From ddc32ddef0f744a7ed6e883d7a8ab39daba753c6 Mon Sep 17 00:00:00 2001 From: "locadex-agent[bot]" <217277504+locadex-agent[bot]@users.noreply.github.com> Date: Sat, 29 Nov 2025 02:46:27 +0000 Subject: [PATCH 1/2] docs(locadex): update translations --- .../data-ingestion/clickpipes/kafka/index.md | 6 +- gt-lock.json | 10987 ++++++++------- i18n/jp/code.json | 1013 +- .../current/Insert_select_settings_tuning.mdx | 15 +- ...ailed-error-using-PowerBI-CH-connector.mdx | 6 +- .../about-quotas-and-query-complexity.mdx | 10 +- .../current/add-column.mdx | 12 +- .../current/alter-user-settings-exception.mdx | 6 +- ...rialized_views_inserted_asynchronously.mdx | 1 - .../async_vs_optimize_read_in_order.mdx | 18 +- .../aws-privatelink-setup-for-clickpipes.mdx | 49 +- ...s-privatelink-setup-for-msk-clickpipes.mdx | 59 +- .../backing_up_a_specific_partition.mdx | 4 +- .../current/calculate-pi-using-sql.mdx | 3 +- ...ate_ratio_of_zero_sparse_serialization.mdx | 3 +- .../cannot-append-data-to-parquet-format.mdx | 3 +- .../certificate_verify_failed_error.mdx | 10 +- .../current/change-billing-email.mdx | 3 +- ...change-the-prompt-in-clickhouse-client.mdx | 9 +- .../current/check-query-cache-in-use.mdx | 7 +- .../check_query_processing_time_only.mdx | 3 +- .../current/check_users_roles.mdx | 1 - .../current/clickhouse_cloud_api_usage.mdx | 3 +- .../current/columnar-database.mdx | 7 +- .../current/common-rbac-queries.mdx | 12 +- .../current/compare_resultsets.mdx | 3 +- .../comparing-metrics-between-queries.mdx | 4 +- .../current/configure-a-user-setting.mdx | 9 +- ...ap_ipc_lock_and_cap_sys_nice_in_docker.mdx | 6 +- ...connection_timeout_remote_remoteSecure.mdx | 3 +- .../current/custom-dns-alias-for-instance.mdx | 12 +- .../current/dbms-naming.mdx | 5 +- .../current/delete-old-data.mdx | 4 +- .../current/dictionaries-consistent-state.mdx | 4 +- .../current/dictionary_using_strings.mdx | 3 +- ..._between_official_builds_and_3rd_party.mdx | 19 +- .../enabling-ssl-with-lets-encrypt.mdx | 5 +- .../current/exception-too-many-parts.mdx | 3 +- .../exchangeStatementToSwitchTables.mdx | 3 +- .../execute-system-queries-in-cloud.mdx | 3 +- .../current/file-export.mdx | 9 +- .../current/filtered-aggregates.mdx | 3 +- .../current/find-expensive-queries.mdx | 9 +- ...ding_expensive_queries_by_memory_usage.mdx | 3 +- ...-developer-verification-error-in-macos.mdx | 17 +- .../current/generate-har-file.mdx | 7 +- ...how-do-i-contribute-code-to-clickhouse.mdx | 1 - ...check-my-clickhouse-cloud-sevice-state.mdx | 9 +- ...-to-connect-to-ch-cloud-using-ssh-keys.mdx | 5 +- ...able-to-query-multiple-remote-clusters.mdx | 3 +- ...-a-clickhouse-table-by-an-array-column.mdx | 5 +- .../how-to-increase-thread-pool-size.mdx | 3 +- ...-to-insert-all-rows-from-another-table.mdx | 3 +- ...set-up-ch-on-docker-odbc-connect-mssql.mdx | 29 +- .../how_to_display_queries_using_MV.mdx | 3 +- .../current/how_to_use_parametrised_views.mdx | 3 +- .../current/ignoring-incorrect-settings.mdx | 3 +- ...ting-geojason-with-nested-object-array.mdx | 8 +- ...ng_and_working_with_JSON_array_objects.mdx | 7 +- .../current/improve-map-performance.mdx | 4 +- .../current/ingest-failures-23-9-release.mdx | 3 +- .../current/ingest-parquet-files-in-s3.mdx | 4 +- .../current/install-clickhouse-windows10.mdx | 4 +- .../current/json-import.mdx | 10 +- .../current/json_extract_example.mdx | 3 +- .../current/json_simple_example.mdx | 5 +- .../current/kafka-clickhouse-json.mdx | 13 +- .../current/kafka-to-clickhouse-setup.mdx | 11 +- .../current/key-value.mdx | 7 +- .../current/llvm-clang-up-to-date.mdx | 3 +- ...f-system-metrics-to-prometheus-metrics.mdx | 101 +- .../current/mapreduce.mdx | 3 +- ...maximum_number_of_tables_and_databases.mdx | 3 +- .../memory-limit-exceeded-for-query.mdx | 5 +- .../current/multi-region-replication.mdx | 1 - .../current/mysql-to-parquet-csv-json.mdx | 12 +- .../current/node-js-example.mdx | 3 +- .../current/olap.mdx | 5 +- .../current/optimize_final_vs_final.mdx | 19 +- .../current/oracle-odbc.mdx | 3 +- .../outputSendLogsLevelTracesToFile.mdx | 6 +- .../current/parquet-to-csv-json.mdx | 15 +- .../current/part_intersects_previous_part.mdx | 15 +- .../current/pivot.mdx | 23 +- .../postgresql-to-parquet-csv-json.mdx | 15 +- .../current/production.mdx | 45 +- .../profiling-clickhouse-with-llvm-xray.mdx | 19 +- .../current/projection_example.mdx | 3 +- .../python-clickhouse-connect-example.mdx | 3 +- .../current/python_http_requests.mdx | 5 +- .../current/query_max_execution_time.mdx | 3 +- .../current/read_consistency.mdx | 3 +- .../recreate_table_across_terminals.mdx | 3 +- .../current/remove-default-user.mdx | 9 +- .../restore-replica-after-storage-failure.mdx | 10 +- .../current/row-column-policy.mdx | 1 - .../s3_export_data_year_month_folders.mdx | 23 +- .../current/schema_migration_tools.mdx | 19 +- ...across-node-for-tables-with-a-wildcard.mdx | 1 - .../current/send_logs_level.mdx | 3 +- .../current/terraform_example.mdx | 4 +- .../current/time-series.mdx | 3 +- ...imizing-basic-data-types-in-clickhouse.mdx | 16 +- .../unable-to-access-cloud-service.mdx | 3 +- .../use-clickhouse-for-log-analytics.mdx | 1 - .../useful-queries-for-troubleshooting.mdx | 36 +- ...y-join-to-extract-and-query-attributes.mdx | 5 +- .../current/vector-search.mdx | 19 +- .../view_number_of_active_mutations.mdx | 3 +- .../current/when_is_ttl_applied.mdx | 9 +- .../which-processes-are-currently-running.mdx | 4 +- .../current/who-is-using-clickhouse.mdx | 1 - .../current/why_default_logging_verbose.mdx | 3 +- .../why_is_my_primary_key_not_used.mdx | 16 +- ...mmend_clickhouse_keeper_over_zookeeper.mdx | 17 +- .../windows_active_directory_to_ch_roles.mdx | 6 +- .../_GCS_authentication_and_bucket.md | 10 +- .../current/_snippets/_add_superset_detail.md | 4 +- .../_clickhouse_mysql_cloud_setup.mdx | 69 +- .../current/_snippets/_clickstack_tagging.mdx | 9 +- ...irect_observability_integration_options.md | 6 +- .../_snippets/_users-and-roles-common.md | 24 +- .../current/about-us/cloud.md | 2 +- .../current/about-us/distinctive-features.md | 2 +- .../current/about-us/index.md | 4 +- .../current/about-us/support.md | 10 +- .../_snippets/_async_inserts.md | 4 +- .../best-practices/avoid_optimize_final.md | 2 +- .../best-practices/choosing_a_primary_key.md | 4 +- .../current/best-practices/index.md | 3 +- .../current/best-practices/json_type.md | 2 +- .../best-practices/partitioning_keys.mdx | 1 - .../best-practices/select_data_type.md | 7 +- .../selecting_an_insert_strategy.md | 13 +- .../sizing-and-hardware-recommendations.md | 2 +- .../using_data_skipping_indices.md | 2 +- .../current/chdb/getting-started.md | 14 +- .../current/chdb/guides/clickhouse-local.md | 13 +- .../current/chdb/guides/jupysql.md | 26 +- .../chdb/guides/query-remote-clickhouse.md | 8 +- .../chdb/guides/querying-apache-arrow.md | 10 +- .../current/chdb/guides/querying-pandas.md | 19 +- .../current/chdb/guides/querying-parquet.md | 11 +- .../current/chdb/guides/querying-s3-bucket.md | 14 +- .../current/chdb/index.md | 2 +- .../current/chdb/install/bun.md | 23 +- .../current/chdb/install/c.md | 26 +- .../current/chdb/install/go.md | 24 +- .../current/chdb/install/nodejs.md | 14 +- .../current/chdb/install/python.md | 180 +- .../current/chdb/install/rust.md | 20 +- .../current/cloud/features/01_cloud_tiers.md | 2 +- .../03_sql_console_features/01_sql-console.md | 8 +- .../02_query-insights.md | 2 +- .../03_query-endpoints.md | 3 +- .../03_sql_console_features/04_dashboards.md | 2 +- .../automatic_scaling/01_auto_scaling.md | 4 +- .../04_infrastructure/deployment-options.md | 2 +- .../replica-aware-routing.md | 2 +- .../04_infrastructure/shared-catalog.md | 2 +- .../04_infrastructure/shared-merge-tree.md | 4 +- .../features/04_infrastructure/warehouses.md | 4 +- .../05_admin_features/api/api-overview.md | 2 +- .../features/05_admin_features/api/openapi.md | 2 +- .../features/05_admin_features/api/postman.md | 14 +- .../features/05_admin_features/upgrades.md | 2 +- .../current/cloud/features/06_security.md | 2 +- .../07_monitoring/advanced_dashboard.md | 8 +- .../features/07_monitoring/prometheus.md | 72 +- .../guides/SQL_console/connection_details.md | 2 +- .../guides/SQL_console/query-endpoints.md | 32 +- .../backups/01_review-and-restore-backups.md | 10 +- .../01_export-backups-to-own-cloud-account.md | 35 +- .../cloud/guides/best_practices/index.md | 2 +- .../guides/best_practices/multitenancy.md | 8 +- .../cloud/guides/cloud-compatibility.md | 2 +- .../data_sources/01_cloud-endpoints-api.md | 3 +- .../02_accessing-s3-data-securely.md | 12 +- .../byoc/03_onboarding/01_aws.md | 14 +- .../byoc/05_observability/01_aws.md | 14 +- .../cloud/guides/production-readiness.md | 12 +- .../03_manage-sql-console-role-assignments.md | 2 +- .../04_manage-database-users.md | 14 +- .../04_saml-sso-setup.md | 2 +- .../05_common-access-management-queries.md | 14 +- .../01_cloud_access_management/index.md | 2 +- .../02_connectivity/01_setting-ip-filters.md | 2 +- .../private_networking/02_aws-privatelink.md | 48 +- .../03_gcp-private-service-connect.md | 52 +- .../04_azure-privatelink.md | 60 +- .../private_networking/index.md | 8 +- .../cloud/guides/security/03_data-masking.md | 20 +- .../current/cloud/guides/security/04_cmek.md | 2 +- .../05_audit_logging/02_database-audit-log.md | 2 +- .../03_byoc-security-playbook.md | 4 +- .../guides/security/05_audit_logging/index.md | 4 +- .../cloud/onboard/01_discover/01_what_is.md | 50 - .../02_use_cases/01_real-time-analytics.md | 2 +- .../01_migration_guides/01_overview.md | 25 +- .../02_postgres/01_overview.md | 2 +- .../02_postgres/appendix.md | 2 +- .../01_migration_guide_part1.md | 20 +- .../02_migration_guide_part2.md | 2 +- .../03_migration_guide_part3.md | 6 +- .../03_bigquery/01_overview.md | 4 +- .../02_migrating-to-clickhouse-cloud.md | 18 +- .../03_bigquery/03_loading-data.md | 4 +- .../04_snowflake/01_overview.md | 2 +- .../04_snowflake/02_migration_guide.md | 6 +- .../03_sql_translation_reference.md | 2 +- .../05_elastic/01_overview.md | 2 +- .../06_redshift/01_overview.md | 2 +- .../06_redshift/02_migration_guide.md | 7 +- .../03_sql_translation_reference.md | 6 +- .../07_OSS_to_Cloud/01_clickhouse-to-cloud.md | 34 +- .../01_clickhouse-local-etl.md | 22 +- .../02_etl-tool-to-clickhouse.md | 11 +- .../03_object-storage-to-clickhouse.md | 11 +- .../cloud/onboard/03_tune/resource_tour.md | 8 +- .../current/cloud/onboard/index.md | 2 +- .../01_changelog/02_release_notes/24_02.md | 8 +- .../01_changelog/02_release_notes/24_05.md | 4 +- .../01_changelog/02_release_notes/24_06.md | 4 +- .../01_changelog/02_release_notes/24_10.md | 2 +- .../cloud/reference/02_architecture.md | 2 +- .../03_billing/01_billing_overview.md | 22 +- .../02_marketplace/aws-marketplace-payg.md | 110 - .../03_billing/04_network-data-transfer.mdx | 2 +- .../03_billing/05_payment-thresholds.md | 4 +- .../03_billing/06_billing_compliance.md | 2 +- .../cloud/reference/05_supported-regions.md | 2 +- .../current/cloud/reference/08_settings.md | 5 +- .../09_security/03_compliance-overview.md | 2 +- .../cloud/reference/11_account-close.md | 2 +- .../current/cloud/reference/index.md | 2 +- .../current/concepts/glossary.md | 2 +- .../current/concepts/olap.md | 2 +- .../concepts/why-clickhouse-is-so-fast.mdx | 26 +- .../compression-in-clickhouse.md | 4 +- .../data-compression/compression-modes.md | 2 +- .../current/data-modeling/backfilling.md | 18 +- .../current/data-modeling/denormalization.md | 14 +- .../current/data-modeling/index.md | 2 +- .../projections/1_projections.md | 298 +- ...2_materialized-views-versus-projections.md | 102 +- .../current/data-modeling/schema-design.md | 1 + .../current/deployment-guides/index.md | 2 +- .../deployment-guides/parallel-replicas.mdx | 264 +- .../01_1_shard_2_replicas.md | 25 +- .../02_2_shards_1_replica.md | 25 +- .../03_2_shards_2_replicas.md | 17 +- .../current/deployment-guides/terminology.md | 2 +- .../current/development/architecture.md | 4 +- .../current/development/build-cross-arm.md | 2 +- .../development/build-cross-loongarch.md | 8 +- .../current/development/build-cross-osx.md | 11 +- .../current/development/build-cross-riscv.md | 8 +- .../current/development/build-cross-s390x.md | 88 +- .../current/development/build-e2k.md | 8 +- .../current/development/build-osx.md | 8 +- .../current/development/build.md | 128 +- .../building_and_benchmarking_deflate_qpl.md | 18 +- .../development/continuous-integration.md | 24 +- .../current/development/contrib.md | 2 +- .../development/developer-instruction.md | 160 +- .../current/development/index.md | 40 - .../development/integrating_rust_libraries.md | 3 +- .../current/development/style.md | 14 +- .../current/development/tests.md | 34 +- .../current/dictionary/index.md | 342 - .../engines/database-engines/atomic.md | 18 +- .../engines/database-engines/backup.md | 11 +- .../engines/database-engines/datalake.md | 6 +- .../current/engines/database-engines/index.md | 41 - .../current/engines/database-engines/lazy.md | 12 +- .../materialized-postgresql.md | 30 +- .../current/engines/database-engines/mysql.md | 8 +- .../engines/database-engines/postgresql.md | 6 +- .../engines/database-engines/replicated.md | 8 +- .../engines/database-engines/shared.md | 4 +- .../engines/database-engines/sqlite.md | 6 +- .../current/engines/table-engines/index.md | 2 +- .../integrations/ExternalDistributed.md | 11 +- .../table-engines/integrations/arrowflight.md | 6 +- .../table-engines/integrations/azure-queue.md | 118 +- .../integrations/azureBlobStorage.md | 14 +- .../table-engines/integrations/deltalake.md | 6 +- .../integrations/embedded-rocksdb.md | 36 +- .../table-engines/integrations/hdfs.md | 16 +- .../table-engines/integrations/hive.md | 36 +- .../table-engines/integrations/hudi.md | 4 +- .../table-engines/integrations/iceberg.md | 26 +- .../table-engines/integrations/index.md | 54 - .../table-engines/integrations/jdbc.md | 6 +- .../table-engines/integrations/kafka.md | 12 +- .../integrations/materialized-postgresql.md | 6 +- .../table-engines/integrations/mongodb.md | 12 +- .../table-engines/integrations/mysql.md | 6 +- .../table-engines/integrations/nats.md | 6 +- .../table-engines/integrations/odbc.md | 6 +- .../table-engines/integrations/postgresql.md | 20 +- .../table-engines/integrations/rabbitmq.md | 6 +- .../table-engines/integrations/redis.md | 6 +- .../engines/table-engines/integrations/s3.md | 36 +- .../table-engines/integrations/s3queue.md | 334 +- .../table-engines/integrations/sqlite.md | 12 +- .../table-engines/integrations/time-series.md | 18 +- .../table-engines/integrations/ytsaurus.md | 6 +- .../engines/table-engines/log-family/index.md | 2 +- .../engines/table-engines/log-family/log.md | 6 +- .../table-engines/log-family/stripelog.md | 6 +- .../table-engines/log-family/tinylog.md | 6 +- .../mergetree-family/aggregatingmergetree.md | 6 +- .../mergetree-family/annindexes.md | 28 +- .../mergetree-family/coalescingmergetree.md | 17 +- .../mergetree-family/collapsingmergetree.md | 16 +- .../custom-partitioning-key.md | 10 +- .../mergetree-family/graphitemergetree.md | 28 +- .../table-engines/mergetree-family/index.md | 4 +- .../mergetree-family/invertedindexes.md | 50 +- .../mergetree-family/mergetree.md | 847 +- .../mergetree-family/replacingmergetree.md | 12 +- .../mergetree-family/replication.md | 357 - .../mergetree-family/summingmergetree.md | 20 +- .../versionedcollapsingmergetree.md | 16 +- .../engines/table-engines/special/alias.md | 127 +- .../engines/table-engines/special/buffer.md | 25 +- .../table-engines/special/dictionary.md | 8 +- .../table-engines/special/distributed.md | 12 +- .../table-engines/special/executable.md | 19 +- .../table-engines/special/external-data.md | 2 +- .../engines/table-engines/special/file.md | 8 +- .../engines/table-engines/special/filelog.md | 4 +- .../engines/table-engines/special/generate.md | 6 +- .../engines/table-engines/special/index.md | 5 +- .../engines/table-engines/special/join.md | 6 +- .../table-engines/special/keepermap.md | 12 +- .../engines/table-engines/special/memory.md | 6 +- .../engines/table-engines/special/merge.md | 6 +- .../engines/table-engines/special/null.md | 2 +- .../engines/table-engines/special/set.md | 2 +- .../engines/table-engines/special/url.md | 4 +- .../engines/table-engines/special/view.md | 2 +- .../current/faq/general/concurrency.md | 2 +- .../current/faq/general/cost-based.md | 2 +- .../current/faq/general/datalake.md | 2 +- .../current/faq/general/dependencies.md | 2 +- .../current/faq/general/distributed-join.md | 2 +- .../current/faq/general/federated.md | 22 +- .../current/faq/general/index.md | 22 +- .../current/faq/general/sql.md | 60 +- .../current/faq/general/updates.md | 2 +- .../current/faq/integration/index.md | 16 +- .../current/faq/integration/json-import.md | 2 +- .../current/faq/integration/oracle-odbc.md | 4 +- .../current/faq/operations/delete-old-data.md | 2 +- .../current/faq/operations/index.md | 18 +- .../current/faq/use-cases/index.md | 6 +- .../example-datasets/amazon-reviews.md | 8 +- .../anon_web_analytics_metrica.md | 22 +- .../example-datasets/brown-benchmark.md | 4 +- .../example-datasets/cell-towers.md | 13 +- .../example-datasets/dbpedia.md | 14 +- .../example-datasets/foursquare-os-places.md | 4 +- .../example-datasets/github.md | 58 +- .../example-datasets/hacker-news.md | 2 +- .../getting-started/example-datasets/laion.md | 28 +- .../getting-started/example-datasets/menus.md | 24 +- .../getting-started/example-datasets/noaa.md | 22 +- .../example-datasets/nyc-taxi.md | 12 +- .../example-datasets/nypd_complaint_data.md | 22 +- .../example-datasets/ontime.md | 6 +- .../example-datasets/stackoverflow.md | 30 +- .../getting-started/example-datasets/tpch.md | 18 +- .../example-datasets/tw-weather.md | 36 +- .../example-datasets/uk-price-paid.md | 12 +- .../example-datasets/youtube-dislikes.md | 10 +- .../current/getting-started/index.md | 5 +- .../install/_snippets/_deb_install.md | 62 +- .../install/_snippets/_docker.md | 36 +- .../install/_snippets/_linux_tar_install.md | 16 +- .../install/_snippets/_macos.md | 10 +- .../install/_snippets/_quick_install.md | 8 +- .../install/_snippets/_rpm_install.md | 8 +- .../install/_snippets/_windows_install.md | 10 +- .../getting-started/install/advanced.md | 2 +- .../getting-started/install/debian_ubuntu.md | 2 +- .../getting-started/install/install.mdx | 28 +- .../current/getting-started/playground.md | 4 +- .../getting-started/quick-start/cloud.mdx | 41 +- .../getting-started/quick-start/oss.mdx | 23 +- .../current/guides/best-practices/index.md | 7 +- .../current/guides/best-practices/prewhere.md | 4 +- .../best-practices/query-optimization.md | 22 +- .../best-practices/query-parallelism.md | 8 +- .../skipping-indexes-examples.md | 18 +- .../guides/best-practices/skipping-indexes.md | 14 +- .../best-practices/sparse-primary-indexes.md | 48 +- .../developer/alternative-query-languages.md | 9 +- .../developer/cascading-materialized-views.md | 31 +- .../developer/debugging-memory-issues.md | 14 +- .../deduplicating-inserts-on-retries.md | 10 +- .../current/guides/developer/deduplication.md | 10 +- .../developer/dynamic-column-selection.md | 19 +- .../current/guides/developer/index.md | 4 +- .../guides/developer/merge-table-function.md | 15 +- .../guides/developer/on-fly-mutations.md | 4 +- .../guides/developer/replacing-merge-tree.md | 8 +- ...ored-procedures-and-prepared-statements.md | 18 +- .../developer/time-series-filling-gaps.md | 27 +- .../current/guides/developer/ttl.md | 14 +- ...nding-query-execution-with-the-analyzer.md | 10 +- .../aggregate_function_combinators/anyIf.md | 2 +- .../argMaxIf.md | 2 +- .../argMinIf.md | 2 +- .../aggregate_function_combinators/avgIf.md | 2 +- .../aggregate_function_combinators/avgMap.md | 2 +- .../avgMergeState.md | 2 +- .../avgResample.md | 4 +- .../avgState.md | 2 +- .../aggregate_function_combinators/countIf.md | 2 +- .../countResample.md | 4 +- .../groupArrayDistinct.md | 2 +- .../groupArrayResample.md | 2 +- .../aggregate_function_combinators/maxMap.md | 2 +- .../aggregate_function_combinators/minMap.md | 2 +- .../minSimpleState.md | 2 +- .../quantilesTimingArrayIf.md | 2 +- .../quantilesTimingIf.md | 2 +- .../sumArray.md | 2 +- .../sumForEach.md | 2 +- .../aggregate_function_combinators/sumIf.md | 6 +- .../aggregate_function_combinators/sumMap.md | 2 +- .../sumSimpleState.md | 4 +- .../uniqArray.md | 2 +- .../uniqArrayIf.md | 4 +- .../current/guides/manage-and-deploy-index.md | 4 +- .../guides/separation-storage-compute.md | 19 +- .../current/guides/sre/configuring-ssl.md | 4 +- .../current/guides/sre/keeper/index.md | 60 +- .../current/guides/sre/network-ports.md | 2 +- .../current/guides/sre/scaling-clusters.md | 2 +- .../sre/user-management/configuring-ldap.md | 4 +- .../guides/sre/user-management/index.md | 4 +- .../sre/user-management/ssl-user-auth.md | 4 +- .../guides/starter_guides/creating-tables.md | 2 +- .../starter_guides/generating-test-data.md | 24 +- .../guides/starter_guides/inserting-data.md | 2 +- .../guides/starter_guides/mutations.md | 14 +- .../starter_guides/working_with_arrays.md | 20 +- .../current/guides/troubleshooting.md | 30 +- .../current/integrations/apache-beam.md | 153 - .../data-ingestion/apache-spark/index.md | 9 +- .../data-ingestion/apache-spark/spark-jdbc.md | 10 +- .../apache-spark/spark-native-connector.md | 20 +- .../data-ingestion/aws-glue/index.md | 2 +- .../azure-data-factory/overview.md | 5 +- .../using_azureblobstorage.md | 4 +- .../using_http_interface.md | 2 +- .../data-ingestion/azure-synapse/index.md | 4 +- .../clickpipes/aws-privatelink.md | 2 +- .../data-ingestion/clickpipes/index.md | 2 +- .../clickpipes/kafka/03_reference.md | 2 +- .../clickpipes/kafka/04_best_practices.md | 8 +- .../data-ingestion/clickpipes/kafka/index.md | 25 - .../data-ingestion/clickpipes/kinesis.md | 2 +- .../clickpipes/mongodb/add_table.md | 2 +- .../data-ingestion/clickpipes/mongodb/faq.md | 18 +- .../clickpipes/mongodb/index.md | 2 +- .../clickpipes/mongodb/quickstart.md | 28 +- .../clickpipes/mongodb/resync.md | 6 +- .../clickpipes/mongodb/source/atlas.md | 2 +- .../clickpipes/mongodb/source/documentdb.md | 10 +- .../clickpipes/mongodb/source/generic.md | 6 +- .../clickpipes/mysql/add_table.md | 2 +- .../data-ingestion/clickpipes/mysql/faq.md | 2 +- .../data-ingestion/clickpipes/mysql/index.md | 2 +- .../data-ingestion/clickpipes/mysql/resync.md | 6 +- .../clickpipes/mysql/source/aurora.md | 8 +- .../clickpipes/mysql/source/gcp.md | 2 +- .../clickpipes/mysql/source/generic.md | 8 +- .../clickpipes/mysql/source/generic_maria.md | 4 +- .../clickpipes/mysql/source/rds.md | 8 +- .../clickpipes/mysql/source/rds_maria.md | 8 +- .../clickpipes/postgres/add_table.md | 2 +- .../clickpipes/postgres/deduplication.md | 23 +- .../data-ingestion/clickpipes/postgres/faq.md | 48 +- .../clickpipes/postgres/index.md | 2 +- .../clickpipes/postgres/maintenance.md | 6 +- .../clickpipes/postgres/ordering_keys.md | 13 +- .../clickpipes/postgres/resync.md | 6 +- .../clickpipes/postgres/scaling.md | 6 +- .../clickpipes/postgres/source/aurora.md | 4 +- .../source/azure-flexible-server-postgres.md | 2 +- .../postgres/source/crunchy-postgres.md | 4 +- .../clickpipes/postgres/source/generic.md | 6 +- .../postgres/source/google-cloudsql.md | 2 +- .../postgres/source/neon-postgres.md | 6 +- .../clickpipes/postgres/source/planetscale.md | 4 +- .../clickpipes/postgres/source/rds.md | 4 +- .../clickpipes/postgres/source/supabase.md | 6 +- .../clickpipes/postgres/source/timescale.md | 4 +- .../clickpipes/postgres/toast.md | 4 +- .../data-ingestion/clickpipes/secure-rds.md | 21 +- .../data-formats/arrow-avro-orc.md | 14 +- .../data-ingestion/data-formats/binary.md | 24 +- .../data-ingestion/data-formats/csv-tsv.md | 32 +- .../data-ingestion/data-formats/intro.md | 2 +- .../data-formats/json/exporting.md | 16 +- .../data-formats/json/formats.md | 36 +- .../data-formats/json/inference.md | 12 +- .../data-ingestion/data-formats/json/intro.md | 27 +- .../data-formats/json/loading.md | 6 +- .../data-ingestion/data-formats/json/other.md | 36 +- .../data-formats/json/schema.md | 20 +- .../data-ingestion/data-formats/parquet.md | 14 +- .../data-ingestion/data-formats/sql.md | 8 +- .../data-formats/templates-regex.md | 14 +- .../data-ingestion/data-ingestion-index.md | 2 +- .../data-ingestion/data-sources-index.md | 2 +- .../data-ingestion/dbms/dynamodb/index.md | 10 +- .../dbms/jdbc-with-clickhouse.md | 6 +- .../postgresql/connecting-to-postgresql.md | 28 +- .../integrations/data-ingestion/emqx/index.md | 24 +- .../etl-tools/airbyte-and-clickhouse.md | 4 +- .../data-ingestion/etl-tools/apache-beam.md | 14 +- .../etl-tools/bladepipe-and-clickhouse.md | 2 +- .../dbt/features-and-configurations.md | 84 +- .../data-ingestion/etl-tools/dbt/guides.md | 16 +- .../data-ingestion/etl-tools/dbt/index.md | 16 +- .../etl-tools/dlt-and-clickhouse.md | 18 +- .../etl-tools/fivetran/index.md | 2 +- .../etl-tools/nifi-and-clickhouse.md | 4 +- .../etl-tools/vector-to-clickhouse.md | 369 +- .../integrations/data-ingestion/gcs/index.md | 62 +- .../google-dataflow/dataflow.md | 2 +- .../google-dataflow/java-runner.md | 2 +- .../google-dataflow/templates.md | 2 +- .../templates/bigquery-to-clickhouse.md | 2 +- .../data-ingestion/insert-local-files.md | 3 +- .../kafka/confluent/confluent-cloud.md | 4 +- .../kafka/confluent/custom-connector.md | 22 +- .../data-ingestion/kafka/confluent/index.md | 2 +- .../kafka/confluent/kafka-connect-http.md | 35 +- .../data-ingestion/kafka/index.md | 2 +- .../kafka/kafka-clickhouse-connect-sink.md | 98 +- .../kafka/kafka-connect-jdbc.md | 10 +- .../kafka-table-engine-named-collections.md | 22 +- .../kafka/kafka-table-engine.md | 62 +- .../data-ingestion/kafka/kafka-vector.md | 13 +- .../data-ingestion/kafka/msk/index.md | 14 +- .../redshift/_snippets/_migration_guide.md | 2 +- .../integrations/data-ingestion/s3-minio.md | 4 +- .../integrations/data-ingestion/s3/index.md | 653 +- .../data-ingestion/s3/performance.md | 271 +- .../integrations/data-sources/cassandra.md | 2 +- .../integrations/data-sources/mysql.md | 16 +- .../integrations/data-sources/postgres.md | 2 +- .../data-sources/table_engines/deltalake.md | 2 +- .../data-sources/table_engines/hive.md | 2 +- .../data-sources/table_engines/hudi.md | 2 +- .../data-sources/table_engines/iceberg.md | 2 +- .../data-sources/table_engines/mongodb.md | 2 +- .../data-sources/table_engines/nats.md | 2 +- .../data-sources/table_engines/rabbitmq.md | 2 +- .../data-sources/table_engines/redis.md | 2 +- .../data-sources/table_engines/rocksdb.md | 2 +- .../data-sources/table_engines/sqlite.md | 2 +- .../astrato-and-clickhouse.md | 4 +- .../chartbrew-and-clickhouse.md | 6 +- .../databrain-and-clickhouse.md | 14 +- .../community_integrations/deepnote.md | 4 +- .../dot-and-clickhouse.md | 2 +- .../draxlr-and-clickhouse.md | 4 +- .../embeddable-and-clickhouse.md | 6 +- .../explo-and-clickhouse.md | 4 +- .../fabi-and-clickhouse.md | 4 +- .../hashboard-and-clickhouse.md | 4 +- .../luzmo-and-clickhouse.md | 4 +- .../mitzu-and-clickhouse.md | 4 +- .../rocketbi-and-clickhouse.md | 6 +- .../zingdata-and-clickhouse.md | 4 +- .../data-visualization/grafana/config.md | 22 +- .../data-visualization/grafana/index.md | 4 +- .../grafana/query-builder.md | 8 +- .../integrations/data-visualization/index.md | 2 +- .../lightdash-and-clickhouse.md | 4 +- .../looker-and-clickhouse.md | 4 +- .../looker-studio-and-clickhouse.md | 6 +- .../metabase-and-clickhouse.md | 4 +- .../data-visualization/omni-and-clickhouse.md | 4 +- .../powerbi-and-clickhouse.md | 4 +- .../quicksight-and-clickhouse.md | 6 +- .../splunk-and-clickhouse.md | 8 +- .../superset-and-clickhouse.md | 4 +- .../tableau/tableau-analysis-tips.md | 8 +- .../tableau/tableau-and-clickhouse.md | 4 +- .../tableau/tableau-connection-tips.md | 6 +- .../tableau/tableau-online-and-clickhouse.md | 6 +- .../current/integrations/index.mdx | 3 +- .../integrations/language-clients/csharp.md | 34 +- .../integrations/language-clients/go/index.md | 112 +- .../java/client/_snippets/_v0_7.mdx | 138 +- .../java/client/_snippets/_v0_8.mdx | 200 +- .../language-clients/java/index.md | 6 +- .../java/jdbc/_snippets/_v0_7.mdx | 171 +- .../java/jdbc/_snippets/_v0_8.mdx | 90 +- .../language-clients/java/r2dbc.md | 10 +- .../integrations/language-clients/js.md | 58 +- .../language-clients/moose-olap.md | 14 +- .../python/additional-options.md | 2 +- .../python/advanced-inserting.md | 44 +- .../python/advanced-querying.md | 138 +- .../language-clients/python/advanced-usage.md | 8 +- .../language-clients/python/driver-api.md | 172 +- .../language-clients/python/index.md | 10 +- .../language-clients/python/sqlalchemy.md | 16 +- .../integrations/language-clients/rust.md | 32 +- .../current/integrations/misc/index.md | 2 +- .../integrations/sql-clients/datagrip.md | 4 +- .../integrations/sql-clients/dbeaver.md | 2 +- .../integrations/sql-clients/dbvisualizer.md | 15 +- .../current/integrations/sql-clients/index.md | 2 +- .../integrations/sql-clients/jupysql.md | 5 +- .../integrations/sql-clients/marimo.md | 6 +- .../integrations/sql-clients/qstudio.md | 15 +- .../integrations/sql-clients/sql-console.md | 8 +- .../integrations/sql-clients/tablum.md | 2 +- .../tools/data-integration/easypanel/index.md | 2 +- .../tools/data-integration/index.md | 2 +- .../tools/data-integration/retool/index.md | 4 +- .../tools/data-integration/splunk/index.md | 8 +- .../current/integrations/tools/index.md | 2 +- .../current/interfaces/arrowflight.md | 6 +- .../current/interfaces/cli.md | 81 +- .../current/interfaces/cpp.md | 4 +- .../current/interfaces/formats/Arrow/Arrow.md | 4 +- .../current/interfaces/formats/Avro/Avro.md | 6 +- .../interfaces/formats/Avro/AvroConfluent.md | 6 +- .../current/interfaces/formats/BSONEachRow.md | 4 +- .../current/interfaces/formats/CSV/CSV.md | 2 +- .../interfaces/formats/CSV/CSVWithNames.md | 4 +- .../formats/CSV/CSVWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/CapnProto.md | 6 +- .../CustomSeparated/CustomSeparated.md | 4 +- .../CustomSeparatedIgnoreSpaces.md | 2 +- .../CustomSeparatedIgnoreSpacesWithNames.md | 2 +- ...mSeparatedIgnoreSpacesWithNamesAndTypes.md | 2 +- .../CustomSeparatedWithNames.md | 4 +- .../CustomSeparatedWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/DWARF.md | 2 +- .../current/interfaces/formats/Form.md | 2 +- .../current/interfaces/formats/Hash.md | 2 +- .../current/interfaces/formats/JSON/JSON.md | 2 +- .../interfaces/formats/JSON/JSONAsObject.md | 6 +- .../interfaces/formats/JSON/JSONAsString.md | 4 +- .../interfaces/formats/JSON/JSONColumns.md | 4 +- .../formats/JSON/JSONColumnsWithMetadata.md | 2 +- .../interfaces/formats/JSON/JSONCompact.md | 4 +- .../formats/JSON/JSONCompactColumns.md | 4 +- .../formats/JSON/JSONCompactEachRow.md | 4 +- .../JSON/JSONCompactEachRowWithNames.md | 4 +- .../JSONCompactEachRowWithNamesAndTypes.md | 4 +- .../JSON/JSONCompactEachRowWithProgress.md | 2 +- .../formats/JSON/JSONCompactStrings.md | 2 +- .../formats/JSON/JSONCompactStringsEachRow.md | 4 +- .../JSONCompactStringsEachRowWithNames.md | 4 +- ...NCompactStringsEachRowWithNamesAndTypes.md | 4 +- .../JSONCompactStringsEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONEachRow.md | 4 +- .../formats/JSON/JSONEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONLines.md | 4 +- .../formats/JSON/JSONObjectEachRow.md | 16 +- .../interfaces/formats/JSON/JSONStrings.md | 2 +- .../formats/JSON/JSONStringsEachRow.md | 6 +- .../JSON/JSONStringsEachRowWithProgress.md | 2 +- .../formats/JSON/PrettyJSONEachRow.md | 2 +- .../formats/LineAsString/LineAsString.md | 2 +- .../LineAsString/LineAsStringWithNames.md | 2 +- .../LineAsStringWithNamesAndTypes.md | 2 +- .../current/interfaces/formats/Markdown.md | 2 +- .../current/interfaces/formats/MsgPack.md | 2 +- .../current/interfaces/formats/MySQLDump.md | 2 +- .../current/interfaces/formats/Npy.md | 8 +- .../current/interfaces/formats/Null.md | 4 +- .../current/interfaces/formats/ORC.md | 6 +- .../current/interfaces/formats/One.md | 2 +- .../interfaces/formats/Parquet/Parquet.md | 6 +- .../formats/Parquet/ParquetMetadata.md | 2 +- .../interfaces/formats/Pretty/Pretty.md | 2 +- .../formats/Pretty/PrettyNoEscapes.md | 2 +- .../current/interfaces/formats/Prometheus.md | 20 +- .../interfaces/formats/Protobuf/Protobuf.md | 6 +- .../formats/Protobuf/ProtobufList.md | 2 +- .../current/interfaces/formats/RawBLOB.md | 6 +- .../current/interfaces/formats/Regexp.md | 2 +- .../RowBinary/RowBinaryWithDefaults.md | 2 +- .../current/interfaces/formats/SQLInsert.md | 2 +- .../interfaces/formats/TabSeparated/TSKV.md | 8 +- .../formats/TabSeparated/TabSeparated.md | 10 +- .../formats/TabSeparated/TabSeparatedRaw.md | 6 +- .../TabSeparated/TabSeparatedRawWithNames.md | 6 +- .../TabSeparatedRawWithNamesAndTypes.md | 6 +- .../TabSeparated/TabSeparatedWithNames.md | 6 +- .../TabSeparatedWithNamesAndTypes.md | 6 +- .../interfaces/formats/Template/Template.md | 16 +- .../formats/Template/TemplateIgnoreSpaces.md | 2 +- .../current/interfaces/formats/Vertical.md | 2 +- .../current/interfaces/formats/XML.md | 2 +- .../current/interfaces/grpc.md | 6 +- .../current/interfaces/http.md | 64 +- .../current/interfaces/jdbc.md | 2 +- .../current/interfaces/mysql.md | 10 +- .../native-clients-interfaces-index.md | 4 +- .../current/interfaces/odbc.md | 2 +- .../current/interfaces/overview.md | 30 +- .../current/interfaces/postgresql.md | 12 +- .../current/interfaces/prometheus.md | 17 +- .../current/interfaces/schema-inference.md | 92 +- .../current/interfaces/ssh.md | 16 +- .../current/interfaces/tcp.md | 2 +- .../third-party/client-libraries.md | 38 +- .../current/interfaces/third-party/gui.md | 2 +- .../current/interfaces/third-party/index.md | 10 +- .../interfaces/third-party/integrations.md | 2 +- .../current/interfaces/third-party/proxy.md | 2 +- .../current/intro.md | 2 +- .../core-concepts/academic_overview.mdx | 172 +- .../managing-data/core-concepts/merges.mdx | 36 +- .../core-concepts/partitions.mdx | 28 +- .../managing-data/core-concepts/parts.md | 4 +- .../core-concepts/primary-indexes.mdx | 41 +- .../managing-data/core-concepts/shards.mdx | 28 +- .../deleting-data/delete_mutations.mdx | 2 +- .../managing-data/deleting-data/overview.mdx | 14 +- .../current/managing-data/drop_partition.mdx | 5 +- .../current/managing-data/truncate.md | 2 +- .../managing-data/updating-data/overview.mdx | 21 +- .../updating-data/update_mutations.mdx | 2 +- .../incremental-materialized-view.md | 32 +- .../refreshable-materialized-view.md | 14 +- .../current/native-protocol/basics.md | 6 +- .../current/native-protocol/client.md | 2 +- .../current/native-protocol/columns.md | 6 +- .../current/native-protocol/hash.md | 2 +- .../current/native-protocol/server.md | 2 +- .../current/operations/_troubleshooting.md | 20 +- .../operations/allocation-profiling-old.md | 22 +- .../operations/allocation-profiling.md | 26 +- .../current/operations/analyzer.md | 37 +- .../current/operations/backup.md | 44 +- .../current/operations/caches.md | 34 +- .../current/operations/cluster-discovery.md | 23 +- .../current/operations/configuration-files.md | 16 +- .../external-authenticators/http.md | 11 +- .../external-authenticators/index.md | 2 +- .../external-authenticators/kerberos.md | 18 +- .../external-authenticators/ldap.md | 102 +- .../external-authenticators/ssl-x509.md | 2 +- .../current/operations/monitoring.md | 2 +- .../current/operations/named-collections.md | 96 +- .../current/operations/opentelemetry.md | 2 +- .../profile-guided-optimization.md | 7 +- .../sampling-query-profiler.md | 8 +- .../current/operations/performance-test.md | 5 +- .../current/operations/query-cache.md | 141 +- .../operations/query-condition-cache.md | 4 +- .../_server_settings_outside_source.md | 186 +- .../settings.md | 5133 +------ .../settings/composable-protocols.md | 16 +- .../settings/constraints-on-settings.md | 14 +- .../current/operations/settings/index.md | 35 - .../operations/settings/memory-overcommit.md | 4 +- .../settings/merge-tree-settings.md | 3120 +---- .../current/operations/settings/overview.md | 4 +- .../settings/permissions-for-queries.md | 2 +- .../operations/settings/query-complexity.md | 2 +- .../operations/settings/server-overload.md | 4 +- .../operations/settings/settings-formats.md | 2800 +--- .../operations/settings/settings-profiles.md | 2 +- .../settings/settings-query-level.md | 32 +- .../operations/settings/settings-users.md | 25 +- .../current/operations/settings/settings.md | 11434 +-------------- .../settings/tcp-connection-limits.md | 2 +- .../current/operations/ssl-zookeeper.md | 4 +- .../current/operations/startup-scripts.md | 2 +- .../current/operations/storing-data.md | 48 +- .../system-tables/asynchronous_insert_log.md | 5 +- .../system-tables/asynchronous_inserts.md | 25 +- .../system-tables/asynchronous_loader.md | 82 +- .../system-tables/asynchronous_metric_log.md | 2 +- .../system-tables/asynchronous_metrics.md | 22 +- .../system-tables/azure_queue_settings.md | 13 +- .../operations/system-tables/backup_log.md | 7 +- .../operations/system-tables/backups.md | 2 +- .../system-tables/blob_storage_log.md | 2 +- .../operations/system-tables/build_options.md | 11 +- .../operations/system-tables/clusters.md | 69 +- .../operations/system-tables/codecs.md | 21 +- .../operations/system-tables/columns.md | 121 +- .../operations/system-tables/contributors.md | 8 +- .../operations/system-tables/crash_log.md | 2 +- .../operations/system-tables/dashboards.md | 14 +- .../system-tables/data_skipping_indices.md | 13 +- .../system-tables/data_type_families.md | 12 +- .../system-tables/database_engines.md | 14 +- .../system-tables/database_replicas.md | 18 +- .../operations/system-tables/databases.md | 19 +- .../system-tables/delta_metadata_log.md | 6 +- .../system-tables/detached_tables.md | 10 +- .../operations/system-tables/dictionaries.md | 50 +- .../system-tables/dimensional_metrics.md | 4 +- .../current/operations/system-tables/disks.md | 28 +- .../system-tables/distributed_ddl_queue.md | 26 +- .../system-tables/distribution_queue.md | 18 +- .../operations/system-tables/dns_cache.md | 15 +- .../system-tables/dropped_tables.md | 20 +- .../operations/system-tables/error_log.md | 2 +- .../operations/system-tables/errors.md | 20 +- .../operations/system-tables/events.md | 32 +- .../operations/system-tables/functions.md | 19 +- .../system-tables/graphite_retentions.md | 17 +- .../system-tables/histogram_metrics.md | 29 +- .../system-tables/iceberg_history.md | 15 +- .../system-tables/iceberg_metadata_log.md | 8 +- .../current/operations/system-tables/index.md | 136 - .../system-tables/information_schema.md | 10 +- .../operations/system-tables/jemalloc_bins.md | 18 +- .../system-tables/kafka_consumers.md | 32 +- .../operations/system-tables/licenses.md | 13 +- .../system-tables/merge_tree_settings.md | 36 +- .../operations/system-tables/merges.md | 36 +- .../operations/system-tables/metric_log.md | 5 +- .../operations/system-tables/metrics.md | 400 +- .../current/operations/system-tables/moves.md | 24 +- .../operations/system-tables/mutations.md | 7 +- .../operations/system-tables/numbers.md | 2 +- .../current/operations/system-tables/one.md | 2 +- .../system-tables/opentelemetry_span_log.md | 5 +- .../operations/system-tables/overview.md | 10 +- .../operations/system-tables/part_log.md | 9 +- .../current/operations/system-tables/parts.md | 3 +- .../operations/system-tables/parts_columns.md | 2 +- .../operations/system-tables/processes.md | 60 +- .../system-tables/processors_profile_log.md | 6 +- .../system-tables/projection_parts.md | 73 +- .../system-tables/projection_parts_columns.md | 61 +- .../operations/system-tables/projections.md | 15 +- .../operations/system-tables/query_cache.md | 21 +- .../system-tables/query_condition_cache.md | 18 +- .../operations/system-tables/query_log.md | 6 +- .../system-tables/query_metric_log.md | 5 +- .../system-tables/query_thread_log.md | 15 +- .../system-tables/query_views_log.md | 73 +- .../operations/system-tables/quota_limits.md | 30 +- .../operations/system-tables/quota_usage.md | 37 +- .../operations/system-tables/quotas.md | 2 +- .../operations/system-tables/quotas_usage.md | 39 +- .../operations/system-tables/replicas.md | 71 +- .../system-tables/replicated_fetches.md | 26 +- .../system-tables/replication_queue.md | 3 +- .../operations/system-tables/resources.md | 22 +- .../operations/system-tables/role_grants.md | 20 +- .../current/operations/system-tables/roles.md | 16 +- .../operations/system-tables/row_policies.md | 22 +- .../system-tables/s3_queue_settings.md | 20 +- .../operations/system-tables/scheduler.md | 44 +- .../system-tables/schema_inference_cache.md | 26 +- .../system-tables/server_settings.md | 6 +- .../operations/system-tables/session_log.md | 6 +- .../operations/system-tables/settings.md | 5 +- .../system-tables/settings_changes.md | 18 +- .../settings_profile_elements.md | 21 +- .../system-tables/settings_profiles.md | 18 +- .../operations/system-tables/stack_trace.md | 7 +- .../system-tables/storage_policies.md | 32 +- .../system-tables/system_warnings.md | 5 +- .../operations/system-tables/table_engines.md | 22 +- .../operations/system-tables/tables.md | 2 +- .../operations/system-tables/text_log.md | 5 +- .../operations/system-tables/time_zones.md | 12 +- .../operations/system-tables/trace_log.md | 6 +- .../operations/system-tables/unicode.md | 4 +- .../system-tables/user_processes.md | 16 +- .../current/operations/system-tables/users.md | 27 +- .../system-tables/view_refreshes.md | 29 +- .../operations/system-tables/workloads.md | 20 +- .../operations/system-tables/zookeeper.md | 2 +- .../system-tables/zookeeper_connection.md | 5 +- .../system-tables/zookeeper_connection_log.md | 6 +- .../operations/system-tables/zookeeper_log.md | 43 +- .../current/operations/tips.md | 80 +- .../current/operations/update.md | 4 +- .../operations/userspace-page-cache.md | 8 +- .../operations/utilities/backupview.md | 20 +- .../utilities/clickhouse-benchmark.md | 6 +- .../utilities/clickhouse-compressor.md | 2 +- .../operations/utilities/clickhouse-disks.md | 2 +- .../operations/utilities/clickhouse-format.md | 34 +- .../utilities/clickhouse-keeper-client.md | 4 +- .../operations/utilities/clickhouse-local.md | 14 +- .../operations/utilities/odbc-bridge.md | 2 +- .../current/operations/workload-scheduling.md | 22 +- .../operations_/backup_restore/00_overview.md | 14 +- .../backup_restore/01_local_disk.md | 28 +- .../backup_restore/02_s3_endpoint.md | 2 +- .../backup_restore/03_azure_blob_storage.md | 4 +- .../backup_restore/04_alternative_methods.md | 2 +- .../aggregate-functions/combinators.md | 12 +- .../aggregate-functions/grouping_function.md | 12 +- .../aggregate-functions/index.md | 13 +- .../parametric-functions.md | 42 +- .../aggregate-functions/reference/aggthrow.md | 2 +- .../reference/analysis_of_variance.md | 2 +- .../aggregate-functions/reference/any.md | 2 +- .../aggregate-functions/reference/anyheavy.md | 2 +- .../aggregate-functions/reference/anylast.md | 2 +- .../reference/approxtopk.md | 57 - .../reference/approxtopsum.md | 53 - .../reference/argandmax.md | 6 +- .../reference/argandmin.md | 3 +- .../aggregate-functions/reference/argmax.md | 2 +- .../aggregate-functions/reference/argmin.md | 3 +- .../aggregate-functions/reference/avg.md | 2 +- .../reference/avgweighted.md | 2 +- .../reference/contingency.md | 2 +- .../aggregate-functions/reference/corr.md | 2 +- .../reference/corrmatrix.md | 2 +- .../reference/corrstable.md | 2 +- .../aggregate-functions/reference/count.md | 2 +- .../aggregate-functions/reference/covarpop.md | 2 +- .../reference/covarpopmatrix.md | 2 +- .../reference/covarpopstable.md | 2 +- .../reference/covarsamp.md | 2 +- .../reference/covarsampmatrix.md | 2 +- .../reference/covarsampstable.md | 2 +- .../aggregate-functions/reference/cramersv.md | 2 +- .../reference/cramersvbiascorrected.md | 2 +- .../aggregate-functions/reference/deltasum.md | 2 +- .../reference/distinctdynamictypes.md | 2 +- .../reference/distinctjsonpaths.md | 5 +- .../aggregate-functions/reference/entropy.md | 2 +- .../reference/estimateCompressionRatio.md | 4 +- .../reference/exponentialmovingaverage.md | 5 +- .../reference/exponentialtimedecayedavg.md | 5 +- .../reference/exponentialtimedecayedcount.md | 5 +- .../reference/exponentialtimedecayedmax.md | 5 +- .../reference/exponentialtimedecayedsum.md | 5 +- .../reference/first_value.md | 16 +- .../reference/flame_graph.md | 14 +- .../reference/grouparray.md | 2 +- .../reference/grouparrayarray.md | 2 +- .../reference/grouparrayinsertat.md | 2 +- .../reference/grouparrayintersect.md | 2 +- .../reference/grouparraylast.md | 2 +- .../reference/grouparraymovingavg.md | 2 +- .../reference/grouparraymovingsum.md | 2 +- .../reference/grouparraysample.md | 2 +- .../reference/grouparraysorted.md | 2 +- .../reference/groupbitand.md | 2 +- .../reference/groupbitmap.md | 2 +- .../reference/groupbitmapor.md | 2 +- .../reference/groupbitmapxor.md | 2 +- .../reference/groupbitor.md | 2 +- .../reference/groupbitxor.md | 2 +- .../reference/groupuniqarray.md | 2 +- .../aggregate-functions/reference/index.md | 164 - .../reference/kolmogorovsmirnovtest.md | 5 +- .../aggregate-functions/reference/kurtpop.md | 2 +- .../aggregate-functions/reference/kurtsamp.md | 2 +- .../reference/largestTriangleThreeBuckets.md | 2 +- .../reference/last_value.md | 16 +- .../reference/mannwhitneyutest.md | 2 +- .../reference/maxintersections.md | 2 +- .../reference/maxintersectionsposition.md | 2 +- .../aggregate-functions/reference/maxmap.md | 2 +- .../reference/meanztest.md | 2 +- .../aggregate-functions/reference/median.md | 2 +- .../aggregate-functions/reference/minmap.md | 2 +- .../aggregate-functions/reference/quantile.md | 2 +- .../reference/quantileGK.md | 2 +- .../reference/quantilebfloat16.md | 2 +- .../reference/quantiledeterministic.md | 2 +- .../reference/quantileexact.md | 20 +- .../reference/quantileexactweighted.md | 2 +- .../quantileexactweightedinterpolated.md | 2 +- .../reference/quantileinterpolatedweighted.md | 2 +- .../reference/quantileprometheushistogram.md | 2 +- .../reference/quantiles.md | 8 +- .../reference/quantiletdigest.md | 2 +- .../reference/quantiletdigestweighted.md | 2 +- .../reference/quantiletiming.md | 2 +- .../reference/quantiletimingweighted.md | 5 +- .../aggregate-functions/reference/rankCorr.md | 2 +- .../reference/simplelinearregression.md | 2 +- .../reference/singlevalueornull.md | 2 +- .../aggregate-functions/reference/skewpop.md | 2 +- .../aggregate-functions/reference/skewsamp.md | 2 +- .../aggregate-functions/reference/sparkbar.md | 2 +- .../reference/stddevpop.md | 2 +- .../reference/stddevpopstable.md | 2 +- .../reference/stddevsamp.md | 2 +- .../reference/stddevsampstable.md | 2 +- .../reference/stochasticlinearregression.md | 15 +- .../reference/stochasticlogisticregression.md | 6 +- .../reference/studentttest.md | 2 +- .../reference/studentttestonesample.md | 2 +- .../aggregate-functions/reference/sum.md | 2 +- .../aggregate-functions/reference/summap.md | 3 +- .../reference/summapwithoverflow.md | 3 +- .../reference/sumwithoverflow.md | 2 +- .../aggregate-functions/reference/theilsu.md | 2 +- .../reference/timeSeriesGroupArray.md | 2 +- .../aggregate-functions/reference/topk.md | 54 - .../reference/topkweighted.md | 2 +- .../aggregate-functions/reference/uniq.md | 2 +- .../reference/uniqcombined.md | 2 +- .../reference/uniqcombined64.md | 2 +- .../reference/uniqexact.md | 2 +- .../reference/uniqhll12.md | 2 +- .../reference/varpopstable.md | 4 +- .../reference/welchttest.md | 2 +- .../data-types/aggregatefunction.md | 78 +- .../current/sql-reference/data-types/array.md | 17 +- .../sql-reference/data-types/boolean.md | 2 +- .../data-types/data-types-binary-encoding.md | 66 +- .../current/sql-reference/data-types/date.md | 2 +- .../sql-reference/data-types/date32.md | 2 +- .../sql-reference/data-types/datetime.md | 6 +- .../sql-reference/data-types/datetime64.md | 8 +- .../sql-reference/data-types/decimal.md | 4 +- .../sql-reference/data-types/domains/index.md | 2 +- .../sql-reference/data-types/dynamic.md | 49 +- .../current/sql-reference/data-types/enum.md | 15 +- .../sql-reference/data-types/fixedstring.md | 2 +- .../current/sql-reference/data-types/float.md | 6 +- .../current/sql-reference/data-types/geo.md | 14 +- .../current/sql-reference/data-types/index.md | 4 +- .../current/sql-reference/data-types/ipv4.md | 6 +- .../current/sql-reference/data-types/ipv6.md | 6 +- .../data-types/lowcardinality.md | 6 +- .../current/sql-reference/data-types/map.md | 6 +- .../nested-data-structures/index.md | 9 +- .../sql-reference/data-types/newjson.md | 38 +- .../sql-reference/data-types/nullable.md | 6 +- .../current/sql-reference/data-types/qbit.md | 4 +- .../data-types/simpleaggregatefunction.md | 6 +- .../special-data-types/expression.md | 2 +- .../data-types/special-data-types/index.md | 2 +- .../data-types/special-data-types/interval.md | 4 +- .../data-types/special-data-types/nothing.md | 2 +- .../data-types/special-data-types/set.md | 2 +- .../sql-reference/data-types/string.md | 2 +- .../current/sql-reference/data-types/time.md | 4 +- .../sql-reference/data-types/time64.md | 4 +- .../current/sql-reference/data-types/tuple.md | 17 +- .../current/sql-reference/data-types/uuid.md | 4 +- .../sql-reference/data-types/variant.md | 34 +- .../sql-reference/dictionaries/index.md | 147 +- .../functions/arithmetic-functions.md | 1298 +- .../functions/array-functions.md | 5004 +------ .../sql-reference/functions/array-join.md | 7 +- .../sql-reference/functions/bit-functions.md | 685 +- .../functions/bitmap-functions.md | 717 +- .../functions/comparison-functions.md | 340 +- .../functions/conditional-functions.md | 405 +- .../functions/date-time-functions.md | 5651 +------- .../functions/distance-functions.md | 621 +- .../functions/embedded-dict-functions.md | 32 +- .../functions/encoding-functions.md | 1096 +- .../functions/encryption-functions.md | 385 +- .../functions/ext-dict-functions.md | 2404 +--- .../current/sql-reference/functions/files.md | 4 +- .../functions/financial-functions.md | 248 +- .../functions/functions-for-nulls.md | 473 +- .../functions/geo/coordinates.md | 16 +- .../functions/geo/flipCoordinates.md | 18 +- .../sql-reference/functions/geo/geohash.md | 6 +- .../current/sql-reference/functions/geo/h3.md | 88 +- .../sql-reference/functions/geo/polygon.md | 240 +- .../current/sql-reference/functions/geo/s2.md | 22 +- .../sql-reference/functions/geo/svg.md | 4 +- .../sql-reference/functions/hash-functions.md | 2641 +--- .../sql-reference/functions/in-functions.md | 2 +- .../sql-reference/functions/introspection.md | 257 +- .../functions/ip-address-functions.md | 1122 +- .../sql-reference/functions/json-functions.md | 2030 +-- .../functions/logical-functions.md | 201 +- .../functions/machine-learning-functions.md | 8 +- .../sql-reference/functions/math-functions.md | 1293 +- .../sql-reference/functions/nlp-functions.md | 339 +- .../numeric-indexed-vector-functions.md | 696 +- .../functions/other-functions.md | 5454 +------- .../sql-reference/functions/overview.md | 6 +- .../functions/random-functions.md | 815 +- .../functions/rounding-functions.md | 463 +- .../functions/splitting-merging-functions.md | 462 +- .../functions/string-functions.md | 3609 +---- .../functions/string-replace-functions.md | 512 +- .../functions/string-search-functions.md | 2695 +--- .../functions/time-series-functions.md | 431 +- .../functions/time-window-functions.md | 233 +- .../functions/tuple-functions.md | 841 +- .../functions/tuple-map-functions.md | 77 +- .../functions/type-conversion-functions.md | 308 +- .../current/sql-reference/functions/udf.md | 12 +- .../sql-reference/functions/ulid-functions.md | 96 +- .../functions/uniqtheta-functions.md | 14 +- .../sql-reference/functions/url-functions.md | 1408 +- .../sql-reference/functions/uuid-functions.md | 1230 +- .../current/sql-reference/index.md | 3 +- .../current/sql-reference/operators/exists.md | 2 +- .../current/sql-reference/operators/in.md | 15 +- .../current/sql-reference/operators/index.md | 20 +- .../sql-reference/sql-reference-links.json | 8 +- .../statements/alter/apply-deleted-mask.md | 6 +- .../sql-reference/statements/alter/column.md | 20 +- .../sql-reference/statements/alter/comment.md | 6 +- .../statements/alter/constraint.md | 2 +- .../statements/alter/database-comment.md | 6 +- .../sql-reference/statements/alter/delete.md | 2 +- .../sql-reference/statements/alter/index.md | 2 +- .../statements/alter/named-collection.md | 3 +- .../statements/alter/order-by.md | 2 +- .../statements/alter/partition.md | 64 +- .../statements/alter/projection.md | 10 +- .../statements/alter/row-policy.md | 2 +- .../statements/alter/sample-by.md | 11 +- .../sql-reference/statements/alter/setting.md | 10 +- .../statements/alter/skipping-index.md | 2 +- .../statements/alter/statistics.md | 22 +- .../sql-reference/statements/alter/ttl.md | 11 +- .../sql-reference/statements/alter/update.md | 2 +- .../sql-reference/statements/alter/user.md | 2 +- .../sql-reference/statements/alter/view.md | 2 +- .../sql-reference/statements/attach.md | 19 +- .../sql-reference/statements/check-grant.md | 4 +- .../sql-reference/statements/check-table.md | 6 +- .../statements/create/database.md | 15 +- .../statements/create/dictionary.md | 15 +- .../sql-reference/statements/create/index.md | 2 +- .../statements/create/named-collection.md | 3 +- .../sql-reference/statements/create/role.md | 2 +- .../statements/create/row-policy.md | 2 +- .../statements/create/settings-profile.md | 3 +- .../sql-reference/statements/create/table.md | 62 +- .../sql-reference/statements/create/user.md | 4 +- .../sql-reference/statements/create/view.md | 52 +- .../sql-reference/statements/delete.md | 4 +- .../current/sql-reference/statements/drop.md | 38 +- .../sql-reference/statements/exchange.md | 51 - .../sql-reference/statements/execute_as.md | 11 +- .../sql-reference/statements/exists.md | 2 +- .../sql-reference/statements/explain.md | 552 - .../current/sql-reference/statements/grant.md | 38 +- .../current/sql-reference/statements/index.md | 2 +- .../sql-reference/statements/insert-into.md | 14 +- .../current/sql-reference/statements/kill.md | 7 +- .../current/sql-reference/statements/move.md | 2 +- .../sql-reference/statements/optimize.md | 15 +- .../sql-reference/statements/parallel_with.md | 6 +- .../sql-reference/statements/rename.md | 13 +- .../sql-reference/statements/revoke.md | 6 +- .../sql-reference/statements/select/all.md | 2 +- .../statements/select/apply_modifier.md | 4 +- .../statements/select/array-join.md | 14 +- .../statements/select/distinct.md | 4 +- .../sql-reference/statements/select/except.md | 29 +- .../statements/select/except_modifier.md | 4 +- .../sql-reference/statements/select/format.md | 2 +- .../sql-reference/statements/select/from.md | 8 +- .../statements/select/group-by.md | 14 +- .../sql-reference/statements/select/having.md | 11 +- .../sql-reference/statements/select/index.md | 10 +- .../statements/select/intersect.md | 12 +- .../statements/select/into-outfile.md | 7 +- .../sql-reference/statements/select/join.md | 16 +- .../statements/select/limit-by.md | 22 +- .../sql-reference/statements/select/limit.md | 20 +- .../sql-reference/statements/select/offset.md | 3 +- .../statements/select/order-by.md | 12 +- .../statements/select/prewhere.md | 4 +- .../statements/select/qualify.md | 4 +- .../statements/select/replace_modifier.md | 4 +- .../sql-reference/statements/select/sample.md | 28 +- .../sql-reference/statements/select/union.md | 2 +- .../sql-reference/statements/select/where.md | 26 +- .../sql-reference/statements/select/with.md | 30 +- .../sql-reference/statements/set-role.md | 6 +- .../current/sql-reference/statements/set.md | 2 +- .../current/sql-reference/statements/show.md | 134 +- .../sql-reference/statements/system.md | 32 +- .../sql-reference/statements/truncate.md | 14 +- .../sql-reference/statements/undrop.md | 2 +- .../sql-reference/statements/update.md | 4 +- .../current/sql-reference/statements/use.md | 2 +- .../current/sql-reference/statements/watch.md | 5 +- .../current/sql-reference/syntax.md | 475 +- .../table-functions/arrowflight.md | 2 +- .../table-functions/azureBlobStorage.md | 14 +- .../azureBlobStorageCluster.md | 6 +- .../sql-reference/table-functions/cluster.md | 6 +- .../table-functions/deltalake.md | 6 +- .../table-functions/deltalakeCluster.md | 4 +- .../table-functions/dictionary.md | 6 +- .../table-functions/executable.md | 6 +- .../sql-reference/table-functions/file.md | 28 +- .../table-functions/fileCluster.md | 6 +- .../sql-reference/table-functions/format.md | 6 +- .../sql-reference/table-functions/fuzzJSON.md | 6 +- .../table-functions/fuzzQuery.md | 6 +- .../sql-reference/table-functions/gcs.md | 12 +- .../sql-reference/table-functions/generate.md | 6 +- .../table-functions/generate_series.md | 11 +- .../sql-reference/table-functions/hdfs.md | 10 +- .../table-functions/hdfsCluster.md | 6 +- .../sql-reference/table-functions/hudi.md | 4 +- .../table-functions/hudiCluster.md | 4 +- .../sql-reference/table-functions/iceberg.md | 50 +- .../table-functions/icebergCluster.md | 6 +- .../sql-reference/table-functions/index.md | 159 - .../sql-reference/table-functions/input.md | 8 +- .../sql-reference/table-functions/jdbc.md | 15 +- .../sql-reference/table-functions/loop.md | 6 +- .../sql-reference/table-functions/merge.md | 4 +- .../table-functions/mergeTreeIndex.md | 6 +- .../table-functions/mergeTreeProjection.md | 6 +- .../sql-reference/table-functions/mongodb.md | 8 +- .../sql-reference/table-functions/mysql.md | 8 +- .../sql-reference/table-functions/null.md | 6 +- .../sql-reference/table-functions/numbers.md | 2 +- .../sql-reference/table-functions/odbc.md | 6 +- .../sql-reference/table-functions/paimon.md | 4 +- .../table-functions/paimonCluster.md | 4 +- .../table-functions/postgresql.md | 8 +- .../table-functions/prometheusQuery.md | 6 +- .../table-functions/prometheusQueryRange.md | 6 +- .../sql-reference/table-functions/redis.md | 6 +- .../sql-reference/table-functions/remote.md | 22 +- .../sql-reference/table-functions/s3.md | 24 +- .../table-functions/s3Cluster.md | 6 +- .../sql-reference/table-functions/sqlite.md | 6 +- .../table-functions/timeSeriesData.md | 2 +- .../table-functions/timeSeriesMetrics.md | 2 +- .../table-functions/timeSeriesSelector.md | 6 +- .../table-functions/timeSeriesTags.md | 2 +- .../sql-reference/table-functions/url.md | 8 +- .../table-functions/urlCluster.md | 6 +- .../sql-reference/table-functions/values.md | 4 +- .../sql-reference/table-functions/view.md | 6 +- .../sql-reference/table-functions/ytsaurus.md | 4 +- .../sql-reference/table-functions/zeros.md | 2 +- .../current/sql-reference/transactions.md | 28 +- .../window-functions/cume_dist.md | 2 +- .../window-functions/dense_rank.md | 2 +- .../window-functions/first_value.md | 2 +- .../sql-reference/window-functions/index.md | 28 +- .../sql-reference/window-functions/lag.md | 2 +- .../window-functions/lagInFrame.md | 2 +- .../window-functions/last_value.md | 2 +- .../sql-reference/window-functions/lead.md | 2 +- .../window-functions/leadInFrame.md | 2 +- .../window-functions/nth_value.md | 2 +- .../window-functions/percent_rank.md | 2 +- .../sql-reference/window-functions/rank.md | 2 +- .../window-functions/row_number.md | 2 +- .../tips-and-tricks/debugging-insights.md | 32 +- .../tips-and-tricks/materialized-views.md | 2 +- .../performance-optimization.md | 6 +- .../current/tips-and-tricks/too-many-parts.md | 2 +- .../static-files-disk-uploader.md | 10 +- .../current/tutorial.md | 4 +- .../current/use-cases/AI_ML/AIChat/index.md | 2 +- .../use-cases/AI_ML/MCP/01_remote_mcp.md | 6 +- .../use-cases/AI_ML/MCP/02_claude-desktop.md | 4 +- .../use-cases/AI_ML/MCP/03_librechat.md | 26 +- .../use-cases/AI_ML/MCP/04_anythingllm.md | 10 +- .../use-cases/AI_ML/MCP/05_open-webui.md | 6 +- .../current/use-cases/AI_ML/MCP/06_ollama.md | 8 +- .../current/use-cases/AI_ML/MCP/07_janai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/agno.md | 6 +- .../AI_ML/MCP/ai_agent_libraries/chainlit.md | 6 +- .../ai_agent_libraries/claude-agent-sdk.md | 6 +- .../MCP/ai_agent_libraries/copilotkit.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/crewai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/dspy.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/index.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/langchain.md | 2 +- .../MCP/ai_agent_libraries/llamaindex.md | 10 +- .../AI_ML/MCP/ai_agent_libraries/mcp-agent.md | 2 +- .../microsoft-agent-framework.md | 2 +- .../MCP/ai_agent_libraries/openai-agents.md | 8 +- .../MCP/ai_agent_libraries/pydantic-ai.md | 8 +- .../AI_ML/MCP/ai_agent_libraries/slackbot.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/streamlit.md | 16 +- .../AI_ML/MCP/ai_agent_libraries/upsonic.md | 2 +- .../AI_ML/ai-powered-sql-generation.md | 16 +- .../data-exploration/jupyter-notebook.md | 30 +- .../AI_ML/data-exploration/marimo-notebook.md | 53 +- .../current/use-cases/AI_ML/index.md | 2 +- .../use-cases/data_lake/glue_catalog.md | 4 +- .../use-cases/data_lake/lakekeeper_catalog.md | 6 +- .../use-cases/data_lake/nessie_catalog.md | 8 +- .../use-cases/data_lake/onelake_catalog.md | 8 +- .../use-cases/data_lake/rest_catalog.md | 8 +- .../use-cases/data_lake/unity_catalog.md | 8 +- .../observability/build-your-own/grafana.md | 16 +- .../integrating-opentelemetry.md | 16 +- .../build-your-own/introduction.md | 4 +- .../build-your-own/managing-data.md | 14 +- .../build-your-own/schema-design.md | 1670 --- .../observability/clickstack/api-reference.md | 15 + .../observability/clickstack/config.md | 6 +- .../observability/clickstack/dashboards.md | 2 +- .../clickstack/deployment/all-in-one.md | 14 +- .../clickstack/deployment/docker-compose.md | 14 +- .../clickstack/deployment/helm/helm-cloud.md | 28 +- .../deployment/helm/helm-configuration.md | 28 +- .../helm/helm-deployment-options.md | 24 +- .../clickstack/deployment/helm/helm.md | 2 +- .../deployment/hyperdx-clickhouse-cloud.md | 2 +- .../clickstack/deployment/hyperdx-only.md | 4 +- .../clickstack/deployment/local-mode-only.md | 4 +- .../clickstack/example-datasets/kubernetes.md | 2 +- .../clickstack/example-datasets/local-data.md | 11 +- .../example-datasets/remote-demo-data.md | 2 +- .../clickstack/getting-started.md | 2 +- .../clickstack/ingesting-data/collector.md | 16 +- .../ingesting-data/opentelemetry.md | 4 +- .../clickstack/ingesting-data/schemas.md | 16 +- .../ingesting-data/sdks/aws-lambda.md | 8 +- .../clickstack/ingesting-data/sdks/browser.md | 10 +- .../clickstack/ingesting-data/sdks/deno.md | 7 +- .../clickstack/ingesting-data/sdks/elixir.md | 8 +- .../clickstack/ingesting-data/sdks/golang.md | 8 +- .../clickstack/ingesting-data/sdks/index.md | 2 +- .../clickstack/ingesting-data/sdks/java.md | 6 +- .../clickstack/ingesting-data/sdks/nestjs.md | 7 +- .../clickstack/ingesting-data/sdks/nextjs.md | 6 +- .../clickstack/ingesting-data/sdks/nodejs.md | 8 +- .../clickstack/ingesting-data/sdks/python.md | 12 +- .../ingesting-data/sdks/react-native.md | 10 +- .../clickstack/ingesting-data/sdks/ruby.md | 8 +- .../integration-examples/kafka-metrics.md | 8 +- .../integration-examples/kubernetes.md | 8 +- .../integration-examples/nginx-traces.md | 16 +- .../integration-examples/postgres-metrics.md | 14 +- .../integration-examples/redis-logs.md | 6 +- .../integration-examples/redis-metrics.md | 8 +- .../migration/elastic/migrating-sdks.md | 3 +- .../observability/clickstack/production.md | 6 +- .../observability/clickstack/search.md | 2 +- .../use-cases/observability/clickstack/ttl.md | 9 +- .../observability/cloud-monitoring.md | 6 +- .../observability/self-managed-monitoring.md | 6 +- .../time-series/01_date-time-data-types.md | 10 +- .../time-series/02_basic-operations.md | 17 +- .../time-series/03_analysis-functions.md | 21 +- .../time-series/04_storage-efficiency.md | 11 +- .../time-series/05_query-performance.md | 16 +- .../06_materialized-view-rollup.mdx | 25 +- .../current/whats-new/changelog/2020.md | 30 +- .../current/whats-new/changelog/2022.md | 40 +- .../current/whats-new/changelog/2023.md | 84 +- .../current/whats-new/changelog/2024.md | 82 +- .../current/whats-new/changelog/cloud.md | 4 +- .../current/whats-new/changelog/index.md | 4379 +++--- .../current/whats-new/security-changelog.md | 2 +- i18n/ru/code.json | 1111 +- .../current/Insert_select_settings_tuning.mdx | 15 +- ...ailed-error-using-PowerBI-CH-connector.mdx | 6 +- .../about-quotas-and-query-complexity.mdx | 10 +- .../current/add-column.mdx | 12 +- .../current/alter-user-settings-exception.mdx | 6 +- ...rialized_views_inserted_asynchronously.mdx | 1 - .../async_vs_optimize_read_in_order.mdx | 20 +- .../aws-privatelink-setup-for-clickpipes.mdx | 49 +- ...s-privatelink-setup-for-msk-clickpipes.mdx | 59 +- .../backing_up_a_specific_partition.mdx | 4 +- .../current/calculate-pi-using-sql.mdx | 3 +- ...ate_ratio_of_zero_sparse_serialization.mdx | 3 +- .../cannot-append-data-to-parquet-format.mdx | 3 +- .../certificate_verify_failed_error.mdx | 10 +- .../current/change-billing-email.mdx | 1 - ...change-the-prompt-in-clickhouse-client.mdx | 9 +- .../current/check-query-cache-in-use.mdx | 7 +- .../check_query_processing_time_only.mdx | 3 +- .../current/check_users_roles.mdx | 1 - .../current/clickhouse_cloud_api_usage.mdx | 3 +- .../current/columnar-database.mdx | 7 +- .../current/common-rbac-queries.mdx | 12 +- .../current/compare_resultsets.mdx | 3 +- .../comparing-metrics-between-queries.mdx | 4 +- .../current/configure-a-user-setting.mdx | 9 +- ...ap_ipc_lock_and_cap_sys_nice_in_docker.mdx | 6 +- ...connection_timeout_remote_remoteSecure.mdx | 3 +- .../current/custom-dns-alias-for-instance.mdx | 16 +- .../current/dbms-naming.mdx | 7 +- .../current/delete-old-data.mdx | 4 +- .../current/dictionaries-consistent-state.mdx | 4 +- .../current/dictionary_using_strings.mdx | 3 +- ..._between_official_builds_and_3rd_party.mdx | 19 +- .../enabling-ssl-with-lets-encrypt.mdx | 5 +- .../current/exception-too-many-parts.mdx | 3 +- .../exchangeStatementToSwitchTables.mdx | 3 +- .../execute-system-queries-in-cloud.mdx | 3 +- .../current/file-export.mdx | 9 +- .../current/filtered-aggregates.mdx | 3 +- .../current/find-expensive-queries.mdx | 9 +- ...ding_expensive_queries_by_memory_usage.mdx | 3 +- ...-developer-verification-error-in-macos.mdx | 17 +- .../current/generate-har-file.mdx | 7 +- ...how-do-i-contribute-code-to-clickhouse.mdx | 1 - ...check-my-clickhouse-cloud-sevice-state.mdx | 9 +- ...-to-connect-to-ch-cloud-using-ssh-keys.mdx | 5 +- ...able-to-query-multiple-remote-clusters.mdx | 3 +- ...-a-clickhouse-table-by-an-array-column.mdx | 5 +- .../how-to-increase-thread-pool-size.mdx | 3 +- ...-to-insert-all-rows-from-another-table.mdx | 3 +- ...set-up-ch-on-docker-odbc-connect-mssql.mdx | 27 +- .../how_to_display_queries_using_MV.mdx | 3 +- .../current/how_to_use_parametrised_views.mdx | 3 +- .../current/ignoring-incorrect-settings.mdx | 3 +- ...ting-geojason-with-nested-object-array.mdx | 8 +- ...ng_and_working_with_JSON_array_objects.mdx | 7 +- .../current/improve-map-performance.mdx | 4 +- .../current/ingest-failures-23-9-release.mdx | 3 +- .../current/ingest-parquet-files-in-s3.mdx | 4 +- .../current/install-clickhouse-windows10.mdx | 4 +- .../current/json-import.mdx | 10 +- .../current/json_extract_example.mdx | 3 +- .../current/json_simple_example.mdx | 5 +- .../current/kafka-clickhouse-json.mdx | 13 +- .../current/kafka-to-clickhouse-setup.mdx | 11 +- .../current/key-value.mdx | 7 +- .../current/llvm-clang-up-to-date.mdx | 3 +- ...f-system-metrics-to-prometheus-metrics.mdx | 101 +- .../current/mapreduce.mdx | 7 +- ...maximum_number_of_tables_and_databases.mdx | 5 +- .../memory-limit-exceeded-for-query.mdx | 3 +- .../current/multi-region-replication.mdx | 1 - .../current/mysql-to-parquet-csv-json.mdx | 12 +- .../current/node-js-example.mdx | 3 +- .../current/olap.mdx | 5 +- .../current/optimize_final_vs_final.mdx | 19 +- .../current/oracle-odbc.mdx | 3 +- .../outputSendLogsLevelTracesToFile.mdx | 8 +- .../current/parquet-to-csv-json.mdx | 15 +- .../current/part_intersects_previous_part.mdx | 15 +- .../current/pivot.mdx | 25 +- .../postgresql-to-parquet-csv-json.mdx | 15 +- .../current/production.mdx | 47 +- .../profiling-clickhouse-with-llvm-xray.mdx | 17 +- .../current/projection_example.mdx | 3 +- .../python-clickhouse-connect-example.mdx | 3 +- .../current/python_http_requests.mdx | 5 +- .../current/query_max_execution_time.mdx | 3 +- .../current/read_consistency.mdx | 3 +- .../recreate_table_across_terminals.mdx | 3 +- .../current/remove-default-user.mdx | 9 +- .../restore-replica-after-storage-failure.mdx | 10 +- .../current/row-column-policy.mdx | 3 +- .../s3_export_data_year_month_folders.mdx | 23 +- .../current/schema_migration_tools.mdx | 21 +- ...across-node-for-tables-with-a-wildcard.mdx | 1 - .../current/send_logs_level.mdx | 3 +- .../current/terraform_example.mdx | 4 +- .../current/time-series.mdx | 5 +- ...imizing-basic-data-types-in-clickhouse.mdx | 12 +- .../unable-to-access-cloud-service.mdx | 3 +- .../use-clickhouse-for-log-analytics.mdx | 3 +- .../useful-queries-for-troubleshooting.mdx | 38 +- ...y-join-to-extract-and-query-attributes.mdx | 5 +- .../current/vector-search.mdx | 19 +- .../view_number_of_active_mutations.mdx | 3 +- .../current/when_is_ttl_applied.mdx | 9 +- .../which-processes-are-currently-running.mdx | 4 +- .../current/who-is-using-clickhouse.mdx | 3 +- .../current/why_default_logging_verbose.mdx | 11 +- .../why_is_my_primary_key_not_used.mdx | 16 +- ...mmend_clickhouse_keeper_over_zookeeper.mdx | 17 +- .../windows_active_directory_to_ch_roles.mdx | 6 +- .../current.json | 6 +- .../_GCS_authentication_and_bucket.md | 10 +- .../current/_snippets/_add_superset_detail.md | 4 +- .../_clickhouse_mysql_cloud_setup.mdx | 69 +- .../current/_snippets/_clickstack_tagging.mdx | 9 +- ...irect_observability_integration_options.md | 6 +- .../_snippets/_users-and-roles-common.md | 24 +- .../current/about-us/cloud.md | 2 +- .../current/about-us/distinctive-features.md | 2 +- .../current/about-us/index.md | 2 +- .../current/about-us/support.md | 10 +- .../_snippets/_async_inserts.md | 4 +- .../best-practices/avoid_optimize_final.md | 2 +- .../best-practices/choosing_a_primary_key.md | 4 +- .../current/best-practices/index.md | 3 +- .../current/best-practices/json_type.md | 2 +- .../best-practices/partitioning_keys.mdx | 5 +- .../best-practices/select_data_type.md | 7 +- .../selecting_an_insert_strategy.md | 13 +- .../sizing-and-hardware-recommendations.md | 2 +- .../using_data_skipping_indices.md | 2 +- .../current/chdb/getting-started.md | 14 +- .../current/chdb/guides/clickhouse-local.md | 13 +- .../current/chdb/guides/jupysql.md | 26 +- .../chdb/guides/query-remote-clickhouse.md | 8 +- .../chdb/guides/querying-apache-arrow.md | 10 +- .../current/chdb/guides/querying-pandas.md | 19 +- .../current/chdb/guides/querying-parquet.md | 11 +- .../current/chdb/guides/querying-s3-bucket.md | 14 +- .../current/chdb/index.md | 2 +- .../current/chdb/install/bun.md | 23 +- .../current/chdb/install/c.md | 26 +- .../current/chdb/install/go.md | 24 +- .../current/chdb/install/nodejs.md | 14 +- .../current/chdb/install/python.md | 180 +- .../current/chdb/install/rust.md | 20 +- .../current/cloud/features/01_cloud_tiers.md | 2 +- .../03_sql_console_features/01_sql-console.md | 213 +- .../02_query-insights.md | 2 +- .../03_query-endpoints.md | 3 +- .../03_sql_console_features/04_dashboards.md | 2 +- .../automatic_scaling/01_auto_scaling.md | 4 +- .../04_infrastructure/deployment-options.md | 2 +- .../replica-aware-routing.md | 2 +- .../04_infrastructure/shared-catalog.md | 2 +- .../04_infrastructure/shared-merge-tree.md | 4 +- .../features/04_infrastructure/warehouses.md | 4 +- .../05_admin_features/api/api-overview.md | 2 +- .../features/05_admin_features/api/openapi.md | 2 +- .../features/05_admin_features/api/postman.md | 14 +- .../features/05_admin_features/upgrades.md | 2 +- .../current/cloud/features/06_security.md | 2 +- .../07_monitoring/advanced_dashboard.md | 10 +- .../features/07_monitoring/prometheus.md | 72 +- .../guides/SQL_console/connection_details.md | 2 +- .../guides/SQL_console/query-endpoints.md | 32 +- .../backups/01_review-and-restore-backups.md | 10 +- .../01_export-backups-to-own-cloud-account.md | 35 +- .../cloud/guides/best_practices/index.md | 2 +- .../guides/best_practices/multitenancy.md | 8 +- .../cloud/guides/cloud-compatibility.md | 2 +- .../data_sources/01_cloud-endpoints-api.md | 3 +- .../02_accessing-s3-data-securely.md | 12 +- .../current/cloud/guides/index.md | 2 - .../byoc/03_onboarding/01_aws.md | 14 +- .../byoc/05_observability/01_aws.md | 16 +- .../cloud/guides/production-readiness.md | 12 +- .../03_manage-sql-console-role-assignments.md | 2 +- .../04_manage-database-users.md | 14 +- .../04_saml-sso-setup.md | 2 +- .../05_common-access-management-queries.md | 14 +- .../01_cloud_access_management/index.md | 2 +- .../02_connectivity/01_setting-ip-filters.md | 2 +- .../private_networking/02_aws-privatelink.md | 48 +- .../03_gcp-private-service-connect.md | 52 +- .../04_azure-privatelink.md | 60 +- .../private_networking/index.md | 8 +- .../cloud/guides/security/03_data-masking.md | 20 +- .../current/cloud/guides/security/04_cmek.md | 2 +- .../05_audit_logging/02_database-audit-log.md | 2 +- .../03_byoc-security-playbook.md | 4 +- .../guides/security/05_audit_logging/index.md | 4 +- .../cloud/onboard/01_discover/01_what_is.md | 63 +- .../01_discover/02_use_cases/00_overview.md | 2 +- .../02_use_cases/01_real-time-analytics.md | 2 +- .../02_use_cases/02_observability.md | 14 +- .../01_machine_learning.md | 2 +- .../01_migration_guides/01_overview.md | 25 +- .../02_postgres/01_overview.md | 2 +- .../02_postgres/appendix.md | 2 +- .../01_migration_guide_part1.md | 20 +- .../02_migration_guide_part2.md | 2 +- .../03_migration_guide_part3.md | 6 +- .../03_bigquery/01_overview.md | 4 +- .../02_migrating-to-clickhouse-cloud.md | 18 +- .../03_bigquery/03_loading-data.md | 4 +- .../04_snowflake/01_overview.md | 2 +- .../04_snowflake/02_migration_guide.md | 6 +- .../03_sql_translation_reference.md | 2 +- .../05_elastic/01_overview.md | 2 +- .../06_redshift/01_overview.md | 2 +- .../06_redshift/02_migration_guide.md | 7 +- .../03_sql_translation_reference.md | 6 +- .../07_OSS_to_Cloud/01_clickhouse-to-cloud.md | 34 +- .../01_clickhouse-local-etl.md | 22 +- .../02_etl-tool-to-clickhouse.md | 11 +- .../03_object-storage-to-clickhouse.md | 11 +- .../cloud/onboard/03_tune/resource_tour.md | 8 +- .../current/cloud/onboard/index.md | 2 +- .../reference/01_changelog/01_changelog.md | 2 +- .../01_changelog/02_release_notes/24_02.md | 8 +- .../01_changelog/02_release_notes/24_05.md | 4 +- .../01_changelog/02_release_notes/24_06.md | 4 +- .../01_changelog/02_release_notes/24_10.md | 2 +- .../01_changelog/02_release_notes/25_08.md | 2 +- .../cloud/reference/02_architecture.md | 2 +- .../03_billing/01_billing_overview.md | 22 +- .../02_marketplace/aws-marketplace-payg.md | 45 +- .../03_billing/04_network-data-transfer.mdx | 2 +- .../03_billing/05_payment-thresholds.md | 4 +- .../03_billing/06_billing_compliance.md | 2 +- .../cloud/reference/05_supported-regions.md | 2 +- .../current/cloud/reference/08_settings.md | 5 +- .../09_security/03_compliance-overview.md | 2 +- .../cloud/reference/11_account-close.md | 2 +- .../current/cloud/reference/index.md | 2 +- .../current/concepts/glossary.md | 2 +- .../current/concepts/olap.md | 2 +- .../concepts/why-clickhouse-is-so-fast.mdx | 26 +- .../compression-in-clickhouse.md | 4 +- .../data-compression/compression-modes.md | 2 +- .../current/data-modeling/backfilling.md | 18 +- .../current/data-modeling/denormalization.md | 14 +- .../current/data-modeling/index.md | 2 +- .../projections/1_projections.md | 344 +- ...2_materialized-views-versus-projections.md | 126 +- .../current/deployment-guides/index.md | 2 +- .../deployment-guides/parallel-replicas.mdx | 306 +- .../01_1_shard_2_replicas.md | 25 +- .../02_2_shards_1_replica.md | 25 +- .../03_2_shards_2_replicas.md | 17 +- .../current/deployment-guides/terminology.md | 2 +- .../current/development/architecture.md | 4 +- .../current/development/build-cross-arm.md | 2 +- .../development/build-cross-loongarch.md | 8 +- .../current/development/build-cross-osx.md | 11 +- .../current/development/build-cross-riscv.md | 8 +- .../current/development/build-cross-s390x.md | 88 +- .../current/development/build-e2k.md | 8 +- .../current/development/build-osx.md | 8 +- .../current/development/build.md | 122 +- .../building_and_benchmarking_deflate_qpl.md | 18 +- .../development/continuous-integration.md | 24 +- .../current/development/contrib.md | 2 +- .../development/developer-instruction.md | 136 +- .../current/development/index.md | 46 +- .../development/integrating_rust_libraries.md | 3 +- .../current/development/style.md | 14 +- .../current/development/tests.md | 34 +- .../current/dictionary/index.md | 130 +- .../engines/database-engines/atomic.md | 18 +- .../engines/database-engines/backup.md | 11 +- .../engines/database-engines/datalake.md | 6 +- .../current/engines/database-engines/index.md | 26 +- .../current/engines/database-engines/lazy.md | 12 +- .../materialized-postgresql.md | 30 +- .../current/engines/database-engines/mysql.md | 15 +- .../engines/database-engines/postgresql.md | 6 +- .../engines/database-engines/replicated.md | 8 +- .../engines/database-engines/shared.md | 4 +- .../engines/database-engines/sqlite.md | 6 +- .../current/engines/table-engines/index.md | 2 +- .../integrations/ExternalDistributed.md | 11 +- .../table-engines/integrations/arrowflight.md | 6 +- .../table-engines/integrations/azure-queue.md | 106 +- .../integrations/azureBlobStorage.md | 14 +- .../table-engines/integrations/deltalake.md | 6 +- .../integrations/embedded-rocksdb.md | 36 +- .../table-engines/integrations/hdfs.md | 16 +- .../table-engines/integrations/hive.md | 36 +- .../table-engines/integrations/hudi.md | 4 +- .../table-engines/integrations/iceberg.md | 26 +- .../table-engines/integrations/index.md | 67 +- .../table-engines/integrations/jdbc.md | 6 +- .../table-engines/integrations/kafka.md | 12 +- .../integrations/materialized-postgresql.md | 6 +- .../table-engines/integrations/mongodb.md | 12 +- .../table-engines/integrations/mysql.md | 6 +- .../table-engines/integrations/nats.md | 6 +- .../table-engines/integrations/odbc.md | 6 +- .../table-engines/integrations/postgresql.md | 20 +- .../table-engines/integrations/rabbitmq.md | 6 +- .../table-engines/integrations/redis.md | 6 +- .../engines/table-engines/integrations/s3.md | 36 +- .../table-engines/integrations/s3queue.md | 315 +- .../table-engines/integrations/sqlite.md | 12 +- .../table-engines/integrations/time-series.md | 18 +- .../table-engines/integrations/ytsaurus.md | 6 +- .../engines/table-engines/log-family/index.md | 2 +- .../engines/table-engines/log-family/log.md | 6 +- .../table-engines/log-family/stripelog.md | 6 +- .../table-engines/log-family/tinylog.md | 6 +- .../mergetree-family/aggregatingmergetree.md | 6 +- .../mergetree-family/annindexes.md | 28 +- .../mergetree-family/coalescingmergetree.md | 17 +- .../mergetree-family/collapsingmergetree.md | 16 +- .../custom-partitioning-key.md | 10 +- .../mergetree-family/graphitemergetree.md | 28 +- .../table-engines/mergetree-family/index.md | 4 +- .../mergetree-family/invertedindexes.md | 50 +- .../mergetree-family/mergetree.md | 802 +- .../mergetree-family/replacingmergetree.md | 12 +- .../mergetree-family/replication.md | 164 +- .../mergetree-family/summingmergetree.md | 20 +- .../versionedcollapsingmergetree.md | 16 +- .../engines/table-engines/special/alias.md | 107 +- .../engines/table-engines/special/buffer.md | 25 +- .../table-engines/special/dictionary.md | 8 +- .../table-engines/special/distributed.md | 12 +- .../table-engines/special/executable.md | 19 +- .../table-engines/special/external-data.md | 2 +- .../engines/table-engines/special/file.md | 8 +- .../engines/table-engines/special/filelog.md | 4 +- .../engines/table-engines/special/generate.md | 6 +- .../engines/table-engines/special/index.md | 5 +- .../engines/table-engines/special/join.md | 6 +- .../table-engines/special/keepermap.md | 12 +- .../engines/table-engines/special/memory.md | 6 +- .../engines/table-engines/special/merge.md | 6 +- .../engines/table-engines/special/null.md | 2 +- .../engines/table-engines/special/set.md | 2 +- .../engines/table-engines/special/url.md | 4 +- .../engines/table-engines/special/view.md | 2 +- .../current/faq/general/concurrency.md | 2 +- .../current/faq/general/cost-based.md | 2 +- .../current/faq/general/datalake.md | 2 +- .../current/faq/general/dependencies.md | 2 +- .../current/faq/general/distributed-join.md | 2 +- .../current/faq/general/federated.md | 22 +- .../current/faq/general/index.md | 22 +- .../current/faq/general/sql.md | 60 +- .../current/faq/general/updates.md | 2 +- .../current/faq/integration/index.md | 16 +- .../current/faq/integration/json-import.md | 2 +- .../current/faq/integration/oracle-odbc.md | 4 +- .../current/faq/operations/delete-old-data.md | 2 +- .../current/faq/operations/index.md | 18 +- .../current/faq/use-cases/index.md | 6 +- .../example-datasets/amazon-reviews.md | 8 +- .../anon_web_analytics_metrica.md | 22 +- .../example-datasets/brown-benchmark.md | 4 +- .../example-datasets/cell-towers.md | 13 +- .../example-datasets/dbpedia.md | 14 +- .../example-datasets/foursquare-os-places.md | 4 +- .../example-datasets/github.md | 58 +- .../example-datasets/hacker-news.md | 2 +- .../getting-started/example-datasets/laion.md | 28 +- .../getting-started/example-datasets/menus.md | 24 +- .../getting-started/example-datasets/noaa.md | 22 +- .../example-datasets/nyc-taxi.md | 12 +- .../example-datasets/nypd_complaint_data.md | 22 +- .../example-datasets/ontime.md | 6 +- .../example-datasets/stackoverflow.md | 30 +- .../getting-started/example-datasets/tpch.md | 18 +- .../example-datasets/tw-weather.md | 36 +- .../example-datasets/uk-price-paid.md | 12 +- .../example-datasets/youtube-dislikes.md | 10 +- .../current/getting-started/index.md | 5 +- .../install/_snippets/_deb_install.md | 117 +- .../install/_snippets/_docker.md | 36 +- .../install/_snippets/_linux_tar_install.md | 16 +- .../install/_snippets/_macos.md | 10 +- .../install/_snippets/_quick_install.md | 8 +- .../install/_snippets/_rpm_install.md | 8 +- .../install/_snippets/_windows_install.md | 10 +- .../getting-started/install/advanced.md | 2 +- .../getting-started/install/install.mdx | 28 +- .../current/getting-started/playground.md | 4 +- .../getting-started/quick-start/cloud.mdx | 43 +- .../getting-started/quick-start/oss.mdx | 25 +- .../current/guides/best-practices/index.md | 11 +- .../current/guides/best-practices/prewhere.md | 4 +- .../best-practices/query-optimization.md | 22 +- .../best-practices/query-parallelism.md | 8 +- .../skipping-indexes-examples.md | 18 +- .../guides/best-practices/skipping-indexes.md | 14 +- .../best-practices/sparse-primary-indexes.md | 48 +- .../developer/alternative-query-languages.md | 9 +- .../developer/cascading-materialized-views.md | 31 +- .../developer/debugging-memory-issues.md | 14 +- .../deduplicating-inserts-on-retries.md | 10 +- .../current/guides/developer/deduplication.md | 10 +- .../developer/dynamic-column-selection.md | 19 +- .../current/guides/developer/index.md | 4 +- .../guides/developer/merge-table-function.md | 15 +- .../guides/developer/on-fly-mutations.md | 4 +- .../guides/developer/replacing-merge-tree.md | 8 +- ...ored-procedures-and-prepared-statements.md | 18 +- .../developer/time-series-filling-gaps.md | 27 +- .../current/guides/developer/ttl.md | 14 +- ...nding-query-execution-with-the-analyzer.md | 10 +- .../aggregate_function_combinators/anyIf.md | 2 +- .../argMaxIf.md | 2 +- .../argMinIf.md | 2 +- .../aggregate_function_combinators/avgIf.md | 2 +- .../aggregate_function_combinators/avgMap.md | 2 +- .../avgMergeState.md | 2 +- .../avgResample.md | 4 +- .../avgState.md | 2 +- .../aggregate_function_combinators/countIf.md | 2 +- .../countResample.md | 4 +- .../groupArrayDistinct.md | 2 +- .../groupArrayResample.md | 2 +- .../aggregate_function_combinators/maxMap.md | 2 +- .../aggregate_function_combinators/minMap.md | 2 +- .../minSimpleState.md | 2 +- .../quantilesTimingArrayIf.md | 2 +- .../quantilesTimingIf.md | 2 +- .../sumArray.md | 2 +- .../sumForEach.md | 2 +- .../aggregate_function_combinators/sumIf.md | 6 +- .../aggregate_function_combinators/sumMap.md | 2 +- .../sumSimpleState.md | 4 +- .../uniqArray.md | 2 +- .../uniqArrayIf.md | 4 +- .../current/guides/manage-and-deploy-index.md | 2 +- .../guides/separation-storage-compute.md | 19 +- .../current/guides/sre/configuring-ssl.md | 26 +- .../current/guides/sre/keeper/index.md | 60 +- .../current/guides/sre/network-ports.md | 2 +- .../current/guides/sre/scaling-clusters.md | 2 +- .../sre/user-management/configuring-ldap.md | 4 +- .../guides/sre/user-management/index.md | 4 +- .../sre/user-management/ssl-user-auth.md | 4 +- .../guides/starter_guides/creating-tables.md | 2 +- .../starter_guides/generating-test-data.md | 24 +- .../guides/starter_guides/inserting-data.md | 2 +- .../guides/starter_guides/mutations.md | 18 +- .../starter_guides/working_with_arrays.md | 20 +- .../current/guides/troubleshooting.md | 30 +- .../data-ingestion/apache-spark/index.md | 9 +- .../data-ingestion/apache-spark/spark-jdbc.md | 10 +- .../apache-spark/spark-native-connector.md | 20 +- .../data-ingestion/aws-glue/index.md | 2 +- .../azure-data-factory/overview.md | 5 +- .../using_azureblobstorage.md | 4 +- .../using_http_interface.md | 2 +- .../data-ingestion/azure-synapse/index.md | 4 +- .../clickpipes/aws-privatelink.md | 2 +- .../data-ingestion/clickpipes/index.md | 2 +- .../clickpipes/kafka/03_reference.md | 2 +- .../clickpipes/kafka/04_best_practices.md | 8 +- .../data-ingestion/clickpipes/kafka/index.md | 18 +- .../data-ingestion/clickpipes/kinesis.md | 2 +- .../clickpipes/mongodb/add_table.md | 2 +- .../data-ingestion/clickpipes/mongodb/faq.md | 18 +- .../clickpipes/mongodb/index.md | 2 +- .../clickpipes/mongodb/quickstart.md | 28 +- .../clickpipes/mongodb/resync.md | 6 +- .../clickpipes/mongodb/source/atlas.md | 2 +- .../clickpipes/mongodb/source/documentdb.md | 10 +- .../clickpipes/mongodb/source/generic.md | 6 +- .../clickpipes/mysql/add_table.md | 2 +- .../data-ingestion/clickpipes/mysql/faq.md | 2 +- .../data-ingestion/clickpipes/mysql/index.md | 2 +- .../data-ingestion/clickpipes/mysql/resync.md | 6 +- .../clickpipes/mysql/source/aurora.md | 8 +- .../clickpipes/mysql/source/gcp.md | 2 +- .../clickpipes/mysql/source/generic.md | 8 +- .../clickpipes/mysql/source/generic_maria.md | 4 +- .../clickpipes/mysql/source/rds.md | 8 +- .../clickpipes/mysql/source/rds_maria.md | 8 +- .../clickpipes/postgres/add_table.md | 2 +- .../clickpipes/postgres/deduplication.md | 23 +- .../data-ingestion/clickpipes/postgres/faq.md | 48 +- .../clickpipes/postgres/index.md | 2 +- .../clickpipes/postgres/maintenance.md | 6 +- .../clickpipes/postgres/ordering_keys.md | 13 +- .../clickpipes/postgres/resync.md | 6 +- .../clickpipes/postgres/scaling.md | 6 +- .../clickpipes/postgres/source/aurora.md | 4 +- .../source/azure-flexible-server-postgres.md | 2 +- .../postgres/source/crunchy-postgres.md | 4 +- .../clickpipes/postgres/source/generic.md | 6 +- .../postgres/source/google-cloudsql.md | 2 +- .../postgres/source/neon-postgres.md | 6 +- .../clickpipes/postgres/source/planetscale.md | 4 +- .../clickpipes/postgres/source/rds.md | 4 +- .../clickpipes/postgres/source/supabase.md | 6 +- .../clickpipes/postgres/source/timescale.md | 4 +- .../clickpipes/postgres/toast.md | 4 +- .../data-ingestion/clickpipes/secure-rds.md | 21 +- .../data-formats/arrow-avro-orc.md | 14 +- .../data-ingestion/data-formats/binary.md | 24 +- .../data-ingestion/data-formats/csv-tsv.md | 32 +- .../data-ingestion/data-formats/intro.md | 2 +- .../data-formats/json/exporting.md | 16 +- .../data-formats/json/formats.md | 36 +- .../data-formats/json/inference.md | 12 +- .../data-ingestion/data-formats/json/intro.md | 27 +- .../data-formats/json/loading.md | 6 +- .../data-ingestion/data-formats/json/other.md | 36 +- .../data-formats/json/schema.md | 20 +- .../data-ingestion/data-formats/parquet.md | 14 +- .../data-ingestion/data-formats/sql.md | 8 +- .../data-formats/templates-regex.md | 14 +- .../data-ingestion/data-ingestion-index.md | 2 +- .../data-ingestion/data-sources-index.md | 2 +- .../data-ingestion/dbms/dynamodb/index.md | 10 +- .../dbms/jdbc-with-clickhouse.md | 6 +- .../postgresql/connecting-to-postgresql.md | 28 +- .../integrations/data-ingestion/emqx/index.md | 24 +- .../etl-tools/airbyte-and-clickhouse.md | 4 +- .../data-ingestion/etl-tools/apache-beam.md | 8 +- .../etl-tools/bladepipe-and-clickhouse.md | 2 +- .../dbt/features-and-configurations.md | 84 +- .../data-ingestion/etl-tools/dbt/guides.md | 16 +- .../data-ingestion/etl-tools/dbt/index.md | 16 +- .../etl-tools/dlt-and-clickhouse.md | 18 +- .../etl-tools/fivetran/index.md | 2 +- .../etl-tools/nifi-and-clickhouse.md | 4 +- .../etl-tools/vector-to-clickhouse.md | 371 +- .../integrations/data-ingestion/gcs/index.md | 62 +- .../google-dataflow/dataflow.md | 2 +- .../google-dataflow/java-runner.md | 2 +- .../google-dataflow/templates.md | 2 +- .../templates/bigquery-to-clickhouse.md | 2 +- .../data-ingestion/insert-local-files.md | 3 +- .../kafka/confluent/confluent-cloud.md | 4 +- .../kafka/confluent/custom-connector.md | 22 +- .../data-ingestion/kafka/confluent/index.md | 2 +- .../kafka/confluent/kafka-connect-http.md | 35 +- .../data-ingestion/kafka/index.md | 2 +- .../kafka/kafka-clickhouse-connect-sink.md | 98 +- .../kafka/kafka-connect-jdbc.md | 10 +- .../kafka-table-engine-named-collections.md | 22 +- .../kafka/kafka-table-engine.md | 62 +- .../data-ingestion/kafka/kafka-vector.md | 15 +- .../data-ingestion/kafka/msk/index.md | 14 +- .../redshift/_snippets/_migration_guide.md | 2 +- .../integrations/data-ingestion/s3-minio.md | 4 +- .../integrations/data-ingestion/s3/index.md | 671 +- .../data-ingestion/s3/performance.md | 24 +- .../integrations/data-sources/cassandra.md | 2 +- .../integrations/data-sources/mysql.md | 16 +- .../integrations/data-sources/postgres.md | 2 +- .../data-sources/table_engines/deltalake.md | 2 +- .../data-sources/table_engines/hive.md | 2 +- .../data-sources/table_engines/hudi.md | 2 +- .../data-sources/table_engines/iceberg.md | 2 +- .../data-sources/table_engines/mongodb.md | 2 +- .../data-sources/table_engines/nats.md | 2 +- .../data-sources/table_engines/rabbitmq.md | 2 +- .../data-sources/table_engines/redis.md | 2 +- .../data-sources/table_engines/rocksdb.md | 2 +- .../data-sources/table_engines/sqlite.md | 2 +- .../astrato-and-clickhouse.md | 4 +- .../chartbrew-and-clickhouse.md | 6 +- .../databrain-and-clickhouse.md | 14 +- .../community_integrations/deepnote.md | 4 +- .../dot-and-clickhouse.md | 2 +- .../draxlr-and-clickhouse.md | 4 +- .../embeddable-and-clickhouse.md | 6 +- .../explo-and-clickhouse.md | 4 +- .../fabi-and-clickhouse.md | 4 +- .../hashboard-and-clickhouse.md | 4 +- .../luzmo-and-clickhouse.md | 4 +- .../mitzu-and-clickhouse.md | 4 +- .../rocketbi-and-clickhouse.md | 6 +- .../zingdata-and-clickhouse.md | 4 +- .../data-visualization/grafana/config.md | 22 +- .../data-visualization/grafana/index.md | 4 +- .../grafana/query-builder.md | 8 +- .../integrations/data-visualization/index.md | 2 +- .../lightdash-and-clickhouse.md | 4 +- .../looker-and-clickhouse.md | 4 +- .../looker-studio-and-clickhouse.md | 6 +- .../metabase-and-clickhouse.md | 4 +- .../data-visualization/omni-and-clickhouse.md | 4 +- .../powerbi-and-clickhouse.md | 4 +- .../quicksight-and-clickhouse.md | 6 +- .../splunk-and-clickhouse.md | 8 +- .../superset-and-clickhouse.md | 4 +- .../tableau/tableau-analysis-tips.md | 8 +- .../tableau/tableau-and-clickhouse.md | 4 +- .../tableau/tableau-connection-tips.md | 6 +- .../tableau/tableau-online-and-clickhouse.md | 6 +- .../current/integrations/index.mdx | 3 +- .../integrations/language-clients/csharp.md | 34 +- .../integrations/language-clients/go/index.md | 112 +- .../java/client/_snippets/_v0_7.mdx | 140 +- .../java/client/_snippets/_v0_8.mdx | 200 +- .../language-clients/java/index.md | 6 +- .../java/jdbc/_snippets/_v0_7.mdx | 171 +- .../java/jdbc/_snippets/_v0_8.mdx | 90 +- .../language-clients/java/r2dbc.md | 10 +- .../integrations/language-clients/js.md | 58 +- .../language-clients/moose-olap.md | 14 +- .../python/additional-options.md | 2 +- .../python/advanced-inserting.md | 44 +- .../python/advanced-querying.md | 138 +- .../language-clients/python/advanced-usage.md | 8 +- .../language-clients/python/driver-api.md | 172 +- .../language-clients/python/index.md | 10 +- .../language-clients/python/sqlalchemy.md | 16 +- .../integrations/language-clients/rust.md | 32 +- .../current/integrations/misc/index.md | 2 +- .../integrations/sql-clients/datagrip.md | 4 +- .../integrations/sql-clients/dbeaver.md | 2 +- .../integrations/sql-clients/dbvisualizer.md | 15 +- .../current/integrations/sql-clients/index.md | 2 +- .../integrations/sql-clients/jupysql.md | 5 +- .../integrations/sql-clients/marimo.md | 6 +- .../integrations/sql-clients/qstudio.md | 15 +- .../integrations/sql-clients/sql-console.md | 15 +- .../integrations/sql-clients/tablum.md | 2 +- .../tools/data-integration/easypanel/index.md | 2 +- .../tools/data-integration/index.md | 2 +- .../tools/data-integration/retool/index.md | 4 +- .../tools/data-integration/splunk/index.md | 8 +- .../current/integrations/tools/index.md | 2 +- .../current/interfaces/arrowflight.md | 6 +- .../current/interfaces/cli.md | 85 +- .../current/interfaces/cpp.md | 4 +- .../current/interfaces/formats/Arrow/Arrow.md | 4 +- .../current/interfaces/formats/Avro/Avro.md | 6 +- .../interfaces/formats/Avro/AvroConfluent.md | 6 +- .../current/interfaces/formats/BSONEachRow.md | 4 +- .../current/interfaces/formats/CSV/CSV.md | 2 +- .../interfaces/formats/CSV/CSVWithNames.md | 4 +- .../formats/CSV/CSVWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/CapnProto.md | 6 +- .../CustomSeparated/CustomSeparated.md | 4 +- .../CustomSeparatedIgnoreSpaces.md | 2 +- .../CustomSeparatedIgnoreSpacesWithNames.md | 2 +- ...mSeparatedIgnoreSpacesWithNamesAndTypes.md | 2 +- .../CustomSeparatedWithNames.md | 4 +- .../CustomSeparatedWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/DWARF.md | 2 +- .../current/interfaces/formats/Form.md | 2 +- .../current/interfaces/formats/Hash.md | 2 +- .../current/interfaces/formats/JSON/JSON.md | 2 +- .../interfaces/formats/JSON/JSONAsObject.md | 6 +- .../interfaces/formats/JSON/JSONAsString.md | 4 +- .../interfaces/formats/JSON/JSONColumns.md | 4 +- .../formats/JSON/JSONColumnsWithMetadata.md | 2 +- .../interfaces/formats/JSON/JSONCompact.md | 4 +- .../formats/JSON/JSONCompactColumns.md | 4 +- .../formats/JSON/JSONCompactEachRow.md | 4 +- .../JSON/JSONCompactEachRowWithNames.md | 4 +- .../JSONCompactEachRowWithNamesAndTypes.md | 4 +- .../JSON/JSONCompactEachRowWithProgress.md | 2 +- .../formats/JSON/JSONCompactStrings.md | 2 +- .../formats/JSON/JSONCompactStringsEachRow.md | 4 +- .../JSONCompactStringsEachRowWithNames.md | 4 +- ...NCompactStringsEachRowWithNamesAndTypes.md | 4 +- .../JSONCompactStringsEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONEachRow.md | 4 +- .../formats/JSON/JSONEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONLines.md | 4 +- .../formats/JSON/JSONObjectEachRow.md | 16 +- .../interfaces/formats/JSON/JSONStrings.md | 2 +- .../formats/JSON/JSONStringsEachRow.md | 6 +- .../JSON/JSONStringsEachRowWithProgress.md | 2 +- .../formats/JSON/PrettyJSONEachRow.md | 2 +- .../formats/LineAsString/LineAsString.md | 2 +- .../LineAsString/LineAsStringWithNames.md | 2 +- .../LineAsStringWithNamesAndTypes.md | 2 +- .../current/interfaces/formats/Markdown.md | 2 +- .../current/interfaces/formats/MsgPack.md | 2 +- .../current/interfaces/formats/MySQLDump.md | 2 +- .../current/interfaces/formats/Npy.md | 8 +- .../current/interfaces/formats/Null.md | 4 +- .../current/interfaces/formats/ORC.md | 6 +- .../current/interfaces/formats/One.md | 2 +- .../interfaces/formats/Parquet/Parquet.md | 6 +- .../formats/Parquet/ParquetMetadata.md | 2 +- .../interfaces/formats/Pretty/Pretty.md | 2 +- .../formats/Pretty/PrettyNoEscapes.md | 2 +- .../current/interfaces/formats/Prometheus.md | 15 +- .../interfaces/formats/Protobuf/Protobuf.md | 6 +- .../formats/Protobuf/ProtobufList.md | 2 +- .../current/interfaces/formats/RawBLOB.md | 6 +- .../current/interfaces/formats/Regexp.md | 2 +- .../RowBinary/RowBinaryWithDefaults.md | 2 +- .../current/interfaces/formats/SQLInsert.md | 2 +- .../interfaces/formats/TabSeparated/TSKV.md | 8 +- .../formats/TabSeparated/TabSeparated.md | 10 +- .../formats/TabSeparated/TabSeparatedRaw.md | 6 +- .../TabSeparated/TabSeparatedRawWithNames.md | 6 +- .../TabSeparatedRawWithNamesAndTypes.md | 6 +- .../TabSeparated/TabSeparatedWithNames.md | 6 +- .../TabSeparatedWithNamesAndTypes.md | 6 +- .../interfaces/formats/Template/Template.md | 16 +- .../formats/Template/TemplateIgnoreSpaces.md | 2 +- .../current/interfaces/formats/Vertical.md | 2 +- .../current/interfaces/formats/XML.md | 2 +- .../current/interfaces/grpc.md | 6 +- .../current/interfaces/http.md | 64 +- .../current/interfaces/jdbc.md | 2 +- .../current/interfaces/mysql.md | 10 +- .../native-clients-interfaces-index.md | 4 +- .../current/interfaces/odbc.md | 2 +- .../current/interfaces/overview.md | 30 +- .../current/interfaces/postgresql.md | 12 +- .../current/interfaces/prometheus.md | 17 +- .../current/interfaces/schema-inference.md | 92 +- .../current/interfaces/ssh.md | 16 +- .../current/interfaces/tcp.md | 2 +- .../third-party/client-libraries.md | 38 +- .../current/interfaces/third-party/gui.md | 2 +- .../current/interfaces/third-party/index.md | 10 +- .../interfaces/third-party/integrations.md | 2 +- .../current/interfaces/third-party/proxy.md | 2 +- .../current/intro.md | 2 +- .../core-concepts/academic_overview.mdx | 178 +- .../managing-data/core-concepts/merges.mdx | 36 +- .../core-concepts/partitions.mdx | 28 +- .../managing-data/core-concepts/parts.md | 4 +- .../core-concepts/primary-indexes.mdx | 43 +- .../managing-data/core-concepts/shards.mdx | 28 +- .../deleting-data/delete_mutations.mdx | 2 +- .../managing-data/deleting-data/overview.mdx | 14 +- .../current/managing-data/drop_partition.mdx | 5 +- .../current/managing-data/truncate.md | 2 +- .../managing-data/updating-data/overview.mdx | 21 +- .../updating-data/update_mutations.mdx | 2 +- .../incremental-materialized-view.md | 32 +- .../refreshable-materialized-view.md | 14 +- .../current/native-protocol/basics.md | 6 +- .../current/native-protocol/client.md | 2 +- .../current/native-protocol/columns.md | 6 +- .../current/native-protocol/hash.md | 2 +- .../current/native-protocol/server.md | 2 +- .../current/operations/_troubleshooting.md | 20 +- .../operations/allocation-profiling-old.md | 22 +- .../operations/allocation-profiling.md | 26 +- .../current/operations/analyzer.md | 37 +- .../current/operations/backup.md | 44 +- .../current/operations/caches.md | 34 +- .../current/operations/cluster-discovery.md | 23 +- .../current/operations/configuration-files.md | 16 +- .../external-authenticators/http.md | 11 +- .../external-authenticators/index.md | 2 +- .../external-authenticators/kerberos.md | 18 +- .../external-authenticators/ldap.md | 102 +- .../external-authenticators/ssl-x509.md | 2 +- .../current/operations/monitoring.md | 4 +- .../current/operations/named-collections.md | 96 +- .../current/operations/opentelemetry.md | 2 +- .../profile-guided-optimization.md | 7 +- .../sampling-query-profiler.md | 8 +- .../current/operations/performance-test.md | 5 +- .../current/operations/query-cache.md | 208 +- .../operations/query-condition-cache.md | 4 +- .../_server_settings_outside_source.md | 186 +- .../settings.md | 5143 +------ .../settings/composable-protocols.md | 16 +- .../settings/constraints-on-settings.md | 14 +- .../current/operations/settings/index.md | 42 +- .../operations/settings/memory-overcommit.md | 4 +- .../settings/merge-tree-settings.md | 3290 +---- .../current/operations/settings/overview.md | 4 +- .../settings/permissions-for-queries.md | 2 +- .../operations/settings/query-complexity.md | 2 +- .../operations/settings/server-overload.md | 4 +- .../operations/settings/settings-formats.md | 2799 +--- .../operations/settings/settings-profiles.md | 2 +- .../settings/settings-query-level.md | 32 +- .../operations/settings/settings-users.md | 25 +- .../current/operations/settings/settings.md | 11477 +--------------- .../settings/tcp-connection-limits.md | 2 +- .../current/operations/ssl-zookeeper.md | 4 +- .../current/operations/startup-scripts.md | 2 +- .../current/operations/storing-data.md | 48 +- .../system-tables/asynchronous_insert_log.md | 5 +- .../system-tables/asynchronous_inserts.md | 21 +- .../system-tables/asynchronous_loader.md | 78 +- .../system-tables/asynchronous_metric_log.md | 2 +- .../system-tables/asynchronous_metrics.md | 22 +- .../system-tables/azure_queue_settings.md | 9 - .../operations/system-tables/backup_log.md | 7 +- .../operations/system-tables/backups.md | 2 +- .../system-tables/blob_storage_log.md | 2 +- .../operations/system-tables/build_options.md | 9 +- .../operations/system-tables/clusters.md | 71 +- .../operations/system-tables/codecs.md | 13 +- .../operations/system-tables/columns.md | 35 +- .../operations/system-tables/contributors.md | 6 +- .../operations/system-tables/crash_log.md | 2 +- .../operations/system-tables/dashboards.md | 12 +- .../system-tables/data_skipping_indices.md | 17 +- .../system-tables/data_type_families.md | 6 +- .../system-tables/database_engines.md | 13 +- .../system-tables/database_replicas.md | 18 +- .../operations/system-tables/databases.md | 22 +- .../system-tables/delta_metadata_log.md | 6 +- .../system-tables/detached_tables.md | 12 +- .../operations/system-tables/dictionaries.md | 44 +- .../system-tables/dimensional_metrics.md | 4 +- .../current/operations/system-tables/disks.md | 22 +- .../system-tables/distributed_ddl_queue.md | 23 +- .../system-tables/distribution_queue.md | 20 +- .../operations/system-tables/dns_cache.md | 11 +- .../system-tables/dropped_tables.md | 14 +- .../operations/system-tables/error_log.md | 39 +- .../operations/system-tables/errors.md | 20 +- .../operations/system-tables/events.md | 19 +- .../operations/system-tables/functions.md | 17 +- .../system-tables/graphite_retentions.md | 15 +- .../system-tables/histogram_metrics.md | 30 +- .../system-tables/iceberg_history.md | 15 +- .../system-tables/iceberg_metadata_log.md | 8 +- .../current/operations/system-tables/index.md | 238 +- .../system-tables/information_schema.md | 10 +- .../operations/system-tables/jemalloc_bins.md | 16 +- .../system-tables/kafka_consumers.md | 28 +- .../operations/system-tables/licenses.md | 14 +- .../system-tables/merge_tree_settings.md | 34 +- .../operations/system-tables/merges.md | 36 +- .../operations/system-tables/metric_log.md | 5 +- .../operations/system-tables/metrics.md | 357 +- .../current/operations/system-tables/moves.md | 20 +- .../operations/system-tables/mutations.md | 7 +- .../operations/system-tables/numbers.md | 2 +- .../current/operations/system-tables/one.md | 2 +- .../system-tables/opentelemetry_span_log.md | 5 +- .../operations/system-tables/overview.md | 10 +- .../operations/system-tables/part_log.md | 9 +- .../current/operations/system-tables/parts.md | 3 +- .../operations/system-tables/parts_columns.md | 2 +- .../operations/system-tables/processes.md | 54 +- .../system-tables/processors_profile_log.md | 6 +- .../system-tables/projection_parts.md | 71 +- .../system-tables/projection_parts_columns.md | 61 +- .../operations/system-tables/projections.md | 17 +- .../operations/system-tables/query_cache.md | 21 +- .../system-tables/query_condition_cache.md | 20 +- .../operations/system-tables/query_log.md | 6 +- .../system-tables/query_metric_log.md | 5 +- .../system-tables/query_thread_log.md | 15 +- .../system-tables/query_views_log.md | 73 +- .../operations/system-tables/quota_limits.md | 30 +- .../operations/system-tables/quota_usage.md | 39 +- .../operations/system-tables/quotas.md | 2 +- .../operations/system-tables/quotas_usage.md | 37 +- .../operations/system-tables/replicas.md | 71 +- .../system-tables/replicated_fetches.md | 26 +- .../system-tables/replication_queue.md | 3 +- .../operations/system-tables/resources.md | 14 +- .../operations/system-tables/role_grants.md | 20 +- .../current/operations/system-tables/roles.md | 12 +- .../operations/system-tables/row_policies.md | 23 +- .../system-tables/s3_queue_settings.md | 19 +- .../operations/system-tables/scheduler.md | 42 +- .../system-tables/schema_inference_cache.md | 22 +- .../system-tables/server_settings.md | 6 +- .../operations/system-tables/session_log.md | 6 +- .../operations/system-tables/settings.md | 5 +- .../system-tables/settings_changes.md | 12 +- .../settings_profile_elements.md | 18 +- .../system-tables/settings_profiles.md | 20 +- .../operations/system-tables/stack_trace.md | 7 +- .../system-tables/storage_policies.md | 32 +- .../system-tables/system_warnings.md | 5 +- .../operations/system-tables/table_engines.md | 22 +- .../operations/system-tables/tables.md | 2 +- .../operations/system-tables/text_log.md | 5 +- .../operations/system-tables/time_zones.md | 13 +- .../operations/system-tables/trace_log.md | 6 +- .../operations/system-tables/unicode.md | 4 +- .../system-tables/user_processes.md | 16 +- .../current/operations/system-tables/users.md | 27 +- .../system-tables/view_refreshes.md | 27 +- .../operations/system-tables/workloads.md | 12 +- .../operations/system-tables/zookeeper.md | 2 +- .../system-tables/zookeeper_connection.md | 5 +- .../system-tables/zookeeper_connection_log.md | 6 +- .../operations/system-tables/zookeeper_log.md | 43 +- .../current/operations/tips.md | 82 +- .../current/operations/update.md | 4 +- .../operations/userspace-page-cache.md | 8 +- .../operations/utilities/backupview.md | 20 +- .../utilities/clickhouse-benchmark.md | 6 +- .../utilities/clickhouse-compressor.md | 2 +- .../operations/utilities/clickhouse-disks.md | 2 +- .../operations/utilities/clickhouse-format.md | 34 +- .../utilities/clickhouse-keeper-client.md | 4 +- .../operations/utilities/clickhouse-local.md | 14 +- .../operations/utilities/odbc-bridge.md | 2 +- .../current/operations/workload-scheduling.md | 22 +- .../operations_/backup_restore/00_overview.md | 14 +- .../backup_restore/01_local_disk.md | 28 +- .../backup_restore/02_s3_endpoint.md | 2 +- .../backup_restore/03_azure_blob_storage.md | 4 +- .../backup_restore/04_alternative_methods.md | 2 +- .../aggregate-functions/combinators.md | 12 +- .../aggregate-functions/grouping_function.md | 12 +- .../aggregate-functions/index.md | 13 +- .../parametric-functions.md | 42 +- .../aggregate-functions/reference/aggthrow.md | 2 +- .../reference/analysis_of_variance.md | 2 +- .../aggregate-functions/reference/any.md | 2 +- .../aggregate-functions/reference/anyheavy.md | 2 +- .../aggregate-functions/reference/anylast.md | 2 +- .../reference/approxtopk.md | 27 +- .../reference/approxtopsum.md | 18 +- .../reference/argandmax.md | 6 +- .../reference/argandmin.md | 3 +- .../aggregate-functions/reference/argmax.md | 2 +- .../aggregate-functions/reference/argmin.md | 3 +- .../aggregate-functions/reference/avg.md | 2 +- .../reference/avgweighted.md | 2 +- .../reference/contingency.md | 2 +- .../aggregate-functions/reference/corr.md | 2 +- .../reference/corrmatrix.md | 2 +- .../reference/corrstable.md | 2 +- .../aggregate-functions/reference/count.md | 2 +- .../aggregate-functions/reference/covarpop.md | 2 +- .../reference/covarpopmatrix.md | 2 +- .../reference/covarpopstable.md | 2 +- .../reference/covarsamp.md | 2 +- .../reference/covarsampmatrix.md | 2 +- .../reference/covarsampstable.md | 2 +- .../aggregate-functions/reference/cramersv.md | 2 +- .../reference/cramersvbiascorrected.md | 2 +- .../aggregate-functions/reference/deltasum.md | 2 +- .../reference/distinctdynamictypes.md | 2 +- .../reference/distinctjsonpaths.md | 5 +- .../aggregate-functions/reference/entropy.md | 2 +- .../reference/estimateCompressionRatio.md | 4 +- .../reference/exponentialmovingaverage.md | 5 +- .../reference/exponentialtimedecayedavg.md | 5 +- .../reference/exponentialtimedecayedcount.md | 5 +- .../reference/exponentialtimedecayedmax.md | 5 +- .../reference/exponentialtimedecayedsum.md | 5 +- .../reference/first_value.md | 16 +- .../reference/flame_graph.md | 14 +- .../reference/grouparray.md | 2 +- .../reference/grouparrayarray.md | 2 +- .../reference/grouparrayinsertat.md | 2 +- .../reference/grouparrayintersect.md | 2 +- .../reference/grouparraylast.md | 2 +- .../reference/grouparraymovingavg.md | 2 +- .../reference/grouparraymovingsum.md | 2 +- .../reference/grouparraysample.md | 2 +- .../reference/grouparraysorted.md | 2 +- .../reference/groupbitand.md | 2 +- .../reference/groupbitmap.md | 2 +- .../reference/groupbitmapor.md | 2 +- .../reference/groupbitmapxor.md | 2 +- .../reference/groupbitor.md | 2 +- .../reference/groupbitxor.md | 2 +- .../reference/groupuniqarray.md | 2 +- .../aggregate-functions/reference/index.md | 283 +- .../reference/kolmogorovsmirnovtest.md | 5 +- .../aggregate-functions/reference/kurtpop.md | 2 +- .../aggregate-functions/reference/kurtsamp.md | 2 +- .../reference/largestTriangleThreeBuckets.md | 2 +- .../reference/last_value.md | 16 +- .../reference/mannwhitneyutest.md | 2 +- .../reference/maxintersections.md | 2 +- .../reference/maxintersectionsposition.md | 2 +- .../aggregate-functions/reference/maxmap.md | 2 +- .../reference/meanztest.md | 2 +- .../aggregate-functions/reference/median.md | 2 +- .../aggregate-functions/reference/minmap.md | 2 +- .../aggregate-functions/reference/quantile.md | 2 +- .../reference/quantileGK.md | 2 +- .../reference/quantilebfloat16.md | 2 +- .../reference/quantiledeterministic.md | 2 +- .../reference/quantileexact.md | 20 +- .../reference/quantileexactweighted.md | 2 +- .../quantileexactweightedinterpolated.md | 2 +- .../reference/quantileinterpolatedweighted.md | 2 +- .../reference/quantileprometheushistogram.md | 2 +- .../reference/quantiles.md | 8 +- .../reference/quantiletdigest.md | 2 +- .../reference/quantiletdigestweighted.md | 2 +- .../reference/quantiletiming.md | 2 +- .../reference/quantiletimingweighted.md | 5 +- .../aggregate-functions/reference/rankCorr.md | 2 +- .../reference/simplelinearregression.md | 2 +- .../reference/singlevalueornull.md | 2 +- .../aggregate-functions/reference/skewpop.md | 2 +- .../aggregate-functions/reference/skewsamp.md | 2 +- .../aggregate-functions/reference/sparkbar.md | 2 +- .../reference/stddevpop.md | 2 +- .../reference/stddevpopstable.md | 2 +- .../reference/stddevsamp.md | 2 +- .../reference/stddevsampstable.md | 2 +- .../reference/stochasticlinearregression.md | 15 +- .../reference/stochasticlogisticregression.md | 6 +- .../reference/studentttest.md | 2 +- .../reference/studentttestonesample.md | 2 +- .../aggregate-functions/reference/sum.md | 2 +- .../aggregate-functions/reference/summap.md | 3 +- .../reference/summapwithoverflow.md | 3 +- .../reference/sumwithoverflow.md | 2 +- .../aggregate-functions/reference/theilsu.md | 2 +- .../reference/timeSeriesGroupArray.md | 2 +- .../aggregate-functions/reference/topk.md | 20 +- .../reference/topkweighted.md | 2 +- .../aggregate-functions/reference/uniq.md | 2 +- .../reference/uniqcombined.md | 2 +- .../reference/uniqcombined64.md | 2 +- .../reference/uniqexact.md | 2 +- .../reference/uniqhll12.md | 2 +- .../reference/varpopstable.md | 4 +- .../reference/welchttest.md | 2 +- .../data-types/aggregatefunction.md | 73 +- .../current/sql-reference/data-types/array.md | 17 +- .../sql-reference/data-types/boolean.md | 2 +- .../data-types/data-types-binary-encoding.md | 104 +- .../current/sql-reference/data-types/date.md | 2 +- .../sql-reference/data-types/date32.md | 2 +- .../sql-reference/data-types/datetime.md | 6 +- .../sql-reference/data-types/datetime64.md | 8 +- .../sql-reference/data-types/decimal.md | 4 +- .../sql-reference/data-types/domains/index.md | 2 +- .../sql-reference/data-types/dynamic.md | 49 +- .../current/sql-reference/data-types/enum.md | 15 +- .../sql-reference/data-types/fixedstring.md | 2 +- .../current/sql-reference/data-types/float.md | 6 +- .../current/sql-reference/data-types/geo.md | 14 +- .../current/sql-reference/data-types/index.md | 4 +- .../current/sql-reference/data-types/ipv4.md | 6 +- .../current/sql-reference/data-types/ipv6.md | 6 +- .../data-types/lowcardinality.md | 6 +- .../current/sql-reference/data-types/map.md | 6 +- .../nested-data-structures/index.md | 9 +- .../sql-reference/data-types/newjson.md | 38 +- .../sql-reference/data-types/nullable.md | 6 +- .../current/sql-reference/data-types/qbit.md | 4 +- .../data-types/simpleaggregatefunction.md | 6 +- .../special-data-types/expression.md | 2 +- .../data-types/special-data-types/index.md | 2 +- .../data-types/special-data-types/interval.md | 4 +- .../data-types/special-data-types/nothing.md | 2 +- .../data-types/special-data-types/set.md | 2 +- .../sql-reference/data-types/string.md | 2 +- .../current/sql-reference/data-types/time.md | 4 +- .../sql-reference/data-types/time64.md | 4 +- .../current/sql-reference/data-types/tuple.md | 17 +- .../current/sql-reference/data-types/uuid.md | 4 +- .../sql-reference/data-types/variant.md | 36 +- .../sql-reference/dictionaries/index.md | 147 +- .../functions/arithmetic-functions.md | 1315 +- .../functions/array-functions.md | 5024 +------ .../sql-reference/functions/array-join.md | 7 +- .../sql-reference/functions/bit-functions.md | 691 +- .../functions/bitmap-functions.md | 718 +- .../functions/comparison-functions.md | 330 +- .../functions/conditional-functions.md | 402 +- .../functions/date-time-functions.md | 5655 +------- .../functions/distance-functions.md | 621 +- .../functions/embedded-dict-functions.md | 34 +- .../functions/encoding-functions.md | 1101 +- .../functions/encryption-functions.md | 381 +- .../functions/ext-dict-functions.md | 2343 +--- .../current/sql-reference/functions/files.md | 4 +- .../functions/financial-functions.md | 246 +- .../functions/functions-for-nulls.md | 467 +- .../functions/geo/coordinates.md | 16 +- .../functions/geo/flipCoordinates.md | 18 +- .../sql-reference/functions/geo/geohash.md | 6 +- .../current/sql-reference/functions/geo/h3.md | 88 +- .../sql-reference/functions/geo/polygon.md | 240 +- .../current/sql-reference/functions/geo/s2.md | 22 +- .../sql-reference/functions/geo/svg.md | 4 +- .../sql-reference/functions/hash-functions.md | 2640 +--- .../sql-reference/functions/in-functions.md | 2 +- .../sql-reference/functions/introspection.md | 257 +- .../functions/ip-address-functions.md | 1120 +- .../sql-reference/functions/json-functions.md | 2024 +-- .../functions/logical-functions.md | 199 +- .../functions/machine-learning-functions.md | 8 +- .../sql-reference/functions/math-functions.md | 1289 +- .../sql-reference/functions/nlp-functions.md | 331 +- .../numeric-indexed-vector-functions.md | 680 +- .../functions/other-functions.md | 5508 +------- .../sql-reference/functions/overview.md | 6 +- .../functions/random-functions.md | 811 +- .../functions/rounding-functions.md | 465 +- .../functions/splitting-merging-functions.md | 462 +- .../functions/string-functions.md | 3603 +---- .../functions/string-replace-functions.md | 506 +- .../functions/string-search-functions.md | 2701 +--- .../functions/time-series-functions.md | 412 +- .../functions/time-window-functions.md | 229 +- .../functions/tuple-functions.md | 831 +- .../functions/tuple-map-functions.md | 77 +- .../functions/type-conversion-functions.md | 308 +- .../current/sql-reference/functions/udf.md | 12 +- .../sql-reference/functions/ulid-functions.md | 94 +- .../functions/uniqtheta-functions.md | 14 +- .../sql-reference/functions/url-functions.md | 1401 +- .../sql-reference/functions/uuid-functions.md | 1145 +- .../current/sql-reference/index.md | 3 +- .../current/sql-reference/operators/exists.md | 2 +- .../current/sql-reference/operators/in.md | 15 +- .../current/sql-reference/operators/index.md | 19 +- .../sql-reference/sql-reference-links.json | 4 +- .../statements/alter/apply-deleted-mask.md | 4 +- .../sql-reference/statements/alter/column.md | 22 +- .../sql-reference/statements/alter/comment.md | 6 +- .../statements/alter/constraint.md | 2 +- .../statements/alter/database-comment.md | 6 +- .../sql-reference/statements/alter/delete.md | 2 +- .../sql-reference/statements/alter/index.md | 2 +- .../statements/alter/named-collection.md | 3 +- .../statements/alter/order-by.md | 2 +- .../statements/alter/partition.md | 64 +- .../statements/alter/projection.md | 10 +- .../statements/alter/row-policy.md | 2 +- .../statements/alter/sample-by.md | 11 +- .../sql-reference/statements/alter/setting.md | 10 +- .../statements/alter/settings-profile.md | 2 +- .../statements/alter/skipping-index.md | 2 +- .../statements/alter/statistics.md | 20 +- .../sql-reference/statements/alter/ttl.md | 11 +- .../sql-reference/statements/alter/update.md | 2 +- .../sql-reference/statements/alter/user.md | 42 +- .../sql-reference/statements/alter/view.md | 2 +- .../sql-reference/statements/attach.md | 19 +- .../sql-reference/statements/check-grant.md | 4 +- .../sql-reference/statements/check-table.md | 6 +- .../statements/create/database.md | 15 +- .../statements/create/dictionary.md | 15 +- .../sql-reference/statements/create/index.md | 2 +- .../statements/create/named-collection.md | 3 +- .../sql-reference/statements/create/quota.md | 18 +- .../sql-reference/statements/create/role.md | 2 +- .../statements/create/row-policy.md | 2 +- .../statements/create/settings-profile.md | 11 +- .../sql-reference/statements/create/table.md | 62 +- .../sql-reference/statements/create/user.md | 106 +- .../sql-reference/statements/create/view.md | 52 +- .../sql-reference/statements/delete.md | 4 +- .../current/sql-reference/statements/drop.md | 38 +- .../sql-reference/statements/exchange.md | 86 +- .../sql-reference/statements/execute_as.md | 11 +- .../sql-reference/statements/exists.md | 2 +- .../sql-reference/statements/explain.md | 168 +- .../current/sql-reference/statements/grant.md | 38 +- .../current/sql-reference/statements/index.md | 2 +- .../sql-reference/statements/insert-into.md | 14 +- .../current/sql-reference/statements/kill.md | 7 +- .../current/sql-reference/statements/move.md | 2 +- .../sql-reference/statements/optimize.md | 15 +- .../sql-reference/statements/parallel_with.md | 6 +- .../sql-reference/statements/rename.md | 13 +- .../sql-reference/statements/revoke.md | 6 +- .../sql-reference/statements/select/all.md | 2 +- .../statements/select/apply_modifier.md | 4 +- .../statements/select/array-join.md | 14 +- .../statements/select/distinct.md | 4 +- .../sql-reference/statements/select/except.md | 29 +- .../statements/select/except_modifier.md | 4 +- .../sql-reference/statements/select/format.md | 2 +- .../sql-reference/statements/select/from.md | 8 +- .../statements/select/group-by.md | 14 +- .../sql-reference/statements/select/having.md | 11 +- .../sql-reference/statements/select/index.md | 9 +- .../statements/select/intersect.md | 12 +- .../statements/select/into-outfile.md | 7 +- .../sql-reference/statements/select/join.md | 16 +- .../statements/select/limit-by.md | 22 +- .../sql-reference/statements/select/limit.md | 20 +- .../sql-reference/statements/select/offset.md | 3 +- .../statements/select/order-by.md | 12 +- .../statements/select/prewhere.md | 4 +- .../statements/select/qualify.md | 4 +- .../statements/select/replace_modifier.md | 4 +- .../sql-reference/statements/select/sample.md | 28 +- .../sql-reference/statements/select/union.md | 2 +- .../sql-reference/statements/select/where.md | 26 +- .../sql-reference/statements/select/with.md | 30 +- .../sql-reference/statements/set-role.md | 6 +- .../current/sql-reference/statements/set.md | 2 +- .../current/sql-reference/statements/show.md | 134 +- .../sql-reference/statements/system.md | 32 +- .../sql-reference/statements/truncate.md | 14 +- .../sql-reference/statements/undrop.md | 2 +- .../sql-reference/statements/update.md | 4 +- .../current/sql-reference/statements/use.md | 2 +- .../current/sql-reference/statements/watch.md | 5 +- .../current/sql-reference/syntax.md | 417 +- .../table-functions/arrowflight.md | 2 +- .../table-functions/azureBlobStorage.md | 14 +- .../azureBlobStorageCluster.md | 6 +- .../sql-reference/table-functions/cluster.md | 6 +- .../table-functions/deltalake.md | 6 +- .../table-functions/deltalakeCluster.md | 4 +- .../table-functions/dictionary.md | 6 +- .../table-functions/executable.md | 6 +- .../sql-reference/table-functions/file.md | 28 +- .../table-functions/fileCluster.md | 6 +- .../sql-reference/table-functions/format.md | 6 +- .../sql-reference/table-functions/fuzzJSON.md | 6 +- .../table-functions/fuzzQuery.md | 6 +- .../sql-reference/table-functions/gcs.md | 12 +- .../sql-reference/table-functions/generate.md | 6 +- .../table-functions/generate_series.md | 11 +- .../sql-reference/table-functions/hdfs.md | 10 +- .../table-functions/hdfsCluster.md | 6 +- .../sql-reference/table-functions/hudi.md | 4 +- .../table-functions/hudiCluster.md | 4 +- .../sql-reference/table-functions/iceberg.md | 50 +- .../table-functions/icebergCluster.md | 6 +- .../sql-reference/table-functions/index.md | 143 +- .../sql-reference/table-functions/input.md | 8 +- .../sql-reference/table-functions/jdbc.md | 13 +- .../sql-reference/table-functions/loop.md | 6 +- .../sql-reference/table-functions/merge.md | 4 +- .../table-functions/mergeTreeIndex.md | 6 +- .../table-functions/mergeTreeProjection.md | 6 +- .../sql-reference/table-functions/mongodb.md | 8 +- .../sql-reference/table-functions/mysql.md | 8 +- .../sql-reference/table-functions/null.md | 6 +- .../sql-reference/table-functions/numbers.md | 2 +- .../sql-reference/table-functions/odbc.md | 6 +- .../sql-reference/table-functions/paimon.md | 4 +- .../table-functions/paimonCluster.md | 4 +- .../table-functions/postgresql.md | 8 +- .../table-functions/prometheusQuery.md | 6 +- .../table-functions/prometheusQueryRange.md | 6 +- .../sql-reference/table-functions/redis.md | 6 +- .../sql-reference/table-functions/remote.md | 22 +- .../sql-reference/table-functions/s3.md | 24 +- .../table-functions/s3Cluster.md | 6 +- .../sql-reference/table-functions/sqlite.md | 6 +- .../table-functions/timeSeriesData.md | 2 +- .../table-functions/timeSeriesMetrics.md | 2 +- .../table-functions/timeSeriesSelector.md | 6 +- .../table-functions/timeSeriesTags.md | 2 +- .../sql-reference/table-functions/url.md | 8 +- .../table-functions/urlCluster.md | 6 +- .../sql-reference/table-functions/values.md | 4 +- .../sql-reference/table-functions/view.md | 6 +- .../sql-reference/table-functions/ytsaurus.md | 4 +- .../sql-reference/table-functions/zeros.md | 2 +- .../current/sql-reference/transactions.md | 28 +- .../window-functions/cume_dist.md | 2 +- .../window-functions/dense_rank.md | 2 +- .../window-functions/first_value.md | 2 +- .../sql-reference/window-functions/index.md | 28 +- .../sql-reference/window-functions/lag.md | 2 +- .../window-functions/lagInFrame.md | 2 +- .../window-functions/last_value.md | 2 +- .../sql-reference/window-functions/lead.md | 2 +- .../window-functions/leadInFrame.md | 2 +- .../window-functions/nth_value.md | 2 +- .../window-functions/percent_rank.md | 2 +- .../sql-reference/window-functions/rank.md | 2 +- .../window-functions/row_number.md | 2 +- .../tips-and-tricks/debugging-insights.md | 32 +- .../tips-and-tricks/materialized-views.md | 2 +- .../performance-optimization.md | 6 +- .../current/tips-and-tricks/too-many-parts.md | 2 +- .../static-files-disk-uploader.md | 10 +- .../current/tutorial.md | 4 +- .../current/use-cases/AI_ML/AIChat/index.md | 2 +- .../use-cases/AI_ML/MCP/01_remote_mcp.md | 6 +- .../use-cases/AI_ML/MCP/02_claude-desktop.md | 4 +- .../use-cases/AI_ML/MCP/03_librechat.md | 26 +- .../use-cases/AI_ML/MCP/04_anythingllm.md | 10 +- .../use-cases/AI_ML/MCP/05_open-webui.md | 6 +- .../current/use-cases/AI_ML/MCP/06_ollama.md | 8 +- .../current/use-cases/AI_ML/MCP/07_janai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/agno.md | 6 +- .../AI_ML/MCP/ai_agent_libraries/chainlit.md | 6 +- .../ai_agent_libraries/claude-agent-sdk.md | 6 +- .../MCP/ai_agent_libraries/copilotkit.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/crewai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/dspy.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/index.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/langchain.md | 2 +- .../MCP/ai_agent_libraries/llamaindex.md | 10 +- .../AI_ML/MCP/ai_agent_libraries/mcp-agent.md | 2 +- .../microsoft-agent-framework.md | 2 +- .../MCP/ai_agent_libraries/openai-agents.md | 8 +- .../MCP/ai_agent_libraries/pydantic-ai.md | 8 +- .../AI_ML/MCP/ai_agent_libraries/slackbot.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/streamlit.md | 16 +- .../AI_ML/MCP/ai_agent_libraries/upsonic.md | 2 +- .../AI_ML/ai-powered-sql-generation.md | 16 +- .../data-exploration/jupyter-notebook.md | 26 +- .../AI_ML/data-exploration/marimo-notebook.md | 226 +- .../current/use-cases/AI_ML/index.md | 2 +- .../use-cases/data_lake/glue_catalog.md | 4 +- .../use-cases/data_lake/lakekeeper_catalog.md | 6 +- .../use-cases/data_lake/nessie_catalog.md | 8 +- .../use-cases/data_lake/onelake_catalog.md | 8 +- .../use-cases/data_lake/rest_catalog.md | 8 +- .../use-cases/data_lake/unity_catalog.md | 8 +- .../observability/build-your-own/grafana.md | 16 +- .../observability/build-your-own/index.md | 2 +- .../integrating-opentelemetry.md | 18 +- .../build-your-own/introduction.md | 8 +- .../build-your-own/managing-data.md | 16 +- .../build-your-own/schema-design.md | 38 +- .../observability/clickstack/api-reference.md | 15 + .../observability/clickstack/config.md | 6 +- .../observability/clickstack/dashboards.md | 2 +- .../clickstack/deployment/all-in-one.md | 14 +- .../clickstack/deployment/docker-compose.md | 14 +- .../clickstack/deployment/helm/helm-cloud.md | 28 +- .../deployment/helm/helm-configuration.md | 28 +- .../helm/helm-deployment-options.md | 24 +- .../clickstack/deployment/helm/helm.md | 2 +- .../deployment/hyperdx-clickhouse-cloud.md | 2 +- .../clickstack/deployment/hyperdx-only.md | 4 +- .../clickstack/deployment/local-mode-only.md | 4 +- .../clickstack/example-datasets/kubernetes.md | 2 +- .../clickstack/example-datasets/local-data.md | 11 +- .../example-datasets/remote-demo-data.md | 2 +- .../clickstack/getting-started.md | 2 +- .../clickstack/ingesting-data/collector.md | 16 +- .../ingesting-data/opentelemetry.md | 4 +- .../clickstack/ingesting-data/schemas.md | 16 +- .../ingesting-data/sdks/aws-lambda.md | 8 +- .../clickstack/ingesting-data/sdks/browser.md | 10 +- .../clickstack/ingesting-data/sdks/deno.md | 7 +- .../clickstack/ingesting-data/sdks/elixir.md | 8 +- .../clickstack/ingesting-data/sdks/golang.md | 8 +- .../clickstack/ingesting-data/sdks/index.md | 2 +- .../clickstack/ingesting-data/sdks/java.md | 6 +- .../clickstack/ingesting-data/sdks/nestjs.md | 7 +- .../clickstack/ingesting-data/sdks/nextjs.md | 6 +- .../clickstack/ingesting-data/sdks/nodejs.md | 8 +- .../clickstack/ingesting-data/sdks/python.md | 12 +- .../ingesting-data/sdks/react-native.md | 10 +- .../clickstack/ingesting-data/sdks/ruby.md | 8 +- .../integration-examples/kafka-metrics.md | 8 +- .../integration-examples/kubernetes.md | 8 +- .../integration-examples/nginx-traces.md | 16 +- .../integration-examples/postgres-metrics.md | 14 +- .../integration-examples/redis-logs.md | 6 +- .../integration-examples/redis-metrics.md | 13 +- .../migration/elastic/migrating-sdks.md | 3 +- .../observability/clickstack/production.md | 6 +- .../observability/clickstack/search.md | 2 +- .../use-cases/observability/clickstack/ttl.md | 11 +- .../observability/cloud-monitoring.md | 6 +- .../observability/self-managed-monitoring.md | 6 +- .../time-series/01_date-time-data-types.md | 10 +- .../time-series/02_basic-operations.md | 17 +- .../time-series/03_analysis-functions.md | 21 +- .../time-series/04_storage-efficiency.md | 11 +- .../time-series/05_query-performance.md | 16 +- .../06_materialized-view-rollup.mdx | 25 +- .../current/whats-new/changelog/2020.md | 30 +- .../current/whats-new/changelog/2022.md | 40 +- .../current/whats-new/changelog/2023.md | 84 +- .../current/whats-new/changelog/2024.md | 82 +- .../current/whats-new/changelog/cloud.md | 4 +- .../current/whats-new/changelog/index.md | 3893 +++--- .../current/whats-new/security-changelog.md | 2 +- i18n/zh/code.json | 1005 +- .../current/Insert_select_settings_tuning.mdx | 15 +- ...ailed-error-using-PowerBI-CH-connector.mdx | 8 +- .../about-quotas-and-query-complexity.mdx | 10 +- .../current/add-column.mdx | 12 +- .../current/alter-user-settings-exception.mdx | 6 +- ...rialized_views_inserted_asynchronously.mdx | 1 - .../async_vs_optimize_read_in_order.mdx | 16 +- .../aws-privatelink-setup-for-clickpipes.mdx | 51 +- ...s-privatelink-setup-for-msk-clickpipes.mdx | 59 +- .../backing_up_a_specific_partition.mdx | 4 +- .../current/calculate-pi-using-sql.mdx | 3 +- ...ate_ratio_of_zero_sparse_serialization.mdx | 3 +- .../cannot-append-data-to-parquet-format.mdx | 3 +- .../certificate_verify_failed_error.mdx | 10 +- .../current/change-billing-email.mdx | 3 +- ...change-the-prompt-in-clickhouse-client.mdx | 9 +- .../current/check-query-cache-in-use.mdx | 7 +- .../check_query_processing_time_only.mdx | 3 +- .../current/check_users_roles.mdx | 1 - .../current/clickhouse_cloud_api_usage.mdx | 3 +- .../current/columnar-database.mdx | 7 +- .../current/common-rbac-queries.mdx | 12 +- .../current/compare_resultsets.mdx | 3 +- .../comparing-metrics-between-queries.mdx | 4 +- .../current/configure-a-user-setting.mdx | 9 +- ...ap_ipc_lock_and_cap_sys_nice_in_docker.mdx | 6 +- ...connection_timeout_remote_remoteSecure.mdx | 3 +- .../current/custom-dns-alias-for-instance.mdx | 12 +- .../current/dbms-naming.mdx | 7 +- .../current/delete-old-data.mdx | 4 +- .../current/dictionaries-consistent-state.mdx | 4 +- .../current/dictionary_using_strings.mdx | 3 +- ..._between_official_builds_and_3rd_party.mdx | 19 +- .../enabling-ssl-with-lets-encrypt.mdx | 5 +- .../current/exception-too-many-parts.mdx | 3 +- .../exchangeStatementToSwitchTables.mdx | 3 +- .../execute-system-queries-in-cloud.mdx | 3 +- .../current/file-export.mdx | 9 +- .../current/filtered-aggregates.mdx | 3 +- .../current/find-expensive-queries.mdx | 9 +- ...ding_expensive_queries_by_memory_usage.mdx | 3 +- ...-developer-verification-error-in-macos.mdx | 17 +- .../current/generate-har-file.mdx | 5 +- ...how-do-i-contribute-code-to-clickhouse.mdx | 1 - ...check-my-clickhouse-cloud-sevice-state.mdx | 9 +- ...-to-connect-to-ch-cloud-using-ssh-keys.mdx | 5 +- ...able-to-query-multiple-remote-clusters.mdx | 3 +- ...-a-clickhouse-table-by-an-array-column.mdx | 5 +- .../how-to-increase-thread-pool-size.mdx | 3 +- ...-to-insert-all-rows-from-another-table.mdx | 3 +- ...set-up-ch-on-docker-odbc-connect-mssql.mdx | 27 +- .../how_to_display_queries_using_MV.mdx | 3 +- .../current/how_to_use_parametrised_views.mdx | 3 +- .../current/ignoring-incorrect-settings.mdx | 3 +- ...ting-geojason-with-nested-object-array.mdx | 8 +- ...ng_and_working_with_JSON_array_objects.mdx | 7 +- .../current/improve-map-performance.mdx | 4 +- .../current/ingest-failures-23-9-release.mdx | 3 +- .../current/ingest-parquet-files-in-s3.mdx | 4 +- .../current/install-clickhouse-windows10.mdx | 4 +- .../current/json-import.mdx | 10 +- .../current/json_extract_example.mdx | 3 +- .../current/json_simple_example.mdx | 5 +- .../current/kafka-clickhouse-json.mdx | 13 +- .../current/kafka-to-clickhouse-setup.mdx | 11 +- .../current/key-value.mdx | 7 +- .../current/llvm-clang-up-to-date.mdx | 3 +- ...f-system-metrics-to-prometheus-metrics.mdx | 101 +- .../current/mapreduce.mdx | 3 +- ...maximum_number_of_tables_and_databases.mdx | 5 +- .../memory-limit-exceeded-for-query.mdx | 3 +- .../current/multi-region-replication.mdx | 1 - .../current/mysql-to-parquet-csv-json.mdx | 12 +- .../current/node-js-example.mdx | 3 +- .../current/olap.mdx | 7 +- .../current/optimize_final_vs_final.mdx | 19 +- .../current/oracle-odbc.mdx | 3 +- .../outputSendLogsLevelTracesToFile.mdx | 6 +- .../current/parquet-to-csv-json.mdx | 15 +- .../current/part_intersects_previous_part.mdx | 17 +- .../current/pivot.mdx | 23 +- .../postgresql-to-parquet-csv-json.mdx | 15 +- .../current/production.mdx | 47 +- .../profiling-clickhouse-with-llvm-xray.mdx | 17 +- .../current/projection_example.mdx | 3 +- .../python-clickhouse-connect-example.mdx | 3 +- .../current/python_http_requests.mdx | 5 +- .../current/query_max_execution_time.mdx | 3 +- .../current/read_consistency.mdx | 3 +- .../recreate_table_across_terminals.mdx | 3 +- .../current/remove-default-user.mdx | 9 +- .../restore-replica-after-storage-failure.mdx | 10 +- .../current/row-column-policy.mdx | 3 +- .../s3_export_data_year_month_folders.mdx | 23 +- .../current/schema_migration_tools.mdx | 19 +- ...across-node-for-tables-with-a-wildcard.mdx | 1 - .../current/send_logs_level.mdx | 3 +- .../current/terraform_example.mdx | 4 +- .../current/time-series.mdx | 5 +- ...imizing-basic-data-types-in-clickhouse.mdx | 16 +- .../unable-to-access-cloud-service.mdx | 3 +- .../use-clickhouse-for-log-analytics.mdx | 1 - .../useful-queries-for-troubleshooting.mdx | 36 +- ...y-join-to-extract-and-query-attributes.mdx | 5 +- .../current/vector-search.mdx | 19 +- .../view_number_of_active_mutations.mdx | 3 +- .../current/when_is_ttl_applied.mdx | 9 +- .../which-processes-are-currently-running.mdx | 4 +- .../current/who-is-using-clickhouse.mdx | 3 +- .../current/why_default_logging_verbose.mdx | 3 +- .../why_is_my_primary_key_not_used.mdx | 16 +- ...mmend_clickhouse_keeper_over_zookeeper.mdx | 17 +- .../windows_active_directory_to_ch_roles.mdx | 6 +- .../_GCS_authentication_and_bucket.md | 10 +- .../current/_snippets/_add_superset_detail.md | 4 +- .../_clickhouse_mysql_cloud_setup.mdx | 69 +- .../current/_snippets/_clickstack_tagging.mdx | 9 +- ...irect_observability_integration_options.md | 6 +- .../_snippets/_users-and-roles-common.md | 24 +- .../current/about-us/cloud.md | 2 +- .../current/about-us/distinctive-features.md | 2 +- .../current/about-us/index.md | 2 +- .../current/about-us/support.md | 10 +- .../_snippets/_async_inserts.md | 4 +- .../best-practices/avoid_optimize_final.md | 2 +- .../best-practices/choosing_a_primary_key.md | 4 +- .../current/best-practices/index.md | 3 +- .../current/best-practices/json_type.md | 2 +- .../best-practices/partitioning_keys.mdx | 3 +- .../best-practices/select_data_type.md | 7 +- .../selecting_an_insert_strategy.md | 13 +- .../sizing-and-hardware-recommendations.md | 2 +- .../using_data_skipping_indices.md | 2 +- .../current/chdb/getting-started.md | 14 +- .../current/chdb/guides/clickhouse-local.md | 13 +- .../current/chdb/guides/jupysql.md | 26 +- .../chdb/guides/query-remote-clickhouse.md | 8 +- .../chdb/guides/querying-apache-arrow.md | 10 +- .../current/chdb/guides/querying-pandas.md | 19 +- .../current/chdb/guides/querying-parquet.md | 11 +- .../current/chdb/guides/querying-s3-bucket.md | 14 +- .../current/chdb/index.md | 2 +- .../current/chdb/install/bun.md | 23 +- .../current/chdb/install/c.md | 26 +- .../current/chdb/install/go.md | 24 +- .../current/chdb/install/nodejs.md | 14 +- .../current/chdb/install/python.md | 180 +- .../current/chdb/install/rust.md | 20 +- .../current/cloud/features/01_cloud_tiers.md | 2 +- .../03_sql_console_features/01_sql-console.md | 362 +- .../02_query-insights.md | 2 +- .../03_query-endpoints.md | 3 +- .../03_sql_console_features/04_dashboards.md | 2 +- .../automatic_scaling/01_auto_scaling.md | 4 +- .../04_infrastructure/deployment-options.md | 2 +- .../replica-aware-routing.md | 2 +- .../04_infrastructure/shared-catalog.md | 2 +- .../04_infrastructure/shared-merge-tree.md | 4 +- .../features/04_infrastructure/warehouses.md | 4 +- .../05_admin_features/api/api-overview.md | 2 +- .../features/05_admin_features/api/openapi.md | 2 +- .../features/05_admin_features/api/postman.md | 14 +- .../features/05_admin_features/upgrades.md | 2 +- .../current/cloud/features/06_security.md | 2 +- .../07_monitoring/advanced_dashboard.md | 8 +- .../features/07_monitoring/prometheus.md | 72 +- .../guides/SQL_console/connection_details.md | 2 +- .../guides/SQL_console/query-endpoints.md | 32 +- .../backups/01_review-and-restore-backups.md | 10 +- .../01_export-backups-to-own-cloud-account.md | 35 +- .../cloud/guides/best_practices/index.md | 2 +- .../guides/best_practices/multitenancy.md | 8 +- .../cloud/guides/cloud-compatibility.md | 2 +- .../data_sources/01_cloud-endpoints-api.md | 3 +- .../02_accessing-s3-data-securely.md | 12 +- .../byoc/03_onboarding/01_aws.md | 14 +- .../byoc/05_observability/01_aws.md | 14 +- .../cloud/guides/production-readiness.md | 12 +- .../03_manage-sql-console-role-assignments.md | 2 +- .../04_manage-database-users.md | 14 +- .../04_saml-sso-setup.md | 2 +- .../05_common-access-management-queries.md | 14 +- .../01_cloud_access_management/index.md | 2 +- .../02_connectivity/01_setting-ip-filters.md | 2 +- .../private_networking/02_aws-privatelink.md | 48 +- .../03_gcp-private-service-connect.md | 52 +- .../04_azure-privatelink.md | 60 +- .../private_networking/index.md | 8 +- .../cloud/guides/security/03_data-masking.md | 20 +- .../current/cloud/guides/security/04_cmek.md | 2 +- .../05_audit_logging/02_database-audit-log.md | 2 +- .../03_byoc-security-playbook.md | 4 +- .../guides/security/05_audit_logging/index.md | 4 +- .../cloud/onboard/01_discover/01_what_is.md | 58 +- .../02_use_cases/01_real-time-analytics.md | 2 +- .../01_migration_guides/01_overview.md | 25 +- .../02_postgres/01_overview.md | 2 +- .../02_postgres/appendix.md | 2 +- .../01_migration_guide_part1.md | 20 +- .../02_migration_guide_part2.md | 2 +- .../03_migration_guide_part3.md | 6 +- .../03_bigquery/01_overview.md | 4 +- .../02_migrating-to-clickhouse-cloud.md | 18 +- .../03_bigquery/03_loading-data.md | 4 +- .../04_snowflake/01_overview.md | 2 +- .../04_snowflake/02_migration_guide.md | 6 +- .../03_sql_translation_reference.md | 2 +- .../05_elastic/01_overview.md | 2 +- .../06_redshift/01_overview.md | 2 +- .../06_redshift/02_migration_guide.md | 7 +- .../03_sql_translation_reference.md | 6 +- .../07_OSS_to_Cloud/01_clickhouse-to-cloud.md | 34 +- .../01_clickhouse-local-etl.md | 22 +- .../02_etl-tool-to-clickhouse.md | 11 +- .../03_object-storage-to-clickhouse.md | 11 +- .../cloud/onboard/03_tune/resource_tour.md | 8 +- .../current/cloud/onboard/index.md | 2 +- .../01_changelog/02_release_notes/24_02.md | 8 +- .../01_changelog/02_release_notes/24_05.md | 4 +- .../01_changelog/02_release_notes/24_06.md | 4 +- .../01_changelog/02_release_notes/24_10.md | 2 +- .../cloud/reference/02_architecture.md | 2 +- .../03_billing/01_billing_overview.md | 22 +- .../02_marketplace/aws-marketplace-payg.md | 55 +- .../03_clickpipes/clickpipes_for_cdc.md | 10 +- .../03_billing/04_network-data-transfer.mdx | 2 +- .../03_billing/05_payment-thresholds.md | 4 +- .../03_billing/06_billing_compliance.md | 2 +- .../cloud/reference/05_supported-regions.md | 2 +- .../current/cloud/reference/08_settings.md | 5 +- .../09_security/03_compliance-overview.md | 2 +- .../cloud/reference/11_account-close.md | 2 +- .../current/cloud/reference/index.md | 2 +- .../current/concepts/glossary.md | 2 +- .../current/concepts/olap.md | 2 +- .../concepts/why-clickhouse-is-so-fast.mdx | 26 +- .../compression-in-clickhouse.md | 4 +- .../data-compression/compression-modes.md | 2 +- .../current/data-modeling/backfilling.md | 18 +- .../current/data-modeling/denormalization.md | 14 +- .../current/data-modeling/index.md | 2 +- .../projections/1_projections.md | 275 +- ...2_materialized-views-versus-projections.md | 96 +- .../current/data-modeling/schema-design.md | 15 +- .../current/deployment-guides/index.md | 2 +- .../deployment-guides/parallel-replicas.mdx | 242 +- .../01_1_shard_2_replicas.md | 25 +- .../02_2_shards_1_replica.md | 25 +- .../03_2_shards_2_replicas.md | 17 +- .../current/deployment-guides/terminology.md | 2 +- .../current/development/architecture.md | 4 +- .../current/development/build-cross-arm.md | 2 +- .../development/build-cross-loongarch.md | 8 +- .../current/development/build-cross-osx.md | 11 +- .../current/development/build-cross-riscv.md | 8 +- .../current/development/build-cross-s390x.md | 92 +- .../current/development/build-e2k.md | 8 +- .../current/development/build-osx.md | 8 +- .../current/development/build.md | 118 +- .../building_and_benchmarking_deflate_qpl.md | 18 +- .../development/continuous-integration.md | 24 +- .../current/development/contrib.md | 2 +- .../development/developer-instruction.md | 172 +- .../current/development/index.md | 20 +- .../development/integrating_rust_libraries.md | 3 +- .../current/development/style.md | 14 +- .../current/development/tests.md | 34 +- .../current/dictionary/index.md | 142 +- .../engines/database-engines/atomic.md | 18 +- .../engines/database-engines/backup.md | 11 +- .../engines/database-engines/datalake.md | 6 +- .../current/engines/database-engines/index.md | 38 +- .../current/engines/database-engines/lazy.md | 12 +- .../materialized-postgresql.md | 30 +- .../current/engines/database-engines/mysql.md | 15 +- .../engines/database-engines/postgresql.md | 6 +- .../engines/database-engines/replicated.md | 8 +- .../engines/database-engines/shared.md | 4 +- .../engines/database-engines/sqlite.md | 6 +- .../current/engines/table-engines/index.md | 2 +- .../integrations/ExternalDistributed.md | 11 +- .../table-engines/integrations/arrowflight.md | 6 +- .../table-engines/integrations/azure-queue.md | 106 +- .../integrations/azureBlobStorage.md | 14 +- .../table-engines/integrations/deltalake.md | 6 +- .../integrations/embedded-rocksdb.md | 36 +- .../table-engines/integrations/hdfs.md | 16 +- .../table-engines/integrations/hive.md | 36 +- .../table-engines/integrations/hudi.md | 4 +- .../table-engines/integrations/iceberg.md | 26 +- .../table-engines/integrations/index.md | 67 +- .../table-engines/integrations/jdbc.md | 6 +- .../table-engines/integrations/kafka.md | 12 +- .../integrations/materialized-postgresql.md | 6 +- .../table-engines/integrations/mongodb.md | 12 +- .../table-engines/integrations/mysql.md | 6 +- .../table-engines/integrations/nats.md | 6 +- .../table-engines/integrations/odbc.md | 6 +- .../table-engines/integrations/postgresql.md | 20 +- .../table-engines/integrations/rabbitmq.md | 6 +- .../table-engines/integrations/redis.md | 6 +- .../engines/table-engines/integrations/s3.md | 36 +- .../table-engines/integrations/s3queue.md | 354 +- .../table-engines/integrations/sqlite.md | 12 +- .../table-engines/integrations/time-series.md | 18 +- .../table-engines/integrations/ytsaurus.md | 6 +- .../engines/table-engines/log-family/index.md | 2 +- .../engines/table-engines/log-family/log.md | 6 +- .../table-engines/log-family/stripelog.md | 6 +- .../table-engines/log-family/tinylog.md | 6 +- .../mergetree-family/aggregatingmergetree.md | 6 +- .../mergetree-family/annindexes.md | 28 +- .../mergetree-family/coalescingmergetree.md | 17 +- .../mergetree-family/collapsingmergetree.md | 16 +- .../custom-partitioning-key.md | 10 +- .../mergetree-family/graphitemergetree.md | 28 +- .../table-engines/mergetree-family/index.md | 4 +- .../mergetree-family/invertedindexes.md | 50 +- .../mergetree-family/mergetree.md | 831 +- .../mergetree-family/replacingmergetree.md | 12 +- .../mergetree-family/replication.md | 189 +- .../mergetree-family/summingmergetree.md | 20 +- .../versionedcollapsingmergetree.md | 16 +- .../engines/table-engines/special/alias.md | 102 +- .../engines/table-engines/special/buffer.md | 25 +- .../table-engines/special/dictionary.md | 8 +- .../table-engines/special/distributed.md | 12 +- .../table-engines/special/executable.md | 19 +- .../table-engines/special/external-data.md | 2 +- .../engines/table-engines/special/file.md | 8 +- .../engines/table-engines/special/filelog.md | 4 +- .../engines/table-engines/special/generate.md | 6 +- .../engines/table-engines/special/index.md | 5 +- .../engines/table-engines/special/join.md | 6 +- .../table-engines/special/keepermap.md | 12 +- .../engines/table-engines/special/memory.md | 6 +- .../engines/table-engines/special/merge.md | 6 +- .../engines/table-engines/special/null.md | 2 +- .../engines/table-engines/special/set.md | 2 +- .../engines/table-engines/special/url.md | 4 +- .../engines/table-engines/special/view.md | 2 +- .../current/faq/general/concurrency.md | 2 +- .../current/faq/general/cost-based.md | 2 +- .../current/faq/general/datalake.md | 2 +- .../current/faq/general/dependencies.md | 2 +- .../current/faq/general/distributed-join.md | 2 +- .../current/faq/general/federated.md | 22 +- .../current/faq/general/index.md | 22 +- .../current/faq/general/sql.md | 60 +- .../current/faq/general/updates.md | 2 +- .../current/faq/integration/index.md | 16 +- .../current/faq/integration/json-import.md | 2 +- .../current/faq/integration/oracle-odbc.md | 4 +- .../current/faq/operations/delete-old-data.md | 2 +- .../current/faq/operations/index.md | 18 +- .../current/faq/use-cases/index.md | 6 +- .../example-datasets/amazon-reviews.md | 8 +- .../anon_web_analytics_metrica.md | 22 +- .../example-datasets/brown-benchmark.md | 4 +- .../example-datasets/cell-towers.md | 13 +- .../example-datasets/dbpedia.md | 14 +- .../example-datasets/foursquare-os-places.md | 4 +- .../example-datasets/github.md | 58 +- .../example-datasets/hacker-news.md | 2 +- .../getting-started/example-datasets/laion.md | 28 +- .../getting-started/example-datasets/menus.md | 24 +- .../getting-started/example-datasets/noaa.md | 22 +- .../example-datasets/nyc-taxi.md | 12 +- .../example-datasets/nypd_complaint_data.md | 22 +- .../example-datasets/ontime.md | 6 +- .../example-datasets/stackoverflow.md | 30 +- .../getting-started/example-datasets/tpch.md | 18 +- .../example-datasets/tw-weather.md | 36 +- .../example-datasets/uk-price-paid.md | 12 +- .../example-datasets/youtube-dislikes.md | 10 +- .../current/getting-started/index.md | 5 +- .../install/_snippets/_deb_install.md | 57 +- .../install/_snippets/_docker.md | 36 +- .../install/_snippets/_linux_tar_install.md | 16 +- .../install/_snippets/_macos.md | 10 +- .../install/_snippets/_quick_install.md | 8 +- .../install/_snippets/_rpm_install.md | 8 +- .../install/_snippets/_windows_install.md | 10 +- .../getting-started/install/advanced.md | 2 +- .../getting-started/install/debian_ubuntu.md | 2 +- .../getting-started/install/install.mdx | 37 +- .../current/getting-started/playground.md | 4 +- .../getting-started/quick-start/cloud.mdx | 39 +- .../getting-started/quick-start/oss.mdx | 19 +- .../current/guides/best-practices/index.md | 7 +- .../current/guides/best-practices/prewhere.md | 4 +- .../best-practices/query-optimization.md | 22 +- .../best-practices/query-parallelism.md | 8 +- .../skipping-indexes-examples.md | 18 +- .../guides/best-practices/skipping-indexes.md | 14 +- .../best-practices/sparse-primary-indexes.md | 48 +- .../developer/alternative-query-languages.md | 9 +- .../developer/cascading-materialized-views.md | 31 +- .../developer/debugging-memory-issues.md | 14 +- .../deduplicating-inserts-on-retries.md | 10 +- .../current/guides/developer/deduplication.md | 10 +- .../developer/dynamic-column-selection.md | 19 +- .../current/guides/developer/index.md | 4 +- .../guides/developer/merge-table-function.md | 15 +- .../guides/developer/on-fly-mutations.md | 4 +- .../guides/developer/replacing-merge-tree.md | 8 +- ...ored-procedures-and-prepared-statements.md | 18 +- .../developer/time-series-filling-gaps.md | 27 +- .../current/guides/developer/ttl.md | 14 +- ...nding-query-execution-with-the-analyzer.md | 10 +- .../aggregate_function_combinators/anyIf.md | 2 +- .../argMaxIf.md | 2 +- .../argMinIf.md | 2 +- .../aggregate_function_combinators/avgIf.md | 2 +- .../aggregate_function_combinators/avgMap.md | 2 +- .../avgMergeState.md | 2 +- .../avgResample.md | 4 +- .../avgState.md | 2 +- .../aggregate_function_combinators/countIf.md | 2 +- .../countResample.md | 4 +- .../groupArrayDistinct.md | 2 +- .../groupArrayResample.md | 2 +- .../aggregate_function_combinators/maxMap.md | 2 +- .../aggregate_function_combinators/minMap.md | 2 +- .../minSimpleState.md | 2 +- .../quantilesTimingArrayIf.md | 2 +- .../quantilesTimingIf.md | 2 +- .../sumArray.md | 2 +- .../sumForEach.md | 2 +- .../aggregate_function_combinators/sumIf.md | 6 +- .../aggregate_function_combinators/sumMap.md | 2 +- .../sumSimpleState.md | 4 +- .../uniqArray.md | 2 +- .../uniqArrayIf.md | 4 +- .../current/guides/manage-and-deploy-index.md | 4 +- .../guides/separation-storage-compute.md | 19 +- .../current/guides/sre/configuring-ssl.md | 4 +- .../current/guides/sre/keeper/index.md | 60 +- .../current/guides/sre/network-ports.md | 2 +- .../current/guides/sre/scaling-clusters.md | 2 +- .../sre/user-management/configuring-ldap.md | 4 +- .../guides/sre/user-management/index.md | 4 +- .../sre/user-management/ssl-user-auth.md | 4 +- .../guides/starter_guides/creating-tables.md | 2 +- .../starter_guides/generating-test-data.md | 24 +- .../guides/starter_guides/inserting-data.md | 2 +- .../guides/starter_guides/mutations.md | 14 +- .../starter_guides/working_with_arrays.md | 20 +- .../current/guides/troubleshooting.md | 30 +- .../data-ingestion/apache-spark/index.md | 9 +- .../data-ingestion/apache-spark/spark-jdbc.md | 10 +- .../apache-spark/spark-native-connector.md | 20 +- .../data-ingestion/aws-glue/index.md | 2 +- .../azure-data-factory/overview.md | 5 +- .../using_azureblobstorage.md | 4 +- .../using_http_interface.md | 2 +- .../data-ingestion/azure-synapse/index.md | 4 +- .../clickpipes/aws-privatelink.md | 2 +- .../data-ingestion/clickpipes/index.md | 2 +- .../clickpipes/kafka/03_reference.md | 2 +- .../clickpipes/kafka/04_best_practices.md | 8 +- .../data-ingestion/clickpipes/kafka/index.md | 18 +- .../data-ingestion/clickpipes/kinesis.md | 2 +- .../clickpipes/mongodb/add_table.md | 2 +- .../data-ingestion/clickpipes/mongodb/faq.md | 18 +- .../clickpipes/mongodb/index.md | 2 +- .../clickpipes/mongodb/quickstart.md | 28 +- .../clickpipes/mongodb/resync.md | 6 +- .../clickpipes/mongodb/source/atlas.md | 2 +- .../clickpipes/mongodb/source/documentdb.md | 10 +- .../clickpipes/mongodb/source/generic.md | 6 +- .../clickpipes/mysql/add_table.md | 2 +- .../data-ingestion/clickpipes/mysql/faq.md | 2 +- .../data-ingestion/clickpipes/mysql/index.md | 2 +- .../data-ingestion/clickpipes/mysql/resync.md | 6 +- .../clickpipes/mysql/source/aurora.md | 8 +- .../clickpipes/mysql/source/gcp.md | 2 +- .../clickpipes/mysql/source/generic.md | 8 +- .../clickpipes/mysql/source/generic_maria.md | 4 +- .../clickpipes/mysql/source/rds.md | 8 +- .../clickpipes/mysql/source/rds_maria.md | 8 +- .../clickpipes/postgres/add_table.md | 2 +- .../clickpipes/postgres/deduplication.md | 23 +- .../data-ingestion/clickpipes/postgres/faq.md | 48 +- .../clickpipes/postgres/index.md | 2 +- .../clickpipes/postgres/maintenance.md | 6 +- .../clickpipes/postgres/ordering_keys.md | 13 +- .../clickpipes/postgres/resync.md | 6 +- .../clickpipes/postgres/scaling.md | 6 +- .../clickpipes/postgres/source/aurora.md | 4 +- .../source/azure-flexible-server-postgres.md | 2 +- .../postgres/source/crunchy-postgres.md | 4 +- .../clickpipes/postgres/source/generic.md | 6 +- .../postgres/source/google-cloudsql.md | 2 +- .../postgres/source/neon-postgres.md | 6 +- .../clickpipes/postgres/source/planetscale.md | 4 +- .../clickpipes/postgres/source/rds.md | 4 +- .../clickpipes/postgres/source/supabase.md | 6 +- .../clickpipes/postgres/source/timescale.md | 4 +- .../clickpipes/postgres/toast.md | 4 +- .../data-ingestion/clickpipes/secure-rds.md | 21 +- .../data-formats/arrow-avro-orc.md | 14 +- .../data-ingestion/data-formats/binary.md | 24 +- .../data-ingestion/data-formats/csv-tsv.md | 32 +- .../data-ingestion/data-formats/intro.md | 2 +- .../data-formats/json/exporting.md | 16 +- .../data-formats/json/formats.md | 36 +- .../data-formats/json/inference.md | 12 +- .../data-ingestion/data-formats/json/intro.md | 27 +- .../data-formats/json/loading.md | 6 +- .../data-ingestion/data-formats/json/other.md | 36 +- .../data-formats/json/schema.md | 20 +- .../data-ingestion/data-formats/parquet.md | 14 +- .../data-ingestion/data-formats/sql.md | 8 +- .../data-formats/templates-regex.md | 14 +- .../data-ingestion/data-ingestion-index.md | 2 +- .../data-ingestion/data-sources-index.md | 2 +- .../data-ingestion/dbms/dynamodb/index.md | 10 +- .../dbms/jdbc-with-clickhouse.md | 6 +- .../postgresql/connecting-to-postgresql.md | 28 +- .../integrations/data-ingestion/emqx/index.md | 24 +- .../etl-tools/airbyte-and-clickhouse.md | 4 +- .../data-ingestion/etl-tools/apache-beam.md | 8 +- .../etl-tools/bladepipe-and-clickhouse.md | 2 +- .../dbt/features-and-configurations.md | 84 +- .../data-ingestion/etl-tools/dbt/guides.md | 16 +- .../data-ingestion/etl-tools/dbt/index.md | 16 +- .../etl-tools/dlt-and-clickhouse.md | 18 +- .../etl-tools/fivetran/index.md | 2 +- .../etl-tools/nifi-and-clickhouse.md | 4 +- .../etl-tools/vector-to-clickhouse.md | 369 +- .../integrations/data-ingestion/gcs/index.md | 62 +- .../google-dataflow/dataflow.md | 2 +- .../google-dataflow/java-runner.md | 2 +- .../google-dataflow/templates.md | 2 +- .../templates/bigquery-to-clickhouse.md | 2 +- .../data-ingestion/insert-local-files.md | 3 +- .../kafka/confluent/confluent-cloud.md | 4 +- .../kafka/confluent/custom-connector.md | 22 +- .../data-ingestion/kafka/confluent/index.md | 2 +- .../kafka/confluent/kafka-connect-http.md | 35 +- .../data-ingestion/kafka/index.md | 2 +- .../kafka/kafka-clickhouse-connect-sink.md | 98 +- .../kafka/kafka-connect-jdbc.md | 10 +- .../kafka-table-engine-named-collections.md | 22 +- .../kafka/kafka-table-engine.md | 62 +- .../data-ingestion/kafka/kafka-vector.md | 13 +- .../data-ingestion/kafka/msk/index.md | 14 +- .../redshift/_snippets/_migration_guide.md | 2 +- .../integrations/data-ingestion/s3-minio.md | 4 +- .../integrations/data-ingestion/s3/index.md | 1488 +- .../data-ingestion/s3/performance.md | 24 +- .../integrations/data-sources/cassandra.md | 2 +- .../integrations/data-sources/mysql.md | 16 +- .../integrations/data-sources/postgres.md | 2 +- .../data-sources/table_engines/deltalake.md | 2 +- .../data-sources/table_engines/hive.md | 2 +- .../data-sources/table_engines/hudi.md | 2 +- .../data-sources/table_engines/iceberg.md | 2 +- .../data-sources/table_engines/mongodb.md | 2 +- .../data-sources/table_engines/nats.md | 2 +- .../data-sources/table_engines/rabbitmq.md | 2 +- .../data-sources/table_engines/redis.md | 2 +- .../data-sources/table_engines/rocksdb.md | 2 +- .../data-sources/table_engines/sqlite.md | 2 +- .../astrato-and-clickhouse.md | 4 +- .../chartbrew-and-clickhouse.md | 6 +- .../databrain-and-clickhouse.md | 14 +- .../community_integrations/deepnote.md | 4 +- .../dot-and-clickhouse.md | 2 +- .../draxlr-and-clickhouse.md | 4 +- .../embeddable-and-clickhouse.md | 6 +- .../explo-and-clickhouse.md | 4 +- .../fabi-and-clickhouse.md | 4 +- .../hashboard-and-clickhouse.md | 4 +- .../luzmo-and-clickhouse.md | 4 +- .../mitzu-and-clickhouse.md | 4 +- .../rocketbi-and-clickhouse.md | 6 +- .../zingdata-and-clickhouse.md | 4 +- .../data-visualization/grafana/config.md | 22 +- .../data-visualization/grafana/index.md | 4 +- .../grafana/query-builder.md | 8 +- .../integrations/data-visualization/index.md | 2 +- .../lightdash-and-clickhouse.md | 4 +- .../looker-and-clickhouse.md | 4 +- .../looker-studio-and-clickhouse.md | 6 +- .../metabase-and-clickhouse.md | 4 +- .../data-visualization/omni-and-clickhouse.md | 4 +- .../powerbi-and-clickhouse.md | 4 +- .../quicksight-and-clickhouse.md | 6 +- .../splunk-and-clickhouse.md | 8 +- .../superset-and-clickhouse.md | 4 +- .../tableau/tableau-analysis-tips.md | 8 +- .../tableau/tableau-and-clickhouse.md | 4 +- .../tableau/tableau-connection-tips.md | 6 +- .../tableau/tableau-online-and-clickhouse.md | 6 +- .../current/integrations/index.mdx | 3 +- .../integrations/language-clients/csharp.md | 34 +- .../integrations/language-clients/go/index.md | 112 +- .../java/client/_snippets/_v0_7.mdx | 138 +- .../java/client/_snippets/_v0_8.mdx | 200 +- .../language-clients/java/index.md | 6 +- .../java/jdbc/_snippets/_v0_7.mdx | 171 +- .../java/jdbc/_snippets/_v0_8.mdx | 90 +- .../language-clients/java/r2dbc.md | 10 +- .../integrations/language-clients/js.md | 58 +- .../language-clients/moose-olap.md | 14 +- .../python/additional-options.md | 2 +- .../python/advanced-inserting.md | 44 +- .../python/advanced-querying.md | 138 +- .../language-clients/python/advanced-usage.md | 8 +- .../language-clients/python/driver-api.md | 172 +- .../language-clients/python/index.md | 10 +- .../language-clients/python/sqlalchemy.md | 16 +- .../integrations/language-clients/rust.md | 32 +- .../current/integrations/misc/index.md | 2 +- .../integrations/sql-clients/datagrip.md | 4 +- .../integrations/sql-clients/dbeaver.md | 2 +- .../integrations/sql-clients/dbvisualizer.md | 15 +- .../current/integrations/sql-clients/index.md | 2 +- .../integrations/sql-clients/jupysql.md | 5 +- .../integrations/sql-clients/marimo.md | 6 +- .../integrations/sql-clients/qstudio.md | 15 +- .../integrations/sql-clients/sql-console.md | 8 +- .../integrations/sql-clients/tablum.md | 2 +- .../tools/data-integration/easypanel/index.md | 2 +- .../tools/data-integration/index.md | 2 +- .../tools/data-integration/retool/index.md | 4 +- .../tools/data-integration/splunk/index.md | 8 +- .../current/integrations/tools/index.md | 2 +- .../current/interfaces/arrowflight.md | 6 +- .../current/interfaces/cli.md | 77 +- .../current/interfaces/cpp.md | 4 +- .../current/interfaces/formats/Arrow/Arrow.md | 4 +- .../current/interfaces/formats/Avro/Avro.md | 6 +- .../interfaces/formats/Avro/AvroConfluent.md | 6 +- .../current/interfaces/formats/BSONEachRow.md | 4 +- .../current/interfaces/formats/CSV/CSV.md | 2 +- .../interfaces/formats/CSV/CSVWithNames.md | 4 +- .../formats/CSV/CSVWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/CapnProto.md | 6 +- .../CustomSeparated/CustomSeparated.md | 4 +- .../CustomSeparatedIgnoreSpaces.md | 2 +- .../CustomSeparatedIgnoreSpacesWithNames.md | 2 +- ...mSeparatedIgnoreSpacesWithNamesAndTypes.md | 2 +- .../CustomSeparatedWithNames.md | 4 +- .../CustomSeparatedWithNamesAndTypes.md | 4 +- .../current/interfaces/formats/DWARF.md | 2 +- .../current/interfaces/formats/Form.md | 2 +- .../current/interfaces/formats/Hash.md | 2 +- .../current/interfaces/formats/JSON/JSON.md | 2 +- .../interfaces/formats/JSON/JSONAsObject.md | 6 +- .../interfaces/formats/JSON/JSONAsString.md | 4 +- .../interfaces/formats/JSON/JSONColumns.md | 4 +- .../formats/JSON/JSONColumnsWithMetadata.md | 2 +- .../interfaces/formats/JSON/JSONCompact.md | 4 +- .../formats/JSON/JSONCompactColumns.md | 4 +- .../formats/JSON/JSONCompactEachRow.md | 4 +- .../JSON/JSONCompactEachRowWithNames.md | 4 +- .../JSONCompactEachRowWithNamesAndTypes.md | 4 +- .../JSON/JSONCompactEachRowWithProgress.md | 2 +- .../formats/JSON/JSONCompactStrings.md | 2 +- .../formats/JSON/JSONCompactStringsEachRow.md | 4 +- .../JSONCompactStringsEachRowWithNames.md | 4 +- ...NCompactStringsEachRowWithNamesAndTypes.md | 4 +- .../JSONCompactStringsEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONEachRow.md | 4 +- .../formats/JSON/JSONEachRowWithProgress.md | 2 +- .../interfaces/formats/JSON/JSONLines.md | 4 +- .../formats/JSON/JSONObjectEachRow.md | 16 +- .../interfaces/formats/JSON/JSONStrings.md | 2 +- .../formats/JSON/JSONStringsEachRow.md | 6 +- .../JSON/JSONStringsEachRowWithProgress.md | 2 +- .../formats/JSON/PrettyJSONEachRow.md | 2 +- .../formats/LineAsString/LineAsString.md | 2 +- .../LineAsString/LineAsStringWithNames.md | 2 +- .../LineAsStringWithNamesAndTypes.md | 2 +- .../current/interfaces/formats/Markdown.md | 2 +- .../current/interfaces/formats/MsgPack.md | 2 +- .../current/interfaces/formats/MySQLDump.md | 2 +- .../current/interfaces/formats/Npy.md | 8 +- .../current/interfaces/formats/Null.md | 4 +- .../current/interfaces/formats/ORC.md | 6 +- .../current/interfaces/formats/One.md | 2 +- .../interfaces/formats/Parquet/Parquet.md | 6 +- .../formats/Parquet/ParquetMetadata.md | 2 +- .../interfaces/formats/Pretty/Pretty.md | 2 +- .../formats/Pretty/PrettyNoEscapes.md | 2 +- .../current/interfaces/formats/Prometheus.md | 20 +- .../interfaces/formats/Protobuf/Protobuf.md | 6 +- .../formats/Protobuf/ProtobufList.md | 2 +- .../current/interfaces/formats/RawBLOB.md | 6 +- .../current/interfaces/formats/Regexp.md | 2 +- .../RowBinary/RowBinaryWithDefaults.md | 2 +- .../current/interfaces/formats/SQLInsert.md | 2 +- .../interfaces/formats/TabSeparated/TSKV.md | 8 +- .../formats/TabSeparated/TabSeparated.md | 10 +- .../formats/TabSeparated/TabSeparatedRaw.md | 6 +- .../TabSeparated/TabSeparatedRawWithNames.md | 6 +- .../TabSeparatedRawWithNamesAndTypes.md | 6 +- .../TabSeparated/TabSeparatedWithNames.md | 6 +- .../TabSeparatedWithNamesAndTypes.md | 6 +- .../interfaces/formats/Template/Template.md | 16 +- .../formats/Template/TemplateIgnoreSpaces.md | 2 +- .../current/interfaces/formats/Vertical.md | 2 +- .../current/interfaces/formats/XML.md | 2 +- .../current/interfaces/grpc.md | 6 +- .../current/interfaces/http.md | 64 +- .../current/interfaces/jdbc.md | 2 +- .../current/interfaces/mysql.md | 10 +- .../native-clients-interfaces-index.md | 4 +- .../current/interfaces/odbc.md | 2 +- .../current/interfaces/overview.md | 30 +- .../current/interfaces/postgresql.md | 12 +- .../current/interfaces/prometheus.md | 17 +- .../current/interfaces/schema-inference.md | 92 +- .../current/interfaces/ssh.md | 16 +- .../current/interfaces/tcp.md | 2 +- .../third-party/client-libraries.md | 38 +- .../current/interfaces/third-party/gui.md | 2 +- .../current/interfaces/third-party/index.md | 10 +- .../interfaces/third-party/integrations.md | 2 +- .../current/interfaces/third-party/proxy.md | 2 +- .../current/intro.md | 2 +- .../core-concepts/academic_overview.mdx | 172 +- .../managing-data/core-concepts/merges.mdx | 36 +- .../core-concepts/partitions.mdx | 28 +- .../managing-data/core-concepts/parts.md | 4 +- .../core-concepts/primary-indexes.mdx | 39 +- .../managing-data/core-concepts/shards.mdx | 28 +- .../deleting-data/delete_mutations.mdx | 2 +- .../managing-data/deleting-data/overview.mdx | 14 +- .../current/managing-data/drop_partition.mdx | 5 +- .../current/managing-data/truncate.md | 2 +- .../managing-data/updating-data/overview.mdx | 21 +- .../updating-data/update_mutations.mdx | 2 +- .../incremental-materialized-view.md | 32 +- .../refreshable-materialized-view.md | 14 +- .../current/native-protocol/basics.md | 6 +- .../current/native-protocol/client.md | 2 +- .../current/native-protocol/columns.md | 6 +- .../current/native-protocol/hash.md | 2 +- .../current/native-protocol/server.md | 2 +- .../current/operations/_troubleshooting.md | 20 +- .../operations/allocation-profiling-old.md | 22 +- .../operations/allocation-profiling.md | 26 +- .../current/operations/analyzer.md | 37 +- .../current/operations/backup.md | 44 +- .../current/operations/caches.md | 34 +- .../current/operations/cluster-discovery.md | 23 +- .../current/operations/configuration-files.md | 16 +- .../external-authenticators/http.md | 11 +- .../external-authenticators/index.md | 2 +- .../external-authenticators/kerberos.md | 18 +- .../external-authenticators/ldap.md | 102 +- .../external-authenticators/ssl-x509.md | 2 +- .../current/operations/monitoring.md | 2 +- .../current/operations/named-collections.md | 96 +- .../current/operations/opentelemetry.md | 2 +- .../profile-guided-optimization.md | 7 +- .../sampling-query-profiler.md | 8 +- .../current/operations/performance-test.md | 5 +- .../current/operations/query-cache.md | 141 +- .../operations/query-condition-cache.md | 4 +- .../_server_settings_outside_source.md | 186 +- .../settings.md | 5130 +------ .../settings/composable-protocols.md | 16 +- .../settings/constraints-on-settings.md | 14 +- .../current/operations/settings/index.md | 28 +- .../operations/settings/memory-overcommit.md | 4 +- .../settings/merge-tree-settings.md | 3032 +--- .../current/operations/settings/overview.md | 4 +- .../settings/permissions-for-queries.md | 2 +- .../operations/settings/query-complexity.md | 2 +- .../operations/settings/server-overload.md | 4 +- .../operations/settings/settings-formats.md | 2798 +--- .../operations/settings/settings-profiles.md | 2 +- .../settings/settings-query-level.md | 32 +- .../operations/settings/settings-users.md | 25 +- .../current/operations/settings/settings.md | 11428 +-------------- .../settings/tcp-connection-limits.md | 2 +- .../current/operations/ssl-zookeeper.md | 4 +- .../current/operations/startup-scripts.md | 2 +- .../current/operations/storing-data.md | 48 +- .../system-tables/asynchronous_insert_log.md | 5 +- .../system-tables/asynchronous_inserts.md | 19 +- .../system-tables/asynchronous_loader.md | 76 +- .../system-tables/asynchronous_metric_log.md | 2 +- .../system-tables/asynchronous_metrics.md | 22 +- .../system-tables/azure_queue_settings.md | 17 +- .../operations/system-tables/backup_log.md | 7 +- .../operations/system-tables/backups.md | 2 +- .../system-tables/blob_storage_log.md | 2 +- .../operations/system-tables/build_options.md | 11 +- .../operations/system-tables/clusters.md | 69 +- .../operations/system-tables/codecs.md | 21 +- .../operations/system-tables/columns.md | 31 +- .../operations/system-tables/contributors.md | 10 +- .../operations/system-tables/crash_log.md | 2 +- .../operations/system-tables/dashboards.md | 20 +- .../system-tables/data_skipping_indices.md | 15 +- .../system-tables/data_type_families.md | 12 +- .../system-tables/database_engines.md | 8 +- .../system-tables/database_replicas.md | 22 +- .../operations/system-tables/databases.md | 17 +- .../system-tables/delta_metadata_log.md | 6 +- .../system-tables/detached_tables.md | 8 +- .../operations/system-tables/dictionaries.md | 50 +- .../system-tables/dimensional_metrics.md | 4 +- .../current/operations/system-tables/disks.md | 26 +- .../system-tables/distributed_ddl_queue.md | 34 +- .../system-tables/distribution_queue.md | 16 +- .../operations/system-tables/dns_cache.md | 19 +- .../system-tables/dropped_tables.md | 16 +- .../operations/system-tables/error_log.md | 2 +- .../operations/system-tables/errors.md | 20 +- .../operations/system-tables/events.md | 14 +- .../operations/system-tables/functions.md | 19 +- .../system-tables/graphite_retentions.md | 15 +- .../system-tables/histogram_metrics.md | 27 +- .../system-tables/iceberg_history.md | 15 +- .../system-tables/iceberg_metadata_log.md | 8 +- .../current/operations/system-tables/index.md | 201 +- .../system-tables/information_schema.md | 10 +- .../operations/system-tables/jemalloc_bins.md | 20 +- .../system-tables/kafka_consumers.md | 26 +- .../operations/system-tables/licenses.md | 13 +- .../system-tables/merge_tree_settings.md | 35 +- .../operations/system-tables/merges.md | 36 +- .../operations/system-tables/metric_log.md | 5 +- .../operations/system-tables/metrics.md | 384 +- .../current/operations/system-tables/moves.md | 21 +- .../operations/system-tables/mutations.md | 7 +- .../operations/system-tables/numbers.md | 2 +- .../current/operations/system-tables/one.md | 2 +- .../system-tables/opentelemetry_span_log.md | 5 +- .../operations/system-tables/overview.md | 10 +- .../operations/system-tables/part_log.md | 9 +- .../current/operations/system-tables/parts.md | 3 +- .../operations/system-tables/parts_columns.md | 2 +- .../operations/system-tables/processes.md | 58 +- .../system-tables/processors_profile_log.md | 6 +- .../system-tables/projection_parts.md | 73 +- .../system-tables/projection_parts_columns.md | 63 +- .../operations/system-tables/projections.md | 17 +- .../operations/system-tables/query_cache.md | 21 +- .../system-tables/query_condition_cache.md | 12 +- .../operations/system-tables/query_log.md | 6 +- .../system-tables/query_metric_log.md | 5 +- .../system-tables/query_thread_log.md | 15 +- .../system-tables/query_views_log.md | 73 +- .../operations/system-tables/quota_limits.md | 30 +- .../operations/system-tables/quota_usage.md | 39 +- .../operations/system-tables/quotas.md | 2 +- .../operations/system-tables/quotas_usage.md | 39 +- .../operations/system-tables/replicas.md | 71 +- .../system-tables/replicated_fetches.md | 28 +- .../system-tables/replication_queue.md | 3 +- .../operations/system-tables/resources.md | 18 +- .../operations/system-tables/role_grants.md | 20 +- .../current/operations/system-tables/roles.md | 16 +- .../operations/system-tables/row_policies.md | 24 +- .../system-tables/s3_queue_settings.md | 18 +- .../operations/system-tables/scheduler.md | 40 +- .../system-tables/schema_inference_cache.md | 37 +- .../system-tables/server_settings.md | 6 +- .../operations/system-tables/session_log.md | 6 +- .../operations/system-tables/settings.md | 5 +- .../system-tables/settings_changes.md | 14 +- .../settings_profile_elements.md | 21 +- .../system-tables/settings_profiles.md | 16 +- .../operations/system-tables/stack_trace.md | 7 +- .../system-tables/storage_policies.md | 32 +- .../system-tables/system_warnings.md | 5 +- .../operations/system-tables/table_engines.md | 22 +- .../operations/system-tables/tables.md | 2 +- .../operations/system-tables/text_log.md | 5 +- .../operations/system-tables/time_zones.md | 12 +- .../operations/system-tables/trace_log.md | 6 +- .../operations/system-tables/unicode.md | 4 +- .../system-tables/user_processes.md | 16 +- .../current/operations/system-tables/users.md | 29 +- .../system-tables/view_refreshes.md | 29 +- .../operations/system-tables/workloads.md | 20 +- .../operations/system-tables/zookeeper.md | 2 +- .../system-tables/zookeeper_connection.md | 5 +- .../system-tables/zookeeper_connection_log.md | 6 +- .../operations/system-tables/zookeeper_log.md | 43 +- .../current/operations/tips.md | 68 +- .../current/operations/update.md | 4 +- .../operations/userspace-page-cache.md | 8 +- .../operations/utilities/backupview.md | 20 +- .../utilities/clickhouse-benchmark.md | 6 +- .../utilities/clickhouse-compressor.md | 2 +- .../operations/utilities/clickhouse-disks.md | 2 +- .../operations/utilities/clickhouse-format.md | 34 +- .../utilities/clickhouse-keeper-client.md | 4 +- .../operations/utilities/clickhouse-local.md | 14 +- .../operations/utilities/odbc-bridge.md | 2 +- .../current/operations/workload-scheduling.md | 22 +- .../operations_/backup_restore/00_overview.md | 14 +- .../backup_restore/01_local_disk.md | 28 +- .../backup_restore/02_s3_endpoint.md | 2 +- .../backup_restore/03_azure_blob_storage.md | 4 +- .../backup_restore/04_alternative_methods.md | 2 +- .../aggregate-functions/combinators.md | 12 +- .../aggregate-functions/grouping_function.md | 12 +- .../aggregate-functions/index.md | 13 +- .../parametric-functions.md | 42 +- .../aggregate-functions/reference/aggthrow.md | 2 +- .../reference/analysis_of_variance.md | 2 +- .../aggregate-functions/reference/any.md | 2 +- .../aggregate-functions/reference/anyheavy.md | 2 +- .../aggregate-functions/reference/anylast.md | 2 +- .../reference/approxtopk.md | 27 +- .../reference/approxtopsum.md | 18 +- .../reference/argandmax.md | 6 +- .../reference/argandmin.md | 3 +- .../aggregate-functions/reference/argmax.md | 2 +- .../aggregate-functions/reference/argmin.md | 3 +- .../aggregate-functions/reference/avg.md | 2 +- .../reference/avgweighted.md | 2 +- .../reference/contingency.md | 2 +- .../aggregate-functions/reference/corr.md | 2 +- .../reference/corrmatrix.md | 2 +- .../reference/corrstable.md | 2 +- .../aggregate-functions/reference/count.md | 2 +- .../aggregate-functions/reference/covarpop.md | 2 +- .../reference/covarpopmatrix.md | 2 +- .../reference/covarpopstable.md | 2 +- .../reference/covarsamp.md | 2 +- .../reference/covarsampmatrix.md | 2 +- .../reference/covarsampstable.md | 2 +- .../aggregate-functions/reference/cramersv.md | 2 +- .../reference/cramersvbiascorrected.md | 2 +- .../aggregate-functions/reference/deltasum.md | 2 +- .../reference/distinctdynamictypes.md | 2 +- .../reference/distinctjsonpaths.md | 5 +- .../aggregate-functions/reference/entropy.md | 2 +- .../reference/estimateCompressionRatio.md | 4 +- .../reference/exponentialmovingaverage.md | 5 +- .../reference/exponentialtimedecayedavg.md | 5 +- .../reference/exponentialtimedecayedcount.md | 5 +- .../reference/exponentialtimedecayedmax.md | 5 +- .../reference/exponentialtimedecayedsum.md | 5 +- .../reference/first_value.md | 16 +- .../reference/flame_graph.md | 14 +- .../reference/grouparray.md | 2 +- .../reference/grouparrayarray.md | 2 +- .../reference/grouparrayinsertat.md | 2 +- .../reference/grouparrayintersect.md | 2 +- .../reference/grouparraylast.md | 2 +- .../reference/grouparraymovingavg.md | 2 +- .../reference/grouparraymovingsum.md | 2 +- .../reference/grouparraysample.md | 2 +- .../reference/grouparraysorted.md | 2 +- .../reference/groupbitand.md | 2 +- .../reference/groupbitmap.md | 2 +- .../reference/groupbitmapor.md | 2 +- .../reference/groupbitmapxor.md | 2 +- .../reference/groupbitor.md | 2 +- .../reference/groupbitxor.md | 2 +- .../reference/groupuniqarray.md | 2 +- .../aggregate-functions/reference/index.md | 287 +- .../reference/kolmogorovsmirnovtest.md | 5 +- .../aggregate-functions/reference/kurtpop.md | 2 +- .../aggregate-functions/reference/kurtsamp.md | 2 +- .../reference/largestTriangleThreeBuckets.md | 2 +- .../reference/last_value.md | 16 +- .../reference/mannwhitneyutest.md | 2 +- .../reference/maxintersections.md | 2 +- .../reference/maxintersectionsposition.md | 2 +- .../aggregate-functions/reference/maxmap.md | 2 +- .../reference/meanztest.md | 2 +- .../aggregate-functions/reference/median.md | 2 +- .../aggregate-functions/reference/minmap.md | 2 +- .../aggregate-functions/reference/quantile.md | 2 +- .../reference/quantileGK.md | 2 +- .../reference/quantilebfloat16.md | 2 +- .../reference/quantiledeterministic.md | 2 +- .../reference/quantileexact.md | 20 +- .../reference/quantileexactweighted.md | 2 +- .../quantileexactweightedinterpolated.md | 2 +- .../reference/quantileinterpolatedweighted.md | 2 +- .../reference/quantileprometheushistogram.md | 2 +- .../reference/quantiles.md | 8 +- .../reference/quantiletdigest.md | 2 +- .../reference/quantiletdigestweighted.md | 2 +- .../reference/quantiletiming.md | 2 +- .../reference/quantiletimingweighted.md | 5 +- .../aggregate-functions/reference/rankCorr.md | 2 +- .../reference/simplelinearregression.md | 2 +- .../reference/singlevalueornull.md | 2 +- .../aggregate-functions/reference/skewpop.md | 2 +- .../aggregate-functions/reference/skewsamp.md | 2 +- .../aggregate-functions/reference/sparkbar.md | 2 +- .../reference/stddevpop.md | 2 +- .../reference/stddevpopstable.md | 2 +- .../reference/stddevsamp.md | 2 +- .../reference/stddevsampstable.md | 2 +- .../reference/stochasticlinearregression.md | 15 +- .../reference/stochasticlogisticregression.md | 6 +- .../reference/studentttest.md | 2 +- .../reference/studentttestonesample.md | 2 +- .../aggregate-functions/reference/sum.md | 2 +- .../aggregate-functions/reference/summap.md | 3 +- .../reference/summapwithoverflow.md | 3 +- .../reference/sumwithoverflow.md | 2 +- .../aggregate-functions/reference/theilsu.md | 2 +- .../reference/timeSeriesGroupArray.md | 2 +- .../aggregate-functions/reference/topk.md | 22 +- .../reference/topkweighted.md | 2 +- .../aggregate-functions/reference/uniq.md | 2 +- .../reference/uniqcombined.md | 2 +- .../reference/uniqcombined64.md | 2 +- .../reference/uniqexact.md | 2 +- .../reference/uniqhll12.md | 2 +- .../reference/varpopstable.md | 4 +- .../reference/welchttest.md | 2 +- .../data-types/aggregatefunction.md | 66 +- .../current/sql-reference/data-types/array.md | 17 +- .../sql-reference/data-types/boolean.md | 2 +- .../data-types/data-types-binary-encoding.md | 60 +- .../current/sql-reference/data-types/date.md | 2 +- .../sql-reference/data-types/date32.md | 2 +- .../sql-reference/data-types/datetime.md | 6 +- .../sql-reference/data-types/datetime64.md | 8 +- .../sql-reference/data-types/decimal.md | 4 +- .../sql-reference/data-types/domains/index.md | 2 +- .../sql-reference/data-types/dynamic.md | 49 +- .../current/sql-reference/data-types/enum.md | 15 +- .../sql-reference/data-types/fixedstring.md | 2 +- .../current/sql-reference/data-types/float.md | 6 +- .../current/sql-reference/data-types/geo.md | 14 +- .../current/sql-reference/data-types/index.md | 4 +- .../current/sql-reference/data-types/ipv4.md | 6 +- .../current/sql-reference/data-types/ipv6.md | 6 +- .../data-types/lowcardinality.md | 6 +- .../current/sql-reference/data-types/map.md | 6 +- .../nested-data-structures/index.md | 9 +- .../sql-reference/data-types/newjson.md | 38 +- .../sql-reference/data-types/nullable.md | 6 +- .../current/sql-reference/data-types/qbit.md | 4 +- .../data-types/simpleaggregatefunction.md | 6 +- .../special-data-types/expression.md | 2 +- .../data-types/special-data-types/index.md | 2 +- .../data-types/special-data-types/interval.md | 4 +- .../data-types/special-data-types/nothing.md | 2 +- .../data-types/special-data-types/set.md | 2 +- .../sql-reference/data-types/string.md | 2 +- .../current/sql-reference/data-types/time.md | 4 +- .../sql-reference/data-types/time64.md | 4 +- .../current/sql-reference/data-types/tuple.md | 17 +- .../current/sql-reference/data-types/uuid.md | 4 +- .../sql-reference/data-types/variant.md | 34 +- .../sql-reference/dictionaries/index.md | 147 +- .../functions/arithmetic-functions.md | 1294 +- .../functions/array-functions.md | 4995 +------ .../sql-reference/functions/array-join.md | 7 +- .../sql-reference/functions/bit-functions.md | 687 +- .../functions/bitmap-functions.md | 715 +- .../functions/comparison-functions.md | 345 +- .../functions/conditional-functions.md | 410 +- .../functions/date-time-functions.md | 5657 +------- .../functions/distance-functions.md | 619 +- .../functions/embedded-dict-functions.md | 34 +- .../functions/encoding-functions.md | 1090 +- .../functions/encryption-functions.md | 385 +- .../functions/ext-dict-functions.md | 2366 +--- .../current/sql-reference/functions/files.md | 4 +- .../functions/financial-functions.md | 248 +- .../functions/functions-for-nulls.md | 475 +- .../functions/geo/coordinates.md | 16 +- .../functions/geo/flipCoordinates.md | 18 +- .../sql-reference/functions/geo/geohash.md | 6 +- .../current/sql-reference/functions/geo/h3.md | 88 +- .../sql-reference/functions/geo/polygon.md | 240 +- .../current/sql-reference/functions/geo/s2.md | 22 +- .../sql-reference/functions/geo/svg.md | 4 +- .../sql-reference/functions/hash-functions.md | 2641 +--- .../sql-reference/functions/in-functions.md | 2 +- .../sql-reference/functions/introspection.md | 259 +- .../functions/ip-address-functions.md | 1122 +- .../sql-reference/functions/json-functions.md | 2018 +-- .../functions/logical-functions.md | 201 +- .../functions/machine-learning-functions.md | 8 +- .../sql-reference/functions/math-functions.md | 1293 +- .../sql-reference/functions/nlp-functions.md | 336 +- .../numeric-indexed-vector-functions.md | 682 +- .../functions/other-functions.md | 5455 +------- .../sql-reference/functions/overview.md | 6 +- .../functions/random-functions.md | 813 +- .../functions/rounding-functions.md | 469 +- .../functions/splitting-merging-functions.md | 464 +- .../functions/string-functions.md | 3601 +---- .../functions/string-replace-functions.md | 512 +- .../functions/string-search-functions.md | 2693 +--- .../functions/time-series-functions.md | 427 +- .../functions/time-window-functions.md | 233 +- .../functions/tuple-functions.md | 833 +- .../functions/tuple-map-functions.md | 77 +- .../functions/type-conversion-functions.md | 308 +- .../current/sql-reference/functions/udf.md | 12 +- .../sql-reference/functions/ulid-functions.md | 98 +- .../functions/uniqtheta-functions.md | 14 +- .../sql-reference/functions/url-functions.md | 1412 +- .../sql-reference/functions/uuid-functions.md | 1166 +- .../current/sql-reference/index.md | 3 +- .../current/sql-reference/operators/exists.md | 2 +- .../current/sql-reference/operators/in.md | 15 +- .../current/sql-reference/operators/index.md | 20 +- .../sql-reference/sql-reference-links.json | 6 +- .../statements/alter/apply-deleted-mask.md | 4 +- .../sql-reference/statements/alter/column.md | 20 +- .../sql-reference/statements/alter/comment.md | 6 +- .../statements/alter/constraint.md | 4 +- .../statements/alter/database-comment.md | 6 +- .../sql-reference/statements/alter/delete.md | 2 +- .../sql-reference/statements/alter/index.md | 2 +- .../statements/alter/named-collection.md | 3 +- .../statements/alter/order-by.md | 2 +- .../statements/alter/partition.md | 66 +- .../statements/alter/projection.md | 12 +- .../statements/alter/row-policy.md | 2 +- .../statements/alter/sample-by.md | 11 +- .../sql-reference/statements/alter/setting.md | 12 +- .../statements/alter/skipping-index.md | 4 +- .../statements/alter/statistics.md | 22 +- .../sql-reference/statements/alter/ttl.md | 11 +- .../sql-reference/statements/alter/update.md | 2 +- .../sql-reference/statements/alter/user.md | 2 +- .../sql-reference/statements/alter/view.md | 4 +- .../sql-reference/statements/attach.md | 19 +- .../sql-reference/statements/check-grant.md | 4 +- .../sql-reference/statements/check-table.md | 6 +- .../statements/create/database.md | 15 +- .../statements/create/dictionary.md | 15 +- .../statements/create/function.md | 34 +- .../sql-reference/statements/create/index.md | 2 +- .../statements/create/named-collection.md | 3 +- .../sql-reference/statements/create/quota.md | 12 +- .../sql-reference/statements/create/role.md | 2 +- .../statements/create/row-policy.md | 50 +- .../statements/create/settings-profile.md | 13 +- .../sql-reference/statements/create/table.md | 62 +- .../sql-reference/statements/create/user.md | 4 +- .../sql-reference/statements/create/view.md | 52 +- .../sql-reference/statements/delete.md | 4 +- .../current/sql-reference/statements/drop.md | 38 +- .../sql-reference/statements/exchange.md | 74 +- .../sql-reference/statements/execute_as.md | 11 +- .../sql-reference/statements/exists.md | 2 +- .../sql-reference/statements/explain.md | 290 +- .../current/sql-reference/statements/grant.md | 38 +- .../current/sql-reference/statements/index.md | 2 +- .../sql-reference/statements/insert-into.md | 14 +- .../current/sql-reference/statements/kill.md | 7 +- .../current/sql-reference/statements/move.md | 2 +- .../sql-reference/statements/optimize.md | 15 +- .../sql-reference/statements/parallel_with.md | 6 +- .../sql-reference/statements/rename.md | 13 +- .../sql-reference/statements/revoke.md | 6 +- .../sql-reference/statements/select/all.md | 2 +- .../statements/select/apply_modifier.md | 4 +- .../statements/select/array-join.md | 14 +- .../statements/select/distinct.md | 4 +- .../sql-reference/statements/select/except.md | 29 +- .../statements/select/except_modifier.md | 4 +- .../sql-reference/statements/select/format.md | 2 +- .../sql-reference/statements/select/from.md | 8 +- .../statements/select/group-by.md | 14 +- .../sql-reference/statements/select/having.md | 11 +- .../sql-reference/statements/select/index.md | 10 +- .../statements/select/intersect.md | 12 +- .../statements/select/into-outfile.md | 7 +- .../sql-reference/statements/select/join.md | 16 +- .../statements/select/limit-by.md | 22 +- .../sql-reference/statements/select/limit.md | 20 +- .../sql-reference/statements/select/offset.md | 3 +- .../statements/select/order-by.md | 12 +- .../statements/select/prewhere.md | 4 +- .../statements/select/qualify.md | 4 +- .../statements/select/replace_modifier.md | 4 +- .../sql-reference/statements/select/sample.md | 28 +- .../sql-reference/statements/select/union.md | 2 +- .../sql-reference/statements/select/where.md | 26 +- .../sql-reference/statements/select/with.md | 30 +- .../sql-reference/statements/set-role.md | 6 +- .../current/sql-reference/statements/set.md | 2 +- .../current/sql-reference/statements/show.md | 134 +- .../sql-reference/statements/system.md | 31 +- .../sql-reference/statements/truncate.md | 14 +- .../sql-reference/statements/undrop.md | 2 +- .../sql-reference/statements/update.md | 4 +- .../current/sql-reference/statements/use.md | 2 +- .../current/sql-reference/statements/watch.md | 5 +- .../current/sql-reference/syntax.md | 466 +- .../table-functions/arrowflight.md | 2 +- .../table-functions/azureBlobStorage.md | 14 +- .../azureBlobStorageCluster.md | 6 +- .../sql-reference/table-functions/cluster.md | 6 +- .../table-functions/deltalake.md | 6 +- .../table-functions/deltalakeCluster.md | 4 +- .../table-functions/dictionary.md | 6 +- .../table-functions/executable.md | 6 +- .../sql-reference/table-functions/file.md | 28 +- .../table-functions/fileCluster.md | 6 +- .../sql-reference/table-functions/format.md | 6 +- .../sql-reference/table-functions/fuzzJSON.md | 6 +- .../table-functions/fuzzQuery.md | 6 +- .../sql-reference/table-functions/gcs.md | 12 +- .../sql-reference/table-functions/generate.md | 6 +- .../table-functions/generate_series.md | 11 +- .../sql-reference/table-functions/hdfs.md | 10 +- .../table-functions/hdfsCluster.md | 6 +- .../sql-reference/table-functions/hudi.md | 4 +- .../table-functions/hudiCluster.md | 4 +- .../sql-reference/table-functions/iceberg.md | 50 +- .../table-functions/icebergCluster.md | 6 +- .../sql-reference/table-functions/index.md | 144 +- .../sql-reference/table-functions/input.md | 8 +- .../sql-reference/table-functions/jdbc.md | 15 +- .../sql-reference/table-functions/loop.md | 6 +- .../sql-reference/table-functions/merge.md | 4 +- .../table-functions/mergeTreeIndex.md | 6 +- .../table-functions/mergeTreeProjection.md | 6 +- .../sql-reference/table-functions/mongodb.md | 8 +- .../sql-reference/table-functions/mysql.md | 8 +- .../sql-reference/table-functions/null.md | 6 +- .../sql-reference/table-functions/numbers.md | 2 +- .../sql-reference/table-functions/odbc.md | 6 +- .../sql-reference/table-functions/paimon.md | 4 +- .../table-functions/paimonCluster.md | 4 +- .../table-functions/postgresql.md | 8 +- .../table-functions/prometheusQuery.md | 6 +- .../table-functions/prometheusQueryRange.md | 6 +- .../sql-reference/table-functions/redis.md | 6 +- .../sql-reference/table-functions/remote.md | 22 +- .../sql-reference/table-functions/s3.md | 24 +- .../table-functions/s3Cluster.md | 6 +- .../sql-reference/table-functions/sqlite.md | 6 +- .../table-functions/timeSeriesData.md | 2 +- .../table-functions/timeSeriesMetrics.md | 2 +- .../table-functions/timeSeriesSelector.md | 6 +- .../table-functions/timeSeriesTags.md | 2 +- .../sql-reference/table-functions/url.md | 8 +- .../table-functions/urlCluster.md | 6 +- .../sql-reference/table-functions/values.md | 4 +- .../sql-reference/table-functions/view.md | 8 +- .../sql-reference/table-functions/ytsaurus.md | 4 +- .../sql-reference/table-functions/zeros.md | 2 +- .../current/sql-reference/transactions.md | 28 +- .../window-functions/cume_dist.md | 2 +- .../window-functions/dense_rank.md | 2 +- .../window-functions/first_value.md | 2 +- .../sql-reference/window-functions/index.md | 28 +- .../sql-reference/window-functions/lag.md | 2 +- .../window-functions/lagInFrame.md | 2 +- .../window-functions/last_value.md | 2 +- .../sql-reference/window-functions/lead.md | 2 +- .../window-functions/leadInFrame.md | 2 +- .../window-functions/nth_value.md | 2 +- .../window-functions/percent_rank.md | 2 +- .../sql-reference/window-functions/rank.md | 2 +- .../window-functions/row_number.md | 2 +- .../tips-and-tricks/debugging-insights.md | 32 +- .../tips-and-tricks/materialized-views.md | 2 +- .../performance-optimization.md | 6 +- .../current/tips-and-tricks/too-many-parts.md | 2 +- .../static-files-disk-uploader.md | 10 +- .../current/tutorial.md | 4 +- .../current/use-cases/AI_ML/AIChat/index.md | 2 +- .../use-cases/AI_ML/MCP/01_remote_mcp.md | 6 +- .../use-cases/AI_ML/MCP/02_claude-desktop.md | 4 +- .../use-cases/AI_ML/MCP/03_librechat.md | 26 +- .../use-cases/AI_ML/MCP/04_anythingllm.md | 10 +- .../use-cases/AI_ML/MCP/05_open-webui.md | 6 +- .../current/use-cases/AI_ML/MCP/06_ollama.md | 8 +- .../current/use-cases/AI_ML/MCP/07_janai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/agno.md | 6 +- .../AI_ML/MCP/ai_agent_libraries/chainlit.md | 6 +- .../ai_agent_libraries/claude-agent-sdk.md | 6 +- .../MCP/ai_agent_libraries/copilotkit.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/crewai.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/dspy.md | 2 +- .../AI_ML/MCP/ai_agent_libraries/index.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/langchain.md | 2 +- .../MCP/ai_agent_libraries/llamaindex.md | 10 +- .../AI_ML/MCP/ai_agent_libraries/mcp-agent.md | 2 +- .../microsoft-agent-framework.md | 2 +- .../MCP/ai_agent_libraries/openai-agents.md | 8 +- .../MCP/ai_agent_libraries/pydantic-ai.md | 8 +- .../AI_ML/MCP/ai_agent_libraries/slackbot.md | 4 +- .../AI_ML/MCP/ai_agent_libraries/streamlit.md | 16 +- .../AI_ML/MCP/ai_agent_libraries/upsonic.md | 2 +- .../AI_ML/ai-powered-sql-generation.md | 16 +- .../data-exploration/jupyter-notebook.md | 26 +- .../AI_ML/data-exploration/marimo-notebook.md | 66 +- .../current/use-cases/AI_ML/index.md | 2 +- .../use-cases/data_lake/glue_catalog.md | 4 +- .../use-cases/data_lake/lakekeeper_catalog.md | 6 +- .../use-cases/data_lake/nessie_catalog.md | 8 +- .../use-cases/data_lake/onelake_catalog.md | 8 +- .../use-cases/data_lake/rest_catalog.md | 8 +- .../use-cases/data_lake/unity_catalog.md | 8 +- .../observability/build-your-own/grafana.md | 16 +- .../integrating-opentelemetry.md | 16 +- .../build-your-own/introduction.md | 4 +- .../build-your-own/managing-data.md | 14 +- .../build-your-own/schema-design.md | 36 +- .../observability/clickstack/api-reference.md | 15 + .../observability/clickstack/config.md | 6 +- .../observability/clickstack/dashboards.md | 2 +- .../clickstack/deployment/all-in-one.md | 14 +- .../clickstack/deployment/docker-compose.md | 14 +- .../clickstack/deployment/helm/helm-cloud.md | 28 +- .../deployment/helm/helm-configuration.md | 28 +- .../helm/helm-deployment-options.md | 24 +- .../clickstack/deployment/helm/helm.md | 2 +- .../deployment/hyperdx-clickhouse-cloud.md | 2 +- .../clickstack/deployment/hyperdx-only.md | 4 +- .../clickstack/deployment/local-mode-only.md | 4 +- .../clickstack/example-datasets/kubernetes.md | 2 +- .../clickstack/example-datasets/local-data.md | 11 +- .../example-datasets/remote-demo-data.md | 2 +- .../clickstack/getting-started.md | 2 +- .../clickstack/ingesting-data/collector.md | 16 +- .../ingesting-data/opentelemetry.md | 4 +- .../clickstack/ingesting-data/schemas.md | 16 +- .../ingesting-data/sdks/aws-lambda.md | 8 +- .../clickstack/ingesting-data/sdks/browser.md | 10 +- .../clickstack/ingesting-data/sdks/deno.md | 7 +- .../clickstack/ingesting-data/sdks/elixir.md | 8 +- .../clickstack/ingesting-data/sdks/golang.md | 8 +- .../clickstack/ingesting-data/sdks/index.md | 2 +- .../clickstack/ingesting-data/sdks/java.md | 6 +- .../clickstack/ingesting-data/sdks/nestjs.md | 7 +- .../clickstack/ingesting-data/sdks/nextjs.md | 6 +- .../clickstack/ingesting-data/sdks/nodejs.md | 8 +- .../clickstack/ingesting-data/sdks/python.md | 12 +- .../ingesting-data/sdks/react-native.md | 10 +- .../clickstack/ingesting-data/sdks/ruby.md | 8 +- .../integration-examples/kafka-metrics.md | 8 +- .../integration-examples/kubernetes.md | 8 +- .../integration-examples/nginx-traces.md | 16 +- .../integration-examples/postgres-metrics.md | 14 +- .../integration-examples/redis-logs.md | 6 +- .../integration-examples/redis-metrics.md | 8 +- .../migration/elastic/migrating-sdks.md | 3 +- .../observability/clickstack/production.md | 6 +- .../observability/clickstack/search.md | 2 +- .../use-cases/observability/clickstack/ttl.md | 9 +- .../observability/cloud-monitoring.md | 6 +- .../observability/self-managed-monitoring.md | 6 +- .../time-series/01_date-time-data-types.md | 10 +- .../time-series/02_basic-operations.md | 17 +- .../time-series/03_analysis-functions.md | 21 +- .../time-series/04_storage-efficiency.md | 11 +- .../time-series/05_query-performance.md | 16 +- .../06_materialized-view-rollup.mdx | 25 +- .../current/whats-new/changelog/2020.md | 30 +- .../current/whats-new/changelog/2022.md | 40 +- .../current/whats-new/changelog/2023.md | 84 +- .../current/whats-new/changelog/2024.md | 82 +- .../current/whats-new/changelog/cloud.md | 4 +- .../current/whats-new/changelog/index.md | 4356 +++--- .../current/whats-new/security-changelog.md | 2 +- 4109 files changed, 47090 insertions(+), 257618 deletions(-) delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/01_discover/01_what_is.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/development/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/dictionary/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/integrations/apache-beam.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/clickpipes/kafka/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/statements/exchange.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/statements/explain.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/table-functions/index.md delete mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/use-cases/observability/build-your-own/schema-design.md create mode 100644 i18n/jp/docusaurus-plugin-content-docs/current/use-cases/observability/clickstack/api-reference.md create mode 100644 i18n/ru/docusaurus-plugin-content-docs/current/use-cases/observability/clickstack/api-reference.md create mode 100644 i18n/zh/docusaurus-plugin-content-docs/current/use-cases/observability/clickstack/api-reference.md diff --git a/docs/integrations/data-ingestion/clickpipes/kafka/index.md b/docs/integrations/data-ingestion/clickpipes/kafka/index.md index 7022039e647..d02835dbce0 100644 --- a/docs/integrations/data-ingestion/clickpipes/kafka/index.md +++ b/docs/integrations/data-ingestion/clickpipes/kafka/index.md @@ -15,9 +15,9 @@ keywords: ['Kafka ClickPipes', 'Apache Kafka', 'streaming ingestion', 'real-time | Page | Description | |-----|-----| -| [Reference](/integrations/clickpipes/kafka/reference) | Details supported formats, sources, delivery semantics, authentication and experimental features supported by Kafka ClickPipes | -| [Schema registries for Kafka ClickPipe](/integrations/clickpipes/kafka/schema-registries) | How to integrate for ClickPipes with a schema registry for schema management | | [Creating your first Kafka ClickPipe](/integrations/clickpipes/kafka/create-your-first-kafka-clickpipe) | Step-by-step guide to creating your first Kafka ClickPipe. | -| [Kafka ClickPipes FAQ](/integrations/clickpipes/kafka/faq) | Frequently asked questions about ClickPipes for Kafka | +| [Schema registries for Kafka ClickPipe](/integrations/clickpipes/kafka/schema-registries) | How to integrate for ClickPipes with a schema registry for schema management | +| [Reference](/integrations/clickpipes/kafka/reference) | Details supported formats, sources, delivery semantics, authentication and experimental features supported by Kafka ClickPipes | | [Best practices](/integrations/clickpipes/kafka/best-practices) | Details best practices to follow when working with Kafka ClickPipes | +| [Kafka ClickPipes FAQ](/integrations/clickpipes/kafka/faq) | Frequently asked questions about ClickPipes for Kafka | \ No newline at end of file diff --git a/gt-lock.json b/gt-lock.json index 1f746dd8d5f..49b47ab4d38 100644 --- a/gt-lock.json +++ b/gt-lock.json @@ -35,161 +35,172 @@ "jp": { "updatedAt": "2025-11-26T07:24:12.065Z" } + }, + "89619e87afb3075fb8d7e908131d96b27f2ee4544b802e747e4ac0033cbb7775": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.099Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.100Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.101Z" + } } }, "24bb0ca99917fdfda706556c75c640db16b12f966ea7bd58e1e9a8bdf4be5146": { "40c867ec4bd9ff53ca41f19ef2fb11bce1cd4d6f82211f50a350bacfd56350a1": { "jp": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.128Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.129Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.118Z" } } }, "2f81498e8b60c281ca710a3a25f611bf79424982fa85bce630e1d4182f252536": { "e5431d96bed4f0f93b507ffa84836d28b1d715ac31c199864a10370ec3b6f040": { "jp": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.107Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.078Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.115Z" } } }, "37e1e1dcfe884bd88e97aa22d6ed7fc14323b326449d12f0a5644f15bd4ba087": { "bd75344d33495d82bb1ddbeeb77d5b1f53a6ecb5f788cb9eadaa606a67b5ba96": { "jp": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.125Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.125Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.130Z" } } }, "49041ac358e6a0f1cdae73923da607add5f9d37fe3320250b5457924d09bcecc": { "d61c6739096f5de9a1f340500324926cc206fe878ab16df77def05d0ba746d3c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.666Z" + "updatedAt": "2025-11-29T02:44:55.130Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.126Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.120Z" } } }, "4abc97ebd23c7b3dacc0e18e77499272b51b908bd0c2a7a823d153d3c00f7613": { "7817d141aff4e4b1ceaca87c554c551bc1add23bd534611e2704fba56223fbfe": { "jp": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.125Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.128Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.119Z" } } }, "4e7333f7ff430819ccfae5b1f2b2ee97508f58db11c3e67c31430385b0618503": { "1a899ad20af5d3dc3c495e6ddc0c3ff5aacc9df838675e487a6910da0a531675": { "jp": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.105Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.115Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.113Z" } } }, "6f13745927dfcaff0a5b759cdfc9dc47aba26e811ab26776ee363cd821f7d585": { "be6c5629590606c77cd44d60b8cb153a6e8b1ae6d9f710967b3ea692cfc8cb6d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.123Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.120Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.121Z" } } }, "8ad40f5399ed36401edb12df869b1d441ff2d635581938c63d4f0a611fb977ae": { "16565c6a0928275a3a601a45f18823227dc886a00aad5531244bec633d3e8af4": { "jp": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.154Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.155Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.117Z" } } }, "a3ea3f0c344313a1d3ad7969f1c82ef13af419e6eec98da91153c8735fd46730": { "df3510130e5bdcdacd162718bb228e62987c548fea96f8a9e94123cc6b9a78d5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.106Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.106Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.105Z" } } }, "b4b7e3ea48cb57c88168d17bf4d4f7d74e58a613803386d3229332939508c542": { "67faf8569421939ba33d4c9fdc3b64f28fcc3bc298cc8c8b43a29bf3499a6898": { "jp": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.156Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.118Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.155Z" } } }, "bed5256b181dbcf92c02187749ebbf45c60b6bbfdee1789c1848984b6be1d78d": { "614647c380ff18e7b1672f19190809fcf15ba05429ff7f93a33f6c77255ba9ba": { "jp": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.122Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.129Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.121Z" } } }, "c2811557e4f56ffd6e37b0f9f6558971e9d45005c22c3c19ebaef586f1591687": { "b9aea39ae1b4e63fef7a92d27750dfc746ac0ac174e77a895050ed0d24ff1ea7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.079Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.109Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.104Z" } } }, @@ -218,403 +229,403 @@ }, "a4c073207b34a9e6e51079c57f0e06190c406d676367e982df527e7379cf105d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.096Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.098Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.097Z" } } }, "d59e7e7594ae44f151cb9c65dc0cf67dc9998af7e3a974cffc3d0f0dabce2e18": { "7f90a5a780c1bb26935f70fb9cdd36714ca975e36d84b530b0b75f565410ba0a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.108Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.110Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.115Z" } } }, "d89bc73ed23da882f0c45593180a3989cb6844bd38d6496ab6cb5ab328d51083": { "42fe50c1e729beb1bfa14d29e80c4f579a068ebbfa39aa1ffe25b2bb963a815a": { "jp": { - "updatedAt": "2025-11-26T07:24:02.672Z" + "updatedAt": "2025-11-29T02:44:55.159Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.158Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.156Z" } } }, "e8ae18c3678baf91b2255e5eab22effc78193c605230851316718cfb95063b2c": { "b8eaf5b30dc66a5bf4e27198f07863a95cd60a2e8b15d9fe7e86cc6f6eb603a7": { "jp": { - "updatedAt": "2025-11-26T07:24:02.672Z" + "updatedAt": "2025-11-29T02:44:55.158Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.156Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.157Z" } } }, "e92405c74b1c19a280775296a5640f2c7646bfabd9d6af48d6359d9a4f09c9d8": { "c9015dfa533bb72f0fe4f1f5a455b0a5497c12b645e908ee88d9686adff07027": { "jp": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.124Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.121Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.666Z" + "updatedAt": "2025-11-29T02:44:55.130Z" } } }, "ea04f6329e37f5487414c9b64a5e1602d705f1fc914807a5e16d95932f4ded16": { "c2794c8cfb2c5d8f3ad408c1a6ee6d92accd0948ff2682cca78897d7cef83daf": { "jp": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.108Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.111Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.109Z" } } }, "ed828a4311942b25614d0fd962b572a6dc329c0d92a3891dce42290c1d8324f1": { "78977a9c19b7aa2ba08361a0d6ca3390d032f6997a67d280a40d8974f768bb52": { "jp": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.116Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.065Z" + "updatedAt": "2025-11-29T02:44:55.104Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.067Z" + "updatedAt": "2025-11-29T02:44:55.116Z" } } }, "ef555d903b99c706a7fbc048a6888f3d3743693968bc76912f338d53af846b0c": { "c84825f7cf888bad7b7b5ec57d4a3941f8dc40c7526398600864fd18a77516ef": { "zh": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.124Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.126Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.664Z" + "updatedAt": "2025-11-29T02:44:55.123Z" } } }, "fb2a4bdb7f2883fa7ac9878a6d4e978def652c408e4ef95784547eef9e313dbb": { "20e4763f0f7057430907de10bf00a918aa2e762becf34af686b125a9da4fe458": { "jp": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.075Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.076Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.066Z" + "updatedAt": "2025-11-29T02:44:55.110Z" } } }, "22760d417a52c66f14bee182587d458d0737a616dd14cb09748b4c725fc5809f": { "c6ca08107fa6822548ad3adc5de4b6fdf1d9860224c2cd62047f42bce72b1c12": { "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.323Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.323Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.350Z" } } }, "3e38c1623307fe1538f034436996c45b6ce42cebe6a35b146ba34a354e7b226a": { "7d9c49d88230712b6849bcab6640651373295cad7888223291eb46da868626e3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.366Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.360Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.359Z" } } }, "434f73c99193146064105db19ce337122de0d78915918472d56f5393dc41a913": { "07acf0a2f2bf2cdedbe6696ce78f98b197df5722eecc5a214cf1d15173619bb2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.383Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.351Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.352Z" } } }, "4fa6a5d68016ad855e418c2e88b5a37793256913a0caceaf33014edf61107509": { "1ed9748c6ebe33e1898f694a866a318e321540cc9186ac29b7621da0715118c5": { "jp": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.367Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.369Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.357Z" } } }, "5dcaaaf5a4d53dc33da6680731d152b4a88a8d4e9c6058a18e521c7629865fb2": { "11c49d7827257644d730176fb691cb3d9705b0b2caafb1ee0ef7b70e70446275": { "jp": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.343Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.345Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.346Z" } } }, "6687b017941230cd05da5ec3d3b0b3a66c8da07927fdb43f62a2732581460749": { "df9979ccd3ace6a4ab1b704d9d4b233c9092cf3e747331f0d940179f918f015b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.163Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.672Z" + "updatedAt": "2025-11-29T02:44:55.160Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.162Z" } } }, "69aa1e22d1867f2dd9082998e597234169f92ed3ba4c3d6af26b34ffa82e4a48": { "aea97333102d80bfe523bef5b3932706938c1ab2307337cf20451a0633f0d7a0": { "jp": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.354Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.351Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.344Z" } } }, "78ae413a62554c8c5ae5ac8301b68726066573d500bb6c8caabdecefd781bb3f": { "754657766dba43bf89b81e0a5c15e318411e3d1782280b5ae5d185edc97b8e9b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.322Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.322Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.350Z" } } }, "7c0a1f3dadfd423b5f66993109c1e8bde0a0b4ff555067d9e6cd275bdaa7a391": { "ef65d65a01188f23ada0aa6a4be2fd11257542de621c6ad17c666a4b0b2aabf4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.380Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.371Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.372Z" } } }, "8d9acd4ed372d08f28519dfb01b6900545df9f42502ac21f0ef6bd86b724c724": { "3ca361084040c6efbaef261b3b4c88e38d022539f3e58645a4023be45b9ed7f2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.369Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.358Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.352Z" } } }, "902fba8b39d6b163ee66698d8bd12433740962a53ec93d756ebdc9d11cc5c531": { "dc72366dcf698c0d7f7b5eed229fd9a7dbb9776362cc9399cf927769376a9098": { "jp": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.343Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.345Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.344Z" } } }, "9414d7ed4d45271d4677e8083e81b7732a750c4dcd8dc182b14ce11749a9ec63": { "55348a4fc23db936f15885093540b12ab2d7159c2a91617e4058fef961a3c4ea": { "jp": { - "updatedAt": "2025-11-26T07:24:02.672Z" + "updatedAt": "2025-11-29T02:44:55.161Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.672Z" + "updatedAt": "2025-11-29T02:44:55.160Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.162Z" } } }, "9ac02f1c2289520aeb58725683781053f5ba6bf828b2e8585540596060f1f416": { "ae1a2a308feb0c5c6d10c67047a4b5867fd643296e4e816743b7e2e297fa0f5b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.348Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.350Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.324Z" } } }, "9c7cf003973e27e4edaecd818b57b1e653cdfc8e936c45c67e314eb7123327be": { "1bca9e04eb1cf2d68b948ebb6ff7b813d50c8faada3f1ee2a8c561e9d96d6882": { "jp": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.163Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.164Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.665Z" + "updatedAt": "2025-11-29T02:44:55.127Z" } } }, "9d1a4cd9443c6b88ad09c86205631318b2554cff25df4445681bef27a63f1923": { "6349c4d8161b7c14d291e4b3a1c44b280c0eb9f067739c2cbeccc19c6800a2de": { "ru": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.371Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.379Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.368Z" } } }, "abe1e09771cc949a765515b0f1ae2d0c4a6ab90fceae133c7ea3efdc49fb65a6": { "3a86b5256b89e144630a5cab1fb1ee8cee76bb30374fd9a861dacc440a7b8bd9": { "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.324Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.321Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.321Z" } } }, "ad7599fbe857ed33f8231ed240a179e73c2e77cfa5e658ffca7502e66d2eeb8d": { "fadc1396d5e8d2ef81e08c49dd8a08b01468ff70c1b1463a692904b2403b88dc": { "jp": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.346Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.348Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.347Z" } } }, "b299c52ac5ef4f3399408a89f700027a4da61547b988cd85ef190b1cd544d809": { "ab3b1379019677d4f056b27b40d79c7a1d368792ecee4e8e04d9224e7f40f825": { "jp": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.364Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.356Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.364Z" } } }, "b776eb4f7e91ceadeb1cd902e4a72be31912c8f40357421634f01d720427d7cf": { "a157eb7d7ffd46e8626bd3b8ed555fd32deed480e675bf80cec9536c2cc53b70": { "jp": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.359Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.353Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.367Z" } } }, "d294fb78df318396bfabb26180e94ed0286f348799a54a338dbcac4df2d501a8": { "2c1ad0e8f79ff31317243d7b0ba63abc05a794bb4cf50ddf3ab6a05a73136433": { "jp": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.349Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.906Z" + "updatedAt": "2025-11-29T02:44:55.347Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.907Z" + "updatedAt": "2025-11-29T02:44:55.348Z" } } }, "fb88079d51f44c0605f104c400878c73b1676f5d7360de0915e1f533962516d7": { "83b74506a046cca4bef2d9a75f263d66bc8cbdf6902a726a083fb24ba240c90a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.372Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.361Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.356Z" } } }, "08c0c301774aaa88b81ec6aa095f55e7824eafa1cbace5b623dc7c79a65127d2": { "69fd950d01a73a4628cd2ff26fd88bc864432af7ec9c2a0b214e105e41696130": { "jp": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.355Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.368Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.358Z" } } }, @@ -632,299 +643,299 @@ }, "9c5e775eb155a68364a368b61b5125ad1624928bd6d74c078e880457d501c280": { "jp": { - "updatedAt": "2025-11-26T07:24:02.707Z" + "updatedAt": "2025-11-29T02:44:55.394Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.707Z" + "updatedAt": "2025-11-29T02:44:55.393Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.707Z" + "updatedAt": "2025-11-29T02:44:55.394Z" } } }, "3a3e4cf73cd863c0103607437eb8b4f6836337cfd7e83bdd562015c4ed9cdd6d": { "086e3e89b5951923ddf12df84d937ba158991125876b5f6d842de358bbe8b3fe": { "jp": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.398Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.398Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.401Z" } } }, "4603feefc1d42187f8cdc33b837b161e0fc3b5c8376d6cd191411f0dd562ad31": { "a1d7bce3db202c9b4687ae92810df1230088ea23aa2c8009af476b02206e65cc": { "jp": { - "updatedAt": "2025-11-26T07:24:18.915Z" + "updatedAt": "2025-11-29T02:44:55.386Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.707Z" + "updatedAt": "2025-11-29T02:44:55.393Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.388Z" } } }, "490be0352814516ee6591ee5f8e07875e2139020d864d540140e0fa494298d5d": { "d23d41d10643691da14255ad0f85c7b97475432325af1c17be68df9efc12be5a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.382Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.382Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.381Z" } } }, "55b28fab1ba94c3606d033461bcc70376b43d613080d008d80ef6eeee311b377": { "256a3209f20639b3de6006d270d351fa95df57bd7f581ffda6773fd8eba690c7": { "jp": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.411Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.397Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.398Z" } } }, "617059ab9b90c50e356730de729f0ae69ee3763a1e279dd764ff91a7fb180dcc": { "d57355b7ce2374ff50888d99d345884771d8478a28a50565e264c7183444541e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.413Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.402Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.400Z" } } }, "752a92de3795a78c42039024533716b1bebd226dc5c16f6d9e6c32db92868aa9": { "3a70208f4d63a66f6cafa72e823a27c94a0b217c643d65060e75846cf03db29d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.417Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.423Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.424Z" } } }, "7c8202b183dd3bd51127bf5cff1d877fc101a710d10076050d3769cec7237315": { "cce8610caf1b6ee18be42bc4b4573a409a2178a60d7e7fdf9aa312bb9a0e96af": { "jp": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.402Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.365Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.370Z" } } }, "7d8c9d047aa047d949a0099bf7badab51bf4cbb1242283616136def6a2087241": { "ae00c1636361dff35e6ca1fc517dd76ec664cbc4f992d5bcfebb7e2a76f626c4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.363Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.366Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.361Z" } } }, "828d49017ac2a72d1fb53055bb4787df9014bcdf6914a82ba88ded05b27ec9d4": { "9d68c2d46ac27369e5a5becf238948336518cad4fd978e7648cd41b1f743b1b1": { "jp": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.430Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.432Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.428Z" } } }, "9970d9a6d501f36cf179c0419231b9d795a4c633dddeb9b278e8ba7a601a3f30": { "5509618b18f9e3d905b42bcca3ca87b185e363a986c08a3c7adaa67ea9d4602e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.368Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.912Z" + "updatedAt": "2025-11-29T02:44:55.370Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.908Z" + "updatedAt": "2025-11-29T02:44:55.353Z" } } }, "a1bad3f4a716dc84c050e5be3e8486b6c74375173ac25b4b6faa1e07928f68dc": { "2ea331fabd4829ebc7e1af163a669bd7da7ebae75dc79796126ab275fd4d3c95": { "jp": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.358Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.357Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.913Z" + "updatedAt": "2025-11-29T02:44:55.372Z" } } }, "b5fa886fabf17ebc48a6ca47fc6a8992d00da4b99785793543e0d888695a2688": { "259e7cbd211ad2a2649e5a8f0da300650ca51664a447e45289d100bfcdfc34d1": { "jp": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.395Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.395Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.416Z" } } }, "c1a7d6a20f956b50b1038cee0a820dd57189fc686d14660b023d1fc67ab2e1e9": { "dbc44ae26a03c1b8c3405262a6dd56a831c655163c2cd640d1e27879c8e4aead": { "jp": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.384Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.381Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.914Z" + "updatedAt": "2025-11-29T02:44:55.383Z" } } }, "c369c0aa928f8264daf73b2cb8b5d20b0f760cd84c596ca63fb6e80bf182b3ac": { "081e5ae543866b5886ecf7decd8d4a80af7f854626b8b8136631cf04a6c7a9f8": { "jp": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.404Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.399Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.399Z" } } }, "c6c433287e0c8c560d8ccfdb9dab1b764948e7aad08d8083787ea5a2ba4ffa25": { "3fa66f5214cb83c0d151b0adefad829fdec772c62100ad8be67b2c2d29a51136": { "jp": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.426Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.412Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.423Z" } } }, "cee9dba64ce2a735e188505d45af71b74c5cd69ece6c6e7512832d84898157a2": { "ba71fb39dc5dad8ed0cc9385af226ee8ccfe87b891afdfae44d4b68d6a6800ce": { "jp": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.401Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.396Z" } } }, "f03ea3286759068addb766b5b98317ea84803343105fd081b75322828bf9d201": { "8049194481456bef5558bf7d7d6cc3b522680055cc050dd06c21001990efaa95": { "jp": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.360Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.354Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.364Z" } } }, "f820ad66299aa0044ecdcc3298f5727903d52ea9ce19686054f70d9df707a8ec": { "1c6d8e151f574eb1c808a7932e470939d01ddf3adbd9a088012790d70765d510": { "jp": { - "updatedAt": "2025-11-26T07:24:18.910Z" + "updatedAt": "2025-11-29T02:44:55.361Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.363Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.355Z" } } }, "fe8dc3e8a42089566aa7dbdc1b163232b94dab755cd8f716e5050a20b9da71be": { "8e5a24d4923c146d3ff29a36c8d08b801a6681568d413d11ee21ab25c5a588ff": { "jp": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.362Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.909Z" + "updatedAt": "2025-11-29T02:44:55.355Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.911Z" + "updatedAt": "2025-11-29T02:44:55.362Z" } } }, "1468ae293b5d12d0ded8668dbb023988cbdb44ac496923a1ef6653864352d921": { "99c4f7270820d4fdcb92c4d24d5487f3eaa377c46e721e913d45645dba75a74f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.447Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.448Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.447Z" } } }, "1c112808a954d78a709e3ae05703950bc5804f9e55e3e98efd93efb0f0f879e0": { "a0ef058ccb99a1b138a4a98ffca0037cb2b496f227c55108b8beef337ba82d66": { "jp": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.422Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.404Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.403Z" } } }, "2317505b4b1b1557458b6ec9caf09937e43cf133543d04e2637e9cd6e0693bc2": { "8b6d58a1ca1a770a40180a524a20350aef1a747a1a0f59ef6bd9eb53764a7d1b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.449Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.446Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.448Z" } } }, @@ -939,694 +950,702 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.083Z" } + }, + "8f390179712abdfde1d16a03f079c6ebbbd781d3f020e59b2ef3af3be4bb0205": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.379Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.387Z" + } } }, "3371d95238c92603c162eaed8138395ca44e47b22ad969c5099f7e599ec16c22": { "2a161bba41a266518443feea5a759cf299dbc3fdeb7b00fd74b546abae68dff0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.425Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.427Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.424Z" } } }, "34148aef91a7ca42367acb2003b2045d6893d713fd20c6ef4a4a8fe6b505125c": { "0df15707cc19ce74ec40c00d884f8f77eb33786d03f5831e131804575fce02b5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.434Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.433Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.434Z" } } }, "5c385581f9c65edaaae75a74b6646a142de547cd3f20a408953b75ba33586e2c": { "8dc4eb869f4a048ed04d5883545cce095cb2df351eba54b486a29c615fe29cb3": { "jp": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.416Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.397Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.415Z" } } }, "650a560be33079fccc4800a89f7ceabf7333009b1cfb0105d0e77b22a9afd9c8": { "609636eeb62cf3a4bd5ce284becb34bda3b97c2382d2dfd89320e13d69bf22d7": { "jp": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.435Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.455Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.452Z" } } }, "66edaa5c8dc32a1f831593b8a49a8f90c9de66304dbe8e78969217a73f2200c0": { "3a20ac6682c2e8633f0f56d7c381698dc85b1777367c924c9a05d2c329c4fda0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.421Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.411Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.416Z" } } }, "67113cbc50d80beb99c25a836c1c97bf312030d10537561666f2d9afcf9f3145": { "bc5d1e200e64a767369cc0ffad68cd1dc62da9a6230b0c00c0c10c90dcbef298": { "jp": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.413Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.415Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.412Z" } } }, "6986025ddfdb6e69c9d68bae98e09599b7bd5252a433fe1c14839522e57376a7": { "6a07a797478a7c19aa592d19f3fd5211e2bae00db7fd3cef33b175016a1b1b29": { "jp": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.417Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.422Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.421Z" } } }, "8e1acaa9709e95b6354d4bb719b069fee08bc3794641756333aba5003eb9475d": { "e8f0f6277f744012426a53a6027257e33c4b16cb2ca45dda3d90d4b73b3d4c5b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.712Z" + "updatedAt": "2025-11-29T02:44:55.426Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.711Z" + "updatedAt": "2025-11-29T02:44:55.418Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.414Z" } } }, "a5aac8ce0e37bc2df7af5f69708607c2c9b46cbe068e3172847b3191394faffe": { "38d2828e9bd727652c3233af76ea089e954aba2db55328f8cf1f43ca609f19ff": { "jp": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.445Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.447Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.446Z" } } }, "af5c9aba153f2323766f5c2833f6dfb1a669b295e319d579f4546ea448e8d7e7": { "0d9634f2d0d51799480d3e5d225d816eb09fdf75e544bf3b04b2fe1385fb9619": { "zh": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.477Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.477Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.475Z" } } }, "b01e9e50dff0d52b1c86ddcce64d477f77a182599c27ebb6752763a0c4cf1884": { "4bf15471d437e48ecaf706869ad9127730c8b915f392e00ca4b38372ff596b01": { "jp": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.430Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.428Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.429Z" } } }, "b721aaf83ea7701a82587311ffcd215fa0fddd0ac9d459193fd26188e0680183": { "906c00a6ef80e7715d21aae24374b2b2d044fcdc7b9d5c6c2c7341ecd0753821": { "jp": { - "updatedAt": "2025-11-26T07:24:02.708Z" + "updatedAt": "2025-11-29T02:44:55.396Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.401Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.709Z" + "updatedAt": "2025-11-29T02:44:55.400Z" } } }, "de08ffcb57e92eb891276970020672bdbe190e2ad13861a7a5a14fe04f7eff24": { "b11091547782b23a3e69aa42aa789855dc525b51b00a033b1cffebdd4f69711f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.452Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.450Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.449Z" } } }, "ea2d038b6989e3982d873c583fb3c15212b691b2e747de62d4d28c3e4b11a23d": { "68f32501aba4af446aa28658a6859e797a66b66f975249f4a21ec435c8e2e471": { "jp": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.454Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.453Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.715Z" + "updatedAt": "2025-11-29T02:44:55.451Z" } } }, "f56b183aebaa9c102a1630d41b724bdd0ef7984c2f5be9f15f51bb83994e0265": { "0e4b6a498cb6259a81c3b89b57fc27d109c9f7c4517473e5f6371c0a4d14e7e7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.483Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.483Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.484Z" } } }, "f80606d0e748135944fda4f0b2bd5df8b58807fb2f4c06c85b06e12fca82e935": { "2aa54fbd8a8eef1da3872abeaa7ad8858d0e7a55684ee9afd514540bcb055f29": { "jp": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.433Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.431Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.713Z" + "updatedAt": "2025-11-29T02:44:55.431Z" } } }, "f90130006ab67f0f1f9729094d7e71d602684a6c03306792b40387ebeda24cbd": { "044f9d08748a2a48a556c183ed0bada874cc4ce848cad6b1bf87fba782fe7d9b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.476Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.471Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.473Z" } } }, "fff1cff77ce23873924a1766144be6a0a4bc145a4beaf1c7902459c008cbd536": { "6b16dc8b034758efca2a7dec7fe695e186e4ef2f750e4a6ba872d28a906012b3": { "jp": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.405Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.410Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.710Z" + "updatedAt": "2025-11-29T02:44:55.405Z" } } }, "0007ef5eb0fc8520aeab373a05b58e2db16ece5be3074e20646fd984e7bb2153": { "534ae688e369810666e881d18767610a7df7671083edd5debe450d3827e074c5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.513Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.470Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.516Z" } } }, "05fa5078290d9319b91e520b8d624cd018e97d963be0d0e1cd22ca7e37e899e9": { "4a4c4d4b7b75c17db47caabac407cb6678d38f795ad11e688adfe6762b928d79": { "jp": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.481Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.481Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.479Z" } } }, "16a9baec9aea4c6dd78355c05288783f630be08b0af1a257fb205b45c7adc066": { "b1a72f898456e3c08b49f6f0e73a4fc33fa3bad39fab513c1db89294a3fb923a": { "jp": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.513Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.508Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.510Z" } } }, "24203d0280dc684588776442ac330a354e834de5789e13b7f7068042627350dc": { "19fc846b48f319f018e4f670ace8976874e318a091bb09940eed158a6c8e8569": { "ru": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.592Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.594Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.583Z" } } }, "2fa693bc37b3a10adc8d79217e3b09168dc83b1d1e169414c8ff196815fec6f9": { "9e33b9e6995d58fab1e0c61f6a5436f2184d7c49af88577359d93f178ead07d6": { "jp": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.588Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.596Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.605Z" } } }, "437aab42be9088c95b44d049c562b541333adb34c7167b0341f25eeb6f1da633": { "673a9dec5d05173b117bf71c194bcbd9250ea1e8e6162c76a5ac07819b4a0314": { "jp": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.480Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.482Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.482Z" } } }, "4d14e175d2ad5b7f1f59197782ca672764811be0a7694da0d93c40a71707c218": { "2f6f3975ac07a17d2e6c12809f029b5fcecdc238f96cab5409c924b908db77fe": { "jp": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.510Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.511Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.512Z" } } }, "57e4e9dfa0451001fd8054b08c62e1b7e7899bf69d75440b300be4c4a727b99e": { "37f3dda8e8d9a3dd2ccbec3bdd564d2de4200f5a0108f14e3cb3cbe1f05fbe96": { "jp": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.515Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.511Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.514Z" } } }, "60e6b2c95bf2975a8ad16addf92ca2f2b8ef9b6f0267eedb1b1609cf83bd7bf0": { "8b24d0eebf933022f5b7646dbd76005a200ed0eb134c91ef2ce37429b92f838e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.547Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.548Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.546Z" } } }, "666059b00db591c1a56ce4963af6165fb3c9b12689bc7bd2d002ad9f8261acdb": { "60035e65e48fd5fdb3a14661c3ac4811bb8496f2b211e4fe284e3d6b420921c0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.593Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.601Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.602Z" } } }, "7431d11049418c30c908694305424392c5e608ecfdf0bd5bb5e82ff877dd01f3": { "3c1e299227977efd8ca6ccf93ac2673c11fbfdfe441a0d0784400200278822ac": { "jp": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.541Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.542Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.541Z" } } }, "7cad50f4cd617547f24613bf26b7d92863268b13a23a167f7afafe1105d9b80d": { "fc4b5c37a2e9cd403b127f9b0e95af107c0815b1c7bb98e1eebae04bc96ad554": { "jp": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.508Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.474Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.507Z" } } }, "8b1e7b5824a25229b63b6cae491572266d76a2f3619bbb37de99f10f9cb281d7": { "b39b1a9501a0d4efe97c7c462447f2f7f762c085e32781115e4e01abed9470bf": { "jp": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.516Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.517Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.515Z" } } }, "a3a1fbd31e0aaa187d657bd8045fa61bc9f31995880bcb5d5758a3e184f5ecb5": { "28ea6e40b848e91414d2d23698b6689414783f37c844f3f15e49942c2f8d0f73": { "jp": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.480Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.478Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.479Z" } } }, "adb57f6c330a361767cc8e018fdeac391e70be9310b007ddc867750c55383217": { "6bffe63c913aa6f222b1d3f7660678d89871583dfc5b85a5472e73ccd48f0852": { "jp": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.603Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.593Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.598Z" } } }, "b0f947d3a4638d92601c813f2511beb5008821e82e066594946d2230ae518888": { "e2d7964de87a21a4f56589f9ef750a5f70e553620f06ce8ed541c52c8e2fd182": { "jp": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.507Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.509Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.512Z" } } }, "b581e8a0971d1d07fd92c09611201fbc0ec1f2ad10e9a9e9462297b6dbe79f67": { "49ae124d0469e31fa1e3318ed468a02b4e75af99b0ad807441a4e18f29afb644": { "jp": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.506Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.718Z" + "updatedAt": "2025-11-29T02:44:55.509Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.473Z" } } }, "c5ec668978bc00da55afaed8bf387ab8e40f7e8cc8a5c3c85b6528469658dbac": { "2760c235bba120190e9792afc2791f4b14241f22634e6dcfd806b0f0c8a2f30f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.083Z" + "updatedAt": "2025-11-29T02:44:55.478Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.472Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.082Z" + "updatedAt": "2025-11-29T02:44:55.476Z" } } }, "d0878e46ea2d9748ef2ef35fa15820d74801d2e823a8c466520717410dca0e30": { "34f3e7285aa8956a7287c85c05fbbc6f82a3d73d51e58594a18dd7c4e673674b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.604Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.595Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.547Z" } } }, "d84d842f939c18587480808dae2c357d93b19f0503165ffbbb5df5723ed8d18f": { "78c6fc1825dfef395f2920f37ae3b83e7a55e08e381e14e11ade4b0633972ca7": { "jp": { - "updatedAt": "2025-11-26T07:24:02.719Z" + "updatedAt": "2025-11-29T02:44:55.512Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.474Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.475Z" } } }, "e16fa51bab7c52534a6634130d4aa9d5f4eaf5a9199be40465cc25c632091ca6": { "9a45c83991713cae83ff2b9ff52e3fac9bc7cf89dc4ce06aee3062459ba62f83": { "jp": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.594Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.588Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.589Z" } } }, "00385c907824ee916e1d2ab90ec1343952049a30fbb273cd705e54e19e5e54dd": { "a1e228059158c6496d116286e96a0ffb78b193d02679d41dffd889c4ae3f4ae5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.606Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.606Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.606Z" } } }, "2026a346cf904938db3b958bccd4a5998e0f9c3e806206b6a7de6c5a43e41346": { "99e01b88c76b26cea06cf6daf392581a33f358c37c5d4b5081a274912cfb4fdd": { "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.583Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.586Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.578Z" } } }, "2283119a59e486c7e332715c4be76c78e6606cc8fef66284fa0397e91f6e9842": { "89e926971a9cb3deeda49f638cbf8679ad56a009190bf99db1a5f7d3b55c106e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.545Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.584Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.545Z" } } }, "3b5e4827235bde4cab69ea0d512c4769c70579291411c713544bf464dec162c8": { "e5ffb2aae3eda69d46997485801b157c3e85f0837446fbd682ac417320b69197": { "jp": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.595Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.586Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.587Z" } } }, "51a4e1d93b002b635941f3a0b969d77f5e76ffcf3ab01cc6c0302553a48f2dea": { "e3cf07cdc5c67cae3f9a9be2ea541fbdda42c2a33f509a3d16926cfb4c4fa296": { "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.582Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.543Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.578Z" } } }, "51ffd052b5e18acec3f8c2fc6fc9f2de6d509c5f9b55c4e653df085e2f4cce96": { "e67a2d890c9d442e3c7a7f02a0d5c6afcdb1928ff906f575bbf304c7f7799b2f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.604Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.597Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.605Z" } } }, "58660987b73352ad4963dda3033196dbfd0c791f7ea7184da7b8ed72a70d23c7": { "e6384b2ee9b82af275d9a7823132ca573a701a7955a267deaca2eba7848c0139": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.624Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.601Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.600Z" } } }, "5ae00fffd365a54fbda628a19a927576375cc455c591c16a26e7ed16b919a10f": { "2c1fe0f08e90b42f0362e7d55eb555bccf6bc9522b4eee5aa410eecb5a6ff63a": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.627Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.629Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.624Z" } } }, "62a28a91cc967d2076cb4a8ae68eb32bb7dc0a91eac1089fc166676f54731dc3": { "4fb613d98fb6ff221944b46d4a102b8b41af0362055b5e31a68dcbedb5e8be6b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.581Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.546Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.592Z" } } }, "62cac186a0d5d595a384019a8da0f2587e8ec388e9fa723441881ad21746e53e": { "5315f9a99c66f3565ee182e7d8faf811aa2e4a227524f9f573eb826dc8b5c51e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.585Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.599Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.584Z" } } }, "64c029683442a95f0d9971d2c2a2f011b21167a916369b96ea20390f74a96eb2": { "27ea13a9d6a87686196565d791a629223843e1c311b9bff9edf44c593e511703": { "jp": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.632Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.630Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.618Z" } } }, "812c31122c49b26a28f2af399b63cac7fdb8dbff9b0eccb1a55146b1f53d9141": { "1ebe27a88b5652f04a87609b29cf3e09b5dc4ad9bfe9681936296ff736f2d7ce": { "jp": { - "updatedAt": "2025-11-26T07:24:02.734Z" + "updatedAt": "2025-11-29T02:44:55.591Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.599Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.597Z" } } }, "8aa6821981ce9839d00fc14d757392848b9750acc4bf8539c334cf2d5871f908": { "a27ad75b9e2993bcfc4ac7d0eda9c06a190e908e4e85725e849767c67999764d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.598Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.602Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.589Z" } } }, "8f6142d5329a13cb865837bf5f90f1676c0ed34132ae0b7413c66ad9fee106c2": { "b5efc55478dd9c26c80dffe9ed741b395f4d2368d8eee6c9c3149cd4fc4eebc1": { "jp": { - "updatedAt": "2025-11-26T07:24:02.735Z" + "updatedAt": "2025-11-29T02:44:55.600Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.590Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.736Z" + "updatedAt": "2025-11-29T02:44:55.603Z" } } }, @@ -1646,494 +1665,494 @@ "afd2f2eebd8416c23bdeb683cdf48c7d32f86769fb59accaa3e0399bedfbc689": { "9b1791199c987e23d27abeedfa5722370720553cfd8a6405ee7112cebcc27c6d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.580Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.544Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.585Z" } } }, "b353f551e48bc3b4c88a7db0d857fefd25c028f8d05216430afdb76e3bd832b4": { "6d6603c2d993968e3e2fb68963df1f14bb64c291c769f84522294cc56cd80d73": { "jp": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.581Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.732Z" + "updatedAt": "2025-11-29T02:44:55.582Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.591Z" } } }, "ba6c4ca640fe7b3f714cda5b21aa83f56d6987a93c06b0f52403fcf16442d4a3": { "73a0749a7a37be27b2b679011c93ceeaf5407fff6130ef17dcbbbc612aee0d5f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.631Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.623Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.630Z" } } }, "c85686859f3f25046db0082f882182fadaaa53c9674e2b8421280d74f206eb40": { "add68d9d7c2384a1f4236b30131c64724392237b73f94a4430f8fd215046f46f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.624Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.635Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.633Z" } } }, "e58beba1ecf7893bfe1389d8eb8c6388801ea9f76c74eaadcbaa400a86832dc0": { "80e13888b6bfca7d175470bafcc2e30a1e88dcbbdaa15cac209fa66c4f44bddb": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.627Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.625Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.628Z" } } }, "f437d5d62e24e71773573d12295d6070b2013b4f10635e752fc5e0c0c6f3d5b6": { "69df1b4df06653852e7ced5d6197d910291dedd2d1b27599cd5608fd1b4a5214": { "jp": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.543Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.731Z" + "updatedAt": "2025-11-29T02:44:55.579Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.733Z" + "updatedAt": "2025-11-29T02:44:55.587Z" } } }, "03f61172ad909b158589d51b6d4f89a053de0b09127cf415c34413087bd48c4b": { "a5c008c72acddb7fec319268bb5dce0d0fb9a1f10d18c2c90a95d993d9f5a960": { "jp": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.574Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.637Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.577Z" } } }, "0b7dab5f7a039f1859e3a70738566e228a8859b0025e472a76cd8fa2c67c6c28": { "1d8df38a053cb69ce2a27d4691e5cdfd13a6b160e9a02fa3f683e748d317ea48": { "jp": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.632Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.636Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.616Z" } } }, "2864b967eeeaa7eaa2100c52550f0c77a534e954059ecfcc0991f21bc889bda3": { "feaa20d52a8757a137658d5066422bdbf2de0a87efa96000934a084ad78bfddf": { "jp": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.637Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.576Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.575Z" } } }, "36663ad730f89d83d4a65b5956ac48db373b0bcfbd0f2bb4062dc5f3bcaf2839": { "8841bb2bfdc1346e286a40346e8503829d958b3bac30b715d775b50f451b49ee": { "jp": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.613Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.612Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.612Z" } } }, "374986e8dd5ccd248058ea18a5c0798d535a4a7501a33eff5fd9b80a782b7c15": { "7b0998df0969746e6c19524cb961e7ff6d7e59afe83c51976450a953fc8b3ffa": { "jp": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.574Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.611Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.577Z" } } }, "3cb23211e097156c0f1a78ad405746a39a30a7fca3e113e221a2bbde60fc5c66": { "30bc5b33601dc47abebcade817fd66b12ac5351751c6ed875945668d80c959b2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.619Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.629Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.617Z" } } }, "6ce48a90c46614816b7c3a238012b7692f39fa7b3d52104f4f0f92d895004b22": { "7e344ba2b2f6753012aae6adc6fcc5f046670439fd5badb29bee696648c4a317": { "jp": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.671Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.671Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.650Z" } } }, "707c0fedc72655fa1c912bcb76b320d66a9ab9c8fe5e939a4df2863fdd7f82b8": { "7c543d5ef5d836f674d6873f133f02c4ab70829715b347650c196ee93273deae": { "zh": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.648Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.608Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.608Z" } } }, "730dcd6bd51a2d8afa76fc973bedd9b4d7162629dcf690b192df4cac1fc39566": { "ed51d6c3026594d0ef90de441bf36dff57ad4a32048a288a0186952eb2f80596": { "jp": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.654Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.666Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.607Z" } } }, "8c4b511502097e5142007ba6bf89d86ef9d582ca174f395180742175d5bd4f05": { "f3274830262e5f01f74d8474761446b9f8a9c83ae245d4cee233a6cd17284b39": { "jp": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.631Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.619Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.628Z" } } }, "8eeb2d38e63485d3f399d528cce00b3fa0310df2d513c8b5aed0077ee217c69c": { "87d6c2b8c54e666cd98b21f88f6b978a41ee92fbde390f5a595aae7d2c59164f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.669Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.669Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.607Z" } } }, "8efa94c2eaa8cf8e3ca888069c1494fbfe5679752549f9d0a41d641f2aad43da": { "481fbe7fef11ec970a0109b0e44e9a8165cf0e73e56a0466f038d0efcf74f657": { "jp": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.672Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.647Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.672Z" } } }, "94747a3cb7498dd41f7f7aaed2f670f003087b3543cf7752be3b39b62c021927": { "f7bca2db0af5de7e2c67ebc1c65c226c309288e7f073d34318c2747b6d1e9327": { "jp": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.614Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.616Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.616Z" } } }, "9be36d6e2bdbfee1f50c6de39175a6e538f2d986429211ef53b12ab0e0031ef0": { "1dee3abbec10bfa0b3995067899a721e47f20ee051715db74e0ac726fa434d54": { "jp": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.576Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.638Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.640Z" } } }, "ba0db243d349404c81abcb5ac1b3df54c29742957ec4ab33b24830ddab68f7a2": { "1f879e7772ed8e095b07f85578bd401df3a64cd4e5498296092756cccd875121": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.626Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.615Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.617Z" } } }, "cb8f8c1219ce7a92277d5329ae659c90b78edb06139fda7cb67e9143f6a4f1a8": { "708faeaebbf5c4dabd6c9a9eb715cafd5178cbb6ceacc376b982a574ba6496b0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.573Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.737Z" + "updatedAt": "2025-11-29T02:44:55.575Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.613Z" } } }, "cf42c21f80f60055d0087c0e795d8976b1d91223e0fe30f342746b23878b6c6d": { "6d3f845905f3f2b2a1be610957281c22628e8585866ee195f1e005cecbd69e88": { "jp": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.634Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.640Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.636Z" } } }, "d0e7cc516637ef8ff263a061c7c16bafdf014cfae7ce60448c7e0fcce8c6dfd7": { "e57a30777e558c8d76cfdd0c7355a7d8d9e150e93787b8eaedcd4f95150f489f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.665Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.608Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.647Z" } } }, "dc560181da04dee98b254f616102cfdbf1969c4c794080bd3b5dd88e33f63287": { "f7b3da6309249ba57146453a20fb02f1da444cc9f6b9ff15796e49d19986d9d8": { "jp": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.634Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.741Z" + "updatedAt": "2025-11-29T02:44:55.633Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.618Z" } } }, "e390b76711ccf2a7eb5d962d037354b40ec5f4bd6b5e88c7a59d4fe98d2af88f": { "959a1807df034b8088bb146f4b89e2d5ea2dea86233fa18c9a28c35bbea95719": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.628Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.638Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.638Z" } } }, "f362b87c61313b355b28fda5f77765651cb599066809f44030b3c1010865fa5c": { "498198cf31ab4d64e31b4a2d37da8c4597bed364756e0feb2aad51f2859ac1fb": { "jp": { - "updatedAt": "2025-11-26T07:24:02.740Z" + "updatedAt": "2025-11-29T02:44:55.626Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.739Z" + "updatedAt": "2025-11-29T02:44:55.615Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.738Z" + "updatedAt": "2025-11-29T02:44:55.614Z" } } }, "f5a9bb73dfebbd60d3ebe96194e16c204bbf24a1a4ad7b46bb262a754dac54b2": { "e78c91e1856bb6bb61d73c20e01d2f69ad12b8495c3f0d7fef84e1558681ea40": { "jp": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.639Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.639Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.742Z" + "updatedAt": "2025-11-29T02:44:55.635Z" } } }, "1370f12b87482a2e8d057a8b41e9ea94795da80127f778fde4628181bbdcc429": { "f8146d175696fd61b1124db8aa052124a23329de9472ab05df373240407f0ecd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.683Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.643Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.695Z" } } }, "25e58c45c99cdd21fc20a817b3bc1c4d1448cfd9024cc4ed56ae9462032d790b": { "6bb9f7de8fea38f23bfe3fefc31fa9cf8d67d55bb09bb2f9a1806c8d39795f52": { "jp": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.656Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.663Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.664Z" } } }, "2f8b276c8b17327a8cd29e02e2e0215fcd7745a9618f81983784ed62a0124e22": { "fd3cd87bae11809fdfbaa52ebbe4afbe61f3cc9cca3ae69d12c598a97c478f9e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.667Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.610Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.609Z" } } }, "376f1f3d79070d024492b0852fcc46275cc6894795ef5c9378fe6a8039d04b64": { "57d1e9d86f14ce94f3b9606be0c45891a1cddf024b0cd28892082e2bebf224ff": { "jp": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.695Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.697Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.680Z" } } }, "3b20b82fd209471b97a1109eecaadcd504d69c6421631143f81852d506036bfa": { "deab720ce649678d8772ed32576b254176937947561eccdb5dd266ddcf5b5d50": { "jp": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.643Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.696Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.691Z" } } }, "5315751710a23b80f9bf1ed7f31831d089dbe93c3c8fb90d20b7744073d0bf57": { "a66560c3d607504cdffd12261e02d0e673e576056f78a84ca9ecdf329603c56d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.682Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.699Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.698Z" } } }, "8d92e8b825b034ea42c644cd23567811b46adb33b6d540b842b64c0196ff3b53": { "292f22bc13c3bd83386dc5ae82bec9ed457e6f79b25efab444ce03844d88e825": { "jp": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.645Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.652Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.651Z" } } }, "9845c4be459de6543c79bb52ebef31089a7b6dde5c4bcbf294e6b614cb8b73ef": { "f7ab2f792dc532d79e54d2172ab842ea8bb45d24fbea3c48d921219d21bb9a5d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.648Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.610Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.654Z" } } }, "994f995f28518f9032c149f031eb9e817c8a85f3b0139de3abda3966eec97f40": { "0299673d875da310e70873db6a17323b8be0705c8b4b17c562c9e797b225acf4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.670Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.667Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.668Z" } } }, "9e812084882765188d8e23b9cfcbf9a3edeb29e9461a1cec110df416342b0289": { "e16e8324972fb51ec759f18c31f84b12438b5b468badc9732e3a35eecb40c277": { "jp": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.679Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.696Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.694Z" } } }, @@ -2148,83 +2167,94 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.089Z" } + }, + "c1ad37c48321ab74d0fa77243447624e6c987f44ca3469a08d251c3e5a869de0": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.659Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.660Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.661Z" + } } }, "a9a515c52dba44d2cbab844922d2f769a5af11a34775d83c1bd8d9c97e4bb6f3": { "85a2a4117446131c96b792674a9cf5594566bfe0b7f1098d2210537e80d0fb0d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.691Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.692Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.693Z" } } }, "af360983b516284a69f939f103f1882eaf99d33139f9033142ae3561946f32c7": { "33be4cf9c98cef60c81c9a896da5f27cad1a7e71f69e85818494ce4b7ec03b2b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.664Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.663Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.657Z" } } }, "b204fb8610ce0fe2a5504ac8ae74eb658b2c80f1a1d885dc2b85d71bc34129bb": { "0aee55116dc7c452f61e8eb411e60595d3f877d5ebfa1d1c034f028155bf44bd": { "jp": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.644Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.666Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.651Z" } } }, "bf09040d678e6760987c57861f7d46d0c83dc84b582441fa87e7ac9756c76f6b": { "ee66bac04fe1df0381e777810c8adb5c9d16229f663ce7ef30a2a0506899ac5c": { "ru": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.692Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.694Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.696Z" } } }, "c50d8bd0ecc6ee24b7f928b73255956cae71fabfe25096539cdb974c7f167191": { "f1fb2f5d8ab4009a1d0458d1d0604ea822a372927443fb49fae37168711e0dc8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.700Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.698Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.697Z" } } }, "cabd7d221b503f016f6d425976074155c6ab65f9918739e83cc1d703e06ce9c9": { "7ca705d224c1a2bae3bf661451d8d9ee2d0404abce117b56dcde2161981ea1cb": { "jp": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.655Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.652Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.747Z" + "updatedAt": "2025-11-29T02:44:55.655Z" } } }, @@ -2239,1630 +2269,1641 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.090Z" } + }, + "ef2a79c4910a450c4d299109576b26b3e4d3c1f0d7cbf8aec0cb3af68cf84848": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.658Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.659Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.660Z" + } } }, "d6382580d57e06e3adb4f63249975c1e63e439afb1528089085bb16be9e0bfd5": { "e66f44bf486dac3ec176125edd9a39b1b6741ccec945cdd42e270d60579a2194": { "jp": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.668Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.645Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.670Z" } } }, "dcbbbc894548f52a28f1dbe2761c66552c70c361ecde98f969015dcee3764a48": { "626e208c3631b5c7c63439845c92c76d534c35cdc0c557b51aac33578683ffb8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.693Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.644Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.679Z" } } }, "e3d2222cd9a6fac9acbf681cd3446dfd1fc4c2866524e4195206634d0d76acc6": { "7dd41862d4380d06fce8d5aee44728bdd5365a42f0ef1ef5d0a91b55cde5c29f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.662Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.662Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.657Z" } } }, "02291322e0d8f494b80f9d8e9c482282d8b548c6ae56afa37d022b164b99b054": { "14c2feb63b9f987820db166804e40ef404c44c5a695f647c2463bc6c7919d96e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.720Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.718Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.721Z" } } }, "13dade465ba8d6e7014eb44c3923f9c834a481123903922ddf6e33bb4ee775db": { "d6e6aa07741897774555a1f0eac0954dd331322344f830c9f304dbdca6fc532c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.731Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.732Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.733Z" } } }, "1e6a9268be90fc10ba5ab851817ae61b0167c3ce6990f2a5d9ebdb1ee6eec11d": { "986717639b58978e5f1cc565ca5bcaef17e4babedbaaace23e272cc8c401372c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.732Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.729Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.730Z" } } }, "290372a9e8da36b9b0dbc38f3a77bf8307b785738d5ba00a31fddfd12681d63a": { "435164419830229ab742e3ae11858464c9c8878bcf4a2bb3d6166ec4642f545e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.734Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.734Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.729Z" } } }, "2aca9c20ab8bbeb98fd4fbb9f62e8ae777bccbfb6417cbddb786afea95af4499": { "866097183364ceafca0351ea2755c4d597ff856cbd6ea28946d96c0d30d28ff7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.701Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.701Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.699Z" } } }, "381da73f1de48015917a484d7c2e45bb2557d1a326b8ff4560cb55a72d1de6ce": { "58f15d2dfce6c37907b066f99ba2b6b1bad2cefdd56e52bb79e7839fed922624": { "jp": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.730Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.725Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.717Z" } } }, "40b25bc5f9906b1b6c1e3fb64539dfc6d270a427153142c668cd16a039ebcb00": { "957d995119871468184ae861bc8fb42689e205013b5a1a037710ce22110de58f": { "jp": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.726Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.725Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.722Z" } } }, "52853976e012785457c250daee6b0280f9e8e88fcbc6a4a02eaf7315f2758fc9": { "35936f5dd5b5ed9baf260d39b24862296fecf4c8c909f41e2a0999a8db0a3772": { "jp": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.719Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.719Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.728Z" } } }, "5a2a174332bfb9a0cdf7cfe65d8e91568153937327d15d632b2c09aba2aba728": { "e8ae2af14396db3064dca28b82c864d44d320c9ce456d8e334f9b158902bf4fe": { "jp": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.716Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.728Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.724Z" } } }, "5f3d913c7a8c4ceda7fa835ce07f7441c4f601788cc68210c993d3dda60335e4": { "758768db465ee7223ab470e594b649587b038bfaa85fe340aea1e4aa3c4bd92a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.714Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.713Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.711Z" } } }, "6312de56aa12f4e18450ee81ed026306d225a603f4e118425c63d475b267e36f": { "05a0d0bd2cfc6068766a3aeeefe69b1d78a4b4f120052aeaeddd00b384c6828c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.727Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.720Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.723Z" } } }, "6439efcca906a994e35faf64afc92947e6ce60d7db71c07200375b94c1ec08a0": { "b590592b2b9abba8d294cbb837fba6f0bf9ec95a8c5f2d979542c7f80d2cae21": { "jp": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.727Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.731Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.726Z" } } }, "645c6b21a98f21e01e529f7978413fd1fd62c80916d3b27f0533877e73361460": { "9190dd15a568419dc8f69602836e2291f52c2c494b8a21b5d535f8100ce666fd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.712Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.688Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.712Z" } } }, "81e55d728a63e9d9620a0aa9a0f3152c86d8f4228a2480791e9cad5a8de39a05": { "0a7dd0ec6b5989e1b77f3754697c20347971441c557b816d890bf2b9648ca561": { "jp": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.689Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.687Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.713Z" } } }, "99dad4c2046d97de9c9a10225dad41defe9ab46dd46ee1ebf18685fa87671a2e": { "06b367fa8b09d7fd9415ccb9f2fa0fb03df266adda026a80d2f81729bad14302": { "jp": { - "updatedAt": "2025-11-26T07:24:02.743Z" + "updatedAt": "2025-11-29T02:44:55.641Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.682Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.700Z" } } }, "9d3e2980fe828b01727089a5b229444dc083a28f187a3ec09ad16a7eb1eb6d78": { "27aa4e4f10c34b32aa528db752d7176b33e61894bc9750f14367f23ebacba5e8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.698Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.702Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.095Z" + "updatedAt": "2025-11-29T02:44:55.699Z" } } }, "c491de2fc423ab10dbad66f7c1ced660f15c3f92d3220edeb2ccd84ee9575232": { "6fd80c5323889b79422bdbfe9fd8a32fb4bc56870affd9f018e096ec38fde0cd": { "jp": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.733Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.735Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.920Z" + "updatedAt": "2025-11-29T02:44:55.734Z" } } }, "cd73972a4d037347d81b6309d5ebdd4973e65b4708a5a1c61e961a7e349f0783": { "9206b8172e5adaad17f8c6eb0cded1360735c838b0a3363c600dce6cc6abbcef": { "jp": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.690Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.686Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.687Z" } } }, "cd764deae766f8c6f6cfe6b576e74bb1f602bfacbb3821340a5850510d57a818": { "b6693ed657d428f4853a8fcd97aaa704f7a982e5e86c5fb8e5ce372b12c11e69": { "jp": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.735Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.736Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.736Z" } } }, "fd4807eb1e777b66ccc79335d7b822af7ba8bb6dcbbf18e3ae8c53f548f20928": { "455e4d7b70315644264125e3a1e3a329d14b945c29bd48454b379b5247f97bdd": { "jp": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.723Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.918Z" + "updatedAt": "2025-11-29T02:44:55.724Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.917Z" + "updatedAt": "2025-11-29T02:44:55.718Z" } } }, "fdc92b085b658968ee987d63323feb976df8a0ac7995cde6f13318c84abd0c59": { "7843455825f9c1828f408c376329311aba7d3c1e14b345e73ef9ad5b93e5b005": { "jp": { - "updatedAt": "2025-11-26T07:24:18.919Z" + "updatedAt": "2025-11-29T02:44:55.728Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.695Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.094Z" + "updatedAt": "2025-11-29T02:44:55.693Z" } } }, "07722b6c1b943ed20bf3ff6d22b1b257d5a8695ae4b4553850f4bd4af3c5d2c7": { "2dcd7f352db514064395ba3b8d67b798124850b8ab716d08d01b773649c588b0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.763Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.765Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.765Z" } } }, "1777f3ba683ace75433dd34468539a9e9d67ef87d9f67a65737e86954611e774": { "3acf5735b7405bf65c0204cd16078ddc21713f4e46ed2d0238fb8871eb19b84c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.748Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.709Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.747Z" } } }, "1d262ab176214afd2615461f6d7dcbc00bf893bd453c8cad2f9b625f5b34ed8e": { "2ba14b7281983a683a98e1fb919f7ee7317f7cf3b6fce775f1d12a76ea1e67e6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.743Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.753Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.708Z" } } }, "34dd8a3ad912132054af7846a7af1d6c6f27c8de9f83f63c9354d5a326b6a82c": { "8e8980f8eff31a76117d3215f17a1cba9a0ee6234c2cce918781f484742ac397": { "jp": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.748Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.751Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.759Z" } } }, "3e5df6c1938919084ef3c24cc3aa0a9f834e4dc5475270adb64943fc1f2c818e": { "a27fbee07ebfb6548a8a222874fceb3def1e176c740f36e8bb3fa452c9d32b53": { "jp": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.756Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.758Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.757Z" } } }, "44b3f5422fc4c4f447ece76e0f8066bb34f3affc30e7419ca74089bfa8055925": { "b2e193e55be108c5582fcb93204b7255923583e86eda8e23c2ec5a7fb530f958": { "jp": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.742Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.750Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.743Z" } } }, "4e56f5a34b33c4d6df45c30f812200b60d80663863d11d65cf1450dcca806457": { "4705c821297fd380138640ab626f9d4f597a2b1853b0c84c3688cc33f5d4dd5e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.716Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.710Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.715Z" } } }, "80d3d6543dd83a7957930d9045025b4815c1307c41a15c290f7acf0eae734cda": { "41c8219de2e81a989c9aa442e0f7b45929280d651e3c0f00f28c5e946e5b9487": { "jp": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.746Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.707Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.750Z" } } }, "92105bab40be708ce10315bc6e1bb96fe7701142a5cccef12ac85e9bd6c2ad0a": { "f2e5adfccb04fbdb23894f4f9466bac4d65302edaa3ab747b455eca21fec899a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.759Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.708Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.754Z" } } }, "98c24f1533f177815a78f76de8765482cd98558271c182e9ea70175821ff82db": { "59cffa3acd22af2478ea31099f73412223d91eb1301172207a61ac51e8cba72d": { "jp": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.706Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.710Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.705Z" } } }, "99393522afef2d07d814a10cdd78d55ffbbf63cbc84caf67a25cbbb6702d7b29": { "df2e38e726ad5701168a107a3233f7e582b27aaddc17196034ab06c247a2cbb1": { "zh": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.755Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.739Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.758Z" } } }, "9c36f42318908cee7394ac7bdffe1f0b4dc78d18dafbeff49083e3a25b9a0c0d": { "e03b65e2958329c1310e8961f72be96a59122375e8ea0f5d7d4a488588e62cf4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.751Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.764Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.757Z" } } }, "a017ed6c9e204f49a926704a46e850a434a064d54ab74c5196dcbbbbf095a5f5": { "a2adde35cfc427e42fa09ac65d97536a831e1059c7400f85820e2263a1b87c36": { "jp": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.705Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.741Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.756Z" } } }, "a02c9804673e90df5f360bc1d48dc4d9b7a2120b7b838de45da8f0bd5dcc7bfb": { "6dba5895ccf72ae7b5a8b301d42e25be067755be6a3b1a5bcb26acdc5cb58138": { "jp": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.688Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.689Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.689Z" } } }, "ae924afae0c519cbcd8b9677316f11c74c707cb0d60447e3f16db759e6be95d7": { "10c1fb6d0471791e0662f2e4888a314601486dae17ed953b398d3ded8b18d82c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.766Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.762Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.761Z" } } }, "bdf6a99d4e97b12fb653dbfa5271fb1f3f759447def3b014fa132fc4a51905e8": { "70ae38ea604bbab68a53eb544cbd0f2cdbeea7e09ac7cd839c84eef1978dec29": { "jp": { - "updatedAt": "2025-11-26T07:24:02.748Z" + "updatedAt": "2025-11-29T02:44:55.762Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.753Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.763Z" } } }, "d85329459d2d377bcf713c7d13c7c96bd1fdcdcc09d41a049d07f84aa752713e": { "92a31f97c45363963ebd5c9ef1e90cde5e6438c8474d31a5257818a31f192d43": { "jp": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.715Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.916Z" + "updatedAt": "2025-11-29T02:44:55.714Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.711Z" } } }, "e0a0f58749dbc99214f8d73d23e3e103bb972d6cb973f80440fb3b9b4b81c305": { "0f27725ca1d515bacca9b8aa1e23bb35c69b895c1d9563452e075aee634e4939": { "jp": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.740Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.752Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.749Z" } } }, "e1027f068c086d68bcd19c94e1c760c747883dda4912d769a49634e99a298bf2": { "327dfaab66de7353575183e0fe7d40b596436f7254ab77cbe35383223ad4ff3a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.745Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.926Z" + "updatedAt": "2025-11-29T02:44:55.752Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.707Z" } } }, "ea52d1bf57d6eca260482d6e9db0b3e4ba187ca46f787a3ec41ccbabccdafc29": { "7792c45b9f12363c758a86312cea564fda8789130772fc2a238a348aa77232bb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.686Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.685Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.097Z" + "updatedAt": "2025-11-29T02:44:55.690Z" } } }, "f2dd481c53ba67e19f970889ce242cd474c9da5ed1571b9d4f5878551ed45889": { "70876690558307749f06728cb6ac14fce7075dc54a6e8cf0695beae2917c50cb": { "jp": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.741Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.704Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.749Z" } } }, "f4deb9d37929966d7a5d1040cf9c1132679840850e80dd91a4e56e059568b817": { "e1dc787a6d679d3f64b0e02ce3b61001ea2f9d7a3aab736ee5ae17d2bc4a4c63": { "jp": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.743Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.742Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.709Z" } } }, "046cf4465fa1fb7186678766ac47cbd78704f28064400523e5b59a245d53c970": { "b13281a5fbb00f880845872b5516d8f9e9039198c4bf031942d0ceec737def68": { "jp": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.788Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.792Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.783Z" } } }, "0cdb3f54f81ff077472e66fb0a57247ee5bf3d2a93abeb493538e948840b724c": { "2beff12ea84429b1b15d3cd9ba939104aa74a91c9770800974ecc16582d6d010": { "jp": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.786Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.785Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.791Z" } } }, "1ac7bdd9222c1e2ffa17fc544f1241b28da0cad7f185b619d29ac27e0aa8c077": { "3f8afe531fdd885aba914642b81b85afea758c6f849a7250bfeebc09887cc894": { "jp": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.747Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.927Z" + "updatedAt": "2025-11-29T02:44:55.755Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.925Z" + "updatedAt": "2025-11-29T02:44:55.746Z" } } }, "2a7b92dadf95743e702b18a30f74eb67e67fef5ea4124220e608f258c6950c9e": { "c66b9e2d0f4d5e382ea43aee7020fd1c7ff170725159ddc185d674bc64b0d24b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.771Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.777Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.777Z" } } }, "2f0873b2704cad58fd2172ec86c842a8262cb2a7c1e6cfbf1e9851fa843f4357": { "d4282945578d91a5ae49277f6ca146ca130e3b3df3c0341a5de3414625c2c903": { "jp": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.796Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.790Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.788Z" } } }, "583a274e308afe89671f40f27b3be757e2c9e98eeb513765c802185f5ec14e29": { "17f1e539b1b6e7759a4aa628833db4667a7e74444abb42880111b4568a28ffe6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.738Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.737Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.778Z" } } }, "60a5d6b5624fc995f6807d15e66c5a5b6bc86dc212e1745ef8bef3f5dc15c3df": { "c3d809b05c72707e6bb1947b6db7f23f641f83155cd0f83a5e6deedee8d07adc": { "jp": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.793Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.786Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.785Z" } } }, "65c3d5357d49f1617e6e959d8e29071102eaf8d0d9e4d1c2fb7cad70b6173a35": { "4cc1991c7b87b22d25ccb176e3b79b021cdde65ce0a2f2e4414efe519cc65f89": { "jp": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.744Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.738Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.922Z" + "updatedAt": "2025-11-29T02:44:55.706Z" } } }, "6e5e66ee5bbbba58fcfeffbe0603dfd55d38dd278fbff14d70aa5595ee971bd7": { "c4a33214adceb28be9e704611bd58cf7f2b17ce705ec29ba0ffd552d82a3d73f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.784Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.779Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.771Z" } } }, "823e98e23e42535697ba133dc5c2d8b57c41a87124d772a59a7bbf545ac0dd84": { "d6ac975393106fe00f3edd51a065ab64a5e99a9aad622632a08705ff68ad4b36": { "ru": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.799Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.799Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.802Z" } } }, "907c6e7bab21d938402f694e7407939915297c82eafd1730100c830df4918778": { "c3a2fac6bf16acdaf76f1ef65dc3317e37696c35df4e8526e1bb887fa5cfdeb2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.782Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.782Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.787Z" } } }, "9840f3c84ffc8a8f881de9eca8454a7e8de6c5bd5c30b6a27784816805453183": { "491cb45d3cfae94c2b0cdeaaaf82b4ad9d2198ed681060717d4f79378fc92714": { "jp": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.791Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.793Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.771Z" } } }, "acee1d54d44425817e527bc2a3215f623d6ebd68804cdb7f18541efb76fb830f": { "53b8019634b145bda892aa14cca4d68066dd9ed1223e68c1450a60c3d6af3368": { "jp": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.772Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.796Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.791Z" } } }, "b66cad86246e7e672bea77de5487ab3238a0cbd0d829ebb54fd0e68f3cbcee09": { "9cf089c5df430ee74bddf608da84394fafc963e1bd03cd0e94fe2c6c179ecce7": { "jp": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.780Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.784Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.794Z" } } }, "bc72b7a9222edd97db763cb5ebbf3a678bd9d11ef3bc3a2c28fd6797dd845434": { "ab1bcc3128e7fca61edfa8cb48cc7969450a097b52da47b30b191820f3c2d949": { "jp": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.798Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.798Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.772Z" } } }, "cbf8d771d3e60af84970fcb0a9a3b216e9fa9d6604d8b59222876988a0f9a23c": { "05073dfddb68903600de4505e9ef4203c4b4f5979a1ad1001059a7e6a6c36293": { "jp": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.795Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.770Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.787Z" } } }, "d259b209c3435b62d81021240a05134c0eea6d95e4ac914c175f7122e5bcdbb9": { "2336e34b998efec4cc83f0643bbd8fc97a4fb0afa00c3343a22139925e777a12": { "jp": { - "updatedAt": "2025-11-26T07:24:18.921Z" + "updatedAt": "2025-11-29T02:44:55.737Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.803Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.801Z" } } }, "e01a6937e1ad5b11af43515d1a1e66db9e6168b75b5528ca1b7b9d7f3c195868": { "2c6fc2afd47aebe8d703b5147ab0245326aebcd6663db747fdeae29badcd7caa": { "jp": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.801Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.800Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.799Z" } } }, "eac642db564baa4ce1f75ca03dc5a23f44db2e588ad4390c7c3cb746e81f695a": { "4bcedeede08560e01db76c1f8f3c871bd8e8aebd281853aeef86bbc3841fd68e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.752Z" + "updatedAt": "2025-11-29T02:44:55.787Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.789Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.781Z" } } }, "f5ec5f1c0bd0776f9a2a7bff868a443e7cbb090161004e7da40277d9999a5e0f": { "1d3bbb34461ec824f8f745ff89fbbe7930bf3ca75ffcf25386fa8e097845e074": { "jp": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.770Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.789Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.790Z" } } }, "faf7c1208ac6cebd805f74b867ef0352238bb675f4e78c25744706e43a0bbf35": { "067bee4f308eb8fb0ee42187bb88647c1df461930299cda269dae6be1e92e2b2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.739Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.923Z" + "updatedAt": "2025-11-29T02:44:55.740Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.924Z" + "updatedAt": "2025-11-29T02:44:55.744Z" } } }, "010f8d66bb60666d0d358628411eeb2b1df5cd86161855532ace6b686750bb2f": { "0feb62388a935eebc64bf1d2d0c742a3b9d17f4ae18ff4e0ed9f4fe6e68ce776": { "jp": { - "updatedAt": "2025-11-26T07:24:02.751Z" + "updatedAt": "2025-11-29T02:44:55.769Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.750Z" + "updatedAt": "2025-11-29T02:44:55.768Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.750Z" + "updatedAt": "2025-11-29T02:44:55.768Z" } } }, "050352a11ca817f5bab4911101cd95e7ae7dc5d8954cd551028380c728411a57": { "6cc2916b976989ba2663dd50f541fbe9751c56f179ac300fc3381ca3479e452b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.820Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.794Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.753Z" + "updatedAt": "2025-11-29T02:44:55.792Z" } } }, "09a42960aa106a95a5cbe49be7b12f9120aefe3ef067ddb1c1b026342239f3be": { "eb1dc019fb90478f30509956caa9e4f342a6e2b031332733edb6a6b927bc71e8": { "jp": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.815Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.814Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.816Z" } } }, "12be1e0f0e04bb9eee1f814b983cb24150e4b9b4f2e86f8c6cf33f7dd28edf16": { "25966e125c5b0d0e09bfbe0bb6c4eced38f8afae86050666be245c00bb7f240c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.776Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.775Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.777Z" } } }, "130f0cbb1d6c82f8ae457bc5d3dfde8dafaeebcec17cebf6c1ec40eb99cd1392": { "4b5db766a70f9027101f584180002e5dd6f63ed99aa3d036eafd61435ddb4812": { "jp": { - "updatedAt": "2025-11-26T07:24:02.762Z" + "updatedAt": "2025-11-29T02:44:55.836Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.761Z" + "updatedAt": "2025-11-29T02:44:55.832Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.775Z" } } }, "30c2729724c6bee568ae40c9d3da452323fc101c8c757841c99647d3bf63f057": { "4eb3058a8a2fa3f5f9687fb24c64b384670b5101c5582da6a348ce515411116b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.763Z" + "updatedAt": "2025-11-29T02:44:55.840Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.763Z" + "updatedAt": "2025-11-29T02:44:55.839Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.763Z" + "updatedAt": "2025-11-29T02:44:55.839Z" } } }, "377591bbd1fd99159d49074a1492a22294d47fb29e338af9ccb456a44f4a181c": { "79d09c5dbf435fb10ca29153a98d5b635aee732c2aa61594fcc2f065989ce327": { "jp": { - "updatedAt": "2025-11-26T07:24:02.761Z" + "updatedAt": "2025-11-29T02:44:55.832Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.826Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.749Z" + "updatedAt": "2025-11-29T02:44:55.773Z" } } }, "40a262fc5e1c5d47aaac955e5d56710c58b184834fced0236175665ec187d93f": { "d9751428d997f059562f26e9bd7ac68c276f0bbf0c513551408f0513601e3d16": { "jp": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.824Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.830Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.823Z" } } }, "46dbee6938843b18fe050245cf7379885770dc6f9f8ed8013ccf5222d1c306d9": { "1c26addde8215daf58446cd375e5e150c2d5ceeefaa8b2acfdb9c9c8afb9953d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.813Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.817Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.818Z" } } }, "4c1ad3942b4184430a7d30de866388373d48c1a27718ee8712e351668b5b2c7b": { "7f0ff3de1f2f3ef36f7c5bcbadc179455a3ae55c4a9e388b8148b18a4dfe6b7b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.795Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.797Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.754Z" + "updatedAt": "2025-11-29T02:44:55.797Z" } } }, "8d0001685270931055d17a8eb50155f983dcec73c892d71e3bffe9004c1cacd4": { "c26606f99e8098f4ed8f1e29ccce89dec0e9cca96fa2536b375d31a3b9fb8036": { "jp": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.830Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.818Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.831Z" } } }, "ac35f8f55935d4ecd0e3d7d6c02b398d04b18c830570d65f6551b9b4ff45bb74": { "09c8a0f7de8fedbc086a20b8603b6ad2439fbb800e29c34ecc840908cfa41148": { "jp": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.781Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.810Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.819Z" } } }, "b949b99783c59002d6d1611f53562639a71143cfb90e027a848ef13b70877e4d": { "65ed1ef87fa32188d6b83d9345586ca7be9701ab437946eec188e8d638e56014": { "jp": { - "updatedAt": "2025-11-26T07:24:02.762Z" + "updatedAt": "2025-11-29T02:44:55.837Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.763Z" + "updatedAt": "2025-11-29T02:44:55.838Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.761Z" + "updatedAt": "2025-11-29T02:44:55.833Z" } } }, "cba0abc4ab65e9d030139163304a053ef5b1fe651a26215e77c9e551fe3b8191": { "62328876676efd5312772f4062f7342ab3fbcced0fec39177d7de554d93c9005": { "jp": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.778Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.784Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.809Z" } } }, "cbf50a3e7f149ed504ecfb70077f07ab9e2fed847c539a9e27d5aa88c015e5f3": { "2db80f4884390b5613f02ed18bdd9194f0f0ca4c9123aaf5b1a68e1c52e263f2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.802Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.804Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.755Z" + "updatedAt": "2025-11-29T02:44:55.803Z" } } }, "cc4204c3e95911221d74a7265dd2e67515e9b01a1b9394863f065398c955594d": { "9538d72bcd29de25ee9a900cfa42747f8ab0f5767963a08a3028ab7f3b189a13": { "jp": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.827Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.761Z" + "updatedAt": "2025-11-29T02:44:55.835Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.762Z" + "updatedAt": "2025-11-29T02:44:55.835Z" } } }, "e15247f6690c752be9eb052c65d6188bf83aa3aa12e45c3969ebd294c52787ad": { "e8049a4edea61ad5f86939e7938566c1c4db909e94409eedf5fec35ac6d46e8c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.761Z" + "updatedAt": "2025-11-29T02:44:55.834Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.750Z" + "updatedAt": "2025-11-29T02:44:55.774Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.762Z" + "updatedAt": "2025-11-29T02:44:55.836Z" } } }, "e979381df042f92438f9f485ef68a9901f7ebe9aae3e09ec14dd65b67c2d842d": { "67bbc03e619fab0b6f99efec8b0f2fb38df1395be3d50b3ed225f0da4b3f4452": { "jp": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.829Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.826Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.819Z" } } }, "edbc39ef9c56e581bb84c8896ac7b1374e3d19f83b554328b6e8d3d38fe01125": { "1f975f6dea1c15645a72a9eac157d5d94cb767124fa4ad2e367bc8233d6b225f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.821Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.750Z" + "updatedAt": "2025-11-29T02:44:55.773Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.828Z" } } }, "fbe0f20b7a71a4be3f802731a84f0eda5afbf565f744180f030a4474dd0b950a": { "acb4ae581b304b32468cac562d7016a47f6ce4fe713075ab12bd276f5d04a0cc": { "jp": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.823Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.817Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.810Z" } } }, "fee41c1b851550b4068c1cdd9e5a18a829a2d27697fe22a9678a8c0d0e87836f": { "5d6d7dab6e54977464b77d2be0fe3967209334b0d1e2cf141000a53098cdb64e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.829Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.763Z" + "updatedAt": "2025-11-29T02:44:55.838Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.831Z" } } }, "00f878a9534e344ca38d2f13a2d0b58a40257c9f7c696adfbc337ee5148c5894": { "d7ae2149e8a1eca5c76f2e499f9ddf19c90a2c840a153acd2e820b96f79c4e3d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.874Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.876Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.875Z" } } }, "262ef21ffee6eb3348b7484a2cb16afdc22c4c02ce86edaa851cad0979d13067": { "5e4f687928ed10c1ab9ee1e070abf78ab7adc2bce9345de9158ca65c8a403093": { "zh": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.869Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.867Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.868Z" } } }, "42014f03b2e5e17b4e9d8b4cd80cfebbf2e3bca570177e4a0f46c977d992d00b": { "1713044e3cccefd79213a1fea6cb08cc00fcb5a3cdf023fa1b265af8ff27f097": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.857Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.855Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.863Z" } } }, "4c05567fa33cc5dc9787df23810bac6451ac3a0fea9f57dbfe51135476f2af9a": { "539aa35729dd5eb9f1172afd319421d880ea5ac2efe1aac243083236a1389aa5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.873Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.872Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.871Z" } } }, "4ec20679bc9c3514801ed7e21c4718d82ab75932df1a07eb0572f662a5730d98": { "86d2c497abf25c94fa112b01bc6df68914ef1bdec7669aac57b740da912b33d9": { "jp": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.851Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.805Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.851Z" } } }, "5a79d1e559ea1ad9f3ddadfdb2a43b047724a8158973368a06de949a009e4a82": { "f10bce44ecc97a7f7fbb9e4dd3135a3443539faf27799c8357608d1f78f0ea0d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.859Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.853Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.854Z" } } }, "5ea715da4571fccc329fc033348aeecf183417b55c28bbdac60956aa1debf704": { "2a8b05277ff4a9cbe8def769d30fe9965fd38e380148a45171afc696a707de97": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.858Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.852Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.862Z" } } }, "6577565180fdc2dd6b58e525087f761a4a641e5fcccec17b8d198f112e8867a2": { "457a7fd8ab504d03ed723c9475bd87417db7fa6b8d538f336eab293e7c2db492": { "jp": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.855Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.852Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.811Z" } } }, "65f86c7c3a06da5be6ca7c02d2ebc67707b92772d464e19a9f17a4ed1f5068e0": { "816a9dda53486f2f740142aa953a0c567c672d1d673898a9ad9493dd248c9c0b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.780Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.808Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.783Z" } } }, "69e3ba4ff50b5b7c3475f46f967bf675b4e5a81f02e3546d810018e6a3fe12c7": { "d64fa7ded50ab81c30dff31ff460cf6ba0811be8f95127b0bbec04487a124039": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.857Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.766Z" + "updatedAt": "2025-11-29T02:44:55.848Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.863Z" } } }, "741985413cbcc54cd08f4f04379dfece525dc97edf44e2f8df78b544f7dd91e9": { "2bd4eecf6148d08318f581143d8ed2830a034f2bd9d72c70252b27c1cf3654bc": { "jp": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.821Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.814Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.822Z" } } }, "8679a4ec12ab69d4d68a5bb2c5cea4b7f0881bbdd39e33ed0dbce1f7a96a02b2": { "6dafd0d4cd13c07a59291f74e30693ff78bc11afb76dbd58ffb368da7e83a065": { "jp": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.807Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.767Z" + "updatedAt": "2025-11-29T02:44:55.850Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.806Z" } } }, "8a737109d61aff4ff62c3cea1b972f0a7863c8fef9c1f3658e42f4cb31df1392": { "132aab96d1afacf12308b65ac1af9345cb2b097664b24dcf5c773ca74a90c659": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.858Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.767Z" + "updatedAt": "2025-11-29T02:44:55.850Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.853Z" } } }, "8b22e50ae696d72046531764b190a2ea3daa28284aebf2f2f2721e1db7b9a752": { "a3ec1a8f31c388fb6d343bd343994dbc83607b4c1aa74c136db042c2472a32d0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.766Z" + "updatedAt": "2025-11-29T02:44:55.849Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.806Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.766Z" + "updatedAt": "2025-11-29T02:44:55.848Z" } } }, "8cf48a0bc486c9b8e497ecc604e48b940db90a9be71802444fc6568bc64fd85a": { "2204d84ab0794f79cb34e93c0671da7bbce325d19d8d5bbb80030251d39917ee": { "jp": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.867Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.868Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.869Z" } } }, "8f767913276b5f3417959156454a31c70c834a5c7093a2081510ef83903f4795": { "bce52080edbc2ef28e9154b8f007ec28a5e436114ad9041d55ab9bd299d603f2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.865Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.861Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.864Z" } } }, "961e4fd08064e39aa2526ab854951754ce9cab815f42e5e159992babeeaa5b0f": { "cf7f511889edff19a30680bf294dfbeedaefa3ea56faf9de40db511b5d58efdd": { "jp": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.874Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.872Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.772Z" + "updatedAt": "2025-11-29T02:44:55.873Z" } } }, "c237b65e74a71bfcdfb33228aa085209a607cb0227f57e434c617a7ced16d975": { "cab8ecccbc0fcc08ad057ca251274b94773a36f8f2f5c0968c4007592472503d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.870Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.771Z" + "updatedAt": "2025-11-29T02:44:55.870Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.866Z" } } }, "c3bbfaf5ba432f3383f9213e2c564cedcf64baf52ca43663bcd031fc79f65fad": { "46c4379cf36fa439d614c84a7b1f2a6e319d2f3a5e352e7f3079aa72e1634e3c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.808Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.807Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.810Z" } } }, "ca7eb037869880c9ebb1a34c0000cdbfc8fdc9225de1f230ad67b8fceeb858de": { "fb2d804909b58e74a6d190031cfb86ce2cfa560d0444d3bb3d0a0af94da23268": { "jp": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.811Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.812Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.765Z" + "updatedAt": "2025-11-29T02:44:55.809Z" } } }, "d6a2aef23a40b1f742ecc4bbf44e21b915daaca32e6106a813cece2855459b4a": { "c2bbc1291a1d9794a9a8424dacda644c27086e3d431d4f0bb25b88182d583c5f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.822Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.758Z" + "updatedAt": "2025-11-29T02:44:55.820Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.757Z" + "updatedAt": "2025-11-29T02:44:55.813Z" } } }, "ddcf8dfb6b1a4d5a1ed98c2017cdd7ae1fe774db2009725b2bf3d5ca4a50b322": { "4f4dfdc7521283f8c0348d0878aa061e186e3e3aad4e92d55841f1902f00e3d3": { "jp": { - "updatedAt": "2025-11-26T07:24:02.760Z" + "updatedAt": "2025-11-29T02:44:55.828Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.825Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.759Z" + "updatedAt": "2025-11-29T02:44:55.824Z" } } }, "059de09a546e5fd7f343688a18f5ae23fe63e31ccd72bd1d8e0ef1ccff248e9e": { "e0133670b30030462807054fabd8948f4d58d68bda6f5fc806435ba96fdc2531": { "jp": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.899Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.899Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.894Z" } } }, "0e59ff691e81e6bb5df727b7bb1a30005ab315602d293b41cb391ed4b5409e8e": { "ab3c2315a32f46dcd77506c38fcb11173ad15a3ad7597e20a3af0f8b3c8e1c02": { "jp": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.884Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.845Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.847Z" } } }, "1be2e6251cf6bfefceeb9a1d2a2cdfcbca4f3dc24d4303c2a666b520ce7dbc5e": { "79ae2db2ede93c3db9f3aa10741077dfe47e966f67fbb578af090bc05ef54683": { "jp": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.898Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.903Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.841Z" } } }, "240885d8a55bf641313a779462f9d5afe9dc23030aa7263fae65179e8d79b9cf": { "0f3c6f532be1ff66173a6f491090bc401c5f5ad396a065d669cf8be23b790fbd": { "jp": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.894Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.900Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.893Z" } } }, "327d9de85dcf4e9908ef639e81f0b4c26211261cdc7427d31c00d57a68f9ea57": { "defbbc0826e47d88fbafb696aa0613a205a13036670b5b16d9f7262852215ad4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.843Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.886Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.885Z" } } }, "34fc130494be0d69639ef51384f698c85712275c82f72ea0884fc912c61fdf98": { "92c9764efaeac8ae2150358dd44c1bb27f41eb7fecfcbaeaa5223b274ca6abf2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.883Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.842Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.882Z" } } }, "3d292af39191f27d31948c49e58c34422323914d2d835dd3b8be63d271aafaeb": { "6c24a188e7d85e8dc525f5000fb2f41b08e17a821ce60ddfa9341db9802fcdb2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.905Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.902Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.902Z" } } }, "4b025a8d2616c548c48183c20f38fd63b3419b7d2754a037d1a3a71de57c5a3b": { "ff303dcd7cec8ced40bda437d563bc42f245742fe9f5d04eda4a24a951b0a458": { "jp": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.897Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.895Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.896Z" } } }, "4be2dfff7ee7eb8ba7e00bea4338d4b73e59739bd67763683573c2c8a87c9e3d": { "37c83798ddd19c1e72b3674657a3635ca49e5d5bf74e74f2fa7bab5c89d58316": { "jp": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.963Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.964Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.964Z" } } }, "508c2be06359376eba3e09eb266a71fd1a64aba5ea5c127642c386bdcf720d00": { "32a1e97aa76cb271770dca75fd904e715623cf504f26d889bcb51a382ae083e8": { "jp": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.903Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.897Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.900Z" } } }, "6547aef5926a6b2487f43dbec05e0957fe924c3749b2e7aeeb9c8724921310c6": { "d72d4d5d1769fb68537cb2b0120c647b9e45e7282fdf4303b4b3b3ba33eb151f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.764Z" + "updatedAt": "2025-11-29T02:44:55.841Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.896Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.905Z" } } }, "742de82015fab9560f32bc67cc0f07a9ca9e1ed3e7aeb11eb4303fa8a580185f": { "e8e388627f1d46545b74abb196d0b01e87cea3cc02063cec9c7cf6835a4f7d7b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.844Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.844Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.843Z" } } }, "77a9c51767cd665f3dd2df3d7ddefaa1effd2f1271cde0211ccbb68de9869a6c": { "1c1de24396b6e6f16f0f9b41c9ee154414738e50d2c294ceeedb57d2b780396f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.864Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.859Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.862Z" } } }, "90aeecc84affbe1a94ebd79e7e3236a66f9c627e327fbaeb50f05aa43d716a7a": { "a7b61a1bd22ae77b9b4f8fe2bc248f5fb8a900c9c853a0f4b28e2114edba6edb": { "jp": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.904Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.902Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.906Z" } } }, "9815f463df07221f4071a1a1bca594afe93b27adf83236c69b1a77b1ebe508a0": { "007c21ba67676302542c1fff75925930501f8226edd684ec93ea8a9d480c18c1": { "jp": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.907Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.907Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.908Z" } } }, @@ -3880,520 +3921,520 @@ }, "471cf465239242ec9f9d784205ced7fc1640f6da4c8228d46163e7757979aa8a": { "ru": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.883Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.756Z" + "updatedAt": "2025-11-29T02:44:55.805Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.845Z" } } }, "af79bbae5029e0964764673ad906f12ea5d0cbd9f6358c69ef5ef5e1e2abf9c8": { "2ac53c6a243d501aa141cc7a46939a9b6d8d89958a13b73f7e3def4acf386114": { "jp": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.904Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.906Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.904Z" } } }, "c26d90fc85acd6879286c1468a93acb164acd86eea2a927516015902a9a832be": { "7cecd0f5d3861eb201c695566fbb8efba35f90080e6ff53cfb99227a455a7433": { "jp": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.896Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.933Z" + "updatedAt": "2025-11-29T02:44:55.898Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.934Z" + "updatedAt": "2025-11-29T02:44:55.901Z" } } }, "c8e894dbaf5047cc3cabc950a4a8ff475057f2bc769f6e66960185717ec18b52": { "53f949f10b8d348067c1f595ef08a9cee2ae03679b3e38fbfe1a67bd2cf12eef": { "jp": { - "updatedAt": "2025-11-26T07:24:02.770Z" + "updatedAt": "2025-11-29T02:44:55.865Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.856Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.861Z" } } }, "d6b97ab54d7597109de2eeed733aaedaf2f8744ebeed7ec1031b8460e9c545c2": { "60328591af08fa91508ef8597f7a9b54e083806e1906b2740d4ec5802abe7ecd": { "ru": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.966Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.967Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.967Z" } } }, "dc33a2eb5786282387491dfbb49c8ff622ea41f11b3278436c7f82ab857f0228": { "6d34c7aa55a8fa5def4e3f2bff389c666852c48291ebab26dbe11069e1977d67": { "jp": { - "updatedAt": "2025-11-26T07:24:02.769Z" + "updatedAt": "2025-11-29T02:44:55.860Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.766Z" + "updatedAt": "2025-11-29T02:44:55.849Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.768Z" + "updatedAt": "2025-11-29T02:44:55.854Z" } } }, "0b6d9c8bcd38a3dcf622f85a9b9f97289107d754955596db63086c5a1f0de013": { "62bc03adcac1853c2ff1d41eab5ec55613571c9634311e2e305ff20b78db334b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.453Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.452Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.454Z" } } }, "13e624cf649963b0430c85b33066c42e9a463e53696049fdef557841854d666d": { "81c2903aa8b7c3295234e5c1b7fdf2be7dbc55fdc9edac19c3d4675fd1215205": { "jp": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.457Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.458Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.464Z" } } }, "2ed1c4bf7fd0d1e9a3aa0e5f13e3c86bcaa77e12c79c9d2fd35be9b8cb485fdb": { "042d7dbf05f1c54ecb628a3aec1b03eb4e2b6e21cb8aa57b9ada88ffcae4f8df": { "jp": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.150Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.180Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.179Z" } } }, "3d2059239ad6af1a2ddfd59349dac15c70518ae11885267fd488f16281699791": { "bb8598cd736f9055ff9d8ee57cfbaf381f8b9b7dd5b8bedf4b973dba8c441a2a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:44:56.528Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.526Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.525Z" } } }, "3ea83a8ef84ec6bbe25f2090619db1abe347ff2b73bca590d6c93d68a42e4e64": { "d03f731b06fef8fcaf928f6e3faf509894d47eaf5b4921a111e9884783dfaf7d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.459Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.463Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.463Z" } } }, "4ac2aa31459a0a92af805200fec9ac7d528d83083a8813c71176539ce30a55d5": { "47965995534ac0fbc4b623464960445019f4dbe230323078f5ba06347fc0188f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.523Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.527Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.508Z" } } }, "4ada93142f1fa23e960fcf0c89e6d17aa2696229485742f034de4ee6593c2071": { "2f19a7e891dd293775fe6638aa903e735c6029210bbf3a17860c69e6f1f6bb6b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.451Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.451Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.452Z" } } }, "5e4520555f04067ffa7eb5af85e61960bb9ef0b5e53db65b7b0471c0eb67e3ca": { "7bb096151a00169df14ef9af359bf6d8949aae217704606f9a6b10a44d8ed7c0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.458Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.460Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.462Z" } } }, "736bf0149d53b024ca3bd9e7977f0bc63d265b1f25ebfb6dfdefeb025d67a838": { "dea965238a83d73269b02031548818dad6e76024fdd545d4ebfad71b6ea7f2f6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.459Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.461Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.493Z" } } }, "78374142cbe93e8f6c7c78c21fae25fb7304d36492b6bf841f120cb0b757622b": { "8c65e21fe9e7b63afe26dee2f144ad334fde661179f2df54cde98ef19f746770": { "jp": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.966Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.965Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.965Z" } } }, "7d77ec1ad6a5f022e0b46f5c3c3ce2c3fea37ff042d1b5dc03023407e067e3da": { "a014826091cc7de6ffe26de700b6870df49479656119a1c4582ab3ba9f32f66c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.147Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.146Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.148Z" } } }, "8c7d4f3fdba3bb4edd06686b726948493ddc13a3c70be44e45a5101013e47060": { "e1a3f32eec379181f97de3483a7955652a71670ed2c2d3ea34c65b17fdc5961d": { "jp": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.151Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.149Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.180Z" } } }, "98ee65248863652e985d674cf1372dd020bd6094b7f3998ae6f4a646d94892b6": { "1bd995b679039ca6bce9ee0b09736ef8f967620b8b89d51a62c70a4d312caa42": { "jp": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.419Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.419Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.418Z" } } }, "995a2e3a8b7d9f74a6263555c02ac239faad9cd474831a38bb8fbe02a8eb4930": { "9cf1d6f4f93a189585be6125df675ba7e1d73f8db3dbffd354c683519bf24dc5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.494Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.494Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.416Z" } } }, "a0768d64d8480213582b5d2b019ac82a6fe9572e3115c707079ccd2a6665834f": { "f53e89f4c4f5f43c018862a8bcb2458cf38a59a2eed7d3a2bac21d2ed57cd772": { "jp": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.417Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.417Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.418Z" } } }, "b5acaeeec7ee7e0b3d8c363ae84792dfc90953fe82cb345bd9a76003f6857008": { "becf724869353de9ac0fbdf72d34274bf02c4477ca8efc26bf383f25cab477b9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.181Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.150Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.149Z" } } }, "b6cd16941758ca4a1cd018e80e59496c19b7711675f9eec3946a989810da8301": { "def5f58d34f9e59ee3bc906fda67f3a9ea90982c852224c86d9d02f3eb4daa81": { "jp": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.462Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.456Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.464Z" } } }, "c5c9fb1e01e8fd89820000126b65de13c1b1aa7723a21af8dd6a22b0c6ce61ab": { "f0bcc513afa858c10cd2907f4b418305889e8287702cf9cdb050972831c885a7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.132Z" + "updatedAt": "2025-11-29T02:44:56.461Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.456Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.493Z" } } }, "ced886ccae611b5ba4d14770da1f424b55ef56a32ab309f10b5ba3de061a0cbe": { "4c6f8e2e7974ca1e44a92dea680f0fe4823cb3dbd478d406583065fef1965c83": { "jp": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.151Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.978Z" + "updatedAt": "2025-11-29T02:44:56.178Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.179Z" } } }, "f3dfcb7d93e8daf7246f1d7b55aef72c87092941f169ec834a03a5094039d22f": { "30c8a47e6bcddf07ce86164218209c750f1bf6a65eaa190202477bb3b35f8686": { "jp": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.963Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.962Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.105Z" + "updatedAt": "2025-11-29T02:44:55.963Z" } } }, "f84c373cff7dbac798db4f00e0218085b87659f099e72d499856efa42972f195": { "4b9492d3cf50402946edb0019de92a07ebf67ee41426a0a31d7cd82149581a9e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.455Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.455Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.131Z" + "updatedAt": "2025-11-29T02:44:56.453Z" } } }, "018a46e784f4216bc572797ae4cfd925900c11b01082ddf5a2c9b5ed08891d85": { "0d31eaa79270bc25ade146c9f275b342537708966bfbae7622a921d0c569a2ee": { "jp": { - "updatedAt": "2025-11-26T07:24:12.136Z" + "updatedAt": "2025-11-29T02:45:01.992Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:45:01.986Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.140Z" + "updatedAt": "2025-11-29T02:45:02.002Z" } } }, "171b148b39ffa6bfa68154f7f794bc9828036c905ec6ea0ed3ab52ea0ab68098": { "9b71315bfc1a5504ea574514ec21f8d0b8c75e646482a4fa10456513e23ec3be": { "jp": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.049Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.155Z" + "updatedAt": "2025-11-29T02:45:02.058Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.052Z" } } }, "24ff6950696e941c133754804fa0f7502ed10e3f273f7828f34d0ec98cc69169": { "9ffff4baa30bb8aedc5b7c4bed60c32432037227f50854a8cf0a554ca74b6742": { "jp": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.044Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.037Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:45:02.035Z" } } }, "2de6c7cb85bc8ce6037011a7cb84ceda700e54852ad5f8048c1b021e9505cfe2": { "cffde22dd20a99321b2469fa4c5f889ab0623f7597c7318cb5c82cc569be15bf": { "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.039Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.043Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.043Z" } } }, "34539b13bc46ae1ff86399ed0b4beced83470a47c23ade3688d97729e239c69b": { "1227956927c2e159479174df3466808d9bd9a1f2cdd1dba3233e8d80391d27c2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.044Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.042Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.037Z" } } }, "397adfde7a860a0707618fd95e8f1f4db83c3ecc2e6141f24b64af0162bec70a": { "fa85899ec41f9998773c9e4dcae84709a75245ca0e0e75850cdc76516b7fd66b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.141Z" + "updatedAt": "2025-11-29T02:45:02.004Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.147Z" + "updatedAt": "2025-11-29T02:45:02.013Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.138Z" + "updatedAt": "2025-11-29T02:45:01.998Z" } } }, "439776d4466dd70e3c5608271d0bffbce0782915faaf2fea75fff7b8e7835bee": { "eb302a76d12c1319056a47c6302ef68febf3a0648e4ce4f94b2b9cfe7bec8c8e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.051Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.048Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.051Z" } } }, "5efbb4c7ed17158323437048f6f32b54a1665e8008c3b499bc27160f7cbf02df": { "06c63df1edaffeb10cb0def08a755d71c765dda9e99144cb3ca1eda2e783c187": { "jp": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:45:01.987Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:45:01.988Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:45:01.988Z" } } }, "61b82c455342cbc02f4a1d8651204017609b443fce1a7cb57a4831730d7fc050": { "1d27a882dcff09d3f22870a4f6707da298747c547d36d3db2d61ebb22253f91e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.155Z" + "updatedAt": "2025-11-29T02:45:02.057Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.156Z" + "updatedAt": "2025-11-29T02:45:02.064Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.055Z" } } }, "65514e61688950cbfdfadc40744ab73dd695de039206e57d83d48b00a2982161": { "c8edcf2ff1eff165beb006860951dfee61d76b4197857f2fbc085e60726d3e38": { "jp": { - "updatedAt": "2025-11-26T07:24:12.156Z" + "updatedAt": "2025-11-29T02:45:02.060Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.139Z" + "updatedAt": "2025-11-29T02:45:02.001Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.147Z" + "updatedAt": "2025-11-29T02:45:02.015Z" } } }, "745a92a844b6497c0310ad16eb03df90d655cde8d7482e58f32d1af9a9c6e68c": { "ed4640fd150472b99b01119068e79ab5dce8af8145d98d8e1f847e482439180c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:44:56.528Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.513Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.522Z" } } }, "7ca5e1494be65fba41fe95ee7a3461cd0844038fb74f44098aa4e3f88607c562": { "ac68f255dfedba5a9d7fc4021983a5c3dfb83430f46eefe29bc3204cdf2720ec": { "jp": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.507Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.514Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:45:02.036Z" } } }, "8bd7dd424981003d655b71b52a223cd72ca57102e28da0b8baca6e8ed3256122": { "8c69f1a1f0d38fc584fc63dfbf0111f2d94d9ce8ad28c47314863119988ad693": { "jp": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.450Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.137Z" + "updatedAt": "2025-11-29T02:45:01.996Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.148Z" + "updatedAt": "2025-11-29T02:45:02.021Z" } } }, @@ -4419,343 +4460,354 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.158Z" } + }, + "32f79342fda1521b32c3cbd5194d1c9682d16a53ade8cb05571f8a298e7705d3": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.498Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.502Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.505Z" + } } }, "9b90a0dfa5536d6059d87dc8f5e817097c8b7bb53db517bff51a83c3e4c282ee": { "3e080983011ca5e98fc432fd4170067d4807f3aaa1e1114b8ec36d58af28fa38": { "jp": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.511Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.514Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.510Z" } } }, "9fe1ae047d397e67707e74c6e97afdec367a2fb4cf27a1ade86e3e2bebd7c4a1": { "9bf44240bd8b0398201f8cc05ed363d4bfa70d08d267473203007c092efe5287": { "jp": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.489Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.129Z" + "updatedAt": "2025-11-29T02:44:56.487Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.042Z" } } }, "b12e51a32d37bb2fb10917322e67f8c70cee8f595c143cd1b629cbf918f6b7b1": { "5014ad055f5a140206335e375c472557e174c690fe089774a9aa8c6d57d28567": { "jp": { - "updatedAt": "2025-11-26T07:24:19.014Z" + "updatedAt": "2025-11-29T02:44:56.447Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.136Z" + "updatedAt": "2025-11-29T02:45:01.990Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.141Z" + "updatedAt": "2025-11-29T02:45:02.006Z" } } }, "bb0a1d7136d43d7b6bb4fa9a54f344ca0e81896a5eaf9cc6ef57b8c3aa682779": { "399cd03c18db8759846f978c253d288ef4caab87adb1838ee5aed970412744bb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.512Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.522Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.040Z" } } }, "c9bb01545754a986ab5cb4d637d8745f995e8c5243183cf90e72563584cc924f": { "efe17e7594347ac3238decf2b1daf336a87a883f6d30bf4a916bc5ae75b80dc6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.523Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.506Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.526Z" } } }, "e814a9ccad02d86ef7c915fb695045529731c882788157b39795b3d624875c39": { "e078c263c4a0f84949c189cd1b90be6b54b0117004a43d0171ca1e7dbbab8fa6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.527Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.135Z" + "updatedAt": "2025-11-29T02:45:01.985Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.525Z" } } }, "f4614a808acf58ed3907dbc80a1d159bc107dde839623cbee9705514996e1fc7": { "ad253066ead1dba2ae292160fbbd6c6d76963231fdc98e27296a51ffab627b05": { "jp": { - "updatedAt": "2025-11-26T07:24:12.134Z" + "updatedAt": "2025-11-29T02:44:56.524Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.510Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.133Z" + "updatedAt": "2025-11-29T02:44:56.511Z" } } }, "fb63ffa66c1f033c420fc44f068aac3712f16d06effcb9b76446564d05a24f47": { "1f15e6976c3b57e0c74fc54faa9674d3ad5afb9a87efa0a5368573923ad33611": { "jp": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.039Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.515Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.039Z" } } }, "168f630aa6a5caf3759f0040f89a49b660bf997cba3f9bb07f93ceae7eaaf55a": { "3b9ccf775a7eb6ed27d87bbe61d94bd4d43913c00f26010a4b8706daf4a6a956": { "jp": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.507Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.130Z" + "updatedAt": "2025-11-29T02:44:56.490Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.514Z" } } }, "23e27d61b6423512c41b26e6c3b22921e93b5b301057fe1f453a2c0d3c1d15fa": { "7a7f792ff342a20689f60f0f265171128a171dee6f6e5a078ebb83a2cdf6ed03": { "jp": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.027Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.028Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.025Z" } } }, "296596880e307a8996c1b3d2f22b414f048332caf4d2083980ef5b77a8a5fdba": { "8891345d058983824a4006d332ff1e3d458871da85894bef04abd4b4a563fce5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.537Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.536Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.537Z" } } }, "3147fdc69c8941ecf865e47d9d8db4472067857ced28a4db9f1732ab44a9e898": { "89c5c15673bafb792cce9d30449c0c07581ad4fc443060edb182f1287d36112c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.046Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.492Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.038Z" } } }, "400e0d7d9471a5d2f998a13e2c607bb42e5b9c72e742d7aeb05f4ee1b2071baf": { "04b93a38ca385c15ca8270dcfa15d68a6941678bdcd2f76c2ad4f06bef8c33c4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.049Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.148Z" + "updatedAt": "2025-11-29T02:44:56.491Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.053Z" } } }, "4088be7256afa16e0829b465fbe851d2600d3bbb21c2610210c4075f713ee668": { "5263f7887931f9fbf63f2e9b15b7ccdd2c7157a7fd13cb67ba7bb5a4724f5c9f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.531Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:45:02.126Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.530Z" } } }, "434c8c6575a1c96da70aa7b25b8f2386d3813854c5fc71c4731982bf93c5b551": { "33868413cbf230f1914b6622c0fa2f639a7ea45c3142a4368aa173e8a03fc411": { "jp": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:45:02.127Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.533Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.534Z" } } }, "5199e54b28e8b206af31f52654ebdf21657caebae6cfe9e8a655ac120217243a": { "cce5c749f00809c0ebd64bf0b902ba923e07ffe3f6cf94b3e416613a539be455": { "jp": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.024Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.026Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.024Z" } } }, "53036f27fabf8d446bbd29d9cc70a86efca0166f01d8c0a80c2e5887bae5dcc7": { "4af9345e8b4d38c273b70d4810c616586320dd68b576345f37b88d764fc3b31a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.535Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.536Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.535Z" } } }, "7e14895b92e515a566c49df3e172fa8ef0794a3911f227fc4a71c6dba5f490d7": { "99b76fc928beec586c17a5cc43f58eacac997ef5729cc011bbfca37d37c70a79": { "jp": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.508Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.506Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.509Z" } } }, "818f1c114e04624a9ce346e723231683afc9efb77f488e698cfae3f76123798c": { "7802fce1dd531f1d9274394e1014f26f608015405f1fca427d28159a91303ceb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.031Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:45:02.129Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.148Z" + "updatedAt": "2025-11-29T02:45:02.022Z" } } }, "89be4ef20c9e5fe95e7e9565ff5aa903eef3eacf9ef5bbff1fa371c4ce7dca62": { "a6c4756c4f81974e9497aa328cf4f067d2e218a364817e6b3353285d9d897dbf": { "ru": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.034Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.532Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.033Z" } } }, "92e7d4855f47bd7172f2143c5bf24c013dcd99fd681ef3d53e75a588365ef40f": { "4aba2abdc8ba16a13f0e130fc8a1c260887158a147901de0d5f87498741d53f4": { "jp": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.530Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.534Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.532Z" } } }, "b82ca3ae71e7ca0bff5a8a1a958e415791b51606240790fabac0a24e99e5a8e5": { "4ed62ba9027cfba50a02993f949860b2fbf583b0d2272c93d49202621bd1c2b9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.045Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.148Z" + "updatedAt": "2025-11-29T02:44:56.491Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.046Z" } } }, "bfa678e1e376ec015ac9221d88b1803ce7811869f141b22241c78abacbd547fe": { "8a6e9f00b55f3b0576f01c6ef20c5163ebaa186d9ca2ba3a241ee00d1040de72": { "jp": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.047Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.048Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.154Z" + "updatedAt": "2025-11-29T02:45:02.050Z" } } }, "c0113692c1b839bd1d0b553b8a746fd8e901fea18a0365874a7d57b5c47410d1": { "fba4fb769bf604e65c2e862ea128a3429d4692c33e0b8ca43bea57e16c6781c6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.495Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.495Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.149Z" + "updatedAt": "2025-11-29T02:44:56.492Z" } } }, "c2fb7179016e62efedb581c777d5b3e618da9208a31a2d9a164ea0308a1143c8": { "795fa89dca9c3b26ee3aeaa8be7c8410b0abd1d329f364f1777a29c3bf6ae7de": { "jp": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.050Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.153Z" + "updatedAt": "2025-11-29T02:45:02.047Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.152Z" + "updatedAt": "2025-11-29T02:45:02.045Z" } } }, "d853a1e0bc487c020a87d920028e6165d0cb3cc395e7fffd09047dee78720588": { "adec2ea632fca207a13f7608230126d9fa9e97108c03848018e30859a7144104": { "jp": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.515Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.150Z" + "updatedAt": "2025-11-29T02:44:56.512Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.151Z" + "updatedAt": "2025-11-29T02:45:02.038Z" } } }, @@ -4770,70 +4822,78 @@ "zh": { "updatedAt": "2025-11-26T07:24:19.018Z" } + }, + "2e9fd2d3c490bc28c36a5d0ec21cb93e844dfdd34f2eb187c7f84f44c2e7cfbe": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.377Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.391Z" + } } }, "ed8b9299efe36240a44450f98875fb20ad838d0c4a97a4d6b56749d87a6c69aa": { "64421077253a2367928552f8ecfca1500ab1a3aa6470e26d805f6aae81b107b2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.148Z" + "updatedAt": "2025-11-29T02:45:02.023Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.030Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.023Z" } } }, "f2a1cb5fbab7a29d73cb9cda36a728e39c8c151cbbf17216e1d17db9fa27ca46": { "476a6d57ad70a1da9a30da84cb37903d939c7a7a7081a3cf8af22e8a9146c1d3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.026Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.027Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.025Z" } } }, "072403da5aaa82666ec2ac55aba2660e421b6711c6cb70c42b4eb6038b58554a": { "aa38bbbb12b1ed94ca667358f90437e09046357f71a6d1e0f8a508d57a4b5568": { "jp": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:45:02.127Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.529Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.034Z" } } }, "242c81539a7d39347e31852ff01c14ca7481b428f62ec2a9a8ef8923e319fd70": { "ff718abf7b9337cb72f9728d2ee59f8366fc732135cec35be718b34d911ff036": { "jp": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.623Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.292Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.627Z" } } }, "2ca1f06020b55585ef361cf2b43c9aa9e23ed32e9d0a80f58141cb6b357e2508": { "e8f70f164f2c79a05e20f2ea7598ea71abec4dd9a196fd990cb3b9f5f5250252": { "jp": { - "updatedAt": "2025-11-26T07:24:19.017Z" + "updatedAt": "2025-11-29T02:44:56.529Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.036Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.125Z" } } }, @@ -4851,117 +4911,117 @@ }, "6e3e04cc7119c0602d04810abb60bd15340766476b6dd90c89c802891040b74f": { "ru": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:44:56.544Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.544Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.541Z" } } }, "516b68aad0c74f76c3c12fe30d1f7258569a0b66643da4924fd24d791f072074": { "55acd998caff6e952b47ceb372ae02d24533c50e2c2a2d341e32d84c2b4a01b1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.585Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.630Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:45:02.137Z" } } }, "52d9303d908cc54e70744ee1b1e2e09e4cf8cb226c9925cebd341f9cac387001": { "71eaa12db00dcad81d12c60186209b5ab377404a70d4a18ee7d26b6ece5ff741": { "jp": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.139Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.138Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.139Z" } } }, "576d725e93d552fa1d974e38954e0acf96bd1d7bdb7ce394aea881a846161589": { "5d83a7ec0232591623da4893b116014b1e37aa25bdbbedda273544d85805f34d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.539Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.538Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.018Z" + "updatedAt": "2025-11-29T02:44:56.538Z" } } }, "59eee6beba7ef7f4b2b1ab657b188c2ad938982b20b45febf1c21c4c7b23d916": { "379215258832c5b1b0beefd2f0012d327e4907cdb0e2564650bdb42214e2e265": { "jp": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.302Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.632Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.629Z" } } }, "671c3c038f46cc2a350b67ff548f3064f3440f0912e1cada9cdbe60cb9c2971b": { "35a6b4b0da582ffce53ec6d62ecfa840b3fd54894bd3063441a0fb637cfcebb0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.028Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:45:02.031Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.030Z" } } }, "6baf2f9dc91a3aafdaf603aa070b6d303e0ca43f60c45975bd126c795f51bf6c": { "21159c4739b98c5874cd3f6e95850d863ba6be6c3a8978b327a9bef2d0bbda5b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.301Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.292Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.585Z" } } }, "85b69398b5611cad0ed97b26cf9ee7ab54989a0ec7615bc3aaabc2e0ae3c33ba": { "3069fe2c05efa1690a8fd9f6e9519528b8d09fe75d6fe914e613400f223a3e0c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.625Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.298Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.626Z" } } }, "8d4d7b2200cef950ad1bc09f8c982ee5def76cb7de03b7265ce2ab6d4a48fc07": { "782ddff0f1b9ecab869f6fba2c946f9fc98a65b12620a1eeeb09e7adfbdef623": { "jp": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.029Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.124Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.124Z" } } }, @@ -4979,1950 +5039,1950 @@ }, "f82d62c4dc19a5c31661c04d7a069bfa0d236fd3870382dd08d9cdbb13e02b93": { "ru": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.543Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.541Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.019Z" + "updatedAt": "2025-11-29T02:44:56.540Z" } } }, "b326d89975e77fc4fe5e09c43f7d7dd72353ad2de4c76604cfa709d60f39cee1": { "41f6f44d6560ff4b7b4a8919ea06169035e1ab5f00669a7875013466734ef23e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.300Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.293Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.297Z" } } }, "c0388925c5cbd757f80d526185840b27148d3d9f44442adba2d651d360e9f8f2": { "fe663d93e8ac7ca2bac8f4753fad3eb0d0150631ba2d2c4e3a85eb5fdd27dcf5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:45:02.129Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:45:02.128Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.032Z" } } }, "c8f7fa8f88301edf51171572623222cac00927836c2b38e0b936dc6808969163": { "0bdde8ad92c2b56e1260138b52e278dda8cd06b984643902593d0d0cd7fb1ef3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.024Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.125Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.126Z" } } }, "cafe8a479283375a185399d18cc4d2444fa4afed320fccd79e4b21ccc00756f3": { "9b037a637113b68681c5e24a1691633df3e7e4ab645c3430fdfbded768ba8392": { "jp": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.138Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.137Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.134Z" } } }, "d66c9f0b5bf68d56ccdb239450a2def7e044ee7dbb38a6df724002e0578ee93a": { "b17e684424dd7e3c1032466ae89d5b4b0753b2b11488a3c5480069b467bdfcd1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:45:02.032Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.029Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.016Z" + "updatedAt": "2025-11-29T02:45:02.035Z" } } }, "dfb826f61e2d3065b29aed9473793d5e9482ca0064907298ee886dcc849a2f30": { "095ffff652d364d8d2d207b5c2495c8f89b149222bdc9348bc26c7785dc49095": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.622Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.631Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.621Z" } } }, "f7ee1a75569ad87e689d0ebfbbc64afa80e4517625c27676aefe4a63952637ab": { "62283411a070bd19b48c75ef32990fea3d01df15e6ce74c1ef8474a50f977cdc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.635Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.634Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.634Z" } } }, "fbf74a86f665ee9aea4f220700429c38da217030a00f7a905ec8171cb63a5f49": { "379c9b448d13ae5617010e62fc925030e206c603b76eb2ab7ab83dddade8d46a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.589Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.295Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.296Z" } } }, "1204bfc3bd6e857b87b1b5a9dd1156c45498c5d9e64e68cdce6f8dfe4987ecfd": { "373f45a715a82081f8e2a3779cc63f874936a6ff999e1d2ee5daf6d9f720ace1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.620Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.657Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.618Z" } } }, "24ceb06f47cf82806e35ac32dfe18ca24087b06cffbe5021739286a56c793b1d": { "4ace68b0458a094405f4c0fd1dc60a5ef026a1a8639846623e86fdff84ae8507": { "zh": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.655Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.617Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.611Z" } } }, "28e0a4a4c7b2f5abc28938acf55ed73d8388d95376bfa8dd13fdecd6bd439e52": { "7b5571b023d676e2979970ede929e965221ec27898362e89cfb8519c41cf3898": { "jp": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.633Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.632Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.631Z" } } }, "4a932aa16f4947c7ef17e42150e4a316f1ffcde90dd8415e4c6bf929ba835846": { "49a5dd5634212d8130c73ae1cd817b3917e322d14b3c96754d53df3d228cd836": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.587Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.588Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.629Z" } } }, "4ca74029aba5db691ad0ec824ca43ed7d92a4d5b9aa1573bc7116ad308b92cde": { "f97238d94d5bdc95a6129e0715198e8a6b955a58fbaa7da4e12e9dfa1348f135": { "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.615Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.615Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.619Z" } } }, "4dec7d00a7f493694d866154c843363d42ed6db4abc5dfbd010fdd90bfcaf67d": { "97c6b3e272815f6b0861c69df01e35d4daeb9dd3a1b81af896dc36740a178f9c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.630Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.294Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.301Z" } } }, "51e35897aeb6c746cdd097c39d7d3d876e62dfc0623f6a3c97974b88226b3a00": { "07eab7fc4983c7ac1da23e4f9c0e0aaefbcbbf2c5cf96b5e1af6a93d9eab9a6e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.620Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.662Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.654Z" } } }, "6faa2072fc3d3a3770d528540726e0fbdb421fa84e62c668a817741883d26440": { "579c8415475bba272d86e61362d88b8f1304de7a7411591652572d7da45590c2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.669Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.669Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.603Z" } } }, "765183c2f979cd15300174a6cbeab761c53e4a2b979f9c1c628c55c69015ae5b": { "aaedfcb72829b8339998ff9e62eb6e54a69755858854804557b9efc3496e73f9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.622Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.294Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.588Z" } } }, "9bd2367031f4ad3ccaa40d2eab23421bb90a176c87631c89d0565908c1c8129d": { "a3d661f00c76cbebde5bfa666feb5af47a4620862c09e2ad2d7ea88d84d8c98d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.299Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.304Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.298Z" } } }, "a61623fa5c7f672b85c730754446bc1a65a53fbfc1fa7eb64e0779690ac3049a": { "e82d7f23954deebeb66e19daaed4363f0e28569d3a42d1de12ffdce2ad3976fb": { "jp": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.633Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.628Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:45:02.305Z" } } }, "b0c4a6145c3f1c781d51adb03f8e4996331d1159cb14cba9c81b851b728253ee": { "d161896a6a88f3dc7f188f95f5ef37b65e50579afa43c7f21b1656e07c5010a7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.658Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.663Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.667Z" } } }, "b6071010708dd9f91932364b3060488201176aeb008d6ba6dceaee25a82a0a2d": { "2007a45c3bc14f5333a4866ed3de37e1c4ce663c0e2b1fd31fbf2030fed127e0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.663Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.586Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.302Z" } } }, "bf4425dd6cb86116b19753768a5420c28985c1fcb442ecd1b5e1d37e6ca2f98f": { "e1eae6052323b0cc1ddca82febd2af06bef603d4809bc06fe09b3e2b0880ed2e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.626Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.625Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.299Z" } } }, "cdab7bf0d8c24f10d2d5dc375305c22f728e0f36fa1e29fdd04c99781fbc6cd5": { "083150d2c3def0d0736d5dbb6a695b7ea5c691ce94fcb5f5e84487727895f4ff": { "jp": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.295Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.622Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.624Z" } } }, "e93967fcdbac2bba7b89b4164ea987452cd09d1070238a59a238036fd94e8618": { "94a465a749cb716926a6ad2a66382c7591719aa2f9d792d5910f48efdc1e20e5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.293Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.304Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.297Z" } } }, "f0e219e3fb45c878fc0c3bc00fdeef1c5dd9c6ab75d1a093efffa9a0a6f002d6": { "f70bbeacf6050f44afacc1a4872c5eb1d3c4e9df491f0c452fdbd869057adb57": { "jp": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.628Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.303Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.040Z" + "updatedAt": "2025-11-29T02:45:02.303Z" } } }, "f39b12efbc001a35d87891fb78f7cc37fe27f3e15abe1f7329d97a2afc1e55dc": { "abf20812398c31c2895cbc7f3902a957857e45b0abdb831d7765f7268fac0928": { "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.617Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.651Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.615Z" } } }, "f44395a43048118c7fe3d4525c225cb5397a7fe3c98ed8d8b8fcfa08e86d5620": { "9d5c04c8e9de527ab629ee91b9ebf0d572f7863c4f88f5651c671a5fff9df8fe": { "jp": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.659Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.652Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.605Z" } } }, "f646fb33e6fccf32e79c0ff130a3e33907e8822e1555b98aa42e7679988ce2ef": { "9c48604413e046bab5cde9bba416d6f9bcc6a7ded493b091e329a27c18ad8b0a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.612Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.650Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.614Z" } } }, "fb8e6138536700f07eca78b5f157d45b6036f77f52782711c91ba183897b4c9a": { "85d1f9adecaf2dd9004cd1e79d1ecdd61c68f65285973b86e6e2ba31e2eadf2f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.041Z" + "updatedAt": "2025-11-29T02:44:56.587Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.624Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.042Z" + "updatedAt": "2025-11-29T02:44:56.627Z" } } }, "fd9477b10ed7d16ef2358b8d1e49ae2377cc94b7a2aa1d03cbf8e6ee55954611": { "36f5cb32c3341f1b52d0987870b8e971b48d9b4ccb72422d895a8e8de42aa565": { "jp": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.291Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.039Z" + "updatedAt": "2025-11-29T02:45:02.297Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:45:02.292Z" } } }, "0a48452290eff051d5083217a35dc08044c6070e028523f7cac12162494649d9": { "007d16df56ba8d29945ba611a8ebd52c915dfd07a9959101cb56729201855baa": { "jp": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.670Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.604Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.662Z" } } }, "1166fa0e4a8285a06447a2c810faea5954a70f41dac027f9312ad41f58d7980c": { "b55b582f39fbb6b1a9a83d90ec6685c4218c3e70536c2a445ad09c9e3380e0e1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.314Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:44:56.660Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.637Z" } } }, "1cc5dc60c755c1b33090726793f332fef7bb57174bac81be1fd840360abec0a9": { "0b2d9a2f1a1de345b24bb2aed0e200731bba362c09de9a98ae9041f3e9312321": { "jp": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.613Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.609Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.611Z" } } }, "1fa73f7fb3f17cb73adf9d2fd3672fb7b1bcea959cdfa4cc1cebebf9783e8493": { "68781891b0d87b8b7fc619dd4fa0e041668116f49851eeb31c8f510173e044b5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.616Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.618Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.616Z" } } }, "277327bc5d1f24036dfcf5127459029b84745c17df9cdbee699b92b7fa8c244a": { "edea05c97af2e9b00969299f942cd800726b3f980c4ecc738e093ae93dac3c2f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.318Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.310Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.320Z" } } }, "2fa7a8042be873e4594c45fc4aa944580ac9957f07dba893bd079f9bd6831739": { "d53dbb06ce9443dcb0eff1d6d827440cd3f32c6995b1495a014f731eb03474e6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.658Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.603Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.661Z" } } }, "3856af0947d30d11556c7934bf86f94916259da90023fd46b0e7c63d1c3a264d": { "13a6600b110e578ff344da3c1257c68c9f1c85b521d7074402b84a2a07f09dee": { "jp": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.664Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.665Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.606Z" } } }, "3cbdf684e4132d36432757c5b2479a68267eb108858510d7f118f4f80e1fe430": { "02a6cbb43f399b26f891350cfb238c12040d0543f4f79b9119f782c965160d27": { "jp": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.671Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.672Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.668Z" } } }, "4efac6c6f465c7a572c85feacf0f06b43a4e0c32c6c920019621529593011d4a": { "90716f5cd329825964992e1323d48a1be73c0b4afe6438deb2f5faa6947cb686": { "jp": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.611Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.610Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.610Z" } } }, "593efc50139609f8ecd70340a6cf0d42b4647499a51d8026ed014bda5df9c3be": { "d22863b43cc42cb50748f21dbf3ca52aa023402a9fd5fe4d478b8ad89b656234": { "jp": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.650Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.647Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.654Z" } } }, "64b5024b5182bfc45a505634c61271260ae40641e132a126b98fdb77fb6a7c95": { "4407c0820a47caebe5b1dfe9eff3d5de80d013db89f0925feb173cff9741369f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.319Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.321Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.320Z" } } }, "803a744763b2e971d43427be40f1173beef9290f8152a79e7047ef5f514f42d2": { "bc19380cbc2e01ee6357dbd1150e6424d9856ad286e39cddde352bb68470ab78": { "jp": { - "updatedAt": "2025-11-26T07:24:19.046Z" + "updatedAt": "2025-11-29T02:44:56.619Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.657Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.614Z" } } }, "81b00d2254d3e49a8edabeaf9d9461d8fb19914c8abfef93d05c71270dbf3786": { "96a507a0b8ed5c5846b4d8f6ffced106a8f7d73ccb668fa851fed8b3be3dbee2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.609Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.045Z" + "updatedAt": "2025-11-29T02:44:56.613Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.608Z" } } }, "81cc4a22f5345ef537b81bda612b5e1b5de5d2fb5b7d6563d33ccac4c53d47c0": { "2264f2a7ed8ccbf74a72e2d8d69d0a56cc35d7bce6b065e30464315bdeee546d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.637Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.644Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.636Z" } } }, "9229ae8ebb059ce61a183f4129a3e0da801e0d4717a314a296909aa6035f7d9e": { "fea4e84293c545f2207f795fa4b98c049df1c2de4dd7351a04e3cfb8dc162c2a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.313Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.316Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.319Z" } } }, "a9c6646ed9b12fe5c1578c74e0f8408353fc82448e8041b1c1d96f9c46e78dea": { "9cf8633b74ca4ae563d8b6514b6ee95e035b912752b8937b25e1ea6d00d6332e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.649Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.647Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.316Z" } } }, "b464890125efe481177f12e2f3c00a28cae107b65627ec59bb17ef93cf157e35": { "4a59992606ccfde9022f21ac63edbdf9bc3e1e8100eaeef04c372952f8c27195": { "jp": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.606Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.608Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.607Z" } } }, "b676683ed68be73eb9635273495e9731122ee184bb63d7293df2bdf22ebad7d0": { "81117b826442551d1cf5856c822f3d1c75ce597cd1faec68ca4ca0233ff5b395": { "jp": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:44:56.661Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:44:56.659Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.646Z" } } }, "ca8c63318081185dadfc8f0c999b2cbe8002743aa40d511bc0efe186e20e334d": { "d058a230016b4adc22efb36e3b3ae2fb018e4b84cf33b6862fd4f520d9e7d3c1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.047Z" + "updatedAt": "2025-11-29T02:44:56.621Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.044Z" + "updatedAt": "2025-11-29T02:44:56.605Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.666Z" } } }, "eb036cf7d16bf188b666a24b079c499f0e91022203931f813a7708c77c75046a": { "d269d0ef9030cc0accc4626f57a4a0fc9fa917b10cf282d13fa57388c6603e4e": { "ru": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.638Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.640Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.644Z" } } }, "f63b4898f4bc0778b3cf11bb85aa91a674b602d4005b001e2977ffa70c3be27a": { "dd2ba17bbdc49a7afba06862b9e2f43e39bf834aefeb4fadb52775d8db69d988": { "jp": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.665Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.048Z" + "updatedAt": "2025-11-29T02:44:56.667Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.049Z" + "updatedAt": "2025-11-29T02:44:56.669Z" } } }, "0850d83ea9ff1a00088e3d64a210fcd073b48986df51234fb69582c6b7fb76d6": { "9a43156c05a1578fda8031ad1f1e9faf8e97b4816647d44bffd71e1f15c3647d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.354Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.356Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.349Z" } } }, "1bb238eff17ee95c127a21dd293881a980bb8f3b0aff1bdd7ecd004fafe3764b": { "d005d0fdfdc2a2469851a9a7d27374e5fcf68c97518463c6aec7498e165ace83": { "jp": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.348Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.673Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.043Z" + "updatedAt": "2025-11-29T02:44:56.635Z" } } }, "23d2246026762ae3494ced9af104cea91f1482d8c7dae1149b7bfa3441618283": { "0e016f2ab261e197b48540cb3d3091ab6d3af62d1c883dcd3281cb2e578a1bfa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.309Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.315Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.312Z" } } }, "29f7d7e079a392736f8e8414574847d7fc12094c29074c197529b77eafd97a46": { "ee468e104feb8b3c7b0aa6d6f466b62ccd0c40d76c88efce2ee623e95b1737ef": { "jp": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.308Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.656Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:44:56.660Z" } } }, "3096aa4bb7832bb2f54010d3c5b6a248f9ebf6a366fb879f82c0eab244f815ae": { "fa532e7e71ef2e3585f03d9f864f4c524338db82a3098d4d46e1abc74f06c4fa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.318Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.315Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.309Z" } } }, "3f380b9290fe7c7d901aac0cb526ca3d522c42c21bc64b85c2e00fbdc953e794": { "e0c1c8cc04e2a4ba817680c61c7923693919ed48ab52a53f3ddf5094909767fb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.312Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.161Z" + "updatedAt": "2025-11-29T02:45:02.307Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.310Z" } } }, "492356529ca75008f683673b06635e91f3cb2d7f1097826262a7957c6cd78136": { "ea6eed1ae135ae1362375bc54a6abf4d9bda82f9cd56e95b97e329d6dfceb889": { "jp": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.643Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.641Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.652Z" } } }, "576c74bc00a8723ea19c093ffe6b3a472b9236e8f3bfcb0b95955083f9cadb86": { "351824c23a3d30665651f9a8eb9f4b521f17129ca1d202c38cbde960046a5d97": { "jp": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.642Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.648Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.646Z" } } }, "57e03de44d901875fb5eb2401640aba105efc70cc184f0f23ff04489b548b151": { "3f8e85fe2d0ca94113aa748a9047c9553cec059c087362ec30bf90a68567a495": { "jp": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:44:56.676Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.685Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.684Z" } } }, "82d75c46385806468ea3f9bb89ec325a34f8717e9925511cf3f746c6793c4178": { "56b23f6722a4743f7d9412ba74c3c4701d0fd1018ab3474c5dceb16bef9ca1c1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.690Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.684Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.688Z" } } }, "835fedb5cc4f200a51753e009ebccb9c5c2703128ecfce3dc53168f68570dd22": { "24e239e6ee1d39ee0ec39c0ebaf4dff703bef48fabe9d4ad32d9fcb51008866a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.674Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.687Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.689Z" } } }, "a218ff0160f1afb6fd940e4797a2159d55a8dbac410f179f5727b567999eaebf": { "aad6f9838da5dc15d37d5f9d16b53754eb0d3ff68a7cf73064f05eaa3669c05b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.645Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.306Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.161Z" + "updatedAt": "2025-11-29T02:45:02.307Z" } } }, "a47af53023e5932aef2db5b77a0ef7cd04c45474a2fe93ea211914667b44e5ec": { "4ff7d90419a50527c3757c649b6725b0da711648246268bc520c1dae8ad9ef97": { "jp": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.347Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.345Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:45:02.341Z" } } }, "ab35a5ab8729c47c7175e9c6cc67e42aba43c58b1e1f2c291dcda4c3977b06bd": { "02d5a608d6ee630f001b827a8fa1c5cad477766220949ac58c83c9ea965c69c2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.347Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.349Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.344Z" } } }, "cd604eef1633b62d027e3e7d70856d9553f233ca6e0180381c2120985643a86d": { "e37d6318a1605b8e2ec28a6a7b49ca74444391f022f98dec4ac9cf1024c821ed": { "jp": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.313Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.163Z" + "updatedAt": "2025-11-29T02:45:02.317Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.162Z" + "updatedAt": "2025-11-29T02:45:02.311Z" } } }, "cfd6efb64f516235ee2ecb43e9da90a4a4f49b69cd47dbfe06c9e1586fb606bd": { "dc206b93eb4f37283d194fc3cd04163bee67e631f232560183ec516accced4b0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.358Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:44:56.675Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.355Z" } } }, "daf8b3e4dde89158cbc831962f60de0ec14cecabcbd44a418f78eb071c12b0c4": { "436bd3437c6e83fc88999652218e47ef4afe3bd262aa9052fd9fbf8900aa176f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.653Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.643Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.642Z" } } }, "df45da7290d6edcd7b6391995f5058013634a6732cc0faaa6bd01d42b9804678": { "b184369e5f189b858945955301721885510add73fe070525f5c066569add5a01": { "jp": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.651Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.653Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.051Z" + "updatedAt": "2025-11-29T02:44:56.645Z" } } }, "e303e41ebcb2d5160248ecceb8943f82399ebc3323390c33a1d6a724c28354fd": { "28a231f853bc9e6425c97ca1c14dcd50898db661a90b51a9e9ef2aaf5c7c2f43": { "jp": { - "updatedAt": "2025-11-26T07:24:12.161Z" + "updatedAt": "2025-11-29T02:45:02.308Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.160Z" + "updatedAt": "2025-11-29T02:45:02.306Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.052Z" + "updatedAt": "2025-11-29T02:44:56.655Z" } } }, "f1754d0c92d25ed65027ccc750febdcca2e7101c72a0eece6697b959d9971621": { "d2cbc57bddda71b0ca36a00fdc52702ffaecf753190fb6095d4a92fca38701f1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.673Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.350Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.358Z" } } }, "ff2e4c3baefa9017265684effd06b1ae64d9d7d79efa83110c92a11de95d2c62": { "7e68dd457179debb6b3b8c9690002e92f3cfcc5539913ccfbd1d0632617d6548": { "jp": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.638Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.639Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.050Z" + "updatedAt": "2025-11-29T02:44:56.640Z" } } }, "10b704f16a650f1802b52736d2c823bd454d8b3dabb76ac91bdcc408b62420cb": { "2d4e7acb59df283f228e25658e527a973db16f341efce41e1ce84944cffa1fae": { "jp": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.337Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.384Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.338Z" } } }, "1e8eecebd2a4e411fc3037074c79ba054debc70b7a76bf53100577ec14359aee": { "5e448cd743d25dd9d490161805e048c3c2f4696c9f46b52a466a1bba220a5eae": { "jp": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.339Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.329Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.328Z" } } }, "3e8e050e4d3fc2dc532df4dd8556aae0bea35f5ab73c2aade8efe957930a412a": { "e8f4b7568afc6590d5203c133ee8873acbea759acf50b34794af4e2cd6b43ad1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.331Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.332Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.329Z" } } }, "47bec243b816f1aff8d7f27262d59edcdc28cb3ec78a655071e9178290bb0578": { "880617a38544a545b4906c62f9009009c13a7ff3ccc2a60fe2e475bb26b6f55c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.686Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.680Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.679Z" } } }, "48ff5e21581a18794244e74d86a13a93c0401d4d23c46f267ead336c36e91cce": { "42db135883af584da69bdb891c2f149df97603eb1cabc3853355aeccb9eef199": { "zh": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.383Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.381Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.385Z" } } }, "4b0ee48c4cbb9020c49cc55e309e9d7f69e89a9ed3a55e9c47bc013ae4ef6d56": { "2ed3bcd79fd5d4e72d74ac905059dc5e77bee95124595bde24fabd5f207ff65d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.330Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.336Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.327Z" } } }, "58026ac4a257b8fe268cb77f7d9da3eab8cee812a5e1d5152bab8b3250885ea9": { "75ab9ab8699432a23f95f427a4d59951ffca9690508f2d181e017be2846fba14": { "jp": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.686Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.681Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.687Z" } } }, "5879b7ee9c3de048a317f9526d9961edba8038230a7d5244320ca051c3377577": { "0a90ec2dd8b2f3498aaafcb347dfa3cda2f9f6da12d64b79f5d5404b53325b70": { "jp": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.356Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.351Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.170Z" + "updatedAt": "2025-11-29T02:45:02.351Z" } } }, "6381d5f736372e0e12418c5b0941665dfa5912b8121475ef968a4b5174f7afda": { "ca830a516bc4a6a4064bd19e68294d34a903114ae0c72112077306844ab37161": { "jp": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.680Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.678Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:44:56.677Z" } } }, "6391f6957c5f75a61373810ad0f0c8f36e0c6ab5b4e5a0f3c373ec2ec25c7f10": { "70a6df8beb04de853a1e2f9d42065e9eafda493219744deb6b08634115f9a498": { "jp": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.327Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.688Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.683Z" } } }, "65aa83e28c6b450bc0daadd14828a7677fb27a998ea9f59faacc7187462718e2": { "3c0cab0fe63f1d762905d3d204e44dff7666b23009b55e1447c9939e7032e82c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.357Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.353Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:44:56.676Z" } } }, "70c04f43190f497d5a2f8677cdc4ca3f609afb464cf98a219e9b600b7d989cf6": { "59c021fe8605f9f4ff5a62d7b51c4f5a7a05acc380d02368ad906c909dd5fa17": { "jp": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.344Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.345Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.346Z" } } }, "7ce6270ebd8e598a5ae3c7e63c99571a865d9289e493233222d36600b8ce255b": { "56a7fab051640f56124193c10c43bab0f0b30eb6b3b43860f813e4335dc69d61": { "jp": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.354Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.171Z" + "updatedAt": "2025-11-29T02:45:02.352Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:45:02.342Z" } } }, "9c1e3cb41e28946be991ff74f6b0fea3622f21ccd94c4e6553aa990de1a4f6b3": { "8fec74d1546ec055cc9bbebd756641fa7e4a28ffd600d29eaf8d88dcf521d25a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.681Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.689Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:44:56.677Z" } } }, "a82c339e6ec19dbf4bf88470d923d48a3cc71cf65c7bae8180edcebcbdffedf7": { "82e1205914218a950a532221e194e1c9da469a4477d36097b83d2a9c2fab0a25": { "jp": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.682Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.682Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.167Z" + "updatedAt": "2025-11-29T02:44:56.685Z" } } }, "ab7133a925d3667ab21eedcaa7493b04d2a7453fa0b3dd6c1545ec18333f6c93": { "3cd87edf3b014d3bf39e15bb926affe5a7484f6efe0143fd80de32aa3bf31d8a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.326Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.327Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.326Z" } } }, "abe38b651cd9f44a9de790429c92f0c07d5d279e5dae34af1329f362738d3a6a": { "0700f00685f173628dfa175ef2fa960a245c5094b60de40155456bae0cf0bece": { "jp": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.381Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.338Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.330Z" } } }, "b0af6145fc6e254fe1ec86adc9d2e005a2c78ca57a92cfbbcb204f22d2b0b206": { "ae6b07939de76cbcba1cb55d37c6d5d3944edcd60cd443a0ae6aad40a42ce5ae": { "jp": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.678Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.690Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.691Z" } } }, "ced28404e4ce6c34312f58e0fa21dc44dc32726f8881c1adb6ed189087c1b289": { "946529a7ef15a484b25d74b9a9f179b04a186b82780a2ea1059020ee8785a2e4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.343Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:45:02.342Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.169Z" + "updatedAt": "2025-11-29T02:45:02.343Z" } } }, "dd5f0d309844443578b1e477b78c685d87f106d689eab41fab33f12709affeef": { "d85b73cbceb154602514bc5dd5ccb07827a65d84bacf59d65c5ddc95c14947c5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.389Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.386Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.388Z" } } }, "e03641b78328c61b637195e74814fe2a13a4f8b55b01fc7b32ac725dd77f1098": { "d7e329d38854c95abf0c4ec667157d6c9e812a6ee76245d01dba66336ccd0ee2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.168Z" + "updatedAt": "2025-11-29T02:44:56.690Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.679Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.166Z" + "updatedAt": "2025-11-29T02:44:56.683Z" } } }, "e1f66fca49c6ff453d4e8a35fdefe650bc1596acc41c176c3c186db3c6b32dcf": { "a953eb312c126bbe30b57606749cd07b7c2b0214177b48b7f6c98c70a8a245ab": { "jp": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.340Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.333Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.384Z" } } }, "028aa3b50c80d12c1dff7886165e9713acd5da0e4c292ec8d74a396e6acb2825": { "1ba8e423cea5af1505e244428a4e315c1ec5b32bcf1289058189844c5da6dc2c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.336Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.337Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.333Z" } } }, "0ae49380ec7f5d307e31d7b631f7f0bf275d679b03f17eb67c5359b37b5242f5": { "f8739620d7524e796b898c8c185a92bf25c2ecbf9cc3893754ede05bce45736b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.325Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.324Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:45:02.323Z" } } }, "15fced5932ede7e35f56539b143eb9b8d0d01a97412450e147ef43084abe420c": { "ec90df838c140604af32f15594fffcd4af40335ecac6a833f13e0158156b0cbc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.383Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.331Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.332Z" } } }, "16db9b76d16ef49e77f68158117027a4829a5968943ae93a509257b7c447f23b": { "04685109a89dab0b5bb34aa000e61426caa176d6790eefce0141144402762ae5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.427Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.425Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.427Z" } } }, "23eb3656e923d758ff491460d9d1bbec7009131392de09276848be0db41fd269": { "3625b1be463613c8fb56424fd4d91f2d85ae950ebd8adce02c7683e4fd11be26": { "jp": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.389Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.390Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.387Z" } } }, "2f2ef25f504a5d8ae76cc6b6b38d72e25aa06fb601145bf8c4555defd3b22c9c": { "3045e21be62572632384525c8e68ac94c74ae489c9d3787b9b86c295740ce2e0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.375Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.366Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.367Z" } } }, "30adceead0e8f58341843c20ba7a1cfc58638b613d0457a74d610123f740dbae": { "e6bcf77b5129d316d4e7eeba39c108e94d974c9844395d380a2ef4f6b5f57283": { "jp": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.428Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.435Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.428Z" } } }, "32d271131b76c30bee10004cc36afd1cc48e48b098944d731a875840a3e1520b": { "483a6ba5cfe7e35e8bd7361dfddd53f126ccf034f9f7e6b101dfc108419b0192": { "jp": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.371Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.364Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.370Z" } } }, "384bbc8a5c6f8f4fd3947610412c719d2877f712b2afbd35874807dc5bf37b5d": { "56a53674a355d521b64bc7d05698ba4051acdbeaca6a3c46a2fda8b450c719e9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.165Z" + "updatedAt": "2025-11-29T02:45:02.322Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.325Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.324Z" } } }, "50e45c22e7e591fcbe4d61812d7d7a9d9626a7f94961516e9f2b08e27d3c36ca": { "4159f227f4e6ff08833e89755d03d3cec73f09d3e9171623e581edcd063d2833": { "jp": { - "updatedAt": "2025-11-26T07:24:12.164Z" + "updatedAt": "2025-11-29T02:45:02.321Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.382Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.385Z" } } }, "8b151a1a26b18205c264eb291e0e0442ddc0a8d5f8b81948e11a1cdd09758259": { "10f61a5bfa1bfc18d47b09dfd27319b441a25e084aea415d11bbbcb64e2a6c0c": { "ru": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.368Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.361Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.369Z" } } }, "b2f66c32f59c426c83078d6b24b7186f54172727a996adce08872051de770134": { "0c794fe311b38eedc683c36f0c611835c85822c536fff3e7f51e45a39493a848": { "jp": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.392Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.393Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.391Z" } } }, "b3581e0b617b1029663a70e779bab6aabd1b97807b23afe26b42a5bb82a2618a": { "38f348198e164923854caf2d5fb911a3b03dff8e5f682f59a476694465af9bd5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.377Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.372Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.376Z" } } }, "b54c21849674b2f133d9a7587a54bf895f7b1a8384d344c53348c14c442b2644": { "ddce74d3907de04d0a9af32787564ecd6b5cba8d6c36159e1e227746999b1540": { "jp": { - "updatedAt": "2025-11-26T07:24:19.053Z" + "updatedAt": "2025-11-29T02:45:02.328Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.380Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.339Z" } } }, "bf6da61b91d435b98dbe4fcfd84c30e4661211a55093b7bd5294d05df5d9018f": { "8df18a3ed0cebffed7ef2a16c2c1feed24d08b38743943e1639bf2e1e83ad9cd": { "jp": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.395Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.391Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.388Z" } } }, "c600219b9f55bdfcea82201926bfe9e4cabf53497d2110e11a6a97d3a6de16d1": { "879e570e6a755b5436d4b4e3e5ee02f6ef2f2b1b56d5e30a0d8ad6d11079deec": { "jp": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.373Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.371Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.363Z" } } }, "d20c2004eff27206aa611fa47101376ca27b19c79a7c22fef935d90c8c7ee0b7": { "31528a8c4089ac02ac4c5cae45bfcf8375faba7dbb39d635e3082a39955f5a65": { "jp": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.386Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.057Z" + "updatedAt": "2025-11-29T02:45:02.387Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.056Z" + "updatedAt": "2025-11-29T02:45:02.382Z" } } }, "d42c8393402232b95f473bddaaa33ac9663e18e070bfb5225b9240cded76bd36": { "469a531fc6c1dbbcdaf79cbc24df46624ad5a44a7c52da48e4665690d6de2002": { "jp": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.362Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.374Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.364Z" } } }, "d55ab4d59e8e430728299d153babb7440fdf1524f75ae30ac017602a393f72f2": { "e946a51dbbf49a6bb72dfb7320ddc89e75e9bca19562498770b9375217a83d34": { "jp": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.360Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.368Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.366Z" } } }, "e9e6900149061b39fd6dd6fa53d0c99f28ffac38d503ec961dd94dce5ebac808": { "aef65ce3391d03e363f980b73f3fa71276203fc5f77a1d75edec615250031f8e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.055Z" + "updatedAt": "2025-11-29T02:45:02.340Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.332Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.054Z" + "updatedAt": "2025-11-29T02:45:02.335Z" } } }, "f5e923aaae110b8d3ec030f52c1731f515c0ed1b9a0e41490e863bb6395bd23b": { "c81f4b30001e6233066eddc0f7a5c166b4369eee24cb505fee91004bc16f3b48": { "jp": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.393Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.394Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.392Z" } } }, "1d0e04973f4a7a2726ce086465182e22cfc8de26b7036f67bf3246dcdcab5c87": { "31f058ab67c32c0251f087188700872a277440d4f0ff0bd41cdc2a390207f441": { "jp": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.512Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.517Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.515Z" } } }, "1d411ae967753b5d27acfdc77c2f68fa873d228cea6cf769ee2c85f10b38628f": { "8c9d1bbb63ac91b1a18b930594b6d354536b4a42a4cefa28e167390053f64f41": { "jp": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.906Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.908Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.901Z" } } }, "32a2dfa24b35817a5fedbfc4895185da11ba73834f024a8c145cb60b3ee324a3": { "8f13f0e888bb91b30f7b56131bf3728f2950f55c2375b05eab6a6c9cabcab037": { "jp": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.861Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.910Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.910Z" } } }, "34fe9aa819ffc70ef68be0505c66c5cb60f94370bfce6edd29d0ef846b1eb245": { "7ef9c6e569280d6e03a986898ccf237a939f4581319206934f40b7e910987b98": { "jp": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.952Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.952Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.953Z" } } }, "5a1049606d2ddeb908a3f87e08c53c766115a2d5315cd4e891c852fa240471ed": { "4340b6e9c5ca9bb508ff61e1f7de601fd3ee092842be32670cf541dd9fe5b76c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.904Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.908Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.904Z" } } }, "6c930d7e263cee0da201aeb82b5afa15d7a0492edd3f17b70d744502c7da16c8": { "2c78d1148a39342c324f60ab8fd48891049dd3af4b2e04e98d60136cac22dac8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.514Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.513Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.479Z" } } }, "7997000584a74b3a4893e2d952e3e74901f5c48d13d2477040f08510ce7fb94a": { "f3a543f784ce343388875d80bf6932364452e41d5c499c0fcdb6193cbc18d2ac": { "jp": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.370Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.360Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.377Z" } } }, "7aeb5a3c848c3ac6401e3621b9731a411c3ffe53b1ec386f511089c819780c4c": { "1f0a4b693ba5e0ec268fafbbe5f0a583b29cfd716f04abb61d43c5813b6ad612": { "jp": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.903Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.898Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.900Z" } } }, "7af81b34b1f80a6579a084fc3f8d1ecb9f0315e228a3b01eca34abc4e963fda6": { "c20825094b802738f9e5eb45bd5ac1dadaadc926f348ad24d8c06cc4e5157994": { "jp": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.897Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.896Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.895Z" } } }, "83eab82a7ad67622f732d278303fd5a55d015c462467d35a81a97662bdec853e": { "2d649e303741fd66ea1aa56354d590ebd300f6ec9c2b2ef22c28c636be7a29cc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.375Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.058Z" + "updatedAt": "2025-11-29T02:45:02.361Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.369Z" } } }, "8aef57a5d0702946541ef4bc66a35386c47ef94c0fbc0f60abf1cf7cff964601": { "1de18ab03988e32b892f506405ca6a01d5a611302a852d3f5e7de174a37be78b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.373Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.172Z" + "updatedAt": "2025-11-29T02:45:02.359Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.378Z" } } }, "a2ec760009faa1e1eff2c135a3d4deb7afa6a079dda0c6d9f99db627647062d5": { "4f03a97491bdbb54d341d453335aff270c60976e7c3ad96cb719e9003ee5ad0c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.909Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.897Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.901Z" } } }, "a81ad531cd4308314f95a3bc7ee7518076cb8b225330a76bdebb309de6c07d84": { "eb1a10c317b4f12f9023e3b4899a6403eac245683d867b105338963ab1df00ca": { "jp": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.518Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.515Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.516Z" } } }, "a8b3a4c7be16228ce7b50cb870cc58cfe39f8c34bd28a3aca5822b90b0f42830": { "f2435d45557de24d303d66a742aeff55e64e2f4b580432c1d1d9f8eaeb1f5d17": { "jp": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.426Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.426Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.424Z" } } }, "b2dcbd4e41cb07eefcbc269f5df931324f8744a9483f6b145243bbc5673c42c1": { "5890daa9787c7983a0d917f5622f02d272e85c52daeee1444ef64b42ce8108d7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.517Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.514Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.513Z" } } }, "db411e0514092e58a10e4b885faa2126f95d2bd39dace283d1e44cbc9831e3dd": { "527580835a672b74a709bacb51a246aba1c88246216cdba2db279817225f4044": { "jp": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.511Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.185Z" + "updatedAt": "2025-11-29T02:45:02.518Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.512Z" } } }, "dc3682d31d860920c0027dc94b51e1f197c5a38ca754f403922910b9b8ba3903": { "668b968f7ffa7b6faf894697548c553b64afd08c5b62258b0eb445aab83c7d88": { "jp": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.905Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.950Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.905Z" } } }, "e72fb86764359e026d92c8940ee6175f5febdbd710006033850bb2ad8aa43023": { "10e1df69f27be8e1de4c2159ec11f7a83395eb9a20a7b729e0fbe4c2bc8bb473": { "jp": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.899Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.899Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.906Z" } } }, "ea7e5e311ec73e96e57ec3343b5c4d7cd8d2c758deae9104dffeb15243a22097": { "a6b1a10073ba1bedb61ae0ed5088f394cf79fd30feddaa919ee25e9e0f4c991c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.951Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.909Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.951Z" } } }, "f46404d0d2b932ed24233530122a903e98fd0ad2e866b50bb50ad16e35006e6f": { "ce6bd20ee80f6f7df45c614920f103f5eb64699dca884aa2e9a55c8adbfcc913": { "jp": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.907Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.903Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.902Z" } } }, "f6103a7698b24fef604602086936cf148c11df516f6f84bf99b48971614c717b": { "2934cd253b5a2e39a317ce455fc2c1d9f94f60e9c0af926ce756c8e2261a0354": { "jp": { - "updatedAt": "2025-11-26T07:24:12.235Z" + "updatedAt": "2025-11-29T02:45:02.907Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.898Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.234Z" + "updatedAt": "2025-11-29T02:45:02.900Z" } } }, "05f8d1acdb9d8a92c6735e4d5dcf8080fa8ee6512cc13dbf3b840c999a094c71": { "97638cef9fdf5d6328f466c856175463ac017bac4780f1d817b5d4729a88aa08": { "jp": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.886Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.883Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.232Z" + "updatedAt": "2025-11-29T02:45:02.887Z" } } }, "0c936deece1cfa87a5970fb553569967ce05687698de65a98ef0315477967bbd": { "a922d6b0d8e112391f7d053fc7058eb1d5659b44c4a9dfa835485d17fbead31d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.927Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.923Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.923Z" } } }, "1582ff8ea3fdbeb1dad986160d1b0999795a555f6d89e98dd145b6f49dfb08eb": { "5e343ab5ab03d0e1fa46bf003992f1eb136b9a12bfad77828128edf71d3afe32": { "jp": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.927Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.921Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.926Z" } } }, "179dbf5bb80545989b2913aca22d0861999dba14106d2380864014877de3c93b": { "114ef0735c99933d93e4c6a570fccf1ca3ef45aed471b8a4eccb902e87cb5043": { "jp": { - "updatedAt": "2025-11-26T07:24:12.232Z" + "updatedAt": "2025-11-29T02:45:02.887Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.863Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.878Z" } } }, "1dccccf586631074a6cd966272c09df3578cce225321b7df5ebc807acd0dcdfb": { "b435aec19ff6ecbb9d88c6d1f945636177e245c9c227442437f370098f0f3e09": { "jp": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.997Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.994Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.997Z" } } }, "2a3e385a0edab430e986558c8176d5e5093f020848f61371fce764ff9195f165": { "b8228ee3face15f90f6ed1245de3feab742bd22410c8360b5dcc4e855e71c22d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.922Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.922Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.924Z" } } }, "2bb9b38a8d5dfd619ee7e2a01589dd2c06c59b11f82f178133c39690b45125c5": { "21f979e19600cd98d3791382f305b11aed31990ab9b8c6cfdaf57719effc558d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.940Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.938Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.931Z" } } }, "32b3dc73599ca183244dc71ff36bc88e62757e5face12c31b14ce042f684120c": { "1bb063448241263bf2f6dc2f55489a21d5cd06be00886e0e9e91d6bceacc47ba": { "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.942Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.946Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.943Z" } } }, "51c48794a66e183ba70935eac117d954a1401f40572a0afc11169b24fcd14820": { "dc661924dc7cd06d16b7ed5abfda37c2ece415c277427ada79d811eff748ebda": { "jp": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.894Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.232Z" + "updatedAt": "2025-11-29T02:45:02.892Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.878Z" } } }, "5565bc89634d0648d7fb44f41fcd9352657cc2b36d57392f0a6561a32e66eb28": { "d223905451f4d931e0e856ce3fd5f35c1c3c25396ff43780337894e768a7242b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.884Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.233Z" + "updatedAt": "2025-11-29T02:45:02.894Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.883Z" } } }, "705e7aed31578540442c080a6cafebaeba2bf1ddb38ec739dd014aec5b25502b": { "29a6c789509cb2e9a587186b93902ad76eec1850c4f01f91eb5c2a4c186d557d": { "zh": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.948Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.946Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.944Z" } } }, "7e47d90d43125cfd56ce110d9bfe1a08ac0c8cecbad7095afeda215f8ebaff80": { "6aa7a3b849b9da4b7d84bb26a3754ab6d9c56ee35825fa788436cb306b81fc00": { "jp": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.861Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.862Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.884Z" } } }, "9368e9ef7da2d3545fdcad02056a63f297099ae569a58d6445ec4175f477bcf7": { "5294da061b84e38e7a5c72fa3738434b348d3c948072b63438f6f8e9041f8d45": { "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.937Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.925Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.929Z" } } }, @@ -6937,18 +6997,26 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.232Z" } + }, + "038b14ac5c1893d1111af35826b4c74e0b753cba46c799f2102d96ef3edb9d42": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.380Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.389Z" + } } }, "9d935527c3051f00d3c44516b5c5b43d9ec31ba4d1ca19553b784a772963e4d6": { "b415e1612fa4c875f71cf858dcdd92606355f03dd3c13b5aef37f79f279ada0c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.938Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.933Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.924Z" } } }, @@ -6966,429 +7034,429 @@ }, "e607066649cfbd50b16b74b56125f76fc7dae7885df303cc620af28c763ce899": { "ru": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.939Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.932Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.936Z" } } }, "ca84649ef742e7064e2d857290ef9d942fcc1d6b9bdfff1813fcdfdbefec62ff": { "555cc07d313afdfd168b7ad11d02f0ab80d39cc85a07b294b44c7401c7ce9620": { "jp": { - "updatedAt": "2025-11-26T07:24:12.240Z" + "updatedAt": "2025-11-29T02:45:02.939Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.929Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.930Z" } } }, "d8908fc8af7a3068c0cc48f8107adaf5bf331be7388208aa9a40ca7f00432b7f": { "561bda26e259939457123ba760b1c473d1ffa5cabb632bd41b00a30024d8ae4e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.232Z" + "updatedAt": "2025-11-29T02:45:02.888Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.231Z" + "updatedAt": "2025-11-29T02:45:02.885Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.232Z" + "updatedAt": "2025-11-29T02:45:02.893Z" } } }, "dbd0d5161d0bd3efeb5fcda68e773df51262f2852a70440882d847c3e8ed79ff": { "558ea55eedb29b8236de463bdebed17358b2ffd17236ba1c7d0c9758543b7b74": { "jp": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.986Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.944Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.949Z" } } }, "df35030ac136ab8f4825db977a911e1874670c6ba73d0e658e95b28fa50c1d70": { "4cebcab8322a8aae73bf90996e95a2e65bb56b941371c8819373abaf169e92db": { "jp": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.935Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.928Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.935Z" } } }, "ee20bc66651b66977783ce3a17b9d4f38b09b4a0774e0791bb9fb26a7f930500": { "e7338142de8dacc4a6fc04e51a78c9dd1fb3bbef6534057d60f8de1db6ed3aab": { "jp": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.934Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.928Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.933Z" } } }, "fe7e045fa5f538d00f569d58a48e0a9285abe27807a38b3ce253116b4cc22e74": { "c2d3019dfd5c9e95d0bc93db0189ffd3ae5bb907d47f6a727f23a3e435164059": { "jp": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.930Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.238Z" + "updatedAt": "2025-11-29T02:45:02.926Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.239Z" + "updatedAt": "2025-11-29T02:45:02.936Z" } } }, "26480489190477329712e0e890231f9ee67f7bae2ec93f1adc5e49bd8705dd0b": { "ca234a63cfee1038a0b6bb5b7e10d7ef8307e9e5239cd0706669420fd2cb62a3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.995Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.995Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.996Z" } } }, "356c6ff78cff0c4de1af14bfafe2c9bd10139292cd3f3c3553d242bfb277d994": { "cf5d9fa224a574f45a3c02cbc85a2617672d37fcaddc77e5adcfc9fa74e326b1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.967Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.968Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.983Z" } } }, "372be1b1091279b14a64c301dd32f570d8ae7c28ebc2b0e65c8d600412c8a6b2": { "24a1775ccfe9d94dbe6ee2e71f12bbcddd22da3de1dd49f2d8ce8e542b33728c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.996Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.987Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.994Z" } } }, "3b4bb74db846ca0f012ad71dfdb33334fa8118040393487ad35fea48bd2470ea": { "3120f1e4d4f08a6ba69af7daa70ffa13d27c3a4aef713d36140278c033dcf2bc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.960Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.965Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.970Z" } } }, "42ae3a5b453abe44edf7cc0c8fb18a3559a3043e9828ca9eecf69cbab0362ecd": { "fb18df11b1efd0c29cdbcd9a0fef8f8e09542882ba6ccb09e3e42d9f3b8aa419": { "jp": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.976Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.965Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.983Z" } } }, "501db638650e5304a9dba8ff4612de47b5da82aaad0a722bd89c11c68a35eb5d": { "f925e25aa54c252061995e84db9939551b2e2035ef3360d06582d778617a054f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.975Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:45:02.993Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.972Z" } } }, "5391d9361d8de859f55fc623438785f034d27921eaf51522b1cfec0b8ae6d057": { "4c5301e6bd068db1c39c7442930c97eb64fc020a710f75519ea91e088c153887": { "jp": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.967Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.968Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.968Z" } } }, "64565318cadde7f90ba96c3e29513ba020adf44fe66a9bf3e5482d23d0dd47dc": { "63452898bc1a5638b696f345c28ff8083c41b2223f3638a2c64f25800a2a5647": { "jp": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.716Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.717Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.703Z" } } }, "68ba9608dff675f309e6f07ee6d6f770a417b027a738a79f138c8d70e2106dbc": { "9dc2946bda2aea97fa9b18c311317369a59c2adf656d6ce6d76316a813616fc1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.991Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.706Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.974Z" } } }, "78fe6d3b89afce471181d779a6a8b475696095ab4ef58d29771279afa02b2997": { "79d3b0b826a742e9b7895789e7402d878b568cd9e4df76a133dc77a70f03c8c0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.947Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.943Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.949Z" } } }, "81915656e6d382d86e051a8fa78d36209f8322f00df9d519bd2aba85055926e2": { "4bc52b2d49860b621c0c2e9203206add44f60ae74179555c48eff9366de95cc3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.985Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.982Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.984Z" } } }, "89db72da570e81ebcb6f667a907e2f846f64923d46a9947f6788299488af58fc": { "bc1f7fd0c55c3e925412c0e368a4ffa88b8fe5c39a7aa535303e0d54e76f2b9c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.947Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.985Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.988Z" } } }, "938d56b6044b6cebcfe8b337190fa6dea927660551790620ca8c19fb31cd39ba": { "2aefd9ad0393f63b7e1ec0b002323afaa8b544c1011e8f3c91b77ac1f84ef487": { "jp": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.990Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.986Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.991Z" } } }, "96e31c277d43b145242840ad838f44b908ce963c352dad86b59211265e87b591": { "482a21b0c27c50eedb13f76a309205d6a1f064bddbb03002a77af2aa8fd7cc3c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:45:02.993Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.711Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.716Z" } } }, "99ff98bca369584f25c59d8f96acd6c1788719989416cfe1d5d478919758fd86": { "139a2b803dd22a097a0fb93f4bf76cd3187b48224be1271d561ce8d6d3b0bdfd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.976Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.971Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.977Z" } } }, "b2aff55ca5954a6970b9f52ac23fc39fc004e51a346a6cd693caccb1417c6519": { "1010abd84c38f96762ed3b8cb461a3bb4e5e229304c1c500e26dc7c6e9d01318": { "jp": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.948Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.241Z" + "updatedAt": "2025-11-29T02:45:02.945Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.988Z" } } }, "b8c212ea80c9bdcc2ba8434c82489b4cd25a84157ab8881924465e669bf2bf1d": { "aad4076142416380448496fbac36524304c81991e5c00dade2ad95e55a087c94": { "ru": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.959Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.992Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.959Z" } } }, "cb227df00b6e64305168553956c1928afd33de9cb76c9d330e9c9eca9290c33e": { "268a8df1fdc77541fc0a6bc99e66097367ea72724a49b591b16c19e00e6685fe": { "jp": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.969Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.973Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:45:02.990Z" } } }, "d1c3b4df71214a3e88455cadb9dda32802eabf8a18de9dd12b4636f3a20001bb": { "407735ce33f5163b7e6c2875f0e2414993e84109f0556ba297b7f1762f038a8a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.977Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.973Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.961Z" } } }, "e9a7a6821acf2148d5fdf59dfb02c842dbeccfe3db8ed78b13af93341b542d82": { "45af94df7fb72c57f3c3954a12bae535b5025b01d4824ae9e4f23b2ab156e1ec": { "jp": { - "updatedAt": "2025-11-26T07:24:12.243Z" + "updatedAt": "2025-11-29T02:44:56.709Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.978Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.242Z" + "updatedAt": "2025-11-29T02:45:02.950Z" } } }, "fee5d5e407a8306e3abcff87b3f147641c908588b209b7c9e107759067db235d": { "35cee660251b87c86ad32e1c0bdaaefadc8dc8d26b278a55c87e87e3de226353": { "jp": { - "updatedAt": "2025-11-26T07:24:12.247Z" + "updatedAt": "2025-11-29T02:45:02.974Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.989Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.705Z" } } }, "16f8cdbdb2db2ec0c442adb2d731ad72ac56b9c830104c642c7d2a3b0353bb51": { "81890c8843157d3e93683e2432db1962f9cd9f3a81498ad22050145df32d70f2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.697Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:45:03.009Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:44:56.693Z" } } }, "22107d57f939a679e2605620f4964bd50775a98cf0a725bb73b27f34f48bb6a8": { "8bee2f4f365485c8773bece43ccdc523ab14c91e5b4d3f38ac552aa09ea55c97": { "jp": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.701Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.695Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.697Z" } } }, "23af5cac91f252ffe2e42d1e7b5a0bcabe7dc844aed8ebeffba1570964d40b4d": { "897a5b0e6ee3fe28e1f105bc25b952d48f233f747b27270188a83040b9b40f90": { "jp": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.969Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.960Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.966Z" } } }, "256b6f8980e5b71a4462fde393b7342ea3301cc72724522761120a29f082680b": { "479ac94cfa44c4916c5e0c53e04c1c5a735ae40232807e8ee57eb3c72f5a8bcd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.009Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.007Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.007Z" } } }, "2a50f26ed5a74514a1bb5535e77a1e4295586acbc14137eeb91bebd950369fe9": { "77daddd248c06a3945d845d9935148cb7d185c9ace0f5a7e2b8d9a52649050c4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.699Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.709Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.715Z" } } }, @@ -7403,96 +7471,104 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.253Z" } + }, + "adee3628812a2e0169c7c436f7c41012c6b0b856ab91c598890be0b181284e63": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.384Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.390Z" + } } }, "55c4dba0327f61353da1e2bb79f11bd89ee95b926dd1a86a38c82a8837d28c4b": { "45ee568fd070f4841a6bc4d3d5720231ff8e0b6b95e3b6558e9f606b852b17aa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:45:03.016Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:45:03.016Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:45:03.015Z" } } }, "6e73db155b7c6964fced099cd2a329a54c570e4567c1e741e45991462993ff89": { "d1aadc2b06df5561a41ec6294f8ba38c60368402b06032d12e12420507c14384": { "jp": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.710Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.703Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.972Z" } } }, "854411037d5e91dafe4510e3bb749eb29c1405966f5c747972f003bea369b464": { "2f5dd362e6719f95a9f300225eac5ed8491245ba11f15bda272d36325d991c01": { "jp": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:03.003Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.961Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.245Z" + "updatedAt": "2025-11-29T02:45:02.966Z" } } }, "906c5c00462e8461e0b7aa1cffaec1f44d3cc275066f474f9ab70cccbf9e9d8d": { "661e85a9d5e8d39ed88218a74a7029ed28519c2e3ed3213707133a5bb6e243c6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:45:03.010Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.701Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.694Z" } } }, "9acecbbe697d2e6d2e334b3b54c514cdcf0ed3d6c83e6748104f8f3b983abbd2": { "4b6046e5cde03661005f0be0ef3f23e778a948c6c005456f94af71b6ea2e484b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.707Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.696Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:44:56.692Z" } } }, "9e489d6bd5311c686f9d5b9d6362b8dc0665508e0aee22c424f6d5045646470b": { "3428be07ec229b0c8189ab88dcf64befbe6259ed64195b3646ffba1c30f4f99c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.710Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.699Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.708Z" } } }, "9ea307eb644dfcb063c696ac334d7efc3c4116379021488a8c90f62910a0ada4": { "fbffc80f20e5309f443ac78918570ef083994e1d627e7abc163966be69f97bbc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.718Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.719Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.989Z" } } }, @@ -7510,104 +7586,104 @@ }, "23da89ecb9cb7533ec667d1796578f33ffa62f9e868634d354c4e0fc15e995c0": { "zh": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.999Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:45:03.001Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:45:03.002Z" } } }, "b9b9941c05a9402833caf9876a84c15fb7e08ee5f6ebf347c9f6516a00134c17": { "c8b48a439aa45e1012fed359c6c51f552f6e1298e6483f894f01fb76cfeb566a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:44:56.749Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.006Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.714Z" } } }, "caf9155f2ad3c6bb6165f0c5a837f80ca0f324d7821ee36716d6a44981b32432": { "c9a20f8ca6d2167945584243cb48aae584ce849963b883da031cb1fa3b57b9d3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.712Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.705Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.706Z" } } }, "cbb612322707858e39d9de4d0c9cc540429b50cdf2909447e753d421fc3212d0": { "4a7d4ef89d791edabbdff46a2878745843ca285c2985ee018c727274960745d4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.249Z" + "updatedAt": "2025-11-29T02:44:56.711Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.992Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.248Z" + "updatedAt": "2025-11-29T02:45:02.984Z" } } }, "cee825351a39183c37f3496b305a2ca1f4323197e9abbe5c4516b74419ab1ae5": { "a0695d4dcbda273aa6429b9ac871a434d4a137ecabd11538b66c6a141105e98c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.714Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.998Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:45:03.010Z" } } }, "dd2163ae2a4256a6c45cea7405483741ac924731427ca35e048e40dbacd4926b": { "1344db208f73dc76c68bb5a882fbdde264c6a27beb40c33d6d5d97679b2e075e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:45:03.012Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:45:03.013Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:45:03.014Z" } } }, "eedd808236db61e2b28ca3ea587227703d2be3b1ced3ffbe6e92ba89ef707e94": { "20f04dac4a93b0fdb3374aba9dc0994fdc280c6ebad124568bf3fd2f999185f8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.970Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.958Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.246Z" + "updatedAt": "2025-11-29T02:45:02.971Z" } } }, "f2d5e46cdcc837e8872864afe7d6050b24a75248c1c3196bb56ddb850f613f07": { "cddb29e069eb3f550f6f7bfa2961bf6e582620eb41b7307ae5bd3772e689f84e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:45:03.013Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.254Z" + "updatedAt": "2025-11-29T02:44:56.698Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:45:03.012Z" } } }, @@ -7622,1214 +7698,1222 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.259Z" } + }, + "be7f4e3331c3fd409e0646bffe9b6357649ebe66e4221085977b0cbfb8bd4a24": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.382Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.386Z" + } } }, "115c23898dca6a5bd85fc79980e071e10196e3e3295527809805baad03df1e8e": { "cc5d85e7940e700fd5d3f8fd7641a3e19d24a033b3c45b51595134cdc91659d3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.049Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.739Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.747Z" } } }, "25fa138ccb807e454af6642c2ed448968e7de55919fd0d0a4ecb3f6e490a397c": { "ab68507bc825afafe53c6d1d0d6f08c53621f3a95a39deec3a4dad7ef103b2c6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.704Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.256Z" + "updatedAt": "2025-11-29T02:44:56.717Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.708Z" } } }, "29098b8e3f1e1a679a5ddc94379ef95f05ce5d74ad32854eb1f4dbf472997cd8": { "a2fdefeb5c115c0929ae0f70cb0135e6ff4857188e411761888474889ae1edda": { "jp": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.744Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.745Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.740Z" } } }, "2e279d80c8ba84fded6bc29580d38a57165294e3bb9ec5ac3177d8fa43594ce7": { "c32887dbd37129abcf60580789e56e42295b227409b866e8d6f639ccb4436f91": { "jp": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:44:56.748Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.047Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.747Z" } } }, "509f6ede51ab34e339503f91928010a06f04655f9ae29650958c5b6768752931": { "b15b0f51d35014ff5faa6f96548eae990708c240d294f1b231da328da35a7588": { "jp": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.008Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:45:03.004Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:45:03.004Z" } } }, "521e12e9546adbbc16980431e680a5ef21ea7b5b3b9b36afb8a2521aa6b377b6": { "6e547ac81c7773f9acb16ff8e8b7c7388a98727bfc4319c29909249791e4ec09": { "jp": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.725Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.734Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.727Z" } } }, "543fafeba882f7e65ffa713c52cc503e06a45708cf5d17f53ac0462449accbf7": { "10b537976cc0e91e97a168611992f05f85e4ed7084a47e4cb1a2f920f41380ac": { "jp": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.730Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.722Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.728Z" } } }, "700af028231b046bfc9ddd5cfa321b3be5e023aaaee235d4d7d86453223b3fdc": { "5feb43870c53151fcd38f8407b9a14613518ef335101c53aa526f6a23caac7ed": { "jp": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:45:03.003Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.007Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:45:03.005Z" } } }, "7d89286b9deca9673424f7ef6646b56ff094fc99d83fea068c7a5b42f3b41f58": { "0cbe2538e6234e4c37b13a2b8892c989968a97ca42dd985663e6d7efb804eeb9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.746Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.049Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.047Z" } } }, "7eb439b32a67cfb0aa3624c9184253dc089e7da15d7e10a23f668083dcbbdb63": { "d75745d1b46f0de5b2028a881660f2bd2ddadc7ddc0b54286beaca30e215e44f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.719Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.048Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.050Z" } } }, "8cd1456e58e9b0f32764599fe1b3c08b4549cd901e4ebe5d8ff994983ffb18dd": { "be2df94d3de1df0b087713bb38516d1a78f6b4313e8daf18309af45c6beb735b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.740Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.733Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.742Z" } } }, "8dbad11f22f37a8dfbe5928d8a4733fffad030ebf6032dcfecd084e9101dba52": { "f92ca8e97f1895ba9a62cdd9bd09b067b16fb3472cb748d5ec26c6d2830bdcc3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.735Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.737Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.733Z" } } }, "9606738dfb47e926dbb72401f97fb8dcdca15e8e7e4c7c8e0b1de1923f128ebd": { "f38bca2728a4ec18acf3801a37e29bd6ce1663c505004c92a4ef0fb8bcfab83d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:44:56.729Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.251Z" + "updatedAt": "2025-11-29T02:44:56.723Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.252Z" + "updatedAt": "2025-11-29T02:44:56.730Z" } } }, "9879a8ecb21ed941282ca62ac8cd46ca90a2e07bea45df3014931af580b18b1c": { "1cee6eed8b351ab527a9d9c859764f01e20c33109d8796baaf74d0bfe5e7498a": { "zh": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.734Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.738Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.738Z" } } }, "9c64eb3f63ed2f4471f8cc3e3a16b5d6f44f4c39e15dce1c2c911d1a94e1a018": { "4af09e0c2db5842c3ba3437a58d8012e6ed6971aac46840180567463da4f8ce8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.244Z" + "updatedAt": "2025-11-29T02:45:02.998Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:45:03.011Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.257Z" + "updatedAt": "2025-11-29T02:44:56.750Z" } } }, "a2fd395ad42270710df1127e0482607ea48ccfe81a62976bedb63b46c8ceb860": { "67cbbbbf1e4f7f85554eebfe9fb09a5afe145f060eefe6aed1c811dfc5891361": { "jp": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.735Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.739Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.730Z" } } }, "aff3738ef426bb03f782516f0c962dc0d4f1e8b1e75422276233e8a61abcbbf9": { "62fbdd6dddf79ab74c534883a022557ea5c732ed713d1fc244291ba771204269": { "jp": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:44:56.694Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:44:56.691Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.253Z" + "updatedAt": "2025-11-29T02:44:56.692Z" } } }, "c61ff854a1d65abf94d196412aea9f3db52e099f903e0aec1c8dbda684f0ee4c": { "6725d42405abcd2763e59c5af20b80e294c49a24e5dfded57358991054e676ae": { "jp": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.732Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.721Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.720Z" } } }, "c85b0e977e47a5de069cf6bc2a4c3c7c368f637081c6c7a74c2b3f09f541da76": { "6a1875203c3c11a5ddaeaf844592c8aa66c906a5f10d8118af659f3188166f2b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.732Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.726Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.728Z" } } }, "cfcb155375b8c7dce0cd7951038c468106245eabdd22e87ceb685a86ad5787b1": { "4f1c6f9f3c784ede710c284000e57bbb2570ca34ccf377e55bb0aa62d9575fb3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:45:03.048Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.743Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.739Z" } } }, "e771f00ee03a6b8ac3a2fe4466ecae0a0ef5fa4a1c06261040efd4c71c7df8ca": { "afaf81983280a59e7aa1584371969108a9f08bbf39abdc8489d3da2cc68c29c7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.700Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.704Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.255Z" + "updatedAt": "2025-11-29T02:44:56.702Z" } } }, "003cc65643f9d9786893e0bde4fee0fde5fc25de83cb44c9b184c9f67f682330": { "7bfbb7c49650987bfda71358fcdb6c75e10f3775e57dd80dfa998cd9df1e42b1": { "ru": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.038Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.084Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:45:03.017Z" } } }, "0bf287012c3e4a1823f4a6d9af97b4ff2ebf50382b88f6e446f2d2462ceff028": { "6fc59c979e71f5ef7d01dffb85d9c0d52f0f7d9af3f0d2364ea573c821dfb4a9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.028Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.036Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.030Z" } } }, "12e31ec44b3dcf65828805450d562ba22004284c24e16fe91cc2a8306183626b": { "9894639ef964614d3ef8027f22a7deb78a5ccec89d41e007f288e7db21591494": { "jp": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.037Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.045Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.030Z" } } }, "1a8c3dc523efbedd8ceca5a1bf0b315be2ac1dcf90f08530d461bd213eef4f7c": { "da9e17112c0ec79d1fa82ab5f0ca3db1c53729e70e3fd6a2c4370c03691b292c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.041Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.042Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.036Z" } } }, "227d51f47fc957dea766831ea43b73c58c9e450c7aadba923fb55f27b830acd0": { "88ce0d6c08629f221dcfe109d7e8a09898443472a62411ee8e84cd0cd4e77851": { "jp": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.032Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.026Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:03.031Z" } } }, "2788d1737d33b8bd86e0aa8f0dbd2c1bed226411e50160a1554ab9361f7532d2": { "d0cbc85c85d4d71c67952d11b3d238be8fc75b6ea16860b09935bd9f96add653": { "jp": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.025Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.020Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.024Z" } } }, "3b065a4f3fc6b25a5184da43b7b0221b5aeccf7b81e1255bd8a6d2a6b86a8ae7": { "c88ae622109bfb3777e96a49c9bfa5f9889a8187d65d687676ef5de1bf070514": { "jp": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.742Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.746Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.261Z" + "updatedAt": "2025-11-29T02:44:56.741Z" } } }, "3ffea18e4142d273a23435211934d60695e426723e88ea42a887c753673da12c": { "9135666001d3b0d949ff7db424b18a4b655d4b8eebcafa75a9e472d040fbb808": { "jp": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.040Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.042Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.040Z" } } }, "6ae9dde7cd947f044ac422d9819b807221ad5825d4f6859ff2c72f3c22d7331f": { "f17b1d4769177c8b7b3260aff487e581de4450f37dd2fbeff3e0a899b7559706": { "jp": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.023Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.031Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.034Z" } } }, "95219024ef9522a55be4e6513f75defbb49883b4a5e32a05d187bbbcc9f53c16": { "069a5c20a99f64397b1d13060b06470148c26b5072a36b8e1b16d746b0e4ad7f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.034Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:45:03.017Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.023Z" } } }, "9829e6d504f03f89cf3f9782f9238f3dec6efd6d3810dd672ec15bd67e65f810": { "e59e26adb9705f2e6456ed1518f0aefb7d0cf0e3b13b040fa78b4a590a1181c2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.022Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.022Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.024Z" } } }, "9f51461ed5499a8b1f450f23f773f55200d3922c76578fac080589c6d4bdb7c9": { "eaded5b9cc370f7c0893d58a270227dd93fe67bc1568a6b674bdee429e92ac10": { "jp": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.033Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.029Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.038Z" } } }, "a4186d2152fae14c248c1297810d8ae84b17536d8f68513f586c1e2d378d79fa": { "da62d5ba1b9b52d86fdf52ef9a5a5fce77010670db44844630fe457d0a64dfda": { "jp": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.028Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.026Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.035Z" } } }, "a98f06f78a3ec0f29bb4f078dbb0c37f77d01618cebf2733ff11b32c497f7b24": { "a9a69fd4a89753f57c102accc6affd4752db865e189ae4cc4e551815c20e9964": { "jp": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.724Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.726Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.723Z" } } }, "b0f8d850504855a8481784c04ab4b0c0a35453e0ccfb3fd1251528b4f77a8b8f": { "0dbab51aa36f5b479c39c4f615a8a9b4493aeae6b1e482a4ccbb9064901d7f3b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.263Z" + "updatedAt": "2025-11-29T02:44:56.748Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.743Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.736Z" } } }, "b7f8c2c6c3c0d8cae21834a515d86c9ba6864e0aa9c968e945adf28aff1bd428": { "bbcd7ca2f8d136d5cdb1c28f0c53253dd6f2040d23646bfbb062d85161da4e08": { "jp": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.086Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.043Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:45:03.041Z" } } }, "bc18991124499a7f66617eb5b243033498a2376e769bee9084fac4cef0b7c045": { "d62f4767bf6ec9661415c60e24e41a90ba047d383b9bfbb29a327253f604da58": { "jp": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.020Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.021Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.019Z" } } }, "c23421b71aced0ac75a1f6d4e5a8b8ae239e457c02367e880b6d1a4ff7277e3a": { "4719e0b0aa1afb513dbea43054775d5c3e22f6638707c72a91d88a4237b487bb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.087Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.086Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.262Z" + "updatedAt": "2025-11-29T02:44:56.744Z" } } }, "c621962c4e9a6c1f2dcb4ec8f98b33faa0d771e9aac97195014471b0f353099e": { "8e462b2a96c9f45baf5c523e8a97e3ffac3676c40724d42a9c5109d5413a54bd": { "jp": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.039Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.035Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.037Z" } } }, "c6addfcf4c2f183d0885f78b6bee455eb175ed28099b76df7d58a87ff79c231e": { "0bdad070e3c15637e1941843f067e2a8ab54f34932a6197c4b57662a1ab08586": { "jp": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.727Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.259Z" + "updatedAt": "2025-11-29T02:44:56.731Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:44:56.724Z" } } }, "d0a117042cd54d2d897e9ff128bb30722802674d738351bc727ad6a48d97c13a": { "ef198e4984503045b3061df3df5083cc081e20ea251352bf6175ea0983742b28": { "jp": { - "updatedAt": "2025-11-26T07:24:12.250Z" + "updatedAt": "2025-11-29T02:44:56.721Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.735Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.260Z" + "updatedAt": "2025-11-29T02:44:56.737Z" } } }, "216d22e459b5339d73b5e5f5efd10ba0d324035b56ffd8c09aca8ff6053e5be7": { "4347cf6fe8d3643c0bc778bc6d6e1a2d7728b22b55e566913fa8326c720d6e54": { "jp": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.075Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.052Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.050Z" } } }, "2f2c64962247267011454aad885684dd07f5230293d18c996004e9a086a48a9e": { "de25513083b27abcf3a1ed0793d26139ab348f9ddbadba05a87914373d86d034": { "jp": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.025Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.021Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.027Z" } } }, "3b502bb7173f6131431ad8322b576ef99ef5e91d3612beb68e0f4ce3b6053bf9": { "c7797285e4835ab50d34203593f5308bddaddec5d13f14f4f6d7be4be2239eb6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.069Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.265Z" + "updatedAt": "2025-11-29T02:45:03.069Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.076Z" } } }, "3c55f6319b00bb5e571612e6f740d049975d5c3de127e0de80d0f34889dd8b12": { "08f719dba95186bb05f8277899abf3443e7cc9fca51a32ba8727dadf82c77879": { "jp": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.060Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.062Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.061Z" } } }, "40ddf7122cbd5708445d09282a9aaaa01b51f15847138bd583939c6bee63c5a8": { "1efde3a11aa977a804768bd9d231b648a793e9638453375585e0f62486abe9f9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.019Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.020Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.029Z" } } }, "44e6428941aba89bd0fd45e401717504047bf2582288d528651664c68b5860ef": { "3dfaa8d64c4eec1438f9e2fdcdf95885e290daa7a1d6f9280e7587fbde343224": { "jp": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.079Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.044Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.045Z" } } }, "5034a9cab8d174bbba4fcce036fa29d5dc6bfa365274ed3cc44a0e5ff13c4738": { "c73720aff6e3013b19ca923ea6650c5399c7cce59157340fcac3ecb68f255f4b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.073Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.051Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.082Z" } } }, "541698242c872ea29127fc2dfe64cbea8b6e3ad3471aea2ac19011a37d71e754": { "08b7c30758e175cbf2a1d09a301193f88562d6e7ab18b078ab6c4b805b81620d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.085Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.084Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.046Z" } } }, "67199cb0b07db7b73e9d48c3856e7a80fa64a401ac9356f38dd56f0ef6af4f87": { "2a193532f966a6fea5015f9758bc034a7cbdfaf8b91c7431fdbc29b0d020b9e8": { "jp": { - "updatedAt": "2025-11-26T07:24:19.081Z" + "updatedAt": "2025-11-29T02:45:03.032Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.082Z" + "updatedAt": "2025-11-29T02:45:03.040Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.046Z" } } }, "74f8cb35854e4cf151ab34a6587a3b0c76868a99d06b7a1b7eb88bfdd101dcc2": { "9431057902d3a29dbfbbd44c8cc88c4dd2b703331d32f31fe7eab5675d5d047c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.070Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.068Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.067Z" } } }, "7e0dc4543c81b33bb19b9b0222c533c95884214b5877d7ed6c08d6101f73935f": { "4d2ea53c6c8b773cda0b23778f9e67b35379e9de8b35e7412e470060aa209fbe": { "jp": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.067Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.055Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.071Z" } } }, "885b5d789ebf32a2edb92bc498ab9f2e881afed86ef284b4892ee15109bb1321": { "b7053e1130cf6901ba2d93962cfe71528955b54a3427effb3f8dd0cb63a10854": { "jp": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.064Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.066Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.064Z" } } }, "8c4025d67d4f83f1787b2935a24ca235fcca456bc7505ac9ac478e5351ad8297": { "3cdb2c61028a51f468d7e958cbdb00bd91b81a31123aacd0a6e4c0f676f159fc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.066Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.052Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.072Z" } } }, "9f2ad018997a5b2a59f6bb176b61937bfa9cd7e81143b53306fe58e2c41400f8": { "79e16644830172d488a3acf805a5b9fe0f8b79fdbba1afe39d5495d561479ee9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.123Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.076Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.056Z" } } }, "b2c15bf0de452ad7ec7d6015900f40d41f66f8080e473ae5d92a9e398fdedca0": { "a7e5cb05a26913f4d5d6b8e23e33097010b909a91fbc9015096bd23deb3ef019": { "jp": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.081Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.077Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.075Z" } } }, "bda6feaa2f751d257d0e9bb7488f846a2998fca7dedddf3b4830683849ba2b58": { "2afea7889acf8ea5044a0d33842f100ab65c6cb7f1df295cd1f21f7e129776fe": { "jp": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.071Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.070Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.058Z" } } }, "d032d67a58a6623fab2b1b66938ad265d806211c7e670b710006fa88c0fa60d9": { "4c0a1b6590854c3a88fa162f08d4611049c85780870affbf3d49f61a3e412fae": { "jp": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.063Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.063Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.061Z" } } }, "db88afafa5d929b34cdf925862a67b324309a0e6c76686a5ccfde2ba82d5226c": { "e4a92a198d90a6cd53c04928fa4fb9c381359603f0c986e9d59a15fa39407592": { "jp": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.083Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.080Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.079Z" } } }, "e18abf4c56dbe146fa998f9070622acba484b5011490469cab0c1e16bc156647": { "58bb991322769280e6d10291b76e02c6a2e7231dc177a9f01f4c729dbe75cc7e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.125Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.121Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.074Z" } } }, "e5455b8e71ca0240dbae9ace48f312b2859517718c9b5597790152f5c5e4c55e": { "70f5e4c518ecfa04a597a86630bfa6b7c13859702dbefa84f43a08c628bb9c6e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.065Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.053Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.074Z" } } }, "f0b04860378a97e43a484e7cfff527be98a82a04b75ec9ff8b95b88bfe017c21": { "4d6e6128d8cb69272312bc10969428b2d7ec14e93843e97641bd6ee1b539f104": { "jp": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.085Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.258Z" + "updatedAt": "2025-11-29T02:45:03.018Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.083Z" + "updatedAt": "2025-11-29T02:45:03.043Z" } } }, "0f826dda16a017686da9cd258a7b36a8a0fa9bf0906faf288ac5dc07e8293c8b": { "7e91c488285e13a646cf4e0be8efc95cc3461d1b565495878e5ce6df5241454f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.104Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.102Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.107Z" } } }, "19d39d202840e88cd6155f2716118be469f38e1c294287f77b535f73052a335f": { "b04b8c8d1915a6a483b4461dc1983e8dc5b3c4131b3c513c2098e3306d1cf01d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.113Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.114Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.111Z" } } }, "20547e4692854c30843291c8c4b85cbaaa2473154a503ada089b47a286e119c6": { "add80eef63fea1cd539d2ca896319743cd0debee7952a9062ff15a5bac9cc978": { "jp": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.162Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.161Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.091Z" } } }, "3f80767faa69da876f277f16dd9152d0f1e8aba3db884130fa4c7ea029eb17e1": { "c8ca096e88fcce6dd3218a70cf039d6d7d8ebfe91be1b6c3b85f141fdc1feac1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.100Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.106Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.103Z" } } }, "4d30111f145343ad89c67951b74bc9c73865620ef3245a428535e3f9bbef259c": { "db4eae957950b7ac6455d1dc6679f4c64802f86d27f4184e0393be153c8bd7a0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.264Z" + "updatedAt": "2025-11-29T02:45:03.051Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.082Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.073Z" } } }, "4f944066028f36b0a6f28232fe75a6ebde995b969ebfd8a3c379cd645f0ff366": { "8ded3d0fa9f33ae122022672fd02b631471b5177e76c368607b554bbb3efce22": { "jp": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.080Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.059Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.123Z" } } }, "74dcbdc993f03875931c0ef548e27e0ecdd4c39c4c084edc6eaf3237a562817e": { "a9ecf8d346bd106208732038ad37c4f2b9861186a25aead51cc7057a47bf2cd5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.108Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.107Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.119Z" } } }, "7ab7a6fd8e115253049873f3b8e272c6c530d1d56d469dbf4d074ab0bc3ae8c5": { "c86ec8922020d2cf9ee2017859793ec04fb19507a65cfb98dd21a9cd2dc99ca5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.055Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.120Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.122Z" } } }, "7ecc3a4ce272f64e4527753ebb952d46d33a160afd6add3fc9a4a9b08d7fae1c": { "ffe09564c5075a69993bd16153054a5cfba44517a2680e60b50e4b87f0a5b83e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.117Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.119Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.094Z" } } }, "7ef33beb95b850b8400aad8ded966f28fd1eb3b61c5de7974983f2270d2b4f7c": { "501d9df3106342436670302f74dd2270b110ee24da435123cc0a1b51633a2284": { "zh": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.095Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.100Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.098Z" } } }, "81154bce9be97a0fc523001b189f4c093458747ff4e9b7f5cdecde64d9163d22": { "126e1bba0f10751cf028401cc1a0f3a944780e4a87fe9b63fb850c58b7d7510d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.118Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.115Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.089Z" } } }, "88d029b112f5fca5e4ba3d06b8c35a6d55e5b557663ed600c6f1b98f59f8ae20": { "1393aaf825d4dab45a6acc1ac4db09d138970e7008f8c78dc434242141a483ba": { "jp": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.104Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.102Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.105Z" } } }, "9831e429c5ccebdd733e3a8b6a49aa6100e584ba3fd39122b79e1a3df27a2feb": { "bde8f222c01ce8541efa7b5346611e02d7e5b13f92c827e80068ede72769e0c5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.054Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.057Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.124Z" } } }, "9b041aa508f2046ee0a4f84858531b8c2507bb6a6989db886c2dd4ea0c11a002": { "23dc86ecd0cc50924f5ea02d06b16b4e395c8e0f2fd73bd76d547ac864d42f36": { "jp": { - "updatedAt": "2025-11-26T07:24:12.270Z" + "updatedAt": "2025-11-29T02:45:03.103Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.118Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.101Z" } } }, "9fdb709a96f96fb011d844ca13cda88bb361212284a327821501551223a4aa9c": { "064e508fcc9e28910cd94c862392084ac9bfbb28d99941ea8a6c7bf60aa11b79": { "jp": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.092Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.111Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.109Z" } } }, "a08c904ab61d1c76fa622a160e0956711547c3a01e8aa50f73e5c58504c9110b": { "65bcd1c2b5d5887e042c81d6e5c21fc2a0db88c65574a53d172b1f40ae25058e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.067Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.086Z" + "updatedAt": "2025-11-29T02:45:03.065Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.068Z" } } }, "acceced538fb290e4499cdbefd4179c4f4d347c0ccfd60840e8eedd522602b6b": { "124022bd2cc51265ce8f1c49ed73363724b1580a4bbe5d35e3c5d6d9b2bb7c01": { "jp": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.162Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.163Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.160Z" } } }, "aeecbc80fabcf65b76af1cc65dd769022e4856381588c8501d1a59b206c10326": { "0ce14d2631d2a24f63a66e4f8b06f82fee405f818a0bcf369ea6485c8ba72681": { "jp": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.088Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.161Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.093Z" } } }, "b5543674ee59dc5d80ec783390644aa03c6a1b7c91bbff001eda92fd5198a064": { "dce1dfac5e498639b6f080315eaf0ea6f42c51bef46d3fb13e621234a36cb996": { "jp": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.116Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.112Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.112Z" } } }, "e6ce65cbfbbe441fc30cf64ab1c1d1dbe1699d91ca18dfbe615e4a83da7007bb": { "366a43842989bf6846e76c26bbf2e87e00bcb25564dfb7941a416bb6c279a332": { "jp": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.113Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.110Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.271Z" + "updatedAt": "2025-11-29T02:45:03.109Z" } } }, "e8bf7b4871a3b921003161fbe9fb3b3e0df205638abb6aa707688886621c9715": { "15aca606b9aecbf11a3de4acfdee9f33ff548522f3411df807128a214f52bae1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.058Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.077Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.089Z" + "updatedAt": "2025-11-29T02:45:03.078Z" } } }, "e90559dea9545d48d4ab67dc41647c74245d5af3f7450472a6a52017b58aaa6e": { "a15b882337784575046360aea947e55fbbbf97d76e32f32fb9e275a833afb47f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.108Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.088Z" + "updatedAt": "2025-11-29T02:45:03.115Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.087Z" + "updatedAt": "2025-11-29T02:45:03.106Z" } } }, "0b209462f1ec411886fda57e810cd3eea5efebe202ca2b4f5dc9f1fb3787ccfb": { "5ecfaa73c3cc92aee3ee2825b0bb89bc857721cc0ed52b56af3b10a539b65498": { "jp": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.151Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.152Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.099Z" } } }, "1d14e004d487902f18fc6c1de04f1ef911152e4d8c2d76455e4956d9cccd132b": { "435800632f77c2f3a43f62396007c869bf0e3310b946c504cec9c7661f101c78": { "jp": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.121Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.090Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.091Z" } } }, "30cf7268ead1225c4a5df7a64fa02d87949b29677780ab8bce4abfe82c42f29a": { "279eb946f79782ab2e8f6e23256ddd3224d1fa15de3a0be112ac3c94bc4312e6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.129Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.131Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.137Z" } } }, "3429435d33feb194cd2815db45da2e05b63eefb419c7039d15215e40384643ba": { "918e8abb0c6066b88bb4b0bdf464c3907f836aae0d601ee87bc91ad879720c8f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.145Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.142Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.138Z" } } }, "3fdff0c8c92ebbc95447e7244075da88510e0c3d4966e3b72af95a6e4c3d8e8f": { "1e45c8cfbc59d4c2fd364a34eb2e7afffd36ea4f0b127f873065e2b176a0133c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.259Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.159Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.259Z" } } }, "4ff60f576a90647ac6859ba05f56e594f54029ca4beea54b1e07f27ee5acfc94": { "b991af90c327a458792ab1640e608a8704cbde6a6f1373636c3d4a5c3445b766": { "jp": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.099Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.097Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.269Z" + "updatedAt": "2025-11-29T02:45:03.098Z" } } }, "5063b2b4bc9b2899fab5998a2b281df0229add76ce268451423a1dfd2ffa5f2c": { "d2af9085fbf80701266de277a6a67f2400d823b5ac0d2ee3f5ffb2eb0b4f0294": { "jp": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.153Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.151Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.153Z" } } }, "53e5bb2209c16605d7273edd1079563619f7fd4e6e5bdfdb95988af1a4694755": { "19b750db7b91f72b4f9666d5cd502557bfaf69581d6fb96105e239e437635657": { "jp": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.156Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.257Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.157Z" } } }, @@ -8849,130 +8933,130 @@ "611b2b0d02709490f3fe4e3331bd31828428e661cdc38d633fd487daabd3cc1c": { "cb8d66eeaa9869bc4e4a0832238beb5e4b8cc2ffa0e3483678d79606c472c326": { "jp": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.140Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.142Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.141Z" } } }, "633a4ffa471ca2244e6ef5a3022d6a46f51861f23239b9b4594d8cac210cc0b0": { "011445c96b51faadcc04ca2af74b4a9de574446918a704bcb7648036f25d38a7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.159Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.158Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.258Z" } } }, "675843b51c582122de910ed4f222f211176c97af172b7f849c0b8ecd0dd2b190": { "a27dbf65b4c9c2e9891bbf450b7163614f6940254a6ad1c1db78fd18c3795fe7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.147Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.132Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.141Z" } } }, "798d0e3eca2e56d6aa7658d85b9a41657e3aacf854913976ea97d89d8865966a": { "767118d90c94b77855b18cc08229cfbb4dd47ceb560ee656c0882c9192c24418": { "jp": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.150Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.154Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.154Z" } } }, "858c3c5ed9eef8bbc65ab6bd6d04421cdd9d8ffdc620d1e8becc8e78f8cc2970": { "272ee010337aa027090ddbeff208e3a6d597280101959048ceb7f7a899d361b9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.129Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.135Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.126Z" } } }, "991e27fab22b52bb4b08b4ae04fdec89d5e6553dc7110f7d24b73408fff315c1": { "a03618c42cb58f95e7e03a4057880d077e66e088f5502749a604eaca3e70f464": { "jp": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.155Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.152Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.155Z" } } }, "a6b9d4c5cae0464959192ad659ed2100cebdeb8bc49e4c041d80a9c6a804808b": { "e888d9f5660cbc8a94390f0efc75e38b61355c7aed5b560ba7c55138aa191993": { "jp": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.127Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.125Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.127Z" } } }, "b7a5608a851a55f00f22ae8d517987b946c9c3eb543370562dc786dab3594714": { "88a876337f46351c9ccac93457f33dc4fb23d9aab3760cae91e020811ac6f19e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.157Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.156Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.272Z" + "updatedAt": "2025-11-29T02:45:03.261Z" } } }, "d9b29cc47744c8dbd75014c191b2d1b6a6cbd834e8f58800d7ab54ee3b380193": { "790a9a7598eac524933e6837e62602bb54c548d8cae162ec8f67203a8285580a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.144Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.136Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.135Z" } } }, "ec3ea94f6a821f3d66e7dc9993bc4fc2b65580f3ce729e89dc7d1d6e9711078e": { "078157aa36205afa5c6e11fa8f7457d8696fb79062fc79c709121c33ed2a7d52": { "jp": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.257Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.260Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.279Z" + "updatedAt": "2025-11-29T02:45:03.159Z" } } }, @@ -8990,1339 +9074,1339 @@ }, "97e44c5927e52bd9073aef11610c839f64450b48880fbf9a450f43177e163506": { "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.136Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.134Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.131Z" } } }, "f62c32cc2359a74d04c720f97feb4119997308d7027c0a473d0878ff90a711e1": { "8bc8554c77b98b4cb1507facebf9be6cda0cbb901ca29360d32136bfd62407a6": { "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.132Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.128Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.128Z" } } }, "0788f71f3701d95084837950d519aaf717087552402cd82dfcf4236628f15af7": { "1840d9cc80dd9c8c8cc0209074557de0b8c1bf9c2ca33bff6ab6effea03e9a16": { "jp": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.168Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.167Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.166Z" } } }, "0a5e03fdd60d6da9d1347b2bf3d6f54bfc297dac14108de9809e6b2bf3665be6": { "21ae110d91d8fdefc2b7d4393674825fad68899404deb0a78de5c902d700b719": { "jp": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.180Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.187Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.178Z" } } }, "178c705b2c62c01727a80327a809f6535d583c870ad39995375a10a363a1d727": { "1589f225c754084b804cb1b7c426921b7979e160c01fe65dd00c2a0a3e3ae3f9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.169Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.164Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.175Z" } } }, "28de08e6f00a1c0b51895715997e43dbe463c1c4cff9b002dd9014edc5579dcb": { "46a435b4ba73651faf0dfb756e1f9ac1d59de7e580a40c74411bfb41b7c958d9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.177Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.167Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.165Z" } } }, "2b86670a1b384ba80cf06cda553945f2f2ae1c3d4a369cf46e8551544379716c": { "4169ee416284b3a8de7ef404d0519465870ec5b969e0dbb0dbb883cd456ef93c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.182Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.188Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.171Z" } } }, "349400436c332178133e694bcd47dc9ccdf4d729cfc274f1f99bf82b54fde8d1": { "312b11780c641636863655dd7b1fd8b57e6cba8bebf6946857812ca2c3afe479": { "jp": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.173Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.172Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.168Z" } } }, "3798000812ded299b4f0b685fe4df133730a45cf67c87419cd05a769b727f03e": { "1810bd73113c23ab353e5810beb46fbccbee2f443e979883aaf33e93c1afa116": { "jp": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.130Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.133Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.138Z" } } }, "4047bbbf1b204976cd120655b4d3e85ae62f7963b1cd9eb60447ba1c7a58d925": { "0250b8f7933c3640e2d5931945a7588921cbebcd1ad9f8227d1301fc00ccdc41": { "jp": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.139Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.144Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.143Z" } } }, "560a771a88f358751a343c140ad56fb357e898496988479f8f38d70c5e6abd73": { "9d206180b799fb716602163eae19d67a30abf8a23c3361889197d04fc78bad52": { "jp": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.174Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.180Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.170Z" } } }, "5c678b3a4246706bfc999afc97cec53d8c982fb87363f87665082241361a5d73": { "3bc705589b64c0ed96008afa97074822fc2196c075b74f42358b16d267c7160a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.171Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.175Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.183Z" } } }, "6021378296fe97cd32f847567e8226b5b01ff3e70c1eaaf35a828c9d29135ea8": { "a116f2580c016c233d50250b989b32bbe09ddafa83b8dc9dddec1dfc676909e5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.177Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.172Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.182Z" } } }, "773e022e6828901db117df504dcb5f22c010a9943c580fc510044d9585197e57": { "b629f3340f4e22116ec115e53eedd044eb499d902c10c1c5d836dbbd184e23b7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.178Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.178Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.176Z" } } }, "8e7b5fa5ffe7d88c028b17edd689a831fdd41f6a2fc82ff78123493c674623d0": { "ede9c0bcd24a013660cd4a3b395c94ffbe54ec3e8cb4f5607bf18d77ba8eca33": { "jp": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.185Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.179Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.184Z" } } }, "9011c5f16cd48892d19d1f38ab22564d7b0466e5517619ebecc2bc7e71bcaed8": { "28412c34a30c1203bcca0cbcea84a0b53297cedb3c9afd82adb465da3b94442f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.186Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.278Z" + "updatedAt": "2025-11-29T02:45:03.149Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.146Z" } } }, "93c3b152cbce0393c9c6f97cf591c3711cbf3f81789b2925cea760c4c5cff676": { "a411bf07df5d4b27d21c51466694fc824b2f2422d09fd22f63570974bf2e2f9b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.276Z" + "updatedAt": "2025-11-29T02:45:03.139Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.275Z" + "updatedAt": "2025-11-29T02:45:03.134Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.277Z" + "updatedAt": "2025-11-29T02:45:03.145Z" } } }, "bbd2cfba8a3c3afd64babc88292f440871b9379ed0c722270cbaee5b28989f92": { "c2ff068859047c7491ef0445b9b895d4133fbbdf15a64d444483ba2ed4225407": { "jp": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.184Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.181Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.187Z" } } }, "be8dc1cd18e614d45a58deaf1024b50c63f3b407079c8195f643d25501e18a86": { "835ca542a441c446925af17c0332381d8c4950cb2f458181ca206b84a34091ae": { "jp": { - "updatedAt": "2025-11-26T07:24:12.282Z" + "updatedAt": "2025-11-29T02:45:03.174Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.183Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.274Z" + "updatedAt": "2025-11-29T02:45:03.165Z" } } }, "bfd7f9d45ae0b09ed83b9f435d6a0720b54f9af64528eea7b98d70d76a9a29ba": { "db0cfeae11a772453aa791ef8a6aad69ed5ff266472101082ba0c8d64ee078c8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.283Z" + "updatedAt": "2025-11-29T02:45:03.181Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.281Z" + "updatedAt": "2025-11-29T02:45:03.169Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.186Z" } } }, "c3760dc390d98e6e5ed5a4a74c5c609915d05a7a63963656f7715607d65ae092": { "2904d7fb4addff8bf173ca68df5b540945453ef8fa9eec76a35d8f9b92ab8b87": { "jp": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.254Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.253Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.254Z" } } }, "e7312c644964f4d389a9171edabe14341e5e6fdd852101cf9f16a264088857b7": { "2904b07971746b903763bbcc8b60c7bc05a984fd6692a24f60eeae21856cf64a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.252Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.190Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.190Z" } } }, "f5e8eec3fa4fdf1b4c16b4b71f35b278d41db6e5586c66a42fe590521942f347": { "f9704f3dd2bb395a82abdb0dd1b7b09ea97a4499075e9bc8ecfcb0ead44a1d69": { "jp": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.293Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.289Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.290Z" } } }, "0c9700318afe07f773f3675286dbd1308302fb5c993fc403ead5ee2c2c311f85": { "26bbf167b8a8bdd6e415d3cf429c935f63ed38563bdb8697297248361bdeffad": { "jp": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.287Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.277Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.282Z" } } }, "1205bf7e48133304fe346efa0309af05787e80fd6f83623b178426d0d89e43ab": { "7a4af08a1b17f2a86db198129d22bf1a71494ef3425bd28e8251e46075a27288": { "jp": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.189Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.190Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.284Z" + "updatedAt": "2025-11-29T02:45:03.189Z" } } }, "3c59f44959e6e9ca6acdb43ffec9355d9485257cc402a212ce1759a0064bb981": { "1ec8c112466533ed9f452783ba3194be3a2def5e0e028ac74fb3913e56cc2f4d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.289Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.326Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.288Z" } } }, "401161c2a7c478b5179bcce758b06f32dba0fdbf9bc64188f2d7f79dac72dfa0": { "f590407909c6eb71eb806ee84a3e8ff079ef373b53b7e0b97152b2c9ea18f318": { "jp": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.327Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.292Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.285Z" } } }, "48ca9336c96e6bf5d7264d6ae62d5ee29644e6c214dc339d83a610716c484ff0": { "6e9ef6dfd8e741fb723339409fd3ec6e0e74d8c83d08b37cb60190c4e83a6762": { "jp": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.274Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.273Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.275Z" } } }, "51b2a652cdf591979b38ae330c8306c66fd1186f911c915e5aa9e108a6876603": { "a570b7d389235335a4604eebb1f1ee2a84cfb547848012a3f76c135f5a18e3f9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.328Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.291Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.286Z" } } }, "5272155dbd5220decd129a5e4b559edddbdf6ce43e7a6b8b33c93f39ff269597": { "976786fd43e7ab3db7efe0f5493c2e4b732add2abc4ca3639e54d6dba7ea3e9c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.271Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.271Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.270Z" } } }, "548de96f50ceaca8e5445f140674be40c17b5c6efd12a9ec257c21dcf7ff409c": { "5b113a68ce6ddc11290d264ed87c01b3577af499c83519374e7d367764d8035d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.261Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.267Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.264Z" } } }, "65a720c2ea0a8158d89758ca90889e7cfba036082ddc99c1900ceb31830b3cd3": { "38abb8321a70b767befbe0fce1b421459ce68d0c40703530c81bd00bb0fe53a2": { "zh": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.268Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.090Z" + "updatedAt": "2025-11-29T02:45:03.265Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.262Z" } } }, "693961a5d1fa6bea79f69b1d2a739db59c41797c8d322ac4dea99c908e8abe46": { "b20f75da8b934d12c4230b89c1dd8cd4bb4b57708832dd5f413fd6ddf12d4434": { "jp": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.277Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.278Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.276Z" } } }, "822e90a8485f6ba254a1b6b4f89bbeea67771bd3cb9f9d6241558e4b9f59e8ca": { "3442662c930110d3e163429ea57e15d27f6132307f6bdd86dd62fc64d01d1c48": { "jp": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.274Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.271Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.272Z" } } }, "8fafd060efa9d7570d6665629f29f511b108ca76567a0f8ab9320536cf4824a3": { "95dc2ad2c072c0167726cf92eb31cd7af87b0eb4785b0fb839363d03a88ae8a5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.295Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.290Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.258Z" } } }, "910c09772c30498ccd96c4a7059798706b5861119f5ae8e46d899e9a4da807d5": { "419e68f0fe31b19a72d7bfd6b1b28c27298c6d38904baf049d3466be88aac0ea": { "jp": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.280Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.281Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.280Z" } } }, "9d303cf2c426cf20096b5dc4321074d62704d00520545ee1bc5184cdba1e2bc6": { "ee164e4761915f7bccd76758188f979de64e4ab9239556b390ffac31d42aaf2a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.293Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.330Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.292Z" } } }, "a0c24fe635a0e43acb3d80e4a7fc854ecdfc143306eb5b0b77baccd6c4ef9468": { "e71bd647b55a5bb2a691655582237e95f9ff246c5f45f2f2f663e74b62968fa9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.286Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.328Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.287Z" } } }, "b60ebbddf877960af38c601bbdbf000beb3124a60fee1f8c23fed49149d1c527": { "a5cf8d2eccddd9b6214fa12aac2b98dd4e514d569be5e26938ee9a3b11a0b411": { "jp": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.256Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.255Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.255Z" } } }, "c11fd5cd4c0e0c76b50b836fc0585b7d897d5c6e619c8530f61e70fb13e7d1cc": { "1fc6d064882a931f2ccd7ae4239ad068568c65a8bef153bd6264d39d45bdf340": { "jp": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.270Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.272Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.275Z" } } }, "d2f8552b7911ad6c378a02252cb789aff8287601b71f6571a0f6e1b7a8e78d04": { "94efb582e11f5b80d588c2052e90bfd70b816f556e6039e851019c86956b10da": { "jp": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.294Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.295Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.260Z" } } }, "eb48ea9cc55a5f79da9d6053e1ddc3e175fac421ecfbf7cdd1fba7409a5937c6": { "4bc78345ed8b814098932537f3fc29577489a1bf65318ccf523d0e7979227a78": { "jp": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.282Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.283Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.285Z" + "updatedAt": "2025-11-29T02:45:03.284Z" } } }, "ec7b68e24b7e06a320d9daf21b92c0a61d2d196ada1c8350b443b492c228b245": { "f255bcf4104dcca52f9807566387c0fcfe6d06f436994fee4179bc40d148cf94": { "jp": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.279Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.283Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.281Z" } } }, "eefff94e72ae2ff41e6e3bdfd308882739e2e719c94cb06245a0ddf4866a91d0": { "1a4e25f6cb4dbccbb5205a184e3f9417ca1d8398e86e5433534abb2f3af17825": { "jp": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.278Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.276Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.093Z" + "updatedAt": "2025-11-29T02:45:03.279Z" } } }, "fe9dca2c6fd3af40bc21440a38de1acbeedfae1588e58844d14185b01797ace9": { "ebf565ec610bd53ba3ec0980a72ee44fcc9e259c89089c7d524937e46cc0037f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.330Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.095Z" + "updatedAt": "2025-11-29T02:45:03.329Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.094Z" + "updatedAt": "2025-11-29T02:45:03.285Z" } } }, "0488cc4c783adb013176b8dd5553d28bd7e7ce03785fd0038e3b2b17e6bdf549": { "718aa60f3c8b05491fd8687c867ff950c98134aa648057ef2a403f78f1807100": { "jp": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.319Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.296Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.302Z" } } }, "1310e9bab89510f4aedd870fa23492b42460b27cc53beb62388b8527a75b4abe": { "5d36c63b3cd649e6c42f53a7e1722d9058261e3c5703e736f85a4081ed299d22": { "jp": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.306Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.310Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.321Z" } } }, "313f2f3a2287ee9166540ad792489898b8322355b28ee91e96ba66cf781aac35": { "813cd15c21bab5b5ae060ddf42a770163642046d3681ff5dd1dd8a48b6578a17": { "jp": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.304Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.362Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.364Z" } } }, "38d86ec85c1c8aaad845db190de05e50994d1a3c494195da910589c64b052751": { "3cff21a72fb101c7dc507cfac07bb03d9d16b6445213a5a7553e646f024ba71f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.092Z" + "updatedAt": "2025-11-29T02:45:03.322Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.302Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.264Z" } } }, "47aae18e89fdc913969ad0dd021c6affb6a825d67862170fab9bf412e150d04a": { "7845706578879f0d6235708b243856e2005db4e602dca78be25078cff83676ac": { "jp": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.365Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.363Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.305Z" } } }, "4a1e810e51a719b0c246d3a43e6419bd4b987b2e7623567a865586ec6ed3fddb": { "ed63b452ccdcc51644ab26c7e164fd9c06b4fb9dd0f29123b7c142d640dfd731": { "jp": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.310Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.303Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.305Z" } } }, "50a5598ee25c450c5fb03f18bc79c9f33c4b2d45dd82d93378770a029449765f": { "d681b2d70ad9048fc005dfbd39784bf38bc368cbb6e601d7be30a81c02aa66d1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.263Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.266Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.268Z" } } }, "58592583285bade083e9bb2abfe89113954c980d6a63fd9134c60920badad2d7": { "8b688b902eb485da1cd904c9a534e7c30138ddc8fe157648544914cd332f7701": { "jp": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.307Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.305Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.322Z" } } }, "5adf48b603a73cafc111898833bb810f6f9d985906f5a28a6b5510c4ad5ed9df": { "ded6c22af292aa253dbdb1b8bcdd3dfedbd38430db398f57c83b96b8b42647f8": { "jp": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.300Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.297Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.286Z" + "updatedAt": "2025-11-29T02:45:03.296Z" } } }, "7d16493aea7d06c09ef802117a0be5f6d628751d0a5c7a7b03ce8eb9dc409bf2": { "5c7a7e89cebe18dd07de910c107fbcee8795947ad29a2d17a6d6024a235a658a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.267Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.269Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.263Z" } } }, "948b9dc936c07fa8b4472138f98721317baa561958a48a6445780ecfc6a1c485": { "2f113bab1b3e6819aa420803e0868837c5a60eed370a5c0708d29084e14f6cdc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.299Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.299Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.304Z" } } }, "94df9a623cfec05c2c5b489fbed533e510d65ccbf937bed27f852c60f3a24b6b": { "4d71c012d9187781ca8fcfad6d17272ce0479d7a403fdf6f4e13744b2054c414": { "jp": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.325Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.363Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.320Z" } } }, "abde8721a04c3899ef6606633f77be77e9d032a8fa7f37d834ba01a23fe119b9": { "580c03f819a51524be5321c5af5b976bf750d39a4e3a64a3dd28f32805924089": { "jp": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.301Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.298Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.302Z" } } }, "ae86875e3a4deeec5b623e90f58d3191bc8e79167da17320095d45b7aefc2243": { "8e8b9e7eee69658acfb5be5d7837a6c6af0457a30ff7676b0d57099a5399ff0e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.309Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.096Z" + "updatedAt": "2025-11-29T02:45:03.300Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.298Z" } } }, "b5d0eacaaf66596432fd2e0164bb5f867e3cac16623e968148a4d757d106c3f9": { "dd01468833f830dab589b0b46480f9b998ba99103d12ff19ec3c342a9f0a9138": { "jp": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.362Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.309Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.325Z" } } }, "bc185d41a81a462e3988685f733423500e79d9186808359cf876254dfc1df6b9": { "873f51e584f0fef0ed5ce12f52eacab370768a902dd8b25575c46c3ea3925c19": { "jp": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.326Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.307Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.324Z" } } }, "bdf609022e3136bdae1def5400ec2932bb8f17ea8d7d49a273b0293defd3affb": { "f21711f3b2d080fbcd8a0d170f14659f54ad538d7c534cc91bee92cd96943824": { "jp": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.308Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.323Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.308Z" } } }, "c5084a09df628aa86f5f8693b10a55f9e8af3ba5b6e50ed69ff474a580871673": { "639b5a2d990e67de3c2c8aab1480380f68b5cffc8cb43a13b0da74c601a38749": { "ru": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.311Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.317Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.314Z" } } }, "c6970f5399e645bf58a1525ef6209242f22690e314c9ec2676aa0f609e60850f": { "857e9c3ca17f16a6e7331b2d62e9f15ea308a426462699ae488f7fd808b8bedf": { "jp": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.323Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.097Z" + "updatedAt": "2025-11-29T02:45:03.306Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.320Z" } } }, "f4e806b4f6f3414588a99250928aefb88bd97521c5fb9f755d133116d676e98f": { "b47d5c0bb52bcb18d0784610fd2a2faa24c97495fc26370a6b4024b1a998b212": { "jp": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.267Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.280Z" + "updatedAt": "2025-11-29T02:45:03.262Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.091Z" + "updatedAt": "2025-11-29T02:45:03.265Z" } } }, "fe52a1835874eff99646b2ecbf9812aaa4ad459489ce76c856750b021e1969fb": { "44b9e40b3ed21a0eb1effa1387bbd83dc88cf7259bae3bbf2af2a134b07516e5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.364Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.287Z" + "updatedAt": "2025-11-29T02:45:03.321Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.288Z" + "updatedAt": "2025-11-29T02:45:03.324Z" } } }, "077683f76fe06aef19e3361bceab4bc549399e0723b4d9d14415d78c7b29cdfb": { "fe9570de03d2029f3efd3701f8a9844fa8bb91810ea7c58923ee8d0766854adc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.352Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.352Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.358Z" } } }, "1d4d6e77bcbd23d001d1913843fc6c9748753173b9770ce333d87441932130ec": { "30da2cbfe92790be7c2f95f485c2ea63c4ff423ade0453d52e65f78a6fe652c0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.350Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.359Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.342Z" } } }, "37b9937d3f28ea06521af2789937cb6974b4bb1da71a4e0e38cd433452943f4b": { "ff41c613f12a073c7cfef1f537c5bef8fc0820fa48eaa7f6ad0cb887283d047d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.346Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.335Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.347Z" } } }, "4503f45c726f639e1a6502e2fa738700aac770245105ecbbc3d6006506fa8e7e": { "b3e6deff1b1839f01fe2fdfb7c34b1a485c8bd5be58b682ad09b971716acc42c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.349Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.334Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.349Z" } } }, "45c696d22175381569602ddc4401df16a7d32249c5f9994c4b98bf2548350315": { "65644f5b55fd61d8fdbe8a764e605ff7e00f6ec53fcdfa43f484a5638a58d2aa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.360Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.368Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.374Z" } } }, "4d6593bbb881e0a74e7a089539eeba4aca7019f581c7caeadeee04c001000773": { "d16ded5082885b0eeb5b28bcee5bf878c87a2cc092934fcfc328a1e535effa1f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.358Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.344Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.339Z" } } }, "5c12094be2a10a85a875ce129adf37c46bdae04160dbb85b3eb63b9c69e7f6ac": { "bf9cdc73e3b5ca0e62d14af59e1854dd6d45176f362f34533c815c278385d1ec": { "jp": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.340Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.334Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.357Z" } } }, "62508936e6b3e7a6b965ed3df755188b154e45270320ca734cb0df2e29a942a9": { "9a4adbb5e86533b1fab803147ed4539c344e121c9526ce249b8e3c49744c7702": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.350Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.336Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.345Z" } } }, "7a1451fe8363988c04d1df2125cc6a560940a7c034905f5e75da236ab427774e": { "7f9fa8dfaab48853ecedafd465b380359704ea83aed218c677074831e1cc0932": { "jp": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.339Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.343Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.346Z" } } }, "83523c78b37179282ea3d0f8a98cd8c0e917e50caaf74f38e237b1b1f1fd7dc1": { "7f172e3eb258a3b4cd3c132303859997ffb354f24a60481f04ae0f80fefe2147": { "jp": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.344Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.356Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.371Z" } } }, "89a6cf75614ffde882ea0e38b857ec20bc3415e924373b586ee53a84d81b8dac": { "212ef1dfe191daf73ed51386731e37ce6d4ca49f4472b7e471586979e69a9a9d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.348Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.355Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.337Z" } } }, "8e4e3758c244f276a3f91f720f08400f7d3280b2729ed2535fe4b0a244bc1eb7": { "a3356389fc2d7537a8464f2e1646f8f51af66a2d715df1807a2fd4184083a70f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.332Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.336Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.361Z" } } }, "92c4e15d1b1edd5a34f950168fa129302400e9f6ef4fa378e3c7af3ed6ec8227": { "3c3fcd6c5352af3e3f90c0a4d954793388177b9bbb34b975eff1c8f384d445ac": { "jp": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.353Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.333Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.368Z" } } }, "a14794c89d955458a9f5af44d7aaca8d68a05b6880e98e008a7c081604143ab7": { "671b0a57421a638325cbf9c110626a9d5b734267bb8f974814c03393141cf7b8": { "jp": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.331Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.336Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.340Z" } } }, "ae39080b133df67d8884d7a8d76cf775ef202d9bf2efb43947344e07462aec23": { "4c42c112034c378e6000b6c987744ecc184d4c90582c11dc33f577b3f2ee44cd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.353Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.343Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.356Z" } } }, "b3e8c57a2ac90416a86e93c4fc87cc9fc69b9ee772adbd854463142bcf0ad103": { "78a6c5fa33437b43f2619fdc05ba1a3ff266f89bafbeb1b78bc71a0ed76a0496": { "jp": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.371Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.370Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.351Z" } } }, "bfa5f357797593cffea8aa625d31e79d5f58effffe1213f1bbb7b709e0c951e9": { "9dbe571f5b98f8fb6c1fe7c120e80cf8fe72a659f77f22e8b74282600d4e9325": { "jp": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.342Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.338Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.341Z" } } }, "c593a21ae24f2adf1116e2099fe2cac24733672a1fdacfbb7d9be523e674a070": { "3888654c7ba7da0474c2c33ac3100faa58509581ecb5ff97147be80f6c3ddc7f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.098Z" + "updatedAt": "2025-11-29T02:45:03.331Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.359Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.351Z" } } }, "d2d56d1eccd2d86a90004069292a4cfc31251986d8bb238fa00ba3a4aab4a56d": { "dc92ad8afa44196810e06c60223ea9ca5b982c40325ac54b37fd95a9f450fdda": { "jp": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.349Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.290Z" + "updatedAt": "2025-11-29T02:45:03.338Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.341Z" } } }, "d36f1827c010ce2e2dcab5998b4c489e963acbe4c2d8322885eae6daf7d3e446": { "2d4e379a75efd761f80eb533b3cf33859ee34ab855d930fab99c5091b13fa5a3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.361Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.332Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.291Z" + "updatedAt": "2025-11-29T02:45:03.345Z" } } }, "f27af8909a343bda58696e815f4b50b00101d0dcd66b99619aa579b381a444cf": { "929021d21964c8a27df287754f3bf673b1e9e43e5b78df9447405b8197530ab2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.367Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.354Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.293Z" + "updatedAt": "2025-11-29T02:45:03.357Z" } } }, "1d24065c2e7fca3ac3f26d0a2b7ccd04f7ff1ae4faa321c7335a8e84eb0ac0de": { "e323f890710302432f3ba708412993f1d391acfb58bf585c82e91d8c3c5b823a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.404Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.402Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.401Z" } } }, "34f6accb938658c99a83aa179d1dfe75fe3f844b0e815b1a8d42a512eb830f06": { "c43e5de4e7fa4afd53423adaa427167edd9077fd3af0bcd8e16a72269e83116f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.405Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.405Z" } } }, "45e2237668cc8f027e43e52ef4443b8a53d2c07dde3b858205c9c43057f4cb8b": { "66380f0ef83c18a16b8296671ad4697deea2b60436ad4259cd3c3df09895bbfc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.366Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.400Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.399Z" } } }, "4e39f1cc2912db452edc06d93f7f0bfcc091c2888f064a3281bd99e46645f722": { "48a7640cd750631e03fa4c3747cd09af737c4ed39ad0a40e22ebcfdbc24b9872": { "jp": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.370Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.292Z" + "updatedAt": "2025-11-29T02:45:03.367Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.294Z" + "updatedAt": "2025-11-29T02:45:03.369Z" } } }, "990553ca9f9ae4591aaae11318ecec98a52d743479ad68505f33d7437ebdcfe5": { "6706062fa424eac816c221cf4a0ecb23afeca8ecbe3f4830da0cee49f3af5b55": { "jp": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.397Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.400Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.289Z" + "updatedAt": "2025-11-29T02:45:03.365Z" } } }, "9b3d838535466c0adcbcf2c1821542686b5932d55c219ecd4c54a8d3d723b617": { "b968225991ebd30f1600f3ad485919d0badeecf3a3e60c5cb52b71a85c5611c6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.402Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.399Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.398Z" } } }, "f8fa9a1c93f8857620e1d2d6052e2a540a9827ab07e015947b84e6fc066cf05a": { "27bc228b35212b29d55733663b0d676059fdafc2d49a527814889b3aa40f6e10": { "jp": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.401Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.295Z" + "updatedAt": "2025-11-29T02:45:03.397Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.296Z" + "updatedAt": "2025-11-29T02:45:03.404Z" } } }, "11aa99a1bdc8390230a974032f545ad7fc914b9d7d7512e6f3d523c3c3315925": { "25ab99f304def64235d114ed61495f4a871f63a473b431f04505d22d84acd92b": { "ru": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.316Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.312Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.318Z" } } }, "15a59bf1722e4b12c28df70766e0baab4b9d5a6f0a0473fcdaa0c562dee3986b": { "38c435040eaac3147a4b165e8f2e2eea100525b71769ee62c7de7604c2c7decd": { "ru": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.314Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.312Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.313Z" } } }, "a840f2497ddf7c24e3414728a66fe927662c74a0473293e11a93429df3ef7e1d": { "14417b042f80b8359063dc1571b796f4f9775e28a90c36436b10c493b04268af": { "ru": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.317Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.318Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.100Z" + "updatedAt": "2025-11-29T02:45:03.347Z" } } }, "e843b874a573838613448a25478fe1be3cfe8e1a5c23c7d816af626567769147": { "8cb205aa323de3c2fa63f58b08365d61b559f9ba1b8554ec982b293d9a83f80b": { "ru": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.313Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.315Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.099Z" + "updatedAt": "2025-11-29T02:45:03.315Z" } } }, "3177435d774099d4ba686628bc971ccc42a54d0a0a211c8a4424bbc544e08540": { "f15d74887e89dbc77f9957e1568c4842460915108734894efa6e2f081275d68b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.077Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.074Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.077Z" } } }, "3caedd95aefa51553be1069772560367e021728814e3e4cb4e732e19460e0502": { "c808220f60eb5bb176af1e26539836830b9934b93a9bc1e1e62fd9b90ce36bc8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.469Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.469Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.468Z" } } }, "853246cca55f655f764269048050edb509e178c1ed6b34530b7a3aae600ec2b8": { "0a1abce96f2027f1611f7096e0422a02de923c3698460cb2c242ae3092e25c81": { "zh": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.436Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.460Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.461Z" } } }, "a030bf426b6662b4674be21ff621cb7fabbfd26f971ddb89ac770557065aa0cc": { "f732d015e8ca7a50761bad6c4404360438b7df18567a96df59faad98662b6017": { "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.461Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.436Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.714Z" + "updatedAt": "2025-11-29T02:44:55.435Z" } } }, "06c88066bda47d4a934bcdcd6f121c4c1e22b06d73242fdfb1ab310a2564cf7a": { "f10ca14dce06ec46cdd4e21bcf3783e50fb8f8e2c7873cc6b828db0e89c91024": { "jp": { - "updatedAt": "2025-11-26T07:24:12.079Z" + "updatedAt": "2025-11-29T02:44:55.464Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.466Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.080Z" + "updatedAt": "2025-11-29T02:44:55.466Z" } } }, @@ -10337,83 +10421,94 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.080Z" } + }, + "fd960e0ad4a4e719414c642095987287a615859dcdfe78dc5e4ade0ad15a3dc3": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.441Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.441Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.442Z" + } } }, "48bd4337b75cd02afdef9e5066ef37aa097bb2376a0997cda1862ec2672e0bb6": { "c01428e3868677f56a7361089108618d1aa1b3f64f9d078f8a9dd079aeceadf1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.500Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.499Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.501Z" } } }, "4a871b3501c8910734e45bfd046fb170eead507a557e7fc029a9720169d74f60": { "a1bfd48d5bf528dd7d49ff5929721a27fac3e265e20a187bfe5603465299248f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.462Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.079Z" + "updatedAt": "2025-11-29T02:44:55.463Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.460Z" } } }, "50f0ba5685aaf3e9d2d05dffeeaa45f47b7ed622dc20465bd6aa71e7192a1a6f": { "430792450e0e247081db5645bfe27bcdf7c5efb4c46fb798c742aecf01bea55d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.486Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.494Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.490Z" } } }, "5929e4805377229948887e5ba720274840b70d5c8448deadfee3a33803c24777": { "4923fea66c23915a7ee88662e5a25bc88b6e63399b5f8007edd0a604f6ff29e9": { "jp": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.456Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.455Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.458Z" } } }, "7f4f10424fd5d15211a9b2e7f5376cd61876478ca1e288c42f77a9d27815ed3b": { "49a85cf8c399228a66495a6ff70df4eb90e968fc2a6386b6d0c3a47d1c6934c0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.502Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.501Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.468Z" } } }, "8fac3eeff35b863ef1c1a857ec5cc7ec6c5e04a3ba1b53c0613d799e0ab40033": { "cff3cef9c9971227c006470a36ab779082e9292add9a0d6480da3c2873a882cb": { "jp": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.458Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.457Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.079Z" + "updatedAt": "2025-11-29T02:44:55.464Z" } } }, @@ -10428,31 +10523,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.079Z" } + }, + "ffc6e2c25867e91947ebe1d8e03113d4066168fa2d6eeb0262027942d80e056b": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.442Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.444Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.444Z" + } } }, "d15bbab335414d4d8b8963bf84d8e6840415a3fc839c797f41e13afb347c0e66": { "7eff53190c5a3759339978f7f7f8df28a9281bca9df3218c5f48b98aefdb5e9b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.459Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.457Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.458Z" } } }, "e524f82a69f9ba0c9ca77d93ce6f9a713d13f108480d3945dba1962f5772ee46": { "fbd98a73453eb2fe0d0b40e9e69f2c6435180be06375fe9f19e1bb909573407f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.717Z" + "updatedAt": "2025-11-29T02:44:55.459Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.455Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.716Z" + "updatedAt": "2025-11-29T02:44:55.456Z" } } }, @@ -10467,135 +10573,146 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.078Z" } + }, + "0d09d70848bc3db09e2e67fdd516909f6d48129455d42ae148932d9d2a956682": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.443Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.443Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.445Z" + } } }, "e9001fe7adae3ee521c4e8d3e207693d2c40ab3153b629428457ad95a126e11f": { "c925c5d3c0431c9ee3487e60721536bea2826b1bda255f0e4e9add7b81f2f4d6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.081Z" + "updatedAt": "2025-11-29T02:44:55.467Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.080Z" + "updatedAt": "2025-11-29T02:44:55.465Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.080Z" + "updatedAt": "2025-11-29T02:44:55.465Z" } } }, "fda80bc8aec70f334713b34cf3411a9095abb45fdde35b4e250c9120d32dc223": { "9447f95299ab26e2dc058db4d4939aabd60236798a21696a52feac53fd714475": { "jp": { - "updatedAt": "2025-11-26T07:24:12.080Z" + "updatedAt": "2025-11-29T02:44:55.465Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.079Z" + "updatedAt": "2025-11-29T02:44:55.463Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.079Z" + "updatedAt": "2025-11-29T02:44:55.462Z" } } }, "14ced74ae89aced82700fb3f104dd8a0694c5c0ea94167d31d43f1c1be2fb09b": { "cb8ca75fddc3df71a3d63cbd9d7f7fe682786749238844ed9083730bc07d7cec": { "jp": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.533Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.538Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.530Z" } } }, "1e26f8437fd2a7483c1f29a2f11a909ff981448ebd08fd7cdce54aaa31e8a511": { "1c028977ab28be717baea644e55afe62584b4eec751926769c92c424bedadeac": { "jp": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.497Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.498Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.496Z" } } }, "1f59d80250965bf621c38a8cbddf5b2e006f67743e1775da704d9729e6c40a23": { "e842588d4a31eebd61f2d17142402ac96b93d20ff0258eb94f82f5209a3ad2a1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.084Z" + "updatedAt": "2025-11-29T02:44:55.483Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.493Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.495Z" } } }, "2024dccfa8cbc20dfede60ca15d6ac2b721789dba97415066cafa133d075bc71": { "ed44ffe66e8c1a1ecf0ca6bc07d18f43272ec161a9e95d0e798e64dfe432b703": { "jp": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.489Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.487Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.491Z" } } }, "44c5128a3edb936a9926b4b513a202ff12d0f2f99f0c07bcfd8be1cc4723be33": { "ae80526735bffb74b999220357c744c052668c14fe1ac555f4b49132850620f3": { "jp": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.520Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.089Z" + "updatedAt": "2025-11-29T02:44:55.471Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.518Z" } } }, "46ae531d604a943d4340ae2e6288c87ed68270217d4e05122a7521d428519ef3": { "fe9a9e2137d1cae06dba9ff8e83ecaa3649ff47e77c5892e5e7eb1529b298c64": { "jp": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.519Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.524Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.519Z" } } }, "54e53c16ab3f42ddf8578d9835bb9d7b843c7a55b19f498defcfab1724ec045c": { "35a38f29e12929f2b225b703480bed8e37445662a61cc1d374ec38bd2400c7f2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.529Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.522Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.525Z" } } }, "687b783276b703fe2b34bfa19c6e6eaaf919ae2edb3b092772cfd3710319c962": { "355157027a1047c82f7755ab15b218d98a8e5232865d69edf8a51337a364b541": { "jp": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.484Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.489Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.494Z" } } }, @@ -10610,148 +10727,159 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.082Z" } + }, + "744f86f0af840c394271f9f85293e3266bb9cf9887e2f173377f48cf9eb8cc0c": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.438Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.439Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.440Z" + } } }, "afabce4e754c0f29b7593d81718787eebb8e7a9f5199eff6f15750cdc8d874f1": { "b814da04f0b9e71448b22e3ba39231b2c53371ce962656e59fcc8215b53d94b5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.520Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.494Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.486Z" } } }, "b27a2ee952ba7dd3cf6b2ca009a0cd8b86282ef4f43897856e759dafd88540fe": { "32217bcd37f2a7cf5746ec775356d9800b981082b7b912e76343ed0600518f76": { "jp": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.493Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.485Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.485Z" } } }, "d1dee74d727375041e64ceadd78df956b10784ab7e1b4ac16460115a7e9d4ef8": { "469305bed4de1b5eb391960ebef6f0f5096cd86b537e42c0f37ee9f35e087a4c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.490Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.487Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.488Z" } } }, "d536aa9054b9ba72e39103f4f8845be09447ae23a9e70d3baf478d3d2c2b8737": { "74f18e7520467c6186fd7fa39a49176648617574146477b17ce7062d7698f2df": { "jp": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.498Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.088Z" + "updatedAt": "2025-11-29T02:44:55.500Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.087Z" + "updatedAt": "2025-11-29T02:44:55.495Z" } } }, "e56710647bd4bc05384aa8d37b2b94c4c5fe719ebbc36c598369a813b3fab06f": { "7fdb5ba5a1e64258c7ea2522a25a1d7238e73b82d6eb92fdda33bd598193863c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.492Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.492Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.491Z" } } }, "e631f7491be262a58a274692876ad3b89d0729e8d0294ce9b3dfa4ed45f95788": { "f3609e7b117bdfa85ee3d950a8fd8f7afee96311aea43d1a833a253a135d50ab": { "jp": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.521Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.518Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.517Z" } } }, "ec626ca92c333df76f0e8018ae7bd3a25ac79ecb449e8db31702fb29bb04506d": { "ec424602c359c5773d3bb1eb5b167bdedb80fb98f907e5848b487a5b40325f67": { "jp": { - "updatedAt": "2025-11-26T07:24:12.085Z" + "updatedAt": "2025-11-29T02:44:55.486Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.488Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.086Z" + "updatedAt": "2025-11-29T02:44:55.489Z" } } }, "fc2a90cf202e8e1844cfa26c61201f10a6c234df6585fc1c8aff86d125238563": { "5680229b7edd18b624f3a4822177aadd2e3930de72a0edd50a0d2924b785a146": { "jp": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.537Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.535Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.538Z" } } }, "1646d3380fb5c92ec41482a9d98b525c37462130d6b01f32e1855b0e5f91c39e": { "ee6d9f1af26926d6377c040c2405ae576469664c532845e1d506079f9a027314": { "jp": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.560Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.558Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.562Z" } } }, "1ca8dfc5de116b6a2aecfd00677ce016075dee9e46cc6f57c85776d3ea9b3bd5": { "e84e0b80c498c3151e15f60e104f2cb38c6e40319081435e228dbfd13acf010e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.529Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.524Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.531Z" } } }, "1d1e36aa27a61854f94b1f60418f1a1d666d53319de3e83255d9388fcdfb4069": { "a0e30e85a93f908ea864b663f52f1dfce2a0d6a87372b01c7bf971316d114876": { "jp": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.568Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.565Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.556Z" } } }, @@ -10769,195 +10897,195 @@ }, "dbf76c8c4d494516e235aa48fe905a94dc6f8466afb98397c14a38f6ae43f681": { "ru": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.503Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.503Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.504Z" } } }, "4d9f585a978f327ccfb66e8db78fa87ec8754d231c098b1a8929e4f912be5651": { "f0713cc147455f15a45af300160f8c01445b53f171e027819d998a3df1dc3b17": { "jp": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.534Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.530Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.531Z" } } }, "509c73a63f9d009e86d97956ae4e1701003ed2be70dd32b5c56c66bd65c22609": { "c01d58d811ef80a75a56846d05c7b54259075a78eb6a2deb665f4405f861a7e2": { "jp": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.558Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.564Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.559Z" } } }, "5bd267d7d3d49be2e95b491604023a269bf78bee49b4a83eefa9352690913107": { "9e71d3c2fa185cdf2d0231b06c410ed213fa00b972cdbfefe21a9aa8916bf03a": { "jp": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.562Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.561Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.566Z" } } }, "66dafb9b646deaa517a7b992eec446570c152c02802db14e18047fc0fba7a0b1": { "f246fb415a6d823d2e1229aaf83e9eb73611213283605b91a0a23a1dbad24f50": { "jp": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.561Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.560Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.555Z" } } }, "7d4c81a663e077a5e75150c0e14d27c4ec51b540adb7aed379113d299f3c76bf": { "9a1b6a07af2168ede1ef0940be49f9f7462ec53241267251f36458e33a1bd688": { "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.552Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.553Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.549Z" } } }, "8b2242e50cc879742f4d4efca957625a1106cb09f45a18de469646abc82467e7": { "343ceb09449e64360e7e7fca397cfc927ac8e348304b9893b3946e0ca65d8fae": { "jp": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.528Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.527Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.532Z" } } }, "c02bec6d7a15ddb4727d64f0c82f001b4a6994e6095794f3b35c713c1c69cd75": { "f05e5879650490f810241a7e1f46402021938daaf4688d3368c183eeb6dd5b65": { "jp": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.554Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.551Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.556Z" } } }, "c35a4c218452080886d36470ffc05c5a0554e095f00432e0d7735900c7ad9435": { "9e5d4bd1e5379d30156d61671b947abb64b0c0e6ce551d838d6da2c7907d2ff3": { "jp": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.557Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.548Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.555Z" } } }, "c97fb19d4fbdf784a9e8916b6965cc8a3ea8fe90f09cfb7c399b3b59efc788a6": { "7b99574846f0eeee45a44964ff5ba57e7c06ca117dc6786a3b1b13201c58cc4b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.523Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.535Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.722Z" + "updatedAt": "2025-11-29T02:44:55.528Z" } } }, "cfcb90141d0b37458f1c2f25a8e506c7c2ceb4cb4e4c27f996e474f6e8c5b159": { "2e2c5497230ef2998811f833ae91e6403540c85762a127d81135370dfbdb4e46": { "jp": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.564Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.559Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.563Z" } } }, "d5c4d2aff5bcd49a39c5a8969a66d9058ea8a6641de98e1f49a707b2a5eb6a06": { "c0bd7005e30dbceab4454c02004199f159d34c9dec509a5c13f2a23d8b720cff": { "jp": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.523Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.522Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.720Z" + "updatedAt": "2025-11-29T02:44:55.521Z" } } }, "eac3b18e7887fa005afb72b037867082f68f247bb61d91f3260e28d28cb1e85a": { "d2aa320a8841951470c1da7b5a35b1b69bf507d11d9b795481a4e587ec4b7bdd": { "jp": { - "updatedAt": "2025-11-26T07:24:02.723Z" + "updatedAt": "2025-11-29T02:44:55.533Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.526Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.721Z" + "updatedAt": "2025-11-29T02:44:55.526Z" } } }, "211a9e255fdac9865968252978823dbe623bf314b09a28779424fb52243ba37e": { "267373ee71eb85826ed3e41dfc0938bb71fbd6c83484df63fbdce933b1a28d1e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.570Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.569Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.572Z" } } }, "4ba1eac8610621c18306898ccbcb9d4eaf5521b4b230d99cc774ec22219c9a28": { "1aafbee1019940fc3e073990ae3817e08af6f7e2ec670ece7d26a194827351bb": { "jp": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.553Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.552Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.552Z" } } }, @@ -10975,676 +11103,676 @@ }, "6b408329e73e771ef16b20b1ea0ca91a113a919ef9db09031d4181b038bc38ec": { "ru": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.505Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.505Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.506Z" } } }, "67ea1760ac764890c103f9795d76f618a583b0bbbe0d32ad38a77c020d119d40": { "9a32d6666fc830213628b9c378f0039bc1280491f729f8bb75dd81bd764f13e5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.573Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.571Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.730Z" + "updatedAt": "2025-11-29T02:44:55.571Z" } } }, "71b7871a9e60b8462bb9bc1ee2ff376b1641403aad826100b88e087426e5841f": { "3ad40142a5980106f0b667308b9b61cd075b9a565aa267c085988df32d9f9d20": { "jp": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.566Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.567Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.728Z" + "updatedAt": "2025-11-29T02:44:55.563Z" } } }, "a9dd86f5f7da605aa9337f714a106fa513a631fcf9a168aa7b4e9a3b7ccaa531": { "ea6fc6dcc9635bc1877901795f75089be17712230ae183401a7e6eeaa9cfcf78": { "jp": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.568Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.567Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.729Z" + "updatedAt": "2025-11-29T02:44:55.565Z" } } }, "b4b5cab881a02e5c4333f93e3149c6242284e0666d745952f3ccdc86593f7b52": { "112d13bcf3046cf70aa9ad7b11bd473fb40eb530504362a77d2a53dd8f9adac1": { "jp": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.549Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.551Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.554Z" } } }, "e21164b6c8802133bb1a3d2aafc3fd517ab74e6f8d293b7d293ae968782a8bd6": { "04d3d33fa3cda8a0df74a6fb806ee0f2d01d7cd25cf9f21c9e07d1830f9a9a6c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.540Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.724Z" + "updatedAt": "2025-11-29T02:44:55.539Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.725Z" + "updatedAt": "2025-11-29T02:44:55.540Z" } } }, "f9aa45e8fc85d0cb2d4c76b0e287f8743a40e6d92257f98ad0691dbde7bc3a9e": { "4866f2bf5a753196ff65a8b94a288fa39116ec9e4deeb7ae77c0598af8d582d9": { "jp": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.557Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.727Z" + "updatedAt": "2025-11-29T02:44:55.557Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.726Z" + "updatedAt": "2025-11-29T02:44:55.550Z" } } }, "3e29eb5aca75381e4ec8ade4e6a0cf7d26b53d4a25cb26660ec2c44647941a73": { "c0bfc76e21aac5582f52b976b44aa4baf44b8f76caa3d562ec73e6e4ef161a92": { "jp": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.673Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.649Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.653Z" } } }, "4b875d4cf08501af46c9a0dc4af0b755918205b50ba44a03d48aab3f7f49ac54": { "658a06aa55917c46e77861ee9b9b9643be0049c255c7052d4f6ae6166e655b01": { "jp": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.678Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.704Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.680Z" } } }, "50ddd976e3ab8042db7b5db277b40561a4de66f66d7343d572a7ddd20ad31bd7": { "0aacc185d8105f7e3ea27585dc11ab225da3bb6c1db23c8daa11af166d8e972a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.677Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.677Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.674Z" } } }, "54e7a0d28f060089af44ed7367d75f254a6d1b252f6ea6274e58dbe249470b30": { "4ced947fe881a2f40e14c2be1395d6c2cc3e15fe93e42e71df52ec929c2dcea4": { "ru": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.678Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.682Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.703Z" } } }, "7a97c0a8a1d7a2b7124253b37f3cdff0f274d654965381e7ee3aeb4db3323631": { "ed2621c01542cd6c73825e5fe7639beff16cce375577d0d908b8b02c4bc1371b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.649Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.642Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.641Z" } } }, "893f6ba96a900463e4a20bfebef45d262bc3a3e1452bbe2f889f333b52e5fee5": { "b3a0a7a9c4f2e4c526bb71ba0bc5e6dac553aa232350b1910ad7fbf035734c06": { "jp": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.646Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.744Z" + "updatedAt": "2025-11-29T02:44:55.609Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.652Z" } } }, "95a73804027437518f6cb49fd17638db0b1d6b9361ef329c1d59b49231f45112": { "e13f5fe9c753ab5e1cd5c3b9ef8db4c7e56caa299572d07d0368d8af887e99a3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.681Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.702Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.680Z" } } }, "b624c3e0df3b6286b5d61538607b9030a6cd27129246f0485ab94c5f1b0efd7c": { "b4c584ccbf84daf8b7fe6aae9e1c393e8220224a9cecec6d5d2024e0cb7aa654": { "jp": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.684Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.684Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.092Z" + "updatedAt": "2025-11-29T02:44:55.675Z" } } }, "e210bad99f1e8a957566f3f34d0853651d4ef532d83ae50fc1fb032d24e2dd28": { "0b6791886d00299fd2b8b71cf58d276a85916e6880c408cdbef78333d00f1d3a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.649Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.093Z" + "updatedAt": "2025-11-29T02:44:55.683Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.096Z" + "updatedAt": "2025-11-29T02:44:55.702Z" } } }, "e77458d405603be885e941ab39a2c03ea7c893b38a1ed1b1c4a5beb9a703c04f": { "f78ef201b8464bb62128fd17fb1bcf8d3f42f167e5b4f4c8547866c5ecfbc7a9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.650Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.674Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.091Z" + "updatedAt": "2025-11-29T02:44:55.643Z" } } }, "f38d701b9050913191927978c4e261dec48019c2bef4c6a3139c4059c71afaf8": { "0e1ad7c4e88f314e2b810b6f27ec43ba78bfe09eca3eec7d023374756f07bc64": { "jp": { - "updatedAt": "2025-11-26T07:24:02.746Z" + "updatedAt": "2025-11-29T02:44:55.653Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.745Z" + "updatedAt": "2025-11-29T02:44:55.646Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.090Z" + "updatedAt": "2025-11-29T02:44:55.673Z" } } }, "06b6f9b31956eb6e3cebe7421e22abac9ad0de32434585b3bedb572ca22fe779": { "ac6f44e72647bc384df3ba5b105e8bc37e9ce25a9c1c104570232ed738108026": { "jp": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.920Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.926Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.878Z" } } }, "088f126360fc7b556a09516cc41a4880d4599464d2cb1ff9f6ea02417c6df429": { "04f510d66c9b376ce9989e4858fb9d1204bb45b666002f527435e252cc2dc4f8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.108Z" + "updatedAt": "2025-11-29T02:44:55.978Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.108Z" + "updatedAt": "2025-11-29T02:44:55.978Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.910Z" } } }, "13195f1c973faf9aadf39f45b6a4df596efad0f6e4df019051e13dc77eb9fdfa": { "948846a8743f4a90ac77c6ba53e93f5386df8d5310a4b8182265798313dc6dc9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.879Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.932Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.936Z" } } }, "2505693dc142fd4f445b3882dc548fa0cc38adca662a63dbfdb437b0f67776ba": { "f86b0dd8e53eca99c2eba408e02d7d92a906e77aee88846c9e24a2d79f1d998e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.882Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.918Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.931Z" } } }, "266e0dc9c395c310374563d981fa2685a69b11a4eb800352e56423b5bd7e2901": { "d344c46f769e848e76522e3e0e64f31e4c4cd999a3de3ea3cc10400f0b2826ae": { "jp": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.937Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.938Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.940Z" } } }, "3c3cdb595236de7ad8f9d05838ec2b8bf3f90caa6bca9eb1dbb703fe9b2c5f67": { "22c4567427f06c4ff596058d0963e1977f619d426a1cb0b04f22ad1721307091": { "jp": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.914Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.930Z" + "updatedAt": "2025-11-29T02:44:55.886Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.887Z" } } }, "3cb2ac954c25f39475156759f2f4f8c8714328c659aaba596322bf83f3e3ecf3": { "da8c2bbfc6c34aa9551b3e0a532d71ec831fc09659ffc38734155072f907743e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.922Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.876Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.917Z" } } }, "3f5009534c38cb29edcc48a3b2c5b50aa0363797569ad9ed3c962e075be3d711": { "e52f05211d11daf47cbab45322de5fb579805427116030493d255d74a6de33e6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.890Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.891Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.846Z" } } }, "51d439a5ad94546b36a253aeeb85868911bfe6475f4fefb30756a75f43e01dc0": { "c9a05803f13e75801b4f09b8c52974299028da9cd5533d505c572edbdd11b9f8": { "jp": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.888Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.887Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.888Z" } } }, "5227584ef900ca7684b844bf9b013a21d6faf12f8833191ac40e941a5fa9878f": { "5405382560ae38c848c605acfb1a4ec134912ef6bcad95aab5381530689e735b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.892Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.893Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.889Z" } } }, "a5397922ad119e6b298a6b4b378a68f864ea43c8323107a35954165809de0589": { "488ca0a5b4cba0af7cf4ca440e3733d6860db7e0e1beb8403ae74e4cfd8e7753": { "jp": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.877Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.920Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.918Z" } } }, "c6e56f828d1b34579ba790f93abaa05b29fb89f9585497258413971007a3a246": { "c2f203731c8694cfaf84b37109a789c0a0167657339f75db8fc7b685f948d2ea": { "jp": { - "updatedAt": "2025-11-26T07:24:18.932Z" + "updatedAt": "2025-11-29T02:44:55.892Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.929Z" + "updatedAt": "2025-11-29T02:44:55.847Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.889Z" } } }, "c8b0b34a39a4f363d421259bdd17b9dd8d0d01f815eda9607f0d9ef245895275": { "1126bfe846bb5fcdc4b0c7c2bfd10807cc64d6e12d190d2c824329258baf5efb": { "jp": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.886Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.890Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.931Z" + "updatedAt": "2025-11-29T02:44:55.891Z" } } }, "ce10e9c3dd234b8bf0fa7265cc3f51606b9f80563a4be89f36f9805412c6a452": { "f80ac33db9f2499ec8763473f9aaab8f92e4f89d4fbb898fbee33da6e7d210d4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.921Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.937Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.934Z" } } }, "e8941cfe3ebe51cf895d37bfced51319951864655bb65ed34110cfbbd542b577": { "1724335ae6c5171c92d1126311524dbb7f3ba7d451a7907320b5c0cbe7ebb3aa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.938Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.919Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.932Z" } } }, "ee1d174b1119575726aa2ce11719dc7482af9a58eb1e4c20075010bcc5bc200a": { "85b1114daba44b005630b9c50a7b4b79dec7d53f4ef54586f1ecd92f3f5c5d72": { "jp": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.916Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.878Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.930Z" } } }, "0cb711998b74fbafee6f9dbe1cf42999cd6bf81fb67aba52da75f9d6e7820916": { "1b31920ed434089b5c438486640b5af358c740bf6e33cef31bc59a7a8cf7708b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.946Z" + "updatedAt": "2025-11-29T02:44:56.008Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.945Z" + "updatedAt": "2025-11-29T02:44:56.007Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.974Z" } } }, "0de197a09c02a6e7de6b2120720f01b2f26dd69cc09e57640234c52fe619cbe1": { "a3b2b2da1705264e477035d4c4f93d27e7c159e13c8fefc67fdbac404fa1df2f": { "jp": { - "updatedAt": "2025-11-26T07:24:18.945Z" + "updatedAt": "2025-11-29T02:44:56.007Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.945Z" + "updatedAt": "2025-11-29T02:44:56.008Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.942Z" + "updatedAt": "2025-11-29T02:44:56.002Z" } } }, "39f0108c94bbc9ceec29295e4a5c4a30bc3ed66e79dcf055c93bcb5e07df95b4": { "f14661437615304886b90084f8db1b8e50ccb8718cce1d8bb57271192cb3f924": { "jp": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.942Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.941Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.942Z" } } }, "4511c24ad879085d0713bffa28b8695c1a87d24872ce30015bb857f43c961627": { "f33dc7dd4c81c9ff62d672ddd22da52fe2b3790feef29653e27d7dbf105dacdc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.933Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.922Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.921Z" } } }, "7209b7ddab6e5d0aa0edb6dd2a9d28893ced1fa4a5e84eca66e18a12cbc9a471": { "b55f055c6ea298013d180b87459ca4cbef2d564e3a47054885bf85eca5781ed7": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.880Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.931Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.930Z" } } }, "8d5ac58622d05dc878c50a9901e001b81276e5c37349076f70389f7ec8731cb4": { "2a5bbf839d622f7ef15b7a5b8575e42dcbd0d1ab16bf6f98ab233f94cdbd68b3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.919Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.935Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.926Z" } } }, "9da34b15afe0cf84a2c73d8d1acfc85dae89be8c90605898caceecbc4626da99": { "ce873407eda99feac5ab7638cb9c330da28e87de5b88e7f7e35b3b8dba2c1ffc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.934Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.916Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.916Z" } } }, "b1eb4813b41c7ccc174d13cca2cec0592da899961dd7a66e59673fce738d90ed": { "d63a4009d7fadde4213a7f160c8741c105b3a63db320d984e375579df904dfc5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.939Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.926Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.940Z" } } }, "bc635d7f6a9111bbbc3d31c625fcda3adb9eadc78253335799d1b3a12a509df7": { "b7a3734788840b662f127af66b64815bd7c85bf39dd4cf42306c85eb6f392d01": { "zh": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.974Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.975Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.976Z" } } }, "bdf357b395b129f57e836477b2fc57675705bcf48e1acda08c190ab17a75951e": { "3a0381755f449a5032606d2fdab638ca733950978814b42e1aceb74203a2235b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.913Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.971Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.913Z" } } }, "c54fab4cf7043c79b8ce701279e089e154ad852ea3c4248cb2c8da671cbc17db": { "b6e7b7146868d159e85bc698be8dd009a8755c7a8c993e4406163a4d71a408a9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.920Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.930Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.923Z" } } }, "c571247fa3e091098d027771a55d5ebe774d6d531b2c5384736de73837552959": { "e5aeca6ca592dd8ef3c7bcf54b278d64dd04a95cd012f8594105429290303c21": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.879Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.938Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.927Z" } } }, "cc311a7d9ae7be3e04c62efd5c5b7aa8cb9d6075749b29e99939d01baa76e3fe": { "3de10984a294ee3ab3e7105d5ba6c42208236c0f01721e7189efb0af99ca2490": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.880Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.928Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.100Z" + "updatedAt": "2025-11-29T02:44:55.935Z" } } }, "d49e422d3649c455dff8cb00cabeffadfc176bab2340583e83d8416f5fbb799a": { "551eaa35224112c0edb3f9e68292a528790f07b1ea2fe15b67e97ec37689af33": { "jp": { - "updatedAt": "2025-11-26T07:24:18.937Z" + "updatedAt": "2025-11-29T02:44:55.917Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.929Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.938Z" + "updatedAt": "2025-11-29T02:44:55.927Z" } } }, "ee343f5a3bf00722c8dacdf2e096fa970da83e5102fcb1446bbc99a4b089a390": { "72f38826fa27979a73a67e5413b3854cc5f5f2bfca9f1efe2890e20dc90a5020": { "jp": { - "updatedAt": "2025-11-26T07:24:12.098Z" + "updatedAt": "2025-11-29T02:44:55.929Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.099Z" + "updatedAt": "2025-11-29T02:44:55.933Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.928Z" + "updatedAt": "2025-11-29T02:44:55.877Z" } } }, "fc30da7ebddc5996d940ca4f9540cee6fa6b79f9c37ee5aa6cd56665488a65e6": { "20ab3ac2e587dcfbf842ef0e2dde364c4fac02225d76cf6a5a4b6a646b77e4d6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.881Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.939Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.881Z" } } }, "fc92ad70da88c48808fdb53f68400c73f6f900eca6a95d544909915d2b22d9f0": { "16c47449f52759987429555de611585f7f1f6d6770d4c1ced0d74ae244ab45df": { "jp": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.970Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.968Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.935Z" + "updatedAt": "2025-11-29T02:44:55.909Z" } } }, "fd2a3635e203221890fdb75fdb12cad083607f12a05af6e46565b58b28626a3f": { "69e391ff6463d09b09730e7e4366b4c486d3bb1759441114546febf2e97601a2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.914Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.103Z" + "updatedAt": "2025-11-29T02:44:55.946Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.103Z" + "updatedAt": "2025-11-29T02:44:55.953Z" } } }, @@ -11659,18 +11787,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.953Z" } + }, + "99dd663f4b6f1f7c866a09ecc4aa890f3ab244afe08834a22596edafef952ca4": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.984Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.033Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.033Z" + } } }, "0f88f2bd27c6a3bc5b20ffd358c1599368da4a7821aed81420035a719675f40a": { "947a7d558e471c72cf79437a217f341c9e6e2083cef8d20956a3839b9c085fa3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.972Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.962Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.969Z" } } }, @@ -11685,83 +11824,94 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.953Z" } + }, + "634a6e8cc715dfe8bb5f0ed37757b192d5e78bbd3d002d9e758fc5e3428bf252": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.991Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.021Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.031Z" + } } }, "24f89815412a9281c45be003f0d9b1edaffe253b9fb6e44d0b69114de2a8bb5c": { "856a0875860cb4e9fdc7fca531785d1b4ba67b93fdace5421889ea8cc500ef1f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.103Z" + "updatedAt": "2025-11-29T02:44:55.915Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.103Z" + "updatedAt": "2025-11-29T02:44:55.946Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.103Z" + "updatedAt": "2025-11-29T02:44:55.951Z" } } }, "309ede6dc77ce6e13f59939faf3901841606c221b6288c941fde4d97ebdb53a0": { "6f8a294f73c0c480fd655b8e9629eea0f71262b277a7a5e6748a17b7458ba88c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.949Z" + "updatedAt": "2025-11-29T02:44:56.013Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.948Z" + "updatedAt": "2025-11-29T02:44:56.012Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.947Z" + "updatedAt": "2025-11-29T02:44:56.010Z" } } }, "417572f3f0c0dee81daaaf436d03f842c9160631b01f165595d06d9e99f3c6c0": { "bedae71b49b3c79b70e3ad0767d167ca7bf7f0cf3792f2786f3be6e243ac41f5": { "ru": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.944Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.945Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.943Z" + "updatedAt": "2025-11-29T02:44:56.005Z" } } }, "453e82594457c450e00def5b4a049c6817c1f11b3242ecdc0c113a4fe824bda1": { "3e341e3a84064fbb72d1f07486692fcc58eba4c23ed96700a8697e160736a689": { "jp": { - "updatedAt": "2025-11-26T07:24:18.940Z" + "updatedAt": "2025-11-29T02:44:55.998Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:56.000Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:56.001Z" } } }, "4f6f1a6da73f8d186f0a18ad4c69138ec62d12a6b38064449c0eaf1293c82145": { "19880790e9525db190f5e72d85ffc766a344cde65183576c30c03ab560c76bad": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.911Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.108Z" + "updatedAt": "2025-11-29T02:44:55.977Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.108Z" + "updatedAt": "2025-11-29T02:44:55.977Z" } } }, "544e14c8df8e9aeba587c7a01debdb6de7b8d0dc480e2a471b321fe3cd637687": { "56a8436026a55bc58795064c90dcf48eb1783d7c4aeb6e25f3c6be910d52bfb0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.912Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.973Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.969Z" } } }, @@ -11798,44 +11948,55 @@ "jp": { "updatedAt": "2025-11-26T07:24:18.940Z" } + }, + "ab2df52995cf3997a3baecd856f3989b8f29d5c7696a2ee4616a5675d7e62391": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.955Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.992Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.992Z" + } } }, "596b0a954cde794b5e64f8babd89a81c1359843a6046215dd00cba539357857d": { "af24567e7b2b1b9a842510afc1c41e6a4f7a9634fdd16e4176a76bc4b3c3e091": { "jp": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.970Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.973Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.101Z" + "updatedAt": "2025-11-29T02:44:55.910Z" } } }, "65351c23daaa6ae3579c1740e82b1f56ce6eb541ff65d23ed1f890694f6ea440": { "b999ab8a06deee210039a2eaf91d71da758c776e64c8fc322d876e73e8db2861": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.913Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.968Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.971Z" } } }, "942eceae58e0a094962eb7383ca418c7a0fb355bbdf35ed09b1fb271b8ef0622": { "a06cd352188c57c4dc80e07b3511cf0c55b644a5eac9806b52fee16a901321cc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.911Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.972Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.106Z" + "updatedAt": "2025-11-29T02:44:55.969Z" } } }, @@ -11850,6 +12011,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.948Z" } + }, + "d74811a43a5692033b0db3a2c94f0a673f6dee42d90914ef696df7c4ddbac655": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.955Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.986Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.988Z" + } } }, "a4265198145ae0d412153c5abd0417df656a57368c80042916957e2c00936a91": { @@ -11874,57 +12046,68 @@ "ru": { "updatedAt": "2025-11-26T07:24:12.104Z" } + }, + "2d62d69aec7a3be320c6934254cbe83967fa5402dfeafb0dbed6ee9c6bb6dd7e": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.954Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.960Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.961Z" + } } }, "acaee03135e8e96bcdcf34c15546b735f613d1e5ae560184c16e47ce55501204": { "8a07567dde3044656ee0f3a1ecdd3437e3653bc1dbd011b4bab9edb2c0e04c95": { "jp": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.975Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.936Z" + "updatedAt": "2025-11-29T02:44:55.909Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.107Z" + "updatedAt": "2025-11-29T02:44:55.976Z" } } }, "ae900fe149a5b14ee7a25d9f850a7fed9bbb24da3497c1861285d73a625852e6": { "178aea88d150360011d964d55863a4f9f7585cb6ddc5b56d142898d29ed03414": { "jp": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.952Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.953Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.952Z" } } }, "cc14be3df8410373edcf3ea623a34273b7005b0668dcb8d261ee3fbada8f972a": { "029f36173935f1b92553f610da6f3be5d9b0976fea74e17265186d40a9f8f8b7": { "jp": { - "updatedAt": "2025-11-26T07:24:18.947Z" + "updatedAt": "2025-11-29T02:44:56.010Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.948Z" + "updatedAt": "2025-11-29T02:44:56.011Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.948Z" + "updatedAt": "2025-11-29T02:44:56.011Z" } } }, "d8cbf85de396e8d762bfdc573d415e4482bb687b9017d25d153c264728283316": { "62c5c6e1debf8e9f65330683895c791394dfa2b8f1cab9a3413558667b58ec1c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.944Z" + "updatedAt": "2025-11-29T02:44:56.006Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.944Z" + "updatedAt": "2025-11-29T02:44:56.005Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.947Z" + "updatedAt": "2025-11-29T02:44:56.009Z" } } }, @@ -11950,18 +12133,29 @@ "jp": { "updatedAt": "2025-11-26T07:24:12.105Z" } + }, + "9833a13f1be1707f9e1e0c52ffe0e9c26df1dca4c21c0968e308a62a79da3578": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.956Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.959Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.959Z" + } } }, "f6c796e2a223d7f3109d7a0e087b02b576111cee44e1affe20a492544e19a35d": { "5c1b2453bc509571ef5a9c5a79343853e690b58e16dd273eb4fedb719f0aabd8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.981Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.980Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.108Z" + "updatedAt": "2025-11-29T02:44:55.979Z" } } }, @@ -11976,6 +12170,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.954Z" } + }, + "3cf60c4c63f78ca99a909e11fdd7b9ea46873225acbb7444b8eb966b6e9c4838": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.985Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.990Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.022Z" + } } }, "134051a83cd4931696ccf591823f46126c6e7feed4971511434916eff09ce79f": { @@ -11992,13 +12197,13 @@ }, "7676a41c6d1e719ba8b13b8d322ace741b11f0fe672d3b38397d5e1d23081fd0": { "zh": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.014Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.016Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.016Z" } } }, @@ -12013,6 +12218,17 @@ "ru": { "updatedAt": "2025-11-26T07:24:18.956Z" } + }, + "4e4923ee24e6317511ddbea23c7b6f8e03c0277f9d6ac0d56eb56dd0caae3746": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.017Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.019Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.050Z" + } } }, "1c4c51a336d1e6dee310539258abd450be7834df46548255e22fae5d3686a247": { @@ -12026,44 +12242,55 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.954Z" } + }, + "5c602e02407de45f557f89046cb7b30682c56eaedacab7a3bfc0f1d208a1813f": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.982Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.024Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.028Z" + } } }, "39df2af9870d3b0cc9ef00711b97902ed3b5f6df0419aadf0841770290785d7b": { "a18203de1411607a70e1437450eccbf17a073e8daa45c5c42ee8e0cba812d5f3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.942Z" + "updatedAt": "2025-11-29T02:44:56.003Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:55.999Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.944Z" } } }, "40220941c00a4eef0a2069b02906e525beca179d4a354e0a2e5a911c363640b5": { "989d53822380f38745d79c1b84562bfb045e678799d0f7110947e9bf5d599700": { "jp": { - "updatedAt": "2025-11-26T07:24:18.943Z" + "updatedAt": "2025-11-29T02:44:56.004Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.940Z" + "updatedAt": "2025-11-29T02:44:55.998Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:56.001Z" } } }, "505cd1f1060fe51777563a177f877b84419bab382656d36901ea1615cd4c5f44": { "0a35a92e535e80b3a150fd73abbc1751ae0fa2688543577feac7ce7f4de53ae8": { "jp": { - "updatedAt": "2025-11-26T07:24:18.953Z" + "updatedAt": "2025-11-29T02:44:56.043Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.950Z" + "updatedAt": "2025-11-29T02:44:56.037Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.041Z" } } }, @@ -12078,6 +12305,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.949Z" } + }, + "4b25da1f59890cfa5a986a65cb021896d1be71d0919069a68ca0edb32d4bcb78": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.993Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.996Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.997Z" + } } }, "6d56ddb9a5b3ccdf4eae29f57959e9374f0ff177ac9800e0d460527344dc64a0": { @@ -12091,6 +12329,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.945Z" } + }, + "eed3064741cb620f55aca288e3255646f75384bcfd7a8d5d63104f69d26df546": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.957Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.993Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.995Z" + } } }, "839030474f427a460a6acfb9a8caa7662e1cd0c337e35995054bd2c956ad05d2": { @@ -12104,6 +12353,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.944Z" } + }, + "06b19ed602eff6e0f4c0b38b69d416263621f402f6591d9f78725c9fb8213249": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.957Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.958Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.995Z" + } } }, "90511d719daa226bb864d0d2bb0fb993971dffcc30b3fda0d86ebc7ff7157a9f": { @@ -12117,18 +12377,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.952Z" } + }, + "1921b13e0a9667a216009da681a38ff65bb6e8e5e69fad4427f31e0dec85b1b7": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.983Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.030Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.031Z" + } } }, "a0e5cd4bbd52095f645996d5a20cc34d462aed2b014ca882138e4ede52f7b410": { "b82f6c4650551ebe5f3c0e03e15ad59d0e9d79edf78e121c65d4de264d1e000e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:56.000Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:56.002Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.944Z" } } }, @@ -12143,6 +12414,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.945Z" } + }, + "5dd603d0ea09ab4c18610adfac733616566b9465cbd159ab38037b65cf3ef036": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.958Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:55.990Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.996Z" + } } }, "b52e68b0fa137214aee6134202f0428952a2f49c83cef796e483c36598106cd9": { @@ -12156,18 +12438,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.950Z" } + }, + "6a2d3b6f6eef53b77da900c4d9383e4c650a1b67df3b4bffcf8c1c982c61e7b0": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.986Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.022Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.024Z" + } } }, "bbbf8ab907626ae0bd4c2e7f8f1e1a30e187356616b813a7f2bafdcb968b16e9": { "64de149ea6c99450b1d0a61247789522cc099815c912ed33f53b378aaf837bbb": { "jp": { - "updatedAt": "2025-11-26T07:24:18.944Z" + "updatedAt": "2025-11-29T02:44:56.006Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.942Z" + "updatedAt": "2025-11-29T02:44:56.003Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.943Z" + "updatedAt": "2025-11-29T02:44:56.004Z" } } }, @@ -12182,6 +12475,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.951Z" } + }, + "805a3c63750885bfc49f7e57abe2e015684ddc6eb6b23a0704a589da9585ba31": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.029Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.030Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.031Z" + } } }, "be5a795a34e525ece1f0651b8ec65280cd3f71026a239b44cb087800474d6992": { @@ -12195,6 +12499,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.953Z" } + }, + "ab85b9c6ab0099af15f4a6be42857b06c4aac25bf43a4a5260304fb4ff3e6f6e": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.027Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.029Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.032Z" + } } }, "c3c4a5cfc613b8b144029f13d913022c2d41ebc3c333e2fa61ed8d2f0df5a81b": { @@ -12208,6 +12523,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.948Z" } + }, + "f9910c6a76a99ff49c7ffee4a3987ae9207e8d7db851b61ec2efe4f6c5c50886": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.989Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.994Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.026Z" + } } }, "d559f4bb7e0e75b052f6989f63565615397e09d8f05bc7535ae634a02281b78a": { @@ -12221,31 +12547,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.948Z" } + }, + "fdd897a9063250be652162e225953c203c05911a364b1cf87f47fa2b3ad6b297": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.991Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.994Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.995Z" + } } }, "e54eba7f7c2e2d6d452b2d73f4934f9ba018e180585b2bbdb2f9f14bb9b5510d": { "d88ed4dda50a3c9ee265b067c0abda94e3cba629d2d6c9a695d77d254c4cd372": { "jp": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.943Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.941Z" + "updatedAt": "2025-11-29T02:44:55.999Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.102Z" + "updatedAt": "2025-11-29T02:44:55.997Z" } } }, "f871545252cead274f81eec090f4a37c79aad733b302ff49eedc5242ba29b1cb": { "5ee24061522cb5a7ed68e5bfa59c658c0cb620eff70e3736f5e3800597533e77": { "jp": { - "updatedAt": "2025-11-26T07:24:12.109Z" + "updatedAt": "2025-11-29T02:44:55.982Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.950Z" + "updatedAt": "2025-11-29T02:44:56.036Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.950Z" + "updatedAt": "2025-11-29T02:44:56.036Z" } } }, @@ -12260,44 +12597,55 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.941Z" } + }, + "295554629ad06cadfbea57e08411547a44046d88bbc5cb20b34c13534fca808f": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.954Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.987Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.993Z" + } } }, "00f0f8e4c4cba686bdd32c7eb510c5ff9cf2847654153d708f69ef3d1fae55b2": { "4cdabdb9af849dd79c526565751107e9b1abf0b12889130ad0f45424328feb65": { "jp": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.090Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.095Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.093Z" } } }, "0819a7d3c5f71c0454ca26bc207870bf57571e75b815f9e6048c755eba88da5b": { "7c183351205668c7bd2a340b5ce1c7a91fbae1b7555a939a4d8e6611fda87e09": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.100Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.096Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.088Z" } } }, "0e624ceaf217ed28aa49746f8a0d8e6f11f50144de84c79c5bfc3cee61b7f1a3": { "2c646c9eed127c879e1e79d90542ee56c28b87e87984ce2e15248bed89ca7aa7": { "jp": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.091Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.092Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.097Z" } } }, @@ -12312,18 +12660,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.957Z" } + }, + "ee5e9f3162a1197524aca6e97cf739a11ca65612be1b597e70bf103f7727993c": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.063Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.065Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.068Z" + } } }, "2395cf7e448505fe5dff52c83b83b0eb98f08d6b30b33dff50d6380fa7e5932f": { "773ced00aebc468e3a46c4cc78b523aab8880ec08d2fdf077d970783ea2663cf": { "jp": { - "updatedAt": "2025-11-26T07:24:18.958Z" + "updatedAt": "2025-11-29T02:44:56.075Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.069Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.071Z" } } }, @@ -12338,6 +12697,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.954Z" } + }, + "b0b56f3e8d31dcc0efeac6d2feb5c7674647f73163b6e6bc288532a7e63ee696": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.988Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.989Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.028Z" + } } }, "56433df9b9399e37671c12717a7e397ab2aec3e086e226fcf8bb3a338e336f38": { @@ -12351,6 +12721,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.956Z" } + }, + "01b3c2c46b1f5b875a5d0c20393042830caf8d92a7c7820943fa80463f760cdd": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.018Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.054Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.062Z" + } } }, "7b92c9515ab243345c2edd443a9f36e432abeb01df31d0d197db37f7733b65f1": { @@ -12364,6 +12745,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.954Z" } + }, + "d2a0556b2ff529a2a6bc6f0f1474e276bb5cf693229f1efb4d5f047dca2bba21": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.987Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.021Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.032Z" + } } }, "8150184b8463d89e5a92277a564104c399220d435ffb6ec7e6d2560672bb49d6": { @@ -12377,6 +12769,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.953Z" } + }, + "6156562270e5d113b2b835c17a4908c76e95978956ef4a57eaa61e1eed78520e": { + "ru": { + "updatedAt": "2025-11-29T02:44:55.983Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.023Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.025Z" + } } }, "8af19e1098601767cbf89d205cfc0d3cd2c79ba5ae84fa11d9cea6cc91850951": { @@ -12390,18 +12793,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.958Z" } + }, + "53c40fa63ea8e3ce50b35d7c9ab69f8ef252980df0deba29ae32d97edd799b2e": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.018Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.019Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.066Z" + } } }, "8fad6511e155deebc0c7b7055ddf993b7213668bd651d77b46f4fef11c363990": { "00a2be5a931770b44b5dabd0013f35d169228fbee45d460fc63c58245bf78264": { "jp": { - "updatedAt": "2025-11-26T07:24:18.952Z" + "updatedAt": "2025-11-29T02:44:56.042Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.040Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.037Z" } } }, @@ -12416,18 +12830,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.952Z" } + }, + "3076a78d7676e145757a5c63a2dae4d7c0c748942ad74b8e147613b5ae9c6a2f": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.985Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.027Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.029Z" + } } }, "9fd477532adc3dadf2dfed8071d354140eb7b667bd012aceca5476a9b5aeb7f1": { "cc0409c62d9e4b650b3ab8a4a2c2ea56b508c8a34ed0235cccc67f60cb557c17": { "jp": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.038Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.952Z" + "updatedAt": "2025-11-29T02:44:56.041Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.952Z" + "updatedAt": "2025-11-29T02:44:56.043Z" } } }, @@ -12442,31 +12867,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.953Z" } + }, + "c338f65f8d100cd47e3a31d6f9e78ba015c1346b791cfa3ff6677795952a0807": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.984Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.023Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.034Z" + } } }, "b24da7e78415a317d4fd792bce74b8acf47ca7b376eb80c5d2a81e9b874b5ec9": { "1b40db05914f87442600e04da552a114b9d6566703fff238531bf2dce4b3fb81": { "jp": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.071Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.072Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.070Z" } } }, "bd066e14efb9c286ea6f6324b04ea5e37363afb94dde1cda3efc2008e77fe6c2": { "ac1b069ca0882ed4666acf6095038e0b7cb288b8596cbf3b1ce1e54a9df05e43": { "jp": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.039Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.039Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.954Z" + "updatedAt": "2025-11-29T02:44:56.044Z" } } }, @@ -12481,31 +12917,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.952Z" } + }, + "c648cd50d62fa017bf9f75cb6b83f2c181571125b4792b76c86198da57d6b234": { + "jp": { + "updatedAt": "2025-11-29T02:44:55.986Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.025Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.026Z" + } } }, "ccb6f7b23e140ff82e19fc0391ef805c0f15507170cf5f60a78b0ea7f7bcf295": { "7b7eb66a4c1f465cbb23aa2d3f377abddba9aaa6d13866786810216306d2eb6e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.040Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.950Z" + "updatedAt": "2025-11-29T02:44:56.035Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.951Z" + "updatedAt": "2025-11-29T02:44:56.038Z" } } }, "d79bc535529875a738bd248165a718dae8d93446b748ae71439f9b822c83972c": { "1a78ff0ba0c6860dc7ce6357e1df29d3b791afd1f3ea81e2713f99d9dd8d0199": { "jp": { - "updatedAt": "2025-11-26T07:24:18.940Z" + "updatedAt": "2025-11-29T02:44:56.034Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.952Z" + "updatedAt": "2025-11-29T02:44:56.042Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.950Z" + "updatedAt": "2025-11-29T02:44:56.035Z" } } }, @@ -12520,6 +12967,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.957Z" } + }, + "703c345dcb09b3d9c115ac6d960bc44df5ebbd21bde9ddaeb3fae08a63b1749a": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.053Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.053Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.054Z" + } } }, "f181f03d87970ee159e60beef4cf41dfdb497fd8d950cab4164f13908b4a893c": { @@ -12533,6 +12991,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.958Z" } + }, + "01426a0b27a8046b6721227a23a347197804e15d2e71c528d46080c264354921": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.050Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.064Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.064Z" + } } }, "0d57e95520df30873578c0c59ade72141faf51c3e622951bb9026968b4b2a96f": { @@ -12546,18 +13015,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.963Z" } + }, + "5f3840a29e61dbec4841a38be6722a8b6d3bc7281cc7d9139a2e112a47d2f251": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.049Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.052Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.056Z" + } } }, "0f2ea76e0db5a6d5b78533ea69f6bf742d59e3c92cd69159341e1c7049a2aa97": { "9da14b2a7b04a5c4ff51174e32fb113e58f6e2c9b60265a9616f729614a2c9ba": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.102Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.097Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.087Z" } } }, @@ -12572,6 +13052,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.971Z" } + }, + "b92611dd6a3b9c353cc2b1383d42601917df331aae94df51cb078400430f456b": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.080Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.081Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.114Z" + } } }, "11f2e3a49b018a860171016a699fa740752c02bc0aa8f5f79a0c57498338ec5e": { @@ -12585,6 +13076,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.966Z" } + }, + "e5c755dea01abaf11bf73c6cfd13729ae4bfa33d43a0a52ea3e0173460d3b39d": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.049Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.050Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.060Z" + } } }, "259e682225d9b71ca3ea983216e57cd82c14b1caf25f00ea510ceadd3a70a0a7": { @@ -12598,31 +13100,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.959Z" } + }, + "e91d3b791b454b1a0ef05b57d40abdbf146ab1474ff1aeb70caabf9e4d32b816": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.051Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.058Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.058Z" + } } }, "3b9d54215d217a013fc4c62df11f26627fb8449a0489b74cc0f54e6b67f41ecc": { "f789cb25007915b6d83be12f4ecf35805e8a487063a7a59b47c497602ae41559": { "jp": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.093Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.960Z" + "updatedAt": "2025-11-29T02:44:56.044Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.069Z" } } }, "45c1b7f8bb110c2b37f34cc31252826058699640eef30ff8486c08761af44c43": { "605cfdad7a54e1e2f7b6a9998f6bfa8f8ff7b6a25aaa39281d58591fed0758e5": { "jp": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.068Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.071Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.070Z" } } }, @@ -12637,18 +13150,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.960Z" } + }, + "2885a781a007a7ada8f0db66ad248161e6af984d5a6d09191834d6a2543db968": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.063Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.065Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.067Z" + } } }, "57f74a21cf2fbbfbe54dc4c14d4c397429d32d51ea09651cbcba81a78f831e03": { "9aff12963c1e1db4b1b461b751a4d72394a3a26138c1713efd31eb628aa3b7c1": { "jp": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.099Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.965Z" + "updatedAt": "2025-11-29T02:44:56.107Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.965Z" + "updatedAt": "2025-11-29T02:44:56.106Z" } } }, @@ -12663,31 +13187,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.957Z" } + }, + "4576bd6efc4271a9bbfd7ff64a67e6a5cd8120b9608e1f0f96745079351d1a69": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.020Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.052Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.062Z" + } } }, "5e82ab99152b96f656e8dbc01527a2124dec1e8c721a629d4ba5aeccc219db56": { "4fe49458ceaccad1ac8e3af48d763a09070b1428ec46ac6e0a3b4c19aa2aff54": { "jp": { - "updatedAt": "2025-11-26T07:24:18.956Z" + "updatedAt": "2025-11-29T02:44:56.072Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.957Z" + "updatedAt": "2025-11-29T02:44:56.073Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.958Z" + "updatedAt": "2025-11-29T02:44:56.075Z" } } }, "61901cc301281214293209e58b53b0298e1dcffad02805348907ec14f5a36253": { "9b549c4be17898687f84e0ef17ef02ef8a374450b44096f17620746288db980c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.091Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.960Z" + "updatedAt": "2025-11-29T02:44:56.045Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.089Z" } } }, @@ -12702,18 +13237,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.960Z" } + }, + "ecb585ec1fef87ae16a59e4392cad2214397ab76b4c1f98f967c69dae8f1c139": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.055Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.066Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.067Z" + } } }, "8232385318fcb8ae5ab151696b217b22f9436e7402f061c4116986347a039995": { "d6b3588b7d8f126d5702902b6c9d58f3929c5d5c37ec39e19523d2d8bfcab2e9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.094Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.046Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.095Z" } } }, @@ -12728,6 +13274,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.957Z" } + }, + "9f3903e67cba6261cb9be580a31bdd170498cc260e9c5d03c285d123770043ec": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.017Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.018Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.020Z" + } } }, "a4b6a047b28cc22275775b0dd79539a2be86c95aa7ced10a1b187f12caf79320": { @@ -12741,6 +13298,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.956Z" } + }, + "af460729a46f510430df896945c2d1083b7a1cba8c13e697cbe6e79e39464dfe": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.051Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.057Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.060Z" + } } }, "bab8d56663945997ddb532be83b22cfbee4c23f88c78951638579d4d2d0d0dc1": { @@ -12754,18 +13322,29 @@ "jp": { "updatedAt": "2025-11-26T07:24:18.966Z" } + }, + "8e64108c88e2501340620384ddf14dd58507b8e1c81219be6f6e610fe3d9c772": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.084Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.085Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.086Z" + } } }, "bb10891887cb78110e7cb4ceb74ff22432d01fac9a3bff7cdeeb1886f79b1a65": { "caa3bae4c975b756d6c9bef7d4ca4f1118fd3ff3418d4538a30aa4c9e33515f9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.096Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.104Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.094Z" } } }, @@ -12783,13 +13362,13 @@ }, "f874e3ae6b9b2413ff9c4415bbd53d217ecc53aa9c8754f7d8b43a840a56a1dd": { "zh": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.014Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.015Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.955Z" + "updatedAt": "2025-11-29T02:44:56.015Z" } } }, @@ -12804,18 +13383,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.959Z" } + }, + "1487bdd2fa2f3944d2af9ce3c81c0bc560bc49e8a960f88b3b1bd574854de890": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.057Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.059Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.061Z" + } } }, "e39ace6f98adf22617bf02b8e1b5e543cc789b8aca34a357f850131c862245ee": { "18eb1c50ac74effbf464a4c046b94e4cb6fa9eb96d70864437ccfb525503aa01": { "jp": { - "updatedAt": "2025-11-26T07:24:18.958Z" + "updatedAt": "2025-11-29T02:44:56.074Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.957Z" + "updatedAt": "2025-11-29T02:44:56.073Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.957Z" + "updatedAt": "2025-11-29T02:44:56.074Z" } } }, @@ -12830,57 +13420,68 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.969Z" } + }, + "ece9e5e8b9ede092044e89a8f22b5f4f8ad893ecb0b80c8ee957c1d036b8e7eb": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.078Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.079Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.118Z" + } } }, "188f9a9bc3bec2ce321905c8a56a28198b42bc1f90b417b6ac00a4d9cf3c147b": { "8e6933142a9b80421dd489117c3233c45a2645cae67fe6bbf99c75fdf827c9ba": { "jp": { - "updatedAt": "2025-11-26T07:24:18.970Z" + "updatedAt": "2025-11-29T02:44:56.138Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.969Z" + "updatedAt": "2025-11-29T02:44:56.136Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.969Z" + "updatedAt": "2025-11-29T02:44:56.135Z" } } }, "1c00ec1111d4c97040f8a6b4705c820bc0afe08ce75657d4021750534563cc33": { "b2e299e5c648bc6c75f661d7ddb0d415bf3f4d2d15b1b81f676f8d781e4ab3d6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.129Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.098Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.101Z" } } }, "251dd0b5213f0a3e9974d1b830a96625f3edc7164c0ea7373201dbcf17869d8f": { "420a595531db98470670f1f677ed2a055a31cba9f965d100ebd8b1fd45bd0c88": { "jp": { - "updatedAt": "2025-11-26T07:24:18.969Z" + "updatedAt": "2025-11-29T02:44:56.136Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.129Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.132Z" } } }, "2fe98a07a0771f0c918a105339c7465f1d1800b749a6786ae052b4f5792f8146": { "bc9d4d641f5b9a05f88360a2ee33515689607102fb6c336b63a7598960ba63de": { "jp": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.128Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.969Z" + "updatedAt": "2025-11-29T02:44:56.135Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.127Z" } } }, @@ -12895,18 +13496,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.970Z" } + }, + "75703e9986b34937934165925fc58e72a3afca84531a9442ab6247ddcf89893e": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.117Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.120Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.123Z" + } } }, "513fe6bad8509823ffdccf71f911e6632a1d6c62bc3828d6880a93c15b106872": { "8b0b91827d9a7c004ba4a826838ebb29f76a0224d429a5d945acb7d900b732fd": { "jp": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.133Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.969Z" + "updatedAt": "2025-11-29T02:44:56.134Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.131Z" } } }, @@ -12921,6 +13533,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.970Z" } + }, + "30c6212c47a3a967e11aed7b52fa64caa3358560d58e4a0931019d75506f6232": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.079Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.082Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.122Z" + } } }, "67b2cf74cdaca50f8911af9d708d0de9b1f69f0efeab9993911fd47c8fe2f59a": { @@ -12934,18 +13557,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.955Z" } + }, + "47b31a12063a879f2beddb3373c4c72ffa7b8dc29a27e09156d7cf03a46cf52b": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.047Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.057Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.085Z" + } } }, "721c2734aaae37ab2cfa24011429e694a791b3fb975c20c543e63c974c336cde": { "9ecec8ec535a5264bf7ad03315791abb102815a602f895880c47fb817859cf24": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.101Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.047Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.092Z" } } }, @@ -12960,31 +13594,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.968Z" } + }, + "28b09452901cdc50d62751f8c7e48dda24bcdeceee8080b1e1efa2058fe428d1": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.120Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.122Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.123Z" + } } }, "8315916bdb3d69fc26c0b36f0b4378146ed63f736e03228e62d22efe01d9dfd4": { "5856087df98f6740b4472f367157e174efdc961ef37e3c1247d0ced2db5782d4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.089Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.963Z" + "updatedAt": "2025-11-29T02:44:56.098Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.961Z" + "updatedAt": "2025-11-29T02:44:56.088Z" } } }, "989eb966fc80a9e76f90dfcbc66e0dea7d1236c5a18dcfc3951a22c271c46183": { "501b56f9eae0cac02eb27cad28e73a3ea80b0a3e66d207d53190032406e903ec": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.103Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.960Z" + "updatedAt": "2025-11-29T02:44:56.046Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.102Z" } } }, @@ -12999,6 +13644,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.955Z" } + }, + "a85beb07457a8cd583ab4626e4a1665d17654cc132cfa7622752e519095ab48d": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.048Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.056Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.086Z" + } } }, "c3e128b68f1271e67f658e6a27e710c60881f8641ac2288d555daa3208c005f9": { @@ -13012,6 +13668,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.961Z" } + }, + "4626508269ebe6c1920ad10ad096b8dba568bff0279bfb0356204ebd6e971f08": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.048Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.059Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.087Z" + } } }, "c484fc5a7f3148583c4468ad2af97f94fd9cc073f7098786a953f31855eb484e": { @@ -13025,6 +13692,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.965Z" } + }, + "8e44081ae47c2cc39c56f929606d16db2e836b56a354be7dc6b701f4b95b4017": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.061Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.085Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.086Z" + } } }, "cb12578467473a3c801b153c6cf4d13a10cf518318fd5f17155acd1793145e1b": { @@ -13038,109 +13716,120 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.962Z" } + }, + "0db5aa15e3dbe18b1c681a26cee8856fc9b32345cfacb7a3bf81dc6e5ea5df3b": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.076Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.083Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.124Z" + } } }, "d7f86ec094d4fd68c7ec3902e09e9c8d6f32e759b1104bbeace470bd65c6ae68": { "aa75faa94f785331aff5bdbe2cbf5c4d6e4d398591d7ba48c786aa44ef7c17d8": { "jp": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.131Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.962Z" + "updatedAt": "2025-11-29T02:44:56.124Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.966Z" + "updatedAt": "2025-11-29T02:44:56.125Z" } } }, "dae06bb227a02c2e0c6a941ce0fc26005199e52c806d58d5e67386d3ec46f9d2": { "7b4e58d24764fbe8ed14bec5a6c802f2f143b902c16c654c45567175ea3ba639": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.100Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.965Z" + "updatedAt": "2025-11-29T02:44:56.106Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.965Z" + "updatedAt": "2025-11-29T02:44:56.105Z" } } }, "dbffe2a957cf5e50f0d77de216e876face0751f13e47da2a20400d54d5665054": { "de205edb219286909fddbd177c0ceefb00f1d4bfa1753f3d37b2539c40ccb3b4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.970Z" + "updatedAt": "2025-11-29T02:44:56.137Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.970Z" + "updatedAt": "2025-11-29T02:44:56.138Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.970Z" + "updatedAt": "2025-11-29T02:44:56.137Z" } } }, "e05629c59a8527d19506d4c60937f73b19f3d5ee1a52750f68b76b2d39b9d9ea": { "746136ea09bf1fea642a7fffc300c1227b17aefa177ec7ad998a0d64c56bbef6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.104Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.099Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.964Z" + "updatedAt": "2025-11-29T02:44:56.103Z" } } }, "125f424723e0504386a4a184da1e7119c6a2785f018e32a19cce5e8d2b7e5836": { "b707bc414a14120fcb5707df2de39c191647cd3b486308a8a5dafb116a49cb6c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.193Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.194Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.184Z" } } }, "19dc76f171fdf3b0cc1a3933538a1ce4395d12a9b9640597e4903ce3f6b18874": { "de4790564f72c39fe581e10e8ac3237721217d6c3c4ea4ad3cd07779bcc8dcf9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.154Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.156Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.160Z" } } }, "1ce6daa0ad295dac3a93f320fa28494beb73c39ee95608595b498a15a3e40ffa": { "85d971b7567c96e52bcd05d9d21b9c8edef12dd133c8c50e8b309d2d5aa75dc9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.196Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.198Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.195Z" } } }, "232c5ecb0f7a4603625517e022985cf3f01e1ead564c3eb970640640aaae8e12": { "3cf3a4419ef85aa0f20d163b55039c8180a0e1cb6acaf80999e00570756a5e6b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.162Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.153Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.163Z" } } }, @@ -13155,18 +13844,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.974Z" } + }, + "86f5cb11e91525fe84a6d7ac02cc19c807d8d6cce44b69717c9bbcefc375cc31": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.110Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.117Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.121Z" + } } }, "70cf97c8fc949e8db59f1ad657a9a53e576e424eaa88498f6a60d5b2e6729885": { "338d9d04b8e82dfebeacc09a54a398e5b4290b074e597a101394bc9922a1ee1c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.130Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.126Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.126Z" } } }, @@ -13181,18 +13881,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.968Z" } + }, + "b37dc6bf582ff09f935ab13559b16785003bbc859edbc25cb5250cf6ed36730e": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.077Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.080Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.123Z" + } } }, "9453ae6e6128bf52ca32a6699d5950c384605875acc28c886ac70d540ffb5753": { "ad123d6a5a72a60f6d9c309147475eb38bbb468cf9e3ff3d8588b1cabffc4b51": { "jp": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.171Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.160Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.158Z" } } }, @@ -13207,6 +13918,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.970Z" } + }, + "8da324568d7b8b5274356df80d91630b941a46f186301011fca9d984a25b20f1": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.080Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.118Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.119Z" + } } }, "9b57ca46e862eddb44a226a1ea028a1678344782bb5bedd683e47de11237eb37": { @@ -13220,6 +13942,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.974Z" } + }, + "68a496a4ec4c57830ab397ac2a3d04f92f17c400130928c2a18b7b319b353710": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.112Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.113Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.114Z" + } } }, "a725d7aefcb81ca44df79432f1da90c48ccc1821c943c4aea64ec662f97fc340": { @@ -13233,6 +13966,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.969Z" } + }, + "1828935083cf996cff81aea1667dc9a5bb681eabb1001a486dc7de9c3d17be77": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.081Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.084Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.116Z" + } } }, "aff518be70e64a7690e4ccddb5d480073f10c95e3ea3c17ad5f290330ba897bf": { @@ -13246,6 +13990,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.976Z" } + }, + "afced390065bbace06e3ad3e7d262d22da585d3d52fc35e6d2659ce9d6f1de55": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.110Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.111Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.115Z" + } } }, "b02ce70d6dcff3632894b67e171d3cc1146833fe54d4b06011bbaa8c85a0884d": { @@ -13259,18 +14014,29 @@ "ru": { "updatedAt": "2025-11-26T07:24:18.977Z" } + }, + "16d290c771287aeaa511447cff48ba3f26790d6964ff9ef4e8d1df0086f94e4c": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.109Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.113Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.144Z" + } } }, "b0eb0aa22feb0a0e87aa525beba888ab6c811439fb42a8629b3439c190197487": { "f0d582626df8dfbb7340a088943ebaa56822080b63fa866b42e18086e898b317": { "jp": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.153Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.157Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.163Z" } } }, @@ -13285,6 +14051,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.974Z" } + }, + "9643b5165e89def525dc375e71c872c60a8f7dd711c239056b81159bc679dcbe": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.116Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.119Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.145Z" + } } }, "d865d8906bab480f2412b8134877a2a96913a3533480602839cb1425678255d8": { @@ -13298,70 +14075,81 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.968Z" } + }, + "9d65695564bebf9c14e110dfe0a95ba918be4e978cdc05a279d1f5d41bd4ee32": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.076Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.077Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.078Z" + } } }, "db1f6b413c1b5c95a7fe86857804d32fa0bf64bd126d0d1bb0a19d36642d1ff9": { "2a09e7a09ae046fb1bc7a86b262a2891822048befffff23b62cc68c9e7e58324": { "jp": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.134Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.966Z" + "updatedAt": "2025-11-29T02:44:56.125Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.128Z" } } }, "deaf9da7af41c9dbd196870e7d946c2d92a2b4098eacc1d9d67ca6e552d438a5": { "fdf52ca20d97fc34fd94ada024eedfd00d77d9abbb0aed5df8411acf741dbddf": { "jp": { - "updatedAt": "2025-11-26T07:24:18.971Z" + "updatedAt": "2025-11-29T02:44:56.139Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.971Z" + "updatedAt": "2025-11-29T02:44:56.140Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.971Z" + "updatedAt": "2025-11-29T02:44:56.139Z" } } }, "ed51dd17995f6639353bb7c4089fa97d4f8dc7203bca3e26312cb31005fd949d": { "a382bedb279fccc3ac9fd5b4fe0ce9a876319b2d0652651cf74622f32f475762": { "jp": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.132Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.967Z" + "updatedAt": "2025-11-29T02:44:56.130Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.968Z" + "updatedAt": "2025-11-29T02:44:56.133Z" } } }, "ef55ad557299e30ca7d8ccbe3f701f3efcfb9407e677358fda64040c88c2a0e3": { "b7534a46cfb2aba578904a3ead55b3a917dd6ea809c434df147c1f98e5defeeb": { "jp": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.165Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.166Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.157Z" } } }, "f4e514c65ad19dadd6e36981ced2004e96119143057123e6f8343003c976414b": { "f9be206d9401669361ef8b3907f74e41604e01c3da770a270a3b262d0cf9e0b7": { "jp": { - "updatedAt": "2025-11-26T07:24:18.966Z" + "updatedAt": "2025-11-29T02:44:56.108Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.155Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.155Z" } } }, @@ -13376,83 +14164,94 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.971Z" } + }, + "0325309625a326b35142132e72492d62272031ba36d1a42a0998e56b1719cc40": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.082Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.083Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.115Z" + } } }, "025fd49fff3f320d5bf6441808dc379cdaa73f78cddd66059a1f1d989a1102a9": { "5cb5606bdf1fcec7d40bb07c9211307f195d39d691aa2cabd78b397dd79771c5": { "jp": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.174Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.226Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.986Z" + "updatedAt": "2025-11-29T02:44:56.178Z" } } }, "1e4b57e276f3147467bca9c9b34ef7237444bbb31a33e9319c88df9db588b8ef": { "781ade8017e15eb182d04e5802e03ea4655dd91aa963a8d3d6d5e111348f2ef9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.205Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.199Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.194Z" } } }, "243d4d43037034f08f1c4e2b8b9dad2508192f28c9087e19fdb7e02cb828ad52": { "8945c696900efad4645c2f95b6f862201f4275bbed3998caa867b1ac37deb350": { "jp": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.232Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.225Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.222Z" } } }, "2d5ce469cb4fcd9ac57756723325805176514ce512b8039ab05e3fde56bb12a1": { "37840663d4e6d0f5bd1b9b294c2b0feff352bd6bdd003b973cd9e9e03ef04b2a": { "jp": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.229Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.230Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.230Z" } } }, "344aa60f54b872aa215951fce76265aad2f3f1d6ff8bacd50188b941ce5098c8": { "7a8f03b82b278bf1a01cbbd7ff1923941fcfc7239248c640ae1b2eec075f2bd0": { "jp": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.172Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.169Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.171Z" } } }, "53d65ec30475ca0007e7da32916549bd02696879f561f268e8e3a58c0dfe9de5": { "e1d20246377ea7703705aeea779bd04141833d80b87084862959aeb3e9a08c2e": { "jp": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.159Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.164Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.156Z" } } }, @@ -13467,135 +14266,146 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.975Z" } + }, + "aea269c29122c965661a61ed9e0de26050201cfa241ccd1b34c694f29cebaf67": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.111Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.112Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.121Z" + } } }, "5c4dcedff3da1da80fb63b9461c1c89223beee53c37a3b5a538edc528453f0b2": { "620bb0c22df1a23b2a8df3eb395373d44296904b0332797c29514f90a31606b2": { "jp": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.160Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.158Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.167Z" } } }, "719a6d655a54f957cec2c65e95d6651040b93a639ad6aa44861b85ae09c1c5c5": { "fafe4a083f40e8f75644ffb779bcedb7065ad373f06a042ecf2238313aeef393": { "jp": { - "updatedAt": "2025-11-26T07:24:18.966Z" + "updatedAt": "2025-11-29T02:44:56.151Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.165Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.168Z" } } }, "82b281d3017bb8cc4db38036df8fbbba3430846e468a784c1b2e6d4d8e43b6d7": { "617961c999f1bf6eb48c03b5f56f99b3a7309dba7bcdb74914b6a76f36a56413": { "jp": { - "updatedAt": "2025-11-26T07:24:18.966Z" + "updatedAt": "2025-11-29T02:44:56.109Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.173Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.173Z" } } }, "8cbea57ac40a6d6358183da1d28c1a09304c1b4a5edf96e2c4a808dc6773ba41": { "39a62a98184d3c0536249ba36e562c954047436e58e929927516fea5318e895b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.170Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.166Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.975Z" + "updatedAt": "2025-11-29T02:44:56.167Z" } } }, "940796a1aae864d0eda15bb34a302626f3ad6a2c1d3af60ba921316d95e81a13": { "301a0a16ec26f11dd9fb52328307087f8c2528fea166cdea553309d6e58106d4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.162Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.161Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.152Z" } } }, "ab91d27df4d8b8148381ccfd51e2bc9b99a1625ef08e73f1d9a0eb197e5397a2": { "a1465aea8fd40bd2a71567dcd05c6ce53e13c60e2ac21919e271ebe1b6782f74": { "jp": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.206Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.191Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.168Z" } } }, "b7c59a245d47fd54f7c7477cbd498ba2937399586e98674be51c6a7c40b2ae70": { "410fd44fe625de2b185ba9098597ace5e062b1884403c90912660d14d188d9bc": { "jp": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.170Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.976Z" + "updatedAt": "2025-11-29T02:44:56.169Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.172Z" } } }, "d03338e91e1f725469cbc573d2b5a49c055fe39e67ab09e92b408e3e6dce3361": { "fee22f53b36f6d80c05058f7c0b07e16a2dbb531dbf640d90efae0a82972bd4c": { "ru": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.204Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.205Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.201Z" } } }, "e6adf4028e4ac7e7c24f84c4aa34ae7d43b6f397a67523ba3f4d24de317db247": { "93b602e423297a4fe948c7f19b73130ce3fb146f09d980aa1450c55a9055f392": { "jp": { - "updatedAt": "2025-11-26T07:24:18.973Z" + "updatedAt": "2025-11-29T02:44:56.159Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.153Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.974Z" + "updatedAt": "2025-11-29T02:44:56.164Z" } } }, "07567d62aae7f94a29e9f4d850ede3f6eec697596681ec8f0be305090388b473": { "781c617b76b44e877e7e119770ca6ecc45863cb3bae1a444fe8807d6ebada97d": { "jp": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.233Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.993Z" + "updatedAt": "2025-11-29T02:44:56.220Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.235Z" } } }, @@ -13610,6 +14420,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.116Z" } + }, + "3a8f100e338d722f8c4dbb2e623e5c4dc5a4d6f3bb6f2d5ba48737830cec8fbf": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.262Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.270Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.271Z" + } } }, "153ff0c08aecf20c420ae5dfa80993225532cf87b7d9c41e419a23934521c9a0": { @@ -13623,31 +14444,42 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.114Z" } + }, + "9d361fb3775a562cb507335e776089308bf758ae2747ae0957bd649d98faedc0": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.255Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.267Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.272Z" + } } }, "24d0c9c911ed73221e135198269c3368d046b7994b57b0fb624351b888e71a8d": { "547964d07a357f1d9316aadc7016d3943cece91207d0037cea7d08bb8914f5fd": { "jp": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.227Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.226Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.176Z" } } }, "32982205f1155c2c2e05fe89e04c9cd20828fb0a653c7c72c7da8d61c3253607": { "641d2a22f3cbbdbb5877f4694e0f7a70c2d4d0ea47aafe7ac478509d2f4bda90": { "jp": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.241Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.110Z" + "updatedAt": "2025-11-29T02:44:56.242Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.110Z" + "updatedAt": "2025-11-29T02:44:56.242Z" } } }, @@ -13662,83 +14494,94 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.981Z" } + }, + "86296003488064b48670c7fa1dea340b94da850eefa6ecaf62711f1d83875b93": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.143Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.145Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.146Z" + } } }, "38b350a818493921c30933efc9a00f13c8de2b1d444f825141d01c27a7c0dd78": { "5c8a7b7c41cedb9f12aa1dfb4a692603fdc40391fd020d73e7415f0890b583d6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.234Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.988Z" + "updatedAt": "2025-11-29T02:44:56.185Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.223Z" } } }, "769f4a7a3d111208fa74381508655c4dc5d7dcae5fe2808879e68d3cdc7b3382": { "489e0fb1db1004ec357920c6836eb4613ef37b11126cdd9c08bcfd3ba4aff449": { "jp": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.200Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.202Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.141Z" } } }, "79e713eaf2edf1bc512ae5d02a7d5d250a9659ca697b83603287e03063cf76ed": { "4ae0bd2c9234eb6b17182e97f10042bb3a03df6b39a2c2156858ba7f8c5537c8": { "jp": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.198Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.201Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.196Z" } } }, "85ba3aaa892ebfeca0dd4e9f91647080ae86a451c4f6a00a9725e2f2d8687ecd": { "b2beddd5e719b038a7b64dcbb0affae7ddf832501e2aa7fafd227bbe1cb45855": { "jp": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.208Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.210Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.209Z" } } }, "8f1cbe44d3d43c4cea34fea884586e29908abcb748f98fa025ccc41b62e45d3e": { "8e89cf7d6f4105f746591f40378eb84bf4bf9932ed4187023e334efc47a4b281": { "jp": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.207Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.195Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.203Z" } } }, "a094ce3a28e694708179862da79fbac7d2795b1716246328a6d1d45989e4d89f": { "01511979759628779536c4426b3446323cd0ba908ba9e69ed46eef6c4e519583": { "jp": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.233Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.231Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.993Z" + "updatedAt": "2025-11-29T02:44:56.221Z" } } }, @@ -13753,109 +14596,120 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.982Z" } + }, + "52272796a3ff10b33a617542859f14d9522e98d92a2f558892a1b3822e8ba86e": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.142Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.143Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.144Z" + } } }, "b28fb4d49a614d643a46b4d31f46daf5e9fe6cda08176cd2f5e078a055407bab": { "4108560a1744ad0710588b9cd75e007435917814d8b73b2316426c9d931d44c6": { "jp": { - "updatedAt": "2025-11-26T07:24:18.988Z" + "updatedAt": "2025-11-29T02:44:56.184Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.988Z" + "updatedAt": "2025-11-29T02:44:56.189Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.988Z" + "updatedAt": "2025-11-29T02:44:56.189Z" } } }, "b626ba6a5d5d3ea8fc4f8b1fbab4067c3c422e1f441d82656ea4e1576b013f77": { "d39e1a92c96f946e67f7b31e6fa41e119a9a923698dbf319033ccb86b70446c3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.140Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.190Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.183Z" } } }, "bfdad58f0ce19b8378572771619d14adf32b34da41695f420ad03ed4496197bf": { "c5d8b4488de9c51f7fa4c711f9885ca220f45c37ba8c7062bb02813316daa7be": { "jp": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.192Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.192Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.185Z" } } }, "cb1ba7289dde002c321160e758dcebe6637312272f6a21430a36ca8d2bd0457e": { "6d2f41b7dfc6a91c7ad657ff5eb668944436fee3888a6396625bc67d1726719c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.237Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.231Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.993Z" + "updatedAt": "2025-11-29T02:44:56.221Z" } } }, "cdbd4e3a0fcbd1a9915e133e9c7749b9e313633614596b23aedac6d6da31105d": { "184622e2d0685a2859808cd7eb92c85650ed8abc39d7a38af056d81ff2c94654": { "jp": { - "updatedAt": "2025-11-26T07:24:18.972Z" + "updatedAt": "2025-11-29T02:44:56.141Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.197Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.979Z" + "updatedAt": "2025-11-29T02:44:56.191Z" } } }, "dedecc80a24539ab5ef48968c83b54eb08fdd06c15720daadff55822ec0b257c": { "5da52f81a0a0c35a9810a8ba27a1945c10ef4931f047eff638a1e08016f6bd12": { "jp": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.204Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.209Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.980Z" + "updatedAt": "2025-11-29T02:44:56.193Z" } } }, "e7ff4d7fd0bd848202048d33c9e285c0b7eaa06b86b242461c43fe7e001d1b39": { "574ff1d32ed4fa6964c51389dc9f9d35f7a76cff9623137d2922ce0856a65215": { "jp": { - "updatedAt": "2025-11-26T07:24:18.982Z" + "updatedAt": "2025-11-29T02:44:56.203Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.199Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.981Z" + "updatedAt": "2025-11-29T02:44:56.200Z" } } }, "e83fb55099e0c1e7efe462a3fc836fad5d3f3480534f4512599d1bb0307a952a": { "00125ab6f5435064f526a97e752f345080fe710b1445d06711d4011db26a78f3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.241Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.206Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.983Z" + "updatedAt": "2025-11-29T02:44:56.207Z" } } }, @@ -13870,6 +14724,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.119Z" } + }, + "86d2b49dce63d0030956d9394380f458d82580fccf11182038c47ae25941e202": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.248Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.291Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.293Z" + } } }, "027f426455e0e6842638722daa037b778ebc144d4ad338fe61f0710ec20e99b4": { @@ -13883,6 +14748,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.112Z" } + }, + "c1078383e79ac339256601f181a193ab1979b13400901d0b702e398f5163d3ca": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.257Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.257Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.265Z" + } } }, "0819d9360d80872f0e20752e84412951fa413fcd532b41e457c8b552f0613288": { @@ -13896,6 +14772,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.119Z" } + }, + "d83c42928fec524182b7980ee07946e52ffe8f524b8943b83d109bf0a5e6b9b4": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.289Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.298Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.298Z" + } } }, "24d804d8ac3ccf43c75fb8b0dabf735d89f27625497f0d7acaf06933ccacda4e": { @@ -13909,18 +14796,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.113Z" } + }, + "9ae107eff975761568ef37cb2d50a6120d87504b7eba954b3c75987d6acfcb56": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.219Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.253Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.268Z" + } } }, "2e14d7ea42f23a61da8855e77c500092cd204a036888c976b84a9a6bf71b8eaf": { "1e988897ad46c538e51b835cd9cd1cf89a4e7059611c53ec91e71868db50124f": { "jp": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.239Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.240Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.238Z" } } }, @@ -13935,57 +14833,68 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.116Z" } + }, + "7557e28163d56959272cc839ee4219d9744ff724198372cda0479d4869f1c55b": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.217Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.219Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.259Z" + } } }, "52ccd94aa1e934784ca02ff91c16f3972d775ebf46b09dc38022536e439186ff": { "c395705e691a1be5ddcfb6d1b262f9a0dfd9470788de834bcb6fbc5d0d8f0c8c": { "jp": { - "updatedAt": "2025-11-26T07:24:18.987Z" + "updatedAt": "2025-11-29T02:44:56.182Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.174Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.225Z" } } }, "5564c05bb16d853324ac6b1bc9b36248c158160a7e1fbac14ae9b86cf5514569": { "ea348755a3baf8bbcd34194da4c4529bb44c7a9a402faf39e87b9d75135b0b81": { "jp": { - "updatedAt": "2025-11-26T07:24:12.117Z" + "updatedAt": "2025-11-29T02:44:56.278Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.117Z" + "updatedAt": "2025-11-29T02:44:56.279Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.118Z" + "updatedAt": "2025-11-29T02:44:56.280Z" } } }, "592a7f7d3a8dbeda07da824c065c0da9b3e247906e6dbf77674f6a63df3136da": { "2293abaeae3fe16820f6c7c9a37b91841e60a17efff63af19cb7a8d4a0eb2456": { "jp": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.177Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.238Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.232Z" } } }, "59e3664663d669e021fbd29e32b23a365ecc37fceaccac1e3c9e74f070873d03": { "664e682e3d269a460d26982803f72d705695f346f7f43cd3b62de24703236061": { "jp": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.175Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.237Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.236Z" } } }, @@ -14000,70 +14909,81 @@ "ru": { "updatedAt": "2025-11-26T07:24:12.113Z" } + }, + "7312a2f89d65b933514be67491fbd4c987461524ae097172ff5410ff3b41a21e": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.261Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.262Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.263Z" + } } }, "650407ab32a2947c9874bd0fc813344a1675577ba430ba4ddefb9497ceec4df4": { "ad334487bb9276e08638e9be4af54b1205755e694d6c1911d00059d8415fae44": { "jp": { - "updatedAt": "2025-11-26T07:24:18.987Z" + "updatedAt": "2025-11-29T02:44:56.183Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.977Z" + "updatedAt": "2025-11-29T02:44:56.175Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.227Z" } } }, "77307f3a7d1b826bb6622b0f3ffa4c1f7706494839393590234d7206bbf2be8f": { "017f574127f909641a3e7c014420c6954edb618ef3d438854515fd0f5dd1e298": { "jp": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.228Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.993Z" + "updatedAt": "2025-11-29T02:44:56.220Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.986Z" + "updatedAt": "2025-11-29T02:44:56.181Z" } } }, "914120dcc6c64903cecac105d4df906767aa83b440456a677f5192431cc83d6e": { "4af035b51000a041cbfd0989fe3c52f7370aaeec63d4f8ae146a2776b899fae3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.176Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.987Z" + "updatedAt": "2025-11-29T02:44:56.182Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.224Z" } } }, "9c50ae2540822f01de38fd832846c44e0815140836bcf8df45e61a172e36831a": { "48e37702889833007771c8e75d0ebddc5a93b178a5f5ae6c2512d72beca89b15": { "jp": { - "updatedAt": "2025-11-26T07:24:12.116Z" + "updatedAt": "2025-11-29T02:44:56.275Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.115Z" + "updatedAt": "2025-11-29T02:44:56.274Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.116Z" + "updatedAt": "2025-11-29T02:44:56.275Z" } } }, "a1a93279f18aea8b2a8afde127dc919f6b9381d84fdb78e820af9fa87a4f85d7": { "8ef32573cad40bd5922dd07f6e65cb11c503497f1996866bd36c8bd70fdbb4a4": { "jp": { - "updatedAt": "2025-11-26T07:24:18.984Z" + "updatedAt": "2025-11-29T02:44:56.177Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.236Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.224Z" } } }, @@ -14078,6 +14998,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.117Z" } + }, + "ea316ef388947a1bd8b8ce3b019ab9377bd0b52bbf557f4ee29826ea0406c8d6": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.215Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.260Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.269Z" + } } }, "b1eb514e8efc1da765f03844ec981e8df30e9e90bffe8f559550b33fcb148386": { @@ -14091,18 +15022,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.114Z" } + }, + "fe5653fd0a01cc377763c0dd39db11ab651632c5116e8e68e5b26336f447b84b": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.216Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.271Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.272Z" + } } }, "c35229fb2bf6081a5aa25c5273a6bc76f7fb1f8586da22277f9b09cdfe9c161e": { "96b4bbf5cd710c7028d1dcff43630fc1346305b9fc31fd06b6feaa5771a11a01": { "jp": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.235Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.995Z" + "updatedAt": "2025-11-29T02:44:56.229Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.996Z" + "updatedAt": "2025-11-29T02:44:56.234Z" } } }, @@ -14117,44 +15059,55 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.114Z" } + }, + "624184c131264cdb4084e3b3202e40b83320cab7475a7b58e74d2b6244ec0c40": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.216Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.256Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.264Z" + } } }, "d88cb52cd1ee657459479ad84c5c952fbde653226d9799e31239473fa8b0fd23": { "fb9e79efbf3a2d62721e7f715f0699a0dc1f1dbc6e75db72c520ba3026346f5b": { "jp": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.240Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.110Z" + "updatedAt": "2025-11-29T02:44:56.244Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.110Z" + "updatedAt": "2025-11-29T02:44:56.243Z" } } }, "fbb5789352a952225705586e3f21b0e7e42cd17127fe8ed8e8ca218112140a27": { "19f784e7b489f48a3d495a2e1c1d68856626b21b4cedf271ef931452b7add1ce": { "jp": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.223Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.988Z" + "updatedAt": "2025-11-29T02:44:56.190Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.994Z" + "updatedAt": "2025-11-29T02:44:56.222Z" } } }, "123aeaa56592e54f31fc778623c345f09749d4e0e65e902af7d1a93337a425bf": { "f2e0676875f34dd5520562d2cd21b217af1b44b68311b6c948988adef7f432a4": { "jp": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.283Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.285Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.332Z" } } }, @@ -14169,18 +15122,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.774Z" } + }, + "0c123e0b345e289824f7799b173b1794fb0e64b2d9bca74c9615d96f2d87b4b3": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.290Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.294Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.302Z" + } } }, "1f24f51d58cccfdaab17312855078466a67ec6632bf8534638b69f8f5f3551c5": { "ac3de3782a6dcd627cb900e0e3c325463324737e43db6385a4a9edbf6ff7796b": { "jp": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.282Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.342Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.325Z" } } }, @@ -14195,6 +15159,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.984Z" } + }, + "56afa0eb5a088f7a9f1763feba7e0a86e58b5f20fd0b80d4eb17728b4ebe4e4f": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.263Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.266Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.270Z" + } } }, "3a83cb18dec6067fc17dcd4bf9d92d724df7894996965a2aa6ddadaa218d8377": { @@ -14208,6 +15183,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.780Z" } + }, + "246e677fdb1037560ce0f99220806100065ce49a0a719ec84b0ef40a87caadcb": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.320Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.320Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.323Z" + } } }, "4db1e2e4946307003f6c8e7296d88d16ea1fa0a50642705d3f4a2f6130b44a03": { @@ -14221,57 +15207,68 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.111Z" } + }, + "c65bebaf1409b6811c25b61ee9e1a29774f1a9a74497375c4b00bb9357be3fa7": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.218Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.264Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.265Z" + } } }, "573d715ca8095f0e4ca44d1cba02fd75a74bbc9c173567252833684110e7eed3": { "87c69d03f3d553568b16a72f5fe7243c7fbedec0f55aa5c55695e0895009d96f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.112Z" + "updatedAt": "2025-11-29T02:44:56.213Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.111Z" + "updatedAt": "2025-11-29T02:44:56.210Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.113Z" + "updatedAt": "2025-11-29T02:44:56.214Z" } } }, "6127321ac3891bee9f802edc9f97eeefd28aa0d40a647d0fa4cda55abfce14ff": { "d3499050f8c6e7b0a1bd1cf5e8bb8e940304335d153d81d9717b6c21c16c2985": { "ru": { - "updatedAt": "2025-11-26T07:24:12.121Z" + "updatedAt": "2025-11-29T02:44:56.306Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.121Z" + "updatedAt": "2025-11-29T02:44:56.247Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.119Z" + "updatedAt": "2025-11-29T02:44:56.245Z" } } }, "650d9f2cc9a940fe5940498f6e144305c01bbf36d3ee2dc4bbd8968c9f8967c6": { "17de42c037b1a363aacffaae4c43b7e7c471839ed6cecff05326ffc1616e8599": { "jp": { - "updatedAt": "2025-11-26T07:24:12.111Z" + "updatedAt": "2025-11-29T02:44:56.212Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.112Z" + "updatedAt": "2025-11-29T02:44:56.213Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.111Z" + "updatedAt": "2025-11-29T02:44:56.211Z" } } }, "6813da4ad4c4af5afb1c7574805fe2dd8caa6c96f485a82e9c901ef475f08fee": { "b0517d0f55cd108acdbbe709883cd25fbda01a6703d9b51ff50bd2116dae6e4b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.116Z" + "updatedAt": "2025-11-29T02:44:56.276Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.116Z" + "updatedAt": "2025-11-29T02:44:56.276Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.116Z" + "updatedAt": "2025-11-29T02:44:56.277Z" } } }, @@ -14286,6 +15283,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.111Z" } + }, + "8f32e4aae111f92315bc93e0ccdf602c223cf64f5840140b6501f1f14e174cbb": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.218Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.258Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.261Z" + } } }, "6fb070f1b02c940c98234a8aaec25f6c6469691d330c72faa861b07763ae4725": { @@ -14299,6 +15307,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.773Z" } + }, + "355de205d2e66f18b00c07141db16e3eae08111fa3207ff29e5d7e2db19cc526": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.249Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.254Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.303Z" + } } }, "9d8c420729f6dd40353fd0b37376eb59e28f1b3a71685df761a9e2ad46f35ca4": { @@ -14312,18 +15331,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.774Z" } + }, + "74b3411891901f287f4f0295c14cd3b703d6197988dcd91f4e53985964af404b": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.289Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.295Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.302Z" + } } }, "9fe9b6ce42a6ad2189bab2836ba94c9f99886df803b81bdc3dec38815dad7c26": { "2a6580470ab1e345d52a27c96f69c6e94d335299083f18b83f4f16b1913c6ee0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.775Z" + "updatedAt": "2025-11-29T02:44:56.312Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.774Z" + "updatedAt": "2025-11-29T02:44:56.309Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.774Z" + "updatedAt": "2025-11-29T02:44:56.311Z" } } }, @@ -14338,6 +15368,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.112Z" } + }, + "5db4f94f0470b0bb3fb558e3b1e3766ee12f80ad603da74665b1a15b3b16c254": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.269Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.273Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.273Z" + } } }, "bca19f630581f8646ca04081842168a1d45e2ea5896cbdbab33c160594c627c3": { @@ -14351,6 +15392,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:18.984Z" } + }, + "0e3c63c854cf8f55abf51be5d8395d72aed010f11ba09ea870f1dd42d4d16794": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.260Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.267Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.268Z" + } } }, "c7f475dc465af2e8a358403f37f7d9ab77fff25bfb2185434a3988227077f61c": { @@ -14375,6 +15427,17 @@ "jp": { "updatedAt": "2025-11-26T07:24:12.121Z" } + }, + "957fef1200e01d1d2a8bc6b685146a1418c4d936418ddfe9ecb18479516293d4": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.291Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.292Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.303Z" + } } }, "cf53b09fb0c34e1e63e41a10d6bc7a6922adc30f419e11b91aa28c4b6550ff94": { @@ -14388,6 +15451,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.114Z" } + }, + "b2680ed1949ad3d0db6340dcfd927b99beee508808edbd641c4f0f3589bb32ec": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.217Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.255Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.259Z" + } } }, "d85a58d074e13f650fae5bc844462e82b569a15037cf4beb81c7fc31334227bd": { @@ -14401,18 +15475,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.115Z" } + }, + "b6d6c294393317eabba321f888c00ac842ed017623140b48620c5b77ecf9538f": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.215Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.254Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.305Z" + } } }, "e014a958a8137fc765da9797a531683aae1075024018fdd2793c345a9ea2837d": { "a3692c0caea63dccb572f30b9f84021d898cc0b99e942bba8475e5cddd746e9c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.112Z" + "updatedAt": "2025-11-29T02:44:56.212Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.113Z" + "updatedAt": "2025-11-29T02:44:56.214Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.114Z" + "updatedAt": "2025-11-29T02:44:56.274Z" } } }, @@ -14427,70 +15512,81 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.776Z" } + }, + "51115dd7a55d295b9a3eb7a76ba8a295d6fcab50c6dbe15e410efebd02969067": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.248Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.296Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.304Z" + } } }, "02fec6942d40034d750c654d9c675a575f12b3a87ec90a6e3786281d265a9b29": { "f8983bc303673b5b9632c8a2f95602dd3f90803ac3e493ee4ff7244ea4b98790": { "jp": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.338Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.339Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.335Z" } } }, "0393512198efa57d46b32a113a35375ccd26518fa34d3bbabef4214d4fb8b53a": { "8103e61160aa52995bd2806ebc1f5871330feb5a4b2c8de0e9221fa8a70d1ac3": { "jp": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.328Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.359Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.328Z" } } }, "0c5a65f577c71fbc834405efc189e3c50da0f84a64b7f1b1ba76d9fa8e7a3e9c": { "2d31634c588cb2805bebfc13a4cefde978ae8d078f32a88954c1ee076a081d1e": { "jp": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.342Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.329Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.341Z" } } }, "0c607ec56978fcf7cc9280f736b890c5fb289db3ddfbeb737f9ed97fa79e3f05": { "c01c4184a1e5108e2ecbbc865220b88f0c11caf78ba66781b92ad690c80fa8d3": { "zh": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.356Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.337Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.362Z" } } }, "16ea5fa75d5d08e032a72f3d2f70dfde100b84192a3a87d58596c7a636e73d4a": { "08b83c6534ed2ed43f2e271298926bbac6bd7c4e552372271ab8f870588ce545": { "jp": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.340Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.326Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.343Z" } } }, @@ -14505,18 +15601,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.119Z" } + }, + "bc07f19137cb6dabf76e068e43903d8f0c0d4a5fd3ef5e4c48ca713d51eae844": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.250Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.292Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.299Z" + } } }, "3e1a6a2d9604853fec0f6b9c21e1534bc36ba5880d4042f71f1d9a03ff9e0c74": { "50a43ff5465e5ed3b333a2938abb5b5a0fe5d616b29d9f1176535339c755b45f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.286Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.284Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.288Z" } } }, @@ -14531,6 +15638,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.773Z" } + }, + "955b894f3bdf01356a56ddde8e74375c28f36a9ddacf9f8cca1d929b82dc3c8a": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.252Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.253Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.290Z" + } } }, "5be58ce97a5c915ff2d4f6bb0a603580ec8a37cc97e4e9b54ce41df65adbfd1a": { @@ -14544,6 +15662,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.120Z" } + }, + "8c7e75be7714da47b7c687d7067a073e9ba05d82b6595b598a376227cab0ee4c": { + "jp": { + "updatedAt": "2025-11-29T02:44:56.249Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.292Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.305Z" + } } }, "5eb08e96fd1bc79722d094e6a779abcf8a842d610d831653012ca3687bc9f9d7": { @@ -14557,6 +15686,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.780Z" } + }, + "a7faed462f96219d02aa7307ac7bc7935bca6700485c60f90a7438b05da3f66e": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.293Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.295Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.323Z" + } } }, "86002a1712dd0604dc6572e5d62e8b5302340810c0d6fb2d6a5457e65c731ba1": { @@ -14570,18 +15710,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.777Z" } + }, + "ef2543418d200cf81a07878edce91cdbee829490d86129bd3c779f48958f1b32": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.297Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.322Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.324Z" + } } }, "927c8bd22c5edb0ea1387b718e08b478ba16818cd5723ac48756552d7439254c": { "11b3f30b750a5e0956357a5f57ba8e702f81cf26a32d8a0825cee915379b0137": { "jp": { - "updatedAt": "2025-11-26T07:24:02.773Z" + "updatedAt": "2025-11-29T02:44:56.308Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.774Z" + "updatedAt": "2025-11-29T02:44:56.310Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.774Z" + "updatedAt": "2025-11-29T02:44:56.309Z" } } }, @@ -14596,18 +15747,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.123Z" } + }, + "c6165f3ca7b3425b951b4a1571401725d547adf52dc6626c445215fb9218c8f1": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.251Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.294Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.300Z" + } } }, "98c18fb7bc391069017a8197ca9b4b5c5a8218b2cc79f1210a3aba08ce470c6c": { "81814115cad79ea901cacf1a4876697b9b219a7ce07476d4edac8f5cfb5017fe": { "jp": { - "updatedAt": "2025-11-26T07:24:02.776Z" + "updatedAt": "2025-11-29T02:44:56.316Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.776Z" + "updatedAt": "2025-11-29T02:44:56.314Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.120Z" + "updatedAt": "2025-11-29T02:44:56.246Z" } } }, @@ -14622,6 +15784,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.774Z" } + }, + "6b17506439ccc3610f6d989dbfd30e8ceef573be8f80fb3230ad3a6b4a276542": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.247Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.296Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.304Z" + } } }, "a5f04cc970babcbd17a73219fd4d3f1d299602d839f96c355b2d5ca53d5cee5b": { @@ -14635,6 +15808,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.773Z" } + }, + "e965cf783f03102b23d52203220a90ea4ad4eeda8ea356dc2888850e3a1ee83c": { + "ru": { + "updatedAt": "2025-11-29T02:44:56.251Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:44:56.301Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.301Z" + } } }, "a6c3bc31e00e18f2175e34ff2c0292484093e1b725e792b68d1075913aa4dcab": { @@ -14659,18 +15843,29 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.121Z" } + }, + "8b70f77a580ae511ac0bf35f88454f6df63aca38b1be27503cffe5bd9b3b0d0f": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.299Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.301Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.306Z" + } } }, "c43792a75d02793708f0f9c298dd1e81a2db715e26bb86c9a3a5e14f34e785c4": { "76526beb43a3126f9cd6e8837bdfd7a2b5b294aba899560796a163b8963fb64c": { "jp": { - "updatedAt": "2025-11-26T07:24:02.782Z" + "updatedAt": "2025-11-29T02:44:56.348Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.774Z" + "updatedAt": "2025-11-29T02:44:56.311Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.775Z" + "updatedAt": "2025-11-29T02:44:56.312Z" } } }, @@ -14685,6 +15880,17 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.124Z" } + }, + "9b5a33767927dbd5b8d2768daa909d9f65bb2a2f716af808a8f3eb55f623603a": { + "zh": { + "updatedAt": "2025-11-29T02:44:56.250Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:56.252Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:56.295Z" + } } }, "e1bc67bc420afe0f3e7e9ad857f155267b70dd99307c6a46ab1d985186a0ca42": { @@ -14701,78 +15907,78 @@ }, "520ab775b334277fd5691012ee012e3b785025e0c7722d8a9b7554a5f3a04f2d": { "jp": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.284Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.287Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.288Z" } } }, "ec512023c5c9e376d6b5a73b27c72543c329f1031a88df1142e318e852a6f5e1": { "e3ddcb697170bf6b9accf7bd9484665c071ebdf44c1609b8c1f4a6ae733dd1c5": { "jp": { - "updatedAt": "2025-11-26T07:24:02.782Z" + "updatedAt": "2025-11-29T02:44:56.349Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.782Z" + "updatedAt": "2025-11-29T02:44:56.348Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.346Z" } } }, "f411f73869f1597bddd3a69a70dcdf627b2f48802b68eb1297c49cf998a1d590": { "6c152f17b58caad6637a04e4d427aba059026b111c90e5aa764f040e05e669bb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.122Z" + "updatedAt": "2025-11-29T02:44:56.306Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.123Z" + "updatedAt": "2025-11-29T02:44:56.307Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.120Z" + "updatedAt": "2025-11-29T02:44:56.245Z" } } }, "0bba267be6ffcbb62a544c365f5d2cd85d6371c78dc289e5697b0225352a76ea": { "95f85b7c7a43494a5f08ae259de69c8952afb7851b1d9a887ad3107d5e6cbc01": { "jp": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.396Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.361Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.372Z" } } }, "12f796f4ae9f25130a8cfc11aff488171e7376f25404278d4e5c173c8bf9ed02": { "55069f671a99d799cfd16eda4312b66b5a321376cc69b52c58ba054f313fa404": { "jp": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.331Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.337Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.330Z" } } }, "16c87bceec945d0aeefa7e76d421913b507e3b04716834b3894e9fd3174d2613": { "b43921e7c1caab150d19b0823696bd909b5e9b9dd41fe7847acfc9dabaec0942": { "ru": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.413Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.412Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.411Z" } } }, @@ -14787,70 +15993,78 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.780Z" } + }, + "7ac8d25006b0218725310bb4f50d2afa2fa76b42500a9587fca779027db7c47f": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.376Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.385Z" + } } }, "2644c145de6d61cff7556d3efdff355e849b2f38b5c7912fbc2eb07360771f61": { "0e301628684a655bb2d5641c57775c3259b037ac338372d82808d6c91cacbd8c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.118Z" + "updatedAt": "2025-11-29T02:44:56.280Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.287Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.286Z" } } }, "2936efa9c627e5f00c7abc38b09aacea29a31481a83c21e719db82d0ba4eed0f": { "60b3bf0f8d4b00d715fe94daf3931f6351a0e38fe807946e5d1f9eee04425171": { "jp": { - "updatedAt": "2025-11-26T07:24:02.782Z" + "updatedAt": "2025-11-29T02:44:56.349Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.351Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.350Z" } } }, "337fa5ffda5b1ce15febb15e28d78f509b83dd0442c0eecb4e5fd5ad01cee570": { "8ad0cc19f45e168f3328286b8c922f25ddb3753ff16efc3a1795161778bbea66": { "jp": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.374Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.379Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.316Z" } } }, "3a39c3cb40c4a84e5848358c7bcda5a305e64fba4846580eecea963760143cbd": { "1a63ea8e13a6c3989444c8189eb5c95920d36ded548a2cbb106db39f91e17f56": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.367Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.369Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.326Z" } } }, "3b2a0db3103ecc795ff82061e46875995689dee845c28a19697c2e8b7d78fb8f": { "84bf17e2315c270d4f26795807428c5ef311a937dd6e53a4b6f3a8e26bf5e771": { "jp": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.336Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.332Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.333Z" } } }, @@ -14865,226 +16079,234 @@ "zh": { "updatedAt": "2025-11-26T07:24:02.782Z" } + }, + "ec159a684acf88c327d88432ca1c26e16cbeb3306673c56da4f7747eee906117": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.383Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.393Z" + } } }, "548882c1623ad246688470b47967ff13ad16868ecad4f09349b0182efc755985": { "76fc9813a272dfbb6dda3bb0096c7a2eeb3bf0a58d344e26c115c075a8cdf7d0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.377Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.375Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.373Z" } } }, "7d0d5bb4af482b39457517f498efb3ea46e33dd5a7cfdef6d18b2beb02e7dc2f": { "a7f9fe4cf3ef806d2de5eb8e1f83c674deb8224319845c32faa2c509b9f0d89a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.372Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.376Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.375Z" } } }, "819fce8b1343a94dee6cf3e428f8d46ff343c43b0d83b49efe18129ccf292430": { "af1d949b76a7c871e4cdce3092a3b2e2b1ea6afca4c4788054f8ff3eddde3ea5": { "jp": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.327Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.338Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.327Z" } } }, "8756460c34802f52ffc72c46fd775666b61d2134d4e3d1de0bf4111a5a049571": { "483cc85982240fd19d9aaf9161c58f6f4b1f2cdf226fb60169450e02caea8384": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.370Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.329Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.365Z" } } }, "99effff387a3391b66ab69348b19106aa7ae02149e5cdda15d9bd9397ddf4c41": { "635055619056b153a2e20b6a09345d76348336b24340ba32f815de9c85a7f2b0": { "jp": { - "updatedAt": "2025-11-26T07:24:02.777Z" + "updatedAt": "2025-11-29T02:44:56.282Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.346Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.782Z" + "updatedAt": "2025-11-29T02:44:56.347Z" } } }, "a0698fed4f549f79afea06d1dac666b108eb6f5dc2fd3c91aff7c13f9d635593": { "7bafd49c863eb3620b55c3516c62d1d11ad2c81b1333e7396261c18b3d55cf9f": { "jp": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.347Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.119Z" + "updatedAt": "2025-11-29T02:44:56.281Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.345Z" } } }, "ad1402ffed17fc7c6fda3f600f70cf8e3bbe5384d766081c16c2c90b4a775b7f": { "623f2f8c2f6006597fa106e18afad1304117a0a599684c3050b5f92f433dadf9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.369Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.363Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.317Z" } } }, "d6127c27c939a8143d6bd93d737c445238b16aea350cd52caa535082aaed407a": { "af21361ca18f3026c0fcb3b223ce74e7a213c2e9016d2f7596b5103f9f243027": { "jp": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.362Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.374Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.378Z" } } }, "d679b331b013310d0726e18cff38795d35a48a549ce862414366ed5d37b17a5a": { "6884d15ae61a9e31fa06e9f6cb793ec44513338525d28650cffaeecfdfd55f59": { "jp": { - "updatedAt": "2025-11-26T07:24:12.119Z" + "updatedAt": "2025-11-29T02:44:56.281Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.345Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.781Z" + "updatedAt": "2025-11-29T02:44:56.344Z" } } }, "f23d1fdbec8862f67bc4eb334787e78bade64fa097b14c338abf676e73a1ca62": { "0206d00d56a0c7be7a356c6499d1bc4c3b24602fe48380f49f1a8be277e30ae9": { "jp": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.341Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.331Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.343Z" } } }, "f5d22ca5e2a60035bc7b1c39046c553ef2238bbf8c611bd22963a3cf3fe67663": { "9a33263baf26f23ddc1d61444b9f0bc17fe15f0d44c6aa520661947f7bc28d34": { "jp": { - "updatedAt": "2025-11-26T07:24:02.780Z" + "updatedAt": "2025-11-29T02:44:56.340Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.334Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.779Z" + "updatedAt": "2025-11-29T02:44:56.333Z" } } }, "0d3a0a09b86406c2c362ede819ee030f9d92d058939579cd1229e361973022f8": { "9fc104791c743a764dffa282d540ca4365e02a6a6590d6c336de81ff7f63da24": { "jp": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.352Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.405Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.406Z" } } }, "14364235369dc388419efc9e290886ddaa202d5023e8adc55d75a61c89fc336a": { "328695ec26f7fc60b0c8aec17edefe2b5cd222a635c116a01ed4259436be44ae": { "jp": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.395Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.435Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.435Z" } } }, "14a65362c725c7a0fae1767f0bdaecab08516f4549961fb82c9b0d3889476e2e": { "4b5208315e755dbc3f295c8a58958e452a782c2f41e4965b7aaafc2ecdf93523": { "jp": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.410Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.410Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.412Z" } } }, "181aa5509e2dd7724e3095fd6c0f17cf6fedab2635b9af1d57fe9d1e2801ec31": { "bf2760368d2fc3a4c455358f8872f13eb6f6e7b8ccd6d529c68dfa016882d216": { "jp": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.434Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.402Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.368Z" } } }, "5181bec59897499f787e1b509cc19c69de2efe0e1437cc2001f2c7dbe8022440": { "54af2191cc8de0b1a73c6bfeceff12420569139b7347df0f18a111a00cfa0d1f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.355Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.363Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.357Z" } } }, "59475ce3a8014935df370b01b5266883e7814a8041f963545d8edaf3119557f2": { "53cc67b8b0dbfb95e446a4d98d10dfc35a026203fad1b1a20cb6bf733faba4e2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.376Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.383Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.381Z" } } }, @@ -15102,962 +16324,962 @@ }, "d6c562ca4d62646ad4de68704867016d1f98e9422a7a7b9d1dab026eb4c410ef": { "jp": { - "updatedAt": "2025-11-26T07:24:02.778Z" + "updatedAt": "2025-11-29T02:44:56.317Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.318Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.319Z" } } }, "6d2c1d43528de97c8dcb1e3618555c13b1ee6ca0cec9035a38fdc267403c6c3a": { "a09b919bea9302ab3ba5a119614ec1de086c78f2af22957d06a895b8b1504bc9": { "jp": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.332Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.358Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.339Z" } } }, "84d27978ad24cbc0448dc0661dc1cf62312406d39568cc877e9bee6c04e93677": { "4120b13b5f03f7c2fd4dd243edcbc718d6bd291d7358050064f6599242eeca09": { "jp": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.336Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.330Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.325Z" } } }, "993eb4cbf451025e383f5baa954ba930c6f9ae51ff01592c72b8d36662548817": { "6397e782e35c68ed2849d7a8210eb150a2820241365b2424b92b3ac99815d60d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.400Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.367Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.398Z" } } }, "bc95ac30c6163794df098cb1c5b0c612d68e460c1fee0982a9fde6ad2158ac24": { "d710ab3ea85690006a2ba44bbff81541eaffd450228382acc7544df0e34c7468": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.366Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.373Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.001Z" + "updatedAt": "2025-11-29T02:44:56.377Z" } } }, "bdea2c6c34b1129be3efdd889576a52c92a915a41e1639ec5331bfe00948aa9e": { "d5c5bd7080a73f05e45d4b278cac9e1b97c489d95a7c80a8edeeccfbc35abb0e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.370Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.335Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.371Z" } } }, "bef9b0e0b7b38c7969e61c98c564c4f45f4514c4992c99602befb825815d3fe5": { "ecbe5e563d38c0a661a9495fe8b3be6dea6041fe9fe0a6e674428f8d203f2c76": { "jp": { - "updatedAt": "2025-11-26T07:24:19.000Z" + "updatedAt": "2025-11-29T02:44:56.371Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.359Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.361Z" } } }, "c86c73e2e1466ca9839d03145d28d089d50433e69d452f195d963042ce89ac2f": { "f65f3977310bfcdd03981a63ac5b1d00c85b04cbbc5ef4d29c352006d88c1be0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.411Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.405Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.404Z" } } }, "cb9c09aa37313bf52611e34b607eaa3775f6ebfd79387f2120b6b2b2ed4b46e5": { "b033c9754be40272847cfcdbce3fd43701961388f8efc8698510876cb0c0fb40": { "jp": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.397Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.368Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.353Z" } } }, "d7bfcfa62fea0cd11e8181ebab38199db1c954694d8230c3cb8be3a89f91c476": { "c1ce68737a5260a794d17040e187ca291588ef715aeba34369597a7058dc2af4": { "jp": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.364Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.409Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.407Z" } } }, "eb4daa639a63e99d988bfe1cd009befb853ba7171f88047823ca4d63e119f46c": { "db3dcac7ca205ca613bb9129a98b90f70d1edd49164206d1bacd86ccbb885f5f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.409Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.401Z" } } }, "ece18ee5cf148911a064ac3aabde31461f3fa90405c4631fe64e67bf35b3df8c": { "babc66efa89a5cb73d9a68a0dceb5ae1559780502d074014931e6370f64030af": { "jp": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.380Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.382Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.379Z" } } }, "eec5db41f767e87040d1a1e1a235ad804968c2645819039af5e1306f75ee2ba6": { "3294839c4121817eb15af16f39ea52c308ef56de049782978aa71dcc4c38777a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.366Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.357Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.358Z" } } }, "fcac219896966a54530a8593af31aa0dd688a431b44e0f3c677722d49352eb30": { "764c0b5706ee7c8505c4e4a557bdfcf617fad088da12e5302081d2d0510f71a1": { "jp": { - "updatedAt": "2025-11-26T07:24:18.999Z" + "updatedAt": "2025-11-29T02:44:56.360Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.997Z" + "updatedAt": "2025-11-29T02:44:56.319Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.998Z" + "updatedAt": "2025-11-29T02:44:56.335Z" } } }, "febea1a8af326ccd97db3bc68c3ffe9b9d02860dfb6225e2ad85613d0fd14f7a": { "96025027a22efdcf22fae68b1f8666c6d43d7197ab56d27461b40b4566ccacf3": { "jp": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.381Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.378Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.380Z" } } }, "07413031937c2e03067e44df8e3cbca1210ee434cba21c6bfa6e51fe5d2f01e5": { "1e96680d8322f2acc44b5d97b8bff6f35462189f2158321fb5f3892804e98d6a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.437Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.440Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.439Z" } } }, "08048e81a0b10f6fc876c8e10e896bba823ef23c25b37974243d3ce6241e95be": { "fa7004278db4a71dffabfc42db57fec5a575fb3dbd7222d4b9792bf19848b5d0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.388Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.393Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.386Z" } } }, "184cb7accedb381551a80c780323d8467fa7bd7b87d916cb1c6e2e1927c800cd": { "20fbfd2eb1f5b24eda2f90fd903779fd0847f0d888d3b04f4c7e56590eff1492": { "jp": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.442Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.443Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.441Z" } } }, "1f9e1a47c221609e49eb77fb61cad9a84a56bdb680185de6655f77145049570f": { "d2bac435d9afc706018821f07927cba0b34f6719b4e95a2c242a869d2c00be3d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.431Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.384Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.386Z" } } }, "1fd11512dba8b586ce114f0a09f748f438a3592e967e6b26fdb70d49b49b5b34": { "528c254f1d39fc4b566d364735917ebd190685375530f8192104891def887095": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.392Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.433Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.384Z" } } }, "22dec589b8fb9f267b747bb6c4caa91619a82b138da7ac22fafdf2a4d36dbe70": { "540a7500cbfe21ee07e22edbd55ff6af883a067d691b6301d93bbec754f9da7f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.356Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.355Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.363Z" } } }, "25a566b63d1b51f62e85f3301907bb9851c8e295092c6c0cbb274855aaf2075a": { "b194d71f6380d7cf9309e9c89f192bff2723d4c46d48e2aa2b48e736c874804d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.401Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.353Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.402Z" } } }, "2dc1b2de19552e6b04e43bcf12a339877b5cd1caa1251210fa995f871b2381a2": { "8023c7e209034ddbc60f9efb8a61b992915d01ab6dcdc5f5a0b08c1c7a1cf28a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.437Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.428Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.438Z" } } }, "356f390220f614f7e13052b738d6bac3386bcb14e99297bc57a7c7bf37c10fd1": { "eed67b4d5e2a37a8d51c1aaf6d8810650b97bd70d00122a88ebb97c212da9ee2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.354Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.365Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.364Z" } } }, "35c7bfba55131ad9d6116db29b6547a45eabafbca7d547b5501ea16d51eede3f": { "6a8e1ca55281999c6130ae572325abcb150b29cfd12ebe451133060b6c502a1a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.432Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.433Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.430Z" } } }, "43b396ef0d459a925fcad74ebe7dbd673c6bb8eab1d24fb377b596b6d6850d5b": { "83184a4d72d70281ddce4c2b92b731c5b7e8f98d6d6bebeedbdc0a053adf947c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.438Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.429Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.429Z" } } }, "4d4c6c8d13e7ac14a5f189e798e199562f2150ad644328ef3e5b7e6d397aacb0": { "7c2190f84db7a1c33916eca37c2632206233059ad999d42ac355995a785c5d81": { "jp": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.406Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.353Z" } } }, "5f7094d809fbf8e07ca4e02020e14a570a112a588701724679f8375a2bfbecb1": { "d84676e935f15fc8eda0f1c0db79ad9cef52b93ee23e53f9891fe1aaaa1180ba": { "jp": { - "updatedAt": "2025-11-26T07:24:19.002Z" + "updatedAt": "2025-11-29T02:44:56.383Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.388Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.391Z" } } }, "7c4de22baba4a87ac92a5d821ddef4976b0c230d25c52c53dfeac09fad83b108": { "6f7f34ba690c91122f3ae8820b83f342061fa594ff253407eb57463d3c34c326": { "jp": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.396Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.399Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.360Z" } } }, "95b82f3948cc48386b85537e6acf7fd6313c1898cc6cfcb47de65f3d618b47b9": { "f133bde7d530ef897396fcb59df2fc8c01b400e070c72c6a25647dfdb9a15538": { "jp": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.413Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.007Z" + "updatedAt": "2025-11-29T02:44:56.414Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.415Z" } } }, "9d97fe2dc29c4e251fff04450d9d510ccf9c2e831a0489dda3f038dcc1a6a2f3": { "e5572fcb5876d8a6b1d1de543d82369a494fcc0715dd6012be5bbf426e7ac03a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.442Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.439Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.440Z" } } }, "9e660b008ccbb63b66a28b42d2ca373909f19186af16b9c41ba664f7930e07fe": { "41b27ab4937c7b922d42316551438b4ad659c0ecc6b4fa06f15edf97230d1798": { "jp": { - "updatedAt": "2025-11-26T07:24:19.003Z" + "updatedAt": "2025-11-29T02:44:56.354Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.399Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.400Z" } } }, "ac6b549d962e823e381f2519f7e5e9ff23ec0d86da8d61b9555feb375c459654": { "0f0b86bed0cbb0312f32be51c009ca122e78f92ff738c6606ff98754fca7f43c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.408Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.004Z" + "updatedAt": "2025-11-29T02:44:56.397Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.407Z" } } }, "af7eb4d69ab4cdae0eb49d2ecd090def503798009a9d8e43c2370f01f9a1219d": { "048d3372a598f7a300c38f0ddefba7da299bf7d8ba7fe1a30bbf53fd7ec3546f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.394Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.389Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.384Z" } } }, "d613460c9b562b02355db0de5e4d5e795d93c8356530d72c4e6943e450e0cd79": { "21c14d0cc95de05e66c6024e0bc731b06c4934474cc10eeacdc8bce66de73868": { "jp": { - "updatedAt": "2025-11-26T07:24:02.783Z" + "updatedAt": "2025-11-29T02:44:56.351Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.006Z" + "updatedAt": "2025-11-29T02:44:56.407Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.005Z" + "updatedAt": "2025-11-29T02:44:56.404Z" } } }, "d79d5c1626358051641a02a5df10627db3ec1f8bfe82c284ecff6fc5d29ba24d": { "4b36bae2acf0c20fa2db7f654f8bc8ca933e4db7d7940a5c9c9a26463fe1a7cd": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.389Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.435Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.436Z" } } }, "09892c5c8c7770850dc4f12c85271ef2eb4054c5c9c132e0c016cfae2c946ba7": { "dc7fead9cdbb478c71bec3f2d3de2e7f32d848c704aedac7d98e3ecb52061139": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.465Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.425Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.467Z" } } }, "2fcab50b97bbc9aee5c0c03f5a35d374e8c3cdd3db10dc78091477c88a2c1277": { "0a0ef87ced393ab506690dadba9b95b3965777f4f3358eb4d004ea111fe10a51": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.390Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.385Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.391Z" } } }, "326c8895de68add9af3b55b704f3bfc1105c0f592e4c66fcf4716d6ad3d6bd4d": { "67ed218e943e01dfd5ac6127ae3673f4c5704dc7e706fa765d94c11dd7f80e59": { "jp": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.473Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.475Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.474Z" } } }, "3aef4f3512c85d4057c69557fd85794d38328c9e61205b126b37d4de45a963e9": { "06d1c97a15302255ab6d9a474e72aa8993ccc93d6749dfd1e5e94970da469d29": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.427Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.424Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.421Z" } } }, "467b72fd8dba8502decf3c42bc9358fa8c4d3014dfcfe6b42bb8f4dce198fd62": { "a67f1de09a8a84f9d6443a0df3a49146ce63494d30ed1c458b9929b32d5a4b7b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.468Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.475Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.477Z" } } }, "4a37cce1f00cda917ca47dd0a1a69934772f9e50b5150732050d2e9f70a019cd": { "f5d8080ef6746049caf9a9d8037b9090eeef2259b54e9f42ef3e6a135b796e6b": { "zh": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.466Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.480Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.472Z" } } }, "4e436a71846d9aca6f15dc8c5445f526f911657bccffd77d51b5a4689a95bbf2": { "1ada5cacc80d636b19794a43afd3d71292a74c9e3f3fa93f182b39eb84ad7355": { "jp": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.432Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.431Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.012Z" + "updatedAt": "2025-11-29T02:44:56.436Z" } } }, "625ac60abe1e4f7ce4df8ac9bffd1f30f906501c1b636c41e7dee039c1280348": { "eaab285929dea7d9ff8f319faad61a28e866d384a56d15e9eb7a2ea10d96b567": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.387Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.434Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.430Z" } } }, "67c93fd175b134b8986f749e1afceefc6f06a4487d9ef161d2ea74e2be618233": { "5418ed61ccd90e17c44bbf1d4246b7b4344bcf595b331971dc74df17def6dcab": { "jp": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.443Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.441Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.444Z" } } }, "8719f0b66c142c6db2cf8773ebaddac3b3d310bd046ca6effa5bb15e73d1a00f": { "9c001ffc30fb8da63ebd6c0931ef3efb9ac209edc160ae449428bb65298622c3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.426Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.420Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.467Z" } } }, "89ea779c156a999fdf17c325da1e81dd07a635d696dfd5a115e631154d3dbb2a": { "ecc1acdcb21d77d65ebcdd760265565e99254e242903d6b4483da0a6b4a59482": { "jp": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.470Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.481Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.416Z" } } }, "9b137d113f115786a055cd8fbc160635ea3e53512ae73d845fd749380bc1f381": { "0e565f9a4b2a92384daeaab520393c6426e3c190a2625839b4ead735b7a693f3": { "jp": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.420Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.469Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.470Z" } } }, "a524ef715c9704a1e92f4f92e0b9799ff822e7bf9860bf751ae2b1ff9edf0afe": { "e0f8014536b364d9d9409cff9471107e76279833faca677b2ccf2c077400b851": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.387Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.394Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.395Z" } } }, "bc010b67445245013c815d8c8dd2a711a400f2ac89689de6a705df179ad8c706": { "58a5d26b93b4269bbcac95ceeeb1329954babd6a907538f5319432f3ac4e6b22": { "jp": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.425Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.471Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.422Z" } } }, "c0ac70d88c31f863cc7a3f822cfa525fe69266c4bf831f94c2029759cb9726db": { "b931df20b4f6c77ea8d226087a40d67fa3ecf5f9d09ed73922e7aa8f8f763fd7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.425Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.428Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.476Z" } } }, "c58c920060a64568fe6e344fe00a5ce4d720ac37a93628692770ada830c5325e": { "4a343784a2e6508b5e218dd32d01eb13fe7c9d806b2cb07a5c39a775f7b2383d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.445Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.445Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.013Z" + "updatedAt": "2025-11-29T02:44:56.444Z" } } }, "d61225a37fe0c4d963dda12e6171915748b61bb4ea252b20fee7017863e0f8cb": { "e22f186111d1f322fd63ea2a2ab6b8dabcc933c9f1a1d547efbcaa1d9f78faec": { "jp": { - "updatedAt": "2025-11-26T07:24:19.009Z" + "updatedAt": "2025-11-29T02:44:56.390Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.385Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.392Z" } } }, "e59d25e659a24273c3eef05daa226fdbfb119134fc9c360fb8f10fa1eda0bc5d": { "cea9fed32032cdfb1fc07ee3fd025b189b279642029231324022cc8c275879fa": { "jp": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.393Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.011Z" + "updatedAt": "2025-11-29T02:44:56.430Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.010Z" + "updatedAt": "2025-11-29T02:44:56.394Z" } } }, "ebcf5c14bcf3f123a8337f0e4d01711d0d5350b19f8fceb4989ba4967a454d71": { "fcbe8a223dbb47bb59f5c3e6880beb175753d21025800e5178cb322086eb6eb5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.424Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.477Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.471Z" } } }, "f8131ef0252b8ff50e0e16a5c5a263d8c4c19d0d5eed0109ad5127d0b7f1e617": { "10eec051f15e6d2b7349c390f8baebb76014741ed3b8e31aa94bf797e786189b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.468Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.479Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.473Z" } } }, "57ad9bdcde77c82a8b9abbf11d3820f549bfb779a29aa35c949fd4b27ff2f01f": { "1e38948feed7f1b2a7b35c47b430e56f07e2438c56f10e45d636ec500990a43d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.469Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.465Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.015Z" + "updatedAt": "2025-11-29T02:44:56.421Z" } } }, "7ceb6e3c9247297b0f7f32b5bcc5fdd805490fb8b1ac4cb176afdba619355e4d": { "ac6e6531f103ea9f5613e39ee61cfcddac7133be00040a3d2577c40543aa27fe": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.467Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.426Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.474Z" } } }, "b623b9673e0f28805a4afdfc8013cc9c06d3af3bc31cc33238b2d1a449d4888f": { "141f6e9d777628dad68e29e4db62adc7411f17cbe61f3611de81835eed95ff15": { "jp": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.423Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.422Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.008Z" + "updatedAt": "2025-11-29T02:44:56.415Z" } } }, "bd529fa629c3f10310f525658df96bc82a80d75ff52d1995cafe1e4b13e747cd": { "ff7e68ef737ee5b8e75daa40f60eb811c121bece05086608bbe25c6ac85d8715": { "jp": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.483Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.481Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.482Z" } } }, "cbd3fd46a4918ee9b9919e72d00bd7ce3d00418bb1705c843c677adb3e595a3a": { "0613ad7af0509f61658a0f7a5e17e617139bdf209f37e63f862416353f1241ef": { "jp": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.478Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.127Z" + "updatedAt": "2025-11-29T02:44:56.476Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.466Z" } } }, "e1167cae2cc6ec3ff98f99cc5fdc814c1613d183ffc5a294e5002a5c76629f89": { "bdc0fd08e9185e494c67e0405a76d6b5ff3f2a66fb66986f38ad9fb1486504d8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.126Z" + "updatedAt": "2025-11-29T02:44:56.472Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.479Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.124Z" + "updatedAt": "2025-11-29T02:44:56.423Z" } } }, "fddeb9c1bb988ad91fa2ab2fd48f16446790394aee1f2ea892b74b4703663d8e": { "40a994cb1728118007e9bcec1d1e95be3ceda608e471c1a73b546b7c438f8ebe": { "jp": { - "updatedAt": "2025-11-26T07:24:12.125Z" + "updatedAt": "2025-11-29T02:44:56.427Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.479Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.128Z" + "updatedAt": "2025-11-29T02:44:56.480Z" } } }, "08bffe1dc74222857bd9847a9079d22910222fbbdc1dd9975d3beb50f284f3ee": { "6ff985dd3eb042cd0776c0981bb360df764da84db1d5f50ba4c7bc2fd4394a58": { "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.571Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.554Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.572Z" } } }, "3a9bf422a9a85629cde7696a05524d19ff37ff8a14e26aa9d363708d50ca69ae": { "3106e22f04396e24e2bcfddd311b6bf015d441abff426e8f3e45320a55f20c46": { "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.577Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.581Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.580Z" } } }, "0b2918c33c54636514f9106c16c6353f3c90189cb35c245462f264976b158e49": { "8b5e41f784e6af3f274d3cbab3bb71a982e9a0b2df5cd5050b3f76eca71460f7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.576Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.545Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.548Z" } } }, "190291cbeb8e03da636d545882454df1f5969a43233fa8547a340888416e0d7a": { "1e21922b278cc488c7ca6142a0b58330666f67ff429c778024409f871aeca347": { "jp": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:44:56.554Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.561Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.560Z" } } }, "1dbfde47d0ccaf8fabcd5ad6a4296155b1b096aae0b5f8d17a8c1b756b2695fb": { "665e7928e61709a3964eb76535bc335c1bee18c8bc09733558199e232956630c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.547Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.550Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.546Z" } } }, "1e6d8899d944f96b533c9b1689dd0f3c45d1f4d88d4d1edd3d0cd126273c28ae": { "874433a820ac2a172772ed12a2a2e43d64d72b5fa3f8c9060c2ea70f9d9969b6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.551Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.570Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.550Z" } } }, "267616b5e710386f1e95782b057051b61f78cf2ab9ab90a87b76171e1110ba0f": { "526635ff55be813366ca95dd8408fe2713af702ad3c42ee3f6df159c36d7d754": { "jp": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.568Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.565Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.568Z" } } }, "3db2189c4ab253714a8026c40657f8d09c5b44880bacd30f5d37a00af55f0af9": { "2e5559b28181e920ab535b8433f1644911413cf5aad2b7f7f2077a2124cdb9a5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.573Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.549Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.549Z" } } }, "4887a31d41443a8cec80c653b5cb1471ad7101392e2a0fd85344bf550b4479de": { "5d542d21d2aeff7420ac405c3efb0280de56bfcdabe3edfdeea55aee2ee0816f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.548Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.547Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.546Z" } } }, "5e3e9bc17b90a0989880b5acd7291677843b0466fc3c36993015c0a7331f4c86": { "50e422154e7d9340b9ae3e608a02ad691373881011458d12ee9329b251e2ee21": { "jp": { - "updatedAt": "2025-11-26T07:24:12.158Z" + "updatedAt": "2025-11-29T02:45:02.118Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.130Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.583Z" } } }, "6820315a7841bbc8c89a60ac5aa8c0fe4457e414cad746f3bed1650c3f297bc6": { "6d8963200cc850f442fe2995954f739d20436c4a7fb4b2ec7f8a636bc53779a7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.131Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.570Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.578Z" } } }, @@ -16075,117 +17297,117 @@ }, "3b63277eca58a6be86961cdf77d03b10bf3995740802c612a1fe8d80ea7d20ea": { "jp": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.120Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.121Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.121Z" } } }, "e0c7e0ffde8dc72698165f5f5a97336beb9082111bdd4a6c98f10c02ab69cd27": { "1bd7f94ef79ae4a259d5eb60f577fdcaa8d2926824240d88238ffb4e9d917715": { "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.552Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.574Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.575Z" } } }, "09967fd0502ac05bc286aeb301c2cc87873b2a18ef14f3e2acde54345b2ce839": { "ced484d2a382f8655c9d000bcfd985aa94545bc671aae3824c264e06b17c1fb5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.283Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:44:56.555Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.280Z" } } }, "181adac272e2abd83cc757fde65fb79cacfbbfdd22c49560ad9938dc95ca360f": { "6aca92cecd7097cb7ee90b10d02efba74d48a3de1843308bf7b14b842592c336": { "jp": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.562Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.561Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.569Z" } } }, "1e8da80bc94e12875fbc8b4285abd87a9ebc00408979ef39716bb53ce4293704": { "cca901fd78a63bb4eb045aec0ee20699b9ea63520630a96e5bc254085761c479": { "jp": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.279Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.281Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.279Z" } } }, "28af1868b1ea9cdd8d1446f03dc1a91a48ed271602879f18d0d3211752aa2e0d": { "38f892b234c9e0a9d0e3e6bf087768671979427a8bbaf831e2c0cd94e5730d2a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.284Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.288Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.291Z" } } }, "352b7210abed12f4834ce3966861f5819c1b015976a552f4d8f3417367d6519c": { "aa0583b1c517ae46447bcd58d7475ba0f4350a3b5974cd1a472f07e84ea2b12b": { "zh": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.151Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:45:02.135Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.595Z" } } }, "3e04e93b41ef14736c12d8caaaae2fd7c113b2b4ab71ad84553b87b688b2ce7c": { "44da72d1f89df587a02ef24e707acb1da8350d35e7f7a73fc92e5b863e479a62": { "jp": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.290Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.289Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.276Z" } } }, "3e44364087b9660f0daf48e77e2d51e82485581dd45ba3635100c92a48f5cc81": { "32a51ef86f8ce4d5a89c21ac07056345cd0ad1dcf0c3eff8343941896b7a8a62": { "jp": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.123Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.123Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:45:02.122Z" } } }, @@ -16203,78 +17425,78 @@ }, "f1f6c6ba727fcac4034e8e533a8a14914f296de5811f8ef69aaccc190ed52c04": { "jp": { - "updatedAt": "2025-11-26T07:24:19.020Z" + "updatedAt": "2025-11-29T02:44:56.556Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.559Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.560Z" } } }, "56a2d0968dd32b192f6e6833bf129bd2a1a73e16d498c9f8a64c8e8cefcb7635": { "85317ab67c21185490c8ce6da9f40ae75c6aa792d046b52122da1555de6a0d7a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.564Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.569Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.565Z" } } }, "57fb93819b163681fc7674df87acd51d16808daf3c9a80875363e714ab6b6f0d": { "589fc5521d34b691619a0775483550005c0339c397f9c5eb2ad84a68d38fc0c5": { "jp": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.572Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.562Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.564Z" } } }, "5f7acdc3b5ad3c4b70f2e0f6421eedcef49bbf5fe1541b93de796181d282e3f8": { "c3b3c36e1615ad52f46683413733ab6deb9809b9216880d962f14d2b316e6812": { "jp": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.579Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.582Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.130Z" } } }, "720286aedee663b0895eadfbb8c855cf28e8c889a5c1e959eba2cb56410fe0ea": { "8b424c806172df3664b5a02f66fa091e75d922eace7c6d17ab06a1cd4d48ded0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.563Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.571Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.027Z" + "updatedAt": "2025-11-29T02:44:56.563Z" } } }, "72359f73659f510486d116f7315a779a8e182fd9217ec98122618754e8e8e523": { "b7f70662c0d64e5760316e2f601553929e92b4cd5b7d382d9d395b743c0236de": { "jp": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.133Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.159Z" + "updatedAt": "2025-11-29T02:45:02.119Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.131Z" } } }, @@ -16292,260 +17514,260 @@ }, "13a75f2e1510d071925413f0a9794c0c5df227d3f3007ca6a25a865fbf3c7afb": { "jp": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.557Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.557Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.026Z" + "updatedAt": "2025-11-29T02:44:56.558Z" } } }, "a27f8d321849f13ef579bf79bd9fb504adce87fc32377cb34f1d87d0247b62fc": { "0af225620d1128bf2b7b6df1fd290b2f9272232c08e057bbcdddcb8da980d877": { "jp": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.580Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.132Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.133Z" } } }, "bf4aa8d8478e9cbccac2af56a2392959e788a6b441ae1d334d378fe41c813431": { "03be8e55e0b7b3239928d3c046bcafe55731c78e43aa66ee2a92c237cad32296": { "jp": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.584Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.576Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.579Z" } } }, "c6f8d4ed5ef7dc56f976117869cc7a69922f064662bcdd47f24b593a903bb511": { "66256e49527646d9c1360a5db02fe360c867281e0fbebf9751bf3d0a5e4e0116": { "jp": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.566Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.553Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.023Z" + "updatedAt": "2025-11-29T02:44:56.574Z" } } }, "cf5cab052feab37e254b75324c3a852334a8eb3c58db22a1686c9494d09f443c": { "d809412f215411acf69b12810108cd424016766dd4d30a992351f8e69bf650e3": { "jp": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.553Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.583Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.158Z" + "updatedAt": "2025-11-29T02:45:02.119Z" } } }, "d9f334133320c651967d1b5b665ba9cb709fe4d09178893258245d70b28c5b25": { "ab1cd75a382114032d421c93d59ddfaae337e9528e1ac6b02cc19764422a2124": { "jp": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.285Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.032Z" + "updatedAt": "2025-11-29T02:45:02.288Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.282Z" } } }, "da0fe2e9eb4d4168fde541e5a4aa216882f11f0fe02c65758804bc42306051b7": { "460c5141199908b2fb1f8ada87d50d25899e1061548dd77278916ae9f0194eb1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.025Z" + "updatedAt": "2025-11-29T02:45:02.132Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.581Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.024Z" + "updatedAt": "2025-11-29T02:44:56.582Z" } } }, "e1c1bce938fcd121a541dda56b44194cec991a3c399320d28f68662d4f2aa155": { "ab303b424478e35d2600d95fd49a29737cadb6308b63d04e43f98d10c38b5cd3": { "jp": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.283Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.286Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.282Z" } } }, "fd5ff75cec53563913c25d3a84cb92ca6b7f928115d7912cef78a22dfc907f29": { "ba4164cf48205f79abd50e8ce1180feb106ddcdda361d67fbf580922f1a8bf3d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.552Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.021Z" + "updatedAt": "2025-11-29T02:44:56.545Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.022Z" + "updatedAt": "2025-11-29T02:44:56.551Z" } } }, "176d0068a5182e14c24f7b86a941e2993dd5d5375dda5f359181472f50bb49a6": { "3c0a49ce0175e9ffb151adc18ac51e16f2d58c189a49b071eddff19741b2773b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.595Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.593Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.598Z" } } }, "2fc9ece7b731c86425713493bf6fdb0053ccce96ffd9f63a70eea4019cdff660": { "547949490f707e9c4812b2f1acebb85c8f7858c6f4c8d030784a54ffa0f6764b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.146Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.142Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.144Z" } } }, "356a5236e325bbd80f92c622b5549c7f59c011b169fdc94f7b59ad1948f64d59": { "32a464d65d3033a6f94c395c523bdf9d52473033f37bc7b58a4c7d5a3374d78c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.140Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.140Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.141Z" } } }, "4dcf3a152974b0406b6bb68f5b1c541fe9249595ec4170e386cdf67f9e97d6c8": { "144e0319e32e38db32a1efd639ffc72bf732e5ea7b5d6a3d0883a97e4bec0cf7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.150Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.601Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.596Z" } } }, "512bf2a261651531d1f44db97f0e2477f9009f4f748fece66e5ca2554439601d": { "f65ce8822ff0abf42d5c376dd8120812baee55885d0c7b7b65bd770ce9d25050": { "jp": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.141Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.148Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.144Z" } } }, "65955c38f425b134d13cac38e2564b302a84b647113466a30fa84df4625f2aff": { "e5d27d0981cb097f6f8db2c3366ef654946ffdaba0ea5433e234e0200fed3d99": { "jp": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.596Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.594Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.593Z" } } }, "70760b9ea84a1492768f54f60022928ceed80c33ef8d2cbbe522324f7979123c": { "5172acba2103f95752ebbc8f74579f1012ec0e81bba84d6402deb3f9ab3b0bfa": { "jp": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.602Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.599Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.598Z" } } }, "832f10a64dee00c5573ad8927271c0f08e6912344a6142b218901f374557d6d4": { "c00fec44d98d20ecff726432315131e9d6815d1bc6d528bba1cbde655c11121f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.278Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.202Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:44:56.578Z" } } }, "85aaa20028d2fe29973bbd19c0fe7f0bbf9b2028122048baf8aa80c366fa2134": { "3e3cfccfbfc2a9aaaa5c073111f54d43e1d4a01447a2fdcb70bbf2ad0fa40c15": { "jp": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.277Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:44:56.575Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.277Z" } } }, "8edf9e4f287ceba4ca2d82f14382e035360e320bcc403a4bd0ffc3569444e7f7": { "0210849faec51fc728046caa3f03b71304bb9c646dc07169ab1c6d9e340a0aec": { "jp": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.202Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.567Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.573Z" } } }, "9c07a7cf8bf10809ed5421b224c9702d1daf802a6511bc28a61380182a3cba5a": { "4e8ed6a1feb2aa52a5a2a4588b3ecb8b8ba68dec83a27b9280790c81f51a60e4": { "jp": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.597Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.602Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.600Z" } } }, @@ -16560,31 +17782,39 @@ "zh": { "updatedAt": "2025-11-26T07:24:19.030Z" } + }, + "2f0734e7c9a31840e186f5a334fbbbc73d1d52db49e8bbda9d6d1527b330a0f4": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.392Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.394Z" + } } }, "b39b9077d3c9edfb0122eda19c18f981a977ba3d4b35e87ca4e9808c93c00357": { "c4806c1db71a5a0e8cfe750303156d37b0c67170fa9901e7e2fcd40bc40df990": { "jp": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.566Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:44:56.577Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.028Z" + "updatedAt": "2025-11-29T02:44:56.566Z" } } }, "b57ac847efe3da698e4e7e930e7c66f735f45e722a25a0fa39bc6f7bfcec60cf": { "9c431dd0d8265db20267a05a0e5cddc327c798c7acfd1be5071f066d5a7aee28": { "jp": { - "updatedAt": "2025-11-26T07:24:19.030Z" + "updatedAt": "2025-11-29T02:45:02.280Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.287Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.285Z" } } }, @@ -16599,1006 +17829,1017 @@ "zh": { "updatedAt": "2025-11-26T07:24:19.038Z" } + }, + "a9aaf3d0acf90c263febea571cd562058a89cc9ae231894d698d45f35f8a8089": { + "zh": { + "updatedAt": "2025-11-29T02:45:02.135Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:45:02.136Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:02.136Z" + } } }, "d6df5c4a34d83a6b139258c32c70c202447dbb8bd299cdbc95f1a4df8e0183c4": { "cc987465957a6c156c8cfb8e7e1f5591232ce2a233ba2e10cdaa0502c5170fc1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.149Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.146Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:44:56.591Z" } } }, "e8326b6e3e229b53f7f7616dad224e62d5aabc8c99d1885fa0b294be36436442": { "e0c19959bdee8150958356d19999762296868f26f8c58d573bd31ee946774713": { "jp": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.150Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:44:56.590Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.147Z" } } }, "f6456b0e678701e28c6a4e322798fee754b4c6d0f806d50583a4b3bd2c244c77": { "b8b48f150dd2033fc11782fa83bfba12af99e2588c361eae29e969d7df966696": { "jp": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.144Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.145Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.143Z" } } }, "581431969901be3a99a89764a4cd843b136cf34d9c36a58c385d297bcf0b5576": { "848b4e2ed1094aeeb74cb89d7d3f155262e075c04ec6a136f164406460b1c404": { "jp": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.142Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.148Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.033Z" + "updatedAt": "2025-11-29T02:45:02.141Z" } } }, "90b8b253ec086b1363c721e07a29dbd20c3e79932831c40618a9e15eaed1259d": { "558092fa5958f7bf2b9c27c89f455619f6ca6f3513e83b59425458536609e8ef": { "jp": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.592Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:44:56.590Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:44:56.591Z" } } }, "b22d1260a64a32ed7c646aebdc8304e5522445a10e936e31715082f3976c0efb": { "0350b0c4a0edef07c101045887230f235288aae9414af376658d84671b54adbe": { "jp": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.145Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.592Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:44:56.589Z" } } }, "ba3d45a637c836f2218890eff93fee4103508fa1c470944799207121717e02a5": { "f3fd1aa8bafa81bb6a7e865a5de62823158a0afcc7ff7586bf136a8b47ee3a88": { "jp": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.147Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.035Z" + "updatedAt": "2025-11-29T02:45:02.149Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.034Z" + "updatedAt": "2025-11-29T02:45:02.143Z" } } }, "fb6facb17dc3579b44508a305bcb4895b64ecd0ac72b1f50f97559b26bc78b2c": { "ad02c360d5787e1cd581329efbb507dd02fe16448697b4344569b5bc44e930ea": { "jp": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.600Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.038Z" + "updatedAt": "2025-11-29T02:44:56.601Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.599Z" } } }, "035ee5282a6d20879fad6bfb6f79b4341721a15ea3c52c10046b1dd3709aa16c": { "58cd6998f41fdded6a804039b2debea7d2278499d73c45aac0012b7439df220c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.423Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.421Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.424Z" } } }, "26fd7d38f92eb5055170efb295d4a4f87a521a38805a47e252302040001b2050": { "6311029c9bad9285962dc8c797429aff225c5d236c038434dbd0c88cfb8a7048": { "jp": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.398Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.415Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.402Z" } } }, "3f43afba791f6baf15364b9b47e22c85a9f1b3dd6af0e12ec732f9dcec39457f": { "1dd4bcf22efaf403e36fb2a77e769a0046ad25b9ce5480ba0ffe16c707a0ef4e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.458Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.416Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.414Z" } } }, "645f7fd9f0334b6f31287f3ff16746bdf9b9befb1bef269261f6079af9ff22a2": { "4cfca9fae37346c2e6b247de1cc83bb1880d5d141f5ad266dea6ae52b8cce258": { "jp": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.413Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.395Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.454Z" } } }, "870cee0b248ecbcf72715dfd0eeb85ec9af5efaca8d3edcf0fe8c5264910fd76": { "31443088162bd3a031a32984a7f4bfd930cc979d324a47439b26f35ddd40c4c4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.456Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.463Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.448Z" } } }, "87cdbf09a8306f33d341ac3e84a3332c186b170f3eaade4500b0517c76c52c33": { "27bd6d01dce2d6441ee156267183789fdfad03cbf3cae1fe51042763a3ae5190": { "jp": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.416Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.408Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.422Z" } } }, "03d4f9de31c6bf8adc70ca8cc91ea13e8e9e9c9401061a886ff406f2ee77507e": { "31a8fa488c7303d5b196d590f58b9ffddcbbaf82dd7d661a3d06b19f60b7ddc5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.438Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.473Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.495Z" } } }, "185920906ded891a9d2e00cce1434c3336837203f6a4afa9c0afd1752f259e14": { "fb5ace8ecf41cd7a84a0650f9d96ead8a0c11e0b73eb701d4b8a50861ed41f3c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.457Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.411Z" } } }, "3b5b38cf7b3fbbf741ef360cdeaf09b58c18acb3ff66337f95d902be5f6db59c": { "b37e005c51f403fc9b37bb6c5b5edef44101e2fc840f20186238b36701cc8e6f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.455Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.460Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.409Z" } } }, "3bc42dea80614a09ae6a300caa882b3109109bbf2c1ff3e4a3cad15872847cb5": { "90eb1bd6cd2087520e2d3b6a42056c3549761f9a48d001c400844b96b08b2d5e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.455Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.456Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.452Z" } } }, "4864254e07b5f2ba04547ffdc42c9fa734db92774140cb47efb6c312ff52493e": { "6dadcbfab042a7bcad0c4076a815d1b10666957ab124f50642fb026d185c6859": { "jp": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.397Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.407Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.459Z" } } }, "4b4055e2a3996b0cc1db8bb8b7f1a428a61fcab906f4eb7fc9e8525523570823": { "fe2aceb75f41309c99fba4ee2a1fcbdba1e53d1591a97e9fee22b69867854012": { "jp": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.472Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.502Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.501Z" } } }, "4c57ae2a858123d1bbd05031233c5f830692e6ff38484e60425dc1e644619e86": { "ac07bacf3135df09429ba59c3085014c51cd2dd6322c81c9cf515a50ac42020d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.464Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.464Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.463Z" } } }, "5f4dd4a5e3b9c2038ce5d97add1c57db4cab04802675890f9a71c7e24d65298e": { "54f6ee288acad5771ea6bb244846d3f7f6f97153a3e95cef843610f79d82f51f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.412Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.402Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.448Z" } } }, "8c9ac06d9f96470f385b45eb7382ea57d23824bef86ddd9dcd04eb31af945385": { "8fd53472854410898a96195caacb583e709b2c67f304949a81fcdc9a6ab77a22": { "ru": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.454Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.452Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.461Z" } } }, "96f086ac06293e9e587823d8e326b7bdd10741ec2cca41ecf709e6dfda01a137": { "8cde4367a08c4c85a443e691e36a03de277bcadbc7b5b8042f83da242fb60262": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.450Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.408Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.447Z" } } }, "98763ad1765b4f7ce59ab7c28c03d9f16eb7ba20340f1fd72f141425b73dfcda": { "2b4ac034aba018ed0128e4b4b5e46817e96795dc002eb687680ef694d17118a7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.415Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.422Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.418Z" } } }, "a1f67d04d8c6c016319715cd37f1aaa7fea045040cd960873db250061b59677d": { "c042f748c77a461dd754ffe542382a34bd504df511e412aaa671006d2a6ce920": { "jp": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.417Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.418Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.417Z" } } }, "b6e6ba59aea8d42356d10f15f3e251c9ecdf84b70f6b284cc535f8f2715be871": { "78c8f7d218a9c211659cb2bb3308ce5d14d1718fcdc5e47d42d5c5f55050e6f9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.174Z" + "updatedAt": "2025-11-29T02:45:02.423Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.420Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.419Z" } } }, "b96f31274279db19ee455ef4a211f35232718d535097413acc9e87b2c16cdee5": { "d1a30df1933d77a7366535efca514780aa4f237e66085e619643f85b025ea495": { "jp": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.406Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.449Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.397Z" } } }, "be5bae686c5d8c7787f030404d9391d2b796570ebe3949ebccadac637ae696ad": { "aa76d4c663800697eb6cffaf9162ddacf8d4a6e9e85ae8732672b1aa668497b2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.421Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.380Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.173Z" + "updatedAt": "2025-11-29T02:45:02.419Z" } } }, "c61d725ce51260e373784d5a559f17b1c985d873f35f4f40d34e5dc3c9d30214": { "164319294d8a4a2d8ae935edd6e5941fde821158fce1cb0fdc3c94aa7eba994f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.414Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.365Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.410Z" } } }, "c9a3c995b2d1f1da16df65c84fc5fcd7d61a80112b46a37925da4d4c5cdfec2c": { "fe45037d34e9b052151f9190e1da1d3bf5cd89744c552cf345b160f37129f8f8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.462Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.459Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.462Z" } } }, "e87d7bb771e6e969df1f4f17a2cea74b1703104f920ba5110ee4c2bc95819b7f": { "c626b9222d67c0a16c11e25def509ff96d4a34afadbccdcc1676284d3fb3c55c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.059Z" + "updatedAt": "2025-11-29T02:45:02.367Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.379Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.409Z" } } }, "f366eb4cbbf4ae87e0ea8145cfd5006bd57589104335fc046ede417d016c390d": { "e26bd50b67b6a44512d1f83c42aa88dd3b0ee7eea44771e913a93704b405e585": { "jp": { - "updatedAt": "2025-11-26T07:24:19.061Z" + "updatedAt": "2025-11-29T02:45:02.378Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.410Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.062Z" + "updatedAt": "2025-11-29T02:45:02.379Z" } } }, "0dec45ecddb0d4b9ff3311f5a670eaeb053be15ec02969e2e3cc776a6771ff5c": { "77a1b67ca7c88505859a9611495e54062c95a3d5051d05c9862ba6120252576d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.485Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.530Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.479Z" } } }, "1345e1194d63be447e8235ac3810d70f7853efd69e98e071d82ffea7cffd7a32": { "40371c6acad0719623ab143c6991d629d5eeef18fd54755245385719989fae91": { "jp": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.489Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.466Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.469Z" } } }, "1784873802e351d4cbfd164226e7d919a480bb1d6312139fa09de23c15d16a8b": { "8742e923d01dd09dc7d8778dca915632a84b942a268948d3212bfca23e4e87e2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.466Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.473Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.468Z" } } }, "1976a270e928ec95aa014d1eb571385ad93c7acfac83fd172543fcf63d413493": { "28f4800b7936b39a171e2fb6c8317b4c9829a963ca30e0d8f2cb33e3e1dba27f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.491Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.480Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.474Z" } } }, "19d053d8db1755b3bac1323b8dc5bdf881a37b3de8c55e8397cfd48c70b492c7": { "a35e75c19a0f228c55c8e74114787fa88e13457d020f241643da1e080c35d9ae": { "jp": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.502Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.465Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.441Z" } } }, "1de644041acf945417d447dae1559f7cba704ddb7f42f4989d75f53b3432bcc7": { "0d354a4bc3cf5327de48753ad84ff21b24119bc6b87f048f6f36a86e9a56461f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.470Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.469Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.467Z" } } }, "21df29d894f5394f8a93a1ff43ddfcea466286f726a703a29d7f5ad5f777ca4f": { "f9004a0faa2530c5a49f802aa2e8e063889d07b4b5779757539ed40941914621": { "jp": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.477Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.478Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.478Z" } } }, "22ff9a2316c586c12132ac52204a80c3282c99ea70504b739a00fc4e769b9090": { "9b6474c5f66a5e775df7e704ab5583bc77d7b503d80449e41bcb0fdca582d72f": { "jp": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.506Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.445Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.507Z" } } }, "642f1cdcfe6481dcca55bd2f485397c27b2cb519506bae85d0903d1022a9a534": { "d58e38a4b38f0454d5c08c7d2887270f277c732f8c21e5a62fa24568ae4fc2a9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.494Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.441Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.498Z" } } }, "76e148edd42e2339581c7f24e0a25ab51ee37d3723b355157641afd3cf2a92ac": { "96f0f82692a94d11ec4bd22df9bf9c367d91f54e7f111247f17715678d4f8a7c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.451Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.407Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.411Z" } } }, "877ff646acb9d8b60cc0a8c397ec6865271899314d2f8d8c3bc6835ea0a51d87": { "cf8035df5e02498f9892ec6d01d716e4e210be81d6a338a2a670b395f2d05b5f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.447Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.406Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.405Z" } } }, "ba2b228d4949b83493253e6cce36fa61e4aab29868007f5c4dea719bd97fe4e3": { "bb371d742e1c3d8bcdd77214bf030643a0331f8f48e7727cbd847a8a32b85ac5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.453Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.400Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.398Z" } } }, "c88c05312ecb48fece611ecb971d8437aee67aab577a01d65950c88e236c100a": { "d28f12f9ff28bee751ec769892ca255d368223c72a14abe462c9cf6ad965a8cc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.491Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.503Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.502Z" } } }, "d517690990eb5a5034e28f3526bde41c42990306742079c31f30f4ed4524ed91": { "9c79376ce670521bff71e976361e6729afb8128c48c2bd62e07e55c58efa6cbc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.400Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.179Z" + "updatedAt": "2025-11-29T02:45:02.449Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.399Z" } } }, "e226489ddbcee1a5f588fea5844e21dcac309588b3ec1f6bbc9f7bfd26b0953b": { "5792c89f06fcaed31fc80316244e3ff2495629cc4d68214bf2ad0fc8b2cafcae": { "jp": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.399Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.401Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.405Z" } } }, "e3904a052cbf5a3387388c389ae010ddc49649dbbbff19900f769f6e6cbfa1ee": { "e3e518cc255f67640d601fecd3cfb11ea7e915ddf282acc6eabba8311aae5b22": { "jp": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.451Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.450Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.453Z" } } }, "e6ad4f2ee58b9c424f0cc4e12e443aa3bb9dfb641432accc87e403a8b0597b0b": { "d64cf4716347332440eb8c9bd7192e0eae84a3f3eb49ad6ba4155f87567e3861": { "jp": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.458Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.458Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.180Z" + "updatedAt": "2025-11-29T02:45:02.460Z" } } }, "e8d810b58d2fc954739ecb8eae76ec7772a7459c01a08dd48ba208a5ab4b2b58": { "0d3df994d73dcce5dc7c4ae8f510488dca241f13863b2cb49c97f6056079afb1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.404Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.404Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.178Z" + "updatedAt": "2025-11-29T02:45:02.412Z" } } }, "ee906a548fde378c55bde17a104978853c964efcc0ac2037f2cc5f90ff301836": { "f49e9e3f91b64b3519c5cc4cdc59ffcf9a84b52eba96cc9a68e95e42dec254a2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.445Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.446Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.446Z" } } }, "f17585a5d8e2bdd6a2ebea5f856955881ef4c473fd73048cf4f26e56bdcb5db2": { "9e7753f5e285750271319abb9baa46c784486772a2b4da88514c28c5141c5c81": { "jp": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.508Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.440Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.509Z" } } }, "fdfddb9175ea6844a8f625eb6ff292798d8dda51dbc62ca44009000f3177a4c8": { "a1fbebb2555661587982370786b093295909d4be9fcca7e32ae5eff02acae18d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.177Z" + "updatedAt": "2025-11-29T02:45:02.406Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.176Z" + "updatedAt": "2025-11-29T02:45:02.403Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.060Z" + "updatedAt": "2025-11-29T02:45:02.396Z" } } }, "04fc2fc59d087b4841db1401316e4d1c9ac88f144242faabf25ec2e969a5215b": { "414e7c4dfb6cd3da8443de0d53c94c82fe3258fa5fdaf93915afe2a8ec3736d4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.540Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.541Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.481Z" } } }, "2fe2ff96c504c59daad55285eb365e9e69fcc5eddd301d8a0409670d1de5a9ac": { "79af085e05f9fd1374cba79aa1eea65a5fa7bcadf0fcbabfc3df348faf04e6e8": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.488Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.540Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.496Z" } } }, "32c8d946bfccbad7f54bc00de27ceee1cc1719758ec7a678b9763d7236502014": { "6c958d1bfa513f4a8e0811e9c383ecdf775c2aa645e088ea7d02462f9209a69c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.487Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.482Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.482Z" } } }, "341eea9182cfeebd2c27c019d06a39d1fcf951c990bcd80fa61f11ffc6f9e196": { "aba92e4ddf93c8ac27c276aa33d276f9987cda30270a7b50881edac3ee8d0b71": { "jp": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.499Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.476Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.475Z" } } }, "3fe31c561edbb5416b22ecceae952bb5b07567cc07d75cd64ad4a2caca7689f8": { "af620cd5ed38d2654712e19961c6712bdc7c780d345e73f17ae49396a20d6df0": { "jp": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.494Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.492Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.492Z" } } }, "4afdda2989ef4d77a80eb6666ee0e5fd90ac3afbba1e33f8e39a07be3bbd203f": { "6d99a0d2cef83d17f6510958c4402246edefbb9b9d564c2e37e017791950e3bd": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.485Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.531Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.493Z" } } }, "4ecdaa59771417d8a6341e9feb60dbd9c4d4fbb10361d6cf230a66334329d458": { "32e97893f5bdae1c411c78d8f927f38c3f5f53f548071542f0aaa587e832cecb": { "jp": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.544Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.543Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.543Z" } } }, "5ed43729b9d1c584d6d2715ce2c8e0e8690a779f998a5295f954f2f562471776": { "1691e237ea64aacab998e397d87c92e5419d9695a9c24f1829f61653d169f1f3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.442Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.471Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.504Z" } } }, "6b19fbc50a3d75e95082802f1b3acf6a3fdda3ff18cd375f0468fb5136f2256d": { "3dcab33a3b2dc5934e1b739f426935f58ec2cc8e37d9a43754b1941d524c7eb7": { "ru": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.576Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.526Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.574Z" } } }, "7043bd98baa35080107f5165fe9bbec5ef39eb9956052fa0c10ef9ac22039a33": { "e6b73b30c4502fd5f9cd04636be35210ae5ea65dc8343c3daaa83eba16905924": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.486Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.497Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.484Z" } } }, "73cc61c275b13e314a195a2bcdc4cbfb3fba91139f9fd1bffb19f48a659d4e6a": { "190e7c7b34bba92cb96c18d30898280711152aa225a02af84331070d834800de": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.536Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.483Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.498Z" } } }, "7478bdb164a78a0066fd05a6a86be0fa7a2ddd64b6f73b9baf2265c59d70f4c4": { "e97df2367ee337a5ad2b8ce514b44485caf7b24462a66eac4a3d178503301830": { "jp": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.504Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.471Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.500Z" } } }, "789c0931dffcacd3c5e4bd954c1cc49f734d35378bd8d9f099bac0b7d7de0017": { "58519a4d43db394ea6d5c15ae1e4f7bfc823bcba6a23e04e1f1b0fc5aea36241": { "jp": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.544Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.544Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.542Z" } } }, "85409384bc3d4ff1f1449d76a33ced011be9773bdbf0758e6975a6dbd1ee1dae": { "1fee80d8af00c415d442c78b9ad825b9a0656bc47f1eb00d9ac9cec8430f1454": { "jp": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.472Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.500Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.493Z" } } }, "940bcfdd1a4ad18a6a6ccd9181dfd460e21675b41985028b535c556f22904357": { "8379073d04e59c3c4b33a28508240fa2ad889504e267a63230a17f0b31b60377": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.537Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.541Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.183Z" + "updatedAt": "2025-11-29T02:45:02.542Z" } } }, "ad44e0a653a277028da523b8bb2ede38b5fb8ae3edb129aec78609264961e45b": { "c58dfcddfe9b317538f8fc75e87174efab26fa62ab436b5d3a1921bdcdb71dcc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.442Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.472Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.490Z" } } }, "b0371f0c5ed81dd8c1a94c3d4fbb5068eda546a915ea97e900025b7967fdc506": { "1adc889763f86e0775ccdc2cb7db8ac95b53182b5f48d36f86a8daf7373c5e8a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.175Z" + "updatedAt": "2025-11-29T02:45:02.439Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.065Z" + "updatedAt": "2025-11-29T02:45:02.496Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.470Z" } } }, "c720ce0e77810fdc639cfe83c2df1fe9c3d97ef4dd59cba6540e1d9e354f6866": { "3f956529d37242046b0834f1c686e59dd0dda8c1b7de96710b47b1ab8e5544f6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.489Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.181Z" + "updatedAt": "2025-11-29T02:45:02.474Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.476Z" } } }, "dfd805b622edd8955d58dd44846aeefbda562b1c575f0740533a458f2478f495": { "c61769f8b34a280fa8e6d8215850f12fe517dd969c26c4527ce9543b9b4052d6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.467Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.064Z" + "updatedAt": "2025-11-29T02:45:02.488Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.063Z" + "updatedAt": "2025-11-29T02:45:02.468Z" } } }, "f7fa9ae5c1d4a87ba39f7e52c396a41963265a407116a9be68d7fbee6e43fa14": { "bfc41c969cf12203663097a7aab28d304963f4e8f2c4310d76f5d5efac423d0a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.506Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.507Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.066Z" + "updatedAt": "2025-11-29T02:45:02.505Z" } } }, "03b5ecbbf39334e6da0c384a22e5e44c1c2d4e7293956c81e85ebc5a1f9684da": { "a8ec8b1cfed8dd821d7646fedd89af692c1d5c39ff7e6c8263486a44277b6811": { "jp": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.524Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.529Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.578Z" } } }, "0b126951a3c780c939a55fe4567f097c8702408c235f214c5763699ad6daaca4": { "5b529866221693a79922a1408a19f5b678c1f0fe4b7ca31e7401ad0a4ce64dfa": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.562Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.613Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.609Z" } } }, "0eafecab32cbe234424b7ea9a0db39d81bfbd85c2d891597fa1309dac8735c8a": { "94efd6e0e379a3b02b71963dbf0699cd5c5ab603e5cbabbb278630c8bc3eed6e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.577Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.579Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.578Z" } } }, @@ -17613,395 +18854,403 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.187Z" } + }, + "21ab0993ec46252ab7b40a1b418b9c04325c81c889a8af72daa16bc54b1f51e6": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.376Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.382Z" + } } }, "22a6c0463fdb5f5bd56c1f342f979b7e0fbc638e39a22abae139379b580611b6": { "c126bede64139b7c6ab143d42c036651e266197fad3b70012de0b058cfc8a7b4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.607Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.563Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.605Z" } } }, "2f0ce2fe6b5d1318ca2c2c11f3ca3100561f2c3b056eac0c92885f76ad381df8": { "22f366f08d6beb4fd69cd03348a69d6ad0fa2634f22a96d663380fcc3e61900c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.521Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.527Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.558Z" } } }, "34bd47b9631a90da1872337df6289e0170c67b0cdd5f5773b7367e05d2dcfe48": { "7bea2cf57bd47e48dbaa0fb6eb99c5614d61a80b75f4b14d7d22036a5315b2a2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.612Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.573Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.522Z" } } }, "3fae7c4513dfdc82bcd5d84b957baba453f6cf9cb2c3086a653e40e66ecab9e5": { "ebb4c00f401d9fc73b63d71739322aba770f129d6784c881ec5e9cd702ebc982": { "jp": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.565Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.519Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.574Z" } } }, "8681a5dfe4cb1dc88d34f905cd6f0b880732c556d84f4c6c1a78c2a42a1e2e94": { "937c3315f8641ae220b02e6527a850efc428a4de748f9fc10c3b23118f915818": { "jp": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.524Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.528Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.525Z" } } }, "9679382c066536c0e977c5bada387315bb3174921875fc2375dab0f8ecb14a9b": { "775c06f4143e15814d67624ccd103ecbff901762e4be69292f9800d11986493a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.561Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.523Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.510Z" } } }, "9f914435087a98e271276ebb87f255c29380082ebf766e89899a246c457e4677": { "71530532e2635eadb067e7bfc1e67c37d37113e6474b6d00295249b91f5e556d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.529Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.497Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.532Z" } } }, "b5043154caba774e4d5afd58609e8705791d168d7b610c441a9f5eb0c01aebe8": { "8640bb0e91d0ce2469cf06735ac41d782b10893d26d5a5e4bdd88f4ddcf19c10": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.534Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.539Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.538Z" } } }, "b6b46b2ddce58f83297d4fd3e22a20c0689c8846b02b00d6c901ad29353143df": { "6526c7597b3e43dfe18fbc51f8dfea10476408a65acfc8c77d19c20114264de2": { "jp": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.484Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.532Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.487Z" } } }, "b760d26fdf8b09ae16032e0dbdd66a6e812e5b85cfc1a2dce387a41c031415a5": { "2a83ac2cbaf9b2ed36fecb623007bef63f6aaaf537e37429095c3057b999a156": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.534Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.483Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.475Z" } } }, "c94404af6396786f2d99e4b9e86fe62f37fba23be9fb0992cb4462421350617d": { "8e9c8e608b5e9c9eb4f01785fa62ca818e1a1957a5723d6cb412ed71f639a50b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.536Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.535Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.182Z" + "updatedAt": "2025-11-29T02:45:02.539Z" } } }, "cb7281a29c8577f9362237b726ab73efa4133f66aa1f532e94603029a6608325": { "e7e9ff403010f7419e6fe70d3329c7fb4d95f62d59d52fda8025ee90af8ad89c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.538Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.068Z" + "updatedAt": "2025-11-29T02:45:02.480Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.531Z" } } }, "cdf00c31e8da5ad17f2b40732cf7e7baf65150deaf7488eac143f7201d1dfb3e": { "3c8db57986756c0b913b89d2204dd19e77508a68267dc6a6d737df290161badc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.070Z" + "updatedAt": "2025-11-29T02:45:02.533Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.486Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.069Z" + "updatedAt": "2025-11-29T02:45:02.495Z" } } }, "d28f5c5276140aee0909af043384a73dc6d1e54e307092d06f03528d2b1110ec": { "c4f358e96fb5460080efb17e46f53d378939fef04b5fcad4e3e2c5a580a10128": { "jp": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.568Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.558Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.525Z" } } }, "04a08061427f75ae80f6c5be1bc33f6ed46cb17ac49c325b49ad3ed082b48721": { "8c2b821e3c5410720085eae977687f3169e4a39395d1aed6e45d331e39dc20b7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.587Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.594Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.635Z" } } }, "0f4329370fc5999889692e3e2374c65bf3f4dd5e8903e64957d654e1c712ee1e": { "87fcfd05b5f0e870d641b6800c171abf3d47bc7484fb7612f4151d70caaaee3c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.580Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.646Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.642Z" } } }, "100c02e77cc38427381d9c58741ebe9d9d8964c870d4cbb14624da2f386e6691": { "2d845a508a5f777e5f61b8dae330312410e821c6f517150d000bebfbc18e03df": { "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.641Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.601Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.582Z" } } }, "1680a9db2b149a398008cc3048b13dba799f74c5bfd3549470992ac1fdd41eea": { "2b8b81210547bd248aa80daed1df50ad236049f83eec7fed484a31e64906811f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.571Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.545Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.564Z" } } }, "1b89e2e1ad09ff845cbc6d24f7a759d61540214cea8a5c79bc2e68f266ebcbba": { "9d8c96f15a9c91e38b4c55448e86a206752b8e56970d31964de0de00beac4133": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.565Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.560Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.569Z" } } }, "25332a58ba046cb340c206ff61639fed4457a1aad56ffaa7b53917205f1bb761": { "ca54f12c897481de5b60e4f4170eccc7217a2e000c56dcbfd023eac144ae760c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.550Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.570Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.606Z" } } }, "45912b7cfa3611f18ba43d31b4bf270d57e4bcee3fdf2ac5e2ff6ded3b672601": { "25bd45fdbb02d82cf7af7820d3acc7ccf1701c6afe3cfae317a6b4ac9289a67d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.648Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.647Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.644Z" } } }, "5a251aa88d6ebebbfbc12913926476ff0da32b3862d705b6ecb28ea2c559b45f": { "b32b63bf76eb2d854a947bc3926ad7d875cc3ed3eeec677de22a5a760014a32d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.572Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.604Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.605Z" } } }, "696be7be6ffadd8471bfb91d7ba6ec45956dc7e449f3fc81dbaa6fa67d66b3be": { "8aa635a63a82ddcda9a254960f313fdd8f129e472d9fe8d3e6dc10d1b38c37ad": { "jp": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.520Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.577Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.527Z" } } }, "79e7241d6edd82b0dc1989f7d3211668c2f24f997b5fb74c55c6db437b7da25e": { "be2734886fbef14228e09151309f47d77c7dc82d6b8d68b9d3f8b6dedeaa8944": { "jp": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.521Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.520Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.561Z" } } }, "7aca79eee9aaf67fab4127f69bfa7778f63bc7a7f6b384bee18e809c672f7b49": { "55febc4e35972c34cb1792867e0fc3cfea4841faadf9de0e30f4502a613b8363": { "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.613Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.611Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.555Z" } } }, "92f16152dca7a77dde912f0a5b22ce16b22c2dc499873bbedb28221aa56e8739": { "f3fafaf3dde2049ce02a32a17ef225150db00d0562e505ad5431a06ed8974f2b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.553Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.547Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.607Z" } } }, "972010f567fed406e9bc7e9fa61c22b7128c4779998665424ab71c46338d1f3e": { "ae02d48d7b29f026ead3f4af508a4e2b3a97657cb5051628dcbbee9111248f7f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.567Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.564Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.580Z" } } }, "a15ab08919f6acfa97670cff9afca686c2351120dfd9d4f8deb2b45ddb99aa0a": { "d344fdb9b77fe64b9863b88b7aea7e3a8e4c7d7db3d3d7a7d7584b626a3c8054": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.567Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.604Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.554Z" } } }, "bb1dea393979951d316dea0be45235c346fe0c28cfe6756a5876f4804290c7e3": { "d3ecf8e3f0da56d9ba8034a953040427b08dc7fa1c165a2173308415b8a6d17e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.568Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.566Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.548Z" } } }, "cdb1f009589e1e0b485965e6b6f62f110d284ec9f225d0eb9717cf9f54e381c0": { "694473bb486e1e21cb8814dc53f5204436b1e5ffbd3f851984bd46f00c011179": { "jp": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.509Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.067Z" + "updatedAt": "2025-11-29T02:45:02.511Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.560Z" } } }, @@ -18016,1435 +19265,1443 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.188Z" } + }, + "7f17768c7754fe62726af95719e525e92c0e64ec5573a51db338fa863d1513be": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.375Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.385Z" + } } }, "d7c9bac812afb3a149a06c61bd4d64e33efbdacc006619f813e625307caa403f": { "bdd5ad8ff2c6c4cbf81696dcd7cf80196be279d10a61a61d0f45caee15d90df1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.570Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.610Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.608Z" } } }, "dd8bec416e1a990e1a7ef46ce2f7761b51432155f4c149641fdc484fffcbe786": { "e43ff80310727083fa06482849132d96578ddd46a8478a49dd3bf42b62882609": { "jp": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.549Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.575Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.605Z" } } }, "de693811d680fc3f30e32c9bc40614fb35f73f55847f45a16264e614a65d74cd": { "8fa72ae7500048bac519db43150657d9500e969b9167f548ec14e8f2a73052c7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.188Z" + "updatedAt": "2025-11-29T02:45:02.557Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.575Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.189Z" + "updatedAt": "2025-11-29T02:45:02.576Z" } } }, "f05e54b97f0cc26e7e6d19993081fe3b3e87410932334968bcda276a9ed28bd3": { "1d9ae46b239c5f237c2f10a2d4e4c6dbc9261c9c864accb4c80b847fe59481d8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.554Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.550Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.573Z" } } }, "fe5b39e56c19c291226f7e3197f67720f7eec98c8343fadf7b1a283589869ee7": { "afa11621c4420fe8e64e6a032e92ea848928d0c35428ff0c7a1b50f7215c04fe": { "jp": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.522Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.187Z" + "updatedAt": "2025-11-29T02:45:02.526Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.186Z" + "updatedAt": "2025-11-29T02:45:02.519Z" } } }, "010d68e65065006bb03ec5afd6da3fb00d5d932dc58d86d356a1fb32041700a1": { "097920bd8ae55b0c4c40422f164718639bf789af17784fc7d268a39285332660": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.592Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.639Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.592Z" } } }, "194312689f754af1eadafa36fb316871d927e7555a7e9237115b13fdf9c16217": { "efdd19891bd36c4b5ee32e3469c4609b62a971eec1305634c7e49ed5d594e5f0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.601Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.596Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.645Z" } } }, "1d9872faa89c7d85b9aedea5b9a72b7f79022036a883f0d76368ba0aab461711": { "70eda1446c7a201ec8f6c37abb74c39a9535a96ae3e057af6538915558876b9a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.584Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.597Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.602Z" } } }, "20fbec5fbf3b5f7168ad991d57410b2a6e132fb9884af790cd2c4a29f361d02f": { "36ab0d9536cb78b918f577f351ad01da73a11122ce416a9654035e7dd9a193bd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.636Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.621Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.675Z" } } }, "286853d39a677e8828ecbe663218f27fedd5bf2bf0e04f6a0845b378f6e8eb8f": { "b384e9d652969f7c44b75186494dd5743f6f7d29a2d07cdc6516f906170b8ecf": { "jp": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.632Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.586Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.583Z" } } }, "31135bad715065dcea06e31337e3a5dd947f27dc411676ba95164d339409a83d": { "763ca58dfeaadfb6457a37642666f8a6557e78cf6969b41e8b1c31735f7e55f1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.672Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.674Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.641Z" } } }, "3d5ae5ca94ad055105e113940b4e5f4f01c26351d5e0aa85b01fb3569699f7c7": { "db7ea4892aba1694aea64f46778e44e4d3a93c6f1d8d5290b4d72c844116181b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.633Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.601Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.590Z" } } }, "42c6ee1d7586b75ddf294b270cd91e6cbfc04990b03c458263060339691f65f0": { "e28070dd3b8f9c8e1de1f84e6213088ded4997089a0463fdced52aa0d7126bee": { "jp": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.634Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.638Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.672Z" } } }, "957a8d1238fb98455672f68cf73445d00c58150afae706f656904ea7f56bbef7": { "438d8f6bebdf4c4f748f67bb045a037db4fe70bfbe607e05bf05fab5e60702e8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.589Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.596Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.633Z" } } }, "bddea4d6e2b142218cf0aa18075b105f560306487a43f98ae93666cc5b0a2088": { "d82b30f533151c915ffd2fccf00cb93c7247a81a9af41c32c0b6ee0a941f1dc4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.548Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.610Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.610Z" } } }, "bf43a73f5fb45ab9aa1813ec5b3c6567e2f43085622a3981fc47bbafb9f28c10": { "e5f80b1293069b81103b1bd7abde9c4afd1e877bec64781bf8b20adfa5b92acd": { "jp": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.551Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.556Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.184Z" + "updatedAt": "2025-11-29T02:45:02.546Z" } } }, "c2092e34e0d63b8d13f5c437a2c60457a006bad8bb89baf8c2cc3dceafc6ec29": { "afa4682df8ae8c2d39481ae157b1d008ea8cf2cf75aa79ffcfdf3cacb4d9b0be": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.563Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.552Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.557Z" } } }, "cf7f50d7a1e362e6ebac5f8205b53d0c8eb6dd0efeecc010f23b8d8f09ea8f80": { "b7f62ebe9c2d110ae4ae2cca482b48cb6a82bf22cf7a6a11933cd85ee6309d22": { "jp": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.555Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.556Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.559Z" } } }, "d1838a9b36c0c0fbc1be851ae978af65ba7e34ab07c37daf5e5c0c741129fd76": { "14af3666e0c05efc3ffcab87b38768b94b93945123edbdb09cb8537e7a7d07b0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.566Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.611Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.572Z" } } }, "d488e5566dd6bf95742db0d7525010310bd38f5971c4a87992a3ec793feba8bf": { "5ba7a81bc990b2456dc8374342d01a7253db45b5183ee93be9b51553586efb4f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.600Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.603Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.584Z" } } }, "dbd27b188ad3cd04691439c723e924796170d0bfdf59a9e9b53d90caca0178bd": { "80b5c91060724e755120c034531a62ece1e13c4c261ac38e2e448b4e2d0e61c2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.647Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.631Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.646Z" } } }, "df4069454374e8aa9593f9687f16b9e3b26d64e2b0082ac22a7123faaef82740": { "e7ed1c4a6adc17da8ad5806d7ebfbb340ba0839bd13951ceea09267bd14c0a6b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.593Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.645Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.637Z" } } }, "e1f18ff34031035a08fe64318b680c893d2d37fb3ac9d30c908d0671a1180f50": { "85e7c92ceca8e1da3949120488020e40a9d10af04a565222bb41223f27a16de2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.600Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.596Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.591Z" } } }, "e4102128c26bcb3f9ad172af76f46d964de749c24c132d5348f9ee3e3de5951e": { "9f2615fd10d6b26b0f5f878a17f58c2100fb6bca45e41b0b5783df222e6dc6e1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.191Z" + "updatedAt": "2025-11-29T02:45:02.559Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.552Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.192Z" + "updatedAt": "2025-11-29T02:45:02.562Z" } } }, "e6493689e0bffb010f12f340386981233dcc9a2f28df11fd9a6e6066d3c5ce8a": { "8de8a8317a3584199eab7b620cccbff20a6c44103452bed63f66cf645cda12ea": { "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.612Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.614Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.608Z" } } }, "f1b73f2ce3c7b6d1ca6f0f28439acb8cc45586fb6f3d1fda35224a8483871689": { "84181ea71df19456b8c88cf67e1c18c054443ce40152a17b3fe3d33911ecc651": { "jp": { - "updatedAt": "2025-11-26T07:24:12.194Z" + "updatedAt": "2025-11-29T02:45:02.608Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.193Z" + "updatedAt": "2025-11-29T02:45:02.571Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.551Z" } } }, "0060f21968d741d1a0f39b19ac2622ebb5065bdb709b03e138eef82e28e31244": { "2670f637399d04628da2e0f038d37565f781605423d4d054185eb0cd33613948": { "jp": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.619Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.630Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.623Z" } } }, "1e131dfad9800937839c4d2f0e5ef58daa6a99e44ee4ff5ea4e66de6069c7c37": { "f76c1f685c5d14375dafd9a42aa84e6f31aeb5b84b0d8c24a2915f02c875d4ca": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.652Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.668Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.709Z" } } }, "2149ec9c3299895bf0097e125705ba36a1e04efee5f43e59c08371caad0cfd45": { "349711e0368c3473e04141d6855c62a92897b88020143c2fd44659089f128368": { "jp": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.711Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.712Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.711Z" } } }, "2a714f7169c51c1804757b5577385bc512ba198c41b0cd228e98a66dc148abb9": { "47388640fbbd48eba401a20cf2754eced76dbed9147e6841f469e2f4acc14075": { "jp": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.675Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.671Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.628Z" } } }, "3f24f2c556bcea3bd4a8da649d898ac0d1aa590efbf76127ecbd252c8df9b55c": { "87fc99663ddeaf7b1d38d03a534b4d0b7cbb70edc9c3b460d5735be114f9f413": { "jp": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.616Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.625Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.617Z" } } }, "42ee6b1cf2d60dae66ca7799b6e3c96a470d6fdbdff801031e35cb9e1891dfdc": { "ab655d464095f3f0a801879f7e0058f71ddf7741b59f1ac855f58f9f7d807344": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.585Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.595Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.636Z" } } }, "46e9f56a57a3931558fcf69333139681d05b4c2f69040e2cfe7a939c976963f3": { "e1fe2166283accd68531dcb58d1682b96cdfd9ca452ab7df14ceb9a7623b7419": { "jp": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.671Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.677Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.635Z" } } }, "46ff568b059cec990fbf679bc4bed642abea08d09f7bafd4747a7036515b95cc": { "714391bd24db523bc05255d05254efcc0766f0f4b43e9f23aaaa7548eef953df": { "jp": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.643Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.599Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.598Z" } } }, "4fbdb5b1520dff0a17e0429f575aa6011097f81752684475262c7ae6aa200bed": { "1ecda987c93d49e1fca1c7c93d39044137cd955db9a36fbd10169f0b85cbdbe1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.629Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.643Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.615Z" } } }, "5083281bd3b547bf9df36adfb2bfba73c9e0cc795d0090fcfa111ce30996f661": { "b09f8b5ff58806fe3abfa9da3b343ccfd7b8e980a6c46bd43dc32927ebac6ce0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.623Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.620Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.677Z" } } }, "5b010922cc17b528fce9cb609955f868e53ad71f5f8622066d24f7b3953f893a": { "83759792792ece10deacdd4a65c5c3b089a7e420df7df362574464fe94fb9408": { "jp": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.657Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.658Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.707Z" } } }, "5c4315a7496585196dea68631d46489e99dab1c8daac61b452a0c580a509d21d": { "4c3fab4892c9a5c579a3017bb4cbe36c271aad9734d4760fecd5bc4ac75d16d6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.625Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.622Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.628Z" } } }, "857a5c3ed29e79d55112d1802865f308f93fcc1035cbad65451f1392ced56b55": { "daf1366f4d86b97aac48da95d72257524192b5104b5dcfd34230427de3762a51": { "jp": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.642Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.627Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.631Z" } } }, "8677ca6f754c9510b46dc0569151e8695270e1ddc3a7791067d3b1b9e5ed0ce4": { "daef99c2eee6c12072def84a2de12f54a7398d20df2b000023b0e91f2100e934": { "ru": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.632Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.624Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.614Z" } } }, "9056916609446b5b12baca0332da8e5e8ad117eb3017488e4c5391bf09af1c65": { "5c1ac19a6dd8304196f8b5c3c4538997259c7d50017642a246b97a60197a70c3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.594Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.634Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.198Z" + "updatedAt": "2025-11-29T02:45:02.603Z" } } }, "93878162d293d38a3f960218a0ee8b1904f199878f15fb0a11f80cc5c6b78ae4": { "38c7da17603cc8d822478a774e4a0851139aaaf988b5e6ac6aebd7c75546c08b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.589Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.581Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.599Z" } } }, "9dd17b4dded970a85df00496de13a74873211a7c3eabb2bfaf1670710eaff639": { "3c77cf690b82a05ac07374c03da339b16bb18f1f69cfa9c51ba296c56cc2f48f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.622Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.675Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.618Z" } } }, "c9c4898b83cd686a39de5d1507a5f2308fdf824b67d0f19322fe25b8230ae68e": { "81003853e9247deae604d51fc5acc18e581a4e0c4f0d79dae6b9207ceefe7142": { "jp": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.676Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.676Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.673Z" } } }, "dc429afc1c845ad436d31b61fb908e473d3a84f5a8919f5d78c6cc647e6e44b7": { "f94a5951c5c6355f3214aef3392f0e31f245f1c2a14bce98a45d190388085326": { "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.585Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.583Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.593Z" } } }, "e8b753d96adf0305cf90b9e579ac4cf927e2e7f187ad62582b9b9a11bab53b3c": { "d6af628ddd5106feb87f50888fafc6fb21ea322d5d658688b385daaa6e2bbc05": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.590Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.190Z" + "updatedAt": "2025-11-29T02:45:02.581Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.199Z" + "updatedAt": "2025-11-29T02:45:02.638Z" } } }, "eaa9435dae8d90063d0ef13fb0d0245e00f3c444e99fd608251e1fbdb283ad76": { "5671b5319cbc284bc1f4dff7b698d72202dcbb66b153aa004c508aa68e5dff04": { "jp": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.586Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.197Z" + "updatedAt": "2025-11-29T02:45:02.598Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.196Z" + "updatedAt": "2025-11-29T02:45:02.591Z" } } }, "0045741a471f4dac9a3db4c43669d28583bac040167b1d39d7e25215fcda5ccc": { "dab964b634db47350d340e0931ec7aea4b46dc1764c4d7c24c6cf164792b3f29": { "jp": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.687Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.693Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.694Z" } } }, "189388fe355c19cd463ff375adbd81bb8d731d323bbf7cf2cdbbc3058b2bd826": { "5ed3b23bcc3bc844b8a42267d9198f127c4ab515a87acd7da5858ed9dd6fe278": { "jp": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.666Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.661Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.705Z" } } }, "23c47eb4902870785fffe1e4baa6e41d6084e1f924e6ae197c27e7b51f843750": { "052d52be77d868d3d26620fa34155f9eb31b5090d664d799d412457b60c3f050": { "jp": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.657Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.660Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.707Z" } } }, "242658032c19f8878ea27fde8bfaf1c2d950073ef6e50d896370f00b777e974b": { "9dced94c1aa74f4a1989dad0844123eff9d336fc99be750b0bd645446ef2190c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.685Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.680Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.691Z" } } }, "302b0ecec3c1a1792dd0c359652c37057d430e76b3e96aa7c8afde8a7172dc09": { "3c7b42d588a09f60f2e03cc7401555f547ab89b4421ae45faee2a50a6b0b0401": { "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.747Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.692Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.700Z" } } }, "52bc7e8ec1f25b547908a73830b6d7664f88e3007b6ea89191268490da4b6c29": { "e4376d6532a2d24e4f86f129429881de208f3ea0ab1bcb5f5e31cb841a06df0e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.709Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.666Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.659Z" } } }, "55d7240c880120e92dc6163e0ae953ba2e5f00fe1352161637e7b7057888a3b6": { "d8020d4cb0f5381e78c97181bfa0e7bd2ff6585f606db5db616fcb0afaff7589": { "jp": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.688Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.698Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.695Z" } } }, "59b41159dfba51bfc26167978b1127378d106b8d443bfaa28a298294319587b0": { "c8e3b18d83b85dea1ed5df57b3bcb5d76702cc3807eb0d8ccc3a2a6bcd46acfc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.685Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.684Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.689Z" } } }, "5a412fdae8fb53a15204e66324bb2d0da4e638bc75ac56e67179382d206d7374": { "02f14c2f65f281503e41f11f04ee9f6cd6ab49c4babb7d84453226444e626ce3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.650Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.651Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.664Z" } } }, "65b791b7c4a125ca183cc9f15c013f5460cca336367cbe0b2dfc01f119a90d1c": { "1a1f03cbe833217e0e2c1ae7fd100d78ebbcc8c0657e571385e72c88889a8da5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.655Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.656Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.705Z" } } }, "6fbd798e9fc4be572840b3ebe3124e7c1982606aa96d7b42be53bd6c1ee9676b": { "123889af8c3d0c2ab264480c584493f0491363fd067fa94edea8459b1555318f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.662Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.667Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.669Z" } } }, "a44ac107c2f03ea1cfc68d15bea4e84005ab3111943ebc6245e22ba05bffe8e9": { "30eb0a47b3a70c9804063775a6d033975254804002f913220b776bebe7566da8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.621Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.644Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.626Z" } } }, "b8b5935e6157dbf3000442e0ae9da10c119186446dab9b0b6ba59ecd8e081b43": { "3aec0cecfebdb1bcc89b6b5e6d7edb63838928162cbed60f94e123b0001dc3e2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.706Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.679Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.678Z" } } }, "c2b6b4b09ba9a1b69a2623b9e76c0169f2800d8215a3a24ec9aaddb566e07410": { "a509a683d08d5f5fa0027e4566599afd99d8661f1932316929ed7b7f5f1434fc": { "jp": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.616Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.619Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.670Z" } } }, "c78d724ce19757f519a89ae81413bdcf8c707c62709608c1fcd90f8f2ad2737c": { "ed98e153a80901d835f37a02ef176c4789e69c4833533e0096f7181d92ddda23": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.652Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.662Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.654Z" } } }, "ca734035b219f4714c9e6c2cdca7a1904792cff5ed4cbd21e39a0c5b2a486565": { "73b78bfc9381e1ef4959ec2997ac7ae0499ef6be647ea0c493a48b57261785b7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.679Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.629Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.640Z" } } }, "d7e629dfded6aa789e7b13dbe976a72e204135dfeb9119292f63ce16cd39473c": { "995d171ddfcf778e23a9288af9f2f3b5372f8ce14a4ce8feb377503b79703cf2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.663Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.668Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.661Z" } } }, "da9d4e8b0bdf930b4854e634849ad3b851aaff67143620d95a9ae1e5cb3a7b9a": { "bbba21070424707e5a6f8591bd4bfaa20069a36dd6b196fdc7050d7a1ab8486f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.195Z" + "updatedAt": "2025-11-29T02:45:02.615Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.204Z" + "updatedAt": "2025-11-29T02:45:02.674Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.620Z" } } }, "e774d95a2c81c53102c61249027c7f00d0f3179aabfad8f71a51ddceb6505a11": { "e0d72b4c4c836c1bd36aac8338da95ae7abce2d57528db5e7d5f1ed3d95b6f29": { "jp": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.665Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.648Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.669Z" } } }, "eaef488a67183c3737450e1c070243954054aca5bcd96f3b4148d73f6a7399fa": { "893797365249d93ec499eaffe4b1ed5f848af3451b59dc62b1c2c0828602a016": { "jp": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.670Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.201Z" + "updatedAt": "2025-11-29T02:45:02.618Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.626Z" } } }, "ec49a0cee949d263027a7b97accd10ea82850898c06f8611df19e985e58a554b": { "33e16cb7d3af2bae2f39127844a9524539563891c9e3db379b8d508c23f9b634": { "jp": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.658Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.663Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.649Z" } } }, "ffb2e794247dc89ebed0e232b0ca7c0962e63c5651c684b4d99f74958eba032f": { "3e28ee25ce5b288bcfcc6aa247be220c6686ae678dc50aa107da3672ec9cea32": { "jp": { - "updatedAt": "2025-11-26T07:24:12.202Z" + "updatedAt": "2025-11-29T02:45:02.627Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.203Z" + "updatedAt": "2025-11-29T02:45:02.639Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.617Z" } } }, "10bf6a851bc722dc218ed84feeaf049930bd2d7b38be10d0175a4b45da4c9e3c": { "72a26e0ef3fe81a02e1eaba48c8ec2828431893b8e50ba8b3dd2152f58c16698": { "jp": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.748Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.746Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.687Z" } } }, "2ba4aedf1481fd714296b22477ae890f08dba4b0496e12c98e62fe2811b6431f": { "e6c19e03fd150258214beab57caf618b7ccc0baf4e6d85d9c67796cb3ea9fd44": { "jp": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.781Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.778Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.704Z" } } }, "2cbf8ac76941d9ddeefe32e9f176ff03397d09339a8d40eb2cfc57efa00fc1d7": { "2d3d7395ba3898aa08ea4bb981e7bffd7607a25fc091046d7a6a359bc9c589ba": { "jp": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.699Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.744Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.701Z" } } }, "2cf9993a309ce837e0def1fde3b9ec81b984bdc367d668342cfcfe3647301013": { "f44de4bedc5c963bcfdfb8f911d7420b96d114fbac92a40412a2594ce4bc5180": { "jp": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.697Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.751Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.699Z" } } }, "3682e2d45de97f6b173cd748b8b4d7583b7f1420f40557e91bf935dd09b009da": { "28eeefee37cae95ff6cae2142c3e8807b596db44875ceafb1b3e3c2b4f5b62be": { "jp": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.701Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.689Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.747Z" } } }, "49403ebf7c98c9603a561ef10166db22cbd8708cc533f76c0feedc9aabdcf4ff": { "512f607384640e8f1cbaf19b2b517930edc16a84b9a618f37f91116a4393bef7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.783Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.783Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.784Z" } } }, "4c4a469c4038db0bd30d547c74475eb77e6b3c4d4eb98a9b5406301541d45581": { "32eae8f070a25e27b3cb7b763fb46241c3e69525a2c4d2ba527136f413a778a2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.693Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.683Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.698Z" } } }, "5e529ee6f1c6b44d742cab16c2436b0f98d61cee3d67b6c243eb91fc94e5747a": { "b5eaa7df44d170d16be268ccac271b07809b8f738fe7f6bc1658432e3f8af2ad": { "ru": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.782Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.741Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.777Z" } } }, "69dc87a0a0efcdc9ce162824232e0caf45af3973a79857510730075407dab81b": { "f55102d7e2ca214c7f9f0866a2bb860df9999592d3a40c6d9b97a2ca5a47cf98": { "jp": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.745Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.702Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.751Z" } } }, "7cf646c7ec8330a693b4b1f30fc05c3ef68f7af5200b4c3d5be55f5e6c627d12": { "b392f20796bafccc3efe1e80f4e6ac3a7db083acc7209c5e540ddcfe853a6127": { "jp": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.682Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.686Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.750Z" } } }, "8b692c2ad787a446b25292433cebf4bef12b92c8e1c334682420d14be45948e3": { "59296f60723eaca7cd5a35c2a97534cb75c9c73d8715867db0a0e547de415157": { "jp": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.700Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.686Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.744Z" } } }, "92ec8f6b08ecfb56cf3d8225b5aff3170cfbbd0aa5775ef3532b3a6f5090f16a": { "24d1012de894e965ee2332b480daaca127319bc8cedb17d9ff8c5d9d4b57de00": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.655Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.656Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.659Z" } } }, "9f34b6230075d04ee88d715b8efa4b4287ac5ef974d0bc4c4940ad96532f8fcc": { "8527ee18d786491e874ba6c6733def703ace3ed743538e924d577e8b8cf2ded0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.660Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.664Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.653Z" } } }, "9f6597744edd6252f669f69c58d2636f8aa9a6b09dbc8b995f9479c4221e22e7": { "308c3f9e814a2ad27043440f48438bae8864dd4493497ab0a517cc656aa82356": { "jp": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.710Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.712Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.667Z" } } }, "ac52e240a096d2b15ce8bfe0c48a2efac10eda017b425c2339c5001cfcb72318": { "56334f7f1fa03f9b3a42096ca5749c43c65a9573954fa56e40e339606f36c1c8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.651Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.653Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.650Z" } } }, "ac7c945a9a70e136f7bf663953e5789b51065cda16bb4013fffa3f1f5633a518": { "79c8e3c46a6ede7e07368f66bfdc60525ced4d42f656a8f57a26ee701ec28b66": { "jp": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.714Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.713Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.713Z" } } }, "c36157e661a0ed678a48034a7b5806bdd2feedb466d46088c035d8bde2fd79e9": { "4b9ecaa4510afe985e77b7c0bf367ca64dcfa7463bb738f45d328855c7efc166": { "jp": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.681Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.682Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.744Z" } } }, "c3d15c85d4784a496cd8acb62a731024d5bb9915807be3522653ec7b1167d18a": { "608f13e19408e1adf4e6688ec8886b26bf677b304247727063c881c2d33f3968": { "jp": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.710Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.207Z" + "updatedAt": "2025-11-29T02:45:02.660Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.706Z" } } }, "cd116d178423eaa55d4970d5d78d398dc1e5099ee13c6221d781e9ee5978b899": { "ec13b6563341c4b7d66f4d675ef48acbc1e40f169c0016ceecaeff7982621eca": { "jp": { - "updatedAt": "2025-11-26T07:24:12.210Z" + "updatedAt": "2025-11-29T02:45:02.708Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.208Z" + "updatedAt": "2025-11-29T02:45:02.665Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.209Z" + "updatedAt": "2025-11-29T02:45:02.705Z" } } }, "d0d17f6390066626b3cd9b1b5cf3bfbe37d88dad9a6142c1db99eeec90102fa3": { "f10f076ae99bcca2c49fc911b738e76676d074aa2444ae614ac526d5065f04f7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.690Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.694Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.684Z" } } }, "f12a63823b74d3b2b90d31871ee06bcf19ba66effba17bcc94c800ce464bb39c": { "5f9e4fad6300cfb262a29845e8e0aaa91d2938f09671d81c5ae2b2c69f9a6483": { "jp": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.648Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.200Z" + "updatedAt": "2025-11-29T02:45:02.649Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.206Z" + "updatedAt": "2025-11-29T02:45:02.654Z" } } }, "15e69bdeb4774e041a333e57689381522781cd859797d0c321068053bd1ac55d": { "ecfdec0409be257ba876146227e2e778ae5f272c3aa56e2fbc1cacb35dd43ca1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.739Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.738Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.740Z" } } }, "2441b704f1648bc3443c9b054ec8854f3764cbbd77801b8747d10f0c1380e055": { "8946d488f9c46e6c14fad461ca002a664b5a2d6561da01977d53a7c95d31e4bc": { "jp": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.813Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.819Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.818Z" } } }, "253c517a16655bd1af2910bca26a946ec5b5257507a84e5c1083bc68edcbaaae": { "383175d865a3e8e5eeeec2ad520a6706a7fe906490a2365a6c124bbbd35fbaea": { "jp": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.719Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.722Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.719Z" } } }, "2c3512a703d975c2b75e7502a141cd8a3e8b086796e9dd5b92d66f1f2a58358c": { "f1c375550607f160ff41977c4e39aad3343f7094f427e196bc55d8e72c22aed3": { "jp": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.767Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.729Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.736Z" } } }, "371cb4852709d9ca0ffc244925c1336472d7b3607e49eb600409ac2634d29c9d": { "2c08ba9df01012e99f6db6d87ed3274138d3991bb7ef1df26cf943bbe938c83c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.761Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.772Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.767Z" } } }, "38065e7c3b022c9edd666529a176fb393cfb28490dd15161ec6ac71c2d9529db": { "35e6467692a1dada24e738d0c85e6530cad77f3c956b13d30d9734eec88985a5": { "jp": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.727Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.731Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.729Z" } } }, "3c1dbc013406b1c31a215c47a6b9edb7f3dcaf68974dc2c38989fd26dd392af4": { "54d4adf41787f75b127c52923ea0abbe3e269714267d20e9e3f8f38afabbaf56": { "jp": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.715Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.716Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.717Z" } } }, "3d0840c01249868fda2bd1e95b3f042cdf2c618bd34004df654106ee3d7fe77b": { "abd6f88511214360a8b3d4a7acb1e68208916aae6edb5e22025418320d437381": { "jp": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.820Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.819Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.815Z" } } }, "3eb17266fde17cf983c1426830939c4712a727fd7eeca3116f2fe348d7489f01": { "d7d5ceeef5f34571ef1e4827cc0966f80aabd85dc08e22be3a3583aa8cbe8a2f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.730Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.734Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.732Z" } } }, "6bf7c7b51f6adc00dec7d08e30d4d16d28e682b5d83a2a9112cfe37d49b6b1ad": { "3faae72ad8b1f70ba0b49e66e434c0ca46525d70f145c05758337bee07817ae9": { "jp": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.690Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.703Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.212Z" + "updatedAt": "2025-11-29T02:45:02.688Z" } } }, "84bcc067be4c969ca78c33fa50f4efff4f2a2daacca3a415c5c86d0fceedd5ac": { "2eb8e19e71aa05266f701be373a387f43f2c6751db4a43fdf67169c2efcd862a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.749Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.697Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.214Z" + "updatedAt": "2025-11-29T02:45:02.696Z" } } }, "85d48d85dd722310026bcee5e22617e344f2aacd9f8e9ec67d816fdb2703a37e": { "92cdab1f6b712fe93f35828375006e26f4c9671ddb601b08780bfafa9a16e196": { "jp": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.702Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.691Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.749Z" } } }, "8d8defb12045ea6e4b617d20e5212582181c730d58236e675147eba18be53d95": { "c53f9e7ae5db8452601cd25c2b2d9ef7eb21620b4522dce992bc50fa2ca137a0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.720Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.742Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.728Z" } } }, "a5236951d982490ee0af310dad8356d6d6153f403e1ee58f4ce2f1c0eda6a81a": { "c1b636cd594663b0ead8b055a758d770ff99552ec72b5c80bc4f4e7f722236c1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.779Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.726Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.736Z" } } }, "d4c8c149a2085ffd9c567e330ccc163bc309990242e7b28d9b404761f935ba4e": { "37cd2110dc9673e6ecc3c129fd27e5e27a8e403857f4a2d17738870cab29a747": { "jp": { - "updatedAt": "2025-11-26T07:24:12.213Z" + "updatedAt": "2025-11-29T02:45:02.692Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.746Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.748Z" } } }, "d9be63b990bb973f2145b0fede5008f532e3efe16cc74b19670e7c30fb33cce3": { "6520ef784c8cb65030b31629babb751b59c90c4785704dd342ccc7196be05ee1": { "jp": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.735Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.731Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.741Z" } } }, "eb49ba497d8db0f37c1298e8ea9f8be1b244b82d159157e8ede112df8f3c919d": { "4b16adf3d0e0aeab42ce3ab01c36acb9cff5de72d7b2802148d15353f359ea9b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.737Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.737Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.735Z" } } }, "ed2113745ac93661c6152589c4303163561a52fecfcb50853a532d0c4d3c4c8c": { "91a36f6307074f27f0253a1a697372b4dbbadd48aaa0cb2381adb6ffad7ec3ee": { "jp": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.739Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.742Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.738Z" } } }, "f7ba33421a28aa3de7f23177b5e40153a4f0e0efc37a2106a3e8b5708fe45005": { "4211afcb557ca12ed79b2828ba3000b6bfc93501ef7266a7012e6f73ca63a27b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.733Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.777Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.780Z" } } }, "fae26c9194eff01a95214ca36a03f887f3e266e90a64a4b894ad55f02c179bb2": { "7386d025ae2748ca0b87ecef00be245390faaaae8fa265f80c33e3480d854a49": { "jp": { - "updatedAt": "2025-11-26T07:24:12.205Z" + "updatedAt": "2025-11-29T02:45:02.681Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.215Z" + "updatedAt": "2025-11-29T02:45:02.703Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.750Z" } } }, @@ -19459,837 +20716,845 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.213Z" } + }, + "ef588f2b6385c55726c920e57be588ac227d274976872debd444eae9c0c673b4": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.383Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.387Z" + } } }, "04c615906de14bff138af4cdd85c3c07b4fc5433296761dca010e8ef60f78e93": { "91810a26e7bbbe9ffcd2f092006cc98930eec1fb41bd4802d4297bf1f45413c7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.758Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.813Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.763Z" } } }, "1580309aeb8bf89a02431ce4e3958695fd0114d89488a627aab1a37097044adc": { "a04bc210be5bcbbe776786b33eff75770784c182f110822abfb00ecf17ff032d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.817Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.771Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.812Z" } } }, "176150c0e3d077975a3bf364d1abf67e535d6c7aead2f176b61c34aca79abd59": { "844838ff96f065aabb06386cc366cf66f183135f983db2d969bbf61b47c89398": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.760Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.757Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.755Z" } } }, "1a55a8d8cd9d21c74eaa692dca8aac6491f16ba3aee28f43616128e2d9ef200b": { "da55650acb4be1e891fe2ae5f1756740a01821cd992f3a8ca4695951fa27e52c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.763Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.760Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.768Z" } } }, "1e61b3b890446ac615cfca4d43e26692a7bc7544426233b862918b5d3fb722da": { "68327a573af2128ef9f8b75c6d3764adaef0d6d6a2518cca36e25acebd3d72ff": { "jp": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.775Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.774Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.770Z" } } }, "2365f342aa73537207eea12c5ea5e59b84982495f018fb65d762d8ced77d7432": { "303a2bb1adcbfc7e719c1aac71a6de6454f8a1ba771cf607483f97b277db1bd4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.758Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.756Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.766Z" } } }, "361b5b1d32de2ebb3e52e8460adeb4b22ec4bc8ca04ceb0e717fedc703a31195": { "10b62158d3216eb8065dd2ff7515e8754275c4c7f5c6d4eed8d2ede3b37286ee": { "zh": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.806Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.809Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.808Z" } } }, "3e3f9cdd02598c16b281b93fb32c30b1be85298c6b705aa31bfbce0e5880e103": { "e9242354e112109aceb1f980cb5bd9997a81807b4b2b9ad51d2e395d6925d743": { "jp": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.753Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.765Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.764Z" } } }, "47db68ab348b969e40c4783275dbc11e1b7c3f0d1e0f7993066d41cd80abc360": { "eb30b9830f751c6d73c63c4e71376e8e862a1b79d67ead319e3a93512cfb332c": { "jp": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.815Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.812Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.811Z" } } }, "49e360371f0bc0d697298f4470438952e521fabefd1b9e98218955be3cdbbcc0": { "974e376db0d1f6bc3a3c2778b18c785b8cbb420855a07c1b3d0cfb100fdf6562": { "jp": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.788Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.796Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.788Z" } } }, "58cd3f4391882ce670046b8d82826c3c127fcee3b6aa2afc15ff717cd3d10d71": { "5015c123581af2b4d332b12ea65e8e6ccfdf0a8a5c76d9fab3a9a30aedfe8767": { "jp": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.775Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.810Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.770Z" } } }, "5d6ff265e282770018f2a3801b1d623cdca059cd587edf9408ad75b7f0427f29": { "7bf23f00d17d99986e4f0927c2dad27c8d9b95293b0f84a3bd9420e9a2cd90c4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.733Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.730Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.721Z" } } }, "92e00a40688842f014f868586a36e069a52b6ebff0afa9668aa0116030f149f7": { "507162d0c5f858cea0fe1b5f4cfb599166143072817567489682c950f1313b5a": { "jp": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.719Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.722Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.717Z" } } }, "94305f7921348204002c5fceee250d8a329b22e97297f5de432297ca6b6ce190": { "68e6800c1c85abed9956b13cc6c1349b8178fe6cfb23ebcc8aa5475efd99f8e7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.755Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.814Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.772Z" } } }, "978146b52bf1385e45bd326ef044217c2dcdc8bb47040c12f8ac16274fa8addc": { "229b20a3b9f2e01d63cbf0aa22d459b44b4535cff9593d53b6edbfdd28847fdf": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.760Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.763Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.768Z" } } }, "b65057e512e1d5ba2b482da61eb41e96a0451b3633379d8bfcd74a74bc5c5255": { "d590e32dca83cbf697fbc724c2b52de9f53b427e55f5e82add0e7c98c670b72f": { "jp": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.781Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.780Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.778Z" } } }, "bbc79010b259fcfbd187a6891a0f4fb7b780904c181f0266b6753f4d179bbd0b": { "9124cca07daf9271adc7984d01efad4c1a6d47441c45c6be540d3204e5502916": { "jp": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.718Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.718Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.715Z" } } }, "c04de4891f93a0ba91486fc9aaf76205c21818b034acf58a753695af7332b3ac": { "783554b75229a238156945270a3356288601a5016510ae7113ea4d4f746a89d9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.816Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.754Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.073Z" + "updatedAt": "2025-11-29T02:45:02.817Z" } } }, "c5ee15352746ad76714767dc88162427e77db4c02b35d0258b67bb1a35882ab6": { "1e07570b89f9d1753c7c6fa5c9dc7f96cd00626361968edca1ee15a898637fe7": { "jp": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.776Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.783Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.782Z" } } }, "c9f381cce8333661e63bd1e01d8c4f1774748ca4686351ffff148b88e9e703cb": { "e4a9139614a7f11d3b10e77e31631df6b358e364a358b51b7e9d35e161a62d0c": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.765Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.773Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.764Z" } } }, "decba6568d82bbae43bf10ae33288e0bb54460fab2d76fb910a5037c036d8b31": { "b3961ee327c6fafcf4999b1abd14b74444d3905528c75bc8bb8c2bfbefbe9765": { "jp": { - "updatedAt": "2025-11-26T07:24:12.211Z" + "updatedAt": "2025-11-29T02:45:02.716Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.218Z" + "updatedAt": "2025-11-29T02:45:02.721Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.720Z" } } }, "f8499afd2bca127eb328fcbbb1d86926a4b6ed99899c57bf912940e11e81fa53": { "57d37a6031f92bd82e315b49237fe134b84352ea376fc2fb6ae7f50d8a63cb03": { "jp": { - "updatedAt": "2025-11-26T07:24:12.220Z" + "updatedAt": "2025-11-29T02:45:02.740Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.221Z" + "updatedAt": "2025-11-29T02:45:02.743Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.219Z" + "updatedAt": "2025-11-29T02:45:02.734Z" } } }, "00801f2886d2097d3f3fd23c2495271df83abfb95d59a9c9a2b4a905b8ec2d19": { "20cf324bd963db14b9a1a4346dec4811329f6ebe733b3eeeaba7616399e4d20d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.843Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.841Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.915Z" } } }, "0d7f085589a701521498ae4f2032eff79402e3efaae1bf069e42f610cc1714dc": { "65b6c024a83d6653e55cb1503b9816b66a3ad761b629019961fe3f8f698afb45": { "jp": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.822Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.826Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.836Z" } } }, "1b24b02c3b8b44ef65014e1185ac74c302c13f1cd510990f907cbfb6af75565c": { "153f09d0dc6e1710e949f8df69bcf6dddffcd2f29e7b48e271192abe56431443": { "ru": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.829Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.825Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.831Z" } } }, "1d3ae6305b61a5daa4272a2fdf5bc89befcde6b3c3cd8ac506e835ebca98d2ce": { "7cfed78448288b1e3ce81098eb348b43d832571045d5f68b5c05727141c3c15b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.921Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.917Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.918Z" } } }, "221df8cc4bd4c59f72e569ef5c6a2072eeed57f90181a227e34cf111231768d7": { "c38114543f910f77d0865008910f7e9c6395ef18ca1ffab216e250ed274cc4f4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.842Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.833Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.825Z" } } }, "2dbf7fe23f006182359a9db8a0997fc25605a170bbf502414f10a4d0445f3741": { "a3d059702798e7975e6104e13702831f09dab10cf354c44b13f40788c8b697a6": { "jp": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.835Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.911Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.912Z" } } }, "36a66d817a53f3419052a70bb1815a864c606b97c1626029b4822b58ad82c762": { "3d820438e1d508017cfc5d486b3974a03a6f0475286a479dfda2cf575d825e99": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.804Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.804Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.806Z" } } }, "424e2c03bd385b0d797e5742bd8f37c67a6d1d9878707ee374ab10fc13b79f63": { "a39308aed08887cbbf1b7ddcfcc47a901be42991586b7b0c15366672b1a8486a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.858Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.860Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.857Z" } } }, "43e8a84fbf33b51194a80d037248d79de4e1436f50520586eff76e3d3f2af304": { "f19d15b264b03da92de17398ccc6c09d43af2b4d24b0b7c5e1a05393cd4b3fa6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.853Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.855Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.853Z" } } }, "519d5b1a64a00744511c1f3e47df4f483237ba323bcad90f4c2eca4ce9a37794": { "f9c93f24237acc26028d821a685b28dcc71dc3b5ef28ed3f611cd0074fd7d695": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.757Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.759Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.071Z" + "updatedAt": "2025-11-29T02:45:02.773Z" } } }, "595165a4c673965a825c2516944ed6da341d1802ba4af0d1f8e1442aba248fa8": { "8396ae84019ca44433161f57c91a29f40404e3a589100e8cca8e8000206607f9": { "jp": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.797Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.786Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.787Z" } } }, "7e455500c000c9cf2e608bee5ea8ceda40748f754e86eb2dfa6fb808fff46087": { "bad6198b79924e96476294bbd990cd527edc29dacccf3bc3408a2a70258e5f0b": { "jp": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.810Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.846Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.809Z" } } }, "976d169b47215323ef4cab38c850345f280e34b651c35ee7a506d07e901ec587": { "91662735bc3f121c2f531adc960066dfb766691e7210f186029e52bc32f80b4a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.796Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.798Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.802Z" } } }, "a2d877584716bec8ddf5f64a0ba5fd7a0a6b67f9077bed82dda87ee72bfffb8c": { "8d6d45dafb5a931c179b3f202896d1e34592ec42eecee9e2f9c96e83bc4cc999": { "jp": { - "updatedAt": "2025-11-26T07:24:12.222Z" + "updatedAt": "2025-11-29T02:45:02.755Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.072Z" + "updatedAt": "2025-11-29T02:45:02.774Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.771Z" } } }, "a5c7b243af8ea45f4cac1779bcbf974f63ad2778759dea05635eca542de84b9b": { "d7c29ef5219d22555b84953c119240e3967ba43e9caba2c80886d14046eb7fc2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.784Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.789Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.790Z" } } }, "d20916d14ade0ee04f39675be5d395d4a057b6b2238ab20a85bf425d1e48c431": { "1ba41582c1e8ebc8a0609ed6a4c503280d425de63584ec900b123ce79c518b7b": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.767Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.216Z" + "updatedAt": "2025-11-29T02:45:02.752Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.217Z" + "updatedAt": "2025-11-29T02:45:02.752Z" } } }, "e4ba3f71170ffd976d7ad1047d155c73155719b1d252f0fe0608a02ffa3d64ca": { "a6ee74f4a5fa3c471abd0d72cdd9151b4614ba229d109564ac3a2e5c5454bd4e": { "jp": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.789Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.786Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.852Z" } } }, "e7b858b48d1c9c70d32c523d9dc6357d0917ee69b16fa5c6a88fd2a2cfac0098": { "092cf9506a86a0643021a3bc1abcb0426387f5124df02aa60181da49a76114c0": { "jp": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.785Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.785Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.790Z" } } }, "eb1a1f01631b05bf1533ffd2c55c66414eb49c1741c154d4907afc8f73f7235f": { "9a41183439ccb685d921031462bb9652422322a842f20a13f989ee4825c98e54": { "jp": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.807Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.808Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.795Z" } } }, "ecc50ef743da07587f50e2e433c891d853c4145a43e14073bee65beca199ca9d": { "e3d9d895a670833c385d032550d1d2f2e8ecc66328713f84bde5f6eb645a9a70": { "jp": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.762Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.761Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.224Z" + "updatedAt": "2025-11-29T02:45:02.762Z" } } }, "f811cef1e60d3d76b1890136803c78a8a4f1f5c0702d5e269d8ea318cf5bc7b7": { "8ed2a0a54a6b4cc5249d9184642124cf15bfe670fcebd8151de225c2a95e77c4": { "jp": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.769Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.756Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.223Z" + "updatedAt": "2025-11-29T02:45:02.759Z" } } }, "037cf7beb5f9d2290c0e219686b1115e8e4b773c79f541f5c81f9a4989e58cd3": { "3f6353039db49376892bd891e326535ed8f03543ad08cc2ad5b7bbbe193ee94e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.833Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.840Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.837Z" } } }, "0a6b34520ca8168f8c366dbf6721239ffec9e0995481a49f17e32bfdf43182b3": { "d12d9428ec537b38678164b4a2d6a7eab105d1f3658778da83f05b64228fece8": { "jp": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.826Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.824Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.830Z" } } }, "391cd20c30f8013777a8a8300b0127fdc765340375b9fa4c319adee3b24ec843": { "c91f5ec1d83b0cec76d7a0b4857bf98e46315d814f9cad3887ee4296fdb30001": { "jp": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.915Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.919Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.917Z" } } }, "4fb36325d48e506203d8d75bcf6b70879d8bb4bd5ac0aef7b03cf1d784b85934": { "e592ec6dc8b770289b11562b8d28fce8a2ed7c9589b8caa85832638eef552890": { "jp": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.954Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.957Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.957Z" } } }, "54668b892baede02e5b6a5cbaf788773fafac8277e523ed65fc920f4ea6df2de": { "0163d4482566b616b6e411361068fbb4094a1c1d66cab5c5f906a2faf1fe96f8": { "jp": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.954Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.823Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.823Z" } } }, "59d8ec79d616c3429926d7db76bd740cbaafbbb1efd8c22b46480f65f400d6ed": { "a215229132a79542224dba0890d25487c4b0a3cc5a0109d4622445912b1ad347": { "jp": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.830Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.831Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.828Z" } } }, "5ad0090f8bb37d66ad4ec384fd8c183a6ce6d85bd5c293bdc484cc9f40bbfc3d": { "fa3251d9fbc086f42e5d133962432d1e0e3306745b593aa2bc755f7b16f5bfa2": { "jp": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.837Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.839Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.834Z" } } }, "67deff08df6c97036b3da071e7956e16555880aeb53c7d8ac63d1316e5f89993": { "8b19006f70430697684ec4194432408cb6d68b05965376bdeba185e83774be1d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.835Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.832Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.080Z" + "updatedAt": "2025-11-29T02:45:02.824Z" } } }, "72054126de2c0ba649ef4842d3a88e42bc8fbabd3ec579abd629308399d48364": { "f53eec1c24f726e22bbfdd53d757a2f052bbadb6e11837183028dab74cbef510": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.799Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.801Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.794Z" } } }, "79354c33a23d98f8b63fe6e965aef5d6b18cdc962e36d20a3b148d8cf335f86c": { "a1b7db6e0aac3869ff670ca64a57cc2cb592944192a99aea022777ca4d6ae73a": { "jp": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.853Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.851Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.854Z" } } }, "8ef8c9df9ddafcf602e81139aa8e6731772a4658d96021c4f2181a1d5e213669": { "bbc0b523bb0b92fbabe619f5570db6bf3895fcc93bc57a5e31d9f3b2110f036d": { "jp": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.859Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.859Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.857Z" } } }, "9a882460cbd2fdc9c5ff521d87a5f2d2b7ccd55f1ba81bfb3906e7ca923d1c1e": { "437e57c81c3f0872003cb47aa8df2359ae68ecc690d887ec26b6e38a740144f6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.804Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.803Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.805Z" } } }, "ad780b9bfd73ed606b7968549e04e8b3334085724088340ad05f2447559d540f": { "2bddef7ed07c45258897c9370efaa505180d67c313bb2d16ef2c830e5636aa00": { "jp": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.829Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.827Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.793Z" } } }, "ae79c700aca5153218493e8a943d16630b2f7ea345ab07e3105236857b43d93b": { "b1e073c8374abc5e997e5c6b5beb49db3202f0731072d2c28d7fbb0d58ae5e38": { "jp": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.913Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.844Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.851Z" } } }, "cad443b0bb3344ed063f7aa4c7fc2b79aced5e32830119e2376d8bc59ea14c52": { "7d224b4658e83885570c772a1a61546603db3deadf2539b9ba2ed630cb97e6a6": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.801Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.797Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.803Z" } } }, "ceefbdcea6747301b15ae01324b1afd1ac12aa220ed2fe99add6fbe53f6c7269": { "5840e875e6ec0ff5abbf5480df1b95d85a50786763ab037f67b711d24e4e67c7": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.800Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.793Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.077Z" + "updatedAt": "2025-11-29T02:45:02.805Z" } } }, "d4d0c35c5f0beed1c59fef3df7f5bfb3c862a52553491c973702a3bc2127649b": { "57ffcbf7d6cac66182cfea77cf8aba9e7c9e489b22f114253119e9ff7f8c1f83": { "jp": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.791Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.078Z" + "updatedAt": "2025-11-29T02:45:02.855Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.079Z" + "updatedAt": "2025-11-29T02:45:02.856Z" } } }, "e14b170922435b64e35287ad9833a81f16ff54cafad9dec0721b50d4150e5eff": { "a7e402c7578841050808aadfed7d6deea52ece0e68f8352e2e942645abf29aa1": { "jp": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.792Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.794Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.800Z" } } }, "f613caf640545aa0661fb14392a49a46e530351b4d92bd137405952d82f5b4c8": { "d8b96ae66a4502def2f78fdd03f27807df147056c6b3fc7bc330500d5a9451ba": { "jp": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.798Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.796Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.076Z" + "updatedAt": "2025-11-29T02:45:02.799Z" } } }, "648b00935dbd871b726296d698650b456ca7a653fa072fd74ce364e23a608928": { "ebc9c5357fa68d5d160cb6ddf6f938a834ac5bfc24d527684b3b9feaa9bc6a60": { "jp": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.842Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.839Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.838Z" } } }, "6d063f7195776042aa3f0a6d982cef56abab4e4b689ea926e2fc79ed09f5a2ff": { "cdca3b6d03d5aff13d620991a578cf9aae185e67396d308d55838c9401281d25": { "jp": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.844Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.845Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.841Z" } } }, @@ -20307,117 +21572,117 @@ }, "000b1489bccc8788cf74aa6329f6c98ad06511f167f46f1b934a958a5c6ce2b4": { "ru": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.820Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.074Z" + "updatedAt": "2025-11-29T02:45:02.821Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:19.075Z" + "updatedAt": "2025-11-29T02:45:02.821Z" } } }, "99b41ad75a6b23d70cb86b644a533c095785f9bb812c802ab52b650473d678ce": { "aa16d1a33d3312895cbf47d1ede82586dfb4df0a3507111d6cc8823a5446a979": { "jp": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.916Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.225Z" + "updatedAt": "2025-11-29T02:45:02.827Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.845Z" } } }, "be4a5f793e39d6e7b18691ba8685878af8c580f898c9f09efc5b93e0979b3902": { "b95eddde3a53a14028e00000ea72057696b55e352e2a30cb66fda415c9ba5d5e": { "jp": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.840Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.913Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.228Z" + "updatedAt": "2025-11-29T02:45:02.912Z" } } }, "c6fb4739e8e0ce948c34c03ed0f585498d9b45c24d566dfb8456926c4160207b": { "1d24888ce8aa77edfe5838c52a804ab3149a5d9497f036556a3e08576311a7ea": { "jp": { - "updatedAt": "2025-11-26T07:24:12.236Z" + "updatedAt": "2025-11-29T02:45:02.955Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.955Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.237Z" + "updatedAt": "2025-11-29T02:45:02.956Z" } } }, "d917e72b0a533c5af5b78c94fe1c05954dfd7ee48fb7ef7ab50f924f25fd68d2": { "b98abd6c9ba813c4b4a7cd9bc3018c8d18d3b4e71c0ec5233cf5d8da0a0f0441": { "jp": { - "updatedAt": "2025-11-26T07:24:12.229Z" + "updatedAt": "2025-11-29T02:45:02.914Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.919Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.230Z" + "updatedAt": "2025-11-29T02:45:02.920Z" } } }, "e05df611d62735d38ef4d916bb8f4ebe7a8d79a8773dcc1e94584527d5291d29": { "6ed109f9852559b92ce5667c817e8c2bc706b8ada65ecb41dd89ea0a07d5a71d": { "jp": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.836Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.227Z" + "updatedAt": "2025-11-29T02:45:02.838Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.226Z" + "updatedAt": "2025-11-29T02:45:02.832Z" } } }, "8e4cc87be65a0de0b75cdf694f1e368b68e721094e28ad05d1ab2af1aa7c97c2": { "b4c7e25600e2e0bab1150a0a7777cdce0d61b9c3e50a9c73e33bae121c92cbba": { "jp": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.096Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.095Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.268Z" + "updatedAt": "2025-11-29T02:45:03.097Z" } } }, "9dbbdc5c5acc11dc5874d8f84c2ec9210659a18cdd63bcc17e5b9addd0e11761": { "ca5dbd38b58fcc4d7a89bbb3e287de8dd7982f758f2a8e314589026ceed00758": { "jp": { - "updatedAt": "2025-11-26T07:24:12.266Z" + "updatedAt": "2025-11-29T02:45:03.090Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.273Z" + "updatedAt": "2025-11-29T02:45:03.164Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.084Z" + "updatedAt": "2025-11-29T02:45:03.088Z" } } }, "a1ae550295a483325655e321e7db058409614a56e29a23b67cbb7b001c387ca1": { "8978ba1f0ad1f751ccb53c78a3aacb61cbebe5e747e9d35fcdd7d9a45f55b790": { "jp": { - "updatedAt": "2025-11-26T07:24:19.085Z" + "updatedAt": "2025-11-29T02:45:03.089Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.092Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.267Z" + "updatedAt": "2025-11-29T02:45:03.093Z" } } }, @@ -20440,6 +21705,14 @@ "ru": { "updatedAt": "2025-11-26T07:24:18.960Z" } + }, + "286b947a3d644f84707e96a4a80f3a46f47bb75bf677493bf29a2c4578bc47c3": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.381Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.396Z" + } } }, "757c782e8d474da0f84ccfdac904b2cece14a9ace176070652ca6e68725754d8": { @@ -20453,13 +21726,13 @@ }, "4123bf4754603cd137b2c347ddc2ecbf727880d70156ebaba4224dfc6513ccdf": { "jp": { - "updatedAt": "2025-11-26T07:24:19.029Z" + "updatedAt": "2025-11-29T02:45:02.276Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.284Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.031Z" + "updatedAt": "2025-11-29T02:45:02.287Z" } } }, @@ -20482,6 +21755,14 @@ "jp": { "updatedAt": "2025-11-26T07:24:18.936Z" } + }, + "b63564ec20730ecb181b7257817dbde4f1541c1b85a389cbe3cd4e6e203c48c5": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.384Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.390Z" + } } }, "955f5f9d51c1666e2ab36912b9d7ce9512a594a94c3d4f68bbd34dc0b4b29a4e": { @@ -20495,13 +21776,13 @@ }, "2020a467b74c2031b09501bd31ebb2d005e1c3d366aa4673be3ded168b7cf3c3": { "jp": { - "updatedAt": "2025-11-26T07:24:19.036Z" + "updatedAt": "2025-11-29T02:44:56.594Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.597Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:19.037Z" + "updatedAt": "2025-11-29T02:44:56.599Z" } } }, @@ -20524,6 +21805,14 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.081Z" } + }, + "1e00e7ce8c07b67a72f3c30424ca0c2d930cacc231adf5a1336f323772ff2edc": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.389Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.393Z" + } } }, "ae8f86a4950495ec40a22f13e62e97bb26b60535bf2661e78523aa5c7694d172": { @@ -20545,6 +21834,14 @@ "jp": { "updatedAt": "2025-11-26T07:24:12.093Z" } + }, + "a99976a7a738ebb33cada2f4d924528e1f6779ca2332591b2c1eaf27105ec883": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.386Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.391Z" + } } }, "b56483af638a1b6f0944e0c00f0fa305176ee14ac4e45829309f7b6e538490d3": { @@ -20566,6 +21863,14 @@ "ru": { "updatedAt": "2025-11-26T07:24:19.025Z" } + }, + "82debe159b38d56f0f7e43e16823ebbfccd913c0fde77cb1d097d676eb7fedb7": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.375Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.377Z" + } } }, "d77d62a6594de01ac3e3969fc055ae87ccde060e7d7acddbf4ec2e3d79a2e678": { @@ -20587,6 +21892,14 @@ "zh": { "updatedAt": "2025-11-26T07:24:12.200Z" } + }, + "18ed02e06f16dfce881d97046fffde26c9f0db28c8ce1161a1f73a89b58682a6": { + "ru": { + "updatedAt": "2025-11-29T02:45:03.379Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:03.395Z" + } } }, "efe7462318a21cb44aed58f2c59069ee34b55da9d484d24f8da7d81958d002e7": { @@ -20608,18 +21921,26 @@ "ru": { "updatedAt": "2025-11-26T07:24:19.008Z" } + }, + "0af616e387db07695b2962dde0bbbd92c2ccccdb78cfa45a093fafcc97b3918c": { + "zh": { + "updatedAt": "2025-11-29T02:45:03.378Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:03.395Z" + } } }, "466fe68cf77ba8d2f7e6b11a62dcea8f2b8466f8161a1a4fb8352442e971815f": { "0fb852baff9f99f784eb97ea0fe1e81f329d845d7e142f0cf03f1c59b7c10b6e": { "ru": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.037Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.057Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.013Z" } }, "35ad145cf388d16f5d4a4011168a1fc7287db6205d3337996d98df0726ff1b0f": { @@ -20634,26 +21955,26 @@ "16c5698666ea7909d9e1753e9b13a5de1a08200f19d637afa8cab711a0379f73": { "38ea5377628be1984cefdabbe1181d528ddf34276864ec19a7193979c8dca03a": { "zh": { - "updatedAt": "2025-11-26T07:24:02.662Z" + "updatedAt": "2025-11-29T02:44:55.050Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.054Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.028Z" } } }, "85911f3bccb6d5539862e976203980d7d51391821089a818a002e7424e1242da": { "d7b1a435f7e4fe293383e5e8731be7cd7008caf825855a2e246a89ce3676aa9a": { "zh": { - "updatedAt": "2025-11-26T07:24:02.662Z" + "updatedAt": "2025-11-29T02:44:55.052Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.057Z" + "updatedAt": "2025-11-29T02:44:55.060Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.023Z" } }, "4ca4488b6240f0969f58775cdd28b38ea19b72d09ed672806dd6066698e103b1": { @@ -20668,13 +21989,13 @@ "237a635525e427bffb1c840b646e1b41486b8ccabc7712217a3d66d8c582f1b8": { "727edae2b97b38f4fc6c0b0dd353075d4fe831d345dda64ac9471ceaf897e490": { "zh": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.053Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.035Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.027Z" } }, "b1a18bb55dc19c1ae6d59cc1d7b85fd42acff628d3ca1636bfb8236189f4e211": { @@ -20692,13 +22013,13 @@ "7f4450440bea714d4def4ce9d273c25160fbc93f8195d945039db1f03871b626": { "98ef39e86680ea8421985ec9e48a11480382a84780d7c51e21ba7c7c08ba5de3": { "zh": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.975Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.976Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.970Z" } }, "046a9d5b6025a5438969893155368430145da58084c1a1cdb51b2f442bd31728": { @@ -20716,13 +22037,13 @@ "96339230d0b0662c9043872f701165e62b1dd1a9ee98448c3678014c12742331": { "f9dcd7d2195374981d74d8864cbac9660f4fe55a672e340bfa424e86bd032bd1": { "zh": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.055Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.016Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.016Z" } }, "6e8878bf1b9d227317cb48a45ab9134707488c84ad6fd609253a2e8ba3d90635": { @@ -20737,13 +22058,13 @@ "7b1152a9f1bfab485338afd2d917ac4d27b6ac598d4df8c416b5d34f5f2f2dc6": { "e85d9475b25d51b62300a450688edb90649a6b929805c4c6c7dc02c5c82425fb": { "ru": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.975Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.645Z" + "updatedAt": "2025-11-29T02:44:54.937Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.652Z" + "updatedAt": "2025-11-29T02:44:54.978Z" } }, "77b3f7333c54264798591573269c51eb187a6530077b8c89d00454a32b3814c7": { @@ -20758,13 +22079,13 @@ "ff3c9f598e696982267c2ce9a91a552bebc66583c1163dc1c4b27f82c5102f1d": { "128e8ba5fd3b5e0981c42ebd31c5b3e87b6845262805a4f4bff3b70534bfda44": { "ru": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.055Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.014Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.658Z" + "updatedAt": "2025-11-29T02:44:55.009Z" } }, "21345561752f0d94aea8069e30373e2f370e4ba9287cafef0820ff4937dc0a05": { @@ -20779,13 +22100,13 @@ "0361e95538168e72e0cf9076b4f8a823f82bca2acba30f30499d1d7ab6a5509f": { "d46f5caa45acdc3ea0cac4ee761116eca50f70acb1faa2569b6101636d3704f8": { "zh": { - "updatedAt": "2025-11-26T07:24:02.663Z" + "updatedAt": "2025-11-29T02:44:55.056Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.058Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.060Z" } }, "7eff5614e108ffbe8fdbb7cb7a60ce43f1c34abb8dce2015fa7c6e289db7874f": { @@ -20800,13 +22121,13 @@ "4914840b74cd4cd05b93446005c1a3f9b45c7e7816eb8b20c953782a78417420": { "66ffb1d1eb8cc149ea48f7ecfeda0ca180b36051bed03928a1992c631dc4c19a": { "zh": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.057Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.658Z" + "updatedAt": "2025-11-29T02:44:55.011Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.026Z" } }, "40abf84cb8f86fa1a58b9ec5523ea457c40aaf25e3b348ce068ffc50600529bd": { @@ -20821,13 +22142,13 @@ "7c40f4e2df36269b352d83d988edf0d606726b28f6527552e7eea3bbecafdef3": { "199bb81cde4d12c23b1adc97c7e2bce05a479079d23a4bb65c6826ef95452990": { "ru": { - "updatedAt": "2025-11-26T07:24:02.652Z" + "updatedAt": "2025-11-29T02:44:54.977Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.649Z" + "updatedAt": "2025-11-29T02:44:54.959Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.652Z" + "updatedAt": "2025-11-29T02:44:54.978Z" } }, "370a91b8533a723d2e4b1549c35c6735c837f2f50c1c4d609903126372f45d30": { @@ -20842,13 +22163,13 @@ "1eff56196650aabbed5f57974122db842d54e3093cc55755e2f4b980a957f4ac": { "598e57a0788cdc232382a72f993fe05e0d9a2ec8e815e0b23e6780d39b245171": { "zh": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.058Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.013Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.024Z" } }, "4adeee00af2b7dc1689945fa1a3ea72870eb8d82635d23c24f5afacdaee2d9cc": { @@ -20863,13 +22184,13 @@ "3c95fa2e161d494b4ae0ef9bf3131f3b028f13b824f5b7ede9ad688d11b58387": { "904fe0150e0e8c168afe250519fee5a4c27e23da832c312dcab667da64fa503d": { "zh": { - "updatedAt": "2025-11-26T07:24:12.056Z" + "updatedAt": "2025-11-29T02:44:55.059Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.011Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.662Z" + "updatedAt": "2025-11-29T02:44:55.051Z" } }, "296904d83e2f05abd0d201b63756e4993fc070bdb04cab19f7310a5f4982f1f8": { @@ -20884,13 +22205,13 @@ "19260fee9e23907e67f7f4589d997bab22cbabd4ffa0aa96806703a3b19aad78": { "1352a2dbb90191a61432180810a0431b454c526d658886e1c33fdb1c71cfc2bc": { "zh": { - "updatedAt": "2025-11-26T07:24:02.658Z" + "updatedAt": "2025-11-29T02:44:55.008Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.019Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.658Z" + "updatedAt": "2025-11-29T02:44:55.010Z" } }, "7032d2644260720142d73bb5705b9bb1dd26018cb12c421cb43c6bd87452858c": { @@ -20905,13 +22226,13 @@ "c71190c424029f1f3166b0dc0c975e43b747cc77aaa7477e6c8834baafd715ec": { "40fb6fb53bc03ff95d4c2a5b88f33db598b6bbba4a8c8273a31dff8b7c9a3fcd": { "zh": { - "updatedAt": "2025-11-26T07:24:02.644Z" + "updatedAt": "2025-11-29T02:44:54.936Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.969Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.965Z" } }, "35eb622d4a1673d7a2b49ac6d4fbe5151e7dad205dad7c16e0e36879a5bbb7da": { @@ -20926,13 +22247,13 @@ "3490c72ebec2d9960e4cc311de931030fc0f1de3f2421d0d2a30876926a983e9": { "20143fdffbf6f144ae3f0a848c2c4135b1dd5359078f18a35f86e5ad0368f0bc": { "ru": { - "updatedAt": "2025-11-26T07:24:02.647Z" + "updatedAt": "2025-11-29T02:44:54.950Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.970Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.973Z" } }, "716d4abd1d53aff3d2fbee3ec30720b9388d98d71e41c650e6551e5ee79417a5": { @@ -20947,13 +22268,13 @@ "df1cbab9f5b7839553ad76ad0b3799099daaf2d5817b6bc1eea8369de5c5842a": { "3a49b42cc312e4959cc3883b924f895ba1f241473240bcbd42a5ff859048c600": { "zh": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.079Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.102Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.080Z" } }, "66576350fa60992186887698075dc59ba76bb735288c2751fa40b91ce10698f2": { @@ -20968,13 +22289,13 @@ "d133c163191364466953c00a3494895f7b213291fa7eec0a3286c15ab6588c48": { "5b79efc25b16535ce983e05832f4052257d44d2790af29323a727be1048bc054": { "ru": { - "updatedAt": "2025-11-26T07:24:02.647Z" + "updatedAt": "2025-11-29T02:44:54.951Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.647Z" + "updatedAt": "2025-11-29T02:44:54.949Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.646Z" + "updatedAt": "2025-11-29T02:44:54.947Z" } }, "de7e11301702e7f96f6fbd025a9103d0ed8c30b19a1bb2d879dbd161c1061ad6": { @@ -20989,13 +22310,13 @@ "5ae13595aec14e94efae48ed27bd30882ef99ca22e926c6eecac01f4a69b6e60": { "4c6c9c998098906955cd0a416322eaf10b8ceb9a33df69bb90b4e0206e58399d": { "ru": { - "updatedAt": "2025-11-26T07:24:02.648Z" + "updatedAt": "2025-11-29T02:44:54.952Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.649Z" + "updatedAt": "2025-11-29T02:44:54.960Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.647Z" + "updatedAt": "2025-11-29T02:44:54.950Z" } }, "832ebe1b67ef5ee07034a93035676b1d6ba9f009d34428f33f25ec2daaa43771": { @@ -21010,13 +22331,13 @@ "52f1e721b650aa5a8bb67053afa7caf447a7332e92f416526d36e8941d726d04": { "8c41257fcdc2d116e76c9a1609bc65adf58513acff260b8f2aa36d74bccf31da": { "zh": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.012Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.021Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.023Z" } }, "630e5b84780be36656bc937645ed65fb88179d11247d1e85ea1205ed29e6f931": { @@ -21031,13 +22352,13 @@ "5a0ce1710868a408e43b0c9859a80ada3b08b93b0d26cb45f2ea004556e9d2b3": { "ccdecf590d1994e9c17ae91e353b32d2f66c08e379ce1eeb73f06a674afd8375": { "ru": { - "updatedAt": "2025-11-26T07:24:02.648Z" + "updatedAt": "2025-11-29T02:44:54.953Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.648Z" + "updatedAt": "2025-11-29T02:44:54.957Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.648Z" + "updatedAt": "2025-11-29T02:44:54.956Z" } }, "a2d654d0961b9427057876a5b47403d5864939d9a0cc302f7941e73ea9093498": { @@ -21052,13 +22373,13 @@ "9b3e13e23b506d9d9ec9b2c5fbf8b9d2a62e1de7d0175c5f6330498124203aac": { "86c47ff8f3b3666e1a6b49b2c8302b448389e1e3b41ab3b1450e055082821549": { "ru": { - "updatedAt": "2025-11-26T07:24:02.646Z" + "updatedAt": "2025-11-29T02:44:54.945Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.646Z" + "updatedAt": "2025-11-29T02:44:54.948Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.974Z" } }, "8c1816d77d3551c7d6dd5710ccc8274f66e5809dd3cea3606629893483ebfef7": { @@ -21073,13 +22394,13 @@ "30f843a3827d19f26bae893b6a89699d15924309d3ee0d771f1309eb391c8171": { "a5eb46f97ff75367e3c2a77e86b555adee47157db34a73cbb68c4faa8e14d033": { "ru": { - "updatedAt": "2025-11-26T07:24:02.659Z" + "updatedAt": "2025-11-29T02:44:55.015Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.972Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.662Z" + "updatedAt": "2025-11-29T02:44:55.052Z" } }, "655ba8e4e20f3b5f89cae3033f51649118b5face2393e69b8ed2d63f7c170bed": { @@ -21094,13 +22415,13 @@ "15cacb127be1afdc884be3ff13c61ff48d4ae41e28740309f5f445002fb0fa90": { "a9c8fa4f53951ce4026e170171a0517a80777e9037e5bb2f16eab83d3ffaa9cc": { "zh": { - "updatedAt": "2025-11-26T07:24:02.649Z" + "updatedAt": "2025-11-29T02:44:54.963Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.651Z" + "updatedAt": "2025-11-29T02:44:54.973Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.648Z" + "updatedAt": "2025-11-29T02:44:54.954Z" } }, "c6b1ffeb8a927241e2108dbeb02a8cbb166d5b270f1e7cdf770147d6ef83a7d2": { @@ -21115,13 +22436,13 @@ "941b4aa0aa9dbadd0a190a16a820e2bcff3884350dd172d2d70c5e4bc21490d1": { "429135ca177730d77f47327bd61c6aecd212a21d1a4625d711d13a6e0c6886bd": { "ru": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.081Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.082Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.080Z" } }, "d744ea0501987d0d0496e17c8100a30396b41d2cb02d4b4937b9c75678cffd0f": { @@ -21136,13 +22457,13 @@ "43aa5066af84a8c935f0fb2dab57ea37c855c50a8c4bf2fe5da1196726ec9767": { "8102f53c258449f037fd5c8bfbe1d4547d061cf4c8af817be8f9e6c45a4504b0": { "ru": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.018Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.022Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.017Z" } }, "bc18044844f416597eef2c300fc30d72ea362c8100b916b3cde37fd6397a9e41": { @@ -21157,13 +22478,13 @@ "a3a2fbdc5aafe02b0407589bc3e1a8e94202c17584b7025219f1bfd6b9bf4a39": { "4874e6e4325e8473fce83ceca9411bf266bf400e8eb78d3c9e8eec128469d820": { "zh": { - "updatedAt": "2025-11-26T07:24:02.660Z" + "updatedAt": "2025-11-29T02:44:55.021Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.661Z" + "updatedAt": "2025-11-29T02:44:55.036Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.082Z" } }, "1b128db269c12be2125d03f195c663118806c04caea0bed54648c79f2879ccee": { @@ -21178,13 +22499,13 @@ "4877e91053b08c2c45734e5085ccf9117e8354554dd8460e2ec3e3afe7aa0ab7": { "1e4f5fb2eb3f3d09c80229402157ba0cccbf2f37d7521185e9cbb71109edeb84": { "ru": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.965Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.650Z" + "updatedAt": "2025-11-29T02:44:54.969Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.649Z" + "updatedAt": "2025-11-29T02:44:54.960Z" } }, "ff50c271592348dfa10d95b4d2fa83784b90178a9865e6dcf8c7996829ea7358": { @@ -21199,858 +22520,858 @@ "a444951bd73cb75b037df1739eb17fc3c4057630058e2cd15b863d55feb1e497": { "be2b70c111bb68681c2eb58d9d87da824e86dac80806aaf1af31eb7e683ee46c": { "zh": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.264Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.692Z" + "updatedAt": "2025-11-29T02:44:55.277Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.283Z" } } }, "b61feee503868b9ae36d297816fda3d2e834c0f1ae6f4deeefcdd9b66b895886": { "4ef342336cc701c4e8d32cd01c1302bec119023fab8a7c695a4baae3e097696f": { "zh": { - "updatedAt": "2025-11-26T07:24:12.057Z" + "updatedAt": "2025-11-29T02:44:55.061Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.071Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.073Z" } } }, "b2e9e9045947db36c00975d8bf16f27ba366df3f4c68a977779fbf5a78b77948": { "046cb0e8076cf8c0b6c68469e0acc454e928a24cf0dfeb0b83292ecb2957f821": { "zh": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.264Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.692Z" + "updatedAt": "2025-11-29T02:44:55.277Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.691Z" + "updatedAt": "2025-11-29T02:44:55.274Z" } } }, "a8580441e057aef43ff213b00764e321caa0062300adad449c1147c6a00554d7": { "803165c43e8eb2cc396419bba2e85a710e5a34fa1c1f8c024a4ef0cd296866fa": { "ru": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.293Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.311Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.295Z" } } }, "581f0a6e4c0d192c8606c68934251365ad7ea4136bd5acf7058f58a76f6d5710": { "ee59cd484bdaa73a60bc061cc701d580ffd417f73fdcd689e3fdd983d9f475d2": { "zh": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.325Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.326Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.903Z" + "updatedAt": "2025-11-29T02:44:55.337Z" } } }, "8d435bf9e6c99e8e1a52f439de6bcbecd2baf3265ece4535053d1e1416ca45c2": { "0c0d01e2f586c0d713dccf1bdfde13a36570342ea30a52d1914566a1af56d594": { "ru": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.294Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.308Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.305Z" } } }, "ff15f334dd81c6f832484d8628568a040ff836d4668005abe916911afbffe911": { "5255a26915e56655751575c9c47141ed725215520f648de9ddb2650d95ec7c9d": { "ru": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.265Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.691Z" + "updatedAt": "2025-11-29T02:44:55.275Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.691Z" + "updatedAt": "2025-11-29T02:44:55.276Z" } } }, "84d3a07f6bb23015f78e31d1cc93e61eaf670a2dcee7c14342d97b32fb037866": { "e5b0ff50a5b4e2b593b51ad0606dd79a8525ea9ba7bc58e22bd24ad8c5a925cc": { "ru": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.265Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.283Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.266Z" } } }, "54d5d67f63f4e8a40581478b2c6f0684322d03116d22c84c5ebed5934c483f47": { "04a1c4adbd60bd15811afb47b49c06837b0eb88b3c5f243bc17465571d25d192": { "ru": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.295Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.319Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.301Z" } } }, "fbe38e1b6e52f3e6a217864f12729f9229c6e773c063d08c4fefa0972fe40f33": { "45c2a21595b33f95e98b84bb6f2c46e0dd1efdfbc1f9b42ba02cf96b3dc4ac1e": { "zh": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.296Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.312Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.299Z" } } }, "e9c8787fbd5d3ab34de4fbc2069baaf46f6986970cc7b8edaffc49a991d61cf1": { "7b366931a91740ebcbb465a17f5142106ecae677c271c9b69d08fa475ef502a6": { "ru": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.265Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.692Z" + "updatedAt": "2025-11-29T02:44:55.278Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.273Z" } } }, "14c0bbca8f7e350393ed679d410ca4b9cd58e0c5ee29885f7e65beae7f51c703": { "82258f2bbaceee1cc2b71c162991c1eb92c67498d494693cd385b4bbbb78fedf": { "zh": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.297Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.320Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.292Z" } } }, "dd1f243e110cd8cd4c72fabd62923f7077ed63859ba2c578b1561943fa5490a9": { "38b8464001ddae6ec2a702908a9a44c1549405c54b818345c5ee01e6079833f1": { "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.327Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.901Z" + "updatedAt": "2025-11-29T02:44:55.333Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.900Z" + "updatedAt": "2025-11-29T02:44:55.332Z" } } }, "ba14369199fbec0937cc2e6400083d328a65fa21e4191586d4474ff60d50b27a": { "687b275c30319ae8712f2bb22a713be7698df9bf60e3f0a3a92687b0ad5813e5": { "zh": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.195Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.199Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.215Z" } } }, "0aefd7962edb78c98f3d45305d81752ebb2eaf0195c75d6a1cd6e7ad0ef5617a": { "5d1d0d81a87bc16246cc6d195a4d9c9f3090b00692a1dcfb3dd44b533128b6dc": { "ru": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.297Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.308Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.298Z" } } }, "6b0a1864f6fd70f19415c4e085caeeff45b83244daed33758454b88d9859c692": { "ecc79a94c617ae9c2438b3b427bea3004cc3f1e8a3f90157b36f8157166a99c0": { "ru": { - "updatedAt": "2025-11-26T07:24:02.653Z" + "updatedAt": "2025-11-29T02:44:54.985Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.653Z" + "updatedAt": "2025-11-29T02:44:54.986Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.653Z" + "updatedAt": "2025-11-29T02:44:54.987Z" } } }, "543d200284e9587853538717503646bf5a945bb43ccdb3b059dbf4eac4c1219f": { "54eb6cb69d7901f33c8b60f1ebf53444695ba214c41ecd088af34c6dde0d4e44": { "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.328Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.904Z" + "updatedAt": "2025-11-29T02:44:55.340Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.327Z" } } }, "fb3d54543e5565bc4305346ef7c2d5312674405acb6e193ffaf4fb30ddd7ce71": { "df9135ddc19fc1bbbb29d708bd2c3afbd621e4a67a544ede4538a80aa5b420b7": { "zh": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.329Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.330Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.325Z" } } }, "14b4676b953c664afb277f933e119c8da2f742590c1a9a4bb7b2beee22a7eb7c": { "5ee021b8f49ccf1b18d5dd6f94a9b7418709365c4195a6b0854ae20f5132dd10": { "ru": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.267Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.270Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.271Z" } } }, "0f67bde502826e1dba901c267e553e40b45a88ea2514fac35224b3011c9eee95": { "40ccc189c309d81655c42b58d6550569ed8e72b0cd53cc36991d1ab17eeb62a2": { "ru": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.267Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.695Z" + "updatedAt": "2025-11-29T02:44:55.284Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.274Z" } } }, "93a056e5b771b1f20f3660dfb370f302960d593ccff14a5684b961c760cac61a": { "b34875547efada966d6f58a27a70b1a17213f7251649cd70a29b9fcfe4aeecfe": { "ru": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.298Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.303Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.300Z" } } }, "ebc5db761ec12b7516bddcdbb93d868ef5c7d1458f56a4288fab25b5e45a980e": { "e20f9f94eb03e49c98c43e022936ac730a22ccaa64a4911703f457858a10f672": { "ru": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.268Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.693Z" + "updatedAt": "2025-11-29T02:44:55.279Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.695Z" + "updatedAt": "2025-11-29T02:44:55.284Z" } } }, "f016a1612cced253f74884a4791ce47126fba584f3ee773967310982b7597b83": { "cc687fc17daeeb33c7c5bef1a2bc7ce51ba437f92c4354369ab58a024c2123b9": { "ru": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.268Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.273Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.274Z" } } }, "f657cce435f5bbd4c37d13d06e137048b8588d93820f3ee19d2b600ed82b6819": { "f4e41d0b3fe1c04866d1690f92f407974255a1b7b269dd34af873b60f54ecb09": { "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.329Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.902Z" + "updatedAt": "2025-11-29T02:44:55.334Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.326Z" } } }, "5a8a41312c127bc8ee51985dd35b7a34db3722502d9dd3b6517218f83ee15209": { "cdc27bc165065afbf272c456901edc7e818c1288e8bf98aa8115b3cc4184e430": { "ru": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.299Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.316Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.310Z" } } }, "bb301384e711a26eac5ab620725ba3651e9a050418e5c4b03409244a6916096a": { "fa37176654ae0b31692c4310f41376cac060e1fac5de1cd5fa4a6795dccc88be": { "ru": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.269Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.292Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.305Z" } } }, "be5b2c5f34f09aeff162abaf45ccf882807b091723c8992305ab5dd6d9d85255": { "a4494efc6991ad7d0de3d84b86e624697071ddfce8e39ebd42923fd6777c8531": { "zh": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.269Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.271Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.272Z" } } }, "b7ac58ff02407e2eedc607e8ffaadc709667604b213c6400361a10c2a2c6e252": { "ae94f635f518e540a73bbd471cee47b91d539ed719fbffdaf358c667006c4bb0": { "zh": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.270Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.272Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.266Z" } } }, "f2566c10efb98a7e07538653cda7cc2135c5c1aaaef306a48e8e753ebc662a1e": { "86c47ff8f3b3666e1a6b49b2c8302b448389e1e3b41ab3b1450e055082821549": { "zh": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.990Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.989Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:55.001Z" } } }, "c3d6ae1d7c3ab47f1321484233d7e2d4c6960c431966f43a50c94da67e615da5": { "7fe2061b7ffe48c965db16b4f632dfa6a0cb32888881320b91a370311396c437": { "ru": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.234Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.233Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.233Z" } } }, "6f8f89ce13c70fe1235d08203ef798a559154950245f81065ab893d0e5c542e3": { "f96e0b809311db6c2baef6eea1807c7d62c21afafa50f43dcaed5dc333127e20": { "zh": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.302Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.315Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.316Z" } } }, "857f78e82a54d7a2128693b3d739a16697e3d23a8ab3595b336d9da8d6d1d643": { "3fadea060a820d56c666c2cf5cdeb8e49e9c833dfa43de6b17bb735aecf7c763": { "ru": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.302Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.314Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.317Z" } } }, "98f9d0cfd669fd1fa447820ed42dde75e265419fd66cf20c9292293dd4a825b7": { "ef840aa109bf499596594d13130b402a3f00f31d42de8569556571fe1c214cfc": { "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.331Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.330Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.705Z" + "updatedAt": "2025-11-29T02:44:55.326Z" } } }, "0ccba8d2db72b1884bbc46c41967afaeff1aa84c34d44e471d4f0a6956691e16": { "94c625175686dfb070b11d461168883b7020c135e87e95dc215bd6a1888c5c54": { "ru": { - "updatedAt": "2025-11-26T07:24:02.690Z" + "updatedAt": "2025-11-29T02:44:55.272Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.691Z" + "updatedAt": "2025-11-29T02:44:55.275Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.691Z" + "updatedAt": "2025-11-29T02:44:55.276Z" } } }, "c3624723e67987627989b19cf8887d0607b1cfe3b554bdb9b1a4afe0241fb796": { "394ce4286ff89f65fa6b50578d4a94d4eaf540883591642f71afb2825984bad3": { "zh": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.083Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.084Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.085Z" } } }, "3f0eaac3f28ba8b2234626f11889b6f51135f12393d659a739adcfe6bb3acaee": { "b93542926f20e8394566dc0612022ddaf2939a3fdd8e5ae25b2ba31cb94de320": { "zh": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.993Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.003Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.653Z" + "updatedAt": "2025-11-29T02:44:54.988Z" } } }, "101a525d5bb936cf99909df3325b1ed7ac0b685ee9889c47f517b4323eba52db": { "fead6f3f426b4d09ad7d10dd975751d5778ec0e92cce0f8ec88ce01950911970": { "zh": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.067Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.069Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.057Z" + "updatedAt": "2025-11-29T02:44:55.063Z" } } }, "6bc2d77e531374ac95719fbbe878154b867b374b36d2f4e8485da1fa3a3820c6": { "d33c2c466bd3a7370a079c2bfd8752318550c559a12158bcc434cabdaec31040": { "zh": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.303Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.312Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.292Z" } } }, "43bdb45dd285638fe98614183eaf90571d4631d1e726c04b99db3c3faa08af32": { "4ba84b799e9b0e8d9b223c47606c717ef7d6ddd565986bc7b238eb33165681f5": { "ru": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.332Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.904Z" + "updatedAt": "2025-11-29T02:44:55.339Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.903Z" + "updatedAt": "2025-11-29T02:44:55.336Z" } } }, "fbc3d920f0695e12f892f5ecdcfa4bc88cf0bb49809defb12c39db77838dee89": { "505618685d75d6489d64b01bd2297e8b2e4ce44b92900a9edcf4d95a5eebb475": { "ru": { - "updatedAt": "2025-11-26T07:24:02.655Z" + "updatedAt": "2025-11-29T02:44:54.995Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.006Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.002Z" } } }, "67e6b09bfe484e48895cf047e4050cb1f08398f2f779e27c7acf3ff69d9c5e8d": { "7b905336c6f753917b4e006f53075b8ba27cb105a18643882989eab9b01e424f": { "ru": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.304Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.317Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.294Z" } } }, "736363d0859d8864ef39d3c2b3906d5ee9e8520ec754a5daaa253102669dbfe3": { "4c2ab8cb337c681d306ce35ffbf49cc6acb8d68b78b1f946b2757bbefd07e898": { "zh": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.305Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.312Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.308Z" } } }, "f9122446fb42667abd4d27483874244c927449544300488e6be5e19f0f5c5196": { "fc2f22b778e933aded713c592fc3c7f36b5925e3b1dddf870b9f00d0d86f3078": { "ru": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.306Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.311Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.293Z" } } }, "235b40c46d2961005ce3297b1e97ffe8edc82de828ff56822b9e32359796e9a9": { "c5ef2e83c2e151559f9dd5524371a9d5b3447d2d1d74ee4818d09823d6de408d": { "zh": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.246Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.692Z" + "updatedAt": "2025-11-29T02:44:55.277Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.289Z" } } }, "462cdde9af0d98973a369e372516b17fe788292eab3b5888894d73e9cbffb6cd": { "d745f7b346b2c1bf0d164fbdb236d9160be09038c4c9ffee5d2fe13aaa441118": { "zh": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.214Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.219Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.211Z" } } }, "710ad55c0afad6985a560346d1622540e29d92eadcee6064888e0cacbfeda384": { "54f1a9cd08afe76cfdeea722af528c57303609afdc34748e3328885c439ce7bf": { "ru": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.074Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.066Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.068Z" } } }, "0fb5c4c89db0cb83f6bd1cdef9a19071c391929cb24660f2f66de45b10763ba3": { "23aae78ddaf4de455a27e50918cb30da7db97d56977cd4dbe8df7b2e1cd49fc4": { "ru": { - "updatedAt": "2025-11-26T07:24:02.658Z" + "updatedAt": "2025-11-29T02:44:55.007Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:54.999Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.002Z" } } }, "45c65db56b87943c8cc881cc85fe81f875a263a988b758817095b2448ebeab1c": { "ef02a49eb6596c142aa773eb78cf22212510b6f1bb9809d02c025e4d34ab82d7": { "zh": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.247Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.232Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.247Z" } } }, "b58d28384b38660cb355b5217eb858f4bc83ad7155278c40ae4663b230c74fd8": { "f5263d91719fc0aa0d4dc51eba8629ecf707553c3c6fd5144e8f1ca748775d75": { "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.172Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.186Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.166Z" } } }, "8d7c4ba98d5f5bbc6e42b2591da7f2b20f246b15396b0ab2075839fef18b5697": { "157c626f8a13dd4dc09e8313f1bf33c397d35bf379c354eb9d973e648827bef2": { "zh": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.215Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.228Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.201Z" } } }, "4d0528f558f80f4881563682077f001ad134becf467e305c86fc84dd7697b089": { "42d9d42562a4f705923103bf4a3b7173addf1f1dd5adc163a37dbd936aa49889": { "ru": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.248Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.241Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.236Z" } } }, "a5d93e69125f512b3e1f00266e424585b846115536039af5f58cae578c2829e3": { "ecacb8f11638f831b9c20da459d9a74e871ae3943e5721f34aba4985e3a9d9eb": { "zh": { - "updatedAt": "2025-11-26T07:24:02.667Z" + "updatedAt": "2025-11-29T02:44:55.133Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.666Z" + "updatedAt": "2025-11-29T02:44:55.131Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.667Z" + "updatedAt": "2025-11-29T02:44:55.134Z" } } }, "5bb6610775ffe555ff909d1e5c0cb423ff2c15c044240729c60b0ebe39bbca30": { "d2a5526c06779a9c79d3b4da1256e4feac0aed57c3171d923a4e08990a784158": { "ru": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.248Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.237Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.682Z" + "updatedAt": "2025-11-29T02:44:55.261Z" } } }, "2c61f03a4fe808580cff4e8aa1a6939d84eb12b9a43724b98bab278d020bb194": { "4158e73583a46ee61d2835723076f3fd91bdae28b86fb6f4d6ab8870a8146937": { "ru": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.172Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.170Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.169Z" } } }, "8fb2e5e5d61ff6b4830012f315c75ccd22ef6f64c4ee7685c2cd3215aabfe79d": { "c393d1a8b5995f5444564d2d762f97bb4815829fdfb74c4739bd527681d89cee": { "zh": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.249Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.252Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.250Z" } } }, "1a54cbb6d0259ab1c0a7866c16289a6efb190e3d138af3035a9b892ce04da57d": { "35875b5d8355a345f2dea01781d4a86cccffca2873f0f1c8151df687559a6ee2": { "ru": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.216Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.225Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.211Z" } } }, "c5c96baff0600024d6bbb610d9cae24faf4a22e4f54fbcc16da6eea5801d716e": { "75a61fac01b9a0c4dc6479a31dfe0ccf020bf8c906301ce66ddb70adc32e62a1": { "ru": { - "updatedAt": "2025-11-26T07:24:02.696Z" + "updatedAt": "2025-11-29T02:44:55.285Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.693Z" + "updatedAt": "2025-11-29T02:44:55.279Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.287Z" } } }, "be5b364ee73eb51fe865a890d10236c2eae4146ef19afc9755721c110139579f": { "e55f970b0157d55548b665f2a95fc93e3875eadfb7a385687eb591b21d592f97": { "zh": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.086Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.061Z" + "updatedAt": "2025-11-29T02:44:55.083Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.086Z" } } }, "4449f60ff9c38182ac376f1ec8ad4f5c377a1d189bf6b8bd0b3f294437ebd1a5": { "b4657b26faf846e566012308f61103c34dbe662b80add135f7d0720222c74ea5": { "zh": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.216Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.195Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.212Z" } } }, "809f990c2475c0e585de5f7677ad5e69d2c480395ed833dfa2922067881e3350": { "1534d3d5fab78c52b36945dc4157e83845141abc6b963eed5bb780b27e5e23e2": { "zh": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.087Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.093Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.094Z" } } }, "8184344ce9b79685863100b25d75f907caba31a6f26b64caf68881e98ea41913": { "8fe3205e82057a29dc0d8aaa2e33ec896cd304ef416bcfb7264bf8da1fbaaa77": { "zh": { - "updatedAt": "2025-11-26T07:24:18.904Z" + "updatedAt": "2025-11-29T02:44:55.339Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.904Z" + "updatedAt": "2025-11-29T02:44:55.338Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.902Z" + "updatedAt": "2025-11-29T02:44:55.335Z" } } }, "0c03db74eb0923183ef12e6e957c01e6d8255d17051d0474807d2dfe15494516": { "8d293de1b22941bb10fe562a4e677c7c7472f7d882ef5aadce39c9033dabb63f": { "zh": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.288Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.289Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.285Z" } } }, "a1c0860ae09b803ff5ed9c9a0c438bd6b2800982753e32c40c910f32979fca1d": { "48ad888591a6dabb0298398a02a18436095ab5c603d344f9156ff7e7ccdb28ae": { "zh": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.318Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.314Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.296Z" } } }, "86a43cc92512a5c918f3385b494d3169c660273f3661eb8dafdc49055b720698": { "60b60a413c29322369042c265eefb3d9aa56d79f8c71fe607cd1ac9eeb60e393": { "zh": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.319Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.319Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.301Z" } } }, "036300ef3b2f4858d6615f663b03ca7a594a026409e0fe0ca41882745b846afc": { "1ad91e7f68dcee666ce7f7d2260270095678629c6052b5b84bf68dc6d54020c4": { "ru": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.089Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.085Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.062Z" + "updatedAt": "2025-11-29T02:44:55.087Z" } } }, "586898784b2000de57eead4932f64db3ae6900471f06aee84b184f3bf8efdf12": { "9c727f0fda6cea3eb8d9add0737f40fd7c2a246e0b779e6a2ea7559741c3af0b": { "zh": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.090Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.088Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.090Z" } } }, @@ -22065,1097 +23386,1108 @@ "jp": { "updatedAt": "2025-11-26T07:24:02.670Z" } + }, + "5962997760b38b2cb309d629f1dcf48964113a84f277bdc508e98c8bad0fa965": { + "zh": { + "updatedAt": "2025-11-29T02:44:55.153Z" + }, + "jp": { + "updatedAt": "2025-11-29T02:44:55.153Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:44:55.154Z" + } } }, "e79b575c27312875f3076748b2d4de3bfd78216748310c894e316b5c6b915aa6": { "7a7699a4379151bff326d63b86c2e5f5b0c36a7de56625710bbef094f9488e4d": { "zh": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.303Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.318Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.304Z" } } }, "c74acd4897e7b7ee4b2df0bff72a35b3b8acbfe976eaa8215f2fcfc031f94ccf": { "720c459362ca150d27eb7701d7e48ce41817e1142bf4ebb8b4e2a87705715ada": { "ru": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.167Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.177Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.178Z" } } }, "503329b0d4a76ca6bed899e9672f8b552300a0c87af309f4216ae734b9861fd2": { "675e12d63a5beef8dc9c071b80bc5249b9dc320e87ed8e63ab1dba75742d1c49": { "zh": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.167Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.171Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.171Z" } } }, "90f0e15a1b59f060a6f0f952d87af6522508eab261e93dd1ff9d2f135297bc7b": { "b323a03a283828a8dd2bdb1310eabc167e779d51e7e53bc928a0c3475022c6ed": { "zh": { - "updatedAt": "2025-11-26T07:24:18.901Z" + "updatedAt": "2025-11-29T02:44:55.333Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.904Z" + "updatedAt": "2025-11-29T02:44:55.338Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.901Z" + "updatedAt": "2025-11-29T02:44:55.333Z" } } }, "e1777c4c468ab2516b850e57b6f6bc5a611e182371ea737b4494074aa581da40": { "c93f95ca1da1b0eee11a33d644aec21a8b55b826129592b9eba161908812b369": { "ru": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.237Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.244Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.682Z" + "updatedAt": "2025-11-29T02:44:55.260Z" } } }, "64e0092d1db56a02e6a5bca8f0b5056cf1f521390ec3925bb3e50df81aa7ac85": { "9a5dd87bf7b220294da0bc415b255ea64029a767c79b1e6a895b5d3d57801055": { "zh": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.240Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.242Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.243Z" } } }, "d012409948884982e8bdf1e450327b34af2546383469b4fd132b635459e6f305": { "95aa9403608d32399c22cc7fc263d9ab30a605eea3844947170400f89d7e71d1": { "ru": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.246Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.240Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.237Z" } } }, "00174dfb396f321fadf5749558a865565bf4dae8cc5d6fa8f305ef68a7f1c6b2": { "d2f79ac832b7a2d7aaa410633fb001b9e95f4660cc65da2bdbe34ab52df0894a": { "ru": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.249Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.235Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.239Z" } } }, "774db99efcf187fd85ea22f0f07cfb6cf5fb6cc68251b2913b976e914e74a951": { "cc59400f1e7b6cc7c2ce5902dae7bd2a641bff181193f2f3f16b2cc24b094add": { "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.173Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.184Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.165Z" } } }, "4da7a2a8dcc0e8244d17285e749f8d2f66e7c939010b06d93f9506b5e0443395": { "5d4659d3e6e8c514f951b33a0e387bbd5340061d0fa6ede0b8d63a27a889570a": { "zh": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.250Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.256Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.238Z" } } }, "8a9dc951991e7089ccd4e1eedd2df9ce190a4888a63408845057666bec28693d": { "3ea6e01fdab2aaecd5561d6a3738320c4c955d0937ec5157cb9ac2e69e3fa30b": { "ru": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.286Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.288Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.286Z" } } }, "e0416bafda40f9b0abd3190774a6d8b8b6fecab49f9676913bac6e5e053b382e": { "aa3e533069b101ec06bf29cb5c1935709f54b0a36858f4636f093f238b277647": { "zh": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.217Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.200Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.231Z" } } }, "6bec8fb9d627bbc8d58479b40c1ff2e2105bf84d0574e514ce2d4a909b35d280": { "9892fa9d4ee47152dab0a70403163228e13146e378a484ac01ec35395c96a186": { "zh": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.315Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.307Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.307Z" } } }, "d600a99ead8b0977fbdf31462c610327f9207f07a47047e4cfafebac76ac6789": { "ba98a569e23d5a0b5a2bee157907242c18d05d010d12a96d4526528db77500b5": { "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.174Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.176Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.213Z" } } }, "6c4d95e5c9add2129eec07c7db776b15731e42064678712cecf1b19d27e9fe1e": { "26bab87ac6555b58f09e971a206121597dc934bf1607e0bc1d1c1ca74b3c8ab5": { "zh": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.175Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.165Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.177Z" } } }, "c97c8d3fc1255144232e48ef1068845cf9a505bf268924eb00d02e4a764b06d4": { "cbf44b30af8d393437b434943a6b72c84ddfbb0c5021ffa6ee01fcee470fce64": { "zh": { - "updatedAt": "2025-11-26T07:24:02.667Z" + "updatedAt": "2025-11-29T02:44:55.139Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.666Z" + "updatedAt": "2025-11-29T02:44:55.132Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.667Z" + "updatedAt": "2025-11-29T02:44:55.134Z" } } }, "aa965228754f809fd54c3e57e8b77d0a2e9c4a048e0e68cef7ae8c333114457a": { "f9ce484d23646e185c37dd955d8f8211aaac0ff9716bb25cc7a6c1dfc7722732": { "zh": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.218Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.228Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.196Z" } } }, "db4a603afaa721633684ab401b86356ad8252b3e4987d3d7f3a1c55750046ef3": { "c71c72e22f263d7e5cb4b6bc6151025b50d1a6999e50ff20143e7d9570eab7e8": { "zh": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.218Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.221Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.222Z" } } }, "cde7d435635c91652b2386bf1619cb7ad40c1a75333e02d9abeca5d374b5fcd2": { "7ed8aea2f22f07b5e6da1bc31a668115f599b57278bd5f78ed4d027851ee59f9": { "ru": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.219Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.195Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.209Z" } } }, "baa5800841c33574a763c76d84029b7167e28cd0e383b549d3c87bdde30230b1": { "4e66ec48e4681668b3829e07df4225df08079780a33326c20145dbd63d2cf115": { "ru": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.287Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.693Z" + "updatedAt": "2025-11-29T02:44:55.281Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.281Z" } } }, "6999f92f0023fe1dd1e922ddaaf1df722f316e49e43a1f46219683d3add8c812": { "9280cf92c0f64187017d3e623d9d06cf5122c9cca98da66abea3317bbf634e3b": { "zh": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.251Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.234Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.251Z" } } }, "534c97b1abac407b7ffecba5c237d20ca3ad4c270a44ed90b44e77de585a610d": { "7ba7deb86c597b598ca684677abf36c48f1d224dfbe3c8465bb1e2b40a280f81": { "ru": { - "updatedAt": "2025-11-26T07:24:02.667Z" + "updatedAt": "2025-11-29T02:44:55.140Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.668Z" + "updatedAt": "2025-11-29T02:44:55.140Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.147Z" } } }, "2c9a0b8bcdb6bc350cecc5d8a2a8571d9ab75452db549ce31e1fdb37159adb97": { "a30a7c92ea08285c067ff0984feefbb3785f4b1f14d7715bfc668fb4bbc9261f": { "ru": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.175Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.181Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.176Z" } } }, "89191e0f0f3ac7ad6fcbe90e008723be94527b1dc5730c24b0ef28b7567b621a": { "db61043ee1c3c508cdf7d9dd474714bef6965ab628e609c3b20ddf986ef02cc9": { "zh": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.220Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.229Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.231Z" } } }, "e9514b207fd2f0999e54604bcc5f81ff6fdaee6511cc23ec24b5e33bcbd7a748": { "9824c5507b882758b8df0cd7ac8ec6f8ec745839288f88d8cad0156e2ed55258": { "zh": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.221Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.197Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.199Z" } } }, "bb75403cac8908b2d1e0f7435d3c432ee901f13dfdca991fb73204120a85338c": { "0a7663696896ca536cf8c5b6b0059cce8944689bcec816f2b5c5b41720cbd804": { "zh": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.253Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.258Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.242Z" } } }, "c66448d10c048389547620c0efc34decc72e9f80bc32daa2c49d957e3c02fa1b": { "1f29d5a37e6fed39b5f9602645e28d9fa470dce74a39a6c598dbd0a16867a37c": { "ru": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.253Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.245Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.679Z" + "updatedAt": "2025-11-29T02:44:55.250Z" } } }, "7a098dff053dea397b97871863eca7199375f5d95f819134c433310d813f3ae4": { "ea322771a5ea71a865948471da4a31d3c932f43e7f418fbd44d17ba4dd564761": { "zh": { - "updatedAt": "2025-11-26T07:24:02.668Z" + "updatedAt": "2025-11-29T02:44:55.141Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.146Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.143Z" } } }, "a36c558e3cc8eb2a3b03c01a4286bfac9d72237977464d90e7395a10cf2209e0": { "94ce7d6626e94f915dc3f8c3c80748074f7c1a750f5800beccd7406817b5d19f": { "zh": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.223Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.230Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.220Z" } } }, "68ae98d78891d0611568e05de511ec72306b7b3511df399280a7ae2c79b3ee06": { "33c7517467d660435f217ea64c4bf7d1325b67636ba929b3ced122cbffac2355": { "zh": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.088Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.092Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.092Z" } } }, "561284460b1fb2a4c59ce07e83be4fee1a8ff052b66a64ff66141a296715102c": { "30382cd05cdfc447ce68389ab117d0b72fb4faf154b6c67bed6c57d0ed565d98": { "ru": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.255Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.258Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.259Z" } } }, "8b014d0b3ce023d8c15fd8c5eb2d350cacf9cf3c41dd4b69ff25dd2351d35db0": { "891d96677ae497189e4ef48d65804e3b886d35381aa01b9dd409f5c32ee066aa": { "ru": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.255Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.259Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.256Z" } } }, "82fa28546b5677c4d98f580e1e222959c159ae3d9905e0932fbfebe2ebde8218": { "5207e407e3f1eccc511c0aaa51164bd35e4d15543e26e8e004002a81d42f5b90": { "ru": { - "updatedAt": "2025-11-26T07:24:02.668Z" + "updatedAt": "2025-11-29T02:44:55.142Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.668Z" + "updatedAt": "2025-11-29T02:44:55.143Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.148Z" } } }, "9e8c51287c7f24817ec004d3002c1ce3b305ec30df1100b9d028e5ebc63461bd": { "afbdd8bf1a036d21dd54275c5ec03df46552510b37adf7a05917d6570967651d": { "ru": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.179Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.191Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.188Z" } } }, "2e86bca26b2ac693c6c25e4a60919c546b7872ba88d487d37cba83528dd4c1c0": { "82625a723fba7e62c237b3557661bd75bff3e41b4de031a888fc315f70bf8f60": { "ru": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.224Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.200Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.230Z" } } }, "03f4f6675beb54a72bd7e3d62bec8c07f1c24ef51dcd84e88ba10e86e3a5a9b7": { "eb1beb44798239cd7a4b527f6d7acf65bd7638560f8fda08cbea63789789cbab": { "ru": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.179Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.171Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.168Z" } } }, "32ffa87be40ab5f31e20d44d8997706429f8284873cee16bf953aa7c8a533e87": { "987df6e0573b5dadab1d721fb8b42546edd8a72a4c4ef547c90da774cfdc0384": { "zh": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.224Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.213Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.216Z" } } }, "a2e55a90379e6ffc005d5cc760c9bf50e3a6631ad77cd354c2d442860ad851ea": { "a0801c6bb244ad72c6b1b26969b590462545f49f3c2c06d4078fe79f62be5841": { "ru": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.180Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.184Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.173Z" } } }, "9d795e71c1e5b5159a55b2c1f0aef5b2b5ba275de3636e7962e76d9cac324863": { "e14d02d5377204ff07364b01b4777caa9edee903a191d54d14cd619978c349a5": { "ru": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.182Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.185Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.192Z" } } }, "459dcfc8cfcb0c798eda34051037eaf36f0e8bdbf413d5ca0f86faf6d1ae4e24": { "f469d58719f2670441a26ddce21a692caf6821dcb698ad90eba442b062adb5aa": { "ru": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.227Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.202Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.074Z" + "updatedAt": "2025-11-29T02:44:55.222Z" } } }, "6aaffe8268bf79e8f4faf87000cd0de5d6453e5299b00696c4cf31cfb8d96d5b": { "ddba7dec037a2fad87464c00483c81011ad76357b5c4963561b6fb33a626d74e": { "zh": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.103Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.102Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.063Z" + "updatedAt": "2025-11-29T02:44:55.091Z" } } }, "299acd2896dbdcc7fc9ec56b51b4a1990b56dd0fe41acb3e57f9cae1bd915ac7": { "99ca8337276f2850a682286f3aa13f69597377997f305892b1182845150c4e2e": { "zh": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.290Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.693Z" + "updatedAt": "2025-11-29T02:44:55.280Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.689Z" + "updatedAt": "2025-11-29T02:44:55.269Z" } } }, "3246877b14617a738e90832de040052391f7c8fc094ca44b2455eef30fbf314e": { "d6d3906022ccc3319721785ef9aa9f57093fc737336e72eddec0d952f2c844d7": { "zh": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.341Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.342Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.331Z" } } }, "3635e79a2e76bb297d10c5dd4637f4fd94275c1ba1081c959a4f02a8d8049bf6": { "69cff4cb3337c445b437475f175d0c1ab8c863e57aa050035a2284326ea56533": { "zh": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.235Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.243Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.257Z" } } }, "c3ae4d87db64f55d260d37bff7580e0a1ff638a6c1bebc984889a0f53e882bd1": { "c8ec9fc9c8400c3e7fc2098760f4d554623fe5eaab093ad69821218853b4e3b8": { "ru": { - "updatedAt": "2025-11-26T07:24:02.673Z" + "updatedAt": "2025-11-29T02:44:55.164Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.187Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.188Z" } } }, "7974b5d9fc7e953fa4aecd07c2f6f9176f90a9a89310ebe7fcb27dff7fdf734a": { "b66740bd12022ccefeb425eba94ee09c08528b3a5b347793bb597e953e4f21b2": { "ru": { - "updatedAt": "2025-11-26T07:24:12.077Z" + "updatedAt": "2025-11-29T02:44:55.236Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.254Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.252Z" } } }, "0d605e87bc30a8f48c930071ba0ea5cd2b5ebdbd6763536c9fdc4f2ed918a40d": { "8b7aa2cd071eea2c995cc398e7fb600f77f1f9a7fa0450f42449838022126464": { "ru": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.178Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.187Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.676Z" + "updatedAt": "2025-11-29T02:44:55.176Z" } } }, "51f9cca65edfee082630f0b1fb8e3a29f4ab177d7d5452a9abc2e1f9b56e3c53": { "96fa3e43effb19ba6584f2d1ae472b68548bb3a136e72cc23135e36bd3bd7b5a": { "zh": { - "updatedAt": "2025-11-26T07:24:02.697Z" + "updatedAt": "2025-11-29T02:44:55.289Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.291Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.282Z" } } }, "0f09f5442f4b4bac183a39fe7c4ebb5f27e3e93b8fbdd22c1bf04db43e598523": { "8dd4d3197218cd45163cf27ba0c5e57b39a8db91e1ae9ccb34b1ee6871418db0": { "ru": { - "updatedAt": "2025-11-26T07:24:02.668Z" + "updatedAt": "2025-11-29T02:44:55.142Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.145Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.670Z" + "updatedAt": "2025-11-29T02:44:55.148Z" } } }, "04bd894d54eb7791d6c20efe6b82643d60ba5f94079895df60cd832a967a8b72": { "b4b191db3e0a1686174b935b4a408eec87a5d10accead9bfce53f6fdb0c78147": { "ru": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.182Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.183Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.183Z" } } }, "ee7cf7082c51841ba27fc19b990495b38b92128a79d2a323ecbca6bb723f0e8e": { "7deda54447cba9acce76845c952c2c7f4ee86488c276f4a335c96e4c55dc6bcd": { "ru": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.225Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.229Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.227Z" } } }, "cbfa6856b07360063ce643d5dc0c1d3cc2418e2639de759af00c6f665fc517e4": { "0140ef2e17d32f74a3b543e6327533884c8025b049e9fdc7af2a729378577a5e": { "ru": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.226Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.226Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.076Z" + "updatedAt": "2025-11-29T02:44:55.232Z" } } }, "c91f782ae583639337bdc49114576cfdd9c9355b699a68919bf1bd023713faef": { "bec2f91a18ab29d790a84a8d99cfc87824936240769c4e0889827b57e2472e09": { "zh": { - "updatedAt": "2025-11-26T07:24:02.704Z" + "updatedAt": "2025-11-29T02:44:55.320Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.309Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.701Z" + "updatedAt": "2025-11-29T02:44:55.306Z" } } }, "abdc65a73d328d0f6587eba73db81db937a7f67106eeb840b67ebf52e35e6379": { "3d443c4abc73eddf8e334725cfa0abf5cbeb70f4475566a8d40953e253b629bc": { "zh": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.189Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.189Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.186Z" } } }, "6f7b91f9de26806b740498adc5e643a9126c17702c3c29691e1666087c366cf0": { "a1903aea52e9e31c6386a9cb6e37a8b774a6be1ff6724d1c7674a90cee7e9059": { "ru": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.190Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.190Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.068Z" + "updatedAt": "2025-11-29T02:44:55.185Z" } } }, "fae1576558dadb0c932329389ce8fbcbeee0d35379cb6c996673cd93aad35a13": { "3c3975cd182172060059f7637ba3d00c8b28a90dce27de128e912a0c986041da": { "ru": { - "updatedAt": "2025-11-26T07:24:02.682Z" + "updatedAt": "2025-11-29T02:44:55.260Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.683Z" + "updatedAt": "2025-11-29T02:44:55.261Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.681Z" + "updatedAt": "2025-11-29T02:44:55.257Z" } } }, "3f14c9de32cc2309c896fed678c5b28a7dbf39af5a00bc45e0fd013b9c4d05d5": { "30c6636556ee6c7c353538457f6b3b57a9f5c21c15e651b2997b487922e38fc3": { "ru": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.342Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.902Z" + "updatedAt": "2025-11-29T02:44:55.335Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.340Z" } } }, "6bed7e7a83ecb81ba1dd2bac10ae908f5dca2985a1372a02ea6f37edc19fb8d6": { "d69df1442a7aad94ba9096815aac2b779c3a23eed85dba10c8cf5e643215acf7": { "ru": { - "updatedAt": "2025-11-26T07:24:02.683Z" + "updatedAt": "2025-11-29T02:44:55.262Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.683Z" + "updatedAt": "2025-11-29T02:44:55.263Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.683Z" + "updatedAt": "2025-11-29T02:44:55.262Z" } } }, "3e0601c102f0cd71b8eb284da75b1cb579b66391d37fa681cf6d4bc5e1cc1d58": { "4eeb3b260eb5599be93bf2151af54a52820bc5b7145e432d1d16218f6b0c376b": { "zh": { - "updatedAt": "2025-11-26T07:24:02.670Z" + "updatedAt": "2025-11-29T02:44:55.149Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.670Z" + "updatedAt": "2025-11-29T02:44:55.151Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.671Z" + "updatedAt": "2025-11-29T02:44:55.152Z" } } }, "8b38fc05c0c3883d9a4ec8bbf5caa1bbc4260e946b23ad31bf5c97563bd88229": { "58e3bcd0e949f466dc2d6e918d912d126143beea61afa2ee594bb6cb9d60e88d": { "zh": { - "updatedAt": "2025-11-26T07:24:02.670Z" + "updatedAt": "2025-11-29T02:44:55.151Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.666Z" + "updatedAt": "2025-11-29T02:44:55.133Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.669Z" + "updatedAt": "2025-11-29T02:44:55.144Z" } } }, "11f0d2e2fe5381cbdabf5c8d911e42f98d764106a83601de0c96203590ad4cc5": { "125142acfba42f104cc8a667d2cd001ded4684ba6896567aa756cbbcdfe1e975": { "zh": { - "updatedAt": "2025-11-26T07:24:12.069Z" + "updatedAt": "2025-11-29T02:44:55.193Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.193Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.194Z" } } }, "6152b4089faf21cb920f0b0e0f015947f4aa6a6539cc24579a8054117329f175": { "58de10c3764c8ae20317dce26cff68631d85677a41b3f5dbd50c51245bb6c66d": { "zh": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.095Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.098Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.064Z" + "updatedAt": "2025-11-29T02:44:55.095Z" } } }, "bca14edd411fa9f3a8a9611aaacff6972d87258f38acd6410fdf5b4d4cdbaa55": { "6bdb09ec322273b515c242a0a196c778ff9876e649fa65392b8031cb787249d3": { "zh": { - "updatedAt": "2025-11-26T07:24:12.057Z" + "updatedAt": "2025-11-29T02:44:55.062Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.057Z" + "updatedAt": "2025-11-29T02:44:55.062Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.064Z" } } }, "90b37c7973739db627d82b16799b1a59ebcb776db33ad8298491b0bbbed6c3de": { "73ba6fad372ebd5b4ddf82f283b3e7b1f303a8f02d8ddee4e4e8d3c0290b12ee": { "zh": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.064Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.069Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.066Z" } } }, "5dcc85853637a46f967236f293c74ce6629e743899ffb1d793ba5c7ffae90dbf": { "6777f02cb4aba6cf43d71fcfd0acc7ed50b7a116661de2ebd8193b82df093941": { "zh": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.989Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:54.997Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.992Z" } } }, "094593271a536e71ddc700495b0cf0f1048d6ab6c2dad60a929934f0798430ea": { "3dd2ef060c7a1cfaa56099a332e54eba203c50d4214e0f5bf98d281ff70e8d9e": { "ru": { - "updatedAt": "2025-11-26T07:24:02.654Z" + "updatedAt": "2025-11-29T02:44:54.991Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.004Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:55.001Z" } } }, "27e2bd6338f55fdbb9b18fcf67e0a0a67489a58d4e1c0e9ebb6902b05fc36aac": { "8929ff1edb2d47de0f53425237359fc7c4a1036ef99e001d0d30c2d13140051c": { "ru": { - "updatedAt": "2025-11-26T07:24:02.655Z" + "updatedAt": "2025-11-29T02:44:54.994Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.655Z" + "updatedAt": "2025-11-29T02:44:54.996Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.653Z" + "updatedAt": "2025-11-29T02:44:54.986Z" } } }, "19a3aba2f96aa29e64f1d6662e4c6e8d99c98fade3c4d0aa0badaed1632f4c7c": { "dc4c51508caf2bb72e5375d6abe27b369e6eacb14cc00c78c196a37458e79501": { "ru": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.067Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.072Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.060Z" + "updatedAt": "2025-11-29T02:44:55.073Z" } } }, "dfad0bc3b6417f406a00ff9ef3820a57dfc8f664400a7ce0134d81da437d7e07": { "79123cc58b0a88edb3bafb181767cf704d4908d66876b9628ebccd1e31728887": { "zh": { - "updatedAt": "2025-11-26T07:24:02.655Z" + "updatedAt": "2025-11-29T02:44:54.995Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:55.000Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.652Z" + "updatedAt": "2025-11-29T02:44:54.982Z" } } }, "4b87a5344a9b716648c77706eed8254331cf4a6ce21d8a43d267f67270734d1f": { "fb4dfb8f9e647f53e63097ab00045af768eb9222f514d424b3a57634d8f3681e": { "ru": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.070Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.059Z" + "updatedAt": "2025-11-29T02:44:55.071Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.058Z" + "updatedAt": "2025-11-29T02:44:55.065Z" } } }, "f0a5d6e46b2ddd583ab900563a42b7687a1b4924afd5d0cb5260268c8952f6d0": { "3a8f69d0d17e9065a46d4d7456a503262e2f2a05ac3d4b37f49520b5f716b1c3": { "zh": { - "updatedAt": "2025-11-26T07:24:02.692Z" + "updatedAt": "2025-11-29T02:44:55.278Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.693Z" + "updatedAt": "2025-11-29T02:44:55.280Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.688Z" + "updatedAt": "2025-11-29T02:44:55.263Z" } } }, "9027438a5e9e30a2c6e8e4d197b479cebf29c05aaa3a716589f591c0ff697c0d": { "d5d6ea5e34429a4a6f22bad136f5d5eb712bbb922cae22a6c870b906c7befadf": { "zh": { - "updatedAt": "2025-11-26T07:24:18.902Z" + "updatedAt": "2025-11-29T02:44:55.334Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:18.902Z" + "updatedAt": "2025-11-29T02:44:55.335Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.706Z" + "updatedAt": "2025-11-29T02:44:55.328Z" } } }, "492b567700669f175480e40ecf1c553c463e77a0bb36e30e76eb3628c35e7db3": { "84c653bd2e6590cbd982437c2304ff4818581c1e60afb256437642c4a3dc66c5": { "ru": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.238Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.244Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.680Z" + "updatedAt": "2025-11-29T02:44:55.254Z" } } }, "dbc3d877611d9d2c9a27f2ea076decc1afc5907f3c3c02044504a307308653af": { "79b34ec963ce2ab8bc60d33c073caf0fc42c9aed7f3b97c1ed638390938960de": { "zh": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.198Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.212Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.217Z" } } }, "e5b56f33a8458d42151bcbd638d4692704a7d1f97fb2c4ed94143ff1e460a418": { "7eab19fd44668e93c10760c5fe2d6a1421e507a9cec55dfd91ed0fcab85c27f1": { "ru": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.198Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.210Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.196Z" } } }, "3f3b14a0c691ae2b5345864fd4ad20a184225db1e35ffcbd455da1aeec5f0d48": { "a9c8fa4f53951ce4026e170171a0517a80777e9037e5bb2f16eab83d3ffaa9cc": { "zh": { - "updatedAt": "2025-11-26T07:24:02.657Z" + "updatedAt": "2025-11-29T02:44:55.003Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.656Z" + "updatedAt": "2025-11-29T02:44:54.998Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.655Z" + "updatedAt": "2025-11-29T02:44:54.993Z" } } }, "5b41c30593068b713e26045c49b89ef31bda4b2d25564fc71eeafadaa3a88b3b": { "ecb137fd1463f816c7efffc5bf4c604e7cfa7735755e22327018e286ec755267": { "zh": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.309Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.313Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.699Z" + "updatedAt": "2025-11-29T02:44:55.298Z" } } }, "7c145e871f942571130b488686f2c93299c7784ad34d23a45c99e2947f75208c": { "193be2e12900fc92f5c6cf9d55c9d419bf67397ce7c166154cf4356eaee3bb11": { "zh": { - "updatedAt": "2025-11-26T07:24:02.702Z" + "updatedAt": "2025-11-29T02:44:55.310Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:02.703Z" + "updatedAt": "2025-11-29T02:44:55.313Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.700Z" + "updatedAt": "2025-11-29T02:44:55.300Z" } } }, "f5b83279dab37d495f5c4fd259883e2f59a812c65ccc8ed0c351f21a2028e710": { "caa363689f97df04d5bdb8cc80dfede581f616ede687804ff5915657268592d2": { "ru": { - "updatedAt": "2025-11-26T07:24:18.903Z" + "updatedAt": "2025-11-29T02:44:55.336Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:18.903Z" + "updatedAt": "2025-11-29T02:44:55.337Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:18.905Z" + "updatedAt": "2025-11-29T02:44:55.341Z" } } }, "bdeb28bdbd403e8a7dbfd53a18daf2d16a5ec80e2b272afff63299b084ee54d4": { "8d2b2934162408394b787a0c9376fd5fc5d3b70e883799983cb40e9cd3caec2b": { "ru": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.284Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.694Z" + "updatedAt": "2025-11-29T02:44:55.282Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.698Z" + "updatedAt": "2025-11-29T02:44:55.290Z" } } }, "6d9be1cdfeaef3b95b6937fe4da26361e0723bbb44069a88774c3f6c426953ff": { "27c7a63e2afca841ae5e7d6fe8b9f6f3c513769116043f854361c07302afa76a": { "ru": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.169Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.675Z" + "updatedAt": "2025-11-29T02:44:55.170Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:02.674Z" + "updatedAt": "2025-11-29T02:44:55.168Z" } } }, "08f3b123bce337ae576d91effb4c4e0aa8ce5818f4196baa0ba59915bd0d269e": { "a29ff4b6f7e821d9ae449a998417a11cc1c6705210186befa92aa45136de5da9": { "ru": { - "updatedAt": "2025-11-26T07:24:12.071Z" + "updatedAt": "2025-11-29T02:44:55.201Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.213Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.075Z" + "updatedAt": "2025-11-29T02:44:55.223Z" } } }, "0e3c84ac0dcb64d166a9e4cad32d3420219fe50fe305e36aa358456c172e2cf7": { "318568dae18d539030ba9900a07a5c387e0ffd38a7b84468080ad1adcdccfc39": { "ru": { - "updatedAt": "2025-11-26T07:24:02.677Z" + "updatedAt": "2025-11-29T02:44:55.241Z" }, "zh": { - "updatedAt": "2025-11-26T07:24:02.678Z" + "updatedAt": "2025-11-29T02:44:55.245Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.078Z" + "updatedAt": "2025-11-29T02:44:55.239Z" } } }, "808e737b87d86c00037ee9499555e8d37bc7fd2e51f5ef796a4a104d5f453b14": { "4719caa724ba0e2a9f5dae16a0fe1e64ccb82cd37762f0d2649a253c1acc65eb": { "zh": { - "updatedAt": "2025-11-26T07:24:12.072Z" + "updatedAt": "2025-11-29T02:44:55.210Z" }, "ru": { - "updatedAt": "2025-11-26T07:24:12.073Z" + "updatedAt": "2025-11-29T02:44:55.214Z" }, "jp": { - "updatedAt": "2025-11-26T07:24:12.070Z" + "updatedAt": "2025-11-29T02:44:55.197Z" } } }, @@ -23165,6 +24497,19 @@ "updatedAt": "2025-11-26T07:19:33.942Z" } } + }, + "66bbf0d8525a651b70357530fa66ca0f30073bb1460c57979838338b1c0d8749": { + "9a8d534c4d4974d982e6c1d6d31787e905d1215b8eade3bf1524a2e15a6fa2c0": { + "jp": { + "updatedAt": "2025-11-29T02:45:02.940Z" + }, + "ru": { + "updatedAt": "2025-11-29T02:45:02.941Z" + }, + "zh": { + "updatedAt": "2025-11-29T02:45:02.941Z" + } + } } } } diff --git a/i18n/jp/code.json b/i18n/jp/code.json index 74b4afa66a5..1e7cfdc6688 100644 --- a/i18n/jp/code.json +++ b/i18n/jp/code.json @@ -68,8 +68,8 @@ "description": "検索ページ上の検索フィールド用の ARIA ラベル" }, "theme.SearchPage.algoliaLabel": { - "message": "Algolia を使って検索", - "description": "Algolia メンションの ARIA ラベル" + "message": "Algolia 提供の検索", + "description": "Algoliaメンション用のARIAラベル" }, "theme.SearchPage.noResultsText": { "message": "結果が見つかりません", @@ -84,60 +84,60 @@ "description": "ブログサイドバーの「最近の投稿」用 ARIA ラベル" }, "theme.DocSidebarItem.toggleCollapsedCategoryAriaLabel": { - "message": "「{label}」の展開/折りたたみを切り替える", - "description": "折りたたみ式サイドバーカテゴリの切り替え用 ARIA ラベル" + "message": "折りたたみ式サイドバーのカテゴリ「{label}」を開閉する", + "description": "折りたたみ式サイドバーカテゴリを切り替えるためのARIAラベル" }, "theme.NavBar.navAriaLabel": { - "message": "ナビゲーション", + "message": "メイン", "description": "メインナビゲーションの ARIA ラベル" }, "theme.navbar.mobileLanguageDropdown.label": { - "message": "その他の言語", + "message": "言語", "description": "モバイル向け言語切り替えドロップダウンのラベル" }, "theme.blog.post.readMore": { - "message": "詳細を見る", - "description": "ブログ記事の抜粋部分に表示される、記事全文へのリンク用ラベル" + "message": "詳細はこちら", + "description": "ブログ記事の抜粋に表示され、記事全文へのリンクとして使われるラベル" }, "theme.blog.post.readMoreLabel": { - "message": "{title} の詳細を確認する", - "description": "抜粋からブログ記事の全文へのリンクに使用する ARIA ラベル" + "message": "{title} の詳細はこちら", + "description": "抜粋からブログ記事全文へのリンクに使用する ARIA ラベル" }, "theme.blog.post.readingTime.plurals": { - "message": "読了時間:約 {readingTime} 分", - "description": "「{readingTime} 分で読了」というテキストの複数形に対応したラベルです。ご利用の言語でサポートされている複数形の形を、可能な限り多く使用してください(\"|\" で区切ります)。詳しくは https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照してください。" + "message": "1分で読めます|{readingTime}分で読めます", + "description": "「{readingTime} 分で読める」というラベルの複数形用文字列。使用する言語がサポートしている複数形の種類に応じて(\"|\" で区切って)指定してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" }, "theme.docs.breadcrumbs.home": { "message": "ホーム", - "description": "パンくずリスト内のホームリンク用ARIAラベル" + "description": "パンくずリスト内のホームページ項目の ARIA ラベル" }, "theme.docs.sidebar.navAriaLabel": { "message": "ドキュメントのサイドバー", "description": "サイドバー ナビゲーションの ARIA ラベル" }, "theme.docs.sidebar.collapseButtonTitle": { - "message": "サイドバーを閉じる", - "description": "ドキュメントサイドバーの折りたたみボタン用の title 属性" + "message": "サイドバーを折りたたむ", + "description": "ドキュメントサイドバーの折りたたみボタンの title 属性" }, "theme.docs.sidebar.collapseButtonAriaLabel": { - "message": "サイドバーを閉じる", + "message": "サイドバーを折りたたむ", "description": "ドキュメントサイドバーの折りたたみボタンの title 属性" }, "theme.docs.sidebar.closeSidebarButtonAriaLabel": { - "message": "ナビゲーションを閉じる", - "description": "モバイルサイドバーの閉じるボタン用のARIAラベル" + "message": "ナビゲーションバーを閉じる", + "description": "モバイルサイドバーの閉じるボタンの ARIA ラベル" }, "theme.navbar.mobileSidebarSecondaryMenu.backButtonLabel": { "message": "← メインメニューへ戻る", - "description": "モバイルナビバーのサイドバー内にあるセカンダリメニュー(通常はドキュメントサイドバーの表示に使用)からメインメニューに戻るための「戻る」ボタンのラベル" + "description": "モバイルナビバー内のサイドバーにあるセカンダリメニュー(主にドキュメントサイドバーの表示に使用)で、メインメニューに戻るための「戻る」ボタンのラベル" }, "theme.ErrorPageContent.title": { - "message": "エラーが発生しました", - "description": "ページがクラッシュしたときに表示される代替ページのタイトル" + "message": "このページでエラーが発生しました。", + "description": "ページがクラッシュしたときに表示されるフォールバックページのタイトル" }, "theme.BackToTopButton.buttonAriaLabel": { - "message": "ページの先頭へ戻る", - "description": "「先頭に戻る」ボタンの ARIA ラベル" + "message": "ページの先頭に戻る", + "description": "「ページの先頭に戻る」ボタンの ARIA ラベル" }, "theme.blog.title": { "message": "ナレッジベース", @@ -145,283 +145,282 @@ }, "theme.blog.archive.title": { "message": "アーカイブ", - "description": "ブログアーカイブページのページタイトルとヒーロータイトル" + "description": "ブログアーカイブページのページタイトルとヒーローセクションのタイトル" }, "theme.blog.archive.description": { "message": "アーカイブ", - "description": "ブログアーカイブページ用のページおよびヒーローの説明文" + "description": "ブログアーカイブページのページおよびヒーローの説明文" }, "theme.blog.paginator.navAriaLabel": { - "message": "ブログ記事一覧ナビゲーション", - "description": "ブログページネーション用の ARIA ラベル" + "message": "ブログ一覧ページのナビゲーション", + "description": "ブログのページネーションのための ARIA ラベル" }, "theme.blog.paginator.newerEntries": { - "message": "新規記事", - "description": "新しいブログ記事ページ(前のページ)に移動するためのラベル" + "message": "新しい投稿", + "description": "新しいブログ記事のページ(前のページ)に移動するためのラベル" }, "theme.blog.paginator.olderEntries": { - "message": "以前の記事", - "description": "古いブログ記事(次のページ)に移動するためのラベル" + "message": "以前のエントリー", + "description": "古いブログ記事のページ(次ページ)に移動するためのラベル" }, "theme.blog.post.paginator.navAriaLabel": { - "message": "ブログ記事ナビゲーション", - "description": "ブログ記事ページネーション用の ARIA ラベル" + "message": "ブログ記事ページ内ナビゲーション", + "description": "ブログ記事のページネーションの ARIA ラベル" }, "theme.blog.post.paginator.newerPost": { "message": "新しい記事", - "description": "前後のブログ記事に移動するボタンのラベル" + "description": "新しい/前のブログ記事に移動するためのボタンラベル" }, "theme.blog.post.paginator.olderPost": { - "message": "過去の記事", - "description": "前後のブログ記事に移動するナビゲーションボタンのラベル" + "message": "過去の投稿", + "description": "前/次のブログ記事へ移動するためのボタンラベル" }, "theme.tags.tagsPageLink": { "message": "すべてのタグを表示", - "description": "タグリストページへのリンク用ラベル" + "description": "タグ一覧ページへのリンクのラベル" }, "theme.colorToggle.ariaLabel": { - "message": "ダークモードを切り替える(現在:{mode})", - "description": "ナビゲーションバーのカラーモード切り替えトグル用の ARIA ラベル" + "message": "ダーク/ライトモードの切り替え(現在:{mode})", + "description": "ナビバーのカラーモード切り替えボタンの ARIA ラベル" }, "theme.colorToggle.ariaLabel.mode.dark": { "message": "ダークモード", - "description": "ダークモードの名称" + "description": "ダークカラーモードの名前" }, "theme.colorToggle.ariaLabel.mode.light": { "message": "ライトモード", - "description": "ライトカラーモード用の名前" + "description": "ライトカラーモードの名前" }, "theme.docs.DocCard.categoryDescription.plurals": { - "message": "{count} 件", - "description": "生成されたインデックスでカテゴリカードに表示され、そのカテゴリ内のアイテム数を示すデフォルトの説明文" + "message": "1件|{count}件", + "description": "生成されたインデックスで、このカテゴリに含まれるアイテム数を説明するカテゴリカードの既定の説明文" }, "theme.docs.paginator.navAriaLabel": { - "message": "ドキュメントのページング", - "description": "ドキュメントのページネーション用 ARIA ラベル" + "message": "ドキュメントページ", + "description": "ドキュメントのページ送り用 ARIA ラベル" }, "theme.docs.paginator.previous": { "message": "前へ", - "description": "前のドキュメントに移動するためのラベル" + "description": "前のドキュメントに移動するためのナビゲーションラベル" }, "theme.docs.paginator.next": { "message": "次へ", "description": "次のドキュメントに移動するためのラベル" }, "theme.docs.tagDocListPageTitle.nDocsTagged": { - "message": "{count} 件のタグ付き文書", - "description": "「{count} docs tagged」の複数形ラベル。使用する言語がサポートしているだけ多くの複数形フォームを \"|\" で区切って指定してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" + "message": "タグ付けされたドキュメント 1 件|タグ付けされたドキュメント {count} 件", + "description": "「{count} docs tagged」というラベルの複数形表現です。対象言語がサポートする限り多くの複数形(「|」で区切って記述)を使用してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" }, "theme.docs.tagDocListPageTitle": { - "message": "「{tagName}」タグの付いたドキュメント {nDocsTagged} 件", + "message": "「{tagName}」のタグが付いた {nDocsTagged}", "description": "ドキュメントタグのページタイトル" }, "theme.docs.versions.unreleasedVersionLabel": { - "message": "これは、{siteTitle} のプレリリース版 {versionLabel} に関するドキュメントです。", - "description": "ユーザーが未リリース版ドキュメントを閲覧していることを示すラベル" + "message": "これは、{siteTitle} {versionLabel} バージョン向けの未公開ドキュメントです。", + "description": "ユーザーに、未リリースのドキュメントのバージョンを閲覧中であることを知らせるためのラベル" }, "theme.docs.versions.unmaintainedVersionLabel": { - "message": "このドキュメントは {siteTitle} バージョン {versionLabel} 向けのもので、現在は更新されていません。", - "description": "保守されていないドキュメントのバージョンをユーザーが閲覧しているときに表示されるラベル" + "message": "これは、現在はメンテナンスが行われていない {siteTitle} {versionLabel} のドキュメントです。", + "description": "ユーザーに、保守されていないドキュメントのバージョンを閲覧していることを知らせるラベル" }, "theme.docs.versions.latestVersionSuggestionLabel": { "message": "最新のドキュメントは {latestVersionLink}({versionLabel})を参照してください。", - "description": "ユーザーに対し最新バージョンを確認するよう促すラベル" + "description": "ユーザーに最新バージョンの確認を促すためのラベル" }, "theme.docs.versions.latestVersionLinkLabel": { "message": "最新バージョン", - "description": "「最新バージョンを提案」リンクに表示されるラベル" + "description": "最新バージョンを提案するリンクに使用されるラベル" }, "theme.docs.versionBadge.label": { - "message": "バージョン: {versionLabel}", - "description": "バージョンラベル" + "message": "バージョン: {versionLabel}" }, "theme.common.editThisPage": { "message": "このページを編集", - "description": "現在のページ編集リンクのラベル" + "description": "現在のページを編集するリンクのラベル" }, "theme.common.headingLinkTitle": { - "message": "{heading} への固定リンク", - "description": "この見出しへのリンク" + "message": "「{heading}」への直接リンク", + "description": "見出しへのリンク用タイトル" }, "theme.lastUpdated.atDate": { - "message": "{date} に", - "description": "ページの最終更新日を表示する際に使用されるテキスト" + "message": " {date} 時点で", + "description": "ページの最終更新日を示す文言" }, "theme.lastUpdated.byUser": { - "message": "{user}", - "description": "ページの最終更新者を示すラベル" + "message": " 作成者: {user}", + "description": "ページの最終更新者を示す語" }, "theme.lastUpdated.lastUpdatedAtBy": { - "message": "最終更新: {atDate}{byUser}", - "description": "ページの最終更新日時と更新者を示すテキスト" + "message": "最終更新{atDate}{byUser}", + "description": "ページの最終更新日時と更新者を表示するための文" }, "theme.tags.tagsListLabel": { "message": "タグ:", - "description": "タグリストの横に表示されるラベル" + "description": "タグリスト横のラベル" }, "theme.AnnouncementBar.closeButtonAriaLabel": { "message": "閉じる", - "description": "お知らせバーの閉じるボタンの ARIA ラベル" + "description": "お知らせバーの閉じるボタン用の ARIA ラベル" }, "theme.admonition.warning": { "message": "警告", - "description": "警告ブロック(:::warning)のデフォルトラベル" + "description": "Warning アドモニション(:::warning)に使用されるデフォルトのラベル" }, "theme.CodeBlock.copied": { - "message": "コピーしました!", - "description": "コードブロックの「コピー」ボタンのラベル" + "message": "コピー済み", + "description": "コードブロック上の「コピーしました」ボタンラベル" }, "theme.CodeBlock.copyButtonAriaLabel": { - "message": "コードをコピー", - "description": "「コードをコピー」ボタンの ARIA ラベル" + "message": "コードをクリップボードにコピー", + "description": "コードブロックをコピーするボタン用の ARIA ラベル" }, "theme.CodeBlock.copy": { "message": "コピー", - "description": "コードブロックの「コピー」ボタンのラベル" + "description": "コードブロック内の「コピー」ボタンのラベル" }, "theme.CodeBlock.wordWrapToggle": { "message": "行の折り返しを切り替える", - "description": "コードブロックの行折り返し切り替えボタンの title 属性" + "description": "コードブロックの行の折り返しを切り替えるボタンの title 属性" }, "theme.DocSidebarItem.expandCategoryAriaLabel": { - "message": "「{label}」の目次を開く", + "message": "「{label}」カテゴリを展開", "description": "サイドバーのカテゴリを展開するための ARIA ラベル" }, "theme.DocSidebarItem.collapseCategoryAriaLabel": { - "message": "{label} の目次を閉じる", - "description": "サイドバーのカテゴリを折りたたむための ARIA ラベル" + "message": "サイドバーの「{label}」カテゴリを折りたたむ", + "description": "サイドバーのカテゴリを折りたたむためのARIAラベル" }, "theme.NotFound.p1": { - "message": "お探しのページは見つかりませんでした", + "message": "お探しのものが見つかりませんでした。", "description": "404 ページの第1段落" }, "theme.NotFound.p2": { - "message": "このページへリンクしているサイトの管理者に、リンクがリンク切れであることをお知らせください。", - "description": "404ページの第2段落" + "message": "元のURLにリンクしていたサイトの管理者に連絡し、そのリンクが切れていることをお知らせください。", + "description": "404 ページの 2 番目の段落" }, "theme.TOCCollapsible.toggleButtonLabel": { - "message": "このページ内の見出し", + "message": "このページの内容", "description": "折りたたみ式目次コンポーネントのボタンに表示されるラベル" }, "theme.docs.sidebar.toggleSidebarButtonAriaLabel": { - "message": "ナビゲーションバーを開く", - "description": "モバイルナビゲーションのハンバーガーメニューボタンの ARIA ラベル" + "message": "ナビゲーションバーの表示を切り替え", + "description": "モバイルナビゲーションのハンバーガーメニューボタンに設定する ARIA ラベル" }, "theme.docs.sidebar.expandButtonTitle": { "message": "サイドバーを開く", - "description": "ドキュメントサイドバーの展開ボタン用ARIAラベルとtitle属性" + "description": "ドキュメントサイドバーの展開ボタンに使用する ARIA ラベルと title 属性" }, "theme.docs.sidebar.expandButtonAriaLabel": { - "message": "サイドバーを開く", - "description": "ドキュメントサイドバーの展開ボタンで使用される ARIA ラベルと title 属性" + "message": "サイドバーを展開", + "description": "ドキュメントサイドバーの展開ボタンに使用する ARIA ラベルと title 属性" }, "theme.SearchBar.label": { "message": "検索", "description": "検索ボタンの ARIA ラベルとプレースホルダー" }, "theme.SearchModal.searchBox.resetButtonTitle": { - "message": "クリア", - "description": "検索ボックスのリセットボタンのラベルおよび ARIA ラベル" + "message": "クエリをクリア", + "description": "検索ボックスのリセットボタン用のラベルと ARIA ラベル" }, "theme.SearchModal.searchBox.cancelButtonText": { "message": "キャンセル", "description": "検索ボックスのキャンセルボタンのラベルおよび ARIA ラベル" }, "theme.SearchModal.startScreen.recentSearchesTitle": { - "message": "最近の検索", - "description": "最近の検索のタイトル" + "message": "最近", + "description": "最近の検索の見出し" }, "theme.SearchModal.startScreen.noRecentSearchesText": { "message": "最近の検索はありません", - "description": "最近の検索がない場合に表示されるテキスト" + "description": "最近の検索がない場合に表示するテキスト" }, "theme.SearchModal.startScreen.saveRecentSearchButtonTitle": { "message": "この検索条件を保存", - "description": "「最近の検索を保存」ボタンのラベル" + "description": "最近の検索を保存するボタンのラベル" }, "theme.SearchModal.startScreen.removeRecentSearchButtonTitle": { - "message": "履歴からこの検索を削除", - "description": "[最近の検索をクリア]ボタンのラベル" + "message": "この検索を履歴から削除", + "description": "「最近の検索を削除」ボタンのラベル" }, "theme.SearchModal.startScreen.favoriteSearchesTitle": { - "message": "保存した検索", - "description": "お気に入り検索用のタイトル" + "message": "お気に入り", + "description": "お気に入り検索のタイトル" }, "theme.SearchModal.startScreen.removeFavoriteSearchButtonTitle": { - "message": "この検索をお気に入りから削除", - "description": "「お気に入り検索を削除」ボタンのラベル" + "message": "この検索条件をお気に入りから削除", + "description": "お気に入り検索を削除するボタンのラベル" }, "theme.SearchModal.errorScreen.titleText": { - "message": "検索結果を取得できませんでした", + "message": "結果を取得できませんでした", "description": "検索モーダルのエラー画面用タイトル" }, "theme.SearchModal.errorScreen.helpText": { "message": "ネットワーク接続を確認してください。", - "description": "検索モーダルのエラー画面に表示するヘルプテキスト" + "description": "検索モーダルのエラー画面に表示されるヘルプテキスト" }, "theme.SearchModal.footer.selectText": { - "message": "選択", - "description": "Enterキー押下時の動作の説明" + "message": "選択する", + "description": "Enter キーに割り当てられたアクションの説明テキスト" }, "theme.SearchModal.footer.selectKeyAriaLabel": { "message": "Enterキー", - "description": "選択を確定するための Enter キーボタンの ARIA ラベル" + "description": "選択を確定する Enter キー ボタン用の ARIA ラベル" }, "theme.SearchModal.footer.navigateText": { - "message": "実行", - "description": "上矢印キーおよび下矢印キーの動作説明テキスト" + "message": "移動する", + "description": "↑キーおよび↓キーの操作内容を説明するテキスト" }, "theme.SearchModal.footer.navigateUpKeyAriaLabel": { - "message": "上矢印キー", - "description": "ナビゲーション用の上矢印キー ボタンの ARIA ラベル" + "message": "上向き矢印", + "description": "ナビゲーションを行うための「上向き矢印キー」ボタン用の ARIA ラベル" }, "theme.SearchModal.footer.navigateDownKeyAriaLabel": { - "message": "下向き矢印キー", - "description": "ナビゲーションを操作する下向き矢印キー ボタンの ARIA ラベル" + "message": "下向きの矢印", + "description": "ナビゲーション用の下向き矢印キー ボタンの ARIA ラベル" }, "theme.SearchModal.footer.closeText": { "message": "閉じる", - "description": "Esc キー動作の説明" + "description": "Escapeキー押下時の動作を説明するテキスト" }, "theme.SearchModal.footer.closeKeyAriaLabel": { "message": "Escキー", - "description": "モーダルを閉じるための Escape キーボタン用の ARIA ラベル" + "description": "Escape キーでモーダルを閉じるボタンの ARIA ラベル" }, "theme.SearchModal.footer.searchByText": { - "message": "検索", - "description": "この文章は、検索機能が Algolia によって実現されていることを説明しています。" + "message": "検索条件", + "description": "このテキストでは、検索が Algolia によって実行されていることを説明しています。" }, "theme.SearchModal.noResultsScreen.noResultsText": { - "message": "該当する結果はありません", - "description": "このテキストは、次の検索に対して結果が見つからなかったことを示しています" + "message": "該当する結果がありません", + "description": "このテキストは、次の検索に結果がないことを説明しています。" }, "theme.SearchModal.noResultsScreen.suggestedQueryText": { - "message": "次の検索を試してください:", - "description": "以下の検索で結果が見つからなかった場合に表示される、提案検索クエリのテキスト" + "message": "検索してみてください", + "description": "次の検索で結果が見つからなかった場合に表示する、提案クエリのテキスト" }, "theme.SearchModal.noResultsScreen.reportMissingResultsText": { - "message": "もっと良い検索結果はありますか?", - "description": "ユーザーが一部の結果が抜けていると感じたときに表示されるプロンプト用のテキスト" + "message": "このクエリで結果が返るはずだとお考えですか?", + "description": "ユーザーが結果が不足していると感じたときの質問文のテキスト" }, "theme.SearchModal.noResultsScreen.reportMissingResultsLinkText": { - "message": "レポート", - "description": "不足している結果を報告するためのリンクのラベル" + "message": "ご連絡ください。", + "description": "結果が見つからないことを報告するリンク用のテキスト" }, "theme.SearchModal.placeholder": { - "message": "ドキュメントを検索", - "description": "DocSearch ポップアップモーダル内の入力フィールド用プレースホルダーテキスト" + "message": "ドキュメント検索", + "description": "DocSearch ポップアップモーダルの入力フィールドのプレースホルダー" }, "theme.blog.post.plurals": { - "message": "{count} 件", - "description": "「{count} 件の投稿」に対応する複数形ラベル。言語がサポートする複数形の形をすべて使用し(\"|\" で区切る)、詳細は https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照してください。" + "message": "1件の投稿|{count}件の投稿", + "description": "「{count} 件の投稿」に対応する複数形ラベルです。使用している言語でサポートされているだけの複数形(\"|\" で区切る)を使用してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" }, "theme.blog.tagTitle": { - "message": "「{tagName}」タグが付いた投稿が {nPosts} 件あります", + "message": "「{tagName}」タグが付いた投稿 {nPosts} 件", "description": "ブログタグページのタイトル" }, "theme.blog.author.pageTitle": { "message": "{authorName} - 投稿 {nPosts}件", - "description": "ブログ執筆者ページのタイトル" + "description": "ブログ著者向けページのタイトル" }, "theme.blog.authorsList.pageTitle": { "message": "著者", @@ -429,35 +428,35 @@ }, "theme.blog.authorsList.viewAll": { "message": "すべての著者を表示", - "description": "ブログ執筆者ページへのリンクラベル" + "description": "ブログ著者ページへのリンクのラベル" }, "theme.blog.author.noPosts": { - "message": "この著者はまだ何も投稿していません。", - "description": "まだブログ記事を投稿していない著者向けのテキスト" + "message": "この著者はまだ投稿を公開していません。", + "description": "ブログ記事をまだ投稿していない著者向けのテキスト" }, "theme.contentVisibility.unlistedBanner.title": { - "message": "非公開ページ", + "message": "非掲載ページ", "description": "限定公開コンテンツ用バナーのタイトル" }, "theme.contentVisibility.unlistedBanner.message": { - "message": "このページは非公開です。検索結果には表示されず、このページへの直接リンクを知っているユーザーのみが閲覧できます。", - "description": "限定公開コンテンツバナーのメッセージ" + "message": "このページは一覧に表示されません。検索エンジンでインデックスされず、直接のリンクを知っているユーザーだけがアクセスできます。", + "description": "限定公開コンテンツ用バナー メッセージ" }, "theme.contentVisibility.draftBanner.title": { "message": "下書きページ", - "description": "下書きコンテンツ用バナーのタイトル" + "description": "下書きコンテンツのバナータイトル" }, "theme.contentVisibility.draftBanner.message": { - "message": "このページはドラフトであり、本番環境で一般公開されているコンテンツには含まれていません。", - "description": "ドラフトコンテンツのバナーメッセージ" + "message": "このページはドラフトです。開発環境でのみ表示され、本番ビルドからは除外されます。", + "description": "ドラフトコンテンツバナーのメッセージ" }, "theme.ErrorPageContent.tryAgain": { - "message": "再試行してください。", - "description": "React のエラーバウンダリがエラーを検知した際に、レンダリングを再試行するボタンのラベル" + "message": "再試行してください", + "description": "React のエラーバウンダリがエラーをキャプチャしたときに、再度レンダリングを試行するためのボタンのラベル" }, "theme.common.skipToMainContent": { "message": "メインコンテンツへスキップ", - "description": "アクセシビリティのために使用される「コンテンツへスキップ」ラベル。キーボードの Tab/Enter による操作で、ユーザーがメインコンテンツへすばやく移動できるようにします" + "description": "キーボードの Tab/Enter 操作でメインコンテンツへすばやく移動できるようにする、アクセシビリティ対応の「コンテンツへスキップ」ラベル" }, "theme.tags.tagsPageTitle": { "message": "タグ", @@ -586,20 +585,16 @@ "description": "有用なデータセットとチュートリアル" }, "sidebar.dropdownCategories.category.Cloud": { - "message": "クラウド", - "description": "クラウド" + "message": "クラウド" }, "sidebar.dropdownCategories.category.description.Cloud": { - "message": "ClickHouse を最速でデプロイする方法", - "description": "ClickHouse を最速でデプロイする方法" + "message": "ClickHouse を最速でデプロイする方法" }, "sidebar.dropdownCategories.category.Cloud.Get Started": { - "message": "はじめに", - "description": "はじめに" + "message": "はじめに" }, "sidebar.dropdownCategories.category.Cloud.Get Started.description": { - "message": "今すぐ ClickHouse Cloud を始めましょう", - "description": "ClickHouse Cloud をすぐに使い始める" + "message": "ClickHouse Cloud をすぐに使い始める" }, "sidebar.dropdownCategories.category.Cloud.Managing Cloud": { "message": "クラウド管理", @@ -746,36 +741,28 @@ "description": "ClickHouse の運用・管理に役立つメタデータテーブル" }, "sidebar.dropdownCategories.category.Reference": { - "message": "リファレンス", - "description": "リファレンス" + "message": "リファレンス" }, "sidebar.dropdownCategories.category.description.Reference": { - "message": "ClickHouse 機能のリファレンスドキュメント", - "description": "ClickHouse 機能リファレンス" + "message": "ClickHouse 機能のリファレンスドキュメント" }, "sidebar.dropdownCategories.category.Reference.Introduction": { - "message": "概要", - "description": "はじめに" + "message": "概要" }, "sidebar.dropdownCategories.category.Reference.Introduction.description": { - "message": "ClickHouse の構文を学ぶ", - "description": "ClickHouse の構文を学ぶ" + "message": "ClickHouse の構文を学習する" }, "sidebar.dropdownCategories.category.Reference.Functions": { - "message": "関数", - "description": "関数" + "message": "関数" }, "sidebar.dropdownCategories.category.Reference.Functions.description": { - "message": "データ分析向けの数百種類の組み込み関数", - "description": "データ分析のための数百の組み込み関数" + "message": "データ分析に役立つ数百の組み込み関数" }, "sidebar.dropdownCategories.category.Reference.Engines": { - "message": "エンジン", - "description": "エンジン" + "message": "エンジン" }, "sidebar.dropdownCategories.category.Reference.Engines.description": { - "message": "データに最適なテーブルエンジンおよびデータベースエンジンを利用する", - "description": "データに適したテーブルエンジンとデータベースエンジンを選ぶ" + "message": "データに適したテーブルエンジンとデータベースエンジンを選択する" }, "sidebar.dropdownCategories.category.Reference.Other Features": { "message": "追加の機能", @@ -786,12 +773,10 @@ "description": "ClickHouse のその他の機能を詳しく見る" }, "sidebar.dropdownCategories.category.Integrations": { - "message": "連携", - "description": "連携" + "message": "連携" }, "sidebar.dropdownCategories.category.description.Integrations": { - "message": "ClickHouse 用のインテグレーション、クライアント、ドライバー", - "description": "ClickHouse のインテグレーション、クライアント、およびドライバー" + "message": "ClickHouse で利用できるインテグレーション、クライアント、ドライバー" }, "sidebar.dropdownCategories.category.Integrations.All Integrations": { "message": "すべての連携", @@ -810,12 +795,10 @@ "description": "お好みのプログラミング言語から ClickHouse を利用する" }, "sidebar.dropdownCategories.category.Integrations.ClickPipes": { - "message": "ClickPipes", - "description": "ClickPipes" + "message": "ClickPipes" }, "sidebar.dropdownCategories.category.Integrations.ClickPipes.description": { - "message": "ClickHouse へのデータ取り込みを最も簡単に行う方法", - "description": "ClickHouse にデータを取り込む最も簡単な方法" + "message": "ClickHouse へのデータ取り込みを最も簡単に行う方法" }, "sidebar.dropdownCategories.category.Integrations.Native Clients & Interfaces": { "message": "ネイティブクライアントとインターフェース", @@ -858,20 +841,16 @@ "description": "さまざまな ELT ツールで ClickHouse にデータを取り込む" }, "sidebar.dropdownCategories.category.chDB": { - "message": "chDB", - "description": "chDB" + "message": "chDB" }, "sidebar.dropdownCategories.category.description.chDB": { - "message": "chDB は、ClickHouse の組み込み版です。", - "description": "chDB は組み込み型 ClickHouse データベースです" + "message": "chDB は ClickHouse の組み込み版です" }, "sidebar.dropdownCategories.category.chDB.Learn chDB": { - "message": "chDB について詳しく知る", - "description": "chDB を学ぶ" + "message": "chDB を学ぶ" }, "sidebar.dropdownCategories.category.chDB.Learn chDB.description": { - "message": "chDB の使用方法を学ぶ", - "description": "chDB を使用する" + "message": "chDB の使用方法を学ぶ" }, "sidebar.dropdownCategories.category.chDB.Language Integrations": { "message": "言語インテグレーション", @@ -882,44 +861,34 @@ "description": "プログラミング言語向けクライアントライブラリを使用して chDB に接続する" }, "sidebar.dropdownCategories.category.chDB.Guides": { - "message": "ガイド", - "description": "ガイド" + "message": "ガイド" }, "sidebar.dropdownCategories.category.chDB.Guides.description": { - "message": "chDB の利用ガイド", - "description": "chDB の利用ガイド" + "message": "chDB を活用するためのガイド" }, "sidebar.dropdownCategories.category.About": { - "message": "概要", - "description": "概要" + "message": "概要" }, "sidebar.dropdownCategories.category.description.About": { - "message": "ClickHouse の詳細を確認する", - "description": "ClickHouse についてさらに詳しく知る" + "message": "ClickHouse の詳細を確認する" }, "sidebar.dropdownCategories.category.About.Adopters": { - "message": "実装例", - "description": "導入企業" + "message": "採用企業" }, "sidebar.dropdownCategories.category.About.Adopters.description": { - "message": "ClickHouse を導入している企業", - "description": "ClickHouse ユーザー" + "message": "ClickHouse 採用企業" }, "sidebar.dropdownCategories.category.About.Changelogs": { - "message": "変更履歴", - "description": "更新履歴" + "message": "変更履歴" }, "sidebar.dropdownCategories.category.About.Changelogs.description": { - "message": "最新の ClickHouse の変更内容を確認", - "description": "ClickHouse の最近の変更を確認" + "message": "ClickHouse の最新の変更内容を表示" }, "sidebar.dropdownCategories.category.About.Support": { - "message": "サポート", - "description": "サポート" + "message": "サポート" }, "sidebar.dropdownCategories.category.About.Support.description": { - "message": "ClickHouse エンジニアによるサポート", - "description": "ClickHouse エンジニアに相談する" + "message": "ClickHouse エンジニアによるサポート" }, "sidebar.dropdownCategories.category.About.Development and Contributing": { "message": "開発と貢献", @@ -938,408 +907,29 @@ "Report an issue": { "message": "問題を報告する" }, - "theme.DocSidebarItem.toggleCollapsedCategoryAriaLabel": { - "message": "サイドバーの折りたたみカテゴリ「{label}」の表示/非表示を切り替える", - "description": "折りたたみ式サイドバーカテゴリを切り替えるための ARIA ラベル" - }, - "theme.NavBar.navAriaLabel": { - "message": "メイン", - "description": "メインナビゲーションの ARIA ラベル" - }, - "theme.navbar.mobileLanguageDropdown.label": { - "message": "言語", - "description": "モバイル版の言語切り替えドロップダウンのラベル" - }, - "theme.NotFound.p1": { - "message": "お探しの内容が見つかりませんでした。", - "description": "404 ページの第1段落" - }, - "theme.NotFound.p2": { - "message": "元のURLへリンクしていたサイトの管理者に連絡し、そのリンクがリンク切れであることを知らせてください。", - "description": "404ページの2番目の段落" - }, - "theme.blog.post.readingTime.plurals": { - "message": "読了時間1分|読了時間{readingTime}分", - "description": "「{readingTime} 分で読める」のラベルを複数形にしたものです。あなたの言語がサポートする複数形の形ごとにラベルを用意し、\"|\" で区切って指定してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" - }, - "theme.docs.breadcrumbs.home": { - "message": "ホームページ", - "description": "パンくずリスト内のホームページ用の ARIA ラベル" - }, - "theme.blog.post.readMore": { - "message": "続きを読む", - "description": "ブログ記事の抜粋に表示され、完全な記事へのリンクとして使われるラベル" - }, - "theme.blog.post.readMoreLabel": { - "message": "{title} について詳しく見る", - "description": "抜粋からブログ記事の全文へのリンク用の ARIA ラベル" - }, - "theme.docs.sidebar.collapseButtonTitle": { - "message": "サイドバーを折りたたむ", - "description": "ドキュメントサイドバーの折りたたみボタンの title 属性" - }, - "theme.docs.sidebar.collapseButtonAriaLabel": { - "message": "サイドバーを閉じる", - "description": "ドキュメントサイドバーの折りたたみボタンに設定する title 属性" - }, - "theme.docs.sidebar.navAriaLabel": { - "message": "ドキュメントのサイドバー", - "description": "サイドバー ナビゲーションの ARIA ラベル" - }, - "theme.docs.sidebar.closeSidebarButtonAriaLabel": { - "message": "ナビゲーションバーを閉じる", - "description": "モバイルサイドバーの閉じるボタンの ARIA ラベル" - }, - "theme.navbar.mobileSidebarSecondaryMenu.backButtonLabel": { - "message": "← メインメニューに戻る", - "description": "モバイル用ナビバーのサイドバーにあるセカンダリメニュー(主にドキュメントサイドバーの表示に使用)で、メインメニューへ戻るための戻るボタンのラベル" - }, - "theme.ErrorPageContent.title": { - "message": "このページでエラーが発生しました。", - "description": "ページクラッシュ時に表示されるフォールバックページのタイトル" - }, - "theme.BackToTopButton.buttonAriaLabel": { - "message": "ページトップに戻る", - "description": "「ページの先頭に戻る」ボタンのARIAラベル" - }, - "theme.blog.archive.title": { - "message": "アーカイブ", - "description": "ブログアーカイブページのページタイトルおよびヒーロータイトル" - }, - "theme.blog.archive.description": { - "message": "アーカイブ", - "description": "ブログアーカイブページのページ全体およびヒーローセクションの説明文" - }, - "theme.blog.paginator.navAriaLabel": { - "message": "ブログ一覧ページのナビゲーション", - "description": "ブログのページネーション用の ARIA ラベル" - }, - "theme.blog.paginator.newerEntries": { - "message": "新しい投稿", - "description": "新しいブログ記事のページ(前のページ)に移動するためのラベル" - }, - "theme.blog.paginator.olderEntries": { - "message": "過去のエントリ", - "description": "古いブログ記事のページ(次のページ)に移動するためのラベル" - }, - "theme.blog.post.paginator.navAriaLabel": { - "message": "ブログ記事ページのナビゲーション", - "description": "ブログ記事のページネーションに使用する ARIA ラベル" - }, - "theme.blog.post.paginator.newerPost": { - "message": "新しい記事", - "description": "新しい/前のブログ記事へ移動するためのボタンラベル" - }, - "theme.blog.post.paginator.olderPost": { - "message": "以前の投稿", - "description": "前/次のブログ記事へ移動するためのボタンラベル" - }, - "theme.tags.tagsPageLink": { - "message": "すべてのタグを表示", - "description": "タグ一覧ページへのリンクのラベル" - }, - "theme.colorToggle.ariaLabel": { - "message": "ダークモード/ライトモードを切り替える(現在:{mode})", - "description": "ナビゲーションバーのカラーモード切り替えトグル用の ARIA ラベル" - }, - "theme.colorToggle.ariaLabel.mode.dark": { - "message": "ダークモード", - "description": "ダークカラーモードの名称" - }, - "theme.colorToggle.ariaLabel.mode.light": { - "message": "ライトモード", - "description": "ライトカラーモードの名前" - }, - "theme.docs.DocCard.categoryDescription.plurals": { - "message": "1 件|{count} 件", - "description": "自動生成されたインデックスで、このカテゴリに含まれる項目数を示すカテゴリカードの既定の説明文" - }, - "theme.docs.paginator.navAriaLabel": { - "message": "ドキュメントページ", - "description": "ドキュメントページネーション用の ARIA ラベル" - }, - "theme.docs.paginator.previous": { - "message": "前へ", - "description": "前のドキュメントに戻るためのラベル" - }, - "theme.docs.paginator.next": { - "message": "次へ", - "description": "次のドキュメントに進むためのラベル" - }, - "theme.docs.tagDocListPageTitle.nDocsTagged": { - "message": "1 件のドキュメントにタグ付け済み|{count} 件のドキュメントにタグ付け済み", - "description": "「{count} docs tagged」というラベルの複数形。使用する言語がサポートする複数形の形の種類だけ、\"|\" で区切って指定してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" - }, - "theme.docs.tagDocListPageTitle": { - "message": "「{tagName}」タグ付きの{nDocsTagged}", - "description": "ドキュメントタグのページタイトル" - }, - "theme.docs.versionBadge.label": { - "message": "バージョン: {versionLabel}" - }, - "theme.docs.versions.unreleasedVersionLabel": { - "message": "これは、{siteTitle} の {versionLabel} バージョンに関する未リリースのドキュメントです。", - "description": "ユーザーに未リリースのドキュメントバージョンを閲覧中であることを知らせるラベル" - }, - "theme.docs.versions.unmaintainedVersionLabel": { - "message": "これは、{siteTitle} {versionLabel} 向けのドキュメントで、現在は積極的にはメンテナンスされていません。", - "description": "ユーザーに、閲覧中のドキュメントがメンテナンスされていないバージョンであることを知らせるためのラベル" - }, - "theme.docs.versions.latestVersionSuggestionLabel": { - "message": "最新のドキュメントは、{latestVersionLink}({versionLabel})を参照してください。", - "description": "ユーザーに最新バージョンの確認を促すラベル" - }, - "theme.docs.versions.latestVersionLinkLabel": { - "message": "最新バージョン", - "description": "最新バージョンの提案リンク用のラベル" - }, - "theme.common.editThisPage": { - "message": "このページを編集", - "description": "現在のページを編集するリンクのラベル" - }, - "theme.common.headingLinkTitle": { - "message": "{heading} への直接リンク", - "description": "見出しリンクのタイトル" - }, - "theme.lastUpdated.atDate": { - "message": " {date} に", - "description": "ページの最終更新日を示す文言" - }, - "theme.lastUpdated.byUser": { - "message": " 著者: {user}", - "description": "ページの最終更新者を示す文言" - }, - "theme.lastUpdated.lastUpdatedAtBy": { - "message": "最終更新日時{atDate}{byUser}", - "description": "ページの最終更新日時と更新者を表示するための文言" - }, - "theme.tags.tagsListLabel": { - "message": "タグ:", - "description": "タグリストの横にあるラベル" - }, - "theme.admonition.warning": { - "message": "警告", - "description": "Warning アドモニション (:::warning) に使用されるデフォルトのラベル" - }, - "theme.AnnouncementBar.closeButtonAriaLabel": { - "message": "閉じる", - "description": "お知らせバーの閉じるボタンの ARIA ラベル" - }, - "theme.CodeBlock.copied": { - "message": "コピーしました", - "description": "コードブロック上の「コピー済み」ボタンのラベル" - }, - "theme.CodeBlock.copyButtonAriaLabel": { - "message": "コードをクリップボードにコピー", - "description": "コードブロックをコピーするボタン用の ARIA ラベル" - }, - "theme.CodeBlock.copy": { - "message": "コピー", - "description": "コードブロックのコピーボタンのラベル" - }, - "theme.CodeBlock.wordWrapToggle": { - "message": "行の折り返しを切り替える", - "description": "コードブロックの行折り返しを切り替えるボタンの title 属性" - }, - "theme.DocSidebarItem.expandCategoryAriaLabel": { - "message": "サイドバーの「{label}」カテゴリを展開する", - "description": "サイドバーのカテゴリを展開するための ARIA ラベル" - }, - "theme.DocSidebarItem.collapseCategoryAriaLabel": { - "message": "サイドバーの「{label}」カテゴリを折りたたむ", - "description": "サイドバーのカテゴリを折りたたむための ARIA ラベル" - }, - "theme.TOCCollapsible.toggleButtonLabel": { - "message": "このページの内容", - "description": "折りたたみ式目次コンポーネントのボタンに表示されるラベル" - }, - "theme.docs.sidebar.toggleSidebarButtonAriaLabel": { - "message": "ナビゲーションバーの表示切り替え", - "description": "モバイルナビゲーションのハンバーガーメニューボタンのARIAラベル" - }, - "theme.docs.sidebar.expandButtonTitle": { - "message": "サイドバーを展開", - "description": "ドキュメントサイドバーの展開ボタンに使用する ARIA ラベルおよび title 属性" - }, - "theme.docs.sidebar.expandButtonAriaLabel": { - "message": "サイドバーを展開", - "description": "ドキュメントサイドバーの展開ボタンに設定する ARIA ラベルおよび title 属性" - }, - "theme.SearchPage.algoliaLabel": { - "message": "Algolia による検索", - "description": "Algolia メンション用の ARIA ラベル" - }, - "theme.SearchBar.label": { - "message": "検索", - "description": "検索ボタンの ARIA ラベルとプレースホルダー" - }, - "theme.SearchModal.searchBox.resetButtonTitle": { - "message": "クエリをクリア", - "description": "検索ボックスのリセットボタンのラベルと ARIA ラベル" - }, - "theme.SearchModal.searchBox.cancelButtonText": { - "message": "キャンセル", - "description": "検索ボックスのキャンセルボタンのラベルおよび ARIA ラベル" - }, - "theme.SearchModal.startScreen.recentSearchesTitle": { - "message": "最近", - "description": "最近の検索の見出し" - }, - "theme.SearchModal.startScreen.noRecentSearchesText": { - "message": "最近の検索はありません", - "description": "最近の検索がない場合に表示するテキスト" - }, - "theme.SearchModal.startScreen.saveRecentSearchButtonTitle": { - "message": "この検索条件を保存", - "description": "「最近の検索を保存」ボタンのラベル" - }, - "theme.SearchModal.startScreen.removeRecentSearchButtonTitle": { - "message": "この検索を履歴から削除", - "description": "最近の検索の削除ボタンのラベル" - }, - "theme.SearchModal.startScreen.favoriteSearchesTitle": { - "message": "お気に入り", - "description": "お気に入り検索のタイトル" - }, - "theme.SearchModal.startScreen.removeFavoriteSearchButtonTitle": { - "message": "お気に入りからこの検索を削除", - "description": "お気に入り検索の削除ボタンのラベル" - }, - "theme.SearchModal.errorScreen.titleText": { - "message": "結果を取得できませんでした", - "description": "検索モーダルのエラー画面タイトル" - }, - "theme.SearchModal.errorScreen.helpText": { - "message": "ネットワーク接続を確認してください。", - "description": "検索モーダルのエラー画面に表示されるヘルプテキスト" - }, - "theme.SearchModal.footer.selectText": { - "message": "選択する", - "description": "Enterキー押下時の動作の説明文" - }, - "theme.SearchModal.footer.selectKeyAriaLabel": { - "message": "Enterキー", - "description": "選択を確定する Enter キー ボタンの ARIA ラベル" - }, - "theme.SearchModal.footer.navigateText": { - "message": "移動する", - "description": "↑キーおよび↓キーの操作内容を説明するテキスト" - }, - "theme.SearchModal.footer.navigateUpKeyAriaLabel": { - "message": "上矢印", - "description": "ナビゲーション用の上矢印キー ボタンの ARIA ラベル" - }, - "theme.SearchModal.footer.navigateDownKeyAriaLabel": { - "message": "下向きの矢印", - "description": "ナビゲーションを開始する下向き矢印キー ボタンの ARIA ラベル" - }, - "theme.SearchModal.footer.closeText": { - "message": "閉じる", - "description": "Escape キー押下時の動作を説明するテキスト" - }, - "theme.SearchModal.footer.closeKeyAriaLabel": { - "message": "Escキー", - "description": "モーダルを閉じるための Esc キーボタンの ARIA ラベル" - }, - "theme.SearchModal.footer.searchByText": { - "message": "検索方法", - "description": "このテキストでは、検索に Algolia を使用していることを説明しています。" - }, - "theme.SearchModal.noResultsScreen.noResultsText": { - "message": "の検索結果はありません", - "description": "このテキストは、次の検索に対する結果が存在しないことを説明しています" - }, - "theme.SearchModal.noResultsScreen.suggestedQueryText": { - "message": "次のキーワードで検索してみてください", - "description": "次の検索で結果が見つからなかった場合に表示する、提案クエリのテキスト" - }, - "theme.SearchModal.noResultsScreen.reportMissingResultsText": { - "message": "このクエリで結果が返るはずだとお考えですか?", - "description": "ユーザーが結果が不足していると感じたときに表示される質問のテキスト" - }, - "theme.SearchModal.noResultsScreen.reportMissingResultsLinkText": { - "message": "ご連絡ください。", - "description": "不足している結果を報告するためのリンクテキスト" - }, - "theme.SearchModal.placeholder": { - "message": "ドキュメントを検索", - "description": "DocSearch のポップアップモーダルにある入力フィールドのプレースホルダー" - }, - "theme.blog.post.plurals": { - "message": "1件の投稿|{count}件の投稿", - "description": "「{count} 件の投稿」の複数形用ラベルです。使用言語がサポートする複数形のパターンをすべて(\"|\" で区切って)指定してください(https://www.unicode.org/cldr/cldr-aux/charts/34/supplemental/language_plural_rules.html を参照)。" - }, - "theme.blog.tagTitle": { - "message": "「{tagName}」タグが付いた投稿 {nPosts} 件", - "description": "ブログタグ用ページのタイトル" - }, - "theme.blog.author.pageTitle": { - "message": "{authorName} - 投稿 {nPosts}件", - "description": "ブログ著者ページのタイトル" - }, - "theme.blog.authorsList.pageTitle": { - "message": "著者", - "description": "著者ページのタイトル" - }, - "theme.blog.authorsList.viewAll": { - "message": "すべての著者を表示", - "description": "ブログ著者ページへのリンクのラベル" - }, - "theme.blog.author.noPosts": { - "message": "この著者はまだ投稿を作成していません。", - "description": "ブログ投稿が 0 件の著者向けテキスト" - }, - "theme.contentVisibility.unlistedBanner.title": { - "message": "非公開ページ", - "description": "非公開コンテンツバナーのタイトル" - }, - "theme.contentVisibility.unlistedBanner.message": { - "message": "このページは非公開設定になっています。検索エンジンによってインデックスされることはなく、このページへの直接リンクを知っているユーザーだけがアクセスできます。", - "description": "非公開コンテンツバナーのメッセージ" - }, - "theme.contentVisibility.draftBanner.title": { - "message": "下書きページ", - "description": "下書きコンテンツバナーのタイトル" - }, - "theme.contentVisibility.draftBanner.message": { - "message": "このページはドラフトです。開発環境でのみ表示され、本番環境のビルドには含まれません。", - "description": "下書きコンテンツ用バナーのメッセージ" - }, - "theme.ErrorPageContent.tryAgain": { - "message": "再試行", - "description": "React のエラーバウンダリがエラーを捕捉した際に、再レンダリングを試みるボタンのラベル" - }, - "theme.common.skipToMainContent": { - "message": "メインコンテンツへスキップ", - "description": "アクセシビリティ向上のために使用される「コンテンツへスキップ」ラベルで、キーボードの Tab/Enter 操作によってメインコンテンツへ素早く移動できるようにします" - }, - "theme.tags.tagsPageTitle": { - "message": "タグ", - "description": "タグ一覧ページのタイトル" - }, "sidebar.dropdownCategories.category.Get started": { "message": "はじめに" }, "sidebar.dropdownCategories.category.description.Get started": { - "message": "ClickHouse の使い方を学ぶ" + "message": "ClickHouse の使い方を習得する" }, "sidebar.dropdownCategories.category.Get started.Introduction": { - "message": "イントロダクション" + "message": "概要" }, "sidebar.dropdownCategories.category.Get started.Introduction.description": { - "message": "ClickHouse の紹介" + "message": "ClickHouse の概要" }, "sidebar.dropdownCategories.category.Get started.Concepts": { - "message": "コンセプト" + "message": "基本概念" }, "sidebar.dropdownCategories.category.Get started.Concepts.description": { - "message": "知っておくべき中核概念" + "message": "理解しておくべき基本概念" }, "sidebar.dropdownCategories.category.Get started.Starter guides": { - "message": "スターターガイド" + "message": "入門ガイド" }, "sidebar.dropdownCategories.category.Get started.Starter guides.description": { - "message": "ClickHouse を学ぶならここから" + "message": "ClickHouse を学ぶなら、まずはここから" }, "sidebar.dropdownCategories.category.Get started.Best practices": { "message": "ベストプラクティス" @@ -1348,16 +938,16 @@ "message": "ClickHouse のベストプラクティスに従う" }, "sidebar.dropdownCategories.category.Get started.Migration guides": { - "message": "マイグレーションガイド" + "message": "移行ガイド" }, "sidebar.dropdownCategories.category.Get started.Migration guides.description": { "message": "データベースを ClickHouse に移行する" }, "sidebar.dropdownCategories.category.Get started.Use case guides": { - "message": "ユースケースガイド" + "message": "ユースケース別ガイド" }, "sidebar.dropdownCategories.category.Get started.Use case guides.description": { - "message": "ClickHouse の一般的なユースケースガイド" + "message": "ClickHouse の代表的なユースケースガイド" }, "sidebar.dropdownCategories.category.Get started.Example datasets": { "message": "サンプルデータセット" @@ -1366,28 +956,16 @@ "message": "役立つデータセットとチュートリアル" }, "sidebar.dropdownCategories.category.Get started.Tips and community wisdom": { - "message": "ヒントとコミュニティの知恵" + "message": "ヒントとコミュニティからの知見" }, "sidebar.dropdownCategories.category.Get started.Tips and community wisdom.description": { "message": "コミュニティからのヒントとコツ" }, - "sidebar.dropdownCategories.category.Cloud": { - "message": "Cloud" - }, - "sidebar.dropdownCategories.category.description.Cloud": { - "message": "ClickHouse をデプロイする最速の方法" - }, - "sidebar.dropdownCategories.category.Cloud.Get Started": { - "message": "はじめる" - }, - "sidebar.dropdownCategories.category.Cloud.Get Started.description": { - "message": "ClickHouse Cloud をすぐに始める" - }, "sidebar.dropdownCategories.category.Cloud.Features": { "message": "機能" }, "sidebar.dropdownCategories.category.Cloud.Features.description": { - "message": "ClickHouse Cloud が提供する機能" + "message": "ClickHouse Cloud の提供機能" }, "sidebar.dropdownCategories.category.Cloud.Guides": { "message": "ガイド" @@ -1399,52 +977,52 @@ "message": "リファレンス" }, "sidebar.dropdownCategories.category.Cloud.Reference.description": { - "message": "ClickHouse Cloud のリファレンスドキュメント" + "message": "ClickHouse Cloud リファレンスドキュメント" }, "sidebar.dropdownCategories.category.Manage data": { - "message": "データ管理" + "message": "データを管理する" }, "sidebar.dropdownCategories.category.description.Manage data": { - "message": "ClickHouse でデータを管理する方法" + "message": "ClickHouse でのデータ管理方法" }, "sidebar.dropdownCategories.category.Manage data.Core data concepts": { - "message": "コアデータコンセプト" + "message": "データの基本概念" }, "sidebar.dropdownCategories.category.Manage data.Core data concepts.description": { - "message": "ClickHouse の内部概念を理解する" + "message": "ClickHouse の内部的な概念を理解する" }, "sidebar.dropdownCategories.category.Manage data.Updating data": { "message": "データの更新" }, "sidebar.dropdownCategories.category.Manage data.Updating data.description": { - "message": "ClickHouse でデータを更新・置換する" + "message": "ClickHouse のデータの更新と置換" }, "sidebar.dropdownCategories.category.Manage data.Deleting data": { "message": "データの削除" }, "sidebar.dropdownCategories.category.Manage data.Deleting data.description": { - "message": "ClickHouse でデータを削除する" + "message": "ClickHouse でのデータ削除" }, "sidebar.dropdownCategories.category.Manage data.Data modeling": { "message": "データモデリング" }, "sidebar.dropdownCategories.category.Manage data.Data modeling.description": { - "message": "スキーマとデータモデルを最適化する" + "message": "スキーマとデータモデルの最適化" }, "sidebar.dropdownCategories.category.Manage data.Performance and optimizations": { "message": "パフォーマンスと最適化" }, "sidebar.dropdownCategories.category.Manage data.Performance and optimizations.description": { - "message": "ClickHouse を最適化するためのガイド" + "message": "ClickHouse の最適化に役立つガイド" }, "sidebar.dropdownCategories.category.Server admin": { - "message": "サーバー管理" + "message": "サーバー管理者" }, "sidebar.dropdownCategories.category.description.Server admin": { "message": "ClickHouse の管理とデプロイ" }, "sidebar.dropdownCategories.category.Server admin.Deployments and scaling": { - "message": "デプロイとスケーリング" + "message": "デプロイメントとスケーリング" }, "sidebar.dropdownCategories.category.Server admin.Deployments and scaling.description": { "message": "ClickHouse をデプロイする方法" @@ -1453,16 +1031,16 @@ "message": "セキュリティと認証" }, "sidebar.dropdownCategories.category.Server admin.Security and authentication.description": { - "message": "ClickHouse デプロイメントを保護する" + "message": "ClickHouse デプロイメントをセキュアにする" }, "sidebar.dropdownCategories.category.Server admin.Settings": { "message": "設定" }, "sidebar.dropdownCategories.category.Server admin.Settings.description": { - "message": "ClickHouse を設定する" + "message": "ClickHouse の設定" }, "sidebar.dropdownCategories.category.Server admin.Tools and utilities": { - "message": "ツールとユーティリティ" + "message": "ツールおよびユーティリティ" }, "sidebar.dropdownCategories.category.Server admin.Tools and utilities.description": { "message": "ClickHouse を管理するためのツール" @@ -1471,115 +1049,79 @@ "message": "システムテーブル" }, "sidebar.dropdownCategories.category.Server admin.System tables.description": { - "message": "ClickHouse を管理するためのメタデータテーブル" - }, - "sidebar.dropdownCategories.category.Reference": { - "message": "リファレンス" - }, - "sidebar.dropdownCategories.category.description.Reference": { - "message": "ClickHouse 機能のリファレンスドキュメント" - }, - "sidebar.dropdownCategories.category.Reference.Introduction": { - "message": "イントロダクション" - }, - "sidebar.dropdownCategories.category.Reference.Introduction.description": { - "message": "ClickHouse の構文を学ぶ" - }, - "sidebar.dropdownCategories.category.Reference.Functions": { - "message": "関数" - }, - "sidebar.dropdownCategories.category.Reference.Functions.description": { - "message": "データ分析に役立つ数百の組み込み関数" - }, - "sidebar.dropdownCategories.category.Reference.Engines": { - "message": "エンジン" - }, - "sidebar.dropdownCategories.category.Reference.Engines.description": { - "message": "データに適したテーブルおよびデータベースエンジンを使用する" - }, - "sidebar.dropdownCategories.category.Integrations": { - "message": "インテグレーション" - }, - "sidebar.dropdownCategories.category.description.Integrations": { - "message": "ClickHouse で使用するインテグレーション、クライアント、ドライバー" + "message": "ClickHouse の管理を支援するメタデータテーブル" }, "sidebar.dropdownCategories.category.Integrations.All integrations": { - "message": "すべてのインテグレーション" + "message": "すべての統合" }, "sidebar.dropdownCategories.category.Integrations.All integrations.description": { "message": "ClickHouse を他のデータベースやアプリケーションと統合する" }, "sidebar.dropdownCategories.category.Integrations.Language clients": { - "message": "言語クライアント" + "message": "各種言語向けクライアント" }, "sidebar.dropdownCategories.category.Integrations.Language clients.description": { - "message": "お好みの言語で ClickHouse を使用する" - }, - "sidebar.dropdownCategories.category.Integrations.ClickPipes": { - "message": "ClickPipes" - }, - "sidebar.dropdownCategories.category.Integrations.ClickPipes.description": { - "message": "ClickHouse にデータを取り込む最も簡単な方法" + "message": "ClickHouse はお好みの言語から利用できます" }, "sidebar.dropdownCategories.category.Integrations.Native clients & interfaces": { "message": "ネイティブクライアントとインターフェース" }, "sidebar.dropdownCategories.category.Integrations.Native clients & interfaces.description": { - "message": "ClickHouse に接続するクライアントとインターフェースを選択する" + "message": "ClickHouse に接続するためのクライアントとインターフェースを選択する" }, "sidebar.dropdownCategories.category.Integrations.Data sources": { "message": "データソース" }, "sidebar.dropdownCategories.category.Integrations.Data sources.description": { - "message": "お好みのソースから ClickHouse にデータを読み込む" + "message": "お好みのソースから ClickHouse にデータを取り込む" }, "sidebar.dropdownCategories.category.Integrations.Data visualization": { - "message": "データ可視化" + "message": "データの可視化" }, "sidebar.dropdownCategories.category.Integrations.Data visualization.description": { - "message": "ClickHouse をお好みの可視化ツールに接続する" + "message": "ClickHouse をお気に入りの可視化ツールに接続する" }, "sidebar.dropdownCategories.category.Integrations.Data formats": { - "message": "データフォーマット" + "message": "データ形式" }, "sidebar.dropdownCategories.category.Integrations.Data formats.description": { - "message": "ClickHouse がサポートするデータフォーマットを探索する" + "message": "ClickHouse がサポートするデータ形式" }, "sidebar.dropdownCategories.category.Integrations.Data ingestion": { - "message": "データ取り込み" + "message": "データインジェスト" }, "sidebar.dropdownCategories.category.Integrations.Data ingestion.description": { - "message": "さまざまな ELT ツールで ClickHouse にデータを取り込む" + "message": "多様な ELT ツールを活用して ClickHouse にデータを取り込む" }, "sidebar.dropdownCategories.category.ClickStack": { "message": "ClickStack" }, "sidebar.dropdownCategories.category.description.ClickStack": { - "message": "ClickStack - ClickHouse 可観測性スタック" + "message": "ClickStack - ClickHouse オブザーバビリティ スタック" }, "sidebar.dropdownCategories.category.ClickStack.Getting started": { "message": "はじめに" }, "sidebar.dropdownCategories.category.ClickStack.Getting started.description": { - "message": "ClickStack を始める" + "message": "ClickStack の始め方" }, "sidebar.dropdownCategories.category.ClickStack.Sample datasets": { "message": "サンプルデータセット" }, "sidebar.dropdownCategories.category.ClickStack.Sample datasets.description": { - "message": "サンプルデータセットで ClickStack を学ぶ" + "message": "サンプルデータセットを使って ClickStack を学ぶ" }, "sidebar.dropdownCategories.category.ClickStack.Architecture": { "message": "アーキテクチャ" }, "sidebar.dropdownCategories.category.ClickStack.Architecture.description": { - "message": "ClickStack のアーキテクチャに慣れる" + "message": "ClickStack のアーキテクチャを把握する" }, "sidebar.dropdownCategories.category.ClickStack.Deployment": { "message": "デプロイメント" }, "sidebar.dropdownCategories.category.ClickStack.Deployment.description": { - "message": "ClickStack のデプロイメントモードを選択する" + "message": "ClickStack のデプロイ方式を選択する" }, "sidebar.dropdownCategories.category.ClickStack.Ingesting data": { "message": "データの取り込み" @@ -1588,144 +1130,33 @@ "message": "ClickStack にデータを取り込む" }, "sidebar.dropdownCategories.category.ClickStack.Configuration options": { - "message": "設定オプション" + "message": "構成オプション" }, "sidebar.dropdownCategories.category.ClickStack.Configuration options.description": { - "message": "本番環境で ClickStack をデプロイする" + "message": "本番環境での ClickStack のデプロイ" }, "sidebar.dropdownCategories.category.ClickStack.Production": { "message": "本番環境" }, "sidebar.dropdownCategories.category.ClickStack.Production.description": { - "message": "本番環境で ClickStack をデプロイする" + "message": "本番環境に ClickStack をデプロイする" }, "sidebar.dropdownCategories.category.ClickStack.Integration examples": { - "message": "インテグレーション例" + "message": "統合の例" }, "sidebar.dropdownCategories.category.ClickStack.Integration examples.description": { - "message": "インテグレーションクイックスタートガイド" - }, - "sidebar.dropdownCategories.category.chDB": { - "message": "chDB" - }, - "sidebar.dropdownCategories.category.description.chDB": { - "message": "chDB は ClickHouse の組み込みバージョン" - }, - "sidebar.dropdownCategories.category.chDB.Learn chDB": { - "message": "chDB を学ぶ" - }, - "sidebar.dropdownCategories.category.chDB.Learn chDB.description": { - "message": "chDB の使い方を学ぶ" + "message": "インテグレーション用クイックスタートガイド" }, "sidebar.dropdownCategories.category.chDB.Language integrations": { - "message": "言語インテグレーション" + "message": "言語別インテグレーション" }, "sidebar.dropdownCategories.category.chDB.Language integrations.description": { - "message": "言語クライアントを使用して chDB に接続する" - }, - "sidebar.dropdownCategories.category.chDB.Guides": { - "message": "ガイド" - }, - "sidebar.dropdownCategories.category.chDB.Guides.description": { - "message": "chDB を使用するためのガイド" - }, - "sidebar.dropdownCategories.category.About": { - "message": "について" - }, - "sidebar.dropdownCategories.category.description.About": { - "message": "ClickHouse についてもっと知る" - }, - "sidebar.dropdownCategories.category.About.Adopters": { - "message": "採用企業" - }, - "sidebar.dropdownCategories.category.About.Adopters.description": { - "message": "ClickHouse 採用企業" - }, - "sidebar.dropdownCategories.category.About.Changelogs": { - "message": "変更履歴" - }, - "sidebar.dropdownCategories.category.About.Changelogs.description": { - "message": "ClickHouse の最新の変更を確認する" - }, - "sidebar.dropdownCategories.category.About.Support": { - "message": "サポート" - }, - "sidebar.dropdownCategories.category.About.Support.description": { - "message": "ClickHouse エンジニアからサポートを受ける" + "message": "プログラミング言語クライアントを使用して chDB に接続する" }, "sidebar.dropdownCategories.category.About.Development and contributing": { - "message": "開発と貢献" + "message": "開発および貢献" }, "sidebar.dropdownCategories.category.About.Development and contributing.description": { - "message": "ClickHouse に貢献する方法を学ぶ" - }, - "homepage.hero.description": { - "message": "ガイド、リファレンスドキュメント、ビデオを通じて ClickHouse の使い方を学ぶ" - }, - "homepage.hero.quickStart": { - "message": "クイックスタート" - }, - "homepage.hero.getStartedCloud": { - "message": "Cloud を始める" - }, - "homepage.hero.installLocally": { - "message": "ローカルにインストール" - }, - "homepage.hero.getStartedOSS": { - "message": "OSS を始める" - }, - "homepage.search.title": { - "message": "ドキュメントを検索" - }, - "homepage.connect.title": { - "message": "ClickHouse に接続" - }, - "homepage.connect.description": { - "message": "数分でアプリケーションを ClickHouse に接続" - }, - "homepage.connect.viewAll": { - "message": "すべてのクライアントとドライバーを見る" - }, - "homepage.connect.clickhouseCli": { - "message": "ClickHouse CLI" - }, - "homepage.connect.cloudSqlConsole": { - "message": "Cloud SQL コンソール" - }, - "homepage.migrate.title": { - "message": "ClickHouse に移行" - }, - "homepage.migrate.description": { - "message": "他のデータベース、データウェアハウス、オブジェクトストレージからデータを読み込む" - }, - "homepage.migrate.viewAll": { - "message": "すべての統合を見る" - }, - "homepage.deploy.title": { - "message": "ClickHouse をデプロイ" - }, - "homepage.deploy.description": { - "message": "ClickHouse をクラウドまたは独自のインフラストラクチャにデプロイ" - }, - "homepage.deploy.cloud": { - "message": "Cloud" - }, - "homepage.deploy.nodeDeployment": { - "message": "ノードデプロイ" - }, - "homepage.deploy.clusterDeployment": { - "message": "クラスターデプロイ" - }, - "homepage.resources.title": { - "message": "その他のリソース" - }, - "homepage.resources.contactSupport": { - "message": "サポートに連絡" - }, - "homepage.resources.changelog": { - "message": "変更履歴" - }, - "homepage.resources.sampleDatasets": { - "message": "サンプルデータセット" + "message": "ClickHouse への貢献方法を学ぶ" } -} +} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/Insert_select_settings_tuning.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/Insert_select_settings_tuning.mdx index 8796bb08cf3..c22a0c05d58 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/Insert_select_settings_tuning.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/Insert_select_settings_tuning.mdx @@ -9,35 +9,34 @@ tags: ['設定', 'エラーと例外'] {/* トランケート */} - ## Question \{#question\} -`INSERT...SELECT` ステートメントを実行すると、TOO_MANY_PARTS(too many parts)エラーが発生します。 +`INSERT...SELECT` ステートメントを実行すると、TOO_MANY_PARTS(too many parts)エラーが発生します。 -この問題を解決するにはどうすればよいですか? +この問題を解決するにはどうすればよいですか? ## 回答 \{#answer\} このエラーを回避するために調整可能な設定の一部を以下に示します。これは ClickHouse のエキスパートレベルのチューニングであり、ここで示す値は、適用先となる ClickHouse Cloud サービスまたはオンプレミスクラスターの仕様を十分に理解したうえでのみ設定すべきです。そのため、これらの値を「すべてに当てはまる一律の推奨値」として受け取らないでください。 -[max_insert_block_size](https://clickhouse.com/docs/operations/settings/settings#settings-max_insert_block_size) = `100_000_000`(デフォルト `1_048_576`) +[max_insert_block_size](https://clickhouse.com/docs/operations/settings/settings#settings-max_insert_block_size) = `100_000_000`(デフォルト `1_048_576`) 約 100 万から 1 億に増やすことで、より大きなブロックを形成できるようになります。 注意: この設定は、サーバー側でブロックを形成する場合にのみ適用されます。つまり、HTTP インターフェイス経由の INSERT には適用されますが、clickhouse-client には適用されません。 -[min_insert_block_size_rows](https://clickhouse.com/docs/operations/settings/settings#min-insert-block-size-rows) = `100_000_000`(デフォルト `1_048_576`) +[min_insert_block_size_rows](https://clickhouse.com/docs/operations/settings/settings#min-insert-block-size-rows) = `100_000_000`(デフォルト `1_048_576`) 約 100 万から 1 億に増やすことで、より大きなブロックを形成できるようになります。 -[min_insert_block_size_bytes](https://clickhouse.com/docs/operations/settings/settings#min-insert-block-size-bytes) = `500_000_000`(デフォルト `268_435_456`) +[min_insert_block_size_bytes](https://clickhouse.com/docs/operations/settings/settings#min-insert-block-size-bytes) = `500_000_000`(デフォルト `268_435_456`) 268.44 MB から 500 MB に増やすことで、より大きなブロックを形成できるようになります。 -[parts_to_delay_insert](https://clickhouse.com/docs/operations/settings/merge-tree-settings#parts-to-delay-insert) = `500`(デフォルト `150`) +[parts_to_delay_insert](https://clickhouse.com/docs/operations/settings/merge-tree-settings#parts-to-delay-insert) = `500`(デフォルト `150`) 単一パーティション内のアクティブなパーツ数がしきい値に達したときに、INSERT が意図的に遅延されないよう、この値を増やします。 -[parts_to_throw_insert](https://clickhouse.com/docs/operations/settings/merge-tree-settings#parts-to-throw-insert) = `1500`(デフォルト `3000`) +[parts_to_throw_insert](https://clickhouse.com/docs/operations/settings/merge-tree-settings#parts-to-throw-insert) = `1500`(デフォルト `3000`) この値を増やすと、一般的にはテーブルに対するクエリ性能に影響しますが、データ移行の用途であれば問題ありません。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/ODBC-authentication-failed-error-using-PowerBI-CH-connector.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/ODBC-authentication-failed-error-using-PowerBI-CH-connector.mdx index d1ad911a316..9adf54592d8 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/ODBC-authentication-failed-error-using-PowerBI-CH-connector.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/ODBC-authentication-failed-error-using-PowerBI-CH-connector.mdx @@ -13,8 +13,7 @@ import powerbi_error from "@site/static/images/knowledgebase/powerbi_odbc_authen {/* 省略 */} - -## 質問 +## 質問 \{#question\} コネクタを使用して Power BI から ClickHouse に接続しようとすると、認証エラーが発生します。 @@ -32,7 +31,6 @@ ClickHouseがインストールされているサーバー上の /etc/clickhouse Power BI ODBC 認証エラーのダイアログ - ## 回答 \{#answer\} ClickHouse ODBC Driver をバージョン [1.4.1](https://github.com/ClickHouse/clickhouse-odbc/releases/tag/1.4.1.20250523) に更新してください。 @@ -41,6 +39,6 @@ ClickHouse ODBC Driver をバージョン [1.4.1](https://github.com/ClickHouse/ 接続には専用のユーザーアカウントを使用し、パスワードを手動で設定することを推奨します。ClickHouse Cloud を使用していて、`default` ユーザーと同等の管理者レベルのアクセス権が必要な場合は、新しいユーザーを作成し、そのユーザーに `default_role` を割り当ててください。 -詳細については次を参照してください: +詳細については次を参照してください:\ https://clickhouse.com/docs/operations/access-rights#user-account-management https://clickhouse.com/docs/cloud/security/cloud-access-management#database-roles \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/about-quotas-and-query-complexity.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/about-quotas-and-query-complexity.mdx index 2f71c2f2bad..eef3f884781 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/about-quotas-and-query-complexity.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/about-quotas-and-query-complexity.mdx @@ -10,7 +10,6 @@ description: 'クォータとクエリの複雑性は、ユーザーが ClickHou {/* 切り捨て */} - ## クォータとクエリの複雑度について \{#about-quotas-and-query-complexity\} [クォータ](https://clickhouse.com/docs/operations/quotas) と [クエリの複雑度](https://clickhouse.com/docs/operations/settings/query-complexity) は、ユーザーが ClickHouse 上で実行できる操作を制限するための強力な手段です。 @@ -19,7 +18,7 @@ description: 'クォータとクエリの複雑性は、ユーザーが ClickHou このナレッジベース記事では、これら 2 つの異なるアプローチをどのように適用するかの例を示します。 -## サンプルデータ +## サンプルデータ \{#the-sample-data\} 以降の例では、次のような単純なサンプルテーブルを使用します。 @@ -71,8 +70,7 @@ clickhouse-cloud :) SELECT * FROM default.test_table_00006488 LIMIT 5 -- 5 rows in set. Elapsed: 0.003 sec. ``` - -## クォータの使用 +## クォータの使用 \{#using-quotas\} この例では、ロールを作成し、そのロールに対して各 10 秒の間隔ごとに取得できる結果行を最大 10 行に制限するクォータを適用します。 @@ -138,7 +136,6 @@ clickhouse-cloud :) CREATE QUOTA quota_max_10_result_rows_per_10_seconds FOR INT 次に、ユーザー `user_with_quota` としてログインします - ```sql -- クォータがロールを通じて適用されているユーザーとしてログイン clickhouse-cloud :) SELECT user() @@ -238,8 +235,7 @@ clickhouse-cloud :) select now() ユーザーは、新しい 10 行分の結果セットを取得できる「許容量」が再び付与されるまで、さらに 5 秒待つ必要があることに注意してください。 - -## クエリ複雑度の使用 +## クエリ複雑度の使用 \{#using-query-complexity\} この例では、各クエリで返される行数を1行のみに制限するQuery Complexity `SETTING`を適用するロールを作成します。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/add-column.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/add-column.mdx index 5531a22d3ee..dbd7894bbaf 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/add-column.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/add-column.mdx @@ -10,8 +10,7 @@ keywords: ['列の追加'] {/* 省略 */} - -## テーブルにカラムを追加する +## テーブルにカラムを追加する \{#adding-a-column-to-a-table\} ここでは `clickhouse-local` を使用します。 @@ -49,8 +48,7 @@ FROM events; └────────────┴────────┘ ``` - -## 新しいカラムを追加する +## 新しいカラムを追加する \{#adding-a-new-column\} ここでは、`favoriteNumber` という名前の新しいカラムを `Float64` 型で追加するとします。 これは、[`ALTER TABLE...ADD COLUMN`](/sql-reference/statements/alter/column#add-column) 句を使って実行できます。 @@ -87,8 +85,7 @@ INSERT INTO events (name) VALUES ('Tyler'); └────────────┴────────┴────────────────┘ ``` - -## 列のデフォルト値を変更する +## 列のデフォルト値を変更する \{#modifying-a-columns-default-value\} `favoriteNumber` 列の型を [`ALTER TABLE...MODIFY COLUMN`](/sql-reference/statements/alter/column#modify-column) 句を使って別の型に変更すると、興味深い挙動が見られます。 @@ -147,8 +144,7 @@ INSERT INTO events (name) VALUES ('Tanya'); `Tanya` には新しいデフォルト値 `21` が適用されますが、`Alexey` には以前のデフォルト値 `99` がそのまま残ります。 - -## テーブル内のカラム位置の制御 +## テーブル内のカラム位置の制御 \{#controlling-column-position-in-table\} 新しいカラムを追加すると、デフォルトではテーブルの最後に追加されます。 `FIRST` 句と `AFTER` 句を使うことで、カラムを配置する位置を制御できます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/alter-user-settings-exception.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/alter-user-settings-exception.mdx index 36a439005ab..06aa205860c 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/alter-user-settings-exception.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/alter-user-settings-exception.mdx @@ -10,8 +10,7 @@ keywords: ['例外', 'ユーザー設定'] {/* トランケート */} - -## DB::Exception: Cannot update user `default` in users.xml because this storage is readonly. (ACCESS_STORAGE_READONLY) \{#dbexception-cannot-update-user-default-in-usersxml-because-this-storage-is-readonly-access_storage_readonly\} +## DB::Exception: Cannot update user `default` in users.xml because this storage is readonly. (ACCESS_STORAGE_READONLY) \{#dbexception-cannot-update-user-default-in-usersxml-because-this-storage-is-readonly-access_storage_readonly\} ユーザーの設定を変更しようとすると、上記の例外が発生することがあります。 @@ -31,5 +30,4 @@ keywords: ['例外', 'ユーザー設定'] ## SQL ベースのアクセス制御を有効化する \{#enable-sql-driven-access-control\} -default ユーザーに対して、SQL ベースのアクセス制御とアカウント管理を有効にできます。これを有効にするための手順は、この[ページ](/operations/access-rights#enabling-access-control -)で説明されています。 \ No newline at end of file +default ユーザーに対して、SQL ベースのアクセス制御とアカウント管理を有効にできます。これを有効にするための手順は、この[ページ](/operations/access-rights#enabling-access-control)で説明されています。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/are_materialized_views_inserted_asynchronously.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/are_materialized_views_inserted_asynchronously.mdx index 94e823f6765..bcbabb30241 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/are_materialized_views_inserted_asynchronously.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/are_materialized_views_inserted_asynchronously.mdx @@ -10,7 +10,6 @@ keywords: ['マテリアライズドビュー'] {/* 省略 */} - ## マテリアライズドビューへの挿入は同期的に行われますか? \{#are-materialized-views-inserted-synchronously\} **質問:** あるソーステーブルに新しい行が挿入されると、その新しい行はそのソーステーブルに紐づくすべてのマテリアライズドビューにも送られます。マテリアライズドビューへの挿入は同期的に実行されますか?つまり、サーバーからクライアントへ挿入成功が通知された時点で、すべてのマテリアライズドビューが完全に更新され、クエリに利用可能になっていることを意味しますか? diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/async_vs_optimize_read_in_order.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/async_vs_optimize_read_in_order.mdx index f548bb8a701..12437a9e058 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/async_vs_optimize_read_in_order.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/async_vs_optimize_read_in_order.mdx @@ -15,12 +15,11 @@ import optimize_read from "@site/static/images/knowledgebase/optimize_read.png"; {/* 省略 */} - ## 同期データ読み取り \{#synchronous-data-reading\} 新しい設定 `allow_asynchronous_read_from_io_pool_for_merge_tree` によって、読み取りスレッド(ストリーム)の数を、クエリ実行パイプラインの残りの処理で使用されるスレッド数よりも多く設定できます。 -通常、[max_threads](https://clickhouse.com/docs/operations/settings/settings/#settings-max_threads) 設定は、並列読み取りスレッド数と並列クエリ処理スレッド数を[制御](https://clickhouse.com/company/events/query-performance-introspection)します。 +通常、[max_threads](https://clickhouse.com/docs/operations/settings/settings/#settings-max_threads) 設定は、並列読み取りスレッド数と並列クエリ処理スレッド数を[制御](https://clickhouse.com/company/events/query-performance-introspection)します。 同期データ読み取りの図 @@ -28,26 +27,26 @@ import optimize_read from "@site/static/images/knowledgebase/optimize_read.png"; ### 非同期データ読み取り \{#asynchronous-data-reading\} -新しい設定 [allow_asynchronous_read_from_io_pool_for_merge_tree](https://github.com/ClickHouse/ClickHouse/pull/43260) により、**CPU リソースが限られた ClickHouse Cloud サービスでのコールドクエリを高速化**し、**I/O ボトルネックとなるクエリのパフォーマンスを向上**させるために、読み取りスレッド(ストリーム)の数を、クエリ実行パイプラインの残りの部分で使用されるスレッド数より多く設定できるようになりました。 -この設定が有効な場合、読み取りスレッドの数は [max_streams_for_merge_tree_reading](https://github.com/ClickHouse/ClickHouse/pull/43260) 設定によって制御されます。 +新しい設定 [allow_asynchronous_read_from_io_pool_for_merge_tree](https://github.com/ClickHouse/ClickHouse/pull/43260) により、**CPU リソースが限られた ClickHouse Cloud サービスでのコールドクエリを高速化**し、**I/O ボトルネックとなるクエリのパフォーマンスを向上**させるために、読み取りスレッド(ストリーム)の数を、クエリ実行パイプラインの残りの部分で使用されるスレッド数より多く設定できるようになりました。 +この設定が有効な場合、読み取りスレッドの数は [max_streams_for_merge_tree_reading](https://github.com/ClickHouse/ClickHouse/pull/43260) 設定によって制御されます。 非同期データ読み取りの図 データは、異なるカラムから非同期かつ並列に読み取られます。 -また、読み取りスレッド(ストリーム)の数と、クエリ実行パイプラインの残りの部分で使用されるスレッド数との比率を設定するための [max_streams_to_max_threads_ratio](https://github.com/ClickHouse/ClickHouse/pull/43260) 設定も存在します。ただし、ベンチマークでは `max_streams_for_merge_tree_reading` 設定ほどの効果はありませんでした。 +また、読み取りスレッド(ストリーム)の数と、クエリ実行パイプラインの残りの部分で使用されるスレッド数との比率を設定するための [max_streams_to_max_threads_ratio](https://github.com/ClickHouse/ClickHouse/pull/43260) 設定も存在します。ただし、ベンチマークでは `max_streams_for_merge_tree_reading` 設定ほどの効果はありませんでした。 -### optimize_read_in_order の場合はどうなりますか? \{#what-about-optimize_read_in_order\} +### optimize_read_in_order の場合はどうなりますか? \{#what-about-optimize_read_in_order\} -[optimize_read_in_order 最適化](https://clickhouse.com/docs/sql-reference/statements/select/order-by/#optimization-of-data-reading)を有効にすると、クエリのソート順がディスク上のデータの物理的な並び順を反映している場合、ClickHouse はメモリ内でのデータの再ソート処理を[省略](https://clickhouse.com/blog/clickhouse-faster-queries-with-projections-and-primary-indexes)できます。**ただし、そのためにはデータを順序どおりに読み出す必要があります(非同期読み取りとは対照的です)。** +[optimize_read_in_order 最適化](https://clickhouse.com/docs/sql-reference/statements/select/order-by/#optimization-of-data-reading)を有効にすると、クエリのソート順がディスク上のデータの物理的な並び順を反映している場合、ClickHouse はメモリ内でのデータの再ソート処理を[省略](https://clickhouse.com/blog/clickhouse-faster-queries-with-projections-and-primary-indexes)できます。**ただし、そのためにはデータを順序どおりに読み出す必要があります(非同期読み取りとは対照的です)。** 順序どおりの読み取り最適化を示す図 -### optimize_read_in_order は非同期読み取りよりも優先される \{#optimize_read_in_order-has-precedence-over-asynchronous-reading\} +### optimize_read_in_order は非同期読み取りよりも優先される \{#optimize_read_in_order-has-precedence-over-asynchronous-reading\} ClickHouse が `optimize_read_in_order optimization` を適用できると判断した場合、`allow_asynchronous_read_from_io_pool_for_merge_tree` 設定は無視されるか、無効化されます。 -### 上記の内容をすべて踏まえた例 +### 上記の内容をすべて踏まえた例 \{#example-demonstrating-all-of-the-above\} * [UK Property Price Paid テーブル](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid) を作成してデータをロードする @@ -139,7 +138,6 @@ SETTINGS * `optimize_read_in_order optimization` を適用できる場合、 非同期読み取り用に 60 個、残りのクエリ実行パイプライン用に 20 個のスレッドを使用するクエリパイプラインを確認する - ``` EXPLAIN PIPELINE SELECT * diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-clickpipes.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-clickpipes.mdx index 6e094550f38..a6e7d92dadf 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-clickpipes.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-clickpipes.mdx @@ -10,7 +10,6 @@ keywords: ['AWS PrivateLink', 'プライベート RDS', 'ClickPipes'] {/* 切り捨て */} - ## ClickPipes 用にプライベート RDS を公開するための AWS PrivateLink セットアップ \{#aws-privatelink-setup-to-expose-private-rds-for-clickpipes\} ClickPipes からプライベート RDS にアクセスできるようにするために、AWS PrivateLink 経由で公開するセットアップ手順です。クロスリージョンアクセスにも対応します。 @@ -26,46 +25,48 @@ ClickPipes からプライベート RDS にアクセスできるようにする RDS インスタンス向けの **VPC エンドポイントサービス** を作成するには、次の手順に従います。エンドポイントサービスが必要な RDS インスタンスが複数ある場合は(インスタンスごとに異なるリスナーポートを使用している場合も含めて)、これらの手順を繰り返してください。 1. VPC の特定および [NLB の作成](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-network-load-balancer.html) - - 対象の VPC を特定し、Network Load Balancer (NLB) を作成します。NLB はインターネット向け(public)ではなく、内部向け(private)である必要がある点に注意してください。 + * 対象の VPC を特定し、Network Load Balancer (NLB) を作成します。NLB はインターネット向け(public)ではなく、内部向け(private)である必要がある点に注意してください。 2. ターゲットグループの設定 - - ターゲットグループは、RDS インスタンスのエンドポイント IP とポート(通常、PostgreSQL は 5432、MySQL は 3306)を指すように設定します。 - :::note + * ターゲットグループは、RDS インスタンスのエンドポイント IP とポート(通常、PostgreSQL は 5432、MySQL は 3306)を指すように設定します。 + + :::note + + 新しい RDS エンドポイント IP でターゲットグループを更新する処理を自動化したい場合は、AWS Lambda 関数やその他の自動化ツールを使用できます。 + この目的で使用できる Terraform モジュールの 1 つとしては、[こちら](https://github.com/MaterializeInc/terraform-aws-rds-privatelink) があります。 - 新しい RDS エンドポイント IP でターゲットグループを更新する処理を自動化したい場合は、AWS Lambda 関数やその他の自動化ツールを使用できます。 - この目的で使用できる Terraform モジュールの 1 つとしては、[こちら](https://github.com/MaterializeInc/terraform-aws-rds-privatelink) があります。 + ::: - ::: - - **重要**: DB Cluster / Aurora の場合に使用する RDS インスタンスエンドポイントは、共通(クラスター)エンドポイントではなく WRITER エンドポイント **のみ** を使用してください。 - - NLB による TLS 終端を回避するため、プロトコルには TCP を使用していることを確認してください。 + * **重要**: DB Cluster / Aurora の場合に使用する RDS インスタンスエンドポイントは、共通(クラスター)エンドポイントではなく WRITER エンドポイント **のみ** を使用してください。 + * NLB による TLS 終端を回避するため、プロトコルには TCP を使用していることを確認してください。 3. リスナーポートの設定 - - ロードバランサーのリスナーポートは、ターゲットグループで使用しているポート(通常、PostgreSQL は 5432、MySQL は 3306)と一致させる必要があります。 + * ロードバランサーのリスナーポートは、ターゲットグループで使用しているポート(通常、PostgreSQL は 5432、MySQL は 3306)と一致させる必要があります。 4. VPC エンドポイントサービスの作成 - - VPC 内で、NLB を指すエンドポイントサービスを作成します。 - - 特定アカウントからの接続要求を承諾できるように設定します。 + * VPC 内で、NLB を指すエンドポイントサービスを作成します。 + * 特定アカウントからの接続要求を承諾できるように設定します。 5. ClickPipes にエンドポイントサービスの利用を許可 - - ClickPipes アカウントがこのエンドポイントサービスを要求できるように権限を付与します。 - - 許可されたプリンシパルとして、次のプリンシパル ID を追加します: - ``` - arn:aws:iam::072088201116:root - ``` + * ClickPipes アカウントがこのエンドポイントサービスを要求できるように権限を付与します。 + * 許可されたプリンシパルとして、次のプリンシパル ID を追加します: + ``` + arn:aws:iam::072088201116:root + ``` 6. NLB で「Enforce Security Group Inbound Rules on Private Link Traffic」を無効化(NLB にセキュリティグループがアタッチされている場合) - - NLB の設定画面で、セキュリティグループがアタッチされている場合は "Enforce Security Group Inbound Rules on Private Link Traffic" 設定を無効にします。 - - Terraform を使用している場合は、NLB に対して `enforce_security_group_inbound_rules_on_private_link_traffic` 属性を `off` に設定します。 - - ClickPipes の VPC から NLB へのトラフィックを許可するには、この設定が**必須**です。 + * NLB の設定画面で、セキュリティグループがアタッチされている場合は "Enforce Security Group Inbound Rules on Private Link Traffic" 設定を無効にします。 + * Terraform を使用している場合は、NLB に対して `enforce_security_group_inbound_rules_on_private_link_traffic` 属性を `off` に設定します。 + * ClickPipes の VPC から NLB へのトラフィックを許可するには、この設定が**必須**です。 7. **任意**: データベースのリージョンが ClickPipes のリージョンと異なる場合のクロスリージョンサポートの有効化 - - VPC コンソールの `Endpoint Services` で、対象のエンドポイントサービスを選択します。 - - `Actions` ボタンをクリックし、`Modify supported Regions` を選択します。 - - エンドポイントサービスへのアクセスを許可したいリージョンを追加します。ClickPipes のリージョン一覧については、[こちら](/integrations/clickpipes#list-of-static-ips) を参照してください。 + * VPC コンソールの `Endpoint Services` で、対象のエンドポイントサービスを選択します。 + * `Actions` ボタンをクリックし、`Modify supported Regions` を選択します。 + * エンドポイントサービスへのアクセスを許可したいリージョンを追加します。ClickPipes のリージョン一覧については、[こちら](/integrations/clickpipes#list-of-static-ips) を参照してください。 8. **MySQL を使用する場合の推奨設定** - - MySQL を使用している場合は、ソースデータベースのパラメータとして `skip_name_resolve=1` を設定することを推奨します。 + * MySQL を使用している場合は、ソースデータベースのパラメータとして `skip_name_resolve=1` を設定することを推奨します。 NLB からのヘルスチェックが繰り返し行われるため、MySQL が NLB ホストをブロックしてしまう場合があります。このパラメータを設定すると DNS ホスト名ルックアップを完全に回避できるため、NLB ホストがブロックされることはありません。RDS を使用している場合、この変更にはインスタンスの再起動が必要です。 ## 接続の開始 \{#initiating-connection\} diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-msk-clickpipes.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-msk-clickpipes.mdx index fe981b48c62..56a15c61b56 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-msk-clickpipes.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/aws-privatelink-setup-for-msk-clickpipes.mdx @@ -10,7 +10,6 @@ keywords: ['AWS PrivateLink', 'MSK', 'ClickPipes'] {/* 省略 */} - ## 概要 \{#overview\} このガイドでは、[ClickPipes reverse private endpoint](/integrations/clickpipes/aws-privatelink#msk-multi-vpc) と併せて使用する **MSK マルチ VPC 環境** のセットアップ方法を説明します。 @@ -22,37 +21,37 @@ MSK クラスターの VPC は、ClickPipes が提供されているリージョ ## マルチ VPC 接続を有効にする \{#enabling-multi-vpc-connectivity\} 1. MSK クラスターに移動します。 - - Amazon MSK コンソールの左側ナビゲーションペインから「Clusters」を選択します。 - - マルチ VPC 接続を設定したい対象の MSK クラスターを選択します。 + * Amazon MSK コンソールの左側ナビゲーションペインから「Clusters」を選択します。 + * マルチ VPC 接続を設定したい対象の MSK クラスターを選択します。 2. MSK マルチ VPC 接続を有効にします - - **Connectivity** タブで **Multi-VPC connectivity** セクションを探します。 - - **Edit** をクリックします。 - - **Turn-on MSK multi-VPC connectivity** オプションを有効にします。 - - 画面の指示に従って設定を完了します。 + * **Connectivity** タブで **Multi-VPC connectivity** セクションを探します。 + * **Edit** をクリックします。 + * **Turn-on MSK multi-VPC connectivity** オプションを有効にします。 + * 画面の指示に従って設定を完了します。 3. ClickPipes アカウントプリンシパルをクラスターのポリシーに追加します - - **Configuration** タブに移動します。 - - **Cluster policy** セクションで **Edit** をクリックします。 - - **IAM policy** に `arn:aws:iam::072088201116:root` を含めます。例: - ```json - { - "Version": "2012-10-17", - "Statement": [ - { - "Effect": "Allow", - "Principal": { - "AWS": [ - "arn:aws:iam::072088201116:root" - ] - }, - "Action": [ - "kafka-cluster:Connect", - "kafka-cluster:DescribeCluster", - "kafka-cluster:ListClusters" - ] - } - ] - } - ``` + * **Configuration** タブに移動します。 + * **Cluster policy** セクションで **Edit** をクリックします。 + * **IAM policy** に `arn:aws:iam::072088201116:root` を含めます。例: + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": [ + "arn:aws:iam::072088201116:root" + ] + }, + "Action": [ + "kafka-cluster:Connect", + "kafka-cluster:DescribeCluster", + "kafka-cluster:ListClusters" + ] + } + ] + } + ``` ## リバースプライベートエンドポイントの作成 \{#creating-reverse-private-endpoint\} diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/backing_up_a_specific_partition.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/backing_up_a_specific_partition.mdx index da0fd3d5804..edd2acb5ef2 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/backing_up_a_specific_partition.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/backing_up_a_specific_partition.mdx @@ -10,12 +10,11 @@ keywords: ['バックアップ', 'パーティション'] {/* 切り捨て */} - ## 質問 \{#question\} ClickHouseで特定のパーティションだけをバックアップするにはどうすればよいですか? -## 回答 +## 回答 \{#answer\} 以下の例では、[docker compose examples](https://github.com/ClickHouse/examples/blob/main/docker-compose-recipes/README.md) ページに掲載されている S3(Minio)ディスクの[設定](https://github.com/ClickHouse/examples/blob/main/docker-compose-recipes/recipes/ch-and-minio-S3/README.md)を使用しています。 @@ -138,7 +137,6 @@ Ok. バックアップから、ID が `1` のパーティションだけを復元します: - ``` ch_minio_s3 :) RESTORE TABLE my_table PARTITION 1 FROM Disk('s3','backups/'); diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/calculate-pi-using-sql.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/calculate-pi-using-sql.mdx index 54d6912d00f..2cea4aa9887 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/calculate-pi-using-sql.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/calculate-pi-using-sql.mdx @@ -10,8 +10,7 @@ keywords: ['円周率の計算', 'SQL'] {/* 切り捨て */} - -## 円周率の日です!SQLを使用して円周率を計算する +## 円周率の日です!SQLを使用して円周率を計算する \{#its-pi-day-lets-calculate-pi-using-sql\} 円周率の日おめでとうございます!ClickHouseのSQLクエリを使って円周率を計算してみるのは面白いのではないかと考えました。これまでに考案した内容は以下の通りです... diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/calculate_ratio_of_zero_sparse_serialization.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/calculate_ratio_of_zero_sparse_serialization.mdx index f154b22936f..e9404ddbbbf 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/calculate_ratio_of_zero_sparse_serialization.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/calculate_ratio_of_zero_sparse_serialization.mdx @@ -10,8 +10,7 @@ keywords: ['空/ゼロ値比率', '計算'] {/* 途中省略 */} - -## テーブル内の各カラムにおける空値/ゼロ値の比率を計算する方法 +## テーブル内の各カラムにおける空値/ゼロ値の比率を計算する方法 \{#how-to-calculate-the-ratio-of-emptyzero-values-in-every-column-in-a-table\} カラムがスパース(空である、または主にゼロを含む)な場合、ClickHouse はそのカラムをスパース形式でエンコードし、自動的に計算を最適化できます。クエリ処理時にデータを完全に伸長する必要がなくなります。実際、カラムがどの程度スパースかが分かっていれば、[`ratio_of_defaults_for_sparse_serialization` 設定](https://clickhouse.com/docs/operations/settings/merge-tree-settings#ratio_of_defaults_for_sparse_serialization)を使ってその比率を指定し、シリアライゼーションを最適化できます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/cannot-append-data-to-parquet-format.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/cannot-append-data-to-parquet-format.mdx index 91733ef6538..b1443adada1 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/cannot-append-data-to-parquet-format.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/cannot-append-data-to-parquet-format.mdx @@ -10,8 +10,7 @@ keywords: ['Parquet', 'Cannot Append Data'] {/* 切り捨て */} - -## ClickHouse での「Cannot Append Data in Parquet Format」エラーの解消 +## ClickHouse での「Cannot Append Data in Parquet Format」エラーの解消 \{#resolving-cannot-append-data-in-parquet-format-error-in-clickhouse\} ClickHouse で「Cannot append data in format Parquet to file」というエラーが発生していますか? diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/certificate_verify_failed_error.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/certificate_verify_failed_error.mdx index cb83d047e59..82a14bc7db5 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/certificate_verify_failed_error.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/certificate_verify_failed_error.mdx @@ -10,7 +10,6 @@ keywords: ['エラー', 'SSL 証明書', '210'] {/* 途中省略 */} - ## ClickHouse におけるコード 210 の SSL 証明書検証エラーの解決 \{#resolving-ssl-certificate-verify-error-in-clickhouse\} このエラーは通常、次のように報告されます。 @@ -21,10 +20,10 @@ keywords: ['エラー', 'SSL 証明書', '210'] このエラーは、`clickhouse-client` を使用して ClickHouse サーバーへの接続を試みている際に発生します。エラーの原因は次のいずれかです。 -- クライアント設定ファイル `config.xml` に、マシンの CA デフォルトストアにあるルート証明書が含まれていない -- 自己署名証明書または内部 CA 証明書が適切に設定されていない +* クライアント設定ファイル `config.xml` に、マシンの CA デフォルトストアにあるルート証明書が含まれていない +* 自己署名証明書または内部 CA 証明書が適切に設定されていない -## 解決策 +## 解決策 \{#solution\} 内部 CA または自己署名 CA を使用する場合は、クライアントディレクトリ(例: `/etc/clickhouse-client`)内の `config.xml` で CA ルート証明書を設定し、デフォルトの場所から読み込まれるデフォルトのルート CA 証明書を無効にします。 @@ -45,7 +44,6 @@ keywords: ['エラー', 'SSL 証明書', '210'] ``` - ## 追加リソース \{#additional-resources\} -[https://clickhouse.com/docs/interfaces/cli/#configuration_files](https://clickhouse.com/docs/interfaces/cli/#configuration_files) を参照してください \ No newline at end of file +[https://clickhouse.com/docs/interfaces/cli/#configuration_files](https://clickhouse.com/docs/interfaces/cli/#configuration_files) を参照してください \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/change-billing-email.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/change-billing-email.mdx index 0be3e96164e..4d8fe866dc5 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/change-billing-email.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/change-billing-email.mdx @@ -10,7 +10,6 @@ keywords: ['請求担当者の変更', 'ClickHouse Cloud'] {/* 以降を省略 */} - ## 質問 \{#question\} ClickHouse Cloud の Billing Contact(請求担当者)を変更するにはどうすればよいですか? @@ -20,5 +19,5 @@ ClickHouse Cloud の Billing Contact(請求担当者)を変更するには 管理者として請求連絡先を変更するには、以下の手順に従ってください。 1. 新しいユーザーを管理者として Cloud Organization に招待します。 -2. 招待が承諾されたら、ClickHouse Cloud Console の請求ページ(Admin > Billing)に移動し、「Billing contacts」セクションを探します。 +2. 招待が承諾されたら、ClickHouse Cloud Console の請求ページ(Admin > Billing)に移動し、「Billing contacts」セクションを探します。 3. 「Edit」ボタンをクリックし、新しい管理者ユーザーを請求連絡先として設定します。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/change-the-prompt-in-clickhouse-client.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/change-the-prompt-in-clickhouse-client.mdx index 4ec126ad57c..b321715c2d5 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/change-the-prompt-in-clickhouse-client.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/change-the-prompt-in-clickhouse-client.mdx @@ -32,8 +32,7 @@ import prompt_8 from "@site/static/images/knowledgebase/change-the-prompt-in-cli プロンプトを更新する方法はいくつかあり、それぞれについて順に説明します。 - -## --prompt フラグ +## --prompt フラグ \{#--prompt-flag\} フラグを変更する 1 つ目の方法は、`--prompt` を使用することです。 @@ -45,8 +44,7 @@ clickhouse --prompt 👉 指さしの絵文字が追加された ClickHouse クライアントプロンプト - -## 設定ファイル - トップレベル +## 設定ファイル - トップレベル \{#config-file---top-level\} 別の方法として、`config.xml` ファイルでプロンプトのプレフィックスを指定することもできます。 @@ -89,8 +87,7 @@ clickhouse -C christmas.yaml 黄色の丸い絵文字が表示された ClickHouse クライアントのプロンプト - -## 設定ファイル - connections_credentials +## 設定ファイル - connections_credentials \{#config-file---connections_credentials\} 別の方法として、接続ごとの認証情報に対して個別にプロンプトを指定することもできます。 これは、clickhouse-client を使用している場合にのみ有効です。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/check-query-cache-in-use.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/check-query-cache-in-use.mdx index 27d7cd58057..4f46cecc136 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/check-query-cache-in-use.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/check-query-cache-in-use.mdx @@ -10,14 +10,13 @@ keywords: ['クエリキャッシュ'] {/* 切り捨て */} - ## クエリでクエリキャッシュが使用されているかを確認するにはどうすればよいですか? \{#how-can-i-check-that-query-cache-is-being-used-in-my-query\} [clickhouse client](https://clickhouse.com/docs/interfaces/cli) と ClickHouse Cloud サービスを使用した次の例を参照します。 `query_cache_test` テーブルを作成します -## clickhouse client の使い方 +## clickhouse client の使い方 \{#using-clickhouse-client\} ```sql clickhouse-cloud :) CREATE TABLE query_cache_test (name String, age UInt8) ENGINE =MergeTree ORDER BY name @@ -69,7 +68,6 @@ Ok. クエリキャッシュの利用を要求するクエリを実行します(クエリの末尾に `SETTINGS use_query_cache=true` を追加します): - ```sql clickhouse-cloud :) SELECT name FROM query_cache_test WHERE age > 1000 FORMAT Null SETTINGS use_query_cache=true; @@ -153,8 +151,7 @@ Ok. 2回目の実行では、既に保存されていたエントリが見つかったため(`Entry found for query SELECT...`)、クエリはクエリキャッシュを利用しました。 - -## SQL だけを使用する +## SQL だけを使用する \{#using-just-sql\} `clickhouse client` のトレースログを確認せず、SQL コマンドを発行するだけでも、 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/check_query_processing_time_only.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/check_query_processing_time_only.mdx index 577748635e6..88f88451dd1 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/check_query_processing_time_only.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/check_query_processing_time_only.mdx @@ -10,12 +10,11 @@ keywords: ['クエリ処理時間'] {/* トランケート */} - ## 質問 \{#question\} クエリが多数の行を返しますが、知りたいのは処理時間だけです。クエリの出力を表示せずに、処理時間だけを確認するにはどうすればよいですか? -## 回答 +## 回答 \{#answer\} クエリに `FORMAT Null` を追加して、出力形式を `Null` に設定します。これにより、データがクライアントへ送信されるのを防げます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/check_users_roles.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/check_users_roles.mdx index 0cb8dcaab01..7e2cc5b56ec 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/check_users_roles.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/check_users_roles.mdx @@ -10,7 +10,6 @@ keywords: ['ユーザーロール'] {/* 切り捨て */} - ## Question \{#question\} ロールに割り当てられているユーザーおよびユーザーに割り当てられているロールを確認するにはどうすればよいですか? diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/clickhouse_cloud_api_usage.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/clickhouse_cloud_api_usage.mdx index 863a0074736..1af20f726ff 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/clickhouse_cloud_api_usage.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/clickhouse_cloud_api_usage.mdx @@ -10,14 +10,13 @@ keywords: ['cURL', 'クラウド管理', 'ClickHouse API'] {/* 切り捨て */} - ## ClickHouse API と cURL を使用して Cloud サービスを開始・停止・再開する方法 \{#how-to-start-stop-and-resume-a-cloud-service-using-the-clickhouse-api-and-curl\} ## 質問 \{#question\} API エンドポイント経由で ClickHouse Cloud サービスを起動・停止・再開するにはどうすればよいですか? -## 回答 +## 回答 \{#answer\} 1. アイドル状態の Cloud サービスを起動/再開するには、インスタンスに対して ping を送信します。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/columnar-database.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/columnar-database.mdx index 38149d55081..97ee0309949 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/columnar-database.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/columnar-database.mdx @@ -14,7 +14,6 @@ import ColumnOriented from '@site/static/images/column-oriented.gif'; {/* 省略 */} - ## はじめに \{#introduction\} カラム型データベースでは、各カラムのデータが互いに独立して保存されます。これにより、任意のクエリで使用されるカラムのデータだけをディスクから読み取ることができます。その代償として、行全体に影響する処理のコストは相対的に高くなります。カラム型データベースは、カラム指向データベース管理システムと同義です。ClickHouse は、その代表的な例です。 @@ -23,9 +22,9 @@ import ColumnOriented from '@site/static/images/column-oriented.gif'; カラムナ型データベースの主な利点は次のとおりです。 -- 多数のカラムのうち、少数のカラムを使用するクエリ。 -- 大量のデータに対する集約クエリ。 -- カラム単位でのデータ圧縮。 +* 多数のカラムのうち、少数のカラムを使用するクエリ。 +* 大量のデータに対する集約クエリ。 +* カラム単位でのデータ圧縮。 ## 行指向データベース vs 列指向データベース \{#row-oriented-vs-column-oriented-databases\} diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/common-rbac-queries.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/common-rbac-queries.mdx index 7147506b923..dca3a9672a5 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/common-rbac-queries.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/common-rbac-queries.mdx @@ -10,19 +10,17 @@ keywords: ['RBAC', 'クエリ'] {/* 切り捨て */} - ## はじめに \{#introduction\} 以下は、ユーザーに特定の権限を付与するためによく使用されるクエリです。 -## 現在のユーザーと同じ権限を別のユーザーに付与するには? +## 現在のユーザーと同じ権限を別のユーザーに付与するには? \{#how-do-i-grant-the-same-permissions-as-the-current-user-to-another-user\} ```sql GRANT CURRENT GRANTS ON *.* TO another_user; ``` - -## 現在のユーザーに付与されている権限に基づいて、別のユーザーに特定の権限を付与するにはどうすればよいですか? +## 現在のユーザーに付与されている権限に基づいて、別のユーザーに特定の権限を付与するにはどうすればよいですか? \{#how-do-i-grant-a-specific-permission-to-a-user-based-on-the-grants-of-the-current-user\} 次の例では、`another_user` は現在のユーザーがアクセス可能なすべてのデータベースおよびテーブルに対して `SELECT` クエリを実行できるようになります。 @@ -30,8 +28,7 @@ GRANT CURRENT GRANTS ON *.* TO another_user; GRANT CURRENT GRANTS(SELECT ON *.*) TO another_user; ``` - -## 現在のユーザーに付与されている権限に基づいて、特定のデータベースに対する特定の権限をユーザーに付与するにはどうすればよいですか? +## 現在のユーザーに付与されている権限に基づいて、特定のデータベースに対する特定の権限をユーザーに付与するにはどうすればよいですか? \{#how-do-i-grant-a-specific-permission-to-a-user-for-a-specific-database-based-on-the-grants-of-the-current-user\} 次の例では、`another_user` は `my_database` 内のすべてのテーブルに対して `INSERT` 文を実行できるようになります。 @@ -39,8 +36,7 @@ GRANT CURRENT GRANTS(SELECT ON *.*) TO another_user; GRANT INSERT ON my_database.* TO another_user; ``` - -## デフォルトユーザーと同じ権限を特定ユーザーにすべて付与するにはどうすればよいですか? +## デフォルトユーザーと同じ権限を特定ユーザーにすべて付与するにはどうすればよいですか? \{#how-do-i-give-access-to-all-grants-for-a-specific-user-based-on-the-default-user\} ```sql GRANT default_role TO another_user; diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/compare_resultsets.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/compare_resultsets.mdx index cbd02075a83..417b615faef 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/compare_resultsets.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/compare_resultsets.mdx @@ -10,12 +10,11 @@ keywords: ['検証', '結果セット', 'ハッシュ関数'] {/* 省略 */} - ## 質問 \{#question\} 2 つのクエリが同じ結果セットを返すことを検証するにはどうすればよいですか? -## 回答 +## 回答 \{#answer\} 以下の方法を利用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/comparing-metrics-between-queries.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/comparing-metrics-between-queries.mdx index 1dee630ec0e..14110af4b47 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/comparing-metrics-between-queries.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/comparing-metrics-between-queries.mdx @@ -10,8 +10,7 @@ keywords: [クエリの比較, メトリクスの比較, クエリ パフォー {/* 切り捨て */} - -## 2 つのクエリ間でメトリクスを比較する +## 2 つのクエリ間でメトリクスを比較する \{#compare-metrics-between-two-queries\} 2 つのクエリ間でメトリクスを比較するには、まず両方のクエリの `query_id` を取得する必要があります。 @@ -42,7 +41,6 @@ FORMAT PrettyCompactMonoBlock 2つのクエリを比較したメトリクスのテーブルが表示されます。 - ```sql ┌─metric──────────────────────────────────────────────┬───────v1─┬───────v2─┬──────────────────dB─┬───perc─┬─bar───────────────────────────────┐ │ OSReadBytes │ 0 │ 528384 │ inf │ 100 │ █████████████████████████████████ │ diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/configure-a-user-setting.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/configure-a-user-setting.mdx index d490f4c9303..b06efd5828e 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/configure-a-user-setting.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/configure-a-user-setting.mdx @@ -10,12 +10,11 @@ keywords: ['SET', 'ALTER', 'ユーザー', 'クエリ', 'セッション'] {/* トランケート */} - ## はじめに \{#introduction\} ClickHouse でユーザーの設定を定義する方法はいくつかあります。どの方法を取るかは、ユースケースやその設定をどのくらいの期間有効にしておきたいかによって異なります。いくつかのシナリオを見ていきましょう… -## 単一のクエリに対して設定を行う +## 単一のクエリに対して設定を行う \{#configure-a-setting-for-a-single-query\} `SELECT` クエリには `SETTINGS` 句を含めることができ、そこに任意の数の設定を定義できます。これらの設定はそのクエリに対してのみ適用されます。例えば: @@ -27,8 +26,7 @@ SETTINGS max_threads = 8; このクエリでは、使用されるスレッド数の最大値は 8 です。 - -## セッションに対する設定を構成する +## セッションに対する設定を構成する \{#configure-a-setting-for-a-session\} `SET` 句を使用して、クライアントセッションの存続期間に有効な設定を定義できます。これはアドホックなテストや、いくつかのクエリを実行している間だけ有効な設定にしたい場合に便利ですが、それ以上の期間は有効にしたくない場合に適しています。 @@ -39,8 +37,7 @@ SELECT * FROM my_table; ``` - -## 特定のユーザー向けの設定を構成する +## 特定のユーザー向けの設定を構成する \{#configure-a-setting-for-a-particular-user\} `ALTER USER` を使用して、特定のユーザーだけに適用される設定を定義します。例えば: diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/configure_cap_ipc_lock_and_cap_sys_nice_in_docker.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/configure_cap_ipc_lock_and_cap_sys_nice_in_docker.mdx index d1de3ed7b38..3109b05085a 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/configure_cap_ipc_lock_and_cap_sys_nice_in_docker.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/configure_cap_ipc_lock_and_cap_sys_nice_in_docker.mdx @@ -10,8 +10,7 @@ keywords: ['Docker', 'CAP_IPC_LOCK', 'CAP_SYS_NICE'] {/* 切り捨て */} - -## 質問 +## 質問 \{#question\} Docker で ClickHouse を実行していると、システムに `CAP_IPC_LOCK` と `CAP_SYS_NICE` のケーパビリティがないと Docker から警告されます。どのように対処すればよいですか? @@ -31,8 +30,7 @@ docker run -d --name clickhouse-server \ 2023.04.19 08:04:10.065860 [ 1 ] {} Application: プロセスにCAP_SYS_NICE機能がないため、'os_thread_priority'設定は効果がありません。ClickHouseパッケージが正しくインストールされていないことが原因の可能性があります。'sudo setcap cap_sys_nice=+ep /usr/bin/clickhouse'を手動で実行することで問題を解決できます。なお、'nosuid'でマウントされたファイルシステムでは動作しません。 ``` - -## 回答 +## 回答 \{#answer\} 1. コンテナに `IPC_LOCK` および `SYS_NICE` のケーパビリティを付与するために、2つの `--cap-add` 引数を追加します。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/connection_timeout_remote_remoteSecure.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/connection_timeout_remote_remoteSecure.mdx index 6e70bec2ce6..0f94e7aebe9 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/connection_timeout_remote_remoteSecure.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/connection_timeout_remote_remoteSecure.mdx @@ -10,8 +10,7 @@ keywords: ['接続試行が失敗しました'] {/* 省略 */} - -## コード: 279. DB::NetException: All connection tries failed. +## コード: 279. DB::NetException: All connection tries failed. \{#code-279-dbnetexception-all-connection-tries-failed\} **問題** [`remote()` または `remoteSecure()`](https://clickhouse.com/docs/sql-reference/table-functions/remote/) テーブル関数を使用すると、別の ClickHouse ノード上のリモートテーブルにアクセスできます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/custom-dns-alias-for-instance.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/custom-dns-alias-for-instance.mdx index 76d0ae5b984..6efbeab50a5 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/custom-dns-alias-for-instance.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/custom-dns-alias-for-instance.mdx @@ -15,12 +15,11 @@ hide_title: true
- # リバースプロキシを設定してカスタム DNS エイリアスを設定する \{#custom-dns-alias\} > このナレッジベース記事では、Nginx などのリバースプロキシを使用して、ClickHouse ネイティブクライアント向けに ClickHouse Cloud インスタンスのカスタム DNS エイリアスを設定する方法を順を追って説明します。 -## 自己署名証明書を作成する +## 自己署名証明書を作成する \{#create-certificate\} :::note 署名付き証明書を使用している場合、この手順は不要です。 @@ -51,8 +50,7 @@ hide_title: true ``` - -## Nginx 設定を更新する +## Nginx 設定を更新する \{#update-nginx-config\} `nginx.conf` ファイルに以下の内容を追加します: @@ -93,8 +91,7 @@ stream { ここで言う `isrgrootx1.pem` は ClickHouse Cloud のルート証明書で、 [こちら](https://letsencrypt.org/certs/isrgrootx1.pem) からダウンロードできます。 - -## hosts ファイルを更新する +## hosts ファイルを更新する \{#update-hosts-file\} :::note 独自のドメインコントローラーを使用している場合、次の手順は不要です @@ -110,8 +107,7 @@ Nginx サーバー上の `/etc/hosts` ファイルに以下を追加します: ここでの `10.X.Y.Z` は、あなたの環境における特定の Nginx サーバーの IP アドレスを指します。 - -## エイリアスを使用して Cloud に接続する +## エイリアスを使用して Cloud に接続する \{#connect-to-cloud-using-alias\} これで、カスタムエイリアスを使って接続する準備が整いました。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/dbms-naming.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/dbms-naming.mdx index 739b0281a02..64ac74dbbee 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/dbms-naming.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/dbms-naming.mdx @@ -10,15 +10,14 @@ keywords: ['ClickHouse の意味'] {/* 省略 */} - ## ClickHouse という名前の意味 \{#clickhouse-meaning-explained\} これは「**Click**stream(クリックストリーム)」と「Data ware**House**(データウェアハウス)」を組み合わせたものです。Yandex.Metrica における元々のユースケースに由来しており、ClickHouse はインターネット全体から人々のあらゆるクリックを記録することになっていて、現在もその役割を果たしています。このユースケースの詳細は [ClickHouse history](https://clickhouse.com/docs/about-us/history) ページで確認できます。 この 2 つの意味には、次の 2 点の含意があります。 -- `Click**H**ouse` の正しい表記は、必ず H を大文字にすることです。 -- 省略する必要がある場合は **CH** を使用してください。歴史的な理由により、中国では CK と省略する形も人気があります。これは、最初期の中国語での ClickHouse に関する講演の 1 つでこの表記が使われたことが主な理由です。 +* `Click**H**ouse` の正しい表記は、必ず H を大文字にすることです。 +* 省略する必要がある場合は **CH** を使用してください。歴史的な理由により、中国では CK と省略する形も人気があります。これは、最初期の中国語での ClickHouse に関する講演の 1 つでこの表記が使われたことが主な理由です。 :::info ClickHouse に名前が付けられてから何年も後になって、このように、それ自体が意味を持つ 2 つの単語を組み合わせるアプローチが、データベースに名前を付ける最良の方法として取り上げられました。これは、Carnegie Mellon University のデータベース学准教授である Andy Pavlo による [研究](https://www.cs.cmu.edu/~pavlo/blog/2020/03/on-naming-a-database-management-system.html) で示されています。ClickHouse は、彼の「史上最高のデータベース名」賞を Postgres と分け合いました。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/delete-old-data.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/delete-old-data.mdx index de70b4a3166..622c8e7544b 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/delete-old-data.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/delete-old-data.mdx @@ -10,7 +10,6 @@ keywords: ['古いレコードの削除'] {/* 省略 */} - ## 結論は「はい」です \{#the-short-answer-is-yes\} 結論は「はい」です。ClickHouse には、古いデータを削除してディスク使用量を削減できる複数の仕組みがあります。それぞれ異なるシナリオを想定しています。 @@ -27,7 +26,7 @@ TTL はデータを [/dev/null](https://en.wikipedia.org/wiki/Null_device) に [TTL の設定](https://clickhouse.com/docs/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl)についての詳細は、こちらを参照してください。 -## DELETE FROM +## DELETE FROM \{#delete-from\} [DELETE FROM](https://clickhouse.com/docs/sql-reference/statements/delete) を使用すると、標準的な DELETE クエリを ClickHouse で実行できます。フィルター句で対象となった行には削除フラグが付き、以降の結果セットからは除外されます。削除済み行のクリーンアップ(物理削除)は非同期に行われます。 @@ -40,7 +39,6 @@ SET allow_experimental_lightweight_delete = true; ::: - ## ALTER DELETE \{#alter-delete\} `ALTER DELETE` は、非同期バッチ処理を用いて行を削除します。`DELETE FROM` と異なり、`ALTER DELETE` の実行後からバッチ処理が完了するまでの間に実行されたクエリの結果には、削除対象となっている行も含まれます。詳細については [ALTER DELETE](https://clickhouse.com/docs/sql-reference/statements/alter/delete) ドキュメントを参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/dictionaries-consistent-state.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/dictionaries-consistent-state.mdx index 9e35c7de71e..5ab572a8ee7 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/dictionaries-consistent-state.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/dictionaries-consistent-state.mdx @@ -10,8 +10,7 @@ keywords: ['辞書'] {/* トランケート */} - -## ClickHouse におけるディクショナリ +## ClickHouse におけるディクショナリ \{#dictionaries-in-clickhouse\} ClickHouse Cloud で作成されたディクショナリは、作成直後の段階では一時的に不整合な状態になる場合があります。つまり、作成直後にはディクショナリ内にデータがまったく表示されないことがあります。ただし、数回リトライすると、作成クエリが別のレプリカで実行され、データが表示されるようになります。 @@ -44,7 +43,6 @@ SOURCE(CLICKHOUSE(QUERY SETTINGS select_sequential_consistency=1' USER default PASSWORD '')) ``` - ## なぜこの問題が発生するのですか? \{#why-does-this-issue-occur\} データを挿入してから辞書を作成または再読み込みする際、DDL がデータ(または新しいデータ)よりも先にレプリカへ到達する場合があります。これにより、レプリカ間で辞書の内容に不整合が生じます。その結果、どのレプリカがクエリを受信するかによって、得られる結果が異なる可能性があります。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/dictionary_using_strings.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/dictionary_using_strings.mdx index 698414ccdb4..f3182015160 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/dictionary_using_strings.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/dictionary_using_strings.mdx @@ -10,12 +10,11 @@ keywords: ['Dictionary', '文字列キー', '文字列値'] {/* truncate */} - ## 質問 \{#question\} MergeTree テーブルをソースとし、文字列キーおよび文字列値を使用する ClickHouse 辞書の作成方法 -## 回答 +## 回答 \{#answer\} * 辞書の元となるテーブルを作成します diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/difference_between_official_builds_and_3rd_party.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/difference_between_official_builds_and_3rd_party.mdx index b4766fa0fb8..02894a71d24 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/difference_between_official_builds_and_3rd_party.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/difference_between_official_builds_and_3rd_party.mdx @@ -10,7 +10,6 @@ keywords: ['公式ビルド', 'サードパーティ製ビルド'] {/* 切り捨て */} - ## 質問 \{#question\} 他のベンダーが独自ビルドの ClickHouse を提供しているのを見かけます。公式の ClickHouse ビルドとこれらのサードパーティ製ビルドにはどのような違いがありますか? @@ -19,17 +18,17 @@ keywords: ['公式ビルド', 'サードパーティ製ビルド'] 他のビルドで確認された違いには、次のようなものがあります。 -- **"official"** という文言がベンダー名に置き換えられている -- 数か月遅れて公開され、***最近のバグ修正が含まれていない*** ため、公式バージョンではすでに修正済みの脆弱性を含んでいる可能性がある -- ビルドがビットレベルで同一ではなく、コード中のアドレスも異なる。その結果として、これらのビルドから得られたスタックトレースを解析できず、ClickHouse チームはこれらのビルドに関する質問に回答できない -- ビルドは監査不可能かつ再現不可能であり、同じビルドログを持つ公開 CI システムが存在しない -- これらのビルドでは ClickHouse のテストスイートが実行されていないため、テストスイートによる動作検証が行われていない -- すべてのアーキテクチャ(ARM など)向けに提供されていない場合がある -- 特定の顧客向けのパッチが含まれていることがあり、それによって互換性が損なわれたり、追加のリスクが生じたりする可能性がある +* **"official"** という文言がベンダー名に置き換えられている +* 数か月遅れて公開され、***最近のバグ修正が含まれていない*** ため、公式バージョンではすでに修正済みの脆弱性を含んでいる可能性がある +* ビルドがビットレベルで同一ではなく、コード中のアドレスも異なる。その結果として、これらのビルドから得られたスタックトレースを解析できず、ClickHouse チームはこれらのビルドに関する質問に回答できない +* ビルドは監査不可能かつ再現不可能であり、同じビルドログを持つ公開 CI システムが存在しない +* これらのビルドでは ClickHouse のテストスイートが実行されていないため、テストスイートによる動作検証が行われていない +* すべてのアーキテクチャ(ARM など)向けに提供されていない場合がある +* 特定の顧客向けのパッチが含まれていることがあり、それによって互換性が損なわれたり、追加のリスクが生じたりする可能性がある ドキュメントの [インストール手順](https://clickhouse.com/docs/install) に従い、公式ビルドを用いて ClickHouse の最新版を実行することを推奨します。 -- 毎月 **stable バージョン** を 1 つリリースしており、直近 3 つの stable リリースについては、診断およびバグ修正のバックポートという観点でサポートしています。 -- また年に 2 回 **long-term support (LTS) バージョン** をリリースしており、初回リリースから 1 年間サポートされます。これは、頻繁なアップグレードや非 LTS ソフトウェアの利用が許可されていない企業を主な対象としています。(私たちは毎月の stable ビルドを強く推奨しています。) +* 毎月 **stable バージョン** を 1 つリリースしており、直近 3 つの stable リリースについては、診断およびバグ修正のバックポートという観点でサポートしています。 +* また年に 2 回 **long-term support (LTS) バージョン** をリリースしており、初回リリースから 1 年間サポートされます。これは、頻繁なアップグレードや非 LTS ソフトウェアの利用が許可されていない企業を主な対象としています。(私たちは毎月の stable ビルドを強く推奨しています。) ドキュメントには、[stable と LTS リリースの違いに関する詳細](https://clickhouse.com/docs/faq/operations/production#how-to-choose-between-clickhouse-releases) も記載しています。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/enabling-ssl-with-lets-encrypt.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/enabling-ssl-with-lets-encrypt.mdx index 9027b4792d1..4d662279b21 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/enabling-ssl-with-lets-encrypt.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/enabling-ssl-with-lets-encrypt.mdx @@ -13,8 +13,7 @@ import security_group from "@site/static/images/knowledgebase/lets-encrypt-ssl/p {/* truncate */} - -## 手順 +## 手順 \{#steps-to-follow\} 以下の手順では、単一の ClickHouse Server に対して [Let's Encrypt](https://letsencrypt.org/) を使用して SSL を有効にする方法を説明します。Let's Encrypt は無料かつ自動化されたオープンな認証局 (Certificate Authority, CA) であり、誰もが HTTPS を用いて自分の Web サイトを容易に保護できるように設計されています。証明書の発行および更新プロセスを自動化することで、Let's Encrypt は手動による操作を必要とせずに Web サイトの安全性を維持します。 @@ -75,7 +74,6 @@ sudo apt install certbot
- ```bash sudo certbot certonly デバッグログを /var/log/letsencrypt/letsencrypt.log に保存しています @@ -164,7 +162,6 @@ echo "127.0.0.1 product-test-server.clickhouse-dev.com" | sudo tee -a /etc/hosts :::warning ワイルドカードアドレスからの接続を許可する場合は、以下のいずれかの対策を必ず講じてください。 - * サーバーはファイアウォールで保護されており、信頼できないネットワークからはアクセスできないこと。 * すべてのユーザーはネットワークアドレスのサブセットに制限されていること(`users.xml` を参照)。 * すべてのユーザーが十分に強力なパスワードを持ち、セキュアな (TLS) インターフェイスのみがアクセス可能であるか、もしくは接続が TLS インターフェイス経由でのみ行われていること。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/exception-too-many-parts.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/exception-too-many-parts.mdx index c5960131c28..ab5c16127ed 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/exception-too-many-parts.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/exception-too-many-parts.mdx @@ -10,8 +10,7 @@ keywords: ['Too many parts'] {/* 省略 */} - -## DB::Exception: パーツが多すぎます (Error: 252)。マージ処理が挿入よりも著しく遅くなっています +## DB::Exception: パーツが多すぎます (Error: 252)。マージ処理が挿入よりも著しく遅くなっています \{#dbexception-too-many-parts-252-merges-are-processing-significantly-slower-than-inserts\} MergeTree テーブルで `parts_to_throw_insert` 設定の上限に達しました。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/exchangeStatementToSwitchTables.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/exchangeStatementToSwitchTables.mdx index 70956e855f8..2d73da3dd25 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/exchangeStatementToSwitchTables.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/exchangeStatementToSwitchTables.mdx @@ -12,12 +12,11 @@ keywords: ['EXCHANGE'] *** - ## 質問 \{#question\} [`EXCHANGE`](/sql-reference/statements/exchange) コマンドを使ってテーブル名を入れ替えるにはどうすればよいですか? -## 回答 +## 回答 \{#answer\} `EXCHANGE` コマンドは、現在のテーブルを、一時的に使用している別のテーブル(そのテーブルで Primary Key やその他の設定が更新されている可能性があるもの)と入れ替える必要がある場合に役立ちます。 この操作は `RENAME` コマンドと異なり、アトミックに実行されます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/execute-system-queries-in-cloud.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/execute-system-queries-in-cloud.mdx index 57baca78b1b..bcafc523b68 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/execute-system-queries-in-cloud.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/execute-system-queries-in-cloud.mdx @@ -10,8 +10,7 @@ keywords: ['clusterAllReplicas'] {/* 切り捨て */} - -## 解答 +## 解答 \{#answer\} ClickHouse Cloud サービス内のすべてのノードで同じクエリを実行するには、[clusterAllReplicas](https://clickhouse.com/docs/sql-reference/table-functions/cluster/) を使用します。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/file-export.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/file-export.mdx index 516073b525b..aacfdcf62d1 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/file-export.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/file-export.mdx @@ -10,8 +10,7 @@ keywords: ['データのエクスポート', 'INTO OUTFILE', 'File テーブル {/* 省略 */} - -## INTO OUTFILE 句の使用 +## INTO OUTFILE 句の使用 \{#using-into-outfile-clause\} クエリに [INTO OUTFILE](https://clickhouse.com/docs/sql-reference/statements/select/into-outfile) 句を追加します。 @@ -48,8 +47,7 @@ INTO OUTFILE 'taxi_rides.txt' FORMAT CSV ``` - -## File テーブルエンジンを使用する +## File テーブルエンジンを使用する \{#using-a-file-engine-table\} 別の選択肢として、[File](https://clickhouse.com/docs/engines/table-engines/special/file) テーブルエンジンを使用する方法があります。この場合、ClickHouse はデータを保存するためにファイルを利用します。ファイルに対して直接クエリの実行やデータの挿入ができます。 @@ -79,8 +77,7 @@ INSERT INTO my_table VALUES `File` テーブルエンジンは、ファイルシステム上のファイルを作成したりクエリを実行したりするのに非常に便利ですが、`File` テーブルは `MergeTree` テーブルではないため、`MergeTree` が提供するすべての利点は得られないことに注意してください。ClickHouse からデータを使いやすい形式でエクスポートする際の利便性を目的として `File` を使用してください。 ::: - -## コマンドラインでのリダイレクトの使用 +## コマンドラインでのリダイレクトの使用 \{#using-command-line-redirection\} ```bash $ clickhouse-client --query "SELECT * from table" --format FormatName > result.txt diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/filtered-aggregates.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/filtered-aggregates.mdx index cc131d548af..cc98f2b3bc9 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/filtered-aggregates.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/filtered-aggregates.mdx @@ -10,8 +10,7 @@ keywords: ['フィルタ付き集約'] {/* 切り捨て */} - -## フィルタ付き集約の利用 +## フィルタ付き集約の利用 \{#using-filtered-aggregates\} ClickHouse では、*フィルタ付き集約* を記述するためのシンプルで直感的な方法が提供されています。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/find-expensive-queries.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/find-expensive-queries.mdx index ee8901abda7..515159a2c20 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/find-expensive-queries.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/find-expensive-queries.mdx @@ -11,8 +11,7 @@ keywords: ['高コストなクエリ'] {/* 省略 */} - -## 回答 +## 回答 \{#answer\} `system` データベースの `query_log` テーブルは、すべてのクエリを記録しており、次の情報を含みます: @@ -53,7 +52,6 @@ LIMIT 10; └─────────────┴─────────────────────┴──────────────────────────────────────┴────────────┴─────────────┴────────────┴───────────────────────┘ ``` - :::note `initial_query_id` は、リクエストを受信したノードから開始される分散クエリ実行における最初のクエリの ID を表します。`query_id` には、別のノード上で実行される子クエリの ID が含まれます。詳細は [この記事](https://clickhouse.com/docs/knowledgebase/find-expensive-queries#initial_query_id-vs-query_id) を参照してください。 ::: @@ -95,8 +93,7 @@ SELECT FROM s3Cluster('default','https://clickhouse-public-datasets.s3.amazonaws.com/youtube/original/files/*.zst', 'JSONLines') ``` - -## initial_query_id VS query_id +## initial_query_id VS query_id \{#initial_query_id-vs-query_id\} クラスタ構成の ClickHouse 環境(ClickHouse Cloud など)では、`initial_query_id` はリクエストを受け付けたノードから起動される分散クエリ実行における、最初に発行されたクエリの ID を表します。 この場合、`query_id` フィールドには、別のノード上で実行される子クエリの ID が含まれます。 @@ -122,7 +119,6 @@ LIMIT 10 FORMAT Pretty; 次のような結果が得られます: - ``` ┏━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ hostname() ┃ type ┃ event_time ┃ initial_query_id ┃ query_id ┃ memory ┃ userCPU ┃ systemCPU ┃ normalized_query_hash ┃ @@ -185,7 +181,6 @@ LIMIT 10 FORMAT Pretty; └────────────────────────┴─────────────┴─────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴────────────┴─────────┴───────────┴───────────────────────┘ ``` - 一方、`query_id = initial_query_id` を指定すると、分散クエリが最初に発行されたローカルノード上で実行されたクエリのみが返されます。 ```sql diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/finding_expensive_queries_by_memory_usage.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/finding_expensive_queries_by_memory_usage.mdx index 9d67333b23f..348f0800080 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/finding_expensive_queries_by_memory_usage.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/finding_expensive_queries_by_memory_usage.mdx @@ -10,8 +10,7 @@ keywords: ['負荷の高いクエリ', 'メモリ使用量'] {/* 切り詰め */} - -## `system.query_log` テーブルの使用 +## `system.query_log` テーブルの使用 \{#using-the-systemquery_log-table\} 次の便利なクエリを使用すると、実行されたクエリのうち、どれが最も多くのメモリを使用したかを確認できます。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/fix-developer-verification-error-in-macos.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/fix-developer-verification-error-in-macos.mdx index e5d2445e11d..d2b94bebef4 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/fix-developer-verification-error-in-macos.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/fix-developer-verification-error-in-macos.mdx @@ -17,7 +17,6 @@ import privacy_allow from "@site/static/images/knowledgebase/fix-the-developer-v *** - ## macOS の開発元検証エラーを解消する \{#fix-the-developer-verification-error-in-macos\} `brew` を使って ClickHouse をインストールした場合、macOS でエラーが表示されることがあります。 @@ -34,20 +33,22 @@ import privacy_allow from "@site/static/images/knowledgebase/fix-the-developer-v `clickhouse` 実行ファイルを隔離領域から削除する最も簡単な方法は次のとおりです。 1. **システム設定** を開きます。 -1. **プライバシーとセキュリティ** に移動します。 - macOS の「プライバシーとセキュリティ」設定のデフォルト表示 +2. **プライバシーとセキュリティ** に移動します。 + + macOS の「プライバシーとセキュリティ」設定のデフォルト表示 + +3. ウィンドウの一番下までスクロールし、*"clickhouse-macos-aarch64" was blocked from use because it is not from an identified developer"* というメッセージを探します。 -1. ウィンドウの一番下までスクロールし、_"clickhouse-macos-aarch64" was blocked from use because it is not from an identified developer"_ というメッセージを探します。 -1. **このまま開く** をクリックします。 +4. **このまま開く** をクリックします。 - macOS の「プライバシーとセキュリティ」設定で「このまま開く」ボタンが表示されている状態 + macOS の「プライバシーとセキュリティ」設定で「このまま開く」ボタンが表示されている状態 -1. macOS のユーザーパスワードを入力します。 +5. macOS のユーザーパスワードを入力します。 これでターミナルから `clickhouse` コマンドを実行できるようになります。 -## ターミナルでの手順 +## ターミナルでの手順 \{#terminal-process\} `Allow Anyway` ボタンを押してもこの問題が解決しないことがあります。その場合は、コマンドラインから同じ処理を実行することもできます。 あるいは、単にコマンドラインのほうが好みという場合もあるでしょう。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/generate-har-file.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/generate-har-file.mdx index b626020186f..1c8929d98aa 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/generate-har-file.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/generate-har-file.mdx @@ -10,7 +10,6 @@ description: 'HAR (HTTP Archive) ファイルは、ブラウザーのネット {/* 切り捨て */} - ## Google Chrome から \{#from-google-chrome\} 1. F12 キー、または Ctrl + Shift + I(Windows/Linux) / Cmd + Option + I(Mac)を押して、デベロッパーツールを開きます。 @@ -35,9 +34,9 @@ description: 'HAR (HTTP Archive) ファイルは、ブラウザーのネット ## Safari から \{#from-safari\} 1. 開発ツールを有効にします(まだ有効にしていない場合): - - Safari > 設定 > 詳細 を開きます。 - - 一番下の「メニューバーに“開発”メニューを表示」をオンにします。 -2. 「開発」>「Web インスペクタを表示」をクリックします。 + * Safari > 設定 > 詳細 を開きます。 + * 一番下の「メニューバーに“開発”メニューを表示」をオンにします。 +2. 「開発」>「Web インスペクタを表示」をクリックします。 3. 「ネットワーク」タブをクリックします。 4. ページを再読み込みして、問題を再現します。 5. 開発ツールから「書き出す」ボタンをクリックします。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/how-do-i-contribute-code-to-clickhouse.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/how-do-i-contribute-code-to-clickhouse.mdx index 70562c67f45..460b9196b24 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/how-do-i-contribute-code-to-clickhouse.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/how-do-i-contribute-code-to-clickhouse.mdx @@ -10,7 +10,6 @@ keywords: ['コントリビューション', 'オープンソース'] {/* 切り捨て */} - ## ClickHouse への貢献方法 \{#how-to-contribute-in-clickhouse\} ClickHouse は、[GitHub 上で開発されている](https://github.com/ClickHouse/ClickHouse)オープンソースプロジェクトです。 diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/how-to-check-my-clickhouse-cloud-sevice-state.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/how-to-check-my-clickhouse-cloud-sevice-state.mdx index 1e610ee3e4d..9d197ee7ea1 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/how-to-check-my-clickhouse-cloud-sevice-state.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/how-to-check-my-clickhouse-cloud-sevice-state.mdx @@ -10,7 +10,6 @@ keywords: ['クラウドサービスの状態'] {/* 途中省略 */} - ## ClickHouse Cloud サービスの状態を確認する方法 \{#how-to-check-your-clickhouse-cloud-service-state\} ClickHouse Cloud サービスの状態はどのように確認できますか?サービスが停止、アイドル状態、または実行中かを確認したいのですが、その際にサービスを起動させたくありません。 @@ -19,8 +18,8 @@ ClickHouse Cloud サービスの状態はどのように確認できますか? [ClickHouse Cloud API](/cloud/manage/api/api-overview) は、クラウドサービスの状態を確認するのに便利です。Cloud API を使用する前に、対象サービスで API キーを作成する必要があります。これは ClickHouse Cloud の [clickhouse.cloud](https://console.clickhouse.cloud) で行えます: -- [API 概要](/cloud/manage/api/api-overview) -- [Swagger](https://clickhouse.com/docs/cloud/manage/api/swagger) +* [API 概要](/cloud/manage/api/api-overview) +* [Swagger](https://clickhouse.com/docs/cloud/manage/api/swagger) 1. サービスの状態を確認するには、次を実行します。`Key-ID` と `Key-Secret` はそれぞれ自身の値に置き換えてください。 @@ -34,7 +33,7 @@ ClickHouse Cloud サービスの状態はどのように確認できますか? result":{"id":"[Service-ID]","name":"[Service-Name]","provider":"aws","region":"us-east-1","state":"**idle**","endpoints":[{"protocol":"nativesecure","host":"[Connect-URL]","port":9440},{"protocol":"https","host":"[Connect-URL]","port":8443}],"tier":"development","idleScaling":true,"idleTimeoutMinutes":15,"ipAccessList":[{"source":"[my-IP]","description":"[my-IP-name]"}],"createdAt":"2023-04-13T23:47:47Z"},"status":200} ``` -1. [jq ユーティリティ](https://jqlang.github.io/jq/) を使用して `state` キーのみを抽出できます。 +2. [jq ユーティリティ](https://jqlang.github.io/jq/) を使用して `state` キーのみを抽出できます。 ```shell curl --user '[Key-ID]:[Key-Secret]' https://api.clickhouse.cloud/v1/organizations/[Org-ID]/services/[Service-ID] | jq '.state' @@ -46,7 +45,7 @@ ClickHouse Cloud サービスの状態はどのように確認できますか? **idle** ``` -1. 稼働中のサービスに対して同じコマンドを実行すると、次のような出力が得られます。 +3. 稼働中のサービスに対して同じコマンドを実行すると、次のような出力が得られます。 ```json **running** diff --git a/i18n/jp/docusaurus-plugin-content-blog/current/how-to-connect-to-ch-cloud-using-ssh-keys.mdx b/i18n/jp/docusaurus-plugin-content-blog/current/how-to-connect-to-ch-cloud-using-ssh-keys.mdx index 34c74f09a62..f271400f4b6 100644 --- a/i18n/jp/docusaurus-plugin-content-blog/current/how-to-connect-to-ch-cloud-using-ssh-keys.mdx +++ b/i18n/jp/docusaurus-plugin-content-blog/current/how-to-connect-to-ch-cloud-using-ssh-keys.mdx @@ -10,7 +10,6 @@ keywords: ['クラウド', 'SSH'] {/* 切り捨て */} - ## 質問 \{#question\} SSH 鍵認証を使用して ClickHouse に接続するにはどうすればよいですか? @@ -19,9 +18,9 @@ SSH 鍵認証を使用して ClickHouse に接続するにはどうすればよ ここでは例として ClickHouse Cloud を使用していますが、この手順は OSS 版 ClickHouse でも同様に利用できます。 ::: - + + + +fullscreen; +picture-in-picture" + allowfullscreen + /> -
+
現在データがどこにあるかに応じて、ClickHouse Cloud にデータを移行する方法はいくつかあります。 -- [セルフマネージド環境から Cloud へ](/cloud/migration/clickhouse-to-cloud): `remoteSecure` 関数を使用してデータを転送します -- [別の DBMS から](/cloud/migration/clickhouse-local): [clickhouse-local] ETL ツールと、現在利用中の DBMS に対応する ClickHouse のテーブル関数を組み合わせて使用します -- [どこからでも!](/cloud/migration/etl-tool-to-clickhouse): さまざまな種類のデータソースに接続できる、広く利用されている ETL/ELT ツールのいずれかを使用します -- [オブジェクトストレージから](/integrations/migration/object-storage-to-clickhouse): S3 から ClickHouse へ簡単にデータを取り込めます +* [セルフマネージド環境から Cloud へ](/cloud/migration/clickhouse-to-cloud): `remoteSecure` 関数を使用してデータを転送します +* [別の DBMS から](/cloud/migration/clickhouse-local): [clickhouse-local] ETL ツールと、現在利用中の DBMS に対応する ClickHouse のテーブル関数を組み合わせて使用します +* [どこからでも!](/cloud/migration/etl-tool-to-clickhouse): さまざまな種類のデータソースに接続できる、広く利用されている ETL/ELT ツールのいずれかを使用します +* [オブジェクトストレージから](/integrations/migration/object-storage-to-clickhouse): S3 から ClickHouse へ簡単にデータを取り込めます [Redshift からの移行](/migrations/redshift/migration-guide)の例では、ClickHouse にデータを移行する 3 つの異なる方法を紹介しています。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md index 1b1ec86cb56..21c89290d82 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# ClickHouse と PostgreSQL の比較 +# ClickHouse と PostgreSQL の比較 {#comparing-clickhouse-and-postgresql} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md index 91a64ccd7a6..20c74d9da68 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md @@ -129,7 +129,7 @@ ClickHouse は、[限定された構成](/guides/developer/transactional) の下 -## 圧縮 +## 圧縮 {#compression} ClickHouse のカラム指向ストレージでは、Postgres と比較して圧縮効率が大幅に優れることがよくあります。以下は、両方のデータベースで Stack Overflow の全テーブルを格納した場合のストレージ要件を比較した例です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md index 9e5c1fc20d5..ac242a28c78 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md @@ -31,49 +31,49 @@ Postgres から ClickHouse への典型的なマイグレーション例を示 ```bash -# ユーザー +# ユーザー {#users} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/users.sql.gz gzip -d users.sql.gz psql < users.sql ``` -# posts +# posts {#posts} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posts.sql.gz gzip -d posts.sql.gz psql < posts.sql -# posthistory +# posthistory {#posthistory} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posthistory.sql.gz gzip -d posthistory.sql.gz psql < posthistory.sql -# コメント +# コメント {#comments} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/comments.sql.gz gzip -d comments.sql.gz psql < comments.sql -# 投票 +# 投票 {#votes} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/votes.sql.gz gzip -d votes.sql.gz psql < votes.sql -# badges +# badges {#badges} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/badges.sql.gz gzip -d badges.sql.gz psql < badges.sql -# postlinks +# postlinks {#postlinks} wget [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz) gzip -d postlinks.sql.gz @@ -87,9 +87,9 @@ ClickHouseにとっては小規模ですが、このデータセットはPostgre ``` -## データの移行 +## データの移行 {#migrating-data} -### リアルタイムレプリケーション(CDC) +### リアルタイムレプリケーション(CDC) {#real-time-replication-or-cdc} PostgreSQL 用の ClickPipes をセットアップするには、この[ガイド](/integrations/clickpipes/postgres)を参照してください。このガイドでは、さまざまな種類のソースとなる Postgres インスタンスを扱っています。 @@ -125,7 +125,7 @@ ORDER BY id; セットアップが完了すると、ClickPipes は PostgreSQL から ClickHouse へのすべてのデータ移行を開始します。ネットワークやデプロイメントの規模によって所要時間は異なりますが、Stack Overflow データセットであれば数分程度で完了するはずです。 -### 手動による一括ロードと定期更新 +### 手動による一括ロードと定期更新 {#initial-bulk-load-with-periodic-updates} 手動アプローチを用いる場合、データセットの初回一括ロードは次の方法で実施できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md index 803667e73c1..37bd95e03b9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md @@ -21,7 +21,7 @@ CDC を利用したリアルタイムレプリケーションで PostgreSQL か -## ClickHouse でクエリを最適化する +## ClickHouse でクエリを最適化する {#optimize-queries-in-clickhouse} 最小限のクエリ書き換えで移行することも可能ですが、クエリを大幅に単純化し、さらにクエリパフォーマンスを向上させるために、ClickHouse の機能を活用することを推奨します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md index dd7dc90718e..fa6a319abb2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md @@ -49,7 +49,7 @@ CDC を用いたリアルタイムレプリケーションを利用する場合 -## パーティション +## パーティション {#partitions} Postgres ユーザーであれば、テーブルをより小さく扱いやすい単位であるパーティションに分割することで、大規模データベースのパフォーマンスと管理性を向上させるテーブルパーティショニングの概念にはなじみがあるはずです。このパーティショニングは、指定した列(例: 日付)の範囲や、定義済みリスト、あるいはキーに対するハッシュを用いることで実現できます。これにより、管理者は日付範囲や地理的な位置などの特定の条件に基づいてデータを整理できます。パーティショニングは、パーティションプルーニングや効率的なインデックス作成による高速なデータアクセスを可能にすることで、クエリ性能の向上に寄与します。また、テーブル全体ではなく個々のパーティション単位で操作できるため、バックアップやデータ削除といったメンテナンス作業の効率化にもつながります。さらに、負荷を複数のパーティションに分散させることで、PostgreSQL データベースのスケーラビリティを大幅に向上させることができます。 @@ -76,7 +76,7 @@ PARTITION BY toYear(CreationDate) パーティションについての詳細な説明は ["Table partitions"](/partitions) を参照してください。 -### パーティションの用途 +### パーティションの用途 {#applications-of-partitions} ClickHouse におけるパーティションは、Postgres と同様の用途がありますが、いくつか細かな違いがあります。より具体的には次のとおりです。 @@ -132,7 +132,7 @@ OK -## マテリアライズドビューとプロジェクションの比較 +## マテリアライズドビューとプロジェクションの比較 {#materialized-views-vs-projections} Postgres では、単一のテーブルに対して複数のインデックスを作成でき、さまざまなアクセスパターンに最適化できます。この柔軟性により、管理者や開発者は特定のクエリや運用要件に合わせてデータベースのパフォーマンスを調整できます。ClickHouse におけるプロジェクションの概念はこれと完全に同じではありませんが、テーブルに対して複数の `ORDER BY` 句を指定できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md index 1aed389601f..1593003ce1b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md @@ -12,7 +12,7 @@ import bigquery_1 from '@site/static/images/migrations/bigquery-1.png'; import Image from '@theme/IdealImage'; -# ClickHouse Cloud と BigQuery の比較 +# ClickHouse Cloud と BigQuery の比較 {#comparing-clickhouse-cloud-and-bigquery} @@ -186,7 +186,7 @@ ClickHouse は、分析タスクにより適したものとなるよう、多く -## 配列 +## 配列 {#arrays} BigQuery の配列関数が 8 個であるのに対して、ClickHouse には 80 個以上の[組み込み配列関数](/sql-reference/functions/array-functions)があり、幅広い問題をエレガントかつシンプルにモデリング・解決できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md index 07ee62f5e5c..3cdac048853 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md @@ -75,7 +75,7 @@ Change Data Capture(CDC)は、2 つのデータベース間でテーブル -## スキーマの設計 +## スキーマの設計 {#designing-schemas} Stack Overflow のデータセットには、関連する多数のテーブルが含まれています。まずは主要なテーブルの移行に集中することを推奨します。これは必ずしも最大のテーブルとは限らず、むしろ分析クエリの発行が最も多いと想定されるテーブルです。そうすることで、主要な ClickHouse の概念に慣れることができます。このテーブルは、追加のテーブルが増えるにつれて、ClickHouse の機能を最大限に活用し最適なパフォーマンスを得るために、再モデリングが必要になる場合があります。このモデリングプロセスについては、[データモデリングのドキュメント](/data-modeling/schema-design#next-data-modeling-techniques)で解説しています。 @@ -108,7 +108,7 @@ CREATE TABLE stackoverflow.posts ( ); ``` -### 型の最適化 +### 型の最適化 {#optimizing-types} [こちら](/data-modeling/schema-design)で説明しているプロセスに従うと、次のようなスキーマになります。 @@ -174,11 +174,11 @@ ClickHouse で選択したプライマリキーは、インデックスだけで -## データモデリング手法 +## データモデリング手法 {#data-modeling-techniques} BigQuery から移行するユーザーは、まず [ClickHouse におけるデータモデリングガイド](/data-modeling/schema-design) を参照することを推奨します。このガイドでは同じ Stack Overflow データセットを使用し、ClickHouse の機能を活用した複数のアプローチを解説しています。 -### パーティション +### パーティション {#partitions} BigQuery ユーザーは、大規模なデータベースにおいて、テーブルを「パーティション」と呼ばれる小さく扱いやすい単位に分割することで、性能と管理性を向上させるテーブルパーティショニングの概念に馴染みがあるはずです。パーティショニングは、指定したカラム(例: 日付)に対する範囲、定義済みリスト、あるいはキーに対するハッシュによって実現できます。これにより、管理者は日付範囲や地理的位置といった特定の条件に基づいてデータを整理できます。 @@ -205,7 +205,7 @@ ORDER BY (PostTypeId, toDate(CreationDate), CreationDate) PARTITION BY toYear(CreationDate) ``` -#### 用途 +#### 用途 {#applications} ClickHouse におけるパーティショニングの用途は BigQuery と似ていますが、いくつか細かな違いがあります。具体的には次のとおりです。 @@ -259,7 +259,7 @@ ALTER TABLE posts -## マテリアライズドビューとプロジェクション +## マテリアライズドビューとプロジェクション {#materialized-views-vs-projections} ClickHouse のプロジェクションの概念により、ユーザーは 1 つのテーブルに対して複数の `ORDER BY` 句を指定できます。 @@ -396,7 +396,7 @@ WHERE UserId = 8592047 ``` -## ClickHouse 向けの BigQuery クエリの書き換え +## ClickHouse 向けの BigQuery クエリの書き換え {#rewriting-bigquery-queries-in-clickhouse} 以下は、BigQuery と ClickHouse のクエリを比較したサンプルクエリです。このリストは、ClickHouse の機能を活用してクエリを大幅に簡素化する方法を示すことを目的としています。ここでの例では、Stack Overflow の全データセット(2024 年 4 月まで)を使用します。 @@ -464,7 +464,7 @@ LIMIT 5 ``` -## 集約関数 +## 集約関数 {#aggregate-functions} 可能な場合は、ClickHouse の集約関数を活用してください。以下では、各年でもっとも閲覧された質問を求めるために、[`argMax` 集約関数](/sql-reference/aggregate-functions/reference/argmax) を使用する例を示します。 @@ -519,7 +519,7 @@ MaxViewCount: 66975 ``` -## 条件式と配列 +## 条件式と配列 {#conditionals-and-arrays} 条件式と配列関数を使うと、クエリを大幅に簡潔にできます。次のクエリは、2022 年から 2023 年にかけての出現回数が 10000 回を超えるタグのうち、増加率が最も大きいものを算出します。以下の ClickHouse クエリが、条件式・配列関数・`HAVING` 句および `SELECT` 句でエイリアスを再利用できる機能のおかげで簡潔になっている点に注目してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md index 6d4a02d2af3..7357b4280c6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md @@ -30,7 +30,7 @@ BigQuery から ClickHouse へのデータエクスポートにかかる時間 -## テーブルデータを GCS にエクスポートする +## テーブルデータを GCS にエクスポートする {#1-export-table-data-to-gcs} この手順では、[BigQuery SQL ワークスペース](https://cloud.google.com/bigquery/docs/bigquery-web-ui) を使用して SQL 文を実行します。ここでは、[`EXPORT DATA`](https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements) ステートメントを使用して、`mytable` という BigQuery テーブルを GCS のバケットにエクスポートします。 @@ -67,7 +67,7 @@ END WHILE; * 列指向フォーマットである Parquet は、標準で圧縮されており、BigQuery によるエクスポートおよび ClickHouse によるクエリが高速であるため、より優れたデータ交換形式です。 -## GCS から ClickHouse へのデータインポート +## GCS から ClickHouse へのデータインポート {#2-importing-data-into-clickhouse-from-gcs} エクスポートが完了したら、このデータを ClickHouse のテーブルにインポートできます。以下のコマンドを実行するには、[ClickHouse SQL console](/integrations/sql-clients/sql-console) か [`clickhouse-client`](/interfaces/cli) を使用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md index b477425dc1e..9bf8778db6c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md @@ -13,7 +13,7 @@ import cloud_architecture from '@site/static/images/cloud/onboard/discover/use_c import Image from '@theme/IdealImage'; -# Snowflake から ClickHouse への移行 +# Snowflake から ClickHouse への移行 {#snowflake-to-clickhouse-migration} > このドキュメントでは、Snowflake から ClickHouse へのデータ移行の概要を説明します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md index 0699341a4b9..3d616d779c9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md @@ -12,7 +12,7 @@ import migrate_snowflake_clickhouse from '@site/static/images/migrations/migrate import Image from '@theme/IdealImage'; -# SnowflakeからClickHouseへの移行 +# SnowflakeからClickHouseへの移行 {#migrate-from-snowflake-to-clickhouse} > 本ガイドでは、SnowflakeからClickHouseへデータを移行する方法について説明します。 @@ -21,7 +21,7 @@ SnowflakeとClickHouse間でデータを移行するには、転送用の中間 -## Snowflake からデータをエクスポートする +## Snowflake からデータをエクスポートする {#1-exporting-data-from-snowflake} Snowflake から ClickHouse への移行 @@ -59,7 +59,7 @@ COPY INTO @external_stage/mydataset from mydataset max_file_size=157286400 heade 約 5TB のデータセットで最大ファイルサイズが 150MB、かつ同じ AWS `us-east-1` リージョン内にある 2X-Large Snowflake ウェアハウスを使用する場合、S3 バケットへのデータのコピーには約 30 分かかります。 -## ClickHouse へのインポート +## ClickHouse へのインポート {#2-importing-to-clickhouse} データが中間オブジェクトストレージにステージングされたら、以下のように [s3 テーブル関数](/sql-reference/table-functions/s3) などの ClickHouse の関数を使用して、テーブルにデータを挿入できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md index c1a31c32cc2..dacd8e3ec5d 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# Snowflake SQL 変換ガイド +# Snowflake SQL 変換ガイド {#snowflake-sql-translation-guide} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md index b5e1ee5aeb7..d197b1233c8 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md @@ -8,6 +8,6 @@ show_related_blogs: true doc_type: 'landing-page' --- -# Elasticsearch から ClickHouse への移行 +# Elasticsearch から ClickHouse への移行 {#elasticsearch-to-clickhouse-migration} オブザーバビリティのユースケースについては、[Elasticsearch から ClickStack への移行](/use-cases/observability/clickstack/migration/elastic) の移行ドキュメントを参照してください。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md index ce49f001ee5..32f1c411563 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Amazon Redshift から ClickHouse への移行 +# Amazon Redshift から ClickHouse への移行 {#amazon-redshift-to-clickhouse-migration} > このドキュメントでは、Amazon Redshift から ClickHouse へのデータ移行の概要を説明します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md index 33827b48ceb..cd2404f2389 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md @@ -7,9 +7,8 @@ title: 'Amazon Redshift から ClickHouse への移行ガイド' doc_type: 'guide' --- -import MigrationGuide from '@site/docs/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +import MigrationGuide from '@site/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +# Amazon Redshift から ClickHouse への移行ガイド {#amazon-redshift-to-clickhouse-migration-guide} -# Amazon Redshift から ClickHouse への移行ガイド - - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md index 32a05db8eee..98c69a3b873 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Amazon Redshift SQL 変換ガイド +# Amazon Redshift SQL 変換ガイド {#amazon-redshift-sql-translation-guide} @@ -52,9 +52,9 @@ ClickHouse と Redshift 間でデータを移動するユーザーは、ClickHou -## DDL 構文 +## DDL 構文 {#compression} -### ソートキー +### ソートキー {#sorting-keys} ClickHouse と Redshift の両方には「ソートキー」という概念があり、 データを保存する際にどのような順序で格納するかを定義します。Redshift では、 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md index 63ddb3a17e6..e4f4b779086 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md @@ -8,7 +8,7 @@ keywords: ['移行', 'ClickHouse Cloud', 'OSS', 'セルフマネージド環境 --- import Image from '@theme/IdealImage'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import self_managed_01 from '@site/static/images/integrations/migration/self-managed-01.png'; import self_managed_02 from '@site/static/images/integrations/migration/self-managed-02.png'; import self_managed_03 from '@site/static/images/integrations/migration/self-managed-03.png'; @@ -16,16 +16,13 @@ import self_managed_04 from '@site/static/images/integrations/migration/self-man import self_managed_05 from '@site/static/images/integrations/migration/self-managed-05.png'; import self_managed_06 from '@site/static/images/integrations/migration/self-managed-06.png'; +# セルフマネージド ClickHouse と ClickHouse Cloud 間の移行 {#migrating-between-self-managed-clickhouse-and-clickhouse-cloud} -# セルフマネージド ClickHouse と ClickHouse Cloud 間の移行 - -セルフマネージド ClickHouse の移行 +セルフマネージド ClickHouse の移行 このガイドでは、セルフマネージドな ClickHouse サーバーから ClickHouse Cloud への移行方法と、ClickHouse Cloud のサービス間での移行方法を説明します。[`remoteSecure`](/sql-reference/table-functions/remote) 関数は、`SELECT` および `INSERT` クエリでリモートの ClickHouse サーバーにアクセスするために使用します。これにより、`SELECT` を埋め込んだ `INSERT INTO` クエリを書くことで、テーブルを簡単に移行できます。 - - -## 自前運用の ClickHouse から ClickHouse Cloud への移行 +## 自前運用の ClickHouse から ClickHouse Cloud への移行 {#migrating-from-self-managed-clickhouse-to-clickhouse-cloud} 自前運用の ClickHouse からの移行 @@ -36,7 +33,7 @@ ClickHouse Cloud が垂直・水平スケーリングを自動的に処理する この例では、自前運用の ClickHouse サーバーが *ソース*、ClickHouse Cloud サービスが *宛先* になります。 -### 概要 +### 概要 {#overview} 手順は次のとおりです。 @@ -46,11 +43,11 @@ ClickHouse Cloud が垂直・水平スケーリングを自動的に処理する 4. (該当する場合)宛先側の IP アクセスリストからソースサーバーを削除する 5. ソース側のサービスから読み取り専用ユーザーを削除する -### あるシステムから別のシステムへのテーブルの移行 +### あるシステムから別のシステムへのテーブルの移行 {#migration-of-tables-from-one-system-to-another} この例では、1 つのテーブルを自前運用の ClickHouse サーバーから ClickHouse Cloud に移行します。 -### ソース側の ClickHouse システム(現在データを保持しているシステム)での作業 +### ソース側の ClickHouse システム(現在データを保持しているシステム)での作業 {#on-the-source-clickhouse-system-the-system-that-currently-hosts-the-data} * ソーステーブル(この例では `db.table`)を読み取ることができる読み取り専用ユーザーを追加します @@ -72,7 +69,7 @@ FROM system.tables WHERE database = 'db' AND table = 'table' ``` -### 宛先側の ClickHouse Cloud システム上で: +### 宛先側の ClickHouse Cloud システム上で: {#on-the-destination-clickhouse-cloud-system} * 宛先データベースを作成します。 @@ -119,8 +116,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 'default', 'PASS') SELECT * FROM db.table ``` - -## ClickHouse Cloud サービス間での移行 +## ClickHouse Cloud サービス間での移行 {#migrating-between-clickhouse-cloud-services} 自己管理型 ClickHouse の移行 @@ -143,7 +139,7 @@ ClickHouse Cloud サービス間でデータを移行する主なユースケー 6. デスティネーション上で IP Access List を再設定する 7. ソースサービスから読み取り専用ユーザーを削除する -#### ソースサービスに読み取り専用ユーザーを追加する +#### ソースサービスに読み取り専用ユーザーを追加する {#add-a-read-only-user-to-the-source-service} * ソーステーブル(この例では `db.table`)を読み取れる読み取り専用ユーザーを追加します @@ -164,7 +160,7 @@ ClickHouse Cloud サービス間でデータを移行する主なユースケー where database = 'db' and table = 'table' ``` -#### デスティネーションサービス上でテーブル構造を複製する +#### デスティネーションサービス上でテーブル構造を複製する {#duplicate-the-table-structure-on-the-destination-service} デスティネーションにまだデータベースが存在しない場合は、先に作成します: @@ -181,7 +177,7 @@ ClickHouse Cloud サービス間でデータを移行する主なユースケー CREATE TABLE db.table ... ``` -#### ソースサービスへのリモートアクセスを許可する +#### ソースサービスへのリモートアクセスを許可する {#allow-remote-access-to-the-source-service} ソースからデスティネーションへデータをプルするには、ソースサービスが接続を許可している必要があります。ソースサービス上で一時的に「IP Access List」機能を無効化します。 @@ -191,7 +187,7 @@ ClickHouse Cloud サービス間でデータを移行する主なユースケー 許可リストを編集し、一時的に **Anywhere** からのアクセスを許可します。詳細については [IP Access List](/cloud/security/setting-ip-filters) ドキュメントを参照してください。 -#### ソースからデスティネーションへデータをコピーする +#### ソースからデスティネーションへデータをコピーする {#copy-the-data-from-source-to-destination} * `remoteSecure` 関数を使用して、ソースの ClickHouse Cloud サービスからデータをプルします。 デスティネーションに接続し、デスティネーション側の ClickHouse Cloud サービスで次のコマンドを実行します: @@ -203,11 +199,11 @@ ClickHouse Cloud サービス間でデータを移行する主なユースケー * デスティネーションサービス上のデータを確認します -#### ソース上で IP Access List を再設定する +#### ソース上で IP Access List を再設定する {#re-establish-the-ip-access-list-on-the-source} 以前にアクセスリストをエクスポートしている場合は、**Share** を使って再インポートできます。エクスポートしていない場合は、アクセスリストにエントリを再度追加してください。 -#### 読み取り専用ユーザー `exporter` を削除する +#### 読み取り専用ユーザー `exporter` を削除する {#remove-the-read-only-exporter-user} ```sql DROP USER exporter diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md index 497022943f0..dce22244c4e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md @@ -11,14 +11,14 @@ import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import ch_local_01 from '@site/static/images/integrations/migration/ch-local-01.png'; import ch_local_02 from '@site/static/images/integrations/migration/ch-local-02.png'; import ch_local_03 from '@site/static/images/integrations/migration/ch-local-03.png'; import ch_local_04 from '@site/static/images/integrations/migration/ch-local-04.png'; -# clickhouse-local を使用した ClickHouse への移行 +# clickhouse-local を使用した ClickHouse への移行 {#migrating-to-clickhouse-using-clickhouse-local} セルフマネージド ClickHouse の移行 @@ -91,21 +91,21 @@ Mac で `clickhouse-local` を実行するには、`./clickhouse local` を使 -## 例 1: Integration テーブルエンジンを使用して MySQL から ClickHouse Cloud へ移行する +## 例 1: Integration テーブルエンジンを使用して MySQL から ClickHouse Cloud へ移行する {#example-1-migrating-from-mysql-to-clickhouse-cloud-with-an-integration-engine} ソースの MySQL データベースからデータを読み取るために、[mysql テーブル関数](/sql-reference/table-functions/mysql/) によって動的に作成される [integration テーブルエンジン](/engines/table-engines/integrations/mysql/) を使用し、ClickHouse Cloud サービス上の宛先テーブルにデータを書き込むために [remoteSecure テーブル関数](/sql-reference/table-functions/remote/) を使用します。 自己管理型 ClickHouse からの移行 -### 宛先側の ClickHouse Cloud サービスで: +### 宛先側の ClickHouse Cloud サービスで: {#on-the-destination-clickhouse-cloud-service} -#### 宛先データベースを作成する: +#### 宛先データベースを作成する: {#create-the-destination-database} ```sql CREATE DATABASE db ``` -#### MySQL テーブルと同じスキーマを持つ出力先テーブルを作成します: +#### MySQL テーブルと同じスキーマを持つ出力先テーブルを作成します: {#create-a-destination-table-that-has-a-schema-equivalent-to-the-mysql-table} ```sql CREATE TABLE db.table ... @@ -115,9 +115,9 @@ CREATE TABLE db.table ... ClickHouse Cloud の宛先テーブルのスキーマと、元の MySQL テーブルのスキーマは整合している必要があります(カラム名と順序が同一であり、かつカラムのデータ型が互換性を持っている必要があります)。 ::: -### clickhouse-local を実行するホストマシン上で: +### clickhouse-local を実行するホストマシン上で: {#on-the-clickhouse-local-host-machine} -#### マイグレーション用クエリで clickhouse-local を実行します: +#### マイグレーション用クエリで clickhouse-local を実行します: {#run-clickhouse-local-with-the-migration-query} ```sql ./clickhouse-local --query " @@ -131,16 +131,16 @@ SELECT * FROM mysql('host:port', 'database', 'table', 'user', 'password');" ::: -## 例 2: JDBC ブリッジを使用して MySQL から ClickHouse Cloud へ移行する +## 例 2: JDBC ブリッジを使用して MySQL から ClickHouse Cloud へ移行する {#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge} [jdbc テーブル関数](/sql-reference/table-functions/jdbc.md) によってオンザフライで作成される [JDBC integration テーブルエンジン](/engines/table-engines/integrations/jdbc.md) を、[ClickHouse JDBC Bridge](https://github.com/ClickHouse/clickhouse-jdbc-bridge) および MySQL JDBC ドライバーと組み合わせて使用してソースの MySQL データベースからデータを読み取り、[remoteSecure テーブル関数](/sql-reference/table-functions/remote.md) を使用して ClickHouse Cloud サービス上の宛先テーブルにデータを書き込みます。 自己管理型 ClickHouse からの移行 -### 宛先の ClickHouse Cloud サービス側での作業: +### 宛先の ClickHouse Cloud サービス側での作業: {#on-the-destination-clickhouse-cloud-service-1} -#### 宛先データベースを作成する: +#### 宛先データベースを作成する: {#create-the-destination-database-1} ```sql CREATE DATABASE db diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md index 2c02d711cc8..15a23962893 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md @@ -10,15 +10,14 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import third_party_01 from '@site/static/images/integrations/migration/third-party-01.png'; +# サードパーティ製 ETL ツールの利用 {#using-a-3rd-party-etl-tool} -# サードパーティ製 ETL ツールの利用 - -自己管理型 ClickHouse の移行 +自己管理型 ClickHouse の移行 外部データソースから ClickHouse にデータを移動する優れた方法の 1 つは、数多く存在する一般的な ETL/ELT ツールのいずれかを利用することです。以下のツールについてはドキュメントを用意しています。 -- [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) -- [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) -- [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) +* [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) +* [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) +* [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) なお、ClickHouse と連携できる ETL/ELT ツールは他にも多数あります。ご利用中(あるいはお好み)のツールについては、そのツールのドキュメントを参照して詳細を確認してください。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md index b61bfea74cc..fffa8dedad7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md @@ -9,19 +9,18 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import object_storage_01 from '@site/static/images/integrations/migration/object-storage-01.png'; +# クラウドオブジェクトストレージから ClickHouse Cloud へのデータ移行 {#move-data-from-cloud-object-storage-to-clickhouse-cloud} -# クラウドオブジェクトストレージから ClickHouse Cloud へのデータ移行 - -Migrating Self-managed ClickHouse +Migrating Self-managed ClickHouse Cloud Object Storage をデータレイクとして利用していて、そのデータを ClickHouse Cloud にインポートしたい場合や、 現在利用しているデータベースシステムがデータを直接 Cloud Object Storage にオフロードできる場合には、 Cloud Object Storage に保存されたデータを ClickHouse Cloud のテーブルへ移行するために、以下のいずれかの テーブル関数を使用できます。 -- [s3](/sql-reference/table-functions/s3.md) または [s3Cluster](/sql-reference/table-functions/s3Cluster.md) -- [gcs](/sql-reference/table-functions/gcs) -- [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) +* [s3](/sql-reference/table-functions/s3.md) または [s3Cluster](/sql-reference/table-functions/s3Cluster.md) +* [gcs](/sql-reference/table-functions/gcs) +* [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) 現在利用しているデータベースシステムがデータを直接 Cloud Object Storage にオフロードできない場合は、 [サードパーティ製 ETL/ELT ツール](/cloud/migration/etl-tool-to-clickhouse) や [clickhouse-local](/cloud/migration/clickhouse-local) を使って、 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md index 88161598c3a..eced5a28c85 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md @@ -7,12 +7,12 @@ hide_title: true doc_type: 'guide' --- -import TableOfContentsBestPractices from '@site/docs/best-practices/_snippets/_table_of_contents.md'; -import TableOfContentsOptimizationAndPerformance from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; -import TableOfContentsSecurity from '@site/docs/cloud/_snippets/_security_table_of_contents.md'; +import TableOfContentsBestPractices from '@site/i18n/jp/docusaurus-plugin-content-docs/current/best-practices/_snippets/_table_of_contents.md'; +import TableOfContentsOptimizationAndPerformance from '@site/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContentsSecurity from '@site/i18n/jp/docusaurus-plugin-content-docs/current/cloud/_snippets/_security_table_of_contents.md'; -# リソースの概要 +# リソースの概要 {#resource-tour} この記事では、ClickHouse Cloud デプロイメントを最大限に活用するために、 ドキュメント内で利用できる各種リソースの概要を紹介します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/index.md index b91b0c318cf..4e03af7d173 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/onboard/index.md @@ -9,7 +9,7 @@ keywords: ['オンボーディング', 'はじめに', 'クラウド設定', ' -# ClickHouse Cloud を始める +# ClickHouse Cloud を始める {#get-started-with-clickhouse-cloud} ClickHouse Cloud を初めて使用する方で、どこから始めればよいかわからない場合、このドキュメントセクションでは、迅速に稼働させるために必要なすべての手順を説明します。ClickHouse Cloud を探索する過程で各ステップをガイドできるよう、この入門セクションを3つのサブセクションに分けて構成しています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md index 79a62dc9190..7732d5b0c73 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md @@ -12,7 +12,7 @@ doc_type: 'changelog' -#### 後方互換性のない変更 +#### 後方互換性のない変更 {#backward-incompatible-change} * ネストされた型内にある疑わしい/実験的な型を検証するようにしました。これまでは、Array/Tuple/Map のようなネストされた型内では(JSON を除き)そのような型を検証していませんでした。[#59385](https://github.com/ClickHouse/ClickHouse/pull/59385) ([Kruglov Pavel](https://github.com/Avogar))。 * ソート句 `ORDER BY ALL`(v23.12 で導入された)は `ORDER BY *` に置き換えられました。以前の構文は、`all` というカラムを持つテーブルでは誤りの原因になりやすいものでした。[#59450](https://github.com/ClickHouse/ClickHouse/pull/59450)([Robert Schulze](https://github.com/rschu1ze))。 @@ -29,7 +29,7 @@ doc_type: 'changelog' -#### 新機能 +#### 新機能 {#new-feature} * 値の件数とその誤差を返す Topk/topkweighed モードをサポート。 [#54508](https://github.com/ClickHouse/ClickHouse/pull/54508) ([UnamedRus](https://github.com/UnamedRus)). * ビュー/マテリアライズドビューで DEFINER ユーザーを指定できる新しい構文を追加しました。これにより、基盤となるテーブルに対する明示的な権限付与なしに、ビューからの SELECT/INSERT を実行できるようになりました。 [#54901](https://github.com/ClickHouse/ClickHouse/pull/54901) ([pufit](https://github.com/pufit)). @@ -57,7 +57,7 @@ doc_type: 'changelog' -#### パフォーマンスの向上 +#### パフォーマンスの向上 {#performance-improvement} * SELECT 句の GROUP BY キーに対する min/max/any/anyLast 集約関数を除去します。 [#52230](https://github.com/ClickHouse/ClickHouse/pull/52230) ([JackyWoo](https://github.com/JackyWoo)). * 複数の [nullable] カラムが関係する場合のシリアル化集約メソッドのパフォーマンスを改善しました。これは抽象化の整合性を損なうことのない、[#51399](https://github.com/ClickHouse/ClickHouse/issues/51399) の汎用バージョンです。[#55809](https://github.com/ClickHouse/ClickHouse/pull/55809)([Amos Bird](https://github.com/amosbird))。 @@ -86,7 +86,7 @@ doc_type: 'changelog' -#### 改善 +#### 改善 {#improvement} * マテリアライズドビューに対して `MODIFY COLUMN` クエリを実行する際は、内部テーブルの構造を確認し、すべてのカラムが存在していることを保証してください。 [#47427](https://github.com/ClickHouse/ClickHouse/pull/47427) ([sunny](https://github.com/sunny19930321)). * パーサーが扱うすべてのキーワードを含むテーブル `system.keywords` を追加しました。主に、ファジングおよびシンタックスハイライトを改善するために使用されます。 [#51808](https://github.com/ClickHouse/ClickHouse/pull/51808) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md index 4c5672a6c8b..c7922a7a5a9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# Cloud 向け v24.5 の変更履歴 +# Cloud 向け v24.5 の変更履歴 {#v245-changelog-for-cloud} v24.5 リリースに基づく ClickHouse Cloud サービスに関する変更点です。 @@ -128,7 +128,7 @@ v24.5 リリースに基づく ClickHouse Cloud サービスに関する変更 -# 改良点 +# 改良点 {#improvements} * `optimize_monotonous_functions_in_order_by` 設定を削除しました。この設定は実質的に何もしない(no-op)状態になりつつあります。 [#63004](https://github.com/ClickHouse/ClickHouse/pull/63004) (Raúl Marín). diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md index e38d4ec4f3a..404b7407e9b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# ClickHouse Cloud 向け v24.6 変更ログ +# ClickHouse Cloud 向け v24.6 変更ログ {#v246-changelog-for-cloud} v24.6 リリースに基づき、ClickHouse Cloud サービスに関係する変更点を記載します。 @@ -110,7 +110,7 @@ v24.6 リリースに基づき、ClickHouse Cloud サービスに関係する変 -## バグ修正(公式安定版リリースにおけるユーザー影響のある不具合) +## バグ修正(公式安定版リリースにおけるユーザー影響のある不具合) {#bug-fix-user-visible-misbehavior-in-an-official-stable-release} * IN および indexHint() 使用時に 'set' スキップインデックスが動作しない問題を修正。 #62083 (Michael Kolupaev)。 * テーブルが adaptive granularity を使用していない場合に、FINAL を使用したクエリが誤った結果を返す問題を修正。#62432 (Duc Canh Le)。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md index 9de9d1c8165..227364349d1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md @@ -22,7 +22,7 @@ v24.10 リリースに基づく ClickHouse Cloud サービスに関する主な -## 新機能 +## 新機能 {#new-feature} * リフレッシュ可能なマテリアライズドビューが本番利用向けに安定しました。 [#70550](https://github.com/ClickHouse/ClickHouse/pull/70550) ([Michael Kolupaev](https://github.com/al13n321))。リフレッシュ可能なマテリアライズドビューが Replicated データベースでもサポートされるようになりました。 [#60669](https://github.com/ClickHouse/ClickHouse/pull/60669) ([Michael Kolupaev](https://github.com/al13n321))。 * 関数 `toStartOfInterval()` に、TimescaleDB の `time_bucket()` 関数および PostgreSQL の `date_bin()` 関数の動作をエミュレートする新しいオーバーロードが追加されました([#55619](https://github.com/ClickHouse/ClickHouse/issues/55619))。これにより、日付またはタイムスタンプ値を、*固定* の起点(0000-01-01 00:00:00.000)ではなく、*任意の* 起点からの指定した間隔の整数倍に切り揃えることができます。例えば、`SELECT toStartOfInterval(toDateTime('2023-01-01 14:45:00'), INTERVAL 1 MINUTE, toDateTime('2023-01-01 14:35:30'));` は `2023-01-01 14:44:30` を返します。これは、起点 `2023-01-01 14:35:30` を基準として 1 分間隔の整数倍に切り揃えられた値です。[#56738](https://github.com/ClickHouse/ClickHouse/pull/56738)([Yarik Briukhovetskyi](https://github.com/yariks5s))。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md index dcf02be6892..a33dea964f0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md @@ -11,7 +11,7 @@ import Image from '@theme/IdealImage'; import Architecture from '@site/static/images/cloud/reference/architecture.png'; -# ClickHouse Cloud アーキテクチャ +# ClickHouse Cloud アーキテクチャ {#clickhouse-cloud-architecture} ClickHouse Cloud のアーキテクチャ diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md index f451e9af942..50ae42958c3 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md @@ -181,9 +181,9 @@ ClickHouse Cloud の課金は、コンピュート、ストレージ、[デー -## よくある質問 +## よくある質問 {#faqs} -### ClickHouse Credit (CHC) とは何ですか? +### ClickHouse Credit (CHC) とは何ですか? {#what-is-chc} ClickHouse Credit は、ClickHouse Cloud の利用に対するクレジットの単位であり、1 CHC は 1 米ドル (US$1) に相当します。これは、ClickHouse がその時点で公開している価格表に基づいて適用されます。 @@ -191,27 +191,27 @@ ClickHouse Credit は、ClickHouse Cloud の利用に対するクレジットの Stripe を通じて請求されている場合、Stripe の請求書上では 1 CHC は US$0.01 として表示されます。これは、当社の標準 SKU(1 CHC = US$1)について、Stripe 側の仕様上、端数数量を請求できないためであり、その制約の中で正確な請求を行うためのものです。 ::: -### 旧プラン(レガシー)の料金はどこで確認できますか? +### 旧プラン(レガシー)の料金はどこで確認できますか? {#find-legacy-pricing} 旧プラン(レガシー)の料金情報は[こちら](https://clickhouse.com/pricing?legacy=true)で確認できます。 -### コンピュート(計算リソース)はどのように計測されますか? +### コンピュート(計算リソース)はどのように計測されますか? {#how-is-compute-metered} ClickHouse Cloud は、コンピュートを 8G RAM 単位で 1 分ごとに計測します。 コンピュートコストはティア、リージョン、クラウドサービスプロバイダーによって異なります。 -### ディスク上のストレージはどのように算出されますか? +### ディスク上のストレージはどのように算出されますか? {#how-is-storage-on-disk-calculated} ClickHouse Cloud はクラウドオブジェクトストレージを使用しており、ClickHouse のテーブルに保存されているデータの圧縮後サイズに基づいて使用量を計測します。 ストレージコストはティア間で共通ですが、リージョンおよびクラウドサービスプロバイダーによって異なります。 -### バックアップはストレージの合計に含まれますか? +### バックアップはストレージの合計に含まれますか? {#do-backups-count-toward-total-storage} ストレージとバックアップはいずれもストレージコストの対象となり、個別に請求されます。 すべてのサービスは、デフォルトで 1 日間保持される 1 つのバックアップが有効になっています。 追加のバックアップが必要なユーザーは、Cloud コンソールの設定タブで追加の[バックアップ](/cloud/manage/backups/overview)を構成することで対応できます。 -### 圧縮率はどのように見積もればよいですか? +### 圧縮率はどのように見積もればよいですか? {#how-do-i-estimate-compression} 圧縮率はデータセットごとに大きく異なります。 どの程度変動するかは、そもそもデータがどれだけ圧縮しやすいか(高カーディナリティと低カーディナリティのフィールドの数など)や、 @@ -228,12 +228,12 @@ FROM system.tables WHERE name = <任意のテーブル名> ``` -### セルフマネージドで運用している場合、クラウドでサービスを実行する際のコストを見積もるために ClickHouse はどのようなツールを提供していますか? +### セルフマネージドで運用している場合、クラウドでサービスを実行する際のコストを見積もるために ClickHouse はどのようなツールを提供していますか? {#what-tools-does-clickhouse-offer-to-estimate-the-cost-of-running-a-service-in-the-cloud-if-i-have-a-self-managed-deployment} ClickHouse のクエリログは、ClickHouse Cloud でワークロードを実行するためのコストを見積もる際に利用できる[主要なメトリクス](/operations/system-tables/query_log)を記録します。 セルフマネージド環境から ClickHouse Cloud への移行の詳細については[移行ドキュメント](/cloud/migration/clickhouse-to-cloud)を参照し、さらに質問がある場合は [ClickHouse Cloud support](https://console.clickhouse.cloud/support) までお問い合わせください。 -### ClickHouse Cloud にはどのような課金オプションがありますか? +### ClickHouse Cloud にはどのような課金オプションがありますか? {#what-billing-options-are-available-for-clickhouse-cloud} ClickHouse Cloud は次の課金オプションをサポートしています: @@ -245,11 +245,11 @@ ClickHouse Cloud は次の課金オプションをサポートしています: PAYG 向けの ClickHouse Cloud クレジットは 0.01 ドル単位で請求されるため、利用状況に応じてクレジットの端数も含めて課金できます。これは、事前購入するコミット型の ClickHouse クレジット(1 ドル単位の整数額で購入)とは異なります。 ::: -### クレジットカードを削除できますか? +### クレジットカードを削除できますか? {#can-i-delete-my-credit-card} Billing UI からクレジットカードを削除することはできませんが、いつでも更新することはできます。これにより、常に有効な支払い方法が組織に設定されていることを保証します。クレジットカードを削除する必要がある場合は、[ClickHouse Cloud support](https://console.clickhouse.cloud/support) までお問い合わせください。 -### 課金サイクルはどのくらいの期間ですか? +### 課金サイクルはどのくらいの期間ですか? {#how-long-is-the-billing-cycle} 課金は月次サイクルに従い、開始日は ClickHouse Cloud の組織が作成された日となります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md deleted file mode 100644 index ee416883e96..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -slug: /cloud/billing/marketplace/aws-marketplace-payg -title: 'AWS Marketplace の従量課金 (PAYG)' -description: 'AWS Marketplace(PAYG 従量課金)経由で ClickHouse Cloud に登録します。' -keywords: ['aws', 'marketplace', 'billing', 'PAYG'] -doc_type: 'guide' ---- - -import aws_marketplace_payg_1 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-1.png'; -import aws_marketplace_payg_2 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-2.png'; -import aws_marketplace_payg_3 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-3.png'; -import aws_marketplace_payg_4 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-4.png'; -import aws_marketplace_payg_5 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-5.png'; -import aws_marketplace_payg_6 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-6.png'; -import aws_marketplace_payg_7 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-7.png'; -import aws_marketplace_payg_8 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-8.png'; -import aws_marketplace_payg_9 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-9.png'; -import Image from '@theme/IdealImage'; - -PAYG(従量課金制)のパブリックオファーを利用して、[AWS Marketplace](https://aws.amazon.com/marketplace) から ClickHouse Cloud の利用を開始しましょう。 - - -## 前提条件 {#prerequisites} - -- 課金管理者によって購入権限が付与されている AWS アカウント。 -- 購入するには、そのアカウントで AWS Marketplace にログインしている必要があります。 -- サブスクリプションに ClickHouse の組織を接続するには、その組織の管理者である必要があります。 - -:::note -1 つの AWS アカウントは、「ClickHouse Cloud - Pay As You Go」サブスクリプション 1 件のみに登録でき、そのサブスクリプションは 1 つの ClickHouse 組織にのみリンクできます。 -::: - - - -## サインアップ手順 {#steps-to-sign-up} - - - -### ClickHouse Cloud - Pay As You Go を検索する {#search-payg} - -[AWS Marketplace](https://aws.amazon.com/marketplace) にアクセスし、「ClickHouse Cloud - Pay As You Go」を検索します。 - -ClickHouse を検索している AWS Marketplace - -### 購入オプションを表示する {#purchase-options} - -[リスティング](https://aws.amazon.com/marketplace/pp/prodview-p4gwofrqpkltu) をクリックし、続いて **View purchase options** をクリックします。 - -AWS Marketplace の購入オプション表示画面 - -### 購読する {#subscribe} - -次の画面で、**Subscribe** をクリックします。 - -:::note -**Purchase order (PO) number** は任意入力のため、省略してかまいません。 -::: - -AWS Marketplace の購読画面 - -### アカウントをセットアップする {#set-up-your-account} - -この時点ではセットアップは完了しておらず、まだ ClickHouse Cloud 組織の請求は AWS Marketplace 経由になっていないことに注意してください。Marketplace のサブスクリプション画面で **Set up your account** をクリックし、ClickHouse Cloud にリダイレクトしてセットアップを完了させる必要があります。 - -アカウントのセットアップ - -ClickHouse Cloud にリダイレクトされたら、既存アカウントでログインするか、新しいアカウントを登録できます。ClickHouse Cloud 組織を AWS Marketplace の課金に紐付けるために、このステップは非常に重要です。 - -:::note[新規 ClickHouse Cloud ユーザー] -ClickHouse Cloud を初めて利用する場合は、以下の手順に従ってください。 -::: - -
-新規ユーザー向けの手順 - -ClickHouse Cloud を初めて利用する場合は、ページ下部の **Register** をクリックします。新規ユーザーの作成とメールアドレスの確認を求められます。メールアドレスを確認したら、ClickHouse Cloud のログインページは閉じてかまいません。https://console.clickhouse.cloud にアクセスし、新しく作成したユーザー名でログインします。 - -ClickHouse Cloud のサインアップ - -:::note[新規ユーザー] -ビジネスに関する基本的な情報も入力する必要があります。以下のスクリーンショットを参照してください。 -::: - -開始前の入力画面 - -開始前の入力画面(続き) - -
- -すでに ClickHouse Cloud を利用している場合は、既存の認証情報を使用してログインするだけでかまいません。 - -### Marketplace サブスクリプションを Organization に追加する {#add-marketplace-subscription} - -正常にログインしたら、この Marketplace サブスクリプションで課金する新しい Organization を作成するか、このサブスクリプションに紐付けて課金する既存の Organization を選択します。 - -Marketplace サブスクリプションの追加 - -このステップを完了すると、組織はこの AWS サブスクリプションに接続され、すべての利用料金は AWS アカウント経由で請求されます。 - -ClickHouse UI の組織の請求ページから、請求が AWS Marketplace にリンクされていることを確認できます。 - -請求ページの確認 - -
- - - -## サポート {#support} - -問題が発生した場合は、[サポートチーム](https://clickhouse.com/support/program)までお気軽にお問い合わせください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx index 66d69f31285..5fd00ebc2c1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['課金', 'ネットワーク転送量', 'データ送信量', 'コスト', '料金'] --- -import NetworkPricing from '@site/docs/cloud/reference/_snippets/_network_transfer_rates.md'; +import NetworkPricing from '@site/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/_snippets/_network_transfer_rates.md'; ClickHouse Cloud は、転送されるデータのイングレスとエグレスを計測します。 これには、ClickHouse Cloud への入出力データだけでなく、同一リージョン内および diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md index 34e6ddd0bbf..1bc939d3e74 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md @@ -7,9 +7,9 @@ keywords: ['請求', '支払いしきい値', '自動請求書発行', '請求 doc_type: 'guide' --- -# 支払いしきい値 +# 支払いしきい値 {#payment-thresholds} -請求期間中に ClickHouse Cloud のご利用金額が 10,000 米ドル(または同等額)に達すると、ご登録のお支払い方法から自動的に決済されます。決済に失敗した場合、猶予期間の後にサービスが一時停止または終了されます。 +請求期間中に ClickHouse Cloud のご利用金額が 10,000 米ドル(または同等額)に達すると、ご登録のお支払い方法から自動的に決済されます。決済に失敗した場合、猶予期間の後にサービスが一時停止または終了されます。 :::note この支払いしきい値は、一定額の利用を約束する契約や、その他 ClickHouse と個別に合意した契約を締結しているお客様には適用されません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md index dc1b7de79b7..949d67078f1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md @@ -11,7 +11,7 @@ import billing_compliance from '@site/static/images/cloud/manage/billing_complia import Image from '@theme/IdealImage'; -# ClickHouse Cloud の課金コンプライアンス +# ClickHouse Cloud の課金コンプライアンス {#clickhouse-cloud-billing-compliance} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md index 97fead5c6b5..ad4739223e1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md @@ -10,7 +10,7 @@ doc_type: 'reference' import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge' -# サポート対象のクラウドリージョン +# サポート対象のクラウドリージョン {#supported-cloud-regions} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md index f39d46c5d76..ec51c52efd9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md @@ -10,8 +10,7 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-settings-sidebar.png'; - -# 設定の構成方法 +# 設定の構成方法 {#configuring-settings} 特定の[ユーザー](/operations/access-rights#user-account-management)または[ロール](/operations/access-rights#role-management)向けに ClickHouse Cloud サービスの設定を指定するには、[SQL ベースの Settings Profile](/operations/access-rights#settings-profiles-management)を使用する必要があります。Settings Profile を適用することで、サービスが停止したりアイドル状態になったりアップグレードされた場合でも、構成した設定が保持されます。Settings Profile の詳細については[こちらのページ](/operations/settings/settings-profiles.md)を参照してください。 @@ -19,4 +18,4 @@ import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-setti ClickHouse Cloud サービスに対して指定できる設定の詳細については、[ドキュメント](/operations/settings)内のカテゴリ別の全設定一覧を参照してください。 -Cloud 設定サイドバー \ No newline at end of file +Cloud 設定サイドバー \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md index 7cfb7ca656d..4761d1b347a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md @@ -10,7 +10,7 @@ import BetaBadge from '@theme/badges/BetaBadge'; import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge'; -# セキュリティおよびコンプライアンスレポート +# セキュリティおよびコンプライアンスレポート {#security-and-compliance-reports} ClickHouse はお客様のセキュリティおよびコンプライアンスに関するニーズを評価し、追加のレポートに対するご要望に応じてプログラムの拡充を継続しています。詳細やレポートのダウンロードは、[Trust Center](https://trust.clickhouse.com) をご覧ください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md index 0f8964f21da..b5dc8d9ce75 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md @@ -37,7 +37,7 @@ doc_type: 'guide' -## アカウントの閉鎖をリクエストする +## アカウントの閉鎖をリクエストする {#request-account-closure} 閉鎖および削除のリクエストについては、認証が必要です。リクエストを迅速に処理できるよう、以下の手順に従ってください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/index.md index c15a604a50e..88bf9b736af 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/cloud/reference/index.md @@ -7,7 +7,7 @@ description: 'Cloud リファレンスセクション用ランディングペー doc_type: 'landing-page' --- -# Cloud リファレンス +# Cloud リファレンス {#cloud-reference} このセクションは ClickHouse Cloud のより技術的な詳細に関するリファレンスガイドで、次のページで構成されています: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/glossary.md b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/glossary.md index 32138525a70..25d7f65ecc2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/glossary.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/glossary.md @@ -10,7 +10,7 @@ doc_type: 'reference' {/* no-glossary */ } -# 用語集 +# 用語集 {#glossary} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/olap.md b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/olap.md index 627c3f4582b..6e4289f6f26 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/olap.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/olap.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# OLAP とは何ですか? +# OLAP とは何ですか? {#what-is-olap} [OLAP](https://en.wikipedia.org/wiki/Online_analytical_processing) は Online Analytical Processing(オンライン分析処理)の略です。これは広い意味を持つ用語で、技術的観点とビジネス的観点の 2 つの視点から捉えることができます。最も抽象的なレベルでは、これらの単語を逆から読むとイメージしやすくなります: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx index 0a1d4d292d8..8f376dd892a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx @@ -18,7 +18,7 @@ doc_type: 'guide' ## ストレージ層:同時に行われる INSERT は互いに独立している \{#storage-layer-concurrent-inserts-are-isolated-from-each-other\} - + + + + + + + \ No newline at end of file + - + -## Projections はどのように動作しますか? {#how-do-projections-work} +## Projection はどのように動作しますか? {#how-do-projections-work} -実務的には、Projection は元のテーブルに対する追加の「非表示テーブル」と考えることができます。Projection は元のテーブルとは異なる行順序を持つことができ、その結果として異なるプライマリインデックスを持つことができ、さらに集約値を自動的かつ段階的に事前計算できます。その結果、Projections を使用すると、クエリ実行を高速化するための 2 つの「チューニング手段」を得られます: +実際には、Projection は元のテーブルに付随する追加の「不可視なテーブル」のようなものと考えることができます。Projection は元のテーブルとは異なる行の順序を持つことができ、その結果として異なるプライマリインデックスを持つことができ、さらに集約値を自動的かつインクリメンタルに事前計算できます。その結果、Projection を使用すると、クエリ実行を高速化するための 2 つの「チューニング手段」を得られます: -- **プライマリインデックスを適切に利用する** -- **集約を事前計算する** +- **プライマリインデックスを適切に活用すること** +- **集約を事前計算すること** -Projections は、複数の行順序を持つテーブルを用意でき、挿入時に集約を事前計算できるという点で [Materialized Views](/materialized-views) と似ています。 -Projections は自動的に更新され、元のテーブルと同期された状態に保たれますが、Materialized Views は明示的に更新する必要があります。クエリが元のテーブルを対象とする場合、 -ClickHouse はプライマリキーを自動的にサンプリングし、同じ正しい結果を生成でき、かつ読み取るデータ量が最も少なくて済むテーブルを選択します。これは次の図に示すとおりです: +Projection は、複数の行順序を持ち、挿入時に集約を事前計算できるという点で [Materialized Views](/materialized-views) +(マテリアライズドビュー)と似ています。 +Projection は自動的に更新され、元のテーブルと同期が維持されますが、マテリアライズドビューは明示的に更新する必要があります。クエリが元のテーブルを対象とする場合、 +ClickHouse はプライマリキーを自動的にサンプリングし、同じ正しい結果を生成でき、かつ読み取る必要があるデータ量が最も少ないテーブルを、以下の図のように選択します: -ClickHouse における Projections +ClickHouse における Projection -### `_part_offset` を使ったよりスマートなストレージ {#smarter_storage_with_part_offset} +### `_part_offset` を用いたよりスマートなストレージ {#smarter_storage_with_part_offset} -バージョン 25.5 以降、ClickHouse は Projection 内で仮想カラム `_part_offset` をサポートしており、Projection を定義する新しい方法を提供します。 +バージョン 25.5 以降、ClickHouse はプロジェクション内で仮想カラム `_part_offset` を +サポートしており、プロジェクションの新しい定義方法を提供します。 -現在、Projection を定義する方法は 2 つあります: +現在、プロジェクションを定義する方法は 2 通りあります。 -- **全カラムを保存する(従来の動作)**: Projection には完全なデータが含まれ、直接読み取ることができます。そのため、フィルタが Projection のソート順と一致する場合に高速なパフォーマンスを提供します。 +- **全カラムを保存する(従来の動作)**: プロジェクションには完全なデータが含まれ、 + プロジェクションのソート順とフィルタが一致する場合は、プロジェクションから直接読み取ることで + 高速に処理できます。 -- **ソートキーと `_part_offset` のみを保存する**: Projection はインデックスのように動作します。 - ClickHouse は Projection のプライマリインデックスを使用して一致する行を特定しますが、実際のデータはベーステーブルから読み取ります。これにより、クエリ時の I/O がわずかに増える代わりに、ストレージのオーバーヘッドを削減できます。 +- **ソートキー + `_part_offset` のみを保存する**: プロジェクションはインデックスのように動作します。 + ClickHouse はプロジェクションのプライマリインデックスを使って一致する行を特定しますが、 + 実際のデータはベーステーブルから読み取ります。これにより、クエリ時の I/O がやや増える + 代わりにストレージのオーバーヘッドを削減できます。 -上記のアプローチは組み合わせて使用することもでき、一部のカラムを Projection 内に保存し、その他を `_part_offset` を介して間接的に保存できます。 +上記のアプローチは組み合わせることもでき、プロジェクションに一部のカラムを直接保存し、 +その他のカラムを `_part_offset` を介して間接的に保存できます。 +## プロジェクションを使用するタイミング {#when-to-use-projections} +プロジェクションは、新しいユーザーにとって魅力的な機能です。というのも、データ挿入時に自動的に +維持されるためです。さらに、クエリは 1 つのテーブルに対して送信するだけでよく、可能な場合には +プロジェクションが利用されて応答時間が短縮されます。 -## プロジェクションをいつ使用すべきか {#when-to-use-projections} - -プロジェクションは、新規ユーザーにとって魅力的な機能です。データの挿入に応じて自動的に -メンテナンスされるためです。さらに、クエリは単一のテーブルに対して送るだけでよく、可能な場合には -プロジェクションが活用されて応答時間を短縮できます。 - -これはマテリアライズドビューとは対照的です。マテリアライズドビューでは、ユーザーがフィルタ条件に +これはマテリアライズドビューとは対照的であり、マテリアライズドビューでは、ユーザーはフィルタに 応じて適切に最適化されたターゲットテーブルを選択するか、クエリを書き換える必要があります。 -そのためユーザーアプリケーション側への要求が大きくなり、クライアント側の複雑さが増加します。 +これにより、ユーザーアプリケーションへの依存度が高まり、クライアント側の複雑さが増します。 -これらの利点にもかかわらず、プロジェクションには本質的な制約がいくつか存在するため、ユーザーはそれらを -理解したうえで、慎重に使用する必要があります。 +これらの利点にもかかわらず、プロジェクションには本質的な制約がいくつか存在するため、 +ユーザーはその点を理解したうえで、必要最小限の利用にとどめるべきです。 -- プロジェクションでは、ソーステーブルと(非表示の)ターゲットテーブルで異なる TTL を使用できませんが、 - マテリアライズドビューでは異なる TTL を使用できます。 +- プロジェクションでは、ソーステーブルと(非表示の)ターゲットテーブルで異なる TTL を + 使用することはできませんが、マテリアライズドビューでは異なる TTL を使用できます。 - プロジェクションを持つテーブルでは、軽量な更新および削除はサポートされていません。 -- マテリアライズドビューは連鎖させることができます。1 つのマテリアライズドビューのターゲットテーブルを、 - 別のマテリアライズドビューのソーステーブルとして利用することができます。しかし、プロジェクションでは - これはできません。 -- プロジェクションは JOIN をサポートしませんが、マテリアライズドビューはサポートします。 -- プロジェクションはフィルタ(`WHERE` 句)をサポートしませんが、マテリアライズドビューはサポートします。 - -次のような場合にはプロジェクションの使用を推奨します。 - -- データの完全な並べ替えが必要な場合。理論上、プロジェクション内の式で `GROUP BY` を使用することも - できますが、集計の維持にはマテリアライズドビューの方が効果的です。クエリオプティマイザも、 - `SELECT * ORDER BY x` のような単純な並べ替えを行うプロジェクションをより積極的に活用する傾向があります。 - この式では、ストレージ使用量を削減するために、一部の列のみを選択することもできます。 -- ストレージ使用量の増加およびデータを 2 回書き込むことによるオーバーヘッドが発生する可能性を +- マテリアライズドビューはチェーン化できます。1 つのマテリアライズドビューのターゲットテーブルを、 + 別のマテリアライズドビューのソーステーブルとして利用することができます。このようなことは + プロジェクションでは不可能です。 +- プロジェクション定義では JOIN はサポートされませんが、マテリアライズドビューではサポートされます。 + ただし、プロジェクションを持つテーブルに対するクエリでは、自由に JOIN を使用できます。 +- プロジェクション定義ではフィルタ(`WHERE` 句)はサポートされませんが、マテリアライズドビューでは + サポートされます。とはいえ、プロジェクションを持つテーブルに対するクエリでは、自由にフィルタできます。 + +プロジェクションの使用を推奨するのは、次のような場合です。 + +- データの完全な並べ替えが必要な場合。理論上は、プロジェクション内の式で `GROUP BY` を + 使用することも可能ですが、集計の維持にはマテリアライズドビューの方が効果的です。 + クエリオプティマイザも、単純な並べ替え、すなわち `SELECT * ORDER BY x` を使用する + プロジェクションの方をより活用しやすくなります。 + ユーザーは、この式内で列のサブセットを選択することで、ストレージフットプリントを削減できます。 +- ストレージフットプリントの増加や、データを書き込む処理が 2 回になることに伴うオーバーヘッドを 許容できる場合。挿入速度への影響をテストし、 [ストレージオーバーヘッドを評価](/data-compression/compression-in-clickhouse)してください。 +## 例 {#examples} - -## 例 - -### プライマリキーに含まれていない列でのフィルタリング +### プライマリキーに含まれないカラムでのフィルタリング {#filtering-without-using-primary-keys} この例では、テーブルに Projection を追加する方法を説明します。 -また、テーブルのプライマリキーに含まれていない列を条件とするクエリを高速化するために、 -Projection をどのように利用できるかも見ていきます。 +また、この Projection を使用して、テーブルのプライマリキーに含まれない +カラムでフィルタリングを行うクエリを高速化する方法も見ていきます。 -この例では、`pickup_datetime` でソートされている、 +この例では、`pickup_datetime` で並べ替えられている、 [sql.clickhouse.com](https://sql.clickhouse.com/) で利用可能な New York Taxi Data -というデータセットを使用します。 +データセットを使用します。 -乗客がドライバーに 200 ドルを超えるチップを支払ったすべてのトリップ ID を検索する、 -単純なクエリを書いてみましょう。 +乗客が運転手に 200 ドルを超える額のチップを支払った +すべてのトリップ ID を検索する、簡単なクエリを書いてみましょう。 ```sql runnable SELECT @@ -115,16 +118,16 @@ FROM nyc_taxi.trips WHERE tip_amount > 200 AND trip_duration_min > 0 ORDER BY tip_amount, trip_id ASC ``` -`ORDER BY` に含まれていない `tip_amount` でフィルタリングしているため、ClickHouse はテーブル全体をスキャンする必要がありました。クエリを高速化しましょう。 +`ORDER BY` に含まれていない `tip_amount` でフィルタリングしているため、ClickHouse はテーブル全体をスキャンする必要がありました。このクエリを高速化していきましょう。 -元のテーブルとその結果を保持するために、新しいテーブルを作成し、`INSERT INTO SELECT` を使ってデータをコピーします。 +元のテーブルと結果を保持するために、新しいテーブルを作成し、`INSERT INTO SELECT` を使ってデータをコピーします。 ```sql CREATE TABLE nyc_taxi.trips_with_projection AS nyc_taxi.trips; INSERT INTO nyc_taxi.trips_with_projection SELECT * FROM nyc_taxi.trips; ``` -プロジェクションを追加するには、`ALTER TABLE` ステートメントと `ADD PROJECTION` ステートメントを組み合わせて使用します。 +プロジェクションを追加するには、`ALTER TABLE` 文と `ADD PROJECTION` 文を組み合わせて使用します。 ```sql ALTER TABLE nyc_taxi.trips_with_projection @@ -135,7 +138,7 @@ ADD PROJECTION prj_tip_amount ) ``` -プロジェクションを追加した後は、その中のデータを上記で指定したクエリに従って物理的に並び替えて書き換えるために、`MATERIALIZE PROJECTION` ステートメントを使用する必要があります。 +プロジェクションを追加した後は、上記で指定したクエリに従って、その中のデータが物理的に並べ替えおよび再書き込みされるように、`MATERIALIZE PROJECTION` ステートメントを実行する必要があります。 ```sql ALTER TABLE nyc.trips_with_projection MATERIALIZE PROJECTION prj_tip_amount @@ -152,10 +155,10 @@ FROM nyc_taxi.trips_with_projection WHERE tip_amount > 200 AND trip_duration_min ORDER BY tip_amount, trip_id ASC ``` -クエリの実行時間を大幅に短縮できており、スキャンする行数も少なくなっていることに注目してください。 +クエリ時間を大幅に短縮でき、スキャンが必要な行数も少なくなっていることに注目してください。 -上記のクエリが実際に作成したプロジェクションを使用していることは、 -`system.query_log` テーブルをクエリすることで確認できます。 +上記のクエリが実際に作成したプロジェクションを使用していたことは、 +`system.query_log` テーブルを参照することで確認できます。 ```sql SELECT query, projections @@ -173,17 +176,14 @@ WHERE query_id='' └───────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────┘ ``` -### プロジェクションを使用した UK Price Paid クエリの高速化 -プロジェクションを使用してクエリパフォーマンスを高速化できることを示すために、実際のデータセットを用いた例を見ていきます。この例では、チュートリアル [UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid) で使用している 3,003 万行のテーブルを使用します。このデータセットは -[sql.clickhouse.com](https://sql.clickhouse.com/?query_id=6IDMHK3OMR1C97J6M9EUQS) -環境内でも利用可能です。 +### プロジェクションを使用してUK不動産価格データのクエリを高速化する {#using-projections-to-speed-up-UK-price-paid} + +プロジェクションを使用してクエリパフォーマンスを高速化する方法を実証するため、実際のデータセットを使用した例を見ていきます。この例では、3,003万行を含む[UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid)チュートリアルのテーブルを使用します。このデータセットは[sql.clickhouse.com](https://sql.clickhouse.com/?query_id=6IDMHK3OMR1C97J6M9EUQS)環境でも利用可能です。 -テーブルの作成方法とデータの挿入方法を確認したい場合は、 -[「The UK property prices dataset」](/getting-started/example-datasets/uk-price-paid) -ページを参照してください。 +テーブルの作成方法とデータの挿入方法を確認する場合は、["英国不動産価格データセット"](/getting-started/example-datasets/uk-price-paid)のページを参照してください。 -このデータセットに対して 2 つの簡単なクエリを実行してみます。1 つ目はロンドンにある郡を、支払われた価格が高い順に一覧表示するもので、2 つ目は各郡の平均価格を計算するものです。 +このデータセットに対して2つの簡単なクエリを実行できます。1つ目はロンドンで最も高い支払価格を記録した郡を一覧表示し、2つ目は郡ごとの平均価格を計算します。 ```sql runnable SELECT @@ -195,7 +195,6 @@ ORDER BY price DESC LIMIT 3 ``` - ```sql runnable SELECT county, @@ -206,7 +205,7 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -クエリ自体は非常に高速ですが、どちらのクエリでも 3,003 万行すべてに対してフルテーブルスキャンが発生している点に注目してください。これは、テーブル作成時の `ORDER BY` 句に `town` と `price` のどちらも含めなかったためです。 +両方のクエリで全3,003万行のフルテーブルスキャンが発生したことに注意してください。非常に高速ではありますが、これはテーブル作成時の ORDER BY 句に`town`も`price`も含まれていなかったためです: ```sql CREATE TABLE uk.uk_price_paid @@ -218,16 +217,16 @@ ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2); ``` -プロジェクションを使ってこのクエリを高速化できるか確認してみましょう。 +プロジェクションを使用してこのクエリを高速化できるか見てみましょう。 -元のテーブルと結果を保持するために、新しいテーブルを作成し、`INSERT INTO SELECT` を使ってデータをコピーします。 +元のテーブルと結果を保持するために、新しいテーブルを作成し、`INSERT INTO SELECT`を使用してデータをコピーします: ```sql CREATE TABLE uk.uk_price_paid_with_projections AS uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections SELECT * FROM uk.uk_price_paid; ``` -ここでは、町と価格で並べ替えられたプライマリインデックスを持つ追加の(非表示の)テーブルを生成するプロジェクション `prj_oby_town_price` を作成し、データを投入します。これは、特定の町における最高支払額に対する郡の一覧を取得するクエリを最適化するためのものです。 +プロジェクション `prj_oby_town_price` を作成してデータを投入します。これにより、町と価格で順序付けされたプライマリインデックスを持つ追加の(非表示)テーブルが生成され、特定の町で最高価格が支払われた郡を一覧表示するクエリが最適化されます: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -246,11 +245,9 @@ ALTER TABLE uk.uk_price_paid_with_projections SETTINGS mutations_sync = 1 ``` -[`mutations_sync`](/operations/settings/settings#mutations_sync) 設定は、 -同期実行を強制するために使用します。 +[`mutations_sync`](/operations/settings/settings#mutations_sync)設定を使用して、同期実行を強制します。 -投影 `prj_gby_county` を作成してデータを投入します。これは追加の(非表示の)テーブルで、 -既存の英国 130 郡すべてについて avg(price) 集約値を増分的に前計算します。 +プロジェクション `prj_gby_county` を作成して投入します。これは追加の(非表示の)テーブルであり、既存の英国130郡すべてについて avg(price) 集計値を段階的に事前計算します: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -270,15 +267,14 @@ SETTINGS mutations_sync = 1 ``` :::note -上記の `prj_gby_county` プロジェクションのように、プロジェクション内で `GROUP BY` 句が使用されている場合、(非表示の)テーブルの基盤となるストレージエンジンは `AggregatingMergeTree` になり、すべての集約関数は `AggregateFunction` に変換されます。これにより、増分集約が適切に行われます。 +上記の `prj_gby_county` プロジェクションのように、プロジェクション内で `GROUP BY` 句が使用されている場合、(隠された)テーブルの基盤となるストレージエンジンは `AggregatingMergeTree` になり、すべての集約関数は `AggregateFunction` に変換されます。これにより、適切な増分データ集約が保証されます。 ::: -以下の図はメインテーブル `uk_price_paid_with_projections` -と、その 2 つのプロジェクションの可視化です。 +以下の図は、メインテーブル `uk_price_paid_with_projections` とその2つのプロジェクションの視覚化です: -メインテーブル uk_price_paid_with_projections とその 2 つのプロジェクションの可視化 +メインテーブル uk_price_paid_with_projections と、その 2 つのプロジェクションの可視化 -ロンドンにおける支払価格が最も高い上位 3 件の郡を列挙するクエリを再度実行すると、クエリパフォーマンスが向上していることが分かります。 +ロンドンにおける上位3件の高額取引価格の郡を一覧表示するクエリを再実行すると、クエリパフォーマンスが向上していることが確認できます: ```sql runnable SELECT @@ -290,7 +286,7 @@ ORDER BY price DESC LIMIT 3 ``` -同様に、平均支払価格が最も高い英国の郡を上位 3 件取得するクエリは次のとおりです。 +同様に、平均支払価格が最も高い上位3つの英国カウンティをリストするクエリの場合: ```sql runnable SELECT @@ -302,19 +298,13 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -両方のクエリが元のテーブルを対象としており、2 つのプロジェクションを作成する前は、 -いずれのクエリもフルテーブルスキャン(ディスクから 3,003 万行すべてがストリーミングされた) -になっていることに注意してください。 +両方のクエリが元のテーブルを対象としており、2つのプロジェクションを作成する前は、両方のクエリでフルテーブルスキャン(全3,003万行がディスクから読み込まれる)が発生していたことに注意してください。 -また、支払価格が最も高い上位 3 件についてロンドンのカウンティを列挙するクエリでは、 -217 万行がストリーミングされていることにも注意してください。このクエリ用に最適化された -2 つ目のテーブルを直接使用した場合、ディスクからストリーミングされたのは 8.192 万行だけでした。 +また、ロンドンの郡を支払価格が最も高い上位 3 件について列挙するクエリでは、2.17 百万行がストリーミングされている点にも注意してください。このクエリ向けに最適化された 2 つ目のテーブルを直接使用した場合、ディスクから読み出されたのは 8.192 万行だけでした。 -この違いが生じる理由は、前述の `optimize_read_in_order` 最適化が、現時点では -プロジェクションではサポートされていないためです。 +この差が生じる理由は、上で述べた `optimize_read_in_order` 最適化が、現時点ではプロジェクションではサポートされていないためです。 -`system.query_log` テーブルを調査して、上記 2 つのクエリに対して ClickHouse が -2 つのプロジェクションを自動的に使用していることを確認します(以下の projections 列を参照): +`system.query_log` テーブルを確認すると、上記 2 つのクエリに対して ClickHouse が自動的に 2 つのプロジェクションを使用していることが分かります(下の projections 列を参照): ```sql SELECT @@ -330,7 +320,6 @@ ORDER BY initial_query_start_time DESC FORMAT Vertical ``` - ```response 行 1: ────── @@ -361,23 +350,25 @@ query_duration: 11 ms read_rows: 2.29 million projections: ['uk.uk_price_paid_with_projections.prj_obj_town_price'] -2行のセット。経過時間: 0.006秒。 +2行のセット。経過時間: 0.006秒 ``` -### さらに例を見てみましょう -次の例では、同じUKの価格データセットを使用し、プロジェクションあり/なしのクエリを比較します。 +### さらに例を示します {#further-examples} -オリジナルのテーブル(およびそのパフォーマンス)を保持するため、再度 `CREATE AS` と `INSERT INTO SELECT` を使用してテーブルのコピーを作成します。 +次の例では、同じ英国の価格データセットを使用し、プロジェクションを使用するクエリと使用しないクエリを比較します。 + +元のテーブル(とそのパフォーマンス)を維持するため、ここでも `CREATE AS` と `INSERT INTO SELECT` を使ってテーブルのコピーを作成します。 ```sql CREATE TABLE uk.uk_price_paid_with_projections_v2 AS uk.uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections_v2 SELECT * FROM uk.uk_price_paid; ``` -#### プロジェクションを作成する -`toYear(date)`、`district`、`town` をディメンションとする集約プロジェクションを作成してみましょう。 +#### プロジェクションを作成する {#build-projection} + +`toYear(date)`、`district`、`town` をディメンションとする集約プロジェクションを作成します: ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -397,7 +388,7 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 ) ``` -既存データに対してプロジェクションを適用します。(マテリアライズしない場合、プロジェクションは新規に挿入されるデータに対してのみ作成されます): +既存データに対してプロジェクションをマテリアライズします。(マテリアライズしない場合、プロジェクションは新たに挿入されるデータに対してのみ作成されます): ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -405,9 +396,10 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 SETTINGS mutations_sync = 1 ``` -以下のクエリでは、プロジェクションあり/なしの場合のパフォーマンスを比較します。プロジェクションを無効にするには、デフォルトで有効になっている設定 [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections) を使用します。 +次のクエリでは、プロジェクションあり/なしの場合のパフォーマンスを比較します。プロジェクションの使用を無効にするには、デフォルトで有効になっている設定 [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections) を変更します。 + -#### クエリ 1. 年ごとの平均価格 +#### クエリ 1. 年ごとの平均価格 {#average-price-projections} ```sql runnable SELECT @@ -431,9 +423,10 @@ ORDER BY year ASC ``` -結果は同じですが、後者の例の方がパフォーマンスに優れています。 +結果は同じになるはずですが、後者の例のほうがパフォーマンスは良くなります。 -#### クエリ 2. ロンドンの年別平均価格 + +#### クエリ 2. ロンドンにおける年ごとの平均価格 {#average-price-london-projections} ```sql runnable SELECT @@ -458,9 +451,10 @@ GROUP BY year ORDER BY year ASC ``` -#### クエリ 3. 最も高価な地区 -条件 `(date >= '2020-01-01')` は、投影ディメンション(`toYear(date) >= 2020)`)に一致するように変更する必要があります。 +#### クエリ 3. 最も高価な地域 {#most-expensive-neighborhoods-projections} + +条件 (date >= '2020-01-01') を、プロジェクションのディメンション (`toYear(date) >= 2020)` と一致するように変更します。 ```sql runnable SELECT @@ -480,7 +474,6 @@ LIMIT 100 SETTINGS optimize_use_projections=0 ``` - ```sql runnable SELECT town, @@ -498,22 +491,26 @@ ORDER BY price DESC LIMIT 100 ``` -同じ結果になりますが、2 番目のクエリではクエリ性能が向上している点に注目してください。 +今回も結果は同じですが、2 番目のクエリではクエリ性能が向上している点に注目してください。 -### 1 つのクエリでプロジェクションを組み合わせる -バージョン 25.6 からは、前のバージョンで導入された `_part_offset` サポートを基盤として、 -ClickHouse は複数のプロジェクションを用いて、複数のフィルタ条件を持つ 1 つのクエリを高速化できるようになりました。 +### 1 つのクエリでプロジェクションを組み合わせる {#combining-projections} -重要なのは、ClickHouse は依然として 1 つのプロジェクション(またはベーステーブル)からのみデータを読み取りますが、 -読み取り前に不要なパーツを除外するために、他のプロジェクションのプライマリインデックスを利用できる点です。 -これは、複数のカラムでフィルタし、それぞれが異なるプロジェクションに一致し得るようなクエリで特に有用です。 +バージョン 25.6 以降では、前のバージョンで導入された `_part_offset` のサポートに基づき、 +ClickHouse は複数のプロジェクションを使用して、複数のフィルター条件を持つ +単一のクエリを高速化できるようになりました。 -> 現在、この仕組みはパーツ全体のみを除外します。グラニュールレベルでの除外は -> まだサポートされていません。 +重要な点として、ClickHouse は依然として 1 つのプロジェクション(またはベーステーブル) +からしかデータを読み取りませんが、読み取り前に不要なパーツを除外するために、 +他のプロジェクションのプライマリインデックスを利用できます。 +これは、複数の列でフィルタリングを行い、それぞれが異なるプロジェクションに +マッチする可能性があるクエリに特に有用です。 -これを示すために、`_part_offset` カラムを利用するプロジェクション付きのテーブルを定義し、 -上記の図に対応する 5 行のサンプルデータを挿入します。 +> 現在、このメカニズムはパーツ全体のみをプルーニングします。 +> グラニュールレベルでのプルーニングはまだサポートされていません。 + +これを示すため、(`_part_offset` 列を使用するプロジェクションを持つ)テーブルを定義し、 +上の図に対応する 5 行のサンプルデータを挿入します。 ```sql CREATE TABLE page_views @@ -535,11 +532,11 @@ CREATE TABLE page_views ENGINE = MergeTree ORDER BY (event_date, id) SETTINGS - index_granularity = 1, -- 1グラニュールあたり1行 - max_bytes_to_merge_at_max_space_in_pool = 1; -- マージを無効にする + index_granularity = 1, -- グラニュールあたり1行 + max_bytes_to_merge_at_max_space_in_pool = 1; -- マージを無効化 ``` -次に、テーブルにデータを挿入します。 +次にテーブルにデータを挿入します。 ```sql INSERT INTO page_views VALUES ( @@ -555,30 +552,30 @@ INSERT INTO page_views VALUES ( ``` :::note -注意: このテーブルでは、説明用に 1 行単位のグラニュールやパーツマージの無効化などのカスタム設定を使用していますが、本番環境での使用は推奨されません。 +注意: このテーブルは説明のために、1 行ごとの granule や part のマージ無効化といったカスタム設定を使用していますが、これらは本番環境での利用には推奨されません。 ::: -この構成により、次のような状態になります: +このセットアップにより、次のような状態になります: -* 5 つの個別のパーツ(挿入された各行につき 1 パーツ) -* 各行に対して 1 つのプライマリインデックスエントリ(ベーステーブルおよび各プロジェクション) -* 各パーツにはちょうど 1 行のみが含まれる +* 5 つの個別の part(挿入された各行につき 1 つ) +* ベーステーブルおよび各 projection で、行ごとに 1 つのプライマリインデックスエントリ +* 各 part にはちょうど 1 行のみが含まれる この構成で、`region` と `user_id` の両方でフィルタするクエリを実行します。 -ベーステーブルのプライマリインデックスは `event_date` と `id` から構築されているため、 -ここでは有用ではなく、そのため ClickHouse は次を使用します: +ベーステーブルのプライマリインデックスは `event_date` と `id` から構築されているため +ここでは役に立たないため、ClickHouse は代わりに次を使用します: -* `region_proj` を使って region ごとにパーツを絞り込み -* `user_id_proj` を使ってさらに `user_id` で絞り込み +* `region_proj` を用いて region に基づき part を絞り込む +* `user_id_proj` を用いてさらに `user_id` による絞り込みを行う -この挙動は `EXPLAIN projections = 1` を使うと確認でき、ClickHouse がプロジェクションをどのように選択し、適用しているかが分かります。 +この挙動は `EXPLAIN projections = 1` を使うことで確認できます。これにより、 +ClickHouse が projection をどのように選択し適用するかを確認できます。 ```sql EXPLAIN projections=1 SELECT * FROM page_views WHERE region = 'us_west' AND user_id = 107; ``` - ```response ┌─explain────────────────────────────────────────────────────────────────────────────────┐ 1. │ Expression ((Project names + Projection)) │ @@ -606,20 +603,21 @@ SELECT * FROM page_views WHERE region = 'us_west' AND user_id = 107; └────────────────────────────────────────────────────────────────────────────────────────┘ ``` -上に示した `EXPLAIN` の出力は、論理クエリプランを上から順に示しています: +上に示した `EXPLAIN` の出力は、論理クエリプランを上から下へと示しています。 + -| Row number | Description | -| ---------- | ---------------------------------------------------------------------------------------- | -| 3 | `page_views` ベーステーブルから読み取るプラン | -| 5-13 | `region_proj` を使用して region = 'us_west' である 3 つのパーツを特定し、5 つのパーツのうち 2 つをプルーニング | -| 14-22 | `user_id = 107` である 1 つのパーツを特定するために `user_id_proj` を使用し、残り 3 つのパーツのうちさらに 2 つをプルーニング | +| 行番号 | 説明 | +|--------|--------------------------------------------------------------------------------------------------------------| +| 3 | `page_views` ベーステーブルから読み取りを行う | +| 5-13 | `region_proj` を使用して region = 'us_west' である 3 つのパーツを特定し、5 つのパーツのうち 2 つを除外 | +| 14-22 | `user_id_proj` を使用して `user_id = 107` である 1 つのパーツを特定し、残り 3 つのパーツのうちさらに 2 つを除外 | 最終的に、ベーステーブルから読み取られるのは **5 つのパーツのうち 1 つだけ** です。 -複数のプロジェクションのインデックス分析を組み合わせることで、ClickHouse はスキャンするデータ量を大幅に削減し、 +複数の Projection に対するインデックス解析を組み合わせることで、ClickHouse はスキャンするデータ量を大幅に削減し、 ストレージのオーバーヘッドを抑えつつパフォーマンスを向上させます。 - ## 関連コンテンツ {#related-content} -- [ClickHouse のプライマリインデックスに関する実践的入門](/guides/best-practices/sparse-primary-indexes#option-3-projections) + +- [ClickHouse のプライマリインデックス実践入門](/guides/best-practices/sparse-primary-indexes#option-3-projections) - [マテリアライズドビュー](/docs/materialized-views) -- [ALTER PROJECTION](/sql-reference/statements/alter/projection) +- [ALTER PROJECTION](/sql-reference/statements/alter/projection) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md b/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md index 359497f2ee1..a94b9cfd346 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md @@ -1,83 +1,77 @@ --- slug: /managing-data/materialized-views-versus-projections -sidebar_label: 'マテリアライズドビューとプロジェクションの比較' -title: 'マテリアライズドビューとプロジェクション' +sidebar_label: 'マテリアライズドビュー vs プロジェクション' +title: 'マテリアライズドビューとプロジェクションの比較' hide_title: false -description: 'ClickHouse におけるマテリアライズドビューとプロジェクションを、ユースケース、パフォーマンス、制約とあわせて比較する記事。' +description: 'ClickHouse におけるマテリアライズドビューとプロジェクションを、ユースケース、パフォーマンス、制約の観点から比較する記事です。' doc_type: 'reference' -keywords: ['materialized views', 'projections', 'differences'] +keywords: ['マテリアライズドビュー', 'プロジェクション', '違い'] --- -> ユーザーからよく寄せられる質問に、どのような場合にマテリアライズドビューを使用し、どのような場合に -> プロジェクションを使用すべきかというものがあります。本記事では、これら 2 つの機能の主な違いと、特定のシナリオでどちらを選択すべきかについて説明します。 +> ユーザーからよく寄せられる質問のひとつが、マテリアライズドビューと +プロジェクションのどちらをいつ使うべきかという点です。本記事では、この 2 つの主な違いと、どのような場面で一方を他方より選択すべきかを解説します。 +## 重要な違いのまとめ {#key-differences} - -## 主な違いの概要 {#key-differences} - -以下の表は、さまざまな観点から見たマテリアライズドビューとプロジェクションの主な違いをまとめたものです。 +以下の表は、検討すべきさまざまな観点における、マテリアライズドビューとプロジェクションの主な違いをまとめたものです。 | Aspect | Materialized views | Projections | |----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Data storage and location | 結果を**別個の明示的なターゲットテーブル**に保存し、ソーステーブルへの挿入時に挿入トリガーとして動作します。 | プロジェクションは最適化されたデータレイアウトを作成し、それらは物理的に**メインテーブルのデータと並んで保存**され、ユーザーからは見えません。 | -| Update mechanism | (増分マテリアライズドビューの場合)ソーステーブルへの `INSERT` に対して**同期的に**動作します。注: リフレッシュ可能なマテリアライズドビューを使用して**スケジュール実行**することもできます。 | メインテーブルへの `INSERT` 時に、バックグラウンドで**非同期的**に更新されます。 | -| Query interaction | マテリアライズドビューを利用する場合は、**ターゲットテーブルを直接クエリ**する必要があるため、クエリを作成する際にマテリアライズドビューの存在をユーザーが認識している必要があります。 | プロジェクションは ClickHouse のクエリオプティマイザによって**自動的に選択**され、ユーザーがプロジェクションを利用するために、そのテーブルに対するクエリを変更する必要がないという意味で透過的です。バージョン 25.6 からは、複数のプロジェクションによるフィルタリングも可能です。 | -| Handling `UPDATE` / `DELETE` | マテリアライズドビューはソーステーブルについての知識を持たず、ソーステーブル_への_挿入トリガーとしてのみ動作するため、ソーステーブルに対する `UPDATE` や `DELETE` 操作に**自動的には反応しません**。これによりソーステーブルとターゲットテーブルの間でデータの不整合や陳腐化が発生する可能性があり、回避にはワークアラウンドや定期的なフルリフレッシュ(リフレッシュ可能なマテリアライズドビュー経由)が必要です。 | 既定では、(特に lightweight delete(軽量削除)において)**`DELETED` 行とは互換性がありません**。`lightweight_mutation_projection_mode`(v24.7+)を使用することで互換性を有効化できます。 | -| `JOIN` support | 対応。リフレッシュ可能なマテリアライズドビューを用いて複雑な非正規化を行うことができます。増分マテリアライズドビューは、最も左側のテーブルへの挿入のみでトリガーされます。 | 非対応。プロジェクション定義内では、マテリアライズされたデータをフィルタリングするための `JOIN` 演算はサポートされていません。 | -| `WHERE` clause in definition | 対応。マテリアライズ前にデータをフィルタリングするために `WHERE` 句を含めることができます。 | 非対応。プロジェクション定義内では、マテリアライズされたデータをフィルタリングするための `WHERE` 句はサポートされていません。 | -| Chaining capabilities | 対応。あるマテリアライズドビューのターゲットテーブルを別のマテリアライズドビューのソースとして使用でき、多段パイプラインを構築できます。 | 非対応。プロジェクションを連鎖させることはできません。 | -| Applicable table engines | さまざまなソーステーブルエンジンで使用できますが、ターゲットテーブルは通常 `MergeTree` ファミリです。 | `MergeTree` ファミリのテーブルエンジンでのみ**利用可能**です。 | -| Failure handling | データ挿入時に失敗するとターゲットテーブル側でデータが失われるため、不整合が発生する可能性があります。 | 障害はバックグラウンドで**サイレントに**処理されます。クエリでは、マテリアライズ済みパーツと未マテリアライズパーツを透過的に混在させることができます。 | -| Operational overhead | 明示的なターゲットテーブルの作成と、しばしば手動でのバックフィルが必要です。`UPDATE`/`DELETE` との整合性維持には追加の複雑さが伴います。 | プロジェクションは自動的に管理され同期が保たれるため、一般に運用負荷は低くなります。 | -| `FINAL` query compatibility | 概ね互換性がありますが、多くの場合、ターゲットテーブル側で `GROUP BY` が必要になります。 | `FINAL` クエリでは**動作しません**。 | -| Lazy materialization | 対応。 | マテリアライゼーション関連機能を使用する際は、プロジェクション互換性の問題を監視してください。`query_plan_optimize_lazy_materialization = false` を設定する必要がある場合があります。 | -| Parallel replicas | 対応。 | 非対応。 | -| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | 対応。 | 対応。 | -| Lightweight updates and deletes | 対応。 | 非対応。 | - - +| Data storage and location | 結果を**別個の明示的なターゲットテーブル**に保存し、ソーステーブルへの `INSERT` 時に挿入トリガーとして動作します。 | プロジェクションは最適化されたデータレイアウトを作成し、物理的には**メインテーブルのデータと並んで保存**され、ユーザーからは見えません。 | +| Update mechanism | (インクリメンタルなマテリアライズドビューの場合)ソーステーブルへの `INSERT` に対して**同期的**に動作します。注: refreshable materialized view を使用することで、**スケジュール**実行も可能です。 | メインテーブルへの `INSERT` 後にバックグラウンドで**非同期的**に更新されます。 | +| Query interaction | Materialized View を利用するには**ターゲットテーブルを直接クエリ**する必要があり、クエリを書く際にユーザーはマテリアライズドビューの存在を意識する必要があります。 | プロジェクションは ClickHouse のクエリオプティマイザによって**自動的に選択**されます。ユーザーはプロジェクション付きテーブルに対してクエリを書き換える必要はなく、その点で透過的です。バージョン 25.6 以降では、複数のプロジェクションを条件に応じて併用することも可能です。 | +| Handling `UPDATE` / `DELETE` | materialized view は、ソーステーブルに関する情報を持たず、ソーステーブルへの挿入トリガーとしてのみ動作するため、ソーステーブルでの `UPDATE` や `DELETE` 操作に**自動では反応しません**。このためソーステーブルとターゲットテーブル間でデータの陳腐化や不整合が生じる可能性があり、回避にはワークアラウンドや、refreshable materialized view による定期的な完全リフレッシュが必要です。 | 既定では、(特に lightweight delete において)**`DELETED` 行とは非互換**です。`lightweight_mutation_projection_mode`(v24.7+)を有効にすることで互換性を持たせることができます。 | +| `JOIN` support | 対応あり。Refreshable materialized view は複雑な非正規化に利用できます。インクリメンタルなマテリアライズドビューは、左端のテーブルへの挿入時のみトリガーされます。 | 対応なし。プロジェクション定義内では、materialized なデータをフィルタリングするための `JOIN` 操作はサポートされません。ただし、プロジェクションを持つテーブル同士を結合するクエリ自体は通常どおり動作し、プロジェクションは各テーブルへのアクセスを個別に最適化します。 | +| `WHERE` clause in definition | 対応あり。マテリアライズ前にデータをフィルタするために `WHERE` 句を含めることができます。 | 対応なし。プロジェクション定義内では、materialized なデータをフィルタリングするための `WHERE` 句はサポートされていません。 | +| Chaining capabilities | 対応あり。あるマテリアライズドビューのターゲットテーブルを別のマテリアライズドビューのソースとして利用でき、多段パイプラインを構成できます。 | 対応なし。プロジェクションを連結(チェーン)することはできません。 | +| Applicable table engines | 各種ソーステーブルエンジンで利用できますが、ターゲットテーブルは通常 `MergeTree` ファミリーです。 | `MergeTree` ファミリーのテーブルエンジンに対してのみ**利用可能**です。 | +| Failure handling | データ挿入中に失敗した場合、ターゲットテーブル側でデータが失われ、整合性に問題が生じる可能性があります。 | 失敗はバックグラウンドで**サイレントに**処理されます。クエリは materialized なパーツと非 materialized なパーツをシームレスに混在させて参照できます。 | +| Operational overhead | 明示的なターゲットテーブルの作成と、多くの場合手動でのバックフィルが必要です。`UPDATE`/`DELETE` と整合性を保つための管理は複雑さを増します。 | プロジェクションは自動的に維持・同期され、一般的に運用上の負荷は低くなります。 | +| `FINAL` query compatibility | 概ね互換性がありますが、ターゲットテーブル上で `GROUP BY` が必要になることが多いです。 | `FINAL` クエリとは**併用できません**。 | +| Lazy materialization | 対応あり。 | materialization 関連機能を使用する際は、プロジェクションの互換性問題に注意してください。必要に応じて `query_plan_optimize_lazy_materialization = false` を設定する必要があります。 | +| Parallel replicas | 対応あり。 | 対応なし。 | +| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | 対応あり。 | 対応あり。 | +| Lightweight updates and deletes | 対応あり。 | 対応なし。 | ## マテリアライズドビューとプロジェクションの比較 {#choose-between} -### マテリアライズドビューを選ぶべきケース {#choosing-materialized-views} - -マテリアライズドビューの利用を検討すべきなのは次のような場合です: - -- **リアルタイム ETL と多段階データパイプライン** を扱う場合: データ到着時に複雑な変換や集約を実行したり、ビューをチェインして複数ステージにわたってデータをルーティングする必要がある場合。 -- **複雑な非正規化** が必要な場合: 複数のソース(テーブル、サブクエリ、ディクショナリ)からのデータを 1 つのクエリ最適化済みテーブルに事前結合する必要があり、特にリフレッシュ可能なマテリアライズドビューを使った定期的なフルリフレッシュが許容できる場合。 -- **スキーマを明示的に制御したい** 場合: 事前計算済み結果用に、独自のスキーマとエンジンを持つ独立したターゲットテーブルが必要であり、データモデリングの柔軟性を高めたい場合。 -- **インジェスト時にフィルタリングしたい** 場合: データがマテリアライズされる _前に_ フィルタリングし、ターゲットテーブルに書き込まれるデータ量を削減する必要がある場合。 +### マテリアライズドビューを選択するタイミング {#choosing-materialized-views} -### マテリアライズドビューを避けるべきケース {#avoid-materialized-views} +次のような場合は、マテリアライズドビューの利用を検討してください。 -マテリアライズドビューの使用を避けることを検討すべきなのは次のような場合です: +- **リアルタイム ETL と多段階のデータパイプライン**を扱う場合: データが到着したタイミングで複雑な変換や集計を実行したり、ビューを連結することで複数のステージにまたがってデータをルーティングする必要がある場合。 +- **複雑な非正規化**が必要な場合: 複数のソース(テーブル、サブクエリ、またはディクショナリ)のデータを 1 つのクエリ最適化されたテーブルに事前結合する必要があり、特にリフレッシュ可能なマテリアライズドビューを用いた定期的なフルリフレッシュが許容できる場合。 +- **スキーマを明示的に制御**したい場合: 事前計算済みの結果を保持するために、それ専用のスキーマとエンジンを持つ独立したターゲットテーブルが必要であり、データモデリングの柔軟性を高めたい場合。 +- **インジェスト時にフィルタリング**したい場合: データがマテリアライズされる_前に_フィルタリングを行い、ターゲットテーブルに書き込まれるデータ量を削減する必要がある場合。 -- **ソースデータが頻繁に更新または削除される** 場合: ソーステーブルとターゲットテーブル間の整合性を維持するための追加戦略がないと、増分マテリアライズドビューが古くなり、不整合を起こす可能性があります。 -- **シンプルさと自動最適化を優先したい** 場合: 別個のターゲットテーブルを管理したくない場合。 +### マテリアライズドビューの使用を避けるべき場合 {#avoid-materialized-views} -### プロジェクションを選ぶべきケース {#choosing-projections} +次のような場合には、マテリアライズドビューの使用を避けることを検討してください。 -プロジェクションの利用を検討すべきなのは次のような場合です: +- **ソースデータが頻繁に更新または削除される場合**: ソーステーブルとターゲットテーブル間の一貫性を保つための追加の戦略がないと、増分マテリアライズドビューは古くなり、一貫性が失われる可能性があります。 +- **シンプルさや自動最適化を重視する場合**: 別個のターゲットテーブルを管理することを避けたい場合。 -- **単一テーブルに対するクエリを最適化する** 場合: 主な目的が、単一のベーステーブルに対するクエリを高速化することであり、そのために別のソート順を提供したり、プライマリキーに含まれない列上のフィルタを最適化したり、単一テーブルに対する集約を事前計算したりする場合。 -- **クエリの透過性** を求める場合: クエリを変更せずに元のテーブルを対象にし、ClickHouse がクエリごとに最適なデータレイアウトを選択することに任せたい場合。 +### プロジェクションを選択すべきケース {#choosing-projections} -### プロジェクションを避けるべきケース {#avoid-projections} +次のような場合には、プロジェクションの利用を検討してください。 -プロジェクションの使用を避けることを検討すべきなのは次のような場合です: +- **単一テーブルに対するクエリの最適化**: 主な目的が、単一のベーステーブルに対するクエリを高速化することであり、そのために別のソート順を用意したり、プライマリキーに含まれない列に対するフィルタを最適化したり、単一テーブルに対する集計をあらかじめ計算しておきたい場合。 +- **クエリの透過性**を求める場合: クエリでは元のテーブルをそのまま対象とし、変更を加えずに、特定のクエリに対して ClickHouse が最適なデータレイアウトを自動的に選択するようにしたい場合。 -- **複雑なデータ変換や多段階 ETL が必要** な場合: プロジェクションは、その定義内で `JOIN` 演算をサポートせず、多段階パイプラインを構築するように変更することもできず、ウィンドウ関数や複雑な `CASE` 式など一部の SQL 機能も扱えません。そのため、複雑なデータ変換には適していません。 -- **マテリアライズされるデータを明示的にフィルタリングする必要がある** 場合: プロジェクションは、その定義内で `WHERE` 句をサポートしておらず、プロジェクション自体にマテリアライズされるデータをフィルタリングできません。 -- **MergeTree 以外のテーブルエンジンが使用されている** 場合: プロジェクションは `MergeTree` ファミリーのエンジンを使用するテーブルに対してのみ利用できます。 -- `FINAL` クエリが不可欠な場合: プロジェクションは、重複排除のために使用されることがある `FINAL` クエリでは機能しません。 -- [parallel replicas](/deployment-guides/parallel-replicas) が必要な場合: プロジェクションではサポートされていません。 +### プロジェクションの利用を避けるべき場合 {#avoid-projections} +次のような場合には、プロジェクションの使用を避けることを検討してください。 +- **複雑なデータ変換や多段階の ETL が必要な場合**: プロジェクション定義は `JOIN` 演算をサポートせず、多段階パイプラインを構築するために連鎖させることもできません。また、ウィンドウ関数や複雑な `CASE` 式など、一部の SQL 機能を扱うこともできません。プロジェクションを持つテーブルに対するクエリは自由に結合できますが、プロジェクション自体は複雑なデータ変換には適していません。 +- **マテリアライズするデータを明示的にフィルタリングする必要がある場合**: プロジェクションは、そのプロジェクションにマテリアライズされるデータをフィルタリングするための `WHERE` 句を定義内でサポートしていません。 +- **MergeTree 以外のテーブルエンジンを使用している場合**: プロジェクションは、`MergeTree` ファミリーのエンジンを使用するテーブルでのみ利用可能です。 +- **`FINAL` クエリが不可欠な場合**: プロジェクションは、重複排除などの目的で使用されることがある `FINAL` クエリとは併用できません。 +- **[parallel replicas](/deployment-guides/parallel-replicas) が必要な場合**: parallel replicas はプロジェクションと併用できません。 -## Summary {#summary} +## まとめ {#summary} -マテリアライズドビューとプロジェクションは、どちらもクエリの最適化やデータ変換のための強力なツールであり、一般的には「どちらか一方を選ぶ」ものとして考えることは推奨しません。代わりに、これらを補完的に組み合わせて使用することで、クエリのパフォーマンスを最大限に引き出すことができます。そのため、ClickHouse でマテリアライズドビューとプロジェクションのどちらを選択するかは、具体的なユースケースやアクセスパターンに大きく依存します。 +マテリアライズドビューとプロジェクションは、いずれもクエリの最適化やデータ変換のための強力なツールであり、一般的には「どちらか一方だけを選ぶ」ものとして捉える必要はありません。むしろ、両者を補完的に組み合わせて使用することで、クエリから最大限のパフォーマンスを引き出すことができます。そのため、ClickHouse においてマテリアライズドビューとプロジェクションのどちらを選択するかは、特定のユースケースやアクセスパターンに大きく依存します。 -一般的な指針として、1 つ以上のソーステーブルからターゲットテーブルへデータを集約したり、大規模な複雑な変換処理を実行する必要がある場合には、マテリアライズドビューの利用を検討すべきです。マテリアライズドビューは、高コストな集約処理の負荷をクエリ実行時から挿入時へと移すのに非常に優れています。日次・月次のロールアップやリアルタイムダッシュボード、データサマリなどに最適な選択肢です。 +一般的な指針として、1 つ以上のソーステーブルからターゲットテーブルへデータを集約したり、大規模かつ複雑な変換処理を行う必要がある場合は、マテリアライズドビューの利用を検討すべきです。マテリアライズドビューは、高コストな集計処理の負荷をクエリ時ではなく挿入時に移すことに非常に優れています。日次・月次ロールアップ、リアルタイムダッシュボード、サマリーデータなどの用途に最適な選択肢です。 -一方で、テーブルのプライマリキー(ディスク上のデータの物理的な並び順を決定するカラム)とは異なるカラムでフィルタするクエリを最適化する必要がある場合には、プロジェクションを使用すべきです。特に、テーブルのプライマリキーをもはや変更できない状況や、アクセスパターンがプライマリキーで表現できる範囲よりも多様である場合に有用です。 +一方で、プロジェクションを使うべきなのは、テーブルのプライマリキー(ディスク上のデータの物理的な並び順を決定する列)とは異なる列でフィルタリングするクエリを最適化する必要がある場合です。特に、テーブルのプライマリキーを変更することがもはや不可能な場合や、アクセスパターンがプライマリキーで想定されているものより多様な場合に有用です。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md b/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md index 7b3aba6c19f..afe692c2e6b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md @@ -122,6 +122,7 @@ With our initial schema defined, we can populate the data using an `INSERT INTO INSERT INTO posts SELECT * FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/*.parquet') 0 rows in set. Elapsed: 148.140 sec. Processed 59.82 million rows, 38.07 GB (403.80 thousand rows/s., 257.00 MB/s.) + ``` > The above query loads 60m rows. While small for ClickHouse, users with slower internet connections may wish to load a subset of data. This can be achieved by simply specifying the years they wish to load via a glob pattern e.g. `https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2008.parquet` or `https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/{2008, 2009}.parquet`. See [here](/sql-reference/table-functions/file#globs-in-path) for how glob patterns can be used to target subsets of files. diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/index.md index be98cb2de83..a9c9baf38c6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/index.md @@ -6,7 +6,7 @@ keywords: ['デプロイメントガイド', 'スケーリング', 'クラスタ doc_type: 'landing-page' --- -# デプロイメントとスケーリング +# デプロイメントとスケーリング {#deployment-and-scaling} 本セクションでは、次のトピックについて説明します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx index 5ac0b01b885..7dacdf00c37 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx @@ -20,11 +20,10 @@ import image_9 from '@site/static/images/deployment-guides/parallel-replicas-9.p - ## はじめに \{#introduction\} ClickHouse はクエリを非常に高速に処理しますが、これらのクエリはどのようにして -複数のサーバーに分散され、並列実行されているのでしょうか。 +複数のサーバーに分散され、並列実行されているのでしょうか。 > このガイドではまず、ClickHouse が分散テーブルを用いてクエリを複数のシャードにどのように分散するかを説明し、その後、クエリの実行に複数のレプリカをどのように活用できるかを解説します。 @@ -39,24 +38,27 @@ ClickHouse はクエリを非常に高速に処理しますが、これらのク 上図は、クライアントが分散テーブルにクエリを実行したときに何が起こるかを示しています。
    -
  1. - SELECT クエリは、任意のノード上の分散テーブルに送信されます - (ラウンドロビン戦略で選択されたノード、またはロードバランサーによって - 特定のサーバーにルーティングされたノードなど)。このノードが - コーディネータとして動作します。 -
  2. -
  3. - ノードは、分散テーブルに指定された情報を用いて、クエリを実行する必要がある - 各シャードを特定し、クエリをそれぞれのシャードに送信します。 -
  4. -
  5. - 各シャードはローカルでデータを読み取り、フィルタおよび集約したうえで、 - マージ可能な状態をコーディネータに返します。 -
  6. -
  7. - コーディネータノードがデータをマージし、その結果をクライアントに - 応答として返します。 -
  8. +
  9. + SELECT クエリは、任意のノード上の分散テーブルに送信されます + (ラウンドロビン戦略で選択されたノード、またはロードバランサーによって + 特定のサーバーにルーティングされたノードなど)。このノードが + コーディネータとして動作します。 +
  10. + +
  11. + ノードは、分散テーブルに指定された情報を用いて、クエリを実行する必要がある + 各シャードを特定し、クエリをそれぞれのシャードに送信します。 +
  12. + +
  13. + 各シャードはローカルでデータを読み取り、フィルタおよび集約したうえで、 + マージ可能な状態をコーディネータに返します。 +
  14. + +
  15. + コーディネータノードがデータをマージし、その結果をクライアントに + 応答として返します。 +
レプリカを追加した場合もプロセスはほぼ同様で、唯一の違いは、各シャードからは単一のレプリカのみがクエリを実行する点です。これにより、より多くのクエリを並列に処理できるようになります。 @@ -64,7 +66,7 @@ ClickHouse はクエリを非常に高速に処理しますが、これらのク ## 非シャーディングアーキテクチャ \{#non-sharded-architecture\} ClickHouse Cloud は、前述のアーキテクチャとは大きく異なるアーキテクチャを採用しています。 -(詳細については、["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) +(詳細については、["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) を参照してください)。コンピュートとストレージが分離され、事実上無制限のストレージ容量が 利用できるため、シャードの必要性はそれほど重要ではなくなります。 @@ -103,47 +105,54 @@ Keeper クラスタは、メタデータに関する単一の信頼できる情 並列レプリカでは、次のようになります。
    -
  1. - クライアントからのクエリはロードバランサーを通過した後、1 つのノードに - 送信されます。このノードがこのクエリのコーディネーターになります。 -
  2. -
  3. - ノードは各パーツのインデックスを解析し、処理すべき適切なパーツと - granule を選択します。 -
  4. -
  5. - コーディネーターは、ワークロードを複数の granule の集合に分割し、 - それぞれを異なるレプリカに割り当てられるようにします。 -
  6. -
  7. - 各 granule の集合は対応するレプリカによって処理され、完了後、 - マージ可能な状態がコーディネーターに送信されます。 -
  8. -
  9. - 最後に、コーディネーターがレプリカからのすべての結果をマージし、 - レスポンスをクライアントに返します。 -
  10. +
  11. + クライアントからのクエリはロードバランサーを通過した後、1 つのノードに + 送信されます。このノードがこのクエリのコーディネーターになります。 +
  12. + +
  13. + ノードは各パーツのインデックスを解析し、処理すべき適切なパーツと + granule を選択します。 +
  14. + +
  15. + コーディネーターは、ワークロードを複数の granule の集合に分割し、 + それぞれを異なるレプリカに割り当てられるようにします。 +
  16. + +
  17. + 各 granule の集合は対応するレプリカによって処理され、完了後、 + マージ可能な状態がコーディネーターに送信されます。 +
  18. + +
  19. + 最後に、コーディネーターがレプリカからのすべての結果をマージし、 + レスポンスをクライアントに返します。 +
上記のステップは、並列レプリカが理論上どのように動作するかを説明したものです。 しかし、実際には、このようなロジックが完全には機能しない要因が多数存在します。
    -
  1. - 一部のレプリカが利用不能になっている場合があります。 -
  2. -
  3. - ClickHouse におけるレプリケーションは非同期であるため、ある時点では - 一部のレプリカが同じパーツを保持していない可能性があります。 -
  4. -
  5. - レプリカ間のテールレイテンシを何らかの方法で扱う必要があります。 -
  6. -
  7. - ファイルシステムキャッシュは各レプリカ上のアクティビティに応じて - レプリカごとに異なるため、タスクをランダムに割り当てると、 - キャッシュの局所性を考慮した場合に最適とは言えない性能につながることがあります。 -
  8. +
  9. + 一部のレプリカが利用不能になっている場合があります。 +
  10. + +
  11. + ClickHouse におけるレプリケーションは非同期であるため、ある時点では + 一部のレプリカが同じパーツを保持していない可能性があります。 +
  12. + +
  13. + レプリカ間のテールレイテンシを何らかの方法で扱う必要があります。 +
  14. + +
  15. + ファイルシステムキャッシュは各レプリカ上のアクティビティに応じて + レプリカごとに異なるため、タスクをランダムに割り当てると、 + キャッシュの局所性を考慮した場合に最適とは言えない性能につながることがあります。 +
これらの要因をどのように克服するかについては、次のセクションで説明します。 @@ -155,18 +164,21 @@ Keeper クラスタは、メタデータに関する単一の信頼できる情 Announcements
    -
  1. - クライアントからのクエリはロードバランサーを経由して 1 つのノードに送信されます。このノードがこのクエリのコーディネーターになります。 -
  2. -
  3. - コーディネーターノードは、クラスター内のすべてのレプリカからアナウンスメントを取得するリクエストを送信します。レプリカは、テーブルの現在のパーツ集合について、わずかに異なる見え方をしている場合があります。そのため、誤ったスケジューリング判断を避けるために、この情報を収集する必要があります。 -
  4. -
  5. - その後、コーディネーターノードはアナウンスメントを使用して、異なるレプリカに割り当て可能なグラニュールの集合を定義します。たとえばここでは、レプリカ 2 は自分のアナウンスメントで part 3 を提示していないため、part 3 のグラニュールはレプリカ 2 には一切割り当てられていないことがわかります。また、レプリカ 3 はアナウンスメントを提供していないため、このレプリカにはタスクが割り当てられていない点にも注意してください。 -
  6. -
  7. - 各レプリカがそれぞれのグラニュールのサブセットに対してクエリ処理を行い、マージ可能な状態をコーディネーターに送り返した後、コーディネーターは結果をマージし、そのレスポンスをクライアントに送信します。 -
  8. +
  9. + クライアントからのクエリはロードバランサーを経由して 1 つのノードに送信されます。このノードがこのクエリのコーディネーターになります。 +
  10. + +
  11. + コーディネーターノードは、クラスター内のすべてのレプリカからアナウンスメントを取得するリクエストを送信します。レプリカは、テーブルの現在のパーツ集合について、わずかに異なる見え方をしている場合があります。そのため、誤ったスケジューリング判断を避けるために、この情報を収集する必要があります。 +
  12. + +
  13. + その後、コーディネーターノードはアナウンスメントを使用して、異なるレプリカに割り当て可能なグラニュールの集合を定義します。たとえばここでは、レプリカ 2 は自分のアナウンスメントで part 3 を提示していないため、part 3 のグラニュールはレプリカ 2 には一切割り当てられていないことがわかります。また、レプリカ 3 はアナウンスメントを提供していないため、このレプリカにはタスクが割り当てられていない点にも注意してください。 +
  14. + +
  15. + 各レプリカがそれぞれのグラニュールのサブセットに対してクエリ処理を行い、マージ可能な状態をコーディネーターに送り返した後、コーディネーターは結果をマージし、そのレスポンスをクライアントに送信します。 +
### 動的なコーディネーション \{#dynamic-coordination\} @@ -180,37 +192,41 @@ Keeper クラスタは、メタデータに関する単一の信頼できる情 動的コーディネーション - パート 1
    -
  1. - レプリカは、タスクを処理可能であること、またどの程度の作業量を処理可能かをコーディネータノードに通知します。 -
  2. -
  3. - コーディネータは、レプリカにタスクを割り当てます。 -
  4. +
  5. + レプリカは、タスクを処理可能であること、またどの程度の作業量を処理可能かをコーディネータノードに通知します。 +
  6. + +
  7. + コーディネータは、レプリカにタスクを割り当てます。 +
動的コーディネーション - パート 2
    -
  1. - レプリカ 1 と 2 はタスクを非常に速く完了できます。そのため、コーディネータノードに別のタスクを要求します。 -
  2. -
  3. - コーディネータは、新しいタスクをレプリカ 1 と 2 に割り当てます。 -
  4. +
  5. + レプリカ 1 と 2 はタスクを非常に速く完了できます。そのため、コーディネータノードに別のタスクを要求します。 +
  6. + +
  7. + コーディネータは、新しいタスクをレプリカ 1 と 2 に割り当てます。 +
動的コーディネーション - パート 3
    -
  1. - すべてのレプリカが、自身に割り当てられたタスクの処理を完了しました。そこで、さらにタスクを要求します。 -
  2. -
  3. - コーディネータは、アナウンスを使って残っているタスクを確認しますが、もはや残りのタスクはありません。 -
  4. -
  5. - コーディネータは、すべての処理が完了したことをレプリカに通知します。次に、マージ可能な状態をすべてマージし、クエリに応答します。 -
  6. +
  7. + すべてのレプリカが、自身に割り当てられたタスクの処理を完了しました。そこで、さらにタスクを要求します。 +
  8. + +
  9. + コーディネータは、アナウンスを使って残っているタスクを確認しますが、もはや残りのタスクはありません。 +
  10. + +
  11. + コーディネータは、すべての処理が完了したことをレプリカに通知します。次に、マージ可能な状態をすべてマージし、クエリに応答します。 +
### キャッシュローカリティの管理 \{#managing-cache-locality\} @@ -220,34 +236,38 @@ Keeper クラスタは、メタデータに関する単一の信頼できる情 保証できるでしょうか。前の例では、次のようにタスクが割り当てられていました。 - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
レプリカ 1レプリカ 2レプリカ 3
パート 1g1, g6, g7g2, g4, g5g3
パート 2g1g2, g4, g5g3
パート 3g1, g6g2, g4, g5g3
+ + レプリカ 1レプリカ 2レプリカ 3
パート 1g1, g6, g7g2, g4, g5g3
パート 2g1g2, g4, g5g3
パート 3g1, g6g2, g4, g5g3
同じタスクが同じレプリカに割り当てられ、キャッシュの恩恵を受けられるようにするために、 @@ -284,13 +304,13 @@ Keeper クラスタは、メタデータに関する単一の信頼できる情 | Setting | Description | |----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `enable_parallel_replicas` | `0`: 無効
`1`: 有効
`2`: 並列レプリカの使用を強制し、使用されない場合は例外をスローします。 | +| `enable_parallel_replicas` | `0`: 無効
`1`: 有効
`2`: 並列レプリカの使用を強制し、使用されない場合は例外をスローします。 | | `cluster_for_parallel_replicas` | 並列レプリケーションに使用するクラスタ名。ClickHouse Cloud を使用している場合は `default` を指定します。 | | `max_parallel_replicas` | 複数レプリカ上でクエリを実行する際に使用するレプリカの最大数。クラスタ内のレプリカ数より小さい値を指定した場合、ノードはランダムに選択されます。この値は水平方向のスケーリングを考慮してオーバーコミットとして設定することもできます。 | -| `parallel_replicas_min_number_of_rows_per_replica` | 処理が必要な行数に基づいて使用するレプリカ数を制限するのに役立ちます。使用されるレプリカ数は次の式で決まります:
`estimated rows to read` / `min_number_of_rows_per_replica`. | -| `allow_experimental_analyzer` | `0`: 旧アナライザを使用
`1`: 新アナライザを使用

使用するアナライザによって、並列レプリカの動作が変わる場合があります。 | +| `parallel_replicas_min_number_of_rows_per_replica` | 処理が必要な行数に基づいて使用するレプリカ数を制限するのに役立ちます。使用されるレプリカ数は次の式で決まります:
`estimated rows to read` / `min_number_of_rows_per_replica`. | +| `allow_experimental_analyzer` | `0`: 旧アナライザを使用
`1`: 新アナライザを使用

使用するアナライザによって、並列レプリカの動作が変わる場合があります。 | -## パラレルレプリカに関する問題の調査 +## パラレルレプリカに関する問題の調査 \{#investigating-issues-with-parallel-replicas\} 各クエリでどの設定が使用されているかは、 [`system.query_log`](/docs/operations/system-tables/query_log) テーブルで確認できます。 @@ -308,7 +328,6 @@ FROM clusterAllReplicas('default', system.events) WHERE event ILIKE '%ParallelReplicas%' ``` -
レスポンス @@ -370,7 +389,6 @@ WHERE query_id = 'ad40c712-d25d-45c4-b1a1-a28ba8d4019c' ORDER BY event_time_microseconds ASC ``` -
レスポンス diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md index a8b481d0a43..d751cfa1ba2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md @@ -9,19 +9,19 @@ keywords: ['レプリケーション', '高可用性', 'クラスター構成', --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ReplicationArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/replication.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > この例では、データをレプリケートするシンプルな ClickHouse クラスターのセットアップ方法を学びます。ここでは 5 台のサーバーを構成します。うち 2 台はデータのコピーを保持するために使用します。残りの 3 台のサーバーは、データレプリケーションを調整するために使用します。 @@ -31,7 +31,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前提条件 {#pre-requisites} - 以前に [ローカルの ClickHouse サーバー](/install) をセットアップしている diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md index 1a3de02acd9..0d0da37adfc 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md @@ -9,19 +9,19 @@ keywords: ['シャーディング', '水平スケーリング', '分散データ --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ShardingArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/sharding.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > この例では、スケールするシンプルな ClickHouse クラスターのセットアップ方法を学びます。 > ここでは 5 台のサーバーを構成します。うち 2 台はデータのシャーディングに使用し、 @@ -33,7 +33,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前提条件 {#pre-requisites} - 以前に [ローカル ClickHouse サーバー](/install) をセットアップしたことがある diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md index 783be44d886..bb00080f6b9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md @@ -10,14 +10,14 @@ keywords: ['クラスター デプロイメント', 'レプリケーション', import Image from '@theme/IdealImage'; import SharedReplicatedArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/both.png'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import KeeperConfig from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > この例では、レプリケーションとスケールの両方に対応したシンプルな ClickHouse クラスターをセットアップする方法を学びます。クラスターは 2 つのシャードと 2 つのレプリカに加え、クラスター内の調整とクォーラムの維持を行う 3 ノード構成の ClickHouse Keeper クラスターで構成されています。 @@ -27,7 +27,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前提条件 {#prerequisites} - すでに [ローカルの ClickHouse サーバー](/install) をセットアップしている diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md index e2027e59947..4c8289717f2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md @@ -8,7 +8,7 @@ doc_type: 'guide' keywords: ['デプロイメント', 'アーキテクチャ', 'レプリケーション', 'シャーディング', 'クラスタ構成'] --- -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; このセクションのデプロイメント例は、ClickHouse の Support and Services チームが ClickHouse ユーザーに提供しているアドバイスに基づいています。これらは実際に動作する例であり、 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/architecture.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/architecture.md index 8d9187de727..e5546f6a7f7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/architecture.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/architecture.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# アーキテクチャ概要 +# アーキテクチャ概要 {#architecture-overview} ClickHouse は真のカラム指向 DBMS です。データはカラム単位で保存され、クエリ実行時には配列(カラムのベクターまたはチャンク)単位で処理されます。 可能な限り、個々の値ではなく配列に対して演算が行われます。 @@ -242,7 +242,7 @@ IO スレッドプールは、`IOThreadPool::get()` メソッドからアクセ -## 同時実行制御 +## 同時実行制御 {#concurrency-control} 並列実行が可能なクエリは、自身を制限するために `max_threads` 設定を使用します。この設定のデフォルト値は、単一クエリがすべての CPU コアを最適な形で利用できるように選択されています。では、複数のクエリが同時に実行されており、それぞれがデフォルトの `max_threads` 設定値を使用している場合はどうなるでしょうか。その場合、クエリ同士で CPU リソースを共有することになります。OS はスレッドを頻繁に切り替えることで公平性を保証しますが、これはある程度の性能低下を招きます。`ConcurrencyControl` は、このペナルティに対処し、大量のスレッド割り当てを避けるのに役立ちます。構成設定 `concurrent_threads_soft_limit_num` は、CPU に負荷をかけ始める前に、どれだけ多くのスレッドを同時に割り当てられるかを制限するために使用されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-arm.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-arm.md index 9fb456d8d01..0dd3ffde8a7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-arm.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-arm.md @@ -7,7 +7,7 @@ title: 'Linux 上で AARCH64 向け ClickHouse をビルドする方法' doc_type: 'guide' --- -# Linux 上で AArch64 向けに ClickHouse をビルドする方法 +# Linux 上で AArch64 向けに ClickHouse をビルドする方法 {#how-to-build-clickhouse-on-linux-for-aarch64} AArch64 マシン上で AArch64 向けに ClickHouse をビルドする場合、特別な手順は必要ありません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md index c729981e2df..5ea1889e8a9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md @@ -7,15 +7,11 @@ title: 'LoongArch64 向け Linux 上でのビルド' doc_type: 'guide' --- - - -# Linux 上での LoongArch64 向けビルド +# Linux 上での LoongArch64 向けビルド {#build-on-linux-for-loongarch64} ClickHouse は LoongArch64 を実験的にサポートしています - - -## ClickHouse をビルドする +## ClickHouse をビルドする {#build-clickhouse} ビルドに必要な LLVM のバージョンは 19.1.0 以上である必要があります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-osx.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-osx.md index b279a8ab85d..b7483aa4fd1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-osx.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-osx.md @@ -7,9 +7,7 @@ title: 'Linux で macOS 向けにビルド' doc_type: 'guide' --- - - -# Linux 上で macOS 向けの ClickHouse をビルドする方法 +# Linux 上で macOS 向けの ClickHouse をビルドする方法 {#how-to-build-clickhouse-on-linux-for-macos} このドキュメントは、Linux マシンを使って OS X 上で実行する `clickhouse` バイナリをビルドしたい場合の手順です。 主なユースケースは、Linux マシン上で実行される継続的インテグレーション(CI)チェックです。 @@ -21,9 +19,7 @@ macOS 向けのクロスビルドは [ビルド手順](../development/build.md) ARM アーキテクチャをターゲットにする場合は、すべての `x86_64` を `aarch64` に置き換えてください。 たとえば、手順全体を通して `x86_64-apple-darwin` を `aarch64-apple-darwin` に置き換えます。 - - -## クロスコンパイル用ツールセットをインストールする +## クロスコンパイル用ツールセットをインストールする {#install-cross-compilation-toolset} `cctools` をインストールしたパスを `${CCTOOLS}` として覚えておきましょう。 @@ -53,8 +49,7 @@ cd ClickHouse/cmake/toolchain/darwin-x86_64 curl -L 'https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX11.0.sdk.tar.xz' | tar xJ --strip-components=1 ``` - -## ClickHouse をビルドする +## ClickHouse をビルドする {#build-clickhouse} ```bash cd ClickHouse diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md index 9344d23a940..d28f8b77f28 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md @@ -7,15 +7,11 @@ title: 'Linux 上で RISC-V 64 向けに ClickHouse をビルドする方法' doc_type: 'guide' --- - - -# Linux 上で RISC-V 64 向けに ClickHouse をビルドする方法 +# Linux 上で RISC-V 64 向けに ClickHouse をビルドする方法 {#how-to-build-clickhouse-on-linux-for-risc-v-64} ClickHouse は RISC-V を実験的にサポートしています。すべての機能を有効にできるわけではありません。 - - -## ClickHouse をビルドする +## ClickHouse をビルドする {#build-clickhouse} RISC-V ではないマシン上で RISC-V 向けにクロスコンパイルするには: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md index 969418a150e..2e702066df2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md @@ -7,38 +7,23 @@ title: 'Linux 上での s390x (zLinux) 向けビルド' doc_type: 'guide' --- - - -# Linux 上での s390x(zLinux)向けビルド +# Linux 上での s390x (zLinux) 向けビルド {#build-on-linux-for-s390x-zlinux} ClickHouse は s390x を実験的にサポートしています。 +## s390x 向けの ClickHouse のビルド {#building-clickhouse-for-s390x} +s390x では、他のプラットフォームと同様に、OpenSSL はスタティックライブラリとしてビルドされます。OpenSSL を動的ライブラリとしてビルドしたい場合は、CMake に `-DENABLE_OPENSSL_DYNAMIC=1` を指定する必要があります。 -## s390x 向けの ClickHouse のビルド - -s390x には、OpenSSL 関連のビルドオプションが 2 つあります。 - -* デフォルトでは、s390x 上では OpenSSL は共有ライブラリとしてビルドされます。これは、他のすべてのプラットフォームでは OpenSSL が静的ライブラリとしてビルドされるのとは異なります。 -* それでも OpenSSL を静的ライブラリとしてビルドしたい場合は、CMake に `-DENABLE_OPENSSL_DYNAMIC=0` を渡します。 - -これらの手順は、ホストマシンが x86_64 であり、[ビルド手順](../development/build.md) に従ってネイティブビルドに必要なツール一式がすべてインストールされていることを前提とします。また、ホストが Ubuntu 22.04 であることを想定していますが、以下の手順は Ubuntu 20.04 でも動作するはずです。 - -ネイティブビルドに使用するツールをインストールすることに加えて、以下の追加パッケージをインストールする必要があります。 - -```bash -apt-get install binutils-s390x-linux-gnu libc6-dev-s390x-cross gcc-s390x-linux-gnu binfmt-support qemu-user-static -``` +これらの手順では、ホストマシンが Linux x86_64/ARM であり、[ビルド手順](../development/build.md) に基づいてネイティブビルドに必要なツールが一通りそろっていることを前提としています。また、ホストが Ubuntu 22.04 であることを想定していますが、以下の手順は Ubuntu 20.04 でも動作するはずです。 -Rust コードをクロスコンパイルする場合は、s390x 向けの Rust クロスコンパイルターゲットをインストールしてください。 +ネイティブビルドに使用するツールのインストールに加えて、以下の追加パッケージをインストールする必要があります。 ```bash +apt-get mold rustup target add s390x-unknown-linux-gnu ``` -s390x ビルドでは mold リンカーを使用します。[https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz](https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz) -からダウンロードし、`$PATH` に配置してください。 - s390x 向けにビルドするには: ```bash @@ -47,30 +32,37 @@ ninja ``` -## 実行 +## 実行 {#running} + +エミュレーションを行うには、s390x 用の QEMU user static バイナリが必要です。Ubuntu では次のコマンドでインストールできます。 + +```bash +apt-get install binfmt-support binutils-s390x-linux-gnu qemu-user-static +``` -ビルドが完了したら、たとえば次のようにバイナリを実行できます。 +ビルドが完了したら、たとえば次のようにバイナリを実行できます: ```bash -qemu-s390x-static -L /usr/s390x-linux-gnu ./clickhouse +qemu-s390x-static -L /usr/s390x-linux-gnu ./programs/clickhouse local --query "Select 2" +2 ``` -## デバッグ +## デバッグ {#debugging} LLDB をインストールします: ```bash -apt-get install lldb-15 +apt-get install lldb-21 ``` -s390x 実行ファイルをデバッグするには、QEMU をデバッグモードで実行して ClickHouse を起動します: +s390x 向け実行ファイルをデバッグするには、デバッグモードで QEMU を使用して ClickHouse を実行します。 ```bash qemu-s390x-static -g 31338 -L /usr/s390x-linux-gnu ./clickhouse ``` -別のシェルで LLDB を実行してアタッチし、`` と `` を、お使いの環境に対応する値に置き換えてください。 +別のシェルで LLDB を実行してプロセスにアタッチし、`<ClickHouse Parent Directory>` と `<build directory>` をお使いの環境に対応する値に置き換えてください。 ```bash lldb-15 @@ -102,16 +94,16 @@ Process 1 stopped ``` -## Visual Studio Code 連携 +## Visual Studio Code との連携 {#visual-studio-code-integration} -* ビジュアルデバッグには [CodeLLDB](https://github.com/vadimcn/vscode-lldb) 拡張機能が必要です。 -* [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md) を使用している場合、[Command Variable](https://github.com/rioj7/command-variable) 拡張機能を使うと動的な起動に役立ちます。 -* バックエンドとして使用する LLVM のインストール先を指すように設定してください(例: `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so"`)。 -* 起動前に ClickHouse 実行ファイルをデバッグモードで実行しておいてください(これを自動化する `preLaunchTask` を作成することも可能です)。 +- ビジュアルデバッグを行うには [CodeLLDB](https://github.com/vadimcn/vscode-lldb) 拡張機能が必要です。 +- [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md) を使用する場合、動的な起動設定には [Command Variable](https://github.com/rioj7/command-variable) 拡張機能が役立ちます。 +- バックエンドが使用している LLVM のインストール先に設定されていることを確認してください。例: `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so"` +- 起動前に ClickHouse の実行ファイルをデバッグモードで実行しておいてください(これを自動化する `preLaunchTask` を作成することも可能です)。 -### 設定例 +### 構成例 {#example-configurations} -#### cmake-variants.yaml +#### cmake-variants.yaml {#cmake-variantsyaml} ```yaml buildType: @@ -119,11 +111,11 @@ buildType: choices: debug: short: Debug - long: デバッグ情報を出力する + long: デバッグ情報を出力 buildType: Debug release: short: Release - long: 生成コードを最適化する + long: 生成コードを最適化 buildType: Release relwithdebinfo: short: RelWithDebInfo @@ -136,7 +128,7 @@ buildType: toolchain: default: default - description: ツールチェーンを選択する + description: ツールチェーンを選択 choices: default: short: x86_64 @@ -148,7 +140,8 @@ toolchain: CMAKE_TOOLCHAIN_FILE: cmake/linux/toolchain-s390x.cmake ``` -#### launch.json + +#### launch.json {#launchjson} ```json { @@ -166,29 +159,32 @@ toolchain: } ``` -#### settings.json -これにより、異なるビルドが `build` フォルダー内の別々のサブフォルダーに配置されるようになります。 +#### settings.json {#settingsjson} + +これにより、ビルドごとに `build` フォルダー内の別々のサブフォルダーに配置されます。 ```json { "cmake.buildDirectory": "${workspaceFolder}/build/${buildKitVendor}-${buildKitVersion}-${variant:toolchain}-${variant:buildType}", - "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so" + "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so" } ``` -#### run-debug.sh + +#### run-debug.sh {#run-debugsh} ```sh #! /bin/sh -echo 'デバッガセッションを開始します' +echo 'デバッガーセッションを開始します' cd $1 qemu-s390x-static -g 2159 -L /usr/s390x-linux-gnu $2 $3 $4 ``` -#### tasks.json -`programs/server/config.xml` の設定を使用して、バイナリと同じディレクトリ内にある `tmp` フォルダーでコンパイル済み実行ファイルを `server` モードで実行するタスクを定義します。 +#### tasks.json {#tasksjson} + +`server` モードでコンパイル済み実行ファイルを実行するタスクを定義します。バイナリと同じディレクトリ内の `tmp` フォルダを使用し、`programs/server/config.xml` にある設定を利用します。 ```json { diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-e2k.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-e2k.md index b859fdb2de3..a95b0486a8e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-e2k.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-e2k.md @@ -7,15 +7,11 @@ title: 'Linux での E2K 向けビルド' doc_type: 'guide' --- - - -# E2K 向け Linux 上でのビルド +# E2K 向け Linux 上でのビルド {#build-on-linux-for-e2k} ClickHouse は E2K (Elbrus-2000) を非常に実験的にサポートしており、boost、croaring、libunwind、zstd などの e2k 向けにカスタムビルドされたライブラリを使用した最小限の構成で、ネイティブモードでのみコンパイル可能です。 - - -## ClickHouse をビルドする +## ClickHouse をビルドする {#build-clickhouse} ビルドに必要な LLVM のバージョンは 20.1.8 以上が必要です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-osx.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-osx.md index 200c296a2fe..47b91ce316f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build-osx.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build-osx.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# macOS 向けに macOS 上で ClickHouse をビルドする方法 +# macOS 向けに macOS 上で ClickHouse をビルドする方法 {#how-to-build-clickhouse-on-macos-for-macos} :::info ClickHouse を自分でビルドする必要はありません! [Quick Start](https://clickhouse.com/#quick-start) に記載されている手順に従って、事前にビルド済みの ClickHouse をインストールできます。 @@ -22,7 +22,7 @@ ClickHouse は、macOS 10.15 (Catalina) 以降の macOS 上で、x86_64 (Intel) -## 前提条件をインストールする +## 前提条件をインストールする {#install-prerequisites} まず、共通の[前提条件ドキュメント](developer-instruction.md)を参照してください。 @@ -53,7 +53,7 @@ mkdir build export PATH=$(brew --prefix llvm)/bin:$PATH cmake -S . -B build cmake --build build -# 生成されるバイナリは次の場所に作成されます: build/programs/clickhouse +# 生成されるバイナリは次の場所に作成されます: build/programs/clickhouse {#the-resulting-binary-will-be-created-at-buildprogramsclickhouse} ``` :::note @@ -62,7 +62,7 @@ cmake --build build ::: -## 注意点 +## 注意点 {#caveats} `clickhouse-server` を実行する場合は、システムの `maxfiles` 変数の値を増やしておいてください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/build.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/build.md index 07da31b680c..c9bfb79fe3b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/build.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/build.md @@ -1,21 +1,19 @@ --- -description: 'Linux システム上で ClickHouse をソースからビルドするためのステップバイステップガイド' -sidebar_label: 'Linux でビルド' +description: 'Linux 上で ClickHouse をソースコードからビルドするためのステップバイステップガイド' +sidebar_label: 'Linux でのビルド' sidebar_position: 10 slug: /development/build title: 'Linux で ClickHouse をビルドする方法' doc_type: 'guide' --- +# Linux で ClickHouse をビルドする方法 {#how-to-build-clickhouse-on-linux} - -# Linux 上で ClickHouse をビルドする方法 - -:::info ClickHouse を自分でビルドする必要はありません! -[Quick Start](https://clickhouse.com/#quick-start) に記載されているように、事前にビルド済みの ClickHouse をインストールできます。 +:::info 自分で ClickHouse をビルドする必要はありません! +[Quick Start](https://clickhouse.com/#quick-start) に記載されている手順に従って、事前にビルド済みの ClickHouse をインストールできます。 ::: -ClickHouse は以下のプラットフォーム上でビルドできます: +ClickHouse は次のプラットフォーム上でビルドできます: - x86_64 - AArch64 @@ -23,24 +21,20 @@ ClickHouse は以下のプラットフォーム上でビルドできます: - s390/x(実験的) - RISC-V 64(実験的) - - ## 前提条件 {#assumptions} -このチュートリアルは Ubuntu Linux をベースとしていますが、適切な変更を加えれば他の Linux ディストリビューションでも動作するはずです。 -開発用として推奨される最小の Ubuntu バージョンは 24.04 LTS です。 - -このチュートリアルは、ClickHouse リポジトリとそのすべてのサブモジュールがローカル環境にチェックアウト済みであることを前提としています。 - +以下のチュートリアルは Ubuntu Linux を前提としていますが、適宜調整すれば他の Linux ディストリビューション上でも動作します。 +開発環境として推奨される Ubuntu の最低バージョンは 24.04 LTS です。 +このチュートリアルでは、ClickHouse のリポジトリとすべてのサブモジュールがローカルにチェックアウトされていることを前提としています。 -## 前提条件をインストールする +## 前提条件をインストールする {#install-prerequisites} -まず、共通の[前提条件のドキュメント](developer-instruction.md)を参照してください。 +まず、共通の[前提条件ドキュメント](developer-instruction.md)を参照してください。 ClickHouse はビルドに CMake と Ninja を使用します。 -ビルド時に既にコンパイル済みのオブジェクトファイルを再利用できるように、任意で ccache をインストールできます。 +ビルドで既にコンパイル済みのオブジェクトファイルを再利用できるように、必要に応じて ccache をインストールすることもできます。 ```bash sudo apt-get update @@ -48,34 +42,32 @@ sudo apt-get install build-essential git cmake ccache python3 ninja-build nasm y ``` -## Clang コンパイラをインストールする +## Clang コンパイラをインストールする {#install-the-clang-compiler} -Ubuntu/Debian に Clang をインストールするには、[こちら](https://apt.llvm.org/)から LLVM 提供の自動インストールスクリプトを使用してください。 +Ubuntu/Debian に Clang をインストールするには、[こちら](https://apt.llvm.org/) から LLVM の自動インストールスクリプトを使用してください。 ```bash sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ``` -他の Linux ディストリビューションの場合は、LLVM の[事前ビルド済みパッケージ](https://releases.llvm.org/download.html)をインストールできるか確認してください。 +他の Linux ディストリビューションについては、LLVM の[事前ビルド済みパッケージ](https://releases.llvm.org/download.html)をインストールできるか確認してください。 -2025 年 3 月時点では、Clang 19 以降が必要です。 -GCC やその他のコンパイラはサポートされていません。 +2025 年 3 月時点では、Clang 19 以上が必要です。 +GCC などの他のコンパイラはサポートされていません。 -## Rust コンパイラをインストールする(任意) {#install-the-rust-compiler-optional} +## Rust コンパイラのインストール(任意) {#install-the-rust-compiler-optional} :::note Rust は ClickHouse のオプションの依存関係です。 Rust がインストールされていない場合、ClickHouse の一部の機能はコンパイルされません。 ::: -まず、公式の [Rust ドキュメント](https://www.rust-lang.org/tools/install)の手順に従って `rustup` をインストールします。 - -C++ の依存関係と同様に、ClickHouse は vendoring(ベンダリング)を使用して、何がインストールされるかを正確に制御し、(`crates.io` レジストリのような)サードパーティサービスへの依存を避けています。 - -リリースモードでは、最新の rustup ツールチェーンであればどのバージョンでもこれらの依存関係は動作するはずですが、サニタイザを有効にする予定がある場合は、CI で使用しているものとまったく同じ `std` に一致するバージョン(そのための crate を vendoring しています)を使用する必要があります。 +まず、公式の [Rust ドキュメント](https://www.rust-lang.org/tools/install)に記載されている手順に従って `rustup` をインストールします。 +C++ の依存関係と同様に、ClickHouse はインストール内容を正確に制御し、`crates.io` レジストリのようなサードパーティサービスへの依存を避けるために vendoring を使用します。 +リリースモードでは、これらの依存関係に対して任意の最新の rustup ツールチェーンバージョンが動作するはずですが、サニタイザーを有効にする予定がある場合は、CI で使用しているものとまったく同じ `std` に対応するバージョンを使用する必要があります(そのためのクレートは vendoring 済みです)。 ```bash rustup toolchain install nightly-2025-07-07 @@ -83,16 +75,17 @@ rustup default nightly-2025-07-07 rustup component add rust-src ``` -## ClickHouse のビルド -すべてのビルド成果物を格納する専用ディレクトリ `build` を `ClickHouse` ディレクトリ内に作成することを推奨します。 +## ClickHouse をビルドする {#build-clickhouse} + +すべてのビルド成果物を格納するために、`ClickHouse` ディレクトリ内に専用の `build` ディレクトリを作成することを推奨します。 ```sh mkdir build cd build ``` -異なるビルドタイプごとに(例:`build_release`、`build_debug` など)、複数のディレクトリを用意できます。 +ビルドタイプごとに、(例: `build_release`、`build_debug` など)複数のディレクトリを用意できます。 オプション: 複数のコンパイラバージョンがインストールされている場合、使用するコンパイラを明示的に指定することもできます。 @@ -101,86 +94,87 @@ export CC=clang-19 export CXX=clang++-19 ``` -開発用途では、`debug` ビルドを推奨します。 -`release` ビルドと比較してコンパイラ最適化レベル(`-O`)が低く、よりデバッグしやすくなります。 -また、`LOGICAL_ERROR` 型の内部例外は、穏やかに処理されるのではなく、即座にクラッシュを引き起こします。 +開発用途には、`debug` ビルドの使用を推奨します。 +`release` ビルドと比較してコンパイラの最適化レベル(`-O`)が低く、デバッグ時の利便性が向上します。 +また、`LOGICAL_ERROR` 型の内部例外は、エラーからの復旧を試みず、即座にクラッシュします。 ```sh cmake -D CMAKE_BUILD_TYPE=Debug .. ``` :::note -gdb などのデバッガを使用したい場合は、上記のコマンドに `-D DEBUG_O_LEVEL="0"` を追加して、すべてのコンパイラ最適化を無効にしてください。コンパイラ最適化により、gdb が変数を表示・参照できなくなる場合があります。 +gdb のようなデバッガを使用したい場合は、上記のコマンドに `-D DEBUG_O_LEVEL="0"` を追加して、すべてのコンパイラ最適化を無効にしてください。最適化が有効なままだと、gdb による変数の表示やアクセスの妨げになる可能性があります。 ::: -ビルドするには ninja を実行します。 +ビルドするには `ninja` を実行します。 ```sh ninja clickhouse ``` -すべてのバイナリ(ユーティリティとテスト)をビルドするには、引数を付けずに `ninja` を実行します: +すべてのバイナリ(ユーティリティおよびテスト)をビルドするには、引数を付けずに `ninja` を実行します。 ```sh ninja ``` -並列ビルドジョブの数は、パラメーター `-j` で制御できます。 +パラメータ `-j` を使用して、並列ビルドジョブの数を指定できます。 ```sh ninja -j 1 clickhouse-server clickhouse-client ``` :::tip -CMake では、上記のコマンドを簡略化するためのショートカットが用意されています。 +CMake には、上記のコマンドを簡略化するショートカットが用意されています: ```sh -cmake -S . -B build # ビルドを構成、リポジトリのトップレベルディレクトリから実行 -cmake --build build # コンパイル +cmake -S . -B build # ビルドを構成します。リポジトリのトップレベルディレクトリから実行してください +cmake --build build # コンパイルします ``` ::: -## ClickHouse 実行ファイルの実行 +## ClickHouse 実行ファイルの起動 {#running-the-clickhouse-executable} -ビルドが正常に完了すると、実行ファイルは `ClickHouse//programs/` に生成されます。 +ビルドが正常に完了すると、`ClickHouse//programs/` に実行ファイルが生成されます。 -ClickHouse サーバーは、カレントディレクトリで設定ファイル `config.xml` を探します。 -または、コマンドラインで `-C` オプションを指定して設定ファイルを明示的に指定できます。 +ClickHouse サーバーは、カレントディレクトリ内の `config.xml` という設定ファイルを探します。 +代わりに、コマンドラインで `-C` オプションを指定して使用する設定ファイルを明示的に指定することもできます。 -`clickhouse-client` を使用して ClickHouse サーバーに接続するには、別のターミナルを開き、`ClickHouse/build/programs/` に移動して `./clickhouse client` を実行します。 +`clickhouse-client` で ClickHouse サーバーに接続するには、別のターミナルを開き、`ClickHouse/build/programs/` に移動してから `./clickhouse client` を実行します。 -macOS または FreeBSD で `Connection refused` というメッセージが表示される場合は、ホストアドレスとして 127.0.0.1 を指定してください。 +macOS または FreeBSD で `Connection refused` というメッセージが表示される場合は、ホストアドレスを 127.0.0.1 に指定してみてください。 ```bash clickhouse client --host 127.0.0.1 ``` -## 高度なオプション +## 高度なオプション {#advanced-options} -### 最小構成でのビルド +### 最小構成でのビルド {#minimal-build} -サードパーティ製ライブラリが提供する機能が不要な場合は、ビルド時間をさらに短縮できます。 +サードパーティ製ライブラリが提供する機能が不要な場合は、ビルドをさらに高速化できます。 ```sh cmake -DENABLE_LIBRARIES=OFF ``` -問題が発生した場合は、すべて自己解決していただくことになります... +問題が発生した場合は自己対応となります… -Rust の利用にはインターネット接続が必要です。Rust サポートを無効にするには、次のようにします: +Rust にはインターネット接続が必要です。Rust サポートを無効化するには: ```sh cmake -DENABLE_RUST=OFF ``` -### ClickHouse 実行ファイルの実行 -システムにインストールされている本番環境版の ClickHouse バイナリを、コンパイル済みの ClickHouse バイナリに置き換えることができます。 -そのためには、公式ウェブサイトの手順に従ってマシンに ClickHouse をインストールしてください。 -次に、以下を実行します: +### ClickHouse バイナリの実行 {#running-the-clickhouse-executable-1} + +システムにインストールされている本番用の ClickHouse バイナリを、コンパイル済みの ClickHouse バイナリに置き換えることができます。 +そのためには、まず公式サイトの手順に従ってマシンに ClickHouse をインストールします。 +次に、以下を実行します。 ```bash sudo service clickhouse-server stop @@ -190,16 +184,17 @@ sudo service clickhouse-server start `clickhouse-client`、`clickhouse-server` などは、共通の `clickhouse` バイナリへのシンボリックリンクであることに注意してください。 -システムにインストールされている ClickHouse パッケージの設定ファイルを使って、独自にビルドした ClickHouse バイナリを実行することもできます。 +システムにインストールされている ClickHouse パッケージに含まれる設定ファイルを使用して、独自にビルドした ClickHouse バイナリを実行することもできます。 ```bash sudo service clickhouse-server stop sudo -u clickhouse ClickHouse/build/programs/clickhouse server --config-file /etc/clickhouse-server/config.xml ``` -### 任意の Linux 上でのビルド -OpenSUSE Tumbleweed に必要なパッケージをインストールします: +### 任意の Linux 環境でのビルド {#building-on-any-linux} + +openSUSE Tumbleweed に必要な前提パッケージをインストールします: ```bash sudo zypper install git cmake ninja clang-c++ python lld nasm yasm gawk @@ -220,19 +215,20 @@ cmake -S . -B build cmake --build build ``` -### Docker でのビルド -CI と同様の環境で、任意のビルドを次のコマンドを使ってローカルで実行できます: +### Docker でのビルド {#building-in-docker} + +次のコマンドを使用して、CI と同様の環境で任意のビルドをローカル環境で実行できます。 ```bash -python -m ci.praktika "BUILD_JOB_NAME" +python -m ci.praktika run "BUILD_JOB_NAME" ``` -ここで BUILD_JOB_NAME は CI レポートに表示されるジョブ名であり、例えば "Build (arm_release)" や "Build (amd_debug)" などです。 +ここで BUILD_JOB_NAME は、CI レポートに表示されるジョブ名であり、例として "Build (arm_release)" や "Build (amd_debug)" などがあります。 -このコマンドは、必要な依存関係がすべて含まれた適切な Docker イメージ `clickhouse/binary-builder` を取得して、 +このコマンドは、必要な依存関係をすべて含む適切な Docker イメージ `clickhouse/binary-builder` を取得して、 その中でビルドスクリプト `./ci/jobs/build_clickhouse.py` を実行します。 ビルド成果物は `./ci/tmp/` に配置されます。 -これは AMD と ARM の両方のアーキテクチャで動作し、Docker 以外に追加の依存関係は必要ありません。 +これは AMD と ARM の両方のアーキテクチャで動作し、`requests` モジュールが利用可能な Python と Docker 以外に、追加の依存関係は必要ありません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md index 0cd5b7f0f56..090a4f15541 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# DEFLATE_QPL を使用して ClickHouse をビルドする +# DEFLATE_QPL を使用して ClickHouse をビルドする {#build-clickhouse-with-deflate_qpl} - ホストマシンが QPL の要求する[前提条件](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#prerequisites)を満たしていることを確認してください - `cmake` ビルド時には `deflate_qpl` はデフォルトで有効になっています。誤って設定を変更してしまった場合は、ビルドフラグ `ENABLE_QPL=1` になっていることを必ず再確認してください @@ -18,7 +18,7 @@ doc_type: 'guide' -# DEFLATE_QPL を使ってベンチマークを実行する +# DEFLATE_QPL を使ってベンチマークを実行する {#run-benchmark-with-deflate_qpl} @@ -35,7 +35,7 @@ doc_type: 'guide' -## スター・スキーマ向けベンチマークを自動実行する: +## スター・スキーマ向けベンチマークを自動実行する: {#run-benchmark-automatically-for-star-schema} ```bash $ cd ./benchmark_sample/client_scripts @@ -53,7 +53,7 @@ $ sh run_ssb.sh -## 環境 +## 環境 {#environment} * CPU: Sapphire Rapid * OS 要件については [System Requirements for QPL](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#system-requirements) を参照してください @@ -81,7 +81,7 @@ $ accel-config list | grep -P 'iax|state' 何も出力されない場合は、IAA の準備がまだ整っていないことを意味します。IAA のセットアップを再度確認してください。 -## 未加工データを生成する +## 未加工データを生成する {#generate-raw-data} ```bash $ cd ./benchmark_sample @@ -94,7 +94,7 @@ $ mkdir rawdata_dir && cd rawdata_dir `*.tbl` のようなファイルは、`./benchmark_sample/rawdata_dir/ssb-dbgen` 配下に出力されます: -## データベースのセットアップ +## データベースのセットアップ {#database-setup} LZ4 コーデックを使用したデータベースのセットアップ @@ -164,7 +164,7 @@ SELECT count() FROM lineorder_flat これは IAA デバイスが使用可能な状態になっていないことを意味します。IAA のセットアップをもう一度確認する必要があります。 -## 単一インスタンスでのベンチマーク +## 単一インスタンスでのベンチマーク {#benchmark-with-single-instance} * ベンチマークを開始する前に、C6 を無効化し、CPU周波数ガバナーを `performance` に設定してください @@ -218,7 +218,7 @@ zstd.log QPS を中心に確認します。キーワード `QPS_Final` を検索し、統計情報を収集してください -## マルチインスタンスでのベンチマーク +## マルチインスタンスでのベンチマーク {#benchmark-with-multi-instances} * メモリボトルネックがスレッド数の増加に与える影響を抑えるため、マルチインスタンス構成でベンチマークを実行することを推奨します。 * マルチインスタンスとは、複数(2 または 4)台のサーバーがそれぞれ個別のクライアントに接続されている構成を指します。 @@ -350,7 +350,7 @@ zstd_2insts.log レビュー用の最終レポートには、2 インスタンス構成のベンチマークデータを採用することを推奨します。 -## ヒント +## ヒント {#tips} 新しい ClickHouse サーバーを起動する前には毎回、バックグラウンドで動作している ClickHouse プロセスがないことを必ず確認し、残っている古いプロセスがあれば終了させてください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/continuous-integration.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/continuous-integration.md index 73f5809c047..b705033c126 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/continuous-integration.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/continuous-integration.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# 継続的インテグレーション (CI) +# 継続的インテグレーション (CI) {#continuous-integration-ci} プルリクエストを作成すると、ClickHouse の [継続的インテグレーション (CI) システム](tests.md#test-automation) によって、あなたのコードに対していくつかの自動チェックが実行されます。 これは、リポジトリのメンテナー (ClickHouse チームのメンバー) があなたのコードを確認し、プルリクエストに `can be tested` ラベルを追加した後に行われます。 @@ -75,26 +75,26 @@ ClickHouse server および keeper 用の Docker イメージをビルドし、 -## スタイルチェック +## スタイルチェック {#style-check} コードベースに対してさまざまなスタイルチェックを実行します。 Style Check ジョブで実行される基本的なチェックは次のとおりです。 -##### cpp +##### cpp {#cpp} [`ci/jobs/scripts/check_style/check_cpp.sh`](https://github.com/ClickHouse/ClickHouse/blob/master/ci/jobs/scripts/check_style/check_cpp.sh) スクリプト(ローカルでも実行可能)を使用して、単純な正規表現ベースのコードスタイルチェックを行います。\ 失敗した場合は、[コードスタイルガイド](style.md) に従ってスタイル上の問題を修正してください。 -##### codespell, aspell +##### codespell, aspell {#codespell} 文法ミスや綴りの誤りをチェックします。 -##### mypy +##### mypy {#mypy} Python コードに対して静的型チェックを実行します。 -### スタイルチェックジョブをローカルで実行する +### スタイルチェックジョブをローカルで実行する {#running-style-check-locally} *Style Check* ジョブ全体は、Docker コンテナ内でローカルに実行できます: @@ -112,14 +112,14 @@ python -m ci.praktika run "Style check" --test cpp Python 3 と Docker 以外に必要な依存関係はありません。 -## Fast test +## Fast test {#fast-test} 通常、これは PR に対して最初に実行されるチェックです。 ClickHouse をビルドし、一部を除くほとんどの [stateless functional tests](tests.md#functional-tests) を実行します。 これが失敗すると、問題が修正されるまで後続のチェックは実行されません。 どのテストが失敗したかを確認するにはレポートを参照し、その後[こちら](/development/tests#running-a-test-locally)で説明されているとおりにローカルでその失敗を再現します。 -#### Fast test をローカルで実行する: +#### Fast test をローカルで実行する: {#running-fast-test-locally} ```sh python -m ci.praktika run "Fast test" [--test テスト名] @@ -129,11 +129,11 @@ python -m ci.praktika run "Fast test" [--test テスト名] Python 3 と Docker 以外の依存関係は必要ありません。 -## ビルドチェック +## ビルドチェック {#build-check} 以降の手順で利用するために、さまざまな構成で ClickHouse をビルドします。 -### ローカルでのビルドの実行 +### ローカルでのビルドの実行 {#running-builds-locally} ビルドは、次のコマンドを使用して CI 環境に近い形でローカル実行できます。 @@ -143,7 +143,7 @@ python -m ci.praktika run "<ビルドジョブ名>" Python 3 と Docker 以外に必要な依存関係はありません。 -#### 利用可能なビルドジョブ +#### 利用可能なビルドジョブ {#available-build-jobs} ビルドジョブ名は CI レポートに表示されるものと完全に同一です。 @@ -181,7 +181,7 @@ Python 3 と Docker 以外に必要な依存関係はありません。 **注意:** クロスコンパイルを使用する「Other Architectures」カテゴリ以外のビルドでは、指定した `BUILD_JOB_NAME` どおりのビルドを生成するために、ローカルマシンのアーキテクチャがビルドタイプと一致している必要があります。 -#### 例 +#### 例 {#example-run-local} ローカルでデバッグビルドを実行するには: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/contrib.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/contrib.md index e8ef83d04ad..552b8ed75d5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/contrib.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/contrib.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# サードパーティライブラリ +# サードパーティライブラリ {#third-party-libraries} ClickHouse は、さまざまな目的でサードパーティライブラリを利用します。たとえば、他のデータベースへの接続、ディスクへの保存/ディスクからの読み込み時のデータのデコード/エンコード、あるいは特定の SQL 関数の実装などです。 対象システムにインストールされているライブラリに依存しないようにするため、各サードパーティライブラリは Git サブモジュールとして ClickHouse のソースツリーにインポートされ、ClickHouse とともにコンパイルおよびリンクされます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/developer-instruction.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/developer-instruction.md index 2538b5bb412..3ba6214e8df 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/developer-instruction.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/developer-instruction.md @@ -1,62 +1,58 @@ --- -description: 'ClickHouse 開発用の前提条件とセットアップ手順' +description: 'ClickHouse 開発のための前提条件とセットアップ手順' sidebar_label: '前提条件' sidebar_position: 5 slug: /development/developer-instruction -title: '開発者向けの前提条件' +title: '開発者向け前提条件' doc_type: 'guide' --- - - -# 前提条件 +# 前提条件 {#prerequisites} ClickHouse は Linux、FreeBSD、macOS 上でビルドできます。 -Windows を使用している場合でも、Ubuntu を実行している [VirtualBox](https://www.virtualbox.org/) などの Linux 仮想マシン上で ClickHouse をビルドできます。 - +Windows を使用している場合でも、Linux を実行している仮想マシン上で ClickHouse をビルドできます。たとえば、Ubuntu を実行する [VirtualBox](https://www.virtualbox.org/) などです。 +## GitHub にリポジトリを作成する {#create-a-repository-on-github} -## GitHub にリポジトリを作成する +ClickHouse の開発を開始するには、[GitHub](https://www.github.com/) アカウントが必要です。 +まだ SSH キーをお持ちでない場合はローカルで SSH キーを生成し、その公開鍵を GitHub にアップロードしてください。これはパッチをコントリビュートするための前提条件です。 -ClickHouse 向けの開発を始めるには、[GitHub](https://www.github.com/) アカウントが必要です。 -まだ SSH キーを持っていない場合は、ローカルで SSH キーを作成し、その公開鍵を GitHub にアップロードしてください。これはパッチを貢献するための前提条件です。 +次に、右上隅にある「fork」ボタンをクリックして、ご自身のアカウントに [ClickHouse リポジトリ](https://github.com/ClickHouse/ClickHouse/) の fork を作成します。 -次に、右上隅の「fork」ボタンをクリックして、ご自身のアカウントの下に [ClickHouse リポジトリ](https://github.com/ClickHouse/ClickHouse/) をフォークします。 +変更をコントリビュートするには(例: issue の修正や新機能の追加など)、まず fork 先のブランチに変更を commit し、その変更をメインリポジトリに反映する「Pull Request」を作成します。 -Issue の修正や機能追加などの変更を貢献するには、まずフォークしたリポジトリ内のブランチに変更をコミットし、その変更をメインのリポジトリに対して反映する「Pull Request」を作成します。 - -Git リポジトリを操作するには、Git をインストールしてください。たとえば、Ubuntu では次のコマンドを実行します。 +Git リポジトリを扱うために、Git をインストールしてください。たとえば Ubuntu では、次のコマンドを実行します。 ```sh sudo apt update sudo apt install git ``` -Git のチートシートは[こちら](https://education.github.com/git-cheat-sheet-education.pdf)から確認できます。 -Git の詳細なマニュアルは[こちら](https://git-scm.com/book/en/v2)を参照してください。 +Git チートシートは [こちら](https://education.github.com/git-cheat-sheet-education.pdf) から入手できます。 +より詳細な Git マニュアルは [こちら](https://git-scm.com/book/en/v2) にあります。 -## リポジトリを開発マシンにクローンする +## リポジトリを開発マシンにクローンする {#clone-the-repository-to-your-development-machine} -まず、作業用マシンにソースファイルを取得します。具体的には、リポジトリをクローンします。 +まず、作業用マシンにソースファイルを取得するため、リポジトリをクローンします。 ```sh -git clone git@github.com:your_github_username/ClickHouse.git # プレースホルダーをご自身のGitHubユーザー名に置き換えてください +git clone git@github.com:your_github_username/ClickHouse.git # プレースホルダーを自分のGitHubユーザー名に置き換えてください cd ClickHouse ``` -このコマンドは、ソースコード、テスト、その他のファイルを含むディレクトリ `ClickHouse/` を作成します。 -URL の後ろに任意のディレクトリ名を指定してチェックアウト先を変更できますが、このパスに空白(スペース)を含めないことが重要です。空白が含まれていると、その後のビルドが失敗する可能性があります。 +このコマンドは、ソースコード、テスト、およびその他のファイルを含むディレクトリ `ClickHouse/` を作成します。 +URL の後にチェックアウト先のカスタムディレクトリを指定できますが、このパスに空白文字を含めないことが重要です。空白を含むと、後続のビルドが失敗する可能性があります。 ClickHouse の Git リポジトリは、サードパーティライブラリを取得するためにサブモジュールを使用しています。 サブモジュールはデフォルトではチェックアウトされません。 -次のいずれかの方法を使用できます。 +次のいずれかを実行できます。 -* `git clone` を `--recurse-submodules` オプション付きで実行する。 +* オプション `--recurse-submodules` を付けて `git clone` を実行する。 -* `git clone` を `--recurse-submodules` なしで実行した場合、`git submodule update --init --jobs ` を実行して、すべてのサブモジュールを明示的にチェックアウトする(`` は、たとえば `12` に設定してダウンロードを並列化できます)。 +* `git clone` を `--recurse-submodules` なしで実行した場合、`git submodule update --init --jobs ` を実行して、すべてのサブモジュールを明示的にチェックアウトする(`` はダウンロードを並列化するために、例えば `12` に設定できます)。 -* `git clone` を `--recurse-submodules` なしで実行し、不要なファイルや履歴をサブモジュールから省いてディスク使用量を削減するために [sparse](https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/) および [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) なサブモジュールチェックアウトを利用したい場合は、`./contrib/update-submodules.sh` を実行します。この方法は CI では使用されていますが、サブモジュールの操作が不便になり遅くもなるため、ローカル開発には推奨されません。 +* `git clone` を `--recurse-submodules` なしで実行し、サブモジュールの履歴を省略して容量を節約するために [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) なサブモジュールチェックアウトを使用したい場合は、`./contrib/update-submodules.sh` を実行します。この方法は CI では使用されていますが、サブモジュールの操作が不便になり遅くなるため、ローカル開発には推奨されません。 Git サブモジュールのステータスを確認するには、`git submodule status` を実行します。 @@ -70,92 +66,86 @@ fatal: Could not read from remote repository. およびリポジトリが存在することを確認してください。 ``` -GitHub に接続するための SSH キーが見つかりません。 +GitHub に SSH 接続するための SSH キーが存在しません。 これらのキーは通常 `~/.ssh` に保存されています。 -SSH キーを利用できるようにするには、GitHub の設定画面からアップロードする必要があります。 +SSH キーを認証に利用するには、GitHub の設定画面からアップロードしておく必要があります。 -また、HTTPS 経由でリポジトリをクローンすることもできます。 +HTTPS 経由でリポジトリをクローンすることもできます: ```sh git clone https://github.com/ClickHouse/ClickHouse.git ``` -しかし、この方法だけでは変更内容をサーバーにプッシュすることはできません。 -一時的にこのまま利用し、後から SSH キーを追加して、`git remote` コマンドでリポジトリのリモートアドレスを置き換えることもできます。 +ただし、この設定のままでは変更内容をサーバーに送信(プッシュ)することはできません。 +一時的にはこのまま使用し、後から SSH キーを追加して、`git remote` コマンドでリポジトリのリモートアドレスを置き換えることもできます。 -また、オリジナルの ClickHouse リポジトリのアドレスをローカルリポジトリに追加して、そこから更新を取得することもできます。 +また、元の ClickHouse リポジトリのアドレスをローカルリポジトリに追加して、そこから更新を取得することもできます。 ```sh git remote add upstream git@github.com:ClickHouse/ClickHouse.git ``` -このコマンドの実行に成功すると、`git pull upstream master` を実行して、メインの ClickHouse リポジトリから更新を取得できるようになります。 +このコマンドの実行に成功すると、`git pull upstream master` を実行することで、ClickHouse のメインリポジトリから更新を取り込めるようになります。 :::tip -`git push` をそのまま使用しないでください。誤って間違ったリモートやブランチに push してしまう可能性があります。 +`git push` をそのまま実行しないでください。誤ったリモートやブランチに push してしまう可能性があります。 `git push origin my_branch_name` のように、リモート名とブランチ名を明示的に指定することを推奨します。 ::: -## コードの記述 {#writing-code} +## コードを書く {#writing-code} -以下に、ClickHouse 用のコードを書く際に役立つクイックリンクを示します。 +以下は、ClickHouse 向けのコードを記述する際に役立つクイックリンクです。 - [ClickHouse のアーキテクチャ](/development/architecture/). - [コードスタイルガイド](/development/style/). - [サードパーティライブラリ](/development/contrib#adding-and-maintaining-third-party-libraries) -- [テストの記述](/development/tests/) -- [未解決の Issue](https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aopen+is%3Aissue+label%3A%22easy+task%22) +- [テストの作成](/development/tests/) +- [未解決の課題](https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aopen+is%3Aissue+label%3A%22easy+task%22) ### IDE {#ide} -[Visual Studio Code](https://code.visualstudio.com/) と [Neovim](https://neovim.io/) は、ClickHouse の開発で実績のある 2 つの選択肢です。VS Code を使用する場合は、IntelliSense の代わりに、はるかに高性能な [clangd 拡張機能](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) を使用することを推奨します。 - -[CLion](https://www.jetbrains.com/clion/) も優れた選択肢です。ただし、ClickHouse のような大規模なプロジェクトでは動作が遅くなる場合があります。CLion を使用する際は、次の点に留意してください。 +[Visual Studio Code](https://code.visualstudio.com/) と [Neovim](https://neovim.io/) は、ClickHouse の開発環境としてこれまで実績のある 2 つの選択肢です。VS Code を使用する場合は、パフォーマンスが大幅に高いため、IntelliSense の代わりに [clangd extension](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) を使用することを推奨します。 -- CLion は自動的に `build` ディレクトリを作成し、ビルドタイプとして `debug` を自動選択します -- ユーザーがインストールしたものではなく、CLion 内で定義されている CMake のバージョンを使用します -- CLion は、ビルドタスクの実行に `ninja` ではなく `make` を使用します(これは通常の動作です) - -使用できるその他の IDE としては、[Sublime Text](https://www.sublimetext.com/)、[Qt Creator](https://www.qt.io/product/development-tools)、[Kate](https://kate-editor.org/) などがあります。 +[CLion](https://www.jetbrains.com/clion/) も優れた選択肢です。ただし、ClickHouse のような大規模なプロジェクトでは動作が遅くなる場合があります。CLion を使用する際の注意点は次のとおりです。 +- CLion は独自に `build` パスを作成し、ビルドタイプとして自動的に `debug` を選択します +- 使用される CMake のバージョンは、ローカルにインストールしたものではなく、CLion 内で定義されているものです +- CLion はビルドタスクの実行に `ninja` ではなく `make` を使用します(これは通常の動作です) +使用可能なその他の IDE としては、[Sublime Text](https://www.sublimetext.com/)、[Qt Creator](https://www.qt.io/product/development-tools)、[Kate](https://kate-editor.org/) があります。 ## プルリクエストを作成する {#create-a-pull-request} -GitHub の UI で自分の fork リポジトリに移動します。 +GitHub の UI で自分の fork リポジトリを開きます。 ブランチで開発している場合は、そのブランチを選択する必要があります。 -画面上に「Pull request」ボタンが表示されます。 -つまり、これは「自分の変更をメインリポジトリに取り込んでもらうようにリクエストを作成する」という意味です。 +画面上に「Pull request」ボタンが表示されているはずです。 +これは要するに、「自分の変更をメインリポジトリに取り込んでもらうためのリクエストを作成する」という意味です。 -作業がまだ完了していなくても、プルリクエストは作成できます。 -この場合、タイトルの先頭に「WIP」(work in progress)を付けてください。後で変更してもかまいません。 -これは、変更内容の共同レビューやディスカッション、および利用可能なすべてのテストを実行するのに役立ちます。 -変更内容について簡潔な説明を記載することが重要です。これは後でリリースの変更履歴(changelog)を生成する際に使用されます。 +作業がまだ完了していなくても、プルリクエストを作成することができます。 +この場合、タイトルの先頭に「WIP」(work in progress)という単語を付けてください。これは後で変更可能です。 +これは、変更内容の共同レビューやディスカッション、および利用可能なすべてのテストの実行に役立ちます。 +変更内容について簡潔な説明を記載することが重要です。これは後にリリースの変更履歴を生成する際に使用されます。 -ClickHouse の担当者があなたの PR に「can be tested」タグを付けると、テストが開始されます。 +ClickHouse のメンバーがあなたのプルリクエスト(PR)に「can be tested」というタグを付けると、テストが開始されます。 最初のいくつかのチェック結果(例: コードスタイル)は数分以内に返ってきます。 ビルドチェックの結果は 30 分以内に届きます。 -主要なテストセットの結果は 1 時間以内に報告されます。 - -システムは、あなたのプルリクエスト専用の ClickHouse バイナリビルドを用意します。 -これらのビルドを取得するには、チェック一覧の「Builds」項目の横にある「Details」リンクをクリックしてください。 -そこには、ClickHouse のビルド済み .deb パッケージへの直接リンクがあり、必要であれば本番サーバーにデプロイすることもできます。 - +メインのテストセットは 1 時間以内に完了します。 +システムは、あなたのプルリクエスト専用の ClickHouse バイナリビルドを準備します。 +これらのビルドを取得するには、チェック一覧で「Builds」項目の横にある「Details」リンクをクリックします。 +そこから ClickHouse のビルド済み .deb パッケージへの直接リンクにアクセスでき、それらは(問題なければ)本番サーバーにそのままデプロイすることもできます。 ## ドキュメントを作成する {#write-documentation} -新機能を追加するプルリクエストには、必ず適切なドキュメントを含めてください。 -ドキュメントの変更をプレビューしたい場合は、ドキュメントページをローカルでビルドする手順が、README.md ファイル内の[こちら](https://github.com/ClickHouse/clickhouse-docs)に記載されています。 -ClickHouse に新しい関数を追加する際は、以下のテンプレートをガイドとして使用できます。 - - +新機能を追加するプルリクエストには、必ずそれに対応するドキュメントを含めてください。 +ドキュメントの変更内容をプレビューしたい場合は、ドキュメントページをローカルでビルドする手順が README.md ファイル内の[こちら](https://github.com/ClickHouse/clickhouse-docs)に記載されています。 +ClickHouse に新しい関数を追加する際は、以下のテンプレートをガイドとして利用できます。 ````markdown -# newFunctionName +# newFunctionName {#newfunctionname} -関数の簡単な説明をここに記載します。この関数が何を行うか、および典型的な使用例を簡潔に説明してください。 +関数の簡潔な説明をここに記載します。関数の機能と典型的な使用例を簡潔に説明してください。 **構文** @@ -165,17 +155,17 @@ newFunctionName(arg1, arg2[, arg3]) **引数** -- `arg1` — 引数の説明。 [DataType](../data-types/float.md) -- `arg2` — 引数の説明。 [DataType](../data-types/float.md) -- `arg3` — オプション引数の説明(省略可能)。 [DataType](../data-types/float.md) +- `arg1` — 引数の説明。[DataType](../data-types/float.md) +- `arg2` — 引数の説明。[DataType](../data-types/float.md) +- `arg3` — オプション引数の説明(省略可能)。[DataType](../data-types/float.md) **実装の詳細** -関連する場合、実装の詳細についての説明を記載します。 +関連する実装の詳細についての説明。 **戻り値** -- {関数が返す内容をここに挿入}を返します。 [DataType](../data-types/float.md) +- {関数が返す内容をここに記載}を返します。[DataType](../data-types/float.md) **例** @@ -185,7 +175,7 @@ newFunctionName(arg1, arg2[, arg3]) SELECT 'write your example query here'; \``` -結果: +レスポンス: \```response ┌───────────────────────────────────┐ @@ -195,12 +185,12 @@ SELECT 'write your example query here'; ```` -## テストデータの使用 +## テストデータの利用 {#using-test-data} -ClickHouse の開発にあたっては、実際の利用状況に近いデータセットを読み込む必要があることがよくあります。 -これは特にパフォーマンス テストにおいて重要です。 -そのために、Web アナリティクスの匿名化データを特別に用意しています。 -これを利用するには、追加で約 3 GB の空きディスク容量が必要です。 +ClickHouse の開発では、実運用に近いデータセットを読み込む必要があります。 +これは特にパフォーマンステストにおいて重要です。 +そのために、匿名化されたウェブアナリティクスのデータセットを特別に用意しています。 +これには、追加で約 3GB の空きディスク容量が必要です。 ```sh sudo apt install wget xz-utils @@ -214,24 +204,20 @@ ClickHouse の開発にあたっては、実際の利用状況に近いデータ clickhouse-client ``` -clickhouse-client で実行します: +clickhouse-client で実行: + ```sql CREATE DATABASE IF NOT EXISTS test; -``` - CREATE TABLE test.hits ( WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree PARTITION BY toYYYYMM(EventDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID), EventTime); +CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); +``` - -CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); - -```` - -データをインポートします: +データをインポートする: ```bash clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.hits FORMAT TSV" < hits_v1.tsv clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.visits FORMAT TSV" < visits_v1.tsv -```` +``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/index.md deleted file mode 100644 index 4ad8d48567c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/index.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '開発と貢献のインデックスページ' -slug: /development/ -title: '開発と貢献' -doc_type: 'landing-page' ---- - -このセクションには、次のページが含まれます。 - -{/* 以下の目次は、YAML フロントマターの title、description、slug フィールドから、 - 次のスクリプトによって自動生成されています: - - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - - 誤りを見つけた場合や説明を改善したい場合は、これらのファイル自体を - 直接編集してください。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ------------------------------------------------------------------------------------------- | --------------------------------------------------------- | -| [Developer Prerequisites](/development/developer-instruction) | ClickHouse 開発の前提条件とセットアップ手順 | -| [How to Build ClickHouse on Linux](/development/build) | Linux システム上で ClickHouse をソースからビルドするためのステップバイステップガイド | -| [Build on macOS for macOS](/development/build-osx) | macOS システム上で ClickHouse をソースからビルドするためのガイド | -| [Build on Linux for macOS](/development/build-cross-osx) | Linux 上で macOS 向けに ClickHouse をクロスコンパイルするためのガイド | -| [How to Build ClickHouse on Linux for AARCH64](/development/build-cross-arm) | AARCH64 アーキテクチャ向けに ClickHouse をソースからビルドするためのガイド | -| [How to Build ClickHouse on Linux for RISC-V 64](/development/build-cross-riscv) | RISC-V 64 アーキテクチャ向けに ClickHouse をソースからビルドするためのガイド | -| [Build on Linux for s390x (zLinux)](/development/build-cross-s390x) | s390x アーキテクチャ向けに ClickHouse をソースからビルドするためのガイド | -| [Build on Linux for E2K](/development/build-e2k) | E2K アーキテクチャ向けに ClickHouse をソースからビルドするためのガイド | -| [Build on Linux for LoongArch64](/development/build-cross-loongarch) | LoongArch64 アーキテクチャ向けに ClickHouse をソースからビルドするためのガイド | -| [Testing ClickHouse](/development/tests) | ClickHouse のテスト実行およびテストスイートの実行方法に関するガイド | -| [Architecture Overview](/development/architecture) | ClickHouse のアーキテクチャとカラム指向設計に関する包括的な概要 | -| [Continuous Integration (CI)](/development/continuous-integration) | ClickHouse の継続的インテグレーション (CI) システムの概要 | -| [Third-Party Libraries](/development/contrib) | ClickHouse におけるサードパーティライブラリの利用方法、およびライブラリの追加と保守方法を説明するページ | -| [C++ Style Guide](/development/style) | ClickHouse の C++ 開発におけるコーディングスタイルガイドライン | -| [Build Clickhouse with DEFLATE_QPL](/development/building_and_benchmarking_deflate_qpl) | ClickHouse を DEFLATE_QPL コーデック付きでビルドし、ベンチマークを実行する方法 | -| [Integrating Rust Libraries](/development/integrating_rust_libraries) | Rust ライブラリを ClickHouse に統合するためのガイド | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md index 4e585a1a138..257f428d292 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md @@ -6,7 +6,7 @@ title: 'Rust ライブラリの統合' doc_type: 'guide' --- -# Rust ライブラリ +# Rust ライブラリ {#rust-libraries} Rust ライブラリの統合については、BLAKE3 ハッシュ関数の統合を例に説明します。 @@ -80,7 +80,6 @@ pub unsafe extern "C" fn blake3_apply_shim( また、すべての C 互換の項目に対して属性 #[no_mangle] と `extern "C"` を使用する必要があります。これらがないと、ライブラリが正しくコンパイルされず、cbindgen によるヘッダーの自動生成が実行されません。 - これらすべての手順が完了したら、小さなプロジェクトでライブラリをテストして、互換性やヘッダー生成に関する問題をすべて洗い出してください。ヘッダー生成中に問題が発生した場合は、`cbindgen.toml` ファイルで設定を行ってみてください(テンプレートはここで参照できます: [https://github.com/eqrion/cbindgen/blob/master/template.toml](https://github.com/eqrion/cbindgen/blob/master/template.toml))。 BLAKE3 を統合した際に発生した問題についても触れておきます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/style.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/style.md index 6a4630f655c..83161cf0342 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/style.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/style.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# C++ スタイルガイド +# C++ スタイルガイド {#c-style-guide} @@ -22,7 +22,7 @@ doc_type: 'guide' -## フォーマット +## フォーマット {#formatting} **1.** ほとんどのコード整形は `clang-format` によって自動的に行われます。 @@ -214,7 +214,7 @@ for (Names::const_iterator it = column_names.begin(); it != column_names.end(); ``` -## コメント +## コメント {#comments} **1.** 自明ではない箇所には、必ずコメントを追加してください。 @@ -312,7 +312,7 @@ void executeQuery( ``` -## 名前 +## 名前 {#names} **1.** 変数およびクラスメンバーの名前には、小文字とアンダースコアからなるスネークケースを使用します。 @@ -431,7 +431,7 @@ T_PAAMAYIM_NEKUDOTAYIM のような名前は使用しないでくださ **17.** C++ ソースコードのファイル名には `.cpp` 拡張子を付けなければなりません。ヘッダーファイルには `.h` 拡張子を付けなければなりません。 -## コードの書き方 +## コードの書き方 {#how-to-write-code} **1.** メモリ管理。 @@ -700,7 +700,7 @@ auto s = std::string{"Hello"}; **26.** 仮想関数を宣言する場合、基底クラスでは `virtual` を記述し、派生クラスでは `virtual` の代わりに `override` を記述します。 -## C++で使用しない機能 +## C++で使用しない機能 {#unused-features-of-c} **1.** 仮想継承は用いない。 @@ -830,7 +830,7 @@ CPU の命令セットは、当社サーバー群で共通してサポートさ -## 追加の推奨事項 +## 追加の推奨事項 {#additional-recommendations} **1.** `stddef.h` に含まれる型に対して `std::` を明示的に指定すること diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/development/tests.md b/i18n/jp/docusaurus-plugin-content-docs/current/development/tests.md index 6468d88f0b7..deca78415a2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/development/tests.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/development/tests.md @@ -9,11 +9,11 @@ doc_type: 'guide' -# ClickHouse のテスト +# ClickHouse のテスト {#testing-clickhouse} -## 機能テスト +## 機能テスト {#functional-tests} 機能テストは最もシンプルで扱いやすいテストです。 ClickHouse の機能のほとんどは機能テストで検証でき、この方法でテスト可能な ClickHouse のコード変更については、機能テストの実行が必須です。 @@ -34,7 +34,7 @@ SQL だけでは検証できない機能、たとえば入力データを `click `DateTime` および `DateTime64` のデータ型をテストする際のよくある誤りは、サーバーが特定のタイムゾーン(例: "UTC")を使用していると想定してしまうことです。実際にはそうではなく、CI テスト実行時のタイムゾーンは意図的にランダム化されています。最も簡単な回避策は、テスト値に対してタイムゾーンを明示的に指定することです。例: `toDateTime64(val, 3, 'Europe/Amsterdam')`。 ::: -### ローカルでテストを実行する +### ローカルでテストを実行する {#running-a-test-locally} ClickHouse サーバーをローカルで起動し、デフォルトポート(9000)で待ち受けるようにします。 たとえばテスト `01428_hash_set_nan_key` を実行するには、リポジトリのフォルダーに移動して次のコマンドを実行します。 @@ -49,7 +49,7 @@ PATH=:$PATH tests/clickhouse-test 01428_hash_set_ すべてのテストを実行することも、テスト名に対するフィルターを指定して一部のテストのみを実行することもできます: `./clickhouse-test substring`。 テストを並列で実行したり、ランダムな順序で実行したりするオプションもあります。 -### 新しいテストの追加 +### 新しいテストの追加 {#adding-a-new-test} 新しいテストを追加するには、まず `queries/0_stateless` ディレクトリに `.sql` または `.sh` ファイルを作成します。 次に、`clickhouse-client < 12345_test.sql > 12345_test.reference` または `./12345_test.sh > ./12345_test.reference` を使用して、対応する `.reference` ファイルを生成します。 @@ -76,7 +76,7 @@ sudo ./install.sh * 他のテストが同じ内容をテストしていないことを確認すること(つまり、まず grep して確認する) ::: -### テスト実行の制限 +### テスト実行の制限 {#restricting-test-runs} テストには 0 個以上の *タグ* を付けることができ、CI 上でどのコンテキストで実行されるかを制御できます。 @@ -95,9 +95,9 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Tags: no-fasttest, no-replicated-database -# - no-fasttest: <ここにタグの理由を記載> -# - no-replicated-database: <ここに理由を記載> +# Tags: no-fasttest, no-replicated-database {#tags-no-fasttest-no-replicated-database} +# - no-fasttest: <ここにタグの理由を記載> {#no-fasttest-provide_a_reason_for_the_tag_here} +# - no-replicated-database: <ここに理由を記載> {#no-replicated-database-provide_a_reason_here} ``` 利用可能なタグの一覧は次のとおりです: @@ -134,7 +134,7 @@ SELECT 1 上記の設定に加えて、特定の ClickHouse 機能を使用するかどうかを指定するために、`system.build_options` の `USE_*` フラグを使用できます。 たとえば、テストで MySQL テーブルを使用する場合は、タグ `use-mysql` を追加する必要があります。 -### ランダム設定の制限の指定 +### ランダム設定の制限の指定 {#specifying-limits-for-random-settings} テストでは、テスト実行中にランダム化される可能性がある設定について、許可される最小値と最大値を指定できます。 @@ -143,8 +143,8 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Tags: no-fasttest -# ランダム設定の制限: max_block_size=(1000, 10000); index_granularity=(100, None) +# Tags: no-fasttest {#tags-no-fasttest} +# ランダム設定の制限: max_block_size=(1000, 10000); index_granularity=(100, None) {#random-settings-limits-max_block_size1000-10000-index_granularity100-none} ``` `.sql` テストでは、タグは対象行の直後の行か、先頭行に SQL コメントとして記述します。 @@ -157,7 +157,7 @@ SELECT 1 片方の上限だけを指定する場合は、もう一方には `None` を指定できます。 -### テスト名の決め方 +### テスト名の決め方 {#choosing-the-test-name} テストの名前は、`00422_hash_function_constexpr.sql` のように、5桁のプレフィックスの後に内容を表す名前を付けます。 プレフィックスを決めるには、ディレクトリ内で既に存在する最大のプレフィックスを確認し、その値に 1 を加えてください。 @@ -168,7 +168,7 @@ ls tests/queries/0_stateless/[0-9]*.reference | tail -n 1 その間に、同じ数値プレフィックスを持つ別のテストが追加されることもありますが、それでも問題はなく、そのままで構いません。後から変更する必要はありません。 -### 必ず発生するエラーの確認 +### 必ず発生するエラーの確認 {#checking-for-an-error-that-must-occur} 誤ったクエリに対してサーバーエラーが発生することを確認したい場合があります。そのために、SQL テストでは次の形式の特別なアノテーションをサポートしています。 @@ -184,12 +184,12 @@ SELECT x; -- { serverError 49 } エラーコードのみを確認してください。 既存のエラーコードが要件に対して十分に厳密でない場合は、新しいエラーコードの追加を検討してください。 -### 分散クエリのテスト +### 分散クエリのテスト {#testing-a-distributed-query} 機能テストで分散クエリを使用したい場合、サーバー自身に対してクエリを実行するために、アドレス `127.0.0.{1..2}` を指定した `remote` テーブル関数を利用できます。または、サーバー設定ファイル内であらかじめ定義された `test_shard_localhost` のようなテスト用クラスタを使用することもできます。 テスト名には必ず `shard` または `distributed` という単語を含めてください。そうすることで、サーバーが分散クエリをサポートするように設定されている正しい構成で、CI 上でテストが実行されるようになります。 -### 一時ファイルの扱い +### 一時ファイルの扱い {#working-with-temporary-files} シェルテストの中で、その場でファイルを作成して利用する必要が生じる場合があります。 一部の CI チェックではテストが並列に実行されることに注意してください。そのため、一意ではない名前でスクリプト内から一時ファイルを作成または削除していると、`Flaky` などの CI チェックが失敗する原因になります。 @@ -217,7 +217,7 @@ SELECT x; -- { serverError 49 } -## ユニットテスト +## ユニットテスト {#unit-tests} ユニットテストは、ClickHouse 全体ではなく、特定のライブラリやクラス単体をテストしたい場合に有用です。 テストのビルドは、`ENABLE_TESTS` という CMake オプションで有効または無効にできます。 @@ -272,7 +272,7 @@ $ ./src/unit_tests_dbms --gtest_filter=LocalAddress* -## 手動テスト +## 手動テスト {#manual-testing} 新しい機能を開発した場合、その機能を手動でもテストするのは妥当です。 次の手順で行うことができます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/dictionary/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/dictionary/index.md deleted file mode 100644 index 3ffaa3a0a60..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/dictionary/index.md +++ /dev/null @@ -1,342 +0,0 @@ ---- -slug: /dictionary -title: '辞書' -keywords: ['辞書', '辞書型'] -description: '辞書は、高速なルックアップのためにデータをキーと値のペアで表現する構造です。' -doc_type: 'reference' ---- - -import dictionaryUseCases from '@site/static/images/dictionary/dictionary-use-cases.png'; -import dictionaryLeftAnyJoin from '@site/static/images/dictionary/dictionary-left-any-join.png'; -import Image from '@theme/IdealImage'; - - -# Dictionary - -ClickHouse における Dictionary は、さまざまな[内部および外部ソース](/sql-reference/dictionaries#dictionary-sources)からのデータをメモリ上の [key-value](https://en.wikipedia.org/wiki/Key%E2%80%93value_database) 形式で表現し、きわめて低レイテンシなルックアップクエリ向けに最適化したものです。 - -Dictionary は次のような用途に役立ちます: -- 特に `JOIN` と組み合わせて使用する場合に、クエリのパフォーマンスを向上させる -- インジェスト処理を低速化させることなく、その場でインジェストされたデータを拡充する - -ClickHouse における Dictionary のユースケース - - - -## Dictionary を使用した JOIN の高速化 - -Dictionary は、特定タイプの `JOIN`、すなわち結合キーが基礎となるキー・バリュー型ストレージのキー属性と一致する必要がある [`LEFT ANY` 型](/sql-reference/statements/select/join#supported-types-of-join) を高速化するために利用できます。 - -LEFT ANY JOIN で Dictionary を使用する - -この条件が満たされる場合、ClickHouse は Dictionary を利用して [Direct Join](https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4#direct-join) を実行できます。これは ClickHouse における最速の JOIN アルゴリズムであり、右側のテーブルに対して使用されている [table engine](/engines/table-engines) が低レイテンシなキー・バリュー要求をサポートしている場合に適用可能です。ClickHouse にはこれに対応する table engine が 3 種類あります: [Join](/engines/table-engines/special/join)(事前計算済みのハッシュテーブルに相当)、[EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb)、および [Dictionary](/engines/table-engines/special/dictionary) です。ここでは Dictionary ベースのアプローチについて説明しますが、仕組み自体は 3 つのエンジンで共通です。 - -Direct Join アルゴリズムでは、右側のテーブルが Dictionary をバックエンドとして持っている必要があります。これにより、そのテーブルから結合対象となるデータが、低レイテンシなキー・バリュー型データ構造としてすでにメモリ上に存在している状態になります。 - -### 例 - -Stack Overflow データセットを使用して、次の疑問に答えてみます: -*Hacker News において、SQL に関する最も「物議を醸した」投稿はどれか?* - -ここでは「物議を醸した」を、賛成票と反対票の数が近い投稿と定義します。この絶対差を計算し、値が 0 に近いほど議論を呼んでいるとみなします。また、投稿には少なくとも 10 件以上の賛成票と反対票があるものに限定します — そもそもほとんど投票されていない投稿は、あまり物議を醸しているとはいえないためです。 - -データが正規化されている前提では、このクエリには現在、`posts` テーブルと `votes` テーブルを用いた `JOIN` が必要です: - -```sql -WITH PostIds AS -( - SELECT Id - FROM posts - WHERE Title ILIKE '%SQL%' -) -SELECT - Id, - Title, - UpVotes, - DownVotes, - abs(UpVotes - DownVotes) AS Controversial_ratio -FROM posts -INNER JOIN -( - SELECT - PostId, - countIf(VoteTypeId = 2) AS UpVotes, - countIf(VoteTypeId = 3) AS DownVotes - FROM votes - WHERE PostId IN (PostIds) - GROUP BY PostId - HAVING (UpVotes > 10) AND (DownVotes > 10) -) AS votes ON posts.Id = votes.PostId -WHERE Id IN (PostIds) -ORDER BY Controversial_ratio ASC -LIMIT 1 - -行 1: -────── -Id: 25372161 -Title: How to add exception handling to SqlDataSource.UpdateCommand -UpVotes: 13 -DownVotes: 13 -Controversial_ratio: 0 - -1行を取得しました。経過時間: 1.283秒。処理行数: 4億1844万行、7.23 GB (3億2607万行/秒、5.63 GB/秒) -ピークメモリ使用量: 3.18 GiB。 -``` - -> **`JOIN` の右側にはより小さいデータセットを使用する**: このクエリでは、外側とサブクエリの両方で `PostId` をフィルタリングしているため、必要以上に冗長に見えるかもしれません。これは、クエリの応答時間を高速に保つためのパフォーマンス最適化です。最適なパフォーマンスを得るには、常に `JOIN` の右側がより小さな集合となるようにし、可能な限り小さく保ってください。`JOIN` のパフォーマンス最適化や利用可能なアルゴリズムの理解に関するヒントとしては、[このブログ記事シリーズ](https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1)を参照することを推奨します。 - -このクエリは高速ですが、良好なパフォーマンスを得るには `JOIN` を慎重に記述する必要があります。本来であれば、まず投稿を「SQL」を含むものだけにフィルタリングし、その後で対象となる一部のブログについて `UpVote` と `DownVote` の数を確認してメトリクスを計算したいところです。 - -#### 辞書の適用 - -これらの概念を示すために、投票データに辞書を使用します。辞書は通常メモリ上に保持されるため(例外は [ssd_cache](/sql-reference/dictionaries#ssd_cache))、ユーザーはデータサイズを意識する必要があります。次に `votes` テーブルのサイズを確認します: - - -```sql -SELECT table, - formatReadableSize(sum(data_compressed_bytes)) AS compressed_size, - formatReadableSize(sum(data_uncompressed_bytes)) AS uncompressed_size, - round(sum(data_uncompressed_bytes) / sum(data_compressed_bytes), 2) AS ratio -FROM system.columns -WHERE table IN ('votes') -GROUP BY table - -┌─table───────────┬─compressed_size─┬─uncompressed_size─┬─ratio─┐ -│ votes │ 1.25 GiB │ 3.79 GiB │ 3.04 │ -└─────────────────┴─────────────────┴───────────────────┴───────┘ -``` - -データはディクショナリ内に非圧縮のまま保存されるため、すべてのカラム(実際にはそうしません)をディクショナリに保存すると仮定すると、少なくとも 4GB のメモリが必要になります。ディクショナリはクラスター全体にレプリケートされるため、このメモリ量は *ノードごとに* 確保する必要があります。 - -> 以下の例では、ディクショナリ用のデータは ClickHouse テーブルに由来しています。これはディクショナリの最も一般的なソースですが、ファイル、HTTP、[Postgres](/sql-reference/dictionaries#postgresql) を含むデータベースなど、[多数のソース](/sql-reference/dictionaries#dictionary-sources) がサポートされています。後述するように、ディクショナリは自動的に更新できるため、頻繁に変更される小さなデータセットを直接結合で利用可能にする理想的な方法となります。 - -ディクショナリには、ルックアップを行うための主キーが必要です。これは概念的にはトランザクションデータベースの主キーと同様であり、一意である必要があります。上記のクエリでは、結合キー `PostId` に対するルックアップが必要です。ディクショナリには、`votes` テーブルから各 `PostId` について賛成票と反対票の合計値が格納されている必要があります。以下が、このディクショナリデータを取得するためのクエリです: - -```sql -SELECT PostId, - countIf(VoteTypeId = 2) AS UpVotes, - countIf(VoteTypeId = 3) AS DownVotes -FROM votes -GROUP BY PostId -``` - -このディクショナリを作成するには、次の DDL が必要です。ここで、先ほどのクエリを使用している点に注意してください。 - -```sql -CREATE DICTIONARY votes_dict -( - `PostId` UInt64, - `UpVotes` UInt32, - `DownVotes` UInt32 -) -PRIMARY KEY PostId -SOURCE(CLICKHOUSE(QUERY 'SELECT PostId, countIf(VoteTypeId = 2) AS UpVotes, countIf(VoteTypeId = 3) AS DownVotes FROM votes GROUP BY PostId')) -LIFETIME(MIN 600 MAX 900) -LAYOUT(HASHED()) - -0 rows in set. Elapsed: 36.063 sec. -``` - -> 自己管理の OSS 環境では、上記のコマンドをすべてのノードで実行する必要があります。ClickHouse Cloud では、ディクショナリはすべてのノードに自動的にレプリケートされます。上記の処理は、RAM 64GB の ClickHouse Cloud ノード上で実行しており、ロードに 36 秒を要しました。 - -ディクショナリが消費しているメモリ量を確認するには、次のようにします。 - -```sql -SELECT formatReadableSize(bytes_allocated) AS size -FROM system.dictionaries -WHERE name = 'votes_dict' - -┌─size─────┐ -│ 4.00 GiB │ -└──────────┘ -``` - -特定の `PostId` に対する賛成票と反対票は、シンプルな `dictGet` 関数で取得できるようになりました。以下の例では、投稿 `11227902` の値を取得しています。 - -```sql -SELECT dictGet('votes_dict', ('UpVotes', 'DownVotes'), '11227902') AS votes - -┌─votes──────┐ -│ (34999,32) │ -└────────────┘ - -これを先ほどのクエリに適用することで、JOINを削除できます: - -WITH PostIds AS -( - SELECT Id - FROM posts - WHERE Title ILIKE '%SQL%' -) -SELECT Id, Title, - dictGet('votes_dict', 'UpVotes', Id) AS UpVotes, - dictGet('votes_dict', 'DownVotes', Id) AS DownVotes, - abs(UpVotes - DownVotes) AS Controversial_ratio -FROM posts -WHERE (Id IN (PostIds)) AND (UpVotes > 10) AND (DownVotes > 10) -ORDER BY Controversial_ratio ASC -LIMIT 3 - -3行のセット。経過時間: 0.551秒。処理: 1億1964万行、3.29 GB (毎秒2億1696万行、毎秒5.97 GB)。 -ピークメモリ使用量: 552.26 MiB。 -``` - -このクエリははるかに単純なだけでなく、実行速度も2倍以上高速です! さらに最適化するには、賛成票と反対票の合計が10を超える投稿だけを辞書に読み込み、あらかじめ計算した「controversial」値だけを保存するようにします。 - - -## クエリ時の拡張 - -辞書は、クエリ実行時に値を参照するために使用できます。これらの値は結果として返したり、集約処理で使用したりできます。たとえば、ユーザー ID をロケーションに対応付ける辞書を作成するとします。 - -```sql -CREATE DICTIONARY users_dict -( - `Id` Int32, - `Location` String -) -PRIMARY KEY Id -SOURCE(CLICKHOUSE(QUERY 'SELECT Id, Location FROM stackoverflow.users')) -LIFETIME(MIN 600 MAX 900) -LAYOUT(HASHED()) -``` - -この辞書を使用して、POST の結果に情報を付加できます。 - -```sql -SELECT - Id, - Title, - dictGet('users_dict', 'Location', CAST(OwnerUserId, 'UInt64')) AS location -FROM posts -WHERE Title ILIKE '%clickhouse%' -LIMIT 5 -FORMAT PrettyCompactMonoBlock - -┌───────Id─┬─Title─────────────────────────────────────────────────────────┬─Location──────────────┐ -│ 52296928 │ ClickHouseにおける2つの文字列の比較 │ Spain │ -│ 52345137 │ ファイルを使用してMySQLからClickHouseにデータを移行する方法 │ 中国江苏省Nanjing Shi │ -│ 61452077 │ ClickHouseでPARTITIONを変更する方法 │ Guangzhou, 广东省中国 │ -│ 55608325 │ ClickHouseでmax()を使わずにテーブルの最後のレコードを選択 │ Moscow, Russia │ -│ 55758594 │ ClickHouseで一時テーブルを作成 │ Perm', Russia │ -└──────────┴───────────────────────────────────────────────────────────────┴───────────────────────┘ - -5行を取得しました。経過時間: 0.033秒。処理行数: 425万行、82.84 MB (1億3062万行/秒、2.55 GB/秒) -ピークメモリ使用量: 249.32 MiB。 -``` - -先ほどの結合の例と同様に、同じ辞書を使って、投稿のおもな発信元を効率的に判定できます。 - -```sql -SELECT - dictGet('users_dict', 'Location', CAST(OwnerUserId, 'UInt64')) AS location, - count() AS c -FROM posts -WHERE location != '' -GROUP BY location -ORDER BY c DESC -LIMIT 5 - -┌─location───────────────┬──────c─┐ -│ India │ 787814 │ -│ Germany │ 685347 │ -│ United States │ 595818 │ -│ London, United Kingdom │ 538738 │ -│ United Kingdom │ 537699 │ -└────────────────────────┴────────┘ - -5 rows in set. Elapsed: 0.763 sec. Processed 59.82 million rows, 239.28 MB (78.40 million rows/s., 313.60 MB/s.) -Peak memory usage: 248.84 MiB. -``` - - -## インデックス時のエンリッチメント - -上記の例では、クエリ時に辞書を使用して結合を排除しました。辞書は、挿入時に行をエンリッチするためにも使用できます。これは通常、エンリッチに用いる値が変わらず、かつ辞書を埋めるために利用できる外部ソースに存在する場合に適しています。この場合、挿入時に行をエンリッチしておけば、クエリ時の辞書ルックアップを回避できます。 - -Stack Overflow におけるユーザーの `Location` が決して変わらないと仮定します(実際には変わります)— 具体的には、`users` テーブルの `Location` 列です。投稿テーブルをロケーション別に分析するクエリを実行したいとします。このテーブルには `UserId` が含まれています。 - -辞書は、`users` テーブルをデータソースとして、ユーザー ID からロケーションへのマッピングを提供します。 - -```sql -CREATE DICTIONARY users_dict -( - `Id` UInt64, - `Location` String -) -PRIMARY KEY Id -SOURCE(CLICKHOUSE(QUERY 'SELECT Id, Location FROM users WHERE Id >= 0')) -LIFETIME(MIN 600 MAX 900) -LAYOUT(HASHED()) -``` - -> `Hashed` 辞書型を使用できるようにするため、`Id < 0` のユーザーを除外しています。`Id < 0` のユーザーはシステムユーザーです。 - -この辞書を posts テーブルへの挿入時に利用するには、スキーマを変更する必要があります。 - -```sql -CREATE TABLE posts_with_location -( - `Id` UInt32, - `PostTypeId` Enum8('Question' = 1, 'Answer' = 2, 'Wiki' = 3, 'TagWikiExcerpt' = 4, 'TagWiki' = 5, 'ModeratorNomination' = 6, 'WikiPlaceholder' = 7, 'PrivilegeWiki' = 8), - ... - `Location` MATERIALIZED dictGet(users_dict, 'Location', OwnerUserId::'UInt64') -) -ENGINE = MergeTree -ORDER BY (PostTypeId, toDate(CreationDate), CommentCount) -``` - -上記の例では、`Location` は `MATERIALIZED` カラムとして宣言されています。これは、値を `INSERT` クエリの一部として指定することもできますが、その値に関わらず常に計算されることを意味します。 - -> ClickHouse は [`DEFAULT` カラム](/sql-reference/statements/create/table#default_values) もサポートしています(値を明示的に挿入することも、指定されていない場合に計算させることもできます)。 - -テーブルにデータを投入するには、通常どおり S3 からの `INSERT INTO SELECT` を使用できます。 - -```sql -INSERT INTO posts_with_location SELECT Id, PostTypeId::UInt8, AcceptedAnswerId, CreationDate, Score, ViewCount, Body, OwnerUserId, OwnerDisplayName, LastEditorUserId, LastEditorDisplayName, LastEditDate, LastActivityDate, Title, Tags, AnswerCount, CommentCount, FavoriteCount, ContentLicense, ParentId, CommunityOwnedDate, ClosedDate FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/*.parquet') - -0 rows in set. Elapsed: 36.830 sec. Processed 238.98 million rows, 2.64 GB (6.49 million rows/s., 71.79 MB/s.) -``` - -これで、投稿の大半がどの場所から発信されているか、その場所名を取得できます。 - -```sql -SELECT Location, count() AS c -FROM posts_with_location -WHERE Location != '' -GROUP BY Location -ORDER BY c DESC -LIMIT 4 - -┌─Location───────────────┬──────c─┐ -│ India │ 787814 │ -│ Germany │ 685347 │ -│ United States │ 595818 │ -│ London, United Kingdom │ 538738 │ -└────────────────────────┴────────┘ - -4 rows in set. Elapsed: 0.142 sec. Processed 59.82 million rows, 1.08 GB (420.73 million rows/s., 7.60 GB/s.) -Peak memory usage: 666.82 MiB. -``` - - -## 辞書に関する高度なトピック {#advanced-dictionary-topics} - -### 辞書の `LAYOUT` の選択 {#choosing-the-dictionary-layout} - -`LAYOUT` 句は、辞書の内部データ構造を制御します。利用可能なオプションはいくつかあり、その内容は[こちら](/sql-reference/dictionaries#ways-to-store-dictionaries-in-memory)に記載されています。適切なレイアウトを選択するためのヒントは[こちら](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout)にあります。 - -### 辞書の更新 {#refreshing-dictionaries} - -辞書の `LIFETIME` として `MIN 600 MAX 900` を指定しました。LIFETIME は辞書の更新間隔であり、ここで指定した値により 600~900 秒のランダムな間隔で定期的に再読み込みが行われます。このランダムな間隔は、多数のサーバーで更新を行う際に辞書のソースへの負荷を分散するために必要です。更新中でも、古いバージョンの辞書には引き続きクエリを実行できます。初回の読み込みのみがクエリをブロックします。`(LIFETIME(0))` を設定すると、辞書は更新されなくなる点に注意してください。 -辞書は `SYSTEM RELOAD DICTIONARY` コマンドを使用して強制的に再読み込みできます。 - -ClickHouse や Postgres のようなデータベースソースでは、一定間隔ではなく、実際に変更があった場合にのみ辞書を更新するクエリを設定できます(クエリの応答結果によって更新の要否が決まります)。詳細は[こちら](/sql-reference/dictionaries#refreshing-dictionary-data-using-lifetime)を参照してください。 - -### その他の辞書タイプ {#other-dictionary-types} - -ClickHouse は [階層型](/sql-reference/dictionaries#hierarchical-dictionaries)、[Polygon](/sql-reference/dictionaries#polygon-dictionaries)、[正規表現](/sql-reference/dictionaries#regexp-tree-dictionary) 辞書もサポートしています。 - -### 参考資料 {#more-reading} - -- [Using Dictionaries to Accelerate Queries](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse) -- [Advanced Configuration for Dictionaries](/sql-reference/dictionaries) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md index a5893eaab15..a1a2c5057d7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Atomic +# Atomic {#atomic} `Atomic` エンジンは、ノンブロッキングな [`DROP TABLE`](#drop-detach-table) および [`RENAME TABLE`](#rename-table) クエリに加え、アトミックな [`EXCHANGE TABLES`](#exchange-tables) クエリをサポートします。`Atomic` データベースエンジンは、オープンソース版の ClickHouse でデフォルトとして使用されています。 @@ -19,16 +19,16 @@ ClickHouse Cloud では、デフォルトで [`Shared` データベースエン -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE test [ENGINE = Atomic] [SETTINGS disk=...]; ``` -## 詳細と推奨事項 +## 詳細と推奨事項 {#specifics-and-recommendations} -### テーブル UUID +### テーブル UUID {#table-uuid} `Atomic` データベース内の各テーブルには永続的な [UUID](../../sql-reference/data-types/uuid.md) が付与されており、そのデータは以下のディレクトリに保存されます。 @@ -50,16 +50,16 @@ CREATE TABLE name UUID '28f1c61c-2970-457a-bffe-454156ddcfef' (n UInt64) ENGINE [`SHOW CREATE` クエリで UUID を表示するには、[show_table_uuid_in_table_create_query_if_not_nil](../../operations/settings/settings.md#show_table_uuid_in_table_create_query_if_not_nil) 設定を使用できます。 ::: -### RENAME TABLE +### RENAME TABLE {#rename-table} [`RENAME`](../../sql-reference/statements/rename.md) クエリは UUID を変更せず、テーブルデータも移動しません。これらのクエリは即座に実行され、そのテーブルを使用している他のクエリの完了を待ちません。 -### DROP/DETACH TABLE +### DROP/DETACH TABLE {#drop-detach-table} `DROP TABLE` を使用しても、データはすぐには削除されません。`Atomic` エンジンは、メタデータを `/clickhouse_path/metadata_dropped/` に移動してテーブルを削除済みとしてマークし、バックグラウンドスレッドに通知するだけです。テーブルデータが最終的に削除されるまでの遅延は、[`database_atomic_delay_before_drop_table_sec`](../../operations/server-configuration-parameters/settings.md#database_atomic_delay_before_drop_table_sec) 設定で指定します。 `SYNC` 修飾子を使用して同期モードを指定できます。これを行うには、[`database_atomic_wait_for_drop_and_detach_synchronously`](../../operations/settings/settings.md#database_atomic_wait_for_drop_and_detach_synchronously) 設定を使用します。この場合、`DROP` はテーブルを使用している実行中の `SELECT`、`INSERT` などのクエリが終了するまで待機します。テーブルは使用されていない状態になったときに削除されます。 -### EXCHANGE TABLES/DICTIONARIES +### EXCHANGE TABLES/DICTIONARIES {#exchange-tables} [`EXCHANGE`](../../sql-reference/statements/exchange.md) クエリは、テーブルやディクショナリをアトミックに入れ替えます。たとえば、次のような非アトミックな操作の代わりに使用できます。 @@ -73,11 +73,11 @@ RENAME TABLE new_table TO tmp, old_table TO new_table, tmp TO old_table; EXCHANGE TABLES new_table AND old_table; ``` -### atomic データベースにおける ReplicatedMergeTree +### atomic データベースにおける ReplicatedMergeTree {#replicatedmergetree-in-atomic-database} [`ReplicatedMergeTree`](/engines/table-engines/mergetree-family/replication) テーブルでは、ZooKeeper 内のパスおよびレプリカ名を指定するエンジンパラメータは設定しないことを推奨します。この場合、設定パラメータ [`default_replica_path`](../../operations/server-configuration-parameters/settings.md#default_replica_path) と [`default_replica_name`](../../operations/server-configuration-parameters/settings.md#default_replica_name) が使用されます。エンジンパラメータを明示的に指定したい場合は、`{uuid}` マクロを使用することを推奨します。これにより、ZooKeeper 内でテーブルごとに一意なパスが自動的に生成されます。 -### メタデータディスク +### メタデータディスク {#metadata-disk} `SETTINGS` 内で `disk` が指定されている場合、そのディスクはテーブルのメタデータファイルの保存に使用されます。 例えば次のとおりです。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md index 917606ca327..476c6e6c551 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md @@ -7,17 +7,13 @@ title: 'バックアップ' doc_type: 'reference' --- - - -# バックアップ +# バックアップ {#backup} データベースのバックアップ機能を使用すると、[バックアップ](../../operations/backup) からテーブルやデータベースを読み取り専用モードで即座にアタッチできます。 データベースのバックアップは、増分バックアップと非増分バックアップの両方に対応しています。 - - -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE backup_database @@ -38,8 +34,7 @@ ENGINE = Backup('database_name_inside_backup', Disk('disk_name', 'backup_name')) * `database_name_inside_backup` — バックアップ内のデータベースの名前。 * `backup_destination` — バックアップ先。 - -## 使用例 +## 使用例 {#usage-example} `Disk` バックアップ先を使った例を見てみましょう。まずは `storage.xml` でバックアップ用ディスクを設定します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md index c24b3bd2eb4..14d1611a99f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' -# DataLakeCatalog +# DataLakeCatalog {#datalakecatalog} `DataLakeCatalog` データベースエンジンを使用すると、ClickHouse を外部の データカタログに接続し、データを複製することなくオープンテーブルフォーマットのデータをクエリできます。 @@ -28,7 +28,7 @@ doc_type: 'reference' -## データベースの作成 +## データベースの作成 {#creating-a-database} `DataLakeCatalog` エンジンを使用するには、以下の必要な設定を有効にする必要があります。 @@ -66,7 +66,7 @@ catalog_type, | `region` | サービスの AWS リージョン(例: `us-east-1`) | -## 例 +## 例 {#examples} `DataLakeCatalog` エンジンの使用例については、以下のセクションを参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/index.md deleted file mode 100644 index b6421800e80..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/index.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'データベースエンジンに関するドキュメント' -slug: /engines/database-engines/ -toc_folder_title: 'データベースエンジン' -toc_priority: 27 -toc_title: '概要' -title: 'データベースエンジン' -doc_type: 'landing-page' ---- - - - -# データベースエンジン - -データベースエンジンは、テーブルを操作するための仕組みです。デフォルトでは、ClickHouse は [Atomic](../../engines/database-engines/atomic.md) データベースエンジンを使用しており、これにより設定可能な[テーブルエンジン](../../engines/table-engines/index.md)と [SQL の方言](../../sql-reference/syntax.md)が提供されます。 - -利用可能なデータベースエンジンの一覧は次のとおりです。詳細については、各リンクを参照してください。 - -{/* このページの目次は、YAML フロントマターのフィールド slug、description、title から - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - によって自動生成されています。 - - 誤りを見つけた場合は、各ページの YML フロントマターを編集してください。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| --------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | -| [Shared](/engines/database-engines/shared) | ClickHouse Cloud で利用可能な `Shared` データベースエンジンについて説明します。 | -| [Atomic](/engines/database-engines/atomic) | `Atomic` エンジンは、ブロックしない `DROP TABLE` および `RENAME TABLE` クエリと、アトミックな `EXCHANGE TABLES` クエリをサポートします。`Atomic` データベースエンジンはデフォルトで使用されます。 | -| [Lazy](/engines/database-engines/lazy) | 最終アクセスから `expiration_time_in_seconds` 秒間のみテーブルを RAM に保持します。Log 型テーブルでのみ使用できます。 | -| [Replicated](/engines/database-engines/replicated) | このエンジンは Atomic エンジンを基盤としています。ZooKeeper に書き込まれる DDL ログを介してメタデータのレプリケーションをサポートし、対象データベースのすべてのレプリカで実行されます。 | -| [PostgreSQL](/engines/database-engines/postgresql) | リモートの PostgreSQL サーバー上のデータベースに接続できます。 | -| [MySQL](/engines/database-engines/mysql) | リモートの MySQL サーバー上のデータベースに接続し、ClickHouse と MySQL 間でデータを交換するために `INSERT` および `SELECT` クエリを実行できます。 | -| [SQLite](/engines/database-engines/sqlite) | SQLite データベースに接続し、ClickHouse と SQLite 間でデータを交換するために `INSERT` および `SELECT` クエリを実行できます。 | -| [MaterializedPostgreSQL](/engines/database-engines/materialized-postgresql) | PostgreSQL データベース内のテーブルを用いて ClickHouse データベースを作成します。 | -| [Backup](/engines/database-engines/backup) | バックアップからテーブル/データベースを読み取り専用モードで即座にアタッチできます。 | -| [DataLakeCatalog](/engines/database-engines/datalakecatalog) | DataLakeCatalog データベースエンジンを使用すると、ClickHouse を外部データカタログに接続し、オープンテーブル形式のデータをクエリできます。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md index 216fe8f5698..54ca2f612ca 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md @@ -7,17 +7,13 @@ title: 'Lazy' doc_type: 'reference' --- +# Lazy {#lazy} +テーブルを最後のアクセスから `expiration_time_in_seconds` 秒間だけ RAM 内に保持します。 *Log テーブルでのみ使用できます。 -# Lazy +アクセス間隔が長い多数の小さな *Log テーブルを格納する用途に最適化されています。 -テーブルを最後のアクセスから `expiration_time_in_seconds` 秒間だけ RAM 内に保持します。 \*Log テーブルでのみ使用できます。 - -アクセス間隔が長い多数の小さな \*Log テーブルを格納する用途に最適化されています。 - - - -## データベースを作成する +## データベースを作成する {#creating-a-database} ```sql CREATE DATABASE testlazy diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md index 0dfe9d79b00..9f42bd4e7b6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MaterializedPostgreSQL +# MaterializedPostgreSQL {#materializedpostgresql} @@ -35,7 +35,7 @@ SET allow_experimental_database_materialized_postgresql=1 ::: -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -50,7 +50,7 @@ ENGINE = MaterializedPostgreSQL('host:port', 'database', 'user', 'password') [SE * `password` — ユーザーのパスワード。 -## 使用例 +## 使用例 {#example-of-use} ```sql CREATE DATABASE postgres_db @@ -66,7 +66,7 @@ SELECT * FROM postgresql_db.postgres_table; ``` -## レプリケーションへの新しいテーブルの動的追加 +## レプリケーションへの新しいテーブルの動的追加 {#dynamically-adding-table-to-replication} `MaterializedPostgreSQL` データベースが作成された後は、対応する PostgreSQL データベース内の新しいテーブルは自動的には検出されません。こうしたテーブルは手動で追加できます。 @@ -79,7 +79,7 @@ ATTACH TABLE postgres_database.new_table; ::: -## レプリケーションからテーブルを動的に除外する +## レプリケーションからテーブルを動的に除外する {#dynamically-removing-table-from-replication} 特定のテーブルをレプリケーション対象から除外することができます。 @@ -88,7 +88,7 @@ DETACH TABLE postgres_database.table_to_remove PERMANENTLY; ``` -## PostgreSQL スキーマ +## PostgreSQL スキーマ {#schema} PostgreSQL の [スキーマ](https://www.postgresql.org/docs/9.1/ddl-schemas.html) は、3 通りの方法で設定できます(バージョン 21.12 以降)。 @@ -136,7 +136,7 @@ SELECT * FROM database1.`schema2.table2`; 警告: このケースでは、テーブル名にピリオド(.)を含めることはできません。 -## 要件 +## 要件 {#requirements} 1. PostgreSQL の設定ファイルにおいて、[wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) 設定は `logical` にし、`max_replication_slots` パラメータは少なくとも `2` に設定する必要があります。 @@ -172,9 +172,9 @@ WHERE oid = 'postgres_table'::regclass; ::: -## 設定 +## 設定 {#settings} -### `materialized_postgresql_tables_list` +### `materialized_postgresql_tables_list` {#materialized-postgresql-tables-list} [MaterializedPostgreSQL](../../engines/database-engines/materialized-postgresql.md) データベースエンジンによって複製される、PostgreSQL データベースのテーブルのコンマ区切りリストを設定します。 @@ -186,15 +186,15 @@ materialized_postgresql_tables_list = 'table1(co1, col2),table2,table3(co3, col5 デフォルト値: 空リスト。空リストの場合、PostgreSQL データベース全体がレプリケートされます。 -### `materialized_postgresql_schema` +### `materialized_postgresql_schema` {#materialized-postgresql-schema} デフォルト値: 空文字列(デフォルトスキーマが使用されます)。 -### `materialized_postgresql_schema_list` +### `materialized_postgresql_schema_list` {#materialized-postgresql-schema-list} デフォルト値: 空リスト(デフォルトスキーマが使用されます)。 -### `materialized_postgresql_max_block_size` +### `materialized_postgresql_max_block_size` {#materialized-postgresql-max-block-size} PostgreSQL データベースのテーブルにデータをフラッシュする前に、メモリ内に蓄積する行数を設定します。 @@ -204,11 +204,11 @@ PostgreSQL データベースのテーブルにデータをフラッシュする デフォルト値: `65536`。 -### `materialized_postgresql_replication_slot` +### `materialized_postgresql_replication_slot` {#materialized-postgresql-replication-slot} ユーザーが作成したレプリケーションスロットです。`materialized_postgresql_snapshot` と一緒に使用する必要があります。 -### `materialized_postgresql_snapshot` +### `materialized_postgresql_snapshot` {#materialized-postgresql-snapshot} スナップショットを識別する文字列であり、このスナップショットから [PostgreSQL テーブルの初回ダンプ](../../engines/database-engines/materialized-postgresql.md) が実行されます。`materialized_postgresql_replication_slot` と一緒に使用する必要があります。 @@ -226,7 +226,7 @@ SELECT * FROM database1.table1; ALTER DATABASE postgres_database MODIFY SETTING materialized_postgresql_max_block_size = <新しいサイズ>; ``` -### `materialized_postgresql_use_unique_replication_consumer_identifier` +### `materialized_postgresql_use_unique_replication_consumer_identifier` {#materialized_postgresql_use_unique_replication_consumer_identifier} レプリケーション用に一意のレプリケーションコンシューマ識別子を使用します。デフォルト: `0`。 `1` に設定すると、同じ `PostgreSQL` テーブルを参照する複数の `MaterializedPostgreSQL` テーブルを設定できるようになります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md index 0d96fca92b6..f2628f548d6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MySQL データベースエンジン +# MySQL データベースエンジン {#mysql-database-engine} @@ -26,7 +26,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -65,7 +65,7 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') -## グローバル変数のサポート +## グローバル変数のサポート {#global-variables-support} 互換性を高めるために、グローバル変数を MySQL 互換の `@@identifier` 形式で参照できます。 @@ -85,7 +85,7 @@ SELECT @@version; ``` -## 使用例 +## 使用例 {#examples-of-use} MySQL のテーブル: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md index 83bb0870368..85be4e88481 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# PostgreSQL +# PostgreSQL {#postgresql} リモート [PostgreSQL](https://www.postgresql.org) サーバー上のデータベースに接続できます。ClickHouse と PostgreSQL 間でデータをやり取りするために、読み取りおよび書き込み操作(`SELECT` および `INSERT` クエリ)をサポートします。 @@ -19,7 +19,7 @@ doc_type: 'guide' -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE test_database @@ -56,7 +56,7 @@ ENGINE = PostgreSQL('host:port', 'database', 'user', 'password'[, `schema`, `use -## 利用例 +## 利用例 {#examples-of-use} ClickHouse 上のデータベースが PostgreSQL サーバーとデータを交換する例: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md index 11660d30875..474e08d08d4 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md @@ -9,7 +9,7 @@ doc_type: "reference" -# Replicated +# Replicated {#replicated} このエンジンは [Atomic](../../engines/database-engines/atomic.md) エンジンをベースとしています。ZooKeeper に書き込まれる DDL ログを介したメタデータのレプリケーションをサポートしており、特定のデータベースに対するすべてのレプリカで実行されます。 @@ -17,7 +17,7 @@ doc_type: "reference" -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE testdb [UUID '...'] ENGINE = Replicated('zoo_path', 'shard_name', 'replica_name') [SETTINGS ...] @@ -54,7 +54,7 @@ CREATE DATABASE testdb [UUID '...'] ENGINE = Replicated('zoo_path', 'shard_name' -## 使用例 +## 使用例 {#usage-example} 3 つのホストを持つクラスターの作成: @@ -153,7 +153,7 @@ node2 :) SELECT materialize(hostName()) AS host, groupArray(n) FROM r.d GROUP BY ``` -## 設定 +## 設定 {#settings} サポートされている設定は次のとおりです: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md index 949076750dc..9961eeeabea 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md @@ -12,7 +12,7 @@ import CloudOnlyBadge from '@theme/badges/CloudOnlyBadge'; -# Shared データベースエンジン +# Shared データベースエンジン {#shared-database-engine} `Shared` データベースエンジンは、Shared Catalog と連携して、[`SharedMergeTree`](/cloud/reference/shared-merge-tree) などのステートレスなテーブルエンジンを使用するデータベースを管理します。 これらのテーブルエンジンは永続的な状態をディスクに書き込まず、動的なコンピュート環境と互換性があります。 @@ -30,7 +30,7 @@ ClickHouse Cloud における `Shared` データベースエンジンは、ロ -## 構文 +## 構文 {#syntax} エンドユーザーが Shared Catalog と Shared データベースエンジンを利用する際に、特別な設定は必要ありません。データベースの作成手順は従来どおりです。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md index ec3f790a4fe..86e191d2bbb 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md @@ -10,13 +10,13 @@ doc_type: 'reference' -# SQLite +# SQLite {#sqlite} [SQLite](https://www.sqlite.org/index.html) データベースに接続し、`INSERT` および `SELECT` クエリを実行して、ClickHouse と SQLite 間でデータを交換できるようにします。 -## データベースの作成 +## データベースの作成 {#creating-a-database} ```sql CREATE DATABASE sqlite_database @@ -46,7 +46,7 @@ SQLite には、サービスとしての管理(起動スクリプトなど) -## 使用例 +## 使用例 {#usage-example} SQLite に接続された ClickHouse のデータベース: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/index.md index 2923e5a08cb..26eb7a236a0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/index.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# テーブルエンジン +# テーブルエンジン {#table-engines} テーブルエンジン(テーブルの種類)は、次の点を決定します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md index c0b4b97b07e..9f113595864 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md @@ -7,15 +7,11 @@ title: 'ExternalDistributed テーブルエンジン' doc_type: 'reference' --- - - -# ExternalDistributed テーブルエンジン +# ExternalDistributed テーブルエンジン {#externaldistributed-table-engine} `ExternalDistributed` エンジンを使用すると、リモートサーバー上の MySQL または PostgreSQL データベースに保存されているデータに対して `SELECT` クエリを実行できます。引数として [MySQL](../../../engines/table-engines/integrations/mysql.md) または [PostgreSQL](../../../engines/table-engines/integrations/postgresql.md) エンジンを指定できるため、シャーディングが可能です。 - - -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -42,8 +38,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `user` — ユーザー名。 * `password` — ユーザーのパスワード。 - -## 実装の詳細 +## 実装の詳細 {#implementation-details} 複数レプリカ構成をサポートしており、レプリカは `|` で、シャードは `,` で区切って列挙する必要があります。例えば次のようになります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md index 339f3533e35..9bc1782155d 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md @@ -9,14 +9,14 @@ doc_type: 'reference' -# ArrowFlight テーブルエンジン +# ArrowFlight テーブルエンジン {#arrowflight-table-engine} ArrowFlight テーブルエンジンを使用すると、ClickHouse は [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) プロトコル経由でリモートのデータセットに対してクエリを実行できます。 この統合により、ClickHouse は外部の Flight 対応サーバーから、列指向の Arrow 形式で高いパフォーマンスでデータを取得できます。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) (その場合、Arrow Flight サーバー側で未認証アクセスが許可されている必要があります)。 -## 使用例 +## 使用例 {#usage-example} この例では、リモートの Arrow Flight サーバーからデータを読み込むテーブルを作成する方法を示します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md index 1fe1381f930..3fd28dfe261 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md @@ -1,5 +1,5 @@ --- -description: 'このエンジンは Azure Blob Storage エコシステムとの統合を提供し、ストリーミングデータの取り込みを可能にします。' +description: 'このエンジンは Azure Blob Storage エコシステムとの統合を提供し、ストリーミングデータのインポートを可能にします。' sidebar_label: 'AzureQueue' sidebar_position: 181 slug: /engines/table-engines/integrations/azure-queue @@ -7,13 +7,9 @@ title: 'AzureQueue テーブルエンジン' doc_type: 'reference' --- +# AzureQueue テーブルエンジン {#azurequeue-table-engine} - -# AzureQueue テーブルエンジン - -このエンジンは [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) エコシステムとの統合を提供し、ストリーミングデータの取り込みを可能にします。 - - +このエンジンは [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) エコシステムとの統合を提供し、ストリーミングデータのインポートを可能にします。 ## テーブルの作成 {#creating-a-table} @@ -29,9 +25,9 @@ CREATE TABLE test (name String, value UInt32) **エンジンパラメータ** -`AzureQueue`のパラメータは`AzureBlobStorage`テーブルエンジンと同じです。パラメータの詳細については[こちら](../../../engines/table-engines/integrations/azureBlobStorage.md)を参照してください。 +`AzureQueue` のパラメータは、`AzureBlobStorage` テーブルエンジンでサポートされるものと同一です。パラメータについては[こちら](../../../engines/table-engines/integrations/azureBlobStorage.md)を参照してください。 -[AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage)テーブルエンジンと同様に、ローカルでのAzure Storage開発にはAzuriteエミュレータを使用できます。詳細については[こちら](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage)を参照してください。 +[AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage) テーブルエンジンと同様に、ローカル環境での Azure Storage 開発には Azurite エミュレーターを利用できます。詳細は[こちら](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage)を参照してください。 **例** @@ -45,22 +41,58 @@ ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1; SETTINGS mode = 'unordered' ``` +## Settings {#settings} + +サポートされている設定項目は、ほとんどが `S3Queue` テーブルエンジンと同じですが、`s3queue_` プレフィックスは付きません。[設定の全リスト](../../../engines/table-engines/integrations/s3queue.md#settings)を参照してください。 +テーブルに対して構成されている設定の一覧を取得するには、`system.azure_queue_settings` テーブルを使用します。`24.10` 以降で利用可能です。 + +以下は、AzureQueue にのみ対応し、S3Queue には適用されない設定です。 + +### `after_processing_move_connection_string` {#after_processing_move_connection_string} -## 設定 {#settings} +宛先が別の Azure コンテナーである場合に、正常に処理されたファイルを移動するための Azure Blob Storage 接続文字列。 -サポートされている設定は`S3Queue`テーブルエンジンと同じですが、`s3queue_`プレフィックスは付きません。[設定の完全なリスト](../../../engines/table-engines/integrations/s3queue.md#settings)を参照してください。 -テーブルに設定されている設定のリストを取得するには、`system.azure_queue_settings`テーブルを使用します。`24.10`から利用可能です。 +指定可能な値: +* 文字列。 -## Description {#description} +デフォルト値: 空文字列。 -`SELECT`はストリーミングインポートにはあまり有用ではありません(デバッグを除く)。各ファイルは一度しかインポートできないためです。[マテリアライズドビュー](../../../sql-reference/statements/create/view.md)を使用してリアルタイム処理を作成する方が実用的です。これを行うには: +### `after_processing_move_container` {#after_processing_move_container} -1. エンジンを使用してS3の指定されたパスから読み取るテーブルを作成し、それをデータストリームとして扱います。 -2. 必要な構造を持つテーブルを作成します。 -3. エンジンからデータを変換し、事前に作成したテーブルに格納するマテリアライズドビューを作成します。 +移動先が別の Azure コンテナである場合に、正常に処理されたファイルを移動する移動先コンテナ名。 -`MATERIALIZED VIEW`がエンジンに接続されると、バックグラウンドでデータの収集が開始されます。 +指定可能な値: + +* 文字列。 + +デフォルト値: 空文字列。 + +例: + +```sql +CREATE TABLE azure_queue_engine_table +( + `key` UInt64, + `data` String +) +ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', 'testcontainer', '*', 'CSV') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_move_connection_string = 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', + after_processing_move_container = 'dst-container'; +``` + +## 説明 {#description} + +`SELECT` は、各ファイルを 1 回しかインポートできないため(デバッグ用途を除き)ストリーミングインポートにはあまり有用ではありません。代わりに、[マテリアライズドビュー](../../../sql-reference/statements/create/view.md) を使用してリアルタイム処理フローを作成する方が実用的です。これを行うには、次のようにします。 + +1. エンジンを使用して、S3 内の指定パスからデータを取り込むテーブルを作成し、それをデータストリームとみなします。 +2. 目的の構造を持つテーブルを作成します。 +3. エンジンからのデータを変換し、事前に作成したテーブルに格納するマテリアライズドビューを作成します。 + +`MATERIALIZED VIEW` をエンジンと関連付けると、バックグラウンドでデータの取り込みを開始します。 例: @@ -79,61 +111,59 @@ CREATE MATERIALIZED VIEW consumer TO stats SELECT * FROM stats ORDER BY key; ``` - ## 仮想カラム {#virtual-columns} -- `_path` — ファイルへのパス -- `_file` — ファイル名 - -仮想カラムの詳細については、[こちら](../../../engines/table-engines/index.md#table_engines-virtual_columns)を参照してください。 +* `_path` — ファイルパス。 +* `_file` — ファイル名。 +仮想カラムの詳細については[こちら](../../../engines/table-engines/index.md#table_engines-virtual_columns)を参照してください。 ## イントロスペクション {#introspection} -テーブル設定 `enable_logging_to_queue_log=1` でテーブルのログ記録を有効にします。 +テーブル設定 `enable_logging_to_queue_log=1` を有効にして、テーブルに対するログ記録を有効化します。 -イントロスペクション機能は [S3Queueテーブルエンジン](/engines/table-engines/integrations/s3queue#introspection) と同じですが、いくつかの相違点があります: +イントロスペクション機能は [S3Queue テーブルエンジン](/engines/table-engines/integrations/s3queue#introspection) と同じですが、いくつか明確な違いがあります: -1. サーババージョン >= 25.1 では、キューのインメモリ状態に `system.azure_queue` を使用します。それ以前のバージョンでは `system.s3queue` を使用します(`azure` テーブルの情報も含まれます)。 -2. メインのClickHouse設定で `system.azure_queue_log` を有効にします。例: +1. サーバーバージョンが >= 25.1 の場合、キューのインメモリ状態には `system.azure_queue` を使用します。古いバージョンでは `system.s3queue` を使用します(こちらにも `azure` テーブルに関する情報が含まれます)。 +2. メインの ClickHouse 設定で `system.azure_queue_log` を有効化します。例: ```xml - - system - azure_queue_log
-
+ + system + azure_queue_log
+
``` -この永続テーブルは `system.s3queue` と同じ情報を持ちますが、処理済みおよび失敗したファイルに関するものです。 +この永続テーブルは、`system.s3queue` と同様の情報を保持しますが、対象は処理済みおよび失敗したファイルです。 -テーブルは以下の構造を持ちます: +このテーブルの構造は次のとおりです。 ```sql CREATE TABLE system.azure_queue_log ( `hostname` LowCardinality(String) COMMENT 'ホスト名', - `event_date` Date COMMENT 'このログ行を書き込んだイベント日付', - `event_time` DateTime COMMENT 'このログ行を書き込んだイベント時刻', - `database` String COMMENT '現在のS3Queueテーブルが存在するデータベースの名前', - `table` String COMMENT 'S3Queueテーブルの名前', + `event_date` Date COMMENT 'このログ行の書き込みイベント日付', + `event_time` DateTime COMMENT 'このログ行の書き込みイベント時刻', + `database` String COMMENT '現在のS3Queueテーブルが存在するデータベース名。', + `table` String COMMENT 'S3Queueテーブル名。', `uuid` String COMMENT 'S3QueueテーブルのUUID', - `file_name` String COMMENT '処理ファイルのファイル名', + `file_name` String COMMENT '処理対象ファイルのファイル名', `rows_processed` UInt64 COMMENT '処理された行数', - `status` Enum8('Processed' = 0, 'Failed' = 1) COMMENT '処理ファイルのステータス', + `status` Enum8('Processed' = 0, 'Failed' = 1) COMMENT 'ファイル処理のステータス', `processing_start_time` Nullable(DateTime) COMMENT 'ファイル処理の開始時刻', `processing_end_time` Nullable(DateTime) COMMENT 'ファイル処理の終了時刻', - `exception` String COMMENT '発生した例外メッセージ' + `exception` String COMMENT '例外が発生した場合の例外メッセージ' ) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) SETTINGS index_granularity = 8192 -COMMENT 'S3Queueエンジンによって処理されたファイルの情報を含むログエントリを格納します。' +COMMENT 'S3Queueエンジンによって処理されるファイルの情報を含むログエントリを格納する。' ``` -例: +例: ```sql SELECT * @@ -141,7 +171,7 @@ FROM system.azure_queue_log LIMIT 1 FORMAT Vertical -Row 1: +行 1: ────── hostname: clickhouse event_date: 2024-12-16 @@ -156,6 +186,6 @@ processing_start_time: 2024-12-16 13:42:47 processing_end_time: 2024-12-16 13:42:47 exception: -1 row in set. Elapsed: 0.002 sec. +1行が結果セットに含まれています。経過時間: 0.002秒。 ``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md index c07db77f886..72d6c5b2374 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# AzureBlobStorage テーブルエンジン +# AzureBlobStorage テーブルエンジン {#azureblobstorage-table-engine} このエンジンは、[Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) エコシステムとの統合機能を提供します。 -## テーブルを作成する +## テーブルを作成する {#create-table} ```sql CREATE TABLE azure_blob_storage_table (name String, value UInt32) @@ -24,7 +24,7 @@ CREATE TABLE azure_blob_storage_table (name String, value UInt32) [SETTINGS ...] ``` -### エンジンパラメータ +### エンジンパラメータ {#engine-parameters} * `endpoint` — コンテナおよびプレフィックスを含む Azure Blob Storage のエンドポイント URL。使用する認証方式で必要な場合は、任意で account_name を含めることもできます(`http://azurite1:{port}/[account_name]{container_name}/{data_prefix}`)。あるいは、これらのパラメータを storage_account_url、account_name、container を用いて個別に指定することもできます。プレフィックスを指定する場合は、endpoint を使用する必要があります。 * `endpoint_contains_account_name` - endpoint に account_name が含まれているかどうかを指定するためのフラグです。これは特定の認証方式でのみ必要となります(デフォルト: true)。 @@ -70,7 +70,7 @@ SELECT * FROM test_table; -## 認証 +## 認証 {#authentication} 現在、認証方法は 3 つあります: @@ -78,7 +78,7 @@ SELECT * FROM test_table; * `SAS Token` — `endpoint`、`connection_string`、または `storage_account_url` を指定することで利用できます。URL 内に '?' が含まれていることで識別されます。例については [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage#using-shared-access-signatures-sas-sas-tokens) を参照してください。 * `Workload Identity` — `endpoint` または `storage_account_url` を指定することで利用できます。設定で `use_workload_identity` パラメータが指定されている場合、認証には [workload identity](https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/identity/azure-identity#authenticate-azure-hosted-applications) が使用されます。 -### データキャッシュ +### データキャッシュ {#data-cache} `Azure` テーブルエンジンはローカルディスク上でのデータキャッシュをサポートします。 ファイルシステムキャッシュの設定オプションと利用方法については、この [セクション](/operations/storing-data.md/#using-local-cache) を参照してください。 @@ -107,13 +107,13 @@ SETTINGS filesystem_cache_name = 'cache_for_azure', enable_filesystem_cache = 1; 2. ClickHouse の `storage_configuration` セクションで定義されたキャッシュ設定(およびキャッシュストレージ)を再利用します。詳細は[こちら](/operations/storing-data.md/#using-local-cache)を参照してください。 -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — 任意です。ほとんどの場合、パーティションキーは不要であり、必要な場合でも月単位より細かいパーティションキーが必要になることは通常ありません。パーティショニングは(ORDER BY 式とは対照的に)クエリの高速化には寄与しません。パーティションを細かくし過ぎてはいけません。データをクライアント識別子やクライアント名でパーティション分割しないでください(代わりに、クライアント識別子または名前を ORDER BY 式の先頭のカラムにします)。 月単位でパーティショニングするには、`toYYYYMM(date_column)` 式を使用します。ここで `date_column` は型が [Date](/sql-reference/data-types/date.md) の日付カラムです。ここでのパーティション名は `"YYYYMM"` 形式になります。 -#### パーティション戦略 +#### パーティション戦略 {#partition-strategy} `WILDCARD`(デフォルト):ファイルパス内の `{_partition_id}` ワイルドカードを実際のパーティションキーに置き換えます。読み取りはサポートされていません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md index ced4221a56e..27330a45065 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# DeltaLake テーブルエンジン +# DeltaLake テーブルエンジン {#deltalake-table-engine} このエンジンは、Amazon S3 上に存在する既存の [Delta Lake](https://github.com/delta-io/delta) テーブルとの読み取り専用の連携を提供します。 -## テーブルを作成する +## テーブルを作成する {#create-table} Delta Lake テーブルはあらかじめ S3 上に存在している必要があり、このコマンドでは新しいテーブルを作成するための DDL パラメータは指定できません。 @@ -55,7 +55,7 @@ CREATE TABLE deltalake ENGINE=DeltaLake('http://mars-doc-test.s3.amazonaws.com/c CREATE TABLE deltalake ENGINE=DeltaLake(deltalake_conf, filename = 'test_table') ``` -### データキャッシュ +### データキャッシュ {#data-cache} `Iceberg` テーブルエンジンおよびテーブル関数は、`S3`、`AzureBlobStorage`、`HDFS` ストレージと同様に、データキャッシュをサポートします。詳細は[こちら](../../../engines/table-engines/integrations/s3.md#data-cache)を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md index b4bcbbdefcf..a825a735218 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# EmbeddedRocksDB テーブルエンジン +# EmbeddedRocksDB テーブルエンジン {#embeddedrocksdb-table-engine} このエンジンを使用すると、ClickHouse を [RocksDB](http://rocksdb.org/) と統合できます。 - - -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,8 +56,7 @@ ENGINE = EmbeddedRocksDB PRIMARY KEY key ``` - -## メトリクス +## メトリクス {#metrics} RocksDB の統計情報を提供する `system.rocksdb` というテーブルもあります。 @@ -76,8 +72,7 @@ FROM system.rocksdb └───────────────────────────┴───────┘ ``` - -## 設定 +## 設定 {#configuration} `config` を使用して、[rocksdb の任意のオプション](https://github.com/facebook/rocksdb/wiki/Option-String-and-Option-Map) を変更することもできます。 @@ -107,10 +102,9 @@ FROM system.rocksdb `optimize_trivial_approximate_count_query = 1` を設定します。また、この設定は EmbeddedRocksDB エンジンにおける `system.tables` にも影響し、 `total_rows` および `total_bytes` の近似値を表示するには、この設定を有効にしておく必要があります。 +## サポートされている操作 {#supported-operations} -## サポートされている操作 - -### 挿入 +### 挿入 {#inserts} 新しい行が `EmbeddedRocksDB` に挿入されると、キーがすでに存在している場合は値が更新され、存在しない場合は新しいキーが作成されます。 @@ -120,7 +114,7 @@ FROM system.rocksdb INSERT INTO test VALUES ('some key', 1, 'value', 3.2); ``` -### 削除 +### 削除 {#deletes} 行は `DELETE` クエリまたは `TRUNCATE` クエリを使用して削除できます。 @@ -136,7 +130,7 @@ ALTER TABLE test DELETE WHERE key LIKE 'some%' AND v1 > 1; テーブル test の全データを削除; ``` -### 更新 +### 更新 {#updates} 値は `ALTER TABLE` クエリを使用して更新できます。主キーは変更できません。 @@ -144,7 +138,7 @@ ALTER TABLE test DELETE WHERE key LIKE 'some%' AND v1 > 1; ALTER TABLE test UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; ``` -### 結合 +### 結合 {#joins} EmbeddedRocksDB テーブルでは、特殊な `direct` 結合がサポートされています。 この `direct` 結合ではメモリ内にハッシュテーブルを構築せず、 @@ -163,9 +157,9 @@ SET join_algorithm = 'direct, hash' `join_algorithm` が `direct, hash` に設定されている場合、可能なときは direct join が使用され、それ以外の場合は hash join が使用されます。 ::: -#### 例 +#### 例 {#example} -##### EmbeddedRocksDB テーブルを作成してデータを投入する +##### EmbeddedRocksDB テーブルを作成してデータを投入する {#create-and-populate-an-embeddedrocksdb-table} ```sql CREATE TABLE rdb @@ -187,7 +181,7 @@ INSERT INTO rdb FROM numbers_mt(10); ``` -##### `rdb` テーブルと結合するためのテーブルを作成し、データを投入する +##### `rdb` テーブルと結合するためのテーブルを作成し、データを投入する {#create-and-populate-a-table-to-join-with-table-rdb} ```sql CREATE TABLE t2 @@ -202,13 +196,13 @@ INSERT INTO t2 SELECT number AS k FROM numbers_mt(10) ``` -##### 結合アルゴリズムを `direct` に設定 +##### 結合アルゴリズムを `direct` に設定 {#set-the-join-algorithm-to-direct} ```sql SET join_algorithm = 'direct' ``` -##### INNER JOIN の例 +##### INNER JOIN の例 {#an-inner-join} ```sql SELECT * @@ -233,7 +227,7 @@ ORDER BY key ASC └─────┴─────────┴────────┴────────┘ ``` -### JOIN の詳細情報 +### JOIN の詳細情報 {#more-information-on-joins} * [`join_algorithm` 設定](/operations/settings/settings.md#join_algorithm) * [JOIN 句](/sql-reference/statements/select/join.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md index c5cc68f30aa..88c01de61de 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# HDFS テーブルエンジン +# HDFS テーブルエンジン {#hdfs-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 使用方法 +## 使用方法 {#usage} ```sql ENGINE = HDFS(URI, format) @@ -34,7 +34,7 @@ ENGINE = HDFS(URI, format) [Formats](/sql-reference/formats#formats-overview) セクションに一覧されています。 * [PARTITION BY expr] -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — 任意です。ほとんどの場合、パーティションキーは不要であり、必要な場合でも、一般的には月単位より細かいパーティションキーは不要です。パーティショニングは(ORDER BY 式とは対照的に)クエリを高速化しません。細かすぎるパーティショニングは決して行うべきではありません。クライアント ID や名前でデータをパーティションしないでください(代わりに、ORDER BY 式の先頭のカラムとしてクライアント ID または名前を指定してください)。 @@ -68,7 +68,7 @@ SELECT * FROM hdfs_engine_table LIMIT 2 ``` -## 実装の詳細 +## 実装の詳細 {#implementation-details} * 読み取りと書き込みは並列に実行できます。 * 次の機能はサポートされません: @@ -136,7 +136,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ENGINE = HDFS('hdfs CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9000/big_dir/file{0..9}{0..9}{0..9}', 'CSV') ``` -## 設定 +## 設定 {#configuration} GraphiteMergeTree と同様に、HDFS エンジンでは ClickHouse の設定ファイルを用いた拡張的な設定が可能です。使用できる設定キーは 2 種類あり、グローバル (`hdfs`) とユーザーレベル (`hdfs_*`) です。最初にグローバル設定が適用され、その後に(存在する場合は)ユーザーレベルの設定が適用されます。 @@ -154,9 +154,9 @@ GraphiteMergeTree と同様に、HDFS エンジンでは ClickHouse の設定フ ``` -### 設定オプション +### 設定オプション {#configuration-options} -#### libhdfs3 がサポートする項目 +#### libhdfs3 がサポートする項目 {#supported-by-libhdfs3} | **parameter** | **default value** | @@ -230,7 +230,7 @@ libhdfs3 の制限により、古典的な方式のみがサポートされて `hadoop_kerberos_keytab`、`hadoop_kerberos_principal` または `hadoop_security_kerberos_ticket_cache_path` が指定されている場合、Kerberos 認証が使用されます。この場合、`hadoop_kerberos_keytab` と `hadoop_kerberos_principal` は必須となります。 -## HDFS NameNode HA サポート +## HDFS NameNode HA サポート {#namenode-ha} libhdfs3 は HDFS NameNode の HA をサポートします。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md index 40d5b433858..b727f9daf34 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md @@ -9,22 +9,19 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# Hive テーブルエンジン {#hive-table-engine} -# Hive テーブルエンジン - - + Hive エンジンを使用すると、HDFS 上の Hive テーブルに対して `SELECT` クエリを実行できます。現在、以下の入力フォーマットをサポートしています。 -- Text: `binary` を除く単純なスカラー列型のみをサポート - -- ORC: `char` を除く単純なスカラー列型をサポートし、複合型は `array` のみサポート - -- Parquet: すべての単純なスカラー列型をサポートし、複合型は `array` のみサポート +* Text: `binary` を除く単純なスカラー列型のみをサポート +* ORC: `char` を除く単純なスカラー列型をサポートし、複合型は `array` のみサポート +* Parquet: すべての単純なスカラー列型をサポートし、複合型は `array` のみサポート -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -52,10 +49,9 @@ PARTITION BY expr * `table` — リモートテーブル名。 +## 使用例 {#usage-example} -## 使用例 - -### HDFS ファイルシステムでローカルキャッシュを使用する方法 +### HDFS ファイルシステムでローカルキャッシュを使用する方法 {#how-to-use-local-cache-for-hdfs-filesystem} リモートファイルシステムに対しては、ローカルキャッシュを有効にすることを強く推奨します。ベンチマークでは、キャッシュありの場合はほぼ 2 倍高速になることが示されています。 @@ -75,9 +71,9 @@ PARTITION BY expr * limit_size: 必須。ローカルキャッシュファイルの最大サイズ (バイト単位) です。 * bytes_read_before_flush: リモートファイルシステムからファイルをダウンロードする際に、ローカルファイルシステムへフラッシュするまでの読み取りバイト数を制御します。デフォルト値は 1MB です。 -### ORC 入力フォーマットで Hive テーブルをクエリする +### ORC 入力フォーマットで Hive テーブルをクエリする {#query-hive-table-with-orc-input-format} -#### Hive でテーブルを作成する +#### Hive でテーブルを作成する {#create-table-in-hive} ```text hive > CREATE TABLE `test`.`test_orc`( @@ -125,8 +121,7 @@ OK Time taken: 0.295 seconds, Fetched: 1 row(s) ``` -#### ClickHouse でテーブルを作成 - +#### ClickHouse でテーブルを作成 {#create-table-in-clickhouse} 次の ClickHouse テーブルは、上で作成した Hive テーブルからデータを取得します: @@ -199,9 +194,9 @@ day: 2021-09-18 1 rows in set. Elapsed: 0.078 sec. ``` -### Parquet 入力フォーマットの Hive テーブルをクエリする +### Parquet 入力フォーマットの Hive テーブルをクエリする {#query-hive-table-with-parquet-input-format} -#### Hive でテーブルを作成する +#### Hive でテーブルを作成する {#create-table-in-hive-1} ```text hive > @@ -241,7 +236,6 @@ OK Time taken: 0.51 seconds ``` - hive > insert into test.test_parquet partition(day='2021-09-18') select 1, 2, 3, 4, 5, 6.11, 7.22, 8.333, current_timestamp(), current_date(), 'hello world', 'hello world', 'hello world', true, 'hello world', array(1, 2, 3), array('hello world', 'hello world'), array(float(1.1), float(1.2)), array(array(1, 2), array(3, 4)), array(array('a', 'b'), array('c', 'd')), array(array(float(1.11), float(2.22)), array(float(3.33), float(4.44))); OK 処理時間: 36.025秒 @@ -329,7 +323,6 @@ day: 2021-09-18 #### Hive でテーブルを作成する - ```text hive > CREATE TABLE `test`.`test_text`( @@ -377,7 +370,7 @@ OK Time taken: 0.624 seconds, Fetched: 1 row(s) ``` -#### ClickHouse でテーブルを作成する +#### ClickHouse でテーブルを作成する {#create-table-in-hive-2} 上で作成した Hive テーブルからデータを取得する ClickHouse テーブル: @@ -416,7 +409,6 @@ SETTINGS input_format_skip_unknown_fields = 1, input_format_with_names_use_heade Query id: 55b79d35-56de-45b9-8be6-57282fbf1f44 ``` - Row 1: ────── f_tinyint: 1 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md index 727502e2e99..594a2acc67a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Hudi テーブルエンジン +# Hudi テーブルエンジン {#hudi-table-engine} このエンジンは、Amazon S3 上の既存の Apache [Hudi](https://hudi.apache.org/) テーブルと読み取り専用で統合する機能を提供します。 -## テーブルを作成 +## テーブルを作成 {#create-table} Hudi テーブルはあらかじめ S3 上に存在している必要がある点に注意してください。このコマンドでは、新しいテーブルを作成するための DDL パラメータを指定することはできません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md index 406ce597883..34ea2f0dff7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md @@ -23,7 +23,7 @@ Iceberg Table Engine も利用可能ですが、いくつかの制限があり -## テーブルを作成 +## テーブルを作成 {#create-table} Iceberg テーブルはあらかじめストレージ上に存在している必要がある点に注意してください。このコマンドには、新しいテーブルを作成するための DDL パラメータを指定できません。 @@ -42,14 +42,14 @@ CREATE TABLE iceberg_table_local ``` -## エンジン引数 +## エンジン引数 {#engine-arguments} 引数の説明は、それぞれ `S3`、`AzureBlobStorage`、`HDFS` および `File` エンジンにおける引数の説明と同様です。 `format` は Iceberg テーブル内のデータファイルの形式を表します。 エンジンパラメータは [Named Collections](../../../operations/named-collections.md) を使用して指定できます。 -### 例 +### 例 {#example} ```sql CREATE TABLE iceberg_table ENGINE=IcebergS3('http://test.s3.amazonaws.com/clickhouse-bucket/test_table', 'test', 'test') @@ -105,7 +105,7 @@ ClickHouse は Iceberg テーブルに対するタイムトラベルをサポー -## 削除行を含むテーブルの処理 +## 削除行を含むテーブルの処理 {#deleted-rows} 現在サポートされているのは、[position deletes](https://iceberg.apache.org/spec/#position-delete-files) を使用する Iceberg テーブルのみです。 @@ -114,7 +114,7 @@ ClickHouse は Iceberg テーブルに対するタイムトラベルをサポー * [Equality deletes](https://iceberg.apache.org/spec/#equality-delete-files) * [Deletion vectors](https://iceberg.apache.org/spec/#deletion-vectors)(v3 で導入) -### 基本的な使用方法 +### 基本的な使用方法 {#basic-usage} ```sql SELECT * FROM example_table ORDER BY 1 @@ -128,7 +128,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 注意: 同じクエリ内で `iceberg_timestamp_ms` パラメータと `iceberg_snapshot_id` パラメータを同時に指定することはできません。 -### 重要な考慮事項 +### 重要な考慮事項 {#important-considerations} * **スナップショット** は通常、次のタイミングで作成されます: * 新しいデータがテーブルに書き込まれたとき @@ -136,11 +136,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * **スキーマ変更によってスナップショットが作成されることは通常ない** — このため、スキーマ進化を行ったテーブルでタイムトラベルを使用する場合に特有の挙動が発生します。 -### シナリオ例 +### シナリオ例 {#example-scenarios} すべてのシナリオは Spark を用いて記述されています。これは、CH がまだ Iceberg テーブルへの書き込みをサポートしていないためです。 -#### シナリオ 1: 新しいスナップショットを伴わないスキーマ変更 +#### シナリオ 1: 新しいスナップショットを伴わないスキーマ変更 {#scenario-1} 次の一連の操作を考えてみます: @@ -200,7 +200,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * ts1 と ts2 の時点: 元の 2 列のみが表示される * ts3 の時点: 3 列すべてが表示され、1 行目の price 列は NULL になる -#### シナリオ 2: 履歴スキーマと現在のスキーマの差異 +#### シナリオ 2: 履歴スキーマと現在のスキーマの差異 {#scenario-2} 現在時点を指定したタイムトラベルクエリでは、現在のテーブルとは異なるスキーマが表示される場合があります: @@ -243,7 +243,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 これは、`ALTER TABLE` が新しいスナップショットを作成せず、現在のテーブルについては Spark がスナップショットではなく最新のメタデータファイルから `schema_id` の値を取得するために発生します。 -#### シナリオ 3: 過去と現在のスキーマの差異 +#### シナリオ 3: 過去と現在のスキーマの差異 {#scenario-3} 2つ目の制約は、タイムトラベルを行っても、テーブルに最初のデータが書き込まれる前の状態は取得できないという点です。 @@ -265,11 +265,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 ClickHouse における挙動は Spark と同じです。概念的には Spark の SELECT クエリを ClickHouse の SELECT クエリに置き換えて考えれば、同じように動作します。 -## メタデータファイルの解決 +## メタデータファイルの解決 {#metadata-file-resolution} ClickHouse で `Iceberg` テーブルエンジンを使用する場合、システムは Iceberg テーブル構造を記述する適切な metadata.json ファイルを特定する必要があります。以下は、この解決プロセスの概要です。 -### 候補の検索 +### 候補の検索 {#candidate-search} 1. **パスの直接指定**: @@ -286,7 +286,7 @@ ClickHouse で `Iceberg` テーブルエンジンを使用する場合、シス * 上記いずれの設定も指定されていない場合、`metadata` ディレクトリ内のすべての `.metadata.json` ファイルが候補になります -### 最新のファイルの選択 +### 最新のファイルの選択 {#most-recent-file} 上記のルールで候補ファイルを特定した後、システムはその中で最も新しいものを決定します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md deleted file mode 100644 index 1723df5a009..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: '統合向けテーブルエンジンのドキュメント' -sidebar_label: '統合' -sidebar_position: 40 -slug: /engines/table-engines/integrations/ -title: '統合向けテーブルエンジン' -doc_type: 'reference' ---- - - - -# 連携向けテーブルエンジン - -ClickHouse は、テーブルエンジンを含むさまざまな手段で外部システムと連携できます。他のすべてのテーブルエンジンと同様に、設定は `CREATE TABLE` または `ALTER TABLE` クエリを使って行います。ユーザーの視点から見ると、設定された連携は通常のテーブルのように見えますが、そのテーブルへのクエリは外部システムにプロキシされます。この透過的なクエリ実行は、辞書やテーブル関数のような代替的な連携方法に比べた、このアプローチの主な利点の 1 つです。代替方法では、利用のたびにカスタムクエリ手法を用いる必要があります。 - -{/* このページの目次テーブルは、YAML フロントマターのフィールド slug、description、title から - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - によって自動生成されます。 - - 誤りを見つけた場合は、対象ページの YAML フロントマターを編集してください。 - */ } - - -{/*AUTOGENERATED_START*/ } - -| ページ | 概要 | -| ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| [AzureBlobStorage テーブルエンジン](/engines/table-engines/integrations/azureBlobStorage) | このエンジンは、Azure Blob Storage エコシステムとの統合機能を提供します。 | -| [DeltaLake テーブルエンジン](/engines/table-engines/integrations/deltalake) | このエンジンにより、Amazon S3 上の既存の Delta Lake テーブルと読み取り専用で連携できます。 | -| [EmbeddedRocksDB テーブルエンジン](/engines/table-engines/integrations/embedded-rocksdb) | このエンジンを使用すると、ClickHouse と RocksDB を統合できます | -| [ExternalDistributed テーブルエンジン](/engines/table-engines/integrations/ExternalDistributed) | `ExternalDistributed` エンジンを使用すると、リモートサーバー上の MySQL または PostgreSQL に格納されたデータに対して `SELECT` クエリを実行できます。MySQL または PostgreSQL エンジンを引数として受け取るため、シャーディングが可能です。 | -| [TimeSeries テーブルエンジン](/engines/table-engines/special/time_series) | タイムスタンプおよびタグ(またはラベル)に関連付けられた値の集合としての時系列データを格納するテーブルエンジン。 | -| [HDFS テーブルエンジン](/engines/table-engines/integrations/hdfs) | このエンジンは、ClickHouse 経由で HDFS 上のデータを管理できるようにすることで、Apache Hadoop エコシステムとの統合を実現します。このエンジンは File エンジンおよび URL エンジンに類似していますが、Hadoop 固有の機能を提供します。 | -| [Hive テーブルエンジン](/engines/table-engines/integrations/hive) | Hive エンジンを使用すると、HDFS 上の Hive テーブルに対して `SELECT` クエリを実行できます。 | -| [Hudi テーブルエンジン](/engines/table-engines/integrations/hudi) | このエンジンは、Amazon S3 上の既存の Apache Hudi テーブルとの読み取り専用統合を提供します。 | -| [Iceberg テーブルエンジン](/engines/table-engines/integrations/iceberg) | このエンジンは、Amazon S3、Azure、HDFS およびローカルに保存された既存の Apache Iceberg テーブルと読み取り専用で連携します。 | -| [JDBC テーブルエンジン](/engines/table-engines/integrations/jdbc) | ClickHouse が JDBC を介して外部データベースに接続できるようにします。 | -| [Kafka テーブルエンジン](/engines/table-engines/integrations/kafka) | Kafka Table Engine は Apache Kafka と連携して使用でき、データフローの公開および購読、耐障害性のあるストレージの構成、そしてストリームが利用可能になり次第その処理を行うことができます。 | -| [MaterializedPostgreSQL テーブルエンジン](/engines/table-engines/integrations/materialized-postgresql) | PostgreSQL テーブルのデータを初期ダンプして ClickHouse テーブルを作成し、レプリケーションを開始します。 | -| [MongoDB テーブルエンジン](/engines/table-engines/integrations/mongodb) | MongoDB エンジンは、リモートのコレクションからデータを読み出すための読み取り専用テーブルエンジンです。 | -| [MySQL テーブルエンジン](/engines/table-engines/integrations/mysql) | MySQL テーブルエンジンに関するドキュメント | -| [NATS テーブルエンジン](/engines/table-engines/integrations/nats) | このエンジンを使用すると、ClickHouse を NATS と連携させてメッセージのサブジェクトを公開・購読し、新着メッセージを利用可能になり次第処理できます。 | -| [ODBC テーブルエンジン](/engines/table-engines/integrations/odbc) | ClickHouse が ODBC 経由で外部データベースに接続できるようにします。 | -| [PostgreSQL テーブルエンジン](/engines/table-engines/integrations/postgresql) | PostgreSQL エンジンを使用すると、リモートの PostgreSQL サーバー上に保存されたデータに対して `SELECT` および `INSERT` クエリを実行できます。 | -| [RabbitMQ テーブルエンジン](/engines/table-engines/integrations/rabbitmq) | このエンジンを使用すると、ClickHouse を RabbitMQ と統合できます。 | -| [Redis テーブルエンジン](/engines/table-engines/integrations/redis) | このエンジンを使用すると、ClickHouse を Redis と統合できます。 | -| [S3 テーブルエンジン](/engines/table-engines/integrations/s3) | このエンジンは Amazon S3 エコシステムとの統合を提供します。HDFS エンジンに似ていますが、S3 に特化した機能も備えています。 | -| [S3Queue テーブルエンジン](/engines/table-engines/integrations/s3queue) | このエンジンは Amazon S3 エコシステムと統合されており、ストリーミングによるインポートをサポートします。Kafka や RabbitMQ エンジンと同様ですが、S3 固有の機能を備えています。 | -| [AzureQueue テーブルエンジン](/engines/table-engines/integrations/azure-queue) | このエンジンは Azure Blob Storage エコシステムと統合されており、ストリーミングによるデータ取り込みをサポートします。 | -| [YTsaurus テーブルエンジン](/engines/table-engines/integrations/ytsaurus) | YTsaurus クラスターからデータをインポートするためのテーブルエンジン。 | -| [SQLite テーブルエンジン](/engines/table-engines/integrations/sqlite) | このエンジンでは、SQLite との間でデータのインポートおよびエクスポートを行え、ClickHouse から SQLite のテーブルに対して直接クエリを実行できます。 | -| [ArrowFlight テーブルエンジン](/engines/table-engines/integrations/arrowflight) | このエンジンは、Apache Arrow Flight を介してリモートデータセットに対してクエリを実行できます。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md index 84e338b3832..04a8ac2dc66 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# JDBC テーブルエンジン +# JDBC テーブルエンジン {#jdbc-table-engine} @@ -27,7 +27,7 @@ JDBC 接続を実現するために、ClickHouse はデーモンとして実行 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -51,7 +51,7 @@ ENGINE = JDBC(datasource, external_database, external_table) * これらのパラメータは、[名前付きコレクション](operations/named-collections.md)を使用して指定することもできます。 -## 使用例 +## 使用例 {#usage-example} コンソールクライアントから MySQL サーバーに直接接続してテーブルを作成します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md index 591f990a11d..722a8726e12 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md @@ -11,7 +11,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Kafka テーブルエンジン +# Kafka テーブルエンジン {#kafka-table-engine} :::tip ClickHouse Cloud をご利用の場合は、代わりに [ClickPipes](/integrations/clickpipes) の利用を推奨します。ClickPipes は、プライベートネットワーク接続のネイティブサポート、インジェスト処理とクラスタリソースを独立してスケールさせる機能、そして Kafka のストリーミングデータを ClickHouse に取り込むための包括的なモニタリング機能を提供します。 @@ -21,7 +21,7 @@ ClickHouse Cloud をご利用の場合は、代わりに [ClickPipes](/integrati - フォールトトレラントなストレージの構成。 - ストリームを利用可能になり次第処理。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -135,7 +135,7 @@ Kafka テーブルエンジンは[デフォルト値](/sql-reference/statements/ ::: -## 説明 +## 説明 {#description} 配信されたメッセージは自動的に追跡されるため、グループ内の各メッセージは 1 回だけカウントされます。データを 2 回取得したい場合は、別のグループ名でテーブルのコピーを作成してください。 @@ -186,7 +186,7 @@ Kafka テーブルエンジンは[デフォルト値](/sql-reference/statements/ `ALTER` を使用してターゲットテーブルを変更する場合は、ターゲットテーブルとビューからのデータとの不整合を避けるため、マテリアライズドビューを無効化することを推奨します。 -## 設定 +## 設定 {#configuration} GraphiteMergeTree と同様に、Kafka エンジンは ClickHouse の設定ファイルを用いた詳細な設定をサポートしています。使用できる設定キーは 2 種類あり、グローバル(`` の下)とトピックレベル(`` の下)です。まずグローバル設定が適用され、その後にトピックレベルの設定(存在する場合)が適用されます。 @@ -233,7 +233,7 @@ GraphiteMergeTree と同様に、Kafka エンジンは ClickHouse の設定フ 利用可能な設定オプションの一覧については、[librdkafka の configuration reference](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md) を参照してください。ClickHouse の設定では、ドットの代わりにアンダースコア(`_`)を使用します。たとえば、`check.crcs=true` は `true` に対応します。 -### Kerberos サポート +### Kerberos サポート {#kafka-kerberos-support} Kerberos 対応の Kafka を扱うには、`security_protocol` の子要素として `sasl_plaintext` を追加します。Kerberos のチケット授与チケット (TGT) が OS の機能によって取得・キャッシュされていれば十分です。 ClickHouse は keytab ファイルを使用して Kerberos 資格情報を管理できます。`sasl_kerberos_service_name`、`sasl_kerberos_keytab`、`sasl_kerberos_principal` の子要素を指定します。 @@ -276,7 +276,7 @@ Kafka エンジンは、ClickHouse でサポートされているすべての[ - 行ベースのフォーマットでは、1 つの Kafka メッセージ内の行数は `kafka_max_rows_per_message` の設定で制御できます。 - ブロックベースのフォーマットでは、ブロックをより小さな部分に分割することはできませんが、1 ブロック内の行数は共通設定 [max_block_size](/operations/settings/settings#max_block_size) によって制御できます。 -## ClickHouse Keeper にコミット済みオフセットを保存するエンジン +## ClickHouse Keeper にコミット済みオフセットを保存するエンジン {#engine-to-store-committed-offsets-in-clickhouse-keeper} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md index 9d30551efc8..d8f94e49658 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MaterializedPostgreSQL テーブルエンジン +# MaterializedPostgreSQL テーブルエンジン {#materializedpostgresql-table-engine} @@ -36,7 +36,7 @@ SET allow_experimental_materialized_postgresql_table=1 複数のテーブルが必要な場合は、テーブルエンジンではなく [MaterializedPostgreSQL](../../../engines/database-engines/materialized-postgresql.md) データベースエンジンを使用し、レプリケートするテーブルを指定する `materialized_postgresql_tables_list` 設定(将来的にはデータベースの `schema` も追加可能になる予定)を利用することを強く推奨します。これにより、CPU 使用量を抑えつつ、接続数およびリモート PostgreSQL データベース内のレプリケーションスロット数を減らすことができ、はるかに効率的になります。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE postgresql_db.postgresql_replica (key UInt64, value UInt64) @@ -65,7 +65,7 @@ PRIMARY KEY key; -## 仮想カラム +## 仮想カラム {#virtual-columns} * `_version` — トランザクションカウンター。型: [UInt64](../../../sql-reference/data-types/int-uint.md)。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md index 56b79eb09f0..69fa3f5efc1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# MongoDB テーブルエンジン +# MongoDB テーブルエンジン {#mongodb-table-engine} MongoDB エンジンは、リモートの [MongoDB](https://www.mongodb.com/) コレクションからデータを読み取るための、読み取り専用のテーブルエンジンです。 @@ -18,7 +18,7 @@ MongoDB v3.6 以降のサーバーのみがサポートされています。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -61,7 +61,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); | `oid_columns` | WHERE 句で `oid` として扱う列をカンマ区切りで指定します。既定では `_id` です。 | -## 型マッピング +## 型マッピング {#types-mappings} | MongoDB | ClickHouse | | ----------------------- | --------------------------------------------------- | @@ -78,7 +78,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); キーが MongoDB ドキュメント内に存在しない場合 (たとえばカラム名が一致しない場合)、デフォルト値または (カラムが Nullable の場合は) `NULL` が挿入されます。 -### OID +### OID {#oid} WHERE 句で `String` を `oid` として扱いたい場合は、テーブルエンジンの最後の引数にそのカラム名を指定します。 これは、MongoDB でデフォルトで `oid` 型を持つ `_id` カラムでレコードをクエリする際に必要になる場合があります。 @@ -132,7 +132,7 @@ SELECT count() FROM sample_oid WHERE another_oid_column = '67bf6cc40000000000ea4 ``` -## サポートされている句 +## サポートされている句 {#supported-clauses} 単純な式を含むクエリのみがサポートされます(例: `WHERE field = ORDER BY field2 LIMIT `)。 このような式は MongoDB のクエリ言語に変換され、サーバー側で実行されます。 @@ -158,7 +158,7 @@ SELECT * FROM mongo_table WHERE date = '2024-01-01'::Date OR date = toDate('2024 ::: -## 使用例 +## 使用例 {#usage-example} MongoDB に [sample_mflix](https://www.mongodb.com/docs/atlas/sample-data/sample-mflix) データセットが読み込まれていることを前提とします diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md index 1f74c72fd6f..26f96d206c9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# MySQL テーブルエンジン +# MySQL テーブルエンジン {#mysql-table-engine} MySQL エンジンを使用すると、リモートの MySQL サーバー上に保存されているデータに対して `SELECT` および `INSERT` クエリを実行できます。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -67,7 +67,7 @@ CREATE TABLE test_replicas (id UInt32, name String, age UInt32, money UInt32) EN ``` -## 使用例 +## 使用例 {#usage-example} MySQL でテーブルを作成します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md index 7b24cf139ee..bfed0ce9ab9 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md @@ -20,7 +20,7 @@ doc_type: 'guide' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -133,7 +133,7 @@ NATS サーバーの設定は、ClickHouse の設定ファイルに追加でき ``` -## 説明 +## 説明 {#description} 各メッセージは一度しか読み取れないため、(デバッグを除いて)メッセージの読み取りに `SELECT` を使ってもあまり有用ではありません。代わりに、[マテリアライズドビュー](../../../sql-reference/statements/create/view.md) を使ってリアルタイムの処理フローを作成する方が実用的です。そのためには、次の手順を実行します。 @@ -198,7 +198,7 @@ NATS エンジンは、ClickHouse がサポートするすべての[フォーマ -## JetStream の使用 +## JetStream の使用 {#using-jetstream} NATS JetStream とともに NATS エンジンを使用する前に、NATS ストリームと永続プルコンシューマを作成する必要があります。これには、[NATS CLI](https://github.com/nats-io/natscli) パッケージに含まれる `nats` ユーティリティなどを使用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md index 6bc95454554..587aa410d07 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# ODBC テーブルエンジン +# ODBC テーブルエンジン {#odbc-table-engine} @@ -22,7 +22,7 @@ ODBC 接続を安全に実装するために、ClickHouse は別のプログラ -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -51,7 +51,7 @@ ENGINE = ODBC(datasource, external_database, external_table) これらのパラメータは、[named collections](operations/named-collections.md) を使用して指定することもできます。 -## 使用例 +## 使用例 {#usage-example} **ODBC を介してローカルの MySQL インストールからデータを取得する** diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md index d781104c93a..869f84bbdd5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# PostgreSQL テーブルエンジン +# PostgreSQL テーブルエンジン {#postgresql-table-engine} PostgreSQLエンジンを使用すると、リモートのPostgreSQLサーバーに保存されたデータに対して `SELECT` および `INSERT` クエリを実行できます。 @@ -23,7 +23,7 @@ ClickHouse Cloud ユーザーには、Postgres データを ClickHouse にスト -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -73,7 +73,7 @@ SELECT * FROM postgresql(postgres_creds, table='table1'); ``` -## 実装の詳細 +## 実装の詳細 {#implementation-details} PostgreSQL 側での `SELECT` クエリは、読み取り専用の PostgreSQL トランザクション内で `COPY (SELECT ...) TO STDOUT` として実行され、各 `SELECT` クエリの後にコミットされます。 @@ -121,9 +121,9 @@ PostgreSQL の辞書ソースではレプリカの優先度指定がサポート ``` -## 使用例 +## 使用例 {#usage-example} -### PostgreSQL のテーブル +### PostgreSQL のテーブル {#table-in-postgresql} ```text postgres=# CREATE TABLE "public"."test" ( @@ -146,7 +146,7 @@ postgresql> SELECT * FROM test; (1 row) ``` -### ClickHouse でテーブルを作成し、前述の PostgreSQL テーブルに接続する +### ClickHouse でテーブルを作成し、前述の PostgreSQL テーブルに接続する {#creating-table-in-clickhouse-and-connecting-to--postgresql-table-created-above} この例では、[PostgreSQL table engine](/engines/table-engines/integrations/postgresql.md) を使用して、ClickHouse テーブルを PostgreSQL テーブルに接続し、PostgreSQL データベースに対して SELECT 文および INSERT 文を実行します。 @@ -160,7 +160,7 @@ CREATE TABLE default.postgresql_table ENGINE = PostgreSQL('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### SELECT クエリを使用して PostgreSQL テーブルから ClickHouse テーブルへ初期データを挿入する +### SELECT クエリを使用して PostgreSQL テーブルから ClickHouse テーブルへ初期データを挿入する {#inserting-initial-data-from-postgresql-table-into-clickhouse-table-using-a-select-query} [postgresql テーブル関数](/sql-reference/table-functions/postgresql.md) は PostgreSQL から ClickHouse へデータをコピーします。これは、多くの場合、PostgreSQL ではなく ClickHouse 上でクエリや分析を実行することでデータのクエリパフォーマンスを向上させるため、あるいは PostgreSQL から ClickHouse へデータを移行するために使用されます。ここでは PostgreSQL から ClickHouse へデータをコピーするため、ClickHouse では MergeTree テーブルエンジンを使用したテーブルを作成し、名前を postgresql_copy とします。 @@ -180,7 +180,7 @@ INSERT INTO default.postgresql_copy SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### PostgreSQL テーブルから ClickHouse テーブルへの増分データ挿入 +### PostgreSQL テーブルから ClickHouse テーブルへの増分データ挿入 {#inserting-incremental-data-from-postgresql-table-into-clickhouse-table} 初回の挿入後に PostgreSQL テーブルと ClickHouse テーブルの間で継続的な同期を行う場合、ClickHouse 側で `WHERE` 句を使用して、タイムスタンプまたは一意のシーケンス ID に基づき、PostgreSQL に新たに追加されたデータのみを挿入できます。 @@ -198,7 +198,7 @@ SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postges_user', 'po WHERE int_id > maxIntID; ``` -### 生成された ClickHouse テーブルからデータを選択する +### 生成された ClickHouse テーブルからデータを選択する {#selecting-data-from-the-resulting-clickhouse-table} ```sql SELECT * FROM postgresql_copy WHERE str IN ('test'); @@ -210,7 +210,7 @@ SELECT * FROM postgresql_copy WHERE str IN ('test'); └────────────────┴──────┴────────┘ ``` -### デフォルト以外のスキーマを使用する +### デフォルト以外のスキーマを使用する {#using-non-default-schema} ```text postgres=# CREATE SCHEMA "nice.schema"; diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md index 056cf863842..de3c7cff726 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# RabbitMQ テーブルエンジン +# RabbitMQ テーブルエンジン {#rabbitmq-table-engine} このエンジンを使用すると、ClickHouse を [RabbitMQ](https://www.rabbitmq.com) と統合できます。 @@ -20,7 +20,7 @@ doc_type: 'guide' -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -131,7 +131,7 @@ RabbitMQ サーバーの設定は、ClickHouse の設定ファイルに追加す ``` -## 説明 +## 説明 {#description} 各メッセージは一度しか読み取れないため、メッセージの読み取りに `SELECT` を使うのは(デバッグ用途を除き)あまり有用ではありません。代わりに、[マテリアライズドビュー](../../../sql-reference/statements/create/view.md) を使ってリアルタイム処理用のパイプラインを作成する方が実用的です。そのためには次の手順を実行します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md index 57be2cf1b88..2a9593e4531 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md @@ -9,13 +9,13 @@ doc_type: 'guide' -# Redis テーブルエンジン +# Redis テーブルエンジン {#redis-table-engine} このエンジンにより、ClickHouse を [Redis](https://redis.io/) と連携させることができます。Redis はキー・バリュー(KV)モデルを採用しているため、`where k=xx` や `where k in (xx, xx)` のようなポイントアクセスのクエリに限定して利用することを強く推奨します。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -47,7 +47,7 @@ PRIMARY KEY(primary_key_name); ::: -## 使用例 +## 使用例 {#usage-example} 単純な引数を用いて、`Redis` エンジンを使用する ClickHouse のテーブルを作成します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md index 358736967cf..a8a14dbbf45 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# S3 table engine +# S3 table engine {#s3-table-engine} このエンジンは、[Amazon S3](https://aws.amazon.com/s3/) エコシステムと連携します。このエンジンは [HDFS](/engines/table-engines/integrations/hdfs) エンジンと類似していますが、S3 固有の機能を備えています。 -## 例 +## 例 {#example} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -35,7 +35,7 @@ SELECT * FROM s3_engine_table LIMIT 2; ``` -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -44,7 +44,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) [SETTINGS ...] ``` -### エンジンパラメータ +### エンジンパラメータ {#parameters} * `path` — ファイルへのパスを含むバケット URL。読み取り専用モードでは `*`、`**`、`?`、`{abc,def}`、`{N..M}` というワイルドカードをサポートします。ここで `N`、`M` は数値、`'abc'`、`'def'` は文字列です。詳細は[下記](#wildcards-in-path)を参照してください。 * `NOSIGN` - 認証情報の代わりにこのキーワードが指定された場合、すべてのリクエストは署名されません。 @@ -55,7 +55,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) * `partition_columns_in_data_file` - `HIVE` パーティション戦略でのみ使用されます。データファイル内にパーティション列が書き込まれていることを ClickHouse が想定すべきかどうかを指定します。デフォルトは `false` です。 * `storage_class_name` - オプション: `STANDARD` または `INTELLIGENT_TIERING`。 [AWS S3 Intelligent Tiering](https://aws.amazon.com/s3/storage-classes/intelligent-tiering/) を指定できます。 -### データキャッシュ +### データキャッシュ {#data-cache} `S3` テーブルエンジンはローカルディスク上でのデータキャッシュをサポートします。 ファイルシステムキャッシュの設定オプションと使用方法については、この[セクション](/operations/storing-data.md/#using-local-cache)を参照してください。 @@ -86,13 +86,13 @@ SETTINGS filesystem_cache_name = 'cache_for_s3', enable_filesystem_cache = 1; 2. ClickHouse の `storage_configuration` セクションで設定されたキャッシュ構成(およびそれに伴うキャッシュストレージ)を再利用します。詳しくは[こちら](/operations/storing-data.md/#using-local-cache)を参照してください。 -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — 省略可能です。ほとんどの場合、パーティションキーは不要であり、必要な場合でも月単位より細かいパーティションキーが求められることはほぼありません。パーティション分割は(ORDER BY 式とは対照的に)クエリを高速化しません。過度に細かいパーティション分割は避けてください。データをクライアント識別子や名前でパーティション分割しないでください(代わりに、ORDER BY 式の最初の列としてクライアント識別子または名前を指定してください)。 月単位でパーティション分割するには、`date_column` が型 [Date](/sql-reference/data-types/date.md) の日付を持つ列であるとき、`toYYYYMM(date_column)` 式を使用します。ここでのパーティション名は `"YYYYMM"` 形式になります。 -#### パーティション戦略 +#### パーティション戦略 {#partition-strategy} `WILDCARD`(デフォルト): ファイルパス内の `{_partition_id}` ワイルドカードを実際のパーティションキーに置き換えます。読み取りはサポートされていません。 @@ -139,7 +139,7 @@ arthur :) select _path, * from t_03363_parquet; └────────────────────────────────────────────────────────────────────────────┴──────┴─────────┴─────────┘ ``` -### パーティション化されたデータのクエリ実行 +### パーティション化されたデータのクエリ実行 {#querying-partitioned-data} この例では、ClickHouse と MinIO を統合した [docker compose recipe](https://github.com/ClickHouse/examples/tree/5fdc6ff72f4e5137e23ea075c88d3f44b0202490/docker-compose-recipes/recipes/ch-and-minio-S3) を使用します。エンドポイントと認証情報の値を差し替えることで、S3 を使って同じクエリを再現できます。 @@ -159,7 +159,7 @@ ClickHouse のデータセットは非常に大きくなることが多く、ま 転送する、すなわちパーティション分割して書き込むのが理にかなっています。 ::: -#### テーブルを作成する +#### テーブルを作成する {#create-the-table} ```sql CREATE TABLE p @@ -177,13 +177,13 @@ ENGINE = S3( PARTITION BY column3 ``` -#### データを挿入する +#### データを挿入する {#insert-data} ```sql INSERT INTO p VALUES (1, 2, 3), (3, 2, 1), (78, 43, 45) ``` -#### パーティション 3 からの選択 +#### パーティション 3 からの選択 {#select-from-partition-3} :::tip このクエリは S3 テーブル関数を使用します。 @@ -200,7 +200,7 @@ FROM s3('http://minio:10000/clickhouse//test_3.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### パーティション1からの SELECT +#### パーティション1からの SELECT {#select-from-partition-1} ```sql SELECT * @@ -213,7 +213,7 @@ FROM s3('http://minio:10000/clickhouse//test_1.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### パーティション45からのSELECT +#### パーティション45からのSELECT {#select-from-partition-45} ```sql SELECT * @@ -226,7 +226,7 @@ FROM s3('http://minio:10000/clickhouse//test_45.csv', 'minioadmin', 'minioadminp └────┴────┴────┘ ``` -#### 制限事項 +#### 制限事項 {#limitation} つい `Select * from p` を実行したくなるかもしれませんが、上述のとおりこのクエリは失敗します。前述のクエリを使用してください。 @@ -273,7 +273,7 @@ Code: 48. DB::Exception: localhost:9000 から受信しました。DB::Exception -## パスのワイルドカード +## パスのワイルドカード {#wildcards-in-path} `path` 引数では、bash 形式のワイルドカードを使用して複数ファイルを指定できます。処理対象となるには、ファイルが存在し、パスパターン全体に一致している必要があります。ファイルの一覧は `SELECT` 実行時に決定されます(`CREATE` 時点ではありません)。 @@ -361,7 +361,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) -## エンドポイント単位の設定 +## エンドポイント単位の設定 {#endpoint-settings} 次の設定は、特定のエンドポイント用に設定ファイル内で指定できます(URL のプレフィックスの完全一致で判定されます)。 @@ -405,7 +405,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ``` -## アーカイブの操作 +## アーカイブの操作 {#working-with-archives} S3 上に、次の URI を持つ複数のアーカイブファイルがあるとします: @@ -431,7 +431,7 @@ ZIP および TAR アーカイブはサポートされている任意のスト ::: -## 公開バケットへのアクセス +## 公開バケットへのアクセス {#accessing-public-buckets} ClickHouse は、さまざまな種類のソースから認証情報を取得しようとします。 そのため、公開されている一部のバケットにアクセスする際に問題が発生し、クライアントが `403` エラーコードを返すことがあります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md index 7c2d73bb4c7..0f056972c0f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md @@ -1,5 +1,5 @@ --- -description: 'このエンジンは Amazon S3 エコシステムとの統合を提供し、ストリーミングによるインポートを可能にします。Kafka エンジンや RabbitMQ エンジンと類似していますが、S3 固有の機能も備えています。' +description: 'このエンジンは Amazon S3 エコシステムとの統合機能を提供し、ストリーミングインポートを可能にします。Kafka エンジンおよび RabbitMQ エンジンと同様ですが、S3 固有の機能を提供します。' sidebar_label: 'S3Queue' sidebar_position: 181 slug: /engines/table-engines/integrations/s3queue @@ -9,16 +9,13 @@ doc_type: 'reference' import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge' +# S3Queue テーブルエンジン {#s3queue-table-engine} -# S3Queue テーブルエンジン +このエンジンは [Amazon S3](https://aws.amazon.com/s3/) エコシステムとの統合を提供し、ストリーミングインポートを可能にします。このエンジンは [Kafka](../../../engines/table-engines/integrations/kafka.md)、[RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md) エンジンと類似していますが、S3 固有の機能を提供します。 -このエンジンは [Amazon S3](https://aws.amazon.com/s3/) エコシステムとの統合を提供し、ストリーミングによるインポートを可能にします。このエンジンは [Kafka](../../../engines/table-engines/integrations/kafka.md)、[RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md) エンジンに似ていますが、S3 固有の機能を備えています。 +[S3Queue 実装の元となった PR](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183) に記載されている次の注意点を理解しておくことが重要です。`MATERIALIZED VIEW` がこのエンジンに接続されると、S3Queue テーブルエンジンはバックグラウンドでデータの収集を開始します。 -[S3Queue 実装の元となった PR](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183) にある次の注意事項を理解しておくことが重要です。`MATERIALIZED VIEW` がこのエンジンに紐付けられると、S3Queue テーブルエンジンはバックグラウンドでデータの収集を開始します。 - - - -## テーブルの作成 {#creating-a-table} +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE s3_queue_engine_table (name String, value UInt32) @@ -49,12 +46,12 @@ CREATE TABLE s3_queue_engine_table (name String, value UInt32) ``` :::warning -`24.7`より前では、`mode`、`after_processing`、`keeper_path`以外のすべての設定に`s3queue_`プレフィックスを使用する必要があります。 +`24.7` より前のバージョンでは、`mode`、`after_processing`、`keeper_path` を除くすべての設定項目で `s3queue_` プレフィックスを使用する必要があります。 ::: **エンジンパラメータ** -`S3Queue`のパラメータは`S3`テーブルエンジンがサポートするものと同じです。パラメータセクションは[こちら](../../../engines/table-engines/integrations/s3.md#parameters)を参照してください。 +`S3Queue` のパラメータは、`S3` テーブルエンジンがサポートしているものと同一です。パラメータについては[こちら](../../../engines/table-engines/integrations/s3.md#parameters)を参照してください。 **例** @@ -65,7 +62,7 @@ SETTINGS mode = 'unordered'; ``` -名前付きコレクションの使用: +名前付きコレクションの使用: ```xml @@ -86,164 +83,256 @@ SETTINGS mode = 'ordered'; ``` - ## 設定 {#settings} -テーブルに設定された設定のリストを取得するには、`system.s3_queue_settings`テーブルを使用します。`24.10`から利用可能です。 +テーブルに対して構成された設定の一覧を取得するには、`system.s3_queue_settings` テーブルを使用します。`24.10` 以降で利用可能です。 -### モード {#mode} +### Mode {#mode} -指定可能な値: +指定可能な値: -- unordered — 順序なしモードでは、すでに処理されたすべてのファイルのセットがZooKeeper内の永続ノードで追跡されます。 -- ordered — 順序ありモードでは、ファイルは辞書順で処理されます。これは、'BBB'という名前のファイルがある時点で処理され、その後'AA'という名前のファイルがバケットに追加された場合、それは無視されることを意味します。正常に処理されたファイルの最大名(辞書順)と、読み込み失敗後に再試行されるファイルの名前のみがZooKeeperに保存されます。 +* unordered — `unordered` モードでは、すでに処理されたすべてのファイルの集合が ZooKeeper 内の永続ノードとして管理されます。 +* ordered — `ordered` モードでは、ファイルは辞書順で処理されます。つまり、ファイル名 `BBB` のファイルがある時点で処理され、その後にファイル名 `AA` のファイルがバケットに追加された場合、そのファイルは無視されます。ZooKeeper には、正常に取り込まれたファイル名のうち最大のもの(辞書順での最大値)と、読み込みに失敗して再試行対象となるファイル名のみが保存されます。 -デフォルト値: 24.6より前のバージョンでは`ordered`。24.6以降はデフォルト値がなく、手動で指定する必要があります。以前のバージョンで作成されたテーブルについては、互換性のためデフォルト値は`Ordered`のままとなります。 +デフォルト値: バージョン 24.6 より前は `ordered`。24.6 以降ではデフォルト値は存在せず、この設定は手動で指定する必要があります。以前のバージョンで作成されたテーブルについては、互換性のためデフォルト値は引き続き `Ordered` のままです。 -### `after_processing` {#after_processing} +### `after_processing` {#after_processing} + +ファイルの処理が正常に完了した後の扱い方法。 -処理が正常に完了した後、ファイルを削除するか保持するかを指定します。 指定可能な値: -- keep。 -- delete。 +* keep。 +* delete。 +* move。 +* tag。 デフォルト値: `keep`。 -### `keeper_path` {#keeper_path} +move を使用するには追加の設定が必要です。同一バケット内で移動する場合は、新しいパスプレフィックスを `after_processing_move_prefix` として指定する必要があります。 + +別の S3 バケットへ移動する場合は、移動先バケットの URI を `after_processing_move_uri` として、S3 クレデンシャルを `after_processing_move_access_key_id` および `after_processing_move_secret_access_key` として指定する必要があります。 + +例: + +```sql +CREATE TABLE s3queue_engine_table (name String, value UInt32) +ENGINE=S3Queue('https://clickhouse-public-datasets.s3.amazonaws.com/my-test-bucket-768/*', 'CSV', 'gzip') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_retries = 20, + after_processing_move_prefix = 'dst_prefix', + after_processing_move_uri = 'https://clickhouse-public-datasets.s3.amazonaws.com/dst-bucket', + after_processing_move_access_key_id = 'test', + after_processing_move_secret_access_key = 'test'; +``` + +Azure コンテナから別の Azure コンテナに移動するには、Blob Storage の接続文字列を `after_processing_move_connection_string` に、コンテナ名を `after_processing_move_container` に指定します。詳しくは [AzureQueue の設定](../../../engines/table-engines/integrations/azure-queue.md#settings) を参照してください。 + +タグ付けを行うには、タグキーと値をそれぞれ `after_processing_tag_key` および `after_processing_tag_value` として指定します。 + +### `after_processing_retries` {#after_processing_retries} + +要求された後処理アクションに対して、処理を中止するまでに行う再試行回数。 + +設定可能な値: + +* 0 以上の整数。 + +デフォルト値: `10`。 + +### `after_processing_move_access_key_id` {#after_processing_move_access_key_id} + +移動先が別の S3 バケットである場合に、正常に処理されたファイルをそのバケットへ移動するための Access Key ID。 + +設定可能な値: + +* 文字列。 + +デフォルト値: 空文字列。 + +### `after_processing_move_prefix` {#after_processing_move_prefix} + +正常に処理されたファイルを移動する先のパスプレフィックスです。同一バケット内での移動と、別のバケットへの移動の両方で有効です。 -ZooKeeper内のパスは、テーブルエンジンの設定として指定するか、グローバル設定で提供されるパスとテーブルUUIDから形成されるデフォルトパスを使用できます。 指定可能な値: -- 文字列。 +* String。 -デフォルト値: `/`。 +デフォルト値: 空文字列。 -### `s3queue_loading_retries` {#loading_retries} +### `after_processing_move_secret_access_key` {#after_processing_move_secret_access_key} + +移動先が別の S3 バケットである場合に、正常に処理されたファイルをそのバケットへ移動するための Secret Access Key。 + +設定可能な値: + +* 文字列。 + +デフォルト値: 空文字列。 + +### `after_processing_move_uri` {#after_processing_move_uri} + +宛先が別の S3 バケットである場合に、正常に処理されたファイルを移動する先となる S3 バケットの URI。 + +取りうる値: + +* 文字列。 + +デフォルト値: 空文字列。 + +### `after_processing_tag_key` {#after_processing_tag_key} + +`after_processing='tag'` の場合に、正常に処理されたファイルへタグ付けを行うためのタグキー。 + +指定可能な値: + +* 文字列。 + +デフォルト値: 空文字列。 + +### `after_processing_tag_value` {#after_processing_tag_value} + +`after_processing` が `tag` の場合に、正常に処理されたファイルに付与するタグ値。 -指定された回数までファイルの読み込みを再試行します。デフォルトでは再試行は行われません。 指定可能な値: -- 正の整数。 +* 文字列。 + +デフォルト値: 空文字列。 + +### `keeper_path` {#keeper_path} + +ZooKeeper 内のパスはテーブルエンジンの設定として指定するか、グローバル設定で指定されたパスとテーブル UUID から既定パスを生成できます。 +取りうる値: + +* 文字列。 + +既定値: `/`。 + +### `s3queue_loading_retries` {#loading_retries} + +指定された回数までファイルの読み込みを再試行します。デフォルトでは再試行は行われません。 +取りうる値: + +* 正の整数。 デフォルト値: `0`。 -### `s3queue_processing_threads_num` {#processing_threads_num} +### `s3queue_processing_threads_num` {#processing_threads_num} -処理を実行するスレッド数。`Unordered`モードにのみ適用されます。 +処理を実行するスレッド数。`Unordered` モードでのみ適用されます。 -デフォルト値: CPUの数または16。 +デフォルト値: CPU の数または 16。 -### `s3queue_parallel_inserts` {#parallel_inserts} +### `s3queue_parallel_inserts` {#parallel_inserts} -デフォルトでは、`processing_threads_num`は1つの`INSERT`を生成するため、複数のスレッドでファイルのダウンロードと解析のみを行います。 -しかし、これは並列性を制限するため、より高いスループットを得るには`parallel_inserts=true`を使用してください。これによりデータを並列に挿入できます(ただし、MergeTreeファミリーでは生成されるデータパーツの数が増加することに注意してください)。 +デフォルトでは、`processing_threads_num` は 1 つの `INSERT` しか生成されないため、複数スレッドで実行されるのはファイルのダウンロードとパース処理だけです。 +しかし、これは並列度を制限するため、スループットを向上させるには `parallel_inserts=true` を使用してください。これによりデータを並列に挿入できるようになります(ただし、その結果として MergeTree ファミリーのテーブルに対して生成されるデータパーツの数が増加する点に注意してください)。 :::note -`INSERT`は`max_process*_before_commit`設定に従って生成されます。 +`INSERT` は `max_process*_before_commit` 設定を考慮して生成されます。 ::: デフォルト値: `false`。 -### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} +### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} -`system.s3queue_log`へのログ記録を有効にします。 +`system.s3queue_log` へのログ記録を有効にします。 デフォルト値: `0`。 -### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} +### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} -次のポーリング試行を行う前にClickHouseが待機する最小時間をミリ秒単位で指定します。 +ClickHouse が次のポーリングを実行する前に待機する最小時間をミリ秒単位で指定します。 -指定可能な値: +設定可能な値: -- 正の整数。 +* 正の整数。 デフォルト値: `1000`。 -### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} +### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} -次のポーリング試行を開始する前にClickHouseが待機する最大時間をミリ秒単位で定義します。 +ClickHouse が次のポーリング試行を開始するまでに待機する最大時間を、ミリ秒単位で定義します。 -指定可能な値: +可能な値: -- 正の整数。 +* 正の整数。 -デフォルト値: `10000`。 +デフォルト値: `10000`. -### `s3queue_polling_backoff_ms` {#polling_backoff_ms} +### `s3queue_polling_backoff_ms` {#polling_backoff_ms} -新しいファイルが見つからない場合に、前回のポーリング間隔に追加される待機時間を決定します。次のポーリングは、前回の間隔とこのバックオフ値の合計、または最大間隔のいずれか小さい方の後に実行されます。 +新しいファイルが見つからなかった場合に、前回のポーリング間隔に追加される待機時間を決定します。次回のポーリングは、前回の間隔にこのバックオフ値を加えた値と最大間隔のうち、短い方の時間が経過した後に行われます。 指定可能な値: -- 正の整数。 +* 正の整数。 デフォルト値: `0`。 -### `s3queue_tracked_files_limit` {#tracked_files_limit} +### `s3queue_tracked_files_limit` {#tracked_files_limit} -'unordered'モードが使用されている場合にZooKeeperノードの数を制限できます。'ordered'モードでは何も行いません。 -制限に達すると、最も古い処理済みファイルがZooKeeperノードから削除され、再度処理されます。 +`unordered` モードが使用されている場合に、ZooKeeper ノードの数に上限を設けるための設定です。`ordered` モードでは何も行いません。 +上限に達した場合、最も古く処理されたファイルが ZooKeeper ノードから削除され、再度処理されます。 -指定可能な値: +取り得る値: -- 正の整数。 +* 正の整数。 デフォルト値: `1000`。 -### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} +### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} -'unordered'モードにおいて、処理済みファイルをZooKeeperノードに保存する最大秒数(デフォルトでは永続的に保存)。'ordered'モードでは何も行いません。 -指定された秒数が経過すると、ファイルは再インポートされます。 +`unordered` モードにおいて、処理済みファイルを ZooKeeper のノードに保持しておく最大秒数(デフォルトでは無期限に保存)を指定します。`ordered` モードでは何もしません。 +指定された秒数が経過すると、そのファイルは再インポートされます。 -指定可能な値: +設定可能な値: -- 正の整数。 +* 正の整数 デフォルト値: `0`。 -### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} +### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} -'Ordered'モード用。追跡ファイルのTTLと最大追跡ファイルセットの維持を担当するバックグラウンドタスクの再スケジュール間隔の最小境界を定義します。 +'Ordered' モード用。追跡対象ファイルの TTL および追跡対象ファイル集合の最大数を維持するバックグラウンドタスクについて、その再スケジュールの間隔の下限値を定義します。 デフォルト値: `10000`。 -### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} +### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} - -'Ordered'モード用。追跡ファイルのTTLと追跡ファイル数の上限を管理するバックグラウンドタスクの再スケジュール間隔の最大値を定義します。 +「Ordered」モード用。追跡対象ファイルの TTL と、追跡対象ファイル集合の最大数を維持するバックグラウンドタスクの再スケジュール間隔に対する上限値を定義します。 デフォルト値: `30000`。 ### `s3queue_buckets` {#buckets} -'Ordered'モード用。`24.6`以降で利用可能。S3Queueテーブルの複数のレプリカが存在し、それぞれがkeeperの同じメタデータディレクトリで動作している場合、`s3queue_buckets`の値は少なくともレプリカ数以上である必要があります。`s3queue_processing_threads`設定も併用する場合は、`s3queue_buckets`設定の値をさらに増やすことが推奨されます。これは`S3Queue`処理の実際の並列度を定義するためです。 - -### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} +「Ordered」モードで使用します。`24.6` から利用可能です。S3Queue テーブルのレプリカが複数あり、それぞれが keeper 内の同一のメタデータディレクトリを使用している場合、`s3queue_buckets` の値はレプリカ数以上に設定する必要があります。`s3queue_processing_threads` 設定も併用している場合は、`S3Queue` の処理における実際の並列度を決定するため、`s3queue_buckets` 設定の値をさらに大きくすることが推奨されます。 -デフォルトでは、S3Queueテーブルは常にエフェメラル処理ノードを使用しています。これにより、S3Queueが処理を開始した後、処理済みファイルをzookeeperにコミットする前にzookeeperセッションが期限切れになった場合、データの重複が発生する可能性があります。この設定により、keeperセッションが期限切れになった場合でも重複の可能性を排除します。 +### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} -### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} +デフォルトでは、S3Queue テーブルは常に一時的な処理ノードを使用しており、ZooKeeper セッションが、S3Queue が処理済みファイルを ZooKeeper にコミットする前に期限切れになり、かつ処理は開始されていた場合、データが重複する可能性がありました。この設定は、keeper セッションの期限切れに起因する重複が発生しないよう、サーバーに強制します。 -サーバーが正常終了しなかった場合、`use_persistent_processing_nodes`が有効になっていると、削除されていない処理ノードが残る可能性があります。この設定は、これらの処理ノードを安全にクリーンアップできる期間を定義します。 +### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} -デフォルト値: `3600`(1時間)。 +サーバーが正常終了しなかった場合、`use_persistent_processing_nodes` が有効になっていると、処理ノードが削除されずに残る可能性があります。この設定は、それらの処理ノードを安全にクリーンアップできる猶予時間を定義します。 +デフォルト値: `3600`(1時間)。 -## S3関連の設定 {#s3-settings} +## S3 に関連する設定 {#s3-settings} -このエンジンはすべてのS3関連設定をサポートしています。S3設定の詳細については、[こちら](../../../engines/table-engines/integrations/s3.md)を参照してください。 +このエンジンでは、すべての S3 関連設定が利用できます。S3 設定の詳細は[こちら](../../../engines/table-engines/integrations/s3.md)を参照してください。 +## S3 ロールベースアクセス {#s3-role-based-access} -## S3ロールベースアクセス {#s3-role-based-access} + - +`s3Queue` テーブルエンジンはロールベースのアクセスをサポートしています。 +バケットにアクセスするためのロールを設定する手順については、[こちら](/cloud/data-sources/secure-s3) のドキュメントを参照してください。 -s3Queueテーブルエンジンは、ロールベースアクセスをサポートしています。 -バケットへのアクセスに必要なロールの設定手順については、[こちら](/cloud/data-sources/secure-s3)のドキュメントを参照してください。 - -ロールの設定が完了したら、以下のように`extra_credentials`パラメータを使用して`roleARN`を指定できます: +ロールを設定したら、以下のように `extra_credentials` パラメータで `roleARN` を指定できます: ```sql CREATE TABLE s3_table @@ -259,29 +348,24 @@ SETTINGS ... ``` +## S3Queue の ordered モード {#ordered-mode} -## S3Queue順序付きモード {#ordered-mode} - -`S3Queue`処理モードでは、ZooKeeper内に保存するメタデータを削減できますが、時間的に後から追加されるファイルは、アルファベット順で大きい名前を持つ必要があるという制限があります。 +`S3Queue` の処理モードでは、ZooKeeper に保存するメタデータ量を減らせますが、時間的に後から追加されるファイルの名前は、英数字としてそれ以前のファイル名より大きくなる必要があるという制約があります。 -`S3Queue`の`ordered`モードは、`unordered`モードと同様に、`(s3queue_)processing_threads_num`設定(`s3queue_`プレフィックスは省略可能)をサポートしており、サーバー上でローカルに`S3`ファイルを処理するスレッド数を制御できます。 +`S3Queue` の `ordered` モードは、`unordered` と同様に `(s3queue_)processing_threads_num` 設定(`s3queue_` プレフィックスは任意)をサポートしており、サーバー上で `S3` ファイルのローカル処理を行うスレッド数を制御できます。 +さらに、`ordered` モードでは、論理スレッド(logical threads)を意味する `(s3queue_)buckets` という別の設定も導入されています。これは、`S3Queue` テーブルのレプリカを持つ複数のサーバーが存在するような分散シナリオにおいて、この設定が処理単位の数を定義することを意味します。例えば、各 `S3Queue` レプリカ上の各処理スレッドは、処理対象として特定の `bucket` をロックしようとし、各 `bucket` はファイル名のハッシュによって特定のファイルに割り当てられます。そのため、分散シナリオでは、`(s3queue_)buckets` 設定をレプリカ数以上、またはそれより大きく設定することが強く推奨されます。`bucket` の数がレプリカ数より多くても問題ありません。最適な構成は、`(s3queue_)buckets` 設定が `number_of_replicas` と `(s3queue_)processing_threads_num` の積と等しくなるようにすることです。 +`(s3queue_)processing_threads_num` 設定は、バージョン `24.6` より前での使用は推奨されません。 +`(s3queue_)buckets` 設定は、バージョン `24.6` から利用可能です。 -さらに、`ordered`モードでは`(s3queue_)buckets`という別の設定が導入されており、これは「論理スレッド」を意味します。分散環境において、複数のサーバーに`S3Queue`テーブルのレプリカが存在する場合、この設定が処理単位の数を定義します。例えば、各`S3Queue`レプリカ上の各処理スレッドは、処理のために特定の`bucket`をロックしようとし、各`bucket`はファイル名のハッシュによって特定のファイルに割り当てられます。したがって、分散環境では、`(s3queue_)buckets`設定をレプリカ数以上に設定することを強く推奨します。バケット数がレプリカ数より多くても問題ありません。最も最適なシナリオは、`(s3queue_)buckets`設定を`number_of_replicas`と`(s3queue_)processing_threads_num`の積に等しくすることです。 +## 説明 {#description} -`(s3queue_)processing_threads_num`設定は、バージョン`24.6`より前での使用は推奨されません。 +`SELECT` は、各ファイルを 1 回しかインポートできないため(デバッグ用途を除いて)ストリーミングインポートにはあまり有用ではありません。代わりに、[マテリアライズドビュー](../../../sql-reference/statements/create/view.md) を使ってリアルタイムの処理フローを作成する方が実用的です。そのためには次のようにします。 -`(s3queue_)buckets`設定は、バージョン`24.6`以降で利用可能です。 +1. 指定した S3 内のパスからデータを消費するテーブルを S3 エンジンで作成し、それをデータストリームと見なします。 +2. 目的の構造を持つテーブルを作成します。 +3. エンジンからのデータを変換し、事前に作成したテーブルに書き込むマテリアライズドビューを作成します。 - -## Description {#description} - -`SELECT`はストリーミングインポートにはあまり有用ではありません(デバッグを除く)。各ファイルは一度しかインポートできないためです。[マテリアライズドビュー](../../../sql-reference/statements/create/view.md)を使用してリアルタイム処理を作成する方が実用的です。これを行うには: - -1. エンジンを使用してS3の指定されたパスからデータを取得するテーブルを作成し、それをデータストリームとして扱います。 -2. 必要な構造を持つテーブルを作成します。 -3. エンジンからデータを変換し、事前に作成したテーブルに格納するマテリアライズドビューを作成します。 - -`MATERIALIZED VIEW`がエンジンに接続されると、バックグラウンドでデータの収集が開始されます。 +`MATERIALIZED VIEW` がエンジンに紐付けられると、バックグラウンドでデータの取り込みを開始します。 例: @@ -300,48 +384,44 @@ SETTINGS SELECT * FROM stats ORDER BY name; ``` - ## 仮想カラム {#virtual-columns} -- `_path` — ファイルへのパス。 -- `_file` — ファイル名。 -- `_size` — ファイルのサイズ。 -- `_time` — ファイルの作成時刻。 - -仮想カラムの詳細については、[こちら](../../../engines/table-engines/index.md#table_engines-virtual_columns)を参照してください。 +* `_path` — ファイルへのパス。 +* `_file` — ファイル名。 +* `_size` — ファイルサイズ。 +* `_time` — ファイルの作成時刻。 +仮想カラムの詳細については[こちら](../../../engines/table-engines/index.md#table_engines-virtual_columns)を参照してください。 ## パス内のワイルドカード {#wildcards-in-path} -`path` 引数では、bashライクなワイルドカードを使用して複数のファイルを指定できます。処理されるファイルは存在し、パスパターン全体に一致している必要があります。ファイルのリストは `SELECT` 実行時に決定されます(`CREATE` 時ではありません)。 - -- `*` — `/` を除く任意の文字を任意の数(空文字列を含む)にマッチします。 -- `**` — `/` を含む任意の文字を任意の数(空文字列を含む)にマッチします。 -- `?` — 任意の1文字にマッチします。 -- `{some_string,another_string,yet_another_one}` — `'some_string'`、`'another_string'`、`'yet_another_one'` のいずれかの文字列にマッチします。 -- `{N..M}` — N から M までの範囲内の任意の数値にマッチします(両端を含む)。N と M は先頭にゼロを含むことができます(例: `000..078`)。 +`path` 引数では、bash 風のワイルドカードを使用して複数のファイルを指定できます。ファイルが処理対象となるには、実際に存在し、パスパターン全体に完全一致している必要があります。ファイルの一覧は `SELECT` 実行時に決定されます(`CREATE` 時点ではありません)。 -`{}` を使用した構文は、[remote](../../../sql-reference/table-functions/remote.md) テーブル関数と同様です。 +* `*` — `/` を除く任意の文字列(空文字列を含む)に対して任意の長さでマッチします。 +* `**` — `/` を含む任意の文字列(空文字列を含む)に対して任意の長さでマッチします。 +* `?` — 任意の 1 文字にマッチします。 +* `{some_string,another_string,yet_another_one}` — 文字列 `'some_string', 'another_string', 'yet_another_one'` のいずれか 1 つにマッチします。 +* `{N..M}` — N から M までの範囲(両端を含む)の任意の数値にマッチします。N と M には先頭にゼロを含めることができ、例えば `000..078` のように指定できます。 +`{}` を用いた構文は [remote](../../../sql-reference/table-functions/remote.md) テーブル関数と類似しています。 ## 制限事項 {#limitations} -1. 重複行は以下の理由により発生する可能性があります: +1. 行の重複は以下の要因により発生する可能性があります: -- ファイル処理の途中で解析中に例外が発生し、`s3queue_loading_retries`による再試行が有効になっている場合; +* ファイル処理の途中でパース中に例外が発生し、`s3queue_loading_retries` によってリトライが有効になっている場合。 -- `S3Queue`が複数のサーバーでZooKeeperの同じパスを指すように設定されており、あるサーバーが処理済みファイルのコミットを完了する前にKeeperセッションが期限切れになった場合、別のサーバーがそのファイルの処理を引き継ぐ可能性があります。このファイルは最初のサーバーによって部分的または完全に処理されている可能性があります。ただし、`use_persistent_processing_nodes = 1`の場合、バージョン25.8以降ではこの問題は発生しません; +* 複数のサーバーで同じ zookeeper のパスを指すように `S3Queue` が構成されており、1つのサーバーが処理済みファイルをコミットする前に keeper セッションが期限切れになった場合。これにより、別のサーバーがそのファイルの処理を引き継ぐ可能性があり、そのファイルは最初のサーバーによって部分的または完全に処理されている場合があります。ただし、バージョン 25.8 以降で `use_persistent_processing_nodes = 1` が設定されている場合、これは発生しません。 -- サーバーの異常終了。 +* サーバーの異常終了。 -2. `S3Queue`が複数のサーバーでZooKeeperの同じパスを指すように設定されており、`Ordered`モードが使用されている場合、`s3queue_loading_retries`は機能しません。この問題は近日中に修正される予定です。 +2. 複数のサーバーで同じ zookeeper のパスを指すように `S3Queue` が構成されており、かつ `Ordered` モードが使用されている場合、`s3queue_loading_retries` は動作しません。これは近いうちに修正される予定です。 +## 内部状態の確認 {#introspection} -## イントロスペクション {#introspection} +内部状態の確認には、ステートレスな `system.s3queue` テーブルと永続的な `system.s3queue_log` テーブルを使用します。 -イントロスペクションには、`system.s3queue`ステートレステーブルと`system.s3queue_log`永続テーブルを使用します。 - -1. `system.s3queue`。このテーブルは永続化されず、`S3Queue`のインメモリ状態を表示します。現在処理中のファイル、処理済みまたは失敗したファイルを示します。 +1. `system.s3queue`。このテーブルは永続化されず、`S3Queue` のメモリ上の状態を表示します。現在処理中のファイル、処理済みのファイル、失敗したファイルを確認できます。 ```sql ┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ @@ -358,11 +438,11 @@ SETTINGS `exception` String ) ENGINE = SystemS3Queue -COMMENT 'S3Queueメタデータのインメモリ状態とファイルごとの現在処理中の行数を含みます。' │ +COMMENT 'S3Queueメタデータのインメモリ状態と、ファイルごとに現在処理されている行を含みます。' │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -例: +例: ```sql @@ -381,14 +461,14 @@ ProfileEvents: {'ZooKeeperTransactions':3,'ZooKeeperGet':2,'ZooKeeperMul exception: ``` -2. `system.s3queue_log`。永続テーブル。`system.s3queue`と同じ情報を持ちますが、`processed`および`failed`ファイルに関するものです。 +2. `system.s3queue_log`。永続テーブル。`system.s3queue` と同じ情報を持ちますが、対象は `processed` および `failed` のファイルです。 -このテーブルは以下の構造を持ちます: +このテーブルは次の構造を持ちます。 ```sql SHOW CREATE TABLE system.s3queue_log -Query id: 0ad619c3-0f2a-4ee4-8b40-c73d86e04314 +クエリ ID: 0ad619c3-0f2a-4ee4-8b40-c73d86e04314 ┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ CREATE TABLE system.s3queue_log @@ -411,7 +491,7 @@ SETTINGS index_granularity = 8192 │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -`system.s3queue_log`を使用するには、サーバー設定ファイルでその設定を定義します: +`system.s3queue_log` を使用するには、サーバーの設定ファイルでその構成を定義してください。 ```xml diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md index 45fbc69acf6..6ee70ea4efd 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# SQLite テーブルエンジン {#sqlite-table-engine} -# SQLite テーブルエンジン - - + このエンジンを使用すると、SQLite へのデータのインポートおよびエクスポートが可能になり、ClickHouse から SQLite テーブルへ直接クエリを実行することもできます。 - - -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -33,8 +30,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; * `db_path` — データベースを含む SQLite ファイルへのパス。 * `table` — SQLite データベース内のテーブル名。 - -## 使用例 +## 使用例 {#usage-example} SQLite テーブルを作成するクエリを次に示します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md index 21982415035..f56d5908e0f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# TimeSeries テーブルエンジン +# TimeSeries テーブルエンジン {#timeseries-table-engine} @@ -31,7 +31,7 @@ TimeSeries テーブルエンジンを使用するには、[allow_experiment ::: -## 構文 +## 構文 {#syntax} ```sql CREATE TABLE name [(columns)] ENGINE=TimeSeries @@ -42,7 +42,7 @@ CREATE TABLE name [(columns)] ENGINE=TimeSeries ``` -## 使用方法 +## 使用方法 {#usage} すべてをデフォルト設定のままにして開始するのが簡単です(列の一覧を指定しなくても `TimeSeries` テーブルを作成できます): @@ -116,7 +116,7 @@ _metrics_ テーブルは次のカラムを持たなければなりません: -## 作成 +## 作成 {#creation} `TimeSeries` テーブルエンジンでテーブルを作成する方法はいくつかあります。 最も簡単なステートメントは @@ -201,7 +201,7 @@ ORDER BY metric_family_name ``` -## 列型の調整 +## 列型の調整 {#adjusting-column-types} メインテーブルを定義する際に型を明示的に指定することで、内部ターゲットテーブルのほとんどすべての列型を調整できます。例えば、 @@ -226,7 +226,7 @@ ORDER BY (id, timestamp) ``` -## `id` 列 +## `id` 列 {#id-column} `id` 列には識別子が格納されており、各識別子はメトリクス名とタグの組み合わせに対して計算されます。 `id` 列の DEFAULT 式は、これらの識別子を計算するために使用される式です。 @@ -241,7 +241,7 @@ ENGINE=TimeSeries ``` -## `tags` 列と `all_tags` 列 +## `tags` 列と `all_tags` 列 {#tags-and-all-tags} タグのマップを含む列が 2 つあります。`tags` と `all_tags` です。この例では同じものですが、`tags_to_columns` 設定を使用した場合には異なる場合があります。この設定を使用すると、特定のタグを `tags` 列内のマップとしてではなく、別の列に保存するよう指定できます。 @@ -272,7 +272,7 @@ SETTINGS tags_to_columns = {'instance': 'instance', 'job': 'job'} ``` -## 内部ターゲットテーブルのテーブルエンジン +## 内部ターゲットテーブルのテーブルエンジン {#inner-table-engines} デフォルトでは、内部ターゲットテーブルは次のテーブルエンジンを使用します。 @@ -290,7 +290,7 @@ METRICS ENGINE=ReplicatedReplacingMergeTree ``` -## 外部ターゲットテーブル +## 外部ターゲットテーブル {#external-target-tables} `TimeSeries` テーブルが手動で作成したテーブルを使用するように設定できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md index ca15e112e3e..280b0b7a660 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md @@ -12,7 +12,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# YTsaurus テーブルエンジン +# YTsaurus テーブルエンジン {#ytsaurus-table-engine} @@ -21,7 +21,7 @@ YTsaurus テーブルエンジンを使用すると、YTsaurus クラスター -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -47,7 +47,7 @@ YTsaurus テーブルエンジンを使用すると、YTsaurus クラスター * `oauth_token` — OAuth トークン。 -## 使用例 +## 使用例 {#usage-example} YTsaurus テーブルを作成するクエリの例です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md index 0dad62231a5..1ac04994bd5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md @@ -10,7 +10,7 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Log テーブルエンジンファミリー +# Log テーブルエンジンファミリー {#log-table-engine-family} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md index 1bdb46cf405..9ba953b4e27 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Log テーブルエンジン +# Log テーブルエンジン {#log-table-engine} @@ -22,7 +22,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## テーブルを作成する +## テーブルを作成する {#table_engines-log-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,7 +59,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用例 +## 使用例 {#table_engines-log-example-of-use} テーブルの作成: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md index e553e83bcdd..83b423504ef 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# StripeLog テーブルエンジン +# StripeLog テーブルエンジン {#stripelog-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## テーブルの作成 +## テーブルの作成 {#table_engines-stripelog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -53,7 +53,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用例 +## 使用例 {#table_engines-stripelog-example-of-use} テーブルを作成: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md index df248d194d8..84dd0c2c058 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# TinyLog テーブルエンジン +# TinyLog テーブルエンジン {#tinylog-table-engine} @@ -32,7 +32,7 @@ Log エンジンとは異なり、TinyLog は mark ファイルを使用しま -## テーブルの作成 +## テーブルの作成 {#table_engines-tinylog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -58,7 +58,7 @@ ClickHouse は各テーブルに対して次のファイルを作成します。 -## 使用例 +## 使用例 {#table_engines-tinylog-example-of-use} テーブルの作成: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md index 076b06acabc..898e2700ce5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# AggregatingMergeTree テーブルエンジン +# AggregatingMergeTree テーブルエンジン {#aggregatingmergetree-table-engine} このエンジンは [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) から継承しており、データパーツのマージロジックを変更します。ClickHouse は、同じ主キー(より正確には、同じ[ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md))を持つすべての行を 1 行(単一のデータパーツ内)にまとめ、その行に集約関数の状態を組み合わせて格納します。 @@ -29,7 +29,7 @@ doc_type: 'reference' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -80,7 +80,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 集約マテリアライズドビューの例 +## 集約マテリアライズドビューの例 {#example-of-an-aggregated-materialized-view} 次の例では、`test` という名前のデータベースが既に存在すると仮定します。まだない場合は、以下のコマンドで作成してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md index fe4f6ded1ef..bd01ea888cd 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md @@ -10,7 +10,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# 厳密ベクトル検索と近似ベクトル検索 +# 厳密ベクトル検索と近似ベクトル検索 {#exact-and-approximate-vector-search} 多次元(ベクトル)空間において、ある点に最も近い N 個の点を見つける問題は、[nearest neighbor search](https://en.wikipedia.org/wiki/Nearest_neighbor_search)(最近傍探索)、または略してベクトル検索と呼ばれます。 ベクトル検索を行うための一般的なアプローチは 2 つあります: @@ -36,13 +36,13 @@ LIMIT `<N>` は、返すべき近傍点の数を指定します。 -## 厳密なベクトル検索 +## 厳密なベクトル検索 {#exact-nearest-neighbor-search} 厳密なベクトル検索は、上記の SELECT クエリをそのまま使用して実行できます。 このようなクエリの実行時間は、一般的に保存されているベクトル数とその次元数、つまり配列要素数に比例します。 また、ClickHouse はすべてのベクトルに対して総当たりスキャンを行うため、クエリで使用されるスレッド数(設定項目 [max_threads](../../../operations/settings/settings.md#max_threads) を参照)にも実行時間が依存します。 -### 例 +### 例 {#exact-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; @@ -67,9 +67,9 @@ LIMIT 3; ``` -## 近似ベクトル検索 +## 近似ベクトル検索 {#approximate-nearest-neighbor-search} -### ベクトル類似度インデックス +### ベクトル類似度インデックス {#vector-similarity-index} ClickHouse は、近似ベクトル検索を実行するための特別な「ベクトル類似度」インデックスを提供します。 @@ -78,7 +78,7 @@ ClickHouse は、近似ベクトル検索を実行するための特別な「ベ 問題が発生した場合は、[ClickHouse リポジトリ](https://github.com/clickhouse/clickhouse/issues) に issue を作成してください。 ::: -#### ベクトル類似度インデックスの作成 +#### ベクトル類似度インデックスの作成 {#creating-a-vector-similarity-index} 新しいテーブルに対して、次のようにベクトル類似度インデックスを作成できます。 @@ -196,7 +196,7 @@ ORDER BY [...] 上記の式には、事前割り当てバッファやキャッシュなど、ベクトル類似性インデックスがランタイムのデータ構造を割り当てるために必要となる追加メモリは含まれていません。 -#### ベクトル類似性インデックスの使用 +#### ベクトル類似性インデックスの使用 {#using-a-vector-similarity-index} :::note ベクトル類似性インデックスを使用するには、[compatibility](../../../operations/settings/settings.md) 設定を `''`(デフォルト値)、または `'25.1'` 以降に設定する必要があります。 @@ -407,7 +407,7 @@ Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 `vector_search_with_rescoring = 0` で再スコアリングなしのクエリを実行し、かつ並列レプリカを有効にしている場合、再スコアリングにフォールバックして実行されることがあります。 ::: -#### パフォーマンスチューニング +#### パフォーマンスチューニング {#performance-tuning} **圧縮のチューニング** @@ -541,7 +541,7 @@ result = chclient.query( この例では、参照ベクターはそのままバイナリ形式で送信され、サーバー側で浮動小数点数配列として再解釈されます。 これにより、サーバー側の CPU 時間を節約し、サーバーログおよび `system.query_log` の肥大化を防ぐことができます。 -#### 管理と監視 +#### 管理と監視 {#administration} ベクトル類似性インデックスのディスク上のサイズは、[system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) から取得できます。 @@ -559,7 +559,7 @@ WHERE type = 'vector_similarity'; └──────────┴───────┴──────┴──────────────────────────┘ ``` -#### 通常のスキッピングインデックスとの違い +#### 通常のスキッピングインデックスとの違い {#differences-to-regular-skipping-indexes} 通常の[スキッピングインデックス](/optimize/skipping-indexes)と同様に、ベクトル類似度インデックスはグラニュール単位で構築され、各インデックスブロックは `GRANULARITY = [N]` 個のグラニュールで構成されます(通常のスキッピングインデックスではデフォルトで `[N]` = 1)。 たとえば、テーブルのプライマリインデックスのグラニュラリティが 8192(`index_granularity = 8192` の設定)で `GRANULARITY = 2` の場合、各インデックスブロックには 16384 行が含まれます。 @@ -585,7 +585,7 @@ WHERE type = 'vector_similarity'; 一般的には、ベクトル類似度インデックスに対しては大きな `GRANULARITY` を使用し、ベクトル類似度構造によるメモリ消費が過大になるなどの問題が発生した場合にのみ、より小さな `GRANULARITY` の値に切り替えることを推奨します。 ベクトル類似度インデックスに対して `GRANULARITY` が指定されていない場合、デフォルト値は 1 億です。 -#### 例 +#### 例 {#approximate-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; @@ -616,7 +616,7 @@ LIMIT 3; * [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) * [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) -### Quantized Bit (QBit) +### Quantized Bit (QBit) {#approximate-nearest-neighbor-search-qbit} @@ -650,7 +650,7 @@ column_name QBit(element_type, dimension) * `element_type` – 各ベクトル要素の型。サポートされている型は `BFloat16`、`Float32`、`Float64` です * `dimension` – 各ベクトル内の要素数 -#### `QBit` テーブルの作成とデータの追加 +#### `QBit` テーブルの作成とデータの追加 {#qbit-create} ```sql CREATE TABLE fruit_animal ( @@ -668,7 +668,7 @@ INSERT INTO fruit_animal VALUES ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); ``` -#### `QBit` を使ったベクトル検索 +#### `QBit` を使ったベクトル検索 {#qbit-search} L2 距離を用いて、単語 'lemon' を表すベクトルに最も近い近傍を検索します。距離関数の第 3 引数ではビット単位の精度を指定します。値を大きくすると精度は向上しますが、計算コストも増加します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md index 857724da71b..4c81481dd5b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md @@ -9,9 +9,7 @@ show_related_blogs: true doc_type: 'reference' --- - - -# CoalescingMergeTree テーブルエンジン +# CoalescingMergeTree テーブルエンジン {#coalescingmergetree-table-engine} :::note Available from version 25.6 このテーブルエンジンは、OSS と Cloud の両方でバージョン 25.6 以降で利用可能です。 @@ -23,9 +21,7 @@ doc_type: 'reference' `CoalescingMergeTree` は、キー以外のカラムで Nullable 型と併用することを想定しています。カラムが Nullable でない場合は、その動作は [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree) と同じになります。 - - -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -42,16 +38,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] リクエストパラメータの説明については、[リクエストの説明](../../../sql-reference/statements/create/table.md)を参照してください。 -### CoalescingMergeTree のパラメータ +### CoalescingMergeTree のパラメータ {#parameters-of-coalescingmergetree} -#### Columns +#### Columns {#columns} `columns` - 値が統合されるカラム名のタプルです。省略可能なパラメータです。\ カラムは数値型である必要があり、パーティションキーまたはソートキーに含まれていてはなりません。 `columns` が指定されていない場合、ClickHouse はソートキーに含まれていないすべてのカラムの値を統合します。 -### クエリ句 +### クエリ句 {#query-clauses} `CoalescingMergeTree` テーブルを作成する際には、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) が必要です。 @@ -76,8 +72,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `columns` — 値が合計されるカラム名のタプルです。省略可能なパラメータです。詳細な説明については上記のテキストを参照してください。
- -## 使用例 +## 使用例 {#usage-example} 次のテーブルを例にします。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md index 306f76d95e9..709781c6480 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# CollapsingMergeTree テーブルエンジン +# CollapsingMergeTree テーブルエンジン {#collapsingmergetree-table-engine} @@ -40,7 +40,7 @@ doc_type: 'guide' -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -81,9 +81,9 @@ ENGINE = CollapsingMergeTree(Sign) * `CollapsingMergeTree` テーブルを作成する場合は、`MergeTree` テーブルを作成する場合と同じ [クエリ句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table) が必要です。 -## Collapsing +## Collapsing {#table_engine-collapsingmergetree-collapsing} -### Data +### Data {#data} あるオブジェクトに対して、継続的に変化するデータを保存する必要がある状況を考えます。 各オブジェクトにつき 1 行だけを持ち、何か変更があるたびにその行を更新する、というのは論理的に思えますが、 @@ -140,7 +140,7 @@ ENGINE = CollapsingMergeTree(Sign) 2. カラム内で長く伸び続ける配列は、書き込み負荷の増大によりエンジンの効率を低下させます。データが単純であればあるほど効率は高くなります。 3. `SELECT` の結果は、オブジェクト変更履歴の一貫性に大きく依存します。挿入用データを準備する際には注意してください。一貫性のないデータでは予測不能な結果が生じる可能性があります。たとえば、セッション深度のような非負のメトリクスに対して負の値が出力されることがあります。 -### Algorithm +### Algorithm {#table_engine-collapsingmergetree-collapsing-algorithm} ClickHouse がデータ[パーツ](/concepts/glossary#parts)をマージする際、 同じソートキー(`ORDER BY`)を持つ連続した行の各グループは、高々 2 行にまでまとめられます。 @@ -186,9 +186,9 @@ collapsing を最終確定させるには、`GROUP BY` 句と、`Sign` を考慮 -## 例 +## 例 {#examples} -### 使用例 +### 使用例 {#example-of-use} 次のサンプルデータを前提とします。 @@ -289,7 +289,7 @@ SELECT * FROM UAct FINAL このようなデータの選択方法は効率が悪く、スキャン対象データが多い場合(数百万行規模)には使用しないことを推奨します。 ::: -### 別のアプローチの例 +### 別のアプローチの例 {#example-of-another-approach} このアプローチの考え方は、マージ処理がキー列のみを考慮するという点にあります。 そのため「cancel」行では、`Sign` 列を使用せずに集計したときにその行の以前のバージョンと相殺されるような diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md index cc06feb766e..3c950ad0bac 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md @@ -7,9 +7,7 @@ title: 'カスタムパーティショニングキー' doc_type: 'guide' --- - - -# カスタムパーティションキー +# カスタムパーティションキー {#custom-partitioning-key} :::note ほとんどの場合、パーティションキーは不要であり、それ以外の多くの場合でも、月単位より細かいパーティションキーは不要です。例外はオブザーバビリティ向けのユースケースで、この場合は日単位のパーティションが一般的です。 @@ -78,7 +76,6 @@ WHERE table = 'visits' `partition` 列にはパーティション名が含まれています。この例では、`201901` と `201902` の 2 つのパーティションがあります。この列の値を使用して、[ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md) クエリでパーティション名を指定できます。 - `name` 列には、パーティションのデータパーツ名が含まれます。この列を使用して、[ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart) クエリでパーツ名を指定できます。 パーツ名 `201901_1_9_2_11` を分解して説明します。 @@ -134,16 +131,13 @@ drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached フォルダ '201901_1_1_0'、'201901_1_7_1' などは、各パートのディレクトリです。各パートは対応するパーティションに紐づいており、特定の月のデータだけを含みます(この例のテーブルは月単位でパーティション分割されています)。 - `detached` ディレクトリには、[DETACH](/sql-reference/statements/detach) クエリを使用してテーブルから切り離されたパーツが含まれます。破損したパーツも削除されるのではなく、このディレクトリへ移動されます。サーバーは `detached` ディレクトリ内のパーツを使用しません。このディレクトリ内のデータは、いつでも追加、削除、または変更できます。サーバーは、[ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart) クエリを実行するまでこれらを認識しません。 稼働中のサーバーでは、サーバーが認識できないため、ファイルシステム上のパーツの集合やそのデータを手動で変更することはできません。非レプリケートテーブルの場合、サーバー停止中であればこれを行うことはできますが、推奨されません。レプリケートテーブルの場合は、いかなる場合でもパーツの集合を変更することはできません。 ClickHouse ではパーティションに対して操作を実行できます。削除したり、あるテーブルから別のテーブルへコピーしたり、バックアップを作成したりできます。すべての操作の一覧は、[パーティションおよびパーツの操作](/sql-reference/statements/alter/partition) セクションを参照してください。 - - -## パーティションキーを利用した Group By 最適化 +## パーティションキーを利用した Group By 最適化 {#group-by-optimisation-using-partition-key} テーブルのパーティションキーとクエリの Group By キーの組み合わせによっては、 各パーティションごとに独立して集約処理を実行できる場合があります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md index c2dbfba48cf..f432ea61618 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md @@ -7,9 +7,7 @@ title: 'GraphiteMergeTree テーブルエンジン' doc_type: 'guide' --- - - -# GraphiteMergeTree テーブルエンジン +# GraphiteMergeTree テーブルエンジン {#graphitemergetree-table-engine} このエンジンは、[Graphite](http://graphite.readthedocs.io/en/latest/index.html) データの間引きおよび集約・平均化(ロールアップ)のために設計されています。Graphite のデータストアとして ClickHouse を使用したい開発者に役立ちます。 @@ -17,9 +15,7 @@ doc_type: 'guide' このエンジンは [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) の特性を継承します。 - - -## テーブルの作成 +## テーブルの作成 {#creating-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -82,8 +78,7 @@ Graphite データ用のテーブルには、以下のデータを格納する * `config_section` — 設定ファイル内のセクション名。このセクションに rollup のルールを設定します。
- -## ロールアップ設定 +## ロールアップ設定 {#rollup-configuration} ロールアップの設定は、サーバー設定内の [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) パラメータによって定義されます。パラメータ名は任意の名前を付けることができます。複数の設定を作成し、異なるテーブルに対して使い分けることができます。 @@ -92,25 +87,25 @@ Graphite データ用のテーブルには、以下のデータを格納する required-columns patterns -### 必須カラム +### 必須カラム {#required-columns} -#### `path_column_name` +#### `path_column_name` {#path_column_name} `path_column_name` — メトリック名(Graphite センサー)を保存するカラム名。デフォルト値: `Path`。 -#### `time_column_name` +#### `time_column_name` {#time_column_name} `time_column_name` — メトリックを計測した時刻を保存するカラム名。デフォルト値: `Time`。 -#### `value_column_name` +#### `value_column_name` {#value_column_name} `value_column_name` — `time_column_name` で指定された時刻におけるメトリック値を保存するカラム名。デフォルト値: `Value`。 -#### `version_column_name` +#### `version_column_name` {#version_column_name} `version_column_name` — メトリックのバージョンを保存するカラム名。デフォルト値: `Timestamp`。 -### パターン +### パターン {#patterns} `patterns` セクションの構造: @@ -163,8 +158,7 @@ default * `precision`– データの経過時間を秒単位でどの程度の精度で定義するか。86400(1 日の秒数)を割り切れる値である必要があります。 * `function` – 経過時間が `[age, age + precision]` の範囲に入るデータに適用する集約関数の名前。使用可能な関数: min / max / any / avg。平均は、平均値同士の平均を取るのと同様に、厳密ではない平均として計算されます。 -### ルールタイプなしの設定例 - +### ルールタイプなしの設定例 {#configuration-example} ```xml @@ -199,7 +193,7 @@ default ``` -### ルールタイプ別の設定例 +### ルールタイプ別の設定例 {#configuration-typed-example} ```xml diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md index 75251b18fea..4e89b13e7f2 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md @@ -7,9 +7,7 @@ title: 'MergeTree エンジンファミリー' doc_type: 'reference' --- - - -# MergeTree エンジンファミリー +# MergeTree エンジンファミリー {#mergetree-engine-family} MergeTree ファミリーに属するテーブルエンジンは、ClickHouse のデータストレージ機能の中核となります。これらは、カラム型ストレージ、カスタムパーティショニング、疎なプライマリインデックス、セカンダリのデータスキップインデックスなど、耐障害性と高性能なデータ取得のためのほとんどの機能を提供します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md index 24a3927a6ca..647bba4c279 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md @@ -10,7 +10,7 @@ doc_type: 'reference' import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; -# テキストインデックスを使用した全文検索 +# テキストインデックスを使用した全文検索 {#full-text-search-using-text-indexes} @@ -22,7 +22,7 @@ ClickHouse のテキストインデックス(["inverted indexes"](https://en.w -## テキストインデックスの作成 +## テキストインデックスの作成 {#creating-a-text-index} テキストインデックスを作成するには、まず対応する実験的な設定を有効にしてください。 @@ -186,12 +186,12 @@ ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); ``` -## テキストインデックスの使用 +## テキストインデックスの使用 {#using-a-text-index} SELECT クエリでテキストインデックスを使用するのは簡単で、一般的な文字列検索関数では自動的にインデックスが利用されます。 インデックスが存在しない場合、以下の文字列検索関数は低速な総当たりスキャンによる処理にフォールバックします。 -### サポートされている関数 +### サポートされている関数 {#functions-support} SELECT クエリの `WHERE` 句でテキスト関数が使用されている場合、テキストインデックスを利用できます。 @@ -201,7 +201,7 @@ FROM [...] WHERE string_search_function(column_with_text_index) ``` -#### `=` と `!=` +#### `=` と `!=` {#functions-example-equals-notequals} `=` ([equals](/sql-reference/functions/comparison-functions.md/#equals)) と `!=` ([notEquals](/sql-reference/functions/comparison-functions.md/#notEquals)) は、指定された検索語全体と一致します。 @@ -213,7 +213,7 @@ SELECT * from tab WHERE str = 'こんにちは'; テキストインデックスは `=` と `!=` をサポートしますが、等値・不等値検索が意味を持つのは `array` トークナイザを使用する場合のみです(このトークナイザではインデックスに行全体の値が保存されます)。 -#### `IN` と `NOT IN` +#### `IN` と `NOT IN` {#functions-example-in-notin} `IN`([`in`](/sql-reference/functions/in-functions))と `NOT IN`([`notIn`](/sql-reference/functions/in-functions))は `equals` および `notEquals` 関数と似ていますが、検索語句のすべてに一致するもの(`IN`)、あるいはどれにも一致しないもの(`NOT IN`)を判定します。 @@ -225,7 +225,7 @@ SELECT * from tab WHERE str IN ('Hello', 'World'); `=` および `!=` に対する制限と同じ制限が適用されます。つまり、`IN` と `NOT IN` は `array` トークナイザーと組み合わせて使用する場合にのみ意味があります。 -#### `LIKE`、`NOT LIKE` および `match` +#### `LIKE`、`NOT LIKE` および `match` {#functions-example-like-notlike-match} :::note これらの関数がフィルタリングのためにテキストインデックスを使用するのは、インデックストークナイザーが `splitByNonAlpha` または `ngrams` のいずれかである場合に限られます。 @@ -250,7 +250,7 @@ SELECT count() FROM tab WHERE comment LIKE ' support %'; -- または `% support `support` の左右に空白を入れておくと、その語をトークンとして抽出できるようになります。 -#### `startsWith` と `endsWith` +#### `startsWith` と `endsWith` {#functions-example-startswith-endswith} `LIKE` と同様に、関数 [startsWith](/sql-reference/functions/string-functions.md/#startsWith) と [endsWith](/sql-reference/functions/string-functions.md/#endsWith) は、検索語から完全なトークンを抽出できる場合にのみテキストインデックスを使用できます。 @@ -275,7 +275,7 @@ startsWith(comment, 'clickhouse supports ')` SELECT count() FROM tab WHERE endsWith(comment, ' olap engine'); ``` -#### `hasToken` と `hasTokenOrNull` +#### `hasToken` と `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} 関数 [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) および [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) は、指定された単一のトークンを対象にマッチングを行います。 @@ -289,7 +289,7 @@ SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); 関数 `hasToken` と `hasTokenOrNull` は、`text` インデックスと組み合わせて使用する関数として最も高いパフォーマンスを発揮します。 -#### `hasAnyTokens` と `hasAllTokens` +#### `hasAnyTokens` と `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} 関数 [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) と [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) は、指定されたトークンの一部またはすべてにマッチします。 @@ -309,7 +309,7 @@ SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); ``` -#### `has` +#### `has` {#functions-example-has} 配列関数 [has](/sql-reference/functions/array-functions#has) は、文字列配列内の単一のトークンとの一致を判定します。 @@ -319,7 +319,7 @@ SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE has(array, 'clickhouse'); ``` -#### `mapContains` +#### `mapContains` {#functions-example-mapcontains} 関数 [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains)(`mapContainsKey` のエイリアス)は、マップのキーに含まれる単一のトークンにマッチさせます。 @@ -331,7 +331,7 @@ SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); ``` -#### `operator[]` +#### `operator[]` {#functions-example-access-operator} アクセス演算子 [operator[]](/sql-reference/operators#access-operators)は、テキストインデックスと併用してキーおよび値をフィルタリングするために使用できます。 @@ -343,9 +343,9 @@ SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; `Array(T)` 型および `Map(K, V)` 型のカラムをテキストインデックスと併用する場合の例を以下に示します。 -### テキストインデックスを使用した `Array` および `Map` カラムの例 +### テキストインデックスを使用した `Array` および `Map` カラムの例 {#text-index-array-and-map-examples} -#### Array(String) カラムへのインデックス作成 +#### Array(String) カラムへのインデックス作成 {#text-index-example-array} ブログプラットフォームを想像してください。著者はキーワードを使って自身のブログ記事にカテゴリー付けを行います。 ユーザーには、トピックを検索したりクリックしたりすることで関連するコンテンツを見つけてほしいと考えています。 @@ -377,7 +377,7 @@ ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitBy ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- 既存データに対してインデックスを再構築してください ``` -#### Map 列のインデックス作成 +#### Map 列のインデックス作成 {#text-index-example-map} 多くのオブザーバビリティのユースケースでは、ログメッセージは「コンポーネント」に分割され、タイムスタンプには日時型、ログレベルには enum 型など、適切なデータ型で保存されます。 メトリクスのフィールドはキーと値のペアとして保存するのが最適です。 @@ -435,9 +435,9 @@ SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- fast ``` -## パフォーマンスチューニング +## パフォーマンスチューニング {#performance-tuning} -### 直接読み取り +### 直接読み取り {#direct-read} 特定の種類のテキストクエリは、「direct read」と呼ばれる最適化によって大幅に高速化されます。 より正確には、`SELECT` クエリがテキスト列を *選択しない* 場合に、この最適化を適用できます。 @@ -515,7 +515,7 @@ Positions: 2番目の EXPLAIN PLAN の出力には、仮想カラム `__text_index___` が含まれます。 このカラムが存在する場合は、直接読み出しが使用されています。 -### キャッシュ +### キャッシュ {#caching} テキストインデックスの一部をメモリ上でバッファリングするために、複数のキャッシュが利用可能です([Implementation Details](#implementation) セクションを参照)。 現在、I/O を削減するために、テキストインデックスのデシリアライズ済みディクショナリブロック、ヘッダー、およびポスティングリスト用のキャッシュが用意されています。 @@ -524,7 +524,7 @@ Positions: キャッシュを設定するには、以下のサーバー設定を参照してください。 -#### ディクショナリブロックキャッシュの設定 +#### ディクショナリブロックキャッシュの設定 {#caching-dictionary} | Setting | Description | @@ -583,7 +583,7 @@ Index granules file (.idx) には、各辞書ブロックについて、その -## 例:Hackernews データセット +## 例:Hackernews データセット {#hacker-news-dataset} 大量のテキストを含む大規模なデータセットに対するテキストインデックスのパフォーマンス向上を確認していきます。 ここでは、人気サイトである Hacker News 上のコメント 2,870 万件を使用します。 @@ -650,7 +650,7 @@ ALTER TABLE hackernews MATERIALIZE INDEX comment_idx SETTINGS mutations_sync = 2 では、`hasToken`、`hasAnyTokens`、`hasAllTokens` 関数を使ってクエリを実行してみます。 次の例では、標準的なインデックススキャンとダイレクトリード最適化の間で、どれほど大きな性能差が生じるかを示します。 -### 1. `hasToken` を使用する +### 1. `hasToken` を使用する {#using-hasToken} `hasToken` は、テキストに特定の単一トークンが含まれているかどうかをチェックします。 ここでは大文字小文字を区別して、`ClickHouse` というトークンを検索します。 @@ -690,7 +690,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re 直接読み取りクエリは 45 倍以上高速 (0.362s 対 0.008s) で、インデックスのみを読み取ることで処理するデータ量も大幅に削減されます (9.51 GB 対 3.15 MB)。 -### 2. `hasAnyTokens` を使用する +### 2. `hasAnyTokens` を使用する {#using-hasAnyTokens} `hasAnyTokens` は、テキストに指定したトークンのうち少なくとも 1 つが含まれているかどうかをチェックします。 ここでは、'love' または 'ClickHouse' のいずれかを含むコメントを検索します。 @@ -730,7 +730,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re この一般的な「OR」検索では、高速化がさらに顕著です。 フルカラムスキャンを回避することで、クエリは約89倍高速化されます(1.329秒 vs 0.015秒)。 -### 3. `hasAllTokens`の使用 +### 3. `hasAllTokens`の使用 {#using-hasAllTokens} `hasAllTokens`は、テキストに指定されたすべてのトークンが含まれているかを確認します。 'love'と'ClickHouse'の両方を含むコメントを検索します。 @@ -770,7 +770,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re この AND 検索では、ダイレクトリード最適化は標準のスキップインデックススキャンと比べて 26 倍以上高速です (0.184s 対 0.007s)。 -### 4. 複合検索: OR, AND, NOT, ... +### 4. 複合検索: OR, AND, NOT, ... {#compound-search} ダイレクトリード最適化は、複合ブール式にも適用されます。 ここでは、大文字小文字を区別しない検索で「ClickHouse」または「clickhouse」を検索します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md index a1f7a01e209..b472dc8f3f5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md @@ -1,8 +1,8 @@ --- -description: '`MergeTree` ファミリーのテーブルエンジンは、高いデータ取り込み速度と大量のデータを扱うために設計されています。' +description: '`MergeTree` ファミリーのテーブルエンジンは、高速なデータ取り込みと膨大なデータ量に対応するよう設計されています。' sidebar_label: 'MergeTree' sidebar_position: 11 -slug: /engines/table-engines/mergetree-family/mergetree +slug: /engines/table-engines/mergetree-family/mergettree title: 'MergeTree テーブルエンジン' doc_type: 'reference' --- @@ -10,31 +10,28 @@ doc_type: 'reference' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# MergeTree テーブルエンジン {#mergetree-table-engine} -# MergeTree テーブルエンジン +`MergeTree` エンジンおよび `MergeTree` ファミリーの他のエンジン(例: `ReplacingMergeTree`、`AggregatingMergeTree`)は、ClickHouse で最も一般的に使用され、最も堅牢なテーブルエンジンです。 -`MergeTree` エンジンおよび `ReplacingMergeTree`、`AggregatingMergeTree` などの `MergeTree` ファミリーの他のエンジンは、ClickHouse で最も一般的に使用され、最も堅牢なテーブルエンジンです。 +`MergeTree` ファミリーのテーブルエンジンは、高いデータ取り込みレートと巨大なデータ量を扱えるように設計されています。 +`INSERT` 操作によりテーブルパーツが作成され、バックグラウンドプロセスによって他のテーブルパーツとマージされます。 -`MergeTree` ファミリーのテーブルエンジンは、高いデータ取り込みレートと膨大なデータ量を扱えるように設計されています。 -INSERT 操作によりテーブルパーツが作成され、それらはバックグラウンドプロセスによって他のテーブルパーツとマージされます。 +`MergeTree` ファミリーのテーブルエンジンの主な特徴は次のとおりです。 -`MergeTree` ファミリーのテーブルエンジンの主な特長: +* テーブルの主キーは、各テーブルパーツ内でのソート順(クラスタ化インデックス)を決定します。主キーは個々の行ではなく、グラニュールと呼ばれる 8192 行のブロックを参照します。これにより、巨大なデータセットに対しても主キーをメインメモリに常駐できる程度の小ささに保ちながら、ディスク上のデータへ高速にアクセスできます。 -- テーブルの主キーは、各テーブルパーツ内のソート順(クラスタ化インデックス)を決定します。主キーは個々の行ではなく、グラニュールと呼ばれる 8192 行のブロックを参照します。これにより、巨大なデータセットの主キーであっても主メモリに常駐できる程度に小さく保たれつつ、ディスク上のデータへの高速アクセスが可能になります。 +* テーブルは任意のパーティション式でパーティション分割できます。パーティションプルーニングにより、クエリ条件に応じて読み取り対象からパーティションを除外できます。 -- テーブルは任意のパーティション式を用いてパーティション分割できます。パーティションプルーニングにより、クエリが許す場合は読み取り対象からパーティションが除外されます。 +* 可用性の向上、フェイルオーバー、およびゼロダウンタイムでのアップグレードのために、データを複数のクラスタノード間でレプリケートできます。詳しくは [Data replication](/engines/table-engines/mergetree-family/replication.md) を参照してください。 -- 高可用性、フェイルオーバー、およびゼロダウンタイムでのアップグレードのために、複数のクラスタノード間でデータをレプリケートできます。詳細は [Data replication](/engines/table-engines/mergetree-family/replication.md) を参照してください。 - -- `MergeTree` テーブルエンジンは、クエリ最適化を支援するために、さまざまな統計情報の種類とサンプリング手法をサポートします。 +* `MergeTree` テーブルエンジンは、クエリ最適化に役立つさまざまな種類の統計情報とサンプリング手法をサポートします。 :::note 名前は似ていますが、[Merge](/engines/table-engines/special/merge) エンジンは `*MergeTree` エンジンとは異なります。 ::: - - -## テーブルの作成 {#table_engine-mergetree-creating-a-table} +## テーブルの作成 {#table_engine-mergetree-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,197 +56,193 @@ ORDER BY expr [SETTINGS name = value, ...] ``` -パラメータの詳細については、[CREATE TABLE](/sql-reference/statements/create/table.md)文を参照してください。 +パラメータの詳細については、[CREATE TABLE](/sql-reference/statements/create/table.md) ステートメントを参照してください。 -### クエリ句 {#mergetree-query-clauses} +### クエリ構文 {#mergetree-query-clauses} #### ENGINE {#engine} -`ENGINE` — エンジンの名前とパラメータ。`ENGINE = MergeTree()`。`MergeTree`エンジンにはパラメータがありません。 +`ENGINE` — エンジンの名前とパラメータを指定します。`ENGINE = MergeTree()`。`MergeTree` エンジンにはパラメータがありません。 -#### ORDER BY {#order_by} +#### ORDER BY {#order_by} -`ORDER BY` — ソートキー。 +`ORDER BY` — ソートキーです。 -カラム名または任意の式のタプル。例: `ORDER BY (CounterID + 1, EventDate)`。 +カラム名または任意の式のタプルを指定します。例: `ORDER BY (CounterID + 1, EventDate)`。 -プライマリキーが定義されていない場合(つまり`PRIMARY KEY`が指定されていない場合)、ClickHouseはソートキーをプライマリキーとして使用します。 +`PRIMARY KEY` が定義されていない場合(つまり `PRIMARY KEY` が指定されていない場合)、ClickHouse はそのソートキーをプライマリキーとして使用します。 -ソートが不要な場合は、`ORDER BY tuple()`という構文を使用できます。 -または、設定`create_table_empty_primary_key_by_default`が有効になっている場合、`ORDER BY ()`が`CREATE TABLE`文に暗黙的に追加されます。[プライマリキーの選択](#selecting-a-primary-key)を参照してください。 +ソートが不要な場合は、`ORDER BY tuple()` 構文を使用できます。 +また、設定 `create_table_empty_primary_key_by_default` が有効な場合、`ORDER BY ()` が `CREATE TABLE` 文に暗黙的に追加されます。[プライマリキーの選択](#selecting-a-primary-key) を参照してください。 #### PARTITION BY {#partition-by} -`PARTITION BY` — [パーティショニングキー](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。オプション。ほとんどの場合、パーティションキーは不要です。パーティション分割が必要な場合でも、通常は月単位よりも細かいパーティションキーは必要ありません。パーティション分割はクエリを高速化しません(ORDER BY式とは対照的です)。過度に細かいパーティション分割は決して使用しないでください。クライアント識別子や名前でデータをパーティション分割しないでください(代わりに、クライアント識別子や名前をORDER BY式の最初のカラムにしてください)。 +`PARTITION BY` — [パーティションキー](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。省略可能です。ほとんどの場合、パーティションキーは不要であり、パーティション分割が必要な場合でも、通常は月単位より細かい粒度のパーティションキーは必要ありません。パーティション分割は(ORDER BY 式とは対照的に)クエリを高速化しません。パーティションは決して細かくしすぎないでください。データをクライアント識別子や名前でパーティション分割しないでください(その代わりに、クライアント識別子または名前を ORDER BY 式の先頭のカラムにしてください)。 -月単位でパーティション分割するには、`toYYYYMM(date_column)`式を使用します。ここで`date_column`は[Date](/sql-reference/data-types/date.md)型の日付を持つカラムです。この場合のパーティション名は`"YYYYMM"`形式になります。 +月単位でパーティション分割するには、`toYYYYMM(date_column)` 式を使用します。ここで、`date_column` は [Date](/sql-reference/data-types/date.md) 型の日付を保持するカラムです。この場合のパーティション名は `"YYYYMM"` 形式になります。 #### PRIMARY KEY {#primary-key} -`PRIMARY KEY` — [ソートキーと異なる](#choosing-a-primary-key-that-differs-from-the-sorting-key)場合のプライマリキー。オプション。 +`PRIMARY KEY` — [ソートキーと異なる場合](#choosing-a-primary-key-that-differs-from-the-sorting-key)に指定するプライマリキーです。省略可能です。 -ソートキーを指定すると(`ORDER BY`句を使用)、暗黙的にプライマリキーが指定されます。 -通常、ソートキーに加えてプライマリキーを指定する必要はありません。 +ソートキー(`ORDER BY` 句)を指定すると、暗黙的にプライマリキーも指定されます。 +通常、ソートキーとは別にプライマリキーを指定する必要はありません。 #### SAMPLE BY {#sample-by} -`SAMPLE BY` — サンプリング式。オプション。 +`SAMPLE BY` — サンプリング用の式です。省略可能です。 -指定する場合は、プライマリキーに含まれている必要があります。 -サンプリング式は符号なし整数を返す必要があります。 +指定する場合は、主キーに含まれている必要があります。 +サンプリング式は符号なし整数値を返さなければなりません。 -例: `SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`。 +例: `SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`. #### TTL {#ttl} -`TTL` — 行の保存期間と[ディスクとボリューム間](#table_engine-mergetree-multiple-volumes)での自動パーツ移動のロジックを指定するルールのリスト。オプション。 +`TTL` — 行の保存期間と、[ディスクおよびボリューム間](#table_engine-mergetree-multiple-volumes)の自動的なパーツ移動ロジックを指定するルールのリストです。省略可能です。 -式は`Date`または`DateTime`を返す必要があります。例: `TTL date + INTERVAL 1 DAY`。 +式は `Date` または `DateTime` 型の値を返す必要があります。例: `TTL date + INTERVAL 1 DAY`。 +ルール種別 `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` は、式が満たされた(現在時刻に達した)ときにそのパーツに対して実行されるアクションを指定します。具体的には、有効期限切れの行の削除、パーツ内のすべての行に対して式が満たされている場合にそのパーツを指定したディスク(`TO DISK 'xxx'`)またはボリューム(`TO VOLUME 'xxx'`)へ移動すること、あるいは有効期限切れの行の値を集約することです。ルールのデフォルト種別は削除(`DELETE`)です。複数のルールを指定できますが、`DELETE` ルールは 1 つまでにする必要があります。 -ルール `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` のタイプは、式が満たされた場合(現在時刻に達した場合)にパートに対して実行されるアクションを指定します:期限切れ行の削除、パートの移動(パート内のすべての行で式が満たされた場合)を指定されたディスク(`TO DISK 'xxx'`)またはボリューム(`TO VOLUME 'xxx'`)へ、または期限切れ行の値の集約です。ルールのデフォルトタイプは削除(`DELETE`)です。複数のルールのリストを指定できますが、`DELETE` ルールは1つまでです。 +詳細については、[カラムおよびテーブルの TTL](#table_engine-mergetree-ttl) を参照してください。 -詳細については、[カラムとテーブルのTTL](#table_engine-mergetree-ttl)を参照してください。 +#### 設定 {#settings} -#### SETTINGS {#settings} +[MergeTree Settings](../../../operations/settings/merge-tree-settings.md) を参照してください。 -[MergeTree設定](../../../operations/settings/merge-tree-settings.md)を参照してください。 - -**セクション設定の例** +**Sections 設定の例** ```sql ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity=8192 ``` -この例では、月ごとのパーティショニングを設定しています。 +この例では、パーティションを月単位で設定しています。 -また、ユーザーIDによるハッシュとしてサンプリング用の式を設定しています。これにより、各`CounterID`と`EventDate`に対してテーブル内のデータを疑似ランダム化できます。データを選択する際に[SAMPLE](/sql-reference/statements/select/sample)句を定義すると、ClickHouseはユーザーのサブセットに対して均等に疑似ランダムなデータサンプルを返します。 +また、ユーザー ID をハッシュ化したものをサンプリング用の式として設定しています。これにより、テーブル内のデータを各 `CounterID` と `EventDate` ごとに擬似乱数的に分散させることができます。データを選択する際に [SAMPLE](/sql-reference/statements/select/sample) 句を指定すると、ClickHouse はユーザーのサブセットに対して偏りの少ない擬似乱数サンプルデータを返します。 -`index_granularity`設定は、8192がデフォルト値であるため省略できます。 +`index_granularity` 設定は省略できます。デフォルト値が 8192 のためです。
+ テーブル作成の非推奨メソッド -非推奨のテーブル作成方法 + :::note + 新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上記で説明した方法に切り替えてください。 + ::: -:::note -新しいプロジェクトではこの方法を使用しないでください。可能であれば、古いプロジェクトを上記の方法に切り替えてください。 -::: + ```sql + CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] + ( + name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], + name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], + ... + ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) + ``` -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) -``` + **MergeTree() のパラメータ** -**MergeTree()パラメータ** + * `date-column` — [Date](/sql-reference/data-types/date.md) 型のカラム名。ClickHouse はこのカラムに基づいて自動的に月ごとのパーティションを作成します。パーティション名は `"YYYYMM"` 形式になります。 + * `sampling_expression` — サンプリング用の式。 + * `(primary, key)` — 主キー。型: [Tuple()](/sql-reference/data-types/tuple.md) + * `index_granularity` — インデックスの粒度。インデックスの「マーク」間のデータ行数。8192 という値はほとんどのユースケースに適しています。 -- `date-column` — [Date](/sql-reference/data-types/date.md)型のカラムの名前。ClickHouseはこのカラムに基づいて月ごとのパーティションを自動的に作成します。パーティション名は`"YYYYMM"`形式です。 -- `sampling_expression` — サンプリング用の式。 -- `(primary, key)` — プライマリキー。型: [Tuple()](/sql-reference/data-types/tuple.md) -- `index_granularity` — インデックスの粒度。インデックスの「マーク」間のデータ行数。ほとんどのタスクでは8192の値が適切です。 + **例** -**例** - -```sql -MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) -``` - -`MergeTree`エンジンは、メインエンジン設定方法の上記の例と同じ方法で設定されます。 + ```sql + MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) + ``` + `MergeTree` エンジンは、メインのエンジン構成方法について上記の例と同様に設定されます。
- ## データストレージ {#mergetree-data-storage} -テーブルは、プライマリキーでソートされたデータパートで構成されます。 - -テーブルにデータが挿入されると、個別のデータパートが作成され、それぞれがプライマリキーで辞書順にソートされます。例えば、プライマリキーが`(CounterID, Date)`の場合、パート内のデータは`CounterID`でソートされ、各`CounterID`内では`Date`で順序付けられます。 +テーブルは、主キーでソートされたデータパーツから構成されます。 -異なるパーティションに属するデータは、異なるパートに分離されます。ClickHouseはバックグラウンドで、より効率的なストレージのためにデータパートをマージします。異なるパーティションに属するパートはマージされません。マージメカニズムは、同じプライマリキーを持つすべての行が同じデータパートに存在することを保証しません。 +テーブルにデータが挿入されると、個別のデータパーツが作成され、それぞれが主キーに従って辞書順でソートされます。たとえば、主キーが `(CounterID, Date)` の場合、そのパーツ内のデータはまず `CounterID` でソートされ、各 `CounterID` の範囲内では `Date` の順に並びます。 -データパートは`Wide`または`Compact`形式で保存できます。`Wide`形式では、各カラムがファイルシステム内の個別のファイルに保存され、`Compact`形式ではすべてのカラムが1つのファイルに保存されます。`Compact`形式は、小規模で頻繁な挿入のパフォーマンスを向上させるために使用できます。 +異なるパーティションに属するデータは、異なるパーツに分離されます。バックグラウンドで ClickHouse は、より効率的に保存できるようにデータパーツをマージします。異なるパーティションに属するパーツはマージされません。マージの仕組みは、同じ主キーを持つすべての行が同じデータパーツ内に配置されることを保証しません。 -データ保存形式は、テーブルエンジンの`min_bytes_for_wide_part`および`min_rows_for_wide_part`設定によって制御されます。データパート内のバイト数または行数が対応する設定値より少ない場合、パートは`Compact`形式で保存されます。それ以外の場合は`Wide`形式で保存されます。これらの設定がいずれも設定されていない場合、データパートは`Wide`形式で保存されます。 +データパーツは `Wide` または `Compact` フォーマットで保存できます。`Wide` フォーマットでは各カラムがファイルシステム上の別々のファイルに保存され、`Compact` フォーマットではすべてのカラムが 1 つのファイルに保存されます。`Compact` フォーマットは、小さなデータの頻繁な挿入時のパフォーマンス向上に利用できます。 -各データパートは論理的にグラニュールに分割されます。グラニュールは、ClickHouseがデータを選択する際に読み取る最小の不可分なデータセットです。ClickHouseは行や値を分割しないため、各グラニュールには常に整数個の行が含まれます。グラニュールの最初の行には、その行のプライマリキーの値がマークされます。各データパートに対して、ClickHouseはマークを保存するインデックスファイルを作成します。各カラムについて、プライマリキーに含まれるかどうかに関わらず、ClickHouseは同じマークを保存します。これらのマークにより、カラムファイル内のデータを直接検索できます。 +データの保存形式は、テーブルエンジンの `min_bytes_for_wide_part` および `min_rows_for_wide_part` 設定によって制御されます。データパーツ内のバイト数または行数が対応する設定値より小さい場合、そのパーツは `Compact` フォーマットで保存されます。そうでない場合は `Wide` フォーマットで保存されます。これらの設定がどちらも指定されていない場合、データパーツは `Wide` フォーマットで保存されます。 -グラニュールのサイズは、テーブルエンジンの`index_granularity`および`index_granularity_bytes`設定によって制限されます。グラニュール内の行数は、行のサイズに応じて`[1, index_granularity]`の範囲内にあります。単一行のサイズが設定値より大きい場合、グラニュールのサイズは`index_granularity_bytes`を超えることがあります。この場合、グラニュールのサイズは行のサイズと等しくなります。 +各データパーツは論理的にグラニュールに分割されます。グラニュールは、ClickHouse がデータを選択する際に読み取る最小の不可分なデータ集合です。ClickHouse は行や値を分割しないため、各グラニュールは常に整数個の行を含みます。グラニュールの先頭行には、その行の主キーの値によってマークが付けられます。各データパーツごとに、ClickHouse はこれらのマークを保存するインデックスファイルを作成します。各カラムに対して、主キーに含まれるかどうかに関わらず、ClickHouse は同じマークも保存します。これらのマークによって、カラムファイル内のデータを直接特定できます。 +グラニュールサイズは、テーブルエンジンの `index_granularity` および `index_granularity_bytes` 設定によって制限されます。グラニュール内の行数は、行のサイズに応じて `[1, index_granularity]` の範囲になります。単一行のサイズが設定値より大きい場合、グラニュールのサイズは `index_granularity_bytes` を超えることがあります。この場合、グラニュールのサイズはその行のサイズと等しくなります。 ## クエリにおける主キーとインデックス {#primary-keys-and-indexes-in-queries} -`(CounterID, Date)` という主キーを例に取ります。この場合、ソートとインデックスは次のように図示できます: +例として、主キー `(CounterID, Date)` を取り上げます。この場合、並び順とインデックスは次のように示されます。 ```text -Whole data: [---------------------------------------------] +全データ: [---------------------------------------------] CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] -Marks: | | | | | | | | | | | +マーク: | | | | | | | | | | | a,1 a,2 a,3 b,3 e,2 e,3 g,1 h,2 i,1 i,3 l,3 -Marks numbers: 0 1 2 3 4 5 6 7 8 9 10 +マーク番号: 0 1 2 3 4 5 6 7 8 9 10 ``` -データクエリが次のように指定された場合: +データクエリが次のように指定されている場合: -- `CounterID in ('a', 'h')` の場合、サーバーはマーク `[0, 3)` および `[6, 8)` の範囲のデータを読み取ります。 -- `CounterID IN ('a', 'h') AND Date = 3` の場合、サーバーはマーク `[1, 3)` および `[7, 8)` の範囲のデータを読み取ります。 -- `Date = 3` の場合、サーバーはマーク `[1, 10]` の範囲のデータを読み取ります。 +* `CounterID in ('a', 'h')` の場合、サーバーはマーク範囲 `[0, 3)` および `[6, 8)` のデータを読み込みます。 +* `CounterID IN ('a', 'h') AND Date = 3` の場合、サーバーはマーク範囲 `[1, 3)` および `[7, 8)` のデータを読み込みます。 +* `Date = 3` の場合、サーバーはマーク範囲 `[1, 10]` のデータを読み込みます。 -上記の例は、フルスキャンよりもインデックスを使用する方が常に効果的であることを示しています。 +上記の例から、常にフルスキャンよりもインデックスを使用する方が効率的であることがわかります。 -スパースインデックスでは、余分なデータが読み取られることがあります。主キーの単一範囲を読み取る際、各データブロックで最大 `index_granularity * 2` 行の余分な行が読み取られる可能性があります。 +疎なインデックスでは、追加のデータが読み込まれることがあります。主キーの単一の範囲を読み込む場合、各データブロックで最大 `index_granularity * 2` 行まで余分な行が読み込まれる可能性があります。 -スパースインデックスを使用すると、非常に多数のテーブル行を扱うことができます。これは、ほとんどの場合、このようなインデックスがコンピュータのRAMに収まるためです。 +疎なインデックスを使用すると、非常に多くのテーブル行を扱うことができます。ほとんどの場合、このようなインデックスはコンピュータの RAM に収まるためです。 -ClickHouseは一意な主キーを必要としません。同じ主キーを持つ複数の行を挿入できます。 +ClickHouse では、一意なプライマリキーは不要です。同じプライマリキーを持つ複数の行を挿入できます。 -`PRIMARY KEY` および `ORDER BY` 句で `Nullable` 型の式を使用できますが、強く非推奨です。この機能を有効にするには、[allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 設定をオンにしてください。`ORDER BY` 句における `NULL` 値には [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) の原則が適用されます。 +`PRIMARY KEY` および `ORDER BY` 句では `Nullable` 型の式を使用できますが、これは強く非推奨です。この機能を許可するには、[allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 設定を有効にします。`ORDER BY` 句での `NULL` 値には、[NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) の原則が適用されます。 ### 主キーの選択 {#selecting-a-primary-key} -主キーの列数に明示的な制限はありません。データ構造に応じて、主キーに含める列を増減できます。これにより次のような効果が得られる可能性があります: +主キーに含める列数には明確な制限はありません。データ構造に応じて、主キーに含める列を増やしたり減らしたりできます。これにより次の効果が得られる可能性があります。 -- インデックスのパフォーマンスを向上させる。 +* インデックスのパフォーマンスが向上する。 - 主キーが `(a, b)` の場合、次の条件が満たされていれば、別の列 `c` を追加することでパフォーマンスが向上します: - - 列 `c` に対する条件を持つクエリが存在する。 - - `(a, b)` の値が同一である長いデータ範囲(`index_granularity` の数倍の長さ)が一般的である。言い換えれば、別の列を追加することで、かなり長いデータ範囲をスキップできる場合。 + 主キーが `(a, b)` の場合に、さらに列 `c` を追加すると、次の条件が満たされるときにパフォーマンスが向上します。 -- データ圧縮を向上させる。 + * 列 `c` に条件を持つクエリが存在する。 + * `(a, b)` の値が同一である長いデータ範囲(`index_granularity` の数倍の長さ)がよく現れる。言い換えると、列を追加することで、かなり長いデータ範囲をスキップできる場合です。 - ClickHouseは主キーでデータをソートするため、一貫性が高いほど圧縮率が向上します。 +* データ圧縮が向上する。 -- [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) および [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) エンジンでデータパーツをマージする際に追加のロジックを提供する。 + ClickHouse はデータを主キーでソートして保存するため、データの並びの一貫性が高いほど圧縮率が向上します。 - この場合、主キーとは異なる_ソートキー_を指定することが理にかなっています。 +* [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) および [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) エンジンでデータパーツをマージする際に、追加のロジックを提供できる。 -長い主キーは挿入パフォーマンスとメモリ消費に悪影響を及ぼしますが、主キーの余分な列は `SELECT` クエリ実行時のClickHouseのパフォーマンスには影響しません。 + この場合、主キーとは異なる *sorting key* を指定することが有効です。 -`ORDER BY tuple()` 構文を使用して、主キーなしでテーブルを作成できます。この場合、ClickHouseはデータを挿入順に格納します。`INSERT ... SELECT` クエリでデータを挿入する際にデータの順序を保持したい場合は、[max_insert_threads = 1](/operations/settings/settings#max_insert_threads) を設定してください。 +長い主キーは挿入パフォーマンスやメモリ消費に悪影響を及ぼしますが、主キーに余分な列があっても、`SELECT` クエリ時の ClickHouse のパフォーマンスには影響しません。 -初期順序でデータを選択するには、[シングルスレッド](/operations/settings/settings.md/#max_threads)の `SELECT` クエリを使用してください。 +`ORDER BY tuple()` 構文を使うことで、主キーなしのテーブルを作成できます。この場合、ClickHouse はデータを挿入順に保存します。`INSERT ... SELECT` クエリで挿入時のデータ順序を保持したい場合は、[max_insert_threads = 1](/operations/settings/settings#max_insert_threads) を設定してください。 -### ソートキーとは異なる主キーの選択 {#choosing-a-primary-key-that-differs-from-the-sorting-key} +挿入時の順序でデータを取得するには、[single-threaded](/operations/settings/settings.md/#max_threads) な `SELECT` クエリを使用します。 +### 並べ替えキーと異なる主キーを選択する {#choosing-a-primary-key-that-differs-from-the-sorting-key} -プライマリキー(各マークに対してインデックスファイルに書き込まれる値の式)を、ソートキー(データパート内の行をソートするための式)とは異なるものとして指定することができます。この場合、プライマリキー式のタプルは、ソートキー式のタプルの接頭辞である必要があります。 +主キー(各マークごとのインデックスファイルに書き込まれる値を持つ式)は、並べ替えキー(データパーツ内の行を並べ替えるための式)とは異なるものを指定できます。この場合、主キーの式タプルは、並べ替えキーの式タプルのプレフィックスでなければなりません。 -この機能は、[SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md)および[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md)テーブルエンジンを使用する際に有用です。これらのエンジンを使用する一般的なケースでは、テーブルには2種類のカラムがあります:_ディメンション_と_メジャー_です。典型的なクエリは、任意の`GROUP BY`でメジャーカラムの値を集計し、ディメンションでフィルタリングします。SummingMergeTreeとAggregatingMergeTreeは、ソートキーの値が同じ行を集計するため、すべてのディメンションをソートキーに追加することが自然です。その結果、キー式は長いカラムのリストで構成され、このリストは新しく追加されるディメンションで頻繁に更新する必要があります。 +この機能は [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) および +[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md) テーブルエンジンを使用する際に有用です。これらのエンジンを利用する一般的なケースでは、テーブルには 2 種類のカラム、つまり *dimensions* と *measures* があります。典型的なクエリでは、任意の `GROUP BY` を用いて measure カラムの値を集約し、dimensions でフィルタリングします。SummingMergeTree と AggregatingMergeTree は並べ替えキーの値が同一の行を集約するため、すべての dimensions を並べ替えキーに含めるのが自然です。その結果、キー式は多数のカラムから成る長いリストとなり、新たな dimensions を追加するたびにこのリストを頻繁に更新しなければなりません。 -この場合、効率的な範囲スキャンを提供する少数のカラムのみをプライマリキーに残し、残りのディメンションカラムをソートキーのタプルに追加することが合理的です。 +このような場合、効率的なレンジスキャンを行うために必要な少数のカラムだけを主キーに残し、残りの dimension カラムを並べ替えキーのタプルに追加するのが理にかなっています。 -ソートキーの[ALTER](/sql-reference/statements/alter/index.md)は軽量な操作です。新しいカラムがテーブルとソートキーに同時に追加される際、既存のデータパートを変更する必要がないためです。古いソートキーが新しいソートキーの接頭辞であり、新しく追加されたカラムにはデータが存在しないため、テーブル変更の時点でデータは古いソートキーと新しいソートキーの両方でソートされています。 +並べ替えキーの [ALTER](/sql-reference/statements/alter/index.md) は軽量な操作です。新しいカラムがテーブルおよび並べ替えキーに同時に追加される場合、既存のデータパーツを変更する必要がないためです。古い並べ替えキーが新しい並べ替えキーのプレフィックスであり、新しく追加されたカラムにはデータが存在しないので、テーブルを変更した時点では、データは古い並べ替えキーと新しい並べ替えキーの両方に従って並べ替えられていることになります。 -### クエリにおけるインデックスとパーティションの使用 {#use-of-indexes-and-partitions-in-queries} +### クエリにおけるインデックスとパーティションの利用 {#use-of-indexes-and-partitions-in-queries} -`SELECT`クエリに対して、ClickHouseはインデックスが使用可能かどうかを分析します。`WHERE/PREWHERE`句に、等価または不等価比較演算を表す式(結合要素の1つとして、または全体として)がある場合、またはプライマリキーまたはパーティショニングキーに含まれるカラムや式、あるいはこれらのカラムの特定の部分的反復関数、またはこれらの式の論理関係に対して、固定接頭辞を持つ`IN`または`LIKE`がある場合、インデックスを使用できます。 +`SELECT` クエリに対して、ClickHouse はインデックスを利用できるかどうかを解析して判断します。`WHERE/PREWHERE` 句に、等価比較または不等価比較の演算(連言要素の 1 つ、もしくは式全体)を表す式が含まれている場合、あるいは、プライマリキーまたはパーティションキーに含まれる列や式、それらの列に対する特定の部分的に反復的な関数、さらにそれらの式同士の論理関係に対して、固定プレフィックスを伴う `IN` または `LIKE` が含まれている場合、インデックスを使用できます。 -したがって、プライマリキーの1つまたは複数の範囲に対してクエリを高速に実行することができます。この例では、特定のトラッキングタグ、特定のタグと日付範囲、特定のタグと日付、日付範囲を持つ複数のタグなどに対してクエリを実行する場合に高速になります。 +そのため、プライマリキーの 1 つまたは複数の範囲に対してクエリを高速に実行できます。この例では、特定のトラッキングタグ、特定のタグと日付範囲、特定のタグと特定の日付、複数のタグと日付範囲などに対するクエリは高速に実行されます。 -次のように設定されたエンジンを見てみましょう: +次のように設定されたエンジンを見てみましょう。 ```sql ENGINE MergeTree() @@ -258,7 +251,7 @@ ORDER BY (CounterID, EventDate) SETTINGS index_granularity=8192 ``` -この場合、以下のクエリにおいて: +この場合、クエリは次のようになります: ```sql SELECT count() FROM table @@ -276,42 +269,41 @@ AND CounterID IN (101500, 731962, 160656) AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) ``` -ClickHouseは、プライマリキーインデックスを使用して不適切なデータを削減し、月次パーティショニングキーを使用して不適切な日付範囲のパーティションを削減します。 +ClickHouse は、プライマリキーインデックスを使用して不適切なデータを除外し、月単位のパーティショニングキーを使用して不適切な日付範囲にあるパーティションを除外します。 -上記のクエリは、複雑な式に対してもインデックスが使用されることを示しています。テーブルからの読み取りは、インデックスの使用がフルスキャンよりも遅くならないように構成されています。 +上記のクエリは、複雑な式に対してもインデックスが使用されることを示しています。テーブルからの読み取り処理は、インデックスを使用してもフルスキャンより遅くならないように設計されています。 -以下の例では、インデックスを使用できません。 +次の例では、インデックスを利用できません。 ```sql SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' ``` -クエリ実行時にClickHouseがインデックスを使用できるかどうかを確認するには、[force_index_by_date](/operations/settings/settings.md/#force_index_by_date)および[force_primary_key](/operations/settings/settings#force_primary_key)設定を使用してください。 - -月次パーティショニングのキーにより、適切な範囲の日付を含むデータブロックのみを読み取ることができます。この場合、データブロックには多数の日付(最大で1か月全体)のデータが含まれる可能性があります。ブロック内では、データはプライマリキーでソートされており、プライマリキーの最初のカラムに日付が含まれていない場合があります。このため、プライマリキーの接頭辞を指定しない日付条件のみを使用するクエリでは、単一の日付に対するよりも多くのデータが読み取られることになります。 +クエリ実行時に ClickHouse がインデックスを利用できるかどうかを確認するには、設定 [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) と [force_primary_key](/operations/settings/settings#force_primary_key) を使用します。 -### 部分的に単調なプライマリキーに対するインデックスの使用 {#use-of-index-for-partially-monotonic-primary-keys} +月単位のパーティションキーは、指定した範囲に含まれる日付を持つデータブロックだけを読み取れるようにします。この場合、データブロックには多数の日付(最大で 1 か月分)に対応するデータが含まれている可能性があります。ブロック内ではデータは主キーでソートされていますが、主キーの先頭の列として日付が含まれていない場合があります。そのため、主キーのプレフィックスを指定せずに日付条件のみを含むクエリを使用すると、単一の日付だけを対象とする場合よりも多くのデータを読み取ることになります。 +### 部分的に単調な主キーに対するインデックスの利用 {#use-of-index-for-partially-monotonic-primary-keys} -例えば、月の日付を考えてみましょう。これらは1か月の間は[単調数列](https://en.wikipedia.org/wiki/Monotonic_function)を形成しますが、より長い期間では単調ではありません。これは部分的単調数列です。ユーザーが部分的単調プライマリキーを持つテーブルを作成すると、ClickHouseは通常通りスパースインデックスを作成します。ユーザーがこの種のテーブルからデータを選択すると、ClickHouseはクエリ条件を分析します。ユーザーがインデックスの2つのマーク間のデータを取得したい場合で、両方のマークが1か月以内に収まる場合、ClickHouseはこの特定のケースでインデックスを使用できます。これは、クエリのパラメータとインデックスマーク間の距離を計算できるためです。 +例として、月の日付を考えてみます。1 か月の中では [単調な数列](https://en.wikipedia.org/wiki/Monotonic_function) を形成しますが、より長い期間では単調ではありません。これは部分的に単調な数列です。ユーザーが部分的に単調な主キーでテーブルを作成した場合、ClickHouse は通常どおりスパースインデックスを作成します。ユーザーがこの種のテーブルからデータを `SELECT` する際、ClickHouse はクエリ条件を解析します。ユーザーがインデックスの 2 つのマークの間のデータを取得しようとしており、その両方のマークが同じ 1 か月の範囲内に収まる場合、ClickHouse はこの特定のケースではインデックスを利用できます。これは、クエリパラメータとインデックスマークとの距離を計算できるためです。 -クエリパラメータ範囲内のプライマリキーの値が単調数列を表さない場合、ClickHouseはインデックスを使用できません。この場合、ClickHouseはフルスキャン方式を使用します。 +クエリのパラメータ範囲内に含まれる主キーの値が単調な数列を表さない場合、ClickHouse はインデックスを利用できません。この場合、ClickHouse はフルスキャンを行います。 -ClickHouseはこのロジックを月の日付の数列だけでなく、部分的単調数列を表すあらゆるプライマリキーに対して使用します。 +ClickHouse は、このロジックを月の日付の数列に対してだけでなく、部分的に単調な数列を表すあらゆる主キーに対しても適用します。 -### データスキップインデックス {#table_engine-mergetree-data_skipping-indexes} +### データスキップインデックス {#table_engine-mergetree-data_skipping-indexes} -インデックス宣言は`CREATE`クエリのカラムセクションに記述します。 +インデックスの宣言は、`CREATE` クエリのカラム定義セクション内に記述します。 ```sql INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] ``` -`*MergeTree`ファミリーのテーブルでは、データスキップインデックスを指定できます。 +`*MergeTree` ファミリーのテーブルでは、データスキップインデックスを指定できます。 -これらのインデックスは、`granularity_value`個のグラニュール(グラニュールのサイズはテーブルエンジンの`index_granularity`設定で指定されます)で構成されるブロック上の指定された式に関する情報を集約します。その後、これらの集約は`SELECT`クエリで使用され、`where`句を満たさない大きなデータブロックをスキップすることで、ディスクから読み取るデータ量を削減します。 +これらのインデックスは、ブロック上で指定された式に関する情報の一部を集約します。ブロックは `granularity_value` 個のグラニュールから構成されます(グラニュールのサイズはテーブルエンジンの `index_granularity` 設定で指定します)。その後、これらの集約値は `SELECT` クエリの実行時に使用され、`WHERE` 句の条件を満たし得ない大きなデータブロックをスキップすることで、ディスクから読み取るデータ量を削減します。 -`GRANULARITY`句は省略可能で、`granularity_value`のデフォルト値は1です。 +`GRANULARITY` 句は省略可能であり、`granularity_value` のデフォルト値は 1 です。 **例** @@ -329,7 +321,7 @@ CREATE TABLE table_name ... ``` -この例のインデックスは、以下のクエリでディスクから読み取るデータ量を削減するためにClickHouseで使用できます: +サンプルで定義したインデックスは、以下のクエリでは ClickHouse がディスクから読み取るデータ量を減らすために利用できます。 ```sql SELECT count() FROM table WHERE u64 == 10; @@ -337,7 +329,7 @@ SELECT count() FROM table WHERE u64 * i32 >= 1234 SELECT count() FROM table WHERE u64 * length(s) == 1234 ``` -データスキップインデックスは複合カラムに対しても作成できます: +データスキップインデックスは複合列にも作成できます: ```sql -- Map型のカラムに対して: @@ -355,88 +347,87 @@ INDEX nested_2_index col.nested_col2 TYPE bloom_filter ### スキップインデックスの種類 {#skip-index-types} -`MergeTree`テーブルエンジンは以下の種類のスキップインデックスをサポートしています。 -スキップインデックスをパフォーマンス最適化に使用する方法の詳細については、 -["ClickHouseデータスキップインデックスの理解"](/optimize/skipping-indexes)を参照してください。 +`MergeTree` テーブルエンジンは、次の種類のスキップインデックスをサポートします。 +スキップインデックスをパフォーマンス最適化にどのように利用できるかについては、 +["ClickHouse のデータスキッピングインデックスを理解する"](/optimize/skipping-indexes) を参照してください。 -- [`MinMax`](#minmax)インデックス -- [`Set`](#set)インデックス -- [`bloom_filter`](#bloom-filter)インデックス -- [`ngrambf_v1`](#n-gram-bloom-filter)インデックス -- [`tokenbf_v1`](#token-bloom-filter)インデックス +* [`MinMax`](#minmax) インデックス +* [`Set`](#set) インデックス +* [`bloom_filter`](#bloom-filter) インデックス +* [`ngrambf_v1`](#n-gram-bloom-filter) インデックス +* [`tokenbf_v1`](#token-bloom-filter) インデックス -#### MinMaxスキップインデックス {#minmax} +#### MinMax スキップインデックス {#minmax} -各インデックスグラニュールに対して、式の最小値と最大値が格納されます。 -(式が`tuple`型の場合、各タプル要素の最小値と最大値が格納されます。) +各インデックスグラニュールごとに、式の最小値と最大値が格納されます。 +(式の型が `tuple` の場合は、各タプル要素ごとに最小値と最大値が格納されます。) -```text title="構文" +```text title="Syntax" minmax ``` #### Set {#set} -各インデックスグラニュールに対して、指定された式の最大`max_rows`個のユニークな値が格納されます。 -`max_rows = 0`は「すべてのユニークな値を格納する」ことを意味します。 +各インデックスグラニュールごとに、指定された式のユニークな値が最大 `max_rows` 個まで保存されます。 +`max_rows = 0` の場合は「すべてのユニークな値を保存する」ことを意味します。 -```text title="構文" +```text title="Syntax" set(max_rows) ``` -#### Bloomフィルタ {#bloom-filter} +#### ブルームフィルタ {#bloom-filter} -各インデックスグラニュールに対して、指定されたカラムの[Bloomフィルタ](https://en.wikipedia.org/wiki/Bloom_filter)を格納します。 +各インデックスグラニュールは、指定された列に対して [Bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) を保持します。 -```text title="構文" +```text title="Syntax" bloom_filter([false_positive_rate]) ``` -`false_positive_rate`パラメータは0から1の間の値を取ることができ(デフォルト: `0.025`)、偽陽性を生成する確率を指定します(これにより読み取るデータ量が増加します)。 - - -以下のデータ型がサポートされています: - -- `(U)Int*` -- `Float*` -- `Enum` -- `Date` -- `DateTime` -- `String` -- `FixedString` -- `Array` -- `LowCardinality` -- `Nullable` -- `UUID` -- `Map` - -:::note Mapデータ型: キーまたは値を使用したインデックス作成の指定 -`Map`データ型の場合、[`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys)または[`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues)関数を使用して、キーまたは値のどちらに対してインデックスを作成するかを指定できます。 +`false_positive_rate` パラメータには 0 から 1 の値を指定できます(既定値: `0.025`)。このパラメータは、偽陽性が発生する確率(これにより読み取られるデータ量が増加します)を指定します。 + +次のデータ型がサポートされています。 + +* `(U)Int*` +* `Float*` +* `Enum` +* `Date` +* `DateTime` +* `String` +* `FixedString` +* `Array` +* `LowCardinality` +* `Nullable` +* `UUID` +* `Map` + +:::note Map データ型: キーまたは値に対するインデックス作成の指定 +`Map` データ型では、クライアントは [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) または [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 関数を使用して、キーに対してインデックスを作成するか、値に対してインデックスを作成するかを指定できます。 ::: -#### N-gramブルームフィルタ {#n-gram-bloom-filter} +#### N-gram ブルームフィルタ {#n-gram-bloom-filter} -各インデックスグラニュールに対して、指定されたカラムの[n-gram](https://en.wikipedia.org/wiki/N-gram)用の[ブルームフィルタ](https://en.wikipedia.org/wiki/Bloom_filter)を格納します。 +各インデックスグラニュールには、指定された列の [n-gram](https://en.wikipedia.org/wiki/N-gram) に対する [bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) が格納されます。 -```text title="構文" +```text title="Syntax" ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` -| パラメータ | 説明 | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `n` | ngramのサイズ | -| `size_of_bloom_filter_in_bytes` | ブルームフィルタのサイズ(バイト単位)。圧縮効率が高いため、例えば`256`や`512`などの大きな値を使用できます。 | -| `number_of_hash_functions` | ブルームフィルタで使用されるハッシュ関数の数。 | -| `random_seed` | ブルームフィルタのハッシュ関数用のシード値。 | +| Parameter | Description | +| ------------------------------- | -------------------------------------------------------------------------------- | +| `n` | n-gram のサイズ | +| `size_of_bloom_filter_in_bytes` | Bloom フィルタのサイズ(バイト単位)。ここには `256` や `512` などの大きな値を指定できます。Bloom フィルタは高い圧縮率で格納できます。 | +| `number_of_hash_functions` | Bloom フィルタで使用されるハッシュ関数の数。 | +| `random_seed` | Bloom フィルタのハッシュ関数に使用するシード値。 | -このインデックスは以下のデータ型でのみ動作します: +このインデックスが利用できるデータ型は次のとおりです。 -- [`String`](/sql-reference/data-types/string.md) -- [`FixedString`](/sql-reference/data-types/fixedstring.md) -- [`Map`](/sql-reference/data-types/map.md) +* [`String`](/sql-reference/data-types/string.md) +* [`FixedString`](/sql-reference/data-types/fixedstring.md) +* [`Map`](/sql-reference/data-types/map.md) -`ngrambf_v1`のパラメータを推定するには、以下の[ユーザー定義関数(UDF)](/sql-reference/statements/create/function.md)を使用できます。 +`ngrambf_v1` のパラメータを見積もるには、次の [ユーザー定義関数 (UDF)](/sql-reference/statements/create/function.md) を使用できます。 -```sql title="ngrambf_v1用のUDF" +```sql title="UDFs for ngrambf_v1" CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] AS (total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); @@ -454,16 +445,16 @@ AS (number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) ``` -これらの関数を使用するには、少なくとも以下の2つのパラメータを指定する必要があります: +これらの関数を使用するには、少なくとも次の 2 つのパラメータを指定する必要があります。 -- `total_number_of_all_grams` -- `probability_of_false_positives` +* `total_number_of_all_grams` +* `probability_of_false_positives` -例えば、グラニュールに`4300`個のngramがあり、偽陽性率を`0.0001`未満にしたい場合、 -以下のクエリを実行することで他のパラメータを推定できます: +たとえば、ある granule に `4300` 個の ngram があり、偽陽性の確率を `0.0001` 未満にしたいとします。 +この場合、他のパラメータは次のクエリを実行することで推定できます。 ```sql ---- フィルタのビット数を推定 +--- フィルタ内のビット数を推定 SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; ┌─size_of_bloom_filter_in_bytes─┐ @@ -478,169 +469,161 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_ha └──────────────────────────┘ ``` -もちろん、これらの関数を使用して他の条件に対するパラメータを推定することもできます。 -上記の関数は[こちら](https://hur.st/bloomfilter)のブルームフィルタ計算機を参照しています。 +もちろん、これらの関数は他の条件のパラメータを見積もるためにも使用できます。 +上記の関数は、ブルームフィルター計算ツール[こちら](https://hur.st/bloomfilter)を参照しています。 -#### トークンブルームフィルタ {#token-bloom-filter} +#### トークンブルームフィルター {#token-bloom-filter} -トークンブルームフィルタは`ngrambf_v1`と同じですが、ngramの代わりにトークン(英数字以外の文字で区切られたシーケンス)を格納します。 +トークンブルームフィルターは `ngrambf_v1` と同様ですが、n-gram ではなく、英数字以外の文字で区切られたトークン(文字列のまとまり)を保存します。 -```text title="構文" -tokenbf_v1(size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) +```text title="Syntax" +tokenbf_v1(ブルームフィルタのサイズ(バイト), ハッシュ関数の数, ランダムシード) ``` +#### スパースグラム Bloom フィルター {#sparse-grams-bloom-filter} -#### スパースグラムブルームフィルター {#sparse-grams-bloom-filter} - -スパースグラムブルームフィルターは`ngrambf_v1`と類似していますが、nグラムの代わりに[スパースグラムトークン](/sql-reference/functions/string-functions.md/#sparseGrams)を使用します。 +スパースグラム Bloom フィルターは `ngrambf_v1` と似ていますが、n-gram の代わりに [スパースグラムトークン](/sql-reference/functions/string-functions.md/#sparseGrams) を使用します。 -```text title="構文" +```text title="Syntax" sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` ### テキストインデックス {#text} -全文検索をサポートしています。詳細は[こちら](invertedindexes.md)を参照してください。 +全文検索をサポートします。詳細は[こちら](invertedindexes.md)を参照してください。 -#### ベクトル類似度 {#vector-similarity} +#### ベクトル類似性 {#vector-similarity} -近似最近傍探索をサポートしています。詳細は[こちら](annindexes.md)を参照してください。 +近似最近傍検索をサポートしています。詳細は[こちら](annindexes.md)をご覧ください。 ### 関数のサポート {#functions-support} -`WHERE`句の条件には、カラムを操作する関数の呼び出しが含まれます。カラムがインデックスの一部である場合、ClickHouseは関数を実行する際にこのインデックスの使用を試みます。ClickHouseは、インデックスを使用するための異なる関数のサブセットをサポートしています。 - -`set`型のインデックスはすべての関数で利用できます。その他のインデックス型は以下のようにサポートされています: - - -| 関数(演算子)/インデックス | 主キー | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | text | sparse_grams | -| ------------------------------------------------------------------------------------------------------------------------- | --- | ------ | -------------- | -------------- | ---------------- | ---- | ---------------- | -| [equals(=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [より小さい (`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [大なり (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | - - - -`ngrambf_v1` では、定数引数が ngram のサイズより小さい関数はクエリ最適化に使用できません。 - -(*) `hasTokenCaseInsensitive` および `hasTokenCaseInsensitiveOrNull` を効果的に機能させるには、`tokenbf_v1` インデックスを小文字化したデータに対して作成する必要があります。例えば `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)` のようにします。 +`WHERE` 句の条件式には、カラムを操作する関数呼び出しが含まれます。カラムがインデックスの一部である場合、ClickHouse は関数を評価する際にそのインデックスを利用しようとします。ClickHouse では、インデックスで利用できる関数のサブセットがインデックスタイプごとに異なります。 + +`set` 型のインデックスは、あらゆる関数で利用できます。その他のインデックスタイプは次のようにサポートされます。 + +| 関数(演算子)/ 索引 | 主キー | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | テキスト | +| ------------------------------------------------------------------------------------------------------------------------- | --- | ------ | -------------- | -------------- | ---------------- | ---------------- | ---- | +| [equals (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | +| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [less(`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [greater(`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [以下 (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | +| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | + +`ngrambf_v1` によるクエリ最適化では、定数引数の値が ngram サイズより小さい関数は使用できません。 + +(*) `hasTokenCaseInsensitive` および `hasTokenCaseInsensitiveOrNull` を効果的に機能させるには、`tokenbf_v1` インデックスを小文字化されたデータ上に作成する必要があります。たとえば `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)` のように指定します。 :::note -Bloom filter には偽陽性が発生する可能性があるため、`ngrambf_v1`、`tokenbf_v1`、`sparse_grams`、および `bloom_filter` インデックスは、関数の結果が false であることが期待されるクエリの最適化には使用できません。 - -例えば: - -- 最適化できるもの: - - `s LIKE '%test%'` - - `NOT s NOT LIKE '%test%'` - - `s = 1` - - `NOT s != 1` - - `startsWith(s, 'test')` -- 最適化できないもの: - - `NOT s LIKE '%test%'` - - `s NOT LIKE '%test%'` - - `NOT s = 1` - - `s != 1` - - `NOT startsWith(s, 'test')` -::: +Bloom filter では偽陽性が発生し得るため、`ngrambf_v1`、`tokenbf_v1`、`sparse_grams`、`bloom_filter` 各インデックスは、関数の結果が false になることを前提としてクエリを最適化する目的には使用できません。 +例: +* 最適化可能: + * `s LIKE '%test%'` + * `NOT s NOT LIKE '%test%'` + * `s = 1` + * `NOT s != 1` + * `startsWith(s, 'test')` +* 最適化不可能: + * `NOT s LIKE '%test%'` + * `s NOT LIKE '%test%'` + * `NOT s = 1` + * `s != 1` + * `NOT startsWith(s, 'test')` + ::: ## プロジェクション {#projections} -プロジェクションは[マテリアライズドビュー](/sql-reference/statements/create/view)に似ていますが、パートレベルで定義されます。クエリでの自動使用とともに一貫性を保証します。 +プロジェクションは [マテリアライズドビュー](/sql-reference/statements/create/view) に似ていますが、パーツレベルで定義されます。クエリで自動的に利用されることに加えて、一貫性を保証します。 :::note -プロジェクションを実装する際は、[force_optimize_projection](/operations/settings/settings#force_optimize_projection)設定も考慮してください。 +プロジェクションを実装する際には、[force_optimize_projection](/operations/settings/settings#force_optimize_projection) の設定も併せて検討する必要があります。 ::: -プロジェクションは[FINAL](/sql-reference/statements/select/from#final-modifier)修飾子を使用した`SELECT`文ではサポートされていません。 +プロジェクションは、[FINAL](/sql-reference/statements/select/from#final-modifier) 修飾子付きの `SELECT` ステートメントではサポートされていません。 ### プロジェクションクエリ {#projection-query} -プロジェクションクエリはプロジェクションを定義するものです。親テーブルから暗黙的にデータを選択します。 +プロジェクションクエリは、プロジェクションを定義するクエリです。親テーブルからデータを暗黙的に選択します。 **構文** ```sql SELECT [GROUP BY] [ORDER BY] ``` -プロジェクションは[ALTER](/sql-reference/statements/alter/projection.md)文で変更または削除できます。 +プロジェクションは [ALTER](/sql-reference/statements/alter/projection.md) 文で変更または削除できます。 -### プロジェクションストレージ {#projection-storage} +### プロジェクションのストレージ {#projection-storage} -プロジェクションはパートディレクトリ内に保存されます。インデックスに似ていますが、匿名の`MergeTree`テーブルのパートを保存するサブディレクトリを含みます。テーブルはプロジェクションの定義クエリによって生成されます。`GROUP BY`句がある場合、基盤となるストレージエンジンは[AggregatingMergeTree](aggregatingmergetree.md)となり、すべての集約関数は`AggregateFunction`に変換されます。`ORDER BY`句がある場合、`MergeTree`テーブルはそれをプライマリキー式として使用します。マージプロセス中、プロジェクションパートはそのストレージのマージルーチンを介してマージされます。親テーブルのパートのチェックサムはプロジェクションのパートと結合されます。その他のメンテナンス作業はスキップインデックスと同様です。 +プロジェクションはパーツディレクトリ内に保存されます。インデックスに似ていますが、匿名の `MergeTree` テーブルのパーツを保存するサブディレクトリを含みます。このテーブルは、プロジェクションの定義クエリに基づいて定義されます。`GROUP BY` 句がある場合、下層のストレージエンジンは [AggregatingMergeTree](aggregatingmergetree.md) になり、すべての集約関数は `AggregateFunction` に変換されます。`ORDER BY` 句がある場合、`MergeTree` テーブルはそれを主キー式として使用します。マージ処理中、プロジェクションのパーツは、そのストレージのマージルーチンによってマージされます。親テーブルのパーツのチェックサムは、プロジェクションのパーツと組み合わされます。その他のメンテナンス処理はスキップインデックスと同様です。 ### クエリ解析 {#projection-query-analysis} -1. プロジェクションが指定されたクエリに応答するために使用できるかどうか、つまりベーステーブルへのクエリと同じ結果を生成するかどうかを確認します。 -2. 読み取るグラニュールが最も少ない、最適な実行可能な一致を選択します。 -3. プロジェクションを使用するクエリパイプラインは、元のパートを使用するものとは異なります。一部のパートでプロジェクションが存在しない場合、その場で「プロジェクト」するパイプラインを追加できます。 - +1. プロジェクションがベーステーブルに対するクエリと同じ結果を生成し、与えられたクエリに対する回答として利用できるかを確認します。 +2. 読み取る必要があるグラニュール数が最も少ない、実行可能な最適な一致を選択します。 +3. プロジェクションを利用するクエリパイプラインは、元のパーツを利用するものとは異なります。あるパーツにプロジェクションが存在しない場合、その場で「投影」するためのパイプラインを追加できます。 ## 同時データアクセス {#concurrent-data-access} -テーブルへの同時アクセスには、マルチバージョニングを使用しています。つまり、テーブルの読み取りと更新が同時に行われる場合、データはクエリ実行時点で最新のパーツセットから読み取られます。長時間のロックは発生しません。挿入操作が読み取り操作を妨げることはありません。 +テーブルへの同時アクセスにはマルチバージョン方式を使用します。つまり、テーブルで読み取りと更新が同時に行われている場合でも、クエリ時点で有効なパーツの集合からデータが読み取られます。長時間にわたるロックは発生しません。挿入処理が読み取り操作の妨げになることもありません。 テーブルからの読み取りは自動的に並列化されます。 +## 列およびテーブルの TTL {#table_engine-mergetree-ttl} -## カラムとテーブルのTTL {#table_engine-mergetree-ttl} - -値の有効期間を決定します。 +値の有効期間(time-to-live)を決定します。 -`TTL`句は、テーブル全体および個々のカラムに対して設定できます。テーブルレベルの`TTL`では、ディスクとボリューム間でのデータの自動移動や、すべてのデータが期限切れになったパーツの再圧縮のロジックを指定することもできます。 +`TTL` 句はテーブル全体にも、各列ごとにも設定できます。テーブルレベルの `TTL` では、ディスクやボリューム間でデータを自動的に移動するロジックや、すべてのデータが期限切れになったパーツを再圧縮するロジックも指定できます。 -式は[Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md)、または[DateTime64](/sql-reference/data-types/datetime64.md)データ型として評価される必要があります。 +式は [Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md)、または [DateTime64](/sql-reference/data-types/datetime64.md) 型として評価されなければなりません。 **構文** -カラムの有効期間を設定する: +列の TTL を設定する場合: ```sql TTL time_column TTL time_column + interval ``` -`interval`を定義するには、[時間間隔](/sql-reference/operators#operators-for-working-with-dates-and-times)演算子を使用します。例: +`interval` を定義するには、[time interval](/sql-reference/operators#operators-for-working-with-dates-and-times) 演算子を使用します。たとえば、次のようにします。 ```sql TTL date_time + INTERVAL 1 MONTH TTL date_time + INTERVAL 15 HOUR ``` -### カラムTTL {#mergetree-column-ttl} +### カラム TTL {#mergetree-column-ttl} -カラム内の値が期限切れになると、ClickHouseはそれらをカラムデータ型のデフォルト値で置き換えます。データパート内のすべてのカラム値が期限切れになった場合、ClickHouseはファイルシステム内のデータパートからこのカラムを削除します。 +カラム内の値の有効期限が切れると、ClickHouse はそれらをカラムのデータ型におけるデフォルト値に置き換えます。データパート内のそのカラムの値がすべて有効期限切れになった場合、ClickHouse はファイルシステム上のそのデータパートからこのカラムを削除します。 -`TTL`句はキーカラムには使用できません。 +`TTL` 句はキー列には使用できません。 **例** -#### `TTL`を使用したテーブルの作成: {#creating-a-table-with-ttl} +#### `TTL` を設定したテーブルの作成: {#creating-a-table-with-ttl} ```sql CREATE TABLE tab @@ -655,7 +638,7 @@ PARTITION BY toYYYYMM(d) ORDER BY d; ``` -#### 既存テーブルのカラムにTTLを追加する {#adding-ttl-to-a-column-of-an-existing-table} +#### 既存のテーブルの列に TTL を追加する {#adding-ttl-to-a-column-of-an-existing-table} ```sql ALTER TABLE tab @@ -663,7 +646,7 @@ ALTER TABLE tab c String TTL d + INTERVAL 1 DAY; ``` -#### カラムのTTLを変更する {#altering-ttl-of-the-column} +#### 列のTTLを変更する {#altering-ttl-of-the-column} ```sql ALTER TABLE tab @@ -671,9 +654,9 @@ ALTER TABLE tab c String TTL d + INTERVAL 1 MONTH; ``` -### テーブルTTL {#mergetree-table-ttl} +### テーブルの TTL {#mergetree-table-ttl} -テーブルには、期限切れ行を削除するための式と、[ディスクまたはボリューム](#table_engine-mergetree-multiple-volumes)間でパーツを自動的に移動するための複数の式を設定できます。テーブル内の行が期限切れになると、ClickHouseは対応するすべての行を削除します。パーツの移動または再圧縮の場合、パーツのすべての行が`TTL`式の条件を満たす必要があります。 +テーブルには、有効期限が切れた行を削除するための式と、[ディスクまたはボリューム](#table_engine-mergetree-multiple-volumes)間でパーツを自動的に移動するための複数の式を定義できます。テーブル内の行が有効期限切れになると、ClickHouse は対応する行をすべて削除します。パーツの移動または再圧縮が行われるためには、そのパーツ内のすべての行が `TTL` 式の条件を満たしている必要があります。 ```sql TTL expr @@ -682,27 +665,27 @@ TTL expr [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] ``` -TTLルールのタイプは各TTL式の後に指定できます。これは、式が満たされた(現在時刻に達した)ときに実行されるアクションを決定します: +TTL ルールの種類は、それぞれの TTL 式の後に続けて指定できます。これは、式が満たされた(現在時刻に達した)ときに実行されるアクションを決定します: -- `DELETE` - 期限切れ行を削除する(デフォルトアクション) -- `RECOMPRESS codec_name` - `codec_name`でデータパーツを再圧縮する -- `TO DISK 'aaa'` - パーツをディスク`aaa`に移動する -- `TO VOLUME 'bbb'` - パーツをボリューム`bbb`に移動する -- `GROUP BY` - 期限切れ行を集約する +* `DELETE` - 期限切れの行を削除します(デフォルトのアクション); +* `RECOMPRESS codec_name` - データパートを `codec_name` で再圧縮します; +* `TO DISK 'aaa'` - パートをディスク `aaa` に移動します; +* `TO VOLUME 'bbb'` - パートをボリューム `bbb` に移動します; +* `GROUP BY` - 期限切れの行を集約します。 -`DELETE`アクションは`WHERE`句と組み合わせて使用でき、フィルタリング条件に基づいて期限切れ行の一部のみを削除できます: +`DELETE` アクションは `WHERE` 句と組み合わせて使用でき、フィルタリング条件に基づいて期限切れの行の一部のみを削除できます: ```sql TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' ``` -`GROUP BY`式は、テーブルのプライマリキーの接頭辞である必要があります。 +`GROUP BY` 式はテーブルの主キーの先頭部分でなければなりません。 -カラムが`GROUP BY`式の一部ではなく、`SET`句で明示的に設定されていない場合、結果行にはグループ化された行からの任意の値が含まれます(集約関数`any`が適用されたかのように)。 +ある列が `GROUP BY` 式の一部ではなく、かつ `SET` 句で明示的に設定されていない場合、その列の値には、結果行でグループ化された行のいずれか 1 つからの値が入ります(あたかも集約関数 `any` が適用されたかのように振る舞います)。 **例** -#### `TTL`を使用したテーブルの作成: {#creating-a-table-with-ttl-1} +#### `TTL` を指定したテーブルの作成: {#creating-a-table-with-ttl-1} ```sql CREATE TABLE tab @@ -718,15 +701,14 @@ TTL d + INTERVAL 1 MONTH DELETE, d + INTERVAL 2 WEEK TO DISK 'bbb'; ``` -#### テーブルの`TTL`を変更する: {#altering-ttl-of-the-table} +#### テーブルの `TTL` を変更する: {#altering-ttl-of-the-table} ```sql ALTER TABLE tab MODIFY TTL d + INTERVAL 1 DAY; ``` - -1ヶ月後に行の有効期限が切れるテーブルを作成します。日付が月曜日である期限切れの行は削除されます: +行が 1 か月後に期限切れとなるテーブルを作成します。期限切れとなった行のうち、日付が月曜日であるものが削除されます。 ```sql CREATE TABLE table_with_where @@ -755,7 +737,7 @@ TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPR SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; ``` -期限切れの行が集約されるテーブルを作成します。結果の行では、`x`はグループ化された行全体の最大値、`y`は最小値、`d`はグループ化された行からの任意の値を含みます。 +有効期限切れの行を集約するテーブルを作成します。結果の行では、`x` にはグループ化された行全体での最大値が、`y` には最小値が、`d` にはグループ化された行からのいずれか 1 つの値が含まれます。 ```sql CREATE TABLE table_for_aggregation @@ -773,63 +755,61 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ### 期限切れデータの削除 {#mergetree-removing-expired-data} -`TTL`が期限切れとなったデータは、ClickHouseがデータパートをマージする際に削除されます。 +`TTL` が期限切れになったデータは、ClickHouse がデータパーツをマージする際に削除されます。 -ClickHouseがデータの期限切れを検出すると、スケジュール外のマージを実行します。このようなマージの頻度を制御するには、`merge_with_ttl_timeout`を設定できます。値が低すぎると、多くのリソースを消費する可能性のあるスケジュール外のマージが頻繁に実行されます。 +ClickHouse がデータの期限切れを検出すると、スケジュール外のマージを実行します。このようなマージの頻度を制御するには、`merge_with_ttl_timeout` を設定できます。値を低くしすぎると、多数のスケジュール外マージが実行され、多くのリソースを消費する可能性があります。 -マージの間に`SELECT`クエリを実行すると、期限切れのデータが取得される可能性があります。これを回避するには、`SELECT`の前に[OPTIMIZE](/sql-reference/statements/optimize.md)クエリを使用してください。 +マージとマージの間に `SELECT` クエリを実行すると、期限切れデータが返される場合があります。これを避けるには、`SELECT` の前に [OPTIMIZE](/sql-reference/statements/optimize.md) クエリを実行してください。 **関連項目** -- [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts)設定 - - -## ディスクタイプ {#disk-types} +* [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) 設定 -ローカルブロックデバイスに加えて、ClickHouseは以下のストレージタイプをサポートしています: +## ディスクの種類 {#disk-types} -- [`s3` S3およびMinIO向け](#table_engine-mergetree-s3) -- [`gcs` GCS向け](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -- [`blob_storage_disk` Azure Blob Storage向け](/operations/storing-data#azure-blob-storage) -- [`hdfs` HDFS向け](/engines/table-engines/integrations/hdfs) -- [`web` Webからの読み取り専用アクセス向け](/operations/storing-data#web-storage) -- [`cache` ローカルキャッシュ向け](/operations/storing-data#using-local-cache) -- [`s3_plain` S3へのバックアップ向け](/operations/backup#backuprestore-using-an-s3-disk) -- [`s3_plain_rewritable` S3上の不変・非レプリケートテーブル向け](/operations/storing-data.md#s3-plain-rewritable-storage) +ローカルブロックデバイスに加えて、ClickHouse は次のストレージタイプをサポートします: +* [`s3` — S3 および MinIO 用](#table_engine-mergetree-s3) +* [`gcs` — GCS 用](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) +* [`blob_storage_disk` — Azure Blob Storage 用](/operations/storing-data#azure-blob-storage) +* [`hdfs` — HDFS 用](/engines/table-engines/integrations/hdfs) +* [`web` — Web からの読み取り専用](/operations/storing-data#web-storage) +* [`cache` — ローカルキャッシュ用](/operations/storing-data#using-local-cache) +* [`s3_plain` — S3 へのバックアップ用](/operations/backup#backuprestore-using-an-s3-disk) +* [`s3_plain_rewritable` — S3 上の変更不可な非レプリケートテーブル用](/operations/storing-data.md#s3-plain-rewritable-storage) -## データ保存のための複数のブロックデバイスを使用する {#table_engine-mergetree-multiple-volumes} +## データストレージで複数のブロックデバイスを利用する {#table_engine-mergetree-multiple-volumes} ### はじめに {#introduction} -`MergeTree` ファミリーのテーブルエンジンは、複数のブロックデバイスにデータを格納できます。例えば、あるテーブルのデータが「ホット」と「コールド」に暗黙のうちに分けられる場合に有用です。最近のデータは頻繁にアクセスされますが、必要なストレージ容量は少量です。一方、ファットテールの古い履歴データはアクセス頻度が低いです。複数のディスクが利用可能な場合、「ホット」データは高速ディスク(例: NVMe SSD やメモリ内)に、「コールド」データは比較的低速なディスク(例: HDD)に配置できます。 +`MergeTree` ファミリーのテーブルエンジンは、複数のブロックデバイス上にデータを保存できます。たとえば、特定のテーブルのデータが事実上「ホット」と「コールド」に分割されている場合に有用です。最新のデータは頻繁に参照されますが、必要な容量は小さくて済みます。対照的に、裾の重い履歴データはまれにしか参照されません。複数のディスクが利用可能な場合、「ホット」データは高速なディスク(たとえば NVMe SSD やメモリ上)に配置し、「コールド」データは比較的低速なディスク(たとえば HDD)上に配置できます。 -データパーツは、`MergeTree` エンジンのテーブルにおける最小の移動単位です。一つのパーツに属するデータは一つのディスクに格納されます。データパーツは、バックグラウンドで(ユーザー設定に基づいて)ディスク間で移動でき、また [ALTER](/sql-reference/statements/alter/partition) クエリによっても移動可能です。 +データパーツは、`MergeTree` エンジンのテーブルにおける最小の移動可能な単位です。1 つのパーツに属するデータは 1 台のディスク上に保存されます。データパーツは、バックグラウンドでユーザー設定に従ってディスク間を移動できるほか、[ALTER](/sql-reference/statements/alter/partition) クエリを使用して移動することもできます。 ### 用語 {#terms} -- Disk — ファイルシステムにマウントされたブロックデバイス。 -- Default disk — [path](/operations/server-configuration-parameters/settings.md/#path) サーバー設定で指定されたパスにデータを格納するディスク。 -- Volume — 同一のディスクからなる順序付き集合([JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) に類似)。 -- Storage policy — ボリュームの集合と、それら間でデータを移動するためのルール。 +* ディスク — ファイルシステムにマウントされたブロックデバイス。 +* デフォルトディスク — [path](/operations/server-configuration-parameters/settings.md/#path) サーバー設定で指定されたパス上にデータを保存するディスク。 +* ボリューム — 同一条件のディスクを順序付きで並べた集合([JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) に類似)。 +* ストレージポリシー — ボリュームの集合と、それらの間でデータを移動するためのルール。 -記述されたエンティティの名前は、システムテーブル [system.storage_policies](/operations/system-tables/storage_policies) および [system.disks](/operations/system-tables/disks) で確認できます。構成済みのストレージポリシーのうちの一つをテーブルに適用するには、`MergeTree` エンジンファミリーテーブルの `storage_policy` 設定を使用します。 +ここで説明した各エンティティの名称は、システムテーブル [system.storage_policies](/operations/system-tables/storage_policies) および [system.disks](/operations/system-tables/disks) で確認できます。テーブルに対して設定済みのストレージポリシーのいずれかを適用するには、`MergeTree` エンジンファミリーのテーブルで `storage_policy` 設定を使用します。 -### 設定 {#table_engine-mergetree-multiple-volumes_configure} +### 設定 {#table_engine-mergetree-multiple-volumes_configure} -ディスク、ボリューム、およびストレージポリシーは、`config.d` ディレクトリのファイル内の `` タグで宣言します。 +ディスク、ボリューム、およびストレージポリシーは、`config.d` ディレクトリ内のファイルにある `` タグ内で宣言する必要があります。 :::tip -ディスクはクエリの `SETTINGS` セクションで宣言することも可能です。これは、例えば URL でホストされたディスクを一時的にアタッチするアドホック分析に便利です。 -詳細は [dynamic storage](/operations/storing-data#dynamic-configuration) を参照してください。 +ディスクはクエリの `SETTINGS` セクション内で宣言することもできます。これは、例えば URL で公開されているディスクを一時的にアタッチしてアドホックな分析を行う場合に便利です。 +詳細については、[dynamic storage](/operations/storing-data#dynamic-configuration) を参照してください。 ::: -設定構造: +設定の構造: ```xml - + /mnt/fast_ssd/clickhouse/ @@ -850,13 +830,13 @@ ClickHouseがデータの期限切れを検出すると、スケジュール外 タグ: -- `` — ディスク名。すべてのディスクで名前は一意でなければなりません。 -- `path` — サーバーがデータを格納するパス(`data` および `shadow` フォルダ)、末尾に '/' を付ける必要があります。 -- `keep_free_space_bytes` — 予約する空きディスク容量の量。 +* `` — ディスク名。すべてのディスクで名前が重複しないようにする必要があります。 +* `path` — サーバーがデータ(`data` および `shadow` フォルダ)を保存するパス。末尾は '/' で終わる必要があります。 +* `keep_free_space_bytes` — 予約しておく空きディスク容量のバイト数。 -ディスクの定義順序は重要ではありません。 +ディスク定義の順序は重要ではありません。 -ストレージポリシーの設定マークアップ: +ストレージポリシー構成のマークアップ: ```xml @@ -870,17 +850,17 @@ ClickHouseがデータの期限切れを検出すると、スケジュール外 round_robin - + - + 0.2 - + - + ... @@ -888,23 +868,22 @@ ClickHouseがデータの期限切れを検出すると、スケジュール外 タグ: - -* `policy_name_N` — ポリシー名。ポリシー名は一意でなければなりません。 -* `volume_name_N` — ボリューム名。ボリューム名は一意でなければなりません。 +* `policy_name_N` — ポリシー名。ポリシー名は一意である必要があります。 +* `volume_name_N` — ボリューム名。ボリューム名は一意である必要があります。 * `disk` — ボリューム内のディスク。 -* `max_data_part_size_bytes` — そのボリューム内のいずれのディスク上にも保存できるパーツの最大サイズ。マージ後のパーツサイズが `max_data_part_size_bytes` より大きくなると見積もられた場合、そのパーツは次のボリュームに書き込まれます。基本的にこの機能により、新しい/小さいパーツをホット(SSD)ボリュームに保持し、大きなサイズに達したときにコールド(HDD)ボリュームへ移動できます。ポリシーにボリュームが 1 つしかない場合は、この設定を使用しないでください。 -* `move_factor` — 利用可能な空き容量がこの係数より小さくなったとき、自動的にデータを次のボリューム(存在する場合)へ移動し始めます(デフォルトは 0.1)。ClickHouse は既存のパーツをサイズ順に大きいものから小さいものへ(降順で)ソートし、`move_factor` 条件を満たすのに十分な合計サイズとなるようにパーツを選択します。全パーツの合計サイズが不十分な場合は、すべてのパーツが移動されます。 -* `perform_ttl_move_on_insert` — データパーツ INSERT 時の TTL による移動を無効にします。デフォルト(有効な場合)では、TTL move ルールですでに期限切れとなっているデータパーツを挿入すると、直ちに移動ルールで指定されたボリューム/ディスクへ移動されます。宛先のボリューム/ディスクが低速(例:S3 のようなリモートストレージ)の場合、これは INSERT を大幅に遅くする可能性があります。無効にした場合、すでに期限切れのデータパーツはいったんデフォルトボリュームに書き込まれ、その直後に TTL ボリュームへ移動されます。 -* `load_balancing` - ディスクバランシングのポリシー。`round_robin` または `least_used`。 -* `least_used_ttl_ms` - すべてのディスクの利用可能な空き容量を更新するためのタイムアウト(ミリ秒単位)を設定します(`0` - 常に更新、`-1` - 決して更新しない、デフォルトは `60000`)。なお、そのディスクが ClickHouse のみで使用され、オンラインでのファイルシステムのサイズ変更(拡張/縮小)の対象とならない場合は `-1` を使用できます。それ以外のケースでは推奨されません。時間の経過とともに空き容量の把握が不正確になり、容量分布が偏る原因となるためです。 -* `prefer_not_to_merge` — この設定は使用しないでください。このボリューム上のデータパーツのマージを無効にします(有害であり、パフォーマンス低下を招きます)。この設定が有効な場合(有効にしないでください)、このボリューム上でのマージは許可されません(これは望ましくありません)。これにより(ただし、その必要はありません)、ClickHouse が低速ディスクをどのように扱うかを制御できますが(何かを制御したくなる時点で誤りです)、ClickHouse の方が適切に扱うため、この設定は使用しないでください。 -* `volume_priority` — ボリュームがどの優先度(順序)で埋められるかを定義します。値が小さいほど優先度が高くなります。パラメータ値は自然数でなければならず、1 から N(与えられた中で最も低い優先度)までの範囲を欠番なしで網羅する必要があります。 - * *すべての* ボリュームがタグ付けされている場合、それらは指定された順序で優先されます。 - * 一部のボリュームのみがタグ付けされている場合、タグのないボリュームは最も優先度が低く、設定ファイル内で定義された順序で優先されます。 - * *一つも* ボリュームがタグ付けされていない場合、その優先度は設定ファイル内で宣言された順序に対応して設定されます。 - * 2 つのボリュームが同じ優先度値を持つことはできません。 - -Configuration examples: +* `max_data_part_size_bytes` — いずれのボリューム上のディスクにも保存可能なパーツの最大サイズ。マージされたパーツのサイズが `max_data_part_size_bytes` より大きくなると見積もられた場合、そのパーツは次のボリュームに書き込まれます。この機能により、新規/小さいパーツをホット (SSD) ボリュームに保持し、サイズが大きくなったときにコールド (HDD) ボリュームへ移動できます。ポリシーにボリュームが 1 つしかない場合、この設定は使用しないでください。 +* `move_factor` — 空き容量がこの係数より小さくなったとき、自動的にデータを次のボリュームに移動し始めます (既定値は 0.1)。ClickHouse は既存のパーツをサイズの大きい順 (降順) にソートし、`move_factor` 条件を満たすのに十分な合計サイズとなるようにパーツを選択します。すべてのパーツの合計サイズが不十分な場合は、すべてのパーツが移動されます。 +* `perform_ttl_move_on_insert` — データパーツの INSERT 時の TTL move を無効化します。既定では (有効な場合)、挿入するデータパーツが TTL move ルールによりすでに期限切れとなっている場合、そのパーツは直ちに move ルールで指定されたボリューム/ディスクに配置されます。宛先ボリューム/ディスクが遅い場合 (例: S3)、これは INSERT を大幅に遅くする可能性があります。無効にした場合、すでに期限切れのデータパーツはいったんデフォルトボリュームに書き込まれ、その直後に TTL ボリュームへ移動されます。 +* `load_balancing` — ディスクのバランシングポリシー。`round_robin` または `least_used`。 +* `least_used_ttl_ms` — すべてのディスク上の空き容量情報を更新するためのタイムアウト (ミリ秒単位) を設定します (`0` - 常に更新、`-1` - 更新しない、既定値は `60000`)。注意: ディスクが ClickHouse 専用であり、オンラインのファイルシステムのリサイズ/縮小の対象にならない場合は `-1` を使用できますが、それ以外のケースでは推奨されません。最終的に空き容量の不適切な分散を招くためです。 +* `prefer_not_to_merge` — この設定は使用しないでください。このボリューム上のデータパーツのマージを無効化します (これは有害であり、パフォーマンス低下につながります)。この設定が有効な場合 (有効にしないでください)、このボリューム上でのマージは許可されません (これは望ましくありません)。これは (必要ありませんが) ClickHouse が遅いディスクをどのように扱うかを制御することを可能にします (しかし、何かを制御しようとしている時点で誤りであり、ClickHouse の方が賢いので、この設定は使用しないでください)。 +* `volume_priority` — ボリュームが埋められる優先度 (順序) を定義します。値が小さいほど優先度が高くなります。このパラメータの値は自然数とし、1 から N (最も低い優先度の値) までの範囲を欠番なくすべて網羅する必要があります。 + * *すべての* ボリュームにタグが付いている場合、指定された順序で優先されます。 + * 一部のボリュームのみにタグが付いている場合、タグのないボリュームは最も低い優先度となり、設定ファイル内で定義された順に優先されます。 + * ボリュームに *まったく* タグが付いていない場合、設定ファイル内で宣言された順序に対応して優先度が設定されます。 + * 2 つのボリュームが同じ優先度の値を持つことはできません。 + +構成例: ```xml @@ -947,16 +926,15 @@ Configuration examples: ``` +この例では、`hdd_in_order` ポリシーは [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling) 方式を実装しています。そのため、このポリシーは 1 つのボリューム(`single`)のみを定義し、そのすべてのディスク上にデータパーツをラウンドロビンで保存します。RAID を構成していないものの、同種のディスクが複数台システムにマウントされている場合、このようなポリシーは非常に有用です。各ディスクドライブ単体は信頼性が高くないことに注意し、レプリケーション係数を 3 以上にして補償することを検討してください。 -この例では、`hdd_in_order` ポリシーは [ラウンドロビン](https://en.wikipedia.org/wiki/Round-robin_scheduling) 方式を実装しています。このポリシーは1つのボリューム(`single`)のみを定義し、データパーツはすべてのディスクに循環順序で格納されます。このようなポリシーは、複数の類似したディスクがシステムにマウントされているが RAID が構成されていない場合に非常に有用です。個々のディスクドライブは信頼性が高くないため、レプリケーション係数を3以上にして補う必要があることに留意してください。 - -システムに異なる種類のディスクが利用可能な場合は、代わりに `moving_from_ssd_to_hdd` ポリシーを使用できます。ボリューム `hot` は SSD ディスク(`fast_ssd`)で構成され、このボリュームに格納できるパーツの最大サイズは 1GB です。1GB より大きいサイズのパーツはすべて、HDD ディスク `disk1` を含む `cold` ボリュームに直接格納されます。 -また、ディスク `fast_ssd` が 80% 以上満たされると、バックグラウンドプロセスによってデータが `disk1` に転送されます。 +システムに異なる種類のディスクが存在する場合は、代わりに `moving_from_ssd_to_hdd` ポリシーを使用できます。`hot` ボリュームは SSD ディスク(`fast_ssd`)で構成されており、このボリュームに保存できる 1 パートの最大サイズは 1GB です。サイズが 1GB を超えるすべてのパーツは、HDD ディスク `disk1` を含む `cold` ボリュームに直接保存されます。 +また、ディスク `fast_ssd` の使用率が 80% を超えると、バックグラウンドプロセスによってデータが `disk1` に転送されます。 -ストレージポリシー内のボリューム列挙の順序は、リストされたボリュームの少なくとも1つに明示的な `volume_priority` パラメータがない場合に重要です。 -ボリュームが満杯になると、データは次のボリュームに移動されます。ディスク列挙の順序も重要です。なぜなら、データは順番にディスクに格納されるためです。 +ストレージポリシー内でのボリュームの列挙順序は、列挙されたボリュームのうち少なくとも 1 つに明示的な `volume_priority` パラメータが設定されていない場合に重要です。 +あるボリュームが満杯になると、データは次のボリュームへ移動されます。ディスクの列挙順も同様に重要であり、データはそれらに順番に保存されます。 -テーブルを作成する際、構成されたストレージポリシーの1つを適用できます: +テーブルを作成する際、そのテーブルに対して設定済みストレージポリシーのいずれかを適用できます。 ```sql CREATE TABLE table_with_non_default_policy ( @@ -970,47 +948,44 @@ PARTITION BY toYYYYMM(EventDate) SETTINGS storage_policy = 'moving_from_ssd_to_hdd' ``` -`default` ストレージポリシーは、`` で指定された1つのディスクのみで構成される1つのボリュームのみを使用することを意味します。 -テーブル作成後に [ALTER TABLE ... MODIFY SETTING] クエリでストレージポリシーを変更できますが、新しいポリシーには同じ名前のすべての古いディスクとボリュームを含める必要があります。 +`default` ストレージポリシーは、`` で指定された 1 つのディスクのみから構成される 1 つのボリュームだけを使用することを意味します。 +テーブル作成後でも [ALTER TABLE ... MODIFY SETTING] クエリを使用してストレージポリシーを変更できますが、新しいポリシーには、同じ名前を持つすべての既存ディスクおよびボリュームを含める必要があります。 -データパーツのバックグラウンド移動を実行するスレッド数は、[background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 設定で変更できます。 +データパーツのバックグラウンド移動を実行するスレッド数は、[background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 設定で変更できます。 ### 詳細 {#details} -`MergeTree` テーブルの場合、データは以下のさまざまな方法でディスクに書き込まれます: - -- 挿入の結果として(`INSERT` クエリ)。 -- バックグラウンドマージおよび [ミューテーション](/sql-reference/statements/alter#mutations) 中。 -- 別のレプリカからダウンロードする際。 -- パーティションフリーズの結果として [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition)。 - -ミューテーションとパーティションフリーズを除くこれらすべてのケースにおいて、パーツは指定されたストレージポリシーに従ってボリュームとディスクに格納されます: - -1. パーツを格納するのに十分なディスク容量(`unreserved_space > current_part_size`)があり、指定されたサイズのパーツの格納を許可する(`max_data_part_size_bytes > current_part_size`)最初のボリューム(定義順)が選択されます。 -2. このボリューム内で、前のデータチャンクの格納に使用されたディスクの次のディスクであり、パーツサイズより大きい空き容量(`unreserved_space - keep_free_space_bytes > current_part_size`)を持つディスクが選択されます。 +`MergeTree` テーブルの場合、データは次のようなさまざまな方法でディスクに書き込まれます。 -内部的には、ミューテーションとパーティションフリーズは [ハードリンク](https://en.wikipedia.org/wiki/Hard_link) を使用します。異なるディスク間のハードリンクはサポートされていないため、このような場合、結果のパーツは初期のパーツと同じディスクに格納されます。 +* 挿入(`INSERT` クエリ)の結果として。 +* バックグラウンドでのマージおよび[ミューテーション](/sql-reference/statements/alter#mutations)の実行中。 +* 別のレプリカからのダウンロード時。 +* パーティションのフリーズ [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) の結果として。 -バックグラウンドでは、構成ファイルでボリュームが宣言された順序に従って、空き容量(`move_factor` パラメータ)に基づいてパーツがボリューム間で移動されます。 -データは最後のボリュームから最初のボリュームに転送されることはありません。バックグラウンド移動を監視するには、システムテーブル [system.part_log](/operations/system-tables/part_log)(フィールド `type = MOVE_PART`)および [system.parts](/operations/system-tables/parts.md)(フィールド `path` および `disk`)を使用できます。また、詳細情報はサーバーログで確認できます。 +ミューテーションとパーティションのフリーズを除くすべての場合において、パーツは指定されたストレージポリシーに従ってボリュームおよびディスク上に保存されます。 -ユーザーは、クエリ [ALTER TABLE ... MOVE PART\|PARTITION ... TO VOLUME\|DISK ...](/sql-reference/statements/alter/partition) を使用して、パーツまたはパーティションをあるボリュームから別のボリュームに強制的に移動できます。バックグラウンド操作のすべての制限が考慮されます。このクエリは独自に移動を開始し、バックグラウンド操作の完了を待ちません。十分な空き容量がない場合、または必要な条件のいずれかが満たされていない場合、ユーザーはエラーメッセージを受け取ります。 +1. パーツを保存するのに十分な空きディスク容量があり(`unreserved_space > current_part_size`)、かつ指定サイズのパーツの保存が許可されている(`max_data_part_size_bytes > current_part_size`)最初のボリューム(定義順)が選択されます。 +2. このボリューム内では、直前のデータチャンクを保存していたディスクの次のディスクであって、かつその空き容量がパーツサイズを上回るもの(`unreserved_space - keep_free_space_bytes > current_part_size`)が選択されます。 -データの移動はデータレプリケーションに干渉しません。したがって、異なるレプリカ上の同じテーブルに対して異なるストレージポリシーを指定できます。 +内部的には、ミューテーションとパーティションのフリーズは[ハードリンク](https://en.wikipedia.org/wiki/Hard_link)を利用します。異なるディスク間のハードリンクはサポートされないため、このような場合には結果として生成されるパーツは元のパーツと同じディスク上に保存されます。 +バックグラウンドでは、設定ファイル内で宣言されたボリュームの順序に従い、空き容量(`move_factor` パラメータ)に基づいてパーツがボリューム間で移動されます。 +最後のボリュームから他のボリュームへの移動および他のボリュームから最初のボリュームへの移動は行われません。バックグラウンドでの移動は、システムテーブル [system.part_log](/operations/system-tables/part_log)(フィールド `type = MOVE_PART`)および [system.parts](/operations/system-tables/parts.md)(フィールド `path` と `disk`)を使用して監視できます。より詳細な情報はサーバーログで確認できます。 -バックグラウンドで実行されるマージおよびミューテーションが完了した後、古いパーツは、一定時間(`old_parts_lifetime`)が経過してから削除されます。 -この間、それらは他のボリュームやディスクに移動されません。したがって、パーツが最終的に削除されるまでは、ディスク使用量の算出に引き続き含まれます。 +ユーザーは、クエリ [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition) を使用して、パーツまたはパーティションをあるボリュームから別のボリュームへ強制的に移動できます。バックグラウンド操作に対するすべての制約が考慮されます。このクエリは独自に移動処理を開始し、バックグラウンド操作の完了を待ちません。必要な空き容量が不足している場合や、必要条件のいずれかが満たされていない場合、ユーザーにはエラーメッセージが返されます。 -ユーザーは、[min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) 設定を使用することで、新しい大きなパーツを [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) ボリューム内の複数ディスクにバランス良く割り当てることができます。 +データの移動はデータレプリケーションの動作を妨げません。そのため、同じテーブルに対しても、レプリカごとに異なるストレージポリシーを指定できます。 +バックグラウンドでのマージおよびミューテーションが完了した後、古いパーツは一定時間(`old_parts_lifetime`)経過してから削除されます。 +この期間中、それらのパーツは他のボリュームやディスクには移動されません。したがってパーツが最終的に削除されるまでは、使用中ディスク容量の計算に引き続き含まれます。 +ユーザーは、[JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) ボリュームの複数ディスクに新しい大きなパーツをバランス良く割り当てるために、設定 [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) を使用できます。 -## 外部ストレージを使用したデータ保存 {#table_engine-mergetree-s3} +## データの保存に外部ストレージを使用する {#table_engine-mergetree-s3} -[MergeTree](/engines/table-engines/mergetree-family/mergetree.md)ファミリーのテーブルエンジンは、`s3`、`azure_blob_storage`、`hdfs`のディスクタイプを使用して、それぞれ`S3`、`AzureBlobStorage`、`HDFS`にデータを保存できます。詳細については、[外部ストレージオプションの設定](/operations/storing-data.md/#configuring-external-storage)を参照してください。 +[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) ファミリーのテーブルエンジンは、それぞれ `s3`、`azure_blob_storage`、`hdfs` タイプのディスクを使用して、データを `S3`、`AzureBlobStorage`、`HDFS` に保存できます。詳細は、[外部ストレージオプションの設定](/operations/storing-data.md/#configuring-external-storage)を参照してください。 -`s3`タイプのディスクを使用した外部ストレージとしての[S3](https://aws.amazon.com/s3/)の例を以下に示します。 +ディスクタイプ `s3` を使用して [S3](https://aws.amazon.com/s3/) を外部ストレージとして利用する例を以下に示します。 設定マークアップ: @@ -1056,33 +1031,32 @@ SETTINGS storage_policy = 'moving_from_ssd_to_hdd' [外部ストレージオプションの設定](/operations/storing-data.md/#configuring-external-storage)も参照してください。 :::note キャッシュ設定 -ClickHouseバージョン22.3から22.7では異なるキャッシュ設定を使用します。これらのバージョンを使用している場合は、[ローカルキャッシュの使用](/operations/storing-data.md/#using-local-cache)を参照してください。 +ClickHouse バージョン 22.3 から 22.7 までは異なるキャッシュ設定が使用されています。これらのバージョンのいずれかを使用している場合は、[ローカルキャッシュの使用](/operations/storing-data.md/#using-local-cache)を参照してください。 ::: - ## 仮想カラム {#virtual-columns} -- `_part` — パートの名前。 -- `_part_index` — クエリ結果内のパートの連番インデックス。 -- `_part_starting_offset` — クエリ結果内のパートの累積開始行。 -- `_part_offset` — パート内の行番号。 -- `_part_granule_offset` — パート内のグラニュール番号。 -- `_partition_id` — パーティションの名前。 -- `_part_uuid` — 一意のパート識別子(MergeTree設定`assign_part_uuids`が有効な場合)。 -- `_part_data_version` — パートのデータバージョン(最小ブロック番号またはミューテーションバージョンのいずれか)。 -- `_partition_value` — `partition by`式の値(タプル)。 -- `_sample_factor` — サンプリング係数(クエリより)。 -- `_block_number` — 挿入時に割り当てられた行の元のブロック番号。設定`enable_block_number_column`が有効な場合、マージ時に保持される。 -- `_block_offset` — 挿入時に割り当てられたブロック内の元の行番号。設定`enable_block_offset_column`が有効な場合、マージ時に保持される。 -- `_disk_name` — ストレージに使用されるディスク名。 - +* `_part` — パーツ名。 +* `_part_index` — クエリ結果内でのパーツの連番インデックス番号。 +* `_part_starting_offset` — クエリ結果内でのパーツの累積開始行番号。 +* `_part_offset` — パーツ内での行番号。 +* `_part_granule_offset` — パーツ内でのグラニュール番号。 +* `_partition_id` — パーティション名。 +* `_part_uuid` — 一意のパーツ識別子(MergeTree 設定 `assign_part_uuids` が有効な場合)。 +* `_part_data_version` — パーツのデータバージョン(最小ブロック番号またはミューテーションバージョンのいずれか)。 +* `_partition_value` — `partition by` 式の値(タプル)。 +* `_sample_factor` — クエリで指定されたサンプル係数。 +* `_block_number` — 行に挿入時に割り当てられた元のブロック番号で、`enable_block_number_column` 設定が有効な場合はマージ時も保持される。 +* `_block_offset` — ブロック内の行に挿入時に割り当てられた元の行番号で、`enable_block_offset_column` 設定が有効な場合はマージ時も保持される。 +* `_disk_name` — ストレージで使用されるディスク名。 ## カラム統計 {#column-statistics} + -統計の宣言は、`set allow_experimental_statistics = 1`を有効にした場合、`*MergeTree*`ファミリーのテーブルに対する`CREATE`クエリのカラムセクションに記述します。 +`set allow_experimental_statistics = 1` を有効にすると、`*MergeTree*` ファミリーのテーブルに対する `CREATE` クエリの `COLUMNS` セクション内で統計を宣言します。 ```sql CREATE TABLE tab @@ -1094,69 +1068,68 @@ ENGINE = MergeTree ORDER BY a ``` -`ALTER`文を使用して統計を操作することもできます。 +`ALTER` ステートメントを使用して統計情報を変更することもできます。 ```sql ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; ALTER TABLE tab DROP STATISTICS a; ``` -これらの軽量な統計は、カラム内の値の分布に関する情報を集約します。統計は各パートに保存され、挿入が行われるたびに更新されます。 -統計は、`set allow_statistics_optimize = 1`を有効にした場合にのみ、prewhere最適化に使用できます。 +これらの軽量な統計情報は、列内の値の分布に関する情報を集約します。統計情報は各パートに保存され、挿入が行われるたびに更新されます。 +`set allow_statistics_optimize = 1` を有効にした場合にのみ、PREWHERE 最適化に利用できます。 -### 利用可能なカラム統計のタイプ {#available-types-of-column-statistics} +### 利用可能な列統計の種類 {#available-types-of-column-statistics} -- `MinMax` +* `MinMax` - 数値カラムに対する範囲フィルタの選択性を推定するための、カラムの最小値と最大値です。 + 数値型列に対する範囲フィルターの選択性を推定できるようにする、列の最小値と最大値。 構文: `minmax` -- `TDigest` +* `TDigest` - 数値カラムに対して近似パーセンタイル(例: 90パーセンタイル)を計算できる[TDigest](https://github.com/tdunning/t-digest)スケッチです。 + 数値型列に対して近似パーセンタイル(例: 第90パーセンタイル)を計算できる [TDigest](https://github.com/tdunning/t-digest) スケッチ。 構文: `tdigest` -- `Uniq` +* `Uniq` - カラムに含まれる個別値の数を推定する[HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog)スケッチです。 + 列に含まれる異なる値の個数を推定する [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) スケッチ。 構文: `uniq` -- `CountMin` +* `CountMin` - カラム内の各値の出現頻度の近似カウントを提供する[CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch)スケッチです。 + 列内の各値の出現頻度を近似的にカウントする [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch) スケッチ。 構文: `countmin` -### サポートされるデータ型 {#supported-data-types} +### サポートされているデータ型 {#supported-data-types} -| | (U)Int*, Float*, Decimal(_), Date_, Boolean, Enum\* | String or FixedString | -| -------- | --------------------------------------------------- | --------------------- | -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | +| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String または FixedString | +|-----------|----------------------------------------------------|---------------------------| +| CountMin | ✔ | ✔ | +| MinMax | ✔ | ✗ | +| TDigest | ✔ | ✗ | +| Uniq | ✔ | ✔ | ### サポートされる操作 {#supported-operations} -| | 等価フィルタ (==) | 範囲フィルタ (`>, >=, <, <=`) | -| -------- | --------------------- | ------------------------------ | -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | +| | 等値フィルター (==) | 範囲フィルター (`>, >=, <, <=`) | +|-----------|---------------------|------------------------------| +| CountMin | ✔ | ✗ | +| MinMax | ✗ | ✔ | +| TDigest | ✗ | ✔ | +| Uniq | ✔ | ✗ | +## 列レベルの設定 {#column-level-settings} -## カラムレベルの設定 {#column-level-settings} +一部の MergeTree の設定は列レベルで上書きできます。 -特定のMergeTree設定はカラムレベルで上書きできます: +* `max_compress_block_size` — テーブルに書き込む際に、圧縮前のデータブロックの最大サイズ。 +* `min_compress_block_size` — 次のマークを書き込む際に圧縮を行うために必要となる、圧縮前のデータブロックの最小サイズ。 -- `max_compress_block_size` — テーブルへの書き込み時に圧縮する前の非圧縮データブロックの最大サイズ。 -- `min_compress_block_size` — 次のマークを書き込む際に圧縮に必要な非圧縮データブロックの最小サイズ。 - -例: +例: ```sql CREATE TABLE tab @@ -1168,21 +1141,21 @@ ENGINE = MergeTree ORDER BY id ``` -カラムレベルの設定は[ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md)を使用して変更または削除できます。例: +カラムレベルの設定は、たとえば [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md) を使用して変更または削除できます。 -- カラム宣言から`SETTINGS`を削除する: +* カラム定義の `SETTINGS` を削除する: ```sql ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; ``` -- 設定を変更する: +* 設定を変更します: ```sql ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; ``` -- 1つ以上の設定をリセットする。これによりテーブルのCREATEクエリのカラム式における設定宣言も削除されます。 +* 1 つ以上の設定をリセットします。また、テーブルの CREATE クエリの列式から設定宣言も削除します。 ```sql ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md index f99bba71216..b99d8c9080b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# ReplacingMergeTree テーブルエンジン +# ReplacingMergeTree テーブルエンジン {#replacingmergetree-table-engine} このエンジンは、[MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) とは異なり、同じ[ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md)値(テーブル定義の `ORDER BY` セクションで指定されるもので、`PRIMARY KEY` ではありません)を持つ重複エントリを削除します。 @@ -24,7 +24,7 @@ doc_type: 'reference' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -47,9 +47,9 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ::: -## ReplacingMergeTree のパラメーター +## ReplacingMergeTree のパラメーター {#replacingmergetree-parameters} -### `ver` +### `ver` {#ver} `ver` — バージョン番号を保持するカラム。型は `UInt*`、`Date`、`DateTime` または `DateTime64`。省略可能なパラメーターです。 @@ -101,7 +101,7 @@ SELECT * FROM mySecondReplacingMT FINAL; └─────┴─────────┴─────────────────────┘ ``` -### `is_deleted` +### `is_deleted` {#is_deleted} `is_deleted` — マージ処理の際に、この行のデータが「状態」を表すのか、あるいは削除対象なのかを判定するために使用されるカラム名です。`1` は「削除された」行、`0` は「状態」の行を表します。 @@ -190,7 +190,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## クエリ時の重複排除と `FINAL` +## クエリ時の重複排除と `FINAL` {#query-time-de-duplication--final} マージ処理の際に、ReplacingMergeTree は `ORDER BY` 列(テーブル作成時に使用した列)の値を一意の識別子として用いて重複行を識別し、最も新しいバージョンのみを保持します。ただし、これはあくまで最終的な整合性しか提供せず、行が必ず重複排除されることを保証するものではないため、これに依存すべきではありません。その結果、更新行や削除行がクエリで考慮されることにより、クエリが誤った結果を返す可能性があります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md deleted file mode 100644 index 6688eef453e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md +++ /dev/null @@ -1,357 +0,0 @@ ---- -description: 'ClickHouse における Replicated* ファミリーのテーブルエンジンによるデータレプリケーションの概要' -sidebar_label: 'Replicated*' -sidebar_position: 20 -slug: /engines/table-engines/mergetree-family/replication -title: 'Replicated* テーブルエンジン' -doc_type: 'reference' ---- - - - -# Replicated* テーブルエンジン - -:::note -ClickHouse Cloud ではレプリケーションは自動的に管理されます。引数を追加せずにテーブルを作成してください。たとえば、以下のテキスト中の次の部分を置き換えてください: - -```sql -ENGINE = ReplicatedMergeTree( - '/clickhouse/tables/{shard}/table_name', - '{replica}' -) -``` - -次の内容で: - -```sql -ENGINE = ReplicatedMergeTree -``` - -::: - -レプリケーションは MergeTree ファミリーのテーブルに対してのみサポートされます: - -* ReplicatedMergeTree -* ReplicatedSummingMergeTree -* ReplicatedReplacingMergeTree -* ReplicatedAggregatingMergeTree -* ReplicatedCollapsingMergeTree -* ReplicatedVersionedCollapsingMergeTree -* ReplicatedGraphiteMergeTree - -レプリケーションはサーバー全体ではなく、個々のテーブル単位で行われます。1 つのサーバー上に、レプリケートされたテーブルとレプリケートされていないテーブルを同時に保持できます。 - -レプリケーションはシャーディングに依存しません。各シャードがそれぞれ独立してレプリケーションを行います。 - -`INSERT` および `ALTER` クエリで圧縮されたデータはレプリケートされます(詳細については、[ALTER](/sql-reference/statements/alter) のドキュメントを参照してください)。 - -`CREATE`、`DROP`、`ATTACH`、`DETACH`、`RENAME` クエリは単一のサーバー上で実行され、レプリケートされません: - -* `CREATE TABLE` クエリは、そのクエリが実行されたサーバー上に新しいレプリケート可能なテーブルを作成します。このテーブルがすでに他のサーバー上に存在する場合は、新しいレプリカを追加します。 -* `DROP TABLE` クエリは、そのクエリが実行されたサーバー上にあるレプリカを削除します。 -* `RENAME` クエリは、レプリカの 1 つにあるテーブルの名前を変更します。言い換えると、レプリケートされたテーブルは、レプリカごとに異なる名前を持つことができます。 - -ClickHouse はレプリカのメタ情報を保存するために [ClickHouse Keeper](/guides/sre/keeper/index.md) を使用します。ZooKeeper のバージョン 3.4.5 以降を使用することも可能ですが、ClickHouse Keeper を推奨します。 - -レプリケーションを使用するには、[zookeeper](/operations/server-configuration-parameters/settings#zookeeper) サーバー設定セクションでパラメータを設定します。 - -:::note -セキュリティ設定をおろそかにしないでください。ClickHouse は ZooKeeper のセキュリティサブシステムの `digest` [ACL スキーム](https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) をサポートしています。 -::: - -ClickHouse Keeper クラスターのアドレスを設定する例: - -```xml - - - example1 - 2181 - - - example2 - 2181 - - - example3 - 2181 - - -``` - -ClickHouse は、補助的な ZooKeeper クラスターにレプリカのメタ情報を保存することもサポートしています。これを行うには、エンジンの引数として ZooKeeper クラスター名とパスを指定します。 -言い換えると、テーブルごとに異なる ZooKeeper クラスターにメタデータを保存することをサポートしています。 - -補助的な ZooKeeper クラスターのアドレス設定例: - -```xml - - - - example_2_1 - 2181 - - - example_2_2 - 2181 - - - example_2_3 - 2181 - - - - - example_3_1 - 2181 - - - -``` - -テーブルメタデータをデフォルトの ZooKeeper クラスターではなく補助的な ZooKeeper クラスターに保存するには、次のように -ReplicatedMergeTree エンジンを指定してテーブルを作成する SQL を使用します。 - -```sql -CREATE TABLE table_name ( ... ) ENGINE = ReplicatedMergeTree('zookeeper_name_configured_in_auxiliary_zookeepers:path', 'replica_name') ... -``` - -任意の既存の ZooKeeper クラスターを指定でき、システムはそのクラスター上のディレクトリを自身のデータ用として使用します(このディレクトリはレプリケーテッドテーブルを作成するときに指定します)。 - -config ファイルで ZooKeeper が設定されていない場合、レプリケーテッドテーブルを作成できず、既存のレプリケーテッドテーブルは読み取り専用になります。 - - -ZooKeeper は `SELECT` クエリでは使用されません。これは、レプリケーションが `SELECT` のパフォーマンスに影響せず、レプリケートされていないテーブルと同じ速度でクエリが実行されるためです。分散レプリケートテーブルに対してクエリを実行する場合、ClickHouse の動作は設定項目 [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) と [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries) によって制御されます。 - -各 `INSERT` クエリごとに、おおよそ 10 個のエントリが複数のトランザクションを通して ZooKeeper に追加されます。(より正確には、これは挿入される各データブロックごとです。1 つの `INSERT` クエリには 1 つのブロック、もしくは `max_insert_block_size = 1048576` 行ごとに 1 ブロックが含まれます。)このため、レプリケートされていないテーブルと比較して `INSERT` のレイテンシーがわずかに長くなります。ただし、1 秒あたり 1 回以下の `INSERT` でデータをバッチ挿入するという推奨事項に従えば、問題は発生しません。1 つの ZooKeeper クラスターをコーディネーションに使用している ClickHouse クラスター全体で、1 秒あたり数百件程度の `INSERT` が発生します。データ挿入時のスループット(1 秒あたりの行数)は、レプリケートされていないデータと同程度に高いままです。 - -非常に大規模なクラスターでは、シャードごとに別々の ZooKeeper クラスターを使用できます。しかし、運用クラスター(およそ 300 台のサーバー)での実運用経験からは、その必要性は確認されていません。 - -レプリケーションは非同期かつマルチマスターです。`INSERT` クエリ(および `ALTER`)は、利用可能な任意のサーバーに送信できます。データはクエリが実行されたサーバーに挿入され、その後他のサーバーへコピーされます。非同期であるため、最近挿入されたデータは、他のレプリカには一定のレイテンシーを伴って反映されます。一部のレプリカが利用できない場合、利用可能になった時点でデータが書き込まれます。レプリカが利用可能な場合、レイテンシーは圧縮データブロックをネットワーク経由で転送する時間に相当します。レプリケートテーブルに対してバックグラウンドタスクを実行するスレッド数は、[background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) 設定で指定できます。 - -`ReplicatedMergeTree` エンジンは、レプリケーションのフェッチ用に専用のスレッドプールを使用します。プールのサイズは [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size) 設定によって制限されており、サーバーの再起動によって調整できます。 - -デフォルトでは、`INSERT` クエリは 1 つのレプリカからのデータ書き込みの確認だけを待ちます。もしデータが 1 つのレプリカにしか正常に書き込まれず、そのレプリカを持つサーバーが消失した場合、そのデータは失われます。複数のレプリカからの書き込み確認を有効にするには、`insert_quorum` オプションを使用します。 - -各データブロックはアトミックに書き込まれます。`INSERT` クエリは、最大 `max_insert_block_size = 1048576` 行までのブロックに分割されます。言い換えると、`INSERT` クエリの行数が 1048576 行未満であれば、そのクエリ全体がアトミックに処理されます。 - -データブロックは重複排除されます。同じデータブロック(同じサイズで、同じ行が同じ順序で含まれているデータブロック)を複数回書き込もうとした場合、そのブロックは 1 度だけ書き込まれます。これは、ネットワーク障害が発生した際に、クライアントアプリケーション側でデータが DB に書き込まれたかどうかを判断できない場合でも、`INSERT` クエリを単純に再送できるようにするためです。同一データを持つ `INSERT` がどのレプリカに送信されたかは関係ありません。`INSERTs` は冪等です。重複排除のパラメータは、[merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree) サーバー設定で制御されます。 - -レプリケーション時には、挿入対象の元データのみがネットワーク経由で転送されます。その後のデータ変換(マージ)は、すべてのレプリカで同じようにコーディネートされ、実行されます。これによりネットワーク使用量が最小化されるため、レプリカが異なるデータセンターに配置されている場合でもレプリケーションは良好に動作します(異なるデータセンター間でデータを複製することがレプリケーションの主な目的である点に注意してください)。 - -同じデータに対して任意の数のレプリカを持つことができます。これまでの経験では、本番運用では二重レプリケーション(ダブルレプリケーション)を用い、各サーバーで RAID-5 または RAID-6(場合によっては RAID-10)を使用する構成が、比較的信頼性が高く扱いやすいソリューションとなり得ます。 - -システムはレプリカ上のデータの同期状態を監視し、障害発生後に復旧することができます。フェイルオーバーは自動(データ差分が小さい場合)あるいは半自動(データの差異が大きい場合。この場合は設定ミスを示している可能性があります)で行われます。 - - - -## レプリケーテッドテーブルの作成 - -:::note -ClickHouse Cloud では、レプリケーションは自動的に行われます。 - -レプリケーション引数を指定せずに [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) を使用してテーブルを作成します。システムは内部的に、レプリケーションとデータ分散のために [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) を [`SharedMergeTree`](/cloud/reference/shared-merge-tree) に書き換えます。 - -レプリケーションはプラットフォームによって管理されるため、`ReplicatedMergeTree` を使用したり、レプリケーションパラメータを指定したりすることは避けてください。 - -::: - -### Replicated*MergeTree のパラメータ - -| Parameter | Description | -| ------------------ | ------------------------------------------------------------------------------- | -| `zoo_path` | ClickHouse Keeper 内のテーブルへのパス。 | -| `replica_name` | ClickHouse Keeper 内のレプリカ名。 | -| `other_parameters` | レプリケートされたバージョンを作成するために使用されるエンジンのパラメータ。たとえば、`ReplacingMergeTree` の `version` など。 | - -例: - -```sql -CREATE TABLE table_name -( - EventDate DateTime, - CounterID UInt32, - UserID UInt32, - ver UInt16 -ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{layer}-{shard}/table_name', '{replica}', ver) -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID); -``` - -
- 非推奨構文の例 - - ```sql - CREATE TABLE table_name - ( - EventDate DateTime, - CounterID UInt32, - UserID UInt32 - ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/table_name', '{replica}', EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID), EventTime), 8192); - ``` -
- -この例が示すように、これらのパラメータには波括弧内に置換文字列を含めることができます。置換される値は、設定ファイルの[macros](/operations/server-configuration-parameters/settings.md/#macros)セクションから取得されます。 - -例: - -```xml - - 02 - example05-02-1 - -``` - -ClickHouse Keeper 内のテーブルへのパスは、レプリケートされた各テーブルごとに一意である必要があります。異なるシャード上のテーブルは、それぞれ異なるパスを持つ必要があります。 -この場合、パスは次の要素から構成されます。 - -`/clickhouse/tables/` は共通のプレフィックスです。この文字列をそのまま使用することを推奨します。 - -`{shard}` はシャード識別子に展開されます。 - -`table_name` は ClickHouse Keeper 内でのテーブル用ノードの名前です。テーブル名と同じにしておくと良いでしょう。テーブル名とは異なり、`RENAME` クエリを実行しても変わらないため、明示的に定義します。 -*HINT*: `table_name` の前にデータベース名を付けることもできます。例: `db_name.table_name` - -2 つの組み込み置換 `{database}` と `{table}` を使用できます。これらはそれぞれテーブル名とデータベース名に展開されます(`macros` セクションでこれらのマクロが定義されていない場合)。したがって、ZooKeeper パスは `'/clickhouse/tables/{shard}/{database}/{table}'` のように指定できます。 -これらの組み込み置換を使用する場合、テーブル名の変更には注意してください。ClickHouse Keeper 内のパスは変更できず、テーブル名を変更するとマクロは別のパスに展開されます。その結果、テーブルは ClickHouse Keeper 内に存在しないパスを参照することになり、読み取り専用モードに移行します。 - -レプリカ名は同一テーブルの異なるレプリカを識別します。例にあるようにサーバー名を使用できます。名前は各シャード内で一意であれば十分です。 - -置換を使用せずにパラメータを明示的に定義することもできます。これはテスト時や小規模クラスタの設定には便利な場合があります。しかし、この場合は分散 DDL クエリ(`ON CLUSTER`)は使用できません。 - -大規模クラスタを扱う場合は、誤りの可能性を減らすために置換を使用することを推奨します。 - -サーバー設定ファイル内で `Replicated` テーブルエンジンのデフォルト引数を指定できます。例えば次のようにします: - -```xml -/clickhouse/tables/{shard}/{database}/{table} -{replica} -``` - -この場合、テーブル作成時の引数は省略できます。 - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree -ORDER BY x; -``` - -これは次と同等です: - - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/{database}/table_name', '{replica}') -ORDER BY x; -``` - -各レプリカで `CREATE TABLE` クエリを実行します。このクエリは新しいレプリケーテッドテーブルを作成するか、既存のテーブルに新しいレプリカを追加します。 - -テーブルにすでに他のレプリカ上のデータが存在する状態で新しいレプリカを追加した場合、そのクエリを実行した後に、他のレプリカから新しいレプリカへデータがコピーされます。言い換えると、新しいレプリカは自動的に他のレプリカと同期されます。 - -レプリカを削除するには、`DROP TABLE` を実行します。ただし、削除されるのは 1 つのレプリカのみであり、それはクエリを実行したサーバー上に存在するレプリカです。 - - -## 障害発生後のリカバリ - -サーバー起動時に ClickHouse Keeper が使用できない場合、レプリケートテーブルは読み取り専用モードに切り替わります。システムは定期的に ClickHouse Keeper への接続を試行します。 - -`INSERT` の実行中に ClickHouse Keeper が使用できない場合、または ClickHouse Keeper とのやり取り時にエラーが発生した場合は、例外がスローされます。 - -ClickHouse Keeper への接続後、システムはローカルファイルシステム上のデータセットが想定されているデータセット(この情報は ClickHouse Keeper が保持)と一致しているかをチェックします。軽微な不整合であれば、システムはレプリカとのデータ同期によってこれを解消します。 - -システムが破損したデータパーツ(ファイルサイズが不正なもの)や、認識されないパーツ(ファイルシステムに書き込まれているが ClickHouse Keeper に記録されていないパーツ)を検出した場合、それらを `detached` サブディレクトリに移動します(削除はされません)。不足しているパーツはレプリカからコピーされます。 - -ClickHouse は、大量のデータを自動削除するといった破壊的な操作は実行しないことに注意してください。 - -サーバー起動時(または ClickHouse Keeper との新しいセッション確立時)には、すべてのファイルの個数とサイズのみをチェックします。ファイルサイズが一致していても、途中のバイトがどこかで変更されているような場合は、すぐには検出されず、`SELECT` クエリでデータを読み取ろうとしたときに初めて検出されます。このクエリは、チェックサムの不一致または圧縮ブロックのサイズ不一致に関する例外をスローします。この場合、データパーツは検証キューに追加され、必要に応じてレプリカからコピーされます。 - -ローカルのデータセットが想定されているものと大きく異なる場合、安全機構がトリガーされます。サーバーはその旨をログに記録し、起動を拒否します。これは、あるシャード上のレプリカが誤って別のシャード上のレプリカと同じ構成にされてしまった、といった構成ミスを示している可能性があるためです。ただし、この機構のしきい値はかなり低く設定されており、通常の障害復旧中にもこの状況が発生することがあります。この場合、データは「ボタンを押す」ことで半自動的に復元されます。 - -リカバリを開始するには、任意の内容で ClickHouse Keeper 内に `/path_to_table/replica_name/flags/force_restore_data` ノードを作成するか、すべてのレプリケートテーブルを復元するためのコマンドを実行します。 - -```bash -sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data -``` - -その後、サーバーを再起動します。起動時にサーバーはこれらのフラグを削除し、リカバリを開始します。 - - -## 完全なデータ損失後の復旧 {#recovery-after-complete-data-loss} - -あるサーバーからすべてのデータとメタデータが消失した場合は、次の手順に従って復旧します。 - -1. そのサーバーに ClickHouse をインストールします。シャード識別子およびレプリカを使用している場合は、それらを含む設定ファイル内の置換設定を正しく定義します。 -2. 各サーバーに手動で複製する必要がある非レプリケートテーブルがある場合は、レプリカ上のデータをコピーします(ディレクトリ `/var/lib/clickhouse/data/db_name/table_name/` 内)。 -3. レプリカから `/var/lib/clickhouse/metadata/` にあるテーブル定義をコピーします。テーブル定義内でシャードまたはレプリカ識別子が明示的に定義されている場合は、このレプリカに対応するように修正します。(別の方法としては、サーバーを起動し、`/var/lib/clickhouse/metadata/` 内の .sql ファイルに含まれているはずのすべての `ATTACH TABLE` クエリを実行します。) -4. 復旧を開始するには、任意の内容で ClickHouse Keeper ノード `/path_to_table/replica_name/flags/force_restore_data` を作成するか、すべてのレプリケートテーブルを復旧するために次のコマンドを実行します: `sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` - -その後、サーバーを起動します(すでに起動している場合は再起動します)。データはレプリカからダウンロードされます。 - -別の復旧方法としては、ClickHouse Keeper から失われたレプリカに関する情報(`/path_to_table/replica_name`)を削除し、「[レプリケートテーブルの作成](#creating-replicated-tables)」で説明されている手順に従ってレプリカを再作成する方法があります。 - -復旧時のネットワーク帯域幅には制限がありません。同時に多数のレプリカを復旧する場合は、この点に注意してください。 - - - -## MergeTree から ReplicatedMergeTree への変換 - -`MergeTree` という用語は、`MergeTree family` に属するすべてのテーブルエンジンを指すために使用しており、`ReplicatedMergeTree` についても同様です。 - -手動でレプリケーションしていた `MergeTree` テーブルがある場合、それをレプリケーション対応のテーブルに変換できます。`MergeTree` テーブルにすでに大量のデータを蓄積していて、これからレプリケーションを有効化したい場合に、この操作が必要になることがあります。 - -[ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) ステートメントを使用すると、デタッチされた `MergeTree` テーブルを `ReplicatedMergeTree` としてアタッチできます。 - -`convert_to_replicated` フラグがテーブルのデータディレクトリ(`Atomic` データベースの場合は `/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/`)に設定されている場合、サーバー再起動時に `MergeTree` テーブルは自動的に変換されます。 -空の `convert_to_replicated` ファイルを作成すると、次回のサーバー再起動時に、そのテーブルはレプリケーション対応テーブルとしてロードされます。 - -次のクエリを使用して、テーブルのデータパスを取得できます。テーブルに複数のデータパスがある場合は、先頭のパスを使用する必要があります。 - -```sql -SELECT data_paths FROM system.tables WHERE table = 'table_name' AND database = 'database_name'; -``` - -`ReplicatedMergeTree` テーブルは、`default_replica_path` および `default_replica_name` 設定の値を用いて作成される点に注意してください。 -他のレプリカ上で変換後のテーブルを作成するには、`ReplicatedMergeTree` エンジンの第 1 引数でそのパスを明示的に指定する必要があります。次のクエリでそのパスを取得できます。 - -```sql -SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; -``` - -これを行う別の方法として、手動で行うこともできます。 - -複数のレプリカ間でデータが異なっている場合は、まず同期をとるか、1つを除くすべてのレプリカからそのデータを削除します。 - -既存の MergeTree テーブルの名前を変更し、同じ古い名前で `ReplicatedMergeTree` テーブルを作成します。 -古いテーブルから、新しいテーブルデータのディレクトリ(`/var/lib/clickhouse/data/db_name/table_name/`)内にある `detached` サブディレクトリへデータを移動します。 -その後、いずれかのレプリカ上で `ALTER TABLE ATTACH PARTITION` を実行し、これらのデータパーツを稼働中のデータセットに追加します。 - - -## ReplicatedMergeTree から MergeTree への変換 {#converting-from-replicatedmergetree-to-mergetree} - -単一サーバー上で切り離されている `ReplicatedMergeTree` テーブルを `MergeTree` として再アタッチするには、[ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) ステートメントを使用します。 - -これを行う別の方法として、サーバーの再起動を伴う手順があります。別の名前の MergeTree テーブルを作成し、`ReplicatedMergeTree` テーブルのデータがあるディレクトリから、すべてのデータを新しいテーブルのデータディレクトリに移動します。その後、`ReplicatedMergeTree` テーブルを削除してサーバーを再起動します。 - -サーバーを起動せずに `ReplicatedMergeTree` テーブルを削除したい場合は、次の手順を実行します。 - -- メタデータディレクトリ(`/var/lib/clickhouse/metadata/`)内の対応する `.sql` ファイルを削除します。 -- ClickHouse Keeper 内の対応するパス(`/path_to_table/replica_name`)を削除します。 - -これが完了したら、サーバーを起動し、`MergeTree` テーブルを作成して、そのデータディレクトリにデータを移動し、サーバーを再起動できます。 - - - -## ClickHouse Keeper クラスター内のメタデータが失われたり破損した場合の復旧 {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} - -ClickHouse Keeper 内のデータが失われたり破損した場合は、上記で説明したように、データを非レプリケートテーブルへ移動することで保持できます。 - -**関連項目** - -- [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) -- [background_fetches_pool_size](/operations/server-configuration-parameters/settings.md/#background_fetches_pool_size) -- [execute_merges_on_single_replica_time_threshold](/operations/settings/merge-tree-settings#execute_merges_on_single_replica_time_threshold) -- [max_replicated_fetches_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_fetches_network_bandwidth) -- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md index bdf14e7f005..721ae5661cc 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# SummingMergeTree テーブルエンジン +# SummingMergeTree テーブルエンジン {#summingmergetree-table-engine} このエンジンは [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) を継承しています。違いは、`SummingMergeTree` テーブルでデータパーツをマージする際に、ClickHouse が同じ主キー(より正確には同じ [ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md))を持つすべての行を、数値データ型のカラムの値を合計した 1 行に置き換える点です。ソートキーの構成によって、1 つのキー値に多数の行が対応する場合、これにより必要なストレージ容量を大幅に削減し、データ取得の高速化を実現できます。 @@ -17,7 +17,7 @@ doc_type: 'reference' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -34,16 +34,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] リクエストパラメータの説明については、[リクエストの説明](../../../sql-reference/statements/create/table.md)を参照してください。 -### SummingMergeTree のパラメータ +### SummingMergeTree のパラメータ {#parameters-of-summingmergetree} -#### カラム +#### カラム {#columns} `columns` - 値を集計(合計)するカラム名を含むタプルです。省略可能なパラメータです。 カラムは数値型である必要があり、パーティションキーまたはソートキーに含めることはできません。 `columns` が指定されていない場合、ClickHouse はソートキーに含まれていない数値データ型のすべてのカラムの値を集計します。 -### クエリ句 +### クエリ句 {#query-clauses} `SummingMergeTree` テーブルを作成する際には、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) を指定する必要があります。 @@ -69,7 +69,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用例 +## 使用例 {#usage-example} 次のテーブルを例にします。 @@ -103,13 +103,13 @@ SELECT key, sum(value) FROM summtt GROUP BY key ``` -## データ処理 +## データ処理 {#data-processing} データがテーブルに挿入されると、そのままの形で保存されます。ClickHouse は挿入されたデータパートを定期的にマージし、その際に同じ主キーを持つ行が合計され、各結果データパートごとに 1 行に置き換えられます。 ClickHouse はデータパートをマージする際、マージの結果として異なるデータパート同士に同じ主キーを持つ行が分かれて存在する場合があります。つまり、合計処理が不完全になる可能性があります。そのため、上記の例で説明したように、クエリでは集約関数 [`sum()`](/sql-reference/aggregate-functions/reference/sum) と `GROUP BY` 句を組み合わせて使用する必要があります。 -### 集計に関する共通ルール +### 集計に関する共通ルール {#common-rules-for-summation} 数値データ型の列に含まれる値は合計されます。対象となる列の集合はパラメータ `columns` で定義されます。 @@ -119,11 +119,11 @@ ClickHouse はデータパートをマージする際、マージの結果とし 主キーに含まれる列の値は合計されません。 -### AggregateFunction 列における集計 +### AggregateFunction 列における集計 {#the-summation-in-the-aggregatefunction-columns} [AggregateFunction 型](../../../sql-reference/data-types/aggregatefunction.md)の列に対しては、ClickHouse は [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md) エンジンと同様に、関数に従って集約処理を行います。 -### ネストした構造 +### ネストした構造 {#nested-structures} テーブルには特別な方法で処理されるネストしたデータ構造を含めることができます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md index a2343e1536d..e993a76f5d0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# VersionedCollapsingMergeTree テーブルエンジン +# VersionedCollapsingMergeTree テーブルエンジン {#versionedcollapsingmergetree-table-engine} このエンジンは次のことができます: @@ -22,7 +22,7 @@ doc_type: 'reference' -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -39,7 +39,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] クエリパラメータの説明は、[クエリの説明](../../../sql-reference/statements/create/table.md)を参照してください。 -### エンジンのパラメータ +### エンジンのパラメータ {#engine-parameters} ```sql VersionedCollapsingMergeTree(サイン, バージョン) @@ -50,7 +50,7 @@ VersionedCollapsingMergeTree(サイン, バージョン) | `sign` | 行の種類を示す列の名前。`1` は「state」行、`-1` は「cancel」行を表します。 | [`Int8`](/sql-reference/data-types/int-uint) | | `version` | オブジェクト状態のバージョンを表す列の名前。 | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) または [`DateTime64`](/sql-reference/data-types/datetime64) | -### クエリ句 +### クエリ句 {#query-clauses} `VersionedCollapsingMergeTree` テーブルを作成する際は、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) が必要です。 @@ -82,9 +82,9 @@ VersionedCollapsingMergeTree(サイン, バージョン) -## 折りたたみ(Collapsing) +## 折りたたみ(Collapsing) {#table_engines_versionedcollapsingmergetree} -### データ +### データ {#data} あるオブジェクトについて、継続的に変化するデータを保存する必要がある状況を考えます。オブジェクトごとに 1 行を持ち、変更があるたびにその行を更新するのは合理的に思えます。しかし、更新操作はストレージ上のデータを書き換える必要があるため、DBMS にとっては高コストかつ低速です。データを高速に書き込む必要がある場合、更新は適していませんが、その代わりにオブジェクトに対する変更を次のように逐次書き込むことができます。 @@ -130,7 +130,7 @@ VersionedCollapsingMergeTree(サイン, バージョン) 2. 列内で長く伸び続ける配列は、書き込み時の負荷によりエンジンの効率を低下させます。データが単純であればあるほど効率は高くなります。 3. `SELECT` の結果は、オブジェクト変更履歴の一貫性に大きく依存します。挿入するデータを準備する際は注意してください。セッション深度のような本来非負であるメトリクスに対して負の値が得られるなど、不整合なデータでは予測不能な結果になる可能性があります。 -### Algorithm +### Algorithm {#table_engines-versionedcollapsingmergetree-algorithm} ClickHouse がデータパートをマージする際、同じ主キーとバージョンを持ち、`Sign` が異なる行のペアを削除します。行の順序は関係ありません。 @@ -149,7 +149,7 @@ ClickHouse は、同じプライマリキーを持つすべての行が、同じ -## 使用例 +## 使用例 {#example-of-use} サンプルデータ: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md index 0a81e922891..e3f964afeaa 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md @@ -1,5 +1,5 @@ --- -description: 'Alias テーブルエンジンは、別のテーブルへの透過的なプロキシを作成します。すべての操作は対象テーブルに委譲され、Alias 自体にはデータは保持されません。' +description: 'Alias テーブルエンジンは、別のテーブルへの透過的なプロキシとして機能します。すべての操作は対象テーブルに転送され、エイリアス自体はデータを保持しません。' sidebar_label: 'Alias' sidebar_position: 5 slug: /engines/table-engines/special/alias @@ -7,22 +7,30 @@ title: 'Alias テーブルエンジン' doc_type: 'reference' --- +import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Alias テーブルエンジン +# Alias テーブルエンジン {#alias-table-engine} -`Alias` エンジンは、別のテーブルへのプロキシを作成するテーブルエンジンです。すべての読み取りおよび書き込み操作は対象テーブルに転送され、エイリアス自体はデータを保持せず、対象テーブルへの参照のみを保持します。 + +`Alias` エンジンは、別のテーブルへのプロキシを作成します。すべての読み取りおよび書き込み操作は対象テーブルに転送され、エイリアス自体はデータを保持せず、対象テーブルへの参照のみを保持します。 +:::info +これは実験的な機能であり、将来のリリースで後方互換性を損なう形で変更される可能性があります。 +`Alias` テーブルエンジンを使用するには、 +[allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine) 設定を有効にしてください。 +次のコマンドを実行します: `set allow_experimental_alias_table_engine = 1`。 +::: -## テーブルを作成する +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [db_name.]alias_name ENGINE = Alias(target_table) ``` -または、データベース名を明示的に指定して: +または、データベース名を明示的に指定する場合: ```sql CREATE TABLE [db_name.]alias_name @@ -30,53 +38,50 @@ ENGINE = Alias(target_db, target_table) ``` :::note -`Alias` テーブルでは、カラムを明示的に定義することはできません。カラムは自動的に対象テーブルから継承されます。これにより、エイリアスは常に対象テーブルのスキーマと一致することが保証されます。 +`Alias` テーブルでは、明示的な列定義を行うことはできません。列はターゲットテーブルから自動的に継承されます。これにより、エイリアスは常にターゲットテーブルのスキーマと一致します。 ::: ## エンジンパラメータ {#engine-parameters} -- **`target_db (optional)`** — 対象テーブルを含むデータベース名。 -- **`target_table`** — 対象テーブル名。 - - +- **`target_db (optional)`** — 対象テーブルを含むデータベースの名前(省略可能)。 +- **`target_table`** — 対象テーブルの名前。 -## サポートされる操作 {#supported-operations} +## サポートされている操作 {#supported-operations} `Alias` テーブルエンジンは、主要な操作をすべてサポートします。 -### 対象テーブルへの操作 {#operations-on-target} -これらの操作は、エイリアスを介して対象テーブルに対して実行されます: +### ターゲットテーブルに対する操作 {#operations-on-target} + +これらの操作はターゲットテーブルに対してプロキシ経由で実行されます: | Operation | Support | Description | |-----------|---------|-------------| -| `SELECT` | ✅ | 対象テーブルからデータを読み取る | -| `INSERT` | ✅ | 対象テーブルにデータを書き込む | -| `INSERT SELECT` | ✅ | 対象テーブルへのバッチ挿入を行う | -| `ALTER TABLE ADD COLUMN` | ✅ | 対象テーブルにカラムを追加する | -| `ALTER TABLE MODIFY SETTING` | ✅ | 対象テーブルの設定を変更する | -| `ALTER TABLE PARTITION` | ✅ | 対象テーブルに対するパーティション操作 (DETACH/ATTACH/DROP) を行う | -| `ALTER TABLE UPDATE` | ✅ | 対象テーブル内の行を更新する (ミューテーション) | -| `ALTER TABLE DELETE` | ✅ | 対象テーブルから行を削除する (ミューテーション) | -| `OPTIMIZE TABLE` | ✅ | 対象テーブルを最適化する (パーツをマージする) | -| `TRUNCATE TABLE` | ✅ | 対象テーブルを空にする | +| `SELECT` | ✅ | ターゲットテーブルからデータを読み取る | +| `INSERT` | ✅ | ターゲットテーブルにデータを書き込む | +| `INSERT SELECT` | ✅ | ターゲットテーブルへのバッチ挿入を行う | +| `ALTER TABLE ADD COLUMN` | ✅ | ターゲットテーブルに列を追加する | +| `ALTER TABLE MODIFY SETTING` | ✅ | ターゲットテーブルの設定を変更する | +| `ALTER TABLE PARTITION` | ✅ | ターゲットに対するパーティション操作 (DETACH/ATTACH/DROP) を行う | +| `ALTER TABLE UPDATE` | ✅ | ターゲットテーブルの行を更新する (mutation) | +| `ALTER TABLE DELETE` | ✅ | ターゲットテーブルから行を削除する (mutation) | +| `OPTIMIZE TABLE` | ✅ | ターゲットテーブルを最適化する (パートをマージ) | +| `TRUNCATE TABLE` | ✅ | ターゲットテーブルを空にする | -### `Alias` 自体への操作 {#operations-on-alias} +### エイリアス自体への操作 {#operations-on-alias} -これらの操作はエイリアスのみに影響し、対象テーブルには **影響しません**: +これらの操作はエイリアスのみに作用し、ターゲットテーブルには**影響しません**。 | Operation | Support | Description | |-----------|---------|-------------| -| `DROP TABLE` | ✅ | エイリアスのみを削除し、対象テーブルは変更されない | -| `RENAME TABLE` | ✅ | エイリアスの名前のみを変更し、対象テーブルは変更されない | - - +| `DROP TABLE` | ✅ | エイリアスのみを削除し、ターゲットテーブルには変更が加わらない | +| `RENAME TABLE` | ✅ | エイリアスの名前だけを変更し、ターゲットテーブルには変更が加わらない | -## 使用例 +## 使用例 {#usage-examples} -### 基本的なエイリアスの作成 +### 基本的なエイリアスの作成 {#basic-alias-creation} -同じデータベース内にシンプルなエイリアスを作成します。 +同一データベース内で簡単なエイリアスを作成します。 ```sql -- ソーステーブルを作成 @@ -93,7 +98,7 @@ INSERT INTO source_data VALUES (1, 'one', 10.1), (2, 'two', 20.2); -- エイリアスを作成 CREATE TABLE data_alias ENGINE = Alias('source_data'); --- エイリアス経由でクエリを実行 +-- エイリアス経由でクエリ SELECT * FROM data_alias; ``` @@ -104,9 +109,10 @@ SELECT * FROM data_alias; └────┴──────┴───────┘ ``` -### データベース間エイリアス -別のデータベース内のテーブルを指すエイリアスを作成します: +### データベース間エイリアス {#cross-database-alias} + +別のデータベース内のテーブルを参照するエイリアスを作成します。 ```sql -- データベースを作成 @@ -127,14 +133,15 @@ CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); -- またはdatabase.table形式を使用 CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); --- 両方のエイリアスは同一に動作 +-- どちらのエイリアスも同じように動作します INSERT INTO db2.events_alias VALUES (now(), 'click', 100); SELECT * FROM db2.events_alias2; ``` -### エイリアス経由の書き込み操作 -すべての書き込み操作はターゲットテーブルに転送されます。 +### エイリアス経由での書き込み操作 {#write-operations} + +すべての書き込みはターゲットテーブルに転送されます。 ```sql CREATE TABLE metrics ( @@ -146,7 +153,7 @@ ORDER BY ts; CREATE TABLE metrics_alias ENGINE = Alias('metrics'); --- エイリアスを介して挿入 +-- エイリアス経由で挿入 INSERT INTO metrics_alias VALUES (now(), 'cpu_usage', 45.2), (now(), 'memory_usage', 78.5); @@ -162,9 +169,10 @@ SELECT count() FROM metrics; -- 7を返す SELECT count() FROM metrics_alias; -- 7を返す ``` -### スキーマの変更 -`ALTER` 操作は対象テーブルのスキーマを変更します。 +### スキーマの変更 {#schema-modification} + +ALTER 操作は対象テーブルのスキーマを変更します。 ```sql CREATE TABLE users ( @@ -190,9 +198,10 @@ DESCRIBE users; └───────┴────────┴──────────────┴────────────────────┘ ``` -### データの変更 -`UPDATE` および `DELETE` 操作がサポートされています。 +### データの変更 {#data-mutations} + +UPDATE 文および DELETE 文がサポートされています。 ```sql CREATE TABLE products ( @@ -216,7 +225,7 @@ ALTER TABLE products_alias UPDATE price = price * 1.1 WHERE status = 'active'; -- エイリアス経由で削除 ALTER TABLE products_alias DELETE WHERE status = 'inactive'; --- 変更は対象テーブルに適用されます +-- 変更は対象テーブルに適用される SELECT * FROM products ORDER BY id; ``` @@ -227,10 +236,10 @@ SELECT * FROM products ORDER BY id; └────┴──────────┴───────┴────────┘ ``` -### パーティション操作 -パーティション化されたテーブルでは、パーティション操作はフォワードされます。 +### パーティション操作 {#partition-operations} +パーティション化されたテーブルでは、パーティション操作はそのまま伝播されます。 ```sql CREATE TABLE logs ( @@ -259,9 +268,10 @@ ALTER TABLE logs_alias ATTACH PARTITION '202402'; SELECT count() FROM logs_alias; -- 3を返す ``` -### テーブルの最適化 -ターゲットテーブルに対するパーツのマージ処理を最適化します。 +### テーブル最適化 {#table-optimization} + +ターゲットテーブル内のパーツをマージする処理を最適化します。 ```sql CREATE TABLE events ( @@ -272,30 +282,31 @@ ORDER BY id; CREATE TABLE events_alias ENGINE = Alias('events'); --- 複数の挿入により複数のパートが作成されます +-- 複数の INSERT により複数のパートが作成される INSERT INTO events_alias VALUES (1, 'data1'); INSERT INTO events_alias VALUES (2, 'data2'); INSERT INTO events_alias VALUES (3, 'data3'); --- パート数を確認します +-- パート数を確認 SELECT count() FROM system.parts WHERE database = currentDatabase() AND table = 'events' AND active; --- エイリアス経由で最適化します +-- エイリアス経由で最適化 OPTIMIZE TABLE events_alias FINAL; --- ターゲットテーブルでパートがマージされます +-- ターゲットテーブルでパートがマージされる SELECT count() FROM system.parts WHERE database = currentDatabase() AND table = 'events' - AND active; -- 1を返します + AND active; -- 1 を返す ``` -### エイリアスの管理 -エイリアスは個別に名前を変更したり削除したりできます。 +### エイリアスの管理 {#alias-management} + +エイリアスはそれぞれ独立して名前を変更したり削除したりできます。 ```sql CREATE TABLE important_data ( @@ -308,15 +319,15 @@ INSERT INTO important_data VALUES (1, 'critical'), (2, 'important'); CREATE TABLE old_alias ENGINE = Alias('important_data'); --- エイリアスの名前を変更(ターゲットテーブルは変更されません) +-- エイリアスの名前を変更(対象テーブルは変更されません) RENAME TABLE old_alias TO new_alias; -- 同じテーブルに別のエイリアスを作成 CREATE TABLE another_alias ENGINE = Alias('important_data'); --- エイリアスを1つ削除(ターゲットテーブルと他のエイリアスは変更されません) +-- エイリアスを1つ削除(対象テーブルと他のエイリアスは変更されません) DROP TABLE new_alias; -SELECT * FROM another_alias; -- 引き続き機能します +SELECT * FROM another_alias; -- 引き続き動作します SELECT count() FROM important_data; -- データは保持されており、2を返します ``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md index adf2f7643f6..4a68e468807 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md @@ -7,9 +7,7 @@ title: 'Buffer テーブルエンジン' doc_type: 'reference' --- - - -# Buffer テーブルエンジン +# Buffer テーブルエンジン {#buffer-table-engine} 書き込み対象のデータを RAM 上でバッファリングし、定期的に別のテーブルへフラッシュします。読み取り時には、バッファとその別のテーブルの両方から同時にデータを読み取ります。 @@ -21,27 +19,27 @@ Buffer テーブルエンジンの代替として推奨されるのは、[非同 Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) ``` -### エンジンパラメータ +### エンジンパラメータ {#engine-parameters} -#### `database` +#### `database` {#database} `database` – データベース名。`currentDatabase()` もしくは文字列を返す他の定数式を使用できます。 -#### `table` +#### `table` {#table} `table` – データをフラッシュする対象のテーブル。 -#### `num_layers` +#### `num_layers` {#num_layers} `num_layers` – 並列レイヤー数。物理的には、テーブルは互いに独立したバッファが `num_layers` 個ある形で表現されます。 -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, および `max_bytes` +#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, および `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} バッファからデータをフラッシュするための条件。 -### オプションのエンジンパラメータ +### オプションのエンジンパラメータ {#optional-engine-parameters} -#### `flush_time`, `flush_rows`, および `flush_bytes` +#### `flush_time`, `flush_rows`, および `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} バックグラウンドでバッファからデータをフラッシュするための条件(省略または 0 の場合は、`flush*` パラメータは存在しないものとして扱われます)。 @@ -49,15 +47,15 @@ Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_ また、少なくとも 1 つの `flush*` 条件が満たされると、バックグラウンドでフラッシュが開始されます。これは `max*` とは異なり、`flush*` によって、Buffer テーブルへの `INSERT` クエリにレイテンシを追加しないように、バックグラウンドフラッシュを個別に構成できます。 -#### `min_time`, `max_time`, および `flush_time` +#### `min_time`, `max_time`, および `flush_time` {#min_time-max_time-and-flush_time} 最初の書き込み時点からの経過時間(秒)に関する条件。 -#### `min_rows`, `max_rows`, および `flush_rows` +#### `min_rows`, `max_rows`, および `flush_rows` {#min_rows-max_rows-and-flush_rows} バッファ内の行数に関する条件。 -#### `min_bytes`, `max_bytes`, および `flush_bytes` +#### `min_bytes`, `max_bytes`, および `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} バッファ内のバイト数に関する条件。 @@ -84,7 +82,6 @@ CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, データベース名およびテーブル名には、シングルクォートで囲んだ空文字列を設定できます。これは、宛先テーブルが存在しないことを示します。この場合、データのフラッシュ条件に達すると、バッファは単にクリアされます。これは、メモリ上に一定期間分のデータだけを保持しておきたい場合に有用です。 - Buffer テーブルから読み取る場合、データはバッファと宛先テーブル(存在する場合)の両方から処理されます。 なお、Buffer テーブルはインデックスをサポートしません。つまり、バッファ内のデータはフルスキャンされるため、大きなバッファでは低速になる可能性があります(下位テーブル内のデータについては、そのテーブルがサポートするインデックスが使用されます)。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md index 6281e3f7900..82534891b7d 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md @@ -7,15 +7,11 @@ title: 'Dictionary テーブルエンジン' doc_type: 'リファレンス' --- - - -# Dictionary テーブルエンジン +# Dictionary テーブルエンジン {#dictionary-table-engine} `Dictionary` エンジンは、[dictionary](../../../sql-reference/dictionaries/index.md) のデータを ClickHouse のテーブルとして扱います。 - - -## 例 +## 例 {#example} 例として、次のように構成された `products` 辞書を考えます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md index 52070299b6c..847d7b76a0e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Distributed テーブルエンジン +# Distributed テーブルエンジン {#distributed-table-engine} :::warning ClickHouse Cloud における Distributed エンジン ClickHouse Cloud で Distributed テーブルエンジンを作成するには、[`remote` および `remoteSecure`](../../../sql-reference/table-functions/remote) テーブル関数を使用します。 @@ -21,7 +21,7 @@ Distributed エンジンを持つテーブル自体はデータを一切保存 -## テーブルの作成 +## テーブルの作成 {#distributed-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] [SETTINGS name=value, ...] ``` -### テーブルから +### テーブルから {#distributed-from-a-table} `Distributed` テーブルが現在のサーバー上のテーブルを参照している場合、そのテーブルのスキーマを利用できます。 @@ -41,7 +41,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] ``` -### Distributed パラメータ +### Distributed パラメータ {#distributed-parameters} | Parameter | Description | | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -56,7 +56,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 * [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) 設定 * 例については [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) を参照 -### Distributed 設定 +### Distributed 設定 {#distributed-settings} | Setting | Description | Default value | @@ -102,7 +102,7 @@ SETTINGS データベース名の代わりに、文字列を返す定数式を使用できます。例えば、`currentDatabase()` です。 -## クラスター +## クラスター {#distributed-clusters} クラスターは[サーバー設定ファイル](../../../operations/configuration-files.md)で構成されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md index 666d7949b08..e44d4b1d19a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md @@ -7,20 +7,16 @@ title: '`Executable` および `ExecutablePool` テーブルエンジン' doc_type: 'reference' --- - - -# Executable および ExecutablePool テーブルエンジン +# Executable および ExecutablePool テーブルエンジン {#executable-and-executablepool-table-engines} `Executable` および `ExecutablePool` テーブルエンジンを使用すると、(行を **stdout** に書き出すことで)ユーザー定義のスクリプトによって行が生成されるテーブルを定義できます。実行可能スクリプトは `users_scripts` ディレクトリに保存され、任意のソースからデータを読み取ることができます。 -- `Executable` テーブル: クエリごとにスクリプトが実行されます -- `ExecutablePool` テーブル: 永続プロセスのプールを維持し、読み取り時にそのプールからプロセスを取得します +* `Executable` テーブル: クエリごとにスクリプトが実行されます +* `ExecutablePool` テーブル: 永続プロセスのプールを維持し、読み取り時にそのプールからプロセスを取得します オプションとして、1 つ以上の入力用クエリを含めることができ、その結果を **stdin** にストリームしてスクリプトが読み取れるようにできます。 - - -## `Executable` テーブルの作成 +## `Executable` テーブルの作成 {#creating-an-executable-table} `Executable` テーブルエンジンには、スクリプト名と入力データの形式という 2 つのパラメータを指定する必要があります。必要に応じて、1 つ以上の入力クエリを渡すこともできます。 @@ -102,8 +98,7 @@ SELECT * FROM my_executable_table └───┴────────────┘ ``` - -## クエリ結果をスクリプトに渡す +## クエリ結果をスクリプトに渡す {#passing-query-results-to-a-script} Hacker News サイトのユーザーはコメントを投稿します。Python には自然言語処理ツールキット (`nltk`) があり、その中の `SentimentIntensityAnalyzer` を使うと、コメントがポジティブかネガティブかニュートラルかを判定し、-1(非常にネガティブなコメント)から 1(非常にポジティブなコメント)の値を割り当てることができます。`nltk` を使って Hacker News のコメントのセンチメント(感情)を計算する `Executable` テーブルを作成してみましょう。 @@ -177,7 +172,6 @@ FROM sentiment レスポンスは次のとおりです。 - ```response ┌───────id─┬─sentiment─┐ │ 7398199 │ 0.4404 │ @@ -203,8 +197,7 @@ FROM sentiment └──────────┴───────────┘ ``` - -## `ExecutablePool` テーブルの作成 +## `ExecutablePool` テーブルの作成 {#creating-an-executablepool-table} `ExecutablePool` の構文は `Executable` と似ていますが、`ExecutablePool` テーブルに固有の重要な設定がいくつかあります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md index db6c663ecc1..a7bba2403f7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md @@ -7,7 +7,7 @@ title: 'クエリ処理のための外部データ' doc_type: 'reference' --- -# クエリ処理のための外部データ +# クエリ処理のための外部データ {#external-data-for-query-processing} ClickHouse では、`SELECT` クエリと一緒に、そのクエリの処理に必要なデータをサーバーに送信できます。これらのデータは一時テーブル(「Temporary tables」のセクションを参照)に格納され、クエリ内(たとえば `IN` 演算子)で利用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md index 2e9784010ee..3f2683934f5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# File テーブルエンジン +# File テーブルエンジン {#file-table-engine} File テーブルエンジンは、サポートされている[ファイルフォーマット](/interfaces/formats#formats-overview)(`TabSeparated`、`Native` など)のいずれかでデータをファイルに保存します。 @@ -25,7 +25,7 @@ File テーブルエンジンは、サポートされている[ファイルフ -## ClickHouse サーバーでの利用方法 +## ClickHouse サーバーでの利用方法 {#usage-in-clickhouse-server} ```sql File(Format) @@ -44,7 +44,7 @@ ClickHouse では、`File` に対してファイルシステムのパスを指 ::: -## 例 +## 例 {#example} **1.** `file_engine_table` テーブルを作成します。 @@ -76,7 +76,7 @@ SELECT * FROM file_engine_table ``` -## ClickHouse-local での使用方法 +## ClickHouse-local での使用方法 {#usage-in-clickhouse-local} [clickhouse-local](../../../operations/utilities/clickhouse-local.md) では、File エンジンは `Format` に加えてファイルパスも指定できます。デフォルトの入出力ストリームは、`0` や `stdin`、`1` や `stdout` のような数値または人間が読める名前で指定できます。追加のエンジンパラメータまたはファイル拡張子(`gz`、`br`、`xz`)に基づいて、圧縮ファイルの読み書きを行えます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md index 0870bcf5825..f7d49039b45 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md @@ -20,7 +20,7 @@ doc_type: 'reference' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -56,7 +56,7 @@ Engine 引数: * `handle_error_mode` — FileLog エンジンでエラーをどのように処理するか。指定可能な値: `default`(メッセージのパースに失敗した場合に例外をスロー)、`stream`(例外メッセージと元のメッセージを仮想カラム `_error` と `_raw_message` に保存)。 -## 説明 +## 説明 {#description} 取り込まれたレコードは自動的に追跡されるため、ログファイル内の各レコードは一度だけカウントされます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md index 0e22ebc3efe..80617691ca0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# GenerateRandom テーブルエンジン +# GenerateRandom テーブルエンジン {#generaterandom-table-engine} GenerateRandom テーブルエンジンは、指定されたテーブルスキーマに基づいてランダムなデータを生成します。 @@ -20,7 +20,7 @@ GenerateRandom テーブルエンジンは、指定されたテーブルスキ -## ClickHouse サーバーでの利用方法 +## ClickHouse サーバーでの利用方法 {#usage-in-clickhouse-server} ```sql ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) @@ -33,7 +33,7 @@ Generate テーブルエンジンは `SELECT` クエリのみをサポートし テーブルに保存可能な [DataTypes](../../../sql-reference/data-types/index.md) のうち、`AggregateFunction` を除くすべてをサポートします。 -## 例 +## 例 {#example} **1.** `generate_engine_table` テーブルを作成します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md index e6134f97d1c..ff1b2715358 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md @@ -7,9 +7,7 @@ title: '特殊テーブルエンジン' doc_type: 'reference' --- - - -# 特殊なテーブルエンジン +# 特殊なテーブルエンジン {#special-table-engines} テーブルエンジンには、主に次の 3 つのカテゴリがあります。 @@ -26,7 +24,6 @@ doc_type: 'reference' 誤りに気付いた場合は、該当ページの YML フロントマターを編集してください。 */ } - {/*AUTOGENERATED_START*/ } | Page | 説明 | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md index 4cba07ffe27..9b083a61439 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Join テーブルエンジン +# Join テーブルエンジン {#join-table-engine} [JOIN](/sql-reference/statements/select/join) 演算で使用するための、オプションの事前構築済みデータ構造です。 @@ -19,7 +19,7 @@ ClickHouse Cloud で、サービスがバージョン 25.4 より前に作成さ -## テーブルの作成 +## テーブルの作成 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -115,7 +115,7 @@ I/O オーバーヘッドを削減します。性能を重視し、永続化を -## 使用例 +## 使用例 {#example} 左側テーブルの作成: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md index 60922cac90f..3c04723fc6c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# KeeperMap テーブルエンジン +# KeeperMap テーブルエンジン {#keepermap-table-engine} このエンジンを使用すると、Keeper/ZooKeeper クラスターを、線形化可能な書き込みと逐次一貫な読み取りを備えた一貫性のあるキー・バリュー ストアとして利用できます。 @@ -26,7 +26,7 @@ KeeperMap ストレージエンジンを有効化するには、テーブルを ここで path には任意の有効な ZooKeeper パスを指定できます。 -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -80,9 +80,9 @@ PRIMARY KEY key もちろん、関連しない ClickHouse インスタンス間であっても、同じパスを指定して手動で `CREATE TABLE` を実行することで、同様のデータ共有効果を得ることができます。 -## サポートされている操作 +## サポートされている操作 {#supported-operations} -### 挿入 +### 挿入 {#inserts} 新しい行が `KeeperMap` に挿入されるとき、キーが存在しない場合は、そのキー用の新しいエントリが作成されます。 キーが存在し、かつ `keeper_map_strict_mode` が `true` に設定されている場合は、例外がスローされます。そうでない場合、そのキーに対する値は上書きされます。 @@ -93,7 +93,7 @@ PRIMARY KEY key INSERT INTO keeper_map_table VALUES ('some key', 1, 'value', 3.2); ``` -### 削除 +### 削除 {#deletes} 行は `DELETE` クエリまたは `TRUNCATE` を使用して削除できます。 キーが存在しており、設定 `keeper_map_strict_mode` が `true` の場合、データの取得および削除は、それらをアトミックに実行できる場合にのみ成功します。 @@ -110,7 +110,7 @@ ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; TRUNCATE TABLE keeper_map_table; ``` -### 更新 +### 更新 {#updates} 値は `ALTER TABLE` クエリを使用して更新できます。プライマリキーは更新できません。 `keeper_map_strict_mode` を `true` に設定すると、データの取得および更新は、アトミックに実行された場合にのみ成功します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md index 2eb34ab15af..ad156123378 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Memory テーブルエンジン +# Memory テーブルエンジン {#memory-table-engine} :::note ClickHouse Cloud 上で Memory テーブルエンジンを使用する場合、データは(設計上)すべてのノード間でレプリケートされません。すべてのクエリが同じノードにルーティングされ、Memory テーブルエンジンが期待どおりに動作することを保証するには、次のいずれかを行ってください: @@ -48,7 +48,7 @@ Memory エンジンのテーブルサイズを制限するために上限およ -## 使用方法 +## 使用方法 {#usage} **設定を初期化する** @@ -65,7 +65,7 @@ ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 100 **注意:** `bytes` と `rows` の両方の上限パラメータは同時に設定できますが、`max` と `min` のうち小さい方の値が優先されます。 -## 例 +## 例 {#examples} ```sql CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md index 90a696086c3..90e44282d19 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Merge テーブルエンジン +# Merge テーブルエンジン {#merge-table-engine} `Merge` エンジン(`MergeTree` と混同しないでください)は、自身ではデータを保存せず、任意の数の他のテーブルから同時に読み取ることができます。 @@ -17,7 +17,7 @@ doc_type: 'reference' -## テーブルを作成する +## テーブルを作成する {#creating-a-table} ```sql CREATE TABLE ... Engine=Merge(db_name, tables_regexp) @@ -51,7 +51,7 @@ CREATE TABLE ... Engine=Merge(db_name, tables_regexp) -## 例 +## 例 {#examples} **例 1** diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md index 15c23ae1b61..8a3d9a0cfdd 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md @@ -7,7 +7,7 @@ title: 'Null テーブルエンジン' doc_type: 'reference' --- -# Null テーブルエンジン +# Null テーブルエンジン {#null-table-engine} `Null` テーブルにデータを書き込むと、そのデータは無視されます。 `Null` テーブルから読み出すと、レスポンスは空になります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md index 986c831b63b..0cb957d8913 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Set テーブルエンジン +# Set テーブルエンジン {#set-table-engine} :::note ClickHouse Cloud では、サービスが 25.4 より前のバージョンで作成されている場合、`SET compatibility=25.4` を使って互換性を少なくとも 25.4 に設定する必要があります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md index 9b7c997f90c..013a4ca2277 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# URL テーブルエンジン +# URL テーブルエンジン {#url-table-engine} リモートの HTTP/HTTPS サーバーとの間でデータをクエリします。このエンジンは [File](../../../engines/table-engines/special/file.md) エンジンに類似しています。 @@ -51,7 +51,7 @@ doc_type: 'reference' -## 例 +## 例 {#example} **1.** サーバー上に `url_engine_table` テーブルを作成します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md index 0616c593050..16757bf68e1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md @@ -7,6 +7,6 @@ title: 'View テーブルエンジン' doc_type: 'reference' --- -# View テーブルエンジン +# View テーブルエンジン {#view-table-engine} ビューを実装するために使用されます(詳細は `CREATE VIEW query` を参照してください)。データ自体は保存せず、指定された `SELECT` クエリだけを保持します。テーブルから読み取る際には、このクエリを実行し(不要なカラムはクエリからすべて削除され)、結果を返します。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/concurrency.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/concurrency.md index f5e674b2725..c0ea1fbb6b3 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/concurrency.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/concurrency.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['concurrency', 'QPS'] --- -# ClickHouse は高頻度かつ同時実行されるクエリをサポートしていますか? +# ClickHouse は高頻度かつ同時実行されるクエリをサポートしていますか? {#does-clickhouse-support-frequent-concurrent-queries} ClickHouse は、外部ユーザーに直接応答するリアルタイム分析アプリケーション向けに設計されています。ペタバイト規模のデータベースに対して、履歴データとリアルタイム挿入データを組み合わせつつ、低レイテンシ(10 ミリ秒未満)かつ高い同時実行性(1 秒あたり 10,000 クエリ超)で分析クエリを処理できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/cost-based.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/cost-based.md index eaec07b2442..47a30fb7c0a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/cost-based.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/cost-based.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['CBE', 'optimizer'] --- -# ClickHouse はコストベースオプティマイザを持っていますか? +# ClickHouse はコストベースオプティマイザを持っていますか? {#does-clickhouse-have-a-cost-based-optimizer} ClickHouse には、いくつかの限定的な形でコストベース最適化の仕組みがあります。たとえば、カラムの読み取り順序は、ディスクから圧縮されたデータ範囲を読み取るコストによって決定されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/datalake.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/datalake.md index 537a73c76f4..4a1f70b3b9d 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/datalake.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['データレイク', 'レイクハウス'] --- -# ClickHouse はデータレイクをサポートしていますか? +# ClickHouse はデータレイクをサポートしていますか? {#does-clickhouse-support-data-lakes} ClickHouse は Iceberg、Delta Lake、Apache Hudi、Apache Paimon、Hive を含むデータレイクをサポートしています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/dependencies.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/dependencies.md index d38c9d0558f..bd63aae30c8 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/dependencies.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/dependencies.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['dependencies', '3rd-party'] --- -# ClickHouse を実行する際に必要なサードパーティ製の依存関係はありますか? +# ClickHouse を実行する際に必要なサードパーティ製の依存関係はありますか? {#what-are-the-3rd-party-dependencies-for-running-clickhouse} ClickHouse には実行時の依存関係は一切ありません。完全に自己完結した単一バイナリアプリケーションとして配布されており、このアプリケーションだけでクラスタのすべての機能を提供します。クエリの処理に加え、クラスタ内のワーカーノードとして、RAFT コンセンサスアルゴリズムを提供する調整システムとして、さらにはクライアントやローカルクエリエンジンとして動作します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md index 2453ad2a5a9..80ea9d9a9f6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['distributed', 'join'] --- -# ClickHouse は分散 JOIN をサポートしていますか? +# ClickHouse は分散 JOIN をサポートしていますか? {#does-clickhouse-support-distributed-join} ClickHouse はクラスタ上での分散 JOIN をサポートしています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/federated.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/federated.md index 342fad39fe3..06c4db0bb3a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/federated.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/federated.md @@ -8,22 +8,22 @@ doc_type: 'reference' keywords: ['federated', 'hybrid', 'postgres', 'mysql', 'sqlite', 'odbc', 'jdbc'] --- -# ClickHouse はフェデレーテッドクエリをサポートしていますか? +# ClickHouse はフェデレーテッドクエリをサポートしていますか? {#does-clickhouse-support-federated-queries} ClickHouse は、分析用データベースの中でもフェデレーテッドクエリおよびハイブリッドクエリ実行について、最も包括的なサポートを提供します。 ClickHouse は外部データベースに対するクエリをサポートしています: -- PostgreSQL -- MySQL -- MongoDB -- Redis -- 任意の ODBC データソース -- 任意の JDBC データソース -- 任意の Arrow Flight データソース -- Kafka や RabbitMQ などのストリーミングデータソース -- Iceberg、Delta Lake、Apache Hudi、Apache Paimon などの Data Lake -- AWS S3、GCS、Minio、Cloudflare R2、Azure Blob Storage、Alicloud OSS、Tencent COS などの共有ストレージ上にある外部ファイル、ならびにローカルストレージ上の外部ファイル(幅広いデータフォーマットに対応) +* PostgreSQL +* MySQL +* MongoDB +* Redis +* 任意の ODBC データソース +* 任意の JDBC データソース +* 任意の Arrow Flight データソース +* Kafka や RabbitMQ などのストリーミングデータソース +* Iceberg、Delta Lake、Apache Hudi、Apache Paimon などの Data Lake +* AWS S3、GCS、Minio、Cloudflare R2、Azure Blob Storage、Alicloud OSS、Tencent COS などの共有ストレージ上にある外部ファイル、ならびにローカルストレージ上の外部ファイル(幅広いデータフォーマットに対応) ClickHouse は、1 つのクエリ内で複数の異なるデータソースを結合できます。また、ローカルリソースを活用しつつ、クエリの一部をリモートマシンにオフロードするハイブリッドクエリ実行オプションも提供します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/index.md index 0f5aba06085..0fc115e1aa0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/index.md @@ -8,18 +8,18 @@ description: 'ClickHouse に関する一般的な質問をまとめたインデ doc_type: 'landing-page' --- -# ClickHouse に関する一般的な質問 +# ClickHouse に関する一般的な質問 {#general-questions-about-clickhouse} -- [ClickHouse とは何ですか?](../../intro.md) -- [なぜ ClickHouse はそんなに高速なのですか?](../../concepts/why-clickhouse-is-so-fast.mdx) -- [誰が ClickHouse を使っていますか?](../../faq/general/who-is-using-clickhouse.md) -- 「[ClickHouse」はどういう意味ですか?](../../faq/general/dbms-naming.md) -- 「[Не тормозит」はどういう意味ですか?](../../faq/general/ne-tormozit.md) -- [OLAP とは何ですか?](../../faq/general/olap.md) -- [カラム型データベースとは何ですか?](../../faq/general/columnar-database.md) -- [プライマリキーはどのように選べばよいですか?](../../guides/best-practices/sparse-primary-indexes.md) -- [なぜ MapReduce のようなものを使わないのですか?](../../faq/general/mapreduce.md) -- [ClickHouse にコードをコントリビュートするにはどうすればよいですか?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) +* [ClickHouse とは何ですか?](../../intro.md) +* [なぜ ClickHouse はそんなに高速なのですか?](../../concepts/why-clickhouse-is-so-fast.mdx) +* [誰が ClickHouse を使っていますか?](../../faq/general/who-is-using-clickhouse.md) +* 「[ClickHouse」はどういう意味ですか?](../../faq/general/dbms-naming.md) +* 「[Не тормозит」はどういう意味ですか?](../../faq/general/ne-tormozit.md) +* [OLAP とは何ですか?](../../faq/general/olap.md) +* [カラム型データベースとは何ですか?](../../faq/general/columnar-database.md) +* [プライマリキーはどのように選べばよいですか?](../../guides/best-practices/sparse-primary-indexes.md) +* [なぜ MapReduce のようなものを使わないのですか?](../../faq/general/mapreduce.md) +* [ClickHouse にコードをコントリビュートするにはどうすればよいですか?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) :::info お探しの内容が見つかりませんか? [Knowledge Base](/knowledgebase/) を確認し、本ドキュメント内にある多数の有用な記事も参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/sql.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/sql.md index 9bbe1b382ae..c7754898585 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/sql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/sql.md @@ -8,46 +8,46 @@ doc_type: 'reference' keywords: ['SQL 構文', 'ANSI SQL'] --- -# ClickHouse はどのような SQL 構文をサポートしていますか? +# ClickHouse はどのような SQL 構文をサポートしていますか? {#what-sql-syntax-does-clickhouse-support} ClickHouse は SQL 構文を完全にサポートしており、次のような機能が含まれます。 -- SQL/JSON と JSON データ型 (SQL-2023) -- ウィンドウ関数 (SQL-2003) -- 共通テーブル式 (CTE) と再帰クエリ (SQL-1999) -- ROLLUP、CUBE、GROUPING SETS (SQL-1999) -- RBAC の完全サポート (SQL-1999) -- 相関サブクエリ (SQL-1992) +* SQL/JSON と JSON データ型 (SQL-2023) +* ウィンドウ関数 (SQL-2003) +* 共通テーブル式 (CTE) と再帰クエリ (SQL-1999) +* ROLLUP、CUBE、GROUPING SETS (SQL-1999) +* RBAC の完全サポート (SQL-1999) +* 相関サブクエリ (SQL-1992) このサポートは、TPC-H および TPC-DS ベンチマーク、ならびに SQLTest によって検証されています。 ClickHouse は、ISO/IEC によって標準化される以前から、次のような多くの機能を導入してきました。 -- 条件付き集約関数 -- `any` 集約関数 -- `least` と `greatest` -- `GROUP BY ALL` -- エイリアスの拡張的な利用 -- 数値リテラル内でのアンダースコア +* 条件付き集約関数 +* `any` 集約関数 +* `least` と `greatest` +* `GROUP BY ALL` +* エイリアスの拡張的な利用 +* 数値リテラル内でのアンダースコア ClickHouse は、次のような大きな利便性向上機能を導入することで、SQL を拡張しています。 -- エイリアスの自由な利用 -- WITH 句内でのエイリアス -- 集約関数コンビネータ -- パラメータ化された集約関数 -- 近似集約関数 -- ネイティブおよびビッグ整数の数値データ型、拡張精度の decimal -- 配列操作のための高階関数 -- ARRAY JOIN 句および arrayJoin 関数 -- 配列集約 -- LIMIT BY 句 -- GROUP BY WITH TOTALS -- AS OF JOIN -- ANY/ALL JOIN -- JSON の自然な構文 -- カラムリスト末尾のカンマ -- FROM ... SELECT の句順 -- 型安全なクエリパラメータおよびパラメータ化ビュー +* エイリアスの自由な利用 +* WITH 句内でのエイリアス +* 集約関数コンビネータ +* パラメータ化された集約関数 +* 近似集約関数 +* ネイティブおよびビッグ整数の数値データ型、拡張精度の decimal +* 配列操作のための高階関数 +* ARRAY JOIN 句および arrayJoin 関数 +* 配列集約 +* LIMIT BY 句 +* GROUP BY WITH TOTALS +* AS OF JOIN +* ANY/ALL JOIN +* JSON の自然な構文 +* カラムリスト末尾のカンマ +* FROM ... SELECT の句順 +* 型安全なクエリパラメータおよびパラメータ化ビュー これらの一部は、今後の SQL 標準に取り込まれる可能性がありますが、ClickHouse ではすでに利用可能です。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/updates.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/updates.md index d9d508763d6..b0fb47739e4 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/updates.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/general/updates.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['更新', 'リアルタイム'] --- -# ClickHouse はリアルタイム更新をサポートしていますか? +# ClickHouse はリアルタイム更新をサポートしていますか? {#does-clickhouse-support-real-time-updates} ClickHouse は `UPDATE` 文をサポートしており、`INSERT` と同等の速度でリアルタイムに更新を実行できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/index.md index 630c608140d..b7781104e85 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/index.md @@ -8,15 +8,15 @@ description: 'ClickHouse を他のシステムと連携する際の質問を一 doc_type: 'landing-page' --- -# ClickHouse と他のシステムの連携に関する質問 +# ClickHouse と他のシステムの連携に関する質問 {#questions-about-integrating-clickhouse-and-other-systems} -- [ClickHouse からファイルへデータをエクスポートするにはどうすればよいですか?](https://clickhouse.com/docs/knowledgebase/file-export) -- [JSON を ClickHouse にインポートするにはどうすればよいですか?](/integrations/data-ingestion/data-formats/json/intro.md) -- [Kafka を ClickHouse に接続するにはどうすればよいですか?](/integrations/data-ingestion/kafka/index.md) -- [Java アプリケーションを ClickHouse に接続できますか?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) -- [ClickHouse は MySQL のテーブルを読み込むことができますか?](/integrations/mysql) -- [ClickHouse は PostgreSQL のテーブルを読み込むことができますか?](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) -- [ODBC 経由で Oracle に接続するときにエンコーディングの問題が発生する場合はどうすればよいですか?](/faq/integration/oracle-odbc.md) +* [ClickHouse からファイルへデータをエクスポートするにはどうすればよいですか?](https://clickhouse.com/docs/knowledgebase/file-export) +* [JSON を ClickHouse にインポートするにはどうすればよいですか?](/integrations/data-ingestion/data-formats/json/intro.md) +* [Kafka を ClickHouse に接続するにはどうすればよいですか?](/integrations/data-ingestion/kafka/index.md) +* [Java アプリケーションを ClickHouse に接続できますか?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) +* [ClickHouse は MySQL のテーブルを読み込むことができますか?](/integrations/mysql) +* [ClickHouse は PostgreSQL のテーブルを読み込むことができますか?](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) +* [ODBC 経由で Oracle に接続するときにエンコーディングの問題が発生する場合はどうすればよいですか?](/faq/integration/oracle-odbc.md) :::info お探しの情報が見つかりませんか? [Knowledge Base](/knowledgebase/) を確認し、あわせて本ドキュメント内の多数の有用な記事も参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/json-import.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/json-import.md index 8032ffda69c..e72f8bbac96 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/json-import.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/json-import.md @@ -16,7 +16,7 @@ ClickHouse は、[入出力用のさまざまなデータ形式](/interfaces/for -## 例 +## 例 {#examples} [HTTP インターフェース](../../interfaces/http.md)を使用する場合: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md index b89e7ca3b9c..58cffcbf35b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md @@ -8,9 +8,7 @@ doc_type: 'guide' keywords: ['oracle', 'odbc', 'encoding', 'integration', 'external dictionary'] --- - - -# ODBC 経由で Oracle を使用する際に文字コードの問題が発生した場合はどうすればよいですか? +# ODBC 経由で Oracle を使用する際に文字コードの問題が発生した場合はどうすればよいですか? {#oracle-odbc-encodings} Oracle ODBC ドライバを介して ClickHouse の外部ディクショナリのソースとして Oracle を使用する場合は、`/etc/default/clickhouse` 内の `NLS_LANG` 環境変数に正しい値を設定する必要があります。詳細については、[Oracle NLS_LANG FAQ](https://www.oracle.com/technetwork/products/globalization/nls-lang-099431.html) を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md index 56b805d23d6..4aaf57e72f4 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md @@ -30,7 +30,7 @@ TTL は、データを [/dev/null](https://en.wikipedia.org/wiki/Null_device) -## DELETE FROM +## DELETE FROM {#delete-from} [DELETE FROM](/sql-reference/statements/delete.md) を使用すると、ClickHouse で標準的な DELETE クエリを実行できます。フィルター句で対象となった行は削除済みとしてマークされ、今後の結果セットには含まれません。行のクリーンアップは非同期に行われます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/index.md index ea1ddd01eed..577fb6d3ca3 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/operations/index.md @@ -8,16 +8,16 @@ doc_type: 'landing-page' keywords: ['運用', '管理', 'デプロイメント', 'クラスター管理', 'FAQ'] --- -# ClickHouse サーバーおよびクラスターの運用に関する質問 +# ClickHouse サーバーおよびクラスターの運用に関する質問 {#question-about-operating-clickhouse-servers-and-clusters} -- [本番環境ではどの ClickHouse バージョンを使用すべきですか?](/faq/operations/production.md) -- [ストレージとコンピュートを分離した構成で ClickHouse をデプロイできますか?](/faq/operations/separate_storage.md) -- [ClickHouse テーブルから古いレコードを削除できますか?](/faq/operations/delete-old-data.md) -- [ClickHouse Keeper はどのように構成すればよいですか?](/guides/sre/keeper/index.md) -- [ClickHouse を LDAP と連携できますか?](/guides/sre/user-management/configuring-ldap.md) -- [ClickHouse でユーザー、ロール、権限をどのように設定すればよいですか?](/guides/sre/user-management/index.md) -- [ClickHouse で行の更新や削除はできますか?](/guides/starter_guides/mutations.md) -- [ClickHouse はマルチリージョンレプリケーションをサポートしていますか?](/faq/operations/multi-region-replication.md) +* [本番環境ではどの ClickHouse バージョンを使用すべきですか?](/faq/operations/production.md) +* [ストレージとコンピュートを分離した構成で ClickHouse をデプロイできますか?](/faq/operations/separate_storage.md) +* [ClickHouse テーブルから古いレコードを削除できますか?](/faq/operations/delete-old-data.md) +* [ClickHouse Keeper はどのように構成すればよいですか?](/guides/sre/keeper/index.md) +* [ClickHouse を LDAP と連携できますか?](/guides/sre/user-management/configuring-ldap.md) +* [ClickHouse でユーザー、ロール、権限をどのように設定すればよいですか?](/guides/sre/user-management/index.md) +* [ClickHouse で行の更新や削除はできますか?](/guides/starter_guides/mutations.md) +* [ClickHouse はマルチリージョンレプリケーションをサポートしていますか?](/faq/operations/multi-region-replication.md) :::info お探しの内容が見つかりませんか? [Knowledge Base](/knowledgebase/) をご覧いただき、このドキュメント内にある多数の有用な記事もあわせて参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/faq/use-cases/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/faq/use-cases/index.md index f87d9568bb3..d3df69feefd 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/faq/use-cases/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/faq/use-cases/index.md @@ -8,10 +8,10 @@ keywords: ['ClickHouse use cases', '時系列データベース', 'キーバリ doc_type: 'landing-page' --- -# ClickHouse のユースケースに関する質問 +# ClickHouse のユースケースに関する質問 {#questions-about-clickhouse-use-cases} -- [ClickHouse を時系列データベースとして利用できますか?](/knowledgebase/time-series) -- [ClickHouse をキーバリューストレージとして利用できますか?](/knowledgebase/key-value) +* [ClickHouse を時系列データベースとして利用できますか?](/knowledgebase/time-series) +* [ClickHouse をキーバリューストレージとして利用できますか?](/knowledgebase/key-value) :::info お探しの情報が見つかりませんか? [Knowledge Base](/knowledgebase/) を確認し、本ドキュメント内にある多数の役立つ記事も参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md index 538ef960540..6a3e3d66220 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md @@ -11,11 +11,11 @@ keywords: ['Amazon レビュー', 'カスタマーレビュー データセッ :::note 以下のクエリは、**Production** 環境の ClickHouse Cloud インスタンス上で実行されています。詳細については -["Playground specifications"](/getting-started/playground#specifications) +["Playground specifications"](/getting-started/playground#specifications) を参照してください。 ::: -## データセットの読み込み +## データセットの読み込み {#loading-the-dataset} 1. データを ClickHouse に挿入しなくても、元の場所に対して直接クエリを実行できます。どのようなデータか確認するために、いくつか行を取得してみましょう。 @@ -123,7 +123,6 @@ FROM s3Cluster('default', 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/amazon_reviews/amazon_reviews_*.snappy.parquet') ``` - :::tip ClickHouse Cloud では、クラスター名は `default` です。`default` を環境のクラスター名に置き換えてください。クラスターがない場合は、`s3Cluster` の代わりに `s3` テーブル関数を使用してください。 ::: @@ -153,8 +152,7 @@ ORDER BY size DESC 元のデータは約 70G ありましたが、ClickHouse で圧縮されると約 30G に収まります。 - -## クエリ例 +## クエリ例 {#example-queries} 7. クエリをいくつか実行してみましょう。こちらは、このデータセットで最も役に立ったレビューの上位 10 件です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md index eb779872fed..d023dd1cafe 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md @@ -7,7 +7,7 @@ title: '匿名化されたウェブアナリティクス' doc_type: 'guide' --- -# 匿名化されたウェブ解析データ +# 匿名化されたウェブ解析データ {#anonymized-web-analytics-data} このデータセットは、匿名化されたウェブ解析データを含む 2 つのテーブルで構成されており、ヒット(`hits_v1`)と訪問(`visits_v1`)のデータが含まれています。 @@ -15,17 +15,17 @@ doc_type: 'guide' ## データをダウンロードして取り込む {#download-and-ingest-the-data} -### hits の圧縮 TSV ファイルをダウンロードする +### hits の圧縮 TSV ファイルをダウンロードする {#download-the-hits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/hits/tsv/hits_v1.tsv.xz | unxz --threads=`nproc` > hits_v1.tsv -# チェックサムを検証する +# チェックサムを検証する {#validate-the-checksum} md5sum hits_v1.tsv -# チェックサムは次の値と一致する必要があります: f3631b6295bf06989c1437491f7592cb +# チェックサムは次の値と一致する必要があります: f3631b6295bf06989c1437491f7592cb {#checksum-should-be-equal-to-f3631b6295bf06989c1437491f7592cb} ``` -### データベースとテーブルを作成 +### データベースとテーブルを作成 {#create-the-database-and-table} ```bash clickhouse-client --query "CREATE DATABASE IF NOT EXISTS datasets" @@ -45,7 +45,7 @@ clickhouse-client --query="CREATE TABLE default.hits_100m_obfuscated (WatchID UI ``` -### hits データをインポートする +### hits データをインポートする {#import-the-hits-data} ```bash cat hits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.hits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -62,13 +62,13 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.hits_v1" ``` -### 「visits」の圧縮 TSV ファイルをダウンロード +### 「visits」の圧縮 TSV ファイルをダウンロード {#download-the-visits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/visits/tsv/visits_v1.tsv.xz | unxz --threads=`nproc` > visits_v1.tsv -# チェックサムを検証する +# チェックサムを検証する {#validate-the-checksum} md5sum visits_v1.tsv -# チェックサムは次の値と一致する必要があります: 6dafe1a0f24e59e3fc2d0fed85601de6 +# チェックサムは次の値と一致する必要があります: 6dafe1a0f24e59e3fc2d0fed85601de6 {#checksum-should-be-equal-to-6dafe1a0f24e59e3fc2d0fed85601de6} ``` @@ -78,7 +78,7 @@ md5sum visits_v1.tsv clickhouse-client --query "CREATE TABLE datasets.visits_v1 ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), Goals Nested(ID UInt32, Serial UInt32, EventTime DateTime, Price Int64, OrderID String, CurrencyID UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, TraficSource Nested(ID Int8, SearchEngineID UInt16, AdvEngineID UInt8, PlaceID UInt16, SocialSourceNetworkID UInt8, Domain String, SearchPhrase String, SocialSourcePage String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), Market Nested(Type UInt8, GoalID UInt32, OrderID String, OrderPrice Int64, PP UInt32, DirectPlaceID UInt32, DirectOrderID UInt32, DirectBannerID UInt32, GoodID String, GoodName String, GoodQuantity Int32, GoodPrice Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192" ``` -### visits データを取り込む +### visits データを取り込む {#import-the-visits-data} ```bash cat visits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.visits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -95,7 +95,7 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.visits_v1" ``` -## JOIN の例 +## JOIN の例 {#an-example-join} hits と visits のデータセットは ClickHouse のテストルーチンで使用されており、これはテストスイートに含まれるクエリの 1 つです。残りのテストは、このページの最後にある *次のステップ* セクションを参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md index 255aca035a4..801f78df415 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md @@ -94,8 +94,7 @@ clickhouse-client --query "INSERT INTO mgbench.logs2 FORMAT CSVWithNames" < mgbe clickhouse-client --query "INSERT INTO mgbench.logs3 FORMAT CSVWithNames" < mgbench3.csv ``` - -## ベンチマーク用クエリを実行する +## ベンチマーク用クエリを実行する {#run-benchmark-queries} ```sql USE mgbench; @@ -238,7 +237,6 @@ ORDER BY machine_name, dt; ``` - ```sql -- Q1.6: すべてのファイルサーバーにおける1時間あたりの総ネットワークトラフィック量は? diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md index b9cfb96e712..edbab07a3a5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md @@ -7,15 +7,15 @@ keywords: ['基地局データ', 'ジオデータ', 'OpenCelliD', '地理空間 doc_type: 'guide' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import ActionsMenu from '@site/docs/_snippets/_service_actions_menu.md'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; -import SupersetDocker from '@site/docs/_snippets/_add_superset_detail.md'; +import ActionsMenu from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_service_actions_menu.md'; +import SQLConsoleDetail from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; +import SupersetDocker from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_add_superset_detail.md'; import cloud_load_data_sample from '@site/static/images/_snippets/cloud-load-data-sample.png'; import cell_towers_1 from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' import add_a_database from '@site/static/images/getting-started/example-datasets/superset-add.png' @@ -29,7 +29,6 @@ import superset_radio_umts from '@site/static/images/getting-started/example-dat import superset_umts_netherlands from '@site/static/images/getting-started/example-datasets/superset-umts-netherlands.png' import superset_cell_tower_dashboard from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' - ## 目標 {#goal} このガイドでは次のことを学びます。 @@ -124,7 +123,7 @@ INSERT INTO cell_towers SELECT * FROM s3('https://datasets-documentation.s3.amaz -## サンプルクエリをいくつか実行する +## サンプルクエリをいくつか実行する {#examples} 1. 種類別のセルタワー数: @@ -171,7 +170,6 @@ SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10 これらの値をデコードするために、ClickHouse で [Dictionary](../../sql-reference/dictionaries/index.md) を作成することを検討してもよいでしょう。 - ## ユースケース:地理データの活用 {#use-case} [`pointInPolygon`](/sql-reference/functions/geo/coordinates.md/#pointinpolygon) 関数を使用します。 @@ -266,7 +264,6 @@ WHERE pointInPolygon((lon, lat), (SELECT * FROM moscow)) 1 rows in set. Elapsed: 0.067 sec. Processed 43.28 million rows, 692.42 MB (645.83 million rows/s., 10.33 GB/s.) ``` - ## スキーマの確認 {#review-of-the-schema} Superset で可視化を作成する前に、使用するカラムを確認しておきましょう。このデータセットは主に、世界中の携帯電話基地局の位置情報(経度と緯度)と無線方式(radio types)の種類を提供します。カラムの説明は [community forum](https://community.opencellid.org/t/documenting-the-columns-in-the-downloadable-cells-database-csv/186) にあります。これから作成する可視化で使用するカラムについては、以下で説明します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md index 6003bffc61f..a7459c0d78d 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md @@ -15,7 +15,7 @@ doc_type: 'guide' このデータセットには、[huggingface.co](https://huggingface.co/api/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M/parquet/default/train/) 上にある 26 個の `Parquet` ファイルが含まれています。ファイル名は `0.parquet`、`1.parquet`、…、`25.parquet` です。データセットのサンプル行を確認するには、この [Hugging Face ページ](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M) を参照してください。 -## テーブルを作成する +## テーブルを作成する {#create-table} 記事 ID、タイトル、テキスト、および埋め込みベクトルを格納する `dbpedia` テーブルを作成します: @@ -31,7 +31,7 @@ CREATE TABLE dbpedia ``` -## テーブルの読み込み +## テーブルの読み込み {#load-table} すべての Parquet ファイルからデータセットを読み込むには、次のシェルコマンドを実行します。 @@ -74,7 +74,7 @@ FROM dbpedia _最近傍(nearest neighbours)_ とは、ユーザーのクエリに関連する結果となる文書、画像、その他のコンテンツを指します。 取得された結果は、生成 AI アプリケーションにおける検索拡張生成(Retrieval Augmented Generation, RAG)の重要な入力となります。 -## 総当たり方式でベクトル類似度検索を実行する +## 総当たり方式でベクトル類似度検索を実行する {#run-a-brute-force-vector-similarity-search} KNN(k-Nearest Neighbours)検索、または総当たり(ブルートフォース)検索では、データセット内の各ベクトルと 検索用埋め込みベクトルとの距離を計算し、その距離を並べ替えて最も近いベクトル(近傍)を取得します。`dbpedia` データセットでは、 @@ -119,7 +119,7 @@ LIMIT 20 また、実際の計算リソース使用量とストレージ帯域幅使用量を把握するため、OS ファイルキャッシュがコールドな状態および `max_threads=1` の条件でのクエリレイテンシも記録してください(それを基に、数百万ベクトル規模の本番データセットでの値を外挿します)。 -## ベクトル類似度インデックスを作成する +## ベクトル類似度インデックスを作成する {#build-vector-similarity-index} `vector` 列にベクトル類似度インデックスを定義・作成するには、次の SQL を実行します。 @@ -134,7 +134,7 @@ ALTER TABLE dbpedia MATERIALIZE INDEX vector_index SETTINGS mutations_sync = 2; インデックスの構築および保存処理には、利用可能な CPU コア数やストレージ帯域幅によっては数分かかる場合があります。 -## ANN 検索を実行する +## ANN 検索を実行する {#perform-ann-search} *Approximate Nearest Neighbours*(ANN、近似最近傍)とは、正確なベクトル検索よりもはるかに高速に結果を取得できる一群の手法(たとえば、グラフやランダムフォレストのような専用データ構造)を指します。結果の精度は、実用上はたいてい「十分よい」レベルです。多くの近似手法では、結果精度と検索時間のトレードオフを調整するためのパラメータが提供されています。 @@ -181,7 +181,7 @@ LIMIT 20 ``` -## 検索クエリ用の埋め込みベクトルの生成 +## 検索クエリ用の埋め込みベクトルの生成 {#generating-embeddings-for-search-query} これまでに見てきた類似度検索クエリでは、`dbpedia` テーブル内に既に存在するベクトルの1つを検索ベクトルとして使用していました。実際のアプリケーションでは、検索ベクトルは、自然言語で記述されたものを含むユーザーの入力クエリに対して生成する必要があります。検索ベクトルは、データセット用の埋め込みベクトルを生成する際に使用したものと同じ LLM モデルを使って生成しなければなりません。 @@ -223,7 +223,7 @@ while True: ``` -## Q&A デモアプリケーション +## Q&A デモアプリケーション {#q-and-a-demo-application} 上記の例では、ClickHouse を用いたセマンティック検索とドキュメント検索を示しました。次に紹介するのは、非常にシンプルでありながら高い可能性を持つ生成 AI のサンプルアプリケーションです。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md index 6fb55bd0f42..2868e4a99aa 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md @@ -23,7 +23,7 @@ Foursquare によるこのデータセットは、[ダウンロード](https://d 1 億件以上のレコードが含まれています。さらに、それらの場所に関するカテゴリやソーシャルメディア情報といった 追加のメタデータも含まれています。 -## データ探索 +## データ探索 {#data-exploration} データ探索には、[`clickhouse-local`](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) を使用します。これはフル機能の ClickHouse エンジンを提供する軽量なコマンドラインツールですが、代わりに ClickHouse Cloud や `clickhouse-client`、あるいは `chDB` を使用することもできます。 @@ -147,7 +147,7 @@ DESCRIBE s3('s3://fsq-os-places-us-east-1/release/dt=2025-04-08/places/parquet/* ``` -## データを ClickHouse に取り込む +## データを ClickHouse に取り込む {#loading-the-data} データをディスクに永続化したい場合は、`clickhouse-server` または ClickHouse Cloud を使用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md index e97039f9309..ae76e5c4ebf 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md @@ -29,7 +29,7 @@ import superset_authors_matrix_v2 from '@site/static/images/getting-started/exam * `line_changes` - 2.7G - 7,535,157 行 -## データの生成 +## データの生成 {#generating-the-data} この手順は任意です。データは無償で配布しています。「[データのダウンロードと挿入](#downloading-and-inserting-the-data)」を参照してください。 @@ -73,7 +73,7 @@ CREATE TABLE git.commits * Linux - `~/clickhouse git-import` - 160分 -## データのダウンロードと挿入 +## データのダウンロードと挿入 {#downloading-and-inserting-the-data} 以下のデータを使用すると、動作環境を再現できます。また、このデータセットは play.clickhouse.com からも利用できます。詳しくは [Queries](#queries) を参照してください。 @@ -220,7 +220,7 @@ FROM s3('https://datasets-documentation.s3.amazonaws.com/github/commits/clickhou このデータセットは、`git_clickhouse` データベース内にある [play.clickhouse.com](https://sql.clickhouse.com?query_id=DCQPNPAIMAQXRLHYURLKVJ) で利用できます。すべてのクエリについて、この環境へのリンクを、必要に応じてデータベース名を調整したうえで提供します。なお、データ収集時期の違いにより、play.clickhouse.com 上での結果は、ここで示すものと異なる場合があります。 -### 単一ファイルの履歴 +### 単一ファイルの履歴 {#history-of-a-single-file} 最も単純なクエリです。ここでは `StorageReplicatedMergeTree.cpp` のすべてのコミットメッセージを確認します。これらのほうが興味深いと考えられるため、最新のメッセージが先頭に来るように並べ替えます。 @@ -296,7 +296,7 @@ LIMIT 10 ファイル名の変更も考慮して、[ファイルの行ごとのコミット履歴](#line-by-line-commit-history-of-a-file)を取得する、より複雑な形のクエリも存在します。 -### 現在アクティブなファイルを特定する +### 現在アクティブなファイルを特定する {#find-the-current-active-files} これは、リポジトリ内の現在のファイルだけを対象に分析したい後続の処理において重要です。ここでは、「名前変更も削除もされておらず(その後に再追加/再リネームされていない)ファイル」の集合として推定します。 @@ -428,7 +428,7 @@ git ls-files | grep -v -E 'generated\.cpp|^(contrib|docs?|website|libs/(libcityh これらの差分は、私たちの分析に本質的な影響を与えることはないはずです。**このクエリの改善版をぜひお寄せください**。 -### 変更数が最も多いファイルを一覧表示する +### 変更数が最も多いファイルを一覧表示する {#list-files-with-most-modifications} 現在のファイルに限定すると、変更数は削除数と追加数の合計とみなします。 @@ -484,7 +484,7 @@ LIMIT 10 ``` -### コミットは通常、週のどの曜日に行われることが多いですか? +### コミットは通常、週のどの曜日に行われることが多いですか? {#what-day-of-the-week-do-commits-usually-occur} [play](https://sql.clickhouse.com?query_id=GED2STFSYJDRAA59H8RLIV) @@ -510,7 +510,7 @@ GROUP BY dayOfWeek(time) AS day_of_week 金曜日には生産性が少し落ちることを考えると、これは納得のいく結果です。週末にもコードをコミットしてくれている人がいるのは素晴らしいですね。貢献してくださっている皆さん、本当にありがとうございます! -### サブディレクトリ/ファイルの履歴 - 行数、コミット数、コントリビューター数の推移 +### サブディレクトリ/ファイルの履歴 - 行数、コミット数、コントリビューター数の推移 {#history-of-subdirectoryfile---number-of-lines-commits-and-contributors-over-time} このクエリは、フィルタしない場合、非常に大きな結果セットとなり、現実的には表示や可視化が困難です。そこで、次の例ではファイルまたはサブディレクトリでフィルタリングできるようにしています。ここでは `toStartOfWeek` 関数を使って週ごとにグルーピングしていますが、必要に応じて調整してください。 @@ -555,7 +555,7 @@ LIMIT 10 コミットおよび作者 -### 著者数が最も多いファイルを一覧表示する +### 著者数が最も多いファイルを一覧表示する {#list-files-with-maximum-number-of-authors} 現在のファイルのみを対象とします。 @@ -611,7 +611,7 @@ LIMIT 10 ``` -### リポジトリ内で最も古いコード行 +### リポジトリ内で最も古いコード行 {#oldest-lines-of-code-in-the-repository} 現在存在するファイルのみが対象です。 @@ -669,7 +669,7 @@ LIMIT 10 ``` -### 履歴が最も長いファイル +### 履歴が最も長いファイル {#files-with-longest-history} 現在存在するファイルのみを対象とします。 @@ -728,7 +728,7 @@ LIMIT 10 コアとなるデータ構造である MergeTree は、言うまでもなく、長年にわたる数多くの改良を経て、今もなお進化し続けています。 -### 月内におけるドキュメントとコード別のコントリビューター分布 +### 月内におけるドキュメントとコード別のコントリビューター分布 {#distribution-of-contributors-with-respect-to-docs-and-code-over-the-month} **データ取得時には、コミット履歴が非常に入り組んでいたため `docs/` フォルダに対する変更を除外しています。そのため、このクエリから得られる結果は正確ではありません。** @@ -792,7 +792,7 @@ FROM 月末にかけてやや多くなるかもしれませんが、全体としては概ね均等に分布しています。とはいえ、データ挿入時に `docs` フィルタで絞り込んでいるため、この結果の信頼性は高くありません。 -### 最も多様な貢献をしている著者 +### 最も多様な貢献をしている著者 {#authors-with-the-most-diverse-impact} ここでいう「多様性」とは、ある著者が貢献したユニークなファイル数を指します。 @@ -870,7 +870,7 @@ LIMIT 10 ``` -### 著者のお気に入りファイル +### 著者のお気に入りファイル {#favorite-files-for-an-author} ここでは創業者の [Alexey Milovidov](https://github.com/alexey-milovidov) を選択し、分析対象を現行のファイルに限定します。 @@ -957,7 +957,7 @@ LIMIT 10 こちらの方が、彼の関心のある分野をより適切に反映しているかもしれません。 -### 著者数が最も少ない巨大ファイル +### 著者数が最も少ない巨大ファイル {#largest-files-with-lowest-number-of-authors} このためには、まず最大サイズのファイルを特定する必要があります。コミット履歴からすべてのファイルについて完全なファイル再構成を行ってサイズを見積もるのは、非常に高コストです。 @@ -1130,7 +1130,7 @@ LIMIT 10 ``` -### 時間帯別、曜日別、作者別、特定サブディレクトリ別のコミット数とコード行数の分布 +### 時間帯別、曜日別、作者別、特定サブディレクトリ別のコミット数とコード行数の分布 {#commits-and-lines-of-code-distribution-by-time-by-weekday-by-author-for-specific-subdirectories} ここでは、これを曜日ごとの追加行数と削除行数として解釈します。この例では、[Functions ディレクトリ](https://github.com/ClickHouse/ClickHouse/tree/master/src/Functions)に注目します。 @@ -1258,7 +1258,7 @@ FROM ``` -### 著者同士が互いのコードを書き換える傾向を示すマトリクス +### 著者同士が互いのコードを書き換える傾向を示すマトリクス {#matrix-of-authors-that-shows-what-authors-tends-to-rewrite-another-authors-code} `sign = -1` はコードの削除を示します。句読点や空行の挿入は除外しています。 @@ -1313,7 +1313,7 @@ Alexey は明らかに他人のコードを削除するのが好きなようで Superset authors matrix v2 -### 曜日ごとに最も高い割合でコミットしているのは誰か? +### 曜日ごとに最も高い割合でコミットしているのは誰か? {#who-is-the-highest-percentage-contributor-per-day-of-week} コミット数だけで見ると: @@ -1514,7 +1514,7 @@ LIMIT 5 BY root ``` -### ある著者のコードのうち、どれだけの割合が他の著者によって削除されたか? +### ある著者のコードのうち、どれだけの割合が他の著者によって削除されたか? {#what-percentage-of-code-for-an-author-has-been-removed-by-other-authors} このクエリでは、特定の著者が作成した行数を、その著者のコードのうち他のコントリビューターによって削除された行数の合計で割った値が必要です。 @@ -1565,7 +1565,7 @@ LIMIT 10 ``` -### 最も多く書き換えられたファイルを一覧表示するには? +### 最も多く書き換えられたファイルを一覧表示するには? {#list-files-that-were-rewritten-most-number-of-times} この問いに対する最も単純なアプローチは、(現在存在するファイルに限定して)パスごとの行の変更回数を数えることです。例: @@ -1708,7 +1708,7 @@ LIMIT 10 ``` -### どの曜日に追加されたコードが最もリポジトリ内に残りやすいか? +### どの曜日に追加されたコードが最もリポジトリ内に残りやすいか? {#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository} このためには、コードの各行を一意に識別する必要があります。同じ行がファイル内に複数回現れる可能性があるため、パスと行内容の組み合わせで推定します。 @@ -1771,7 +1771,7 @@ GROUP BY dayOfWeek(added_day) AS day_of_week_added ``` -### 平均コード年齢でソートされたファイル +### 平均コード年齢でソートされたファイル {#files-sorted-by-average-code-age} このクエリは、[リポジトリに最も長く残りやすい曜日はいつか](#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository) と同じ原理を用い、パスと行の内容によってコード行を一意に識別することを目指します。 これにより、あるコード行が追加されてから削除されるまでの時間を特定できます。ただし対象は現在存在するファイルおよびコードのみに絞り込み、各ファイルについて行ごとの時間を平均します。 @@ -1862,7 +1862,7 @@ LIMIT 10 ``` -### 誰がより多くのテスト / C++ コード / コメントを書く傾向があるのか? +### 誰がより多くのテスト / C++ コード / コメントを書く傾向があるのか? {#who-tends-to-write-more-tests--cpp-code--comments} この問いにはいくつかのアプローチがあります。コードとテストの比率に着目すると、このクエリは比較的単純で、`tests` を含むフォルダへのコントリビューション数を数え、それを全コントリビューション数で割って比率を算出します。 @@ -1996,7 +1996,7 @@ LIMIT 10 コードへの貢献数でソートしている点に注意してください。上位の主要なコントリビューターはいずれも、驚くほど高い割合を占めており、これがコードの可読性の高さにもつながっています。 -### コードとコメントの割合という観点で、ある作者のコミットは時間とともにどのように変化するでしょうか? +### コードとコメントの割合という観点で、ある作者のコミットは時間とともにどのように変化するでしょうか? {#how-does-an-authors-commits-change-over-time-with-respect-to-codecomments-percentage} 作者ごとにこの値を計算するのは容易です。 @@ -2114,7 +2114,7 @@ LIMIT 20 励みになることに、コメント率はほぼ一定で、著者が長期間にわたって貢献しても低下していません。 -### コードが書き換えられるまでの平均時間と中央値(コード劣化の半減期)はどのくらいか? +### コードが書き換えられるまでの平均時間と中央値(コード劣化の半減期)はどのくらいか? {#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay} [最も多く書き換えられた、あるいは最も多くの著者によって編集されたファイルを一覧表示する](#list-files-that-were-rewritten-most-number-of-times)ときと同じ原理を、すべてのファイルを対象にして書き換えを特定するために使うことができます。各ファイルについて、書き換えの間隔を算出するためにウィンドウ関数を使用します。そこから、すべてのファイルにわたる平均値と中央値を算出できます。 @@ -2175,7 +2175,7 @@ FROM rewrites ``` -### 将来書き直される可能性が最も高いという観点で、コードを書くのに最悪なタイミングはいつでしょうか? +### 将来書き直される可能性が最も高いという観点で、コードを書くのに最悪なタイミングはいつでしょうか? {#what-is-the-worst-time-to-write-code-in-sense-that-the-code-has-highest-chance-to-be-re-written} [What is the average time before code will be rewritten and the median (half-life of code decay)?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) および [List files that were rewritten most number of time or by most of authors](#list-files-that-were-rewritten-most-number-of-times) と同様ですが、ここでは集計単位を曜日としています。必要に応じて、たとえば月ごとなどに調整してください。 @@ -2240,7 +2240,7 @@ GROUP BY dayOfWeek ``` -### どの著者のコードが最も「定着」しているか? +### どの著者のコードが最も「定着」しているか? {#which-authors-code-is-the-most-sticky} ここでは「sticky(定着度)」を、「著者のコードが書き換えられるまでどれくらいの期間残り続けるか」という意味で定義します。前の質問 [コードが書き換えられるまでの平均時間と中央値(コード崩壊の半減期)は?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) と同様に、書き換えの指標として、ファイルに対する 50% の追加と 50% の削除を用います。著者ごとに平均書き換え時間を算出し、2 ファイルより多くのファイルに貢献しているコントリビューターのみを対象とします。 @@ -2318,7 +2318,7 @@ LIMIT 10 ``` -### 著者ごとの連続コミット日数の最大値 +### 著者ごとの連続コミット日数の最大値 {#most-consecutive-days-of-commits-by-an-author} このクエリではまず、著者がコミットを行った日付を算出する必要があります。ウィンドウ関数を使用し、著者ごとにパーティション分割することで、コミット間の日数を計算します。各コミットについて、前回のコミットからの経過日数が1日であれば連続 (1)、それ以外は0としてマークし、その結果を `consecutive_day` に保存します。 @@ -2374,7 +2374,7 @@ LIMIT 10 ``` -### ファイルの行ごとのコミット履歴 +### ファイルの行ごとのコミット履歴 {#line-by-line-commit-history-of-a-file} ファイルはリネームされることがあります。この場合、リネームイベントが発生し、その際に `path` カラムにはファイルの新しいパスが、`old_path` には以前の場所が設定されます。例えば次のとおりです。 @@ -2455,7 +2455,7 @@ FORMAT PrettyCompactMonoBlock ## 未解決の問題 {#unsolved-questions} -### Git blame +### Git blame {#git-blame} これは、配列関数内で現在は状態を保持できないため、厳密な結果を得るのが特に難しいケースです。各イテレーションで状態を保持できるようにする `arrayFold` や `arrayReduce` が利用可能になれば、これが実現できるようになります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md index 73915053ac5..da1fba68b4b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['サンプルデータセット', 'Hacker News', 'サンプルデータ', 'テキスト分析', 'ベクトル検索'] --- -# Hacker News データセット +# Hacker News データセット {#hacker-news-dataset} > このチュートリアルでは、Hacker News のデータ 2,800 万行を、CSV および Parquet 形式から ClickHouse > のテーブルに挿入し、そのデータを探索するための簡単なクエリを実行します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md index 2dd6fb3a6b3..73b90f05945 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md @@ -11,7 +11,7 @@ keywords: ['サンプルデータセット', 'laion', '画像埋め込み', 'サ このデータセットには、画像の URL、画像および画像キャプションそれぞれの埋め込みベクトル、画像と画像キャプション間の類似度スコアに加え、画像の幅/高さ、ライセンス、NSFW フラグといったメタデータが含まれます。このデータセットを使って、ClickHouse における[近似最近傍検索](../../engines/table-engines/mergetree-family/annindexes.md)を実演できます。 -## データ準備 +## データ準備 {#data-preparation} 生データでは、埋め込みとメタデータは別々のファイルに保存されています。データ準備のステップでは、データをダウンロードしてファイルを結合し、 CSV に変換して ClickHouse にインポートします。そのために、次の `download.sh` スクリプトを使用できます。 @@ -40,28 +40,28 @@ npy_file = "img_emb_" + str_i + '.npy' metadata_file = "metadata_" + str_i + '.parquet' text_npy = "text_emb_" + str_i + '.npy' -# 全ファイルを読み込み +# 全ファイルを読み込み {#load-all-files} im_emb = np.load(npy_file) text_emb = np.load(text_npy) data = pd.read_parquet(metadata_file) -# ファイルを結合 +# ファイルを結合 {#combine-files} data = pd.concat([data, pd.DataFrame({"image_embedding" : [*im_emb]}), pd.DataFrame({"text_embedding" : [*text_emb]})], axis=1, copy=False) -# ClickHouseへインポートする列 +# ClickHouseへインポートする列 {#columns-to-be-imported-into-clickhouse} data = data[['url', 'caption', 'NSFW', 'similarity', "image_embedding", "text_embedding"]] -# np.arraysをリストへ変換 +# np.arraysをリストへ変換 {#transform-nparrays-to-lists} data['image_embedding'] = data['image_embedding'].apply(lambda x: x.tolist()) data['text_embedding'] = data['text_embedding'].apply(lambda x: x.tolist()) -# captionに様々な引用符が含まれる場合があるため、この回避策が必要 +# captionに様々な引用符が含まれる場合があるため、この回避策が必要 {#this-small-hack-is-needed-because-caption-sometimes-contains-all-kind-of-quotes} data['caption'] = data['caption'].apply(lambda x: x.replace("'", " ").replace('"', " ")) -# データをCSVファイルとしてエクスポート +# データをCSVファイルとしてエクスポート {#export-data-as-csv-file} data.to_csv(str_i + '.csv', header=False) -# 生データファイルを削除 +# 生データファイルを削除 {#removed-raw-data-files} os.system(f"rm {npy_file} {metadata_file} {text_npy}") ``` @@ -76,7 +76,7 @@ seq 0 409 | xargs -P1 -I{} bash -c './download.sh {}' (上記の Python スクリプトは非常に遅く(1 ファイルあたり約 2〜10 分)、大量のメモリを消費し(1 ファイルあたり 41 GB)、生成される CSV ファイルも大きい(各 10 GB)ため、注意してください。十分な RAM がある場合は、より高い並列度を得るために `-P1` の値を増やしてください。これでもまだ遅い場合は、より良いインジェスト手順を検討してください。たとえば .npy ファイルを Parquet に変換してから、残りの処理をすべて ClickHouse で行うなどです。) -## テーブルを作成する +## テーブルを作成する {#create-table} 最初にインデックスなしでテーブルを作成するには、次を実行します。 @@ -105,7 +105,7 @@ INSERT INTO laion FROM INFILE '{path_to_csv_files}/*.csv' `id` 列はあくまで例示用のものであり、スクリプトによって一意ではない値が入力されている点に注意してください。 -## 総当たり方式でベクトル類似度検索を実行する +## 総当たり方式でベクトル類似度検索を実行する {#run-a-brute-force-vector-similarity-search} 総当たり方式の近似ベクトル検索を実行するには、次を実行します。 @@ -137,7 +137,7 @@ SELECT url, caption FROM laion ORDER BY cosineDistance(image_embedding, {target: ``` -## ベクトル類似度インデックスを使って近似ベクトル類似検索を実行する +## ベクトル類似度インデックスを使って近似ベクトル類似検索を実行する {#run-an-approximate-vector-similarity-search-with-a-vector-similarity-index} ここでは、テーブルに 2 つのベクトル類似度インデックスを定義します。 @@ -194,7 +194,7 @@ HNSW インデックスは、HNSW パラメータを慎重に選定し、イン 通常は、新しい画像や新しい画像キャプションに対して埋め込みを作成し、そのデータ内で類似する画像/画像キャプションのペアを検索します。[UDF](/sql-reference/functions/udf) を使用すると、クライアント環境を離れることなく `target` ベクトルを作成できます。データ作成時と検索用の新しい埋め込みを生成する際には、同じモデルを使用することが重要です。以下のスクリプトは、データセットの基盤にもなっている `ViT-B/32` モデルを利用します。 -### テキスト埋め込み +### テキスト埋め込み {#text-embeddings} まず、次の Python スクリプトを ClickHouse のデータパス配下にある `user_scripts/` ディレクトリに保存し、実行可能にします(`chmod +x encode_text.py`)。 @@ -256,7 +256,7 @@ LIMIT 10 `encode_text()` UDF 自体が計算を行い埋め込みベクトルを出力するまでに、数秒かかる場合があることに注意してください。 -### 画像埋め込み +### 画像埋め込み {#image-embeddings} 画像埋め込みも同様に作成できるように、ローカルにファイルとして保存されている画像の埋め込みを生成するための Python スクリプトを用意しています。 @@ -304,7 +304,7 @@ if __name__ == '__main__': 検索用のサンプル画像を取得: ```shell -# LEGOセットのランダムな画像を取得 +# LEGOセットのランダムな画像を取得 {#get-a-random-image-of-a-lego-set} $ wget http://cdn.firstcry.com/brainbees/images/products/thumb/191325a.jpg ``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md index cb8b2529975..dfcbae74b10 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md @@ -15,22 +15,22 @@ keywords: ['example dataset', 'menus', 'historical data', 'sample data', 'nypl'] このデータは図書館アーカイブ由来であるため、不完全であったり、統計解析には扱いづらい場合があります。それでも、とても「おいしい」データです。 メニューに掲載された料理に関するレコードはわずか130万件で、ClickHouse にとってはごく小さなデータ量ですが、良いサンプルとして利用できます。 -## データセットをダウンロードする +## データセットをダウンロードする {#download-dataset} 次のコマンドを実行します。 ```bash wget https://s3.amazonaws.com/menusdata.nypl.org/gzips/2021_08_01_07_01_17_data.tgz -# オプション: チェックサムの検証 +# オプション: チェックサムの検証 {#option-validate-the-checksum} md5sum 2021_08_01_07_01_17_data.tgz -# チェックサムは次と一致する必要があります: db6126724de939a5481e3160a2d67d15 +# チェックサムは次と一致する必要があります: db6126724de939a5481e3160a2d67d15 {#checksum-should-be-equal-to-db6126724de939a5481e3160a2d67d15} ``` 必要に応じて、[http://menus.nypl.org/data](http://menus.nypl.org/data) にある最新のリンクに差し替えてください。 ダウンロードサイズは約 35 MB です。 -## データセットの展開 +## データセットの展開 {#unpack-dataset} ```bash tar xvf 2021_08_01_07_01_17_data.tgz @@ -46,7 +46,7 @@ tar xvf 2021_08_01_07_01_17_data.tgz * `MenuItem` — メニュー項目。特定のメニューページ上での料理とその価格を表し、`Dish` と `MenuPage` へのリンクを持ちます。 -## テーブルを作成する +## テーブルを作成する {#create-tables} 価格を格納するために[Decimal](../../sql-reference/data-types/decimal.md)データ型を使用します。 @@ -114,7 +114,7 @@ CREATE TABLE menu_item ``` -## データをインポートする +## データをインポートする {#import-data} ClickHouse にデータをアップロードするには、次のコマンドを実行します: @@ -134,7 +134,7 @@ clickhouse-client --format_csv_allow_single_quotes 0 --input_format_null_as_defa [date_time_input_format best_effort](/operations/settings/formats#date_time_input_format) 設定により、[DateTime](../../sql-reference/data-types/datetime.md) フィールドをさまざまなフォーマットでパースできます。例えば、秒なしの ISO-8601 形式である「2000-01-01 01:02」も認識されます。この設定を有効にしない場合、固定形式の DateTime フォーマットのみが許可されます。 -## データを非正規化する +## データを非正規化する {#denormalize-data} データは[正規化形式](https://en.wikipedia.org/wiki/Database_normalization#Normal_forms)で複数のテーブルに分かれて格納されています。これは、例えばメニュー項目から料理名をクエリしたい場合などに、[JOIN](/sql-reference/statements/select/join) を実行する必要があるということです。 典型的な分析タスクでは、毎回 `JOIN` を実行しないよう、あらかじめ JOIN 済みのデータを扱う方がはるかに効率的です。これを「非正規化」データと呼びます。 @@ -187,7 +187,7 @@ FROM menu_item ``` -## データを検証する +## データを検証する {#validate-data} クエリ: @@ -206,7 +206,7 @@ SELECT count() FROM menu_item_denorm; ## いくつかのクエリを実行してみる {#run-queries} -### 料理の過去平均価格 +### 料理の過去平均価格 {#query-averaged-historical-prices} クエリ: @@ -249,7 +249,7 @@ ORDER BY d ASC; あくまで目安としてお考えください。 -### ハンバーガーの価格 +### ハンバーガーの価格 {#query-burger-prices} クエリ: @@ -287,7 +287,7 @@ ORDER BY d ASC; ``` -### ウォッカ +### ウォッカ {#query-vodka} クエリ: @@ -322,7 +322,7 @@ ORDER BY d ASC; ウォッカを取得するには `ILIKE '%vodka%'` と書く必要があり、これはなかなかインパクトのある書き方です。 -### キャビア +### キャビア {#query-caviar} キャビアの価格を表示しましょう。また、キャビア料理の名前をひとつ表示しましょう。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md index 0cab1334a30..e46b2b4a78f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md @@ -28,7 +28,7 @@ The sections below give a brief overview of the steps that were involved in brin - ClickHouse 向けに[事前に用意されたデータ](#pre-prepared-data)。クレンジングや再構造化、拡張が行われており、1900 年から 2022 年までをカバーしています。 - [元データをダウンロード](#original-data)して、ClickHouse に必要な形式へ変換します。独自のカラムを追加したいユーザーは、このアプローチを検討するとよいでしょう。 -### 事前処理済みデータ +### 事前処理済みデータ {#pre-prepared-data} より具体的には、NOAA による品質保証チェックで一度も失敗しなかった行は削除されています。また、データは 1 行につき 1 測定値という構造から、station id と日付ごとに 1 行という構造に再編成されています。つまり、 @@ -55,7 +55,7 @@ wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enriche 以下では、ClickHouse にロードするための準備として、元データをダウンロードおよび変換する手順を説明します。 -#### ダウンロード +#### ダウンロード {#download} 元データをダウンロードするには、次の手順を行います。 @@ -64,7 +64,7 @@ for i in {1900..2023}; do wget https://noaa-ghcn-pds.s3.amazonaws.com/csv.gz/${i ``` -#### データのサンプリング +#### データのサンプリング {#sampling-the-data} ```bash $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format PrettyCompact @@ -108,7 +108,7 @@ $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format Prett 1 行あたり 1 件の測定値とする形式では、ClickHouse ではスパースなテーブル構造になってしまいます。時刻と観測所ごとに 1 行とし、測定値を列として持つ形式に変換する必要があります。まず、問題のない行、すなわち `qFlag` が空文字列に等しい行にデータセットを絞り込みます。 -#### データをクリーンアップする +#### データをクリーンアップする {#clean-the-data} [ClickHouse local](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) を使用して、関心のある測定値を表し、かつ品質要件を満たす行だけが残るようにフィルタリングできます。 @@ -122,7 +122,7 @@ FROM file('*.csv.gz', CSV, 'station_id String, date String, measurement String, 26億行以上あるため、すべてのファイルをパースするこのクエリは高速ではありません。8コアのマシンでは、完了までに約160秒かかります。 -### データのピボット +### データのピボット {#pivot-data} 1 行に 1 つの計測値を持つ構造も ClickHouse で利用できますが、将来のクエリを不必要に複雑にしてしまいます。理想的には、各 station id と日付ごとに 1 行とし、各計測タイプとその値がそれぞれ列になる形が望ましいです。つまり、次のようなイメージです。 @@ -161,7 +161,7 @@ done このクエリにより、サイズが 50GB の単一ファイル `noaa.csv` が生成されます。 -### データの拡充 +### データの拡充 {#enriching-the-data} 現在のデータには、国コードのプレフィックスを含むステーション ID 以外に位置情報に関する情報がありません。本来であれば、各ステーションには緯度と経度が紐づいていることが望ましいです。これを実現するために、NOAA は各ステーションの詳細を別ファイルとして提供しており、それが [ghcnd-stations.txt](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file) です。このファイルには[複数の列](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file)があり、そのうち今後の分析で有用なのは 5 つの列、すなわち id、latitude、longitude、elevation、name です。 @@ -194,7 +194,7 @@ FROM file('noaa.csv', CSV, このクエリは実行に数分かかり、サイズ 6.4 GB の `noaa_enriched.parquet` ファイルを生成します。 -## テーブルの作成 +## テーブルの作成 {#create-table} ClickHouse クライアントから、ClickHouse 上に MergeTree テーブルを作成します。 @@ -223,7 +223,7 @@ CREATE TABLE noaa ## ClickHouse へのデータ挿入 {#inserting-into-clickhouse} -### ローカルファイルからの挿入 +### ローカルファイルからの挿入 {#inserting-from-local-file} データは、ClickHouse クライアントから次のようにローカルファイルから挿入できます: @@ -236,7 +236,7 @@ INSERT INTO noaa FROM INFILE '/noaa_enriched.parquet' この読み込みを高速化する方法については[こちら](https://clickhouse.com/blog/real-world-data-noaa-climate-data#load-the-data)を参照してください。 -### S3 からの挿入 +### S3 からの挿入 {#inserting-from-s3} ```sql INSERT INTO noaa SELECT * @@ -249,7 +249,7 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enr ## サンプルクエリ {#sample-queries} -### 観測史上最高気温 +### 観測史上最高気温 {#highest-temperature-ever} ```sql SELECT @@ -278,7 +278,7 @@ LIMIT 5 2023 年時点での [Furnace Creek](https://www.google.com/maps/place/36%C2%B027'00.0%22N+116%C2%B052'00.1%22W/@36.1329666,-116.1104099,8.95z/data=!4m5!3m4!1s0x0:0xf2ed901b860f4446!8m2!3d36.45!4d-116.8667) における[記録上の最高気温](https://en.wikipedia.org/wiki/List_of_weather_records#Highest_temperatures_ever_recorded)と比べても、安心できるほどよく一致しています。 -### 最高のスキーリゾート +### 最高のスキーリゾート {#best-ski-resorts} アメリカ合衆国の[スキーリゾート一覧](https://gist.githubusercontent.com/gingerwizard/dd022f754fd128fdaf270e58fa052e35/raw/622e03c37460f17ef72907afe554cb1c07f91f23/ski_resort_stats.csv)とそれぞれの所在地を用い、過去5年間のいずれかの月における降雪量が最大だった上位1000件の気象観測所と結合します。この結合結果を[geoDistance](/sql-reference/functions/geo/coordinates/#geodistance)でソートし、距離が20km未満のものに絞り込んだうえで、リゾートごとに最上位の結果を選択し、それらを合計降雪量で並べ替えます。なお、良好なスキーコンディションの大まかな指標として、標高1800m以上のリゾートのみに制限しています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md index 1268ecf07d8..9407ab26c93 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md @@ -24,7 +24,7 @@ import TabItem from '@theme/TabItem'; ::: -## trips テーブルを作成する +## trips テーブルを作成する {#create-the-table-trips} まずはタクシー乗車データ用のテーブルを作成します。 @@ -125,7 +125,7 @@ FROM gcs( -## サンプルクエリ +## サンプルクエリ {#sample-queries} 以下のクエリは、前述のサンプルに対して実行されます。ユーザーは、以下のクエリを修正してテーブル `nyc_taxi.trips` を使用することで、完全なデータセットに対してサンプルクエリを [sql.clickhouse.com](https://sql.clickhouse.com/?query=U0VMRUNUIGNvdW50KCkgRlJPTSBueWNfdGF4aS50cmlwcw\&chart=eyJ0eXBlIjoibGluZSIsImNvbmZpZyI6eyJ0aXRsZSI6IlRlbXBlcmF0dXJlIGJ5IGNvdW50cnkgYW5kIHllYXIiLCJ4YXhpcyI6InllYXIiLCJ5YXhpcyI6ImNvdW50KCkiLCJzZXJpZXMiOiJDQVNUKHBhc3Nlbmdlcl9jb3VudCwgJ1N0cmluZycpIn19) 上で実行できます。 @@ -182,7 +182,7 @@ ORDER BY passenger_count ASC ``` -## 準備済みパーティションのダウンロード +## 準備済みパーティションのダウンロード {#download-of-prepared-partitions} :::note 以下の手順では、元のデータセットに関する情報と、準備済みパーティションをセルフマネージドな ClickHouse サーバー環境にロードする方法を説明します。 @@ -195,9 +195,9 @@ ORDER BY passenger_count ASC ```bash $ curl -O https://datasets.clickhouse.com/trips_mergetree/partitions/trips_mergetree.tar -# チェックサムを検証する +# チェックサムを検証する {#validate-the-checksum} $ md5sum trips_mergetree.tar -# チェックサムは次の値と一致する必要があります: f3b8d469b41d9a82da064ded7245d12c +# チェックサムは次の値と一致する必要があります: f3b8d469b41d9a82da064ded7245d12c {#checksum-should-be-equal-to-f3b8d469b41d9a82da064ded7245d12c} $ tar xvf trips_mergetree.tar -C /var/lib/clickhouse # ClickHouse データディレクトリへのパス $ # 展開されたデータの権限を確認し、必要に応じて修正する $ sudo service clickhouse-server restart @@ -209,7 +209,7 @@ $ clickhouse-client --query "select count(*) from datasets.trips_mergetree" ::: -## 単一サーバー環境での結果 +## 単一サーバー環境での結果 {#results-on-single-server} Q1: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md index 08afe191fdd..b417e399de8 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md @@ -42,7 +42,7 @@ keywords: ['サンプルデータセット', 'nypd', '犯罪データ', 'サン ClickHouse データベースを扱い始める前に、まずデータの内容を確認しておいてください。 -### 元の TSV ファイルのフィールドを確認する +### 元の TSV ファイルのフィールドを確認する {#look-at-the-fields-in-the-source-tsv-file} これは TSV ファイルをクエリするコマンド例ですが、まだ実行しないでください。 @@ -119,7 +119,7 @@ New Georeferenced Column Nullable(String) ここで、TSV ファイル内のカラムが、[dataset web page](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243) の **Columns in this Dataset** セクションに記載されている名前と型と一致しているか確認してください。データ型はあまり厳密ではなく、数値フィールドはすべて `Nullable(Float64)`、それ以外のフィールドはすべて `Nullable(String)` になっています。データを保存するための ClickHouse テーブルを作成する際には、より適切でパフォーマンスに優れた型を指定できます。 -### 適切なスキーマを決定する +### 適切なスキーマを決定する {#determine-the-proper-schema} 各フィールドにどの型を使用すべきか判断するには、データがどのような内容かを把握しておく必要があります。たとえば、フィールド `JURISDICTION_CODE` は数値ですが、これは `UInt8` にすべきでしょうか、それとも `Enum` にすべきでしょうか、あるいは `Float64` が適切でしょうか? @@ -210,7 +210,7 @@ clickhouse-local --input_format_max_rows_to_read_for_schema_inference=2000 \ この執筆時点で使用しているデータセットでは、`PARK_NM` 列に含まれる公園および遊び場の値は数百種類しかありません。これは、`LowCardinality(String)` フィールド内の異なる文字列数を 10,000 未満に抑えることを推奨している [LowCardinality](/sql-reference/data-types/lowcardinality#description) のガイドラインと比較しても小さい数です。 -### DateTime フィールド +### DateTime フィールド {#datetime-fields} [データセットのウェブページ](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243) の **Columns in this Dataset** セクションによると、報告されたイベントの開始時刻と終了時刻を表す日時フィールドが用意されています。`CMPLNT_FR_DT` と `CMPLT_TO_DT` の最小値と最大値を確認すると、これらのフィールドが常に値で埋められているかどうかを把握できます。 @@ -297,7 +297,7 @@ FORMAT PrettyCompact" 型に対して行うべき変更はまだ多くありますが、いずれも同じ調査手順に従うことで判断できます。フィールド内の文字列の異なる値の数や、数値の最小値と最大値を確認し、それに基づいて決定してください。このガイドの後半で示すテーブルスキーマでは、低カーディナリティの文字列と符号なし整数フィールドが多数あり、浮動小数点数値はごくわずかです。 ::: -## 日付フィールドと時刻フィールドを連結する +## 日付フィールドと時刻フィールドを連結する {#concatenate-the-date-and-time-fields} 日付フィールド `CMPLNT_FR_DT` と時刻フィールド `CMPLNT_FR_TM` を、`DateTime` 型にキャスト可能な 1 つの `String` 型の値に連結するには、連結演算子で結合した 2 つのフィールドを選択します: `CMPLNT_FR_DT || ' ' || CMPLNT_FR_TM`。`CMPLNT_TO_DT` と `CMPLNT_TO_TM` フィールドも同様に処理されます。 @@ -328,7 +328,7 @@ FORMAT PrettyCompact" ``` -## 日付と時刻の文字列を DateTime64 型に変換する +## 日付と時刻の文字列を DateTime64 型に変換する {#convert-the-date-and-time-string-to-a-datetime64-type} このガイドの前のセクションで、TSV ファイル内に 1970 年 1 月 1 日より前の日付が含まれていることを確認しました。これは、日付に 64 ビットの DateTime 型が必要であることを意味します。さらに、日付は `MM/DD/YYYY` 形式から `YYYY/MM/DD` 形式へ変換する必要があります。これらはどちらも、[`parseDateTime64BestEffort()`](../../sql-reference/functions/type-conversion-functions.md#parsedatetime64besteffort) を使用して実行できます。 @@ -389,7 +389,7 @@ FORMAT PrettyCompact" これまでに決定した、カラムに使用するデータ型は、以下のテーブルスキーマに反映されています。テーブルで使用する `ORDER BY` と `PRIMARY KEY` も決める必要があります。`ORDER BY` と `PRIMARY KEY` の少なくとも一方は必ず指定しなければなりません。`ORDER BY` に含めるカラムを決定するためのガイドラインを以下に示します。さらに詳しい情報は、このドキュメント末尾の *Next Steps* セクションで説明します。 -### `ORDER BY` と `PRIMARY KEY` 句 +### `ORDER BY` と `PRIMARY KEY` 句 {#order-by-and-primary-key-clauses} * `ORDER BY` タプルには、クエリのフィルタで使用されるフィールドを含めるべきです * ディスク上での圧縮率を最大化するには、`ORDER BY` タプルはカーディナリティが低いものから高いものへ昇順になるように並べるべきです @@ -485,7 +485,7 @@ CREATE TABLE NYPD_Complaint ( ``` -### テーブルの主キーを確認する +### テーブルの主キーを確認する {#finding-the-primary-key-of-a-table} ClickHouse の `system` データベースのうち、特に `system.table` には、作成したばかりのテーブルに関するすべての情報が含まれています。次のクエリで、`ORDER BY`(ソートキー)と `PRIMARY KEY` を確認できます。 @@ -520,7 +520,7 @@ Query id: 6a5b10bf-9333-4090-b36e-c7f08b1d9e01 データの前処理には `clickhouse-local` ツールを使用し、取り込みには `clickhouse-client` を使用します。 -### `clickhouse-local` で使用される引数 +### `clickhouse-local` で使用される引数 {#clickhouse-local-arguments-used} :::tip `table='input'` は、以下の `clickhouse-local` の引数に含まれています。`clickhouse-local` は指定された入力(`cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv`)を受け取り、その入力をテーブルに挿入します。デフォルトではテーブル名は `table` です。このガイドではデータフローを分かりやすくするために、テーブル名を `input` に設定しています。`clickhouse-local` の最後の引数は、そのテーブル(`FROM input`)から SELECT するクエリであり、その結果がパイプで `clickhouse-client` に渡されて、`NYPD_Complaint` テーブルにデータが投入されます。 @@ -571,7 +571,7 @@ cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv \ ``` -## データを検証する +## データを検証する {#validate-data} :::note このデータセットは年に1回以上更新される可能性があるため、このドキュメントに記載されている数値と一致しない場合があります。 @@ -615,7 +615,7 @@ WHERE name = 'NYPD_Complaint' ## いくつかクエリを実行する {#run-queries} -### クエリ 1. 月別の苦情件数を比較する +### クエリ 1. 月別の苦情件数を比較する {#query-1-compare-the-number-of-complaints-by-month} クエリ: @@ -653,7 +653,7 @@ Query id: 7fbd4244-b32a-4acf-b1f3-c3aa198e74d9 ``` -### クエリ 2. 区ごとの苦情件数の合計を比較する +### クエリ 2. 区ごとの苦情件数の合計を比較する {#query-2-compare-total-number-of-complaints-by-borough} クエリ: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md index 69889409f1c..eaeb2d0e5ec 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md @@ -127,7 +127,7 @@ CREATE TABLE `ontime` ORDER BY (Year, Quarter, Month, DayofMonth, FlightDate, IATA_CODE_Reporting_Airline); ``` -## 生データのインポート +## 生データのインポート {#import-from-raw-data} データのダウンロード: @@ -144,7 +144,7 @@ ls -1 *.zip | xargs -I{} -P $(nproc) bash -c "echo {}; unzip -cq {} '*.csv' | se (サーバーでメモリ不足などの問題が発生する場合は、`-P $(nproc)` の部分を削除してください) -## 保存したコピーからのインポート +## 保存したコピーからのインポート {#import-from-a-saved-copy} 別の方法として、次のクエリを使用して保存したコピーからデータをインポートできます。 @@ -155,7 +155,7 @@ INSERT INTO ontime SELECT * FROM s3('https://clickhouse-public-datasets.s3.amazo このスナップショットは2022-05-29に作成されました。 -## クエリ +## クエリ {#queries} Q0. diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md index 67432072214..77cd2c9b11c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md @@ -22,7 +22,7 @@ import stackoverflow from '@site/static/images/getting-started/example-datasets/ このデータのスキーマの説明は[こちら](https://meta.stackexchange.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-sede)で確認できます。 -## あらかじめ用意されたデータ +## あらかじめ用意されたデータ {#pre-prepared-data} このデータのコピーを Parquet 形式で提供しており、内容は 2024 年 4 月時点のものです。行数(6,000 万件の投稿)の点では ClickHouse にとっては小規模ですが、このデータセットには大量のテキストと大きな String 型カラムが含まれています。 @@ -33,7 +33,7 @@ CREATE DATABASE stackoverflow 以下の計測結果は、`eu-west-2` に配置された 96 GiB・24 vCPU 構成の ClickHouse Cloud クラスターに対するものです。データセットは `eu-west-3` にあります。 -### 投稿 +### 投稿 {#posts} ```sql CREATE TABLE stackoverflow.posts @@ -73,7 +73,7 @@ INSERT INTO stackoverflow.posts SELECT * FROM s3('https://datasets-documentation 投稿データは年別のファイルとしても利用できます。例: [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet) -### 投票 +### 投票 {#votes} ```sql CREATE TABLE stackoverflow.votes @@ -96,7 +96,7 @@ INSERT INTO stackoverflow.votes SELECT * FROM s3('https://datasets-documentation 投票データも年ごとに利用できます。例: [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/votes/2020.parquet) -### コメント +### コメント {#comments} ```sql CREATE TABLE stackoverflow.comments @@ -120,7 +120,7 @@ INSERT INTO stackoverflow.comments SELECT * FROM s3('https://datasets-documentat コメントについても年ごとのデータが利用可能です。例: [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/comments/2020.parquet) -### ユーザー +### ユーザー {#users} ```sql CREATE TABLE stackoverflow.users @@ -147,7 +147,7 @@ INSERT INTO stackoverflow.users SELECT * FROM s3('https://datasets-documentation ``` -### バッジ +### バッジ {#badges} ```sql CREATE TABLE stackoverflow.badges @@ -168,7 +168,7 @@ INSERT INTO stackoverflow.badges SELECT * FROM s3('https://datasets-documentatio ``` -### PostLinks +### PostLinks {#postlinks} ```sql CREATE TABLE stackoverflow.postlinks @@ -188,7 +188,7 @@ INSERT INTO stackoverflow.postlinks SELECT * FROM s3('https://datasets-documenta ``` -### PostHistory +### PostHistory {#posthistory} ```sql CREATE TABLE stackoverflow.posthistory @@ -217,7 +217,7 @@ INSERT INTO stackoverflow.posthistory SELECT * FROM s3('https://datasets-documen 元のデータセットは、7zip 形式で圧縮された XML ファイルとして [https://archive.org/download/stackexchange](https://archive.org/download/stackexchange) から入手できます。`stackoverflow.com*` というプレフィックスを持つファイルが対象です。 -### ダウンロード +### ダウンロード {#download} ```bash wget https://archive.org/download/stackexchange/stackoverflow.com-Badges.7z @@ -232,7 +232,7 @@ wget https://archive.org/download/stackexchange/stackoverflow.com-Votes.7z これらのファイルは最大 35GB あり、インターネット接続状況によってはダウンロードに約 30 分かかる場合があります。ダウンロードサーバー側で帯域が制限されており、おおよそ 20MB/秒が上限となります。 -### JSON への変換 +### JSON への変換 {#convert-to-json} 本ドキュメント執筆時点では、ClickHouse は入力フォーマットとして XML をネイティブにサポートしていません。ClickHouse にデータをロードするため、まず NDJSON に変換します。 @@ -260,7 +260,7 @@ p7zip -d stackoverflow.com-Posts.7z ```bash mkdir posts cd posts -# 以下は入力XMLファイルを10000行ごとのサブファイルに分割します +# 以下は入力XMLファイルを10000行ごとのサブファイルに分割します {#the-following-splits-the-input-xml-file-into-sub-files-of-10000-rows} tail +3 ../Posts.xml | head -n -1 | split -l 10000 --filter='{ printf "\n"; cat - ; printf "\n"; } > $FILE' - ``` @@ -283,7 +283,7 @@ clickhouse local --query "SELECT * FROM file('posts.json', JSONEachRow, 'Id Int3 ここから始めるための、いくつかの簡単なクエリです。 -### Stack Overflowで最も人気の高いタグ +### Stack Overflowで最も人気の高いタグ {#most-popular-tags-on-stack-overflow} ```sql @@ -313,7 +313,7 @@ Peak memory usage: 224.03 MiB. ``` -### 最も多く回答しているユーザー(アクティブなアカウント) +### 最も多く回答しているユーザー(アクティブなアカウント) {#user-with-the-most-answers-active-accounts} アカウントには `UserId` が必要です。 @@ -340,7 +340,7 @@ LIMIT 5 ``` -### 閲覧数が多い ClickHouse 関連記事 +### 閲覧数が多い ClickHouse 関連記事 {#clickhouse-related-posts-with-the-most-views} ```sql SELECT @@ -371,7 +371,7 @@ LIMIT 10 ``` -### 最も物議を醸した投稿 +### 最も物議を醸した投稿 {#most-controversial-posts} ```sql SELECT diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md index 8d1e2759661..ef6003e52a1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md @@ -13,12 +13,12 @@ keywords: ['サンプルデータセット', 'tpch', 'ベンチマーク', 'サ **参考文献** -- [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) -- [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291) (Poess et. al., 2000) -- [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5) (Boncz et. al.), 2013 -- [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138) (Dresseler et. al.), 2020 +* [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) +* [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291) (Poess et. al., 2000) +* [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5) (Boncz et. al.), 2013 +* [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138) (Dresseler et. al.), 2020 -## データ生成とインポート +## データ生成とインポート {#data-generation-and-import} まず、TPC-H リポジトリを取得し、データ生成ツールをコンパイルします。 @@ -60,7 +60,6 @@ TPC-H 仕様のルールにできるだけ忠実に従います: * セクション 1.3.1 に従い、抽象的なデータ型(例: `Identifier`, `Variable text, size N`)を実装するために、ClickHouse ネイティブのデータ型(例: `Int32`, `String`)を使用します。 これにより読みやすさが向上するだけであり、`dbgen` によって生成される SQL-92 のデータ型(例: `INTEGER`, `VARCHAR(40)`)も ClickHouse で問題なく動作します。 - ```sql CREATE TABLE nation ( n_nationkey Int32, @@ -170,7 +169,6 @@ clickhouse-client --format_csv_delimiter '|' --query "INSERT INTO lineitem FORMA :::note tpch-kit を使用して自分でテーブルを生成する代わりに、公開 S3 バケットからデータをインポートすることもできます。必ず、上記の `CREATE` 文を使って、先に空のテーブルを作成してください。 - ```sql -- スケーリングファクター 1 INSERT INTO nation SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws.com/h/1/nation.tbl', NOSIGN, CSV) SETTINGS format_csv_delimiter = '|', input_format_defaults_for_omitted_fields = 1, input_format_csv_empty_as_default = 1; @@ -195,8 +193,7 @@ INSERT INTO lineitem SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws. ::: - -## クエリ +## クエリ {#queries} :::note SQL 標準に従った正しい結果を得るために、[`join_use_nulls`](../../operations/settings/settings.md#join_use_nulls) の設定を有効にする必要があります。 @@ -349,7 +346,6 @@ ORDER BY **Q5** - ```sql SELECT n_name, @@ -497,7 +493,6 @@ ORDER BY **Q9** - ```sql SELECT nation, @@ -851,7 +846,6 @@ WHERE **Q20** - ```sql SELECT s_name, diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md index 774d9a2be43..e748a928976 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md @@ -26,7 +26,7 @@ keywords: ['サンプルデータセット', '天気', '台湾', 'サンプル - ClickHouse 向けにクリーンアップ、再構成、拡充された[前処理済みデータ](#pre-processed-data)。このデータセットは 1896 年から 2023 年までをカバーします。 - [元の生データをダウンロード](#original-raw-data)し、ClickHouse が要求する形式に変換します。独自のカラムを追加したいユーザーは、このデータを調査したり、自身のアプローチを検討・完成させたりするのに利用できます。 -### 前処理済みデータ +### 前処理済みデータ {#pre-processed-data} このデータセットは、「測定ごとに 1 行」の形式から、「気象観測所 ID」と「測定日」ごとに 1 行となる形式へと再構成されています。つまり、次のような形です。 @@ -47,16 +47,16 @@ C0X100,2016-01-01 04:00:00,1021.2,15.8,74,1.7,8.0,,,,,,,,,,,,,,,,,,,,,,, ```bash wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/preprocessed_weather_daily_1896_2023.tar.gz -# オプション: チェックサムを検証する +# オプション: チェックサムを検証する {#option-validate-the-checksum} md5sum preprocessed_weather_daily_1896_2023.tar.gz -# チェックサムは次と一致する必要があります: 11b484f5bd9ddafec5cfb131eb2dd008 +# チェックサムは次と一致する必要があります: 11b484f5bd9ddafec5cfb131eb2dd008 {#checksum-should-be-equal-to-11b484f5bd9ddafec5cfb131eb2dd008} tar -xzvf preprocessed_weather_daily_1896_2023.tar.gz daily_weather_preprocessed_1896_2023.csv -# オプション: チェックサムを検証する +# オプション: チェックサムを検証する {#option-validate-the-checksum} md5sum daily_weather_preprocessed_1896_2023.csv -# チェックサムは次と一致する必要があります: 1132248c78195c43d93f843753881754 +# チェックサムは次と一致する必要があります: 1132248c78195c43d93f843753881754 {#checksum-should-be-equal-to-1132248c78195c43d93f843753881754} ``` @@ -64,7 +64,7 @@ md5sum daily_weather_preprocessed_1896_2023.csv 以下では、目的に応じて変換や加工を行うための元の生データをダウンロードする手順について説明します。 -#### ダウンロード +#### ダウンロード {#download} 元の生データをダウンロードするには: @@ -73,9 +73,9 @@ mkdir tw_raw_weather_data && cd tw_raw_weather_data wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/raw_data_weather_daily_1896_2023.tar.gz -# オプション: チェックサムの検証 +# オプション: チェックサムの検証 {#option-validate-the-checksum} md5sum raw_data_weather_daily_1896_2023.tar.gz -# チェックサムは b66b9f137217454d655e3004d7d1b51a と一致する必要があります +# チェックサムは b66b9f137217454d655e3004d7d1b51a と一致する必要があります {#checksum-should-be-equal-to-b66b9f137217454d655e3004d7d1b51a} tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1928.csv @@ -84,23 +84,23 @@ tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1931.csv ... -# オプション: チェックサムの検証 +# オプション: チェックサムの検証 {#option-validate-the-checksum} cat *.csv | md5sum -# チェックサムは b26db404bf84d4063fac42e576464ce1 と一致する必要があります +# チェックサムは b26db404bf84d4063fac42e576464ce1 と一致する必要があります {#checksum-should-be-equal-to-b26db404bf84d4063fac42e576464ce1} ``` -#### 台湾の気象観測所情報を取得する +#### 台湾の気象観測所情報を取得する {#retrieve-the-taiwan-weather-stations} ```bash wget -O weather_sta_list.csv https://github.com/Raingel/weather_station_list/raw/main/data/weather_sta_list.csv -# オプション: UTF-8-BOMからUTF-8エンコーディングへ変換 +# オプション: UTF-8-BOMからUTF-8エンコーディングへ変換 {#option-convert-the-utf-8-bom-to-utf-8-encoding} sed -i '1s/^\xEF\xBB\xBF//' weather_sta_list.csv ``` -## テーブルスキーマの作成 +## テーブルスキーマの作成 {#create-table-schema} ClickHouse クライアントから、ClickHouse 上に MergeTree テーブルを作成します。 @@ -144,7 +144,7 @@ ORDER BY (MeasuredDate); ## ClickHouse への挿入 {#inserting-into-clickhouse} -### ローカルファイルからの挿入 +### ローカルファイルからの挿入 {#inserting-from-local-file} データは、ClickHouse クライアントから次のようにローカルファイルを利用して挿入できます: @@ -165,7 +165,7 @@ Peak memory usage: 583.23 MiB. ``` -### URL からのデータ挿入 +### URL からのデータ挿入 {#inserting-from-url} ```sql INSERT INTO tw_weather_data SELECT * @@ -176,7 +176,7 @@ FROM url('https://storage.googleapis.com/taiwan-weather-observaiton-datasets/dai これをより高速化する方法の詳細については、[大規模データロードのチューニング](https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part2)に関するブログ記事を参照してください。 -## データ行数とサイズを確認する +## データ行数とサイズを確認する {#check-data-rows-and-sizes} 1. 何行挿入されたか確認してみましょう。 @@ -210,7 +210,7 @@ WHERE (`table` = 'tw_weather_data') AND active ## クエリ例 {#sample-queries} -### Q1: 特定の年における気象観測所ごとの最高露点温度を取得する +### Q1: 特定の年における気象観測所ごとの最高露点温度を取得する {#q1-retrieve-the-highest-dew-point-temperature-for-each-weather-station-in-the-specific-year} ```sql SELECT @@ -257,7 +257,7 @@ GROUP BY StationId ``` -### Q2: 特定の期間・フィールド・気象観測所を指定した生データの取得 +### Q2: 特定の期間・フィールド・気象観測所を指定した生データの取得 {#q2-raw-data-fetching-with-the-specific-duration-time-range-fields-and-weather-station} ```sql SELECT diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md index adf4b7124f6..2ac0b456647 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md @@ -13,7 +13,7 @@ keywords: ['example dataset', 'uk property', 'sample data', 'real estate', 'gett - 項目の説明: https://www.gov.uk/guidance/about-the-price-paid-data - HM Land Registry のデータを含みます © Crown copyright and database right 2021。このデータは Open Government Licence v3.0 に基づきライセンスされています。 -## テーブルの作成 +## テーブルの作成 {#create-table} ```sql CREATE DATABASE uk; @@ -40,7 +40,7 @@ ORDER BY (postcode1, postcode2, addr1, addr2); ``` -## データの前処理と挿入 +## データの前処理と挿入 {#preprocess-import-data} `url` 関数を使用してデータを ClickHouse にストリーミングします。その前に、受信データの一部を前処理する必要があります。内容は次のとおりです: @@ -95,7 +95,7 @@ FROM url( データの挿入が完了するまで待ちます。ネットワーク速度にもよりますが、1~2分ほどかかります。 -## データを検証する +## データを検証する {#validate-data} 何行挿入されたかを確認して、正しく動作したことを検証します。 @@ -119,7 +119,7 @@ WHERE name = 'uk_price_paid' データを分析するために、いくつかクエリを実行します。 -### クエリ 1: 年ごとの平均価格 +### クエリ 1: 年ごとの平均価格 {#average-price} ```sql runnable SELECT @@ -133,7 +133,7 @@ ORDER BY year ``` -### クエリ 2. ロンドンの1年あたりの平均価格 +### クエリ 2. ロンドンの1年あたりの平均価格 {#average-price-london} ```sql runnable SELECT @@ -150,7 +150,7 @@ ORDER BY year 2020年に住宅価格に異変が起きました!もっとも、それはおそらく驚くことではないでしょうが…… -### クエリ3. 最も高価なエリア +### クエリ3. 最も高価なエリア {#most-expensive-neighborhoods} ```sql runnable SELECT diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md index b25c39bc953..6729471296a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md @@ -241,7 +241,7 @@ keywords: ['サンプルデータセット', 'youtube', 'サンプルデータ', ## 質問 {#questions} -### コメントが無効化されていると、高評価や低評価が押される可能性は下がりますか? +### コメントが無効化されていると、高評価や低評価が押される可能性は下がりますか? {#create-the-table} コメントが無効化されている場合、視聴者は動画に対する気持ちを表そうとして、高評価や低評価を押す傾向が強くなるのでしょうか? @@ -297,7 +297,7 @@ ORDER BY コメントを有効にすると、エンゲージメント率が高くなる傾向があります。 -### 時間の経過に伴う動画数の変化と主なイベント +### 時間の経過に伴う動画数の変化と主なイベント {#insert-data} ```sql SELECT @@ -336,7 +336,7 @@ ORDER BY month ASC; アップローダー数の急増が、[新型コロナウイルス流行期のあたりで明確に見て取れます](https://www.theverge.com/2020/3/27/21197642/youtube-with-me-style-videos-views-coronavirus-cook-workout-study-home-beauty)。 -### 字幕はいつ頃からどのように増えてきたのか +### 字幕はいつ頃からどのように増えてきたのか {#count-row-numbers} 音声認識技術の進歩により、動画に字幕を付けることはいままでになく簡単になりました。YouTube では 2009 年末に自動キャプション機能が追加されましたが、あれが転換点だったのでしょうか? @@ -374,7 +374,7 @@ ORDER BY month ASC; これをきっかけに、難聴者やろう者の視聴者のために、クリエイターが自分の動画に字幕を追加することを促す、大きな成功を収めたキャンペーンが展開されました。 -### 時系列でのアップロード数上位ユーザー +### 時系列でのアップロード数上位ユーザー {#explore-the-data} ```sql WITH uploaders AS @@ -467,7 +467,7 @@ ORDER BY ``` -### ビューはどのように分散されますか? +### ビューはどのように分散されますか? {#if-someone-disables-comments-does-it-lower-the-chance-someone-will-actually-click-like-or-dislike} ```sql SELECT diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/index.md index 75f2efc074c..d1d01b4deca 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/index.md @@ -9,9 +9,7 @@ title: 'チュートリアルとサンプルデータセット' doc_type: 'landing-page' --- - - -# チュートリアルとサンプルデータセット +# チュートリアルとサンプルデータセット {#tutorials-and-example-datasets} ClickHouse の使い方を学び、すぐに使い始めるためのリソースを多数用意しています。 @@ -25,7 +23,6 @@ ClickHouse の使い方を学び、すぐに使い始めるためのリソース {/* 以下の表は、ビルド時に次のスクリプトによって自動生成されます https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh */ } - {/*AUTOGENERATED_START*/ } | ページ | 概要 | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md index ff1d2595a4b..91330b557b3 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md @@ -1,6 +1,8 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; + + # Debian/UbuntuへのClickHouseのインストール {#install-from-deb-packages} > **Debian**または**Ubuntu**では、公式のプリコンパイル済み`deb`パッケージの使用を推奨します。 @@ -15,20 +17,30 @@ ClickHouse をインストールするには、次のコマンドを実行しま ```bash -# 前提パッケージをインストール +# 前提パッケージをインストール {#install-prerequisite-packages} sudo apt-get install -y apt-transport-https ca-certificates curl gnupg +``` + -# ClickHouse の GPG キーをダウンロードしてキーリングに保存する +# ClickHouse の GPG キーをダウンロードしてキーリングに保存する {#download-the-clickhouse-gpg-key-and-store-it-in-the-keyring} curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | sudo gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg -# システムのアーキテクチャを取得する + + +# システムのアーキテクチャを取得する {#get-the-system-architecture} ARCH=$(dpkg --print-architecture) -# ClickHouse リポジトリを apt のソースリストに追加する + + +# ClickHouse リポジトリを apt のソースリストに追加する {#add-the-clickhouse-repository-to-apt-sources} echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=${ARCH}] https://packages.clickhouse.com/deb stable main" | sudo tee /etc/apt/sources.list.d/clickhouse.list -# apt パッケージリストを更新する + + +# apt パッケージリストを更新する {#update-apt-package-lists} + sudo apt-get update + ``` - 必要に応じて、`stable`を`lts`に置き換えることで、異なる[リリース種別](/knowledgebase/production)を使用できます。 @@ -36,42 +48,59 @@ sudo apt-get update
debパッケージをインストールする旧ディストリビューション方式 +``` + ```bash -# 前提パッケージのインストール +# 前提パッケージのインストール {#install-prerequisite-packages} sudo apt-get install apt-transport-https ca-certificates dirmngr +``` -# パッケージの認証に使用する ClickHouse の GPG キーを追加する + +# パッケージの認証に使用する ClickHouse の GPG キーを追加する {#add-the-clickhouse-gpg-key-to-authenticate-packages} sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754 -# APT のソースリストに ClickHouse リポジトリを追加する + + +# APT のソースリストに ClickHouse リポジトリを追加する {#add-the-clickhouse-repository-to-apt-sources} echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \ /etc/apt/sources.list.d/clickhouse.list + + -# apt パッケージリストを更新 +# apt パッケージリストを更新 {#update-apt-package-lists} sudo apt-get update -# ClickHouse サーバーおよびクライアントのパッケージをインストールする + + +# ClickHouse サーバーおよびクライアントのパッケージをインストールする {#install-clickhouse-server-and-client-packages} sudo apt-get install -y clickhouse-server clickhouse-client -# ClickHouse サーバーのサービスを起動する + + +# ClickHouse サーバーのサービスを起動する {#start-the-clickhouse-server-service} sudo service clickhouse-server start -# ClickHouse コマンドラインクライアントを起動する -clickhouse-client # パスワードを設定している場合は "clickhouse-client --password" を使用します。 + + +# ClickHouse コマンドラインクライアントを起動する {#launch-the-clickhouse-command-line-client} + +clickhouse-client # パスワードを設定している場合は "clickhouse-client --password" を使用します。 + ```
+``` -## ClickHouse サーバーとクライアントのインストール +## ClickHouse サーバーとクライアントのインストール {#install-clickhouse-server-and-client} ```bash sudo apt-get install -y clickhouse-server clickhouse-client ``` -## ClickHouse を起動する +## ClickHouse を起動する {#start-clickhouse-server} ClickHouse サーバーを起動するには、次のコマンドを実行します。 @@ -92,7 +121,7 @@ clickhouse-client --password ``` -## スタンドアロン構成の ClickHouse Keeper をインストールする +## スタンドアロン構成の ClickHouse Keeper をインストールする {#install-standalone-clickhouse-keeper} :::tip 本番環境では、ClickHouse Keeper を専用ノード上で実行することを強く推奨します。 @@ -131,7 +160,6 @@ sudo systemctl status clickhouse-keeper | `clickhouse-keeper` | 専用の ClickHouse Keeper ノードに ClickHouse Keeper をインストールするために使用します。ClickHouse server と同じサーバー上で ClickHouse Keeper を実行している場合、このパッケージをインストールする必要はありません。ClickHouse Keeper 本体とデフォルトの ClickHouse Keeper 設定ファイルをインストールします。 |
- :::info 特定のバージョンの ClickHouse をインストールする必要がある場合は、同じバージョンのパッケージをすべてインストールする必要があります: `sudo apt-get install clickhouse-server=21.8.5.7 clickhouse-client=21.8.5.7 clickhouse-common-static=21.8.5.7` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md index 8feffbdda87..1c02b3bc266 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md @@ -1,4 +1,4 @@ -# Docker を使用して ClickHouse をインストールする +# Docker を使用して ClickHouse をインストールする {#install-clickhouse-using-docker} [Docker Hub](https://hub.docker.com/r/clickhouse/clickhouse-server/) のガイドを、 便宜上、以下に再掲します。提供されている Docker イメージは、 @@ -33,9 +33,9 @@ docker pull clickhouse/clickhouse-server -## このイメージの使い方 +## このイメージの使い方 {#how-to-use-image} -### サーバーインスタンスを起動する +### サーバーインスタンスを起動する {#start-server-instance} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickhouse/clickhouse-server @@ -45,18 +45,18 @@ docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickh デフォルトでは、上記のサーバーインスタンスは、パスワードなしの `default` ユーザーとして実行されます。 -### ネイティブクライアントから接続する +### ネイティブクライアントから接続する {#connect-to-it-from-native-client} ```bash docker run -it --rm --network=container:some-clickhouse-server --entrypoint clickhouse-client clickhouse/clickhouse-server -# または +# または {#or} docker exec -it some-clickhouse-server clickhouse-client ``` ClickHouse クライアントの詳細については、[ClickHouse client](/interfaces/cli) を参照してください。 -### curl で接続する +### curl で接続する {#connect-to-it-using-curl} ```bash echo "SELECT 'こんにちは、ClickHouse!'" | docker run -i --rm --network=container:some-clickhouse-server buildpack-deps:curl curl 'http://localhost:8123/?query=' -s --data-binary @- @@ -64,14 +64,14 @@ echo "SELECT 'こんにちは、ClickHouse!'" | docker run -i --rm --network=con HTTP インターフェイスの詳細については、[ClickHouse HTTP Interface](/interfaces/http) を参照してください。 -### コンテナの停止と削除 +### コンテナの停止と削除 {#stopping-removing-container} ```bash docker stop some-clickhouse-server docker rm some-clickhouse-server ``` -### ネットワーキング +### ネットワーキング {#networking} :::note あらかじめ定義されているユーザー `default` は、パスワードが設定されていない限りネットワークにアクセスできません。 @@ -97,7 +97,7 @@ echo 'SELECT version()' | curl 'http://localhost:8123/' --data-binary @- 上記の例でのユーザーのデフォルト設定は、localhost からのリクエストに対してのみ利用可能です ::: -### ボリューム +### ボリューム {#volumes} 永続化を行うために、通常はコンテナ内に次のフォルダをマウントします: @@ -118,7 +118,7 @@ docker run -d \ * `/docker-entrypoint-initdb.d/` - データベース初期化スクリプトを配置するフォルダー(後述)。 -## Linux capabilities +## Linux capabilities {#linear-capabilities} ClickHouse には、複数の [Linux capabilities](https://man7.org/linux/man-pages/man7/capabilities.7.html) の有効化を必要とする高度な機能があります。 @@ -133,29 +133,29 @@ docker run -d \ 詳細は ["Docker における CAP_IPC_LOCK および CAP_SYS_NICE ケーパビリティの設定"](/knowledgebase/configure_cap_ipc_lock_and_cap_sys_nice_in_docker) を参照してください。 -## 設定 +## 設定 {#configuration} コンテナは [HTTP インターフェイス](https://clickhouse.com/docs/interfaces/http_interface/) 用にポート 8123 を、[ネイティブクライアント](https://clickhouse.com/docs/interfaces/tcp/) 用にポート 9000 を公開します。 ClickHouse の設定はファイル "config.xml"([ドキュメント](https://clickhouse.com/docs/operations/configuration_files/))で定義されます。 -### カスタム設定でサーバーインスタンスを起動する +### カスタム設定でサーバーインスタンスを起動する {#start-server-instance-with-custom-config} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 -v /path/to/your/config.xml:/etc/clickhouse-server/config.xml clickhouse/clickhouse-server ``` -### 任意のユーザーとしてサーバーを起動する +### 任意のユーザーとしてサーバーを起動する {#start-server-custom-user} ```bash -# $PWD/data/clickhouse が存在し、現在のユーザーが所有者である必要があります +# $PWD/data/clickhouse が存在し、現在のユーザーが所有者である必要があります {#pwddataclickhouse-should-exist-and-be-owned-by-current-user} docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit nofile=262144:262144 -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` ローカルディレクトリをマウントしてこのイメージを使用する場合、適切なファイル所有権を維持するためにユーザーを指定する必要があります。`--user` 引数を使用し、コンテナ内に `/var/lib/clickhouse` と `/var/log/clickhouse-server` をマウントしてください。そうしないと、コンテナイメージがエラーとなり起動しません。 -### root でサーバーを起動する +### root でサーバーを起動する {#start-server-from-root} ユーザーネームスペースが有効になっている場合、root でサーバーを起動することが有効です。 その場合は、次を実行します。 @@ -164,7 +164,7 @@ docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit no docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` -### 起動時にデフォルトのデータベースとユーザーを作成する方法 +### 起動時にデフォルトのデータベースとユーザーを作成する方法 {#how-to-create-default-db-and-user} コンテナの起動時にユーザー(デフォルトでは `default` という名前のユーザーが使用されます)とデータベースを作成したい場合には、環境変数 `CLICKHOUSE_DB`、`CLICKHOUSE_USER`、`CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT`、`CLICKHOUSE_PASSWORD` を使用して設定できます。 @@ -172,7 +172,7 @@ docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v " docker run --rm -e CLICKHOUSE_DB=my_database -e CLICKHOUSE_USER=username -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 -e CLICKHOUSE_PASSWORD=password -p 9000:9000/tcp clickhouse/clickhouse-server ``` -#### `default` ユーザーの管理 +#### `default` ユーザーの管理 {#managing-default-user} `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD`、`CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` のいずれも設定されていない場合、`default` ユーザーはデフォルトではネットワークアクセスが無効化されています。 @@ -183,7 +183,7 @@ docker run --rm -e CLICKHOUSE_SKIP_USER_SETUP=1 -p 9000:9000/tcp clickhouse/clic ``` -## このイメージを拡張する方法 +## このイメージを拡張する方法 {#how-to-extend-image} このイメージを元にした派生イメージで追加の初期化処理を行うには、`/docker-entrypoint-initdb.d` 配下に `*.sql`、`*.sql.gz`、または `*.sh` スクリプトを 1 つ以上追加します。エントリポイントが `initdb` を呼び出した後、そのディレクトリ内にあるすべての `*.sql` ファイルを実行し、実行可能な `*.sh` スクリプトを実行し、実行可能でない `*.sh` スクリプトは読み込んで(source して)、サービスを起動する前にさらに初期化処理を行います。\ また、初期化中に clickhouse-client で使用される環境変数 `CLICKHOUSE_USER` と `CLICKHOUSE_PASSWORD` を指定することもできます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md index 0b431719605..36c6e2e0641 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md @@ -1,4 +1,4 @@ -# tgzアーカイブを使用したClickHouseのインストール +# tgzアーカイブを使用したClickHouseのインストール {#install-clickhouse-using-tgz-archives} > `deb`または`rpm`パッケージのインストールができないすべてのLinuxディストリビューションでは、公式のプリコンパイル済み`tgz`アーカイブの使用を推奨します。 @@ -19,7 +19,7 @@ -## 最新の ClickHouse バージョンを取得する +## 最新の ClickHouse バージョンを取得する {#get-latest-version} GitHub から最新の ClickHouse バージョンを取得し、`LATEST_VERSION` 変数に設定します。 @@ -30,7 +30,7 @@ export LATEST_VERSION ``` -## システムアーキテクチャを特定する +## システムアーキテクチャを特定する {#detect-system-architecture} システムアーキテクチャを特定し、それに応じて `ARCH` 変数を設定します。 @@ -43,7 +43,7 @@ esac ``` -## 各 ClickHouse コンポーネント用の tarball をダウンロードする +## 各 ClickHouse コンポーネント用の tarball をダウンロードする {#download-tarballs} 各 ClickHouse コンポーネント用の tarball をダウンロードします。ループではまずアーキテクチャ固有のパッケージを試し、なければ汎用パッケージにフォールバックします。 @@ -64,7 +64,7 @@ done ```bash -# clickhouse-common-static パッケージの展開とインストール +# clickhouse-common-static パッケージの展開とインストール {#extract-and-install-clickhouse-common-static-package} tar -xzvf "clickhouse-common-static-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" @@ -74,7 +74,7 @@ sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" ```bash -# デバッグシンボルパッケージの展開とインストール +# デバッグシンボルパッケージの展開とインストール {#extract-and-install-debug-symbols-package} tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" @@ -84,7 +84,7 @@ sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" ```bash -# サーバーパッケージを展開し、設定を行ってインストール +# サーバーパッケージを展開し、設定を行ってインストール {#extract-and-install-server-package-with-configuration} tar -xzvf "clickhouse-server-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-server-$LATEST_VERSION.tgz" sudo "clickhouse-server-$LATEST_VERSION/install/doinst.sh" configure @@ -95,7 +95,7 @@ sudo /etc/init.d/clickhouse-server start # サーバーを起動 ```bash -# クライアントパッケージを展開してインストール +# クライアントパッケージを展開してインストール {#extract-and-install-client-package} tar -xzvf "clickhouse-client-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-client-$LATEST_VERSION.tgz" sudo "clickhouse-client-$LATEST_VERSION/install/doinst.sh" diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md index 781cf93b6ce..569ea917e0e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md @@ -5,12 +5,12 @@ import privacy_allow from "@site/static/images/knowledgebase/fix-the-developer-v -# HomebrewによるClickHouseのインストール +# HomebrewによるClickHouseのインストール {#install-clickhouse-using-homebrew} -## コミュニティ版 Homebrew フォーミュラを使用してインストールする +## コミュニティ版 Homebrew フォーミュラを使用してインストールする {#install-using-community-homebrew-formula} macOS で [Homebrew](https://brew.sh/) を使用して ClickHouse をインストールするには、 ClickHouse コミュニティの [Homebrew フォーミュラ](https://formulae.brew.sh/cask/clickhouse) を使用できます。 @@ -20,7 +20,7 @@ brew install --cask clickhouse ``` -## macOS での開発元検証エラーの解消 +## macOS での開発元検証エラーの解消 {#fix-developer-verification-error-macos} `brew` を使用して ClickHouse をインストールした場合、macOS からエラーが表示されることがあります。 デフォルトでは、macOS は確認できない開発元によって作成されたアプリケーションやツールを実行しません。 @@ -31,7 +31,7 @@ brew install --cask clickhouse この検証エラーを回避するには、システム設定ウィンドウで該当する設定を変更するか、ターミナルを使用するか、または ClickHouse を再インストールするなどして、いずれかの方法で macOS の隔離領域からアプリを削除する必要があります。 -### システム設定での手順 +### システム設定での手順 {#system-settings-process} `clickhouse` 実行ファイルを隔離領域から削除する最も簡単な方法は次のとおりです。 @@ -51,7 +51,7 @@ brew install --cask clickhouse これでターミナルで `clickhouse` コマンドを実行できるようになるはずです。 -### ターミナルでの手順 +### ターミナルでの手順 {#terminal-process} `Allow Anyway` ボタンを押してもこの問題が解消しない場合は、コマンドラインを使って同じ処理を行うことができます。 あるいは、単にコマンドラインを使う方が好みの場合もあるでしょう。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md index b13e2c98feb..3a57c6e1ed7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md @@ -1,11 +1,11 @@ -# curl でスクリプトを実行して ClickHouse をインストールする +# curl でスクリプトを実行して ClickHouse をインストールする {#install-clickhouse-via-script-using-curl} 本番環境向けに ClickHouse をインストールする必要がない場合、最も手早くセットアップする方法は、curl を使ってインストールスクリプトを実行することです。このスクリプトは、利用中の OS に適したバイナリを自動的に判別します。 -## curl を使用して ClickHouse をインストールする +## curl を使用して ClickHouse をインストールする {#install-clickhouse-using-curl} 以下のコマンドを実行して、使用しているオペレーティングシステム向けの単一のバイナリをダウンロードします。 @@ -18,7 +18,7 @@ Mac をお使いの方へ: バイナリの開発元を検証できないとい ::: -## clickhouse-local を起動する +## clickhouse-local を起動する {#start-clickhouse-local} `clickhouse-local` を使用すると、ClickHouse の強力な SQL 構文を利用して、 ローカルおよびリモートファイルを事前の設定なしに処理できます。テーブルデータは一時領域に保存されるため、 @@ -31,7 +31,7 @@ Mac をお使いの方へ: バイナリの開発元を検証できないとい ``` -## clickhouse-server を起動する +## clickhouse-server を起動する {#start-clickhouse-server} データを永続化する場合は、`clickhouse-server` を起動します。ClickHouse サーバーは、次のコマンドで起動できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md index dbc4f24138e..cb213946b80 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md @@ -5,7 +5,7 @@ -## RPM リポジトリの設定 +## RPM リポジトリの設定 {#setup-the-rpm-repository} 次のコマンドを実行して公式リポジトリを追加します: @@ -24,7 +24,7 @@ sudo zypper --gpg-auto-import-keys refresh clickhouse-stable 以下の手順では、利用しているパッケージマネージャに応じて、`yum install` を `zypper install` に置き換えて構いません。 -## ClickHouse サーバーとクライアントをインストールする +## ClickHouse サーバーとクライアントをインストールする {#install-clickhouse-server-and-client-1} ClickHouse をインストールするには、次のコマンドを実行します。 @@ -42,7 +42,7 @@ sudo yum install clickhouse-server-22.8.7.34 ``` -## ClickHouse サーバーを起動する +## ClickHouse サーバーを起動する {#start-clickhouse-server-1} ClickHouse サーバーを起動するには、以下を実行します。 @@ -65,7 +65,7 @@ clickhouse-client --password ``` -## スタンドアロン ClickHouse Keeper をインストールする +## スタンドアロン ClickHouse Keeper をインストールする {#install-standalone-clickhouse-keeper-1} :::tip 本番環境では、ClickHouse Keeper を専用ノード上で実行することを強く推奨します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md index 0a487d97607..2304fb24124 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md @@ -1,4 +1,4 @@ -# WSL を使って Windows に ClickHouse をインストールする +# WSL を使って Windows に ClickHouse をインストールする {#install-clickhouse-on-windows-with-wsl} @@ -11,7 +11,7 @@ WindowsにClickHouseをインストールする場合は、WSL(Windows Subsyst -## WSL をインストールする +## WSL をインストールする {#install-wsl} Windows PowerShell を管理者権限で開き、次のコマンドを実行します。 @@ -26,7 +26,7 @@ Ubuntu 24.04.1 LTS へようこそ (GNU/Linux 5.15.133.1-microsoft-WSL2 x86_64) ``` -## curl を使ったスクリプトで ClickHouse をインストールする +## curl を使ったスクリプトで ClickHouse をインストールする {#install-clickhouse-via-script-using-curl} curl を使ったスクリプトで ClickHouse をインストールするには、次のコマンドを実行します。 @@ -42,7 +42,7 @@ ClickHouseバイナリのダウンロードが完了しました。以下のコ ``` -## clickhouse-local を起動する +## clickhouse-local を起動する {#start-clickhouse-local} `clickhouse-local` を使用すると、ClickHouse の強力な SQL 構文を利用して、 ローカルおよびリモートのファイルを設定なしで処理できます。テーブルデータは @@ -56,7 +56,7 @@ ClickHouseバイナリのダウンロードが完了しました。以下のコ ``` -## clickhouse-server を起動する +## clickhouse-server を起動する {#start-clickhouse-server} データを永続化したい場合は、`clickhouse-server` を実行します。ClickHouse サーバーは次のコマンドで起動できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md index e4485f33a1b..ae90f270a22 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md @@ -10,7 +10,7 @@ doc_type: 'guide' -## ソースからコンパイルする +## ソースからコンパイルする {#compile-from-source} ClickHouse を手動でコンパイルするには、[Linux](/development/build.md) または [macOS](/development/build-osx.md) 向けの手順に従ってください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md index f1036c846c6..61f9992212b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md @@ -10,4 +10,4 @@ doc_type: 'guide' import DebianProd from './_snippets/_deb_install.md' - + diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx index 7ac1817e233..1768651b3fd 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx @@ -18,33 +18,13 @@ import Docker from './_snippets/_docker.md' import {CardPrimary} from '@clickhouse/click-ui/bundled'; import {galaxyOnClick} from '@site/src/lib/galaxy/galaxy' +# インストール手順 \{#installation-instructions\} -# インストール手順 + - - -
+
または、以下からプラットフォーム、ディストリビューション、およびインストール方法を選択して、 オープンソース版 ClickHouse のインストール手順を表示します。 -} - quickinstall={} - debian_prod={} - rpm_prod={} - tar_prod={} - macos_prod={} - docker={} -/> \ No newline at end of file +} quickinstall={} debian_prod={} rpm_prod={} tar_prod={} macos_prod={} docker={} /> \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/playground.md b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/playground.md index 40e70936d80..d2c3bfe027c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/playground.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/playground.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# ClickHouse playground +# ClickHouse playground {#clickhouse-playground} [ClickHouse Playground](https://sql.clickhouse.com) は、ユーザーが独自のサーバーやクラスターをセットアップすることなく、すぐにクエリを実行して ClickHouse を試せる環境です。 Playground には、いくつかのサンプルデータセットが用意されています。 @@ -40,7 +40,7 @@ Playground には、任意の HTTP クライアントからクエリを送信で -## 例 +## 例 {#examples} `curl` を使って HTTPS エンドポイントにアクセスする例: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx index 46491fd5197..618619e1ec5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx @@ -20,16 +20,15 @@ import data_sources from '@site/static/images/_snippets/data_sources.png'; import select_data_source from '@site/static/images/_snippets/select_data_source.png'; import client_details from '@site/static/images/_snippets/client_details.png'; import new_rows_from_csv from '@site/static/images/_snippets/new_rows_from_csv.png'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; +import SQLConsoleDetail from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; +# ClickHouse Cloud クイックスタート \{#clickhouse-cloud-quick-start\} -# ClickHouse Cloud クイックスタート - -> ClickHouse を最も手早く簡単に使い始める方法は、[ClickHouse Cloud](https://console.clickhouse.cloud) で新しいサービスを作成することです。 +> ClickHouse を最も手早く簡単に使い始める方法は、[ClickHouse Cloud](https://console.clickhouse.cloud) で新しいサービスを作成することです。\ > このクイックスタートガイドでは、3 つの簡単なステップでセットアップする手順を説明します。 - ## ClickHouseサービスを作成する + ## ClickHouseサービスを作成する \{#1-create-a-clickhouse-service\} [ClickHouse Cloud](https://console.clickhouse.cloud)で無料のClickHouseサービスを作成するには、以下の手順に従ってサインアップしてください: @@ -58,7 +57,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; おめでとうございます!ClickHouse Cloudサービスが稼働し、オンボーディングが完了しました。データの取り込みとクエリの実行方法については、以下をご参照ください。 - ## ClickHouseに接続する + ## ClickHouseに接続する \{#2-connect-to-clickhouse\} ClickHouseへの接続方法は2つあります: @@ -67,7 +66,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### SQLコンソールを使用して接続する + ### SQLコンソールを使用して接続する \{#connect-using-sql-console\} 迅速に開始するには、ClickHouseがWebベースのSQLコンソールを提供しており、オンボーディング完了後に自動的にリダイレクトされます。 @@ -87,7 +86,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 以上で完了です。新しいClickHouseサービスの使用を開始できます。 - ### アプリケーションへの接続 + ### アプリケーションへの接続 \{#connect-with-your-app\} ナビゲーションメニューから接続ボタンをクリックします。モーダルが開き、サービスの認証情報と、使用するインターフェースまたは言語クライアントでの接続手順が表示されます。 @@ -97,7 +96,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; ご使用の言語クライアントが表示されない場合は、[インテグレーション](/integrations)のリストを参照してください。 - ## データを追加する + ## データを追加する \{#3-add-data\} ClickHouseはデータを取り込むことで真価を発揮します。データを追加する方法は複数あり、そのほとんどはナビゲーションメニューからアクセス可能なData Sourcesページで利用できます。 @@ -113,7 +112,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; * ファイルをアップロード - 対応形式は JSON、CSV、TSV です * ファイルURLからデータをアップロードする - ### ClickPipes + ### ClickPipes \{#clickpipes\} [ClickPipes](http://clickhouse.com/docs/integrations/clickpipes)は、多様なソースからのデータ取り込みを数クリックで実現するマネージド統合プラットフォームです。 最も要求の厳しいワークロードに対応するよう設計されたClickPipesの堅牢でスケーラブルなアーキテクチャは、一貫したパフォーマンスと信頼性を保証します。 ClickPipesは、長期的なストリーミングニーズや一回限りのデータロードジョブに使用できます。 @@ -121,7 +120,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### SQLコンソールを使用したデータの追加 + ### SQLコンソールを使用したデータの追加 \{#add-data-using-the-sql-console\} 多くのデータベース管理システムと同様に、ClickHouseはテーブルを論理的に**データベース**にグループ化します。ClickHouseで新しいデータベースを作成するには、[`CREATE DATABASE`](../../sql-reference/statements/create/database.md)コマンドを使用します。 @@ -162,7 +161,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 選択可能なテーブルエンジンは多数ありますが、単一ノードのClickHouseサーバー上のシンプルなテーブルには、[`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md)が最適な選択となるでしょう。 ::: - #### プライマリキーの簡単な紹介 + #### プライマリキーの簡単な紹介 \{#a-brief-intro-to-primary-keys\} 先に進む前に、ClickHouseにおけるプライマリキーの動作原理を理解しておくことが重要です(プライマリキーの実装は予想外に感じられるかもしれません): @@ -176,7 +175,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; ClickHouseの中核概念について詳しく知るには、["コア概念"](../../managing-data/core-concepts/index.md)を参照してください。 - #### テーブルへのデータの挿入 + #### テーブルへのデータの挿入 \{#insert-data-into-your-table\} ClickHouseでは、おなじみの[`INSERT INTO TABLE`](../../sql-reference/statements/insert-into.md)コマンドを使用できますが、[`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md)テーブルへの各挿入により、ストレージ内に**パート**が作成されることを理解しておく必要があります。 @@ -206,7 +205,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; SELECT * FROM helloworld.my_first_table ``` - ### ClickHouseクライアントを使用してデータを追加する + ### ClickHouseクライアントを使用してデータを追加する \{#add-data-using-the-clickhouse-client\} [**clickhouse client**](/interfaces/cli)というコマンドラインツールを使用して、ClickHouse Cloudサービスに接続することもできます。左側のメニューで`Connect`をクリックし、詳細情報にアクセスしてください。ダイアログのドロップダウンから`Native`を選択します: @@ -286,7 +285,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; exit ``` - ### ファイルのアップロード + ### ファイルのアップロード \{#upload-a-file\} データベースの利用開始時によくあるタスクとして、既存のファイルに保存されているデータの挿入があります。クリックストリームデータを表すサンプルデータをオンラインで提供しており、これを挿入することができます。このデータには、ユーザーID、訪問されたURL、およびイベントのタイムスタンプが含まれています。 @@ -321,9 +320,9 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; ## 次のステップ \{#whats-next\} -- [チュートリアル](/tutorial.md) では、テーブルに 200 万行を挿入し、いくつかの分析クエリを実行します -- [サンプルデータセット](/getting-started/index.md) と、その挿入手順を一覧で提供しています -- [Getting Started with ClickHouse](https://clickhouse.com/company/events/getting-started-with-clickhouse/) の 25 分間の動画をご覧ください -- データが外部ソースから来る場合は、メッセージキュー、データベース、パイプラインなどへの接続方法をまとめた[連携ガイド集](/integrations/index.mdx)を参照してください -- UI/BI 可視化ツールを使用している場合は、[UI を ClickHouse に接続するためのユーザーガイド](/integrations/data-visualization) を参照してください -- [プライマリキー](/guides/best-practices/sparse-primary-indexes.md) に関するユーザーガイドには、プライマリキーについて知っておくべきことと、その定義方法がすべて記載されています \ No newline at end of file +* [チュートリアル](/tutorial.md) では、テーブルに 200 万行を挿入し、いくつかの分析クエリを実行します +* [サンプルデータセット](/getting-started/index.md) と、その挿入手順を一覧で提供しています +* [Getting Started with ClickHouse](https://clickhouse.com/company/events/getting-started-with-clickhouse/) の 25 分間の動画をご覧ください +* データが外部ソースから来る場合は、メッセージキュー、データベース、パイプラインなどへの接続方法をまとめた[連携ガイド集](/integrations/index.mdx)を参照してください +* UI/BI 可視化ツールを使用している場合は、[UI を ClickHouse に接続するためのユーザーガイド](/integrations/data-visualization) を参照してください +* [プライマリキー](/guides/best-practices/sparse-primary-indexes.md) に関するユーザーガイドには、プライマリキーについて知っておくべきことと、その定義方法がすべて記載されています \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx index b02a92d333e..8ccc9bdb73f 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx +++ b/i18n/jp/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx @@ -14,15 +14,14 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import {VerticalStepper} from '@clickhouse/click-ui/bundled'; - -# ClickHouse OSS クイックスタート +# ClickHouse OSS クイックスタート \{#clickhouse-oss-quick-start\} > このクイックスタートチュートリアルでは、8 つの簡単なステップで OSS 版 ClickHouse をセットアップします。お使いの OS に適したバイナリをダウンロードし、 -ClickHouse サーバーの起動方法を学び、ClickHouse クライアントを使ってテーブルを作成し、 -そのテーブルにデータを挿入して、そのデータを取得するクエリを実行します。 +> ClickHouse サーバーの起動方法を学び、ClickHouse クライアントを使ってテーブルを作成し、 +> そのテーブルにデータを挿入して、そのデータを取得するクエリを実行します。 - ## ClickHouseのダウンロード + ## ClickHouseのダウンロード \{#download-the-binary\} ClickHouseはLinux、FreeBSD、macOS上でネイティブに動作し、Windowsでは[WSL](https://learn.microsoft.com/en-us/windows/wsl/about)経由で動作します。 ClickHouseをローカルにダウンロードする最も簡単な方法は、以下の`curl`コマンドを実行することです。 このコマンドは使用中のオペレーティングシステムがサポートされているかを判定し、masterブランチからビルドされた適切なClickHouseバイナリをダウンロードします。 @@ -53,7 +52,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント Macユーザーの方へ:バイナリの開発者を検証できないというエラーが表示される場合は、["MacOSにおける開発者検証エラーの修正方法"](https://clickhouse.com/docs/knowledgebase/fix-developer-verification-error-in-macos)を参照してください。 ::: - ## サーバーを起動する + ## サーバーを起動する \{#start-the-server\} 以下のコマンドを実行してClickHouseサーバーを起動します: @@ -65,7 +64,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント [デフォルトのログレベル](https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose) が`warning`ではなく`trace`に設定されています。 - ## クライアントを起動する + ## クライアントを起動する \{#start-the-client\} `clickhouse-client`を使用してClickHouseサービスに接続します。新しいターミナルを開き、`clickhouse`バイナリが保存されているディレクトリに移動して、以下のコマンドを実行します: @@ -79,7 +78,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント my-host :) ``` - ## テーブルを作成する + ## テーブルを作成する \{#create-a-table\} `CREATE TABLE`を使用して新しいテーブルを定義します。一般的なSQL DDLコマンドはClickHouseで動作しますが、1つ追加があります - ClickHouseのテーブルには`ENGINE`句が必要です。ClickHouseのパフォーマンス上の利点を活用するには、[`MergeTree`](/engines/table-engines/mergetree-family/mergetree)を使用してください: @@ -95,7 +94,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント PRIMARY KEY (user_id, timestamp) ``` - ## データの挿入 + ## データの挿入 \{#insert-data\} ClickHouseでは一般的な`INSERT INTO TABLE`コマンドを使用できますが、`MergeTree`テーブルへの各挿入によって、ストレージ内にClickHouseで**パート**と呼ばれるものが作成されることを理解しておく必要があります。これらの^^パート^^は後でClickHouseによってバックグラウンドでマージされます。 @@ -111,7 +110,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント (101, 'グラニュールは読み取られるデータの最小チャンク', now() + 5, 3.14159 ) ``` - ## 新しいテーブルをクエリする + ## 新しいテーブルをクエリする \{#query-your-new-table\} 他のSQLデータベースと同様に`SELECT`クエリを記述できます: @@ -134,7 +133,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント 4行が返されました。経過時間: 0.008秒 ``` - ## 独自データの挿入 + ## 独自データの挿入 \{#insert-your-own-data\} 次のステップは、お客様のデータをClickHouseに取り込むことです。データ取り込み用の[テーブル関数](/sql-reference/table-functions/index.md)と[インテグレーション](/integrations)を多数ご用意しています。以下のタブにいくつかの例を掲載していますが、ClickHouseと統合可能な技術の詳細なリストについては[インテグレーション](/integrations)ページをご確認ください。 @@ -348,7 +347,7 @@ ClickHouse サーバーの起動方法を学び、ClickHouse クライアント - ## 探索 + ## 探索 \{#explore\} * ClickHouse の仕組みの基本を学ぶには、[Core Concepts](/managing-data/core-concepts) セクションを参照してください。 * ClickHouse の主要なコンセプトや機能を、より踏み込んで解説している [Advanced Tutorial](tutorial.md) もご覧ください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/index.md index 476c767ce21..98b80092534 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/index.md @@ -7,12 +7,11 @@ keywords: ['パフォーマンス最適化', 'ベストプラクティス', '最 doc_type: 'reference' --- -import TableOfContents from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContents from '@site/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; - -# パフォーマンスと最適化 +# パフォーマンスと最適化 {#performance-and-optimizations} このセクションでは、ClickHouse のパフォーマンスを向上させるためのヒントとベストプラクティスを紹介します。 パフォーマンス向上に必要な基本概念を解説している [Core Concepts](/parts) を、このセクションを読む前に参照することを推奨します。 - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md index 5b64d277986..83aafe06bab 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/prewhere_05.gif' import Image from '@theme/IdealImage'; -# PREWHERE 最適化はどのように動作しますか? +# PREWHERE 最適化はどのように動作しますか? {#how-does-the-prewhere-optimization-work} [PREWHERE 句](/sql-reference/statements/select/prewhere) は、ClickHouse におけるクエリ実行の最適化機構です。不要なデータの読み取りを回避し、フィルタ条件に含まれない列をディスクから読み込む前に関係のないデータを除外することで、I/O を削減しクエリ速度を向上させます。 @@ -109,7 +109,7 @@ ClickHouse はバージョン [23.2](https://clickhouse.com/blog/clickhouse-rele -## PREWHERE の効果を測定する方法 +## PREWHERE の効果を測定する方法 {#how-to-measure-prewhere-impact} PREWHERE がクエリに効果を発揮しているか検証するには、`optimize_move_to_prewhere setting` を有効にした場合と無効にした場合でクエリのパフォーマンスを比較します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md index 8925d292794..2a8fca2439c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md @@ -11,7 +11,7 @@ import queryOptimizationDiagram1 from '@site/static/images/guides/best-practices import Image from '@theme/IdealImage'; -# クエリ最適化のためのシンプルガイド +# クエリ最適化のためのシンプルガイド {#a-simple-guide-for-query-optimization} このセクションでは、一般的なシナリオを通じて [analyzer](/operations/analyzer)、[query profiling](/operations/optimizing-performance/sampling-query-profiler)、[avoid nullable Columns](/optimize/avoid-nullable-columns) など、さまざまなパフォーマンス最適化手法の使い方を示し、ClickHouse のクエリ性能を向上させる方法を説明します。 @@ -61,7 +61,7 @@ ClickHouse には、クエリがどのように実行され、その実行にど -## データセット +## データセット {#dataset} 実際の例を用いて、クエリ性能へのアプローチを説明します。  @@ -112,9 +112,9 @@ ORDER BY tuple() ``` -## 遅いクエリを見つける +## 遅いクエリを見つける {#spot-the-slow-queries} -### クエリログ +### クエリログ {#query-logs} デフォルトでは、ClickHouse は実行された各クエリに関する情報を収集し、[クエリログ](/operations/system-tables/query_log) に書き込みます。このデータはテーブル `system.query_log` に保存されます。  @@ -431,7 +431,7 @@ _最後に、外れ値には注意してください。ユーザーがアドホ -## 基本的な最適化 +## 基本的な最適化 {#basic-optimization} テスト用のフレームワークが用意できたので、ここから最適化を始めていきます。 @@ -439,7 +439,7 @@ _最後に、外れ値には注意してください。ユーザーがアドホ データをどのように取り込んだかによっては、インジェストされたデータに基づいてテーブルスキーマを推論するために、ClickHouse の[機能](/interfaces/schema-inference)を利用しているかもしれません。これは使い始めるうえでは非常に便利ですが、クエリ性能を最適化したい場合は、ユースケースに最も適した形になるようテーブルスキーマを見直す必要があります。 -### Nullable +### Nullable {#nullable} [ベストプラクティスのドキュメント](/best-practices/select-data-types#avoid-nullable-columns)に記載されているとおり、可能な限り Nullable 列は避けてください。データのインジェスト機構を柔軟にできるため多用したくなりますが、そのたびに余分な列を処理する必要があるため、パフォーマンスに悪影響を与えます。 @@ -485,7 +485,7 @@ dropoff_location_id_nulls: 0 `null` 値を含む列は `mta_tax` と `payment_type` の 2 つだけです。その他のフィールドは `Nullable` 列にする必要はありません。 -### 低カーディナリティ +### 低カーディナリティ {#low-cardinality} String 型に対して簡単に適用できる最適化として、LowCardinality データ型を有効活用する方法があります。低カーディナリティに関しては [ドキュメント](/sql-reference/data-types/lowcardinality) に記載されているとおり、ClickHouse は LowCardinality 型の列に辞書エンコーディングを適用し、クエリ性能を大きく向上させます。  @@ -515,7 +515,7 @@ uniq(vendor_id): 3 カーディナリティが低い場合、これら 4 つのカラム `ratecode_id`、`pickup_location_id`、`dropoff_location_id`、`vendor_id` は、LowCardinality データ型の有力な候補となります。 -### データ型の最適化 +### データ型の最適化 {#optimize-data-type} ClickHouse は多数のデータ型をサポートしています。パフォーマンスを最適化し、ディスク上のデータ保存容量を削減するために、ユースケースに適合する範囲で可能な限り小さいデータ型を選択してください。  @@ -538,7 +538,7 @@ Query id: 4306a8e1-2a9c-4b06-97b4-4d902d2233eb 日付の精度は、データセットに合致し、実行する予定のクエリに最も適したものを選択してください。 -### 最適化を適用する +### 最適化を適用する {#apply-the-optimizations} 最適化済みのスキーマを使用する新しいテーブルを作成し、データを再取り込みします。 @@ -604,7 +604,7 @@ Query id: 72b5eb1c-ff33-4fdb-9d29-dd076ac6f532 新しいテーブルは、以前のテーブルと比べてかなり小さくなっています。テーブル全体のディスク使用量は約 34% 削減されており、7.38 GiB から 4.89 GiB になっています。 -## 主キーの重要性 +## 主キーの重要性 {#the-importance-of-primary-keys} ClickHouse における主キーは、多くの従来型データベースシステムとは異なる動作をします。従来のシステムでは、主キーは一意性とデータ整合性を保証します。重複した主キー値を挿入しようとすると拒否され、通常、高速な検索のために B-tree またはハッシュベースのインデックスが作成されます。  @@ -616,7 +616,7 @@ ClickHouse では、主キーの[目的](/guides/best-practices/sparse-primary-i ClickHouse がサポートしている他のオプション(Projection やマテリアライズドビューなど)を使うことで、同じデータに対して異なる主キーのセットを利用することもできます。このブログシリーズの第 2 部では、この点についてさらに詳しく説明します。  -### 主キーを選択する +### 主キーを選択する {#choose-primary-keys} 正しい主キーのセットを選択することは複雑なテーマであり、最適な組み合わせを見つけるにはトレードオフや試行錯誤が必要になる場合があります。  diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md index 6918a2188e3..fd7dd32a5b0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/query-parallelis import Image from '@theme/IdealImage'; -# ClickHouse がクエリを並列実行する仕組み +# ClickHouse がクエリを並列実行する仕組み {#how-clickhouse-executes-a-query-in-parallel} ClickHouse は [高速に動作するよう設計されています](/concepts/why-clickhouse-is-so-fast)。利用可能なすべての CPU コアを使用し、データを複数の処理レーンに分散させることで、高い並列性でクエリを実行し、多くの場合ハードウェアを限界近くまで活用します。 @@ -66,7 +66,7 @@ ClickHouse Cloud では、同様の並列性を [parallel replicas](https://clic -## クエリの並列実行の監視 +## クエリの並列実行の監視 {#monitoring-query-parallelism} これらのツールを使用して、クエリが利用可能な CPU リソースを十分に活用しているかを確認し、そうでない場合の原因を診断します。 @@ -126,12 +126,12 @@ ClickHouse の [組み込み Web UI](/interfaces/http)(`/play` エンドポイ Note: 可視化は左から右へ読んでください。各行は、データをブロック単位でストリーミングし、フィルタリング、集約、最終処理ステージなどの変換を適用する並列処理レーンを表します。この例では、`max_threads = 4` の設定に対応する 4 本の並列レーンが確認できます。 -### 処理レーン間での負荷分散 +### 処理レーン間での負荷分散 {#load-balancing-across-processing-lanes} 上記の物理プラン内の `Resize` オペレーターは、処理レーン間でデータブロックストリームを[再パーティションおよび再分配](/academic_overview#4-2-multi-core-parallelization)することで、各レーンの利用率を均等に保っている点に注意してください。この再バランシングは、データ範囲ごとにクエリ述語にマッチする行数が異なる場合に特に重要です。そうしないと、一部のレーンだけが過負荷になり、他のレーンがアイドル状態になる可能性があります。作業を再分配することで、速いレーンが遅いレーンを実質的に助ける形となり、クエリ全体の実行時間を最適化できます。 -## なぜ max_threads が常にそのとおりには動作しないのか +## なぜ max_threads が常にそのとおりには動作しないのか {#why-max-threads-isnt-always-respected} 前述のとおり、並列処理レーンの数 `n` は `max_threads` 設定によって制御されます。デフォルトでは、この設定値はサーバー上で ClickHouse が利用可能な CPU コア数と同じになります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md index 4c2812935cb..6653926f2f0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md @@ -10,7 +10,7 @@ keywords: ['スキップインデックス', 'データスキップ', 'パフォ -# データスキップインデックスの例 +# データスキップインデックスの例 {#data-skipping-index-examples} このページでは、ClickHouse のデータスキップインデックスの例をまとめ、各インデックスの定義方法、使用すべきタイミング、適用されているかを検証する方法を示します。これらの機能はすべて [MergeTree-family テーブル](/engines/table-engines/mergetree-family/mergetree) で動作します。 @@ -33,7 +33,7 @@ ClickHouse は 5 種類のスキップインデックスをサポートしてい 各セクションではサンプルデータを用いた例を示し、クエリ実行時にインデックスが利用されているかを確認する方法を説明します。 -## MinMax インデックス +## MinMax インデックス {#minmax-index} `minmax` インデックスは、おおまかにソートされたデータや、`ORDER BY` と相関のあるカラムに対する範囲条件に最適です。 @@ -64,7 +64,7 @@ SELECT count() FROM events WHERE ts >= now() - 3600; `EXPLAIN` とプルーニングを用いた[具体例](/best-practices/use-data-skipping-indices-where-appropriate#example)を参照してください。 -## Set インデックス +## Set インデックス {#set-index} ローカル(ブロック単位)のカーディナリティが低い場合に `set` インデックスを使用します。各ブロック内に多数の異なる値が存在する場合には効果がありません。 @@ -81,7 +81,7 @@ SELECT * FROM events WHERE user_id IN (101, 202); 作成/マテリアライズのワークフローと、その適用前後の効果は、[基本的な操作ガイド](/optimize/skipping-indexes#basic-operation)で確認できます。 -## 汎用 Bloom フィルター(スカラー) +## 汎用 Bloom フィルター(スカラー) {#generic-bloom-filter-scalar} `bloom_filter` インデックスは、「干し草の山から針を探すような」等価比較や IN によるメンバーシップ判定に適しています。偽陽性率(デフォルト 0.025)を指定するオプションのパラメータを受け取ります。 @@ -96,7 +96,7 @@ SELECT * FROM events WHERE value IN (7, 42, 99); ``` -## 部分文字列検索用の N-gram Bloom フィルター (ngrambf_v1) +## 部分文字列検索用の N-gram Bloom フィルター (ngrambf_v1) {#n-gram-bloom-filter-ngrambf-v1-for-substring-search} `ngrambf_v1` インデックスは、文字列を N-gram に分割します。`LIKE '%...%'` クエリに対して有効です。String/FixedString/Map(mapKeys/mapValues 経由)をサポートし、サイズ、ハッシュ数、シードを調整できます。詳細については、[N-gram Bloom filter](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter) のドキュメントを参照してください。 @@ -133,7 +133,7 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) AS k; -- 約13 チューニングに関する完全なガイダンスについては、[パラメータのドキュメント](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter)を参照してください。 -## 単語ベース検索用の Token Bloom フィルタ (tokenbf_v1) +## 単語ベース検索用の Token Bloom フィルタ (tokenbf_v1) {#token-bloom-filter-tokenbf-v1-for-word-based-search} `tokenbf_v1` は、英数字以外の文字で区切られたトークンをインデックス化します。[`hasToken`](/sql-reference/functions/string-search-functions#hasToken)、`LIKE` による単語パターン、または `=` / `IN` 演算子と併用して使用することを推奨します。`String`/`FixedString`/`Map` 型をサポートします。 @@ -153,7 +153,7 @@ SELECT count() FROM logs WHERE hasToken(lower(msg), 'exception'); トークンと ngram に関するオブザーバビリティの例とガイダンスについては、[こちら](/use-cases/observability/schema-design#bloom-filters-for-text-search)を参照してください。 -## CREATE TABLE 時にインデックスを追加する(複数の例) +## CREATE TABLE 時にインデックスを追加する(複数の例) {#add-indexes-during-create-table-multiple-examples} スキップインデックスは、複合式や `Map` / `Tuple` / `Nested` 型もサポートします。これは以下の例で示します。 @@ -175,7 +175,7 @@ ORDER BY u64; ``` -## 既存データのマテリアライズと検証 +## 既存データのマテリアライズと検証 {#materializing-on-existing-data-and-verifying} `MATERIALIZE` を使って既存のデータパーツにインデックスを追加し、以下のように `EXPLAIN` やトレースログでプルーニングの動作を確認できます。 @@ -211,7 +211,7 @@ SET send_logs_level = 'trace'; -## 一時的にインデックスを無視または強制する +## 一時的にインデックスを無視または強制する {#temporarily-ignore-or-force-indexes} テストやトラブルシューティングの際に、個々のクエリごとに名前を指定して特定のインデックスを無効化できます。必要に応じてインデックスの使用を強制するための設定もあります。[`ignore_data_skipping_indices`](/operations/settings/settings#ignore_data_skipping_indices) を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md index 52cbdb1bcbb..49fff5a86b1 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md @@ -13,7 +13,7 @@ import bad_skip from '@site/static/images/guides/best-practices/bad_skip.png'; import Image from '@theme/IdealImage'; -# ClickHouse のデータスキップインデックスを理解する +# ClickHouse のデータスキップインデックスを理解する {#understanding-clickhouse-data-skipping-indexes} @@ -29,7 +29,7 @@ ClickHouse のクエリパフォーマンスには多くの要因が影響しま -## 基本的な動作 +## 基本的な動作 {#basic-operation} ユーザーが Data Skipping Indexes を使用できるのは、MergeTree ファミリーのテーブルのみです。各データスキップインデックスには、主に次の 4 つの引数があります。 @@ -122,11 +122,11 @@ SET send_logs_level='trace'; default.skip_table (933d4b2c-8cea-4bf9-8c93-c56e900eefd1) (SelectExecutor): インデックス `vix` により、6104 個のグラニュールのうち 6102 個が除外されました。 ``` -## スキップインデックスのタイプ +## スキップインデックスのタイプ {#skip-index-types} {/* vale off */ } -### minmax +### minmax {#minmax} {/* vale on */ } @@ -136,7 +136,7 @@ SET send_logs_level='trace'; {/* vale off */ } -### set +### set {#set} {/* vale on */ } @@ -144,7 +144,7 @@ SET send_logs_level='trace'; このインデックスのコスト、性能、および有効性は、ブロック内のカーディナリティに依存します。各ブロックに多数のユニークな値が含まれる場合、大きなインデックス集合に対してクエリ条件を評価するのが非常に高コストになるか、あるいは `max_size` を超えた結果としてインデックスが空になり、インデックスが適用されない可能性があります。 -### Bloom フィルター型 +### Bloom フィルター型 {#bloom-filter-types} *Bloom フィルター*は、わずかな偽陽性を許容する代わりに、集合への所属(メンバーシップ)を空間効率良くテストできるデータ構造です。スキップインデックスの場合、偽陽性は大きな問題にはなりません。唯一の不利益は、不要なブロックをいくつか読み込むことだけだからです。しかし、偽陽性が起こり得るということは、インデックス化された式が真になることが期待されるケースで使うべきであり、そうでないと有効なデータがスキップされる可能性があることを意味します。 @@ -185,7 +185,7 @@ Bloom フィルターに基づくデータスキッピングインデックス -## スキップインデックスのベストプラクティス +## スキップインデックスのベストプラクティス {#skip-best-practices} スキップインデックスは必ずしも直感的ではなく、特に RDBMS の行ベースのセカンダリインデックスやドキュメントストアの転置インデックスに慣れたユーザーにはわかりにくいものです。効果を得るには、ClickHouse のデータスキップインデックスを適用することで、インデックスを計算するコストを上回るだけの十分なグラニュールの読み取りを回避する必要があります。重要なのは、インデックス化されたブロック内にある値が 1 回でも出現すると、そのブロック全体をメモリに読み込んで評価しなければならず、その場合インデックスのコストが無駄に発生してしまうという点です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md index 63b75249947..29eab8abf2e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md @@ -36,7 +36,7 @@ import sparsePrimaryIndexes15b from '@site/static/images/guides/best-practices/s import Image from '@theme/IdealImage'; -# ClickHouse におけるプライマリインデックスの実践入門 +# ClickHouse におけるプライマリインデックスの実践入門 {#a-practical-introduction-to-primary-indexes-in-clickhouse} ## はじめに {#introduction} @@ -73,7 +73,7 @@ ClickHouse の [セカンダリ data skipping インデックス](/engines/table このドキュメントで示しているすべての実行時の数値は、Apple M1 Pro チップと 16GB の RAM を搭載した MacBook Pro 上で ClickHouse 22.2.1 をローカルで実行した際の結果に基づいています。 -### フルテーブルスキャン +### フルテーブルスキャン {#a-full-table-scan} 主キーなしのデータセットに対してクエリがどのように実行されるかを確認するために、次の SQL DDL ステートメントを実行して、MergeTree テーブルエンジンを使用するテーブルを作成します。 @@ -150,7 +150,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 10 rows in set. Elapsed: 0.022 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} 処理行数: 887万行、 70.45 MB (3億9853万行/秒、3.17 GB/秒) ``` @@ -172,7 +172,7 @@ B-Tree インデックスに伴う課題を踏まえ、ClickHouse のテーブ 以下では、ClickHouse がスパースプライマリインデックスをどのように構築・利用しているかを詳細に説明します。記事の後半では、インデックス(プライマリキー列)を構築するために使用するテーブル列の選択・削除・並び替えについて、いくつかのベストプラクティスを解説します。 -### 主キーを持つテーブル +### 主キーを持つテーブル {#a-table-with-a-primary-key} `UserID` と `URL` をキー列とする複合主キーを持つテーブルを作成します。 @@ -494,7 +494,7 @@ SELECT UserID, URL FROM file('primary-hits_UserID_URL.idx', 'RowBinary', 'UserID これがクエリ実行性能に与える影響については、このあと詳しく説明します。 -### プライマリインデックスはグラニュールを選択するために使用される +### プライマリインデックスはグラニュールを選択するために使用される {#the-primary-index-is-used-for-selecting-granules} これで、プライマリインデックスを活用してクエリを実行できるようになりました。 @@ -526,7 +526,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 10行のセット。経過時間: 0.005秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 8.19千行を処理、 740.18 KB (1.53百万行/秒、138.59 MB/秒) ``` @@ -537,13 +537,13 @@ ClickHouse クライアントの出力を見ると、フルテーブルスキャ ```response ...Executor): キー条件: (列 0 が [749927693, 749927693] 内) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): パート all_1_9_2 のインデックス範囲で二分探索を実行中 (1083 マーク) ...Executor): 左境界マークを検出: 176 ...Executor): 右境界マークを検出: 177 ...Executor): 19 ステップで連続範囲を検出 ...Executor): パーティションキーで 1/1 パートを選択、プライマリキーで 1 パートを選択、 -# highlight-next-line +# highlight-next-line {#highlight-next-line} プライマリキーで 1/1083 マークを選択、1 範囲から 1 マークを読み取り ...Reading ...1441792 から開始して約 8192 行を読み取り中 ``` @@ -592,7 +592,7 @@ LIMIT 10; │ UserID │ │ Condition: (UserID in [749927693, 749927693]) │ │ Parts: 1/1 │ -# highlight-next-line +# highlight-next-line {#highlight-next-line} │ Granules: 1/1083 │ └───────────────────────────────────────────────────────────────────────────────────────┘ @@ -711,7 +711,7 @@ ClickHouse は、今回の例のクエリ(UserID が 749.927.693 のインタ -### セカンダリキー列は(必ずしも)非効率とは限らない +### セカンダリキー列は(必ずしも)非効率とは限らない {#secondary-key-columns-can-not-be-inefficient} クエリが複合キーの一部であり、かつ先頭のキー列である列を条件にフィルタリングしている場合、[ClickHouse はそのキー列のインデックスマークに対して二分探索アルゴリズムを実行します](#the-primary-index-is-used-for-selecting-granules)。 @@ -757,7 +757,7 @@ LIMIT 10; └────────────┴───────┘ 10 rows in set. Elapsed: 0.086 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} 処理: 881万行、 799.69 MB (1億211万行/秒、9.27 GB/秒) ``` @@ -769,11 +769,11 @@ LIMIT 10; ```response ...Executor): Key condition: (column 1 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Used generic exclusion search over index for part all_1_9_2 with 1537 steps ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 1076/1083 marks by primary key, 1076 marks to read from 5 ranges ...Executor): Reading approx. 8814592 rows with 10 streams ``` @@ -842,7 +842,7 @@ UserID のカーディナリティが高い場合、同じ UserID 値が複数 このサンプルデータセットでは、両方のキー列(UserID, URL)は同程度に高いカーディナリティを持っており、前述のとおり、URL 列の直前のキー列のカーディナリティが高い、あるいは同程度に高い場合には、汎用排他検索アルゴリズムはあまり効果的ではありません。 -### データスキッピングインデックスに関する注意 +### データスキッピングインデックスに関する注意 {#note-about-data-skipping-index} UserID と URL はどちらも同様にカーディナリティが高いため、[URL でのクエリフィルタリング](/guides/best-practices/sparse-primary-indexes#secondary-key-columns-can-not-be-inefficient) についても、[複合主キー (UserID, URL) を持つテーブル](#a-table-with-a-primary-key) の URL 列に [セカンダリのデータスキッピングインデックス](./skipping-indexes.md) を作成しても、得られる効果はそれほど大きくありません。 @@ -906,7 +906,7 @@ UserIDとURLのカーディナリティがともに高いため、この二次 -### オプション 1: セカンダリテーブル +### オプション 1: セカンダリテーブル {#option-1-secondary-tables} @@ -986,7 +986,7 @@ LIMIT 10; └────────────┴───────┘ 10行を取得しました。経過時間: 0.017秒 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 処理行数: 319.49千行、 11.38 MB (18.41百万行/秒、655.75 MB/秒) ``` @@ -1002,13 +1002,13 @@ ClickHouse サーバーのログファイル内に出力される対応する tr ```response ...Executor): Key condition: (column 0 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): パート all_1_9_2 のインデックス範囲でバイナリサーチを実行中 (1083 marks) ...Executor): (LEFT) 境界マークを検出: 644 ...Executor): (RIGHT) 境界マークを検出: 683 ...Executor): 19 ステップで連続範囲を検出 ...Executor): パーティションキーで 1/1 parts を選択、プライマリキーで 1 parts を選択、 -# highlight-next-line +# highlight-next-line {#highlight-next-line} プライマリキーで 39/1083 marks、1 ranges から 39 marks を読み取り ...Executor): 2 streams で約 319488 行を読み取り中 ``` @@ -1143,7 +1143,7 @@ LIMIT 10; └────────────┴───────┘ 10行が設定されています。経過時間: 0.026秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 335.87千行を処理しました、 13.54 MB (12.91百万行/秒、520.38 MB/秒) ``` @@ -1156,7 +1156,7 @@ ClickHouse のサーバーログファイル内の対応するトレースログ ```response ...Executor): Key condition: (column 0 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Running binary search on index range ... ... ...Executor): Selected 4/4 parts by partition key, 4 parts by primary key, @@ -1233,7 +1233,7 @@ LIMIT 10; └────────────┴───────┘ 10 rows in set. Elapsed: 0.029 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Processed 319.49 thousand rows, 1 1.38 MB (11.05 million rows/s., 393.58 MB/s.) ``` @@ -1246,14 +1246,14 @@ ClickHouse のサーバーログファイル中の該当トレースログから ```response ...Executor): Key condition: (column 0 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Running binary search on index range for part prj_url_userid (1083 marks) ...Executor): ... # highlight-next-line ...Executor): Choose complete Normal projection prj_url_userid ...Executor): projection required columns: URL, UserID ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 39/1083 marks by primary key, 39 marks to read from 1 ranges ...Executor): Reading approx. 319488 rows with 2 streams ``` @@ -1444,7 +1444,7 @@ Processed 20.32 thousand rows, その理由は、[汎用除外検索アルゴリズム](https://github.com/ClickHouse/ClickHouse/blob/22.3/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp#L1444) は、直前のキー列のカーディナリティがより低い場合に、その次のセカンダリキー列を用いて [granules](#the-primary-index-is-used-for-selecting-granules) を選択するときに最も効果的に動作するためです。この点については、本ガイドの[前のセクション](#generic-exclusion-search-algorithm)で詳しく説明しました。 -### データファイルの最適な圧縮率 +### データファイルの最適な圧縮率 {#efficient-filtering-on-secondary-key-columns} 次のクエリでは、上記で作成した 2 つのテーブル間における `UserID` 列の圧縮率を比較します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md index 6130bcc0f27..863cc4e6275 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md @@ -19,8 +19,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; どのクエリ言語を使用するかは、`dialect` の設定によって制御されます。 - -## 標準SQL +## 標準SQL {#standard-sql} 標準SQLは ClickHouse のデフォルトのクエリ言語です。 @@ -28,8 +27,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; SET dialect = 'clickhouse' ``` - -## パイプライン型リレーショナルクエリ言語 (PRQL) +## パイプライン型リレーショナルクエリ言語 (PRQL) {#pipelined-relational-query-language-prql} @@ -52,8 +50,7 @@ aggregate { 内部的には、ClickHouse は PRQL クエリを実行する際、PRQL を SQL にトランスパイルして処理します。 - -## Kusto クエリ言語 (KQL) +## Kusto クエリ言語 (KQL) {#kusto-query-language-kql} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md b/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md index 9c61509e8ca..a2e4db08110 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md @@ -6,13 +6,11 @@ keywords: ['マテリアライズドビュー', '集約'] doc_type: 'guide' --- - - -# カスケードするマテリアライズドビュー +# カスケードするマテリアライズドビュー {#cascading-materialized-views} この例では、まずマテリアライズドビューの作成方法を示し、その後、2つ目のマテリアライズドビューを1つ目にカスケードさせる方法を説明します。このページでは、その手順、さまざまな活用方法、および制約について説明します。2つ目のマテリアライズドビューをソースとして使用してマテリアライズドビューを作成することで、さまざまなユースケースに対応できます。 - + + + + allowfullscreen +/> -
+
ClickHouse では JSON を扱うための複数のアプローチを提供しており、それぞれに長所・短所や適したユースケースがあります。このガイドでは、JSON の読み込み方法とスキーマを最適に設計する方法について説明します。このガイドは次のセクションで構成されています。 -- [Loading JSON](/integrations/data-formats/json/loading) - シンプルなスキーマを用いて、ClickHouse で構造化および半構造化 JSON を読み込み、クエリする方法。 -- [JSON schema inference](/integrations/data-formats/json/inference) - JSON schema inference を使用して JSON に対してクエリを実行し、テーブルスキーマを作成する方法。 -- [Designing JSON schema](/integrations/data-formats/json/schema) - JSON スキーマを設計および最適化するための手順。 -- [Exporting JSON](/integrations/data-formats/json/exporting) - JSON をエクスポートする方法。 -- [Handling other JSON Formats](/integrations/data-formats/json/other-formats) - 改行区切り JSON (NDJSON) 以外の JSON フォーマットを扱う際のヒント。 -- [Other approaches to modeling JSON](/integrations/data-formats/json/other-approaches) - JSON をモデリングする従来のアプローチ。**非推奨です。** \ No newline at end of file +* [Loading JSON](/integrations/data-formats/json/loading) - シンプルなスキーマを用いて、ClickHouse で構造化および半構造化 JSON を読み込み、クエリする方法。 +* [JSON schema inference](/integrations/data-formats/json/inference) - JSON schema inference を使用して JSON に対してクエリを実行し、テーブルスキーマを作成する方法。 +* [Designing JSON schema](/integrations/data-formats/json/schema) - JSON スキーマを設計および最適化するための手順。 +* [Exporting JSON](/integrations/data-formats/json/exporting) - JSON をエクスポートする方法。 +* [Handling other JSON Formats](/integrations/data-formats/json/other-formats) - 改行区切り JSON (NDJSON) 以外の JSON フォーマットを扱う際のヒント。 +* [Other approaches to modeling JSON](/integrations/data-formats/json/other-approaches) - JSON をモデリングする従来のアプローチ。**非推奨です。** \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md index 6c35a997f9b..b8229f4a32a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md @@ -17,7 +17,7 @@ doc_type: 'guide' -## 構造化された JSON の読み込み +## 構造化された JSON の読み込み {#loading-structured-json} このセクションでは、JSON データが [`NDJSON`](https://github.com/ndjson/ndjson-spec) (Newline delimited JSON) 形式であり、ClickHouse では [`JSONEachRow`](/interfaces/formats/JSONEachRow) として知られ、かつ列名と型が固定された適切に構造化されたデータであると仮定します。`NDJSON` は、その簡潔さとストレージ効率の良さから JSON を読み込む際に推奨される形式ですが、他の形式も [入力と出力](/interfaces/formats/JSON) の両方でサポートされています。 @@ -124,7 +124,7 @@ FORMAT JSONEachRow これらの例では、`JSONEachRow` 形式の使用を想定しています。その他の一般的な JSON 形式もサポートされており、それらを取り込む例は[こちら](/integrations/data-formats/json/other-formats)にあります。 -## セミ構造化 JSON の読み込み +## セミ構造化 JSON の読み込み {#loading-semi-structured-json} 前の例では、キー名と型がよく分かっている静的な JSON を読み込みました。実際にはそうとは限らず、キーが追加されたり、その型が変化したりします。これは Observability データなどのユースケースでよく見られます。 @@ -200,7 +200,7 @@ LIMIT 2 ここではデータ読み込み時のパフォーマンスの違いに注目してください。`JSON` 列は、挿入時に型推論が必要であり、さらに 1 つの列に複数の型が存在する場合は追加のストレージも必要になります。`JSON` 型は([JSON スキーマの設計](/integrations/data-formats/json/schema)を参照)明示的に列を宣言した場合と同等のパフォーマンスになるように設定できますが、デフォルトではあえて柔軟に使えるように設計されています。しかし、この柔軟性にはある程度のコストが伴います。 -### JSON 型を使用するタイミング +### JSON 型を使用するタイミング {#when-to-use-the-json-type} 次のようなデータの場合は JSON 型を使用します: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md index 6ce5583d5c9..f05cacb0fb5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md @@ -6,9 +6,7 @@ keywords: ['json', 'formats'] doc_type: 'reference' --- - - -# JSON をモデリングするその他のアプローチ +# JSON をモデリングするその他のアプローチ {#other-approaches-to-modeling-json} **以下は、ClickHouse における JSON モデリングの別手法です。網羅性のために記載していますが、これらは JSON 型が登場する以前に有用だったものであり、現在では多くのユースケースにおいて推奨されず、ほとんどの場合適用されません。** @@ -16,9 +14,7 @@ doc_type: 'reference' 同じスキーマ内でも、オブジェクトごとに異なる手法を適用できます。たとえば、一部のオブジェクトには `String` 型が最適であり、別のものには `Map` 型が最適な場合があります。`String` 型を使用した場合、それ以降にスキーマに関する追加の決定を行う必要はない点に注意してください。逆に、以下で示すように、JSON を表す `String` を含め、サブオブジェクトを `Map` のキーに対応する値としてネストすることも可能です。 ::: - - -## String 型の使用 +## String 型の使用 {#using-string} オブジェクトが非常に動的で、予測可能な構造がなく、任意のネストされたオブジェクトを含む場合は、`String` 型を使用することが推奨されます。値は、以下で示すように、クエリ実行時に JSON 関数を使用して抽出できます。 @@ -95,7 +91,6 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/arxiv/arxiv.j 0 rows in set. Elapsed: 25.186 sec. Processed 2.52 million rows, 1.38 GB (99.89 thousand rows/s., 54.79 MB/s.) ``` - 例えば、発行年ごとの論文数を集計したいとします。次のクエリを、単一の文字列カラムのみを使う場合と、スキーマの[構造化されたバージョン](/integrations/data-formats/json/inference#creating-tables)を使う場合で比較してみましょう。 ```sql @@ -156,7 +151,7 @@ Peak memory usage: 205.98 MiB. このアプローチは柔軟性が高い一方で、明確なパフォーマンスおよび構文上のコストを伴うため、スキーマ内で非常に動的なオブジェクトに対してのみ使用すべきです。 -### Simple JSON functions +### Simple JSON functions {#simple-json-functions} 上記の例では JSON* 系の関数を使用しています。これらは [simdjson](https://github.com/simdjson/simdjson) に基づく完全な JSON パーサーを利用しており、厳密なパースを行い、異なる階層にネストされた同名フィールドを区別します。これらの関数は、構文的には正しいものの体裁が整っていない JSON(例: キー間に二重スペースがあるなど)も扱うことができます。 @@ -174,7 +169,6 @@ Peak memory usage: 205.98 MiB. 一方、次の例は正しくパースされます。 - ````json {"@timestamp":893964617,"clientip":"40.135.0.0","request":{"method":"GET", "path":"/images/hm_bg.jpg","version":"HTTP/1.0"},"status":200,"size":24736} @@ -209,8 +203,7 @@ LIMIT 10 上記のクエリでは、公開日時は先頭の値のみで十分であるという事実を利用し、`simpleJSONExtractString` を使って `created` キーを抽出しています。この場合、パフォーマンス向上という利点を考えると、`simpleJSON*` 関数の制約は許容できます。 - -## Map 型の使用 +## Map 型の使用 {#using-map} オブジェクトが任意のキーを格納するために使われ、その多くが単一の型である場合は、`Map` 型の使用を検討してください。理想的には、一意なキーの数は数百を超えないことが望まれます。サブオブジェクトを持つオブジェクトについても、サブオブジェクトの型が一様であれば `Map` 型を検討できます。一般的には、ラベルやタグ、たとえばログデータ中の Kubernetes ポッドラベルなどに対して `Map` 型の使用を推奨します。 @@ -224,7 +217,7 @@ LIMIT 10 オブジェクトを `Map` としてモデリングする場合、JSON のキー名を格納するのに `String` キーが使われます。そのため `Map` は常に `Map(String, T)` となり、`T` はデータに応じて決まります。 ::: -#### プリミティブ値 +#### プリミティブ値 {#primitive-values} `Map` の最も単純な適用例は、オブジェクトが同じプリミティブ型を値として含む場合です。多くの場合、値 `T` として `String` 型を使用します。 @@ -281,13 +274,12 @@ SELECT company.labels['type'] AS type FROM people この型をクエリするための `Map` 関数の完全なセットが利用可能であり、[こちら](/sql-reference/functions/tuple-map-functions.md)で説明されています。データの型が一貫していない場合には、[必要な型変換](/sql-reference/functions/type-conversion-functions)を行うための関数が用意されています。 -#### オブジェクト値 +#### オブジェクト値 {#object-values} `Map` 型は、サブオブジェクトを持つオブジェクトにも適用できます。ただし、サブオブジェクト側の型に一貫性がある必要があります。 `persons` オブジェクトの `tags` キーについて、各 `tag` のサブオブジェクトが `name` と `time` 列を持つ、一定の構造である必要があるとします。そのような JSON ドキュメントの単純化した例は、次のようになります。 - ```json { "id": 1, @@ -360,8 +352,7 @@ FORMAT JSONEachRow } ``` - -## Nested 型の使用 +## Nested 型の使用 {#using-nested} [Nested 型](/sql-reference/data-types/nested-data-structures/nested) は、ほとんど変更されない静的オブジェクトをモデリングするために使用でき、`Tuple` や `Array(Tuple)` の代替手段となります。挙動が分かりにくい場合が多いため、JSON に対してこの型を使用するのは一般的に避けることを推奨します。`Nested` の主な利点は、サブカラムを並び替えキー(ORDER BY キー)として使用できることです。 @@ -396,11 +387,11 @@ CREATE table http ) ENGINE = MergeTree() ORDER BY (status, timestamp); ``` -### flatten_nested +### flatten_nested {#flatten_nested} `flatten_nested` 設定は Nested の動作を制御します。 -#### flatten_nested=1 +#### flatten_nested=1 {#flatten_nested1} 値が `1`(デフォルト)の場合、任意のレベルのネストはサポートされません。この値では、ネストされたデータ構造は、同じ長さを持つ複数の [Array](/sql-reference/data-types/array) カラムと考えると分かりやすいです。`method`、`path`、`version` フィールドは、すべて実質的には別々の `Array(Type)` カラムですが、重要な制約が 1 つあります。**`method`、`path`、`version` フィールドの長さはすべて同じでなければなりません。** `SHOW CREATE TABLE` を使用すると、これは次のように示されます。 @@ -471,10 +462,9 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth 1 rows in set. Elapsed: 0.002 sec. ``` - サブカラムに `Array` を使用することで、[Array functions](/sql-reference/functions/array-functions) の機能全体を、[`ARRAY JOIN`](/sql-reference/statements/select/array-join) 句も含めて活用できる可能性があります。これは、カラムが複数の値を持つ場合に有用です。 -#### flatten_nested=0 +#### flatten_nested=0 {#flatten_nested0} これは任意のレベルのネストを許可し、ネストされたカラムは単一の `Tuple` 配列として保持されることを意味します。実質的に、それらは `Array(Tuple)` と同じになります。 @@ -546,7 +536,7 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth 1 行のセット。経過時間: 0.002 秒。 ``` -### 例 +### 例 {#example} 上記データのより大きなサンプルが、S3 のパブリックバケット `s3://datasets-documentation/http/` に用意されています。 @@ -584,7 +574,6 @@ size FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/doc このデータをクエリするには、リクエストフィールドに配列としてアクセスする必要があります。以下では、一定期間におけるエラーと HTTP メソッドを集計します。 - ```sql SELECT status, request.method[1] AS method, count() AS c FROM http @@ -604,7 +593,7 @@ ORDER BY c DESC LIMIT 5; 5 rows in set. Elapsed: 0.007 sec. ``` -### ペアワイズ配列の使用 +### ペアワイズ配列の使用 {#using-pairwise-arrays} ペアワイズ配列は、JSON を文字列として表現する際の柔軟性と、より構造化されたアプローチによる高い性能とのバランスを提供します。スキーマは柔軟であり、新しいフィールドを任意にルートレベルへ追加することが可能です。しかしその一方で、クエリ構文が大幅に複雑になり、ネストされた構造とは互換性がありません。 @@ -668,7 +657,6 @@ GROUP BY method, status ORDER BY c DESC LIMIT 5; └────────┴────────┴───────┘ ``` - 5 行が返されました。経過時間: 0.383 秒。8.22 百万行、1.97 GB を処理しました (21.45 百万行/秒、5.15 GB/秒)。 ピークメモリ使用量: 51.35 MiB。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md index 94f466a9213..927b493f81b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md @@ -13,11 +13,11 @@ import json_offsets from '@site/static/images/integrations/data-ingestion/data-f import shared_json_column from '@site/static/images/integrations/data-ingestion/data-formats/json_shared_column.png'; -# スキーマの設計 +# スキーマの設計 {#designing-your-schema} [スキーマ推論](/integrations/data-formats/json/inference) を使用すると、JSON データの初期スキーマを確立し、S3 上にあるなどの JSON データファイルをそのままクエリできますが、ユーザーは自分たちのデータに対して、最適化されたバージョン管理済みスキーマを確立することを目指すべきです。以下では、JSON 構造をモデリングする際の推奨アプローチについて説明します。 -## 静的 JSON と動的 JSON +## 静的 JSON と動的 JSON {#static-vs-dynamic-json} JSON のスキーマを定義する際の主なタスクは、各キーの値に対して適切な型を決定することです。各キーに対して適切な型を決定するため、JSON 階層内のそれぞれのキーに以下のルールを再帰的に適用することを推奨します。 @@ -92,7 +92,7 @@ JSON のスキーマを定義する際の主なタスクは、各キーの値に 数百から数千もの静的キーを持つ構造体は、これらの列を静的に宣言するのが現実的でないことが多いため、動的なものと見なすことができます。ただし、可能な箇所では、ストレージと推論の両方のオーバーヘッドを削減するために、不要な[パスをスキップ](#using-type-hints-and-skipping-paths)してください。 ::: -## 静的な構造の扱い方 +## 静的な構造の扱い方 {#handling-static-structures} 静的な構造は名前付きタプル、つまり `Tuple` 型で扱うことを推奨します。オブジェクトの配列は、`Array(Tuple)` のようにタプルの配列として保持できます。タプル内でも、カラムとその型は同じルールに従って定義する必要があります。その結果、以下のように入れ子オブジェクトを表現するためにタプルをネストして用いることになります。 @@ -203,7 +203,7 @@ ORDER BY company.name ``` -### デフォルト値の扱い +### デフォルト値の扱い {#handling-default-values} JSON オブジェクトが構造化されていても、多くの場合は既知のキーの一部しか含まれない疎な形式になることがよくあります。幸い、`Tuple` 型では JSON ペイロード内のすべての列が必須というわけではありません。指定されていない場合にはデフォルト値が使用されます。 @@ -282,7 +282,7 @@ FORMAT PrettyJSONEachRow ::: -### 新しいカラムの扱い +### 新しいカラムの扱い {#handling-new-columns} JSON キーが静的な場合は構造化されたアプローチが最も簡単ですが、スキーマへの変更を事前に計画できる、つまり新しいキーがあらかじめ分かっており、それに応じてスキーマを変更できるのであれば、このアプローチはその場合でも引き続き使用できます。 @@ -360,7 +360,7 @@ SELECT id, nickname FROM people ``` -## 半構造化・動的な構造の扱い +## 半構造化・動的な構造の扱い {#handling-semi-structured-dynamic-structures} JSON データが半構造化されており、キーが動的に追加されたり、複数の型を取りうる場合は、[`JSON`](/sql-reference/data-types/newjson) 型を推奨します。 @@ -491,7 +491,7 @@ JSON データが半構造化されており、キーが動的に追加された - **カラム爆発のリスク回避** - JSON 型はサブカラムを専用カラムとして保存することで潜在的に数千のカラムまでスケールできますが、その結果として過剰な数のカラムファイルが作成され、パフォーマンスに悪影響を与える「カラムファイル爆発」が発生する可能性があります。これを軽減するために、JSON で使用される基盤の [Dynamic 型](/sql-reference/data-types/dynamic) では、[`max_dynamic_paths`](/sql-reference/data-types/newjson#reading-json-paths-as-sub-columns) パラメータが提供されており、個別のカラムファイルとして保存される一意なパスの数を制限します。しきい値に達すると、それ以降のパスはコンパクトなエンコード形式を用いて共有カラムファイル内に保存され、柔軟なデータのインジェストをサポートしつつ、パフォーマンスとストレージ効率を維持します。ただし、この共有カラムファイルへのアクセスは、専用カラムほど高速ではありません。なお、JSON カラムは [type hints](#using-type-hints-and-skipping-paths) と併用できます。「ヒント付き」のカラムは、専用カラムと同等のパフォーマンスを提供します。 - **パスおよび型のインスペクションの容易さ** - JSON 型は、推論された型やパスを判定するための [インスペクション関数](/sql-reference/data-types/newjson#introspection-functions) をサポートしていますが、`DESCRIBE` などを用いた静的な構造のほうが、確認・探索がより簡単な場合があります。 -### 単一の JSON カラム +### 単一の JSON カラム {#single-json-column} この手法はプロトタイピングやデータエンジニアリングのタスクに有用です。本番環境では、必要な場合に限り、動的なサブ構造に対してのみ `JSON` を使用することを推奨します。 @@ -667,7 +667,7 @@ FROM people ``` -### 対象の JSON 列 +### 対象の JSON 列 {#targeted-json-column} プロトタイピングやデータエンジニアリング上の課題に対処するうえでは有用ですが、本番環境では可能な限り明示的なスキーマを使用することを推奨します。 @@ -767,7 +767,7 @@ FORMAT PrettyJsonEachRow ``` -### 型ヒントの使用とパスのスキップ +### 型ヒントの使用とパスのスキップ {#using-type-hints-and-skipping-paths} 型ヒントを使うと、パスとそのサブカラムの型を指定できるため、不必要な型推論を防げます。次の例では、JSON カラム `company.labels` 内の JSON キー `dissolved`、`employees`、`founded` に対して型を指定しています。 @@ -934,7 +934,7 @@ FORMAT PrettyJSONEachRow その結果、大部分が一貫していながらも JSON の柔軟性が有用なデータセットに対して、型ヒントはスキーマやデータ取り込みパイプラインを再構成することなくパフォーマンスを維持する便利な手段となります。 -### 動的パスの設定 +### 動的パスの設定 {#configuring-dynamic-paths} ClickHouse は各 JSON パスを純粋なカラム型レイアウトにおけるサブカラムとして保存し、圧縮、SIMD による高速処理、最小限のディスク I/O といった、従来のカラムと同様のパフォーマンス上の利点を実現します。JSON データ内のそれぞれの一意なパスと型の組み合わせは、それぞれ専用のカラムファイルとしてディスク上に保存されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md index f84c3b1f965..7556e62ded8 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md @@ -10,7 +10,7 @@ keywords: ['parquet', 'columnar format', 'data format', 'compression', 'apache p -# ClickHouse での Parquet の利用 +# ClickHouse での Parquet の利用 {#working-with-parquet-in-clickhouse} Parquet は、データをカラム指向で効率的に保存できるファイル形式です。 ClickHouse は、Parquet ファイルの読み取りと書き込みの両方をサポートします。 @@ -24,7 +24,7 @@ ClickHouse は、Parquet ファイルの読み取りと書き込みの両方を -## Parquet からのインポート +## Parquet からのインポート {#importing-from-parquet} データをロードする前に、[file()](/sql-reference/functions/files.md/#file) 関数を使用して、[Parquet 形式のサンプルファイル](assets/data.parquet) の構造を確認できます。 @@ -64,7 +64,7 @@ LIMIT 3; ::: -## 既存テーブルへのインポート +## 既存テーブルへのインポート {#importing-to-an-existing-table} Parquet データをインポートするためのテーブルを作成します。 @@ -103,7 +103,7 @@ LIMIT 5; ClickHouse が `date` 列の Parquet の文字列データを自動的に `Date` 型へ変換していることに注目してください。これは、ClickHouse がターゲットテーブルの列の型に基づいて自動的に型変換を行うためです。 -## ローカルファイルをリモートサーバーに挿入する +## ローカルファイルをリモートサーバーに挿入する {#inserting-a-local-file-to-remote-server} ローカルの Parquet ファイルをリモートの ClickHouse サーバーに挿入したい場合は、次のようにファイルの内容を `clickhouse-client` にパイプすることで行えます。 @@ -112,7 +112,7 @@ clickhouse client -q "INSERT INTO sometable FORMAT Parquet" < data.parquet ``` -## Parquet ファイルから新しいテーブルを作成する +## Parquet ファイルから新しいテーブルを作成する {#creating-new-tables-from-parquet-files} ClickHouse は Parquet ファイルのスキーマを読み取ることができるため、テーブルを動的に作成できます。 @@ -141,7 +141,7 @@ DESCRIBE TABLE imported_from_parquet; デフォルトでは、ClickHouse はカラム名やデータ型、値に対して厳格に動作します。ただし、状況によっては、インポート時に存在しないカラムやサポートされていない値をスキップすることができます。これは [Parquet 設定](/interfaces/formats/Parquet#format-settings) で制御できます。 -## Parquet 形式へのエクスポート +## Parquet 形式へのエクスポート {#exporting-to-parquet-format} :::tip ClickHouse Cloud で `INTO OUTFILE` を使用する場合、ファイルが書き込まれるマシン上で `clickhouse client` を使ってコマンドを実行する必要があります。 @@ -159,7 +159,7 @@ FORMAT Parquet これにより、作業ディレクトリに `export.parquet` ファイルが作成されます。 -## ClickHouse と Parquet のデータ型 +## ClickHouse と Parquet のデータ型 {#clickhouse-and-parquet-data-types} ClickHouse と Parquet のデータ型はほとんど同一ですが、[いくつか違いがあります](/interfaces/formats/Parquet#data-types-matching-parquet)。たとえば、ClickHouse は `DateTime` 型を Parquet 側の `int64` としてエクスポートします。その後それを ClickHouse にインポートし直すと、([time.parquet ファイル](assets/time.parquet) のように)数値として表示されます: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md index 5aa6cfa68b3..13c61727e27 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md @@ -9,13 +9,13 @@ keywords: ['SQL 形式', 'データエクスポート', 'データインポー -# ClickHouse での SQL データの挿入とダンプ +# ClickHouse での SQL データの挿入とダンプ {#inserting-and-dumping-sql-data-in-clickhouse} ClickHouse は、さまざまな方法で OLTP データベース基盤に容易に統合できます。その 1 つの方法として、SQL ダンプを使用して他のデータベースと ClickHouse 間でデータを転送することが挙げられます。 -## SQL ダンプの作成 +## SQL ダンプの作成 {#creating-sql-dumps} [SQLInsert](/interfaces/formats/SQLInsert) を使用すると、データを SQL 形式でダンプできます。ClickHouse はデータを `INSERT INTO <table name> VALUES(...` 形式で出力し、テーブル名として [`output_format_sql_insert_table_name`](/operations/settings/settings-formats.md/#output_format_sql_insert_table_name) 設定オプションを使用します。 @@ -46,7 +46,7 @@ mysql some_db < dump.sql SET output_format_sql_insert_max_batch_size = 1000; ``` -### 値の集合のエクスポート +### 値の集合のエクスポート {#exporting-a-set-of-values} ClickHouse には [Values](/interfaces/formats/Values) フォーマットがあり、SQL の INSERT 文に似ていますが、`INSERT INTO table VALUES` の部分を省き、値の集合だけを返します。 @@ -59,7 +59,7 @@ SELECT * FROM some_data LIMIT 3 FORMAT Values ``` -## SQLダンプからのデータ挿入 +## SQLダンプからのデータ挿入 {#inserting-data-from-sql-dumps} SQL ダンプを読み込むには、[MySQLDump](/interfaces/formats/MySQLDump) を使用します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md index c02fb8638d9..50a512ac8e0 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md @@ -10,13 +10,13 @@ keywords: ['データ形式', 'テンプレート', '正規表現', 'カスタ -# ClickHouse で Templates と Regex を使用してカスタムテキストデータをインポートおよびエクスポートする +# ClickHouse で Templates と Regex を使用してカスタムテキストデータをインポートおよびエクスポートする {#importing-and-exporting-custom-text-data-using-templates-and-regex-in-clickhouse} 独自テキスト形式のデータ、たとえば非標準的なフォーマット、不正な JSON、壊れた CSV などを扱わなければならないことはよくあります。CSV や JSON といった標準パーサーでは、こうしたすべてのケースを扱えるとは限りません。しかし ClickHouse には強力な Template フォーマットと Regex フォーマットが用意されており、これらのケースにも対応できます。 -## テンプレートに基づくインポート +## テンプレートに基づくインポート {#importing-based-on-a-template} 次の[ログファイル](assets/error.log)からデータをインポートしたいとします。 @@ -88,7 +88,7 @@ GROUP BY request └──────────────────────────────────────────────────┴─────────┘ ``` -### 空白のスキップ +### 空白のスキップ {#skipping-whitespaces} テンプレート内の区切り文字同士の間にある空白を無視できるようにするには、[TemplateIgnoreSpaces](/interfaces/formats/TemplateIgnoreSpaces) の利用を検討してください。 @@ -98,7 +98,7 @@ TemplateIgnoreSpaces --> "p1:${p1:CSV}, p2:${p2:CSV}" ``` -## テンプレートを使用したデータのエクスポート +## テンプレートを使用したデータのエクスポート {#exporting-data-using-templates} テンプレートを使用して任意のテキスト形式でデータをエクスポートすることもできます。この場合は、次の 2 つのファイルを作成する必要があります。 @@ -142,7 +142,7 @@ FORMAT Template SETTINGS format_template_resultset = 'output.results', --- 1000行を0.001380604秒で読み取り --- ``` -### HTML ファイルへのエクスポート +### HTML ファイルへのエクスポート {#exporting-to-html-files} テンプレートベースの結果は、[`INTO OUTFILE`](/sql-reference/statements/select/into-outfile.md) 句を使用してファイルにエクスポートすることもできます。次の [resultset](assets/html.results) および [row](assets/html.row) のフォーマットに基づいて HTML ファイルを生成してみましょう。 @@ -157,7 +157,7 @@ SETTINGS format_template_resultset = 'html.results', format_template_row = 'html.row' ``` -### XML へのエクスポート +### XML へのエクスポート {#exporting-to-xml} Template フォーマットは、XML を含むあらゆるテキスト形式ファイルを生成するために使用できます。適切なテンプレートを用意してエクスポートを実行してください。 @@ -203,7 +203,7 @@ FORMAT XML ``` -## 正規表現に基づくデータのインポート +## 正規表現に基づくデータのインポート {#importing-data-based-on-regular-expressions} [Regexp](/interfaces/formats/Regexp) フォーマットは、入力データをより複雑な方法で解析する必要がある、高度なユースケースに対応します。ここでは [error.log](assets/error.log) のサンプルファイルを解析し、今回はファイル名とプロトコルも抽出して、それぞれ別のカラムに保存します。まず、そのための新しいテーブルを準備します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md index 69351d3e176..d5e5dbc90fc 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md @@ -6,7 +6,7 @@ description: 'データインジェストセクション用ランディングペ doc_type: 'landing-page' --- -# データインジェスト +# データインジェスト {#data-ingestion} ClickHouse は、データ統合および変換のために数多くのソリューションと連携しています。 詳しくは、以下のページを参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md index 27287a524b6..65e9ba1df17 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md @@ -6,7 +6,7 @@ title: 'データソース' doc_type: 'landing-page' --- -# データソース +# データソース {#data-sources} ClickHouse を使用すると、さまざまなソースからデータベースにデータを簡単に取り込むことができます。 詳細については、以下のページを参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md index b3d6b9c5914..9c6b803dac7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md @@ -17,7 +17,7 @@ import dynamodb_map_columns from '@site/static/images/integrations/data-ingestio import Image from '@theme/IdealImage'; -# DynamoDB から ClickHouse への CDC +# DynamoDB から ClickHouse への CDC {#cdc-from-dynamodb-to-clickhouse} @@ -50,9 +50,9 @@ import Image from '@theme/IdealImage'; -## 3. スナップショットを ClickHouse に読み込む +## 3. スナップショットを ClickHouse に読み込む {#3-load-the-snapshot-into-clickhouse} -### 必要なテーブルを作成する +### 必要なテーブルを作成する {#create-necessary-tables} DynamoDB からのスナップショットデータはおおよそ次のようになります: @@ -115,7 +115,7 @@ ORDER BY id; * テーブルはパーティションキーをソートキー(`ORDER BY` で指定)として使用する必要があります。 * 同じソートキーを持つ行は、`version` カラムに基づいて重複排除されます。 -### スナップショット用 ClickPipe の作成 +### スナップショット用 ClickPipe の作成 {#create-the-snapshot-clickpipe} ここまでで、S3 から ClickHouse へスナップショットデータをロードするための ClickPipe を作成できます。S3 ClickPipe ガイドは[こちら](/integrations/clickpipes/object-storage)に従いますが、次の設定を使用してください。 @@ -146,7 +146,7 @@ https://{bucket}.s3.amazonaws.com/{prefix}/AWSDynamoDB/{export-id}/data/* -## 5. クリーンアップ(オプション) +## 5. クリーンアップ(オプション) {#5-cleanup-optional} スナップショット ClickPipe の処理が完了したら、スナップショットテーブルとマテリアライズドビューを削除できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md index c3ee1ce5d84..f585ff40b2a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md @@ -16,7 +16,7 @@ import Jdbc02 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-02 import Jdbc03 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-03.png'; -# JDBC を使用して ClickHouse を外部データソースに接続する +# JDBC を使用して ClickHouse を外部データソースに接続する {#connecting-clickhouse-to-external-data-sources-with-jdbc} :::note JDBC を使用するには ClickHouse JDBC Bridge が必要なため、ローカルマシン上で `clickhouse-local` を使用して、データベースから ClickHouse Cloud へデータをストリーミングする必要があります。詳細については、ドキュメントの **Migrate** セクションにある [**Using clickhouse-local**](/cloud/migration/clickhouse-local#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge) ページを参照してください。 @@ -44,7 +44,7 @@ ClickHouse JDBC Bridge は、読み取りと書き込みの両方に使用でき -## ClickHouse JDBC Bridge をローカルにインストールする +## ClickHouse JDBC Bridge をローカルにインストールする {#install-the-clickhouse-jdbc-bridge-locally} ClickHouse JDBC Bridge を使用する最も簡単な方法は、ClickHouse が動作しているのと同じホスト上にインストールして実行することです。ClickHouse JDBC Bridge をローカルにデプロイした構成図 @@ -107,7 +107,7 @@ ClickHouse JDBC Bridge をフォアグラウンドモードで起動しました ::: -## ClickHouse 内から JDBC 接続を使用する +## ClickHouse 内から JDBC 接続を使用する {#use-the-jdbc-connection-from-within-clickhouse} ClickHouse は、[jdbc テーブル関数](/sql-reference/table-functions/jdbc.md) または [JDBC テーブルエンジン](/engines/table-engines/integrations/jdbc.md) を使用して、MySQL のデータにアクセスできます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md index 6a532a86cd2..42c9395c3e7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md @@ -10,27 +10,24 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - -# ClickHouse と PostgreSQL の接続 +# ClickHouse と PostgreSQL の接続 {#connecting-clickhouse-to-postgresql} このページでは、PostgreSQL と ClickHouse を統合するための次のオプションについて説明します。 -- PostgreSQL のテーブルから読み取るための `PostgreSQL` テーブルエンジンの利用 -- PostgreSQL 内のデータベースと ClickHouse 内のデータベースを同期するための、実験的な `MaterializedPostgreSQL` データベースエンジンの利用 +* PostgreSQL のテーブルから読み取るための `PostgreSQL` テーブルエンジンの利用 +* PostgreSQL 内のデータベースと ClickHouse 内のデータベースを同期するための、実験的な `MaterializedPostgreSQL` データベースエンジンの利用 :::tip [ClickPipes](/integrations/clickpipes/postgres) は、PeerDB を基盤とした ClickHouse Cloud 向けのマネージド連携サービスであり、こちらの利用を推奨します。 また、代替手段として [PeerDB](https://github.com/PeerDB-io/peerdb) は、セルフホスト型の ClickHouse および ClickHouse Cloud 双方への PostgreSQL データベースレプリケーション向けに特化して設計された、オープンソースの CDC(変更データキャプチャ)ツールとして利用できます。 ::: - - -## PostgreSQL テーブルエンジンの使用 +## PostgreSQL テーブルエンジンの使用 {#using-the-postgresql-table-engine} `PostgreSQL` テーブルエンジンを使用すると、リモートの PostgreSQL サーバー上に保存されているデータに対して、ClickHouse から **SELECT** および **INSERT** 操作を行うことができます。 この記事では、1 つのテーブルを使った基本的な連携方法を説明します。 -### 1. PostgreSQL のセットアップ +### 1. PostgreSQL のセットアップ {#1-setting-up-postgresql} 1. `postgresql.conf` で、PostgreSQL がネットワークインターフェイスで待ち受けできるようにするため、次の設定を追加します。 @@ -93,7 +90,7 @@ ClickHouse Cloud 上でこの機能を利用している場合、ClickHouse Clou 外向きトラフィックの詳細については、ClickHouse の [Cloud Endpoints API](/cloud/get-started/query-endpoints) を確認してください。 ::: -### 2. ClickHouse にテーブルを定義する +### 2. ClickHouse にテーブルを定義する {#2-define-a-table-in-clickhouse} 1. `clickhouse-client` にログインします: @@ -131,7 +128,7 @@ ENGINE = PostgreSQL('postgres-host.domain.com:5432', 'db_in_psg', 'table1', 'cli 利用可能なパラメータの完全な一覧については、[PostgreSQL table engine](/engines/table-engines/integrations/postgresql) のドキュメントページを参照してください。 ::: -### 3 統合をテストする +### 3 統合をテストする {#3-test-the-integration} 1. ClickHouse で初期の行を表示します: @@ -166,7 +163,6 @@ VALUES SELECT * FROM db_in_ch.table1 ``` - レスポンスは次のとおりです。 ```response @@ -208,8 +204,7 @@ id | column1 この例では、`PostrgeSQL` テーブルエンジンを使用して、PostgreSQL と ClickHouse の間の基本的な連携方法を示しました。 スキーマの指定、特定のカラムのみを返す設定、複数レプリカへの接続など、さらに多くの機能については、[PostgreSQL テーブルエンジンのドキュメントページ](/engines/table-engines/integrations/postgresql) を参照してください。また、ブログ記事 [ClickHouse and PostgreSQL - a match made in data heaven - part 1](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres) もあわせてご覧ください。 - -## MaterializedPostgreSQL データベースエンジンの使用 +## MaterializedPostgreSQL データベースエンジンの使用 {#using-the-materializedpostgresql-database-engine} @@ -220,7 +215,7 @@ PostgreSQL データベースエンジンは、PostgreSQL のレプリケーシ ***以下の手順では、PostgreSQL CLI (`psql`) と ClickHouse CLI (`clickhouse-client`) を使用します。PostgreSQL サーバーは Linux 上にインストールされています。以下の内容は、PostgreSQL データベースを新規にテストインストールした場合の最小設定です。*** -### 1. PostgreSQL 側の設定 +### 1. PostgreSQL 側の設定 {#1-in-postgresql} 1. `postgresql.conf` で、最低限の listen 設定、レプリケーション用の `wal_level`、レプリケーションスロットを設定します: @@ -275,7 +270,6 @@ VALUES 7. レプリケーション用に、新しいユーザーが新しいデータベースへ接続できるよう PostgreSQL を設定します。以下は `pg_hba.conf` ファイルに追加する最小限のエントリです。 - ```text # TYPE DATABASE USER ADDRESS METHOD host db1 clickhouse_user 192.168.1.0/24 password @@ -349,7 +343,7 @@ Query id: df2381ac-4e30-4535-b22e-8be3894aaafc └────┴─────────┘ ``` -### 3. 基本的なレプリケーションをテストする +### 3. 基本的なレプリケーションをテストする {#2-in-clickhouse} 1. PostgreSQL に新しい行を追加します: @@ -385,7 +379,7 @@ Query id: b0729816-3917-44d3-8d1a-fed912fb59ce └────┴─────────┘ ``` -### 4. まとめ +### 4. まとめ {#3-test-basic-replication} このインテグレーションガイドでは、テーブルを含むデータベースをレプリケートするためのシンプルな例を扱いましたが、データベース全体をレプリケートしたり、既存のレプリケーションに新しいテーブルやスキーマを追加したりするなど、より高度なオプションも存在します。このレプリケーションでは DDL コマンドはサポートされませんが、エンジンを設定することで変更を検出し、スキーマ変更などの構造的な変更が行われた際にテーブルを再読み込みさせることができます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md index ff672310a6c..65c2460e3fb 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md @@ -40,7 +40,7 @@ import clickhouse_result from '@site/static/images/integrations/data-ingestion/e import Image from '@theme/IdealImage'; -# EMQX と ClickHouse の統合 +# EMQX と ClickHouse の統合 {#integrating-emqx-with-clickhouse} @@ -63,7 +63,7 @@ import Image from '@theme/IdealImage'; -## ClickHouse Cloud サービスを取得する +## ClickHouse Cloud サービスを取得する {#get-your-clickhouse-cloudservice} このセットアップでは、ClickHouse インスタンスを N. Virginia (us-east-1) の AWS 上にデプロイし、同じリージョンに EMQX Cloud インスタンスもデプロイしました。 @@ -153,7 +153,7 @@ Overview ページに戻り、ページの一番下までスクロールする -## EMQX Cloud と ClickHouse Cloud の統合 +## EMQX Cloud と ClickHouse Cloud の統合 {#integration-emqx-cloud-with-clickhouse-cloud} [EMQX Cloud Data Integrations](https://docs.emqx.com/en/cloud/latest/rule_engine/introduction.html#general-flow) 機能は、EMQX のメッセージフローおよびデバイスイベントを処理し応答するためのルールを構成するために使用されます。Data Integrations は、明確かつ柔軟な「設定可能な」アーキテクチャソリューションを提供するだけでなく、開発プロセスを簡素化し、ユーザビリティを向上させ、業務システムと EMQX Cloud 間の結合度を低減します。また、EMQX Cloud 固有機能のカスタマイズに対して優れた基盤インフラストラクチャも提供します。 @@ -163,7 +163,7 @@ EMQX Cloud は、代表的なデータシステム向けに 30 を超えるネ EMQX Cloud ClickHouse Data Integration コネクタの詳細 -### ClickHouse リソースの作成 +### ClickHouse リソースの作成 {#create-clickhouse-resource} 左側メニューの「Data Integrations」をクリックし、「View All Resources」をクリックします。Data Persistence セクションで ClickHouse を見つけるか、ClickHouse を検索します。 @@ -177,7 +177,7 @@ ClickHouse カードをクリックして新しいリソースを作成します 接続情報を入力する EMQX Cloud ClickHouse Resource 設定フォーム -### 新しいルールの作成 +### 新しいルールの作成 {#create-a-new-rule} リソース作成時にポップアップが表示され、「New」をクリックするとルール作成ページに移動します。 @@ -212,7 +212,7 @@ SQL のテスト機能を使ってテストを実行し、結果を確認でき 次に「NEXT」ボタンをクリックします。このステップでは、EMQX Cloud に対して、整形されたデータを ClickHouse データベースにどのように挿入するかを指定します。 -### レスポンスアクションを追加する +### レスポンスアクションを追加する {#add-a-response-action} リソースが 1 つだけの場合は、「Resource」と「Action Type」を変更する必要はありません。 SQL テンプレートを設定するだけで構いません。このチュートリアルで使用している例は次のとおりです。 @@ -225,7 +225,7 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ これは ClickHouse にデータを挿入するためのテンプレートです。ここで変数がどのように利用されているか確認できます。 -### ルールの詳細を表示 +### ルールの詳細を表示 {#view-rules-details} 「Confirm」と「View Details」をクリックします。これで、すべての設定が完了しているはずです。ルール詳細ページから、データ統合が正しく動作していることを確認できます。 @@ -234,13 +234,13 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ `temp_hum/emqx` トピックに送信されたすべての MQTT メッセージは、ClickHouse Cloud データベースに永続化されます。 -## ClickHouse へのデータ保存 +## ClickHouse へのデータ保存 {#saving-data-into-clickhouse} 温度と湿度のデータをシミュレートし、そのデータを MQTT X を介して EMQX Cloud に送信します。その後、EMQX Cloud Data Integrations を使用してデータを ClickHouse Cloud に保存します。 EMQX Cloud から ClickHouse へのワークフローを示すデータフロー図 -### MQTT メッセージを EMQX Cloud にパブリッシュする +### MQTT メッセージを EMQX Cloud にパブリッシュする {#publish-mqtt-messages-to-emqx-cloud} メッセージのパブリッシュには、任意の MQTT クライアントまたは SDK を使用できます。本チュートリアルでは、EMQ が提供するユーザーフレンドリーな MQTT クライアントアプリケーションである [MQTT X](https://mqttx.app/) を使用します。 @@ -274,13 +274,13 @@ EMQX Cloud に送信されたデータは、ルールエンジンによって処 MQTTX Publish MQTT Messages インターフェースにおけるメッセージ作成画面 -### ルール監視を確認する +### ルール監視を確認する {#view-rules-monitoring} ルール監視を開き、成功数が 1 件増えていることを確認します。 EMQX Cloud Rule Monitoring ダッシュボードにおけるメッセージ処理メトリクス -### 永続化されたデータを確認する +### 永続化されたデータを確認する {#check-the-data-persisted} ClickHouse Cloud 上のデータを確認します。理想的には、MQTTX を使って送信したデータは EMQX Cloud に送られ、ネイティブなデータ統合機能により ClickHouse Cloud のデータベースに永続化されます。 @@ -293,6 +293,6 @@ SELECT * FROM emqx.temp_hum; ClickHouse のクエリ結果で永続化された IoT データを表示している画面 -### まとめ +### まとめ {#summary} コードを一行も書くことなく、MQTT データを EMQX Cloud から ClickHouse Cloud へ送れるようになりました。EMQX Cloud と ClickHouse Cloud を使えば、インフラの運用・管理は不要になり、ClickHouse Cloud に安全に保存されたデータを活用して IoT アプリケーションの開発に専念できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md index 6ad9700b285..116e929b94e 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md @@ -25,7 +25,7 @@ import airbyte09 from '@site/static/images/integrations/data-ingestion/etl-tools import PartnerBadge from '@theme/badges/PartnerBadge'; -# AirbyteをClickHouseに接続する +# AirbyteをClickHouseに接続する {#connect-airbyte-to-clickhouse} @@ -70,7 +70,7 @@ ClickHouse用のAirbyteソースおよびデスティネーションは現在ア -## ClickHouse を送信先として追加する +## ClickHouse を送信先として追加する {#2-add-clickhouse-as-a-destination} このセクションでは、ClickHouse インスタンスを送信先として追加する方法を説明します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md index 18cf2d96176..191fae2856a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md @@ -13,7 +13,7 @@ keywords: ['apache beam', 'ストリーム処理', 'バッチ処理', 'JDBC コ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Apache Beam と ClickHouse の統合 +# Apache Beam と ClickHouse の統合 {#integrating-apache-beam-and-clickhouse} @@ -29,9 +29,9 @@ Apache Beam と ClickHouse を統合するために必要なインテグレー -## Apache Beam ClickHouse パッケージのセットアップ +## Apache Beam ClickHouse パッケージのセットアップ {#setup-of-the-apache-beam-clickhouse-package} -### パッケージのインストール +### パッケージのインストール {#package-installation} ご利用のパッケージ管理フレームワークに、次の依存関係を追加します: @@ -50,7 +50,7 @@ Apache Beam と ClickHouse を統合するために必要なインテグレー アーティファクトは [公式 Maven リポジトリ](https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-clickhouse)から入手できます。 -### コード例 +### コード例 {#code-example} 次の例では、`input.csv` という名前の CSV ファイルを `PCollection` として読み込み、定義済みのスキーマを使用して `Row` オブジェクトに変換し、`ClickHouseIO` を使用してローカルの ClickHouse インスタンスに挿入します。 @@ -133,6 +133,12 @@ public class Main { | | `Schema.TypeName#DECIMAL` | ❌ | | | | `Schema.TypeName#MAP` | ❌ | | + + + + + + ## ClickHouseIO.Write のパラメータ {#clickhouseiowrite-parameters} 次のセッターメソッドを使用して `ClickHouseIO.Write` の設定を調整できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md index c8c9a685083..b0e1ded81f7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md @@ -21,7 +21,7 @@ import bp_ck_9 from '@site/static/images/integrations/data-ingestion/etl-tools/b import PartnerBadge from '@theme/badges/PartnerBadge'; -# BladePipe を ClickHouse に接続する +# BladePipe を ClickHouse に接続する {#connect-bladepipe-to-clickhouse} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md index 0c8a16f8016..afdb75efd5a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md @@ -12,7 +12,7 @@ import TOCInline from '@theme/TOCInline'; import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# 機能と設定 +# 機能と設定 {#features-and-configurations} @@ -22,7 +22,7 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## Profile.yml の設定 +## Profile.yml の設定 {#profile-yml-configurations} dbt から ClickHouse に接続するには、`profiles.yml` ファイルに[プロファイル](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles)を追加する必要があります。ClickHouse のプロファイルは、次の構文に従います。 @@ -65,14 +65,14 @@ your_profile_name: compress_block_size: [1048576] # 圧縮有効時の圧縮ブロックサイズ ``` -### スキーマとデータベースの違い +### スキーマとデータベースの違い {#schema-vs-database} dbt モデルのリレーション識別子 `database.schema.table` は ClickHouse では互換性がありません。これは、ClickHouse が `schema` をサポートしていないためです。 そのため、`schema.table` という簡略化したアプローチを使用します。この場合の `schema` は ClickHouse のデータベースを意味します。 `default` データベースの使用は推奨されません。 -### SET ステートメントに関する警告 +### SET ステートメントに関する警告 {#set-statement-warning} 多くの環境では、すべての dbt クエリに対して ClickHouse の設定を永続させる目的で SET ステートメントを使用する方法は信頼性が低く、 予期しない失敗を引き起こす可能性があります。これは特に、ロードバランサーを介して複数ノードにクエリを分散する HTTP 接続 @@ -80,7 +80,7 @@ dbt モデルのリレーション識別子 `database.schema.table` は ClickHou したがって、事前フックの "SET" ステートメントに依存するのではなく、ベストプラクティスとして、dbt プロファイルの "custom_settings" プロパティで必要な ClickHouse の設定を構成することを推奨します。 -### `quote_columns` の設定 +### `quote_columns` の設定 {#setting-quote_columns} 警告が出ないようにするため、`dbt_project.yml` 内で `quote_columns` に値を明示的に設定してください。詳細については、[quote_columns に関するドキュメント](https://docs.getdbt.com/reference/resource-configs/quote_columns) を参照してください。 @@ -90,14 +90,14 @@ seeds: +quote_columns: false # CSV列ヘッダーにスペースが含まれている場合は `true` に設定 ``` -### ClickHouse クラスターについて +### ClickHouse クラスターについて {#about-the-clickhouse-cluster} ClickHouse クラスターを使用する場合、次の 2 点を考慮する必要があります。 * `cluster` 設定を行うこと * 特に複数の `threads` を使用している場合に、書き込み直後の読み取り一貫性を確保すること -#### クラスター設定 +#### クラスター設定 {#cluster-setting} プロファイル内の `cluster` 設定により、dbt-clickhouse は ClickHouse クラスターを対象に実行されます。プロファイルで `cluster` が設定されている場合、**すべてのモデルはデフォルトで `ON CLUSTER` 句付きで作成されます**(**Replicated** エンジンを使用するものを除く)。これには以下が含まれます。 @@ -126,7 +126,7 @@ non-replicated エンジンを使用する table および incremental マテリ モデルが `cluster` 設定なしで作成されている場合、dbt-clickhouse はその状況を検知し、そのモデルに対するすべての DDL/DML を `on cluster` 句なしで実行します。 -#### 書き込み直後の読み取り一貫性 +#### 書き込み直後の読み取り一貫性 {#read-after-write-consistency} dbt は read-after-insert 一貫性モデルに依存しています。これは、すべての操作が同じレプリカに送られることを保証できない場合、複数レプリカを持つ ClickHouse クラスターとは互換性がありません。日常的な dbt の利用においては問題が発生しないこともありますが、この保証を満たすために、クラスターの種類に応じた戦略がいくつかあります。 @@ -134,9 +134,9 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは * 自前でホストしているクラスターを使用している場合は、すべての dbt リクエストが同じ ClickHouse レプリカに送信されるようにしてください。その上にロードバランサーがある場合は、常に同じレプリカに到達できるように、`replica aware routing` / `sticky sessions` メカニズムの利用を検討してください。ClickHouse Cloud 以外のクラスターで `select_sequential_consistency = 1` 設定を追加することは[推奨されません](https://clickhouse.com/docs/operations/settings/settings#select_sequential_consistency)。 -## 機能に関する一般情報 +## 機能に関する一般情報 {#general-information-about-features} -### テーブルの一般的な設定 +### テーブルの一般的な設定 {#general-table-configurations} | Option | Description | Default if any | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | @@ -154,7 +154,7 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは | definer | `sql_security` に `definer` を設定した場合、`definer` 句で既存のユーザーまたは `CURRENT_USER` を指定する必要があります。 | | | projections | 作成する[プロジェクション](/data-modeling/projections)の一覧。[プロジェクションについて](#projections)を参照してください。 | | -#### データスキッピングインデックスについて +#### データスキッピングインデックスについて {#data-skipping-indexes} データスキッピングインデックスは `table` マテリアライゼーションでのみ利用可能です。テーブルにデータスキッピングインデックスの一覧を追加するには、`indexes` 設定を使用します。 @@ -168,7 +168,7 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは ) }} ``` -#### プロジェクションについて +#### プロジェクションについて {#projections} `projections` 設定を使用して、`table` および `distributed_table` マテリアライゼーションに [プロジェクション](/data-modeling/projections) を追加できます。 @@ -186,7 +186,7 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは **注記**: 分散テーブルでは、プロジェクションは分散プロキシテーブルではなく `_local` テーブルに適用されます。 -### サポートされているテーブルエンジン +### サポートされているテーブルエンジン {#supported-table-engines} | 種類 | 詳細 | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -197,7 +197,7 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは | EmbeddedRocksDB | [https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb](https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb) | | Hive | [https://clickhouse.com/docs/en/engines/table-engines/integrations/hive](https://clickhouse.com/docs/en/engines/table-engines/integrations/hive) | -### 実験的サポート対象のテーブルエンジン +### 実験的サポート対象のテーブルエンジン {#experimental-supported-table-engines} | Type | Details | @@ -208,7 +208,7 @@ dbt は read-after-insert 一貫性モデルに依存しています。これは 上記のいずれかのエンジンを使った dbt から ClickHouse への接続で問題が発生した場合は、 [こちら](https://github.com/ClickHouse/dbt-clickhouse/issues) から issue を報告してください。 -### モデル設定についての注意事項 +### モデル設定についての注意事項 {#a-note-on-model-settings} ClickHouse には複数の種類やレベルの「settings(設定)」があります。上記のモデル設定では、そのうち 2 種類を 設定できます。`settings` は、`CREATE TABLE/VIEW` タイプの DDL 文で使用される `SETTINGS` @@ -218,7 +218,7 @@ ClickHouse の設定は数百個あり、「テーブル」設定と「ユーザ 必ずしも明確でないものもあります(ただし後者は一般的に `system.settings` テーブルで確認できます)。 一般的にはデフォルト値の使用が推奨され、これらのプロパティを利用する場合は、事前に十分な調査とテストを行ってください。 -### カラム設定 +### カラム設定 {#column-configuration} > ***NOTE:*** 以下のカラム設定オプションを利用するには、[model contracts](https://docs.getdbt.com/docs/collaborate/govern/model-contracts) が適用・強制されている必要があります。 @@ -227,7 +227,7 @@ ClickHouse の設定は数百個あり、「テーブル」設定と「ユーザ | codec | カラムの DDL 内で `CODEC()` に渡される引数からなる文字列。例: `codec: "Delta, ZSTD"` は `CODEC(Delta, ZSTD)` として解釈されます。 | | | ttl | カラムの DDL 内で TTL ルールを定義する [TTL(time-to-live)式](https://clickhouse.com/docs/guides/developer/ttl) からなる文字列。例: `ttl: ts + INTERVAL 1 DAY` は `TTL ts + INTERVAL 1 DAY` として解釈されます。 | | -#### スキーマ設定の例 +#### スキーマ設定の例 {#example-of-schema-configuration} ```yaml models: @@ -245,7 +245,7 @@ models: ttl: ts + INTERVAL 1 DAY ``` -#### 複合型の追加 +#### 複合型の追加 {#adding-complex-types} dbt は、モデルの定義に用いられた SQL を解析して、各カラムのデータ型を自動的に判定します。ただし、場合によってはこの処理では正確にデータ型を判定できず、コントラクトの `data_type` プロパティで指定された型と矛盾が生じることがあります。これに対処するため、モデルの SQL 内で `CAST()` 関数を使用して、意図した型を明示的に指定することを推奨します。例えば、次のようになります。 @@ -270,9 +270,9 @@ group by event_type ``` -## 機能 +## 機能 {#features} -### マテリアライゼーション: view +### マテリアライゼーション: view {#materialization-view} dbt モデルは [ClickHouse view](https://clickhouse.com/docs/en/sql-reference/table-functions/view/) として作成し、次の構文で設定できます。 @@ -291,7 +291,7 @@ models: {{ config(materialized = "view") }} ``` -### マテリアライゼーション: テーブル +### マテリアライゼーション: テーブル {#materialization-table} dbt モデルは [ClickHouse のテーブル](https://clickhouse.com/docs/en/operations/system-tables/tables/) として作成し、 次の構文で設定できます。 @@ -320,7 +320,7 @@ models: ) }} ``` -### マテリアライゼーション: incremental +### マテリアライゼーション: incremental {#materialization-incremental} テーブルモデルは、dbt が実行されるたびに再構築されます。これは、結果セットが大きい場合や変換が複雑な場合には、実行不可能であったり非常に高コストになることがあります。この課題に対処し、ビルド時間を短縮するために、dbt モデルをインクリメンタルな ClickHouse テーブルとして作成し、次の構文を用いて設定できます。 @@ -352,7 +352,7 @@ models: ) }} ``` -#### 設定 +#### 設定 {#configurations} このマテリアライゼーションタイプに特有の設定は以下のとおりです。 @@ -363,11 +363,11 @@ models: | `incremental_strategy` | インクリメンタルマテリアライゼーションに使用する戦略。`delete+insert`、`append`、`insert_overwrite`、`microbatch` がサポートされています。戦略の詳細については[こちら](/integrations/dbt/features-and-configurations#incremental-model-strategies)を参照してください。 | 任意 (デフォルト: 'default') | | `incremental_predicates` | インクリメンタルマテリアライゼーションに適用される追加条件(`delete+insert` 戦略にのみ適用されます) | 任意 | -#### インクリメンタルモデルの戦略 +#### インクリメンタルモデルの戦略 {#incremental-model-strategies} `dbt-clickhouse` は 3 つのインクリメンタルモデル戦略をサポートしています。 -##### デフォルト(レガシー)戦略 +##### デフォルト(レガシー)戦略 {#default-legacy-strategy} これまで ClickHouse は、非同期の「mutations」という形式による、限定的な更新および削除のサポートしか持っていませんでした。 期待される dbt の挙動をエミュレートするために、 @@ -377,7 +377,7 @@ dbt-clickhouse はデフォルトで、影響を受けない(削除されて 処理が完了する前に問題が発生した場合でも元のリレーションを保持する唯一の戦略ですが、 元のテーブルの完全なコピーを伴うため、非常にコストが高く、実行が遅くなる可能性があります。 -##### Delete+Insert 戦略 +##### Delete+Insert 戦略 {#delete-insert-strategy} ClickHouse はバージョン 22.8 で実験的機能として「lightweight deletes」を追加しました。Lightweight deletes は @@ -456,7 +456,7 @@ WHERE 句/フィルタで除外される場合には、最も高速なアプ この戦略を利用するには、モデル設定で `partition_by` を指定している必要があります。モデル設定におけるその他の戦略固有の パラメータはすべて無視されます。 -### Materialization: materialized_view (Experimental) +### Materialization: materialized_view (Experimental) {#materialized-view} `materialized_view` マテリアライゼーションは、既存の(ソース)テーブルを対象とした `SELECT` 文である必要があります。アダプターはモデル名を用いた ターゲットテーブルと、`_mv` という名前の ClickHouse MATERIALIZED VIEW を作成します。PostgreSQL と異なり、ClickHouse のマテリアライズドビューは @@ -489,7 +489,7 @@ select a,b,c from {{ source('raw', 'table_2') }} > 次の警告が表示されます: > `Warning - Table was detected with the same pattern as model name but was not found in this run. In case it is a renamed mv that was previously part of this model, drop it manually (!!!) ` -#### データのキャッチアップ +#### データのキャッチアップ {#data-catch-up} 現在、マテリアライズドビュー (MV) を作成する場合、MV 自体が作成される前に、ターゲットテーブルはまず履歴データで埋められます。 @@ -506,7 +506,7 @@ MV の作成時に履歴データをプリロードしたくない場合は、`c )}} ``` -#### リフレッシュ可能なマテリアライズドビュー +#### リフレッシュ可能なマテリアライズドビュー {#refreshable-materialized-views} [Refreshable Materialized View](https://clickhouse.com/docs/en/materialized-view/refreshable-materialized-view) を利用するには、 MV モデル内で必要に応じて次の設定を調整してください(これらの設定はすべて、 @@ -538,19 +538,19 @@ refreshable 設定オブジェクト内に記述します)。 }} ``` -#### 制限事項 +#### 制限事項 {#limitations} * 依存関係を持つリフレッシュ可能なマテリアライズドビュー (MV) を ClickHouse で作成する際、指定した依存関係が作成時点で存在しなくても、ClickHouse はエラーをスローしません。代わりに、そのリフレッシュ可能な MV は非アクティブ状態のままとなり、依存関係が満たされるまで更新処理やリフレッシュを開始しません。この挙動は設計によるものですが、必要な依存関係への対応が遅れた場合、データの利用可能性に遅延が生じる可能性があります。ユーザーは、リフレッシュ可能なマテリアライズドビューを作成する前に、すべての依存関係が正しく定義され、実在していることを確認するよう推奨されます。 * 現時点では、MV とその依存関係の間に実際の「dbt linkage」は存在しないため、作成順序は保証されません。 * リフレッシュ可能機能は、同一のターゲットモデルを指す複数の MV に対してはテストされていません。 -### マテリアライゼーション: dictionary(実験的) +### マテリアライゼーション: dictionary(実験的) {#materialization-dictionary} ClickHouse の dictionary 用マテリアライゼーションの実装例については、 [https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py](https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py) のテストを参照してください。 -### マテリアライゼーション: distributed_table(実験的) +### マテリアライゼーション: distributed_table(実験的) {#materialization-distributed-table} 分散テーブルは次の手順で作成されます: @@ -563,7 +563,7 @@ ClickHouse の dictionary 用マテリアライゼーションの実装例につ * 下流のインクリメンタルマテリアライゼーション処理が正しく実行されるようにするため、dbt-clickhouse のクエリには自動的に `insert_distributed_sync = 1` 設定が含まれるようになりました。これにより、一部の分散テーブルへの INSERT が想定より遅くなる可能性があります。 -#### 分散テーブルモデルの例 +#### 分散テーブルモデルの例 {#distributed-table-model-example} ```sql {{ @@ -579,7 +579,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### 生成されたマイグレーション +#### 生成されたマイグレーション {#distributed-table-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -599,7 +599,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### materialization: distributed_incremental (experimental) +### materialization: distributed_incremental (experimental) {#materialization-distributed-incremental} 分散テーブルと同様の考え方に基づくインクリメンタルモデルであり、主な難しさはすべてのインクリメンタル戦略を正しく処理することにあります。 @@ -610,7 +610,7 @@ CREATE TABLE db.table on cluster cluster ( 分散テーブルはデータを保持しないため、置き換えられるのはシャードテーブルのみです。 分散テーブルが再読み込みされるのは、`full_refresh` モードが有効になっている場合か、テーブル構造が変更された可能性がある場合のみです。 -#### 分散インクリメンタルモデルの例 +#### 分散インクリメンタルモデルの例 {#distributed-incremental-model-example} ```sql @@ -627,7 +627,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### 自動生成されたマイグレーション +#### 自動生成されたマイグレーション {#distributed-incremental-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -646,7 +646,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### スナップショット +### スナップショット {#snapshot} dbt のスナップショット機能を使用すると、可変なモデルに対する変更を時間の経過とともに記録できます。これにより、アナリストはモデルに対して、モデルの以前の状態を「過去にさかのぼって」確認できる時点指定クエリを実行できます。この機能は ClickHouse コネクタでサポートされており、次の構文で設定します。 @@ -665,7 +665,7 @@ dbt のスナップショット機能を使用すると、可変なモデルに 設定の詳細については、[snapshot configs](https://docs.getdbt.com/docs/build/snapshots#snapshot-configs) のリファレンスページを参照してください。 -### コントラクトと制約 +### コントラクトと制約 {#contracts-and-constraints} 厳密なカラム型コントラクトのみがサポートされます。たとえば、UInt32 カラム型のコントラクトがある場合、モデルが UInt64 もしくはその他の整数型を返すと失敗します。 @@ -673,9 +673,9 @@ ClickHouse は、テーブル/モデル全体に対する `CHECK` 制約*の およびカラムレベルの CHECK 制約はサポートされません。 (プライマリキー/ORDER BY キーについては ClickHouse のドキュメントを参照してください。) -### 追加の ClickHouse マクロ +### 追加の ClickHouse マクロ {#additional-clickhouse-macros} -#### モデルのマテリアライゼーション用ユーティリティマクロ +#### モデルのマテリアライゼーション用ユーティリティマクロ {#model-materialization-utility-macros} ClickHouse 固有のテーブルおよびビューを作成しやすくするために、次のマクロが含まれています。 @@ -692,7 +692,7 @@ ClickHouse 固有のテーブルおよびビューを作成しやすくするた * `ttl_config` -- `ttl` モデル設定プロパティを使用して、ClickHouse のテーブル TTL 式を割り当てます。デフォルトでは TTL は割り当てられません。 -#### s3Source ヘルパーマクロ +#### s3Source ヘルパーマクロ {#s3source-helper-macro} `s3source` マクロは、ClickHouse の S3 テーブル関数を使用して S3 から直接 ClickHouse データを選択する処理を簡素化します。 このマクロは、 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md index 2522503b2af..d9fbf9ae53c 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md @@ -20,7 +20,7 @@ import dbt_07 from '@site/static/images/integrations/data-ingestion/etl-tools/db import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# ガイド +# ガイド {#guides} @@ -39,13 +39,13 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## セットアップ +## セットアップ {#setup} 環境を準備するには、[dbt と ClickHouse アダプターのセットアップ](/integrations/dbt) セクションの手順に従ってください。 **重要: 以下は python 3.9 環境で検証されています。** -### ClickHouse の準備 +### ClickHouse の準備 {#prepare-clickhouse} dbt はリレーショナル性の高いデータのモデリングに優れています。例として、次のようなリレーショナルスキーマを持つ小さな IMDB データセットを用意しています。このデータセットは [relational dataset repository](https://relational.fit.cvut.cz/dataset/IMDb) に由来します。これは dbt で一般的に使われるスキーマと比べると単純ですが、扱いやすいサンプルになっています。 @@ -672,7 +672,7 @@ SELECT * FROM imdb_dbt.actor_summary ORDER BY num_movies DESC LIMIT 2; +------+-------------------+----------+------------------+------+---------+-------------------+ ``` -### 内部動作 +### 内部動作 {#internals} 上記の増分更新を実現するために実行されたステートメントは、ClickHouse のクエリログを参照することで確認できます。 @@ -694,7 +694,7 @@ AND event_time > subtractMinutes(now(), 15) ORDER BY event_time LIMIT 100; この戦略は、非常に大きなモデルでは問題が発生する可能性があります。詳細については [Limitations](/integrations/dbt#limitations) を参照してください。 -### Append Strategy(挿入のみモード) +### Append Strategy(挿入のみモード) {#append-strategy-inserts-only-mode} インクリメンタルモデルにおける大規模データセットの制約を回避するために、アダプターは dbt の設定パラメータ `incremental_strategy` を使用します。これは `append` に設定できます。これを設定すると、更新された行はターゲットテーブル(`imdb_dbt.actor_summary`)に直接挿入され、一時テーブルは作成されません。 注意: Append only モードでは、データが不変であるか、重複を許容できる必要があります。更新された行をサポートするインクリメンタルテーブルモデルが必要な場合、このモードは使用しないでください。 @@ -796,7 +796,7 @@ WHERE id > (SELECT max(id) FROM imdb_dbt.actor_summary) OR updated_at > (SELECT この実行では、新しい行だけが直接 `imdb_dbt.actor_summary` テーブルに追加され、テーブルの作成は行われません。 -### 削除および挿入モード(実験的) +### 削除および挿入モード(実験的) {#deleteinsert-mode-experimental} 歴史的には、ClickHouse は非同期の [Mutations](/sql-reference/statements/alter/index.md) としてのみ、更新および削除を限定的にサポートしていました。これらは非常に I/O 負荷が高く、一般的には避けるべきです。 @@ -821,7 +821,7 @@ This process is shown below: 軽量な delete インクリメンタル -### insert_overwrite mode (experimental) +### insert_overwrite mode (experimental) {#insert_overwrite-mode-experimental} Performs the following steps: @@ -840,7 +840,7 @@ This approach has the following advantages: insert overwrite インクリメンタル -## スナップショットの作成 +## スナップショットの作成 {#creating-a-snapshot} dbt のスナップショット機能を使用すると、更新可能なモデルに対する変更を時間の経過とともに記録できます。これにより、アナリストはモデルに対して任意時点のクエリを実行し、モデルの過去の状態を「遡って」確認できるようになります。これは、行が有効であった期間を記録する from 日付列および to 日付列を持つ [タイプ 2 のゆっくり変化する次元 (Slowly Changing Dimensions)](https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row) を使用して実現されます。この機能は ClickHouse アダプターでサポートされており、以下に示します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md index 8439abea8c7..f228f238aba 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md @@ -95,9 +95,9 @@ ClickHouse 向けの[現在のアダプタ](https://github.com/silentsokolov/dbt -## dbt と ClickHouse アダプターのセットアップ +## dbt と ClickHouse アダプターのセットアップ {#setup-of-dbt-and-the-clickhouse-adapter} -### dbt-core と dbt-clickhouse のインストール +### dbt-core と dbt-clickhouse のインストール {#install-dbt-core-and-dbt-clickhouse} dbt のコマンドラインインターフェイス (CLI) のインストール方法にはいくつかの選択肢があり、詳細は[こちら](https://docs.getdbt.com/dbt-cli/install/overview)に記載されています。dbt と dbt-clickhouse の両方をインストールするには、`pip` の利用を推奨します。 @@ -105,7 +105,7 @@ dbt のコマンドラインインターフェイス (CLI) のインストール pip install dbt-core dbt-clickhouse ``` -### ClickHouse インスタンスへの接続情報を dbt に提供する +### ClickHouse インスタンスへの接続情報を dbt に提供する {#provide-dbt-with-the-connection-details-for-our-clickhouse-instance} `~/.dbt/profiles.yml` ファイル内で `clickhouse-service` プロファイルを構成し、`schema`、`host`、`port`、`user`、`password` プロパティを指定します。接続構成オプションの全一覧は、[機能と設定](/integrations/dbt/features-and-configurations) ページに記載されています。 @@ -125,7 +125,7 @@ clickhouse-service: secure: True # TLS(ネイティブプロトコル)またはHTTPS(httpプロトコル)を使用 ``` -### dbt プロジェクトを作成する +### dbt プロジェクトを作成する {#create-a-dbt-project} これで、このプロファイルを既存のいずれかのプロジェクトで使用することも、次の手順で新しいプロジェクトを作成することもできます。 @@ -139,17 +139,17 @@ dbt init project_name profile: 'clickhouse-service' ``` -### 接続テスト +### 接続テスト {#test-connection} CLI で `dbt debug` を実行し、dbt が ClickHouse に接続できるかどうかを確認します。レスポンスに `Connection test: [OK connection ok]` が含まれていることを確認し、接続が成功していることを確かめてください。 ClickHouse と dbt の連携方法の詳細については、[ガイドページ](/integrations/dbt/guides) を参照してください。 -### モデルのテストとデプロイ (CI/CD) +### モデルのテストとデプロイ (CI/CD) {#testing-and-deploying-your-models-ci-cd} dbt プロジェクトをテストおよびデプロイする方法は多数あります。dbt では、[ベストプラクティスとされるワークフロー](https://docs.getdbt.com/best-practices/best-practice-workflows#pro-tips-for-workflows) や [CI ジョブ](https://docs.getdbt.com/docs/deploy/ci-jobs) に関する提案を提供しています。ここではいくつかの戦略について説明しますが、これらの戦略はユースケースに合わせて大きく調整する必要がある場合がある点に注意してください。 -#### シンプルなデータテストおよびユニットテストによる CI/CD +#### シンプルなデータテストおよびユニットテストによる CI/CD {#ci-with-simple-data-tests-and-unit-tests} CI パイプラインを手軽に立ち上げる方法の 1 つは、ジョブ内で ClickHouse クラスターを起動し、そのクラスターに対してモデルを実行することです。モデルを実行する前に、このクラスターにデモデータを挿入できます。[seed](https://docs.getdbt.com/reference/commands/seed) を使って、本番データのサブセットをステージング環境に投入することもできます。 @@ -157,7 +157,7 @@ CI パイプラインを手軽に立ち上げる方法の 1 つは、ジョブ CD ステップは、本番の ClickHouse クラスターに対して `dbt build` を実行するだけのシンプルなものにできます。 -#### より包括的な CI/CD ステージ: 最新のデータを使用し、影響を受けたモデルのみをテスト +#### より包括的な CI/CD ステージ: 最新のデータを使用し、影響を受けたモデルのみをテスト {#more-complete-ci-stage} 一般的な戦略として、変更されたモデル (およびその上流・下流の依存関係) のみを再デプロイする [Slim CI](https://docs.getdbt.com/best-practices/best-practice-workflows#run-only-modified-models-to-test-changes-slim-ci) ジョブを利用する方法があります。このアプローチでは、本番実行の成果物 (例: [dbt manifest](https://docs.getdbt.com/reference/artifacts/manifest-json)) を利用して、プロジェクトの実行時間を短縮し、環境間でスキーマのずれが生じないようにします。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md index 6207100b4fe..c64f4515c65 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md @@ -10,7 +10,7 @@ doc_type: 'guide' import PartnerBadge from '@theme/badges/PartnerBadge'; -# dlt を ClickHouse に接続する +# dlt を ClickHouse に接続する {#connect-dlt-to-clickhouse} @@ -18,9 +18,9 @@ import PartnerBadge from '@theme/badges/PartnerBadge'; -## ClickHouse と併せて dlt をインストールする +## ClickHouse と併せて dlt をインストールする {#install-dlt-with-clickhouse} -### ClickHouse の依存関係付きで `dlt` ライブラリをインストールするには: +### ClickHouse の依存関係付きで `dlt` ライブラリをインストールするには: {#to-install-the-dlt-library-with-clickhouse-dependencies} ```bash pip install "dlt[clickhouse]" @@ -99,7 +99,7 @@ ClickHouseサーバーが`http_port`で指定されたポートでHTTP接続を ```bash -# tomlファイルの先頭、セクション開始前に記述してください。 +# tomlファイルの先頭、セクション開始前に記述してください。 {#keep-it-at-the-top-of-your-toml-file-before-any-section-starts} destination.clickhouse.credentials="clickhouse://dlt:Dlt*12345789234567@localhost:9000/dlt?secure=1" ``` @@ -157,7 +157,7 @@ ClickHouse は、以下の
GCS テーブル関数 によって自動的に処理されます。 @@ -241,10 +241,10 @@ dlt はこれらの認証情報を ClickHouse に渡し、認証および GCS * filesystem destination を S3 互換モードで GCS と連携できるようにする * Google Cloud Storage ステージングエリアのサポート -### dbt サポート +### dbt サポート {#dbt-support} dbt との連携は、一般に dbt-clickhouse を通じてサポートされています。 -### `dlt` state の同期 +### `dlt` state の同期 {#syncing-of-dlt-state} この destination は、dlt state の同期を完全にサポートしています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md index 254b24d6158..9f1ed82585a 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md @@ -14,7 +14,7 @@ keywords: ['fivetran', 'データ移動', 'ETL', 'ClickHouse 宛先', '自動化 import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Fivetran と ClickHouse Cloud +# Fivetran と ClickHouse Cloud {#fivetran-and-clickhouse-cloud} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md index 05f92a7548d..bd4aff29010 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md @@ -11,7 +11,7 @@ integration: - category: 'data_ingestion' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import nifi01 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_01.png'; import nifi02 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_02.png'; @@ -31,7 +31,7 @@ import nifi15 from '@site/static/images/integrations/data-ingestion/etl-tools/ni import CommunityMaintainedBadge from '@theme/badges/CommunityMaintained'; -# Apache NiFiをClickHouseに接続する +# Apache NiFiをClickHouseに接続する {#connect-apache-nifi-to-clickhouse} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md index 934779a27bd..fad3e4819f6 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md @@ -18,8 +18,7 @@ import vector01 from '@site/static/images/integrations/data-ingestion/etl-tools/ import vector02 from '@site/static/images/integrations/data-ingestion/etl-tools/vector_02.png'; import PartnerBadge from '@theme/badges/PartnerBadge'; - -# Vector と ClickHouse の統合 +# Vector と ClickHouse の統合 {#integrating-vector-with-clickhouse} @@ -31,214 +30,202 @@ ClickHouse は、優れた圧縮率(ログでは最大 [170x](https://clickhou **前提条件:** -- ClickHouse がすでに稼働していること -- Vector がインストールされていること +* ClickHouse がすでに稼働していること +* Vector がインストールされていること + ## データベースとテーブルを作成する {#1-create-a-database-and-table} + ログイベントを保存するためのテーブルを定義します。 -## データベースとテーブルを作成する - -ログイベントを保存するためのテーブルを定義します。 - -1. まず、`nginxdb` という名前の新しいデータベースを作成します: - -```sql -CREATE DATABASE IF NOT EXISTS nginxdb -``` - -2. ログイベント全体を1つの文字列として挿入します。もちろん、この形式はログデータを分析するのに適したものではありませんが、その点は後ほど***マテリアライズドビュー***を使って解決していきます。 - -```sql -CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( - message String -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -:::note -**ORDER BY** は、まだ主キーが不要なため、空のタプルである **tuple()** に設定されています。 -::: - - -## Nginx を構成する - -このステップでは、Nginx のログ出力を構成する方法を説明します。 - -1. 次の `access_log` プロパティは、ログを **combined** 形式で `/var/log/nginx/my_access.log` に出力します。 - この値は `nginx.conf` ファイルの `http` セクションに記述します: - -```bash -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - access_log /var/log/nginx/my_access.log combined; - sendfile on; - keepalive_timeout 65; - include /etc/nginx/conf.d/*.conf; -} -``` - -2. `nginx.conf` を変更した場合は、必ず Nginx を再起動してください。 - -3. Web サーバー上のページにアクセスして、アクセスログにログイベントをいくつか生成してください。 - **combined** 形式のログは次のようになります。 - -```bash -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - + 1. まず、`nginxdb` という名前の新しいデータベースを作成します: -## Vector を設定する + ```sql + CREATE DATABASE IF NOT EXISTS nginxdb + ``` -Vector はログ、メトリクス、トレース(**sources** と呼ばれます)を収集・変換・ルーティングし、ClickHouse を含む多数の異なるベンダー(**sinks** と呼ばれます)へ送信します。 -Source と sink は **vector.toml** という名前の設定ファイルで定義します。 + 2. ログイベント全体を1つの文字列として挿入します。もちろん、この形式はログデータを分析するのに適したものではありませんが、その点は後ほど***マテリアライズドビュー***を使って解決していきます。 -1. 次の **vector.toml** ファイルでは、**my_access.log** の末尾を tail する **file** 型の **source** と、上で定義した **access_logs** テーブルを **sink** として定義しています。 + ```sql + CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( + message String + ) + ENGINE = MergeTree() + ORDER BY tuple() + ``` -```bash -[sources.nginx_logs] -type = "file" -include = [ "/var/log/nginx/my_access.log" ] -read_from = "end" + :::note + **ORDER BY** は、まだ主キーが不要なため、空のタプルである **tuple()** に設定されています。 + ::: -[sinks.clickhouse] -type = "clickhouse" -inputs = ["nginx_logs"] -endpoint = "http://clickhouse-server:8123" -database = "nginxdb" -table = "access_logs" -skip_unknown_fields = true -``` + ## Nginx を構成する {#2--configure-nginx} -2. 上記の構成を使用して Vector を起動します。ソースおよびシンクの定義についての詳細は、Vector の[ドキュメント](https://vector.dev/docs/)を参照してください。 + このステップでは、Nginx のログ出力を構成する方法を説明します。 -3. 次のクエリを実行して、アクセスログが ClickHouse に取り込まれていることを確認します。テーブル内にアクセスログが表示されるはずです。 + 1. 次の `access_log` プロパティは、ログを **combined** 形式で `/var/log/nginx/my_access.log` に出力します。 + この値は `nginx.conf` ファイルの `http` セクションに記述します: -```sql -SELECT * FROM nginxdb.access_logs -``` + ```bash + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + access_log /var/log/nginx/my_access.log combined; + sendfile on; + keepalive_timeout 65; + include /etc/nginx/conf.d/*.conf; + } + ``` -テーブル形式で ClickHouse のログを表示する + 2. `nginx.conf` を変更した場合は、必ず Nginx を再起動してください。 + 3. Web サーバー上のページにアクセスして、アクセスログにログイベントをいくつか生成してください。 + **combined** 形式のログは次のようになります。 -## ログをパースする + ```bash + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` -ログを ClickHouse に保存できるのは有用ですが、各イベントを 1 つの文字列として保存するだけでは、十分なデータ分析は行えません。 -次に、[マテリアライズドビュー](/materialized-view/incremental-materialized-view) を使用してログイベントをどのようにパースするかを見ていきます。 + ## Vector を設定する {#3-configure-vector} -**マテリアライズドビュー** は、SQL における INSERT トリガーと同様に機能します。ソーステーブルにデータ行が挿入されると、マテリアライズドビューはそれらの行を変換し、その結果をターゲットテーブルに挿入します。 -マテリアライズドビューを設定して、**access_logs** 内のログイベントのパース済み表現を生成できます。 -そのようなログイベントの 1 つの例を以下に示します。 + Vector はログ、メトリクス、トレース(**sources** と呼ばれます)を収集・変換・ルーティングし、ClickHouse を含む多数の異なるベンダー(**sinks** と呼ばれます)へ送信します。 + Source と sink は **vector.toml** という名前の設定ファイルで定義します。 -```bash -192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - -ClickHouse には、上記の文字列を解析するためのさまざまな関数があります。[`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) 関数は、文字列を空白文字で分割し、各トークンを配列で返します。 -動作を確認するには、次のコマンドを実行します。 - -```sql title="Query" -SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -```text title="Response" -["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] -``` - -いくつかの文字列には余分な文字が含まれており、ユーザーエージェント(ブラウザ情報)は解析する必要はありませんでしたが、 -結果として得られた配列は、必要なものにかなり近い形になっています。 - -`splitByWhitespace` と同様に、[`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) 関数は、正規表現に基づいて文字列を配列に分割します。 -次のコマンドを実行します。2 つの文字列が返されます。 - -```sql -SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -2 番目に返される文字列が、ログから正常に解析されたユーザーエージェントであることに注目してください。 - -```text -["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] -``` - -最終的な `CREATE MATERIALIZED VIEW` コマンドを見る前に、データをクリーンアップするために使用する、いくつかの関数をさらに確認しておきます。 -たとえば、`RequestMethod` の値は先頭に不要な二重引用符が付いた `"GET` になっています。 -この二重引用符を削除するには、[`trimBoth`(エイリアス `trim`)](/sql-reference/functions/string-functions#trimBoth) 関数を使用できます。 - -```sql -SELECT trim(LEADING '"' FROM '"GET') -``` - -時刻文字列は先頭に角括弧が付いており、ClickHouse が日付として解析できる形式にもなっていません。 -しかし、区切り文字をコロン(**:**)からカンマ(**,**)に変更すると、問題なく解析できるようになります: - -```sql -SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) -``` - - -これでマテリアライズドビューを定義する準備が整いました。 -以下の定義には `POPULATE` 句が含まれており、これは **access_logs** に既に存在する行がすぐに処理され、その場で挿入されることを意味します。 -次の SQL ステートメントを実行してください: - -```sql -CREATE MATERIALIZED VIEW nginxdb.access_logs_view -( - RemoteAddr String, - Client String, - RemoteUser String, - TimeLocal DateTime, - RequestMethod String, - Request String, - HttpVersion String, - Status Int32, - BytesSent Int64, - UserAgent String -) -ENGINE = MergeTree() -ORDER BY RemoteAddr -POPULATE AS -WITH - splitByWhitespace(message) as split, - splitByRegexp('\S \d+ "([^"]*)"', message) as referer -SELECT - split[1] AS RemoteAddr, - split[2] AS Client, - split[3] AS RemoteUser, - parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, - trim(LEADING '"' FROM split[6]) AS RequestMethod, - split[7] AS Request, - trim(TRAILING '"' FROM split[8]) AS HttpVersion, - split[9] AS Status, - split[10] AS BytesSent, - trim(BOTH '"' from referer[2]) AS UserAgent -FROM - (SELECT message FROM nginxdb.access_logs) -``` - -続いて、正しく動作しているか確認します。 -アクセスログが列ごとにきれいにパースされて表示されるはずです: - -```sql -SELECT * FROM nginxdb.access_logs_view -``` - -パース済みの ClickHouse ログをテーブル形式で表示 - -:::note -上記のレッスンではデータを 2 つのテーブルに保存しましたが、最初の `nginxdb.access_logs` テーブルを [`Null`](/engines/table-engines/special/null) テーブルエンジンを使用するように変更することもできます。 -パース済みデータは変わらず `nginxdb.access_logs_view` テーブルに格納されますが、生のデータはテーブルには保存されません。 -::: + 1. 次の **vector.toml** ファイルでは、**my_access.log** の末尾を tail する **file** 型の **source** と、上で定義した **access_logs** テーブルを **sink** として定義しています。 + ```bash + [sources.nginx_logs] + type = "file" + include = [ "/var/log/nginx/my_access.log" ] + read_from = "end" + + [sinks.clickhouse] + type = "clickhouse" + inputs = ["nginx_logs"] + endpoint = "http://clickhouse-server:8123" + database = "nginxdb" + table = "access_logs" + skip_unknown_fields = true + ``` + + 2. 上記の構成を使用して Vector を起動します。ソースおよびシンクの定義についての詳細は、Vector の[ドキュメント](https://vector.dev/docs/)を参照してください。 + + 3. 次のクエリを実行して、アクセスログが ClickHouse に取り込まれていることを確認します。テーブル内にアクセスログが表示されるはずです。 + + ```sql + SELECT * FROM nginxdb.access_logs + ``` + + テーブル形式で ClickHouse のログを表示する + + ## ログをパースする {#4-parse-the-logs} + + ログを ClickHouse に保存できるのは有用ですが、各イベントを 1 つの文字列として保存するだけでは、十分なデータ分析は行えません。 + 次に、[マテリアライズドビュー](/materialized-view/incremental-materialized-view) を使用してログイベントをどのようにパースするかを見ていきます。 + + **マテリアライズドビュー** は、SQL における INSERT トリガーと同様に機能します。ソーステーブルにデータ行が挿入されると、マテリアライズドビューはそれらの行を変換し、その結果をターゲットテーブルに挿入します。 + マテリアライズドビューを設定して、**access_logs** 内のログイベントのパース済み表現を生成できます。 + そのようなログイベントの 1 つの例を以下に示します。 + + ```bash + 192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` + + ClickHouse には、上記の文字列を解析するためのさまざまな関数があります。[`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) 関数は、文字列を空白文字で分割し、各トークンを配列で返します。 + 動作を確認するには、次のコマンドを実行します。 + + ```sql title="Query" + SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + ```text title="Response" + ["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] + ``` + + いくつかの文字列には余分な文字が含まれており、ユーザーエージェント(ブラウザ情報)は解析する必要はありませんでしたが、 + 結果として得られた配列は、必要なものにかなり近い形になっています。 + + `splitByWhitespace` と同様に、[`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) 関数は、正規表現に基づいて文字列を配列に分割します。 + 次のコマンドを実行します。2 つの文字列が返されます。 + + ```sql + SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + 2 番目に返される文字列が、ログから正常に解析されたユーザーエージェントであることに注目してください。 + + ```text + ["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] + ``` + + 最終的な `CREATE MATERIALIZED VIEW` コマンドを見る前に、データをクリーンアップするために使用する、いくつかの関数をさらに確認しておきます。 + たとえば、`RequestMethod` の値は先頭に不要な二重引用符が付いた `"GET` になっています。 + この二重引用符を削除するには、[`trimBoth`(エイリアス `trim`)](/sql-reference/functions/string-functions#trimBoth) 関数を使用できます。 + + ```sql + SELECT trim(LEADING '"' FROM '"GET') + ``` + + 時刻文字列は先頭に角括弧が付いており、ClickHouse が日付として解析できる形式にもなっていません。 + しかし、区切り文字をコロン(**:**)からカンマ(**,**)に変更すると、問題なく解析できるようになります: + + ```sql + SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) + ``` + + これでマテリアライズドビューを定義する準備が整いました。 + 以下の定義には `POPULATE` 句が含まれており、これは **access_logs** に既に存在する行がすぐに処理され、その場で挿入されることを意味します。 + 次の SQL ステートメントを実行してください: + + ```sql + CREATE MATERIALIZED VIEW nginxdb.access_logs_view + ( + RemoteAddr String, + Client String, + RemoteUser String, + TimeLocal DateTime, + RequestMethod String, + Request String, + HttpVersion String, + Status Int32, + BytesSent Int64, + UserAgent String + ) + ENGINE = MergeTree() + ORDER BY RemoteAddr + POPULATE AS + WITH + splitByWhitespace(message) as split, + splitByRegexp('\S \d+ "([^"]*)"', message) as referer + SELECT + split[1] AS RemoteAddr, + split[2] AS Client, + split[3] AS RemoteUser, + parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, + trim(LEADING '"' FROM split[6]) AS RequestMethod, + split[7] AS Request, + trim(TRAILING '"' FROM split[8]) AS HttpVersion, + split[9] AS Status, + split[10] AS BytesSent, + trim(BOTH '"' from referer[2]) AS UserAgent + FROM + (SELECT message FROM nginxdb.access_logs) + ``` + + 続いて、正しく動作しているか確認します。 + アクセスログが列ごとにきれいにパースされて表示されるはずです: + + ```sql + SELECT * FROM nginxdb.access_logs_view + ``` + + パース済みの ClickHouse ログをテーブル形式で表示 + + :::note + 上記のレッスンではデータを 2 つのテーブルに保存しましたが、最初の `nginxdb.access_logs` テーブルを [`Null`](/engines/table-engines/special/null) テーブルエンジンを使用するように変更することもできます。 + パース済みデータは変わらず `nginxdb.access_logs_view` テーブルに格納されますが、生のデータはテーブルには保存されません。 + ::: > シンプルなインストールと短時間の設定だけで利用できる Vector を使うと、Nginx サーバーからのログを ClickHouse のテーブルに送信できます。マテリアライズドビューを使用すれば、それらのログを列にパースして、より簡単に分析できるようにできます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md index 3bcf12710e0..0e0d4eafddb 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md @@ -8,13 +8,13 @@ doc_type: 'guide' keywords: ['Google Cloud Storage ClickHouse', 'GCS ClickHouse 統合', 'GCS バックエンド MergeTree', 'ClickHouse GCS ストレージ', 'Google Cloud ClickHouse'] --- -import BucketDetails from '@site/docs/_snippets/_GCS_authentication_and_bucket.md'; +import BucketDetails from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_GCS_authentication_and_bucket.md'; import Image from '@theme/IdealImage'; import GCS_examine_bucket_1 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-1.png'; import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-2.png'; -# Google Cloud Storage を ClickHouse と統合する +# Google Cloud Storage を ClickHouse と統合する {#integrate-google-cloud-storage-with-clickhouse} :::note [Google Cloud](https://cloud.google.com) 上の ClickHouse Cloud を利用している場合、このページは対象外です。サービスはすでに [Google Cloud Storage](https://cloud.google.com/storage) を使用しているためです。GCS から `SELECT` または `INSERT` でデータを扱いたい場合は、[`gcs` テーブル関数](/sql-reference/table-functions/gcs) を参照してください。 @@ -24,13 +24,13 @@ ClickHouse は、ストレージとコンピュートを分離したいユーザ -## GCS バックエンドの MergeTree +## GCS バックエンドの MergeTree {#gcs-backed-mergetree} -### ディスクの作成 +### ディスクの作成 {#creating-a-disk} GCS バケットをディスクとして利用するには、まず `conf.d` 配下のファイルで ClickHouse の設定にディスクを定義する必要があります。GCS ディスク定義の例を以下に示します。この設定には、GCS の「disk」、キャッシュ、およびテーブルを GCS ディスク上に作成する際に DDL クエリで指定されるポリシーを構成するための複数のセクションが含まれます。それぞれについて以下で説明します。 -#### Storage configuration > disks > gcs +#### Storage configuration > disks > gcs {#storage_configuration--disks--gcs} この設定の該当部分はハイライトされているセクションであり、次の内容を指定します。 @@ -68,7 +68,7 @@ GCS バケットをディスクとして利用するには、まず `conf.d` 配 ``` -#### ストレージ設定 > disks > cache +#### ストレージ設定 > disks > cache {#storage_configuration--disks--cache} 次の例の設定では、ディスク `gcs` に対して 10Gi のメモリ キャッシュを有効化します。 @@ -106,7 +106,7 @@ GCS バケットをディスクとして利用するには、まず `conf.d` 配 ``` -#### Storage configuration > policies > gcs_main +#### Storage configuration > policies > gcs_main {#storage_configuration--policies--gcs_main} ストレージ構成ポリシーを使用すると、データを保存する場所を選択できます。以下でハイライトされているポリシーでは、ポリシー `gcs_main` を指定することで、ディスク `gcs` 上にデータを保存できます。たとえば、`CREATE TABLE ... SETTINGS storage_policy='gcs_main'` のように指定します。 @@ -141,7 +141,7 @@ GCS バケットをディスクとして利用するには、まず `conf.d` 配 このディスク定義に関連するすべての設定項目の一覧は[こちら](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3)にあります。 -### テーブルの作成 +### テーブルの作成 {#creating-a-table} 書き込み権限のあるバケットを使用するようにディスクを設定してあると仮定すると、以下の例のようなテーブルを作成できるはずです。簡潔にするため、NYC タクシー データセットのカラムの一部のみを使用し、データを GCS をバックエンドとするテーブルに直接ストリーミングします。 @@ -179,11 +179,11 @@ INSERT INTO trips_gcs SELECT trip_id, pickup_date, pickup_datetime, dropoff_date SELECT passenger_count, avg(tip_amount) AS avg_tip, avg(total_amount) AS avg_amount FROM trips_gcs GROUP BY passenger_count; ``` -### レプリケーションの処理 +### レプリケーションの処理 {#handling-replication} GCS ディスクを用いたレプリケーションは、`ReplicatedMergeTree` テーブルエンジンを使用することで実現できます。詳細については、[GCS を使用して 2 つの GCP リージョン間で単一シャードをレプリケートする](#gcs-multi-region) ガイドを参照してください。 -### さらに詳しく +### さらに詳しく {#learn-more} [Cloud Storage XML API](https://cloud.google.com/storage/docs/xml-api/overview) は、Amazon Simple Storage Service (Amazon S3) などのサービスで動作する一部のツールおよびライブラリと相互運用性があります。 @@ -305,13 +305,13 @@ ClickHouse Keeper ノードでデプロイメント手順を実行する際は ``` -### ClickHouse サーバーを構成する +### ClickHouse サーバーを構成する {#configure-clickhouse-server} :::note best practice このガイドのいくつかの手順では、設定ファイルを `/etc/clickhouse-server/config.d/` に配置するように求められます。これは、Linux システムにおける設定オーバーライド用ファイルのデフォルトの配置場所です。このディレクトリにファイルを配置すると、ClickHouse はその内容をデフォルト設定とマージします。`config.d` ディレクトリにこれらのファイルを置くことで、アップグレード時に設定が失われるのを防ぐことができます。 ::: -#### ネットワーキング +#### ネットワーキング {#networking} デフォルトでは、ClickHouse はループバックインターフェイスで待ち受けますが、レプリケーション構成ではマシン間のネットワーク接続が必要です。すべてのインターフェイスで待ち受けるには、次のように設定します。 @@ -321,7 +321,7 @@ ClickHouse Keeper ノードでデプロイメント手順を実行する際は ``` -#### リモート ClickHouse Keeper サーバー +#### リモート ClickHouse Keeper サーバー {#remote-clickhouse-keeper-servers} レプリケーションは ClickHouse Keeper によって制御されます。この設定ファイルでは、ClickHouse Keeper ノードをホスト名とポート番号で識別します。 @@ -346,7 +346,7 @@ ClickHouse Keeper ノードでデプロイメント手順を実行する際は ``` -#### リモート ClickHouse サーバー +#### リモート ClickHouse サーバー {#remote-clickhouse-servers} このファイルでは、クラスタ内の各 ClickHouse サーバーのホスト名とポートを設定します。デフォルトの設定ファイルにはサンプルのクラスタ定義が含まれています。完全に構成されたクラスタのみを使用するために、この設定がデフォルト設定とマージされた際に `remote_servers` セクションへ追加されるのではなく、その内容を置き換えるよう、`remote_servers` エントリにはタグ `replace="true"` が追加されています。 @@ -372,7 +372,7 @@ ClickHouse Keeper ノードでデプロイメント手順を実行する際は ``` -#### レプリカの識別 +#### レプリカの識別 {#replica-identification} このファイルでは、ClickHouse Keeper 上のパスに関連する設定を行います。具体的には、データがどのレプリカに属しているかを識別するためのマクロを設定します。1 台目のサーバーではレプリカを `replica_1`、もう一方のサーバーでは `replica_2` と指定します。名前は変更しても構いません。たとえば、1 つのレプリカをサウスカロライナ、もう 1 つをノーザンバージニアに配置する例に基づくと、値は `carolina` と `virginia` のようにしてもかまいません。ただし、各マシンで異なる値になるようにしてください。 @@ -390,7 +390,7 @@ ClickHouse Keeper ノードでデプロイメント手順を実行する際は ``` -#### GCS でのストレージ +#### GCS でのストレージ {#storage-in-gcs} ClickHouse のストレージ構成には `disks` と `policies` が含まれます。以下で構成しているディスク名は `gcs` で、`type` は `s3` です。`type` が s3 になっているのは、ClickHouse が GCS バケットに対して、AWS S3 バケットと同様の方法でアクセスするためです。この構成は 2 部用意し、ClickHouse サーバーノードごとに 1 つずつ使用します。 @@ -438,7 +438,7 @@ ClickHouse のストレージ構成には `disks` と `policies` が含まれま ``` -### ClickHouse Keeper を起動する +### ClickHouse Keeper を起動する {#start-clickhouse-keeper} お使いのオペレーティングシステム向けのコマンドを使用してください。たとえば、次のとおりです。 @@ -448,7 +448,7 @@ sudo systemctl start clickhouse-keeper sudo systemctl status clickhouse-keeper ``` -#### ClickHouse Keeper のステータスを確認する +#### ClickHouse Keeper のステータスを確認する {#check-clickhouse-keeper-status} `netcat` を使って ClickHouse Keeper にコマンドを送信します。たとえば、`mntr` は ClickHouse Keeper クラスターの状態を返します。各 Keeper ノードでこのコマンドを実行すると、1 つがリーダーで、残りの 2 つがフォロワーであることがわかります。 @@ -464,11 +464,11 @@ zk_max_latency 11 zk_min_latency 0 zk_packets_received 1783 zk_packets_sent 1783 -# highlight-start +# highlight-start {#highlight-start} zk_num_alive_connections 2 zk_outstanding_requests 0 zk_server_state leader -# highlight-end +# highlight-end {#highlight-end} zk_znode_count 135 zk_watch_count 8 zk_ephemerals_count 3 @@ -477,13 +477,13 @@ zk_key_arena_size 28672 zk_latest_snapshot_size 0 zk_open_file_descriptor_count 182 zk_max_file_descriptor_count 18446744073709551615 -# highlight-start +# highlight-start {#highlight-start} zk_followers 2 zk_synced_followers 2 -# highlight-end +# highlight-end {#highlight-end} ``` -### ClickHouse サーバーを起動する +### ClickHouse サーバーを起動する {#start-clickhouse-server} `chnode1` と `chnode` で以下を実行します。 @@ -495,9 +495,9 @@ sudo service clickhouse-server start sudo service clickhouse-server status ``` -### 検証 +### 検証 {#verification} -#### ディスク構成の検証 +#### ディスク構成の検証 {#verify-disk-configuration} `system.disks` には、各ディスクのレコードが含まれている必要があります: @@ -565,7 +565,7 @@ cache_path: 3 行が結果セットに含まれています。経過時間: 0.002 秒。 ```` -#### クラスタ上で作成されたテーブルが両ノードに作成されていることを確認する +#### クラスタ上で作成されたテーブルが両ノードに作成されていることを確認する {#verify-that-tables-created-on-the-cluster-are-created-on-both-nodes} ```sql -- highlight-next-line create table trips on cluster 'cluster_1S_2R' ( @@ -600,7 +600,7 @@ SETTINGS storage_policy='gcs_main' 2行のデータセット。経過時間: 0.641秒 ``` -#### データを挿入できることを確認する +#### データを挿入できることを確認する {#verify-that-data-can-be-inserted} ```sql INSERT INTO trips SELECT @@ -621,7 +621,7 @@ FROM s3('https://ch-nyc-taxi.s3.eu-west-3.amazonaws.com/tsv/trips_{0..9}.tsv.gz' LIMIT 1000000 ``` -#### テーブルでストレージポリシー `gcs_main` が使用されていることを確認します。 +#### テーブルでストレージポリシー `gcs_main` が使用されていることを確認します。 {#verify-that-the-storage-policy-gcs_main-is-used-for-the-table} ```sql SELECT @@ -647,14 +647,14 @@ formatReadableSize(total_bytes): 36.42 MiB 1行を取得しました。経過時間: 0.002秒 ``` -#### Google Cloud コンソールでの確認 +#### Google Cloud コンソールでの確認 {#verify-in-google-cloud-console} バケットを確認すると、`storage.xml` 構成ファイルで指定した名前のフォルダが、各バケット内に作成されていることがわかります。フォルダを展開すると、多数のファイルがあり、これらがデータパーティションを表しています。 -#### レプリカ 1 用バケット +#### レプリカ 1 用バケット {#bucket-for-replica-one} Google Cloud Storage におけるレプリカ 1 のバケット。データパーティションを含むフォルダ構造が表示されている -#### レプリカ 2 用バケット +#### レプリカ 2 用バケット {#bucket-for-replica-two} Google Cloud Storage におけるレプリカ 2 のバケット。データパーティションを含むフォルダ構造が表示されている diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md index e05c5a858e1..170b5f065a5 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md @@ -11,7 +11,7 @@ keywords: ['Google Dataflow ClickHouse', 'Dataflow ClickHouse integration', 'Apa import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Google Dataflow と ClickHouse の統合 +# Google Dataflow と ClickHouse の統合 {#integrating-google-dataflow-with-clickhouse} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md index def78a71397..9262e134ff7 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md @@ -11,7 +11,7 @@ keywords: ['Dataflow Java Runner', 'Google Dataflow ClickHouse', 'Apache Beam Ja import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Dataflow Java ランナー +# Dataflow Java ランナー {#dataflow-java-runner} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md index 03b9690b8ad..45ccd449073 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md @@ -11,7 +11,7 @@ keywords: ['google dataflow', 'gcp', 'データパイプライン', 'テンプ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Google Dataflow テンプレート +# Google Dataflow テンプレート {#google-dataflow-templates} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md index 13f288394fc..c6f666db706 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md @@ -19,7 +19,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Dataflow BigQuery から ClickHouse へのテンプレート +# Dataflow BigQuery から ClickHouse へのテンプレート {#dataflow-bigquery-to-clickhouse-template} BigQuery から ClickHouse への Dataflow テンプレートは、BigQuery テーブルから ClickHouse テーブルへデータをバッチで取り込むパイプラインです。 このテンプレートは、テーブル全体を読み取ることも、指定された SQL クエリを使用して特定のレコードに絞り込むこともできます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md index 332f305972b..a6bf3d5162b 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md @@ -9,7 +9,7 @@ doc_type: 'guide' keywords: ['ローカルファイル インポート ClickHouse', 'ClickHouse ローカルファイル インポート', 'clickhouse-client ファイルアップロード'] --- -# ローカルファイルの挿入 +# ローカルファイルの挿入 {#insert-local-files} `clickhouse-client` を使用して、ローカルファイルを ClickHouse サービスにストリーミングできます。これにより、ClickHouse が備える数多くの強力かつ便利な関数を使ってデータを前処理できます。例を見てみましょう... @@ -79,7 +79,6 @@ LIMIT 7 結果は以下のとおりです。 - ```response │ 488 │ comment │ mynameishere │ 2007-02-22 14:48:18 │ "It's too bad. Javascript-in-the-browser and Ajax are both nasty hacks that force programmers to do all sorts of shameful things. And the result is--wanky html tricks. Java, for its faults, is fairly clean when run in the applet environment. It has every superiority over JITBAJAX, except for install issues and a chunky load process. Yahoo games seems like just about the only applet success story. Of course, back in the day, non-trivial Applets tended to be too large for the dial-up accounts people had. At least that is changed." │ [454927] │ ['It','s','too','bad','Javascript','in','the','browser','and','Ajax','are','both','nasty','hacks','that','force','programmers','to','do','all','sorts','of','shameful','things','And','the','result','is','wanky','html','tricks','Java','for','its','faults','is','fairly','clean','when','run','in','the','applet','environment','It','has','every','superiority','over','JITBAJAX','except','for','install','issues','and','a','chunky','load','process','Yahoo','games','seems','like','just','about','the','only','applet','success','story','Of','course','back','in','the','day','non','trivial','Applets','tended','to','be','too','large','for','the','dial','up','accounts','people','had','At','least','that','is','changed'] │ │ 575 │ comment │ leoc │ 2007-02-23 00:09:49 │ "I can't find the reference now, but I *think* I've just read something suggesting that the install process for an Apollo applet will involve an "install-this-application?" confirmation dialog followed by a download of 30 seconds or so. If so then Apollo's less promising than I hoped. That kind of install may be low-friction by desktop-app standards but it doesn't compare to the ease of starting a browser-based AJAX or Flash application. (Consider how easy it is to use maps.google.com for the first time.)

Surely it will at least be that Apollo applications will run untrusted by default, and that an already-installed app will start automatically whenever you take your browser to the URL you downloaded it from?" │ [455071] │ ['I','can','t','find','the','reference','now','but','I','think','I','ve','just','read','something','suggesting','that','the','install','process','for','an','Apollo','applet','will','involve','an','34','install','this','application','34','confirmation','dialog','followed','by','a','download','of','30','seconds','or','so','If','so','then','Apollo','s','less','promising','than','I','hoped','That','kind','of','install','may','be','low','friction','by','desktop','app','standards','but','it','doesn','t','compare','to','the','ease','of','starting','a','browser','based','AJAX','or','Flash','application','Consider','how','easy','it','is','to','use','maps','google','com','for','the','first','time','p','Surely','it','will','at','least','be','that','Apollo','applications','will','run','untrusted','by','default','and','that','an','already','installed','app','will','start','automatically','whenever','you','take','your','browser','to','the','URL','you','downloaded','it','from'] │ diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md index d817065d007..99485d24e43 100644 --- a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md @@ -12,11 +12,11 @@ integration: - website: 'https://clickhouse.com/cloud/clickpipes' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; -# Confluent Cloud と ClickHouse との連携 +# Confluent Cloud と ClickHouse との連携 {#integrating-confluent-cloud-with-clickhouse}

\ No newline at end of file + + + + +fullscreen; +picture-in-picture" + allowfullscreen + />
-
+
Существует несколько вариантов миграции данных в ClickHouse Cloud в зависимости от того, где сейчас находятся ваши данные: -- [Self-managed to Cloud](/cloud/migration/clickhouse-to-cloud): используйте функцию `remoteSecure` для переноса данных -- [Another DBMS](/cloud/migration/clickhouse-local): используйте ETL-инструмент [clickhouse-local] вместе с соответствующей табличной функцией ClickHouse для вашей текущей СУБД -- [Anywhere!](/cloud/migration/etl-tool-to-clickhouse): используйте один из множества популярных ETL/ELT-инструментов, которые подключаются к самым разным источникам данных -- [Object Storage](/integrations/migration/object-storage-to-clickhouse): просто загружайте данные из S3 в ClickHouse +* [Self-managed to Cloud](/cloud/migration/clickhouse-to-cloud): используйте функцию `remoteSecure` для переноса данных +* [Another DBMS](/cloud/migration/clickhouse-local): используйте ETL-инструмент [clickhouse-local] вместе с соответствующей табличной функцией ClickHouse для вашей текущей СУБД +* [Anywhere!](/cloud/migration/etl-tool-to-clickhouse): используйте один из множества популярных ETL/ELT-инструментов, которые подключаются к самым разным источникам данных +* [Object Storage](/integrations/migration/object-storage-to-clickhouse): просто загружайте данные из S3 в ClickHouse В примере [Migrate from Redshift](/migrations/redshift/migration-guide) мы приводим три способа миграции данных в ClickHouse. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md index a95770adac4..d64cbafdc04 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Сравнение ClickHouse с PostgreSQL +# Сравнение ClickHouse с PostgreSQL {#comparing-clickhouse-and-postgresql} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md index 8b1fb22997e..de3376e2b83 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md @@ -129,7 +129,7 @@ ClickHouse обеспечивает свойства ACID при [огранич -## Сжатие +## Сжатие {#compression} Колоночное хранилище ClickHouse означает, что степень сжатия часто будет значительно выше по сравнению с Postgres. Ниже показано, как отличаются требования к хранилищу для всех таблиц Stack Overflow в обеих базах данных: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md index 251b14c75f0..d9a55d4d96d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md @@ -31,49 +31,49 @@ import Image from '@theme/IdealImage'; ```bash -# пользователи +# пользователи {#users} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/users.sql.gz gzip -d users.sql.gz psql < users.sql ``` -# posts +# posts {#posts} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posts.sql.gz gzip -d posts.sql.gz psql < posts.sql -# posthistory +# posthistory {#posthistory} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posthistory.sql.gz gzip -d posthistory.sql.gz psql < posthistory.sql -# комментарии +# комментарии {#comments} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/comments.sql.gz gzip -d comments.sql.gz psql < comments.sql -# Голоса +# Голоса {#votes} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/votes.sql.gz gzip -d votes.sql.gz psql < votes.sql -# Значки +# Значки {#badges} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/badges.sql.gz gzip -d badges.sql.gz psql < badges.sql -# postlinks +# postlinks {#postlinks} wget [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz) gzip -d postlinks.sql.gz @@ -87,9 +87,9 @@ psql < postlinks.sql ``` -## Миграция данных +## Миграция данных {#migrating-data} -### Репликация в режиме реального времени (CDC) +### Репликация в режиме реального времени (CDC) {#real-time-replication-or-cdc} Обратитесь к этому [руководству](/integrations/clickpipes/postgres), чтобы настроить ClickPipes для PostgreSQL. В нём рассматриваются многие типы исходных экземпляров Postgres. @@ -125,7 +125,7 @@ ORDER BY id; После настройки ClickPipes начинает миграцию всех данных из PostgreSQL в ClickHouse. В зависимости от сети и масштаба развертывания для набора данных Stack Overflow это должно занять всего несколько минут. -### Ручная массовая загрузка с периодическими обновлениями +### Ручная массовая загрузка с периодическими обновлениями {#initial-bulk-load-with-periodic-updates} При использовании ручного подхода первоначальная массовая загрузка набора данных может быть выполнена с помощью: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md index 5e942043279..7eb2f2f4140 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md @@ -21,7 +21,7 @@ doc_type: 'guide' -## Оптимизация запросов в ClickHouse +## Оптимизация запросов в ClickHouse {#optimize-queries-in-clickhouse} Хотя можно выполнить миграцию с минимальной переработкой запросов, рекомендуется использовать возможности ClickHouse, чтобы существенно упростить запросы и дополнительно улучшить их производительность. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md index b3d641f0702..225bb153a08 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md @@ -49,7 +49,7 @@ import Image from '@theme/IdealImage'; -## Партиции +## Партиции {#partitions} Пользователям Postgres знакома концепция партиционирования таблиц, которая используется для повышения производительности и управляемости крупных баз данных за счёт разделения таблиц на более мелкие, удобные в обслуживании части, называемые партициями. Такое партиционирование может выполняться по диапазону значений для указанного столбца (например, по датам), по заданным спискам или по хешу ключа. Это позволяет администраторам организовывать данные на основе конкретных критериев, таких как диапазоны дат или географические регионы. Партиционирование помогает улучшить производительность запросов за счёт более быстрого доступа к данным через отсечение партиций (partition pruning) и более эффективного индексирования. Оно также упрощает задачи обслуживания, такие как резервное копирование и очистка данных, позволяя выполнять операции над отдельными партициями, а не над всей таблицей целиком. Кроме того, партиционирование может существенно повысить масштабируемость баз данных PostgreSQL за счёт распределения нагрузки по нескольким партициям. @@ -76,7 +76,7 @@ PARTITION BY toYear(CreationDate) Для полного описания партиционирования см. ["Партиции таблиц"](/partitions). -### Применение партиционирования +### Применение партиционирования {#applications-of-partitions} Партиционирование в ClickHouse имеет схожие области применения с Postgres, но с некоторыми тонкими отличиями. В частности: @@ -132,7 +132,7 @@ Ok. -## Материализованные представления и проекции +## Материализованные представления и проекции {#materialized-views-vs-projections} Postgres позволяет создавать несколько индексов для одной таблицы, что позволяет оптимизировать ее под различные паттерны доступа. Эта гибкость позволяет администраторам и разработчикам настраивать производительность базы данных под конкретные запросы и операционные потребности. Концепция проекций в ClickHouse, хотя и не является полной аналогией этого, позволяет пользователям задавать несколько выражений `ORDER BY` для таблицы. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md index d46e2ba546a..d92e3e2e0c1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md @@ -12,7 +12,7 @@ import bigquery_1 from '@site/static/images/migrations/bigquery-1.png'; import Image from '@theme/IdealImage'; -# Сравнение ClickHouse Cloud и BigQuery +# Сравнение ClickHouse Cloud и BigQuery {#comparing-clickhouse-cloud-and-bigquery} @@ -186,7 +186,7 @@ ClickHouse использует стандартный SQL с множество -## Массивы +## Массивы {#arrays} По сравнению с восемью функциями работы с массивами в BigQuery, в ClickHouse доступно более 80 [встроенных функций для работы с массивами](/sql-reference/functions/array-functions), которые позволяют элегантно и просто моделировать и решать широкий круг задач. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md index 9354bd14098..4e904afd8dd 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md @@ -75,7 +75,7 @@ CDC (фиксация изменений данных) — это процесс -## Проектирование схем +## Проектирование схем {#designing-schemas} Набор данных Stack Overflow содержит ряд связанных таблиц. Рекомендуем сначала сосредоточиться на миграции основной таблицы. Это не обязательно самая большая таблица, а та, по которой вы ожидаете получать больше всего аналитических запросов. Это позволит вам познакомиться с основными концепциями ClickHouse. По мере добавления дополнительных таблиц структура этой таблицы может потребовать пересмотра, чтобы в полной мере использовать возможности ClickHouse и обеспечить оптимальную производительность. Мы рассматриваем этот процесс моделирования в нашей [документации по моделированию данных](/data-modeling/schema-design#next-data-modeling-techniques). @@ -108,7 +108,7 @@ CREATE TABLE stackoverflow.posts ( ); ``` -### Оптимизация типов +### Оптимизация типов {#optimizing-types} Применение процесса, [описанного здесь](/data-modeling/schema-design), приводит к следующей схеме: @@ -174,11 +174,11 @@ INSERT INTO stackoverflow.posts SELECT * FROM gcs( 'gs://clickhouse-public-datas -## Подходы к моделированию данных +## Подходы к моделированию данных {#data-modeling-techniques} Мы рекомендуем пользователям, мигрирующим с BigQuery, ознакомиться с [руководством по моделированию данных в ClickHouse](/data-modeling/schema-design). В этом руководстве используется тот же набор данных Stack Overflow и рассматриваются несколько подходов с использованием возможностей ClickHouse. -### Партиционирование +### Партиционирование {#partitions} Пользователи BigQuery знакомы с концепцией партиционирования таблиц, которая повышает производительность и управляемость крупных баз данных за счёт деления таблиц на более мелкие, удобные для управления части, называемые партициями. Такое партиционирование может осуществляться с использованием диапазона по заданному столбцу (например, по датам), определённых списков или хеша по ключу. Это позволяет администраторам организовывать данные на основе конкретных критериев, таких как диапазоны дат или географическое расположение. @@ -205,7 +205,7 @@ ORDER BY (PostTypeId, toDate(CreationDate), CreationDate) PARTITION BY toYear(CreationDate) ``` -#### Применение +#### Применение {#applications} Разбиение на разделы (partitioning) в ClickHouse имеет сходные области применения с BigQuery, но и некоторые тонкие отличия. В частности: @@ -259,7 +259,7 @@ Ok. -## Материализованные представления и проекции +## Материализованные представления и проекции {#materialized-views-vs-projections} Концепция проекций в ClickHouse позволяет задавать для одной таблицы несколько вариантов `ORDER BY`. @@ -413,7 +413,7 @@ WHERE UserId = 8592047 ``` -## Переписывание запросов BigQuery на ClickHouse +## Переписывание запросов BigQuery на ClickHouse {#rewriting-bigquery-queries-in-clickhouse} Ниже приведены примеры запросов для сравнения BigQuery и ClickHouse. Этот список призван показать, как использовать возможности ClickHouse для значительного упрощения запросов. В примерах используется полный набор данных Stack Overflow (до апреля 2024 года включительно). @@ -481,7 +481,7 @@ LIMIT 5 ``` -## Агрегатные функции +## Агрегатные функции {#aggregate-functions} Где это возможно, пользователям следует использовать агрегатные функции ClickHouse. Ниже показано использование [функции `argMax`](/sql-reference/aggregate-functions/reference/argmax) для вычисления самого просматриваемого вопроса за каждый год. @@ -536,7 +536,7 @@ Peak memory usage: 377.26 MiB. ``` -## Условные выражения и массивы +## Условные выражения и массивы {#conditionals-and-arrays} Условные выражения и функции для работы с массивами значительно упрощают запросы. Следующий запрос вычисляет теги (встречающиеся более 10 000 раз) с наибольшим процентным ростом с 2022 по 2023 год. Обратите внимание, насколько лаконичен следующий запрос ClickHouse благодаря условным выражениям, функциям для работы с массивами и возможности повторно использовать псевдонимы в предложениях `HAVING` и `SELECT`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md index 58487c10548..0907106fb45 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md @@ -30,7 +30,7 @@ _Это руководство подходит для ClickHouse Cloud и дл -## Экспорт данных таблицы в GCS +## Экспорт данных таблицы в GCS {#1-export-table-data-to-gcs} На этом шаге мы используем [рабочую область BigQuery SQL](https://cloud.google.com/bigquery/docs/bigquery-web-ui) для выполнения SQL-команд. Ниже мы экспортируем таблицу BigQuery с именем `mytable` в бакет GCS с помощью оператора [`EXPORT DATA`](https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements). @@ -67,7 +67,7 @@ END WHILE; * Parquet как колоночный формат является более подходящим форматом обмена данными, поскольку он изначально сжат и позволяет BigQuery быстрее выполнять экспорт, а ClickHouse — быстрее выполнять запросы. -## Импорт данных в ClickHouse из GCS +## Импорт данных в ClickHouse из GCS {#2-importing-data-into-clickhouse-from-gcs} После завершения экспорта мы можем импортировать эти данные в таблицу ClickHouse. Вы можете использовать [консоль ClickHouse SQL](/integrations/sql-clients/sql-console) или [`clickhouse-client`](/interfaces/cli) для выполнения приведённых ниже команд. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md index 8a7ad8b4166..9fec245b382 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md @@ -13,7 +13,7 @@ import cloud_architecture from '@site/static/images/cloud/onboard/discover/use_c import Image from '@theme/IdealImage'; -# Миграция с Snowflake на ClickHouse +# Миграция с Snowflake на ClickHouse {#snowflake-to-clickhouse-migration} > Этот документ является введением в миграцию данных из Snowflake в ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md index 974b5cb07dd..419371e52c6 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md @@ -12,7 +12,7 @@ import migrate_snowflake_clickhouse from '@site/static/images/migrations/migrate import Image from '@theme/IdealImage'; -# Миграция из Snowflake в ClickHouse +# Миграция из Snowflake в ClickHouse {#migrate-from-snowflake-to-clickhouse} > В этом руководстве описывается процесс миграции данных из Snowflake в ClickHouse. @@ -24,7 +24,7 @@ import Image from '@theme/IdealImage'; -## Экспорт данных из Snowflake +## Экспорт данных из Snowflake {#1-exporting-data-from-snowflake} Миграция с Snowflake на ClickHouse @@ -62,7 +62,7 @@ COPY INTO @external_stage/mydataset from mydataset max_file_size=157286400 heade Для набора данных объемом около 5 ТБ с максимальным размером файла 150 МБ и при использовании виртуального склада Snowflake типа 2X-Large, расположенного в том же регионе AWS `us-east-1`, копирование данных в бакет S3 займет примерно 30 минут. -## Импорт в ClickHouse +## Импорт в ClickHouse {#2-importing-to-clickhouse} После того как данные размещены во временном объектном хранилище, функции ClickHouse, такие как [табличная функция s3](/sql-reference/table-functions/s3), можно использовать для вставки данных в таблицу, как показано ниже. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md index bdfc01b1a38..1cd3450242b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# Руководство по преобразованию SQL-запросов Snowflake +# Руководство по преобразованию SQL-запросов Snowflake {#snowflake-sql-translation-guide} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md index 88fe3de3174..8365ea362ee 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md @@ -8,6 +8,6 @@ show_related_blogs: true doc_type: 'landing-page' --- -# Миграция с Elasticsearch на ClickHouse +# Миграция с Elasticsearch на ClickHouse {#elasticsearch-to-clickhouse-migration} Для сценариев наблюдаемости см. документацию по миграции [с Elasticsearch на ClickStack](/use-cases/observability/clickstack/migration/elastic). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md index 860c119f448..852d600c25c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Миграция с Amazon Redshift на ClickHouse +# Миграция с Amazon Redshift на ClickHouse {#amazon-redshift-to-clickhouse-migration} > Этот документ является вводным руководством по миграции данных из Amazon Redshift в ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md index 54a3d34c932..9c30be7822e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md @@ -7,9 +7,8 @@ title: 'Руководство по миграции с Amazon Redshift на Cli doc_type: 'guide' --- -import MigrationGuide from '@site/docs/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +import MigrationGuide from '@site/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +# Руководство по миграции с Amazon Redshift на ClickHouse {#amazon-redshift-to-clickhouse-migration-guide} -# Руководство по миграции с Amazon Redshift на ClickHouse - - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md index 92d41d4e724..c03693dc915 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Руководство по преобразованию SQL-запросов Amazon Redshift +# Руководство по преобразованию SQL-запросов Amazon Redshift {#amazon-redshift-sql-translation-guide} @@ -65,9 +65,9 @@ String в ClickHouse, таким образом, не имеет огранич -## Синтаксис DDL +## Синтаксис DDL {#compression} -### Ключи сортировки +### Ключи сортировки {#sorting-keys} И в ClickHouse, и в Redshift есть понятие «ключ сортировки», который определяет, как данные упорядочиваются при сохранении. В Redshift ключ сортировки задаётся с помощью diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md index 894633ec6ad..176b740deef 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md @@ -8,7 +8,7 @@ keywords: ['миграция', 'ClickHouse Cloud', 'OSS', 'миграция са --- import Image from '@theme/IdealImage'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import self_managed_01 from '@site/static/images/integrations/migration/self-managed-01.png'; import self_managed_02 from '@site/static/images/integrations/migration/self-managed-02.png'; import self_managed_03 from '@site/static/images/integrations/migration/self-managed-03.png'; @@ -16,16 +16,13 @@ import self_managed_04 from '@site/static/images/integrations/migration/self-man import self_managed_05 from '@site/static/images/integrations/migration/self-managed-05.png'; import self_managed_06 from '@site/static/images/integrations/migration/self-managed-06.png'; +# Миграция между самостоятельно управляемым ClickHouse и ClickHouse Cloud {#migrating-between-self-managed-clickhouse-and-clickhouse-cloud} -# Миграция между самостоятельно управляемым ClickHouse и ClickHouse Cloud - -Миграция самостоятельно управляемого ClickHouse +Миграция самостоятельно управляемого ClickHouse В этом руководстве показано, как выполнить миграцию с самостоятельно управляемого сервера ClickHouse в ClickHouse Cloud, а также как выполнять миграцию между сервисами ClickHouse Cloud. Функция [`remoteSecure`](/sql-reference/table-functions/remote) используется в запросах `SELECT` и `INSERT` для обеспечения доступа к удалённым серверам ClickHouse, что делает миграцию таблиц настолько же простой, как написание запроса `INSERT INTO` с вложенным `SELECT`. - - -## Миграция с самостоятельно управляемого ClickHouse в ClickHouse Cloud +## Миграция с самостоятельно управляемого ClickHouse в ClickHouse Cloud {#migrating-from-self-managed-clickhouse-to-clickhouse-cloud} Migrating Self-managed ClickHouse @@ -36,7 +33,7 @@ import self_managed_06 from '@site/static/images/integrations/migration/self-man В этом примере самостоятельно управляемый сервер ClickHouse является *источником*, а сервис ClickHouse Cloud — *приёмником*. -### Обзор +### Обзор {#overview} Процесс выглядит следующим образом: @@ -46,11 +43,11 @@ import self_managed_06 from '@site/static/images/integrations/migration/self-man 4. Удалить исходный сервер из IP Access List на целевой стороне (если применимо) 5. Удалить пользователя с правами только на чтение из исходного сервиса -### Миграция таблиц из одной системы в другую: +### Миграция таблиц из одной системы в другую: {#migration-of-tables-from-one-system-to-another} Этот пример переносит одну таблицу с самостоятельно управляемого сервера ClickHouse в ClickHouse Cloud. -### На исходной системе ClickHouse (системе, которая в данный момент хранит данные) +### На исходной системе ClickHouse (системе, которая в данный момент хранит данные) {#on-the-source-clickhouse-system-the-system-that-currently-hosts-the-data} * Добавьте пользователя с правами только на чтение, который может читать исходную таблицу (`db.table` в этом примере) @@ -72,7 +69,7 @@ FROM system.tables WHERE database = 'db' AND table = 'table' ``` -### В целевой системе ClickHouse Cloud: +### В целевой системе ClickHouse Cloud: {#on-the-destination-clickhouse-cloud-system} * Создайте целевую базу данных: @@ -119,8 +116,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 'default', 'PASS') SELECT * FROM db.table ``` - -## Миграция между сервисами ClickHouse Cloud +## Миграция между сервисами ClickHouse Cloud {#migrating-between-clickhouse-cloud-services} Миграция самоуправляемого ClickHouse @@ -143,7 +139,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 6. Восстановите IP Access List на сервисе destination 7. Удалите пользователя только для чтения из сервиса source -#### Добавьте пользователя только для чтения в сервис source +#### Добавьте пользователя только для чтения в сервис source {#add-a-read-only-user-to-the-source-service} * Добавьте пользователя только для чтения, который может читать таблицу source (`db.table` в этом примере) @@ -164,7 +160,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', where database = 'db' and table = 'table' ``` -#### Продублируйте структуру таблицы в сервисе destination +#### Продублируйте структуру таблицы в сервисе destination {#duplicate-the-table-structure-on-the-destination-service} В сервисе destination создайте базу данных, если её ещё нет: @@ -181,7 +177,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', CREATE TABLE db.table ... ``` -#### Разрешите удалённый доступ к сервису source +#### Разрешите удалённый доступ к сервису source {#allow-remote-access-to-the-source-service} Чтобы забирать данные из source в destination, сервис source должен разрешать подключения. Временно отключите функциональность "IP Access List" на сервисе source. @@ -191,7 +187,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', Измените allow list и временно разрешите доступ из **Anywhere**. Подробности см. в документации по [IP Access List](/cloud/security/setting-ip-filters). -#### Скопируйте данные из source в destination +#### Скопируйте данные из source в destination {#copy-the-data-from-source-to-destination} * Используйте функцию `remoteSecure` для получения данных из сервиса ClickHouse Cloud source. Подключитесь к destination. Выполните эту команду в сервисе ClickHouse Cloud destination: @@ -203,11 +199,11 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', * Проверьте данные в сервисе destination -#### Восстановите IP Access List на сервисе source +#### Восстановите IP Access List на сервисе source {#re-establish-the-ip-access-list-on-the-source} Если вы ранее экспортировали список доступа, то можете повторно импортировать его с помощью **Share**, иначе заново добавьте свои записи в список доступа. -#### Удалите пользователя только для чтения `exporter` +#### Удалите пользователя только для чтения `exporter` {#remove-the-read-only-exporter-user} ```sql DROP USER exporter diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md index 76a652199ba..ecd078ff360 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md @@ -11,14 +11,14 @@ import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import ch_local_01 from '@site/static/images/integrations/migration/ch-local-01.png'; import ch_local_02 from '@site/static/images/integrations/migration/ch-local-02.png'; import ch_local_03 from '@site/static/images/integrations/migration/ch-local-03.png'; import ch_local_04 from '@site/static/images/integrations/migration/ch-local-04.png'; -# Миграция в ClickHouse с использованием clickhouse-local +# Миграция в ClickHouse с использованием clickhouse-local {#migrating-to-clickhouse-using-clickhouse-local} Миграция в самоуправляемый ClickHouse @@ -94,21 +94,21 @@ ClickHouse предоставляет движки интеграции и та -## Пример 1: Миграция с MySQL в ClickHouse Cloud с использованием интеграционного движка +## Пример 1: Миграция с MySQL в ClickHouse Cloud с использованием интеграционного движка {#example-1-migrating-from-mysql-to-clickhouse-cloud-with-an-integration-engine} Мы будем использовать [интеграционный движок таблиц](/engines/table-engines/integrations/mysql/) (создаваемый на лету с помощью [табличной функции mysql](/sql-reference/table-functions/mysql/)) для чтения данных из исходной базы данных MySQL, а для записи данных в целевую таблицу в вашем облачном сервисе ClickHouse Cloud — [табличную функцию remoteSecure](/sql-reference/table-functions/remote/). Миграция самоуправляемого ClickHouse -### На целевом сервисе ClickHouse Cloud: +### На целевом сервисе ClickHouse Cloud: {#on-the-destination-clickhouse-cloud-service} -#### Создайте целевую базу данных: +#### Создайте целевую базу данных: {#create-the-destination-database} ```sql CREATE DATABASE db ``` -#### Создайте целевую таблицу с такой же схемой, как у таблицы MySQL: +#### Создайте целевую таблицу с такой же схемой, как у таблицы MySQL: {#create-a-destination-table-that-has-a-schema-equivalent-to-the-mysql-table} ```sql CREATE TABLE db.table ... @@ -118,9 +118,9 @@ CREATE TABLE db.table ... Схемы целевой таблицы ClickHouse Cloud и исходной таблицы MySQL должны совпадать (имена и порядок столбцов должны быть одинаковыми, а типы данных столбцов — совместимыми). ::: -### На хосте с clickhouse-local: +### На хосте с clickhouse-local: {#on-the-clickhouse-local-host-machine} -#### Запустите clickhouse-local с миграционным запросом: +#### Запустите clickhouse-local с миграционным запросом: {#run-clickhouse-local-with-the-migration-query} ```sql ./clickhouse-local --query " @@ -135,16 +135,16 @@ SELECT * FROM mysql('host:port', 'database', 'table', 'user', 'password');" ::: -## Пример 2: миграция с MySQL в ClickHouse Cloud с использованием моста JDBC +## Пример 2: миграция с MySQL в ClickHouse Cloud с использованием моста JDBC {#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge} Мы будем использовать [табличный движок интеграции с JDBC](/engines/table-engines/integrations/jdbc.md) (создаваемый на лету с помощью [табличной функции JDBC](/sql-reference/table-functions/jdbc.md)) вместе с [ClickHouse JDBC Bridge](https://github.com/ClickHouse/clickhouse-jdbc-bridge) и драйвером MySQL JDBC для чтения данных из исходной базы данных MySQL, а также [табличную функцию remoteSecure](/sql-reference/table-functions/remote.md) для записи данных в целевую таблицу в вашем сервисе ClickHouse Cloud. Миграция самоуправляемого ClickHouse -### На целевом сервисе ClickHouse Cloud: +### На целевом сервисе ClickHouse Cloud: {#on-the-destination-clickhouse-cloud-service-1} -#### Создайте целевую базу данных: +#### Создайте целевую базу данных: {#create-the-destination-database-1} ```sql CREATE DATABASE db diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md index 007f7ca232c..86bb2f04640 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md @@ -10,15 +10,14 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import third_party_01 from '@site/static/images/integrations/migration/third-party-01.png'; +# Использование стороннего ETL-инструмента {#using-a-3rd-party-etl-tool} -# Использование стороннего ETL-инструмента - -Миграция самостоятельно управляемого ClickHouse +Миграция самостоятельно управляемого ClickHouse Отличный вариант переноса данных из внешнего источника в ClickHouse — использовать один из многих популярных ETL- и ELT-инструментов. У нас есть документация по следующим решениям: -- [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) -- [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) -- [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) +* [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) +* [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) +* [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) Но существует и множество других ETL/ELT-инструментов, которые интегрируются с ClickHouse, поэтому обратитесь к документации используемого вами инструмента за подробной информацией. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md index 3c75c3a39dd..2d71bc1aa64 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md @@ -9,18 +9,17 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import object_storage_01 from '@site/static/images/integrations/migration/object-storage-01.png'; +# Перенос данных из облачного объектного хранилища в ClickHouse Cloud {#move-data-from-cloud-object-storage-to-clickhouse-cloud} -# Перенос данных из облачного объектного хранилища в ClickHouse Cloud - -Миграция самостоятельно развернутого ClickHouse +Миграция самостоятельно развернутого ClickHouse Если вы используете облачное объектное хранилище как озеро данных и хотите импортировать эти данные в ClickHouse Cloud, или если ваша текущая система управления базами данных может напрямую выгружать данные в облачное объектное хранилище, то вы можете использовать одну из табличных функций для миграции данных, хранящихся в облачном объектном хранилище, в таблицу ClickHouse Cloud: -- [s3](/sql-reference/table-functions/s3.md) или [s3Cluster](/sql-reference/table-functions/s3Cluster.md) -- [gcs](/sql-reference/table-functions/gcs) -- [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) +* [s3](/sql-reference/table-functions/s3.md) или [s3Cluster](/sql-reference/table-functions/s3Cluster.md) +* [gcs](/sql-reference/table-functions/gcs) +* [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) Если ваша текущая система управления базами данных не может напрямую выгружать данные в облачное объектное хранилище, вы можете использовать [сторонний ETL/ELT-инструмент](/cloud/migration/etl-tool-to-clickhouse) или [clickhouse-local](/cloud/migration/clickhouse-local) для переноса данных из вашей текущей системы управления базами данных в облачное объектное хранилище, чтобы на втором этапе мигрировать эти данные в таблицу ClickHouse Cloud. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md index e3209d56939..33fd8034d5c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md @@ -7,12 +7,12 @@ hide_title: true doc_type: 'guide' --- -import TableOfContentsBestPractices from '@site/docs/best-practices/_snippets/_table_of_contents.md'; -import TableOfContentsOptimizationAndPerformance from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; -import TableOfContentsSecurity from '@site/docs/cloud/_snippets/_security_table_of_contents.md'; +import TableOfContentsBestPractices from '@site/i18n/ru/docusaurus-plugin-content-docs/current/best-practices/_snippets/_table_of_contents.md'; +import TableOfContentsOptimizationAndPerformance from '@site/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContentsSecurity from '@site/i18n/ru/docusaurus-plugin-content-docs/current/cloud/_snippets/_security_table_of_contents.md'; -# Обзор ресурсов +# Обзор ресурсов {#resource-tour} В этой статье представлен обзор доступных в документации ресурсов, которые помогут вам максимально эффективно использовать развертывание ClickHouse Cloud. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/index.md index 16834d39c86..870014d9c92 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/onboard/index.md @@ -9,7 +9,7 @@ keywords: ['онбординг', 'начало работы', 'настройк -# Начало работы с ClickHouse Cloud +# Начало работы с ClickHouse Cloud {#get-started-with-clickhouse-cloud} Впервые используете ClickHouse Cloud и не знаете, с чего начать? В этом разделе документации мы покажем вам всё необходимое для быстрого запуска и начала работы. Мы diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/01_changelog.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/01_changelog.md index b75a67a460e..8455857e59e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/01_changelog.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/01_changelog.md @@ -214,7 +214,7 @@ import dashboards from '@site/static/images/cloud/reference/may-30-dashboards.pn созданного для упрощения мониторинга ваших сервисов ClickHouse Cloud. Этот миксин использует наш совместимый с Prometheus конечный API-адрес для бесшовной интеграции метрик ClickHouse в вашу существующую инсталляцию Prometheus и Grafana. В его состав - входит преднастроенная панель мониторинга, которая обеспечивает Обзервабилити в режиме реального времени + входит преднастроенная панель мониторинга, которая обеспечивает наблюдаемость в режиме реального времени за состоянием и производительностью ваших сервисов. Дополнительную информацию см. в [блоге о запуске](https://clickhouse.com/blog/monitor-with-new-prometheus-grafana-mix-in). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md index 45c212b1caa..b88ceca6abc 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md @@ -12,7 +12,7 @@ doc_type: 'changelog' -#### Обратно несовместимое изменение +#### Обратно несовместимое изменение {#backward-incompatible-change} * Теперь выполняется проверка подозрительных/экспериментальных типов во вложенных типах. Ранее такие типы (кроме JSON) не проверялись во вложенных типах, таких как Array/Tuple/Map. [#59385](https://github.com/ClickHouse/ClickHouse/pull/59385) ([Kruglov Pavel](https://github.com/Avogar)). * Условие сортировки `ORDER BY ALL` (введённое в версии 23.12) заменено на `ORDER BY *`. Предыдущий синтаксис слишком часто приводил к ошибкам в таблицах со столбцом `all`. [#59450](https://github.com/ClickHouse/ClickHouse/pull/59450) ([Robert Schulze](https://github.com/rschu1ze)). @@ -29,7 +29,7 @@ doc_type: 'changelog' -#### Новая возможность +#### Новая возможность {#new-feature} * Режим работы topK/topKWeighted, возвращающий количество значений и оценку погрешности. [#54508](https://github.com/ClickHouse/ClickHouse/pull/54508) ([UnamedRus](https://github.com/UnamedRus)). * Добавлен новый синтаксис, позволяющий указать пользователя-определителя в представлении/материализованном представлении. Это позволяет выполнять SELECT/INSERT из представлений без явной выдачи прав на базовые таблицы. [#54901](https://github.com/ClickHouse/ClickHouse/pull/54901) ([pufit](https://github.com/pufit)). @@ -57,7 +57,7 @@ doc_type: 'changelog' -#### Повышение производительности +#### Повышение производительности {#performance-improvement} * Удаляет агрегаторы min/max/any/anyLast по ключам GROUP BY в списке SELECT. [#52230](https://github.com/ClickHouse/ClickHouse/pull/52230) ([JackyWoo](https://github.com/JackyWoo)). * Улучшена производительность метода сериализованной агрегации при работе с несколькими столбцами [nullable]. Это обобщённая версия [#51399](https://github.com/ClickHouse/ClickHouse/issues/51399), не нарушающая целостность абстракции. [#55809](https://github.com/ClickHouse/ClickHouse/pull/55809) ([Amos Bird](https://github.com/amosbird)). @@ -86,7 +86,7 @@ doc_type: 'changelog' -#### Улучшение +#### Улучшение {#improvement} * При выполнении запроса MODIFY COLUMN для материализованных представлений проверьте структуру внутренней таблицы, чтобы убедиться, что все столбцы присутствуют. [#47427](https://github.com/ClickHouse/ClickHouse/pull/47427) ([sunny](https://github.com/sunny19930321)). * Добавлена таблица `system.keywords`, которая содержит все ключевые слова из парсера. Она предназначена главным образом для улучшения фаззинга и подсветки синтаксиса. [#51808](https://github.com/ClickHouse/ClickHouse/pull/51808) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov)). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md index d3faa4c8ef6..633c1da6378 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# Журнал изменений версии 24.5 для Cloud +# Журнал изменений версии 24.5 для Cloud {#v245-changelog-for-cloud} Актуальные изменения в сервисах ClickHouse Cloud в релизе v24.5. @@ -128,7 +128,7 @@ doc_type: 'changelog' -# Улучшения +# Улучшения {#improvements} * Удалена настройка `optimize_monotonous_functions_in_order_by`, так как она по сути стала пустой операцией (no-op). [#63004](https://github.com/ClickHouse/ClickHouse/pull/63004) (Raúl Marín). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md index 897ddd1ee1b..852f9e8de5d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# Журнал изменений v24.6 для Cloud +# Журнал изменений v24.6 для Cloud {#v246-changelog-for-cloud} Актуальные изменения для сервисов ClickHouse Cloud в релизе v24.6. @@ -110,7 +110,7 @@ doc_type: 'changelog' -## Исправление ошибки (некорректное поведение, заметное пользователям, в официальном стабильном релизе) +## Исправление ошибки (некорректное поведение, заметное пользователям, в официальном стабильном релизе) {#bug-fix-user-visible-misbehavior-in-an-official-stable-release} * Исправлена проблема, из-за которой пропускной индекс 'set' не работал с IN и indexHint(). #62083 (Michael Kolupaev). * Исправлены запросы с FINAL, которые давали неверный результат, если таблица не использовала адаптивную гранулярность. #62432 (Duc Canh Le). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md index fb4202315e9..80e11827548 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md @@ -22,7 +22,7 @@ doc_type: 'changelog' -## Новая возможность +## Новая возможность {#new-feature} * Обновляемые материализованные представления готовы к промышленной эксплуатации. [#70550](https://github.com/ClickHouse/ClickHouse/pull/70550) ([Michael Kolupaev](https://github.com/al13n321)). Обновляемые материализованные представления теперь поддерживаются в реплицируемых базах данных. [#60669](https://github.com/ClickHouse/ClickHouse/pull/60669) ([Michael Kolupaev](https://github.com/al13n321)). * Функция `toStartOfInterval()` теперь имеет новую перегрузку, которая эмулирует функцию TimescaleDB `time_bucket()`, а также функцию PostgreSQL `date_bin()` ([#55619](https://github.com/ClickHouse/ClickHouse/issues/55619)). Она позволяет выравнивать значения даты или временной метки по кратным заданному интервалу от *произвольной* точки отсчёта (вместо 0000-01-01 00:00:00.000 как *фиксированной* точки отсчёта). Например, `SELECT toStartOfInterval(toDateTime('2023-01-01 14:45:00'), INTERVAL 1 MINUTE, toDateTime('2023-01-01 14:35:30'));` возвращает `2023-01-01 14:44:30`, которое кратно интервалу в 1 минуту, начиная от точки отсчёта `2023-01-01 14:35:30`. [#56738](https://github.com/ClickHouse/ClickHouse/pull/56738) ([Yarik Briukhovetskyi](https://github.com/yariks5s)). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/25_08.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/25_08.md index bf872d469b7..86379116691 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/25_08.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/25_08.md @@ -380,7 +380,7 @@ doc_type: 'changelog' * Некоррелированные `EXISTS` выполняются как скалярный подзапрос. Это позволяет использовать кэш скалярного подзапроса и выполнять константное свёртывание результата, что полезно для индексов. Для совместимости добавлена новая настройка `execute_exists_as_scalar_subquery=1`. [#85481](https://github.com/ClickHouse/ClickHouse/pull/85481) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). * Добавлена поддержка большего числа случаев разрешения составных идентификаторов. В частности, это улучшает совместимость `ARRAY JOIN` со старым анализатором. Добавлена новая настройка `analyzer_compatibility_allow_compound_identifiers_in_unflatten_nested` для сохранения старого поведения. [#85492](https://github.com/ClickHouse/ClickHouse/pull/85492) ([Nikolai Kochetov](https://github.com/KochetovNicolai)). -### Системные таблицы и Обзервабилити \{#system-tables-and-observability\} +### Системные таблицы и наблюдаемость \{#system-tables-and-observability\} * Добавлены метрики нагрузки в асинхронные метрики ClickHouse. [#80779](https://github.com/ClickHouse/ClickHouse/pull/80779) ([Xander Garbett](https://github.com/Garbett1)). * Добавлены метрики `MarkCacheEvictedBytes`, `MarkCacheEvictedMarks`, `MarkCacheEvictedFiles` для отслеживания вытеснений из кэша меток (issue [#60989](https://github.com/ClickHouse/ClickHouse/issues/60989)). [#80799](https://github.com/ClickHouse/ClickHouse/pull/80799) ([Shivji Kumar Jha](https://github.com/shiv4289)). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md index a9530916a74..bf883feb215 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md @@ -11,7 +11,7 @@ import Image from '@theme/IdealImage'; import Architecture from '@site/static/images/cloud/reference/architecture.png'; -# Архитектура ClickHouse Cloud +# Архитектура ClickHouse Cloud {#clickhouse-cloud-architecture} Архитектура ClickHouse Cloud diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md index 9334fe0563b..973ac7dd258 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md @@ -181,9 +181,9 @@ ClickHouse Cloud выставляет счета исходя из исполь -## Часто задаваемые вопросы +## Часто задаваемые вопросы {#faqs} -### Что такое ClickHouse Credit (CHC)? +### Что такое ClickHouse Credit (CHC)? {#what-is-chc} ClickHouse Credit — это единица кредита на использование ClickHouse Cloud Клиентом, равная одному (1) доллару США и применяемая на основе актуального опубликованного прайс‑листа ClickHouse. @@ -191,27 +191,27 @@ ClickHouse Credit — это единица кредита на использо Если вы получаете счета через Stripe, то в своём счёте Stripe вы увидите, что 1 CHC равен $0.01 USD. Это необходимо для точного выставления счетов в Stripe из‑за ограничения сервиса, не позволяющего выставлять счета за дробные количества нашей стандартной позиции (SKU) 1 CHC = $1 USD. ::: -### Где я могу найти устаревшие (legacy) цены? +### Где я могу найти устаревшие (legacy) цены? {#find-legacy-pricing} Информацию об устаревших ценах можно найти [здесь](https://clickhouse.com/pricing?legacy=true). -### Как измеряется потребление вычислительных ресурсов (compute)? +### Как измеряется потребление вычислительных ресурсов (compute)? {#how-is-compute-metered} ClickHouse Cloud измеряет потребление вычислительных ресурсов поминутно, с шагом 8 ГБ ОЗУ. Стоимость вычислений зависит от уровня (tier), региона и поставщика облачных услуг. -### Как рассчитывается использование дискового хранилища? +### Как рассчитывается использование дискового хранилища? {#how-is-storage-on-disk-calculated} ClickHouse Cloud использует облачное объектное хранилище, а использование измеряется по сжатому размеру данных, хранящихся в таблицах ClickHouse. Стоимость хранения одинакова для всех уровней и зависит от региона и поставщика облачных услуг. -### Засчитываются ли резервные копии в общий объём хранилища? +### Засчитываются ли резервные копии в общий объём хранилища? {#do-backups-count-toward-total-storage} Хранилище и резервные копии учитываются при расчёте стоимости хранения и тарифицируются отдельными позициями. Все сервисы по умолчанию имеют одну резервную копию с хранением в течение одного дня. Пользователи, которым требуется больше резервных копий, могут настроить дополнительные [резервные копии](/cloud/manage/backups/overview) на вкладке настроек консоли Cloud. -### Как оценить степень сжатия? +### Как оценить степень сжатия? {#how-do-i-estimate-compression} Степень сжатия может различаться для разных наборов данных. Насколько именно — зависит от того, насколько данные вообще поддаются сжатию (количество полей с высокой и низкой кардинальностью) @@ -228,12 +228,12 @@ FROM system.tables WHERE name = <имя_вашей_таблицы> ``` -### Какие инструменты предоставляет ClickHouse для оценки стоимости работы сервиса в облаке, если у меня есть самостоятельное развертывание? +### Какие инструменты предоставляет ClickHouse для оценки стоимости работы сервиса в облаке, если у меня есть самостоятельное развертывание? {#what-tools-does-clickhouse-offer-to-estimate-the-cost-of-running-a-service-in-the-cloud-if-i-have-a-self-managed-deployment} Журнал запросов ClickHouse фиксирует [ключевые метрики](/operations/system-tables/query_log), которые можно использовать для оценки стоимости выполнения рабочей нагрузки в ClickHouse Cloud. Подробнее о миграции с самостоятельного развертывания в ClickHouse Cloud см. в [документации по миграции](/cloud/migration/clickhouse-to-cloud). Если у вас есть дополнительные вопросы, свяжитесь со [службой поддержки ClickHouse Cloud](https://console.clickhouse.cloud/support). -### Какие варианты биллинга доступны для ClickHouse Cloud? +### Какие варианты биллинга доступны для ClickHouse Cloud? {#what-billing-options-are-available-for-clickhouse-cloud} ClickHouse Cloud поддерживает следующие варианты биллинга: @@ -245,11 +245,11 @@ ClickHouse Cloud поддерживает следующие варианты б Кредиты ClickHouse Cloud для PAYG выставляются счетом с шагом $0.01, что позволяет взимать плату с клиентов за доли кредитов ClickHouse в зависимости от фактического использования. Это отличается от кредитов ClickHouse при фиксированном обязательстве по расходам, которые приобретаются авансом целыми единицами по $1. ::: -### Могу ли я удалить свою банковскую карту? +### Могу ли я удалить свою банковскую карту? {#can-i-delete-my-credit-card} Вы не можете удалить банковскую карту в интерфейсе Billing, но в любое время можете обновить данные карты. Это помогает гарантировать, что у вашей организации всегда есть действительный способ оплаты. Если вам необходимо удалить банковскую карту, обратитесь в [службу поддержки ClickHouse Cloud](https://console.clickhouse.cloud/support) за помощью. -### Какова продолжительность расчетного периода (billing cycle)? +### Какова продолжительность расчетного периода (billing cycle)? {#how-long-is-the-billing-cycle} Начисление платежей происходит помесячно, а датой начала считается день создания организации ClickHouse Cloud. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md index b6ff1f73648..7d06f313bdd 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md @@ -17,21 +17,19 @@ import aws_marketplace_payg_8 from '@site/static/images/cloud/manage/billing/mar import aws_marketplace_payg_9 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-9.png'; import Image from '@theme/IdealImage'; -Начните работу с ClickHouse Cloud в [AWS Marketplace](https://aws.amazon.com/marketplace) через публичное предложение по модели PAYG (Pay-as-you-go, оплата по мере использования). +Начните работу с ClickHouse Cloud на [AWS Marketplace](https://aws.amazon.com/marketplace), воспользовавшись публичным предложением по модели PAYG (Pay-as-you-go). ## Предварительные требования {#prerequisites} -- Учетная запись AWS с правами на совершение покупок, предоставленными вашим администратором по биллингу. -- Для совершения покупки вы должны войти в AWS Marketplace под этой учетной записью. +- Учетная запись AWS с правами на совершение покупок, предоставленными администратором биллинга. +- Для совершения покупки вы должны быть авторизованы в AWS Marketplace под этой учетной записью. - Чтобы подключить организацию ClickHouse к вашей подписке, вы должны быть администратором этой организации. :::note -Одна учетная запись AWS может оформить только одну подписку «ClickHouse Cloud - Pay As You Go», которую можно связать только с одной организацией ClickHouse. +Одна учетная запись AWS может оформить только одну подписку «ClickHouse Cloud - Pay As You Go», которая может быть связана только с одной организацией ClickHouse. ::: - - ## Этапы регистрации {#steps-to-sign-up} @@ -44,46 +42,47 @@ import Image from '@theme/IdealImage'; ### Просмотрите варианты покупки {#purchase-options} -Нажмите на [страницу продукта](https://aws.amazon.com/marketplace/pp/prodview-p4gwofrqpkltu), а затем на **View purchase options**. +Нажмите на [листинг](https://aws.amazon.com/marketplace/pp/prodview-p4gwofrqpkltu), а затем на **View purchase options**. -Варианты покупки в AWS Marketplace +AWS Marketplace — просмотр вариантов покупки ### Оформите подписку {#subscribe} На следующем экране нажмите **Subscribe**. :::note -**Номер заказа на покупку (PO)** указывать необязательно, его можно не заполнять. +**Номер заказа на покупку (Purchase order, PO)** необязателен, им можно пренебречь. +**В этом листинге доступны два предложения.** Если вы выберете вариант «ClickHouse Cloud - Pay As You Go Free Trial», вы подпишетесь на управляемую AWS 30‑дневную бесплатную пробную версию. Однако по истечении 30 дней подписка на листинг завершится, и вам потребуется повторно оформить подписку уже на другое предложение «ClickHouse Cloud - Pay As You Go» в этом листинге, чтобы продолжить использовать ClickHouse Pay As You Go. ::: Оформление подписки в AWS Marketplace -### Настройте свою учетную запись {#set-up-your-account} +### Настройте учетную запись {#set-up-your-account} -Обратите внимание, что на данном этапе настройка еще не завершена, и вашей организации ClickHouse Cloud пока не выставляются счета через Marketplace. Теперь вам нужно нажать **Set up your account** в вашей подписке Marketplace, чтобы перейти в ClickHouse Cloud и завершить настройку. +Обратите внимание, что на этом этапе настройка еще не завершена, и ваша организация ClickHouse Cloud пока не тарифицируется через AWS Marketplace. Теперь вам нужно нажать **Set up your account** в подписке AWS Marketplace, чтобы перейти в ClickHouse Cloud и завершить настройку. Настройка учетной записи -После перенаправления в ClickHouse Cloud вы можете войти под существующей учетной записью или зарегистрировать новую. Этот шаг очень важен, поскольку он позволяет привязать вашу организацию ClickHouse Cloud к выставлению счетов через AWS Marketplace. +После перехода в ClickHouse Cloud вы можете либо войти с существующей учетной записью, либо зарегистрировать новую. Этот шаг очень важен, поскольку он позволяет привязать вашу организацию ClickHouse Cloud к биллингу AWS Marketplace. :::note[Новые пользователи ClickHouse Cloud] -Если вы новый пользователь ClickHouse Cloud, выполните действия, приведенные ниже. +Если вы новый пользователь ClickHouse Cloud, выполните шаги ниже. :::
Шаги для новых пользователей -Если вы новый пользователь ClickHouse Cloud, нажмите **Register** в нижней части страницы. Вам будет предложено создать нового пользователя и подтвердить адрес электронной почты. После подтверждения электронной почты вы можете закрыть страницу входа в ClickHouse Cloud и войти, используя новое имя пользователя, на https://console.clickhouse.cloud. +Если вы новый пользователь ClickHouse Cloud, нажмите **Register** внизу страницы. Вам будет предложено создать нового пользователя и подтвердить адрес электронной почты. После подтверждения email вы можете закрыть страницу входа ClickHouse Cloud и войти, используя новое имя пользователя, на https://console.clickhouse.cloud. Регистрация в ClickHouse Cloud :::note[Новые пользователи] -Вам также потребуется предоставить некоторую основную информацию о вашей компании. См. скриншоты ниже. +Вам также потребуется предоставить некоторую основную информацию о вашем бизнесе. См. скриншоты ниже. ::: -Прежде чем начать +Перед началом работы -Прежде чем начать, продолжение +Перед началом работы — продолжение
@@ -91,20 +90,18 @@ import Image from '@theme/IdealImage'; ### Добавьте подписку Marketplace к организации {#add-marketplace-subscription} -После успешного входа вы можете выбрать: создать новую организацию для выставления счетов по этой подписке Marketplace или использовать существующую организацию для выставления счетов по этой подписке. +После успешного входа вы можете выбрать, создать ли новую организацию для выставления счетов по этой подписке AWS Marketplace или использовать существующую организацию для выставления счетов по данной подписке. Добавление подписки Marketplace -После завершения этого шага ваша организация будет подключена к этой подписке AWS, и все потребление будет выставляться на оплату через ваш аккаунт AWS. +После завершения этого шага ваша организация будет подключена к этой подписке AWS, и все использование будет тарифицироваться через ваш AWS‑аккаунт. -На странице выставления счетов организации в интерфейсе ClickHouse вы можете убедиться, что биллинг теперь действительно связан с AWS Marketplace. +Вы можете убедиться на странице биллинга организации в интерфейсе ClickHouse, что биллинг теперь действительно связан с AWS Marketplace. -Подтверждение на странице выставления счетов +Подтверждение на странице биллинга
- - ## Поддержка {#support} -Если у вас возникнут какие-либо проблемы, обращайтесь в [нашу службу поддержки](https://clickhouse.com/support/program). +Если у вас возникнут проблемы, обращайтесь в [нашу службу поддержки](https://clickhouse.com/support/program). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx index 3b558008273..92da42236dd 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['выставление счетов', 'передача данных по сети', 'исходящий трафик данных', 'затраты', 'тарифы'] --- -import NetworkPricing from '@site/docs/cloud/reference/_snippets/_network_transfer_rates.md'; +import NetworkPricing from '@site/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/_snippets/_network_transfer_rates.md'; ClickHouse Cloud измеряет объем входящего и исходящего трафика данных. Сюда включаются любые данные, поступающие в ClickHouse Cloud и исходящие из него, а также diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md index fb7196cc702..6c1eb47aa27 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md @@ -7,9 +7,9 @@ keywords: ['выставление счетов', 'платёжные порог doc_type: 'guide' --- -# Платёжные пороги +# Платёжные пороги {#payment-thresholds} -Когда сумма к оплате за расчётный период в ClickHouse Cloud достигает 10 000 долларов США или эквивалентного значения, с вашего способа оплаты автоматически будет списана эта сумма. В случае неудачного списания по истечении льготного периода произойдёт приостановка или прекращение предоставления услуг. +Когда сумма к оплате за расчётный период в ClickHouse Cloud достигает 10 000 долларов США или эквивалентного значения, с вашего способа оплаты автоматически будет списана эта сумма. В случае неудачного списания по истечении льготного периода произойдёт приостановка или прекращение предоставления услуг. :::note Этот платёжный порог не применяется к клиентам, у которых есть контракт с обязательствами по расходам или иное согласованное договорное соглашение с ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md index c8be1b20642..a626d5451a4 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md @@ -11,7 +11,7 @@ import billing_compliance from '@site/static/images/cloud/manage/billing_complia import Image from '@theme/IdealImage'; -# Соответствие биллинга ClickHouse Cloud нормативным требованиям +# Соответствие биллинга ClickHouse Cloud нормативным требованиям {#clickhouse-cloud-billing-compliance} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md index 1d55cea14f4..d73ce271c60 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md @@ -10,7 +10,7 @@ doc_type: 'reference' import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge' -# Поддерживаемые регионы облака +# Поддерживаемые регионы облака {#supported-cloud-regions} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md index 76ba9a1f762..b1672c3c75a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md @@ -10,8 +10,7 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-settings-sidebar.png'; - -# Настройка параметров +# Настройка параметров {#configuring-settings} Чтобы задать настройки для сервиса ClickHouse Cloud для конкретного [пользователя](/operations/access-rights#user-account-management) или [роли](/operations/access-rights#role-management), необходимо использовать [профили настроек на основе SQL](/operations/access-rights#settings-profiles-management). Применение профилей настроек гарантирует, что настроенные вами параметры сохранятся, даже когда ваши сервисы останавливаются, простаивают или обновляются. Чтобы узнать больше о профилях настроек, см. [эту страницу](/operations/settings/settings-profiles.md). @@ -19,4 +18,4 @@ import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-setti Чтобы узнать больше о параметрах, которые вы можете задать для сервиса ClickHouse Cloud, ознакомьтесь со всеми возможными настройками по категориям в [нашей документации](/operations/settings). -Боковая панель настроек Cloud \ No newline at end of file +Боковая панель настроек Cloud \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md index edbf038555c..f11b063df23 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md @@ -10,7 +10,7 @@ import BetaBadge from '@theme/badges/BetaBadge'; import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge'; -# Отчеты по безопасности и соответствию требованиям +# Отчеты по безопасности и соответствию требованиям {#security-and-compliance-reports} ClickHouse оценивает потребности наших клиентов в области безопасности и соответствия требованиям и постоянно расширяет программу по мере появления запросов на дополнительные отчеты. Для получения дополнительной информации или загрузки отчетов посетите наш [Центр доверия](https://trust.clickhouse.com). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md index d6deef65810..c9cd437f449 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md @@ -42,7 +42,7 @@ doc_type: 'guide' -## Запрос на закрытие учетной записи +## Запрос на закрытие учетной записи {#request-account-closure} Мы обязаны подтверждать подлинность запросов как на закрытие, так и на удаление. Чтобы обеспечить быструю обработку вашего запроса, выполните шаги, описанные ниже. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/index.md index 981f51e6dce..3f155deeacc 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/cloud/reference/index.md @@ -7,7 +7,7 @@ description: 'Посадочная страница справочного ра doc_type: 'landing-page' --- -# Справочник по ClickHouse Cloud +# Справочник по ClickHouse Cloud {#cloud-reference} Этот раздел служит справочным руководством по более техническим аспектам ClickHouse Cloud и содержит следующие страницы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/glossary.md b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/glossary.md index a2f99fd44b2..fe2c746f7ac 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/glossary.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/glossary.md @@ -10,7 +10,7 @@ doc_type: 'reference' {/* no-glossary */ } -# Глоссарий +# Глоссарий {#glossary} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/olap.md b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/olap.md index 62f06dd3cdb..4d2ba1dc70c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/olap.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/olap.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Что такое OLAP? +# Что такое OLAP? {#what-is-olap} [OLAP](https://en.wikipedia.org/wiki/Online_analytical_processing) расшифровывается как Online Analytical Processing. Это обобщающий термин, на который можно смотреть с двух точек зрения: технической и с точки зрения бизнеса. На самом базовом уровне можно просто прочитать эти слова в обратном порядке: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx index 1e9f2b6055e..026e1f11fb7 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx @@ -18,7 +18,7 @@ doc_type: 'guide' ## Уровень хранения: одновременные вставки изолированы друг от друга \{#storage-layer-concurrent-inserts-are-isolated-from-each-other\} - + + + + + + + \ No newline at end of file + - + ## Как работают Projections? {#how-do-projections-work} -Практически Projection можно рассматривать как дополнительную, скрытую таблицу -к исходной таблице. Проекция может иметь иной порядок строк и, следовательно, -другой первичный индекс по сравнению с исходной таблицей, а также может -автоматически и по мере вставки данных предварительно вычислять агрегированные значения. -В результате использование Projections дает два «рычага настройки» для -ускорения выполнения запросов: +На практике Projection можно рассматривать как дополнительную скрытую таблицу для +исходной таблицы. Projection может иметь иной порядок строк и, следовательно, +другой первичный индекс по сравнению с исходной таблицей, а также может автоматически +и инкрементально предварительно вычислять агрегированные значения. В результате использование Projections +предоставляет два механизма оптимизации для ускорения выполнения запросов: - **Корректное использование первичных индексов** - **Предварительное вычисление агрегатов** Projections в некотором смысле похожи на [Materialized Views](/materialized-views), -которые также позволяют иметь несколько порядков строк и предварительно -вычислять агрегации во время вставки. +которые также позволяют иметь несколько порядков строк и предварительно вычислять агрегации +в момент вставки. Projections автоматически обновляются и -поддерживаются в актуальном состоянии и синхронизированными с исходной таблицей, -в отличие от Materialized Views, которые обновляются явно. Когда запрос направлен -к исходной таблице, ClickHouse автоматически сэмплирует первичные ключи и -выбирает таблицу, которая может сгенерировать тот же корректный результат, но -требует чтения наименьшего объема данных, как показано на рисунке ниже: +остаются синхронизированными с исходной таблицей, в отличие от Materialized Views, которые +обновляются явно. Когда запрос направлен к исходной таблице, +ClickHouse автоматически выбирает первичные ключи и таблицу, которая может +сгенерировать тот же корректный результат, но требует чтения наименьшего объема данных, как показано на рисунке ниже: Projections в ClickHouse -### Более умное хранение с `_part_offset` {#smarter_storage_with_part_offset} +### Более эффективное хранение с `_part_offset` {#smarter_storage_with_part_offset} -Начиная с версии 25.5, ClickHouse поддерживает виртуальный столбец `_part_offset` -в проекциях, который предлагает новый способ определения проекции. +Начиная с версии 25.5, ClickHouse поддерживает виртуальный столбец `_part_offset` в +проекциях, что предоставляет новый способ определения проекций. -Теперь есть два способа определить проекцию: +Теперь есть два способа определения проекции: - **Хранить полные столбцы (исходное поведение)**: проекция содержит полные - данные и может читаться напрямую, обеспечивая более высокую производительность, - когда фильтры соответствуют порядку сортировки проекции. + данные и может читаться напрямую, обеспечивая более высокую производительность, когда фильтры соответствуют порядку сортировки проекции. - **Хранить только ключ сортировки + `_part_offset`**: проекция работает как индекс. - ClickHouse использует первичный индекс проекции для поиска соответствующих строк, - но читает фактические данные из базовой таблицы. Это снижает накладные расходы - на хранение ценой немного большего объема операций ввода-вывода во время выполнения запроса. + ClickHouse использует первичный индекс проекции, чтобы найти подходящие строки, но читает + фактические данные из базовой таблицы. Это снижает накладные расходы на хранение ценой + немного большего объёма операций ввода-вывода во время выполнения запроса. -Указанные подходы также можно комбинировать, храня часть столбцов в проекции, а +Эти подходы также можно комбинировать, храня часть столбцов в проекции, а остальные — косвенно через `_part_offset`. - - ## Когда использовать проекции? {#when-to-use-projections} -Проекции являются привлекательной возможностью для новых пользователей, поскольку они автоматически -поддерживаются по мере вставки данных. Кроме того, запросы могут отправляться просто к -одной таблице, где проекции при возможности используются для ускорения +Проекции — привлекательная возможность для новых пользователей, так как они автоматически +поддерживаются по мере вставки данных. Более того, запросы могут отправляться +к одной таблице, где проекции по возможности используются для ускорения времени отклика. -В отличие от этого, при использовании материализованных представлений пользователю необходимо выбирать -подходящую оптимизированную целевую таблицу или переписывать запрос в зависимости от -фильтров. Это создает большую нагрузку на пользовательские приложения и увеличивает +В отличие от материализованных представлений, где пользователю необходимо выбирать +соответствующую оптимизированную целевую таблицу или переписывать запрос в зависимости +от фильтров. Это накладывает больше требований на пользовательские приложения и увеличивает сложность на стороне клиента. -Несмотря на эти преимущества, у проекций есть ряд встроенных ограничений, о которых -пользователям следует знать, поэтому их следует использовать избирательно. +Несмотря на эти преимущества, у проекций есть некоторые присущие им ограничения, +о которых пользователям следует знать, поэтому применять их стоит выборочно. - Проекции не позволяют использовать разные TTL для исходной таблицы и - (скрытой) целевой таблицы, тогда как материализованные представления допускают разные TTL. + (скрытой) целевой таблицы, тогда как материализованные представления позволяют задавать разные TTL. - Легковесные операции обновления и удаления не поддерживаются для таблиц с проекциями. -- Материализованные представления можно выстраивать в цепочки: целевая таблица одного материализованного представления - может быть исходной таблицей для другого материализованного представления и так далее. Это +- Материализованные представления можно выстраивать в цепочку: целевая таблица одного материализованного представления + может быть исходной таблицей другого материализованного представления и так далее. Это невозможно для проекций. -- Проекции не поддерживают `JOIN`, тогда как материализованные представления поддерживают. -- Проекции не поддерживают фильтры (клауза `WHERE`), тогда как материализованные представления поддерживают. +- Определения проекций не поддерживают соединения (JOIN), тогда как материализованные представления их поддерживают. Однако запросы к таблицам с проекциями могут свободно использовать соединения. +- Определения проекций не поддерживают фильтры (оператор `WHERE`), тогда как материализованные представления их поддерживают. Однако запросы к таблицам с проекциями могут свободно использовать фильтрацию. Мы рекомендуем использовать проекции, когда: -- Требуется полная переупорядоченная организация данных. Хотя выражение в +- Требуется полное переупорядочивание данных. Хотя выражение в проекции теоретически может использовать `GROUP BY`, материализованные представления более эффективны для поддержки агрегатов. Оптимизатор запросов также с большей вероятностью будет использовать проекции, выполняющие простое переупорядочивание, то есть `SELECT * ORDER BY x`. Пользователи могут выбрать подмножество столбцов в этом выражении, чтобы уменьшить - объем хранимых данных. -- Пользователи готовы к возможному увеличению объема хранимых данных и - накладным расходам на двойную запись данных. Протестируйте влияние на скорость вставки и + занимаемый объём хранения. +- Пользователи готовы к потенциальному увеличению занимаемого объёма хранения и + накладным расходам на двукратную запись данных. Протестируйте влияние на скорость вставки и [оцените накладные расходы на хранение](/data-compression/compression-in-clickhouse). +## Примеры {#examples} - -## Примеры - -### Фильтрация по столбцам, которые не входят в первичный ключ +### Фильтрация по столбцам, которые не входят в первичный ключ {#filtering-without-using-primary-keys} В этом примере мы покажем, как добавить проекцию к таблице. -Мы также рассмотрим, как можно использовать проекцию для ускорения запросов, которые фильтруют +Мы также рассмотрим, как проекция может использоваться для ускорения запросов, которые фильтруют по столбцам, не входящим в первичный ключ таблицы. В этом примере мы будем использовать набор данных New York Taxi Data, доступный на [sql.clickhouse.com](https://sql.clickhouse.com/), который упорядочен по `pickup_datetime`. -Напишем простой запрос, чтобы найти все идентификаторы поездок, в которых пассажиры -дали водителю чаевые свыше 200 $: +Напишем простой запрос, чтобы найти все идентификаторы поездок, для которых пассажиры +дали водителю чаевые свыше $200: ```sql runnable SELECT @@ -140,8 +130,8 @@ FROM nyc_taxi.trips WHERE tip_amount > 200 AND trip_duration_min > 0 ORDER BY tip_amount, trip_id ASC ``` -Обратите внимание, что поскольку мы фильтруем по `tip_amount`, который не участвует в `ORDER BY`, ClickHouse -вынужден выполнить полное сканирование таблицы. Ускорим этот запрос. +Обратите внимание, что из‑за того, что мы фильтруем по `tip_amount`, который не входит в `ORDER BY`, ClickHouse +приходится выполнять полное сканирование таблицы. Давайте ускорим этот запрос. Чтобы сохранить исходную таблицу и результаты, мы создадим новую таблицу и скопируем данные с помощью `INSERT INTO SELECT`: @@ -150,7 +140,7 @@ CREATE TABLE nyc_taxi.trips_with_projection AS nyc_taxi.trips; INSERT INTO nyc_taxi.trips_with_projection SELECT * FROM nyc_taxi.trips; ``` -Чтобы добавить проекцию, мы используем оператор `ALTER TABLE` вместе с оператором `ADD PROJECTION`: +Чтобы добавить проекцию, используем оператор `ALTER TABLE` вместе с оператором `ADD PROJECTION`: ```sql ALTER TABLE nyc_taxi.trips_with_projection @@ -161,9 +151,9 @@ ADD PROJECTION prj_tip_amount ) ``` -После добавления проекции необходимо выполнить оператор `MATERIALIZE PROJECTION`, -чтобы данные в ней были физически упорядочены и перезаписаны -в соответствии с указанным выше запросом: +После добавления проекции необходимо использовать оператор `MATERIALIZE PROJECTION`, +чтобы данные в ней были физически отсортированы и перезаписаны в соответствии +с приведённым выше запросом: ```sql ALTER TABLE nyc.trips_with_projection MATERIALIZE PROJECTION prj_tip_amount @@ -180,9 +170,9 @@ FROM nyc_taxi.trips_with_projection WHERE tip_amount > 200 AND trip_duration_min ORDER BY tip_amount, trip_id ASC ``` -Обратите внимание, что нам удалось существенно сократить время выполнения запроса и при этом сканировать меньше строк. +Обратите внимание, что нам удалось существенно сократить время выполнения запроса и при этом просканировать меньше строк. -Мы можем подтвердить, что приведённый выше запрос действительно использовал созданную нами проекцию, обратившись к таблице `system.query_log`: +Мы можем подтвердить, что наш запрос выше действительно использовал созданную нами проекцию, обратившись к таблице `system.query_log`: ```sql SELECT query, projections @@ -200,18 +190,18 @@ WHERE query_id='' └───────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────┘ ``` -### Использование проекций для ускорения запросов к данным UK Price Paid -Чтобы продемонстрировать, как проекции могут использоваться для ускорения выполнения запросов, рассмотрим пример с использованием реального набора данных. В этом примере мы будем -использовать таблицу из нашего руководства [UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid) -с 30,03 миллионами строк. Этот набор данных также доступен в нашей среде -[sql.clickhouse.com](https://sql.clickhouse.com/?query_id=6IDMHK3OMR1C97J6M9EUQS). +### Использование проекций для ускорения запросов к данным UK price paid {#using-projections-to-speed-up-UK-price-paid} -Если вы хотите посмотреть, как была создана таблица и как в неё были вставлены данные, вы можете -обратиться к разделу [«The UK property prices dataset»](/getting-started/example-datasets/uk-price-paid). +Чтобы продемонстрировать, как проекции могут использоваться для ускорения выполнения запросов, +рассмотрим пример на реальном наборе данных. В этом примере мы будем +использовать таблицу из руководства [UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid), +содержащую 30,03 миллиона строк. Этот набор данных также доступен в +среде [sql.clickhouse.com](https://sql.clickhouse.com/?query_id=6IDMHK3OMR1C97J6M9EUQS). -Мы можем выполнить два простых запроса к этому набору данных. Первый выводит список графств Лондона, -в которых были заплачены самые высокие цены, а второй вычисляет среднюю цену по этим графствам: +Если вы хотите узнать, как была создана таблица и загружены данные, обратитесь к странице ["Набор данных о ценах на недвижимость в Великобритании"](/getting-started/example-datasets/uk-price-paid). + +Мы можем выполнить два простых запроса к этому набору данных. Первый выводит список районов Лондона с наибольшими суммами оплаты, а второй вычисляет среднюю цену по районам: ```sql runnable SELECT @@ -223,7 +213,6 @@ ORDER BY price DESC LIMIT 3 ``` - ```sql runnable SELECT county, @@ -234,7 +223,7 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -Обратите внимание, что, несмотря на очень высокую скорость выполнения, для обоих запросов был выполнен полный скан всей таблицы с 30,03 миллионами строк из‑за того, что ни `town`, ни `price` не были указаны в нашем операторе `ORDER BY` при создании таблицы: +Обратите внимание, что несмотря на высокую скорость выполнения, для обоих запросов было выполнено полное сканирование всех 30,03 миллионов строк, так как ни `town`, ни `price` не были включены в ORDER BY при создании таблицы: ```sql CREATE TABLE uk.uk_price_paid @@ -246,18 +235,19 @@ ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2); ``` -Давайте посмотрим, удастся ли нам ускорить выполнение этого запроса с помощью проекций. +Проверим, можно ли ускорить этот запрос с помощью проекций. -Чтобы сохранить исходную таблицу и результаты, мы создадим новую таблицу и скопируем данные с помощью оператора `INSERT INTO SELECT`: +Чтобы сохранить исходную таблицу и результаты, создадим новую таблицу и скопируем данные с помощью `INSERT INTO SELECT`: ```sql CREATE TABLE uk.uk_price_paid_with_projections AS uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections SELECT * FROM uk.uk_price_paid; ``` -Мы создаём и заполняем проекцию `prj_oby_town_price`, которая создаёт -дополнительную (скрытую) таблицу с первичным индексом и сортировкой по городу и цене, -чтобы оптимизировать запрос, возвращающий графства в конкретном городе по наивысшим ценам: +Создаём и заполняем проекцию `prj_oby_town_price`, которая создаёт +дополнительную (скрытую) таблицу с первичным индексом, упорядоченную по городу и цене, для +оптимизации запроса, который выводит список округов в указанном городе с максимальными +ценами: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -276,12 +266,11 @@ ALTER TABLE uk.uk_price_paid_with_projections SETTINGS mutations_sync = 1 ``` -Параметр [`mutations_sync`](/operations/settings/settings#mutations_sync) -используется для принудительного синхронного выполнения. +Настройка [`mutations_sync`](/operations/settings/settings#mutations_sync) используется для принудительного синхронного выполнения. -Мы создаём и заполняем проекцию `prj_gby_county` — дополнительную (скрытую) таблицу, -которая инкрементально предварительно вычисляет агрегатные значения avg(price) для всех 130 существующих -графств Великобритании: +Создаём и заполняем проекцию `prj_gby_county` — дополнительную (скрытую) таблицу, +которая инкрементно предвычисляет агрегированные значения avg(price) для всех существующих +130 округов Великобритании: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -301,19 +290,18 @@ SETTINGS mutations_sync = 1 ``` :::note -Если в проекции, как в `prj_gby_county` выше, используется предложение `GROUP BY`, -то базовым движком хранения для (скрытой) таблицы становится `AggregatingMergeTree`, -а все агрегатные функции преобразуются в `AggregateFunction`. Это обеспечивает -корректную инкрементальную агрегацию данных. +Если в проекции используется предложение `GROUP BY`, как в проекции `prj_gby_county` +выше, то базовым движком хранения для (скрытой) таблицы +становится `AggregatingMergeTree`, и все агрегатные функции преобразуются в +`AggregateFunction`. Это обеспечивает правильную инкрементную агрегацию данных. ::: -Рисунок ниже представляет собой визуализацию основной таблицы `uk_price_paid_with_projections` -и её двух проекций: +На рисунке ниже показана визуализация основной таблицы `uk_price_paid_with_projections` +и двух её проекций: -Визуализация основной таблицы uk_price_paid_with_projections и её двух проекций +Визуализация основной таблицы uk_price_paid_with_projections и двух её проекций -Если теперь снова выполнить запрос, который выводит округа Лондона с тремя -наибольшими ценами покупки, мы увидим улучшение производительности запроса: +Если теперь снова выполнить запрос, который выводит районы Лондона с тремя самыми высокими ценами продажи, мы увидим улучшение производительности запроса: ```sql runnable SELECT @@ -325,7 +313,8 @@ ORDER BY price DESC LIMIT 3 ``` -Аналогично для запроса, выводящего графства Великобритании с тремя наибольшими средними ценами покупки: +Аналогично для запроса, который выводит три округа Великобритании с наибольшими +средними ценами: ```sql runnable SELECT @@ -337,21 +326,18 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -Обратите внимание, что оба запроса обращаются к исходной таблице и что оба -запроса привели к полному сканированию таблицы (все 30,03 миллиона строк были -прочитаны с диска) до того, как мы создали две проекции. +Обратите внимание, что оба запроса обращаются к исходной таблице, и оба запроса привели к полному сканированию таблицы (все 30,03 миллиона строк были считаны с диска) до создания двух проекций. -Также обратите внимание, что запрос, который перечисляет графства в Лондоне по -трём самым высоким ценам продажи, считывает (стримит) 2,17 миллиона строк. Когда мы -напрямую использовали вторую таблицу, специально оптимизированную под этот запрос, с диска -было прочитано всего 81,92 тысячи строк. +Также обратите внимание, что запрос, который выводит графства Лондона с тремя +наиболее высокими ценами, считывает в потоковом режиме 2,17 миллиона строк. Когда мы использовали вторую таблицу, +оптимизированную под этот запрос, с диска было прочитано только 81,92 тысячи строк. -Причина этой разницы в том, что в данный момент оптимизация -`optimize_read_in_order`, упомянутая выше, не поддерживается для проекций. +Причина этой разницы в том, что в настоящее время оптимизация `optimize_read_in_order`, +упомянутая выше, не поддерживается для проекций. -Мы проверяем таблицу `system.query_log`, чтобы увидеть, что ClickHouse -автоматически использовал две проекции для двух приведённых выше запросов -(см. столбец projections ниже): +Мы анализируем таблицу `system.query_log` и видим, что ClickHouse +автоматически использовал две проекции для двух приведённых выше запросов (см. столбец +projections ниже): ```sql SELECT @@ -367,10 +353,9 @@ ORDER BY initial_query_start_time DESC FORMAT Vertical ``` - ```response Строка 1: -───────── +────── tables: ['uk.uk_price_paid_with_projections'] query: SELECT county, @@ -384,7 +369,7 @@ read_rows: 132.00 projections: ['uk.uk_price_paid_with_projections.prj_gby_county'] Строка 2: -───────── +────── tables: ['uk.uk_price_paid_with_projections'] query: SELECT county, @@ -398,23 +383,25 @@ query_duration: 11 мс read_rows: 2.29 млн projections: ['uk.uk_price_paid_with_projections.prj_obj_town_price'] -Получено 2 строки. Прошло: 0.006 сек. +2 строки. Затрачено: 0.006 сек. ``` -### Дополнительные примеры -В следующих примерах используется тот же набор данных по ценам в Великобритании, чтобы сравнить запросы с проекциями и без них. +### Дополнительные примеры {#further-examples} -Чтобы сохранить нашу исходную таблицу (и производительность), мы снова создаём копию таблицы с помощью `CREATE AS` и `INSERT INTO SELECT`. +В следующих примерах используется тот же набор данных с ценами в Великобритании, и сравниваются запросы с использованием проекций и без них. + +Чтобы сохранить нашу исходную таблицу (и производительность), мы снова создадим копию таблицы с помощью `CREATE AS` и `INSERT INTO SELECT`. ```sql CREATE TABLE uk.uk_price_paid_with_projections_v2 AS uk.uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections_v2 SELECT * FROM uk.uk_price_paid; ``` -#### Создание проекции -Создадим агрегирующую проекцию по измерениям `toYear(date)`, `district` и `town`: +#### Построим проекцию {#build-projection} + +Давайте создадим агрегатную проекцию по измерениям `toYear(date)`, `district` и `town`: ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -434,7 +421,7 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 ) ``` -Заполните проекцию для существующих данных. (Если не выполнять материализацию, проекция будет создаваться только для вновь вставляемых данных): +Заполните проекцию для существующих данных. (Без материализации проекция будет создаваться только для данных, вставляемых после этого): ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -442,9 +429,10 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 SETTINGS mutations_sync = 1 ``` -Следующие запросы сравнивают производительность с проекциями и без них. Для отключения использования проекций используется настройка [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections), которая включена по умолчанию. +Следующие запросы сравнивают производительность при использовании проекций и без них. Чтобы отключить использование проекций, мы используем настройку [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections), которая включена по умолчанию. + -#### Запрос 1. Средняя цена по годам +#### Запрос 1. Средняя годовая цена {#average-price-projections} ```sql runnable SELECT @@ -468,9 +456,10 @@ ORDER BY year ASC ``` -Результаты должны совпадать, но во втором примере производительность будет лучше! +Результат должен быть таким же, но производительность во втором примере будет лучше! -#### Запрос 2. Средняя цена по годам для Лондона + +#### Запрос 2. Средняя цена по годам в Лондоне {#average-price-london-projections} ```sql runnable SELECT @@ -495,9 +484,10 @@ GROUP BY year ORDER BY year ASC ``` -#### Запрос 3. Самые дорогие районы -Условие (date >= '2020-01-01') необходимо изменить так, чтобы оно соответствовало измерению проекции (`toYear(date) >= 2020)`: +#### Запрос 3. Самые дорогие районы {#most-expensive-neighborhoods-projections} + +Условие (date >= '2020-01-01') нужно изменить так, чтобы оно соответствовало измерению проекции (`toYear(date) >= 2020)`: ```sql runnable SELECT @@ -517,7 +507,6 @@ LIMIT 100 SETTINGS optimize_use_projections=0 ``` - ```sql runnable SELECT town, @@ -535,24 +524,25 @@ ORDER BY price DESC LIMIT 100 ``` -Снова результат тот же, но обратите внимание на улучшение производительности второго запроса. +Результат по-прежнему тот же, но обратите внимание на улучшение производительности второго запроса. -### Комбинирование проекций в одном запросе -Начиная с версии 25.6, на основе поддержки `_part_offset`, -появившейся в предыдущей версии, ClickHouse теперь может использовать несколько -проекций для ускорения одного запроса с несколькими фильтрами. +### Комбинирование проекций в одном запросе {#combining-projections} -Важно, что ClickHouse по-прежнему читает данные только из одной проекции (или базовой таблицы), -но может использовать первичные индексы других проекций, чтобы отбросить ненужные парты до чтения. -Это особенно полезно для запросов, фильтрующих по нескольким столбцам, каждый из которых +Начиная с версии 25.6, на основе поддержки `_part_offset`, добавленной в +предыдущей версии, ClickHouse теперь может использовать несколько проекций для ускорения +одного запроса с несколькими фильтрами. + +Важно, что ClickHouse по‑прежнему считывает данные только из одной проекции (или базовой таблицы), +но может использовать первичные индексы других проекций для отсечения ненужных кусков данных (parts) перед чтением. +Это особенно полезно для запросов, которые фильтруют по нескольким столбцам, при этом каждый из них может соответствовать своей проекции. -> В настоящее время этот механизм позволяет отбрасывать только целые парты. Фильтрация -> на уровне гранул пока не поддерживается. +> В настоящее время этот механизм отсекает только целые части (parts). Отсечение на уровне гранул +> пока не поддерживается. Чтобы продемонстрировать это, мы определим таблицу (с проекциями, использующими столбцы `_part_offset`) -и вставим пять примерных строк, соответствующих диаграммам выше. +и вставим пять примерных строк, соответствующих приведённым выше диаграммам. ```sql CREATE TABLE page_views @@ -594,24 +584,24 @@ INSERT INTO page_views VALUES ( ``` :::note -Примечание: в таблице используются специальные настройки для иллюстрации, -например гранулы размером в одну строку и отключённые слияния частей данных, что не рекомендуется для использования в продакшене. +Примечание: в таблице для наглядности используются нестандартные настройки, такие как гранулы по одной строке +и отключённое слияние частей (parts), что не рекомендуется для использования в продакшене. ::: -Эта конфигурация приводит к следующему: +Эта конфигурация даёт следующий результат: -* Пяти отдельным частям (по одной на каждую вставленную строку) -* По одной записи в первичном индексе на строку (в базовой таблице и в каждой проекции) +* Пять отдельных частей (по одной на каждую вставленную строку) +* По одной записи первичного индекса на строку (в базовой таблице и в каждой проекции) * Каждая часть содержит ровно одну строку -С такой конфигурацией мы выполняем запрос с фильтрацией сразу по `region` и `user_id`. +С такой конфигурацией мы выполняем запрос с фильтрацией и по `region`, и по `user_id`. Поскольку первичный индекс базовой таблицы построен по `event_date` и `id`, он здесь бесполезен, поэтому ClickHouse использует: * `region_proj` для отсечения частей по региону * `user_id_proj` для дополнительного отсечения по `user_id` -Это поведение видно с помощью `EXPLAIN projections = 1`, который показывает, +Это поведение видно при использовании `EXPLAIN projections = 1`, который показывает, как ClickHouse выбирает и применяет проекции. ```sql @@ -619,48 +609,48 @@ EXPLAIN projections=1 SELECT * FROM page_views WHERE region = 'us_west' AND user_id = 107; ``` - ```response ┌─explain────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression ((Имена проектов + Проекция)) │ - 2. │ Expression │ + 1. │ Выражение ((Project names + Projection)) │ + 2. │ Выражение │ 3. │ ReadFromMergeTree (default.page_views) │ 4. │ Проекции: │ - 5. │ Имя: region_proj │ - 6. │ Описание: Проекция проанализирована и используется для фильтрации на уровне частей │ - 7. │ Условие: (region in ['us_west', 'us_west']) │ - 8. │ Алгоритм поиска: бинарный поиск │ + 5. │ Название: region_proj │ + 6. │ Описание: проекция проанализирована и используется для фильтрации на уровне частей │ + 7. │ Условие: (region in ['us_west', 'us_west']) │ + 8. │ Алгоритм поиска: двоичный поиск │ 9. │ Части: 3 │ 10. │ Метки: 3 │ -11. │ Диапазоны: 3 │ -12. │ Строки: 3 │ -13. │ Отфильтрованные части: 2 │ -14. │ Имя: user_id_proj │ -15. │ Описание: Проекция проанализирована и используется для фильтрации на уровне частей │ -16. │ Условие: (user_id in [107, 107]) │ -17. │ Алгоритм поиска: бинарный поиск │ +11. │ Диапазоны: 3 │ +12. │ Строки: 3 │ +13. │ Filtered Части: 2 │ +14. │ Название: user_id_proj │ +15. │ Описание: проекция проанализирована и используется для фильтрации на уровне частей │ +16. │ Условие: (user_id in [107, 107]) │ +17. │ Алгоритм поиска: двоичный поиск │ 18. │ Части: 1 │ 19. │ Метки: 1 │ -20. │ Диапазоны: 1 │ -21. │ Строки: 1 │ -22. │ Отфильтрованные части: 2 │ +20. │ Диапазоны: 1 │ +21. │ Строки: 1 │ +22. │ Filtered Части: 2 │ └────────────────────────────────────────────────────────────────────────────────────────┘ ``` -Вывод `EXPLAIN` (показан выше) показывает логический план запроса сверху вниз: +Вывод `EXPLAIN` (показан выше) отображает логический план запроса, сверху вниз: -| Row number | Description | -| ---------- | ------------------------------------------------------------------------------------------------------------------------ | -| 3 | План чтения из базовой таблицы `page_views` | -| 5-13 | Использует `region_proj`, чтобы определить 3 части, где `region = 'us_west'`, отсекая 2 из 5 частей | -| 14-22 | Использует `user_id_proj`, чтобы определить 1 часть, где `user_id = 107`, дополнительно отсекая 2 из 3 оставшихся частей | -В итоге из базовой таблицы читается только **1 часть из 5**. -Комбинируя анализ индексов нескольких проекций, ClickHouse значительно сокращает объём сканируемых данных, -повышая производительность при низких накладных расходах на хранение. +| Номер строки | Описание | +|--------------|--------------------------------------------------------------------------------------------------------| +| 3 | Планирует чтение из базовой таблицы `page_views` | +| 5-13 | Использует `region_proj` для определения 3 частей, где `region = 'us_west'`, отбрасывая 2 из 5 частей | +| 14-22 | Использует `user_id_proj` для определения 1 части, где `user_id = 107`, дополнительно отбрасывая 2 из 3 оставшихся частей | +В итоге из базовой таблицы читается только **1 из 5 частей**. +За счет комбинированного анализа индексов нескольких проекций ClickHouse существенно снижает объем сканируемых данных, +повышая производительность при низких накладных расходах на хранение. ## Связанные материалы {#related-content} + - [Практическое введение в первичные индексы ClickHouse](/guides/best-practices/sparse-primary-indexes#option-3-projections) - [Материализованные представления](/docs/materialized-views) -- [ALTER PROJECTION](/sql-reference/statements/alter/projection) +- [ALTER PROJECTION](/sql-reference/statements/alter/projection) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md b/i18n/ru/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md index 452575b7335..11853a2f3c5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md @@ -1,97 +1,93 @@ --- slug: /managing-data/materialized-views-versus-projections -sidebar_label: 'Материализованные представления и проекции' +sidebar_label: 'Материализованные представления vs проекции' title: 'Материализованные представления и проекции' hide_title: false -description: 'Статья, сравнивающая материализованные представления и проекции в ClickHouse, включая сценарии использования, производительность и ограничения.' +description: 'Статья, сравнивающая материализованные представления и проекции в ClickHouse, включая их варианты использования, производительность и ограничения.' doc_type: 'reference' keywords: ['materialized views', 'projections', 'differences'] --- -> Один из частых вопросов пользователей — когда следует использовать материализованные представления, а когда проекции. В этой статье мы рассмотрим ключевые различия между ними и объясним, почему в определённых сценариях вы можете предпочесть один вариант другому. +> Один из частых вопросов пользователей — когда следует использовать материализованные представления, а когда +проекции. В этой статье мы рассмотрим ключевые различия между ними и разберёмся, почему в определённых сценариях +стоит предпочесть один подход другому. +## Сводка ключевых различий {#key-differences} - -## Краткое описание ключевых различий {#key-differences} - -В таблице ниже обобщены ключевые различия между материализованными представлениями и проекциями по ряду аспектов. +В таблице ниже приведены ключевые различия между материализованными представлениями и проекциями по различным аспектам. | Аспект | Материализованные представления | Проекции | |----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Хранение данных и их расположение | Сохраняют свои результаты в **отдельной явной целевой таблице**, действуя как триггеры вставки при `INSERT` в исходную таблицу. | Проекции создают оптимизированные схемы хранения данных, которые физически **хранятся рядом с основными данными таблицы** и невидимы для пользователя. | -| Механизм обновления | Работают **синхронно** при `INSERT` в исходную таблицу (для инкрементальных материализованных представлений). Примечание: их также можно **запускать по расписанию** с помощью обновляемых материализованных представлений. | **Асинхронные** обновления в фоновом режиме при `INSERT` в основную таблицу. | -| Взаимодействие с запросами | Работа с материализованными представлениями требует **прямого запроса к целевой таблице**, то есть пользователи должны знать о существовании материализованных представлений при написании запросов. | Проекции **автоматически выбираются** оптимизатором запросов ClickHouse и являются прозрачными в том смысле, что пользователю не нужно изменять свои запросы к таблице с проекцией, чтобы её использовать. Начиная с версии 25.6 также возможно фильтровать более чем по одной проекции. | -| Обработка `UPDATE` / `DELETE` | **Не реагируют автоматически** на операции `UPDATE` или `DELETE` в исходной таблице, поскольку материализованные представления не имеют информации об исходной таблице и действуют только как триггеры вставки _в_ исходную таблицу. Это может привести к потенциальной несогласованности данных между исходной и целевой таблицами и требует обходных решений или периодического полного обновления (через обновляемое материализованное представление). | По умолчанию **несовместимы со строками `DELETED`** (особенно с lightweight‑удалениями). `lightweight_mutation_projection_mode` (v24.7+) может включить совместимость. | -| Поддержка `JOIN` | Да. Обновляемые материализованные представления могут использоваться для сложной денормализации. Инкрементальные материализованные представления срабатывают только при вставках в крайнюю левую таблицу. | Нет. Операции `JOIN` не поддерживаются в определениях проекций для фильтрации материализованных данных. | -| Условие `WHERE` в определении | Да. Условия `WHERE` могут быть включены для фильтрации данных перед материализацией. | Нет. Условия `WHERE` не поддерживаются в определениях проекций для фильтрации материализованных данных. | -| Возможность построения цепочек | Да, целевая таблица одного материализованного представления может быть источником для другого материализованного представления, что позволяет строить многостадийные конвейеры обработки. | Нет. Проекции не могут быть связаны в цепочку. | -| Поддерживаемые движки таблиц | Могут использоваться с различными движками исходных таблиц, но целевые таблицы обычно относятся к семейству `MergeTree`. | **Доступны только** для движков таблиц семейства `MergeTree`. | -| Обработка сбоев | Сбой во время вставки данных означает потерю данных в целевой таблице, что может привести к потенциальной несогласованности. | Сбои **тихо** обрабатываются в фоновом режиме. Запросы могут прозрачно сочетать материализованные и нематериализованные части данных. | -| Операционные накладные расходы | Требуют явного создания целевой таблицы и зачастую ручного начального заполнения. Управление согласованностью при `UPDATE`/`DELETE` повышает сложность. | Проекции автоматически поддерживаются и синхронизируются и, как правило, создают меньшую операционную нагрузку. | -| Совместимость с запросами `FINAL` | В целом совместимы, но часто требуют `GROUP BY` по целевой таблице. | **Не работают** с запросами `FINAL`. | -| Ленивая материализация | Да. | Следите за проблемами совместимости проекций при использовании возможностей ленивой материализации. Возможно, потребуется установить `query_plan_optimize_lazy_materialization = false`. | -| Параллельные реплики | Да. | Нет. | -| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | Да. | Да. | -| Лёгкие (lightweight) операции обновления и удаления | Да. | Нет. | - - +| Data storage and location | Сохраняют свои результаты в **отдельной, явно заданной целевой таблице**, выступая в роли триггеров вставки при `INSERT` в исходную таблицу. | Проекции создают оптимизированные структуры данных, которые физически **хранятся рядом с основными данными таблицы** и невидимы для пользователя. | +| Update mechanism | Работают **синхронно** при `INSERT` в исходную таблицу (для инкрементальных материализованных представлений). Примечание: их также можно **планировать по расписанию** с помощью обновляемых материализованных представлений. | **Асинхронные** обновления в фоновом режиме при `INSERT` в основную таблицу. | +| Query interaction | Работа с материализованными представлениями требует прямого выполнения запросов к **целевой таблице**, то есть пользователи должны знать о существовании материализованных представлений при написании запросов. | Проекции **автоматически выбираются** оптимизатором запросов ClickHouse и являются прозрачными в том смысле, что пользователю не нужно изменять свои запросы к таблице с проекцией, чтобы её использовать. Начиная с версии 25.6 также можно фильтровать более чем по одной проекции. | +| Handling `UPDATE` / `DELETE` | **Не реагируют автоматически** на операции `UPDATE` или `DELETE` над исходной таблицей, так как материализованные представления не обладают информацией об исходной таблице и работают только как триггеры вставки _в_ исходную таблицу. Это может приводить к потенциальной несвежести данных между исходной и целевой таблицами и требует обходных решений или периодического полного обновления (через обновляемое материализованное представление). | По умолчанию **несовместимы с удалёнными (`DELETED`) строками** (особенно с облегчёнными удалениями — lightweight deletes). Параметр `lightweight_mutation_projection_mode` (v24.7+) может включить совместимость. | +| `JOIN` support | Да. Обновляемые материализованные представления могут использоваться для сложной денормализации. Инкрементальные материализованные представления срабатывают только при вставках в самую левую таблицу. | Нет. Операции `JOIN` не поддерживаются в определениях проекций для фильтрации материализованных данных. Однако запросы, которые делают `JOIN` таблиц с проекциями, работают нормально — проекции оптимизируют доступ к отдельным таблицам. | +| `WHERE` clause in definition | Да. Условия `WHERE` могут быть включены для фильтрации данных до материализации. | Нет. Условия `WHERE` не поддерживаются в определениях проекций для фильтрации материализованных данных. | +| Chaining capabilities | Да, целевая таблица одного материализованного представления может быть источником для другого материализованного представления, что позволяет строить многоступенчатые конвейеры. | Нет. Проекции нельзя связывать в цепочку. | +| Applicable table engines | Могут использоваться с различными движками исходных таблиц, но целевые таблицы обычно принадлежат семейству `MergeTree`. | **Доступны только** для движков таблиц семейства `MergeTree`. | +| Failure handling | Сбой во время вставки данных означает потерю данных в целевой таблице, что может привести к несогласованности данных. | Сбои **тихо** обрабатываются в фоновом режиме. Запросы могут бесшовно комбинировать материализованные и нематериализованные части данных. | +| Operational overhead | Требуют явного создания целевой таблицы и часто ручного дозаполнения (backfill). Управление согласованностью данных при `UPDATE`/`DELETE` повышает сложность. | Проекции автоматически поддерживаются и синхронизируются и, как правило, создают меньшую операционную нагрузку. | +| `FINAL` query compatibility | В целом совместимы, но часто требуют `GROUP BY` по целевой таблице. | **Не работают** с запросами `FINAL`. | +| Lazy materialization | Да. | Следите за проблемами совместимости проекций при использовании механизмов ленивой материализации. Возможно, потребуется установить `query_plan_optimize_lazy_materialization = false`. | +| Parallel replicas | Да. | Нет. | +| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | Да. | Да. | +| Lightweight updates and deletes | Да. | Нет. | ## Сравнение материализованных представлений и проекций {#choose-between} -### Когда выбирать материализованные представления {#choosing-materialized-views} +### Когда стоит выбирать материализованные представления {#choosing-materialized-views} -Рассмотрите возможность использования материализованных представлений, когда: +Использование материализованных представлений стоит рассмотреть, когда: -- Вы работаете с **ETL в реальном времени и многостадийными конвейерами обработки данных:** необходимо выполнять сложные преобразования, агрегации или маршрутизацию данных по мере их поступления, потенциально через несколько стадий путём связывания представлений в цепочку. -- Вам требуется **сложная денормализация**: необходимо предварительно объединять данные из нескольких источников (таблиц, подзапросов или словарей) в одну таблицу, оптимизированную для запросов, особенно если допустимы периодические полные обновления с использованием обновляемых материализованных представлений. -- Вам нужен **явный контроль схемы**: требуется отдельная целевая таблица с собственной схемой и движком для предвычисленных результатов, что обеспечивает большую гибкость при моделировании данных. -- Вы хотите **фильтровать на этапе приёма данных**: необходимо отфильтровать данные _до того_, как они материализуются, уменьшая объём данных, записываемых в целевую таблицу. +- Вы работаете с **ETL в режиме реального времени и многостадийными конвейерами обработки данных:** Нужно выполнять сложные преобразования, агрегации или маршрутизацию данных по мере их поступления, возможно, через несколько стадий, связывая представления в цепочки. +- Вам требуется **сложная денормализация**: Нужно заранее объединить данные из нескольких источников (таблиц, подзапросов или словарей) в одну таблицу, оптимизированную для запросов, особенно если допустимы периодические полные обновления с использованием обновляемых материализованных представлений. +- Вам нужен **явный контроль над схемой**: Требуется отдельная целевая таблица с собственной схемой и движком для предварительно вычисленных результатов, что обеспечивает большую гибкость при моделировании данных. +- Вы хотите **фильтровать на этапе ингестии**: Нужно отфильтровать данные _до того_, как они будут материализованы, уменьшая объём данных, записываемых в целевую таблицу. ### Когда следует избегать материализованных представлений {#avoid-materialized-views} -Рассмотрите возможность отказа от использования материализованных представлений, когда: +Следует рассмотреть отказ от использования материализованных представлений, когда: -- **Исходные данные часто обновляются или удаляются**: без дополнительных стратегий обеспечения согласованности между исходной и целевой таблицами инкрементные материализованные представления могут устаревать и становиться несогласованными. -- **Предпочтительны простота и автоматическая оптимизация**: если вы хотите избежать управления отдельными целевыми таблицами. +- **Исходные данные часто обновляются или удаляются**: Без дополнительных стратегий поддержания согласованности между исходными и целевыми таблицами инкрементальные материализованные представления могут устаревать и становиться несогласованными. +- **Предпочитаются простота и автоматическая оптимизация**: Если вы не хотите управлять отдельными целевыми таблицами. -### Когда выбирать проекции {#choosing-projections} +### Когда следует использовать проекции {#choosing-projections} -Рассмотрите возможность использования проекций, когда: +Рекомендуется рассматривать использование проекций, когда: -- **Оптимизируются запросы к одной таблице**: ваша основная цель — ускорить запросы к одной базовой таблице за счёт предоставления альтернативных порядков сортировки, оптимизации фильтрации по столбцам, не входящим в первичный ключ, или предвычисления агрегатов для одной таблицы. -- Вам нужна **прозрачность запросов**: вы хотите, чтобы запросы по-прежнему были нацелены на исходную таблицу без модификаций, полагаясь на ClickHouse при выборе наилучшего формата данных для конкретного запроса. +- **Оптимизируете запросы для одной таблицы**: ваша основная цель — ускорить запросы к одной базовой таблице за счёт задания альтернативных порядков сортировки, оптимизации фильтров по столбцам, которые не входят в первичный ключ, или предварительного вычисления агрегаций для одной таблицы. +- Вам нужна **прозрачность запросов**: вы хотите, чтобы запросы по-прежнему выполнялись к исходной таблице без изменений, полагаясь на ClickHouse при выборе наилучшего расположения данных для конкретного запроса. ### Когда следует избегать проекций {#avoid-projections} -Рассмотрите возможность отказа от использования проекций, когда: - -- **Требуются сложные преобразования данных или многостадийный ETL**: проекции не поддерживают операции `JOIN` в своих определениях, не могут изменяться для построения многошаговых конвейеров и не поддерживают некоторые возможности SQL, такие как оконные функции или сложные выражения `CASE`. Поэтому они не подходят для сложных преобразований данных. -- **Нужно явное фильтрование материализованных данных**: проекции не поддерживают `WHERE` в определении для фильтрации данных, которые материализуются в самой проекции. -- **Используются таблицы на движках, отличных от MergeTree**: проекции доступны исключительно для таблиц, использующих семейство движков `MergeTree`. -- `FINAL`-запросы имеют ключевое значение: проекции не работают с `FINAL`-запросами, которые иногда используются для дедупликации. -- Вам нужны [параллельные реплики](/deployment-guides/parallel-replicas), так как они не поддерживаются проекциями. - +Рекомендуется избегать использования проекций в следующих случаях: +- **Требуется сложная трансформация данных или многоэтапный ETL**: Определения проекций не поддерживают операции `JOIN`, не могут быть объединены в цепочки для построения многошаговых конвейеров и не работают с некоторыми возможностями SQL, такими как оконные функции или сложные выражения `CASE`. Хотя запросы к таблицам с проекциями могут свободно использовать `JOIN`, сами проекции не подходят для сложной трансформации данных. +- **Нужна явная фильтрация материализованных данных**: Проекции не поддерживают использование предложения `WHERE` в определении для фильтрации данных, которые материализуются в саму проекцию. +- **Используются табличные движки, отличные от MergeTree**: Проекции доступны исключительно для таблиц, использующих семейство движков `MergeTree`. +- Запросы с `FINAL` являются критически важными: проекции не работают с запросами `FINAL`, которые иногда используются для дедупликации. +- Вам нужны [параллельные реплики](/deployment-guides/parallel-replicas), так как они не поддерживаются при использовании проекций. ## Резюме {#summary} -Материализованные представления и проекции — оба мощных инструмента в вашем арсенале -для оптимизации запросов и преобразования данных, и в целом мы не рекомендуем рассматривать -их использование как взаимоисключающий выбор. Вместо этого их можно применять -взаимодополняющим образом, чтобы получить максимум от ваших запросов. Таким образом, выбор -между материализованными представлениями и проекциями в ClickHouse действительно зависит -от вашего конкретного сценария использования и паттернов доступа. - -В качестве общего практического правила вам следует рассматривать использование -материализованных представлений, когда нужно агрегировать данные из одной или -нескольких исходных таблиц в целевую таблицу или выполнять сложные преобразования -в больших масштабах. Материализованные представления отлично подходят для переноса выполнения -дорогостоящих агрегаций с времени выполнения запроса на время вставки. Они являются -отличным выбором для ежедневных или ежемесячных сводок, дашбордов в реальном времени или -сводной информации по данным. - -С другой стороны, вам следует использовать проекции, когда нужно оптимизировать -запросы, которые фильтруют по другим столбцам, чем те, которые используются в первичном -ключе таблицы и определяют физический порядок данных на диске. Они особенно полезны, -когда больше нет возможности изменить первичный ключ таблицы или когда ваши паттерны -доступа более разнообразны, чем то, что может обеспечить первичный ключ. +Материализованные представления и проекции — это мощные инструменты в вашем арсенале +для оптимизации запросов и преобразования данных, и в целом мы не рекомендуем +рассматривать их использование как взаимоисключающий выбор. Вместо этого их можно +применять взаимодополняющим образом, чтобы получить максимальную отдачу от запросов. +Таким образом, выбор между материализованными представлениями и проекциями в ClickHouse +на самом деле зависит от конкретного сценария использования и паттернов доступа. + +В качестве общего практического правила стоит рассматривать использование +материализованных представлений, когда вам нужно агрегировать данные из одной или нескольких +исходных таблиц в целевую таблицу или выполнять сложные преобразования в больших масштабах. +Материализованные представления отлично подходят для переноса работы по выполнению +ресурсоёмких агрегирующих вычислений с момента выполнения запроса на момент вставки данных. +Это отличный выбор для ежедневных или ежемесячных сводок, дашбордов в реальном времени +или агрегированных обзоров данных. + +С другой стороны, имеет смысл использовать проекции, когда необходимо оптимизировать запросы, +которые фильтруют по другим столбцам, чем те, что используются в первичном ключе таблицы, +определяющем физический порядок данных на диске. Они особенно полезны, когда изменить +первичный ключ таблицы больше невозможно или когда ваши паттерны доступа более разнообразны, +чем то, что может обеспечить первичный ключ. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/index.md index 8a88b6e12d5..0cc75531d29 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/index.md @@ -6,7 +6,7 @@ keywords: ['руководства по развертыванию', 'масшт doc_type: 'landing-page' --- -# Развертывание и масштабирование +# Развертывание и масштабирование {#deployment-and-scaling} В этом разделе рассматриваются следующие темы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx index 3b2ba9bedcf..d47248021a3 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx @@ -20,15 +20,14 @@ import image_9 from '@site/static/images/deployment-guides/parallel-replicas-9.p - ## Введение \{#introduction\} -ClickHouse обрабатывает запросы чрезвычайно быстро, но как эти запросы -распределяются и выполняются параллельно на нескольких серверах? +ClickHouse обрабатывает запросы чрезвычайно быстро, но как эти запросы +распределяются и выполняются параллельно на нескольких серверах? > В этом руководстве мы сначала рассмотрим, как ClickHouse распределяет запрос -по нескольким шардам с помощью распределённых таблиц, а затем как запрос может -задействовать несколько реплик при выполнении. +> по нескольким шардам с помощью распределённых таблиц, а затем как запрос может +> задействовать несколько реплик при выполнении. ## Шардированная архитектура \{#sharded-architecture\} @@ -48,24 +47,27 @@ ClickHouse обрабатывает запросы чрезвычайно быс На рисунке выше показано, что происходит, когда клиент отправляет запрос к распределённой таблице:
    -
  1. - Запрос SELECT отправляется к распределённой таблице на произвольный узел - (по стратегии round-robin или после маршрутизации балансировщиком нагрузки - на конкретный сервер). Этот узел теперь будет выступать координатором. -
  2. -
  3. - Узел определяет, какие шарды должны выполнить запрос, - используя информацию, указанную в распределённой таблице, и отправляет - запрос на каждый шард. -
  4. -
  5. - Каждый шард локально читает, фильтрует и агрегирует данные, а затем - отправляет агрегируемое состояние координатору. -
  6. -
  7. - Координирующий узел объединяет данные и затем отправляет ответ - обратно клиенту. -
  8. +
  9. + Запрос SELECT отправляется к распределённой таблице на произвольный узел + (по стратегии round-robin или после маршрутизации балансировщиком нагрузки + на конкретный сервер). Этот узел теперь будет выступать координатором. +
  10. + +
  11. + Узел определяет, какие шарды должны выполнить запрос, + используя информацию, указанную в распределённой таблице, и отправляет + запрос на каждый шард. +
  12. + +
  13. + Каждый шард локально читает, фильтрует и агрегирует данные, а затем + отправляет агрегируемое состояние координатору. +
  14. + +
  15. + Координирующий узел объединяет данные и затем отправляет ответ + обратно клиенту. +
Когда мы добавляем реплики, процесс в целом остаётся похожим, с единственным @@ -75,7 +77,7 @@ ClickHouse обрабатывает запросы чрезвычайно быс ## Нешардированная архитектура \{#non-sharded-architecture\} ClickHouse Cloud имеет архитектуру, существенно отличающуюся от представленной выше. -(См. раздел ["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) +(См. раздел ["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) для получения более подробной информации). Благодаря разделению вычислительных ресурсов и хранилища, а также практически неограниченному объёму хранилища, необходимость в шардах во многом отпадает. @@ -112,26 +114,30 @@ ClickHouse Cloud имеет архитектуру, существенно от При использовании параллельных реплик:
    -
  1. - Запрос от клиента отправляется на один узел после прохождения через - балансировщик нагрузки. Этот узел становится координатором для данного запроса. -
  2. -
  3. - Узел анализирует индекс каждой партиции и выбирает подходящие партиции и - гранулы для обработки. -
  4. -
  5. - Координатор разбивает нагрузку на набор гранул, который можно - распределить между разными репликами. -
  6. -
  7. - Каждый набор гранул обрабатывается соответствующими репликами, и по - завершении на координатор отправляется состояние, пригодное для слияния. -
  8. -
  9. - Наконец, координатор объединяет все результаты от реплик и - возвращает ответ клиенту. -
  10. +
  11. + Запрос от клиента отправляется на один узел после прохождения через + балансировщик нагрузки. Этот узел становится координатором для данного запроса. +
  12. + +
  13. + Узел анализирует индекс каждой партиции и выбирает подходящие партиции и + гранулы для обработки. +
  14. + +
  15. + Координатор разбивает нагрузку на набор гранул, который можно + распределить между разными репликами. +
  16. + +
  17. + Каждый набор гранул обрабатывается соответствующими репликами, и по + завершении на координатор отправляется состояние, пригодное для слияния. +
  18. + +
  19. + Наконец, координатор объединяет все результаты от реплик и + возвращает ответ клиенту. +
Вышеописанные шаги показывают, как параллельные реплики работают в теории. @@ -139,22 +145,25 @@ ClickHouse Cloud имеет архитектуру, существенно от идеальной работе такого механизма:
    -
  1. - Некоторые реплики могут быть недоступны. -
  2. -
  3. - Репликация в ClickHouse асинхронная, поэтому в какой-то момент времени - на некоторых репликах могут отсутствовать те же самые части. -
  4. -
  5. - Нужно как‑то обрабатывать «хвостовые» задержки между репликами. -
  6. -
  7. - Кэш файловой системы различается от реплики к реплике в зависимости - от активности на каждой реплике, поэтому случайное назначение - задач может привести к менее оптимальной производительности с точки - зрения локальности кэша. -
  8. +
  9. + Некоторые реплики могут быть недоступны. +
  10. + +
  11. + Репликация в ClickHouse асинхронная, поэтому в какой-то момент времени + на некоторых репликах могут отсутствовать те же самые части. +
  12. + +
  13. + Нужно как‑то обрабатывать «хвостовые» задержки между репликами. +
  14. + +
  15. + Кэш файловой системы различается от реплики к реплике в зависимости + от активности на каждой реплике, поэтому случайное назначение + задач может привести к менее оптимальной производительности с точки + зрения локальности кэша. +
В следующих разделах мы рассмотрим, как удаётся преодолеть эти факторы. @@ -167,30 +176,33 @@ ClickHouse Cloud имеет архитектуру, существенно от Объявления
    -
  1. - Запрос от клиента после прохождения через балансировщик нагрузки - отправляется на один из узлов. Этот узел становится координатором для данного запроса. -
  2. -
  3. - Узел-координатор отправляет запрос на получение объявлений от всех - реплик в кластере. У реплик могут быть немного разные представления о - текущем наборе частей (parts) таблицы. Поэтому нам нужно - собрать эту информацию, чтобы избежать неверных решений при - планировании. -
  4. -
  5. - Затем узел-координатор использует объявления, чтобы определить набор - гранул, которые можно назначить различным репликам. Здесь, например, - мы видим, что ни одна гранула из part 3 не была назначена - реплике 2, потому что эта реплика не указала эту часть в своем - объявлении. Также обратите внимание, что реплике 3 не были назначены - никакие задачи, потому что она не предоставила объявление. -
  6. -
  7. - После того как каждая реплика обработала запрос на своем подмножестве - гранул и объединяемое состояние было отправлено обратно координатору, - координатор объединяет результаты, и ответ отправляется клиенту. -
  8. +
  9. + Запрос от клиента после прохождения через балансировщик нагрузки + отправляется на один из узлов. Этот узел становится координатором для данного запроса. +
  10. + +
  11. + Узел-координатор отправляет запрос на получение объявлений от всех + реплик в кластере. У реплик могут быть немного разные представления о + текущем наборе частей (parts) таблицы. Поэтому нам нужно + собрать эту информацию, чтобы избежать неверных решений при + планировании. +
  12. + +
  13. + Затем узел-координатор использует объявления, чтобы определить набор + гранул, которые можно назначить различным репликам. Здесь, например, + мы видим, что ни одна гранула из part 3 не была назначена + реплике 2, потому что эта реплика не указала эту часть в своем + объявлении. Также обратите внимание, что реплике 3 не были назначены + никакие задачи, потому что она не предоставила объявление. +
  14. + +
  15. + После того как каждая реплика обработала запрос на своем подмножестве + гранул и объединяемое состояние было отправлено обратно координатору, + координатор объединяет результаты, и ответ отправляется клиенту. +
### Динамическая координация \{#dynamic-coordination\} @@ -208,42 +220,46 @@ ClickHouse Cloud имеет архитектуру, существенно от Динамическая координация — часть 1
    -
  1. - Реплики уведомляют узел-координатор, что они готовы обрабатывать - задания, также они могут указать, какой объём работы могут выполнить. -
  2. -
  3. - Координатор назначает задания репликам. -
  4. +
  5. + Реплики уведомляют узел-координатор, что они готовы обрабатывать + задания, также они могут указать, какой объём работы могут выполнить. +
  6. + +
  7. + Координатор назначает задания репликам. +
Динамическая координация — часть 2
    -
  1. - Реплики 1 и 2 могут очень быстро завершить свои задания. Они - запрашивают у узла-координатора новое задание. -
  2. -
  3. - Координатор назначает новые задания репликам 1 и 2. -
  4. +
  5. + Реплики 1 и 2 могут очень быстро завершить свои задания. Они + запрашивают у узла-координатора новое задание. +
  6. + +
  7. + Координатор назначает новые задания репликам 1 и 2. +
Динамическая координация — часть 3
    -
  1. - Все реплики завершили обработку своих заданий. Они - запрашивают дополнительные задания. -
  2. -
  3. - Координатор, используя объявления, проверяет, какие задания - ещё нужно обработать, но оставшихся заданий нет. -
  4. -
  5. - Координатор сообщает репликам, что всё обработано. - Теперь он объединит все состояния, подлежащие слиянию, и вернёт ответ на запрос. -
  6. +
  7. + Все реплики завершили обработку своих заданий. Они + запрашивают дополнительные задания. +
  8. + +
  9. + Координатор, используя объявления, проверяет, какие задания + ещё нужно обработать, но оставшихся заданий нет. +
  10. + +
  11. + Координатор сообщает репликам, что всё обработано. + Теперь он объединит все состояния, подлежащие слиянию, и вернёт ответ на запрос. +
### Управление локальностью кэша \{#managing-cache-locality\} @@ -253,34 +269,38 @@ ClickHouse Cloud имеет архитектуру, существенно от реплику? В предыдущем примере у нас были назначены следующие задачи: - - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Реплика 1Реплика 2Реплика 3
Часть 1g1, g6, g7g2, g4, g5g3
Часть 2g1g2, g4, г5g3
Часть 3g1, g6g2, g4, g5g3
+ + Реплика 1Реплика 2Реплика 3
Часть 1g1, g6, g7g2, g4, g5g3
Часть 2g1g2, g4, г5g3
Часть 3g1, g6g2, g4, g5g3
Чтобы гарантировать, что одни и те же задачи назначаются одним и тем же репликам и могут @@ -322,13 +342,13 @@ ClickHouse Cloud имеет архитектуру, существенно от | Setting | Description | |----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `enable_parallel_replicas` | `0`: отключено
`1`: включено
`2`: принудительное использование параллельных реплик, будет сгенерировано исключение, если они не используются. | +| `enable_parallel_replicas` | `0`: отключено
`1`: включено
`2`: принудительное использование параллельных реплик, будет сгенерировано исключение, если они не используются. | | `cluster_for_parallel_replicas` | Имя кластера, используемого для параллельной репликации; если вы используете ClickHouse Cloud, используйте `default`. | | `max_parallel_replicas` | Максимальное количество реплик, используемых для выполнения запроса на нескольких репликах; если указано число, меньшее количества реплик в кластере, узлы будут выбираться случайным образом. Это значение также может быть завышено с учетом горизонтального масштабирования. | -| `parallel_replicas_min_number_of_rows_per_replica` | Позволяет ограничить количество используемых реплик на основе числа строк, которые необходимо обработать. Количество реплик определяется как:
`estimated rows to read` / `min_number_of_rows_per_replica`. | -| `allow_experimental_analyzer` | `0`: использовать старый анализатор
`1`: использовать новый анализатор.

Поведение параллельных реплик может изменяться в зависимости от используемого анализатора. | +| `parallel_replicas_min_number_of_rows_per_replica` | Позволяет ограничить количество используемых реплик на основе числа строк, которые необходимо обработать. Количество реплик определяется как:
`estimated rows to read` / `min_number_of_rows_per_replica`. | +| `allow_experimental_analyzer` | `0`: использовать старый анализатор
`1`: использовать новый анализатор.

Поведение параллельных реплик может изменяться в зависимости от используемого анализатора. | -## Диагностика проблем с параллельными репликами +## Диагностика проблем с параллельными репликами \{#investigating-issues-with-parallel-replicas\} Вы можете проверить, какие настройки используются для каждого запроса, в таблице [`system.query_log`](/docs/operations/system-tables/query_log). Также можно @@ -345,7 +365,6 @@ FROM clusterAllReplicas('default', system.events) WHERE event ILIKE '%ParallelReplicas%' ``` -
Ответ @@ -407,7 +426,6 @@ WHERE query_id = 'ad40c712-d25d-45c4-b1a1-a28ba8d4019c' ORDER BY event_time_microseconds ASC ``` -
Ответ diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md index 422b9c6299b..a296abf4133 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md @@ -9,19 +9,19 @@ keywords: ['репликация', 'высокая доступность', 'н --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ReplicationArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/replication.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > В этом примере вы узнаете, как настроить простой кластер ClickHouse, > который реплицирует данные. Настроено пять серверов. Два из них используются для @@ -34,7 +34,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## Предварительные требования {#pre-requisites} - Ранее вы уже развернули [локальный сервер ClickHouse](/install) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md index 7427ef5c9a6..58bcf8416d9 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md @@ -9,19 +9,19 @@ keywords: ['шардинг', 'горизонтальное масштабиро --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ShardingArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/sharding.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > В этом примере вы узнаете, как настроить простой масштабируемый кластер ClickHouse. > Настроено пять серверов: два используются для шардинга данных, @@ -33,7 +33,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## Предварительные требования {#pre-requisites} - Вы уже развернули [локальный сервер ClickHouse](/install) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md index cb9c3396663..39f4926da33 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md @@ -10,14 +10,14 @@ keywords: ['развертывание кластера', 'репликация' import Image from '@theme/IdealImage'; import SharedReplicatedArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/both.png'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import KeeperConfig from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > В этом примере вы узнаете, как развернуть простой кластер ClickHouse, который > одновременно обеспечивает репликацию и масштабирование. Он состоит из двух шардов и двух реплик, @@ -30,7 +30,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## Предварительные требования {#prerequisites} - Вы уже развернули [локальный сервер ClickHouse](/install) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md index 79c7c7312a7..939fb1924f7 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md @@ -8,7 +8,7 @@ doc_type: 'guide' keywords: ['развертывание', 'архитектура', 'репликация', 'шардинг', 'настройка кластера'] --- -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; Примеры развертывания в этом разделе основаны на рекомендациях, которые организация ClickHouse Support and Services предоставляет пользователям ClickHouse. Это рабочие примеры, и мы рекомендуем опробовать их, а затем адаптировать под свои нужды. Возможно, вы найдете здесь пример, который точно соответствует вашим требованиям. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/architecture.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/architecture.md index 15fe339f87b..68393afea8a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/architecture.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/architecture.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Обзор архитектуры +# Обзор архитектуры {#architecture-overview} ClickHouse — это по-настоящему колоночная СУБД. Данные хранятся по столбцам и при выполнении запросов обрабатываются массивами (векторами или чанками столбцов). По возможности операции выполняются над массивами, а не над отдельными значениями. @@ -242,7 +242,7 @@ IO‑пул потоков реализован как обычный `ThreadPoo -## Контроль параллелизма +## Контроль параллелизма {#concurrency-control} Запрос, который может выполняться параллельно, использует настройку `max_threads`, чтобы ограничить количество потоков. Значение по умолчанию для этой настройки выбирается таким образом, чтобы один запрос мог наилучшим образом использовать все ядра CPU. Но что, если есть несколько одновременно выполняющихся запросов, и каждый из них использует значение `max_threads` по умолчанию? Тогда запросы будут делить ресурсы CPU. ОС будет обеспечивать справедливое распределение, постоянно переключая потоки, что вносит дополнительные накладные расходы и снижает производительность. `ConcurrencyControl` помогает снизить эти накладные расходы и избежать создания слишком большого числа потоков. Настройка конфигурации `concurrent_threads_soft_limit_num` используется для ограничения того, сколько одновременно выполняющихся потоков может быть выделено до начала применения механизма ограничения нагрузки на CPU. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-arm.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-arm.md index f501a0db522..0a9f8a05c4e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-arm.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-arm.md @@ -7,7 +7,7 @@ title: 'Как собрать ClickHouse на Linux для AARCH64' doc_type: 'guide' --- -# Как собрать ClickHouse на Linux для AArch64 +# Как собрать ClickHouse на Linux для AArch64 {#how-to-build-clickhouse-on-linux-for-aarch64} Для сборки ClickHouse для AArch64 на машине с архитектурой AArch64 не требуются специальные действия. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md index 81da9c998c9..4909b955874 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md @@ -7,15 +7,11 @@ title: 'Сборка на Linux для LoongArch64' doc_type: 'guide' --- - - -# Сборка под Linux для LoongArch64 +# Сборка под Linux для LoongArch64 {#build-on-linux-for-loongarch64} ClickHouse экспериментально поддерживает LoongArch64 - - -## Сборка ClickHouse +## Сборка ClickHouse {#build-clickhouse} Для сборки требуется версия LLVM не ниже 19.1.0. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-osx.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-osx.md index c60419fe21c..8db0f37e816 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-osx.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-osx.md @@ -7,9 +7,7 @@ title: 'Сборка на Linux для macOS' doc_type: 'guide' --- - - -# Как собрать ClickHouse на Linux для macOS +# Как собрать ClickHouse на Linux для macOS {#how-to-build-clickhouse-on-linux-for-macos} Этот документ описывает случай, когда у вас есть машина под управлением Linux, и вы хотите использовать её для сборки бинарного файла `clickhouse`, который будет запускаться на OS X. Основной сценарий использования — проверки в системе непрерывной интеграции, выполняющиеся на Linux-машинах. @@ -21,9 +19,7 @@ doc_type: 'guide' Если вы нацеливаетесь на архитектуру ARM, просто замените все вхождения `x86_64` на `aarch64`. Например, замените `x86_64-apple-darwin` на `aarch64-apple-darwin` на всех этапах. - - -## Установите набор инструментов для кросс-компиляции +## Установите набор инструментов для кросс-компиляции {#install-cross-compilation-toolset} Запомните путь, по которому установлен `cctools`, и обозначьте его как `${CCTOOLS}` @@ -53,8 +49,7 @@ cd ClickHouse/cmake/toolchain/darwin-x86_64 curl -L 'https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX11.0.sdk.tar.xz' | tar xJ --strip-components=1 ``` - -## Сборка ClickHouse +## Сборка ClickHouse {#build-clickhouse} ```bash cd ClickHouse diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md index 2dc84d6b925..5a005dfb09a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md @@ -7,15 +7,11 @@ title: 'Как собрать ClickHouse на Linux для RISC-V 64' doc_type: 'guide' --- - - -# Как собрать ClickHouse на Linux для RISC-V 64 +# Как собрать ClickHouse на Linux для RISC-V 64 {#how-to-build-clickhouse-on-linux-for-risc-v-64} В ClickHouse есть экспериментальная поддержка архитектуры RISC-V. Доступны не все возможности. - - -## Сборка ClickHouse +## Сборка ClickHouse {#build-clickhouse} Для кросс-компиляции под RISC-V на машине, не основанной на RISC-V: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md index f746fd08ccf..ce92400081d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md @@ -7,39 +7,24 @@ title: 'Сборка на Linux для s390x (zLinux)' doc_type: 'guide' --- +# Сборка в Linux для s390x (zLinux) {#build-on-linux-for-s390x-zlinux} +ClickHouse в экспериментальном режиме поддерживает архитектуру s390x. -# Сборка в Linux для s390x (zLinux) +## Сборка ClickHouse для s390x {#building-clickhouse-for-s390x} -В ClickHouse имеется экспериментальная поддержка архитектуры s390x. +На платформе s390x, как и на других платформах, OpenSSL собирается как статическая библиотека. Если вы хотите собрать с динамическим OpenSSL, необходимо передать `-DENABLE_OPENSSL_DYNAMIC=1` в CMake. - - -## Сборка ClickHouse для s390x - -Для s390x есть две опции сборки, связанные с OpenSSL: - -* По умолчанию OpenSSL на s390x собирается как динамическая библиотека. Это отличается от всех остальных платформ, где OpenSSL собирается как статическая библиотека. -* Чтобы, несмотря на это, собрать OpenSSL как статическую библиотеку, передайте `-DENABLE_OPENSSL_DYNAMIC=0` в CMake. - -В этих инструкциях предполагается, что хост‑система — x86_64 и на ней установлены все инструменты, необходимые для нативной сборки, согласно [инструкциям по сборке](../development/build.md). Также предполагается, что хост работает под управлением Ubuntu 22.04, но следующие инструкции также должны работать на Ubuntu 20.04. +В этих инструкциях предполагается, что хостовая система — Linux x86_64/ARM и на ней установлены все инструменты, необходимые для нативной сборки в соответствии с [инструкцией по сборке](../development/build.md). Также предполагается, что хост — Ubuntu 22.04, но приведённые ниже инструкции также должны работать на Ubuntu 20.04. Помимо установки инструментов, используемых для нативной сборки, необходимо установить следующие дополнительные пакеты: ```bash -apt-get install binutils-s390x-linux-gnu libc6-dev-s390x-cross gcc-s390x-linux-gnu binfmt-support qemu-user-static -``` - -Если вы хотите выполнить кросс-компиляцию кода на Rust, установите целевой таргет Rust для архитектуры s390x: - -```bash +apt-get mold rustup target add s390x-unknown-linux-gnu ``` -Для сборки под s390x используется линкер mold. Скачайте его по ссылке [https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz](https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz) -и добавьте его в `$PATH`. - -Чтобы выполнить сборку для s390x: +Сборка для s390x: ```bash cmake -DCMAKE_TOOLCHAIN_FILE=cmake/linux/toolchain-s390x.cmake .. @@ -47,30 +32,37 @@ ninja ``` -## Запуск +## Запуск {#running} + +Для эмуляции вам понадобится статический бинарник qemu-user для s390x. В Ubuntu его можно установить с помощью: + +```bash +apt-get install binfmt-support binutils-s390x-linux-gnu qemu-user-static +``` -После сборки бинарного файла его можно запустить, например, так: +После сборки бинарный файл можно запустить, например, так: ```bash -qemu-s390x-static -L /usr/s390x-linux-gnu ./clickhouse +qemu-s390x-static -L /usr/s390x-linux-gnu ./programs/clickhouse local --query "Select 2" +2 ``` -## Отладка +## Отладка {#debugging} Установите LLDB: ```bash -apt-get install lldb-15 +apt-get install lldb-21 ``` -Чтобы отладить исполняемый файл s390x, запустите ClickHouse под QEMU в отладочном режиме: +Чтобы отладить исполняемый файл s390x, запустите ClickHouse с помощью QEMU в режиме отладки: ```bash qemu-s390x-static -g 31338 -L /usr/s390x-linux-gnu ./clickhouse ``` -В другом терминале запустите LLDB и подключитесь к процессу, заменив `` и `` на значения, соответствующие вашей среде. +В другом терминале запустите LLDB и присоединитесь к процессу, заменив `` и `` на значения, соответствующие вашей среде. ```bash lldb-15 @@ -102,16 +94,16 @@ Process 1 stopped ``` -## Интеграция с Visual Studio Code +## Интеграция с Visual Studio Code {#visual-studio-code-integration} -* Для визуальной отладки требуется расширение [CodeLLDB](https://github.com/vadimcn/vscode-lldb). -* Расширение [Command Variable](https://github.com/rioj7/command-variable) может помочь с динамическим запуском при использовании [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md). -* Убедитесь, что в качестве бэкенда указана ваша установка LLVM, например: `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so"`. -* Перед запуском обязательно запустите исполняемый файл clickhouse в режиме отладки. (Также можно создать `preLaunchTask`, который автоматизирует это.) +- Для визуальной отладки требуется расширение [CodeLLDB](https://github.com/vadimcn/vscode-lldb). +- Расширение [Command Variable](https://github.com/rioj7/command-variable) может упростить динамический запуск при использовании [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md). +- Убедитесь, что бекенд настроен на вашу установку LLVM, например: `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so"`. +- Перед запуском визуальной отладки предварительно запустите исполняемый файл clickhouse в режиме отладки. (Также можно создать задачу `preLaunchTask`, которая автоматизирует это.) -### Примеры конфигураций +### Примеры конфигураций {#example-configurations} -#### cmake-variants.yaml +#### cmake-variants.yaml {#cmake-variantsyaml} ```yaml buildType: @@ -127,11 +119,11 @@ buildType: buildType: Release relwithdebinfo: short: RelWithDebInfo - long: Релиз с отладочной информацией + long: Релизная сборка с отладочной информацией buildType: RelWithDebInfo tsan: short: MinSizeRel - long: Релиз минимального размера + long: Релизная сборка минимального размера buildType: MinSizeRel toolchain: @@ -148,7 +140,8 @@ toolchain: CMAKE_TOOLCHAIN_FILE: cmake/linux/toolchain-s390x.cmake ``` -#### launch.json + +#### launch.json {#launchjson} ```json { @@ -160,24 +153,26 @@ toolchain: "name": "(lldb) Запуск s390x с qemu", "targetCreateCommands": ["target create ${command:cmake.launchTargetPath}"], "processCreateCommands": ["gdb-remote 2159"], - "preLaunchTask": "Запустить ClickHouse" + "preLaunchTask": "Запуск ClickHouse" } ] } ``` -#### settings.json -Это также поместит разные сборки в разные подкаталоги каталога `build`. +#### settings.json {#settingsjson} + +Это также поместит разные сборки в разные подпапки в каталоге `build`. ```json { "cmake.buildDirectory": "${workspaceFolder}/build/${buildKitVendor}-${buildKitVersion}-${variant:toolchain}-${variant:buildType}", - "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so" + "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so" } ``` -#### run-debug.sh + +#### run-debug.sh {#run-debugsh} ```sh #! /bin/sh @@ -186,9 +181,10 @@ cd $1 qemu-s390x-static -g 2159 -L /usr/s390x-linux-gnu $2 $3 $4 ``` -#### tasks.json -Определяет задачу для запуска скомпилированного исполняемого файла в режиме `server` в папке `tmp` рядом с бинарными файлами, с использованием конфигурации из `programs/server/config.xml`. +#### tasks.json {#tasksjson} + +Определяет задачу для запуска скомпилированного исполняемого файла в режиме `server` в подкаталоге `tmp` рядом с бинарными файлами, с использованием конфигурации из файла `programs/server/config.xml`. ```json { diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-e2k.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-e2k.md index 55afd833d76..bf39aaab6de 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-e2k.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-e2k.md @@ -7,15 +7,11 @@ title: 'Сборка на Linux для E2K' doc_type: 'guide' --- - - -# Сборка на Linux для E2K +# Сборка на Linux для E2K {#build-on-linux-for-e2k} В ClickHouse имеется крайне экспериментальная поддержка архитектуры E2K (Эльбрус‑2000), и он может быть скомпилирован только в нативном режиме с минимальной конфигурацией, используя специально собранные для E2K библиотеки, такие как Boost, CRoaring, libunwind, Zstd. - - -## Сборка ClickHouse +## Сборка ClickHouse {#build-clickhouse} Требуемая версия LLVM для сборки должна быть не ниже 20.1.8. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-osx.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-osx.md index 256755ec317..268a7a472cf 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build-osx.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build-osx.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# Как собрать ClickHouse на macOS для macOS +# Как собрать ClickHouse на macOS для macOS {#how-to-build-clickhouse-on-macos-for-macos} :::info Вам не нужно собирать ClickHouse самостоятельно! Вы можете установить предварительно собранный ClickHouse, как описано в разделе [Quick Start](https://clickhouse.com/#quick-start). @@ -22,7 +22,7 @@ ClickHouse можно скомпилировать на macOS x86_64 (Intel) и -## Установка необходимых компонентов +## Установка необходимых компонентов {#install-prerequisites} Сначала ознакомьтесь с общей [документацией по предварительным требованиям](developer-instruction.md). @@ -53,7 +53,7 @@ mkdir build export PATH=$(brew --prefix llvm)/bin:$PATH cmake -S . -B build cmake --build build -# Итоговый исполняемый файл будет создан по пути: build/programs/clickhouse +# Итоговый исполняемый файл будет создан по пути: build/programs/clickhouse {#the-resulting-binary-will-be-created-at-buildprogramsclickhouse} ``` :::note @@ -62,7 +62,7 @@ cmake --build build ::: -## Особенности +## Особенности {#caveats} Если вы планируете запускать `clickhouse-server`, убедитесь, что значение системной переменной `maxfiles` увеличено. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/build.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/build.md index 2c7bead66c9..ce75f3f75d1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/build.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/build.md @@ -1,5 +1,5 @@ --- -description: 'Пошаговое руководство по сборке ClickHouse из исходного кода на Linux-системах' +description: 'Пошаговое руководство по сборке ClickHouse из исходников в операционных системах Linux' sidebar_label: 'Сборка на Linux' sidebar_position: 10 slug: /development/build @@ -7,15 +7,13 @@ title: 'Как собрать ClickHouse на Linux' doc_type: 'guide' --- - - -# Как собрать ClickHouse под Linux +# Как собрать ClickHouse под Linux {#how-to-build-clickhouse-on-linux} :::info Вам не обязательно собирать ClickHouse самостоятельно! -Вы можете установить уже собранный ClickHouse, как описано в разделе [Quick Start](https://clickhouse.com/#quick-start). +Вы можете установить уже собранный ClickHouse, как описано в разделе [Быстрый старт](https://clickhouse.com/#quick-start). ::: -ClickHouse можно собрать на следующих платформах: +ClickHouse может быть собран на следующих платформах: - x86_64 - AArch64 @@ -23,24 +21,20 @@ ClickHouse можно собрать на следующих платформа - s390/x (экспериментально) - RISC-V 64 (экспериментально) - - ## Предположения {#assumptions} -Данное руководство основано на Ubuntu Linux, но при соответствующих изменениях оно должно работать и на любом другом дистрибутиве Linux. +Данное руководство рассчитано на использование в Ubuntu Linux, но при соответствующей настройке оно также должно работать и на любом другом дистрибутиве Linux. Минимально рекомендуемая версия Ubuntu для разработки — 24.04 LTS. -В руководстве предполагается, что у вас локально клонирован репозиторий ClickHouse со всеми подмодулями. - - +Руководство исходит из того, что вы локально клонировали репозиторий ClickHouse со всеми подмодулями. -## Установка предварительных требований +## Установка необходимых зависимостей {#install-prerequisites} Сначала ознакомьтесь с общей [документацией по предварительным требованиям](developer-instruction.md). -ClickHouse при сборке использует CMake и Ninja. +ClickHouse использует CMake и Ninja для сборки. -При желании вы можете установить ccache, чтобы сборка повторно использовала уже скомпилированные объектные файлы. +При желании вы можете установить ccache, чтобы при сборке повторно использовать уже скомпилированные объектные файлы. ```bash sudo apt-get update @@ -48,34 +42,32 @@ sudo apt-get install build-essential git cmake ccache python3 ninja-build nasm y ``` -## Установите компилятор Clang +## Установите компилятор Clang {#install-the-clang-compiler} -Чтобы установить Clang на Ubuntu/Debian, используйте скрипт автоматической установки LLVM с сайта [apt.llvm.org](https://apt.llvm.org/). +Чтобы установить Clang в Ubuntu/Debian, используйте автоматический скрипт установки LLVM, доступный [здесь](https://apt.llvm.org/). ```bash sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ``` -Для других дистрибутивов Linux проверьте, доступны ли для установки какие-либо [предварительно собранные пакеты LLVM](https://releases.llvm.org/download.html). +Для других дистрибутивов Linux проверьте, можно ли установить один из [предварительно собранных пакетов LLVM](https://releases.llvm.org/download.html). -По состоянию на март 2025 года требуется Clang 19 или новее. -GCC и другие компиляторы не поддерживаются. +По состоянию на март 2025 года требуется Clang 19 или выше. +Компилятор GCC и другие компиляторы не поддерживаются. -## Установите компилятор Rust (необязательно) {#install-the-rust-compiler-optional} +## Установка компилятора Rust (необязательно) {#install-the-rust-compiler-optional} :::note -Rust является необязательной зависимостью ClickHouse. -Если Rust не установлен, некоторые возможности ClickHouse не будут включены в сборку. +Rust является необязательной зависимостью для ClickHouse. +Если Rust не установлен, некоторые функции ClickHouse не будут включены в сборку. ::: Сначала выполните шаги из официальной [документации по Rust](https://www.rust-lang.org/tools/install), чтобы установить `rustup`. -Как и для зависимостей C++, ClickHouse использует vendoring, чтобы точно контролировать, что именно устанавливается, и избежать зависимости от сторонних сервисов (таких как реестр `crates.io`). - -Хотя в режиме release любая современная версия toolchain в rustup для Rust должна работать с этими зависимостями, если вы планируете включить санитайзеры, необходимо использовать версию, которая использует точно такую же `std`, как и та, что применяется в CI (для которой мы вендорим crates): - +Так же, как и для зависимостей C++, ClickHouse использует vendoring, чтобы точно контролировать состав устанавливаемых компонентов и не зависеть от сторонних сервисов (таких как реестр `crates.io`). +Хотя в режиме release любая современная версия toolchain `rustup` должна работать с этими зависимостями, если вы планируете включить санитайзеры, необходимо использовать версию, у которой `std` в точности совпадает с той, что используется в CI (для которой мы вендорим соответствующие crates): ```bash rustup toolchain install nightly-2025-07-07 @@ -83,34 +75,35 @@ rustup default nightly-2025-07-07 rustup component add rust-src ``` -## Сборка ClickHouse -Рекомендуем создать отдельный каталог `build` внутри `ClickHouse`, в котором будут находиться все артефакты сборки: +## Сборка ClickHouse {#build-clickhouse} + +Мы рекомендуем создать отдельный каталог `build` внутри `ClickHouse`, в котором будут храниться все артефакты сборки: ```sh mkdir build cd build ``` -Вы можете создавать несколько разных каталогов (например, `build_release`, `build_debug` и т.д.) для различных типов сборки. +Вы можете использовать несколько разных каталогов (например, `build_release`, `build_debug` и т. д.) для разных типов сборок. -Необязательно: если у вас установлено несколько версий компилятора, вы можете при необходимости указать, какой компилятор использовать. +Необязательный шаг: если у вас установлено несколько версий компилятора, при необходимости вы можете указать конкретный компилятор, который следует использовать. ```sh export CC=clang-19 export CXX=clang++-19 ``` -Для разработки рекомендуется использовать отладочные сборки. -По сравнению с релизными сборками, у них ниже уровень оптимизации компилятора (`-O`), что обеспечивает более удобную отладку. -Кроме того, внутренние исключения типа `LOGICAL_ERROR` приводят к немедленному падению вместо корректной обработки ошибки. +Для целей разработки рекомендуется использовать отладочные сборки. +По сравнению с релизными, у них ниже уровень оптимизации компилятора (`-O`), что обеспечивает более удобную отладку. +Также внутренние исключения типа `LOGICAL_ERROR` приводят к немедленному аварийному завершению работы вместо корректной обработки ошибки. ```sh cmake -D CMAKE_BUILD_TYPE=Debug .. ``` :::note -Если вы хотите использовать отладчик, например gdb, добавьте `-D DEBUG_O_LEVEL="0"` к приведённой выше команде, чтобы отключить все оптимизации компилятора, которые могут мешать gdb просматривать и получать доступ к переменным. +Если вы хотите использовать отладчик, такой как gdb, добавьте `-D DEBUG_O_LEVEL="0"` к приведённой выше команде, чтобы полностью отключить оптимизации компилятора, которые могут мешать gdb просматривать переменные и получать к ним доступ. ::: Запустите ninja для сборки: @@ -119,7 +112,7 @@ cmake -D CMAKE_BUILD_TYPE=Debug .. ninja clickhouse ``` -Если вы хотите собрать все бинарные артефакты (утилиты и тесты), запустите ninja без параметров: +Если вы хотите собрать все двоичные файлы (утилиты и тесты), запустите команду `ninja` без параметров: ```sh ninja @@ -132,54 +125,55 @@ ninja -j 1 clickhouse-server clickhouse-client ``` :::tip -CMake предоставляет сокращённые формы для приведённых выше команд: +CMake предоставляет сокращённые варианты для приведённых выше команд: ```sh -cmake -S . -B build # конфигурация сборки, запускается из корневого каталога репозитория +cmake -S . -B build # настройка сборки, запускается из корневого каталога репозитория cmake --build build # компиляция ``` ::: -## Запуск исполняемого файла ClickHouse +## Запуск исполняемого файла ClickHouse {#running-the-clickhouse-executable} -После успешного завершения сборки вы найдете исполняемый файл в `ClickHouse//programs/`: +После успешной сборки вы найдёте исполняемый файл в `ClickHouse//programs/`: Сервер ClickHouse пытается найти файл конфигурации `config.xml` в текущем каталоге. -Также вы можете указать файл конфигурации в командной строке с помощью параметра `-C`. +При необходимости вы можете указать другой файл конфигурации в командной строке через параметр `-C`. -Чтобы подключиться к серверу ClickHouse с помощью `clickhouse-client`, откройте другой терминал, перейдите в `ClickHouse/build/programs/` и выполните `./clickhouse client`. +Чтобы подключиться к серверу ClickHouse с помощью `clickhouse-client`, откройте ещё один терминал, перейдите в `ClickHouse/build/programs/` и выполните `./clickhouse client`. -Если вы видите сообщение `Connection refused` на macOS или FreeBSD, попробуйте указать адрес хоста 127.0.0.1: +Если в macOS или FreeBSD вы получаете сообщение `Connection refused`, попробуйте указать адрес хоста 127.0.0.1: ```bash clickhouse client --host 127.0.0.1 ``` -## Расширенные параметры +## Расширенные настройки {#advanced-options} -### Минимальная сборка +### Минимальная сборка {#minimal-build} -Если вам не нужна функциональность, предоставляемая сторонними библиотеками, вы можете ещё больше ускорить сборку: +Если вам не нужна функциональность, предоставляемая сторонними библиотеками, вы можете ещё больше ускорить процесс сборки: ```sh cmake -DENABLE_LIBRARIES=OFF ``` -В случае проблем вам придётся разбираться самостоятельно ... +В случае проблем вам придётся разбираться самостоятельно… -Для работы Rust требуется подключение к интернету. Чтобы отключить поддержку Rust: +Rust требует подключения к интернету. Чтобы отключить поддержку Rust: ```sh cmake -DENABLE_RUST=OFF ``` -### Запуск исполняемого файла ClickHouse -Вы можете заменить продакшн-версию бинарного файла ClickHouse, установленную в вашей системе, скомпилированным бинарным файлом ClickHouse. -Для этого установите ClickHouse на свою машину, следуя инструкциям на официальном веб‑сайте. +### Запуск исполняемого файла ClickHouse {#running-the-clickhouse-executable-1} + +Вы можете заменить продакшн-версию бинарного файла ClickHouse, установленную в вашей системе, на скомпилированный бинарный файл ClickHouse. +Для этого установите ClickHouse на свою машину, следуя инструкциям на официальном сайте. Затем выполните: ```bash @@ -188,18 +182,19 @@ sudo cp ClickHouse/build/programs/clickhouse /usr/bin/ sudo service clickhouse-server start ``` -Обратите внимание, что `clickhouse-client`, `clickhouse-server` и другие — это символьные ссылки на общий исполняемый файл `clickhouse`. +Обратите внимание, что `clickhouse-client`, `clickhouse-server` и другие — это симлинки на общий бинарный файл `clickhouse`. -Вы также можете запустить собранный вами исполняемый файл ClickHouse с файлом конфигурации из пакета ClickHouse, установленного в вашей системе: +Вы также можете запустить собранный вами бинарный файл ClickHouse с файлом конфигурации из пакета ClickHouse, установленного в вашей системе: ```bash sudo service clickhouse-server stop sudo -u clickhouse ClickHouse/build/programs/clickhouse server --config-file /etc/clickhouse-server/config.xml ``` -### Сборка на любом дистрибутиве Linux -Установите необходимые компоненты в дистрибутиве openSUSE Tumbleweed: +### Сборка в любом дистрибутиве Linux {#building-on-any-linux} + +Установите необходимые зависимости в дистрибутиве openSUSE Tumbleweed: ```bash sudo zypper install git cmake ninja clang-c++ python lld nasm yasm gawk @@ -209,7 +204,7 @@ cmake -S . -B build cmake --build build ``` -Установите необходимые пакеты в Fedora Rawhide: +Установите необходимые зависимости на Fedora Rawhide: ```bash sudo yum update @@ -220,19 +215,20 @@ cmake -S . -B build cmake --build build ``` -### Сборка в Docker -Вы можете запустить любую сборку локально в среде, аналогичной CI, с помощью: +### Сборка в Docker {#building-in-docker} + +Вы можете запустить любую сборку локально в среде, аналогичной CI, используя: ```bash -python -m ci.praktika "BUILD_JOB_NAME" +python -m ci.praktika run "BUILD_JOB_NAME" ``` -где BUILD_JOB_NAME — это имя job'а, отображаемое в отчёте CI, например: "Build (arm_release)", "Build (amd_debug)" +где BUILD_JOB_NAME — это имя задания (job) так, как оно указано в отчёте CI, например, "Build (arm_release)", "Build (amd_debug)". Эта команда загружает соответствующий Docker-образ `clickhouse/binary-builder` со всеми необходимыми зависимостями -и запускает внутри него скрипт сборки: `./ci/jobs/build_clickhouse.py` +и запускает внутри него скрипт сборки: `./ci/jobs/build_clickhouse.py`. -Результат сборки будет размещён в `./ci/tmp/`. +Результат сборки будет помещён в `./ci/tmp/`. -Команда работает как на архитектуре AMD, так и на ARM и не требует дополнительных зависимостей, кроме Docker. +Эта команда работает как на архитектуре AMD, так и на ARM и не требует никаких дополнительных зависимостей, кроме установленного Python с модулем `requests` и Docker. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md index dbf56a721c2..3d0b2524235 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Сборка ClickHouse с DEFLATE_QPL +# Сборка ClickHouse с DEFLATE_QPL {#build-clickhouse-with-deflate_qpl} - Убедитесь, что ваша хост-система удовлетворяет требуемым для QPL [предварительным требованиям](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#prerequisites) - deflate_qpl включён по умолчанию при сборке с помощью CMake. Если вы случайно изменили это, пожалуйста, проверьте флаг сборки: ENABLE_QPL=1 @@ -18,7 +18,7 @@ doc_type: 'guide' -# Запуск бенчмарка с DEFLATE_QPL +# Запуск бенчмарка с DEFLATE_QPL {#run-benchmark-with-deflate_qpl} @@ -35,7 +35,7 @@ doc_type: 'guide' -## Автоматический запуск бенчмарка для схемы «звезда»: +## Автоматический запуск бенчмарка для схемы «звезда»: {#run-benchmark-automatically-for-star-schema} ```bash $ cd ./benchmark_sample/client_scripts @@ -53,7 +53,7 @@ $ sh run_ssb.sh -## Среда +## Среда {#environment} * CPU: Sapphire Rapids * Требования к ОС см. раздел [System Requirements for QPL](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#system-requirements) @@ -81,7 +81,7 @@ $ accel-config list | grep -P 'iax|state' Если вывода нет, это означает, что IAA еще не готов к работе. Пожалуйста, проверьте настройку IAA. -## Генерация необработанных данных +## Генерация необработанных данных {#generate-raw-data} ```bash $ cd ./benchmark_sample @@ -94,7 +94,7 @@ $ mkdir rawdata_dir && cd rawdata_dir Файлы с расширением `*.tbl` должны быть сгенерированы в каталоге `./benchmark_sample/rawdata_dir/ssb-dbgen`: -## Настройка базы данных +## Настройка базы данных {#database-setup} Настройте базу данных с использованием кодека LZ4 @@ -164,7 +164,7 @@ SELECT count() FROM lineorder_flat Это означает, что устройства IAA не готовы; вам нужно заново проверить их настройку. -## Тестирование производительности с одним экземпляром +## Тестирование производительности с одним экземпляром {#benchmark-with-single-instance} * Перед началом бенчмарка отключите режим C6 и переведите регулятор частоты CPU в режим `performance` @@ -218,7 +218,7 @@ zstd.log Нас интересует показатель QPS. Найдите по ключевому слову `QPS_Final` и соберите статистику. -## Тестирование производительности с несколькими инстансами +## Тестирование производительности с несколькими инстансами {#benchmark-with-multi-instances} * Чтобы снизить влияние ограничений по памяти при использовании слишком большого числа потоков, мы рекомендуем запускать тестирование производительности с несколькими инстансами. * Конфигурация с несколькими инстансами означает использование нескольких (2 или 4) серверов, каждый из которых подключён к своему клиенту. @@ -350,7 +350,7 @@ zstd_2insts.log Мы рекомендуем использовать данные бенчмарка для 2 инстансов в качестве итогового отчёта для рассмотрения. -## Советы +## Советы {#tips} Каждый раз перед запуском нового сервера ClickHouse убедитесь, что не осталось запущенных фоновых процессов ClickHouse; при необходимости найдите и завершите старые процессы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/continuous-integration.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/continuous-integration.md index b9b73dc23ec..157b57572ba 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/continuous-integration.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/continuous-integration.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Непрерывная интеграция (CI) +# Непрерывная интеграция (CI) {#continuous-integration-ci} Когда вы отправляете pull request, для вашего кода выполняются автоматические проверки системой ClickHouse [непрерывной интеграции (CI)](tests.md#test-automation). Это происходит после того, как сопровождающий репозитория (кто‑то из команды ClickHouse) просмотрел ваш код и добавил к вашему pull request метку `can be tested`. @@ -75,26 +75,26 @@ git push -## Проверка стиля +## Проверка стиля {#style-check} Выполняет различные проверки стиля кода. В задаче Style Check выполняются следующие базовые проверки: -##### cpp +##### cpp {#cpp} Выполняет простые проверки стиля кода на основе регулярных выражений с помощью скрипта [`ci/jobs/scripts/check_style/check_cpp.sh`](https://github.com/ClickHouse/ClickHouse/blob/master/ci/jobs/scripts/check_style/check_cpp.sh) (его также можно запускать локально).\ Если проверка завершается с ошибкой, исправьте проблемы со стилем в соответствии с [руководством по стилю кода](style.md). -##### codespell, aspell +##### codespell, aspell {#codespell} Проверяют грамматические ошибки и опечатки. -##### mypy +##### mypy {#mypy} Выполняет статическую проверку типов для кода на Python. -### Запуск задачи проверки стиля локально +### Запуск задачи проверки стиля локально {#running-style-check-locally} Всю задачу *Style Check* можно запустить локально в Docker-контейнере с помощью: @@ -112,14 +112,14 @@ python -m ci.praktika run "Style check" --test cpp Дополнительные зависимости не требуются — достаточно Python 3 и Docker. -## Быстрый тест +## Быстрый тест {#fast-test} Обычно это первая проверка, которая запускается для PR. Она собирает ClickHouse и запускает большинство [stateless-функциональных тестов](tests.md#functional-tests), пропуская некоторые. Если она не проходит, последующие проверки не запускаются, пока проблема не будет устранена. Ознакомьтесь с отчетом, чтобы увидеть, какие тесты завершились с ошибкой, затем воспроизведите сбой локально, как описано [здесь](/development/tests#running-a-test-locally). -#### Запуск быстрого теста локально: +#### Запуск быстрого теста локально: {#running-fast-test-locally} ```sh python -m ci.praktika run "Fast test" [--test имя_теста] @@ -129,11 +129,11 @@ python -m ci.praktika run "Fast test" [--test имя_теста] Никаких зависимостей, кроме Python 3 и Docker, не требуется. -## Проверка сборки +## Проверка сборки {#build-check} Выполняет сборку ClickHouse в различных конфигурациях для использования на следующих шагах. -### Локальный запуск сборки +### Локальный запуск сборки {#running-builds-locally} Сборку можно запустить локально в среде, имитирующей CI, с помощью: @@ -143,7 +143,7 @@ python -m ci.praktika run "<ИМЯ_ЗАДАНИЯ_СБОРКИ>" Никаких дополнительных зависимостей, кроме Python 3 и Docker, не требуется. -#### Доступные задания сборки +#### Доступные задания сборки {#available-build-jobs} Имена заданий сборки указаны в точности так, как они отображаются в отчёте CI: @@ -181,7 +181,7 @@ python -m ci.praktika run "<ИМЯ_ЗАДАНИЯ_СБОРКИ>" **Примечание:** Для сборок, не относящихся к категории «Другие архитектуры» (сборки из категории «Другие архитектуры» используют кросс-компиляцию), архитектура вашей локальной машины должна совпадать с типом сборки, чтобы она была выполнена так, как запрошено в `BUILD_JOB_NAME`. -#### Пример +#### Пример {#example-run-local} Чтобы выполнить локальную отладочную сборку: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/contrib.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/contrib.md index 5ea6eb0e29f..c2a7840bcf2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/contrib.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/contrib.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Сторонние библиотеки +# Сторонние библиотеки {#third-party-libraries} ClickHouse использует сторонние библиотеки для различных целей, например, для подключения к другим базам данных, декодирования/кодирования данных при чтении/записи с диска/на диск или для реализации отдельных специализированных SQL‑функций. Чтобы не зависеть от набора библиотек, доступных в целевой системе, каждая сторонняя библиотека импортируется как подмодуль Git в дерево исходного кода ClickHouse и затем компилируется и компонуется вместе с ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/developer-instruction.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/developer-instruction.md index b87e28ac2b4..df3508713d5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/developer-instruction.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/developer-instruction.md @@ -1,5 +1,5 @@ --- -description: 'Предварительные требования и инструкции по настройке среды разработки ClickHouse' +description: 'Предварительные требования и инструкции по настройке для разработки с использованием ClickHouse' sidebar_label: 'Предварительные требования' sidebar_position: 5 slug: /development/developer-instruction @@ -7,46 +7,42 @@ title: 'Предварительные требования для разраб doc_type: 'guide' --- +# Предварительные требования {#prerequisites} +ClickHouse можно собрать под Linux, FreeBSD и macOS. +Если вы используете Windows, вы всё равно можете собрать ClickHouse в виртуальной машине с установленным Linux, например в [VirtualBox](https://www.virtualbox.org/) с Ubuntu. -# Предварительные требования +## Создание репозитория на GitHub {#create-a-repository-on-github} -ClickHouse можно собирать под Linux, FreeBSD и macOS. -Если вы используете Windows, вы всё равно можете собирать ClickHouse в виртуальной машине с Linux, например, в [VirtualBox](https://www.virtualbox.org/) с Ubuntu. - - - -## Создание репозитория на GitHub - -Чтобы начать вносить вклад в разработку ClickHouse, вам понадобится учетная запись на [GitHub](https://www.github.com/). -Также сгенерируйте локально SSH-ключ (если у вас его ещё нет) и загрузите открытый ключ в GitHub, так как это является необходимым условием для отправки патчей. +Чтобы начать разрабатывать для ClickHouse, вам понадобится аккаунт на [GitHub](https://www.github.com/). +Также сгенерируйте локально SSH‑ключ (если у вас его ещё нет) и загрузите открытый ключ в GitHub, так как это является необходимым условием для отправки патчей. Затем форкните [репозиторий ClickHouse](https://github.com/ClickHouse/ClickHouse/) в свой личный аккаунт, нажав кнопку «fork» в правом верхнем углу. -Чтобы внести изменения, например, исправление ошибки или новую функциональность, сначала закоммитьте свои изменения в ветку в вашем форке, затем создайте «Pull Request» с этими изменениями в основной репозиторий. +Чтобы внести изменения — например, исправление ошибки или новую функциональность, — сначала закоммитьте изменения в ветку в своём форке, затем создайте Pull Request с этими изменениями в основной репозиторий. -Для работы с Git-репозиториями установите Git. Например, в Ubuntu выполните: +Для работы с Git‑репозиториями установите Git. Например, в Ubuntu выполните: ```sh sudo apt update sudo apt install git ``` -Шпаргалка по Git доступна [здесь](https://education.github.com/git-cheat-sheet-education.pdf). +Шпаргалку по Git можно найти [здесь](https://education.github.com/git-cheat-sheet-education.pdf). Подробное руководство по Git доступно [здесь](https://git-scm.com/book/en/v2). -## Клонируйте репозиторий на свою локальную машину +## Клонируйте репозиторий на рабочую машину {#clone-the-repository-to-your-development-machine} -Сначала скачайте исходные файлы на локальную машину, то есть клонируйте репозиторий: +Сначала скачайте исходные файлы на рабочую машину, то есть клонируйте репозиторий: ```sh -git clone git@github.com:your_github_username/ClickHouse.git # замените заполнитель своим именем пользователя GitHub +git clone git@github.com:your_github_username/ClickHouse.git # замените your_github_username на ваше имя пользователя GitHub cd ClickHouse ``` Эта команда создаёт директорию `ClickHouse/`, содержащую исходный код, тесты и другие файлы. -Вы можете указать собственную директорию для клонирования после URL, но важно, чтобы этот путь не содержал пробелов, так как это может привести к сбою сборки на более позднем этапе. +Вы можете указать собственный каталог для клонирования после URL, но важно, чтобы этот путь не содержал пробелов, так как это может привести к сбою сборки в дальнейшем. Git-репозиторий ClickHouse использует подмодули для подключения сторонних библиотек. Подмодули по умолчанию не клонируются. @@ -54,9 +50,9 @@ Git-репозиторий ClickHouse использует подмодули д * запустить `git clone` с опцией `--recurse-submodules`, -* если `git clone` запущен без `--recurse-submodules`, выполнить `git submodule update --init --jobs `, чтобы явно клонировать все подмодули. (`` можно, например, установить в `12`, чтобы параллелизировать загрузку.) +* если `git clone` выполняется без `--recurse-submodules`, запустить `git submodule update --init --jobs `, чтобы явно клонировать все подмодули. (`` можно, например, установить в `12`, чтобы распараллелить загрузку.) -* если `git clone` запущен без `--recurse-submodules` и вы хотите использовать [sparse](https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/) и [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) клонирование подмодулей, чтобы исключить ненужные файлы и историю в подмодулях и сэкономить место (около 5 GB вместо примерно 15 GB), выполните `./contrib/update-submodules.sh`. Этот вариант используется в CI, но не рекомендуется для локальной разработки, так как делает работу с подмодулями менее удобной и более медленной. +* если `git clone` выполняется без `--recurse-submodules` и вы хотите использовать [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) клонирование подмодулей, чтобы не загружать историю в подмодулях и сэкономить место, запустите `./contrib/update-submodules.sh`. Эта альтернатива используется в CI, но не рекомендуется для локальной разработки, так как делает работу с подмодулями менее удобной и более медленной. Чтобы проверить состояние Git-подмодулей, выполните `git submodule status`. @@ -70,36 +66,36 @@ fatal: Could not read from remote repository. и репозиторий существует. ``` -Отсутствуют SSH-ключи для подключения к GitHub. +SSH-ключи для подключения к GitHub не найдены. Обычно эти ключи находятся в `~/.ssh`. -Чтобы SSH-ключи были приняты, необходимо добавить их в настройках GitHub. +Чтобы SSH-ключи были приняты, нужно загрузить их в настройках GitHub. -Вы также можете клонировать репозиторий по протоколу HTTPS: +Вы также можете клонировать репозиторий через HTTPS: ```sh git clone https://github.com/ClickHouse/ClickHouse.git ``` -Однако это не позволит вам отправлять свои изменения на сервер. -Вы всё равно можете временно использовать его и позже добавить SSH-ключи, заменив удалённый URL репозитория с помощью команды `git remote`. +Однако это не позволит отправлять изменения на сервер. +Вы по-прежнему можете временно так работать и добавить SSH-ключи позже, заменив адрес удалённого репозитория с помощью команды `git remote`. -Вы также можете добавить оригинальный URL репозитория ClickHouse в свой локальный репозиторий, чтобы получать оттуда обновления: +Вы также можете добавить оригинальный адрес репозитория ClickHouse в локальный репозиторий, чтобы получать из него обновления: ```sh git remote add upstream git@github.com:ClickHouse/ClickHouse.git ``` -После успешного выполнения этой команды вы сможете получать обновления из основного репозитория ClickHouse, выполняя `git pull upstream master`. +После успешного выполнения этой команды вы сможете получать обновления из основного репозитория ClickHouse, выполнив `git pull upstream master`. :::tip -Пожалуйста, не запускайте просто `git push`, иначе вы можете отправить изменения в неправильный удалённый репозиторий и/или в неправильную ветку. +Пожалуйста, не используйте просто `git push`: так вы можете отправить изменения не в тот удалённый репозиторий и/или не в ту ветку. Лучше явно указывать имена удалённого репозитория и ветки, например: `git push origin my_branch_name`. ::: ## Написание кода {#writing-code} -Ниже вы найдёте несколько полезных ссылок, которые могут пригодиться при написании кода для ClickHouse: +Ниже приведены несколько ссылок, которые могут быть полезны при написании кода для ClickHouse: - [Архитектура ClickHouse](/development/architecture/). - [Руководство по стилю кода](/development/style/). @@ -109,53 +105,47 @@ git remote add upstream git@github.com:ClickHouse/ClickHouse.git ### IDE {#ide} -[Visual Studio Code](https://code.visualstudio.com/) и [Neovim](https://neovim.io/) — два варианта, которые хорошо зарекомендовали себя для разработки ClickHouse. Если вы используете VS Code, мы рекомендуем установить [расширение clangd](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) вместо IntelliSense, так как оно значительно быстрее и производительнее. +[Visual Studio Code](https://code.visualstudio.com/) и [Neovim](https://neovim.io/) — два проверенных варианта, которые хорошо подходят для разработки ClickHouse. Если вы используете VS Code, мы рекомендуем установить расширение [clangd](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) вместо IntelliSense, так как оно значительно быстрее и эффективнее. -[CLion](https://www.jetbrains.com/clion/) — ещё одна хорошая альтернатива. Однако на крупных проектах, таких как ClickHouse, он может работать медленнее. Несколько моментов, которые стоит учитывать при использовании CLion: +[CLion](https://www.jetbrains.com/clion/) — ещё одна отличная альтернатива. Однако он может работать медленнее на крупных проектах, таких как ClickHouse. Несколько моментов, которые стоит учитывать при использовании CLion: -- CLion создаёт собственный каталог `build` и автоматически выбирает `debug` в качестве типа сборки -- используется версия CMake, определённая в CLion, а не установленная вами -- CLion будет использовать `make` для выполнения задач сборки вместо `ninja` (это ожидаемое поведение) +- CLion самостоятельно создаёт каталог `build` и автоматически выбирает `debug` как тип сборки +- Он использует версию CMake, определённую в CLion, а не установленную вами +- CLion будет использовать `make` для выполнения задач сборки вместо `ninja` (это нормальное поведение) Другие IDE, которые вы можете использовать, — [Sublime Text](https://www.sublimetext.com/), [Qt Creator](https://www.qt.io/product/development-tools) или [Kate](https://kate-editor.org/). - - ## Создание pull request {#create-a-pull-request} -Перейдите к вашему форк-репозиторию в интерфейсе GitHub. -Если вы разрабатывали в отдельной ветке, вам нужно выбрать именно эту ветку. -На экране будет кнопка «Pull request». -По сути это означает «создать запрос на принятие моих изменений в основной репозиторий». +Перейдите к своему fork‑репозиторию в интерфейсе GitHub. +Если вы разрабатывали в отдельной ветке, выберите эту ветку. +На экране будет отображаться кнопка «Pull request». +По сути, это означает «создать запрос на принятие моих изменений в основной репозиторий». -Pull request можно создать, даже если работа еще не завершена. +Pull request можно создать даже в том случае, если работа ещё не завершена. В этом случае, пожалуйста, добавьте слово «WIP» (work in progress) в начало заголовка, его можно будет изменить позже. Это полезно для совместного ревью и обсуждения изменений, а также для запуска всех доступных тестов. -Важно, чтобы вы добавили краткое описание ваших изменений — позже оно будет использовано для формирования списка изменений (changelog) релиза. +Важно, чтобы вы добавили краткое описание своих изменений — позже оно будет использовано для формирования журнала изменений релиза. -Тестирование начнется, как только сотрудники ClickHouse пометят ваш PR меткой «can be tested». -Результаты некоторых первичных проверок (например, стиля кода) появятся в течение нескольких минут. -Результаты проверки сборки будут доступны в течение получаса. -Основной набор тестов сообщит о результатах примерно в течение часа. +Тестирование начнётся, как только сотрудники ClickHouse пометят ваш PR меткой «can be tested». +Результаты первых проверок (например, code style) появятся в течение нескольких минут. +Результаты проверки сборки поступят примерно через полчаса. +Основной набор тестов завершится примерно через час. Система подготовит отдельные бинарные сборки ClickHouse для вашего pull request. Чтобы получить эти сборки, нажмите ссылку «Details» рядом с пунктом «Builds» в списке проверок. -Там вы найдете прямые ссылки на собранные .deb-пакеты ClickHouse, которые вы можете развернуть даже на своих production-серверах (если не боитесь). - - +Там вы найдёте прямые ссылки на собранные .deb‑пакеты ClickHouse, которые вы можете развернуть даже на своих production‑серверах (если не боитесь). -## Написание документации {#write-documentation} +## Создавайте документацию {#write-documentation} -Каждый pull request, добавляющий новую функцию, должен сопровождаться соответствующей документацией. +Каждый pull request, который добавляет новую функциональность, должен сопровождаться соответствующей документацией. Если вы хотите предварительно просмотреть изменения в документации, инструкции по локальной сборке страницы документации доступны в файле README.md [здесь](https://github.com/ClickHouse/clickhouse-docs). При добавлении новой функции в ClickHouse вы можете использовать приведённый ниже шаблон в качестве ориентира: - - ````markdown -# newFunctionName +# newFunctionName {#newfunctionname} -Краткое описание функции. Здесь следует кратко описать её назначение и типичный сценарий использования. +Краткое описание функции. Должно кратко описывать её назначение и типичный сценарий использования. **Синтаксис** @@ -167,40 +157,40 @@ newFunctionName(arg1, arg2[, arg3]) - `arg1` — Описание аргумента. [DataType](../data-types/float.md) - `arg2` — Описание аргумента. [DataType](../data-types/float.md) -- `arg3` — Описание необязательного аргумента. [DataType](../data-types/float.md) +- `arg3` — Описание необязательного аргумента (необязательный). [DataType](../data-types/float.md) -**Особенности реализации** +**Детали реализации** -Описание деталей реализации, если это применимо. +Описание деталей реализации, если применимо. **Возвращаемое значение** -- Возвращает {укажите, что именно возвращает функция}. [DataType](../data-types/float.md) +- Возвращает {укажите, что возвращает функция}. [DataType](../data-types/float.md) **Пример** Запрос: \```sql -SELECT 'укажите здесь пример запроса'; +SELECT 'write your example query here'; \``` -Ответ: +Результат: \```response ┌───────────────────────────────────┐ -│ результат запроса │ +│ the result of the query │ └───────────────────────────────────┘ \``` ```` -## Использование тестовых данных +## Использование тестовых данных {#using-test-data} -Разработка ClickHouse часто требует загрузки реалистичных наборов данных. +При разработке под ClickHouse часто требуется загрузка реалистичных наборов данных. Это особенно важно для тестирования производительности. -Мы подготовили специальный набор обезличенных данных по веб‑аналитике. -Для него требуется дополнительно около 3 ГБ свободного дискового пространства. +У нас есть специально подготовленный набор анонимизированных данных веб‑аналитики. +Для него требуется около 3 ГБ свободного дискового пространства. ```sh sudo apt install wget xz-utils @@ -214,24 +204,20 @@ SELECT 'укажите здесь пример запроса'; clickhouse-client ``` -В клиенте clickhouse-client: +В clickhouse-client: + ```sql CREATE DATABASE IF NOT EXISTS test; -``` - CREATE TABLE test.hits ( WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree PARTITION BY toYYYYMM(EventDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID), EventTime); - - -CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); - -```` +CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); +``` Импортируйте данные: ```bash clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.hits FORMAT TSV" < hits_v1.tsv clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.visits FORMAT TSV" < visits_v1.tsv -```` +``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/index.md index 8551d98399f..17868c47a57 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/index.md @@ -1,40 +1,40 @@ --- -description: 'Главная страница раздела «Разработка и участие»' +description: 'Индексная страница раздела «Разработка и вклад в проект»' slug: /development/ -title: 'Разработка и участие' +title: 'Разработка и вклад в проект' doc_type: 'landing-page' --- В этом разделе документации представлены следующие страницы: -{/* Приведённое ниже оглавление автоматически генерируется из полей YAML - front matter: title, description, slug с помощью скрипта, расположенного здесь: +{/* Приведённое ниже оглавление автоматически генерируется на основе полей + front matter в YAML: title, description, slug с помощью скрипта отсюда: https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - + Если вы заметили ошибку или хотите улучшить описания, отредактируйте соответствующие файлы напрямую. */ } {/*AUTOGENERATED_START*/ } -| Страница | Описание | -| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | -| [Developer Prerequisites](/development/developer-instruction) | Требования и инструкции по настройке среды разработки ClickHouse | -| [How to Build ClickHouse on Linux](/development/build) | Пошаговое руководство по сборке ClickHouse из исходного кода на Linux-системах | -| [Build on macOS for macOS](/development/build-osx) | Руководство по сборке ClickHouse из исходного кода на системах macOS | -| [Build on Linux for macOS](/development/build-cross-osx) | Руководство по кросс-компиляции ClickHouse на Linux для систем macOS | -| [How to Build ClickHouse on Linux for AARCH64](/development/build-cross-arm) | Руководство по сборке ClickHouse из исходного кода для архитектуры AARCH64 | -| [How to Build ClickHouse on Linux for RISC-V 64](/development/build-cross-riscv) | Руководство по сборке ClickHouse из исходного кода для архитектуры RISC-V 64 | -| [Build on Linux for s390x (zLinux)](/development/build-cross-s390x) | Руководство по сборке ClickHouse из исходного кода для архитектуры s390x | -| [Build on Linux for E2K](/development/build-e2k) | Руководство по сборке ClickHouse из исходного кода для архитектуры E2K | -| [Build on Linux for LoongArch64](/development/build-cross-loongarch) | Руководство по сборке ClickHouse из исходного кода для архитектуры LoongArch64 | -| [Testing ClickHouse](/development/tests) | Руководство по тестированию ClickHouse и запуску набора тестов | -| [Architecture Overview](/development/architecture) | Подробный обзор архитектуры ClickHouse и его колонко-ориентированной организации данных | -| [Continuous Integration (CI)](/development/continuous-integration) | Обзор системы непрерывной интеграции (CI) ClickHouse | -| [Third-Party Libraries](/development/contrib) | Страница, описывающая использование сторонних библиотек в ClickHouse и порядок их добавления и сопровождения | -| [C++ Style Guide](/development/style) | Рекомендации по стилю кодирования для разработки ClickHouse на C++ | -| [Build Clickhouse with DEFLATE_QPL](/development/building_and_benchmarking_deflate_qpl) | Как собрать ClickHouse и запустить бенчмарк с кодеком DEFLATE_QPL | -| [Integrating Rust Libraries](/development/integrating_rust_libraries) | Руководство по интеграции библиотек Rust в ClickHouse | +| Страница | Описание | +| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| [Developer Prerequisites](/development/developer-instruction) | Предварительные требования и инструкции по настройке среды для разработки ClickHouse | +| [How to Build ClickHouse on Linux](/development/build) | Пошаговое руководство по сборке ClickHouse из исходного кода в системах Linux | +| [Build on macOS for macOS](/development/build-osx) | Руководство по сборке ClickHouse из исходного кода в системах macOS | +| [Build on Linux for macOS](/development/build-cross-osx) | Руководство по кросс-компиляции ClickHouse в Linux для систем macOS | +| [How to Build ClickHouse on Linux for AARCH64](/development/build-cross-arm) | Руководство по сборке ClickHouse из исходного кода для архитектуры AARCH64 | +| [How to Build ClickHouse on Linux for RISC-V 64](/development/build-cross-riscv) | Руководство по сборке ClickHouse из исходного кода для архитектуры RISC-V 64 | +| [Build on Linux for s390x (zLinux)](/development/build-cross-s390x) | Руководство по сборке ClickHouse из исходного кода для архитектуры s390x | +| [Build on Linux for LoongArch64](/development/build-cross-loongarch) | Руководство по сборке ClickHouse из исходного кода для архитектуры LoongArch64 | +| [Build on Linux for E2K](/development/build-e2k) | Руководство по сборке ClickHouse из исходного кода для архитектуры E2K | +| [Testing ClickHouse](/development/tests) | Руководство по тестированию ClickHouse и запуску набора тестов | +| [Architecture Overview](/development/architecture) | Подробный обзор архитектуры ClickHouse и его столбцово-ориентированной организации | +| [Continuous Integration (CI)](/development/continuous-integration) | Обзор системы непрерывной интеграции ClickHouse | +| [Third-Party Libraries](/development/contrib) | Страница, описывающая использование сторонних библиотек в ClickHouse, а также порядок их добавления и сопровождения | +| [C++ Style Guide](/development/style) | Рекомендации по стилю кодирования для разработки ClickHouse на C++ | +| [Build Clickhouse with DEFLATE_QPL](/development/building_and_benchmarking_deflate_qpl) | Как собрать ClickHouse и запустить бенчмарк с кодеком DEFLATE_QPL | +| [Integrating Rust Libraries](/development/integrating_rust_libraries) | Руководство по интеграции библиотек Rust в ClickHouse | {/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md index 855b7814529..d27596b00f5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md @@ -6,7 +6,7 @@ title: 'Интеграция библиотек на Rust' doc_type: 'guide' --- -# Библиотеки Rust +# Библиотеки Rust {#rust-libraries} Интеграция библиотеки Rust будет описана на примере интеграции хеш-функции BLAKE3. @@ -80,7 +80,6 @@ pub unsafe extern "C" fn blake3_apply_shim( Также следует использовать атрибут #[no_mangle] и `extern "C"` для каждого C-совместимого элемента. В противном случае библиотека может быть собрана некорректно, а cbindgen не сможет автоматически сгенерировать заголовочный файл. - После выполнения всех этих шагов вы можете протестировать свою библиотеку в небольшом проекте, чтобы выявить все проблемы с совместимостью или генерацией заголовков. Если при генерации заголовков возникают какие-либо проблемы, вы можете попробовать настроить её с помощью файла cbindgen.toml (шаблон можно найти здесь: [https://github.com/eqrion/cbindgen/blob/master/template.toml](https://github.com/eqrion/cbindgen/blob/master/template.toml)). Стоит отметить проблему, возникшую при интеграции BLAKE3: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/style.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/style.md index 4fdfe3eb18a..6c9d00369ae 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/style.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/style.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Руководство по стилю кода C++ +# Руководство по стилю кода C++ {#c-style-guide} @@ -22,7 +22,7 @@ doc_type: 'guide' -## Форматирование +## Форматирование {#formatting} **1.** Большая часть форматирования выполняется автоматически с помощью `clang-format`. @@ -214,7 +214,7 @@ for (Names::const_iterator it = column_names.begin(); it != column_names.end(); ``` -## Комментарии +## Комментарии {#comments} **1.** Обязательно добавляйте комментарии ко всем нетривиальным фрагментам кода. @@ -312,7 +312,7 @@ void executeQuery( ``` -## Имена +## Имена {#names} **1.** Используйте строчные буквы и символ нижнего подчеркивания в именах переменных и членов классов. @@ -431,7 +431,7 @@ enum class CompressionMethod **17.** Имена файлов с исходным кодом C++ должны иметь расширение `.cpp`. Заголовочные файлы должны иметь расширение `.h`. -## Как писать код +## Как писать код {#how-to-write-code} **1.** Управление памятью. @@ -700,7 +700,7 @@ auto s = std::string{"Hello"}; **26.** Для виртуальных функций пишите `virtual` в базовом классе, а в производных классах пишите `override` вместо `virtual`. -## Неиспользуемые возможности C++ +## Неиспользуемые возможности C++ {#unused-features-of-c} **1.** Виртуальное наследование не используется. @@ -830,7 +830,7 @@ auto func(const E & e) // автовыведение возвращаемог -## Дополнительные рекомендации +## Дополнительные рекомендации {#additional-recommendations} **1.** Явное указание `std::` для типов из `stddef.h` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/development/tests.md b/i18n/ru/docusaurus-plugin-content-docs/current/development/tests.md index 994e55b38f0..a8c6a0e7207 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/development/tests.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/development/tests.md @@ -9,11 +9,11 @@ doc_type: 'guide' -# Проверка ClickHouse +# Проверка ClickHouse {#testing-clickhouse} -## Функциональные тесты +## Функциональные тесты {#functional-tests} Функциональные тесты — наиболее простые и удобные в использовании. Большинство возможностей ClickHouse можно протестировать с их помощью, и их обязательно использовать для каждого изменения в коде ClickHouse, которое может быть протестировано таким образом. @@ -34,7 +34,7 @@ doc_type: 'guide' Распространённая ошибка при тестировании типов данных `DateTime` и `DateTime64` — предполагать, что сервер использует конкретный часовой пояс (например, «UTC»). Это не так: часовые пояса в CI‑запусках тестов преднамеренно выбираются случайным образом. Самое простое решение — явно указывать часовой пояс для тестовых значений, например: `toDateTime64(val, 3, 'Europe/Amsterdam')`. ::: -### Запуск теста локально +### Запуск теста локально {#running-a-test-locally} Запустите сервер ClickHouse локально, прослушивающий порт по умолчанию (9000). Чтобы запустить, например, тест `01428_hash_set_nan_key`, перейдите в каталог репозитория и выполните следующую команду: @@ -49,7 +49,7 @@ PATH=:$PATH tests/clickhouse-test 01428_hash_set_nan_ Вы можете запустить все тесты или только их подмножество, указав фильтр по именам тестов: `./clickhouse-test substring`. Также есть параметры для запуска тестов параллельно или в случайном порядке. -### Добавление нового теста +### Добавление нового теста {#adding-a-new-test} Чтобы добавить новый тест, сначала создайте файл `.sql` или `.sh` в директории `queries/0_stateless`. Затем сгенерируйте соответствующий файл `.reference`, используя `clickhouse-client < 12345_test.sql > 12345_test.reference` или `./12345_test.sh > ./12345_test.reference`. @@ -76,7 +76,7 @@ sudo ./install.sh * удостоверяться, что другие тесты не проверяют то же самое (т.е. сначала выполните grep). ::: -### Ограничение запусков тестов +### Ограничение запусков тестов {#restricting-test-runs} Тест может иметь ноль или более *тегов*, задающих ограничения, в каких контекстах тест запускается в CI. @@ -95,9 +95,9 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Tags: no-fasttest, no-replicated-database -# - no-fasttest: <укажите причину для данного тега> -# - no-replicated-database: <укажите причину для данного тега> +# Tags: no-fasttest, no-replicated-database {#tags-no-fasttest-no-replicated-database} +# - no-fasttest: <укажите причину для данного тега> {#no-fasttest-provide_a_reason_for_the_tag_here} +# - no-replicated-database: <укажите причину для данного тега> {#no-replicated-database-provide_a_reason_here} ``` Список доступных тегов: @@ -134,7 +134,7 @@ SELECT 1 В дополнение к приведённым выше настройкам вы можете использовать флаги `USE_*` из `system.build_options` для указания использования отдельных возможностей ClickHouse. Например, если ваш тест использует таблицу MySQL, вам следует добавить тег `use-mysql`. -### Указание ограничений для случайных настроек +### Указание ограничений для случайных настроек {#specifying-limits-for-random-settings} Тест может задавать минимальные и максимальные допустимые значения для настроек, которые могут случайным образом изменяться во время выполнения теста. @@ -143,8 +143,8 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Теги: no-fasttest -# Ограничения случайных настроек: max_block_size=(1000, 10000); index_granularity=(100, None) +# Теги: no-fasttest {#tags-no-fasttest} +# Ограничения случайных настроек: max_block_size=(1000, 10000); index_granularity=(100, None) {#random-settings-limits-max_block_size1000-10000-index_granularity100-none} ``` Для `.sql`-тестов теги размещаются как SQL-комментарий в строке рядом с тестом или в первой строке: @@ -157,7 +157,7 @@ SELECT 1 Если вам нужно указать только одно ограничение, вы можете использовать `None` для второго. -### Выбор имени теста +### Выбор имени теста {#choosing-the-test-name} Имя теста начинается с пятизначного префикса, за которым следует описательное имя, например `00422_hash_function_constexpr.sql`. Чтобы выбрать префикс, найдите наибольший префикс, уже присутствующий в каталоге, и увеличьте его на единицу. @@ -168,7 +168,7 @@ ls tests/queries/0_stateless/[0-9]*.reference | tail -n 1 Параллельно могут быть добавлены другие тесты с тем же числовым префиксом, но это допустимо и не приводит к каким-либо проблемам; впоследствии ничего менять не нужно. -### Проверка ошибки, которая должна произойти +### Проверка ошибки, которая должна произойти {#checking-for-an-error-that-must-occur} Иногда требуется протестировать, что для некорректного запроса возникает ошибка сервера. В SQL-тестах для этого поддерживаются специальные аннотации следующего вида: @@ -184,12 +184,12 @@ SELECT x; -- { serverError 49 } Проверяйте только код ошибки. Если существующий код ошибки недостаточно точен для ваших нужд, рассмотрите возможность добавления нового. -### Тестирование распределённого запроса +### Тестирование распределённого запроса {#testing-a-distributed-query} Если вы хотите использовать распределённые запросы в функциональных тестах, вы можете воспользоваться табличной функцией `remote` с адресами `127.0.0.{1..2}`, чтобы сервер выполнял запрос к самому себе; либо вы можете использовать предопределённые тестовые кластеры в конфигурационном файле сервера, такие как `test_shard_localhost`. Не забудьте добавить слова `shard` или `distributed` в имя теста, чтобы он запускался в CI в корректных конфигурациях, где сервер настроен на поддержку распределённых запросов. -### Работа с временными файлами +### Работа с временными файлами {#working-with-temporary-files} Иногда в shell-тесте может потребоваться на лету создать файл для работы. Имейте в виду, что некоторые проверки в CI запускают тесты параллельно, поэтому если вы создаёте или удаляете временный файл в своём скрипте без уникального имени, это может привести к сбоям некоторых проверок CI, таких как Flaky. @@ -217,7 +217,7 @@ SELECT x; -- { serverError 49 } -## Модульные тесты +## Модульные тесты {#unit-tests} Модульные тесты полезны, когда вы хотите протестировать не ClickHouse целиком, а отдельную изолированную библиотеку или класс. Сборку тестов можно включить или отключить с помощью опции CMake `ENABLE_TESTS`. @@ -272,7 +272,7 @@ $ ./src/unit_tests_dbms --gtest_filter=LocalAddress* -## Ручное тестирование +## Ручное тестирование {#manual-testing} Когда вы разрабатываете новую функциональность, имеет смысл также протестировать её вручную. Вы можете сделать это, выполнив следующие шаги: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/dictionary/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/dictionary/index.md index b9ca5aa5de7..ebaf4f79298 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/dictionary/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/dictionary/index.md @@ -2,7 +2,7 @@ slug: /dictionary title: 'Словарь' keywords: ['словарь', 'словари'] -description: 'Словарь предоставляет представление данных в формате ключ-значение для быстрого поиска.' +description: 'Словарь представляет данные в формате ключ-значение для быстрого поиска.' doc_type: 'reference' --- @@ -11,36 +11,35 @@ import dictionaryLeftAnyJoin from '@site/static/images/dictionary/dictionary-lef import Image from '@theme/IdealImage'; -# Словарь +# Словарь {#dictionary} -Словарь в ClickHouse предоставляет представление данных в оперативной памяти в формате [key-value](https://en.wikipedia.org/wiki/Key%E2%80%93value_database) из различных [внутренних и внешних источников](/sql-reference/dictionaries#dictionary-sources), оптимизированное для запросов с крайне низкой задержкой поиска. +Словарь в ClickHouse предоставляет хранящееся в памяти представление данных в формате [key-value](https://en.wikipedia.org/wiki/Key%E2%80%93value_database) из различных [внутренних и внешних источников](/sql-reference/dictionaries#dictionary-sources), оптимизированное для операций поиска с крайне низкой задержкой. Словари полезны для: -- Повышения производительности запросов, особенно при использовании в операциях `JOIN` -- Обогащения данных на лету в процессе ингестии, не замедляя её -Сценарии использования словарей в ClickHouse +- Повышения производительности запросов, особенно при использовании с операциями `JOIN` +- Обогащения поступающих данных «на лету» без замедления процесса ингестии +Сценарии использования словаря в ClickHouse +## Ускорение соединений с использованием словаря {#speeding-up-joins-using-a-dictionary} -## Ускорение соединений с помощью Dictionary +Словари можно использовать для ускорения определённого типа операции `JOIN`: типа [`LEFT ANY`](/sql-reference/statements/select/join#supported-types-of-join), когда ключ соединения совпадает с ключевым атрибутом подлежащего хранилища ключ-значение. -Dictionary можно использовать для ускорения определённого типа `JOIN`: [`LEFT ANY`](/sql-reference/statements/select/join#supported-types-of-join), когда ключ соединения должен совпадать с ключевым атрибутом базового key-value-хранилища. +Использование словаря с LEFT ANY JOIN -Использование Dictionary с LEFT ANY JOIN +В таком случае ClickHouse может использовать словарь для выполнения [Direct Join](https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4#direct-join). Это самый быстрый алгоритм соединения в ClickHouse, применимый, когда базовый [движок таблицы](/engines/table-engines) для таблицы справа поддерживает запросы к хранилищу ключ-значение с низкой задержкой. В ClickHouse есть три движка таблиц, которые это поддерживают: [Join](/engines/table-engines/special/join) (по сути, предварительно вычисленная хеш-таблица), [EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) и [Dictionary](/engines/table-engines/special/dictionary). Мы опишем подход, основанный на словаре, но механика одинакова для всех трёх движков. -В таком случае ClickHouse может использовать Dictionary для выполнения [Direct Join](https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4#direct-join). Это самый быстрый алгоритм соединения в ClickHouse, и он применим, когда базовый [движок таблицы](/engines/table-engines) для таблицы правой части поддерживает key-value-запросы с низкой задержкой. В ClickHouse есть три движка таблиц, обеспечивающие это: [Join](/engines/table-engines/special/join) (по сути, предварительно вычисленная хеш-таблица), [EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) и [Dictionary](/engines/table-engines/special/dictionary). Мы опишем подход на основе Dictionary, но механика одинакова для всех трёх движков. +Алгоритм Direct Join требует, чтобы правая таблица опиралась на словарь, благодаря чему данные для соединения из этой таблицы уже находятся в памяти в виде структуры данных хранилища ключ-значение с низкой задержкой. -Алгоритм Direct Join требует, чтобы правая таблица была реализована на основе Dictionary, так, чтобы данные, которые нужно присоединять из этой таблицы, уже находились в памяти в виде key-value-структуры данных с низкой задержкой. +### Пример {#example} -### Пример - -Используя набор данных Stack Overflow, ответим на вопрос: +Используя [датасет Stack Overflow](/getting-started/example-datasets/stackoverflow), ответим на вопрос: *Какой пост, касающийся SQL, является самым спорным на Hacker News?* -Мы будем считать пост спорным, если у него схожее количество голосов «за» и «против». Мы вычислим эту абсолютную разницу, где значение, ближе к нулю, означает большую спорность. Будем считать, что у поста должно быть как минимум 10 голосов «за» и 10 «против» — посты, за которые почти не голосуют, не слишком спорные. +Мы будем считать пост спорным, если у него схожее количество голосов «за» и «против». Мы вычислим абсолютную разницу между ними: чем ближе значение к 0, тем сильнее спорность. Также будем считать, что у поста должно быть как минимум 10 голосов «за» и 10 голосов «против» — посты, за которые почти не голосуют, вряд ли можно считать спорными. -При нормализованных данных этот запрос в текущем виде требует `JOIN` с использованием таблиц `posts` и `votes`: +При нормализованных данных этот запрос требует `JOIN` с использованием таблиц `posts` и `votes`: ```sql WITH PostIds AS @@ -79,38 +78,38 @@ UpVotes: 13 DownVotes: 13 Controversial_ratio: 0 -1 строка в наборе. Прошло: 1.283 сек. Обработано 418.44 млн строк, 7.23 ГБ (326.07 млн строк/с., 5.63 ГБ/с.) -Пиковое использование памяти: 3.18 ГиБ. +Обработана 1 строка. Затрачено: 1,283 сек. Обработано 418,44 млн строк, 7,23 ГБ (326,07 млн строк/с., 5,63 ГБ/с.) +Пиковое использование памяти: 3,18 ГиБ. ``` -> **Используйте меньшие наборы данных в правой части `JOIN`**: Этот запрос может показаться избыточно многословным, так как фильтрация по `PostId` выполняется и во внешнем, и во вложенном запросах. Это оптимизация производительности, которая обеспечивает быстрое время отклика запроса. Для оптимальной производительности всегда следите за тем, чтобы правая сторона `JOIN` была меньшим набором и по возможности как можно меньшего размера. Советы по оптимизации производительности `JOIN` и обзору доступных алгоритмов приведены в [этой серии статей в блоге](https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1). +> **Используйте меньшие наборы данных в правой части `JOIN`**: Этот запрос может показаться более многословным, чем требуется, поскольку фильтрация по `PostId` выполняется как во внешнем, так и во вложенном запросе. Это оптимизация производительности, которая обеспечивает быстрое время ответа. Для оптимальной производительности всегда следите за тем, чтобы правая сторона `JOIN` была меньшим набором данных и оставалась как можно меньше. Советы по оптимизации производительности JOIN и обзору доступных алгоритмов приведены в [этой серии статей в блоге](https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1). -Хотя этот запрос работает быстро, он требует от нас аккуратного использования `JOIN`, чтобы достичь хорошей производительности. В идеале мы бы просто отфильтровали посты по тем, которые содержат «SQL», прежде чем анализировать значения `UpVote` и `DownVote` для этого подмножества блогов, чтобы вычислить нашу метрику. +Хотя этот запрос и работает быстро, он требует от нас аккуратного использования `JOIN`, чтобы добиться хорошей производительности. В идеале мы бы просто отфильтровали записи до тех, которые содержат «SQL», прежде чем смотреть на значения `UpVote` и `DownVote` для подмножества блогов, чтобы вычислить нашу метрику. -#### Применение словаря -Чтобы продемонстрировать эти концепции, мы используем словарь для наших данных о голосах. Поскольку словари обычно хранятся в памяти ([ssd_cache](/sql-reference/dictionaries#ssd_cache) — исключение), пользователям следует учитывать размер данных. Проверим размер нашей таблицы `votes`: +#### Применение словаря {#applying-a-dictionary} +Чтобы продемонстрировать эти концепции, мы используем словарь для наших данных о голосовании. Поскольку словари обычно хранятся в памяти ([ssd_cache](/sql-reference/dictionaries#ssd_cache) — исключение), пользователям следует учитывать объём данных. Проверим размер нашей таблицы `votes`: ```sql SELECT table, - formatReadableSize(sum(data_compressed_bytes)) AS размер_сжатых_данных, - formatReadableSize(sum(data_uncompressed_bytes)) AS размер_несжатых_данных, - round(sum(data_uncompressed_bytes) / sum(data_compressed_bytes), 2) AS коэффициент + formatReadableSize(sum(data_compressed_bytes)) AS compressed_size, + formatReadableSize(sum(data_uncompressed_bytes)) AS uncompressed_size, + round(sum(data_uncompressed_bytes) / sum(data_compressed_bytes), 2) AS ratio FROM system.columns WHERE table IN ('votes') GROUP BY table -┌─table───────────┬─размер_сжатых_данных─┬─размер_несжатых_данных─┬─коэффициент─┐ -│ votes │ 1.25 GiB │ 3.79 GiB │ 3.04 │ -└─────────────────┴──────────────────────┴────────────────────────┴─────────────┘ +┌─table───────────┬─compressed_size─┬─uncompressed_size─┬─ratio─┐ +│ votes │ 1.25 GiB │ 3.79 GiB │ 3.04 │ +└─────────────────┴─────────────────┴───────────────────┴───────┘ ``` -Данные будут храниться в нашем словаре без сжатия, поэтому нам требуется как минимум 4 ГБ памяти, если бы мы сохраняли все столбцы (мы этого делать не будем) в словаре. Словарь будет реплицирован по нашему кластеру, поэтому этот объём памяти нужно зарезервировать *на каждый узел*. +Данные будут храниться в нашем словаре без сжатия, поэтому нам потребуется как минимум 4 ГБ памяти, если бы мы собирались хранить все столбцы (мы не будем) в словаре. Словарь будет реплицирован по нашему кластеру, поэтому этот объём памяти должен быть зарезервирован *на каждый узел*. -> В приведённом ниже примере данные для нашего словаря берутся из таблицы ClickHouse. Хотя это и является наиболее распространённым источником словарей, поддерживается [ряд источников](/sql-reference/dictionaries#dictionary-sources), включая файлы, HTTP и базы данных, в том числе [Postgres](/sql-reference/dictionaries#postgresql). Как мы покажем, словари могут автоматически обновляться, что является идеальным способом обеспечить доступность небольших наборов данных, подверженных частым изменениям, для прямых JOIN-ов. +> В примере ниже данные для нашего словаря поступают из таблицы ClickHouse. Хотя это и является наиболее распространённым источником словарей, поддерживается [ряд источников](/sql-reference/dictionaries#dictionary-sources), включая файлы, HTTP и базы данных, в том числе [Postgres](/sql-reference/dictionaries#postgresql). Как мы покажем, словари могут автоматически обновляться, что делает их идеальным способом обеспечить доступность небольших наборов данных, подверженных частым изменениям, для прямых JOIN. -Для нашего словаря необходим первичный ключ, по которому будут выполняться поиски. Концептуально он идентичен первичному ключу в транзакционной базе данных и должен быть уникальным. Наш запрос выше требует поиска по ключу соединения — `PostId`. Словарь, в свою очередь, должен быть заполнен суммарным количеством голосов «за» и «против» для каждого `PostId` из нашей таблицы `votes`. Ниже приведён запрос для получения данных для этого словаря: +Для нашего словаря требуется первичный ключ, по которому будут выполняться обращения. Концептуально это идентично первичному ключу транзакционной базы данных и должно быть уникальным. Наш запрос выше требует обращения по ключу соединения — `PostId`. Словарь, в свою очередь, должен быть заполнен суммарными значениями голосов «за» и «против» по `PostId` из нашей таблицы `votes`. Ниже приведён запрос для получения данных этого словаря: ```sql SELECT PostId, @@ -120,7 +119,7 @@ FROM votes GROUP BY PostId ``` -Чтобы создать наш словарь, потребуется следующий DDL — обратите внимание на использование нашего запроса, приведённого выше: +Для создания нашего словаря используем следующий DDL — обратите внимание на использование приведённого выше запроса: ```sql CREATE DICTIONARY votes_dict @@ -137,9 +136,9 @@ LAYOUT(HASHED()) 0 строк в наборе. Затрачено: 36.063 сек. ``` -> В самоуправляемой установке OSS приведённую выше команду необходимо выполнить на всех узлах. В ClickHouse Cloud словарь будет автоматически реплицирован на все узлы. Эту операцию выполняли на узле ClickHouse Cloud с 64 ГБ ОЗУ, загрузка заняла 36 с. +> В самоуправляемой OSS-установке указанную выше команду нужно выполнить на всех узлах. В ClickHouse Cloud словарь будет автоматически реплицирован на все узлы. Эта команда была выполнена на узле ClickHouse Cloud с 64 ГБ ОЗУ, загрузка заняла 36 секунд. -Чтобы проверить объём памяти, потребляемой нашим словарём: +Чтобы подтвердить объём памяти, потребляемый нашим словарём: ```sql SELECT formatReadableSize(bytes_allocated) AS size @@ -151,7 +150,8 @@ WHERE name = 'votes_dict' └──────────┘ ``` -Теперь получение голосов «за» и «против» для конкретного `PostId` сводится к использованию простой функции `dictGet`. Ниже показано, как получить значения для поста `11227902`: +Получить количество голосов «за» и «против» для конкретного `PostId` теперь можно с помощью простой функции `dictGet`. Ниже мы получаем значения для поста `11227902`: + ```sql SELECT dictGet('votes_dict', ('UpVotes', 'DownVotes'), '11227902') AS votes @@ -181,12 +181,12 @@ LIMIT 3 Пиковое использование памяти: 552.26 МиБ. ``` -Мало того, что этот запрос значительно проще, он ещё и более чем вдвое быстрее! Его можно дополнительно оптимизировать, загружая в словарь только посты с более чем 10 голосами «за» и «против» и сохраняя лишь заранее вычисленное значение спорности. +Этот запрос не только гораздо проще, но и более чем в два раза быстрее! Его можно дополнительно оптимизировать, загружая в словарь только посты с более чем 10 голосами «за» и «против» и сохраняя только предварительно вычисленное значение степени спорности. -## Обогащение данных при выполнении запроса +## Обогащение при выполнении запроса {#query-time-enrichment} -Словари можно использовать для поиска значений в момент выполнения запроса. Эти значения могут возвращаться в результатах запроса или использоваться в агрегациях. Предположим, мы создаём словарь для сопоставления идентификаторов пользователей с их местоположением: +Словари можно использовать для поиска значений при выполнении запроса. Эти значения могут возвращаться в результатах или использоваться в агрегациях. Предположим, мы создаём словарь для отображения идентификаторов пользователей на их местоположения: ```sql CREATE DICTIONARY users_dict @@ -200,7 +200,7 @@ LIFETIME(MIN 600 MAX 900) LAYOUT(HASHED()) ``` -Мы можем использовать этот словарь для обогащения результатов: +Мы можем использовать этот словарь для обогащения результатов по постам: ```sql SELECT @@ -213,18 +213,18 @@ LIMIT 5 FORMAT PrettyCompactMonoBlock ┌───────Id─┬─Title─────────────────────────────────────────────────────────┬─Location──────────────┐ -│ 52296928 │ Сравнение двух строк в ClickHouse │ Spain │ -│ 52345137 │ Как использовать файл для миграции данных из MySQL в ClickHouse? │ 中国江苏省Nanjing Shi │ -│ 61452077 │ Как изменить PARTITION в ClickHouse │ Guangzhou, 广东省中国 │ -│ 55608325 │ Выбор последней записи в ClickHouse без max() для всей таблицы │ Moscow, Russia │ -│ 55758594 │ Создание временной таблицы в ClickHouse │ Perm', Russia │ +│ 52296928 │ Comparison between two Strings in ClickHouse │ Spain │ +│ 52345137 │ How to use a file to migrate data from mysql to a clickhouse? │ 中国江苏省Nanjing Shi │ +│ 61452077 │ How to change PARTITION in clickhouse │ Guangzhou, 广东省中国 │ +│ 55608325 │ Clickhouse select last record without max() on all table │ Moscow, Russia │ +│ 55758594 │ ClickHouse create temporary table │ Perm', Russia │ └──────────┴───────────────────────────────────────────────────────────────┴───────────────────────┘ Получено 5 строк. Затрачено: 0,033 сек. Обработано 4,25 млн строк, 82,84 МБ (130,62 млн строк/с., 2,55 ГБ/с.) Пиковое использование памяти: 249,32 МиБ. ``` -Аналогично приведённому выше примеру join, мы можем использовать тот же словарь, чтобы эффективно определить, откуда происходит большинство постов: +Аналогично нашему примеру выше с JOIN, мы можем использовать тот же словарь, чтобы эффективно определить, откуда происходит большинство постов: ```sql SELECT @@ -249,13 +249,13 @@ LIMIT 5 ``` -## Обогащение во время индексации +## Обогащение на этапе вставки (index time) {#index-time-enrichment} -В приведённом выше примере мы использовали словарь на этапе выполнения запроса, чтобы убрать `JOIN`. Словари также можно использовать для обогащения строк на этапе вставки. Это обычно уместно, если значение для обогащения не меняется и существует во внешнем источнике, который можно использовать для заполнения словаря. В таком случае обогащение строки на этапе вставки позволяет избежать обращения к словарю во время выполнения запроса. +В приведённом выше примере мы использовали словарь на этапе выполнения запроса, чтобы убрать операцию JOIN. Словари также можно использовать для обогащения строк на этапе вставки. Это обычно целесообразно, если значение для обогащения не меняется и существует во внешнем источнике, который можно использовать для заполнения словаря. В этом случае обогащение строки на этапе вставки позволяет избежать поиска в словаре во время выполнения запроса. -Предположим, что `Location` пользователя в Stack Overflow никогда не меняется (в реальности это не так), а именно столбец `Location` таблицы `users`. Допустим, мы хотим выполнить аналитический запрос к таблице `posts` по местоположению. В этой таблице есть столбец `UserId`. +Предположим, что `Location` пользователя в Stack Overflow никогда не меняется (в реальности это не так), а именно столбец `Location` таблицы `users`. Допустим, мы хотим выполнить аналитический запрос к таблице `posts` по местоположению. В ней содержится столбец `UserId`. -Словарь предоставляет соответствие между идентификатором пользователя и его местоположением, используя таблицу `users`: +Словарь задаёт соответствие между идентификатором пользователя и его местоположением, опираясь на таблицу `users`: ```sql CREATE DICTIONARY users_dict @@ -269,9 +269,9 @@ LIFETIME(MIN 600 MAX 900) LAYOUT(HASHED()) ``` -> Мы исключаем пользователей с `Id < 0`, что позволяет использовать тип словаря `Hashed`. Пользователи с `Id < 0` — это системные пользователи. +> Мы исключаем пользователей с `Id < 0`, что позволяет нам использовать тип словаря `Hashed`. Пользователи с `Id < 0` являются системными пользователями. -Чтобы задействовать этот словарь на этапе вставки данных в таблицу posts, необходимо изменить схему: +Чтобы использовать этот словарь при вставке данных в таблицу posts, нам нужно изменить схему: ```sql CREATE TABLE posts_with_location @@ -285,11 +285,11 @@ ENGINE = MergeTree ORDER BY (PostTypeId, toDate(CreationDate), CommentCount) ``` -В приведённом выше примере `Location` объявлен как столбец `MATERIALIZED`. Это означает, что значение может быть передано в запросе `INSERT` и всегда будет вычислено. +В приведённом выше примере `Location` объявлен как столбец типа `MATERIALIZED`. Это означает, что значение может быть указано в запросе `INSERT` и при этом всегда будет вычислено. -> ClickHouse также поддерживает [столбцы с типом `DEFAULT`](/sql-reference/statements/create/table#default_values) (когда значение может быть явно указано при вставке или вычислено, если не задано). +> ClickHouse также поддерживает [`DEFAULT` столбцы](/sql-reference/statements/create/table#default_values) (когда значение может быть вставлено или вычислено, если оно не указано). -Чтобы заполнить таблицу, мы можем использовать обычный `INSERT INTO SELECT` из S3: +Чтобы заполнить таблицу, мы можем использовать привычный `INSERT INTO SELECT` из S3: ```sql INSERT INTO posts_with_location SELECT Id, PostTypeId::UInt8, AcceptedAnswerId, CreationDate, Score, ViewCount, Body, OwnerUserId, OwnerDisplayName, LastEditorUserId, LastEditorDisplayName, LastEditDate, LastActivityDate, Title, Tags, AnswerCount, CommentCount, FavoriteCount, ContentLicense, ParentId, CommunityOwnedDate, ClosedDate FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/*.parquet') @@ -297,7 +297,7 @@ INSERT INTO posts_with_location SELECT Id, PostTypeId::UInt8, AcceptedAnswerId, 0 строк в наборе. Затрачено: 36.830 сек. Обработано 238.98 млн строк, 2.64 ГБ (6.49 млн строк/с., 71.79 МБ/с.) ``` -Теперь мы можем определить название местоположения, откуда поступает большинство сообщений: +Теперь мы можем узнать название места, из которого поступает большинство записей: ```sql SELECT Location, count() AS c @@ -314,29 +314,29 @@ LIMIT 4 │ London, United Kingdom │ 538738 │ └────────────────────────┴────────┘ -Получено 4 строки. Время выполнения: 0.142 сек. Обработано 59.82 млн строк, 1.08 ГБ (420.73 млн строк/с., 7.60 ГБ/с.) -Пиковое потребление памяти: 666.82 МиБ. +Получено 4 строки. Прошло: 0.142 сек. Обработано 59.82 млн строк, 1.08 ГБ (420.73 млн строк/сек., 7.60 ГБ/сек.) +Пиковое использование памяти: 666.82 МиБ. ``` -## Расширенные темы по словарям {#advanced-dictionary-topics} +## Расширенные темы о словарях {#advanced-dictionary-topics} ### Выбор `LAYOUT` словаря {#choosing-the-dictionary-layout} -Секция `LAYOUT` управляет внутренней структурой данных словаря. Существует несколько вариантов, которые описаны [здесь](/sql-reference/dictionaries#ways-to-store-dictionaries-in-memory). Некоторые рекомендации по выбору подходящего варианта структуры можно найти [здесь](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout). +Клауза `LAYOUT` управляет внутренней структурой данных словаря. Существует несколько вариантов, описанных [здесь](/sql-reference/dictionaries#ways-to-store-dictionaries-in-memory). Некоторые рекомендации по выбору подходящего `LAYOUT` можно найти [здесь](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout). ### Обновление словарей {#refreshing-dictionaries} -Мы указали для словаря `LIFETIME` со значением `MIN 600 MAX 900`. LIFETIME — это интервал обновления словаря; в данном случае значения приводят к периодической перезагрузке через случайный интервал между 600 и 900 с. Этот случайный интервал необходим для распределения нагрузки на источник словаря при обновлении на большом количестве серверов. Во время обновления старая версия словаря по‑прежнему может запрашиваться; только начальная загрузка блокирует запросы. Обратите внимание, что установка `(LIFETIME(0))` предотвращает обновление словарей. -Словари можно принудительно перезагрузить с помощью команды `SYSTEM RELOAD DICTIONARY`. +Мы указали для словаря `LIFETIME` со значением `MIN 600 MAX 900`. LIFETIME — это интервал обновления словаря; в данном случае значения приводят к периодической перезагрузке через случайный интервал между 600 и 900 секундами. Такой случайный интервал необходим для распределения нагрузки на источник словаря при обновлении на большом числе серверов. Во время обновления старая версия словаря по-прежнему может использоваться в запросах, при этом только начальная загрузка блокирует запросы. Обратите внимание, что задание `(LIFETIME(0))` предотвращает обновление словарей. +Принудительную перезагрузку словарей можно выполнить с помощью команды `SYSTEM RELOAD DICTIONARY`. -Для источников баз данных, таких как ClickHouse и Postgres, можно настроить запрос, который будет обновлять словари только в том случае, если они действительно изменились (это определяется ответом запроса), а не через фиксированный периодический интервал. Дополнительные сведения можно найти [здесь](/sql-reference/dictionaries#refreshing-dictionary-data-using-lifetime). +Для источников данных, таких как ClickHouse и Postgres, вы можете настроить запрос, который будет обновлять словари только в том случае, если они действительно изменились (это определяется ответом на запрос), а не с периодическим интервалом. Дополнительные подробности можно найти [здесь](/sql-reference/dictionaries#refreshing-dictionary-data-using-lifetime). ### Другие типы словарей {#other-dictionary-types} -ClickHouse также поддерживает [иерархические словари](/sql-reference/dictionaries#hierarchical-dictionaries), [полигональные словари](/sql-reference/dictionaries#polygon-dictionaries) и словари на основе [регулярных выражений](/sql-reference/dictionaries#regexp-tree-dictionary). +ClickHouse также поддерживает [иерархические](/sql-reference/dictionaries#hierarchical-dictionaries), [многоугольные](/sql-reference/dictionaries#polygon-dictionaries) и [словарі на основе регулярных выражений](/sql-reference/dictionaries#regexp-tree-dictionary) словари. -### Дополнительные материалы {#more-reading} +### Дополнительные материалы для чтения {#more-reading} - [Использование словарей для ускорения запросов](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse) -- [Расширенная конфигурация словарей](/sql-reference/dictionaries) +- [Расширенная конфигурация словарей](/sql-reference/dictionaries) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md index 42c51c80d44..3fb71630005 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Atomic +# Atomic {#atomic} Движок `Atomic` поддерживает неблокирующие запросы [`DROP TABLE`](#drop-detach-table) и [`RENAME TABLE`](#rename-table), а также атомарные запросы [`EXCHANGE TABLES`](#exchange-tables). Движок базы данных `Atomic` по умолчанию используется в open-source версии ClickHouse. @@ -19,16 +19,16 @@ doc_type: 'reference' -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE test [ENGINE = Atomic] [SETTINGS disk=...]; ``` -## Особенности и рекомендации +## Особенности и рекомендации {#specifics-and-recommendations} -### UUID таблицы +### UUID таблицы {#table-uuid} Каждая таблица в базе данных `Atomic` имеет постоянный идентификатор [UUID](../../sql-reference/data-types/uuid.md) и хранит свои данные в следующем каталоге: @@ -50,16 +50,16 @@ CREATE TABLE name UUID '28f1c61c-2970-457a-bffe-454156ddcfef' (n UInt64) ENGINE Вы можете использовать настройку [show_table_uuid_in_table_create_query_if_not_nil](../../operations/settings/settings.md#show_table_uuid_in_table_create_query_if_not_nil), чтобы отображать UUID в запросе `SHOW CREATE`. ::: -### RENAME TABLE +### RENAME TABLE {#rename-table} Запросы [`RENAME`](../../sql-reference/statements/rename.md) не изменяют UUID и не перемещают данные таблицы. Эти запросы выполняются сразу и не ждут завершения других запросов, использующих таблицу. -### DROP/DETACH TABLE +### DROP/DETACH TABLE {#drop-detach-table} При использовании `DROP TABLE` данные сразу не удаляются. Движок `Atomic` просто помечает таблицу как удалённую, перемещая её метаданные в `/clickhouse_path/metadata_dropped/` и уведомляя фоновый поток. Задержка перед окончательным удалением данных таблицы задаётся настройкой [`database_atomic_delay_before_drop_table_sec`](../../operations/server-configuration-parameters/settings.md#database_atomic_delay_before_drop_table_sec). Вы можете указать синхронный режим с помощью модификатора `SYNC`. Для этого используйте настройку [`database_atomic_wait_for_drop_and_detach_synchronously`](../../operations/settings/settings.md#database_atomic_wait_for_drop_and_detach_synchronously). В этом случае `DROP` ожидает завершения выполняющихся `SELECT`, `INSERT` и других запросов, которые используют таблицу. Таблица будет удалена, когда она больше не используется. -### EXCHANGE TABLES/DICTIONARIES +### EXCHANGE TABLES/DICTIONARIES {#exchange-tables} Запрос [`EXCHANGE`](../../sql-reference/statements/exchange.md) атомарно меняет местами таблицы или словари. Например, вместо этой неатомарной операции: @@ -73,11 +73,11 @@ RENAME TABLE new_table TO tmp, old_table TO new_table, tmp TO old_table; EXCHANGE TABLES новая_таблица AND старая_таблица; ``` -### ReplicatedMergeTree в базе данных Atomic +### ReplicatedMergeTree в базе данных Atomic {#replicatedmergetree-in-atomic-database} Для таблиц [`ReplicatedMergeTree`](/engines/table-engines/mergetree-family/replication) рекомендуется не указывать параметры движка для пути в ZooKeeper и имени реплики. В этом случае будут использоваться параметры конфигурации [`default_replica_path`](../../operations/server-configuration-parameters/settings.md#default_replica_path) и [`default_replica_name`](../../operations/server-configuration-parameters/settings.md#default_replica_name). Если вы хотите явно задать параметры движка, рекомендуется использовать макрос `{uuid}`. Это гарантирует, что для каждой таблицы в ZooKeeper автоматически генерируются уникальные пути. -### Диск для метаданных +### Диск для метаданных {#metadata-disk} Если в `SETTINGS` указан `disk`, этот диск используется для хранения файлов метаданных таблицы. Например: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md index fe431c7cba0..3e8ac70a2d8 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md @@ -7,17 +7,13 @@ title: 'Backup' doc_type: 'reference' --- - - -# Backup +# Backup {#backup} База данных Backup позволяет мгновенно подключить таблицу или базу данных из [резервных копий](../../operations/backup) в режиме только для чтения. База данных Backup работает как с инкрементными, так и с неинкрементными резервными копиями. - - -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE backup_database @@ -38,8 +34,7 @@ ENGINE = Backup('database_name_inside_backup', Disk('disk_name', 'backup_name')) * `database_name_inside_backup` — Имя базы данных внутри резервной копии. * `backup_destination` — Место размещения резервной копии. - -## Пример использования +## Пример использования {#usage-example} Рассмотрим пример с местом назначения резервных копий типа `Disk`. Сначала настроим диск для резервных копий в `storage.xml`: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md index 437d1c815d9..dd5744386f0 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' -# DataLakeCatalog +# DataLakeCatalog {#datalakecatalog} Движок базы данных `DataLakeCatalog` позволяет подключить ClickHouse к внешним каталогам данных и выполнять запросы к данным в открытых табличных форматах без необходимости дублирования данных. @@ -28,7 +28,7 @@ doc_type: 'reference' -## Создание базы данных +## Создание базы данных {#creating-a-database} Чтобы использовать движок `DataLakeCatalog`, необходимо включить приведённые ниже настройки: @@ -66,7 +66,7 @@ SETTINGS | `region` | Регион AWS для сервиса (например, `us-east-1`) | -## Примеры +## Примеры {#examples} Ниже приведены примеры использования движка `DataLakeCatalog`: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/index.md index 4b07f0477e1..4f4cc2b35f7 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/index.md @@ -8,9 +8,7 @@ title: 'Движки баз данных' doc_type: 'landing-page' --- - - -# Движки баз данных +# Движки баз данных {#database-engines} Движки баз данных позволяют работать с таблицами. По умолчанию ClickHouse использует движок базы данных [Atomic](../../engines/database-engines/atomic.md), который предоставляет настраиваемые [движки таблиц](../../engines/table-engines/index.md) и [диалект SQL](../../sql-reference/syntax.md). @@ -18,24 +16,24 @@ doc_type: 'landing-page' {/* Оглавление для этой страницы автоматически генерируется скриптом https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - на основе полей фронтматтера YAML: slug, description, title. + из полей YAML front matter: slug, description, title. - Если вы заметили ошибку, отредактируйте YML-фронтматтер самих страниц. + Если вы заметили ошибку, отредактируйте YML front matter самих страниц. */ } -{/*АВТОГЕНЕРИРОВАНО_НАЧАЛО*/ } +{/*AUTOGENERATED_START*/ } -| Страница | Описание | +| Page | Description | | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [Shared](/engines/database-engines/shared) | Страница, описывающая движок базы данных `Shared`, доступный в ClickHouse Cloud. | | [Atomic](/engines/database-engines/atomic) | Движок `Atomic` поддерживает неблокирующие запросы `DROP TABLE` и `RENAME TABLE`, а также атомарные запросы `EXCHANGE TABLES`. Движок базы данных `Atomic` используется по умолчанию. | -| [Lazy](/engines/database-engines/lazy) | Хранит таблицы в оперативной памяти только `expiration_time_in_seconds` секунд после последнего обращения. Может использоваться только с таблицами типа Log. | -| [Replicated](/engines/database-engines/replicated) | Движок основан на движке Atomic. Он поддерживает репликацию метаданных через DDL-лог, который записывается в ZooKeeper и выполняется на всех репликах заданной базы данных. | +| [Shared](/engines/database-engines/shared) | Страница, описывающая движок базы данных `Shared`, доступный в ClickHouse Cloud. | +| [Lazy](/engines/database-engines/lazy) | Хранит таблицы в RAM только в течение `expiration_time_in_seconds` секунд после последнего доступа. Может использоваться только с таблицами типа Log. | +| [Replicated](/engines/database-engines/replicated) | Движок основан на движке `Atomic`. Поддерживает репликацию метаданных через DDL‑лог, который записывается в ZooKeeper и выполняется на всех репликах для заданной базы данных. | | [PostgreSQL](/engines/database-engines/postgresql) | Позволяет подключаться к базам данных на удалённом сервере PostgreSQL. | | [MySQL](/engines/database-engines/mysql) | Позволяет подключаться к базам данных на удалённом сервере MySQL и выполнять запросы `INSERT` и `SELECT` для обмена данными между ClickHouse и MySQL. | | [SQLite](/engines/database-engines/sqlite) | Позволяет подключаться к базам данных SQLite и выполнять запросы `INSERT` и `SELECT` для обмена данными между ClickHouse и SQLite. | -| [MaterializedPostgreSQL](/engines/database-engines/materialized-postgresql) | Создаёт базу данных ClickHouse на основе таблиц из базы данных PostgreSQL. | -| [Backup](/engines/database-engines/backup) | Позволяет мгновенно подключать таблицы/базы данных из резервных копий в режиме только чтения. | -| [DataLakeCatalog](/engines/database-engines/datalakecatalog) | Движок базы данных DataLakeCatalog позволяет подключать ClickHouse к внешним каталогам данных и выполнять запросы к данным в формате открытых таблиц. | +| [Backup](/engines/database-engines/backup) | Позволяет мгновенно подключать таблицу или базу данных из резервных копий в режиме только для чтения. | +| [MaterializedPostgreSQL](/engines/database-engines/materialized-postgresql) | Создаёт в ClickHouse базу данных с таблицами из базы данных PostgreSQL. | +| [DataLakeCatalog](/engines/database-engines/datalakecatalog) | Движок базы данных `DataLakeCatalog` позволяет подключить ClickHouse к внешним каталогам данных и выполнять запросы к данным в открытых табличных форматах | -{/*AUTOGENERATED_END*/ } +{/*АВТОГЕНЕРИРОВАНО_НАЧАЛО*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md index 806de8919fa..258c83dd07e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md @@ -7,17 +7,13 @@ title: 'Lazy' doc_type: 'reference' --- +# Lazy {#lazy} +Удерживает таблицы в RAM только в течение `expiration_time_in_seconds` секунд после последнего обращения. Может использоваться только с таблицами типа *Log. -# Lazy +Оптимизирован для хранения множества небольших таблиц *Log, между обращениями к которым проходят большие промежутки времени. -Удерживает таблицы в RAM только в течение `expiration_time_in_seconds` секунд после последнего обращения. Может использоваться только с таблицами типа \*Log. - -Оптимизирован для хранения множества небольших таблиц \*Log, между обращениями к которым проходят большие промежутки времени. - - - -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE testlazy diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md index 0708017bdc9..16804b2daaf 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MaterializedPostgreSQL +# MaterializedPostgreSQL {#materializedpostgresql} @@ -35,7 +35,7 @@ SET allow_experimental_database_materialized_postgresql=1 ::: -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -50,7 +50,7 @@ ENGINE = MaterializedPostgreSQL('host:port', 'database', 'user', 'password') [SE * `password` — пароль пользователя. -## Пример использования +## Пример использования {#example-of-use} ```sql CREATE DATABASE postgres_db @@ -66,7 +66,7 @@ SELECT * FROM postgresql_db.postgres_table; ``` -## Динамическое добавление новых таблиц в репликацию +## Динамическое добавление новых таблиц в репликацию {#dynamically-adding-table-to-replication} После создания базы данных `MaterializedPostgreSQL` она не будет автоматически обнаруживать новые таблицы в соответствующей базе данных PostgreSQL. Такие таблицы можно добавить вручную: @@ -79,7 +79,7 @@ ATTACH TABLE postgres_database.new_table; ::: -## Динамическое исключение таблиц из репликации +## Динамическое исключение таблиц из репликации {#dynamically-removing-table-from-replication} Можно исключить отдельные таблицы из репликации: @@ -88,7 +88,7 @@ DETACH TABLE postgres_database.table_to_remove PERMANENTLY; ``` -## Схема PostgreSQL +## Схема PostgreSQL {#schema} Схемы PostgreSQL ([schema](https://www.postgresql.org/docs/9.1/ddl-schemas.html)) можно настраивать тремя способами (начиная с версии 21.12). @@ -136,7 +136,7 @@ SELECT * FROM database1.`schema2.table2`; Предупреждение: в данном случае точки в имени таблицы не допускаются. -## Требования +## Требования {#requirements} 1. Параметр [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) должен иметь значение `logical`, а параметр `max_replication_slots` — значение не менее `2` в конфигурационном файле PostgreSQL. @@ -171,9 +171,9 @@ WHERE oid = 'postgres_table'::regclass; ::: -## Настройки +## Настройки {#settings} -### `materialized_postgresql_tables_list` +### `materialized_postgresql_tables_list` {#materialized-postgresql-tables-list} Задает список таблиц базы данных PostgreSQL, разделенный запятыми, которые будут реплицироваться с помощью движка базы данных [MaterializedPostgreSQL](../../engines/database-engines/materialized-postgresql.md). @@ -185,15 +185,15 @@ materialized_postgresql_tables_list = 'table1(co1, col2),table2,table3(co3, col5 Значение по умолчанию: пустой список — означает, что будет реплицирована вся база данных PostgreSQL. -### `materialized_postgresql_schema` +### `materialized_postgresql_schema` {#materialized-postgresql-schema} Значение по умолчанию: пустая строка (используется схема по умолчанию). -### `materialized_postgresql_schema_list` +### `materialized_postgresql_schema_list` {#materialized-postgresql-schema-list} Значение по умолчанию: пустой список (используется схема по умолчанию). -### `materialized_postgresql_max_block_size` +### `materialized_postgresql_max_block_size` {#materialized-postgresql-max-block-size} Задаёт количество строк, собираемых в памяти перед сбросом данных в таблицу базы данных PostgreSQL. @@ -203,11 +203,11 @@ materialized_postgresql_tables_list = 'table1(co1, col2),table2,table3(co3, col5 Значение по умолчанию: `65536`. -### `materialized_postgresql_replication_slot` +### `materialized_postgresql_replication_slot` {#materialized-postgresql-replication-slot} Слот репликации, созданный пользователем. Должен использоваться вместе с `materialized_postgresql_snapshot`. -### `materialized_postgresql_snapshot` +### `materialized_postgresql_snapshot` {#materialized-postgresql-snapshot} Текстовая строка, идентифицирующая снимок, из которого будет выполнен [начальный дамп таблиц PostgreSQL](../../engines/database-engines/materialized-postgresql.md). Должен использоваться вместе с `materialized_postgresql_replication_slot`. @@ -225,7 +225,7 @@ SELECT * FROM database1.table1; ALTER DATABASE postgres_database MODIFY SETTING materialized_postgresql_max_block_size = <новый_размер>; ``` -### `materialized_postgresql_use_unique_replication_consumer_identifier` +### `materialized_postgresql_use_unique_replication_consumer_identifier` {#materialized_postgresql_use_unique_replication_consumer_identifier} Использует уникальный идентификатор потребителя при репликации. Значение по умолчанию — `0`. Если установлено в `1`, позволяет настроить несколько таблиц `MaterializedPostgreSQL`, указывающих на одну и ту же таблицу `PostgreSQL`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md index 27bd9dbfb1a..9dab4fbcd49 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md @@ -10,8 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# Движок базы данных MySQL +# Движок базы данных MySQL {#mysql-database-engine} @@ -25,9 +24,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - `CREATE TABLE` - `ALTER` - - -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -41,7 +38,6 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') * `user` — пользователь MySQL. * `password` — пароль пользователя. - ## Поддержка типов данных {#data_types-support} | MySQL | ClickHouse | @@ -64,9 +60,7 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') Поддерживается тип [Nullable](../../sql-reference/data-types/nullable.md). - - -## Поддержка глобальных переменных +## Поддержка глобальных переменных {#global-variables-support} Для лучшей совместимости вы можете обращаться к глобальным переменным в стиле MySQL — через `@@identifier`. @@ -85,8 +79,7 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') SELECT @@version; ``` - -## Примеры использования +## Примеры использования {#examples-of-use} Таблица в MySQL: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md index f0e0ee1274c..651f61d18f3 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# PostgreSQL +# PostgreSQL {#postgresql} Позволяет подключаться к базам данных на удалённом сервере [PostgreSQL](https://www.postgresql.org). Поддерживает операции чтения и записи (запросы `SELECT` и `INSERT`) для обмена данными между ClickHouse и PostgreSQL. @@ -19,7 +19,7 @@ doc_type: 'guide' -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE test_database @@ -56,7 +56,7 @@ ENGINE = PostgreSQL('host:port', 'database', 'user', 'password'[, `schema`, `use -## Примеры использования +## Примеры использования {#examples-of-use} База данных в ClickHouse, обменивающаяся данными с сервером PostgreSQL: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md index 2208376ea83..41025a6ad98 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md @@ -11,7 +11,7 @@ doc_type: 'reference' -# Replicated +# Replicated {#replicated} Движок основан на движке [Atomic](../../engines/database-engines/atomic.md). Он поддерживает репликацию метаданных по журналу DDL, который записывается в ZooKeeper и выполняется на всех репликах заданной базы данных. @@ -19,7 +19,7 @@ doc_type: 'reference' -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE testdb [UUID '...'] ENGINE = Replicated('zoo_path', 'shard_name', 'replica_name') [SETTINGS ...] @@ -56,7 +56,7 @@ DDL-запросы с базой данных `Replicated` работают ан -## Пример использования +## Пример использования {#usage-example} Создание кластера с тремя хостами: @@ -155,7 +155,7 @@ node2 :) SELECT materialize(hostName()) AS host, groupArray(n) FROM r.d GROUP BY ``` -## Параметры +## Параметры {#settings} Поддерживаются следующие параметры: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md index 03f815ab6f6..8e1d241a36c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md @@ -12,7 +12,7 @@ import CloudOnlyBadge from '@theme/badges/CloudOnlyBadge'; -# Движок базы данных Shared +# Движок базы данных Shared {#shared-database-engine} Движок базы данных `Shared` работает совместно с Shared Catalog для управления базами данных, таблицы которых используют stateless‑движки таблиц, такие как [`SharedMergeTree`](/cloud/reference/shared-merge-tree). Эти движки таблиц не записывают постоянное состояние на диск и совместимы с динамическими вычислительными средами. @@ -30,7 +30,7 @@ import CloudOnlyBadge from '@theme/badges/CloudOnlyBadge'; -## Синтаксис +## Синтаксис {#syntax} Для конечных пользователей работа с Shared Catalog и движком базы данных Shared не требует дополнительной конфигурации. Создание базы данных выполняется как обычно: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md index 1807def09b5..05cb67cdcee 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md @@ -10,13 +10,13 @@ doc_type: 'reference' -# SQLite +# SQLite {#sqlite} Позволяет подключаться к базе данных [SQLite](https://www.sqlite.org/index.html) и выполнять запросы `INSERT` и `SELECT` для обмена данными между ClickHouse и SQLite. -## Создание базы данных +## Создание базы данных {#creating-a-database} ```sql CREATE DATABASE sqlite_database @@ -46,7 +46,7 @@ SQLite не требует отдельного управления служб -## Пример использования +## Пример использования {#usage-example} База данных в ClickHouse, подключённая к SQLite: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/index.md index 561f2e07e52..ada5fb3706d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/index.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Движки таблиц +# Движки таблиц {#table-engines} Движок таблицы (тип таблицы) определяет: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md index 4e51b90f647..75b0d8122fb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md @@ -9,15 +9,11 @@ title: 'Табличный движок ExternalDistributed' doc_type: 'reference' --- - - -# Движок таблицы ExternalDistributed +# Движок таблицы ExternalDistributed {#externaldistributed-table-engine} Движок `ExternalDistributed` позволяет выполнять запросы `SELECT` к данным, которые хранятся на удалённых серверах с MySQL или PostgreSQL. Принимает в качестве аргумента движки [MySQL](../../../engines/table-engines/integrations/mysql.md) или [PostgreSQL](../../../engines/table-engines/integrations/postgresql.md), поэтому возможен шардинг. - - -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -44,8 +40,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `user` — Имя пользователя. * `password` — Пароль пользователя. - -## Детали реализации +## Детали реализации {#implementation-details} Поддерживаются несколько реплик; их необходимо перечислять через `|`, а шарды — через `,`. Например: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md index 45138339727..b9bb1f91db4 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md @@ -9,14 +9,14 @@ doc_type: 'reference' -# Движок таблицы ArrowFlight +# Движок таблицы ArrowFlight {#arrowflight-table-engine} Движок таблицы ArrowFlight позволяет ClickHouse выполнять запросы к удалённым наборам данных по протоколу [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html). Эта интеграция даёт возможность ClickHouse получать данные с внешних серверов с поддержкой Flight в колонном формате Arrow с высокой производительностью. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) (это будет работать только в том случае, если сервер Arrow Flight это допускает). -## Пример использования +## Пример использования {#usage-example} В этом примере показано, как создать таблицу для чтения данных с удалённого сервера Arrow Flight: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md index db9d948a95a..765b39acf7d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md @@ -8,13 +8,9 @@ title: 'Табличный движок AzureQueue' doc_type: 'reference' --- +# Табличный движок AzureQueue {#azurequeue-table-engine} - -# Движок таблицы AzureQueue - -Этот движок обеспечивает интеграцию с экосистемой [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs), позволяя выполнять потоковую загрузку данных. - - +Этот движок обеспечивает интеграцию с экосистемой [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) и поддерживает потоковый импорт данных. ## Создание таблицы {#creating-a-table} @@ -30,9 +26,9 @@ CREATE TABLE test (name String, value UInt32) **Параметры движка** -Параметры `AzureQueue` совпадают с параметрами движка таблиц `AzureBlobStorage`. См. описание параметров [здесь](../../../engines/table-engines/integrations/azureBlobStorage.md). +Параметры `AzureQueue` такие же, как у табличного движка `AzureBlobStorage`. См. раздел с параметрами [здесь](../../../engines/table-engines/integrations/azureBlobStorage.md). -Аналогично движку таблиц [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage), можно использовать эмулятор Azurite для локальной разработки с Azure Storage. Подробнее [здесь](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage). +Аналогично табличному движку [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage), пользователи могут использовать эмулятор Azurite для локальной разработки хранилища Azure. Дополнительные сведения см. [здесь](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage). **Пример** @@ -46,22 +42,58 @@ ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1; SETTINGS mode = 'unordered' ``` +## Settings {#settings} + +Набор поддерживаемых настроек в основном совпадает с настройками для движка таблиц `S3Queue`, но без префикса `s3queue_`. См. [полный список настроек](../../../engines/table-engines/integrations/s3queue.md#settings). +Чтобы получить список настроек, заданных для таблицы, используйте таблицу `system.azure_queue_settings`. Доступно начиная с версии 24.10. + +Ниже приведены настройки, совместимые только с AzureQueue и не применимые к S3Queue. + +### `after_processing_move_connection_string` {#after_processing_move_connection_string} + +Строка подключения к Azure Blob Storage, в которую будут перемещаться успешно обработанные файлы, если в качестве назначения используется другой контейнер Azure. + +Возможные значения: + +* String. + +Значение по умолчанию: пустая строка. + +### `after_processing_move_container` {#after_processing_move_container} + +Имя контейнера, в который необходимо переместить успешно обработанные файлы, если целевым контейнером является другой контейнер Azure. -## Настройки {#settings} +Возможные значения: -Набор поддерживаемых настроек совпадает с настройками движка таблиц `S3Queue`, но без префикса `s3queue_`. См. [полный список настроек](../../../engines/table-engines/integrations/s3queue.md#settings). -Чтобы получить список настроек, настроенных для таблицы, используйте таблицу `system.azure_queue_settings`. Доступно начиная с версии `24.10`. +* Строка. +Значение по умолчанию: пустая строка. -## Description {#description} +Пример: -`SELECT` не особенно полезен для потоковой загрузки данных (за исключением отладки), поскольку каждый файл может быть импортирован только один раз. Более практичным является создание потоков обработки в реальном времени с использованием [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: +```sql +CREATE TABLE azure_queue_engine_table +( + `key` UInt64, + `data` String +) +ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', 'testcontainer', '*', 'CSV') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_move_connection_string = 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', + after_processing_move_container = 'dst-container'; +``` + +## Описание {#description} -1. Используйте движок для создания таблицы, которая будет получать данные из указанного пути в S3, и рассматривайте её как поток данных. -2. Создайте таблицу с требуемой структурой. -3. Создайте материализованное представление, которое преобразует данные из движка и помещает их в ранее созданную таблицу. +`SELECT` не особенно полезен для потокового импорта (кроме отладки), потому что каждый файл можно импортировать только один раз. Гораздо практичнее создавать потоки в реальном времени с использованием [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: -Когда `MATERIALIZED VIEW` подключается к движку, оно начинает собирать данные в фоновом режиме. +1. Используйте табличный движок для создания таблицы, потребляющей данные из указанного пути в S3, и рассматривайте её как поток данных. +2. Создайте таблицу с требуемой структурой. +3. Создайте материализованное представление, которое преобразует данные из табличного движка и записывает их в ранее созданную таблицу. + +Когда `MATERIALIZED VIEW` связывается с табличным движком, оно начинает собирать данные в фоновом режиме. Пример: @@ -80,32 +112,30 @@ CREATE MATERIALIZED VIEW consumer TO stats SELECT * FROM stats ORDER BY key; ``` - ## Виртуальные столбцы {#virtual-columns} -- `_path` — Путь к файлу. -- `_file` — Имя файла. - -Подробнее о виртуальных столбцах см. [здесь](../../../engines/table-engines/index.md#table_engines-virtual_columns). +* `_path` — Путь к файлу. +* `_file` — Имя файла. +Дополнительные сведения о виртуальных столбцах см. [здесь](../../../engines/table-engines/index.md#table_engines-virtual_columns). ## Интроспекция {#introspection} Включите логирование для таблицы с помощью настройки таблицы `enable_logging_to_queue_log=1`. -Возможности интроспекции аналогичны [движку таблиц S3Queue](/engines/table-engines/integrations/s3queue#introspection) с несколькими существенными отличиями: +Возможности интроспекции такие же, как у [движка таблиц S3Queue](/engines/table-engines/integrations/s3queue#introspection), с несколькими важными отличиями: -1. Используйте `system.azure_queue` для хранения состояния очереди в памяти для версий сервера >= 25.1. Для более старых версий используйте `system.s3queue` (она также будет содержать информацию для таблиц `azure`). -2. Включите `system.azure_queue_log` через основной конфигурационный файл ClickHouse, например: +1. Используйте `system.azure_queue` для представления состояния очереди в памяти для версий сервера >= 25.1. Для более старых версий используйте `system.s3queue` (он также будет содержать информацию для таблиц `azure`). +2. Включите `system.azure_queue_log` через основную конфигурацию ClickHouse, например: ```xml - - system - azure_queue_log
-
+ + system + azure_queue_log
+
``` -Эта персистентная таблица содержит ту же информацию, что и `system.s3queue`, но для обработанных и неудачно обработанных файлов. +Эта постоянная таблица содержит ту же информацию, что и `system.s3queue`, но для обработанных и завершившихся ошибкой файлов. Таблица имеет следующую структуру: @@ -114,23 +144,23 @@ SELECT * FROM stats ORDER BY key; CREATE TABLE system.azure_queue_log ( `hostname` LowCardinality(String) COMMENT 'Имя хоста', - `event_date` Date COMMENT 'Дата события записи этой строки лога', - `event_time` DateTime COMMENT 'Время события записи этой строки лога', - `database` String COMMENT 'Имя базы данных, в которой находится текущая таблица S3Queue.', + `event_date` Date COMMENT 'Дата события записи данной строки журнала', + `event_time` DateTime COMMENT 'Время события записи данной строки журнала', + `database` String COMMENT 'Имя базы данных, в которой находится таблица S3Queue.', `table` String COMMENT 'Имя таблицы S3Queue.', `uuid` String COMMENT 'UUID таблицы S3Queue', `file_name` String COMMENT 'Имя обрабатываемого файла', `rows_processed` UInt64 COMMENT 'Количество обработанных строк', `status` Enum8('Processed' = 0, 'Failed' = 1) COMMENT 'Статус обработки файла', `processing_start_time` Nullable(DateTime) COMMENT 'Время начала обработки файла', - `processing_end_time` Nullable(DateTime) COMMENT 'Время окончания обработки файла', - `exception` String COMMENT 'Сообщение об исключении, если произошло' + `processing_end_time` Nullable(DateTime) COMMENT 'Время завершения обработки файла', + `exception` String COMMENT 'Сообщение об исключении при его возникновении' ) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) SETTINGS index_granularity = 8192 -COMMENT 'Содержит записи логирования с информацией о файлах, обработанных движком S3Queue.' +COMMENT 'Содержит записи журнала с информацией о файлах, обработанных движком S3Queue.' ``` @@ -142,7 +172,7 @@ FROM system.azure_queue_log LIMIT 1 FORMAT Vertical -Row 1: +Строка 1: ────── hostname: clickhouse event_date: 2024-12-16 @@ -157,6 +187,6 @@ processing_start_time: 2024-12-16 13:42:47 processing_end_time: 2024-12-16 13:42:47 exception: -1 row in set. Elapsed: 0.002 sec. +Получена 1 строка. Затрачено: 0.002 сек. ``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md index 05dbb08786b..ccc134f3794 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Табличный движок AzureBlobStorage +# Табличный движок AzureBlobStorage {#azureblobstorage-table-engine} Этот движок предоставляет интеграцию с экосистемой [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs). -## Создание таблицы +## Создание таблицы {#create-table} ```sql CREATE TABLE azure_blob_storage_table (name String, value UInt32) @@ -24,7 +24,7 @@ CREATE TABLE azure_blob_storage_table (name String, value UInt32) [SETTINGS ...] ``` -### Параметры движка +### Параметры движка {#engine-parameters} * `endpoint` — URL конечной точки AzureBlobStorage с контейнером и префиксом. Дополнительно может содержать `account_name`, если это требуется используемому методу аутентификации (`http://azurite1:{port}/[account_name]{container_name}/{data_prefix}`), либо эти параметры могут быть переданы отдельно с помощью `storage_account_url`, `account_name` и `container`. Для указания префикса должен использоваться `endpoint`. * `endpoint_contains_account_name` — флаг, указывающий, содержит ли `endpoint` `account_name`, так как это требуется только для некоторых методов аутентификации. (По умолчанию: `true`) @@ -70,7 +70,7 @@ SELECT * FROM test_table; -## Аутентификация +## Аутентификация {#authentication} В настоящее время есть три способа аутентификации: @@ -78,7 +78,7 @@ SELECT * FROM test_table; * `SAS Token` — может использоваться при указании `endpoint`, `connection_string` или `storage_account_url`. Определяется по наличию символа '?' в URL. См. раздел [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage#using-shared-access-signatures-sas-sas-tokens) с примерами. * `Workload Identity` — может использоваться при указании `endpoint` или `storage_account_url`. Если параметр `use_workload_identity` установлен в конфигурации, для аутентификации используется механизм [workload identity](https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/identity/azure-identity#authenticate-azure-hosted-applications). -### Кэш данных +### Кэш данных {#data-cache} Движок таблиц `Azure` поддерживает кэширование данных на локальном диске. Параметры конфигурации и использование кэша файловой системы описаны в этом [разделе](/operations/storing-data.md/#using-local-cache). @@ -107,13 +107,13 @@ SETTINGS filesystem_cache_name = 'cache_for_azure', enable_filesystem_cache = 1; 2. повторно используйте конфигурацию кеша (и, соответственно, хранилище кеша) из секции `storage_configuration` ClickHouse, [описанной здесь](/operations/storing-data.md/#using-local-cache) -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — необязательный параметр. В большинстве случаев ключ партиционирования не нужен, а когда он все же требуется, обычно нет необходимости делать его более детализированным, чем по месяцам. Партиционирование не ускоряет выполнение запросов (в отличие от выражения ORDER BY). Не следует использовать слишком детализированное партиционирование. Не партиционируйте данные по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — это столбец с датой типа [Date](/sql-reference/data-types/date.md). Имена партиций в этом случае имеют формат `"YYYYMM"`. -#### Стратегия партиционирования +#### Стратегия партиционирования {#partition-strategy} `WILDCARD` (по умолчанию): заменяет подстановочный шаблон `{_partition_id}` в пути к файлу фактическим ключом партиции. Чтение не поддерживается. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md index 3e530298ff5..3de508eafa6 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Табличный движок DeltaLake +# Табличный движок DeltaLake {#deltalake-table-engine} Этот табличный движок обеспечивает доступ только для чтения к существующим таблицам [Delta Lake](https://github.com/delta-io/delta) в Amazon S3. -## Создание таблицы +## Создание таблицы {#create-table} Учтите, что таблица Delta Lake уже должна существовать в S3: эта команда не принимает параметры DDL для создания новой таблицы. @@ -55,7 +55,7 @@ CREATE TABLE deltalake ENGINE=DeltaLake('http://mars-doc-test.s3.amazonaws.com/c CREATE TABLE deltalake ENGINE=DeltaLake(deltalake_conf, filename = 'test_table') ``` -### Кэш данных +### Кэш данных {#data-cache} Движок таблиц `Iceberg` и табличная функция поддерживают кэширование данных так же, как хранилища `S3`, `AzureBlobStorage`, `HDFS`. См. [здесь](../../../engines/table-engines/integrations/s3.md#data-cache). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md index b84f7467e40..c0b2552cb28 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# Табличный движок EmbeddedRocksDB +# Табличный движок EmbeddedRocksDB {#embeddedrocksdb-table-engine} Этот движок позволяет интегрировать ClickHouse с [RocksDB](http://rocksdb.org/). - - -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,8 +56,7 @@ ENGINE = EmbeddedRocksDB PRIMARY KEY key ``` - -## Метрики +## Метрики {#metrics} Кроме того, есть таблица `system.rocksdb`, содержащая статистику RocksDB: @@ -76,8 +72,7 @@ FROM system.rocksdb └───────────────────────────┴───────┘ ``` - -## Конфигурация +## Конфигурация {#configuration} Вы также можете изменить любые [параметры RocksDB](https://github.com/facebook/rocksdb/wiki/Option-String-and-Option-Map) с помощью конфигурации: @@ -105,10 +100,9 @@ FROM system.rocksdb По умолчанию тривиальная оптимизация приблизительного подсчёта отключена, что может снизить производительность запросов `count()`. Чтобы включить эту оптимизацию, установите `optimize_trivial_approximate_count_query = 1`. Также этот параметр влияет на `system.tables` для движка EmbeddedRocksDB — включите его, чтобы увидеть приблизительные значения для `total_rows` и `total_bytes`. +## Поддерживаемые операции {#supported-operations} -## Поддерживаемые операции - -### Вставки +### Вставки {#inserts} При вставке новых строк в `EmbeddedRocksDB`, если ключ уже существует, его значение обновляется, иначе создаётся новый ключ. @@ -118,7 +112,7 @@ FROM system.rocksdb INSERT INTO test VALUES ('некоторый ключ', 1, 'значение', 3.2); ``` -### Удаление +### Удаление {#deletes} Строки можно удалять с помощью запросов `DELETE` или `TRUNCATE`. @@ -134,7 +128,7 @@ ALTER TABLE test DELETE WHERE key LIKE 'some%' AND v1 > 1; TRUNCATE TABLE test; ``` -### Обновления +### Обновления {#updates} Значения можно обновлять с помощью запроса `ALTER TABLE`. Первичный ключ нельзя изменять. @@ -142,7 +136,7 @@ TRUNCATE TABLE test; ALTER TABLE test UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; ``` -### Соединения +### Соединения {#joins} Поддерживается специальный тип соединения `direct` с таблицами EmbeddedRocksDB. Такое прямое соединение позволяет избежать формирования хеш-таблицы в памяти и @@ -161,9 +155,9 @@ SET join_algorithm = 'direct, hash' Когда параметр `join_algorithm` установлен в значение `direct, hash`, по возможности будут использоваться прямые соединения, а в остальных случаях — хеш-соединения. ::: -#### Пример +#### Пример {#example} -##### Создание и заполнение таблицы EmbeddedRocksDB +##### Создание и заполнение таблицы EmbeddedRocksDB {#create-and-populate-an-embeddedrocksdb-table} ```sql CREATE TABLE rdb @@ -185,7 +179,7 @@ INSERT INTO rdb FROM numbers_mt(10); ``` -##### Создайте и заполните таблицу для объединения с таблицей `rdb` +##### Создайте и заполните таблицу для объединения с таблицей `rdb` {#create-and-populate-a-table-to-join-with-table-rdb} ```sql CREATE TABLE t2 @@ -200,13 +194,13 @@ INSERT INTO t2 SELECT number AS k FROM numbers_mt(10) ``` -##### Установите алгоритм соединения в значение `direct` +##### Установите алгоритм соединения в значение `direct` {#set-the-join-algorithm-to-direct} ```sql SET join_algorithm = 'direct' ``` -##### Внутреннее соединение (INNER JOIN) +##### Внутреннее соединение (INNER JOIN) {#an-inner-join} ```sql SELECT * @@ -231,7 +225,7 @@ ORDER BY key ASC └─────┴─────────┴────────┴────────┘ ``` -### Подробнее о соединениях +### Подробнее о соединениях {#more-information-on-joins} * [настройка `join_algorithm`](/operations/settings/settings.md#join_algorithm) * [оператор JOIN](/sql-reference/statements/select/join.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md index c5c440f2548..75d677a601e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы HDFS +# Движок таблицы HDFS {#hdfs-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## Использование +## Использование {#usage} ```sql ENGINE = HDFS(URI, format) @@ -35,7 +35,7 @@ ENGINE = HDFS(URI, format) [Formats](/sql-reference/formats#formats-overview). * [PARTITION BY expr] -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — необязательный параметр. В большинстве случаев ключ партиционирования не требуется, а если и требуется, обычно нет необходимости делать его более детализированным, чем по месяцам. Партиционирование не ускоряет выполнение запросов (в отличие от выражения ORDER BY). Не следует использовать излишне детализированное партиционирование. Не выполняйте партиционирование данных по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). @@ -69,7 +69,7 @@ SELECT * FROM hdfs_engine_table LIMIT 2 ``` -## Подробности реализации +## Подробности реализации {#implementation-details} * Операции чтения и записи могут выполняться параллельно. * Не поддерживаются: @@ -137,7 +137,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ENGINE = HDFS('hdfs CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9000/big_dir/file{0..9}{0..9}{0..9}', 'CSV') ``` -## Конфигурация +## Конфигурация {#configuration} Как и GraphiteMergeTree, движок HDFS поддерживает расширенную настройку с помощью конфигурационного файла ClickHouse. Доступны два ключа конфигурации: глобальный (`hdfs`) и пользовательский (`hdfs_*`). Сначала применяется глобальная конфигурация, а затем — пользовательская (если она есть). @@ -155,9 +155,9 @@ CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9 ``` -### Параметры конфигурации +### Параметры конфигурации {#configuration-options} -#### Поддерживаемые libhdfs3 +#### Поддерживаемые libhdfs3 {#supported-by-libhdfs3} | **параметр** | **значение по умолчанию** | @@ -231,7 +231,7 @@ CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9 Если указаны `hadoop_kerberos_keytab`, `hadoop_kerberos_principal` или `hadoop_security_kerberos_ticket_cache_path`, будет использоваться аутентификация Kerberos. В этом случае `hadoop_kerberos_keytab` и `hadoop_kerberos_principal` являются обязательными. -## Поддержка HDFS Namenode HA +## Поддержка HDFS Namenode HA {#namenode-ha} libhdfs3 поддерживает HDFS Namenode HA. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md index 6849972b2bb..c438dd8772a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md @@ -9,22 +9,19 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# Движок таблицы Hive {#hive-table-engine} -# Движок таблицы Hive - - + Движок Hive позволяет выполнять запросы `SELECT` к таблицам Hive в HDFS. В данный момент он поддерживает следующие форматы входных данных: -- Text: поддерживает только простые скалярные типы столбцов, за исключением `binary` - -- ORC: поддерживает простые скалярные типы столбцов, за исключением `char`; из сложных типов поддерживает только тип `array` - -- Parquet: поддерживает все простые скалярные типы столбцов; из сложных типов поддерживает только тип `array` +* Text: поддерживает только простые скалярные типы столбцов, за исключением `binary` +* ORC: поддерживает простые скалярные типы столбцов, за исключением `char`; из сложных типов поддерживает только тип `array` +* Parquet: поддерживает все простые скалярные типы столбцов; из сложных типов поддерживает только тип `array` -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -52,10 +49,9 @@ PARTITION BY expr * `table` — имя удалённой таблицы. +## Пример использования {#usage-example} -## Пример использования - -### Как использовать локальный кэш файловой системы HDFS +### Как использовать локальный кэш файловой системы HDFS {#how-to-use-local-cache-for-hdfs-filesystem} Мы настоятельно рекомендуем включать локальное кэширование для удалённых файловых систем. Тесты производительности показывают, что при использовании кэша работа почти в 2 раза быстрее. @@ -75,9 +71,9 @@ PARTITION BY expr * limit_size: Обязательный параметр. Максимальный размер (в байтах) файлов локального кэша. * bytes_read_before_flush: Объём данных (в байтах), который будет прочитан перед сбросом на локальную файловую систему при загрузке файла из удалённой файловой системы. Значение по умолчанию — 1 МБ. -### Запрос к таблице Hive с форматом ввода ORC +### Запрос к таблице Hive с форматом ввода ORC {#query-hive-table-with-orc-input-format} -#### Создание таблицы в Hive +#### Создание таблицы в Hive {#create-table-in-hive} ```text hive > CREATE TABLE `test`.`test_orc`( @@ -125,8 +121,7 @@ OK Time taken: 0.295 seconds, Fetched: 1 row(s) ``` -#### Создание таблицы в ClickHouse - +#### Создание таблицы в ClickHouse {#create-table-in-clickhouse} Таблица в ClickHouse, которая получает данные из созданной выше таблицы Hive: @@ -199,9 +194,9 @@ day: 2021-09-18 1 rows in set. Elapsed: 0.078 sec. ``` -### Выполнение запроса к таблице Hive с форматом ввода Parquet +### Выполнение запроса к таблице Hive с форматом ввода Parquet {#query-hive-table-with-parquet-input-format} -#### Создание таблицы в Hive +#### Создание таблицы в Hive {#create-table-in-hive-1} ```text hive > @@ -241,7 +236,6 @@ OK Затрачено времени: 0,51 секунд ``` - hive > insert into test.test_parquet partition(day='2021-09-18') select 1, 2, 3, 4, 5, 6.11, 7.22, 8.333, current_timestamp(), current_date(), 'hello world', 'hello world', 'hello world', true, 'hello world', array(1, 2, 3), array('hello world', 'hello world'), array(float(1.1), float(1.2)), array(array(1, 2), array(3, 4)), array(array('a', 'b'), array('c', 'd')), array(array(float(1.11), float(2.22)), array(float(3.33), float(4.44))); OK Затраченное время: 36.025 сек. @@ -329,7 +323,6 @@ day: 2021-09-18 #### Создание таблицы в Hive - ```text hive > CREATE TABLE `test`.`test_text`( @@ -377,7 +370,7 @@ OK Time taken: 0.624 seconds, Fetched: 1 row(s) ``` -#### Создание таблицы в ClickHouse +#### Создание таблицы в ClickHouse {#create-table-in-hive-2} Таблица в ClickHouse, получающая данные из таблицы Hive, созданной выше: @@ -416,7 +409,6 @@ SETTINGS input_format_skip_unknown_fields = 1, input_format_with_names_use_heade Query id: 55b79d35-56de-45b9-8be6-57282fbf1f44 ``` - Row 1: ────── f_tinyint: 1 diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md index 94cb2998385..310adfc77ab 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Табличный движок Hudi +# Табличный движок Hudi {#hudi-table-engine} Этот движок предоставляет доступ только для чтения к существующим таблицам Apache [Hudi](https://hudi.apache.org/) в Amazon S3. -## Создание таблицы +## Создание таблицы {#create-table} Имейте в виду, что таблица Hudi должна уже существовать в S3: эта команда не принимает DDL‑параметры для создания новой таблицы. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md index 19d3c619c59..edc030b9d9b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md @@ -23,7 +23,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#create-table} Обратите внимание, что таблица Iceberg должна уже существовать в хранилище: эта команда не принимает параметры DDL для создания новой таблицы. @@ -42,14 +42,14 @@ CREATE TABLE iceberg_table_local ``` -## Параметры движка +## Параметры движка {#engine-arguments} Описание аргументов соответствует описанию аргументов в движках `S3`, `AzureBlobStorage`, `HDFS` и `File` соответственно. `format` обозначает формат файлов с данными в таблице Iceberg. Параметры движка могут быть заданы с использованием [Named Collections](../../../operations/named-collections.md). -### Пример +### Пример {#example} ```sql CREATE TABLE iceberg_table ENGINE=IcebergS3('http://test.s3.amazonaws.com/clickhouse-bucket/test_table', 'test', 'test') @@ -105,7 +105,7 @@ ClickHouse поддерживает отсечение партиций в за -## Обработка таблиц с удалёнными строками +## Обработка таблиц с удалёнными строками {#deleted-rows} В настоящее время поддерживаются только таблицы Iceberg с [позиционными удалениями (position deletes)](https://iceberg.apache.org/spec/#position-delete-files). @@ -114,7 +114,7 @@ ClickHouse поддерживает отсечение партиций в за * [удаления по равенству (equality deletes)](https://iceberg.apache.org/spec/#equality-delete-files) * [векторы удаления (deletion vectors)](https://iceberg.apache.org/spec/#deletion-vectors) (введены в версии 3) -### Базовое использование +### Базовое использование {#basic-usage} ```sql SELECT * FROM example_table ORDER BY 1 @@ -128,7 +128,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 Примечание: Нельзя указывать параметры `iceberg_timestamp_ms` и `iceberg_snapshot_id` в одном запросе одновременно. -### Важные замечания +### Важные замечания {#important-considerations} * **Снапшоты** обычно создаются, когда: * В таблицу записываются новые данные @@ -136,11 +136,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * **Изменения схемы, как правило, не создают снапшоты** — это приводит к важным особенностям поведения при использовании механизма time travel для таблиц, которые претерпели эволюцию схемы. -### Примеры сценариев +### Примеры сценариев {#example-scenarios} Все сценарии приведены на Spark, потому что ClickHouse (CH) пока не поддерживает запись в таблицы Iceberg. -#### Сценарий 1: Изменения схемы без новых снапшотов +#### Сценарий 1: Изменения схемы без новых снапшотов {#scenario-1} Рассмотрим следующую последовательность операций: @@ -200,7 +200,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * В моменты ts1 и ts2: отображаются только исходные два столбца * В момент ts3: отображаются все три столбца, при этом для первой строки значение price равно NULL -#### Сценарий 2: Отличия между исторической и текущей схемами +#### Сценарий 2: Отличия между исторической и текущей схемами {#scenario-2} Запрос time travel, выполненный в текущий момент времени, может показать схему, отличающуюся от текущей схемы таблицы: @@ -243,7 +243,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 Это происходит потому, что `ALTER TABLE` не создает новый снимок, а для текущей таблицы Spark берет значение `schema_id` из последнего файла метаданных, а не из снимка. -#### Сценарий 3: Отличия между исторической и текущей схемами +#### Сценарий 3: Отличия между исторической и текущей схемами {#scenario-3} Второй момент заключается в том, что при выполнении операции time travel вы не можете получить состояние таблицы до того, как в неё были записаны какие-либо данные: @@ -265,11 +265,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 В ClickHouse поведение аналогично Spark. Вы можете мысленно заменить запросы Select в Spark на запросы Select в ClickHouse — и всё будет работать так же. -## Определение файла метаданных +## Определение файла метаданных {#metadata-file-resolution} При использовании движка таблиц `Iceberg` в ClickHouse системе необходимо найти корректный файл metadata.json, который описывает структуру таблицы Iceberg. Ниже описано, как работает этот процесс определения: -### Поиск кандидатов +### Поиск кандидатов {#candidate-search} 1. **Явное указание пути**: @@ -286,7 +286,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * Если ни один из вышеперечисленных параметров не задан, все файлы `.metadata.json` в директории `metadata` рассматриваются как кандидаты -### Выбор самого нового файла +### Выбор самого нового файла {#most-recent-file} После определения файлов-кандидатов по указанным правилам система решает, какой из них является самым новым: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md index 36d0a797d64..f05b4829174 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md @@ -7,48 +7,45 @@ title: 'Табличные движки для интеграций' doc_type: 'reference' --- +# Движки таблиц для интеграций {#table-engines-for-integrations} +ClickHouse предоставляет различные средства интеграции с внешними системами, включая движки таблиц. Как и для всех остальных движков таблиц, конфигурация выполняется с помощью запросов `CREATE TABLE` или `ALTER TABLE`. Затем, с точки зрения пользователя, настроенная интеграция выглядит как обычная таблица, но запросы к ней проксируются во внешнюю систему. Такая прозрачность выполнения запросов является одним из ключевых преимуществ этого подхода по сравнению с альтернативными методами интеграции, такими как словари или табличные функции, которые требуют использования специальных способов выполнения запросов при каждом обращении. -# Движки таблиц для интеграций - -ClickHouse предоставляет различные способы интеграции с внешними системами, включая движки таблиц. Как и для всех остальных движков таблиц, конфигурация выполняется с помощью запросов `CREATE TABLE` или `ALTER TABLE`. Затем, с точки зрения пользователя, настроенная интеграция выглядит как обычная таблица, но запросы к ней проксируются во внешнюю систему. Такое прозрачное выполнение запросов является одним из ключевых преимуществ этого подхода по сравнению с альтернативными методами интеграции, такими как словари или табличные функции, которые требуют использования специальных способов обращения при каждом запросе. - -{/* Таблица оглавления для этой страницы автоматически генерируется скриптом +{/* Таблица оглавления для этой страницы автоматически генерируется скриптом https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - на основе полей фронтматтера YAML: slug, description, title. + из полей YAML front matter: slug, description, title. - Если вы заметили ошибку, отредактируйте YML-фронтматтер самих страниц. + Если вы заметили ошибку, отредактируйте YAML front matter непосредственно в самих страницах. */ } - {/*AUTOGENERATED_START*/ } -| Страница | Описание | -| --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [Движок таблицы AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage) | Этот движок реализует интеграцию с экосистемой Azure Blob Storage. | -| [Табличный движок DeltaLake](/engines/table-engines/integrations/deltalake) | Этот движок предоставляет доступ только для чтения к существующим таблицам Delta Lake в Amazon S3. | -| [Движок таблицы EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) | Этот движок обеспечивает интеграцию ClickHouse с RocksDB | -| [Движок таблицы ExternalDistributed](/engines/table-engines/integrations/ExternalDistributed) | Движок `ExternalDistributed` позволяет выполнять запросы `SELECT` к данным, которые хранятся на удалённых серверах MySQL или PostgreSQL. Принимает движки MySQL или PostgreSQL в качестве аргумента, что позволяет использовать шардинг. | -| [Движок таблицы TimeSeries](/engines/table-engines/special/time_series) | Движок таблицы, хранящий временные ряды, то есть набор значений, связанных с временными метками и тегами (метками). | -| [Движок таблицы HDFS](/engines/table-engines/integrations/hdfs) | Этот движок обеспечивает интеграцию с экосистемой Apache Hadoop, позволяя управлять данными в HDFS из ClickHouse. Он похож на движки File и URL, но предоставляет специализированные для Hadoop возможности. | -| [Движок таблиц Hive](/engines/table-engines/integrations/hive) | Движок Hive позволяет выполнять запросы `SELECT` по таблице Hive в HDFS. | -| [Движок таблиц Hudi](/engines/table-engines/integrations/hudi) | Этот движок обеспечивает доступ только для чтения к существующим таблицам Apache Hudi в Amazon S3. | -| [Табличный движок Iceberg](/engines/table-engines/integrations/iceberg) | Этот движок обеспечивает доступную только для чтения интеграцию с существующими таблицами Apache Iceberg в Amazon S3, Azure, HDFS, а также с локально хранящимися таблицами. | -| [Табличный движок JDBC](/engines/table-engines/integrations/jdbc) | Позволяет ClickHouse подключаться к внешним базам данных посредством JDBC. | -| [Движок таблиц Kafka](/engines/table-engines/integrations/kafka) | Движок таблиц Kafka можно использовать для работы с Apache Kafka; он позволяет публиковать данные и подписываться на потоки данных, организовывать отказоустойчивое хранение данных и обрабатывать потоки по мере их поступления. | -| [Движок таблиц MaterializedPostgreSQL](/engines/table-engines/integrations/materialized-postgresql) | Создаёт таблицу ClickHouse, заполняя её начальными данными из дампа таблицы PostgreSQL, и запускает процесс репликации. | -| [Табличный движок MongoDB](/engines/table-engines/integrations/mongodb) | Движок MongoDB — это табличный движок только для чтения, который позволяет читать данные из удалённой коллекции. | -| [Движок таблиц MySQL](/engines/table-engines/integrations/mysql) | Документация по табличному движку MySQL | -| [Движок таблицы NATS](/engines/table-engines/integrations/nats) | Этот движок позволяет осуществлять интеграцию ClickHouse с NATS для публикации и подписки на сообщения по определённым темам, а также обработки новых сообщений по мере их появления. | -| [Движок таблицы ODBC](/engines/table-engines/integrations/odbc) | Позволяет ClickHouse подключаться к внешним базам данных по ODBC. | -| [Движок таблиц PostgreSQL](/engines/table-engines/integrations/postgresql) | Движок PostgreSQL позволяет выполнять запросы `SELECT` и `INSERT` к данным, хранящимся на удалённом сервере PostgreSQL. | -| [Движок таблиц RabbitMQ](/engines/table-engines/integrations/rabbitmq) | Этот движок позволяет интегрировать ClickHouse с RabbitMQ. | -| [Табличный движок Redis](/engines/table-engines/integrations/redis) | Этот движок позволяет интегрировать ClickHouse с Redis. | -| [Движок таблицы S3](/engines/table-engines/integrations/s3) | Этот движок обеспечивает интеграцию с экосистемой Amazon S3. Аналогичен движку HDFS, но поддерживает функции, специфичные для S3. | -| [Движок таблицы S3Queue](/engines/table-engines/integrations/s3queue) | Этот движок обеспечивает интеграцию с экосистемой Amazon S3 и позволяет выполнять потоковый импорт. Аналогичен движкам Kafka и RabbitMQ, но предоставляет возможности, специфичные для S3. | -| [Движок таблицы AzureQueue](/engines/table-engines/integrations/azure-queue) | Этот движок интегрируется с экосистемой Azure Blob Storage и поддерживает потоковый импорт данных. | -| [Движок таблиц YTsaurus](/engines/table-engines/integrations/ytsaurus) | Табличный движок для импорта данных из кластера YTsaurus. | -| [Движок таблиц SQLite](/engines/table-engines/integrations/sqlite) | Движок позволяет импортировать данные в SQLite и экспортировать их из него, а также поддерживает выполнение запросов к таблицам SQLite напрямую из ClickHouse. | -| [Движок таблицы ArrowFlight](/engines/table-engines/integrations/arrowflight) | Движок позволяет выполнять запросы к удалённым наборам данных по протоколу Apache Arrow Flight. | +| Страница | Описание | +| ---------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [Табличный движок AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage) | Этот движок предоставляет интеграцию с экосистемой Azure Blob Storage. | +| [Табличный движок DeltaLake](/engines/table-engines/integrations/deltalake) | Этот движок предоставляет интеграцию только для чтения с существующими таблицами Delta Lake в Amazon S3. | +| [табличный движок EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) | Этот движок позволяет интегрировать ClickHouse с RocksDB. | +| [Движок таблицы ExternalDistributed](/engines/table-engines/integrations/ExternalDistributed) | Движок `ExternalDistributed` позволяет выполнять запросы `SELECT` к данным, хранящимся на удалённых серверах с MySQL или PostgreSQL. Принимает движки MySQL или PostgreSQL в качестве аргумента, что позволяет организовать сегментацию данных. | +| [Табличный движок TimeSeries](/engines/table-engines/special/time_series) | Движок таблицы, предназначенный для хранения временных рядов — набора значений, привязанных к временным меткам и тегам (или меткам). | +| [Табличный движок HDFS](/engines/table-engines/integrations/hdfs) | Этот движок обеспечивает интеграцию с экосистемой Apache Hadoop, позволяя управлять данными в HDFS из ClickHouse. Он подобен движкам File и URL, но предоставляет функции, специфичные для Hadoop. | +| [Табличный движок Hive](/engines/table-engines/integrations/hive) | Движок Hive позволяет выполнять запросы `SELECT` по таблице Hive, размещённой в HDFS. | +| [Движок таблицы Hudi](/engines/table-engines/integrations/hudi) | Этот движок обеспечивает интеграцию только для чтения с существующими таблицами Apache Hudi в Amazon S3. | +| [Движок таблицы Iceberg](/engines/table-engines/integrations/iceberg) | Этот движок обеспечивает интеграцию только для чтения с существующими таблицами Apache Iceberg, размещёнными в Amazon S3, Azure, HDFS и в локальном хранилище. | +| [Табличный движок JDBC](/engines/table-engines/integrations/jdbc) | Позволяет ClickHouse подключаться к внешним базам данных через JDBC. | +| [Движок таблицы Kafka](/engines/table-engines/integrations/kafka) | Движок таблиц Kafka (Kafka Table Engine) можно использовать для работы с Apache Kafka. Он позволяет публиковать и получать данные (подписываться на потоки), организовывать отказоустойчивое хранилище и обрабатывать потоки по мере их поступления. | +| [Движок таблицы MaterializedPostgreSQL](/engines/table-engines/integrations/materialized-postgresql) | Создаёт таблицу ClickHouse с первоначальным дампом данных таблицы PostgreSQL и запускает процесс репликации. | +| [Табличный движок MongoDB](/engines/table-engines/integrations/mongodb) | Движок MongoDB — это движок таблиц, предназначенный только для чтения и позволяющий считывать данные из удалённой коллекции. | +| [Табличный движок MySQL](/engines/table-engines/integrations/mysql) | Документация по табличному движку MySQL | +| [Табличный движок NATS](/engines/table-engines/integrations/nats) | Этот движок позволяет интегрировать ClickHouse с NATS для публикации сообщений, подписки на их темы и обработки новых сообщений по мере их поступления. | +| [Табличный движок ODBC](/engines/table-engines/integrations/odbc) | Позволяет ClickHouse подключаться к внешним базам данных через ODBC. | +| [Движок таблицы PostgreSQL](/engines/table-engines/integrations/postgresql) | Движок PostgreSQL позволяет выполнять запросы `SELECT` и `INSERT` к данным, хранящимся на удалённом сервере PostgreSQL. | +| [Движок таблиц RabbitMQ](/engines/table-engines/integrations/rabbitmq) | Этот движок позволяет интегрировать ClickHouse с RabbitMQ. | +| [Табличный движок Redis](/engines/table-engines/integrations/redis) | Этот движок обеспечивает интеграцию ClickHouse с Redis. | +| [табличный движок S3](/engines/table-engines/integrations/s3) | Этот движок обеспечивает интеграцию с экосистемой Amazon S3. Аналогичен движку HDFS, но предоставляет функции, специфичные для S3. | +| [Движок таблицы AzureQueue](/engines/table-engines/integrations/azure-queue) | Этот движок предоставляет интеграцию с экосистемой Azure Blob Storage и позволяет выполнять потоковый импорт данных. | +| [Табличный движок S3Queue](/engines/table-engines/integrations/s3queue) | Этот движок обеспечивает интеграцию с экосистемой Amazon S3 и позволяет выполнять потоковый импорт данных. Аналогичен движкам Kafka и RabbitMQ, но поддерживает специфичные для S3 возможности. | +| [Движок таблицы SQLite](/engines/table-engines/integrations/sqlite) | Этот движок позволяет импортировать и экспортировать данные в/из SQLite и поддерживает выполнение запросов к таблицам SQLite непосредственно из ClickHouse. | +| [Табличный движок YTsaurus](/engines/table-engines/integrations/ytsaurus) | Движок таблицы, позволяющий импортировать данные из кластера YTsaurus. | +| [Табличный движок ArrowFlight](/engines/table-engines/integrations/arrowflight) | Движок позволяет выполнять запросы к удалённым наборам данных по протоколу Apache Arrow Flight. | {/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md index ab2de4f7cf0..7787178e62e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы JDBC +# Движок таблицы JDBC {#jdbc-table-engine} @@ -27,7 +27,7 @@ ClickHouse рекомендует использовать встроенные -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]имя_таблицы @@ -51,7 +51,7 @@ ENGINE = JDBC(источник_данных, внешняя_база_данны * Эти параметры также можно передавать с использованием [именованных коллекций](operations/named-collections.md). -## Пример использования +## Пример использования {#usage-example} Создание таблицы на сервере MySQL, подключившись к нему напрямую через консольный клиент: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md index 5f5d9e4280f..ea9e047be4b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md @@ -11,7 +11,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Табличный движок Kafka +# Табличный движок Kafka {#kafka-table-engine} :::tip Если вы используете ClickHouse Cloud, мы рекомендуем вместо этого использовать [ClickPipes](/integrations/clickpipes). ClickPipes нативно поддерживает приватные сетевые соединения, независимое масштабирование ресурсов для приёма данных и ресурсов кластера, а также всесторонний мониторинг при потоковой загрузке данных из Kafka в ClickHouse. @@ -21,7 +21,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - Организовывать отказоустойчивое хранилище. - Обрабатывать потоки по мере их поступления. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -135,7 +135,7 @@ Examples: ::: -## Описание +## Описание {#description} Доставленные сообщения отслеживаются автоматически, поэтому каждое сообщение в группе учитывается только один раз. Если вам нужно получить данные дважды, создайте копию таблицы с другим именем группы. @@ -186,7 +186,7 @@ Examples: Если вы хотите изменить целевую таблицу с помощью `ALTER`, рекомендуем отключить материализованное представление, чтобы избежать расхождений между целевой таблицей и данными из представления. -## Конфигурация +## Конфигурация {#configuration} Как и движок GraphiteMergeTree, движок Kafka поддерживает расширенную конфигурацию с использованием файла конфигурации ClickHouse. Вы можете использовать два ключа конфигурации: глобальный (в секции ``) и на уровне топика (в секции ``). Сначала применяется глобальная конфигурация, а затем — конфигурация для конкретного топика (если она задана). @@ -233,7 +233,7 @@ Examples: Список доступных параметров конфигурации приведён в [справочнике по конфигурации librdkafka](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md). Используйте символ подчёркивания (`_`) вместо точки в конфигурации ClickHouse. Например, `check.crcs=true` будет записано как `true`. -### Поддержка Kerberos +### Поддержка Kerberos {#kafka-kerberos-support} Для работы с Kafka с поддержкой Kerberos добавьте дочерний элемент `security_protocol` со значением `sasl_plaintext`. Этого достаточно при наличии действительного Kerberos ticket-granting ticket (TGT), уже полученного и закэшированного средствами операционной системы. ClickHouse может самостоятельно поддерживать учетные данные Kerberos с использованием файла keytab. Для этого используйте дочерние элементы `sasl_kerberos_service_name`, `sasl_kerberos_keytab` и `sasl_kerberos_principal`. @@ -276,7 +276,7 @@ ClickHouse может самостоятельно поддерживать уч - Для построчных форматов количество строк в одном сообщении Kafka можно контролировать с помощью настройки `kafka_max_rows_per_message`. - Для блочных форматов мы не можем разделить блок на более мелкие части, но количество строк в одном блоке можно контролировать с помощью общего параметра настройки [max_block_size](/operations/settings/settings#max_block_size). -## Движок для хранения зафиксированных смещений в ClickHouse Keeper +## Движок для хранения зафиксированных смещений в ClickHouse Keeper {#engine-to-store-committed-offsets-in-clickhouse-keeper} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md index d2005238932..844e25794d8 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы MaterializedPostgreSQL +# Движок таблицы MaterializedPostgreSQL {#materializedpostgresql-table-engine} @@ -35,7 +35,7 @@ SET allow_experimental_materialized_postgresql_table=1 Если требуется более одной таблицы, настоятельно рекомендуется использовать движок базы данных [MaterializedPostgreSQL](../../../engines/database-engines/materialized-postgresql.md) вместо движка таблицы и настройку `materialized_postgresql_tables_list`, которая задаёт список реплицируемых таблиц (также можно будет указать `schema` базы данных). Такой подход значительно лучше с точки зрения нагрузки на CPU, количества подключений и числа слотов репликации в удалённой базе данных PostgreSQL. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE postgresql_db.postgresql_replica (key UInt64, value UInt64) @@ -64,7 +64,7 @@ PRIMARY KEY key; -## Виртуальные столбцы +## Виртуальные столбцы {#virtual-columns} * `_version` — счётчик транзакций. Тип: [UInt64](../../../sql-reference/data-types/int-uint.md). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md index 75fee13a8d4..777959f80f4 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Табличный движок MongoDB +# Табличный движок MongoDB {#mongodb-table-engine} Табличный движок MongoDB — это движок только для чтения, который позволяет читать данные из удалённой коллекции [MongoDB](https://www.mongodb.com/). @@ -18,7 +18,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -61,7 +61,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); | `oid_columns` | Список имён столбцов, разделённых запятыми, которые в предложении WHERE должны интерпретироваться как `oid`. По умолчанию — `_id`. | -## Сопоставление типов +## Сопоставление типов {#types-mappings} | MongoDB | ClickHouse | | ----------------------- | ------------------------------------------------------------------------------------ | @@ -78,7 +78,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); Если ключ не найден в документе MongoDB (например, имя столбца не совпадает), будет вставлено значение по умолчанию или `NULL` (если столбец допускает значения `NULL`). -### OID +### OID {#oid} Если вы хотите, чтобы `String` обрабатывался как `oid` в условии WHERE, просто укажите имя столбца в последнем аргументе движка таблицы. Это может понадобиться при выборке записи по столбцу `_id`, который по умолчанию имеет тип `oid` в MongoDB. @@ -132,7 +132,7 @@ SELECT count() FROM sample_oid WHERE another_oid_column = '67bf6cc40000000000ea4 ``` -## Поддерживаемые предложения +## Поддерживаемые предложения {#supported-clauses} Поддерживаются только запросы с простыми выражениями (например, `WHERE field = ORDER BY field2 LIMIT `). Такие выражения переводятся в язык запросов MongoDB и выполняются на стороне сервера. @@ -158,7 +158,7 @@ SELECT * FROM mongo_table WHERE date = '2024-01-01'::Date OR date = toDate('2024 ::: -## Пример использования +## Пример использования {#usage-example} Предположим, что в MongoDB загружен набор данных [sample_mflix](https://www.mongodb.com/docs/atlas/sample-data/sample-mflix). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md index c9414d2e3e4..df651240b10 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Движок таблицы MySQL +# Движок таблицы MySQL {#mysql-table-engine} Движок MySQL позволяет выполнять запросы `SELECT` и `INSERT` к данным, которые хранятся на удалённом сервере MySQL. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -67,7 +67,7 @@ CREATE TABLE test_replicas (id UInt32, name String, age UInt32, money UInt32) EN ``` -## Пример использования +## Пример использования {#usage-example} Создайте таблицу в MySQL: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md index be0abe14840..3d9d263fdf7 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md @@ -21,7 +21,7 @@ doc_type: 'guide' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -134,7 +134,7 @@ SSL-подключение: ``` -## Описание +## Описание {#description} `SELECT` не особенно полезен для чтения сообщений (кроме отладки), потому что каждое сообщение может быть прочитано только один раз. Гораздо практичнее создавать потоки в реальном времени с помощью [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: @@ -199,7 +199,7 @@ SSL-подключение: -## Использование JetStream +## Использование JetStream {#using-jetstream} Прежде чем использовать движок NATS с NATS JetStream, необходимо создать поток NATS (stream) и устойчивого (durable) pull‑consumer'а. Для этого можно использовать, например, утилиту `nats` из пакета [NATS CLI](https://github.com/nats-io/natscli): diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md index a2e2691350e..fe72c04a0f0 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы ODBC +# Движок таблицы ODBC {#odbc-table-engine} @@ -22,7 +22,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -51,7 +51,7 @@ ENGINE = ODBC(источник_данных, внешняя_база_данны Эти параметры также можно передавать с помощью [именованных коллекций](operations/named-collections.md). -## Пример использования +## Пример использования {#usage-example} **Получение данных из локального экземпляра MySQL через ODBC** diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md index 0258bcab40f..a2085483fb1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Движок таблиц PostgreSQL +# Движок таблиц PostgreSQL {#postgresql-table-engine} Движок PostgreSQL позволяет выполнять запросы `SELECT` и `INSERT` к данным, хранящимся на удалённом сервере PostgreSQL. @@ -23,7 +23,7 @@ doc_type: 'guide' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -73,7 +73,7 @@ SELECT * FROM postgresql(postgres_creds, table='table1'); ``` -## Особенности реализации +## Особенности реализации {#implementation-details} Запросы `SELECT` на стороне PostgreSQL выполняются как `COPY (SELECT ...) TO STDOUT` внутри транзакции PostgreSQL только для чтения с фиксацией (commit) после каждого запроса `SELECT`. @@ -121,9 +121,9 @@ CREATE TABLE test_replicas (id UInt32, name String) ENGINE = PostgreSQL(`postgre ``` -## Пример использования +## Пример использования {#usage-example} -### Таблица в PostgreSQL +### Таблица в PostgreSQL {#table-in-postgresql} ```text postgres=# CREATE TABLE "public"."test" ( @@ -146,7 +146,7 @@ postgresql> SELECT * FROM test; (1 row) ``` -### Создание таблицы в ClickHouse и подключение к таблице PostgreSQL, созданной выше +### Создание таблицы в ClickHouse и подключение к таблице PostgreSQL, созданной выше {#creating-table-in-clickhouse-and-connecting-to--postgresql-table-created-above} В этом примере используется [движок таблицы PostgreSQL](/engines/table-engines/integrations/postgresql.md) для подключения таблицы ClickHouse к таблице PostgreSQL и выполнения операторов SELECT и INSERT над базой данных PostgreSQL: @@ -160,7 +160,7 @@ CREATE TABLE default.postgresql_table ENGINE = PostgreSQL('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### Вставка начальных данных из таблицы PostgreSQL в таблицу ClickHouse с использованием запроса SELECT +### Вставка начальных данных из таблицы PostgreSQL в таблицу ClickHouse с использованием запроса SELECT {#inserting-initial-data-from-postgresql-table-into-clickhouse-table-using-a-select-query} [Табличная функция postgresql](/sql-reference/table-functions/postgresql.md) копирует данные из PostgreSQL в ClickHouse. Её часто используют для повышения производительности запросов за счёт выполнения запросов и аналитики в ClickHouse, а не в PostgreSQL, а также для миграции данных из PostgreSQL в ClickHouse. Поскольку мы будем копировать данные из PostgreSQL в ClickHouse, мы используем в ClickHouse табличный движок MergeTree и назовём таблицу postgresql_copy: @@ -180,7 +180,7 @@ INSERT INTO default.postgresql_copy SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### Вставка инкрементальных данных из таблицы PostgreSQL в таблицу ClickHouse +### Вставка инкрементальных данных из таблицы PostgreSQL в таблицу ClickHouse {#inserting-incremental-data-from-postgresql-table-into-clickhouse-table} Если после первоначальной вставки вы выполняете дальнейшую синхронизацию между таблицей PostgreSQL и таблицей ClickHouse, вы можете использовать предложение WHERE в ClickHouse, чтобы вставлять только данные, добавленные в PostgreSQL, на основе метки времени или уникального последовательного идентификатора. @@ -198,7 +198,7 @@ SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postges_user', 'po WHERE int_id > maxIntID; ``` -### Выбор данных из полученной таблицы ClickHouse +### Выбор данных из полученной таблицы ClickHouse {#selecting-data-from-the-resulting-clickhouse-table} ```sql SELECT * FROM postgresql_copy WHERE str IN ('test'); @@ -210,7 +210,7 @@ SELECT * FROM postgresql_copy WHERE str IN ('test'); └────────────────┴──────┴────────┘ ``` -### Использование схемы, отличной от схемы по умолчанию +### Использование схемы, отличной от схемы по умолчанию {#using-non-default-schema} ```text postgres=# CREATE SCHEMA "nice.schema"; diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md index 357253a9623..b5ca214728c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Табличный движок RabbitMQ +# Табличный движок RabbitMQ {#rabbitmq-table-engine} Этот движок позволяет интегрировать ClickHouse с [RabbitMQ](https://www.rabbitmq.com). @@ -20,7 +20,7 @@ doc_type: 'guide' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -131,7 +131,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ``` -## Описание +## Описание {#description} `SELECT` не особенно полезен для чтения сообщений (кроме отладки), потому что каждое сообщение можно прочитать только один раз. Гораздо практичнее создавать потоки в реальном времени с помощью [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md index 37de63a250a..8e58eaa31ea 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md @@ -9,13 +9,13 @@ doc_type: 'guide' -# Табличный движок Redis +# Табличный движок Redis {#redis-table-engine} Этот движок позволяет интегрировать ClickHouse с [Redis](https://redis.io/). Поскольку Redis использует модель ключ–значение (kv), настоятельно рекомендуется выполнять только точечные запросы, например `where k=xx` или `where k in (xx, xx)`. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -47,7 +47,7 @@ PRIMARY KEY(primary_key_name); ::: -## Пример использования +## Пример использования {#usage-example} Создайте таблицу в ClickHouse с движком `Redis`, явно указав аргументы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md index 7b21b4b5543..fe5d8bbfac6 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md @@ -10,13 +10,13 @@ doc_type: 'reference' -# Движок таблицы S3 +# Движок таблицы S3 {#s3-table-engine} Этот движок обеспечивает интеграцию с экосистемой [Amazon S3](https://aws.amazon.com/s3/). Он похож на движок [HDFS](/engines/table-engines/integrations/hdfs), но поддерживает специфичные для S3 возможности. -## Пример +## Пример {#example} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -36,7 +36,7 @@ SELECT * FROM s3_engine_table LIMIT 2; ``` -## Создайте таблицу +## Создайте таблицу {#creating-a-table} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -45,7 +45,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) [SETTINGS ...] ``` -### Параметры движка +### Параметры движка {#parameters} * `path` — URL бакета с путем к файлу. В режиме только для чтения поддерживаются следующие шаблоны: `*`, `**`, `?`, `{abc,def}` и `{N..M}`, где `N`, `M` — числа, `'abc'`, `'def'` — строки. Дополнительную информацию см. [ниже](#wildcards-in-path). * `NOSIGN` — Если это ключевое слово указано вместо учетных данных, все запросы не будут подписываться. @@ -56,7 +56,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) * `partition_columns_in_data_file` — Используется только со стратегией партиционирования `HIVE`. Указывает ClickHouse, следует ли ожидать, что столбцы партиционирования будут записаны в файл данных. По умолчанию `false`. * `storage_class_name` — Варианты: `STANDARD` или `INTELLIGENT_TIERING`, позволяют указать [AWS S3 Intelligent Tiering](https://aws.amazon.com/s3/storage-classes/intelligent-tiering/). -### Кэш данных +### Кэш данных {#data-cache} Движок таблиц `S3` поддерживает кэширование данных на локальном диске. Параметры конфигурации файлового кэша и примеры использования приведены в этом [разделе](/operations/storing-data.md/#using-local-cache). @@ -87,13 +87,13 @@ SETTINGS filesystem_cache_name = 'cache_for_s3', enable_filesystem_cache = 1; 2. повторно используйте конфигурацию кеша (и, следовательно, хранилище кеша) из секции ClickHouse `storage_configuration`, [описанной здесь](/operations/storing-data.md/#using-local-cache) -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — необязательный параметр. В большинстве случаев вам не нужен ключ партиционирования, а если он все же требуется, как правило, не нужен ключ с более мелкой детализацией, чем по месяцам. Партиционирование не ускоряет запросы (в отличие от выражения ORDER BY). Никогда не используйте слишком мелкое партиционирование. Не делите данные на партиции по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — это столбец с датой типа [Date](/sql-reference/data-types/date.md). Имена партиций здесь имеют формат `"YYYYMM"`. -#### Стратегия партиционирования +#### Стратегия партиционирования {#partition-strategy} `WILDCARD` (по умолчанию): заменяет подстановочный шаблон `{_partition_id}` в пути к файлу фактическим ключом партиционирования. Чтение не поддерживается. @@ -140,7 +140,7 @@ arthur :) select _path, * from t_03363_parquet; └────────────────────────────────────────────────────────────────────────────┴──────┴─────────┴─────────┘ ``` -### Запросы к секционированным данным +### Запросы к секционированным данным {#querying-partitioned-data} В этом примере используется [рецепт для docker compose](https://github.com/ClickHouse/examples/tree/5fdc6ff72f4e5137e23ea075c88d3f44b0202490/docker-compose-recipes/recipes/ch-and-minio-S3), который интегрирует ClickHouse и MinIO. Вы сможете воспроизвести те же запросы, используя S3, заменив endpoint и значения аутентификации. @@ -160,7 +160,7 @@ Cloud). Поскольку наборы данных ClickHouse часто оч данных частями, то есть использовать партиционированную запись. ::: -#### Создайте таблицу +#### Создайте таблицу {#create-the-table} ```sql CREATE TABLE p @@ -178,13 +178,13 @@ ENGINE = S3( PARTITION BY column3 ``` -#### Вставка данных +#### Вставка данных {#insert-data} ```sql INSERT INTO p VALUES (1, 2, 3), (3, 2, 1), (78, 43, 45) ``` -#### Выборка из партиции 3 +#### Выборка из партиции 3 {#select-from-partition-3} :::tip Этот запрос использует табличную функцию s3 @@ -201,7 +201,7 @@ FROM s3('http://minio:10000/clickhouse//test_3.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### Выбор данных из партиции 1 +#### Выбор данных из партиции 1 {#select-from-partition-1} ```sql SELECT * @@ -214,7 +214,7 @@ FROM s3('http://minio:10000/clickhouse//test_1.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### Выбор из партиции 45 +#### Выбор из партиции 45 {#select-from-partition-45} ```sql SELECT * @@ -227,7 +227,7 @@ FROM s3('http://minio:10000/clickhouse//test_45.csv', 'minioadmin', 'minioadminp └────┴────┴────┘ ``` -#### Ограничение +#### Ограничение {#limitation} Логично попробовать выполнить `Select * from p`, но, как отмечалось выше, этот запрос завершится с ошибкой; используйте приведённый выше запрос. @@ -274,7 +274,7 @@ SELECT * FROM p -## Подстановочные шаблоны в пути +## Подстановочные шаблоны в пути {#wildcards-in-path} Аргумент `path` может задавать несколько файлов, используя подстановочные шаблоны в стиле bash. Для обработки файл должен существовать и полностью совпадать с шаблоном пути. Список файлов определяется во время выполнения `SELECT` (а не в момент `CREATE`). @@ -362,7 +362,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) -## Параметры для отдельных endpoint +## Параметры для отдельных endpoint {#endpoint-settings} Следующие параметры могут быть указаны в конфигурационном файле для заданного endpoint (который будет сопоставляться по точному префиксу URL): @@ -406,7 +406,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ``` -## Работа с архивами +## Работа с архивами {#working-with-archives} Предположим, что у нас есть несколько файлов-архивов со следующими URI в S3: @@ -432,7 +432,7 @@ TAR ::: -## Доступ к общедоступным бакетам +## Доступ к общедоступным бакетам {#accessing-public-buckets} ClickHouse пытается получить учетные данные из множества различных источников. Иногда это может приводить к проблемам при доступе к некоторым общедоступным бакетам, в результате чего клиент возвращает код ошибки `403`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md index 79fb3196b4f..b8090eb3032 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md @@ -1,7 +1,7 @@ --- -description: 'Этот движок обеспечивает интеграцию с экосистемой Amazon S3 и поддерживает - потоковый импорт данных. Аналогичен движкам Kafka и RabbitMQ, но предоставляет функции, - специфичные для S3.' +description: 'Этот движок обеспечивает интеграцию с экосистемой Amazon S3 и позволяет + выполнять потоковый импорт. Аналогичен движкам Kafka и RabbitMQ, но предоставляет + специфичные для S3 возможности.' sidebar_label: 'S3Queue' sidebar_position: 181 slug: /engines/table-engines/integrations/s3queue @@ -11,16 +11,13 @@ doc_type: 'reference' import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge' +# Движок таблицы S3Queue {#s3queue-table-engine} -# Движок таблицы S3Queue +Этот движок обеспечивает интеграцию с экосистемой [Amazon S3](https://aws.amazon.com/s3/) и позволяет выполнять потоковый импорт. Он аналогичен движкам [Kafka](../../../engines/table-engines/integrations/kafka.md) и [RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md), но предоставляет функции, специфичные для S3. -Этот движок обеспечивает интеграцию с экосистемой [Amazon S3](https://aws.amazon.com/s3/) и поддерживает потоковый импорт. Движок аналогичен движкам [Kafka](../../../engines/table-engines/integrations/kafka.md) и [RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md), но предоставляет возможности, специфичные для S3. +Важно учитывать следующее примечание из [исходного PR по реализации S3Queue](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183): когда к таблице с этим движком присоединяется `MATERIALIZED VIEW`, движок таблицы S3Queue начинает собирать данные в фоновом режиме. -Важно учитывать следующее замечание из [оригинального PR с реализацией S3Queue](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183): когда к этому движку подключается `MATERIALIZED VIEW`, движок таблицы S3Queue начинает собирать данные в фоновом режиме. - - - -## Создание таблицы {#creating-a-table} +## Создать таблицу {#creating-a-table} ```sql CREATE TABLE s3_queue_engine_table (name String, value UInt32) @@ -51,12 +48,12 @@ CREATE TABLE s3_queue_engine_table (name String, value UInt32) ``` :::warning -До версии `24.7` необходимо использовать префикс `s3queue_` для всех настроек, за исключением `mode`, `after_processing` и `keeper_path`. +До версии `24.7` требуется использовать префикс `s3queue_` для всех настроек, кроме `mode`, `after_processing` и `keeper_path`. ::: **Параметры движка** -Параметры `S3Queue` совпадают с параметрами табличного движка `S3`. См. раздел о параметрах [здесь](../../../engines/table-engines/integrations/s3.md#parameters). +Параметры `S3Queue` такие же, как у табличного движка `S3`. См. раздел «Параметры» [здесь](../../../engines/table-engines/integrations/s3.md#parameters). **Пример** @@ -67,7 +64,7 @@ SETTINGS mode = 'unordered'; ``` -Использование именованных коллекций: +Использование коллекций с именами: ```xml @@ -88,164 +85,256 @@ SETTINGS mode = 'ordered'; ``` - ## Настройки {#settings} -Чтобы получить список настроек, настроенных для таблицы, используйте таблицу `system.s3_queue_settings`. Доступно начиная с версии `24.10`. +Чтобы получить список настроек, заданных для таблицы, используйте таблицу `system.s3_queue_settings`. Доступно, начиная с версии `24.10`. ### Mode {#mode} Возможные значения: -- unordered — В неупорядоченном режиме набор всех уже обработанных файлов отслеживается с помощью постоянных узлов в ZooKeeper. -- ordered — В упорядоченном режиме файлы обрабатываются в лексикографическом порядке. Это означает, что если файл с именем 'BBB' был обработан в какой-то момент, а позже в бакет добавляется файл с именем 'AA', он будет проигнорирован. В ZooKeeper хранятся только максимальное имя (в лексикографическом смысле) успешно обработанного файла и имена файлов, для которых будет выполнена повторная попытка после неудачной загрузки. +* unordered — В режиме unordered множество всех уже обработанных файлов отслеживается с помощью постоянных узлов в ZooKeeper. +* ordered — В режиме ordered файлы обрабатываются в лексикографическом порядке. Это означает, что если файл с именем `BBB` был обработан в какой‑то момент, а позже в бакет был добавлен файл с именем `AA`, он будет проигнорирован. В ZooKeeper сохраняются только максимальное имя (в лексикографическом смысле) успешно обработанного файла и имена файлов, которые будут повторно загружены после неудачной попытки загрузки. + +Значение по умолчанию: `ordered` в версиях до 24.6. Начиная с 24.6 значение по умолчанию отсутствует, настройку требуется указывать вручную. Для таблиц, созданных в более ранних версиях, значение по умолчанию останется `ordered` для сохранения совместимости. -Значение по умолчанию: `ordered` в версиях до 24.6. Начиная с версии 24.6 значение по умолчанию отсутствует, настройку необходимо указывать вручную. Для таблиц, созданных в более ранних версиях, значение по умолчанию останется `Ordered` для обеспечения совместимости. +### `after_processing` {#after_processing} -### `after_processing` {#after_processing} +Что делать с файлом после успешной обработки. -Удалить или сохранить файл после успешной обработки. Возможные значения: -- keep. -- delete. +* keep. +* delete. +* move. +* tag. Значение по умолчанию: `keep`. -### `keeper_path` {#keeper_path} +Для варианта `move` требуются дополнительные настройки. В случае перемещения в пределах того же бакета необходимо указать новый префикс пути в параметре `after_processing_move_prefix`. + +Перемещение в другой S3‑бакет требует указания URI целевого бакета в параметре `after_processing_move_uri`, а также учетных данных доступа к S3 в параметрах `after_processing_move_access_key_id` и `after_processing_move_secret_access_key`. + +Пример: + +```sql +CREATE TABLE s3queue_engine_table (name String, value UInt32) +ENGINE=S3Queue('https://clickhouse-public-datasets.s3.amazonaws.com/my-test-bucket-768/*', 'CSV', 'gzip') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_retries = 20, + after_processing_move_prefix = 'dst_prefix', + after_processing_move_uri = 'https://clickhouse-public-datasets.s3.amazonaws.com/dst-bucket', + after_processing_move_access_key_id = 'test', + after_processing_move_secret_access_key = 'test'; +``` + +Для перемещения данных из одного контейнера Azure в другой необходимо указать строку подключения Blob Storage в параметре `after_processing_move_connection_string` и имя контейнера в параметре `after_processing_move_container`. См. [настройки AzureQueue](../../../engines/table-engines/integrations/azure-queue.md#settings). + +Для добавления тегов необходимо указать ключ и значение тега в параметрах `after_processing_tag_key` и `after_processing_tag_value`. + +### `after_processing_retries` {#after_processing_retries} + +Количество повторных попыток выполнения запрошенного действия постобработки перед тем, как прекратить попытки. + +Возможные значения: + +* Неотрицательное целое число. + +Значение по умолчанию: `10`. + +### `after_processing_move_access_key_id` {#after_processing_move_access_key_id} + +ID ключа доступа (Access Key ID) для S3‑бакета, в который нужно переместить успешно обработанные файлы, если целевым местом назначения является другой S3‑бакет. + +Возможные значения: + +* Строка. + +Значение по умолчанию: пустая строка. + +### `after_processing_move_prefix` {#after_processing_move_prefix} + +Префикс пути, в который перемещаются успешно обработанные файлы. Применимо как при перемещении в пределах того же бакета, так и в другой бакет. + +Возможные значения: + +* Строка. + +Значение по умолчанию: пустая строка. + +### `after_processing_move_secret_access_key` {#after_processing_move_secret_access_key} + +Secret Access Key для S3‑бакета, в который нужно перемещать успешно обработанные файлы, если целевой ресурс — другой S3‑бакет. + +Возможные значения: + +* Строка. + +Значение по умолчанию: пустая строка. + +### `after_processing_move_uri` {#after_processing_move_uri} + +URI S3-бакета, в который следует перемещать успешно обработанные файлы, если местом назначения является другой S3-бакет. + +Возможные значения: + +* Строка. + +Значение по умолчанию: пустая строка. + +### `after_processing_tag_key` {#after_processing_tag_key} + +Ключ тега, который будет использоваться для пометки успешно обработанных файлов, если `after_processing='tag'`. -Путь в ZooKeeper может быть указан как настройка движка таблицы, или путь по умолчанию может быть сформирован из пути, предоставленного глобальной конфигурацией, и UUID таблицы. Возможные значения: -- String. +* Строка. + +Значение по умолчанию: пустая строка. + +### `after_processing_tag_value` {#after_processing_tag_value} + +Значение тега, которое будет присвоено успешно обработанным файлам, если `after_processing='tag'`. + +Возможные значения: + +* Строка. + +Значение по умолчанию: пустая строка. + +### `keeper_path` {#keeper_path} + +Путь в ZooKeeper может быть задан в параметрах движка таблицы, либо путь по умолчанию может быть сформирован из пути, указанного в глобальной конфигурации, и UUID таблицы. +Возможные значения: + +* строка. Значение по умолчанию: `/`. -### `s3queue_loading_retries` {#loading_retries} +### `s3queue_loading_retries` {#loading_retries} -Повторять попытки загрузки файла до указанного количества раз. По умолчанию повторные попытки не выполняются. +Повторять загрузку файла до указанного количества раз. По умолчанию повторы не выполняются. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `0`. -### `s3queue_processing_threads_num` {#processing_threads_num} +### `s3queue_processing_threads_num` {#processing_threads_num} -Количество потоков для выполнения обработки. Применяется только для режима `Unordered`. +Количество потоков обработки. Применяется только в режиме `Unordered`. -Значение по умолчанию: количество процессоров или 16. +Значение по умолчанию: количество CPU или 16. -### `s3queue_parallel_inserts` {#parallel_inserts} +### `s3queue_parallel_inserts` {#parallel_inserts} -По умолчанию `processing_threads_num` создаёт один `INSERT`, поэтому загрузка файлов и парсинг выполняются только в нескольких потоках. -Но это ограничивает параллелизм, поэтому для лучшей пропускной способности используйте `parallel_inserts=true`, это позволит вставлять данные параллельно (но имейте в виду, что это приведёт к большему количеству сгенерированных кусков данных для семейства MergeTree). +По умолчанию `processing_threads_num` будет выполнять один `INSERT`, поэтому файлы будут только загружаться и парситься в несколько потоков. +Но это ограничивает степень параллелизма, поэтому для лучшей пропускной способности используйте `parallel_inserts=true` — это позволит вставлять данные параллельно (но имейте в виду, что это приведёт к большему количеству создаваемых частей данных для семейства движков MergeTree). :::note -`INSERT` будут создаваться с учётом настроек `max_process*_before_commit`. +`INSERT`-запросы будут создаваться с учётом настроек `max_process*_before_commit`. ::: Значение по умолчанию: `false`. -### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} +### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} -Включить логирование в `system.s3queue_log`. +Включает логирование в `system.s3queue_log`. Значение по умолчанию: `0`. -### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} +### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} -Указывает минимальное время в миллисекундах, которое ClickHouse ожидает перед выполнением следующей попытки опроса. +Указывает минимальное время в миллисекундах, которое ClickHouse ожидает перед следующей попыткой опроса. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `1000`. -### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} +### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} -Определяет максимальное время в миллисекундах, которое ClickHouse ожидает перед инициацией следующей попытки опроса. +Определяет максимальное время в миллисекундах, в течение которого ClickHouse ждёт перед запуском следующей попытки опроса. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `10000`. -### `s3queue_polling_backoff_ms` {#polling_backoff_ms} +### `s3queue_polling_backoff_ms` {#polling_backoff_ms} -Определяет дополнительное время ожидания, добавляемое к предыдущему интервалу опроса, когда новые файлы не найдены. Следующий опрос происходит после суммы предыдущего интервала и этого значения отсрочки, или максимального интервала, в зависимости от того, что меньше. +Определяет дополнительное время ожидания, добавляемое к предыдущему интервалу опроса при отсутствии новых файлов. Следующий опрос выполняется после истечения времени, равного сумме предыдущего интервала и этого значения backoff, либо по наступлении максимального интервала — в зависимости от того, что меньше. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `0`. -### `s3queue_tracked_files_limit` {#tracked_files_limit} +### `s3queue_tracked_files_limit` {#tracked_files_limit} -Позволяет ограничить количество узлов ZooKeeper при использовании режима 'unordered', не действует для режима 'ordered'. -При достижении лимита самые старые обработанные файлы будут удалены из узла ZooKeeper и обработаны снова. +Позволяет ограничить количество узлов ZooKeeper при использовании режима `unordered`; не влияет на режим `ordered`. +Если лимит достигнут, самые старые обработанные файлы будут удалены из узла ZooKeeper и обработаны повторно. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `1000`. -### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} +### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} -Максимальное количество секунд для хранения обработанных файлов в узле ZooKeeper (по умолчанию хранятся бессрочно) для режима 'unordered', не действует для режима 'ordered'. -По истечении указанного количества секунд файл будет импортирован повторно. +Максимальное время в секундах для хранения обработанных файлов в узле ZooKeeper (по умолчанию хранятся бессрочно) в режиме `unordered`; не влияет на режим `ordered`. +По истечении указанного времени файл будет загружен повторно. Возможные значения: -- Положительное целое число. +* Положительное целое число. Значение по умолчанию: `0`. -### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} +### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} -Для режима 'Ordered'. Определяет минимальную границу интервала перепланирования для фоновой задачи, которая отвечает за поддержание TTL отслеживаемых файлов и максимального набора отслеживаемых файлов. +Для режима `Ordered`. Определяет минимальное значение интервала перепланирования фоновой задачи, которая отвечает за поддержание TTL отслеживаемых файлов и ограничения на их максимальное количество. Значение по умолчанию: `10000`. -### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} - +### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} -Для режима 'Ordered'. Определяет максимальную границу интервала перепланирования для фоновой задачи, отвечающей за поддержание TTL отслеживаемых файлов и максимального количества отслеживаемых файлов. +Для режима `Ordered`. Определяет максимальный интервал между переназначениями фоновой задачи, которая отвечает за поддержание TTL отслеживаемых файлов и максимального набора отслеживаемых файлов. Значение по умолчанию: `30000`. ### `s3queue_buckets` {#buckets} -Для режима 'Ordered'. Доступно начиная с версии `24.6`. Если существует несколько реплик таблицы S3Queue, каждая из которых работает с одним и тем же каталогом метаданных в keeper, значение `s3queue_buckets` должно быть не меньше количества реплик. Если также используется параметр `s3queue_processing_threads`, имеет смысл дополнительно увеличить значение параметра `s3queue_buckets`, так как он определяет фактическую степень параллелизма обработки `S3Queue`. +Для режима `Ordered`. Доступно начиная с версии `24.6`. Если существует несколько реплик таблицы `S3Queue`, каждая из которых работает с одним и тем же каталогом метаданных в keeper, значение `s3queue_buckets` должно быть не меньше количества реплик. Если также используется настройка `s3queue_processing_threads`, имеет смысл дополнительно увеличить значение `s3queue_buckets`, так как она определяет фактический уровень параллелизма обработки `S3Queue`. -### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} +### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} -По умолчанию таблица S3Queue всегда использовала эфемерные узлы обработки, что могло приводить к дублированию данных в случае истечения сессии zookeeper до того, как S3Queue зафиксирует обработанные файлы в zookeeper, но после начала обработки. Этот параметр заставляет сервер исключить возможность появления дубликатов при истечении сессии keeper. +По умолчанию таблица S3Queue всегда использовала эфемерные узлы обработки, что могло приводить к дублированию данных в случае, если сессия ZooKeeper истекает до того, как S3Queue зафиксирует обработанные файлы в ZooKeeper, но после того, как обработка уже началась. Эта настройка заставляет сервер исключить возможность появления дубликатов при истечении сессии Keeper. -### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} +### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} -В случае аварийного завершения работы сервера возможна ситуация, когда при включённом параметре `use_persistent_processing_nodes` узлы обработки не будут удалены. Этот параметр определяет период времени, в течение которого эти узлы обработки могут быть безопасно удалены. +В случае некорректного завершения работы сервера, если включён `use_persistent_processing_nodes`, могут остаться неудалённые узлы обработки. Этот параметр определяет период времени, по истечении которого эти узлы обработки могут быть безопасно удалены. Значение по умолчанию: `3600` (1 час). +## Настройки S3 {#s3-settings} -## Настройки, связанные с S3 {#s3-settings} - -Движок поддерживает все настройки, связанные с S3. Подробнее о настройках S3 см. [здесь](../../../engines/table-engines/integrations/s3.md). - +Движок поддерживает все настройки S3. Дополнительную информацию о настройках S3 см. [здесь](../../../engines/table-engines/integrations/s3.md). -## Доступ к S3 на основе ролей {#s3-role-based-access} +## Ролевой доступ к S3 {#s3-role-based-access} - + -Движок таблиц s3Queue поддерживает доступ на основе ролей. -Инструкции по настройке роли для доступа к вашему бакету см. в документации [здесь](/cloud/data-sources/secure-s3). +Движок таблицы s3Queue поддерживает ролевую модель доступа. +См. документацию [здесь](/cloud/data-sources/secure-s3) с описанием шагов по настройке роли для доступа к вашему бакету. -После настройки роли можно передать `roleARN` через параметр `extra_credentials`, как показано ниже: +После настройки роли значение `roleARN` можно передать через параметр `extra_credentials`, как показано ниже: ```sql CREATE TABLE s3_table @@ -261,26 +350,24 @@ SETTINGS ... ``` +## Упорядоченный режим (ordered) для S3Queue {#ordered-mode} -## Режим ordered для S3Queue {#ordered-mode} +Режим обработки `S3Queue` позволяет хранить меньше метаданных в ZooKeeper, но имеет ограничение: файлы, добавленные позже по времени, должны иметь имена, которые в алфавитно-цифровом порядке больше имён ранее добавленных файлов. -Режим обработки `S3Queue` позволяет хранить меньше метаданных в ZooKeeper, но имеет ограничение: файлы, добавленные позже по времени, должны иметь алфавитно-цифровые имена, которые идут дальше в порядке сортировки. - -Режим `ordered` для `S3Queue`, как и режим `unordered`, поддерживает настройку `(s3queue_)processing_threads_num` (префикс `s3queue_` необязателен), которая позволяет управлять количеством потоков, обрабатывающих файлы `S3` локально на сервере. -Кроме того, режим `ordered` также вводит другую настройку `(s3queue_)buckets`, которая означает «логические потоки». В распределённом сценарии, когда имеется несколько серверов с репликами таблицы `S3Queue`, эта настройка определяет количество единиц обработки. Например, каждый поток обработки на каждой реплике `S3Queue` будет пытаться заблокировать определённый `bucket` для обработки; каждый `bucket` привязывается к определённым файлам по хешу имени файла. Поэтому в распределённом сценарии настоятельно рекомендуется, чтобы значение настройки `(s3queue_)buckets` было как минимум равно количеству реплик или больше. Допустимо иметь количество buckets больше, чем количество реплик. Наиболее оптимальным сценарием является случай, когда значение настройки `(s3queue_)buckets` равно произведению `number_of_replicas` и `(s3queue_)processing_threads_num`. -Настройка `(s3queue_)processing_threads_num` не рекомендуется к использованию в версиях до `24.6`. +Режим `ordered` в `S3Queue`, так же как и `unordered`, поддерживает настройку `(s3queue_)processing_threads_num` (префикс `s3queue_` является необязательным), которая позволяет управлять количеством потоков, обрабатывающих файлы `S3` локально на сервере. +Кроме того, режим `ordered` вводит дополнительную настройку `(s3queue_)buckets`, которая означает «логические потоки». В распределённой конфигурации, когда есть несколько серверов с репликами таблиц `S3Queue`, эта настройка определяет количество единиц обработки. Например, каждый поток обработки на каждой реплике `S3Queue` будет пытаться захватить определённый `bucket` для обработки; каждый `bucket` сопоставляется определённым файлам по хэшу имени файла. Поэтому в распределённом сценарии настоятельно рекомендуется, чтобы значение настройки `(s3queue_)buckets` было как минимум равно количеству реплик или больше. Допустимо, если число бакетов больше количества реплик. Оптимальным будет сценарий, когда настройка `(s3queue_)buckets` равна произведению `number_of_replicas` и `(s3queue_)processing_threads_num`. +Использование настройки `(s3queue_)processing_threads_num` не рекомендуется до версии `24.6`. Настройка `(s3queue_)buckets` доступна начиная с версии `24.6`. +## Описание {#description} -## Description {#description} - -`SELECT` не особенно полезен для потоковой загрузки данных (за исключением отладки), поскольку каждый файл может быть импортирован только один раз. Более практичным является создание потоков обработки в реальном времени с использованием [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: +`SELECT` мало полезен для потокового импорта (кроме отладки), потому что каждый файл можно импортировать только один раз. Более практично создавать потоки в реальном времени с помощью [материализованных представлений](../../../sql-reference/statements/create/view.md). Для этого: -1. Используйте движок для создания таблицы, которая будет получать данные из указанного пути в S3, и рассматривайте её как поток данных. -2. Создайте таблицу с требуемой структурой. -3. Создайте материализованное представление, которое преобразует данные из движка и помещает их в ранее созданную таблицу. +1. Используйте этот движок для создания таблицы, которая будет читать данные из указанного пути в S3 и рассматриваться как поток данных. +2. Создайте таблицу с требуемой структурой. +3. Создайте материализованное представление, которое преобразует данные из этого движка и помещает их в ранее созданную таблицу. -Когда `MATERIALIZED VIEW` подключается к движку, он начинает собирать данные в фоновом режиме. +Когда `MATERIALIZED VIEW` подключено к этому движку, оно начинает собирать данные в фоновом режиме. Пример: @@ -299,48 +386,44 @@ SETTINGS SELECT * FROM stats ORDER BY name; ``` - ## Виртуальные столбцы {#virtual-columns} -- `_path` — Путь к файлу. -- `_file` — Имя файла. -- `_size` — Размер файла. -- `_time` — Время создания файла. - -Подробнее о виртуальных столбцах см. [здесь](../../../engines/table-engines/index.md#table_engines-virtual_columns). +* `_path` — Путь к файлу. +* `_file` — Имя файла. +* `_size` — Размер файла. +* `_time` — Время создания файла. +Дополнительную информацию о виртуальных столбцах см. [здесь](../../../engines/table-engines/index.md#table_engines-virtual_columns). -## Подстановочные символы в пути {#wildcards-in-path} +## Подстановочные символы в `path` {#wildcards-in-path} -Аргумент `path` может указывать на несколько файлов с использованием подстановочных символов в стиле bash. Для обработки файл должен существовать и соответствовать всему шаблону пути. Список файлов определяется во время выполнения запроса `SELECT` (а не в момент выполнения `CREATE`). +Аргумент `path` может задавать несколько файлов, используя подстановочные шаблоны в стиле bash. Чтобы файл был обработан, он должен существовать и полностью соответствовать шаблону пути. Перечень файлов определяется во время выполнения `SELECT` (а не в момент `CREATE`). -- `*` — Соответствует любому количеству любых символов, кроме `/`, включая пустую строку. -- `**` — Соответствует любому количеству любых символов, включая `/`, включая пустую строку. -- `?` — Соответствует любому одиночному символу. -- `{some_string,another_string,yet_another_one}` — Соответствует любой из строк `'some_string', 'another_string', 'yet_another_one'`. -- `{N..M}` — Соответствует любому числу в диапазоне от N до M, включая обе границы. N и M могут содержать ведущие нули, например `000..078`. +* `*` — Заменяет любое количество любых символов, кроме `/`, включая пустую строку. +* `**` — Заменяет любое количество любых символов, включая `/`, включая пустую строку. +* `?` — Заменяет ровно один произвольный символ. +* `{some_string,another_string,yet_another_one}` — Заменяет любую из строк `'some_string', 'another_string', 'yet_another_one'`. +* `{N..M}` — Заменяет любое число в диапазоне от N до M включительно. N и M могут содержать ведущие нули, например `000..078`. Конструкции с `{}` аналогичны табличной функции [remote](../../../sql-reference/table-functions/remote.md). - ## Ограничения {#limitations} -1. Дублирование строк может происходить в следующих случаях: - -- возникает исключение во время парсинга в процессе обработки файла, и включены повторные попытки через параметр `s3queue_loading_retries`; +1. Дубликаты строк могут возникать в результате: -- `S3Queue` настроен на нескольких серверах, указывающих на один и тот же путь в ZooKeeper, и сессия keeper истекает до того, как один из серверов успевает зафиксировать обработанный файл, что может привести к тому, что другой сервер начнёт обработку файла, который мог быть частично или полностью обработан первым сервером; однако это не относится к версии 25.8 и выше при `use_persistent_processing_nodes = 1`. +* во время парсинга происходит исключение в середине обработки файла, и включены повторные попытки через `s3queue_loading_retries`; -- происходит аварийное завершение работы сервера. +* `S3Queue` настроен на нескольких серверах, указывающих на один и тот же путь в ZooKeeper, и сессия Keeper завершается до того, как один из серверов успел зафиксировать обработанный файл, что может привести к тому, что другой сервер возьмет в обработку файл, который уже мог быть частично или полностью обработан первым сервером; однако это не актуально, начиная с версии 25.8, если `use_persistent_processing_nodes = 1`. -2. Если `S3Queue` настроен на нескольких серверах, указывающих на один и тот же путь в ZooKeeper, и используется режим `Ordered`, то параметр `s3queue_loading_retries` не будет работать. Эта проблема будет исправлена в ближайшее время. +* аварийного завершения работы сервера. +2. Если `S3Queue` настроен на нескольких серверах, указывающих на один и тот же путь в ZooKeeper, и используется режим `Ordered`, то `s3queue_loading_retries` не будет работать. Это будет скоро исправлено. ## Интроспекция {#introspection} -Для интроспекции используйте таблицу `system.s3queue` (без сохранения состояния) и постоянную таблицу `system.s3queue_log`. +Для интроспекции используйте неперсистентную таблицу `system.s3queue` и персистентную таблицу `system.s3queue_log`. -1. `system.s3queue`. Эта таблица не сохраняет данные и отображает состояние `S3Queue` в памяти: какие файлы обрабатываются в данный момент, какие файлы обработаны или завершились с ошибкой. +1. `system.s3queue`. Эта таблица неперсистентная и отображает состояние `S3Queue` в памяти: какие файлы в данный момент обрабатываются, какие файлы уже обработаны или завершились с ошибкой. ```sql ┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ @@ -357,7 +440,7 @@ SETTINGS `exception` String ) ENGINE = SystemS3Queue -COMMENT 'Contains in-memory state of S3Queue metadata and currently processed rows per file.' │ +COMMENT 'Содержит состояние метаданных S3Queue в памяти и текущее количество обработанных строк по каждому файлу.' │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` @@ -368,26 +451,26 @@ COMMENT 'Contains in-memory state of S3Queue metadata and currently processed ro SELECT * FROM system.s3queue -Row 1: +Строка 1: ────── zookeeper_path: /clickhouse/s3queue/25ea5621-ae8c-40c7-96d0-cec959c5ab88/3b3f66a1-9866-4c2e-ba78-b6bfa154207e file_name: wikistat/original/pageviews-20150501-030000.gz rows_processed: 5068534 -status: Processed +status: Обработано processing_start_time: 2023-10-13 13:09:48 processing_end_time: 2023-10-13 13:10:31 ProfileEvents: {'ZooKeeperTransactions':3,'ZooKeeperGet':2,'ZooKeeperMulti':1,'SelectedRows':5068534,'SelectedBytes':198132283,'ContextLock':1,'S3QueueSetFileProcessingMicroseconds':2480,'S3QueueSetFileProcessedMicroseconds':9985,'S3QueuePullMicroseconds':273776,'LogTest':17} exception: ``` -2. `system.s3queue_log`. Постоянная таблица. Содержит ту же информацию, что и `system.s3queue`, но для обработанных (`processed`) и завершившихся с ошибкой (`failed`) файлов. +2. `system.s3queue_log`. Персистентная таблица. Содержит ту же информацию, что и `system.s3queue`, но для файлов со статусами `processed` и `failed`. -Структура таблицы: +Таблица имеет следующую структуру: ```sql SHOW CREATE TABLE system.s3queue_log -Query id: 0ad619c3-0f2a-4ee4-8b40-c73d86e04314 +ID запроса: 0ad619c3-0f2a-4ee4-8b40-c73d86e04314 ┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ │ CREATE TABLE system.s3queue_log @@ -410,7 +493,7 @@ SETTINGS index_granularity = 8192 │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -Чтобы использовать `system.s3queue_log`, определите её конфигурацию в конфигурационном файле сервера: +Чтобы использовать `system.s3queue_log`, задайте его конфигурацию в файле конфигурации сервера: ```xml diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md index 2268c7f7761..7ec2a4ca14c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# Движок таблиц SQLite {#sqlite-table-engine} -# Движок таблиц SQLite - - + Этот движок позволяет импортировать данные в SQLite и экспортировать их из неё, а также выполнять запросы к таблицам SQLite непосредственно из ClickHouse. - - -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -33,8 +30,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; * `db_path` — Путь к файлу базы данных SQLite. * `table` — Название таблицы в базе данных SQLite. - -## Пример использования +## Пример использования {#usage-example} Пример запроса для создания таблицы SQLite: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md index 4f5b9435884..b22478cfa9d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Табличный движок TimeSeries +# Табличный движок TimeSeries {#timeseries-table-engine} @@ -32,7 +32,7 @@ metric_name2[...] = ... ::: -## Синтаксис +## Синтаксис {#syntax} ```sql CREATE TABLE name [(columns)] ENGINE=TimeSeries @@ -43,7 +43,7 @@ CREATE TABLE name [(columns)] ENGINE=TimeSeries ``` -## Использование +## Использование {#usage} Проще всего начать, оставив все настройки по умолчанию (можно создать таблицу `TimeSeries` без явного указания списка столбцов): @@ -117,7 +117,7 @@ CREATE TABLE my_table ENGINE=TimeSeries -## Создание +## Создание {#creation} Существует несколько способов создать таблицу с движком `TimeSeries`. Самый простой запрос @@ -202,7 +202,7 @@ ORDER BY metric_family_name ``` -## Настройка типов столбцов +## Настройка типов столбцов {#adjusting-column-types} Вы можете изменить тип почти любого столбца во внутренних целевых таблицах, явно указав его при определении основной таблицы. Например, @@ -228,7 +228,7 @@ ORDER BY (id, timestamp) ``` -## Столбец `id` +## Столбец `id` {#id-column} Столбец `id` содержит идентификаторы; каждый идентификатор вычисляется для комбинации имени метрики и тегов. Выражение DEFAULT для столбца `id` — это выражение, которое будет использоваться для вычисления таких идентификаторов. @@ -243,7 +243,7 @@ ENGINE=TimeSeries ``` -## Столбцы `tags` и `all_tags` +## Столбцы `tags` и `all_tags` {#tags-and-all-tags} Есть два столбца, содержащих отображения тегов, — `tags` и `all_tags`. В этом примере они по сути эквивалентны, однако могут отличаться, если используется настройка `tags_to_columns`. Эта настройка позволяет указать, что конкретный тег должен храниться в отдельном столбце вместо хранения @@ -278,7 +278,7 @@ SETTINGS tags_to_columns = {'instance': 'instance', 'job': 'job'} ``` -## Движки внутренних целевых таблиц +## Движки внутренних целевых таблиц {#inner-table-engines} По умолчанию внутренние целевые таблицы используют следующие движки таблиц: @@ -298,7 +298,7 @@ METRICS ENGINE=ReplicatedReplacingMergeTree ``` -## Внешние таблицы назначения +## Внешние таблицы назначения {#external-target-tables} Можно настроить таблицу `TimeSeries` на использование созданной вручную таблицы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md index 3403e3953f2..a85b241d9d2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md @@ -12,7 +12,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Движок таблицы YTsaurus +# Движок таблицы YTsaurus {#ytsaurus-table-engine} @@ -21,7 +21,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -48,7 +48,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; * `oauth_token` — OAuth-токен. -## Пример использования +## Пример использования {#usage-example} Пример запроса для создания таблицы YTsaurus: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md index 177f681a8bf..bb5c6bd3f37 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md @@ -10,7 +10,7 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Семейство движков таблиц Log +# Семейство движков таблиц Log {#log-table-engine-family} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md index 6d786230c94..b3633198efb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы Log +# Движок таблицы Log {#log-table-engine} @@ -22,7 +22,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## Создание таблицы +## Создание таблицы {#table_engines-log-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,7 +59,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Пример использования +## Пример использования {#table_engines-log-example-of-use} Создание таблицы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md index b0cfb49f8d6..6283cc281e9 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблицы StripeLog +# Движок таблицы StripeLog {#stripelog-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## Создание таблицы +## Создание таблицы {#table_engines-stripelog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -53,7 +53,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Пример использования +## Пример использования {#table_engines-stripelog-example-of-use} Создание таблицы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md index 988c67ef87d..6bdba72acda 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Движок таблиц TinyLog +# Движок таблиц TinyLog {#tinylog-table-engine} @@ -32,7 +32,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## Создание таблицы +## Создание таблицы {#table_engines-tinylog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -58,7 +58,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Пример использования +## Пример использования {#table_engines-tinylog-example-of-use} Создание таблицы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md index af0991fb0bb..08e9ca4ecb4 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок таблиц AggregatingMergeTree +# Движок таблиц AggregatingMergeTree {#aggregatingmergetree-table-engine} Движок наследуется от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) и изменяет логику слияния частей данных. ClickHouse заменяет все строки с одинаковым первичным ключом (или, точнее, с одинаковым [ключом сортировки](../../../engines/table-engines/mergetree-family/mergetree.md)) одной строкой (в пределах одной части данных), которая хранит комбинацию состояний агрегатных функций. @@ -29,7 +29,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -80,7 +80,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Пример агрегированного материализованного представления +## Пример агрегированного материализованного представления {#example-of-an-aggregated-materialized-view} В этом примере предполагается, что у вас есть база данных под названием `test`. Создайте её, если она ещё не существует, с помощью приведённой ниже команды: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md index 5e480489647..0c49b86fed3 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md @@ -10,7 +10,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Точный и приближённый векторный поиск +# Точный и приближённый векторный поиск {#exact-and-approximate-vector-search} Задача нахождения N ближайших точек в многомерном (векторном) пространстве для заданной точки известна как [поиск ближайших соседей](https://en.wikipedia.org/wiki/Nearest_neighbor_search) или, кратко, векторный поиск. Существуют два общих подхода к решению задачи векторного поиска: @@ -36,13 +36,13 @@ LIMIT `<N>` задаёт, сколько соседей нужно вернуть. -## Точный поиск по векторам +## Точный поиск по векторам {#exact-nearest-neighbor-search} Точный поиск по векторам можно выполнить с использованием приведённого выше запроса SELECT без изменений. Время выполнения таких запросов, как правило, пропорционально количеству сохранённых векторов и их размерности, то есть количеству элементов массива. Кроме того, поскольку ClickHouse выполняет полный перебор всех векторов, время выполнения таких запросов также зависит от количества потоков, используемых запросом (см. настройку [max_threads](../../../operations/settings/settings.md#max_threads)). -### Пример +### Пример {#exact-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; @@ -67,9 +67,9 @@ LIMIT 3; ``` -## Приблизительный векторный поиск +## Приблизительный векторный поиск {#approximate-nearest-neighbor-search} -### Индексы сходства векторов +### Индексы сходства векторов {#vector-similarity-index} ClickHouse предоставляет специальный индекс сходства векторов («vector similarity») для выполнения приблизительного векторного поиска. @@ -78,7 +78,7 @@ ClickHouse предоставляет специальный индекс схо Если вы столкнётесь с проблемами, пожалуйста, создайте issue в [репозитории ClickHouse](https://github.com/clickhouse/clickhouse/issues). ::: -#### Создание индекса сходства векторов +#### Создание индекса сходства векторов {#creating-a-vector-similarity-index} Индекс сходства векторов можно создать на новой таблице следующим образом: @@ -196,7 +196,7 @@ ORDER BY [...] Приведенная выше формула не учитывает дополнительную память, необходимую индексам векторного сходства для выделения структур данных, используемых во время выполнения, таких как заранее выделенные буферы и кэши. -#### Использование индекса векторного сходства +#### Использование индекса векторного сходства {#using-a-vector-similarity-index} :::note Чтобы использовать индексы векторного сходства, настройка [compatibility](../../../operations/settings/settings.md) должна быть равна `''` (значение по умолчанию) или `'25.1'` либо новее. @@ -406,7 +406,7 @@ Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 Запрос, выполняемый без повторной оценки (`vector_search_with_rescoring = 0`) и с включёнными параллельными репликами, может всё равно перейти к повторной оценке результатов. ::: -#### Оптимизация производительности +#### Оптимизация производительности {#performance-tuning} **Настройка сжатия** @@ -540,7 +540,7 @@ result = chclient.query( В этом примере опорный вектор отправляется как есть в бинарном виде и на сервере интерпретируется как массив чисел с плавающей запятой. Это экономит процессорное время на стороне сервера и предотвращает избыточный рост серверных логов и `system.query_log`. -#### Администрирование и мониторинг +#### Администрирование и мониторинг {#administration} Объём на диске индексов векторного сходства можно получить из [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices): @@ -558,7 +558,7 @@ WHERE type = 'vector_similarity'; └──────────┴───────┴──────┴──────────────────────────┘ ``` -#### Отличия от обычных пропускающих индексов +#### Отличия от обычных пропускающих индексов {#differences-to-regular-skipping-indexes} Как и все обычные [пропускающие индексы](/optimize/skipping-indexes), индексы векторного сходства строятся поверх гранул, и каждый индексируемый блок состоит из `GRANULARITY = [N]` гранул (`[N]` = 1 по умолчанию для обычных пропускающих индексов). Например, если гранулярность первичного индекса таблицы равна 8192 (параметр `index_granularity = 8192`) и `GRANULARITY = 2`, то каждый индексируемый блок будет содержать 16384 строки. @@ -584,7 +584,7 @@ WHERE type = 'vector_similarity'; Обычно рекомендуется использовать большое значение `GRANULARITY` для индексов векторного сходства и переходить к меньшим значениям `GRANULARITY` только в случае проблем, например чрезмерного потребления памяти структурами векторного сходства. Если `GRANULARITY` для индексов векторного сходства не задан, значение по умолчанию — 100 миллионов. -#### Пример +#### Пример {#approximate-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; @@ -615,7 +615,7 @@ LIMIT 3; * [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) * [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) -### Квантованный бит (QBit) +### Квантованный бит (QBit) {#approximate-nearest-neighbor-search-qbit} @@ -649,7 +649,7 @@ column_name QBit(element_type, dimension) * `element_type` – тип каждого элемента вектора. Поддерживаемые типы: `BFloat16`, `Float32` и `Float64` * `dimension` – количество элементов в каждом векторе -#### Создание таблицы `QBit` и добавление данных +#### Создание таблицы `QBit` и добавление данных {#qbit-create} ```sql CREATE TABLE fruit_animal ( @@ -667,7 +667,7 @@ INSERT INTO fruit_animal VALUES ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); ``` -#### Векторный поиск с `QBit` +#### Векторный поиск с `QBit` {#qbit-search} Найдём ближайших соседей к вектору, соответствующему слову «lemon», используя L2-расстояние. Третий параметр в функции расстояния задаёт точность в битах: более высокие значения обеспечивают большую точность, но требуют больше вычислительных ресурсов. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md index ab0708f5c5c..d552544b967 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md @@ -9,9 +9,7 @@ show_related_blogs: true doc_type: 'reference' --- - - -# Движок таблицы CoalescingMergeTree +# Движок таблицы CoalescingMergeTree {#coalescingmergetree-table-engine} :::note Доступно начиная с версии 25.6 Этот движок таблицы доступен начиная с версии 25.6 как в OSS, так и в Cloud. @@ -23,9 +21,7 @@ doc_type: 'reference' `CoalescingMergeTree` предназначен для использования с типами Nullable в неклю́чевых столбцах. Если столбцы не являются Nullable, поведение такое же, как у [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree). - - -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -42,16 +38,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] Описание параметров запроса см. в разделе [описание запроса](../../../sql-reference/statements/create/table.md). -### Параметры движка CoalescingMergeTree +### Параметры движка CoalescingMergeTree {#parameters-of-coalescingmergetree} -#### Столбцы +#### Столбцы {#columns} `columns` — кортеж имен столбцов, значения в которых будут объединены. Необязательный параметр. Столбцы должны иметь числовой тип и не должны входить в ключ партиционирования или сортировки. Если `columns` не указан, ClickHouse объединяет значения во всех столбцах, которые не входят в ключ сортировки. -### Части запроса +### Части запроса {#query-clauses} При создании таблицы `CoalescingMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. @@ -76,8 +72,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `columns` — кортеж имен столбцов, значения которых будут суммироваться. Необязательный параметр. Описание см. в тексте выше.
- -## Пример использования +## Пример использования {#usage-example} Рассмотрим следующую таблицу: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md index e45cf8d2fd0..9604373a546 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# Движок таблицы CollapsingMergeTree +# Движок таблицы CollapsingMergeTree {#collapsingmergetree-table-engine} @@ -41,7 +41,7 @@ doc_type: 'guide' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -82,9 +82,9 @@ ENGINE = CollapsingMergeTree(Sign) * При создании таблицы `CollapsingMergeTree` используются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table), что и при создании таблицы `MergeTree`. -## Схлопывание +## Схлопывание {#table_engine-collapsingmergetree-collapsing} -### Данные +### Данные {#data} Рассмотрим ситуацию, когда необходимо сохранять постоянно меняющиеся данные для некоторого объекта. Может показаться логичным иметь по одной строке на объект и обновлять её каждый раз при изменении данных, @@ -142,7 +142,7 @@ ENGINE = CollapsingMergeTree(Sign) 2. Длинные растущие массивы в столбцах снижают эффективность движка из-за увеличенной нагрузки на запись. Чем проще данные, тем выше эффективность. 3. Результаты `SELECT` в значительной степени зависят от согласованности истории изменений объекта. Будьте аккуратны при подготовке данных для вставки. При несогласованных данных можно получить непредсказуемые результаты. Например, отрицательные значения для неотрицательных метрик, таких как глубина сессии. -### Algorithm +### Algorithm {#table_engine-collapsingmergetree-collapsing-algorithm} Когда ClickHouse сливает [части](/concepts/glossary#parts) данных, каждая группа последовательных строк с одинаковым ключом сортировки (`ORDER BY`) сокращается до не более чем двух строк: @@ -190,9 +190,9 @@ ClickHouse обрабатывает запросы `SELECT` в нескольк -## Примеры +## Примеры {#examples} -### Пример использования +### Пример использования {#example-of-use} Даны следующие исходные данные: @@ -292,7 +292,7 @@ SELECT * FROM UAct FINAL Этот способ выборки данных менее эффективен и не рекомендуется при работе с большими объемами сканируемых данных (миллионы строк). ::: -### Пример другого подхода +### Пример другого подхода {#example-of-another-approach} Идея этого подхода состоит в том, что при слияниях учитываются только ключевые поля. В строке "cancel" мы, соответственно, можем указать отрицательные значения, diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md index 9829780e278..5464b210ba1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md @@ -7,9 +7,7 @@ title: 'Пользовательский ключ партиционирован doc_type: 'guide' --- - - -# Пользовательский ключ партиционирования +# Пользовательский ключ партиционирования {#custom-partitioning-key} :::note В большинстве случаев ключ партиционирования не нужен, а в большинстве остальных случаев нет необходимости в ключе партиционирования с более мелкой детализацией, чем по месяцам, за исключением сценариев наблюдаемости (observability), где часто используется партиционирование по дням. @@ -78,7 +76,6 @@ WHERE table = 'visits' Столбец `partition` содержит имена партиций. В этом примере есть две партиции: `201901` и `201902`. Вы можете использовать значение этого столбца, чтобы указать имя партиции в запросах [ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md). - Столбец `name` содержит имена частей данных партиции. Вы можете использовать этот столбец, чтобы указать имя части в запросе [ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart). Разберём имя части: `201901_1_9_2_11`: @@ -134,16 +131,13 @@ drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached Папки '201901_1_1_0', '201901_1_7_1' и так далее являются каталогами частей. Каждая часть относится к соответствующему разделу (partition) и содержит данные только за определённый месяц (в этом примере таблица разбита на разделы по месяцам). - Каталог `detached` содержит части, которые были отсоединены от таблицы с помощью запроса [DETACH](/sql-reference/statements/detach). Повреждённые части также перемещаются в этот каталог вместо удаления. Сервер не использует части из каталога `detached`. Вы можете добавлять, удалять или изменять данные в этом каталоге в любое время — сервер не узнает об этом, пока вы не выполните запрос [ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart). Обратите внимание, что на рабочем сервере вы не можете вручную изменять набор частей или их данные в файловой системе, так как сервер не узнает об этом. Для нереплицируемых таблиц вы можете делать это, когда сервер остановлен, но это не рекомендуется. Для реплицируемых таблиц набор частей не может быть изменён ни при каких обстоятельствах. ClickHouse позволяет выполнять операции с партициями: удалять их, копировать из одной таблицы в другую или создавать резервную копию. См. полный список операций в разделе [Manipulations With Partitions and Parts](/sql-reference/statements/alter/partition). - - -## Оптимизация Group By с использованием ключа партиционирования +## Оптимизация Group By с использованием ключа партиционирования {#group-by-optimisation-using-partition-key} Для некоторых комбинаций ключа партиционирования таблицы и ключа группировки запроса (`GROUP BY`) агрегацию можно выполнять независимо для каждой партиции. Тогда нам не придется в конце объединять частично агрегированные данные со всех потоков выполнения, diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md index 305019b1c39..8a705ad5061 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md @@ -7,9 +7,7 @@ title: 'Движок таблицы GraphiteMergeTree' doc_type: 'guide' --- - - -# Движок таблицы GraphiteMergeTree +# Движок таблицы GraphiteMergeTree {#graphitemergetree-table-engine} Этот движок предназначен для прореживания и агрегирования/усреднения (rollup) данных [Graphite](http://graphite.readthedocs.io/en/latest/index.html). Он может быть полезен разработчикам, которые хотят использовать ClickHouse в качестве хранилища данных для Graphite. @@ -17,9 +15,7 @@ doc_type: 'guide' Движок наследует свойства от [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md). - - -## Создание таблицы +## Создание таблицы {#creating-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -82,8 +78,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `config_section` — Имя раздела в файле конфигурации, где заданы правила rollup.
- -## Конфигурация rollup +## Конфигурация rollup {#rollup-configuration} Настройки rollup задаются параметром [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) в конфигурации сервера. Имя параметра может быть любым. Вы можете создать несколько конфигураций и использовать их для разных таблиц. @@ -92,25 +87,25 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] required-columns patterns -### Обязательные столбцы +### Обязательные столбцы {#required-columns} -#### `path_column_name` +#### `path_column_name` {#path_column_name} `path_column_name` — имя столбца, в котором хранится имя метрики (датчик Graphite). Значение по умолчанию: `Path`. -#### `time_column_name` +#### `time_column_name` {#time_column_name} `time_column_name` — имя столбца, в котором хранится время измерения метрики. Значение по умолчанию: `Time`. -#### `value_column_name` +#### `value_column_name` {#value_column_name} `value_column_name` — имя столбца, в котором хранится значение метрики в момент времени, указанной в `time_column_name`. Значение по умолчанию: `Value`. -#### `version_column_name` +#### `version_column_name` {#version_column_name} `version_column_name` — имя столбца, в котором хранится версия метрики. Значение по умолчанию: `Timestamp`. -### Шаблоны +### Шаблоны {#patterns} Структура раздела `patterns`: @@ -163,8 +158,7 @@ default * `precision` – точность определения возраста данных в секундах. Должно быть делителем 86400 (количество секунд в сутках). * `function` – имя агрегирующей функции, применяемой к данным, возраст которых попадает в диапазон `[age, age + precision]`. Допустимые функции: min / max / any / avg. Среднее значение рассчитывается неточно, как среднее от средних значений. -### Пример конфигурации без типов правил - +### Пример конфигурации без типов правил {#configuration-example} ```xml @@ -199,7 +193,7 @@ default ``` -### Пример конфигурации с типами правил +### Пример конфигурации с типами правил {#configuration-typed-example} ```xml diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md index 357aa5f16db..08f8de9d12d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md @@ -7,9 +7,7 @@ title: 'Семейство движков MergeTree' doc_type: 'reference' --- - - -# Семейство движков MergeTree +# Семейство движков MergeTree {#mergetree-engine-family} Табличные движки из семейства MergeTree являются ядром возможностей ClickHouse по хранению данных. Они предоставляют большинство функций для обеспечения отказоустойчивости и высокопроизводительного чтения данных: колоночное хранение, настраиваемое разбиение на партиции, разрежённый первичный индекс, вторичные индексы пропуска данных и т. д. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md index b27704e82ac..5516126eae6 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md @@ -10,7 +10,7 @@ doc_type: 'reference' import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; -# Полнотекстовый поиск с использованием текстовых индексов +# Полнотекстовый поиск с использованием текстовых индексов {#full-text-search-using-text-indexes} @@ -22,7 +22,7 @@ import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; -## Создание текстового индекса +## Создание текстового индекса {#creating-a-text-index} Чтобы создать текстовый индекс, сначала включите соответствующий экспериментальный параметр: @@ -186,12 +186,12 @@ ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); ``` -## Использование текстового индекса +## Использование текстового индекса {#using-a-text-index} Использовать текстовый индекс в запросах SELECT просто, так как стандартные функции строкового поиска автоматически используют индекс. Если индекс отсутствует, приведённые ниже функции строкового поиска будут выполнять медленный полный перебор по всем данным. -### Поддерживаемые функции +### Поддерживаемые функции {#functions-support} Текстовый индекс может быть использован, если текстовые функции применяются в условии `WHERE` запроса SELECT: @@ -201,7 +201,7 @@ FROM [...] WHERE функция_поиска_по_строке(столбец_с_текстовым_индексом) ``` -#### `=` и `!=` +#### `=` и `!=` {#functions-example-equals-notequals} `=` ([equals](/sql-reference/functions/comparison-functions.md/#equals)) и `!=` ([notEquals](/sql-reference/functions/comparison-functions.md/#notEquals)) совпадают с заданным поисковым выражением целиком. @@ -213,7 +213,7 @@ SELECT * from tab WHERE str = 'Привет'; Индекс по тексту поддерживает операторы `=` и `!=`, однако поиск по равенству и неравенству имеет смысл только с токенизатором `array` (он приводит к тому, что индекс хранит значения целых строк). -#### `IN` и `NOT IN` +#### `IN` и `NOT IN` {#functions-example-in-notin} `IN` ([in](/sql-reference/functions/in-functions)) и `NOT IN` ([notIn](/sql-reference/functions/in-functions)) похожи на функции `equals` и `notEquals`, но они выбирают все (`IN`) или ни одного (`NOT IN`) из искомых значений. @@ -225,7 +225,7 @@ SELECT * from tab WHERE str IN ('Привет', 'Мир'); Те же ограничения, что и для `=` и `!=`, действуют и здесь: `IN` и `NOT IN` имеют смысл только в сочетании с токенизатором `array`. -#### `LIKE`, `NOT LIKE` и `match` +#### `LIKE`, `NOT LIKE` и `match` {#functions-example-like-notlike-match} :::note В настоящее время эти функции используют текстовый индекс для фильтрации только в том случае, если токенизатор индекса — `splitByNonAlpha` или `ngrams`. @@ -250,7 +250,7 @@ SELECT count() FROM tab WHERE comment LIKE ' support %'; -- или `% support %` Пробелы слева и справа от `support` гарантируют, что термин может быть извлечён как токен. -#### `startsWith` и `endsWith` +#### `startsWith` и `endsWith` {#functions-example-startswith-endswith} Аналогично оператору `LIKE`, функции [startsWith](/sql-reference/functions/string-functions.md/#startsWith) и [endsWith](/sql-reference/functions/string-functions.md/#endsWith) могут использовать текстовый индекс только в том случае, если из поискового выражения могут быть извлечены полные токены. @@ -275,7 +275,7 @@ startsWith(comment, 'clickhouse supports ')` SELECT count() FROM tab WHERE endsWith(comment, ' olap engine'); ``` -#### `hasToken` и `hasTokenOrNull` +#### `hasToken` и `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} Функции [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) и [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) сопоставляют значение с одним заданным токеном. @@ -289,7 +289,7 @@ SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); Функции `hasToken` и `hasTokenOrNull` являются наиболее эффективными при использовании с индексом `text`. -#### `hasAnyTokens` и `hasAllTokens` +#### `hasAnyTokens` и `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} Функции [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) и [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) сопоставляют строку соответственно с одним или со всеми указанными токенами. @@ -309,7 +309,7 @@ SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); ``` -#### `has` +#### `has` {#functions-example-has} Функция для работы с массивами [`has`](/sql-reference/functions/array-functions#has) проверяет наличие отдельного токена в массиве строк. @@ -319,7 +319,7 @@ SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE has(array, 'clickhouse'); ``` -#### `mapContains` +#### `mapContains` {#functions-example-mapcontains} Функция [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) (псевдоним `mapContainsKey`) проверяет, содержится ли заданный токен среди ключей отображения. @@ -331,7 +331,7 @@ SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); ``` -#### `operator[]` +#### `operator[]` {#functions-example-access-operator} Оператор доступа [`operator[]`](/sql-reference/operators#access-operators) можно использовать с текстовым индексом для фильтрации по ключам и значениям. @@ -343,9 +343,9 @@ SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; Рассмотрим следующие примеры использования столбцов типов `Array(T)` и `Map(K, V)` с текстовым индексом. -### Примеры для столбцов `Array` и `Map` с текстовыми индексами +### Примеры для столбцов `Array` и `Map` с текстовыми индексами {#text-index-array-and-map-examples} -#### Индексирование столбцов `Array(String)` +#### Индексирование столбцов `Array(String)` {#text-index-example-array} Представьте платформу для ведения блогов, где авторы классифицируют свои записи с помощью ключевых слов. Мы хотим, чтобы пользователи могли находить похожие материалы, выполняя поиск или переходя по темам. @@ -377,7 +377,7 @@ ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitBy ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- Не забудьте материализовать индекс для существующих данных ``` -#### Индексация столбцов типа Map +#### Индексация столбцов типа Map {#text-index-example-map} Во многих сценариях систем наблюдаемости сообщения логов разбиваются на отдельные «компоненты» и сохраняются в виде соответствующих типов данных, например тип `DateTime` для временной метки, `Enum` для уровня логирования и т. д. Поля метрик оптимально хранить в виде пар «ключ–значение». @@ -435,9 +435,9 @@ SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- быст ``` -## Оптимизация производительности +## Оптимизация производительности {#performance-tuning} -### Прямое чтение +### Прямое чтение {#direct-read} Некоторые типы текстовых запросов можно значительно ускорить с помощью оптимизации, называемой «прямое чтение». Более точно, эту оптимизацию можно применить, если в запросе SELECT текстовый столбец *не* выбирается. @@ -515,7 +515,7 @@ Expression (перед GROUP BY) Вывод второго EXPLAIN PLAN содержит виртуальный столбец `__text_index___`. Если этот столбец присутствует, используется прямое чтение. -### Кэширование +### Кэширование {#caching} Доступны различные кэши для буферизации частей текстового индекса в памяти (см. раздел [Сведения о реализации](#implementation)): В настоящее время доступны кэши для десериализованных блоков словаря, заголовков и списков постингов текстового индекса для снижения объема операций ввода-вывода (I/O). @@ -524,7 +524,7 @@ Expression (перед GROUP BY) Для настройки кэшей используйте следующие параметры сервера. -#### Настройки кэша блоков словаря +#### Настройки кэша блоков словаря {#caching-dictionary} | Параметр | Описание | @@ -583,7 +583,7 @@ Expression (перед GROUP BY) -## Пример: датасет Hacker News +## Пример: датасет Hacker News {#hacker-news-dataset} Рассмотрим, как текстовые индексы улучшают производительность на большом текстовом наборе данных. Мы будем использовать 28,7 млн строк комментариев с популярного сайта Hacker News. @@ -650,7 +650,7 @@ ALTER TABLE hackernews MATERIALIZE INDEX comment_idx SETTINGS mutations_sync = 2 Теперь давайте выполним запросы с использованием функций `hasToken`, `hasAnyTokens` и `hasAllTokens`. Следующие примеры покажут существенную разницу в производительности между стандартным сканированием индекса и оптимизацией прямого чтения. -### 1. Использование `hasToken` +### 1. Использование `hasToken` {#using-hasToken} `hasToken` проверяет, содержит ли текст указанный одиночный токен. Мы будем искать регистрозависимый токен `ClickHouse`. @@ -690,7 +690,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re Запрос прямого чтения более чем в 45 раз быстрее (0,362 с против 0,008 с) и обрабатывает значительно меньше данных (9,51 ГБ против 3,15 МБ), поскольку читает только из индекса. -### 2. Использование `hasAnyTokens` +### 2. Использование `hasAnyTokens` {#using-hasAnyTokens} `hasAnyTokens` проверяет, содержит ли текст хотя бы один из заданных токенов. Мы будем искать комментарии, содержащие либо «love», либо «ClickHouse». @@ -730,7 +730,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re Ускорение ещё более впечатляющее для этого распространённого поиска с условием "ИЛИ". Запрос выполняется почти в 89 раз быстрее (1.329 с против 0.015 с) благодаря отсутствию полного сканирования столбца. -### 3. Использование `hasAllTokens` +### 3. Использование `hasAllTokens` {#using-hasAllTokens} `hasAllTokens` проверяет, содержит ли текст все указанные токены. Выполним поиск комментариев, содержащих одновременно 'love' и 'ClickHouse'. @@ -770,7 +770,7 @@ SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_re Для этого поиска с оператором "AND" оптимизация прямого чтения более чем в 26 раз быстрее (0.184 s против 0.007 s), чем стандартное сканирование с использованием skip-индекса. -### 4. Составной поиск: OR, AND, NOT, ... +### 4. Составной поиск: OR, AND, NOT, ... {#compound-search} Оптимизация прямого чтения также применима к составным булевым выражениям. Здесь мы выполним поиск без учета регистра для 'ClickHouse' OR 'clickhouse'. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md index c7a12b9bbc4..5b32f798d56 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md @@ -1,40 +1,37 @@ --- -description: 'Движки таблиц семейства `MergeTree` разработаны для высокой скорости загрузки и обработки огромных объемов данных.' +description: 'Семейство табличных движков `MergeTree` предназначено для высоких скоростей приёма данных и работы с очень большими объёмами данных.' sidebar_label: 'MergeTree' sidebar_position: 11 slug: /engines/table-engines/mergetree-family/mergetree -title: 'Движок таблицы MergeTree' +title: 'Табличный движок MergeTree' doc_type: 'reference' --- import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# Движок таблицы MergeTree {#mergetree-table-engine} -# Движок таблиц MergeTree +Движок `MergeTree` и другие движки семейства `MergeTree` (например, `ReplacingMergeTree`, `AggregatingMergeTree`) являются наиболее часто используемыми и наиболее надёжными движками таблиц в ClickHouse. -Движок `MergeTree` и другие движки семейства `MergeTree` (например, `ReplacingMergeTree`, `AggregatingMergeTree`) являются наиболее часто используемыми и наиболее надежными движками таблиц в ClickHouse. +Движки таблиц семейства `MergeTree` спроектированы для высокой скорости приёма данных и работы с очень большими объёмами. +Операции вставки создают части таблицы, которые затем объединяются фоновым процессом с другими частями таблицы. -Движки таблиц семейства `MergeTree` разработаны для высокой скорости загрузки данных и работы с очень большими объемами данных. -Операции вставки создают части таблицы (table parts), которые фоновым процессом сливаются с другими частями таблицы. +Основные особенности движков таблиц семейства `MergeTree`. -Основные особенности движков таблиц семейства `MergeTree`: +* Первичный ключ таблицы определяет порядок сортировки внутри каждой части таблицы (кластерный индекс). При этом первичный ключ указывает не на отдельные строки, а на блоки по 8192 строки, которые называются гранулами. Это делает первичные ключи для очень больших наборов данных достаточно компактными, чтобы оставаться загруженными в основную память, при этом обеспечивая быстрый доступ к данным на диске. -- Первичный ключ таблицы определяет порядок сортировки внутри каждой части таблицы (кластерный индекс). При этом первичный ключ ссылается не на отдельные строки, а на блоки по 8192 строки, называемые гранулами. Это делает первичные ключи огромных наборов данных достаточно компактными, чтобы оставаться загруженными в оперативной памяти, при этом обеспечивая быстрый доступ к данным на диске. +* Таблицы могут быть разбиты на разделы (партиции) с использованием произвольного выражения секционирования. Исключение разделов (partition pruning) гарантирует, что такие разделы пропускаются при чтении, когда это допускает запрос. -- Таблицы могут быть секционированы (разделены на партиции) с использованием произвольного выражения партиционирования. Отсечение партиций (partition pruning) обеспечивает пропуск чтения партиций, если запрос позволяет это сделать. +* Данные могут реплицироваться между несколькими узлами кластера для обеспечения высокой доступности, отказоустойчивости и обновлений без простоя. См. раздел [Репликация данных](/engines/table-engines/mergetree-family/replication.md). -- Данные могут реплицироваться между несколькими узлами кластера для обеспечения высокой доступности, отказоустойчивости и обновлений без простоя. См. раздел [Data replication](/engines/table-engines/mergetree-family/replication.md). - -- Движки таблиц `MergeTree` поддерживают различные типы статистики и методы семплирования, которые помогают в оптимизации запросов. +* Движки таблиц `MergeTree` поддерживают различные виды статистики и методы выборочного чтения (sampling), помогающие оптимизировать запросы. :::note -Несмотря на схожее название, движок [Merge](/engines/table-engines/special/merge) отличается от движков `*MergeTree`. +Несмотря на похожее название, движок [Merge](/engines/table-engines/special/merge) отличается от движков `*MergeTree`. ::: - - -## Создание таблиц {#table_engine-mergetree-creating-a-table} +## Создание таблиц {#table_engine-mergetree-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,132 +56,127 @@ ORDER BY expr [SETTINGS name = value, ...] ``` -Подробное описание параметров см. в описании оператора [CREATE TABLE](/sql-reference/statements/create/table.md). +Подробное описание параметров см. в описании оператора [CREATE TABLE](/sql-reference/statements/create/table.md) -### Секции запроса {#mergetree-query-clauses} +### Части запроса {#mergetree-query-clauses} #### ENGINE {#engine} -`ENGINE` — имя и параметры движка. `ENGINE = MergeTree()`. Движок `MergeTree` не имеет параметров. +`ENGINE` — имя и параметры движка таблицы. `ENGINE = MergeTree()`. Движок таблицы `MergeTree` не имеет параметров. -#### ORDER BY {#order_by} +#### ORDER BY {#order_by} `ORDER BY` — ключ сортировки. -Кортеж из имен столбцов или произвольных выражений. Пример: `ORDER BY (CounterID + 1, EventDate)`. +Кортеж имён столбцов или произвольных выражений. Пример: `ORDER BY (CounterID + 1, EventDate)`. -Если первичный ключ не определен (т.е. секция `PRIMARY KEY` не указана), ClickHouse использует ключ сортировки в качестве первичного ключа. +Если первичный ключ не определён (то есть `PRIMARY KEY` не был указан), ClickHouse использует ключ сортировки в качестве первичного ключа. Если сортировка не требуется, можно использовать синтаксис `ORDER BY tuple()`. -Альтернативно, если включена настройка `create_table_empty_primary_key_by_default`, секция `ORDER BY ()` неявно добавляется к операторам `CREATE TABLE`. См. [Выбор первичного ключа](#selecting-a-primary-key). +Либо, если включена настройка `create_table_empty_primary_key_by_default`, `ORDER BY ()` неявно добавляется к операторам `CREATE TABLE`. См. раздел [Выбор первичного ключа](#selecting-a-primary-key). #### PARTITION BY {#partition-by} -`PARTITION BY` — [ключ партиционирования](/engines/table-engines/mergetree-family/custom-partitioning-key.md). Необязательный параметр. В большинстве случаев ключ партиционирования не требуется, а если партиционирование необходимо, обычно не нужен ключ более детальный, чем по месяцам. Партиционирование не ускоряет запросы (в отличие от выражения ORDER BY). Не следует использовать слишком детальное партиционирование. Не партиционируйте данные по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). +`PARTITION BY` — [ключ партиционирования](/engines/table-engines/mergetree-family/custom-partitioning-key.md). Необязателен. В большинстве случаев ключ партиционирования не нужен, а если и требуется партиционирование, как правило, нет необходимости использовать ключ с более высокой детализацией, чем по месяцам. Партиционирование не ускоряет выполнение запросов (в отличие от выражения ORDER BY). Никогда не используйте слишком мелкое партиционирование. Не разбивайте данные по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). -Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — столбец с датой типа [Date](/sql-reference/data-types/date.md). Имена партиций в этом случае имеют формат `"YYYYMM"`. +Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — это столбец с датой типа [Date](/sql-reference/data-types/date.md). Имена партиций в этом случае имеют формат `"YYYYMM"`. #### PRIMARY KEY {#primary-key} -`PRIMARY KEY` — первичный ключ, если он [отличается от ключа сортировки](#choosing-a-primary-key-that-differs-from-the-sorting-key). Необязательный параметр. +`PRIMARY KEY` — первичный ключ, если он [отличается от сортировочного ключа](#choosing-a-primary-key-that-differs-from-the-sorting-key). Необязательный параметр. -Указание ключа сортировки (с помощью секции `ORDER BY`) неявно задает первичный ключ. -Обычно нет необходимости указывать первичный ключ дополнительно к ключу сортировки. +Указание сортировочного ключа (с помощью клаузы `ORDER BY`) неявно задаёт первичный ключ. +Обычно нет необходимости указывать первичный ключ дополнительно к сортировочному ключу. #### SAMPLE BY {#sample-by} -`SAMPLE BY` — выражение для сэмплирования. Необязательный параметр. +`SAMPLE BY` — выражение для семплирования (sampling expression). Необязательное выражение. -Если указано, оно должно содержаться в первичном ключе. -Выражение для сэмплирования должно возвращать беззнаковое целое число. +Если указано, оно должно входить в первичный ключ. +Результат этого выражения должен быть беззнаковым целым числом. Пример: `SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`. #### TTL {#ttl} -`TTL` — список правил, определяющих длительность хранения строк и логику автоматического перемещения частей [между дисками и томами](#table_engine-mergetree-multiple-volumes). Необязательный параметр. +`TTL` — список правил, которые задают срок хранения строк и логику автоматического перемещения частей [между дисками и томами](#table_engine-mergetree-multiple-volumes). Необязательный параметр. Выражение должно возвращать `Date` или `DateTime`, например, `TTL date + INTERVAL 1 DAY`. - -Тип правила `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` определяет действие, которое будет выполнено с частью данных при выполнении условия выражения (достижении текущего времени): удаление устаревших строк, перемещение части (если выражение выполнено для всех строк в части) на указанный диск (`TO DISK 'xxx'`) или том (`TO VOLUME 'xxx'`), либо агрегирование значений в устаревших строках. Тип правила по умолчанию — удаление (`DELETE`). Можно указать список из нескольких правил, но должно быть не более одного правила `DELETE`. +Тип правила `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` определяет действие, которое выполняется с частью, если выражение удовлетворяется (достигает текущего времени): удаление истёкших строк, перемещение части (если выражение выполняется для всех строк в части) на указанный диск (`TO DISK 'xxx'`) или на том (`TO VOLUME 'xxx'`), либо агрегация значений в истёкших строках. Тип правила по умолчанию — удаление (`DELETE`). Можно задать список из нескольких правил, но не более одного правила `DELETE`. Подробнее см. [TTL для столбцов и таблиц](#table_engine-mergetree-ttl) -#### SETTINGS {#settings} +#### ПАРАМЕТРЫ {#settings} -См. [Настройки MergeTree](../../../operations/settings/merge-tree-settings.md). +См. [настройки MergeTree](../../../operations/settings/merge-tree-settings.md). -**Пример настройки секций** +**Пример настройки параметра sections** ```sql ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity=8192 ``` -В примере задано партиционирование по месяцам. +В этом примере мы задаём секционирование по месяцам. -Также задано выражение для сэмплирования в виде хеша по идентификатору пользователя. Это позволяет псевдослучайным образом перемешать данные в таблице для каждого `CounterID` и `EventDate`. Если при выборке данных указать секцию [SAMPLE](/sql-reference/statements/select/sample), ClickHouse вернёт равномерную псевдослучайную выборку данных для подмножества пользователей. +Мы также задаём выражение для выборочного чтения данных в виде хэша по ID пользователя. Это позволяет псевдослучайно распределить данные в таблице для каждого `CounterID` и `EventDate`. Если вы укажете предложение [SAMPLE](/sql-reference/statements/select/sample) при выборке данных, ClickHouse вернёт равномерную псевдослучайную выборку данных для подмножества пользователей. -Настройку `index_granularity` можно опустить, так как 8192 является значением по умолчанию. +Параметр `index_granularity` можно опустить, так как 8192 — это значение по умолчанию.
+ Устаревший метод создания таблицы -Устаревший метод создания таблицы - -:::note -Не используйте этот метод в новых проектах. По возможности переведите старые проекты на метод, описанный выше. -::: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) -``` + :::note + Не используйте этот метод в новых проектах. По возможности переведите старые проекты на метод, описанный выше. + ::: -**Параметры MergeTree()** + ```sql + CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] + ( + name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], + name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], + ... + ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) + ``` -- `date-column` — Имя столбца типа [Date](/sql-reference/data-types/date.md). ClickHouse автоматически создаёт партиции по месяцам на основе этого столбца. Имена партиций имеют формат `"YYYYMM"`. -- `sampling_expression` — Выражение для сэмплирования. -- `(primary, key)` — Первичный ключ. Тип: [Tuple()](/sql-reference/data-types/tuple.md) -- `index_granularity` — Гранулярность индекса. Количество строк данных между «метками» индекса. Значение 8192 подходит для большинства задач. + **Параметры MergeTree()** -**Пример** + * `date-column` — Имя столбца типа [Date](/sql-reference/data-types/date.md). ClickHouse автоматически создаёт партиции по месяцам на основе этого столбца. Имена партиций имеют формат `"YYYYMM"`. + * `sampling_expression` — Выражение для выборочного чтения данных. + * `(primary, key)` — Первичный ключ. Тип: [Tuple()](/sql-reference/data-types/tuple.md) + * `index_granularity` — Гранулярность индекса. Количество строк данных между «метками» индекса. Значение 8192 подходит для большинства задач. -```sql -MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) -``` + **Пример** -Движок `MergeTree` настраивается так же, как в примере выше для основного метода конфигурации движка. + ```sql + MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) + ``` + Движок `MergeTree` настраивается так же, как в примере выше для основного метода конфигурации движка.
- ## Хранение данных {#mergetree-data-storage} -Таблица состоит из кусков данных, отсортированных по первичному ключу. - -При вставке данных в таблицу создаются отдельные куски данных, каждый из которых лексикографически сортируется по первичному ключу. Например, если первичный ключ — `(CounterID, Date)`, данные в куске сортируются по `CounterID`, а внутри каждого `CounterID` упорядочиваются по `Date`. +Таблица состоит из частей данных, отсортированных по первичному ключу. -Данные, относящиеся к разным партициям, разделяются на разные куски. В фоновом режиме ClickHouse объединяет куски данных для более эффективного хранения. Куски, принадлежащие разным партициям, не объединяются. Механизм слияния не гарантирует, что все строки с одинаковым первичным ключом окажутся в одном куске данных. +При вставке данных в таблицу создаются отдельные части данных, и каждая из них лексикографически сортируется по первичному ключу. Например, если первичный ключ — `(CounterID, Date)`, данные в части сортируются по `CounterID`, а внутри каждого `CounterID` упорядочиваются по `Date`. -Куски данных могут храниться в формате `Wide` или `Compact`. В формате `Wide` каждый столбец хранится в отдельном файле файловой системы, в формате `Compact` все столбцы хранятся в одном файле. Формат `Compact` может использоваться для повышения производительности при небольших и частых вставках. +Данные, принадлежащие разным партициям, разделяются на отдельные части. В фоновом режиме ClickHouse сливает части данных для более эффективного хранения. Части, принадлежащие разным партициям, не сливаются. Механизм слияния не гарантирует, что все строки с одинаковым первичным ключом окажутся в одной и той же части. -Формат хранения данных управляется настройками движка таблицы `min_bytes_for_wide_part` и `min_rows_for_wide_part`. Если количество байтов или строк в куске данных меньше значения соответствующей настройки, кусок хранится в формате `Compact`. В противном случае он хранится в формате `Wide`. Если ни одна из этих настроек не задана, куски данных хранятся в формате `Wide`. +Части данных могут храниться в форматах `Wide` или `Compact`. В формате `Wide` каждый столбец хранится в отдельном файле в файловой системе, в формате `Compact` все столбцы хранятся в одном файле. Формат `Compact` может использоваться для повышения производительности при небольших и частых вставках. -Каждый кусок данных логически разделён на гранулы. Гранула — это наименьший неделимый набор данных, который ClickHouse читает при выборке данных. ClickHouse не разделяет строки или значения, поэтому каждая гранула всегда содержит целое число строк. Первая строка гранулы отмечается значением первичного ключа для этой строки. Для каждого куска данных ClickHouse создаёт индексный файл, в котором хранятся метки. Для каждого столбца, независимо от того, входит ли он в первичный ключ или нет, ClickHouse также сохраняет те же метки. Эти метки позволяют находить данные непосредственно в файлах столбцов. +Формат хранения данных контролируется параметрами движка таблицы `min_bytes_for_wide_part` и `min_rows_for_wide_part`. Если количество байт или строк в части данных меньше соответствующего значения параметра, часть хранится в формате `Compact`. В противном случае она хранится в формате `Wide`. Если ни один из этих параметров не задан, части данных хранятся в формате `Wide`. -Размер гранулы ограничивается настройками движка таблицы `index_granularity` и `index_granularity_bytes`. Количество строк в грануле находится в диапазоне `[1, index_granularity]` в зависимости от размера строк. Размер гранулы может превышать `index_granularity_bytes`, если размер одной строки больше значения настройки. В этом случае размер гранулы равен размеру строки. +Каждая часть данных логически разделена на гранулы. Гранула — это наименьший неделимый набор данных, который ClickHouse читает при выборке. ClickHouse не разбивает строки или значения, поэтому каждая гранула всегда содержит целое число строк. Первая строка гранулы помечается значением первичного ключа для этой строки. Для каждой части данных ClickHouse создает файл индекса, в котором хранятся эти метки. Для каждого столбца, независимо от того, входит он в первичный ключ или нет, ClickHouse также хранит те же метки. Эти метки позволяют находить данные непосредственно в файлах столбцов. +Размер гранулы ограничивается параметрами движка таблицы `index_granularity` и `index_granularity_bytes`. Число строк в грануле находится в диапазоне `[1, index_granularity]` и зависит от размера строк. Размер гранулы может превышать `index_granularity_bytes`, если размер одной строки больше значения этого параметра. В этом случае размер гранулы равен размеру строки. ## Первичные ключи и индексы в запросах {#primary-keys-and-indexes-in-queries} -Рассмотрим первичный ключ `(CounterID, Date)` в качестве примера. В этом случае сортировка и индекс могут быть проиллюстрированы следующим образом: +Рассмотрим в качестве примера первичный ключ `(CounterID, Date)`. В этом случае сортировку и индекс можно представить следующим образом: ```text -Все данные: [---------------------------------------------] +Все данные: [---------------------------------------------] CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] Метки: | | | | | | | | | | | @@ -192,63 +184,63 @@ Date: [111111122222223333123321111122222233321111111212222222311111222 Номера меток: 0 1 2 3 4 5 6 7 8 9 10 ``` -Если в запросе данных указано: +Если в запросе к данным указано: -- `CounterID in ('a', 'h')`, сервер читает данные в диапазонах меток `[0, 3)` и `[6, 8)`. -- `CounterID IN ('a', 'h') AND Date = 3`, сервер читает данные в диапазонах меток `[1, 3)` и `[7, 8)`. -- `Date = 3`, сервер читает данные в диапазоне меток `[1, 10]`. +* `CounterID in ('a', 'h')`, сервер читает данные в диапазонах меток `[0, 3)` и `[6, 8)`. +* `CounterID IN ('a', 'h') AND Date = 3`, сервер читает данные в диапазонах меток `[1, 3)` и `[7, 8)`. +* `Date = 3`, сервер читает данные в диапазоне меток `[1, 10]`. -Приведенные выше примеры показывают, что использование индекса всегда эффективнее полного сканирования. +Приведённые выше примеры показывают, что использование индекса всегда эффективнее, чем полное сканирование. -Разреженный индекс допускает чтение дополнительных данных. При чтении одного диапазона первичного ключа может быть прочитано до `index_granularity * 2` дополнительных строк в каждом блоке данных. +Разреженный индекс допускает чтение лишних данных. При чтении одного диапазона первичного ключа в каждом блоке данных может быть прочитано до `index_granularity * 2` дополнительных строк. -Разреженные индексы позволяют работать с очень большим количеством строк таблицы, поскольку в большинстве случаев такие индексы помещаются в оперативной памяти компьютера. +Разреженные индексы позволяют работать с очень большим числом строк в таблице, потому что в большинстве случаев такие индексы помещаются в оперативную память. -ClickHouse не требует уникальности первичного ключа. Вы можете вставлять несколько строк с одинаковым первичным ключом. +ClickHouse не требует уникального первичного ключа. Вы можете вставлять несколько строк с одинаковым первичным ключом. -Вы можете использовать выражения типа `Nullable` в секциях `PRIMARY KEY` и `ORDER BY`, но это крайне не рекомендуется. Чтобы разрешить эту возможность, включите настройку [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key). Для значений `NULL` в секции `ORDER BY` применяется принцип [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values). +Вы можете использовать выражения типа `Nullable` в выражениях `PRIMARY KEY` и `ORDER BY`, но это настоятельно не рекомендуется. Чтобы включить эту возможность, активируйте настройку [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key). Принцип [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) применяется к значениям `NULL` в выражении `ORDER BY`. ### Выбор первичного ключа {#selecting-a-primary-key} -Количество столбцов в первичном ключе явно не ограничено. В зависимости от структуры данных вы можете включить в первичный ключ больше или меньше столбцов. Это может: +Количество столбцов в первичном ключе явно не ограничено. В зависимости от структуры данных вы можете включать больше или меньше столбцов в первичный ключ. Это может: -- Улучшить производительность индекса. +* Повысить производительность индекса. - Если первичный ключ — это `(a, b)`, то добавление еще одного столбца `c` улучшит производительность, если выполняются следующие условия: - - Существуют запросы с условием на столбец `c`. - - Длинные диапазоны данных (в несколько раз длиннее, чем `index_granularity`) с идентичными значениями для `(a, b)` встречаются часто. Другими словами, когда добавление еще одного столбца позволяет пропускать довольно длинные диапазоны данных. + Если первичный ключ — `(a, b)`, то добавление дополнительного столбца `c` улучшит производительность, если выполняются следующие условия: -- Улучшить сжатие данных. + * Есть запросы с условием по столбцу `c`. + * Длинные диапазоны данных (в несколько раз длиннее, чем `index_granularity`) с одинаковыми значениями для `(a, b)` встречаются часто. Другими словами, добавление еще одного столбца позволяет пропускать достаточно длинные диапазоны данных. - ClickHouse сортирует данные по первичному ключу, поэтому чем выше согласованность данных, тем лучше сжатие. +* Улучшить сжатие данных. -- Обеспечить дополнительную логику при слиянии частей данных в движках [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) и [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md). + ClickHouse сортирует данные по первичному ключу, поэтому чем выше упорядоченность, тем лучше сжатие. - В этом случае имеет смысл указать _ключ сортировки_, отличающийся от первичного ключа. +* Обеспечить дополнительную логику при слиянии частей данных в движках [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) и [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md). -Длинный первичный ключ негативно повлияет на производительность вставки и потребление памяти, но дополнительные столбцы в первичном ключе не влияют на производительность ClickHouse при выполнении запросов `SELECT`. + В этом случае имеет смысл указать *ключ сортировки*, отличающийся от первичного ключа. -Вы можете создать таблицу без первичного ключа, используя синтаксис `ORDER BY tuple()`. В этом случае ClickHouse хранит данные в порядке вставки. Если вы хотите сохранить порядок данных при вставке данных запросами `INSERT ... SELECT`, установите [max_insert_threads = 1](/operations/settings/settings#max_insert_threads). +Длинный первичный ключ негативно влияет на производительность операций вставки и потребление памяти, но дополнительные столбцы в первичном ключе не влияют на производительность ClickHouse при выполнении `SELECT`‑запросов. -Для выборки данных в исходном порядке используйте [однопоточные](/operations/settings/settings.md/#max_threads) запросы `SELECT`. +Вы можете создать таблицу без первичного ключа, используя синтаксис `ORDER BY tuple()`. В этом случае ClickHouse хранит данные в порядке вставки. Если вы хотите сохранить порядок данных при вставке через запросы `INSERT ... SELECT`, установите [max_insert_threads = 1](/operations/settings/settings#max_insert_threads). -### Выбор первичного ключа, отличающегося от ключа сортировки {#choosing-a-primary-key-that-differs-from-the-sorting-key} +Чтобы выбирать данные в исходном порядке, используйте [однопоточные](/operations/settings/settings.md/#max_threads) `SELECT`‑запросы. +### Выбор первичного ключа, отличного от ключа сортировки {#choosing-a-primary-key-that-differs-from-the-sorting-key} -Возможно указать первичный ключ (выражение, значения которого записываются в индексный файл для каждой отметки), отличающийся от ключа сортировки (выражения для сортировки строк в частях данных). В этом случае кортеж выражения первичного ключа должен быть префиксом кортежа выражения ключа сортировки. +Можно задать первичный ключ (выражение со значениями, которые записываются в файл индекса для каждой метки), отличающийся от ключа сортировки (выражение для сортировки строк в частях данных). В этом случае кортеж выражений первичного ключа должен быть префиксом кортежа выражений ключа сортировки. Эта возможность полезна при использовании движков таблиц [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) и -[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md). В типичном случае при использовании этих движков таблица имеет два типа столбцов: _измерения_ и _метрики_. Типичные запросы агрегируют значения столбцов метрик с произвольным `GROUP BY` и фильтрацией по измерениям. Поскольку SummingMergeTree и AggregatingMergeTree агрегируют строки с одинаковым значением ключа сортировки, естественно добавить в него все измерения. В результате выражение ключа состоит из длинного списка столбцов, и этот список необходимо часто обновлять при добавлении новых измерений. +[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md). В типичном случае при использовании этих движков таблица содержит два типа столбцов: *измерения* и *показатели*. Типичные запросы агрегируют значения столбцов-показателей с произвольным `GROUP BY` и фильтрацией по измерениям. Поскольку SummingMergeTree и AggregatingMergeTree агрегируют строки с одинаковым значением ключа сортировки, естественно включить в него все измерения. В результате выражение ключа состоит из длинного списка столбцов, и этот список необходимо часто обновлять при добавлении новых измерений. -В этом случае имеет смысл оставить в первичном ключе только несколько столбцов, которые обеспечат эффективное сканирование диапазонов, и добавить остальные столбцы измерений в кортеж ключа сортировки. +В этом случае имеет смысл оставить в первичном ключе только несколько столбцов, которые обеспечат эффективное диапазонное сканирование, а оставшиеся столбцы-измерения добавить в кортеж ключа сортировки. -[ALTER](/sql-reference/statements/alter/index.md) ключа сортировки является лёгкой операцией, поскольку при одновременном добавлении нового столбца в таблицу и в ключ сортировки существующие части данных не требуют изменения. Так как старый ключ сортировки является префиксом нового ключа сортировки, а во вновь добавленном столбце нет данных, данные отсортированы как по старому, так и по новому ключу сортировки в момент модификации таблицы. +[ALTER](/sql-reference/statements/alter/index.md) ключа сортировки — это лёгкая операция, потому что когда новый столбец одновременно добавляется в таблицу и в ключ сортировки, существующие части данных не нужно изменять. Поскольку старый ключ сортировки является префиксом нового ключа сортировки и в только что добавленном столбце ещё нет данных, данные на момент изменения таблицы отсортированы как по старому, так и по новому ключам сортировки. ### Использование индексов и партиций в запросах {#use-of-indexes-and-partitions-in-queries} -Для запросов `SELECT` ClickHouse анализирует, может ли быть использован индекс. Индекс может быть использован, если в секции `WHERE/PREWHERE` присутствует выражение (как один из элементов конъюнкции или полностью), представляющее операцию сравнения на равенство или неравенство, или если оно содержит `IN` или `LIKE` с фиксированным префиксом для столбцов или выражений, которые входят в первичный ключ или ключ партиционирования, или для определённых частично повторяющихся функций этих столбцов, или логических отношений этих выражений. +Для запросов `SELECT` ClickHouse анализирует, может ли быть использован индекс. Индекс может быть использован, если предложение `WHERE/PREWHERE` содержит выражение (как один из элементов конъюнкции или целиком), представляющее собой операцию сравнения на равенство или неравенство, или если оно содержит `IN` или `LIKE` с фиксированным префиксом по столбцам или выражениям, входящим в первичный ключ или ключ партиционирования, или по определённым частично повторяющимся функциям этих столбцов, или логические комбинации этих выражений. -Таким образом, возможно быстро выполнять запросы по одному или нескольким диапазонам первичного ключа. В этом примере запросы будут выполняться быстро для конкретного тега отслеживания, для конкретного тега и диапазона дат, для конкретного тега и даты, для нескольких тегов с диапазоном дат и так далее. +Таким образом, можно быстро выполнять запросы по одному или нескольким диапазонам первичного ключа. В этом примере запросы будут выполняться быстро при выборке по конкретному тегу отслеживания, по конкретному тегу и диапазону дат, по конкретному тегу и дате, по нескольким тегам с диапазоном дат и так далее. Рассмотрим движок, настроенный следующим образом: @@ -259,7 +251,7 @@ ORDER BY (CounterID, EventDate) SETTINGS index_granularity=8192 ``` -В этом случае в запросах: +В таком случае в запросах: ```sql SELECT count() FROM table @@ -277,42 +269,41 @@ AND CounterID IN (101500, 731962, 160656) AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) ``` -ClickHouse будет использовать индекс первичного ключа для отсечения неподходящих данных и ключ месячного партиционирования для отсечения партиций, находящихся в неподходящих диапазонах дат. +ClickHouse будет использовать индекс по первичному ключу для отсечения нерелевантных данных и ежемесячный ключ партиционирования для отсечения партиций, попадающих в неподходящие диапазоны дат. -Приведённые выше запросы показывают, что индекс используется даже для сложных выражений. Чтение из таблицы организовано таким образом, что использование индекса не может быть медленнее полного сканирования. +Приведённые выше запросы показывают, что индекс используется даже для сложных выражений. Чтение из таблицы организовано так, что использование индекса не может быть медленнее полного сканирования. -В примере ниже индекс не может быть использован. +В приведённом ниже примере индекс использоваться не будет. ```sql SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' ``` -Чтобы проверить, может ли ClickHouse использовать индекс при выполнении запроса, используйте настройки [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) и [force_primary_key](/operations/settings/settings#force_primary_key). +Чтобы проверить, может ли ClickHouse использовать индекс при выполнении запроса, используйте настройки [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) и [force_primary_key](/operations/settings/settings#force_primary_key). -Ключ партиционирования по месяцам позволяет читать только те блоки данных, которые содержат даты из соответствующего диапазона. В этом случае блок данных может содержать данные для многих дат (до целого месяца). Внутри блока данные отсортированы по первичному ключу, который может не содержать дату в качестве первого столбца. Из-за этого использование запроса только с условием по дате, не указывающим префикс первичного ключа, приведёт к чтению большего объёма данных, чем для одной даты. +Ключ партиционирования по месяцам позволяет читать только те блоки данных, которые содержат даты из нужного диапазона. В этом случае блок данных может содержать данные за множество дат (вплоть до целого месяца). Внутри блока данные отсортированы по первичному ключу, который может не содержать дату в качестве первого столбца. Из-за этого использование запроса только с условием по дате, без указания префикса первичного ключа, приведёт к чтению большего объёма данных, чем при выборке за одну дату. -### Использование индекса для частично монотонных первичных ключей {#use-of-index-for-partially-monotonic-primary-keys} +### Использование индекса для частично-монотонных первичных ключей {#use-of-index-for-partially-monotonic-primary-keys} +Рассмотрим, например, дни месяца. Они образуют [монотонную последовательность](https://en.wikipedia.org/wiki/Monotonic_function) в пределах одного месяца, но не являются монотонными на более длительных промежутках времени. Это частично-монотонная последовательность. Если пользователь создаёт таблицу с частично-монотонным первичным ключом, ClickHouse создаёт разреженный индекс как обычно. Когда пользователь выбирает данные из такой таблицы, ClickHouse анализирует условия запроса. Если пользователь хочет получить данные между двумя метками индекса и обе эти метки попадают в один месяц, ClickHouse может использовать индекс в этом конкретном случае, потому что он может вычислить расстояние между параметрами запроса и метками индекса. -Рассмотрим, например, дни месяца. Они образуют [монотонную последовательность](https://en.wikipedia.org/wiki/Monotonic_function) в пределах одного месяца, но не являются монотонными для более длительных периодов. Это частично монотонная последовательность. Если пользователь создает таблицу с частично монотонным первичным ключом, ClickHouse создает разреженный индекс как обычно. Когда пользователь выбирает данные из такой таблицы, ClickHouse анализирует условия запроса. Если пользователь хочет получить данные между двумя отметками индекса и обе эти отметки попадают в пределы одного месяца, ClickHouse может использовать индекс в этом конкретном случае, поскольку он может вычислить расстояние между параметрами запроса и отметками индекса. +ClickHouse не может использовать индекс, если значения первичного ключа в заданном в параметрах запроса диапазоне не образуют монотонную последовательность. В этом случае ClickHouse использует полное сканирование. -ClickHouse не может использовать индекс, если значения первичного ключа в диапазоне параметров запроса не представляют собой монотонную последовательность. В этом случае ClickHouse использует метод полного сканирования. +ClickHouse применяет эту логику не только к последовательностям дней месяца, но и к любому первичному ключу, который представляет частично-монотонную последовательность. -ClickHouse использует эту логику не только для последовательностей дней месяца, но и для любого первичного ключа, который представляет собой частично монотонную последовательность. +### Индексы пропуска данных {#table_engine-mergetree-data_skipping-indexes} -### Индексы пропуска данных {#table_engine-mergetree-data_skipping-indexes} - -Объявление индекса находится в секции столбцов запроса `CREATE`. +Объявление индекса указывается в разделе `COLUMNS` оператора `CREATE`. ```sql INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] ``` -Для таблиц семейства `*MergeTree` можно указывать индексы пропуска данных. +Для таблиц из семейства `*MergeTree` можно задать индексы пропуска данных (data skipping indices). -Эти индексы агрегируют некоторую информацию о заданном выражении на блоках, которые состоят из `granularity_value` гранул (размер гранулы задается с помощью настройки `index_granularity` в движке таблицы). Затем эти агрегаты используются в запросах `SELECT` для уменьшения объема данных, считываемых с диска, путем пропуска больших блоков данных, где условие `where` не может быть выполнено. +Эти индексы агрегируют некоторую информацию об указанном выражении по блокам, которые состоят из гранул размера `granularity_value` (размер гранулы задаётся с помощью настройки `index_granularity` в движке таблицы). Затем эти агрегаты используются в запросах `SELECT` для уменьшения объёма данных, считываемых с диска, за счёт пропуска крупных блоков данных, в которых условие секции `WHERE` не может быть выполнено. -Предложение `GRANULARITY` может быть опущено, значение `granularity_value` по умолчанию равно 1. +Секцию `GRANULARITY` можно опустить, значение `granularity_value` по умолчанию равно 1. **Пример** @@ -330,7 +321,7 @@ CREATE TABLE table_name ... ``` -Индексы из примера могут использоваться ClickHouse для уменьшения объема данных, считываемых с диска, в следующих запросах: +ClickHouse может использовать индексы из примера, чтобы сократить объём данных, считываемых с диска, в следующих запросах: ```sql SELECT count() FROM table WHERE u64 == 10; @@ -338,106 +329,105 @@ SELECT count() FROM table WHERE u64 * i32 >= 1234 SELECT count() FROM table WHERE u64 * length(s) == 1234 ``` -Индексы пропуска данных также могут быть созданы на составных столбцах: +Индексы пропуска данных также могут создаваться для составных столбцов: ```sql --- на столбцах типа Map: +-- для столбцов типа Map: INDEX map_key_index mapKeys(map_column) TYPE bloom_filter INDEX map_value_index mapValues(map_column) TYPE bloom_filter --- на столбцах типа Tuple: +-- для столбцов типа Tuple: INDEX tuple_1_index tuple_column.1 TYPE bloom_filter INDEX tuple_2_index tuple_column.2 TYPE bloom_filter --- на столбцах типа Nested: +-- для столбцов типа Nested: INDEX nested_1_index col.nested_col1 TYPE bloom_filter INDEX nested_2_index col.nested_col2 TYPE bloom_filter ``` -### Типы индексов пропуска {#skip-index-types} +### Типы пропускающих индексов {#skip-index-types} -Движок таблиц `MergeTree` поддерживает следующие типы индексов пропуска. -Для получения дополнительной информации о том, как индексы пропуска могут использоваться для оптимизации производительности, -см. [«Понимание индексов пропуска данных ClickHouse»](/optimize/skipping-indexes). +Движок таблицы `MergeTree` поддерживает следующие типы пропускающих индексов. +Подробнее об использовании пропускающих индексов для оптимизации производительности +см. в разделе ["Понимание пропускающих индексов данных в ClickHouse"](/optimize/skipping-indexes). -- Индекс [`MinMax`](#minmax) -- Индекс [`Set`](#set) -- Индекс [`bloom_filter`](#bloom-filter) -- Индекс [`ngrambf_v1`](#n-gram-bloom-filter) -- Индекс [`tokenbf_v1`](#token-bloom-filter) +* индекс [`MinMax`](#minmax) +* индекс [`Set`](#set) +* индекс [`bloom_filter`](#bloom-filter) +* индекс [`ngrambf_v1`](#n-gram-bloom-filter) +* индекс [`tokenbf_v1`](#token-bloom-filter) -#### Индекс пропуска MinMax {#minmax} +#### Индекс MinMax {#minmax} -Для каждой гранулы индекса сохраняются минимальное и максимальное значения выражения. -(Если выражение имеет тип `tuple`, сохраняются минимум и максимум для каждого элемента кортежа.) +Для каждой гранулы индекса сохраняются минимальные и максимальные значения выражения. +(Если выражение имеет тип `tuple`, сохраняются минимальные и максимальные значения для каждого элемента кортежа.) -```text title="Синтаксис" +```text title="Syntax" minmax ``` #### Set {#set} Для каждой гранулы индекса сохраняется не более `max_rows` уникальных значений указанного выражения. -`max_rows = 0` означает «сохранять все уникальные значения». +`max_rows = 0` означает «хранить все уникальные значения». -```text title="Синтаксис" +```text title="Syntax" set(max_rows) ``` -#### Bloom filter {#bloom-filter} +#### Фильтр Блума {#bloom-filter} -Для каждой гранулы индекса сохраняется [фильтр Блума](https://en.wikipedia.org/wiki/Bloom_filter) для указанных столбцов. +Для каждой гранулы индекса хранится [фильтр Блума](https://en.wikipedia.org/wiki/Bloom_filter) по указанным столбцам. -```text title="Синтаксис" +```text title="Syntax" bloom_filter([false_positive_rate]) ``` -Параметр `false_positive_rate` может принимать значение от 0 до 1 (по умолчанию: `0.025`) и задает вероятность ложноположительного срабатывания (что увеличивает объем считываемых данных). - +Параметр `false_positive_rate` может принимать значение от 0 до 1 (по умолчанию: `0.025`) и задаёт вероятность положительного срабатывания (что увеличивает объём считываемых данных). Поддерживаются следующие типы данных: -- `(U)Int*` -- `Float*` -- `Enum` -- `Date` -- `DateTime` -- `String` -- `FixedString` -- `Array` -- `LowCardinality` -- `Nullable` -- `UUID` -- `Map` +* `(U)Int*` +* `Float*` +* `Enum` +* `Date` +* `DateTime` +* `String` +* `FixedString` +* `Array` +* `LowCardinality` +* `Nullable` +* `UUID` +* `Map` :::note Тип данных Map: указание создания индекса по ключам или значениям -Для типа данных `Map` можно указать, должен ли индекс создаваться по ключам или по значениям, используя функции [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) или [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues). +Для типа данных `Map` клиент может указать, должен ли индекс создаваться по ключам или по значениям, с помощью функций [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) или [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues). ::: -#### N-граммный фильтр Блума {#n-gram-bloom-filter} +#### N-граммовый Bloom-фильтр {#n-gram-bloom-filter} -Для каждой гранулы индекса сохраняется [фильтр Блума](https://en.wikipedia.org/wiki/Bloom_filter) для [n-грамм](https://en.wikipedia.org/wiki/N-gram) указанных столбцов. +Для каждой гранулы индекса хранится [Bloom-фильтр](https://en.wikipedia.org/wiki/Bloom_filter) по [n-граммам](https://en.wikipedia.org/wiki/N-gram) указанных столбцов. -```text title="Синтаксис" +```text title="Syntax" ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` -| Параметр | Описание | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `n` | Размер n-граммы | -| `size_of_bloom_filter_in_bytes` | Размер фильтра Блума в байтах. Здесь можно использовать большое значение, например `256` или `512`, так как оно хорошо сжимается. | -| `number_of_hash_functions` | Количество хеш-функций, используемых в фильтре Блума. | -| `random_seed` | Начальное значение для хеш-функций фильтра Блума. | +| Parameter | Description | +| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `n` | размер n-граммы | +| `size_of_bloom_filter_in_bytes` | Размер фильтра Блума в байтах. Здесь можно использовать большое значение, например `256` или `512`, поскольку оно хорошо сжимается. | +| `number_of_hash_functions` | Количество хеш-функций, используемых в фильтре Блума. | +| `random_seed` | Начальное значение (seed) для хеш-функций фильтра Блума. | Этот индекс работает только со следующими типами данных: -- [`String`](/sql-reference/data-types/string.md) -- [`FixedString`](/sql-reference/data-types/fixedstring.md) -- [`Map`](/sql-reference/data-types/map.md) +* [`String`](/sql-reference/data-types/string.md) +* [`FixedString`](/sql-reference/data-types/fixedstring.md) +* [`Map`](/sql-reference/data-types/map.md) -Для оценки параметров `ngrambf_v1` можно использовать следующие [пользовательские функции (UDF)](/sql-reference/statements/create/function.md). +Чтобы оценить параметры `ngrambf_v1`, вы можете использовать следующие [пользовательские функции (UDF)](/sql-reference/statements/create/function.md). -```sql title="UDF для ngrambf_v1" +```sql title="UDFs for ngrambf_v1" CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] AS (total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); @@ -455,23 +445,23 @@ AS (number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) ``` -Для использования этих функций необходимо указать как минимум два параметра: +Чтобы использовать эти функции, необходимо указать не менее двух параметров: -- `total_number_of_all_grams` -- `probability_of_false_positives` +* `total_number_of_all_grams` +* `probability_of_false_positives` -Например, в грануле содержится `4300` n-грамм, и вы ожидаете, что вероятность ложных срабатываний будет меньше `0.0001`. -Остальные параметры можно оценить, выполнив следующие запросы: +Например, в грануле есть `4300` n-грамм, и вы ожидаете, что вероятность ложных срабатываний будет меньше `0.0001`. +Остальные параметры можно затем оценить, выполнив следующие запросы: ```sql ---- оценка количества битов в фильтре +--- оценить количество битов в фильтре SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; ┌─size_of_bloom_filter_in_bytes─┐ │ 10304 │ └───────────────────────────────┘ ---- оценка количества хеш-функций +--- оценить количество хеш-функций SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_hash_functions ┌─number_of_hash_functions─┐ @@ -479,103 +469,97 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_ha └──────────────────────────┘ ``` -Разумеется, эти функции также можно использовать для оценки параметров при других условиях. -Приведенные выше функции основаны на калькуляторе фильтра Блума, доступном [здесь](https://hur.st/bloomfilter). +Разумеется, вы также можете использовать эти функции для оценки параметров и в других условиях. +Приведённые выше функции соответствуют калькулятору фильтра Блума, доступному по адресу [здесь](https://hur.st/bloomfilter). -#### Токенный фильтр Блума {#token-bloom-filter} +#### Фильтр Блума по токенам {#token-bloom-filter} -Токенный фильтр Блума аналогичен `ngrambf_v1`, но сохраняет токены (последовательности, разделенные неалфавитно-цифровыми символами) вместо n-грамм. +Фильтр Блума по токенам аналогичен `ngrambf_v1`, но вместо n-грамм хранит токены (последовательности, разделённые небуквенно-цифровыми символами). -```text title="Синтаксис" +```text title="Syntax" tokenbf_v1(size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` +#### Разрежённый n-граммный фильтр Блума {#sparse-grams-bloom-filter} -#### Фильтр Блума на основе разреженных грамм {#sparse-grams-bloom-filter} +Разрежённый n-граммный фильтр Блума аналогичен `ngrambf_v1`, но использует [токены разрежённых n-грамм](/sql-reference/functions/string-functions.md/#sparseGrams) вместо n-грамм. -Фильтр Блума на основе разреженных грамм аналогичен `ngrambf_v1`, но использует [токены разреженных грамм](/sql-reference/functions/string-functions.md/#sparseGrams) вместо n-грамм. - -```text title="Синтаксис" +```text title="Syntax" sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` ### Текстовый индекс {#text} -Поддерживает полнотекстовый поиск, подробности см. [здесь](invertedindexes.md). +Поддерживает полнотекстовый поиск; подробности см. [здесь](invertedindexes.md). -#### Векторное сходство {#vector-similarity} +#### Сходство векторов {#vector-similarity} -Поддерживает приближённый поиск ближайших соседей, подробности см. [здесь](annindexes.md). +Поддерживает приближённый поиск ближайших соседей, подробнее см. [здесь](annindexes.md). ### Поддержка функций {#functions-support} -Условия в секции `WHERE` содержат вызовы функций, работающих со столбцами. Если столбец является частью индекса, ClickHouse пытается использовать этот индекс при выполнении функций. ClickHouse поддерживает различные подмножества функций для использования индексов. - -Индексы типа `set` могут использоваться всеми функциями. Другие типы индексов поддерживаются следующим образом: - - -| Функция (оператор) / индекс | первичный ключ | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | text | sparse_grams | -| ------------------------------------------------------------------------------------------------------------------------- | -------------- | ------ | -------------- | -------------- | ---------------- | ---- | ---------------- | -| [equals (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [меньше (`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [больше (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | - - - -Функции с константным аргументом, который меньше размера n-граммы, не могут использоваться индексом `ngrambf_v1` для оптимизации запросов. - -(*) Чтобы `hasTokenCaseInsensitive` и `hasTokenCaseInsensitiveOrNull` были эффективны, индекс `tokenbf_v1` должен быть создан по данным в нижнем регистре, например `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`. +Условия в предложении `WHERE` содержат вызовы функций, которые работают со столбцами. Если столбец является частью индекса, ClickHouse пытается использовать этот индекс при вычислении этих функций. ClickHouse поддерживает различные подмножества функций для работы с индексами. + +Индексы типа `set` могут использоваться всеми функциями. Остальные типы индексов поддерживаются следующим образом: + +| Функция (оператор) / Индекс | первичный ключ | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | текст | +| ------------------------------------------------------------------------------------------------------------------------- | -------------- | ------ | -------------- | -------------- | ---------------- | ---------------- | ----- | +| [равно (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | +| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [меньше (`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [greater (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | +| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | + +Функции с константным аргументом, значение которого меньше размера n-граммы, не могут использоваться индексом `ngrambf_v1` для оптимизации запросов. + +(*) Чтобы `hasTokenCaseInsensitive` и `hasTokenCaseInsensitiveOrNull` были эффективны, индекс `tokenbf_v1` должен быть создан по данным в нижнем регистре, например: `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`. :::note -Фильтры Блума могут давать ложноположительные совпадения, поэтому индексы `ngrambf_v1`, `tokenbf_v1`, `sparse_grams` и `bloom_filter` не могут использоваться для оптимизации запросов, в которых ожидается, что результат функции будет ложным. +У фильтров Блума возможны ложноположительные срабатывания, поэтому индексы `ngrambf_v1`, `tokenbf_v1`, `sparse_grams` и `bloom_filter` не могут использоваться для оптимизации запросов, в которых ожидается, что результат функции будет ложным. Например: -- Может быть оптимизировано: - - `s LIKE '%test%'` - - `NOT s NOT LIKE '%test%'` - - `s = 1` - - `NOT s != 1` - - `startsWith(s, 'test')` -- Не может быть оптимизировано: - - `NOT s LIKE '%test%'` - - `s NOT LIKE '%test%'` - - `NOT s = 1` - - `s != 1` - - `NOT startsWith(s, 'test')` -::: - - +* Может быть оптимизировано: + * `s LIKE '%test%'` + * `NOT s NOT LIKE '%test%'` + * `s = 1` + * `NOT s != 1` + * `startsWith(s, 'test')` +* Не может быть оптимизировано: + * `NOT s LIKE '%test%'` + * `s NOT LIKE '%test%'` + * `NOT s = 1` + * `s != 1` + * `NOT startsWith(s, 'test')` + ::: ## Проекции {#projections} -Проекции похожи на [материализованные представления](/sql-reference/statements/create/view), но определяются на уровне части таблицы. Они обеспечивают гарантии согласованности и автоматически используются в запросах. +Проекции похожи на [materialized views](/sql-reference/statements/create/view), но определяются на уровне частей таблицы (parts). Они обеспечивают гарантии согласованности, а также автоматическое использование в запросах. :::note -При использовании проекций следует также учитывать настройку [force_optimize_projection](/operations/settings/settings#force_optimize_projection). +При использовании проекций следует также учитывать настройку [force_optimize_projection](/operations/settings/settings#force_optimize_projection). ::: Проекции не поддерживаются в операторах `SELECT` с модификатором [FINAL](/sql-reference/statements/select/from#final-modifier). @@ -586,36 +570,34 @@ sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloo **Синтаксис** ```sql -SELECT [GROUP BY] [ORDER BY] +SELECT <выражение списка столбцов> [GROUP BY] <выражение ключей группировки> [ORDER BY] <выражение> ``` Проекции можно изменять или удалять с помощью оператора [ALTER](/sql-reference/statements/alter/projection.md). ### Хранение проекций {#projection-storage} -Проекции хранятся внутри директории части таблицы. Это похоже на индекс, но содержит поддиректорию, в которой хранится часть анонимной таблицы `MergeTree`. Таблица создается на основе определяющего запроса проекции. Если присутствует секция `GROUP BY`, базовым движком хранения становится [AggregatingMergeTree](aggregatingmergetree.md), и все агрегатные функции преобразуются в `AggregateFunction`. Если присутствует секция `ORDER BY`, таблица `MergeTree` использует её в качестве выражения первичного ключа. В процессе слияния часть проекции объединяется через процедуру слияния её хранилища. Контрольная сумма части родительской таблицы объединяется с контрольной суммой части проекции. Другие операции обслуживания аналогичны индексам с пропуском данных. +Проекции хранятся внутри каталога части. По сути это похоже на индекс, но включает подкаталог, в котором хранится часть анонимной таблицы `MergeTree`. Таблица задаётся запросом, определяющим проекцию. Если в определении есть предложение `GROUP BY`, базовый движок хранения становится [AggregatingMergeTree](aggregatingmergetree.md), и все агрегатные функции приводятся к типу `AggregateFunction`. Если есть предложение `ORDER BY`, таблица `MergeTree` использует его как выражение первичного ключа. Во время процесса слияния часть проекции объединяется с использованием процедуры слияния её движка хранения. Контрольная сумма части родительской таблицы объединяется с частью проекции. Остальные операции обслуживания аналогичны операциям для skip-индексов. ### Анализ запросов {#projection-query-analysis} -1. Проверяется, может ли проекция использоваться для ответа на данный запрос, то есть генерирует ли она тот же результат, что и запрос к базовой таблице. -2. Выбирается наилучшее подходящее совпадение, которое содержит наименьшее количество гранул для чтения. -3. Конвейер запроса, использующий проекции, будет отличаться от того, который использует исходные части. Если проекция отсутствует в некоторых частях, можно добавить конвейер для её создания «на лету». - +1. Проверьте, может ли проекция быть использована для ответа на данный запрос, то есть даёт ли она тот же результат, что и запрос к базовой таблице. +2. Выберите оптимальное соответствие, для которого нужно прочитать наименьшее количество гранул. +3. Конвейер обработки запроса, использующий проекции, будет отличаться от конвейера, работающего с исходными частями. Если в некоторых частях проекция отсутствует, можно добавить конвейер, чтобы «спроецировать» её на лету. -## Параллельный доступ к данным {#concurrent-data-access} +## Одновременный доступ к данным {#concurrent-data-access} -Для параллельного доступа к таблице используется многоверсионность. Иными словами, когда таблица одновременно читается и обновляется, данные читаются из набора кусков, актуального на момент выполнения запроса. Длительные блокировки отсутствуют. Операции вставки не мешают операциям чтения. +Для одновременного доступа к таблице используется многоверсионность. Иными словами, когда таблица одновременно читается и обновляется, данные читаются из набора частей, актуального на момент выполнения запроса. Длительные блокировки отсутствуют. Вставки не мешают операциям чтения. Чтение из таблицы автоматически распараллеливается. - -## TTL для столбцов и таблиц {#table_engine-mergetree-ttl} +## TTL для столбцов и таблиц {#table_engine-mergetree-ttl} Определяет время жизни значений. -Секция `TTL` может быть задана как для всей таблицы, так и для каждого отдельного столбца. `TTL` на уровне таблицы также может определять логику автоматического перемещения данных между дисками и томами или повторного сжатия кусков, в которых все данные устарели. +Выражение `TTL` может быть задано как для всей таблицы, так и для каждого отдельного столбца. `TTL` на уровне таблицы также может задавать логику автоматического перемещения данных между дисками и томами, а также перекомпрессии частей, в которых срок жизни всех данных истёк. -Выражения должны возвращать тип данных [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md), [DateTime](/sql-reference/data-types/datetime.md) или [DateTime64](/sql-reference/data-types/datetime64.md). +Выражения должны вычисляться в значение типа данных [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md), [DateTime](/sql-reference/data-types/datetime.md) или [DateTime64](/sql-reference/data-types/datetime64.md). **Синтаксис** @@ -626,7 +608,7 @@ TTL time_column TTL time_column + interval ``` -Для определения `interval` используйте операторы [временных интервалов](/sql-reference/operators#operators-for-working-with-dates-and-times), например: +Чтобы задать `interval`, используйте операторы [интервалов времени](/sql-reference/operators#operators-for-working-with-dates-and-times), например: ```sql TTL date_time + INTERVAL 1 MONTH @@ -635,13 +617,13 @@ TTL date_time + INTERVAL 15 HOUR ### TTL столбца {#mergetree-column-ttl} -Когда значения в столбце устаревают, ClickHouse заменяет их значениями по умолчанию для типа данных столбца. Если все значения столбца в куске данных устаревают, ClickHouse удаляет этот столбец из куска данных в файловой системе. +Когда срок жизни значений в столбце истекает, ClickHouse заменяет их значениями по умолчанию для типа данных столбца. Если срок жизни всех значений столбца в части данных истекает, ClickHouse удаляет этот столбец из соответствующей части данных в файловой системе. -Секция `TTL` не может использоваться для ключевых столбцов. +Предложение `TTL` нельзя использовать для ключевых столбцов. **Примеры** -#### Создание таблицы с `TTL`: {#creating-a-table-with-ttl} +#### Создание таблицы с параметром `TTL`: {#creating-a-table-with-ttl} ```sql CREATE TABLE tab @@ -656,7 +638,7 @@ PARTITION BY toYYYYMM(d) ORDER BY d; ``` -#### Добавление TTL к столбцу существующей таблицы {#adding-ttl-to-a-column-of-an-existing-table} +#### Добавление TTL для столбца существующей таблицы {#adding-ttl-to-a-column-of-an-existing-table} ```sql ALTER TABLE tab @@ -664,7 +646,7 @@ ALTER TABLE tab c String TTL d + INTERVAL 1 DAY; ``` -#### Изменение TTL столбца {#altering-ttl-of-the-column} +#### Изменение TTL для столбца {#altering-ttl-of-the-column} ```sql ALTER TABLE tab @@ -674,32 +656,32 @@ ALTER TABLE tab ### TTL таблицы {#mergetree-table-ttl} -Таблица может иметь выражение для удаления устаревших строк и несколько выражений для автоматического перемещения кусков между [дисками или томами](#table_engine-mergetree-multiple-volumes). Когда строки в таблице устаревают, ClickHouse удаляет все соответствующие строки. Для перемещения или повторного сжатия кусков все строки куска должны удовлетворять критериям выражения `TTL`. +Для таблицы может быть задано выражение для удаления строк с истекшим сроком жизни и несколько выражений для автоматического перемещения частей между [дисками или томами](#table_engine-mergetree-multiple-volumes). Когда срок жизни строк в таблице истекает, ClickHouse удаляет все соответствующие строки. Для перемещения или перекомпрессии частей все строки части должны удовлетворять критериям выражения `TTL`. ```sql TTL expr - [DELETE|RECOMPRESS codec_name1|TO DISK 'xxx'|TO VOLUME 'xxx'][, DELETE|RECOMPRESS codec_name2|TO DISK 'aaa'|TO VOLUME 'bbb'] ... - [WHERE conditions] + [DELETE|RECOMPRESS codec_name1|НА ДИСК 'xxx'|НА ТОМ 'xxx'][, DELETE|RECOMPRESS codec_name2|НА ДИСК 'aaa'|НА ТОМ 'bbb'] ... + [WHERE условия] [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] ``` -Тип правила TTL может следовать за каждым выражением TTL. Он определяет действие, которое должно быть выполнено после того, как выражение будет удовлетворено (достигнет текущего времени): +Тип правила TTL может следовать за каждым выражением TTL. Он определяет действие, которое будет выполнено, когда выражение будет выполнено (достигнет текущего времени): -- `DELETE` — удалить устаревшие строки (действие по умолчанию); -- `RECOMPRESS codec_name` — повторно сжать кусок данных с помощью `codec_name`; -- `TO DISK 'aaa'` — переместить кусок на диск `aaa`; -- `TO VOLUME 'bbb'` — переместить кусок на том `bbb`; -- `GROUP BY` — агрегировать устаревшие строки. +* `DELETE` — удалить истекшие строки (действие по умолчанию); +* `RECOMPRESS codec_name` — перекомпрессировать часть данных с использованием `codec_name`; +* `TO DISK 'aaa'` — перенести часть на диск `aaa`; +* `TO VOLUME 'bbb'` — перенести часть в том `bbb`; +* `GROUP BY` — агрегировать истекшие строки. -Действие `DELETE` может использоваться вместе с секцией `WHERE` для удаления только некоторых устаревших строк на основе условия фильтрации: +Действие `DELETE` может использоваться вместе с предложением `WHERE`, чтобы удалять только часть истекших строк на основе условия фильтрации: ```sql -TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' +TTL time_column + INTERVAL 1 MONTH УДАЛИТЬ ГДЕ column = 'value' ``` Выражение `GROUP BY` должно быть префиксом первичного ключа таблицы. -Если столбец не является частью выражения `GROUP BY` и не задан явно в секции `SET`, в результирующей строке он содержит произвольное значение из сгруппированных строк (как если бы к нему была применена агрегатная функция `any`). +Если столбец не входит в выражение `GROUP BY` и явно не задан в предложении `SET`, в результирующей строке он будет содержать произвольное значение из сгруппированных строк (как если бы к нему была применена агрегатная функция `any`). **Примеры** @@ -719,15 +701,14 @@ TTL d + INTERVAL 1 MONTH DELETE, d + INTERVAL 2 WEEK TO DISK 'bbb'; ``` -#### Изменение `TTL` таблицы: {#altering-ttl-of-the-table} +#### Изменение `TTL` для таблицы: {#altering-ttl-of-the-table} ```sql ALTER TABLE tab MODIFY TTL d + INTERVAL 1 DAY; ``` - -Создание таблицы, в которой строки истекают через месяц. Истекшие строки, даты которых приходятся на понедельники, удаляются: +Создание таблицы, в которой строки автоматически удаляются через один месяц. Просроченные строки с датами, приходящимися на понедельник, удаляются: ```sql CREATE TABLE table_with_where @@ -741,7 +722,7 @@ ORDER BY d TTL d + INTERVAL 1 MONTH DELETE WHERE toDayOfWeek(d) = 1; ``` -#### Создание таблицы, в которой истекшие строки перекомпрессируются: {#creating-a-table-where-expired-rows-are-recompressed} +#### Создание таблицы, в которой строки с истёкшим сроком хранения повторно сжимаются: {#creating-a-table-where-expired-rows-are-recompressed} ```sql CREATE TABLE table_for_recompression @@ -756,7 +737,7 @@ TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPR SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; ``` -Создание таблицы, в которой истекшие строки агрегируются. В результирующих строках `x` содержит максимальное значение среди сгруппированных строк, `y` — минимальное значение, а `d` — любое случайное значение из сгруппированных строк. +Создание таблицы, в которой агрегируются просроченные строки. В результате в столбце `x` содержится максимальное значение по сгруппированным строкам, в `y` — минимальное значение, а в `d` — произвольное значение из сгруппированных строк. ```sql CREATE TABLE table_for_aggregation @@ -772,58 +753,56 @@ ORDER BY (k1, k2) TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ``` -### Удаление истекших данных {#mergetree-removing-expired-data} +### Удаление просроченных данных {#mergetree-removing-expired-data} -Данные с истекшим `TTL` удаляются при слиянии частей данных в ClickHouse. +Данные с истёкшим `TTL` удаляются, когда ClickHouse объединяет части данных. -Когда ClickHouse обнаруживает истекшие данные, он выполняет внеплановое слияние. Для управления частотой таких слияний можно задать параметр `merge_with_ttl_timeout`. Если значение слишком низкое, будет выполняться много внеплановых слияний, что может потреблять значительные ресурсы. +Когда ClickHouse обнаруживает, что данные просрочены, он выполняет внеплановое слияние. Чтобы контролировать частоту таких слияний, вы можете задать `merge_with_ttl_timeout`. Если значение слишком мало, будет выполняться много внеплановых слияний, которые могут потреблять значительный объём ресурсов. -Если вы выполняете запрос `SELECT` между слияниями, вы можете получить истекшие данные. Чтобы этого избежать, используйте запрос [OPTIMIZE](/sql-reference/statements/optimize.md) перед `SELECT`. +Если вы выполняете запрос `SELECT` между слияниями, вы можете получить просроченные данные. Чтобы этого избежать, используйте запрос [OPTIMIZE](/sql-reference/statements/optimize.md) перед `SELECT`. **См. также** -- настройка [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) - +* настройка [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) ## Типы дисков {#disk-types} Помимо локальных блочных устройств, ClickHouse поддерживает следующие типы хранилищ: -- [`s3` для S3 и MinIO](#table_engine-mergetree-s3) -- [`gcs` для GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -- [`blob_storage_disk` для Azure Blob Storage](/operations/storing-data#azure-blob-storage) -- [`hdfs` для HDFS](/engines/table-engines/integrations/hdfs) -- [`web` для чтения данных из веб-источников в режиме только для чтения](/operations/storing-data#web-storage) -- [`cache` для локального кэширования](/operations/storing-data#using-local-cache) -- [`s3_plain` для резервного копирования в S3](/operations/backup#backuprestore-using-an-s3-disk) -- [`s3_plain_rewritable` для неизменяемых нереплицируемых таблиц в S3](/operations/storing-data.md#s3-plain-rewritable-storage) - +* [`s3` для S3 и MinIO](#table_engine-mergetree-s3) +* [`gcs` для GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) +* [`blob_storage_disk` для Azure Blob Storage](/operations/storing-data#azure-blob-storage) +* [`hdfs` для HDFS](/engines/table-engines/integrations/hdfs) +* [`web` для режима только чтения с веба](/operations/storing-data#web-storage) +* [`cache` для локального кэширования](/operations/storing-data#using-local-cache) +* [`s3_plain` для резервных копий в S3](/operations/backup#backuprestore-using-an-s3-disk) +* [`s3_plain_rewritable` для неизменяемых, нереплицируемых таблиц в S3](/operations/storing-data.md#s3-plain-rewritable-storage) -## Использование нескольких блочных устройств для хранения данных {#table_engine-mergetree-multiple-volumes} +## Использование нескольких блочных устройств для хранения данных {#table_engine-mergetree-multiple-volumes} ### Введение {#introduction} -Семейство движков таблиц `MergeTree` может хранить данные на нескольких блочных устройствах. Например, это полезно, когда данные определённой таблицы неявно делятся на «горячие» и «холодные». К наиболее свежим данным регулярно обращаются, но они занимают небольшой объём. Напротив, исторические данные с «длинным хвостом» запрашиваются редко. Если доступно несколько дисков, «горячие» данные можно размещать на быстрых дисках (например, NVMe SSD или в памяти), а «холодные» данные — на относительно медленных (например, HDD). +Семейство движков таблиц `MergeTree` может хранить данные на нескольких блочных устройствах. Например, это может быть полезно, когда данные определённой таблицы фактически разделены на «горячие» и «холодные». Самые свежие данные запрашиваются регулярно, но занимают небольшой объём. Напротив, большой «хвост» исторических данных запрашивается редко. Если доступно несколько дисков, «горячие» данные могут располагаться на быстрых дисках (например, NVMe SSD или в памяти), а «холодные» — на относительно медленных (например, HDD). -Часть данных является минимальной перемещаемой единицей для таблиц на движке `MergeTree`. Данные, относящиеся к одной части, хранятся на одном диске. Части данных могут перемещаться между дисками в фоновом режиме (в соответствии с пользовательскими настройками), а также с помощью запросов [ALTER](/sql-reference/statements/alter/partition). +Часть данных (data part) — минимальная единица, которую можно перемещать, для таблиц на движке `MergeTree`. Данные, принадлежащие одной части, хранятся на одном диске. Части данных могут перемещаться между дисками в фоновом режиме (в соответствии с пользовательскими настройками), а также с помощью запросов [ALTER](/sql-reference/statements/alter/partition). ### Термины {#terms} -- Диск — блочное устройство, подключённое к файловой системе. -- Диск по умолчанию — диск, на котором расположен путь, указанный в настройке сервера [path](/operations/server-configuration-parameters/settings.md/#path). -- Том — упорядоченный набор однотипных дисков (аналогично [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures)). -- Политика хранения — набор томов и правила перемещения данных между ними. +* Диск — блочное устройство, смонтированное к файловой системе. +* Диск по умолчанию — диск, на котором расположен путь, указанный в серверной настройке [path](/operations/server-configuration-parameters/settings.md/#path). +* Том — упорядоченный набор одинаковых дисков (аналогично [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures)). +* Политика хранения — набор томов и правил перемещения данных между ними. -Названия указанных сущностей можно найти в системных таблицах [system.storage_policies](/operations/system-tables/storage_policies) и [system.disks](/operations/system-tables/disks). Чтобы применить одну из настроенных политик хранения к таблице, используйте настройку `storage_policy` для таблиц семейства движков `MergeTree`. +Названия описанных сущностей можно найти в системных таблицах [system.storage_policies](/operations/system-tables/storage_policies) и [system.disks](/operations/system-tables/disks). Чтобы применить одну из настроенных политик хранения к таблице, используйте настройку `storage_policy` для таблиц семейства движков `MergeTree`. -### Конфигурация {#table_engine-mergetree-multiple-volumes_configure} +### Конфигурация {#table_engine-mergetree-multiple-volumes_configure} -Диски, тома и политики хранения должны быть объявлены внутри тега `` в файле в каталоге `config.d`. +Диски, тома и политики хранения должны быть объявлены внутри тега `` или в файле в каталоге `config.d`. :::tip Диски также могут быть объявлены в секции `SETTINGS` запроса. Это полезно -для разового анализа, когда необходимо временно подключить диск, который, например, доступен по URL. -Подробнее см. в разделе [динамическое хранилище](/operations/storing-data#dynamic-configuration). +для разового анализа, когда нужно временно подключить диск, который, например, доступен по URL-адресу. +См. раздел [dynamic storage](/operations/storing-data#dynamic-configuration) для получения дополнительной информации. ::: Структура конфигурации: @@ -831,7 +810,7 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ```xml - + /mnt/fast_ssd/clickhouse/ @@ -852,9 +831,9 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); Теги: -- `` — имя диска. Имена должны отличаться для всех дисков. -- `path` — путь, по которому сервер будет хранить данные (каталоги `data` и `shadow`); должен заканчиваться на «/». -- `keep_free_space_bytes` — объём свободного дискового пространства, который необходимо зарезервировать. +* `` — имя диска. Имена должны быть разными для всех дисков. +* `path` — путь, по которому сервер будет хранить данные (каталоги `data` и `shadow`); должен заканчиваться символом '/'. +* `keep_free_space_bytes` — объем свободного дискового пространства, который необходимо зарезервировать. Порядок определения дисков не имеет значения. @@ -874,7 +853,7 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); - + 0.2 @@ -882,7 +861,7 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); - + ... @@ -890,20 +869,19 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); Теги: - -* `policy_name_N` — имя политики. Имена политик должны быть уникальными. -* `volume_name_N` — имя тома. Имена томов должны быть уникальными. +* `policy_name_N` — Имя политики. Имена политик должны быть уникальными. +* `volume_name_N` — Имя тома. Имена томов должны быть уникальными. * `disk` — диск внутри тома. -* `max_data_part_size_bytes` — максимальный размер части данных, которая может быть сохранена на любом из дисков тома. Если оцениваемый размер слитой части данных больше, чем `max_data_part_size_bytes`, то эта часть будет записана на следующий том. По сути, эта возможность позволяет хранить новые/маленькие части данных на «горячем» томе (SSD) и перемещать их на «холодный» том (HDD), когда они становятся достаточно крупными. Не используйте этот параметр, если ваша политика содержит только один том. -* `move_factor` — когда количество доступного места становится меньше этого коэффициента, данные автоматически начинают перемещаться на следующий том, если он есть (по умолчанию 0.1). ClickHouse сортирует существующие части данных по размеру от наибольшей к наименьшей (в порядке убывания) и выбирает части с суммарным размером, достаточным для выполнения условия `move_factor`. Если суммарного размера всех частей недостаточно, будут перемещены все части. -* `perform_ttl_move_on_insert` — отключает перемещение по TTL при INSERT части данных. По умолчанию (если параметр включен), если вставляется часть данных, которая уже просрочена по правилу перемещения TTL, она сразу записывается на том/диск, указанный в правиле перемещения. Это может значительно замедлить вставку, если целевой том/диск медленный (например, S3). Если параметр отключен, уже просроченная часть данных сначала записывается на том по умолчанию, а затем сразу же переносится на том, указанный правилом TTL. -* `load_balancing` — политика балансировки по дискам: `round_robin` или `least_used`. -* `least_used_ttl_ms` — настройка тайм-аута (в миллисекундах) для обновления информации о доступном пространстве на всех дисках (`0` — обновлять всегда, `-1` — никогда не обновлять, значение по умолчанию — `60000`). Обратите внимание: если диск может использоваться только ClickHouse и не подвержен онлайн-изменению размера/сжатию файловой системы, можно использовать `-1`; во всех остальных случаях это не рекомендуется, так как в итоге это приведёт к некорректному распределению пространства. -* `prefer_not_to_merge` — не следует использовать этот параметр. Отключает слияние частей данных на этом томе (это вредно и приводит к деградации производительности). Когда этот параметр включён (не делайте этого), слияние данных на этом томе не допускается (что плохо). Это позволяет (но вам это не нужно) управлять (если вы хотите чем-то управлять, вы совершаете ошибку) тем, как ClickHouse работает с медленными дисками (но ClickHouse знает лучше, поэтому, пожалуйста, не используйте этот параметр). -* `volume_priority` — определяет приоритет (порядок), в котором заполняются тома. Чем меньше значение, тем выше приоритет. Значения параметра должны быть натуральными числами и совместно покрывать диапазон от 1 до N (где N — наименьший приоритет) без пропуска каких-либо чисел. - * Если *у всех* томов задан приоритет, они используются в указанном порядке. - * Если только *у некоторых* томов задан приоритет, тома без приоритета имеют наименьший приоритет и используются в том порядке, в котором они определены в конфигурации. - * Если *ни у одного* тома приоритет не задан, их приоритет устанавливается в соответствии с порядком их объявления в конфигурации. +* `max_data_part_size_bytes` — максимальный размер части данных, которая может быть сохранена на любом из дисков тома. Если оценочный размер сливаемой части будет больше, чем `max_data_part_size_bytes`, то эта часть будет записана на следующий том. По сути эта возможность позволяет держать новые/маленькие части на «горячем» томе (SSD) и перемещать их на «холодный» том (HDD), когда они достигают большого размера. Не используйте этот параметр, если в вашей политике только один том. +* `move_factor` — когда доступное пространство становится меньше этого коэффициента, данные автоматически начинают перемещаться на следующий том, если он есть (по умолчанию 0.1). ClickHouse сортирует существующие части данных по размеру от наибольшей к наименьшей (по убыванию) и выбирает части с суммарным размером, достаточным для выполнения условия `move_factor`. Если суммарный размер всех частей недостаточен, будут перемещены все части. +* `perform_ttl_move_on_insert` — Отключает перемещение по TTL при INSERT части данных. По умолчанию (если включено), если мы вставляем часть данных, которая уже просрочена по правилу перемещения TTL, она сразу попадает на том/диск, указанный в правиле перемещения. Это может существенно замедлить вставку, если целевой том/диск медленный (например, S3). Если выключено, то уже просроченная часть данных записывается на том по умолчанию, а затем сразу перемещается на том, указанный в правиле TTL. +* `load_balancing` — политика балансировки дисков: `round_robin` или `least_used`. +* `least_used_ttl_ms` — настройка таймаута (в миллисекундах) для обновления информации о доступном пространстве на всех дисках (`0` — всегда обновлять, `-1` — никогда не обновлять, по умолчанию `60000`). Обратите внимание: если диск может использоваться только ClickHouse и не подвержен онлайн-изменению размера файловой системы (расширению/сжатию), вы можете использовать `-1`; во всех остальных случаях это не рекомендуется, так как в итоге это приведёт к некорректному распределению пространства. +* `prefer_not_to_merge` — Не следует использовать этот параметр. Отключает слияние частей данных на этом томе (это вредно и приводит к деградации производительности). При включённом параметре (не делайте этого) слияние данных на этом томе не допускается (что плохо). Это позволяет (но вам это не нужно) управлять (если вы хотите что‑то контролировать, вы совершаете ошибку) тем, как ClickHouse работает с медленными дисками (но ClickHouse знает лучше, поэтому, пожалуйста, не используйте этот параметр). +* `volume_priority` — Определяет приоритет (порядок), в котором заполняются тома. Меньшее значение означает более высокий приоритет. Значения параметра должны быть натуральными числами и совместно покрывать диапазон от 1 до N (для самого низкого приоритета) без пропусков. + * Если *все* тома помечены, они получают приоритет в указанном порядке. + * Если помечены только *некоторые* тома, те, у которых нет метки, имеют самый низкий приоритет и получают приоритет в порядке, в котором они определены в конфигурации. + * Если *ни один* том не помечен, их приоритет задаётся в соответствии с порядком, в котором они объявлены в конфигурации. * Два тома не могут иметь одинаковое значение приоритета. Примеры конфигурации: @@ -912,9 +890,9 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ... - + - + disk1 disk2 @@ -949,14 +927,13 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ``` +В приведённом примере политика `hdd_in_order` реализует стратегию [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling). Поэтому эта политика определяет только один том (`single`), а части данных хранятся на всех его дисках по кругу. Такая политика может быть весьма полезна, если в системе подключено несколько однотипных дисков, но RAID не настроен. Имейте в виду, что каждый отдельный диск ненадёжен, и может потребоваться компенсировать это фактором репликации 3 или более. -В данном примере политика `hdd_in_order` реализует подход [циклического перебора](https://en.wikipedia.org/wiki/Round-robin_scheduling). Эта политика определяет только один том (`single`), куски данных хранятся на всех его дисках в циклическом порядке. Такая политика может быть весьма полезна, если к системе подключено несколько однотипных дисков, но RAID не настроен. Имейте в виду, что каждый отдельный дисковый накопитель не является надёжным, и вы можете компенсировать это коэффициентом репликации 3 или более. - -Если в системе доступны различные типы дисков, можно использовать политику `moving_from_ssd_to_hdd`. Том `hot` состоит из SSD-диска (`fast_ssd`), максимальный размер куска, который может храниться на этом томе, составляет 1 ГБ. Все куски размером более 1 ГБ будут храниться непосредственно на томе `cold`, который содержит HDD-диск `disk1`. -Кроме того, как только диск `fast_ssd` заполнится более чем на 80%, данные будут перенесены на `disk1` фоновым процессом. +Если в системе доступны разные типы дисков, вместо этого можно использовать политику `moving_from_ssd_to_hdd`. Том `hot` состоит из SSD-диска (`fast_ssd`), и максимальный размер части, которая может храниться на этом томе, составляет 1 ГБ. Все части размером более 1 ГБ будут храниться непосредственно на томе `cold`, который содержит HDD-диск `disk1`. +Кроме того, как только диск `fast_ssd` будет заполнен более чем на 80%, данные будут перенесены на `disk1` фоновым процессом. -Порядок перечисления томов в политике хранения важен в случае, если хотя бы один из перечисленных томов не имеет явного параметра `volume_priority`. -Как только том переполняется, данные перемещаются на следующий. Порядок перечисления дисков также важен, поскольку данные хранятся на них по очереди. +Порядок перечисления томов в политике хранения важен в случае, если хотя бы один из перечисленных томов не имеет явно заданного параметра `volume_priority`. +Когда том переполнен, данные переносятся на следующий. Порядок перечисления дисков также важен, поскольку данные записываются на них по очереди. При создании таблицы к ней можно применить одну из настроенных политик хранения: @@ -972,49 +949,46 @@ PARTITION BY toYYYYMM(EventDate) SETTINGS storage_policy = 'moving_from_ssd_to_hdd' ``` -Политика хранения `default` подразумевает использование только одного тома, который состоит из одного диска, указанного в ``. -Политику хранения можно изменить после создания таблицы с помощью запроса [ALTER TABLE ... MODIFY SETTING], новая политика должна включать все старые диски и тома с теми же именами. +Политика хранения `default` подразумевает использование только одного тома, который включает один диск, заданный в ``. +Вы можете изменить политику хранения после создания таблицы с помощью запроса [ALTER TABLE ... MODIFY SETTING]; при этом новая политика должна включать все старые диски и тома с теми же именами. -Количество потоков, выполняющих фоновое перемещение кусков данных, можно изменить с помощью настройки [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size). +Количество потоков, выполняющих фоновое перемещение частей данных, можно изменить с помощью настройки [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size). ### Подробности {#details} -В случае таблиц `MergeTree` данные попадают на диск различными способами: - -- В результате вставки (запрос `INSERT`). -- Во время фоновых слияний и [мутаций](/sql-reference/statements/alter#mutations). -- При загрузке с другой реплики. -- В результате заморозки партиции [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition). - -Во всех этих случаях, за исключением мутаций и заморозки партиций, кусок хранится на томе и диске в соответствии с заданной политикой хранения: - -1. Выбирается первый том (в порядке определения), который имеет достаточно дискового пространства для хранения куска (`unreserved_space > current_part_size`) и позволяет хранить куски заданного размера (`max_data_part_size_bytes > current_part_size`). -2. В пределах этого тома выбирается диск, который следует за тем, который использовался для хранения предыдущего фрагмента данных, и который имеет свободное пространство больше размера куска (`unreserved_space - keep_free_space_bytes > current_part_size`). +В случае таблиц `MergeTree` данные попадают на диск разными способами: -Внутри мутации и заморозка партиций используют [жёсткие ссылки](https://en.wikipedia.org/wiki/Hard_link). Жёсткие ссылки между различными дисками не поддерживаются, поэтому в таких случаях результирующие куски хранятся на тех же дисках, что и исходные. +* В результате вставки (запрос `INSERT`). +* Во время фоновых слияний и [мутаций](/sql-reference/statements/alter#mutations). +* При загрузке с другой реплики. +* В результате заморозки партиции [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition). -В фоновом режиме куски перемещаются между томами на основе объёма свободного пространства (параметр `move_factor`) в соответствии с порядком, в котором тома объявлены в конфигурационном файле. -Данные никогда не переносятся с последнего тома на первый. Для мониторинга фоновых перемещений можно использовать системные таблицы [system.part_log](/operations/system-tables/part_log) (поле `type = MOVE_PART`) и [system.parts](/operations/system-tables/parts.md) (поля `path` и `disk`). Также подробная информация доступна в логах сервера. +Во всех этих случаях, за исключением мутаций и заморозки партиций, часть данных сохраняется на томе и диске в соответствии с заданной политикой хранения: -Пользователь может принудительно переместить кусок или партицию с одного тома на другой с помощью запроса [ALTER TABLE ... MOVE PART\|PARTITION ... TO VOLUME\|DISK ...](/sql-reference/statements/alter/partition), при этом учитываются все ограничения для фоновых операций. Запрос инициирует перемещение самостоятельно и не ожидает завершения фоновых операций. Пользователь получит сообщение об ошибке, если недостаточно свободного пространства или если какое-либо из требуемых условий не выполнено. +1. Выбирается первый том (в порядке объявления), у которого достаточно свободного дискового пространства для хранения части (`unreserved_space > current_part_size`) и который допускает хранение частей заданного размера (`max_data_part_size_bytes > current_part_size`). +2. Внутри этого тома выбирается тот диск, который следует за диском, использованным для хранения предыдущей части данных, и у которого свободное пространство больше размера части (`unreserved_space - keep_free_space_bytes > current_part_size`). -Перемещение данных не влияет на репликацию данных. Поэтому для одной и той же таблицы на разных репликах могут быть указаны различные политики хранения. +Внутренним образом мутации и заморозка партиций используют [жёсткие ссылки](https://en.wikipedia.org/wiki/Hard_link). Жёсткие ссылки между разными дисками не поддерживаются, поэтому в таких случаях результирующие части сохраняются на тех же дисках, что и исходные. +В фоновом режиме части перемещаются между томами на основе количества свободного места (параметр `move_factor`) в соответствии с порядком, в котором тома объявлены в конфигурационном файле. +Данные никогда не переносятся ни с последнего тома, ни на первый том. Для мониторинга фоновых перемещений можно использовать системные таблицы [system.part_log](/operations/system-tables/part_log) (поле `type = MOVE_PART`) и [system.parts](/operations/system-tables/parts.md) (поля `path` и `disk`). Также подробная информация может быть найдена в логах сервера. -После завершения фоновых слияний и мутаций старые части удаляются только по истечении заданного времени (`old_parts_lifetime`). -В течение этого времени они не перемещаются на другие тома или диски. Поэтому, пока части окончательно не удалены, они продолжают учитываться при расчёте занятого дискового пространства. +Пользователь может принудительно переместить часть или партицию с одного тома на другой с помощью запроса [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition); при этом учитываются все ограничения для фоновых операций. Запрос самостоятельно инициирует перемещение и не ждёт завершения фоновых операций. Пользователь получит сообщение об ошибке, если недостаточно свободного места или не выполнено какое-либо из требуемых условий. -Пользователь может равномерно распределять новые крупные части по разным дискам тома [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) с помощью настройки [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod). +Перемещение данных не мешает репликации данных. Поэтому для одной и той же таблицы на разных репликах могут быть заданы разные политики хранения. +После завершения фоновых слияний и мутаций старые части удаляются только спустя определённый промежуток времени (`old_parts_lifetime`). +В течение этого времени они не перемещаются на другие тома или диски. Поэтому до окончательного удаления части по-прежнему учитываются при оценке занятого дискового пространства. +Пользователь может более равномерно распределять новые большие части по разным дискам тома [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) с помощью настройки [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod). -## Использование внешнего хранилища для хранения данных {#table_engine-mergetree-s3} +## Использование внешнего хранилища для хранения данных {#table_engine-mergetree-s3} -Движки таблиц семейства [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) могут хранить данные в `S3`, `AzureBlobStorage`, `HDFS`, используя диски с типами `s3`, `azure_blob_storage`, `hdfs` соответственно. Подробнее см. в разделе [настройка параметров внешнего хранилища](/operations/storing-data.md/#configuring-external-storage). +Движки таблиц семейства [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) могут хранить данные в `S3`, `AzureBlobStorage`, `HDFS`, используя диск с типом `s3`, `azure_blob_storage`, `hdfs` соответственно. Для получения дополнительной информации см. раздел [настройка параметров внешнего хранилища](/operations/storing-data.md/#configuring-external-storage). Пример использования [S3](https://aws.amazon.com/s3/) в качестве внешнего хранилища с диском типа `s3`. -Разметка конфигурации: +Фрагмент конфигурации: ```xml @@ -1055,36 +1029,35 @@ SETTINGS storage_policy = 'moving_from_ssd_to_hdd' ``` -См. также [настройка параметров внешнего хранилища](/operations/storing-data.md/#configuring-external-storage). +См. также [настройку вариантов внешних хранилищ](/operations/storing-data.md/#configuring-external-storage). -:::note конфигурация кеша -В версиях ClickHouse с 22.3 по 22.7 используется другая конфигурация кеша. См. раздел [использование локального кеша](/operations/storing-data.md/#using-local-cache), если вы используете одну из этих версий. +:::note конфигурация кэша +Версии ClickHouse с 22.3 по 22.7 используют другую конфигурацию кэша, см. [использование локального кэша](/operations/storing-data.md/#using-local-cache), если вы используете одну из этих версий. ::: - ## Виртуальные столбцы {#virtual-columns} -- `_part` — Имя части. -- `_part_index` — Порядковый индекс части в результате запроса. -- `_part_starting_offset` — Накопительное смещение начальной строки части в результате запроса. -- `_part_offset` — Номер строки в части. -- `_part_granule_offset` — Номер гранулы в части. -- `_partition_id` — Имя партиции. -- `_part_uuid` — Уникальный идентификатор части (если включена настройка MergeTree `assign_part_uuids`). -- `_part_data_version` — Версия данных части (минимальный номер блока или версия мутации). -- `_partition_value` — Значения (кортеж) выражения `partition by`. -- `_sample_factor` — Коэффициент выборки (из запроса). -- `_block_number` — Исходный номер блока для строки, назначенный при вставке; сохраняется при слияниях, если включена настройка `enable_block_number_column`. -- `_block_offset` — Исходный номер строки в блоке, назначенный при вставке; сохраняется при слияниях, если включена настройка `enable_block_offset_column`. -- `_disk_name` — Имя диска, используемого для хранения данных. - - -## Статистика столбцов {#column-statistics} +* `_part` — Имя парта. +* `_part_index` — Последовательный индекс парта в результате запроса. +* `_part_starting_offset` — Кумулятивный номер первой строки парта в результате запроса. +* `_part_offset` — Номер строки в парте. +* `_part_granule_offset` — Номер гранулы в парте. +* `_partition_id` — Имя партиции. +* `_part_uuid` — Уникальный идентификатор парта (если включена настройка MergeTree `assign_part_uuids`). +* `_part_data_version` — Версия данных парта (либо минимальный номер блока, либо версия мутации). +* `_partition_value` — Значения (кортеж) выражения `PARTITION BY`. +* `_sample_factor` — Коэффициент выборки (из запроса). +* `_block_number` — Исходный номер блока для строки, который был присвоен при вставке; сохраняется при слияниях, когда включена настройка `enable_block_number_column`. +* `_block_offset` — Исходный номер строки в блоке, который был присвоен при вставке; сохраняется при слияниях, когда включена настройка `enable_block_offset_column`. +* `_disk_name` — Имя диска, используемого для хранения. + +## Статистика по столбцам {#column-statistics} + -Объявление статистики находится в секции столбцов запроса `CREATE` для таблиц семейства `*MergeTree*` при включении `set allow_experimental_statistics = 1`. +Объявление статистики задаётся в секции `COLUMNS` запроса `CREATE` для таблиц из семейства `*MergeTree*` при включённой настройке `set allow_experimental_statistics = 1`. ```sql CREATE TABLE tab @@ -1096,67 +1069,66 @@ ENGINE = MergeTree ORDER BY a ``` -Статистикой также можно управлять с помощью операторов `ALTER`. +Мы также можем изменять статистику с помощью операторов `ALTER`. ```sql ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; ALTER TABLE tab DROP STATISTICS a; ``` -Эта легковесная статистика агрегирует информацию о распределении значений в столбцах. Статистика хранится в каждом куске и обновляется при каждой вставке данных. -Она может использоваться для оптимизации prewhere только при включении `set allow_statistics_optimize = 1`. +Эта легковесная статистика агрегирует информацию о распределении значений в столбцах. Статистика хранится в каждой части и обновляется при каждой вставке. +Её можно использовать для оптимизации `PREWHERE` только при включённом параметре `set allow_statistics_optimize = 1`. -### Доступные типы статистики столбцов {#available-types-of-column-statistics} +### Доступные типы статистики по столбцам {#available-types-of-column-statistics} -- `MinMax` +* `MinMax` - Минимальное и максимальное значения столбца, что позволяет оценить селективность диапазонных фильтров для числовых столбцов. + Минимальное и максимальное значения столбца, что позволяет оценивать селективность диапазонных фильтров по числовым столбцам. Синтаксис: `minmax` -- `TDigest` +* `TDigest` - Скетчи [TDigest](https://github.com/tdunning/t-digest), которые позволяют вычислять приблизительные процентили (например, 90-й процентиль) для числовых столбцов. + Скетчи [TDigest](https://github.com/tdunning/t-digest), которые позволяют вычислять приблизительные перцентили (например, 90-й перцентиль) для числовых столбцов. Синтаксис: `tdigest` -- `Uniq` +* `Uniq` - Скетчи [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog), которые предоставляют оценку количества уникальных значений в столбце. + Скетчи [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog), которые позволяют оценить, сколько различных значений содержит столбец. Синтаксис: `uniq` -- `CountMin` +* `CountMin` - Скетчи [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch), которые предоставляют приблизительный подсчет частоты каждого значения в столбце. + Скетчи [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch), которые предоставляют приблизительный подсчёт частоты каждого значения в столбце. Синтаксис: `countmin` ### Поддерживаемые типы данных {#supported-data-types} -| | (U)Int*, Float*, Decimal(_), Date_, Boolean, Enum\* | String или FixedString | -| -------- | --------------------------------------------------- | --------------------- | -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | +| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String или FixedString | +|-----------|----------------------------------------------------|------------------------| +| CountMin | ✔ | ✔ | +| MinMax | ✔ | ✗ | +| TDigest | ✔ | ✗ | +| Uniq | ✔ | ✔ | ### Поддерживаемые операции {#supported-operations} -| | Фильтры равенства (==) | Диапазонные фильтры (`>, >=, <, <=`) | -| -------- | --------------------- | ------------------------------ | -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | - +| | Фильтры равенства (==) | Фильтры по диапазону (`>, >=, <, <=`) | +|-----------|------------------------|---------------------------------------| +| CountMin | ✔ | ✗ | +| MinMax | ✗ | ✔ | +| TDigest | ✗ | ✔ | +| Uniq | ✔ | ✗ | -## Настройки на уровне столбцов {#column-level-settings} +## Параметры на уровне столбцов {#column-level-settings} -Некоторые настройки MergeTree можно переопределить на уровне столбцов: +Некоторые настройки MergeTree можно переопределять на уровне столбцов: -- `max_compress_block_size` — Максимальный размер блоков несжатых данных перед сжатием при записи в таблицу. -- `min_compress_block_size` — Минимальный размер блоков несжатых данных, требуемый для сжатия при записи следующей метки. +* `max_compress_block_size` — максимальный размер блоков несжатых данных перед их сжатием при записи в таблицу. +* `min_compress_block_size` — минимальный размер блоков несжатых данных, необходимый для сжатия при записи следующей метки. Пример: @@ -1170,21 +1142,21 @@ ENGINE = MergeTree ORDER BY id ``` -Настройки на уровне столбцов можно изменить или удалить с помощью [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md), например: +Настройки для отдельных столбцов можно изменять или удалять с помощью [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md), например: -- Удаление `SETTINGS` из объявления столбца: +* Удалить `SETTINGS` из определения столбца: ```sql ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; ``` -- Изменение настройки: +* Измените параметр: ```sql ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; ``` -- Сброс одной или нескольких настроек также удаляет объявление настройки в выражении столбца запроса CREATE таблицы. +* Сбрасывает одну или несколько настроек, а также удаляет объявление настройки в определении столбца в запросе CREATE для таблицы. ```sql ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md index b02445758f3..e920e05b138 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Движок таблиц ReplacingMergeTree +# Движок таблиц ReplacingMergeTree {#replacingmergetree-table-engine} Этот движок отличается от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) тем, что удаляет дублирующиеся записи с одинаковым значением [ключа сортировки](../../../engines/table-engines/mergetree-family/mergetree.md) (раздел `ORDER BY` в определении таблицы, а не `PRIMARY KEY`). @@ -24,7 +24,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -47,9 +47,9 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ::: -## Параметры ReplacingMergeTree +## Параметры ReplacingMergeTree {#replacingmergetree-parameters} -### `ver` +### `ver` {#ver} `ver` — столбец с номером версии. Тип `UInt*`, `Date`, `DateTime` или `DateTime64`. Необязательный параметр. @@ -101,7 +101,7 @@ SELECT * FROM mySecondReplacingMT FINAL; └─────┴─────────┴─────────────────────┘ ``` -### `is_deleted` +### `is_deleted` {#is_deleted} `is_deleted` — имя столбца, используемого во время слияния для определения, представляет ли строка состояние или подлежит удалению; `1` — строка-удаление, `0` — строка-состояние. @@ -190,7 +190,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Дедупликация при выполнении запроса & FINAL +## Дедупликация при выполнении запроса & FINAL {#query-time-de-duplication--final} Во время слияния ReplacingMergeTree определяет дублирующиеся строки, используя значения столбцов `ORDER BY` (указанных при создании таблицы) в качестве уникального идентификатора и сохраняя только самую позднюю версию. Однако это обеспечивает лишь корректность «в конечном счёте» — нет гарантии, что строки будут дедуплицированы, и полагаться на это не следует. Поэтому запросы могут возвращать некорректные результаты, так как строки с обновлениями и удалениями учитываются в запросах. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md index 6ac7b1d8bbd..a469af6ea44 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md @@ -1,5 +1,5 @@ --- -description: 'Обзор репликации данных с помощью семейства движков таблиц Replicated* в ClickHouse' +description: 'Обзор репликации данных с использованием семейства движков таблиц Replicated* в ClickHouse' sidebar_label: 'Replicated*' sidebar_position: 20 slug: /engines/table-engines/mergetree-family/replication @@ -7,12 +7,10 @@ title: 'Движки таблиц Replicated*' doc_type: 'reference' --- - - -# Движки таблиц Replicated* +# Движки таблиц Replicated* {#replicated-table-engines} :::note -В ClickHouse Cloud репликация выполняется автоматически. Пожалуйста, создавайте таблицы без указания аргументов. Например, в тексте ниже вы бы заменили: +В ClickHouse Cloud репликацией управляет платформа. Пожалуйста, создавайте таблицы без дополнительных аргументов. Например, в тексте ниже вы бы заменили: ```sql ENGINE = ReplicatedMergeTree( @@ -21,7 +19,7 @@ ENGINE = ReplicatedMergeTree( ) ``` -с: +с помощью: ```sql ENGINE = ReplicatedMergeTree @@ -39,17 +37,19 @@ ENGINE = ReplicatedMergeTree * ReplicatedVersionedCollapsingMergeTree * ReplicatedGraphiteMergeTree -Репликация осуществляется на уровне отдельной таблицы, а не всего сервера. Сервер может одновременно хранить как реплицируемые, так и нереплицируемые таблицы. +Репликация работает на уровне отдельной таблицы, а не всего сервера. Один сервер может одновременно хранить как реплицируемые, так и нереплицируемые таблицы. + +Репликация не зависит от разбиения на сегменты. Каждый сегмент имеет собственную независимую репликацию. Сжатые данные для запросов `INSERT` и `ALTER` реплицируются (дополнительную информацию см. в документации по [ALTER](/sql-reference/statements/alter)). Запросы `CREATE`, `DROP`, `ATTACH`, `DETACH` и `RENAME` выполняются на одном сервере и не реплицируются: -* Запрос `CREATE TABLE` создаёт новую реплицируемую таблицу на сервере, где выполняется запрос. Если эта таблица уже существует на других серверах, добавляется новая реплика. -* Запрос `DROP TABLE` удаляет реплику, расположенную на сервере, где выполняется запрос. +* Запрос `CREATE TABLE` создаёт новую реплицируемую таблицу на сервере, на котором выполняется запрос. Если такая таблица уже существует на других серверах, создаётся новая реплика. +* Запрос `DROP TABLE` удаляет реплику, расположенную на сервере, на котором выполняется запрос. * Запрос `RENAME` переименовывает таблицу на одной из реплик. Другими словами, реплицируемые таблицы могут иметь разные имена на разных репликах. -ClickHouse использует [ClickHouse Keeper](/guides/sre/keeper/index.md) для хранения метаданных о репликах. Возможно использование ZooKeeper версии 3.4.5 или более новой, но рекомендуется ClickHouse Keeper. +ClickHouse использует [ClickHouse Keeper](/guides/sre/keeper/index.md) для хранения метаинформации о репликах. Можно использовать ZooKeeper версии 3.4.5 или новее, но рекомендуется ClickHouse Keeper. Чтобы использовать репликацию, задайте параметры в разделе конфигурации сервера [zookeeper](/operations/server-configuration-parameters/settings#zookeeper). @@ -76,8 +76,8 @@ ClickHouse использует [ClickHouse Keeper](/guides/sre/keeper/index.md) ``` -ClickHouse также поддерживает хранение метаинформации о репликах во вспомогательном кластере ZooKeeper. Для этого укажите имя кластера ZooKeeper и путь в качестве аргументов движка. -Иными словами, поддерживается хранение метаданных разных таблиц в разных кластерах ZooKeeper. +ClickHouse также поддерживает хранение метаинформации о репликах во вспомогательном кластере ZooKeeper. Для этого укажите имя кластера ZooKeeper и путь в параметрах движка. +Другими словами, поддерживается хранение метаданных разных таблиц в разных кластерах ZooKeeper. Пример задания адресов вспомогательного кластера ZooKeeper: @@ -106,60 +106,58 @@ ClickHouse также поддерживает хранение метаинфо ``` -Чтобы хранить метаданные таблицы во вспомогательном кластере ZooKeeper вместо кластера ZooKeeper по умолчанию, можно использовать SQL, чтобы создать таблицу с -движком ReplicatedMergeTree следующим образом: +Чтобы хранить метаданные таблицы в дополнительном кластере ZooKeeper вместо кластера ZooKeeper по умолчанию, можно создать таблицу с +движком ReplicatedMergeTree с помощью следующего SQL-запроса: ```sql CREATE TABLE table_name ( ... ) ENGINE = ReplicatedMergeTree('zookeeper_name_configured_in_auxiliary_zookeepers:path', 'replica_name') ... ``` -Вы можете указать любой существующий кластер ZooKeeper, и система будет использовать в нём каталог для своих данных (каталог задаётся при создании реплицируемой таблицы). +Вы можете указать любой существующий кластер ZooKeeper, и система будет использовать на нём каталог для собственных данных (каталог задаётся при создании реплицируемой таблицы). -Если ZooKeeper не задан в конфигурационном файле, вы не сможете создавать реплицируемые таблицы, а любые существующие реплицируемые таблицы будут доступны только для чтения. +Если ZooKeeper не указан в конфигурационном файле, вы не сможете создавать реплицируемые таблицы, а любые существующие реплицируемые таблицы будут доступны только для чтения. -ZooKeeper не используется в запросах `SELECT`, поскольку репликация не влияет на производительность `SELECT`, и запросы выполняются так же быстро, как и для нереплицируемых таблиц. При запросе распределённых реплицируемых таблиц поведение ClickHouse контролируется настройками [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) и [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries). +ZooKeeper не используется в запросах `SELECT`, потому что репликация не влияет на производительность `SELECT`, и запросы выполняются так же быстро, как и для нереплицируемых таблиц. При выполнении запросов к распределённым реплицируемым таблицам поведение ClickHouse управляется настройками [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) и [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries). -Для каждого запроса `INSERT` в ZooKeeper добавляется примерно десять записей через несколько транзакций. (Более точно — для каждого вставленного блока данных; один запрос `INSERT` содержит один блок или один блок на `max_insert_block_size = 1048576` строк.) Это приводит к немного большей задержке для `INSERT` по сравнению с нереплицируемыми таблицами. Но если вы следуете рекомендациям вставлять данные пачками не более одного `INSERT` в секунду, это не создаёт никаких проблем. Весь кластер ClickHouse, который использует один кластер ZooKeeper для координации, в сумме выполняет несколько сотен `INSERT` в секунду. Пропускная способность по вставке данных (количество строк в секунду) столь же высока, как и для нереплицируемых данных. +Для каждого запроса `INSERT` в ZooKeeper добавляется примерно десять записей через несколько транзакций. (Более точно: для каждого вставленного блока данных; один запрос `INSERT` содержит один блок или один блок на `max_insert_block_size = 1048576` строк.) Это приводит к немного большему времени ожидания для `INSERT` по сравнению с нереплицируемыми таблицами. Но если вы следуете рекомендациям и вставляете данные пакетами не чаще одного `INSERT` в секунду, это не создаёт никаких проблем. Весь кластер ClickHouse, используемый для координации одного кластера ZooKeeper, в сумме обрабатывает несколько сотен запросов `INSERT` в секунду. Пропускная способность по вставке данных (количество строк в секунду) столь же высока, как и для нереплицируемых данных. -Для очень больших кластеров вы можете использовать разные кластеры ZooKeeper для разных шардов. Однако, исходя из нашего опыта, в этом не было необходимости даже для промышленных кластеров примерно из 300 серверов. +Для очень больших кластеров вы можете использовать разные кластеры ZooKeeper для разных сегментов. Однако, по нашему опыту, в боевых кластерах примерно из 300 серверов в этом не было необходимости. -Репликация асинхронная и многомастерная (multi-master). Запросы `INSERT` (а также `ALTER`) могут быть отправлены на любой доступный сервер. Данные вставляются на тот сервер, на котором выполняется запрос, а затем копируются на остальные серверы. Поскольку это асинхронно, недавно вставленные данные появляются на других репликах с некоторой задержкой. Если часть реплик недоступна, данные будут записаны, когда они станут доступны. Если реплика доступна, задержка равна времени, необходимому для передачи блока сжатых данных по сети. Количество потоков, выполняющих фоновые задачи для реплицируемых таблиц, можно задать настройкой [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size). +Репликация асинхронная и multi-master. Запросы `INSERT` (а также `ALTER`) могут быть отправлены на любой доступный сервер. Данные вставляются на сервере, где выполняется запрос, а затем копируются на другие серверы. Поскольку это происходит асинхронно, недавно вставленные данные появляются на других репликах с некоторой задержкой. Если часть реплик недоступна, данные будут записаны, когда они станут доступными. Если реплика доступна, задержка равна времени передачи блока сжатых данных по сети. Количество потоков, выполняющих фоновые задачи для реплицируемых таблиц, может быть задано настройкой [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size). -Движок `ReplicatedMergeTree` использует отдельный пул потоков для реплицируемых выборок данных (fetch). Размер пула ограничивается настройкой [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size), которую можно изменить с перезапуском сервера. +Движок `ReplicatedMergeTree` использует отдельный пул потоков для реплицируемых загрузок данных (fetches). Размер пула ограничен настройкой [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size), которую можно изменить с перезапуском сервера. -По умолчанию запрос `INSERT` ждёт подтверждения записи данных только от одной реплики. Если данные были успешно записаны только на одну реплику и сервер с этой репликой перестаёт существовать, сохранённые данные будут потеряны. Чтобы получать подтверждение записи данных от нескольких реплик, используйте опцию `insert_quorum`. +По умолчанию запрос `INSERT` ожидает подтверждения записи данных только от одной реплики. Если данные были успешно записаны только на одну реплику и сервер с этой репликой перестаёт существовать, сохранённые данные будут утеряны. Чтобы включить получение подтверждения о записи данных от нескольких реплик, используйте опцию `insert_quorum`. Каждый блок данных записывается атомарно. Запрос `INSERT` разбивается на блоки до `max_insert_block_size = 1048576` строк. Другими словами, если запрос `INSERT` содержит меньше 1048576 строк, он выполняется атомарно. -Блоки данных дедуплицируются. При многократной записи одного и того же блока данных (блоков одинакового размера, содержащих одинаковые строки в одном и том же порядке) блок записывается только один раз. Причина в том, что при сетевых сбоях клиентское приложение может не знать, были ли данные записаны в БД, поэтому запрос `INSERT` можно просто повторить. Не имеет значения, на какие реплики отправлялись `INSERT` с идентичными данными. Запросы `INSERT` являются идемпотентными. Параметры дедупликации управляются серверными настройками [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree). +Блоки данных дедуплицируются. При многократной записи одного и того же блока данных (блоки данных одного размера, содержащие одни и те же строки в одном и том же порядке) блок записывается только один раз. Это сделано для случая сбоев сети, когда клиентское приложение не знает, были ли данные записаны в БД, поэтому запрос `INSERT` можно просто повторить. Неважно, на какие реплики отправлялись запросы `INSERT` с идентичными данными. Запросы `INSERT` являются идемпотентными. Параметры дедупликации управляются серверными настройками [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree). -Во время репликации по сети передаются только исходные данные для вставки. Дальнейшая обработка данных (слияние) координируется и выполняется на всех репликах одинаковым образом. Это минимизирует использование сети, что означает, что репликация хорошо работает, когда реплики расположены в разных дата-центрах. (Обратите внимание, что дублирование данных в разных дата-центрах является основной целью репликации.) +Во время репликации по сети передаются только исходные данные для вставки. Дальнейшее преобразование данных (слияние) координируется и выполняется на всех репликах одинаковым образом. Это минимизирует сетевой трафик, что означает, что репликация хорошо работает, когда реплики находятся в разных дата-центрах. (Обратите внимание, что дублирование данных в разных дата-центрах является основной целью репликации.) -Вы можете иметь любое количество реплик одних и тех же данных. По нашему опыту, относительно надёжным и удобным решением в продакшене может быть двойная репликация, при этом каждый сервер использует RAID-5 или RAID-6 (а в некоторых случаях RAID-10). +Вы можете иметь любое количество реплик одних и тех же данных. По нашему опыту, относительно надёжным и удобным решением может быть двойная репликация в продакшене, когда каждый сервер использует RAID-5 или RAID-6 (и RAID-10 в некоторых случаях). -Система отслеживает синхронность данных на репликах и способна восстанавливаться после сбоя. Переключение осуществляется автоматически (при небольших расхождениях в данных) или полуавтоматически (когда данные отличаются слишком сильно, что может указывать на ошибку конфигурации). +Система отслеживает синхронность данных на репликах и способна восстанавливаться после сбоя. Переключение на резерв (failover) происходит автоматически (при небольших различиях в данных) или полуавтоматически (когда данные различаются слишком сильно, что может указывать на ошибку конфигурации). - - -## Создание реплицируемых таблиц +## Создание реплицируемых таблиц {#creating-replicated-tables} :::note -В ClickHouse Cloud репликация выполняется автоматически. +В ClickHouse Cloud репликация осуществляется автоматически. -Создавайте таблицы с использованием [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) без аргументов репликации. Система автоматически преобразует [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) в [`SharedMergeTree`](/cloud/reference/shared-merge-tree) для репликации и распределения данных. +Создавайте таблицы, используя [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) без аргументов репликации. Система внутренне преобразует [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) в [`SharedMergeTree`](/cloud/reference/shared-merge-tree) для репликации и распределения данных. Не используйте `ReplicatedMergeTree` и не задавайте параметры репликации, так как репликацией управляет платформа. ::: -### Параметры Replicated*MergeTree +### Параметры Replicated*MergeTree {#replicatedmergetree-parameters} -| Parameter | Description | -| ------------------ | ------------------------------------------------------------------------------------------------------------------- | -| `zoo_path` | Путь к таблице в ClickHouse Keeper. | -| `replica_name` | Имя реплики в ClickHouse Keeper. | -| `other_parameters` | Параметры движка, используемого для создания реплицируемой версии, например параметр версии в `ReplacingMergeTree`. | +| Параметр | Описание | +| ------------------ | --------------------------------------------------------------------------------------------------------------- | +| `zoo_path` | Путь к таблице в ClickHouse Keeper. | +| `replica_name` | Имя реплики в ClickHouse Keeper. | +| `other_parameters` | Параметры движка, используемого для создания реплицированной версии, например `version` в `ReplacingMergeTree`. | Пример: @@ -170,6 +168,7 @@ CREATE TABLE table_name CounterID UInt32, UserID UInt32, ver UInt16 +) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{layer}-{shard}/table_name', '{replica}', ver) PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) @@ -189,7 +188,7 @@ SAMPLE BY intHash32(UserID); ``` -Как показано в примере, эти параметры могут содержать подстановки в фигурных скобках. Подставляемые значения берутся из раздела [macros](/operations/server-configuration-parameters/settings.md/#macros) конфигурационного файла. +Как видно из примера, эти параметры могут содержать подстановки в фигурных скобках. Значения для подстановки берутся из раздела [macros](/operations/server-configuration-parameters/settings.md/#macros) файла конфигурации. Пример: @@ -200,26 +199,26 @@ SAMPLE BY intHash32(UserID); ``` -Путь к таблице в ClickHouse Keeper должен быть уникальным для каждой реплицируемой таблицы. Таблицы на разных шардах должны иметь разные пути. +Путь к таблице в ClickHouse Keeper должен быть уникальным для каждой реплицируемой таблицы. Таблицы на разных сегментах должны иметь разные пути. В данном случае путь состоит из следующих частей: -`/clickhouse/tables/` — общий префикс. Мы рекомендуем использовать именно его. +`/clickhouse/tables/` — это общий префикс. Мы рекомендуем использовать именно его. -`{shard}` будет подставлен как идентификатор шарда. +`{shard}` будет подставлен как идентификатор сегмента. -`table_name` — это имя узла для таблицы в ClickHouse Keeper. Желательно сделать его таким же, как имя таблицы. Оно задаётся явно, потому что, в отличие от имени таблицы, не меняется после выполнения запроса RENAME. +`table_name` — это имя узла для таблицы в ClickHouse Keeper. Хорошей идеей будет сделать его таким же, как имя таблицы. Оно задаётся явно, потому что, в отличие от имени таблицы, не изменяется после запроса RENAME. *ПОДСКАЗКА*: вы также можете добавить имя базы данных перед `table_name`. Например, `db_name.table_name` -Можно использовать две встроенные подстановки `{database}` и `{table}`, они разворачиваются в имя таблицы и имя базы данных соответственно (при условии, что эти макросы не определены в секции `macros`). Таким образом, путь в ClickHouse Keeper можно указать как `'/clickhouse/tables/{shard}/{database}/{table}'`. -Будьте осторожны с переименованием таблиц при использовании этих встроенных подстановок. Путь в ClickHouse Keeper нельзя изменить, и когда таблица будет переименована, макросы развернутся в другой путь, таблица будет ссылаться на путь, который не существует в ClickHouse Keeper, и перейдёт в режим «только для чтения». +Можно использовать две встроенные подстановки `{database}` и `{table}`, которые подставляются как имя таблицы и имя базы данных соответственно (если только эти макросы не определены в секции `macros`). Таким образом, путь в ClickHouse Keeper можно задать как `'/clickhouse/tables/{shard}/{database}/{table}'`. +Будьте осторожны с переименованием таблиц при использовании этих встроенных подстановок. Путь в ClickHouse Keeper нельзя изменить, и когда таблица переименовывается, макросы будут подставляться в другой путь, таблица будет ссылаться на путь, который не существует в ClickHouse Keeper, и перейдёт в режим только для чтения. -Имя реплики идентифицирует разные реплики одной и той же таблицы. Для этого можно использовать имя сервера, как в примере. Имя должно быть уникальным только внутри каждого шарда. +Имя реплики идентифицирует разные реплики одной и той же таблицы. Для этого вы можете использовать имя сервера, как в примере. Имя должно быть уникальным только внутри каждого сегмента. Вы можете задать параметры явно вместо использования подстановок. Это может быть удобно для тестирования и настройки небольших кластеров. Однако в этом случае вы не можете использовать распределённые DDL-запросы (`ON CLUSTER`). -При работе с большими кластерами мы рекомендуем использовать подстановки, поскольку они снижают вероятность ошибки. +При работе с большими кластерами мы рекомендуем использовать подстановки, так как они снижают вероятность ошибок. -Вы можете указать аргументы по умолчанию для движка таблицы `Replicated` в конфигурационном файле сервера. Например: +Вы можете задать аргументы по умолчанию для движка таблиц `Replicated` в конфигурационном файле сервера. Например: ```xml /clickhouse/tables/{shard}/{database}/{table} @@ -237,7 +236,6 @@ ORDER BY x; Это эквивалентно следующему: - ```sql CREATE TABLE table_name ( x UInt32 @@ -245,30 +243,30 @@ CREATE TABLE table_name ( ORDER BY x; ``` -Выполните запрос `CREATE TABLE` на каждой реплике. Этот запрос создаёт новую реплицируемую таблицу или добавляет новую реплику к уже существующей. +Выполните запрос `CREATE TABLE` на каждой реплике. Этот запрос создает новую реплицируемую таблицу или добавляет новую реплику к уже существующей. -Если вы добавляете новую реплику после того, как таблица уже содержит данные на других репликах, данные будут скопированы с других реплик на новую реплику после выполнения запроса. Другими словами, новая реплика синхронизируется с остальными. +Если вы добавляете новую реплику после того, как таблица уже содержит какие‑то данные на других репликах, данные будут скопированы с других реплик на новую реплику после выполнения этого запроса. Другими словами, новая реплика синхронизируется с остальными. Чтобы удалить реплику, выполните `DROP TABLE`. Однако удаляется только одна реплика — та, которая находится на сервере, где вы выполняете запрос. -## Восстановление после сбоев +## Восстановление после сбоев {#recovery-after-failures} Если ClickHouse Keeper недоступен при запуске сервера, реплицируемые таблицы переключаются в режим только для чтения. Система периодически пытается подключиться к ClickHouse Keeper. -Если ClickHouse Keeper недоступен во время выполнения `INSERT` или при взаимодействии с ClickHouse Keeper возникает ошибка, будет выброшено исключение. +Если ClickHouse Keeper недоступен во время выполнения `INSERT` или при взаимодействии с ClickHouse Keeper возникает ошибка, выбрасывается исключение. -После подключения к ClickHouse Keeper система проверяет, соответствует ли набор данных в локальной файловой системе ожидаемому набору данных (ClickHouse Keeper хранит эту информацию). При небольших несоответствиях система устраняет их, синхронизируя данные с репликами. +После подключения к ClickHouse Keeper система проверяет, соответствует ли набор данных в локальной файловой системе ожидаемому набору данных (ClickHouse Keeper хранит эту информацию). При незначительных несоответствиях система устраняет их, синхронизируя данные с репликами. -Если система обнаруживает повреждённые части данных (с некорректным размером файлов) или неизвестные части (записанные в файловую систему, но не зафиксированные в ClickHouse Keeper), они перемещаются в подкаталог `detached` (они не удаляются). Все отсутствующие части копируются с реплик. +Если система обнаруживает поврежденные части данных (с неправильным размером файлов) или нераспознанные части (части, записанные в файловую систему, но не зарегистрированные в ClickHouse Keeper), она перемещает их в подкаталог `detached` (они не удаляются). Любые отсутствующие части копируются с реплик. -Обратите внимание, что ClickHouse не выполняет никаких разрушающих действий, таких как автоматическое удаление большого объёма данных. +Обратите внимание, что ClickHouse не выполняет никаких разрушающих действий, таких как автоматическое удаление большого объема данных. -При запуске сервера (или установлении нового сеанса с ClickHouse Keeper) он проверяет только количество и размеры всех файлов. Если размеры файлов совпадают, но байты где-то посередине были изменены, это сразу не обнаруживается, а только при попытке чтения данных для запроса `SELECT`. Запрос в этом случае завершится исключением о несовпадающей контрольной сумме или размере сжатого блока. В таком случае части данных добавляются в очередь проверки и при необходимости копируются с реплик. +При запуске сервера (или установлении новой сессии с ClickHouse Keeper) он проверяет только количество и размеры всех файлов. Если размеры файлов совпадают, но байты были изменены где-то в середине файла, это обнаруживается не сразу, а только при попытке чтения данных для запроса `SELECT`. В этом случае запрос завершится исключением о несовпадающей контрольной сумме или размере сжатого блока. Части данных добавляются в очередь проверки и при необходимости копируются с реплик. -Если локальный набор данных слишком сильно отличается от ожидаемого, срабатывает защитный механизм. Сервер записывает это в лог и отказывается запускаться. Причина в том, что такая ситуация может указывать на ошибку конфигурации, например, если реплика на одном шарде была по ошибке сконфигурирована как реплика на другом шарде. Однако пороговые значения для этого механизма установлены достаточно низкими, и такая ситуация может возникнуть при нормальном восстановлении после сбоя. В этом случае данные восстанавливаются полуавтоматически — «по нажатию кнопки». +Если локальный набор данных существенно отличается от ожидаемого, срабатывает защитный механизм. Сервер записывает это в лог и отказывается запускаться. Причина в том, что такая ситуация может указывать на ошибку конфигурации, например, если реплика на одном сегменте была по ошибке настроена как реплика на другом сегменте. Однако пороговые значения для этого механизма установлены достаточно низко, и такая ситуация может возникнуть при обычном восстановлении после сбоя. В этом случае данные восстанавливаются полуавтоматически — «нажатием кнопки». -Чтобы запустить восстановление, создайте узел `/path_to_table/replica_name/flags/force_restore_data` в ClickHouse Keeper с любым содержимым или выполните команду для восстановления всех реплицируемых таблиц: +Чтобы начать восстановление, создайте в ClickHouse Keeper узел `/path_to_table/replica_name/flags/force_restore_data` с любым содержимым или выполните команду для восстановления всех реплицируемых таблиц: ```bash sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data @@ -279,59 +277,57 @@ sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data ## Восстановление после полной потери данных {#recovery-after-complete-data-loss} -Если все данные и метаданные исчезли с одного из серверов, выполните следующие действия для восстановления: +Если все данные и метаданные исчезли с одного из серверов, выполните следующие шаги для восстановления: -1. Установите ClickHouse на сервер. Корректно задайте подстановки в конфигурационном файле, который содержит идентификатор шарда и реплик, если вы их используете. -2. Если у вас были нереплицируемые таблицы, которые необходимо вручную продублировать на серверах, скопируйте их данные с реплики (из каталога `/var/lib/clickhouse/data/db_name/table_name/`). -3. Скопируйте определения таблиц, расположенные в `/var/lib/clickhouse/metadata/`, с реплики. Если идентификатор шарда или реплики явно задан в определениях таблиц, скорректируйте его так, чтобы он соответствовал этой реплике. (Либо запустите сервер и выполните все запросы `ATTACH TABLE`, которые должны были находиться в .sql-файлах в `/var/lib/clickhouse/metadata/`.) -4. Для запуска процедуры восстановления создайте узел в ClickHouse Keeper `/path_to_table/replica_name/flags/force_restore_data` с любым содержимым или выполните команду для восстановления всех реплицируемых таблиц: `sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` +1. Установите ClickHouse на сервер. Корректно задайте подстановки в конфигурационном файле, который содержит идентификатор сегмента и реплик, если вы их используете. +2. Если у вас были нереплицированные таблицы, которые необходимо вручную продублировать на серверах, скопируйте их данные с реплики (из каталога `/var/lib/clickhouse/data/db_name/table_name/`). +3. Скопируйте определения таблиц, расположенные в `/var/lib/clickhouse/metadata/`, с реплики. Если в определениях таблиц явно задан идентификатор сегмента или реплики, скорректируйте его так, чтобы он соответствовал этой реплике. (Либо запустите сервер и выполните все запросы `ATTACH TABLE`, которые должны были быть в .sql-файлах в `/var/lib/clickhouse/metadata/`.) +4. Чтобы запустить восстановление, создайте узел ClickHouse Keeper `/path_to_table/replica_name/flags/force_restore_data` с любым содержимым или выполните команду для восстановления всех реплицированных таблиц: `sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` Затем запустите сервер (или перезапустите, если он уже работает). Данные будут загружены с реплик. -Альтернативный вариант восстановления — удалить информацию о потерянной реплике из ClickHouse Keeper (`/path_to_table/replica_name`), затем создать реплику заново, как описано в разделе "[Создание реплицируемых таблиц](#creating-replicated-tables)". - -Во время восстановления пропускная способность сети никак не ограничивается. Учитывайте это, если вы восстанавливаете сразу много реплик. - +Альтернативный вариант восстановления — удалить информацию о потерянной реплике из ClickHouse Keeper (`/path_to_table/replica_name`), затем создать реплику заново, как описано в разделе "[Создание реплицированных таблиц](#creating-replicated-tables)". +Во время восстановления нет ограничений на пропускную способность сети. Учитывайте это, если вы восстанавливаете большое количество реплик одновременно. -## Преобразование из MergeTree в ReplicatedMergeTree +## Преобразование из MergeTree в ReplicatedMergeTree {#converting-from-mergetree-to-replicatedmergetree} -Мы используем термин `MergeTree` для обозначения всех движков таблиц из `семейства MergeTree`, аналогично для `ReplicatedMergeTree`. +Мы используем термин `MergeTree` для обозначения всех движков таблиц семейства `MergeTree`, аналогично и для `ReplicatedMergeTree`. -Если у вас была таблица `MergeTree`, реплицированная вручную, вы можете преобразовать её в реплицируемую таблицу. Это может понадобиться, если вы уже накопили большой объём данных в таблице `MergeTree` и теперь хотите включить репликацию. +Если у вас была таблица `MergeTree`, которая реплицировалась вручную, вы можете преобразовать её в реплицируемую таблицу. Это может понадобиться, если вы уже накопили большой объём данных в таблице `MergeTree` и теперь хотите включить репликацию. -Оператор [ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) позволяет присоединить отсоединённую таблицу `MergeTree` как `ReplicatedMergeTree`. +Команда [ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) позволяет прикрепить отсоединённую таблицу `MergeTree` как таблицу `ReplicatedMergeTree`. -Таблица `MergeTree` может быть автоматически преобразована при перезапуске сервера, если флаг `convert_to_replicated` установлен в каталоге данных таблицы (`/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/` для базы данных `Atomic`). +Таблица `MergeTree` может быть автоматически преобразована при перезапуске сервера, если флаг `convert_to_replicated` установлен в каталоге с данными таблицы (`/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/` для базы данных `Atomic`). Создайте пустой файл `convert_to_replicated`, и при следующем перезапуске сервера таблица будет загружена как реплицируемая. -Этот запрос можно использовать, чтобы получить путь к данным таблицы. Если у таблицы несколько путей к данным, необходимо использовать первый из них. +Этот запрос можно использовать, чтобы получить путь к данным таблицы. Если у таблицы несколько путей к данным, необходимо использовать первый. ```sql SELECT data_paths FROM system.tables WHERE table = 'table_name' AND database = 'database_name'; ``` Обратите внимание, что таблица ReplicatedMergeTree будет создана со значениями настроек `default_replica_path` и `default_replica_name`. -Чтобы создать преобразованную таблицу на других репликах, вам нужно явно указать её путь в первом аргументе движка `ReplicatedMergeTree`. Для получения этого пути можно использовать следующий запрос. +Чтобы создать преобразованную таблицу на других репликах, вам нужно явно указать ее путь в первом аргументе движка таблицы `ReplicatedMergeTree`. Для получения этого пути можно выполнить следующий запрос. ```sql SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; ``` -Существует также ручной способ это сделать. +Есть и способ сделать это вручную. -Если данные отличаются на разных репликах, сначала синхронизируйте их или удалите эти данные на всех репликах, кроме одной. +Если данные различаются на разных репликах, сначала синхронизируйте их или удалите эти данные на всех репликах, кроме одной. Переименуйте существующую таблицу MergeTree, затем создайте таблицу `ReplicatedMergeTree` со старым именем. Переместите данные из старой таблицы в подкаталог `detached` внутри каталога с данными новой таблицы (`/var/lib/clickhouse/data/db_name/table_name/`). -Затем выполните `ALTER TABLE ATTACH PARTITION` на одной из реплик, чтобы добавить эти части данных в рабочий набор данных. +Затем выполните `ALTER TABLE ATTACH PARTITION` на одной из реплик, чтобы добавить эти части в рабочий набор. -## Преобразование ReplicatedMergeTree в MergeTree {#converting-from-replicatedmergetree-to-mergetree} +## Преобразование из ReplicatedMergeTree в MergeTree {#converting-from-replicatedmergetree-to-mergetree} -Используйте оператор [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree), чтобы подключить отсоединённую таблицу `ReplicatedMergeTree` как таблицу `MergeTree` на одном сервере. +Используйте команду [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree), чтобы подключить отсоединённую таблицу `ReplicatedMergeTree` как `MergeTree` на одном сервере. -Другой способ требует перезапуска сервера. Создайте таблицу MergeTree с другим именем. Переместите все данные из каталога с данными таблицы `ReplicatedMergeTree` в каталог данных новой таблицы. Затем удалите таблицу `ReplicatedMergeTree` и перезапустите сервер. +Другой способ сделать это требует перезапуска сервера. Создайте таблицу MergeTree с другим именем. Переместите все данные из каталога с данными таблицы `ReplicatedMergeTree` в каталог данных новой таблицы. Затем удалите таблицу `ReplicatedMergeTree` и перезапустите сервер. Если вы хотите избавиться от таблицы `ReplicatedMergeTree`, не запуская сервер: @@ -340,11 +336,9 @@ SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; После этого вы можете запустить сервер, создать таблицу `MergeTree`, переместить данные в её каталог, а затем перезапустить сервер. +## Восстановление после потери или повреждения метаданных в кластере ClickHouse Keeper {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} - -## Восстановление при потере или повреждении метаданных в кластере ClickHouse Keeper {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} - -Если данные в ClickHouse Keeper были утеряны или повреждены, их можно сохранить, переместив в нереплицируемую таблицу, как описано выше. +Если данные в ClickHouse Keeper были утеряны или повреждены, вы можете сохранить их, переместив в нереплицируемую таблицу, как описано выше. **См. также** @@ -352,4 +346,4 @@ SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; - [background_fetches_pool_size](/operations/server-configuration-parameters/settings.md/#background_fetches_pool_size) - [execute_merges_on_single_replica_time_threshold](/operations/settings/merge-tree-settings#execute_merges_on_single_replica_time_threshold) - [max_replicated_fetches_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_fetches_network_bandwidth) -- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) +- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md index 65742cd803d..56e740f3701 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок таблиц SummingMergeTree +# Движок таблиц SummingMergeTree {#summingmergetree-table-engine} Этот движок наследуется от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree). Разница в том, что при слиянии частей данных для таблиц `SummingMergeTree` ClickHouse заменяет все строки с одинаковым первичным ключом (или, точнее, с одинаковым [ключом сортировки](../../../engines/table-engines/mergetree-family/mergetree.md)) одной строкой, которая содержит суммы значений для столбцов с числовым типом данных. Если ключ сортировки построен таким образом, что одному значению ключа соответствует большое количество строк, это существенно уменьшает объем хранимых данных и ускоряет выборку. @@ -17,7 +17,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -34,16 +34,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] Описание параметров запроса см. в [описании запроса](../../../sql-reference/statements/create/table.md). -### Параметры SummingMergeTree +### Параметры SummingMergeTree {#parameters-of-summingmergetree} -#### Столбцы +#### Столбцы {#columns} `columns` — кортеж с именами столбцов, значения в которых будут суммироваться. Необязательный параметр. Столбцы должны иметь числовой тип и не должны входить в ключ партиционирования или сортировки. Если `columns` не указан, ClickHouse суммирует значения во всех столбцах с числовым типом данных, которые не входят в ключ сортировки. -### Части запроса +### Части запроса {#query-clauses} При создании таблицы `SummingMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. @@ -69,7 +69,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Пример использования +## Пример использования {#usage-example} Рассмотрим следующую таблицу: @@ -103,13 +103,13 @@ SELECT key, sum(value) FROM summtt GROUP BY key ``` -## Обработка данных +## Обработка данных {#data-processing} Когда данные вставляются в таблицу, они сохраняются как есть. ClickHouse периодически сливает вставленные части данных, и именно в этот момент строки с одинаковым первичным ключом суммируются и заменяются одной строкой для каждой получившейся части данных. ClickHouse может сливать части данных таким образом, что разные получившиеся части данных могут содержать строки с одинаковым первичным ключом, т. е. суммирование будет неполным. Поэтому при выполнении запроса (`SELECT`) следует использовать агрегатную функцию [sum()](/sql-reference/aggregate-functions/reference/sum) и предложение `GROUP BY`, как описано в примере выше. -### Общие правила суммирования +### Общие правила суммирования {#common-rules-for-summation} Значения в столбцах с числовым типом данных суммируются. Набор столбцов определяется параметром `columns`. @@ -119,11 +119,11 @@ ClickHouse может сливать части данных таким обра Значения не суммируются для столбцов, входящих в первичный ключ. -### Суммирование в столбцах AggregateFunction +### Суммирование в столбцах AggregateFunction {#the-summation-in-the-aggregatefunction-columns} Для столбцов типа [AggregateFunction](../../../sql-reference/data-types/aggregatefunction.md) ClickHouse ведёт себя как движок [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md), агрегируя в соответствии с функцией. -### Вложенные структуры +### Вложенные структуры {#nested-structures} Таблица может содержать вложенные структуры данных, которые обрабатываются особым образом. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md index 3be3f2fe0aa..d9462a14649 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Движок таблицы VersionedCollapsingMergeTree +# Движок таблицы VersionedCollapsingMergeTree {#versionedcollapsingmergetree-table-engine} Этот движок: @@ -23,7 +23,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -40,7 +40,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] Описание параметров запроса см. в разделе [описание запроса](../../../sql-reference/statements/create/table.md). -### Параметры движка +### Параметры движка {#engine-parameters} ```sql VersionedCollapsingMergeTree(sign, version) @@ -51,7 +51,7 @@ VersionedCollapsingMergeTree(sign, version) | `sign` | Имя столбца с типом записи: `1` — запись «state», `-1` — запись «cancel». | [`Int8`](/sql-reference/data-types/int-uint) | | `version` | Имя столбца с версией состояния объекта. | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) или [`DateTime64`](/sql-reference/data-types/datetime64) | -### Клаузы запроса +### Клаузы запроса {#query-clauses} При создании таблицы `VersionedCollapsingMergeTree` требуются те же [клаузы](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. @@ -83,9 +83,9 @@ VersionedCollapsingMergeTree(sign, version) -## Коллапсирование +## Коллапсирование {#table_engines_versionedcollapsingmergetree} -### Данные +### Данные {#data} Рассмотрим ситуацию, когда нужно сохранять постоянно изменяющиеся данные для некоторого объекта. Разумно иметь одну строку на объект и обновлять эту строку при каждом изменении. Однако операция UPDATE для СУБД дорогая и медленная, поскольку требует перезаписи данных в хранилище. Обновление неприемлемо, если нужно быстро записывать данные, но вы можете последовательно записывать изменения объекта следующим образом. @@ -131,7 +131,7 @@ VersionedCollapsingMergeTree(sign, version) 2. Длинные постоянно растущие массивы в столбцах снижают эффективность движка из‑за нагрузки на запись. Чем проще данные, тем выше эффективность. 3. Результаты `SELECT` сильно зависят от согласованности истории изменений объекта. Будьте внимательны при подготовке данных для вставки. При несогласованных данных вы можете получить непредсказуемые результаты, например отрицательные значения для неотрицательных метрик, таких как глубина сессии. -### Algorithm +### Algorithm {#table_engines-versionedcollapsingmergetree-algorithm} Когда ClickHouse сливает части данных, он удаляет каждую пару строк с одинаковым первичным ключом и версией и разным `Sign`. Порядок строк не имеет значения. @@ -150,7 +150,7 @@ ClickHouse не гарантирует, что все строки с одина -## Пример использования +## Пример использования {#example-of-use} Пример данных: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md index f90e756406a..fdd588065d7 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md @@ -1,28 +1,36 @@ --- -description: 'Движок таблицы Alias создает прозрачный прокси к другой таблице. Все операции перенаправляются в целевую таблицу, при этом сам алиас не хранит никаких данных.' +description: 'Табличный движок Alias создает прозрачный прокси к другой таблице. Все операции перенаправляются в целевую таблицу, при этом сам алиас не хранит данные.' sidebar_label: 'Alias' sidebar_position: 5 slug: /engines/table-engines/special/alias -title: 'Движок таблицы Alias' +title: 'Табличный движок Alias' doc_type: 'reference' --- +import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Движок таблицы Alias +# Движок таблицы Alias {#alias-table-engine} -Движок `Alias` создает прокси к другой таблице. Все операции чтения и записи перенаправляются в целевую таблицу, при этом сама таблица-алиас не хранит данные и лишь содержит ссылку на целевую таблицу. + +Движок `Alias` создаёт прокси для другой таблицы. Все операции чтения и записи перенаправляются в целевую таблицу, при этом сама таблица-алиас не хранит данных и только поддерживает ссылку на целевую таблицу. +:::info +Это экспериментальная функция, которая может измениться в будущих релизах с нарушением обратной совместимости. +Включите использование движка таблицы Alias +с помощью настройки [allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine). +Введите команду `set allow_experimental_alias_table_engine = 1`. +::: -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [db_name.]alias_name ENGINE = Alias(target_table) ``` -Либо с явным указанием имени базы данных: +Или с указанием имени базы данных: ```sql CREATE TABLE [db_name.]alias_name @@ -30,7 +38,7 @@ ENGINE = Alias(target_db, target_table) ``` :::note -Таблица `Alias` не поддерживает явное задание столбцов. Столбцы автоматически наследуются от целевой таблицы. Это гарантирует, что псевдоним всегда соответствует схеме целевой таблицы. +Таблица `Alias` не поддерживает явное определение столбцов. Столбцы автоматически наследуются от целевой таблицы. Это гарантирует, что таблица `Alias` всегда соответствует схеме целевой таблицы. ::: @@ -39,17 +47,16 @@ ENGINE = Alias(target_db, target_table) - **`target_db (optional)`** — Имя базы данных, содержащей целевую таблицу. - **`target_table`** — Имя целевой таблицы. - - ## Поддерживаемые операции {#supported-operations} Движок таблицы `Alias` поддерживает все основные операции. + ### Операции с целевой таблицей {#operations-on-target} -Эти операции пробрасываются в целевую таблицу: +Эти операции проксируются на целевую таблицу: -| Operation | Support | Description | -|-----------|---------|-------------| +| Операция | Поддержка | Описание | +|-----------|-----------|----------| | `SELECT` | ✅ | Чтение данных из целевой таблицы | | `INSERT` | ✅ | Запись данных в целевую таблицу | | `INSERT SELECT` | ✅ | Пакетная вставка в целевую таблицу | @@ -63,20 +70,18 @@ ENGINE = Alias(target_db, target_table) ### Операции с самим алиасом {#operations-on-alias} -Эти операции затрагивают только алиас, а **не** целевую таблицу: +Эти операции применяются только к алиасу, **а не** к целевой таблице: -| Operation | Support | Description | -|-----------|---------|-------------| +| Операция | Поддержка | Описание | +|----------|-----------|----------| | `DROP TABLE` | ✅ | Удаляет только алиас, целевая таблица остаётся без изменений | | `RENAME TABLE` | ✅ | Переименовывает только алиас, целевая таблица остаётся без изменений | +## Примеры использования {#usage-examples} +### Создание простого алиаса {#basic-alias-creation} -## Примеры использования - -### Создание простого псевдонима - -Создайте простой псевдоним в той же базе данных: +Создайте простой алиас в этой же базе данных: ```sql -- Создать исходную таблицу @@ -104,16 +109,17 @@ SELECT * FROM data_alias; └────┴──────┴───────┘ ``` -### Псевдоним для другой базы данных -Создайте псевдоним, который ссылается на таблицу в другой базе данных: +### Межбазовый псевдоним {#cross-database-alias} + +Создайте псевдоним, ссылающийся на таблицу в другой базе данных: ```sql --- Создание баз данных +-- Создать базы данных CREATE DATABASE db1; CREATE DATABASE db2; --- Создание исходной таблицы в db1 +-- Создать исходную таблицу в db1 CREATE TABLE db1.events ( timestamp DateTime, event_type String, @@ -121,10 +127,10 @@ CREATE TABLE db1.events ( ) ENGINE = MergeTree ORDER BY timestamp; --- Создание псевдонима в db2, ссылающегося на db1.events +-- Создать псевдоним в db2, указывающий на db1.events CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); --- Или с использованием формата database.table +-- Или используя формат database.table CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); -- Оба псевдонима работают одинаково @@ -132,7 +138,8 @@ INSERT INTO db2.events_alias VALUES (now(), 'click', 100); SELECT * FROM db2.events_alias2; ``` -### Операции записи через алиас + +### Операции записи через алиас {#write-operations} Все операции записи перенаправляются в целевую таблицу: @@ -151,20 +158,21 @@ INSERT INTO metrics_alias VALUES (now(), 'cpu_usage', 45.2), (now(), 'memory_usage', 78.5); --- Вставка с помощью SELECT +-- Вставка с SELECT INSERT INTO metrics_alias SELECT now(), 'disk_usage', number * 10 FROM system.numbers LIMIT 5; --- Проверка наличия данных в целевой таблице +-- Проверка данных в целевой таблице SELECT count() FROM metrics; -- Возвращает 7 SELECT count() FROM metrics_alias; -- Возвращает 7 ``` -### Изменение схемы -Операции `ALTER` изменяют схему целевой таблицы: +### Изменение схемы {#schema-modification} + +Операции ALTER изменяют схему целевой таблицы: ```sql CREATE TABLE users ( @@ -175,7 +183,7 @@ ORDER BY id; CREATE TABLE users_alias ENGINE = Alias('users'); --- Добавить столбец через псевдоним +-- Добавление столбца через псевдоним ALTER TABLE users_alias ADD COLUMN email String DEFAULT ''; -- Столбец добавляется в целевую таблицу @@ -190,7 +198,8 @@ DESCRIBE users; └───────┴────────┴──────────────┴────────────────────┘ ``` -### Модификации данных + +### Мутации данных {#data-mutations} Поддерживаются операции UPDATE и DELETE: @@ -227,10 +236,10 @@ SELECT * FROM products ORDER BY id; └────┴──────────┴───────┴────────┘ ``` -### Операции с партициями -Для партиционированных таблиц операции с партициями перенаправляются: +### Операции с партициями {#partition-operations} +Для секционированных таблиц операции с партициями передаются далее: ```sql CREATE TABLE logs ( @@ -248,20 +257,21 @@ INSERT INTO logs_alias VALUES ('2024-02-15', 'ERROR', 'message2'), ('2024-03-15', 'INFO', 'message3'); --- Отключение партиции через псевдоним +-- Отсоединить партицию через псевдоним ALTER TABLE logs_alias DETACH PARTITION '202402'; -SELECT count() FROM logs_alias; -- Возвращает 2 (партиция 202402 отключена) +SELECT count() FROM logs_alias; -- Возвращает 2 (партиция 202402 отсоединена) --- Подключение партиции обратно +-- Присоединить партицию обратно ALTER TABLE logs_alias ATTACH PARTITION '202402'; SELECT count() FROM logs_alias; -- Возвращает 3 ``` -### Оптимизация таблицы -Выполните операцию `OPTIMIZE` для слияния частей в целевой таблице: +### Оптимизация таблицы {#table-optimization} + +Оптимизируйте операции по слиянию частей в целевой таблице: ```sql CREATE TABLE events ( @@ -283,19 +293,20 @@ WHERE database = currentDatabase() AND table = 'events' AND active; --- Оптимизация через псевдоним +-- Оптимизация через алиас OPTIMIZE TABLE events_alias FINAL; --- Части объединены в целевой таблице +-- Части объединяются в целевой таблице SELECT count() FROM system.parts WHERE database = currentDatabase() AND table = 'events' AND active; -- Возвращает 1 ``` -### Управление псевдонимами -Псевдонимы можно переименовывать или удалять по отдельности: +### Управление алиасами {#alias-management} + +Алиасы можно переименовывать или удалять независимо: ```sql CREATE TABLE important_data ( @@ -308,15 +319,15 @@ INSERT INTO important_data VALUES (1, 'critical'), (2, 'important'); CREATE TABLE old_alias ENGINE = Alias('important_data'); --- Переименование псевдонима (целевая таблица остаётся без изменений) +-- Переименование псевдонима (целевая таблица не изменяется) RENAME TABLE old_alias TO new_alias; -- Создание ещё одного псевдонима для той же таблицы CREATE TABLE another_alias ENGINE = Alias('important_data'); --- Удаление одного псевдонима (целевая таблица и остальные псевдонимы остаются без изменений) +-- Удаление одного псевдонима (целевая таблица и другие псевдонимы не изменяются) DROP TABLE new_alias; -SELECT * FROM another_alias; -- Всё ещё работает -SELECT count() FROM important_data; -- Данные целы, возвращает 2 +SELECT * FROM another_alias; -- По-прежнему работает +SELECT count() FROM important_data; -- Данные не повреждены, возвращается 2 ``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md index 4220516d3e6..ac3ddf72127 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md @@ -7,9 +7,7 @@ title: 'Движок таблицы Buffer' doc_type: 'reference' --- - - -# Движок таблицы Buffer +# Движок таблицы Buffer {#buffer-table-engine} Буферизует данные, подлежащие записи, в оперативной памяти и периодически сбрасывает их в другую таблицу. При чтении данные одновременно считываются из буфера и из другой таблицы. @@ -21,27 +19,27 @@ doc_type: 'reference' Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) ``` -### Параметры движка +### Параметры движка {#engine-parameters} -#### `database` +#### `database` {#database} `database` – Имя базы данных. Можно использовать `currentDatabase()` или другое константное выражение, возвращающее строку. -#### `table` +#### `table` {#table} `table` – Таблица, в которую сбрасываются данные. -#### `num_layers` +#### `num_layers` {#num_layers} `num_layers` – Уровень параллелизма. Физически таблица будет представлена как `num_layers` независимых буферов. -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes` и `max_bytes` +#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes` и `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} Условия для сброса данных из буфера. -### Необязательные параметры движка +### Необязательные параметры движка {#optional-engine-parameters} -#### `flush_time`, `flush_rows` и `flush_bytes` +#### `flush_time`, `flush_rows` и `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} Условия для фонового сброса данных из буфера (отсутствие параметра или значение 0 означает отсутствие параметров `flush*`). @@ -49,15 +47,15 @@ Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_ Также, если выполнено хотя бы одно условие `flush*`, в фоновом режиме инициируется сброс. В отличие от параметров `max*`, параметры `flush*` позволяют настраивать фоновые сбросы отдельно, чтобы избежать добавления задержки для запросов `INSERT` в таблицы Buffer. -#### `min_time`, `max_time` и `flush_time` +#### `min_time`, `max_time` и `flush_time` {#min_time-max_time-and-flush_time} Условие по времени в секундах с момента первой записи в буфер. -#### `min_rows`, `max_rows` и `flush_rows` +#### `min_rows`, `max_rows` и `flush_rows` {#min_rows-max_rows-and-flush_rows} Условие по количеству строк в буфере. -#### `min_bytes`, `max_bytes` и `flush_bytes` +#### `min_bytes`, `max_bytes` и `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} Условие по количеству байт в буфере. @@ -84,7 +82,6 @@ CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, Вы можете задать пустые строковые литералы в одинарных кавычках для имени базы данных и имени таблицы. Это указывает на отсутствие целевой таблицы. В этом случае, когда условия сброса данных выполняются, буфер просто очищается. Это может быть полезно для хранения окна данных в памяти. - При чтении из таблицы Buffer данные обрабатываются как из буфера, так и из целевой таблицы (если она существует). Обратите внимание, что таблица Buffer не поддерживает индекс. Иными словами, данные в буфере полностью сканируются, что может быть медленно для больших буферов. (Для данных в подчинённой таблице будет использоваться индекс, который она поддерживает.) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md index f04780ecca4..883a540fbfb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md @@ -7,15 +7,11 @@ title: 'Движок таблицы Dictionary' doc_type: 'reference' --- - - -# Движок таблицы Dictionary +# Движок таблицы Dictionary {#dictionary-table-engine} Движок `Dictionary` представляет данные [словаря](../../../sql-reference/dictionaries/index.md) в виде таблицы ClickHouse. - - -## Пример +## Пример {#example} В качестве примера рассмотрим словарь `products` со следующей конфигурацией: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md index 38629ecd720..13e629e2e7b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок распределённой таблицы +# Движок распределённой таблицы {#distributed-table-engine} :::warning Распределённый движок в Cloud Чтобы создать движок распределённой таблицы в ClickHouse Cloud, можно использовать табличные функции [`remote` и `remoteSecure`](../../../sql-reference/table-functions/remote). @@ -21,7 +21,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#distributed-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] [SETTINGS name=value, ...] ``` -### Из таблицы +### Из таблицы {#distributed-from-a-table} Когда таблица `Distributed` указывает на таблицу на текущем сервере, вы можете заимствовать её схему: @@ -41,7 +41,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] ``` -### Параметры движка Distributed +### Параметры движка Distributed {#distributed-parameters} | Параметр | Описание | | ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -56,7 +56,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 * настройка [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) * [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) для примеров -### Настройки движка Distributed +### Настройки движка Distributed {#distributed-settings} | Параметр | Описание | Значение по умолчанию | @@ -102,7 +102,7 @@ SETTINGS Вместо имени базы данных вы можете использовать константное выражение, которое возвращает строку. Например: `currentDatabase()`. -## Кластеры +## Кластеры {#distributed-clusters} Кластеры настраиваются в [конфигурационном файле сервера](../../../operations/configuration-files.md): diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md index 05a23ce4e17..32bd946ad29 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md @@ -9,20 +9,16 @@ title: 'Движки таблиц Executable и ExecutablePool' doc_type: 'reference' --- - - -# Движки таблиц Executable и ExecutablePool +# Движки таблиц Executable и ExecutablePool {#executable-and-executablepool-table-engines} Движки таблиц `Executable` и `ExecutablePool` позволяют задать таблицу, строки которой генерируются скриптом, который вы пишете (путём вывода строк в **stdout**). Исполняемый скрипт хранится в каталоге `users_scripts` и может читать данные из любого источника. -- Таблицы `Executable`: скрипт выполняется при каждом запросе -- Таблицы `ExecutablePool`: поддерживается пул долгоживущих процессов, и для чтения процессы берутся из этого пула +* Таблицы `Executable`: скрипт выполняется при каждом запросе +* Таблицы `ExecutablePool`: поддерживается пул долгоживущих процессов, и для чтения процессы берутся из этого пула Дополнительно вы можете указать один или несколько входных запросов, результаты которых будут передаваться в **stdin** для чтения скриптом. - - -## Создание таблицы `Executable` +## Создание таблицы `Executable` {#creating-an-executable-table} Движку таблицы `Executable` требуются два параметра: имя скрипта и формат входящих данных. При необходимости можно передать один или несколько входных запросов: @@ -104,8 +100,7 @@ SELECT * FROM my_executable_table └───┴────────────┘ ``` - -## Передача результатов запроса в скрипт +## Передача результатов запроса в скрипт {#passing-query-results-to-a-script} Пользователи веб‑сайта Hacker News оставляют комментарии. В Python есть набор инструментов для обработки естественного языка (`nltk`) с `SentimentIntensityAnalyzer` для определения, являются ли комментарии положительными, отрицательными или нейтральными, — в том числе для присвоения значения от -1 (очень негативный комментарий) до 1 (очень позитивный комментарий). Давайте создадим таблицу `Executable`, которая вычисляет тональность комментариев Hacker News с помощью `nltk`. @@ -179,7 +174,6 @@ FROM sentiment Ответ будет выглядеть следующим образом: - ```response ┌───────id─┬─sentiment─┐ │ 7398199 │ 0.4404 │ @@ -205,8 +199,7 @@ FROM sentiment └──────────┴───────────┘ ``` - -## Создание таблицы `ExecutablePool` +## Создание таблицы `ExecutablePool` {#creating-an-executablepool-table} Синтаксис `ExecutablePool` похож на `Executable`, но у таблицы `ExecutablePool` есть несколько параметров, характерных именно для нее: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md index 491770e1a5f..bec15dea888 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md @@ -7,7 +7,7 @@ title: 'Внешние данные для обработки запросов' doc_type: 'reference' --- -# Внешние данные для обработки запросов +# Внешние данные для обработки запросов {#external-data-for-query-processing} ClickHouse позволяет отправлять на сервер данные, которые необходимы для обработки запроса, вместе с запросом `SELECT`. Эти данные помещаются во временную таблицу (см. раздел «Временные таблицы») и могут использоваться в запросе (например, в операторах `IN`). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md index 7295bed385e..e2fe7d77234 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Движок таблицы File +# Движок таблицы File {#file-table-engine} Движок таблицы File хранит данные в файле в одном из поддерживаемых [форматов файлов](/interfaces/formats#formats-overview) (`TabSeparated`, `Native` и т. д.). @@ -26,7 +26,7 @@ doc_type: 'reference' -## Использование на сервере ClickHouse +## Использование на сервере ClickHouse {#usage-in-clickhouse-server} ```sql File(Format) @@ -48,7 +48,7 @@ ClickHouse не позволяет указывать путь в файлово ::: -## Пример +## Пример {#example} **1.** Настройте таблицу `file_engine_table`: @@ -80,7 +80,7 @@ SELECT * FROM file_engine_table ``` -## Использование в clickhouse-local +## Использование в clickhouse-local {#usage-in-clickhouse-local} В [clickhouse-local](../../../operations/utilities/clickhouse-local.md) движок File, помимо параметра `Format`, принимает путь к файлу. Потоки ввода/вывода по умолчанию можно указывать с помощью числовых или понятных имён, таких как `0` или `stdin`, `1` или `stdout`. Можно читать и записывать сжатые файлы, исходя из дополнительного параметра движка или расширения файла (`gz`, `br` или `xz`). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md index ee7c142bd24..cc1006fcc4f 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md @@ -21,7 +21,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -57,7 +57,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `handle_error_mode` — Как обрабатывать ошибки в движке FileLog. Возможные значения: default (будет сгенерировано исключение, если не удалось разобрать сообщение), stream (сообщение об исключении и исходное сообщение будут сохранены во виртуальных столбцах `_error` и `_raw_message`). -## Описание +## Описание {#description} Поступившие записи отслеживаются автоматически, поэтому каждая запись в файле журнала учитывается только один раз. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md index 5987da1a3b8..9d9543fe694 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Движок таблицы GenerateRandom +# Движок таблицы GenerateRandom {#generaterandom-table-engine} Движок таблицы GenerateRandom генерирует случайные данные в соответствии с заданной схемой таблицы. @@ -21,7 +21,7 @@ doc_type: 'reference' -## Использование в ClickHouse Server +## Использование в ClickHouse Server {#usage-in-clickhouse-server} ```sql ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) @@ -34,7 +34,7 @@ ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) Он поддерживает все [типы данных](../../../sql-reference/data-types/index.md), которые могут храниться в таблице, за исключением `AggregateFunction`. -## Пример +## Пример {#example} **1.** Создайте таблицу `generate_engine_table`: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md index f4058e0abc2..ec76dffe7d2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md @@ -7,9 +7,7 @@ title: 'Специальные движки таблиц' doc_type: 'reference' --- - - -# Специальные движки таблиц +# Специальные движки таблиц {#special-table-engines} Существует три основные категории движков таблиц: @@ -26,7 +24,6 @@ doc_type: 'reference' Если вы заметили ошибку, отредактируйте YAML front matter соответствующих страниц. */ } - {/*AUTOGENERATED_START*/ } | Page | Description | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md index 5d41830f0d9..ad165544a38 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Табличный движок Join +# Табличный движок Join {#join-table-engine} Дополнительная подготовленная структура данных для использования в операциях [JOIN](/sql-reference/statements/select/join). @@ -19,7 +19,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -115,7 +115,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## Примеры использования +## Примеры использования {#example} Создание левой таблицы: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md index 0dedd9c57b3..9dc42339bda 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# Табличный движок KeeperMap +# Табличный движок KeeperMap {#keepermap-table-engine} Этот движок позволяет использовать кластер Keeper/ZooKeeper как согласованное хранилище ключ–значение с линеаризуемыми записями и последовательно согласованными чтениями. @@ -27,7 +27,7 @@ doc_type: 'reference' где path может быть любым другим допустимым путем в ZooKeeper. -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -81,9 +81,9 @@ PRIMARY KEY key Разумеется, можно вручную выполнить `CREATE TABLE` с тем же путём на несвязанных экземплярах ClickHouse, чтобы получить тот же эффект совместного использования данных. -## Поддерживаемые операции +## Поддерживаемые операции {#supported-operations} -### Вставки +### Вставки {#inserts} Когда в `KeeperMap` вставляются новые строки, если ключ не существует, создаётся новая запись с этим ключом. Если ключ существует и настройка `keeper_map_strict_mode` имеет значение `true`, выбрасывается исключение, в противном случае значение для ключа перезаписывается. @@ -94,7 +94,7 @@ PRIMARY KEY key INSERT INTO keeper_map_table VALUES ('какой-то ключ', 1, 'значение', 3.2); ``` -### Удаления +### Удаления {#deletes} Строки можно удалять с помощью запроса `DELETE` или команды `TRUNCATE`. Если ключ существует и параметр `keeper_map_strict_mode` установлен в значение `true`, операции выборки и удаления данных завершатся успешно только в том случае, если их можно выполнить атомарно. @@ -111,7 +111,7 @@ ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; TRUNCATE TABLE keeper_map_table; ``` -### Обновления +### Обновления {#updates} Значения можно изменять с помощью запроса `ALTER TABLE`. Первичный ключ изменить нельзя. Если параметр `keeper_map_strict_mode` имеет значение `true`, выборка и обновление данных выполнятся успешно только при атомарном выполнении операции. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md index dd8dc33029c..1c3c7e69810 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md @@ -11,7 +11,7 @@ doc_type: 'reference' -# Движок таблиц Memory +# Движок таблиц Memory {#memory-table-engine} :::note При использовании движка таблиц Memory в ClickHouse Cloud данные по проекту не реплицируются на все узлы (так задумано). Чтобы гарантировать, что все запросы направляются на один и тот же узел и движок Memory работает предсказуемо, вы можете: @@ -50,7 +50,7 @@ doc_type: 'reference' -## Использование +## Использование {#usage} **Инициализация настроек** @@ -67,7 +67,7 @@ ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 100 **Примечание:** Параметры ограничения `bytes` и `rows` могут быть заданы одновременно, однако будет использоваться меньшее из значений `max` и `min`. -## Примеры +## Примеры {#examples} ```sql CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md index ce5334d76ed..0455e83c946 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок таблицы Merge +# Движок таблицы Merge {#merge-table-engine} Движок `Merge` (не путать с `MergeTree`) сам не хранит данные, но позволяет одновременно читать из любого количества других таблиц. @@ -17,7 +17,7 @@ doc_type: 'reference' -## Создание таблицы +## Создание таблицы {#creating-a-table} ```sql CREATE TABLE ... Engine=Merge(db_name, tables_regexp) @@ -51,7 +51,7 @@ CREATE TABLE ... Engine=Merge(db_name, tables_regexp) -## Примеры +## Примеры {#examples} **Пример 1** diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md index b4ecdb6cd07..245785fc0ee 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md @@ -8,7 +8,7 @@ title: 'Движок таблицы Null' doc_type: 'reference' --- -# Движок таблицы Null +# Движок таблицы Null {#null-table-engine} При записи данных в таблицу `Null` они игнорируются. При чтении из таблицы `Null` результат будет пустым. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md index 384300df37f..0e9166dbaa5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок таблицы Set +# Движок таблицы Set {#set-table-engine} :::note В ClickHouse Cloud, если ваш сервис был создан с версией ранее 25.4, вам необходимо установить совместимость как минимум с 25.4 с помощью `SET compatibility=25.4`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md index 9e632a8c144..0766455d5c8 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Движок таблицы URL +# Движок таблицы URL {#url-table-engine} Выполняет чтение и запись данных на удалённый HTTP/HTTPS-сервер. Этот движок похож на движок [File](../../../engines/table-engines/special/file.md). @@ -51,7 +51,7 @@ doc_type: 'reference' -## Пример +## Пример {#example} **1.** Создайте таблицу `url_engine_table` на сервере: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md index 3f350814d00..e39fb505ef2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md @@ -7,6 +7,6 @@ title: 'Движок таблицы View' doc_type: 'reference' --- -# Движок таблицы View +# Движок таблицы View {#view-table-engine} Используется для реализации представлений (подробнее см. запрос `CREATE VIEW`). Он не хранит данные, а только сохраняет указанный запрос `SELECT`. При чтении из таблицы выполняется этот запрос (при этом из него удаляются все ненужные столбцы). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/concurrency.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/concurrency.md index c03fbce81ec..0e43f40950e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/concurrency.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/concurrency.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['concurrency', 'QPS'] --- -# Поддерживает ли ClickHouse частые одновременные запросы? +# Поддерживает ли ClickHouse частые одновременные запросы? {#does-clickhouse-support-frequent-concurrent-queries} ClickHouse разработан для систем оперативной аналитики, которые могут напрямую обслуживать внешних пользователей. Он способен обрабатывать аналитические запросы с низкой задержкой (менее 10 миллисекунд) и высокой степенью параллелизма (более 10 000 запросов в секунду) на базах данных петабайтного масштаба, объединяя исторические данные и вставки в реальном времени. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/cost-based.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/cost-based.md index 1d175685621..9c423be122b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/cost-based.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/cost-based.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['CBE', 'optimizer'] --- -# Есть ли у ClickHouse стоимостной оптимизатор? +# Есть ли у ClickHouse стоимостной оптимизатор? {#does-clickhouse-have-a-cost-based-optimizer} В ClickHouse есть отдельные механизмы стоимостной оптимизации, например, порядок чтения столбцов определяется оценочной стоимостью чтения сжатых диапазонов данных с диска. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/datalake.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/datalake.md index f6d59ecb2b7..e1baba198e2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/datalake.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['озеро данных', 'lakehouse'] --- -# Поддерживает ли ClickHouse озёра данных? +# Поддерживает ли ClickHouse озёра данных? {#does-clickhouse-support-data-lakes} ClickHouse поддерживает озёра данных (Data Lakes), включая Iceberg, Delta Lake, Apache Hudi, Apache Paimon, Hive. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/dependencies.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/dependencies.md index da880aed909..1b42e44c47c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/dependencies.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/dependencies.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['зависимости', 'сторонние'] --- -# Какие сторонние зависимости нужны для запуска ClickHouse? +# Какие сторонние зависимости нужны для запуска ClickHouse? {#what-are-the-3rd-party-dependencies-for-running-clickhouse} ClickHouse не имеет никаких runtime-зависимостей. Он распространяется как один исполняемый бинарный файл, который является полностью самодостаточным. Это приложение предоставляет всю функциональность кластера, обрабатывает запросы, работает как рабочий узел кластера, как система координации, реализующая алгоритм консенсуса RAFT, а также как клиент или локальный движок запросов. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md index b6eea1eebba..ee93d03b28a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['distributed', 'join'] --- -# Поддерживает ли ClickHouse распределённый JOIN? +# Поддерживает ли ClickHouse распределённый JOIN? {#does-clickhouse-support-distributed-join} ClickHouse поддерживает распределённый JOIN на кластере. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/federated.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/federated.md index e64fa273549..42170743872 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/federated.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/federated.md @@ -8,22 +8,22 @@ doc_type: 'reference' keywords: ['federated', 'hybrid', 'postgres', 'mysql', 'sqlite', 'odbc', 'jdbc'] --- -# Поддерживает ли ClickHouse федеративные запросы? +# Поддерживает ли ClickHouse федеративные запросы? {#does-clickhouse-support-federated-queries} ClickHouse имеет наиболее широкую поддержку федеративных запросов и гибридного выполнения запросов среди аналитических баз данных. Он поддерживает выполнение запросов к внешним базам данных: -- PostgreSQL -- MySQL -- MongoDB -- Redis -- любые источники данных через ODBC -- любые источники данных через JDBC -- любые источники данных Arrow Flight -- потоковые источники данных, такие как Kafka и RabbitMQ -- Data Lake‑хранилища, такие как Iceberg, Delta Lake, Apache Hudi, Apache Paimon -- внешние файлы, расположенные в общих системах хранения, таких как AWS S3, GCS, Minio, Cloudflare R2, Azure Blob Storage, Alicloud OSS, Tencent COS, а также в локальном хранилище, с широким набором форматов данных +* PostgreSQL +* MySQL +* MongoDB +* Redis +* любые источники данных через ODBC +* любые источники данных через JDBC +* любые источники данных Arrow Flight +* потоковые источники данных, такие как Kafka и RabbitMQ +* Data Lake‑хранилища, такие как Iceberg, Delta Lake, Apache Hudi, Apache Paimon +* внешние файлы, расположенные в общих системах хранения, таких как AWS S3, GCS, Minio, Cloudflare R2, Azure Blob Storage, Alicloud OSS, Tencent COS, а также в локальном хранилище, с широким набором форматов данных ClickHouse может объединять несколько разнородных источников данных в одном запросе. Также он предоставляет возможность гибридного выполнения запросов, комбинируя локальные ресурсы и передавая часть запроса на удалённые машины. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/index.md index de43f0aa585..146b0fea535 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/index.md @@ -8,18 +8,18 @@ description: 'Страница-индекс со списком общих во doc_type: 'landing-page' --- -# Общие вопросы о ClickHouse +# Общие вопросы о ClickHouse {#general-questions-about-clickhouse} -- [Что такое ClickHouse?](../../intro.md) -- [Почему ClickHouse такой быстрый?](../../concepts/why-clickhouse-is-so-fast.mdx) -- [Кто использует ClickHouse?](../../faq/general/who-is-using-clickhouse.md) -- [Что означает название «ClickHouse»?](../../faq/general/dbms-naming.md) -- [Что означает «Не тормозит»?](../../faq/general/ne-tormozit.md) -- [Что такое OLAP?](../../faq/general/olap.md) -- [Что такое колоночная база данных?](../../faq/general/columnar-database.md) -- [Как выбрать первичный ключ?](../../guides/best-practices/sparse-primary-indexes.md) -- [Почему не использовать что-то вроде MapReduce?](../../faq/general/mapreduce.md) -- [Как внести вклад в разработку ClickHouse?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) +* [Что такое ClickHouse?](../../intro.md) +* [Почему ClickHouse такой быстрый?](../../concepts/why-clickhouse-is-so-fast.mdx) +* [Кто использует ClickHouse?](../../faq/general/who-is-using-clickhouse.md) +* [Что означает название «ClickHouse»?](../../faq/general/dbms-naming.md) +* [Что означает «Не тормозит»?](../../faq/general/ne-tormozit.md) +* [Что такое OLAP?](../../faq/general/olap.md) +* [Что такое колоночная база данных?](../../faq/general/columnar-database.md) +* [Как выбрать первичный ключ?](../../guides/best-practices/sparse-primary-indexes.md) +* [Почему не использовать что-то вроде MapReduce?](../../faq/general/mapreduce.md) +* [Как внести вклад в разработку ClickHouse?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) :::info Не нашли то, что искали? Ознакомьтесь с нашей [Базой знаний](/knowledgebase/), а также просмотрите множество полезных статей в этой документации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/sql.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/sql.md index 3eaec0c1439..aa08ff2637a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/sql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/sql.md @@ -8,46 +8,46 @@ doc_type: 'reference' keywords: ['синтаксис SQL', 'ANSI SQL'] --- -# Какой синтаксис SQL поддерживает ClickHouse? +# Какой синтаксис SQL поддерживает ClickHouse? {#what-sql-syntax-does-clickhouse-support} ClickHouse полностью поддерживает синтаксис SQL, включая такие возможности, как: -- SQL/JSON и тип данных JSON (SQL-2023) -- оконные функции (SQL-2003) -- общие табличные выражения (CTE) и рекурсивные запросы (SQL-1999) -- ROLLUP, CUBE и GROUPING SETS (SQL-1999) -- полная поддержка RBAC (SQL-1999) -- коррелированные подзапросы (SQL-1992); +* SQL/JSON и тип данных JSON (SQL-2023) +* оконные функции (SQL-2003) +* общие табличные выражения (CTE) и рекурсивные запросы (SQL-1999) +* ROLLUP, CUBE и GROUPING SETS (SQL-1999) +* полная поддержка RBAC (SQL-1999) +* коррелированные подзапросы (SQL-1992); Поддержка подтверждена бенчмарками TPC-H и TPC-DS, а также SQLTest. ClickHouse внедрил многие функции до того, как они были стандартизованы ISO/IEC, в том числе: -- условные агрегатные функции -- агрегатные функции `any` -- `least` и `greatest` -- `GROUP BY ALL` -- расширенное использование псевдонимов -- символ подчёркивания в числовых литералах +* условные агрегатные функции +* агрегатные функции `any` +* `least` и `greatest` +* `GROUP BY ALL` +* расширенное использование псевдонимов +* символ подчёркивания в числовых литералах ClickHouse расширяет SQL, добавляя важные улучшения, повышающие удобство работы: -- неограниченное использование псевдонимов -- псевдонимы внутри предложения WITH -- комбинаторы агрегатных функций -- параметризованные агрегатные функции -- приближённые агрегатные функции -- встроенные и большие целочисленные типы данных, десятичные числа повышенной точности -- функции высшего порядка для обработки массивов -- предложение ARRAY JOIN и функция arrayJoin -- агрегирование массивов -- предложение LIMIT BY -- GROUP BY WITH TOTALS -- AS OF JOIN -- ANY/ALL JOIN -- естественный синтаксис для JSON -- завершающая запятая в списке столбцов -- порядок предложений FROM ... SELECT -- типобезопасные параметры запросов и параметризованные представления +* неограниченное использование псевдонимов +* псевдонимы внутри предложения WITH +* комбинаторы агрегатных функций +* параметризованные агрегатные функции +* приближённые агрегатные функции +* встроенные и большие целочисленные типы данных, десятичные числа повышенной точности +* функции высшего порядка для обработки массивов +* предложение ARRAY JOIN и функция arrayJoin +* агрегирование массивов +* предложение LIMIT BY +* GROUP BY WITH TOTALS +* AS OF JOIN +* ANY/ALL JOIN +* естественный синтаксис для JSON +* завершающая запятая в списке столбцов +* порядок предложений FROM ... SELECT +* типобезопасные параметры запросов и параметризованные представления Некоторые из них потенциально могут быть включены в будущие стандарты SQL, при этом уже доступны пользователям ClickHouse. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/updates.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/updates.md index 0bc89452d86..f15ed8ece48 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/updates.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/general/updates.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['обновления', 'реальное время'] --- -# Поддерживает ли ClickHouse обновления в режиме реального времени? +# Поддерживает ли ClickHouse обновления в режиме реального времени? {#does-clickhouse-support-real-time-updates} ClickHouse поддерживает оператор UPDATE и способен выполнять обновления в реальном времени с той же скоростью, с какой выполняет операции INSERT. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/index.md index ec26f9bd2c5..f9fc554fb47 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/index.md @@ -8,15 +8,15 @@ description: 'Главная страница со списком вопросо doc_type: 'landing-page' --- -# Вопросы об интеграции ClickHouse с другими системами +# Вопросы об интеграции ClickHouse с другими системами {#questions-about-integrating-clickhouse-and-other-systems} -- [Как экспортировать данные из ClickHouse в файл?](https://clickhouse.com/docs/knowledgebase/file-export) -- [Как импортировать JSON в ClickHouse?](/integrations/data-ingestion/data-formats/json/intro.md) -- [Как подключить Kafka к ClickHouse?](/integrations/data-ingestion/kafka/index.md) -- [Могу ли я подключить своё Java‑приложение к ClickHouse?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) -- [Может ли ClickHouse читать таблицы из MySQL?](/integrations/mysql) -- [Может ли ClickHouse читать таблицы из PostgreSQL](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) -- [Что делать, если возникают проблемы с кодировками при подключении к Oracle через ODBC?](/faq/integration/oracle-odbc.md) +* [Как экспортировать данные из ClickHouse в файл?](https://clickhouse.com/docs/knowledgebase/file-export) +* [Как импортировать JSON в ClickHouse?](/integrations/data-ingestion/data-formats/json/intro.md) +* [Как подключить Kafka к ClickHouse?](/integrations/data-ingestion/kafka/index.md) +* [Могу ли я подключить своё Java‑приложение к ClickHouse?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) +* [Может ли ClickHouse читать таблицы из MySQL?](/integrations/mysql) +* [Может ли ClickHouse читать таблицы из PostgreSQL](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) +* [Что делать, если возникают проблемы с кодировками при подключении к Oracle через ODBC?](/faq/integration/oracle-odbc.md) :::info Не нашли то, что искали? Ознакомьтесь с нашей [Базой знаний](/knowledgebase/) и также просмотрите множество полезных статей в документации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/json-import.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/json-import.md index ed9f79a092e..81eeae88c47 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/json-import.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/json-import.md @@ -16,7 +16,7 @@ ClickHouse поддерживает широкий спектр [формато -## Примеры +## Примеры {#examples} С помощью [HTTP-интерфейса](../../interfaces/http.md): diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md index 7ec753572ed..2ddcfdbf4e6 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md @@ -8,9 +8,7 @@ doc_type: 'guide' keywords: ['oracle', 'odbc', 'encoding', 'integration', 'external dictionary'] --- - - -# Что делать, если у меня возникают проблемы с кодировками при работе с Oracle через ODBC? +# Что делать, если у меня возникают проблемы с кодировками при работе с Oracle через ODBC? {#oracle-odbc-encodings} Если вы используете Oracle как источник внешних словарей ClickHouse через драйвер Oracle ODBC, необходимо установить корректное значение переменной окружения `NLS_LANG` в файле `/etc/default/clickhouse`. Дополнительную информацию см. в [Oracle NLS_LANG FAQ](https://www.oracle.com/technetwork/products/globalization/nls-lang-099431.html). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md index 23d95d0b2b4..8067e2b45d9 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md @@ -30,7 +30,7 @@ TTL также может использоваться не только для -## DELETE FROM +## DELETE FROM {#delete-from} [DELETE FROM](/sql-reference/statements/delete.md) позволяет выполнять стандартные запросы DELETE в ClickHouse. Строки, отобранные условием фильтрации, помечаются как удалённые и не возвращаются в будущих результатах запросов. Очистка строк происходит асинхронно. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/index.md index c87e91e99be..88853edcfc4 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/operations/index.md @@ -8,16 +8,16 @@ doc_type: 'landing-page' keywords: ['эксплуатация', 'администрирование', 'развертывание', 'управление кластерами', 'частые вопросы'] --- -# Вопросы по эксплуатации серверов и кластеров ClickHouse +# Вопросы по эксплуатации серверов и кластеров ClickHouse {#question-about-operating-clickhouse-servers-and-clusters} -- [Какую версию ClickHouse использовать в продакшене?](/faq/operations/production.md) -- [Можно ли развернуть ClickHouse с раздельными хранилищем и вычислительными ресурсами?](/faq/operations/separate_storage.md) -- [Можно ли удалять старые данные из таблицы ClickHouse?](/faq/operations/delete-old-data.md) -- [Как настроить ClickHouse Keeper?](/guides/sre/keeper/index.md) -- [Может ли ClickHouse интегрироваться с LDAP?](/guides/sre/user-management/configuring-ldap.md) -- [Как настроить пользователей, роли и права доступа в ClickHouse?](/guides/sre/user-management/index.md) -- [Можно ли обновлять или удалять строки в ClickHouse?](/guides/starter_guides/mutations.md) -- [Поддерживает ли ClickHouse мульти-региональную (multi-region) репликацию?](/faq/operations/multi-region-replication.md) +* [Какую версию ClickHouse использовать в продакшене?](/faq/operations/production.md) +* [Можно ли развернуть ClickHouse с раздельными хранилищем и вычислительными ресурсами?](/faq/operations/separate_storage.md) +* [Можно ли удалять старые данные из таблицы ClickHouse?](/faq/operations/delete-old-data.md) +* [Как настроить ClickHouse Keeper?](/guides/sre/keeper/index.md) +* [Может ли ClickHouse интегрироваться с LDAP?](/guides/sre/user-management/configuring-ldap.md) +* [Как настроить пользователей, роли и права доступа в ClickHouse?](/guides/sre/user-management/index.md) +* [Можно ли обновлять или удалять строки в ClickHouse?](/guides/starter_guides/mutations.md) +* [Поддерживает ли ClickHouse мульти-региональную (multi-region) репликацию?](/faq/operations/multi-region-replication.md) :::info Не нашли то, что искали? Ознакомьтесь с нашей [Базой знаний](/knowledgebase/) и просмотрите множество полезных статей в документации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/faq/use-cases/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/faq/use-cases/index.md index bed752b2c7b..2f0d96db811 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/faq/use-cases/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/faq/use-cases/index.md @@ -8,10 +8,10 @@ keywords: ['варианты использования ClickHouse', 'база doc_type: 'landing-page' --- -# Вопросы по использованию ClickHouse +# Вопросы по использованию ClickHouse {#questions-about-clickhouse-use-cases} -- [Могу ли я использовать ClickHouse в качестве базы данных временных рядов?](/knowledgebase/time-series) -- [Могу ли я использовать ClickHouse в качестве хранилища ключ-значение?](/knowledgebase/key-value) +* [Могу ли я использовать ClickHouse в качестве базы данных временных рядов?](/knowledgebase/time-series) +* [Могу ли я использовать ClickHouse в качестве хранилища ключ-значение?](/knowledgebase/key-value) :::info Не нашли то, что искали? Ознакомьтесь с нашей [Базой знаний](/knowledgebase/), а также с многочисленными полезными статьями в этой документации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md index 60073ad20dd..e55fb10b357 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md @@ -11,10 +11,10 @@ keywords: ['отзывы Amazon', 'датасет с отзывами покуп :::note Приведённые ниже запросы выполнялись на **Production**-инстансе ClickHouse Cloud. Для получения дополнительной информации см. раздел -["Playground specifications"](/getting-started/playground#specifications). +["Playground specifications"](/getting-started/playground#specifications). ::: -## Загрузка набора данных +## Загрузка набора данных {#loading-the-dataset} 1. Не загружая данные в ClickHouse, мы можем обращаться к ним напрямую. Давайте выберем несколько строк, чтобы посмотреть, как они выглядят: @@ -122,7 +122,6 @@ FROM s3Cluster('default', 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/amazon_reviews/amazon_reviews_*.snappy.parquet') ``` - :::tip В ClickHouse Cloud имя кластера — `default`. Замените `default` на имя вашего кластера... или используйте табличную функцию `s3` (вместо `s3Cluster`), если кластера у вас нет. ::: @@ -152,8 +151,7 @@ ORDER BY size DESC Объём исходных данных составлял около 70 ГБ, но после сжатия в ClickHouse они занимают около 30 ГБ. - -## Примеры запросов +## Примеры запросов {#example-queries} 7. Давайте выполним несколько запросов. Ниже приведены 10 наиболее полезных отзывов в этом наборе данных: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md index 6be9205b8b6..e890b6540d1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md @@ -7,7 +7,7 @@ title: 'Анонимизированная веб-аналитика' doc_type: 'guide' --- -# Обезличенные данные веб-аналитики +# Обезличенные данные веб-аналитики {#anonymized-web-analytics-data} Этот набор данных состоит из двух таблиц, содержащих обезличенные данные веб-аналитики с запросами (`hits_v1`) и визитами (`visits_v1`). @@ -15,17 +15,17 @@ doc_type: 'guide' ## Скачать данные и выполнить их приём {#download-and-ingest-the-data} -### Скачайте сжатый TSV‑файл hits +### Скачайте сжатый TSV‑файл hits {#download-the-hits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/hits/tsv/hits_v1.tsv.xz | unxz --threads=`nproc` > hits_v1.tsv -# Проверьте контрольную сумму +# Проверьте контрольную сумму {#validate-the-checksum} md5sum hits_v1.tsv -# Контрольная сумма должна быть: f3631b6295bf06989c1437491f7592cb +# Контрольная сумма должна быть: f3631b6295bf06989c1437491f7592cb {#checksum-should-be-equal-to-f3631b6295bf06989c1437491f7592cb} ``` -### Создание базы данных и таблицы +### Создание базы данных и таблицы {#create-the-database-and-table} ```bash clickhouse-client --query "CREATE DATABASE IF NOT EXISTS datasets" @@ -45,7 +45,7 @@ clickhouse-client --query="CREATE TABLE default.hits_100m_obfuscated (WatchID UI ``` -### Импортируйте данные таблицы hits +### Импортируйте данные таблицы hits {#import-the-hits-data} ```bash cat hits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.hits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -62,13 +62,13 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.hits_v1" ``` -### Скачайте сжатый TSV‑файл visits +### Скачайте сжатый TSV‑файл visits {#download-the-visits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/visits/tsv/visits_v1.tsv.xz | unxz --threads=`nproc` > visits_v1.tsv -# Проверьте контрольную сумму +# Проверьте контрольную сумму {#validate-the-checksum} md5sum visits_v1.tsv -# Контрольная сумма должна быть: 6dafe1a0f24e59e3fc2d0fed85601de6 +# Контрольная сумма должна быть: 6dafe1a0f24e59e3fc2d0fed85601de6 {#checksum-should-be-equal-to-6dafe1a0f24e59e3fc2d0fed85601de6} ``` @@ -78,7 +78,7 @@ md5sum visits_v1.tsv clickhouse-client --query "CREATE TABLE datasets.visits_v1 ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), Goals Nested(ID UInt32, Serial UInt32, EventTime DateTime, Price Int64, OrderID String, CurrencyID UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, TraficSource Nested(ID Int8, SearchEngineID UInt16, AdvEngineID UInt8, PlaceID UInt16, SocialSourceNetworkID UInt8, Domain String, SearchPhrase String, SocialSourcePage String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), Market Nested(Type UInt8, GoalID UInt32, OrderID String, OrderPrice Int64, PP UInt32, DirectPlaceID UInt32, DirectOrderID UInt32, DirectBannerID UInt32, GoodID String, GoodName String, GoodQuantity Int32, GoodPrice Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192" ``` -### Импортируйте данные о посещениях +### Импортируйте данные о посещениях {#import-the-visits-data} ```bash cat visits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.visits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -95,7 +95,7 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.visits_v1" ``` -## Пример JOIN +## Пример JOIN {#an-example-join} Наборы данных hits и visits используются в тестовых процедурах ClickHouse; это один из запросов из набора тестов. Остальные diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md index ccf41157c95..8801c597b7d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md @@ -94,8 +94,7 @@ clickhouse-client --query "INSERT INTO mgbench.logs2 FORMAT CSVWithNames" < mgbe clickhouse-client --query "INSERT INTO mgbench.logs3 FORMAT CSVWithNames" < mgbench3.csv ``` - -## Выполните тестовые запросы +## Выполните тестовые запросы {#run-benchmark-queries} ```sql USE mgbench; @@ -209,7 +208,6 @@ ORDER BY machine_name, dt; ``` - ```sql -- Q1.6: Каков общий почасовой объём сетевого трафика по всем файловым серверам? diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md index 4fa7a616374..b1625e0f6fb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md @@ -8,15 +8,15 @@ keywords: ['данные о сотовых вышках', 'геоданные', doc_type: 'guide' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import ActionsMenu from '@site/docs/_snippets/_service_actions_menu.md'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; -import SupersetDocker from '@site/docs/_snippets/_add_superset_detail.md'; +import ActionsMenu from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_service_actions_menu.md'; +import SQLConsoleDetail from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; +import SupersetDocker from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_add_superset_detail.md'; import cloud_load_data_sample from '@site/static/images/_snippets/cloud-load-data-sample.png'; import cell_towers_1 from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' import add_a_database from '@site/static/images/getting-started/example-datasets/superset-add.png' @@ -30,7 +30,6 @@ import superset_radio_umts from '@site/static/images/getting-started/example-dat import superset_umts_netherlands from '@site/static/images/getting-started/example-datasets/superset-umts-netherlands.png' import superset_cell_tower_dashboard from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' - ## Цель {#goal} В этом руководстве вы узнаете, как: @@ -125,7 +124,7 @@ INSERT INTO cell_towers SELECT * FROM s3('https://datasets-documentation.s3.amaz -## Выполните несколько примеров запросов +## Выполните несколько примеров запросов {#examples} 1. Количество сотовых вышек по типу: @@ -172,7 +171,6 @@ SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10 Вы можете создать в ClickHouse [словарь (Dictionary)](../../sql-reference/dictionaries/index.md) для расшифровки этих значений. - ## Сценарий использования: использование геоданных {#use-case} Использование функции [`pointInPolygon`](/sql-reference/functions/geo/coordinates.md/#pointinpolygon). @@ -267,7 +265,6 @@ WHERE pointInPolygon((lon, lat), (SELECT * FROM moscow)) Получено 1 строк. Затрачено: 0.067 сек. Обработано 43.28 млн строк, 692.42 МБ (645.83 млн строк/сек., 10.33 ГБ/сек.) ``` - ## Обзор схемы {#review-of-the-schema} Прежде чем создавать визуализации в Superset, ознакомьтесь со столбцами, которые вы будете использовать. Этот набор данных в первую очередь содержит сведения о местоположении (долгота и широта) и типах радиотехнологий на базовых станциях мобильной связи по всему миру. Описание столбцов можно найти на [форуме сообщества](https://community.opencellid.org/t/documenting-the-columns-in-the-downloadable-cells-database-csv/186). Столбцы, используемые в визуализациях, которые вы будете строить, описаны ниже. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md index d9ac9090303..3a53af2496d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md @@ -15,7 +15,7 @@ doc_type: 'guide' Набор данных содержит 26 файлов в формате `Parquet`, размещённых на [huggingface.co](https://huggingface.co/api/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M/parquet/default/train/). Файлы называются `0.parquet`, `1.parquet`, ..., `25.parquet`. Чтобы просмотреть несколько строк этого набора данных в качестве примера, перейдите на эту [страницу Hugging Face](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M). -## Создание таблицы +## Создание таблицы {#create-table} Создайте таблицу `dbpedia` для хранения идентификатора статьи, заголовка, текста и векторного представления (эмбеддинга): @@ -31,7 +31,7 @@ CREATE TABLE dbpedia ``` -## Загрузка таблицы +## Загрузка таблицы {#load-table} Чтобы загрузить набор данных из всех файлов Parquet, выполните следующую команду в оболочке: @@ -74,7 +74,7 @@ FROM dbpedia _Ближайшими соседями_ являются документы, изображения или другой контент, релевантный запросу пользователя. Извлечённые результаты являются ключевыми входными данными для Retrieval Augmented Generation (RAG) в приложениях генеративного ИИ. -## Выполнить векторный поиск ближайших соседей методом полного перебора +## Выполнить векторный поиск ближайших соседей методом полного перебора {#run-a-brute-force-vector-similarity-search} Поиск KNN (k ближайших соседей) или поиск методом полного перебора предполагает вычисление расстояния от каждого вектора в наборе данных до вектора-запроса (эмбеддинга), а затем упорядочивание этих расстояний для получения ближайших соседей. Для набора данных `dbpedia` @@ -119,7 +119,7 @@ LIMIT 20 Также зафиксируйте задержку выполнения запроса при холодном файловом кэше ОС и с `max_threads=1`, чтобы оценить реальное использование вычислительных ресурсов и пропускную способность подсистемы хранения (экстраполируйте это на производственный набор данных с миллионами векторов!). -## Создание индекса векторного сходства +## Создание индекса векторного сходства {#build-vector-similarity-index} Выполните следующий SQL-запрос, чтобы определить и построить индекс векторного сходства для столбца `vector`: @@ -134,7 +134,7 @@ ALTER TABLE dbpedia MATERIALIZE INDEX vector_index SETTINGS mutations_sync = 2; Построение и сохранение индекса может занять несколько минут в зависимости от количества доступных ядер CPU и пропускной способности подсистемы хранения. -## Выполнение ANN-поиска +## Выполнение ANN-поиска {#perform-ann-search} *Approximate Nearest Neighbours* (ANN, приближённые ближайшие соседи) — это группа техник (например, специальные структуры данных, такие как графы и случайные леса), которые позволяют выполнять поиск значительно быстрее, чем точный векторный поиск. Точность результатов, как правило, «достаточно хороша» для практического применения. Во многих приближённых техниках предусмотрены параметры для настройки компромисса между точностью результатов и временем поиска. @@ -181,7 +181,7 @@ LIMIT 20 ``` -## Генерация эмбеддингов для поискового запроса +## Генерация эмбеддингов для поискового запроса {#generating-embeddings-for-search-query} Запросы поиска по сходству, рассмотренные до этого момента, используют один из существующих векторов в таблице `dbpedia` в качестве поискового вектора. В реальных приложениях поисковый вектор необходимо @@ -230,7 +230,7 @@ while True: ``` -## Демонстрационное приложение «Вопрос-ответ» +## Демонстрационное приложение «Вопрос-ответ» {#q-and-a-demo-application} Приведённые выше примеры демонстрировали семантический поиск и извлечение документов с использованием ClickHouse. Далее представлено очень простое, но обладающее высоким потенциалом демонстрационное приложение генеративного ИИ. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md index 9f0aad188f2..6dadef5c183 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md @@ -24,7 +24,7 @@ import visualization_4 from '@site/static/images/getting-started/example-dataset таких как магазины, рестораны, парки, игровые площадки и памятники. Он также включает дополнительные метаданные об этих местах, такие как категории и данные из социальных сетей. -## Исследование данных +## Исследование данных {#data-exploration} Для исследования данных мы будем использовать [`clickhouse-local`](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) — небольшую утилиту командной строки, которая предоставляет полноценный движок ClickHouse. Также вы можете использовать @@ -148,7 +148,7 @@ DESCRIBE s3('s3://fsq-os-places-us-east-1/release/dt=2025-04-08/places/parquet/* ``` -## Загрузка данных в ClickHouse +## Загрузка данных в ClickHouse {#loading-the-data} Если вы хотите сохранять данные на диске, вы можете использовать `clickhouse-server` или ClickHouse Cloud. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md index 78fb0c5d90e..4371f6f28d3 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md @@ -29,7 +29,7 @@ import superset_authors_matrix_v2 from '@site/static/images/getting-started/exam * `line_changes` — 2.7G — 7,535,157 строк -## Генерация данных +## Генерация данных {#generating-the-data} Этот шаг необязателен. Мы свободно распространяем эти данные — см. раздел [Downloading and inserting the data](#downloading-and-inserting-the-data). @@ -73,7 +73,7 @@ CREATE TABLE git.commits * Linux — `~/clickhouse git-import` — 160 минут -## Загрузка и вставка данных +## Загрузка и вставка данных {#downloading-and-inserting-the-data} Следующие данные можно использовать для воспроизведения рабочего окружения. Также этот набор данных доступен на play.clickhouse.com — подробности см. в разделе [Queries](#queries). @@ -220,7 +220,7 @@ FROM s3('https://datasets-documentation.s3.amazonaws.com/github/commits/clickhou Этот набор данных доступен на [play.clickhouse.com](https://sql.clickhouse.com?query_id=DCQPNPAIMAQXRLHYURLKVJ) в базе данных `git_clickhouse`. Для всех запросов мы предоставляем ссылку на эту среду, при необходимости подставляя соответствующее имя базы данных. Обратите внимание, что результаты в play могут отличаться от представленных здесь из‑за различий во времени сбора данных. -### История одного файла +### История одного файла {#history-of-a-single-file} Самый простой из запросов. Здесь мы просматриваем все сообщения коммитов для `StorageReplicatedMergeTree.cpp`. Поскольку они, вероятно, более интересны, мы сортируем их по дате, начиная с самых свежих сообщений. @@ -296,7 +296,7 @@ LIMIT 10 Обратите внимание, что существует более сложный вариант этого запроса, позволяющий получить [построчную историю коммитов файла](#line-by-line-commit-history-of-a-file) с учётом переименований. -### Найти текущие активные файлы +### Найти текущие активные файлы {#find-the-current-active-files} Это важно для последующего анализа, когда нам нужно учитывать только текущие файлы в репозитории. Мы определяем этот набор как файлы, которые не были переименованы или удалены (а затем заново добавлены/переименованы). @@ -428,7 +428,7 @@ git ls-files | grep -v -E 'generated\.cpp|^(contrib|docs?|website|libs/(libcityh Эти различия не должны существенно повлиять на наш анализ. **Будем признательны за улучшенные версии этого запроса**. -### Список файлов с наибольшим числом изменений +### Список файлов с наибольшим числом изменений {#list-files-with-most-modifications} Если ограничиться текущими файлами, считаем числом изменений сумму удалений и добавлений. @@ -484,7 +484,7 @@ LIMIT 10 ``` -### В какой день недели обычно совершаются коммиты? +### В какой день недели обычно совершаются коммиты? {#what-day-of-the-week-do-commits-usually-occur} [play](https://sql.clickhouse.com?query_id=GED2STFSYJDRAA59H8RLIV) @@ -510,7 +510,7 @@ GROUP BY dayOfWeek(time) AS day_of_week Это вполне логично: по пятницам наблюдается небольшое снижение продуктивности. Здорово видеть, что люди коммитят код по выходным! Огромное спасибо нашим контрибьюторам! -### История подкаталога/файла — количество строк, коммитов и контрибьюторов в динамике +### История подкаталога/файла — количество строк, коммитов и контрибьюторов в динамике {#history-of-subdirectoryfile---number-of-lines-commits-and-contributors-over-time} Это приведёт к очень большому результату запроса, который невозможно разумно отобразить или визуализировать без фильтрации. Поэтому в следующем примере мы позволяем отфильтровать данные по файлу или подкаталогу. Здесь мы группируем по неделям с помощью функции `toStartOfWeek` — при необходимости адаптируйте под свои задачи. @@ -555,7 +555,7 @@ LIMIT 10 For commits and authors -### Вывести список файлов с максимальным числом авторов +### Вывести список файлов с максимальным числом авторов {#list-files-with-maximum-number-of-authors} Ограничить выборку только текущими файлами. @@ -611,7 +611,7 @@ LIMIT 10 ``` -### Самые старые строки кода в репозитории +### Самые старые строки кода в репозитории {#oldest-lines-of-code-in-the-repository} Только по текущим файлам. @@ -669,7 +669,7 @@ LIMIT 10 ``` -### Файлы с самой длинной историей +### Файлы с самой длинной историей {#files-with-longest-history} Только текущие файлы. @@ -728,7 +728,7 @@ LIMIT 10 Наша основная структура данных MergeTree, разумеется, постоянно развивается и имеет долгую историю доработок! -### Распределение вклада контрибьюторов между документацией и кодом в течение месяца +### Распределение вклада контрибьюторов между документацией и кодом в течение месяца {#distribution-of-contributors-with-respect-to-docs-and-code-over-the-month} **Во время сбора данных изменения в папке `docs/` были исключены из анализа из‑за очень запутанной истории коммитов. Поэтому результаты этого запроса не являются точными.** @@ -792,7 +792,7 @@ FROM Возможно, чуть больше ближе к концу месяца, но в целом распределение остаётся достаточно равномерным. Однако на эти данные нельзя полностью полагаться из‑за фильтрации фильтром docs при вставке. -### Авторы с наиболее разнообразным вкладом +### Авторы с наиболее разнообразным вкладом {#authors-with-the-most-diverse-impact} Под разнообразием вклада здесь мы понимаем количество уникальных файлов, в которые автор вносил изменения. @@ -870,7 +870,7 @@ LIMIT 10 ``` -### Избранные файлы автора +### Избранные файлы автора {#favorite-files-for-an-author} Здесь мы выбираем нашего основателя, [Алексея Миловидова](https://github.com/alexey-milovidov), и ограничиваем анализ только текущими файлами. @@ -957,7 +957,7 @@ LIMIT 10 Это, возможно, лучше отражает его сферы интересов. -### Самые большие файлы с наименьшим числом авторов +### Самые большие файлы с наименьшим числом авторов {#largest-files-with-lowest-number-of-authors} Для этого сначала нужно определить самые большие файлы. Оценивать их с помощью полного восстановления каждого файла по всей истории коммитов — слишком затратно! @@ -1130,7 +1130,7 @@ LIMIT 10 ``` -### Распределение коммитов и строк кода по времени: по дням недели, по авторам, для отдельных поддиректорий +### Распределение коммитов и строк кода по времени: по дням недели, по авторам, для отдельных поддиректорий {#commits-and-lines-of-code-distribution-by-time-by-weekday-by-author-for-specific-subdirectories} Мы рассматриваем это как количество добавленных и удалённых строк по дням недели. В данном случае мы сосредотачиваемся на [директории Functions](https://github.com/ClickHouse/ClickHouse/tree/master/src/Functions) @@ -1258,7 +1258,7 @@ FROM ``` -### Матрица авторов, показывающая, какие из них склонны переписывать код других авторов +### Матрица авторов, показывающая, какие из них склонны переписывать код других авторов {#matrix-of-authors-that-shows-what-authors-tends-to-rewrite-another-authors-code} `sign = -1` указывает на удаление кода. Мы не учитываем знаки препинания и вставку пустых строк. @@ -1313,7 +1313,7 @@ LIMIT 100 Superset матрица авторов v2 -### Кто делает наибольшую долю коммитов в каждый день недели? +### Кто делает наибольшую долю коммитов в каждый день недели? {#who-is-the-highest-percentage-contributor-per-day-of-week} Если смотреть только на количество коммитов: @@ -1514,7 +1514,7 @@ LIMIT 5 BY root ``` -### Какой процент кода автора был удалён другими авторами? +### Какой процент кода автора был удалён другими авторами? {#what-percentage-of-code-for-an-author-has-been-removed-by-other-authors} Для ответа на этот вопрос нам нужно количество строк, написанных автором, разделить на общее количество его строк, которые были удалены другим участником. @@ -1565,7 +1565,7 @@ LIMIT 10 ``` -### Список файлов, которые переписывались чаще всего? +### Список файлов, которые переписывались чаще всего? {#list-files-that-were-rewritten-most-number-of-times} Самый простой способ ответить на этот вопрос — просто посчитать максимальное количество изменений строк для каждого пути (ограничившись текущими файлами), например: @@ -1708,7 +1708,7 @@ LIMIT 10 ``` -### В какой день недели у кода наибольший шанс остаться в репозитории? +### В какой день недели у кода наибольший шанс остаться в репозитории? {#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository} Для этого нам нужно однозначно идентифицировать строку кода. Мы делаем это (так как одна и та же строка может встречаться в файле несколько раз) с помощью пути и содержимого строки. @@ -1771,7 +1771,7 @@ GROUP BY dayOfWeek(added_day) AS day_of_week_added ``` -### Файлы, отсортированные по среднему возрасту кода +### Файлы, отсортированные по среднему возрасту кода {#files-sorted-by-average-code-age} Этот запрос использует тот же принцип, что и [В какой день недели у кода наибольший шанс остаться в репозитории](#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository): мы стремимся однозначно идентифицировать строку кода по пути и её содержимому. Это позволяет определить интервал между моментом, когда строка была добавлена, и моментом, когда она была удалена. При этом мы ограничиваемся текущими файлами и актуальным кодом и усредняем время для каждого файла по всем строкам. @@ -1862,7 +1862,7 @@ LIMIT 10 ``` -### Кто, как правило, пишет больше тестов / C++‑кода / комментариев? +### Кто, как правило, пишет больше тестов / C++‑кода / комментариев? {#who-tends-to-write-more-tests--cpp-code--comments} Есть несколько способов подойти к этому вопросу. Если сосредоточиться на соотношении кода и тестов, то этот запрос относительно прост — нужно посчитать количество изменений в каталогах, содержащих `tests`, и вычислить их долю от общего числа изменений. @@ -1996,7 +1996,7 @@ LIMIT 10 Обратите внимание, что сортировка ведётся по количеству вкладов в код. У всех наших крупнейших контрибьюторов удивительно высокий процент — это одна из причин, почему наш код такой читаемый. -### Как со временем меняется доля кода и комментариев в коммитах автора? +### Как со временем меняется доля кода и комментариев в коммитах автора? {#how-does-an-authors-commits-change-over-time-with-respect-to-codecomments-percentage} Вычислить это по каждому автору тривиально, @@ -2114,7 +2114,7 @@ LIMIT 20 Обнадеживает, что наша доля комментариев остается примерно постоянной и не снижается по мере роста стажа авторов. -### Каково среднее время до переписывания кода и медиана (период «полураспада» кода)? +### Каково среднее время до переписывания кода и медиана (период «полураспада» кода)? {#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay} Мы можем использовать тот же принцип, что и в разделе [List files that were rewritten most number of time or by most of authors](#list-files-that-were-rewritten-most-number-of-times) для определения переписываний, но применить его ко всем файлам. Оконная функция позволяет вычислить время между переписываниями для каждого файла. На этой основе мы можем посчитать среднее и медиану по всем файлам. @@ -2175,7 +2175,7 @@ FROM rewrites ``` -### В какое время писать код хуже всего с точки зрения наибольшей вероятности его последующего переписывания? +### В какое время писать код хуже всего с точки зрения наибольшей вероятности его последующего переписывания? {#what-is-the-worst-time-to-write-code-in-sense-that-the-code-has-highest-chance-to-be-re-written} Аналогично разделам [Каково среднее время до переписывания кода и медиана (период полураспада кода)?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) и [Список файлов, которые переписывались больше всего раз или наибольшим числом авторов](#list-files-that-were-rewritten-most-number-of-times), за исключением того, что здесь мы агрегируем по дням недели. При необходимости скорректируйте, например, по месяцам года. @@ -2240,7 +2240,7 @@ GROUP BY dayOfWeek ``` -### Чей код «держится» дольше всего? +### Чей код «держится» дольше всего? {#which-authors-code-is-the-most-sticky} Мы определяем «липкость» как то, как долго код автора остается в репозитории, прежде чем его перепишут. Аналогично предыдущему вопросу [Какое среднее время до переписывания кода и медианное значение (период полураспада деградации кода)?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) — используем тот же критерий переписывания, то есть 50% добавлений и 50% удалений в файле. Мы вычисляем среднее время до переписывания для каждого автора и учитываем только контрибьюторов с более чем двумя файлами. @@ -2318,7 +2318,7 @@ LIMIT 10 ``` -### Наибольшее количество последовательных дней с коммитами у автора +### Наибольшее количество последовательных дней с коммитами у автора {#most-consecutive-days-of-commits-by-an-author} Сначала в этом запросе необходимо определить дни, в которые автор делал коммиты. Используя оконную функцию с разбиением по автору, мы можем вычислить количество дней между его коммитами. Для каждого коммита, если с момента предыдущего прошёл 1 день, мы помечаем его как последовательный (1), в противном случае — 0, сохраняя результат в поле `consecutive_day`. @@ -2374,7 +2374,7 @@ LIMIT 10 ``` -### Построчная история коммитов файла +### Построчная история коммитов файла {#line-by-line-commit-history-of-a-file} Файлы могут быть переименованы. Когда это происходит, мы получаем событие переименования, в котором столбец `path` содержит новый путь к файлу, а `old_path` — его прежнее расположение, например: @@ -2455,7 +2455,7 @@ FORMAT PrettyCompactMonoBlock ## Открытые вопросы {#unsolved-questions} -### Git blame +### Git blame {#git-blame} Получить точный результат особенно сложно из‑за того, что в настоящий момент в функциях для работы с массивами нельзя хранить состояние. Это станет возможным с помощью `arrayFold` или `arrayReduce`, которые позволяют сохранять состояние на каждой итерации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md index ec10b335995..d60d555c526 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['пример набора данных', 'hacker news', 'пример данных', 'анализ текста', 'векторный поиск'] --- -# Набор данных Hacker News +# Набор данных Hacker News {#hacker-news-dataset} > В этом руководстве вы загрузите в таблицу ClickHouse 28 миллионов строк данных Hacker News из форматов CSV и Parquet и выполните несколько простых запросов, чтобы изучить данные. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md index 187c9d3b129..954067b6288 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md @@ -11,7 +11,7 @@ keywords: ['пример набора данных', 'laion', 'векторны Набор данных содержит URL-адрес изображения, векторные представления (эмбеддинги) как для изображения, так и для подписи, оценку сходства между изображением и подписью, а также метаданные, например ширину и высоту изображения, лицензию и флаг NSFW. Мы можем использовать этот набор данных для демонстрации [поиска приблизительных ближайших соседей](../../engines/table-engines/mergetree-family/annindexes.md) в ClickHouse. -## Подготовка данных +## Подготовка данных {#data-preparation} Эмбеддинги и метаданные хранятся в отдельных файлах в исходных данных. На этапе подготовки данные загружаются, файлы объединяются, преобразуются в CSV и импортируются в ClickHouse. Для этого вы можете использовать следующий скрипт `download.sh`: @@ -40,28 +40,28 @@ npy_file = "img_emb_" + str_i + '.npy' metadata_file = "metadata_" + str_i + '.parquet' text_npy = "text_emb_" + str_i + '.npy' -# загрузка всех файлов +# загрузка всех файлов {#load-all-files} im_emb = np.load(npy_file) text_emb = np.load(text_npy) data = pd.read_parquet(metadata_file) -# объединение файлов +# объединение файлов {#combine-files} data = pd.concat([data, pd.DataFrame({"image_embedding" : [*im_emb]}), pd.DataFrame({"text_embedding" : [*text_emb]})], axis=1, copy=False) -# столбцы для импорта в ClickHouse +# столбцы для импорта в ClickHouse {#columns-to-be-imported-into-clickhouse} data = data[['url', 'caption', 'NSFW', 'similarity', "image_embedding", "text_embedding"]] -# преобразование np.arrays в списки +# преобразование np.arrays в списки {#transform-nparrays-to-lists} data['image_embedding'] = data['image_embedding'].apply(lambda x: x.tolist()) data['text_embedding'] = data['text_embedding'].apply(lambda x: x.tolist()) -# этот небольшой хак необходим, поскольку caption иногда содержит различные типы кавычек +# этот небольшой хак необходим, поскольку caption иногда содержит различные типы кавычек {#this-small-hack-is-needed-because-caption-sometimes-contains-all-kind-of-quotes} data['caption'] = data['caption'].apply(lambda x: x.replace("'", " ").replace('"', " ")) -# экспорт данных в CSV-файл +# экспорт данных в CSV-файл {#export-data-as-csv-file} data.to_csv(str_i + '.csv', header=False) -# удаление исходных файлов данных +# удаление исходных файлов данных {#removed-raw-data-files} os.system(f"rm {npy_file} {metadata_file} {text_npy}") ``` @@ -76,7 +76,7 @@ seq 0 409 | xargs -P1 -I{} bash -c './download.sh {}' (Приведённый выше скрипт на Python очень медленный (~2–10 минут на файл), потребляет много памяти (41 ГБ на файл), а получающиеся CSV‑файлы большие (10 ГБ каждый), поэтому будьте осторожны. Если у вас достаточно RAM, увеличьте значение `-P1` для большего параллелизма. Если этого всё ещё недостаточно и обработка остаётся слишком медленной, рассмотрите более эффективную процедуру ингестии — например, сначала конвертировать файлы .npy в parquet, а затем выполнять всю остальную обработку с помощью ClickHouse.) -## Создание таблицы +## Создание таблицы {#create-table} Чтобы сначала создать таблицу без индексов, выполните: @@ -105,7 +105,7 @@ INSERT INTO laion FROM INFILE '{path_to_csv_files}/*.csv' Обратите внимание, что столбец `id` служит лишь для иллюстрации и заполняется скриптом неуникальными значениями. -## Запуск векторного поиска методом перебора +## Запуск векторного поиска методом перебора {#run-a-brute-force-vector-similarity-search} Чтобы выполнить векторный поиск методом перебора, выполните: @@ -137,7 +137,7 @@ SELECT url, caption FROM laion ORDER BY cosineDistance(image_embedding, {target: ``` -## Выполните приближённый поиск по сходству векторов с использованием индекса сходства векторов +## Выполните приближённый поиск по сходству векторов с использованием индекса сходства векторов {#run-an-approximate-vector-similarity-search-with-a-vector-similarity-index} Теперь определим два индекса сходства векторов для таблицы. @@ -194,7 +194,7 @@ SELECT url, caption FROM laion ORDER BY cosineDistance(image_embedding, {target: Обычно требуется создавать эмбеддинги для новых изображений или новых подписей к изображениям и искать в данных похожие пары «изображение — подпись к изображению». Мы можем использовать [UDF](/sql-reference/functions/udf), чтобы создать вектор `target`, не выходя за пределы клиентского приложения. Важно использовать одну и ту же модель и для исходных данных, и для новых эмбеддингов при выполнении поисковых запросов. В следующих скриптах используется модель `ViT-B/32`, которая также лежит в основе набора данных. -### Текстовые эмбеддинги +### Текстовые эмбеддинги {#text-embeddings} Сначала поместите следующий скрипт Python в каталог `user_scripts/` в каталоге данных ClickHouse и сделайте его исполняемым (`chmod +x encode_text.py`). @@ -256,7 +256,7 @@ LIMIT 10 Обратите внимание, что сам UDF `encode_text()` может занимать несколько секунд на вычисление и выдачу векторного представления (эмбеддинга). -### Векторные представления изображений +### Векторные представления изображений {#image-embeddings} Векторные представления изображений можно создать аналогичным образом; для этого мы предоставляем сценарий на Python, который генерирует векторное представление изображения, сохранённого локально в файле. @@ -304,7 +304,7 @@ if __name__ == '__main__': Загрузите пример изображения для поиска: ```shell -# получить случайное изображение набора LEGO +# получить случайное изображение набора LEGO {#get-a-random-image-of-a-lego-set} $ wget http://cdn.firstcry.com/brainbees/images/products/thumb/191325a.jpg ``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md index 4ce6d538c66..2a720e37590 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md @@ -16,22 +16,22 @@ keywords: ['пример набора данных', 'меню', 'историч Данные взяты из архива библиотеки, поэтому они могут быть неполными и неудобными для статистического анализа. Тем не менее, они ещё и очень «аппетитные». Объём составляет всего 1,3 миллиона записей о блюдах в меню — это очень небольшой объём данных для ClickHouse, но всё же это хороший пример. -## Загрузите набор данных +## Загрузите набор данных {#download-dataset} Выполните команду: ```bash wget https://s3.amazonaws.com/menusdata.nypl.org/gzips/2021_08_01_07_01_17_data.tgz -# Опционально: Проверка контрольной суммы +# Опционально: Проверка контрольной суммы {#option-validate-the-checksum} md5sum 2021_08_01_07_01_17_data.tgz -# Контрольная сумма должна быть равна: db6126724de939a5481e3160a2d67d15 +# Контрольная сумма должна быть равна: db6126724de939a5481e3160a2d67d15 {#checksum-should-be-equal-to-db6126724de939a5481e3160a2d67d15} ``` При необходимости замените ссылку на актуальную с [http://menus.nypl.org/data](http://menus.nypl.org/data). Размер загрузки составляет около 35 МБ. -## Распакуйте датасет +## Распакуйте датасет {#unpack-dataset} ```bash tar xvf 2021_08_01_07_01_17_data.tgz @@ -47,7 +47,7 @@ tar xvf 2021_08_01_07_01_17_data.tgz * `MenuItem` — Позиция меню. Блюдо вместе с его ценой на определённой странице меню: ссылки на блюдо и страницу меню. -## Создайте таблицы +## Создайте таблицы {#create-tables} Для хранения цен мы используем тип данных [Decimal](../../sql-reference/data-types/decimal.md). @@ -115,7 +115,7 @@ CREATE TABLE menu_item ``` -## Импортируйте данные +## Импортируйте данные {#import-data} Чтобы загрузить данные в ClickHouse, выполните следующую команду: @@ -135,7 +135,7 @@ clickhouse-client --format_csv_allow_single_quotes 0 --input_format_null_as_defa Настройка [date_time_input_format best_effort](/operations/settings/formats#date_time_input_format) позволяет разбирать поля [DateTime](../../sql-reference/data-types/datetime.md) в широком диапазоне форматов. Например, будет распознан формат ISO-8601 без секунд, такой как '2000-01-01 01:02'. Без этой настройки разрешён только фиксированный формат DateTime. -## Денормализация данных +## Денормализация данных {#denormalize-data} Данные представлены в нескольких таблицах в [нормализованной форме](https://en.wikipedia.org/wiki/Database_normalization#Normal_forms). Это означает, что вам нужно выполнять [JOIN](/sql-reference/statements/select/join), если вы хотите, например, получить названия блюд из пунктов меню. Для типичных аналитических задач гораздо эффективнее работать с данными, заранее объединёнными с помощью `JOIN`, чтобы не выполнять `JOIN` каждый раз. Такие данные называются «денормализованными». @@ -188,7 +188,7 @@ FROM menu_item ``` -## Проверьте данные +## Проверьте данные {#validate-data} Запрос: @@ -207,7 +207,7 @@ SELECT count() FROM menu_item_denorm; ## Выполните несколько запросов {#run-queries} -### Средние исторические цены на блюда +### Средние исторические цены на блюда {#query-averaged-historical-prices} Запрос: @@ -250,7 +250,7 @@ ORDER BY d ASC; Не воспринимайте это слишком всерьёз. -### Цены на бургеры +### Цены на бургеры {#query-burger-prices} Запрос: @@ -288,7 +288,7 @@ ORDER BY d ASC; ``` -### Водка +### Водка {#query-vodka} Запрос: @@ -323,7 +323,7 @@ ORDER BY d ASC; Чтобы получить «водку», нам нужно написать `ILIKE '%vodka%'`, и это, мягко говоря, звучит многозначительно. -### Икра +### Икра {#query-caviar} Выведем цены на икру, а также название любого блюда с икрой. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md index 51a12af366d..816e4a6af05 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md @@ -28,7 +28,7 @@ keywords: ['пример набора данных', 'noaa', 'погодные - [Предварительно подготовленная версия](#pre-prepared-data) данных для ClickHouse, очищенная, переработанная и обогащённая. Эти данные охватывают период с 1900 по 2022 годы. - [Скачать исходные данные](#original-data) и преобразовать их в формат, требуемый ClickHouse. Пользователи, желающие добавить собственные столбцы, могут использовать этот подход. -### Предварительно подготовленные данные +### Предварительно подготовленные данные {#pre-prepared-data} Более конкретно, были удалены строки, не провалившие ни одной проверки качества со стороны NOAA. Данные также были реструктурированы из формата «одно измерение на строку» в формат «одна строка на идентификатор станции и дату», т. е. @@ -55,7 +55,7 @@ wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enriche Ниже приведены шаги по скачиванию и преобразованию исходных данных перед их загрузкой в ClickHouse. -#### Загрузка +#### Загрузка {#download} Чтобы скачать исходные данные: @@ -64,7 +64,7 @@ for i in {1900..2023}; do wget https://noaa-ghcn-pds.s3.amazonaws.com/csv.gz/${i ``` -#### Сэмплирование данных +#### Сэмплирование данных {#sampling-the-data} ```bash $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format PrettyCompact @@ -108,7 +108,7 @@ $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format Prett Формат «одно измерение на строку» приведёт к разрежённой структуре таблицы в ClickHouse. Мы должны преобразовать данные к формату «одна строка на момент времени и станцию», с измерениями в виде столбцов. Сначала мы ограничим набор данных строками без проблем, т.е. где `qFlag` равен пустой строке. -#### Очистите данные +#### Очистите данные {#clean-the-data} Используя [ClickHouse local](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local), мы можем отфильтровать строки, которые представляют интересующие нас измерения и соответствуют нашим требованиям к качеству: @@ -122,7 +122,7 @@ FROM file('*.csv.gz', CSV, 'station_id String, date String, measurement String, При объёме более 2,6 миллиарда строк этот запрос получается не очень быстрым, так как нужно разобрать все файлы. На нашей машине с 8 ядрами это занимает около 160 секунд. -### Преобразование данных (pivot) +### Преобразование данных (pivot) {#pivot-data} Хотя структуру «одно измерение на строку» можно использовать с ClickHouse, она будет излишне усложнять будущие запросы. В идеале нам нужна одна строка по идентификатору станции и дате, где каждый тип измерения и соответствующее ему значение представлены отдельным столбцом, т. е. @@ -161,7 +161,7 @@ done Этот запрос создаёт один файл `noaa.csv` размером 50 ГБ. -### Обогащение данных +### Обогащение данных {#enriching-the-data} В данных нет информации о местоположении, кроме идентификатора станции, который включает префикс с кодом страны. В идеале у каждой станции должны быть указаны широта и долгота. Для этого NOAA предоставляет сведения о каждой станции в отдельном файле [ghcnd-stations.txt](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file). Этот файл содержит [несколько столбцов](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file), из которых пять пригодятся для нашего дальнейшего анализа: id, latitude, longitude, elevation и name. @@ -194,7 +194,7 @@ FROM file('noaa.csv', CSV, Этот запрос выполняется в течение нескольких минут и создаёт файл размером 6,4 ГБ, `noaa_enriched.parquet`. -## Создание таблицы +## Создание таблицы {#create-table} Создайте таблицу MergeTree в ClickHouse (в клиенте ClickHouse). @@ -223,7 +223,7 @@ CREATE TABLE noaa ## Добавление данных в ClickHouse {#inserting-into-clickhouse} -### Вставка из локального файла +### Вставка из локального файла {#inserting-from-local-file} Данные можно вставить из локального файла следующим образом (в клиенте ClickHouse): @@ -236,7 +236,7 @@ INSERT INTO noaa FROM INFILE '<путь>/noaa_enriched.parquet' См. [здесь](https://clickhouse.com/blog/real-world-data-noaa-climate-data#load-the-data), чтобы узнать, как ускорить загрузку данных. -### Вставка из S3 +### Вставка из S3 {#inserting-from-s3} ```sql INSERT INTO noaa SELECT * @@ -249,7 +249,7 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enr ## Примеры запросов {#sample-queries} -### Максимальная температура за всё время +### Максимальная температура за всё время {#highest-temperature-ever} ```sql SELECT @@ -278,7 +278,7 @@ LIMIT 5 Обнадеживающе соответствует [задокументированному рекорду](https://en.wikipedia.org/wiki/List_of_weather_records#Highest_temperatures_ever_recorded) в [Furnace Creek](https://www.google.com/maps/place/36%C2%B027'00.0%22N+116%C2%B052'00.1%22W/@36.1329666,-116.1104099,8.95z/data=!4m5!3m4!1s0x0:0xf2ed901b860f4446!8m2!3d36.45!4d-116.8667) по состоянию на 2023 год. -### Лучшие горнолыжные курорты +### Лучшие горнолыжные курорты {#best-ski-resorts} Используя [список горнолыжных курортов](https://gist.githubusercontent.com/gingerwizard/dd022f754fd128fdaf270e58fa052e35/raw/622e03c37460f17ef72907afe554cb1c07f91f23/ski_resort_stats.csv) в США и их координаты, мы объединяем его с топ-1000 метеостанций с наибольшими снегопадами в любой месяц за последние 5 лет. Отсортировав результат этого объединения по [geoDistance](/sql-reference/functions/geo/coordinates/#geodistance) и ограничив выборку расстоянием менее 20 км, мы выбираем лучший результат для каждого курорта и затем сортируем полученный список по суммарному количеству снега. Обратите внимание, что мы также ограничиваем выборку курортами, расположенными выше 1800 м, как общим индикатором хороших условий для катания. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md index 7e7bcad570d..0dafaa6a050 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md @@ -24,7 +24,7 @@ import TabItem from '@theme/TabItem'; ::: -## Создайте таблицу trips +## Создайте таблицу trips {#create-the-table-trips} Сначала создайте таблицу для поездок на такси: @@ -126,7 +126,7 @@ FROM gcs( -## Примеры запросов +## Примеры запросов {#sample-queries} Следующие запросы выполняются для описанного выше примера. Пользователи могут запускать эти примерные запросы на полном наборе данных в [sql.clickhouse.com](https://sql.clickhouse.com/?query=U0VMRUNUIGNvdW50KCkgRlJPTSBueWNfdGF4aS50cmlwcw\&chart=eyJ0eXBlIjoibGluZSIsImNvbmZpZyI6eyJ0aXRsZSI6IlRlbXBlcmF0dXJlIGJ5IGNvdW50cnkgYW5kIHllYXIiLCJ4YXhpcyI6InllYXIiLCJ5YXhpcyI6ImNvdW50KCkiLCJzZXJpZXMiOiJDQVNUKHBhc3Nlbmdlcl9jb3VudCwgJ1N0cmluZycpIn19), изменив приведённые ниже запросы для использования таблицы `nyc_taxi.trips`. @@ -183,7 +183,7 @@ ORDER BY passenger_count ASC ``` -## Скачивание подготовленных партиций +## Скачивание подготовленных партиций {#download-of-prepared-partitions} :::note Следующие шаги содержат информацию об исходном наборе данных и метод загрузки подготовленных партиций в самостоятельно управляемую среду сервера ClickHouse. @@ -196,9 +196,9 @@ ORDER BY passenger_count ASC ```bash $ curl -O https://datasets.clickhouse.com/trips_mergetree/partitions/trips_mergetree.tar -# Проверьте контрольную сумму +# Проверьте контрольную сумму {#validate-the-checksum} $ md5sum trips_mergetree.tar -# Контрольная сумма должна быть: f3b8d469b41d9a82da064ded7245d12c +# Контрольная сумма должна быть: f3b8d469b41d9a82da064ded7245d12c {#checksum-should-be-equal-to-f3b8d469b41d9a82da064ded7245d12c} $ tar xvf trips_mergetree.tar -C /var/lib/clickhouse # путь к каталогу данных ClickHouse $ # проверьте права доступа к распакованным данным и исправьте при необходимости $ sudo service clickhouse-server restart @@ -210,7 +210,7 @@ $ clickhouse-client --query "select count(*) from datasets.trips_mergetree" ::: -## Результаты на одном сервере +## Результаты на одном сервере {#results-on-single-server} Q1: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md index a51d21eb170..89f81878b6e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md @@ -42,7 +42,7 @@ keywords: ['пример набора данных', 'nypd', 'данные о п Перед началом работы с базой данных ClickHouse ознакомьтесь с данными. -### Посмотрите на поля из исходного TSV-файла +### Посмотрите на поля из исходного TSV-файла {#look-at-the-fields-in-the-source-tsv-file} Это пример команды для выполнения запроса к TSV-файлу, но пока не запускайте её. @@ -120,7 +120,7 @@ New Georeferenced Column Nullable(String) На этом этапе вам следует проверить, что столбцы в TSV‑файле совпадают по названиям и типам с указанными в разделе **Columns in this Dataset** на [веб‑странице набора данных](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243). Типы данных заданы не очень строго: все числовые поля имеют тип `Nullable(Float64)`, а все остальные поля — `Nullable(String)`. При создании таблицы ClickHouse для хранения этих данных вы можете задать более подходящие и эффективные типы. -### Определите подходящую схему +### Определите подходящую схему {#determine-the-proper-schema} Чтобы определить, какие типы следует использовать для полей, необходимо понимать, как выглядят данные. Например, поле `JURISDICTION_CODE` представляет собой числовое значение: должно ли оно иметь тип `UInt8`, `Enum` или подойдет `Float64`? @@ -211,7 +211,7 @@ clickhouse-local --input_format_max_rows_to_read_for_schema_inference=2000 \ В используемом на момент написания наборе данных всего несколько сотен уникальных парков и игровых площадок в столбце `PARK_NM`. Это небольшое количество и оно укладывается в рекомендацию для [LowCardinality](/sql-reference/data-types/lowcardinality#description) — не превышать 10 000 различных строк в поле типа `LowCardinality(String)`. -### Поля DateTime +### Поля DateTime {#datetime-fields} Согласно разделу **Columns in this Dataset** на [веб‑странице набора данных](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243), в этом наборе данных есть поля даты и времени для начала и окончания зарегистрированного события. Анализ минимальных и максимальных значений полей `CMPLNT_FR_DT` и `CMPLT_TO_DT` позволяет понять, всегда ли эти поля заполняются: @@ -298,7 +298,7 @@ FORMAT PrettyCompact" Требуется внести ещё множество изменений в типы; все их можно определить, следуя тем же шагам анализа. Посмотрите на количество различных строковых значений в поле, минимальные и максимальные значения числовых полей и принимайте решения. Схема таблицы, которая приводится далее в руководстве, содержит много строковых полей с низкой кардинальностью и полей с беззнаковыми целыми числами и очень мало чисел с плавающей точкой. ::: -## Объединение полей даты и времени +## Объединение полей даты и времени {#concatenate-the-date-and-time-fields} Чтобы объединить поля даты и времени `CMPLNT_FR_DT` и `CMPLNT_FR_TM` в одну строку (`String`), которую затем можно привести к типу `DateTime`, выберите выражение с двумя полями, соединёнными оператором конкатенации: `CMPLNT_FR_DT || ' ' || CMPLNT_FR_TM`. Поля `CMPLNT_TO_DT` и `CMPLNT_TO_TM` обрабатываются аналогичным образом. @@ -329,7 +329,7 @@ FORMAT PrettyCompact" ``` -## Преобразование строкового значения даты и времени в тип DateTime64 +## Преобразование строкового значения даты и времени в тип DateTime64 {#convert-the-date-and-time-string-to-a-datetime64-type} Ранее в этом руководстве мы обнаружили, что в TSV-файле есть даты до 1 января 1970 года, что означает, что нам нужен 64-битный тип DateTime64 для хранения дат. Даты также нужно преобразовать из формата `MM/DD/YYYY` в формат `YYYY/MM/DD`. Оба этих действия можно выполнить с помощью функции [`parseDateTime64BestEffort()`](../../sql-reference/functions/type-conversion-functions.md#parsedatetime64besteffort). @@ -390,7 +390,7 @@ FORMAT PrettyCompact" Принятые выше решения относительно типов данных, используемых для столбцов, отражены в приведённой ниже схеме таблицы. Нам также необходимо определить `ORDER BY` и `PRIMARY KEY`, которые будут использоваться в таблице. Должен быть указан как минимум один из `ORDER BY` или `PRIMARY KEY`. Ниже приведены некоторые рекомендации по выбору столбцов для включения в `ORDER BY`; дополнительная информация приведена в разделе *Next Steps* в конце этого документа. -### Операторы `ORDER BY` и `PRIMARY KEY` +### Операторы `ORDER BY` и `PRIMARY KEY` {#order-by-and-primary-key-clauses} * Кортеж `ORDER BY` должен включать поля, которые используются в фильтрах запросов * Для максимального сжатия на диске кортеж `ORDER BY` должен быть упорядочен по возрастанию кардинальности полей @@ -486,7 +486,7 @@ CREATE TABLE NYPD_Complaint ( ``` -### Поиск первичного ключа таблицы +### Поиск первичного ключа таблицы {#finding-the-primary-key-of-a-table} Системная база данных ClickHouse `system`, в частности таблица `system.table`, содержит всю информацию о таблице, которую вы только что создали. Этот запрос показывает `ORDER BY` (ключ сортировки) и `PRIMARY KEY`: @@ -521,7 +521,7 @@ table: NYPD_Complaint Используем утилиту `clickhouse-local` для предварительной обработки данных и `clickhouse-client` для их загрузки. -### Используемые аргументы `clickhouse-local` +### Используемые аргументы `clickhouse-local` {#clickhouse-local-arguments-used} :::tip `table='input'` присутствует в аргументах clickhouse-local ниже. clickhouse-local принимает переданные входные данные (`cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv`) и вставляет их в таблицу. По умолчанию таблица называется `table`. В этом руководстве имя таблицы задано как `input`, чтобы сделать поток данных более наглядным. Последний аргумент clickhouse-local — это запрос, который выбирает данные из таблицы (`FROM input`), после чего результат перенаправляется в `clickhouse-client` для заполнения таблицы `NYPD_Complaint`. @@ -572,7 +572,7 @@ cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv \ ``` -## Проверка данных +## Проверка данных {#validate-data} :::note Набор данных обновляется один или несколько раз в год, поэтому ваши результаты подсчёта могут отличаться от приведённых в этом документе. @@ -616,7 +616,7 @@ WHERE name = 'NYPD_Complaint' ## Выполните несколько запросов {#run-queries} -### Запрос 1. Сравнение количества жалоб по месяцам +### Запрос 1. Сравнение количества жалоб по месяцам {#query-1-compare-the-number-of-complaints-by-month} Запрос: @@ -654,7 +654,7 @@ Query id: 7fbd4244-b32a-4acf-b1f3-c3aa198e74d9 ``` -### Запрос 2. Сравнение общего числа жалоб по районам +### Запрос 2. Сравнение общего числа жалоб по районам {#query-2-compare-total-number-of-complaints-by-borough} Запрос: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md index c09016063a7..13c6e0c4de2 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md @@ -127,7 +127,7 @@ CREATE TABLE `ontime` ORDER BY (Year, Quarter, Month, DayofMonth, FlightDate, IATA_CODE_Reporting_Airline); ``` -## Импорт сырых данных +## Импорт сырых данных {#import-from-raw-data} Загрузка данных: @@ -144,7 +144,7 @@ ls -1 *.zip | xargs -I{} -P $(nproc) bash -c "echo {}; unzip -cq {} '*.csv' | se (если на вашем сервере будет нехватка памяти или возникнут другие проблемы, удалите флаг `-P $(nproc)`) -## Импорт из сохранённой копии +## Импорт из сохранённой копии {#import-from-a-saved-copy} Также вы можете импортировать данные из сохранённой копии с помощью следующего запроса: @@ -155,7 +155,7 @@ INSERT INTO ontime SELECT * FROM s3('https://clickhouse-public-datasets.s3.amazo Снимок был создан 29.05.2022. -## Запросы +## Запросы {#queries} Q0. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md index 80578c5ccda..55aa0bcb402 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md @@ -22,7 +22,7 @@ import stackoverflow from '@site/static/images/getting-started/example-datasets/ Описание схемы этих данных можно найти [здесь](https://meta.stackexchange.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-sede). -## Заранее подготовленные данные +## Заранее подготовленные данные {#pre-prepared-data} Мы предоставляем копию этих данных в формате Parquet, актуальную по состоянию на апрель 2024 года. Хотя этот набор данных и невелик для ClickHouse с точки зрения количества строк (60 миллионов постов), он содержит значительные объемы текста и большие столбцы строкового типа (String). @@ -33,7 +33,7 @@ CREATE DATABASE stackoverflow Приведённые ниже замеры времени относятся к кластеру ClickHouse Cloud с 96 GiB памяти и 24 vCPU, расположенному в регионе `eu-west-2`. Набор данных находится в регионе `eu-west-3`. -### Публикации +### Публикации {#posts} ```sql CREATE TABLE stackoverflow.posts @@ -73,7 +73,7 @@ INSERT INTO stackoverflow.posts SELECT * FROM s3('https://datasets-documentation Публикации также доступны по годам, например, по адресу [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet) -### Голоса +### Голоса {#votes} ```sql CREATE TABLE stackoverflow.votes @@ -96,7 +96,7 @@ INSERT INTO stackoverflow.votes SELECT * FROM s3('https://datasets-documentation Голоса также доступны по годам, например, по адресу [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/votes/2020.parquet) -### Комментарии +### Комментарии {#comments} ```sql CREATE TABLE stackoverflow.comments @@ -120,7 +120,7 @@ INSERT INTO stackoverflow.comments SELECT * FROM s3('https://datasets-documentat Комментарии также доступны по годам, например, по адресу: [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/comments/2020.parquet) -### Пользователи +### Пользователи {#users} ```sql CREATE TABLE stackoverflow.users @@ -147,7 +147,7 @@ INSERT INTO stackoverflow.users SELECT * FROM s3('https://datasets-documentation ``` -### Значки +### Значки {#badges} ```sql CREATE TABLE stackoverflow.badges @@ -168,7 +168,7 @@ INSERT INTO stackoverflow.badges SELECT * FROM s3('https://datasets-documentatio ``` -### PostLinks +### PostLinks {#postlinks} ```sql CREATE TABLE stackoverflow.postlinks @@ -188,7 +188,7 @@ INSERT INTO stackoverflow.postlinks SELECT * FROM s3('https://datasets-documenta ``` -### PostHistory +### PostHistory {#posthistory} ```sql CREATE TABLE stackoverflow.posthistory @@ -217,7 +217,7 @@ INSERT INTO stackoverflow.posthistory SELECT * FROM s3('https://datasets-documen Исходный набор данных доступен в сжатом XML-формате (7zip) по адресу [https://archive.org/download/stackexchange](https://archive.org/download/stackexchange) — файлы с префиксом `stackoverflow.com*`. -### Загрузка +### Загрузка {#download} ```bash wget https://archive.org/download/stackexchange/stackoverflow.com-Badges.7z @@ -232,7 +232,7 @@ wget https://archive.org/download/stackexchange/stackoverflow.com-Votes.7z Размер этих файлов может достигать 35 ГБ, и их загрузка может занять около 30 минут в зависимости от интернет-соединения — сервер загрузки ограничивает скорость примерно до 20 МБ/с. -### Преобразование в JSON +### Преобразование в JSON {#convert-to-json} На момент написания ClickHouse не имеет встроенной поддержки XML в качестве входного формата. Чтобы загрузить данные в ClickHouse, мы сначала преобразуем их в NDJSON. @@ -260,7 +260,7 @@ p7zip -d stackoverflow.com-Posts.7z ```bash mkdir posts cd posts -# следующая команда разбивает входной xml-файл на подфайлы по 10000 строк +# следующая команда разбивает входной xml-файл на подфайлы по 10000 строк {#the-following-splits-the-input-xml-file-into-sub-files-of-10000-rows} tail +3 ../Posts.xml | head -n -1 | split -l 10000 --filter='{ printf "\n"; cat - ; printf "\n"; } > $FILE' - ``` @@ -283,7 +283,7 @@ clickhouse local --query "SELECT * FROM file('posts.json', JSONEachRow, 'Id Int3 Несколько простых запросов для начала работы. -### Самые популярные теги на Stack Overflow +### Самые популярные теги на Stack Overflow {#most-popular-tags-on-stack-overflow} ```sql @@ -313,7 +313,7 @@ LIMIT 10 ``` -### Пользователь с наибольшим количеством ответов (активные учётные записи) +### Пользователь с наибольшим количеством ответов (активные учётные записи) {#user-with-the-most-answers-active-accounts} Для учётной записи требуется `UserId`. @@ -340,7 +340,7 @@ LIMIT 5 ``` -### Самые популярные статьи о ClickHouse +### Самые популярные статьи о ClickHouse {#clickhouse-related-posts-with-the-most-views} ```sql SELECT @@ -371,7 +371,7 @@ LIMIT 10 ``` -### Самые спорные публикации +### Самые спорные публикации {#most-controversial-posts} ```sql SELECT diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md index 4d5645690b4..41684aad26c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md @@ -13,12 +13,12 @@ keywords: ['пример набора данных', 'tpch', 'бенчмарк', **Ссылки** -- [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) -- [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291) (Poess et. al., 2000) -- [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5) (Boncz et. al.), 2013 -- [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138) (Dresseler et. al.), 2020 +* [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) +* [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291) (Poess et. al., 2000) +* [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5) (Boncz et. al.), 2013 +* [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138) (Dresseler et. al.), 2020 -## Генерация и импорт данных +## Генерация и импорт данных {#data-generation-and-import} Сначала клонируйте репозиторий TPC-H и скомпилируйте генератор данных: @@ -58,7 +58,6 @@ make * В соответствии с разделом 1.4.2.1 определения таблиц не используют необязательные ограничения `NOT NULL`, даже если `dbgen` генерирует их по умолчанию. Производительность запросов `SELECT` в ClickHouse не зависит от наличия или отсутствия ограничений `NOT NULL`. * В соответствии с разделом 1.3.1 мы используем собственные типы данных ClickHouse (например, `Int32`, `String`) для реализации абстрактных типов данных, указанных в спецификации (например, `Identifier`, `Variable text, size N`). Единственный эффект этого — лучшая читаемость; типы данных SQL-92, генерируемые `dbgen` (например, `INTEGER`, `VARCHAR(40)`), также будут работать в ClickHouse. - ```sql CREATE TABLE nation ( n_nationkey Int32, @@ -169,7 +168,6 @@ clickhouse-client --format_csv_delimiter '|' --query "INSERT INTO lineitem FORMA Вместо использования tpch-kit и самостоятельной генерации таблиц вы можете импортировать данные из общедоступного бакета S3. Перед этим обязательно создайте пустые таблицы, используя приведённые выше операторы `CREATE`. - ```sql -- Коэффициент масштабирования 1 INSERT INTO nation SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws.com/h/1/nation.tbl', NOSIGN, CSV) SETTINGS format_csv_delimiter = '|', input_format_defaults_for_omitted_fields = 1, input_format_csv_empty_as_default = 1; @@ -194,8 +192,7 @@ INSERT INTO lineitem SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws. ::: - -## Запросы +## Запросы {#queries} :::note Для получения корректных результатов в соответствии со стандартом SQL необходимо включить настройку [`join_use_nulls`](../../operations/settings/settings.md#join_use_nulls). @@ -348,7 +345,6 @@ ORDER BY **Q5** - ```sql SELECT n_name, @@ -496,7 +492,6 @@ ORDER BY **Q9** - ```sql SELECT nation, @@ -850,7 +845,6 @@ WHERE **Q20** - ```sql SELECT s_name, diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md index 69e792d9e55..2ecf71eeafa 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md @@ -26,7 +26,7 @@ keywords: ['пример набора данных', 'погода', 'тайва - [Предварительно обработанная версия](#pre-processed-data) данных для ClickHouse, которые были очищены, переработаны и обогащены. Этот набор данных охватывает период с 1896 по 2023 год. - [Скачать исходные «сырые» данные](#original-raw-data) и преобразовать их в формат, требуемый ClickHouse. Пользователи, которые хотят добавить собственные столбцы, могут изучить или доработать свои подходы. -### Предобработанные данные +### Предобработанные данные {#pre-processed-data} Набор данных был преобразован из формата «одно измерение на строку» в формат «одна строка по идентификатору метеостанции и дате измерения», т.е. @@ -47,16 +47,16 @@ C0X100,2016-01-01 04:00:00,1021.2,15.8,74,1.7,8.0,,,,,,,,,,,,,,,,,,,,,,, ```bash wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/preprocessed_weather_daily_1896_2023.tar.gz -# Опционально: Проверьте контрольную сумму +# Опционально: Проверьте контрольную сумму {#option-validate-the-checksum} md5sum preprocessed_weather_daily_1896_2023.tar.gz -# Контрольная сумма должна совпадать с: 11b484f5bd9ddafec5cfb131eb2dd008 +# Контрольная сумма должна совпадать с: 11b484f5bd9ddafec5cfb131eb2dd008 {#checksum-should-be-equal-to-11b484f5bd9ddafec5cfb131eb2dd008} tar -xzvf preprocessed_weather_daily_1896_2023.tar.gz daily_weather_preprocessed_1896_2023.csv -# Опционально: Проверьте контрольную сумму +# Опционально: Проверьте контрольную сумму {#option-validate-the-checksum} md5sum daily_weather_preprocessed_1896_2023.csv -# Контрольная сумма должна совпадать с: 1132248c78195c43d93f843753881754 +# Контрольная сумма должна совпадать с: 1132248c78195c43d93f843753881754 {#checksum-should-be-equal-to-1132248c78195c43d93f843753881754} ``` @@ -64,7 +64,7 @@ md5sum daily_weather_preprocessed_1896_2023.csv Далее приведены сведения о шагах по загрузке исходных необработанных данных, которые затем можно преобразовать и конвертировать по своему усмотрению. -#### Загрузка +#### Загрузка {#download} Чтобы скачать исходные сырые данные: @@ -73,9 +73,9 @@ mkdir tw_raw_weather_data && cd tw_raw_weather_data wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/raw_data_weather_daily_1896_2023.tar.gz -# Опционально: Проверка контрольной суммы +# Опционально: Проверка контрольной суммы {#option-validate-the-checksum} md5sum raw_data_weather_daily_1896_2023.tar.gz -# Контрольная сумма должна совпадать с: b66b9f137217454d655e3004d7d1b51a +# Контрольная сумма должна совпадать с: b66b9f137217454d655e3004d7d1b51a {#checksum-should-be-equal-to-b66b9f137217454d655e3004d7d1b51a} tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1928.csv @@ -84,23 +84,23 @@ tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1931.csv ... -# Опционально: Проверка контрольной суммы +# Опционально: Проверка контрольной суммы {#option-validate-the-checksum} cat *.csv | md5sum -# Контрольная сумма должна совпадать с: b26db404bf84d4063fac42e576464ce1 +# Контрольная сумма должна совпадать с: b26db404bf84d4063fac42e576464ce1 {#checksum-should-be-equal-to-b26db404bf84d4063fac42e576464ce1} ``` -#### Получение данных метеостанций Тайваня +#### Получение данных метеостанций Тайваня {#retrieve-the-taiwan-weather-stations} ```bash wget -O weather_sta_list.csv https://github.com/Raingel/weather_station_list/raw/main/data/weather_sta_list.csv -# Опционально: Преобразование кодировки UTF-8-BOM в UTF-8 +# Опционально: Преобразование кодировки UTF-8-BOM в UTF-8 {#option-convert-the-utf-8-bom-to-utf-8-encoding} sed -i '1s/^\xEF\xBB\xBF//' weather_sta_list.csv ``` -## Создание схемы таблицы +## Создание схемы таблицы {#create-table-schema} Создайте таблицу MergeTree в ClickHouse (через клиент ClickHouse). @@ -144,7 +144,7 @@ ORDER BY (MeasuredDate); ## Вставка данных в ClickHouse {#inserting-into-clickhouse} -### Вставка данных из локального файла +### Вставка данных из локального файла {#inserting-from-local-file} Данные можно вставить из локального файла следующим образом (в клиенте ClickHouse): @@ -165,7 +165,7 @@ INSERT INTO tw_weather_data FROM INFILE '/path/to/daily_weather_preprocessed_189 ``` -### Вставка из URL +### Вставка из URL {#inserting-from-url} ```sql INSERT INTO tw_weather_data SELECT * @@ -176,7 +176,7 @@ FROM url('https://storage.googleapis.com/taiwan-weather-observaiton-datasets/dai Чтобы узнать, как ускорить этот процесс, ознакомьтесь с нашей публикацией в блоге о [настройке загрузки больших объёмов данных](https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part2). -## Проверка числа строк и объёма данных +## Проверка числа строк и объёма данных {#check-data-rows-and-sizes} 1. Посмотрим, сколько строк было вставлено: @@ -210,7 +210,7 @@ WHERE (`table` = 'tw_weather_data') AND active ## Примеры запросов {#sample-queries} -### Q1: Получите максимальное значение температуры точки росы для каждой метеостанции за заданный год +### Q1: Получите максимальное значение температуры точки росы для каждой метеостанции за заданный год {#q1-retrieve-the-highest-dew-point-temperature-for-each-weather-station-in-the-specific-year} ```sql SELECT @@ -257,7 +257,7 @@ GROUP BY StationId ``` -### Q2: Выборка сырых данных за заданный интервал времени, по полям и метеостанции +### Q2: Выборка сырых данных за заданный интервал времени, по полям и метеостанции {#q2-raw-data-fetching-with-the-specific-duration-time-range-fields-and-weather-station} ```sql SELECT diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md index b784a8a0bc4..25dfa2e506a 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md @@ -13,7 +13,7 @@ keywords: ['пример набора данных', 'недвижимость - Описание полей: https://www.gov.uk/guidance/about-the-price-paid-data - Содержит данные HM Land Registry © Crown copyright and database right 2021. Эти данные распространяются по лицензии Open Government Licence v3.0. -## Создание таблицы +## Создание таблицы {#create-table} ```sql CREATE DATABASE uk; @@ -40,7 +40,7 @@ ORDER BY (postcode1, postcode2, addr1, addr2); ``` -## Предобработка и вставка данных +## Предобработка и вставка данных {#preprocess-import-data} Мы будем использовать функцию `url` для потоковой передачи данных в ClickHouse. Сначала нам нужно предварительно обработать часть входящих данных, включая: @@ -95,7 +95,7 @@ FROM url( Дождитесь, пока данные будут вставлены — это может занять одну-две минуты в зависимости от скорости сети. -## Проверка данных +## Проверка данных {#validate-data} Проверим, что всё сработало, посмотрев, сколько строк было записано: @@ -119,7 +119,7 @@ WHERE name = 'uk_price_paid' Давайте выполним несколько запросов, чтобы проанализировать данные: -### Запрос 1. Средняя цена по годам +### Запрос 1. Средняя цена по годам {#average-price} ```sql runnable SELECT @@ -133,7 +133,7 @@ ORDER BY year ``` -### Запрос 2. Средняя цена по годам в Лондоне +### Запрос 2. Средняя цена по годам в Лондоне {#average-price-london} ```sql runnable SELECT @@ -150,7 +150,7 @@ ORDER BY year Что-то произошло с ценами на жильё в 2020 году! Однако, вероятно, это не стало сюрпризом... -### Запрос 3. Самые дорогие районы +### Запрос 3. Самые дорогие районы {#most-expensive-neighborhoods} ```sql runnable SELECT diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md index 7e958bd2ce4..0e75010ffb5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md @@ -241,7 +241,7 @@ keywords: ['пример набора данных', 'youtube', 'образцы ## Вопросы {#questions} -### Если отключить комментарии, уменьшится ли вероятность того, что кто-то поставит лайк или дизлайк? +### Если отключить комментарии, уменьшится ли вероятность того, что кто-то поставит лайк или дизлайк? {#create-the-table} Когда комментарии отключены, станут ли люди чаще ставить лайки или дизлайки, чтобы выразить своё отношение к видео? @@ -297,7 +297,7 @@ ORDER BY Включение комментариев, как правило, коррелирует с более высоким уровнем вовлечённости. -### Как со временем меняется количество видео — какие при этом можно выделить события? +### Как со временем меняется количество видео — какие при этом можно выделить события? {#insert-data} ```sql SELECT @@ -336,7 +336,7 @@ ORDER BY month ASC; Всплеск числа авторов, загружающих видео, [в период COVID-19 заметен](https://www.theverge.com/2020/3/27/21197642/youtube-with-me-style-videos-views-coronavirus-cook-workout-study-home-beauty). -### Больше субтитров со временем: когда это произошло +### Больше субтитров со временем: когда это произошло {#count-row-numbers} С развитием технологий распознавания речи создавать субтитры для видео стало проще, чем когда-либо: YouTube добавил автоматическое создание субтитров в конце 2009 года — стал ли это переломным моментом? @@ -374,7 +374,7 @@ ORDER BY month ASC; Это привело к запуску очень успешной кампании, призывавшей авторов добавлять субтитры к своим видео для слабослышащих и глухих зрителей. -### Лидеры по загрузкам во времени +### Лидеры по загрузкам во времени {#explore-the-data} ```sql WITH uploaders AS @@ -467,7 +467,7 @@ ORDER BY ``` -### Как распределяются представления? +### Как распределяются представления? {#if-someone-disables-comments-does-it-lower-the-chance-someone-will-actually-click-like-or-dislike} ```sql SELECT diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/index.md index fd852dc3ec7..e7da3264997 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/index.md @@ -9,9 +9,7 @@ title: 'Учебные руководства и примеры наборов doc_type: 'landing-page' --- - - -# Учебные материалы и примеры наборов данных +# Учебные материалы и примеры наборов данных {#tutorials-and-example-datasets} У нас есть множество ресурсов, которые помогут вам начать работу и понять, как работает ClickHouse: @@ -25,7 +23,6 @@ doc_type: 'landing-page' {/* Приведённая ниже таблица автоматически генерируется на этапе сборки скриптом https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh */ } - {/*AUTOGENERATED_START*/ } | Страница | Описание | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md index 5fcbfa1a77b..df327cf753f 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md @@ -1,99 +1,143 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; + + # Установка ClickHouse на Debian/Ubuntu {#install-from-deb-packages} -> Рекомендуется использовать официальные предварительно скомпилированные `deb` пакеты для **Debian** или **Ubuntu**. +> Рекомендуется использовать официальные предкомпилированные пакеты `deb` для **Debian** или **Ubuntu**. + ## Настройка репозитория Debian {#setup-the-debian-repository} -Для установки ClickHouse выполните следующие команды: +Чтобы установить ClickHouse, выполните следующие команды: + + + ```bash -# Установка необходимых пакетов +# Установите необходимые пакеты {#install-prerequisite-packages} sudo apt-get install -y apt-transport-https ca-certificates curl gnupg +``` + -# Загрузка GPG ключа ClickHouse и сохранение его в связке ключей +# Загрузите GPG‑ключ ClickHouse и сохраните его в ключевом хранилище {#download-the-clickhouse-gpg-key-and-store-it-in-the-keyring} curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | sudo gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg -# Получение архитектуры системы + + +# Определите архитектуру системы {#get-the-system-architecture} ARCH=$(dpkg --print-architecture) -# Добавление репозитория ClickHouse в источники apt + + +# Добавьте репозиторий ClickHouse в список источников пакетов apt {#add-the-clickhouse-repository-to-apt-sources} echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=${ARCH}] https://packages.clickhouse.com/deb stable main" | sudo tee /etc/apt/sources.list.d/clickhouse.list -# Обновление списков пакетов apt + + +# Обновить списки пакетов apt {#update-apt-package-lists} + sudo apt-get update + ``` -- Вы можете заменить `stable` на `lts` для использования различных [типов релизов](/knowledgebase/production) в зависимости от ваших потребностей. -- Вы можете загрузить и установить пакеты вручную с [packages.clickhouse.com](https://packages.clickhouse.com/deb/pool/main/c/). -
+- Вы можете заменить `stable` на `lts`, чтобы использовать различные [типы релизов](/knowledgebase/production) в зависимости от ваших потребностей. +- Вы можете скачать и установить пакеты вручную с [packages.clickhouse.com](https://packages.clickhouse.com/deb/pool/main/c/). +
-Старый метод для установки deb-пакетов для устаревших дистрибутивов +Устаревший метод установки deb-пакетов для дистрибутивов +``` + + ```bash -# Установка необходимых пакетов +# Установите необходимые пакеты {#install-prerequisite-packages} sudo apt-get install apt-transport-https ca-certificates dirmngr +``` -# Добавление GPG ключа ClickHouse для аутентификации пакетов + +# Добавьте GPG-ключ ClickHouse для аутентификации пакетов {#add-the-clickhouse-gpg-key-to-authenticate-packages} sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754 -# Добавление репозитория ClickHouse в источники apt + + +# Добавьте репозиторий ClickHouse в список источников APT {#add-the-clickhouse-repository-to-apt-sources} echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \ -/etc/apt/sources.list.d/clickhouse.list + /etc/apt/sources.list.d/clickhouse.list + + -# Обновление списков пакетов apt +# Обновите списки пакетов apt {#update-apt-package-lists} sudo apt-get update -# Установка пакетов сервера и клиента ClickHouse + + +# Установите пакеты сервера и клиента ClickHouse {#install-clickhouse-server-and-client-packages} sudo apt-get install -y clickhouse-server clickhouse-client -# Запуск службы сервера ClickHouse + + +# Запустите службу сервера ClickHouse {#start-the-clickhouse-server-service} sudo service clickhouse-server start -# Запуск клиента командной строки ClickHouse -clickhouse-client # или "clickhouse-client --password", если вы установили пароль. + + +# Запустите клиент командной строки ClickHouse {#launch-the-clickhouse-command-line-client} + +clickhouse-client # или "clickhouse-client --password", если вы указали пароль. + ```
+``` + ## Установка сервера и клиента ClickHouse {#install-clickhouse-server-and-client} + ```bash sudo apt-get install -y clickhouse-server clickhouse-client ``` + ## Запуск ClickHouse {#start-clickhouse-server} -Для запуска сервера ClickHouse выполните: +Чтобы запустить сервер ClickHouse, выполните следующую команду: + ```bash sudo service clickhouse-server start ``` -Для запуска клиента ClickHouse выполните: +Чтобы запустить клиент ClickHouse, выполните: + ```bash clickhouse-client ``` -Если вы установили пароль для вашего сервера, то вам нужно будет выполнить: +Если вы задали пароль для своего сервера, вам потребуется выполнить: + ```bash clickhouse-client --password ``` + ## Установка автономного ClickHouse Keeper {#install-standalone-clickhouse-keeper} :::tip -В производственных средах мы настоятельно рекомендуем запускать ClickHouse Keeper на выделенных узлах. +В производственных средах настоятельно рекомендуется запускать ClickHouse Keeper на выделенных узлах. В тестовых средах, если вы решите запускать ClickHouse Server и ClickHouse Keeper на одном сервере, -то вам не нужно устанавливать ClickHouse Keeper, так как он включен в ClickHouse server. +то вам не нужно отдельно устанавливать ClickHouse Keeper, так как он включён в ClickHouse Server. ::: -Для установки `clickhouse-keeper` на автономных серверах ClickHouse Keeper выполните: +Чтобы установить `clickhouse-keeper` на автономные серверы ClickHouse Keeper, выполните: + ```bash sudo apt-get install -y clickhouse-keeper ``` + ## Включение и запуск ClickHouse Keeper {#enable-and-start-clickhouse-keeper} + ```bash sudo systemctl enable clickhouse-keeper sudo systemctl start clickhouse-keeper @@ -102,20 +146,21 @@ sudo systemctl status clickhouse-keeper
+ ## Пакеты {#packages} -Ниже подробно описаны различные доступные deb пакеты: +Доступные deb-пакеты описаны ниже: -| Пакет | Описание | -|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `clickhouse-common-static` | Устанавливает скомпилированные бинарные файлы ClickHouse. | -| `clickhouse-server` | Создает символическую ссылку для `clickhouse-server` и устанавливает конфигурацию сервера по умолчанию. | -| `clickhouse-client` | Создает символическую ссылку для `clickhouse-client` и других инструментов, связанных с клиентом, и устанавливает файлы конфигурации клиента. | -| `clickhouse-common-static-dbg` | Устанавливает скомпилированные бинарные файлы ClickHouse с отладочной информацией. | -| `clickhouse-keeper` | Используется для установки ClickHouse Keeper на выделенных узлах ClickHouse Keeper. Если вы запускаете ClickHouse Keeper на том же сервере, что и сервер ClickHouse, то вам не нужно устанавливать этот пакет. Устанавливает ClickHouse Keeper и файлы конфигурации ClickHouse Keeper по умолчанию. | +| Package | Description | +|--------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `clickhouse-common-static` | Устанавливает скомпилированные бинарные файлы ClickHouse. | +| `clickhouse-server` | Создает символическую ссылку для `clickhouse-server` и устанавливает конфигурацию сервера по умолчанию. | +| `clickhouse-client` | Создает символическую ссылку для `clickhouse-client` и другие клиентские утилиты, а также устанавливает файлы конфигурации клиента. | +| `clickhouse-common-static-dbg` | Устанавливает скомпилированные бинарные файлы ClickHouse с отладочной информацией. | +| `clickhouse-keeper` | Используется для установки ClickHouse Keeper на выделенные узлы ClickHouse Keeper. Если вы запускаете ClickHouse Keeper на том же сервере, что и сервер ClickHouse, устанавливать этот пакет не нужно. Устанавливает ClickHouse Keeper и конфигурационные файлы ClickHouse Keeper по умолчанию. |
:::info -Если вам нужно установить конкретную версию ClickHouse, вы должны установить все пакеты одной и той же версии: +Если вам нужно установить определенную версию ClickHouse, необходимо установить все пакеты одной и той же версии: `sudo apt-get install clickhouse-server=21.8.5.7 clickhouse-client=21.8.5.7 clickhouse-common-static=21.8.5.7` -::: \ No newline at end of file +::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md index f49c5d136ce..aa6e0a09b84 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md @@ -1,4 +1,4 @@ -# Установка ClickHouse с помощью Docker +# Установка ClickHouse с помощью Docker {#install-clickhouse-using-docker} Руководство на [Docker Hub](https://hub.docker.com/r/clickhouse/clickhouse-server/) приведено ниже для удобства. Доступные Docker-образы используют @@ -33,9 +33,9 @@ docker pull clickhouse/clickhouse-server -## Как использовать этот образ +## Как использовать этот образ {#how-to-use-image} -### Запуск экземпляра сервера +### Запуск экземпляра сервера {#start-server-instance} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickhouse/clickhouse-server @@ -45,18 +45,18 @@ docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickh По умолчанию описанный выше экземпляр сервера запускается от имени пользователя `default` без пароля. -### Подключение к нему из нативного клиента +### Подключение к нему из нативного клиента {#connect-to-it-from-native-client} ```bash docker run -it --rm --network=container:some-clickhouse-server --entrypoint clickhouse-client clickhouse/clickhouse-server -# ИЛИ +# ИЛИ {#or} docker exec -it some-clickhouse-server clickhouse-client ``` Подробнее о клиенте ClickHouse см. в разделе [ClickHouse client](/interfaces/cli). -### Подключение с помощью curl +### Подключение с помощью curl {#connect-to-it-using-curl} ```bash echo "SELECT 'Hello, ClickHouse!'" | docker run -i --rm --network=container:some-clickhouse-server buildpack-deps:curl curl 'http://localhost:8123/?query=' -s --data-binary @- @@ -64,14 +64,14 @@ echo "SELECT 'Hello, ClickHouse!'" | docker run -i --rm --network=container:some Подробную информацию об HTTP-интерфейсе см. в разделе [ClickHouse HTTP Interface](/interfaces/http). -### Остановка и удаление контейнера +### Остановка и удаление контейнера {#stopping-removing-container} ```bash docker stop some-clickhouse-server docker rm some-clickhouse-server ``` -### Сеть +### Сеть {#networking} :::note Предопределённый пользователь `default` не имеет сетевого доступа, пока для него не задан пароль, @@ -98,7 +98,7 @@ echo 'SELECT version()' | curl 'http://localhost:8123/' --data-binary @- Пользователь по умолчанию в приведённом выше примере доступен только для запросов с localhost ::: -### Томa +### Томa {#volumes} Обычно для обеспечения сохранности данных имеет смысл смонтировать в контейнер следующие каталоги: @@ -119,7 +119,7 @@ docker run -d \ * `/docker-entrypoint-initdb.d/` - каталог со скриптами инициализации базы данных (см. ниже). -## Возможности Linux +## Возможности Linux {#linear-capabilities} У ClickHouse есть дополнительная функциональность, для работы которой требуется включить несколько [возможностей Linux (capabilities)](https://man7.org/linux/man-pages/man7/capabilities.7.html). @@ -134,29 +134,29 @@ docker run -d \ Дополнительные сведения см. в разделе ["Настройка возможностей CAP_IPC_LOCK и CAP_SYS_NICE в Docker"](/knowledgebase/configure_cap_ipc_lock_and_cap_sys_nice_in_docker) -## Конфигурация +## Конфигурация {#configuration} Контейнер открывает порт 8123 для [HTTP-интерфейса](https://clickhouse.com/docs/interfaces/http_interface/) и порт 9000 для [нативного клиента](https://clickhouse.com/docs/interfaces/tcp/). Конфигурация ClickHouse представлена файлом «config.xml» ([документация](https://clickhouse.com/docs/operations/configuration_files/)). -### Запуск экземпляра сервера с собственной конфигурацией +### Запуск экземпляра сервера с собственной конфигурацией {#start-server-instance-with-custom-config} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 -v /path/to/your/config.xml:/etc/clickhouse-server/config.xml clickhouse/clickhouse-server ``` -### Запуск сервера от имени отдельного пользователя +### Запуск сервера от имени отдельного пользователя {#start-server-custom-user} ```bash -# Директория $PWD/data/clickhouse должна существовать и принадлежать текущему пользователю +# Директория $PWD/data/clickhouse должна существовать и принадлежать текущему пользователю {#pwddataclickhouse-should-exist-and-be-owned-by-current-user} docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit nofile=262144:262144 -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` Когда вы используете образ с примонтированными локальными каталогами, вам, вероятно, нужно указать пользователя, чтобы сохранить корректное владение файлами. Используйте аргумент `--user` и смонтируйте `/var/lib/clickhouse` и `/var/log/clickhouse-server` внутрь контейнера. В противном случае образ будет выдавать ошибку и не запустится. -### Запуск сервера от root +### Запуск сервера от root {#start-server-from-root} Запуск сервера от root полезен в случаях, когда включено пространство имён пользователей. Чтобы сделать это, выполните: @@ -165,7 +165,7 @@ docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit no docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` -### Как создать базу данных и пользователя по умолчанию при запуске +### Как создать базу данных и пользователя по умолчанию при запуске {#how-to-create-default-db-and-user} Иногда может потребоваться при запуске контейнера создать пользователя (по умолчанию используется пользователь с именем `default`) и базу данных. Это можно сделать с помощью переменных окружения `CLICKHOUSE_DB`, `CLICKHOUSE_USER`, `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` и `CLICKHOUSE_PASSWORD`: @@ -173,7 +173,7 @@ docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v " docker run --rm -e CLICKHOUSE_DB=my_database -e CLICKHOUSE_USER=username -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 -e CLICKHOUSE_PASSWORD=password -p 9000:9000/tcp clickhouse/clickhouse-server ``` -#### Управление пользователем `default` +#### Управление пользователем `default` {#managing-default-user} Пользователь `default` по умолчанию не имеет сетевого доступа, если не заданы ни `CLICKHOUSE_USER`, ни `CLICKHOUSE_PASSWORD`, ни `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT`. @@ -184,7 +184,7 @@ docker run --rm -e CLICKHOUSE_SKIP_USER_SETUP=1 -p 9000:9000/tcp clickhouse/clic ``` -## Как расширить этот образ +## Как расширить этот образ {#how-to-extend-image} Чтобы выполнить дополнительную инициализацию в образе, производном от этого, добавьте один или несколько скриптов `*.sql`, `*.sql.gz` или `*.sh` в каталог `/docker-entrypoint-initdb.d`. После того как entrypoint-скрипт вызовет `initdb`, он выполнит все файлы `*.sql`, запустит все исполняемые скрипты `*.sh` и подключит (source) все неисполняемые скрипты `*.sh`, найденные в этом каталоге, для дальнейшей инициализации перед запуском сервиса.\ Также вы можете задать переменные окружения `CLICKHOUSE_USER` & `CLICKHOUSE_PASSWORD`, которые будут использоваться clickhouse-client во время инициализации. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md index 3a16793617a..a20cc32147d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md @@ -1,4 +1,4 @@ -# Установка ClickHouse с помощью tgz-архивов +# Установка ClickHouse с помощью tgz-архивов {#install-clickhouse-using-tgz-archives} > Рекомендуется использовать официальные предкомпилированные `tgz`-архивы для всех дистрибутивов Linux, где установка пакетов `deb` или `rpm` невозможна. @@ -20,7 +20,7 @@ -## Получите последнюю версию ClickHouse +## Получите последнюю версию ClickHouse {#get-latest-version} Получите последнюю версию ClickHouse с GitHub и сохраните её в переменной `LATEST_VERSION`. @@ -31,7 +31,7 @@ export LATEST_VERSION ``` -## Определите архитектуру системы +## Определите архитектуру системы {#detect-system-architecture} Определите архитектуру системы и задайте переменную ARCH соответствующим образом: @@ -44,7 +44,7 @@ esac ``` -## Загрузка tar-архивов для каждого компонента ClickHouse +## Загрузка tar-архивов для каждого компонента ClickHouse {#download-tarballs} Скачайте tar-архивы для каждого компонента ClickHouse. Цикл сначала пытается использовать пакеты, специфичные для архитектуры, затем при необходимости переходит к универсальным. @@ -65,7 +65,7 @@ done ```bash -# Извлечение и установка пакета clickhouse-common-static +# Извлечение и установка пакета clickhouse-common-static {#extract-and-install-clickhouse-common-static-package} tar -xzvf "clickhouse-common-static-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" @@ -75,7 +75,7 @@ sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" ```bash -# Извлеките и установите пакет отладочных символов +# Извлеките и установите пакет отладочных символов {#extract-and-install-debug-symbols-package} tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" @@ -85,7 +85,7 @@ sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" ```bash -# Извлечение и установка серверного пакета с конфигурацией +# Извлечение и установка серверного пакета с конфигурацией {#extract-and-install-server-package-with-configuration} tar -xzvf "clickhouse-server-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-server-$LATEST_VERSION.tgz" sudo "clickhouse-server-$LATEST_VERSION/install/doinst.sh" configure @@ -96,7 +96,7 @@ sudo /etc/init.d/clickhouse-server start # Запуск сервера ```bash -# Извлечь и установить клиентский пакет +# Извлечь и установить клиентский пакет {#extract-and-install-client-package} tar -xzvf "clickhouse-client-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-client-$LATEST_VERSION.tgz" sudo "clickhouse-client-$LATEST_VERSION/install/doinst.sh" diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md index 9f49a18deb2..610900f143c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md @@ -5,12 +5,12 @@ import privacy_allow from "@site/static/images/knowledgebase/fix-the-developer-v -# Установите ClickHouse с помощью Homebrew +# Установите ClickHouse с помощью Homebrew {#install-clickhouse-using-homebrew} -## Установка с помощью формулы Homebrew сообщества +## Установка с помощью формулы Homebrew сообщества {#install-using-community-homebrew-formula} Чтобы установить ClickHouse на macOS с помощью [Homebrew](https://brew.sh/), воспользуйтесь формулой Homebrew, поддерживаемой сообществом ClickHouse ([clickhouse](https://formulae.brew.sh/cask/clickhouse)). @@ -19,7 +19,7 @@ brew install --cask clickhouse ``` -## Исправление ошибки проверки разработчика в macOS +## Исправление ошибки проверки разработчика в macOS {#fix-developer-verification-error-macos} Если вы устанавливаете ClickHouse с помощью `brew`, вы можете столкнуться с ошибкой со стороны macOS. По умолчанию macOS не запускает приложения или инструменты, созданные разработчиком, подлинность которого не может быть подтверждена. @@ -30,7 +30,7 @@ brew install --cask clickhouse Чтобы обойти эту ошибку проверки, нужно убрать приложение из карантина macOS — либо найдя соответствующую настройку в окне **System Settings**, используя терминал, либо переустановив ClickHouse. -### Процесс через системные настройки +### Процесс через системные настройки {#system-settings-process} Самый простой способ убрать исполняемый файл `clickhouse` из карантина: @@ -50,7 +50,7 @@ brew install --cask clickhouse Теперь вы должны иметь возможность запускать команды `clickhouse` в терминале. -### Процесс через терминал +### Процесс через терминал {#terminal-process} Иногда нажатие кнопки `Allow Anyway` не решает эту проблему, и в этом случае вы можете выполнить этот процесс через командную строку. Или вы можете просто предпочитать использовать командную строку! diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md index a9b25d487e0..c7b0e931dab 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md @@ -1,11 +1,11 @@ -# Установка ClickHouse через скрипт с использованием curl +# Установка ClickHouse через скрипт с использованием curl {#install-clickhouse-via-script-using-curl} Если вам не нужно устанавливать ClickHouse для production-среды, самый быстрый способ настройки — запустить установочный скрипт с помощью curl. Скрипт автоматически определит подходящий бинарный файл для вашей ОС. -## Установка ClickHouse с помощью curl +## Установка ClickHouse с помощью curl {#install-clickhouse-using-curl} Выполните следующую команду, чтобы скачать один бинарный файл для вашей операционной системы. @@ -18,7 +18,7 @@ curl https://clickhouse.com/ | sh ::: -## Запустите clickhouse-local +## Запустите clickhouse-local {#start-clickhouse-local} `clickhouse-local` позволяет обрабатывать локальные и удалённые файлы, используя мощный SQL-синтаксис ClickHouse и без какой-либо предварительной настройки. Данные таблиц @@ -32,7 +32,7 @@ curl https://clickhouse.com/ | sh ``` -## Запуск clickhouse-server +## Запуск clickhouse-server {#start-clickhouse-server} Если вы хотите хранить данные, вам потребуется запустить `clickhouse-server`. Вы можете запустить сервер ClickHouse с помощью следующей команды: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md index de97b67d73a..32576a9da4e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md @@ -5,7 +5,7 @@ -## Настройка RPM-репозитория +## Настройка RPM-репозитория {#setup-the-rpm-repository} Добавьте официальный репозиторий, выполнив следующую команду: @@ -24,7 +24,7 @@ sudo zypper --gpg-auto-import-keys refresh clickhouse-stable В шагах ниже команду `yum install` можно заменить на `zypper install` в зависимости от используемого менеджера пакетов. -## Установка сервера и клиента ClickHouse +## Установка сервера и клиента ClickHouse {#install-clickhouse-server-and-client-1} Установите ClickHouse, выполнив следующие команды: @@ -42,7 +42,7 @@ sudo yum install clickhouse-server-22.8.7.34 ``` -## Запуск сервера ClickHouse +## Запуск сервера ClickHouse {#start-clickhouse-server-1} Чтобы запустить сервер ClickHouse, выполните команду: @@ -65,7 +65,7 @@ clickhouse-client --password ``` -## Установка отдельного ClickHouse Keeper +## Установка отдельного ClickHouse Keeper {#install-standalone-clickhouse-keeper-1} :::tip В производственных средах мы настоятельно рекомендуем запускать ClickHouse Keeper на выделенных узлах. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md index 5075f1bc89d..3db2fcee5ed 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md @@ -1,4 +1,4 @@ -# Установка ClickHouse на Windows с помощью WSL +# Установка ClickHouse на Windows с помощью WSL {#install-clickhouse-on-windows-with-wsl} @@ -11,7 +11,7 @@ -## Установка WSL +## Установка WSL {#install-wsl} Откройте Windows PowerShell от имени администратора и выполните следующую команду: @@ -26,7 +26,7 @@ wsl --install ``` -## Установите ClickHouse с помощью скрипта curl +## Установите ClickHouse с помощью скрипта curl {#install-clickhouse-via-script-using-curl} Выполните следующую команду, чтобы установить ClickHouse с помощью скрипта curl: @@ -42,7 +42,7 @@ curl https://clickhouse.com/ | sh ``` -## Запуск clickhouse-local +## Запуск clickhouse-local {#start-clickhouse-local} `clickhouse-local` позволяет обрабатывать локальные и удалённые файлы с использованием мощного SQL-синтаксиса ClickHouse и без необходимости настройки. Данные таблиц @@ -56,7 +56,7 @@ curl https://clickhouse.com/ | sh ``` -## Запуск clickhouse-server +## Запуск clickhouse-server {#start-clickhouse-server} Если вы хотите обеспечить сохранность данных, вам нужно запустить `clickhouse-server`. Вы можете запустить сервер ClickHouse с помощью следующей команды: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md index 990d3f39aa4..5142035a658 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md @@ -10,7 +10,7 @@ doc_type: 'guide' -## Сборка из исходных кодов +## Сборка из исходных кодов {#compile-from-source} Чтобы вручную собрать ClickHouse, следуйте инструкциям по сборке для [Linux](/development/build.md) или [macOS](/development/build-osx.md). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx index f587de616b5..4dd9b2770db 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx @@ -18,32 +18,12 @@ import Docker from './_snippets/_docker.md' import {CardPrimary} from '@clickhouse/click-ui/bundled'; import {galaxyOnClick} from '@site/src/lib/galaxy/galaxy' +# Инструкции по установке \{#installation-instructions\} -# Инструкции по установке + - - -
+
Или выберите ниже платформу, дистрибутив и способ установки, чтобы просмотреть инструкции по установке ClickHouse (open source): -} - quickinstall={} - debian_prod={} - rpm_prod={} - tar_prod={} - macos_prod={} - docker={} -/> \ No newline at end of file +} quickinstall={} debian_prod={} rpm_prod={} tar_prod={} macos_prod={} docker={} /> \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/playground.md b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/playground.md index fc626e5b653..0a45394e905 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/playground.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/playground.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# Песочница ClickHouse +# Песочница ClickHouse {#clickhouse-playground} [ClickHouse Playground](https://sql.clickhouse.com) позволяет экспериментировать с ClickHouse, мгновенно выполняя запросы без необходимости развертывать собственный сервер или кластер. В Playground доступно несколько примеров наборов данных. @@ -40,7 +40,7 @@ doc_type: 'guide' -## Примеры +## Примеры {#examples} Пример HTTPS-эндпоинта с использованием `curl`: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx index ecef4f95d81..dc52945164c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx @@ -20,17 +20,16 @@ import data_sources from '@site/static/images/_snippets/data_sources.png'; import select_data_source from '@site/static/images/_snippets/select_data_source.png'; import client_details from '@site/static/images/_snippets/client_details.png'; import new_rows_from_csv from '@site/static/images/_snippets/new_rows_from_csv.png'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; +import SQLConsoleDetail from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; - -# Быстрый старт с ClickHouse Cloud +# Быстрый старт с ClickHouse Cloud \{#clickhouse-cloud-quick-start\} > Самый быстрый и простой способ начать работу с ClickHouse — создать новый -сервис в [ClickHouse Cloud](https://console.clickhouse.cloud). В этом руководстве по быстрому старту вы сможете настроить систему -в три простых шага. +> сервис в [ClickHouse Cloud](https://console.clickhouse.cloud). В этом руководстве по быстрому старту вы сможете настроить систему +> в три простых шага. - ## Создайте сервис ClickHouse + ## Создайте сервис ClickHouse \{#1-create-a-clickhouse-service\} Чтобы создать бесплатный сервис ClickHouse в [ClickHouse Cloud](https://console.clickhouse.cloud), необходимо зарегистрироваться, выполнив следующие действия: @@ -59,7 +58,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; Поздравляем! Ваш сервис ClickHouse Cloud запущен и работает, процесс подключения завершён. Продолжайте чтение, чтобы узнать, как начать приём данных и выполнять запросы к ним. - ## Подключение к ClickHouse + ## Подключение к ClickHouse \{#2-connect-to-clickhouse\} Существует два способа подключения к ClickHouse: @@ -68,7 +67,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### Подключение через SQL-консоль + ### Подключение через SQL-консоль \{#connect-using-sql-console\} Для быстрого начала работы ClickHouse предоставляет веб-консоль SQL, на которую вы будете перенаправлены после завершения онбординга. @@ -88,7 +87,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; Вот и всё — вы готовы начать использовать свой новый сервис ClickHouse! - ### Подключение приложения + ### Подключение приложения \{#connect-with-your-app\} Нажмите кнопку подключения в меню навигации. Откроется модальное окно с учетными данными для вашего сервиса и инструкциями по подключению с использованием вашего интерфейса или клиентских библиотек. @@ -98,7 +97,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; Если вы не видите клиент для вашего языка программирования, ознакомьтесь со списком [интеграций](/integrations). - ## Добавление данных + ## Добавление данных \{#3-add-data\} ClickHouse эффективнее всего работает с данными! Существует несколько способов добавления данных, большинство из которых доступны на странице Data Sources в навигационном меню. @@ -114,7 +113,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; * Загрузите файл — поддерживаемые форматы: JSON, CSV и TSV * Загрузка данных из файла по URL - ### ClickPipes + ### ClickPipes \{#clickpipes\} [ClickPipes](http://clickhouse.com/docs/integrations/clickpipes) — управляемая платформа интеграции, которая упрощает приём данных из различных источников до нескольких нажатий кнопок. Разработанная для самых требовательных рабочих нагрузок, надёжная и масштабируемая архитектура ClickPipes обеспечивает стабильную производительность и отказоустойчивость. ClickPipes можно использовать как для долгосрочной потоковой передачи данных, так и для однократной загрузки. @@ -122,7 +121,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### Добавление данных через SQL Console + ### Добавление данных через SQL Console \{#add-data-using-the-sql-console\} Как и большинство систем управления базами данных, ClickHouse логически группирует таблицы в **базы данных**. Для создания новой базы данных в ClickHouse используйте команду [`CREATE DATABASE`](../../sql-reference/statements/create/database.md): @@ -163,7 +162,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; Существует множество движков таблиц на выбор, но для простой таблицы на одноузловом сервере ClickHouse оптимальным выбором будет [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md). ::: - #### Краткое введение в первичные ключи + #### Краткое введение в первичные ключи \{#a-brief-intro-to-primary-keys\} Прежде чем двигаться дальше, важно понять, как работают первичные ключи в ClickHouse (их реализация может оказаться неожиданной!): @@ -183,7 +182,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; Для углубленного изучения основных концепций ClickHouse см. ["Основные концепции"](../../managing-data/core-concepts/index.md). - #### Вставка данных в таблицу + #### Вставка данных в таблицу \{#insert-data-into-your-table\} Вы можете использовать знакомую команду [`INSERT INTO TABLE`](../../sql-reference/statements/insert-into.md) в ClickHouse, но важно понимать, что каждая вставка в таблицу [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) приводит к созданию **части** в хранилище. @@ -213,7 +212,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; SELECT * FROM helloworld.my_first_table ``` - ### Добавление данных с помощью клиента ClickHouse + ### Добавление данных с помощью клиента ClickHouse \{#add-data-using-the-clickhouse-client\} Вы также можете подключиться к вашему сервису ClickHouse Cloud с помощью инструмента командной строки [**clickhouse client**](/interfaces/cli). Нажмите `Connect` в левом меню для доступа к этим данным. В диалоговом окне выберите `Native` из выпадающего списка: @@ -293,7 +292,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; exit ``` - ### Загрузка файла + ### Загрузка файла \{#upload-a-file\} Распространённая задача при начале работы с базой данных — загрузить имеющиеся данные из файлов. Мы предоставляем образцы данных онлайн, которые можно использовать для демонстрации работы с данными о кликах (clickstream) — они включают идентификатор пользователя, посещённый URL и временную метку события. @@ -328,9 +327,9 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; ## Что дальше? \{#whats-next\} -- В [руководстве](/tutorial.md) вы загрузите 2 миллиона строк в таблицу и выполните несколько аналитических запросов -- У нас есть список [примеров наборов данных](/getting-started/index.md) с инструкциями по их загрузке -- Посмотрите наше 25-минутное видео [Начало работы с ClickHouse](https://clickhouse.com/company/events/getting-started-with-clickhouse/) -- Если ваши данные поступают из внешнего источника, ознакомьтесь с [коллекцией руководств по интеграции](/integrations/index.mdx) по подключению к очередям сообщений, базам данных, пайплайнам и многому другому -- Если вы используете инструмент визуализации (UI/BI), ознакомьтесь с [руководствами по подключению пользовательского интерфейса к ClickHouse](/integrations/data-visualization) -- Руководство по [первичным ключам](/guides/best-practices/sparse-primary-indexes.md) содержит всё, что вам нужно знать о первичных ключах и о том, как их определять \ No newline at end of file +* В [руководстве](/tutorial.md) вы загрузите 2 миллиона строк в таблицу и выполните несколько аналитических запросов +* У нас есть список [примеров наборов данных](/getting-started/index.md) с инструкциями по их загрузке +* Посмотрите наше 25-минутное видео [Начало работы с ClickHouse](https://clickhouse.com/company/events/getting-started-with-clickhouse/) +* Если ваши данные поступают из внешнего источника, ознакомьтесь с [коллекцией руководств по интеграции](/integrations/index.mdx) по подключению к очередям сообщений, базам данных, пайплайнам и многому другому +* Если вы используете инструмент визуализации (UI/BI), ознакомьтесь с [руководствами по подключению пользовательского интерфейса к ClickHouse](/integrations/data-visualization) +* Руководство по [первичным ключам](/guides/best-practices/sparse-primary-indexes.md) содержит всё, что вам нужно знать о первичных ключах и о том, как их определять \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx index 8d1929f269d..d8e3c1e1e86 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx +++ b/i18n/ru/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx @@ -14,16 +14,15 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import {VerticalStepper} from '@clickhouse/click-ui/bundled'; - -# Быстрый старт с ClickHouse OSS +# Быстрый старт с ClickHouse OSS \{#clickhouse-oss-quick-start\} > В этом кратком руководстве по быстрому старту вы за 8 -простых шагов настроите ClickHouse OSS. Вы скачаете подходящий бинарный файл для своей ОС, -узнаете, как запускать сервер ClickHouse, и воспользуетесь клиентом ClickHouse, чтобы создать таблицу, -затем вставите в неё данные и выполните запрос для выборки этих данных. +> простых шагов настроите ClickHouse OSS. Вы скачаете подходящий бинарный файл для своей ОС, +> узнаете, как запускать сервер ClickHouse, и воспользуетесь клиентом ClickHouse, чтобы создать таблицу, +> затем вставите в неё данные и выполните запрос для выборки этих данных. - ## Загрузка ClickHouse + ## Загрузка ClickHouse \{#download-the-binary\} ClickHouse работает нативно на Linux, FreeBSD и macOS, а на Windows — через [WSL](https://learn.microsoft.com/en-us/windows/wsl/about). Самый простой способ загрузить ClickHouse локально — выполнить @@ -57,7 +56,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; Для пользователей Mac: Если вы получаете ошибки о том, что разработчик бинарного файла не может быть проверен, см. ["Исправление ошибки проверки разработчика в MacOS"](https://clickhouse.com/docs/knowledgebase/fix-developer-verification-error-in-macos). ::: - ## Запуск сервера + ## Запуск сервера \{#start-the-server\} Выполните следующую команду для запуска сервера ClickHouse: @@ -69,7 +68,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; [уровень логирования по умолчанию](https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose) установлен в `trace`, а не в `warning`. - ## Запуск клиента + ## Запуск клиента \{#start-the-client\} Используйте `clickhouse-client` для подключения к вашему сервису ClickHouse. Откройте новый терминал, перейдите в каталог, где сохранён бинарный файл `clickhouse`, и @@ -85,7 +84,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; my-host :) ``` - ## Создание таблицы + ## Создание таблицы \{#create-a-table\} Используйте `CREATE TABLE` для создания новой таблицы. Стандартные SQL DDL-команды работают в ClickHouse с одним дополнением — таблицы в ClickHouse требуют @@ -104,7 +103,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; PRIMARY KEY (user_id, timestamp) ``` - ## Вставка данных + ## Вставка данных \{#insert-data\} Вы можете использовать знакомую команду `INSERT INTO TABLE` с ClickHouse, но важно понимать, что каждая вставка в таблицу `MergeTree` приводит к созданию в хранилище @@ -125,7 +124,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; (101, 'Гранулы — это наименьшие порции читаемых данных', now() + 5, 3.14159 ) ``` - ## Запросы к новой таблице + ## Запросы к новой таблице \{#query-your-new-table\} Запрос `SELECT` можно написать так же, как в любой другой SQL-базе данных: @@ -148,7 +147,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; 4 строки в наборе. Прошло: 0.008 сек. ``` - ## Вставка собственных данных + ## Вставка собственных данных \{#insert-your-own-data\} Следующий шаг — загрузить ваши данные в ClickHouse. Для приёма данных доступно множество [табличных функций](/sql-reference/table-functions/index.md) и [интеграций](/integrations). Примеры приведены на вкладках @@ -366,7 +365,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; - ## Исследование данных + ## Исследование данных \{#explore\} * Ознакомьтесь с разделом [Основные концепции](/managing-data/core-concepts), чтобы познакомиться с основами того, как ClickHouse работает «под капотом». * Ознакомьтесь с [углублённым руководством](tutorial.md), которое гораздо глубже раскрывает ключевые концепции и возможности ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/index.md index b013cbb0be6..53aca3cdf5e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/index.md @@ -7,13 +7,12 @@ keywords: ['оптимизация производительности', 'лу doc_type: 'reference' --- -import TableOfContents from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContents from '@site/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +# Производительность и оптимизация {#performance-and-optimizations} -# Производительность и оптимизация - -В этом разделе приведены рекомендации и лучшие практики по улучшению производительности при работе с ClickHouse. -Мы рекомендуем пользователям сначала ознакомиться с разделом [Основные концепции](/parts), +В этом разделе приведены рекомендации и лучшие практики по улучшению производительности при работе с ClickHouse. +Мы рекомендуем пользователям сначала ознакомиться с разделом [Основные концепции](/parts), в котором рассмотрены ключевые понятия, необходимые для оптимизации производительности. - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md index ba689533664..af8d05f6252 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/prewhere_05.gif' import Image from '@theme/IdealImage'; -# Как работает оптимизация PREWHERE? +# Как работает оптимизация PREWHERE? {#how-does-the-prewhere-optimization-work} [Предложение PREWHERE](/sql-reference/statements/select/prewhere) — это оптимизация выполнения запроса в ClickHouse. Она уменьшает объём операций ввода-вывода и ускоряет выполнение запроса, избегая ненужного чтения данных и отфильтровывая лишние данные до чтения со встроенного диска столбцов, не участвующих в фильтрации. @@ -109,7 +109,7 @@ ClickHouse начинает обработку PREWHERE, ① читая выбр -## Как измерить влияние PREWHERE +## Как измерить влияние PREWHERE {#how-to-measure-prewhere-impact} Чтобы убедиться, что PREWHERE действительно ускоряет ваши запросы, вы можете сравнить их производительность с включённой и выключенной настройкой `optimize_move_to_prewhere`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md index 332c2023cce..1b0d0f96257 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md @@ -11,7 +11,7 @@ import queryOptimizationDiagram1 from '@site/static/images/guides/best-practices import Image from '@theme/IdealImage'; -# Простое руководство по оптимизации запросов +# Простое руководство по оптимизации запросов {#a-simple-guide-for-query-optimization} В этом разделе на распространённых сценариях показано, как использовать различные методы повышения производительности и оптимизации, такие как [анализатор](/operations/analyzer), [профилирование запросов](/operations/optimizing-performance/sampling-query-profiler) и [отказ от использования Nullable-столбцов](/optimize/avoid-nullable-columns), чтобы улучшить производительность выполнения запросов в ClickHouse. @@ -61,7 +61,7 @@ ClickHouse предоставляет богатый набор инструме -## Набор данных +## Набор данных {#dataset} Мы используем реальный пример, чтобы проиллюстрировать наш подход к оптимизации производительности запросов.  @@ -112,9 +112,9 @@ ORDER BY tuple() ``` -## Найдите медленные запросы +## Найдите медленные запросы {#spot-the-slow-queries} -### Журнал запросов +### Журнал запросов {#query-logs} По умолчанию ClickHouse собирает и записывает информацию о каждом выполненном запросе в [журналы запросов](/operations/system-tables/query_log). Эти данные хранятся в таблице `system.query_log`.  @@ -431,7 +431,7 @@ _Наконец, будьте внимательны к выбросам: дов -## Базовая оптимизация +## Базовая оптимизация {#basic-optimization} Теперь, когда у нас есть фреймворк для тестирования, можно приступать к оптимизации. @@ -439,7 +439,7 @@ _Наконец, будьте внимательны к выбросам: дов В зависимости от того, как вы осуществляли приём данных, вы могли использовать [возможности](/interfaces/schema-inference) ClickHouse для вывода схемы таблицы на основе принятых данных. Хотя это очень удобно на начальном этапе, если вы хотите оптимизировать производительность запросов, вам потребуется пересмотреть схему данных, чтобы она наилучшим образом соответствовала вашему сценарию применения. -### Nullable +### Nullable {#nullable} Как описано в [документации по лучшим практикам](/best-practices/select-data-types#avoid-nullable-columns), по возможности избегайте столбцов с типом Nullable. Их часто хочется использовать, так как они делают механизм ингестии данных более гибким, но они негативно влияют на производительность, поскольку каждый раз приходится обрабатывать дополнительный столбец. @@ -485,7 +485,7 @@ dropoff_location_id_nulls: 0 У нас есть только два столбца со значениями NULL: `mta_tax` и `payment_type`. Остальные поля не должны использовать тип столбца `Nullable`. -### Низкая кардинальность +### Низкая кардинальность {#low-cardinality} Простая оптимизация для строковых типов — максимально эффективно использовать тип данных LowCardinality. Как описано в [документации по низкой кардинальности](/sql-reference/data-types/lowcardinality), ClickHouse применяет словарное кодирование к столбцам LowCardinality, что значительно повышает производительность запросов.  @@ -515,7 +515,7 @@ uniq(vendor_id): 3 Благодаря низкой кардинальности эти четыре столбца — `ratecode_id`, `pickup_location_id`, `dropoff_location_id` и `vendor_id` — являются хорошими кандидатами для типа данных LowCardinality. -### Оптимизируйте тип данных +### Оптимизируйте тип данных {#optimize-data-type} ClickHouse поддерживает большое количество типов данных. Для оптимизации производительности и уменьшения объёма занимаемого на диске пространства данных убедитесь, что вы выбираете наименьший возможный тип данных, подходящий для вашего сценария использования.  @@ -538,7 +538,7 @@ Query id: 4306a8e1-2a9c-4b06-97b4-4d902d2233eb Для дат следует выбирать такую точность, которая соответствует вашему набору данных и лучше всего подходит для выполнения запросов, которые вы планируете запускать. -### Применим оптимизации +### Применим оптимизации {#apply-the-optimizations} Давайте создадим новую таблицу, чтобы использовать оптимизированную схему и повторно выполнить приём данных. @@ -604,7 +604,7 @@ ORDER BY size DESC Новая таблица значительно меньше предыдущей. Мы наблюдаем сокращение объёма дискового пространства, занимаемого таблицей, примерно на 34% (7,38 GiB против 4,89 GiB). -## Важность первичных ключей +## Важность первичных ключей {#the-importance-of-primary-keys} Первичные ключи в ClickHouse работают иначе, чем в большинстве традиционных систем управления базами данных. В таких системах первичные ключи обеспечивают уникальность и целостность данных. Любая попытка вставки дублирующихся значений первичного ключа отклоняется, а для быстрого поиска обычно создаётся индекс на основе B-tree или хэша.  @@ -616,7 +616,7 @@ ORDER BY size DESC Другие возможности, поддерживаемые ClickHouse, такие как проекция (Projection) или материализованное представление, позволяют использовать другой набор первичных ключей для тех же данных. Во второй части этой серии статей в блоге это будет рассмотрено подробнее.  -### Выбор первичных ключей +### Выбор первичных ключей {#choose-primary-keys} Выбор корректного набора первичных ключей — сложная тема, и для нахождения наилучшей комбинации могут потребоваться компромиссы и эксперименты.  diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md index f007f07f47c..befb3ef4e91 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/query-parallelis import Image from '@theme/IdealImage'; -# Как ClickHouse выполняет запросы параллельно +# Как ClickHouse выполняет запросы параллельно {#how-clickhouse-executes-a-query-in-parallel} ClickHouse [создан для высокой скорости](/concepts/why-clickhouse-is-so-fast). Он выполняет запросы в высокопараллельном режиме, используя все доступные ядра CPU, распределяя данные по потокам обработки и часто подводя оборудование к пределам его возможностей. @@ -66,7 +66,7 @@ ClickHouse [создан для высокой скорости](/concepts/why-c -## Мониторинг параллелизма запросов +## Мониторинг параллелизма запросов {#monitoring-query-parallelism} Используйте эти инструменты, чтобы убедиться, что ваш запрос полностью задействует доступные ресурсы CPU и диагностировать случаи, когда это не так. @@ -126,12 +126,12 @@ FROM Примечание: читайте визуализацию слева направо. Каждая строка представляет параллельный конвейер обработки, который передаёт данные блок за блоком, применяя преобразования, такие как фильтрация, агрегация и финальные стадии обработки. В этом примере вы видите четыре параллельных конвейера, соответствующих настройке `max_threads = 4`. -### Балансировка нагрузки между конвейерами обработки +### Балансировка нагрузки между конвейерами обработки {#load-balancing-across-processing-lanes} Обратите внимание, что операторы `Resize` в физическом плане выше [переразбивают на части и перераспределяют](/academic_overview#4-2-multi-core-parallelization) потоки блоков данных между конвейерами обработки, чтобы поддерживать их равномерную загрузку. Такое перебалансирование особенно важно, когда диапазоны данных различаются по количеству строк, удовлетворяющих предикатам запроса, иначе некоторые конвейеры могут быть перегружены, в то время как другие простаивают. Перераспределяя работу, более быстрые конвейеры фактически помогают более медленным, оптимизируя общее время выполнения запроса. -## Почему max_threads не всегда соблюдается +## Почему max_threads не всегда соблюдается {#why-max-threads-isnt-always-respected} Как отмечалось выше, количество параллельных потоков обработки `n` контролируется параметром `max_threads`, который по умолчанию равен числу ядер CPU, доступных ClickHouse на сервере: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md index a51f9455a12..c8e59e8fbde 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md @@ -10,7 +10,7 @@ keywords: ['индексы пропуска данных', 'пропуск да -# Примеры индексов пропуска данных +# Примеры индексов пропуска данных {#data-skipping-index-examples} На этой странице собраны примеры индексов пропуска данных ClickHouse, показано, как объявить каждый тип, когда их использовать и как проверить, что они используются. Все возможности работают с [таблицами семейства MergeTree](/engines/table-engines/mergetree-family/mergetree). @@ -33,7 +33,7 @@ ClickHouse поддерживает пять типов индексов про В каждом разделе приводятся примеры с демонстрационными данными и показывается, как проверить использование индекса при выполнении запроса. -## Индекс MinMax +## Индекс MinMax {#minmax-index} Индекс `minmax` лучше всего подходит для диапазонных предикатов по слабо упорядоченным данным или по столбцам, коррелированным с `ORDER BY`. @@ -64,7 +64,7 @@ SELECT count() FROM events WHERE ts >= now() - 3600; См. [подробный пример](/best-practices/use-data-skipping-indices-where-appropriate#example) с `EXPLAIN` и отсечением данных. -## Индекс set +## Индекс set {#set-index} Используйте индекс `set`, когда локальная кардинальность на уровне блока низкая; он неэффективен, если в каждом блоке много различных значений. @@ -81,7 +81,7 @@ SELECT * FROM events WHERE user_id IN (101, 202); Процесс создания и материализации, а также эффект до/после показаны в [базовом руководстве по работе](/optimize/skipping-indexes#basic-operation). -## Универсальный Bloom-фильтр (скалярный) +## Универсальный Bloom-фильтр (скалярный) {#generic-bloom-filter-scalar} Индекс `bloom_filter` хорошо подходит для поиска «иголки в стоге сена» по условию равенства или проверки принадлежности (оператор IN). Он принимает необязательный параметр — вероятность ложноположительных срабатываний (по умолчанию 0.025). @@ -96,7 +96,7 @@ SELECT * FROM events WHERE value IN (7, 42, 99); ``` -## N-граммный фильтр Блума (ngrambf_v1) для поиска подстрок +## N-граммный фильтр Блума (ngrambf_v1) для поиска подстрок {#n-gram-bloom-filter-ngrambf-v1-for-substring-search} Индекс `ngrambf_v1` разбивает строки на n-граммы. Он хорошо подходит для запросов вида `LIKE '%...%'`. Поддерживаются типы String/FixedString/Map (через mapKeys/mapValues), а также настраиваемые размер, количество хэшей и значение seed. Дополнительные сведения см. в документации по [N-граммному фильтру Блума](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter). @@ -133,7 +133,7 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) AS k; -- ~13 См. [документацию по параметрам](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter) для получения подробных рекомендаций по настройке. -## Токен-блум-фильтр (tokenbf_v1) для поиска по словам +## Токен-блум-фильтр (tokenbf_v1) для поиска по словам {#token-bloom-filter-tokenbf-v1-for-word-based-search} `tokenbf_v1` индексирует токены, разделённые небуквенно-цифровыми символами. Используйте его с [`hasToken`](/sql-reference/functions/string-search-functions#hasToken), с шаблонами слов `LIKE` или с операторами равенства (`=`) и `IN`. Поддерживает типы `String`/`FixedString`/`Map`. @@ -153,7 +153,7 @@ SELECT count() FROM logs WHERE hasToken(lower(msg), 'exception'); См. примеры по наблюдаемости и рекомендации по выбору token vs ngram [здесь](/use-cases/observability/schema-design#bloom-filters-for-text-search). -## Добавление индексов при выполнении CREATE TABLE (несколько примеров) +## Добавление индексов при выполнении CREATE TABLE (несколько примеров) {#add-indexes-during-create-table-multiple-examples} Индексы пропуска данных также поддерживают составные выражения и типы `Map`/`Tuple`/`Nested`. Это показано в примере ниже: @@ -175,7 +175,7 @@ ORDER BY u64; ``` -## Материализация индекса на существующих данных и проверка +## Материализация индекса на существующих данных и проверка {#materializing-on-existing-data-and-verifying} Вы можете добавить индекс к уже существующим частям данных с помощью `MATERIALIZE` и проверить отсечение с помощью `EXPLAIN` или трассировочных журналов, как показано ниже: @@ -211,7 +211,7 @@ SET send_logs_level = 'trace'; -## Временно игнорировать или принудительно использовать индексы +## Временно игнорировать или принудительно использовать индексы {#temporarily-ignore-or-force-indexes} Отключайте отдельные индексы по имени для конкретных запросов во время тестирования и устранения неполадок. Также доступны настройки для принудительного использования индексов при необходимости. См. [`ignore_data_skipping_indices`](/operations/settings/settings#ignore_data_skipping_indices). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md index 45006697f1b..7e09f3a40c9 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md @@ -13,7 +13,7 @@ import bad_skip from '@site/static/images/guides/best-practices/bad_skip.png'; import Image from '@theme/IdealImage'; -# Что такое индексы пропуска данных в ClickHouse +# Что такое индексы пропуска данных в ClickHouse {#understanding-clickhouse-data-skipping-indexes} @@ -29,7 +29,7 @@ import Image from '@theme/IdealImage'; -## Базовый принцип работы +## Базовый принцип работы {#basic-operation} Пользователи могут использовать индексы пропуска данных (Data Skipping Indexes) только для таблиц семейства MergeTree. Каждый индекс пропуска данных имеет четыре основных аргумента: @@ -121,11 +121,11 @@ SET send_logs_level='trace'; default.skip_table (933d4b2c-8cea-4bf9-8c93-c56e900eefd1) (SelectExecutor): Индекс `vix` пропустил 6102/6104 гранул. ``` -## Типы индексов пропуска данных +## Типы индексов пропуска данных {#skip-index-types} {/* vale off */ } -### minmax +### minmax {#minmax} {/* vale on */ } @@ -137,7 +137,7 @@ SET send_logs_level='trace'; {/* vale off */ } -### SET +### SET {#set} {/* vale on */ } @@ -146,7 +146,7 @@ SET send_logs_level='trace'; Затраты, производительность и эффективность этого индекса зависят от кардинальности внутри блоков. Если каждый блок содержит большое количество уникальных значений, то либо вычисление условия запроса по большому набору индекса будет очень ресурсоёмким, либо индекс не будет применён, потому что он пуст из‑за превышения max_size. -### Типы фильтров Блума +### Типы фильтров Блума {#bloom-filter-types} *Фильтр Блума* — это структура данных, которая позволяет эффективно по памяти проверять принадлежность к набору ценой небольшой вероятности ложноположительных результатов. Ложноположительные срабатывания не являются существенной проблемой в случае пропускающих индексов, поскольку единственный недостаток — чтение нескольких лишних блоков. Однако возможность ложноположительных срабатываний означает, что индексированное выражение, как правило, должно быть истинным, иначе часть корректных данных может быть пропущена. @@ -187,7 +187,7 @@ SET send_logs_level='trace'; -## Рекомендации по использованию skip index +## Рекомендации по использованию skip index {#skip-best-practices} Skip-индексы неочевидны, особенно для пользователей, привыкших к вторичным строковым индексам из мира СУБД или инвертированным индексам из документных хранилищ. Чтобы получить от них пользу, применение индекса пропуска данных (data skipping index) в ClickHouse должно позволять избежать достаточного количества чтений гранул, чтобы компенсировать стоимость вычисления индекса. Критически важно, что если значение встречается хотя бы один раз в индексируемом блоке, это означает, что весь блок должен быть считан в память и обработан, а затраты на вычисление индекса окажутся напрасными. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md index d029a951e34..c8440883b2e 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md @@ -36,7 +36,7 @@ import sparsePrimaryIndexes15b from '@site/static/images/guides/best-practices/s import Image from '@theme/IdealImage'; -# Практическое введение в первичные индексы ClickHouse +# Практическое введение в первичные индексы ClickHouse {#a-practical-introduction-to-primary-indexes-in-clickhouse} ## Введение {#introduction} @@ -73,7 +73,7 @@ import Image from '@theme/IdealImage'; Все показатели времени выполнения, приведённые в этом документе, основаны на локальном запуске ClickHouse 22.2.1 на MacBook Pro с чипом Apple M1 Pro и 16 ГБ оперативной памяти. -### Полное сканирование таблицы +### Полное сканирование таблицы {#a-full-table-scan} Чтобы увидеть, как выполняется запрос по нашему набору данных без первичного ключа, создадим таблицу (с движком таблиц MergeTree), выполнив следующий SQL DDL‑оператор: @@ -150,7 +150,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 10 rows in set. Elapsed: 0.022 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 8,87 млн строк, 70,45 МБ (398,53 млн строк/с., 3,17 ГБ/с.) ``` @@ -172,7 +172,7 @@ LIMIT 10; Ниже подробно показано, как ClickHouse строит и использует свой разрежённый первичный индекс. Позже в статье мы обсудим лучшие практики выбора, удаления и упорядочивания столбцов таблицы, используемых для построения индекса (столбцов первичного ключа). -### Таблица с первичным ключом +### Таблица с первичным ключом {#a-table-with-a-primary-key} Создайте таблицу с составным первичным ключом по столбцам UserID и URL: @@ -496,7 +496,7 @@ SELECT UserID, URL FROM file('primary-hits_UserID_URL.idx', 'RowBinary', 'UserID Последствия этого для производительности выполнения запросов мы рассмотрим подробнее позже. -### Первичный индекс используется для отбора гранул +### Первичный индекс используется для отбора гранул {#the-primary-index-is-used-for-selecting-granules} Теперь мы можем выполнять запросы с использованием первичного индекса. @@ -528,7 +528,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 10 rows in set. Elapsed: 0.005 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 8,19 тыс. строк, 740,18 КБ (1,53 млн. строк/с., 138,59 МБ/с.) ``` @@ -539,13 +539,13 @@ LIMIT 10; ```response ...Executor): Key condition: (column 0 in [749927693, 749927693]) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Running binary search on index range for part all_1_9_2 (1083 marks) ...Executor): Found (LEFT) boundary mark: 176 ...Executor): Found (RIGHT) boundary mark: 177 ...Executor): Found continuous range in 19 steps ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 1/1083 marks by primary key, 1 marks to read from 1 ranges ...Reading ...approx. 8192 rows starting from 1441792 ``` @@ -594,7 +594,7 @@ LIMIT 10; │ UserID │ │ Condition: (UserID in [749927693, 749927693]) │ │ Parts: 1/1 │ -# highlight-next-line +# highlight-next-line {#highlight-next-line} │ Granules: 1/1083 │ └───────────────────────────────────────────────────────────────────────────────────────┘ @@ -713,7 +713,7 @@ ClickHouse нужно найти (и потоково считать все зн -### Вторичные ключевые столбцы могут быть (не)эффективными +### Вторичные ключевые столбцы могут быть (не)эффективными {#secondary-key-columns-can-not-be-inefficient} Когда запрос фильтрует данные по столбцу, который является частью составного ключа и при этом является первым ключевым столбцом, [ClickHouse выполняет алгоритм бинарного поиска по индексным меткам этого ключевого столбца](#the-primary-index-is-used-for-selecting-granules). @@ -759,7 +759,7 @@ LIMIT 10; └────────────┴───────┘ 10 строк в наборе. Прошло: 0.086 сек. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 8.81 млн строк, 799.69 МБ (102.11 млн строк/с., 9.27 ГБ/с.) ``` @@ -771,11 +771,11 @@ LIMIT 10; ```response ...Executor): Условие по ключу: (столбец 1 в ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Использован общий поиск с исключениями по индексу для части all_1_9_2 за 1537 шагов ...Executor): Выбрано 1/1 частей по ключу партиционирования, 1 часть по первичному ключу, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 1076/1083 засечек по первичному ключу, 1076 засечек для чтения из 5 диапазонов ...Executor): Чтение примерно 8814592 строк в 10 потоков ``` @@ -844,7 +844,7 @@ LIMIT 10; В нашем примерном наборе данных оба ключевых столбца (UserID, URL) имеют схожую высокую кардинальность, и, как уже объяснялось, универсальный алгоритм поиска с исключением не очень эффективен, когда предшествующий ключевой столбец для столбца URL имеет высокую (или схожую) кардинальность. -### Примечание об индексе пропуска данных +### Примечание об индексе пропуска данных {#note-about-data-skipping-index} Из-за схожей высокой кардинальности `UserID` и `URL` наша [фильтрация запросов по URL](/guides/best-practices/sparse-primary-indexes#secondary-key-columns-can-not-be-inefficient) также не дала бы существенного выигрыша от создания [вторичного индекса пропуска данных](./skipping-indexes.md) на столбце `URL` в нашей [таблице с составным первичным ключом (UserID, URL)](#a-table-with-a-primary-key). @@ -909,7 +909,7 @@ ClickHouse создал дополнительный индекс, которы -### Вариант 1: вторичные таблицы +### Вариант 1: вторичные таблицы {#option-1-secondary-tables} @@ -989,7 +989,7 @@ LIMIT 10; └────────────┴───────┘ 10 строк в наборе. Затрачено: 0.017 сек. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 319.49 тысяч строк, 11.38 MB (18.41 million rows/s., 655.75 MB/s.) ``` @@ -1005,13 +1005,13 @@ LIMIT 10; ```response ...Executor): Условие по ключу: (столбец 0 в ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Выполняется бинарный поиск по диапазону индекса для части all_1_9_2 (1083 гранулы) ...Executor): Найдена (ЛЕВАЯ) граничная гранула: 644 ...Executor): Найдена (ПРАВАЯ) граничная гранула: 683 ...Executor): Найден непрерывный диапазон за 19 шагов ...Executor): Выбрано 1/1 частей по ключу партиционирования, 1 часть по первичному ключу, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 39/1083 гранул по первичному ключу, 39 гранул для чтения из 1 диапазона ...Executor): Чтение примерно 319488 строк с использованием 2 потоков ``` @@ -1146,7 +1146,7 @@ LIMIT 10; └────────────┴───────┘ 10 строк в наборе. Прошло: 0.026 сек. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 335.87 тыс. строк, 13.54 МБ (12.91 млн строк/с., 520.38 МБ/с.) ``` @@ -1159,7 +1159,7 @@ LIMIT 10; ```response ...Executor): Key condition: (column 0 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Running binary search on index range ... ... ...Executor): Selected 4/4 parts by partition key, 4 parts by primary key, @@ -1236,7 +1236,7 @@ LIMIT 10; └────────────┴───────┘ 10 строк в наборе. Прошло: 0.029 сек. -# highlight-next-line +# highlight-next-line {#highlight-next-line} Обработано 319.49 тыс. строк, 1 1.38 МБ (11.05 млн строк/сек., 393.58 МБ/сек.) ``` @@ -1249,14 +1249,14 @@ LIMIT 10; ```response ...Executor): Условие по ключу: (столбец 0 в ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Выполняется бинарный поиск по диапазону индекса для части prj_url_userid (1083 гранулы) ...Executor): ... # highlight-next-line ...Executor): Выбрана полная обычная проекция prj_url_userid ...Executor): требуемые столбцы проекции: URL, UserID ...Executor): Выбрано 1/1 частей по ключу партиции, 1 частей по первичному ключу, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 39/1083 гранул по первичному ключу, 39 гранул для чтения из 1 диапазона ...Executor): Чтение приблизительно 319488 строк с использованием 2 потоков ``` @@ -1448,7 +1448,7 @@ WHERE UserID = 112304 Причина в том, что [generic exclusion search algorithm](https://github.com/ClickHouse/ClickHouse/blob/22.3/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp#L1444) работает наиболее эффективно, когда [гранулы](#the-primary-index-is-used-for-selecting-granules) выбираются по столбцу вторичного ключа, при этом предшествующий ему столбец ключа имеет более низкую кардинальность. Мы подробно показали это в [предыдущем разделе](#generic-exclusion-search-algorithm) данного руководства. -### Оптимальный коэффициент сжатия файлов данных +### Оптимальный коэффициент сжатия файлов данных {#efficient-filtering-on-secondary-key-columns} Этот запрос сравнивает коэффициент сжатия столбца `UserID` в двух таблицах, созданных выше: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md index 855aa7ff57a..83890616327 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md @@ -19,8 +19,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; Используемый язык запросов задаётся параметром `dialect`. - -## Стандартный SQL +## Стандартный SQL {#standard-sql} Стандартный SQL — язык запросов, используемый в ClickHouse по умолчанию. @@ -28,8 +27,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; SET dialect = 'clickhouse' ``` - -## Конвейерный реляционный язык запросов (PRQL) +## Конвейерный реляционный язык запросов (PRQL) {#pipelined-relational-query-language-prql} @@ -52,8 +50,7 @@ aggregate { Внутренне ClickHouse транспилирует PRQL в SQL для выполнения запросов PRQL. - -## Язык запросов Kusto (KQL) +## Язык запросов Kusto (KQL) {#kusto-query-language-kql} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md b/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md index 9e565577392..cd2f4576e75 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md @@ -6,13 +6,11 @@ keywords: ['materialized view', 'агрегация'] doc_type: 'guide' --- - - -# Каскадные материализованные представления +# Каскадные материализованные представления {#cascading-materialized-views} Этот пример демонстрирует, как создать материализованное представление, а затем — как создать второе материализованное представление на основе первого (каскадно). На этой странице вы увидите, как это сделать, узнаете о многих возможностях и ограничениях. Разные варианты использования можно реализовать, создавая материализованное представление, использующее в качестве источника другое материализованное представление. - + + + + allowfullscreen +/> -
+
ClickHouse предоставляет несколько подходов к работе с JSON, каждый из которых имеет свои преимущества, недостатки и области применения. В этом руководстве мы рассмотрим, как загружать JSON и оптимально проектировать схему. Руководство состоит из следующих разделов: -- [Loading JSON](/integrations/data-formats/json/loading) - Загрузка и выполнение запросов к структурированному и полуструктурированному JSON в ClickHouse с использованием простых схем. -- [JSON schema inference](/integrations/data-formats/json/inference) - Использование автоматического вывода схемы JSON для выполнения запросов к JSON и создания схем таблиц. -- [Designing JSON schema](/integrations/data-formats/json/schema) - Этапы проектирования и оптимизации схемы JSON. -- [Exporting JSON](/integrations/data-formats/json/exporting) - Как экспортировать JSON. -- [Handling other JSON Formats](/integrations/data-formats/json/other-formats) - Рекомендации по работе с форматами JSON, отличными от формата с разделением по строкам (NDJSON). -- [Other approaches to modeling JSON](/integrations/data-formats/json/other-approaches) - Устаревшие подходы к моделированию JSON. **Не рекомендуется к использованию.** \ No newline at end of file +* [Loading JSON](/integrations/data-formats/json/loading) - Загрузка и выполнение запросов к структурированному и полуструктурированному JSON в ClickHouse с использованием простых схем. +* [JSON schema inference](/integrations/data-formats/json/inference) - Использование автоматического вывода схемы JSON для выполнения запросов к JSON и создания схем таблиц. +* [Designing JSON schema](/integrations/data-formats/json/schema) - Этапы проектирования и оптимизации схемы JSON. +* [Exporting JSON](/integrations/data-formats/json/exporting) - Как экспортировать JSON. +* [Handling other JSON Formats](/integrations/data-formats/json/other-formats) - Рекомендации по работе с форматами JSON, отличными от формата с разделением по строкам (NDJSON). +* [Other approaches to modeling JSON](/integrations/data-formats/json/other-approaches) - Устаревшие подходы к моделированию JSON. **Не рекомендуется к использованию.** \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md index f7691d59497..7dc6de90385 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md @@ -17,7 +17,7 @@ doc_type: 'guide' -## Загрузка структурированного JSON +## Загрузка структурированного JSON {#loading-structured-json} В этом разделе предполагается, что данные находятся в формате [`NDJSON`](https://github.com/ndjson/ndjson-spec) (Newline delimited JSON — JSON, где объекты разделены переводами строки), который в ClickHouse известен как [`JSONEachRow`](/interfaces/formats/JSONEachRow), и хорошо структурированы, то есть имена и типы столбцов фиксированы. Формат `NDJSON` является предпочтительным для загрузки JSON из‑за своей компактности и эффективного использования места, но поддерживаются и другие форматы как для [ввода, так и вывода](/interfaces/formats/JSON). @@ -124,7 +124,7 @@ FORMAT JSONEachRow В этих примерах используется формат `JSONEachRow`. Поддерживаются и другие распространённые форматы JSON; примеры их загрузки приведены [здесь](/integrations/data-formats/json/other-formats). -## Загрузка полуструктурированного JSON +## Загрузка полуструктурированного JSON {#loading-semi-structured-json} В нашем предыдущем примере мы загружали JSON с фиксированной структурой и хорошо известными именами ключей и типами. На практике так бывает не всегда — ключи могут добавляться, а их типы меняться. Это часто встречается в сценариях, связанных с данными для наблюдаемости (Observability). @@ -200,7 +200,7 @@ LIMIT 2 Обратите внимание на разницу в производительности при загрузке данных. Для JSON-столбца при вставке требуется определение типов, а также дополнительное место для хранения, если есть столбцы, содержащие значения более чем одного типа. Хотя тип JSON можно настроить (см. [Проектирование схемы JSON](/integrations/data-formats/json/schema)) так, чтобы его производительность была сопоставима с явным объявлением столбцов, по умолчанию он намеренно остаётся гибким. Однако эта гибкость имеет свою цену. -### Когда использовать тип JSON +### Когда использовать тип JSON {#when-to-use-the-json-type} Используйте тип JSON, когда ваши данные: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md index 0464a38de60..dc9bb5522cb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md @@ -6,9 +6,7 @@ keywords: ['json', 'formats'] doc_type: 'reference' --- - - -# Другие подходы к моделированию JSON +# Другие подходы к моделированию JSON {#other-approaches-to-modeling-json} **Ниже приведены альтернативные подходы к моделированию JSON в ClickHouse. Они приведены для полноты и использовались до появления типа JSON, поэтому, как правило, не рекомендуются и не применяются в большинстве сценариев.** @@ -16,9 +14,7 @@ doc_type: 'reference' К разным объектам в одной и той же схеме можно применять разные техники. Например, для одних объектов лучше всего подойдет тип `String`, а для других — тип `Map`. Обратите внимание, что после выбора типа `String` больше не требуется принимать какие-либо решения о схеме. Напротив, в ключ `Map` можно вложить подчиненные объекты, включая `String`, представляющую JSON, как показано ниже: ::: - - -## Использование типа String +## Использование типа String {#using-string} Если объекты очень динамичны, не имеют предсказуемой структуры и содержат произвольные вложенные объекты, следует использовать тип `String`. Значения можно извлекать во время выполнения запроса с помощью JSON‑функций, как показано ниже. @@ -95,7 +91,6 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/arxiv/arxiv.j 0 строк в наборе. Прошло: 25.186 с. Обработано 2.52 миллиона строк, 1.38 ГБ (99.89 тысячи строк/с, 54.79 МБ/с.) ``` - Предположим, мы хотим посчитать количество статей, выпущенных по годам. Сравним следующий запрос, использующий только строковое поле, с [структурированной версией](/integrations/data-formats/json/inference#creating-tables) схемы: ```sql @@ -156,7 +151,7 @@ LIMIT 10 Гибкость такого подхода имеет очевидную цену в виде потерь производительности и усложнения синтаксиса, поэтому его следует использовать только для высокодинамичных объектов в схеме. -### Простые JSON-функции +### Простые JSON-функции {#simple-json-functions} Выше приведены примеры использования семейства функций JSON*. Они используют полноценный JSON-парсер на базе [simdjson](https://github.com/simdjson/simdjson), который строго относится к разбору и различает одноимённые поля на разных уровнях вложенности. Эти функции способны корректно обрабатывать синтаксически правильный, но плохо отформатированный JSON, например с двойными пробелами между ключами. @@ -174,7 +169,6 @@ LIMIT 10 В то время как следующий пример будет успешно разобран: - ````json {"@timestamp":893964617,"clientip":"40.135.0.0","request":{"method":"GET", "path":"/images/hm_bg.jpg","version":"HTTP/1.0"},"status":200,"size":24736} @@ -209,8 +203,7 @@ LIMIT 10 Приведённый выше запрос использует `simpleJSONExtractString` для извлечения ключа `created`, учитывая, что нам нужно только первое значение для даты публикации. В этом случае ограничения функций `simpleJSON*` приемлемы ради повышения производительности. - -## Использование типа Map +## Использование типа Map {#using-map} Если объект используется для хранения произвольных ключей (преимущественно одного типа), рассмотрите использование типа `Map`. В идеале количество уникальных ключей не должно превышать нескольких сотен. Тип `Map` также можно использовать для объектов с вложенными объектами при условии, что последние однородны по своим типам. В целом мы рекомендуем использовать тип `Map` для лейблов и тегов, например лейблов подов Kubernetes в логах. @@ -224,7 +217,7 @@ LIMIT 10 При моделировании объектов как `Map` в качестве ключа используется строка (`String`), в которой хранится имя ключа JSON. Поэтому `Map` всегда будет иметь вид `Map(String, T)`, где `T` зависит от данных. ::: -#### Примитивные значения +#### Примитивные значения {#primitive-values} Простейшее применение `Map` — когда объект содержит значения одного и того же примитивного типа. В большинстве случаев это означает использование типа `String` для значения `T`. @@ -281,13 +274,12 @@ SELECT company.labels['type'] AS type FROM people Полный набор функций `Map`, доступных для выполнения запросов, описан [здесь](/sql-reference/functions/tuple-map-functions.md). Если ваши данные не имеют согласованного типа, существуют функции для выполнения [необходимого приведения типов](/sql-reference/functions/type-conversion-functions). -#### Значения объектов +#### Значения объектов {#object-values} Тип `Map` также можно использовать для представления объектов с вложенными объектами, при условии, что у последних согласованы их типы. Предположим, ключ `tags` для нашего объекта `persons` требует согласованной структуры, где вложенный объект для каждого `tag` имеет столбцы `name` и `time`. Упрощённый пример такого JSON-документа может выглядеть следующим образом: - ```json { "id": 1, @@ -360,8 +352,7 @@ FORMAT JSONEachRow } ``` - -## Использование типа Nested +## Использование типа Nested {#using-nested} Тип [Nested](/sql-reference/data-types/nested-data-structures/nested) можно использовать для моделирования статических объектов, которые редко изменяются, в качестве альтернативы `Tuple` и `Array(Tuple)`. В целом мы рекомендуем избегать использования этого типа для JSON, поскольку его поведение часто оказывается запутанным. Основное преимущество `Nested` заключается в том, что подколонки могут использоваться в ключах сортировки. @@ -396,11 +387,11 @@ CREATE table http ) ENGINE = MergeTree() ORDER BY (status, timestamp); ``` -### flatten_nested +### flatten_nested {#flatten_nested} Параметр `flatten_nested` управляет поведением типа данных `Nested`. -#### flatten_nested=1 +#### flatten_nested=1 {#flatten_nested1} Значение `1` (по умолчанию) не поддерживает произвольную глубину вложенности. При таком значении проще всего рассматривать вложенную структуру данных как несколько столбцов [Array](/sql-reference/data-types/array) одинаковой длины. Поля `method`, `path` и `version` фактически являются отдельными столбцами `Array(Type)` с одним критическим ограничением: **длина полей `method`, `path` и `version` должна быть одинаковой.** Если мы воспользуемся `SHOW CREATE TABLE`, это иллюстрируется следующим образом: @@ -471,10 +462,9 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth Получена 1 строка. Время выполнения: 0.002 сек. ``` - Обратите внимание, что использование `Array` для подстолбцов означает, что можно потенциально использовать весь спектр [функций работы с массивами](/sql-reference/functions/array-functions), включая клаузу [`ARRAY JOIN`](/sql-reference/statements/select/array-join), что полезно, если ваши столбцы содержат несколько значений. -#### flatten_nested=0 +#### flatten_nested=0 {#flatten_nested0} Это позволяет использовать произвольный уровень вложенности и означает, что вложенные столбцы остаются одним массивом `Tuple` — по сути, они становятся тем же самым, что и `Array(Tuple)`. @@ -546,7 +536,7 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth Получена 1 строка. Время выполнения: 0.002 сек. ``` -### Пример +### Пример {#example} Более объёмный пример приведённых выше данных доступен в общедоступном бакете в S3 по адресу: `s3://datasets-documentation/http/`. @@ -584,7 +574,6 @@ size FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/doc Чтобы выполнять запросы к этим данным, нам необходимо обращаться к полям запроса как к массивам. Ниже мы агрегируем ошибки и HTTP-методы за фиксированный период времени. - ```sql SELECT status, request.method[1] AS method, count() AS c FROM http @@ -604,7 +593,7 @@ ORDER BY c DESC LIMIT 5; 5 строк в наборе. Прошло: 0,007 сек. ``` -### Использование парных массивов +### Использование парных массивов {#using-pairwise-arrays} Парные массивы обеспечивают баланс между гибкостью представления JSON в виде строк (`String`) и производительностью более структурированного подхода. Схема гибкая в том смысле, что любые новые поля потенциально могут быть добавлены в корень. Однако это требует значительно более сложного синтаксиса запросов и несовместимо с вложенными структурами. @@ -668,7 +657,6 @@ GROUP BY method, status ORDER BY c DESC LIMIT 5; └────────┴────────┴───────┘ ``` - 5 строк в наборе. Прошло времени: 0.383 сек. Обработано 8.22 млн строк, 1.97 ГБ (21.45 млн строк/с, 5.15 ГБ/с.) Пиковое потребление памяти: 51.35 МиБ. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md index 4e5974300f5..db143fbcf73 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md @@ -13,11 +13,11 @@ import json_offsets from '@site/static/images/integrations/data-ingestion/data-f import shared_json_column from '@site/static/images/integrations/data-ingestion/data-formats/json_shared_column.png'; -# Проектирование схемы +# Проектирование схемы {#designing-your-schema} Хотя [вывод схемы](/integrations/data-formats/json/inference) можно использовать для формирования начальной схемы JSON‑данных и выполнения запросов к JSON‑файлам непосредственно в хранилище, например в S3, пользователям следует стремиться разработать оптимизированную версионируемую схему для своих данных. Ниже рассматривается рекомендуемый подход к моделированию JSON‑структур. -## Статический и динамический JSON +## Статический и динамический JSON {#static-vs-dynamic-json} Основная задача при определении схемы для JSON — определить подходящий тип для значения каждого ключа. Мы рекомендуем рекурсивно применять к каждому ключу в иерархии JSON следующие правила, чтобы определить соответствующий тип для каждого ключа. @@ -92,7 +92,7 @@ import shared_json_column from '@site/static/images/integrations/data-ingestion/ Структуры с сотнями или тысячами статических ключей можно считать динамическими, так как редко бывает реалистично статически объявлять для них столбцы. Тем не менее, при возможности [пропускайте пути](#using-type-hints-and-skipping-paths), которые не нужны, чтобы сократить затраты как на хранение, так и на определение типов. ::: -## Обработка статических структур +## Обработка статических структур {#handling-static-structures} Мы рекомендуем обрабатывать статические структуры с помощью именованных кортежей (`Tuple`). Массивы объектов могут храниться в массивах кортежей, то есть `Array(Tuple)`. Внутри самих кортежей столбцы и их соответствующие типы должны определяться по тем же правилам. Это может приводить к вложенным `Tuple` для представления вложенных объектов, как показано ниже. @@ -203,7 +203,7 @@ ORDER BY company.name ``` -### Обработка значений по умолчанию +### Обработка значений по умолчанию {#handling-default-values} Даже если объекты JSON структурированы, они часто разреженные и содержат только подмножество известных ключей. К счастью, тип `Tuple` не требует, чтобы в JSON-полезной нагрузке присутствовали все столбцы. Если какие-либо из них не указаны, будут использованы значения по умолчанию. @@ -282,7 +282,7 @@ FORMAT PrettyJSONEachRow ::: -### Обработка новых столбцов +### Обработка новых столбцов {#handling-new-columns} Хотя структурированный подход наиболее прост, когда ключи JSON статичны, его все же можно использовать и в том случае, если изменения схемы можно спланировать, то есть новые ключи известны заранее и схему можно соответствующим образом изменить. @@ -360,7 +360,7 @@ SELECT id, nickname FROM people ``` -## Обработка полуструктурированных/динамических структур +## Обработка полуструктурированных/динамических структур {#handling-semi-structured-dynamic-structures} Если JSON-данные являются полуструктурированными, где ключи могут динамически добавляться и/или иметь несколько типов, рекомендуется использовать тип [`JSON`](/sql-reference/data-types/newjson). @@ -491,7 +491,7 @@ SELECT id, nickname FROM people - **Избежание риска взрывного роста числа столбцов** – хотя тип JSON масштабируется потенциально до тысяч столбцов, где подстолбцы хранятся как отдельные столбцы, это может привести к «взрыву» файлов столбцов, когда создаётся чрезмерное количество файлов столбцов, что ухудшает производительность. Чтобы минимизировать этот риск, базовый [тип Dynamic](/sql-reference/data-types/dynamic), используемый JSON, предоставляет параметр [`max_dynamic_paths`](/sql-reference/data-types/newjson#reading-json-paths-as-sub-columns), который ограничивает количество уникальных путей, хранимых как отдельные файлы столбцов. После достижения порога дополнительные пути сохраняются в общем файле столбца с использованием компактного кодированного формата, что позволяет сохранить производительность и эффективность хранения при обеспечении гибкости ингестии данных. Однако доступ к этому общему файлу столбца менее эффективен по производительности. Обратите внимание, что столбец JSON может использоваться с [подсказками типов](#using-type-hints-and-skipping-paths). Столбцы с «подсказками» будут обеспечивать ту же производительность, что и выделенные столбцы. - **Упрощённая интроспекция путей и типов** – хотя тип JSON поддерживает [функции интроспекции](/sql-reference/data-types/newjson#introspection-functions) для определения выведенных типов и путей, статические структуры зачастую проще исследовать, например, с помощью `DESCRIBE`. -### Один столбец JSON +### Один столбец JSON {#single-json-column} Этот подход полезен для прототипирования и задач data engineering. В продакшене старайтесь использовать `JSON` только для динамических подструктур, когда это действительно необходимо. @@ -667,7 +667,7 @@ FROM people ``` -### Целевой столбец JSON +### Целевой столбец JSON {#targeted-json-column} Хотя такой подход полезен при прототипировании и решении задач инженерии данных, в продуктивной среде мы рекомендуем по возможности использовать явную схему. @@ -767,7 +767,7 @@ FORMAT PrettyJsonEachRow ``` -### Использование подсказок типов и пропуска путей +### Использование подсказок типов и пропуска путей {#using-type-hints-and-skipping-paths} Подсказки типов позволяют указать тип для пути и его подстолбца, предотвращая излишний вывод типов. Рассмотрим следующий пример, в котором мы задаём типы для JSON-ключей `dissolved`, `employees` и `founded` внутри JSON-столбца `company.labels`. @@ -934,7 +934,7 @@ FORMAT PrettyJSONEachRow В результате для наборов данных, которые в основном однородны, но при этом выигрывают от гибкости JSON, подсказки типов предоставляют удобный способ сохранить производительность без необходимости переработки схемы или конвейера приёма данных. -### Настройка динамических путей +### Настройка динамических путей {#configuring-dynamic-paths} ClickHouse хранит каждый JSON-путь как подколонку в настоящем колоночном формате, обеспечивая те же преимущества по производительности, что и для традиционных столбцов — например, сжатие, SIMD-ускоренную обработку и минимальное дисковое I/O. Каждая уникальная комбинация пути и типа в ваших JSON-данных может быть вынесена в отдельный файловый столбец на диске. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md index 0faf3e900d3..9652f1d8def 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md @@ -10,7 +10,7 @@ keywords: ['parquet', 'колоночный формат', 'формат дан -# Работа с Parquet в ClickHouse +# Работа с Parquet в ClickHouse {#working-with-parquet-in-clickhouse} Parquet — это эффективный файловый формат для хранения данных в колоночном формате. ClickHouse поддерживает как чтение, так и запись файлов Parquet. @@ -24,7 +24,7 @@ ClickHouse поддерживает как чтение, так и запись -## Импорт из Parquet +## Импорт из Parquet {#importing-from-parquet} Перед загрузкой данных мы можем использовать функцию [file()](/sql-reference/functions/files.md/#file), чтобы изучить структуру [примерного файла формата Parquet](assets/data.parquet): @@ -64,7 +64,7 @@ LIMIT 3; ::: -## Импорт в существующую таблицу +## Импорт в существующую таблицу {#importing-to-an-existing-table} Создадим таблицу, в которую будем импортировать данные в формате Parquet: @@ -103,7 +103,7 @@ LIMIT 5; Обратите внимание, что ClickHouse автоматически преобразовал строки формата Parquet (в столбце `date`) в тип `Date`. Это происходит потому, что ClickHouse выполняет приведение типов на основе типов в целевой таблице. -## Загрузка локального файла на удалённый сервер +## Загрузка локального файла на удалённый сервер {#inserting-a-local-file-to-remote-server} Если вы хотите загрузить локальный файл Parquet на удалённый сервер ClickHouse, вы можете сделать это, передав его содержимое в `clickhouse-client` через pipe, как показано ниже: @@ -112,7 +112,7 @@ clickhouse client -q "INSERT INTO sometable FORMAT Parquet" < data.parquet ``` -## Создание новых таблиц из файлов Parquet +## Создание новых таблиц из файлов Parquet {#creating-new-tables-from-parquet-files} Поскольку ClickHouse читает схему файлов Parquet, мы можем динамически создавать таблицы: @@ -141,7 +141,7 @@ DESCRIBE TABLE imported_from_parquet; По умолчанию ClickHouse строго проверяет имена столбцов, их типы и значения. Но иногда при импорте можно игнорировать несуществующие столбцы или неподдерживаемые значения. Это можно настроить с помощью [настроек Parquet](/interfaces/formats/Parquet#format-settings). -## Экспорт в формат Parquet +## Экспорт в формат Parquet {#exporting-to-parquet-format} :::tip При использовании `INTO OUTFILE` с ClickHouse Cloud команды в `clickhouse client` нужно запускать на той машине (хосте), на которую будет записан файл. @@ -159,7 +159,7 @@ FORMAT Parquet В результате в текущем рабочем каталоге будет создан файл `export.parquet`. -## Типы данных ClickHouse и Parquet +## Типы данных ClickHouse и Parquet {#clickhouse-and-parquet-data-types} Типы данных ClickHouse и Parquet в основном совпадают, но всё же [имеют некоторые отличия](/interfaces/formats/Parquet#data-types-matching-parquet). Например, ClickHouse экспортирует тип `DateTime` как значение типа `int64` в формате Parquet. Если затем импортировать его обратно в ClickHouse, мы увидим числа ([файл time.parquet](assets/time.parquet)): diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md index dfeb10aafaa..c6717999b4d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md @@ -9,13 +9,13 @@ keywords: ['формат SQL', 'экспорт данных', 'импорт да -# Вставка и выгрузка SQL-данных в ClickHouse +# Вставка и выгрузка SQL-данных в ClickHouse {#inserting-and-dumping-sql-data-in-clickhouse} ClickHouse можно легко интегрировать в OLTP‑инфраструктуры баз данных разными способами. Один из вариантов — передавать данные между другими базами данных и ClickHouse с помощью SQL‑дампов. -## Создание SQL-дампов +## Создание SQL-дампов {#creating-sql-dumps} Данные можно выгрузить в формате SQL с помощью [SQLInsert](/interfaces/formats/SQLInsert). ClickHouse запишет данные в виде `INSERT INTO VALUES(...` и будет использовать настройку [`output_format_sql_insert_table_name`](/operations/settings/settings-formats.md/#output_format_sql_insert_table_name) в качестве имени таблицы: @@ -46,7 +46,7 @@ mysql some_db < dump.sql SET output_format_sql_insert_max_batch_size = 1000; ``` -### Экспорт набора значений +### Экспорт набора значений {#exporting-a-set-of-values} В ClickHouse есть формат [Values](/interfaces/formats/Values), который аналогичен SQL INSERT, но опускает оператор `INSERT INTO table VALUES` и содержит только набор значений: @@ -59,7 +59,7 @@ SELECT * FROM some_data LIMIT 3 FORMAT Values ``` -## Импорт данных из SQL-дампов +## Импорт данных из SQL-дампов {#inserting-data-from-sql-dumps} Для чтения SQL-дампов используется формат [MySQLDump](/interfaces/formats/MySQLDump): diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md index 4dfdbc17ceb..f65e20847f1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md @@ -10,13 +10,13 @@ keywords: ['форматы данных', 'шаблоны', 'regex', 'польз -# Импорт и экспорт произвольных текстовых данных с помощью форматов Template и Regex в ClickHouse +# Импорт и экспорт произвольных текстовых данных с помощью форматов Template и Regex в ClickHouse {#importing-and-exporting-custom-text-data-using-templates-and-regex-in-clickhouse} Нам часто приходится работать с данными в произвольных текстовых форматах. Это может быть нестандартный формат, некорректный JSON или «сломанный» CSV. Использование стандартных парсеров, таких как CSV или JSON, в таких случаях не всегда работает. Но в ClickHouse для этого предусмотрены мощные форматы Template и Regex. -## Импорт на основе шаблона +## Импорт на основе шаблона {#importing-based-on-a-template} Предположим, что мы хотим импортировать данные из следующего [файла журнала](assets/error.log): @@ -88,7 +88,7 @@ GROUP BY request └──────────────────────────────────────────────────┴─────────┘ ``` -### Пропуск пробелов +### Пропуск пробелов {#skipping-whitespaces} Рекомендуется использовать [TemplateIgnoreSpaces](/interfaces/formats/TemplateIgnoreSpaces), который позволяет игнорировать пробелы между разделителями в шаблоне: @@ -98,7 +98,7 @@ TemplateIgnoreSpaces --> "p1:${p1:CSV}, p2:${p2:CSV}" ``` -## Экспорт данных с использованием шаблонов +## Экспорт данных с использованием шаблонов {#exporting-data-using-templates} Мы также можем экспортировать данные в любой текстовый формат с помощью шаблонов. В этом случае нужно создать два файла: @@ -142,7 +142,7 @@ FORMAT Template SETTINGS format_template_resultset = 'output.results', --- Прочитано 1000 строк за 0.001380604 --- ``` -### Экспорт в HTML-файлы +### Экспорт в HTML-файлы {#exporting-to-html-files} Результаты, сформированные по шаблону, также можно экспортировать в файлы с помощью предложения [`INTO OUTFILE`](/sql-reference/statements/select/into-outfile.md). Сгенерируем HTML-файлы на основе указанных форматов [resultset](assets/html.results) и [row](assets/html.row): @@ -157,7 +157,7 @@ SETTINGS format_template_resultset = 'html.results', format_template_row = 'html.row' ``` -### Экспорт в XML +### Экспорт в XML {#exporting-to-xml} Формат шаблона можно использовать для генерации файлов любого текстового формата, включая XML. Просто подготовьте соответствующий шаблон и выполните экспорт. @@ -203,7 +203,7 @@ FORMAT XML ``` -## Импорт данных на основе регулярных выражений +## Импорт данных на основе регулярных выражений {#importing-data-based-on-regular-expressions} Формат [Regexp](/interfaces/formats/Regexp) предназначен для более сложных случаев, когда входные данные необходимо разбирать более сложным способом. Давайте разберём наш пример с файлом [error.log](assets/error.log), но на этот раз извлечём имя файла и протокол, чтобы сохранить их в отдельные столбцы. Для начала подготовим для этого новую таблицу: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md index d37457d767b..6e2d3f1b687 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md @@ -6,7 +6,7 @@ description: 'Главная страница раздела об ингести doc_type: 'landing-page' --- -# Ингестия данных +# Ингестия данных {#data-ingestion} ClickHouse интегрируется с рядом решений для интеграции и трансформации данных. Для получения дополнительной информации ознакомьтесь со страницами ниже: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md index 94cb56c155a..63838b7a23c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md @@ -6,7 +6,7 @@ title: 'Источники данных' doc_type: 'landing-page' --- -# Источники данных +# Источники данных {#data-sources} ClickHouse позволяет с лёгкостью выполнять приём данных в базу данных из различных источников. Дополнительную информацию см. на следующих страницах: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md index 6d621c92536..1cb59452e6c 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md @@ -17,7 +17,7 @@ import dynamodb_map_columns from '@site/static/images/integrations/data-ingestio import Image from '@theme/IdealImage'; -# CDC из DynamoDB в ClickHouse +# CDC из DynamoDB в ClickHouse {#cdc-from-dynamodb-to-clickhouse} @@ -50,9 +50,9 @@ import Image from '@theme/IdealImage'; -## 3. Загрузка снимка в ClickHouse +## 3. Загрузка снимка в ClickHouse {#3-load-the-snapshot-into-clickhouse} -### Создайте необходимые таблицы +### Создайте необходимые таблицы {#create-necessary-tables} Данные снимка из DynamoDB будут выглядеть примерно так: @@ -115,7 +115,7 @@ ORDER BY id; * Таблица должна использовать ключ партиционирования в качестве ключа сортировки (задаваемого в `ORDER BY`) * Строки с одинаковым ключом сортировки будут очищаться от дубликатов на основе столбца `version`. -### Создание snapshot ClickPipe +### Создание snapshot ClickPipe {#create-the-snapshot-clickpipe} Теперь вы можете создать ClickPipe для загрузки snapshot-данных из S3 в ClickHouse. Следуйте руководству по S3 ClickPipe [здесь](/integrations/clickpipes/object-storage), но используйте следующие настройки: @@ -146,7 +146,7 @@ https://{bucket}.s3.amazonaws.com/{prefix}/AWSDynamoDB/{export-id}/data/* -## 5. Очистка (необязательно) +## 5. Очистка (необязательно) {#5-cleanup-optional} После завершения снапшотного ClickPipe вы можете удалить таблицу снимка и материализованное представление. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md index 802f77d3da7..9b57b6703ae 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md @@ -16,7 +16,7 @@ import Jdbc02 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-02 import Jdbc03 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-03.png'; -# Подключение ClickHouse к внешним источникам данных с помощью JDBC +# Подключение ClickHouse к внешним источникам данных с помощью JDBC {#connecting-clickhouse-to-external-data-sources-with-jdbc} :::note Использование JDBC требует ClickHouse JDBC Bridge, поэтому вам понадобится использовать `clickhouse-local` на локальной машине, чтобы передавать данные из вашей базы данных в ClickHouse Cloud. Перейдите на страницу [**Using clickhouse-local**](/cloud/migration/clickhouse-local#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge) в разделе **Migrate** документации для получения подробной информации. @@ -44,7 +44,7 @@ import Jdbc03 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-03 -## Установка ClickHouse JDBC Bridge локально +## Установка ClickHouse JDBC Bridge локально {#install-the-clickhouse-jdbc-bridge-locally} Самый простой способ использовать ClickHouse JDBC Bridge — установить и запустить его на той же машине, где работает ClickHouse: @@ -107,7 +107,7 @@ java -jar clickhouse-jdbc-bridge-2.0.7-shaded.jar ::: -## Использование JDBC-подключения из ClickHouse +## Использование JDBC-подключения из ClickHouse {#use-the-jdbc-connection-from-within-clickhouse} Теперь ClickHouse может получать доступ к данным MySQL с помощью [табличной функции jdbc](/sql-reference/table-functions/jdbc.md) или [движка таблицы JDBC](/engines/table-engines/integrations/jdbc.md). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md index f60acf8d735..8f824b5de13 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md @@ -10,27 +10,24 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - -# Подключение ClickHouse к PostgreSQL +# Подключение ClickHouse к PostgreSQL {#connecting-clickhouse-to-postgresql} На этой странице рассматриваются следующие варианты интеграции PostgreSQL с ClickHouse: -- использование движка таблиц `PostgreSQL` для чтения данных из таблицы PostgreSQL -- использование экспериментального движка баз данных `MaterializedPostgreSQL` для синхронизации базы данных в PostgreSQL с базой данных в ClickHouse +* использование движка таблиц `PostgreSQL` для чтения данных из таблицы PostgreSQL +* использование экспериментального движка баз данных `MaterializedPostgreSQL` для синхронизации базы данных в PostgreSQL с базой данных в ClickHouse :::tip Мы рекомендуем использовать [ClickPipes](/integrations/clickpipes/postgres) — управляемый сервис интеграции для ClickHouse Cloud на базе PeerDB. В качестве альтернативы [PeerDB](https://github.com/PeerDB-io/peerdb) доступен как open-source CDC‑инструмент, специально разработанный для репликации базы данных PostgreSQL как в самостоятельно развернутый ClickHouse, так и в ClickHouse Cloud. ::: - - -## Использование табличного движка PostgreSQL +## Использование табличного движка PostgreSQL {#using-the-postgresql-table-engine} Табличный движок `PostgreSQL` позволяет выполнять операции **SELECT** и **INSERT** над данными, хранящимися на удалённом сервере PostgreSQL, из ClickHouse. В этой статье иллюстрируются базовые способы интеграции на примере одной таблицы. -### 1. Настройка PostgreSQL +### 1. Настройка PostgreSQL {#1-setting-up-postgresql} 1. В `postgresql.conf` добавьте следующую запись, чтобы разрешить PostgreSQL прослушивать сетевые интерфейсы: @@ -93,7 +90,7 @@ psql -U clickhouse_user -W -d db_in_psg -h <ваш_хост_postgresql> Обратитесь к ClickHouse [Cloud Endpoints API](/cloud/get-started/query-endpoints), чтобы получить сведения об исходящем трафике. ::: -### 2. Определите таблицу в ClickHouse +### 2. Определите таблицу в ClickHouse {#2-define-a-table-in-clickhouse} 1. Подключитесь к `clickhouse-client`: @@ -131,7 +128,7 @@ ENGINE = PostgreSQL('postgres-host.domain.com:5432', 'db_in_psg', 'table1', 'cli См. страницу документации [PostgreSQL table engine](/engines/table-engines/integrations/postgresql) для полного списка параметров. ::: -### 3 Тестирование интеграции +### 3 Тестирование интеграции {#3-test-the-integration} 1. В ClickHouse просмотрите несколько первых строк: @@ -166,7 +163,6 @@ VALUES SELECT * FROM db_in_ch.table1 ``` - Ответ должен выглядеть следующим образом: ```response @@ -208,8 +204,7 @@ id | column1 В этом примере была продемонстрирована базовая интеграция между PostgreSQL и ClickHouse с использованием движка таблицы `PostrgeSQL`. Ознакомьтесь с [документацией по движку таблицы PostgreSQL](/engines/table-engines/integrations/postgresql), чтобы узнать о дополнительных возможностях, таких как указание схем, возврат только подмножества столбцов и подключение к нескольким репликам. Также рекомендуем ознакомиться с записью в блоге [ClickHouse and PostgreSQL - a match made in data heaven - part 1](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres). - -## Использование движка базы данных MaterializedPostgreSQL +## Использование движка базы данных MaterializedPostgreSQL {#using-the-materializedpostgresql-database-engine} @@ -220,7 +215,7 @@ id | column1 ***В описанных ниже процедурах используются PostgreSQL CLI (psql) и ClickHouse CLI (clickhouse-client). Сервер PostgreSQL установлен на Linux. Далее приведены минимальные настройки для новой тестовой установки базы данных PostgreSQL.*** -### 1. В PostgreSQL +### 1. В PostgreSQL {#1-in-postgresql} 1. В `postgresql.conf` установите минимальные параметры прослушивания, уровень WAL для репликации и слоты репликации: @@ -275,7 +270,6 @@ VALUES 7. Настройте PostgreSQL так, чтобы он разрешал подключения к новой базе данных новому пользователю для репликации. Ниже приведена минимально необходимая запись, которую нужно добавить в файл `pg_hba.conf`: - ```text # TYPE DATABASE USER ADDRESS METHOD host db1 clickhouse_user 192.168.1.0/24 password @@ -349,7 +343,7 @@ Query id: df2381ac-4e30-4535-b22e-8be3894aaafc └────┴─────────┘ ``` -### 3. Проверьте базовую репликацию +### 3. Проверьте базовую репликацию {#2-in-clickhouse} 1. В PostgreSQL добавьте новые строки: @@ -385,7 +379,7 @@ Query id: b0729816-3917-44d3-8d1a-fed912fb59ce └────┴─────────┘ ``` -### 4. Итоги +### 4. Итоги {#3-test-basic-replication} В данном руководстве по интеграции был рассмотрен простой пример репликации базы данных с одной таблицей, однако существуют и более продвинутые варианты, включая репликацию всей базы данных или добавление новых таблиц и схем к уже настроенным репликациям. Хотя DDL-команды не поддерживаются в этой схеме репликации, движок можно настроить на обнаружение изменений и перезагрузку таблиц при внесении структурных изменений. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md index 5765282fa5c..6decd83a1f8 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md @@ -40,7 +40,7 @@ import clickhouse_result from '@site/static/images/integrations/data-ingestion/e import Image from '@theme/IdealImage'; -# Интеграция EMQX с ClickHouse +# Интеграция EMQX с ClickHouse {#integrating-emqx-with-clickhouse} @@ -63,7 +63,7 @@ import Image from '@theme/IdealImage'; -## Получение сервиса ClickHouse Cloud +## Получение сервиса ClickHouse Cloud {#get-your-clickhouse-cloudservice} В ходе этой настройки мы развернули экземпляр ClickHouse в AWS в регионе N. Virginia (us-east -1), при этом экземпляр EMQX Cloud также был развернут в том же регионе. @@ -153,7 +153,7 @@ EMQX Cloud по умолчанию не допускает анонимные п -## Интеграция EMQX Cloud с ClickHouse Cloud +## Интеграция EMQX Cloud с ClickHouse Cloud {#integration-emqx-cloud-with-clickhouse-cloud} [EMQX Cloud Data Integrations](https://docs.emqx.com/en/cloud/latest/rule_engine/introduction.html#general-flow) используется для настройки правил обработки и реакции на потоки сообщений EMQX и события устройств. Data Integrations не только предоставляет понятное и гибкое «конфигурируемое» архитектурное решение, но также упрощает процесс разработки, повышает удобство использования и снижает степень связности между бизнес-системой и EMQX Cloud. Кроме того, он обеспечивает превосходную инфраструктуру для кастомизации проприетарных возможностей EMQX Cloud. @@ -163,7 +163,7 @@ EMQX Cloud предлагает более 30 нативных интеграц -### Создание ресурса ClickHouse +### Создание ресурса ClickHouse {#create-clickhouse-resource} Нажмите «Data Integrations» в левом меню и затем «View All Resources». Вы найдете ClickHouse в разделе Data Persistence или можете выполнить поиск по ClickHouse. @@ -177,7 +177,7 @@ EMQX Cloud предлагает более 30 нативных интеграц -### Создание нового правила +### Создание нового правила {#create-a-new-rule} Во время создания ресурса вы увидите всплывающее окно, и нажатие кнопки «New» перенаправит вас на страницу создания правила. @@ -212,7 +212,7 @@ FROM Теперь нажмите кнопку «NEXT». На этом шаге вы указываете EMQX Cloud, как записывать обработанные данные в вашу базу данных ClickHouse. -### Добавьте ответное действие +### Добавьте ответное действие {#add-a-response-action} Если у вас только один ресурс, вам не нужно изменять «Resource» и «Action Type». Вам нужно только задать SQL‑шаблон. Ниже приведён пример, используемый в этом руководстве: @@ -225,7 +225,7 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ Это шаблон для вставки данных в ClickHouse; здесь вы можете увидеть, как используются переменные. -### Просмотр сведений о правиле +### Просмотр сведений о правиле {#view-rules-details} Нажмите «Confirm» и «View Details». Теперь всё должно быть корректно настроено. На странице сведений о правиле вы можете убедиться, что интеграция данных работает. @@ -234,13 +234,13 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ Все MQTT-сообщения, отправленные в топик `temp_hum/emqx`, будут сохранены в вашей базе данных ClickHouse Cloud. -## Сохранение данных в ClickHouse +## Сохранение данных в ClickHouse {#saving-data-into-clickhouse} Мы будем имитировать данные о температуре и влажности и отправлять их в EMQX Cloud через MQTT X, а затем использовать EMQX Cloud Data Integrations для сохранения данных в ClickHouse Cloud. -### Публикация MQTT‑сообщений в EMQX Cloud +### Публикация MQTT‑сообщений в EMQX Cloud {#publish-mqtt-messages-to-emqx-cloud} Вы можете использовать любой MQTT‑клиент или SDK для публикации сообщений. В этом руководстве мы будем использовать [MQTT X](https://mqttx.app/) — удобный MQTT‑клиент, предоставляемый EMQ. @@ -274,13 +274,13 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ -### Просмотр мониторинга правил +### Просмотр мониторинга правил {#view-rules-monitoring} Проверьте мониторинг правил и убедитесь, что счётчик успешных срабатываний увеличился на единицу. -### Проверка сохранённых данных +### Проверка сохранённых данных {#check-the-data-persisted} Теперь пришло время посмотреть на данные в ClickHouse Cloud. В идеале данные, которые вы отправляете с помощью MQTTX, попадут в EMQX Cloud и будут сохранены в базе данных ClickHouse Cloud с помощью встроенной интеграции данных. @@ -293,6 +293,6 @@ SELECT * FROM emqx.temp_hum; -### Итоги +### Итоги {#summary} Вы не написали ни единой строки кода, а данные MQTT уже поступают из EMQX Cloud в ClickHouse Cloud. С EMQX Cloud и ClickHouse Cloud вам не нужно управлять инфраструктурой — вы можете сосредоточиться на разработке IoT‑приложений, пока данные надёжно хранятся в ClickHouse Cloud. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md index fb80c5ac0cf..c01cd2bab10 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md @@ -25,7 +25,7 @@ import airbyte09 from '@site/static/images/integrations/data-ingestion/etl-tools import PartnerBadge from '@theme/badges/PartnerBadge'; -# Подключение Airbyte к ClickHouse +# Подключение Airbyte к ClickHouse {#connect-airbyte-to-clickhouse} @@ -72,7 +72,7 @@ import PartnerBadge from '@theme/badges/PartnerBadge'; -## Добавьте ClickHouse в качестве приемника данных +## Добавьте ClickHouse в качестве приемника данных {#2-add-clickhouse-as-a-destination} В этом разделе мы покажем, как добавить экземпляр ClickHouse в качестве приемника данных. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md index fb9cf050ab2..f3ccbdf96a3 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md @@ -13,7 +13,7 @@ keywords: ['apache beam', 'потоковая обработка', 'пакетн import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Интеграция Apache Beam и ClickHouse +# Интеграция Apache Beam и ClickHouse {#integrating-apache-beam-and-clickhouse} @@ -29,9 +29,9 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## Настройка пакета ClickHouse для Apache Beam +## Настройка пакета ClickHouse для Apache Beam {#setup-of-the-apache-beam-clickhouse-package} -### Установка пакета +### Установка пакета {#package-installation} Добавьте следующую зависимость в используемую систему управления пакетами: @@ -50,7 +50,7 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; Артефакты можно найти в [официальном репозитории Maven](https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-clickhouse). -### Пример кода +### Пример кода {#code-example} Следующий пример считывает CSV‑файл с именем `input.csv` как коллекцию `PCollection`, преобразует его в объект `Row` (используя определённую схему) и вставляет в локальный экземпляр ClickHouse с помощью `ClickHouseIO`: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md index 6048ae1c2ca..188cd0480b1 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md @@ -21,7 +21,7 @@ import bp_ck_9 from '@site/static/images/integrations/data-ingestion/etl-tools/b import PartnerBadge from '@theme/badges/PartnerBadge'; -# Подключение BladePipe к ClickHouse +# Подключение BladePipe к ClickHouse {#connect-bladepipe-to-clickhouse} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md index 7220213d65a..9667959ac4b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md @@ -12,7 +12,7 @@ import TOCInline from '@theme/TOCInline'; import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Возможности и конфигурации +# Возможности и конфигурации {#features-and-configurations} @@ -22,7 +22,7 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## Конфигурация Profile.yml +## Конфигурация Profile.yml {#profile-yml-configurations} Чтобы подключиться к ClickHouse из dbt, вам потребуется добавить [профиль](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles) в файл `profiles.yml`. Профиль ClickHouse должен соответствовать следующему синтаксису: @@ -65,14 +65,14 @@ your_profile_name: compress_block_size: [1048576] # Размер блока сжатия, если сжатие включено ``` -### Схема и база данных +### Схема и база данных {#schema-vs-database} Идентификатор отношения модели dbt `database.schema.table` не совместим с ClickHouse, поскольку ClickHouse не поддерживает `schema`. Поэтому используется упрощённый вариант `schema.table`, где `schema` — это база данных ClickHouse. Использование базы данных `default` не рекомендуется. -### Предупреждение об операторе SET +### Предупреждение об операторе SET {#set-statement-warning} Во многих средах использование оператора SET для сохранения настройки ClickHouse, применяемой ко всем запросам dbt, ненадёжно и может приводить к неожиданным сбоям. Это особенно актуально при использовании HTTP‑подключений через балансировщик нагрузки, @@ -81,7 +81,7 @@ your_profile_name: в свойстве "custom_settings" профиля dbt как рекомендуемую практику, вместо того чтобы полагаться на оператор "SET" в pre-hook, как иногда предлагается. -### Настройка `quote_columns` +### Настройка `quote_columns` {#setting-quote_columns} Чтобы избежать предупреждения, обязательно явно задайте значение параметра `quote_columns` в файле `dbt_project.yml`. Дополнительную информацию смотрите в [документации по quote_columns](https://docs.getdbt.com/reference/resource-configs/quote_columns). @@ -91,14 +91,14 @@ seeds: +quote_columns: false #или `true`, если в заголовках столбцов CSV есть пробелы ``` -### О кластере ClickHouse +### О кластере ClickHouse {#about-the-clickhouse-cluster} При использовании кластера ClickHouse нужно учитывать две вещи: * Установку настройки `cluster`. * Обеспечение согласованности чтения после записи (read-after-write), особенно если вы используете более одного потока (`threads`). -#### Настройка кластера +#### Настройка кластера {#cluster-setting} Настройка `cluster` в профиле позволяет dbt-clickhouse работать с кластером ClickHouse. Если `cluster` задан в профиле, по умолчанию **все модели будут создаваться с оператором `ON CLUSTER`**, за исключением моделей, использующих движок **Replicated**. К ним относятся: @@ -129,7 +129,7 @@ seeds: Если модель была создана без настройки `cluster`, dbt-clickhouse обнаружит это и выполнит все DDL/DML без предложения `on cluster` для этой модели. -#### Согласованность чтения после записи (read-after-write) +#### Согласованность чтения после записи (read-after-write) {#read-after-write-consistency} dbt полагается на модель согласованности чтения после вставки (read-after-insert). Это несовместимо с кластерами ClickHouse с более чем одной репликой, если вы не можете гарантировать, что все операции будут направляться на одну и ту же реплику. В повседневной работе с dbt вы можете не столкнуться с проблемами, но в зависимости от конфигурации кластера есть несколько стратегий, позволяющих обеспечить такую гарантию: @@ -137,9 +137,9 @@ dbt полагается на модель согласованности чте * Если вы используете кластер с самостоятельным размещением (self-hosted), убедитесь, что все запросы dbt отправляются на одну и ту же реплику ClickHouse. Если поверх него есть балансировщик нагрузки, попробуйте использовать механизм `replica aware routing`/`sticky sessions`, чтобы всегда попадать на одну и ту же реплику. Добавление настройки `select_sequential_consistency = 1` в кластерах вне ClickHouse Cloud [не рекомендуется](https://clickhouse.com/docs/operations/settings/settings#select_sequential_consistency). -## Общая информация о возможностях +## Общая информация о возможностях {#general-information-about-features} -### Общие конфигурации таблиц +### Общие конфигурации таблиц {#general-table-configurations} | Option | Description | Default if any | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | @@ -157,7 +157,7 @@ dbt полагается на модель согласованности чте | definer | Если `sql_security` установлено в значение `definer`, необходимо указать любого существующего пользователя или `CURRENT_USER` в предложении `definer`. | | | projections | Список [проекций](/data-modeling/projections), которые будут созданы. Подробности см. в разделе [О проекциях](#projections). | | -#### О data skipping индексах +#### О data skipping индексах {#data-skipping-indexes} Data skipping индексы доступны только для материализации `table`. Чтобы добавить список data skipping индексов в таблицу, используйте конфигурацию `indexes`: @@ -171,7 +171,7 @@ Data skipping индексы доступны только для материа ) }} ``` -#### О проекциях +#### О проекциях {#projections} Вы можете добавить [проекции](/data-modeling/projections) к материализациям типов `table` и `distributed_table` с помощью конфигурации `projections`: @@ -189,7 +189,7 @@ Data skipping индексы доступны только для материа **Примечание**: Для распределённых таблиц проекция применяется к таблицам `_local`, а не к распределённой прокси-таблице. -### Поддерживаемые движки таблиц +### Поддерживаемые движки таблиц {#supported-table-engines} | Тип | Подробности | | ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -200,7 +200,7 @@ Data skipping индексы доступны только для материа | EmbeddedRocksDB | [https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb](https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb) | | Hive | [https://clickhouse.com/docs/en/engines/table-engines/integrations/hive](https://clickhouse.com/docs/en/engines/table-engines/integrations/hive) | -### Поддерживаемые экспериментальные движки таблиц +### Поддерживаемые экспериментальные движки таблиц {#experimental-supported-table-engines} | Тип | Подробности | @@ -211,7 +211,7 @@ Data skipping индексы доступны только для материа Если вы столкнётесь с проблемами при подключении к ClickHouse из dbt с одним из вышеуказанных движков, пожалуйста, сообщите о проблеме [здесь](https://github.com/ClickHouse/dbt-clickhouse/issues). -### Примечание о настройках моделей +### Примечание о настройках моделей {#a-note-on-model-settings} В ClickHouse есть несколько типов/уровней «настроек». В конфигурации модели выше настраиваются два их типа. `settings` означает секцию `SETTINGS`, @@ -224,7 +224,7 @@ Data skipping индексы доступны только для материа доступны в таблице `system.settings`). В целом рекомендуются настройки по умолчанию, а любое использование этих свойств следует тщательно исследовать и протестировать. -### Конфигурация столбцов +### Конфигурация столбцов {#column-configuration} > ***ПРИМЕЧАНИЕ:*** Приведённые ниже параметры конфигурации столбцов требуют применения [контрактов моделей](https://docs.getdbt.com/docs/collaborate/govern/model-contracts). @@ -233,7 +233,7 @@ Data skipping индексы доступны только для материа | codec | Строка, состоящая из аргументов, передаваемых в `CODEC()` в DDL-описании столбца. Например: `codec: "Delta, ZSTD"` будет скомпилирована в выражение `CODEC(Delta, ZSTD)`. | | | ttl | Строка, состоящая из [TTL-выражения (time-to-live)](https://clickhouse.com/docs/guides/developer/ttl), которое определяет TTL-правило в DDL-описании столбца. Например: `ttl: ts + INTERVAL 1 DAY` будет скомпилирована в выражение `TTL ts + INTERVAL 1 DAY`. | | -#### Пример конфигурации схемы +#### Пример конфигурации схемы {#example-of-schema-configuration} ```yaml models: @@ -251,7 +251,7 @@ models: ttl: ts + INTERVAL 1 DAY ``` -#### Добавление сложных типов данных +#### Добавление сложных типов данных {#adding-complex-types} dbt автоматически определяет тип данных каждого столбца, анализируя SQL, используемый для создания модели. Однако в некоторых случаях этот процесс может некорректно определить тип данных, что приводит к конфликтам с типами, указанными в свойстве контракта `data_type`. Чтобы избежать этого, рекомендуется использовать функцию `CAST()` в SQL-коде модели для явного указания требуемого типа. Например: @@ -276,9 +276,9 @@ group by event_type ``` -## Возможности +## Возможности {#features} -### Материализация: view +### Материализация: view {#materialization-view} Модель dbt может быть создана как [представление ClickHouse](https://clickhouse.com/docs/en/sql-reference/table-functions/view/) и настроена с использованием следующего синтаксиса: @@ -297,7 +297,7 @@ models: {{ config(materialized = "view") }} ``` -### Материализация: таблица +### Материализация: таблица {#materialization-table} Модель dbt может быть создана как [таблица ClickHouse](https://clickhouse.com/docs/en/operations/system-tables/tables/) и настроена с использованием следующего синтаксиса: @@ -326,7 +326,7 @@ models: ) }} ``` -### Материализация: incremental +### Материализация: incremental {#materialization-incremental} Модель таблицы будет пересоздаваться при каждом выполнении dbt. Это может быть неосуществимо и крайне затратно для больших наборов данных или сложных трансформаций. Чтобы решить эту проблему и сократить время сборки, модель dbt может быть создана как инкрементальная таблица ClickHouse и настраивается с помощью следующего синтаксиса: @@ -358,7 +358,7 @@ models: ) }} ``` -#### Конфигурации +#### Конфигурации {#configurations} Конфигурации, специфические для этого типа материализации, перечислены ниже: @@ -369,11 +369,11 @@ models: | `incremental_strategy` | Стратегия, используемая для инкрементальной материализации. Поддерживаются `delete+insert`, `append`, `insert_overwrite` или `microbatch`. Дополнительные сведения о стратегиях см. [здесь](/integrations/dbt/features-and-configurations#incremental-model-strategies). | Необязателен (по умолчанию: 'default') | | `incremental_predicates` | Дополнительные условия, которые будут применяться к инкрементальной материализации (применяются только для стратегии `delete+insert`). | Необязателен | -#### Стратегии инкрементальных моделей +#### Стратегии инкрементальных моделей {#incremental-model-strategies} `dbt-clickhouse` поддерживает три стратегии инкрементальных моделей. -##### Стратегия по умолчанию (устаревшая) +##### Стратегия по умолчанию (устаревшая) {#default-legacy-strategy} Исторически ClickHouse имел лишь ограниченную поддержку операций обновления и удаления в форме асинхронных «мутаций». Чтобы эмулировать ожидаемое поведение dbt, @@ -384,7 +384,7 @@ dbt-clickhouse по умолчанию создаёт новую временн идёт не так до завершения операции; однако, поскольку она включает полное копирование исходной таблицы, её выполнение может быть дорогим и медленным. -##### Стратегия Delete+Insert +##### Стратегия Delete+Insert {#delete-insert-strategy} ClickHouse добавил «облегчённые удаления» (lightweight deletes) как экспериментальную возможность в версии 22.8. Облегчённые удаления значительно @@ -466,7 +466,7 @@ ClickHouse добавил «облегчённые удаления» (lightweig Для этой стратегии требуется, чтобы `partition_by` был задан в конфигурации модели. Все остальные параметры конфигурации модели, относящиеся к стратегиям, игнорируются. -### Materialization: materialized_view (экспериментально) +### Materialization: materialized_view (экспериментально) {#materialized-view} Материализация `materialized_view` должна представлять собой запрос `SELECT` из существующей (исходной) таблицы. Адаптер создаст целевую таблицу с именем модели @@ -501,7 +501,7 @@ select a,b,c from {{ source('raw', 'table_2') }} > вы получите следующее предупреждение: > `Warning - Table was detected with the same pattern as model name but was not found in this run. In case it is a renamed mv that was previously part of this model, drop it manually (!!!) ` -#### Догрузка данных +#### Догрузка данных {#data-catch-up} В настоящее время при создании материализованного представления (MV) целевая таблица сначала заполняется историческими данными, и только затем создается само MV. @@ -518,7 +518,7 @@ select a,b,c from {{ source('raw', 'table_2') }} )}} ``` -#### Обновляемые материализованные представления +#### Обновляемые материализованные представления {#refreshable-materialized-views} Чтобы использовать [Refreshable Materialized View](https://clickhouse.com/docs/en/materialized-view/refreshable-materialized-view), при необходимости скорректируйте следующие параметры в вашей MV‑модели (все эти параметры должны быть заданы внутри @@ -550,7 +550,7 @@ select a,b,c from {{ source('raw', 'table_2') }} }} ``` -#### Ограничения +#### Ограничения {#limitations} * При создании обновляемого материализованного представления (MV) в ClickHouse, которое имеет зависимость, ClickHouse не выдаёт ошибку, если указанная зависимость не существует на момент создания. Вместо этого обновляемое MV остаётся в @@ -563,14 +563,14 @@ select a,b,c from {{ source('raw', 'table_2') }} гарантируется. * Функциональность обновляемости не тестировалась с несколькими mv, направляющими данные в одну и ту же целевую модель. -### Материализация: dictionary (экспериментальная) +### Материализация: dictionary (экспериментальная) {#materialization-dictionary} См. тесты в [https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py](https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py) для примеров того, как реализовывать материализации для словарей ClickHouse. -### Материализация: distributed_table (экспериментальная) +### Материализация: distributed_table (экспериментальная) {#materialization-distributed-table} Распределённая таблица создаётся следующими шагами: @@ -585,7 +585,7 @@ select a,b,c from {{ source('raw', 'table_2') }} корректное выполнение последующих операций инкрементальной материализации. Это может привести к тому, что некоторые вставки в распределённые таблицы будут выполняться медленнее, чем ожидается. -#### Пример модели распределённой таблицы +#### Пример модели распределённой таблицы {#distributed-table-model-example} ```sql {{ @@ -601,7 +601,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### Сгенерированные миграции +#### Сгенерированные миграции {#distributed-table-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -621,7 +621,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### materialization: distributed_incremental (экспериментальная) +### materialization: distributed_incremental (экспериментальная) {#materialization-distributed-incremental} Инкрементальная модель, основанная на той же идее, что и распределённая таблица; основная сложность — корректно обрабатывать все стратегии инкрементального обновления. @@ -632,7 +632,7 @@ CREATE TABLE db.table on cluster cluster ( Заменяются только таблицы шардов, потому что распределённая таблица не хранит данные. Распределённая таблица пересоздаётся только при включённом режиме full_refresh или если структура таблицы могла измениться. -#### Пример распределённой инкрементальной модели +#### Пример распределённой инкрементальной модели {#distributed-incremental-model-example} ```sql @@ -649,7 +649,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### Сгенерированные миграции +#### Сгенерированные миграции {#distributed-incremental-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -668,7 +668,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### Snapshot +### Snapshot {#snapshot} Снимки dbt позволяют фиксировать изменения изменяемой модели со временем. В свою очередь, это позволяет выполнять запросы к моделям в разрезе конкретного момента времени, когда аналитики могут «заглянуть в прошлое» и увидеть предыдущее состояние модели. Эта функциональность поддерживается коннектором ClickHouse и настраивается с помощью следующего синтаксиса: @@ -687,7 +687,7 @@ CREATE TABLE db.table on cluster cluster ( Для получения дополнительной информации о конфигурации см. справочную страницу [snapshot configs](https://docs.getdbt.com/docs/build/snapshots#snapshot-configs). -### Контракты и ограничения +### Контракты и ограничения {#contracts-and-constraints} Поддерживаются только контракты с точным совпадением типов столбцов. Например, контракт с типом столбца UInt32 завершится с ошибкой, если модель вернёт UInt64 или другой целочисленный тип. @@ -695,9 +695,9 @@ ClickHouse также поддерживает *только* ограничен ограничения CHECK на уровне отдельных столбцов не поддерживаются. (См. документацию ClickHouse по первичным ключам/ключам ORDER BY.) -### Дополнительные макросы ClickHouse +### Дополнительные макросы ClickHouse {#additional-clickhouse-macros} -#### Вспомогательные макросы материализации моделей +#### Вспомогательные макросы материализации моделей {#model-materialization-utility-macros} Следующие макросы включены для упрощения создания специфичных для ClickHouse таблиц и представлений: @@ -714,7 +714,7 @@ ClickHouse также поддерживает *только* ограничен * `ttl_config` -- использует свойство конфигурации модели `ttl` для назначения выражения TTL таблицы ClickHouse. По умолчанию TTL не назначен. -#### Вспомогательный макрос s3Source +#### Вспомогательный макрос s3Source {#s3source-helper-macro} Макрос `s3source` упрощает процесс выборки данных ClickHouse непосредственно из S3 с помощью табличной функции ClickHouse S3. Он работает за счёт diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md index 649eb95cf67..ffddbfc0a79 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md @@ -20,7 +20,7 @@ import dbt_07 from '@site/static/images/integrations/data-ingestion/etl-tools/db import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Руководства +# Руководства {#guides} @@ -39,13 +39,13 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## Настройка +## Настройка {#setup} Следуйте инструкциям из раздела [Настройка dbt и адаптера ClickHouse](/integrations/dbt), чтобы подготовить ваше окружение. **Важно: приведённые ниже инструкции протестированы с Python 3.9.** -### Подготовка ClickHouse +### Подготовка ClickHouse {#prepare-clickhouse} dbt особенно эффективен при моделировании сильно связанных реляционных данных. В качестве примера мы предоставляем небольшой набор данных IMDB со следующей реляционной схемой. Этот набор данных взят из [репозитория реляционных наборов данных](https://relational.fit.cvut.cz/dataset/IMDb). Он тривиален по сравнению с типичными схемами, используемыми с dbt, но является удобным для работы примером: @@ -672,7 +672,7 @@ SELECT * FROM imdb_dbt.actor_summary ORDER BY num_movies DESC LIMIT 2; +------+-------------------+----------+------------------+------+---------+-------------------+ ``` -### Внутреннее устройство +### Внутреннее устройство {#internals} Мы можем определить, какие запросы выполнялись для реализации описанного выше инкрементального обновления, обратившись к журналу запросов ClickHouse. @@ -694,7 +694,7 @@ AND event_time > subtractMinutes(now(), 15) ORDER BY event_time LIMIT 100; Такая стратегия может создавать сложности на очень больших моделях. Для получения дополнительной информации см. раздел [Limitations](/integrations/dbt#limitations). -### Стратегия append (режим только вставки) +### Стратегия append (режим только вставки) {#append-strategy-inserts-only-mode} Чтобы обойти ограничения, связанные с большими наборами данных в инкрементальных моделях, адаптер использует параметр конфигурации dbt `incremental_strategy`. Его можно установить в значение `append`. В этом случае обновлённые строки вставляются непосредственно в целевую таблицу (т.е. `imdb_dbt.actor_summary`), и временная таблица не создаётся. Примечание: режим только append требует, чтобы ваши данные были неизменяемыми или чтобы дубликаты были допустимы. Если вам нужна инкрементальная модель таблицы, поддерживающая изменение строк, не используйте этот режим! @@ -796,7 +796,7 @@ WHERE id > (SELECT max(id) FROM imdb_dbt.actor_summary) OR updated_at > (SELECT В этом прогоне непосредственно в таблицу `imdb_dbt.actor_summary` добавляются только новые строки, при этом создание таблицы не выполняется. -### Режим удаления и вставки (экспериментальный) +### Режим удаления и вставки (экспериментальный) {#deleteinsert-mode-experimental} Традиционно ClickHouse имел лишь ограниченную поддержку операций обновления и удаления в виде асинхронных [Mutations](/sql-reference/statements/alter/index.md). Эти операции могут быть крайне ресурсоёмкими по вводу-выводу, и, как правило, их следует избегать. @@ -821,7 +821,7 @@ WHERE id > (SELECT max(id) FROM imdb_dbt.actor_summary) OR updated_at > (SELECT -### режим insert_overwrite (экспериментальный) +### режим insert_overwrite (экспериментальный) {#insert_overwrite-mode-experimental} Выполняет следующие шаги: @@ -840,7 +840,7 @@ WHERE id > (SELECT max(id) FROM imdb_dbt.actor_summary) OR updated_at > (SELECT -## Создание снимка +## Создание снимка {#creating-a-snapshot} Снимки dbt позволяют зафиксировать изменения изменяемой модели во времени. Это, в свою очередь, позволяет выполнять запросы к моделям на состояние в конкретный момент времени, когда аналитики могут «оглядываться назад» и просматривать предыдущее состояние модели. Это достигается с помощью [измерений типа 2 (Slowly Changing Dimensions)](https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row), где столбцы from и to фиксируют период, в течение которого строка считалась актуальной. Эта функциональность поддерживается адаптером ClickHouse и продемонстрирована ниже. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md index 160aa72e408..312be5a3a71 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md @@ -95,9 +95,9 @@ dbt предоставляет 4 типа материализации: -## Настройка dbt и адаптера ClickHouse +## Настройка dbt и адаптера ClickHouse {#setup-of-dbt-and-the-clickhouse-adapter} -### Установка dbt-core и dbt-clickhouse +### Установка dbt-core и dbt-clickhouse {#install-dbt-core-and-dbt-clickhouse} dbt предоставляет несколько способов установки интерфейса командной строки (CLI), которые подробно описаны [здесь](https://docs.getdbt.com/dbt-cli/install/overview). Мы рекомендуем использовать `pip` для установки как dbt, так и dbt-clickhouse. @@ -105,7 +105,7 @@ dbt предоставляет несколько способов устано pip install dbt-core dbt-clickhouse ``` -### Укажите в dbt параметры подключения к нашему экземпляру ClickHouse. +### Укажите в dbt параметры подключения к нашему экземпляру ClickHouse. {#provide-dbt-with-the-connection-details-for-our-clickhouse-instance} Настройте профиль `clickhouse-service` в файле `~/.dbt/profiles.yml` и задайте параметры schema, host, port, user и password. Полный список вариантов конфигурации подключения доступен на странице [Возможности и параметры конфигурации](/integrations/dbt/features-and-configurations): @@ -125,7 +125,7 @@ clickhouse-service: secure: True # Use TLS (native protocol) or HTTPS (http protocol) ``` -### Создайте проект dbt +### Создайте проект dbt {#create-a-dbt-project} Теперь вы можете использовать этот профиль в одном из существующих проектов или создать новый с помощью: @@ -139,17 +139,17 @@ dbt init project_name profile: 'clickhouse-service' ``` -### Тестирование соединения +### Тестирование соединения {#test-connection} Выполните `dbt debug` с помощью инструмента командной строки (CLI), чтобы проверить, может ли dbt подключиться к ClickHouse. Убедитесь, что в ответе присутствует строка `Connection test: [OK connection ok]`, означающая успешное соединение. Перейдите на [страницу руководств](/integrations/dbt/guides), чтобы узнать больше о том, как использовать dbt с ClickHouse. -### Тестирование и развертывание моделей (CI/CD) +### Тестирование и развертывание моделей (CI/CD) {#testing-and-deploying-your-models-ci-cd} Существует множество способов тестировать и развертывать ваш dbt‑проект. У dbt есть рекомендации по [рекомендуемым рабочим процессам](https://docs.getdbt.com/best-practices/best-practice-workflows#pro-tips-for-workflows) и [CI‑заданиям](https://docs.getdbt.com/docs/deploy/ci-jobs). Мы рассмотрим несколько стратегий, но имейте в виду, что их может потребоваться существенно адаптировать под ваш конкретный сценарий. -#### CI/CD с простыми тестами данных и модульными тестами +#### CI/CD с простыми тестами данных и модульными тестами {#ci-with-simple-data-tests-and-unit-tests} Один из простых способов запустить ваш CI‑конвейер — развернуть кластер ClickHouse внутри задания и запускать ваши модели на нём. Перед запуском моделей вы можете загрузить демонстрационные данные в этот кластер. Можно просто использовать [seed](https://docs.getdbt.com/reference/commands/seed), чтобы наполнить промежуточную среду подмножеством ваших боевых данных. @@ -157,7 +157,7 @@ profile: 'clickhouse-service' Шаг CD может быть столь же простым, как запуск `dbt build` для вашего боевого кластера ClickHouse. -#### Более полный этап CI/CD: использование свежих данных, тестирование только затронутых моделей +#### Более полный этап CI/CD: использование свежих данных, тестирование только затронутых моделей {#more-complete-ci-stage} Одна из распространённых стратегий — использовать задания [Slim CI](https://docs.getdbt.com/best-practices/best-practice-workflows#run-only-modified-models-to-test-changes-slim-ci), в которых повторно разворачиваются только изменённые модели (и их зависимости вверх и вниз по потоку). Этот подход использует артефакты ваших боевых прогонов (например, [dbt manifest](https://docs.getdbt.com/reference/artifacts/manifest-json)), чтобы сократить время выполнения проекта и гарантировать отсутствие расхождений схем между средами. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md index 4d80f22edb9..d642e695838 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md @@ -10,7 +10,7 @@ doc_type: 'guide' import PartnerBadge from '@theme/badges/PartnerBadge'; -# Подключение dlt к ClickHouse +# Подключение dlt к ClickHouse {#connect-dlt-to-clickhouse} @@ -18,9 +18,9 @@ import PartnerBadge from '@theme/badges/PartnerBadge'; -## Установка dlt с ClickHouse +## Установка dlt с ClickHouse {#install-dlt-with-clickhouse} -### Установите библиотеку `dlt` с зависимостями для ClickHouse: +### Установите библиотеку `dlt` с зависимостями для ClickHouse: {#to-install-the-dlt-library-with-clickhouse-dependencies} ```bash pip install "dlt[clickhouse]" @@ -99,7 +99,7 @@ dataset_table_separator = "___" # Разделитель для имё ```bash -# разместите это в начале вашего toml-файла, перед началом любых секций. +# разместите это в начале вашего toml-файла, перед началом любых секций. {#keep-it-at-the-top-of-your-toml-file-before-any-section-starts} destination.clickhouse.credentials="clickhouse://dlt:Dlt*12345789234567@localhost:9000/dlt?secure=1" ``` @@ -156,7 +156,7 @@ ClickHouse поддерживает следующие табличной функции GCS ClickHouse, которую dlt использует под капотом. @@ -240,10 +240,10 @@ dlt передаст эти учетные данные в ClickHouse, кото * Обеспечить работу файлового назначения с GCS в режиме совместимости с S3 * Поддержка области промежуточного размещения Google Cloud Storage -### Поддержка dbt +### Поддержка dbt {#dbt-support} Интеграция с dbt в целом поддерживается через dbt-clickhouse. -### Синхронизация состояния `dlt` +### Синхронизация состояния `dlt` {#syncing-of-dlt-state} Это назначение полностью поддерживает синхронизацию состояния dlt. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md index 5b36309f52f..9e0045831f5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md @@ -14,7 +14,7 @@ keywords: ['fivetran', 'перемещение данных', 'etl', 'ClickHouse import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Fivetran и ClickHouse Cloud +# Fivetran и ClickHouse Cloud {#fivetran-and-clickhouse-cloud} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md index 8db1dd93213..1398184be7d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md @@ -11,7 +11,7 @@ integration: - category: 'data_ingestion' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import nifi01 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_01.png'; import nifi02 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_02.png'; @@ -31,7 +31,7 @@ import nifi15 from '@site/static/images/integrations/data-ingestion/etl-tools/ni import CommunityMaintainedBadge from '@theme/badges/CommunityMaintained'; -# Подключение Apache NiFi к ClickHouse +# Подключение Apache NiFi к ClickHouse {#connect-apache-nifi-to-clickhouse} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md index 86c85ff1d07..20c56ada410 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md @@ -10,7 +10,7 @@ integration: - support_level: 'partner' - category: 'data_ingestion' - website: 'https://vector.dev/' -keywords: ['vector', 'сбор логов', 'Обзервабилити', 'ингестия данных', 'конвейер'] +keywords: ['vector', 'сбор логов', 'наблюдаемость', 'ингестия данных', 'конвейер'] --- import Image from '@theme/IdealImage'; @@ -18,8 +18,7 @@ import vector01 from '@site/static/images/integrations/data-ingestion/etl-tools/ import vector02 from '@site/static/images/integrations/data-ingestion/etl-tools/vector_02.png'; import PartnerBadge from '@theme/badges/PartnerBadge'; - -# Integrating Vector with ClickHouse +# Integrating Vector with ClickHouse {#integrating-vector-with-clickhouse} @@ -32,214 +31,202 @@ ClickHouse превосходно справляется с хранением **Предварительные требования:** -- У вас уже установлен и запущен ClickHouse -- У вас установлен Vector +* У вас уже установлен и запущен ClickHouse +* У вас установлен Vector + ## Создайте базу данных и таблицу {#1-create-a-database-and-table} + Создайте таблицу для хранения событий логов: -## Создайте базу данных и таблицу - -Создайте таблицу для хранения событий логов: - -1. Начните с новой базы данных с именем `nginxdb`: - -```sql -CREATE DATABASE IF NOT EXISTS nginxdb -``` - -2. Вставьте всё событие лога одной строкой. Очевидно, что это не лучший формат для аналитики по данным логов, но ниже мы разберёмся с этим, используя ***материализованные представления***. - -```sql -CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( - message String -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -:::note -**ORDER BY** установлен в значение **tuple()** (пустой кортеж), так как пока нет необходимости задавать первичный ключ. -::: - - -## Настройка Nginx - -На этом шаге будет показано, как настроить логирование Nginx. - -1. Следующее свойство `access_log` отправляет логи в `/var/log/nginx/my_access.log` в формате **combined**. - Это значение указывается в секции `http` файла `nginx.conf`: - -```bash -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - access_log /var/log/nginx/my_access.log combined; - sendfile on; - keepalive_timeout 65; - include /etc/nginx/conf.d/*.conf; -} -``` - -2. Обязательно перезапустите Nginx, если вам пришлось изменить `nginx.conf`. - -3. Сгенерируйте несколько записей в журнале доступа, посетив страницы на вашем веб-сервере. - Логи в формате **combined** выглядят следующим образом: - -```bash -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - + 1. Начните с новой базы данных с именем `nginxdb`: -## Настройка Vector + ```sql + CREATE DATABASE IF NOT EXISTS nginxdb + ``` -Vector собирает, преобразует и маршрутизирует логи, метрики и трейсы (далее — **sources**) в различные системы/клиентов (далее — **sinks**), включая поддержку ClickHouse «из коробки». -Sources и sinks задаются в конфигурационном файле **vector.toml**. + 2. Вставьте всё событие лога одной строкой. Очевидно, что это не лучший формат для аналитики по данным логов, но ниже мы разберёмся с этим, используя ***материализованные представления***. -1. Следующий файл **vector.toml** определяет **source** типа **file**, который отслеживает (tail) конец файла **my_access.log**, а также **sink**, использующий таблицу **access_logs**, описанную выше: + ```sql + CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( + message String + ) + ENGINE = MergeTree() + ORDER BY tuple() + ``` -```bash -[sources.nginx_logs] -type = "file" -include = [ "/var/log/nginx/my_access.log" ] -read_from = "end" + :::note + **ORDER BY** установлен в значение **tuple()** (пустой кортеж), так как пока нет необходимости задавать первичный ключ. + ::: -[sinks.clickhouse] -type = "clickhouse" -inputs = ["nginx_logs"] -endpoint = "http://clickhouse-server:8123" -database = "nginxdb" -table = "access_logs" -skip_unknown_fields = true -``` + ## Настройка Nginx {#2--configure-nginx} -2. Запустите Vector, используя приведённую выше конфигурацию. Обратитесь к [документации Vector](https://vector.dev/docs/) для получения дополнительных сведений об определении источников и приёмников. + На этом шаге будет показано, как настроить логирование Nginx. -3. Убедитесь, что журналы доступа записываются в ClickHouse, выполнив следующий запрос. В вашей таблице должны отобразиться журналы доступа: + 1. Следующее свойство `access_log` отправляет логи в `/var/log/nginx/my_access.log` в формате **combined**. + Это значение указывается в секции `http` файла `nginx.conf`: -```sql -SELECT * FROM nginxdb.access_logs -``` + ```bash + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + access_log /var/log/nginx/my_access.log combined; + sendfile on; + keepalive_timeout 65; + include /etc/nginx/conf.d/*.conf; + } + ``` - + 2. Обязательно перезапустите Nginx, если вам пришлось изменить `nginx.conf`. + 3. Сгенерируйте несколько записей в журнале доступа, посетив страницы на вашем веб-сервере. + Логи в формате **combined** выглядят следующим образом: -## Разбор логов + ```bash + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` -Хранить логи в ClickHouse полезно, но сохранение каждого события в виде одной строки не дает больших возможностей для анализа данных. -Далее мы рассмотрим, как разбирать события логов с помощью [материализованного представления](/materialized-view/incremental-materialized-view). + ## Настройка Vector {#3-configure-vector} -**Материализованное представление** работает подобно триггеру на INSERT в SQL. Когда строки данных вставляются в исходную таблицу, материализованное представление выполняет над ними некоторые преобразования и вставляет результаты в целевую таблицу. -Материализованное представление можно настроить так, чтобы формировать разобранное представление событий логов в **access_logs**. -Ниже приведен пример одного такого события лога: + Vector собирает, преобразует и маршрутизирует логи, метрики и трейсы (далее — **sources**) в различные системы/клиентов (далее — **sinks**), включая поддержку ClickHouse «из коробки». + Sources и sinks задаются в конфигурационном файле **vector.toml**. -```bash -192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - -В ClickHouse есть различные функции для разбора приведённой выше строки. Функция [`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) разбирает строку по пробельным символам и возвращает каждый токен в виде элемента массива. -Для демонстрации выполните следующую команду: - -```sql title="Query" -SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -```text title="Response" -["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] -``` - -В некоторых строках есть лишние символы, а user agent (сведения о браузере) не требуется парсить, но -получившийся массив близок к нужному. - -Аналогично функции `splitByWhitespace`, функция [`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) разбивает строку на массив по регулярному выражению. -Выполните следующую команду, которая вернёт две строки. - -```sql -SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -Обратите внимание, что вторая возвращаемая строка — это User-Agent, успешно разобранный из лога: - -```text -["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] -``` - -Прежде чем перейти к итоговой команде `CREATE MATERIALIZED VIEW`, давайте рассмотрим ещё пару функций, используемых для очистки данных. -Например, поле `RequestMethod` имеет значение `"GET`, то есть содержит лишнюю двойную кавычку. -Вы можете использовать функцию [`trimBoth` (псевдоним `trim`)](/sql-reference/functions/string-functions#trimBoth), чтобы удалить двойную кавычку: - -```sql -SELECT trim(LEADING '"' FROM '"GET') -``` - -Строка с датой начинается с квадратной скобки и при этом имеет формат, который ClickHouse не может разобрать как дату. -Однако, если мы заменим разделитель с двоеточия (**:**) на запятую (**,**), разбор уже работает отлично: - -```sql -SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) -``` - - -Теперь мы готовы определить материализованное представление. -Определение ниже включает `POPULATE`, что означает, что существующие строки в **access_logs** будут обработаны и вставлены сразу же. -Выполните следующую SQL-команду: - -```sql -CREATE MATERIALIZED VIEW nginxdb.access_logs_view -( - RemoteAddr String, - Client String, - RemoteUser String, - TimeLocal DateTime, - RequestMethod String, - Request String, - HttpVersion String, - Status Int32, - BytesSent Int64, - UserAgent String -) -ENGINE = MergeTree() -ORDER BY RemoteAddr -POPULATE AS -WITH - splitByWhitespace(message) as split, - splitByRegexp('\S \d+ "([^"]*)"', message) as referer -SELECT - split[1] AS RemoteAddr, - split[2] AS Client, - split[3] AS RemoteUser, - parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, - trim(LEADING '"' FROM split[6]) AS RequestMethod, - split[7] AS Request, - trim(TRAILING '"' FROM split[8]) AS HttpVersion, - split[9] AS Status, - split[10] AS BytesSent, - trim(BOTH '"' from referer[2]) AS UserAgent -FROM - (SELECT message FROM nginxdb.access_logs) -``` - -Теперь убедитесь, что всё работает. -Вы должны увидеть журналы доступа, корректно разобранные по столбцам: - -```sql -SELECT * FROM nginxdb.access_logs_view -``` - - - -:::note -В приведённом выше примере данные сохраняются в двух таблицах, но вы можете изменить исходную таблицу `nginxdb.access_logs`, чтобы использовать движок таблиц [`Null`](/engines/table-engines/special/null). -Разобранные данные по-прежнему будут попадать в таблицу `nginxdb.access_logs_view`, но исходные данные не будут сохраняться в таблице. -::: + 1. Следующий файл **vector.toml** определяет **source** типа **file**, который отслеживает (tail) конец файла **my_access.log**, а также **sink**, использующий таблицу **access_logs**, описанную выше: + ```bash + [sources.nginx_logs] + type = "file" + include = [ "/var/log/nginx/my_access.log" ] + read_from = "end" + + [sinks.clickhouse] + type = "clickhouse" + inputs = ["nginx_logs"] + endpoint = "http://clickhouse-server:8123" + database = "nginxdb" + table = "access_logs" + skip_unknown_fields = true + ``` + + 2. Запустите Vector, используя приведённую выше конфигурацию. Обратитесь к [документации Vector](https://vector.dev/docs/) для получения дополнительных сведений об определении источников и приёмников. + + 3. Убедитесь, что журналы доступа записываются в ClickHouse, выполнив следующий запрос. В вашей таблице должны отобразиться журналы доступа: + + ```sql + SELECT * FROM nginxdb.access_logs + ``` + + + + ## Разбор логов {#4-parse-the-logs} + + Хранить логи в ClickHouse полезно, но сохранение каждого события в виде одной строки не дает больших возможностей для анализа данных. + Далее мы рассмотрим, как разбирать события логов с помощью [материализованного представления](/materialized-view/incremental-materialized-view). + + **Материализованное представление** работает подобно триггеру на INSERT в SQL. Когда строки данных вставляются в исходную таблицу, материализованное представление выполняет над ними некоторые преобразования и вставляет результаты в целевую таблицу. + Материализованное представление можно настроить так, чтобы формировать разобранное представление событий логов в **access_logs**. + Ниже приведен пример одного такого события лога: + + ```bash + 192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` + + В ClickHouse есть различные функции для разбора приведённой выше строки. Функция [`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) разбирает строку по пробельным символам и возвращает каждый токен в виде элемента массива. + Для демонстрации выполните следующую команду: + + ```sql title="Query" + SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + ```text title="Response" + ["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] + ``` + + В некоторых строках есть лишние символы, а user agent (сведения о браузере) не требуется парсить, но + получившийся массив близок к нужному. + + Аналогично функции `splitByWhitespace`, функция [`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) разбивает строку на массив по регулярному выражению. + Выполните следующую команду, которая вернёт две строки. + + ```sql + SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + Обратите внимание, что вторая возвращаемая строка — это User-Agent, успешно разобранный из лога: + + ```text + ["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] + ``` + + Прежде чем перейти к итоговой команде `CREATE MATERIALIZED VIEW`, давайте рассмотрим ещё пару функций, используемых для очистки данных. + Например, поле `RequestMethod` имеет значение `"GET`, то есть содержит лишнюю двойную кавычку. + Вы можете использовать функцию [`trimBoth` (псевдоним `trim`)](/sql-reference/functions/string-functions#trimBoth), чтобы удалить двойную кавычку: + + ```sql + SELECT trim(LEADING '"' FROM '"GET') + ``` + + Строка с датой начинается с квадратной скобки и при этом имеет формат, который ClickHouse не может разобрать как дату. + Однако, если мы заменим разделитель с двоеточия (**:**) на запятую (**,**), разбор уже работает отлично: + + ```sql + SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) + ``` + + Теперь мы готовы определить материализованное представление. + Определение ниже включает `POPULATE`, что означает, что существующие строки в **access_logs** будут обработаны и вставлены сразу же. + Выполните следующую SQL-команду: + + ```sql + CREATE MATERIALIZED VIEW nginxdb.access_logs_view + ( + RemoteAddr String, + Client String, + RemoteUser String, + TimeLocal DateTime, + RequestMethod String, + Request String, + HttpVersion String, + Status Int32, + BytesSent Int64, + UserAgent String + ) + ENGINE = MergeTree() + ORDER BY RemoteAddr + POPULATE AS + WITH + splitByWhitespace(message) as split, + splitByRegexp('\S \d+ "([^"]*)"', message) as referer + SELECT + split[1] AS RemoteAddr, + split[2] AS Client, + split[3] AS RemoteUser, + parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, + trim(LEADING '"' FROM split[6]) AS RequestMethod, + split[7] AS Request, + trim(TRAILING '"' FROM split[8]) AS HttpVersion, + split[9] AS Status, + split[10] AS BytesSent, + trim(BOTH '"' from referer[2]) AS UserAgent + FROM + (SELECT message FROM nginxdb.access_logs) + ``` + + Теперь убедитесь, что всё работает. + Вы должны увидеть журналы доступа, корректно разобранные по столбцам: + + ```sql + SELECT * FROM nginxdb.access_logs_view + ``` + + + + :::note + В приведённом выше примере данные сохраняются в двух таблицах, но вы можете изменить исходную таблицу `nginxdb.access_logs`, чтобы использовать движок таблиц [`Null`](/engines/table-engines/special/null). + Разобранные данные по-прежнему будут попадать в таблицу `nginxdb.access_logs_view`, но исходные данные не будут сохраняться в таблице. + ::: > Используя Vector, который требует лишь простой установки и быстрой настройки, вы можете отправлять журналы с сервера Nginx в таблицу ClickHouse. Используя материализованное представление, вы можете разобрать эти журналы по столбцам для упрощения аналитики. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md index 23f45f96d9a..302eb98e87d 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md @@ -8,13 +8,13 @@ doc_type: 'guide' keywords: ['Google Cloud Storage ClickHouse', 'интеграция GCS с ClickHouse', 'MergeTree на базе GCS', 'хранение данных ClickHouse в GCS', 'Google Cloud ClickHouse'] --- -import BucketDetails from '@site/docs/_snippets/_GCS_authentication_and_bucket.md'; +import BucketDetails from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_GCS_authentication_and_bucket.md'; import Image from '@theme/IdealImage'; import GCS_examine_bucket_1 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-1.png'; import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-2.png'; -# Интеграция Google Cloud Storage с ClickHouse +# Интеграция Google Cloud Storage с ClickHouse {#integrate-google-cloud-storage-with-clickhouse} :::note Если вы используете ClickHouse Cloud в [Google Cloud](https://cloud.google.com), эта страница к вам не относится, так как ваши сервисы уже используют [Google Cloud Storage](https://cloud.google.com/storage). Если вы хотите выполнять `SELECT` или `INSERT` данных из GCS, обратитесь к табличной функции [`gcs`](/sql-reference/table-functions/gcs). @@ -24,13 +24,13 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio -## MergeTree с хранилищем в GCS +## MergeTree с хранилищем в GCS {#gcs-backed-mergetree} -### Создание диска +### Создание диска {#creating-a-disk} Чтобы использовать bucket GCS как диск, сначала необходимо объявить его в конфигурации ClickHouse в файле в каталоге `conf.d`. Ниже приведён пример объявления диска GCS. Эта конфигурация включает несколько секций для настройки GCS‑«диска», кэша и политики, которая указывается в DDL‑запросах при создании таблиц на диске GCS. Каждая из них описана ниже. -#### Storage configuration > disks > gcs +#### Storage configuration > disks > gcs {#storage_configuration--disks--gcs} Эта часть конфигурации показана на выделенном фрагменте и задаёт следующее: @@ -68,7 +68,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio ``` -#### Конфигурация хранилища > disks > cache +#### Конфигурация хранилища > disks > cache {#storage_configuration--disks--cache} Пример конфигурации, показанный ниже, включает кэш в памяти объёмом 10Gi для диска `gcs`. @@ -106,7 +106,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio ``` -#### Конфигурация хранилища > policies > gcs_main +#### Конфигурация хранилища > policies > gcs_main {#storage_configuration--policies--gcs_main} Политики хранения в конфигурации позволяют выбирать, где размещаются данные. Политика, показанная ниже, позволяет хранить данные на диске `gcs`, указывая политику `gcs_main`. Например, `CREATE TABLE ... SETTINGS storage_policy='gcs_main'`. @@ -141,7 +141,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio Полный список настроек, относящихся к этому описанию диска, можно найти [здесь](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3). -### Создание таблицы +### Создание таблицы {#creating-a-table} Если вы настроили диск на использование бакета с правом записи, вы сможете создать таблицу, как в примере ниже. Ради краткости мы используем подмножество столбцов набора данных NYC taxi и отправляем данные напрямую в таблицу на базе GCS: @@ -179,11 +179,11 @@ INSERT INTO trips_gcs SELECT trip_id, pickup_date, pickup_datetime, dropoff_date SELECT passenger_count, avg(tip_amount) AS avg_tip, avg(total_amount) AS avg_amount FROM trips_gcs GROUP BY passenger_count; ``` -### Работа с репликацией +### Работа с репликацией {#handling-replication} Репликация с использованием дисков GCS может быть реализована с помощью движка таблиц `ReplicatedMergeTree`. Подробности см. в руководстве [репликация одного шарда в двух регионах GCP с использованием GCS](#gcs-multi-region). -### Дополнительные материалы +### Дополнительные материалы {#learn-more} [Cloud Storage XML API](https://cloud.google.com/storage/docs/xml-api/overview) совместим с некоторыми инструментами и библиотеками, которые работают с такими сервисами, как Amazon Simple Storage Service (Amazon S3). @@ -305,13 +305,13 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -### Настройка сервера ClickHouse +### Настройка сервера ClickHouse {#configure-clickhouse-server} :::note best practice На некоторых шагах этого руководства вам потребуется поместить конфигурационный файл в `/etc/clickhouse-server/config.d/`. Это каталог по умолчанию в системах Linux для файлов переопределения конфигурации. Когда вы помещаете эти файлы в этот каталог, ClickHouse объединяет их содержимое с конфигурацией по умолчанию. Размещая эти файлы в каталоге `config.d`, вы избежите потери настроек при обновлении. ::: -#### Сеть +#### Сеть {#networking} По умолчанию ClickHouse прослушивает только loopback‑интерфейс; в реплицированной конфигурации требуется сетевое взаимодействие между серверами. Включите прослушивание на всех сетевых интерфейсах: @@ -321,7 +321,7 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -#### Удалённые серверы ClickHouse Keeper +#### Удалённые серверы ClickHouse Keeper {#remote-clickhouse-keeper-servers} Репликация координируется ClickHouse Keeper. Этот конфигурационный файл идентифицирует узлы ClickHouse Keeper по имени хоста и номеру порта. @@ -346,7 +346,7 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -#### Удалённые серверы ClickHouse +#### Удалённые серверы ClickHouse {#remote-clickhouse-servers} Этот файл задаёт имя хоста и порт каждого сервера ClickHouse в кластере. Конфигурационный файл по умолчанию содержит примеры описаний кластеров; чтобы отображались только полностью настроенные кластеры, к записи `remote_servers` добавляется тег `replace="true"`, чтобы при слиянии этой конфигурации с конфигурацией по умолчанию она заменяла секцию `remote_servers`, а не дополняла её. @@ -372,7 +372,7 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -#### Идентификация реплики +#### Идентификация реплики {#replica-identification} Этот файл настраивает параметры, связанные с путём в ClickHouse Keeper. В частности, в нём задаются макросы, используемые для идентификации реплики, к которой относятся данные. На одном сервере реплика должна быть указана как `replica_1`, а на другом сервере — как `replica_2`. Эти имена можно изменить: например, в нашем случае, когда одна реплика хранится в Южной Каролине, а другая — в Северной Вирджинии, значениями могут быть `carolina` и `virginia`; просто убедитесь, что на каждой машине они отличаются. @@ -390,7 +390,7 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -#### Хранение в GCS +#### Хранение в GCS {#storage-in-gcs} Конфигурация хранилища ClickHouse включает `disks` и `policies`. Диск, настраиваемый ниже, называется `gcs` и имеет `type` `s3`. Тип `s3` используется, потому что ClickHouse обращается к бакету GCS так, как если бы это был бакет AWS S3. Понадобятся две копии этой конфигурации — по одной для каждого узла сервера ClickHouse. @@ -438,7 +438,7 @@ ClickHouse Keeper для работы требует двух узлов, поэ ``` -### Запустите ClickHouse Keeper +### Запустите ClickHouse Keeper {#start-clickhouse-keeper} Используйте команды, соответствующие вашей операционной системе, например: @@ -448,7 +448,7 @@ sudo systemctl start clickhouse-keeper sudo systemctl status clickhouse-keeper ``` -#### Проверьте статус ClickHouse Keeper +#### Проверьте статус ClickHouse Keeper {#check-clickhouse-keeper-status} Отправляйте команды в ClickHouse Keeper с помощью `netcat`. Например, команда `mntr` возвращает состояние кластера ClickHouse Keeper. Если выполнить эту команду на каждом узле Keeper, вы увидите, что один из них является лидером, а два других — ведомыми: @@ -464,11 +464,11 @@ zk_max_latency 11 zk_min_latency 0 zk_packets_received 1783 zk_packets_sent 1783 -# highlight-start +# highlight-start {#highlight-start} zk_num_alive_connections 2 zk_outstanding_requests 0 zk_server_state leader -# highlight-end +# highlight-end {#highlight-end} zk_znode_count 135 zk_watch_count 8 zk_ephemerals_count 3 @@ -477,13 +477,13 @@ zk_key_arena_size 28672 zk_latest_snapshot_size 0 zk_open_file_descriptor_count 182 zk_max_file_descriptor_count 18446744073709551615 -# highlight-start +# highlight-start {#highlight-start} zk_followers 2 zk_synced_followers 2 -# highlight-end +# highlight-end {#highlight-end} ``` -### Запуск сервера ClickHouse +### Запуск сервера ClickHouse {#start-clickhouse-server} На `chnode1` и `chnode` выполните: @@ -495,9 +495,9 @@ sudo service clickhouse-server start sudo service clickhouse-server status ``` -### Проверка +### Проверка {#verification} -#### Проверка конфигурации дисков +#### Проверка конфигурации дисков {#verify-disk-configuration} `system.disks` должна содержать записи для каждого диска: @@ -565,7 +565,7 @@ cache_path: 3 строки в наборе. Прошло: 0,002 сек. ```` -#### Убедитесь, что таблицы, созданные на кластере, создаются на обоих узлах +#### Убедитесь, что таблицы, созданные на кластере, создаются на обоих узлах {#verify-that-tables-created-on-the-cluster-are-created-on-both-nodes} ```sql -- highlight-next-line create table trips on cluster 'cluster_1S_2R' ( @@ -600,7 +600,7 @@ SETTINGS storage_policy='gcs_main' 2 строки в наборе. Затрачено: 0.641 сек. ``` -#### Убедитесь, что данные можно вставить +#### Убедитесь, что данные можно вставить {#verify-that-data-can-be-inserted} ```sql INSERT INTO trips SELECT @@ -621,7 +621,7 @@ FROM s3('https://ch-nyc-taxi.s3.eu-west-3.amazonaws.com/tsv/trips_{0..9}.tsv.gz' LIMIT 1000000 ``` -#### Убедитесь, что для таблицы используется политика хранения данных `gcs_main`. +#### Убедитесь, что для таблицы используется политика хранения данных `gcs_main`. {#verify-that-the-storage-policy-gcs_main-is-used-for-the-table} ```sql SELECT @@ -647,14 +647,14 @@ formatReadableSize(total_bytes): 36.42 MiB Получена 1 строка. Прошло: 0.002 сек. ``` -#### Проверьте в консоли Google Cloud +#### Проверьте в консоли Google Cloud {#verify-in-google-cloud-console} Если открыть бакеты, вы увидите, что в каждом из них создана папка с именем, указанным в конфигурационном файле `storage.xml`. Разверните папки, и вы увидите множество файлов, соответствующих разделам данных. -#### Бакет для первой реплики +#### Бакет для первой реплики {#bucket-for-replica-one} -#### Бакет для второй реплики +#### Бакет для второй реплики {#bucket-for-replica-two} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md index 165620a4a2f..27254572a12 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md @@ -11,7 +11,7 @@ keywords: ['Google Dataflow ClickHouse', 'Dataflow ClickHouse integration', 'Apa import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Интеграция Google Dataflow с ClickHouse +# Интеграция Google Dataflow с ClickHouse {#integrating-google-dataflow-with-clickhouse} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md index 377baf37e09..83e9ac6257b 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md @@ -11,7 +11,7 @@ keywords: ['Dataflow Java Runner', 'Google Dataflow ClickHouse', 'Apache Beam Ja import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Java-раннер Dataflow +# Java-раннер Dataflow {#dataflow-java-runner} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md index 5287c52fd46..b688aea96f5 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md @@ -11,7 +11,7 @@ keywords: ['google dataflow', 'gcp', 'конвейер данных', 'шабл import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Шаблоны Google Dataflow +# Шаблоны Google Dataflow {#google-dataflow-templates} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md index 188f6dfc2ab..3f3ef364b30 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md @@ -19,7 +19,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Шаблон Dataflow BigQuery to ClickHouse +# Шаблон Dataflow BigQuery to ClickHouse {#dataflow-bigquery-to-clickhouse-template} Шаблон BigQuery to ClickHouse представляет собой пакетный конвейер обработки данных, который выполняет приём данных из таблицы BigQuery в таблицу ClickHouse. Шаблон может считывать всю таблицу или фильтровать определённые записи с помощью заданного SQL-запроса. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md index 2180f44e4df..f55fd0de7be 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md @@ -9,7 +9,7 @@ doc_type: 'guide' keywords: ['загрузка локальных файлов в ClickHouse', 'импорт локальных файлов в ClickHouse', 'загрузка файлов через clickhouse-client'] --- -# Вставка локальных файлов +# Вставка локальных файлов {#insert-local-files} Вы можете использовать `clickhouse-client` для потоковой загрузки локальных файлов в сервис ClickHouse. Это позволяет предварительно обрабатывать данные с помощью множества мощных и удобных функций ClickHouse. Рассмотрим пример... @@ -79,7 +79,6 @@ LIMIT 7 Результат: - ```response │ 488 │ comment │ mynameishere │ 2007-02-22 14:48:18 │ «Жаль. Javascript-в-браузере и Ajax — это оба грязные хаки, которые заставляют программистов делать всевозможные постыдные вещи. И результатом становятся кривоватые HTML‑фокусы. Java, при всех своих недостатках, достаточно чиста, когда запускается в среде апплета. У неё есть все преимущества перед JITBAJAX, за исключением проблем с установкой и тяжёлого процесса загрузки. Yahoo games, похоже, едва ли не единственная успешная история с апплетами. Конечно, раньше сколько-нибудь сложные апплеты обычно были слишком большими для dial‑up‑аккаунтов, которые были у людей. Сейчас, по крайней мере, это уже не так.» │ [454927] │ ['It','s','too','bad','Javascript','in','the','browser','and','Ajax','are','both','nasty','hacks','that','force','programmers','to','do','all','sorts','of','shameful','things','And','the','result','is','wanky','html','tricks','Java','for','its','faults','is','fairly','clean','when','run','in','the','applet','environment','It','has','every','superiority','over','JITBAJAX','except','for','install','issues','and','a','chunky','load','process','Yahoo','games','seems','like','just','about','the','only','applet','success','story','Of','course','back','in','the','day','non','trivial','Applets','tended','to','be','too','large','for','the','dial','up','accounts','people','had','At','least','that','is','changed'] │ │ 575 │ comment │ leoc │ 2007-02-23 00:09:49 │ «Сейчас я не могу найти ссылку, но, *кажется*, только что читал что‑то, из чего следует, что процесс установки апплета Apollo будет включать диалог подтверждения вида "install-this-application?" с последующей загрузкой примерно на 30 секунд. Если так, то Apollo менее многообещающ, чем я надеялся. Такой тип установки может считаться малообременительным по меркам настольных приложений, но он не сравним с лёгкостью запуска браузерного AJAX‑ или Flash‑приложения. (Подумайте, насколько просто впервые воспользоваться maps.google.com.)

Наверняка хотя бы будет так, что приложения Apollo по умолчанию запускаются в недоверенном режиме (untrusted), а уже установленное приложение будет запускаться автоматически, когда вы откроете в браузере URL, с которого его скачали?» │ [455071] │ ['I','can','t','find','the','reference','now','but','I','think','I','ve','just','read','something','suggesting','that','the','install','process','for','an','Apollo','applet','will','involve','an','34','install','this','application','34','confirmation','dialog','followed','by','a','download','of','30','seconds','or','so','If','so','then','Apollo','s','less','promising','than','I','hoped','That','kind','of','install','may','be','low','friction','by','desktop','app','standards','but','it','doesn','t','compare','to','the','ease','of','starting','a','browser','based','AJAX','or','Flash','application','Consider','how','easy','it','is','to','use','maps','google','com','for','the','first','time','p','Surely','it','will','at','least','be','that','Apollo','applications','will','run','untrusted','by','default','and','that','an','already','installed','app','will','start','automatically','whenever','you','take','your','browser','to','the','URL','you','downloaded','it','from'] │ diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md index 3628bc5e175..24d00e76ecb 100644 --- a/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md +++ b/i18n/ru/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md @@ -12,11 +12,11 @@ integration: - website: 'https://clickhouse.com/cloud/clickpipes' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; -# Интеграция Confluent Cloud с ClickHouse +# Интеграция Confluent Cloud с ClickHouse {#integrating-confluent-cloud-with-clickhouse}

\ No newline at end of file + + + + +fullscreen; +picture-in-picture" + allowfullscreen + />
-
+
根据当前数据所在的位置,有多种方式可以将数据迁移到 ClickHouse Cloud: -- [自托管迁移到 Cloud](/cloud/migration/clickhouse-to-cloud):使用 `remoteSecure` 函数传输数据 -- [其他 DBMS](/cloud/migration/clickhouse-local):将 [clickhouse-local] ETL 工具与适用于当前 DBMS 的相应 ClickHouse 表函数配合使用 -- [任意数据源!](/cloud/migration/etl-tool-to-clickhouse):使用众多常用的 ETL/ELT 工具之一连接各种不同的数据源 -- [对象存储](/integrations/migration/object-storage-to-clickhouse):轻松将 S3 中的数据插入到 ClickHouse 中 +* [自托管迁移到 Cloud](/cloud/migration/clickhouse-to-cloud):使用 `remoteSecure` 函数传输数据 +* [其他 DBMS](/cloud/migration/clickhouse-local):将 [clickhouse-local] ETL 工具与适用于当前 DBMS 的相应 ClickHouse 表函数配合使用 +* [任意数据源!](/cloud/migration/etl-tool-to-clickhouse):使用众多常用的 ETL/ELT 工具之一连接各种不同的数据源 +* [对象存储](/integrations/migration/object-storage-to-clickhouse):轻松将 S3 中的数据插入到 ClickHouse 中 在示例 [从 Redshift 迁移](/migrations/redshift/migration-guide) 中,我们展示了三种不同的数据迁移方式,将数据迁移到 ClickHouse。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md index 5481f5ffa19..61b243f1d39 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# ClickHouse 与 PostgreSQL 对比 +# ClickHouse 与 PostgreSQL 对比 {#comparing-clickhouse-and-postgresql} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md index fa1ab35a7a5..24b3f03629b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/appendix.md @@ -129,7 +129,7 @@ ClickHouse Cloud 在 S3 上只保留一份数据副本,并配合多个计算 -## 压缩 +## 压缩 {#compression} 由于 ClickHouse 采用列式存储,相比 Postgres,压缩效果通常会显著更好。如下图所示,我们对比了在这两个数据库中存储所有 Stack Overflow 表时的空间需求: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md index c4a4686eb07..fcc34130fd0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/01_migration_guide_part1.md @@ -31,49 +31,49 @@ import Image from '@theme/IdealImage'; ```bash -# 用户 +# 用户 {#users} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/users.sql.gz gzip -d users.sql.gz psql < users.sql ``` -# posts 表 +# posts 表 {#posts} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posts.sql.gz gzip -d posts.sql.gz psql < posts.sql -# posthistory +# posthistory {#posthistory} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/posthistory.sql.gz gzip -d posthistory.sql.gz psql < posthistory.sql -# 评论 +# 评论 {#comments} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/comments.sql.gz gzip -d comments.sql.gz psql < comments.sql -# votes 表 +# votes 表 {#votes} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/votes.sql.gz gzip -d votes.sql.gz psql < votes.sql -# badges 徽章 +# badges 徽章 {#badges} wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/badges.sql.gz gzip -d badges.sql.gz psql < badges.sql -# postlinks +# postlinks {#postlinks} wget [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/pdump/2024/postlinks.sql.gz) gzip -d postlinks.sql.gz @@ -87,9 +87,9 @@ psql < postlinks.sql ``` -## 迁移数据 +## 迁移数据 {#migrating-data} -### 实时复制(CDC) +### 实时复制(CDC) {#real-time-replication-or-cdc} 请参阅此[指南](/integrations/clickpipes/postgres),为 PostgreSQL 配置 ClickPipes。该指南涵盖了多种不同类型的 Postgres 源实例。 @@ -125,7 +125,7 @@ ORDER BY id; 完成设置后,ClickPipes 会开始将 PostgreSQL 中的所有数据迁移到 ClickHouse。根据网络状况和部署规模,对于 Stack Overflow 数据集,这通常只需要几分钟。 -### 手动批量加载与定期更新 +### 手动批量加载与定期更新 {#initial-bulk-load-with-periodic-updates} 采用手动方式时,可以通过以下方法完成数据集的初始批量加载: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md index ae30c969daa..b4566b4e4ed 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/02_migration_guide_part2.md @@ -21,7 +21,7 @@ doc_type: 'guide' -## 在 ClickHouse 中优化查询 +## 在 ClickHouse 中优化查询 {#optimize-queries-in-clickhouse} 虽然可以在对查询做最少改写的情况下完成迁移,但建议充分利用 ClickHouse 的特性,以显著简化查询并进一步提升查询性能。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md index ec56435b52b..ba4af208d41 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/02_postgres/migration_guide/03_migration_guide_part3.md @@ -49,7 +49,7 @@ import Image from '@theme/IdealImage'; -## 分区 +## 分区 {#partitions} Postgres 用户对表分区这一概念应该很熟悉:通过将表拆分为更小、更易管理的片段(称为分区),以提升大型数据库的性能和可管理性。分区可以通过在指定列(例如日期)上使用范围、指定列表,或基于键的哈希来实现。这使管理员可以基于特定条件(如日期范围或地理位置)来组织数据。分区有助于通过分区裁剪和更高效的索引来提升查询性能,从而实现更快速的数据访问。同时,它也有助于维护任务,例如备份和数据清理,因为可以针对单个分区而不是整个表执行操作。此外,通过将负载分布到多个分区,分区还能显著提高 PostgreSQL 数据库的可扩展性。 @@ -76,7 +76,7 @@ PARTITION BY toYear(CreationDate) 有关分区的完整介绍,请参阅 ["Table partitions"](/partitions)。 -### 分区的应用场景 +### 分区的应用场景 {#applications-of-partitions} ClickHouse 中的分区与 Postgres 的应用场景类似,但也存在一些细微差别。更具体地说: @@ -132,7 +132,7 @@ ALTER TABLE posts -## 物化视图与投影 +## 物化视图与投影 {#materialized-views-vs-projections} Postgres 允许在单个表上创建多个索引,从而可以针对多种访问模式进行优化。这种灵活性使管理员和开发人员能够根据特定查询和运维需求定制数据库性能。ClickHouse 的投影(projections)概念虽然与此并非完全等价,但允许用户为一张表指定多个 `ORDER BY` 子句。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md index 096da1fcf13..8865e6998e7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/01_overview.md @@ -12,7 +12,7 @@ import bigquery_1 from '@site/static/images/migrations/bigquery-1.png'; import Image from '@theme/IdealImage'; -# ClickHouse Cloud 与 BigQuery 对比 +# ClickHouse Cloud 与 BigQuery 对比 {#comparing-clickhouse-cloud-and-bigquery} @@ -186,7 +186,7 @@ ClickHouse 提供了标准 SQL,并在此基础上进行了大量扩展和改 -## 数组 +## 数组 {#arrays} 与 BigQuery 仅有 8 个数组函数相比,ClickHouse 提供了 80 多个[内置数组函数](/sql-reference/functions/array-functions),可以优雅而简洁地对各种问题进行建模和求解。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md index 3896de9a6ec..86b82068521 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/02_migrating-to-clickhouse-cloud.md @@ -75,7 +75,7 @@ CDC(变更数据捕获)是指在两个数据库之间保持表数据同步 -## 模式设计 +## 模式设计 {#designing-schemas} Stack Overflow 数据集包含许多相关的表。我们建议优先迁移主表。这不一定是最大的那张表,而是您预计会收到最多分析查询的那张表。这样可以帮助您熟悉 ClickHouse 的核心概念。随着后续添加更多表,为了充分利用 ClickHouse 的特性并获得最佳性能,可能需要对该表进行重新建模。我们在[数据建模文档](/data-modeling/schema-design#next-data-modeling-techniques)中对这一建模过程进行了探讨。 @@ -108,7 +108,7 @@ CREATE TABLE stackoverflow.posts ( ); ``` -### 优化数据类型 +### 优化数据类型 {#optimizing-types} 按照[此处所述的流程](/data-modeling/schema-design)进行后,将得到如下模式: @@ -174,11 +174,11 @@ INSERT INTO stackoverflow.posts SELECT * FROM gcs( 'gs://clickhouse-public-datas -## 数据建模技术 +## 数据建模技术 {#data-modeling-techniques} 我们建议从 BigQuery 迁移的用户阅读[在 ClickHouse 中进行数据建模的指南](/data-modeling/schema-design)。该指南使用相同的 Stack Overflow 数据集,并结合 ClickHouse 的特性来探索多种建模方法。 -### 分区 +### 分区 {#partitions} BigQuery 用户应该已经熟悉表分区的概念:通过将表拆分为更小、更易管理的部分(称为分区),来提升大型数据库的性能和可管理性。分区可以通过在指定列(例如日期)上使用范围分区、定义列表分区,或者基于键的哈希分区来实现。这使得管理员可以根据特定条件(例如日期范围或地理位置)来组织数据。 @@ -205,7 +205,7 @@ ORDER BY (PostTypeId, toDate(CreationDate), CreationDate) PARTITION BY toYear(CreationDate) ``` -#### 应用场景 +#### 应用场景 {#applications} ClickHouse 中的分区与 BigQuery 有类似的应用场景,但存在一些细微差异。更具体地说: @@ -259,7 +259,7 @@ Ok. -## 物化视图与投影 +## 物化视图与投影 {#materialized-views-vs-projections} ClickHouse 的投影(projection)概念允许用户为同一张表指定多个 `ORDER BY` 子句。 @@ -397,7 +397,7 @@ WHERE UserId = 8592047 ``` -## 在 ClickHouse 中重写 BigQuery 查询 +## 在 ClickHouse 中重写 BigQuery 查询 {#rewriting-bigquery-queries-in-clickhouse} 下文给出了 BigQuery 与 ClickHouse 的对比查询示例。该列表旨在演示如何利用 ClickHouse 的特性来大幅简化查询。这里的示例使用完整的 Stack Overflow 数据集(截至 2024 年 4 月)。 @@ -465,7 +465,7 @@ LIMIT 5 ``` -## 聚合函数 +## 聚合函数 {#aggregate-functions} 在条件允许的情况下,用户应尽可能利用 ClickHouse 的聚合函数。下面我们展示如何使用 [`argMax` 函数](/sql-reference/aggregate-functions/reference/argmax) 来计算每一年浏览次数最多的问题。 @@ -520,7 +520,7 @@ MaxViewCount: 66975 ``` -## 条件和数组 +## 条件和数组 {#conditionals-and-arrays} 条件和数组函数可以显著简化查询。下面的查询会计算在 2022 年到 2023 年间,出现次数超过 10000 次的标签中,百分比增幅最大的那些标签。请注意,得益于条件函数、数组函数以及在 `HAVING` 和 `SELECT` 子句中重复使用别名的能力,下面的 ClickHouse 查询非常简洁。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md index 2276b4afab1..c24a5e61814 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/03_bigquery/03_loading-data.md @@ -30,7 +30,7 @@ _本指南适用于 ClickHouse Cloud 以及自托管的 ClickHouse v23.5 及以 -## 将表数据导出到 GCS +## 将表数据导出到 GCS {#1-export-table-data-to-gcs} 在此步骤中,我们使用 [BigQuery SQL workspace](https://cloud.google.com/bigquery/docs/bigquery-web-ui) 来执行 SQL 命令。下面的示例中,我们使用 [`EXPORT DATA`](https://cloud.google.com/bigquery/docs/reference/standard-sql/other-statements) 语句,将名为 `mytable` 的 BigQuery 表导出到一个 GCS 存储桶中。 @@ -67,7 +67,7 @@ END WHILE; * Parquet 作为列式格式,是更好的交换格式,因为它天然具备压缩特性,并且对 BigQuery 导出和 ClickHouse 查询都更快。 -## 将数据从 GCS 导入 ClickHouse +## 将数据从 GCS 导入 ClickHouse {#2-importing-data-into-clickhouse-from-gcs} 导出完成后,我们即可将这些数据导入到 ClickHouse 表中。可以使用 [ClickHouse SQL console](/integrations/sql-clients/sql-console) 或 [`clickhouse-client`](/interfaces/cli) 来执行以下命令。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md index 227c4659ba0..ddced0f3d36 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/01_overview.md @@ -13,7 +13,7 @@ import cloud_architecture from '@site/static/images/cloud/onboard/discover/use_c import Image from '@theme/IdealImage'; -# 从 Snowflake 迁移到 ClickHouse +# 从 Snowflake 迁移到 ClickHouse {#snowflake-to-clickhouse-migration} > 本文档介绍如何将数据从 Snowflake 迁移到 ClickHouse。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md index 6f05d894808..2fafdbe7714 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/02_migration_guide.md @@ -12,7 +12,7 @@ import migrate_snowflake_clickhouse from '@site/static/images/migrations/migrate import Image from '@theme/IdealImage'; -# 从 Snowflake 迁移到 ClickHouse +# 从 Snowflake 迁移到 ClickHouse {#migrate-from-snowflake-to-clickhouse} > 本指南介绍如何将数据从 Snowflake 迁移到 ClickHouse。 @@ -21,7 +21,7 @@ import Image from '@theme/IdealImage'; -## 从 Snowflake 导出数据 +## 从 Snowflake 导出数据 {#1-exporting-data-from-snowflake} @@ -59,7 +59,7 @@ COPY INTO @external_stage/mydataset from mydataset max_file_size=157286400 heade 对于大约 5TB 的数据集(单个文件最大 150MB),在同一 AWS `us-east-1` 区域使用 2X-Large Snowflake 仓库时,将数据复制到 S3 存储桶大约需要 30 分钟。 -## 导入 ClickHouse +## 导入 ClickHouse {#2-importing-to-clickhouse} 将数据暂存到中间对象存储后,就可以使用 ClickHouse 的函数(例如 [s3 表函数](/sql-reference/table-functions/s3))将数据插入到表中,如下所示。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md index 2218bedf103..72b7745644a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/04_snowflake/03_sql_translation_reference.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# Snowflake SQL 转换指南 +# Snowflake SQL 转换指南 {#snowflake-sql-translation-guide} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md index 9e02ee52184..33ea300a539 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/05_elastic/01_overview.md @@ -8,6 +8,6 @@ show_related_blogs: true doc_type: 'landing-page' --- -# Elasticsearch 到 ClickHouse 的迁移 +# Elasticsearch 到 ClickHouse 的迁移 {#elasticsearch-to-clickhouse-migration} 在可观测性相关的使用场景中,请参阅 [Elasticsearch 到 ClickStack 的迁移文档](/use-cases/observability/clickstack/migration/elastic)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md index 0bf5d4cae1c..f5b6d8fab56 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/01_overview.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# 从 Amazon Redshift 迁移到 ClickHouse +# 从 Amazon Redshift 迁移到 ClickHouse {#amazon-redshift-to-clickhouse-migration} > 本文档对如何将数据从 Amazon Redshift 迁移到 ClickHouse 进行简介。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md index eb6c643cb3c..60414051a8f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/02_migration_guide.md @@ -7,9 +7,8 @@ title: '从 Amazon Redshift 迁移到 ClickHouse 指南' doc_type: 'guide' --- -import MigrationGuide from '@site/docs/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +import MigrationGuide from '@site/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/redshift/_snippets/_migration_guide.md' +# 从 Amazon Redshift 迁移到 ClickHouse 指南 {#amazon-redshift-to-clickhouse-migration-guide} -# 从 Amazon Redshift 迁移到 ClickHouse 指南 - - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md index db4ff876992..68984128a4f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/06_redshift/03_sql_translation_reference.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Amazon Redshift SQL 转换指南 +# Amazon Redshift SQL 转换指南 {#amazon-redshift-sql-translation-guide} @@ -52,9 +52,9 @@ doc_type: 'reference' -## DDL 语法 +## DDL 语法 {#compression} -### 排序键 +### 排序键 {#sorting-keys} ClickHouse 和 Redshift 都有“排序键”的概念,用于定义数据在存储时的排序方式。Redshift 使用 `SORTKEY` 子句来定义排序键: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md index 29e44477964..81531876604 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/07_OSS_to_Cloud/01_clickhouse-to-cloud.md @@ -8,7 +8,7 @@ keywords: ['迁移', 'ClickHouse Cloud', 'OSS', '将自托管迁移到 Cloud'] --- import Image from '@theme/IdealImage'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import self_managed_01 from '@site/static/images/integrations/migration/self-managed-01.png'; import self_managed_02 from '@site/static/images/integrations/migration/self-managed-02.png'; import self_managed_03 from '@site/static/images/integrations/migration/self-managed-03.png'; @@ -16,16 +16,13 @@ import self_managed_04 from '@site/static/images/integrations/migration/self-man import self_managed_05 from '@site/static/images/integrations/migration/self-managed-05.png'; import self_managed_06 from '@site/static/images/integrations/migration/self-managed-06.png'; +# 在自托管 ClickHouse 与 ClickHouse Cloud 之间迁移 {#migrating-between-self-managed-clickhouse-and-clickhouse-cloud} -# 在自托管 ClickHouse 与 ClickHouse Cloud 之间迁移 - - + 本指南将说明如何将自托管 ClickHouse 服务器迁移到 ClickHouse Cloud,以及如何在不同的 ClickHouse Cloud 服务之间进行迁移。[`remoteSecure`](/sql-reference/table-functions/remote) 函数在 `SELECT` 和 `INSERT` 查询中使用,以便访问远程 ClickHouse 服务器,这使得迁移数据表变得非常简单,只需编写一个嵌套 `SELECT` 的 `INSERT INTO` 查询即可完成。 - - -## 从自托管 ClickHouse 迁移到 ClickHouse Cloud +## 从自托管 ClickHouse 迁移到 ClickHouse Cloud {#migrating-from-self-managed-clickhouse-to-clickhouse-cloud} @@ -36,7 +33,7 @@ ClickHouse Cloud 会自动处理纵向和横向扩展。无需再考虑如何对 在本示例中,自托管 ClickHouse 服务器是*源端*,ClickHouse Cloud 服务是*目标端*。 -### 概览 +### 概览 {#overview} 流程如下: @@ -46,11 +43,11 @@ ClickHouse Cloud 会自动处理纵向和横向扩展。无需再考虑如何对 4. 从目标端的 IP 访问列表中移除源服务器(如适用) 5. 从源服务中删除该只读用户 -### 在两个系统之间迁移表: +### 在两个系统之间迁移表: {#migration-of-tables-from-one-system-to-another} 本示例会将一个表从自托管 ClickHouse 服务器迁移到 ClickHouse Cloud。 -### 在源 ClickHouse 系统上(当前托管数据的系统) +### 在源 ClickHouse 系统上(当前托管数据的系统) {#on-the-source-clickhouse-system-the-system-that-currently-hosts-the-data} * 添加一个可以读取源表(本例中为 `db.table`)的只读用户 @@ -72,7 +69,7 @@ FROM system.tables WHERE database = 'db' AND table = 'table' ``` -### 在目标 ClickHouse Cloud 系统上: +### 在目标 ClickHouse Cloud 系统上: {#on-the-destination-clickhouse-cloud-system} * 创建目标数据库: @@ -119,8 +116,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 'default', 'PASS') SELECT * FROM db.table ``` - -## 在 ClickHouse Cloud 服务之间迁移 +## 在 ClickHouse Cloud 服务之间迁移 {#migrating-between-clickhouse-cloud-services} @@ -143,7 +139,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 6. 在目标服务上重新建立 IP 访问列表 7. 从源服务中移除只读用户 -#### 在源服务中添加只读用户 +#### 在源服务中添加只读用户 {#add-a-read-only-user-to-the-source-service} * 添加一个只读用户,用于读取源表(本示例中为 `db.table`): @@ -164,7 +160,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', where database = 'db' and table = 'table' ``` -#### 在目标服务上复制表结构 +#### 在目标服务上复制表结构 {#duplicate-the-table-structure-on-the-destination-service} 在目标服务上,如果数据库尚不存在,则先创建数据库: @@ -181,7 +177,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', CREATE TABLE db.table ... ``` -#### 允许对源服务的远程访问 +#### 允许对源服务的远程访问 {#allow-remote-access-to-the-source-service} 为了将数据从源服务拉取到目标服务,源服务必须允许来自外部的连接。请在源服务上临时禁用“IP Access List(IP 访问列表)”功能。 @@ -191,7 +187,7 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', 修改访问列表,并临时将访问范围设置为 **Anywhere(任意地址)**。详细信息请参阅 [IP Access List](/cloud/security/setting-ip-filters) 文档。 -#### 将数据从源服务复制到目标服务 +#### 将数据从源服务复制到目标服务 {#copy-the-data-from-source-to-destination} * 使用 `remoteSecure` 函数从源 ClickHouse Cloud 服务拉取数据。 连接到目标服务,并在目标 ClickHouse Cloud 服务上运行以下命令: @@ -203,11 +199,11 @@ remoteSecure('HOSTNAME.clickhouse.cloud:9440', 'db.table', * 在目标服务中验证数据 -#### 在源服务上重新建立 IP 访问列表 +#### 在源服务上重新建立 IP 访问列表 {#re-establish-the-ip-access-list-on-the-source} 如果之前导出了访问列表,则可以通过 **Share** 进行重新导入;否则,请重新将各条访问记录添加到访问列表中。 -#### 移除只读的 `exporter` 用户 +#### 移除只读的 `exporter` 用户 {#remove-the-read-only-exporter-user} ```sql DROP USER exporter diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md index 7b886152e93..073f00c42a7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/01_clickhouse-local-etl.md @@ -11,14 +11,14 @@ import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import AddARemoteSystem from '@site/docs/_snippets/_add_remote_ip_access_list_detail.md'; +import AddARemoteSystem from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_add_remote_ip_access_list_detail.md'; import ch_local_01 from '@site/static/images/integrations/migration/ch-local-01.png'; import ch_local_02 from '@site/static/images/integrations/migration/ch-local-02.png'; import ch_local_03 from '@site/static/images/integrations/migration/ch-local-03.png'; import ch_local_04 from '@site/static/images/integrations/migration/ch-local-04.png'; -# 使用 clickhouse-local 迁移到 ClickHouse +# 使用 clickhouse-local 迁移到 ClickHouse {#migrating-to-clickhouse-using-clickhouse-local} @@ -94,21 +94,21 @@ ClickHouse 为 [MySQL](/engines/table-engines/integrations/mysql/)、[PostgreSQL -## 示例 1:使用集成引擎从 MySQL 迁移到 ClickHouse Cloud +## 示例 1:使用集成引擎从 MySQL 迁移到 ClickHouse Cloud {#example-1-migrating-from-mysql-to-clickhouse-cloud-with-an-integration-engine} 我们将使用 [integration 表引擎](/engines/table-engines/integrations/mysql/)(由 [mysql 表函数](/sql-reference/table-functions/mysql/) 动态创建)从源 MySQL 数据库读取数据,并使用 [remoteSecure 表函数](/sql-reference/table-functions/remote/) 将数据写入您在 ClickHouse Cloud 服务上的目标表中。 -### 在目标 ClickHouse Cloud 服务上: +### 在目标 ClickHouse Cloud 服务上: {#on-the-destination-clickhouse-cloud-service} -#### 创建目标数据库: +#### 创建目标数据库: {#create-the-destination-database} ```sql CREATE DATABASE db ``` -#### 创建一个目标表,使其表结构与 MySQL 表相同: +#### 创建一个目标表,使其表结构与 MySQL 表相同: {#create-a-destination-table-that-has-a-schema-equivalent-to-the-mysql-table} ```sql CREATE TABLE db.table ... @@ -118,9 +118,9 @@ CREATE TABLE db.table ... ClickHouse Cloud 目标表的表结构必须与源 MySQL 表的表结构保持一致(列名和列顺序必须相同,且列的数据类型必须兼容)。 ::: -### 在运行 clickhouse-local 的主机上: +### 在运行 clickhouse-local 的主机上: {#on-the-clickhouse-local-host-machine} -#### 使用迁移查询语句运行 clickhouse-local: +#### 使用迁移查询语句运行 clickhouse-local: {#run-clickhouse-local-with-the-migration-query} ```sql ./clickhouse-local --query " @@ -135,16 +135,16 @@ SELECT * FROM mysql('host:port', 'database', 'table', 'user', 'password');" ::: -## 示例 2:使用 JDBC bridge 将 MySQL 迁移到 ClickHouse Cloud +## 示例 2:使用 JDBC bridge 将 MySQL 迁移到 ClickHouse Cloud {#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge} 我们将使用 [JDBC 集成表引擎](/engines/table-engines/integrations/jdbc.md)(由 [jdbc 表函数](/sql-reference/table-functions/jdbc.md) 动态创建),配合 [ClickHouse JDBC Bridge](https://github.com/ClickHouse/clickhouse-jdbc-bridge) 和 MySQL JDBC 驱动,从源 MySQL 数据库读取数据,并使用 [remoteSecure 表函数](/sql-reference/table-functions/remote.md) 将数据写入 ClickHouse Cloud 服务中的目标表。 -### 在目标 ClickHouse Cloud 服务上: +### 在目标 ClickHouse Cloud 服务上: {#on-the-destination-clickhouse-cloud-service-1} -#### 创建目标数据库: +#### 创建目标数据库: {#create-the-destination-database-1} ```sql CREATE DATABASE db diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md index 0a19e1792f1..caff40358f9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/02_etl-tool-to-clickhouse.md @@ -10,15 +10,14 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import third_party_01 from '@site/static/images/integrations/migration/third-party-01.png'; +# 使用第三方 ETL 工具 {#using-a-3rd-party-etl-tool} -# 使用第三方 ETL 工具 - - + 将数据从外部数据源迁移到 ClickHouse 的一个很好的选择,是使用众多流行的 ETL/ELT 工具之一。我们提供以下相关文档: -- [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) -- [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) -- [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) +* [Airbyte](/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md) +* [dbt](/integrations/data-ingestion/etl-tools/dbt/index.md) +* [Vector](/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md) 此外,还有许多其他 ETL/ELT 工具可以与 ClickHouse 集成,请查阅所使用工具的文档以获取详细信息。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md index a4ffa3f686f..3c3ce652b33 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/02_migrate/01_migration_guides/08_other_methods/03_object-storage-to-clickhouse.md @@ -9,16 +9,15 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import object_storage_01 from '@site/static/images/integrations/migration/object-storage-01.png'; +# 将数据从云对象存储迁移到 ClickHouse Cloud {#move-data-from-cloud-object-storage-to-clickhouse-cloud} -# 将数据从云对象存储迁移到 ClickHouse Cloud - - + 如果你将云对象存储用作数据湖并希望将这些数据导入 ClickHouse Cloud,或者当前数据库系统可以直接将数据导出到云对象存储,那么可以使用以下表函数,将存储在云对象存储中的数据迁移到 ClickHouse Cloud 表中: -- [s3](/sql-reference/table-functions/s3.md) 或 [s3Cluster](/sql-reference/table-functions/s3Cluster.md) -- [gcs](/sql-reference/table-functions/gcs) -- [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) +* [s3](/sql-reference/table-functions/s3.md) 或 [s3Cluster](/sql-reference/table-functions/s3Cluster.md) +* [gcs](/sql-reference/table-functions/gcs) +* [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage) 如果当前数据库系统无法直接将数据导出到云对象存储,你可以使用[第三方 ETL/ELT 工具](/cloud/migration/etl-tool-to-clickhouse)或 [clickhouse-local](/cloud/migration/clickhouse-local),先将数据从当前数据库系统迁移到云对象存储,然后在第二步再将这些数据迁移到 ClickHouse Cloud 表中。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md index c632dbef4a4..ba8f98f5738 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/03_tune/resource_tour.md @@ -7,12 +7,12 @@ hide_title: true doc_type: 'guide' --- -import TableOfContentsBestPractices from '@site/docs/best-practices/_snippets/_table_of_contents.md'; -import TableOfContentsOptimizationAndPerformance from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; -import TableOfContentsSecurity from '@site/docs/cloud/_snippets/_security_table_of_contents.md'; +import TableOfContentsBestPractices from '@site/i18n/zh/docusaurus-plugin-content-docs/current/best-practices/_snippets/_table_of_contents.md'; +import TableOfContentsOptimizationAndPerformance from '@site/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContentsSecurity from '@site/i18n/zh/docusaurus-plugin-content-docs/current/cloud/_snippets/_security_table_of_contents.md'; -# 资源导览 +# 资源导览 {#resource-tour} 本文旨在为您概述文档中可用的资源,帮助您充分发挥 ClickHouse Cloud 部署的价值。 您可以按以下主题浏览资源: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/index.md index 21dd61d6c13..78cfbcf5194 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/onboard/index.md @@ -9,7 +9,7 @@ keywords: ['上手引导', '入门', '云端配置', '快速入门', '简介'] -# ClickHouse Cloud 入门 +# ClickHouse Cloud 入门 {#get-started-with-clickhouse-cloud} 初次使用 ClickHouse Cloud,不知从何入手?本文档部分将指导您完成快速部署和运行所需的全部步骤。我们将入门指南划分为三个子部分,帮助您在探索 ClickHouse Cloud 的过程中循序渐进地完成各项配置。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md index ebe542c9ce2..ae0334e3395 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_02.md @@ -12,7 +12,7 @@ doc_type: 'changelog' -#### 不向后兼容的变更 +#### 不向后兼容的变更 {#backward-incompatible-change} * 在嵌套类型中对可疑/实验性类型进行校验。此前我们不会在 Array/Tuple/Map 等嵌套类型中校验这类类型(JSON 除外)。[#59385](https://github.com/ClickHouse/ClickHouse/pull/59385)([Kruglov Pavel](https://github.com/Avogar))。 * 排序子句 `ORDER BY ALL`(在 v23.12 中引入)已被 `ORDER BY *` 取代。之前的语法对于包含名为 `all` 的列的表而言过于容易出错。[#59450](https://github.com/ClickHouse/ClickHouse/pull/59450)([Robert Schulze](https://github.com/rschu1ze))。 @@ -29,7 +29,7 @@ doc_type: 'changelog' -#### 新功能 +#### 新功能 {#new-feature} * 支持返回值计数及其误差的 topk/topkweighed 模式。 [#54508](https://github.com/ClickHouse/ClickHouse/pull/54508) ([UnamedRus](https://github.com/UnamedRus)). * 新增语法,允许在视图/物化视图中指定定义者用户(definer user)。这样可以在未对底层表显式授予权限的情况下,通过视图执行 SELECT/INSERT 操作。[#54901](https://github.com/ClickHouse/ClickHouse/pull/54901) ([pufit](https://github.com/pufit))。 @@ -57,7 +57,7 @@ doc_type: 'changelog' -#### 性能优化 +#### 性能优化 {#performance-improvement} * 在 `SELECT` 子句中消除了对 `GROUP BY` 键使用 `min`/`max`/`any`/`anyLast` 聚合函数的情况。[#52230](https://github.com/ClickHouse/ClickHouse/pull/52230) ([JackyWoo](https://github.com/JackyWoo))。 * 在处理多个 [nullable] 列时提升序列化聚合方法的性能。这是 [#51399](https://github.com/ClickHouse/ClickHouse/issues/51399) 的通用化版本,同时不牺牲抽象层的完整性。[#55809](https://github.com/ClickHouse/ClickHouse/pull/55809)([Amos Bird](https://github.com/amosbird))。 @@ -86,7 +86,7 @@ doc_type: 'changelog' -#### 改进 +#### 改进 {#improvement} * 在对物化视图运行 MODIFY COLUMN 查询时,检查内部表的结构以确保所有列均存在。[#47427](https://github.com/ClickHouse/ClickHouse/pull/47427) ([sunny](https://github.com/sunny19930321))。 * 新增了表 `system.keywords`,其中包含解析器中的所有关键字。主要是为了更好地支持模糊测试和语法高亮。 [#51808](https://github.com/ClickHouse/ClickHouse/pull/51808) ([Nikita Mikhaylov](https://github.com/nikitamikhaylov))。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md index f74695431ea..3460352d38c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_05.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# V24.5 Cloud 版本变更日志 +# V24.5 Cloud 版本变更日志 {#v245-changelog-for-cloud} 基于 v24.5 版本的 ClickHouse Cloud 服务相关变更。 @@ -128,7 +128,7 @@ doc_type: 'changelog' -# 改进 +# 改进 {#improvements} * 移除 `optimize_monotonous_functions_in_order_by` 设置,该设置正逐渐变成空操作(no-op)。[#63004](https://github.com/ClickHouse/ClickHouse/pull/63004) (Raúl Marín)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md index 32de2aef8ea..9a5c5929ef3 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_06.md @@ -10,7 +10,7 @@ doc_type: 'changelog' -# V24.6 版 Cloud 更新日志 +# V24.6 版 Cloud 更新日志 {#v246-changelog-for-cloud} v24.6 版本中与 ClickHouse Cloud 服务相关的更改。 @@ -110,7 +110,7 @@ v24.6 版本中与 ClickHouse Cloud 服务相关的更改。 -## 错误修复(官方稳定版本中用户可见的异常行为) +## 错误修复(官方稳定版本中用户可见的异常行为) {#bug-fix-user-visible-misbehavior-in-an-official-stable-release} * 修复了在与 IN 和 indexHint() 一起使用时 'set' 跳过索引不起作用的问题。#62083 (Michael Kolupaev)。 * 修复在表未使用自适应粒度时,带有 FINAL 关键字的查询返回错误结果的问题。#62432(Duc Canh Le)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md index c4d8c1228ce..91bad566910 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/01_changelog/02_release_notes/24_10.md @@ -22,7 +22,7 @@ doc_type: 'changelog' -## 新功能 +## 新功能 {#new-feature} * 可刷新物化视图已经可以在生产环境中使用。[#70550](https://github.com/ClickHouse/ClickHouse/pull/70550)([Michael Kolupaev](https://github.com/al13n321))。Replicated 数据库现已支持可刷新物化视图。[#60669](https://github.com/ClickHouse/ClickHouse/pull/60669)([Michael Kolupaev](https://github.com/al13n321))。 * 函数 `toStartOfInterval()` 现在新增了一个重载,可用于模拟 TimescaleDB 的 `time_bucket()` 函数以及 PostgreSQL 的 `date_bin()` 函数([#55619](https://github.com/ClickHouse/ClickHouse/issues/55619))。它允许将日期或时间戳值,对齐到从*任意*起点开始的给定时间间隔的整数倍(而不是以 0000-01-01 00:00:00.000 作为*固定*起点)。例如,`SELECT toStartOfInterval(toDateTime('2023-01-01 14:45:00'), INTERVAL 1 MINUTE, toDateTime('2023-01-01 14:35:30'));` 会返回 `2023-01-01 14:44:30`,这是以起点 `2023-01-01 14:35:30` 开始的 1 分钟间隔的整数倍。[#56738](https://github.com/ClickHouse/ClickHouse/pull/56738)([Yarik Briukhovetskyi](https://github.com/yariks5s))。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md index 09c9438efdd..73f50852bd9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/02_architecture.md @@ -11,7 +11,7 @@ import Image from '@theme/IdealImage'; import Architecture from '@site/static/images/cloud/reference/architecture.png'; -# ClickHouse Cloud 架构 +# ClickHouse Cloud 架构 {#clickhouse-cloud-architecture} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md index 38b34fffde7..2510653ffdd 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/01_billing_overview.md @@ -181,9 +181,9 @@ ClickHouse Cloud 会根据计算、存储、[数据传输](/cloud/manage/network -## 常见问题 +## 常见问题 {#faqs} -### 什么是 ClickHouse Credit(CHC)? +### 什么是 ClickHouse Credit(CHC)? {#what-is-chc} ClickHouse Credit 是针对客户使用 ClickHouse Cloud 的一单位额度,等同于一(1)美元,并按 ClickHouse 届时发布的有效价目表进行折算。 @@ -191,27 +191,27 @@ ClickHouse Credit 是针对客户使用 ClickHouse Cloud 的一单位额度, 如果您通过 Stripe 支付账单,那么在您的 Stripe 发票上会看到 1 CHC 等于 0.01 美元。这是为了在 Stripe 上实现精确计费,因为 Stripe 无法针对我们标准 SKU(1 CHC = 1 美元)按非整数数量进行计费。 ::: -### 在哪里可以找到旧版定价信息? +### 在哪里可以找到旧版定价信息? {#find-legacy-pricing} 旧版定价信息可以在[这里](https://clickhouse.com/pricing?legacy=true)找到。 -### 计算资源是如何计量的? +### 计算资源是如何计量的? {#how-is-compute-metered} ClickHouse Cloud 以每分钟为单位计量计算资源,粒度为每 8GB 内存。 计算费用会根据服务等级、区域和云服务提供商而变化。 -### 磁盘存储是如何计算的? +### 磁盘存储是如何计算的? {#how-is-storage-on-disk-calculated} ClickHouse Cloud 使用云对象存储,并根据存储在 ClickHouse 表中的数据压缩后大小来计量用量。 存储费用在各服务等级之间相同,但会根据区域和云服务提供商而变化。 -### 备份是否计入总存储? +### 备份是否计入总存储? {#do-backups-count-toward-total-storage} 存储和备份都会计入存储用量,并分别计费。 所有服务默认保留一个备份,保留时间为一天。 需要额外备份的用户可以在 Cloud 控制台的“设置”标签页中配置额外的[备份](/cloud/manage/backups/overview)。 -### 我该如何估算压缩比? +### 我该如何估算压缩比? {#how-do-i-estimate-compression} 压缩率会因数据集而异。 压缩率变化的程度取决于数据本身的可压缩性(高基数字段和低基数字段的数量), @@ -228,12 +228,12 @@ FROM system.tables WHERE name = <你的表名> ``` -### 如果我有自管部署,ClickHouse 提供哪些工具来预估在云端运行服务的成本? +### 如果我有自管部署,ClickHouse 提供哪些工具来预估在云端运行服务的成本? {#what-tools-does-clickhouse-offer-to-estimate-the-cost-of-running-a-service-in-the-cloud-if-i-have-a-self-managed-deployment} ClickHouse 查询日志会捕获[关键指标](/operations/system-tables/query_log),可用于估算在 ClickHouse Cloud 中运行工作负载的成本。 关于从自管环境迁移到 ClickHouse Cloud 的详细信息,请参阅[迁移文档](/cloud/migration/clickhouse-to-cloud),如有进一步问题,请联系 [ClickHouse Cloud 支持](https://console.clickhouse.cloud/support)。 -### ClickHouse Cloud 提供哪些计费选项? +### ClickHouse Cloud 提供哪些计费选项? {#what-billing-options-are-available-for-clickhouse-cloud} ClickHouse Cloud 支持以下计费选项: @@ -245,11 +245,11 @@ ClickHouse Cloud 支持以下计费选项: 用于 PAYG 的 ClickHouse Cloud credits 按 $0.01 为单位开具发票,使我们能够根据客户的实际使用量按部分 ClickHouse credits 收费。这不同于承诺支出型 ClickHouse credits,后者是以整 $1 为单位预付购买。 ::: -### 我可以删除我的信用卡吗? +### 我可以删除我的信用卡吗? {#can-i-delete-my-credit-card} 您无法在计费 UI 中移除信用卡,但可以随时更新。这有助于确保您的组织始终具有有效的付款方式。如果您需要移除信用卡,请联系 [ClickHouse Cloud 支持](https://console.clickhouse.cloud/support) 获取帮助。 -### 计费周期有多长? +### 计费周期有多长? {#how-long-is-the-billing-cycle} 计费采用月度周期,开始日期为创建 ClickHouse Cloud 组织的日期。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md index 0e247f3b8ea..1225bf6ff4a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/02_marketplace/aws-marketplace-payg.md @@ -1,7 +1,7 @@ --- slug: /cloud/billing/marketplace/aws-marketplace-payg -title: 'AWS Marketplace 按需计费(PAYG)' -description: '通过 AWS Marketplace 以按需计费(PAYG)方式订阅 ClickHouse Cloud。' +title: 'AWS Marketplace 按需付费 (PAYG)' +description: '通过 AWS Marketplace 以按需付费 (PAYG) 方式订阅 ClickHouse Cloud。' keywords: ['aws', 'marketplace', 'billing', 'PAYG'] doc_type: 'guide' --- @@ -17,68 +17,67 @@ import aws_marketplace_payg_8 from '@site/static/images/cloud/manage/billing/mar import aws_marketplace_payg_9 from '@site/static/images/cloud/manage/billing/marketplace/aws-marketplace-payg-9.png'; import Image from '@theme/IdealImage'; -通过 [AWS Marketplace](https://aws.amazon.com/marketplace) 上的按需付费(PAYG)公共报价开始使用 ClickHouse Cloud。 +在 [AWS Marketplace](https://aws.amazon.com/marketplace) 上通过 PAYG(按需付费)公共优惠开始使用 ClickHouse Cloud。 -## 先决条件 {#prerequisites} +## 前提条件 {#prerequisites} - 一个已由计费管理员开通购买权限的 AWS 账户。 - 要进行购买,您必须使用该账户登录 AWS Marketplace。 -- 若要将 ClickHouse 组织关联到您的订阅,您必须是该组织的管理员。 +- 要将 ClickHouse 组织关联到您的订阅,您必须是该组织的管理员。 :::note -一个 AWS 账户只能订阅一个 “ClickHouse Cloud - 按量付费” 订阅,并且该订阅只能关联一个 ClickHouse 组织。 +一个 AWS 账户最多只能订阅一个“ClickHouse Cloud - Pay As You Go”订阅,并且该订阅只能关联到一个 ClickHouse 组织。 ::: - - ## 注册步骤 {#steps-to-sign-up} ### 搜索 ClickHouse Cloud - Pay As You Go {#search-payg} -前往 [AWS Marketplace](https://aws.amazon.com/marketplace),搜索 “ClickHouse Cloud - Pay As You Go”。 +前往 [AWS Marketplace](https://aws.amazon.com/marketplace),搜索“ClickHouse Cloud - Pay As You Go”。 ### 查看购买选项 {#purchase-options} -点击该[产品页面](https://aws.amazon.com/marketplace/pp/prodview-p4gwofrqpkltu),然后点击 **View purchase options**。 +点击该[商品详情页](https://aws.amazon.com/marketplace/pp/prodview-p4gwofrqpkltu),然后点击 **View purchase options**。 - + ### 订阅 {#subscribe} -在下一个页面中,点击 **Subscribe**。 +在下一屏上点击 **subscribe**。 :::note -**Purchase order (PO) number(采购订单号)** 为可选项,可以忽略。 +**采购订单(PO)号** 为可选项,可以忽略。 +**此商品有两个可选报价。** 如果选择 “ClickHouse Cloud - Pay As You Go Free Trial” 报价,您将订阅一个由 AWS 管理的 30 天免费试用。需要注意的是,30 天结束后,此商品订阅将终止,您需要在该商品页面重新订阅另一个 “ClickHouse Cloud - Pay As You Go” 报价,才能继续使用 ClickHouse Pay As You Go。 ::: - + -### 设置你的账户 {#set-up-your-account} +### 设置您的账户 {#set-up-your-account} -请注意,此时设置尚未完成,你的 ClickHouse Cloud 组织尚未通过 Marketplace 计费。你现在需要在 Marketplace 订阅页面中点击 **Set up your account**,跳转到 ClickHouse Cloud 完成配置。 +请注意,此时设置尚未完成,您的 ClickHouse Cloud 组织也尚未通过 Marketplace 计费。您现在需要在 Marketplace 订阅页面中点击 **Set up your account**,跳转到 ClickHouse Cloud 完成最终设置。 - + -跳转到 ClickHouse Cloud 后,你可以使用现有账户登录,或者注册一个新账户。此步骤非常关键,以便我们将你的 ClickHouse Cloud 组织绑定到你的 AWS Marketplace 计费。 +重定向至 ClickHouse Cloud 后,您可以使用现有账户登录,或注册新账户。此步骤非常重要,以便我们将您的 ClickHouse Cloud 组织与 AWS Marketplace 计费绑定。 -:::note[新 ClickHouse Cloud 用户] -如果你是新的 ClickHouse Cloud 用户,请按照下面的步骤操作。 +:::note[ClickHouse Cloud 新用户] +如果您是 ClickHouse Cloud 新用户,请按照以下步骤操作。 :::
新用户步骤 -如果你是新的 ClickHouse Cloud 用户,请点击页面底部的 **Register**。系统会提示你创建新用户并验证邮箱。完成邮箱验证后,你可以关闭 ClickHouse Cloud 登录页面,然后在 https://console.clickhouse.cloud 使用新用户名登录。 +如果您是 ClickHouse Cloud 新用户,请点击页面底部的 **Register**。系统会提示您创建一个新用户并验证邮箱。完成邮箱验证后,您可以关闭 ClickHouse Cloud 登录页面,并在 https://console.clickhouse.cloud 使用新的用户名登录。 :::note[新用户] -你还需要提供一些关于你业务的基本信息。请参见下面的截图。 +您还需要提供一些关于您业务的基本信息。请参见下方截图。 ::: @@ -87,24 +86,22 @@ import Image from '@theme/IdealImage';
-如果你是现有的 ClickHouse Cloud 用户,只需使用你的账户凭证登录即可。 +如果您是现有的 ClickHouse Cloud 用户,只需使用您的账号凭据登录即可。 ### 将 Marketplace 订阅添加到组织 {#add-marketplace-subscription} -成功登录后,你可以选择创建一个新组织,将其计费绑定到此 Marketplace 订阅;也可以选择一个现有组织,将其计费绑定到此订阅。 +成功登录后,您可以选择创建一个新组织,将此 Marketplace 订阅的计费归到该组织;也可以选择一个现有组织,将其计费归到此订阅下。 -完成此步骤后,你的组织将连接到该 AWS 订阅,所有用量将通过你的 AWS 账户计费。 +完成此步骤后,您的组织将关联到此 AWS 订阅,所有用量都会通过您的 AWS 账户计费。 -你可以在 ClickHouse UI 中该组织的计费页面确认计费已成功关联到 AWS Marketplace。 +您可以在 ClickHouse UI 中该组织的计费页面确认,计费现在确实已关联到 AWS Marketplace。
- - ## 支持 {#support} -如果您遇到任何问题,请随时联系[我们的支持团队](https://clickhouse.com/support/program)。 +如果您遇到任何问题,请随时联系[我们的支持团队](https://clickhouse.com/support/program)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/03_clickpipes/clickpipes_for_cdc.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/03_clickpipes/clickpipes_for_cdc.md index 340d796b08f..31801f12579 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/03_clickpipes/clickpipes_for_cdc.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/03_clickpipes/clickpipes_for_cdc.md @@ -63,15 +63,15 @@ Postgres CDC 连接器分为两个主要阶段运行: #### 月度费用明细 {#cost-breakdown} -**摄取数据(CDC)** +**摄取数据(CDC)**: -2 个管道 × 500 GB = 1,000 GB 每月 +$$ 2 \text{ 个管道} \times 500 \text{ GB} = 1,000 \text{ GB 每月} $$ -1,000 GB × $0.20/GB = $200 +$$ 1,000 \text{ GB} \times \$0.20/\text{GB} = \$200 $$ **计算资源**: -1 个计算单元 × $0.20/小时 × 730 小时(约一个月) = $146 +$$1 \text{ 个计算单元} \times \$0.20/\text{小时} \times 730 \text{ 小时(约一个月)} = \$146$$ :::note 计算资源在两个管道之间共享 @@ -79,7 +79,7 @@ Postgres CDC 连接器分为两个主要阶段运行: **月度总费用**: -$200 (摄取) + $146 (计算资源) = $346 +$$\$200 \text{ (摄取)} + \$146 \text{ (计算资源)} = \$346$$ ## Postgres CDC ClickPipes 常见问题解答 {#faq-postgres-cdc-clickpipe} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx index cb106cb441e..14206f5852b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/04_network-data-transfer.mdx @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['计费', '网络传输', '数据出站', '成本', '定价'] --- -import NetworkPricing from '@site/docs/cloud/reference/_snippets/_network_transfer_rates.md'; +import NetworkPricing from '@site/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/_snippets/_network_transfer_rates.md'; ClickHouse Cloud 会监控并计量入站和出站的数据传输量。 这包括所有进出 ClickHouse Cloud 的数据,以及区域内和跨区域的数据传输。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md index ab7ffda14a9..64d77d586de 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/05_payment-thresholds.md @@ -7,9 +7,9 @@ keywords: ['计费', '支付阈值', '自动开票', '发票'] doc_type: 'guide' --- -# 付款阈值 +# 付款阈值 {#payment-thresholds} -当你在某一 ClickHouse Cloud 计费周期内的应付金额达到 10,000 美元(USD)或等值金额时,系统将会自动从你的付款方式中扣款。扣款失败将在宽限期结束后导致你的服务被暂停或终止。 +当你在某一 ClickHouse Cloud 计费周期内的应付金额达到 10,000 美元(USD)或等值金额时,系统将会自动从你的付款方式中扣款。扣款失败将在宽限期结束后导致你的服务被暂停或终止。 :::note 此付款阈值不适用于与 ClickHouse 签订了承诺消费合同或其他协商合同协议的客户。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md index f8f3454ca5d..f05658e7511 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/03_billing/06_billing_compliance.md @@ -11,7 +11,7 @@ import billing_compliance from '@site/static/images/cloud/manage/billing_complia import Image from '@theme/IdealImage'; -# ClickHouse Cloud 计费合规 +# ClickHouse Cloud 计费合规 {#clickhouse-cloud-billing-compliance} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md index 324aebcfbea..f3b5449378d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/05_supported-regions.md @@ -10,7 +10,7 @@ doc_type: 'reference' import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge' -# 受支持的云区域 +# 受支持的云区域 {#supported-cloud-regions} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md index 801199cb76b..161cd7408f4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/08_settings.md @@ -10,8 +10,7 @@ doc_type: 'guide' import Image from '@theme/IdealImage'; import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-settings-sidebar.png'; - -# 配置设置 +# 配置设置 {#configuring-settings} 要为特定[用户](/operations/access-rights#user-account-management)或[角色](/operations/access-rights#role-management)指定 ClickHouse Cloud 服务的设置,必须使用[基于 SQL 的 Settings Profiles](/operations/access-rights#settings-profiles-management)。应用 Settings Profiles 可以确保你配置的设置在服务停止、处于空闲状态或升级时仍然保持不变。要了解有关 Settings Profiles 的更多信息,请参阅[此页面](/operations/settings/settings-profiles.md)。 @@ -19,4 +18,4 @@ import cloud_settings_sidebar from '@site/static/images/cloud/manage/cloud-setti 若要进一步了解可以为 ClickHouse Cloud 服务指定的设置,请在[我们的文档](/operations/settings)中按类别查看所有可用设置。 - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md index 6fb5b3a42bf..633855714f0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/09_security/03_compliance-overview.md @@ -10,7 +10,7 @@ import BetaBadge from '@theme/badges/BetaBadge'; import EnterprisePlanFeatureBadge from '@theme/badges/EnterprisePlanFeatureBadge'; -# 安全与合规报告 +# 安全与合规报告 {#security-and-compliance-reports} ClickHouse 会评估客户在安全与合规方面的需求,并会根据额外报告请求持续扩展相关项目。有关更多信息或下载报告,请访问我们的[信任中心(Trust Center)](https://trust.clickhouse.com)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md index 10950b45226..dd2adbaceab 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/11_account-close.md @@ -37,7 +37,7 @@ doc_type: 'guide' -## 申请关闭账户 +## 申请关闭账户 {#request-account-closure} 我们必须对关闭和删除请求进行身份验证。为确保您的请求能够尽快处理,请按照以下步骤操作。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/index.md index d336123b7e0..0e4cf8d4043 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/cloud/reference/index.md @@ -7,7 +7,7 @@ description: 'Cloud 参考部分的概览页' doc_type: 'landing-page' --- -# 云端参考 +# 云端参考 {#cloud-reference} 本节作为 ClickHouse Cloud 更多技术细节的参考指南,包含以下页面: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/glossary.md b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/glossary.md index a4e8dc8d71e..a7ffce3acd9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/glossary.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/glossary.md @@ -10,7 +10,7 @@ doc_type: 'reference' {/* no-glossary */ } -# 词汇表 +# 词汇表 {#glossary} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/olap.md b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/olap.md index d364b33754b..d3d67924e83 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/olap.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/olap.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# 什么是 OLAP? +# 什么是 OLAP? {#what-is-olap} [OLAP](https://en.wikipedia.org/wiki/Online_analytical_processing) 是 Online Analytical Processing(联机分析处理)的缩写。这个术语范围很广,可以从两个角度来理解:技术角度和业务角度。在最高层面上,你可以简单地将这个短语反向理解: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx index 98ea29515bb..9a151e45baf 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/concepts/why-clickhouse-is-so-fast.mdx @@ -18,7 +18,7 @@ doc_type: 'guide' ## 存储层:并发插入彼此之间是相互隔离的 \{#storage-layer-concurrent-inserts-are-isolated-from-each-other\} - + + + + + + + \ No newline at end of file + - + -## Projection 如何工作? {#how-do-projections-work} +## Projections 如何工作? {#how-do-projections-work} -在实际应用中,可以将 Projection 看作是原始表上的一个额外的、隐藏的表。Projection 可以具有与原始表不同的行排序,因此其主索引也可以不同,并且它可以自动、增量地预计算聚合值。因此,使用 Projection 为加速查询执行提供了两个“调优手段”: +在实践中,可以将 Projection 看作是原始表的一个额外的隐藏表。Projection 可以拥有与原始表不同的行顺序,因此也可以拥有不同的主索引,并且它可以自动、增量地预计算聚合值。因此,使用 Projections 可以通过两个“调优手段”来加速查询执行: - **正确使用主索引** - **预计算聚合** -Projection 在某些方面与 [物化视图](/materialized-views) 类似,它们同样允许你拥有多种行排序方式,并在插入时预计算聚合。 -Projection 会自动更新并与原始表保持同步,而物化视图则需要显式更新。当查询针对原始表时,ClickHouse 会自动采样主键并选择一个既能生成同样正确结果、又需要读取数据量最少的表,如下图所示: +Projections 在某些方面类似于 [物化视图](/materialized-views) +,它们同样允许你使用多种行顺序,并在插入时预计算聚合。 +Projections 会自动更新,并与原始表保持同步;这与物化视图(Materialized Views)不同,后者需要显式更新。当查询针对原始表时, +ClickHouse 会自动对主键进行采样,并选择一个既能生成相同正确结果、又需要读取数据量最少的表,如下图所示: -### 通过 `_part_offset` 实现更智能的存储 {#smarter_storage_with_part_offset} - -从 25.5 版本开始,ClickHouse 在 Projection 中支持虚拟列 `_part_offset`,它提供了一种定义 Projection 的新方式。 - -现在有两种定义 Projection 的方式: +### 使用 `_part_offset` 的更智能存储 {#smarter_storage_with_part_offset} -- **存储完整列(原有行为)**:Projection 包含完整数据,可以被直接读取,当过滤条件与 Projection 的排序顺序匹配时性能更高。 +从 25.5 版本开始,ClickHouse 在 projection 中支持虚拟列 `_part_offset`,这为定义 projection 提供了一种新的方式。 -- **仅存储排序键 + `_part_offset`**:Projection 的工作方式类似索引。 - ClickHouse 使用 Projection 的主索引定位匹配的行,但从基表中读取实际数据。这样可以降低存储开销,但在查询时会稍微增加 I/O。 +现在有两种定义 projection 的方式: -上述两种方式也可以混合使用,将部分列直接存储在 Projection 中,而其他列通过 `_part_offset` 间接访问。 +- **存储完整列(原有行为)**:projection 包含完整数据,可以被直接读取,当过滤条件与 projection 的排序顺序匹配时,可以获得更快的查询性能。 +- **仅存储排序键 + `_part_offset`**:projection 的工作方式类似索引。 + ClickHouse 使用 projection 的主索引来定位匹配的行,但从基表中读取实际数据。 + 这样可以减少存储开销,但在查询时会略微增加 I/O。 +上述方法也可以混合使用,在 projection 中直接存储部分列,而通过 `_part_offset` 间接存储其他列。 ## 何时使用 Projections? {#when-to-use-projections} -Projections 对新用户来说是一个颇具吸引力的功能,因为它会在数据插入时自动维护。此外,查询只需发送到单个表,查询优化器会在可能的情况下利用 Projections 来加快响应时间。 +Projections 对新用户而言非常有吸引力,因为它们会在数据插入时自动维护。并且,查询只需发送到单个表,并在可能时利用 projections 来加速响应时间。 -这与 Materialized Views 形成对比:使用后者时,用户必须根据过滤条件选择合适的已优化目标表,或重写查询。这会对用户应用提出更高要求,并增加客户端侧的复杂度。 +这与物化视图不同,后者要求用户根据过滤条件选择适当的已优化目标表,或者重写查询。这会对用户应用提出更高要求,并增加客户端的复杂度。 -尽管有这些优势,Projections 也存在一些固有的限制,用户应当了解,因此应谨慎、少量地部署。 +尽管有这些优点,projections 也存在一些固有限制,用户应当了解这些限制,因此应谨慎使用,避免过度依赖。 -- Projections 不允许为源表和(隐藏的)目标表使用不同的 TTL,而 Materialized Views 允许使用不同的 TTL。 -- 启用 Projections 的表不支持轻量级更新和删除。 -- Materialized Views 可以链式使用:一个 Materialized View 的目标表可以作为另一个 Materialized View 的源表,依此类推。而 Projections 不支持这种用法。 -- Projections 不支持 join,而 Materialized Views 支持。 -- Projections 不支持过滤(`WHERE` 子句),而 Materialized Views 支持。 +- Projections 不允许对源表和(隐藏的)目标表使用不同的 TTL,而物化视图允许使用不同的 TTL。 +- 对包含 projections 的表,不支持轻量级更新和删除。 +- 物化视图可以链式使用:一个物化视图的目标表可以作为另一个物化视图的源表,依此类推。而 projections 无法做到这一点。 +- Projection 定义不支持 join,但物化视图支持。不过,对包含 projections 的表进行的查询可以自由使用 join。 +- Projection 定义不支持过滤条件(`WHERE` 子句),但物化视图支持。不过,对包含 projections 的表进行的查询可以自由使用过滤条件。 -我们建议在以下场景中使用 Projections: +我们建议在以下场景使用 projections: -- 需要对数据进行完全重新排序时。虽然 Projection 中的表达式理论上可以使用 `GROUP BY`,但 Materialized Views 在维护聚合方面更高效。查询优化器也更有可能利用只进行简单重排的 Projections,即 `SELECT * ORDER BY x`。用户可以在该表达式中仅选择部分列,以降低存储占用。 -- 用户能够接受潜在的存储占用增加以及将数据写入两次所带来的开销。请测试对插入速度的影响,并[评估存储开销](/data-compression/compression-in-clickhouse)。 +- 需要对数据进行完全重新排序时。虽然理论上,projection 中的表达式可以使用 `GROUP BY`,但物化视图在维护聚合方面更有效。查询优化器也更有可能利用仅做简单重排的 projections,即 `SELECT * ORDER BY x`。用户可以在该表达式中选择列的子集,以减少存储占用。 +- 用户能够接受可能带来的存储占用增加,以及将数据写入两次的额外开销时。请测试其对插入速度的影响,并[评估存储开销](/data-compression/compression-in-clickhouse)。 +## 示例 {#examples} +### 在非主键列上进行过滤 {#filtering-without-using-primary-keys} -## 示例 +在这个示例中,我们将向你展示如何为表添加一个投影(projection)。 +我们还将了解如何利用该投影加速在非主键列上进行过滤的查询。 -### 在不属于主键的列上进行过滤 +在本示例中,我们将使用 New York Taxi Data +数据集(可在 [sql.clickhouse.com](https://sql.clickhouse.com/) 获取),该数据集按 `pickup_datetime` 排序。 -在这个示例中,我们将介绍如何为一张表添加一个 projection。 -同时,我们还将说明如何使用该 projection 来加速在不属于表主键的列上进行过滤的查询。 - -在本示例中,我们将使用可在 [sql.clickhouse.com](https://sql.clickhouse.com/) 获取的 New York Taxi Data -数据集,该数据集按 `pickup_datetime` 排序。 - -让我们编写一个简单的查询,找出所有乘客给司机小费 -大于 200 美元的行程 ID: +现在我们来编写一个简单的查询语句,找出所有乘客 +给司机小费超过 200 美元的行程 ID: ```sql runnable SELECT @@ -99,34 +96,35 @@ FROM nyc_taxi.trips WHERE tip_amount > 200 AND trip_duration_min > 0 ORDER BY tip_amount, trip_id ASC ``` -请注意,由于我们在过滤不在 `ORDER BY` 中的 `tip_amount`,ClickHouse 不得不执行全表扫描。下面我们来加快这个查询。 +请注意,由于我们正在对未包含在 `ORDER BY` 中的 `tip_amount` 进行过滤,ClickHouse +不得不执行一次全表扫描。我们来加速这个查询。 -为了保留原始表和查询结果,我们将创建一个新表,并使用 `INSERT INTO SELECT` 来复制数据: +为了保留原始表及其结果,我们将创建一个新表,并使用 `INSERT INTO SELECT` 来复制数据: ```sql CREATE TABLE nyc_taxi.trips_with_projection AS nyc_taxi.trips; INSERT INTO nyc_taxi.trips_with_projection SELECT * FROM nyc_taxi.trips; ``` -要添加一个 projection,我们使用 `ALTER TABLE` 语句结合 `ADD PROJECTION` 语句: +要添加投影,我们使用 `ALTER TABLE` 语句配合 `ADD PROJECTION` 语句: ```sql ALTER TABLE nyc_taxi.trips_with_projection -添加 投影 prj_tip_amount +ADD PROJECTION prj_tip_amount ( SELECT * ORDER BY tip_amount, dateDiff('minutes', pickup_datetime, dropoff_datetime) ) ``` -在添加 projection 之后,必须使用 `MATERIALIZE PROJECTION` -语句,这样其中的数据才会根据上面指定的查询进行物理排序并重写: +在添加 projection 之后,需要使用 `MATERIALIZE PROJECTION` +语句,使其中的数据根据上面指定的查询进行物理排序并重写: ```sql ALTER TABLE nyc.trips_with_projection MATERIALIZE PROJECTION prj_tip_amount ``` -现在我们已经添加了投影,让我们再运行一次这个查询: +现在我们已经添加了投影,再来运行一次该查询: ```sql runnable SELECT @@ -137,9 +135,9 @@ FROM nyc_taxi.trips_with_projection WHERE tip_amount > 200 AND trip_duration_min ORDER BY tip_amount, trip_id ASC ``` -请注意,我们显著减少了查询时间,并且需要扫描的行数也更少了。 +请注意,我们显著减少了查询时间,且需要扫描的行数也更少了。 -我们可以通过查询 `system.query_log` 表来确认上述查询确实使用了我们创建的投影: +我们可以通过查询 `system.query_log` 表来确认上面的查询确实使用了我们创建的投影: ```sql SELECT query, projections @@ -157,29 +155,30 @@ WHERE query_id='' └───────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────┘ ``` -### 使用投影加速英国房价支付查询 -为了演示如何使用投影来提升查询性能,我们来看一个基于真实数据集的示例。在本示例中,我们将使用来自我们的 [UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid) -教程中的表,该表包含 3003 万行记录。该数据集同样可以在我们的 +### 使用投影加速英国房价已付数据查询 {#using-projections-to-speed-up-UK-price-paid} + +为了演示如何使用投影来加速查询性能,我们 +通过一个真实数据集的示例来说明。在本示例中,我们将 +使用 [UK Property Price Paid](https://clickhouse.com/docs/getting-started/example-datasets/uk-price-paid) +教程中的表,该表包含 3003 万行数据。此数据集也可在 [sql.clickhouse.com](https://sql.clickhouse.com/?query_id=6IDMHK3OMR1C97J6M9EUQS) -环境中使用。 +环境中获取。 -如果您希望了解该表是如何创建以及数据是如何插入的,可以参考《[The UK property prices dataset](/getting-started/example-datasets/uk-price-paid)》页面。 +如果您想了解表的创建方式和数据插入过程,可以参阅 ["英国房产价格数据集"](/getting-started/example-datasets/uk-price-paid) 页面。 -我们可以在该数据集上运行两个简单的查询。第一个查询列出伦敦房价最高的各郡, -第二个则计算各个郡的平均价格: +我们可以对此数据集运行两个简单的查询。第一个查询列出伦敦地区支付价格最高的郡县,第二个查询计算各郡县的平均价格: ```sql runnable SELECT county, price FROM uk.uk_price_paid -WHERE town = '伦敦' +WHERE town = 'LONDON' ORDER BY price DESC LIMIT 3 ``` - ```sql runnable SELECT county, @@ -190,8 +189,7 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -请注意,尽管查询速度非常快,这两条查询仍然对整张表的 3003 万行数据进行了全表扫描, -这是因为在创建表时的 `ORDER BY` 子句中,既没有包含 `town` 列,也没有包含 `price` 列: +请注意,尽管查询速度很快,但两个查询都对全部 3003 万行进行了全表扫描,这是因为在创建表时,`town` 和 `price` 都不在 ORDER BY 语句中: ```sql CREATE TABLE uk.uk_price_paid @@ -203,18 +201,16 @@ ENGINE = MergeTree ORDER BY (postcode1, postcode2, addr1, addr2); ``` -让我们看看是否可以使用投影来加速这个查询。 +让我们看看能否使用投影来加速这个查询。 -为保留原始表及其结果,我们将创建一个新表,并使用 `INSERT INTO SELECT` 复制数据: +为了保留原始表和结果,我们将创建一个新表并使用 `INSERT INTO SELECT` 复制数据: ```sql CREATE TABLE uk.uk_price_paid_with_projections AS uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections SELECT * FROM uk.uk_price_paid; ``` -我们创建并填充投影 `prj_oby_town_price`,它会生成一个附加的(隐藏的)表, -该表带有主索引,并按 town 和 price 排序,用于优化如下查询: -在指定 town 中列出最高价格对应的 counties。 +我们创建并填充投影 `prj_oby_town_price`,它会生成一个额外的(隐藏)表,该表具有主索引,按城镇和价格排序,用于优化查询特定城镇中最高成交价格的郡县列表: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -233,10 +229,10 @@ ALTER TABLE uk.uk_price_paid_with_projections SETTINGS mutations_sync = 1 ``` -[`mutations_sync`](/operations/settings/settings#mutations_sync) 设置用于强制以同步方式执行。 +[`mutations_sync`](/operations/settings/settings#mutations_sync) 设置用于强制执行同步操作。 -我们创建并填充投影 `prj_gby_county` —— 一个额外的(隐藏的)表, -用于以增量方式预先计算所有现有的 130 个英国郡的 avg(price) 聚合值: +我们创建并填充投影 `prj_gby_county` —— 一个额外的(隐藏)表, +用于增量预计算所有现有 130 个英国郡的 avg(price) 聚合值: ```sql ALTER TABLE uk.uk_price_paid_with_projections @@ -256,14 +252,14 @@ SETTINGS mutations_sync = 1 ``` :::note -如果在如上面 `prj_gby_county` 投影那样的投影中使用了 `GROUP BY` 子句,那么(隐藏)表的底层存储引擎会变为 `AggregatingMergeTree`,并且所有聚合函数都会被转换为 `AggregateFunction`。这可以确保正确地进行增量数据聚合。 +如果在投影中使用了 `GROUP BY` 子句(例如上面的 `prj_gby_county` 投影),则(隐藏)表的底层存储引擎会变为 `AggregatingMergeTree`,所有聚合函数会被转换为 `AggregateFunction`。这可确保正确的增量数据聚合。 ::: -下图是主表 `uk_price_paid_with_projections` 及其两个投影的可视化示意图: +下图展示了主表 `uk_price_paid_with_projections` 及其两个投影的可视化: - + -如果现在再次运行查询,列出伦敦中成交价格最高的三个价格及其所属郡,我们就会看到查询性能有所提升: +如果现在再次运行查询以列出伦敦成交价格最高的三个区县,可以看到查询性能有所提升: ```sql runnable SELECT @@ -275,7 +271,7 @@ ORDER BY price DESC LIMIT 3 ``` -同样,对于列出平均支付价格最高的三个英国郡县的查询: +同样,对于列出英国平均房价最高的三个郡的查询: ```sql runnable SELECT @@ -287,19 +283,19 @@ ORDER BY avg(price) DESC LIMIT 3 ``` -请注意,这两个查询都是针对原始表执行的,并且在我们创建这两个 projection 之前,这两个查询都会进行一次全表扫描(从磁盘流式读取全部 3003 万行数据)。 +请注意,这两个查询都针对原始表,并且在创建这两个投影之前,两个查询都执行了全表扫描(从磁盘读取了全部 3003 万行数据)。 -另外请注意,用于列出伦敦中支付价格最高的前三个郡的查询,会流式读取 217 万行数据。而当我们直接使用一张专门为该查询优化的第二张表时,仅有 8.192 万行数据从磁盘被流式读取。 +另外请注意,用于列出伦敦中成交价最高的三个县(county)的查询正在流式处理 217 万行数据。而当我们直接使用为该查询优化的第二张表时,只从磁盘流式读取了 8.192 万行数据。 -造成这一差异的原因在于,目前上文提到的 `optimize_read_in_order` 优化尚不支持用于 projection。 +造成这一差异的原因是,目前上文提到的 `optimize_read_in_order` 优化尚不支持应用于投影(projection)。 -我们检查 `system.query_log` 表,以确认 ClickHouse 为上述两个查询自动使用了这两个 projection(见下方的 projections 列): +我们检查 `system.query_log` 表,可以看到 ClickHouse 在上面的两个查询中自动使用了两个投影(参见下面的 projections 列): ```sql SELECT tables, query, - query_duration_ms::String || ' 毫秒' AS query_duration, + query_duration_ms::String || ' ms' AS query_duration, formatReadableQuantity(read_rows) AS read_rows, projections FROM clusterAllReplicas(default, system.query_log) @@ -309,7 +305,6 @@ ORDER BY initial_query_start_time DESC FORMAT Vertical ``` - ```response 第 1 行: ────── @@ -343,20 +338,22 @@ projections: ['uk.uk_price_paid_with_projections.prj_obj_town_price'] 返回 2 行。耗时:0.006 秒。 ``` -### 更多示例 -以下示例使用相同的英国价格数据集,对比使用和不使用投影的查询。 +### 更多示例 {#further-examples} + +以下示例继续使用相同的英国价格数据集,对比使用和不使用投影的查询。 -为了保持原始表(及其性能)不受影响,我们再次使用 `CREATE AS` 和 `INSERT INTO SELECT` 创建该表的副本。 +为了保持原始表及其性能不受影响,我们再次使用 `CREATE AS` 和 `INSERT INTO SELECT` 创建该表的副本。 ```sql CREATE TABLE uk.uk_price_paid_with_projections_v2 AS uk.uk_price_paid; INSERT INTO uk.uk_price_paid_with_projections_v2 SELECT * FROM uk.uk_price_paid; ``` -#### 构建 Projection -让我们基于维度 `toYear(date)`、`district` 和 `town` 创建一个聚合 Projection: +#### 构建 Projection {#build-projection} + +让我们基于 `toYear(date)`、`district` 和 `town` 这三个维度创建一个聚合 Projection: ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -376,7 +373,7 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 ) ``` -为现有数据填充该投影。(如果不进行物化,则该投影只会为之后新插入的数据创建): +为已有数据填充该 projection。(如果不进行物化,则该 projection 只会针对新插入的数据创建): ```sql ALTER TABLE uk.uk_price_paid_with_projections_v2 @@ -384,9 +381,10 @@ ALTER TABLE uk.uk_price_paid_with_projections_v2 SETTINGS mutations_sync = 1 ``` -下面的查询对比了启用和未启用投影(projection)时的性能。要禁用投影,我们使用设置项 [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections),该设置默认开启。 +以下查询对比了启用和未启用投影时的性能。若要禁用投影功能,请使用设置 [`optimize_use_projections`](/operations/settings/settings#optimize_use_projections),该设置默认是启用的。 -#### 查询 1:每年平均价格 + +#### 查询 1:各年份的平均价格 {#average-price-projections} ```sql runnable SELECT @@ -410,9 +408,10 @@ ORDER BY year ASC ``` -结果应该相同,但后一个示例的性能会更好! +结果应该是相同的,但后一个示例的性能会更优! + -#### 查询 2:伦敦每年的平均价格 +#### 查询 2:伦敦历年平均价格 {#average-price-london-projections} ```sql runnable SELECT @@ -437,9 +436,10 @@ GROUP BY year ORDER BY year ASC ``` -#### 查询 3:最昂贵的社区 -需要将条件 (date >= '2020-01-01') 修改为与投影维度 (`toYear(date) >= 2020)` 一致: +#### 查询 3:最昂贵的街区 {#most-expensive-neighborhoods-projections} + +条件 (date >= '2020-01-01') 需要进行修改,以便与投影维度 (`toYear(date) >= 2020)` 保持一致: ```sql runnable SELECT @@ -459,7 +459,6 @@ LIMIT 100 SETTINGS optimize_use_projections=0 ``` - ```sql runnable SELECT town, @@ -477,17 +476,22 @@ ORDER BY price DESC LIMIT 100 ``` -同样,结果是相同的,但请注意第二个查询的性能有所提升。 +同样,结果相同,但请注意第二个查询的执行性能有所提升。 + -### 在一个查询中组合多个 projection +### 在单个查询中组合投影 {#combining-projections} -从 25.6 版本开始,在上一版本引入对 `_part_offset` 的支持基础上,ClickHouse 现在可以在带有多个过滤条件的单个查询中使用多个 projection 来加速查询。 +从 25.6 版本开始,在前一版本引入的 `_part_offset` 支持基础之上,ClickHouse +现在可以在带有多个过滤条件的单个查询中使用多个投影来加速查询。 -需要强调的是,ClickHouse 仍然只会从一个 projection(或基础表)中读取数据,但可以利用其他 projection 的主索引在读取前裁剪掉不必要的 part。对于在多个列上进行过滤、且每个过滤条件可能匹配到不同 projection 的查询,这一特性尤其有用。 +需要注意的是,ClickHouse 仍然只会从一个投影(或基础表)中读取数据, +但可以利用其他投影的主键索引在读取之前剪枝掉不必要的 part。 +这对于在多个列上进行过滤,且每一列可能分别匹配到不同投影的查询尤其有用。 -> 当前,该机制只能裁剪整个 part,尚不支持粒度级别的裁剪。 +> 当前,该机制只会剪枝整个 part。尚不支持粒度级(granule-level)的剪枝。 -为了演示这一点,我们定义了一个表(其 projection 使用 `_part_offset` 列),并插入了五行示例数据,以对应上面的示意图。 +为演示这一点,我们定义表(带有使用 `_part_offset` 列的投影), +并插入与上文图示相对应的五行示例数据。 ```sql CREATE TABLE page_views @@ -513,7 +517,7 @@ SETTINGS max_bytes_to_merge_at_max_space_in_pool = 1; -- 禁用合并 ``` -然后向该表中插入数据: +然后向表中插入数据: ```sql INSERT INTO page_views VALUES ( @@ -529,72 +533,71 @@ INSERT INTO page_views VALUES ( ``` :::note -注意:此表为了演示使用了自定义设置,例如单行 granule 和禁用 part 合并,这些设置不推荐用于生产环境。 +注意:该表为了演示使用了自定义设置,例如单行 granule(粒度单元)以及禁用 part 合并,这些设置不建议在生产环境中使用。 ::: 此设置会产生: * 五个独立的 part(每插入一行生成一个 part) -* 每行一个主键索引项(在基础表和每个 projection 中都是如此) -* 每个 part 中正好包含一行数据 +* 每行一个主键索引条目(在基础表和每个 projection 中) +* 每个 part 都只包含一行 -在此设置下,我们运行一个同时在 `region` 和 `user_id` 上过滤的查询。 +在此设置下,我们运行一个同时按 `region` 和 `user_id` 进行过滤的查询。 由于基础表的主键索引是基于 `event_date` 和 `id` 构建的, -在这里并没有帮助,因此 ClickHouse 会使用: +在这里帮不上忙,因此 ClickHouse 会改为使用: * `region_proj` 按 `region` 剪枝 part * `user_id_proj` 进一步按 `user_id` 剪枝 part -可以通过 `EXPLAIN projections = 1` 看到这一行为,它展示了 -ClickHouse 如何选择并应用 projection。 +可以使用 `EXPLAIN projections = 1` 观察这一行为,它会展示 ClickHouse 如何选择并应用 projection。 ```sql EXPLAIN projections=1 SELECT * FROM page_views WHERE region = 'us_west' AND user_id = 107; ``` - ```response ┌─explain────────────────────────────────────────────────────────────────────────────────┐ - 1. │ 表达式 ((投影名称 + 投影)) │ - 2. │ 表达式 │ - 3. │ 从 MergeTree 读取 (default.page_views) │ + 1. │ Expression ((投影名称 + 投影)) │ + 2. │ Expression │ + 3. │ ReadFromMergeTree (default.page_views) │ 4. │ 投影: │ 5. │ 名称: region_proj │ - 6. │ 描述: 投影已分析并用于数据分片级过滤 │ + 6. │ 描述: 投影已分析并用于数据分片级别过滤 │ 7. │ 条件: (region in ['us_west', 'us_west']) │ 8. │ 搜索算法: 二分查找 │ - 9. │ 数据分片数: 3 │ -10. │ 标记数: 3 │ -11. │ 范围数: 3 │ + 9. │ 数据分片: 3 │ +10. │ 标记: 3 │ +11. │ 范围: 3 │ 12. │ 行数: 3 │ -13. │ 已过滤数据分片数: 2 │ +13. │ 已过滤数据分片: 2 │ 14. │ 名称: user_id_proj │ -15. │ 描述: 投影已分析并用于数据分片级过滤 │ +15. │ 描述: 投影已分析并用于数据分片级别过滤 │ 16. │ 条件: (user_id in [107, 107]) │ 17. │ 搜索算法: 二分查找 │ -18. │ 数据分片数: 1 │ -19. │ 标记数: 1 │ -20. │ 范围数: 1 │ +18. │ 数据分片: 1 │ +19. │ 标记: 1 │ +20. │ 范围: 1 │ 21. │ 行数: 1 │ -22. │ 已过滤数据分片数: 2 │ +22. │ 已过滤数据分片: 2 │ └────────────────────────────────────────────────────────────────────────────────────────┘ ``` -`EXPLAIN` 的输出(如上所示)从上到下展示了逻辑查询计划: +`EXPLAIN` 的输出(如上所示)自上而下展示了逻辑查询计划: -| 行号 | 描述 | -| ----- | ----------------------------------------------------------------------------- | -| 3 | 计划从 `page_views` 基表读取数据 | -| 5-13 | 使用 `region_proj` 来定位满足 region = 'us_west' 的 3 个分片,剪枝掉 5 个分片中的 2 个 | -| 14-22 | 使用 `user_id_proj` 来定位满足 `user_id = 107` 的 1 个分片,进一步剪枝掉剩余 3 个分片中的 2 个 | -最终,基表中只读取了 **5 个分片中的 1 个**。 -通过结合多个投影的索引分析,ClickHouse 显著减少了扫描的数据量, -在保持较低存储开销的同时提升了性能。 +| 行号 | 描述 | +|------|---------------------------------------------------------------------------------------------------| +| 3 | 计划从 `page_views` 基表中读取数据 | +| 5-13 | 使用 `region_proj` 来定位 `region = 'us_west'` 的 3 个分片,从而在 5 个分片中裁剪掉 2 个 | +| 14-22| 使用 `user_id_proj` 来定位 `user_id = 107` 的 1 个分片,进一步从剩余的 3 个分片中再裁剪掉 2 个 | +最终,只需要从基表中读取 **5 个分片中的 1 个**。 +通过结合多个 projection 的索引分析结果,ClickHouse 能显著减少扫描的数据量, +在保持存储开销较低的同时提升性能。 ## 相关内容 {#related-content} -- [ClickHouse 主索引实用介绍](/guides/best-practices/sparse-primary-indexes#option-3-projections) + +- [ClickHouse 主键索引实用入门指南](/guides/best-practices/sparse-primary-indexes#option-3-projections) - [物化视图](/docs/materialized-views) -- [ALTER PROJECTION](/sql-reference/statements/alter/projection) +- [ALTER PROJECTION](/sql-reference/statements/alter/projection) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md b/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md index 285e324dfef..1458736cce9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/projections/2_materialized-views-versus-projections.md @@ -3,80 +3,74 @@ slug: /managing-data/materialized-views-versus-projections sidebar_label: '物化视图 vs 投影' title: '物化视图与投影' hide_title: false -description: '本文比较 ClickHouse 中的物化视图与投影,包括它们的使用场景、性能以及局限性。' +description: '本文比较 ClickHouse 中的物化视图和投影,包括它们的使用场景、性能以及局限性。' doc_type: 'reference' keywords: ['物化视图', '投影', '差异'] --- -> 用户经常会问:在什么情况下应该使用物化视图,而不是投影?在本文中,我们将探讨二者之间的关键差异,以及在某些场景下为何更适合选择其中一种而非另一种。 +> 用户经常会问:在什么情况下应该使用物化视图,而什么时候应该使用投影?在本文中,我们将探讨二者之间的关键差异,以及在某些场景下为何可能更适合选择其中之一。 +## 关键差异概览 {#key-differences} +下表总结了在不同考量维度下,物化视图与投影之间的关键差异。 -## 关键差异总结 {#key-differences} +| Aspect | Materialized views | Projections | +|----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Data storage and location | 将结果存储在一个**单独、显式的目标表**中,在向源表插入数据时充当 `INSERT` 触发器。 | 投影会创建经过优化的数据布局,这些数据在物理上**与主表数据一起存储**,并且对用户不可见。 | +| Update mechanism | 在对源表执行 `INSERT` 时(对于增量物化视图)**同步**运行。注意:也可以通过可刷新的物化视图按计划进行**调度**刷新。 | 在对主表执行 `INSERT` 时,在后台进行**异步**更新。 | +| Query interaction | 使用物化视图时需要**直接查询目标表**,这意味着在编写查询时,用户必须意识到物化视图的存在。 | 投影由 ClickHouse 查询优化器**自动选择**,对用户是透明的,用户无需为利用投影而修改针对该表的查询。从 25.6 版本开始,还可以基于多个投影进行过滤。 | +| Handling `UPDATE` / `DELETE` | 对源表上的 `UPDATE` 或 `DELETE` 操作**不会自动做出反应**,因为物化视图并不了解源表,仅作为对源表的 `INSERT` 触发器。这可能导致源表和目标表之间的数据陈旧,需要通过变通方案或定期全量刷新(通过可刷新的物化视图)来解决。 | 默认情况下,**与 `DELETED` 行不兼容**(尤其是轻量级删除)。可以通过 `lightweight_mutation_projection_mode`(v24.7+)启用兼容性。 | +| `JOIN` support | 支持。可刷新的物化视图可用于复杂的反规范化。增量物化视图仅在最左侧表插入时触发。 | 不支持。在投影定义中不支持使用 `JOIN` 操作来过滤物化后的数据。不过,查询在与包含投影的表进行 `JOIN` 时可以正常工作——投影会优化单个表的访问。 | +| `WHERE` clause in definition | 支持。可以包含 `WHERE` 子句以在物化之前过滤数据。 | 不支持。在投影定义中不支持使用 `WHERE` 子句来过滤物化后的数据。 | +| Chaining capabilities | 支持,一个物化视图的目标表可以作为另一个物化视图的源表,从而实现多阶段流水线。 | 不支持。投影不能进行链式组合。 | +| Applicable table engines | 可用于多种源表引擎,但目标表通常属于 `MergeTree` 家族。 | **仅可用于** `MergeTree` 家族表引擎。 | +| Failure handling | 在插入数据期间发生故障意味着目标表中的数据会丢失,从而导致潜在的不一致性。 | 故障在后台被**静默**处理。查询可以无缝混合已物化和未物化的数据分片。 | +| Operational overhead | 需要显式创建目标表,并且通常需要手动回填历史数据。使用 `UPDATE`/`DELETE` 保持一致性会增加复杂度。 | 投影自动维护并保持同步,通常具有更低的运维负担。 | +| `FINAL` query compatibility | 通常兼容,但往往需要在目标表上使用 `GROUP BY`。 | **无法与** `FINAL` 查询配合使用。 | +| Lazy materialization | 支持。 | 使用延迟物化特性时需要监控投影兼容性问题。可能需要设置 `query_plan_optimize_lazy_materialization = false`。 | +| Parallel replicas | 支持。 | 不支持。 | +| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | 支持。 | 支持。 | +| Lightweight updates and deletes | 支持。 | 不支持。 | -下表总结了在各个考量维度上,物化视图与投影之间的主要差异。 - -| 方面 | 物化视图 | 投影 | -|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| 数据存储与位置 | 将结果存储在一个**单独、显式的目标表**中,在向源表插入数据时充当插入触发器。 | 投影会创建经过优化的数据布局,这些数据在物理上**与主表数据一同存储**,并且对用户不可见。 | -| 更新机制 | 在对源表执行 `INSERT` 时(对于增量物化视图)**同步**运行。注意:也可以通过可刷新的物化视图将其设置为**按计划调度**。 | 在对主表执行 `INSERT` 后,于后台**异步**更新。 | -| 查询交互 | 使用物化视图时,需要**直接查询目标表**,这意味着用户在编写查询时需要了解物化视图的存在。 | 投影会由 ClickHouse 的查询优化器**自动选择**,对用户是透明的,用户无需为了使用投影而修改针对包含投影的表的查询。从 25.6 版本开始,也可以按照多个投影进行过滤。 | -| 处理 `UPDATE` / `DELETE` | 对源表上的 `UPDATE` 或 `DELETE` 操作**不会自动响应**,因为物化视图不了解源表,仅充当源表的插入触发器。这可能导致源表与目标表之间的数据陈旧,需要通过变通方案或定期全量刷新(通过可刷新物化视图)来解决。 | 默认情况下,与被标记为 **`DELETED` 的行不兼容**(尤其是轻量级删除)。可以通过启用 `lightweight_mutation_projection_mode`(v24.7+)来实现兼容性。 | -| `JOIN` 支持 | 支持。可刷新物化视图可用于复杂的反规范化场景。增量物化视图仅在最左侧表插入时触发。 | 不支持。投影定义中不支持 `JOIN` 操作来过滤物化后的数据。 | -| 定义中的 `WHERE` 子句 | 支持。可以在定义中包含 `WHERE` 子句,以在物化前过滤数据。 | 不支持。投影定义中不支持使用 `WHERE` 子句来过滤物化数据。 | -| 串联能力 | 支持。一个物化视图的目标表可以作为另一个物化视图的源表,从而实现多阶段流水线。 | 不支持。投影不能被串联使用。 | -| 适用的表引擎 | 可以与多种源表引擎配合使用,但目标表通常使用 `MergeTree` 系列表引擎。 | **仅适用于** `MergeTree` 系列表引擎。 | -| 失败处理 | 在插入数据期间发生失败时,目标表中的数据会丢失,从而可能导致不一致。 | 失败会在后台被**静默**处理。查询可以无缝混合使用已物化和未物化的数据片段。 | -| 运维开销 | 需要显式创建目标表,并且通常需要手动回填数据。使用 `UPDATE`/`DELETE` 维持一致性会增加复杂度。 | 投影会自动维护并保持同步,通常具有更低的运维负担。 | -| `FINAL` 查询兼容性 | 一般兼容,但通常需要在目标表上使用 `GROUP BY`。 | 与 `FINAL` 查询**不兼容**。 | -| 惰性物化 | 支持。 | 使用惰性物化特性时,需要监控投影兼容性问题。你可能需要设置 `query_plan_optimize_lazy_materialization = false`。 | -| 并行副本 | 支持。 | 不支持。 | -| [`optimize_read_in_order`](/operations/settings/settings#optimize_read_in_order) | 支持。 | 支持。 | -| 轻量级更新与删除 | 支持。 | 不支持。 | - - - -## 比较物化视图和投影 {#choose-between} +## 对比物化视图和投影 {#choose-between} ### 何时选择物化视图 {#choosing-materialized-views} -在以下情况下,你应考虑使用物化视图: - -- 处理 **实时 ETL 和多阶段数据管道**:你需要在数据到达时执行复杂转换、聚合或路由,并可能通过链式视图跨多个阶段处理数据。 -- 需要 **复杂反规范化**:你需要将来自多个来源(表、子查询或字典)的数据预先关联到一个单一、针对查询优化的表中,尤其是在可以接受使用可刷新的物化视图进行周期性全量刷新的情况下。 -- 希望 **显式控制模式(schema)**:你需要一个独立的目标表,用其自身的 schema 和引擎来存储预计算结果,为数据建模提供更大的灵活性。 -- 希望在 **摄取阶段进行过滤**:你需要在数据被物化之前进行过滤,从而减少写入目标表的数据量。 - -### 何时避免使用物化视图 {#avoid-materialized-views} +在以下情况下,应考虑使用物化视图: -在以下情况下,你应考虑避免使用物化视图: +- 处理 **实时 ETL 和多阶段数据管道**:需要在数据到达时执行复杂转换、聚合或路由分发,并且可能通过串联视图跨多个阶段处理。 +- 需要 **复杂的反规范化**:需要将来自多个源(表、子查询或字典)的数据预先关联(join)到单个、针对查询优化的表中,尤其是在可以接受使用可刷新的物化视图进行周期性全量刷新时。 +- 希望获得 **显式的 schema 控制**:需要一个具有自己 schema 和引擎的独立目标表来存放预计算结果,从而在数据建模上获得更大的灵活性。 +- 希望在 **摄取时进行过滤**:需要在数据被物化之前对其进行过滤,从而减少写入目标表的数据量。 -- **源数据经常被更新或删除**:如果没有额外策略来处理源表和目标表之间的一致性,增量物化视图可能会变得陈旧且不一致。 -- **更偏好简单性和自动优化**:例如你希望避免管理单独的目标表。 +### 何时应避免使用物化视图 {#avoid-materialized-views} -### 何时选择投影 {#choosing-projections} +在以下情况下,应考虑避免使用物化视图: -在以下情况下,你应考虑使用投影: +- **源数据经常被更新或删除**:如果没有额外的策略来处理源表和目标表之间的一致性,增量物化视图可能会变得过时且不一致。 +- **更倾向于保持简单并依赖自动优化**:例如你希望避免管理单独的目标表时。 -- **优化单表查询**:你的主要目标是通过提供备用排序顺序、优化对非主键列的过滤,或为单个表预计算聚合结果,以加速对单个基础表的查询。 -- 需要 **查询透明性**:你希望查询直接针对原始表而无需修改,并依赖 ClickHouse 为给定查询选择最优的数据布局。 +### 何时选择使用投影(projections){#choosing-projections} -### 何时避免使用投影 {#avoid-projections} +在以下情况下,应当考虑使用 projections: -在以下情况下,你应考虑避免使用投影: +- **为单个表优化查询**:你的主要目标是通过提供备用排序顺序、优化不属于主键的列上的过滤条件,或为单个表预先计算聚合,从而加速对单个基础表的查询。 +- 你希望实现**查询透明性(query transparency)**:你希望在不修改查询的情况下,始终针对原始表执行查询,并依赖 ClickHouse 为给定查询自动选择最优的数据布局。 -- **需要复杂数据转换或多阶段 ETL**:投影在其定义中不支持 `JOIN` 操作,无法调整为构建多步管道,并且无法处理某些 SQL 特性,如窗口函数或复杂的 `CASE` 表达式。因此,它们不适合复杂数据转换。 -- **需要对物化数据进行显式过滤**:投影在其定义中不支持 `WHERE` 子句来过滤要物化到投影本身的数据。 -- **使用非 MergeTree 表引擎**:投影仅适用于使用 `MergeTree` 系列表引擎的表。 -- **必须依赖 `FINAL` 查询**:投影无法与 `FINAL` 查询配合使用,而后者有时用于去重。 -- 你需要使用 [parallel replicas](/deployment-guides/parallel-replicas),而这在投影中不受支持。 +### 何时应避免使用 Projection {#avoid-projections} +在以下情况下,应考虑避免使用 Projection: +- **需要复杂数据转换或多阶段 ETL 时**:Projection 定义不支持 `JOIN` 操作,不能通过链式组合构建多步管道,也无法处理某些 SQL 特性,如窗口函数或复杂的 `CASE` 表达式。虽然针对带有 Projection 的表执行查询时可以自由使用 `JOIN`,但 Projection 本身并不适合承担复杂的数据转换逻辑。 +- **需要对物化数据进行显式过滤时**:Projection 在定义中不支持使用 `WHERE` 子句来过滤将被物化到该 Projection 中的数据。 +- **使用非 MergeTree 表引擎时**:Projection 仅适用于使用 `MergeTree` 系列表引擎的表。 +- **需要使用 `FINAL` 查询时**:Projection 无法与 `FINAL` 查询配合使用,而 `FINAL` 查询有时会用于去重。 +- **需要使用 [parallel replicas](/deployment-guides/parallel-replicas) 时**:Projection 不支持该特性。 ## 总结 {#summary} -物化视图和投影都是用于优化查询和转换数据的强大工具。总体来说,我们不建议将它们视为非此即彼的选择。相反,可以将它们以互补的方式一起使用,从而最大化查询性能。因此,在 ClickHouse 中选择物化视图还是投影,实际上取决于具体的使用场景和访问模式。 +物化视图和投影(projection)都是用于优化查询和转换数据的强大工具,我们并不建议把它们看作非此即彼的二选一方案。相反,你可以将二者组合、互补使用,以最大化查询性能。因此,在 ClickHouse 中选择物化视图还是投影,很大程度上取决于你的具体用例和访问模式。 -一般来说,当需要将一个或多个源表中的数据聚合到一个目标表中,或在大规模上执行复杂转换时,应考虑使用物化视图。物化视图非常适合将开销昂贵的聚合工作从查询时转移到写入时。它们非常适用于按日或按月的汇总、实时仪表盘或数据摘要。 +作为一个经验法则,当你需要将一个或多个源表中的数据聚合到目标表,或者需要在大规模数据上执行复杂转换时,应优先考虑使用物化视图。物化视图非常适合将代价高昂的聚合工作从查询时刻前移到写入时刻。它们非常适用于按日或按月的汇总(rollup)、实时看板以及数据概览或汇总等场景。 -另一方面,当需要优化那些在与表主键(主键决定了磁盘上数据的物理排序)不同的列上进行过滤的查询时,应使用投影。当无法再更改表的主键,或者访问模式比主键所能支持的更加多样化时,投影尤其有用。 +另一方面,当你需要优化的查询在过滤条件中使用的列,与表主键(决定数据在磁盘上物理排序的列)不同的时候,应考虑使用投影。尤其是在无法再更改表主键,或者你的访问模式比主键能够支持的情况更加多样化时,投影会格外有用。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md b/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md index 8e50d03e9a4..00a2620a01e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/data-modeling/schema-design.md @@ -12,7 +12,6 @@ import Image from '@theme/IdealImage'; 理解高效的 schema 设计是优化 ClickHouse 性能的关键,其中的诸多选择往往需要在不同方案之间进行权衡,最优方案取决于实际查询模式,以及数据更新频率、延迟要求和数据量等因素。本指南概述了用于优化 ClickHouse 性能的 schema 设计最佳实践和数据建模技术。 - ## Stack Overflow 数据集 {#stack-overflow-dataset} 在本指南的示例中,我们使用 Stack Overflow 数据集的一个子集。该数据集包含自 2008 年至 2024 年 4 月在 Stack Overflow 上产生的每一条帖子、投票、用户、评论和徽章。该数据以 Parquet 格式提供,使用下方所示的 schema,存储在 S3 bucket `s3://datasets-documentation/stackoverflow/parquet/` 中: @@ -27,9 +26,7 @@ Stack Overflow 数据集包含多张相互关联的表。在任何数据建模 上述 schema 在本指南中是刻意未进行最优设计的。 - - -## 建立初始 schema +## 建立初始 schema {#establish-initial-schema} 由于 `posts` 表将是大多数分析查询的目标,我们重点为该表建立 schema。此数据位于公共 S3 bucket `s3://datasets-documentation/stackoverflow/parquet/posts/*.parquet` 中,每年一个文件。 @@ -129,7 +126,6 @@ INSERT INTO posts SELECT * FROM s3('https://datasets-documentation.s3.eu-west-3. > 上述查询会加载 6000 万行。对于 ClickHouse 来说这算小规模,但网络连接较慢的用户可能希望只加载一部分数据。可以通过使用 glob 通配模式指定要加载的年份来实现,例如 `https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2008.parquet` 或 `https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/{2008, 2009}.parquet`。关于如何使用 glob 模式来筛选文件子集,请参阅[此处](/sql-reference/table-functions/file#globs-in-path)。 - ## 优化类型 {#optimizing-types} ClickHouse 查询性能的秘密之一是压缩。 @@ -154,8 +150,6 @@ ClickHouse 中的压缩主要受 3 个因素影响:排序键(ordering key) 通过将这些简单规则应用到我们的 posts 表,我们可以为每一列识别出一个最优类型: - - | 列 | 是否为数值 | 最小值,最大值 | 唯一值 | 空值 | 注释 | 优化类型 | | ----------------------- | ----- | ------------------------------------------------------------ | -------- | -- | ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `PostTypeId` | 是 | 1, 8 | 8 | 否 | | `Enum('Question' = 1, 'Answer' = 2, 'Wiki' = 3, 'TagWikiExcerpt' = 4, 'TagWiki' = 5, 'ModeratorNomination' = 6, 'WikiPlaceholder' = 7, 'PrivilegeWiki' = 8)` | @@ -227,8 +221,7 @@ INSERT INTO posts_v2 SELECT * FROM posts ClickHouse 中的主(排序)键 来自 OLTP 数据库的用户通常会在 ClickHouse 中寻找与之对应的等价概念。 - -## 选择排序键 +## 选择排序键 {#choosing-an-ordering-key} 在 ClickHouse 通常使用的规模下,内存和磁盘效率至关重要。数据以称为 part 的数据块形式写入 ClickHouse 表,并在后台根据规则对这些 part 进行合并。在 ClickHouse 中,每个 part 都有自己的主索引。当 part 被合并时,合并后 part 的主索引也会被合并。每个 part 的主索引对每一组行只包含一个索引条目——这种技术称为稀疏索引(sparse indexing)。 @@ -247,7 +240,7 @@ ClickHouse 中的主(排序)键 在确定用于排序键的列子集时,需要按特定顺序声明这些列。此顺序会显著影响查询中对排序键中后续列进行过滤的效率,以及表数据文件的压缩比。通常,最好按基数(cardinality)从低到高来排列键。这需要与以下事实进行权衡:对在排序键中位置靠后的列进行过滤,其效率会低于对位置靠前列的过滤。在这些行为之间取得平衡,并结合你的访问模式进行考虑(最重要的是要测试不同方案)。 -### 示例 +### 示例 {#example} 将上述指南应用于我们的 `posts` 表,假设用户希望执行按日期和帖子类型过滤的分析,例如: @@ -279,7 +272,6 @@ LIMIT 3 让我们选择列 `PostTypeId` 和 `CreationDate` 作为排序键。 - 在我们的场景中,我们假定用户始终会按 `PostTypeId` 进行过滤。它的基数为 8,是作为排序键首个元素的合理选择。鉴于按日期粒度进行过滤通常已经足够(同时按日期时间过滤也会受益),因此我们将 `toDate(CreationDate)` 作为键的第二个组成部分。这样也会生成更小的索引,因为日期可以用 16 位来表示,从而加快过滤速度。我们键中的最后一个条目是 `CommentCount`,用于辅助找到评论数最多的帖子(最终排序)。 ```sql @@ -335,7 +327,6 @@ LIMIT 3 对于希望通过使用特定数据类型和合理排序键来提升压缩效果的用户,请参阅 [Compression in ClickHouse](/data-compression/compression-in-clickhouse)。如果需要进一步提高压缩率,我们还推荐参考其中的 [Choosing the right column compression codec](/data-compression/compression-in-clickhouse#choosing-the-right-column-compression-codec) 部分。 - ## 下一步:数据建模技术 {#next-data-modeling-techniques} 到目前为止,我们只迁移了一张表。虽然这已经让我们能够介绍一些核心的 ClickHouse 概念,但大多数 schema 往往没有这么简单。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/index.md index 33d3134dfee..1408971097d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/index.md @@ -6,7 +6,7 @@ keywords: ['部署指南', '扩展', '集群部署', '复制', '容错'] doc_type: 'landing-page' --- -# 部署与扩展 +# 部署与扩展 {#deployment-and-scaling} 本节涵盖以下主题: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx index d42ce49ccf1..324d42dbbe1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/parallel-replicas.mdx @@ -20,7 +20,6 @@ import image_9 from '@site/static/images/deployment-guides/parallel-replicas-9.p - ## 简介 \{#introduction\} ClickHouse 处理查询的速度极快,但这些查询是如何在多台服务器之间被分发并并行执行的呢? @@ -38,22 +37,25 @@ ClickHouse 处理查询的速度极快,但这些查询是如何在多台服务 上图展示了客户端查询分布式表时发生的过程:
    -
  1. - SELECT 查询被任意发送到某个节点上的分布式表 - (通过轮询策略,或在经过负载均衡器路由到特定服务器之后)。 - 该节点随后将作为协调器。 -
  2. -
  3. - 节点会根据分布式表中指定的信息,定位需要执行查询的每个分片, - 并将查询发送给各个分片。 -
  4. -
  5. - 每个分片在本地读取、过滤并聚合数据,然后将可合并的中间状态 - 返回给协调器。 -
  6. -
  7. - 协调器节点对数据进行合并,然后将响应返回给客户端。 -
  8. +
  9. + SELECT 查询被任意发送到某个节点上的分布式表 + (通过轮询策略,或在经过负载均衡器路由到特定服务器之后)。 + 该节点随后将作为协调器。 +
  10. + +
  11. + 节点会根据分布式表中指定的信息,定位需要执行查询的每个分片, + 并将查询发送给各个分片。 +
  12. + +
  13. + 每个分片在本地读取、过滤并聚合数据,然后将可合并的中间状态 + 返回给协调器。 +
  14. + +
  15. + 协调器节点对数据进行合并,然后将响应返回给客户端。 +
当我们引入副本时,流程基本相同,唯一的区别是每个分片中只有一个副本会执行查询, @@ -62,7 +64,7 @@ ClickHouse 处理查询的速度极快,但这些查询是如何在多台服务 ## 非分片架构 \{#non-sharded-architecture\} ClickHouse Cloud 的架构与上文所介绍的架构有很大差异。 -(参见 ["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) +(参见 ["ClickHouse Cloud Architecture"](https://clickhouse.com/docs/cloud/reference/architecture) 了解更多详情。)由于计算与存储的分离,以及几乎无限的存储容量,对分片的需求就不那么重要了。 下图展示了 ClickHouse Cloud 的架构: @@ -86,39 +88,46 @@ Keeper 集群确保我们拥有元数据的唯一可信来源。各个副本可 使用并行副本时:
    -
  1. - 来自客户端的查询在经过负载均衡器后被发送到某个节点。该节点成为此查询的协调节点。 -
  2. -
  3. - 该节点分析每个 part 的索引,并选择需要处理的合适 part 和粒度。 -
  4. -
  5. - 协调节点将工作负载拆分成一组可以分配给不同副本的粒度。 -
  6. -
  7. - 每组粒度由相应的副本进行处理,完成后将可合并的中间状态发送给协调节点。 -
  8. -
  9. - 最后,协调节点合并来自各副本的所有结果,然后将响应返回给客户端。 -
  10. +
  11. + 来自客户端的查询在经过负载均衡器后被发送到某个节点。该节点成为此查询的协调节点。 +
  12. + +
  13. + 该节点分析每个 part 的索引,并选择需要处理的合适 part 和粒度。 +
  14. + +
  15. + 协调节点将工作负载拆分成一组可以分配给不同副本的粒度。 +
  16. + +
  17. + 每组粒度由相应的副本进行处理,完成后将可合并的中间状态发送给协调节点。 +
  18. + +
  19. + 最后,协调节点合并来自各副本的所有结果,然后将响应返回给客户端。 +
上述步骤描述了并行副本在理论上的工作方式。 然而在实践中,存在许多因素会阻碍这套逻辑完美运行:
    -
  1. - 某些副本可能不可用。 -
  2. -
  3. - ClickHouse 中的复制是异步的,在某些时间点上,一些副本可能不具有相同的 part。 -
  4. -
  5. - 副本之间的尾延迟需要以某种方式进行处理。 -
  6. -
  7. - 文件系统缓存会根据各副本上的活动在副本之间有所差异,这意味着随机分配任务在缓存局部性方面可能导致性能不够理想。 -
  8. +
  9. + 某些副本可能不可用。 +
  10. + +
  11. + ClickHouse 中的复制是异步的,在某些时间点上,一些副本可能不具有相同的 part。 +
  12. + +
  13. + 副本之间的尾延迟需要以某种方式进行处理。 +
  14. + +
  15. + 文件系统缓存会根据各副本上的活动在副本之间有所差异,这意味着随机分配任务在缓存局部性方面可能导致性能不够理想。 +
在接下来的小节中,我们将讨论如何克服这些因素。 @@ -130,18 +139,21 @@ Keeper 集群确保我们拥有元数据的唯一可信来源。各个副本可
    -
  1. - 来自客户端的查询在经过负载均衡器后被发送到某个节点,该节点成为此查询的协调节点。 -
  2. -
  3. - 协调节点发送请求,从集群中所有副本获取公告。副本对于某个表当前分区(parts)集合的视图可能略有不同。因此,我们需要收集这些信息以避免做出错误的调度决策。 -
  4. -
  5. - 随后,协调节点使用这些公告来确定一组可分配给不同副本的粒度单元。例如,在这里我们可以看到,没有来自 part 3 的粒度被分配给副本 2,因为该副本在其公告中未声明该分区。还要注意,没有任务被分配给副本 3,因为该副本没有提供公告。 -
  6. -
  7. - 在每个副本都处理完其粒度子集上的查询,并将可合并状态发送回协调节点之后,协调节点会合并结果,并将响应返回给客户端。 -
  8. +
  9. + 来自客户端的查询在经过负载均衡器后被发送到某个节点,该节点成为此查询的协调节点。 +
  10. + +
  11. + 协调节点发送请求,从集群中所有副本获取公告。副本对于某个表当前分区(parts)集合的视图可能略有不同。因此,我们需要收集这些信息以避免做出错误的调度决策。 +
  12. + +
  13. + 随后,协调节点使用这些公告来确定一组可分配给不同副本的粒度单元。例如,在这里我们可以看到,没有来自 part 3 的粒度被分配给副本 2,因为该副本在其公告中未声明该分区。还要注意,没有任务被分配给副本 3,因为该副本没有提供公告。 +
  14. + +
  15. + 在每个副本都处理完其粒度子集上的查询,并将可合并状态发送回协调节点之后,协调节点会合并结果,并将响应返回给客户端。 +
### 动态协调 \{#dynamic-coordination\} @@ -155,37 +167,41 @@ Keeper 集群确保我们拥有元数据的唯一可信来源。各个副本可
    -
  1. - 各副本向协调节点表明它们可以处理任务,同时还可以指定各自能够处理的工作量。 -
  2. -
  3. - 协调节点将任务分配给这些副本。 -
  4. +
  5. + 各副本向协调节点表明它们可以处理任务,同时还可以指定各自能够处理的工作量。 +
  6. + +
  7. + 协调节点将任务分配给这些副本。 +
    -
  1. - 副本 1 和 2 能够非常快速地完成各自的任务,它们会向协调节点请求新的任务。 -
  2. -
  3. - 协调节点为副本 1 和 2 分配新的任务。 -
  4. +
  5. + 副本 1 和 2 能够非常快速地完成各自的任务,它们会向协调节点请求新的任务。 +
  6. + +
  7. + 协调节点为副本 1 和 2 分配新的任务。 +
    -
  1. - 所有副本现在都已完成各自任务的处理,它们会请求更多任务。 -
  2. -
  3. - 协调节点利用公告检查还有哪些任务尚未被处理,但已经没有剩余任务了。 -
  4. -
  5. - 协调节点会告知副本所有内容都已处理完成。随后它会合并所有可合并的状态,并返回查询结果。 -
  6. +
  7. + 所有副本现在都已完成各自任务的处理,它们会请求更多任务。 +
  8. + +
  9. + 协调节点利用公告检查还有哪些任务尚未被处理,但已经没有剩余任务了。 +
  10. + +
  11. + 协调节点会告知副本所有内容都已处理完成。随后它会合并所有可合并的状态,并返回查询结果。 +
### 管理缓存局部性 \{#managing-cache-locality\} @@ -193,34 +209,38 @@ Keeper 集群确保我们拥有元数据的唯一可信来源。各个副本可 最后一个潜在的问题是如何处理缓存局部性。如果多次执行同一个查询,如何确保相同的任务被路由到同一副本?在前面的示例中,我们有如下任务分配:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
副本 1副本 2副本 3
Part 1g1, g6, g7g2, g4, g5g3
Part 2g1g2, g4, g5g3
Part 3g1, g6g2, g4, g5g3
+ + 副本 1副本 2副本 3
Part 1g1, g6, g7g2, g4, g5g3
Part 2g1g2, g4, g5g3
Part 3g1, g6g2, g4, g5g3
为了确保相同的任务被分配到同一副本并能从缓存中获益,会进行两步操作:首先计算分片 + 一组 granule(即一个任务)的哈希值;然后对副本数量取模以进行任务分配。 @@ -252,13 +272,13 @@ Keeper 集群确保我们拥有元数据的唯一可信来源。各个副本可 | Setting | Description | |----------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `enable_parallel_replicas` | `0`: 禁用
`1`: 启用
`2`: 强制启用并行副本,如果无法使用并行副本则抛出异常。 | +| `enable_parallel_replicas` | `0`: 禁用
`1`: 启用
`2`: 强制启用并行副本,如果无法使用并行副本则抛出异常。 | | `cluster_for_parallel_replicas` | 用于并行副本的集群名称;如果你使用的是 ClickHouse Cloud,请使用 `default`。 | | `max_parallel_replicas` | 在多个副本上执行查询时可使用的最大副本数;如果指定的值小于集群中的副本数,则会随机选择节点。此值也可以设置得高于实际副本数,以预留水平扩展空间。 | -| `parallel_replicas_min_number_of_rows_per_replica` | 用于根据需要处理的行数来限制所使用的副本数量,实际使用的副本数定义为:
`estimated rows to read` / `min_number_of_rows_per_replica`。 | -| `allow_experimental_analyzer` | `0`: 使用旧分析器
`1`: 使用新分析器。

并行副本的行为可能会因所使用的分析器不同而发生变化。 | +| `parallel_replicas_min_number_of_rows_per_replica` | 用于根据需要处理的行数来限制所使用的副本数量,实际使用的副本数定义为:
`estimated rows to read` / `min_number_of_rows_per_replica`。 | +| `allow_experimental_analyzer` | `0`: 使用旧分析器
`1`: 使用新分析器。

并行副本的行为可能会因所使用的分析器不同而发生变化。 | -## 排查并行副本相关问题 +## 排查并行副本相关问题 \{#investigating-issues-with-parallel-replicas\} 你可以在 [`system.query_log`](/docs/operations/system-tables/query_log) 表中检查每个查询实际使用的设置。你也可以查看 [`system.events`](/docs/operations/system-tables/events) 表,了解服务器上发生的所有事件,并使用 [`clusterAllReplicas`](/docs/sql-reference/table-functions/cluster) 表函数查看所有副本上的表(如果你是云用户,请将集群名设为 `default`)。 @@ -270,7 +290,6 @@ FROM clusterAllReplicas('default', system.events) WHERE event ILIKE '%ParallelReplicas%' ``` -
响应 @@ -331,7 +350,6 @@ WHERE query_id = 'ad40c712-d25d-45c4-b1a1-a28ba8d4019c' ORDER BY event_time_microseconds ASC ``` -
响应 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md index 1047dd6df64..30fd13c46bf 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/01_1_shard_2_replicas.md @@ -9,19 +9,19 @@ keywords: ['复制', '高可用性', '集群搭建', '数据冗余', '容错'] --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ReplicationArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/replication.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > 在本示例中,你将学习如何搭建一个用于数据复制的简单 ClickHouse 集群。 > 一共配置了五台服务器,其中两台用于存储数据副本, @@ -33,7 +33,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前置条件 {#pre-requisites} - 您之前已设置过[本地 ClickHouse 服务器](/install) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md index 5a136d87e21..343d6a0aab8 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/02_2_shards_1_replica.md @@ -9,19 +9,19 @@ keywords: ['分片', '水平扩展', '分布式数据', '集群部署', '数据 --- import Image from '@theme/IdealImage'; -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; import ShardingArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/sharding.png'; -import ConfigFileNote from '@site/docs/_snippets/_config-files.md'; -import KeeperConfigFileNote from '@site/docs/_snippets/_keeper-config-files.md'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import ServerParameterTable from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigFileNote from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_config-files.md'; +import KeeperConfigFileNote from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_keeper-config-files.md'; +import ConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import ServerParameterTable from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_server_parameter_table.mdx'; +import KeeperConfig from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > 在本示例中,你将学习如何搭建一个简单且可扩展的 ClickHouse 集群。 > 集群中共配置了五台服务器,其中两台用于对数据进行分片。 @@ -33,7 +33,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前置条件 {#pre-requisites} - 你已经在本地部署过 [ClickHouse 服务器](/install) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md index 0d09c992a3d..fbe30a70f02 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/03_2_shards_2_replicas.md @@ -10,14 +10,14 @@ keywords: ['集群部署', '复制', '分片', '高可用性', '可扩展性'] import Image from '@theme/IdealImage'; import SharedReplicatedArchitecture from '@site/static/images/deployment-guides/replication-sharding-examples/both.png'; -import ConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; -import ListenHost from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; -import KeeperConfig from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; -import KeeperConfigExplanation from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; -import VerifyKeeperStatus from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; -import DedicatedKeeperServers from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; -import ExampleFiles from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; -import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; +import ConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_config_explanation.mdx'; +import ListenHost from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_listen_host.mdx'; +import KeeperConfig from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_config.mdx'; +import KeeperConfigExplanation from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_keeper_explanation.mdx'; +import VerifyKeeperStatus from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_verify_keeper_using_mntr.mdx'; +import DedicatedKeeperServers from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_dedicated_keeper_servers.mdx'; +import ExampleFiles from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_working_example.mdx'; +import CloudTip from '@site/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/replication-sharding-examples/_snippets/_cloud_tip.mdx'; > 在本示例中,你将学习如何搭建一个既支持复制又支持横向扩展的简单 ClickHouse 集群。 > 该集群由两个分片和两个副本组成,并包含一个由 3 个节点构成的 ClickHouse Keeper 集群, @@ -29,7 +29,6 @@ import CloudTip from '@site/docs/deployment-guides/replication-sharding-examples - ## 前提条件 {#prerequisites} - 已经搭建过[本地 ClickHouse 服务器](/install) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md index 4a5778bab57..8fb89b30fd7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/deployment-guides/terminology.md @@ -8,7 +8,7 @@ doc_type: 'guide' keywords: ['部署', '架构', '复制', '分片', '集群配置'] --- -import ReplicationShardingTerminology from '@site/docs/_snippets/_replication-sharding-terminology.md'; +import ReplicationShardingTerminology from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_replication-sharding-terminology.md'; 本节中的部署示例基于 ClickHouse 支持与服务团队向 ClickHouse 用户提供的建议。这些都是可直接使用的示例,我们建议您先尝试运行,然后根据自身需求进行调整。您也许会在这里找到一个完全符合您需求的示例。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/architecture.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/architecture.md index 355f0a963f0..0edace5177f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/architecture.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/architecture.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# 架构概览 +# 架构概览 {#architecture-overview} ClickHouse 是一个真正的列式 DBMS。数据按列存储,并在执行过程中以数组(向量或列块)的形式处理。 在可能的情况下,操作会尽量在数组上执行,而不是针对单个值。 @@ -242,7 +242,7 @@ IO 线程池是一个简单的 `ThreadPool`,可通过 `IOThreadPool::get()` -## 并发控制 +## 并发控制 {#concurrency-control} 可以并行执行的查询通过 `max_threads` 设置来限制自身的并发度。该设置的默认值经过选择,使单个查询能够以最佳方式利用所有 CPU 核心。但如果存在多个并发查询且每个查询都使用默认的 `max_threads` 设置值,会怎样?此时这些查询将共享 CPU 资源。操作系统会通过不断切换线程来保证公平性,而这会引入一定的性能损耗。`ConcurrencyControl` 用来应对这种损耗,并避免分配过多线程。配置项 `concurrent_threads_soft_limit_num` 用于限制在开始施加某种形式的 CPU 压力之前,最多可以分配多少并发线程。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-arm.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-arm.md index c90c3486f0a..2c93eae4430 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-arm.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-arm.md @@ -7,7 +7,7 @@ title: '如何在 Linux 上面向 AARCH64 构建 ClickHouse' doc_type: 'guide' --- -# 如何在 Linux 上为 AArch64 构建 ClickHouse +# 如何在 Linux 上为 AArch64 构建 ClickHouse {#how-to-build-clickhouse-on-linux-for-aarch64} 在 AArch64 机器上构建面向 AArch64 的 ClickHouse 时,无需执行任何特殊步骤。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md index 74d13a4fe35..bc2a08095b5 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-loongarch.md @@ -7,15 +7,11 @@ title: '在 Linux 上为 LoongArch64 构建' doc_type: 'guide' --- - - -# 在 Linux 上为 LoongArch64 构建 +# 在 Linux 上为 LoongArch64 构建 {#build-on-linux-for-loongarch64} ClickHouse 对 LoongArch64 提供了实验性支持 - - -## 构建 ClickHouse +## 构建 ClickHouse {#build-clickhouse} 用于构建的 LLVM 版本必须不低于 19.1.0。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-osx.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-osx.md index a8b12551c08..a5163aef5b3 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-osx.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-osx.md @@ -7,9 +7,7 @@ title: '在 Linux 上为 macOS 构建' doc_type: 'guide' --- - - -# 如何在 Linux 上为 macOS 构建 ClickHouse +# 如何在 Linux 上为 macOS 构建 ClickHouse {#how-to-build-clickhouse-on-linux-for-macos} 本指南适用于:当你有一台 Linux 机器,并希望使用它来构建可在 OS X 上运行的 `clickhouse` 二进制可执行文件的情况。 主要用例是运行在 Linux 机器上的持续集成(CI)检查。 @@ -21,9 +19,7 @@ doc_type: 'guide' 如果你的目标是 ARM 架构,只需将所有出现的 `x86_64` 替换为 `aarch64` 即可。 例如,在整个步骤中将 `x86_64-apple-darwin` 替换为 `aarch64-apple-darwin`。 - - -## 安装交叉编译工具链 +## 安装交叉编译工具链 {#install-cross-compilation-toolset} 记住安装 `cctools` 的路径,并将其记为 `${CCTOOLS}` @@ -53,8 +49,7 @@ cd ClickHouse/cmake/toolchain/darwin-x86_64 curl -L 'https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX11.0.sdk.tar.xz' | tar xJ --strip-components=1 ``` - -## 构建 ClickHouse +## 构建 ClickHouse {#build-clickhouse} ```bash cd ClickHouse diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md index 289c027160e..c155f71b605 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-riscv.md @@ -7,15 +7,11 @@ title: '如何在 Linux 上为 RISC-V 64 构建 ClickHouse' doc_type: 'guide' --- - - -# 如何在 RISC-V 64 架构的 Linux 上构建 ClickHouse +# 如何在 RISC-V 64 架构的 Linux 上构建 ClickHouse {#how-to-build-clickhouse-on-linux-for-risc-v-64} ClickHouse 对 RISC-V 架构提供实验性支持。目前尚无法启用全部功能。 - - -## 构建 ClickHouse +## 构建 ClickHouse {#build-clickhouse} 在非 RISC-V 机器上为 RISC-V 目标进行交叉编译: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md index 45f0c86b61a..93f7abf80ff 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-cross-s390x.md @@ -1,45 +1,30 @@ --- description: '在 s390x 架构上从源代码构建 ClickHouse 的指南' -sidebar_label: '在 Linux 上为 s390x(zLinux)构建' +sidebar_label: '在 Linux 上针对 s390x(zLinux)构建' sidebar_position: 30 slug: /development/build-cross-s390x -title: '在 Linux 上为 s390x(zLinux)构建' +title: '在 Linux 上针对 s390x(zLinux)构建' doc_type: 'guide' --- +# 在 Linux 上为 s390x(zLinux)进行构建 {#build-on-linux-for-s390x-zlinux} +ClickHouse 对 s390x 提供实验性支持。 -# 在 Linux 上构建 s390x(zLinux)版本 +## 为 s390x 构建 ClickHouse {#building-clickhouse-for-s390x} -ClickHouse 目前对 s390x 提供实验性支持。 +与其他平台一样,s390x 会将 OpenSSL 构建为静态库。如果你希望使用动态链接的 OpenSSL 进行构建,则需要向 CMake 传递 `-DENABLE_OPENSSL_DYNAMIC=1`。 +这些说明假定宿主机为 Linux x86_64/ARM,并且已经按照[构建说明](../development/build.md)安装了本机构建所需的全部工具。同时假定宿主机运行的是 Ubuntu 22.04,不过以下说明同样适用于 Ubuntu 20.04。 - -## 为 s390x 构建 ClickHouse - -s390x 有两个与 OpenSSL 相关的构建选项: - -* 默认情况下,在 s390x 上 OpenSSL 会被构建为共享库。这与其他所有平台不同,其他平台上 OpenSSL 会被构建为静态库。 -* 如果仍然希望将 OpenSSL 构建为静态库,请在 CMake 中传入 `-DENABLE_OPENSSL_DYNAMIC=0`。 - -以下说明假定宿主机为 x86_64 架构,并且已经按照[构建说明](../development/build.md)安装了本机构建所需的全部工具。同时还假定宿主机运行的是 Ubuntu 22.04,但下面的说明在 Ubuntu 20.04 上同样应当可行。 - -除了安装用于本机构建的工具外,还需要额外安装以下软件包: - -```bash -apt-get install binutils-s390x-linux-gnu libc6-dev-s390x-cross gcc-s390x-linux-gnu binfmt-support qemu-user-static -``` - -如果您希望交叉编译 Rust 代码,请安装针对 s390x 的 Rust 交叉编译目标: +除了安装用于本机构建的工具外,还需要安装以下额外软件包: ```bash +apt-get mold rustup target add s390x-unknown-linux-gnu ``` -s390x 构建使用 mold 链接器,请从 [https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz](https://github.com/rui314/mold/releases/download/v2.0.0/mold-2.0.0-x86_64-linux.tar.gz) -下载并将其放入你的 `$PATH` 中。 - -要构建 s390x: +构建 s390x 版本: ```bash cmake -DCMAKE_TOOLCHAIN_FILE=cmake/linux/toolchain-s390x.cmake .. @@ -47,30 +32,37 @@ ninja ``` -## 运行 +## 运行 {#running} + +要进行仿真,你需要适用于 s390x 的 QEMU user static 静态二进制文件。在 Ubuntu 上可以通过以下命令安装: + +```bash +apt-get install binfmt-support binutils-s390x-linux-gnu qemu-user-static +``` -构建完成后,例如可以这样运行该二进制文件: +构建完成后,例如可以通过以下命令运行该二进制文件: ```bash -qemu-s390x-static -L /usr/s390x-linux-gnu ./clickhouse +qemu-s390x-static -L /usr/s390x-linux-gnu ./programs/clickhouse local --query "Select 2" +2 ``` -## 调试 +## 调试 {#debugging} 安装 LLDB: ```bash -apt-get install lldb-15 +apt-get install lldb-21 ``` -要调试 s390x 可执行文件,请在调试模式下通过 QEMU 运行 ClickHouse: +要调试 s390x 可执行文件,请使用 QEMU 以调试模式运行 ClickHouse: ```bash qemu-s390x-static -g 31338 -L /usr/s390x-linux-gnu ./clickhouse ``` -在另一个 shell 中运行 LLDB 并附加到进程,将 `` 和 `` 替换为与你环境相对应的值。 +在另一个 shell 中运行 LLDB 并进行附加操作,将 `` 和 `` 替换为与您环境相对应的值。 ```bash lldb-15 @@ -102,16 +94,16 @@ Process 1 stopped ``` -## Visual Studio Code 集成 +## Visual Studio Code 集成 {#visual-studio-code-integration} -* 需要安装 [CodeLLDB](https://github.com/vadimcn/vscode-lldb) 扩展以进行可视化调试。 -* 如果使用 [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md),可以通过 [Command Variable](https://github.com/rioj7/command-variable) 扩展实现动态启动配置。 -* 请确保将后端指向你的 LLVM 安装,例如 `"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so"`。 -* 请确保在启动前以调试模式运行 ClickHouse 可执行文件。(也可以创建一个 `preLaunchTask` 来自动化完成此操作) +- 进行可视化调试需要安装 [CodeLLDB](https://github.com/vadimcn/vscode-lldb) 扩展。 +- 如果使用 [CMake Variants](https://github.com/microsoft/vscode-cmake-tools/blob/main/docs/variants.md),可以安装 [Command Variable](https://github.com/rioj7/command-variable) 扩展来辅助配置动态启动。 +- 请确保将后端设置为你的 LLVM 安装路径,例如:`"lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so"` +- 在启动之前,请确保以调试模式运行 ClickHouse 可执行文件。(也可以创建一个 `preLaunchTask` 来自动完成此操作) -### 示例配置 +### 示例配置 {#example-configurations} -#### cmake-variants.yaml +#### cmake-variants.yaml {#cmake-variantsyaml} ```yaml buildType: @@ -119,7 +111,7 @@ buildType: choices: debug: short: Debug - long: 生成调试信息 + long: 输出调试信息 buildType: Debug release: short: Release @@ -127,7 +119,7 @@ buildType: buildType: Release relwithdebinfo: short: RelWithDebInfo - long: 带调试信息的发布版本 + long: 发布版本(含调试信息) buildType: RelWithDebInfo tsan: short: MinSizeRel @@ -148,7 +140,8 @@ toolchain: CMAKE_TOOLCHAIN_FILE: cmake/linux/toolchain-s390x.cmake ``` -#### launch.json + +#### launch.json {#launchjson} ```json { @@ -166,18 +159,20 @@ toolchain: } ``` -#### settings.json -这也会将不同的构建产物放在 `build` 目录下的不同子目录中。 +#### settings.json {#settingsjson} + +这也会将不同的构建产物放在 `build` 文件夹下的不同子文件夹中。 ```json { "cmake.buildDirectory": "${workspaceFolder}/build/${buildKitVendor}-${buildKitVersion}-${variant:toolchain}-${variant:buildType}", - "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-15.so" + "lldb.library": "/usr/lib/x86_64-linux-gnu/liblldb-21.so" } ``` -#### run-debug.sh + +#### run-debug.sh {#run-debugsh} ```sh #! /bin/sh @@ -186,9 +181,10 @@ cd $1 qemu-s390x-static -g 2159 -L /usr/s390x-linux-gnu $2 $3 $4 ``` -#### tasks.json -定义了一个任务,用于在与二进制文件同级的 `tmp` 文件夹中,以 `server` 模式运行已编译的可执行文件,并使用 `programs/server/config.xml` 下的配置。 +#### tasks.json {#tasksjson} + +定义了一个任务,用于在与二进制文件同级的 `tmp` 目录下,以 `server` 模式运行已编译的可执行文件,并从 `programs/server/config.xml` 加载配置。 ```json { diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-e2k.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-e2k.md index 22fad59b925..8b16f5186b1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-e2k.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-e2k.md @@ -7,15 +7,11 @@ title: '在 Linux 上为 E2K 构建' doc_type: 'guide' --- - - -# 在 Linux 上为 E2K 构建 +# 在 Linux 上为 E2K 构建 {#build-on-linux-for-e2k} ClickHouse 对 E2K(Elbrus-2000)的支持仍处于高度实验阶段,目前只能在原生模式下使用最小化配置进行编译,并依赖 e2k 定制构建的库,例如 boost、croaring、libunwind、zstd。 - - -## 构建 ClickHouse +## 构建 ClickHouse {#build-clickhouse} 构建所需的 LLVM 版本必须大于或等于 20.1.8。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-osx.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-osx.md index 882fa249717..a56b4d18c6e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build-osx.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build-osx.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# 如何在 macOS 上为 macOS 构建 ClickHouse +# 如何在 macOS 上为 macOS 构建 ClickHouse {#how-to-build-clickhouse-on-macos-for-macos} :::info 你不需要自己构建 ClickHouse! 你可以按照 [Quick Start](https://clickhouse.com/#quick-start) 中的说明安装预编译的 ClickHouse。 @@ -22,7 +22,7 @@ ClickHouse 可以在 macOS 10.15(Catalina)或更高版本上编译,支持 -## 安装前提条件 +## 安装前提条件 {#install-prerequisites} 首先,请参阅通用的[前提条件文档](developer-instruction.md)。 @@ -53,7 +53,7 @@ mkdir build export PATH=$(brew --prefix llvm)/bin:$PATH cmake -S . -B build cmake --build build -# 生成的二进制文件将位于:build/programs/clickhouse +# 生成的二进制文件将位于:build/programs/clickhouse {#the-resulting-binary-will-be-created-at-buildprogramsclickhouse} ``` :::note @@ -61,7 +61,7 @@ cmake --build build ::: -## 注意事项 +## 注意事项 {#caveats} 如果您打算运行 `clickhouse-server`,请确保增大系统的 `maxfiles` 参数值。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/build.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/build.md index a0fa3fa7f48..abc32f024d3 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/build.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/build.md @@ -7,12 +7,10 @@ title: '如何在 Linux 上构建 ClickHouse' doc_type: 'guide' --- - - -# 如何在 Linux 上构建 ClickHouse +# 如何在 Linux 上构建 ClickHouse {#how-to-build-clickhouse-on-linux} :::info 无需自行构建 ClickHouse! -可以按照 [快速开始](https://clickhouse.com/#quick-start) 中的说明安装预编译的 ClickHouse。 +你可以按照[快速开始](https://clickhouse.com/#quick-start)中的说明安装预编译的 ClickHouse。 ::: ClickHouse 可以在以下平台上构建: @@ -23,24 +21,20 @@ ClickHouse 可以在以下平台上构建: - s390/x(实验性) - RISC-V 64(实验性) - - ## 前提条件 {#assumptions} -本教程以 Ubuntu Linux 为基础进行讲解,但通过适当调整,也应适用于其他任何 Linux 发行版。 -用于开发的最低推荐 Ubuntu 版本为 24.04 LTS。 - -本教程假定你已经在本地检出(或克隆)了 ClickHouse 仓库及其所有子模块。 - +本教程基于 Ubuntu Linux 编写,但通过适当调整后,也应适用于其他任意 Linux 发行版。 +用于开发的 Ubuntu 最低推荐版本为 24.04 LTS。 +本教程假设你已在本地检出 ClickHouse 仓库及其所有子模块。 -## 安装前提条件 +## 安装前置条件 {#install-prerequisites} -首先,请参阅通用的[前提条件文档](developer-instruction.md)。 +首先,请参阅通用的[前置条件文档](developer-instruction.md)。 ClickHouse 使用 CMake 和 Ninja 进行构建。 -你还可以选择安装 ccache,以便在构建时复用已编译的目标文件。 +你还可以选择安装 ccache,以便在构建时复用已编译好的目标文件。 ```bash sudo apt-get update @@ -48,34 +42,32 @@ sudo apt-get install build-essential git cmake ccache python3 ninja-build nasm y ``` -## 安装 Clang 编译器 +## 安装 Clang 编译器 {#install-the-clang-compiler} -要在 Ubuntu/Debian 上安装 Clang,请使用 [LLVM 提供的自动安装脚本](https://apt.llvm.org/)。 +要在 Ubuntu/Debian 上安装 Clang,请使用 LLVM 的自动安装脚本,详见[此页面](https://apt.llvm.org/)。 ```bash sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)" ``` -对于其他 Linux 发行版,请检查是否可以安装 LLVM 的任一[预构建包](https://releases.llvm.org/download.html)。 +对于其他 Linux 发行版,请查看是否提供可安装的 LLVM [预编译二进制包](https://releases.llvm.org/download.html)。 -截至 2025 年 3 月,需要 Clang 19 或更高版本。 +截至 2025 年 3 月,需要使用 Clang 19 或更高版本。 不支持 GCC 或其他编译器。 ## 安装 Rust 编译器(可选) {#install-the-rust-compiler-optional} :::note -Rust 是 ClickHouse 的一个可选依赖。 -如果未安装 Rust,ClickHouse 的某些功能将不会被编译。 +Rust 是 ClickHouse 的可选依赖。 +如果未安装 Rust,ClickHouse 的某些功能将不会被编译进二进制文件。 ::: -首先,按照官方 [Rust 文档](https://www.rust-lang.org/tools/install) 中的步骤安装 `rustup`。 - -与 C++ 依赖类似,ClickHouse 使用 vendoring 来精确控制安装内容,并避免依赖第三方服务(例如 `crates.io` 注册表)。 - -虽然在发布模式下,任何较新的 rustup 工具链版本通常都可以与这些依赖配合使用,但如果你计划启用 sanitizers,则必须使用与 CI 中所用工具链在 `std` 版本上完全一致的 rustup 工具链版本(我们为该版本预先 vendoring 了相关 crates): +首先,按照官方 [Rust 文档](https://www.rust-lang.org/tools/install)中的步骤安装 `rustup`。 +与 C++ 依赖类似,ClickHouse 使用 vendoring 来精确控制安装内容,并避免依赖第三方服务(例如 `crates.io` registry)。 +虽然在 release 模式下,任意较新的 Rust toolchain 版本通常都可以与这些依赖一起工作,但如果你计划启用 sanitizers,则必须使用与 CI 中所用版本拥有完全相同 `std` 的 toolchain(我们为此对相关 crates 做了 vendoring): ```bash rustup toolchain install nightly-2025-07-07 @@ -83,34 +75,35 @@ rustup default nightly-2025-07-07 rustup component add rust-src ``` -## 构建 ClickHouse -我们建议在 `ClickHouse` 内创建一个单独的 `build` 目录,用于存放所有构建产物: +## 构建 ClickHouse {#build-clickhouse} + +我们建议在 ClickHouse 项目中创建一个单独的 `build` 目录,用于存放所有构建产物: ```sh mkdir build cd build ``` -你可以为不同的构建类型使用多个不同的目录(例如 `build_release`、`build_debug` 等)。 +你可以为不同的构建类型使用多个目录(例如 `build_release`、`build_debug` 等)。 -可选:如果你安装了多个编译器版本,可以选择性地指定要使用的具体编译器。 +可选:如果你安装了多个编译器版本,可以指定要使用的具体编译器。 ```sh export CC=clang-19 export CXX=clang++-19 ``` -在开发阶段,建议使用调试构建。 -与发布构建相比,它们使用较低级别的编译器优化(`-O`),从而带来更好的调试体验。 -此外,类型为 `LOGICAL_ERROR` 的内部异常会立即导致崩溃,而不会被优雅地处理。 +出于开发环境的需求,建议使用调试构建(debug builds)。 +与发布构建(release builds)相比,它们使用更低级别的编译器优化级别(`-O`),从而提供更好的调试体验。 +此外,类型为 `LOGICAL_ERROR` 的内部异常会立即导致崩溃,而不会被优雅地捕获和处理。 ```sh cmake -D CMAKE_BUILD_TYPE=Debug .. ``` :::note -如果你希望使用 gdb 等调试器,请在上述命令中添加 `-D DEBUG_O_LEVEL="0"`,以禁用所有编译器优化,因为这些优化可能会影响 gdb 查看或访问变量的能力。 +如果你希望使用例如 gdb 这样的调试器,请在上述命令中添加 `-D DEBUG_O_LEVEL="0"`,以禁用所有编译器优化,因为这些优化可能会影响 gdb 查看或访问变量的能力。 ::: 运行 ninja 进行构建: @@ -125,14 +118,14 @@ ninja clickhouse ninja ``` -可以通过参数 `-j` 控制并行构建作业的数量: +你可以使用参数 `-j` 来控制并行构建作业的数量: ```sh ninja -j 1 clickhouse-server clickhouse-client ``` :::tip -CMake 为执行上述命令提供了快捷方式: +CMake 为这些命令提供了快捷方式: ```sh cmake -S . -B build # 配置构建,从代码仓库顶层目录运行 @@ -142,44 +135,45 @@ cmake --build build # 编译 ::: -## 运行 ClickHouse 可执行文件 +## 运行 ClickHouse 可执行文件 {#running-the-clickhouse-executable} -在构建成功完成后,可以在 `ClickHouse//programs/` 中找到可执行文件: +构建成功后,你可以在 `ClickHouse//programs/` 中找到可执行文件: -ClickHouse 服务器会尝试在当前目录中查找配置文件 `config.xml`。 -也可以在命令行中通过 `-C` 指定配置文件。 +ClickHouse 服务器会尝试在当前目录查找配置文件 `config.xml`。 +你也可以在命令行中通过 `-C` 选项指定配置文件。 -要使用 `clickhouse-client` 连接到 ClickHouse 服务器,打开另一个终端,进入 `ClickHouse/build/programs/`,然后运行 `./clickhouse client`。 +要使用 `clickhouse-client` 连接到 ClickHouse 服务器,打开另一个终端,进入 `ClickHouse/build/programs/` 并运行 `./clickhouse client`。 -如果在 macOS 或 FreeBSD 上收到 `Connection refused` 提示,请尝试将主机地址指定为 127.0.0.1: +如果在 macOS 或 FreeBSD 上收到 `Connection refused` 消息,请尝试指定主机地址 127.0.0.1: ```bash clickhouse client --host 127.0.0.1 ``` -## 高级选项 +## 高级选项 {#advanced-options} -### 最小构建 +### 最小构建 {#minimal-build} -如果不需要使用第三方库提供的功能,可以进一步加快构建速度: +如果不需要第三方库提供的功能,可以进一步提升构建速度: ```sh cmake -DENABLE_LIBRARIES=OFF ``` -如果出现问题,就只能靠你自己了…… +如果出现问题,则需自行解决…… -Rust 需要网络连接。要禁用 Rust 支持: +Rust 需要网络连接。若要禁用 Rust 支持: ```sh cmake -DENABLE_RUST=OFF ``` -### 运行 ClickHouse 可执行文件 -你可以将系统中安装的生产版本 ClickHouse 二进制文件替换为自己编译的 ClickHouse 二进制文件。 -为此,请按照官方网站上的说明在本机安装 ClickHouse。 +### 运行 ClickHouse 可执行文件 {#running-the-clickhouse-executable-1} + +你可以将系统中安装的生产环境版本 ClickHouse 二进制文件替换为自己编译的 ClickHouse 二进制文件。 +为此,请按照官方文档网站上的说明在你的机器上安装 ClickHouse。 然后运行: ```bash @@ -188,18 +182,19 @@ sudo cp ClickHouse/build/programs/clickhouse /usr/bin/ sudo service clickhouse-server start ``` -请注意,`clickhouse-client`、`clickhouse-server` 等是指向通用共享的 `clickhouse` 二进制文件的符号链接。 +请注意,`clickhouse-client`、`clickhouse-server` 等都是指向公共共享的 `clickhouse` 二进制文件的符号链接。 -也可以使用系统中已安装的 ClickHouse 软件包中的配置文件来运行自定义构建的 ClickHouse 二进制文件: +你也可以使用系统上已安装的 ClickHouse 软件包中的配置文件来运行你自定义构建的 ClickHouse 二进制文件。 ```bash sudo service clickhouse-server stop sudo -u clickhouse ClickHouse/build/programs/clickhouse server --config-file /etc/clickhouse-server/config.xml ``` -### 在任何 Linux 系统上构建 -在 OpenSUSE Tumbleweed 上安装先决条件: +### 在任何 Linux 上构建 {#building-on-any-linux} + +在 OpenSUSE Tumbleweed 上安装依赖项: ```bash sudo zypper install git cmake ninja clang-c++ python lld nasm yasm gawk @@ -209,7 +204,7 @@ cmake -S . -B build cmake --build build ``` -在 Fedora Rawhide 上安装前提条件: +在 Fedora Rawhide 上安装先决条件: ```bash sudo yum update @@ -220,19 +215,20 @@ cmake -S . -B build cmake --build build ``` -### 在 Docker 中构建 -你可以使用以下命令,在本地的、与 CI 类似的环境中运行任意构建: +### 在 Docker 中构建 {#building-in-docker} + +你可以使用以下命令,在与 CI 类似的环境中本地运行任何构建: ```bash -python -m ci.praktika "BUILD_JOB_NAME" +python -m ci.praktika run "BUILD_JOB_NAME" ``` 其中 BUILD_JOB_NAME 是 CI 报告中显示的作业名称,例如 “Build (arm_release)”、“Build (amd_debug)”。 -此命令会拉取包含所有所需依赖项的相应 Docker 镜像 `clickhouse/binary-builder`, -并在其中运行构建脚本:`./ci/jobs/build_clickhouse.py`。 +此命令会拉取包含所有必需依赖项的对应 Docker 镜像 `clickhouse/binary-builder`, +并在该镜像内运行构建脚本:`./ci/jobs/build_clickhouse.py`。 -构建产物将输出到 `./ci/tmp/` 目录中。 +构建产物将生成在 `./ci/tmp/` 中。 -它同时适用于 AMD 和 ARM 架构,除了 Docker 之外不需要任何其他依赖。 +该方式同时适用于 AMD 和 ARM 架构,除已安装 `requests` 模块的 Python 和 Docker 外,无需其他额外依赖。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md index 5b3499cbdd8..77786e515f0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/building_and_benchmarking_deflate_qpl.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# 使用 DEFLATE_QPL 构建 ClickHouse +# 使用 DEFLATE_QPL 构建 ClickHouse {#build-clickhouse-with-deflate_qpl} - 请确保你的主机符合 QPL 所需的[先决条件](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#prerequisites) - 在使用 CMake 构建时,`deflate_qpl` 默认是启用的。如果你不小心修改了该设置,请再次确认构建标志:`ENABLE_QPL=1` @@ -18,7 +18,7 @@ doc_type: 'guide' -# 使用 DEFLATE_QPL 运行基准测试 +# 使用 DEFLATE_QPL 运行基准测试 {#run-benchmark-with-deflate_qpl} @@ -35,7 +35,7 @@ doc_type: 'guide' -## 自动运行星型模式基准测试: +## 自动运行星型模式基准测试: {#run-benchmark-automatically-for-star-schema} ```bash $ cd ./benchmark_sample/client_scripts @@ -53,7 +53,7 @@ $ sh run_ssb.sh -## 环境 +## 环境 {#environment} * CPU:Sapphire Rapids * 操作系统要求请参阅 [System Requirements for QPL](https://intel.github.io/qpl/documentation/get_started_docs/installation.html#system-requirements) @@ -81,7 +81,7 @@ $ accel-config list | grep -P 'iax|state' 如果没有任何输出,说明 IAA 尚未就绪。请重新检查 IAA 的配置。 -## 生成原始数据 +## 生成原始数据 {#generate-raw-data} ```bash $ cd ./benchmark_sample @@ -94,7 +94,7 @@ $ mkdir rawdata_dir && cd rawdata_dir 预计会在 `./benchmark_sample/rawdata_dir/ssb-dbgen` 目录下生成类似 `*.tbl` 的文件: -## 数据库配置 +## 数据库配置 {#database-setup} 将数据库配置为使用 LZ4 编解码器 @@ -164,7 +164,7 @@ SELECT count() FROM lineorder_flat 这表示 IAA 设备尚未准备就绪,你需要重新检查 IAA 的配置。 -## 使用单实例进行基准测试 +## 使用单实例进行基准测试 {#benchmark-with-single-instance} * 在开始基准测试之前,请禁用 C6,并将 CPU 频率调节策略设置为 `performance` @@ -218,7 +218,7 @@ zstd.log 我们主要关注 QPS,请搜索关键字 `QPS_Final` 并收集统计数据。 -## 使用多实例进行基准测试 +## 使用多实例进行基准测试 {#benchmark-with-multi-instances} * 为了减小过多线程导致的内存瓶颈影响,建议使用多实例来运行基准测试。 * 多实例是指在多台(2 或 4 台)服务器上分别连接各自的客户端。 @@ -350,7 +350,7 @@ zstd_2insts.log 我们建议使用 2 个实例的基准测试数据作为最终报告的评审依据。 -## 提示 +## 提示 {#tips} 每次启动新的 ClickHouse 服务器之前,请确保没有 ClickHouse 后台进程在运行。请检查并终止旧进程: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/continuous-integration.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/continuous-integration.md index e07639bfa0a..5a22be94ff1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/continuous-integration.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/continuous-integration.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# 持续集成(CI) +# 持续集成(CI) {#continuous-integration-ci} 当你提交一个 pull request 时,ClickHouse 的[持续集成(CI)系统](tests.md#test-automation)会对你的代码运行一些自动检查。 这会在代码仓库维护者(ClickHouse 团队成员)审查了你的代码并在 pull request 上添加 `can be tested` 标签之后进行。 @@ -75,26 +75,26 @@ git push -## 样式检查 +## 样式检查 {#style-check} 对代码库执行各种样式检查。 *Style Check* 作业中包含的基础检查: -##### cpp +##### cpp {#cpp} 使用 [`ci/jobs/scripts/check_style/check_cpp.sh`](https://github.com/ClickHouse/ClickHouse/blob/master/ci/jobs/scripts/check_style/check_cpp.sh) 脚本执行基于正则表达式的简单代码样式检查(也可以在本地运行)。\ 如果检查失败,请根据[代码样式指南](style.md)修复样式问题。 -##### codespell, aspell +##### codespell, aspell {#codespell} 检查语法错误和拼写错误。 -##### mypy +##### mypy {#mypy} 对 Python 代码执行静态类型检查。 -### 在本地运行样式检查作业 +### 在本地运行样式检查作业 {#running-style-check-locally} 可以在 Docker 容器中本地运行完整的 *Style Check* 作业,命令如下: @@ -112,14 +112,14 @@ python -m ci.praktika run "Style check" --test cpp 除 Python 3 和 Docker 外,无需其他任何依赖。 -## 快速测试 +## 快速测试 {#fast-test} 通常这是在 PR 上运行的第一个检查。 它会构建 ClickHouse 并运行大部分[无状态功能测试](tests.md#functional-tests),但会省略部分测试。 如果该步骤失败,在修复之前不会启动后续检查。 查看报告以确定哪些测试失败,然后按照[这里](/development/tests#running-a-test-locally)的说明在本地重现失败。 -#### 在本地运行快速测试: +#### 在本地运行快速测试: {#running-fast-test-locally} ```sh python -m ci.praktika run "Fast test" [--test 测试名称] @@ -129,11 +129,11 @@ python -m ci.praktika run "Fast test" [--test 测试名称] 只需 Python 3 和 Docker,无需其他依赖。 -## 构建检查 +## 构建检查 {#build-check} 以多种配置构建 ClickHouse,以便在后续步骤中使用。 -### 在本地运行构建 +### 在本地运行构建 {#running-builds-locally} 可以在类似 CI 的本地环境中运行构建,使用: @@ -143,7 +143,7 @@ python -m ci.praktika run "" 除了 Python 3 和 Docker 外不需要其他依赖。 -#### 可用构建任务 +#### 可用构建任务 {#available-build-jobs} 构建任务名称与 CI 报告中的名称完全一致: @@ -181,7 +181,7 @@ python -m ci.praktika run "" **注意:** 对于不属于 “Other Architectures” 类别(该类别使用交叉编译)的构建,你本地机器的架构必须与构建类型一致,才能按照 `BUILD_JOB_NAME` 要求生成构建产物。 -#### 示例 +#### 示例 {#example-run-local} 要在本地运行调试构建: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/contrib.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/contrib.md index 99a4ea29cef..d27dd569045 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/contrib.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/contrib.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# 第三方库 +# 第三方库 {#third-party-libraries} ClickHouse 出于不同目的会使用第三方库,例如连接到其他数据库、在从磁盘加载/保存到磁盘时对数据进行解码/编码,或实现某些专用 SQL 函数。 为避免依赖目标系统中可用的库,每个第三方库都会作为 Git 子模块导入到 ClickHouse 的源代码树中,并与 ClickHouse 一起编译和链接。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/developer-instruction.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/developer-instruction.md index ae1d412973e..7e7528e0bd6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/developer-instruction.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/developer-instruction.md @@ -1,78 +1,74 @@ --- -description: 'ClickHouse 开发的前提条件和设置指南' +description: 'ClickHouse 开发的前提条件与设置指南' sidebar_label: '前提条件' sidebar_position: 5 slug: /development/developer-instruction -title: '开发前提条件' +title: '开发者前提条件' doc_type: 'guide' --- +# 前提条件 {#prerequisites} +ClickHouse 可以在 Linux、FreeBSD 和 macOS 上构建。 +如果你使用的是 Windows,仍然可以在运行 Linux 的虚拟机中构建 ClickHouse,例如在安装了 Ubuntu 的 [VirtualBox](https://www.virtualbox.org/) 中。 -# 前置条件 - -ClickHouse 可以在 Linux、FreeBSD 和 macOS 上编译构建。 -如果您使用 Windows,仍然可以在运行 Linux 的虚拟机中编译构建 ClickHouse,例如使用运行 Ubuntu 的 [VirtualBox](https://www.virtualbox.org/)。 - - - -## 在 GitHub 上创建代码仓库 +## 在 GitHub 上创建仓库 {#create-a-repository-on-github} 要开始为 ClickHouse 开发,你需要一个 [GitHub](https://www.github.com/) 账号。 -如果你还没有 SSH 密钥,请先在本地生成一个 SSH 密钥,并将公钥上传到 GitHub,这是提交补丁的前置条件。 +如果你还没有 SSH 密钥,请在本地生成一个 SSH 密钥,并将公钥上传到 GitHub,因为这是提交补丁的前置条件。 -接下来,在你的个人账号下 fork [ClickHouse 仓库](https://github.com/ClickHouse/ClickHouse/),点击右上角的 “fork” 按钮即可。 +接下来,在你的个人账号中 fork [ClickHouse 仓库](https://github.com/ClickHouse/ClickHouse/),方法是在右上角点击 “Fork” 按钮。 -要贡献更改(例如修复一个 issue 或添加一个功能),请先将你的更改提交到你 fork 中的某个分支,然后创建一个包含这些更改的 “Pull Request” 到主仓库。 +要贡献更改(例如修复 issue 或添加功能),请先将你的修改提交到 fork 后仓库中的某个分支,然后向主仓库创建一个包含这些更改的 Pull Request。 -要使用 Git 仓库,请先安装 Git。比如在 Ubuntu 中,运行: +若要操作 Git 仓库,请先安装 Git。例如,在 Ubuntu 中运行: ```sh sudo apt update sudo apt install git ``` -Git 速查表可在[这里](https://education.github.com/git-cheat-sheet-education.pdf)找到。 -详细的 Git 手册在[这里](https://git-scm.com/book/en/v2)。 +Git 速查表可在[此处](https://education.github.com/git-cheat-sheet-education.pdf)查阅。 +Git 详细手册在[此处](https://git-scm.com/book/en/v2)。 -## 将代码仓库克隆到你的开发机器上 +## 将仓库克隆到你的开发环境中 {#clone-the-repository-to-your-development-machine} -首先,将源文件下载到你的本地开发环境,即克隆该代码仓库: +首先,将源文件下载到你的工作环境中,也就是克隆该仓库: ```sh -git clone git@github.com:your_github_username/ClickHouse.git # 将占位符替换为你的 GitHub 用户名 +git clone git@github.com:your_github_username/ClickHouse.git # 将占位符替换为您的 GitHub 用户名 cd ClickHouse ``` -此命令会创建一个名为 `ClickHouse/` 的目录,其中包含源代码、测试以及其他文件。 -你可以在 URL 后指定一个自定义目录用于检出,但务必确保该路径中不包含空格字符,否则后续的构建过程可能会失败。 +此命令会创建一个 `ClickHouse/` 目录,其中包含源代码、测试以及其他文件。 +你可以在 URL 之后指定一个自定义检出目录,但务必要确保该路径中不包含空格,否则可能会在后续构建过程中导致失败。 -ClickHouse 的 Git 仓库使用子模块来拉取第三方库代码。 -子模块默认不会被检出。 -你可以执行以下任一操作: +ClickHouse 的 Git 仓库使用子模块来拉取第三方库。 +默认情况下不会检出子模块。 +你可以: -* 使用带有 `--recurse-submodules` 选项的 `git clone` 命令; +* 使用带有 `--recurse-submodules` 选项的 `git clone`; -* 如果运行 `git clone` 时未使用 `--recurse-submodules`,则运行 `git submodule update --init --jobs <N>` 来显式检出所有子模块。(例如可以将 `<N>` 设置为 `12` 以并行下载。) +* 如果 `git clone` 未使用 `--recurse-submodules`,则运行 `git submodule update --init --jobs ` 显式检出所有子模块。(例如可以将 `` 设为 `12` 以并行下载。) -* 如果运行 `git clone` 时未使用 `--recurse-submodules`,并且你希望使用 [sparse](https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/) 和 [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) 子模块检出,以省略子模块中不需要的文件和历史、从而节省空间(约 5 GB 而不是约 15 GB),请运行 `./contrib/update-submodules.sh`。这种方式在 CI 中使用,但不推荐用于本地开发,因为它会让使用子模块变得不太方便且更慢。 +* 如果 `git clone` 未使用 `--recurse-submodules`,并且你希望使用 [shallow](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/) 子模块检出以省略子模块的历史记录从而节省空间,则运行 `./contrib/update-submodules.sh`。这种方式由 CI 使用,但不推荐用于本地开发,因为它会让使用子模块变得不太方便且更慢。 -要检查 Git 子模块的状态,请运行 `git submodule status`。 +要检查 Git 子模块的状态,运行 `git submodule status`。 如果你收到如下错误信息 ```bash -权限被拒绝(publickey)。 -致命错误:无法从远程仓库读取。 +权限被拒绝(publickey)。 +致命错误:无法从远程仓库读取。 -请确保您拥有正确的访问权限, +请确保您拥有正确的访问权限, 且该仓库存在。 ``` -缺少用于连接 GitHub 的 SSH 密钥。 +用于连接 GitHub 的 SSH 密钥不存在。 这些密钥通常位于 `~/.ssh`。 -要使 SSH 密钥生效,你需要在 GitHub 的设置中上传它们。 +要让 SSH 密钥被 GitHub 接受,你需要在 GitHub 的设置页面中上传它们。 你也可以通过 HTTPS 克隆该仓库: @@ -80,82 +76,76 @@ ClickHouse 的 Git 仓库使用子模块来拉取第三方库代码。 git clone https://github.com/ClickHouse/ClickHouse.git ``` -然而,这样做并不能让你将更改推送到服务器。 -你仍然可以暂时这样使用,并在之后添加 SSH 密钥,再通过 `git remote` 命令替换仓库的远程地址。 +不过,这样你还不能将更改推送到服务器。 +你仍然可以先暂时这样使用,之后再添加 SSH 密钥,并通过 `git remote` 命令替换仓库的远程地址。 -你也可以将原始 ClickHouse 仓库地址添加到本地仓库,以便从那里拉取更新: +你也可以在本地仓库中添加原始 ClickHouse 仓库地址,以便从那里拉取更新: ```sh git remote add upstream git@github.com:ClickHouse/ClickHouse.git ``` -成功运行此命令后,你就可以通过运行 `git pull upstream master` 从 ClickHouse 主仓库拉取更新。 +成功运行此命令后,你就可以通过执行 `git pull upstream master` 从 ClickHouse 主仓库拉取更新。 :::tip -请不要直接使用 `git push`,你可能会推送到错误的远程仓库和/或错误的分支。 -最好显式指定远程和分支的名称,例如 `git push origin my_branch_name`。 +请不要直接使用 `git push`,否则你可能会推送到错误的远程仓库或错误的分支。 +最好显式指定远程和分支名,例如 `git push origin my_branch_name`。 ::: ## 编写代码 {#writing-code} -下面是一些在为 ClickHouse 编写代码时可能有用的快速链接: +下面是一些在为 ClickHouse 编写代码时可能会用到的快速链接: -- [ClickHouse 架构](/development/architecture/). -- [代码风格指南](/development/style/). +- [ClickHouse 架构](/development/architecture/) +- [代码风格指南](/development/style/) - [第三方库](/development/contrib#adding-and-maintaining-third-party-libraries) - [编写测试](/development/tests/) -- [开放的 Issue](https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aopen+is%3Aissue+label%3A%22easy+task%22) +- [开放的 issue](https://github.com/ClickHouse/ClickHouse/issues?q=is%3Aopen+is%3Aissue+label%3A%22easy+task%22) ### IDE {#ide} -[Visual Studio Code](https://code.visualstudio.com/) 和 [Neovim](https://neovim.io/) 是在开发 ClickHouse 时实践证明很好用的两个选项。如果你使用 VS Code,我们推荐使用 [clangd 扩展](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) 替代 IntelliSense,因为它的性能要高得多。 +[Visual Studio Code](https://code.visualstudio.com/) 和 [Neovim](https://neovim.io/) 在开发 ClickHouse 的实践中被证明是两个效果不错的选项。若你使用 VS Code,我们建议使用 [clangd 扩展](https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd) 来替换 IntelliSense,因为它的性能要高得多。 -[CLion](https://www.jetbrains.com/clion/) 是另一个很好的选择。不过,在像 ClickHouse 这样的大型项目上,它可能会比较慢。使用 CLion 时需要注意以下几点: +[CLion](https://www.jetbrains.com/clion/) 是另一个很好的选择。不过,在像 ClickHouse 这样的大型项目上,它可能会更慢。使用 CLion 时需要注意以下几点: - CLion 会自行创建一个 `build` 目录,并自动选择 `debug` 作为构建类型 -- 它使用的是在 CLion 中定义的 CMake 版本,而不是你自己安装的版本 +- 它使用的是在 CLion 中定义的 CMake 版本,而不是你自行安装的版本 - CLion 会使用 `make` 来运行构建任务,而不是 `ninja`(这是正常行为) 你也可以使用其他 IDE,例如 [Sublime Text](https://www.sublimetext.com/)、[Qt Creator](https://www.qt.io/product/development-tools) 或 [Kate](https://kate-editor.org/)。 +## 创建 Pull Request {#create-a-pull-request} - -## 创建拉取请求 {#create-a-pull-request} - -在 GitHub 的界面中进入你自己的 fork 仓库。 +在 GitHub 的 UI 中打开你 fork 出来的仓库。 如果你是在某个分支上开发的,需要先选择该分支。 -页面上会有一个 “Pull request” 按钮。 -本质上,这表示“创建一个请求,将我的更改合并到主仓库中”。 - -即使工作尚未完成,你也可以创建拉取请求。 -在这种情况下,请在标题开头加入 “WIP”(work in progress),之后可以再修改。 -这有助于协作进行评审和讨论更改,也便于运行所有可用的测试。 -请务必提供你所做更改的简要说明,之后会用它来生成发布版本的变更日志。 - -当 ClickHouse 员工为你的 PR 添加 “can be tested” 标签后,测试将开始执行。 -部分初始检查结果(例如代码风格)会在几分钟内完成。 -构建检查结果通常会在半小时内返回。 -主测试集的结果通常会在一小时内完成。 +界面上会有一个名为 “Pull request” 的按钮。 +本质上,这意味着“创建一个请求,将我的更改合并到主仓库中”。 -系统会为你的拉取请求单独准备 ClickHouse 二进制构建。 -要获取这些构建,请点击检查列表中 “Builds” 条目旁边的 “Details” 链接。 -在那里你会找到已构建的 ClickHouse .deb 软件包的直接链接,你甚至可以将其部署到生产服务器上(如果你不惧风险的话)。 +即使工作尚未完成,也可以创建 Pull Request。 +在这种情况下,请在标题开头加上 “WIP”(work in progress,进行中),之后可以再修改。 +这对于协同审阅和讨论更改,以及运行所有可用测试都非常有用。 +请务必对你的更改内容做一个简要说明,之后会用它来生成发布版本的变更日志。 +当 ClickHouse 员工给你的 PR 加上 “can be tested” 标签后,测试就会开始。 +部分初始检查结果(例如代码风格)会在几分钟内给出。 +构建检查结果会在半小时内完成。 +主要测试集的结果通常会在一小时内返回。 +系统会为你的 Pull Request 单独准备 ClickHouse 二进制构建。 +要获取这些构建,点击检查列表中 “Builds” 条目旁边的 “Details” 链接。 +在那里你会找到已构建的 ClickHouse `.deb` 包的直接链接,你甚至可以将它们部署到生产环境服务器上(如果你不担心的话)。 ## 编写文档 {#write-documentation} -每个添加新功能的 pull request 都必须附带相应文档。 -如果你希望预览文档修改内容,本地构建文档页面的步骤说明可在 README.md 文件中找到,链接在[这里](https://github.com/ClickHouse/clickhouse-docs)。 -在向 ClickHouse 添加新函数时,你可以参考以下模板来编写文档: - - +每个添加新功能的 pull request 都必须附带相应的文档。 +如果你想预览文档修改内容,如何在本地构建文档页面的说明请参见 README.md 文件:[链接](https://github.com/ClickHouse/clickhouse-docs)。 +在向 ClickHouse 添加新函数时,你可以参考下面的模板: ````markdown -# newFunctionName +# newFunctionName {#newfunctionname} -此处为该函数的简要说明,应简要描述其功能以及一个典型的使用场景。 +此处简要描述该函数。应简要说明其功能及典型使用场景。 **语法** @@ -165,42 +155,42 @@ newFunctionName(arg1, arg2[, arg3]) **参数** -- `arg1` — 参数说明。 [DataType](../data-types/float.md) -- `arg2` — 参数说明。 [DataType](../data-types/float.md) -- `arg3` — 可选参数说明(可选)。 [DataType](../data-types/float.md) +- `arg1` — 参数描述。 [DataType](../data-types/float.md) +- `arg2` — 参数描述。 [DataType](../data-types/float.md) +- `arg3` — 可选参数描述(可选)。 [DataType](../data-types/float.md) **实现细节** -如有需要,在此说明实现细节。 +如有必要,此处描述实现细节。 **返回值** -- 返回 {在此填写函数返回内容}。 [DataType](../data-types/float.md) +- 返回 {在此插入函数返回内容}。 [DataType](../data-types/float.md) **示例** 查询: \```sql -SELECT '在此编写示例查询'; +SELECT 'write your example query here'; \``` 响应: \```response ┌───────────────────────────────────┐ -│ 查询结果 │ +│ the result of the query │ └───────────────────────────────────┘ \``` ```` -## 使用测试数据 +## 使用测试数据 {#using-test-data} -在开发 ClickHouse 时,经常需要加载贴近真实场景的数据集。 -这对于性能测试尤为重要。 -我们专门准备了一套经过匿名化处理的 Web 分析数据。 -它额外需要大约 3GB 的可用磁盘空间。 +在开发 ClickHouse 时,经常需要加载接近真实的数据集。 +这对性能测试尤为重要。 +我们准备了一套经过匿名化处理的网站分析数据。 +使用它大约需要额外 3GB 的可用磁盘空间。 ```sh sudo apt install wget xz-utils @@ -216,22 +206,18 @@ SELECT '在此编写示例查询'; 在 clickhouse-client 中: + ```sql CREATE DATABASE IF NOT EXISTS test; -``` - CREATE TABLE test.hits ( WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree PARTITION BY toYYYYMM(EventDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID), EventTime); +CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); +``` - -CREATE TABLE test.visits ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), `Goals.ID` Array(UInt32), `Goals.Serial` Array(UInt32), `Goals.EventTime` Array(DateTime), `Goals.Price` Array(Int64), `Goals.OrderID` Array(String), `Goals.CurrencyID` Array(UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, `TraficSource.ID` Array(Int8), `TraficSource.SearchEngineID` Array(UInt16), `TraficSource.AdvEngineID` Array(UInt8), `TraficSource.PlaceID` Array(UInt16), `TraficSource.SocialSourceNetworkID` Array(UInt8), `TraficSource.Domain` Array(String), `TraficSource.SearchPhrase` Array(String), `TraficSource.SocialSourcePage` Array(String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, `ParsedParams.Key1` Array(String), `ParsedParams.Key2` Array(String), `ParsedParams.Key3` Array(String), `ParsedParams.Key4` Array(String), `ParsedParams.Key5` Array(String), `ParsedParams.ValueDouble` Array(Float64), `Market.Type` Array(UInt8), `Market.GoalID` Array(UInt32), `Market.OrderID` Array(String), `Market.OrderPrice` Array(Int64), `Market.PP` Array(UInt32), `Market.DirectPlaceID` Array(UInt32), `Market.DirectOrderID` Array(UInt32), `Market.DirectBannerID` Array(UInt32), `Market.GoodID` Array(String), `Market.GoodName` Array(String), `Market.GoodQuantity` Array(Int32), `Market.GoodPrice` Array(Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) SAMPLE BY intHash32(UserID) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID); - -```` - -导入数据: +导入数据: ```bash clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.hits FORMAT TSV" < hits_v1.tsv clickhouse-client --max_insert_block_size 100000 --query "INSERT INTO test.visits FORMAT TSV" < visits_v1.tsv -```` +``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/index.md index 6d821704281..0b7003dc064 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/index.md @@ -7,8 +7,8 @@ doc_type: 'landing-page' 在本节文档中,您将找到以下页面: -{/* 下面的目录是由如下脚本根据 YAML - front matter 字段:title、description、slug 自动生成的: +{/* 下方的目录由 YAML front matter 中的以下字段自动生成: + title、description、slug。生成脚本位于: https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh @@ -19,21 +19,21 @@ doc_type: 'landing-page' | Page | Description | | ------------------------------------------------------------------------------------------- | ----------------------------------------------- | -| [Developer Prerequisites](/development/developer-instruction) | ClickHouse 开发的前提条件和环境配置说明 | +| [Developer Prerequisites](/development/developer-instruction) | ClickHouse 开发的前置条件和环境配置说明 | | [How to Build ClickHouse on Linux](/development/build) | 在 Linux 系统上从源码构建 ClickHouse 的分步指南 | | [Build on macOS for macOS](/development/build-osx) | 在 macOS 系统上从源码构建 ClickHouse 的指南 | -| [Build on Linux for macOS](/development/build-cross-osx) | 在 Linux 上为 macOS 系统交叉编译 ClickHouse 的指南 | +| [Build on Linux for macOS](/development/build-cross-osx) | 从 Linux 交叉编译面向 macOS 系统的 ClickHouse 的指南 | | [How to Build ClickHouse on Linux for AARCH64](/development/build-cross-arm) | 在 Linux 上为 AARCH64 架构从源码构建 ClickHouse 的指南 | | [How to Build ClickHouse on Linux for RISC-V 64](/development/build-cross-riscv) | 在 Linux 上为 RISC-V 64 架构从源码构建 ClickHouse 的指南 | | [Build on Linux for s390x (zLinux)](/development/build-cross-s390x) | 在 Linux 上为 s390x 架构从源码构建 ClickHouse 的指南 | -| [Build on Linux for E2K](/development/build-e2k) | 在 Linux 上为 E2K 架构从源码构建 ClickHouse 的指南 | | [Build on Linux for LoongArch64](/development/build-cross-loongarch) | 在 Linux 上为 LoongArch64 架构从源码构建 ClickHouse 的指南 | +| [Build on Linux for E2K](/development/build-e2k) | 在 Linux 上为 E2K 架构从源码构建 ClickHouse 的指南 | | [Testing ClickHouse](/development/tests) | 关于测试 ClickHouse 并运行测试套件的指南 | | [Architecture Overview](/development/architecture) | 对 ClickHouse 架构及其列式设计的全面概述 | -| [Continuous Integration (CI)](/development/continuous-integration) | ClickHouse 持续集成系统概览 | -| [Third-Party Libraries](/development/contrib) | 介绍 ClickHouse 第三方库使用方式以及如何添加和维护第三方库的页面 | +| [Continuous Integration (CI)](/development/continuous-integration) | ClickHouse 持续集成系统的概览 | +| [Third-Party Libraries](/development/contrib) | 介绍 ClickHouse 使用第三方库的情况,以及如何添加和维护第三方库 | | [C++ Style Guide](/development/style) | ClickHouse C++ 开发的代码风格规范 | -| [Build Clickhouse with DEFLATE_QPL](/development/building_and_benchmarking_deflate_qpl) | 使用 DEFLATE_QPL 编解码器构建 ClickHouse 并运行基准测试的指南 | -| [Integrating Rust Libraries](/development/integrating_rust_libraries) | 将 Rust 库集成到 ClickHouse 的指南 | +| [Build Clickhouse with DEFLATE_QPL](/development/building_and_benchmarking_deflate_qpl) | 使用 DEFLATE_QPL 编解码器构建 ClickHouse 并运行基准测试的说明 | +| [Integrating Rust Libraries](/development/integrating_rust_libraries) | 将 Rust 库集成到 ClickHouse 中的指南 | -{/*AUTOGENERATED_END*/ } +{/*AUTOGENERATED_START*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md index 3567871bd97..8455712f00b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/integrating_rust_libraries.md @@ -6,7 +6,7 @@ title: '集成 Rust 库' doc_type: 'guide' --- -# Rust 库 +# Rust 库 {#rust-libraries} Rust 库的集成将以集成 BLAKE3 哈希函数为例进行说明。 @@ -80,7 +80,6 @@ pub unsafe extern "C" fn blake3_apply_shim( 此外,对于每个与 C 兼容的项,你都应使用属性 #[no_mangle] 和 `extern "C"`。否则库可能会被错误地编译,并且 cbindgen 将无法进行头文件的自动生成。 - 在完成以上所有步骤之后,可以在一个小型项目中测试该库,以发现所有与兼容性或头文件生成相关的问题。若在头文件生成过程中出现任何问题,可以尝试通过 `cbindgen.toml` 文件进行配置(可以在此处找到一个模板:[https://github.com/eqrion/cbindgen/blob/master/template.toml](https://github.com/eqrion/cbindgen/blob/master/template.toml))。 值得一提的是在集成 BLAKE3 时出现的一个问题: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/style.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/style.md index f14095ad23b..a1bb7a6f8dd 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/style.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/style.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# C++ 编码风格指南 +# C++ 编码风格指南 {#c-style-guide} @@ -22,7 +22,7 @@ doc_type: 'guide' -## 格式 +## 格式 {#formatting} **1.** 大部分格式由 `clang-format` 自动完成。 @@ -214,7 +214,7 @@ for (Names::const_iterator it = column_names.begin(); it != column_names.end(); ``` -## 注释 +## 注释 {#comments} **1.** 一定要为所有非一目了然的代码部分添加注释。 @@ -312,7 +312,7 @@ void executeQuery( ``` -## 命名 +## 命名 {#names} **1.** 变量名和类成员名应使用小写字母并以下划线分隔。 @@ -431,7 +431,7 @@ enum class CompressionMethod **17.** 包含 C++ 源代码的文件必须使用 `.cpp` 扩展名。头文件必须使用 `.h` 扩展名。 -## 如何编写代码 +## 如何编写代码 {#how-to-write-code} **1.** 内存管理。 @@ -700,7 +700,7 @@ auto s = std::string{"Hello"}; **26.** 对于虚函数,在基类中使用 `virtual` 关键字,而在派生类中不要再写 `virtual`,改为写 `override`。 -## 未使用的 C++ 特性 +## 未使用的 C++ 特性 {#unused-features-of-c} **1.** 不使用虚继承。 @@ -830,7 +830,7 @@ CPU 指令集为我们服务器中支持的最小公共子集,目前为 SSE 4. -## 其他补充建议 +## 其他补充建议 {#additional-recommendations} **1.** 对来自 `stddef.h` 的类型显式添加 `std::` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/development/tests.md b/i18n/zh/docusaurus-plugin-content-docs/current/development/tests.md index 8c2ee07ad81..52055a1ef01 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/development/tests.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/development/tests.md @@ -9,11 +9,11 @@ doc_type: 'guide' -# 测试 ClickHouse +# 测试 ClickHouse {#testing-clickhouse} -## 功能测试 +## 功能测试 {#functional-tests} 功能测试是最简单、最易使用的一类测试。 ClickHouse 的大部分特性都可以通过功能测试来验证,并且对于每一个可以用这种方式测试的 ClickHouse 代码变更,功能测试都是必需的。 @@ -34,7 +34,7 @@ ClickHouse 的大部分特性都可以通过功能测试来验证,并且对于 在测试 `DateTime` 和 `DateTime64` 数据类型时,一个常见错误是误以为服务器会使用某个特定时区(例如 “UTC”)。实际情况并非如此,在 CI 测试运行中,时区是被刻意随机化的。最简单的解决办法是为测试值显式指定时区,例如 `toDateTime64(val, 3, 'Europe/Amsterdam')`。 ::: -### 在本地运行测试 +### 在本地运行测试 {#running-a-test-locally} 在本地启动 ClickHouse 服务器,并监听默认端口(9000)。 例如,要运行测试 `01428_hash_set_nan_key`,切换到代码仓库目录并执行以下命令: @@ -49,7 +49,7 @@ PATH=:$PATH tests/clickhouse-test 01428_hash_set_nan_ 可以运行全部测试,或者通过为测试名称提供过滤字符串来运行部分测试:`./clickhouse-test substring`。 也可以选择并行运行测试,或以随机顺序运行测试。 -### 添加新测试 +### 添加新测试 {#adding-a-new-test} 要添加新测试,首先在 `queries/0_stateless` 目录中创建一个 `.sql` 或 `.sh` 文件。 然后使用 `clickhouse-client < 12345_test.sql > 12345_test.reference` 或 `./12345_test.sh > ./12345_test.reference` 生成对应的 `.reference` 文件。 @@ -76,7 +76,7 @@ sudo ./install.sh * 确保其他测试不会测试相同内容(即先用 grep 检查)。 ::: -### 限制测试运行 +### 限制测试运行 {#restricting-test-runs} 一个测试可以具有零个或多个*标签*,用于指定该测试在 CI 中在哪些上下文中运行。 @@ -95,9 +95,9 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Tags: no-fasttest, no-replicated-database -# - no-fasttest: <请在此处说明使用该标签的原因> -# - no-replicated-database: <请在此处说明原因> +# Tags: no-fasttest, no-replicated-database {#tags-no-fasttest-no-replicated-database} +# - no-fasttest: <请在此处说明使用该标签的原因> {#no-fasttest-provide_a_reason_for_the_tag_here} +# - no-replicated-database: <请在此处说明原因> {#no-replicated-database-provide_a_reason_here} ``` 可用标签列表: @@ -134,7 +134,7 @@ SELECT 1 除上述设置外,你还可以使用 `system.build_options` 中的 `USE_*` 标志来定义是否使用特定的 ClickHouse 特性。 例如,如果你的测试使用了 MySQL 表,则应添加标签 `use-mysql`。 -### 为随机设置指定限制 +### 为随机设置指定限制 {#specifying-limits-for-random-settings} 测试可以为在测试运行期间被随机化的设置指定允许的最小值和最大值。 @@ -143,8 +143,8 @@ SELECT 1 ```bash #!/usr/bin/env bash -# Tags: no-fasttest -# 随机设置限制:max_block_size=(1000, 10000); index_granularity=(100, None) +# Tags: no-fasttest {#tags-no-fasttest} +# 随机设置限制:max_block_size=(1000, 10000); index_granularity=(100, None) {#random-settings-limits-max_block_size1000-10000-index_granularity100-none} ``` 对于 `.sql` 测试,标签以 SQL 注释的形式写在相邻的一行,或写在第一行: @@ -157,7 +157,7 @@ SELECT 1 如果你只需要指定其中一个限制值,可以将另一个设置为 `None`。 -### 选择测试名称 +### 选择测试名称 {#choosing-the-test-name} 测试名称以五位数字前缀开头,后面跟一个描述性名称,例如 `00422_hash_function_constexpr.sql`。 要选择前缀,先在目录中找到已经存在的最大前缀,然后将其加一。 @@ -168,7 +168,7 @@ ls tests/queries/0_stateless/[0-9]*.reference | tail -n 1 与此同时,可能会添加一些具有相同数字前缀的其他测试,但这没问题,不会导致任何问题,你之后也不需要对其进行修改。 -### 检查必须出现的错误 +### 检查必须出现的错误 {#checking-for-an-error-that-must-occur} 有时你希望测试,当查询不正确时会出现服务器错误。我们在 SQL 测试中为此提供了特殊注解,形式如下: @@ -184,12 +184,12 @@ SELECT x; -- { serverError 49 } 只检查错误码。 如果现有的错误码对你的需求不够精确,可以考虑新增一个错误码。 -### 测试分布式查询 +### 测试分布式查询 {#testing-a-distributed-query} 如果你想在功能测试中使用分布式查询,可以使用 `remote` 表函数,并使用 `127.0.0.{1..2}` 这些地址让服务器查询自身;或者你也可以在服务器配置文件中使用预定义的测试集群,例如 `test_shard_localhost`。 记得在测试名称中加入 `shard` 或 `distributed` 这样的关键词,以便在 CI 中在正确的配置下运行该测试,即服务器已配置为支持分布式查询。 -### 使用临时文件 +### 使用临时文件 {#working-with-temporary-files} 有时在 shell 测试中,你可能需要临时创建一个文件用于操作。 请注意,某些 CI 检查会并行运行测试,因此如果你在脚本中创建或删除的临时文件没有唯一名称,就可能导致某些 CI 检查(例如 “Flaky”)失败。 @@ -217,7 +217,7 @@ SELECT x; -- { serverError 49 } -## 单元测试 +## 单元测试 {#unit-tests} 当你希望测试的不是整个 ClickHouse,而是某个独立的库或类时,单元测试会非常有用。 你可以通过 `ENABLE_TESTS` CMake 选项来启用或禁用测试的构建。 @@ -272,7 +272,7 @@ Quorum 测试是在 ClickHouse 开源之前由一个独立团队编写的。 -## 手动测试 +## 手动测试 {#manual-testing} 在开发新功能时,对其进行手动测试也是合理的。 你可以按照以下步骤进行: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/dictionary/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/dictionary/index.md index 1b6bdf69811..03d717ad336 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/dictionary/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/dictionary/index.md @@ -1,8 +1,8 @@ --- slug: /dictionary -title: '数据字典' -keywords: ['数据字典', '字典'] -description: '数据字典以键值对形式组织数据,便于快速查找。' +title: '字典' +keywords: ['dictionary', 'dictionaries'] +description: '字典以键值对形式表示数据,以支持快速查找。' doc_type: 'reference' --- @@ -11,36 +11,35 @@ import dictionaryLeftAnyJoin from '@site/static/images/dictionary/dictionary-lef import Image from '@theme/IdealImage'; -# Dictionary +# 字典 {#dictionary} -ClickHouse 中的 Dictionary 提供了一种在内存中以[键值](https://en.wikipedia.org/wiki/Key%E2%80%93value_database)形式表示来自多种[内部和外部数据源](/sql-reference/dictionaries#dictionary-sources)的数据的机制,并针对超低延迟的查找查询进行了优化。 +ClickHouse 中的字典以内存中的 [key-value](https://en.wikipedia.org/wiki/Key%E2%80%93value_database) 形式表示来自各种[内部和外部数据源](/sql-reference/dictionaries#dictionary-sources)的数据,并针对超低延迟的查找查询进行了优化。 -Dictionary 适用于: -- 提升查询性能,特别是在与 `JOIN` 配合使用时 -- 在不减慢摄取过程的情况下,对摄取数据进行实时丰富 +字典可用于: -ClickHouse 中 Dictionary 的使用场景 +- 提高查询性能,尤其是在与 `JOIN` 一起使用时 +- 在不减慢摄取过程的情况下,实时丰富已摄取的数据 +ClickHouse 中字典的使用场景 +## 使用字典加速 JOIN {#speeding-up-joins-using-a-dictionary} -## 使用 Dictionary 加速 JOIN +字典(Dictionary)可以用于加速特定类型的 `JOIN`:即 [`LEFT ANY` 类型](/sql-reference/statements/select/join#supported-types-of-join),其中 JOIN 键需要与底层键值存储的键属性匹配。 -Dictionaries 可用于加速特定类型的 `JOIN`:即 [`LEFT ANY` 类型](/sql-reference/statements/select/join#supported-types-of-join),其中 JOIN 键需要与底层键值存储的键属性匹配。 +Using Dictionary with LEFT ANY JOIN -在 LEFT ANY JOIN 中使用 Dictionary +在这种情况下,ClickHouse 可以利用字典来执行 [Direct Join](https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4#direct-join)。这是 ClickHouse 中最快的 JOIN 算法,适用于右表的底层[表引擎](/engines/table-engines)支持低延迟键值请求的场景。ClickHouse 为此提供了三种表引擎:[Join](/engines/table-engines/special/join)(本质上是预计算的哈希表)、[EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) 和 [Dictionary](/engines/table-engines/special/dictionary)。我们将介绍基于字典的方案,但三种引擎的机制是相同的。 -在这种情况下,ClickHouse 可以利用 Dictionary 执行 [Direct Join](https://clickhouse.com/blog/clickhouse-fully-supports-joins-direct-join-part4#direct-join)。这是 ClickHouse 速度最快的 JOIN 算法,适用于右表的底层 [table engine](/engines/table-engines) 支持低延迟键值请求的场景。ClickHouse 提供了三种满足该条件的表引擎:[Join](/engines/table-engines/special/join)(本质上是一个预计算哈希表)、[EmbeddedRocksDB](/engines/table-engines/integrations/embedded-rocksdb) 和 [Dictionary](/engines/table-engines/special/dictionary)。我们将重点介绍基于 Dictionary 的方法,但这三种引擎的机制是相同的。 +Direct Join 算法要求右表由字典驱动,这样来自该表、需要参与 JOIN 的数据就已经以内存中的低延迟键值数据结构形式存在。 -Direct Join 算法要求右表由 Dictionary 提供支持,即该表中需要参与 JOIN 的数据已以内存中的低延迟键值数据结构形式存在。 +### 示例 {#example} -### 示例 +使用 [Stack Overflow 数据集](/getting-started/example-datasets/stackoverflow),来回答这样一个问题: +*在 Hacker News 上,关于 SQL 的帖子中,哪一条是最具争议性的?* -使用 Stack Overflow 数据集,让我们来回答这个问题: -*关于 SQL 的哪条 Hacker News 帖子最具争议性?* +我们将“有争议性”定义为:帖子获得的赞成票和反对票数量相近。我们会计算这两者的绝对差值,数值越接近 0,代表争议越大。我们还假设帖子必须至少有 10 个赞成票和 10 个反对票——几乎没人投票的帖子并不算很有争议。 -我们将“具争议性”定义为:帖子获得的赞成票和反对票数量相近。我们计算两者的绝对差值,值越接近 0 表示争议越大。我们假定帖子至少要有 10 个赞成票和 10 个反对票——没人投票的帖子并不算很有争议。 - -在数据已规范化的前提下,这个查询目前需要在 `posts` 和 `votes` 表之间执行一次 `JOIN`: +在对数据进行规范化之后,这个查询目前需要对 `posts` 表和 `votes` 表执行一次 `JOIN` 操作: ```sql WITH PostIds AS @@ -79,38 +78,38 @@ UpVotes: 13 DownVotes: 13 Controversial_ratio: 0 -返回 1 行。用时:1.283 秒。已处理 4.1844 亿行,7.23 GB(3.2607 亿行/秒,5.63 GB/秒)。 -内存峰值:3.18 GiB。 +结果集包含 1 行。耗时: 1.283 秒。处理了 4.1844 亿行,7.23 GB (每秒 3.2607 亿行,5.63 GB/秒)。 +峰值内存使用: 3.18 GiB。 ``` -> **在 `JOIN` 的右侧使用较小的数据集**:这个查询看起来比必要的更冗长,因为对 `PostId` 的过滤同时发生在外层查询和子查询中。这是一种性能优化,用于确保查询响应时间足够快。为了获得最佳性能,请始终确保 `JOIN` 右侧的数据集较小,并尽可能小。关于如何优化 JOIN 性能以及了解可用算法的更多提示,我们推荐阅读[这一系列博客文章](https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1)。 +> **在 `JOIN` 的右侧使用较小的数据集**:这个查询看起来比实际需要的更啰嗦一些,因为对 `PostId` 的过滤同时出现在外层查询和子查询中。这是一种性能优化,用于确保查询响应时间足够快。为了获得最佳性能,应始终确保 `JOIN` 右侧的数据集更小,并且尽可能小。关于优化 JOIN 性能以及理解可用算法的建议,我们推荐阅读[这系列博客文章](https://clickhouse.com/blog/clickhouse-fully-supports-joins-part1)。 -虽然这个查询很快,但它依赖我们谨慎地编写 `JOIN` 才能获得良好的性能。理想情况下,我们只需先将帖子过滤为那些包含 “SQL” 的内容,然后再查看这一子集博客的 `UpVote` 和 `DownVote` 计数来计算我们的指标。 +虽然这个查询很快,但它要求我们在编写 `JOIN` 时格外小心,才能获得良好的性能。理想情况下,我们只需先过滤出包含“SQL”的帖子,然后再查看这部分帖子对应的 `UpVote` 和 `DownVote` 计数来计算我们的指标。 -#### 使用字典 -为了演示这些概念,我们为投票数据使用字典。由于字典通常存放在内存中([ssd_cache](/sql-reference/dictionaries#ssd_cache) 是个例外),用户应当注意数据的大小。先确认一下我们的 `votes` 表的大小: +#### 应用字典 {#applying-a-dictionary} +为了演示这些概念,我们为投票数据使用一个字典。由于字典通常存放在内存中([ssd_cache](/sql-reference/dictionaries#ssd_cache) 是一个例外),用户应当注意数据的大小。先确认一下我们的 `votes` 表的大小: ```sql -SELECT 表, - formatReadableSize(sum(data_compressed_bytes)) AS 压缩后大小, - formatReadableSize(sum(data_uncompressed_bytes)) AS 未压缩大小, - round(sum(data_uncompressed_bytes) / sum(data_compressed_bytes), 2) AS 压缩比 +SELECT table, + formatReadableSize(sum(data_compressed_bytes)) AS compressed_size, + formatReadableSize(sum(data_uncompressed_bytes)) AS uncompressed_size, + round(sum(data_uncompressed_bytes) / sum(data_compressed_bytes), 2) AS ratio FROM system.columns -WHERE 表 IN ('votes') -GROUP BY 表 +WHERE table IN ('votes') +GROUP BY table -┌─表───────────┬─压缩后大小─┬─未压缩大小─┬─压缩比─┐ +┌─table───────────┬─compressed_size─┬─uncompressed_size─┬─ratio─┐ │ votes │ 1.25 GiB │ 3.79 GiB │ 3.04 │ └─────────────────┴─────────────────┴───────────────────┴───────┘ ``` -数据将在我们的字典中以未压缩形式存储,因此如果要在字典中存储所有列(实际我们不会这么做),则至少需要 4GB 内存。字典会在集群中的各个节点间进行复制,因此这部分内存需要为*每个节点*预留。 +数据将在我们的字典中以未压缩形式存储,因此如果要将所有列(实际上我们不会这样做)都存入字典,至少需要 4GB 内存。字典会在集群中进行复制,因此这部分内存需要 *按节点* 预留。 -> 在下面的示例中,我们字典的数据来源于一张 ClickHouse 表。虽然这是最常见的字典数据源,但还支持[多种数据源](/sql-reference/dictionaries#dictionary-sources),包括文件、HTTP,以及包含 [Postgres](/sql-reference/dictionaries#postgresql) 在内的各类数据库。正如下文所示,字典可以自动刷新,是确保那些经常变动的小型数据集始终可以用于直接 JOIN 的理想方式。 +> 在下面的示例中,我们字典的数据来源于一个 ClickHouse 表。虽然这是字典最常见的数据源,但还支持[多种数据源](/sql-reference/dictionaries#dictionary-sources),包括文件、HTTP 以及包括 [Postgres](/sql-reference/dictionaries#postgresql) 在内的各类数据库。正如我们将展示的那样,字典可以自动刷新,为小型且经常变更的数据集提供了一种理想方式,使其可用于直接进行 join 操作。 -我们的字典需要一个用于执行查找的主键。在概念上,它与事务型数据库中的主键相同,并且应当是唯一的。上面的查询需要在关联键 `PostId` 上进行查找。字典本身则应填充为来自 `votes` 表的每个 `PostId` 的赞成票和反对票的总数。下面是获取该字典数据的查询: +我们的字典需要一个用于执行查找的主键。这在概念上与事务型数据库中的主键相同,并且必须唯一。上面的查询需要在 join 键 `PostId` 上执行查找。字典应相应地填充为来自 `votes` 表的每个 `PostId` 的赞成票和反对票总数。下面是获取该字典数据的查询: ```sql SELECT PostId, @@ -120,7 +119,7 @@ FROM votes GROUP BY PostId ``` -要创建我们的字典,我们需要执行以下 DDL——请注意其中使用了上面的查询: +要创建我们的字典,需要使用以下 DDL——请注意其中使用了上面的查询: ```sql CREATE DICTIONARY votes_dict @@ -137,9 +136,9 @@ LAYOUT(HASHED()) 0 rows in set. Elapsed: 36.063 sec. ``` -> 在自管理的开源部署中,需要在所有节点上执行上述命令。在 ClickHouse Cloud 中,字典会自动复制到所有节点。上述命令是在一台具有 64GB 内存的 ClickHouse Cloud 节点上执行的,加载耗时 36 秒。 +> 在自管理 OSS 中,需要在所有节点上执行上述命令。在 ClickHouse Cloud 中,字典会自动复制到所有节点。上述操作是在一台具有 64GB 内存的 ClickHouse Cloud 节点上执行的,加载耗时 36 秒。 -要确认我们的字典占用的内存: +要确认我们的字典内存占用情况: ```sql SELECT formatReadableSize(bytes_allocated) AS size @@ -151,7 +150,8 @@ WHERE name = 'votes_dict' └──────────┘ ``` -现在,你可以通过一个简单的 `dictGet` 函数来获取特定 `PostId` 的赞成票和反对票数。下面我们检索帖子 `11227902` 的这些值: +现在,可以通过一个简单的 `dictGet` FUNCTION 来获取特定 `PostId` 的赞成票和反对票。下面我们检索 `PostId` 为 `11227902` 的帖子对应的这些值: + ```sql SELECT dictGet('votes_dict', ('UpVotes', 'DownVotes'), '11227902') AS votes @@ -177,16 +177,16 @@ WHERE (Id IN (PostIds)) AND (UpVotes > 10) AND (DownVotes > 10) ORDER BY Controversial_ratio ASC LIMIT 3 -返回 3 行。耗时:0.551 秒。处理了 1.1964 亿行,3.29 GB(2.1696 亿行/秒,5.97 GB/秒)。 +返回 3 行。耗时:0.551 秒。处理了 1.1964 亿行,3.29 GB(每秒 2.1696 亿行,5.97 GB/秒)。 峰值内存使用量:552.26 MiB。 ``` -这个查询不仅更加简单,而且执行速度也快了两倍多!还可以进一步优化,只将赞成票和反对票合计超过 10 的帖子加载到字典中,并只存储预先计算好的“争议度”数值。 +这个查询不仅简单得多,而且速度也提升了两倍多!还可以通过只将赞成票和反对票之和超过 10 的帖子加载到字典中,并仅存储预先计算好的争议度值来进一步优化。 -## 查询时富化 +## 查询时富化 {#query-time-enrichment} -可以在查询时使用字典来查找值。这些值可以直接在结果中返回,或用于聚合操作。假设我们创建一个字典,用于将用户 ID 映射到其位置: +字典可以用于在查询时查找值。这些值可以在查询结果中返回,或用于聚合运算。假设我们创建一个字典,将用户 ID 映射到其所在地: ```sql CREATE DICTIONARY users_dict @@ -200,7 +200,7 @@ LIFETIME(MIN 600 MAX 900) LAYOUT(HASHED()) ``` -我们可以使用此字典来丰富查询结果: +我们可以使用该字典来丰富 POST 请求的结果: ```sql SELECT @@ -213,18 +213,18 @@ LIMIT 5 FORMAT PrettyCompactMonoBlock ┌───────Id─┬─Title─────────────────────────────────────────────────────────┬─Location──────────────┐ -│ 52296928 │ ClickHouse 中两个字符串的比较 │ Spain │ -│ 52345137 │ 如何使用文件将数据从 MySQL 迁移到 ClickHouse? │ 中国江苏省Nanjing Shi │ -│ 61452077 │ 如何在 ClickHouse 中更改 PARTITION │ Guangzhou, 广东省中国 │ -│ 55608325 │ ClickHouse 在不使用 max() 的情况下选择表中最后一条记录 │ Moscow, Russia │ -│ 55758594 │ ClickHouse 创建临时表 │ Perm', Russia │ +│ 52296928 │ Comparison between two Strings in ClickHouse │ Spain │ +│ 52345137 │ How to use a file to migrate data from mysql to a clickhouse? │ 中国江苏省Nanjing Shi │ +│ 61452077 │ How to change PARTITION in clickhouse │ Guangzhou, 广东省中国 │ +│ 55608325 │ Clickhouse select last record without max() on all table │ Moscow, Russia │ +│ 55758594 │ ClickHouse create temporary table │ Perm', Russia │ └──────────┴───────────────────────────────────────────────────────────────┴───────────────────────┘ -返回 5 行。耗时:0.033 秒。处理了 425 万行,82.84 MB(每秒 1.3062 亿行,2.55 GB/秒)。 -峰值内存使用量:249.32 MiB。 +返回 5 行。用时:0.033 秒。处理了 425 万行,82.84 MB(每秒 1.3062 亿行,2.55 GB/秒)。 +内存峰值:249.32 MiB。 ``` -与上面的 JOIN 示例类似,我们可以使用同一个字典高效地确定大多数帖子来自哪里: +与上面的 join 示例类似,我们可以使用同一个字典,高效判断大部分帖子是从哪里发出的: ```sql SELECT @@ -249,13 +249,13 @@ LIMIT 5 ``` -## 索引时富化 +## 索引时富化 {#index-time-enrichment} -在上面的示例中,我们在查询时使用了字典来去掉一次 JOIN。字典也可以在插入时用于对行进行富化。通常当富化值不会变化,并且存在于可用于填充字典的外部数据源中时,这种方式是合适的。在这种情况下,在插入时对行进行富化可以避免在查询时再去查找字典。 +在上面的示例中,我们在查询时使用字典来避免一次 join 操作。字典也可以用于在插入时对行进行富化。如果富化值不会变化,并且存在于可用于填充字典的外部数据源中,这通常是一个合适的做法。在这种情况下,在插入时对行进行富化可以避免在查询时对字典进行查找。 -假设 Stack Overflow 中用户的 `Location` 从不改变(实际上是会变的)——更具体地说是 `users` 表的 `Location` 列。假设我们希望在 `posts` 表上按位置进行分析查询。`posts` 表中包含一个 `UserId`。 +假设 Stack Overflow 中某个用户的 `Location` 永远不变(实际上会变)——具体来说,是 `users` 表中的 `Location` 列。假设我们希望对 `posts` 表按地点进行分析查询。该表包含一个 `UserId`。 -字典提供了一个从用户 id 到位置的映射,并以 `users` 表为后端: +字典提供了从用户 ID 到位置信息的映射,并以 `users` 表为数据源: ```sql CREATE DICTIONARY users_dict @@ -269,9 +269,9 @@ LIFETIME(MIN 600 MAX 900) LAYOUT(HASHED()) ``` -> 我们忽略 `Id < 0` 的用户,以便可以使用 `Hashed` 字典类型。`Id < 0` 的用户是系统用户。 +> 我们排除 `Id < 0` 的用户,这样就可以使用 `Hashed` 字典类型。`Id < 0` 的用户是系统用户。 -要在向 `posts` 表插入数据时利用这个字典,我们需要修改该表的 schema: +为了在向 posts 表插入数据时利用这个字典,我们需要修改表结构(schema): ```sql CREATE TABLE posts_with_location @@ -285,11 +285,11 @@ ENGINE = MergeTree ORDER BY (PostTypeId, toDate(CreationDate), CommentCount) ``` -在上面的示例中,`Location` 被声明为一个 `MATERIALIZED` 列。这意味着该值可以作为 `INSERT` 查询的一部分提供,并且始终会被计算。 +在上面的示例中,`Location` 被声明为 `MATERIALIZED` 列。这意味着其值可以作为 `INSERT` 查询的一部分提供,并且始终会被计算。 -> ClickHouse 也支持 [`DEFAULT` 列](/sql-reference/statements/create/table#default_values)(在这种情况下,值既可以被显式插入,也可以在未提供时自动计算)。 +> ClickHouse 还支持 [`DEFAULT` columns](/sql-reference/statements/create/table#default_values)(在这种情况下,如果未提供值,则可以插入该值或由系统计算)。 -为了向该表填充数据,我们可以像往常一样从 S3 使用 `INSERT INTO SELECT`: +为了填充该表,我们可以像往常一样使用来自 S3 的 `INSERT INTO SELECT`: ```sql INSERT INTO posts_with_location SELECT Id, PostTypeId::UInt8, AcceptedAnswerId, CreationDate, Score, ViewCount, Body, OwnerUserId, OwnerDisplayName, LastEditorUserId, LastEditorDisplayName, LastEditDate, LastActivityDate, Title, Tags, AnswerCount, CommentCount, FavoriteCount, ContentLicense, ParentId, CommunityOwnedDate, ClosedDate FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/*.parquet') @@ -297,7 +297,7 @@ INSERT INTO posts_with_location SELECT Id, PostTypeId::UInt8, AcceptedAnswerId, 返回 0 行。耗时:36.830 秒。处理了 2.3898 亿行,2.64 GB(649 万行/秒,71.79 MB/秒) ``` -现在我们可以找出大多数帖子发布地点的名称: +现在我们可以获取大多数帖子发布地的名称: ```sql SELECT Location, count() AS c @@ -314,29 +314,29 @@ LIMIT 4 │ London, United Kingdom │ 538738 │ └────────────────────────┴────────┘ -4 行结果。耗时:0.142 秒。处理了 59.82 百万行,1.08 GB(420.73 百万行/秒,7.60 GB/秒)。 -峰值内存使用:666.82 MiB。 +返回 4 行。用时:0.142 秒。已处理 5982 万行,1.08 GB(420.73 百万行/秒,7.60 GB/秒)。 +内存峰值:666.82 MiB。 ``` -## 字典进阶主题 {#advanced-dictionary-topics} +## 字典高级主题 {#advanced-dictionary-topics} ### 选择字典 `LAYOUT` {#choosing-the-dictionary-layout} -`LAYOUT` 子句控制字典的内部数据结构。可用的布局类型有多种,其说明文档见[此处](/sql-reference/dictionaries#ways-to-store-dictionaries-in-memory)。关于如何选择合适布局的一些建议可以在[这里](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout)找到。 +`LAYOUT` 子句控制字典的内部数据结构。有多种可用选项,其文档见[此处](/sql-reference/dictionaries#ways-to-store-dictionaries-in-memory)。关于如何选择合适布局的一些建议见[这里](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout)。 ### 刷新字典 {#refreshing-dictionaries} -我们为该字典指定的 `LIFETIME` 为 `MIN 600 MAX 900`。LIFETIME 表示字典的更新间隔,上述配置会使字典在 600 到 900 秒之间的随机时间点进行周期性重新加载。这个随机间隔是必要的,用于在大量服务器更新时分散对字典源的负载。在更新期间,旧版本的字典仍然可以被查询,只有初次加载会阻塞查询。请注意,设置 `(LIFETIME(0))` 将禁止字典更新。 +我们为字典指定了 `LIFETIME MIN 600 MAX 900`。`LIFETIME` 用于控制字典的更新间隔,上述取值会使字典在 600 到 900 秒之间的随机时间间隔内周期性地重新加载。这个随机间隔是必要的,以便在大量服务器进行更新时分散对字典数据源的负载。在更新过程中,旧版本的字典仍然可以被查询,只有初始加载时才会阻塞查询。注意,将 `LIFETIME(0)` 进行设置会禁止字典更新。 可以使用 `SYSTEM RELOAD DICTIONARY` 命令强制重新加载字典。 -对于 ClickHouse 和 Postgres 等数据库源,可以配置查询,让字典仅在数据确实发生变化时才更新(由该查询的响应决定),而不是按固定周期更新。更多细节请参见[此处](/sql-reference/dictionaries#refreshing-dictionary-data-using-lifetime)。 +对于 ClickHouse 和 Postgres 等数据库数据源,你可以设置一个查询,仅在字典数据确实发生变化时才更新字典(由该查询的响应来决定),而不是按固定周期更新。更多详细信息请参见[此处](/sql-reference/dictionaries#refreshing-dictionary-data-using-lifetime)。 ### 其他字典类型 {#other-dictionary-types} -ClickHouse 还支持[层次字典](/sql-reference/dictionaries#hierarchical-dictionaries)、[多边形字典](/sql-reference/dictionaries#polygon-dictionaries)和[正则表达式字典](/sql-reference/dictionaries#regexp-tree-dictionary)。 +ClickHouse 还支持[层次结构字典](/sql-reference/dictionaries#hierarchical-dictionaries)、[多边形字典](/sql-reference/dictionaries#polygon-dictionaries)和[正则表达式字典](/sql-reference/dictionaries#regexp-tree-dictionary)。 ### 延伸阅读 {#more-reading} - [使用字典加速查询](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse) -- [字典的高级配置](/sql-reference/dictionaries) +- [字典的高级配置](/sql-reference/dictionaries) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md index 094aee6160d..9ae853009db 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/atomic.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Atomic +# Atomic {#atomic} `Atomic` 引擎支持非阻塞的 [`DROP TABLE`](#drop-detach-table) 和 [`RENAME TABLE`](#rename-table) 查询,以及原子性的 [`EXCHANGE TABLES`](#exchange-tables) 查询。在开源 ClickHouse 中,默认使用 `Atomic` 数据库引擎。 @@ -19,16 +19,16 @@ doc_type: 'reference' -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE test [ENGINE = Atomic] [SETTINGS disk=...]; ``` -## 具体说明和建议 +## 具体说明和建议 {#specifics-and-recommendations} -### 表 UUID +### 表 UUID {#table-uuid} `Atomic` 数据库中的每个表都有一个持久的 [UUID](../../sql-reference/data-types/uuid.md),其数据存储在以下目录中: @@ -50,16 +50,16 @@ CREATE TABLE name UUID '28f1c61c-2970-457a-bffe-454156ddcfef' (n UInt64) ENGINE 您可以使用 [show_table_uuid_in_table_create_query_if_not_nil](../../operations/settings/settings.md#show_table_uuid_in_table_create_query_if_not_nil) 设置,在 `SHOW CREATE` 查询中显示 UUID。 ::: -### RENAME TABLE +### RENAME TABLE {#rename-table} [`RENAME`](../../sql-reference/statements/rename.md) 查询不会修改 UUID,也不会移动表数据。此类查询会立即执行,并且不会等待正在使用该表的其他查询完成。 -### DROP/DETACH TABLE +### DROP/DETACH TABLE {#drop-detach-table} 使用 `DROP TABLE` 时,不会立即删除任何数据。`Atomic` 引擎只是通过将表的元数据移动到 `/clickhouse_path/metadata_dropped/` 并通知后台线程,将该表标记为已删除。最终删除表数据前的延迟由 [`database_atomic_delay_before_drop_table_sec`](../../operations/server-configuration-parameters/settings.md#database_atomic_delay_before_drop_table_sec) 设置指定。 您可以使用 `SYNC` 修饰符指定同步模式,可以通过 [`database_atomic_wait_for_drop_and_detach_synchronously`](../../operations/settings/settings.md#database_atomic_wait_for_drop_and_detach_synchronously) 设置来实现。在这种情况下,`DROP` 会等待正在运行且使用该表的 `SELECT`、`INSERT` 以及其他查询完成。当表不再被使用时,它将被删除。 -### EXCHANGE TABLES/DICTIONARIES +### EXCHANGE TABLES/DICTIONARIES {#exchange-tables} [`EXCHANGE`](../../sql-reference/statements/exchange.md) 查询会以原子方式交换表或字典。例如,您可以用它替代如下非原子操作: @@ -73,11 +73,11 @@ RENAME TABLE new_table TO tmp, old_table TO new_table, tmp TO old_table; EXCHANGE TABLES new_table AND old_table; ``` -### 原子数据库中的 ReplicatedMergeTree +### 原子数据库中的 ReplicatedMergeTree {#replicatedmergetree-in-atomic-database} 对于 [`ReplicatedMergeTree`](/engines/table-engines/mergetree-family/replication) 表,建议不要在引擎参数中显式指定 ZooKeeper 路径和副本名称。在这种情况下,将使用配置参数 [`default_replica_path`](../../operations/server-configuration-parameters/settings.md#default_replica_path) 和 [`default_replica_name`](../../operations/server-configuration-parameters/settings.md#default_replica_name)。如果需要显式指定引擎参数,建议使用 `{uuid}` 宏。这样可以确保为 ZooKeeper 中的每个表自动生成唯一路径。 -### 元数据磁盘 +### 元数据磁盘 {#metadata-disk} 当在 `SETTINGS` 中指定了 `disk` 时,将使用该磁盘来存储表的元数据文件。 例如: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md index 046bbd77c8c..6a885cf3b7b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/backup.md @@ -7,17 +7,13 @@ title: '备份' doc_type: '参考' --- - - -# 备份 +# 备份 {#backup} 数据库备份允许从[备份](../../operations/backup)中即时附加表或数据库,并以只读模式访问。 数据库备份同时支持增量和非增量备份。 - - -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE backup_database @@ -38,8 +34,7 @@ ENGINE = Backup('database_name_inside_backup', Disk('disk_name', 'backup_name')) * `database_name_inside_backup` — 备份中数据库的名称。 * `backup_destination` — 备份存储位置。 - -## 使用示例 +## 使用示例 {#usage-example} 以 `Disk` 作为备份目标为例。首先在 `storage.xml` 中配置备份磁盘: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md index 2b60244dd3d..6c9d8e6c8f4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' -# DataLakeCatalog +# DataLakeCatalog {#datalakecatalog} `DataLakeCatalog` 数据库引擎使您能够将 ClickHouse 连接到外部数据目录,并在无需复制数据的情况下查询开放表格式数据。 这使 ClickHouse 成为一个功能强大的查询引擎,能够与您现有的数据湖基础设施无缝协同工作。 @@ -26,7 +26,7 @@ doc_type: 'reference' -## 创建数据库 +## 创建数据库 {#creating-a-database} 要使用 `DataLakeCatalog` 引擎,需要启用下列相关设置: @@ -64,7 +64,7 @@ catalog_type, | `region` | 服务所在的 AWS 区域(例如 `us-east-1`) | -## 示例 +## 示例 {#examples} 请参阅以下部分,了解如何使用 `DataLakeCatalog` 引擎的示例: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/index.md index 633141dd6ab..8ca86a91738 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/index.md @@ -8,34 +8,32 @@ title: '数据库引擎' doc_type: 'landing-page' --- +# 数据库引擎 {#database-engines} - -# 数据库引擎 - -数据库引擎用于操作表。默认情况下,ClickHouse 使用 [Atomic](../../engines/database-engines/atomic.md) 数据库引擎,它提供可配置的[表引擎](../../engines/table-engines/index.md)以及一种 [SQL 方言](../../sql-reference/syntax.md)。 +数据库引擎用于操作表。默认情况下,ClickHouse 使用 [Atomic](../../engines/database-engines/atomic.md) 数据库引擎,它提供可配置的 [表引擎](../../engines/table-engines/index.md) 和 [SQL 方言](../../sql-reference/syntax.md)。 下面是可用数据库引擎的完整列表。请通过以下链接了解更多详细信息: -{/* 本页面的目录由以下脚本自动生成: +{/* 本页的目录由以下脚本自动生成: https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 基于 YAML front matter 字段:slug、description、title。 + 根据 YAML front matter 中的 slug、description、title 字段生成。 如果发现错误,请直接编辑页面本身的 YAML front matter。 */ } {/*AUTOGENERATED_START*/ } -| 页面 | 描述 | -| --------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | -| [Shared](/engines/database-engines/shared) | 介绍 `Shared` 数据库引擎的页面,该引擎在 ClickHouse Cloud 中可用。 | -| [Atomic](/engines/database-engines/atomic) | `Atomic` 引擎支持非阻塞的 `DROP TABLE` 和 `RENAME TABLE` 查询,以及原子性的 `EXCHANGE TABLES` 查询。默认使用 `Atomic` 数据库引擎。 | -| [Lazy](/engines/database-engines/lazy) | 在最后一次访问后的 `expiration_time_in_seconds` 秒内,仅在内存(RAM)中保留这些表。只能用于 Log 类型表。 | -| [Replicated](/engines/database-engines/replicated) | 该引擎基于 Atomic 引擎构建。它通过将 DDL 日志写入 ZooKeeper,并在给定数据库的所有副本上执行,从而支持元数据复制。 | -| [PostgreSQL](/engines/database-engines/postgresql) | 允许连接到远程 PostgreSQL 服务器上的数据库。 | -| [MySQL](/engines/database-engines/mysql) | 允许连接到远程 MySQL 服务器上的数据库,并执行 `INSERT` 和 `SELECT` 查询,以在 ClickHouse 和 MySQL 之间交换数据。 | -| [SQLite](/engines/database-engines/sqlite) | 允许连接到 SQLite 数据库,并执行 `INSERT` 和 `SELECT` 查询,以在 ClickHouse 和 SQLite 之间交换数据。 | -| [MaterializedPostgreSQL](/engines/database-engines/materialized-postgresql) | 基于 PostgreSQL 数据库中的表创建一个 ClickHouse 数据库。 | -| [Backup](/engines/database-engines/backup) | 允许以只读模式从备份中立即挂载表或数据库。 | -| [DataLakeCatalog](/engines/database-engines/datalakecatalog) | DataLakeCatalog 数据库引擎使您能够将 ClickHouse 连接到外部数据目录,并查询开放表格式的数据。 | - -{/*AUTOGENERATED_END*/ } +| Page | Description | +| --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | +| [Atomic](/engines/database-engines/atomic) | `Atomic` 引擎支持非阻塞的 `DROP TABLE` 和 `RENAME TABLE` 查询,以及原子的 `EXCHANGE TABLES` 查询。`Atomic` 是默认的数据库引擎。 | +| [Shared](/engines/database-engines/shared) | 描述 `Shared` 数据库引擎的页面,该引擎可在 ClickHouse Cloud 中使用。 | +| [Lazy](/engines/database-engines/lazy) | 在最后一次访问之后,仅在 RAM 中保留表 `expiration_time_in_seconds` 秒。只能与 Log 类型的表一起使用。 | +| [Replicated](/engines/database-engines/replicated) | 该引擎基于 Atomic 引擎构建。它通过将 DDL 日志写入 ZooKeeper,并在给定数据库的所有副本上执行,从而实现元数据复制。 | +| [PostgreSQL](/engines/database-engines/postgresql) | 用于连接远程 PostgreSQL 服务器上的数据库。 | +| [MySQL](/engines/database-engines/mysql) | 用于连接远程 MySQL 服务器上的数据库,并执行 `INSERT` 和 `SELECT` 查询,以在 ClickHouse 和 MySQL 之间交换数据。 | +| [SQLite](/engines/database-engines/sqlite) | 用于连接 SQLite 数据库,并执行 `INSERT` 和 `SELECT` 查询,以在 ClickHouse 和 SQLite 之间交换数据。 | +| [Backup](/engines/database-engines/backup) | 用于以只读模式即时挂载来自备份的表/数据库。 | +| [MaterializedPostgreSQL](/engines/database-engines/materialized-postgresql) | 使用 PostgreSQL 数据库中的表创建一个 ClickHouse 数据库。 | +| [DataLakeCatalog](/engines/database-engines/datalakecatalog) | DataLakeCatalog 数据库引擎用于将 ClickHouse 连接到外部数据目录,并查询开放表格式数据。 | + +{/*AUTOGENERATED_START*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md index bede945ee4c..10e7f441684 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/lazy.md @@ -7,17 +7,13 @@ title: 'Lazy' doc_type: 'reference' --- +# Lazy {#lazy} +只会在上次访问后的 `expiration_time_in_seconds` 秒内将表保留在 RAM 中。只能用于 *Log 表。 -# Lazy +它针对存储大量小型 *Log 表进行了优化,这些表的访问之间存在较长的时间间隔。 -只会在上次访问后的 `expiration_time_in_seconds` 秒内将表保留在 RAM 中。只能用于 \*Log 表。 - -它针对存储大量小型 \*Log 表进行了优化,这些表的访问之间存在较长的时间间隔。 - - - -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE testlazy diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md index f2892726e69..80764db4cd7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MaterializedPostgreSQL +# MaterializedPostgreSQL {#materializedpostgresql} @@ -35,7 +35,7 @@ SET allow_experimental_database_materialized_postgresql=1 ::: -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -50,7 +50,7 @@ ENGINE = MaterializedPostgreSQL('host:port', 'database', 'user', 'password') [SE * `password` — 用户密码。 -## 使用示例 +## 使用示例 {#example-of-use} ```sql CREATE DATABASE postgres_db @@ -66,7 +66,7 @@ SELECT * FROM postgresql_db.postgres_table; ``` -## 动态向复制中添加新表 +## 动态向复制中添加新表 {#dynamically-adding-table-to-replication} 创建 `MaterializedPostgreSQL` 数据库后,它不会自动检测对应 PostgreSQL 数据库中的新表。可以手动添加这类表: @@ -79,7 +79,7 @@ ATTACH TABLE postgres_database.new_table; ::: -## 动态移除复制中的表 +## 动态移除复制中的表 {#dynamically-removing-table-from-replication} 可以将特定的表从复制中移除: @@ -88,7 +88,7 @@ DETACH TABLE postgres_database.table_to_remove PERMANENTLY; ``` -## PostgreSQL 模式 +## PostgreSQL 模式 {#schema} 从 21.12 版本开始,PostgreSQL 的[模式(schema)](https://www.postgresql.org/docs/9.1/ddl-schemas.html)可以通过 3 种方式进行配置。 @@ -136,7 +136,7 @@ SELECT * FROM database1.`schema2.table2`; 警告:在这种情况下,表名中不允许包含点号。 -## 要求 +## 要求 {#requirements} 1. 在 PostgreSQL 配置文件中,必须将 [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) 参数设置为 `logical`,并且将 `max_replication_slots` 参数设置为至少 `2`。 @@ -172,9 +172,9 @@ WHERE oid = 'postgres_table'::regclass; ::: -## 设置 +## 设置 {#settings} -### `materialized_postgresql_tables_list` +### `materialized_postgresql_tables_list` {#materialized-postgresql-tables-list} 设置一个以逗号分隔的 PostgreSQL 数据库表列表,这些表将通过 [MaterializedPostgreSQL](../../engines/database-engines/materialized-postgresql.md) 数据库引擎进行复制。 @@ -186,15 +186,15 @@ materialized_postgresql_tables_list = 'table1(co1, col2),table2,table3(co3, col5 默认值:空列表 —— 表示将复制整个 PostgreSQL 数据库。 -### `materialized_postgresql_schema` +### `materialized_postgresql_schema` {#materialized-postgresql-schema} 默认值:空字符串。(使用默认 schema) -### `materialized_postgresql_schema_list` +### `materialized_postgresql_schema_list` {#materialized-postgresql-schema-list} 默认值:空列表。(使用默认 schema) -### `materialized_postgresql_max_block_size` +### `materialized_postgresql_max_block_size` {#materialized-postgresql-max-block-size} 设置在将数据刷新到 PostgreSQL 数据库表之前,先在内存中累积的行数。 @@ -204,11 +204,11 @@ materialized_postgresql_tables_list = 'table1(co1, col2),table2,table3(co3, col5 默认值:`65536`。 -### `materialized_postgresql_replication_slot` +### `materialized_postgresql_replication_slot` {#materialized-postgresql-replication-slot} 由用户创建的 replication slot。必须与 `materialized_postgresql_snapshot` 一起使用。 -### `materialized_postgresql_snapshot` +### `materialized_postgresql_snapshot` {#materialized-postgresql-snapshot} 用于标识快照的文本字符串,将基于该快照执行 [PostgreSQL 表的初始导出](../../engines/database-engines/materialized-postgresql.md)。必须与 `materialized_postgresql_replication_slot` 一起使用。 @@ -226,7 +226,7 @@ SELECT * FROM database1.table1; ALTER DATABASE postgres_database MODIFY SETTING materialized_postgresql_max_block_size = <新大小>; ``` -### `materialized_postgresql_use_unique_replication_consumer_identifier` +### `materialized_postgresql_use_unique_replication_consumer_identifier` {#materialized_postgresql_use_unique_replication_consumer_identifier} 使用唯一的复制消费者标识符进行复制。默认值:`0`。 如果设置为 `1`,则允许创建多个指向同一 `PostgreSQL` 表的 `MaterializedPostgreSQL` 表。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md index ce1316cb981..4cec39a282f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/mysql.md @@ -10,8 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# MySQL 数据库引擎 +# MySQL 数据库引擎 {#mysql-database-engine} @@ -25,9 +24,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - `CREATE TABLE` - `ALTER` - - -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE [IF NOT EXISTS] db_name [ON CLUSTER cluster] @@ -41,7 +38,6 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') * `user` — MySQL 用户。 * `password` — 用户密码。 - ## 数据类型支持 {#data_types-support} | MySQL | ClickHouse | @@ -64,9 +60,7 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') 支持 [Nullable](../../sql-reference/data-types/nullable.md)。 - - -## 全局变量支持 +## 全局变量支持 {#global-variables-support} 为提高兼容性,可以使用 MySQL 风格来引用全局变量,即 `@@identifier`。 @@ -85,8 +79,7 @@ ENGINE = MySQL('host:port', ['database' | database], 'user', 'password') SELECT @@version; ``` - -## 使用示例 +## 使用示例 {#examples-of-use} 在 MySQL 中的表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md index 84bec7a06d7..03510b2731b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/postgresql.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# PostgreSQL +# PostgreSQL {#postgresql} 允许连接到远程 [PostgreSQL](https://www.postgresql.org) 服务器上的数据库。支持读写操作(`SELECT` 和 `INSERT` 查询),用于在 ClickHouse 和 PostgreSQL 之间交换数据。 @@ -19,7 +19,7 @@ doc_type: 'guide' -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE test_database @@ -56,7 +56,7 @@ ENGINE = PostgreSQL('host:port', 'database', 'user', 'password'[, `schema`, `use -## 使用示例 +## 使用示例 {#examples-of-use} 与 PostgreSQL 服务器进行数据交换的 ClickHouse 数据库: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md index 1b15c22a0ce..8bbb1b024d7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/replicated.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Replicated +# Replicated {#replicated} 该引擎基于 [Atomic](../../engines/database-engines/atomic.md) 引擎。它通过将 DDL 日志写入 ZooKeeper,并在给定数据库的所有副本上执行该日志,从而实现元数据复制。 @@ -17,7 +17,7 @@ doc_type: 'reference' -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE testdb [UUID '...'] ENGINE = Replicated('zoo_path', 'shard_name', 'replica_name') [SETTINGS ...] @@ -54,7 +54,7 @@ CREATE DATABASE testdb [UUID '...'] ENGINE = Replicated('zoo_path', 'shard_name' -## 使用示例 +## 使用示例 {#usage-example} 创建一个由三台主机组成的集群: @@ -153,7 +153,7 @@ node2 :) SELECT materialize(hostName()) AS host, groupArray(n) FROM r.d GROUP BY ``` -## 设置 +## 设置 {#settings} 支持以下设置: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md index 1c4f99a7156..b3b6ffa5c1d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/shared.md @@ -12,7 +12,7 @@ import CloudOnlyBadge from '@theme/badges/CloudOnlyBadge'; -# Shared 数据库引擎 +# Shared 数据库引擎 {#shared-database-engine} `Shared` 数据库引擎与 Shared Catalog 配合使用,用于管理其表使用无状态表引擎(例如 [`SharedMergeTree`](/cloud/reference/shared-merge-tree))的数据库。 这些表引擎不会将任何持久化状态写入磁盘,并且与动态计算环境兼容。 @@ -30,7 +30,7 @@ Cloud 中的 `Shared` 数据库引擎消除了对本地磁盘的依赖。 -## 语法 +## 语法 {#syntax} 对于最终用户而言,使用 Shared Catalog 和 Shared 数据库引擎无需任何额外配置。数据库的创建方式与以往完全相同: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md index dd89b2699b2..48ce642fb7b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/database-engines/sqlite.md @@ -10,13 +10,13 @@ doc_type: 'reference' -# SQLite +# SQLite {#sqlite} 用于连接 [SQLite](https://www.sqlite.org/index.html) 数据库,并执行 `INSERT` 和 `SELECT` 查询,以在 ClickHouse 与 SQLite 之间交换数据。 -## 创建数据库 +## 创建数据库 {#creating-a-database} ```sql CREATE DATABASE sqlite_database @@ -46,7 +46,7 @@ SQLite 不需要服务管理(例如启动脚本)或基于 `GRANT` 和密码 -## 使用示例 +## 使用示例 {#usage-example} ClickHouse 中连接到 SQLite 的数据库: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/index.md index e2970821e1e..944574609ca 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/index.md @@ -10,7 +10,7 @@ doc_type: 'reference' -# 表引擎 +# 表引擎 {#table-engines} 表引擎(表的类型)决定: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md index fd8517fa7c3..69a50e92a68 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ExternalDistributed.md @@ -7,15 +7,11 @@ title: 'ExternalDistributed 表引擎' doc_type: 'reference' --- - - -# ExternalDistributed 表引擎 +# ExternalDistributed 表引擎 {#externaldistributed-table-engine} `ExternalDistributed` 引擎允许对存储在远程服务器上 MySQL 或 PostgreSQL 实例中的数据执行 `SELECT` 查询。它接受 [MySQL](../../../engines/table-engines/integrations/mysql.md) 或 [PostgreSQL](../../../engines/table-engines/integrations/postgresql.md) 引擎作为参数,从而支持分片。 - - -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -42,8 +38,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `user` — 用户名。 * `password` — 用户密码。 - -## 实现细节 +## 实现细节 {#implementation-details} 支持多个副本,多个副本之间必须使用 `|` 分隔,多个分片之间必须使用 `,` 分隔。例如: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md index 3ee86b61dae..437133a9bad 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/arrowflight.md @@ -9,14 +9,14 @@ doc_type: 'reference' -# ArrowFlight 表引擎 +# ArrowFlight 表引擎 {#arrowflight-table-engine} ArrowFlight 表引擎使 ClickHouse 能够通过 [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) 协议查询远程数据集。 此集成允许 ClickHouse 高效地从支持 Flight 的外部服务器获取列式 Arrow 格式的数据。 -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name (name1 [type1], name2 [type2], ...) (仅当 Arrow Flight 服务器允许无认证访问时才可用)。 -## 使用示例 +## 使用示例 {#usage-example} 本示例演示如何创建一个从远程 Arrow Flight 服务器读取数据的表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md index 803385122f6..74f731a576c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azure-queue.md @@ -1,5 +1,5 @@ --- -description: '该引擎提供与 Azure Blob Storage 生态系统的集成,支持流式数据导入。' +description: '此引擎提供与 Azure Blob Storage 生态系统的集成,支持流式数据导入。' sidebar_label: 'AzureQueue' sidebar_position: 181 slug: /engines/table-engines/integrations/azure-queue @@ -7,13 +7,9 @@ title: 'AzureQueue 表引擎' doc_type: 'reference' --- +# AzureQueue 表引擎 {#azurequeue-table-engine} - -# AzureQueue 表引擎 - -此引擎提供与 [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) 生态系统的集成,支持流式数据导入。 - - +该引擎实现了与 [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) 生态系统的集成,支持以流式方式导入数据。 ## 创建表 {#creating-a-table} @@ -29,9 +25,9 @@ CREATE TABLE test (name String, value UInt32) **引擎参数** -`AzureQueue` 的参数与 `AzureBlobStorage` 表引擎支持的参数相同。参数说明请参见[此处](../../../engines/table-engines/integrations/azureBlobStorage.md)。 +`AzureQueue` 的参数与 `AzureBlobStorage` 表引擎支持的参数相同。请参阅[此处](../../../engines/table-engines/integrations/azureBlobStorage.md)中的参数部分。 -与 [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage) 表引擎类似,用户可以使用 Azurite 模拟器进行本地 Azure Storage 开发。更多详细信息请参见[此处](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage)。 +与 [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage) 表引擎类似,用户可以使用 Azurite 模拟器在本地进行 Azure Storage 开发。更多详细信息请参见[此处](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azurite?tabs=docker-hub%2Cblob-storage)。 **示例** @@ -45,24 +41,60 @@ ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1; SETTINGS mode = 'unordered' ``` +## Settings {#settings} + +支持的设置大多与 `S3Queue` 表引擎相同,只是没有 `s3queue_` 前缀。参见[完整的设置列表](../../../engines/table-engines/integrations/s3queue.md#settings)。 +要获取为该表配置的设置列表,请使用 `system.azure_queue_settings` 表。从 `24.10` 版本起可用。 + +下面是仅与 AzureQueue 兼容且不适用于 S3Queue 的设置。 + +### `after_processing_move_connection_string` {#after_processing_move_connection_string} -## 设置 {#settings} +当目标是另一个 Azure 容器时,用于将已成功处理的文件移动到该容器的 Azure Blob Storage 连接字符串。 -支持的设置集与 `S3Queue` 表引擎相同,但不带 `s3queue_` 前缀。请参阅[完整设置列表](../../../engines/table-engines/integrations/s3queue.md#settings)。 -要获取表的配置设置列表,请使用 `system.azure_queue_settings` 表。此功能从 `24.10` 版本开始提供。 +可能的取值: +* 字符串。 -## Description {#description} +默认值:空字符串。 -`SELECT` 对于流式导入并不特别有用(除了调试场景),因为每个文件只能导入一次。更实用的做法是使用[物化视图](../../../sql-reference/statements/create/view.md)创建实时处理流程。操作步骤如下: +### `after_processing_move_container` {#after_processing_move_container} -1. 使用该引擎创建一个表,用于从 S3 指定路径消费数据,并将其视为数据流。 -2. 创建一个具有所需结构的目标表。 -3. 创建一个物化视图,将引擎中的数据进行转换并写入之前创建的表中。 +当目标是另一个 Azure 容器时,用于指定成功处理后文件要移动到的容器名称。 -当 `MATERIALIZED VIEW` 关联到引擎后,它会在后台开始收集数据。 +可能的取值: -示例: +* 字符串。 + +默认值:空字符串。 + +示例: + +```sql +CREATE TABLE azure_queue_engine_table +( + `key` UInt64, + `data` String +) +ENGINE = AzureQueue('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', 'testcontainer', '*', 'CSV') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_move_connection_string = 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', + after_processing_move_container = 'dst-container'; +``` + +## 描述 {#description} + +`SELECT` 对于流式导入并不是特别有用(除非用于调试),因为每个文件只能被导入一次。更实际的做法是使用[物化视图](../../../sql-reference/statements/create/view.md)来创建实时管道。为此: + +1. 使用该引擎创建一个从 S3 中指定路径消费数据的表,并将其视为数据流。 +2. 创建一个具有所需结构的表。 +3. 创建一个物化视图,将数据从该引擎转换后写入先前创建的表中。 + +当 `MATERIALIZED VIEW` 与该引擎关联后,它会开始在后台收集数据。 + +示例: ```sql CREATE TABLE azure_queue_engine_table (key UInt64, data String) @@ -79,34 +111,32 @@ CREATE MATERIALIZED VIEW consumer TO stats SELECT * FROM stats ORDER BY key; ``` - ## 虚拟列 {#virtual-columns} -- `_path` — 文件的路径。 -- `_file` — 文件的名称。 +* `_path` — 文件路径。 +* `_file` — 文件名。 有关虚拟列的更多信息,请参阅[此处](../../../engines/table-engines/index.md#table_engines-virtual_columns)。 +## 自省 {#introspection} -## 内省 {#introspection} - -通过表设置 `enable_logging_to_queue_log=1` 为表启用日志记录。 +通过表设置 `enable_logging_to_queue_log=1` 为该表启用日志记录。 -内省功能与 [S3Queue 表引擎](/engines/table-engines/integrations/s3queue#introspection) 相同,但存在以下几个明显差异: +自省功能与 [S3Queue 表引擎](/engines/table-engines/integrations/s3queue#introspection) 相同,但有以下几个明显差异: -1. 对于服务器版本 >= 25.1,使用 `system.azure_queue` 查看队列的内存状态。对于旧版本,使用 `system.s3queue`(它也包含 `azure` 表的信息)。 -2. 通过 ClickHouse 主配置文件启用 `system.azure_queue_log`,例如: +1. 对于服务器版本 >= 25.1,使用 `system.azure_queue` 表示队列的内存状态。对于更早的版本,使用 `system.s3queue`(其中也会包含 `azure` 表的信息)。 +2. 在主 ClickHouse 配置中启用 `system.azure_queue_log`,例如: ```xml - - system - azure_queue_log
-
+ + system + azure_queue_log
+
``` -此持久化表包含与 `system.s3queue` 相同的信息,但针对已处理和失败的文件。 +这个持久化表与 `system.s3queue` 表包含相同的信息,但记录的是已处理和失败的文件。 -该表具有以下结构: +该表具有以下结构: ```sql @@ -123,7 +153,7 @@ CREATE TABLE system.azure_queue_log `status` Enum8('Processed' = 0, 'Failed' = 1) COMMENT '文件处理状态', `processing_start_time` Nullable(DateTime) COMMENT '文件处理开始时间', `processing_end_time` Nullable(DateTime) COMMENT '文件处理结束时间', - `exception` String COMMENT '发生异常时的异常消息' + `exception` String COMMENT '异常消息(如有发生)' ) ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) @@ -133,7 +163,7 @@ COMMENT '包含 S3Queue 引擎处理文件信息的日志条目。' ``` -示例: +示例: ```sql SELECT * @@ -141,7 +171,7 @@ FROM system.azure_queue_log LIMIT 1 FORMAT Vertical -Row 1: +第 1 行: ────── hostname: clickhouse event_date: 2024-12-16 @@ -156,6 +186,6 @@ processing_start_time: 2024-12-16 13:42:47 processing_end_time: 2024-12-16 13:42:47 exception: -1 row in set. Elapsed: 0.002 sec. +返回 1 行。用时:0.002 秒。 ``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md index 9c926ad8c47..921c6d5d753 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/azureBlobStorage.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# AzureBlobStorage 表引擎 +# AzureBlobStorage 表引擎 {#azureblobstorage-table-engine} 该引擎用于与 [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs) 生态系统集成。 -## 创建表 +## 创建表 {#create-table} ```sql CREATE TABLE azure_blob_storage_table (name String, value UInt32) @@ -24,7 +24,7 @@ CREATE TABLE azure_blob_storage_table (name String, value UInt32) [SETTINGS ...] ``` -### 引擎参数 +### 引擎参数 {#engine-parameters} * `endpoint` — 带有容器和前缀的 Azure Blob Storage 端点 URL。如所用的认证方式需要,还可以选择在其中包含 account_name。(`http://azurite1:{port}/[account_name]{container_name}/{data_prefix}`)或者也可以通过 `storage_account_url`、`account_name` 和 `container` 单独提供这些参数。要指定前缀时,应使用 `endpoint`。 * `endpoint_contains_account_name` - 此标志用于指定 `endpoint` 是否包含 `account_name`,因为只有某些认证方式才需要它。(默认值:true) @@ -70,7 +70,7 @@ SELECT * FROM test_table; -## 身份验证 +## 身份验证 {#authentication} 当前有 3 种身份验证方式: @@ -78,7 +78,7 @@ SELECT * FROM test_table; * `SAS Token` - 可以通过提供 `endpoint`、`connection_string` 或 `storage_account_url` 进行身份验证。通过 URL 中是否包含 `?` 来识别。示例参见 [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage#using-shared-access-signatures-sas-sas-tokens)。 * `Workload Identity` - 可以通过提供 `endpoint` 或 `storage_account_url` 进行身份验证。如果在配置中设置了 `use_workload_identity` 参数,则会使用 [workload identity](https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/identity/azure-identity#authenticate-azure-hosted-applications) 进行身份验证。 -### 数据缓存 +### 数据缓存 {#data-cache} `Azure` 表引擎支持在本地磁盘上进行数据缓存。 有关文件系统缓存配置选项及用法,请参见本[节](/operations/storing-data.md/#using-local-cache)。 @@ -107,13 +107,13 @@ SETTINGS filesystem_cache_name = 'cache_for_azure', enable_filesystem_cache = 1; 2. 复用 ClickHouse `storage_configuration` 部分中的缓存配置(以及相应的缓存存储),[见此处](/operations/storing-data.md/#using-local-cache) -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` —— 可选。在大多数情况下不需要分区键;即便需要,一般也不需要比按月更细的分区键。分区不会加速查询(与 ORDER BY 表达式相反)。切勿使用粒度过细的分区。不要按客户端标识符或名称对数据进行分区(相反,应将客户端标识符或名称作为 ORDER BY 表达式中的第一列)。 对于按月分区,使用 `toYYYYMM(date_column)` 表达式,其中 `date_column` 是类型为 [Date](/sql-reference/data-types/date.md) 的日期列。这里的分区名称采用 `"YYYYMM"` 格式。 -#### 分区策略 +#### 分区策略 {#partition-strategy} `WILDCARD`(默认):将文件路径中的 `{_partition_id}` 通配符替换为实际的分区键。不支持读取。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md index 6161fc4d8e3..905c5c82966 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/deltalake.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Delta Lake 表引擎 +# Delta Lake 表引擎 {#deltalake-table-engine} 此引擎与 Amazon S3 中现有的 [Delta Lake](https://github.com/delta-io/delta) 表进行只读集成。 -## 创建表 +## 创建表 {#create-table} 请注意,Delta Lake 表必须已存在于 S3 中,并且该命令不支持通过 DDL 参数创建新表。 @@ -55,7 +55,7 @@ CREATE TABLE deltalake ENGINE=DeltaLake('http://mars-doc-test.s3.amazonaws.com/c CREATE TABLE deltalake ENGINE=DeltaLake(deltalake_conf, filename = 'test_table') ``` -### 数据缓存 +### 数据缓存 {#data-cache} `Iceberg` 表引擎和表函数支持与 `S3`、`AzureBlobStorage`、`HDFS` 存储相同的数据缓存机制。请参阅[此处](../../../engines/table-engines/integrations/s3.md#data-cache)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md index 70404eb6bd6..ee75edbaaeb 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/embedded-rocksdb.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# EmbeddedRocksDB 表引擎 +# EmbeddedRocksDB 表引擎 {#embeddedrocksdb-table-engine} 该引擎用于将 ClickHouse 与 [RocksDB](http://rocksdb.org/) 集成。 - - -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,8 +56,7 @@ ENGINE = EmbeddedRocksDB PRIMARY KEY key ``` - -## 指标 +## 指标 {#metrics} 还有一个 `system.rocksdb` 表,用于公开 RocksDB 的统计信息: @@ -76,8 +72,7 @@ FROM system.rocksdb └───────────────────────────┴───────┘ ``` - -## 配置 +## 配置 {#configuration} 你也可以通过配置更改任何 [RocksDB 选项](https://github.com/facebook/rocksdb/wiki/Option-String-and-Option-Map): @@ -105,10 +100,9 @@ FROM system.rocksdb 默认情况下,“trivial approximate count” 优化是关闭的,这可能会影响 `count()` 查询的性能。要启用此优化,请将 `optimize_trivial_approximate_count_query` 设置为 `1`。此外,该设置还会影响 EmbeddedRocksDB 引擎的 `system.tables`,启用后可以看到 `total_rows` 和 `total_bytes` 的近似值。 +## 支持的操作 {#supported-operations} -## 支持的操作 - -### 插入 +### 插入 {#inserts} 当向 `EmbeddedRocksDB` 插入新行时,如果键已存在,则会更新对应的值;否则会创建一个新的键。 @@ -118,7 +112,7 @@ FROM system.rocksdb INSERT INTO test VALUES ('some key', 1, 'value', 3.2); ``` -### 删除 +### 删除 {#deletes} 可以使用 `DELETE` 查询或 `TRUNCATE` 语句删除行。 @@ -134,7 +128,7 @@ ALTER TABLE test DELETE WHERE key LIKE 'some%' AND v1 > 1; TRUNCATE TABLE test; ``` -### 更新 +### 更新 {#updates} 可以使用 `ALTER TABLE` 语句来更新值。主键不允许更新。 @@ -142,7 +136,7 @@ TRUNCATE TABLE test; ALTER TABLE test UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; ``` -### 联接 +### 联接 {#joins} 在 EmbeddedRocksDB 表上支持一种特殊的 `direct` 联接。 这种 `direct` 联接避免在内存中构建哈希表,而是直接从 EmbeddedRocksDB 访问数据。 @@ -159,9 +153,9 @@ SET join_algorithm = 'direct, hash' 当 `join_algorithm` 设置为 `direct, hash` 时,将在可行时优先使用 direct 联接,否则使用 hash 联接。 ::: -#### 示例 +#### 示例 {#example} -##### 创建并填充 EmbeddedRocksDB 表 +##### 创建并填充 EmbeddedRocksDB 表 {#create-and-populate-an-embeddedrocksdb-table} ```sql CREATE TABLE rdb @@ -183,7 +177,7 @@ INSERT INTO rdb FROM numbers_mt(10); ``` -##### 创建并填充一个表,以便与 `rdb` 表进行 JOIN +##### 创建并填充一个表,以便与 `rdb` 表进行 JOIN {#create-and-populate-a-table-to-join-with-table-rdb} ```sql CREATE TABLE t2 @@ -198,13 +192,13 @@ INSERT INTO t2 SELECT number AS k FROM numbers_mt(10) ``` -##### 将 join 算法设为 `direct` +##### 将 join 算法设为 `direct` {#set-the-join-algorithm-to-direct} ```sql SET join_algorithm = 'direct' ``` -##### INNER JOIN(内连接) +##### INNER JOIN(内连接) {#an-inner-join} ```sql SELECT * @@ -229,7 +223,7 @@ ORDER BY key ASC └─────┴─────────┴────────┴────────┘ ``` -### 关于 JOIN 的更多信息 +### 关于 JOIN 的更多信息 {#more-information-on-joins} * [`join_algorithm` 设置](/operations/settings/settings.md#join_algorithm) * [JOIN 子句](/sql-reference/statements/select/join.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md index 62515dbbc71..8b9c5bf2fbc 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hdfs.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# HDFS 表引擎 +# HDFS 表引擎 {#hdfs-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 用法 +## 用法 {#usage} ```sql ENGINE = HDFS(URI, format) @@ -35,7 +35,7 @@ ENGINE = HDFS(URI, format) [Formats](/sql-reference/formats#formats-overview) 部分。 * [PARTITION BY expr] -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — 可选。在大多数情况下不需要分区键,即使需要,一般也不需要比“按月”更细的分区键。分区并不会加速查询(与 ORDER BY 表达式不同)。切勿使用过于细粒度的分区。不要按客户标识符或名称对数据进行分区(相反,应将客户标识符或名称设为 ORDER BY 表达式中的第一列)。 @@ -69,7 +69,7 @@ SELECT * FROM hdfs_engine_table LIMIT 2 ``` -## 实现细节 +## 实现细节 {#implementation-details} * 读写操作可以并行进行。 * 不支持: @@ -137,7 +137,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ENGINE = HDFS('hdfs CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9000/big_dir/file{0..9}{0..9}{0..9}', 'CSV') ``` -## 配置 +## 配置 {#configuration} 与 GraphiteMergeTree 类似,HDFS 引擎支持通过 ClickHouse 配置文件进行扩展配置。可以使用两个配置项:全局级(`hdfs`)和用户级(`hdfs_*`)。系统会先应用全局配置,然后再应用用户级配置(如果存在)。 @@ -155,9 +155,9 @@ CREATE TABLE big_table (name String, value UInt32) ENGINE = HDFS('hdfs://hdfs1:9 ``` -### 配置选项 +### 配置选项 {#configuration-options} -#### libhdfs3 支持的选项 +#### libhdfs3 支持的选项 {#supported-by-libhdfs3} | **参数** | **默认值** | @@ -230,7 +230,7 @@ datanode 通信不会通过 SASL 进行安全保护(`HADOOP_SECURE_DN_USER` 如果指定了 `hadoop_kerberos_keytab`、`hadoop_kerberos_principal` 或 `hadoop_security_kerberos_ticket_cache_path`,则会使用 Kerberos 身份验证。在这种情况下,`hadoop_kerberos_keytab` 和 `hadoop_kerberos_principal` 是必需的。 -## HDFS Namenode HA 支持 +## HDFS Namenode HA 支持 {#namenode-ha} libhdfs3 支持 HDFS Namenode 高可用(HA)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md index 16f4c300237..8ae1ceb588b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hive.md @@ -9,22 +9,19 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# Hive 表引擎 {#hive-table-engine} -# Hive 表引擎 - - + Hive 引擎允许对 HDFS 中的 Hive 表执行 `SELECT` 查询。目前支持的输入格式如下: -- Text:仅支持除 `binary` 外的简单标量列类型 - -- ORC:支持除 `char` 外的简单标量列类型;复杂类型仅支持 `array` 等 - -- Parquet:支持所有简单标量列类型;复杂类型仅支持 `array` 等 +* Text:仅支持除 `binary` 外的简单标量列类型 +* ORC:支持除 `char` 外的简单标量列类型;复杂类型仅支持 `array` 等 +* Parquet:支持所有简单标量列类型;复杂类型仅支持 `array` 等 -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -52,10 +49,9 @@ PARTITION BY expr * `table` — 远程表名称。 +## 使用示例 {#usage-example} -## 使用示例 - -### 如何在 HDFS 文件系统中使用本地缓存 +### 如何在 HDFS 文件系统中使用本地缓存 {#how-to-use-local-cache-for-hdfs-filesystem} 我们强烈建议为远程文件系统启用本地缓存。基准测试结果表明,启用缓存后速度几乎提升 2 倍。 @@ -75,9 +71,9 @@ PARTITION BY expr * limit_size: 必需。本地缓存文件的最大大小(以字节为单位)。 * bytes_read_before_flush: 控制从远程文件系统下载文件时,在刷新到本地文件系统之前读取的字节数。默认值为 1MB。 -### 使用 ORC 输入格式查询 Hive 表 +### 使用 ORC 输入格式查询 Hive 表 {#query-hive-table-with-orc-input-format} -#### 在 Hive 中创建表 +#### 在 Hive 中创建表 {#create-table-in-hive} ```text hive > CREATE TABLE `test`.`test_orc`( @@ -125,8 +121,7 @@ OK Time taken: 0.295 seconds, Fetched: 1 row(s) ``` -#### 在 ClickHouse 中创建表 - +#### 在 ClickHouse 中创建表 {#create-table-in-clickhouse} ClickHouse 中的一个表,用于从上面创建的 Hive 表中读取数据: @@ -199,9 +194,9 @@ day: 2021-09-18 1 rows in set. Elapsed: 0.078 sec. ``` -### 使用 Parquet 输入格式查询 Hive 表 +### 使用 Parquet 输入格式查询 Hive 表 {#query-hive-table-with-parquet-input-format} -#### 在 Hive 中创建表 +#### 在 Hive 中创建表 {#create-table-in-hive-1} ```text hive > @@ -241,7 +236,6 @@ OK Time taken: 0.51 seconds ``` - hive > insert into test.test_parquet partition(day='2021-09-18') select 1, 2, 3, 4, 5, 6.11, 7.22, 8.333, current_timestamp(), current_date(), 'hello world', 'hello world', 'hello world', true, 'hello world', array(1, 2, 3), array('hello world', 'hello world'), array(float(1.1), float(1.2)), array(array(1, 2), array(3, 4)), array(array('a', 'b'), array('c', 'd')), array(array(float(1.11), float(2.22)), array(float(3.33), float(4.44))); OK 耗时:36.025 秒 @@ -329,7 +323,6 @@ day: 2021-09-18 #### 在 Hive 中创建表 - ```text hive > CREATE TABLE `test`.`test_text`( @@ -377,7 +370,7 @@ OK Time taken: 0.624 seconds, Fetched: 1 row(s) ``` -#### 在 ClickHouse 中创建表 +#### 在 ClickHouse 中创建表 {#create-table-in-hive-2} 在 ClickHouse 中创建一个表,用于从上述创建的 Hive 表中读取数据: @@ -416,7 +409,6 @@ SETTINGS input_format_skip_unknown_fields = 1, input_format_with_names_use_heade 查询 ID: 55b79d35-56de-45b9-8be6-57282fbf1f44 ``` - 第 1 行: ────── f_tinyint: 1 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md index db538335be8..83946c9813f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/hudi.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# Hudi 表引擎 +# Hudi 表引擎 {#hudi-table-engine} 该引擎提供与 Amazon S3 中现有 Apache [Hudi](https://hudi.apache.org/) 表的只读方式集成。 -## 创建表 +## 创建表 {#create-table} 请注意,S3 中必须已经存在该 Hudi 表,此命令不支持通过 DDL 参数创建新表。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md index 4be8b5b86c9..7fa51955ff2 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/iceberg.md @@ -23,7 +23,7 @@ Iceberg 表引擎是可用的,但可能存在一些限制。ClickHouse 最初 -## 创建表 +## 创建表 {#create-table} 请注意,Iceberg 表必须已经存在于存储中,此命令不接受用于创建新表的 DDL 参数。 @@ -42,14 +42,14 @@ CREATE TABLE iceberg_table_local ``` -## 引擎参数 +## 引擎参数 {#engine-arguments} 参数说明与引擎 `S3`、`AzureBlobStorage`、`HDFS` 和 `File` 中参数的说明相同。\ `format` 表示 Iceberg 表中数据文件的格式。 可以使用 [Named Collections](../../../operations/named-collections.md) 来指定引擎参数。 -### 示例 +### 示例 {#example} ```sql CREATE TABLE iceberg_table ENGINE=IcebergS3('http://test.s3.amazonaws.com/clickhouse-bucket/test_table', 'test', 'test') @@ -105,7 +105,7 @@ ClickHouse 支持 Iceberg 表的时间旅行功能,允许您在指定的时间 -## 处理包含已删除行的表 +## 处理包含已删除行的表 {#deleted-rows} 目前,仅支持带有 [position deletes](https://iceberg.apache.org/spec/#position-delete-files) 的 Iceberg 表。 @@ -114,7 +114,7 @@ ClickHouse 支持 Iceberg 表的时间旅行功能,允许您在指定的时间 * [Equality deletes](https://iceberg.apache.org/spec/#equality-delete-files)(等值删除) * [Deletion vectors](https://iceberg.apache.org/spec/#deletion-vectors)(在 v3 中引入的删除向量) -### 基本用法 +### 基本用法 {#basic-usage} ```sql SELECT * FROM example_table ORDER BY 1 @@ -128,7 +128,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 注意:你不能在同一个查询中同时指定 `iceberg_timestamp_ms` 和 `iceberg_snapshot_id` 参数。 -### 重要注意事项 +### 重要注意事项 {#important-considerations} * **快照** 通常在以下情况下创建: * 向表写入新数据时 @@ -136,11 +136,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * **模式变更通常不会创建快照** —— 这会在对经过模式演进的表使用 time travel(时间穿梭查询)时产生一些需要注意的重要行为。 -### 示例场景 +### 示例场景 {#example-scenarios} 所有场景都使用 Spark 编写,因为 ClickHouse(CH)目前尚不支持向 Iceberg 表写入数据。 -#### 场景 1:仅发生模式变更但没有新的快照 +#### 场景 1:仅发生模式变更但没有新的快照 {#scenario-1} 考虑以下一系列操作: @@ -200,7 +200,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * 在 ts1 和 ts2:只显示原来的两列 * 在 ts3:显示全部三列,且第一行的 price 为 NULL -#### 场景 2:历史表结构与当前表结构的差异 +#### 场景 2:历史表结构与当前表结构的差异 {#scenario-2} 在当前时刻执行的时间旅行查询,可能会显示与当前表不同的表结构: @@ -243,7 +243,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 这是因为 `ALTER TABLE` 不会创建新的快照;对于当前表,Spark 会从最新的元数据文件中读取 `schema_id` 的值,而不是从快照中读取。 -#### 场景 3:历史与当前表结构差异 +#### 场景 3:历史与当前表结构差异 {#scenario-3} 第二点是在进行时间旅行时,你无法获取表在尚未写入任何数据之前的状态: @@ -265,11 +265,11 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 在 ClickHouse 中,其行为与 Spark 保持一致。你可以在概念上将 Spark 的 Select 查询替换为 ClickHouse 的 Select 查询,它们的工作方式是相同的。 -## 元数据文件解析 +## 元数据文件解析 {#metadata-file-resolution} 在 ClickHouse 中使用 `Iceberg` 表引擎时,系统需要定位描述 Iceberg 表结构的正确 metadata.json 文件。下面是该解析过程的具体流程: -### 候选文件搜索 +### 候选文件搜索 {#candidate-search} 1. **直接路径指定**: @@ -286,7 +286,7 @@ SETTINGS iceberg_snapshot_id = 3547395809148285433 * 如果上述设置都未提供,则 `metadata` 目录中的所有 `.metadata.json` 文件都将作为候选。 -### 选择最新的文件 +### 选择最新的文件 {#most-recent-file} 根据上述规则确定候选文件后,系统会判断其中哪个是最新的文件: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md index 616ff87f55e..dde5a751dc6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/index.md @@ -7,48 +7,45 @@ title: '用于集成的表引擎' doc_type: 'reference' --- +# 用于集成的表引擎 {#table-engines-for-integrations} +ClickHouse 提供多种与外部系统集成的方式,其中包括表引擎。与所有其他表引擎一样,配置是通过 `CREATE TABLE` 或 `ALTER TABLE` 查询完成的。之后,对用户来说,已配置的集成就像一个普通表,但对其发起的查询会被转发到外部系统。这种透明查询能力是该方案优于其他集成方法(如需要在每次使用时采用自定义查询方式的字典或表函数)的关键优势之一。 -# 用于集成的表引擎 - -ClickHouse 提供多种与外部系统集成的方式,其中包括表引擎。与所有其他表引擎一样,其配置是通过 `CREATE TABLE` 或 `ALTER TABLE` 语句完成的。随后,从用户的角度来看,已配置的集成与普通表无异,但对其发起的查询会被转发到外部系统。这种透明的查询特性是该方案相较于其他集成方法(如字典或表函数,需要在每次使用时采用自定义查询方式)的关键优势之一。 - -{/* 本页的目录由以下脚本自动生成: +{/* 本页的目录表由以下脚本自动生成: https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 该脚本基于 YAML front matter 字段:slug、description、title。 + 它会根据 YAML front matter 中的 slug、description、title 字段生成。 - 如发现错误,请直接编辑相应页面的 YAML front matter。 + 如果你发现错误,请直接编辑各页面本身的 YML front matter。 */ } - {/*AUTOGENERATED_START*/ } -| 页面 | 描述 | -| ----------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | -| [AzureBlobStorage 表引擎](/engines/table-engines/integrations/azureBlobStorage) | 该引擎支持与 Azure Blob Storage 生态系统集成。 | -| [DeltaLake 表引擎](/engines/table-engines/integrations/deltalake) | 此引擎以只读方式集成位于 Amazon S3 上的现有 Delta Lake 表。 | -| [EmbeddedRocksDB 表引擎](/engines/table-engines/integrations/embedded-rocksdb) | 该引擎允许将 ClickHouse 与 RocksDB 集成 | -| [ExternalDistributed 表引擎](/engines/table-engines/integrations/ExternalDistributed) | `ExternalDistributed` 引擎允许对存储在远程 MySQL 或 PostgreSQL 服务器中的数据执行 `SELECT` 查询。它接收 MySQL 或 PostgreSQL 引擎作为参数,因此可以实现分片。 | -| [TimeSeries 表引擎](/engines/table-engines/special/time_series) | 用于存储时间序列的表引擎,即一组与时间戳和标签(或标记)关联的值。 | -| [HDFS 表引擎](/engines/table-engines/integrations/hdfs) | 该引擎支持通过 ClickHouse 管理 HDFS 上的数据,从而与 Apache Hadoop 生态系统集成。该引擎类似于 File 和 URL 引擎,但提供了 Hadoop 特有的功能。 | -| [Hive 表引擎](/engines/table-engines/integrations/hive) | Hive 引擎允许你对存储在 HDFS 上的 Hive 表执行 `SELECT` 查询。 | -| [Hudi 表引擎](/engines/table-engines/integrations/hudi) | 该引擎为位于 Amazon S3 中的现有 Apache Hudi 表提供只读集成。 | -| [Iceberg 表引擎](/engines/table-engines/integrations/iceberg) | 此引擎为现有的 Apache Iceberg 表提供只读集成,支持存储在 Amazon S3、Azure、HDFS 以及本地存储的表。 | -| [JDBC 表引擎](/engines/table-engines/integrations/jdbc) | 允许 ClickHouse 通过 JDBC 连接到外部数据库。 | -| [Kafka 表引擎](/engines/table-engines/integrations/kafka) | Kafka 表引擎可用于与 Apache Kafka 协同工作,并支持发布或订阅数据流、构建容错存储,以及在数据流可用时对其进行处理。 | -| [MaterializedPostgreSQL 表引擎](/engines/table-engines/integrations/materialized-postgresql) | 根据 PostgreSQL 表的初始数据转储创建一个 ClickHouse 表,并启动复制流程。 | -| [MongoDB 表引擎](/engines/table-engines/integrations/mongodb) | MongoDB 引擎是一种只读表引擎,可用于从远程集合中读取数据。 | -| [MySQL 表引擎](/engines/table-engines/integrations/mysql) | MySQL 表引擎文档 | -| [NATS 表引擎](/engines/table-engines/integrations/nats) | 此引擎可将 ClickHouse 与 NATS 集成,用于发布或订阅消息主题,并在有新消息时对其进行处理。 | -| [ODBC 表引擎](/engines/table-engines/integrations/odbc) | 使 ClickHouse 能够通过 ODBC 连接到外部数据库。 | -| [PostgreSQL 表引擎](/engines/table-engines/integrations/postgresql) | PostgreSQL 引擎允许对存储在远程 PostgreSQL 服务器上的数据进行 `SELECT` 和 `INSERT` 操作。 | -| [RabbitMQ 表引擎](/engines/table-engines/integrations/rabbitmq) | 该引擎支持将 ClickHouse 与 RabbitMQ 集成。 | -| [Redis 表引擎](/engines/table-engines/integrations/redis) | 该引擎允许将 ClickHouse 与 Redis 集成。 | -| [S3 表引擎](/engines/table-engines/integrations/s3) | 此引擎提供与 Amazon S3 生态系统的集成。类似于 HDFS 引擎,但提供 S3 特有的功能。 | -| [S3Queue 表引擎](/engines/table-engines/integrations/s3queue) | 该引擎提供与 Amazon S3 生态系统的集成,并支持流式导入。类似于 Kafka 和 RabbitMQ 引擎,但提供 S3 特有的功能。 | -| [AzureQueue 表引擎](/engines/table-engines/integrations/azure-queue) | 该引擎实现了与 Azure Blob Storage 生态系统的集成,从而支持流式数据导入。 | -| [YTsaurus 表引擎](/engines/table-engines/integrations/ytsaurus) | 一种表引擎,可从 YTsaurus 集群导入数据。 | -| [SQLite 表引擎](/engines/table-engines/integrations/sqlite) | 该引擎支持向 SQLite 导入数据、从 SQLite 导出数据,并支持在 ClickHouse 中直接查询 SQLite 表。 | -| [ArrowFlight 表引擎](/engines/table-engines/integrations/arrowflight) | 该引擎支持通过 Apache Arrow Flight 对远程数据集进行查询。 | +| 页面 | 说明 | +| ----------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | +| [AzureBlobStorage 表引擎](/engines/table-engines/integrations/azureBlobStorage) | 该引擎支持与 Azure Blob Storage 生态系统集成。 | +| [DeltaLake 表引擎](/engines/table-engines/integrations/deltalake) | 该引擎可与 Amazon S3 中已有的 Delta Lake 表进行只读集成。 | +| [EmbeddedRocksDB 表引擎](/engines/table-engines/integrations/embedded-rocksdb) | 该引擎允许将 ClickHouse 与 RocksDB 集成 | +| [ExternalDistributed 表引擎](/engines/table-engines/integrations/ExternalDistributed) | `ExternalDistributed` 引擎允许对存储在远程服务器上 MySQL 或 PostgreSQL 中的数据执行 `SELECT` 查询。它接受 MySQL 或 PostgreSQL 引擎作为参数,从而支持分片。 | +| [TimeSeries 表引擎](/engines/table-engines/special/time_series) | 一种用于存储时间序列的表引擎,即由与时间戳和标签(或标记)关联的一组数值组成的数据。 | +| [HDFS 表引擎](/engines/table-engines/integrations/hdfs) | 此引擎通过在 ClickHouse 中管理 HDFS 上的数据,与 Apache Hadoop 生态系统集成。该引擎类似于 File 和 URL 引擎,但提供了 Hadoop 特有的功能。 | +| [Hive 表引擎](/engines/table-engines/integrations/hive) | Hive 引擎允许你对存储在 HDFS 上的 Hive 表执行 `SELECT` 查询。 | +| [Hudi 表引擎](/engines/table-engines/integrations/hudi) | 此引擎为存储在 Amazon S3 中的现有 Apache Hudi 表提供只读方式的集成。 | +| [Iceberg 表引擎](/engines/table-engines/integrations/iceberg) | 该引擎以只读方式与现有的 Apache Iceberg 表集成,支持位于 Amazon S3、Azure、HDFS 以及本地存储的表。 | +| [JDBC 表引擎](/engines/table-engines/integrations/jdbc) | 使 ClickHouse 能够通过 JDBC 连接到外部数据库。 | +| [Kafka 表引擎](/engines/table-engines/integrations/kafka) | Kafka 表引擎可以与 Apache Kafka 配合使用,使你能够发布或订阅数据流、构建容错存储,并在数据流可用时对其进行处理。 | +| [MaterializedPostgreSQL 表引擎](/engines/table-engines/integrations/materialized-postgresql) | 创建一个基于 PostgreSQL 表初始数据导出的 ClickHouse 表,并启动复制过程。 | +| [MongoDB 表引擎](/engines/table-engines/integrations/mongodb) | MongoDB 引擎是一种只读的表引擎,允许从远程集合中读取数据。 | +| [MySQL 表引擎](/engines/table-engines/integrations/mysql) | MySQL 表引擎文档 | +| [NATS 表引擎](/engines/table-engines/integrations/nats) | 此引擎可将 ClickHouse 与 NATS 集成,用于发布或订阅消息主题,并在有新消息产生时进行处理。 | +| [ODBC 表引擎](/engines/table-engines/integrations/odbc) | 使 ClickHouse 能够通过 ODBC 连接到外部数据库。 | +| [PostgreSQL 表引擎](/engines/table-engines/integrations/postgresql) | PostgreSQL 引擎允许对位于远程 PostgreSQL 服务器上的数据执行 `SELECT` 和 `INSERT` 查询。 | +| [RabbitMQ 表引擎](/engines/table-engines/integrations/rabbitmq) | 该引擎支持将 ClickHouse 与 RabbitMQ 集成。 | +| [Redis 表引擎](/engines/table-engines/integrations/redis) | 该引擎允许将 ClickHouse 与 Redis 集成。 | +| [S3 表引擎](/engines/table-engines/integrations/s3) | 此引擎用于与 Amazon S3 生态系统集成。类似于 HDFS 引擎,但提供 S3 特有的功能。 | +| [AzureQueue 表引擎](/engines/table-engines/integrations/azure-queue) | 此引擎实现了与 Azure Blob Storage 生态系统的集成,支持数据流式导入。 | +| [S3Queue 表引擎](/engines/table-engines/integrations/s3queue) | 此引擎可与 Amazon S3 生态系统集成,并支持流式导入。类似于 Kafka 和 RabbitMQ 引擎,但提供 S3 特有的功能。 | +| [SQLite 表引擎](/engines/table-engines/integrations/sqlite) | 该引擎支持向 SQLite 导入数据并从 SQLite 导出数据,并支持在 ClickHouse 中直接查询 SQLite 表。 | +| [YTsaurus 表引擎](/engines/table-engines/integrations/ytsaurus) | 一种支持从 YTsaurus 集群导入数据的表引擎。 | +| [ArrowFlight 表引擎](/engines/table-engines/integrations/arrowflight) | 该引擎支持通过 Apache Arrow Flight 查询远程数据集。 | {/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md index 2cd604d8d9d..060465f47f1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/jdbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# JDBC 表引擎 +# JDBC 表引擎 {#jdbc-table-engine} @@ -27,7 +27,7 @@ ClickHouse 推荐使用 ClickHouse 内置的表函数,作为临时(即席) -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -51,7 +51,7 @@ ENGINE = JDBC(数据源名称, 外部数据库, 外部表) * 这些参数也可以通过 [命名集合](operations/named-collections.md) 传递。 -## 使用示例 +## 使用示例 {#usage-example} 使用 MySQL 的控制台客户端直接连接到服务器来创建一张表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md index 48a538a1114..9b39adedf9f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/kafka.md @@ -11,7 +11,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Kafka 表引擎 +# Kafka 表引擎 {#kafka-table-engine} :::tip 如果您在使用 ClickHouse Cloud,我们推荐改用 [ClickPipes](/integrations/clickpipes)。ClickPipes 原生支持私有网络连接,可分别扩展摄取层和集群资源,并为将 Kafka 流式数据摄取到 ClickHouse 提供完善的监控能力。 @@ -21,7 +21,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - 构建具备容错能力的存储。 - 在数据流到达时进行处理。 -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -135,7 +135,7 @@ Kafka 表引擎不支持包含[默认值](/sql-reference/statements/create/table ::: -## 描述 +## 描述 {#description} 已投递的消息会被自动跟踪,因此每个组中的每条消息只会被计数一次。如果你希望获取同一批数据两次,请创建一个使用不同 group 名称的表副本。 @@ -186,7 +186,7 @@ Group 十分灵活,并且会在集群中同步。例如,如果你在一个 如果你想通过 `ALTER` 语句更改目标表,建议先禁用该物化视图,以避免目标表与视图产出的数据之间出现不一致。 -## 配置 +## 配置 {#configuration} 与 GraphiteMergeTree 类似,Kafka 引擎支持通过 ClickHouse 配置文件进行扩展配置。可以使用两个配置键:全局配置(位于 `` 下)和主题级配置(位于 `` 下)。会先应用全局配置,然后再应用主题级配置(如果存在)。 @@ -233,7 +233,7 @@ Group 十分灵活,并且会在集群中同步。例如,如果你在一个 有关可用配置选项的列表,请参阅 [librdkafka 配置参考](https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md)。在 ClickHouse 配置中,应使用下划线(`_`)而不是点号。例如,`check.crcs=true` 将写作 `true`。 -### Kerberos 支持 +### Kerberos 支持 {#kafka-kerberos-support} 要与支持 Kerberos 的 Kafka 配合使用,请添加值为 `sasl_plaintext` 的 `security_protocol` 子元素。如果操作系统已经获取并缓存了 Kerberos 票据授予票(TGT,ticket-granting ticket),这就足够了。 ClickHouse 可以使用 keytab 文件维护 Kerberos 凭证。请考虑配置 `sasl_kerberos_service_name`、`sasl_kerberos_keytab` 和 `sasl_kerberos_principal` 子元素。 @@ -276,7 +276,7 @@ Kafka 引擎支持 ClickHouse 所支持的所有[格式](../../../interfaces/for - 对于行级格式,可以通过设置 `kafka_max_rows_per_message` 来控制单个 Kafka 消息中的行数。 - 对于块级格式,我们无法将一个块再拆分为更小的部分,但可以通过通用设置 [max_block_size](/operations/settings/settings#max_block_size) 来控制单个块中的行数。 -## 在 ClickHouse Keeper 中存储已提交 offset 的引擎 +## 在 ClickHouse Keeper 中存储已提交 offset 的引擎 {#engine-to-store-committed-offsets-in-clickhouse-keeper} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md index b99233a0d7b..b0411bd02ce 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/materialized-postgresql.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# MaterializedPostgreSQL 表引擎 +# MaterializedPostgreSQL 表引擎 {#materializedpostgresql-table-engine} @@ -35,7 +35,7 @@ SET allow_experimental_materialized_postgresql_table=1 如果需要使用多个表,强烈建议使用 [MaterializedPostgreSQL](../../../engines/database-engines/materialized-postgresql.md) 数据库引擎而不是表引擎,并通过 `materialized_postgresql_tables_list` 设置来指定要复制的表(后续也可以添加数据库 `schema`)。在 CPU 占用、连接数以及远程 PostgreSQL 数据库中所占用的复制槽数量方面,这种方式都会更优。 -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE postgresql_db.postgresql_replica (key UInt64, value UInt64) @@ -64,7 +64,7 @@ PRIMARY KEY key; -## 虚拟列 +## 虚拟列 {#virtual-columns} * `_version` — 事务计数器。类型:[UInt64](../../../sql-reference/data-types/int-uint.md)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md index 8615cab3803..246359f7d81 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mongodb.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# MongoDB 表引擎 +# MongoDB 表引擎 {#mongodb-table-engine} MongoDB 引擎是一种只读表引擎,用于从远程 [MongoDB](https://www.mongodb.com/) 集合中读取数据。 @@ -18,7 +18,7 @@ MongoDB 引擎是一种只读表引擎,用于从远程 [MongoDB](https://www.m -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -61,7 +61,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); | `oid_columns` | 以逗号分隔的列名列表,这些列在 WHERE 子句中将被视为 `oid` 类型。默认值为 `_id`。 | -## 类型映射 +## 类型映射 {#types-mappings} | MongoDB | ClickHouse | | ----------------------- | ---------------------------------------- | @@ -78,7 +78,7 @@ ENGINE = MongoDB(uri, collection[, oid_columns]); 如果在 MongoDB 文档中未找到键(例如列名不匹配),将插入默认值,或者在列可为 `NULL` 的情况下插入 `NULL`。 -### OID +### OID {#oid} 如果希望在 WHERE 子句中将某个 `String` 视为 `oid`,只需将该列名作为表引擎的最后一个参数传入。 在按 `_id` 列查询记录时可能需要这样做,因为在 MongoDB 中 `_id` 列默认具有 `oid` 类型。 @@ -132,7 +132,7 @@ SELECT count() FROM sample_oid WHERE another_oid_column = '67bf6cc40000000000ea4 ``` -## 支持的子句 +## 支持的子句 {#supported-clauses} 仅支持包含简单表达式的查询(例如,`WHERE field = ORDER BY field2 LIMIT `)。 此类表达式会被转换为 MongoDB 查询语言并在服务器端执行。 @@ -156,7 +156,7 @@ SELECT * FROM mongo_table WHERE date = '2024-01-01'::Date OR date = toDate('2024 这适用于 `Date`、`Date32`、`DateTime`、`Bool` 和 `UUID` 类型。 -## 使用示例 +## 使用示例 {#usage-example} 假设 MongoDB 中已经加载了 [sample_mflix](https://www.mongodb.com/docs/atlas/sample-data/sample-mflix) 数据集 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md index 5cc47e8d14f..b5a880dccb4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/mysql.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# MySQL 表引擎 +# MySQL 表引擎 {#mysql-table-engine} MySQL 引擎允许对存储在远程 MySQL 服务器上的数据执行 `SELECT` 和 `INSERT` 查询。 -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -67,7 +67,7 @@ CREATE TABLE test_replicas (id UInt32, name String, age UInt32, money UInt32) EN ``` -## 使用示例 +## 使用示例 {#usage-example} 在 MySQL 中创建表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md index f76f3426384..e2695a21f05 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/nats.md @@ -20,7 +20,7 @@ doc_type: 'guide' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -133,7 +133,7 @@ SSL 连接: ``` -## 描述 +## 描述 {#description} `SELECT` 对于读取消息(除调试用途外)并不是特别有用,因为每条消息只能被读取一次。更实用的方式是使用[物化视图](../../../sql-reference/statements/create/view.md)来创建实时处理流水线。为此,您需要: @@ -198,7 +198,7 @@ NATS 引擎支持 ClickHouse 所支持的所有[格式](../../../interfaces/form -## 使用 JetStream +## 使用 JetStream {#using-jetstream} 在将 NATS 引擎与 NATS JetStream 配合使用之前,必须先创建一个 NATS 流(stream)和一个持久拉取型消费者(durable pull consumer)。为此,可以使用 [NATS CLI](https://github.com/nats-io/natscli) 包中的 `nats` 工具,例如: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md index 385aa2c0f6b..9451ffbd470 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/odbc.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# ODBC 表引擎 +# ODBC 表引擎 {#odbc-table-engine} @@ -22,7 +22,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -51,7 +51,7 @@ ENGINE = ODBC(数据源, external_database, external_table) 这些参数也可以通过[命名集合](operations/named-collections.md)传递。 -## 使用示例 +## 使用示例 {#usage-example} **通过 ODBC 从本地 MySQL 实例中检索数据** diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md index 4db7dc6ad92..316e8de067e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/postgresql.md @@ -9,7 +9,7 @@ doc_type: '指南' -# PostgreSQL 表引擎 +# PostgreSQL 表引擎 {#postgresql-table-engine} PostgreSQL 引擎允许对存储在远程 PostgreSQL 服务器上的数据执行 `SELECT` 和 `INSERT` 查询。 @@ -23,7 +23,7 @@ PostgreSQL 引擎允许对存储在远程 PostgreSQL 服务器上的数据执行 -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -73,7 +73,7 @@ SELECT * FROM postgresql(postgres_creds, table='table1'); ``` -## 实现细节 +## 实现细节 {#implementation-details} PostgreSQL 端的 `SELECT` 查询在只读 PostgreSQL 事务中以 `COPY (SELECT ...) TO STDOUT` 的形式运行,每个 `SELECT` 查询结束后都会提交事务。 @@ -121,9 +121,9 @@ PostgreSQL 字典源支持副本优先级。映射中的数字越大,优先级 ``` -## 使用示例 +## 使用示例 {#usage-example} -### PostgreSQL 中的表 +### PostgreSQL 中的表 {#table-in-postgresql} ```text postgres=# CREATE TABLE "public"."test" ( @@ -146,7 +146,7 @@ postgresql> SELECT * FROM test; (1 row) ``` -### 在 ClickHouse 中创建表并连接到上文创建的 PostgreSQL 表 +### 在 ClickHouse 中创建表并连接到上文创建的 PostgreSQL 表 {#creating-table-in-clickhouse-and-connecting-to--postgresql-table-created-above} 此示例使用 [PostgreSQL 表引擎](/engines/table-engines/integrations/postgresql.md),将 ClickHouse 表连接到 PostgreSQL 表,并在 PostgreSQL 数据库上同时执行 SELECT 和 INSERT 查询: @@ -160,7 +160,7 @@ CREATE TABLE default.postgresql_table ENGINE = PostgreSQL('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### 使用 SELECT 查询将 PostgreSQL 表中的初始数据插入到 ClickHouse 表中 +### 使用 SELECT 查询将 PostgreSQL 表中的初始数据插入到 ClickHouse 表中 {#inserting-initial-data-from-postgresql-table-into-clickhouse-table-using-a-select-query} [postgresql table function](/sql-reference/table-functions/postgresql.md) 会将数据从 PostgreSQL 复制到 ClickHouse,通常用于在 ClickHouse 而非 PostgreSQL 中执行查询或分析,从而提升数据的查询性能,也可用于将数据从 PostgreSQL 迁移到 ClickHouse。由于我们将数据从 PostgreSQL 复制到 ClickHouse,因此会在 ClickHouse 中使用 MergeTree 表引擎,并将其命名为 postgresql_copy: @@ -180,7 +180,7 @@ INSERT INTO default.postgresql_copy SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postgres_user', 'postgres_password'); ``` -### 将 PostgreSQL 表中的增量数据插入到 ClickHouse 表中 +### 将 PostgreSQL 表中的增量数据插入到 ClickHouse 表中 {#inserting-incremental-data-from-postgresql-table-into-clickhouse-table} 如果在初始插入之后,随后需要在 PostgreSQL 表和 ClickHouse 表之间执行持续同步,可以在 ClickHouse 中使用 WHERE 子句,根据时间戳或唯一序列 ID,仅插入新增到 PostgreSQL 的数据。 @@ -198,7 +198,7 @@ SELECT * FROM postgresql('localhost:5432', 'public', 'test', 'postges_user', 'po WHERE int_id > maxIntID; ``` -### 从生成的 ClickHouse 表中查询数据 +### 从生成的 ClickHouse 表中查询数据 {#selecting-data-from-the-resulting-clickhouse-table} ```sql SELECT * FROM postgresql_copy WHERE str IN ('test'); @@ -210,7 +210,7 @@ SELECT * FROM postgresql_copy WHERE str IN ('test'); └────────────────┴──────┴────────┘ ``` -### 使用非默认模式 +### 使用非默认模式 {#using-non-default-schema} ```text postgres=# CREATE SCHEMA "nice.schema"; diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md index e122b83eb69..73185624a42 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/rabbitmq.md @@ -9,7 +9,7 @@ doc_type: 'guide' -# RabbitMQ 表引擎 +# RabbitMQ 表引擎 {#rabbitmq-table-engine} 该引擎用于将 ClickHouse 与 [RabbitMQ](https://www.rabbitmq.com) 集成。 @@ -20,7 +20,7 @@ doc_type: 'guide' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -131,7 +131,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ``` -## 描述 +## 描述 {#description} `SELECT` 对于读取消息并不是特别有用(除非用于调试),因为每条消息只能被读取一次。更实用的方式是使用[物化视图](../../../sql-reference/statements/create/view.md)创建实时处理流程。为此: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md index 2aa1b175082..63105254d6a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/redis.md @@ -9,13 +9,13 @@ doc_type: 'guide' -# Redis 表引擎 +# Redis 表引擎 {#redis-table-engine} 该引擎允许将 ClickHouse 与 [Redis](https://redis.io/) 集成。由于 Redis 采用键值(KV)模型,我们强烈建议仅执行点查询,例如使用 `where k = xx` 或 `where k in (xx, xx)`。 -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -47,7 +47,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name ::: -## 使用示例 +## 使用示例 {#usage-example} 在 ClickHouse 中使用 `Redis` 引擎和基本参数创建一张表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md index 18f17438425..a9452e685d0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3.md @@ -9,13 +9,13 @@ doc_type: 'reference' -# S3 表引擎 +# S3 表引擎 {#s3-table-engine} 该引擎提供与 [Amazon S3](https://aws.amazon.com/s3/) 生态系统的集成能力。此引擎类似于 [HDFS](/engines/table-engines/integrations/hdfs) 引擎,但提供了 S3 专用功能。 -## 示例 +## 示例 {#example} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -35,7 +35,7 @@ SELECT * FROM s3_engine_table LIMIT 2; ``` -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE s3_engine_table (name String, value UInt32) @@ -44,7 +44,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) [SETTINGS ...] ``` -### 引擎参数 +### 引擎参数 {#parameters} * `path` — 带有文件路径的存储桶 URL。在只读模式下支持以下通配符:`*`、`**`、`?`、`{abc,def}` 和 `{N..M}`,其中 `N`、`M` 为数字,`'abc'`、`'def'` 为字符串。更多信息参见[下文](#wildcards-in-path)。 * `NOSIGN` - 如果在凭证位置提供此关键字,所有请求都不会被签名。 @@ -55,7 +55,7 @@ CREATE TABLE s3_engine_table (name String, value UInt32) * `partition_columns_in_data_file` - 仅在使用 `HIVE` 分区策略时使用。用于指示 ClickHouse 数据文件中是否包含分区列。默认值为 `false`。 * `storage_class_name` - 选项:`STANDARD` 或 `INTELLIGENT_TIERING`,用于指定 [AWS S3 Intelligent Tiering](https://aws.amazon.com/s3/storage-classes/intelligent-tiering/) 存储类别。 -### 数据缓存 +### 数据缓存 {#data-cache} `S3` 表引擎支持在本地磁盘上进行数据缓存。 有关文件系统缓存配置选项和使用方法,请参见本[节](/operations/storing-data.md/#using-local-cache)。 @@ -86,13 +86,13 @@ SETTINGS filesystem_cache_name = 'cache_for_s3', enable_filesystem_cache = 1; 2. 复用 ClickHouse 中 `storage_configuration` 部分的缓存配置(以及相应的缓存存储),[如本节所述](/operations/storing-data.md/#using-local-cache) -### PARTITION BY +### PARTITION BY {#partition-by} `PARTITION BY` — 可选。大多数情况下您不需要分区键,即便需要,一般也不需要比“按月”更细粒度的分区键。分区并不会加快查询速度(与 ORDER BY 表达式相反)。绝不要使用粒度过细的分区。不要按客户端标识符或名称对数据进行分区(而应将客户端标识符或名称设置为 ORDER BY 表达式中的第一列)。 对于按月分区,使用 `toYYYYMM(date_column)` 表达式,其中 `date_column` 是类型为 [Date](/sql-reference/data-types/date.md) 的日期列。此时的分区名称采用 `"YYYYMM"` 格式。 -#### 分区策略 +#### 分区策略 {#partition-strategy} `WILDCARD`(默认):将文件路径中的 `{_partition_id}` 通配符替换为实际的分区键。不支持读取操作。 @@ -139,7 +139,7 @@ arthur :) select _path, * from t_03363_parquet; └────────────────────────────────────────────────────────────────────────────┴──────┴─────────┴─────────┘ ``` -### 查询分区数据 +### 查询分区数据 {#querying-partitioned-data} 本示例使用了集成 ClickHouse 和 MinIO 的 [docker compose 配置示例](https://github.com/ClickHouse/examples/tree/5fdc6ff72f4e5137e23ea075c88d3f44b0202490/docker-compose-recipes/recipes/ch-and-minio-S3)。你只需替换 endpoint 和认证参数即可在 S3 上复现相同的查询。 @@ -156,7 +156,7 @@ Cloud)。由于 ClickHouse 数据集通常非常庞大,且网络可靠性有 因此将数据集拆分为多个子集进行传输是合理的做法,这也正是进行分区写入的原因。 ::: -#### 创建表 +#### 创建表 {#create-the-table} ```sql CREATE TABLE p @@ -174,13 +174,13 @@ ENGINE = S3( PARTITION BY column3 ``` -#### 插入数据 +#### 插入数据 {#insert-data} ```sql INSERT INTO p VALUES (1, 2, 3), (3, 2, 1), (78, 43, 45) ``` -#### 查询分区 3 +#### 查询分区 3 {#select-from-partition-3} :::tip 此查询使用 S3 表函数。 @@ -197,7 +197,7 @@ FROM s3('http://minio:10000/clickhouse//test_3.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### 从分区 1 查询数据 +#### 从分区 1 查询数据 {#select-from-partition-1} ```sql SELECT * @@ -210,7 +210,7 @@ FROM s3('http://minio:10000/clickhouse//test_1.csv', 'minioadmin', 'minioadminpa └────┴────┴────┘ ``` -#### 从分区 45 中查询数据 +#### 从分区 45 中查询数据 {#select-from-partition-45} ```sql SELECT * @@ -223,7 +223,7 @@ FROM s3('http://minio:10000/clickhouse//test_45.csv', 'minioadmin', 'minioadminp └────┴────┴────┘ ``` -#### 限制 +#### 限制 {#limitation} 你可能很自然地会尝试执行 `Select * from p`,但如上所述,此查询会失败;请改用前面的查询。 @@ -270,7 +270,7 @@ SELECT * FROM p -## 路径中的通配符 +## 路径中的通配符 {#wildcards-in-path} `path` 参数可以使用类似 bash 的通配符来指定多个文件。要参与处理,文件必须实际存在,并且与整个路径模式完全匹配。文件列表在执行 `SELECT` 时确定(而不是在 `CREATE` 时)。 @@ -358,7 +358,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) -## 基于 endpoint 的设置 +## 基于 endpoint 的设置 {#endpoint-settings} 可以在配置文件中为指定的 endpoint 设置以下配置(通过 URL 的精确前缀进行匹配): @@ -402,7 +402,7 @@ CREATE TABLE table_with_asterisk (name String, value UInt32) ``` -## 使用归档文件 +## 使用归档文件 {#working-with-archives} 假设我们在 S3 上有若干归档文件,其 URI 如下: @@ -428,7 +428,7 @@ TAR ::: -## 访问公共 bucket +## 访问公共 bucket {#accessing-public-buckets} ClickHouse 会尝试从多种不同类型的来源自动获取凭证。 有时,在访问某些公共 bucket 时,这可能会导致问题,从而导致客户端返回 `403` 错误码。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md index 569aa8e8a23..690d01860c6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/s3queue.md @@ -1,5 +1,5 @@ --- -description: '此引擎提供与 Amazon S3 生态系统的集成,并支持流式导入。类似于 Kafka 和 RabbitMQ 引擎,但提供 S3 专有功能。' +description: '此引擎提供与 Amazon S3 生态系统的集成,并支持流式导入。类似于 Kafka 和 RabbitMQ 引擎,但提供 S3 专属特性。' sidebar_label: 'S3Queue' sidebar_position: 181 slug: /engines/table-engines/integrations/s3queue @@ -9,14 +9,11 @@ doc_type: 'reference' import ScalePlanFeatureBadge from '@theme/badges/ScalePlanFeatureBadge' +# S3Queue 表引擎 {#s3queue-table-engine} -# S3Queue 表引擎 - -该引擎提供与 [Amazon S3](https://aws.amazon.com/s3/) 生态系统的集成能力,并支持流式导入。此引擎类似于 [Kafka](../../../engines/table-engines/integrations/kafka.md)、[RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md) 等表引擎,但提供了特定于 S3 的功能。 - -需要注意 [S3Queue 实现的原始 PR](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183) 中的这条说明:当 `MATERIALIZED VIEW` 与该引擎建立关联后,S3Queue 表引擎会在后台开始收集数据。 - +该引擎提供与 [Amazon S3](https://aws.amazon.com/s3/) 生态系统的集成,并支持流式导入。该引擎类似于 [Kafka](../../../engines/table-engines/integrations/kafka.md)、[RabbitMQ](../../../engines/table-engines/integrations/rabbitmq.md) 引擎,但提供了 S3 特有的功能。 +需要特别注意 [S3Queue 实现的原始 PR](https://github.com/ClickHouse/ClickHouse/pull/49086/files#diff-e1106769c9c8fbe48dd84f18310ef1a250f2c248800fde97586b3104e9cd6af8R183) 中的这一说明:当有 `MATERIALIZED VIEW` 关联到该引擎时,S3Queue 表引擎会在后台开始收集数据。 ## 创建表 {#creating-a-table} @@ -49,12 +46,12 @@ CREATE TABLE s3_queue_engine_table (name String, value UInt32) ``` :::warning -在 `24.7` 版本之前,除 `mode`、`after_processing` 和 `keeper_path` 外,所有设置都必须使用 `s3queue_` 前缀。 +在 `24.7` 版本之前,除 `mode`、`after_processing` 和 `keeper_path` 之外的所有配置项都必须使用 `s3queue_` 前缀。 ::: **引擎参数** -`S3Queue` 的参数与 `S3` 表引擎支持的参数相同。请参阅[此处](../../../engines/table-engines/integrations/s3.md#parameters)的参数部分。 +`S3Queue` 的参数与 `S3` 表引擎支持的参数相同。请参见[此处](../../../engines/table-engines/integrations/s3.md#parameters)的参数部分。 **示例** @@ -65,7 +62,7 @@ SETTINGS mode = 'unordered'; ``` -使用命名集合: +使用命名集合: ```xml @@ -86,164 +83,256 @@ SETTINGS mode = 'ordered'; ``` - ## 设置 {#settings} -要获取表的配置设置列表,请使用 `system.s3_queue_settings` 表。从 `24.10` 版本开始可用。 +要获取为该表配置的设置列表,请查询 `system.s3_queue_settings` 系统表。自 `24.10` 版本起可用。 + +### Mode {#mode} + +可能的取值: + +* unordered — 在 `unordered` 模式下,所有已处理文件的集合会通过 ZooKeeper 中的持久节点进行跟踪和持久化。 +* ordered — 在 `ordered` 模式下,文件按照字典序进行处理。这意味着如果名为 `BBB` 的文件在某个时间点被处理了,之后又向 bucket 中添加了名为 `AA` 的文件,那么该文件将被忽略。ZooKeeper 中只会存储已成功消费文件中的最大文件名(按字典序),以及那些在加载失败后需要重试的文件名。 + +默认值:在 24.6 之前的版本中默认为 `ordered`。从 24.6 开始不再提供默认值,该设置必须手动指定。对于在更早版本中创建的表,为了兼容性其默认值仍为 `Ordered`。 -### 模式 {#mode} +### `after_processing` {#after_processing} -可能的值: +如何在文件成功处理后进行后续操作。 -- unordered — 在无序模式下,所有已处理文件的集合通过 ZooKeeper 中的持久节点进行跟踪。 -- ordered — 在有序模式下,文件按字典序处理。这意味着如果名为 'BBB' 的文件在某个时刻被处理,之后又向存储桶添加了名为 'AA' 的文件,该文件将被忽略。ZooKeeper 中仅存储成功消费文件的最大名称(按字典序)以及加载失败后将重试的文件名称。 +可能的取值: -默认值:在 24.6 之前的版本中为 `ordered`。从 24.6 开始没有默认值,该设置需要手动指定。对于在早期版本上创建的表,为保持兼容性,默认值将保持为 `Ordered`。 +* keep。 +* delete。 +* move。 +* tag。 -### `after_processing` {#after_processing} +默认值:`keep`。 -成功处理后删除或保留文件。 -可能的值: +`move` 需要额外配置。若在同一 bucket 内移动,必须通过 `after_processing_move_prefix` 提供新的路径前缀。 -- keep。 -- delete。 +移动到另一个 S3 bucket 时,需要通过 `after_processing_move_uri` 提供目标 bucket 的 URI,并通过 `after_processing_move_access_key_id` 和 `after_processing_move_secret_access_key` 提供 S3 凭证。 -默认值:`keep`。 +示例: -### `keeper_path` {#keeper_path} +```sql +CREATE TABLE s3queue_engine_table (name String, value UInt32) +ENGINE=S3Queue('https://clickhouse-public-datasets.s3.amazonaws.com/my-test-bucket-768/*', 'CSV', 'gzip') +SETTINGS + mode = 'unordered', + after_processing = 'move', + after_processing_retries = 20, + after_processing_move_prefix = 'dst_prefix', + after_processing_move_uri = 'https://clickhouse-public-datasets.s3.amazonaws.com/dst-bucket', + after_processing_move_access_key_id = 'test', + after_processing_move_secret_access_key = 'test'; +``` -ZooKeeper 中的路径可以指定为表引擎设置,或者可以从全局配置提供的路径和表 UUID 组成默认路径。 -可能的值: +从一个 Azure 容器移动到另一个 Azure 容器时,需要提供 Blob Storage 连接字符串作为 `after_processing_move_connection_string`,以及容器名称作为 `after_processing_move_container`。参见 [AzureQueue 设置](../../../engines/table-engines/integrations/azure-queue.md#settings)。 -- 字符串。 +标记操作需要提供标签键和值,分别通过 `after_processing_tag_key` 和 `after_processing_tag_value` 配置。 -默认值:`/`。 +### `after_processing_retries` {#after_processing_retries} -### `s3queue_loading_retries` {#loading_retries} +在放弃之前,对所请求的后处理操作进行重试的次数。 -重试文件加载最多指定次数。默认情况下不进行重试。 -可能的值: +可能的取值: -- 正整数。 +* 非负整数。 -默认值:`0`。 +默认值:`10`。 -### `s3queue_processing_threads_num` {#processing_threads_num} +### `after_processing_move_access_key_id` {#after_processing_move_access_key_id} -执行处理的线程数。仅适用于 `Unordered` 模式。 +如果目标是另一个 S3 bucket,则该选项为将成功处理的文件移动到目标 S3 bucket 所使用的 Access Key ID。 -默认值:CPU 数量或 16。 +可能的取值: -### `s3queue_parallel_inserts` {#parallel_inserts} +* 字符串。 -默认情况下,`processing_threads_num` 将产生一个 `INSERT`,因此它只会在多个线程中下载文件和解析。 -但这限制了并行性,因此为了获得更好的吞吐量,请使用 `parallel_inserts=true`,这将允许并行插入数据(但请注意,这将导致 MergeTree 系列生成更多的数据部分)。 +默认值:空字符串。 -:::note -`INSERT` 将根据 `max_process*_before_commit` 设置生成。 -::: +### `after_processing_move_prefix` {#after_processing_move_prefix} -默认值:`false`。 +成功处理后用于存放文件的路径前缀。适用于两种情况:在同一 bucket 内移动文件,或移动到另一个 bucket。 -### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} +可能的值: -启用记录到 `system.s3queue_log`。 +* 字符串。 -默认值:`0`。 +默认值:空字符串。 -### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} +### `after_processing_move_secret_access_key` {#after_processing_move_secret_access_key} -指定 ClickHouse 在进行下一次轮询尝试之前等待的最短时间(以毫秒为单位)。 +当目标是另一个 S3 bucket 时,用于将成功处理的文件移动到该目标 bucket 的 Secret Access Key。 -可能的值: +可能的取值: -- 正整数。 +* 字符串。 -默认值:`1000`。 +默认值:空字符串。 -### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} +### `after_processing_move_uri` {#after_processing_move_uri} -定义 ClickHouse 在启动下一次轮询尝试之前等待的最长时间(以毫秒为单位)。 +当目标是另一个 S3 bucket 时,用于存放已成功处理文件的 S3 bucket 的 URI。 -可能的值: +可能的取值: -- 正整数。 +* 字符串。 -默认值:`10000`。 +默认值:空字符串。 -### `s3queue_polling_backoff_ms` {#polling_backoff_ms} +### `after_processing_tag_key` {#after_processing_tag_key} -确定当未找到新文件时添加到上一次轮询间隔的额外等待时间。下一次轮询在上一次间隔与此退避值之和或最大间隔(以较小者为准)之后发生。 +当 `after_processing='tag'` 时,用于给成功处理的文件打标签的标签键。 -可能的值: +可能的取值: -- 正整数。 +* 字符串。 -默认值:`0`。 +默认值:空字符串。 -### `s3queue_tracked_files_limit` {#tracked_files_limit} +### `after_processing_tag_value` {#after_processing_tag_value} -如果使用 'unordered' 模式,允许限制 Zookeeper 节点的数量,对 'ordered' 模式不起作用。 -如果达到限制,最旧的已处理文件将从 ZooKeeper 节点中删除并再次处理。 +当 `after_processing='tag'` 时,为成功处理的文件设置标签时使用的标签值。 -可能的值: +可能的取值: -- 正整数。 +* 字符串。 -默认值:`1000`。 +默认值:空字符串。 -### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} +### `keeper_path` {#keeper_path} -在 ZooKeeper 节点中存储已处理文件的最大秒数(默认永久存储),适用于 'unordered' 模式,对 'ordered' 模式不起作用。 -在指定的秒数后,文件将被重新导入。 +ZooKeeper 中的路径可以通过表引擎设置单独指定;如果未指定,则默认路径由全局配置中提供的路径和表的 UUID 组成。 +可能的取值: -可能的值: +* 字符串。 -- 正整数。 +默认值:`/`。 -默认值:`0`。 +### `s3queue_loading_retries` {#loading_retries} -### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} +文件加载最多重试指定的次数。默认情况下,不进行重试。 +可能的取值: -适用于 'Ordered' 模式。定义后台任务重新调度间隔的最小边界,该任务负责维护跟踪文件的 TTL 和最大跟踪文件集。 +* 正整数。 -默认值:`10000`。 +默认值:`0`。 -### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} +### `s3queue_processing_threads_num` {#processing_threads_num} +用于执行处理的线程数。仅适用于 `Unordered` 模式。 -用于 'Ordered' 模式。定义后台任务重新调度间隔的最大边界值,该任务负责维护已跟踪文件的 TTL 和最大跟踪文件集合。 +默认值:CPU 数量或 16。 -默认值:`30000`。 +### `s3queue_parallel_inserts` {#parallel_inserts} -### `s3queue_buckets` {#buckets} +默认情况下,`processing_threads_num` 只会产生一个 `INSERT`,因此只是以多线程方式下载文件并进行解析。 +但这会限制并行度,因此为获得更高吞吐量,应使用 `parallel_inserts=true`,这将允许并行插入数据(但请注意,这会导致为 MergeTree 系列表引擎生成更多数据片段)。 + +:::note +`INSERT` 的触发会受 `max_process*_before_commit` 相关设置约束。 +::: + +默认值:`false`。 + +### `s3queue_enable_logging_to_s3queue_log` {#enable_logging_to_s3queue_log} + +启用将日志记录写入 `system.s3queue_log`。 + +默认值:`0`。 + +### `s3queue_polling_min_timeout_ms` {#polling_min_timeout_ms} + +指定 ClickHouse 在进行下一次轮询尝试之前等待的最短时间(以毫秒为单位)。 + +可能的取值: + +* 正整数。 + +默认值:`1000`。 + +### `s3queue_polling_max_timeout_ms` {#polling_max_timeout_ms} + +定义 ClickHouse 在发起下一次轮询尝试之前的最大等待时间(以毫秒为单位)。 + +可能的取值: -用于 'Ordered' 模式。自 `24.6` 版本起可用。如果存在多个 S3Queue 表副本,且每个副本都使用 keeper 中的同一元数据目录,则 `s3queue_buckets` 的值至少需要等于副本数量。如果同时使用了 `s3queue_processing_threads` 设置,则进一步增加 `s3queue_buckets` 设置的值是合理的,因为它定义了 `S3Queue` 处理的实际并行度。 +* 正整数。 -### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} +默认值:`10000`。 -默认情况下,S3Queue 表始终使用临时处理节点,这可能导致数据重复,即当 zookeeper 会话在 S3Queue 开始处理之后但在 zookeeper 中提交已处理文件之前过期时。此设置强制服务器在 keeper 会话过期的情况下消除数据重复的可能性。 +### `s3queue_polling_backoff_ms` {#polling_backoff_ms} -### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} +确定在未发现新文件时,需要加到上一次轮询间隔上的额外等待时间。下一次轮询会在上一次间隔与该退避值之和或最大间隔(取二者中较小值)之后触发。 -在服务器非正常终止的情况下,如果启用了 `use_persistent_processing_nodes`,可能会存在未删除的处理节点。此设置定义了可以安全清理这些处理节点的时间周期。 +可能的取值: -默认值:`3600`(1 小时)。 +* 正整数。 +默认值:`0`。 -## S3 相关设置 {#s3-settings} +### `s3queue_tracked_files_limit` {#tracked_files_limit} -引擎支持所有 S3 相关设置。有关 S3 设置的更多信息,请参阅[此处](../../../engines/table-engines/integrations/s3.md)。 +在使用 `unordered` 模式时,用于限制 ZooKeeper 节点的数量,对 `ordered` 模式不起作用。 +一旦达到该限制,最早已处理的文件会从 ZooKeeper 节点中删除,并再次被处理。 +可能的取值: -## S3 基于角色的访问 {#s3-role-based-access} +* 正整数。 - +默认值:`1000`。 -s3Queue 表引擎支持基于角色的访问。 -有关配置角色以访问存储桶的步骤,请参阅[此处](/cloud/data-sources/secure-s3)的文档。 +### `s3queue_tracked_file_ttl_sec` {#tracked_file_ttl_sec} -配置角色后,可以通过 `extra_credentials` 参数传递 `roleARN`,如下所示: +在“unordered”模式下,在 ZooKeeper 节点中保留已处理文件的最长时间(以秒为单位,默认永久保存)。对“ordered”模式无效。 +在经过指定秒数后,该文件会被重新导入。 + +可能的取值: + +* 正整数。 + +默认值:`0`。 + +### `s3queue_cleanup_interval_min_ms` {#cleanup_interval_min_ms} + +用于 `Ordered` 模式。定义负责维护已跟踪文件 TTL 和最大已跟踪文件集合的后台任务在重新调度间隔上的最小值。 + +默认值:`10000`。 + +### `s3queue_cleanup_interval_max_ms` {#cleanup_interval_max_ms} + +用于 “Ordered” 模式。定义重新调度后台任务的时间间隔的最大上限,该后台任务负责维护已跟踪文件的 TTL,以及已跟踪文件集合的最大大小。 + +默认值:`30000`。 + +### `s3queue_buckets` {#buckets} + +用于 Ordered 模式。从 `24.6` 版本开始可用。如果存在多个 S3Queue 表副本,并且它们都使用 Keeper 中相同的元数据目录,那么 `s3queue_buckets` 的值至少要等于副本数量。如果同时使用了 `s3queue_processing_threads` 设置,那么进一步增大 `s3queue_buckets` 的值是合理的,因为它决定了 `S3Queue` 处理的实际并行度。 + +### `use_persistent_processing_nodes` {#use_persistent_processing_nodes} + +默认情况下,S3Queue 表始终使用临时处理节点。如果在 S3Queue 将已处理文件提交到 ZooKeeper 之前、但在其开始处理之后,ZooKeeper 会话过期,就可能导致数据重复。此设置会强制服务器在 Keeper 会话过期的情况下消除产生重复数据的可能性。 + +### `persistent_processing_nodes_ttl_seconds` {#persistent_processing_nodes_ttl_seconds} + +在服务器异常终止的情况下,如果启用了 `use_persistent_processing_nodes`,就有可能出现尚未被移除的处理节点(processing nodes)。此设置用于定义一个时间段,在该时间段内,这些处理节点可以被安全清理。 + +默认值:`3600`(1 小时)。 + +## 与 S3 相关的设置 {#s3-settings} + +该引擎支持所有 S3 相关设置。有关 S3 设置的更多信息,请参阅[此处](../../../engines/table-engines/integrations/s3.md)。 + +## 基于角色的 S3 访问 {#s3-role-based-access} + + + +`s3Queue` 表引擎支持基于角色的访问控制。 +请参阅[此处](/cloud/data-sources/secure-s3)的文档,了解如何配置用于访问您的 bucket 的角色。 + +角色配置完成后,可以通过 `extra_credentials` 参数传递一个 `roleARN`,如下所示: ```sql CREATE TABLE s3_table @@ -259,28 +348,27 @@ SETTINGS ... ``` - ## S3Queue 有序模式 {#ordered-mode} -`S3Queue` 处理模式允许在 ZooKeeper 中存储更少的元数据,但存在一个限制:后续添加的文件必须具有按字母数字顺序更大的名称。 +`S3Queue` 处理模式可以在 ZooKeeper 中存储更少的元数据,但有一个限制:按时间更晚添加的文件,其名称在字母数字顺序上必须更大。 -`S3Queue` 的 `ordered` 模式与 `unordered` 模式一样,支持 `(s3queue_)processing_threads_num` 设置(`s3queue_` 前缀可选),用于控制服务器本地处理 `S3` 文件的线程数量。 -此外,`ordered` 模式还引入了另一个名为 `(s3queue_)buckets` 的设置,表示"逻辑线程"。在分布式场景中,当存在多个具有 `S3Queue` 表副本的服务器时,此设置定义处理单元的数量。例如,每个 `S3Queue` 副本上的每个处理线程都会尝试锁定某个 `bucket` 进行处理,每个 `bucket` 通过文件名的哈希值关联到特定文件。因此,在分布式场景中,强烈建议将 `(s3queue_)buckets` 设置为至少等于副本数量或更大。bucket 数量大于副本数量是允许的。最优配置是将 `(s3queue_)buckets` 设置为 `number_of_replicas` 与 `(s3queue_)processing_threads_num` 的乘积。 -不建议在 `24.6` 版本之前使用 `(s3queue_)processing_threads_num` 设置。 -`(s3queue_)buckets` 设置从 `24.6` 版本开始提供。 +`S3Queue` 的 `ordered` 模式与 `unordered` 模式一样,支持 `(s3queue_)processing_threads_num` 设置(`s3queue_` 前缀是可选的),用于控制在服务器本地处理 `S3` 文件的线程数量。 +此外,`ordered` 模式还引入了另一个名为 `(s3queue_)buckets` 的设置,表示“逻辑线程”。在分布式场景中,当存在多个带有 `S3Queue` 表副本的服务器时,该设置定义了处理单元的数量。比如,每个 `S3Queue` 副本上的每个处理线程都会尝试锁定某个用于处理的 `bucket`,每个 `bucket` 通过文件名的哈希与特定文件关联。因此,在分布式场景下,强烈建议将 `(s3queue_)buckets` 设置为至少等于副本数量或更大。`buckets` 数量大于副本数量是可以的。最优的情况是将 `(s3queue_)buckets` 设置为 `number_of_replicas` 与 `(s3queue_)processing_threads_num` 的乘积。 +不建议在 `24.6` 版本之前使用 `(s3queue_)processing_threads_num` 设置。 +`(s3queue_)buckets` 设置从 `24.6` 版本开始可用。 -## Description {#description} +## 描述 {#description} -`SELECT` 对于流式导入并不特别有用(除了调试场景),因为每个文件只能导入一次。更实用的做法是使用[物化视图](../../../sql-reference/statements/create/view.md)创建实时数据流。操作步骤如下: +`SELECT` 对于流式导入并不是特别有用(除调试外),因为每个文件只能被导入一次。更实用的做法是使用[物化视图](../../../sql-reference/statements/create/view.md)来创建实时数据流。为此: -1. 使用该引擎创建一个表,从 S3 指定路径消费数据,并将其视为数据流。 -2. 创建一个具有所需结构的表。 -3. 创建一个物化视图,将引擎中的数据转换后写入之前创建的表中。 +1. 使用该引擎创建一张表,从 S3 中指定的路径消费数据,并将其视为数据流。 +2. 创建一张具有所需结构的表。 +3. 创建一个物化视图,将来自该引擎的数据转换后写入事先创建好的表中。 -当 `MATERIALIZED VIEW` 关联到引擎后,它会在后台开始收集数据。 +当 `MATERIALIZED VIEW` 与该引擎关联后,它会在后台开始收集数据。 -示例: +示例: ```sql CREATE TABLE s3queue_engine_table (name String, value UInt32) @@ -297,48 +385,44 @@ SETTINGS SELECT * FROM stats ORDER BY name; ``` - ## 虚拟列 {#virtual-columns} -- `_path` — 文件的路径。 -- `_file` — 文件的名称。 -- `_size` — 文件的大小。 -- `_time` — 文件的创建时间。 - -有关虚拟列的更多信息,请参阅[此处](../../../engines/table-engines/index.md#table_engines-virtual_columns)。 +* `_path` — 文件路径。 +* `_file` — 文件名。 +* `_size` — 文件大小。 +* `_time` — 文件创建时间。 +有关虚拟列的更多信息,请参阅[此处](../../../engines/table-engines/index.md#table_engines-virtual_columns)。 ## 路径中的通配符 {#wildcards-in-path} -`path` 参数可以使用类似 bash 的通配符来指定多个文件。待处理的文件必须存在且匹配完整的路径模式。文件列表在执行 `SELECT` 时确定(而非在 `CREATE` 时)。 - -- `*` — 匹配除 `/` 之外的任意数量字符,包括空字符串。 -- `**` — 匹配任意数量字符,包括 `/`,也包括空字符串。 -- `?` — 匹配任意单个字符。 -- `{some_string,another_string,yet_another_one}` — 匹配字符串 `'some_string'`、`'another_string'`、`'yet_another_one'` 中的任意一个。 -- `{N..M}` — 匹配从 N 到 M 范围内的任意数字,包括两端边界。N 和 M 可以包含前导零,例如 `000..078`。 +`path` 参数可以使用类似 bash 的通配符来指定多个文件。要参与处理,文件必须存在并且与整个路径模式完全匹配。文件列表在执行 `SELECT` 时确定(而不是在 `CREATE` 时)。 -使用 `{}` 的语法与 [remote](../../../sql-reference/table-functions/remote.md) 表函数类似。 +* `*` — 匹配除 `/` 之外的任意数量任意字符,包括空字符串。 +* `**` — 匹配包括 `/` 在内的任意数量任意字符,包括空字符串。 +* `?` — 匹配任意单个字符。 +* `{some_string,another_string,yet_another_one}` — 匹配字符串 `'some_string'、'another_string'、'yet_another_one'` 中的任意一个。 +* `{N..M}` — 匹配从 N 到 M(包含两端)范围内的任意数字。N 和 M 可以有前导零,例如 `000..078`。 +带有 `{}` 的结构类似于 [remote](../../../sql-reference/table-functions/remote.md) 表函数。 ## 限制 {#limitations} -1. 重复行可能由以下原因导致: - -- 在文件处理过程中解析时发生异常,且通过 `s3queue_loading_retries` 启用了重试; +1. 出现重复行可能是由于以下原因导致: -- `S3Queue` 在多个服务器上配置并指向 ZooKeeper 中的同一路径,且在某个服务器成功提交已处理文件之前 Keeper 会话过期,这可能导致另一个服务器接管该文件的处理,而该文件可能已被第一个服务器部分或完全处理;但是,从 25.8 版本开始,如果设置 `use_persistent_processing_nodes = 1`,则不会出现此情况。 +* 在文件处理中途解析时发生异常,并且通过 `s3queue_loading_retries` 启用了重试; -- 服务器异常终止。 +* 在多个服务器上配置了 `S3Queue`,它们在 zookeeper 中指向同一路径,并且在某个服务器成功提交已处理文件之前 Keeper 会话过期,这可能导致另一台服务器接手处理该文件,而该文件可能已被第一台服务器部分或全部处理;然而,自 25.8 版本起,如果 `use_persistent_processing_nodes = 1`,则不会出现此问题。 -2. 如果 `S3Queue` 在多个服务器上配置并指向 ZooKeeper 中的同一路径,且使用 `Ordered` 模式,则 `s3queue_loading_retries` 将不起作用。此问题将很快修复。 +* 服务器异常终止。 +2. 在多个服务器上配置了 `S3Queue` 且它们在 zookeeper 中指向同一路径,并使用 `Ordered` 模式时,`s3queue_loading_retries` 将不起作用。该问题很快将得到修复。 -## 内省 {#introspection} +## 自省 {#introspection} -用于内省可使用 `system.s3queue` 无状态表和 `system.s3queue_log` 持久化表。 +要进行自省,请使用无状态表 `system.s3queue` 和持久化表 `system.s3queue_log`。 -1. `system.s3queue`。此表非持久化,显示 `S3Queue` 的内存状态:当前正在处理的文件、已处理或失败的文件。 +1. `system.s3queue`。此表为非持久化表,用于显示 `S3Queue` 的内存中状态:当前正在处理哪些文件、哪些文件已处理完成或处理失败。 ```sql ┌─statement──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ @@ -359,7 +443,7 @@ COMMENT '包含 S3Queue 元数据的内存状态以及每个文件当前处理 └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -示例: +示例: ```sql @@ -378,9 +462,9 @@ ProfileEvents: {'ZooKeeperTransactions':3,'ZooKeeperGet':2,'ZooKeeperMul exception: ``` -2. `system.s3queue_log`。持久化表。包含与 `system.s3queue` 相同的信息,但仅针对 `processed` 和 `failed` 状态的文件。 +2. `system.s3queue_log`。持久化表。包含与 `system.s3queue` 相同的信息,但记录的是 `processed` 和 `failed` 文件。 -该表具有以下结构: +该表具有以下结构: ```sql SHOW CREATE TABLE system.s3queue_log @@ -408,7 +492,7 @@ SETTINGS index_granularity = 8192 │ └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ ``` -要使用 `system.s3queue_log`,需在服务器配置文件中定义其配置: +要使用 `system.s3queue_log`,需要在服务器配置文件中为其添加相应配置: ```xml @@ -417,7 +501,7 @@ SETTINGS index_granularity = 8192 │ ``` -示例: +示例: ```sql SELECT * diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md index b1ae9767817..545e272e4d1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/sqlite.md @@ -9,16 +9,13 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; +# SQLite 表引擎 {#sqlite-table-engine} -# SQLite 表引擎 - - + 该引擎用于向 SQLite 导入数据或从 SQLite 导出数据,并支持在 ClickHouse 中直接查询 SQLite 表。 - - -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -33,8 +30,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; * `db_path` — SQLite 数据库文件的路径。 * `table` — SQLite 数据库中表的名称。 - -## 使用示例 +## 使用示例 {#usage-example} 示例查询,用于创建 SQLite 表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md index 4d215ee4c3e..0670c1d5549 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/time-series.md @@ -11,7 +11,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# TimeSeries 表引擎 +# TimeSeries 表引擎 {#timeseries-table-engine} @@ -31,7 +31,7 @@ metric_name2[...] = ... ::: -## 语法 +## 语法 {#syntax} ```sql CREATE TABLE name [(columns)] ENGINE=TimeSeries @@ -42,7 +42,7 @@ CREATE TABLE name [(columns)] ENGINE=TimeSeries ``` -## 用法 +## 用法 {#usage} 使用全部默认设置开始会更简单(允许在不指定列列表的情况下创建 `TimeSeries` 表): @@ -116,7 +116,7 @@ _metrics_ 表必须包含以下列: -## 创建 +## 创建 {#creation} 使用 `TimeSeries` 表引擎创建表有多种方式。 最简单的语句如下: @@ -201,7 +201,7 @@ ORDER BY metric_family_name ``` -## 调整列类型 +## 调整列类型 {#adjusting-column-types} 在定义主表时,通过显式指定列类型,可以调整内部目标表中几乎任意列的类型。例如, @@ -226,7 +226,7 @@ ORDER BY (id, timestamp) ``` -## `id` 列 +## `id` 列 {#id-column} `id` 列包含标识符,每个标识符是根据指标名称与标签的组合计算得到的。 `id` 列的 DEFAULT 表达式是用于计算这些标识符的表达式。 @@ -241,7 +241,7 @@ ENGINE=TimeSeries ``` -## `tags` 与 `all_tags` 列 +## `tags` 与 `all_tags` 列 {#tags-and-all-tags} 有两列包含标签映射——`tags` 和 `all_tags`。在本例中它们含义相同,但在使用 `tags_to_columns` 设置项时,它们可能会不同。该设置项允许指定某个特定标签应存储在单独的列中,而不是作为映射存储在 `tags` 列中: @@ -273,7 +273,7 @@ SETTINGS tags_to_columns = {'instance': 'instance', 'job': 'job'} ``` -## 内部目标表的表引擎 +## 内部目标表的表引擎 {#inner-table-engines} 默认情况下,内部目标表使用以下表引擎: @@ -293,7 +293,7 @@ METRICS ENGINE=ReplicatedReplacingMergeTree ``` -## 外部目标表 +## 外部目标表 {#external-target-tables} 可以让 `TimeSeries` 表使用一个手动创建的表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md index d0be599a80a..3f595ddb64f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/integrations/ytsaurus.md @@ -12,7 +12,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# YTsaurus 表引擎 +# YTsaurus 表引擎 {#ytsaurus-table-engine} @@ -21,7 +21,7 @@ YTsaurus 表引擎用于从 YTsaurus 集群导入数据。 -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name @@ -48,7 +48,7 @@ YTsaurus 表引擎用于从 YTsaurus 集群导入数据。 * `oauth_token` — OAuth 令牌。 -## 使用示例 +## 使用示例 {#usage-example} 以下是一个用于创建 YTsaurus 表的查询: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md index a4f3eb8a9b9..de8baa9ad0b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/index.md @@ -10,7 +10,7 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Log 表引擎系列 +# Log 表引擎系列 {#log-table-engine-family} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md index f074b96cc1f..5a6805e08d9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/log.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# Log 表引擎 +# Log 表引擎 {#log-table-engine} @@ -22,7 +22,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 创建表 +## 创建表 {#table_engines-log-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -59,7 +59,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用示例 +## 使用示例 {#table_engines-log-example-of-use} 创建表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md index e7c6892c914..cb34e5762ae 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/stripelog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# StripeLog 表引擎 +# StripeLog 表引擎 {#stripelog-table-engine} @@ -20,7 +20,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 创建数据表 +## 创建数据表 {#table_engines-stripelog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -53,7 +53,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用示例 +## 使用示例 {#table_engines-stripelog-example-of-use} 创建表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md index 4f992397be9..5ed29f3ca43 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/log-family/tinylog.md @@ -10,7 +10,7 @@ doc_type: 'reference' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -# TinyLog 表引擎 +# TinyLog 表引擎 {#tinylog-table-engine} @@ -32,7 +32,7 @@ import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; -## 创建表 +## 创建表 {#table_engines-tinylog-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -58,7 +58,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 示例用法 +## 示例用法 {#table_engines-tinylog-example-of-use} 创建表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md index 2474458e7cb..285eb6e0cab 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/aggregatingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# AggregatingMergeTree 表引擎 +# AggregatingMergeTree 表引擎 {#aggregatingmergetree-table-engine} 该引擎继承自 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree),并对数据部分的合并逻辑进行了调整。ClickHouse 会将所有具有相同主键(更准确地说,是具有相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md))的行在单个数据部分内合并为一行,该行存储了聚合函数状态的组合。 @@ -29,7 +29,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -80,7 +80,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 聚合物化视图示例 +## 聚合物化视图示例 {#example-of-an-aggregated-materialized-view} 以下示例假设已存在名为 `test` 的数据库。如尚不存在,请使用以下命令创建: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md index cb2c3e1082e..188027b0fa0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md @@ -10,7 +10,7 @@ doc_type: 'guide' import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# 精确与近似向量搜索 +# 精确与近似向量搜索 {#exact-and-approximate-vector-search} 在给定多维(向量)空间中的一个点时,寻找与其距离最近的 N 个点的问题,被称为[最近邻搜索](https://en.wikipedia.org/wiki/Nearest_neighbor_search),简称向量搜索。 解决向量搜索通常有两种通用方法: @@ -36,13 +36,13 @@ LIMIT `<N>` 指定应返回多少个近邻。 -## 精确向量搜索 +## 精确向量搜索 {#exact-nearest-neighbor-search} 可以直接使用上面的 SELECT 查询执行精确向量搜索。 此类查询的运行时间通常与已存储向量的数量及其维度成正比,即数组元素的数量。 此外,由于 ClickHouse 会对所有向量进行暴力扫描(brute-force scan),运行时间还取决于查询使用的线程数(参见设置 [max_threads](../../../operations/settings/settings.md#max_threads))。 -### 示例 +### 示例 {#exact-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; @@ -67,9 +67,9 @@ LIMIT 3; ``` -## 近似向量搜索 +## 近似向量搜索 {#approximate-nearest-neighbor-search} -### 向量相似度索引 +### 向量相似度索引 {#vector-similarity-index} ClickHouse 提供了一种专用的“向量相似度”索引,用于执行近似向量搜索。 @@ -78,7 +78,7 @@ ClickHouse 提供了一种专用的“向量相似度”索引,用于执行近 如果遇到问题,请在 [ClickHouse 仓库](https://github.com/clickhouse/clickhouse/issues) 中提交 issue。 ::: -#### 创建向量相似度索引 +#### 创建向量相似度索引 {#creating-a-vector-similarity-index} 可以在新表上按如下方式创建向量相似度索引: @@ -196,7 +196,7 @@ ORDER BY [...] 上述公式未将向量相似度索引在分配运行时数据结构(例如预分配缓冲区和缓存)时所需的额外内存考虑在内。 -#### 使用向量相似度索引 +#### 使用向量相似度索引 {#using-a-vector-similarity-index} :::note 要使用向量相似度索引,[compatibility](../../../operations/settings/settings.md) 设置必须为 `''`(默认值)或不低于 `'25.1'` 的版本。 @@ -406,7 +406,7 @@ Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 在未启用重打分(`vector_search_with_rescoring = 0`)且启用了并行副本的情况下运行的查询,可能会回退为执行重打分。 ::: -#### 性能调优 +#### 性能调优 {#performance-tuning} **压缩调优** @@ -540,7 +540,7 @@ result = chclient.query( 在本示例中,参考向量以原始二进制形式发送到服务器,并在服务器端被重新解释为浮点数数组。 这可以节省服务器端的 CPU 时间,并避免服务器日志和 `system.query_log` 的膨胀。 -#### 管理和监控 +#### 管理和监控 {#administration} 向量相似性索引在磁盘上的大小可以通过 [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) 获取: @@ -558,7 +558,7 @@ WHERE type = 'vector_similarity'; └──────────┴───────┴──────┴──────────────────────────┘ ``` -#### 与常规跳过索引的区别 +#### 与常规跳过索引的区别 {#differences-to-regular-skipping-indexes} 与所有常规[跳过索引](/optimize/skipping-indexes)类似,向量相似度索引也是在 granule 之上构建的,每个已建立索引的块由 `GRANULARITY = [N]` 个 granule 组成(对普通跳过索引而言,`[N]` 默认为 1)。 例如,如果表的主索引粒度为 8192(设置 `index_granularity = 8192`)且 `GRANULARITY = 2`,则每个已建立索引的块将包含 16384 行。 @@ -584,7 +584,7 @@ WHERE type = 'vector_similarity'; 通常建议为向量相似度索引使用较大的 `GRANULARITY`,只有在出现例如向量相似度结构内存占用过高等问题时,才改用较小的 `GRANULARITY` 值。 如果没有为向量相似度索引指定 `GRANULARITY`,则默认值为 1 亿。 -#### 示例 +#### 示例 {#approximate-nearest-neighbor-search-example} ```sql CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; @@ -615,7 +615,7 @@ LIMIT 3; * [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) * [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) -### 量化比特(QBit) +### 量化比特(QBit) {#approximate-nearest-neighbor-search-qbit} @@ -649,7 +649,7 @@ ClickHouse 提供了 Quantized Bit(`QBit`)数据类型,通过以下方式 * `element_type` – 每个向量元素的类型。支持的类型包括 `BFloat16`、`Float32` 和 `Float64` * `dimension` – 每个向量中的元素个数 -#### 创建 `QBit` 表并添加数据 +#### 创建 `QBit` 表并添加数据 {#qbit-create} ```sql CREATE TABLE fruit_animal ( @@ -667,7 +667,7 @@ INSERT INTO fruit_animal VALUES ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); ``` -#### 使用 `QBit` 进行向量搜索 +#### 使用 `QBit` 进行向量搜索 {#qbit-search} 我们使用 L2 距离查找与表示单词 “lemon” 的向量最接近的邻居向量。距离函数的第三个参数指定精度的位数——值越高,精度越高,但计算量也越大。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md index 602ad0ed321..5217eb20dc4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md @@ -9,9 +9,7 @@ show_related_blogs: true doc_type: 'reference' --- - - -# CoalescingMergeTree 表引擎 +# CoalescingMergeTree 表引擎 {#coalescingmergetree-table-engine} :::note Available from version 25.6 此表引擎从 25.6 及更高版本开始在 OSS 和 Cloud 中可用。 @@ -23,9 +21,7 @@ doc_type: 'reference' `CoalescingMergeTree` 旨在与非键列中的 Nullable 类型配合使用。如果这些列不是 Nullable,其行为与 [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree) 相同。 - - -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -42,16 +38,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] 有关请求参数的说明,请参阅[请求描述](../../../sql-reference/statements/create/table.md)。 -### CoalescingMergeTree 的参数 +### CoalescingMergeTree 的参数 {#parameters-of-coalescingmergetree} -#### 列 +#### 列 {#columns} `columns` - 一个包含需要合并其值的列名的元组(tuple)。可选参数。 这些列必须是数值类型,并且不能出现在分区键或排序键中。 如果未指定 `columns`,ClickHouse 会合并所有不在排序键中的列的值。 -### 查询子句 +### 查询子句 {#query-clauses} 在创建 `CoalescingMergeTree` 表时,所需的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)与创建 `MergeTree` 表时相同。 @@ -76,8 +72,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `columns` — 一个包含列名的元组(tuple),这些列的值将被求和。可选参数。相关说明见上文。
- -## 使用示例 +## 使用示例 {#usage-example} 请看下表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md index dc6faddbb44..bab22872ef8 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md @@ -10,7 +10,7 @@ doc_type: 'guide' -# CollapsingMergeTree 表引擎 +# CollapsingMergeTree 表引擎 {#collapsingmergetree-table-engine} @@ -40,7 +40,7 @@ doc_type: 'guide' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -81,9 +81,9 @@ ENGINE = CollapsingMergeTree(Sign) * 创建 `CollapsingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[查询子句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。 -## 折叠 +## 折叠 {#table_engine-collapsingmergetree-collapsing} -### 数据 +### 数据 {#data} 考虑这样一种情况:你需要为某个给定对象保存持续变化的数据。 看起来为每个对象只保留一行并在有变化时更新它似乎是合乎逻辑的, @@ -141,7 +141,7 @@ ENGINE = CollapsingMergeTree(Sign) 2. 列中不断增长的长数组会因为写入负载增加而降低引擎效率。数据越简单,效率越高。 3. `SELECT` 的结果高度依赖于对象变更历史的一致性。在准备要插入的数据时要谨慎。对于不一致的数据,可能会得到不可预测的结果。例如,本应非负的指标(如会话深度)出现负值。 -### 算法 +### 算法 {#table_engine-collapsingmergetree-collapsing-algorithm} 当 ClickHouse 合并数据[分片](/concepts/glossary#parts)时, 每组具有相同排序键(`ORDER BY`)的连续行会被折叠为最多两行, @@ -186,9 +186,9 @@ ClickHouse 使用多个线程处理 `SELECT` 查询,因此无法预测结果 -## 示例 +## 示例 {#examples} -### 使用示例 +### 使用示例 {#example-of-use} 给出以下示例数据: @@ -288,7 +288,7 @@ SELECT * FROM UAct FINAL 这种数据选取方式效率较低,不建议在扫描数据量很大(数百万行)时使用。 ::: -### 另一种方法示例 +### 另一种方法示例 {#example-of-another-approach} 这种方法的思路是,合并操作只考虑键字段。 因此,在 "cancel" 行中,我们可以指定负值, diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md index 21830741a9d..8f6caa9e637 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md @@ -7,9 +7,7 @@ title: '自定义分区键' doc_type: 'guide' --- - - -# 自定义分区键 +# 自定义分区键 {#custom-partitioning-key} :::note 在大多数情况下,无需使用分区键;在其他大多数情况下,除非是针对按天分区较为常见的可观测性场景,否则也不需要比“按月”更细粒度的分区键。 @@ -78,7 +76,6 @@ WHERE table = 'visits' `partition` 列包含分区的名称。在此示例中有两个分区:`201901` 和 `201902`。可以使用该列的值在 [ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md) 查询中指定分区名称。 - `name` 列包含分区数据 part 的名称。你可以在 [ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart) 查询中使用该列来指定 part 的名称。 下面我们来拆解这个 part 名称:`201901_1_9_2_11`: @@ -134,16 +131,13 @@ drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached 文件夹 '201901_1_1_0'、'201901_1_7_1' 等,是各个 part 的目录。每个 part 属于一个分区,并且只包含某一个月的数据(本示例中的表是按月分区的)。 - `detached` 目录包含通过 [DETACH](/sql-reference/statements/detach) 查询从表中分离出去的 part。损坏的 part 也会被移动到该目录,而不是直接删除。服务器不会使用 `detached` 目录中的这些 part。你可以在任何时候在该目录中添加、删除或修改数据——在你执行 [ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart) 查询之前,服务器都不会察觉到这些更改。 请注意,在正在运行的服务器上,你不能在文件系统中手动更改 part 的集合或其数据,因为服务器不会感知到这些变更。对于非复制表,你可以在服务器停止时进行此操作,但不推荐这样做。对于复制表,在任何情况下都不能更改 part 的集合。 ClickHouse 允许你对分区执行操作:删除分区、在表之间复制分区,或者创建备份。所有操作的完整列表请参见 [Manipulations With Partitions and Parts](/sql-reference/statements/alter/partition) 一节。 - - -## 使用分区键进行 GROUP BY 优化 +## 使用分区键进行 GROUP BY 优化 {#group-by-optimisation-using-partition-key} 对于某些表的分区键与查询的 GROUP BY 键的特定组合,可以针对每个分区独立执行聚合。 这样在最后我们就不必合并所有执行线程产生的部分聚合结果, diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md index 1dcccded6d1..69e4427819e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md @@ -7,9 +7,7 @@ title: 'GraphiteMergeTree 表引擎' doc_type: 'guide' --- - - -# GraphiteMergeTree 表引擎 +# GraphiteMergeTree 表引擎 {#graphitemergetree-table-engine} 该引擎用于对 [Graphite](http://graphite.readthedocs.io/en/latest/index.html) 数据进行稀疏化和聚合/平均(rollup)处理。它对希望使用 ClickHouse 作为 Graphite 数据存储的开发者非常有用。 @@ -17,9 +15,7 @@ doc_type: 'guide' 该引擎继承自 [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) 的属性。 - - -## 创建表 +## 创建表 {#creating-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -82,8 +78,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `config_section` — 配置文件中定义 rollup 规则的节名称。
- -## Rollup 配置 +## Rollup 配置 {#rollup-configuration} Rollup 的配置由服务器配置中的 [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) 参数定义。该参数的名称可以任意。你可以创建多个配置,并将它们用于不同的表。 @@ -92,25 +87,25 @@ Rollup 配置结构: required-columns patterns -### 必需列 +### 必需列 {#required-columns} -#### `path_column_name` +#### `path_column_name` {#path_column_name} `path_column_name` — 存储指标名称(Graphite 指标)的列名。默认值:`Path`。 -#### `time_column_name` +#### `time_column_name` {#time_column_name} `time_column_name` — 存储该指标采集时间的列名。默认值:`Time`。 -#### `value_column_name` +#### `value_column_name` {#value_column_name} `value_column_name` — 存储在 `time_column_name` 中指定时间点的指标值的列名。默认值:`Value`。 -#### `version_column_name` +#### `version_column_name` {#version_column_name} `version_column_name` — 存储指标版本的列名。默认值:`Timestamp`。 -### 模式(Patterns) +### 模式(Patterns) {#patterns} `patterns` 部分的结构: @@ -163,8 +158,7 @@ default * `precision` – 以秒为单位定义数据年龄的精度。应当是 86400(一天的秒数)的约数。 * `function` – 要应用于其年龄落在 `[age, age + precision]` 区间内数据的聚合函数名称。可用函数:min / max / any / avg。平均值的计算是近似的,即“平均的平均值”。 -### 没有规则类型的配置示例 - +### 没有规则类型的配置示例 {#configuration-example} ```xml @@ -199,7 +193,7 @@ default ``` -### 不同规则类型的配置示例 +### 不同规则类型的配置示例 {#configuration-typed-example} ```xml diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md index 12217788d1f..e5cc8cbc4c4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md @@ -7,9 +7,7 @@ title: 'MergeTree 引擎系列' doc_type: 'reference' --- - - -# MergeTree 引擎家族 +# MergeTree 引擎家族 {#mergetree-engine-family} MergeTree 家族中的表引擎是 ClickHouse 数据存储能力的核心。它们提供了实现高可靠性和高性能数据检索的大部分特性:列式存储、自定义分区、稀疏主索引、二级数据跳过索引等。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md index 32b7e5db450..89c513dbddc 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md @@ -10,7 +10,7 @@ doc_type: 'reference' import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; -# 使用文本索引进行全文搜索 +# 使用文本索引进行全文搜索 {#full-text-search-using-text-indexes} @@ -22,7 +22,7 @@ ClickHouse 中的文本索引(也称为["倒排索引"](https://en.wikipedia.o -## 创建文本索引 +## 创建文本索引 {#creating-a-text-index} 要创建文本索引,首先启用相应的实验性设置: @@ -186,12 +186,12 @@ ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); ``` -## 使用文本索引 +## 使用文本索引 {#using-a-text-index} 在 SELECT 查询中使用文本索引非常简单,常见的字符串搜索函数会自动使用该索引。 如果不存在索引,下面这些字符串搜索函数将会回退到较慢的暴力扫描。 -### 支持的函数 +### 支持的函数 {#functions-support} 当在 SELECT 查询的 `WHERE` 子句中使用文本函数时,即可使用文本索引: @@ -201,7 +201,7 @@ FROM [...] WHERE string_search_function(column_with_text_index) ``` -#### `=` 和 `!=` +#### `=` 和 `!=` {#functions-example-equals-notequals} `=`([`equals`](/sql-reference/functions/comparison-functions.md/#equals))和 `!=`([`notEquals`](/sql-reference/functions/comparison-functions.md/#notEquals))会匹配给定搜索词的整体。 @@ -213,7 +213,7 @@ SELECT * from tab WHERE str = 'Hello'; 文本索引支持 `=` 和 `!=` 运算,但只有在使用 `array` 分词器时,等值和不等值搜索才有意义(它会使索引存储整行的值)。 -#### `IN` 和 `NOT IN` +#### `IN` 和 `NOT IN` {#functions-example-in-notin} `IN`([in](/sql-reference/functions/in-functions))和 `NOT IN`([notIn](/sql-reference/functions/in-functions))类似于函数 `equals` 和 `notEquals`,但它们分别要求匹配所有搜索词(`IN`)或完全与所有搜索词都不匹配(`NOT IN`)。 @@ -225,7 +225,7 @@ SELECT * from tab WHERE str IN ('Hello', 'World'); 适用与 `=` 和 `!=` 相同的限制,也就是说,只有在配合 `array` tokenizer 使用时,`IN` 和 `NOT IN` 才有意义。 -#### `LIKE`、`NOT LIKE` 和 `match` +#### `LIKE`、`NOT LIKE` 和 `match` {#functions-example-like-notlike-match} :::note 这些函数目前仅在索引 tokenizer 为 `splitByNonAlpha` 或 `ngrams` 时才会使用文本索引进行过滤。 @@ -250,7 +250,7 @@ SELECT count() FROM tab WHERE comment LIKE ' support %'; -- 或者 `% support %` `support` 左右的空格确保该术语可以被提取为一个 token。 -#### `startsWith` 和 `endsWith` +#### `startsWith` 和 `endsWith` {#functions-example-startswith-endswith} 与 `LIKE` 类似,[startsWith](/sql-reference/functions/string-functions.md/#startsWith) 和 [endsWith](/sql-reference/functions/string-functions.md/#endsWith) 函数只有在能从搜索词中提取出完整的 token 时,才能使用文本索引。 @@ -275,7 +275,7 @@ startsWith(comment, 'clickhouse supports ')` SELECT count() FROM tab WHERE endsWith(comment, ' olap 引擎'); ``` -#### `hasToken` 和 `hasTokenOrNull` +#### `hasToken` 和 `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} 函数 [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) 和 [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) 用于匹配单个指定的 token。 @@ -289,7 +289,7 @@ SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); 函数 `hasToken` 和 `hasTokenOrNull` 是在与 `text` 索引配合使用时性能最优的函数。 -#### `hasAnyTokens` 和 `hasAllTokens` +#### `hasAnyTokens` 和 `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} 函数 [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) 和 [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) 分别用于匹配给定 token 中的任意一个或全部。 @@ -309,7 +309,7 @@ SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); ``` -#### `has` +#### `has` {#functions-example-has} 数组函数 [has](/sql-reference/functions/array-functions#has) 用于判断字符串数组中是否包含某个单独的元素。 @@ -319,7 +319,7 @@ SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); SELECT count() FROM tab WHERE has(array, 'clickhouse'); ``` -#### `mapContains` +#### `mapContains` {#functions-example-mapcontains} 函数 [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains)(`mapContainsKey` 的别名)用于在 map 的键中匹配单个 token。 @@ -331,7 +331,7 @@ SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); ``` -#### `operator[]` +#### `operator[]` {#functions-example-access-operator} 可以将访问运算符 [operator[]](/sql-reference/operators#access-operators) 与文本索引配合使用,以过滤键和值。 @@ -343,9 +343,9 @@ SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; 请参阅以下示例,了解如何配合文本索引使用 `Array(T)` 和 `Map(K, V)` 类型的列。 -### 带有文本索引的 `Array` 和 `Map` 列示例 +### 带有文本索引的 `Array` 和 `Map` 列示例 {#text-index-array-and-map-examples} -#### 为 Array(String) 列建立索引 +#### 为 Array(String) 列建立索引 {#text-index-example-array} 想象一个博客平台,作者会使用关键词为他们的博文打标签并进行分类。 我们希望用户能够通过搜索或点击主题来发现相关内容。 @@ -377,7 +377,7 @@ ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitBy ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- 务必为现有数据重建索引 ``` -#### 为 Map 列建立索引 +#### 为 Map 列建立索引 {#text-index-example-map} 在许多可观测性场景中,日志消息会被拆分为「组件」,并按合适的数据类型存储,例如时间戳使用 DateTime 类型、日志级别使用 Enum 类型等。 指标字段最好以键值对的形式存储。 @@ -435,9 +435,9 @@ SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- 快速 ``` -## 性能调优 +## 性能调优 {#performance-tuning} -### 直接读取 +### 直接读取 {#direct-read} 某些类型的文本查询可以通过一种称为“直接读取”(direct read)的优化显著提速。 更具体地说,当 SELECT 查询中 *没有* 从文本列进行投影时,可以应用此优化。 @@ -515,7 +515,7 @@ Positions: EXPLAIN PLAN 的第二个输出结果中包含一个虚拟列 `__text_index___`。 如果存在该列,则会使用直接读取。 -### 缓存 +### 缓存 {#caching} 可以使用不同的缓存将文本索引的部分内容缓存在内存中(参见[实现细节](#implementation)部分): 目前,对文本索引的反序列化字典块、头信息和倒排列表都提供了缓存,以减少 I/O。 @@ -524,7 +524,7 @@ EXPLAIN PLAN 的第二个输出结果中包含一个虚拟列 `__text_index_ + 已弃用的建表方法 -已弃用的建表方法 + :::note + 不要在新项目中使用此方法。如有可能,请将旧项目切换到上面描述的方法。 + ::: -:::note -请勿在新项目中使用此方法。如果可能,请将旧项目切换到上述方法。 -::: + ```sql + CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] + ( + name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], + name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], + ... + ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) + ``` -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) -``` + **MergeTree() 参数** -**MergeTree() 参数** + * `date-column` — [Date](/sql-reference/data-types/date.md) 类型的列名。ClickHouse 会基于该列按月自动创建分区。分区名称采用 `"YYYYMM"` 格式。 + * `sampling_expression` — 用于采样的表达式。 + * `(primary, key)` — 主键。类型:[Tuple()](/sql-reference/data-types/tuple.md) + * `index_granularity` — 索引粒度,即索引“marks”之间的数据行数。8192 这一数值适用于大多数任务。 -- `date-column` — [Date](/sql-reference/data-types/date.md) 类型列的名称。ClickHouse 会根据此列自动按月创建分区。分区名称采用 `"YYYYMM"` 格式。 -- `sampling_expression` — 采样表达式。 -- `(primary, key)` — 主键。类型:[Tuple()](/sql-reference/data-types/tuple.md) -- `index_granularity` — 索引粒度。索引"标记"之间的数据行数。值 8192 适用于大多数任务。 + **示例** -**示例** - -```sql -MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) -``` - -`MergeTree` 引擎的配置方式与上述主引擎配置方法示例中的方式相同。 + ```sql + MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) + ``` + `MergeTree` 引擎的配置方式与上面主要引擎配置方法中的示例相同。 - ## 数据存储 {#mergetree-data-storage} -表由按主键排序的数据部分(data part)组成。 - -当数据插入表中时,会创建独立的数据部分,每个部分都按主键进行字典序排序。例如,如果主键是 `(CounterID, Date)`,则部分中的数据按 `CounterID` 排序,在每个 `CounterID` 内,再按 `Date` 排序。 +一张表由按主键排序的数据部分(data parts)组成。 -属于不同分区的数据被分离到不同的部分中。ClickHouse 会在后台合并数据部分以实现更高效的存储。属于不同分区的部分不会被合并。合并机制不保证具有相同主键的所有行都位于同一个数据部分中。 +当向表中插入数据时,会创建独立的数据部分,每个数据部分都会按主键进行字典序排序。比如,如果主键是 `(CounterID, Date)`,那么该数据部分中的数据首先按 `CounterID` 排序,并且在每个 `CounterID` 内部再按 `Date` 排序。 -数据部分可以以 `Wide` 或 `Compact` 格式存储。在 `Wide` 格式中,每列存储在文件系统中的单独文件中;在 `Compact` 格式中,所有列存储在一个文件中。`Compact` 格式可用于提高小规模频繁插入的性能。 +属于不同分区的数据会被存放到不同的数据部分中。在后台,ClickHouse 会合并数据部分以实现更高效的存储。属于不同分区的数据部分不会被合并。合并机制并不保证具有相同主键的所有行都会落在同一个数据部分中。 -数据存储格式由表引擎的 `min_bytes_for_wide_part` 和 `min_rows_for_wide_part` 设置控制。如果数据部分中的字节数或行数小于相应设置的值,则该部分以 `Compact` 格式存储;否则以 `Wide` 格式存储。如果未设置这些参数,则数据部分以 `Wide` 格式存储。 +数据部分可以以 `Wide` 或 `Compact` 格式存储。在 `Wide` 格式下,每一列都作为单独的文件存储在文件系统中;在 `Compact` 格式下,所有列都存储在同一个文件中。`Compact` 格式可用于提升小批量且频繁插入场景下的性能。 -每个数据部分在逻辑上被划分为多个颗粒(granule)。颗粒是 ClickHouse 在查询数据时读取的最小不可分割数据集。ClickHouse 不会拆分行或值,因此每个颗粒始终包含整数个行。颗粒的第一行用该行的主键值进行标记。对于每个数据部分,ClickHouse 会创建一个索引文件来存储这些标记(mark)。对于每一列,无论是否在主键中,ClickHouse 也会存储相同的标记。这些标记使您能够直接在列文件中定位数据。 +数据存储格式由表引擎的 `min_bytes_for_wide_part` 和 `min_rows_for_wide_part` 设置控制。如果某个数据部分中的字节数或行数小于对应设置的值,则该数据部分会以 `Compact` 格式存储;否则将以 `Wide` 格式存储。如果这两个设置都未配置,数据部分将以 `Wide` 格式存储。 -颗粒大小受表引擎的 `index_granularity` 和 `index_granularity_bytes` 设置限制。颗粒中的行数位于 `[1, index_granularity]` 范围内,具体取决于行的大小。如果单行的大小大于该设置的值,则颗粒的大小可以超过 `index_granularity_bytes`。在这种情况下,颗粒的大小等于该行的大小。 +每个数据部分在逻辑上被划分为多个粒度(granule)。粒度是 ClickHouse 在查询数据时读取的最小不可再分的数据集。ClickHouse 不会拆分行或单个值,因此每个粒度始终包含整数数量的行。粒度的第一行会用该行的主键值进行标记。对于每个数据部分,ClickHouse 会创建一个索引文件来存储这些标记(marks)。对于每一列(无论是否包含在主键中),ClickHouse 也会存储相同的标记。这些标记可以让系统直接在列文件中定位数据。 +粒度大小受表引擎的 `index_granularity` 和 `index_granularity_bytes` 设置限制。每个粒度中的行数位于 `[1, index_granularity]` 范围内,具体取决于每行数据的大小。如果单行数据的大小超过 `index_granularity_bytes` 的值,则粒度的大小可以超过 `index_granularity_bytes`。在这种情况下,粒度大小等于该行数据的大小。 ## 查询中的主键和索引 {#primary-keys-and-indexes-in-queries} -以 `(CounterID, Date)` 主键为例。在这种情况下,排序和索引可以如下图所示: +以 `(CounterID, Date)` 主键为例。在这种情况下,排序和索引可以表示如下: ```text -完整数据: [---------------------------------------------] +全部数据: [---------------------------------------------] CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] -标记: | | | | | | | | | | | +标记点: | | | | | | | | | | | a,1 a,2 a,3 b,3 e,2 e,3 g,1 h,2 i,1 i,3 l,3 -标记编号: 0 1 2 3 4 5 6 7 8 9 10 +标记点编号: 0 1 2 3 4 5 6 7 8 9 10 ``` -如果数据查询指定: +如果数据查询包含以下条件: -- `CounterID in ('a', 'h')`,服务器读取标记范围 `[0, 3)` 和 `[6, 8)` 中的数据。 -- `CounterID IN ('a', 'h') AND Date = 3`,服务器读取标记范围 `[1, 3)` 和 `[7, 8)` 中的数据。 -- `Date = 3`,服务器读取标记范围 `[1, 10]` 中的数据。 +* `CounterID in ('a', 'h')`,服务器会读取标记区间 `[0, 3)` 和 `[6, 8)` 内的数据。 +* `CounterID IN ('a', 'h') AND Date = 3`,服务器会读取标记区间 `[1, 3)` 和 `[7, 8)` 内的数据。 +* `Date = 3`,服务器会读取标记区间 `[1, 10]` 内的数据。 -上述示例表明,使用索引总是比全表扫描更高效。 +上面的示例表明,使用索引总是比全表扫描更高效。 -稀疏索引允许读取额外的数据。当读取主键的单个范围时,每个数据块中最多可以读取 `index_granularity * 2` 个额外的行。 +稀疏索引会多读一些额外数据。在读取一个主键范围时,每个数据块中最多会额外读取 `index_granularity * 2` 行。 -稀疏索引允许您处理非常大量的表行,因为在大多数情况下,这些索引可以完全放入计算机的 RAM 中。 +稀疏索引允许你处理行数非常巨大的表,因为在大多数情况下,这类索引可以完全放入计算机内存中。 -ClickHouse 不要求主键唯一。您可以插入具有相同主键的多行数据。 +ClickHouse 不要求主键唯一。你可以插入多行具有相同主键的记录。 -您可以在 `PRIMARY KEY` 和 `ORDER BY` 子句中使用 `Nullable` 类型的表达式,但强烈不建议这样做。要启用此功能,请开启 [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 设置。在 `ORDER BY` 子句中,`NULL` 值遵循 [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) 原则。 +你可以在 `PRIMARY KEY` 和 `ORDER BY` 子句中使用 `Nullable` 类型的表达式,但强烈不建议这样做。要启用此功能,请开启 [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 设置。对于 `ORDER BY` 子句中的 `NULL` 值,适用 [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) 原则。 ### 选择主键 {#selecting-a-primary-key} -主键中的列数没有明确限制。根据数据结构,您可以在主键中包含更多或更少的列。这可能会: +主键中的列数没有显式限制。可以根据数据结构,在主键中包含更多或更少的列。这可能会: -- 提高索引性能。 +* 提高索引性能。 - 如果主键是 `(a, b)`,那么在满足以下条件时,添加另一列 `c` 将提高性能: - - 存在对列 `c` 有条件的查询。 - - 具有相同 `(a, b)` 值的长数据范围(比 `index_granularity` 长几倍)很常见。换句话说,当添加另一列可以让您跳过相当长的数据范围时。 + 如果主键是 `(a, b)`,那么在满足以下条件时,添加另一列 `c` 会提高性能: -- 提高数据压缩率。 + * 存在带有列 `c` 条件的查询。 + * 通常会出现较长的数据范围(长度是 `index_granularity` 的数倍)在 `(a, b)` 上具有相同的值。换句话说,添加另一列可以使系统跳过相当长的数据范围。 - ClickHouse 按主键对数据进行排序,因此一致性越高,压缩效果越好。 +* 改善数据压缩。 -- 在 [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) 和 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 引擎中合并数据部分时提供额外的逻辑。 + ClickHouse 会按主键对数据进行排序,因此数据按主键越集中、有序,压缩效果越好。 - 在这种情况下,指定与主键不同的_排序键_是有意义的。 +* 在 [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) 和 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 引擎中,为合并数据部分提供额外的逻辑。 -过长的主键会对插入性能和内存消耗产生负面影响,但主键中的额外列不会影响 ClickHouse 在 `SELECT` 查询时的性能。 + 在这种情况下,指定与主键不同的*排序键(sorting key)*是有意义的。 -您可以使用 `ORDER BY tuple()` 语法创建没有主键的表。在这种情况下,ClickHouse 按插入顺序存储数据。如果您希望在通过 `INSERT ... SELECT` 查询插入数据时保持数据顺序,请设置 [max_insert_threads = 1](/operations/settings/settings#max_insert_threads)。 +较长的主键会对插入性能和内存消耗产生负面影响,但在执行 `SELECT` 查询时,主键中的额外列不会影响 ClickHouse 的性能。 -要按初始顺序选择数据,请使用[单线程](/operations/settings/settings.md/#max_threads) `SELECT` 查询。 +可以使用 `ORDER BY tuple()` 语法创建没有主键的表。在这种情况下,ClickHouse 按插入顺序存储数据。如果希望在使用 `INSERT ... SELECT` 查询插入数据时保持数据顺序,请将 [max_insert_threads = 1](/operations/settings/settings#max_insert_threads) 设置为 1。 -### 选择与排序键不同的主键 {#choosing-a-primary-key-that-differs-from-the-sorting-key} +要按初始顺序选择数据,请使用[单线程](/operations/settings/settings.md/#max_threads)的 `SELECT` 查询。 +### 选择与排序键不同的主键 {#choosing-a-primary-key-that-differs-from-the-sorting-key} -可以指定与排序键(用于对数据部分中的行进行排序的表达式)不同的主键(其值会写入索引文件中每个标记的表达式)。在这种情况下,主键表达式元组必须是排序键表达式元组的前缀。 +可以指定一个与排序键不同的主键(一个表达式,其值会在每个标记的索引文件中写入)。在这种情况下,主键表达式元组必须是排序键表达式元组的前缀。 -此功能在使用 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 和 -[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md) 表引擎时非常有用。在使用这些引擎的常见场景中,表包含两种类型的列:_维度列_ 和 _度量列_。典型的查询会使用任意 `GROUP BY` 对度量列的值进行聚合,并按维度进行过滤。由于 SummingMergeTree 和 AggregatingMergeTree 会聚合具有相同排序键值的行,因此自然会将所有维度添加到排序键中。这样一来,键表达式就由一长串列组成,并且必须频繁更新此列表以包含新添加的维度。 +在使用 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 和 +[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md) 表引擎时,这一特性非常有用。在这些引擎的常见使用场景中,表通常有两类列:*维度(dimensions)* 和 *度量(measures)*。典型查询会对度量列的值在任意 `GROUP BY` 条件下进行聚合,并按维度进行过滤。由于 SummingMergeTree 和 AggregatingMergeTree 会对具有相同排序键值的行进行聚合,因此将所有维度都加入排序键是很自然的做法。结果是,键表达式会由一个很长的列列表组成,并且在新增维度时必须频繁更新该列表。 -在这种情况下,合理的做法是在主键中仅保留少数几列以提供高效的范围扫描,并将其余维度列添加到排序键元组中。 +在这种情况下,更合理的做法是只在主键中保留少数几列,以保证高效的范围扫描,并将其余维度列加入排序键元组中。 -排序键的 [ALTER](/sql-reference/statements/alter/index.md) 是一个轻量级操作,因为当新列同时添加到表和排序键时,现有数据部分无需更改。由于旧排序键是新排序键的前缀,并且新添加的列中没有数据,因此在表修改时,数据同时按旧排序键和新排序键排序。 +对排序键执行 [ALTER](/sql-reference/statements/alter/index.md) 是一项轻量级操作,因为当新列同时被添加到表和排序键中时,现有数据部分不需要被修改。由于旧排序键是新排序键的前缀,并且在新添加的列中还没有数据,因此在进行表修改时,数据在逻辑上同时满足按旧排序键和新排序键排序。 ### 在查询中使用索引和分区 {#use-of-indexes-and-partitions-in-queries} -对于 `SELECT` 查询,ClickHouse 会分析是否可以使用索引。如果 `WHERE/PREWHERE` 子句包含表示相等或不等比较操作的表达式(作为合取元素之一或整体),或者在主键或分区键中的列或表达式上包含带固定前缀的 `IN` 或 `LIKE`,或者在这些列的某些部分重复函数上,或者这些表达式的逻辑关系上,则可以使用索引。 +对于 `SELECT` 查询,ClickHouse 会分析是否可以使用索引。若 `WHERE/PREWHERE` 子句中包含(作为某个合取项或整体)表示等值或不等比较运算的表达式,或者在主键或分区键中的列或表达式,或这些列上的某些特定函数,或这些表达式的逻辑组合上使用了带固定前缀的 `IN` 或 `LIKE`,则可以使用索引。 -因此,可以在主键的一个或多个范围上快速运行查询。在此示例中,针对特定跟踪标签、特定标签和日期范围、特定标签和日期、具有日期范围的多个标签等运行查询时,查询速度会很快。 +因此,可以对主键的一个或多个范围快速执行查询。在此示例中,当针对特定的跟踪标签、特定标签与日期范围、特定标签与日期、带日期范围的多个标签等进行查询时,查询都会很快。 -让我们看一下如下配置的引擎: +来看一个如下配置的引擎: ```sql ENGINE MergeTree() @@ -259,7 +251,7 @@ ORDER BY (CounterID, EventDate) SETTINGS index_granularity=8192 ``` -在这种情况下,在以下查询中: +在这种情况下,在查询时: ```sql SELECT count() FROM table @@ -277,42 +269,41 @@ AND CounterID IN (101500, 731962, 160656) AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) ``` -ClickHouse 将使用主键索引来修剪不相关的数据,并使用月度分区键来修剪不在适当日期范围内的分区。 +ClickHouse 将使用主键索引来跳过不符合条件的数据,并使用按月分区键来跳过处于不符合日期范围内的分区。 -上述查询表明,即使对于复杂表达式也会使用索引。从表中读取数据的组织方式确保使用索引不会比全表扫描慢。 +上面的查询展示了,即使是复杂表达式也会使用索引。表的数据读取经过组织,保证使用索引不会比全表扫描更慢。 -在下面的示例中,无法使用索引。 +在下面的示例中,将无法利用索引。 ```sql SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' ``` -要检查 ClickHouse 在运行查询时是否可以使用索引,请使用设置 [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) 和 [force_primary_key](/operations/settings/settings#force_primary_key)。 +要检查 ClickHouse 在运行查询时是否可以使用索引,请使用设置项 [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) 和 [force_primary_key](/operations/settings/settings#force_primary_key)。 -按月分区的键允许仅读取包含适当范围内日期的数据块。在这种情况下,数据块可能包含多个日期的数据(最多整个月)。在块内,数据按主键排序,而主键可能不包含日期作为第一列。因此,使用仅包含日期条件且未指定主键前缀的查询将导致读取的数据多于单个日期所需的数据。 +按月分区的分区键可以使查询仅读取包含目标日期范围的数据块。在这种情况下,一个数据块可能包含多个日期的数据(最多可覆盖整个月)。在一个数据块内,数据按主键排序,而主键的首列不一定是日期。正因为如此,如果查询中只包含日期条件而未指定主键前缀,就会为获取某个单一日期而读取比实际需要更多的数据。 ### 对部分单调主键使用索引 {#use-of-index-for-partially-monotonic-primary-keys} +以月份中的日期为例。在一个月内,它们构成一个[单调序列](https://en.wikipedia.org/wiki/Monotonic_function),但在更长的时间范围内则不是单调的。这就是一个部分单调序列。如果用户使用部分单调的主键创建表,ClickHouse 会像往常一样创建稀疏索引。当用户从这种类型的表中查询数据时,ClickHouse 会分析查询条件。如果用户希望获取索引中两个标记点之间的数据,并且这两个标记点都落在同一个月内,ClickHouse 就可以在这种特定情况下使用索引,因为它可以计算查询参数与索引标记之间的距离。 -例如,考虑月份中的日期。它们在一个月内形成[单调序列](https://en.wikipedia.org/wiki/Monotonic_function),但在更长的时间段内则不是单调的。这是一个部分单调序列。如果用户使用部分单调主键创建表,ClickHouse 会像往常一样创建稀疏索引。当用户从这种表中查询数据时,ClickHouse 会分析查询条件。如果用户想要获取索引的两个标记之间的数据,并且这两个标记都在同一个月内,ClickHouse 可以在这种特定情况下使用索引,因为它可以计算查询参数与索引标记之间的距离。 - -如果查询参数范围内的主键值不构成单调序列,ClickHouse 无法使用索引。在这种情况下,ClickHouse 会使用全表扫描方法。 +如果查询参数范围内的主键值不构成单调序列,ClickHouse 无法使用索引。在这种情况下,ClickHouse 会使用全表扫描方法。 -ClickHouse 不仅对月份日期序列使用此逻辑,对任何表示部分单调序列的主键都使用此逻辑。 +ClickHouse 不仅对月份日期序列使用这一逻辑,也会对任何表示部分单调序列的主键使用这一逻辑。 -### 数据跳过索引 {#table_engine-mergetree-data_skipping-indexes} +### 数据跳过索引 {#table_engine-mergetree-data_skipping-indexes} -索引声明位于 `CREATE` 查询的列部分。 +索引声明在 `CREATE` 查询的 `columns` 部分中。 ```sql INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] ``` -对于 `*MergeTree` 系列的表,可以指定数据跳过索引。 +对于 `*MergeTree` 家族的表,可以指定数据跳过索引。 -这些索引在由 `granularity_value` 个颗粒组成的块上聚合指定表达式的某些信息(颗粒的大小使用表引擎中的 `index_granularity` 设置指定)。然后在 `SELECT` 查询中使用这些聚合信息,通过跳过 `where` 查询条件无法满足的大数据块来减少从磁盘读取的数据量。 +这些索引会在由 `granularity_value` 个粒度组成的数据块上聚合指定表达式的一些信息(粒度的大小通过表引擎中的 `index_granularity` 设置指定)。随后,这些聚合结果会在 `SELECT` 查询中用于减少从磁盘读取的数据量,通过跳过那些不可能满足 `where` 查询条件的大数据块来实现。 -`GRANULARITY` 子句可以省略,`granularity_value` 的默认值为 1。 +可以省略 `GRANULARITY` 子句,此时 `granularity_value` 的默认值为 1。 **示例** @@ -330,7 +321,7 @@ CREATE TABLE table_name ... ``` -ClickHouse 可以使用示例中的索引来减少以下查询中从磁盘读取的数据量: +示例中的索引可供 ClickHouse 在以下查询中使用,以减少从磁盘读取的数据量: ```sql SELECT count() FROM table WHERE u64 == 10; @@ -338,106 +329,105 @@ SELECT count() FROM table WHERE u64 * i32 >= 1234 SELECT count() FROM table WHERE u64 * length(s) == 1234 ``` -数据跳过索引也可以在复合列上创建: +数据跳过索引也可以建立在复合列上: ```sql --- 在 Map 类型的列上: +-- 在 Map 类型的列上: INDEX map_key_index mapKeys(map_column) TYPE bloom_filter INDEX map_value_index mapValues(map_column) TYPE bloom_filter --- 在 Tuple 类型的列上: +-- 在 Tuple 类型的列上: INDEX tuple_1_index tuple_column.1 TYPE bloom_filter INDEX tuple_2_index tuple_column.2 TYPE bloom_filter --- 在 Nested 类型的列上: +-- 在 Nested 类型的列上: INDEX nested_1_index col.nested_col1 TYPE bloom_filter INDEX nested_2_index col.nested_col2 TYPE bloom_filter ``` ### 跳过索引类型 {#skip-index-types} -`MergeTree` 表引擎支持以下类型的跳过索引。 -有关如何使用跳过索引进行性能优化的更多信息, -请参阅["理解 ClickHouse 数据跳过索引"](/optimize/skipping-indexes)。 +`MergeTree` 表引擎支持以下几种跳过索引类型。\ +有关如何使用跳过索引进行性能优化的更多信息,\ +请参阅[《理解 ClickHouse 数据跳过索引》](/optimize/skipping-indexes)。 -- [`MinMax`](#minmax) 索引 -- [`Set`](#set) 索引 -- [`bloom_filter`](#bloom-filter) 索引 -- [`ngrambf_v1`](#n-gram-bloom-filter) 索引 -- [`tokenbf_v1`](#token-bloom-filter) 索引 +* [`MinMax`](#minmax) 索引 +* [`Set`](#set) 索引 +* [`bloom_filter`](#bloom-filter) 索引 +* [`ngrambf_v1`](#n-gram-bloom-filter) 索引 +* [`tokenbf_v1`](#token-bloom-filter) 索引 #### MinMax 跳过索引 {#minmax} -对于每个索引颗粒,存储表达式的最小值和最大值。 -(如果表达式是 `tuple` 类型,则存储每个元组元素的最小值和最大值。) +对于每个索引粒度,会存储某个表达式的最小值和最大值。 +(如果表达式的类型是 `tuple`,则会为元组中的每个元素分别存储最小值和最大值。) -```text title="语法" +```text title="Syntax" minmax ``` -#### Set 索引 {#set} +#### Set {#set} -对于每个索引颗粒,最多存储指定表达式的 `max_rows` 个唯一值。 -`max_rows = 0` 表示"存储所有唯一值"。 +对于每个索引粒度,最多会存储 `max_rows` 个指定表达式的唯一值。 +`max_rows = 0` 表示“存储所有唯一值”。 -```text title="语法" +```text title="Syntax" set(max_rows) ``` -#### Bloom filter 索引 {#bloom-filter} +#### 布隆过滤器 {#bloom-filter} -对于每个索引颗粒,为指定的列存储一个 [bloom filter](https://en.wikipedia.org/wiki/Bloom_filter)。 +对于每个索引粒度,都会为指定列存储一个[布隆过滤器](https://en.wikipedia.org/wiki/Bloom_filter)。 -```text title="语法" +```text title="Syntax" bloom_filter([false_positive_rate]) ``` -`false_positive_rate` 参数可以取 0 到 1 之间的值(默认值:`0.025`),用于指定产生假阳性的概率(这会增加需要读取的数据量)。 - - -支持以下数据类型: - -- `(U)Int*` -- `Float*` -- `Enum` -- `Date` -- `DateTime` -- `String` -- `FixedString` -- `Array` -- `LowCardinality` -- `Nullable` -- `UUID` -- `Map` - -:::note Map 数据类型:使用键或值指定索引创建 -对于 `Map` 数据类型,客户端可以使用 [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) 或 [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 函数指定为键或值创建索引。 +`false_positive_rate` 参数可以取 0 到 1 之间的值(默认值:`0.025`),用于指定产生假阳性(false positive)结果的概率(该值越大,需要读取的数据量越多)。 + +支持以下数据类型: + +* `(U)Int*` +* `Float*` +* `Enum` +* `Date` +* `DateTime` +* `String` +* `FixedString` +* `Array` +* `LowCardinality` +* `Nullable` +* `UUID` +* `Map` + +:::note Map 数据类型:使用键或值创建索引 +对于 `Map` 数据类型,客户端可以通过 [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) 或 [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 函数指定索引是针对键还是针对值创建。 ::: #### N-gram 布隆过滤器 {#n-gram-bloom-filter} -为每个索引粒度存储指定列的 [n-grams](https://en.wikipedia.org/wiki/N-gram) 的[布隆过滤器](https://en.wikipedia.org/wiki/Bloom_filter)。 +对于每个索引粒度,会为指定列的 [n-gram](https://en.wikipedia.org/wiki/N-gram) 存储一个 [布隆过滤器](https://en.wikipedia.org/wiki/Bloom_filter)。 -```text title="语法" +```text title="Syntax" ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` -| 参数 | 描述 | -| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `n` | ngram 大小 | -| `size_of_bloom_filter_in_bytes` | 布隆过滤器大小(以字节为单位)。可以使用较大的值,例如 `256` 或 `512`,因为它可以很好地压缩)。 | -| `number_of_hash_functions` | 布隆过滤器中使用的哈希函数数量。 | -| `random_seed` | 布隆过滤器哈希函数的随机种子。 | +| 参数 | 描述 | +| ------------------------------- | ---------------------------------------------------------------- | +| `n` | n-gram 大小 | +| `size_of_bloom_filter_in_bytes` | 布隆过滤器(Bloom filter)的字节大小。此处可以使用较大的值,例如 `256` 或 `512`,因为它可以很好地压缩。 | +| `number_of_hash_functions` | 布隆过滤器中使用的哈希函数数量。 | +| `random_seed` | 布隆过滤器哈希函数使用的随机种子。 | -此索引仅适用于以下数据类型: +此索引仅适用于以下数据类型: -- [`String`](/sql-reference/data-types/string.md) -- [`FixedString`](/sql-reference/data-types/fixedstring.md) -- [`Map`](/sql-reference/data-types/map.md) +* [`String`](/sql-reference/data-types/string.md) +* [`FixedString`](/sql-reference/data-types/fixedstring.md) +* [`Map`](/sql-reference/data-types/map.md) -要估算 `ngrambf_v1` 的参数,可以使用以下[用户定义函数 (UDF)](/sql-reference/statements/create/function.md)。 +要估算 `ngrambf_v1` 的参数,可以使用以下[用户自定义函数(UDF)](/sql-reference/statements/create/function.md)。 -```sql title="ngrambf_v1 的 UDF" +```sql title="UDFs for ngrambf_v1" CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] AS (total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); @@ -455,13 +445,13 @@ AS (number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) ``` -要使用这些函数,需要至少指定两个参数: +要使用这些函数,您至少需要指定两个参数: -- `total_number_of_all_grams` -- `probability_of_false_positives` +* `total_number_of_all_grams` +* `probability_of_false_positives` -例如,粒度中有 `4300` 个 ngram,并且期望误报率低于 `0.0001`。 -然后可以通过执行以下查询来估算其他参数: +例如,在一个 granule 中有 `4300` 个 ngram,并且您预期误报率小于 `0.0001`。 +然后可以通过执行以下查询来估算其余参数: ```sql --- 估算过滤器中的位数 @@ -471,7 +461,7 @@ SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; │ 10304 │ └───────────────────────────────┘ ---- 估算哈希函数数量 +--- 估算哈希函数的数量 SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_hash_functions ┌─number_of_hash_functions─┐ @@ -479,111 +469,104 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_ha └──────────────────────────┘ ``` -当然,也可以使用这些函数来估算其他条件下的参数。 +当然,你也可以使用这些函数来估算其他条件下的参数。 上述函数参考了[此处](https://hur.st/bloomfilter)的布隆过滤器计算器。 -#### Token 布隆过滤器 {#token-bloom-filter} +#### Token bloom filter {#token-bloom-filter} -Token 布隆过滤器与 `ngrambf_v1` 相同,但存储的是 token(由非字母数字字符分隔的序列)而不是 ngram。 +Token bloom filter 与 `ngrambf_v1` 相同,但存储的是 token(由非字母数字字符分隔的序列),而不是 ngram。 -```text title="语法" -tokenbf_v1(size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) +```text title="Syntax" +tokenbf_v1(布隆过滤器字节大小, 哈希函数数量, 随机种子) ``` +#### 稀疏 grams 布隆过滤器 {#sparse-grams-bloom-filter} -#### 稀疏 gram 布隆过滤器 {#sparse-grams-bloom-filter} - -稀疏 gram 布隆过滤器与 `ngrambf_v1` 类似,但使用[稀疏 gram 标记](/sql-reference/functions/string-functions.md/#sparseGrams)代替 ngram。 +稀疏 grams 布隆过滤器与 `ngrambf_v1` 类似,但使用的是[稀疏 grams 标记](/sql-reference/functions/string-functions.md/#sparseGrams)而不是 ngrams。 -```text title="语法" +```text title="Syntax" sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) ``` ### 文本索引 {#text} -支持全文搜索,详情请参阅[此处](invertedindexes.md)。 +支持全文搜索,详情见[这里](invertedindexes.md)。 #### 向量相似度 {#vector-similarity} -支持近似最近邻搜索,详情请参阅[此处](annindexes.md)。 +支持近似最近邻检索,详见[此处](annindexes.md)。 ### 函数支持 {#functions-support} -`WHERE` 子句中的条件包含对列进行操作的函数调用。如果列是索引的一部分,ClickHouse 在执行函数时会尝试使用该索引。ClickHouse 针对索引使用支持不同的函数子集。 - -`set` 类型的索引可被所有函数使用。其他索引类型的支持情况如下: - - -| 函数(运算符)/索引 | 主键 | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | text | sparse_grams | -| ------------------------------------------------------------------------------------------------------------------------- | -- | ------ | -------------- | -------------- | ---------------- | ---- | ---------------- | -| [等于 (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [小于 (`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [大于(`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [lessOrEquals(`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greaterOrEquals(`>=`,大于等于)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✗ | ✔ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✔ | ✗ | -| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | ✗ | - - - -当函数的常量参数小于 ngram 大小时,`ngrambf_v1` 无法用于查询优化。 - -(*) 要使 `hasTokenCaseInsensitive` 和 `hasTokenCaseInsensitiveOrNull` 有效,必须在已转换为小写的数据上创建 `tokenbf_v1` 索引,例如:`INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`。 +`WHERE` 子句中的条件可能包含对作用于列的函数的调用。如果该列是索引的一部分,ClickHouse 会在执行这些函数时尝试使用该索引。ClickHouse 对可用于索引的函数提供了不同的支持子集。 + +类型为 `set` 的索引可被所有函数使用。其他类型的索引支持情况如下: + +| 函数(运算符)/ 索引 | 主键 | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | text | +| ------------------------------------------------------------------------------------------------------------------------- | -- | ------ | -------------- | -------------- | ---------------- | ---------------- | ---- | +| [equals(=,==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | +| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | +| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [小于(`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [大于 (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [大于等于 (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | +| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | +| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | +| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | +| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | +| [hasTokenCaseInsensitive(`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | +| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | +| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | + +对于常量参数小于 ngram 大小的函数,`ngrambf_v1` 不能用于查询优化。 + +(*) 要让 `hasTokenCaseInsensitive` 和 `hasTokenCaseInsensitiveOrNull` 生效,必须在转为小写的数据上创建 `tokenbf_v1` 索引,例如:`INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`。 :::note -Bloom filter 可能产生误报,因此 `ngrambf_v1`、`tokenbf_v1`、`sparse_grams` 和 `bloom_filter` 索引不能用于优化那些期望函数结果为 false 的查询。 +由于布隆过滤器可能产生假阳性匹配,因此在期望函数结果为 false 的查询中,`ngrambf_v1`、`tokenbf_v1`、`sparse_grams` 和 `bloom_filter` 索引不能用于查询优化。 例如: -- 可以被优化: - - `s LIKE '%test%'` - - `NOT s NOT LIKE '%test%'` - - `s = 1` - - `NOT s != 1` - - `startsWith(s, 'test')` -- 不可以被优化: - - `NOT s LIKE '%test%'` - - `s NOT LIKE '%test%'` - - `NOT s = 1` - - `s != 1` - - `NOT startsWith(s, 'test')` -::: - - +* 可以被优化: + * `s LIKE '%test%'` + * `NOT s NOT LIKE '%test%'` + * `s = 1` + * `NOT s != 1` + * `startsWith(s, 'test')` +* 不能被优化: + * `NOT s LIKE '%test%'` + * `s NOT LIKE '%test%'` + * `NOT s = 1` + * `s != 1` + * `NOT startsWith(s, 'test')` + ::: ## 投影 {#projections} -投影类似于[物化视图](/sql-reference/statements/create/view),但在数据分区级别定义。它提供一致性保证,并在查询中自动使用。 +投影类似于[物化视图](/sql-reference/statements/create/view),但定义在数据片段(part)级别。它在提供一致性保证的同时,还能在查询中被自动使用。 :::note -在实现投影时,您还应考虑 [force_optimize_projection](/operations/settings/settings#force_optimize_projection) 设置。 +在使用投影时,你还应考虑 [force_optimize_projection](/operations/settings/settings#force_optimize_projection) 设置。 ::: -带有 [FINAL](/sql-reference/statements/select/from#final-modifier) 修饰符的 `SELECT` 语句不支持投影。 +在带有 [FINAL](/sql-reference/statements/select/from#final-modifier) 修饰符的 `SELECT` 语句中不支持投影。 ### 投影查询 {#projection-query} -投影查询用于定义投影。它隐式地从父表中选择数据。 - +投影查询用于定义一个投影。它会隐式地从父表中选取数据。 **语法** ```sql @@ -594,40 +577,38 @@ SELECT [GROUP BY] [ORDER BY] ### 投影存储 {#projection-storage} -投影存储在数据分区目录内。它类似于索引,但包含一个子目录,用于存储匿名 `MergeTree` 表的数据分区。该表由投影的定义查询生成。如果存在 `GROUP BY` 子句,底层存储引擎将变为 [AggregatingMergeTree](aggregatingmergetree.md),并且所有聚合函数都会转换为 `AggregateFunction`。如果存在 `ORDER BY` 子句,`MergeTree` 表将其用作主键表达式。在合并过程中,投影分区通过其存储的合并例程进行合并。父表分区的校验和与投影分区的校验和合并。其他维护任务与跳数索引类似。 +投影存储在数据分片(part)目录中。它类似于索引,但包含一个子目录,用于存放一个匿名 `MergeTree` 表的分片。该表由投影的定义查询所派生。如果存在 `GROUP BY` 子句,则其底层存储引擎变为 [AggregatingMergeTree](aggregatingmergetree.md),并且所有聚合函数都会被转换为 `AggregateFunction`。如果存在 `ORDER BY` 子句,则该 `MergeTree` 表会将其作为主键表达式使用。在合并过程中,投影分片通过其存储引擎的合并流程进行合并。父表分片的校验和会与投影分片的校验和组合在一起。其他维护任务与跳过索引(skip index)类似。 ### 查询分析 {#projection-query-analysis} -1. 检查投影是否可用于回答给定查询,即它是否生成与查询基表相同的结果。 -2. 选择最佳可行匹配,即需要读取最少颗粒数的匹配。 -3. 使用投影的查询管道将与使用原始分区的查询管道不同。如果某些分区中不存在投影,我们可以添加管道以即时"投影"它。 - +1. 检查投影是否可以用于回答给定查询,即它是否能生成与查询基表相同的结果。 +2. 选择最优的可行匹配方案,即需要读取的数据颗粒(granule)最少的那个。 +3. 使用投影的查询管道将不同于使用原始数据分片的管道。如果某些数据分片中缺少该投影,可以在查询管道中动态增加步骤以“实时投影”出来。 ## 并发数据访问 {#concurrent-data-access} -对于并发表访问,ClickHouse 使用多版本控制机制。换句话说,当表被同时读取和更新时,数据从查询时刻的当前数据分片集合中读取。不存在长时间锁定。插入操作不会阻塞读取操作。 - -表的读取操作会自动并行化。 +对于对表的并发访问,我们使用多版本机制。换句话说,当一个表被同时读取和更新时,查询会从在查询时刻“当前”的那一组数据分片中读取数据。不会出现长时间持有的锁。插入操作不会阻塞读取操作。 +从表中读取会自动并行执行。 -## 列和表的 TTL {#table_engine-mergetree-ttl} +## 列和表的 TTL {#table_engine-mergetree-ttl} -确定数据值的生存时间。 +用于指定数据值的生命周期。 -`TTL` 子句可以为整个表和每个单独的列设置。表级 `TTL` 还可以指定在磁盘和卷之间自动移动数据的逻辑,或对所有数据已过期的部分进行重新压缩。 +可以为整张表以及每个单独的列设置 `TTL` 子句。表级 `TTL` 还可以指定在不同磁盘和卷之间自动迁移数据的逻辑,或者对数据已全部过期的部件进行重新压缩。 -表达式的求值结果必须为 [Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md) 或 [DateTime64](/sql-reference/data-types/datetime64.md) 数据类型。 +表达式的计算结果必须是 [Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md) 或 [DateTime64](/sql-reference/data-types/datetime64.md) 数据类型。 **语法** -为列设置生存时间: +为列设置 TTL(生存时间): ```sql TTL time_column TTL time_column + interval ``` -要定义 `interval`,请使用[时间间隔](/sql-reference/operators#operators-for-working-with-dates-and-times)运算符,例如: +要定义 `interval`,请使用 [时间间隔](/sql-reference/operators#operators-for-working-with-dates-and-times) 运算符,例如: ```sql TTL date_time + INTERVAL 1 MONTH @@ -636,13 +617,13 @@ TTL date_time + INTERVAL 15 HOUR ### 列 TTL {#mergetree-column-ttl} -当列中的值过期时,ClickHouse 会将它们替换为该列数据类型的默认值。如果数据部分中某列的所有值都过期,ClickHouse 会从文件系统的数据部分中删除该列。 +当列中的值过期时,ClickHouse 会将其替换为该列数据类型的默认值。如果某个数据部分中该列的所有值都已过期,ClickHouse 会从文件系统中的该数据部分删除此列。 `TTL` 子句不能用于键列。 **示例** -#### 创建带有 `TTL` 的表: {#creating-a-table-with-ttl} +#### 创建带 `TTL` 的表: {#creating-a-table-with-ttl} ```sql CREATE TABLE tab @@ -657,7 +638,7 @@ PARTITION BY toYYYYMM(d) ORDER BY d; ``` -#### 为现有表的列添加 TTL {#adding-ttl-to-a-column-of-an-existing-table} +#### 向现有表的列添加 TTL {#adding-ttl-to-a-column-of-an-existing-table} ```sql ALTER TABLE tab @@ -665,7 +646,7 @@ ALTER TABLE tab c String TTL d + INTERVAL 1 DAY; ``` -#### 修改列的 TTL {#altering-ttl-of-the-column} +#### 更改列的 TTL {#altering-ttl-of-the-column} ```sql ALTER TABLE tab @@ -675,24 +656,24 @@ ALTER TABLE tab ### 表 TTL {#mergetree-table-ttl} -表可以有一个用于删除过期行的表达式,以及多个用于在[磁盘或卷](#table_engine-mergetree-multiple-volumes)之间自动移动部分的表达式。当表中的行过期时,ClickHouse 会删除所有相应的行。对于部分的移动或重新压缩,该部分的所有行都必须满足 `TTL` 表达式条件。 +表可以定义一个用于删除过期行的表达式,以及多个用于在[磁盘或卷](#table_engine-mergetree-multiple-volumes)之间自动迁移分片的表达式。当表中的行过期时,ClickHouse 会删除所有对应的行。对于分片移动或重新压缩操作,某个分片中的所有行都必须满足 `TTL` 表达式所定义的条件。 ```sql -TTL expr - [DELETE|RECOMPRESS codec_name1|TO DISK 'xxx'|TO VOLUME 'xxx'][, DELETE|RECOMPRESS codec_name2|TO DISK 'aaa'|TO VOLUME 'bbb'] ... - [WHERE conditions] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] +TTL 表达式 + [DELETE|RECOMPRESS 编解码器名称1|TO DISK 'xxx'|TO VOLUME 'xxx'][, DELETE|RECOMPRESS 编解码器名称2|TO DISK 'aaa'|TO VOLUME 'bbb'] ... + [WHERE 条件] + [GROUP BY 键表达式 [SET v1 = 聚合函数(v1) [, v2 = 聚合函数(v2) ...]] ] ``` -每个 TTL 表达式后面可以跟 TTL 规则的类型。它决定了当表达式满足条件(达到当前时间)时要执行的操作: +TTL 规则的类型可以紧跟在每个 TTL 表达式之后。它会影响在表达式满足条件(到达当前时间)时要执行的操作: -- `DELETE` - 删除过期行(默认操作); -- `RECOMPRESS codec_name` - 使用 `codec_name` 重新压缩数据部分; -- `TO DISK 'aaa'` - 将部分移动到磁盘 `aaa`; -- `TO VOLUME 'bbb'` - 将部分移动到卷 `bbb`; -- `GROUP BY` - 聚合过期行。 +* `DELETE` - 删除已过期的行(默认操作); +* `RECOMPRESS codec_name` - 使用 `codec_name` 重新压缩数据分片; +* `TO DISK 'aaa'` - 将分片移动到名为 `aaa` 的磁盘; +* `TO VOLUME 'bbb'` - 将分片移动到名为 `bbb` 的卷; +* `GROUP BY` - 聚合已过期的行。 -`DELETE` 操作可以与 `WHERE` 子句一起使用,根据过滤条件仅删除部分过期行: +`DELETE` 操作可以与 `WHERE` 子句一起使用,根据筛选条件只删除部分已过期的行: ```sql TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' @@ -700,11 +681,11 @@ TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' `GROUP BY` 表达式必须是表主键的前缀。 -如果某列不是 `GROUP BY` 表达式的一部分,且未在 `SET` 子句中显式设置,则结果行中该列将包含分组行中的任意值(就像对其应用了聚合函数 `any`)。 +如果某列既不是 `GROUP BY` 表达式的一部分,又没有在 `SET` 子句中显式设置,那么在结果行中,该列会包含分组行中的任意一个值(就像对该列应用了聚合函数 `any` 一样)。 **示例** -#### 创建带有 `TTL` 的表: {#creating-a-table-with-ttl-1} +#### 创建带有 `TTL` 的表: {#creating-a-table-with-ttl-1} ```sql CREATE TABLE tab @@ -720,15 +701,14 @@ TTL d + INTERVAL 1 MONTH DELETE, d + INTERVAL 2 WEEK TO DISK 'bbb'; ``` -#### 修改表的 `TTL`: {#altering-ttl-of-the-table} +#### 修改表的 `TTL`: {#altering-ttl-of-the-table} ```sql ALTER TABLE tab MODIFY TTL d + INTERVAL 1 DAY; ``` - -创建一个表,其中的行在一个月后过期。日期为星期一的过期行将被删除: +创建一个表,行在一个月后过期。对已过期的行,仅删除日期为星期一的记录: ```sql CREATE TABLE table_with_where @@ -742,7 +722,7 @@ ORDER BY d TTL d + INTERVAL 1 MONTH DELETE WHERE toDayOfWeek(d) = 1; ``` -#### 创建一个表,其中过期的行会被重新压缩: {#creating-a-table-where-expired-rows-are-recompressed} +#### 创建对过期行重新压缩的表: {#creating-a-table-where-expired-rows-are-recompressed} ```sql CREATE TABLE table_for_recompression @@ -757,7 +737,7 @@ TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPR SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; ``` -创建一个表,其中过期的行会被聚合。在结果行中,`x` 包含分组行中的最大值,`y` 包含最小值,`d` 包含分组行中的任意偶然值。 +创建一个用于聚合已过期行的表。最终结果行中,`x` 包含该分组内的最大值,`y` 为最小值,`d` 为该分组中的任意一个值。 ```sql CREATE TABLE table_for_aggregation @@ -775,55 +755,53 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ### 删除过期数据 {#mergetree-removing-expired-data} -当 ClickHouse 合并数据部分时,具有过期 `TTL` 的数据会被删除。 +TTL 已过期的数据会在 ClickHouse 合并数据部分时被删除。 -当 ClickHouse 检测到数据已过期时,它会执行计划外合并。要控制此类合并的频率,可以设置 `merge_with_ttl_timeout`。如果该值设置得过低,将执行大量计划外合并,可能会消耗大量资源。 +当 ClickHouse 检测到数据已过期时,会执行一次非计划合并。要控制此类合并的频率,可以设置 `merge_with_ttl_timeout`。如果该值过低,可能会触发大量非计划合并,消耗大量资源。 -如果在合并之间执行 `SELECT` 查询,可能会获取到过期数据。为避免这种情况,请在 `SELECT` 之前使用 [OPTIMIZE](/sql-reference/statements/optimize.md) 查询。 +在两次合并之间执行 `SELECT` 查询时,可能会读到已过期的数据。为避免这种情况,请在执行 `SELECT` 之前先执行 [OPTIMIZE](/sql-reference/statements/optimize.md) 查询。 **另请参阅** -- [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) 设置 - +* [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) 设置 ## 磁盘类型 {#disk-types} -除本地块设备外,ClickHouse 还支持以下存储类型: +除了本地块设备之外,ClickHouse 还支持以下存储类型: -- [`s3` 用于 S3 和 MinIO](#table_engine-mergetree-s3) -- [`gcs` 用于 GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -- [`blob_storage_disk` 用于 Azure Blob Storage](/operations/storing-data#azure-blob-storage) -- [`hdfs` 用于 HDFS](/engines/table-engines/integrations/hdfs) -- [`web` 用于 Web 只读访问](/operations/storing-data#web-storage) -- [`cache` 用于本地缓存](/operations/storing-data#using-local-cache) -- [`s3_plain` 用于备份到 S3](/operations/backup#backuprestore-using-an-s3-disk) -- [`s3_plain_rewritable` 用于 S3 中的不可变非复制表](/operations/storing-data.md#s3-plain-rewritable-storage) +* [`s3` 用于 S3 和 MinIO](#table_engine-mergetree-s3) +* [`gcs` 用于 GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) +* [`blob_storage_disk` 用于 Azure Blob Storage](/operations/storing-data#azure-blob-storage) +* [`hdfs` 用于 HDFS](/engines/table-engines/integrations/hdfs) +* [`web` 用于从 Web 进行只读访问](/operations/storing-data#web-storage) +* [`cache` 用于本地缓存](/operations/storing-data#using-local-cache) +* [`s3_plain` 用于备份到 S3](/operations/backup#backuprestore-using-an-s3-disk) +* [`s3_plain_rewritable` 用于 S3 中的不可变且非复制的表](/operations/storing-data.md#s3-plain-rewritable-storage) - -## 使用多个块设备存储数据 {#table_engine-mergetree-multiple-volumes} +## 使用多个块设备用于数据存储 {#table_engine-mergetree-multiple-volumes} ### 简介 {#introduction} -`MergeTree` 系列表引擎可以在多个块设备上存储数据。比如,当某个表的数据被隐式划分为“热数据”和“冷数据”时,这一特性就非常有用。最新数据会被频繁访问,但只占用少量空间;相反,长尾的历史数据很少被请求。如果有多块磁盘可用,“热数据”可以放在快速磁盘上(例如 NVMe SSD 或内存中),而“冷数据”则可以放在相对较慢的磁盘上(例如 HDD)。 +`MergeTree` 系列表引擎可以将数据存储在多个块设备上。举例来说,当某个表中的数据被隐式划分为「热数据」和「冷数据」时,这会非常有用。最新的数据会被频繁访问,但只占用很小的空间。相反,具有长尾特征的历史数据则很少被访问。如果有多块磁盘可用,可以将「热数据」放在高速磁盘上(例如 NVMe SSD 或内存中),而将「冷数据」放在相对较慢的磁盘上(例如 HDD)。 -数据部分(data part)是 `MergeTree` 引擎表中最小的可移动单元。属于同一数据部分的数据存储在同一块磁盘上。数据部分既可以根据用户设置在后台在磁盘之间移动,也可以通过 [ALTER](/sql-reference/statements/alter/partition) 查询进行移动。 +数据片段是 `MergeTree` 引擎表中可移动的最小单元。属于同一数据片段的数据存储在同一块磁盘上。数据片段既可以在后台根据用户设置在磁盘之间移动,也可以通过 [ALTER](/sql-reference/statements/alter/partition) 查询进行移动。 ### 术语 {#terms} -- 磁盘(Disk)— 挂载到文件系统的块设备。 -- 默认磁盘(Default disk)— 存储 [path](/operations/server-configuration-parameters/settings.md/#path) 服务器设置中指定路径的磁盘。 -- 卷(Volume)— 一组同类磁盘的有序集合(类似于 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures))。 -- 存储策略(Storage policy)— 一组卷及其之间数据迁移规则的组合。 +* Disk — 挂载到文件系统的块设备。 +* Default disk — 存储 [path](/operations/server-configuration-parameters/settings.md/#path) 服务器设置中所指定路径数据的磁盘。 +* Volume — 由一组相同磁盘按顺序组织而成的集合(类似于 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures))。 +* Storage policy — 卷的集合以及在这些卷之间迁移数据的规则。 -上述实体的名称可以在系统表 [system.storage_policies](/operations/system-tables/storage_policies) 和 [system.disks](/operations/system-tables/disks) 中找到。要为某个表应用已配置的存储策略,请使用 `MergeTree` 系列表引擎的 `storage_policy` 设置。 +这些实体的名称可以在系统表 [system.storage_policies](/operations/system-tables/storage_policies) 和 [system.disks](/operations/system-tables/disks) 中找到。要为某个表应用已配置的存储策略之一,请在 `MergeTree` 引擎族表中使用 `storage_policy` 设置。 -### 配置 {#table_engine-mergetree-multiple-volumes_configure} +### 配置 {#table_engine-mergetree-multiple-volumes_configure} -磁盘、卷和存储策略应在 `config.d` 目录中的文件里,在 `` 标签内进行声明。 +应在 `config.d` 目录下的配置文件中,通过 `` 标签声明磁盘、卷和存储策略。 :::tip -磁盘也可以在查询的 `SETTINGS` 部分中声明。这对于临时分析(ad-hoc)场景非常有用,比如临时挂载某个通过 URL 提供的磁盘。 -参见 [动态存储](/operations/storing-data#dynamic-configuration) 以了解更多细节。 +也可以在查询的 `SETTINGS` 部分声明磁盘。这对于临时分析时挂载磁盘(例如托管在某个 URL 上的磁盘)非常有用。 +更多详情参见[动态存储](/operations/storing-data#dynamic-configuration)。 ::: 配置结构: @@ -831,7 +809,7 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ```xml - + /mnt/fast_ssd/clickhouse/ @@ -852,11 +830,11 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); 标签: -- `` — 磁盘名称。所有磁盘的名称必须各不相同。 -- `path` — 服务器用于存储数据(`data` 和 `shadow` 目录)的路径,应以 “/” 结尾。 -- `keep_free_space_bytes` — 需要预留的磁盘空闲空间大小。 +* `` — 磁盘名称。所有磁盘的名称必须互不相同。 +* `path` — 服务器用于存储数据(`data` 和 `shadow` 目录)的路径,必须以 '/' 结尾。 +* `keep_free_space_bytes` — 需要预留的空闲磁盘空间大小。 -磁盘定义的顺序并不重要。 +磁盘定义的顺序无关紧要。 存储策略配置标记: @@ -872,17 +850,17 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); round_robin - + - + 0.2 - + - + ... @@ -890,20 +868,19 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); 标签: - * `policy_name_N` — 策略名称。策略名称必须唯一。 -* `volume_name_N` — 卷名称。卷名称必须唯一。 +* `volume_name_N` — 卷名。卷名必须唯一。 * `disk` — 卷中的一个磁盘。 -* `max_data_part_size_bytes` — 可以存储在该卷任意磁盘上的数据分片的最大大小。如果预估合并后的分片大小会大于 `max_data_part_size_bytes`,则该分片会被写入下一个卷。该功能基本上允许将新的/较小的分片保存在热点(SSD)卷上,并在它们变大时将其移动到冷存储(HDD)卷。对于仅包含一个卷的策略,不要使用此设置。 -* `move_factor` — 当可用空间比例低于该因子时,数据会自动开始移动到下一个卷(如果存在)(默认值为 0.1)。ClickHouse 会按大小将已有分片从大到小(降序)排序,并选择总大小足以满足 `move_factor` 条件的分片。如果所有分片的总大小仍不足,则会移动所有分片。 -* `perform_ttl_move_on_insert` — 是否在数据分片 INSERT 时执行 TTL 迁移。默认情况下(启用时),如果插入的数据分片已经满足 TTL 迁移规则而过期,则会立即被写入迁移规则中声明的卷/磁盘。如果目标卷/磁盘较慢(例如 S3),这会显著降低插入速度。如果禁用,则已过期的数据分片会先写入默认卷,然后再立即迁移到 TTL 卷。 -* `load_balancing` - 磁盘负载均衡策略,可选值为 `round_robin` 或 `least_used`。 -* `least_used_ttl_ms` - 配置所有磁盘可用空间信息的更新超时时间(毫秒)(`0` - 始终更新,`-1` - 从不更新,默认值为 `60000`)。注意,如果磁盘仅供 ClickHouse 使用,并且不会进行在线文件系统扩/缩容,则可以使用 `-1`;在其他所有情况下不推荐这样做,因为最终会导致空间分配不正确。 -* `prefer_not_to_merge` — 不应使用此设置。禁用该卷上的数据分片合并(这会有害并导致性能下降)。当启用该设置时(不要这么做),不允许在此卷上合并数据(这很糟糕)。这虽然允许(但你并不需要)控制(如果你想控制什么,那就是在犯错)ClickHouse 如何与慢磁盘交互(但 ClickHouse 更清楚,所以请不要使用此设置)。 -* `volume_priority` — 定义卷被填充的优先级(顺序)。数值越小优先级越高。参数值应为自然数,并整体覆盖从 1 到 N(最低优先级)之间的连续区间,中间不能跳过任何数字。 - * 如果 *所有* 卷都被打了标签,则按照给定顺序设定优先级。 - * 如果只有 *部分* 卷被打了标签,则未打标签的卷具有最低优先级,并按照它们在配置中定义的顺序确定优先级。 - * 如果 *没有* 卷被打标签,则其优先级按照它们在配置文件中的声明顺序确定。 +* `max_data_part_size_bytes` — 可以存储在任意卷磁盘上的数据分片的最大大小。如果预计合并后的分片大小会大于 `max_data_part_size_bytes`,那么该分片会被写入下一个卷。基本上,此功能允许将新的/较小的分片保存在“热”(SSD)卷上,并在它们变大时移动到“冷”(HDD)卷。如果你的策略只有一个卷,请不要使用此设置。 +* `move_factor` — 当可用空间比例低于该系数时,如果存在下一个卷,数据会自动开始移动到下一个卷(默认值为 0.1)。ClickHouse 会按从大到小(降序)对现有分片按大小排序,并选择其总大小足以满足 `move_factor` 条件的分片。如果所有分片的总大小仍不足,则会移动所有分片。 +* `perform_ttl_move_on_insert` — 禁用在插入数据分片(data part)时执行 TTL 移动。默认情况下(启用时),如果插入的分片根据 TTL 移动规则已经过期,它会立即被写入移动规则中指定的卷/磁盘。如果目标卷/磁盘较慢(例如 S3),这可能会显著减慢插入速度。如果禁用,则已过期的数据分片会先写入默认卷,然后紧接着再移动到 TTL 卷。 +* `load_balancing` - 磁盘负载均衡策略,`round_robin` 或 `least_used`。 +* `least_used_ttl_ms` - 配置在所有磁盘上更新可用空间的超时时间(毫秒)(`0` - 始终更新,`-1` - 从不更新,默认值为 `60000`)。注意,如果磁盘只能被 ClickHouse 使用,并且不会进行在线文件系统扩容/缩容,则可以使用 `-1`;在其他所有情况下都不推荐使用,因为最终会导致空间分布不正确。 +* `prefer_not_to_merge` — 你不应该使用此设置。禁用在该卷上合并数据分片(这有害并会导致性能下降)。当启用此设置时(不要这样做),不允许在该卷上进行数据合并(这很糟糕)。这允许你(但你并不需要)控制(如果你想控制什么,你就是在犯错)ClickHouse 如何与慢磁盘交互(但 ClickHouse 更了解情况,所以请不要使用此设置)。 +* `volume_priority` — 定义填充卷的优先级(顺序)。数值越小优先级越高。该参数的取值应为自然数,并且整体上从 1 到 N(给出的最低优先级)连续覆盖,中间不能缺少任何数字。 + * 如果 *所有* 卷都打了标签,则按给定顺序确定它们的优先级。 + * 如果只有 *部分* 卷打了标签,则未打标签的卷具有最低优先级,并按它们在配置中的定义顺序确定优先级。 + * 如果 *没有* 卷打标签,则它们的优先级对应于它们在配置中声明的顺序。 * 两个卷不能具有相同的优先级值。 配置示例: @@ -949,16 +926,15 @@ TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); ``` +在给定的示例中,`hdd_in_order` 策略实现了 [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling)(轮询)方式。因此,该策略仅定义了一个卷(`single`),数据 part 会以循环顺序存储在该卷的所有磁盘上。如果系统中挂载了多个相似的磁盘但没有配置 RAID,此类策略会非常有用。请记住,每个单独的磁盘驱动器都不够可靠,您可能需要通过将复制因子设置为 3 或更多来进行补偿。 -在给定的示例中,`hdd_in_order` 策略实现了[轮询](https://en.wikipedia.org/wiki/Round-robin_scheduling)方法。因此,该策略仅定义一个卷(`single`),数据部分按循环顺序存储在其所有磁盘上。如果系统挂载了多个相似的磁盘但未配置 RAID,这种策略会非常有用。请注意,每个单独的磁盘驱动器并不可靠,您可能需要使用 3 或更高的副本因子来补偿这一点。 - -如果系统中有不同类型的磁盘可用,则可以使用 `moving_from_ssd_to_hdd` 策略。卷 `hot` 由一个 SSD 磁盘(`fast_ssd`)组成,可以存储在该卷上的数据部分的最大大小为 1GB。所有大于 1GB 的数据部分将直接存储在 `cold` 卷上,该卷包含一个 HDD 磁盘 `disk1`。 -此外,一旦磁盘 `fast_ssd` 的填充率超过 80%,数据将通过后台进程传输到 `disk1`。 +如果系统中存在不同类型的磁盘,可以使用 `moving_from_ssd_to_hdd` 策略。卷 `hot` 由一个 SSD 磁盘(`fast_ssd`)组成,可以存储在该卷上的单个 part 的最大大小为 1GB。所有大小超过 1GB 的 part 将直接存储在 `cold` 卷上,该卷包含一个 HDD 磁盘 `disk1`。 +另外,一旦磁盘 `fast_ssd` 的使用率超过 80%,后台进程会将数据迁移到 `disk1` 上。 -如果列出的卷中至少有一个没有明确的 `volume_priority` 参数,则存储策略中卷的枚举顺序很重要。 -一旦某个卷被填满,数据将移动到下一个卷。磁盘的枚举顺序同样重要,因为数据会轮流存储在这些磁盘上。 +在存储策略中,卷的列举顺序非常重要,尤其是在列出的卷中至少有一个未显式设置 `volume_priority` 参数时。 +一旦某个卷被写满,数据会被移动到下一个卷。磁盘的列举顺序同样重要,因为数据会依次存储到这些磁盘上。 -创建表时,可以对其应用已配置的存储策略之一: +在创建表时,可以为其应用一个已配置好的存储策略: ```sql CREATE TABLE table_with_non_default_policy ( @@ -972,49 +948,46 @@ PARTITION BY toYYYYMM(EventDate) SETTINGS storage_policy = 'moving_from_ssd_to_hdd' ``` -`default` 存储策略意味着仅使用一个卷,该卷仅包含 `` 中给定的一个磁盘。 -您可以在创建表后使用 [ALTER TABLE ... MODIFY SETTING] 查询更改存储策略,新策略应包含所有具有相同名称的旧磁盘和卷。 - -执行数据部分后台移动的线程数可以通过 [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 设置进行更改。 - -### 详细信息 {#details} - -对于 `MergeTree` 表,数据通过不同方式写入磁盘: +`default` 存储策略意味着只使用一个卷,该卷仅由 `` 中指定的一块磁盘组成。 +你可以在建表之后通过 [ALTER TABLE ... MODIFY SETTING] 查询更改存储策略,新策略必须包含所有原有的磁盘和卷,并使用相同的名称。 -- 作为插入操作的结果(`INSERT` 查询)。 -- 在后台合并和[变更](/sql-reference/statements/alter#mutations)期间。 -- 从另一个副本下载时。 -- 作为分区冻结的结果 [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition)。 +用于在后台移动数据部分的线程数量可以通过 [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 设置进行修改。 -在除变更和分区冻结之外的所有这些情况下,数据部分根据给定的存储策略存储在卷和磁盘上: +### 细节 {#details} -1. 选择第一个(按定义顺序)具有足够磁盘空间来存储数据部分(`unreserved_space > current_part_size`)并允许存储给定大小的数据部分(`max_data_part_size_bytes > current_part_size`)的卷。 -2. 在该卷内,选择紧随用于存储前一个数据块的磁盘之后的磁盘,并且该磁盘的可用空间大于数据部分大小(`unreserved_space - keep_free_space_bytes > current_part_size`)。 +对于 `MergeTree` 表,数据以不同的方式写入磁盘: -在底层,变更和分区冻结使用[硬链接](https://en.wikipedia.org/wiki/Hard_link)。不支持不同磁盘之间的硬链接,因此在这种情况下,生成的数据部分存储在与初始数据部分相同的磁盘上。 +* 作为插入操作(`INSERT` 查询)的结果。 +* 在执行后台合并和[变更](/sql-reference/statements/alter#mutations)时。 +* 从其他副本下载时。 +* 作为分区冻结 [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) 的结果。 -在后台,数据部分根据可用空间量(`move_factor` 参数)按照配置文件中声明卷的顺序在卷之间移动。 -数据永远不会从最后一个卷传输到第一个卷。可以使用系统表 [system.part_log](/operations/system-tables/part_log)(字段 `type = MOVE_PART`)和 [system.parts](/operations/system-tables/parts.md)(字段 `path` 和 `disk`)来监控后台移动。此外,详细信息可以在服务器日志中找到。 +在除变更和分区冻结之外的所有这些情况下,数据部件(part)会根据给定的存储策略被存储在某个卷和磁盘上: -用户可以使用查询 [ALTER TABLE ... MOVE PART\|PARTITION ... TO VOLUME\|DISK ...](/sql-reference/statements/alter/partition) 强制将数据部分或分区从一个卷移动到另一个卷,所有后台操作的限制都会被考虑在内。该查询会自行启动移动操作,不会等待后台操作完成。如果可用空间不足或不满足任何必需条件,用户将收到错误消息。 +1. 选择第一个(按定义顺序)同时满足以下条件的卷:该卷拥有足够的磁盘空间来存储该部件(`unreserved_space > current_part_size`),并且允许存储该大小的部件(`max_data_part_size_bytes > current_part_size`)。 +2. 在该卷中,选择这样的磁盘:在上一次用于存储数据块的磁盘之后的那个磁盘,且其可用空间大于该部件的大小(`unreserved_space - keep_free_space_bytes > current_part_size`)。 -移动数据不会干扰数据复制。因此,可以为不同副本上的同一张表指定不同的存储策略。 +在底层实现中,变更和分区冻结会使用[硬链接](https://en.wikipedia.org/wiki/Hard_link)。不同磁盘之间不支持硬链接,因此在这些情况下,生成的部件会存储在与初始部件相同的磁盘上。 +在后台,部件会基于空闲空间的大小(`move_factor` 参数),按照配置文件中声明卷的顺序在卷之间移动。 +数据永远不会从最后一个卷迁出,也不会被迁移到第一个卷中。可以使用系统表 [system.part_log](/operations/system-tables/part_log)(字段 `type = MOVE_PART`)和 [system.parts](/operations/system-tables/parts.md)(字段 `path` 和 `disk`)来监控后台移动操作。更详细的信息可以在服务器日志中找到。 -在后台合并和变更操作完成后,旧的数据块只有在经过一定时间(`old_parts_lifetime`)后才会被删除。 -在此期间,它们不会被移动到其他卷或磁盘。因此,在这些数据块被最终删除之前,评估已占用磁盘空间时仍会将其计算在内。 +用户可以通过查询 [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition) 强制将某个部件或分区从一个卷移动到另一个卷,所有对后台操作的限制同样会被考虑在内。该查询会自行发起移动操作,而不会等待后台操作完成。如果没有足够的可用空间,或任一必需条件未满足,用户将收到一条错误信息。 -用户可以使用 [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) 设置,将新的大型数据块以均衡的方式分配到 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) 卷的不同磁盘上。 +数据移动不会影响数据复制。因此,可以在不同副本上的同一张表上指定不同的存储策略。 +在后台合并和变更完成后,旧部件只会在经过一定时间(`old_parts_lifetime`)后才会被删除。 +在此期间,它们不会被移动到其他卷或磁盘。因此,在这些部件被最终删除之前,它们仍然会被计入磁盘空间占用情况的计算中。 +用户可以使用 [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) 设置,将新的大型部件在 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) 卷的不同磁盘上以均衡的方式进行分配。 -## 使用外部存储存储数据 {#table_engine-mergetree-s3} +## 使用外部存储进行数据存储 {#table_engine-mergetree-s3} -[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 系列表引擎可以通过类型为 `s3`、`azure_blob_storage`、`hdfs` 的磁盘将数据分别存储到 `S3`、`AzureBlobStorage`、`HDFS`。更多详情请参阅[配置外部存储选项](/operations/storing-data.md/#configuring-external-storage)。 +[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 系列表引擎可以通过类型分别为 `s3`、`azure_blob_storage`、`hdfs` 的磁盘,将数据存储到 `S3`、`AzureBlobStorage`、`HDFS` 中。有关更多详细信息,请参见[配置外部存储选项](/operations/storing-data.md/#configuring-external-storage)。 -以下示例展示了如何使用类型为 `s3` 的磁盘将 [S3](https://aws.amazon.com/s3/) 作为外部存储。 +下面是使用类型为 `s3` 的磁盘,将 [S3](https://aws.amazon.com/s3/) 用作外部存储的示例。 -配置标记: +配置示例: ```xml @@ -1058,33 +1031,32 @@ SETTINGS storage_policy = 'moving_from_ssd_to_hdd' 另请参阅[配置外部存储选项](/operations/storing-data.md/#configuring-external-storage)。 :::note 缓存配置 -ClickHouse 版本 22.3 至 22.7 使用不同的缓存配置,如果您使用的是这些版本,请参阅[使用本地缓存](/operations/storing-data.md/#using-local-cache)。 +ClickHouse 版本 22.3 至 22.7 使用了不同的缓存配置,如果你正在使用这些版本之一,请参阅[使用本地缓存](/operations/storing-data.md/#using-local-cache)。 ::: - ## 虚拟列 {#virtual-columns} -- `_part` — 数据分片的名称。 -- `_part_index` — 数据分片在查询结果中的顺序索引。 -- `_part_starting_offset` — 数据分片在查询结果中的累计起始行号。 -- `_part_offset` — 数据分片中的行号。 -- `_part_granule_offset` — 数据分片中的颗粒编号。 -- `_partition_id` — 分区的名称。 -- `_part_uuid` — 数据分片的唯一标识符(如果启用了 MergeTree 设置 `assign_part_uuids`)。 -- `_part_data_version` — 数据分片的数据版本(最小块编号或变更版本)。 -- `_partition_value` — `partition by` 表达式的值(元组)。 -- `_sample_factor` — 采样因子(来自查询)。 -- `_block_number` — 插入时为行分配的原始块编号,当启用设置 `enable_block_number_column` 时在合并操作中保留。 -- `_block_offset` — 插入时为行分配的块内原始行号,当启用设置 `enable_block_offset_column` 时在合并操作中保留。 -- `_disk_name` — 用于存储的磁盘名称。 - +* `_part` — 数据部分(part)的名称。 +* `_part_index` — 该数据部分在查询结果中的顺序索引。 +* `_part_starting_offset` — 该数据部分在查询结果中的累计起始行号。 +* `_part_offset` — 该数据部分中的行号。 +* `_part_granule_offset` — 该数据部分中的 granule 编号。 +* `_partition_id` — 分区的名称。 +* `_part_uuid` — 数据部分的唯一标识符(如果启用了 MergeTree 设置 `assign_part_uuids`)。 +* `_part_data_version` — 数据部分的数据版本(最小块号或变更版本)。 +* `_partition_value` — `partition by` 表达式的值(一个元组)。 +* `_sample_factor` — 采样因子(来自查询)。 +* `_block_number` — 插入时为该行分配的原始块号;在启用设置 `enable_block_number_column` 时,在合并过程中会被保留。 +* `_block_offset` — 插入时为该行在块中的原始行号;在启用设置 `enable_block_offset_column` 时,在合并过程中会被保留。 +* `_disk_name` — 用于存储的磁盘名称。 ## 列统计信息 {#column-statistics} + -在启用 `set allow_experimental_statistics = 1` 后,可以在 `*MergeTree*` 系列表的 `CREATE` 查询的列定义部分声明统计信息。 +在启用 `set allow_experimental_statistics = 1` 时,对于 `*MergeTree*` 系列表,可以在 `CREATE` 查询的列(columns)部分中声明统计信息。 ```sql CREATE TABLE tab @@ -1096,69 +1068,68 @@ ENGINE = MergeTree ORDER BY a ``` -也可以使用 `ALTER` 语句来操作统计信息。 +我们也可以使用 `ALTER` 语句来调整统计信息。 ```sql ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; ALTER TABLE tab DROP STATISTICS a; ``` -这些轻量级统计信息汇总了列中值的分布信息。统计信息存储在每个数据部分中,并在每次插入数据时更新。 -只有在启用 `set allow_statistics_optimize = 1` 时,才能将其用于 prewhere 优化。 +这些轻量级统计信息汇总了列中值的分布情况。统计信息存储在每个数据片段中,并在每次插入时都会更新。 +只有在启用 `set allow_statistics_optimize = 1` 时,它们才会用于 `PREWHERE` 优化。 -### 可用的列统计信息类型 {#available-types-of-column-statistics} +### 可用的列统计类型 {#available-types-of-column-statistics} -- `MinMax` +* `MinMax` - 列的最小值和最大值,用于估算数值列上范围过滤器的选择性。 + 列的最小值和最大值,用于估计数值列上范围过滤条件的选择性。 - 语法:`minmax` + 语法:`minmax` -- `TDigest` +* `TDigest` - [TDigest](https://github.com/tdunning/t-digest) 草图,用于计算数值列的近似百分位数(例如第 90 百分位数)。 + [TDigest](https://github.com/tdunning/t-digest) Sketch 数据结构,用于计算数值列的近似分位数(例如第 90 个百分位数)。 - 语法:`tdigest` + 语法:`tdigest` -- `Uniq` +* `Uniq` - [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) 草图,用于估算列中不同值的数量。 + [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) Sketch 数据结构,用于估算某列中包含的不同值的数量。 - 语法:`uniq` + 语法:`uniq` -- `CountMin` +* `CountMin` - [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch) 草图,用于提供列中每个值出现频率的近似计数。 + [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch) Sketch 数据结构,用于对某列中每个值的出现频率进行近似计数。 - 语法:`countmin` + 语法:`countmin` ### 支持的数据类型 {#supported-data-types} -| | (U)Int*, Float*, Decimal(_), Date_, Boolean, Enum\* | String 或 FixedString | -| -------- | --------------------------------------------------- | --------------------- | -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | +| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String 或 FixedString | +|-----------|----------------------------------------------------|-----------------------| +| CountMin | ✔ | ✔ | +| MinMax | ✔ | ✗ | +| TDigest | ✔ | ✗ | +| Uniq | ✔ | ✔ | ### 支持的操作 {#supported-operations} -| | 等值过滤器 (==) | 范围过滤器 (`>, >=, <, <=`) | -| -------- | --------------------- | ------------------------------ | -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | - +| | 等值过滤 (==) | 范围过滤 (`>, >=, <, <=`) | +|-----------|----------------|---------------------------| +| CountMin | ✔ | ✗ | +| MinMax | ✗ | ✔ | +| TDigest | ✗ | ✔ | +| Uniq | ✔ | ✗ | -## 列级设置 {#column-level-settings} +## 列级别设置 {#column-level-settings} -某些 MergeTree 设置可以在列级别覆盖: +某些 MergeTree 设置可以在列级别进行覆盖: -- `max_compress_block_size` — 写入表时压缩前未压缩数据块的最大大小。 -- `min_compress_block_size` — 写入下一个标记时压缩所需的未压缩数据块的最小大小。 +* `max_compress_block_size` — 在写入表之前,对未压缩数据块进行压缩时所允许的最大未压缩数据块大小。 +* `min_compress_block_size` — 在写入下一个标记时,为执行压缩所需的最小未压缩数据块大小。 -示例: +示例: ```sql CREATE TABLE tab @@ -1170,21 +1141,21 @@ ENGINE = MergeTree ORDER BY id ``` -列级设置可以使用 [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md) 进行修改或删除,例如: +可以使用 [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md) 修改或删除列级设置,例如: -- 从列声明中删除 `SETTINGS`: +* 从列定义中删除 `SETTINGS`: ```sql ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; ``` -- 修改设置: +* 修改某项设置: ```sql ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; ``` -- 重置一个或多个设置,同时从表的 CREATE 查询的列表达式中删除设置声明。 +* 重置一个或多个设置,同时会从该表的 `CREATE` 查询语句中的列表达式里删除相应的设置声明。 ```sql ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md index c0316989b98..976823f2af9 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# ReplacingMergeTree 表引擎 +# ReplacingMergeTree 表引擎 {#replacingmergetree-table-engine} 该引擎与 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) 的不同之处在于,它会删除具有相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md)值的重复记录(指表的 `ORDER BY` 子句,而非 `PRIMARY KEY`)。 @@ -23,7 +23,7 @@ doc_type: 'reference' -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -46,9 +46,9 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] ::: -## ReplacingMergeTree 参数 +## ReplacingMergeTree 参数 {#replacingmergetree-parameters} -### `ver` +### `ver` {#ver} `ver` — 带有版本号的列。类型为 `UInt*`、`Date`、`DateTime` 或 `DateTime64`。可选参数。 @@ -100,7 +100,7 @@ SELECT * FROM mySecondReplacingMT FINAL; └─────┴─────────┴─────────────────────┘ ``` -### `is_deleted` +### `is_deleted` {#is_deleted} `is_deleted` — 在合并过程中用于确定该行数据表示的是当前状态还是应被删除的列名;`1` 表示“删除”行,`0` 表示“状态”行。 @@ -187,7 +187,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 查询时去重 & FINAL +## 查询时去重 & FINAL {#query-time-de-duplication--final} 在合并阶段,ReplacingMergeTree 使用 `ORDER BY` 列(用于创建表)中的值作为唯一标识来识别重复行,并仅保留版本最高的那一行。不过,这种方式只能在最终状态上接近正确——它并不保证所有重复行都会被去重,因此不应将其作为严格依赖。由于更新和删除记录在查询时仍可能被计算在内,查询结果因此可能不正确。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md index 23ff93782e9..35717f42c4d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md @@ -1,18 +1,16 @@ --- -description: 'ClickHouse 中 Replicated* 系列表引擎数据复制概览' +description: '基于 ClickHouse 中 Replicated* 系列表引擎的数据复制概述' sidebar_label: 'Replicated*' sidebar_position: 20 slug: /engines/table-engines/mergetree-family/replication -title: 'Replicated* 表引擎' +title: 'Replicated* 系列表引擎' doc_type: 'reference' --- - - -# Replicated* 表引擎 +# Replicated* 表引擎 {#replicated-table-engines} :::note -在 ClickHouse Cloud 中,副本由系统自动管理。请在创建表时不要添加参数。例如,在下面的文本中,应将以下内容替换为: +在 ClickHouse Cloud 中,复制由系统自动管理。请在创建表时不要添加这些参数。例如,在下面的文本中,你应将其替换为: ```sql ENGINE = ReplicatedMergeTree( @@ -21,7 +19,7 @@ ENGINE = ReplicatedMergeTree( ) ``` -包含: +包括: ```sql ENGINE = ReplicatedMergeTree @@ -29,7 +27,7 @@ ENGINE = ReplicatedMergeTree ::: -复制仅支持 MergeTree 系列的表引擎: +复制仅支持属于 MergeTree 系列的表: * ReplicatedMergeTree * ReplicatedSummingMergeTree @@ -39,21 +37,21 @@ ENGINE = ReplicatedMergeTree * ReplicatedVersionedCollapsingMergeTree * ReplicatedGraphiteMergeTree -复制是在单个表级别进行的,而不是在整个服务器级别。一个服务器可以同时存储已复制的表和未复制的表。 +复制在单表级别进行,而不是在整个服务器级别进行。单个服务器可以同时存储已复制表和未复制表。 复制不依赖于分片。每个分片都有自己独立的复制。 -针对 `INSERT` 和 `ALTER` 查询的压缩数据会被复制(更多信息,参见 [ALTER](/sql-reference/statements/alter) 文档)。 +`INSERT` 和 `ALTER` 查询的压缩数据会被复制(更多信息,参见 [ALTER](/sql-reference/statements/alter) 文档)。 -`CREATE`、`DROP`、`ATTACH`、`DETACH` 和 `RENAME` 查询只在单个服务器上执行,并不会被复制: +`CREATE`、`DROP`、`ATTACH`、`DETACH` 和 `RENAME` 查询在单个服务器上执行,不会被复制: -* `CREATE TABLE` 查询会在执行该查询的服务器上创建一个新的可复制的表。如果该表在其他服务器上已经存在,则会添加一个新副本。 -* `DROP TABLE` 查询会删除位于执行该查询的服务器上的副本。 +* `CREATE TABLE` 查询会在执行该查询的服务器上创建一个新的可复制表。如果此表已经存在于其他服务器上,它会增加一个新的副本。 +* `DROP TABLE` 查询会删除位于执行该查询的服务器上的该副本。 * `RENAME` 查询会在其中一个副本上重命名表。换句话说,复制表在不同副本上可以有不同的名称。 -ClickHouse 使用 [ClickHouse Keeper](/guides/sre/keeper/index.md) 来存储副本的元信息。也可以使用 3.4.5 或更高版本的 ZooKeeper,但推荐使用 ClickHouse Keeper。 +ClickHouse 使用 [ClickHouse Keeper](/guides/sre/keeper/index.md) 来存储副本的元信息。也可以使用 3.4.5 或更新版本的 ZooKeeper,但推荐使用 ClickHouse Keeper。 -要使用复制功能,请在服务器配置的 [zookeeper](/operations/server-configuration-parameters/settings#zookeeper) 部分中设置相关参数。 +要使用复制功能,请在服务器配置的 [zookeeper](/operations/server-configuration-parameters/settings#zookeeper) 部分设置相关参数。 :::note 不要忽视安全设置。ClickHouse 支持 ZooKeeper 安全子系统的 `digest` [ACL 方案](https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#sc_ZooKeeperAccessControl)。 @@ -78,10 +76,10 @@ ClickHouse 使用 [ClickHouse Keeper](/guides/sre/keeper/index.md) 来存储副 ``` -ClickHouse 还支持将副本元数据存储在辅助 ZooKeeper 集群中。方法是:在引擎参数中指定 ZooKeeper 集群名称和路径。 +ClickHouse 也支持将副本元数据存储在辅助 ZooKeeper 集群中。可以通过在引擎参数中指定 ZooKeeper 集群名称和路径来实现。 换句话说,它支持将不同表的元数据存储在不同的 ZooKeeper 集群中。 -设置辅助 ZooKeeper 集群地址的示例: +辅助 ZooKeeper 集群地址配置示例: ```xml @@ -108,59 +106,57 @@ ClickHouse 还支持将副本元数据存储在辅助 ZooKeeper 集群中。方 ``` -要将表元数据存储在一个额外的 ZooKeeper 集群中,而不是默认的 ZooKeeper 集群,可以使用 SQL 创建一个基于 ReplicatedMergeTree 引擎的表,如下所示: +要将表元数据存储在辅助 ZooKeeper 集群而不是默认的 ZooKeeper 集群中,可以使用 SQL 创建基于 ReplicatedMergeTree 引擎的表,如下所示: ```sql CREATE TABLE table_name ( ... ) ENGINE = ReplicatedMergeTree('zookeeper_name_configured_in_auxiliary_zookeepers:path', 'replica_name') ... ``` -你可以指定任何现有的 ZooKeeper 集群,系统会在其上使用一个目录用于存储自身数据(该目录在创建复制表时指定)。 - -如果在配置文件中未配置 ZooKeeper,则无法创建复制表,且所有已有的复制表都将是只读的。 +你可以指定任意现有的 ZooKeeper 集群,系统会在该集群上使用一个目录来存放自身数据(该目录在创建副本表时指定)。 +如果在配置文件中未配置 ZooKeeper,你将无法创建副本表,且任何已有的副本表都将变为只读。 -ZooKeeper 不用于 `SELECT` 查询,因为复制不会影响 `SELECT` 的性能,查询速度与非复制表一样快。在查询分布式复制表时,ClickHouse 的行为由 [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) 和 [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries) 这两个设置控制。 -对于每个 `INSERT` 查询,大约会通过若干事务向 ZooKeeper 添加十条记录。(更精确地说,这是针对每个插入的数据块;一个 `INSERT` 查询包含一个数据块,或者每 `max_insert_block_size = 1048576` 行一个数据块。)这会导致与非复制表相比,`INSERT` 的延迟略有增加。但如果按照建议以每秒不超过一次 `INSERT` 的批量方式插入数据,就不会产生任何问题。用于与单个 ZooKeeper 集群协同工作的整个 ClickHouse 集群,总共每秒只有数百个 `INSERT`。数据插入的吞吐量(每秒行数)与非复制数据一样高。 +ZooKeeper 不参与 `SELECT` 查询,因为复制不会影响 `SELECT` 的性能,查询速度与非复制表一样快。对于分布式复制表的查询,ClickHouse 的行为由 [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) 和 [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries) 这两个设置控制。 -对于非常大的集群,可以为不同的分片使用不同的 ZooKeeper 集群。然而,根据我们的经验,在大约 300 台服务器的生产集群中,这并不是必要的。 +对于每个 `INSERT` 查询,会通过若干事务向 ZooKeeper 添加大约十条记录。(更精确地说,这是针对每个被插入的数据块;一个 INSERT 查询包含一个数据块,或者每 `max_insert_block_size = 1048576` 行一个数据块。)这会导致 `INSERT` 相比非复制表有略高的延迟。但如果遵循建议,以每秒不超过一个 `INSERT` 的批量方式插入数据,就不会产生任何问题。一个用于协调单个 ZooKeeper 集群的整个 ClickHouse 集群,总共每秒可处理数百个 `INSERT`。数据插入的吞吐量(每秒行数)与非复制数据一样高。 -复制是异步且多主的。`INSERT` 查询(以及 `ALTER`)可以发送到任何可用的服务器。数据会先插入到执行查询的服务器上,然后再复制到其他服务器上。由于是异步的,最近插入的数据会在一定延迟后才出现在其他副本上。如果部分副本不可用,则会在它们重新可用时再写入数据。如果某个副本是可用的,延迟就是通过网络传输压缩数据块所需的时间。用于为复制表执行后台任务的线程数可以通过 [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) 设置进行配置。 +对于非常大的集群,可以为不同的分片使用不同的 ZooKeeper 集群。不过根据我们的经验,在大约 300 台服务器的生产集群中,还没有发现这种做法是必要的。 -`ReplicatedMergeTree` 引擎为副本抓取(fetch)操作使用了单独的线程池。该线程池的大小由 [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size) 设置限制,并且可以通过重启服务器来调整。 +复制是异步的,并且是多主(multi-master)。`INSERT` 查询(以及 `ALTER`)可以发送到任何可用服务器。数据会先插入到执行查询的服务器上,然后再复制到其他服务器。由于复制是异步的,新插入的数据会在一定延迟后才出现在其他副本上。如果部分副本不可用,则当这些副本重新可用时数据才会被写入。如果某个副本是可用的,延迟就是在网络上传输压缩数据块所需的时间。用于处理复制表后台任务的线程数量可以通过 [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) 这个设置来配置。 -默认情况下,`INSERT` 查询只等待来自一个副本的数据写入确认。如果数据仅成功写入一个副本,而该副本所在的服务器随后宕机,存储的数据就会丢失。要启用从多个副本获取数据写入确认,请使用 `insert_quorum` 选项。 +`ReplicatedMergeTree` 引擎使用单独的线程池来处理复制数据拉取(replicated fetches)。该线程池的大小由 [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size) 设置限制,并且可以在重启服务器时进行调整。 -每个数据块都是原子写入的。`INSERT` 查询会被拆分为最多 `max_insert_block_size = 1048576` 行的数据块。换句话说,如果 `INSERT` 查询少于 1048576 行,则该查询是以原子方式完成写入的。 +默认情况下,INSERT 查询只等待来自一个副本的数据写入确认。如果数据仅成功写入一个副本,而该副本所在的服务器随之完全宕机,则已存储的数据会丢失。要启用从多个副本获取写入确认的能力,请使用 `insert_quorum` 选项。 -数据块会去重。对于多次写入相同的数据块(大小相同且包含相同行并且顺序相同的数据块),该数据块只会被写入一次。这样做是为了在发生网络故障、客户端应用无法确定数据是否已写入数据库时,可以简单地重复执行 `INSERT` 查询。将相同数据的 `INSERT` 发送到哪个副本并不重要。`INSERTs` 是幂等的。去重参数由 [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree) 服务器设置控制。 +每个数据块的写入是原子的。INSERT 查询会被拆分为最多 `max_insert_block_size = 1048576` 行的数据块。换句话说,如果 `INSERT` 查询中少于 1048576 行,则整个查询是以原子方式完成的。 -在复制过程中,只有要插入的源数据会通过网络传输。后续的数据转换(合并)会在所有副本上以相同方式协调并执行。这样可以将网络使用量降到最低,这意味着当副本位于不同数据中心时,复制也能良好工作。(注意,在不同数据中心中复制数据是复制的主要目标。) +数据块会被去重。对于多次写入同一个数据块(具有相同大小、包含相同行且行顺序相同的数据块),该数据块只会被写入一次。这样设计的原因在于,网络故障时客户端应用可能不知道数据是否已写入数据库,因此可以直接重试 `INSERT` 查询。对于包含相同数据的 INSERT 来说,它被发送到了哪个副本并不重要。`INSERT` 操作是幂等的。去重参数由 [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree) 服务器设置控制。 -你可以为相同的数据配置任意数量的副本。根据我们的经验,一个相对可靠且易于运维的方案是在生产环境中使用双副本复制,并且每台服务器使用 RAID-5 或 RAID-6(某些情况下使用 RAID-10)。 +在复制过程中,只有要插入的源数据会通过网络传输。后续的数据转换(合并)会在所有副本上以相同方式进行协调并执行。这样可以最大限度地减少网络使用量,这意味着当副本位于不同数据中心时,复制依然工作良好。(注意,将数据复制到不同数据中心是复制的主要目标。) -系统会监控副本上的数据同步情况,并能够在故障后进行恢复。故障切换可以是自动的(当数据差异较小时),也可以是半自动的(当数据差异过大时,这可能表明存在配置错误)。 +同一份数据可以拥有任意数量的副本。根据我们的经验,在生产环境中,一个相对可靠且易于运维的方案是使用双副本(double replication),并在每台服务器上使用 RAID-5 或 RAID-6(在某些情况下使用 RAID-10)。 +系统会监控副本上的数据同步情况,并且能够在故障后恢复。对于数据差异较小的情况,故障转移是自动完成的;对于数据差异过大的情况(可能表明存在配置错误),故障转移是半自动完成的。 - -## 创建复制表 +## 创建复制表 {#creating-replicated-tables} :::note -在 ClickHouse Cloud 中,副本复制由平台自动处理。 +在 ClickHouse Cloud 中,复制由系统自动处理。 -创建表时使用不带复制参数的 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree)。系统会在内部将 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 重写为 [`SharedMergeTree`](/cloud/reference/shared-merge-tree),以实现复制和数据分发。 +使用不带复制参数的 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 创建表。系统会在内部将 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 重写为用于复制和数据分布的 [`SharedMergeTree`](/cloud/reference/shared-merge-tree)。 -避免使用 `ReplicatedMergeTree` 或手动指定复制参数,因为复制由平台统一管理。 +请避免使用 `ReplicatedMergeTree` 或指定复制参数,因为复制由平台统一管理。 ::: -### Replicated*MergeTree 参数 +### Replicated*MergeTree 参数 {#replicatedmergetree-parameters} -| 参数 | 说明 | -| ------------------ | ----------------------------------------------- | -| `zoo_path` | 在 ClickHouse Keeper 中表的路径。 | -| `replica_name` | 在 ClickHouse Keeper 中副本的名称。 | -| `other_parameters` | 用于创建复制版本的底层引擎参数,例如 `ReplacingMergeTree` 中的版本参数。 | +| 参数 | 描述 | +| ------------------ | -------------------------------------------------- | +| `zoo_path` | ClickHouse Keeper 中该表的路径。 | +| `replica_name` | ClickHouse Keeper 中的副本名称。 | +| `other_parameters` | 用于创建该副本引擎版本的参数,例如 `ReplacingMergeTree` 中的 version。 | 示例: @@ -171,6 +167,7 @@ CREATE TABLE table_name CounterID UInt32, UserID UInt32, ver UInt16 +) ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{layer}-{shard}/table_name', '{replica}', ver) PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) @@ -190,7 +187,7 @@ SAMPLE BY intHash32(UserID); ``` -如示例所示,这些参数可以包含花括号中的替换占位符。替换值取自配置文件的 [macros](/operations/server-configuration-parameters/settings.md/#macros) 部分。 +如该示例所示,这些参数可以在花括号中包含可替换的占位符。替换后的值取自配置文件的 [macros](/operations/server-configuration-parameters/settings.md/#macros) 部分。 示例: @@ -201,33 +198,33 @@ SAMPLE BY intHash32(UserID); ``` -在 ClickHouse Keeper 中,每个复制表的路径都必须唯一。不同分片上的表应当具有不同的路径。 +在 ClickHouse Keeper 中,指向表的路径对于每个 Replicated 表都必须是唯一的。不同分片上的表应使用不同的路径。 在本例中,路径由以下部分组成: -`/clickhouse/tables/` 是通用前缀。建议严格使用这一前缀。 +`/clickhouse/tables/` 是公共前缀。建议原样使用该前缀。 `{shard}` 将被展开为分片标识符。 -`table_name` 是 ClickHouse Keeper 中该表对应节点的名称。将其设置为与表名相同是一个不错的选择。之所以要显式定义,是因为与表名不同,它在执行 RENAME 查询后不会改变。 -*提示*:也可以在 `table_name` 前添加数据库名,例如:`db_name.table_name` +`table_name` 是该表在 ClickHouse Keeper 中对应节点的名称。通常将其设置为与表名相同是个好主意。之所以要显式定义,是因为与表名不同的是,它在执行 RENAME 查询后不会改变。 +*提示*:也可以在 `table_name` 前加上数据库名,例如 `db_name.table_name`。 -可以使用两个内置替换量 `{database}` 和 `{table}`,它们分别展开为表名和数据库名(除非在 `macros` 部分中定义了这些宏)。因此,可以将 ZooKeeper 路径指定为 `'/clickhouse/tables/{shard}/{database}/{table}'`。 -在使用这些内置替换量时,请谨慎处理表重命名。ClickHouse Keeper 中的路径无法更改,而当表被重命名后,宏会展开为不同的路径,表将引用 ClickHouse Keeper 中不存在的路径,并进入只读模式。 +可以使用两个内置替换 `{database}` 和 `{table}`,它们分别展开为表名和数据库名(除非在 `macros` 部分中定义了这些宏)。因此,ZooKeeper 路径可以指定为 `'/clickhouse/tables/{shard}/{database}/{table}'`。 +在使用这些内置替换时,请谨慎处理表重命名。ClickHouse Keeper 中的路径无法更改,当表被重命名后,宏会展开为不同的路径,表将指向 ClickHouse Keeper 中不存在的路径,并进入只读模式。 副本名称用于标识同一张表的不同副本。可以像示例中那样使用服务器名。该名称只需要在每个分片内保持唯一即可。 -你可以不使用替换量,而是显式定义这些参数。对于测试以及配置小型集群时,这可能更加方便。不过,在这种情况下,将无法使用分布式 DDL 查询(`ON CLUSTER`)。 +可以显式定义这些参数,而不是使用替换。这在测试以及配置小型集群时可能更为方便。但是,在这种情况下将无法使用分布式 DDL 查询(`ON CLUSTER`)。 -在处理大型集群时,建议使用替换量,因为这可以降低出错的概率。 +在处理大型集群时,建议使用替换,因为它们可以降低出错概率。 -你可以在服务器配置文件中为 `Replicated` 表引擎指定默认参数。例如: +可以在服务器配置文件中为 `Replicated` 表引擎指定默认参数。例如: ```xml /clickhouse/tables/{shard}/{database}/{table} {replica} ``` -在这种情况下,在创建表时可以省略参数: +在这种情况下,创建表时可以省略参数: ```sql CREATE TABLE table_name ( @@ -236,8 +233,7 @@ CREATE TABLE table_name ( ORDER BY x; ``` -这相当于: - +它等同于: ```sql CREATE TABLE table_name ( @@ -246,105 +242,102 @@ CREATE TABLE table_name ( ORDER BY x; ``` -在每个副本上运行 `CREATE TABLE` 查询。此查询会创建一个新的复制表,或向现有复制表中添加一个新的副本。 +在每个副本上运行 `CREATE TABLE` 查询。此查询会创建一个新的复制表,或者将一个新的副本添加到现有表中。 -如果是在表在其他副本上已经包含一些数据之后才添加新副本,那么在运行查询后,这些数据会从其他副本复制到新的副本。换句话说,新副本会与其他副本进行同步。 +如果是在其他副本上已经存在部分数据之后才添加新的副本,那么在运行该查询后,数据会从其他副本复制到新的副本。换句话说,新副本会与其他副本进行同步。 -要删除副本,请运行 `DROP TABLE`。但这只会删除一个副本——也就是你执行该查询所在服务器上的那个副本。 +要删除一个副本,请运行 `DROP TABLE`。但这只会删除一个副本——也就是你运行该查询所在服务器上的那个副本。 -## 故障后的恢复 +## 故障后的恢复 {#recovery-after-failures} -如果在服务器启动时 ClickHouse Keeper 不可用,复制表会切换到只读模式。系统会定期尝试连接 ClickHouse Keeper。 +如果在服务器启动时 ClickHouse Keeper 不可用,复制表会切换为只读模式。系统会定期尝试连接 ClickHouse Keeper。 -如果在执行 `INSERT` 时 ClickHouse Keeper 不可用,或者在与 ClickHouse Keeper 交互时发生错误,则会抛出异常。 +如果在执行 `INSERT` 期间 ClickHouse Keeper 不可用,或者在与 ClickHouse Keeper 交互时发生错误,系统会抛出异常。 -连接到 ClickHouse Keeper 之后,系统会检查本地文件系统中的数据集是否与预期的数据集匹配(ClickHouse Keeper 会存储这些信息)。如果存在轻微的不一致,系统会通过与副本同步数据来解决。 +连接到 ClickHouse Keeper 之后,系统会检查本地文件系统中的数据集是否与预期的数据集匹配(ClickHouse Keeper 中保存了这些信息)。如果存在轻微不一致,系统会通过与副本同步数据来解决。 -如果系统检测到损坏的数据分片(文件大小错误)或无法识别的分片(已写入文件系统但未记录到 ClickHouse Keeper 的分片),则会将它们移动到 `detached` 子目录中(不会被删除)。任何缺失的分片都会从副本中复制。 +如果系统检测到损坏的数据分区片段(文件大小错误)或无法识别的分区片段(已写入文件系统但未在 ClickHouse Keeper 中记录的分区片段),则会将它们移动到 `detached` 子目录中(不会被删除)。任何缺失的分区片段都会从副本中复制。 请注意,ClickHouse 不会执行任何破坏性操作,例如自动删除大量数据。 -当服务器启动(或与 ClickHouse Keeper 建立新会话)时,它只检查所有文件的数量和大小。如果文件大小相同但中间某处字节发生了变化,则不会立即检测到,只会在尝试为 `SELECT` 查询读取数据时才会发现。此时查询会抛出关于校验和不匹配或压缩块大小不匹配的异常。在这种情况下,数据分片会被加入到验证队列中,并在必要时从副本中复制。 +在服务器启动时(或与 ClickHouse Keeper 建立新会话时),系统只会检查所有文件的数量和大小。如果文件大小匹配,但中间某处的字节发生了更改,则不会立刻被检测到,只有在尝试为 `SELECT` 查询读取数据时才会发现。此时查询会抛出关于校验和不匹配或压缩块大小不匹配的异常。在这种情况下,数据分区片段会被加入验证队列,并在必要时从副本中复制。 -如果本地数据集与预期数据集的差异过大,就会触发安全机制。服务器会将此记录到日志中并拒绝启动。这样设计的原因是,这种情况可能表明配置错误,例如某个分片上的副本被误配置成了另一个分片上的副本。不过,该机制的阈值设置得相当低,因此这种情况也可能在正常的故障恢复过程中发生。在这种情况下,数据通过“按下一个按钮”实现半自动恢复。 +如果本地数据集与预期的数据集差异过大,会触发安全机制。服务器会将此写入日志,并拒绝启动。原因是这种情况可能表明存在配置错误,例如某个分片上的副本被误配置为另一个分片上的副本。然而,该机制的阈值设置得相当低,这种情况也可能在正常的故障恢复过程中出现。在这种情况下,数据会通过“按下一个按钮”的方式进行半自动恢复。 -要开始恢复,在 ClickHouse Keeper 中创建节点 `/path_to_table/replica_name/flags/force_restore_data` 并填入任意内容,或者运行以下命令来恢复所有复制表: +要开始恢复,请在 ClickHouse Keeper 中创建节点 `/path_to_table/replica_name/flags/force_restore_data`(内容任意),或者运行命令以恢复所有复制表: ```bash sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data ``` -然后重启服务器。服务器在启动时会删除这些标志位并开始恢复。 +随后重启服务器。服务器在启动时会删除这些标志文件并开始恢复。 ## 完全数据丢失后的恢复 {#recovery-after-complete-data-loss} -如果某个服务器上的所有数据和元数据都丢失了,请按照以下步骤进行恢复: +如果某台服务器上的所有数据和元数据都丢失了,请按以下步骤进行恢复: -1. 在该服务器上安装 ClickHouse。正确配置包含分片标识符和副本信息的配置文件中的 substitution(替换规则)(如果你使用了它们)。 -2. 如果存在未复制的表且需要在服务器上手动恢复,请从其某个副本复制这些表的数据(目录为 `/var/lib/clickhouse/data/db_name/table_name/`)。 -3. 从某个副本复制位于 `/var/lib/clickhouse/metadata/` 中的表定义文件。如果在表定义中显式指定了分片或副本标识符,请将其修改为与当前副本相对应。(或者,启动服务器并执行所有本应位于 `/var/lib/clickhouse/metadata/` 目录下 .sql 文件中的 `ATTACH TABLE` 查询。) +1. 在该服务器上安装 ClickHouse。在包含分片标识符和副本信息(如使用副本)的配置文件中正确配置 substitutions。 +2. 如果有需要在服务器上手动复制的非复制表(unreplicated tables),请从副本中拷贝其数据(目录 `/var/lib/clickhouse/data/db_name/table_name/`)。 +3. 从副本中拷贝位于 `/var/lib/clickhouse/metadata/` 的表定义。如果在表定义中显式定义了分片或副本标识符,请更正为与此副本相对应。(或者,启动服务器并执行所有本应位于 `/var/lib/clickhouse/metadata/` 目录下 .sql 文件中的 `ATTACH TABLE` 查询。) 4. 要开始恢复,请在 ClickHouse Keeper 中创建节点 `/path_to_table/replica_name/flags/force_restore_data`(内容任意),或者运行命令以恢复所有复制表:`sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` -然后启动服务器(如果已在运行,则重启)。数据会从其他副本中下载。 - -另一种恢复方式是从 ClickHouse Keeper 中删除丢失副本的信息(`/path_to_table/replica_name`),然后按照“[创建复制表](#creating-replicated-tables)”中所述重新创建该副本。 - -恢复过程中对网络带宽没有限制。如果你一次恢复许多副本,请注意这一点。 +然后启动服务器(如果已经在运行,则重启)。数据会从其他副本中下载。 +另一种恢复方式是从 ClickHouse Keeper 中删除与丢失副本相关的信息(`/path_to_table/replica_name`),然后按照“[创建复制表](#creating-replicated-tables)”中的说明重新创建该副本。 +恢复过程中对网络带宽没有限制。如果你同时恢复大量副本,请注意这一点。 -## 从 MergeTree 转换为 ReplicatedMergeTree +## 从 MergeTree 转换为 ReplicatedMergeTree {#converting-from-mergetree-to-replicatedmergetree} -我们使用术语 `MergeTree` 来统称 `MergeTree family` 中的所有表引擎,对 `ReplicatedMergeTree` 也是同样的用法。 +我们使用术语 `MergeTree` 指代 `MergeTree family` 中的所有表引擎,对 `ReplicatedMergeTree` 也采用相同的约定。 -如果您有一个是通过手动方式实现复制的 `MergeTree` 表,可以将它转换为复制表。如果您已经在一个 `MergeTree` 表中收集了大量数据,现在希望启用复制功能,则可能需要执行此操作。 +如果有一个通过手动方式进行复制的 `MergeTree` 表,可以将其转换为复制表。如果已经在某个 `MergeTree` 表中累积了大量数据,并且现在希望启用复制功能,则可能需要这样做。 -[ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句可以将已分离的 `MergeTree` 表以 `ReplicatedMergeTree` 的形式附加。 +[ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句允许将已分离的 `MergeTree` 表附加为 `ReplicatedMergeTree`。 -如果在表的数据目录中(对于 `Atomic` 数据库,即 `/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/`)设置了 `convert_to_replicated` 标志,则在服务器重启时可以自动将 `MergeTree` 表转换。创建一个空的 `convert_to_replicated` 文件,下一次服务器重启时该表将作为复制表加载。 +如果在表的数据目录(对于 `Atomic` 数据库为 `/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/`)下设置了 `convert_to_replicated` 标记,则在服务器重启时可以自动将 `MergeTree` 表转换为复制表。 +创建一个空的 `convert_to_replicated` 文件,下一次服务器重启时,该表将作为复制表加载。 -可以使用此查询来获取表的数据路径。如果表有多个数据路径,则必须使用第一个路径。 +可以使用以下查询获取表的数据路径。如果表有多个数据路径,则必须使用第一个。 ```sql SELECT data_paths FROM system.tables WHERE table = 'table_name' AND database = 'database_name'; ``` 请注意,ReplicatedMergeTree 表将会使用 `default_replica_path` 和 `default_replica_name` 设置的值来创建。 -要在其他副本上创建已转换的表,需要在 `ReplicatedMergeTree` 引擎的第一个参数中显式指定其路径。可以使用以下查询来获取其路径。 +要在其他副本上创建转换后的表,需要在 `ReplicatedMergeTree` 引擎的第一个参数中显式指定其路径。可以使用以下查询来获取该路径。 ```sql SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; ``` -还有一种手动完成此操作的方法。 +还有一种手动方式可以实现此操作。 -如果各个副本上的数据不一致,先进行同步,或者在除一个副本外的所有副本上删除这部分数据。 +如果各个副本上的数据不一致,请先对其进行同步,或者在除一个副本外的所有副本上删除这些数据。 -将现有的 MergeTree 表重命名,然后使用原来的名称创建一个 `ReplicatedMergeTree` 表。 -将旧表中的数据移动到新表数据目录内的 `detached` 子目录(`/var/lib/clickhouse/data/db_name/table_name/`)。 -然后在其中一个副本上运行 `ALTER TABLE ATTACH PARTITION`,将这些数据部件添加到工作集中。 +先重命名现有的 MergeTree 表,然后使用旧表名创建一个 `ReplicatedMergeTree` 表。 +将旧表中的数据移动到新表数据目录下的 `detached` 子目录中(`/var/lib/clickhouse/data/db_name/table_name/`)。 +然后在其中一个副本上运行 `ALTER TABLE ATTACH PARTITION`,将这些分区片段添加到正在使用的工作集中。 ## 从 ReplicatedMergeTree 转换为 MergeTree {#converting-from-replicatedmergetree-to-mergetree} -使用 [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句,在单个服务器上将已分离(detached)的 `ReplicatedMergeTree` 表作为 `MergeTree` 表附加。 +使用 [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句,在单个服务器上将已分离的 `ReplicatedMergeTree` 表作为 `MergeTree` 表附加。 -另一种方式需要重启服务器。先创建一个名称不同的 MergeTree 表。将 `ReplicatedMergeTree` 表数据目录中的所有数据移动到新表的数据目录。然后删除该 `ReplicatedMergeTree` 表并重启服务器。 +另一种做法需要重启服务器。先创建一个名称不同的 MergeTree 表。将 `ReplicatedMergeTree` 表数据目录中的所有数据移动到新表的数据目录中。然后删除 `ReplicatedMergeTree` 表并重启服务器。 -如果希望在不启动服务器的情况下移除一个 `ReplicatedMergeTree` 表: +如果希望在不启动服务器的情况下移除 `ReplicatedMergeTree` 表: - 删除元数据目录(`/var/lib/clickhouse/metadata/`)中对应的 `.sql` 文件。 -- 删除 ClickHouse Keeper 中对应的路径(`/path_to_table/replica_name`)。 - -之后,可以启动服务器,创建一个 `MergeTree` 表,将数据移动到其数据目录,然后重启服务器。 - +- 删除 ClickHouse Keeper 中相应的路径(`/path_to_table/replica_name`)。 +完成上述操作后,可以启动服务器,创建一个 `MergeTree` 表,将数据移动到该表的数据目录中,然后重启服务器。 ## 当 ClickHouse Keeper 集群中的元数据丢失或损坏时的恢复 {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} -如果 ClickHouse Keeper 中的数据丢失或损坏,可以按照上文所述,将数据迁移到一个非复制表以进行保留。 +如果 ClickHouse Keeper 中的数据丢失或损坏,可以按照上文所述,将数据迁移到未复制表中以进行保存。 **另请参阅** @@ -352,4 +345,4 @@ SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; - [background_fetches_pool_size](/operations/server-configuration-parameters/settings.md/#background_fetches_pool_size) - [execute_merges_on_single_replica_time_threshold](/operations/settings/merge-tree-settings#execute_merges_on_single_replica_time_threshold) - [max_replicated_fetches_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_fetches_network_bandwidth) -- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) +- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md index 0dff8da5d6a..8d57f59172d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# SummingMergeTree 表引擎 +# SummingMergeTree 表引擎 {#summingmergetree-table-engine} 该引擎继承自 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree)。不同之处在于,当对 `SummingMergeTree` 表的数据部分进行合并时,ClickHouse 会将所有具有相同主键(更准确地说,是具有相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md))的多行,替换为一行,其中数值数据类型列的值为这些行的求和结果。如果排序键的设计使得单个键值对应大量行,这种方式可以显著减少存储体积并加速数据查询。 @@ -17,7 +17,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -34,16 +34,16 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] 有关请求参数的描述,请参阅[请求描述](../../../sql-reference/statements/create/table.md)。 -### SummingMergeTree 的参数 +### SummingMergeTree 的参数 {#parameters-of-summingmergetree} -#### 列 +#### 列 {#columns} `columns` — 一个包含需要求和的列名的元组(tuple)。可选参数。 这些列必须是数值类型,且不能出现在分区键或排序键中。 如果未指定 `columns`,ClickHouse 会对所有数值类型且不在排序键中的列进行求和。 -### 查询子句 +### 查询子句 {#query-clauses} 在创建 `SummingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)。 @@ -69,7 +69,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 使用示例 +## 使用示例 {#usage-example} 请看下表: @@ -103,13 +103,13 @@ SELECT key, sum(value) FROM summtt GROUP BY key ``` -## 数据处理 +## 数据处理 {#data-processing} 当数据被插入到表中时,会按原样保存。ClickHouse 会定期合并已插入的数据部分,在此过程中,具有相同主键的行会被求和,并在每个合并结果的数据部分中替换为一行。 ClickHouse 合并数据部分的方式可能导致:不同的合并结果数据部分中仍然可能包含具有相同主键的行,即求和可能是不完整的。因此,在执行查询时(`SELECT`),应按照上面的示例使用聚合函数 [sum()](/sql-reference/aggregate-functions/reference/sum) 和 `GROUP BY` 子句。 -### 求和的通用规则 +### 求和的通用规则 {#common-rules-for-summation} 数值数据类型列中的值会被求和。参与求和的列集合由参数 `columns` 定义。 @@ -119,11 +119,11 @@ ClickHouse 合并数据部分的方式可能导致:不同的合并结果数据 主键列中的值不会被求和。 -### AggregateFunction 列中的求和 +### AggregateFunction 列中的求和 {#the-summation-in-the-aggregatefunction-columns} 对于 [AggregateFunction 类型](../../../sql-reference/data-types/aggregatefunction.md) 的列,ClickHouse 的行为类似于 [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md) 引擎,会根据该函数进行聚合。 -### 嵌套结构 +### 嵌套结构 {#nested-structures} 表可以包含以特殊方式处理的嵌套数据结构。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md index 14c2b0668d3..31d7d8e3314 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# VersionedCollapsingMergeTree 表引擎 +# VersionedCollapsingMergeTree 表引擎 {#versionedcollapsingmergetree-table-engine} 该引擎: @@ -22,7 +22,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -39,7 +39,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] 有关查询参数的详细说明,请参阅[查询说明](../../../sql-reference/statements/create/table.md)。 -### 引擎参数 +### 引擎参数 {#engine-parameters} ```sql VersionedCollapsingMergeTree(sign, version) @@ -50,7 +50,7 @@ VersionedCollapsingMergeTree(sign, version) | `sign` | 行类型列的列名:`1` 表示“state”行,`-1` 表示“cancel”行。 | [`Int8`](/sql-reference/data-types/int-uint) | | `version` | 对象状态版本列的列名。 | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) 或 [`DateTime64`](/sql-reference/data-types/datetime64) | -### 查询子句 +### 查询子句 {#query-clauses} 在创建 `VersionedCollapsingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)。 @@ -82,9 +82,9 @@ VersionedCollapsingMergeTree(sign, version) -## 折叠 +## 折叠 {#table_engines_versionedcollapsingmergetree} -### 数据 +### 数据 {#data} 考虑这样一种情况:你需要为某个对象保存不断变化的数据。为某个对象仅保留一行记录,并在有变化时更新这一行是合理的。然而,对于 DBMS 来说,执行 `UPDATE` 操作代价高且速度慢,因为这需要在存储中重写数据。如果你需要快速写入数据,则不适合使用 `UPDATE`,但可以按如下方式顺序写入对象的变更。 @@ -130,7 +130,7 @@ VersionedCollapsingMergeTree(sign, version) 2. 列中持续增长的长数组会因为写入负载而降低引擎效率。数据越简单直接,引擎效率越高。 3. `SELECT` 结果高度依赖于对象变更历史的一致性。在准备要插入的数据时要非常谨慎。对于不一致的数据,你可能会得到不可预测的结果,例如本应为非负指标(如会话深度)的负值。 -### 算法 +### 算法 {#table_engines-versionedcollapsingmergetree-algorithm} 当 ClickHouse 合并数据分片时,会删除每一对具有相同主键和版本、但 `Sign` 不同的行。行的顺序无关紧要。 @@ -149,7 +149,7 @@ ClickHouse 不保证具有相同主键的所有行会位于同一个结果数据 -## 使用示例 +## 使用示例 {#example-of-use} 示例数据: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md index 51bd7bf315c..ad12189d81c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md @@ -1,5 +1,5 @@ --- -description: 'Alias 表引擎会创建到另一张表的透明代理。所有操作都会被转发到目标表,而别名表自身不存储任何数据。' +description: 'Alias 表引擎创建一个指向另一张表的透明代理。所有操作都会转发至目标表,而别名本身不存储任何数据。' sidebar_label: 'Alias' sidebar_position: 5 slug: /engines/table-engines/special/alias @@ -7,19 +7,26 @@ title: 'Alias 表引擎' doc_type: 'reference' --- +import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -# Alias 表引擎 +# Alias 表引擎 {#alias-table-engine} -`Alias` 引擎会创建一个指向另一张表的代理表。所有读写操作都会转发到目标表,而别名表本身不存储任何数据,只维护对目标表的引用。 + +`Alias` 引擎会创建指向另一张表的代理。所有读写操作都会被转发到目标表,而别名表本身不存储任何数据,只维护对目标表的引用。 +:::info +这是一个实验性特性,在未来版本中可能会以不向后兼容的方式发生变更。 +要启用 Alias 表引擎,请通过设置 [allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine)。 +输入命令 `set allow_experimental_alias_table_engine = 1`。 +::: -## 创建表 +## 创建表 {#creating-a-table} ```sql -CREATE TABLE [数据库名.]别名表名 -ENGINE = Alias(目标表) +CREATE TABLE [db_name.]alias_name +ENGINE = Alias(target_table) ``` 或者显式地指定数据库名称: @@ -30,20 +37,19 @@ ENGINE = Alias(target_db, target_table) ``` :::note -`Alias` 表不支持显式定义列。列会自动从目标表继承,这可以确保别名表始终与目标表的表结构保持一致。 +`Alias` 表不支持显式定义列。列会自动从目标表继承,从而确保该别名表始终与目标表的 schema 保持一致。 ::: ## 引擎参数 {#engine-parameters} -- **`target_db (optional)`** — 目标表所在数据库的名称。 -- **`target_table`** — 目标表名称。 - - +- **`target_db(可选)`** — 包含目标表的数据库名称。 +- **`target_table`** — 目标表的名称。 ## 支持的操作 {#supported-operations} `Alias` 表引擎支持所有主要操作。 + ### 目标表上的操作 {#operations-on-target} 这些操作会被转发到目标表: @@ -52,29 +58,27 @@ ENGINE = Alias(target_db, target_table) |-----------|---------|-------------| | `SELECT` | ✅ | 从目标表读取数据 | | `INSERT` | ✅ | 向目标表写入数据 | -| `INSERT SELECT` | ✅ | 向目标表执行批量插入 | +| `INSERT SELECT` | ✅ | 批量向目标表插入数据 | | `ALTER TABLE ADD COLUMN` | ✅ | 向目标表添加列 | -| `ALTER TABLE MODIFY SETTING` | ✅ | 修改目标表设置 | +| `ALTER TABLE MODIFY SETTING` | ✅ | 修改目标表的设置 | | `ALTER TABLE PARTITION` | ✅ | 在目标表上执行分区操作(DETACH/ATTACH/DROP) | -| `ALTER TABLE UPDATE` | ✅ | 更新目标表中的行(变更,mutation) | -| `ALTER TABLE DELETE` | ✅ | 从目标表删除行(变更,mutation) | -| `OPTIMIZE TABLE` | ✅ | 优化目标表(合并数据分片) | +| `ALTER TABLE UPDATE` | ✅ | 更新目标表中的行(mutation 变更) | +| `ALTER TABLE DELETE` | ✅ | 从目标表删除行(mutation 变更) | +| `OPTIMIZE TABLE` | ✅ | 优化目标表(合并数据片段) | | `TRUNCATE TABLE` | ✅ | 截断目标表 | ### 对别名本身的操作 {#operations-on-alias} -这些操作只影响别名,**不会**影响目标表: +这些操作只会影响别名,**不会**影响目标表: -| Operation | Support | Description | +| 操作 | 支持情况 | 描述 | |-----------|---------|-------------| | `DROP TABLE` | ✅ | 仅删除别名,目标表保持不变 | | `RENAME TABLE` | ✅ | 仅重命名别名,目标表保持不变 | +## 使用示例 {#usage-examples} - -## 用法示例 - -### 创建基本别名 +### 创建基本别名 {#basic-alias-creation} 在同一数据库中创建一个简单的别名: @@ -104,9 +108,10 @@ SELECT * FROM data_alias; └────┴──────┴───────┘ ``` -### 跨数据库别名 -创建一个指向其他数据库中某张表的别名: +### 跨数据库别名 {#cross-database-alias} + +创建一个指向不同数据库中某个表的别名: ```sql -- 创建数据库 @@ -121,20 +126,21 @@ CREATE TABLE db1.events ( ) ENGINE = MergeTree ORDER BY timestamp; --- 在 db2 中创建指向 db1.events 的别名 +-- 在 db2 中创建指向 db1.events 的别名表 CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); -- 或使用 database.table 格式 CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); --- 两个别名的工作方式相同 +-- 两个别名表的功能完全相同 INSERT INTO db2.events_alias VALUES (now(), 'click', 100); SELECT * FROM db2.events_alias2; ``` -### 通过别名进行写入操作 -所有写入操作都会转发到目标表: +### 通过别名执行写入操作 {#write-operations} + +经由别名的所有写入操作都会被转发到其目标表: ```sql CREATE TABLE metrics ( @@ -146,25 +152,26 @@ ORDER BY ts; CREATE TABLE metrics_alias ENGINE = Alias('metrics'); --- 通过别名插入数据 +-- 通过别名插入 INSERT INTO metrics_alias VALUES (now(), 'cpu_usage', 45.2), (now(), 'memory_usage', 78.5); --- 使用 SELECT 语句插入数据 +-- 使用 SELECT 插入 INSERT INTO metrics_alias SELECT now(), 'disk_usage', number * 10 FROM system.numbers LIMIT 5; --- 验证数据已写入目标表 +-- 验证目标表中的数据 SELECT count() FROM metrics; -- 返回 7 SELECT count() FROM metrics_alias; -- 返回 7 ``` -### 表结构修改 -`ALTER` 操作会修改目标表的表结构: +### 表结构修改 {#schema-modification} + +`ALTER` 操作用于修改目标表的表结构: ```sql CREATE TABLE users ( @@ -178,7 +185,7 @@ CREATE TABLE users_alias ENGINE = Alias('users'); -- 通过别名添加列 ALTER TABLE users_alias ADD COLUMN email String DEFAULT ''; --- 列被添加到目标表 +-- 列已添加至目标表 DESCRIBE users; ``` @@ -190,9 +197,10 @@ DESCRIBE users; └───────┴────────┴──────────────┴────────────────────┘ ``` -### 数据变更 -支持 `UPDATE` 和 `DELETE` 操作: +### 数据变更 {#data-mutations} + +支持 UPDATE 和 DELETE 操作: ```sql CREATE TABLE products ( @@ -216,7 +224,7 @@ ALTER TABLE products_alias UPDATE price = price * 1.1 WHERE status = 'active'; -- 通过别名进行删除 ALTER TABLE products_alias DELETE WHERE status = 'inactive'; --- 变更将应用到目标表 +-- 更改将应用到目标表 SELECT * FROM products ORDER BY id; ``` @@ -227,10 +235,10 @@ SELECT * FROM products ORDER BY id; └────┴──────────┴───────┴────────┘ ``` -### 分区操作 -对于分区表,分区操作会转发: +### 分区操作 {#partition-operations} +对于分区表,分区操作将被转发: ```sql CREATE TABLE logs ( @@ -259,9 +267,10 @@ ALTER TABLE logs_alias ATTACH PARTITION '202402'; SELECT count() FROM logs_alias; -- 返回 3 ``` -### 表优化 -OPTIMIZE 操作会在目标表中合并数据分片: +### 表优化 {#table-optimization} + +对目标表中的分片执行合并优化操作: ```sql CREATE TABLE events ( @@ -272,28 +281,29 @@ ORDER BY id; CREATE TABLE events_alias ENGINE = Alias('events'); --- 多次插入创建多个数据部分 +-- 多次插入创建多个部分 INSERT INTO events_alias VALUES (1, 'data1'); INSERT INTO events_alias VALUES (2, 'data2'); INSERT INTO events_alias VALUES (3, 'data3'); --- 检查数据部分数量 +-- 检查部分数量 SELECT count() FROM system.parts WHERE database = currentDatabase() AND table = 'events' AND active; --- 通过别名表优化 +-- 通过别名优化 OPTIMIZE TABLE events_alias FINAL; --- 数据部分在目标表中合并 +-- 部分在目标表中合并 SELECT count() FROM system.parts WHERE database = currentDatabase() AND table = 'events' AND active; -- 返回 1 ``` -### 别名管理 + +### 别名管理 {#alias-management} 可以分别对别名进行重命名或删除: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md index 531e2a971d4..92663abc42d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md @@ -7,9 +7,7 @@ title: 'Buffer 表引擎' doc_type: 'reference' --- - - -# Buffer 表引擎 +# Buffer 表引擎 {#buffer-table-engine} 在写入时先将要写入的数据缓存在 RAM 中,并定期刷新到另一张表。读取时会同时从缓冲区和另一张表中读取数据。 @@ -21,27 +19,27 @@ doc_type: 'reference' Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) ``` -### 引擎参数 +### 引擎参数 {#engine-parameters} -#### `database` +#### `database` {#database} `database` – 数据库名称。可以使用 `currentDatabase()` 或其他返回字符串的常量表达式。 -#### `table` +#### `table` {#table} `table` – 要将数据刷新到的目标表。 -#### `num_layers` +#### `num_layers` {#num_layers} `num_layers` – 并行层数。从物理上看,该表将表示为 `num_layers` 个彼此独立的缓冲区。 -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, 和 `max_bytes` +#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, 和 `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} 用于从缓冲区刷新数据的触发条件。 -### 可选引擎参数 +### 可选引擎参数 {#optional-engine-parameters} -#### `flush_time`, `flush_rows`, 和 `flush_bytes` +#### `flush_time`, `flush_rows`, 和 `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} 在后台从缓冲区刷新数据的触发条件(省略或设为零表示没有 `flush*` 参数)。 @@ -49,15 +47,15 @@ Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_ 另外,如果至少一个 `flush*` 条件满足,就会在后台发起一次刷新操作。这与 `max*` 不同,因为 `flush*` 允许单独配置后台刷新,以避免对写入 Buffer 表的 `INSERT` 查询增加延迟。 -#### `min_time`, `max_time`, 和 `flush_time` +#### `min_time`, `max_time`, 和 `flush_time` {#min_time-max_time-and-flush_time} 从第一次写入缓冲区开始计算的时间(秒)条件。 -#### `min_rows`, `max_rows`, 和 `flush_rows` +#### `min_rows`, `max_rows`, 和 `flush_rows` {#min_rows-max_rows-and-flush_rows} 缓冲区中行数的条件。 -#### `min_bytes`, `max_bytes`, 和 `flush_bytes` +#### `min_bytes`, `max_bytes`, 和 `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} 缓冲区中字节数的条件。 @@ -84,7 +82,6 @@ CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, 可以为数据库名和表名设置用单引号括起来的空字符串。这表示不存在目标表。在这种情况下,当达到数据刷新条件时,缓冲区会直接被清空。这对于在内存中保留一个数据时间窗口可能很有用。 - 从 Buffer 表中读取时,数据会同时从缓冲区和目标表(如果存在)中进行处理。 注意,Buffer 表不支持索引。换句话说,缓冲区中的数据会被全量扫描,这在缓冲区很大时可能会比较慢。(对于从属表中的数据,将使用该表所支持的索引。) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md index e3fe4f14967..b01ccb05e74 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md @@ -8,15 +8,11 @@ title: 'Dictionary 表引擎' doc_type: 'reference' --- - - -# Dictionary 表引擎 +# Dictionary 表引擎 {#dictionary-table-engine} `Dictionary` 引擎将 [dictionary](../../../sql-reference/dictionaries/index.md) 数据显示为 ClickHouse 表。 - - -## 示例 +## 示例 {#example} 例如,假设有一个名为 `products` 的字典,其配置如下: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md index 03b80031c73..4378297fbb4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# 分布式表引擎 +# 分布式表引擎 {#distributed-table-engine} :::warning 在 Cloud 中使用 Distributed 引擎 要在 ClickHouse Cloud 中创建分布式表引擎,可以使用 [`remote` 和 `remoteSecure`](../../../sql-reference/table-functions/remote) 表函数。 @@ -21,7 +21,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#distributed-creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -33,7 +33,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] [SETTINGS name=value, ...] ``` -### 基于现有表 +### 基于现有表 {#distributed-from-a-table} 当 `Distributed` 表指向当前服务器上的某个表时,你可以沿用该表的表结构: @@ -41,7 +41,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] ``` -### 分布式参数 +### 分布式参数 {#distributed-parameters} | Parameter | Description | | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | @@ -56,7 +56,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 * [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) 设置 * [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) 使用示例 -### 分布式设置 +### 分布式设置 {#distributed-settings} | Setting | Description | Default value | @@ -102,7 +102,7 @@ SETTINGS 在数据库名的位置上,你可以使用返回字符串的常量表达式。例如:`currentDatabase()`。 -## 集群 +## 集群 {#distributed-clusters} 集群是在[服务器配置文件](../../../operations/configuration-files.md)中配置的: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md index 7a5c4cf69ce..d467f21d67f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md @@ -7,20 +7,16 @@ title: 'Executable 和 ExecutablePool 表引擎' doc_type: 'reference' --- - - -# Executable 和 ExecutablePool 表引擎 +# Executable 和 ExecutablePool 表引擎 {#executable-and-executablepool-table-engines} `Executable` 和 `ExecutablePool` 表引擎允许你定义一张表,其行由你编写的脚本生成(通过向 **stdout** 写入行)。可执行脚本存储在 `users_scripts` 目录中,并且可以从任意数据源读取数据。 -- `Executable` 表:每次查询都会运行脚本 -- `ExecutablePool` 表:维护一个持久进程池,并从池中获取进程来执行读取操作 +* `Executable` 表:每次查询都会运行脚本 +* `ExecutablePool` 表:维护一个持久进程池,并从池中获取进程来执行读取操作 你还可以选择性地添加一个或多个输入查询,将其结果以流式方式写入 **stdin**,供脚本读取。 - - -## 创建 `Executable` 表 +## 创建 `Executable` 表 {#creating-an-executable-table} `Executable` 表引擎需要两个参数:脚本名称和输入数据的格式。你还可以可选地传入一个或多个输入查询: @@ -102,8 +98,7 @@ SELECT * FROM my_executable_table └───┴────────────┘ ``` - -## 将查询结果传递给脚本 +## 将查询结果传递给脚本 {#passing-query-results-to-a-script} Hacker News 网站的用户会留下评论。Python 提供了一个自然语言处理工具包(`nltk`),其中的 `SentimentIntensityAnalyzer` 可用于判断评论是正面、负面还是中性,并为其打分,分值范围在 -1(极度负面评论)到 1(极度正面评论)之间。我们来创建一个 `Executable` 表,使用 `nltk` 计算 Hacker News 评论的情感。 @@ -177,7 +172,6 @@ FROM sentiment 响应如下: - ```response ┌───────id─┬─情感值────┐ │ 7398199 │ 0.4404 │ @@ -203,8 +197,7 @@ FROM sentiment └──────────┴───────────┘ ``` - -## 创建 `ExecutablePool` 表 +## 创建 `ExecutablePool` 表 {#creating-an-executablepool-table} `ExecutablePool` 的语法与 `Executable` 类似,但 `ExecutablePool` 表有几个特有的相关设置: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md index 0353a40ee42..5685af197ad 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md @@ -7,7 +7,7 @@ title: '用于查询处理的外部数据' doc_type: 'reference' --- -# 用于查询处理的外部数据 +# 用于查询处理的外部数据 {#external-data-for-query-processing} ClickHouse 允许在发送 `SELECT` 查询时,将查询处理所需的数据一并发送到服务器。此数据会被放入一张临时表(参见“临时表”一节),并且可以在查询中使用(例如用于 `IN` 运算符)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md index e916851dad6..1b775e7e30d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# File 表引擎 +# File 表引擎 {#file-table-engine} `File` 表引擎将数据保存在一个文件中,文件使用受支持的[文件格式](/interfaces/formats#formats-overview)之一(如 `TabSeparated`、`Native` 等)。 @@ -25,7 +25,7 @@ doc_type: 'reference' -## 在 ClickHouse 服务器中的使用 +## 在 ClickHouse 服务器中的使用 {#usage-in-clickhouse-server} ```sql File(Format) @@ -44,7 +44,7 @@ ClickHouse 不允许为 `File` 指定文件系统路径。它将使用服务器 ::: -## 示例 +## 示例 {#example} **1.** 创建 `file_engine_table` 表: @@ -76,7 +76,7 @@ SELECT * FROM file_engine_table ``` -## 在 ClickHouse-local 中的用法 +## 在 ClickHouse-local 中的用法 {#usage-in-clickhouse-local} 在 [clickhouse-local](../../../operations/utilities/clickhouse-local.md) 中,File 引擎除了 `Format` 外还可以接收文件路径。可以使用数字或人类可读的名称(例如 `0` 或 `stdin`、`1` 或 `stdout`)来指定默认输入/输出流。可以根据额外的引擎参数或文件扩展名(`gz`、`br` 或 `xz`)来读写压缩文件。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md index 34e711e0ae7..d2b5a7fdf9e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md @@ -20,7 +20,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -56,7 +56,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] * `handle_error_mode` — FileLog 引擎的错误处理方式。可选值:`default`(如果解析消息失败则抛出异常)、`stream`(异常信息和原始消息将保存在虚拟列 `_error` 和 `_raw_message` 中)。 -## 描述 +## 描述 {#description} 已送达的记录会被自动跟踪,因此日志文件中的每条记录只会被计数一次。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md index 8b101c88712..392d398434f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# GenerateRandom 表引擎 +# GenerateRandom 表引擎 {#generaterandom-table-engine} `GenerateRandom` 表引擎根据给定的表结构生成随机数据。 @@ -20,7 +20,7 @@ doc_type: 'reference' -## 在 ClickHouse Server 中的使用 +## 在 ClickHouse Server 中的使用 {#usage-in-clickhouse-server} ```sql ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) @@ -33,7 +33,7 @@ ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) 它支持所有可以存储在表中的 [DataTypes](../../../sql-reference/data-types/index.md),`AggregateFunction` 类型除外。 -## 示例 +## 示例 {#example} **1.** 创建 `generate_engine_table` 表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md index 52362661982..6323ae0b209 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md @@ -7,9 +7,7 @@ title: '特殊表引擎' doc_type: 'reference' --- - - -# 特殊表引擎 +# 特殊表引擎 {#special-table-engines} 表引擎主要分为三大类: @@ -26,7 +24,6 @@ doc_type: 'reference' 如果发现错误,请直接编辑对应页面的 YAML front matter。 */ } - {/*AUTOGENERATED_START*/ } | Page | Description | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md index 4fa163e1f1a..2a243707b83 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Join 表引擎 +# Join 表引擎 {#join-table-engine} 用于 [JOIN](/sql-reference/statements/select/join) 操作的可选预构建数据结构。 @@ -19,7 +19,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -115,7 +115,7 @@ CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -## 用法示例 +## 用法示例 {#example} 创建左侧的表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md index 6c33ab3835a..1a720510d55 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# KeeperMap 表引擎 +# KeeperMap 表引擎 {#keepermap-table-engine} 此引擎允许你将 Keeper/ZooKeeper 集群用作一致性的键值存储,支持线性化写入和顺序一致的读取。 @@ -26,7 +26,7 @@ doc_type: 'reference' 其中 `path` 可以是任意其他有效的 ZooKeeper 路径。 -## 创建数据表 +## 创建数据表 {#creating-a-table} ```sql CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] @@ -80,9 +80,9 @@ PRIMARY KEY key 当然,也可以在彼此无关联的 ClickHouse 实例上,手动使用相同路径运行 `CREATE TABLE`,以达到相同的数据共享效果。 -## 支持的操作 +## 支持的操作 {#supported-operations} -### 插入 +### 插入 {#inserts} 当向 `KeeperMap` 插入新行时,如果键不存在,则会为该键创建一个新条目。 如果键已存在且 `keeper_map_strict_mode` 被设为 `true`,则会抛出异常;否则,该键对应的值将被覆盖。 @@ -93,7 +93,7 @@ PRIMARY KEY key INSERT INTO keeper_map_table VALUES ('some key', 1, 'value', 3.2); ``` -### 删除 +### 删除 {#deletes} 可以使用 `DELETE` 查询或 `TRUNCATE` 删除行。 如果键存在,并且将 `keeper_map_strict_mode` 设置为 `true`,则只有在能够以原子方式执行时,获取和删除数据才会成功。 @@ -110,7 +110,7 @@ ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; TRUNCATE TABLE keeper_map_table; ``` -### 更新 +### 更新 {#updates} 可以使用 `ALTER TABLE` 查询来更新值。主键不可更新。 如果将 `keeper_map_strict_mode` 设置为 `true`,只有在以原子方式执行时,读取和更新数据才会成功。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md index 71e37e79d82..91317761af5 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Memory 表引擎 +# Memory 表引擎 {#memory-table-engine} :::note 在 ClickHouse Cloud 上使用 Memory 表引擎时,数据出于设计原因不会在所有节点之间复制。若要保证所有查询都被路由到同一节点,并使 Memory 表引擎按预期工作,可以采用以下任一方式: @@ -48,7 +48,7 @@ Memory 引擎被系统用于带有外部查询数据的临时表(参见“Exte -## 使用说明 +## 使用说明 {#usage} **初始化配置** @@ -65,7 +65,7 @@ ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 100 **注意:** `bytes` 和 `rows` 的封顶参数可以同时设置,但会始终遵守由 `max` 和 `min` 所定义的最低限制。 -## 示例 +## 示例 {#examples} ```sql CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md index 9c836320bcd..facd208ad44 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Merge 表引擎 +# Merge 表引擎 {#merge-table-engine} `Merge` 引擎(不要与 `MergeTree` 混淆)自身不存储数据,而是允许同时从任意数量的其他表中读取数据。 @@ -17,7 +17,7 @@ doc_type: 'reference' -## 创建表 +## 创建表 {#creating-a-table} ```sql CREATE TABLE ... 引擎=Merge(db_name, tables_regexp) @@ -51,7 +51,7 @@ CREATE TABLE ... 引擎=Merge(db_name, tables_regexp) -## 示例 +## 示例 {#examples} **示例 1** diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md index 5747c28ffc3..c2cb4630004 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md @@ -7,7 +7,7 @@ title: 'Null 表引擎' doc_type: 'reference' --- -# Null 表引擎 +# Null 表引擎 {#null-table-engine} 当向 `Null` 表写入数据时,这些数据会被忽略。 当从 `Null` 表读取数据时,返回结果是空的。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md index c173222b514..8466531679c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# Set 表引擎 +# Set 表引擎 {#set-table-engine} :::note 在 ClickHouse Cloud 中,如果服务创建时使用的版本早于 25.4,则需要使用 `SET compatibility=25.4` 将兼容性设置为至少 25.4。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md index 33ab7358a1c..6e23b82cf74 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md @@ -9,7 +9,7 @@ doc_type: 'reference' -# URL 表引擎 +# URL 表引擎 {#url-table-engine} 对远程 HTTP/HTTPS 服务器进行数据查询和写入。该引擎类似于 [File](../../../engines/table-engines/special/file.md) 引擎。 @@ -52,7 +52,7 @@ doc_type: 'reference' -## 示例 +## 示例 {#example} **1.** 在服务器上创建一个 `url_engine_table` 表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md index 8fc437b3f98..ae9b9d56a94 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md @@ -7,6 +7,6 @@ title: 'View 表引擎' doc_type: 'reference' --- -# View 表引擎 +# View 表引擎 {#view-table-engine} 用于实现视图(更多信息请参见 `CREATE VIEW` 查询)。它本身不存储数据,而只存储指定的 `SELECT` 查询。在从该表读取数据时,会运行此查询(并从查询中删除所有不需要的列)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/concurrency.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/concurrency.md index f9d25f4415e..b42acc28c7e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/concurrency.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/concurrency.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['并发', 'QPS'] --- -# ClickHouse 是否支持高频并发查询? +# ClickHouse 是否支持高频并发查询? {#does-clickhouse-support-frequent-concurrent-queries} ClickHouse 专为可直接对外服务的实时分析型应用而设计。它可以在 PB 级数据库上,将历史数据与实时写入相结合,以低延迟(小于 10 毫秒)和高并发(每秒超过 10,000 次查询)处理分析查询。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/cost-based.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/cost-based.md index 891f7fb97cd..d17a372243b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/cost-based.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/cost-based.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['CBE', 'optimizer'] --- -# ClickHouse 是否有基于成本的优化器? +# ClickHouse 是否有基于成本的优化器? {#does-clickhouse-have-a-cost-based-optimizer} ClickHouse 具备一些局部的基于成本的优化机制,例如:读取列的顺序由从磁盘读取压缩数据范围的成本来决定。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/datalake.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/datalake.md index 54e906551f7..0daca22a540 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/datalake.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/datalake.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['数据湖', '湖仓'] --- -# ClickHouse 是否支持数据湖? +# ClickHouse 是否支持数据湖? {#does-clickhouse-support-data-lakes} ClickHouse 支持数据湖,包括 Iceberg、Delta Lake、Apache Hudi、Apache Paimon、Hive。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/dependencies.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/dependencies.md index b2bcbedcb98..6ee40440f69 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/dependencies.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/dependencies.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['依赖', '第三方'] --- -# 运行 ClickHouse 需要哪些第三方依赖? +# 运行 ClickHouse 需要哪些第三方依赖? {#what-are-the-3rd-party-dependencies-for-running-clickhouse} ClickHouse 没有任何运行时依赖。它以单个二进制可执行文件的形式发布,完全自包含。该应用程序提供集群的全部功能:处理查询、作为集群中的工作节点、作为提供 Raft 共识算法的协调系统,以及作为客户端或本地查询引擎。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md index 6c151dca2e3..ead08a51735 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/distributed-join.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['分布式', 'JOIN'] --- -# ClickHouse 是否支持分布式 JOIN? +# ClickHouse 是否支持分布式 JOIN? {#does-clickhouse-support-distributed-join} ClickHouse 在集群上支持分布式 JOIN。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/federated.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/federated.md index b979afe7b1e..3f3090a7e26 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/federated.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/federated.md @@ -8,22 +8,22 @@ doc_type: 'reference' keywords: ['federated', 'hybrid', 'postgres', 'mysql', 'sqlite', 'odbc', 'jdbc'] --- -# ClickHouse 是否支持联邦查询? +# ClickHouse 是否支持联邦查询? {#does-clickhouse-support-federated-queries} 在分析型数据库中,ClickHouse 对联邦查询和混合查询执行的支持最为全面。 它支持查询外部数据库: -- PostgreSQL -- MySQL -- MongoDB -- Redis -- 任何 ODBC 数据源 -- 任何 JDBC 数据源 -- 任何 Arrow Flight 数据源 -- 流式数据源,例如 Kafka 和 RabbitMQ -- 数据湖,例如 Iceberg、Delta Lake、Apache Hudi、Apache Paimon -- 位于共享存储上的外部文件,例如 AWS S3、GCS、Minio、Cloudflare R2、Azure Blob Storage、Alicloud OSS、Tencent COS,以及本地存储,支持多种数据格式 +* PostgreSQL +* MySQL +* MongoDB +* Redis +* 任何 ODBC 数据源 +* 任何 JDBC 数据源 +* 任何 Arrow Flight 数据源 +* 流式数据源,例如 Kafka 和 RabbitMQ +* 数据湖,例如 Iceberg、Delta Lake、Apache Hudi、Apache Paimon +* 位于共享存储上的外部文件,例如 AWS S3、GCS、Minio、Cloudflare R2、Azure Blob Storage、Alicloud OSS、Tencent COS,以及本地存储,支持多种数据格式 ClickHouse 可以在单个查询中对多个不同的数据源进行 join。它还提供混合查询执行选项,既利用本地资源,又将部分查询卸载到远程机器上。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/index.md index 2ba3a15c958..461eedccbf5 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/index.md @@ -8,18 +8,18 @@ description: '列出关于 ClickHouse 常见问题的索引页面' doc_type: 'landing-page' --- -# 关于 ClickHouse 的常见问题 +# 关于 ClickHouse 的常见问题 {#general-questions-about-clickhouse} -- [什么是 ClickHouse?](../../intro.md) -- [为什么 ClickHouse 如此之快?](../../concepts/why-clickhouse-is-so-fast.mdx) -- [谁在使用 ClickHouse?](../../faq/general/who-is-using-clickhouse.md) -- [“ClickHouse” 是什么意思?](../../faq/general/dbms-naming.md) -- [“Не тормозит” 是什么意思?](../../faq/general/ne-tormozit.md) -- [什么是 OLAP?](../../faq/general/olap.md) -- [什么是列式数据库?](../../faq/general/columnar-database.md) -- [如何选择主键?](../../guides/best-practices/sparse-primary-indexes.md) -- [为什么不使用类似 MapReduce 的方案?](../../faq/general/mapreduce.md) -- [如何向 ClickHouse 贡献代码?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) +* [什么是 ClickHouse?](../../intro.md) +* [为什么 ClickHouse 如此之快?](../../concepts/why-clickhouse-is-so-fast.mdx) +* [谁在使用 ClickHouse?](../../faq/general/who-is-using-clickhouse.md) +* [“ClickHouse” 是什么意思?](../../faq/general/dbms-naming.md) +* [“Не тормозит” 是什么意思?](../../faq/general/ne-tormozit.md) +* [什么是 OLAP?](../../faq/general/olap.md) +* [什么是列式数据库?](../../faq/general/columnar-database.md) +* [如何选择主键?](../../guides/best-practices/sparse-primary-indexes.md) +* [为什么不使用类似 MapReduce 的方案?](../../faq/general/mapreduce.md) +* [如何向 ClickHouse 贡献代码?](/knowledgebase/how-do-i-contribute-code-to-clickhouse) :::info 没有找到所需内容? 请查阅我们的[知识库](/knowledgebase/),并浏览文档中更多实用的文章。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/sql.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/sql.md index 838674e2e54..bb3c6113bb2 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/sql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/sql.md @@ -8,46 +8,46 @@ doc_type: 'reference' keywords: ['SQL 语法', 'ANSI SQL'] --- -# ClickHouse 支持哪些 SQL 语法? +# ClickHouse 支持哪些 SQL 语法? {#what-sql-syntax-does-clickhouse-support} ClickHouse 对 SQL 语法提供了完整支持,包括以下特性: -- SQL/JSON 和 JSON 数据类型(SQL-2023) -- 窗口函数(SQL-2003) -- 公用表表达式和递归查询(SQL-1999) -- ROLLUP、CUBE 和 GROUPING SETS(SQL-1999) -- 对 RBAC 的完整支持(SQL-1999) -- 关联子查询(SQL-1992) +* SQL/JSON 和 JSON 数据类型(SQL-2023) +* 窗口函数(SQL-2003) +* 公用表表达式和递归查询(SQL-1999) +* ROLLUP、CUBE 和 GROUPING SETS(SQL-1999) +* 对 RBAC 的完整支持(SQL-1999) +* 关联子查询(SQL-1992) 这一支持已经通过 TPC-H 和 TPC-DS 基准测试以及 SQLTest 得到验证。 ClickHouse 在许多特性被 ISO/IEC 标准化之前就已经引入了它们,例如: -- 条件聚合函数 -- `any` 聚合函数 -- `least` 和 `greatest` -- `GROUP BY ALL` -- 对别名的扩展使用 -- 数字字面量中的下划线 +* 条件聚合函数 +* `any` 聚合函数 +* `least` 和 `greatest` +* `GROUP BY ALL` +* 对别名的扩展使用 +* 数字字面量中的下划线 ClickHouse 通过引入大量提升使用体验的改进来扩展 SQL: -- 不受限制地使用别名 -- WITH 子句中的别名 -- 聚合函数组合器 -- 参数化聚合函数 -- 近似聚合函数 -- 原生大整数数值类型和扩展精度小数类型 -- 用于数组操作的高阶函数 -- ARRAY JOIN 子句和 arrayJoin 函数 -- 数组聚合 -- LIMIT BY 子句 -- GROUP BY WITH TOTALS -- AS OF JOIN -- ANY/ALL JOIN -- 更自然的 JSON 语法 -- 列表中的尾随逗号 -- FROM ... SELECT 子句顺序 -- 类型安全的查询参数和参数化视图 +* 不受限制地使用别名 +* WITH 子句中的别名 +* 聚合函数组合器 +* 参数化聚合函数 +* 近似聚合函数 +* 原生大整数数值类型和扩展精度小数类型 +* 用于数组操作的高阶函数 +* ARRAY JOIN 子句和 arrayJoin 函数 +* 数组聚合 +* LIMIT BY 子句 +* GROUP BY WITH TOTALS +* AS OF JOIN +* ANY/ALL JOIN +* 更自然的 JSON 语法 +* 列表中的尾随逗号 +* FROM ... SELECT 子句顺序 +* 类型安全的查询参数和参数化视图 其中一些特性有机会被纳入即将发布的 SQL 标准,而它们已经可以在 ClickHouse 中使用。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/updates.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/updates.md index 9653ddd0d42..b25e3c99795 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/updates.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/general/updates.md @@ -8,7 +8,7 @@ doc_type: 'reference' keywords: ['更新', '实时'] --- -# ClickHouse 是否支持实时更新? +# ClickHouse 是否支持实时更新? {#does-clickhouse-support-real-time-updates} ClickHouse 支持 `UPDATE` 语句,并且能够以与执行 `INSERT` 一样快的速度执行实时更新。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/index.md index 047ecfe9ddd..c3b1b40e500 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/index.md @@ -8,15 +8,15 @@ description: '汇总与 ClickHouse 集成到其他系统相关问题的落地页 doc_type: 'landing-page' --- -# 关于将 ClickHouse 与其他系统集成的问题 +# 关于将 ClickHouse 与其他系统集成的问题 {#questions-about-integrating-clickhouse-and-other-systems} -- [如何将 ClickHouse 中的数据导出到文件?](https://clickhouse.com/docs/knowledgebase/file-export) -- [如何将 JSON 导入 ClickHouse?](/integrations/data-ingestion/data-formats/json/intro.md) -- [如何将 Kafka 连接到 ClickHouse?](/integrations/data-ingestion/kafka/index.md) -- [可以将 Java 应用程序连接到 ClickHouse 吗?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) -- [ClickHouse 可以读取 MySQL 中的表吗?](/integrations/mysql) -- [ClickHouse 可以读取 PostgreSQL 中的表吗?](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) -- [通过 ODBC 连接 Oracle 时,如果遇到编码问题该怎么办?](/faq/integration/oracle-odbc.md) +* [如何将 ClickHouse 中的数据导出到文件?](https://clickhouse.com/docs/knowledgebase/file-export) +* [如何将 JSON 导入 ClickHouse?](/integrations/data-ingestion/data-formats/json/intro.md) +* [如何将 Kafka 连接到 ClickHouse?](/integrations/data-ingestion/kafka/index.md) +* [可以将 Java 应用程序连接到 ClickHouse 吗?](/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md) +* [ClickHouse 可以读取 MySQL 中的表吗?](/integrations/mysql) +* [ClickHouse 可以读取 PostgreSQL 中的表吗?](/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md) +* [通过 ODBC 连接 Oracle 时,如果遇到编码问题该怎么办?](/faq/integration/oracle-odbc.md) :::info 没有找到所需内容? 请查看我们的[知识库](/knowledgebase/),以及浏览本文档中许多其他有用的文章。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/json-import.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/json-import.md index 94781804f48..2d332231094 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/json-import.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/json-import.md @@ -16,7 +16,7 @@ ClickHouse 支持多种[输入和输出数据格式](/interfaces/formats)。其 -## 示例 +## 示例 {#examples} 使用 [HTTP 接口](../../interfaces/http.md): diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md index f59ca525691..7db7500b5dc 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/integration/oracle-odbc.md @@ -8,9 +8,7 @@ doc_type: 'guide' keywords: ['oracle', 'odbc', '编码', '集成', '外部字典'] --- - - -# 通过 ODBC 使用 Oracle 时如果遇到编码问题怎么办? +# 通过 ODBC 使用 Oracle 时如果遇到编码问题怎么办? {#oracle-odbc-encodings} 如果通过 Oracle ODBC 驱动将 Oracle 用作 ClickHouse 外部字典的数据源,则需要在 `/etc/default/clickhouse` 中为 `NLS_LANG` 环境变量设置正确的值。有关更多信息,请参阅 [Oracle NLS_LANG 常见问题解答](https://www.oracle.com/technetwork/products/globalization/nls-lang-099431.html)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md index 841024fcce6..af2f280d5be 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/delete-old-data.md @@ -30,7 +30,7 @@ TTL 不仅可以用来将数据“移动”到 [/dev/null](https://en.wikipedia. -## DELETE FROM +## DELETE FROM {#delete-from} [DELETE FROM](/sql-reference/statements/delete.md) 允许在 ClickHouse 中运行标准的 DELETE 查询。满足过滤条件的行会被标记为已删除,并且在后续的结果集中不再返回。行的清理是以异步方式执行的。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/index.md index 92deb488e62..cce5432aaff 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/operations/index.md @@ -8,16 +8,16 @@ doc_type: 'landing-page' keywords: ['运维', '管理', '部署', '集群管理', '常见问题'] --- -# 关于运维 ClickHouse 服务器和集群的问题 +# 关于运维 ClickHouse 服务器和集群的问题 {#question-about-operating-clickhouse-servers-and-clusters} -- [在生产环境中应该使用哪个 ClickHouse 版本?](/faq/operations/production.md) -- [是否可以采用存储与计算分离的方式部署 ClickHouse?](/faq/operations/separate_storage.md) -- [是否可以从 ClickHouse 表中删除旧记录?](/faq/operations/delete-old-data.md) -- [如何配置 ClickHouse Keeper?](/guides/sre/keeper/index.md) -- [ClickHouse 能否与 LDAP 集成?](/guides/sre/user-management/configuring-ldap.md) -- [如何在 ClickHouse 中配置用户、角色和权限?](/guides/sre/user-management/index.md) -- [能否在 ClickHouse 中更新或删除行?](/guides/starter_guides/mutations.md) -- [ClickHouse 是否支持多区域复制?](/faq/operations/multi-region-replication.md) +* [在生产环境中应该使用哪个 ClickHouse 版本?](/faq/operations/production.md) +* [是否可以采用存储与计算分离的方式部署 ClickHouse?](/faq/operations/separate_storage.md) +* [是否可以从 ClickHouse 表中删除旧记录?](/faq/operations/delete-old-data.md) +* [如何配置 ClickHouse Keeper?](/guides/sre/keeper/index.md) +* [ClickHouse 能否与 LDAP 集成?](/guides/sre/user-management/configuring-ldap.md) +* [如何在 ClickHouse 中配置用户、角色和权限?](/guides/sre/user-management/index.md) +* [能否在 ClickHouse 中更新或删除行?](/guides/starter_guides/mutations.md) +* [ClickHouse 是否支持多区域复制?](/faq/operations/multi-region-replication.md) :::info 没有找到需要的内容? 请查阅我们的[知识库](/knowledgebase/),并浏览文档中提供的更多实用文章。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/faq/use-cases/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/faq/use-cases/index.md index ed9687016e0..300abb822cd 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/faq/use-cases/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/faq/use-cases/index.md @@ -8,10 +8,10 @@ keywords: ['ClickHouse 使用场景', '时间序列数据库', '键值存储', ' doc_type: 'landing-page' --- -# 关于 ClickHouse 使用场景的常见问题 +# 关于 ClickHouse 使用场景的常见问题 {#questions-about-clickhouse-use-cases} -- [我可以将 ClickHouse 用作时序数据库吗?](/knowledgebase/time-series) -- [我可以将 ClickHouse 用作键值存储吗?](/knowledgebase/key-value) +* [我可以将 ClickHouse 用作时序数据库吗?](/knowledgebase/time-series) +* [我可以将 ClickHouse 用作键值存储吗?](/knowledgebase/key-value) :::info 没有找到您需要的内容? 请查看我们的[知识库](/knowledgebase/),并浏览文档中大量实用的文章。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md index a8dcd9bb096..a18d278f198 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/amazon-reviews.md @@ -11,10 +11,10 @@ keywords: ['Amazon reviews', 'customer reviews dataset', 'e-commerce data', 'exa :::note 下面的查询是在 **Production** 环境的 ClickHouse Cloud 实例上执行的。更多信息请参阅 -["Playground 规格说明"](/getting-started/playground#specifications)。 +["Playground 规格说明"](/getting-started/playground#specifications)。 ::: -## 加载数据集 +## 加载数据集 {#loading-the-dataset} 1. 在不将数据插入 ClickHouse 的情况下,我们可以直接在原处对其进行查询。先取出几行数据,看看它们的样子: @@ -122,7 +122,6 @@ FROM s3Cluster('default', 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/amazon_reviews/amazon_reviews_*.snappy.parquet') ``` - :::tip 在 ClickHouse Cloud 中,集群名称为 `default`。请将 `default` 更改为你的集群名称……或者如果你没有集群,可以使用 `s3` 表函数(而不是 `s3Cluster`)。 ::: @@ -152,8 +151,7 @@ ORDER BY size DESC 原始数据约为 70GB,在 ClickHouse 中压缩后仅占约 30GB。 - -## 示例查询 +## 示例查询 {#example-queries} 7. 现在来运行一些查询。下面是数据集中最有帮助的前 10 条评论: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md index d198ae2aec1..210d1671def 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/anon_web_analytics_metrica.md @@ -7,7 +7,7 @@ title: '匿名化网站分析' doc_type: 'guide' --- -# 匿名化的网站分析数据 +# 匿名化的网站分析数据 {#anonymized-web-analytics-data} 此数据集由两个表组成,存储匿名化的网站分析数据:一个为 hits(`hits_v1`),一个为 visits(`visits_v1`)。 @@ -15,17 +15,17 @@ doc_type: 'guide' ## 下载并摄取数据 {#download-and-ingest-the-data} -### 下载 hits TSV 压缩文件 +### 下载 hits TSV 压缩文件 {#download-the-hits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/hits/tsv/hits_v1.tsv.xz | unxz --threads=`nproc` > hits_v1.tsv -# 验证校验和 +# 验证校验和 {#validate-the-checksum} md5sum hits_v1.tsv -# 校验和应为:f3631b6295bf06989c1437491f7592cb +# 校验和应为:f3631b6295bf06989c1437491f7592cb {#checksum-should-be-equal-to-f3631b6295bf06989c1437491f7592cb} ``` -### 创建数据库和表 +### 创建数据库和表 {#create-the-database-and-table} ```bash clickhouse-client --query "CREATE DATABASE IF NOT EXISTS datasets" @@ -45,7 +45,7 @@ clickhouse-client --query="CREATE TABLE default.hits_100m_obfuscated (WatchID UI ``` -### 导入 hits 表的数据 +### 导入 hits 表的数据 {#import-the-hits-data} ```bash cat hits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.hits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -62,13 +62,13 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.hits_v1" ``` -### 下载压缩的 visits TSV 文件 +### 下载压缩的 visits TSV 文件 {#download-the-visits-compressed-tsv-file} ```bash curl https://datasets.clickhouse.com/visits/tsv/visits_v1.tsv.xz | unxz --threads=`nproc` > visits_v1.tsv -# 验证校验和 +# 验证校验和 {#validate-the-checksum} md5sum visits_v1.tsv -# 校验和应为:6dafe1a0f24e59e3fc2d0fed85601de6 +# 校验和应为:6dafe1a0f24e59e3fc2d0fed85601de6 {#checksum-should-be-equal-to-6dafe1a0f24e59e3fc2d0fed85601de6} ``` @@ -78,7 +78,7 @@ md5sum visits_v1.tsv clickhouse-client --query "CREATE TABLE datasets.visits_v1 ( CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), Goals Nested(ID UInt32, Serial UInt32, EventTime DateTime, Price Int64, OrderID String, CurrencyID UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, TraficSource Nested(ID Int8, SearchEngineID UInt16, AdvEngineID UInt8, PlaceID UInt16, SocialSourceNetworkID UInt8, Domain String, SearchPhrase String, SocialSourcePage String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), Market Nested(Type UInt8, GoalID UInt32, OrderID String, OrderPrice Int64, PP UInt32, DirectPlaceID UInt32, DirectOrderID UInt32, DirectBannerID UInt32, GoodID String, GoodName String, GoodQuantity Int32, GoodPrice Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192" ``` -### 导入访问数据 +### 导入访问数据 {#import-the-visits-data} ```bash cat visits_v1.tsv | clickhouse-client --query "INSERT INTO datasets.visits_v1 FORMAT TSV" --max_insert_block_size=100000 @@ -95,7 +95,7 @@ clickhouse-client --query "SELECT COUNT(*) FROM datasets.visits_v1" ``` -## 一个 JOIN 示例 +## 一个 JOIN 示例 {#an-example-join} hits 和 visits 数据集用于 ClickHouse 的测试例程,这是测试套件中的一个查询。其余测试在本页末尾的 *下一步* 部分中有引用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md index 738971e5045..2c008e2033e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/brown-benchmark.md @@ -94,8 +94,7 @@ clickhouse-client --query "INSERT INTO mgbench.logs2 FORMAT CSVWithNames" < mgbe clickhouse-client --query "INSERT INTO mgbench.logs3 FORMAT CSVWithNames" < mgbench3.csv ``` - -## 运行基准查询 +## 运行基准查询 {#run-benchmark-queries} ```sql USE mgbench; @@ -209,7 +208,6 @@ ORDER BY machine_name, dt; ``` - ```sql -- Q1.6: 所有文件服务器的每小时网络流量总和是多少? diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md index cce072b914e..c497446382c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/cell-towers.md @@ -7,15 +7,15 @@ keywords: ['蜂窝基站数据', '地理数据', 'OpenCelliD', '地理空间数 doc_type: '指南' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; -import ActionsMenu from '@site/docs/_snippets/_service_actions_menu.md'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; -import SupersetDocker from '@site/docs/_snippets/_add_superset_detail.md'; +import ActionsMenu from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_service_actions_menu.md'; +import SQLConsoleDetail from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; +import SupersetDocker from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_add_superset_detail.md'; import cloud_load_data_sample from '@site/static/images/_snippets/cloud-load-data-sample.png'; import cell_towers_1 from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' import add_a_database from '@site/static/images/getting-started/example-datasets/superset-add.png' @@ -29,7 +29,6 @@ import superset_radio_umts from '@site/static/images/getting-started/example-dat import superset_umts_netherlands from '@site/static/images/getting-started/example-datasets/superset-umts-netherlands.png' import superset_cell_tower_dashboard from '@site/static/images/getting-started/example-datasets/superset-cell-tower-dashboard.png' - ## 目标 {#goal} 在本指南中,您将学习如何: @@ -124,7 +123,7 @@ INSERT INTO cell_towers SELECT * FROM s3('https://datasets-documentation.s3.amaz -## 运行一些示例查询 +## 运行一些示例查询 {#examples} 1. 各类型蜂窝基站的数量: @@ -171,7 +170,6 @@ SELECT mcc, count() FROM cell_towers GROUP BY mcc ORDER BY count() DESC LIMIT 10 您可以考虑在 ClickHouse 中创建一个 [Dictionary](../../sql-reference/dictionaries/index.md) 来对这些值进行解码。 - ## 使用场景:集成地理数据 {#use-case} 使用 [`pointInPolygon`](/sql-reference/functions/geo/coordinates.md/#pointinpolygon) 函数。 @@ -266,7 +264,6 @@ WHERE pointInPolygon((lon, lat), (SELECT * FROM moscow)) 返回 1 行。耗时:0.067 秒。处理了 4328 万行,692.42 MB(每秒 6.4583 亿行,10.33 GB/秒)。 ``` - ## 查看模式 {#review-of-the-schema} 在 Superset 中构建可视化之前,请先查看你将要使用的列。此数据集主要提供全球移动蜂窝基站的位置(经度和纬度)以及无线接入技术类型。各列的说明可以在[社区论坛](https://community.opencellid.org/t/documenting-the-columns-in-the-downloadable-cells-database-csv/186)中找到。下面描述将用于构建可视化的列。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md index 49d9f674323..2ea98618f16 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/dbpedia.md @@ -15,7 +15,7 @@ doc_type: 'guide' 该数据集包含位于 [huggingface.co](https://huggingface.co/api/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M/parquet/default/train/) 上的 26 个 `Parquet` 文件。文件名为 `0.parquet`、`1.parquet`、...、`25.parquet`。要查看该数据集的一些示例记录,请访问此 [Hugging Face 页面](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-1536-1M)。 -## 创建表 +## 创建表 {#create-table} 创建 `dbpedia` 表,用于存储文章 ID、标题、正文和嵌入向量: @@ -31,7 +31,7 @@ CREATE TABLE dbpedia ``` -## 加载表 +## 加载表 {#load-table} 要从所有 Parquet 文件中加载数据集,请运行以下 Shell 命令: @@ -74,7 +74,7 @@ FROM dbpedia _最近邻_ 指的是与用户查询最相关的文档、图像或其他内容。 检索结果是生成式 AI 应用中 RAG(检索增强生成,Retrieval Augmented Generation)的关键输入。 -## 运行暴力向量相似度搜索 +## 运行暴力向量相似度搜索 {#run-a-brute-force-vector-similarity-search} KNN(k 近邻)搜索或暴力搜索是指计算数据集中每个向量与搜索嵌入向量之间的距离,然后对这些距离进行排序以获得最近邻。使用 `dbpedia` 数据集时,一种快速、直观地观察语义搜索效果的方法,是直接使用数据集本身的嵌入向量作为搜索向量。例如: @@ -116,7 +116,7 @@ LIMIT 20 同时分别记录在 OS 文件缓存为冷状态时,以及在将 `max_threads` 设置为 1 时的查询延迟,以识别实际的计算资源占用和存储带宽使用情况(从而可以将结果推算到包含数百万向量的生产数据集!) -## 构建向量相似度索引 +## 构建向量相似度索引 {#build-vector-similarity-index} 运行以下 SQL,在 `vector` 列上定义并构建向量相似度索引: @@ -131,7 +131,7 @@ ALTER TABLE dbpedia MATERIALIZE INDEX vector_index SETTINGS mutations_sync = 2; 根据可用 CPU 核心数量和存储带宽的不同,构建并保存索引可能需要几分钟时间。 -## 执行 ANN 搜索 +## 执行 ANN 搜索 {#perform-ann-search} *Approximate Nearest Neighbours*(近似最近邻,ANN)指一类技术(例如采用图结构、随机森林等特殊数据结构),可以比精确向量搜索更快地计算结果。结果的准确度通常在实际使用中已经“足够好”。许多近似算法提供参数,用于在结果准确度和搜索时间之间进行权衡和调优。 @@ -178,7 +178,7 @@ LIMIT 20 ``` -## 为搜索查询生成嵌入向量 +## 为搜索查询生成嵌入向量 {#generating-embeddings-for-search-query} 目前为止看到的相似度搜索查询,都是使用 `dbpedia` 表中已有的向量之一作为搜索向量。在实际应用中,需要根据用户输入的查询(可能是自然语言)来生成搜索向量。搜索向量应当使用与为数据集生成嵌入向量时相同的 LLM 模型生成。 @@ -220,7 +220,7 @@ while True: ``` -## 问答演示应用 +## 问答演示应用 {#q-and-a-demo-application} 上面的示例演示了使用 ClickHouse 进行语义搜索和文档检索。下面将介绍一个非常简单但具有巨大潜力的生成式 AI 示例应用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md index 7d7324534c8..6861be2c6d7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/foursquare-os-places.md @@ -24,7 +24,7 @@ import visualization_4 from '@site/static/images/getting-started/example-dataset 关于这些地点的附加元数据,例如类别和社交媒体 信息。 -## 数据探索 +## 数据探索 {#data-exploration} 为了探索这些数据,我们将使用 [`clickhouse-local`](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local),这是一个小型命令行工具, 虽然体积小巧,但提供了完整的 ClickHouse 引擎。当然,也可以使用 @@ -148,7 +148,7 @@ DESCRIBE s3('s3://fsq-os-places-us-east-1/release/dt=2025-04-08/places/parquet/* ``` -## 将数据加载到 ClickHouse 中 +## 将数据加载到 ClickHouse 中 {#loading-the-data} 如果希望将数据持久化到磁盘上,可以使用 `clickhouse-server` 或 ClickHouse Cloud。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md index ffd2cac861e..509ad631dd7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/github.md @@ -29,7 +29,7 @@ import superset_authors_matrix_v2 from '@site/static/images/getting-started/exam * `line_changes` - 2.7G - 7,535,157 行 -## 生成数据 +## 生成数据 {#generating-the-data} 此步骤为可选。我们免费提供这些数据——请参阅[下载并插入数据](#downloading-and-inserting-the-data)。 @@ -73,7 +73,7 @@ CREATE TABLE git.commits * Linux - `~/clickhouse git-import` - 160 分钟 -## 下载和插入数据 +## 下载和插入数据 {#downloading-and-inserting-the-data} 以下数据可用于复现一个可工作的环境。或者,可以在 play.clickhouse.com 中获取该数据集——参见 [Queries](#queries) 获取更多详情。 @@ -220,7 +220,7 @@ FROM s3('https://datasets-documentation.s3.amazonaws.com/github/commits/clickhou 该数据集可以在 [play.clickhouse.com](https://sql.clickhouse.com?query_id=DCQPNPAIMAQXRLHYURLKVJ) 的 `git_clickhouse` 数据库中获取。我们为所有查询都提供了指向该环境的链接,并在需要时调整数据库名称。请注意,由于数据采集时间不同,play 环境中的查询结果可能与此处展示的结果有所差异。 -### 单个文件的历史记录 +### 单个文件的历史记录 {#history-of-a-single-file} 这是最简单的查询之一。在这里我们查看 `StorageReplicatedMergeTree.cpp` 的所有提交信息。由于这些通常更值得关注,我们按最新的提交优先排序。 @@ -296,7 +296,7 @@ LIMIT 10 请注意,还有一个更复杂的查询版本,它会在考虑重命名的情况下查找文件的[逐行提交历史](#line-by-line-commit-history-of-a-file)。 -### 查找当前有效文件 +### 查找当前有效文件 {#find-the-current-active-files} 这对于后续分析很重要,因为那时我们只想考虑代码仓库中当前存在的文件。我们将这组文件近似定义为那些没有被重命名或删除(然后又被重新添加或重新命名)的文件。 @@ -428,7 +428,7 @@ git ls-files | grep -v -E 'generated\.cpp|^(contrib|docs?|website|libs/(libcityh 这些差异不应对我们的分析产生实质性影响。**欢迎提供该查询的改进版本**。 -### 列出修改最多的文件 +### 列出修改最多的文件 {#list-files-with-most-modifications} 在当前文件范围内,我们将删除和新增的行数相加,作为修改次数。 @@ -484,7 +484,7 @@ LIMIT 10 ``` -### 通常在一周中的哪一天进行代码提交? +### 通常在一周中的哪一天进行代码提交? {#what-day-of-the-week-do-commits-usually-occur} [运行](https://sql.clickhouse.com?query_id=GED2STFSYJDRAA59H8RLIV) @@ -510,7 +510,7 @@ GROUP BY dayOfWeek(time) AS day_of_week 这很合理,周五的工作效率往往会有所下降。很高兴看到大家在周末也继续提交代码!非常感谢所有贡献者! -### 子目录/文件的历史——随时间变化的行数、提交次数和贡献者数量 +### 子目录/文件的历史——随时间变化的行数、提交次数和贡献者数量 {#history-of-subdirectoryfile---number-of-lines-commits-and-contributors-over-time} 如果不进行过滤,查询结果会非常庞大,不便展示或可视化。因此,在下面的示例中,我们支持按文件或子目录进行过滤。这里我们使用 `toStartOfWeek` 函数按周分组——可根据需要进行调整。 @@ -555,7 +555,7 @@ LIMIT 10 For commits and authors -### 列出作者数量最多的文件 +### 列出作者数量最多的文件 {#list-files-with-maximum-number-of-authors} 仅限当前文件。 @@ -611,7 +611,7 @@ LIMIT 10 ``` -### 仓库中最早的代码行 +### 仓库中最早的代码行 {#oldest-lines-of-code-in-the-repository} 仅针对当前文件。 @@ -669,7 +669,7 @@ LIMIT 10 ``` -### 历史最悠久的文件 +### 历史最悠久的文件 {#files-with-longest-history} 仅包含当前仍存在的文件。 @@ -728,7 +728,7 @@ LIMIT 10 我们的核心数据结构 MergeTree 显然也在持续演进,迄今已经历了大量改进! -### 本月在文档和代码方面的贡献分布 +### 本月在文档和代码方面的贡献分布 {#distribution-of-contributors-with-respect-to-docs-and-code-over-the-month} **在数据采集过程中,由于 `docs/` 目录的提交历史非常杂乱,其变更已被过滤掉。因此,本查询结果并不完全准确。** @@ -792,7 +792,7 @@ FROM 在月底时可能会稍微多一点,但整体来看分布比较均匀。需要再次说明的是,由于在数据插入过程中应用了 docs 过滤器进行筛选,因此这些结果并不可靠。 -### 影响最广泛的作者 +### 影响最广泛的作者 {#authors-with-the-most-diverse-impact} 这里我们将多样性定义为某位作者参与贡献过的不同文件数量。 @@ -870,7 +870,7 @@ LIMIT 10 ``` -### 作者的常用文件 +### 作者的常用文件 {#favorite-files-for-an-author} 在这里,我们选择我们的创始人 [Alexey Milovidov](https://github.com/alexey-milovidov),并将分析范围限定为当前的文件。 @@ -957,7 +957,7 @@ LIMIT 10 这可能更能体现他的兴趣领域。 -### 作者最少的大型文件 +### 作者最少的大型文件 {#largest-files-with-lowest-number-of-authors} 为此,我们首先需要找出最大的文件。如果要基于提交历史,为每个文件完整重建所有版本来估算大小,代价会非常高! @@ -1130,7 +1130,7 @@ LIMIT 10 ``` -### 按时间、按星期几、按作者以及按特定子目录统计提交与代码行数分布 +### 按时间、按星期几、按作者以及按特定子目录统计提交与代码行数分布 {#commits-and-lines-of-code-distribution-by-time-by-weekday-by-author-for-specific-subdirectories} 这里的含义是:按一周中每一天统计新增和删除的代码行数。在本示例中,我们聚焦于 [Functions 目录](https://github.com/ClickHouse/ClickHouse/tree/master/src/Functions) @@ -1258,7 +1258,7 @@ FROM ``` -### 用于展示不同作者之间谁更倾向于重写他人代码的矩阵 +### 用于展示不同作者之间谁更倾向于重写他人代码的矩阵 {#matrix-of-authors-that-shows-what-authors-tends-to-rewrite-another-authors-code} `sign = -1` 表示代码删除。我们会排除标点符号以及空行的插入。 @@ -1313,7 +1313,7 @@ Alexey 显然很喜欢删除别人的代码。我们把他排除在外,以获 Superset 作者矩阵 v2 -### 按一周中的每一天来看,谁是贡献占比最高的提交者? +### 按一周中的每一天来看,谁是贡献占比最高的提交者? {#who-is-the-highest-percentage-contributor-per-day-of-week} 如果我们只按提交次数来考虑: @@ -1514,7 +1514,7 @@ LIMIT 5 BY root ``` -### 某位作者的代码中,有多少百分比被其他作者删除? +### 某位作者的代码中,有多少百分比被其他作者删除? {#what-percentage-of-code-for-an-author-has-been-removed-by-other-authors} 对于这个问题,我们需要将某位作者编写的代码行数,除以该作者被其他贡献者删除的代码总行数。 @@ -1565,7 +1565,7 @@ LIMIT 10 ``` -### 列出被重写次数最多的文件 +### 列出被重写次数最多的文件 {#list-files-that-were-rewritten-most-number-of-times} 对于这个问题,最简单的方法可能是按文件路径统计行修改次数(仅限当前仍存在的文件),例如: @@ -1708,7 +1708,7 @@ LIMIT 10 ``` -### 在一周中的哪一天,代码留存在代码库中的概率最高? +### 在一周中的哪一天,代码留存在代码库中的概率最高? {#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository} 为此,我们需要对每一行代码进行唯一标识。由于同一行可能在一个文件中出现多次,我们通过文件路径和该行的内容来进行估算。 @@ -1771,7 +1771,7 @@ GROUP BY dayOfWeek(added_day) AS day_of_week_added ``` -### 按平均代码年龄排序的文件 +### 按平均代码年龄排序的文件 {#files-sorted-by-average-code-age} 此查询使用与[代码在仓库中保留概率最高的是一周中的哪一天](#what-weekday-does-the-code-have-the-highest-chance-to-stay-in-the-repository)相同的原理——通过路径和行内容来唯一标识一行代码。 这使我们能够识别一行代码从被添加到被删除之间的时间。我们仅筛选当前仍存在的文件和代码,并对每个文件内所有代码行的时间进行平均。 @@ -1862,7 +1862,7 @@ LIMIT 10 ``` -### 谁更倾向于编写更多的测试 / CPP 代码 / 注释? +### 谁更倾向于编写更多的测试 / CPP 代码 / 注释? {#who-tends-to-write-more-tests--cpp-code--comments} 我们可以用几种方式来回答这个问题。若聚焦于代码与测试的比例,这个查询相对简单——统计对包含 `tests` 的文件夹的贡献次数,并计算其占总贡献次数的比例。 @@ -1996,7 +1996,7 @@ LIMIT 10 请注意,我们是按代码贡献排序的。对于所有主要贡献者来说,这个比例都出奇地高,这也是我们代码之所以如此易读的原因之一。 -### 不同作者的提交中代码/注释比例随时间如何变化? +### 不同作者的提交中代码/注释比例随时间如何变化? {#how-does-an-authors-commits-change-over-time-with-respect-to-codecomments-percentage} 按作者维度计算这个指标非常简单, @@ -2114,7 +2114,7 @@ LIMIT 20 令人鼓舞的是,我们的评论比例相当稳定,并不会随着作者贡献时间的增加而下降。 -### 代码在被重写之前的平均时间是多少?中位数(代码衰减的“半衰期”)又是多少? +### 代码在被重写之前的平均时间是多少?中位数(代码衰减的“半衰期”)又是多少? {#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay} 我们可以使用与[列出被重写次数最多或被最多作者重写的文件](#list-files-that-were-rewritten-most-number-of-times)相同的原则来识别重写行为,但这次针对所有文件。使用窗口函数计算每个文件两次重写之间的时间间隔,在此基础上,我们可以计算所有文件整体的平均值和中位数。 @@ -2175,7 +2175,7 @@ FROM rewrites ``` -### 从代码最有可能被重写的角度来看,什么时候写代码是最“糟糕”的? +### 从代码最有可能被重写的角度来看,什么时候写代码是最“糟糕”的? {#what-is-the-worst-time-to-write-code-in-sense-that-the-code-has-highest-chance-to-be-re-written} 类似于 [What is the average time before code will be rewritten and the median (half-life of code decay)?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) 和 [List files that were rewritten most number of time or by most of authors](#list-files-that-were-rewritten-most-number-of-times),只是这里我们按一周中的星期几进行聚合。可按需要进行调整,例如按一年中的月份。 @@ -2240,7 +2240,7 @@ GROUP BY dayOfWeek ``` -### 哪位作者的代码「黏性」最高? +### 哪位作者的代码「黏性」最高? {#which-authors-code-is-the-most-sticky} 我们将「黏性」定义为:一位作者的代码在被重写之前能保留多长时间。类似于前一个问题 [代码在被重写前的平均时间是多少?以及中位数(代码衰减的半衰期)?](#what-is-the-average-time-before-code-will-be-rewritten-and-the-median-half-life-of-code-decay) —— 使用相同的重写度量方式,即文件中有 50% 的内容为新增、50% 为删除。我们按作者计算平均重写时间,并且只考虑在超过两个文件上有贡献的作者。 @@ -2318,7 +2318,7 @@ LIMIT 10 ``` -### 作者连续提交天数最多 +### 作者连续提交天数最多 {#most-consecutive-days-of-commits-by-an-author} 此查询首先需要计算每位作者在哪些日期进行了提交。使用按作者分区的窗口函数,我们可以计算各次提交之间相隔的天数。对于每次提交,如果距离上一次提交相隔 1 天,则将其标记为连续(1),否则为 0,并将这一结果存入 `consecutive_day`。 @@ -2374,7 +2374,7 @@ LIMIT 10 ``` -### 文件的逐行提交历史 +### 文件的逐行提交历史 {#line-by-line-commit-history-of-a-file} 文件可能会被重命名。发生这种情况时,我们会产生一个重命名事件,其中 `path` 列会被设置为文件的新路径,而 `old_path` 则表示之前的位置,例如: @@ -2455,7 +2455,7 @@ FORMAT PrettyCompactMonoBlock ## 未解决的问题 {#unsolved-questions} -### Git blame +### Git blame {#git-blame} 由于目前无法在数组函数中保持状态,因此在实践中很难得到精确的结果。使用 `arrayFold` 或 `arrayReduce` 之后就可以做到,因为它们允许在每次迭代时保存状态。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md index b578cde5ded..27c0ef870bf 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/hacker-news.md @@ -7,7 +7,7 @@ doc_type: 'guide' keywords: ['示例数据集', 'Hacker News', '样本数据', '文本分析', '向量搜索'] --- -# Hacker News 数据集 +# Hacker News 数据集 {#hacker-news-dataset} > 在本教程中,你将把 2800 万行 Hacker News 数据(来自 CSV 和 Parquet 两种格式)插入到一个 ClickHouse 表中,并运行一些简单的查询来探索这些数据。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md index 984d1a75820..1d6c3ddcb14 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/laion.md @@ -11,7 +11,7 @@ keywords: ['示例数据集', 'laion', '图像嵌入', '示例数据', '机器 该数据集包含图像 URL、图像及其说明文字各自的嵌入向量、图像与图像说明文字之间的相似度评分,以及元数据,例如图像宽度/高度、许可证类型和 NSFW 标志。我们可以使用该数据集来演示 ClickHouse 中的[近似最近邻搜索](../../engines/table-engines/mergetree-family/annindexes.md)。 -## 数据准备 +## 数据准备 {#data-preparation} 在原始数据中,embedding 向量和元数据存储在不同的文件中。数据准备步骤会下载数据、合并这些文件, 将它们转换为 CSV 格式并导入 ClickHouse。可以使用下面的 `download.sh` 脚本来完成这一操作: @@ -40,28 +40,28 @@ npy_file = "img_emb_" + str_i + '.npy' metadata_file = "metadata_" + str_i + '.parquet' text_npy = "text_emb_" + str_i + '.npy' -# 加载所有文件 +# 加载所有文件 {#load-all-files} im_emb = np.load(npy_file) text_emb = np.load(text_npy) data = pd.read_parquet(metadata_file) -# 合并文件 +# 合并文件 {#combine-files} data = pd.concat([data, pd.DataFrame({"image_embedding" : [*im_emb]}), pd.DataFrame({"text_embedding" : [*text_emb]})], axis=1, copy=False) -# 待导入 ClickHouse 的列 +# 待导入 ClickHouse 的列 {#columns-to-be-imported-into-clickhouse} data = data[['url', 'caption', 'NSFW', 'similarity', "image_embedding", "text_embedding"]] -# 将 np.arrays 转换为列表 +# 将 np.arrays 转换为列表 {#transform-nparrays-to-lists} data['image_embedding'] = data['image_embedding'].apply(lambda x: x.tolist()) data['text_embedding'] = data['text_embedding'].apply(lambda x: x.tolist()) -# 此处需要使用这个小技巧,因为 caption 有时会包含各种引号 +# 此处需要使用这个小技巧,因为 caption 有时会包含各种引号 {#this-small-hack-is-needed-because-caption-sometimes-contains-all-kind-of-quotes} data['caption'] = data['caption'].apply(lambda x: x.replace("'", " ").replace('"', " ")) -# 导出数据为 CSV 文件 +# 导出数据为 CSV 文件 {#export-data-as-csv-file} data.to_csv(str_i + '.csv', header=False) -# 删除原始数据文件 +# 删除原始数据文件 {#removed-raw-data-files} os.system(f"rm {npy_file} {metadata_file} {text_npy}") ``` @@ -76,7 +76,7 @@ seq 0 409 | xargs -P1 -I{} bash -c './download.sh {}' (上面的 Python 脚本非常慢(每个文件大约需要 2–10 分钟),占用大量内存(每个文件 41 GB),且生成的 CSV 文件也很大(每个 10 GB),使用时请谨慎。如果你的 RAM 足够多,可以增大 `-P1` 的数值以提高并行度。如果这仍然太慢,可以考虑设计更好的摄取流程——例如先将 .npy 文件转换为 Parquet,然后使用 ClickHouse 完成其余处理。) -## 创建表 +## 创建表 {#create-table} 要先创建一个不带索引的表,运行: @@ -105,7 +105,7 @@ INSERT INTO laion FROM INFILE '{path_to_csv_files}/*.csv' 请注意,`id` 列仅用于示例说明,脚本会向其中写入非唯一值。 -## 运行基于暴力算法的向量相似度搜索 +## 运行基于暴力算法的向量相似度搜索 {#run-a-brute-force-vector-similarity-search} 要运行一次基于暴力算法的近似向量搜索,请执行: @@ -137,7 +137,7 @@ SELECT url, caption FROM laion ORDER BY cosineDistance(image_embedding, {target: ``` -## 使用向量相似性索引执行近似向量相似性搜索 +## 使用向量相似性索引执行近似向量相似性搜索 {#run-an-approximate-vector-similarity-search-with-a-vector-similarity-index} 现在我们在该表上定义两个向量相似性索引。 @@ -194,7 +194,7 @@ SELECT url, caption FROM laion ORDER BY cosineDistance(image_embedding, {target: 通常我们希望为新的图像或新的图像描述创建嵌入向量,并在数据中搜索相似的图像/图像描述对。我们可以使用 [UDF](/sql-reference/functions/udf) 直接在客户端中创建 `target` 向量。务必使用同一模型来生成原始数据和搜索时用的新嵌入向量。下面的脚本使用的是 `ViT-B/32` 模型,该模型也是此数据集所基于的模型。 -### 文本嵌入 +### 文本嵌入 {#text-embeddings} 首先,将以下 Python 脚本保存到 ClickHouse 数据路径下的 `user_scripts/` 目录中,并将其设为可执行文件(`chmod +x encode_text.py`)。 @@ -256,7 +256,7 @@ LIMIT 10 请注意,`encode_text()` UDF 本身的计算可能需要几秒钟才能生成嵌入向量。 -### 图像嵌入 +### 图像嵌入 {#image-embeddings} 图像嵌入的创建方式类似,我们提供了一个 Python 脚本,可为以本地图像文件形式存储的图像生成嵌入向量。 @@ -304,7 +304,7 @@ if __name__ == '__main__': 获取示例图片进行搜索: ```shell -# 获取乐高套装的随机图片 +# 获取乐高套装的随机图片 {#get-a-random-image-of-a-lego-set} $ wget http://cdn.firstcry.com/brainbees/images/products/thumb/191325a.jpg ``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md index 000c71bd970..26aa0c85cae 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/menus.md @@ -15,22 +15,22 @@ keywords: ['example dataset', 'menus', 'historical data', 'sample data', 'nypl'] 这些数据来自图书馆馆藏档案,可能不完整,也不一定适合严格的统计分析。不过它也非常有趣且“美味”。 数据规模仅为关于菜单菜品的 130 万条记录——对 ClickHouse 来说,这个数据量非常小,但仍然是一个很好的示例。 -## 下载数据集 +## 下载数据集 {#download-dataset} 运行以下命令: ```bash wget https://s3.amazonaws.com/menusdata.nypl.org/gzips/2021_08_01_07_01_17_data.tgz -# 可选:验证校验和 +# 可选:验证校验和 {#option-validate-the-checksum} md5sum 2021_08_01_07_01_17_data.tgz -# 校验和应为:db6126724de939a5481e3160a2d67d15 +# 校验和应为:db6126724de939a5481e3160a2d67d15 {#checksum-should-be-equal-to-db6126724de939a5481e3160a2d67d15} ``` 如有需要,请将链接替换为来自 [http://menus.nypl.org/data](http://menus.nypl.org/data) 的最新更新链接。 下载大小约为 35 MB。 -## 解压数据集 +## 解压数据集 {#unpack-dataset} ```bash tar xvf 2021_08_01_07_01_17_data.tgz @@ -46,7 +46,7 @@ tar xvf 2021_08_01_07_01_17_data.tgz * `MenuItem` — 菜单中的一项。某个菜单页面上的一道菜及其价格,并包含指向菜品和菜单页面的链接。 -## 创建表 +## 创建表 {#create-tables} 我们使用 [Decimal](../../sql-reference/data-types/decimal.md) 数据类型存储价格。 @@ -114,7 +114,7 @@ CREATE TABLE menu_item ``` -## 导入数据 +## 导入数据 {#import-data} 将数据上传到 ClickHouse,请运行: @@ -134,7 +134,7 @@ clickhouse-client --format_csv_allow_single_quotes 0 --input_format_null_as_defa 设置 [date_time_input_format best_effort](/operations/settings/formats#date_time_input_format) 允许以多种格式解析 [DateTime](../../sql-reference/data-types/datetime.md) 字段。例如,不带秒的 ISO-8601 时间(如 `2000-01-01 01:02`)也会被识别。如果不启用此设置,则只允许固定格式的 DateTime。 -## 数据反规范化 +## 数据反规范化 {#denormalize-data} 数据以[规范化形式](https://en.wikipedia.org/wiki/Database_normalization#Normal_forms)分布在多个表中。这意味着,如果你想查询例如菜单项对应的菜品名称,就必须执行 [JOIN](/sql-reference/statements/select/join)。 对于典型的分析任务,预先对数据进行 JOIN 合并,以避免每次查询都执行 `JOIN`,要高效得多。这被称为“反规范化”数据。 @@ -187,7 +187,7 @@ FROM menu_item ``` -## 验证数据 +## 验证数据 {#validate-data} 查询: @@ -206,7 +206,7 @@ SELECT count() FROM menu_item_denorm; ## 执行一些查询 {#run-queries} -### 菜品历史价格平均值 +### 菜品历史价格平均值 {#query-averaged-historical-prices} 查询: @@ -249,7 +249,7 @@ ORDER BY d ASC; 请谨慎看待,仅供参考。 -### 汉堡价格 +### 汉堡价格 {#query-burger-prices} 查询: @@ -287,7 +287,7 @@ ORDER BY d ASC; ``` -### 伏特加 +### 伏特加 {#query-vodka} 查询: @@ -322,7 +322,7 @@ ORDER BY d ASC; 为了查到 vodka,我们就得写 `ILIKE '%vodka%'`,本身就很能说明问题。 -### 鱼子酱 +### 鱼子酱 {#query-caviar} 我们来打印鱼子酱的价格,同时打印任意一道包含鱼子酱的菜肴名称。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md index 38618a5ca35..22545e4e922 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/noaa.md @@ -28,7 +28,7 @@ keywords: ['示例数据集', 'noaa', '天气数据', '样本数据', '气候'] - 一个适用于 ClickHouse 的[预先准备好的版本](#pre-prepared-data)数据,已完成清洗、重组和富化。该数据覆盖 1900 年至 2022 年。 - [下载原始数据](#original-data)并转换为 ClickHouse 所需的格式。希望添加自定义列的用户可以考虑采用这种方式。 -### 预先准备的数据 +### 预先准备的数据 {#pre-prepared-data} 更具体地说,已经移除了所有在 NOAA 的质量检查中未出现任何失败的行。数据也从“每行一个观测值”重构为“每个测站 ID 与日期对应一行”,即: @@ -55,7 +55,7 @@ wget https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enriche 下面将说明下载并转换原始数据,以便将其加载到 ClickHouse 的步骤。 -#### 下载 +#### 下载 {#download} 下载原始数据: @@ -64,7 +64,7 @@ for i in {1900..2023}; do wget https://noaa-ghcn-pds.s3.amazonaws.com/csv.gz/${i ``` -#### 数据采样 +#### 数据采样 {#sampling-the-data} ```bash $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format PrettyCompact @@ -108,7 +108,7 @@ $ clickhouse-local --query "SELECT * FROM '2021.csv.gz' LIMIT 10" --format Prett 每行一个测量值会在 ClickHouse 中产生稀疏的表结构。我们应当转换为每个时间点和测站一行的形式,将各测量项作为列。首先,我们将数据集限制为没有问题的行,即 `qFlag` 等于空字符串的记录。 -#### 清洗数据 +#### 清洗数据 {#clean-the-data} 使用 [ClickHouse local](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local),我们可以筛选出代表感兴趣测量数据并满足质量要求的行: @@ -122,7 +122,7 @@ FROM file('*.csv.gz', CSV, 'station_id String, date String, measurement String, 由于数据量超过 26 亿行,这个查询执行得并不快,因为需要解析所有文件。在我们的 8 核机器上,这大约需要 160 秒。 -### 透视数据 +### 透视数据 {#pivot-data} 虽然“每行一条测量记录”的结构也可以在 ClickHouse 中使用,但会让后续查询不必要地变得复杂。理想情况下,我们希望每个站点 ID 和日期对应一行,其中每种测量类型及其对应的数值都是单独的一列,即: @@ -161,7 +161,7 @@ done 此查询会生成一个 50GB 的单个文件 `noaa.csv`。 -### 丰富数据 +### 丰富数据 {#enriching-the-data} 这些数据除了包含前缀为国家代码的站点 ID 外,不包含任何其他位置信息。理想情况下,每个站点都应关联有纬度和经度。为此,NOAA 贴心地将每个站点的详细信息单独提供在一个 [ghcnd-stations.txt](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file) 文件中。该文件包含[多列数据](https://github.com/awslabs/open-data-docs/tree/main/docs/noaa/noaa-ghcn#format-of-ghcnd-stationstxt-file),其中有五列对后续分析很有用:id、latitude、longitude、elevation 和 name。 @@ -194,7 +194,7 @@ FROM file('noaa.csv', CSV, 此查询运行需要几分钟时间,并生成一个大小为 6.4 GB 的文件 `noaa_enriched.parquet`。 -## 创建表 +## 创建表 {#create-table} 使用 ClickHouse 客户端在 ClickHouse 中创建一个 MergeTree 表。 @@ -223,7 +223,7 @@ CREATE TABLE noaa ## 向 ClickHouse 写入数据 {#inserting-into-clickhouse} -### 从本地文件插入 +### 从本地文件插入 {#inserting-from-local-file} 在 ClickHouse 客户端中,可以按如下方式从本地文件插入数据: @@ -236,7 +236,7 @@ INSERT INTO noaa FROM INFILE '<路径>/noaa_enriched.parquet' 有关如何加快加载速度,请参见[此处](https://clickhouse.com/blog/real-world-data-noaa-climate-data#load-the-data)。 -### 从 S3 中导入数据 +### 从 S3 中导入数据 {#inserting-from-s3} ```sql INSERT INTO noaa SELECT * @@ -249,7 +249,7 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/noaa/noaa_enr ## 查询示例 {#sample-queries} -### 有史以来的最高气温 +### 有史以来的最高气温 {#highest-temperature-ever} ```sql SELECT @@ -278,7 +278,7 @@ LIMIT 5 令人放心的是,其与截至 2023 年 [Furnace Creek](https://www.google.com/maps/place/36%C2%B027'00.0%22N+116%C2%B052'00.1%22W/@36.1329666,-116.1104099,8.95z/data=!4m5!3m4!1s0x0:0xf2ed901b860f4446!8m2!3d36.45!4d-116.8667) 的[官方记录](https://en.wikipedia.org/wiki/List_of_weather_records#Highest_temperatures_ever_recorded)高度一致。 -### 最佳滑雪胜地 +### 最佳滑雪胜地 {#best-ski-resorts} 使用这份[美国滑雪胜地列表](https://gist.githubusercontent.com/gingerwizard/dd022f754fd128fdaf270e58fa052e35/raw/622e03c37460f17ef72907afe554cb1c07f91f23/ski_resort_stats.csv)及其各自位置,将其与过去 5 年中任意单月降雪量最高的前 1000 个气象站进行关联。按 [geoDistance](/sql-reference/functions/geo/coordinates/#geodistance) 对关联结果排序,并将距离限制在 20 公里以内,从中为每个滑雪胜地选取距离最近的一条记录,然后按总降雪量排序。注意,我们还将滑雪胜地限制在海拔 1800 米以上,将其作为良好滑雪条件的一个粗略指标。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md index 4c0df670537..d17faeeb76b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nyc-taxi.md @@ -24,7 +24,7 @@ import TabItem from '@theme/TabItem'; ::: -## 创建 `trips` 表 +## 创建 `trips` 表 {#create-the-table-trips} 首先为出租车行程创建一张表: @@ -125,7 +125,7 @@ FROM gcs( -## 示例查询 +## 示例查询 {#sample-queries} 下面的查询是在上文所述的示例数据上执行的。用户可以在 [sql.clickhouse.com](https://sql.clickhouse.com/?query=U0VMRUNUIGNvdW50KCkgRlJPTSBueWNfdGF4aS50cmlwcw\&chart=eyJ0eXBlIjoibGluZSIsImNvbmZpZyI6eyJ0aXRsZSI6IlRlbXBlcmF0dXJlIGJ5IGNvdW50cnkgYW5kIHllYXIiLCJ4YXhpcyI6InllYXIiLCJ5YXhpcyI6ImNvdW50KCkiLCJzZXJpZXMiOiJDQVNUKHBhc3Nlbmdlcl9jb3VudCwgJ1N0cmluZycpIn19) 上针对完整数据集运行这些示例查询,只需将下面的查询修改为使用表 `nyc_taxi.trips`。 @@ -182,7 +182,7 @@ ORDER BY passenger_count ASC ``` -## 下载预先生成的分区 +## 下载预先生成的分区 {#download-of-prepared-partitions} :::note 以下步骤介绍原始数据集的信息,以及将预先生成的分区加载到自行管理的 ClickHouse 服务器环境中的方法。 @@ -195,9 +195,9 @@ ORDER BY passenger_count ASC ```bash $ curl -O https://datasets.clickhouse.com/trips_mergetree/partitions/trips_mergetree.tar -# 验证校验和 +# 验证校验和 {#validate-the-checksum} $ md5sum trips_mergetree.tar -# 校验和应为:f3b8d469b41d9a82da064ded7245d12c +# 校验和应为:f3b8d469b41d9a82da064ded7245d12c {#checksum-should-be-equal-to-f3b8d469b41d9a82da064ded7245d12c} $ tar xvf trips_mergetree.tar -C /var/lib/clickhouse # ClickHouse 数据目录的路径 $ # 检查解压数据的权限,必要时进行修复 $ sudo service clickhouse-server restart @@ -209,7 +209,7 @@ $ clickhouse-client --query "select count(*) from datasets.trips_mergetree" ::: -## 单机环境下的结果 +## 单机环境下的结果 {#results-on-single-server} 问题 1: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md index 7f261114132..d66f75a12f1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/nypd_complaint_data.md @@ -42,7 +42,7 @@ keywords: ['示例数据集', 'nypd', '犯罪数据', '样本数据', '公共数 在开始使用 ClickHouse 数据库之前,请先熟悉一下该 TSV 文件中的数据。 -### 查看源 TSV 文件中的字段 +### 查看源 TSV 文件中的字段 {#look-at-the-fields-in-the-source-tsv-file} 这是一个查询 TSV 文件的示例命令,但先不要运行它。 @@ -119,7 +119,7 @@ New Georeferenced Column Nullable(String) 此时,应检查 TSV 文件中的列是否与[数据集网页](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243)中 **Columns in this Dataset** 部分指定的列名和类型一致。该数据集中的数据类型定义并不十分具体,所有数值型字段都被设置为 `Nullable(Float64)`,而其他所有字段则为 `Nullable(String)`。在创建用于存储这些数据的 ClickHouse 表时,可以指定更合适、性能更佳的数据类型。 -### 确定合适的表结构 +### 确定合适的表结构 {#determine-the-proper-schema} 要确定字段应使用哪些数据类型,必须先了解数据的实际形态。例如,字段 `JURISDICTION_CODE` 是一个数值类型:它应该是 `UInt8`,还是 `Enum`,或者使用 `Float64` 更为合适? @@ -210,7 +210,7 @@ clickhouse-local --input_format_max_rows_to_read_for_schema_inference=2000 \ 在撰写本文时使用的数据集中,`PARK_NM` 列中只有几百个不同的公园和游乐场名称。根据 [LowCardinality](/sql-reference/data-types/lowcardinality#description) 的建议,`LowCardinality(String)` 字段中的不同字符串数量应控制在 10,000 个以下,因此这个数量相对较小。 -### DateTime 字段 +### DateTime 字段 {#datetime-fields} 根据该[数据集网页](https://data.cityofnewyork.us/Public-Safety/NYPD-Complaint-Data-Current-Year-To-Date-/5uac-w243)中的 **Columns in this Dataset** 部分,可以看到为报告事件的开始和结束分别提供了日期和时间字段。查看 `CMPLNT_FR_DT` 和 `CMPLT_TO_DT` 的最小值和最大值,可以帮助判断这些字段是否始终有值: @@ -297,7 +297,7 @@ FORMAT PrettyCompact" 还需要对更多字段类型进行修改,这些都可以通过遵循相同的分析步骤来确定。查看字段中不同字符串取值的数量、数值字段的最小值和最大值,然后做出你的决策。本指南后面给出的表结构中包含了许多低基数的字符串字段和无符号整数字段,而浮点数值字段则很少。 ::: -## 连接日期和时间字段 +## 连接日期和时间字段 {#concatenate-the-date-and-time-fields} 要将日期和时间字段 `CMPLNT_FR_DT` 和 `CMPLNT_FR_TM` 拼接为一个可以转换为 `DateTime` 的 `String`,请选择这两个字段,并使用连接运算符进行拼接:`CMPLNT_FR_DT || ' ' || CMPLNT_FR_TM`。`CMPLNT_TO_DT` 和 `CMPLNT_TO_TM` 字段的处理方式相同。 @@ -328,7 +328,7 @@ FORMAT PrettyCompact" ``` -## 将日期和时间字符串转换为 DateTime64 类型 +## 将日期和时间字符串转换为 DateTime64 类型 {#convert-the-date-and-time-string-to-a-datetime64-type} 在本指南的前面部分中,我们发现 TSV 文件中存在早于 1970 年 1 月 1 日的日期,这意味着这些日期需要使用 64 位的 DateTime 类型。同时,还需要将日期格式从 `MM/DD/YYYY` 转换为 `YYYY/MM/DD`。这两项操作都可以通过 [`parseDateTime64BestEffort()`](../../sql-reference/functions/type-conversion-functions.md#parsedatetime64besteffort) 来完成。 @@ -389,7 +389,7 @@ FORMAT PrettyCompact" 上文中针对各列所选用的数据类型会体现在下面的表结构中。我们还需要确定表所使用的 `ORDER BY` 和 `PRIMARY KEY`。必须至少指定 `ORDER BY` 或 `PRIMARY KEY` 其中之一。下面是关于如何决定在 `ORDER BY` 中包含哪些列的一些指导原则,更多信息请参见本文档末尾的 *后续步骤* 部分。 -### `ORDER BY` 和 `PRIMARY KEY` 子句 +### `ORDER BY` 和 `PRIMARY KEY` 子句 {#order-by-and-primary-key-clauses} * `ORDER BY` 元组应包含在查询过滤条件中使用的字段 * 为了最大化磁盘压缩率,`ORDER BY` 元组中的字段应按基数从低到高排序 @@ -483,7 +483,7 @@ CREATE TABLE NYPD_Complaint ( ``` -### 查找表的主键 +### 查找表的主键 {#finding-the-primary-key-of-a-table} ClickHouse 的 `system` 数据库,特别是 `system.table`,存储了你刚刚创建的表的所有信息。下面的查询会显示 `ORDER BY`(排序键)和 `PRIMARY KEY`: @@ -518,7 +518,7 @@ Query id: 6a5b10bf-9333-4090-b36e-c7f08b1d9e01 我们将使用 `clickhouse-local` 工具对数据进行预处理,并使用 `clickhouse-client` 将其上传。 -### 使用的 `clickhouse-local` 参数 +### 使用的 `clickhouse-local` 参数 {#clickhouse-local-arguments-used} :::tip `table='input'` 出现在下面传给 clickhouse-local 的参数中。clickhouse-local 会获取提供的输入(`cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv`),并将该输入插入到一张表中。默认情况下,这张表被命名为 `table`。在本指南中,将这张表的名称设置为 `input`,以使数据流更加清晰。传给 clickhouse-local 的最后一个参数是一个从该表查询的语句(`FROM input`),随后通过管道传给 `clickhouse-client`,用于向表 `NYPD_Complaint` 写入数据。 @@ -569,7 +569,7 @@ cat ${HOME}/NYPD_Complaint_Data_Current__Year_To_Date_.tsv \ ``` -## 验证数据 +## 验证数据 {#validate-data} :::note 数据集每年会变动一到多次,因此你得到的计数结果可能与本文档中的示例不完全一致。 @@ -613,7 +613,7 @@ WHERE name = 'NYPD_Complaint' ## 执行一些查询 {#run-queries} -### 查询 1:按月对比投诉数量 +### 查询 1:按月对比投诉数量 {#query-1-compare-the-number-of-complaints-by-month} 查询: @@ -651,7 +651,7 @@ Query id: 7fbd4244-b32a-4acf-b1f3-c3aa198e74d9 ``` -### 查询 2:按行政区对比投诉总数 +### 查询 2:按行政区对比投诉总数 {#query-2-compare-total-number-of-complaints-by-borough} 查询: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md index 249ee95a712..c4b5f58e370 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/ontime.md @@ -127,7 +127,7 @@ CREATE TABLE `ontime` ORDER BY (Year, Quarter, Month, DayofMonth, FlightDate, IATA_CODE_Reporting_Airline); ``` -## 从原始数据导入 +## 从原始数据导入 {#import-from-raw-data} 下载数据: @@ -144,7 +144,7 @@ ls -1 *.zip | xargs -I{} -P $(nproc) bash -c "echo {}; unzip -cq {} '*.csv' | se (如果你的服务器出现内存不足或其他问题,请删除 `-P $(nproc)` 这一部分) -## 从已保存的副本导入 +## 从已保存的副本导入 {#import-from-a-saved-copy} 你也可以通过以下查询,从已保存的副本中导入数据: @@ -155,7 +155,7 @@ INSERT INTO ontime SELECT * FROM s3('https://clickhouse-public-datasets.s3.amazo 该快照创建于 2022-05-29。 -## 查询 +## 查询 {#queries} Q0. diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md index 2831d7e3671..32ec8b4f04f 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/stackoverflow.md @@ -22,7 +22,7 @@ import stackoverflow from '@site/static/images/getting-started/example-datasets/ 该数据集的模式说明可在[此处](https://meta.stackexchange.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-sede)找到。 -## 预置数据 +## 预置数据 {#pre-prepared-data} 我们提供了一份 Parquet 格式的数据副本,内容更新至 2024 年 4 月。虽然从行数规模来看(6,000 万条帖子),这个数据集对 ClickHouse 来说并不算大,但其中包含了大量文本以及体积较大的 String 类型列。 @@ -33,7 +33,7 @@ CREATE DATABASE stackoverflow 以下时间测试结果基于一个位于 `eu-west-2`、具有 96 GiB 内存和 24 个 vCPU 的 ClickHouse Cloud 集群。数据集位于 `eu-west-3`。 -### 文章 +### 文章 {#posts} ```sql CREATE TABLE stackoverflow.posts @@ -73,7 +73,7 @@ INSERT INTO stackoverflow.posts SELECT * FROM s3('https://datasets-documentation Posts 数据集也可以按年份获取,例如 [https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet) -### 投票 +### 投票 {#votes} ```sql CREATE TABLE stackoverflow.votes @@ -96,7 +96,7 @@ INSERT INTO stackoverflow.votes SELECT * FROM s3('https://datasets-documentation 投票数据也可以按年份获取,例如:[https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/votes/2020.parquet) -### 评论 +### 评论 {#comments} ```sql CREATE TABLE stackoverflow.comments @@ -120,7 +120,7 @@ INSERT INTO stackoverflow.comments SELECT * FROM s3('https://datasets-documentat 评论数据也可以按年份获取,例如:[https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/posts/2020.parquet](https://datasets-documentation.s3.eu-west-3.amazonaws.com/stackoverflow/parquet/comments/2020.parquet) -### 用户 +### 用户 {#users} ```sql CREATE TABLE stackoverflow.users @@ -147,7 +147,7 @@ INSERT INTO stackoverflow.users SELECT * FROM s3('https://datasets-documentation ``` -### 徽章 +### 徽章 {#badges} ```sql CREATE TABLE stackoverflow.badges @@ -168,7 +168,7 @@ INSERT INTO stackoverflow.badges SELECT * FROM s3('https://datasets-documentatio ``` -### 文章链接 +### 文章链接 {#postlinks} ```sql CREATE TABLE stackoverflow.postlinks @@ -188,7 +188,7 @@ INSERT INTO stackoverflow.postlinks SELECT * FROM s3('https://datasets-documenta ``` -### PostHistory +### PostHistory {#posthistory} ```sql CREATE TABLE stackoverflow.posthistory @@ -217,7 +217,7 @@ INSERT INTO stackoverflow.posthistory SELECT * FROM s3('https://datasets-documen 原始数据集以压缩的 7-Zip XML 格式提供,可从 [https://archive.org/download/stackexchange](https://archive.org/download/stackexchange) 获取,文件前缀为 `stackoverflow.com*`。 -### 下载 +### 下载 {#download} ```bash wget https://archive.org/download/stackexchange/stackoverflow.com-Badges.7z @@ -232,7 +232,7 @@ wget https://archive.org/download/stackexchange/stackoverflow.com-Votes.7z 这些文件大小可达 35GB,具体下载时间取决于网络状况,大约需要 30 分钟——下载服务器的限速约为 20 MB/秒。 -### 转换为 JSON +### 转换为 JSON {#convert-to-json} 在撰写本文时,ClickHouse 暂不原生支持将 XML 作为输入格式。要将数据加载到 ClickHouse,我们首先将其转换为 NDJSON。 @@ -260,7 +260,7 @@ p7zip -d stackoverflow.com-Posts.7z ```bash mkdir posts cd posts -# 以下命令将输入的 XML 文件拆分为每个 10000 行的子文件 +# 以下命令将输入的 XML 文件拆分为每个 10000 行的子文件 {#the-following-splits-the-input-xml-file-into-sub-files-of-10000-rows} tail +3 ../Posts.xml | head -n -1 | split -l 10000 --filter='{ printf "\n"; cat - ; printf "\n"; } > $FILE' - ``` @@ -283,7 +283,7 @@ clickhouse local --query "SELECT * FROM file('posts.json', JSONEachRow, 'Id Int3 下面是一些简单的查询示例,帮助你开始使用。 -### Stack Overflow 上最常用的标签 +### Stack Overflow 上最常用的标签 {#most-popular-tags-on-stack-overflow} ```sql @@ -313,7 +313,7 @@ LIMIT 10 ``` -### 回答数最多的用户(活跃账户) +### 回答数最多的用户(活跃账户) {#user-with-the-most-answers-active-accounts} 账户必须包含 `UserId`。 @@ -340,7 +340,7 @@ LIMIT 5 ``` -### 阅读量最高的 ClickHouse 相关文章 +### 阅读量最高的 ClickHouse 相关文章 {#clickhouse-related-posts-with-the-most-views} ```sql SELECT @@ -371,7 +371,7 @@ LIMIT 10 ``` -### 最具争议的帖子 +### 最具争议的帖子 {#most-controversial-posts} ```sql SELECT diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md index 4344f035678..3df93f349fa 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tpch.md @@ -13,12 +13,12 @@ keywords: ['示例数据集', 'tpch', '基准测试', '示例数据', '性能测 **参考文献** -- [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) -- [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291)(Poess 等,2000) -- [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5)(Boncz 等,2013) -- [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138)(Dresseler 等,2020) +* [TPC-H](https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp) +* [New TPC Benchmarks for Decision Support and Web Commerce](https://doi.org/10.1145/369275.369291)(Poess 等,2000) +* [TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark](https://doi.org/10.1007/978-3-319-04936-6_5)(Boncz 等,2013) +* [Quantifying TPC-H Choke Points and Their Optimizations](https://doi.org/10.14778/3389133.3389138)(Dresseler 等,2020) -## 数据生成与导入 +## 数据生成与导入 {#data-generation-and-import} 首先,检出 TPC-H 仓库代码并编译数据生成器: @@ -61,7 +61,6 @@ make (例如 `Identifier`、`Variable text, size N`)。这仅带来更好的可读性,由 `dbgen` 生成的 SQL-92 数据类型 (例如 `INTEGER`、`VARCHAR(40)`) 在 ClickHouse 中同样可以正常工作。 - ```sql CREATE TABLE nation ( n_nationkey Int32, @@ -169,7 +168,6 @@ clickhouse-client --format_csv_delimiter '|' --query "INSERT INTO lineitem FORMA :::note 你也可以选择从公共 S3 存储桶中导入数据,而不是使用 tpch-kit 自行生成这些表。请确保先使用上面的 `CREATE` 语句创建空表。 - ```sql -- 扩展因子 1 INSERT INTO nation SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws.com/h/1/nation.tbl', NOSIGN, CSV) SETTINGS format_csv_delimiter = '|', input_format_defaults_for_omitted_fields = 1, input_format_csv_empty_as_default = 1; @@ -194,8 +192,7 @@ INSERT INTO lineitem SELECT * FROM s3('https://clickhouse-datasets.s3.amazonaws. ::: - -## 查询 +## 查询 {#queries} :::note 应启用 [`join_use_nulls`](../../operations/settings/settings.md#join_use_nulls) 设置,以获得符合 SQL 标准的正确结果。 @@ -348,7 +345,6 @@ ORDER BY **问题 5** - ```sql SELECT n_name, @@ -496,7 +492,6 @@ ORDER BY **Q9** - ```sql SELECT nation, @@ -850,7 +845,6 @@ WHERE **Q20** - ```sql SELECT s_name, diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md index 3feb9f29161..c20b0876e96 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/tw-weather.md @@ -26,7 +26,7 @@ keywords: ['示例数据集', 'weather', 'taiwan', '示例数据', '气候数据 - 为 ClickHouse 准备的[预处理数据版本](#pre-processed-data),已经过清洗、重构和富化。该数据集覆盖 1896 年至 2023 年。 - [下载原始数据](#original-raw-data)并转换为 ClickHouse 所需的格式。希望添加自定义列的用户可以在此基础上探索或完善自己的方案。 -### 预处理数据 +### 预处理数据 {#pre-processed-data} 该数据集也已经从“每行一条测量记录”的结构重组为“每个气象站 ID 与测量日期对应一行”的结构,即: @@ -47,16 +47,16 @@ C0X100,2016-01-01 04:00:00,1021.2,15.8,74,1.7,8.0,,,,,,,,,,,,,,,,,,,,,,, ```bash wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/preprocessed_weather_daily_1896_2023.tar.gz -# 可选:验证校验和 +# 可选:验证校验和 {#option-validate-the-checksum} md5sum preprocessed_weather_daily_1896_2023.tar.gz -# 校验和应为:11b484f5bd9ddafec5cfb131eb2dd008 +# 校验和应为:11b484f5bd9ddafec5cfb131eb2dd008 {#checksum-should-be-equal-to-11b484f5bd9ddafec5cfb131eb2dd008} tar -xzvf preprocessed_weather_daily_1896_2023.tar.gz daily_weather_preprocessed_1896_2023.csv -# 可选:验证校验和 +# 可选:验证校验和 {#option-validate-the-checksum} md5sum daily_weather_preprocessed_1896_2023.csv -# 校验和应为:1132248c78195c43d93f843753881754 +# 校验和应为:1132248c78195c43d93f843753881754 {#checksum-should-be-equal-to-1132248c78195c43d93f843753881754} ``` @@ -64,7 +64,7 @@ md5sum daily_weather_preprocessed_1896_2023.csv 以下内容介绍如何下载原始数据,以便按需进行转换和处理。 -#### 下载 +#### 下载 {#download} 要下载原始数据: @@ -73,9 +73,9 @@ mkdir tw_raw_weather_data && cd tw_raw_weather_data wget https://storage.googleapis.com/taiwan-weather-observaiton-datasets/raw_data_weather_daily_1896_2023.tar.gz -# 可选:验证校验和 +# 可选:验证校验和 {#option-validate-the-checksum} md5sum raw_data_weather_daily_1896_2023.tar.gz -# 校验和应为:b66b9f137217454d655e3004d7d1b51a +# 校验和应为:b66b9f137217454d655e3004d7d1b51a {#checksum-should-be-equal-to-b66b9f137217454d655e3004d7d1b51a} tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1928.csv @@ -84,23 +84,23 @@ tar -xzvf raw_data_weather_daily_1896_2023.tar.gz 466920_1931.csv ... -# 可选:验证校验和 +# 可选:验证校验和 {#option-validate-the-checksum} cat *.csv | md5sum -# 校验和应为:b26db404bf84d4063fac42e576464ce1 +# 校验和应为:b26db404bf84d4063fac42e576464ce1 {#checksum-should-be-equal-to-b26db404bf84d4063fac42e576464ce1} ``` -#### 获取台湾地区气象站列表 +#### 获取台湾地区气象站列表 {#retrieve-the-taiwan-weather-stations} ```bash wget -O weather_sta_list.csv https://github.com/Raingel/weather_station_list/raw/main/data/weather_sta_list.csv -# 可选:将 UTF-8-BOM 转换为 UTF-8 编码 +# 可选:将 UTF-8-BOM 转换为 UTF-8 编码 {#option-convert-the-utf-8-bom-to-utf-8-encoding} sed -i '1s/^\xEF\xBB\xBF//' weather_sta_list.csv ``` -## 创建表结构 +## 创建表结构 {#create-table-schema} 使用 ClickHouse 客户端在 ClickHouse 中创建 MergeTree 表。 @@ -144,7 +144,7 @@ ORDER BY (MeasuredDate); ## 向 ClickHouse 插入数据 {#inserting-into-clickhouse} -### 从本地文件插入 +### 从本地文件插入 {#inserting-from-local-file} 可以在 ClickHouse 客户端中通过以下方式从本地文件插入数据: @@ -165,7 +165,7 @@ INSERT INTO tw_weather_data FROM INFILE '/path/to/daily_weather_preprocessed_189 ``` -### 从 URL 插入数据 +### 从 URL 插入数据 {#inserting-from-url} ```sql INSERT INTO tw_weather_data SELECT * @@ -176,7 +176,7 @@ FROM url('https://storage.googleapis.com/taiwan-weather-observaiton-datasets/dai 如需了解如何加快这一过程,请参阅我们关于[优化大规模数据加载](https://clickhouse.com/blog/supercharge-your-clickhouse-data-loads-part2)的博文。 -## 检查数据行和大小 +## 检查数据行和大小 {#check-data-rows-and-sizes} 1. 先查看已插入了多少行: @@ -210,7 +210,7 @@ WHERE (`table` = 'tw_weather_data') AND active ## 查询示例 {#sample-queries} -### Q1: 查询指定年份中每个气象站的最高露点温度 +### Q1: 查询指定年份中每个气象站的最高露点温度 {#q1-retrieve-the-highest-dew-point-temperature-for-each-weather-station-in-the-specific-year} ```sql SELECT @@ -257,7 +257,7 @@ GROUP BY StationId ``` -### Q2: 在特定时间范围内按字段和气象站获取原始数据 +### Q2: 在特定时间范围内按字段和气象站获取原始数据 {#q2-raw-data-fetching-with-the-specific-duration-time-range-fields-and-weather-station} ```sql SELECT diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md index af1ced26910..7a2796ae642 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/uk-price-paid.md @@ -13,7 +13,7 @@ keywords: ['示例数据集', '英国房产', '示例数据', '房地产', '入 - 字段说明: https://www.gov.uk/guidance/about-the-price-paid-data - 包含 HM Land Registry 数据 © Crown copyright and database right 2021。本数据依据 Open Government Licence v3.0 授权许可使用。 -## 创建数据表 +## 创建数据表 {#create-table} ```sql CREATE DATABASE uk; @@ -40,7 +40,7 @@ ORDER BY (postcode1, postcode2, addr1, addr2); ``` -## 预处理并插入数据 +## 预处理并插入数据 {#preprocess-import-data} 我们将使用 `url` 函数将数据流式写入 ClickHouse。首先需要对部分传入数据进行预处理,包括: @@ -95,7 +95,7 @@ FROM url( 等待数据插入完成;根据网络速度,这可能需要一到两分钟。 -## 验证数据 +## 验证数据 {#validate-data} 通过查看插入了多少行来验证是否生效: @@ -119,7 +119,7 @@ WHERE name = 'uk_price_paid' 我们来运行一些查询来分析数据: -### 查询 1:各年份的平均价格 +### 查询 1:各年份的平均价格 {#average-price} ```sql runnable SELECT @@ -133,7 +133,7 @@ ORDER BY year ``` -### 查询 2:伦敦每年的平均价格 +### 查询 2:伦敦每年的平均价格 {#average-price-london} ```sql runnable SELECT @@ -150,7 +150,7 @@ ORDER BY year 2020 年房价发生了点变化!不过这大概不算什么意外…… -### 查询 3:最昂贵的街区 +### 查询 3:最昂贵的街区 {#most-expensive-neighborhoods} ```sql runnable SELECT diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md index 29eb820262b..90beb9a30ed 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/example-datasets/youtube-dislikes.md @@ -241,7 +241,7 @@ keywords: ['示例数据集', 'youtube', '示例数据', '视频分析', '点踩 ## 常见问题 {#questions} -### 如果视频关闭了评论功能,会不会降低观众点点赞或点踩的可能性? +### 如果视频关闭了评论功能,会不会降低观众点点赞或点踩的可能性? {#create-the-table} 当评论被关闭时,观众是否更倾向于通过点赞或点踩来表达他们对视频的看法? @@ -297,7 +297,7 @@ ORDER BY 启用评论似乎与更高的参与度有关。 -### 视频数量随时间如何变化——有哪些值得关注的事件? +### 视频数量随时间如何变化——有哪些值得关注的事件? {#insert-data} ```sql SELECT @@ -336,7 +336,7 @@ ORDER BY month ASC; 可以明显看出,在新冠疫情前后,上传者数量出现了激增([相关现象可见此处](https://www.theverge.com/2020/3/27/21197642/youtube-with-me-style-videos-views-coronavirus-cook-workout-study-home-beauty))。 -### 字幕数量随时间的变化及其出现时间 +### 字幕数量随时间的变化及其出现时间 {#count-row-numbers} 随着语音识别技术的进步,为视频创建字幕比以往任何时候都更容易。YouTube 在 2009 年底推出自动字幕功能——转折点就是那时吗? @@ -374,7 +374,7 @@ ORDER BY month ASC; 这引发了一场非常成功的活动,号召创作者为他们的视频添加字幕,以方便听力障碍和失聪的观众。 -### 各时间段上传量最高的用户 +### 各时间段上传量最高的用户 {#explore-the-data} ```sql WITH uploaders AS @@ -467,7 +467,7 @@ ORDER BY ``` -### 视图是如何分布的? +### 视图是如何分布的? {#if-someone-disables-comments-does-it-lower-the-chance-someone-will-actually-click-like-or-dislike} ```sql SELECT diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/index.md index fbc2cdfe715..75f01c972c0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/index.md @@ -9,9 +9,7 @@ title: '教程和示例数据集' doc_type: 'landing-page' --- - - -# 教程和示例数据集 +# 教程和示例数据集 {#tutorials-and-example-datasets} 我们提供了大量资源,帮助你上手并了解 ClickHouse 的工作原理: @@ -24,7 +22,6 @@ doc_type: 'landing-page' {/* 下表在构建时由脚本 https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh 自动生成 */ } - {/*AUTOGENERATED_START*/ } | 页面 | 描述 | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md index 397f2b6f8d9..5f8e1c50406 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_deb_install.md @@ -17,20 +17,30 @@ import TabItem from '@theme/TabItem'; ```bash -# 安装先决条件软件包 +# 安装先决条件软件包 {#install-prerequisite-packages} sudo apt-get install -y apt-transport-https ca-certificates curl gnupg +``` + -# 下载 ClickHouse 的 GPG 密钥并将其存储到密钥环中 +# 下载 ClickHouse 的 GPG 密钥并将其存储到密钥环中 {#download-the-clickhouse-gpg-key-and-store-it-in-the-keyring} curl -fsSL 'https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key' | sudo gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg -# 获取系统架构 + + +# 获取系统架构 {#get-the-system-architecture} ARCH=$(dpkg --print-architecture) -# 将 ClickHouse 仓库添加到 APT 软件源列表 + + +# 将 ClickHouse 仓库添加到 APT 软件源列表 {#add-the-clickhouse-repository-to-apt-sources} echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=${ARCH}] https://packages.clickhouse.com/deb stable main" | sudo tee /etc/apt/sources.list.d/clickhouse.list -# 更新 apt 软件包列表 + + +# 更新 apt 软件包列表 {#update-apt-package-lists} + sudo apt-get update + ``` - 您可以将 `stable` 替换为 `lts`,以根据需要使用不同的[发布通道](/knowledgebase/production)。 @@ -38,42 +48,59 @@ sudo apt-get update
用于安装 deb 软件包的旧分发方式 +``` + ```bash -# 安装前置依赖包 +# 安装前置依赖包 {#install-prerequisite-packages} sudo apt-get install apt-transport-https ca-certificates dirmngr +``` -# 添加 ClickHouse GPG 密钥用于验证软件包 + +# 添加 ClickHouse GPG 密钥用于验证软件包 {#add-the-clickhouse-gpg-key-to-authenticate-packages} sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754 -# 将 ClickHouse 软件仓库添加到 apt 源列表 + + +# 将 ClickHouse 软件仓库添加到 apt 源列表 {#add-the-clickhouse-repository-to-apt-sources} echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \ /etc/apt/sources.list.d/clickhouse.list + + -# 更新 apt 软件包列表 +# 更新 apt 软件包列表 {#update-apt-package-lists} sudo apt-get update -# 安装 ClickHouse 服务端和客户端软件包 + + +# 安装 ClickHouse 服务端和客户端软件包 {#install-clickhouse-server-and-client-packages} sudo apt-get install -y clickhouse-server clickhouse-client -# 启动 ClickHouse 服务 + + +# 启动 ClickHouse 服务 {#start-the-clickhouse-server-service} sudo service clickhouse-server start -# 启动 ClickHouse 命令行客户端 + + +# 启动 ClickHouse 命令行客户端 {#launch-the-clickhouse-command-line-client} + clickhouse-client # 如果已设置密码,请使用 "clickhouse-client --password"。 + ```
+``` -## 安装 ClickHouse 服务端和客户端 +## 安装 ClickHouse 服务端和客户端 {#install-clickhouse-server-and-client} ```bash sudo apt-get install -y clickhouse-server clickhouse-client ``` -## 启动 ClickHouse +## 启动 ClickHouse {#start-clickhouse-server} 要启动 ClickHouse 服务器,请运行: @@ -94,7 +121,7 @@ clickhouse-client --password ``` -## 安装独立的 ClickHouse Keeper +## 安装独立的 ClickHouse Keeper {#install-standalone-clickhouse-keeper} :::tip 在生产环境中,我们强烈建议在专用节点上运行 ClickHouse Keeper。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md index 6553cdc7ef6..9ac903655ae 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_docker.md @@ -1,4 +1,4 @@ -# 使用 Docker 安装 ClickHouse +# 使用 Docker 安装 ClickHouse {#install-clickhouse-using-docker} 为方便起见,下面转载了 [Docker Hub](https://hub.docker.com/r/clickhouse/clickhouse-server/) 上的指南。可用的 Docker 镜像使用的是官方提供的 ClickHouse deb 软件包。 @@ -31,9 +31,9 @@ docker pull clickhouse/clickhouse-server -## 如何使用该镜像 +## 如何使用该镜像 {#how-to-use-image} -### 启动服务器实例 +### 启动服务器实例 {#start-server-instance} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickhouse/clickhouse-server @@ -43,18 +43,18 @@ docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 clickh 默认情况下,上面启动的服务器实例将以无密码的 `default` 用户身份运行。 -### 从原生客户端连接到它 +### 从原生客户端连接到它 {#connect-to-it-from-native-client} ```bash docker run -it --rm --network=container:some-clickhouse-server --entrypoint clickhouse-client clickhouse/clickhouse-server -# 或者 +# 或者 {#or} docker exec -it some-clickhouse-server clickhouse-client ``` 有关 ClickHouse 客户端的更多信息,请参阅[ClickHouse 客户端](/interfaces/cli)。 -### 使用 curl 进行连接 +### 使用 curl 进行连接 {#connect-to-it-using-curl} ```bash echo "SELECT 'Hello, ClickHouse!'" | docker run -i --rm --network=container:some-clickhouse-server buildpack-deps:curl curl 'http://localhost:8123/?query=' -s --data-binary @- @@ -62,14 +62,14 @@ echo "SELECT 'Hello, ClickHouse!'" | docker run -i --rm --network=container:some 有关 HTTP 接口的更多信息,请参阅 [ClickHouse HTTP 接口](/interfaces/http)。 -### 停止/移除容器 +### 停止/移除容器 {#stopping-removing-container} ```bash docker stop some-clickhouse-server docker rm some-clickhouse-server ``` -### 网络 +### 网络 {#networking} :::note 预定义用户 `default` 在未设置密码之前不具备网络访问权限, @@ -95,7 +95,7 @@ echo 'SELECT version()' | curl 'http://localhost:8123/' --data-binary @- 上面示例中的用户默认配置仅适用于本机(localhost)请求 ::: -### 卷(Volumes) +### 卷(Volumes) {#volumes} 通常,为了实现持久化,你可能需要在容器内挂载以下目录: @@ -116,7 +116,7 @@ docker run -d \ * `/docker-entrypoint-initdb.d/` - 包含数据库初始化脚本的文件夹(见下文)。 -## Linux 能力 +## Linux 能力 {#linear-capabilities} ClickHouse 提供了一些高级功能,这些功能需要启用若干 [Linux 能力(capabilities)](https://man7.org/linux/man-pages/man7/capabilities.7.html)。 @@ -131,29 +131,29 @@ docker run -d \ 如需了解更多信息,请参阅 ["在 Docker 中配置 CAP_IPC_LOCK 和 CAP_SYS_NICE 权限"](/knowledgebase/configure_cap_ipc_lock_and_cap_sys_nice_in_docker) -## 配置 +## 配置 {#configuration} 该容器对外暴露 8123 端口用于 [HTTP 接口](https://clickhouse.com/docs/interfaces/http_interface/),以及 9000 端口用于 [原生客户端](https://clickhouse.com/docs/interfaces/tcp/)。 ClickHouse 的配置由名为 “config.xml” 的文件进行定义([文档](https://clickhouse.com/docs/operations/configuration_files/))。 -### 使用自定义配置启动服务器实例 +### 使用自定义配置启动服务器实例 {#start-server-instance-with-custom-config} ```bash docker run -d --name some-clickhouse-server --ulimit nofile=262144:262144 -v /path/to/your/config.xml:/etc/clickhouse-server/config.xml clickhouse/clickhouse-server ``` -### 以自定义用户身份启动服务器 +### 以自定义用户身份启动服务器 {#start-server-custom-user} ```bash -# $PWD/data/clickhouse 目录应存在且归当前用户所有 +# $PWD/data/clickhouse 目录应存在且归当前用户所有 {#pwddataclickhouse-should-exist-and-be-owned-by-current-user} docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit nofile=262144:262144 -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` 当你使用挂载本地目录的镜像时,通常需要指定用户以保持正确的文件所有权。使用 `--user` 参数,并在容器内挂载 `/var/lib/clickhouse` 和 `/var/log/clickhouse-server`。否则,镜像会报错,容器将无法启动。 -### 以 root 启动服务器 +### 以 root 启动服务器 {#start-server-from-root} 在启用了用户命名空间的场景下,以 root 用户身份启动服务器会很有用。 要这样做,请运行: @@ -162,7 +162,7 @@ docker run --rm --user "${UID}:${GID}" --name some-clickhouse-server --ulimit no docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v "$PWD/logs/clickhouse:/var/log/clickhouse-server" -v "$PWD/data/clickhouse:/var/lib/clickhouse" clickhouse/clickhouse-server ``` -### 如何在启动时创建默认数据库和用户 +### 如何在启动时创建默认数据库和用户 {#how-to-create-default-db-and-user} 有时你可能希望在容器启动时创建一个用户(系统默认使用名为 `default` 的用户)和一个数据库。你可以通过环境变量 `CLICKHOUSE_DB`、`CLICKHOUSE_USER`、`CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` 和 `CLICKHOUSE_PASSWORD` 来实现: @@ -170,7 +170,7 @@ docker run --rm -e CLICKHOUSE_RUN_AS_ROOT=1 --name clickhouse-server-userns -v " docker run --rm -e CLICKHOUSE_DB=my_database -e CLICKHOUSE_USER=username -e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 -e CLICKHOUSE_PASSWORD=password -p 9000:9000/tcp clickhouse/clickhouse-server ``` -#### 管理 `default` 用户 +#### 管理 `default` 用户 {#managing-default-user} 当未设置 `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD` 或 `CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT` 中任意一个时,`default` 用户默认禁用网络访问。 @@ -181,7 +181,7 @@ docker run --rm -e CLICKHOUSE_SKIP_USER_SETUP=1 -p 9000:9000/tcp clickhouse/clic ``` -## 如何扩展此镜像 +## 如何扩展此镜像 {#how-to-extend-image} 要在基于本镜像的自定义镜像中执行额外的初始化操作,请在 `/docker-entrypoint-initdb.d` 目录下添加一个或多个 `*.sql`、`*.sql.gz` 或 `*.sh` 脚本。入口点脚本调用 `initdb` 之后,会运行所有 `*.sql` 文件、执行所有可执行的 `*.sh` 脚本,并对该目录中所有不可执行的 `*.sh` 脚本执行 `source` 引入操作,以在启动服务前完成进一步初始化。\ 另外,你也可以提供环境变量 `CLICKHOUSE_USER` 和 `CLICKHOUSE_PASSWORD`,它们会在初始化期间被 clickhouse-client 使用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md index 231282dc539..d0b37f7109c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_linux_tar_install.md @@ -1,4 +1,4 @@ -# 使用 tgz 归档文件安装 ClickHouse +# 使用 tgz 归档文件安装 ClickHouse {#install-clickhouse-using-tgz-archives} > 对于无法安装 `deb` 或 `rpm` 软件包的 Linux 发行版,建议使用官方预编译的 `tgz` 归档文件。 @@ -20,7 +20,7 @@ -## 获取最新的 ClickHouse 版本 +## 获取最新的 ClickHouse 版本 {#get-latest-version} 从 GitHub 获取最新的 ClickHouse 版本,并将其保存到 `LATEST_VERSION` 变量中。 @@ -31,7 +31,7 @@ export LATEST_VERSION ``` -## 检测系统架构 +## 检测系统架构 {#detect-system-architecture} 检测系统架构,并相应设置 ARCH 变量: @@ -44,7 +44,7 @@ esac ``` -## 为每个 ClickHouse 组件下载 tar 包 +## 为每个 ClickHouse 组件下载 tar 包 {#download-tarballs} 为每个 ClickHouse 组件下载 tar 包。该循环会优先尝试特定架构的 软件包,如果不存在则退回到通用软件包。 @@ -66,7 +66,7 @@ done ```bash -# 解压并安装 clickhouse-common-static 软件包 +# 解压并安装 clickhouse-common-static 软件包 {#extract-and-install-clickhouse-common-static-package} tar -xzvf "clickhouse-common-static-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" @@ -76,7 +76,7 @@ sudo "clickhouse-common-static-$LATEST_VERSION/install/doinst.sh" ```bash -# 提取并安装调试符号包 +# 提取并安装调试符号包 {#extract-and-install-debug-symbols-package} tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-common-static-dbg-$LATEST_VERSION.tgz" sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" @@ -86,7 +86,7 @@ sudo "clickhouse-common-static-dbg-$LATEST_VERSION/install/doinst.sh" ```bash -# 解压并安装服务器软件包及配置 +# 解压并安装服务器软件包及配置 {#extract-and-install-server-package-with-configuration} tar -xzvf "clickhouse-server-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-server-$LATEST_VERSION.tgz" sudo "clickhouse-server-$LATEST_VERSION/install/doinst.sh" configure @@ -97,7 +97,7 @@ sudo /etc/init.d/clickhouse-server start # 启动服务器 ```bash -# 提取并安装客户端软件包 +# 提取并安装客户端软件包 {#extract-and-install-client-package} tar -xzvf "clickhouse-client-$LATEST_VERSION-${ARCH}.tgz" \ || tar -xzvf "clickhouse-client-$LATEST_VERSION.tgz" sudo "clickhouse-client-$LATEST_VERSION/install/doinst.sh" diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md index b8c0c09030c..e79c70711d2 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_macos.md @@ -5,12 +5,12 @@ import privacy_allow from "@site/static/images/knowledgebase/fix-the-developer-v -# 使用 Homebrew 安装 ClickHouse +# 使用 Homebrew 安装 ClickHouse {#install-clickhouse-using-homebrew} -## 使用社区 Homebrew 配方进行安装 +## 使用社区 Homebrew 配方进行安装 {#install-using-community-homebrew-formula} 要在 macOS 上使用 [Homebrew](https://brew.sh/) 安装 ClickHouse,可以使用 ClickHouse 社区提供的 [Homebrew 配方](https://formulae.brew.sh/cask/clickhouse)。 @@ -19,7 +19,7 @@ brew install --cask clickhouse ``` -## 在 macOS 中修复开发者验证错误 +## 在 macOS 中修复开发者验证错误 {#fix-developer-verification-error-macos} 如果你使用 `brew` 安装 ClickHouse,可能会遇到来自 macOS 的错误提示。 默认情况下,macOS 不会运行由无法验证身份的开发者创建的应用程序或工具。 @@ -30,7 +30,7 @@ brew install --cask clickhouse 要绕过此验证错误,你需要将该应用从 macOS 的隔离区中移除,可以通过以下任一方式完成:在系统设置窗口中找到相应设置、使用终端,或者重新安装 ClickHouse。 -### 系统设置流程 +### 系统设置流程 {#system-settings-process} 将 `clickhouse` 可执行文件从隔离区移除的最简单方式是: @@ -50,7 +50,7 @@ brew install --cask clickhouse 现在你应该可以在终端中运行 `clickhouse` 命令了。 -### 终端流程 +### 终端流程 {#terminal-process} 有时点击 `Allow Anyway` 按钮并不能解决该问题,在这种情况下,你也可以通过命令行来完成这一流程。 或者你可能只是更喜欢使用命令行! diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md index 49e53e4f083..1528b065cce 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_quick_install.md @@ -1,11 +1,11 @@ -# 使用 curl 脚本安装 ClickHouse +# 使用 curl 脚本安装 ClickHouse {#install-clickhouse-via-script-using-curl} 如果无需在生产环境中安装 ClickHouse,最快的安装方式是使用 curl 运行安装脚本。该脚本会自动为您的操作系统选择合适的二进制文件。 -## 使用 curl 安装 ClickHouse +## 使用 curl 安装 ClickHouse {#install-clickhouse-using-curl} 运行以下命令,为你的操作系统下载一个独立的二进制可执行文件。 @@ -18,7 +18,7 @@ Mac 用户:如果你遇到提示无法验证二进制文件开发者的错误 ::: -## 启动 clickhouse-local +## 启动 clickhouse-local {#start-clickhouse-local} `clickhouse-local` 允许使用 ClickHouse 强大的 SQL 语法,在无需任何配置的情况下处理本地和远程文件。表数据会存储在临时位置,这意味着在重启 `clickhouse-local` 之后,先前创建的表将不再可用。 @@ -29,7 +29,7 @@ Mac 用户:如果你遇到提示无法验证二进制文件开发者的错误 ``` -## 启动 clickhouse-server +## 启动 clickhouse-server {#start-clickhouse-server} 如果您希望持久化存储数据,需要运行 `clickhouse-server`。您可以使用以下命令启动 ClickHouse 服务器: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md index 56bedaaca51..1a637e0d893 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_rpm_install.md @@ -5,7 +5,7 @@ -## 配置 RPM 软件源 +## 配置 RPM 软件源 {#setup-the-rpm-repository} 运行以下命令添加官方软件源: @@ -24,7 +24,7 @@ sudo zypper --gpg-auto-import-keys refresh clickhouse-stable 在以下步骤中,您可以根据所使用的包管理器,将 `yum install` 替换为 `zypper install`。 -## 安装 ClickHouse 服务端和客户端 +## 安装 ClickHouse 服务端和客户端 {#install-clickhouse-server-and-client-1} 要安装 ClickHouse,请运行以下命令: @@ -41,7 +41,7 @@ sudo yum install clickhouse-server-22.8.7.34 ``` -## 启动 ClickHouse 服务器 +## 启动 ClickHouse 服务器 {#start-clickhouse-server-1} 要启动 ClickHouse 服务器,运行以下命令: @@ -64,7 +64,7 @@ clickhouse-client --password ``` -## 安装独立的 ClickHouse Keeper +## 安装独立的 ClickHouse Keeper {#install-standalone-clickhouse-keeper-1} :::tip 在生产环境中,我们强烈建议在独立节点上运行 ClickHouse Keeper。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md index db56ce4aee4..94e749b7ee7 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/_snippets/_windows_install.md @@ -1,4 +1,4 @@ -# 在 Windows 上通过 WSL 安装 ClickHouse +# 在 Windows 上通过 WSL 安装 ClickHouse {#install-clickhouse-on-windows-with-wsl} @@ -11,7 +11,7 @@ -## 安装 WSL +## 安装 WSL {#install-wsl} 以管理员身份打开 Windows PowerShell,并运行以下命令: @@ -26,7 +26,7 @@ wsl --install ``` -## 使用 curl 脚本安装 ClickHouse +## 使用 curl 脚本安装 ClickHouse {#install-clickhouse-via-script-using-curl} 运行以下命令,通过 curl 脚本安装 ClickHouse: @@ -42,7 +42,7 @@ curl https://clickhouse.com/ | sh ``` -## 启动 clickhouse-local +## 启动 clickhouse-local {#start-clickhouse-local} `clickhouse-local` 可用于在无需任何配置的情况下,借助 ClickHouse 强大的 SQL 语法处理本地和远程文件。表数据会存储在临时位置,这意味着在重启 `clickhouse-local` 后,此前创建的表将不再可用。 @@ -53,7 +53,7 @@ curl https://clickhouse.com/ | sh ``` -## 启动 clickhouse-server +## 启动 clickhouse-server {#start-clickhouse-server} 若要持久化数据,应运行 `clickhouse-server`。可以使用以下命令启动 ClickHouse 服务器: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md index 2421e3bf855..526cdb70b90 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/advanced.md @@ -10,7 +10,7 @@ doc_type: 'guide' -## 从源码编译 +## 从源码编译 {#compile-from-source} 要手动编译 ClickHouse,请按照 [Linux](/development/build.md) 或 [macOS](/development/build-osx.md) 的说明进行操作。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md index 9f5e1f73f24..073feaeb65a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/debian_ubuntu.md @@ -10,4 +10,4 @@ doc_type: 'guide' import DebianProd from './_snippets/_deb_install.md' - + diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx index d808bfce012..e9148c3f012 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/install/install.mdx @@ -18,33 +18,24 @@ import Docker from './_snippets/_docker.md' import {CardPrimary} from '@clickhouse/click-ui/bundled'; import {galaxyOnClick} from '@site/src/lib/galaxy/galaxy' - -# 安装指南 +# 安装指南 \{#installation-instructions\} -
+
或者,在下方选择平台、发行版和安装方式,以查看适用于开源 ClickHouse 的安装指南: -} - quickinstall={} - debian_prod={} - rpm_prod={} - tar_prod={} - macos_prod={} - docker={} -/> \ No newline at end of file +} quickinstall={} debian_prod={} rpm_prod={} tar_prod={} macos_prod={} docker={} /> \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/playground.md b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/playground.md index 6134f5bfb2e..5d5a2bf4928 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/playground.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/playground.md @@ -9,7 +9,7 @@ doc_type: '指南' -# ClickHouse Playground +# ClickHouse Playground {#clickhouse-playground} [ClickHouse Playground](https://sql.clickhouse.com) 允许用户无需自行搭建服务器或集群,即可通过即时运行查询来试用和探索 ClickHouse。 Playground 中提供了若干示例数据集。 @@ -40,7 +40,7 @@ Playground 中提供了若干示例数据集。 -## 示例 +## 示例 {#examples} 使用 `curl` 访问 HTTPS 端点的示例: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx index b58df2ad326..86f37009e0c 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/cloud.mdx @@ -20,15 +20,14 @@ import data_sources from '@site/static/images/_snippets/data_sources.png'; import select_data_source from '@site/static/images/_snippets/select_data_source.png'; import client_details from '@site/static/images/_snippets/client_details.png'; import new_rows_from_csv from '@site/static/images/_snippets/new_rows_from_csv.png'; -import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; +import SQLConsoleDetail from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_launch_sql_console.md'; - -# ClickHouse Cloud 快速开始 +# ClickHouse Cloud 快速开始 \{#clickhouse-cloud-quick-start\} > 使用 ClickHouse 的最快、最简便方式,是在 [ClickHouse Cloud](https://console.clickhouse.cloud) 中创建一个新服务。在本快速入门指南中,我们将通过三个简单步骤帮助您完成设置。 - ## 创建 ClickHouse 服务 + ## 创建 ClickHouse 服务 \{#1-create-a-clickhouse-service\} 要在 [ClickHouse Cloud](https://console.clickhouse.cloud) 中创建免费的 ClickHouse 服务,只需按照以下步骤完成注册: @@ -57,7 +56,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 恭喜!您的 ClickHouse Cloud 服务已启动并运行,初始配置已完成。请继续阅读,了解如何开始摄取和查询数据。 - ## 连接到 ClickHouse + ## 连接到 ClickHouse \{#2-connect-to-clickhouse\} 连接到 ClickHouse 有两种方式: @@ -66,7 +65,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### 使用 SQL 控制台连接 + ### 使用 SQL 控制台连接 \{#connect-using-sql-console\} 为便于快速上手,ClickHouse 提供了基于 Web 的 SQL 控制台,完成初始配置后将自动跳转至该控制台。 @@ -86,7 +85,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 完成了 - 您现在可以开始使用新的 ClickHouse 服务了! - ### 连接您的应用 + ### 连接您的应用 \{#connect-with-your-app\} 从导航菜单中点击连接按钮。将弹出一个对话框,显示您的服务凭据,并提供使用界面或语言客户端进行连接的操作说明。 @@ -96,7 +95,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 如果您找不到所需的语言客户端,请查看我们的[集成列表](/integrations)。 - ## 添加数据 + ## 添加数据 \{#3-add-data\} ClickHouse 需要数据才能发挥最佳性能!添加数据有多种方式,大部分可通过导航菜单中的数据源页面访问。 @@ -112,7 +111,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; * 上传文件 - 接受的格式包括 JSON、CSV 和 TSV * 通过文件 URL 导入数据 - ### ClickPipes + ### ClickPipes \{#clickpipes\} [ClickPipes](http://clickhouse.com/docs/integrations/clickpipes) 是一个托管集成平台,只需点击几下即可轻松完成从各种数据源摄取数据。ClickPipes 专为最苛刻的工作负载而设计,其强大且可扩展的架构确保了一致的性能和可靠性。ClickPipes 既可用于长期流式传输需求,也可用于一次性数据加载作业。 @@ -120,7 +119,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md';
- ### 使用 SQL 控制台添加数据 + ### 使用 SQL 控制台添加数据 \{#add-data-using-the-sql-console\} 与大多数数据库管理系统一样,ClickHouse 在逻辑上将表组织到**数据库**中。使用 [`CREATE DATABASE`](../../sql-reference/statements/create/database.md) 命令在 ClickHouse 中创建新数据库: @@ -161,7 +160,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 有多种表引擎可供选择,但对于单节点 ClickHouse 服务器上的简单表,建议使用 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md)。 ::: - #### 主键简介 + #### 主键简介 \{#a-brief-intro-to-primary-keys\} 在继续操作之前,务必了解 ClickHouse 中主键的工作原理(主键的实现方式可能会出乎意料!): @@ -176,7 +175,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; 如需深入了解 ClickHouse 核心概念,请参阅["核心概念"](../../managing-data/core-concepts/index.md)。 - #### 向表中插入数据 + #### 向表中插入数据 \{#insert-data-into-your-table\} 您可以使用熟悉的 [`INSERT INTO TABLE`](../../sql-reference/statements/insert-into.md) 命令向 ClickHouse 插入数据,但需要注意的是,每次向 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) 表执行插入操作都会在存储中创建一个**数据部分**(part)。 @@ -206,7 +205,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; SELECT * FROM helloworld.my_first_table ``` - ### 使用 ClickHouse 客户端添加数据 + ### 使用 ClickHouse 客户端添加数据 \{#add-data-using-the-clickhouse-client\} 您还可以使用名为 [**clickhouse client**](/interfaces/cli) 的命令行工具连接到 ClickHouse Cloud 服务。点击左侧菜单中的 `Connect` 访问相关详细信息。在弹出的对话框中,从下拉菜单选择 `Native`: @@ -286,7 +285,7 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; exit ``` - ### 上传文件 + ### 上传文件 \{#upload-a-file\} 开始使用数据库时的一项常见任务是插入已有文件中的数据。我们提供了一些在线示例数据供您插入,这些数据代表点击流数据,包括用户 ID、访问的 URL 以及事件时间戳。 @@ -321,9 +320,9 @@ import SQLConsoleDetail from '@site/docs/_snippets/_launch_sql_console.md'; ## 接下来? \{#whats-next\} -- 在 [教程](/tutorial.md) 中,您将向一张表插入 200 万行数据,并编写一些分析查询 -- 我们提供了一份[示例数据集](/getting-started/index.md)列表,并附有插入这些数据集的详细说明 -- 查看我们时长 25 分钟的 [ClickHouse 入门](https://clickhouse.com/company/events/getting-started-with-clickhouse/) 视频 -- 如果您的数据来自外部来源,请查看我们的[集成指南合集](/integrations/index.mdx),了解如何连接消息队列、数据库、数据管道等 -- 如果您在使用 UI/BI 可视化工具,请查看[将 UI 连接到 ClickHouse 的用户指南](/integrations/data-visualization) -- [主键](/guides/best-practices/sparse-primary-indexes.md)用户指南涵盖了您需要了解的所有主键信息及其定义方式 \ No newline at end of file +* 在 [教程](/tutorial.md) 中,您将向一张表插入 200 万行数据,并编写一些分析查询 +* 我们提供了一份[示例数据集](/getting-started/index.md)列表,并附有插入这些数据集的详细说明 +* 查看我们时长 25 分钟的 [ClickHouse 入门](https://clickhouse.com/company/events/getting-started-with-clickhouse/) 视频 +* 如果您的数据来自外部来源,请查看我们的[集成指南合集](/integrations/index.mdx),了解如何连接消息队列、数据库、数据管道等 +* 如果您在使用 UI/BI 可视化工具,请查看[将 UI 连接到 ClickHouse 的用户指南](/integrations/data-visualization) +* [主键](/guides/best-practices/sparse-primary-indexes.md)用户指南涵盖了您需要了解的所有主键信息及其定义方式 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx index 6c64699fd32..2b3969577e8 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx +++ b/i18n/zh/docusaurus-plugin-content-docs/current/getting-started/quick-start/oss.mdx @@ -14,13 +14,12 @@ import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import {VerticalStepper} from '@clickhouse/click-ui/bundled'; - -# ClickHouse OSS 快速入门 +# ClickHouse OSS 快速入门 \{#clickhouse-oss-quick-start\} > 在本快速入门教程中,我们将通过 8 个简单步骤帮助你完成 OSS ClickHouse 的设置。你将下载适用于你操作系统的二进制文件,学习如何运行 ClickHouse server,并使用 ClickHouse client 创建一张表,然后向其中插入数据并运行查询来选取这些数据。 - ## 下载 ClickHouse + ## 下载 ClickHouse \{#download-the-binary\} ClickHouse 原生支持 Linux、FreeBSD 和 macOS,并可通过 [WSL](https://learn.microsoft.com/en-us/windows/wsl/about) 在 Windows 上运行。 在本地下载 ClickHouse 最简单的方式是运行以下 `curl` 命令。 该命令将检测您的操作系统是否受支持,然后下载从 master 分支构建的相应 ClickHouse 二进制文件。 @@ -51,7 +50,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; Mac 用户注意:如果您遇到无法验证二进制文件开发者的错误,请参阅["修复 macOS 中的开发者验证错误"](https://clickhouse.com/docs/knowledgebase/fix-developer-verification-error-in-macos)。 ::: - ## 启动服务器 + ## 启动服务器 \{#start-the-server\} 运行以下命令启动 ClickHouse 服务器: @@ -63,7 +62,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; [默认日志级别](https://clickhouse.com/docs/knowledgebase/why_default_logging_verbose) 设置为 `trace` 而非 `warning`。 - ## 启动客户端 + ## 启动客户端 \{#start-the-client\} 使用 `clickhouse-client` 连接到您的 ClickHouse 服务。打开新终端,切换到 `clickhouse` 二进制文件所在的目录,然后运行以下命令: @@ -77,7 +76,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; my-host :) ``` - ## 创建表 + ## 创建表 \{#create-a-table\} 使用 `CREATE TABLE` 定义新表。常规 SQL DDL 命令在 ClickHouse 中均可使用,但需注意一点——ClickHouse 中的表必须指定 `ENGINE` 子句。使用 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 引擎可充分发挥 ClickHouse 的性能优势: @@ -93,7 +92,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; PRIMARY KEY (user_id, timestamp) ``` - ## 插入数据 + ## 插入数据 \{#insert-data\} 您可以使用熟悉的 `INSERT INTO TABLE` 命令操作 ClickHouse,但需要理解的是,每次向 `MergeTree` 表插入数据都会在存储中创建我们所称的 **part**(数据部分)。这些 ^^part^^ 随后会由 ClickHouse 在后台进行合并。 @@ -109,7 +108,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; (101, 'Granule 是数据读取的最小单元', now() + 5, 3.14159 ) ``` - ## 查询您的新表 + ## 查询您的新表 \{#query-your-new-table\} 您可以像在任何 SQL 数据库中一样编写 `SELECT` 查询: @@ -132,7 +131,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; 返回 4 行。耗时:0.008 秒。 ``` - ## 插入您自己的数据 + ## 插入您自己的数据 \{#insert-your-own-data\} 下一步是将您自己的数据导入 ClickHouse。我们提供了大量的[表函数](/sql-reference/table-functions/index.md)和[集成](/integrations)用于摄取数据。您可以参考下方选项卡中的示例,或访问我们的[集成](/integrations)页面查看与 ClickHouse 集成的完整技术列表。 @@ -344,7 +343,7 @@ import {VerticalStepper} from '@clickhouse/click-ui/bundled'; - ## 探索 + ## 探索 \{#explore\} * 查看我们的[核心概念](/managing-data/core-concepts)章节,了解一些 ClickHouse 底层工作原理的基础概念。 * 请查看 [进阶教程](tutorial.md),该教程将更深入地探讨 ClickHouse 的核心概念和功能。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/index.md index ea6c16c91e2..46b317a5a16 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/index.md @@ -7,13 +7,12 @@ keywords: ['性能优化', '最佳实践', '优化指南', 'ClickHouse 性能', doc_type: 'reference' --- -import TableOfContents from '@site/docs/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; +import TableOfContents from '@site/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/_snippets/_performance_optimizations_table_of_contents.md'; - -# 性能与优化 +# 性能与优化 {#performance-and-optimizations} 本节提供有关提升 ClickHouse 性能的建议和最佳实践。 建议读者在阅读本节之前先查阅[核心概念](/parts), 其中介绍了改进性能所需掌握的主要概念。 - \ No newline at end of file + \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md index d2de24829ef..bfce716c5ba 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/prewhere.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/prewhere_05.gif' import Image from '@theme/IdealImage'; -# PREWHERE 优化是如何工作的? +# PREWHERE 优化是如何工作的? {#how-does-the-prewhere-optimization-work} [PREWHERE 子句](/sql-reference/statements/select/prewhere) 是 ClickHouse 中的一种查询执行优化机制。它通过避免不必要的数据读取、在从磁盘读取非过滤列之前先过滤掉无关数据,从而减少 I/O 并提升查询速度。 @@ -109,7 +109,7 @@ ClickHouse 首先通过 ① 从 `town` 列读取选定的 granule,并检查哪 -## 如何衡量 PREWHERE 的影响 +## 如何衡量 PREWHERE 的影响 {#how-to-measure-prewhere-impact} 要验证 PREWHERE 是否提升了查询性能,可以对比在启用和禁用 `optimize_move_to_prewhere` 设置时的查询表现。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md index 882f245d645..42199ebc2f0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-optimization.md @@ -11,7 +11,7 @@ import queryOptimizationDiagram1 from '@site/static/images/guides/best-practices import Image from '@theme/IdealImage'; -# 查询优化简明指南 +# 查询优化简明指南 {#a-simple-guide-for-query-optimization} 本节通过常见场景示例说明如何使用不同的性能优化技术,例如 [analyzer](/operations/analyzer)、[query profiling](/operations/optimizing-performance/sampling-query-profiler) 或 [avoid nullable Columns](/optimize/avoid-nullable-columns),从而提升 ClickHouse 查询性能。 @@ -61,7 +61,7 @@ ClickHouse 提供了一套丰富的工具,帮助你了解查询是如何执行 -## 数据集 +## 数据集 {#dataset} 我们将使用一个真实示例来说明我们是如何处理查询性能的。 @@ -112,9 +112,9 @@ ORDER BY tuple() ``` -## 找出慢查询 +## 找出慢查询 {#spot-the-slow-queries} -### 查询日志 +### 查询日志 {#query-logs} 默认情况下,ClickHouse 会在 [查询日志](/operations/system-tables/query_log) 中收集并记录每条已执行查询的信息。这些数据存储在表 `system.query_log` 中。  @@ -431,7 +431,7 @@ _最后,要留意离群值;某条查询偶尔运行缓慢是很常见的情 -## 基础优化 +## 基础优化 {#basic-optimization} 既然我们已经搭好了用于测试的框架,就可以开始进行优化了。 @@ -439,7 +439,7 @@ _最后,要留意离群值;某条查询偶尔运行缓慢是很常见的情 根据你摄取数据的方式,你可能利用了 ClickHouse 的[功能](/interfaces/schema-inference),基于摄取的数据推断表结构。虽然这在入门阶段非常实用,但如果你希望优化查询性能,就需要重新审视数据表结构,使其尽可能贴合你的具体用例。 -### Nullable +### Nullable {#nullable} 如[最佳实践文档](/best-practices/select-data-types#avoid-nullable-columns)中所述,应尽可能避免使用 Nullable 列。经常使用它们很有诱惑力,因为它们可以让数据摄取机制更加灵活,但每次都需要处理一个额外的列,会对性能产生负面影响。 @@ -485,7 +485,7 @@ dropoff_location_id_nulls: 0 我们只有两列存在 null 值:`mta_tax` 和 `payment_type`。其余字段不应该使用 `Nullable` 列类型。 -### 低基数(Low cardinality) +### 低基数(Low cardinality) {#low-cardinality} 对于 String 类型列,一个简单易行的优化是充分利用 LowCardinality 数据类型。正如低基数[文档](/sql-reference/data-types/lowcardinality)中所述,ClickHouse 会对 LowCardinality 列应用字典编码,从而显著提升查询性能。 @@ -515,7 +515,7 @@ uniq(vendor_id): 3 在基数较低的情况下,这四列 `ratecode_id`、`pickup_location_id`、`dropoff_location_id` 和 `vendor_id` 非常适合使用 LowCardinality 类型。 -### 优化数据类型 +### 优化数据类型 {#optimize-data-type} ClickHouse 支持大量数据类型。请务必在满足用例需求的前提下选择尽可能小的数据类型,以优化性能并减少磁盘上的数据存储空间。  @@ -538,7 +538,7 @@ Query id: 4306a8e1-2a9c-4b06-97b4-4d902d2233eb 对于日期,你应选择既符合数据集特性、又最适合支持你计划执行查询的精度。 -### 应用这些优化 +### 应用这些优化 {#apply-the-optimizations} 让我们创建一个新表来使用优化后的 schema,并重新摄取这些数据。 @@ -604,7 +604,7 @@ ORDER BY size DESC 新表相比之前的表小了很多。可以看到,该表的磁盘空间占用减少了约 34%(从 7.38 GiB 降至 4.89 GiB)。 -## 主键的重要性 +## 主键的重要性 {#the-importance-of-primary-keys} ClickHouse 中的主键与大多数传统数据库系统中的工作方式不同。在那些系统中,主键用于保证唯一性和数据完整性。任何插入重复主键值的尝试都会被拒绝,并且通常会创建基于 B-tree 或哈希的索引用于快速查找。  @@ -616,7 +616,7 @@ ClickHouse 中的主键与大多数传统数据库系统中的工作方式不同 ClickHouse 支持的其他选项,例如 Projection(投影)或物化视图,可以让你在相同数据上使用不同的主键集合。本系列博客的第二部分将更详细地讨论这一点。  -### 选择主键 +### 选择主键 {#choose-primary-keys} 选择正确的主键集合是一个复杂的话题,可能需要权衡和试验,才能找到最佳组合。  diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md index 7f8f25d5b12..2c217926c12 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/query-parallelism.md @@ -17,7 +17,7 @@ import visual05 from '@site/static/images/guides/best-practices/query-parallelis import Image from '@theme/IdealImage'; -# ClickHouse 如何并行执行查询 +# ClickHouse 如何并行执行查询 {#how-clickhouse-executes-a-query-in-parallel} ClickHouse [为速度而生](/concepts/why-clickhouse-is-so-fast)。它以高度并行的方式执行查询,利用所有可用的 CPU 核心,将数据分布到各个处理通道,并且经常将硬件推至其性能极限。 @@ -66,7 +66,7 @@ ClickHouse [为速度而生](/concepts/why-clickhouse-is-so-fast)。它以高度 -## 监控查询并行度 +## 监控查询并行度 {#monitoring-query-parallelism} 使用这些工具来验证你的查询是否充分利用了可用的 CPU 资源,并在没有充分利用时进行诊断。 @@ -126,12 +126,12 @@ ClickHouse 的[内嵌 Web UI](/interfaces/http)(在 `/play` 端点可用)可 注意:请从左到右阅读该可视化。每一行代表一条并行处理通道,它以数据块为单位进行流式处理,并应用过滤、聚合以及最终处理阶段等转换。在本例中,你可以看到与 `max_threads = 4` 设置对应的四条并行通道。 -### 在处理通道之间进行负载均衡 +### 在处理通道之间进行负载均衡 {#load-balancing-across-processing-lanes} 请注意,物理计划中的 `Resize` 算子会[重新分区并重新分发](/academic_overview#4-2-multi-core-parallelization)数据块流到各个处理通道,以保持它们的利用率均衡。当不同数据范围中满足查询谓词的行数相差较大时,这种再平衡尤为重要,否则某些通道可能会过载,而其他通道则处于空闲状态。通过重新分配工作量,较快的通道可以有效帮助较慢的通道,从而优化整体查询执行时间。 -## 为什么 max_threads 并不总是被严格遵守 +## 为什么 max_threads 并不总是被严格遵守 {#why-max-threads-isnt-always-respected} 如上所述,`n` 条并行处理通道的数量由 `max_threads` 参数控制,其默认值等于 ClickHouse 在该服务器上可用的 CPU 内核数量: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md index 55eebac9fd9..63a6c24cd75 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes-examples.md @@ -10,7 +10,7 @@ keywords: ['跳过索引', '数据跳过', '性能', '索引', '最佳实践'] -# 数据跳过索引示例 +# 数据跳过索引示例 {#data-skipping-index-examples} 本文汇总了 ClickHouse 中数据跳过索引的示例,展示如何定义每种类型、在什么场景下使用它们,以及如何验证它们是否已生效。所有功能都适用于 [MergeTree-family 表](/engines/table-engines/mergetree-family/mergetree)。 @@ -33,7 +33,7 @@ ClickHouse 支持五种跳过索引类型: 每一节都会通过示例数据展示用法,并演示如何在查询执行中验证索引是否被使用。 -## MinMax 索引 +## MinMax 索引 {#minmax-index} `minmax` 索引最适合在松散排序的数据上执行范围条件,或用于与 `ORDER BY` 相关联的列。 @@ -64,7 +64,7 @@ SELECT count() FROM events WHERE ts >= now() - 3600; 请参阅一个包含 `EXPLAIN` 和剪枝的[示例](/best-practices/use-data-skipping-indices-where-appropriate#example)。 -## Set 索引 +## Set 索引 {#set-index} 当本地(按块)基数较低时使用 `set` 索引;如果每个数据块中包含许多不同的值,则效果不明显。 @@ -81,7 +81,7 @@ SELECT * FROM events WHERE user_id IN (101, 202); 在[基本操作指南](/optimize/skipping-indexes#basic-operation)中展示了创建/物化流程以及应用前后的效果。 -## 通用 Bloom 过滤器(标量) +## 通用 Bloom 过滤器(标量) {#generic-bloom-filter-scalar} `bloom_filter` 索引非常适合用于“在干草堆里找针”式的等值/`IN` 成员匹配查询。它接受一个可选参数,用于指定假阳性(误报)率(默认值为 0.025)。 @@ -96,7 +96,7 @@ SELECT * FROM events WHERE value IN (7, 42, 99); ``` -## 用于子串搜索的 N-gram Bloom 过滤器(ngrambf_v1) +## 用于子串搜索的 N-gram Bloom 过滤器(ngrambf_v1) {#n-gram-bloom-filter-ngrambf-v1-for-substring-search} `ngrambf_v1` 索引将字符串划分为 n-gram。它非常适用于 `LIKE '%...%'` 查询。它支持 String/FixedString/Map(通过 mapKeys/mapValues),并且可以调节大小、哈希次数和种子。有关更多详细信息,请参阅 [N-gram Bloom 过滤器](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter) 的文档。 @@ -133,7 +133,7 @@ SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) AS k; -- 约 13 有关完整的调优指导,请参阅[参数文档](/engines/table-engines/mergetree-family/mergetree#n-gram-bloom-filter)。 -## 用于基于单词搜索的 Token Bloom 过滤器(tokenbf_v1) +## 用于基于单词搜索的 Token Bloom 过滤器(tokenbf_v1) {#token-bloom-filter-tokenbf-v1-for-word-based-search} `tokenbf_v1` 会为由非字母数字字符分隔的 token 建立索引。你应将其与 [`hasToken`](/sql-reference/functions/string-search-functions#hasToken)、`LIKE` 单词模式或 `=` / `IN` 一起使用。它支持 `String` / `FixedString` / `Map` 类型。 @@ -153,7 +153,7 @@ SELECT count() FROM logs WHERE hasToken(lower(msg), 'exception'); 可在[此处](/use-cases/observability/schema-design#bloom-filters-for-text-search)查看可观测性示例,以及关于 token 与 ngram 的使用指导。 -## 在 CREATE TABLE 时添加索引(多个示例) +## 在 CREATE TABLE 时添加索引(多个示例) {#add-indexes-during-create-table-multiple-examples} 跳过索引也支持组合表达式以及 `Map`/`Tuple`/`Nested` 类型。下面的示例对此进行了演示: @@ -175,7 +175,7 @@ ORDER BY u64; ``` -## 在现有数据上物化并验证 +## 在现有数据上物化并验证 {#materializing-on-existing-data-and-verifying} 你可以使用 `MATERIALIZE` 为现有的数据部分添加索引,并通过 `EXPLAIN` 或跟踪日志来检查裁剪效果,如下所示: @@ -211,7 +211,7 @@ SET send_logs_level = 'trace'; -## 临时忽略或强制使用索引 +## 临时忽略或强制使用索引 {#temporarily-ignore-or-force-indexes} 在测试和故障排查期间,可以针对单个查询按名称禁用特定索引。也可以通过相关设置在需要时强制使用索引。参见 [`ignore_data_skipping_indices`](/operations/settings/settings#ignore_data_skipping_indices)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md index 5cb47d94066..f90c3848937 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/skipping-indexes.md @@ -13,7 +13,7 @@ import bad_skip from '@site/static/images/guides/best-practices/bad_skip.png'; import Image from '@theme/IdealImage'; -# 深入了解 ClickHouse 数据跳过索引 +# 深入了解 ClickHouse 数据跳过索引 {#understanding-clickhouse-data-skipping-indexes} @@ -29,7 +29,7 @@ import Image from '@theme/IdealImage'; -## 基本操作 +## 基本操作 {#basic-operation} 用户只能在 MergeTree 系列的表上使用数据跳过索引(Data Skipping Indexes)。每个数据跳过索引有四个主要参数: @@ -121,11 +121,11 @@ SET send_logs_level='trace'; default.skip_table (933d4b2c-8cea-4bf9-8c93-c56e900eefd1) (SelectExecutor): 索引 `vix` 已丢弃 6102/6104 个颗粒。 ``` -## 跳过索引类型 +## 跳过索引类型 {#skip-index-types} {/* vale off */ } -### minmax +### minmax {#minmax} {/* vale on */ } @@ -135,7 +135,7 @@ SET send_logs_level='trace'; {/* vale off */ } -### set +### set {#set} {/* vale on */ } @@ -143,7 +143,7 @@ SET send_logs_level='trace'; 此索引的成本、性能和有效性取决于数据块内部的基数。如果每个数据块包含大量唯一值,要么针对一个很大的索引集合评估查询条件会非常昂贵,要么由于超过 `max_size` 导致索引为空而无法应用索引。 -### Bloom filter 类型 +### Bloom filter 类型 {#bloom-filter-types} *Bloom filter*(布隆过滤器)是一种数据结构,它以极高的空间效率实现集合成员测试,但代价是存在一定概率的误报。对于跳过索引而言,误报并不是一个重要问题,因为唯一的劣势是会多读取一些不必要的数据块。然而,存在误报的可能性也意味着被索引的表达式通常应当为 true,否则可能会跳过有效数据。 @@ -184,7 +184,7 @@ SET send_logs_level='trace'; -## Skip 索引最佳实践 +## Skip 索引最佳实践 {#skip-best-practices} Skip 索引并不直观,尤其是对于那些习惯于关系型数据库(RDBMS)中的行式二级索引,或文档存储中的倒排索引的用户而言。要真正获益,在应用 ClickHouse 数据跳过索引时,必须跳过足够多的 granule 读取,以抵消计算索引本身的开销。关键在于,如果某个值在一个已建立索引的数据块(block)中哪怕只出现一次,就意味着整个 block 都必须被读入内存并进行评估,而索引的计算成本在这种情况下就成了多余的开销。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md index 68abe1691cc..fd832422cb6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/best-practices/sparse-primary-indexes.md @@ -36,7 +36,7 @@ import sparsePrimaryIndexes15b from '@site/static/images/guides/best-practices/s import Image from '@theme/IdealImage'; -# ClickHouse 主索引实用入门指南 +# ClickHouse 主索引实用入门指南 {#a-practical-introduction-to-primary-indexes-in-clickhouse} ## 介绍 {#introduction} @@ -73,7 +73,7 @@ import Image from '@theme/IdealImage'; 本文中给出的所有运行时数据均基于在一台配备 Apple M1 Pro 芯片和 16GB 内存的 MacBook Pro 上本地运行 ClickHouse 22.2.1 所得。 -### 全表扫描 +### 全表扫描 {#a-full-table-scan} 为了了解在没有主键的数据集上查询是如何执行的,我们通过执行以下 SQL DDL 语句来创建一张使用 MergeTree 表引擎的表: @@ -150,7 +150,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 返回了 10 行。耗时:0.022 秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 处理了 887 万行, 70.45 MB (398.53 million rows/s., 3.17 GB/s.) ``` @@ -172,7 +172,7 @@ ClickHouse 客户端的结果输出表明,ClickHouse 执行了一次全表扫 下面将详细说明 ClickHouse 如何构建和使用其稀疏主索引。在本文后续部分,我们还将讨论在构建索引(主键列)时,关于选择、移除和排序相关表列的一些最佳实践。 -### 带主键的表 +### 带主键的表 {#a-table-with-a-primary-key} 创建一个具有复合主键的表,主键列为 UserID 和 URL: @@ -494,7 +494,7 @@ SELECT UserID, URL FROM file('primary-hits_UserID_URL.idx', 'RowBinary', 'UserID 我们将在后面更详细地讨论这对查询执行性能的影响。 -### 主索引用于选择索引颗粒 +### 主索引用于选择索引颗粒 {#the-primary-index-is-used-for-selecting-granules} 现在我们可以在主索引的支持下执行查询。 @@ -526,7 +526,7 @@ LIMIT 10; └────────────────────────────────┴───────┘ 10 rows in set. Elapsed: 0.005 sec. -# highlight-next-line +# highlight-next-line {#highlight-next-line} 已处理 8.19 千行, 740.18 KB (153 万行/秒,138.59 MB/秒) ``` @@ -537,13 +537,13 @@ LIMIT 10; ```response ...Executor): 键条件:(列 0 在 [749927693, 749927693] 范围内) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): 对数据分片 all_1_9_2 的索引范围执行二分查找(1083 个标记) ...Executor): 找到左边界标记:176 ...Executor): 找到右边界标记:177 ...Executor): 经过 19 步找到连续范围 ...Executor): 通过分区键选择了 1/1 个分片,通过主键选择了 1 个分片, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 通过主键选择了 1/1083 个标记,需从 1 个范围读取 1 个标记 ...Reading ...从 1441792 开始读取约 8192 行数据 ``` @@ -592,7 +592,7 @@ LIMIT 10; │ UserID │ │ Condition: (UserID in [749927693, 749927693]) │ │ Parts: 1/1 │ -# highlight-next-line +# highlight-next-line {#highlight-next-line} │ Granules: 1/1083 │ └───────────────────────────────────────────────────────────────────────────────────────┘ @@ -711,7 +711,7 @@ ClickHouse 现在使用索引中选中的标记号(176),在 UserID.mrk 标 -### 次级键列可能(不)高效 +### 次级键列可能(不)高效 {#secondary-key-columns-can-not-be-inefficient} 当查询在一个复合键中、且作为首个键列的列上进行过滤时,[ClickHouse 会在该键列的索引标记上运行二分查找算法](#the-primary-index-is-used-for-selecting-granules)。 @@ -757,7 +757,7 @@ LIMIT 10; └────────────┴───────┘ 10 行结果。耗时:0.086 秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 已处理 881 万行, 799.69 MB(1.0211 亿行/秒,9.27 GB/秒) ``` @@ -769,11 +769,11 @@ LIMIT 10; ```response ...Executor): Key condition: (column 1 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Used generic exclusion search over index for part all_1_9_2 with 1537 steps ...Executor): Selected 1/1 parts by partition key, 1 parts by primary key, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 1076/1083 marks by primary key, 1076 marks to read from 5 ranges ...Executor): Reading approx. 8814592 rows with 10 streams ``` @@ -842,7 +842,7 @@ LIMIT 10; 在我们的示例数据集中,两个键列(UserID、URL)都具有类似的高基数。如前所述,当位于 URL 列之前的键列具有较高或相近的基数时,通用排除搜索算法的效果并不理想。 -### 关于 data skipping index 的说明 +### 关于 data skipping index 的说明 {#note-about-data-skipping-index} 由于 UserID 和 URL 都具有类似的高基数,我们在 URL 上的[查询过滤](/guides/best-practices/sparse-primary-indexes#secondary-key-columns-can-not-be-inefficient),即使在来自我们[复合主键表 (UserID, URL)](#a-table-with-a-primary-key)的 URL 列上创建一个[辅助 data skipping index](./skipping-indexes.md),收益也不会太大。 @@ -906,7 +906,7 @@ ClickHouse 现在创建了一个额外的索引,该索引针对每组 4 个连 -### 选项 1:辅助表 +### 选项 1:辅助表 {#option-1-secondary-tables} @@ -986,7 +986,7 @@ LIMIT 10; └────────────┴───────┘ 10 行数据。耗时: 0.017 秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 处理了 319.49 千行数据, 11.38 MB (18.41 百万行/秒, 655.75 MB/秒) ``` @@ -1002,13 +1002,13 @@ ClickHouse 服务器日志文件中的相应 trace 日志也印证了这一点 ```response ...Executor): 键条件:(列 0 在 ['http://public_search', 'http://public_search'] 中) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): 对数据分片 all_1_9_2 的索引范围执行二分查找(1083 个标记) ...Executor): 找到(左)边界标记:644 ...Executor): 找到(右)边界标记:683 ...Executor): 经过 19 步找到连续范围 ...Executor): 通过分区键选择了 1/1 个分片,通过主键选择了 1 个分片, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 通过主键选择了 39/1083 个标记,需从 1 个范围读取 39 个标记 ...Executor): 使用 2 个流读取约 319488 行数据 ``` @@ -1143,7 +1143,7 @@ LIMIT 10; └────────────┴───────┘ 返回 10 行。耗时:0.026 秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 已处理 33.587 万行, 13.54 MB(1291 万行/秒,520.38 MB/秒)。 ``` @@ -1156,7 +1156,7 @@ ClickHouse 服务器日志文件中的对应跟踪日志确认,ClickHouse 正 ```response ...Executor): Key condition: (column 0 in ['http://public_search', 'http://public_search']) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): Running binary search on index range ... ... ...Executor): Selected 4/4 parts by partition key, 4 parts by primary key, @@ -1233,7 +1233,7 @@ LIMIT 10; └────────────┴───────┘ 返回了 10 行。耗时:0.029 秒。 -# highlight-next-line +# highlight-next-line {#highlight-next-line} 处理了 319.49 千行,1.38 MB(11.05 百万行/秒,393.58 MB/秒) ``` @@ -1245,14 +1245,14 @@ ClickHouse 服务器日志文件中的相应 trace 日志证实 ClickHouse 正 ```response ...Executor): 键条件:(列 0 在 ['http://public_search', 'http://public_search'] 中) -# highlight-next-line +# highlight-next-line {#highlight-next-line} ...Executor): 对数据分片 prj_url_userid 的索引范围执行二分查找(1083 个标记) ...Executor): ... # highlight-next-line ...Executor): 选择完整的普通投影 prj_url_userid ...Executor): 投影所需列:URL、UserID ...Executor): 按分区键选中 1/1 个分片,按主键选中 1 个分片, -# highlight-next-line +# highlight-next-line {#highlight-next-line} 按主键选中 39/1083 个标记,将从 1 个范围读取 39 个标记 ...Executor): 使用 2 个数据流读取约 319488 行 ``` @@ -1444,7 +1444,7 @@ WHERE UserID = 112304 其原因在于,当通过某个次级键列来选择 [granules](#the-primary-index-is-used-for-selecting-granules),且其前一个键列具有更低的基数时,[generic exclusion search algorithm](https://github.com/ClickHouse/ClickHouse/blob/22.3/src/Storages/MergeTree/MergeTreeDataSelectExecutor.cpp#L1444) 的效果最佳。我们已经在本指南的[前一节](#generic-exclusion-search-algorithm)中对这一点进行了详细说明。 -### 数据文件的最佳压缩率 +### 数据文件的最佳压缩率 {#efficient-filtering-on-secondary-key-columns} 此查询比较了我们在上面创建的两个表中 `UserID` 列的压缩率: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md index 9f2f1556c63..a8cc676f9eb 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/alternative-query-languages.md @@ -19,8 +19,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; 具体使用哪种查询语言由 `dialect` 设置项控制。 - -## 标准 SQL +## 标准 SQL {#standard-sql} 标准 SQL 是 ClickHouse 的默认查询语言。 @@ -28,8 +27,7 @@ import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; SET dialect = 'clickhouse' ``` - -## 管道式关系查询语言(PRQL) +## 管道式关系查询语言(PRQL) {#pipelined-relational-query-language-prql} @@ -52,8 +50,7 @@ aggregate { 在底层,ClickHouse 会先将 PRQL 转译为 SQL,然后再执行 PRQL 查询。 - -## Kusto 查询语言 (KQL) +## Kusto 查询语言 (KQL) {#kusto-query-language-kql} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md b/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md index 64ee7988d64..f26b1fb63bd 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/guides/developer/cascading-materialized-views.md @@ -6,13 +6,11 @@ keywords: ['物化视图', '聚合'] doc_type: 'guide' --- - - -# 级联物化视图 +# 级联物化视图 {#cascading-materialized-views} 本示例演示如何创建一个物化视图,然后在其基础上再级联创建第二个物化视图。在本页中,你将看到具体操作步骤、多种可能的使用方式以及相应的限制。针对不同用例,可以通过创建一个以另一个物化视图作为数据源的物化视图来实现。 - + + + + allowfullscreen +/> -
+
ClickHouse 提供了多种处理 JSON 的方法,每种方法都有各自的优缺点和适用场景。本指南将介绍如何加载 JSON,并以最优方式设计 JSON schema。本文包括以下部分: -- [加载 JSON](/integrations/data-formats/json/loading) - 在 ClickHouse 中使用简单 schema 加载和查询结构化与半结构化 JSON。 -- [JSON schema 推断](/integrations/data-formats/json/inference) - 使用 JSON schema 推断来查询 JSON 并创建表的 schema。 -- [设计 JSON schema](/integrations/data-formats/json/schema) - 设计和优化 JSON schema 的步骤。 -- [导出 JSON](/integrations/data-formats/json/exporting) - 如何导出 JSON。 -- [处理其他 JSON 格式](/integrations/data-formats/json/other-formats) - 关于处理非换行分隔(NDJSON)的其他 JSON 格式的一些建议。 -- [建模 JSON 的其他方法](/integrations/data-formats/json/other-approaches) - 旧版 JSON 建模方法。**不推荐使用。** \ No newline at end of file +* [加载 JSON](/integrations/data-formats/json/loading) - 在 ClickHouse 中使用简单 schema 加载和查询结构化与半结构化 JSON。 +* [JSON schema 推断](/integrations/data-formats/json/inference) - 使用 JSON schema 推断来查询 JSON 并创建表的 schema。 +* [设计 JSON schema](/integrations/data-formats/json/schema) - 设计和优化 JSON schema 的步骤。 +* [导出 JSON](/integrations/data-formats/json/exporting) - 如何导出 JSON。 +* [处理其他 JSON 格式](/integrations/data-formats/json/other-formats) - 关于处理非换行分隔(NDJSON)的其他 JSON 格式的一些建议。 +* [建模 JSON 的其他方法](/integrations/data-formats/json/other-approaches) - 旧版 JSON 建模方法。**不推荐使用。** \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md index ca59fc8ce47..e74b561b2e4 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/loading.md @@ -17,7 +17,7 @@ doc_type: 'guide' -## 加载结构化 JSON +## 加载结构化 JSON {#loading-structured-json} 在本节中,我们假定 JSON 数据为 [`NDJSON`](https://github.com/ndjson/ndjson-spec)(以换行分隔的 JSON)格式,在 ClickHouse 中称为 [`JSONEachRow`](/interfaces/formats/JSONEachRow),且结构规范,即列名和类型是固定的。由于 `NDJSON` 简洁且空间利用率高,是加载 JSON 数据的首选格式,但 ClickHouse 也支持其他格式用于[输入和输出](/interfaces/formats/JSON)。 @@ -124,7 +124,7 @@ FORMAT JSONEachRow 这些示例假定使用 `JSONEachRow` 格式。系统同样支持其他常见的 JSON 格式,加载这些格式的示例请参见[此处](/integrations/data-formats/json/other-formats)。 -## 加载半结构化 JSON +## 加载半结构化 JSON {#loading-semi-structured-json} 前面的示例加载的是结构固定、键名和类型都已知的 JSON。现实中往往并非如此——可以新增键,或者键的类型会发生变化。这在可观测性数据等场景中非常常见。 @@ -200,7 +200,7 @@ LIMIT 2 请注意此处在加载数据时的性能差异。`JSON` 列在插入时需要进行类型推断,并且如果某些列中存在多种类型的值,还需要额外的存储空间。尽管可以通过配置 `JSON` 类型(参见 [Designing JSON schema](/integrations/data-formats/json/schema))来获得与显式声明列相当的性能,但它在开箱即用时被刻意设计为更加灵活。不过,这种灵活性也会带来一定的代价。 -### 何时使用 JSON 类型 +### 何时使用 JSON 类型 {#when-to-use-the-json-type} 在以下情况下使用 `JSON` 类型: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md index 95ff9ffb030..a87cad357fb 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/other.md @@ -6,9 +6,7 @@ keywords: ['json', 'formats'] doc_type: 'reference' --- - - -# 对 JSON 建模的其他方法 +# 对 JSON 建模的其他方法 {#other-approaches-to-modeling-json} **以下是在 ClickHouse 中对 JSON 建模的替代方法。这些方法为了文档完整性而被记录下来,主要适用于 JSON 类型尚未出现之前的阶段,因此在大多数用例中通常不推荐使用或不再适用。** @@ -16,9 +14,7 @@ doc_type: 'reference' 在同一个 schema 中,可以对不同对象采用不同的技术。例如,一些对象最适合使用 `String` 类型,另一些则适合使用 `Map` 类型。请注意,一旦使用了 `String` 类型,就不再需要做进一步的 schema 决策。相反,我们也可以在 `Map` 的某个 key 下嵌套子对象——包括用 `String` 表示的 JSON——如下所示: ::: - - -## 使用 String 类型 +## 使用 String 类型 {#using-string} 如果对象高度动态、没有可预测的结构并且包含任意嵌套对象,建议使用 `String` 类型。可以在查询时使用 JSON 函数提取值,如下所示。 @@ -95,7 +91,6 @@ FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/arxiv/arxiv.j 0 rows in set. Elapsed: 25.186 sec. Processed 2.52 million rows, 1.38 GB (99.89 thousand rows/s., 54.79 MB/s.) ``` - 假设我们希望按年份统计论文发表数量。对比如下两条查询语句:一条仅使用字符串,另一条使用该模式的[结构化版本](/integrations/data-formats/json/inference#creating-tables): ```sql @@ -156,7 +151,7 @@ LIMIT 10 这种方法的灵活性带来了显著的性能和语法开销,只应在模式中对象高度动态的情况下使用。 -### 简单 JSON 函数 +### 简单 JSON 函数 {#simple-json-functions} 上面的示例使用了 JSON* 函数族。这些函数使用基于 [simdjson](https://github.com/simdjson/simdjson) 的完整 JSON 解析器,解析严格,并且会区分位于不同嵌套层级的同名字段。这些函数能够处理语法正确但格式不佳的 JSON,例如键之间存在双空格。 @@ -174,7 +169,6 @@ LIMIT 10 而下面的示例将会被正确解析: - ````json {"@timestamp":893964617,"clientip":"40.135.0.0","request":{"method":"GET", "path":"/images/hm_bg.jpg","version":"HTTP/1.0"},"status":200,"size":24736} @@ -209,8 +203,7 @@ LIMIT 10 上述查询使用 `simpleJSONExtractString` 来提取 `created` 键,利用了我们在发布日期上只需要第一个值这一点。在这种情况下,为了获得性能提升,可以接受 `simpleJSON*` 函数带来的局限性。 - -## 使用 Map 类型 +## 使用 Map 类型 {#using-map} 如果对象用于存储任意键,并且这些键大多为同一类型,可以考虑使用 `Map` 类型。理想情况下,唯一键的数量不应超过数百个。对于包含子对象的对象,在这些子对象的类型足够统一的前提下,也可以考虑使用 `Map` 类型。一般来说,我们推荐使用 `Map` 类型来存储标签和标记(labels / tags),例如日志数据中的 Kubernetes pod(容器组)标签。 @@ -224,7 +217,7 @@ LIMIT 10 在将对象建模为 `Map` 时,使用 `String` 键来存储 JSON 键名。因此 map 始终是 `Map(String, T)`,其中 `T` 取决于数据。 ::: -#### 原始类型值 +#### 原始类型值 {#primitive-values} `Map` 最简单的用法是对象将同一种原始类型作为值。在大多数情况下,这意味着对值 `T` 使用 `String` 类型。 @@ -281,13 +274,12 @@ SELECT company.labels['type'] AS type FROM people 完整的 `Map` 函数集可用于对其进行查询,相关说明见[此处](/sql-reference/functions/tuple-map-functions.md)。如果你的数据类型不一致,可以使用相应函数执行[必要的类型强制转换](/sql-reference/functions/type-conversion-functions)。 -#### 对象值 +#### 对象值 {#object-values} 对于具有子对象的对象,只要其子对象的类型保持一致,也可以考虑使用 `Map` 类型。 假设我们的 `persons` 对象中的 `tags` 键需要一个结构一致的子对象,其中每个 `tag` 的子对象都包含 `name` 和 `time` 列。此类 JSON 文档的简化示例如下所示: - ```json { "id": 1, @@ -360,8 +352,7 @@ FORMAT JSONEachRow } ``` - -## 使用 Nested 类型 +## 使用 Nested 类型 {#using-nested} [Nested 类型](/sql-reference/data-types/nested-data-structures/nested) 可用于表示很少发生变化的静态对象,可作为 `Tuple` 和 `Array(Tuple)` 的一种替代方案。我们通常建议避免在处理 JSON 时使用此类型,因为它的行为往往令人困惑。`Nested` 的主要优势在于其子列可以用于排序键。 @@ -396,11 +387,11 @@ CREATE table http ) ENGINE = MergeTree() ORDER BY (status, timestamp); ``` -### flatten_nested +### flatten_nested {#flatten_nested} `flatten_nested` 设置用于控制 `Nested` 类型的行为。 -#### flatten_nested=1 +#### flatten_nested=1 {#flatten_nested1} 当值为 `1`(默认)时,不支持任意深度的嵌套。在该设置下,最简单的理解方式是:将嵌套数据结构视为多个长度相同的 [Array](/sql-reference/data-types/array) 列。字段 `method`、`path` 和 `version` 实际上分别是独立的 `Array(Type)` 列,但有一个关键约束:**`method`、`path` 和 `version` 字段的长度必须相同。** 如果使用 `SHOW CREATE TABLE`,就可以看到这一点: @@ -471,10 +462,9 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth 结果集包含 1 行。执行耗时:0.002 秒。 ``` - 请注意,对子列使用 `Array` 意味着可以充分利用完整的 [Array 函数](/sql-reference/functions/array-functions) 功能集,包括 [`ARRAY JOIN`](/sql-reference/statements/select/array-join) 子句——在列中包含多个值时非常有用。 -#### flatten_nested=0 +#### flatten_nested=0 {#flatten_nested0} 这允许任意级别的嵌套,并意味着嵌套列会保持为一个由 `Tuple` 组成的单个数组——实质上它们与 `Array(Tuple)` 相同。 @@ -546,7 +536,7 @@ SELECT clientip, status, size, `request.method` FROM http WHERE has(request.meth 结果集包含 1 行。执行耗时:0.002 秒。 ``` -### 示例 +### 示例 {#example} 上述数据的更大规模示例可在 S3 的公共存储桶中获取,路径为:`s3://datasets-documentation/http/`。 @@ -584,7 +574,6 @@ size FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/doc 要查询这些数据,我们需要以数组形式访问请求字段。下面,我们将在固定时间段内汇总错误和 HTTP 方法。 - ```sql SELECT status, request.method[1] AS method, count() AS c FROM http @@ -604,7 +593,7 @@ ORDER BY c DESC LIMIT 5; 5 rows in set. Elapsed: 0.007 sec. ``` -### 使用成对数组 +### 使用成对数组 {#using-pairwise-arrays} 成对数组在将 JSON 表示为 `String` 的灵活性与更结构化方案的性能之间提供了一种折中。该模式比较灵活,因为可以在根级别添加任意新的字段。不过,这也需要明显更复杂的查询语法,并且与嵌套结构不兼容。 @@ -668,7 +657,6 @@ GROUP BY method, status ORDER BY c DESC LIMIT 5; └────────┴────────┴───────┘ ``` - 5 行结果。耗时:0.383 秒。已处理 8.22 百万行,1.97 GB(21.45 百万行/秒,5.15 GB/秒)。 峰值内存占用:51.35 MiB。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md index 1e388a17bc4..97a7609eb0e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/json/schema.md @@ -13,11 +13,11 @@ import json_offsets from '@site/static/images/integrations/data-ingestion/data-f import shared_json_column from '@site/static/images/integrations/data-ingestion/data-formats/json_shared_column.png'; -# 设计你的 schema +# 设计你的 schema {#designing-your-schema} 虽然可以使用 [schema 推断](/integrations/data-formats/json/inference) 为 JSON 数据建立初始 schema,并直接查询存储在 S3 等位置的 JSON 数据文件,但用户仍应以为其数据建立经过优化的版本化 schema 为目标。下面我们将讨论对 JSON 结构建模的推荐方法。 -## 静态 JSON 与动态 JSON +## 静态 JSON 与动态 JSON {#static-vs-dynamic-json} 为 JSON 定义模式的首要任务是确定每个键的合适值类型。我们建议用户在 JSON 层级结构中的每一个键上递归应用以下规则,以确定每个键的合适类型。 @@ -92,7 +92,7 @@ import shared_json_column from '@site/static/images/integrations/data-ingestion/ 包含数百或数千个静态键的结构也可以视为动态结构,因为在实际场景中很少会为这些键静态声明所有列。不过,在可能的情况下,应[跳过不需要的路径](#using-type-hints-and-skipping-paths),以同时节省存储和推断开销。 ::: -## 处理静态结构 +## 处理静态结构 {#handling-static-structures} 我们建议使用具名元组(即 `Tuple`)来处理静态结构。对象数组可以使用元组数组(即 `Array(Tuple)`)来存储。在元组内部,列及其对应的类型也应按照相同的规则进行定义。这样就可以通过嵌套的 Tuple 来表示嵌套对象,如下所示。 @@ -203,7 +203,7 @@ ORDER BY company.name ``` -### 处理默认值 +### 处理默认值 {#handling-default-values} 即使 JSON 对象是结构化的,它们通常也比较稀疏,只提供部分已知键。好在,`Tuple` 类型并不要求 JSON 载荷中必须包含所有列;如果某些列未提供,将使用默认值。 @@ -282,7 +282,7 @@ FORMAT PrettyJSONEachRow ::: -### 处理新列 +### 处理新列 {#handling-new-columns} 当 JSON 键是固定的时,使用结构化方法是最简单的。但即使可以事先规划模式变更(即预先知道会新增哪些键,并且可以相应修改模式),这种方法仍然适用。 @@ -360,7 +360,7 @@ SELECT id, nickname FROM people ``` -## 处理半结构化/动态结构 +## 处理半结构化/动态结构 {#handling-semi-structured-dynamic-structures} 如果 JSON 数据是半结构化的,其中的键可以动态增加和/或具有多种类型,则推荐使用 [`JSON`](/sql-reference/data-types/newjson) 类型。 @@ -491,7 +491,7 @@ SELECT id, nickname FROM people - **避免列爆炸风险** - 尽管 JSON 类型可以扩展到潜在的数千个列(其中子列作为独立列存储),但这可能导致列文件数量爆炸,产生过多的列文件,从而影响性能。为缓解这一问题,JSON 所使用的底层 [Dynamic type](/sql-reference/data-types/dynamic) 提供了 [`max_dynamic_paths`](/sql-reference/data-types/newjson#reading-json-paths-as-sub-columns) 参数,用于限制以独立列文件形式存储的唯一路径数量。一旦达到阈值,额外的路径将存储在一个共享的列文件中,并使用紧凑的编码格式,从而在支持灵活数据摄取的同时保持性能和存储效率。但需要注意的是,访问这个共享列文件的性能会略低一些。另请注意,JSON 列可以结合使用[类型提示](#using-type-hints-and-skipping-paths)。带有“提示”的列在性能上将与独立列相当。 - **更简单的路径和类型自省** - 虽然 JSON 类型支持用于确定已推断类型和路径的[自省函数](/sql-reference/data-types/newjson#introspection-functions),但静态结构在探索时(例如使用 `DESCRIBE`)可能更为简单。 -### 单一 JSON 列 +### 单一 JSON 列 {#single-json-column} 此方法适用于原型开发和数据工程任务。对于生产环境,建议仅在确有需要的动态子结构中使用 `JSON`。 @@ -667,7 +667,7 @@ FROM people ``` -### 专用 JSON 列 +### 专用 JSON 列 {#targeted-json-column} 尽管在原型设计和数据工程场景中很有用,但在生产环境中我们建议在可能的情况下使用显式的 schema(模式)定义。 @@ -767,7 +767,7 @@ FORMAT PrettyJsonEachRow ``` -### 使用类型提示和跳过路径 +### 使用类型提示和跳过路径 {#using-type-hints-and-skipping-paths} 类型提示允许我们为某个路径及其子列显式指定类型,从而避免不必要的类型推断。来看下面这个示例,我们为 JSON 列 `company.labels` 中的 JSON 键 `dissolved`、`employees` 和 `founded` 指定了类型。 @@ -934,7 +934,7 @@ FORMAT PrettyJSONEachRow 因此,对于大体结构一致、但仍希望保留 JSON 灵活性的数据集,类型提示提供了一种便利方式,从而在无需重构 schema 或摄取流水线的前提下,保持良好的性能表现。 -### 配置动态路径 +### 配置动态路径 {#configuring-dynamic-paths} ClickHouse 将每个 JSON 路径作为一个子列存储在真正的列式存储布局中,从而能够享受到与传统列相同的性能优势——例如压缩、SIMD 加速处理以及最小化磁盘 I/O。JSON 数据中每个唯一的路径与类型组合都可以在磁盘上对应到自己的列文件。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md index f0a71427883..002182988ca 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/parquet.md @@ -10,7 +10,7 @@ keywords: ['parquet', '列式格式', '数据格式', '压缩', 'Apache Parquet' -# 在 ClickHouse 中使用 Parquet +# 在 ClickHouse 中使用 Parquet {#working-with-parquet-in-clickhouse} Parquet 是一种高效的文件格式,用于以列式方式存储数据。 ClickHouse 支持读取和写入 Parquet 文件。 @@ -24,7 +24,7 @@ ClickHouse 支持读取和写入 Parquet 文件。 -## 从 Parquet 导入 +## 从 Parquet 导入 {#importing-from-parquet} 在加载数据之前,我们可以使用 [file()](/sql-reference/functions/files.md/#file) 函数来查看[示例 Parquet 文件](assets/data.parquet)的结构: @@ -64,7 +64,7 @@ LIMIT 3; ::: -## 导入到现有表 +## 导入到现有表 {#importing-to-an-existing-table} 我们先创建一个用于导入 Parquet 数据的表: @@ -103,7 +103,7 @@ LIMIT 5; 请注意 ClickHouse 如何自动将 Parquet 字符串(`date` 列中的值)转换为 `Date` 类型。这是因为 ClickHouse 会根据目标表中的列类型自动进行类型转换。 -## 将本地文件插入到远程服务器 +## 将本地文件插入到远程服务器 {#inserting-a-local-file-to-remote-server} 如果您想将本地 Parquet 文件插入到远程 ClickHouse 服务器,可以像下面这样通过管道将文件内容传递给 `clickhouse-client`: @@ -112,7 +112,7 @@ clickhouse client -q "INSERT INTO sometable FORMAT Parquet" < data.parquet ``` -## 基于 Parquet 文件创建新表 +## 基于 Parquet 文件创建新表 {#creating-new-tables-from-parquet-files} 由于 ClickHouse 能读取 Parquet 文件的 schema,我们可以动态创建表: @@ -141,7 +141,7 @@ DESCRIBE TABLE imported_from_parquet; 默认情况下,ClickHouse 对列名、类型和值要求非常严格。但在某些情况下,我们可以在导入时跳过不存在的列或不支持的值。可以通过 [Parquet 设置](/interfaces/formats/Parquet#format-settings) 来控制这一行为。 -## 导出为 Parquet 格式 +## 导出为 Parquet 格式 {#exporting-to-parquet-format} :::tip 在 ClickHouse Cloud 中使用 `INTO OUTFILE` 时,需要在将要写入该文件的那台机器上,通过 `clickhouse client` 来运行这些命令。 @@ -159,7 +159,7 @@ FORMAT Parquet 这将在当前工作目录中创建 `export.parquet` 文件。 -## ClickHouse 与 Parquet 数据类型 +## ClickHouse 与 Parquet 数据类型 {#clickhouse-and-parquet-data-types} ClickHouse 与 Parquet 的数据类型在大多数情况下是相同的,但仍然[存在一些差异](/interfaces/formats/Parquet#data-types-matching-parquet)。例如,ClickHouse 会将 `DateTime` 类型导出为 Parquet 的 `int64`。如果我们随后再将该数据导入回 ClickHouse,看到的将是一串数字([time.parquet 文件](assets/time.parquet)): diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md index b4030b84c1c..c841be3c06a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/sql.md @@ -9,13 +9,13 @@ keywords: ['SQL 格式', '数据导出', '数据导入', '备份', 'SQL 转储'] -# 在 ClickHouse 中插入和导出 SQL 数据 +# 在 ClickHouse 中插入和导出 SQL 数据 {#inserting-and-dumping-sql-data-in-clickhouse} ClickHouse 可以通过多种方式轻松集成到 OLTP 数据库基础架构中。其中一种方式是使用 SQL 转储文件在其他数据库与 ClickHouse 之间传输数据。 -## 创建 SQL 转储 +## 创建 SQL 转储 {#creating-sql-dumps} 可以使用 [SQLInsert](/interfaces/formats/SQLInsert) 以 SQL 格式导出数据。ClickHouse 会以 `INSERT INTO VALUES(...` 的形式输出数据,并使用 [`output_format_sql_insert_table_name`](/operations/settings/settings-formats.md/#output_format_sql_insert_table_name) 设置项作为表名: @@ -46,7 +46,7 @@ mysql some_db < dump.sql SET output_format_sql_insert_max_batch_size = 1000; ``` -### 导出一组值 +### 导出一组值 {#exporting-a-set-of-values} ClickHouse 提供了一种 [Values](/interfaces/formats/Values) 格式,类似于 SQL INSERT,但会省略 `INSERT INTO table VALUES` 部分,仅返回一组值: @@ -59,7 +59,7 @@ SELECT * FROM some_data LIMIT 3 FORMAT Values ``` -## 从 SQL 转储中插入数据 +## 从 SQL 转储中插入数据 {#inserting-data-from-sql-dumps} 要读取 SQL 转储文件,使用 [MySQLDump](/interfaces/formats/MySQLDump): diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md index 6bb64f7a4e3..7be91425d62 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-formats/templates-regex.md @@ -10,13 +10,13 @@ keywords: ['数据格式', '模板', '正则表达式', '自定义格式', '解 -# 在 ClickHouse 中使用 Templates 和 Regex 导入与导出自定义文本数据 +# 在 ClickHouse 中使用 Templates 和 Regex 导入与导出自定义文本数据 {#importing-and-exporting-custom-text-data-using-templates-and-regex-in-clickhouse} 我们经常需要处理自定义文本格式的数据,这些数据可能是非标准格式、无效的 JSON,或损坏的 CSV。在这些情况下,使用 CSV 或 JSON 等标准解析器并不总是可行。好在 ClickHouse 提供了功能强大的 Template 和 Regex 格式,可以很好地应对这些场景。 -## 基于模板导入 +## 基于模板导入 {#importing-based-on-a-template} 假设我们要从以下[日志文件](assets/error.log)中导入数据: @@ -88,7 +88,7 @@ GROUP BY request └──────────────────────────────────────────────────┴─────────┘ ``` -### 跳过空白字符 +### 跳过空白字符 {#skipping-whitespaces} 建议使用 [TemplateIgnoreSpaces](/interfaces/formats/TemplateIgnoreSpaces),它可以忽略模板中分隔符之间的空白字符: @@ -98,7 +98,7 @@ TemplateIgnoreSpaces --> "p1:${p1:CSV}, p2:${p2:CSV}" ``` -## 使用模板导出数据 +## 使用模板导出数据 {#exporting-data-using-templates} 我们也可以使用模板将数据导出为任何文本格式。在这种情况下,我们需要创建两个文件: @@ -142,7 +142,7 @@ FORMAT Template SETTINGS format_template_resultset = 'output.results', --- 已读取 1000 行,用时 0.001380604 秒 --- ``` -### 导出为 HTML 文件 +### 导出为 HTML 文件 {#exporting-to-html-files} 基于模板的结果也可以使用 [`INTO OUTFILE`](/sql-reference/statements/select/into-outfile.md) 子句导出到文件。我们来基于给定的 [结果集](assets/html.results) 和 [行](assets/html.row) 格式生成 HTML 文件: @@ -157,7 +157,7 @@ SETTINGS format_template_resultset = 'html.results', format_template_row = 'html.row' ``` -### 导出为 XML +### 导出为 XML {#exporting-to-xml} 模板格式可用于生成几乎所有可以想象的文本格式文件,包括 XML。只需准备相应的模板并执行导出即可。 @@ -203,7 +203,7 @@ FORMAT XML ``` -## 基于正则表达式导入数据 +## 基于正则表达式导入数据 {#importing-data-based-on-regular-expressions} [Regexp](/interfaces/formats/Regexp) 格式适用于需要以更复杂方式解析输入数据的场景。我们再次解析 [error.log](assets/error.log) 示例文件,不过这次要提取文件名和协议,并将它们保存到单独的列中。首先,为此准备一张新表: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md index 7f4be496488..6d9f3a41102 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-ingestion-index.md @@ -6,7 +6,7 @@ description: '数据摄取部分的概览页' doc_type: 'landing-page' --- -# 数据摄取 +# 数据摄取 {#data-ingestion} ClickHouse 集成了多种用于数据集成和转换的解决方案。 如需更多信息,请参阅以下页面: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md index 584f01f48e8..2e87aaa4323 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/data-sources-index.md @@ -6,7 +6,7 @@ title: '数据源' doc_type: 'landing-page' --- -# 数据源 +# 数据源 {#data-sources} ClickHouse 允许你轻松地从多种数据源将数据摄取到数据库中。 有关更多信息,请参阅下列页面: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md index 72a9df5f46f..f3f334d2f6d 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/dynamodb/index.md @@ -17,7 +17,7 @@ import dynamodb_map_columns from '@site/static/images/integrations/data-ingestio import Image from '@theme/IdealImage'; -# 从 DynamoDB 到 ClickHouse 的 CDC +# 从 DynamoDB 到 ClickHouse 的 CDC {#cdc-from-dynamodb-to-clickhouse} @@ -50,9 +50,9 @@ import Image from '@theme/IdealImage'; -## 3. 将快照加载到 ClickHouse 中 +## 3. 将快照加载到 ClickHouse 中 {#3-load-the-snapshot-into-clickhouse} -### 创建必要的表 +### 创建必要的表 {#create-necessary-tables} 来自 DynamoDB 的快照数据大致如下所示: @@ -115,7 +115,7 @@ ORDER BY id; * 表应使用分区键作为排序键(通过 `ORDER BY` 指定) * 具有相同排序键的行会基于 `version` 列进行去重。 -### 创建快照 ClickPipe +### 创建快照 ClickPipe {#create-the-snapshot-clickpipe} 现在可以创建一个 ClickPipe,将来自 S3 的快照数据加载到 ClickHouse 中。请按照 S3 ClickPipe 指南[此处](/integrations/clickpipes/object-storage)的说明进行操作,但使用以下设置: @@ -146,7 +146,7 @@ https://{bucket}.s3.amazonaws.com/{prefix}/AWSDynamoDB/{export-id}/data/* -## 5. 清理(可选) +## 5. 清理(可选) {#5-cleanup-optional} 当快照 ClickPipe 运行完成后,可以删除快照表和物化视图。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md index 78a19ef372b..c55d0e6bc8b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/jdbc-with-clickhouse.md @@ -16,7 +16,7 @@ import Jdbc02 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-02 import Jdbc03 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-03.png'; -# 使用 JDBC 将 ClickHouse 连接到外部数据源 +# 使用 JDBC 将 ClickHouse 连接到外部数据源 {#connecting-clickhouse-to-external-data-sources-with-jdbc} :::note 使用 JDBC 需要 ClickHouse JDBC Bridge,因此您需要在本地机器上使用 `clickhouse-local`,将数据库中的数据以流式方式传输到 ClickHouse Cloud。请访问文档 **Migrate** 部分中的 [**Using clickhouse-local**](/cloud/migration/clickhouse-local#example-2-migrating-from-mysql-to-clickhouse-cloud-with-the-jdbc-bridge) 页面了解详细信息。 @@ -44,7 +44,7 @@ import Jdbc03 from '@site/static/images/integrations/data-ingestion/dbms/jdbc-03 -## 在本地安装 ClickHouse JDBC Bridge +## 在本地安装 ClickHouse JDBC Bridge {#install-the-clickhouse-jdbc-bridge-locally} 使用 ClickHouse JDBC Bridge 最简单的方式,是将其安装并运行在与 ClickHouse 相同的主机上: @@ -107,7 +107,7 @@ java -jar clickhouse-jdbc-bridge-2.0.7-shaded.jar ::: -## 在 ClickHouse 中使用 JDBC 连接 +## 在 ClickHouse 中使用 JDBC 连接 {#use-the-jdbc-connection-from-within-clickhouse} ClickHouse 现在可以通过使用 [jdbc 表函数](/sql-reference/table-functions/jdbc.md) 或 [JDBC 表引擎](/engines/table-engines/integrations/jdbc.md) 来访问 MySQL 数据。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md index a26143f7d74..f4308b7d8e5 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/dbms/postgresql/connecting-to-postgresql.md @@ -10,27 +10,24 @@ doc_type: 'guide' import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - -# 将 ClickHouse 连接到 PostgreSQL +# 将 ClickHouse 连接到 PostgreSQL {#connecting-clickhouse-to-postgresql} 本页介绍以下几种将 PostgreSQL 与 ClickHouse 集成的方式: -- 使用 `PostgreSQL` 表引擎,从 PostgreSQL 表中读取数据 -- 使用试验性的 `MaterializedPostgreSQL` 数据库引擎,将 PostgreSQL 中的数据库与 ClickHouse 中的数据库进行同步 +* 使用 `PostgreSQL` 表引擎,从 PostgreSQL 表中读取数据 +* 使用试验性的 `MaterializedPostgreSQL` 数据库引擎,将 PostgreSQL 中的数据库与 ClickHouse 中的数据库进行同步 :::tip 我们推荐使用由 PeerDB 提供支持的 [ClickPipes](/integrations/clickpipes/postgres),这是一项 ClickHouse Cloud 的托管集成服务。 或者,你也可以使用 [PeerDB](https://github.com/PeerDB-io/peerdb),它是一个专门为将 PostgreSQL 数据库复制到自托管 ClickHouse 和 ClickHouse Cloud 而设计的开源 CDC 工具。 ::: - - -## 使用 PostgreSQL 表引擎 +## 使用 PostgreSQL 表引擎 {#using-the-postgresql-table-engine} `PostgreSQL` 表引擎允许 ClickHouse 对存储在远程 PostgreSQL 服务器上的数据执行 **SELECT** 和 **INSERT** 操作。 本文将通过单个表来演示基本的集成方法。 -### 1. 设置 PostgreSQL +### 1. 设置 PostgreSQL {#1-setting-up-postgresql} 1. 在 `postgresql.conf` 中添加以下条目,以便让 PostgreSQL 监听网络接口: @@ -93,7 +90,7 @@ psql -U clickhouse_user -W -d db_in_psg -h <你的_postgresql_主机地址> 有关出站流量的详细信息,请查看 ClickHouse 的 [Cloud Endpoints API](/cloud/get-started/query-endpoints)。 ::: -### 2. 在 ClickHouse 中定义一张表 +### 2. 在 ClickHouse 中定义一张表 {#2-define-a-table-in-clickhouse} 1. 登录 `clickhouse-client`: @@ -131,7 +128,7 @@ ENGINE = PostgreSQL('postgres-host.domain.com:5432', 'db_in_psg', 'table1', 'cli 查看 [PostgreSQL 表引擎](/engines/table-engines/integrations/postgresql) 文档页面以获取完整的参数列表。 ::: -### 3 测试集成 +### 3 测试集成 {#3-test-the-integration} 1. 在 ClickHouse 中查看初始数据行: @@ -166,7 +163,6 @@ VALUES SELECT * FROM db_in_ch.table1 ``` - 响应应如下: ```response @@ -209,8 +205,7 @@ id | column1 请参阅 [PostgreSQL 表引擎的文档页面](/engines/table-engines/integrations/postgresql),了解更多功能,例如指定 schema、仅返回部分列以及连接到多个副本。另请参阅博客文章:[ClickHouse and PostgreSQL - a match made in data heaven - part 1](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres)。 - -## 使用 MaterializedPostgreSQL 数据库引擎 +## 使用 MaterializedPostgreSQL 数据库引擎 {#using-the-materializedpostgresql-database-engine} @@ -221,7 +216,7 @@ PostgreSQL 数据库引擎利用 PostgreSQL 的复制功能来创建该数据库 ***在以下步骤中,将使用 PostgreSQL 命令行客户端 (`psql`) 和 ClickHouse 命令行客户端 (`clickhouse-client`)。PostgreSQL 服务器安装在 Linux 上。下面给出的配置是针对全新测试安装的 PostgreSQL 数据库的最小配置。*** -### 1. 在 PostgreSQL 中 +### 1. 在 PostgreSQL 中 {#1-in-postgresql} 1. 在 `postgresql.conf` 中,设置基本的监听参数、WAL 复制级别以及复制槽: @@ -276,7 +271,6 @@ VALUES 7. 配置 PostgreSQL,允许使用新用户连接到新数据库以进行复制。下面是在 `pg_hba.conf` 文件中需要添加的最小必要条目: - ```text # TYPE DATABASE USER ADDRESS METHOD host db1 clickhouse_user 192.168.1.0/24 password @@ -350,7 +344,7 @@ Query id: df2381ac-4e30-4535-b22e-8be3894aaafc └────┴─────────┘ ``` -### 3. 测试基本复制 +### 3. 测试基本复制 {#2-in-clickhouse} 1. 在 PostgreSQL 中添加新行: @@ -386,7 +380,7 @@ Query id: b0729816-3917-44d3-8d1a-fed912fb59ce └────┴─────────┘ ``` -### 4. 总结 +### 4. 总结 {#3-test-basic-replication} 本集成指南重点通过一个简单示例说明如何复制一个包含单个表的数据库,不过也有更高级的选项,例如复制整个数据库,或在现有复制基础上新增表和模式(schema)。虽然此复制方式不支持 DDL 命令,但可以将引擎配置为在发生结构性变更时检测更改并重新加载表。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md index 6f13040568f..0e022f79bf0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/emqx/index.md @@ -40,7 +40,7 @@ import clickhouse_result from '@site/static/images/integrations/data-ingestion/e import Image from '@theme/IdealImage'; -# 将 EMQX 与 ClickHouse 集成 +# 将 EMQX 与 ClickHouse 集成 {#integrating-emqx-with-clickhouse} @@ -63,7 +63,7 @@ import Image from '@theme/IdealImage'; -## 获取 ClickHouse Cloud 服务 +## 获取 ClickHouse Cloud 服务 {#get-your-clickhouse-cloudservice} 在本次部署过程中,我们在 AWS 美国北弗吉尼亚(us-east-1)区域部署了一个 ClickHouse 实例,并在同一地区部署了一个 EMQX Cloud 实例。 @@ -153,7 +153,7 @@ EMQX Cloud 默认不允许匿名连接,所以你需要添加一个客户端凭 -## 将 EMQX Cloud 与 ClickHouse Cloud 集成 +## 将 EMQX Cloud 与 ClickHouse Cloud 集成 {#integration-emqx-cloud-with-clickhouse-cloud} [EMQX Cloud Data Integrations](https://docs.emqx.com/en/cloud/latest/rule_engine/introduction.html#general-flow) 用于配置处理和响应 EMQX 消息流与设备事件的规则。Data Integrations 不仅提供了清晰且灵活的可配置架构方案,还简化了开发流程、提升了用户体验,并降低了业务系统与 EMQX Cloud 之间的耦合度。同时,它还为 EMQX Cloud 专有能力的定制化提供了完善的基础设施。 @@ -163,7 +163,7 @@ EMQX Cloud 为常见数据系统提供了 30 多种原生集成方案,ClickHou -### 创建 ClickHouse 资源 +### 创建 ClickHouse 资源 {#create-clickhouse-resource} 点击左侧菜单中的 “Data Integrations”,然后点击 “View All Resources”。您可以在 Data Persistence 部分找到 ClickHouse,或者直接搜索 ClickHouse。 @@ -177,7 +177,7 @@ EMQX Cloud 为常见数据系统提供了 30 多种原生集成方案,ClickHou -### 创建新规则 +### 创建新规则 {#create-a-new-rule} 在创建资源的过程中,您会看到一个弹窗,点击 “New” 会跳转到规则创建页面。 @@ -212,7 +212,7 @@ FROM 现在点击 "NEXT" 按钮。此步骤是告诉 EMQX Cloud 如何将处理后的数据写入你的 ClickHouse 数据库。 -### 添加响应操作 +### 添加响应操作 {#add-a-response-action} 如果你只有一个资源,则无需修改 'Resource' 和 'Action Type'。 你只需要设置 SQL 模板。以下是本教程中使用的示例: @@ -225,7 +225,7 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ 这是一个用于向 ClickHouse 写入数据的模板,可以看到这里使用了变量。 -### 查看规则详情 +### 查看规则详情 {#view-rules-details} 点击 “Confirm” 和 “View Details”。现在一切都已配置就绪。你可以在规则详情页面看到数据集成的运行情况。 @@ -234,13 +234,13 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ 发送到 `temp_hum/emqx` 主题的所有 MQTT 消息都会被持久化到你的 ClickHouse Cloud 数据库中。 -## 将数据保存到 ClickHouse +## 将数据保存到 ClickHouse {#saving-data-into-clickhouse} 我们将模拟温度和湿度数据,通过 MQTT X 将这些数据上报到 EMQX Cloud,然后使用 EMQX Cloud 的数据集成功能将数据保存到 ClickHouse Cloud 中。 -### 向 EMQX Cloud 发布 MQTT 消息 +### 向 EMQX Cloud 发布 MQTT 消息 {#publish-mqtt-messages-to-emqx-cloud} 你可以使用任意 MQTT 客户端或 SDK 发布消息。在本教程中,我们将使用 [MQTT X](https://mqttx.app/),这是由 EMQ 提供的一款用户友好的 MQTT 客户端应用程序。 @@ -274,13 +274,13 @@ INSERT INTO temp_hum (client_id, timestamp, topic, temp, hum) VALUES ('${client_ -### 查看规则监控 +### 查看规则监控 {#view-rules-monitoring} 检查规则监控,确认成功次数已增加 1。 -### 检查持久化数据 +### 检查持久化数据 {#check-the-data-persisted} 现在是时候查看 ClickHouse Cloud 上的数据了。理想情况下,你使用 MQTTX 发送的数据会进入 EMQX Cloud,并在原生数据集成的帮助下持久化到 ClickHouse Cloud 的数据库中。 @@ -293,6 +293,6 @@ SELECT * FROM emqx.temp_hum; -### 总结 +### 总结 {#summary} 你无需编写任何代码,就已经让 MQTT 数据从 EMQX Cloud 流转到了 ClickHouse Cloud。借助 EMQX Cloud 和 ClickHouse Cloud,你无需自行管理基础设施,只需专注于编写物联网 (IoT) 应用,而数据会安全地存储在 ClickHouse Cloud 中。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md index 9c9b98a987d..78681183269 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/airbyte-and-clickhouse.md @@ -25,7 +25,7 @@ import airbyte09 from '@site/static/images/integrations/data-ingestion/etl-tools import PartnerBadge from '@theme/badges/PartnerBadge'; -# 将 Airbyte 连接到 ClickHouse +# 将 Airbyte 连接到 ClickHouse {#connect-airbyte-to-clickhouse} @@ -70,7 +70,7 @@ import PartnerBadge from '@theme/badges/PartnerBadge'; -## 将 ClickHouse 添加为目标 +## 将 ClickHouse 添加为目标 {#2-add-clickhouse-as-a-destination} 在本节中,我们将展示如何将一个 ClickHouse 实例添加为目标。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md index 441455abc73..b56dbf2cfe0 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/apache-beam.md @@ -13,7 +13,7 @@ keywords: ['apache beam', 'stream processing', 'batch processing', 'jdbc connect import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# 集成 Apache Beam 与 ClickHouse +# 集成 Apache Beam 与 ClickHouse {#integrating-apache-beam-and-clickhouse} @@ -29,9 +29,9 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## 设置 Apache Beam ClickHouse 包 +## 设置 Apache Beam ClickHouse 包 {#setup-of-the-apache-beam-clickhouse-package} -### 安装包 +### 安装包 {#package-installation} 将以下依赖添加到你的包管理工具中: @@ -50,7 +50,7 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; 相关构件可以在[官方 Maven 仓库](https://mvnrepository.com/artifact/org.apache.beam/beam-sdks-java-io-clickhouse)中找到。 -### 代码示例 +### 代码示例 {#code-example} 以下示例将名为 `input.csv` 的 CSV 文件读取为 `PCollection`,将其转换为 Row 对象(基于已定义的 schema),并使用 `ClickHouseIO` 将其插入到本地 ClickHouse 实例中: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md index b712370db52..34a438409e6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/bladepipe-and-clickhouse.md @@ -21,7 +21,7 @@ import bp_ck_9 from '@site/static/images/integrations/data-ingestion/etl-tools/b import PartnerBadge from '@theme/badges/PartnerBadge'; -# 将 BladePipe 连接到 ClickHouse +# 将 BladePipe 连接到 ClickHouse {#connect-bladepipe-to-clickhouse} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md index f281fbe18cd..314e8685fad 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/features-and-configurations.md @@ -12,7 +12,7 @@ import TOCInline from '@theme/TOCInline'; import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# 功能和配置 +# 功能和配置 {#features-and-configurations} @@ -22,7 +22,7 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## Profile.yml 配置 +## Profile.yml 配置 {#profile-yml-configurations} 要使用 dbt 连接到 ClickHouse,需在 `profiles.yml` 文件中添加一个 [profile](https://docs.getdbt.com/docs/core/connect-data-platform/connection-profiles)。ClickHouse 的 profile 遵循以下语法: @@ -65,16 +65,16 @@ your_profile_name: compress_block_size: [1048576] # 启用压缩时的压缩块大小 ``` -### 模式与数据库 +### 模式与数据库 {#schema-vs-database} dbt 模型关系标识符 `database.schema.table` 与 ClickHouse 不兼容,因为 ClickHouse 不支持 `schema`。 因此我们采用简化形式 `schema.table`,其中 `schema` 实际上就是 ClickHouse 的 database。不推荐使用 `default` 数据库。 -### SET 语句警告 +### SET 语句警告 {#set-statement-warning} 在许多环境中,使用 SET 语句在所有 dbt 查询之间持久化 ClickHouse 设置并不可靠,并可能导致意外失败。对于通过负载均衡器使用 HTTP 连接并将查询分发到多个节点的场景(例如 ClickHouse Cloud),这一点尤为明显;在某些情况下,这种问题在原生 ClickHouse 连接中也可能出现。因此,我们建议将所有必需的 ClickHouse 设置配置在 dbt profile 的 "custom_settings" 属性中,作为最佳实践,而不是依赖在 pre-hook 中执行 "SET" 语句(这一做法曾被偶尔建议过)。 -### 设置 `quote_columns` +### 设置 `quote_columns` {#setting-quote_columns} 为避免出现警告,请务必在 `dbt_project.yml` 中明确设置 `quote_columns` 的值。更多信息请参阅[有关 `quote_columns` 的文档](https://docs.getdbt.com/reference/resource-configs/quote_columns)。 @@ -84,14 +84,14 @@ seeds: +quote_columns: false #若 CSV 列标题含有空格,则设为 `true` ``` -### 关于 ClickHouse 集群 +### 关于 ClickHouse 集群 {#about-the-clickhouse-cluster} 在使用 ClickHouse 集群时,需要考虑两点: * 设置 `cluster` 参数。 * 确保写入后的读取一致性,尤其是在使用多个 `threads` 时。 -#### 集群设置 +#### 集群设置 {#cluster-setting} 配置文件中的 `cluster` 参数允许 dbt-clickhouse 在 ClickHouse 集群上运行。若在配置文件中设置了 `cluster`,则**所有模型默认都会使用 `ON CLUSTER` 子句进行创建**——使用 **Replicated** 引擎的模型除外。这包括: @@ -120,7 +120,7 @@ Replicated 引擎**不会**包含 `ON CLUSTER` 子句,因为它们被设计为 如果某个模型是在没有 `cluster` 设置的情况下创建的,dbt-clickhouse 会识别这一情况,并在对该模型执行所有 DDL/DML 时不使用 `on cluster` 子句。 -#### 写后读一致性 +#### 写后读一致性 {#read-after-write-consistency} dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无法保证所有操作都发送到同一个副本,那么对于具有多个副本的 ClickHouse 集群,这种一致性模型就不兼容。在你日常使用 dbt 的过程中,可能不会遇到问题,但可以根据集群情况采用一些策略来保证这一点: @@ -128,9 +128,9 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 * 如果你使用的是自托管集群,请确保所有 dbt 请求都发送到同一个 ClickHouse 副本。如果其前面有负载均衡器,尝试使用 `replica aware routing` / `sticky sessions` 等机制,以始终访问同一副本。在 ClickHouse Cloud 之外的集群中添加设置 `select_sequential_consistency = 1`[是不推荐的](https://clickhouse.com/docs/operations/settings/settings#select_sequential_consistency)。 -## 功能概览 +## 功能概览 {#general-information-about-features} -### 通用表配置 +### 通用表配置 {#general-table-configurations} | Option | Description | Default if any | | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------- | -------------- | @@ -148,7 +148,7 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 | definer | 如果 `sql_security` 设置为 `definer`,则必须在 `definer` 子句中指定某个已存在的用户或 `CURRENT_USER`。 | | | projections | 要创建的[投影(projections)列表](/data-modeling/projections)。详细信息参见[关于投影](#projections)。 | | -#### 关于数据跳过索引 +#### 关于数据跳过索引 {#data-skipping-indexes} 数据跳过索引仅适用于 `table` 物化方式。要为表添加数据跳过索引列表,请使用 `indexes` 配置: @@ -162,7 +162,7 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 ) }} ``` -#### 关于投影 +#### 关于投影 {#projections} 你可以通过 `projections` 配置为 `table` 和 `distributed_table` 物化方式添加[投影](/data-modeling/projections): @@ -180,7 +180,7 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 **注意**:对于分布式表,投影会应用到 `_local` 表,而不是分布式代理表。 -### 支持的表引擎 +### 支持的表引擎 {#supported-table-engines} | 类型 | 详情 | | ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -191,7 +191,7 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 | EmbeddedRocksDB | [https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb](https://clickhouse.com/docs/en/engines/table-engines/integrations/embedded-rocksdb) | | Hive | [https://clickhouse.com/docs/en/engines/table-engines/integrations/hive](https://clickhouse.com/docs/en/engines/table-engines/integrations/hive) | -### 实验性支持的表引擎 +### 实验性支持的表引擎 {#experimental-supported-table-engines} | 类型 | 详情 | @@ -201,13 +201,13 @@ dbt 依赖写入后读取(read-after-insert)的一致性模型。如果无 如果你在使用上述任一引擎时,从 dbt 连接 ClickHouse 遇到问题,请在[这里](https://github.com/ClickHouse/dbt-clickhouse/issues)提交 issue。 -### 关于模型设置的说明 +### 关于模型设置的说明 {#a-note-on-model-settings} ClickHouse 有多种类型/级别的“设置(settings)”。在上面的模型配置中,其中两类是可配置的。`settings` 指的是在 `CREATE TABLE/VIEW` 这类 DDL 语句中使用的 `SETTINGS` 子句,因此通常是特定于某个 ClickHouse 表引擎的设置。新的 `query_settings` 用于在用于模型物化的 `INSERT` 和 `DELETE` 查询中添加 `SETTINGS` 子句(包括增量物化)。 ClickHouse 中有数百个设置,而且并不总是很清楚哪些是“表”级设置,哪些是“用户”级设置(尽管后者通常可以在 `system.settings` 表中查看)。通常推荐使用默认值,若要使用这些属性,应进行充分的调研和测试。 -### 列配置 +### 列配置 {#column-configuration} > ***注意:*** 下列列配置选项要求启用并强制执行[模型契约(model contracts)](https://docs.getdbt.com/docs/collaborate/govern/model-contracts)。 @@ -216,7 +216,7 @@ ClickHouse 中有数百个设置,而且并不总是很清楚哪些是“表” | codec | 一个字符串,由传递给列 DDL 中 `CODEC()` 的参数组成。例如:`codec: "Delta, ZSTD"` 会被编译为 `CODEC(Delta, ZSTD)`。 | | | ttl | 一个字符串,由[TTL(time-to-live)表达式](https://clickhouse.com/docs/guides/developer/ttl)组成,用于在列的 DDL 中定义 TTL 规则。例如:`ttl: ts + INTERVAL 1 DAY` 会被编译为 `TTL ts + INTERVAL 1 DAY`。 | | -#### Schema 配置示例 +#### Schema 配置示例 {#example-of-schema-configuration} ```yaml models: @@ -234,7 +234,7 @@ models: ttl: ts + INTERVAL 1 DAY ``` -#### 添加复杂类型 +#### 添加复杂类型 {#adding-complex-types} dbt 会通过分析用于创建模型的 SQL,自动推断每一列的数据类型。然而,在某些情况下,此过程可能无法准确确定数据类型,进而与契约中 `data_type` 属性指定的类型产生冲突。为了解决这一问题,我们建议在模型 SQL 中使用 `CAST()` 函数显式指定所需类型。例如: @@ -259,9 +259,9 @@ group by event_type ``` -## 功能 +## 功能 {#features} -### 物化类型:view +### 物化类型:view {#materialization-view} 可以将 dbt 模型创建为 [ClickHouse view](https://clickhouse.com/docs/en/sql-reference/table-functions/view/), 并使用以下语法进行配置: @@ -280,7 +280,7 @@ models: {{ config(materialized = "view") }} ``` -### 物化:table +### 物化:table {#materialization-table} 可以将 dbt 模型物化为 [ClickHouse 表](https://clickhouse.com/docs/en/operations/system-tables/tables/),并使用以下语法进行配置: @@ -308,7 +308,7 @@ models: ) }} ``` -### 物化方式:增量(incremental) +### 物化方式:增量(incremental) {#materialization-incremental} 每次执行 dbt 时,表模型都会被重新构建。对于较大的结果集或复杂的转换,这可能不可行且代价极高。为了解决这一问题并减少构建时间,可以将 dbt 模型创建为 ClickHouse 增量表,并使用以下语法进行配置: @@ -340,7 +340,7 @@ models: ) }} ``` -#### 配置 +#### 配置 {#configurations} 针对此物化类型的特定配置如下所示: @@ -351,11 +351,11 @@ models: | `incremental_strategy` | 用于增量物化的策略。支持 `delete+insert`、`append`、`insert_overwrite` 或 `microbatch`。有关各策略的更多详细信息,请参见[此处](/integrations/dbt/features-and-configurations#incremental-model-strategies)。 | 可选(默认:'default') | | `incremental_predicates` | 需要应用于增量物化的附加条件(仅适用于 `delete+insert` 策略) | 可选 | -#### 增量模型策略 +#### 增量模型策略 {#incremental-model-strategies} `dbt-clickhouse` 支持三种增量模型策略。 -##### 默认(传统)策略 +##### 默认(传统)策略 {#default-legacy-strategy} 一直以来,ClickHouse 仅通过异步的 “mutations” 形式有限支持更新和删除操作。 为了模拟预期的 dbt 行为, @@ -363,7 +363,7 @@ dbt-clickhouse 默认会创建一个新的临时表,该表包含所有未受 记录,以及所有新增或更新的记录, 然后将此临时表与现有的增量模型关系进行交换。这是唯一一种在操作完成之前如果出现问题仍能保留原始关系的策略;但是,由于它需要对原始表进行完整拷贝,因此执行代价较高且速度较慢。 -##### Delete+Insert 策略 +##### Delete+Insert 策略 {#delete-insert-strategy} ClickHouse 在 22.8 版本中新增了实验性功能 “lightweight deletes(轻量级删除)”。与 `ALTER TABLE ... DELETE` @@ -437,7 +437,7 @@ ClickHouse 在 22.8 版本中新增了实验性功能 “lightweight deletes( 该策略要求在模型配置中设置 `partition_by`,并会忽略模型配置中所有其他特定于策略的参数。 -### 物化:materialized_view(实验性) +### 物化:materialized_view(实验性) {#materialized-view} `materialized_view` 物化应为对现有(源)表的 `SELECT`。适配器会创建一个名称为模型名的目标表, 以及一个名为 `_mv` 的 ClickHouse MATERIALIZED VIEW。与 PostgreSQL 不同,ClickHouse 的物化视图不是“静态的”(且 @@ -466,7 +466,7 @@ select a,b,c from {{ source('raw', 'table_2') }} > 将会看到如下警告: > `Warning - Table was detected with the same pattern as model name but was not found in this run. In case it is a renamed mv that was previously part of this model, drop it manually (!!!) ` -#### 数据补齐 +#### 数据补齐 {#data-catch-up} 目前,在创建物化视图(MV)时,目标表会先使用历史数据进行填充,然后才创建 MV 本身。 @@ -483,7 +483,7 @@ select a,b,c from {{ source('raw', 'table_2') }} )}} ``` -#### 可刷新物化视图 +#### 可刷新物化视图 {#refreshable-materialized-views} 要使用 [Refreshable Materialized View](https://clickhouse.com/docs/en/materialized-view/refreshable-materialized-view), 请在 MV 模型中按需调整以下配置(所有这些配置都应在一个 refreshable 配置对象中进行设置): @@ -514,7 +514,7 @@ select a,b,c from {{ source('raw', 'table_2') }} }} ``` -#### 限制 +#### 限制 {#limitations} * 在 ClickHouse 中创建具有依赖项的可刷新的物化视图(MV)时,如果在创建时指定的依赖项不存在,ClickHouse 不会抛出错误。 相反,该可刷新 MV 会保持在未激活状态,等待依赖项被满足之后,才会开始处理更新或刷新。 @@ -523,7 +523,7 @@ select a,b,c from {{ source('raw', 'table_2') }} * 截至目前,MV 与其依赖项之间并没有真正的 “dbt 链接(dbt linkage)”,因此无法保证创建顺序。 * 尚未对多个 MV 指向同一目标模型场景下的可刷新特性进行测试。 -### 物化:dictionary(实验性) +### 物化:dictionary(实验性) {#materialization-dictionary} 请参阅 [https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py](https://github.com/ClickHouse/dbt-clickhouse/blob/main/tests/integration/adapter/dictionary/test_dictionary.py) @@ -531,7 +531,7 @@ select a,b,c from {{ source('raw', 'table_2') }} 以获取如何为 ClickHouse dictionaries 实现物化的示例。 -### 物化:distributed_table(实验性) +### 物化:distributed_table(实验性) {#materialization-distributed-table} 通过以下步骤创建 Distributed 表: @@ -545,7 +545,7 @@ select a,b,c from {{ source('raw', 'table_2') }} * 为了确保下游增量物化操作能够正确执行,dbt-clickhouse 查询现在会自动包含设置 `insert_distributed_sync = 1`。 这可能会导致某些 Distributed 表插入操作比预期更慢。 -#### Distributed 表模型示例 +#### Distributed 表模型示例 {#distributed-table-model-example} ```sql {{ @@ -561,7 +561,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### 自动生成的迁移 +#### 自动生成的迁移 {#distributed-table-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -581,7 +581,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### materialization: distributed_incremental(实验性) +### materialization: distributed_incremental(实验性) {#materialization-distributed-incremental} 基于与 Distributed 表相同理念的增量模型,主要难点在于要正确处理所有增量策略。 @@ -592,7 +592,7 @@ CREATE TABLE db.table on cluster cluster ( 只会替换各分片上的本地表,因为 Distributed 表本身不存储数据。 Distributed 表仅在启用 full_refresh 模式或表结构可能发生变化时才会重新加载。 -#### 分布式增量模型示例 +#### 分布式增量模型示例 {#distributed-incremental-model-example} ```sql @@ -609,7 +609,7 @@ select id, created_at, item from {{ source('db', 'table') }} ``` -#### 自动生成的迁移 +#### 自动生成的迁移 {#distributed-incremental-generated-migrations} ```sql CREATE TABLE db.table_local on cluster cluster ( @@ -628,7 +628,7 @@ CREATE TABLE db.table on cluster cluster ( ENGINE = Distributed ('cluster', 'db', 'table_local', cityHash64(id)); ``` -### 快照 +### 快照 {#snapshot} dbt 快照允许对可变模型随时间发生的变更进行记录。这样一来,就可以在模型上执行按时间点的查询,使分析人员能够“回到过去”查看模型之前的状态。ClickHouse 连接器支持此功能,并可通过以下语法进行配置: @@ -647,15 +647,15 @@ dbt 快照允许对可变模型随时间发生的变更进行记录。这样一 有关配置的更多信息,请查看 [snapshot configs](https://docs.getdbt.com/docs/build/snapshots#snapshot-configs) 参考页面。 -### 合约与约束 +### 合约与约束 {#contracts-and-constraints} 仅支持精确匹配的列类型合约。例如,如果合约中列类型为 UInt32,而模型返回 UInt64 或其他整数类型,则会失败。 ClickHouse *仅* 支持在整个表/模型上的 `CHECK` 约束。不支持主键、外键、唯一键以及列级别的 `CHECK` 约束。 (参见 ClickHouse 关于 primary/ORDER BY 键的文档。) -### 其他 ClickHouse 宏 +### 其他 ClickHouse 宏 {#additional-clickhouse-macros} -#### 模型物化辅助宏 +#### 模型物化辅助宏 {#model-materialization-utility-macros} 包含以下宏,用于简化创建 ClickHouse 特定的表和视图: @@ -672,7 +672,7 @@ ClickHouse *仅* 支持在整个表/模型上的 `CHECK` 约束。不支持主 * `ttl_config` -- 使用 `ttl` 模型配置属性来指定 ClickHouse 表的 TTL 表达式。 默认不设置 TTL。 -#### s3Source 辅助宏 +#### s3Source 辅助宏 {#s3source-helper-macro} `s3source` 宏简化了使用 ClickHouse S3 表函数直接从 S3 中选择 ClickHouse 数据的流程。其工作方式是 从具名配置字典(字典名称必须以 `s3` 结尾)中填充 S3 表函数参数。该宏会 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md index da92b149a67..840f49d5dba 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/guides.md @@ -20,7 +20,7 @@ import dbt_07 from '@site/static/images/integrations/data-ingestion/etl-tools/db import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# 指南 +# 指南 {#guides} @@ -39,13 +39,13 @@ import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -## 设置 +## 设置 {#setup} 请按照[设置 dbt 和 ClickHouse 适配器](/integrations/dbt)部分中的说明来准备环境。 **重要:以下内容已在 Python 3.9 环境下测试通过。** -### 准备 ClickHouse +### 准备 ClickHouse {#prepare-clickhouse} dbt 在对高度关系型的数据进行建模时表现出色。作为示例,我们提供了一个包含如下关系型模式的小型 IMDb 数据集。该数据集来源于[关系型数据集仓库](https://relational.fit.cvut.cz/dataset/IMDb)。与 dbt 常见的模式相比,这个示例非常简单,但可以作为一个便于上手的示例样本: @@ -672,7 +672,7 @@ SELECT * FROM imdb_dbt.actor_summary ORDER BY num_movies DESC LIMIT 2; +------+-------------------+----------+------------------+------+---------+-------------------+ ``` -### 内部实现 +### 内部实现 {#internals} 我们可以通过查询 ClickHouse 的查询日志,找出为完成上述增量更新而执行的语句。 @@ -694,7 +694,7 @@ AND event_time > subtractMinutes(now(), 15) ORDER BY event_time LIMIT 100; 这种策略在非常大的模型上可能会遇到挑战。更多细节请参见 [Limitations](/integrations/dbt#limitations)。 -### 追加策略(仅插入模式) +### 追加策略(仅插入模式) {#append-strategy-inserts-only-mode} 为克服在大数据集上使用增量模型的限制,适配器使用 dbt 配置参数 `incremental_strategy`。该参数可以设置为 `append`。设置后,更新的行会被直接插入到目标表(即 `imdb_dbt.actor_summary`)中,而不会创建临时表。 注意:仅追加模式要求你的数据是不可变的,或者可以接受重复数据。如果你需要一个支持已修改行的增量表模型,请不要使用此模式! @@ -796,7 +796,7 @@ WHERE id > (SELECT max(id) FROM imdb_dbt.actor_summary) OR updated_at > (SELECT 在本次运行中,只有新增的行会直接添加到 `imdb_dbt.actor_summary` 表中,不会涉及创建新表。 -### 删除并插入模式(实验性) +### 删除并插入模式(实验性) {#deleteinsert-mode-experimental} 一直以来,ClickHouse 仅通过异步的 [变更(Mutations)](/sql-reference/statements/alter/index.md) 对更新和删除提供有限支持。这些操作对 IO 消耗极大,通常应尽量避免。 @@ -821,7 +821,7 @@ ClickHouse 22.8 引入了[轻量级删除](/sql-reference/statements/delete.md) -### insert_overwrite 模式(实验性) +### insert_overwrite 模式(实验性) {#insert_overwrite-mode-experimental} 执行以下步骤: @@ -840,7 +840,7 @@ ClickHouse 22.8 引入了[轻量级删除](/sql-reference/statements/delete.md) -## 创建快照 +## 创建快照 {#creating-a-snapshot} dbt 快照允许随着时间推移记录可变模型的变更。这使得可以在模型上执行时间点查询,从而让分析人员“回溯”查看模型先前的状态。其通过使用[类型 2 缓慢变化维度](https://en.wikipedia.org/wiki/Slowly_changing_dimension#Type_2:_add_new_row)实现,其中 from 和 to 日期列用于记录某一行数据在什么时间段内有效。ClickHouse 适配器支持此功能,下面将进行演示。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md index 5d664753e77..52588aa666a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dbt/index.md @@ -95,9 +95,9 @@ dbt 提供 4 种物化方式: -## 配置 dbt 和 ClickHouse 适配器 +## 配置 dbt 和 ClickHouse 适配器 {#setup-of-dbt-and-the-clickhouse-adapter} -### 安装 dbt-core 和 dbt-clickhouse +### 安装 dbt-core 和 dbt-clickhouse {#install-dbt-core-and-dbt-clickhouse} dbt 提供了多种安装命令行界面(CLI)的方法,详细说明见 [此处](https://docs.getdbt.com/dbt-cli/install/overview)。我们建议使用 `pip` 安装 dbt 和 dbt-clickhouse。 @@ -105,7 +105,7 @@ dbt 提供了多种安装命令行界面(CLI)的方法,详细说明见 [ pip install dbt-core dbt-clickhouse ``` -### 为 dbt 提供 ClickHouse 实例的连接详细信息。 +### 为 dbt 提供 ClickHouse 实例的连接详细信息。 {#provide-dbt-with-the-connection-details-for-our-clickhouse-instance} 在 `~/.dbt/profiles.yml` 文件中配置名为 `clickhouse-service` 的 profile,并提供 schema、host、port、user 和 password 属性。完整的连接配置选项列表请参见 [功能与配置](/integrations/dbt/features-and-configurations) 页面: @@ -125,7 +125,7 @@ clickhouse-service: secure: True # 使用 TLS(原生协议)或 HTTPS(HTTP 协议) ``` -### 创建 dbt 项目 +### 创建 dbt 项目 {#create-a-dbt-project} 现在你可以在现有项目中使用此配置,或使用以下命令创建新项目: @@ -139,17 +139,17 @@ dbt init <项目名称> profile: 'clickhouse-service' ``` -### 测试连接 +### 测试连接 {#test-connection} 使用 CLI 工具执行 `dbt debug`,以确认 dbt 是否能够连接到 ClickHouse。确认输出中包含 `Connection test: [OK connection ok]`,这表示连接成功。 前往[指南页面](/integrations/dbt/guides)以了解更多关于如何在 ClickHouse 中使用 dbt 的信息。 -### 测试和部署你的模型(CI/CD) +### 测试和部署你的模型(CI/CD) {#testing-and-deploying-your-models-ci-cd} 有多种方式可以测试和部署你的 dbt 项目。dbt 提供了一些关于[最佳实践工作流](https://docs.getdbt.com/best-practices/best-practice-workflows#pro-tips-for-workflows)和[CI 作业](https://docs.getdbt.com/docs/deploy/ci-jobs)的建议。我们将讨论几种策略,但请记住,这些策略可能需要进行较大幅度的调整以适配你的具体用例。 -#### 使用简单数据测试和单元测试的 CI/CD +#### 使用简单数据测试和单元测试的 CI/CD {#ci-with-simple-data-tests-and-unit-tests} 启动 CI 流水线的一种简单方式,是在作业内部运行一个 ClickHouse 集群,然后在其上运行你的模型。在运行模型之前,你可以向该集群插入演示数据。你可以直接使用一个 [seed](https://docs.getdbt.com/reference/commands/seed) 来用生产数据的一个子集填充暂存环境(staging 环境)。 @@ -157,7 +157,7 @@ profile: 'clickhouse-service' 你的 CD 步骤可以非常简单,只需针对生产 ClickHouse 集群运行 `dbt build` 即可。 -#### 更完整的 CI/CD 阶段:使用最新数据,仅测试受影响的模型 +#### 更完整的 CI/CD 阶段:使用最新数据,仅测试受影响的模型 {#more-complete-ci-stage} 一种常见策略是使用 [Slim CI](https://docs.getdbt.com/best-practices/best-practice-workflows#run-only-modified-models-to-test-changes-slim-ci) 作业,只重新部署被修改的模型(以及其上下游依赖)。这种方法利用生产运行生成的制品(例如 [dbt manifest](https://docs.getdbt.com/reference/artifacts/manifest-json))来缩短项目运行时间,并确保各环境之间的模式不会发生漂移。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md index e0a73d59f69..e5069e15042 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/dlt-and-clickhouse.md @@ -10,7 +10,7 @@ doc_type: 'guide' import PartnerBadge from '@theme/badges/PartnerBadge'; -# 将 dlt 连接到 ClickHouse +# 将 dlt 连接到 ClickHouse {#connect-dlt-to-clickhouse} @@ -18,9 +18,9 @@ import PartnerBadge from '@theme/badges/PartnerBadge'; -## 安装适用于 ClickHouse 的 dlt +## 安装适用于 ClickHouse 的 dlt {#install-dlt-with-clickhouse} -### 安装包含 ClickHouse 依赖的 `dlt` 库: +### 安装包含 ClickHouse 依赖的 `dlt` 库: {#to-install-the-dlt-library-with-clickhouse-dependencies} ```bash pip install "dlt[clickhouse]" @@ -99,7 +99,7 @@ dataset_table_separator = "___" # 数据集表名与数据集之间的 ```bash -# 将此配置保持在 toml 文件的顶部,在任何节(section)开始之前。 +# 将此配置保持在 toml 文件的顶部,在任何节(section)开始之前。 {#keep-it-at-the-top-of-your-toml-file-before-any-section-starts} destination.clickhouse.credentials="clickhouse://dlt:Dlt*12345789234567@localhost:9000/dlt?secure=1" ``` @@ -156,7 +156,7 @@ ClickHouse 支持以下GCS table function 自动处理,dlt 在内部会调用该函数。 @@ -240,10 +240,10 @@ dlt 会将这些凭据传递给 ClickHouse,由 ClickHouse 处理认证并访 * 使 filesystem 目标在 s3 兼容模式下与 gcs 正常工作 * Google Cloud Storage staging 区域 支持 -### Dbt 支持 +### Dbt 支持 {#dbt-support} 与 dbt 的集成通常通过 dbt-clickhouse 提供支持。 -### `dlt` 状态同步 +### `dlt` 状态同步 {#syncing-of-dlt-state} 此目标完全支持 dlt 状态同步。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md index cf7fa7d6030..66906e534c6 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/fivetran/index.md @@ -14,7 +14,7 @@ keywords: ['fivetran', '数据迁移', 'etl', 'ClickHouse 目标端', '自动化 import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Fivetran 与 ClickHouse Cloud +# Fivetran 与 ClickHouse Cloud {#fivetran-and-clickhouse-cloud} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md index d5d863f1e5b..e05292332fa 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/nifi-and-clickhouse.md @@ -11,7 +11,7 @@ integration: - category: 'data_ingestion' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; import nifi01 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_01.png'; import nifi02 from '@site/static/images/integrations/data-ingestion/etl-tools/nifi_02.png'; @@ -31,7 +31,7 @@ import nifi15 from '@site/static/images/integrations/data-ingestion/etl-tools/ni import CommunityMaintainedBadge from '@theme/badges/CommunityMaintained'; -# 将 Apache NiFi 连接到 ClickHouse +# 将 Apache NiFi 连接到 ClickHouse {#connect-apache-nifi-to-clickhouse} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md index 885d9368360..4bc877d7b2a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/etl-tools/vector-to-clickhouse.md @@ -18,8 +18,7 @@ import vector01 from '@site/static/images/integrations/data-ingestion/etl-tools/ import vector02 from '@site/static/images/integrations/data-ingestion/etl-tools/vector_02.png'; import PartnerBadge from '@theme/badges/PartnerBadge'; - -# 将 Vector 与 ClickHouse 集成 +# 将 Vector 与 ClickHouse 集成 {#integrating-vector-with-clickhouse} @@ -31,214 +30,202 @@ ClickHouse 在存储和分析日志数据方面表现卓越,这得益于其出 **前置条件:** -- 您已部署并运行 ClickHouse -- 您已安装 Vector +* 您已部署并运行 ClickHouse +* 您已安装 Vector + ## 创建数据库和表 {#1-create-a-database-and-table} + 定义一个用于存储日志事件的表: -## 创建数据库和表 - -定义一个用于存储日志事件的表: - -1. 首先创建一个名为 `nginxdb` 的新数据库: - -```sql -CREATE DATABASE IF NOT EXISTS nginxdb -``` - -2. 将整条日志事件作为一个字符串插入。显然,这并不是对日志数据进行分析的理想格式,不过我们会在下文中借助***物化视图***来解决这一问题。 - -```sql -CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( - message String -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -:::note -**ORDER BY** 被设置为 **tuple()**(一个空元组),因为当前还不需要主键。 -::: - - -## 配置 Nginx - -在本步骤中,将演示如何配置 Nginx 日志记录。 - -1. 以下 `access_log` 属性会以 **combined** 格式将日志写入 `/var/log/nginx/my_access.log`。 - 该配置应放在 `nginx.conf` 文件的 `http` 块中: - -```bash -http { - include /etc/nginx/mime.types; - default_type application/octet-stream; - access_log /var/log/nginx/my_access.log combined; - sendfile on; - keepalive_timeout 65; - include /etc/nginx/conf.d/*.conf; -} -``` - -2. 如果你修改了 `nginx.conf`,务必重启 Nginx。 - -3. 通过访问 Web 服务器上的页面,在访问日志(access log)中生成一些日志事件。\ - 以 **combined** 格式记录的日志大致如下: - -```bash -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - + 1. 首先创建一个名为 `nginxdb` 的新数据库: -## 配置 Vector + ```sql + CREATE DATABASE IF NOT EXISTS nginxdb + ``` -Vector 会收集、转换并路由日志、指标和追踪数据(统称为 **sources**),将其发送到多个不同的后端目标(统称为 **sinks**),并且开箱即用地兼容 ClickHouse。 -Sources 和 sinks 都在名为 **vector.toml** 的配置文件中定义。 + 2. 将整条日志事件作为一个字符串插入。显然,这并不是对日志数据进行分析的理想格式,不过我们会在下文中借助***物化视图***来解决这一问题。 -1. 以下 **vector.toml** 文件定义了一个类型为 **file** 的 **source**,用于持续跟踪读取 **my_access.log** 文件末尾的内容,同时还定义了一个 **sink**,即上文定义的 **access_logs** 表: + ```sql + CREATE TABLE IF NOT EXISTS nginxdb.access_logs ( + message String + ) + ENGINE = MergeTree() + ORDER BY tuple() + ``` -```bash -[sources.nginx_logs] -type = "file" -include = [ "/var/log/nginx/my_access.log" ] -read_from = "end" + :::note + **ORDER BY** 被设置为 **tuple()**(一个空元组),因为当前还不需要主键。 + ::: -[sinks.clickhouse] -type = "clickhouse" -inputs = ["nginx_logs"] -endpoint = "http://clickhouse-server:8123" -database = "nginxdb" -table = "access_logs" -skip_unknown_fields = true -``` + ## 配置 Nginx {#2--configure-nginx} -2. 使用上述配置启动 Vector。请参阅 Vector 的[文档](https://vector.dev/docs/),以了解更多关于定义 source 和 sink 的详细信息。 + 在本步骤中,将演示如何配置 Nginx 日志记录。 -3. 通过运行以下查询验证访问日志是否已经写入 ClickHouse。您应该能在表中看到这些访问日志: + 1. 以下 `access_log` 属性会以 **combined** 格式将日志写入 `/var/log/nginx/my_access.log`。 + 该配置应放在 `nginx.conf` 文件的 `http` 块中: -```sql -SELECT * FROM nginxdb.access_logs -``` + ```bash + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + access_log /var/log/nginx/my_access.log combined; + sendfile on; + keepalive_timeout 65; + include /etc/nginx/conf.d/*.conf; + } + ``` - + 2. 如果你修改了 `nginx.conf`,务必重启 Nginx。 + 3. 通过访问 Web 服务器上的页面,在访问日志(access log)中生成一些日志事件。\ + 以 **combined** 格式记录的日志大致如下: -## 解析日志 + ```bash + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET / HTTP/1.1" 200 615 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:44 +0000] "GET /favicon.ico HTTP/1.1" 404 555 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + 192.168.208.1 - - [12/Oct/2021:03:31:49 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` -将日志存储在 ClickHouse 中固然很好,但如果将每个事件都存储为单个字符串,就很难进行有效的数据分析。 -接下来我们将介绍如何使用[物化视图](/materialized-view/incremental-materialized-view)来解析日志事件。 + ## 配置 Vector {#3-configure-vector} -**物化视图**的作用类似于 SQL 中的插入触发器。当数据行被插入到源表时,物化视图会对这些行进行转换,并将结果插入到目标表中。 -我们可以配置物化视图,将 **access_logs** 中的日志事件解析为结构化表示。 -下面是一个此类日志事件的示例: + Vector 会收集、转换并路由日志、指标和追踪数据(统称为 **sources**),将其发送到多个不同的后端目标(统称为 **sinks**),并且开箱即用地兼容 ClickHouse。 + Sources 和 sinks 都在名为 **vector.toml** 的配置文件中定义。 -```bash -192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" -``` - -ClickHouse 中有多种函数可以用来解析上述字符串。[`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) 函数按空白字符对字符串进行分割,并将每个标记作为数组元素返回。 -为演示这一点,运行以下命令: - -```sql title="Query" -SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -```text title="Response" -["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] -``` - -有些字符串包含一些多余字符,而且用户代理字符串(浏览器信息)其实无需解析,不过 -生成的数组已经与所需结果非常接近。 - -类似于 `splitByWhitespace`,[`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) 函数会基于正则表达式将字符串拆分为数组。 -运行以下命令,它会返回两个字符串。 - -```sql -SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') -``` - -请注意,返回的第二个字符串是从日志中成功解析出的 User-Agent: - -```text -["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] -``` - -在查看最终的 `CREATE MATERIALIZED VIEW` 命令之前,先来看几个用于清理数据的函数。 -例如,`RequestMethod` 的值是 `"GET`,其中包含一个多余的双引号。 -可以使用 [`trimBoth`(别名 `trim`)](/sql-reference/functions/string-functions#trimBoth) 函数来移除这个双引号: - -```sql -SELECT trim(LEADING '"' FROM '"GET') -``` - -时间字符串开头有一个左方括号,而且其格式也不是 ClickHouse 能够解析为日期的格式。 -但是,如果我们把分隔符从冒号(**:**)改成逗号(**,**),那么就可以顺利完成解析: - -```sql -SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) -``` - - -现在我们可以定义物化视图了。 -下面的定义包含 `POPULATE`,这意味着 **access_logs** 中的现有行将立即被处理并插入。 -运行以下 SQL 语句: - -```sql -CREATE MATERIALIZED VIEW nginxdb.access_logs_view -( - RemoteAddr String, - Client String, - RemoteUser String, - TimeLocal DateTime, - RequestMethod String, - Request String, - HttpVersion String, - Status Int32, - BytesSent Int64, - UserAgent String -) -ENGINE = MergeTree() -ORDER BY RemoteAddr -POPULATE AS -WITH - splitByWhitespace(message) as split, - splitByRegexp('\S \d+ "([^"]*)"', message) as referer -SELECT - split[1] AS RemoteAddr, - split[2] AS Client, - split[3] AS RemoteUser, - parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, - trim(LEADING '"' FROM split[6]) AS RequestMethod, - split[7] AS Request, - trim(TRAILING '"' FROM split[8]) AS HttpVersion, - split[9] AS Status, - split[10] AS BytesSent, - trim(BOTH '"' from referer[2]) AS UserAgent -FROM - (SELECT message FROM nginxdb.access_logs) -``` - -现在验证其是否生效。 -您应该看到访问日志已被正确解析为列: - -```sql -SELECT * FROM nginxdb.access_logs_view -``` - - - -:::note -上述示例将数据存储在两个表中,但您可以将初始的 `nginxdb.access_logs` 表更改为使用 [`Null`](/engines/table-engines/special/null) 表引擎。 -解析后的数据仍将存储在 `nginxdb.access_logs_view` 表中,但原始数据不会存储在表中。 -::: + 1. 以下 **vector.toml** 文件定义了一个类型为 **file** 的 **source**,用于持续跟踪读取 **my_access.log** 文件末尾的内容,同时还定义了一个 **sink**,即上文定义的 **access_logs** 表: + ```bash + [sources.nginx_logs] + type = "file" + include = [ "/var/log/nginx/my_access.log" ] + read_from = "end" + + [sinks.clickhouse] + type = "clickhouse" + inputs = ["nginx_logs"] + endpoint = "http://clickhouse-server:8123" + database = "nginxdb" + table = "access_logs" + skip_unknown_fields = true + ``` + + 2. 使用上述配置启动 Vector。请参阅 Vector 的[文档](https://vector.dev/docs/),以了解更多关于定义 source 和 sink 的详细信息。 + + 3. 通过运行以下查询验证访问日志是否已经写入 ClickHouse。您应该能在表中看到这些访问日志: + + ```sql + SELECT * FROM nginxdb.access_logs + ``` + + + + ## 解析日志 {#4-parse-the-logs} + + 将日志存储在 ClickHouse 中固然很好,但如果将每个事件都存储为单个字符串,就很难进行有效的数据分析。 + 接下来我们将介绍如何使用[物化视图](/materialized-view/incremental-materialized-view)来解析日志事件。 + + **物化视图**的作用类似于 SQL 中的插入触发器。当数据行被插入到源表时,物化视图会对这些行进行转换,并将结果插入到目标表中。 + 我们可以配置物化视图,将 **access_logs** 中的日志事件解析为结构化表示。 + 下面是一个此类日志事件的示例: + + ```bash + 192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36" + ``` + + ClickHouse 中有多种函数可以用来解析上述字符串。[`splitByWhitespace`](/sql-reference/functions/splitting-merging-functions#splitByWhitespace) 函数按空白字符对字符串进行分割,并将每个标记作为数组元素返回。 + 为演示这一点,运行以下命令: + + ```sql title="Query" + SELECT splitByWhitespace('192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + ```text title="Response" + ["192.168.208.1","-","-","[12/Oct/2021:15:32:43","+0000]","\"GET","/","HTTP/1.1\"","304","0","\"-\"","\"Mozilla/5.0","(Macintosh;","Intel","Mac","OS","X","10_15_7)","AppleWebKit/537.36","(KHTML,","like","Gecko)","Chrome/93.0.4577.63","Safari/537.36\""] + ``` + + 有些字符串包含一些多余字符,而且用户代理字符串(浏览器信息)其实无需解析,不过 + 生成的数组已经与所需结果非常接近。 + + 类似于 `splitByWhitespace`,[`splitByRegexp`](/sql-reference/functions/splitting-merging-functions#splitByRegexp) 函数会基于正则表达式将字符串拆分为数组。 + 运行以下命令,它会返回两个字符串。 + + ```sql + SELECT splitByRegexp('\S \d+ "([^"]*)"', '192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"') + ``` + + 请注意,返回的第二个字符串是从日志中成功解析出的 User-Agent: + + ```text + ["192.168.208.1 - - [12/Oct/2021:15:32:43 +0000] \"GET / HTTP/1.1\" 30"," \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36\""] + ``` + + 在查看最终的 `CREATE MATERIALIZED VIEW` 命令之前,先来看几个用于清理数据的函数。 + 例如,`RequestMethod` 的值是 `"GET`,其中包含一个多余的双引号。 + 可以使用 [`trimBoth`(别名 `trim`)](/sql-reference/functions/string-functions#trimBoth) 函数来移除这个双引号: + + ```sql + SELECT trim(LEADING '"' FROM '"GET') + ``` + + 时间字符串开头有一个左方括号,而且其格式也不是 ClickHouse 能够解析为日期的格式。 + 但是,如果我们把分隔符从冒号(**:**)改成逗号(**,**),那么就可以顺利完成解析: + + ```sql + SELECT parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM '[12/Oct/2021:15:32:43'), ':', ' ')) + ``` + + 现在我们可以定义物化视图了。 + 下面的定义包含 `POPULATE`,这意味着 **access_logs** 中的现有行将立即被处理并插入。 + 运行以下 SQL 语句: + + ```sql + CREATE MATERIALIZED VIEW nginxdb.access_logs_view + ( + RemoteAddr String, + Client String, + RemoteUser String, + TimeLocal DateTime, + RequestMethod String, + Request String, + HttpVersion String, + Status Int32, + BytesSent Int64, + UserAgent String + ) + ENGINE = MergeTree() + ORDER BY RemoteAddr + POPULATE AS + WITH + splitByWhitespace(message) as split, + splitByRegexp('\S \d+ "([^"]*)"', message) as referer + SELECT + split[1] AS RemoteAddr, + split[2] AS Client, + split[3] AS RemoteUser, + parseDateTimeBestEffort(replaceOne(trim(LEADING '[' FROM split[4]), ':', ' ')) AS TimeLocal, + trim(LEADING '"' FROM split[6]) AS RequestMethod, + split[7] AS Request, + trim(TRAILING '"' FROM split[8]) AS HttpVersion, + split[9] AS Status, + split[10] AS BytesSent, + trim(BOTH '"' from referer[2]) AS UserAgent + FROM + (SELECT message FROM nginxdb.access_logs) + ``` + + 现在验证其是否生效。 + 您应该看到访问日志已被正确解析为列: + + ```sql + SELECT * FROM nginxdb.access_logs_view + ``` + + + + :::note + 上述示例将数据存储在两个表中,但您可以将初始的 `nginxdb.access_logs` 表更改为使用 [`Null`](/engines/table-engines/special/null) 表引擎。 + 解析后的数据仍将存储在 `nginxdb.access_logs_view` 表中,但原始数据不会存储在表中。 + ::: > 通过使用 Vector(只需简单安装和快速配置),您可以将 Nginx 服务器的日志发送到 ClickHouse 表中。通过使用物化视图,您可以将这些日志解析为列,以便更轻松地进行分析。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md index 45af2f56446..bf23b97b44b 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/gcs/index.md @@ -8,13 +8,13 @@ doc_type: 'guide' keywords: ['Google Cloud Storage ClickHouse', 'GCS ClickHouse 集成', 'GCS 后端 MergeTree', 'ClickHouse GCS 存储', 'Google Cloud ClickHouse'] --- -import BucketDetails from '@site/docs/_snippets/_GCS_authentication_and_bucket.md'; +import BucketDetails from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_GCS_authentication_and_bucket.md'; import Image from '@theme/IdealImage'; import GCS_examine_bucket_1 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-1.png'; import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestion/s3/GCS-examine-bucket-2.png'; -# 将 Google Cloud Storage 与 ClickHouse 集成 +# 将 Google Cloud Storage 与 ClickHouse 集成 {#integrate-google-cloud-storage-with-clickhouse} :::note 如果您在 [Google Cloud](https://cloud.google.com) 上使用 ClickHouse Cloud,则本页内容不适用,因为您的服务已经在使用 [Google Cloud Storage](https://cloud.google.com/storage)。如果您希望从 GCS 中执行 `SELECT` 或向 GCS 中执行 `INSERT` 操作,请参阅 [`gcs` 表函数](/sql-reference/table-functions/gcs)。 @@ -24,13 +24,13 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio -## 基于 GCS 的 MergeTree +## 基于 GCS 的 MergeTree {#gcs-backed-mergetree} -### 创建磁盘 +### 创建磁盘 {#creating-a-disk} 要将 GCS 存储桶用作磁盘,首先必须在 `conf.d` 目录下的文件中,在 ClickHouse 配置中声明它。下面展示了一个 GCS 磁盘声明示例。该配置包含多个部分,用于配置 GCS “磁盘”、缓存,以及在需要在 GCS 磁盘上创建表时在 DDL 查询中指定的策略。下面分别对这些部分进行说明。 -#### Storage configuration > disks > gcs +#### Storage configuration > disks > gcs {#storage_configuration--disks--gcs} 配置中高亮显示的这部分内容表示: @@ -68,7 +68,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio ``` -#### Storage configuration > disks > cache +#### Storage configuration > disks > cache {#storage_configuration--disks--cache} 下方高亮显示的示例配置为磁盘 `gcs` 启用了 10Gi 内存缓存。 @@ -106,7 +106,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio ``` -#### Storage configuration > policies > gcs_main +#### Storage configuration > policies > gcs_main {#storage_configuration--policies--gcs_main} 存储配置中的策略用于选择数据的存放位置。下面高亮的策略通过指定策略 `gcs_main`,允许将数据存储在名为 `gcs` 的磁盘上。例如 `CREATE TABLE ... SETTINGS storage_policy='gcs_main'`。 @@ -141,7 +141,7 @@ import GCS_examine_bucket_2 from '@site/static/images/integrations/data-ingestio 与此磁盘配置相关的设置完整列表可在[此处](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3)中找到。 -### 创建表 +### 创建表 {#creating-a-table} 假设你已经将磁盘配置为使用具有写权限的存储桶,现在应该可以创建如下示例中的表。为简洁起见,我们只使用 NYC taxi 数据集中的部分列,并将数据直接流式写入由 GCS 作为后端存储的表中: @@ -179,11 +179,11 @@ INSERT INTO trips_gcs SELECT trip_id, pickup_date, pickup_datetime, dropoff_date SELECT passenger_count, avg(tip_amount) AS avg_tip, avg(total_amount) AS avg_amount FROM trips_gcs GROUP BY passenger_count; ``` -### 处理复制 +### 处理复制 {#handling-replication} 使用 GCS 磁盘时,可以通过 `ReplicatedMergeTree` 表引擎来实现复制。有关详细信息,请参阅[使用 GCS 在两个 GCP 区域之间复制单个分片](#gcs-multi-region)指南。 -### 了解更多 +### 了解更多 {#learn-more} [Cloud Storage XML API](https://cloud.google.com/storage/docs/xml-api/overview) 可与某些适用于 Amazon Simple Storage Service(Amazon S3)等服务的工具和库互操作。 @@ -305,13 +305,13 @@ ClickHouse Keeper 至少需要两个节点才能工作,因此为了实现高 ``` -### 配置 ClickHouse 服务器 +### 配置 ClickHouse 服务器 {#configure-clickhouse-server} :::note best practice 本指南中的某些步骤会要求你将配置文件放置在 `/etc/clickhouse-server/config.d/` 中。这是 Linux 系统上用于放置覆盖默认配置文件的默认位置。当你将这些文件放入该目录时,ClickHouse 会将其内容与默认配置进行合并。通过将这些文件放在 `config.d` 目录中,你可以在升级过程中避免丢失自己的配置。 ::: -#### 网络 +#### 网络 {#networking} 默认情况下,ClickHouse 监听回环接口;在副本部署环境中,机器之间需要进行网络通信。要监听所有接口: @@ -321,7 +321,7 @@ ClickHouse Keeper 至少需要两个节点才能工作,因此为了实现高 ``` -#### 远程 ClickHouse Keeper 服务器 +#### 远程 ClickHouse Keeper 服务器 {#remote-clickhouse-keeper-servers} 副本复制由 ClickHouse Keeper 协调完成。此配置文件通过主机名和端口号来标识 ClickHouse Keeper 节点。 @@ -346,7 +346,7 @@ ClickHouse Keeper 至少需要两个节点才能工作,因此为了实现高 ``` -#### 远程 ClickHouse 服务器 +#### 远程 ClickHouse 服务器 {#remote-clickhouse-servers} 此文件用于配置集群中每个 ClickHouse 服务器的主机名和端口。默认配置文件包含示例集群定义。为了只显示已完全配置的集群,会在 `remote_servers` 条目中添加标签 `replace="true"`,这样当此配置与默认配置合并时,会替换 `remote_servers` 部分,而不是在其基础上追加内容。 @@ -372,7 +372,7 @@ ClickHouse Keeper 至少需要两个节点才能工作,因此为了实现高 ``` -#### 副本标识 +#### 副本标识 {#replica-identification} 此文件用于配置与 ClickHouse Keeper 路径相关的设置,尤其是用于标识数据属于哪个副本的宏。在一台服务器上,应将副本指定为 `replica_1`,在另一台服务器上指定为 `replica_2`。这些名称可以修改,例如在我们的示例中,一个副本存储在南卡罗来纳州,另一个存储在北弗吉尼亚州,则可以分别命名为 `carolina` 和 `virginia`;只需确保每台机器上的名称彼此不同即可。 @@ -390,7 +390,7 @@ ClickHouse Keeper 至少需要两个节点才能工作,因此为了实现高 ``` -#### 在 GCS 中配置存储 +#### 在 GCS 中配置存储 {#storage-in-gcs} ClickHouse 的存储配置包括 `disks` 和 `policies`。下面配置的磁盘名为 `gcs`,其 `type` 为 `s3`。之所以使用 s3 类型,是因为 ClickHouse 访问 GCS bucket 的方式与访问 AWS S3 bucket 相同。此配置需要准备两份,分别应用于两个 ClickHouse 服务器节点。 @@ -438,7 +438,7 @@ ClickHouse 的存储配置包括 `disks` 和 `policies`。下面配置的磁盘 ``` -### 启动 ClickHouse Keeper +### 启动 ClickHouse Keeper {#start-clickhouse-keeper} 根据所使用的操作系统运行相应的命令,例如: @@ -448,7 +448,7 @@ sudo systemctl start clickhouse-keeper sudo systemctl status clickhouse-keeper ``` -#### 检查 ClickHouse Keeper 状态 +#### 检查 ClickHouse Keeper 状态 {#check-clickhouse-keeper-status} 通过 `netcat` 向 ClickHouse Keeper 发送命令。例如,`mntr` 会返回 ClickHouse Keeper 集群的状态。如果你在每个 Keeper 节点上执行该命令,你会看到其中一个是 leader,另外两个是 follower: @@ -464,11 +464,11 @@ zk_max_latency 11 zk_min_latency 0 zk_packets_received 1783 zk_packets_sent 1783 -# highlight-start +# highlight-start {#highlight-start} zk_num_alive_connections 2 zk_outstanding_requests 0 zk_server_state leader -# highlight-end +# highlight-end {#highlight-end} zk_znode_count 135 zk_watch_count 8 zk_ephemerals_count 3 @@ -477,13 +477,13 @@ zk_key_arena_size 28672 zk_latest_snapshot_size 0 zk_open_file_descriptor_count 182 zk_max_file_descriptor_count 18446744073709551615 -# highlight-start +# highlight-start {#highlight-start} zk_followers 2 zk_synced_followers 2 -# highlight-end +# highlight-end {#highlight-end} ``` -### 启动 ClickHouse 服务器 +### 启动 ClickHouse 服务器 {#start-clickhouse-server} 在 `chnode1` 和 `chnode` 上运行: @@ -495,9 +495,9 @@ sudo service clickhouse-server start sudo service clickhouse-server status ``` -### 验证 +### 验证 {#verification} -#### 验证磁盘配置 +#### 验证磁盘配置 {#verify-disk-configuration} `system.disks` 中应包含每个磁盘对应的一条记录: @@ -565,7 +565,7 @@ cache_path: 3 行数据,耗时 0.002 秒。 ```` -#### 验证在集群上创建的表已在两个节点上创建 +#### 验证在集群上创建的表已在两个节点上创建 {#verify-that-tables-created-on-the-cluster-are-created-on-both-nodes} ```sql -- highlight-next-line create table trips on cluster 'cluster_1S_2R' ( @@ -600,7 +600,7 @@ SETTINGS storage_policy='gcs_main' 2 rows in set. Elapsed: 0.641 sec. ``` -#### 验证数据能否插入 +#### 验证数据能否插入 {#verify-that-data-can-be-inserted} ```sql INSERT INTO trips SELECT @@ -621,7 +621,7 @@ FROM s3('https://ch-nyc-taxi.s3.eu-west-3.amazonaws.com/tsv/trips_{0..9}.tsv.gz' LIMIT 1000000 ``` -#### 验证该表是否使用了存储策略 `gcs_main`。 +#### 验证该表是否使用了存储策略 `gcs_main`。 {#verify-that-the-storage-policy-gcs_main-is-used-for-the-table} ```sql SELECT @@ -647,14 +647,14 @@ formatReadableSize(total_bytes): 36.42 MiB 返回 1 行。用时:0.002 秒。 ``` -#### 在 Google Cloud 控制台中验证 +#### 在 Google Cloud 控制台中验证 {#verify-in-google-cloud-console} 查看这些 bucket,你会发现每个 bucket 中都创建了一个文件夹,文件夹名称与 `storage.xml` 配置文件中使用的名称相同。展开这些文件夹,你会看到许多文件,对应各个数据分区。 -#### 副本一的 Bucket +#### 副本一的 Bucket {#bucket-for-replica-one} -#### 副本二的 Bucket +#### 副本二的 Bucket {#bucket-for-replica-two} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md index 93aaf4a8927..1e20936a9ce 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/dataflow.md @@ -11,7 +11,7 @@ keywords: ['Google Dataflow ClickHouse', 'Dataflow ClickHouse integration', 'Apa import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# 将 Google Dataflow 与 ClickHouse 集成 +# 将 Google Dataflow 与 ClickHouse 集成 {#integrating-google-dataflow-with-clickhouse} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md index 526ce4ae60b..efd68be8a6e 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/java-runner.md @@ -11,7 +11,7 @@ keywords: ['Dataflow Java Runner', 'Google Dataflow ClickHouse', 'Apache Beam Ja import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Dataflow Java 运行器 +# Dataflow Java 运行器 {#dataflow-java-runner} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md index 636f1d19107..12e84e30270 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates.md @@ -11,7 +11,7 @@ keywords: ['Google Dataflow', 'GCP', '数据管道', '模板', '批处理'] import ClickHouseSupportedBadge from '@theme/badges/ClickHouseSupported'; -# Google Dataflow 模板 +# Google Dataflow 模板 {#google-dataflow-templates} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md index 8ae6a40e99b..80defca71f1 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/google-dataflow/templates/bigquery-to-clickhouse.md @@ -19,7 +19,7 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -# Dataflow BigQuery 到 ClickHouse 模板 +# Dataflow BigQuery 到 ClickHouse 模板 {#dataflow-bigquery-to-clickhouse-template} BigQuery 到 ClickHouse 模板是一个批处理管道,用于将 BigQuery 表中的数据摄取到 ClickHouse 表中。 该模板可以读取整个表,或使用提供的 SQL 查询筛选特定记录。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md index 8662a276f66..62dee8a883a 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/insert-local-files.md @@ -9,7 +9,7 @@ doc_type: 'guide' keywords: ['插入本地文件 ClickHouse', 'ClickHouse 本地文件导入', 'clickhouse-client 文件上传'] --- -# 插入本地文件 +# 插入本地文件 {#insert-local-files} 你可以使用 `clickhouse-client` 将本地文件以流方式导入到你的 ClickHouse 服务中。这样你就可以在插入前利用众多强大且便捷的 ClickHouse 函数对数据进行预处理。下面来看一个示例…… @@ -79,7 +79,6 @@ LIMIT 7 结果如下: - ```response │ 488 │ comment │ mynameishere │ 2007-02-22 14:48:18 │ "It's too bad. Javascript-in-the-browser and Ajax are both nasty hacks that force programmers to do all sorts of shameful things. And the result is--wanky html tricks. Java, for its faults, is fairly clean when run in the applet environment. It has every superiority over JITBAJAX, except for install issues and a chunky load process. Yahoo games seems like just about the only applet success story. Of course, back in the day, non-trivial Applets tended to be too large for the dial-up accounts people had. At least that is changed." │ [454927] │ ['It','s','too','bad','Javascript','in','the','browser','and','Ajax','are','both','nasty','hacks','that','force','programmers','to','do','all','sorts','of','shameful','things','And','the','result','is','wanky','html','tricks','Java','for','its','faults','is','fairly','clean','when','run','in','the','applet','environment','It','has','every','superiority','over','JITBAJAX','except','for','install','issues','and','a','chunky','load','process','Yahoo','games','seems','like','just','about','the','only','applet','success','story','Of','course','back','in','the','day','non','trivial','Applets','tended','to','be','too','large','for','the','dial','up','accounts','people','had','At','least','that','is','changed'] │ │ 575 │ comment │ leoc │ 2007-02-23 00:09:49 │ "I can't find the reference now, but I *think* I've just read something suggesting that the install process for an Apollo applet will involve an "install-this-application?" confirmation dialog followed by a download of 30 seconds or so. If so then Apollo's less promising than I hoped. That kind of install may be low-friction by desktop-app standards but it doesn't compare to the ease of starting a browser-based AJAX or Flash application. (Consider how easy it is to use maps.google.com for the first time.)

Surely it will at least be that Apollo applications will run untrusted by default, and that an already-installed app will start automatically whenever you take your browser to the URL you downloaded it from?" │ [455071] │ ['I','can','t','find','the','reference','now','but','I','think','I','ve','just','read','something','suggesting','that','the','install','process','for','an','Apollo','applet','will','involve','an','34','install','this','application','34','confirmation','dialog','followed','by','a','download','of','30','seconds','or','so','If','so','then','Apollo','s','less','promising','than','I','hoped','That','kind','of','install','may','be','low','friction','by','desktop','app','standards','but','it','doesn','t','compare','to','the','ease','of','starting','a','browser','based','AJAX','or','Flash','application','Consider','how','easy','it','is','to','use','maps','google','com','for','the','first','time','p','Surely','it','will','at','least','be','that','Apollo','applications','will','run','untrusted','by','default','and','that','an','already','installed','app','will','start','automatically','whenever','you','take','your','browser','to','the','URL','you','downloaded','it','from'] │ diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md index aa96e925a0c..b74dde6ea29 100644 --- a/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md +++ b/i18n/zh/docusaurus-plugin-content-docs/current/integrations/data-ingestion/kafka/confluent/confluent-cloud.md @@ -12,11 +12,11 @@ integration: - website: 'https://clickhouse.com/cloud/clickpipes' --- -import ConnectionDetails from '@site/docs/_snippets/_gather_your_details_http.mdx'; +import ConnectionDetails from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_gather_your_details_http.mdx'; import Image from '@theme/IdealImage'; -# 将 Confluent Cloud 与 ClickHouse 集成 +# 将 Confluent Cloud 与 ClickHouse 集成 {#integrating-confluent-cloud-with-clickhouse}

\ No newline at end of file + -
- -このエンジンは、次の型を持つすべてのカラムを処理します。 - -- [`AggregateFunction`](../../../sql-reference/data-types/aggregatefunction.md) -- [`SimpleAggregateFunction`](../../../sql-reference/data-types/simpleaggregatefunction.md) - -行数を桁違いに削減できる場合には、`AggregatingMergeTree` を使用するのが適切です。 - - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = AggregatingMergeTree() -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[TTL expr] -[SETTINGS name=value, ...] -``` - -リクエストパラメータの説明は、[リクエストの説明](../../../sql-reference/statements/create/table.md)を参照してください。 - -**クエリの句** - -`AggregatingMergeTree` テーブルを作成する場合、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) を指定する必要があります。 - -
- テーブル作成の非推奨の方法 - - :::note - 新規プロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上で説明した方法へ移行してください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] AggregatingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - すべてのパラメータの意味は `MergeTree` と同じです。 -
- - -## SELECT と INSERT {#select-and-insert} - -データを挿入するには、集約関数の `-State` バージョンを用いた [INSERT SELECT](../../../sql-reference/statements/insert-into.md) クエリを使用します。 -`AggregatingMergeTree` テーブルからデータを選択する場合は、`GROUP BY` 句と、挿入時と同じ集約関数を使用しますが、`-Merge` 接尾辞を付けて使用します。 - -`SELECT` クエリの結果において、`AggregateFunction` 型の値は、すべての ClickHouse 出力形式で実装依存のバイナリ表現になります。たとえば、`SELECT` クエリでデータを `TabSeparated` 形式にダンプした場合、このダンプは `INSERT` クエリを使用して再度ロードできます。 - - - -## 集約マテリアライズドビューの例 {#example-of-an-aggregated-materialized-view} - -次の例では、`test` という名前のデータベースが既に存在すると仮定します。まだない場合は、以下のコマンドで作成してください。 - -```sql -CREATE DATABASE test; -``` - -次に、生データを格納するテーブル `test.visits` を作成します: - -```sql -CREATE TABLE test.visits - ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Sign Nullable(Int32), - UserID Nullable(Int32) -) ENGINE = MergeTree ORDER BY (StartDate, CounterID); -``` - -次に、訪問数の合計とユニークユーザー数を追跡する `AggregationFunction` を格納するための `AggregatingMergeTree` テーブルが必要です。 - -`test.visits` テーブルを監視し、[`AggregateFunction`](/sql-reference/data-types/aggregatefunction) 型を使用する `AggregatingMergeTree` マテリアライズドビューを作成します。 - -```sql -CREATE TABLE test.agg_visits ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Visits AggregateFunction(sum, Nullable(Int32)), - Users AggregateFunction(uniq, Nullable(Int32)) -) -ENGINE = AggregatingMergeTree() ORDER BY (StartDate, CounterID); -``` - -`test.visits` からデータを取り込み、`test.agg_visits` を更新するマテリアライズドビューを作成します: - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - sumState(Sign) AS Visits, - uniqState(UserID) AS Users -FROM test.visits -GROUP BY StartDate, CounterID; -``` - -`test.visits` テーブルにデータを挿入します。` - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1667446031000, 1, 3, 4), (1667446031000, 1, 6, 3); -``` - -データは `test.visits` と `test.agg_visits` の両方に挿入されます。 - -集約されたデータを取得するには、マテリアライズドビュー `test.visits_mv` に対して `SELECT ... GROUP BY ...` といったクエリを実行します。 - -```sql -SELECT - StartDate, - sumMerge(Visits) AS Visits, - uniqMerge(Users) AS Users -FROM test.visits_mv -GROUP BY StartDate -ORDER BY StartDate; -``` - -```text -┌───────────────StartDate─┬─Visits─┬─Users─┐ -│ 2022-11-03 03:27:11.000 │ 9 │ 2 │ -└─────────────────────────┴────────┴───────┘ -``` - -`test.visits` にさらに 2 件のレコードを追加します。ただし、今回はそのうち 1 件には別のタイムスタンプを指定してみてください。 - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1669446031000, 2, 5, 10), (1667446031000, 3, 7, 5); -``` - -`SELECT` クエリを再度実行すると、次のような出力が返されます。 - -```text -┌───────────────開始日─┬─訪問数─┬─ユーザー数─┐ -│ 2022-11-03 03:27:11.000 │ 16 │ 3 │ -│ 2022-11-26 07:00:31.000 │ 5 │ 1 │ -└─────────────────────────┴────────┴───────┘ -``` - -場合によっては、挿入時に行を事前集計せず、集計のコストを挿入時からマージ時へ移したいことがあります。通常は、エラーを避けるために、マテリアライズドビュー定義の `GROUP BY` -句に、集計の対象ではない列も含める必要があります。しかし、この動作を実現するには、`optimize_on_insert = 0`(デフォルトで有効になっています)を設定したうえで、[`initializeAggregation`](/sql-reference/functions/other-functions#initializeAggregation) -関数を利用できます。この場合、`GROUP BY` -を使用する必要はなくなります。 - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - initializeAggregation('sumState', Sign) AS Visits, - initializeAggregation('uniqState', UserID) AS Users -FROM test.visits; -``` - - -:::note -`initializeAggregation` を使用する場合、グループ化を行わずに各行に対して集約状態が作成されます。 -各ソース行はマテリアライズドビュー内で 1 行を生成し、実際の集約は後で `AggregatingMergeTree` がパートをマージする際に行われます。 -これは `optimize_on_insert = 0` の場合にのみ当てはまります。 -::: - - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Using Aggregate Combinators in ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md deleted file mode 100644 index bd01ea888cd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ /dev/null @@ -1,737 +0,0 @@ ---- -description: '厳密および近似ベクトル検索に関するドキュメント' -keywords: ['ベクトル類似度検索', 'ANN', 'kNN', 'HNSW', 'インデックス', 'インデックス作成', '最近傍探索', 'ベクトル検索'] -sidebar_label: '厳密および近似ベクトル検索' -slug: /engines/table-engines/mergetree-family/annindexes -title: '厳密および近似ベクトル検索' -doc_type: 'guide' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# 厳密ベクトル検索と近似ベクトル検索 {#exact-and-approximate-vector-search} - -多次元(ベクトル)空間において、ある点に最も近い N 個の点を見つける問題は、[nearest neighbor search](https://en.wikipedia.org/wiki/Nearest_neighbor_search)(最近傍探索)、または略してベクトル検索と呼ばれます。 -ベクトル検索を行うための一般的なアプローチは 2 つあります: - -* 厳密ベクトル検索は、与えられた点とベクトル空間内のすべての点との距離を計算します。これにより可能な限り最高の精度が保証され、返される点は必ず実際の最近傍になります。ベクトル空間を総当たりで探索するため、厳密ベクトル検索は実運用では遅くなり過ぎる場合があります。 -* 近似ベクトル検索は、一連の手法(例えばグラフやランダムフォレストといった特殊なデータ構造)を指し、厳密ベクトル検索よりもはるかに高速に結果を計算します。結果の精度は通常、実用上「十分良い」レベルです。多くの近似手法は、結果の精度と検索時間のトレードオフを調整するためのパラメータを提供します。 - -ベクトル検索(厳密または近似)は、次のように SQL で記述できます: - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- WHERE 句は任意です -ORDER BY (vectors, reference_vector) -LIMIT -``` - -ベクトル空間内の点は、配列型の `vectors` 列に格納されます。例えば [Array(Float64)](../../../sql-reference/data-types/array.md)、[Array(Float32)](../../../sql-reference/data-types/array.md)、または [Array(BFloat16)](../../../sql-reference/data-types/array.md) です。 -参照ベクトルは定数配列であり、共通テーブル式として与えられます。 -`<DistanceFunction>` は、参照点と格納されているすべての点との距離を計算します。 -この処理には、利用可能な任意の [distance function](/sql-reference/functions/distance-functions) を使用できます。 -`<N>` は、返すべき近傍点の数を指定します。 - - -## 厳密なベクトル検索 {#exact-nearest-neighbor-search} - -厳密なベクトル検索は、上記の SELECT クエリをそのまま使用して実行できます。 -このようなクエリの実行時間は、一般的に保存されているベクトル数とその次元数、つまり配列要素数に比例します。 -また、ClickHouse はすべてのベクトルに対して総当たりスキャンを行うため、クエリで使用されるスレッド数(設定項目 [max_threads](../../../operations/settings/settings.md#max_threads) を参照)にも実行時間が依存します。 - -### 例 {#exact-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -戻り値 - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - - -## 近似ベクトル検索 {#approximate-nearest-neighbor-search} - -### ベクトル類似度インデックス {#vector-similarity-index} - -ClickHouse は、近似ベクトル検索を実行するための特別な「ベクトル類似度」インデックスを提供します。 - -:::note -ベクトル類似度インデックスは ClickHouse バージョン 25.8 以降で利用可能です。 -問題が発生した場合は、[ClickHouse リポジトリ](https://github.com/clickhouse/clickhouse/issues) に issue を作成してください。 -::: - -#### ベクトル類似度インデックスの作成 {#creating-a-vector-similarity-index} - -新しいテーブルに対して、次のようにベクトル類似度インデックスを作成できます。 - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -別の方法として、既存のテーブルにベクトル類似インデックスを追加するには次のようにします。 - -```sql -ALTER TABLE table ADD INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ]; -``` - -ベクトル類似性インデックスは特別な種類のスキッピングインデックスにあたります([こちら](mergetree.md#table_engine-mergetree-data_skipping-indexes)および[こちら](../../../optimize/skipping-indexes)を参照してください)。 -したがって、上記の `ALTER TABLE` 文では、テーブルに今後挿入される新規データに対してのみインデックスが作成されます。 -既存データに対してもインデックスを構築するには、インデックスをマテリアライズする必要があります。 - -```sql -ALTER TABLE table MATERIALIZE INDEX SETTINGS mutations_sync = 2; -``` - -関数 `` には、次のいずれかを指定する必要があります。 - -* `L2Distance` — [ユークリッド距離](https://en.wikipedia.org/wiki/Euclidean_distance)。ユークリッド空間における 2 点間を結ぶ線分の長さを表します。 -* `cosineDistance` — [コサイン距離](https://en.wikipedia.org/wiki/Cosine_similarity#Cosine_distance)。ゼロではない 2 つのベクトル間の角度を表します。 - -正規化済みデータに対しては、通常 `L2Distance` が最適な選択肢です。それ以外の場合はスケールの違いを補正するために `cosineDistance` を推奨します。 - -`` は、基礎となるカラムにおける配列のカーディナリティ(要素数)を指定します。 -ClickHouse がインデックス作成中に異なるカーディナリティの配列を検出した場合、そのインデックスは破棄され、エラーが返されます。 - -オプションの GRANULARITY パラメータ `` は、インデックスグラニュールのサイズを表します([こちら](../../../optimize/skipping-indexes)を参照)。 -デフォルト値の 1 億は、ほとんどのユースケースで十分に良好に動作しますが、チューニングすることも可能です。 -その影響を理解している上級ユーザーのみがチューニングすることをお勧めします([下記](#differences-to-regular-skipping-indexes)を参照)。 - -ベクトル類似度インデックスは汎用的であり、さまざまな近似検索手法を利用できます。 -実際に使用される手法は、パラメータ `` によって指定されます。 -現時点で利用可能な手法は HNSW([論文](https://arxiv.org/abs/1603.09320))のみであり、階層的近接グラフに基づく、近似ベクトル検索のための一般的かつ最先端の手法です。 -`type` として HNSW を使用する場合、ユーザーは任意で HNSW 固有の追加パラメータを指定できます。 - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX index_name vectors TYPE vector_similarity('hnsw', , [, , , ]) [GRANULARITY N] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -これらの HNSW 固有パラメータが利用可能です: - -* `` は近傍グラフ内のベクトルの量子化方式を制御します。指定可能な値は `f64`、`f32`、`f16`、`bf16`、`i8`、`b1` です。デフォルト値は `bf16` です。このパラメータは、基盤となるカラム内でのベクトル表現には影響しない点に注意してください。 -* `` は、グラフ内の各ノードあたりの近傍ノード数を制御します。これは HNSW のハイパーパラメータ `M` としても知られています。デフォルト値は `32` です。値 `0` はデフォルト値を使用することを意味します。 -* `` は、HNSW グラフ構築時の動的候補リストのサイズを制御します。これは HNSW のハイパーパラメータ `ef_construction` としても知られています。デフォルト値は `128` です。値 `0` はデフォルト値を使用することを意味します。 - -すべての HNSW 固有パラメータのデフォルト値は、ほとんどのユースケースで良好に機能します。 -したがって、HNSW 固有パラメータのカスタマイズは推奨しません。 - - -さらなる制限があります: - -* ベクター類似度インデックスは、[Array(Float32)](../../../sql-reference/data-types/array.md)、[Array(Float64)](../../../sql-reference/data-types/array.md)、または [Array(BFloat16)](../../../sql-reference/data-types/array.md) 型の列に対してのみ作成できます。`Array(Nullable(Float32))` や `Array(LowCardinality(Float32))` のような nullable や low-cardinality の浮動小数点数配列は使用できません。 -* ベクター類似度インデックスは単一列に対してのみ作成しなければなりません。 -* ベクター類似度インデックスは計算式(例: `INDEX index_name arraySort(vectors) TYPE vector_similarity([...])`)に対して作成することもできますが、そのようなインデックスは後で近似近傍探索に利用することはできません。 -* ベクター類似度インデックスでは、基になる列中のすべての配列が `` 個の要素を持つ必要があります。この条件はインデックス作成時に検査されます。この要件違反をできるだけ早期に検出するために、ユーザーはベクター列に対して [constraint](/sql-reference/statements/create/table.md#constraints) を追加できます(例: `CONSTRAINT same_length CHECK length(vectors) = 256`)。 -* 同様に、基になる列中の配列値は空 (`[]`) であってはならず、デフォルト値(同じく `[]`)であってもいけません。 - -**ストレージおよびメモリ消費量の見積もり** - -典型的な AI モデル(例: 大規模言語モデル [LLM](https://en.wikipedia.org/wiki/Large_language_model))で使用するために生成されるベクターは、数百から数千の浮動小数点値で構成されます。 -そのため、単一のベクター値でも複数キロバイトのメモリを消費する可能性があります。 -テーブル内の基になるベクター列に必要なストレージ量、およびベクター類似度インデックスに必要なメインメモリ量を見積もりたい場合は、以下の 2 つの式を利用できます。 - -テーブル内のベクター列のストレージ使用量(非圧縮): - -```text -ストレージ消費量 = ベクトル数 × 次元数 × カラムデータ型のサイズ -``` - -[dbpedia データセット](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) の例: - -```text -ストレージ消費量 = 100万 × 1536 × 4(Float32の場合)= 6.1 GB -``` - -ベクトル類似度インデックスで検索を行うには、ディスクから主メモリに完全に読み込まれている必要があります。 -同様に、ベクトルインデックスもメモリ上で完全に構築してからディスクに保存されます。 - -ベクトルインデックスをロードする際に必要なメモリ使用量: - -```text -インデックス内のベクトル用メモリ (mv) = ベクトル数 * 次元数 * 量子化データ型のサイズ -インメモリグラフ用メモリ (mg) = ベクトル数 * hnsw_max_connections_per_layer * ノードIDあたりのバイト数 (= 4) * レイヤーノード繰り返し係数 (= 2) - -メモリ消費量: mv + mg -``` - -[DBpedia データセット](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) の例: - -```text -インデックス内のベクトル用メモリ (mv) = 100万 × 1536 × 2 (BFloat16の場合) = 3072 MB -インメモリグラフ用メモリ (mg) = 100万 × 64 × 2 × 4 = 512 MB - -メモリ消費量 = 3072 + 512 = 3584 MB -``` - -上記の式には、事前割り当てバッファやキャッシュなど、ベクトル類似性インデックスがランタイムのデータ構造を割り当てるために必要となる追加メモリは含まれていません。 - -#### ベクトル類似性インデックスの使用 {#using-a-vector-similarity-index} - -:::note -ベクトル類似性インデックスを使用するには、[compatibility](../../../operations/settings/settings.md) 設定を `''`(デフォルト値)、または `'25.1'` 以降に設定する必要があります。 -::: - -ベクトル類似性インデックスは、次の形式の SELECT クエリをサポートします。 - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- WHERE 句は任意 -ORDER BY (vectors, reference_vector) -LIMIT -``` - -ClickHouse のクエリオプティマイザは、上記のクエリテンプレートと照合し、利用可能なベクトル類似度インデックスを活用します。 -SELECT クエリ内の距離関数がインデックス定義で指定されている距離関数と同じである場合にのみ、そのクエリはベクトル類似度インデックスを使用できます。 - -上級ユーザーは、検索時の候補リストのサイズを調整するために、[hnsw_candidate_list_size_for_search](../../../operations/settings/settings.md#hnsw_candidate_list_size_for_search)(HNSW のハイパーパラメータ「ef_search」としても知られる)に任意の値を設定できます(例: `SELECT [...] SETTINGS hnsw_candidate_list_size_for_search = `)。 -この設定のデフォルト値である 256 は、ほとんどのユースケースで良好に機能します。 -この値を大きくすると精度は向上しますが、その分パフォーマンスが低下します。 - - -クエリでベクター類似性インデックスを使用する場合、ClickHouse は SELECT クエリで指定された LIMIT `` が妥当な範囲内かどうかをチェックします。 -より具体的には、`` が設定値 [max_limit_for_vector_search_queries](../../../operations/settings/settings.md#max_limit_for_vector_search_queries)(デフォルト値は 100)より大きい場合はエラーが返されます。 -LIMIT の値が大きすぎると検索が遅くなり、通常は誤った使用方法であることを示しています。 - -SELECT クエリがベクター類似性インデックスを使用しているかどうかを確認するには、クエリの先頭に `EXPLAIN indexes = 1` を付けて実行します。 - -例として、次のクエリを示します。 - -```sql -EXPLAIN indexes = 1 -WITH [0.462, 0.084, ..., -0.110] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 10; -``` - -返す場合があります - -```result - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (プロジェクト名) │ - 2. │ Limit (予備LIMIT (OFFSETなし)) │ - 3. │ Sorting (ORDER BY用のソート) │ - 4. │ Expression ((ORDER BY前 + (射影 + 列名を列識別子に変更))) │ - 5. │ ReadFromMergeTree (default.tab) │ - 6. │ Indexes: │ - 7. │ PrimaryKey │ - 8. │ Condition: true │ - 9. │ Parts: 1/1 │ -10. │ Granules: 575/575 │ -11. │ Skip │ -12. │ Name: idx │ -13. │ Description: vector_similarity GRANULARITY 100000000 │ -14. │ Parts: 1/1 │ -15. │ Granules: 10/575 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -この例では、[dbpedia dataset](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) の 100 万個のベクトル(各ベクトルの次元は 1536)が 575 個の granule(グラニュール)に格納されており、つまり 1 granule あたり約 1.7k 行となります。 -クエリは 10 個の近傍を要求し、ベクトル類似度インデックスはこれら 10 個の近傍を 10 個の別々の granule から見つけます。 -クエリ実行中には、これら 10 個の granule が読み込まれます。 - -出力に `Skip` とベクトルインデックスの名前と型(この例では `idx` および `vector_similarity`)が含まれている場合、ベクトル類似度インデックスが使用されています。 -このケースでは、ベクトル類似度インデックスによって 4 個の granule のうち 2 個がスキップされ、つまりデータの 50% が読み取り対象から外れました。 -スキップできる granule が多いほど、インデックスの利用はより効果的になります。 - -:::tip -インデックスの使用を強制するには、設定 [force_data_skipping_indexes](../../../operations/settings/settings#force_data_skipping_indices) を指定して SELECT クエリを実行します(インデックス名を設定値として指定します)。 -::: - -**Post-filtering と Pre-filtering** - -ユーザーはオプションで、SELECT クエリに対して追加のフィルター条件を含む `WHERE` 句を指定できます。 -ClickHouse は、post-filtering あるいは pre-filtering 戦略を用いてこれらのフィルター条件を評価します。 -要するに、どちらの戦略もフィルターを評価する順序を決定します。 - -* Post-filtering とは、まずベクトル類似度インデックスが評価され、その後に ClickHouse が `WHERE` 句で指定された追加フィルターを評価することを意味します。 -* Pre-filtering とは、フィルター評価の順序がその逆になることを意味します。 - -これらの戦略には、それぞれ異なるトレードオフがあります。 - -* Post-filtering には、`LIMIT ` 句で要求された行数より少ない結果しか返せない可能性があるという一般的な問題があります。これは、ベクトル類似度インデックスから返された 1 行以上の結果行が、追加フィルターを満たさない場合に発生します。 -* Pre-filtering は、一般的には未解決の問題です。一部の特化したベクトルデータベースは pre-filtering アルゴリズムを提供していますが、ほとんどのリレーショナルデータベース(ClickHouse を含む)は、インデックスを使わない総当たり走査、すなわち厳密な近傍探索へフォールバックします。 - -どの戦略が使用されるかは、フィルター条件に依存します。 - -*追加フィルターがパーティションキーの一部である場合* - -追加のフィルター条件がパーティションキーの一部である場合、ClickHouse はパーティションプルーニングを適用します。 -例として、テーブルが列 `year` によってレンジパーティションされており、次のクエリが実行されるとします。 - -```sql -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -WHERE year = 2025 -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -ClickHouse は、2025 年のパーティション以外をすべてプルーニングします。 - -*追加フィルタはインデックスを使って評価できない場合があります* - -追加のフィルタ条件がインデックス(PRIMARY KEY インデックス、スキッピングインデックス)を使って評価できない場合、ClickHouse はポストフィルタリングを適用します。 - - -*追加のフィルターはプライマリキーインデックスを用いて評価できる* - -追加のフィルター条件が [プライマリキー](mergetree.md#primary-key) を用いて評価可能な場合(すなわち、プライマリキーのプレフィックスを構成している場合)、かつ - -* フィルター条件がパーツ内で少なくとも 1 行を除外できる場合、ClickHouse はそのパーツ内の「生き残る」範囲に対してプレフィルタリングに切り替えます。 -* フィルター条件がパーツ内で 1 行も除外しない場合、ClickHouse はそのパーツに対してポストフィルタリングを実行します。 - -実際のユースケースでは、後者のケースが発生することはほとんどありません。 - -*追加のフィルターは skipping index を用いて評価できる* - -追加のフィルター条件が [skipping indexes](mergetree.md#table_engine-mergetree-data_skipping-indexes)(minmax index、set index など)を用いて評価可能な場合、ClickHouse はポストフィルタリングを実行します。 -このような場合、他の skipping index と比較して最も多くの行を除外できると期待されるため、まずベクトル類似度インデックスが評価されます。 - -ポストフィルタリングとプレフィルタリングをより細かく制御するために、2 つの設定を使用できます。 - -[vector_search_filter_strategy](../../../operations/settings/settings#vector_search_filter_strategy) 設定(デフォルト: 上記のヒューリスティクスを実装する `auto`)は `prefilter` に設定できます。 -これは、追加のフィルター条件が非常に選択的な場合に、プレフィルタリングを強制したいケースで有用です。 -例として、次のクエリはプレフィルタリングの恩恵を受ける可能性があります: - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('古代アジアの帝国に関する書籍')) -LIMIT 10 -``` - -2ドル未満の本がごく少数しか存在しないと仮定すると、ベクターインデックスから返される上位10件の結果がすべて2ドルより高い価格である可能性があるため、ポストフィルタリングでは行が0件になることがあります。 -プリフィルタリングを強制することで(クエリに `SETTINGS vector_search_filter_strategy = 'prefilter'` を追加)、ClickHouse はまず価格が2ドル未満のすべての本を検索し、その後、それらの本に対して総当たりのベクター検索を実行します。 - -上記の問題を解決する別のアプローチとして、[vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier)(デフォルト: `1.0`、最大: `1000.0`)を `1.0` より大きい値(たとえば `2.0`)に設定することができます。 -ベクターインデックスから取得される最近傍の数は、この設定値を掛けた数となり、その後、それらの行に対して追加のフィルタを適用して、LIMIT で指定された件数の行を返します。 -例として、乗数を `3.0` にして再度クエリを実行できます。 - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('Books on ancient Asian empires')) -LIMIT 10 -SETTING vector_search_index_fetch_multiplier = 3.0; -``` - -ClickHouse は各パーツのベクターインデックスから 3.0 x 10 = 30 個の最近傍を取得し、その後に追加のフィルタを評価します。 -最も近い 10 個の近傍だけが返されます。 -`vector_search_index_fetch_multiplier` を設定することでこの問題を軽減できることに注意してください。 -しかし、極端なケース(非常に選択性の高い WHERE 条件)では、要求された行数 N 未満の行しか返されない可能性があります。 - -**リスコアリング(再スコアリング)** - -ClickHouse のスキップインデックスは一般的にグラニュールレベルでフィルタリングします。つまり、スキップインデックスでのルックアップは(内部的に)一致する可能性のあるグラニュールのリストを返し、その後のスキャンで読み取るデータ量を削減します。 -これはスキップインデックス全般ではうまく機能しますが、ベクター類似性インデックスの場合には「粒度のミスマッチ」を引き起こします。 -詳しく言うと、ベクター類似性インデックスは、ある参照ベクターに対して最も類似した N 個のベクターの行番号を決定しますが、その後これらの行番号をグラニュール番号に外挿する必要があります。 -その後 ClickHouse はこれらのグラニュールをディスクから読み込み、これらのグラニュール内のすべてのベクターに対して距離計算を繰り返します。 -このステップはリスコアリングと呼ばれます。理論的には精度を向上させることができます(ベクター類似性インデックスはあくまで*近似的な*結果しか返さないことを思い出してください)が、パフォーマンスの観点から最適ではないことは明らかです。 - -そのため ClickHouse では、リスコアリングを無効化し、インデックスから最も類似したベクターとその距離を直接返す最適化を提供しています。 -この最適化はデフォルトで有効になっており、設定 [vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring) を参照してください。 -概要としては、ClickHouse が最も類似したベクターとその距離を仮想カラム `_distances` として利用可能にします。 -これを確認するには、`EXPLAIN header = 1` を付けてベクター検索クエリを実行します。 - - -```sql -EXPLAIN header = 1 -WITH [0., 2.] AS reference_vec -SELECT id -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3 -SETTINGS vector_search_with_rescoring = 0 -``` - -```result -Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 - - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (名前の射影) │ - 2. │ Header: id Int32 │ - 3. │ Limit (予備的LIMIT(OFFSETなし)) │ - 4. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 5. │ __table1.id Int32 │ - 6. │ Sorting (ORDER BYのソート) │ - 7. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 8. │ __table1.id Int32 │ - 9. │ Expression ((ORDER BY前 + (射影 + カラム名のカラム識別子への変更))) │ -10. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ -11. │ __table1.id Int32 │ -12. │ ReadFromMergeTree (default.tab) │ -13. │ Header: id Int32 │ -14. │ _distance Float32 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -:::note -`vector_search_with_rescoring = 0` で再スコアリングなしのクエリを実行し、かつ並列レプリカを有効にしている場合、再スコアリングにフォールバックして実行されることがあります。 -::: - -#### パフォーマンスチューニング {#performance-tuning} - -**圧縮のチューニング** - -ほぼすべてのユースケースで、基盤となる列内のベクトルは高密度であり、圧縮が効きにくくなります。 -その結果、[圧縮](/sql-reference/statements/create/table.md#column_compression_codec) はベクトル列への挿入およびベクトル列からの読み取りを遅くします。 -そのため、圧縮を無効にすることを推奨します。 -そのためには、ベクトル列に対して次のように `CODEC(NONE)` を指定します。 - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32) CODEC(NONE), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; -``` - -**インデックス作成のチューニング** - -ベクトル類似度インデックスのライフサイクルは、パーツのライフサイクルに結び付いています。 -言い換えると、ベクトル類似度インデックスが定義された新しいパーツが作成されるたびに、インデックスも作成されます。 -これは通常、データが[挿入](https://clickhouse.com/docs/guides/inserting-data)されたとき、または[マージ](https://clickhouse.com/docs/merges)中に発生します。 -残念ながら、HNSW はインデックス作成時間が長いことで知られており、INSERT やマージを大幅に遅くする可能性があります。 -ベクトル類似度インデックスは、データが不変、またはほとんど変更されない場合にのみ使用するのが理想的です。 - -インデックス作成を高速化するために、次の手法を使用できます。 - -まず、インデックス作成は並列化できます。 -インデックス作成スレッドの最大数は、サーバー設定 [max_build_vector_similarity_index_thread_pool_size](/operations/server-configuration-parameters/settings#max_build_vector_similarity_index_thread_pool_size) を使用して設定できます。 -最適なパフォーマンスのためには、この設定値を CPU コア数に合わせて設定することを推奨します。 - -次に、INSERT 文を高速化するために、セッション設定 [materialize_skip_indexes_on_insert](../../../operations/settings/settings.md#materialize_skip_indexes_on_insert) を使用して、新しく挿入されたパーツに対するスキップインデックスの作成を無効にすることができます。 -そのようなパーツに対する SELECT クエリは、厳密検索にフォールバックします。 -挿入されたパーツは、テーブル全体のサイズと比較して小さい傾向があるため、そのことによるパフォーマンスへの影響は無視できると想定されます。 - -第 3 に、マージを高速化するために、セッション設定 [materialize_skip_indexes_on_merge](../../../operations/settings/merge-tree-settings.md#materialize_skip_indexes_on_merge) を使用して、マージされたパーツに対するスキップインデックスの作成を無効にすることができます。 -これは、ステートメント [ALTER TABLE [...] MATERIALIZE INDEX [...]](../../../sql-reference/statements/alter/skipping-index.md#materialize-index) と組み合わせることで、ベクトル類似度インデックスのライフサイクルを明示的に制御する手段を提供します。 -たとえば、インデックス作成を、すべてのデータが取り込まれるまで、あるいは週末のようなシステム負荷の低い期間まで延期することができます。 - -**インデックス使用のチューニング** - - -SELECT クエリでベクトル類似性インデックスを使用するには、それらをメインメモリにロードする必要があります。 -同じベクトル類似性インデックスが繰り返しメインメモリにロードされることを避けるため、ClickHouse はそのようなインデックス用の専用インメモリキャッシュを提供しています。 -このキャッシュが大きいほど、不要なロードの回数は少なくなります。 -キャッシュの最大サイズは、サーバー設定 [vector_similarity_index_cache_size](../../../operations/server-configuration-parameters/settings.md#vector_similarity_index_cache_size) で設定できます。 -デフォルトでは、このキャッシュは最大 5 GB まで拡張できます。 - -:::note -ベクトル類似性インデックスキャッシュには、ベクトルインデックスのグラニュールが保存されます。 -個々のベクトルインデックスグラニュールがキャッシュサイズより大きい場合、それらはキャッシュされません。 -そのため、「Estimating storage and memory consumption」の式、または [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) に基づいてベクトルインデックスサイズを算出し、それに応じてキャッシュサイズを設定してください。 -::: - -ベクトル類似性インデックスキャッシュの現在のサイズは、[system.metrics](../../../operations/system-tables/metrics.md) に表示されます。 - -```sql -SELECT metric, value -FROM system.metrics -WHERE metric = 'VectorSimilarityIndexCacheBytes' -``` - -特定のクエリ ID に対応するクエリのキャッシュヒットおよびミスは、[system.query_log](../../../operations/system-tables/query_log.md) から取得できます。 - -```sql -SYSTEM FLUSH LOGS query_log; - -SELECT ProfileEvents['VectorSimilarityIndexCacheHits'], ProfileEvents['VectorSimilarityIndexCacheMisses'] -FROM system.query_log -WHERE type = 'QueryFinish' AND query_id = '<...>' -ORDER BY event_time_microseconds; -``` - -本番環境でのユースケースでは、すべてのベクトルインデックスが常にメモリ上に保持されるよう、十分な容量のキャッシュを確保することを推奨します。 - -**量子化のチューニング** - -[量子化](https://huggingface.co/blog/embedding-quantization) は、ベクトルのメモリ使用量と、ベクトルインデックスの構築および走査にかかる計算コストを削減するための手法です。 -ClickHouse のベクトルインデックスは、次の量子化オプションをサポートします: - -| Quantization | Name | Storage per dimension | -| -------------- | ----------------- | --------------------- | -| f32 | 単精度 | 4 bytes | -| f16 | 半精度 | 2 bytes | -| bf16 (default) | 半精度 (brain float) | 2 bytes | -| i8 | 4分の1精度 | 1 byte | -| b1 | バイナリ | 1 bit | - -量子化は、元のフル精度の浮動小数点値 (`f32`) に対する検索と比較して、ベクトル検索の精度を低下させます。 -しかし、ほとんどのデータセットでは、半精度 brain float 量子化 (`bf16`) による精度低下はごくわずかであるため、ベクトル類似度インデックスではこの量子化手法がデフォルトで使用されます。 -4分の1精度 (`i8`) およびバイナリ (`b1`) 量子化は、ベクトル検索において無視できない精度低下を引き起こします。 -これら 2 つの量子化は、ベクトル類似度インデックスのサイズが、利用可能な DRAM 容量を大きく上回る場合にのみ推奨されます。 -このような場合は、精度向上のために再スコアリング([vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier)、[vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring))を有効にすることも推奨します。 -バイナリ量子化は、1) 正規化済み埋め込み(すなわちベクトル長 = 1、OpenAI のモデルは通常正規化されている)、かつ 2) 距離関数としてコサイン距離を使用する場合にのみ推奨されます。 -バイナリ量子化は内部的に Hamming 距離を用いて近傍グラフを構築および検索します。 -再スコアリングのステップでは、テーブルに保存されている元のフル精度ベクトルを用いて、コサイン距離により最近傍を特定します。 - -**データ転送のチューニング** - -ベクトル検索クエリの参照ベクトルはユーザーによって提供され、一般的には大規模言語モデル(LLM)を呼び出して取得されます。 -ClickHouse でベクトル検索を実行する典型的な Python コードは、次のようになります。 - -```python -search_v = openai_client.embeddings.create(input = "[Good Books]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'search_v': search_v} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, %(search_v)s) - LIMIT 10", - parameters = params) -``` - - -埋め込みベクトル(上記スニペットの `search_v`)は、非常に大きな次元数を持つ場合があります。 -たとえば、OpenAI は 1536 次元や 3072 次元の埋め込みベクトルを生成するモデルを提供しています。 -上記のコードでは、ClickHouse Python ドライバは埋め込みベクトルを人間が読める文字列に置き換え、その後 SELECT クエリ全体を文字列として送信します。 -埋め込みベクトルが 1536 個の単精度浮動小数点値で構成されているとすると、送信される文字列は長さが 20 kB に達します。 -これにより、トークナイズ、パース、および数千回におよぶ文字列から浮動小数点への変換で CPU 使用率が高くなります。 -また、ClickHouse サーバーログファイルにも多くのディスク容量が必要となり、`system.query_log` の肥大化も引き起こします。 - -ほとんどの LLM モデルは、埋め込みベクトルをネイティブな float のリストまたは NumPy 配列として返すことに注意してください。 -そのため、Python アプリケーションでは、参照ベクトルのパラメータを次のようなスタイルでバイナリ形式としてバインドすることを推奨します。 - -```python -search_v = openai_client.embeddings.create(input = "[Good Books]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'$search_v_binary$': np.array(search_v, dtype=np.float32).tobytes()} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, (SELECT reinterpret($search_v_binary$, 'Array(Float32)'))) - LIMIT 10" - parameters = params) -``` - -この例では、参照ベクターはそのままバイナリ形式で送信され、サーバー側で浮動小数点数配列として再解釈されます。 -これにより、サーバー側の CPU 時間を節約し、サーバーログおよび `system.query_log` の肥大化を防ぐことができます。 - -#### 管理と監視 {#administration} - -ベクトル類似性インデックスのディスク上のサイズは、[system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) から取得できます。 - -```sql -SELECT database, table, name, formatReadableSize(data_compressed_bytes) -FROM system.data_skipping_indices -WHERE type = 'vector_similarity'; -``` - -出力例: - -```result -┌─database─┬─table─┬─name─┬─formatReadab⋯ssed_bytes)─┐ -│ default │ tab │ idx │ 348.00 MB │ -└──────────┴───────┴──────┴──────────────────────────┘ -``` - -#### 通常のスキッピングインデックスとの違い {#differences-to-regular-skipping-indexes} - -通常の[スキッピングインデックス](/optimize/skipping-indexes)と同様に、ベクトル類似度インデックスはグラニュール単位で構築され、各インデックスブロックは `GRANULARITY = [N]` 個のグラニュールで構成されます(通常のスキッピングインデックスではデフォルトで `[N]` = 1)。 -たとえば、テーブルのプライマリインデックスのグラニュラリティが 8192(`index_granularity = 8192` の設定)で `GRANULARITY = 2` の場合、各インデックスブロックには 16384 行が含まれます。 -しかし、おおよその近傍検索のためのデータ構造とアルゴリズムは、本質的に行指向です。 -これらは行の集合をコンパクトに表現して保存し、ベクトル検索クエリに対しても行を返します。 -このため、ベクトル類似度インデックスの挙動は、通常のスキッピングインデックスと比べて、いくつか直感的ではない違いが生じます。 - -ユーザーがあるカラムにベクトル類似度インデックスを定義すると、ClickHouse は内部的に各インデックスブロックごとにベクトル類似度の「サブインデックス」を作成します。 -サブインデックスは、そのインデックスブロックに含まれる行だけを認識しているという意味で「ローカル」です。 -前述の例で、あるカラムが 65536 行を持つと仮定すると、8 個のグラニュールにまたがる 4 つのインデックスブロックと、それぞれのインデックスブロックに対応するベクトル類似度サブインデックスが得られます。 -サブインデックスは理論的には、そのインデックスブロック内で最も近い N 個の点に対応する行を直接返すことができます。 -しかし、ClickHouse はディスクからメモリへデータを読み込む際にグラニュールの粒度で処理するため、サブインデックスはマッチした行をグラニュール単位にまで拡張して扱います。 -これは、インデックスブロックの粒度でデータをスキップする通常のスキッピングインデックスとは異なります。 - - -`GRANULARITY` パラメータは、いくつのベクトル類似度サブインデックスを作成するかを決定します。 -より大きな `GRANULARITY` の値では、サブインデックスの数は少なくなる一方、それぞれのサブインデックスはより大きくなり、最終的にはあるカラム(またはカラムのデータパーツ)が単一のサブインデックスしか持たない状態になります。 -その場合、そのサブインデックスはカラム行全体に対する「グローバル」なビューを持ち、関連する行を含むカラム(パーツ)のすべてのグラニュールを直接返すことができます(そのようなグラニュールは最大でも `LIMIT [N]` 個です)。 -次のステップとして、ClickHouse はこれらのグラニュールを読み込み、グラニュール内のすべての行に対して総当たりの距離計算を行うことで、実際に最適な行を特定します。 -`GRANULARITY` の値が小さい場合、それぞれのサブインデックスは最大 `LIMIT N` 個のグラニュールを返します。 -その結果、より多くのグラニュールを読み込んで事後フィルタリングする必要が生じます。 -どちらの場合でも検索精度は同等であり、異なるのは処理パフォーマンスのみであることに注意してください。 -一般的には、ベクトル類似度インデックスに対しては大きな `GRANULARITY` を使用し、ベクトル類似度構造によるメモリ消費が過大になるなどの問題が発生した場合にのみ、より小さな `GRANULARITY` の値に切り替えることを推奨します。 -ベクトル類似度インデックスに対して `GRANULARITY` が指定されていない場合、デフォルト値は 1 億です。 - -#### 例 {#approximate-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -戻り値 - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - -近似ベクトル検索を利用する他のサンプルデータセット: - -* [LAION-400M](../../../getting-started/example-datasets/laion-400m-dataset) -* [LAION-5B](../../../getting-started/example-datasets/laion-5b-dataset) -* [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) -* [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) - -### Quantized Bit (QBit) {#approximate-nearest-neighbor-search-qbit} - - - -厳密なベクトル検索を高速化する一般的な方法の 1 つは、低精度の [浮動小数点数型 (float data type)](../../../sql-reference/data-types/float.md) を使用することです。 -例えば、ベクトルを `Array(Float32)` ではなく `Array(BFloat16)` として保存すると、データサイズは半分になり、クエリ実行時間もそれに比例して短くなることが期待されます。 -この手法は量子化と呼ばれます。計算は高速になりますが、すべてのベクトルを完全走査していても、結果の精度は低下する可能性があります。 - -従来の量子化では、検索時とデータ保存時の両方で精度が失われます。上記の例では、`Float32` の代わりに `BFloat16` を保存することになり、たとえ望んだとしても、後からより高精度な検索を実行することはできません。別のアプローチとして、量子化済みデータとフル精度のデータという 2 つのコピーを保存する方法があります。これは有効ですが、冗長なストレージが必要になります。例えば、元データが `Float64` であり、異なる精度 (16 ビット、32 ビット、フル 64 ビット) で検索を実行したいシナリオを考えてみましょう。この場合、データのコピーを 3 つ別々に保存する必要があります。 - -ClickHouse は、これらの制約を解決する Quantized Bit (`QBit`) データ型を提供しており、次のことを実現します: - -1. 元のフル精度データを保存する。 -2. クエリ時に量子化精度を指定できるようにする。 - - -これは、データをビットグループ化形式(すべてのベクトルの i 番目のビットをまとめて保存する形式)で保存することで実現され、要求された精度レベルだけを読み出せるようにします。これにより、量子化による I/O と計算量の削減による高速化の恩恵を受けつつ、必要に応じて元のデータをすべて利用可能な状態に保てます。最大精度が選択された場合、検索は厳密なもの(完全一致)になります。 - -:::note -`QBit` データ型とそれに関連する距離関数は、現在実験的な機能です。これらを有効にするには、`SET allow_experimental_qbit_type = 1` を実行してください。 -問題が発生した場合は、[ClickHouse リポジトリ](https://github.com/clickhouse/clickhouse/issues) に Issue を作成してください。 -::: - -`QBit` 型のカラムを宣言するには、次の構文を使用します。 - -```sql -column_name QBit(element_type, dimension) -``` - -次のとおりです。 - -* `element_type` – 各ベクトル要素の型。サポートされている型は `BFloat16`、`Float32`、`Float64` です -* `dimension` – 各ベクトル内の要素数 - -#### `QBit` テーブルの作成とデータの追加 {#qbit-create} - -```sql -CREATE TABLE fruit_animal ( - word String, - vec QBit(Float64, 5) -) ENGINE = MergeTree -ORDER BY word; - -INSERT INTO fruit_animal VALUES - ('apple', [-0.99105519, 1.28887844, -0.43526649, -0.98520696, 0.66154391]), - ('banana', [-0.69372815, 0.25587061, -0.88226235, -2.54593015, 0.05300475]), - ('orange', [0.93338752, 2.06571317, -0.54612565, -1.51625717, 0.69775337]), - ('dog', [0.72138876, 1.55757105, 2.10953259, -0.33961248, -0.62217325]), - ('cat', [-0.56611276, 0.52267331, 1.27839863, -0.59809804, -1.26721048]), - ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); -``` - -#### `QBit` を使ったベクトル検索 {#qbit-search} - -L2 距離を用いて、単語 'lemon' を表すベクトルに最も近い近傍を検索します。距離関数の第 3 引数ではビット単位の精度を指定します。値を大きくすると精度は向上しますが、計算コストも増加します。 - -`QBit` で使用可能なすべての距離関数は[こちら](../../../sql-reference/data-types/qbit.md#vector-search-functions)で確認できます。 - -**フル精度検索 (64 ビット):** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 64) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬────────────distance─┐ -1. │ apple │ 0.14639757188169716 │ -2. │ banana │ 1.998961369007679 │ -3. │ orange │ 2.039041552613732 │ -4. │ cat │ 2.752802631487914 │ -5. │ horse │ 2.7555776805484813 │ -6. │ dog │ 3.382295083120104 │ - └────────┴─────────────────────┘ -``` - -**精度を下げた検索:** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 12) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬───────────distance─┐ -1. │ apple │ 0.757668703053566 │ -2. │ orange │ 1.5499475034938677 │ -3. │ banana │ 1.6168396735102937 │ -4. │ cat │ 2.429752230904804 │ -5. │ horse │ 2.524650475528617 │ -6. │ dog │ 3.17766975527459 │ - └────────┴────────────────────┘ -``` - - -12 ビット量子化では、クエリ実行が高速化されつつ、距離を良好に近似できることに注目してください。相対的な順位付けも概ね一貫しており、依然として「apple」が最も近い一致となっています。 - -:::note -現時点では、高速化の主な要因は、読み取るデータ量が減ることによる I/O の削減です。元のデータが `Float64` のようにビット幅の大きい型であっても、より低い精度を選択すると、同じ幅のデータに対して距離計算が行われますが、精度だけが下がります。 -::: - -#### パフォーマンス上の考慮事項 {#qbit-performance} - -`QBit` のパフォーマンス上の利点は、より低い精度を使用した場合に、ストレージから読み取る必要のあるデータ量が減ることによる I/O 操作の削減から生じます。さらに、`QBit` に格納されているデータが `Float32` であり、`precision` パラメータが 16 以下の場合は、計算量が減ることによる追加のメリットも得られます。`precision` パラメータは、精度と速度のトレードオフを直接制御します。 - -- **精度が高い場合**(元のデータ幅に近い場合): 結果はより正確だが、クエリは遅くなる -- **精度が低い場合**: 近似結果になるがクエリは高速になり、メモリ使用量も削減される - -### 参考文献 {#references} - -ブログ: -- [Vector Search with ClickHouse - Part 1](https://clickhouse.com/blog/vector-search-clickhouse-p1) -- [Vector Search with ClickHouse - Part 2](https://clickhouse.com/blog/vector-search-clickhouse-p2) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md deleted file mode 100644 index 4c81481dd5b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -description: 'CoalescingMergeTree は MergeTree エンジンを継承しています。その主な特徴は、パーツのマージ時に各列の直近の非 NULL 値を自動的に格納できることです。' -sidebar_label: 'CoalescingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/coalescingmergetree -title: 'CoalescingMergeTree テーブルエンジン' -keywords: ['CoalescingMergeTree'] -show_related_blogs: true -doc_type: 'reference' ---- - -# CoalescingMergeTree テーブルエンジン {#coalescingmergetree-table-engine} - -:::note Available from version 25.6 -このテーブルエンジンは、OSS と Cloud の両方でバージョン 25.6 以降で利用可能です。 -::: - -このエンジンは [MergeTree](/engines/table-engines/mergetree-family/mergetree) を継承しています。主な違いはデータパートのマージ方法です。`CoalescingMergeTree` テーブルでは、ClickHouse は同じ主キー(より正確には、同じ [ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md))を持つすべての行を、各カラムについて最新の非 NULL 値を含む 1 行に置き換えます。 - -これによりカラム単位のアップサートが可能になり、行全体ではなく特定のカラムだけを更新できます。 - -`CoalescingMergeTree` は、キー以外のカラムで Nullable 型と併用することを想定しています。カラムが Nullable でない場合は、その動作は [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree) と同じになります。 - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = CoalescingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -リクエストパラメータの説明については、[リクエストの説明](../../../sql-reference/statements/create/table.md)を参照してください。 - -### CoalescingMergeTree のパラメータ {#parameters-of-coalescingmergetree} - -#### Columns {#columns} - -`columns` - 値が統合されるカラム名のタプルです。省略可能なパラメータです。\ -カラムは数値型である必要があり、パーティションキーまたはソートキーに含まれていてはなりません。 - -`columns` が指定されていない場合、ClickHouse はソートキーに含まれていないすべてのカラムの値を統合します。 - -### クエリ句 {#query-clauses} - -`CoalescingMergeTree` テーブルを作成する際には、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) が必要です。 - -
- 非推奨のテーブル作成方法 - - :::note - 新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上で説明した方法に切り替えてください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] CoalescingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - `columns` を除くすべてのパラメータは、`MergeTree` における意味と同じです。 - - * `columns` — 値が合計されるカラム名のタプルです。省略可能なパラメータです。詳細な説明については上記のテキストを参照してください。 -
- -## 使用例 {#usage-example} - -次のテーブルを例にします。 - -```sql -CREATE TABLE test_table -( - key UInt64, - value_int Nullable(UInt32), - value_string Nullable(String), - value_date Nullable(Date) -) -ENGINE = CoalescingMergeTree() -ORDER BY key -``` - -データを挿入する: - -```sql -INSERT INTO test_table VALUES(1, NULL, NULL, '2025-01-01'), (2, 10, 'test', NULL); -INSERT INTO test_table VALUES(1, 42, 'win', '2025-02-01'); -INSERT INTO test_table(key, value_date) VALUES(2, '2025-02-01'); -``` - -結果は次のようになります。 - -```sql -SELECT * FROM test_table ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 1 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-01-01 │ -│ 2 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-02-01 │ -│ 2 │ 10 │ test │ ᴺᵁᴸᴸ │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -最終的な正しい結果を得るための推奨クエリ: - -```sql -SELECT * FROM test_table FINAL ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 2 │ 10 │ test │ 2025-02-01 │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -`FINAL` 修飾子を使用すると、クエリ実行時に ClickHouse がマージロジックを適用し、各カラムごとに正しい統合後の「最新」値を必ず取得できます。これは、CoalescingMergeTree テーブルに対してクエリを実行する際に、最も安全で精度の高い方法です。 - -:::note - -`GROUP BY` を用いるアプローチは、背後のパーツが完全にはマージされていない場合、誤った結果を返す可能性があります。 - -```sql -SELECT key, last_value(value_int), last_value(value_string), last_value(value_date) FROM test_table GROUP BY key; -- 非推奨です。 -``` - -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md deleted file mode 100644 index 709781c6480..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ /dev/null @@ -1,377 +0,0 @@ ---- -description: 'MergeTree をベースとしており、マージ処理中に行を折りたたむためのロジックを追加したものです。' -keywords: ['更新', '折りたたみ'] -sidebar_label: 'CollapsingMergeTree' -sidebar_position: 70 -slug: /engines/table-engines/mergetree-family/collapsingmergetree -title: 'CollapsingMergeTree テーブルエンジン' -doc_type: 'guide' ---- - - - -# CollapsingMergeTree テーブルエンジン {#collapsingmergetree-table-engine} - - - -## 説明 {#description} - -`CollapsingMergeTree` エンジンは [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) -を継承し、マージ処理中に行を折りたたむ(コラプスする)ためのロジックを追加します。 -`CollapsingMergeTree` テーブルエンジンは、特別なフィールド `Sign` を除くソートキー(`ORDER BY`)内のすべてのフィールドが等しく、 -かつ `Sign` フィールドの値が `1` または `-1` である行のペアを非同期に削除(折りたたみ)します。 -`Sign` が反対の値を持つ対応する行のペアが存在しない行は保持されます。 - -詳細については、このドキュメントの [Collapsing](#table_engine-collapsingmergetree-collapsing) セクションを参照してください。 - -:::note -このエンジンはストレージ容量を大幅に削減でき、 -その結果として `SELECT` クエリの効率を高めることができます。 -::: - - - -## パラメータ {#parameters} - -このテーブルエンジンのすべてのパラメータは、`Sign` パラメータを除き、 -[`MergeTree`](/engines/table-engines/mergetree-family/mergetree) における同名パラメータと同じ意味を持ちます。 - -- `Sign` — 行の種別を示す列に付ける名前で、`1` は「状態」行、`-1` は「取消」行を表します。型: [Int8](/sql-reference/data-types/int-uint)。 - - - -## テーブルの作成 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) -ENGINE = CollapsingMergeTree(Sign) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -
- テーブル作成の非推奨メソッド - - :::note - 以下のメソッドは新しいプロジェクトでの使用は推奨されません。 - 可能であれば、既存のプロジェクトを更新し、新しいメソッドを使用することを推奨します。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) - ENGINE [=] CollapsingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, Sign) - ``` - - `Sign` — `1` が「state」行、`-1` が「cancel」行を表す行種別を持つカラムに付ける名前です。[Int8](/sql-reference/data-types/int-uint)。 -
- -* クエリパラメータの説明については、[クエリの説明](../../../sql-reference/statements/create/table.md)を参照してください。 -* `CollapsingMergeTree` テーブルを作成する場合は、`MergeTree` テーブルを作成する場合と同じ [クエリ句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table) が必要です。 - - -## Collapsing {#table_engine-collapsingmergetree-collapsing} - -### Data {#data} - -あるオブジェクトに対して、継続的に変化するデータを保存する必要がある状況を考えます。 -各オブジェクトにつき 1 行だけを持ち、何か変更があるたびにその行を更新する、というのは論理的に思えますが、 -更新処理はストレージ上のデータを書き直す必要があるため、DBMS にとって高コストかつ低速です。 -データを高速に書き込む必要がある場合、大量の更新を行う方法は現実的ではありませんが、 -あるオブジェクトに対する変更を逐次的に書き込むことはいつでもできます。 -そのために、専用のカラム `Sign` を利用します。 - -* `Sign` = `1` の場合、その行は「状態」行を意味します:*現在の有効な状態を表すフィールドを含む行*。 -* `Sign` = `-1` の場合、その行は「キャンセル」行を意味します:*同じ属性を持つオブジェクトの状態をキャンセルするために使用される行*。 - -例えば、あるウェブサイトでユーザーが閲覧したページ数および滞在時間を計算したいとします。 -ある時点で、ユーザーアクティビティの状態を表す次の行を書き込みます。 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -後のタイミングでユーザーアクティビティの変化を検出し、次の 2 行として書き込みます。 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -最初の行は、(この場合はユーザーを表す)オブジェクトの以前の状態を打ち消します。 -この「canceled」行では、`Sign` を除くすべてのソートキーのフィールドをコピーする必要があります。 -その上の 2 行目が現在の状態を表しています。 - -ユーザーアクティビティの最後の状態だけが必要なため、元の「state」行と、挿入した「cancel」行は、次のように削除して、オブジェクトの無効(古い)状態を畳み込むことができます。 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -- 古い「状態」行は削除可能 -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -- 「キャンセル」行は削除可能 -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -- 新しい「状態」行は残る -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -`CollapsingMergeTree` は、データパーツのマージが行われる際に、まさにこの *collapsing(折りたたみ)* 動作を実行します。 - -:::note -各変更に対して行が 2 行必要となる理由については、 -[Algorithm](#table_engine-collapsingmergetree-collapsing-algorithm) の節でさらに詳しく説明します。 -::: - -**このアプローチ特有の注意点** - -1. データを書き込むプログラムは、取り消しを行えるように、オブジェクトの状態を保持しておく必要があります。「cancel」行には、「state」行のソートキー項目のコピーと、反対の `Sign` を含める必要があります。これにより初期のストレージサイズは増加しますが、データを高速に書き込めるようになります。 -2. カラム内で長く伸び続ける配列は、書き込み負荷の増大によりエンジンの効率を低下させます。データが単純であればあるほど効率は高くなります。 -3. `SELECT` の結果は、オブジェクト変更履歴の一貫性に大きく依存します。挿入用データを準備する際には注意してください。一貫性のないデータでは予測不能な結果が生じる可能性があります。たとえば、セッション深度のような非負のメトリクスに対して負の値が出力されることがあります。 - -### Algorithm {#table_engine-collapsingmergetree-collapsing-algorithm} - -ClickHouse がデータ[パーツ](/concepts/glossary#parts)をマージする際、 -同じソートキー(`ORDER BY`)を持つ連続した行の各グループは、高々 2 行にまでまとめられます。 -すなわち、`Sign` = `1` の「state」行と、`Sign` = `-1` の「cancel」行です。 -言い換えると、ClickHouse ではエントリが collapsing(折りたたみ)されます。 - - -各結果データパーツごとに、ClickHouse は次のように保存します。 - -| | | -|--|-------------------------------------------------------------------------------------------------------------------------------------| -|1.| 「state」行と「cancel」行の数が一致し、かつ最後の行が「state」行である場合、最初の「cancel」行と最後の「state」行。 | -|2.| 「state」行の方が「cancel」行より多い場合、最後の「state」行。 | -|3.| 「cancel」行の方が「state」行より多い場合、最初の「cancel」行。 | -|4.| 上記以外のすべての場合、いずれの行も保存しない。 | - -さらに、「state」行が「cancel」行より少なくとも 2 行多い場合、または「cancel」行が「state」行より少なくとも 2 行多い場合は、マージ処理は継続されます。 -ただし、ClickHouse はこの状況を論理エラーと見なし、サーバーログに記録します。 -同じデータが複数回挿入された場合に、このエラーが発生することがあります。 -したがって、collapsing によって統計値の計算結果が変わることはありません。 -変更は徐々に collapse されていき、最終的にはほぼすべてのオブジェクトについて最後の状態だけが残されます。 - -`Sign` 列が必要なのは、マージアルゴリズムが、同じソートキーを持つすべての行が同じ結果データパーツ、さらには同じ物理サーバー上に入ることを保証しないためです。 -ClickHouse は複数スレッドで `SELECT` クエリを処理するため、結果における行の順序を予測できません。 - -`CollapsingMergeTree` テーブルから完全に「collapse された」データを取得する必要がある場合は、集約処理が必要です。 -collapsing を最終確定させるには、`GROUP BY` 句と、`Sign` を考慮した集約関数を使ってクエリを書きます。 -例えば、件数を計算するには `count()` の代わりに `sum(Sign)` を使用します。 -ある値の合計を計算するには、`sum(x)` の代わりに `sum(Sign * x)` と `HAVING sum(Sign) > 0` を併用します。 -下記の[例](#example-of-use)のようにします。 - -集約関数 `count`、`sum`、`avg` はこの方法で計算できます。 -オブジェクトに少なくとも 1 つの collapse されていない状態が存在する場合、集約関数 `uniq` も計算できます。 -一方で、`min` と `max` は計算できません。 -これは、`CollapsingMergeTree` が collapse された状態の履歴を保存しないためです。 - -:::note -集約を行わずにデータを取り出す必要がある場合 -(例えば、最新の値が特定の条件に一致する行が存在するかどうかを確認する場合など)は、 -`FROM` 句に対して [`FINAL`](../../../sql-reference/statements/select/from.md#final-modifier) 修飾子を使用できます。これは、結果を返す前にデータをマージします。 -`CollapsingMergeTree` では、各キーごとに最新の「state」行のみが返されます。 -::: - - - -## 例 {#examples} - -### 使用例 {#example-of-use} - -次のサンプルデータを前提とします。 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -次に、`CollapsingMergeTree` を使用してテーブル `UAct` を作成します。 - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -次に、データを挿入します。 - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1),(4324182021466249494, 6, 185, 1) -``` - -2つの異なるデータパーツを作成するために、2つの `INSERT` クエリを使用します。 - -:::note -1つのクエリでデータを挿入すると、ClickHouse は1つのデータパーツしか作成せず、その後マージを一切実行しません。 -::: - -次のようにしてデータを取得できます: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -上で返されたデータを見て、collapsing が発生したかどうか確認してみましょう。 -2 つの `INSERT` クエリで、2 つのデータパーツを作成しました。 -`SELECT` クエリは 2 つのスレッドで実行され、行の順序はランダムになりました。 -しかし、データパーツのマージはまだ行われておらず、 -ClickHouse はデータパーツを予測できないタイミングでバックグラウンドでマージするため、 -**collapsing は発生しませんでした**。 - -そのため、集約処理を行う必要があります。 -ここでは、[`sum`](/sql-reference/aggregate-functions/reference/sum) -集約関数と [`HAVING`](/sql-reference/statements/select/having) 句を使って集約を実行します。 - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration -FROM UAct -GROUP BY UserID -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -集約が不要で、行の折りたたみを強制したい場合は、`FROM` 句に `FINAL` 修飾子を指定することもできます。 - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -:::note -このようなデータの選択方法は効率が悪く、スキャン対象データが多い場合(数百万行規模)には使用しないことを推奨します。 -::: - -### 別のアプローチの例 {#example-of-another-approach} - -このアプローチの考え方は、マージ処理がキー列のみを考慮するという点にあります。 -そのため「cancel」行では、`Sign` 列を使用せずに集計したときにその行の以前のバージョンと相殺されるような -負の値を指定できます。 - -この例では、以下のサンプルデータを使用します。 - - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ -5 │ -146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -この方法では、負の値を保存できるようにするために、`PageViews` と `Duration` のデータ型を変更する必要があります。 -そのため、テーブル `UAct` を `collapsingMergeTree` を使って作成する際に、これらの列の型を `UInt8` から `Int16` に変更します。 - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews Int16, - Duration Int16, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -テーブルにデータを挿入して、このアプローチをテストしてみましょう。 - -しかし、例や小さなテーブルの場合であれば問題ありません。 - -```sql -INSERT INTO UAct VALUES(4324182021466249494, 5, 146, 1); -INSERT INTO UAct VALUES(4324182021466249494, -5, -146, -1); -INSERT INTO UAct VALUES(4324182021466249494, 6, 185, 1); - -SELECT * FROM UAct FINAL; -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -```sql -SELECT - UserID, - sum(PageViews) AS PageViews, - sum(Duration) AS Duration -FROM UAct -GROUP BY UserID -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -```sql -SELECT COUNT() FROM UAct -``` - -```text -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -```sql -OPTIMIZE TABLE UAct FINAL; - -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md deleted file mode 100644 index 3c950ad0bac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -description: 'MergeTree テーブルにカスタムパーティショニングキーを追加する方法について説明します。' -sidebar_label: 'カスタムパーティショニングキー' -sidebar_position: 30 -slug: /engines/table-engines/mergetree-family/custom-partitioning-key -title: 'カスタムパーティショニングキー' -doc_type: 'guide' ---- - -# カスタムパーティションキー {#custom-partitioning-key} - -:::note -ほとんどの場合、パーティションキーは不要であり、それ以外の多くの場合でも、月単位より細かいパーティションキーは不要です。例外はオブザーバビリティ向けのユースケースで、この場合は日単位のパーティションが一般的です。 - -パーティションを過度に細かくしてはいけません。クライアント識別子や名前でデータをパーティションしないでください。その代わりに、クライアント識別子や名前を ORDER BY 式の最初の列にします。 -::: - -パーティショニングは、[MergeTree ファミリーのテーブル](../../../engines/table-engines/mergetree-family/mergetree.md)、[レプリケートテーブル](../../../engines/table-engines/mergetree-family/replication.md)、および[マテリアライズドビュー](/sql-reference/statements/create/view#materialized-view)で利用できます。 - -パーティションとは、指定した条件に基づいてテーブル内のレコードを論理的にまとめたものです。パーティションは、月単位、日単位、イベント種別など、任意の条件で設定できます。各パーティションは個別に保存され、このデータの操作が容易になります。データへアクセスする際、ClickHouse は可能な限り最小限のパーティション集合だけを使用します。パーティションキーを含むクエリでは、ClickHouse がまず対象となるパーティションを絞り込んでから、そのパーティション内のパーツやグラニュールを選択するため、パーティションはクエリのパフォーマンスを向上させます。 - -パーティションは、[テーブルを作成](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)するときの `PARTITION BY expr` 句で指定します。パーティションキーにはテーブル列からの任意の式を指定できます。たとえば、月単位でパーティションを指定するには、`toYYYYMM(date_column)` という式を使用します。 - -```sql -CREATE TABLE visits -( - VisitDate Date, - Hour UInt8, - ClientID UUID -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(VisitDate) -ORDER BY Hour; -``` - -パーティションキーは([primary key](../../../engines/table-engines/mergetree-family/mergetree.md#primary-keys-and-indexes-in-queries) と同様に)、式のタプルにすることもできます。例えば、次のように指定します。 - -```sql -ENGINE = ReplicatedCollapsingMergeTree('/clickhouse/tables/name', 'replica1', Sign) -PARTITION BY (toMonday(StartDate), EventType) -ORDER BY (CounterID, StartDate, intHash32(UserID)); -``` - -この例では、その週に発生したイベントタイプごとにパーティション分割を行います。 - -デフォルトでは、浮動小数点数のパーティションキーはサポートされていません。使用するには、設定 [allow_floating_point_partition_key](../../../operations/settings/merge-tree-settings.md#allow_floating_point_partition_key) を有効にします。 - -新しいデータをテーブルに挿入すると、このデータはプライマリキーでソートされた別個のパーツ(チャンク)として保存されます。挿入後 10〜15 分ほどで、同一パーティション内のパーツが 1 つのパーツにマージされます。 - -:::info -マージは、パーティション式の値が同じデータパーツに対してのみ機能します。これは **パーティションを過度に細かくしないでください**(目安としてパーティション数はおおよそ 1000 個以下にする)ということを意味します。そうしないと、ファイルシステム上のファイル数やオープンファイルディスクリプタ数が不当に多くなり、`SELECT` クエリの性能が低下します。 -::: - -テーブルのパーツおよびパーティションを確認するには、[system.parts](../../../operations/system-tables/parts.md) テーブルを使用します。たとえば、月ごとにパーティション分割された `visits` テーブルがあると仮定します。`system.parts` テーブルに対して次のように `SELECT` クエリを実行します。 - -```sql -SELECT - partition, - name, - active -FROM system.parts -WHERE table = 'visits' -``` - -```text -┌─partition─┬─name──────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1_11 │ 1 │ -│ 201902 │ 201902_10_10_0_11 │ 1 │ -│ 201902 │ 201902_11_11_0_11 │ 1 │ -└───────────┴───────────────────┴────────┘ -``` - -`partition` 列にはパーティション名が含まれています。この例では、`201901` と `201902` の 2 つのパーティションがあります。この列の値を使用して、[ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md) クエリでパーティション名を指定できます。 - -`name` 列には、パーティションのデータパーツ名が含まれます。この列を使用して、[ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart) クエリでパーツ名を指定できます。 - -パーツ名 `201901_1_9_2_11` を分解して説明します。 - -* `201901` はパーティション名です。 -* `1` はデータブロックの最小番号です。 -* `9` はデータブロックの最大番号です。 -* `2` はチャンクレベル(作成元の MergeTree における深さ)です。 -* `11` はミューテーションバージョンです(パーツがミューテートされた場合)。 - -:::info -旧タイプのテーブルのパーツ名は `20190117_20190123_2_2_0` という形式です(最小日付 - 最大日付 - 最小ブロック番号 - 最大ブロック番号 - レベル)。 -::: - -`active` 列はパーツの状態を示します。`1` はアクティブ、`0` は非アクティブです。非アクティブなパーツの例としては、より大きなパーツにマージされた後に残る元のパーツがあります。破損したデータパーツも非アクティブとして示されます。 - -例から分かるように、同じパーティションに複数の分割されたパーツが存在します(例: `201901_1_3_1` と `201901_1_9_2`)。これは、これらのパーツがまだマージされていないことを意味します。ClickHouse は挿入されたデータパーツを定期的にマージしており、おおよそ挿入から 15 分後に実行されます。さらに、[OPTIMIZE](../../../sql-reference/statements/optimize.md) クエリを使用して、スケジュール外のマージを実行することもできます。例: - -```sql -OPTIMIZE TABLE visits PARTITION 201902; -``` - -```text -┌─partition─┬─name─────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1 │ 0 │ -│ 201902 │ 201902_4_11_2_11 │ 1 │ -│ 201902 │ 201902_10_10_0 │ 0 │ -│ 201902 │ 201902_11_11_0 │ 0 │ -└───────────┴──────────────────┴────────┘ -``` - -非アクティブなパーツは、マージ後およそ 10 分で削除されます。 - -パーツとパーティションの集合を確認する別の方法として、テーブルのディレクトリ `/var/lib/clickhouse/data//
/` に移動します。例えば次のとおりです。 - -```bash -/var/lib/clickhouse/data/default/visits$ ls -l -total 40 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 201901_1_3_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201901_1_9_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_8_8_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_9_9_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_10_10_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_11_11_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:19 201902_4_11_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 12:09 201902_4_6_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached -``` - -フォルダ '201901_1_1_0'、'201901_1_7_1' などは、各パートのディレクトリです。各パートは対応するパーティションに紐づいており、特定の月のデータだけを含みます(この例のテーブルは月単位でパーティション分割されています)。 - -`detached` ディレクトリには、[DETACH](/sql-reference/statements/detach) クエリを使用してテーブルから切り離されたパーツが含まれます。破損したパーツも削除されるのではなく、このディレクトリへ移動されます。サーバーは `detached` ディレクトリ内のパーツを使用しません。このディレクトリ内のデータは、いつでも追加、削除、または変更できます。サーバーは、[ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart) クエリを実行するまでこれらを認識しません。 - -稼働中のサーバーでは、サーバーが認識できないため、ファイルシステム上のパーツの集合やそのデータを手動で変更することはできません。非レプリケートテーブルの場合、サーバー停止中であればこれを行うことはできますが、推奨されません。レプリケートテーブルの場合は、いかなる場合でもパーツの集合を変更することはできません。 - -ClickHouse ではパーティションに対して操作を実行できます。削除したり、あるテーブルから別のテーブルへコピーしたり、バックアップを作成したりできます。すべての操作の一覧は、[パーティションおよびパーツの操作](/sql-reference/statements/alter/partition) セクションを参照してください。 - -## パーティションキーを利用した Group By 最適化 {#group-by-optimisation-using-partition-key} - -テーブルのパーティションキーとクエリの Group By キーの組み合わせによっては、 -各パーティションごとに独立して集約処理を実行できる場合があります。 -この場合、すべての実行スレッドからの部分集約済みデータを最後にマージする必要がなくなります。 -これは、各 Group By キーの値が、異なる 2 つのスレッドのワーキングセット内に同時に現れないことが保証されているためです。 - -典型的な例としては、次のようなものがあります。 - -```sql -CREATE TABLE session_log -( - UserID UInt64, - SessionID UUID -) -ENGINE = MergeTree -PARTITION BY sipHash64(UserID) % 16 -ORDER BY tuple(); - -SELECT - UserID, - COUNT() -FROM session_log -GROUP BY UserID; -``` - -:::note -この種のクエリのパフォーマンスはテーブルのレイアウトに強く依存します。そのため、この最適化はデフォルトでは有効になっていません。 -::: - -良好なパフォーマンスを得るための主な要因: - -* クエリの対象となるパーティション数は十分に多い必要があります(`max_threads / 2` より大きい)。そうでない場合、クエリはマシン資源を十分に活用できません -* パーティションが小さすぎてはいけません。小さすぎるとバッチ処理が行単位の処理に退化してしまいます -* パーティションのサイズは互いに同程度であるべきです。そうすることで、すべてのスレッドがおおよそ同じ量の作業を行うようになります - -:::info -`partition by` 句のカラムに対して何らかのハッシュ関数を適用し、データがパーティション間で均等に分散されるようにすることが推奨されます。 -::: - -関連する設定は次のとおりです: - -* `allow_aggregate_partitions_independently` - この最適化の利用を有効にするかどうかを制御します -* `force_aggregate_partitions_independently` - 正しさの観点から適用可能である場合に、内部ロジックによる有効性の推定で無効化されていても、この最適化の使用を強制します -* `max_number_of_partitions_for_independent_aggregation` - テーブルが持つことのできるパーティション数の最大値に対するハードリミットです diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md deleted file mode 100644 index f432ea61618..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ /dev/null @@ -1,272 +0,0 @@ ---- -description: 'Graphite データの間引きおよび集約・平均(ロールアップ)のために設計されています。' -sidebar_label: 'GraphiteMergeTree' -sidebar_position: 90 -slug: /engines/table-engines/mergetree-family/graphitemergetree -title: 'GraphiteMergeTree テーブルエンジン' -doc_type: 'guide' ---- - -# GraphiteMergeTree テーブルエンジン {#graphitemergetree-table-engine} - -このエンジンは、[Graphite](http://graphite.readthedocs.io/en/latest/index.html) データの間引きおよび集約・平均化(ロールアップ)のために設計されています。Graphite のデータストアとして ClickHouse を使用したい開発者に役立ちます。 - -ロールアップが不要な場合は任意の ClickHouse テーブルエンジンで Graphite データを保存できますが、ロールアップが必要な場合は `GraphiteMergeTree` を使用してください。このエンジンはストレージ容量を削減し、Graphite からのクエリの効率を向上させます。 - -このエンジンは [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) の特性を継承します。 - -## テーブルの作成 {#creating-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - Path String, - Time DateTime, - Value Float64, - Version - ... -) ENGINE = GraphiteMergeTree(config_section) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -[CREATE TABLE](/sql-reference/statements/create/table) クエリの詳細な説明を参照してください。 - -Graphite データ用のテーブルには、以下のデータを格納するために次の列が必要です。 - -* メトリクス名(Graphite センサー)。データ型: `String`。 - -* メトリクスを計測した時刻。データ型: `DateTime`。 - -* メトリクスの値。データ型: `Float64`。 - -* メトリクスのバージョン。データ型: 任意の数値型(ClickHouse は、バージョンが最大の行、もしくはバージョンが同一の場合は最後に書き込まれた行を保持します。他の行はデータパーツのマージ時に削除されます)。 - -これらの列名は rollup の設定で指定する必要があります。 - -**GraphiteMergeTree のパラメータ** - -* `config_section` — 設定ファイル内のセクション名。このセクションに rollup のルールを設定します。 - -**クエリ句** - -`GraphiteMergeTree` テーブルを作成する場合は、`MergeTree` テーブルを作成する場合と同様に、同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table) が必須です。 - -
- テーブル作成の非推奨メソッド - - :::note - 新規プロジェクトではこのメソッドを使用しないでください。可能であれば、既存プロジェクトも上記で説明したメソッドへ移行してください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - EventDate Date, - Path String, - Time DateTime, - Value Float64, - Version - ... - ) ENGINE [=] GraphiteMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, config_section) - ``` - - `config_section` を除くすべてのパラメータは、`MergeTree` における意味と同じです。 - - * `config_section` — 設定ファイル内のセクション名。このセクションに rollup のルールを設定します。 -
- -## ロールアップ設定 {#rollup-configuration} - -ロールアップの設定は、サーバー設定内の [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) パラメータによって定義されます。パラメータ名は任意の名前を付けることができます。複数の設定を作成し、異なるテーブルに対して使い分けることができます。 - -ロールアップ設定の構造: - -required-columns -patterns - -### 必須カラム {#required-columns} - -#### `path_column_name` {#path_column_name} - -`path_column_name` — メトリック名(Graphite センサー)を保存するカラム名。デフォルト値: `Path`。 - -#### `time_column_name` {#time_column_name} - -`time_column_name` — メトリックを計測した時刻を保存するカラム名。デフォルト値: `Time`。 - -#### `value_column_name` {#value_column_name} - -`value_column_name` — `time_column_name` で指定された時刻におけるメトリック値を保存するカラム名。デフォルト値: `Value`。 - -#### `version_column_name` {#version_column_name} - -`version_column_name` — メトリックのバージョンを保存するカラム名。デフォルト値: `Timestamp`。 - -### パターン {#patterns} - -`patterns` セクションの構造: - -```text -pattern - rule_type - regexp - function -pattern - rule_type - regexp - age + precision - ... -pattern - rule_type - regexp - function - age + precision - ... -pattern - ... -default - function - age + precision - ... -``` - -:::important -パターンは次の厳密な順序で並べる必要があります: - -1. `function` と `retention` のいずれも指定しないパターン。 -2. `function` と `retention` の両方を指定するパターン。 -3. パターン `default`。 - ::: - -行を処理する際、ClickHouse は `pattern` セクション内のルールをチェックします。各 `pattern`(`default` を含む)セクションには、集約用の `function` パラメータ、`retention` パラメータ、またはその両方を含めることができます。メトリクス名が `regexp` にマッチする場合、その `pattern` セクション(または複数セクション)のルールが適用され、それ以外の場合は `default` セクションのルールが適用されます。 - -`pattern` および `default` セクションのフィールド: - -* `rule_type` - ルールの種別。特定の種類のメトリクスにのみ適用されます。エンジンはこれを使用してプレーンメトリクスとタグ付きメトリクスを分離します。省略可能なパラメータです。デフォルト値: `all`。 - パフォーマンスが重要でない場合、またはプレーンメトリクスなど 1 種類のメトリクスのみを使用する場合は不要です。デフォルトでは 1 種類のルールセットのみが作成されます。そうでなく、いずれかの特別なタイプが定義されている場合は、2 つの異なるセットが作成されます。1 つはプレーンメトリクス(root.branch.leaf)用、もう 1 つはタグ付きメトリクス(root.branch.leaf;tag1=value1)用です。 - デフォルトルールは両方のセットに含まれます。 - 有効な値: - * `all` (デフォルト) - `rule_type` が省略された場合に使用される汎用ルール。 - * `plain` - プレーンメトリクス用のルール。フィールド `regexp` は正規表現として処理されます。 - * `tagged` - タグ付きメトリクス用のルール(メトリクスは DB に `someName?tag1=value1&tag2=value2&tag3=value3` 形式で保存されます)。正規表現はタグ名でソートされている必要があり、存在する場合は最初のタグが `__name__` でなければなりません。フィールド `regexp` は正規表現として処理されます。 - * `tag_list` - タグ付きメトリクス用のルールで、graphite 形式 `someName;tag1=value1;tag2=value2`、`someName`、または `tag1=value1;tag2=value2` によるメトリクス記述を簡素化するための DSL です。フィールド `regexp` は `tagged` ルールに変換されます。タグ名でのソートは不要で、自動的に行われます。タグの値(名前ではなく)は正規表現として指定できます(例: `env=(dev|staging)`)。 -* `regexp` – メトリクス名に対するパターン(正規表現または DSL)。 -* `age` – データの最小経過時間(秒)。 -* `precision`– データの経過時間を秒単位でどの程度の精度で定義するか。86400(1 日の秒数)を割り切れる値である必要があります。 -* `function` – 経過時間が `[age, age + precision]` の範囲に入るデータに適用する集約関数の名前。使用可能な関数: min / max / any / avg。平均は、平均値同士の平均を取るのと同様に、厳密ではない平均として計算されます。 - -### ルールタイプなしの設定例 {#configuration-example} - -```xml - - Version - - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -### ルールタイプ別の設定例 {#configuration-typed-example} - -```xml - - Version - - plain - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - tagged - ^((.*)|.)min\? - min - - 0 - 5 - - - 86400 - 60 - - - - tagged - - min - - 0 - 5 - - - 86400 - 60 - - - - tag_list - someName;tag2=value2 - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -:::note -データのロールアップはマージ処理の際に実行されます。通常、古いパーティションではマージが開始されないため、ロールアップを行うには [optimize](../../../sql-reference/statements/optimize.md) を使用して予定外のマージをトリガーする必要があります。あるいは、[graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) などの追加ツールを使用します。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md deleted file mode 100644 index 4e89b13e7f2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: 'MergeTree エンジンファミリーに関するドキュメント' -sidebar_label: 'MergeTree ファミリー' -sidebar_position: 10 -slug: /engines/table-engines/mergetree-family/ -title: 'MergeTree エンジンファミリー' -doc_type: 'reference' ---- - -# MergeTree エンジンファミリー {#mergetree-engine-family} - -MergeTree ファミリーに属するテーブルエンジンは、ClickHouse のデータストレージ機能の中核となります。これらは、カラム型ストレージ、カスタムパーティショニング、疎なプライマリインデックス、セカンダリのデータスキップインデックスなど、耐障害性と高性能なデータ取得のためのほとんどの機能を提供します。 - -ベースとなる [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) テーブルエンジンは、単一ノードの ClickHouse インスタンスにおけるデフォルトのテーブルエンジンとみなすことができます。汎用性が高く、幅広いユースケースに対して実用的だからです。 - -本番環境での利用には [ReplicatedMergeTree](../../../engines/table-engines/mergetree-family/replication.md) を使うのが一般的です。通常の MergeTree エンジンのすべての機能に高可用性が追加されるためです。さらに、データのインジェスト時に自動的な重複排除が行われるため、挿入中にネットワークの問題が発生しても、ソフトウェア側で安全にリトライできます。 - -MergeTree ファミリーのその他すべてのエンジンは、特定のユースケース向けに追加機能を提供します。通常、これはバックグラウンドでの追加のデータ操作として実装されます。 - -MergeTree エンジンの主な欠点は、比較的ヘビーウェイトであることです。そのため、典型的なパターンとしては、それほど多くのテーブルを作成しない構成になります。多くの小さなテーブル(たとえば一時データ用)が必要な場合は、[Log エンジンファミリー](../../../engines/table-engines/log-family/index.md) の利用を検討してください。 - -{/* このページの目次は、 - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - によって、YAML フロントマターのフィールド(slug、description、title)から自動生成されています。 - - 誤りを見つけた場合は、該当ページの YAML フロントマターを編集してください。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | -| [MergeTree テーブルエンジン](/engines/table-engines/mergetree-family/mergetree) | `MergeTree` ファミリーのテーブルエンジンは、高いデータ取り込みレートと大規模なデータ量に対応するよう設計されています。 | -| [Replicated* テーブルエンジン](/engines/table-engines/mergetree-family/replication) | ClickHouse における Replicated* ファミリーのテーブルエンジンを用いたデータレプリケーションの概要です。 | -| [カスタムパーティションキー](/engines/table-engines/mergetree-family/custom-partitioning-key) | MergeTree テーブルにカスタムパーティションキーを追加する方法を説明します。 | -| [ReplacingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/replacingmergetree) | MergeTree と異なり、同じソートキー値(`ORDER BY` 句で指定される列であり、`PRIMARY KEY` ではありません)を持つ重複したエントリを削除します。 | -| [CoalescingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/coalescingmergetree) | CoalescingMergeTree は MergeTree エンジンを継承しています。主な特徴は、パーツのマージ時に各列の最後の非 NULL 値を自動的に保存できることです。 | -| [SummingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/summingmergetree) | SummingMergeTree は MergeTree エンジンを継承しています。主な特徴は、パーツのマージ時に数値データを自動的に集計(合計)できることです。 | -| [AggregatingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/aggregatingmergetree) | 同じプライマリキー(より正確には、同じ [ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md))を持つすべての行を、集約関数の状態を組み合わせて保存した 1 行(単一のデータパーツ内)に置き換えます。 | -| [CollapsingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/collapsingmergetree) | MergeTree を継承していますが、マージ処理中に行を折りたたむロジックが追加されています。 | -| [VersionedCollapsingMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) | 継続的に変化するオブジェクトの状態を高速に書き込み、古いオブジェクト状態をバックグラウンドで削除できるようにします。 | -| [GraphiteMergeTree テーブルエンジン](/engines/table-engines/mergetree-family/graphitemergetree) | Graphite データの間引きおよび集約・平均化(ロールアップ)のために設計されています。 | -| [Exact および Approximate Vector Search](/engines/table-engines/mergetree-family/annindexes) | Exact および Approximate Vector Search についてのドキュメントです。 | -| [テキストインデックスを使用した全文検索](/engines/table-engines/mergetree-family/invertedindexes) | テキスト内から検索語句を高速に見つけます。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md deleted file mode 100644 index 647bba4c279..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ /dev/null @@ -1,816 +0,0 @@ ---- -description: 'テキスト中の検索語句を素早く見つけます。' -keywords: ['全文検索', 'テキストインデックス', 'インデックス', 'インデックス(複数形)'] -sidebar_label: 'テキストインデックスを使用した全文検索' -slug: /engines/table-engines/mergetree-family/invertedindexes -title: 'テキストインデックスを使用した全文検索' -doc_type: 'reference' ---- - -import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; - - -# テキストインデックスを使用した全文検索 {#full-text-search-using-text-indexes} - - - -ClickHouse のテキストインデックス(["inverted indexes"](https://en.wikipedia.org/wiki/Inverted_index) とも呼ばれます)は、文字列データに対して高速な全文検索機能を提供します。 -インデックスは、列内の各トークンを、そのトークンを含む行に対応付けます。 -トークンは、トークン化と呼ばれる処理によって生成されます。 -例えば、ClickHouse は英語の文 "All cat like mice." を、デフォルトでは ["All", "cat", "like", "mice"] のようにトークン化します(末尾のドットは無視される点に注意してください)。 -ログデータ向けなど、より高度なトークナイザーも利用できます。 - - - -## テキストインデックスの作成 {#creating-a-text-index} - -テキストインデックスを作成するには、まず対応する実験的な設定を有効にしてください。 - -```sql -SET allow_experimental_full_text_index = true; -``` - -テキストインデックスは、次の構文を使用して、[String](/sql-reference/data-types/string.md)、[FixedString](/sql-reference/data-types/fixedstring.md)、[Array(String)](/sql-reference/data-types/array.md)、[Array(FixedString)](/sql-reference/data-types/array.md)、および [Map](/sql-reference/data-types/map.md)([mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) および [mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues) マップ関数を通じて)列に定義できます。 - -```sql -CREATE TABLE tab -( - `key` UInt64, - `str` String, - INDEX text_idx(str) TYPE text( - -- 必須パラメータ: - tokenizer = splitByNonAlpha - | splitByString[(S)] - | ngrams[(N)] - | sparseGrams[(min_length[, max_length[, min_cutoff_length]])] - | array - -- オプションパラメータ: - [, preprocessor = expression(str)] - -- オプション高度なパラメータ: - [, dictionary_block_size = D] - [, dictionary_block_frontcoding_compression = B] - [, max_cardinality_for_embedded_postings = M] - [, bloom_filter_false_positive_rate = R] - ) [GRANULARITY 64] -) -ENGINE = MergeTree -ORDER BY key -``` - -**トークナイザー引数(必須)**。`tokenizer` 引数は使用するトークナイザーを指定します: - -* `splitByNonAlpha` は、英数字以外の ASCII 文字で文字列を分割します(関数 [splitByNonAlpha](/sql-reference/functions/splitting-merging-functions.md/#splitByNonAlpha) も参照してください)。 -* `splitByString(S)` は、ユーザー定義の区切り文字列 `S` に従って文字列を分割します(関数 [splitByString](/sql-reference/functions/splitting-merging-functions.md/#splitByString) も参照してください)。 - 区切り文字列はオプションのパラメータで指定できます。たとえば `tokenizer = splitByString([', ', '; ', '\n', '\\'])` のように指定します。 - それぞれの区切り文字列は、複数文字(例では `', '`)から構成されていてもかまいません。 - 区切り文字リストを明示的に指定しない場合(たとえば `tokenizer = splitByString`)、既定の区切り文字リストは単一の空白 `[' ']` です。 -* `ngrams(N)` は、文字列を同じ長さの `N`-gram に分割します(関数 [ngrams](/sql-reference/functions/splitting-merging-functions.md/#ngrams) も参照してください)。 - n-gram の長さは 2 から 8 の整数でオプション引数として指定できます。たとえば `tokenizer = ngrams(3)` のように指定します。 - n-gram の既定サイズは、明示的に指定されない場合(たとえば `tokenizer = ngrams`)は 3 です。 -* `sparseGrams(min_length, max_length, min_cutoff_length)` は、`min_length` 以上 `max_length` 以下(両端を含む)の長さの可変長 n-gram に文字列を分割します(関数 [sparseGrams](/sql-reference/functions/string-functions#sparseGrams) も参照してください)。 - 明示的に指定されない場合、`min_length` と `max_length` の既定値はそれぞれ 3 と 100 です。 - パラメータ `min_cutoff_length` が指定されている場合、その長さ以上の n-gram のみがインデックスに保存されます。 - `ngrams(N)` と比べて、`sparseGrams` トークナイザーは可変長の N-gram を生成するため、元のテキストをより柔軟に表現できます。 - たとえば `tokenizer = sparseGrams(3, 5, 4)` は内部的には入力文字列から 3・4・5-gram を生成しますが、インデックスには 4-gram と 5-gram のみが保存されます。 -* `array` はトークン化を行いません。つまり各行の値全体が 1 つのトークンになります(関数 [array](/sql-reference/functions/array-functions.md/#array) も参照してください)。 - -:::note -`splitByString` トークナイザーは、左から右へ区切り文字列を順に適用します。 -これにより曖昧さが生じる場合があります。 -たとえば、区切り文字列を `['%21', '%']` とすると `%21abc` は `['abc']` にトークン化されますが、区切り文字列の順序を `['%', '%21']` と入れ替えると、出力は `['21abc']` になります。 -多くの場合、より長い区切り文字列が優先的にマッチすることを期待するでしょう。 -これは一般には、区切り文字列を長いものから短いものの順に渡すことで実現できます。 -区切り文字列が [prefix code](https://en.wikipedia.org/wiki/Prefix_code) を形成している場合は、任意の順序で渡すことができます。 -::: - - -:::warning -現時点では、中国語などの非西洋言語のテキストに対してテキストインデックスを構築することは推奨されません。 -現在サポートされているトークナイザーでは、インデックスサイズの肥大化とクエリ時間の増大を引き起こす可能性があります。 -今後、これらのケースをより適切に処理する言語固有の専用トークナイザーを追加する予定です。 -::: - -トークナイザーが入力文字列をどのように分割するかをテストするには、ClickHouseの[tokens](/sql-reference/functions/splitting-merging-functions.md/#tokens)関数を使用します: - -例: - -```sql -SELECT tokens('abc def', 'ngrams', 3); -``` - -結果: - -```result -['abc','bc ','c d',' de','def'] -``` - -**プリプロセッサ引数(オプション)**。`preprocessor`引数は、トークン化の前に入力文字列に適用される式です。 - -プリプロセッサ引数の典型的な使用例には以下が含まれます - -1. 大文字小文字を区別しないマッチングを可能にするための小文字化または大文字化。例:[lower](/sql-reference/functions/string-functions.md/#lower)、[lowerUTF8](/sql-reference/functions/string-functions.md/#lowerUTF8)。以下の最初の例を参照してください。 -2. UTF-8正規化。例:[normalizeUTF8NFC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFC)、[normalizeUTF8NFD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFD)、[normalizeUTF8NFKC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKC)、[normalizeUTF8NFKD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKD)、[toValidUTF8](/sql-reference/functions/string-functions.md/#toValidUTF8)。 -3. 不要な文字または部分文字列の削除または変換。例:[extractTextFromHTML](/sql-reference/functions/string-functions.md/#extractTextFromHTML)、[substring](/sql-reference/functions/string-functions.md/#substring)、[idnaEncode](/sql-reference/functions/string-functions.md/#idnaEncode)。 - -プリプロセッサ式は、[String](/sql-reference/data-types/string.md)型または[FixedString](/sql-reference/data-types/fixedstring.md)型の入力値を同じ型の値に変換する必要があります。 - -例: - -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(col))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = substringIndex(col, '\n', 1))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(extractTextFromHTML(col))` - -また、プリプロセッサ式は、テキストインデックスが定義されている列のみを参照する必要があります。 -非決定的関数の使用は許可されていません。 - -関数[hasToken](/sql-reference/functions/string-search-functions.md/#hasToken)、[hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens)、[hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens)は、プリプロセッサを使用して検索語をトークン化する前にまず変換します。 - -例えば、 - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(str) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(str)) -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, 'Foo'); -``` - -は以下と等価です: - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(lower(str)) TYPE text(tokenizer = 'splitByNonAlpha') -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, lower('Foo')); -``` - -**その他の引数(オプション)**。ClickHouseのテキストインデックスは[セカンダリインデックス](/engines/table-engines/mergetree-family/mergetree.md/#skip-index-types)として実装されています。 -ただし、他のスキップインデックスとは異なり、テキストインデックスのデフォルトのインデックスGRANULARITYは64です。 -この値は経験的に選択されており、ほとんどの使用例において速度とインデックスサイズの適切なトレードオフを提供します。 -上級ユーザーは異なるインデックス粒度を指定できます(ただし、推奨しません)。 - -
- -オプションの高度なパラメータ - -以下の高度なパラメータのデフォルト値は、ほぼすべての状況で適切に機能します。 -これらの変更は推奨しません。 - -オプションパラメータ`dictionary_block_size`(デフォルト:128)は、辞書ブロックのサイズを行数で指定します。 - -オプションパラメータ`dictionary_block_frontcoding_compression`(デフォルト:1)は、辞書ブロックが圧縮としてフロントコーディングを使用するかどうかを指定します。 - -オプションパラメータ`max_cardinality_for_embedded_postings`(デフォルト:16)は、ポスティングリストを辞書ブロックに埋め込むべきカーディナリティの閾値を指定します。 - - -オプションパラメータ `bloom_filter_false_positive_rate`(デフォルト: 0.1)は、辞書ブルームフィルタの偽陽性率を指定します。 - -
- -テキストインデックスは、テーブル作成後にカラムへ追加または削除することができます: - -```sql -ALTER TABLE tab DROP INDEX text_idx; -ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); -``` - - -## テキストインデックスの使用 {#using-a-text-index} - -SELECT クエリでテキストインデックスを使用するのは簡単で、一般的な文字列検索関数では自動的にインデックスが利用されます。 -インデックスが存在しない場合、以下の文字列検索関数は低速な総当たりスキャンによる処理にフォールバックします。 - -### サポートされている関数 {#functions-support} - -SELECT クエリの `WHERE` 句でテキスト関数が使用されている場合、テキストインデックスを利用できます。 - -```sql -SELECT [...] -FROM [...] -WHERE string_search_function(column_with_text_index) -``` - -#### `=` と `!=` {#functions-example-equals-notequals} - -`=` ([equals](/sql-reference/functions/comparison-functions.md/#equals)) と `!=` ([notEquals](/sql-reference/functions/comparison-functions.md/#notEquals)) は、指定された検索語全体と一致します。 - -例: - -```sql -SELECT * from tab WHERE str = 'こんにちは'; -``` - -テキストインデックスは `=` と `!=` をサポートしますが、等値・不等値検索が意味を持つのは `array` トークナイザを使用する場合のみです(このトークナイザではインデックスに行全体の値が保存されます)。 - -#### `IN` と `NOT IN` {#functions-example-in-notin} - -`IN`([`in`](/sql-reference/functions/in-functions))と `NOT IN`([`notIn`](/sql-reference/functions/in-functions))は `equals` および `notEquals` 関数と似ていますが、検索語句のすべてに一致するもの(`IN`)、あるいはどれにも一致しないもの(`NOT IN`)を判定します。 - -例: - -```sql -SELECT * from tab WHERE str IN ('Hello', 'World'); -``` - -`=` および `!=` に対する制限と同じ制限が適用されます。つまり、`IN` と `NOT IN` は `array` トークナイザーと組み合わせて使用する場合にのみ意味があります。 - -#### `LIKE`、`NOT LIKE` および `match` {#functions-example-like-notlike-match} - -:::note -これらの関数がフィルタリングのためにテキストインデックスを使用するのは、インデックストークナイザーが `splitByNonAlpha` または `ngrams` のいずれかである場合に限られます。 -::: - -テキストインデックスで `LIKE` ([like](/sql-reference/functions/string-search-functions.md/#like))、`NOT LIKE` ([notLike](/sql-reference/functions/string-search-functions.md/#notLike))、および [match](/sql-reference/functions/string-search-functions.md/#match) 関数を使用するには、ClickHouse が検索語句から完全なトークンを抽出できる必要があります。 - -例: - -```sql -SELECT count() FROM tab WHERE comment LIKE 'support%'; -``` - -この例の `support` は、`support`、`supports`、`supporting` などにマッチする可能性があります。 -この種のクエリは部分文字列クエリであり、テキストインデックスによって高速化することはできません。 - -LIKE クエリでテキストインデックスを活用するには、LIKE パターンを次のように書き換える必要があります。 - -```sql -SELECT count() FROM tab WHERE comment LIKE ' support %'; -- または `% support %` -``` - -`support` の左右に空白を入れておくと、その語をトークンとして抽出できるようになります。 - -#### `startsWith` と `endsWith` {#functions-example-startswith-endswith} - -`LIKE` と同様に、関数 [startsWith](/sql-reference/functions/string-functions.md/#startsWith) と [endsWith](/sql-reference/functions/string-functions.md/#endsWith) は、検索語から完全なトークンを抽出できる場合にのみテキストインデックスを使用できます。 - -例: - -```sql -SELECT count() FROM tab WHERE startsWith(comment, 'clickhouse support'); -``` - -この例では、`clickhouse` だけがトークンとして扱われます。 -`support` は `support`、`supports`、`supporting` などにマッチする可能性があるため、トークンとはみなされません。 - -`clickhouse supports` で始まるすべての行を見つけるには、検索パターンの末尾にスペースを 1 つ追加してください: - -```sql -startsWith(comment, 'clickhouse supports ')` -``` - -同様に、`endsWith` も先頭にスペースを付けて使用します。 - -```sql -SELECT count() FROM tab WHERE endsWith(comment, ' olap engine'); -``` - -#### `hasToken` と `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} - -関数 [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) および [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) は、指定された単一のトークンを対象にマッチングを行います。 - -前述の関数とは異なり、これらは検索語句をトークン化せず、入力が単一のトークンであると仮定します。 - -例: - -```sql -SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); -``` - -関数 `hasToken` と `hasTokenOrNull` は、`text` インデックスと組み合わせて使用する関数として最も高いパフォーマンスを発揮します。 - -#### `hasAnyTokens` と `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} - - -関数 [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) と [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) は、指定されたトークンの一部またはすべてにマッチします。 - -これら 2 つの関数は、検索トークンを、インデックス列で使用されるものと同じトークナイザでトークン化される文字列として、または検索前にトークナイズ処理を行わない、すでにトークン化済みのトークン配列として受け取ります。 -詳細については、それぞれの関数のドキュメントを参照してください。 - -例: - -```sql --- 検索トークンを文字列引数として渡す -SELECT count() FROM tab WHERE hasAnyTokens(comment, 'clickhouse olap'); -SELECT count() FROM tab WHERE hasAllTokens(comment, 'clickhouse olap'); - --- 検索トークンをArray(String)として渡す -SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); -SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); -``` - -#### `has` {#functions-example-has} - -配列関数 [has](/sql-reference/functions/array-functions#has) は、文字列配列内の単一のトークンとの一致を判定します。 - -例: - -```sql -SELECT count() FROM tab WHERE has(array, 'clickhouse'); -``` - -#### `mapContains` {#functions-example-mapcontains} - -関数 [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains)(`mapContainsKey` のエイリアス)は、マップのキーに含まれる単一のトークンにマッチさせます。 - -例: - -```sql -SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); --- または -SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); -``` - -#### `operator[]` {#functions-example-access-operator} - -アクセス演算子 [operator[]](/sql-reference/operators#access-operators)は、テキストインデックスと併用してキーおよび値をフィルタリングするために使用できます。 - -例: - -```sql -SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; -``` - -`Array(T)` 型および `Map(K, V)` 型のカラムをテキストインデックスと併用する場合の例を以下に示します。 - -### テキストインデックスを使用した `Array` および `Map` カラムの例 {#text-index-array-and-map-examples} - -#### Array(String) カラムへのインデックス作成 {#text-index-example-array} - -ブログプラットフォームを想像してください。著者はキーワードを使って自身のブログ記事にカテゴリー付けを行います。 -ユーザーには、トピックを検索したりクリックしたりすることで関連するコンテンツを見つけてほしいと考えています。 - -次のようなテーブル定義を想定します。 - -```sql -CREATE TABLE posts ( - post_id UInt64, - title String, - content String, - keywords Array(String) -) -ENGINE = MergeTree -ORDER BY (post_id); -``` - -テキストインデックスが存在しない場合、特定のキーワード(例:`clickhouse`)を含む投稿を見つけるには、すべてのエントリを全件スキャンする必要があります。 - -```sql -SELECT count() FROM posts WHERE has(keywords, 'clickhouse'); -- 低速なフルテーブルスキャン - 全投稿の全キーワードを確認 -``` - -プラットフォームが成長するにつれて、クエリはすべての行の `keywords` 配列を走査する必要があるため、次第に処理が遅くなっていきます。 -このパフォーマンス上の問題を解決するために、列 `keywords` に対してテキストインデックスを定義します。 - -```sql -ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitByNonAlpha); -ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- 既存データに対してインデックスを再構築してください -``` - -#### Map 列のインデックス作成 {#text-index-example-map} - -多くのオブザーバビリティのユースケースでは、ログメッセージは「コンポーネント」に分割され、タイムスタンプには日時型、ログレベルには enum 型など、適切なデータ型で保存されます。 -メトリクスのフィールドはキーと値のペアとして保存するのが最適です。 -運用チームは、デバッグ、セキュリティインシデント、監視のために、ログを効率的に検索する必要があります。 - -次のような logs テーブルを考えます: - -```sql -CREATE TABLE logs ( - id UInt64, - timestamp DateTime, - message String, - attributes Map(String, String) -) -ENGINE = MergeTree -ORDER BY (timestamp); -``` - -テキストインデックスがない場合、[Map](/sql-reference/data-types/map.md) 型データを検索するには、テーブルのフルスキャンが必要になります。 - -```sql --- レート制限データを含むすべてのログを検索: -SELECT count() FROM logs WHERE has(mapKeys(attributes), 'rate_limit'); -- 低速なフルテーブルスキャン - --- 特定のIPからのすべてのログを検索: -SELECT count() FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- 低速なフルテーブルスキャン -``` - -ログ量が増加すると、これらのクエリは遅くなります。 - -解決策は、[Map](/sql-reference/data-types/map.md) のキーと値に対してテキストインデックスを作成することです。 -フィールド名や属性タイプでログを検索する必要がある場合は、[mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) を使用してテキストインデックスを作成します。 - - -```sql -ALTER TABLE logs ADD INDEX attributes_keys_idx mapKeys(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_keys_idx; -``` - -属性の実際の内容を検索する必要がある場合は、[mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues) を使用してテキストインデックスを作成します。 - -```sql -ALTER TABLE logs ADD INDEX attributes_vals_idx mapValues(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_vals_idx; -``` - -クエリの例: - -```sql --- レート制限されたリクエストをすべて検索: -SELECT * FROM logs WHERE mapContainsKey(attributes, 'rate_limit'); -- fast - --- 特定のIPアドレスからのログをすべて検索: -SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- fast -``` - - -## パフォーマンスチューニング {#performance-tuning} - -### 直接読み取り {#direct-read} - -特定の種類のテキストクエリは、「direct read」と呼ばれる最適化によって大幅に高速化されます。 -より正確には、`SELECT` クエリがテキスト列を *選択しない* 場合に、この最適化を適用できます。 - -例: - -```sql -SELECT column_a, column_b, ... -- 注: column_with_text_indexは含めない -FROM [...] -WHERE string_search_function(column_with_text_index) -``` - -ClickHouse におけるダイレクトリード最適化は、基盤となるテキスト列にアクセスせずに、テキストインデックスのみ(すなわちテキストインデックスのルックアップ)の使用によってクエリに応答します。 -テキストインデックスのルックアップは比較的少量のデータしか読み取らないため、ClickHouse の通常のスキップインデックス(スキップインデックスのルックアップの後に、残った granule の読み込みとフィルタリングを行う)よりもはるかに高速です。 - -ダイレクトリードは次の 2 つの設定で制御されます。 - -* 設定 [query_plan_direct_read_from_text_index](../../../operations/settings/settings#query_plan_direct_read_from_text_index) は、ダイレクトリードを全体として有効にするかどうかを指定します。 -* 設定 [use_skip_indexes_on_data_read](../../../operations/settings/settings#use_skip_indexes_on_data_read) は、ダイレクトリードのもう 1 つの前提条件です。ClickHouse データベースで [compatibility](../../../operations/settings/settings#compatibility) < 25.10 の場合、`use_skip_indexes_on_data_read` は無効化されているため、compatibility 設定値を引き上げるか、明示的に `SET use_skip_indexes_on_data_read = 1` と設定する必要があります。 - -また、ダイレクトリードを利用するには、テキストインデックスが完全にマテリアライズされている必要があります(そのためには `ALTER TABLE ... MATERIALIZE INDEX` を使用します)。 - -**サポートされている関数** -ダイレクトリード最適化は、`hasToken`、`hasAllTokens`、`hasAnyTokens` 関数をサポートします。 -これらの関数は AND、OR、NOT 演算子と組み合わせることもできます。 -WHERE 句には、(テキスト列やその他の列に対する)追加の非テキスト検索関数によるフィルタも含めることができます。その場合でもダイレクトリード最適化は使用されますが、効果は小さくなります(サポートされているテキスト検索関数にのみ適用されるため)。 - -クエリがダイレクトリードを利用しているかを確認するには、`EXPLAIN PLAN actions = 1` を指定してクエリを実行します。 -例として、ダイレクトリードを無効にしたクエリは次のようになります - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 0, -- disable direct read - use_skip_indexes_on_data_read = 1; -``` - -戻り値 - -```text -[...] -Filter ((WHERE + 列名を列識別子に変更)) -Filter column: hasToken(__table1.col, 'some_token'_String) (削除済み) -Actions: INPUT : 0 -> col String : 0 - COLUMN Const(String) -> 'some_token'_String String : 1 - FUNCTION hasToken(col :: 0, 'some_token'_String :: 1) -> hasToken(__table1.col, 'some_token'_String) UInt8 : 2 -[...] -``` - -一方、同じクエリを `query_plan_direct_read_from_text_index = 1` を指定して実行すると - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 1, -- enable direct read - use_skip_indexes_on_data_read = 1; -``` - -戻り値 - -```text -[...] -Expression (Before GROUP BY) -Positions: - Filter - Filter column: __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 (removed) - Actions: INPUT :: 0 -> __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 UInt8 : 0 -[...] -``` - -2番目の EXPLAIN PLAN の出力には、仮想カラム `__text_index___` が含まれます。 -このカラムが存在する場合は、直接読み出しが使用されています。 - -### キャッシュ {#caching} - -テキストインデックスの一部をメモリ上でバッファリングするために、複数のキャッシュが利用可能です([Implementation Details](#implementation) セクションを参照)。 -現在、I/O を削減するために、テキストインデックスのデシリアライズ済みディクショナリブロック、ヘッダー、およびポスティングリスト用のキャッシュが用意されています。 -これらは設定 [use_text_index_dictionary_cache](/operations/settings/settings#use_text_index_dictionary_cache)、[use_text_index_header_cache](/operations/settings/settings#use_text_index_header_cache)、および [use_text_index_postings_cache](/operations/settings/settings#use_text_index_postings_cache) によって有効化できます。 -デフォルトでは、すべてのキャッシュは無効になっています。 - -キャッシュを設定するには、以下のサーバー設定を参照してください。 - -#### ディクショナリブロックキャッシュの設定 {#caching-dictionary} - - -| Setting | Description | -|----------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------| -| [text_index_dictionary_block_cache_policy](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_policy) | テキストインデックス辞書ブロックキャッシュのポリシー名。 | -| [text_index_dictionary_block_cache_size](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size) | キャッシュの最大サイズ(バイト単位)。 | -| [text_index_dictionary_block_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_max_entries) | キャッシュ内のデシリアライズ済み辞書ブロックの最大数。 | -| [text_index_dictionary_block_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size_ratio) | テキストインデックス辞書ブロックキャッシュにおける保護キューのサイズの、キャッシュ全体サイズに対する割合。 | - -#### ヘッダーキャッシュ設定 {#caching-header} - -| Setting | Description | -|--------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| -| [text_index_header_cache_policy](/operations/server-configuration-parameters/settings#text_index_header_cache_policy) | テキストインデックスヘッダーキャッシュのポリシー名。 | -| [text_index_header_cache_size](/operations/server-configuration-parameters/settings#text_index_header_cache_size) | キャッシュの最大サイズ(バイト単位)。 | -| [text_index_header_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_header_cache_max_entries) | キャッシュ内のデシリアライズ済みヘッダーの最大数。 | -| [text_index_header_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_header_cache_size_ratio) | テキストインデックスヘッダーキャッシュにおける保護キューのサイズの、キャッシュ全体サイズに対する割合。 | - -#### ポスティングリストキャッシュ設定 {#caching-posting-lists} - -| Setting | Description | -|---------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------| -| [text_index_postings_cache_policy](/operations/server-configuration-parameters/settings#text_index_postings_cache_policy) | テキストインデックスのポスティングリストキャッシュのポリシー名。 | -| [text_index_postings_cache_size](/operations/server-configuration-parameters/settings#text_index_postings_cache_size) | キャッシュの最大サイズ(バイト単位)。 | -| [text_index_postings_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_postings_cache_max_entries) | キャッシュ内のデシリアライズ済みポスティングの最大数。 | -| [text_index_postings_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_postings_cache_size_ratio) | テキストインデックスのポスティングリストキャッシュにおける保護キューのサイズの、キャッシュ全体サイズに対する割合。 | - - - -## 実装の詳細 {#implementation} - -各テキストインデックスは、2 つの(抽象的な)データ構造から構成されます: -- 各トークンをポスティングリストにマッピングする辞書 -- 各々が行番号の集合を表すポスティングリストの集合 - -テキストインデックスはスキップインデックスであるため、これらのデータ構造は論理的にはインデックスグラニュール単位で存在します。 - -インデックス作成時には、3 つのファイルが(パートごとに)作成されます。 - -**Dictionary blocks file (.dct)** - -インデックスグラニュール内のトークンはソートされ、128 トークンごとの辞書ブロックに格納されます(ブロックサイズはパラメータ `dictionary_block_size` で設定可能です)。 -Dictionary blocks file (.dct) は、あるパート内のすべてのインデックスグラニュールに対するすべての辞書ブロックから構成されます。 - -**Index granules file (.idx)** - -Index granules file (.idx) には、各辞書ブロックについて、そのブロックの先頭トークン、dictionary blocks file 内での相対オフセット、そしてブロック内のすべてのトークンに対するブルームフィルタが含まれます。 -この疎なインデックス構造は、ClickHouse の [sparse primary key index](https://clickhouse.com/docs/guides/best-practices/sparse-primary-indexes) と類似しています。 -ブルームフィルタにより、検索対象のトークンが辞書ブロックに含まれていない場合、その辞書ブロックを早期にスキップできます。 - -**Postings lists file (.pst)** - -すべてのトークンに対するポスティングリストは、postings lists file 内に連続して配置されます。 -ストレージ容量を節約しつつ、高速な積集合および和集合の演算を可能にするため、ポスティングリストは [roaring bitmaps](https://roaringbitmap.org/) として保存されます。 -ポスティングリストの基数が 16 未満の場合(パラメータ `max_cardinality_for_embedded_postings` で設定可能)、そのリストは辞書内に埋め込まれます。 - - - -## 例:Hackernews データセット {#hacker-news-dataset} - -大量のテキストを含む大規模なデータセットに対するテキストインデックスのパフォーマンス向上を確認していきます。 -ここでは、人気サイトである Hacker News 上のコメント 2,870 万件を使用します。 -以下はテキストインデックスを作成していないテーブルです: - -```sql -CREATE TABLE hackernews ( - id UInt64, - deleted UInt8, - type String, - author String, - timestamp DateTime, - comment String, - dead UInt8, - parent UInt64, - poll UInt64, - children Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32 -) -ENGINE = MergeTree -ORDER BY (type, author); -``` - -この 2,870 万行は S3 上の Parquet ファイルに格納されています。これらを `hackernews` テーブルに挿入しましょう: - -```sql -INSERT INTO hackernews - SELECT * FROM s3Cluster( - 'default', - 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/hackernews/hacknernews.parquet', - 'Parquet', - ' - id UInt64, - deleted UInt8, - type String, - by String, - time DateTime, - text String, - dead UInt8, - parent UInt64, - poll UInt64, - kids Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32'); -``` - -`ALTER TABLE` を使用して comment 列にテキストインデックスを追加し、次にそれをマテリアライズします: - -```sql --- インデックスを追加 -ALTER TABLE hackernews ADD INDEX comment_idx(comment) TYPE text(tokenizer = splitByNonAlpha); - --- 既存データのインデックスをマテリアライズ -ALTER TABLE hackernews MATERIALIZE INDEX comment_idx SETTINGS mutations_sync = 2; -``` - -では、`hasToken`、`hasAnyTokens`、`hasAllTokens` 関数を使ってクエリを実行してみます。 -次の例では、標準的なインデックススキャンとダイレクトリード最適化の間で、どれほど大きな性能差が生じるかを示します。 - -### 1. `hasToken` を使用する {#using-hasToken} - -`hasToken` は、テキストに特定の単一トークンが含まれているかどうかをチェックします。 -ここでは大文字小文字を区別して、`ClickHouse` というトークンを検索します。 - -**ダイレクトリード無効(標準スキャン)** -デフォルトでは、ClickHouse はスキップインデックスを使ってグラニュールをフィルタリングし、そのグラニュールに対してカラムデータを読み込みます。 -ダイレクトリードを無効にすることで、この動作をシミュレートできます。 - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -1行のセット。経過時間: 0.362秒。処理: 2490万行、9.51 GB -``` - -**ダイレクトリード有効(高速インデックス読み取り)** -次に、デフォルトで有効になっているダイレクトリードを使用して同じクエリを実行します。 - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -1 row in set. 経過時間: 0.008秒 処理行数: 315万行、3.15 MB -``` - -直接読み取りクエリは 45 倍以上高速 (0.362s 対 0.008s) で、インデックスのみを読み取ることで処理するデータ量も大幅に削減されます (9.51 GB 対 3.15 MB)。 - -### 2. `hasAnyTokens` を使用する {#using-hasAnyTokens} - -`hasAnyTokens` は、テキストに指定したトークンのうち少なくとも 1 つが含まれているかどうかをチェックします。 -ここでは、'love' または 'ClickHouse' のいずれかを含むコメントを検索します。 - -**直接読み取り無効(標準スキャン)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 408426 │ -└─────────┘ - -1 row in set. Elapsed: 1.329 sec. Processed 28.74 million rows, 9.72 GB -``` - -**ダイレクト読み取り有効(高速インデックス読み出し)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 408426 │ -└─────────┘ -``` - - -1 行がセットに含まれています。経過時間: 0.015 秒。27.99 百万行、27.99 MB を処理しました。 - -```` -この一般的な「OR」検索では、高速化がさらに顕著です。 -フルカラムスキャンを回避することで、クエリは約89倍高速化されます(1.329秒 vs 0.015秒)。 - -### 3. `hasAllTokens`の使用 {#using-hasAllTokens} - -`hasAllTokens`は、テキストに指定されたすべてのトークンが含まれているかを確認します。 -'love'と'ClickHouse'の両方を含むコメントを検索します。 - -**ダイレクトリード無効(標準スキャン)** -ダイレクトリードが無効な場合でも、標準スキップインデックスは依然として有効です。 -2870万行を14.746万行にフィルタリングしますが、カラムから57.03 MBを読み取る必要があります。 - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -1 row in set. Elapsed: 0.184 sec. Processed 147.46 thousand rows, 57.03 MB -```` - -**ダイレクトリード有効(高速インデックス読み取り)** -ダイレクトリードではインデックスデータに対してクエリを実行し、147.46 KB 分のみを読み取って結果を返します。 - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -1行が返されました。経過時間: 0.007秒。処理された行数: 147.46千行、147.46 KB -``` - -この AND 検索では、ダイレクトリード最適化は標準のスキップインデックススキャンと比べて 26 倍以上高速です (0.184s 対 0.007s)。 - -### 4. 複合検索: OR, AND, NOT, ... {#compound-search} - -ダイレクトリード最適化は、複合ブール式にも適用されます。 -ここでは、大文字小文字を区別しない検索で「ClickHouse」または「clickhouse」を検索します。 - -**ダイレクトリード無効時 (標準スキャン)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -1行が返されました。経過時間: 0.450秒。処理された行数: 2587万行、9.58 GB -``` - -**ダイレクトリードが有効(高速インデックス読み取り)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -1 row in set. Elapsed: 0.013 sec. Processed 25.87 million rows, 51.73 MB -``` - -インデックスの結果を組み合わせることで、直接読み取りクエリは 34 倍高速 (0.450 秒対 0.013 秒) になり、9.58 GB のカラムデータを読み込む必要がなくなります。 -この特定のケースでは、`hasAnyTokens(comment, ['ClickHouse', 'clickhouse'])` が推奨される、より効率的な構文です。 - - -## 関連コンテンツ {#related-content} - -- ブログ: [ClickHouse における転置インデックスの紹介](https://clickhouse.com/blog/clickhouse-search-with-inverted-indices) -- ブログ: [ClickHouse の全文検索の内部構造: 高速・ネイティブ・カラム型](https://clickhouse.com/blog/clickhouse-full-text-search) -- 動画: [全文検索インデックス: 設計と実験](https://www.youtube.com/watch?v=O_MnyUkrIq8) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md deleted file mode 100644 index b472dc8f3f5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md +++ /dev/null @@ -1,1162 +0,0 @@ ---- -description: '`MergeTree` ファミリーのテーブルエンジンは、高速なデータ取り込みと膨大なデータ量に対応するよう設計されています。' -sidebar_label: 'MergeTree' -sidebar_position: 11 -slug: /engines/table-engines/mergetree-family/mergettree -title: 'MergeTree テーブルエンジン' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# MergeTree テーブルエンジン {#mergetree-table-engine} - -`MergeTree` エンジンおよび `MergeTree` ファミリーの他のエンジン(例: `ReplacingMergeTree`、`AggregatingMergeTree`)は、ClickHouse で最も一般的に使用され、最も堅牢なテーブルエンジンです。 - -`MergeTree` ファミリーのテーブルエンジンは、高いデータ取り込みレートと巨大なデータ量を扱えるように設計されています。 -`INSERT` 操作によりテーブルパーツが作成され、バックグラウンドプロセスによって他のテーブルパーツとマージされます。 - -`MergeTree` ファミリーのテーブルエンジンの主な特徴は次のとおりです。 - -* テーブルの主キーは、各テーブルパーツ内でのソート順(クラスタ化インデックス)を決定します。主キーは個々の行ではなく、グラニュールと呼ばれる 8192 行のブロックを参照します。これにより、巨大なデータセットに対しても主キーをメインメモリに常駐できる程度の小ささに保ちながら、ディスク上のデータへ高速にアクセスできます。 - -* テーブルは任意のパーティション式でパーティション分割できます。パーティションプルーニングにより、クエリ条件に応じて読み取り対象からパーティションを除外できます。 - -* 可用性の向上、フェイルオーバー、およびゼロダウンタイムでのアップグレードのために、データを複数のクラスタノード間でレプリケートできます。詳しくは [Data replication](/engines/table-engines/mergetree-family/replication.md) を参照してください。 - -* `MergeTree` テーブルエンジンは、クエリ最適化に役立つさまざまな種類の統計情報とサンプリング手法をサポートします。 - -:::note -名前は似ていますが、[Merge](/engines/table-engines/special/merge) エンジンは `*MergeTree` エンジンとは異なります。 -::: - -## テーブルの作成 {#table_engine-mergetree-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr1] [COMMENT ...] [CODEC(codec1)] [STATISTICS(stat1)] [TTL expr1] [PRIMARY KEY] [SETTINGS (name = value, ...)], - name2 [type2] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr2] [COMMENT ...] [CODEC(codec2)] [STATISTICS(stat2)] [TTL expr2] [PRIMARY KEY] [SETTINGS (name = value, ...)], - ... - INDEX index_name1 expr1 TYPE type1(...) [GRANULARITY value1], - INDEX index_name2 expr2 TYPE type2(...) [GRANULARITY value2], - ... - PROJECTION projection_name_1 (SELECT [GROUP BY] [ORDER BY]), - PROJECTION projection_name_2 (SELECT [GROUP BY] [ORDER BY]) -) ENGINE = MergeTree() -ORDER BY expr -[PARTITION BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[TTL expr - [DELETE|TO DISK 'xxx'|TO VOLUME 'xxx' [, ...] ] - [WHERE conditions] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] ] -[SETTINGS name = value, ...] -``` - -パラメータの詳細については、[CREATE TABLE](/sql-reference/statements/create/table.md) ステートメントを参照してください。 - -### クエリ構文 {#mergetree-query-clauses} - -#### ENGINE {#engine} - -`ENGINE` — エンジンの名前とパラメータを指定します。`ENGINE = MergeTree()`。`MergeTree` エンジンにはパラメータがありません。 - -#### ORDER BY {#order_by} - -`ORDER BY` — ソートキーです。 - -カラム名または任意の式のタプルを指定します。例: `ORDER BY (CounterID + 1, EventDate)`。 - -`PRIMARY KEY` が定義されていない場合(つまり `PRIMARY KEY` が指定されていない場合)、ClickHouse はそのソートキーをプライマリキーとして使用します。 - -ソートが不要な場合は、`ORDER BY tuple()` 構文を使用できます。 -また、設定 `create_table_empty_primary_key_by_default` が有効な場合、`ORDER BY ()` が `CREATE TABLE` 文に暗黙的に追加されます。[プライマリキーの選択](#selecting-a-primary-key) を参照してください。 - -#### PARTITION BY {#partition-by} - -`PARTITION BY` — [パーティションキー](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。省略可能です。ほとんどの場合、パーティションキーは不要であり、パーティション分割が必要な場合でも、通常は月単位より細かい粒度のパーティションキーは必要ありません。パーティション分割は(ORDER BY 式とは対照的に)クエリを高速化しません。パーティションは決して細かくしすぎないでください。データをクライアント識別子や名前でパーティション分割しないでください(その代わりに、クライアント識別子または名前を ORDER BY 式の先頭のカラムにしてください)。 - -月単位でパーティション分割するには、`toYYYYMM(date_column)` 式を使用します。ここで、`date_column` は [Date](/sql-reference/data-types/date.md) 型の日付を保持するカラムです。この場合のパーティション名は `"YYYYMM"` 形式になります。 - -#### PRIMARY KEY {#primary-key} - -`PRIMARY KEY` — [ソートキーと異なる場合](#choosing-a-primary-key-that-differs-from-the-sorting-key)に指定するプライマリキーです。省略可能です。 - -ソートキー(`ORDER BY` 句)を指定すると、暗黙的にプライマリキーも指定されます。 -通常、ソートキーとは別にプライマリキーを指定する必要はありません。 - -#### SAMPLE BY {#sample-by} - -`SAMPLE BY` — サンプリング用の式です。省略可能です。 - -指定する場合は、主キーに含まれている必要があります。 -サンプリング式は符号なし整数値を返さなければなりません。 - -例: `SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`. - -#### TTL {#ttl} - -`TTL` — 行の保存期間と、[ディスクおよびボリューム間](#table_engine-mergetree-multiple-volumes)の自動的なパーツ移動ロジックを指定するルールのリストです。省略可能です。 - -式は `Date` または `DateTime` 型の値を返す必要があります。例: `TTL date + INTERVAL 1 DAY`。 - -ルール種別 `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` は、式が満たされた(現在時刻に達した)ときにそのパーツに対して実行されるアクションを指定します。具体的には、有効期限切れの行の削除、パーツ内のすべての行に対して式が満たされている場合にそのパーツを指定したディスク(`TO DISK 'xxx'`)またはボリューム(`TO VOLUME 'xxx'`)へ移動すること、あるいは有効期限切れの行の値を集約することです。ルールのデフォルト種別は削除(`DELETE`)です。複数のルールを指定できますが、`DELETE` ルールは 1 つまでにする必要があります。 - -詳細については、[カラムおよびテーブルの TTL](#table_engine-mergetree-ttl) を参照してください。 - -#### 設定 {#settings} - -[MergeTree Settings](../../../operations/settings/merge-tree-settings.md) を参照してください。 - -**Sections 設定の例** - -```sql -ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity=8192 -``` - -この例では、パーティションを月単位で設定しています。 - -また、ユーザー ID をハッシュ化したものをサンプリング用の式として設定しています。これにより、テーブル内のデータを各 `CounterID` と `EventDate` ごとに擬似乱数的に分散させることができます。データを選択する際に [SAMPLE](/sql-reference/statements/select/sample) 句を指定すると、ClickHouse はユーザーのサブセットに対して偏りの少ない擬似乱数サンプルデータを返します。 - -`index_granularity` 設定は省略できます。デフォルト値が 8192 のためです。 - -
- テーブル作成の非推奨メソッド - - :::note - 新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上記で説明した方法に切り替えてください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - **MergeTree() のパラメータ** - - * `date-column` — [Date](/sql-reference/data-types/date.md) 型のカラム名。ClickHouse はこのカラムに基づいて自動的に月ごとのパーティションを作成します。パーティション名は `"YYYYMM"` 形式になります。 - * `sampling_expression` — サンプリング用の式。 - * `(primary, key)` — 主キー。型: [Tuple()](/sql-reference/data-types/tuple.md) - * `index_granularity` — インデックスの粒度。インデックスの「マーク」間のデータ行数。8192 という値はほとんどのユースケースに適しています。 - - **例** - - ```sql - MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) - ``` - - `MergeTree` エンジンは、メインのエンジン構成方法について上記の例と同様に設定されます。 -
- -## データストレージ {#mergetree-data-storage} - -テーブルは、主キーでソートされたデータパーツから構成されます。 - -テーブルにデータが挿入されると、個別のデータパーツが作成され、それぞれが主キーに従って辞書順でソートされます。たとえば、主キーが `(CounterID, Date)` の場合、そのパーツ内のデータはまず `CounterID` でソートされ、各 `CounterID` の範囲内では `Date` の順に並びます。 - -異なるパーティションに属するデータは、異なるパーツに分離されます。バックグラウンドで ClickHouse は、より効率的に保存できるようにデータパーツをマージします。異なるパーティションに属するパーツはマージされません。マージの仕組みは、同じ主キーを持つすべての行が同じデータパーツ内に配置されることを保証しません。 - -データパーツは `Wide` または `Compact` フォーマットで保存できます。`Wide` フォーマットでは各カラムがファイルシステム上の別々のファイルに保存され、`Compact` フォーマットではすべてのカラムが 1 つのファイルに保存されます。`Compact` フォーマットは、小さなデータの頻繁な挿入時のパフォーマンス向上に利用できます。 - -データの保存形式は、テーブルエンジンの `min_bytes_for_wide_part` および `min_rows_for_wide_part` 設定によって制御されます。データパーツ内のバイト数または行数が対応する設定値より小さい場合、そのパーツは `Compact` フォーマットで保存されます。そうでない場合は `Wide` フォーマットで保存されます。これらの設定がどちらも指定されていない場合、データパーツは `Wide` フォーマットで保存されます。 - -各データパーツは論理的にグラニュールに分割されます。グラニュールは、ClickHouse がデータを選択する際に読み取る最小の不可分なデータ集合です。ClickHouse は行や値を分割しないため、各グラニュールは常に整数個の行を含みます。グラニュールの先頭行には、その行の主キーの値によってマークが付けられます。各データパーツごとに、ClickHouse はこれらのマークを保存するインデックスファイルを作成します。各カラムに対して、主キーに含まれるかどうかに関わらず、ClickHouse は同じマークも保存します。これらのマークによって、カラムファイル内のデータを直接特定できます。 - -グラニュールサイズは、テーブルエンジンの `index_granularity` および `index_granularity_bytes` 設定によって制限されます。グラニュール内の行数は、行のサイズに応じて `[1, index_granularity]` の範囲になります。単一行のサイズが設定値より大きい場合、グラニュールのサイズは `index_granularity_bytes` を超えることがあります。この場合、グラニュールのサイズはその行のサイズと等しくなります。 - -## クエリにおける主キーとインデックス {#primary-keys-and-indexes-in-queries} - -例として、主キー `(CounterID, Date)` を取り上げます。この場合、並び順とインデックスは次のように示されます。 - -```text -全データ: [---------------------------------------------] -CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] -Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] -マーク: | | | | | | | | | | | - a,1 a,2 a,3 b,3 e,2 e,3 g,1 h,2 i,1 i,3 l,3 -マーク番号: 0 1 2 3 4 5 6 7 8 9 10 -``` - -データクエリが次のように指定されている場合: - -* `CounterID in ('a', 'h')` の場合、サーバーはマーク範囲 `[0, 3)` および `[6, 8)` のデータを読み込みます。 -* `CounterID IN ('a', 'h') AND Date = 3` の場合、サーバーはマーク範囲 `[1, 3)` および `[7, 8)` のデータを読み込みます。 -* `Date = 3` の場合、サーバーはマーク範囲 `[1, 10]` のデータを読み込みます。 - -上記の例から、常にフルスキャンよりもインデックスを使用する方が効率的であることがわかります。 - -疎なインデックスでは、追加のデータが読み込まれることがあります。主キーの単一の範囲を読み込む場合、各データブロックで最大 `index_granularity * 2` 行まで余分な行が読み込まれる可能性があります。 - -疎なインデックスを使用すると、非常に多くのテーブル行を扱うことができます。ほとんどの場合、このようなインデックスはコンピュータの RAM に収まるためです。 - -ClickHouse では、一意なプライマリキーは不要です。同じプライマリキーを持つ複数の行を挿入できます。 - -`PRIMARY KEY` および `ORDER BY` 句では `Nullable` 型の式を使用できますが、これは強く非推奨です。この機能を許可するには、[allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 設定を有効にします。`ORDER BY` 句での `NULL` 値には、[NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) の原則が適用されます。 - -### 主キーの選択 {#selecting-a-primary-key} - -主キーに含める列数には明確な制限はありません。データ構造に応じて、主キーに含める列を増やしたり減らしたりできます。これにより次の効果が得られる可能性があります。 - -* インデックスのパフォーマンスが向上する。 - - 主キーが `(a, b)` の場合に、さらに列 `c` を追加すると、次の条件が満たされるときにパフォーマンスが向上します。 - - * 列 `c` に条件を持つクエリが存在する。 - * `(a, b)` の値が同一である長いデータ範囲(`index_granularity` の数倍の長さ)がよく現れる。言い換えると、列を追加することで、かなり長いデータ範囲をスキップできる場合です。 - -* データ圧縮が向上する。 - - ClickHouse はデータを主キーでソートして保存するため、データの並びの一貫性が高いほど圧縮率が向上します。 - -* [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) および [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) エンジンでデータパーツをマージする際に、追加のロジックを提供できる。 - - この場合、主キーとは異なる *sorting key* を指定することが有効です。 - -長い主キーは挿入パフォーマンスやメモリ消費に悪影響を及ぼしますが、主キーに余分な列があっても、`SELECT` クエリ時の ClickHouse のパフォーマンスには影響しません。 - -`ORDER BY tuple()` 構文を使うことで、主キーなしのテーブルを作成できます。この場合、ClickHouse はデータを挿入順に保存します。`INSERT ... SELECT` クエリで挿入時のデータ順序を保持したい場合は、[max_insert_threads = 1](/operations/settings/settings#max_insert_threads) を設定してください。 - -挿入時の順序でデータを取得するには、[single-threaded](/operations/settings/settings.md/#max_threads) な `SELECT` クエリを使用します。 - -### 並べ替えキーと異なる主キーを選択する {#choosing-a-primary-key-that-differs-from-the-sorting-key} - -主キー(各マークごとのインデックスファイルに書き込まれる値を持つ式)は、並べ替えキー(データパーツ内の行を並べ替えるための式)とは異なるものを指定できます。この場合、主キーの式タプルは、並べ替えキーの式タプルのプレフィックスでなければなりません。 - -この機能は [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) および -[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md) テーブルエンジンを使用する際に有用です。これらのエンジンを利用する一般的なケースでは、テーブルには 2 種類のカラム、つまり *dimensions* と *measures* があります。典型的なクエリでは、任意の `GROUP BY` を用いて measure カラムの値を集約し、dimensions でフィルタリングします。SummingMergeTree と AggregatingMergeTree は並べ替えキーの値が同一の行を集約するため、すべての dimensions を並べ替えキーに含めるのが自然です。その結果、キー式は多数のカラムから成る長いリストとなり、新たな dimensions を追加するたびにこのリストを頻繁に更新しなければなりません。 - -このような場合、効率的なレンジスキャンを行うために必要な少数のカラムだけを主キーに残し、残りの dimension カラムを並べ替えキーのタプルに追加するのが理にかなっています。 - -並べ替えキーの [ALTER](/sql-reference/statements/alter/index.md) は軽量な操作です。新しいカラムがテーブルおよび並べ替えキーに同時に追加される場合、既存のデータパーツを変更する必要がないためです。古い並べ替えキーが新しい並べ替えキーのプレフィックスであり、新しく追加されたカラムにはデータが存在しないので、テーブルを変更した時点では、データは古い並べ替えキーと新しい並べ替えキーの両方に従って並べ替えられていることになります。 - -### クエリにおけるインデックスとパーティションの利用 {#use-of-indexes-and-partitions-in-queries} - -`SELECT` クエリに対して、ClickHouse はインデックスを利用できるかどうかを解析して判断します。`WHERE/PREWHERE` 句に、等価比較または不等価比較の演算(連言要素の 1 つ、もしくは式全体)を表す式が含まれている場合、あるいは、プライマリキーまたはパーティションキーに含まれる列や式、それらの列に対する特定の部分的に反復的な関数、さらにそれらの式同士の論理関係に対して、固定プレフィックスを伴う `IN` または `LIKE` が含まれている場合、インデックスを使用できます。 - -そのため、プライマリキーの 1 つまたは複数の範囲に対してクエリを高速に実行できます。この例では、特定のトラッキングタグ、特定のタグと日付範囲、特定のタグと特定の日付、複数のタグと日付範囲などに対するクエリは高速に実行されます。 - -次のように設定されたエンジンを見てみましょう。 - -```sql -ENGINE MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate) -SETTINGS index_granularity=8192 -``` - -この場合、クエリは次のようになります: - -```sql -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND CounterID = 34 - -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND (CounterID = 34 OR CounterID = 42) - -SELECT count() FROM table -WHERE ((EventDate >= toDate('2014-01-01') -AND EventDate <= toDate('2014-01-31')) OR EventDate = toDate('2014-05-01')) -AND CounterID IN (101500, 731962, 160656) -AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) -``` - -ClickHouse は、プライマリキーインデックスを使用して不適切なデータを除外し、月単位のパーティショニングキーを使用して不適切な日付範囲にあるパーティションを除外します。 - -上記のクエリは、複雑な式に対してもインデックスが使用されることを示しています。テーブルからの読み取り処理は、インデックスを使用してもフルスキャンより遅くならないように設計されています。 - -次の例では、インデックスを利用できません。 - -```sql -SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' -``` - -クエリ実行時に ClickHouse がインデックスを利用できるかどうかを確認するには、設定 [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) と [force_primary_key](/operations/settings/settings#force_primary_key) を使用します。 - -月単位のパーティションキーは、指定した範囲に含まれる日付を持つデータブロックだけを読み取れるようにします。この場合、データブロックには多数の日付(最大で 1 か月分)に対応するデータが含まれている可能性があります。ブロック内ではデータは主キーでソートされていますが、主キーの先頭の列として日付が含まれていない場合があります。そのため、主キーのプレフィックスを指定せずに日付条件のみを含むクエリを使用すると、単一の日付だけを対象とする場合よりも多くのデータを読み取ることになります。 - -### 部分的に単調な主キーに対するインデックスの利用 {#use-of-index-for-partially-monotonic-primary-keys} - -例として、月の日付を考えてみます。1 か月の中では [単調な数列](https://en.wikipedia.org/wiki/Monotonic_function) を形成しますが、より長い期間では単調ではありません。これは部分的に単調な数列です。ユーザーが部分的に単調な主キーでテーブルを作成した場合、ClickHouse は通常どおりスパースインデックスを作成します。ユーザーがこの種のテーブルからデータを `SELECT` する際、ClickHouse はクエリ条件を解析します。ユーザーがインデックスの 2 つのマークの間のデータを取得しようとしており、その両方のマークが同じ 1 か月の範囲内に収まる場合、ClickHouse はこの特定のケースではインデックスを利用できます。これは、クエリパラメータとインデックスマークとの距離を計算できるためです。 - -クエリのパラメータ範囲内に含まれる主キーの値が単調な数列を表さない場合、ClickHouse はインデックスを利用できません。この場合、ClickHouse はフルスキャンを行います。 - -ClickHouse は、このロジックを月の日付の数列に対してだけでなく、部分的に単調な数列を表すあらゆる主キーに対しても適用します。 - -### データスキップインデックス {#table_engine-mergetree-data_skipping-indexes} - -インデックスの宣言は、`CREATE` クエリのカラム定義セクション内に記述します。 - -```sql -INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] -``` - -`*MergeTree` ファミリーのテーブルでは、データスキップインデックスを指定できます。 - -これらのインデックスは、ブロック上で指定された式に関する情報の一部を集約します。ブロックは `granularity_value` 個のグラニュールから構成されます(グラニュールのサイズはテーブルエンジンの `index_granularity` 設定で指定します)。その後、これらの集約値は `SELECT` クエリの実行時に使用され、`WHERE` 句の条件を満たし得ない大きなデータブロックをスキップすることで、ディスクから読み取るデータ量を削減します。 - -`GRANULARITY` 句は省略可能であり、`granularity_value` のデフォルト値は 1 です。 - -**例** - -```sql -CREATE TABLE table_name -( - u64 UInt64, - i32 Int32, - s String, - ... - INDEX idx1 u64 TYPE bloom_filter GRANULARITY 3, - INDEX idx2 u64 * i32 TYPE minmax GRANULARITY 3, - INDEX idx3 u64 * length(s) TYPE set(1000) GRANULARITY 4 -) ENGINE = MergeTree() -... -``` - -サンプルで定義したインデックスは、以下のクエリでは ClickHouse がディスクから読み取るデータ量を減らすために利用できます。 - -```sql -SELECT count() FROM table WHERE u64 == 10; -SELECT count() FROM table WHERE u64 * i32 >= 1234 -SELECT count() FROM table WHERE u64 * length(s) == 1234 -``` - -データスキップインデックスは複合列にも作成できます: - -```sql --- Map型のカラムに対して: -INDEX map_key_index mapKeys(map_column) TYPE bloom_filter -INDEX map_value_index mapValues(map_column) TYPE bloom_filter - --- Tuple型のカラムに対して: -INDEX tuple_1_index tuple_column.1 TYPE bloom_filter -INDEX tuple_2_index tuple_column.2 TYPE bloom_filter - --- Nested型のカラムに対して: -INDEX nested_1_index col.nested_col1 TYPE bloom_filter -INDEX nested_2_index col.nested_col2 TYPE bloom_filter -``` - -### スキップインデックスの種類 {#skip-index-types} - -`MergeTree` テーブルエンジンは、次の種類のスキップインデックスをサポートします。 -スキップインデックスをパフォーマンス最適化にどのように利用できるかについては、 -["ClickHouse のデータスキッピングインデックスを理解する"](/optimize/skipping-indexes) を参照してください。 - -* [`MinMax`](#minmax) インデックス -* [`Set`](#set) インデックス -* [`bloom_filter`](#bloom-filter) インデックス -* [`ngrambf_v1`](#n-gram-bloom-filter) インデックス -* [`tokenbf_v1`](#token-bloom-filter) インデックス - -#### MinMax スキップインデックス {#minmax} - -各インデックスグラニュールごとに、式の最小値と最大値が格納されます。 -(式の型が `tuple` の場合は、各タプル要素ごとに最小値と最大値が格納されます。) - -```text title="Syntax" -minmax -``` - -#### Set {#set} - -各インデックスグラニュールごとに、指定された式のユニークな値が最大 `max_rows` 個まで保存されます。 -`max_rows = 0` の場合は「すべてのユニークな値を保存する」ことを意味します。 - -```text title="Syntax" -set(max_rows) -``` - -#### ブルームフィルタ {#bloom-filter} - -各インデックスグラニュールは、指定された列に対して [Bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) を保持します。 - -```text title="Syntax" -bloom_filter([false_positive_rate]) -``` - -`false_positive_rate` パラメータには 0 から 1 の値を指定できます(既定値: `0.025`)。このパラメータは、偽陽性が発生する確率(これにより読み取られるデータ量が増加します)を指定します。 - -次のデータ型がサポートされています。 - -* `(U)Int*` -* `Float*` -* `Enum` -* `Date` -* `DateTime` -* `String` -* `FixedString` -* `Array` -* `LowCardinality` -* `Nullable` -* `UUID` -* `Map` - -:::note Map データ型: キーまたは値に対するインデックス作成の指定 -`Map` データ型では、クライアントは [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) または [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 関数を使用して、キーに対してインデックスを作成するか、値に対してインデックスを作成するかを指定できます。 -::: - -#### N-gram ブルームフィルタ {#n-gram-bloom-filter} - -各インデックスグラニュールには、指定された列の [n-gram](https://en.wikipedia.org/wiki/N-gram) に対する [bloom filter](https://en.wikipedia.org/wiki/Bloom_filter) が格納されます。 - -```text title="Syntax" -ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -| Parameter | Description | -| ------------------------------- | -------------------------------------------------------------------------------- | -| `n` | n-gram のサイズ | -| `size_of_bloom_filter_in_bytes` | Bloom フィルタのサイズ(バイト単位)。ここには `256` や `512` などの大きな値を指定できます。Bloom フィルタは高い圧縮率で格納できます。 | -| `number_of_hash_functions` | Bloom フィルタで使用されるハッシュ関数の数。 | -| `random_seed` | Bloom フィルタのハッシュ関数に使用するシード値。 | - -このインデックスが利用できるデータ型は次のとおりです。 - -* [`String`](/sql-reference/data-types/string.md) -* [`FixedString`](/sql-reference/data-types/fixedstring.md) -* [`Map`](/sql-reference/data-types/map.md) - -`ngrambf_v1` のパラメータを見積もるには、次の [ユーザー定義関数 (UDF)](/sql-reference/statements/create/function.md) を使用できます。 - -```sql title="UDFs for ngrambf_v1" -CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] -AS -(total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); - -CREATE FUNCTION bfEstimateBmSize [ON CLUSTER cluster] -AS -(total_number_of_all_grams, probability_of_false_positives) -> ceil((total_number_of_all_grams * log(probability_of_false_positives)) / log(1 / pow(2, log(2)))); - -CREATE FUNCTION bfEstimateFalsePositive [ON CLUSTER cluster] -AS -(total_number_of_all_grams, number_of_hash_functions, size_of_bloom_filter_in_bytes) -> pow(1 - exp(-number_of_hash_functions/ (size_of_bloom_filter_in_bytes / total_number_of_all_grams)), number_of_hash_functions); - -CREATE FUNCTION bfEstimateGramNumber [ON CLUSTER cluster] -AS -(number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) -``` - -これらの関数を使用するには、少なくとも次の 2 つのパラメータを指定する必要があります。 - -* `total_number_of_all_grams` -* `probability_of_false_positives` - -たとえば、ある granule に `4300` 個の ngram があり、偽陽性の確率を `0.0001` 未満にしたいとします。 -この場合、他のパラメータは次のクエリを実行することで推定できます。 - -```sql ---- フィルタ内のビット数を推定 -SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; - -┌─size_of_bloom_filter_in_bytes─┐ -│ 10304 │ -└───────────────────────────────┘ - ---- ハッシュ関数の数を推定 -SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_hash_functions - -┌─number_of_hash_functions─┐ -│ 13 │ -└──────────────────────────┘ -``` - -もちろん、これらの関数は他の条件のパラメータを見積もるためにも使用できます。 -上記の関数は、ブルームフィルター計算ツール[こちら](https://hur.st/bloomfilter)を参照しています。 - -#### トークンブルームフィルター {#token-bloom-filter} - -トークンブルームフィルターは `ngrambf_v1` と同様ですが、n-gram ではなく、英数字以外の文字で区切られたトークン(文字列のまとまり)を保存します。 - -```text title="Syntax" -tokenbf_v1(ブルームフィルタのサイズ(バイト), ハッシュ関数の数, ランダムシード) -``` - -#### スパースグラム Bloom フィルター {#sparse-grams-bloom-filter} - -スパースグラム Bloom フィルターは `ngrambf_v1` と似ていますが、n-gram の代わりに [スパースグラムトークン](/sql-reference/functions/string-functions.md/#sparseGrams) を使用します。 - -```text title="Syntax" -sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -### テキストインデックス {#text} - -全文検索をサポートします。詳細は[こちら](invertedindexes.md)を参照してください。 - -#### ベクトル類似性 {#vector-similarity} - -近似最近傍検索をサポートしています。詳細は[こちら](annindexes.md)をご覧ください。 - -### 関数のサポート {#functions-support} - -`WHERE` 句の条件式には、カラムを操作する関数呼び出しが含まれます。カラムがインデックスの一部である場合、ClickHouse は関数を評価する際にそのインデックスを利用しようとします。ClickHouse では、インデックスで利用できる関数のサブセットがインデックスタイプごとに異なります。 - -`set` 型のインデックスは、あらゆる関数で利用できます。その他のインデックスタイプは次のようにサポートされます。 - -| 関数(演算子)/ 索引 | 主キー | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | テキスト | -| ------------------------------------------------------------------------------------------------------------------------- | --- | ------ | -------------- | -------------- | ---------------- | ---------------- | ---- | -| [equals (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [less(`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greater(`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [以下 (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | - -`ngrambf_v1` によるクエリ最適化では、定数引数の値が ngram サイズより小さい関数は使用できません。 - -(*) `hasTokenCaseInsensitive` および `hasTokenCaseInsensitiveOrNull` を効果的に機能させるには、`tokenbf_v1` インデックスを小文字化されたデータ上に作成する必要があります。たとえば `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)` のように指定します。 - -:::note -Bloom filter では偽陽性が発生し得るため、`ngrambf_v1`、`tokenbf_v1`、`sparse_grams`、`bloom_filter` 各インデックスは、関数の結果が false になることを前提としてクエリを最適化する目的には使用できません。 - -例: - -* 最適化可能: - * `s LIKE '%test%'` - * `NOT s NOT LIKE '%test%'` - * `s = 1` - * `NOT s != 1` - * `startsWith(s, 'test')` -* 最適化不可能: - * `NOT s LIKE '%test%'` - * `s NOT LIKE '%test%'` - * `NOT s = 1` - * `s != 1` - * `NOT startsWith(s, 'test')` - ::: - -## プロジェクション {#projections} - -プロジェクションは [マテリアライズドビュー](/sql-reference/statements/create/view) に似ていますが、パーツレベルで定義されます。クエリで自動的に利用されることに加えて、一貫性を保証します。 - -:::note -プロジェクションを実装する際には、[force_optimize_projection](/operations/settings/settings#force_optimize_projection) の設定も併せて検討する必要があります。 -::: - -プロジェクションは、[FINAL](/sql-reference/statements/select/from#final-modifier) 修飾子付きの `SELECT` ステートメントではサポートされていません。 - -### プロジェクションクエリ {#projection-query} - -プロジェクションクエリは、プロジェクションを定義するクエリです。親テーブルからデータを暗黙的に選択します。 -**構文** - -```sql -SELECT [GROUP BY] [ORDER BY] -``` - -プロジェクションは [ALTER](/sql-reference/statements/alter/projection.md) 文で変更または削除できます。 - -### プロジェクションのストレージ {#projection-storage} - -プロジェクションはパーツディレクトリ内に保存されます。インデックスに似ていますが、匿名の `MergeTree` テーブルのパーツを保存するサブディレクトリを含みます。このテーブルは、プロジェクションの定義クエリに基づいて定義されます。`GROUP BY` 句がある場合、下層のストレージエンジンは [AggregatingMergeTree](aggregatingmergetree.md) になり、すべての集約関数は `AggregateFunction` に変換されます。`ORDER BY` 句がある場合、`MergeTree` テーブルはそれを主キー式として使用します。マージ処理中、プロジェクションのパーツは、そのストレージのマージルーチンによってマージされます。親テーブルのパーツのチェックサムは、プロジェクションのパーツと組み合わされます。その他のメンテナンス処理はスキップインデックスと同様です。 - -### クエリ解析 {#projection-query-analysis} - -1. プロジェクションがベーステーブルに対するクエリと同じ結果を生成し、与えられたクエリに対する回答として利用できるかを確認します。 -2. 読み取る必要があるグラニュール数が最も少ない、実行可能な最適な一致を選択します。 -3. プロジェクションを利用するクエリパイプラインは、元のパーツを利用するものとは異なります。あるパーツにプロジェクションが存在しない場合、その場で「投影」するためのパイプラインを追加できます。 - -## 同時データアクセス {#concurrent-data-access} - -テーブルへの同時アクセスにはマルチバージョン方式を使用します。つまり、テーブルで読み取りと更新が同時に行われている場合でも、クエリ時点で有効なパーツの集合からデータが読み取られます。長時間にわたるロックは発生しません。挿入処理が読み取り操作の妨げになることもありません。 - -テーブルからの読み取りは自動的に並列化されます。 - -## 列およびテーブルの TTL {#table_engine-mergetree-ttl} - -値の有効期間(time-to-live)を決定します。 - -`TTL` 句はテーブル全体にも、各列ごとにも設定できます。テーブルレベルの `TTL` では、ディスクやボリューム間でデータを自動的に移動するロジックや、すべてのデータが期限切れになったパーツを再圧縮するロジックも指定できます。 - -式は [Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md)、または [DateTime64](/sql-reference/data-types/datetime64.md) 型として評価されなければなりません。 - -**構文** - -列の TTL を設定する場合: - -```sql -TTL time_column -TTL time_column + interval -``` - -`interval` を定義するには、[time interval](/sql-reference/operators#operators-for-working-with-dates-and-times) 演算子を使用します。たとえば、次のようにします。 - -```sql -TTL date_time + INTERVAL 1 MONTH -TTL date_time + INTERVAL 15 HOUR -``` - -### カラム TTL {#mergetree-column-ttl} - -カラム内の値の有効期限が切れると、ClickHouse はそれらをカラムのデータ型におけるデフォルト値に置き換えます。データパート内のそのカラムの値がすべて有効期限切れになった場合、ClickHouse はファイルシステム上のそのデータパートからこのカラムを削除します。 - -`TTL` 句はキー列には使用できません。 - -**例** - -#### `TTL` を設定したテーブルの作成: {#creating-a-table-with-ttl} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int TTL d + INTERVAL 1 MONTH, - b Int TTL d + INTERVAL 1 MONTH, - c String -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d; -``` - -#### 既存のテーブルの列に TTL を追加する {#adding-ttl-to-a-column-of-an-existing-table} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 DAY; -``` - -#### 列のTTLを変更する {#altering-ttl-of-the-column} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 MONTH; -``` - -### テーブルの TTL {#mergetree-table-ttl} - -テーブルには、有効期限が切れた行を削除するための式と、[ディスクまたはボリューム](#table_engine-mergetree-multiple-volumes)間でパーツを自動的に移動するための複数の式を定義できます。テーブル内の行が有効期限切れになると、ClickHouse は対応する行をすべて削除します。パーツの移動または再圧縮が行われるためには、そのパーツ内のすべての行が `TTL` 式の条件を満たしている必要があります。 - -```sql -TTL expr - [DELETE|RECOMPRESS codec_name1|TO DISK 'xxx'|TO VOLUME 'xxx'][, DELETE|RECOMPRESS codec_name2|TO DISK 'aaa'|TO VOLUME 'bbb'] ... - [WHERE conditions] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] -``` - -TTL ルールの種類は、それぞれの TTL 式の後に続けて指定できます。これは、式が満たされた(現在時刻に達した)ときに実行されるアクションを決定します: - -* `DELETE` - 期限切れの行を削除します(デフォルトのアクション); -* `RECOMPRESS codec_name` - データパートを `codec_name` で再圧縮します; -* `TO DISK 'aaa'` - パートをディスク `aaa` に移動します; -* `TO VOLUME 'bbb'` - パートをボリューム `bbb` に移動します; -* `GROUP BY` - 期限切れの行を集約します。 - -`DELETE` アクションは `WHERE` 句と組み合わせて使用でき、フィルタリング条件に基づいて期限切れの行の一部のみを削除できます: - -```sql -TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' -``` - -`GROUP BY` 式はテーブルの主キーの先頭部分でなければなりません。 - -ある列が `GROUP BY` 式の一部ではなく、かつ `SET` 句で明示的に設定されていない場合、その列の値には、結果行でグループ化された行のいずれか 1 つからの値が入ります(あたかも集約関数 `any` が適用されたかのように振る舞います)。 - -**例** - -#### `TTL` を指定したテーブルの作成: {#creating-a-table-with-ttl-1} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE, - d + INTERVAL 1 WEEK TO VOLUME 'aaa', - d + INTERVAL 2 WEEK TO DISK 'bbb'; -``` - -#### テーブルの `TTL` を変更する: {#altering-ttl-of-the-table} - -```sql -ALTER TABLE tab - MODIFY TTL d + INTERVAL 1 DAY; -``` - -行が 1 か月後に期限切れとなるテーブルを作成します。期限切れとなった行のうち、日付が月曜日であるものが削除されます。 - -```sql -CREATE TABLE table_with_where -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE WHERE toDayOfWeek(d) = 1; -``` - -#### 期限切れの行が再圧縮されるテーブルの作成: {#creating-a-table-where-expired-rows-are-recompressed} - -```sql -CREATE TABLE table_for_recompression -( - d DateTime, - key UInt64, - value String -) ENGINE MergeTree() -ORDER BY tuple() -PARTITION BY key -TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPRESS CODEC(LZ4HC(10)) -SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; -``` - -有効期限切れの行を集約するテーブルを作成します。結果の行では、`x` にはグループ化された行全体での最大値が、`y` には最小値が、`d` にはグループ化された行からのいずれか 1 つの値が含まれます。 - -```sql -CREATE TABLE table_for_aggregation -( - d DateTime, - k1 Int, - k2 Int, - x Int, - y Int -) -ENGINE = MergeTree -ORDER BY (k1, k2) -TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); -``` - -### 期限切れデータの削除 {#mergetree-removing-expired-data} - -`TTL` が期限切れになったデータは、ClickHouse がデータパーツをマージする際に削除されます。 - -ClickHouse がデータの期限切れを検出すると、スケジュール外のマージを実行します。このようなマージの頻度を制御するには、`merge_with_ttl_timeout` を設定できます。値を低くしすぎると、多数のスケジュール外マージが実行され、多くのリソースを消費する可能性があります。 - -マージとマージの間に `SELECT` クエリを実行すると、期限切れデータが返される場合があります。これを避けるには、`SELECT` の前に [OPTIMIZE](/sql-reference/statements/optimize.md) クエリを実行してください。 - -**関連項目** - -* [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) 設定 - -## ディスクの種類 {#disk-types} - -ローカルブロックデバイスに加えて、ClickHouse は次のストレージタイプをサポートします: - -* [`s3` — S3 および MinIO 用](#table_engine-mergetree-s3) -* [`gcs` — GCS 用](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -* [`blob_storage_disk` — Azure Blob Storage 用](/operations/storing-data#azure-blob-storage) -* [`hdfs` — HDFS 用](/engines/table-engines/integrations/hdfs) -* [`web` — Web からの読み取り専用](/operations/storing-data#web-storage) -* [`cache` — ローカルキャッシュ用](/operations/storing-data#using-local-cache) -* [`s3_plain` — S3 へのバックアップ用](/operations/backup#backuprestore-using-an-s3-disk) -* [`s3_plain_rewritable` — S3 上の変更不可な非レプリケートテーブル用](/operations/storing-data.md#s3-plain-rewritable-storage) - -## データストレージで複数のブロックデバイスを利用する {#table_engine-mergetree-multiple-volumes} - -### はじめに {#introduction} - -`MergeTree` ファミリーのテーブルエンジンは、複数のブロックデバイス上にデータを保存できます。たとえば、特定のテーブルのデータが事実上「ホット」と「コールド」に分割されている場合に有用です。最新のデータは頻繁に参照されますが、必要な容量は小さくて済みます。対照的に、裾の重い履歴データはまれにしか参照されません。複数のディスクが利用可能な場合、「ホット」データは高速なディスク(たとえば NVMe SSD やメモリ上)に配置し、「コールド」データは比較的低速なディスク(たとえば HDD)上に配置できます。 - -データパーツは、`MergeTree` エンジンのテーブルにおける最小の移動可能な単位です。1 つのパーツに属するデータは 1 台のディスク上に保存されます。データパーツは、バックグラウンドでユーザー設定に従ってディスク間を移動できるほか、[ALTER](/sql-reference/statements/alter/partition) クエリを使用して移動することもできます。 - -### 用語 {#terms} - -* ディスク — ファイルシステムにマウントされたブロックデバイス。 -* デフォルトディスク — [path](/operations/server-configuration-parameters/settings.md/#path) サーバー設定で指定されたパス上にデータを保存するディスク。 -* ボリューム — 同一条件のディスクを順序付きで並べた集合([JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) に類似)。 -* ストレージポリシー — ボリュームの集合と、それらの間でデータを移動するためのルール。 - -ここで説明した各エンティティの名称は、システムテーブル [system.storage_policies](/operations/system-tables/storage_policies) および [system.disks](/operations/system-tables/disks) で確認できます。テーブルに対して設定済みのストレージポリシーのいずれかを適用するには、`MergeTree` エンジンファミリーのテーブルで `storage_policy` 設定を使用します。 - -### 設定 {#table_engine-mergetree-multiple-volumes_configure} - -ディスク、ボリューム、およびストレージポリシーは、`config.d` ディレクトリ内のファイルにある `` タグ内で宣言する必要があります。 - -:::tip -ディスクはクエリの `SETTINGS` セクション内で宣言することもできます。これは、例えば URL で公開されているディスクを一時的にアタッチしてアドホックな分析を行う場合に便利です。 -詳細については、[dynamic storage](/operations/storing-data#dynamic-configuration) を参照してください。 -::: - -設定の構造: - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - - ... - - - ... - -``` - -タグ: - -* `` — ディスク名。すべてのディスクで名前が重複しないようにする必要があります。 -* `path` — サーバーがデータ(`data` および `shadow` フォルダ)を保存するパス。末尾は '/' で終わる必要があります。 -* `keep_free_space_bytes` — 予約しておく空きディスク容量のバイト数。 - -ディスク定義の順序は重要ではありません。 - -ストレージポリシー構成のマークアップ: - -```xml - - ... - - - - - disk_name_from_disks_configuration - 1073741824 - round_robin - - - - - - - 0.2 - - - - - - - - ... - -``` - -タグ: - -* `policy_name_N` — ポリシー名。ポリシー名は一意である必要があります。 -* `volume_name_N` — ボリューム名。ボリューム名は一意である必要があります。 -* `disk` — ボリューム内のディスク。 -* `max_data_part_size_bytes` — いずれのボリューム上のディスクにも保存可能なパーツの最大サイズ。マージされたパーツのサイズが `max_data_part_size_bytes` より大きくなると見積もられた場合、そのパーツは次のボリュームに書き込まれます。この機能により、新規/小さいパーツをホット (SSD) ボリュームに保持し、サイズが大きくなったときにコールド (HDD) ボリュームへ移動できます。ポリシーにボリュームが 1 つしかない場合、この設定は使用しないでください。 -* `move_factor` — 空き容量がこの係数より小さくなったとき、自動的にデータを次のボリュームに移動し始めます (既定値は 0.1)。ClickHouse は既存のパーツをサイズの大きい順 (降順) にソートし、`move_factor` 条件を満たすのに十分な合計サイズとなるようにパーツを選択します。すべてのパーツの合計サイズが不十分な場合は、すべてのパーツが移動されます。 -* `perform_ttl_move_on_insert` — データパーツの INSERT 時の TTL move を無効化します。既定では (有効な場合)、挿入するデータパーツが TTL move ルールによりすでに期限切れとなっている場合、そのパーツは直ちに move ルールで指定されたボリューム/ディスクに配置されます。宛先ボリューム/ディスクが遅い場合 (例: S3)、これは INSERT を大幅に遅くする可能性があります。無効にした場合、すでに期限切れのデータパーツはいったんデフォルトボリュームに書き込まれ、その直後に TTL ボリュームへ移動されます。 -* `load_balancing` — ディスクのバランシングポリシー。`round_robin` または `least_used`。 -* `least_used_ttl_ms` — すべてのディスク上の空き容量情報を更新するためのタイムアウト (ミリ秒単位) を設定します (`0` - 常に更新、`-1` - 更新しない、既定値は `60000`)。注意: ディスクが ClickHouse 専用であり、オンラインのファイルシステムのリサイズ/縮小の対象にならない場合は `-1` を使用できますが、それ以外のケースでは推奨されません。最終的に空き容量の不適切な分散を招くためです。 -* `prefer_not_to_merge` — この設定は使用しないでください。このボリューム上のデータパーツのマージを無効化します (これは有害であり、パフォーマンス低下につながります)。この設定が有効な場合 (有効にしないでください)、このボリューム上でのマージは許可されません (これは望ましくありません)。これは (必要ありませんが) ClickHouse が遅いディスクをどのように扱うかを制御することを可能にします (しかし、何かを制御しようとしている時点で誤りであり、ClickHouse の方が賢いので、この設定は使用しないでください)。 -* `volume_priority` — ボリュームが埋められる優先度 (順序) を定義します。値が小さいほど優先度が高くなります。このパラメータの値は自然数とし、1 から N (最も低い優先度の値) までの範囲を欠番なくすべて網羅する必要があります。 - * *すべての* ボリュームにタグが付いている場合、指定された順序で優先されます。 - * 一部のボリュームのみにタグが付いている場合、タグのないボリュームは最も低い優先度となり、設定ファイル内で定義された順に優先されます。 - * ボリュームに *まったく* タグが付いていない場合、設定ファイル内で宣言された順序に対応して優先度が設定されます。 - * 2 つのボリュームが同じ優先度の値を持つことはできません。 - -構成例: - -```xml - - ... - - - - - disk1 - disk2 - - - - - - - - fast_ssd - 1073741824 - - - disk1 - - - 0.2 - - - - -
- jbod1 -
- - external - -
-
-
- ... -
-``` - -この例では、`hdd_in_order` ポリシーは [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling) 方式を実装しています。そのため、このポリシーは 1 つのボリューム(`single`)のみを定義し、そのすべてのディスク上にデータパーツをラウンドロビンで保存します。RAID を構成していないものの、同種のディスクが複数台システムにマウントされている場合、このようなポリシーは非常に有用です。各ディスクドライブ単体は信頼性が高くないことに注意し、レプリケーション係数を 3 以上にして補償することを検討してください。 - -システムに異なる種類のディスクが存在する場合は、代わりに `moving_from_ssd_to_hdd` ポリシーを使用できます。`hot` ボリュームは SSD ディスク(`fast_ssd`)で構成されており、このボリュームに保存できる 1 パートの最大サイズは 1GB です。サイズが 1GB を超えるすべてのパーツは、HDD ディスク `disk1` を含む `cold` ボリュームに直接保存されます。 -また、ディスク `fast_ssd` の使用率が 80% を超えると、バックグラウンドプロセスによってデータが `disk1` に転送されます。 - -ストレージポリシー内でのボリュームの列挙順序は、列挙されたボリュームのうち少なくとも 1 つに明示的な `volume_priority` パラメータが設定されていない場合に重要です。 -あるボリュームが満杯になると、データは次のボリュームへ移動されます。ディスクの列挙順も同様に重要であり、データはそれらに順番に保存されます。 - -テーブルを作成する際、そのテーブルに対して設定済みストレージポリシーのいずれかを適用できます。 - -```sql -CREATE TABLE table_with_non_default_policy ( - EventDate Date, - OrderID UInt64, - BannerID UInt64, - SearchPhrase String -) ENGINE = MergeTree -ORDER BY (OrderID, BannerID) -PARTITION BY toYYYYMM(EventDate) -SETTINGS storage_policy = 'moving_from_ssd_to_hdd' -``` - -`default` ストレージポリシーは、`` で指定された 1 つのディスクのみから構成される 1 つのボリュームだけを使用することを意味します。 -テーブル作成後でも [ALTER TABLE ... MODIFY SETTING] クエリを使用してストレージポリシーを変更できますが、新しいポリシーには、同じ名前を持つすべての既存ディスクおよびボリュームを含める必要があります。 - -データパーツのバックグラウンド移動を実行するスレッド数は、[background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 設定で変更できます。 - -### 詳細 {#details} - -`MergeTree` テーブルの場合、データは次のようなさまざまな方法でディスクに書き込まれます。 - -* 挿入(`INSERT` クエリ)の結果として。 -* バックグラウンドでのマージおよび[ミューテーション](/sql-reference/statements/alter#mutations)の実行中。 -* 別のレプリカからのダウンロード時。 -* パーティションのフリーズ [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) の結果として。 - -ミューテーションとパーティションのフリーズを除くすべての場合において、パーツは指定されたストレージポリシーに従ってボリュームおよびディスク上に保存されます。 - -1. パーツを保存するのに十分な空きディスク容量があり(`unreserved_space > current_part_size`)、かつ指定サイズのパーツの保存が許可されている(`max_data_part_size_bytes > current_part_size`)最初のボリューム(定義順)が選択されます。 -2. このボリューム内では、直前のデータチャンクを保存していたディスクの次のディスクであって、かつその空き容量がパーツサイズを上回るもの(`unreserved_space - keep_free_space_bytes > current_part_size`)が選択されます。 - -内部的には、ミューテーションとパーティションのフリーズは[ハードリンク](https://en.wikipedia.org/wiki/Hard_link)を利用します。異なるディスク間のハードリンクはサポートされないため、このような場合には結果として生成されるパーツは元のパーツと同じディスク上に保存されます。 - -バックグラウンドでは、設定ファイル内で宣言されたボリュームの順序に従い、空き容量(`move_factor` パラメータ)に基づいてパーツがボリューム間で移動されます。 -最後のボリュームから他のボリュームへの移動および他のボリュームから最初のボリュームへの移動は行われません。バックグラウンドでの移動は、システムテーブル [system.part_log](/operations/system-tables/part_log)(フィールド `type = MOVE_PART`)および [system.parts](/operations/system-tables/parts.md)(フィールド `path` と `disk`)を使用して監視できます。より詳細な情報はサーバーログで確認できます。 - -ユーザーは、クエリ [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition) を使用して、パーツまたはパーティションをあるボリュームから別のボリュームへ強制的に移動できます。バックグラウンド操作に対するすべての制約が考慮されます。このクエリは独自に移動処理を開始し、バックグラウンド操作の完了を待ちません。必要な空き容量が不足している場合や、必要条件のいずれかが満たされていない場合、ユーザーにはエラーメッセージが返されます。 - -データの移動はデータレプリケーションの動作を妨げません。そのため、同じテーブルに対しても、レプリカごとに異なるストレージポリシーを指定できます。 - -バックグラウンドでのマージおよびミューテーションが完了した後、古いパーツは一定時間(`old_parts_lifetime`)経過してから削除されます。 -この期間中、それらのパーツは他のボリュームやディスクには移動されません。したがってパーツが最終的に削除されるまでは、使用中ディスク容量の計算に引き続き含まれます。 - -ユーザーは、[JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) ボリュームの複数ディスクに新しい大きなパーツをバランス良く割り当てるために、設定 [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) を使用できます。 - -## データの保存に外部ストレージを使用する {#table_engine-mergetree-s3} - -[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) ファミリーのテーブルエンジンは、それぞれ `s3`、`azure_blob_storage`、`hdfs` タイプのディスクを使用して、データを `S3`、`AzureBlobStorage`、`HDFS` に保存できます。詳細は、[外部ストレージオプションの設定](/operations/storing-data.md/#configuring-external-storage)を参照してください。 - -ディスクタイプ `s3` を使用して [S3](https://aws.amazon.com/s3/) を外部ストレージとして利用する例を以下に示します。 - -設定マークアップ: - -```xml - - ... - - - s3 - true - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - -
Authorization: Bearer SOME-TOKEN
- your_base64_encoded_customer_key - your_kms_key_id - your_kms_encryption_context - true - - http://proxy1 - http://proxy2 - - 10000 - 5000 - 10 - 4 - 1000 - /var/lib/clickhouse/disks/s3/ - false -
- - cache - s3 - /var/lib/clickhouse/disks/s3_cache/ - 10Gi - -
- ... -
-``` - -[外部ストレージオプションの設定](/operations/storing-data.md/#configuring-external-storage)も参照してください。 - -:::note キャッシュ設定 -ClickHouse バージョン 22.3 から 22.7 までは異なるキャッシュ設定が使用されています。これらのバージョンのいずれかを使用している場合は、[ローカルキャッシュの使用](/operations/storing-data.md/#using-local-cache)を参照してください。 -::: - -## 仮想カラム {#virtual-columns} - -* `_part` — パーツ名。 -* `_part_index` — クエリ結果内でのパーツの連番インデックス番号。 -* `_part_starting_offset` — クエリ結果内でのパーツの累積開始行番号。 -* `_part_offset` — パーツ内での行番号。 -* `_part_granule_offset` — パーツ内でのグラニュール番号。 -* `_partition_id` — パーティション名。 -* `_part_uuid` — 一意のパーツ識別子(MergeTree 設定 `assign_part_uuids` が有効な場合)。 -* `_part_data_version` — パーツのデータバージョン(最小ブロック番号またはミューテーションバージョンのいずれか)。 -* `_partition_value` — `partition by` 式の値(タプル)。 -* `_sample_factor` — クエリで指定されたサンプル係数。 -* `_block_number` — 行に挿入時に割り当てられた元のブロック番号で、`enable_block_number_column` 設定が有効な場合はマージ時も保持される。 -* `_block_offset` — ブロック内の行に挿入時に割り当てられた元の行番号で、`enable_block_offset_column` 設定が有効な場合はマージ時も保持される。 -* `_disk_name` — ストレージで使用されるディスク名。 - -## カラム統計 {#column-statistics} - - - - - -`set allow_experimental_statistics = 1` を有効にすると、`*MergeTree*` ファミリーのテーブルに対する `CREATE` クエリの `COLUMNS` セクション内で統計を宣言します。 - -```sql -CREATE TABLE tab -( - a Int64 STATISTICS(TDigest, Uniq), - b Float64 -) -ENGINE = MergeTree -ORDER BY a -``` - -`ALTER` ステートメントを使用して統計情報を変更することもできます。 - -```sql -ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; -ALTER TABLE tab DROP STATISTICS a; -``` - -これらの軽量な統計情報は、列内の値の分布に関する情報を集約します。統計情報は各パートに保存され、挿入が行われるたびに更新されます。 -`set allow_statistics_optimize = 1` を有効にした場合にのみ、PREWHERE 最適化に利用できます。 - -### 利用可能な列統計の種類 {#available-types-of-column-statistics} - -* `MinMax` - - 数値型列に対する範囲フィルターの選択性を推定できるようにする、列の最小値と最大値。 - - 構文: `minmax` - -* `TDigest` - - 数値型列に対して近似パーセンタイル(例: 第90パーセンタイル)を計算できる [TDigest](https://github.com/tdunning/t-digest) スケッチ。 - - 構文: `tdigest` - -* `Uniq` - - 列に含まれる異なる値の個数を推定する [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) スケッチ。 - - 構文: `uniq` - -* `CountMin` - - 列内の各値の出現頻度を近似的にカウントする [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch) スケッチ。 - - 構文: `countmin` - -### サポートされているデータ型 {#supported-data-types} - -| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String または FixedString | -|-----------|----------------------------------------------------|---------------------------| -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | - -### サポートされる操作 {#supported-operations} - -| | 等値フィルター (==) | 範囲フィルター (`>, >=, <, <=`) | -|-----------|---------------------|------------------------------| -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | - -## 列レベルの設定 {#column-level-settings} - -一部の MergeTree の設定は列レベルで上書きできます。 - -* `max_compress_block_size` — テーブルに書き込む際に、圧縮前のデータブロックの最大サイズ。 -* `min_compress_block_size` — 次のマークを書き込む際に圧縮を行うために必要となる、圧縮前のデータブロックの最小サイズ。 - -例: - -```sql -CREATE TABLE tab -( - id Int64, - document String SETTINGS (min_compress_block_size = 16777216, max_compress_block_size = 16777216) -) -ENGINE = MergeTree -ORDER BY id -``` - -カラムレベルの設定は、たとえば [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md) を使用して変更または削除できます。 - -* カラム定義の `SETTINGS` を削除する: - -```sql -ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; -``` - -* 設定を変更します: - -```sql -ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; -``` - -* 1 つ以上の設定をリセットします。また、テーブルの CREATE クエリの列式から設定宣言も削除します。 - -```sql -ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md deleted file mode 100644 index b99d8c9080b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ /dev/null @@ -1,240 +0,0 @@ ---- -description: 'MergeTree と異なり、同じソートキーの値(`ORDER BY` テーブルセクションで定義されるものであり、 - `PRIMARY KEY` ではありません)を持つ重複エントリを削除します。' -sidebar_label: 'ReplacingMergeTree' -sidebar_position: 40 -slug: /engines/table-engines/mergetree-family/replacingmergetree -title: 'ReplacingMergeTree テーブルエンジン' -doc_type: 'reference' ---- - - - -# ReplacingMergeTree テーブルエンジン {#replacingmergetree-table-engine} - -このエンジンは、[MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) とは異なり、同じ[ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md)値(テーブル定義の `ORDER BY` セクションで指定されるもので、`PRIMARY KEY` ではありません)を持つ重複エントリを削除します。 - -データの重複排除はマージ時にのみ行われます。マージはバックグラウンドで不定のタイミングで実行されるため、そのタイミングを前提として計画することはできません。一部のデータは未処理のまま残る可能性があります。`OPTIMIZE` クエリを使用してオンデマンドでマージを実行することもできますが、`OPTIMIZE` クエリは大量のデータを読み書きするため、それに依存しないでください。 - -したがって、`ReplacingMergeTree` はバックグラウンドで重複データを削除してディスク使用量を削減する用途には適していますが、重複がまったく存在しないことを保証するものではありません。 - -:::note -ベストプラクティスやパフォーマンス最適化の方法を含む ReplacingMergeTree の詳細なガイドは[こちら](/guides/replacing-merge-tree)にあります。 -::: - - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = ReplacingMergeTree([ver [, is_deleted]]) -[PARTITION BY expr] -[ORDER BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -リクエストパラメータの詳細については、[ステートメントの説明](../../../sql-reference/statements/create/table.md)を参照してください。 - -:::note -行の一意性は `PRIMARY KEY` ではなく、テーブルの `ORDER BY` 句によって決定されます。 -::: - - -## ReplacingMergeTree のパラメーター {#replacingmergetree-parameters} - -### `ver` {#ver} - -`ver` — バージョン番号を保持するカラム。型は `UInt*`、`Date`、`DateTime` または `DateTime64`。省略可能なパラメーターです。 - -マージ時に、`ReplacingMergeTree` は同じソートキーを持つすべての行のうち 1 行だけを残します: - -* `ver` が設定されていない場合は、選択集合の中で最後の行。選択集合とは、マージに参加するパーツ集合内の行の集合のことです。もっとも最近作成されたパーツ(最後に挿入されたもの)が選択集合の中で最後になります。したがって、重複排除後は、もっとも新しい挿入からのそれぞれの一意なソートキーについて、いちばん最後の行が残ります。 -* `ver` が指定されている場合は、最大のバージョンを持つ行。複数の行で `ver` が同じ場合、それらには「`ver` が指定されていない場合」のルールが適用されます。つまり、もっとも最近に挿入された行が残ります。 - -Example: - -```sql --- ver なし - 最後に挿入されたものが '勝つ' -CREATE TABLE myFirstReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree -ORDER BY key; - -INSERT INTO myFirstReplacingMT Values (1, 'first', '2020-01-01 01:01:01'); -INSERT INTO myFirstReplacingMT Values (1, 'second', '2020-01-01 00:00:00'); - -SELECT * FROM myFirstReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ second │ 2020-01-01 00:00:00 │ -└─────┴─────────┴─────────────────────┘ - - --- ver あり - 最大の ver を持つ行が '勝つ' -CREATE TABLE mySecondReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree(eventTime) -ORDER BY key; - -INSERT INTO mySecondReplacingMT Values (1, 'first', '2020-01-01 01:01:01'); -INSERT INTO mySecondReplacingMT Values (1, 'second', '2020-01-01 00:00:00'); - -SELECT * FROM mySecondReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ first │ 2020-01-01 01:01:01 │ -└─────┴─────────┴─────────────────────┘ -``` - -### `is_deleted` {#is_deleted} - -`is_deleted` — マージ処理の際に、この行のデータが「状態」を表すのか、あるいは削除対象なのかを判定するために使用されるカラム名です。`1` は「削除された」行、`0` は「状態」の行を表します。 - -カラムのデータ型 — `UInt8`。 - -:::note -`is_deleted` は `ver` が使用されている場合にのみ有効化できます。 - -どのようなデータ操作であっても、バージョン番号は増加させる必要があります。2 つの挿入行が同じバージョン番号を持つ場合、後から挿入された行が保持されます。 - -デフォルトでは、ClickHouse はキーに対して最後の行を保持し、その行が削除行であっても保持します。これは、将来より低いバージョンの行が挿入された場合でも、安全に挿入でき、その削除行が引き続き適用されるようにするためです。 - -このような削除行を恒久的に削除するには、テーブル設定 `allow_experimental_replacing_merge_with_cleanup` を有効にし、次のいずれかを実行します。 - -1. テーブル設定 `enable_replacing_merge_with_cleanup_for_min_age_to_force_merge`、`min_age_to_force_merge_on_partition_only`、`min_age_to_force_merge_seconds` を設定します。パーティション内のすべてのパーツが `min_age_to_force_merge_seconds` より古い場合、ClickHouse はそれらを - 1 つのパーツにマージし、すべての削除行を削除します。 - -2. 手動で `OPTIMIZE TABLE table [PARTITION partition | PARTITION ID 'partition_id'] FINAL CLEANUP` を実行します。 - ::: - -Example: - -```sql --- ver と is_deleted を使用 -CREATE OR REPLACE TABLE myThirdReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime, - `is_deleted` UInt8 -) -ENGINE = ReplacingMergeTree(eventTime, is_deleted) -ORDER BY key -SETTINGS allow_experimental_replacing_merge_with_cleanup = 1; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 0); -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 1); -``` - - -select * from myThirdReplacingMT final; - -0 rows in set. Elapsed: 0.003 sec. - --- is_deleted が設定されている行を削除 -OPTIMIZE TABLE myThirdReplacingMT FINAL CLEANUP; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 00:00:00', 0); - -select * from myThirdReplacingMT final; - -┌─key─┬─someCol─┬───────────eventTime─┬─is_deleted─┐ -│ 1 │ first │ 2020-01-01 00:00:00 │ 0 │ -└─────┴─────────┴─────────────────────┴────────────┘ - -``` -``` - - -## クエリ句 {#query-clauses} - -`ReplacingMergeTree` テーブルを作成する場合は、`MergeTree` テーブルを作成する場合と同様に、同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) が必要です。 - -
- -テーブル作成の非推奨な方法 - -:::note -新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上記で説明した方法に切り替えてください。 -::: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] ReplacingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [ver]) -``` - -`ver` を除くすべてのパラメータは、`MergeTree` の場合と同じ意味を持ちます。 - -- `ver` - バージョンを表すカラム。省略可能なパラメータです。詳細については上記の説明を参照してください。 - -
- - - -## クエリ時の重複排除と `FINAL` {#query-time-de-duplication--final} - -マージ処理の際に、ReplacingMergeTree は `ORDER BY` 列(テーブル作成時に使用した列)の値を一意の識別子として用いて重複行を識別し、最も新しいバージョンのみを保持します。ただし、これはあくまで最終的な整合性しか提供せず、行が必ず重複排除されることを保証するものではないため、これに依存すべきではありません。その結果、更新行や削除行がクエリで考慮されることにより、クエリが誤った結果を返す可能性があります。 - -正しい結果を得るためには、バックグラウンドでのマージ処理に加えて、クエリ時の重複排除および削除済み行の除去を行う必要があります。これは `FINAL` 演算子を使用することで実現できます。例えば、次の例を考えてみます。 - -```sql -CREATE TABLE rmt_example -( - `number` UInt16 -) -ENGINE = ReplacingMergeTree -ORDER BY number - -INSERT INTO rmt_example SELECT floor(randUniform(0, 100)) AS number -FROM numbers(1000000000) - -0 rows in set. Elapsed: 19.958 sec. Processed 1.00 billion rows, 8.00 GB (50.11 million rows/s., 400.84 MB/s.) -``` - -`FINAL` を指定せずにクエリすると、不正確なカウント結果になります(具体的な値はマージ状況によって変動します)。 - -```sql -SELECT count() -FROM rmt_example - -┌─count()─┐ -│ 200 │ -└─────────┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -`FINAL` を追加すると、正しい結果が得られます。 - -```sql -SELECT count() -FROM rmt_example -FINAL - -┌─count()─┐ -│ 100 │ -└─────────┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -`FINAL` の詳細や `FINAL` のパフォーマンス最適化方法については、[ReplacingMergeTree に関する詳細ガイド](/guides/replacing-merge-tree) を参照することを推奨します。` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md deleted file mode 100644 index 721ae5661cc..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -description: 'SummingMergeTree は MergeTree エンジンを継承します。その主な機能は、パーツのマージ時に数値データを自動的に合計できることです。' -sidebar_label: 'SummingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/summingmergetree -title: 'SummingMergeTree テーブルエンジン' -doc_type: 'reference' ---- - - - -# SummingMergeTree テーブルエンジン {#summingmergetree-table-engine} - -このエンジンは [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) を継承しています。違いは、`SummingMergeTree` テーブルでデータパーツをマージする際に、ClickHouse が同じ主キー(より正確には同じ [ソートキー](../../../engines/table-engines/mergetree-family/mergetree.md))を持つすべての行を、数値データ型のカラムの値を合計した 1 行に置き換える点です。ソートキーの構成によって、1 つのキー値に多数の行が対応する場合、これにより必要なストレージ容量を大幅に削減し、データ取得の高速化を実現できます。 - -このエンジンは `MergeTree` と組み合わせて使用することを推奨します。生データ(完全なデータ)は `MergeTree` テーブルに保存し、集計済みデータの保存には `SummingMergeTree` を使用します(たとえばレポートを作成する場合など)。このようなアプローチにより、不適切に構成された主キーが原因で貴重なデータを失うことを防止できます。 - - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = SummingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -リクエストパラメータの説明については、[リクエストの説明](../../../sql-reference/statements/create/table.md)を参照してください。 - -### SummingMergeTree のパラメータ {#parameters-of-summingmergetree} - -#### カラム {#columns} - -`columns` - 値を集計(合計)するカラム名を含むタプルです。省略可能なパラメータです。 -カラムは数値型である必要があり、パーティションキーまたはソートキーに含めることはできません。 - -`columns` が指定されていない場合、ClickHouse はソートキーに含まれていない数値データ型のすべてのカラムの値を集計します。 - -### クエリ句 {#query-clauses} - -`SummingMergeTree` テーブルを作成する際には、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) を指定する必要があります。 - -
- テーブル作成の非推奨メソッド - - :::note - 新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存のプロジェクトも上で説明した方法に切り替えてください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] SummingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - `columns` 以外のすべてのパラメータは、`MergeTree` における意味と同じです。 - - * `columns` — 値を集計(合計)するカラム名を含むタプルです。省略可能なパラメータです。詳細は上記を参照してください。 -
- - -## 使用例 {#usage-example} - -次のテーブルを例にします。 - -```sql -CREATE TABLE summtt -( - key UInt32, - value UInt32 -) -ENGINE = SummingMergeTree() -ORDER BY key -``` - -データを挿入します: - -```sql -INSERT INTO summtt VALUES(1,1),(1,2),(2,1) -``` - -ClickHouse はすべての行を完全には集計しない場合があります([後述](#data-processing) を参照してください)。そのため、クエリでは集約関数 `sum` と `GROUP BY` 句を使用します。 - -```sql -SELECT key, sum(value) FROM summtt GROUP BY key -``` - -```text -┌─key─┬─sum(value)─┐ -│ 2 │ 1 │ -│ 1 │ 3 │ -└─────┴────────────┘ -``` - - -## データ処理 {#data-processing} - -データがテーブルに挿入されると、そのままの形で保存されます。ClickHouse は挿入されたデータパートを定期的にマージし、その際に同じ主キーを持つ行が合計され、各結果データパートごとに 1 行に置き換えられます。 - -ClickHouse はデータパートをマージする際、マージの結果として異なるデータパート同士に同じ主キーを持つ行が分かれて存在する場合があります。つまり、合計処理が不完全になる可能性があります。そのため、上記の例で説明したように、クエリでは集約関数 [`sum()`](/sql-reference/aggregate-functions/reference/sum) と `GROUP BY` 句を組み合わせて使用する必要があります。 - -### 集計に関する共通ルール {#common-rules-for-summation} - -数値データ型の列に含まれる値は合計されます。対象となる列の集合はパラメータ `columns` で定義されます。 - -合計対象となるすべての列の値が 0 であった場合、その行は削除されます。 - -列が主キーに含まれておらず、かつ合計対象でもない場合、既存の値の中から任意の値が選択されます。 - -主キーに含まれる列の値は合計されません。 - -### AggregateFunction 列における集計 {#the-summation-in-the-aggregatefunction-columns} - -[AggregateFunction 型](../../../sql-reference/data-types/aggregatefunction.md)の列に対しては、ClickHouse は [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md) エンジンと同様に、関数に従って集約処理を行います。 - -### ネストした構造 {#nested-structures} - -テーブルには特別な方法で処理されるネストしたデータ構造を含めることができます。 - -ネストしたテーブル名が `Map` で終わり、かつ次の条件を満たす 2 列以上を含んでいる場合: - -* 1 列目が数値型 `(*Int*, Date, DateTime)` または文字列型 `(String, FixedString)` である列(これを `key` と呼びます), -* その他の列が算術型 `(*Int*, Float32/64)` である列(これらを `(values...)` と呼びます)、 - -このネストしたテーブルは `key => (values...)` という対応関係を表すものとして解釈されます。このテーブルの行をマージする際には、2 つのデータセットの要素が `key` を基準にマージされ、それに対応する `(values...)` が合計されます。 - -例: - -```text -DROP TABLE IF EXISTS nested_sum; -CREATE TABLE nested_sum -( - date Date, - site UInt32, - hitsMap Nested( - browser String, - imps UInt32, - clicks UInt32 - ) -) ENGINE = SummingMergeTree -PRIMARY KEY (date, site); - -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Firefox', 'Opera'], [10, 5], [2, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Chrome', 'Firefox'], [20, 1], [1, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['IE'], [22], [0]); -INSERT INTO nested_sum VALUES ('2020-01-01', 10, ['Chrome'], [4], [3]); - -OPTIMIZE TABLE nested_sum FINAL; -- マージをエミュレートする - -SELECT * FROM nested_sum; -┌───────date─┬─site─┬─hitsMap.browser───────────────────┬─hitsMap.imps─┬─hitsMap.clicks─┐ -│ 2020-01-01 │ 10 │ ['Chrome'] │ [4] │ [3] │ -│ 2020-01-01 │ 12 │ ['Chrome','Firefox','IE','Opera'] │ [20,11,22,5] │ [1,3,0,1] │ -└────────────┴──────┴───────────────────────────────────┴──────────────┴────────────────┘ - -SELECT - site, - browser, - impressions, - clicks -FROM -( - SELECT - site, - sumMap(hitsMap.browser, hitsMap.imps, hitsMap.clicks) AS imps_map - FROM nested_sum - GROUP BY site -) -ARRAY JOIN - imps_map.1 AS browser, - imps_map.2 AS impressions, - imps_map.3 AS clicks; - -┌─site─┬─browser─┬─impressions─┬─clicks─┐ -│ 12 │ Chrome │ 20 │ 1 │ -│ 12 │ Firefox │ 11 │ 3 │ -│ 12 │ IE │ 22 │ 0 │ -│ 12 │ Opera │ 5 │ 1 │ -│ 10 │ Chrome │ 4 │ 3 │ -└──────┴─────────┴─────────────┴────────┘ -``` - - -データを取得する際は、`Map` を集計するために [sumMap(key, value)](../../../sql-reference/aggregate-functions/reference/summap.md) 関数を使用します。 - -ネストされたデータ構造の場合、集計対象のカラムのタプル内で、その構造に含まれるカラムを個別に指定する必要はありません。 - - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Using Aggregate Combinators in ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md deleted file mode 100644 index e993a76f5d0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -description: '継続的に変化するオブジェクトの状態を高速に書き込み、古い状態をバックグラウンドで削除できるテーブルエンジンです。' -sidebar_label: 'VersionedCollapsingMergeTree' -sidebar_position: 80 -slug: /engines/table-engines/mergetree-family/versionedcollapsingmergetree -title: 'VersionedCollapsingMergeTree テーブルエンジン' -doc_type: 'reference' ---- - - - -# VersionedCollapsingMergeTree テーブルエンジン {#versionedcollapsingmergetree-table-engine} - -このエンジンは次のことができます: - -- 継続的に変化するオブジェクトの状態を高速に書き込む。 -- 古いオブジェクトの状態をバックグラウンドで削除する。これにより必要なストレージ容量を大幅に削減できます。 - -詳細は [Collapsing](#table_engines_versionedcollapsingmergetree) のセクションを参照してください。 - -このエンジンは [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) を継承し、データパーツのマージアルゴリズムに行の折りたたみロジックを追加します。`VersionedCollapsingMergeTree` は [CollapsingMergeTree](../../../engines/table-engines/mergetree-family/collapsingmergetree.md) と同じ目的で使用されますが、異なる折りたたみアルゴリズムを採用しており、複数スレッドで任意の順序でデータを挿入できます。特に、`Version` 列は、行が誤った順序で挿入された場合でも、適切に行を折りたたむのに役立ちます。これに対して、`CollapsingMergeTree` は厳密に連続した順序での挿入のみをサポートします。 - - - -## テーブルの作成 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = VersionedCollapsingMergeTree(sign, version) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -クエリパラメータの説明は、[クエリの説明](../../../sql-reference/statements/create/table.md)を参照してください。 - -### エンジンのパラメータ {#engine-parameters} - -```sql -VersionedCollapsingMergeTree(サイン, バージョン) -``` - -| パラメータ | 説明 | 型 | -| --------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sign` | 行の種類を示す列の名前。`1` は「state」行、`-1` は「cancel」行を表します。 | [`Int8`](/sql-reference/data-types/int-uint) | -| `version` | オブジェクト状態のバージョンを表す列の名前。 | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) または [`DateTime64`](/sql-reference/data-types/datetime64) | - -### クエリ句 {#query-clauses} - -`VersionedCollapsingMergeTree` テーブルを作成する際は、`MergeTree` テーブルを作成する場合と同じ [句](../../../engines/table-engines/mergetree-family/mergetree.md) が必要です。 - -
- テーブル作成の非推奨メソッド - - :::note - 新しいプロジェクトではこの方法を使用しないでください。可能であれば、既存プロジェクトも上で説明した方法に切り替えてください。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] VersionedCollapsingMergeTree(date-column [, samp#table_engines_versionedcollapsingmergetreeling_expression], (primary, key), index_granularity, sign, version) - ``` - - `sign` と `version` 以外のすべてのパラメータは、`MergeTree` における意味と同じです。 - - * `sign` — 行の種類を示す列の名前。`1` は「state」行、`-1` は「cancel」行を表します。 - - 列のデータ型 — `Int8`。 - - * `version` — オブジェクト状態のバージョンを表す列の名前。 - - 列のデータ型は `UInt*` である必要があります。 -
- - -## 折りたたみ(Collapsing) {#table_engines_versionedcollapsingmergetree} - -### データ {#data} - -あるオブジェクトについて、継続的に変化するデータを保存する必要がある状況を考えます。オブジェクトごとに 1 行を持ち、変更があるたびにその行を更新するのは合理的に思えます。しかし、更新操作はストレージ上のデータを書き換える必要があるため、DBMS にとっては高コストかつ低速です。データを高速に書き込む必要がある場合、更新は適していませんが、その代わりにオブジェクトに対する変更を次のように逐次書き込むことができます。 - -行を書き込むときに `Sign` 列を使用します。`Sign = 1` の場合、その行はオブジェクトの状態を表す(これを「state 行」と呼びます)ことを意味します。`Sign = -1` の場合、同じ属性を持つオブジェクトの状態を取り消す(これを「cancel 行」と呼びます)ことを意味します。また、`Version` 列も使用し、オブジェクトの各状態を一意の番号で識別します。 - -たとえば、あるサイトでユーザーが閲覧したページ数と、そのページに滞在した時間を集計したいとします。ある時点で、ユーザーのアクティビティ状態を表す次の行を書き込みます。 - -```text -┌──────────────ユーザーID─┬─ページビュー数─┬─滞在時間─┬─署名─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -後でユーザーアクティビティの変更を記録し、それを次の 2 行で書き込みます。 - -```text -┌──────────────UserID─┬─ページビュー数─┬─継続時間─┬─符号─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -最初の行は、オブジェクト(ユーザー)の以前の状態を取り消します。この行には、`Sign` 以外の、取り消される状態のすべてのフィールドをコピーして含める必要があります。 - -2 行目には現在の状態が含まれます。 - -ユーザーアクティビティの最後の状態だけが必要なため、これらの行は - -```text -┌──────────────ユーザーID─┬─ページビュー─┬─継続時間─┬─サイン─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -は削除され、オブジェクトの無効(古い)状態が折りたたまれます。`VersionedCollapsingMergeTree` は、データパートをマージする際にこれを行います。 - -各変更ごとに 2 行が必要な理由については、[Algorithm](#table_engines-versionedcollapsingmergetree-algorithm) を参照してください。 - -**使用上の注意** - -1. データを書き込むプログラムは、取り消しができるようにオブジェクトの状態を保持しておく必要があります。"Cancel" 文字列には、主キーのフィールドのコピー、"state" 文字列のバージョン、および反対の `Sign` を含める必要があります。これにより初期のストレージ使用量は増加しますが、高速にデータを書き込むことができます。 -2. 列内で長く伸び続ける配列は、書き込み時の負荷によりエンジンの効率を低下させます。データが単純であればあるほど効率は高くなります。 -3. `SELECT` の結果は、オブジェクト変更履歴の一貫性に大きく依存します。挿入するデータを準備する際は注意してください。セッション深度のような本来非負であるメトリクスに対して負の値が得られるなど、不整合なデータでは予測不能な結果になる可能性があります。 - -### Algorithm {#table_engines-versionedcollapsingmergetree-algorithm} - -ClickHouse がデータパートをマージする際、同じ主キーとバージョンを持ち、`Sign` が異なる行のペアを削除します。行の順序は関係ありません。 - -ClickHouse がデータを挿入する際には、行は主キーでソートされます。`Version` 列が主キーに含まれていない場合、ClickHouse はそれを暗黙的に主キーの最後のフィールドとして追加し、その並び替えに使用します。 - - -## データの選択 {#selecting-data} - -ClickHouse は、同じプライマリキーを持つすべての行が、同じ結果のデータパート内、あるいは同じ物理サーバー上に存在することを保証しません。これは、データを書き込むときと、その後にデータパートをマージするときの両方について当てはまります。さらに、ClickHouse は `SELECT` クエリを複数スレッドで処理するため、結果の行の順序を予測できません。つまり、`VersionedCollapsingMergeTree` テーブルから完全に「折りたたまれた」データを取得する必要がある場合は、集約処理が必要になります。 - -折りたたみを最終的に確定させるには、`GROUP BY` 句と、符号を考慮した集約関数を使ってクエリを書きます。たとえば数量を計算するには、`count()` の代わりに `sum(Sign)` を使用します。何らかの合計を計算するには、`sum(x)` の代わりに `sum(Sign * x)` を使用し、さらに `HAVING sum(Sign) > 0` を追加します。 - -`count`、`sum`、`avg` といった集約は、この方法で計算できます。`uniq` 集約は、オブジェクトに少なくとも 1 つの未折りたたみ状態がある場合に計算できます。`min` および `max` 集約は、`VersionedCollapsingMergeTree` が折りたたまれた状態の値の履歴を保存しないため、計算できません。 - -集約なしで「折りたたみ」を行ったデータを抽出する必要がある場合(たとえば、最新の値が特定の条件に一致する行が存在するかを確認する場合)は、`FROM` 句に対して `FINAL` 修飾子を使用できます。このアプローチは非効率的であり、大きなテーブルには使用すべきではありません。 - - - -## 使用例 {#example-of-use} - -サンプルデータ: - -```text -┌──────────────ユーザーID─┬─ページビュー─┬─滞在時間─┬─符号─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -テーブルの作成: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8, - Version UInt8 -) -ENGINE = VersionedCollapsingMergeTree(Sign, Version) -ORDER BY UserID -``` - -データを挿入する: - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1, 1),(4324182021466249494, 6, 185, 1, 2) -``` - -2つの `INSERT` クエリを使用して、2つの異なるデータパーツを作成します。1つのクエリでデータを挿入した場合、ClickHouse は1つのデータパーツしか作成せず、マージ処理は行いません。 - -データの取得: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────ユーザーID─┬─ページビュー─┬─滞在時間─┬─符号─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -┌──────────────ユーザーID─┬─ページビュー─┬─滞在時間─┬─符号─┬─バージョン─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -ここで何が起きていて、折りたたまれるはずの部分はどこにあるのでしょうか? -2 つの `INSERT` クエリを使って 2 つのデータパーツを作成しました。`SELECT` クエリは 2 つのスレッドで実行され、その結果、行の並び順はランダムになっています。 -データパーツはまだマージされていないため、折りたたみは発生していません。ClickHouse は、いつ行われるか予測できないタイミングでデータパーツをマージします。 - -だからこそ、集約が必要になります。 - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration, - Version -FROM UAct -GROUP BY UserID, Version -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Version─┐ -│ 4324182021466249494 │ 6 │ 185 │ 2 │ -└─────────────────────┴───────────┴──────────┴─────────┘ -``` - -集約が不要で、強制的に折りたたみを行いたい場合は、`FROM` 句に `FINAL` 修飾子を指定できます。 - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────ユーザーID─┬─ページビュー─┬─滞在時間─┬─サイン─┬─バージョン─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -これはデータを抽出する非常に非効率な方法です。大規模なテーブルには使用しないでください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md deleted file mode 100644 index e3f964afeaa..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ /dev/null @@ -1,333 +0,0 @@ ---- -description: 'Alias テーブルエンジンは、別のテーブルへの透過的なプロキシとして機能します。すべての操作は対象テーブルに転送され、エイリアス自体はデータを保持しません。' -sidebar_label: 'Alias' -sidebar_position: 5 -slug: /engines/table-engines/special/alias -title: 'Alias テーブルエンジン' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# Alias テーブルエンジン {#alias-table-engine} - - - -`Alias` エンジンは、別のテーブルへのプロキシを作成します。すべての読み取りおよび書き込み操作は対象テーブルに転送され、エイリアス自体はデータを保持せず、対象テーブルへの参照のみを保持します。 - -:::info -これは実験的な機能であり、将来のリリースで後方互換性を損なう形で変更される可能性があります。 -`Alias` テーブルエンジンを使用するには、 -[allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine) 設定を有効にしてください。 -次のコマンドを実行します: `set allow_experimental_alias_table_engine = 1`。 -::: - -## テーブルの作成 {#creating-a-table} - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_table) -``` - -または、データベース名を明示的に指定する場合: - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_db, target_table) -``` - -:::note -`Alias` テーブルでは、明示的な列定義を行うことはできません。列はターゲットテーブルから自動的に継承されます。これにより、エイリアスは常にターゲットテーブルのスキーマと一致します。 -::: - - -## エンジンパラメータ {#engine-parameters} - -- **`target_db (optional)`** — 対象テーブルを含むデータベースの名前(省略可能)。 -- **`target_table`** — 対象テーブルの名前。 - -## サポートされている操作 {#supported-operations} - -`Alias` テーブルエンジンは、主要な操作をすべてサポートします。 - -### ターゲットテーブルに対する操作 {#operations-on-target} - -これらの操作はターゲットテーブルに対してプロキシ経由で実行されます: - -| Operation | Support | Description | -|-----------|---------|-------------| -| `SELECT` | ✅ | ターゲットテーブルからデータを読み取る | -| `INSERT` | ✅ | ターゲットテーブルにデータを書き込む | -| `INSERT SELECT` | ✅ | ターゲットテーブルへのバッチ挿入を行う | -| `ALTER TABLE ADD COLUMN` | ✅ | ターゲットテーブルに列を追加する | -| `ALTER TABLE MODIFY SETTING` | ✅ | ターゲットテーブルの設定を変更する | -| `ALTER TABLE PARTITION` | ✅ | ターゲットに対するパーティション操作 (DETACH/ATTACH/DROP) を行う | -| `ALTER TABLE UPDATE` | ✅ | ターゲットテーブルの行を更新する (mutation) | -| `ALTER TABLE DELETE` | ✅ | ターゲットテーブルから行を削除する (mutation) | -| `OPTIMIZE TABLE` | ✅ | ターゲットテーブルを最適化する (パートをマージ) | -| `TRUNCATE TABLE` | ✅ | ターゲットテーブルを空にする | - -### エイリアス自体への操作 {#operations-on-alias} - -これらの操作はエイリアスのみに作用し、ターゲットテーブルには**影響しません**。 - -| Operation | Support | Description | -|-----------|---------|-------------| -| `DROP TABLE` | ✅ | エイリアスのみを削除し、ターゲットテーブルには変更が加わらない | -| `RENAME TABLE` | ✅ | エイリアスの名前だけを変更し、ターゲットテーブルには変更が加わらない | - -## 使用例 {#usage-examples} - -### 基本的なエイリアスの作成 {#basic-alias-creation} - -同一データベース内で簡単なエイリアスを作成します。 - -```sql --- ソーステーブルを作成 -CREATE TABLE source_data ( - id UInt32, - name String, - value Float64 -) ENGINE = MergeTree -ORDER BY id; - --- データを挿入 -INSERT INTO source_data VALUES (1, 'one', 10.1), (2, 'two', 20.2); - --- エイリアスを作成 -CREATE TABLE data_alias ENGINE = Alias('source_data'); - --- エイリアス経由でクエリ -SELECT * FROM data_alias; -``` - -```text -┌─id─┬─name─┬─value─┐ -│ 1 │ one │ 10.1 │ -│ 2 │ two │ 20.2 │ -└────┴──────┴───────┘ -``` - - -### データベース間エイリアス {#cross-database-alias} - -別のデータベース内のテーブルを参照するエイリアスを作成します。 - -```sql --- データベースを作成 -CREATE DATABASE db1; -CREATE DATABASE db2; - --- db1にソーステーブルを作成 -CREATE TABLE db1.events ( - timestamp DateTime, - event_type String, - user_id UInt32 -) ENGINE = MergeTree -ORDER BY timestamp; - --- db1.eventsを指すエイリアスをdb2に作成 -CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); - --- またはdatabase.table形式を使用 -CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); - --- どちらのエイリアスも同じように動作します -INSERT INTO db2.events_alias VALUES (now(), 'click', 100); -SELECT * FROM db2.events_alias2; -``` - - -### エイリアス経由での書き込み操作 {#write-operations} - -すべての書き込みはターゲットテーブルに転送されます。 - -```sql -CREATE TABLE metrics ( - ts DateTime, - metric_name String, - value Float64 -) ENGINE = MergeTree -ORDER BY ts; - -CREATE TABLE metrics_alias ENGINE = Alias('metrics'); - --- エイリアス経由で挿入 -INSERT INTO metrics_alias VALUES - (now(), 'cpu_usage', 45.2), - (now(), 'memory_usage', 78.5); - --- SELECTで挿入 -INSERT INTO metrics_alias -SELECT now(), 'disk_usage', number * 10 -FROM system.numbers -LIMIT 5; - --- ターゲットテーブルにデータが格納されていることを確認 -SELECT count() FROM metrics; -- 7を返す -SELECT count() FROM metrics_alias; -- 7を返す -``` - - -### スキーマの変更 {#schema-modification} - -ALTER 操作は対象テーブルのスキーマを変更します。 - -```sql -CREATE TABLE users ( - id UInt32, - name String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE users_alias ENGINE = Alias('users'); - --- エイリアス経由でカラムを追加 -ALTER TABLE users_alias ADD COLUMN email String DEFAULT ''; - --- カラムは対象テーブルに追加される -DESCRIBE users; -``` - -```text -┌─name──┬─type───┬─default_type─┬─default_expression─┐ -│ id │ UInt32 │ │ │ -│ name │ String │ │ │ -│ email │ String │ DEFAULT │ '' │ -└───────┴────────┴──────────────┴────────────────────┘ -``` - - -### データの変更 {#data-mutations} - -UPDATE 文および DELETE 文がサポートされています。 - -```sql -CREATE TABLE products ( - id UInt32, - name String, - price Float64, - status String DEFAULT 'active' -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE products_alias ENGINE = Alias('products'); - -INSERT INTO products_alias VALUES - (1, 'item_one', 100.0, 'active'), - (2, 'item_two', 200.0, 'active'), - (3, 'item_three', 300.0, 'inactive'); - --- エイリアス経由で更新 -ALTER TABLE products_alias UPDATE price = price * 1.1 WHERE status = 'active'; - --- エイリアス経由で削除 -ALTER TABLE products_alias DELETE WHERE status = 'inactive'; - --- 変更は対象テーブルに適用される -SELECT * FROM products ORDER BY id; -``` - -```text -┌─id─┬─name─────┬─price─┬─status─┐ -│ 1 │ item_one │ 110.0 │ active │ -│ 2 │ item_two │ 220.0 │ active │ -└────┴──────────┴───────┴────────┘ -``` - - -### パーティション操作 {#partition-operations} - -パーティション化されたテーブルでは、パーティション操作はそのまま伝播されます。 - -```sql -CREATE TABLE logs ( - date Date, - level String, - message String -) ENGINE = MergeTree -PARTITION BY toYYYYMM(date) -ORDER BY date; - -CREATE TABLE logs_alias ENGINE = Alias('logs'); - -INSERT INTO logs_alias VALUES - ('2024-01-15', 'INFO', 'message1'), - ('2024-02-15', 'ERROR', 'message2'), - ('2024-03-15', 'INFO', 'message3'); - --- エイリアス経由でパーティションをデタッチ -ALTER TABLE logs_alias DETACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- 2を返す(パーティション202402はデタッチ済み) - --- パーティションを再アタッチ -ALTER TABLE logs_alias ATTACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- 3を返す -``` - - -### テーブル最適化 {#table-optimization} - -ターゲットテーブル内のパーツをマージする処理を最適化します。 - -```sql -CREATE TABLE events ( - id UInt32, - data String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE events_alias ENGINE = Alias('events'); - --- 複数の INSERT により複数のパートが作成される -INSERT INTO events_alias VALUES (1, 'data1'); -INSERT INTO events_alias VALUES (2, 'data2'); -INSERT INTO events_alias VALUES (3, 'data3'); - --- パート数を確認 -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; - --- エイリアス経由で最適化 -OPTIMIZE TABLE events_alias FINAL; - --- ターゲットテーブルでパートがマージされる -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; -- 1 を返す -``` - - -### エイリアスの管理 {#alias-management} - -エイリアスはそれぞれ独立して名前を変更したり削除したりできます。 - -```sql -CREATE TABLE important_data ( - id UInt32, - value String -) ENGINE = MergeTree -ORDER BY id; - -INSERT INTO important_data VALUES (1, 'critical'), (2, 'important'); - -CREATE TABLE old_alias ENGINE = Alias('important_data'); - --- エイリアスの名前を変更(対象テーブルは変更されません) -RENAME TABLE old_alias TO new_alias; - --- 同じテーブルに別のエイリアスを作成 -CREATE TABLE another_alias ENGINE = Alias('important_data'); - --- エイリアスを1つ削除(対象テーブルと他のエイリアスは変更されません) -DROP TABLE new_alias; - -SELECT * FROM another_alias; -- 引き続き動作します -SELECT count() FROM important_data; -- データは保持されており、2を返します -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md deleted file mode 100644 index 4a68e468807..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: '書き込むデータをあらかじめ RAM 上でバッファリングし、定期的に別のテーブルへフラッシュします。読み取り時には、バッファと別のテーブルの両方から同時にデータを読み出します。' -sidebar_label: 'Buffer' -sidebar_position: 120 -slug: /engines/table-engines/special/buffer -title: 'Buffer テーブルエンジン' -doc_type: 'reference' ---- - -# Buffer テーブルエンジン {#buffer-table-engine} - -書き込み対象のデータを RAM 上でバッファリングし、定期的に別のテーブルへフラッシュします。読み取り時には、バッファとその別のテーブルの両方から同時にデータを読み取ります。 - -:::note -Buffer テーブルエンジンの代替として推奨されるのは、[非同期インサート](/guides/best-practices/asyncinserts.md) を有効化することです。 -::: - -```sql -Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) -``` - -### エンジンパラメータ {#engine-parameters} - -#### `database` {#database} - -`database` – データベース名。`currentDatabase()` もしくは文字列を返す他の定数式を使用できます。 - -#### `table` {#table} - -`table` – データをフラッシュする対象のテーブル。 - -#### `num_layers` {#num_layers} - -`num_layers` – 並列レイヤー数。物理的には、テーブルは互いに独立したバッファが `num_layers` 個ある形で表現されます。 - -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, および `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} - -バッファからデータをフラッシュするための条件。 - -### オプションのエンジンパラメータ {#optional-engine-parameters} - -#### `flush_time`, `flush_rows`, および `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} - -バックグラウンドでバッファからデータをフラッシュするための条件(省略または 0 の場合は、`flush*` パラメータは存在しないものとして扱われます)。 - -すべての `min*` 条件が満たされるか、少なくとも 1 つの `max*` 条件が満たされた場合、データはバッファからフラッシュされ、出力先テーブルに書き込まれます。 - -また、少なくとも 1 つの `flush*` 条件が満たされると、バックグラウンドでフラッシュが開始されます。これは `max*` とは異なり、`flush*` によって、Buffer テーブルへの `INSERT` クエリにレイテンシを追加しないように、バックグラウンドフラッシュを個別に構成できます。 - -#### `min_time`, `max_time`, および `flush_time` {#min_time-max_time-and-flush_time} - -最初の書き込み時点からの経過時間(秒)に関する条件。 - -#### `min_rows`, `max_rows`, および `flush_rows` {#min_rows-max_rows-and-flush_rows} - -バッファ内の行数に関する条件。 - -#### `min_bytes`, `max_bytes`, および `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} - -バッファ内のバイト数に関する条件。 - -書き込み操作中、データは 1 つ以上のランダムなバッファ(`num_layers` で構成)に挿入されます。あるいは、挿入するデータのまとまりが十分に大きい場合(`max_rows` または `max_bytes` を超える場合)、バッファを経由せずに直接出力先テーブルに書き込まれます。 - -データをフラッシュする条件は、`num_layers` の各バッファごとに個別に評価されます。たとえば、`num_layers = 16` かつ `max_bytes = 100000000` の場合、RAM の最大使用量は 1.6 GB です。 - -例: - -```sql -CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, 100, 10000, 1000000, 10000000, 100000000) -``` - -`merge.hits` と同じ構造を持ち、Buffer エンジンを使用する `merge.hits_buffer` テーブルを作成します。このテーブルに書き込むと、データは RAM 上にバッファされ、その後で 'merge.hits' テーブルに書き込まれます。1 つのバッファが作成され、以下のいずれかの条件を満たすとデータがフラッシュされます: - -* 前回のフラッシュから 100 秒経過した場合(`max_time`) -* 100 万行が書き込まれた場合(`max_rows`) -* 100 MB のデータが書き込まれた場合(`max_bytes`) -* 10 秒が経過しており(`min_time`)、かつ 1 万行(`min_rows`)および 10 MB(`min_bytes`)のデータが書き込まれた場合 - -たとえば、1 行だけが書き込まれた場合でも、100 秒後には必ずフラッシュされます。一方、多数の行が書き込まれている場合は、それより早くデータがフラッシュされます。 - -サーバーの停止時や `DROP TABLE` または `DETACH TABLE` が実行されたときは、バッファされたデータも宛先テーブルにフラッシュされます。 - -データベース名およびテーブル名には、シングルクォートで囲んだ空文字列を設定できます。これは、宛先テーブルが存在しないことを示します。この場合、データのフラッシュ条件に達すると、バッファは単にクリアされます。これは、メモリ上に一定期間分のデータだけを保持しておきたい場合に有用です。 - -Buffer テーブルから読み取る場合、データはバッファと宛先テーブル(存在する場合)の両方から処理されます。 -なお、Buffer テーブルはインデックスをサポートしません。つまり、バッファ内のデータはフルスキャンされるため、大きなバッファでは低速になる可能性があります(下位テーブル内のデータについては、そのテーブルがサポートするインデックスが使用されます)。 - -Buffer テーブル内のカラム集合が下位テーブルのカラム集合と一致しない場合、両方のテーブルに共通するカラムのみが挿入されます。 - -Buffer テーブルと下位テーブルのいずれかのカラムで型が一致しない場合、エラーメッセージがサーバーログに出力され、バッファは消去されます。 -バッファがフラッシュされる時点で下位テーブルが存在しない場合も同様です。 - -:::note -2021 年 10 月 26 日以前のリリースで Buffer テーブルに対して ALTER を実行すると、`Block structure mismatch` エラーが発生します([#15117](https://github.com/ClickHouse/ClickHouse/issues/15117) および [#30565](https://github.com/ClickHouse/ClickHouse/pull/30565) を参照)。そのため、Buffer テーブルを削除して再作成する以外に方法はありません。Buffer テーブルに対して ALTER を実行する前に、お使いのリリースでこのエラーが修正されていることを確認してください。 -::: - -サーバーが異常終了した場合、バッファ内のデータは失われます。 - -`FINAL` と `SAMPLE` は Buffer テーブルでは正しく動作しません。これらの条件は宛先テーブルには渡されますが、バッファ内のデータ処理には使用されません。これらの機能が必要な場合は、Buffer テーブルは書き込み専用とし、読み取りは宛先テーブルからのみ行うことを推奨します。 - -Buffer テーブルにデータを追加する際は、バッファの 1 つがロックされます。そのため、同時にテーブルから読み取り操作が行われていると遅延が発生します。 - -Buffer テーブルに挿入されたデータは、下位テーブル内では異なる順序や異なるブロックに配置される場合があります。このため、Buffer テーブルを CollapsingMergeTree への正しい書き込みに用いるのは困難です。問題を避けるには、`num_layers` を 1 に設定できます。 - -宛先テーブルがレプリケートされている場合、Buffer テーブルに書き込むと、レプリケートされたテーブルに期待される特性の一部が失われます。行の順序やデータパーツのサイズがランダムに変化することで、データ重複排除が機能しなくなり、レプリケートされたテーブルに対して信頼できる「厳密に一度だけ」の書き込みを行うことが不可能になります。 - -これらの欠点により、Buffer テーブルの使用はごく限られたケースにのみ推奨されます。 - -Buffer テーブルは、単位時間あたりに多数のサーバーから大量の INSERT が送信され、挿入前にデータをバッファリングできないために INSERT が十分な速度で実行できない場合に使用されます。 - -なお、Buffer テーブルであっても、1 行ずつデータを挿入することには意味がありません。この方法では、スループットは毎秒数千行程度にとどまり、より大きなブロック単位でデータを挿入することで、毎秒 100 万行を超えるスループットを得ることができます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md deleted file mode 100644 index 82534891b7d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -description: '`Dictionary` エンジンは、辞書データを ClickHouse のテーブルとして扱えるようにします。' -sidebar_label: 'Dictionary' -sidebar_position: 20 -slug: /engines/table-engines/special/dictionary -title: 'Dictionary テーブルエンジン' -doc_type: 'リファレンス' ---- - -# Dictionary テーブルエンジン {#dictionary-table-engine} - -`Dictionary` エンジンは、[dictionary](../../../sql-reference/dictionaries/index.md) のデータを ClickHouse のテーブルとして扱います。 - -## 例 {#example} - -例として、次のように構成された `products` 辞書を考えます。 - -```xml - - - products - - -
products
- DSN=some-db-server - - - - 300 - 360 - - - - - - - product_id - - - title - String - - - - - -``` - -辞書データにクエリを実行する: - -```sql -SELECT - name, - type, - key, - attribute.names, - attribute.types, - bytes_allocated, - element_count, - source -FROM system.dictionaries -WHERE name = 'products' -``` - -```text -┌─name─────┬─type─┬─key────┬─attribute.names─┬─attribute.types─┬─bytes_allocated─┬─element_count─┬─source──────────┐ -│ products │ Flat │ UInt64 │ ['title'] │ ['String'] │ 23065376 │ 175032 │ ODBC: .products │ -└──────────┴──────┴────────┴─────────────────┴─────────────────┴─────────────────┴───────────────┴─────────────────┘ -``` - -この形式で辞書データを取得するには、[dictGet*](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull) 関数を使用できます。 - -生データを取得する必要がある場合や、`JOIN` 操作を実行する場合には、このビューは有用ではありません。これらの場合には、`Dictionary` エンジンを使用することで、辞書データをテーブルとして表示できます。 - -構文: - -```sql -CREATE TABLE %table_name% (%fields%) engine = Dictionary(%dictionary_name%)` -``` - -使用例: - -```sql -CREATE TABLE products (product_id UInt64, title String) ENGINE = Dictionary(products); -``` - -では - -テーブルの内容を確認しましょう。 - -```sql -SELECT * FROM products LIMIT 1; -``` - -```text -┌────product_id─┬─title───────────┐ -│ 152689 │ Some item │ -└───────────────┴─────────────────┘ -``` - -**関連項目** - -* [Dictionary 関数](/sql-reference/table-functions/dictionary) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md deleted file mode 100644 index 847d7b76a0e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ /dev/null @@ -1,251 +0,0 @@ ---- -description: 'Distributed エンジンを持つテーブルは自身では一切データを保存せず、複数サーバー上での分散クエリ処理を可能にします。読み取り処理は自動的に並列化されます。読み取り時には、存在する場合はリモートサーバー上のテーブルインデックスが利用されます。' -sidebar_label: 'Distributed' -sidebar_position: 10 -slug: /engines/table-engines/special/distributed -title: 'Distributed テーブルエンジン' -doc_type: 'reference' ---- - - - -# Distributed テーブルエンジン {#distributed-table-engine} - -:::warning ClickHouse Cloud における Distributed エンジン -ClickHouse Cloud で Distributed テーブルエンジンを作成するには、[`remote` および `remoteSecure`](../../../sql-reference/table-functions/remote) テーブル関数を使用します。 -`Distributed(...)` 構文は ClickHouse Cloud では使用できません。 -::: - -Distributed エンジンを持つテーブル自体はデータを一切保存しませんが、複数のサーバーでの分散クエリ処理を可能にします。 -読み取り処理は自動的に並列化されます。読み取り時には、リモートサーバー上にテーブルインデックスが存在する場合、それらが利用されます。 - - - -## テーブルの作成 {#distributed-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) -[SETTINGS name=value, ...] -``` - -### テーブルから {#distributed-from-a-table} - -`Distributed` テーブルが現在のサーバー上のテーブルを参照している場合、そのテーブルのスキーマを利用できます。 - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] -``` - -### Distributed パラメータ {#distributed-parameters} - -| Parameter | Description | -| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `cluster` | サーバーの設定ファイル内のクラスター名 | -| `database` | リモートデータベース名 | -| `table` | リモートテーブル名 | -| `sharding_key` (Optional) | シャーディングキー。
`sharding_key` の指定が必要となるケースは次のとおりです。
  • Distributed テーブルへの `INSERT` の場合(テーブルエンジンがデータの分割方法を決定するために `sharding_key` を必要とするため)。ただし、`insert_distributed_one_random_shard` 設定が有効な場合は、`INSERT` にシャーディングキーは不要です。
  • `optimize_skip_unused_shards` を利用する場合(どのシャードをクエリするかを決定するために `sharding_key` が必要です)。
| -| `policy_name` (Optional) | ポリシー名。バックグラウンド送信処理で使用する一時ファイルを保存するために使用されます | - -**関連項目** - -* [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) 設定 -* 例については [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) を参照 - -### Distributed 設定 {#distributed-settings} - - -| Setting | Description | Default value | -| ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| `fsync_after_insert` | Distributed へのバックグラウンド挿入後にファイルデータに対して `fsync` を実行します。OS が **イニシエーターノード** のディスク上の挿入済みデータ全体をファイルにフラッシュしたことを保証します。 | `false` | -| `fsync_directories` | ディレクトリに対して `fsync` を実行します。Distributed テーブルでのバックグラウンド挿入に関連する操作(挿入後、シャードへのデータ送信後など)の後で、OS がディレクトリメタデータを更新したことを保証します。 | `false` | -| `skip_unavailable_shards` | true の場合、ClickHouse は利用不能なシャードを黙って自動的にスキップします。シャードは次の場合に利用不能とマークされます: 1) 接続障害によりシャードに到達できない場合。2) シャードが DNS で解決できない場合。3) テーブルがそのシャード上に存在しない場合。 | `false` | -| `bytes_to_throw_insert` | バックグラウンド `INSERT` のために保留中の圧縮データ量(バイト数)がこの値を超えた場合、例外がスローされます。`0` の場合はスローしません。 | `0` | -| `bytes_to_delay_insert` | バックグラウンド `INSERT` のために保留中の圧縮データ量(バイト数)がこの値を超えた場合、クエリは遅延されます。`0` の場合は遅延しません。 | `0` | -| `max_delay_to_insert` | バックグラウンド送信のために保留中のバイト数が多い場合に、Distributed テーブルへのデータ挿入を遅延させる最大秒数。 | `60` | -| `background_insert_batch` | [`distributed_background_insert_batch`](../../../operations/settings/settings.md#distributed_background_insert_batch) と同じです。 | `0` | -| `background_insert_split_batch_on_failure` | [`distributed_background_insert_split_batch_on_failure`](../../../operations/settings/settings.md#distributed_background_insert_split_batch_on_failure) と同じです。 | `0` | -| `background_insert_sleep_time_ms` | [`distributed_background_insert_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms) と同じです。 | `0` | -| `background_insert_max_sleep_time_ms` | [`distributed_background_insert_max_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms) と同じです。 | `0` | -| `flush_on_detach` | `DETACH`/`DROP`/サーバーシャットダウン時に、リモートノードへデータをフラッシュします。 | `true` | - -:::note -**耐久性設定** (`fsync_...`): - -* データがまずイニシエーターノードのディスクに保存され、その後バックグラウンドでシャードへ送信される、バックグラウンド `INSERT`(つまり `distributed_foreground_insert=false`)にのみ影響します。 -* `INSERT` のパフォーマンスを大きく低下させる可能性があります。 -* 分散テーブルフォルダ内に保存されているデータを、**挿入を受け付けたノード** に書き込む処理に影響します。基盤となる MergeTree テーブルへの書き込み保証が必要な場合は、`system.merge_tree_settings` 内の耐久性設定(`...fsync...`)を参照してください。 - -**挿入制限設定**(`..._insert`)については、次も参照してください: - -* [`distributed_foreground_insert`](../../../operations/settings/settings.md#distributed_foreground_insert) 設定 -* [`prefer_localhost_replica`](/operations/settings/settings#prefer_localhost_replica) 設定 -* `bytes_to_throw_insert` は `bytes_to_delay_insert` より前に処理されるため、`bytes_to_delay_insert` より小さい値に設定すべきではありません。 - ::: - -**例** - -```sql -CREATE TABLE hits_all AS hits -ENGINE = Distributed(logs, default, hits[, sharding_key[, policy_name]]) -SETTINGS - fsync_after_insert=0, - fsync_directories=0; -``` - -`logs` クラスター内のすべてのサーバーに存在する `default.hits` テーブルからデータが読み出されます。データは読み出されるだけでなく、可能な範囲でリモートサーバー側で部分的に処理されます。例えば、`GROUP BY` を含むクエリの場合、データはリモートサーバー上で集約され、集約関数の中間状態がリクエスト元のサーバーに送信されます。その後、そのサーバーでデータがさらに集約されます。 - -データベース名の代わりに、文字列を返す定数式を使用できます。例えば、`currentDatabase()` です。 - - -## クラスター {#distributed-clusters} - -クラスターは[サーバー設定ファイル](../../../operations/configuration-files.md)で構成されます。 - -```xml - - - - - - - - - - - 1 - - shard_01 - - false - - - 1 - example01-01-1 - 9000 - - - example01-01-2 - 9000 - - - - 2 - shard_02 - false - - example01-02-1 - 9000 - - - example01-02-2 - 1 - 9440 - - - - -``` - -ここでは、`logs` という名前のクラスタが定義されており、2 つのシャード(各シャードには 2 つのレプリカを含む)で構成されています。シャードとは、データの異なる部分を保持しているサーバーのことであり(すべてのデータを読み取るには、すべてのシャードにアクセスする必要があります)、レプリカはサーバーの複製です(すべてのデータを読み取るには、いずれか 1 つのレプリカ上のデータにアクセスすれば十分です)。 - -クラスタ名にはドットを含めてはいけません。 - -各サーバーには、`host`、`port`、および必要に応じて `user`、`password`、`secure`、`compression`、`bind_host` のパラメータを指定します。 - - -| Parameter | Description | Default Value | -|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| -| `host` | リモートサーバーのアドレス。ドメイン名、IPv4 アドレス、または IPv6 アドレスを使用できます。ドメイン名を指定した場合、サーバー起動時に DNS リクエストが実行され、その結果はサーバーが稼働している間保持されます。DNS リクエストが失敗すると、サーバーは起動しません。DNS レコードを変更した場合は、サーバーを再起動してください。 | - | -| `port` | メッセージ送受信用に使用される TCP ポート(設定ファイル内の `tcp_port`、通常は 9000 に設定)。`http_port` と混同しないでください。 | - | -| `user` | リモートサーバーへ接続するためのユーザー名。このユーザーは指定したサーバーへの接続権限を持っている必要があります。アクセス権限は `users.xml` ファイルで設定します。詳細は [Access rights](../../../guides/sre/user-management/index.md) セクションを参照してください。 | `default` | -| `password` | リモートサーバーへ接続するためのパスワード(マスクされません)。 | '' | -| `secure` | セキュアな SSL/TLS 接続を使用するかどうか。通常、ポートの指定も必要です(デフォルトのセキュアポートは `9440`)。サーバーは `9440` でリッスンし、正しい証明書が設定されている必要があります。 | `false` | -| `compression` | データ圧縮を使用するかどうか。 | `true` | -| `bind_host` | このノードからリモートサーバーへ接続する際に使用する送信元アドレス。IPv4 アドレスのみサポートされます。ClickHouse の分散クエリで使用される送信元 IP アドレスを指定する必要がある、高度なデプロイメントのユースケース向けです。 | - | - -レプリカを指定すると、読み取り時に各シャードに対して利用可能なレプリカのうち 1 つが選択されます。ロードバランシングアルゴリズム(どのレプリカへアクセスするかの優先度)は、[load_balancing](../../../operations/settings/settings.md#load_balancing) 設定で構成できます。サーバーとの接続が確立できない場合、短いタイムアウトで接続を試行します。接続に失敗した場合は次のレプリカが選択され、すべてのレプリカについて同様に繰り返されます。すべてのレプリカへの接続試行が失敗した場合、同じ方法で複数回リトライされます。これはレジリエンス向上には有効ですが、完全なフォールトトレランスを提供するものではありません。リモートサーバーが接続を受け付けても、正常に動作しない、または性能が不十分な場合があるためです。 - -シャードを 1 つだけ指定することもできます(この場合、クエリ処理は「分散」ではなく「リモート」と呼ぶべきです)し、任意の数のシャードを指定することもできます。各シャード内では、1 つから任意の数のレプリカを指定できます。シャードごとに異なる数のレプリカを指定することも可能です。 - -設定内には、必要な数だけクラスターを指定できます。 - -クラスターを確認するには、`system.clusters` テーブルを使用します。 - -`Distributed` エンジンを使用すると、クラスターをローカルサーバーのように扱うことができます。ただし、クラスターの設定は動的に指定することはできず、サーバーの設定ファイルで構成しておく必要があります。通常、クラスター内のすべてのサーバーは同一のクラスター設定を持ちます(必須ではありません)。設定ファイル内のクラスターは、サーバーを再起動することなくオンザフライで更新されます。 - -毎回未知のシャードやレプリカの集合に対してクエリを送信する必要がある場合、`Distributed` テーブルを作成する必要はありません。その代わりに `remote` テーブル関数を使用してください。詳細は [Table functions](../../../sql-reference/table-functions/index.md) セクションを参照してください。 - - - -## データの書き込み {#distributed-writing-data} - -クラスターにデータを書き込む方法は 2 つあります。 - -1 つ目は、どのサーバーにどのデータを書き込むかを自分で定義し、各シャードに直接書き込む方法です。言い換えると、`Distributed` テーブルが参照しているクラスター内のリモートテーブルに対して、直接 `INSERT` 文を実行します。これは、任意のシャーディング方式を使用できるため、対象分野の要件により複雑な方式であっても対応できる、最も柔軟な方法です。また、この方式では異なるシャードに完全に独立してデータを書き込めるため、最も効率的でもあります。 - -2 つ目は、`Distributed` テーブルに対して `INSERT` 文を実行する方法です。この場合、テーブル自体が挿入されたデータをサーバー間に分散します。`Distributed` テーブルに書き込むには、`sharding_key` パラメータが設定されている必要があります(シャードが 1 つしかない場合を除く)。 - -各シャードには、設定ファイル内で `` を定義できます。デフォルトでは weight は `1` です。データは、シャードの weight に比例した量でシャード間に分散されます。すべてのシャードの weight が合計され、その後、各シャードの weight を合計値で割ることで、各シャードの比率が決まります。例えば、2 つのシャードがあり、1 つ目の weight が 1、2 つ目の weight が 2 の場合、1 つ目のシャードには挿入された行の 3 分の 1 (1 / 3)、2 つ目のシャードには 3 分の 2 (2 / 3) が送られます。 - -各シャードには、設定ファイル内で `internal_replication` パラメータを定義できます。このパラメータが `true` に設定されている場合、書き込み処理は最初の正常なレプリカを選択し、そのレプリカにデータを書き込みます。これは、`Distributed` テーブルの背後にあるテーブルがレプリケートされたテーブル(例: 任意の `Replicated*MergeTree` テーブルエンジン)である場合に使用します。テーブルレプリカのうち 1 つが書き込みを受け取り、その後自動的に他のレプリカへレプリケートされます。 - -`internal_replication` が `false`(デフォルト)に設定されている場合、データはすべてのレプリカに書き込まれます。この場合、`Distributed` テーブル自体がデータを複製します。これは、レプリケートされたテーブルを使用する場合よりも劣ります。というのも、レプリカ間の一貫性が検査されず、時間の経過とともに、レプリカごとにわずかに異なるデータを保持するようになるためです。 - -どのシャードに行データを送るかを選択するために、シャーディング式が評価され、その結果をシャードの総 weight で割った余りが取られます。行は、余りが `prev_weights` から `prev_weights + weight` までの半開区間に対応するシャードに送られます。ここで、`prev_weights` は番号がより小さいシャードの総 weight、`weight` はそのシャード自身の weight です。例えば、2 つのシャードがあり、1 つ目の weight が 9、2 つ目の weight が 10 の場合、余りが範囲 \[0, 9) に入る行は 1 つ目のシャードに、範囲 \[9, 19) に入る行は 2 つ目のシャードに送られます。 - -シャーディング式は、定数やテーブル列からなる任意の式であり、整数を返す必要があります。例えば、データをランダムに分散するには `rand()` を使用できますし、ユーザー ID を割った余りで分散するには `UserID` を使用できます(この場合、1 人のユーザーのデータは 1 つのシャードにのみ配置されるため、ユーザー単位の `IN` や `JOIN` を実行しやすくなります)。ある列が十分に均等に分散されない場合は、`intHash64(UserID)` のようにハッシュ関数でラップできます。 - -単純な除算の余りによる方法は、シャーディングの解決策としては限定的であり、常に適切というわけではありません。これは、中〜大規模(サーバーが数十台)のデータ量では機能しますが、非常に大規模(サーバーが数百台以上)のデータ量には向きません。後者の場合、`Distributed` テーブルを使用するのではなく、対象分野で求められるシャーディング方式を使用してください。 - -次のような場合には、シャーディング方式について検討する必要があります。 - - - -- 特定のキーでデータを結合する(`IN` または `JOIN`)クエリを使用している場合、そのキーでデータがシャーディングされていれば、`GLOBAL IN` や `GLOBAL JOIN` よりもはるかに効率的なローカルな `IN` または `JOIN` を使用できます。 -- 多数のサーバー(数百台以上)を使用し、多数の小さなクエリ、たとえば個々のクライアント(ウェブサイト、広告主、パートナーなど)のデータに対するクエリを実行する場合。小さなクエリがクラスター全体に影響しないようにするには、1 クライアントのデータを 1 シャード上に配置するのが理にかなっています。あるいは、二段階のシャーディングを構成することもできます。クラスター全体を複数の「レイヤー」に分割し、レイヤーは複数のシャードから構成されるようにします。1 クライアントのデータは 1 つのレイヤー内に配置されますが、必要に応じてそのレイヤーにシャードを追加でき、データはそれらのシャード内でランダムに分散されます。各レイヤーに対して `Distributed` テーブルを作成し、グローバルなクエリ用に 1 つの共有の `Distributed` テーブルを作成します。 - -データはバックグラウンドで書き込まれます。テーブルに `INSERT` されたとき、データブロックはローカルファイルシステムに書き込まれるだけです。データは可能な限り早くバックグラウンドでリモートサーバーへ送信されます。データ送信の周期は、[distributed_background_insert_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms) および [distributed_background_insert_max_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms) の設定で制御されます。`Distributed` エンジンは挿入されたデータを含むファイルを個別に送信しますが、[distributed_background_insert_batch](../../../operations/settings/settings.md#distributed_background_insert_batch) 設定を有効にすることで、ファイルのバッチ送信を有効化できます。この設定により、ローカルサーバーおよびネットワークリソースをより有効に活用することで、クラスターのパフォーマンスが向上します。テーブルディレクトリ `/var/lib/clickhouse/data/database/table/` にあるファイル(送信待ちデータ)の一覧を確認することで、データが正常に送信されているか確認する必要があります。バックグラウンドタスクを実行するスレッド数は、[background_distributed_schedule_pool_size](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size) 設定で指定できます。 - -`Distributed` テーブルへの `INSERT` 実行後に、サーバーが消失した場合や(ハードウェア障害などにより)クラッシュして強制再起動された場合、挿入されたデータが失われる可能性があります。テーブルディレクトリ内で破損したデータパーツが検出されると、それは `broken` サブディレクトリに移動され、以後は使用されません。 - - - -## データの読み取り {#distributed-reading-data} - -`Distributed` テーブルをクエリする場合、`SELECT` クエリはすべてのシャードに送信され、データがシャード間でどのように分散されているかに関係なく動作します(完全にランダムに分散されていても問題ありません)。新しいシャードを追加する場合、既存のデータをそのシャードへ移行する必要はありません。その代わり、新しいシャードに対してより大きな重み付けを指定して新しいデータを書き込むことができます。この場合、データの分散はやや不均一になりますが、クエリは正しくかつ効率的に動作します。 - -`max_parallel_replicas` オプションが有効な場合、クエリ処理は 1 つのシャード内のすべてのレプリカに対して並列化されます。詳細については、[max_parallel_replicas](../../../operations/settings/settings.md#max_parallel_replicas) セクションを参照してください。 - -分散環境における `in` および `global in` クエリがどのように処理されるかの詳細については、[こちら](/sql-reference/operators/in#distributed-subqueries) のドキュメントを参照してください。 - - - -## 仮想カラム {#virtual-columns} - -#### _Shard_num {#_shard_num} - -`_shard_num` — テーブル `system.clusters` の `shard_num` の値を保持します。型: [UInt32](../../../sql-reference/data-types/int-uint.md)。 - -:::note -[`remote`](../../../sql-reference/table-functions/remote.md) および [`cluster](../../../sql-reference/table-functions/cluster.md) テーブル関数は内部的に一時的な Distributed テーブルを作成するため、`_shard_num` はそれらでも利用可能です。 -::: - -**関連項目** - -- [仮想カラム](../../../engines/table-engines/index.md#table_engines-virtual_columns) の説明 -- [`background_distributed_schedule_pool_size`](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size) 設定 -- [`shardNum()`](../../../sql-reference/functions/other-functions.md#shardNum) および [`shardCount()`](../../../sql-reference/functions/other-functions.md#shardCount) 関数 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md deleted file mode 100644 index e44d4b1d19a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ /dev/null @@ -1,227 +0,0 @@ ---- -description: '`Executable` および `ExecutablePool` テーブルエンジンを使用すると、スクリプトの標準出力(**stdout**)に行を書き出すことで、そのスクリプトから行が生成されるテーブルを定義できます。' -sidebar_label: 'Executable/ExecutablePool' -sidebar_position: 40 -slug: /engines/table-engines/special/executable -title: '`Executable` および `ExecutablePool` テーブルエンジン' -doc_type: 'reference' ---- - -# Executable および ExecutablePool テーブルエンジン {#executable-and-executablepool-table-engines} - -`Executable` および `ExecutablePool` テーブルエンジンを使用すると、(行を **stdout** に書き出すことで)ユーザー定義のスクリプトによって行が生成されるテーブルを定義できます。実行可能スクリプトは `users_scripts` ディレクトリに保存され、任意のソースからデータを読み取ることができます。 - -* `Executable` テーブル: クエリごとにスクリプトが実行されます -* `ExecutablePool` テーブル: 永続プロセスのプールを維持し、読み取り時にそのプールからプロセスを取得します - -オプションとして、1 つ以上の入力用クエリを含めることができ、その結果を **stdin** にストリームしてスクリプトが読み取れるようにできます。 - -## `Executable` テーブルの作成 {#creating-an-executable-table} - -`Executable` テーブルエンジンには、スクリプト名と入力データの形式という 2 つのパラメータを指定する必要があります。必要に応じて、1 つ以上の入力クエリを渡すこともできます。 - -```sql -Executable(script_name, format, [input_query...]) -``` - -`Executable` テーブルに関連する設定は次のとおりです: - -* `send_chunk_header` - * 説明: チャンクを処理に送る前に、そのチャンク内の行数を送信します。この設定を有効にすると、スクリプト側でリソースを事前割り当てするなど、より効率的な記述が可能になります。 - * デフォルト値: false -* `command_termination_timeout` - * 説明: コマンドを終了させるタイムアウト(秒) - * デフォルト値: 10 -* `command_read_timeout` - * 説明: コマンドの標準出力からデータを読み取るタイムアウト(ミリ秒) - * デフォルト値: 10000 -* `command_write_timeout` - * 説明: コマンドの標準入力へデータを書き込むタイムアウト(ミリ秒) - * デフォルト値: 10000 - -例を見てみましょう。次の Python スクリプトは `my_script.py` という名前で、`user_scripts` フォルダ内に保存されています。数値 `i` を入力として受け取り、`i` 個のランダムな文字列を出力します。各文字列の先頭には、タブで区切られた番号が付与されます: - -```python -#!/usr/bin/python3 - -import sys -import string -import random - -def main(): - - # 入力値を読み込む - for number in sys.stdin: - i = int(number) - - # ランダムな行を生成する - for id in range(0, i): - letters = string.ascii_letters - random_string = ''.join(random.choices(letters ,k=10)) - print(str(id) + '\t' + random_string + '\n', end='') - - # 結果を標準出力にフラッシュする - sys.stdout.flush() - -if __name__ == "__main__": - main() -``` - -次の `my_executable_table` は `my_script.py` の出力から作成されたもので、`my_executable_table` に対して `SELECT` を実行するたびに 10 個のランダムな文字列を生成します。 - -```sql -CREATE TABLE my_executable_table ( - x UInt32, - y String -) -ENGINE = Executable('my_script.py', TabSeparated, (SELECT 10)) -``` - -テーブルを作成しても、スクリプトは呼び出されず、即座に処理が返されます。`my_executable_table` をクエリすると、スクリプトが呼び出されます。 - -```sql -SELECT * FROM my_executable_table -``` - -```response -┌─x─┬─y──────────┐ -│ 0 │ BsnKBsNGNH │ -│ 1 │ mgHfBCUrWM │ -│ 2 │ iDQAVhlygr │ -│ 3 │ uNGwDuXyCk │ -│ 4 │ GcFdQWvoLB │ -│ 5 │ UkciuuOTVO │ -│ 6 │ HoKeCdHkbs │ -│ 7 │ xRvySxqAcR │ -│ 8 │ LKbXPHpyDI │ -│ 9 │ zxogHTzEVV │ -└───┴────────────┘ -``` - -## クエリ結果をスクリプトに渡す {#passing-query-results-to-a-script} - -Hacker News サイトのユーザーはコメントを投稿します。Python には自然言語処理ツールキット (`nltk`) があり、その中の `SentimentIntensityAnalyzer` を使うと、コメントがポジティブかネガティブかニュートラルかを判定し、-1(非常にネガティブなコメント)から 1(非常にポジティブなコメント)の値を割り当てることができます。`nltk` を使って Hacker News のコメントのセンチメント(感情)を計算する `Executable` テーブルを作成してみましょう。 - -この例では、[こちら](/engines/table-engines/mergetree-family/invertedindexes/#hacker-news-dataset) で説明している `hackernews` テーブルを使用します。`hackernews` テーブルには、型が `UInt64` の `id` 列と、`comment` という名前の `String` 型の列があります。まずは `Executable` テーブルを定義することから始めましょう。 - -```sql -CREATE TABLE sentiment ( - id UInt64, - sentiment Float32 -) -ENGINE = Executable( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20) -); -``` - -`sentiment` テーブルについての補足: - -* ファイル `sentiment.py` は `user_scripts` フォルダ(`user_scripts_path` 設定のデフォルトフォルダ)に保存されています -* `TabSeparated` フォーマットは、Python スクリプトがタブ区切りの値を含む生データ行を生成する必要があることを意味します -* クエリは `hackernews` から 2 つのカラムを選択します。Python スクリプトでは、入力として渡される各行からこれらのカラム値をパース(抽出)する必要があります - -`sentiment.py` の定義は次のとおりです: - -```python -#!/usr/local/bin/python3.9 - -import sys -import nltk -from nltk.sentiment import SentimentIntensityAnalyzer - -def main(): - sentiment_analyzer = SentimentIntensityAnalyzer() - - while True: - try: - row = sys.stdin.readline() - if row == '': - break - - split_line = row.split("\t") - - id = str(split_line[0]) - comment = split_line[1] - - score = sentiment_analyzer.polarity_scores(comment)['compound'] - print(id + '\t' + str(score) + '\n', end='') - sys.stdout.flush() - except BaseException as x: - break - -if __name__ == "__main__": - main() -``` - -Python スクリプトについての補足説明です。 - -* これを動作させるには、`nltk.downloader.download('vader_lexicon')` を実行する必要があります。これはスクリプト内に含めることもできますが、その場合は `sentiment` テーブルに対してクエリが実行されるたびに毎回ダウンロードされてしまい、非効率です -* `row` のそれぞれの値は、`SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20` の結果セットの 1 行に対応します -* 入力として渡される行はタブ区切りになっているため、Python の `split` 関数を使って `id` と `comment` をパースします -* `polarity_scores` の結果は、いくつかの値を持つ JSON オブジェクトです。ここでは、この JSON オブジェクトから `compound` の値だけを取得することにしました -* ClickHouse の `sentiment` テーブルは `TabSeparated` フォーマットを使用し 2 つのカラムを持っているので、`print` 関数ではそれらのカラムをタブで区切っています - -`sentiment` テーブルから行を選択するクエリを記述するたびに、`SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20` クエリが実行され、その結果が `sentiment.py` に渡されます。実際に試してみましょう。 - -```sql -SELECT * -FROM sentiment -``` - -レスポンスは次のとおりです。 - -```response -┌───────id─┬─sentiment─┐ -│ 7398199 │ 0.4404 │ -│ 21640317 │ 0.1779 │ -│ 21462000 │ 0 │ -│ 25168863 │ 0 │ -│ 25168978 │ -0.1531 │ -│ 25169359 │ 0 │ -│ 25169394 │ -0.9231 │ -│ 25169766 │ 0.4137 │ -│ 25172570 │ 0.7469 │ -│ 25173687 │ 0.6249 │ -│ 28291534 │ 0 │ -│ 28291669 │ -0.4767 │ -│ 28291731 │ 0 │ -│ 28291949 │ -0.4767 │ -│ 28292004 │ 0.3612 │ -│ 28292050 │ -0.296 │ -│ 28292322 │ 0 │ -│ 28295172 │ 0.7717 │ -│ 28295288 │ 0.4404 │ -│ 21465723 │ -0.6956 │ -└──────────┴───────────┘ -``` - -## `ExecutablePool` テーブルの作成 {#creating-an-executablepool-table} - -`ExecutablePool` の構文は `Executable` と似ていますが、`ExecutablePool` テーブルに固有の重要な設定がいくつかあります。 - -* `pool_size` - * 説明: プロセスプールのサイズ。サイズが 0 の場合はサイズ制限がありません。 - * デフォルト値: 16 -* `max_command_execution_time` - * 説明: コマンドの最大実行時間(秒単位) - * デフォルト値: 10 - -上記の `sentiment` テーブルは、`Executable` の代わりに `ExecutablePool` を使用するように容易に変更できます。 - -```sql -CREATE TABLE sentiment_pooled ( - id UInt64, - sentiment Float32 -) -ENGINE = ExecutablePool( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20000) -) -SETTINGS - pool_size = 4; -``` - -クライアントが `sentiment_pooled` テーブルをクエリすると、ClickHouse は必要に応じて 4 つのプロセスを起動して維持します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md deleted file mode 100644 index a7bba2403f7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'ClickHouse では、`SELECT` クエリと一緒に、そのクエリの処理に必要なデータをサーバーに送信することができます。このデータは一時テーブルに格納され、クエリ内で(たとえば `IN` 演算子などで)使用できます。' -sidebar_label: 'クエリ処理のための外部データ' -sidebar_position: 130 -slug: /engines/table-engines/special/external-data -title: 'クエリ処理のための外部データ' -doc_type: 'reference' ---- - -# クエリ処理のための外部データ {#external-data-for-query-processing} - -ClickHouse では、`SELECT` クエリと一緒に、そのクエリの処理に必要なデータをサーバーに送信できます。これらのデータは一時テーブル(「Temporary tables」のセクションを参照)に格納され、クエリ内(たとえば `IN` 演算子)で利用できます。 - -たとえば、重要なユーザー識別子が記載されたテキストファイルがある場合、そのリストでフィルタリングを行うクエリと一緒に、そのファイルをサーバーにアップロードできます。 - -大量の外部データを使って複数のクエリを実行する必要がある場合は、この機能は使用しないでください。その場合は、事前にデータを DB にアップロードしておく方が適切です。 - -外部データは、コマンドラインクライアント(非対話モード)または HTTP インターフェイスを使用してアップロードできます。 - -コマンドラインクライアントでは、次の形式でパラメータセクションを指定できます - -```bash ---external --file=... [--name=...] [--format=...] [--types=...|--structure=...] -``` - -転送されるテーブルの数に応じて、このようなセクションを複数指定することができます。 - -**–external** – 節の開始を示します。 -**–file** – テーブルダンプのファイルへのパス、または stdin を参照する `-` を指定します。 -stdin から取得できるのは単一のテーブルのみです。 - -次のパラメータは省略可能です: **–name** – テーブル名。省略した場合は _data が使用されます。 -**–format** – ファイル内のデータフォーマット。省略した場合は TabSeparated が使用されます。 - -次のパラメータのいずれか一つが必須です: **–types** – カンマ区切りのカラム型の一覧。例: `UInt64,String`。カラム名は _1, _2, ... のように付けられます。 -**–structure** – `UserID UInt64`, `URL String` のような形式で指定するテーブル構造。カラム名と型を定義します。 - -`file` で指定されたファイルは、`format` で指定されたフォーマットとして解析され、`types` または `structure` で指定されたデータ型が使用されます。テーブルはサーバーにアップロードされ、`name` で指定された名前の一時テーブルとしてサーバー側で利用可能になります。 - -例: - -```bash -$ echo -ne "1\n2\n3\n" | clickhouse-client --query="SELECT count() FROM test.visits WHERE TraficSourceID IN _data" --external --file=- --types=Int8 -849897 -$ cat /etc/passwd | sed 's/:/\t/g' | clickhouse-client --query="SELECT shell, count() AS c FROM passwd GROUP BY shell ORDER BY c DESC" --external --file=- --name=passwd --structure='login String, unused String, uid UInt16, gid UInt16, comment String, home String, shell String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -HTTP インターフェイスを使用する場合、外部データは multipart/form-data 形式で渡されます。各テーブルは個別のファイルとして送信されます。テーブル名はファイル名から取得されます。`query_string` にはパラメータ `name_format`、`name_types`、`name_structure` が渡されます。ここで `name` は、これらのパラメータが対応するテーブルの名前を表します。パラメータの意味は、コマンドラインクライアントを使用する場合と同じです。 - -例: - -```bash -$ cat /etc/passwd | sed 's/:/\t/g' > passwd.tsv - -$ curl -F 'passwd=@passwd.tsv;' 'http://localhost:8123/?query=SELECT+shell,+count()+AS+c+FROM+passwd+GROUP+BY+shell+ORDER+BY+c+DESC&passwd_structure=login+String,+unused+String,+uid+UInt16,+gid+UInt16,+comment+String,+home+String,+shell+String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -分散クエリ処理では、一時テーブルがすべてのリモートサーバーに送信されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md deleted file mode 100644 index 3f2683934f5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: 'File テーブルエンジンは、サポートされているファイル形式(`TabSeparated`、`Native` など)のいずれかで、データをファイルに保存します。' -sidebar_label: 'File' -sidebar_position: 40 -slug: /engines/table-engines/special/file -title: 'File テーブルエンジン' -doc_type: 'reference' ---- - - - -# File テーブルエンジン {#file-table-engine} - -File テーブルエンジンは、サポートされている[ファイルフォーマット](/interfaces/formats#formats-overview)(`TabSeparated`、`Native` など)のいずれかでデータをファイルに保存します。 - -利用シナリオ: - -- ClickHouse からファイルへのデータエクスポート。 -- データをあるフォーマットから別のフォーマットへ変換。 -- ディスク上のファイルを編集して ClickHouse 内のデータを更新。 - -:::note -このエンジンは現在 ClickHouse Cloud では利用できません。[代わりに S3 テーブル関数を使用してください](/sql-reference/table-functions/s3.md)。 -::: - - - -## ClickHouse サーバーでの利用方法 {#usage-in-clickhouse-server} - -```sql -File(Format) -``` - -`Format` パラメータは、利用可能なファイルフォーマットのうちの 1 つを指定します。`SELECT` クエリを実行するには、そのフォーマットが入力用としてサポートされている必要があり、`INSERT` クエリを実行するには出力用としてサポートされている必要があります。利用可能なフォーマットは [Formats](/interfaces/formats#formats-overview) セクションに一覧があります。 - -ClickHouse では、`File` に対してファイルシステムのパスを指定することはできません。サーバー設定の [path](../../../operations/server-configuration-parameters/settings.md) 設定で定義されたフォルダが使用されます。 - -`File(Format)` を使用してテーブルを作成すると、そのフォルダ内に空のサブディレクトリが作成されます。そのテーブルにデータが書き込まれると、そのサブディレクトリ内の `data.Format` ファイルに書き込まれます。 - -サーバーのファイルシステム上で、このサブフォルダとファイルを手動で作成し、その後、同じ名前のテーブルとして [ATTACH](../../../sql-reference/statements/attach.md) することで、そのファイルからデータをクエリできるようにすることも可能です。 - -:::note -この機能を使用する際は注意してください。ClickHouse は、この種のファイルに対する外部からの変更を追跡しません。ClickHouse 経由での書き込みと ClickHouse 外部からの書き込みが同時に行われた場合の結果は未定義です。 -::: - - -## 例 {#example} - -**1.** `file_engine_table` テーブルを作成します。 - -```sql -CREATE TABLE file_engine_table (name String, value UInt32) ENGINE=File(TabSeparated) -``` - -デフォルトでは、ClickHouse はフォルダ `/var/lib/clickhouse/data/default/file_engine_table` を作成します。 - -**2.** 手動で `/var/lib/clickhouse/data/default/file_engine_table/data.TabSeparated` を作成し、次の内容を記述します: - -```bash -$ cat data.TabSeparated -one 1 -two 2 -``` - -**3.** データをクエリする: - -```sql -SELECT * FROM file_engine_table -``` - -```text -┌─name─┬─value─┐ -│ one │ 1 │ -│ two │ 2 │ -└──────┴───────┘ -``` - - -## ClickHouse-local での使用方法 {#usage-in-clickhouse-local} - -[clickhouse-local](../../../operations/utilities/clickhouse-local.md) では、File エンジンは `Format` に加えてファイルパスも指定できます。デフォルトの入出力ストリームは、`0` や `stdin`、`1` や `stdout` のような数値または人間が読める名前で指定できます。追加のエンジンパラメータまたはファイル拡張子(`gz`、`br`、`xz`)に基づいて、圧縮ファイルの読み書きを行えます。 - -**例:** - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -q "CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); SELECT a, b FROM table; DROP TABLE table" -``` - - -## 実装の詳細 {#details-of-implementation} - -- 複数の `SELECT` クエリは同時に実行できますが、`INSERT` クエリは直列に実行されます。 -- `INSERT` クエリによる新規ファイルの作成をサポートしています。 -- ファイルが既に存在する場合、`INSERT` はそのファイルに新しい値を追記します。 -- サポートされていません: - - `ALTER` - - `SELECT ... SAMPLE` - - インデックス - - レプリケーション - - - -## PARTITION BY {#partition-by} - -`PARTITION BY` — オプションです。パーティションキーによってデータを分割することで、別々のファイルとして保存できます。ほとんどの場合、パーティションキーは不要であり、必要な場合でも通常は「月」より細かい粒度のパーティションキーは必要ありません。パーティション分割は(`ORDER BY` 式とは対照的に)クエリの高速化にはつながりません。パーティションの粒度を細かくしすぎてはいけません。クライアント識別子や名前でデータをパーティション分割しないでください(その代わりに、`ORDER BY` 式の最初の列としてクライアント識別子または名前を指定します)。 - -月単位でパーティション分割するには、`toYYYYMM(date_column)` 式を使用します。ここで `date_column` は [Date](/sql-reference/data-types/date.md) 型の日付を持つ列です。このときのパーティション名は `"YYYYMM"` 形式になります。 - - - -## 仮想カラム {#virtual-columns} - -- `_path` — ファイルへのパス。型: `LowCardinality(String)`。 -- `_file` — ファイル名。型: `LowCardinality(String)`。 -- `_size` — ファイルサイズ(バイト単位)。型: `Nullable(UInt64)`。サイズが不明な場合、値は `NULL` です。 -- `_time` — ファイルの最終更新時刻。型: `Nullable(DateTime)`。時刻が不明な場合、値は `NULL` です。 - - - -## 設定 {#settings} - -- [engine_file_empty_if_not_exists](/operations/settings/settings#engine_file_empty_if_not_exists) - 存在しないファイルに対して空のデータを選択できるようにします。デフォルトでは無効です。 -- [engine_file_truncate_on_insert](/operations/settings/settings#engine_file_truncate_on_insert) - 挿入前にファイルを切り詰められるようにします。デフォルトでは無効です。 -- [engine_file_allow_create_multiple_files](/operations/settings/settings.md#engine_file_allow_create_multiple_files) - フォーマットにサフィックスがある場合、挿入ごとに新しいファイルを作成できるようにします。デフォルトでは無効です。 -- [engine_file_skip_empty_files](/operations/settings/settings.md#engine_file_skip_empty_files) - 読み取り時に空のファイルをスキップできるようにします。デフォルトでは無効です。 -- [storage_file_read_method](/operations/settings/settings#engine_file_empty_if_not_exists) - ストレージファイルからデータを読み出す方法で、`read`、`pread`、`mmap` のいずれかです。`mmap` メソッドは clickhouse-server には適用されません(clickhouse-local 用です)。デフォルト値は、clickhouse-server では `pread`、clickhouse-local では `mmap` です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md deleted file mode 100644 index f7d49039b45..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -description: 'このエンジンは、アプリケーションログファイルをレコードのストリームとして処理します。' -sidebar_label: 'FileLog' -sidebar_position: 160 -slug: /engines/table-engines/special/filelog -title: 'FileLog テーブルエンジン' -doc_type: 'reference' ---- - - - -# FileLog テーブルエンジン {#filelog-engine} - -このエンジンを使用すると、アプリケーションのログファイルをレコードのストリームとして処理できます。 - -`FileLog` を使用すると、次のことができます: - -- ログファイルを監視する。 -- 監視対象のログファイルに追記された新しいレコードを処理する。 - - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = FileLog('path_to_logs', 'format_name') SETTINGS - [poll_timeout_ms = 0,] - [poll_max_batch_size = 0,] - [max_block_size = 0,] - [max_threads = 0,] - [poll_directory_watch_events_backoff_init = 500,] - [poll_directory_watch_events_backoff_max = 32000,] - [poll_directory_watch_events_backoff_factor = 2,] - [handle_error_mode = 'default'] -``` - -Engine 引数: - -* `path_to_logs` – 購読対象とするログファイルへのパス。ログファイルを含むディレクトリへのパス、または単一のログファイルへのパスを指定できます。なお、ClickHouse は `user_files` ディレクトリ内のパスのみを許可します。 -* `format_name` - レコードフォーマット。FileLog はファイル内の各行を個別のレコードとして処理するため、すべてのデータフォーマットが適しているわけではない点に注意してください。 - -オプションパラメータ: - -* `poll_timeout_ms` - ログファイルからの 1 回のポーリングにおけるタイムアウト。デフォルト: [stream_poll_timeout_ms](../../../operations/settings/settings.md#stream_poll_timeout_ms)。 -* `poll_max_batch_size` — 1 回のポーリングで取得される最大レコード数。デフォルト: [max_block_size](/operations/settings/settings#max_block_size)。 -* `max_block_size` — ポーリング時のバッチサイズ(レコード数)の最大値。デフォルト: [max_insert_block_size](../../../operations/settings/settings.md#max_insert_block_size)。 -* `max_threads` - ファイルをパースするための最大スレッド数。デフォルトは 0 で、その場合の実際の値は max(1, physical_cpu_cores / 4) となります。 -* `poll_directory_watch_events_backoff_init` - ディレクトリ監視スレッドの初期待機時間。デフォルト: `500`。 -* `poll_directory_watch_events_backoff_max` - ディレクトリ監視スレッドの最大待機時間。デフォルト: `32000`。 -* `poll_directory_watch_events_backoff_factor` - バックオフの係数。デフォルトでは指数関数的に増加します。デフォルト: `2`。 -* `handle_error_mode` — FileLog エンジンでエラーをどのように処理するか。指定可能な値: `default`(メッセージのパースに失敗した場合に例外をスロー)、`stream`(例外メッセージと元のメッセージを仮想カラム `_error` と `_raw_message` に保存)。 - - -## 説明 {#description} - -取り込まれたレコードは自動的に追跡されるため、ログファイル内の各レコードは一度だけカウントされます。 - -`SELECT` は(デバッグ用途を除き)レコードの読み取りにはあまり有用ではありません。各レコードは一度しか読み取れないためです。代わりに、[マテリアライズドビュー](../../../sql-reference/statements/create/view.md) を使ってリアルタイム処理パイプラインを作成する方が実用的です。これを行うには、次のようにします。 - -1. FileLog エンジンを使用して FileLog テーブルを作成し、それをデータストリームと見なします。 -2. 目的の構造を持つテーブルを作成します。 -3. エンジンから読み取ったデータを変換して、事前に作成したテーブルに格納するマテリアライズドビューを作成します。 - -`MATERIALIZED VIEW` がエンジンに接続されると、バックグラウンドでデータの収集を開始します。これにより、ログファイルから継続的にレコードを受信し、`SELECT` を使用して必要な形式に変換できます。 -1 つの FileLog テーブルには、必要な数だけマテリアライズドビューを作成できます。これらはテーブルから直接データを読み取るのではなく、新しいレコードを(ブロック単位で)受け取ります。この方法により、詳細度の異なる複数のテーブル(グループ化・集約あり/なし)に書き込むことができます。 - -例: - -```sql - CREATE TABLE logs ( - timestamp UInt64, - level String, - message String - ) ENGINE = FileLog('user_files/my_app/app.log', 'JSONEachRow'); - - CREATE TABLE daily ( - day Date, - level String, - total UInt64 - ) ENGINE = SummingMergeTree(day, (day, level), 8192); - - CREATE MATERIALIZED VIEW consumer TO daily - AS SELECT toDate(toDateTime(timestamp)) AS day, level, count() AS total - FROM queue GROUP BY day, level; - - SELECT level, sum(total) FROM daily GROUP BY level; -``` - -ストリームデータの受信を停止するか、変換ロジックを変更するには、マテリアライズドビューをデタッチします。 - -```sql - DETACH TABLE consumer; - ATTACH TABLE consumer; -``` - -`ALTER` を使用してターゲットテーブルを変更する場合は、ターゲットテーブルとビューからのデータとの不整合を避けるため、マテリアルビューを無効化することを推奨します。 - - -## 仮想カラム {#virtual-columns} - -- `_filename` - ログファイル名。データ型: `LowCardinality(String)`。 -- `_offset` - ログファイル内のオフセット。データ型: `UInt64`。 - -`handle_error_mode='stream'` の場合に追加される仮想カラム: - -- `_raw_record` - 正常にパースできなかった生のレコード。データ型: `Nullable(String)`。 -- `_error` - パース失敗時に発生した例外メッセージ。データ型: `Nullable(String)`。 - -注記: `_raw_record` と `_error` の仮想カラムには、パース中に例外が発生した場合のみ値が設定され、メッセージが正常にパースされた場合は常に `NULL` になります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md deleted file mode 100644 index 80617691ca0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 'GenerateRandom テーブルエンジンは、指定されたテーブルスキーマに従ってランダムなデータを生成します。' -sidebar_label: 'GenerateRandom' -sidebar_position: 140 -slug: /engines/table-engines/special/generate -title: 'GenerateRandom テーブルエンジン' -doc_type: 'reference' ---- - - - -# GenerateRandom テーブルエンジン {#generaterandom-table-engine} - -GenerateRandom テーブルエンジンは、指定されたテーブルスキーマに基づいてランダムなデータを生成します。 - -使用例: - -- テストで再現可能な大規模テーブルを作成するために使用します。 -- ファジングテスト用のランダムな入力データを生成します。 - - - -## ClickHouse サーバーでの利用方法 {#usage-in-clickhouse-server} - -```sql -ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) -``` - -`max_array_length` と `max_string_length` パラメータは、生成されるデータ内のすべての配列型およびマップ型カラムと文字列の最大長をそれぞれ指定します。 - -Generate テーブルエンジンは `SELECT` クエリのみをサポートします。 - -テーブルに保存可能な [DataTypes](../../../sql-reference/data-types/index.md) のうち、`AggregateFunction` を除くすべてをサポートします。 - - -## 例 {#example} - -**1.** `generate_engine_table` テーブルを作成します。 - -```sql -CREATE TABLE generate_engine_table (name String, value UInt32) ENGINE = GenerateRandom(1, 5, 3) -``` - -**2.** データをクエリします: - -```sql -SELECT * FROM generate_engine_table LIMIT 3 -``` - -```text -┌─name─┬──────value─┐ -│ c4xJ │ 1412771199 │ -│ r │ 1791099446 │ -│ 7#$ │ 124312908 │ -└──────┴────────────┘ -``` - - -## 実装の詳細 {#details-of-implementation} - -- サポート対象外: - - `ALTER` - - `SELECT ... SAMPLE` - - `INSERT` - - インデックス - - レプリケーション diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md deleted file mode 100644 index ff1b2715358..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '特殊なテーブルエンジンに関するドキュメント' -sidebar_label: '特殊' -sidebar_position: 50 -slug: /engines/table-engines/special/ -title: '特殊テーブルエンジン' -doc_type: 'reference' ---- - -# 特殊なテーブルエンジン {#special-table-engines} - -テーブルエンジンには、主に次の 3 つのカテゴリがあります。 - -* 本番利用向けの [MergeTree エンジンファミリー](../../../engines/table-engines/mergetree-family/index.md)。 -* 小規模な一時データ向けの [Log エンジンファミリー](../../../engines/table-engines/log-family/index.md)。 -* [連携用テーブルエンジン](../../../engines/table-engines/integrations/index.md)。 - -それ以外のエンジンは用途が固有で、まだファミリーとしてグループ化されていないため、この「特殊」カテゴリに分類されています。 - -{/* このページの目次テーブルは、 - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - によって、YAML フロントマターのフィールド slug、description、title から自動生成されています。 - - 誤りに気付いた場合は、該当ページの YML フロントマターを編集してください。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | 説明 | -| ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| [Alias table engine](/engines/table-engines/special/alias) | Alias テーブルエンジンは、別のテーブルへの透過的なプロキシを作成します。すべての操作は対象テーブルに転送され、エイリアス自体はデータを保持しません。 | -| [Distributed table engine](/engines/table-engines/special/distributed) | Distributed エンジンを持つテーブルは自身のデータを一切保持しませんが、複数サーバー上での分散クエリ処理を可能にします。読み取りは自動的に並列化されます。読み取り時には、存在する場合はリモートサーバー上のテーブルインデックスが使用されます。 | -| [Dictionary table engine](/engines/table-engines/special/dictionary) | `Dictionary` エンジンは、辞書データを ClickHouse テーブルとして表示します。 | -| [Merge table engine](/engines/table-engines/special/merge) | `Merge` エンジン(`MergeTree` と混同しないでください)は自体ではデータを保存しませんが、任意の数の他のテーブルから同時に読み取ることを可能にします。 | -| [Executable and ExecutablePool table engines](/engines/table-engines/special/executable) | `Executable` および `ExecutablePool` テーブルエンジンを使用すると、(行を **stdout** に書き出すことで)定義したスクリプトから行が生成されるテーブルを定義できます。 | -| [File table engine](/engines/table-engines/special/file) | File テーブルエンジンは、サポートされているファイル形式(`TabSeparated`、`Native` など)のいずれかでデータをファイル内に保持します。 | -| [Null table engine](/engines/table-engines/special/null) | `Null` テーブルに書き込むと、データは破棄されます。`Null` テーブルから読み取ると、レスポンスは空になります。 | -| [Set table engine](/engines/table-engines/special/set) | 常に RAM 上に存在するデータセットです。`IN` 演算子の右側での使用を意図しています。 | -| [Join table engine](/engines/table-engines/special/join) | JOIN 演算で使用するためのオプションの事前構築済みデータ構造です。 | -| [URL table engine](/engines/table-engines/special/url) | リモートの HTTP/HTTPS サーバーとの間でデータをクエリします。このエンジンは File エンジンに似ています。 | -| [View table engine](/engines/table-engines/special/view) | ビューを実装するために使用されます(詳細は `CREATE VIEW query` を参照してください)。データは保存せず、指定された `SELECT` クエリのみを保持します。テーブルから読み取るときには、このクエリを実行し(クエリから不要なカラムをすべて削除します)。 | -| [Memory table engine](/engines/table-engines/special/memory) | Memory エンジンは、データを RAM 上に非圧縮形式で保存します。データは、読み出す際にも受信時とまったく同じ形式のまま保持されます。言い換えると、このテーブルからの読み取りには一切の追加処理が発生しません。 | -| [Buffer table engine](/engines/table-engines/special/buffer) | 書き込み対象のデータを RAM 上でバッファリングし、定期的に別のテーブルへフラッシュします。読み取り操作時には、バッファともう一方のテーブルの両方から同時にデータが読み取られます。 | -| [External data for query processing](/engines/table-engines/special/external-data) | ClickHouse では、`SELECT` クエリと一緒に、クエリを処理するために必要なデータをサーバーに送信できます。このデータは一時テーブルに格納され、クエリ内で(たとえば `IN` 演算子で)使用できます。 | -| [GenerateRandom table engine](/engines/table-engines/special/generate) | GenerateRandom テーブルエンジンは、指定されたテーブルスキーマに対してランダムなデータを生成します。 | -| [KeeperMap table engine](/engines/table-engines/special/keeper-map) | このエンジンを使用すると、Keeper/ZooKeeper クラスタを、一貫性のあるキー・バリュー・ストアとして、線形化可能な書き込みと逐次一貫性を持つ読み取りで利用できます。 | -| [FileLog table engine](/engines/table-engines/special/filelog) | このエンジンを使用すると、アプリケーションのログファイルをレコードストリームとして処理できます。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md deleted file mode 100644 index 9b083a61439..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ /dev/null @@ -1,176 +0,0 @@ ---- -description: 'JOIN 操作で使用するためのオプションの事前構築されたデータ構造。' -sidebar_label: 'Join' -sidebar_position: 70 -slug: /engines/table-engines/special/join -title: 'Join テーブルエンジン' -doc_type: 'reference' ---- - - - -# Join テーブルエンジン {#join-table-engine} - -[JOIN](/sql-reference/statements/select/join) 演算で使用するための、オプションの事前構築済みデータ構造です。 - -:::note -ClickHouse Cloud で、サービスがバージョン 25.4 より前に作成されている場合は、`SET compatibility=25.4` を実行して、compatibility を少なくとも 25.4 に設定する必要があります。 -::: - - - -## テーブルの作成 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1] [TTL expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2] [TTL expr2], -) ENGINE = Join(join_strictness, join_type, k1[, k2, ...]) -``` - -[CREATE TABLE](/sql-reference/statements/create/table) クエリの詳細については、説明を参照してください。 - - -## エンジンパラメータ {#engine-parameters} - -### `join_strictness` {#join_strictness} - -`join_strictness` – [JOIN の厳密度](/sql-reference/statements/select/join#supported-types-of-join)。 - -### `join_type` {#join_type} - -`join_type` – [JOIN の種類](/sql-reference/statements/select/join#supported-types-of-join)。 - -### キー列 {#key-columns} - -`k1[, k2, ...]` – `JOIN` 演算が実行される際の、`USING` 句内のキー列。 - -`join_strictness` および `join_type` パラメータは、`Join(ANY, LEFT, col1)` のように引用符なしで指定します。これらは、そのテーブルが使用される `JOIN` 演算と一致していなければなりません。パラメータが一致しない場合でも、ClickHouse は例外をスローせず、誤ったデータを返す可能性があります。 - - - -## 詳細と推奨事項 {#specifics-and-recommendations} - -### データストレージ {#data-storage} - -`Join` テーブルのデータは常に RAM 上にあります。テーブルに行を挿入するとき、ClickHouse はデータブロックをディスク上のディレクトリに書き込み、サーバーの再起動時にそれらを復元できるようにします。 - -サーバーが異常終了したり正しくない手順で再起動された場合、ディスク上のデータブロックが失われたり破損したりすることがあります。この場合、破損したデータを含むファイルを手動で削除する必要が生じることがあります。 - -### データの選択と挿入 {#selecting-and-inserting-data} - -`INSERT` クエリを使用して、`Join` エンジンのテーブルにデータを追加できます。テーブルが `ANY` 厳密度で作成されている場合、重複キーのデータは無視されます。`ALL` 厳密度では、すべての行が追加されます。 - -`Join` エンジンテーブルの主なユースケースは次のとおりです。 - -- `JOIN` 句の右側にテーブルを配置する。 -- [joinGet](/sql-reference/functions/other-functions.md/#joinGet) 関数を呼び出し、辞書と同じ方法でテーブルからデータを抽出する。 - -### データの削除 {#deleting-data} - -`Join` エンジンテーブルに対する `ALTER DELETE` クエリは、[mutation](/sql-reference/statements/alter/index.md#mutations) として実装されています。`DELETE` mutation はフィルタ済みのデータを読み取り、メモリおよびディスク上のデータを上書きします。 - -### 制限事項と設定 {#join-limitations-and-settings} - -テーブル作成時には、次の設定が適用されます。 - -#### `join_use_nulls` {#join_use_nulls} - -[join_use_nulls](/operations/settings/settings.md/#join_use_nulls) - -#### `max_rows_in_join` {#max_rows_in_join} - -[max_rows_in_join](/operations/settings/settings#max_rows_in_join) - -#### `max_bytes_in_join` {#max_bytes_in_join} - -[max_bytes_in_join](/operations/settings/settings#max_bytes_in_join) - -#### `join_overflow_mode` {#join_overflow_mode} - -[join_overflow_mode](/operations/settings/settings#join_overflow_mode) - -#### `join_any_take_last_row` {#join_any_take_last_row} - -[join_any_take_last_row](/operations/settings/settings.md/#join_any_take_last_row) -#### `join_use_nulls` {#join_use_nulls-1} - -#### Persistent {#persistent} - -Join および [Set](/engines/table-engines/special/set.md) テーブルエンジンの永続化を無効にします。 - -I/O オーバーヘッドを削減します。性能を重視し、永続化を必要としないシナリオに適しています。 - -設定可能な値: - -- 1 — 有効。 -- 0 — 無効。 - -デフォルト値: `1`。 - -`Join` エンジンテーブルは `GLOBAL JOIN` 操作では使用できません。 - -`Join` エンジンでは、`CREATE TABLE` 文内で [join_use_nulls](/operations/settings/settings.md/#join_use_nulls) 設定を指定できます。[SELECT](/sql-reference/statements/select/index.md) クエリでも同じ `join_use_nulls` の値を使用する必要があります。 - - - -## 使用例 {#example} - -左側テーブルの作成: - -```sql -CREATE TABLE id_val(`id` UInt32, `val` UInt32) ENGINE = TinyLog; -``` - -```sql -INSERT INTO id_val VALUES (1,11)(2,12)(3,13); -``` - -右側の `Join` テーブルを作成する: - -```sql -CREATE TABLE id_val_join(`id` UInt32, `val` UInt8) ENGINE = Join(ANY, LEFT, id); -``` - -```sql -INSERT INTO id_val_join VALUES (1,21)(1,22)(3,23); -``` - -テーブル結合: - -```sql -SELECT * FROM id_val ANY LEFT JOIN id_val_join USING (id); -``` - -```text -┌─id─┬─val─┬─id_val_join.val─┐ -│ 1 │ 11 │ 21 │ -│ 2 │ 12 │ 0 │ -│ 3 │ 13 │ 23 │ -└────┴─────┴─────────────────┘ -``` - -代わりに、結合キーの値を指定して `Join` テーブルからデータを取得することもできます。 - -```sql -SELECT joinGet('id_val_join', 'val', toUInt32(1)); -``` - -```text -┌─joinGet('id_val_join', 'val', toUInt32(1))─┐ -│ 21 │ -└────────────────────────────────────────────┘ -``` - -`Join` テーブルから行を削除する: - -```sql -ALTER TABLE id_val_join DELETE WHERE id = 3; -``` - -```text -┌─id─┬─val─┐ -│ 1 │ 21 │ -└────┴─────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md deleted file mode 100644 index 3c04723fc6c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -description: 'このエンジンを使用すると、Keeper/ZooKeeper クラスターを、書き込みは線形化可能、読み取りは逐次一貫性を持つ、一貫性のあるキーバリューストアとして利用できます。' -sidebar_label: 'KeeperMap' -sidebar_position: 150 -slug: /engines/table-engines/special/keeper-map -title: 'KeeperMap テーブルエンジン' -doc_type: 'reference' ---- - - - -# KeeperMap テーブルエンジン {#keepermap-table-engine} - -このエンジンを使用すると、Keeper/ZooKeeper クラスターを、線形化可能な書き込みと逐次一貫な読み取りを備えた一貫性のあるキー・バリュー ストアとして利用できます。 - -KeeperMap ストレージエンジンを有効化するには、テーブルを保存する ZooKeeper パスを `` 設定で定義する必要があります。 - -例: - -```xml - - /keeper_map_tables - -``` - -ここで path には任意の有効な ZooKeeper パスを指定できます。 - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = KeeperMap(root_path, [keys_limit]) PRIMARY KEY(primary_key_name) -``` - -エンジンのパラメータ: - -* `root_path` - `table_name` が保存される ZooKeeper パス。\ - このパスには、設定の `` で定義されたプレフィックスを含めないでください。プレフィックスは自動的に `root_path` に付加されます。\ - さらに、`auxiliary_zookeeper_cluster_name:/some/path` の形式もサポートされます。ここで `auxiliary_zookeeper_cluster` は `` 設定内で定義された ZooKeeper クラスタです。\ - 既定では、`` 設定内で定義された ZooKeeper クラスタが使用されます。 -* `keys_limit` - テーブル内で許可されるキー数。\ - この制限はソフトリミットであり、特定のエッジケースではより多くのキーがテーブルに格納される可能性があります。 -* `primary_key_name` – カラムリスト内の任意のカラム名。 -* `primary key` は必ず指定する必要があり、主キーとしては 1 つのカラムのみをサポートします。主キーは ZooKeeper 内で `node name` としてバイナリ形式でシリアライズされます。 -* 主キー以外のカラムは、対応する順序でバイナリにシリアライズされ、シリアライズされたキーによって定義される生成ノードの値として保存されます。 -* キーに対して `equals` または `in` によるフィルタを行うクエリは、`Keeper` からの複数キーのルックアップとして最適化され、それ以外の場合はすべての値を取得します。 - -例: - -```sql -CREATE TABLE keeper_map_table -( - `key` String, - `v1` UInt32, - `v2` String, - `v3` Float32 -) -ENGINE = KeeperMap('/keeper_map_table', 4) -PRIMARY KEY key -``` - -と共に - -```xml - - /keeper_map_tables - -``` - -各値は `(v1, v2, v3)` をバイナリ形式にシリアライズしたものであり、`Keeper` 内の `/keeper_map_tables/keeper_map_table/data/serialized_key` に格納されます。 -また、キー数には 4 というソフトリミットがあります。 - -同じ ZooKeeper パス上に複数のテーブルが作成された場合、そのパスを使用しているテーブルが少なくとも 1 つ存在する限り、値は永続化されます。\ -そのため、テーブル作成時に `ON CLUSTER` 句を使用して、複数の ClickHouse インスタンス間でデータを共有することが可能です。\ -もちろん、関連しない ClickHouse インスタンス間であっても、同じパスを指定して手動で `CREATE TABLE` を実行することで、同様のデータ共有効果を得ることができます。 - - -## サポートされている操作 {#supported-operations} - -### 挿入 {#inserts} - -新しい行が `KeeperMap` に挿入されるとき、キーが存在しない場合は、そのキー用の新しいエントリが作成されます。 -キーが存在し、かつ `keeper_map_strict_mode` が `true` に設定されている場合は、例外がスローされます。そうでない場合、そのキーに対する値は上書きされます。 - -例: - -```sql -INSERT INTO keeper_map_table VALUES ('some key', 1, 'value', 3.2); -``` - -### 削除 {#deletes} - -行は `DELETE` クエリまたは `TRUNCATE` を使用して削除できます。 -キーが存在しており、設定 `keeper_map_strict_mode` が `true` の場合、データの取得および削除は、それらをアトミックに実行できる場合にのみ成功します。 - -```sql -DELETE FROM keeper_map_table WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -TRUNCATE TABLE keeper_map_table; -``` - -### 更新 {#updates} - -値は `ALTER TABLE` クエリを使用して更新できます。プライマリキーは更新できません。 -`keeper_map_strict_mode` を `true` に設定すると、データの取得および更新は、アトミックに実行された場合にのみ成功します。 - -```sql -ALTER TABLE keeper_map_table UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; -``` - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [ClickHouse と Hex を利用したリアルタイム分析アプリの構築](https://clickhouse.com/blog/building-real-time-applications-with-clickhouse-and-hex-notebook-keeper-engine) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md deleted file mode 100644 index ad156123378..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -description: 'Memory エンジンはデータを圧縮せずに RAM 上に保持します。データは取り込まれたときの形式そのままで保存されます。言い換えると、このテーブルからの読み取りでは追加の処理コストが一切発生しません。' -sidebar_label: 'Memory' -sidebar_position: 110 -slug: /engines/table-engines/special/memory -title: 'Memory テーブルエンジン' -doc_type: 'reference' ---- - - - -# Memory テーブルエンジン {#memory-table-engine} - -:::note -ClickHouse Cloud 上で Memory テーブルエンジンを使用する場合、データは(設計上)すべてのノード間でレプリケートされません。すべてのクエリが同じノードにルーティングされ、Memory テーブルエンジンが期待どおりに動作することを保証するには、次のいずれかを行ってください: -- 同一セッション内で、すべての操作を実行する -- TCP またはネイティブインターフェース(スティッキー接続をサポート)を使用するクライアント、たとえば [clickhouse-client](/interfaces/cli) を使用する -::: - -Memory エンジンは、圧縮されていない形式でデータを RAM に保存します。データは読み取り時に受け取ったものとまったく同じ形式で保存されます。言い換えると、このテーブルからの読み取りコストはほぼゼロです。 -同時データアクセスは同期制御されます。ロック時間は短く、読み取りと書き込み操作は互いにブロックしません。 -インデックスはサポートされません。読み取りは並列化されます。 - -ディスクからの読み取りやデータの解凍、デシリアライズがないため、単純なクエリでは最大スループット(10 GB/秒超)が得られます。(多くの場合、MergeTree エンジンのスループットもほぼ同等であることに注意してください。) -サーバーを再起動すると、テーブル内のデータは消失し、テーブルは空になります。 -通常、このテーブルエンジンを使用する必然性はあまりありません。ただし、テスト用途や、比較的少ない行数(おおよそ 100,000,000 行まで)に対して最大速度が求められるタスクには使用できます。 - -Memory エンジンは、クエリの外部データ用一時テーブル(「クエリを処理するための外部データ」のセクションを参照)や、`GLOBAL IN` の実装(「IN 演算子」のセクションを参照)に、システムによって使用されます。 - -Memory エンジンのテーブルサイズを制限するために上限および下限を指定でき、事実上、循環バッファとして動作させることができます([Engine Parameters](#engine-parameters) を参照)。 - - - -## エンジンパラメーター {#engine-parameters} - -- `min_bytes_to_keep` — メモリテーブルにサイズ制限がある場合に保持する最小バイト数。 - - デフォルト値: `0` - - `max_bytes_to_keep` が必要 -- `max_bytes_to_keep` — メモリテーブル内で保持する最大バイト数。各挿入時に最も古い行が削除されます(リングバッファ方式)。大きなブロックを追加する際、削除対象となる最古の行バッチが `min_bytes_to_keep` の制限内に収まる場合は、最大バイト数が指定した上限を超えることがあります。 - - デフォルト値: `0` -- `min_rows_to_keep` — メモリテーブルにサイズ制限がある場合に保持する最小行数。 - - デフォルト値: `0` - - `max_rows_to_keep` が必要 -- `max_rows_to_keep` — メモリテーブル内で保持する最大行数。各挿入時に最も古い行が削除されます(リングバッファ方式)。大きなブロックを追加する際、削除対象となる最古の行バッチが `min_rows_to_keep` の制限内に収まる場合は、最大行数が指定した上限を超えることがあります。 - - デフォルト値: `0` -- `compress` — メモリ上のデータを圧縮するかどうか。 - - デフォルト値: `false` - - - -## 使用方法 {#usage} - -**設定を初期化する** - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**設定の変更** - -```sql -ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**注意:** `bytes` と `rows` の両方の上限パラメータは同時に設定できますが、`max` と `min` のうち小さい方の値が優先されます。 - - -## 例 {#examples} - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; - -/* 1. 最も古いブロックが最小しきい値により削除されないことをテスト - 3000 行 */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 8'192 バイト - -/* 2. 削除されないブロックを追加 */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 1'024 バイト - -/* 3. 最も古いブロックが削除されることをテスト - 9216 バイト - 1100 */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 8'192 バイト - -/* 4. 非常に大きなブロックがすべてを置き換えることを確認 */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 65'536 バイト - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` - -また、行の場合は次のとおりです: - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 4000, max_rows_to_keep = 10000; - -/* 1. 最古のブロックが最小しきい値により削除されないことを確認 - 3000 行 */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 1'600 行 - -/* 2. 削除されないブロックを追加する */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 100 行 - -/* 3. 最古のブロックが削除されることを確認 - 9216 バイト - 1100 行 */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 1'000 行 - -/* 4. 非常に大きなブロックがすべてを置き換えることを確認 */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 10'000 行 - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md deleted file mode 100644 index 90e44282d19..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: '`Merge` エンジン(`MergeTree` と混同しないでください)は自分自身ではデータを保存せず、任意の数の他のテーブルから同時に読み出すことができます。' -sidebar_label: 'Merge' -sidebar_position: 30 -slug: /engines/table-engines/special/merge -title: 'Merge テーブルエンジン' -doc_type: 'reference' ---- - - - -# Merge テーブルエンジン {#merge-table-engine} - -`Merge` エンジン(`MergeTree` と混同しないでください)は、自身ではデータを保存せず、任意の数の他のテーブルから同時に読み取ることができます。 - -読み取りは自動的に並列化されます。テーブルへの書き込みはサポートされていません。読み取り時には、存在する場合は、実際に読み出されるテーブルのインデックスが使用されます。 - - - -## テーブルを作成する {#creating-a-table} - -```sql -CREATE TABLE ... Engine=Merge(db_name, tables_regexp) -``` - - -## エンジンパラメータ {#engine-parameters} - -### `db_name` {#db_name} - -`db_name` — 指定可能な値: - - データベース名 - - データベース名の文字列を返す定数式(例: `currentDatabase()`) - - `REGEXP(expression)`。ここで `expression` は DB 名にマッチする正規表現。 - -### `tables_regexp` {#tables_regexp} - -`tables_regexp` — 指定した DB または複数の DB 内のテーブル名にマッチさせるための正規表現。 - -正規表現 — [re2](https://github.com/google/re2)(PCRE のサブセットをサポート)、大文字小文字を区別します。 -正規表現内での記号のエスケープについては「match」セクションの注意事項を参照してください。 - - - -## 使用方法 {#usage} - -テーブルを読み取り対象として選択する際は、たとえ正規表現にマッチしても `Merge` テーブル自体は選択されません。これはループを避けるためです。 -互いのデータを延々と読み合おうとする 2 つの `Merge` テーブルを作成することも技術的には可能ですが、これは望ましくありません。 - -`Merge` エンジンの典型的な使用方法は、多数の `TinyLog` テーブルを 1 つのテーブルであるかのように扱うことです。 - - - -## 例 {#examples} - -**例 1** - -2 つのデータベース `ABC_corporate_site` と `ABC_store` があるとします。`all_visitors` テーブルには、両方のデータベースにある `visitors` テーブルの ID が含まれます。 - -```sql -CREATE TABLE all_visitors (id UInt32) ENGINE=Merge(REGEXP('ABC_*'), 'visitors'); -``` - -**例 2** - -古いテーブル `WatchLog_old` があり、データを新しいテーブル `WatchLog_new` に移行せずにパーティション分割を変更することにしたとします。この場合、両方のテーブルのデータを参照する必要があります。 - -```sql -CREATE TABLE WatchLog_old( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -ORDER BY (date, UserId, EventType); - -INSERT INTO WatchLog_old VALUES ('2018-01-01', 1, 'hit', 3); - -CREATE TABLE WatchLog_new( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -PARTITION BY date -ORDER BY (UserId, EventType) -SETTINGS index_granularity=8192; - -INSERT INTO WatchLog_new VALUES ('2018-01-02', 2, 'hit', 3); - -CREATE TABLE WatchLog AS WatchLog_old ENGINE=Merge(currentDatabase(), '^WatchLog'); - -SELECT * FROM WatchLog; -``` - -```text -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-01 │ 1 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-02 │ 2 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -``` - - -## 仮想カラム {#virtual-columns} - -- `_table` — データが読み取られたテーブルの名前。型: [String](../../../sql-reference/data-types/string.md)。 - - `_table` に対してフィルタを行う場合(例: `WHERE _table='xyz'`)、フィルタ条件を満たすテーブルのみが読み取られます。 - -- `_database` — データが読み取られたデータベースの名前。型: [String](../../../sql-reference/data-types/string.md)。 - -**参照** - -- [仮想カラム](../../../engines/table-engines/index.md#table_engines-virtual_columns) -- [merge](../../../sql-reference/table-functions/merge.md) テーブル関数 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md deleted file mode 100644 index 8a3d9a0cfdd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '`Null` テーブルに書き込むと、データは無視されます。`Null` テーブルから読み出すと、結果は空になります。' -sidebar_label: 'Null' -sidebar_position: 50 -slug: /engines/table-engines/special/null -title: 'Null テーブルエンジン' -doc_type: 'reference' ---- - -# Null テーブルエンジン {#null-table-engine} - -`Null` テーブルにデータを書き込むと、そのデータは無視されます。 -`Null` テーブルから読み出すと、レスポンスは空になります。 - -`Null` テーブルエンジンは、データ変換後に元のデータが不要になるようなユースケースに有用です。 -このような用途のために、`Null` テーブル上にマテリアライズドビューを作成できます。 -テーブルに書き込まれたデータはビューが消費しますが、元の生データは破棄されます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md deleted file mode 100644 index 0cb957d8913..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '常に RAM 上に保持されるデータセットです。`IN` 演算子の右側での利用を想定しています。' -sidebar_label: 'Set' -sidebar_position: 60 -slug: /engines/table-engines/special/set -title: 'Set テーブルエンジン' -doc_type: 'reference' ---- - - - -# Set テーブルエンジン {#set-table-engine} - -:::note -ClickHouse Cloud では、サービスが 25.4 より前のバージョンで作成されている場合、`SET compatibility=25.4` を使って互換性を少なくとも 25.4 に設定する必要があります。 -::: - -常に RAM 上に保持されるデータセットです。`IN` 演算子の右辺での利用を想定しています(「IN 演算子」のセクションを参照してください)。 - -`INSERT` を使用してテーブルにデータを挿入できます。新しい要素はデータセットに追加され、重複は無視されます。 -ただし、このテーブルに対して `SELECT` を実行することはできません。データを取得する唯一の方法は、`IN` 演算子の右側で使用することです。 - -データは常に RAM に配置されます。`INSERT` の際、挿入されたデータのブロックはディスク上のテーブルのディレクトリにも書き込まれます。サーバー起動時に、このデータが RAM に読み込まれます。つまり、再起動後もデータは保持されます。 - -サーバーが異常再起動した場合、ディスク上のデータブロックが失われたり破損したりする可能性があります。後者の場合、破損したデータを含むファイルを手動で削除する必要があるかもしれません。 - -### 制限事項と設定 {#join-limitations-and-settings} - -テーブルを作成する際、以下の設定が適用されます。 - -#### Persistent {#persistent} - -Set および [Join](/engines/table-engines/special/join) テーブルエンジンに対して永続化を無効化します。 - -I/O のオーバーヘッドを削減します。パフォーマンスを重視し、永続化を必要としないシナリオに適しています。 - -設定可能な値: - -- 1 — 有効。 -- 0 — 無効。 - -デフォルト値: `1`。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md deleted file mode 100644 index 013a4ca2277..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: 'リモート HTTP/HTTPS サーバーとの間でデータをクエリします。このエンジンは File エンジンに似ています。' -sidebar_label: 'URL' -sidebar_position: 80 -slug: /engines/table-engines/special/url -title: 'URL テーブルエンジン' -doc_type: 'reference' ---- - - - -# URL テーブルエンジン {#url-table-engine} - -リモートの HTTP/HTTPS サーバーとの間でデータをクエリします。このエンジンは [File](../../../engines/table-engines/special/file.md) エンジンに類似しています。 - -構文: `URL(URL [,Format] [,CompressionMethod])` - -- `URL` パラメータは、Uniform Resource Locator の構造に準拠している必要があります。指定された URL は HTTP もしくは HTTPS を使用するサーバーを指している必要があります。サーバーからレスポンスを取得するために追加のヘッダーは不要です。 - -- `Format` には、ClickHouse が `SELECT` クエリおよび必要に応じて `INSERT` で使用できるフォーマットを指定します。サポートされているフォーマットの一覧については、[Formats](/interfaces/formats#formats-overview) を参照してください。 - - この引数が指定されていない場合、ClickHouse は `URL` パラメータのサフィックスから自動的にフォーマットを判別します。`URL` パラメータのサフィックスがサポート対象のいずれのフォーマットとも一致しない場合、テーブルの作成は失敗します。たとえば、エンジン式 `URL('http://localhost/test.json')` の場合、`JSON` フォーマットが適用されます。 - -- `CompressionMethod` は、HTTP 本文を圧縮するかどうかを示します。圧縮が有効な場合、URL エンジンが送信する HTTP パケットには、使用している圧縮方式を示す `Content-Encoding` ヘッダーが含まれます。 - -圧縮を有効にする前に、`URL` パラメータで指定されたリモート HTTP エンドポイントが、対応する圧縮アルゴリズムをサポートしていることを確認してください。 - -サポートされている `CompressionMethod` は次のいずれかです: -- gzip または gz -- deflate -- brotli または br -- lzma または xz -- zstd または zst -- lz4 -- bz2 -- snappy -- none -- auto - -`CompressionMethod` が指定されていない場合、デフォルトは `auto` です。これは、ClickHouse が `URL` パラメータのサフィックスから自動的に圧縮方式を判別することを意味します。サフィックスが上記のいずれかの圧縮方式と一致する場合は対応する圧縮が適用され、一致しない場合は圧縮は有効になりません。 - -たとえば、エンジン式 `URL('http://localhost/test.gzip')` の場合、`gzip` 圧縮方式が適用されますが、`URL('http://localhost/test.fr')` の場合、サフィックス `fr` が上記のいずれの圧縮方式とも一致しないため、圧縮は有効になりません。 - - - -## 使用方法 {#using-the-engine-in-the-clickhouse-server} - -`INSERT` および `SELECT` クエリは、それぞれ `POST` および `GET` リクエストに変換されます。`POST` リクエストを処理するには、リモートサーバーが [チャンク転送エンコーディング](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) をサポートしている必要があります。 - -[max_http_get_redirects](/operations/settings/settings#max_http_get_redirects) 設定を使用して、HTTP GET リダイレクトの最大ホップ数を制限できます。 - - - -## 例 {#example} - -**1.** サーバー上に `url_engine_table` テーブルを作成します: - -```sql -CREATE TABLE url_engine_table (word String, value UInt64) -ENGINE=URL('http://127.0.0.1:12345/', CSV) -``` - -**2.** 標準の Python 3 ツールを使って簡易な HTTP サーバーを作成し、 -起動します。 - -```python3 -from http.server import BaseHTTPRequestHandler, HTTPServer - -class CSVHTTPServer(BaseHTTPRequestHandler): - def do_GET(self): - self.send_response(200) - self.send_header('Content-type', 'text/csv') - self.end_headers() - - self.wfile.write(bytes('Hello,1\nWorld,2\n', "utf-8")) - -if __name__ == "__main__": - server_address = ('127.0.0.1', 12345) - HTTPServer(server_address, CSVHTTPServer).serve_forever() -``` - -```bash -$ python3 server.py -``` - -**3.** リクエストデータ: - -```sql -SELECT * FROM url_engine_table -``` - -```text -┌─word──┬─value─┐ -│ Hello │ 1 │ -│ World │ 2 │ -└───────┴───────┘ -``` - - -## 実装の詳細 {#details-of-implementation} - -- 読み取りと書き込みは並行して実行できます -- 以下はサポートされていません: - - `ALTER` および `SELECT...SAMPLE` 操作 - - インデックス - - レプリケーション - - - -## 仮想カラム {#virtual-columns} - -- `_path` — `URL` へのパス。型: `LowCardinality(String)`。 -- `_file` — `URL` のリソース名。型: `LowCardinality(String)`。 -- `_size` — リソースのサイズ(バイト単位)。型: `Nullable(UInt64)`。サイズが不明な場合、値は `NULL`。 -- `_time` — ファイルの最終更新時刻。型: `Nullable(DateTime)`。時刻が不明な場合、値は `NULL`。 -- `_headers` - HTTP レスポンスヘッダー。型: `Map(LowCardinality(String), LowCardinality(String))`。 - - - -## ストレージ設定 {#storage-settings} - -- [engine_url_skip_empty_files](/operations/settings/settings.md#engine_url_skip_empty_files) - 読み込み時に空のファイルをスキップできるようにします。既定では無効です。 -- [enable_url_encoding](/operations/settings/settings.md#enable_url_encoding) - URI パスのエンコード/デコード処理を有効/無効にできます。既定では有効です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md deleted file mode 100644 index 16757bf68e1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ビューの実装に使用されます(詳細は `CREATE VIEW` クエリを参照してください)。データ自体は保存せず、指定された `SELECT` クエリのみを保持します。このエンジンを使用するテーブルから読み取る際には、このクエリを実行し(クエリから不要な列をすべて削除した上で)、結果を返します。' -sidebar_label: 'View' -sidebar_position: 90 -slug: /engines/table-engines/special/view -title: 'View テーブルエンジン' -doc_type: 'reference' ---- - -# View テーブルエンジン {#view-table-engine} - -ビューを実装するために使用されます(詳細は `CREATE VIEW query` を参照してください)。データ自体は保存せず、指定された `SELECT` クエリだけを保持します。テーブルから読み取る際には、このクエリを実行し(不要なカラムはクエリからすべて削除され)、結果を返します。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/clickpipes/kafka/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/clickpipes/kafka/index.md new file mode 100644 index 00000000000..aa04821e35a --- /dev/null +++ b/i18n/jp/docusaurus-plugin-content-docs/current/integrations/data-ingestion/clickpipes/kafka/index.md @@ -0,0 +1,25 @@ +--- + + +description: 'Kafka ClickPipes セクションの目次付きランディングページ' +slug: /integrations/clickpipes/kafka +sidebar_position: 1 +title: 'Kafka ClickPipes' +doc_type: 'landing-page' +integration: + - support_level: 'core' + - category: 'clickpipes' +keywords: ['Kafka ClickPipes', 'Apache Kafka', 'ストリーミングインジェスト', 'リアルタイムデータ', 'メッセージブローカー'] +--- + +{/*AUTOGENERATED_START*/ } + +| ページ | 説明 | +| ------------------------------------------------------------------------------------------- | ------------------------------------------------------------ | +| [最初の Kafka ClickPipe の作成](/integrations/clickpipes/kafka/create-your-first-kafka-clickpipe) | 最初の Kafka ClickPipe を作成するためのステップバイステップガイド。 | +| [Kafka ClickPipe のスキーマレジストリ](/integrations/clickpipes/kafka/schema-registries) | スキーマ管理のためにスキーマレジストリと ClickPipes を連携させる方法。 | +| [リファレンス](/integrations/clickpipes/kafka/reference) | Kafka ClickPipes がサポートするフォーマット、ソース、配信セマンティクス、認証、および実験的機能の詳細。 | +| [ベストプラクティス](/integrations/clickpipes/kafka/best-practices) | Kafka ClickPipes を扱う際に従うべきベストプラクティスの詳細。 | +| [Kafka ClickPipes FAQ](/integrations/clickpipes/kafka/faq) | Kafka 向け ClickPipes に関するよくある質問。 | + +{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md deleted file mode 100644 index b19957946e2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: 'ClickHouse の Apache Arrow Flight インターフェイスに関するドキュメントで、Flight SQL クライアントから ClickHouse への接続を可能にします' -sidebar_label: 'Arrow Flight インターフェイス' -sidebar_position: 26 -slug: /interfaces/arrowflight -title: 'Arrow Flight インターフェイス' -doc_type: 'reference' ---- - - - -# Apache Arrow Flight インターフェイス {#apache-arrow-flight-interface} - -ClickHouse は、Arrow IPC フォーマットを gRPC 上で利用して効率的なカラム型データ転送を行う、高性能な RPC フレームワークである [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) プロトコルとの連携をサポートしています。 - -このインターフェイスにより、Flight SQL クライアントは ClickHouse に対してクエリを実行し、結果を Arrow フォーマットで取得できます。これにより、分析ワークロード向けに高スループットかつ低レイテンシなクエリ処理が可能になります。 - - - -## 機能 {#features} - -* Arrow Flight SQL プロトコル経由で SQL クエリを実行 -* クエリ結果を Apache Arrow 形式でストリーミング配信 -* Arrow Flight をサポートする BI ツールや独自のデータアプリケーションとの統合 -* gRPC を用いた軽量かつ高性能な通信 - - - -## 制限事項 {#limitations} - -Arrow Flight インターフェイスは現在、実験的な段階であり、活発に開発が進められています。既知の制限事項には次のようなものがあります。 - -* ClickHouse 固有の複雑な SQL 機能に対するサポートが限定的です -* すべての Arrow Flight SQL メタデータ操作がまだ実装されていません -* リファレンス実装には、組み込みの認証機能や TLS 設定はありません - -互換性の問題が発生した場合やコントリビュートを希望される場合は、ClickHouse リポジトリで[issue を作成](https://github.com/ClickHouse/ClickHouse/issues)してください。 - - - -## Arrow Flight サーバーの実行 {#running-server} - -自己管理の ClickHouse インスタンスで Arrow Flight サーバーを有効化するには、サーバー設定に次の構成を追加します。 - -```xml - - 9005 - -``` - -ClickHouse サーバーを再起動します。起動に成功すると、次のようなログメッセージが表示されます。 - -```bash -{} Application: Arrow Flight互換プロトコル: 0.0.0.0:9005 -``` - - -## Arrow Flight SQL を使用して ClickHouse に接続する {#connecting-to-clickhouse} - -Arrow Flight SQL をサポートする任意のクライアントを利用できます。たとえば、`pyarrow` を使う場合は次のとおりです。 - -```python -import pyarrow.flight - -client = pyarrow.flight.FlightClient("grpc://localhost:9005") -ticket = pyarrow.flight.Ticket(b"SELECT number FROM system.numbers LIMIT 10") -reader = client.do_get(ticket) - -for batch in reader: - print(batch.to_pandas()) -``` - - -## 互換性 {#compatibility} - -Arrow Flight インターフェースは、次のような技術スタックで構築されたカスタムアプリケーションを含め、Arrow Flight SQL をサポートするツールと互換性があります。 - -* Python (`pyarrow`) -* Java (`arrow-flight`) -* C++ およびその他の gRPC 互換言語 - -利用しているツール向けにネイティブな ClickHouse コネクタ(例: JDBC、ODBC)が利用可能な場合、パフォーマンスやフォーマット互換性の理由で Arrow Flight が明示的に必要な場合を除き、そちらを優先して使用してください。 - - - -## クエリのキャンセル {#query-cancellation} - -長時間実行中のクエリは、クライアント側で gRPC 接続を閉じることでキャンセルできます。より高度なキャンセル機能のサポートの追加が計画されています。 - ---- - -詳しくは次を参照してください。 - -* [Apache Arrow Flight SQL specification](https://arrow.apache.org/docs/format/FlightSql.html) -* [ClickHouse GitHub Issue #7554](https://github.com/ClickHouse/ClickHouse/issues/7554) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cli.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cli.md deleted file mode 100644 index 743e873d7e3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cli.md +++ /dev/null @@ -1,918 +0,0 @@ ---- -description: 'ClickHouse コマンドラインクライアントインターフェースのドキュメント' -sidebar_label: 'ClickHouse クライアント' -sidebar_position: 17 -slug: /interfaces/cli -title: 'ClickHouse クライアント' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; -import cloud_connect_button from '@site/static/images/_snippets/cloud-connect-button.png'; -import connection_details_native from '@site/static/images/_snippets/connection-details-native.png'; -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -ClickHouse は、ClickHouse サーバーに対して直接 SQL クエリを実行するためのネイティブなコマンドラインクライアントを提供します。 -このクライアントは、対話型モード(その場でのクエリ実行)とバッチモード(スクリプトや自動化向け)の両方をサポートします。 -クエリ結果はターミナルに表示することも、ファイルにエクスポートすることもでき、Pretty、CSV、JSON などを含むすべての ClickHouse 出力[フォーマット](formats.md)に対応しています。 - -このクライアントでは、プログレスバーや読み取られた行数、処理されたバイト数、クエリの実行時間などを通じて、クエリの実行状況をリアルタイムに確認できます。 -[コマンドラインオプション](#command-line-options)と[設定ファイル](#configuration_files)の両方をサポートします。 - - -## インストール {#install} - -ClickHouse をダウンロードするには、次のコマンドを実行します。 - -```bash -curl https://clickhouse.com/ | sh -``` - -これもインストールするには、次を実行してください: - -```bash -sudo ./clickhouse install -``` - -他のインストール方法については、[Install ClickHouse](../getting-started/install/install.mdx) を参照してください。 - -クライアントとサーバーのバージョンが異なっていても互換性はありますが、古いクライアントでは一部の機能が利用できない場合があります。クライアントとサーバーには同じバージョンを使用することを推奨します。 - - -## 実行 {#run} - -:::note -ClickHouse をダウンロードしただけでインストールしていない場合は、`clickhouse-client` ではなく `./clickhouse client` を使用してください。 -::: - -ClickHouse サーバーに接続するには、次を実行してください。 - -```bash -$ clickhouse-client --host server - -ClickHouse client version 24.12.2.29 (official build). -Connecting to server:9000 as user default. -Connected to ClickHouse server version 24.12.2. - -:) -``` - -必要に応じて、以下の追加接続設定を指定します: - -| Option | Description | -| -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | -| `--port ` | ClickHouse サーバーが接続を受け付けるポートです。デフォルトのポートは 9440(TLS)と 9000(TLS なし)です。ClickHouse Client は HTTP(S) ではなくネイティブプロトコルを使用する点に注意してください。 | -| `-s [ --secure ]` | TLS を使用するかどうか(通常は自動検出されます)。 | -| `-u [ --user ] ` | 接続に使用するデータベースユーザーです。デフォルトでは `default` ユーザーとして接続します。 | -| `--password ` | データベースユーザーのパスワードです。接続に使用するパスワードは、構成ファイル内で指定することもできます。パスワードを指定しない場合、クライアントが入力を求めます。 | -| `-c [ --config ] ` | ClickHouse Client の構成ファイルのパスです。構成ファイルがデフォルトのいずれかの場所にない場合に指定します。詳しくは [Configuration Files](#configuration_files) を参照してください。 | -| `--connection ` | [configuration file](#connection-credentials) で事前定義された接続設定の名前です。 | - -コマンドラインオプションの完全な一覧については、[Command Line Options](#command-line-options) を参照してください。 - - -### ClickHouse Cloud への接続 {#connecting-cloud} - -ClickHouse Cloud サービスの詳細は、ClickHouse Cloud コンソールで確認できます。接続したいサービスを選択し、**Connect** をクリックします。 - -ClickHouse Cloud サービスの Connect ボタン - -
- -
- -**Native** を選択すると、詳細情報が表示され、例として `clickhouse-client` コマンドが示されます。 - -ClickHouse Cloud のネイティブ TCP 接続の詳細 - -### 接続情報を設定ファイルに保存する {#connection-credentials} - -1台以上のClickHouseサーバーに対する接続情報を[設定ファイル](#configuration_files)に保存できます。 - -フォーマットは次のようになります。 - -```xml - - - - default - hostname - 9440 - 1 - default - password - - - - - - - -``` - -[設定ファイルに関するセクション](#configuration_files)を参照してください。 - -:::note -クエリ構文に焦点を当てるため、以降の例では接続情報(`--host`、`--port` など)を省略しています。実際にコマンドを使用する際は、必ずこれらを指定してください。 -::: - - -## インタラクティブモード {#interactive-mode} - -### インタラクティブモードを使用する {#using-interactive-mode} - -ClickHouse をインタラクティブモードで実行するには、次のコマンドを実行します。 - -```bash -clickhouse-client -``` - -これで Read-Eval-Print Loop (REPL) が開き、対話的に SQL クエリを入力できるようになります。 -接続が確立されると、クエリを入力するためのプロンプトが表示されます。 - -```bash -ClickHouse client version 25.x.x.x -localhost:9000 にユーザー default として接続しています。 -ClickHouse サーバー バージョン 25.x.x.x に接続しました - -hostname :) -``` - -対話モードでは、デフォルトの出力フォーマットは `PrettyCompact` です。 -クエリの `FORMAT` 句でフォーマットを変更するか、コマンドラインオプション `--format` を指定して変更できます。 -`Vertical` フォーマットを使うには、`--vertical` を利用するか、クエリの末尾に `\G` を指定します。 -このフォーマットでは、それぞれの値が別々の行に出力されるため、横に長いテーブルを扱う際に便利です。 - -対話モードでは、`Enter` を押すと入力した内容がそのまま実行されます。 -クエリの末尾にセミコロンは必須ではありません。 - -クライアントは `-m, --multiline` パラメータを付けて起動できます。 -複数行のクエリを入力するには、改行の前にバックスラッシュ `\` を入力します。 -`Enter` を押した後、クエリの次の行の入力が求められます。 -クエリを実行するには、末尾にセミコロンを付けて `Enter` を押します。 - -ClickHouse Client は `replxx`(`readline` に類似)をベースとしているため、使い慣れたキーボードショートカットが利用でき、履歴も保持されます。 -履歴はデフォルトで `~/.clickhouse-client-history` に書き込まれます。 - -クライアントを終了するには、`Ctrl+D` を押すか、クエリの代わりに次のいずれかを入力します。 - -* `exit` または `exit;` -* `quit` または `quit;` -* `q`、`Q` または `:q` -* `logout` または `logout;` - - -### クエリ処理情報 {#processing-info} - -クエリを処理するとき、クライアントは次の情報を表示します。 - -1. 進捗状況。デフォルトでは 1 秒間に最大 10 回まで更新されます。 - 短時間で完了するクエリの場合、進捗が表示される前に処理が完了することがあります。 -2. デバッグ用の、パース後に整形されたクエリ。 -3. 指定されたフォーマットでの結果。 -4. 結果の行数、経過時間、およびクエリ処理の平均速度。 - ここでのデータ量はすべて非圧縮データに対するものです。 - -長時間実行中のクエリは、`Ctrl+C` を押すことでキャンセルできます。 -ただし、サーバー側でリクエストが中断されるまで、しばらく待つ必要があります。 -処理の特定の段階では、クエリをキャンセルすることはできません。 -待たずに 2 回目の `Ctrl+C` を押した場合、クライアントは終了します。 - -ClickHouse Client では、クエリ用に外部データ(外部一時テーブル)を渡すことができます。 -詳細については、「[クエリ処理用の外部データ](../engines/table-engines/special/external-data.md)」のセクションを参照してください。 - -### エイリアス {#cli_aliases} - -REPL 内では次のエイリアスを使用できます: - -- `\l` - SHOW DATABASES -- `\d` - SHOW TABLES -- `\c ` - USE DATABASE -- `.` - 直前のクエリを再実行する - -### キーボードショートカット {#keyboard_shortcuts} - -- `Alt (Option) + Shift + e` - 現在のクエリをエディタで開きます。使用するエディタは環境変数 `EDITOR` で指定できます。デフォルトでは `vim` が使用されます。 -- `Alt (Option) + #` - 行をコメントアウトします。 -- `Ctrl + r` - あいまい検索で履歴を検索します。 - -利用可能なすべてのキーボードショートカットの一覧は [replxx](https://github.com/AmokHuginnsson/replxx/blob/1f149bf/src/replxx_impl.cxx#L262) にあります。 - -:::tip -macOS で Meta キー (Option) を正しく動作させるには: - -iTerm2: Preferences -> Profile -> Keys -> Left Option key に移動し、Esc+ をクリックします。 -::: - -## バッチモード {#batch-mode} - -### バッチモードの使用 {#using-batch-mode} - -ClickHouse Client を対話的に使用する代わりに、バッチモードで実行できます。 -バッチモードでは、ClickHouse は単一のクエリを実行するとすぐに終了し、対話的なプロンプトやループはありません。 - -次のように単一のクエリを指定できます: - -```bash -$ clickhouse-client "SELECT sum(number) FROM numbers(10)" -45 -``` - -`--query` コマンドラインオプションを利用することもできます: - -```bash -$ clickhouse-client --query "SELECT uniq(number) FROM numbers(10)" -10 -``` - -クエリは `stdin` からも指定できます: - -```bash -$ echo "SELECT avg(number) FROM numbers(10)" | clickhouse-client -4.5 -``` - -テーブル `messages` が存在することを前提として、コマンドラインからデータを挿入することもできます。 - -```bash -$ echo "Hello\nGoodbye" | clickhouse-client --query "INSERT INTO messages FORMAT CSV" -``` - -`--query` が指定されている場合、入力された内容は改行文字の後にリクエストへ追加されます。 - - -### リモート ClickHouse サービスに CSV ファイルを挿入する {#cloud-example} - -この例では、サンプルデータセットの CSV ファイル `cell_towers.csv` を、`default` データベース内の既存のテーブル `cell_towers` に挿入します。 - -```bash -clickhouse-client --host HOSTNAME.clickhouse.cloud \ - --port 9440 \ - --user default \ - --password PASSWORD \ - --query "INSERT INTO cell_towers FORMAT CSVWithNames" \ - < cell_towers.csv -``` - - -### コマンドラインからデータを挿入する例 {#more-examples} - -コマンドラインからデータを挿入する方法はいくつかあります。 -以下の例では、バッチモードを使用して、2行の CSV データを ClickHouse テーブルに挿入します。 - -```bash -echo -ne "1, 'some text', '2016-08-14 00:00:00'\n2, 'some more text', '2016-08-14 00:00:01'" | \ - clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -以下の例では、`cat <<_EOF` でヒアドキュメントを開始し、再度 `_EOF` が現れるまでのすべての内容を読み取って出力します。 - -```bash -cat <<_EOF | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -3, 'some text', '2016-08-14 00:00:00' -4, 'some more text', '2016-08-14 00:00:01' -_EOF -``` - -次の例では、`cat` を使用して file.csv の内容を標準出力に書き出し、その内容をパイプで `clickhouse-client` の入力として渡します。 - -```bash -cat file.csv | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -バッチモードでは、デフォルトのデータ[フォーマット](formats.md)は `TabSeparated` です。 -上の例に示したように、クエリの `FORMAT` 句でフォーマットを指定できます。 - - -## パラメーター付きクエリ {#cli-queries-with-parameters} - -クエリ内でパラメーターを指定し、コマンドラインオプションを使って値を渡すことができます。 -これにより、クライアント側で特定の動的な値を埋め込んだクエリ文字列を組み立てる必要がなくなります。 -例えば、次のようにします。 - -```bash -$ clickhouse-client --param_parName="[1, 2]" --query "SELECT {parName: Array(UInt16)}" -[1,2] -``` - -また、[インタラクティブ セッション](#interactive-mode)内からパラメータを設定することもできます。 - -```text -$ clickhouse-client -ClickHouse client version 25.X.X.XXX (official build). - -#highlight-next-line -:) SET param_parName='[1, 2]'; - -SET param_parName = '[1, 2]' - -Query id: 7ac1f84e-e89a-4eeb-a4bb-d24b8f9fd977 - -Ok. - -0 rows in set. Elapsed: 0.000 sec. - -#highlight-next-line -:) SELECT {parName:Array(UInt16)} - -SELECT {parName:Array(UInt16)} - -Query id: 0358a729-7bbe-4191-bb48-29b063c548a7 - - ┌─_CAST([1, 2]⋯y(UInt16)')─┐ -1. │ [1,2] │ - └──────────────────────────┘ - -1 row in set. Elapsed: 0.006 sec. -``` - - -### クエリ構文 {#cli-queries-with-parameters-syntax} - -クエリ内で、コマンドライン引数で指定したい値は、次の形式で中かっこで囲んで記述します。 - -```sql -{:} -``` - -| Parameter | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | プレースホルダー用の識別子。対応するコマンドラインオプションは `--param_=value` です。 | -| `data type` | パラメータの[データ型](../sql-reference/data-types/index.md)。

たとえば、`(integer, ('string', integer))` のようなデータ構造は、`Tuple(UInt8, Tuple(String, UInt8))` 型を持つことができます(他の[整数](../sql-reference/data-types/int-uint.md)型も使用できます)。

テーブル名、データベース名、カラム名をパラメータとして渡すことも可能であり、その場合はデータ型として `Identifier` を使用する必要があります。 | - - -### 使用例 {#cli-queries-with-parameters-examples} - -```bash -$ clickhouse-client --param_tuple_in_tuple="(10, ('dt', 10))" \ - --query "SELECT * FROM table WHERE val = {tuple_in_tuple:Tuple(UInt8, Tuple(String, UInt8))}" - -$ clickhouse-client --param_tbl="numbers" --param_db="system" --param_col="number" --param_alias="top_ten" \ - --query "SELECT {col:Identifier} as {alias:Identifier} FROM {db:Identifier}.{tbl:Identifier} LIMIT 10" -``` - - -## AI を活用した SQL 生成 {#ai-sql-generation} - -ClickHouse クライアントには、自然言語による説明から SQL クエリを生成するための AI 支援機能が組み込まれています。この機能により、ユーザーは高度な SQL の知識がなくても複雑なクエリを作成できます。 - -`OPENAI_API_KEY` または `ANTHROPIC_API_KEY` のいずれかの環境変数が設定されていれば、AI 支援機能は追加の設定なしでそのまま利用できます。より高度な設定については、[Configuration](#ai-sql-generation-configuration) セクションを参照してください。 - -### 使用方法 {#ai-sql-generation-usage} - -AI SQL 生成機能を利用するには、自然言語のクエリの先頭に `??` を付けてください: - -```bash -:) ?? 過去30日間に購入したすべてのユーザーを表示 -``` - -AI は次のことを行います: - -1. データベーススキーマを自動的に解析します -2. 把握したテーブルやカラムに基づいて、適切な SQL を生成します -3. 生成したクエリを直ちに実行します - - -### 例 {#ai-sql-generation-example} - -```bash -:) ?? count orders by product category - -スキーマ検出を伴うAI SQL生成を開始しています... -────────────────────────────────────────────────── - -🔍 list_databases - ➜ system, default, sales_db - -🔍 list_tables_in_database - database: sales_db - ➜ orders, products, categories - -🔍 get_schema_for_table - database: sales_db - table: orders - ➜ CREATE TABLE orders (order_id UInt64, product_id UInt64, quantity UInt32, ...) - -✨ SQLクエリが正常に生成されました! -────────────────────────────────────────────────── - -SELECT - c.name AS category, - COUNT(DISTINCT o.order_id) AS order_count -FROM sales_db.orders o -JOIN sales_db.products p ON o.product_id = p.product_id -JOIN sales_db.categories c ON p.category_id = c.category_id -GROUP BY c.name -ORDER BY order_count DESC -``` - - -### 設定 {#ai-sql-generation-configuration} - -AI による SQL 生成を行うには、ClickHouse Client の設定ファイルで AI プロバイダーを構成する必要があります。OpenAI、Anthropic、または OpenAI 互換の API サービスを使用できます。 - -#### 環境変数によるフォールバック {#ai-sql-generation-fallback} - -設定ファイルで AI 設定が指定されていない場合、ClickHouse Client は自動的に環境変数の利用を試みます。 - -1. まず `OPENAI_API_KEY` 環境変数を確認します -2. 見つからない場合は `ANTHROPIC_API_KEY` 環境変数を確認します -3. どちらも見つからない場合、AI 機能は無効になります - -これにより、設定ファイルなしで迅速にセットアップできます。 - -```bash -# OpenAIを使用する場合 {#using-openai} -export OPENAI_API_KEY=your-openai-key -clickhouse-client - -# Anthropicを使用する場合 {#using-anthropic} -export ANTHROPIC_API_KEY=your-anthropic-key -clickhouse-client -``` - - -#### 設定ファイル {#ai-sql-generation-configuration-file} - -AI 設定をより細かく制御するには、次の場所にある ClickHouse Client の設定ファイルで設定します: - -* `$XDG_CONFIG_HOME/clickhouse/config.xml`(または `XDG_CONFIG_HOME` が設定されていない場合は `~/.config/clickhouse/config.xml`)(XML 形式) -* `$XDG_CONFIG_HOME/clickhouse/config.yaml`(または `XDG_CONFIG_HOME` が設定されていない場合は `~/.config/clickhouse/config.yaml`)(YAML 形式) -* `~/.clickhouse-client/config.xml`(XML 形式、旧来の場所) -* `~/.clickhouse-client/config.yaml`(YAML 形式、旧来の場所) -* または `--config-file` で任意のパスを指定 - - - - ```xml - - - - your-api-key-here - - - openai - - - gpt-4o - - - - - - true - - - 0.0 - 1000 - 30 - 10 - - - - - - ``` - - - - ```yaml - ai: - # 必須: API キー(または環境変数で設定) - api_key: your-api-key-here - - # 必須: プロバイダータイプ (openai, anthropic) - provider: openai - - # 使用するモデル - model: gpt-4o - - # オプション: OpenAI 互換サービス向けのカスタム API エンドポイント - # base_url: https://openrouter.ai/api - - # スキーマアクセスを有効化 - AI にデータベース/テーブル情報の参照を許可 - enable_schema_access: true - - # 生成パラメータ - temperature: 0.0 # ランダム性を制御 (0.0 = 決定的) - max_tokens: 1000 # 応答の最大長 - timeout_seconds: 30 # リクエストのタイムアウト - max_steps: 10 # スキーマ探索ステップの最大回数 - - # オプション: カスタム system prompt - # system_prompt: | - # You are an expert ClickHouse SQL assistant. Convert natural language to SQL. - # Focus on performance and use ClickHouse-specific optimizations. - # Always return executable SQL without explanations. - ``` - - - -
- -**OpenAI 互換 API(例: OpenRouter)の使用:** - -```yaml -ai: - provider: openai # 互換性のため 'openai' を使用してください - api_key: your-openrouter-api-key - base_url: https://openrouter.ai/api/v1 - model: anthropic/claude-3.5-sonnet # OpenRouter のモデル命名規則を使用してください -``` - -**最小限の設定例:** - -```yaml -# 最小構成 - 環境変数のAPIキーを使用 {#minimal-config-uses-environment-variable-for-api-key} -ai: - provider: openai # OPENAI_API_KEY環境変数を使用 - -# 設定なし - 自動フォールバック {#no-config-at-all-automatic-fallback} -# (aiセクションが空または存在しない場合、OPENAI_API_KEY、次にANTHROPIC_API_KEYを試行) {#empty-or-no-ai-section-will-try-openai_api_key-then-anthropic_api_key} - -# モデルのみ上書き - 環境変数のAPIキーを使用 {#only-override-model-uses-env-var-for-api-key} -ai: - provider: openai - model: gpt-3.5-turbo -``` - - -### パラメーター {#ai-sql-generation-parameters} - -
-必須パラメーター - -- `api_key` - AI サービス用の API キー。環境変数で設定している場合は省略可能: - - OpenAI: `OPENAI_API_KEY` - - Anthropic: `ANTHROPIC_API_KEY` - - 注意: 設定ファイル内の API キーが環境変数より優先されます -- `provider` - AI プロバイダー: `openai` または `anthropic` - - 省略した場合は、利用可能な環境変数に基づいて自動的に選択されます - -
- -
-モデル設定 - -- `model` - 使用するモデル (デフォルト: プロバイダーごとのデフォルト値) - - OpenAI: `gpt-4o`, `gpt-4`, `gpt-3.5-turbo` など - - Anthropic: `claude-3-5-sonnet-20241022`, `claude-3-opus-20240229` など - - OpenRouter: `anthropic/claude-3.5-sonnet` のようなモデル名を使用 - -
- -
-接続設定 - -- `base_url` - OpenAI 互換サービス向けのカスタム API エンドポイント (任意) -- `timeout_seconds` - リクエストのタイムアウト秒数 (デフォルト: `30`) - -
- -
-スキーマ探索 - -- `enable_schema_access` - AI にデータベーススキーマを探索させるかどうか (デフォルト: `true`) -- `max_steps` - スキーマ探索でツールを呼び出すステップ数の上限 (デフォルト: `10`) - -
- -
-生成パラメーター - -- `temperature` - 出力のランダム性を制御します。0.0 = 決定的、1.0 = 創造的 (デフォルト: `0.0`) -- `max_tokens` - レスポンスの最大長 (トークン数) (デフォルト: `1000`) -- `system_prompt` - AI へのカスタム指示 (任意) - -
- -### 仕組み {#ai-sql-generation-how-it-works} - -AI SQL ジェネレーターは、複数のステップで処理を行います。 - - - -1. **スキーマ検出** - -AI は組み込みツールを使ってデータベースを探索します。 -- 利用可能なデータベースを一覧表示します -- 関連するデータベース内のテーブルを検出します -- `CREATE TABLE` ステートメントを用いてテーブル構造を確認します - -2. **クエリ生成** - -検出したスキーマに基づいて、AI が次のような SQL を生成します: -- 自然言語による意図に合致する -- 正しいテーブル名とカラム名を使用する -- 適切な結合や集約を適用する - -3. **実行** - -生成された SQL は自動的に実行され、その結果が表示されます。 - - - -### 制限事項 {#ai-sql-generation-limitations} - -- 有効なインターネット接続が必要 -- API の利用には、AI プロバイダーによるレート制限があり、料金が発生する -- 複雑なクエリでは、複数回の調整・改善が必要になる場合がある -- AI はスキーマ情報への読み取り専用アクセスのみを持ち、実データにはアクセスできない - -### セキュリティ {#ai-sql-generation-security} - -- API キーが ClickHouse サーバーに送信されることはありません -- AI はスキーマ情報(テーブル/カラム名と型)のみを参照し、実データにはアクセスしません -- 生成されるすべてのクエリは、既存のデータベース権限に従います - -## 接続文字列 {#connection_string} - -### 使用方法 {#connection-string-usage} - -ClickHouse Client は、[MongoDB](https://www.mongodb.com/docs/manual/reference/connection-string/)、[PostgreSQL](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING)、[MySQL](https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html#connecting-using-uri) と同様の接続文字列を使用して ClickHouse サーバーに接続する方法にも対応しています。構文は次のとおりです。 - -```text -clickhouse:[//[user[:password]@][hosts_and_ports]][/database][?query_parameters] -``` - -| コンポーネント(すべて任意) | 説明 | デフォルト | -| ------------------ | --------------------------------------------------------------------------------------------------- | ---------------- | -| `user` | データベースのユーザー名。 | `default` | -| `password` | データベースユーザーのパスワード。`:` が指定されていてパスワードが空の場合、クライアントはユーザーのパスワードの入力を求めます。 | - | -| `hosts_and_ports` | ホストおよび任意のポートのリスト `host[:port] [, host:[port]], ...`。 | `localhost:9000` | -| `database` | データベース名。 | `default` | -| `query_parameters` | キーと値のペアのリスト `param1=value1[,¶m2=value2], ...`。一部のパラメータでは値を指定する必要はありません。パラメータ名と値は大文字・小文字が区別されます。 | - | - - -### 注意事項 {#connection-string-notes} - -ユーザー名、パスワード、またはデータベースを接続文字列で指定している場合、`--user`、`--password`、`--database` で再度指定することはできません(その逆も同様です)。 - -host コンポーネントには、ホスト名または IPv4 / IPv6 アドレスを指定できます。 -IPv6 アドレスは角括弧で囲む必要があります。 - -```text -clickhouse://[2001:db8::1234] -``` - -接続文字列には複数のホストを含めることができます。 -ClickHouse クライアントは、これらのホストに左から右の順番で接続を試行します。 -一度接続が確立されると、残りのホストへの接続は試行されません。 - -接続文字列は `clickHouse-client` の最初の引数として指定する必要があります。 -接続文字列は、`--host` と `--port` を除く任意個数の他の [コマンドラインオプション](#command-line-options) と組み合わせて使用できます。 - -`query_parameters` には次のキーを指定できます: - -| Key | Description | -| ------------------ | --------------------------------------------------------------------------------------------------------- | -| `secure` (または `s`) | 指定すると、クライアントは TLS を利用した安全な接続でサーバーに接続します。詳細は [コマンドラインオプション](#command-line-options) の `--secure` を参照してください。 | - -**パーセントエンコード** - -以下のパラメータに含まれる US ASCII 以外の文字、スペース、および特殊文字は、[パーセントエンコード](https://en.wikipedia.org/wiki/URL_encoding)する必要があります: - -* `user` -* `password` -* `hosts` -* `database` -* `query parameters` - - -### 例 {#connection_string_examples} - -`localhost` のポート 9000 に接続し、クエリ `SELECT 1` を実行します。 - -```bash -clickhouse-client clickhouse://localhost:9000 --query "SELECT 1" -``` - -ユーザー `john`、パスワード `secret` を使用し、ホスト `127.0.0.1`、ポート `9000` で `localhost` に接続します - -```bash -clickhouse-client clickhouse://john:secret@127.0.0.1:9000 -``` - -`default` ユーザーとして、IPv6 アドレス `[::1]` を持つホスト `localhost` に、ポート `9000` で接続します。 - -```bash -clickhouse-client clickhouse://[::1]:9000 -``` - -マルチラインモードで、ポート 9000 の `localhost` に接続します。 - -```bash -clickhouse-client clickhouse://localhost:9000 '-m' -``` - -ユーザー `default` として、ポート 9000 で `localhost` に接続します。 - -```bash -clickhouse-client clickhouse://default@localhost:9000 - -# 以下と同等: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --user default -``` - -ポート 9000 で `localhost` に接続し、デフォルトのデータベースとして `my_database` を使用します。 - -```bash -clickhouse-client clickhouse://localhost:9000/my_database - -# 次と同等: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --database my_database -``` - -ポート 9000 の `localhost` に接続し、接続文字列で指定された `my_database` をデフォルトデータベースとして使用し、短縮パラメータ `s` によるセキュア接続を行います。 - -```bash -clickhouse-client clickhouse://localhost/my_database?s - -# 以下と同等: {#equivalent-to} -clickhouse-client clickhouse://localhost/my_database -s -``` - -デフォルトのホスト、ポート、ユーザー、データベースを使用して接続します。 - -```bash -clickhouse-client clickhouse: -``` - -デフォルトのホストのデフォルトポートに、ユーザー `my_user` としてパスワードなしで接続します。 - -```bash -clickhouse-client clickhouse://my_user@ - -# :と@の間に空のパスワードを指定すると、接続開始前にユーザーにパスワードの入力を求めます。 {#using-a-blank-password-between-and-means-to-asking-the-user-to-enter-the-password-before-starting-the-connection} -clickhouse-client clickhouse://my_user:@ -``` - -メールアドレスをユーザー名として使用して `localhost` に接続します。`@` 記号は `%40` にパーセントエンコードされます。 - -```bash -clickhouse-client clickhouse://some_user%40some_mail.com@localhost:9000 -``` - -2つのホストのいずれか(`192.168.1.15` または `192.168.1.25`)に接続します。 - -```bash -clickhouse-client clickhouse://192.168.1.15,192.168.1.25 -``` - - -## クエリ ID の形式 {#query-id-format} - -インタラクティブモードでは、ClickHouse Client は各クエリに対してクエリ ID を表示します。既定では、ID は次のような形式です。 - -```sql -クエリ ID: 927f137d-00f1-4175-8914-0dd066365e96 -``` - -設定ファイル内の `query_id_formats` タグでカスタムフォーマットを指定できます。フォーマット文字列内の `{query_id}` プレースホルダーはクエリ ID に置き換えられます。タグ内には複数のフォーマット文字列を指定できます。 -この機能を利用すると、クエリのプロファイリングを容易にする URL を生成できます。 - -**例** - -```xml - - - http://speedscope-host/#profileURL=qp%3Fid%3D{query_id} - - -``` - -上記の設定では、クエリ ID は次の形式で表示されます。 - -```response -speedscope:http://speedscope-host/#profileURL=qp%3Fid%3Dc8ecc783-e753-4b38-97f1-42cddfb98b7d -``` - - -## 設定ファイル {#configuration_files} - -ClickHouse Client は、次のうち最初に存在するファイルを使用します。 - -- `-c [ -C, --config, --config-file ]` パラメータで指定されたファイル -- `./clickhouse-client.[xml|yaml|yml]` -- `$XDG_CONFIG_HOME/clickhouse/config.[xml|yaml|yml]`(`XDG_CONFIG_HOME` が設定されていない場合は `~/.config/clickhouse/config.[xml|yaml|yml]`) -- `~/.clickhouse-client/config.[xml|yaml|yml]` -- `/etc/clickhouse-client/config.[xml|yaml|yml]` - -ClickHouse リポジトリにあるサンプル設定ファイルを参照してください:[`clickhouse-client.xml`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/client/clickhouse-client.xml) - - - - ```xml - - username - password - true - - - /etc/ssl/cert.pem - - - - ``` - - - ```yaml - user: username - password: 'password' - secure: true - openSSL: - client: - caConfig: '/etc/ssl/cert.pem' - ``` - - - -## 環境変数オプション {#environment-variable-options} - -ユーザー名、パスワード、ホストは、環境変数 `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD`、`CLICKHOUSE_HOST` で指定できます。 -コマンドライン引数 `--user`、`--password`、`--host`、または(指定されている場合)[接続文字列](#connection_string) は、環境変数による設定よりも優先されます。 - -## コマンドラインオプション {#command-line-options} - -すべてのコマンドラインオプションは、コマンドラインで直接指定することも、[設定ファイル](#configuration_files)で既定値として指定することもできます。 - -### 一般オプション {#command-line-options-general} - -| Option | Description | Default | -|-----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|------------------------------| -| `-c [ -C, --config, --config-file ] ` | クライアント設定ファイルの場所を指定します。設定ファイルがデフォルトの検索パス上にない場合に使用します。詳細は [Configuration Files](#configuration_files) を参照してください。 | - | -| `--help` | 使用方法の概要を表示して終了します。`--verbose` と組み合わせると、クエリ設定を含む利用可能なすべてのオプションを表示します。 | - | -| `--history_file ` | コマンド履歴を保存するファイルのパスを指定します。 | - | -| `--history_max_entries` | 履歴ファイルに保存するエントリの最大数を指定します。 | `1000000` (100万) | -| `--prompt ` | カスタムプロンプトを指定します。 | サーバーの `display_name` | -| `--verbose` | 出力の詳細度を上げます。 | - | -| `-V [ --version ]` | バージョンを表示して終了します。 | - | - -### 接続オプション {#command-line-options-connection} - -| Option | Description | Default | -|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------| -| `--connection ` | 設定ファイルで事前設定されている接続設定の名前。[Connection credentials](#connection-credentials) を参照してください。 | - | -| `-d [ --database ] ` | この接続でデフォルトとして使用するデータベースを選択します。 | サーバー設定で現在有効なデータベース(デフォルトでは `default`) | -| `-h [ --host ] ` | 接続先の ClickHouse サーバーのホスト名。ホスト名、IPv4 アドレス、または IPv6 アドレスを指定できます。複数のホストを指定する場合は、このオプションを複数回指定します。 | `localhost` | -| `--jwt ` | 認証に JSON Web Token (JWT) を使用します。

JWT によるサーバー側の認可は ClickHouse Cloud でのみ利用可能です。 | - | -| `--no-warnings` | クライアントがサーバーに接続するときに `system.warnings` からの警告を表示しないようにします。 | - | -| `--no-server-client-version-message` | クライアントがサーバーに接続するときに、サーバーとクライアントのバージョン不一致メッセージを表示しないようにします。 | - | -| `--password ` | データベースユーザーのパスワード。接続用パスワードは設定ファイルでも指定できます。パスワードを指定しない場合、クライアントが入力を求めます。 | - | -| `--port ` | サーバーが接続を受け付けるポート番号。デフォルトのポートは 9440 (TLS) と 9000 (非 TLS) です。

注: クライアントは HTTP(S) ではなくネイティブプロトコルを使用します。 | `--secure` が指定されている場合は `9440`、それ以外の場合は `9000`。ホスト名が `.clickhouse.cloud` で終わる場合は常に `9440` がデフォルトになります。 | -| `-s [ --secure ]` | TLS を使用するかどうか。

ポート 9440(デフォルトのセキュアポート)または ClickHouse Cloud に接続する場合は自動的に有効になります。

[設定ファイル](#configuration_files) で CA 証明書を設定する必要がある場合があります。利用可能な設定項目は [サーバー側の TLS 設定](../operations/server-configuration-parameters/settings.md#openssl) と同じです。 | ポート 9440 または ClickHouse Cloud に接続する場合に自動的に有効化 | -| `--ssh-key-file ` | サーバーに対して認証を行うための SSH 秘密鍵を格納したファイル。 | - | -| `--ssh-key-passphrase ` | `--ssh-key-file` で指定した SSH 秘密鍵のパスフレーズ。 | - | -| `-u [ --user ] ` | 接続時に使用するデータベースユーザー。 | `default` | - -:::note -`--host`、`--port`、`--user`、`--password` オプションの代わりに、クライアントは [connection strings](#connection_string)(接続文字列)もサポートしています。 -::: - -### クエリオプション {#command-line-options-query} - -| Option | Description | -|---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--param_=` | [パラメータ付きクエリ](#cli-queries-with-parameters) のパラメータ用の置換値。 | -| `-q [ --query ] ` | バッチモードで実行するクエリ。複数回指定できます(`--query "SELECT 1" --query "SELECT 2"`)、またはセミコロン区切りの複数クエリを 1 回で指定することもできます(`--query "SELECT 1; SELECT 2;"`)。後者の場合、フォーマットが `VALUES` 以外の `INSERT` クエリは空行で区切る必要があります。

単一のクエリは、パラメータ指定なしでも指定できます: `clickhouse-client "SELECT 1"`

`--queries-file` と同時には使用できません。 | -| `--queries-file ` | クエリを含むファイルへのパス。`--queries-file` は複数回指定できます(例: `--queries-file queries1.sql --queries-file queries2.sql`)。

`--query` と同時には使用できません。 | -| `-m [ --multiline ]` | 指定された場合、複数行のクエリを許可します(Enter キーを押してもクエリを送信しません)。クエリは末尾がセミコロンで終わったときのみ送信されます。 | - -### クエリ設定 {#command-line-options-query-settings} - -クエリ設定は、クライアントのコマンドラインオプションとして指定できます。たとえば次のようにします。 - -```bash -$ clickhouse-client --max_threads 1 -``` - -設定の一覧は [Settings](../operations/settings/settings.md) を参照してください。 - - -### フォーマットオプション {#command-line-options-formatting} - -| オプション | 説明 | デフォルト | -|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------| -| `-f [ --format ] ` | 結果を指定した形式で出力します。

サポートされている形式の一覧については、[入力および出力データのフォーマット](formats.md) を参照してください。 | `TabSeparated` | -| `--pager ` | すべての出力をこのコマンドにパイプします。通常は `less`(例: 幅の広い結果セットを表示するための `less -S`)などを指定します。 | - | -| `-E [ --vertical ]` | 結果の出力に [Vertical フォーマット](/interfaces/formats/Vertical) を使用します。これは `–-format Vertical` と同じです。このフォーマットでは、各値が個別の行に出力されるため、幅の広いテーブルを表示する際に便利です。 | - | - -### 実行の詳細 {#command-line-options-execution-details} - -| Option | Description | Default | -|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------| -| `--enable-progress-table-toggle` | Ctrl+Space キーを押すことで進捗テーブルの表示/非表示を切り替えられるようにします。進捗テーブルの出力が有効なインタラクティブモードでのみ有効です。 | `enabled` | -| `--hardware-utilization` | 進捗バーにハードウェア使用状況(利用率)の情報を出力します。 | - | -| `--memory-usage` | 指定した場合、非インタラクティブモードでメモリ使用量を `stderr` に出力します。

指定可能な値:
• `none` - メモリ使用量を出力しない
• `default` - バイト数を出力する
• `readable` - メモリ使用量を人間が読みやすい形式で出力する | - | -| `--print-profile-events` | `ProfileEvents` パケットを出力します。 | - | -| `--progress` | クエリ実行の進捗を出力します。

指定可能な値:
• `tty\|on\|1\|true\|yes` - インタラクティブモードで端末に出力する
• `err` - 非インタラクティブモードで `stderr` に出力する
• `off\|0\|false\|no` - 進捗の出力を無効にする | インタラクティブモードでは `tty`、非インタラクティブ(バッチ)モードでは `off` | -| `--progress-table` | クエリ実行中に変化するメトリクスを含む進捗テーブルを出力します。

指定可能な値:
• `tty\|on\|1\|true\|yes` - インタラクティブモードで端末に出力する
• `err` - 非インタラクティブモードで `stderr` に出力する
• `off\|0\|false\|no` - 進捗テーブルの出力を無効にする | インタラクティブモードでは `tty`、非インタラクティブ(バッチ)モードでは `off` | -| `--stacktrace` | 例外のスタックトレースを出力します。 | - | -| `-t [ --time ]` | 非インタラクティブモードでクエリ実行時間を `stderr` に出力します(ベンチマーク用)。 | - | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cpp.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cpp.md deleted file mode 100644 index fb82434a6e5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/cpp.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'ClickHouse C++ クライアントライブラリおよび u-server フレームワークとの統合に関するドキュメント' -sidebar_label: 'C++ クライアントライブラリ' -sidebar_position: 24 -slug: /interfaces/cpp -title: 'C++ クライアントライブラリ' -doc_type: 'reference' ---- - -# C++ クライアントライブラリ {#c-client-library} - -詳細は [clickhouse-cpp](https://github.com/ClickHouse/clickhouse-cpp) リポジトリの README を参照してください。 - -# Userver 非同期フレームワーク {#userver-asynchronous-framework} - -[userver (beta)](https://github.com/userver-framework/userver) は ClickHouse を組み込みでサポートしています。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats.md deleted file mode 100644 index 6cd9bbf893d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats.md +++ /dev/null @@ -1,171 +0,0 @@ ---- -description: 'ClickHouse における入力および出力でサポートされているデータ形式の概要' -sidebar_label: 'すべての形式を表示...' -sidebar_position: 21 -slug: /interfaces/formats -title: 'データ入出力形式' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - -# 入力および出力データのフォーマット {#formats-for-input-and-output-data} - -ClickHouse は、一般的なテキスト形式およびバイナリ形式のほとんどをサポートしています。これにより、運用中のほぼあらゆるデータパイプラインに容易に統合し、ClickHouse の利点を活用できます。 - - - -## 入力フォーマット {#input-formats} - -入力フォーマットは次の用途に使用されます: -- `INSERT` ステートメントに渡されたデータのパース -- `File`、`URL`、`HDFS` などのファイルをバックエンドに持つテーブルに対する `SELECT` クエリの実行 -- 辞書の読み取り - -ClickHouse にデータを効率的にインジェストするには、適切な入力フォーマットの選択が重要です。70 を超えるフォーマットがサポートされているため、どのフォーマットを選ぶかによって、挿入速度、CPU・メモリ使用量、およびシステム全体の効率が大きく変わります。これらの選択肢を検討しやすくするため、フォーマットごとにインジェスト性能をベンチマークし、次のような主なポイントが明らかになりました: - -- **[Native](formats/Native.md) フォーマットは最も効率的な入力フォーマットであり**、最高の圧縮率、最小のリソース使用量、およびサーバー側処理のオーバーヘッドの最小化を実現します。 -- **圧縮は不可欠です** - LZ4 は CPU コストをほとんど増やさずにデータサイズを削減し、ZSTD は追加の CPU 使用量と引き換えにより高い圧縮率を提供します。 -- **事前のソートの影響は中程度であり**、ClickHouse 自体がすでに効率的なソートを行います。 -- **バッチ処理は効率を大きく改善します** - バッチサイズを大きくすることで挿入時のオーバーヘッドが減り、スループットが向上します。 - -結果とベストプラクティスの詳細については、 -完全版の [ベンチマーク分析](https://www.clickhouse.com/blog/clickhouse-input-format-matchup-which-is-fastest-most-efficient) を参照してください。 -テスト結果の全体像は、[FastFormats](https://fastformats.clickhouse.com/) のオンラインダッシュボードで確認できます。 - - - -## 出力形式 {#output-formats} - -出力としてサポートされている形式は、次の用途に使用されます: -- `SELECT` クエリ結果の整形 -- ファイルベースのテーブルへの `INSERT` 操作の実行 - - - -## フォーマットの概要 {#formats-overview} - -サポートされているフォーマットは以下のとおりです。 - - - -| フォーマット | 入力 | 出力 | -| ---------------------------------------------------------------------------------------------------------- | -- | -- | -| [TabSeparated](./formats/TabSeparated/TabSeparated.md) | ✔ | ✔ | -| [TabSeparatedRaw](./formats/TabSeparated/TabSeparatedRaw.md) | ✔ | ✔ | -| [TabSeparatedWithNames](./formats/TabSeparated/TabSeparatedWithNames.md) | ✔ | ✔ | -| [TabSeparatedWithNamesAndTypes](./formats/TabSeparated/TabSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [TabSeparatedRawWithNames](./formats/TabSeparated/TabSeparatedRawWithNames.md) | ✔ | ✔ | -| [TabSeparatedRawWithNamesAndTypes](./formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md) | ✔ | ✔ | -| [Template](./formats/Template/Template.md) | ✔ | ✔ | -| [TemplateIgnoreSpaces](./formats/Template/TemplateIgnoreSpaces.md) | ✔ | ✗ | -| [CSV](./formats/CSV/CSV.md) | ✔ | ✔ | -| [CSVWithNames](./formats/CSV/CSVWithNames.md) | ✔ | ✔ | -| [CSVWithNamesAndTypes](./formats/CSV/CSVWithNamesAndTypes.md) | ✔ | ✔ | -| [CustomSeparated](./formats/CustomSeparated/CustomSeparated.md) | ✔ | ✔ | -| [CustomSeparatedWithNames](./formats/CustomSeparated/CustomSeparatedWithNames.md) | ✔ | ✔ | -| [CustomSeparatedWithNamesAndTypes](./formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [SQLInsert](./formats/SQLInsert.md) | ✗ | ✔ | -| [値](./formats/Values.md) | ✔ | ✔ | -| [Vertical](./formats/Vertical.md) | ✗ | ✔ | -| [JSON](./formats/JSON/JSON.md) | ✔ | ✔ | -| [JSONAsString](./formats/JSON/JSONAsString.md) | ✔ | ✗ | -| [JSONAsObject](./formats/JSON/JSONAsObject.md) | ✔ | ✗ | -| [JSONStrings](./formats/JSON/JSONStrings.md) | ✔ | ✔ | -| [JSONColumns](./formats/JSON/JSONColumns.md) | ✔ | ✔ | -| [JSONColumnsWithMetadata](./formats/JSON/JSONColumnsWithMetadata.md) | ✔ | ✔ | -| [JSONCompact](./formats/JSON/JSONCompact.md) | ✔ | ✔ | -| [JSONCompactStrings](./formats/JSON/JSONCompactStrings.md) | ✗ | ✔ | -| [JSONCompactColumns](./formats/JSON/JSONCompactColumns.md) | ✔ | ✔ | -| [JSONEachRow](./formats/JSON/JSONEachRow.md) | ✔ | ✔ | -| [PrettyJSONEachRow](./formats/JSON/PrettyJSONEachRow.md) | ✗ | ✔ | -| [JSONEachRowWithProgress](./formats/JSON/JSONEachRowWithProgress.md) | ✗ | ✔ | -| [JSONStringsEachRow](./formats/JSON/JSONStringsEachRow.md) | ✔ | ✔ | -| [JSONStringsEachRowWithProgress](./formats/JSON/JSONStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactEachRow](./formats/JSON/JSONCompactEachRow.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNames](./formats/JSON/JSONCompactEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNamesAndTypes](./formats/JSON/JSONCompactEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactEachRowWithProgress](./formats/JSON/JSONCompactEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactStringsEachRow](./formats/JSON/JSONCompactStringsEachRow.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNames](./formats/JSON/JSONCompactStringsEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNamesAndTypes](./formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithProgress](./formats/JSON/JSONCompactStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONObjectEachRow](./formats/JSON/JSONObjectEachRow.md) | ✔ | ✔ | -| [BSONEachRow](./formats/BSONEachRow.md) | ✔ | ✔ | -| [TSKV](./formats/TabSeparated/TSKV.md) | ✔ | ✔ | -| [Pretty](./formats/Pretty/Pretty.md) | ✗ | ✔ | -| [PrettyNoEscapes](./formats/Pretty/PrettyNoEscapes.md) | ✗ | ✔ | -| [PrettyMonoBlock](./formats/Pretty/PrettyMonoBlock.md) | ✗ | ✔ | -| [PrettyNoEscapesMonoBlock](./formats/Pretty/PrettyNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettyCompact](./formats/Pretty/PrettyCompact.md) | ✗ | ✔ | -| [PrettyCompactNoEscapes](./formats/Pretty/PrettyCompactNoEscapes.md) | ✗ | ✔ | -| [PrettyCompactMonoBlock](./formats/Pretty/PrettyCompactMonoBlock.md) | ✗ | ✔ | -| [PrettyCompactNoEscapesMonoBlock](./formats/Pretty/PrettyCompactNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettySpace](./formats/Pretty/PrettySpace.md) | ✗ | ✔ | -| [PrettySpaceNoEscapes](./formats/Pretty/PrettySpaceNoEscapes.md) | ✗ | ✔ | -| [PrettySpaceMonoBlock](./formats/Pretty/PrettySpaceMonoBlock.md) | ✗ | ✔ | -| [PrettySpaceNoEscapesMonoBlock](./formats/Pretty/PrettySpaceNoEscapesMonoBlock.md) | ✗ | ✔ | -| [Prometheus](./formats/Prometheus.md) | ✗ | ✔ | -| [Protobuf](./formats/Protobuf/Protobuf.md) | ✔ | ✔ | -| [ProtobufSingle](./formats/Protobuf/ProtobufSingle.md) | ✔ | ✔ | -| [ProtobufList](./formats/Protobuf/ProtobufList.md) | ✔ | ✔ | -| [Avro](./formats/Avro/Avro.md) | ✔ | ✔ | -| [AvroConfluent](./formats/Avro/AvroConfluent.md) | ✔ | ✗ | -| [Parquet](./formats/Parquet/Parquet.md) | ✔ | ✔ | -| [ParquetMetadata](./formats/Parquet/ParquetMetadata.md) | ✔ | ✗ | -| [Arrow](./formats/Arrow/Arrow.md) | ✔ | ✔ | -| [ArrowStream](./formats/Arrow/ArrowStream.md) | ✔ | ✔ | -| [ORC](./formats/ORC.md) | ✔ | ✔ | -| [One](./formats/One.md) | ✔ | ✗ | -| [Npy](./formats/Npy.md) | ✔ | ✔ | -| [RowBinary](./formats/RowBinary/RowBinary.md) | ✔ | ✔ | -| [RowBinaryWithNames](./formats/RowBinary/RowBinaryWithNames.md) | ✔ | ✔ | -| [RowBinaryWithNamesAndTypes](./formats/RowBinary/RowBinaryWithNamesAndTypes.md) | ✔ | ✔ | -| [RowBinaryWithDefaults](./formats/RowBinary/RowBinaryWithDefaults.md) | ✔ | ✗ | -| [Native](./formats/Native.md) | ✔ | ✔ | -| [Null](./formats/Null.md) | ✗ | ✔ | -| [Hash](./formats/Hash.md) | ✗ | ✔ | -| [XML](./formats/XML.md) | ✗ | ✔ | -| [CapnProto](./formats/CapnProto.md) | ✔ | ✔ | -| [LineAsString](./formats/LineAsString/LineAsString.md) | ✔ | ✔ | -| [LineAsStringWithNames](./formats/LineAsString/LineAsStringWithNames.md) | ✔ | ✔ | -| [LineAsStringWithNamesAndTypes](./formats/LineAsString/LineAsStringWithNamesAndTypes.md) | ✔ | ✔ | -| [正規表現(Regexp)](./formats/Regexp.md) | ✔ | ✗ | -| [RawBLOB](./formats/RawBLOB.md) | ✔ | ✔ | -| [MsgPack](./formats/MsgPack.md) | ✔ | ✔ | -| [MySQLDump](./formats/MySQLDump.md) | ✔ | ✗ | -| [DWARF](./formats/DWARF.md) | ✔ | ✗ | -| [Markdown](./formats/Markdown.md) | ✗ | ✔ | -| [フォーム](./formats/Form.md) | ✔ | ✗ | - - - -ClickHouse の設定を使用して、一部のフォーマット処理パラメータを制御できます。詳細については、[Settings](/operations/settings/settings-formats.md) セクションを参照してください。 - - - -## フォーマットスキーマ {#formatschema} - -フォーマットスキーマを格納したファイル名は、設定 `format_schema` で指定します。 -この設定は、`Cap'n Proto` または `Protobuf` のいずれかのフォーマットを使用する場合に必須です。 -フォーマットスキーマは、コロンで区切られた「ファイル名」と、そのファイル内のメッセージ型の名前の組み合わせです。 -例: `schemafile.proto:MessageType`。 -ファイルがそのフォーマットの標準拡張子(たとえば `Protobuf` の `.proto`)を持つ場合は、拡張子を省略でき、その場合フォーマットスキーマは `schemafile:MessageType` のようになります。 - -[クライアント](/interfaces/cli.md)を対話モードで使用してデータを入力または出力する場合、フォーマットスキーマに指定するファイル名には、 -クライアント側のカレントディレクトリからの相対パス、または絶対パスを指定できます。 -クライアントを[バッチモード](/interfaces/cli.md/#batch-mode)で使用する場合、セキュリティ上の理由から、スキーマへのパスは相対パスでなければなりません。 - -[HTTP インターフェイス](/interfaces/http.md)経由でデータを入力または出力する場合、フォーマットスキーマで指定するファイル名は、 -サーバー設定の [format_schema_path](/operations/server-configuration-parameters/settings.md/#format_schema_path) で指定されたディレクトリ内に存在している必要があります。 - - - -## エラーのスキップ {#skippingerrors} - -`CSV`、`TabSeparated`、`TSKV`、`JSONEachRow`、`Template`、`CustomSeparated`、`Protobuf` などの一部の形式では、パースエラーが発生した場合に不正な行をスキップし、次の行の先頭からパースを継続できます。詳細は [input_format_allow_errors_num](/operations/settings/settings-formats.md/#input_format_allow_errors_num) および -[input_format_allow_errors_ratio](/operations/settings/settings-formats.md/#input_format_allow_errors_ratio) 設定を参照してください。 -制限事項: -- パースエラーが発生した場合、`JSONEachRow` は改行(または EOF)までのすべてのデータをスキップするため、エラーを正しくカウントするには、行を `\n` で区切る必要があります。 -- `Template` と `CustomSeparated` は、次の行の先頭を見つけるために、最後の列の後の区切り文字と行間の区切り文字を使用するため、少なくとも一方が空でない場合にのみエラーをスキップできます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md deleted file mode 100644 index 67ed5a8c56a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Arrow 形式に関するドキュメント' -input_format: true -keywords: ['Arrow'] -output_format: true -slug: /interfaces/formats/Arrow -title: 'Arrow' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[Apache Arrow](https://arrow.apache.org/) には、組み込みのカラムナ型ストレージフォーマットが 2 つ用意されています。ClickHouse はこれらのフォーマットに対する読み書き処理をサポートしています。 -`Arrow` は Apache Arrow の「ファイルモード」フォーマットです。メモリ上でのランダムアクセス向けに設計されています。 - -## データ型の対応関係 {#data-types-matching} - -次の表は、サポートされているデータ型と、それらが `INSERT` および `SELECT` クエリで ClickHouse の [データ型](/sql-reference/data-types/index.md) にどのように対応するかを示しています。 - -| Arrow data type (`INSERT`) | ClickHouse data type | Arrow data type (`SELECT`) | -|-----------------------------------------|------------------------------------------------------------------------------------------------------------|----------------------------| -| `BOOL` | [Bool](/sql-reference/data-types/boolean.md) | `BOOL` | -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md) | `INT64` | -| `FLOAT`, `HALF_FLOAT` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `DATE32` | [Date32](/sql-reference/data-types/date32.md) | `UINT16` | -| `DATE64` | [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](/sql-reference/data-types/datetime64.md) | `TIMESTAMP` | -| `STRING`, `BINARY` | [String](/sql-reference/data-types/string.md) | `BINARY` | -| `STRING`, `BINARY`, `FIXED_SIZE_BINARY` | [FixedString](/sql-reference/data-types/fixedstring.md) | `FIXED_SIZE_BINARY` | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | `DECIMAL` | -| `DECIMAL256` | [Decimal256](/sql-reference/data-types/decimal.md) | `DECIMAL256` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `MAP` | [Map](/sql-reference/data-types/map.md) | `MAP` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `FIXED_SIZE_BINARY`, `BINARY` | [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_SIZE_BINARY` | -| `FIXED_SIZE_BINARY`, `BINARY` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_SIZE_BINARY` | - -配列は入れ子(ネスト)にすることができ、引数として `Nullable` 型の値を持つことができます。`Tuple` 型および `Map` 型も入れ子にすることができます。 - -`DICTIONARY` 型は `INSERT` クエリでサポートされており、`SELECT` クエリに対しては、[LowCardinality](/sql-reference/data-types/lowcardinality.md) 型を `DICTIONARY` 型として出力できるようにする [`output_format_arrow_low_cardinality_as_dictionary`](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary) 設定があります。 - -サポートされていない Arrow データ型は次のとおりです: - -- `FIXED_SIZE_BINARY` -- `JSON` -- `UUID` -- `ENUM`. - -ClickHouse テーブル列のデータ型は、対応する Arrow データフィールドと一致している必要はありません。データを挿入する際、ClickHouse はまず上記の表に従ってデータ型を解釈し、その後、ClickHouse テーブル列に設定されているデータ型へデータを[キャスト](/sql-reference/functions/type-conversion-functions#cast)します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のコマンドを使用して、ファイルから ClickHouse テーブルに Arrow 形式のデータを挿入できます。 - -```bash -$ cat filename.arrow | clickhouse-client --query="INSERT INTO some_table FORMAT Arrow" -``` - - -### データの選択 {#selecting-data} - -次のコマンドを使用して、ClickHouse のテーブルからデータを抽出し、Arrow 形式のファイルに保存できます。 - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Arrow" > {filename.arrow} -``` - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | デフォルト | -|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|--------------| -| `input_format_arrow_allow_missing_columns` | Arrow 入力フォーマットの読み取り時に列の欠落を許可する | `1` | -| `input_format_arrow_case_insensitive_column_matching` | Arrow の列と CH の列を照合する際に大文字小文字を区別しない | `0` | -| `input_format_arrow_import_nested` | 廃止された設定で、何もしない | `0` | -| `input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference` | Arrow フォーマットのスキーマ推論時に、未対応の型を持つ列をスキップする | `0` | -| `output_format_arrow_compression_method` | Arrow 出力フォーマットの圧縮方式。サポートされるコーデック: lz4_frame, zstd, none (非圧縮) | `lz4_frame` | -| `output_format_arrow_fixed_string_as_fixed_byte_array` | FixedString 列に対して Binary 型の代わりに Arrow の FIXED_SIZE_BINARY 型を使用する | `1` | -| `output_format_arrow_low_cardinality_as_dictionary` | LowCardinality 型を Dictionary Arrow 型として出力することを有効にする | `0` | -| `output_format_arrow_string_as_string` | String 列に対して Binary 型の代わりに Arrow の String 型を使用する | `1` | -| `output_format_arrow_use_64_bit_indexes_for_dictionary` | Arrow フォーマットのディクショナリインデックスに常に 64 ビット整数を使用する | `0` | -| `output_format_arrow_use_signed_indexes_for_dictionary` | Arrow フォーマットのディクショナリインデックスに符号付き整数を使用する | `1` | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md deleted file mode 100644 index 6c405376ef7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -alias: [] -description: 'ArrowStream フォーマットのドキュメント' -input_format: true -keywords: ['ArrowStream'] -output_format: true -slug: /interfaces/formats/ArrowStream -title: 'ArrowStream' -doc_type: 'reference' ---- - -| Input | Output | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -`ArrowStream` は Apache Arrow の「ストリームモード」形式です。インメモリストリーム処理向けに設計されています。 - -## 使用例 {#example-usage} - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md deleted file mode 100644 index 18add860eda..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Avro 形式に関するドキュメント' -input_format: true -keywords: ['Avro'] -output_format: true -slug: /interfaces/formats/Avro -title: 'Avro' -doc_type: 'reference' ---- - -import DataTypeMapping from './_snippets/data-types-matching.md' - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✔ | | - - -## 説明 {#description} - -[Apache Avro](https://avro.apache.org/) は、効率的なデータ処理のためにバイナリエンコーディングを使用する行指向のシリアル化フォーマットです。`Avro` フォーマットは、[Avro データファイル](https://avro.apache.org/docs/++version++/specification/#object-container-files) の読み書きをサポートします。このフォーマットは、スキーマを埋め込んだ自己記述型のメッセージを前提としています。Avro をスキーマレジストリと併用している場合は、[`AvroConfluent`](./AvroConfluent.md) フォーマットを参照してください。 - -## データ型マッピング {#data-type-mapping} - - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | デフォルト | -|---------------------------------------------|-----------------------------------------------------------------------------------------------------|-----------| -| `input_format_avro_allow_missing_fields` | スキーマ内にフィールドが存在しない場合にエラーとする代わりにデフォルト値を使用するかどうか。 | `0` | -| `input_format_avro_null_as_default` | NULL 不可の列に `null` 値を挿入する際にエラーとする代わりにデフォルト値を使用するかどうか。 | `0` | -| `output_format_avro_codec` | Avro 出力ファイルの圧縮アルゴリズム。指定可能な値: `null`, `deflate`, `snappy`, `zstd`. | | -| `output_format_avro_sync_interval` | Avro ファイルにおける同期マーカーの頻度(バイト単位)。 | `16384` | -| `output_format_avro_string_column_pattern` | Avro の `string` 型にマッピングする対象となる `String` 列を識別するための正規表現。デフォルトでは、ClickHouse の `String` 列は Avro の `bytes` 型として書き出される。 | | -| `output_format_avro_rows_in_file` | 1 つの Avro 出力ファイルあたりの最大行数。この上限に達すると、新しいファイルが作成される(ストレージシステムがファイル分割をサポートしている場合)。 | `1` | - -## 例 {#examples} - -### Avro データの読み取り {#reading-avro-data} - -Avro ファイルから ClickHouse テーブルにデータを読み込むには、次のとおりです。 - -```bash -$ cat file.avro | clickhouse-client --query="INSERT INTO {some_table} FORMAT Avro" -``` - -取り込まれた Avro ファイルのルートスキーマは、`record` 型でなければなりません。 - -テーブルのカラムと Avro スキーマのフィールドを対応付けるために、ClickHouse はそれらの名前を比較します。 -この比較は大文字と小文字を区別し、未使用のフィールドはスキップされます。 - -ClickHouse テーブルのカラムのデータ型は、挿入される Avro データの対応するフィールドと異なる場合があります。データを挿入する際、ClickHouse は上記のテーブルに従ってデータ型を解釈し、その後データを対応するカラム型に[キャスト](/sql-reference/functions/type-conversion-functions#cast)します。 - -データをインポートする際、スキーマ内でフィールドが見つからず、設定 [`input_format_avro_allow_missing_fields`](/operations/settings/settings-formats.md/#input_format_avro_allow_missing_fields) が有効になっている場合は、エラーを発生させる代わりにデフォルト値が使用されます。 - - -### Avro データの書き込み {#writing-avro-data} - -ClickHouse テーブルのデータを Avro ファイルに書き出すには、次のようにします。 - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Avro" > file.avro -``` - -列名は次を満たす必要があります: - -* `[A-Za-z_]` で始まること -* 続く文字は `[A-Za-z0-9_]` のみであること - -Avro ファイルの出力圧縮と同期間隔は、それぞれ [`output_format_avro_codec`](/operations/settings/settings-formats.md/#output_format_avro_codec) および [`output_format_avro_sync_interval`](/operations/settings/settings-formats.md/#output_format_avro_sync_interval) 設定を使用して構成できます。 - - -### Avro スキーマの推論 {#inferring-the-avro-schema} - -ClickHouse の [`DESCRIBE`](/sql-reference/statements/describe-table) 関数を使用すると、次の例のように Avro ファイルの推論されたスキーマをすばやく確認できます。 -この例には、ClickHouse の S3 パブリックバケット内にある、公開アクセス可能な Avro ファイルの URL が含まれています。 - -```sql -DESCRIBE url('https://clickhouse-public-datasets.s3.eu-central-1.amazonaws.com/hits.avro','Avro); - -┌─name───────────────────────┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ WatchID │ Int64 │ │ │ │ │ │ -│ JavaEnable │ Int32 │ │ │ │ │ │ -│ Title │ String │ │ │ │ │ │ -│ GoodEvent │ Int32 │ │ │ │ │ │ -│ EventTime │ Int32 │ │ │ │ │ │ -│ EventDate │ Date32 │ │ │ │ │ │ -│ CounterID │ Int32 │ │ │ │ │ │ -│ ClientIP │ Int32 │ │ │ │ │ │ -│ ClientIP6 │ FixedString(16) │ │ │ │ │ │ -│ RegionID │ Int32 │ │ │ │ │ │ -... -│ IslandID │ FixedString(16) │ │ │ │ │ │ -│ RequestNum │ Int32 │ │ │ │ │ │ -│ RequestTry │ Int32 │ │ │ │ │ │ -└────────────────────────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md deleted file mode 100644 index cb2a879932a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -alias: [] -description: 'AvroConfluent フォーマットに関するドキュメント' -input_format: true -keywords: ['AvroConfluent'] -output_format: false -slug: /interfaces/formats/AvroConfluent -title: 'AvroConfluent' -doc_type: 'reference' ---- - -import DataTypesMatching from './_snippets/data-types-matching.md' - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✗ | | - - -## 説明 {#description} - -[Apache Avro](https://avro.apache.org/) は、効率的なデータ処理のためにバイナリエンコードを使用する行指向のシリアル化フォーマットです。`AvroConfluent` フォーマットは、[Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/index.html)(またはその API 互換サービス)を用いてシリアル化された、単一オブジェクト形式の Avro でエンコードされた Kafka メッセージのデコードをサポートします。 - -各 Avro メッセージにはスキーマ ID が埋め込まれており、ClickHouse は設定済みの Schema Registry に対してクエリを実行することで自動的にスキーマを解決します。一度解決されたスキーマは、パフォーマンスを最適化するためにキャッシュされます。 - - - -## データ型のマッピング {#data-type-mapping} - - - -## フォーマット設定 {#format-settings} - -[//]: # "NOTE これらの設定はセッション・レベルでも設定できますが、それが行われることはあまりなく、あまり目立つ形で文書化するとユーザーを混乱させる可能性があります。" - -| Setting | Description | Default | -|---------------------------------------------|-----------------------------------------------------------------------------------------------------|---------| -| `input_format_avro_allow_missing_fields` | スキーマ内にフィールドが見つからない場合にエラーを発生させる代わりに、デフォルト値を使用するかどうか。 | `0` | -| `input_format_avro_null_as_default` | NULL を許容しないカラムに `null` 値を挿入する際にエラーを発生させる代わりに、デフォルト値を使用するかどうか。 | `0` | -| `format_avro_schema_registry_url` | Confluent Schema Registry の URL。Basic 認証を利用する場合は、URL エンコードした認証情報を URL のパス部分に直接含めることができます。 | | - -## 例 {#examples} - -### スキーマレジストリの使用 {#using-a-schema-registry} - -[Kafka table engine](/engines/table-engines/integrations/kafka.md) を使用して Avro でエンコードされた Kafka トピックを読み取るには、`format_avro_schema_registry_url` 設定を使用してスキーマレジストリの URL を指定します。 - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'http://schema-registry-url'; - -SELECT * FROM topic1_stream; -``` - - -#### Basic 認証の使用 {#using-basic-authentication} - -スキーマレジストリが Basic 認証を必要とする場合(例:Confluent Cloud を使用している場合)、`format_avro_schema_registry_url` 設定に URL エンコード済みの認証情報を指定できます。 - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'https://:@schema-registry-url'; -``` - - -## トラブルシューティング {#troubleshooting} - -インジェスト処理の進行状況を監視し、Kafka コンシューマーで発生するエラーをデバッグするには、[`system.kafka_consumers` システムテーブル](../../../operations/system-tables/kafka_consumers.md)に対してクエリを実行できます。デプロイメントに複数のレプリカがある場合(例:ClickHouse Cloud)、[`clusterAllReplicas`](../../../sql-reference/table-functions/cluster.md) テーブル関数を使用する必要があります。 - -```sql -SELECT * FROM clusterAllReplicas('default',system.kafka_consumers) -ORDER BY assignments.partition_id ASC; -``` - -スキーマの解決で問題が発生した場合は、[kafkacat](https://github.com/edenhill/kafkacat) と [clickhouse-local](/operations/utilities/clickhouse-local.md) を使用してトラブルシューティングできます。 - -```bash -$ kafkacat -b kafka-broker -C -t topic1 -o beginning -f '%s' -c 3 | clickhouse-local --input-format AvroConfluent --format_avro_schema_registry_url 'http://schema-registry' -S "field1 Int64, field2 String" -q 'select * from table' -1 a -2 b -3 c -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md deleted file mode 100644 index 7ff21914cd5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md +++ /dev/null @@ -1,40 +0,0 @@ -次の表は、Apache Avro 形式でサポートされているすべてのデータ型と、`INSERT` クエリおよび `SELECT` クエリにおける対応する ClickHouse の[データ型](/sql-reference/data-types/index.md)を示しています。 - -| Avro データ型 `INSERT` | ClickHouse データ型 | Avro データ型 `SELECT` | -|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------------------| -| `boolean`, `int`, `long`, `float`, `double` | [Int(8\16\32)](/sql-reference/data-types/int-uint.md), [UInt(8\16\32)](/sql-reference/data-types/int-uint.md) | `int` | -| `boolean`, `int`, `long`, `float`, `double` | [Int64](/sql-reference/data-types/int-uint.md), [UInt64](/sql-reference/data-types/int-uint.md) | `long` | -| `boolean`, `int`, `long`, `float`, `double` | [Float32](/sql-reference/data-types/float.md) | `float` | -| `boolean`, `int`, `long`, `float`, `double` | [Float64](/sql-reference/data-types/float.md) | `double` | -| `bytes`, `string`, `fixed`, `enum` | [String](/sql-reference/data-types/string.md) | `bytes` または `string` \* | -| `bytes`, `string`, `fixed` | [FixedString(N)](/sql-reference/data-types/fixedstring.md) | `fixed(N)` | -| `enum` | [Enum(8\16)](/sql-reference/data-types/enum.md) | `enum` | -| `array(T)` | [Array(T)](/sql-reference/data-types/array.md) | `array(T)` | -| `map(V, K)` | [Map(V, K)](/sql-reference/data-types/map.md) | `map(string, K)` | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(null, T)` | -| `union(T1, T2, …)` \** | [Variant(T1, T2, …)](/sql-reference/data-types/variant.md) | `union(T1, T2, …)` \** | -| `null` | [Nullable(Nothing)](/sql-reference/data-types/special-data-types/nothing.md) | `null` | -| `int (date)` \**\* | [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md) | `int (date)` \**\* | -| `long (timestamp-millis)` \**\* | [DateTime64(3)](/sql-reference/data-types/datetime.md) | `long (timestamp-millis)` \**\* | -| `long (timestamp-micros)` \**\* | [DateTime64(6)](/sql-reference/data-types/datetime.md) | `long (timestamp-micros)` \**\* | -| `bytes (decimal)` \**\* | [DateTime64(N)](/sql-reference/data-types/datetime.md) | `bytes (decimal)` \**\* | -| `int` | [IPv4](/sql-reference/data-types/ipv4.md) | `int` | -| `fixed(16)` | [IPv6](/sql-reference/data-types/ipv6.md) | `fixed(16)` | -| `bytes (decimal)` \**\* | [Decimal(P, S)](/sql-reference/data-types/decimal.md) | `bytes (decimal)` \**\* | -| `string (uuid)` \**\* | [UUID](/sql-reference/data-types/uuid.md) | `string (uuid)` \**\* | -| `fixed(16)` | [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `fixed(16)` | -| `fixed(32)` | [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `fixed(32)` | -| `record` | [Tuple](/sql-reference/data-types/tuple.md) | `record` | - -\* `bytes` がデフォルトであり、この挙動は設定 [`output_format_avro_string_column_pattern`](/operations/settings/settings-formats.md/#output_format_avro_string_column_pattern) によって制御されます - -\** [Variant type](/sql-reference/data-types/variant) はフィールド値として暗黙的に `null` を受け入れるため、たとえば Avro の `union(T1, T2, null)` は `Variant(T1, T2)` に変換されます。 -その結果、ClickHouse から Avro を生成する際には、スキーマ推論中に任意の値が実際に `null` かどうかを判断できないため、常に Avro の `union` 型の集合に `null` 型を含める必要があります。 - -\**\* [Avro logical types](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -サポートされていない Avro の論理データ型: - -- `time-millis` -- `time-micros` -- `duration` \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md deleted file mode 100644 index bac4c7cc95b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -alias: [] -description: 'BSONEachRow 形式についてのドキュメント' -input_format: true -keywords: ['BSONEachRow'] -output_format: true -slug: /interfaces/formats/BSONEachRow -title: 'BSONEachRow' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -`BSONEachRow` フォーマットは、データを区切りなしの連続した Binary JSON (BSON) ドキュメントのシーケンスとして解釈します。 -各行は 1 つのドキュメントとして表現され、各列は列名をキーとする 1 つの BSON ドキュメントフィールドとして表現されます。 - -## データ型の対応 {#data-types-matching} - -出力時には、ClickHouse の型と BSON の型の間で次の対応を使用します。 - -| ClickHouse type | BSON Type | -|-----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| -| [Bool](/sql-reference/data-types/boolean.md) | `\x08` boolean | -| [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int32](/sql-reference/data-types/int-uint.md) | `\x10` int32 | -| [UInt32](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Int64/UInt64](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Float32/Float64](/sql-reference/data-types/float.md) | `\x01` double | -| [Date](/sql-reference/data-types/date.md)/[Date32](/sql-reference/data-types/date32.md) | `\x10` int32 | -| [DateTime](/sql-reference/data-types/datetime.md) | `\x12` int64 | -| [DateTime64](/sql-reference/data-types/datetime64.md) | `\x09` datetime | -| [Decimal32](/sql-reference/data-types/decimal.md) | `\x10` int32 | -| [Decimal64](/sql-reference/data-types/decimal.md) | `\x12` int64 | -| [Decimal128](/sql-reference/data-types/decimal.md) | `\x05` binary、`\x00` binary サブタイプ、サイズ = 16 | -| [Decimal256](/sql-reference/data-types/decimal.md) | `\x05` binary、`\x00` binary サブタイプ、サイズ = 32 | -| [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `\x05` binary、`\x00` binary サブタイプ、サイズ = 16 | -| [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `\x05` binary、`\x00` binary サブタイプ、サイズ = 32 | -| [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | `\x05` binary、`\x00` binary サブタイプ、または設定 output_format_bson_string_as_string が有効な場合は `\x02` string | -| [UUID](/sql-reference/data-types/uuid.md) | `\x05` binary、`\x04` uuid サブタイプ、サイズ = 16 | -| [Array](/sql-reference/data-types/array.md) | `\x04` array | -| [Tuple](/sql-reference/data-types/tuple.md) | `\x04` array | -| [Named Tuple](/sql-reference/data-types/tuple.md) | `\x03` document | -| [Map](/sql-reference/data-types/map.md) | `\x03` document | -| [IPv4](/sql-reference/data-types/ipv4.md) | `\x10` int32 | -| [IPv6](/sql-reference/data-types/ipv6.md) | `\x05` binary、`\x00` binary サブタイプ | - -入力時には、BSON の型と ClickHouse の型の間で次の対応を使用します。 - -| BSON Type | ClickHouse Type | -|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `\x01` double | [Float32/Float64](/sql-reference/data-types/float.md) | -| `\x02` string | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x03` document | [Map](/sql-reference/data-types/map.md)/[Named Tuple](/sql-reference/data-types/tuple.md) | -| `\x04` array | [Array](/sql-reference/data-types/array.md)/[Tuple](/sql-reference/data-types/tuple.md) | -| `\x05` binary, `\x00` binary subtype | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md)/[IPv6](/sql-reference/data-types/ipv6.md) | -| `\x05` binary, `\x02` old binary subtype | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x05` binary, `\x03` old uuid subtype | [UUID](/sql-reference/data-types/uuid.md) | -| `\x05` binary, `\x04` uuid subtype | [UUID](/sql-reference/data-types/uuid.md) | -| `\x07` ObjectId | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x08` boolean | [Bool](/sql-reference/data-types/boolean.md) | -| `\x09` datetime | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `\x0A` null value | [NULL](/sql-reference/data-types/nullable.md) | -| `\x0D` JavaScript code | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x0E` symbol | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x10` int32 | [Int32/UInt32](/sql-reference/data-types/int-uint.md)/[Decimal32](/sql-reference/data-types/decimal.md)/[IPv4](/sql-reference/data-types/ipv4.md)/[Enum8/Enum16](/sql-reference/data-types/enum.md) | -| `\x12` int64 | [Int64/UInt64](/sql-reference/data-types/int-uint.md)/[Decimal64](/sql-reference/data-types/decimal.md)/[DateTime64](/sql-reference/data-types/datetime64.md) | - -その他の BSON 型はサポートされていません。さらに、異なる整数型どうしの変換も行われます。 -たとえば、BSON の `int32` 値を ClickHouse の [`UInt8`](../../sql-reference/data-types/int-uint.md) として挿入することが可能です。 - -`Int128`/`UInt128`/`Int256`/`UInt256`/`Decimal128`/`Decimal256` のような大きな整数および小数値は、バイナリサブタイプが `\x00` の BSON Binary 値から解析できます。 -この場合、この形式はバイナリデータのサイズが期待される値のサイズと等しいことを検証します。 - -:::note -この形式はビッグエンディアンプラットフォームでは正しく動作しません。 -::: - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む、`football.bson` という名前の BSON ファイルを使用します。 - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -データの挿入: - -```sql -INSERT INTO football FROM INFILE 'football.bson' FORMAT BSONEachRow; -``` - - -### データの読み込み {#reading-data} - -`BSONEachRow` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football INTO OUTFILE 'docs_data/bson/football.bson' -FORMAT BSONEachRow -``` - -:::tip -BSON はバイナリ形式のデータであり、ターミナル上では人間が読める形では表示されません。`INTO OUTFILE` を使用して BSON ファイルとして出力してください。 -::: - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | 既定値 | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------|----------| -| [`output_format_bson_string_as_string`](../../operations/settings/settings-formats.md/#output_format_bson_string_as_string) | String 列に対しては、Binary 型ではなく BSON の String 型を使用します。 | `false` | -| [`input_format_bson_skip_fields_with_unsupported_types_in_schema_inference`](../../operations/settings/settings-formats.md/#input_format_bson_skip_fields_with_unsupported_types_in_schema_inference) | BSONEachRow フォーマットのスキーマ推論時に、サポートされていない型を持つ列をスキップすることを許可します。 | `false` | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md deleted file mode 100644 index 298df0a5995..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -alias: [] -description: 'CSV 形式に関するドキュメント' -input_format: true -keywords: ['CSV'] -output_format: true -slug: /interfaces/formats/CSV -title: 'CSV' -doc_type: 'reference' ---- - -## 説明 {#description} - -コンマ区切り値(Comma Separated Values)形式([RFC](https://tools.ietf.org/html/rfc4180))。 -フォーマット時、行は二重引用符で囲まれます。文字列中の二重引用符は、連続した 2 つの二重引用符として出力されます。 -文字のエスケープに関するその他のルールはありません。 - -* 日付および日時は二重引用符で囲まれます。 -* 数値は引用符なしで出力されます。 -* 値は区切り文字で区切られ、デフォルトでは `,` が使用されます。区切り文字は設定 [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) で定義されます。 -* 行は Unix の改行文字(LF)で区切られます。 -* 配列は CSV では次のようにシリアライズされます: - * まず、配列を TabSeparated 形式と同様に文字列へシリアライズします。 - * 得られた文字列を CSV では二重引用符で囲んで出力します。 -* CSV 形式におけるタプルは、個別のカラムとしてシリアライズされます(つまり、タプル内でのネスト情報は失われます)。 - -```bash -$ clickhouse-client --format_csv_delimiter="|" --query="INSERT INTO test.csv FORMAT CSV" < data.csv -``` - -:::note -デフォルトでは、区切り文字は `,` です。 -詳細は、設定 [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) を参照してください。 -::: - -パース時、すべての値はクォートあり/なしのどちらでも解釈できます。ダブルクォートとシングルクォートの両方がサポートされています。 - -行はクォートなしで記述することもできます。この場合、区切り文字または改行文字(CR または LF)までを 1 つの値としてパースします。 -ただし、RFC に反しますが、クォートなしの行をパースする場合は、先頭および末尾のスペースとタブは無視されます。 -改行コードとしては、Unix (LF)、Windows (CR LF)、Mac OS Classic (CR LF) がサポートされています。 - -`NULL` は設定 [format_csv_null_representation](/operations/settings/settings-formats.md/#format_csv_null_representation) に従ってフォーマットされます(デフォルト値は `\N`)。 - -入力データ内の `ENUM` 値は、名前または ID として表現できます。 -最初に、入力値を ENUM 名と照合しようとします。 -照合に失敗し、かつ入力値が数値である場合は、この数値を ENUM ID と照合しようとします。 -入力データに ENUM ID のみが含まれている場合は、`ENUM` のパースを最適化するために、設定 [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) を有効にすることを推奨します。 - - -## 使用例 {#example-usage} - -## フォーマット設定 {#format-settings} - -| Setting | 説明 | デフォルト | 備考 | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) | CSV データで区切り文字として扱う文字。 | `,` | | -| [format_csv_allow_single_quotes](/operations/settings/settings-formats.md/#format_csv_allow_single_quotes) | 文字列で単一引用符(シングルクォート)の使用を許可する。 | `true` | | -| [format_csv_allow_double_quotes](/operations/settings/settings-formats.md/#format_csv_allow_double_quotes) | 文字列で二重引用符(ダブルクォート)の使用を許可する。 | `true` | | -| [format_csv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation) | CSV フォーマットにおける NULL のカスタム表現。 | `\N` | | -| [input_format_csv_empty_as_default](/operations/settings/settings-formats.md/#input_format_csv_empty_as_default) | CSV 入力で空フィールドをデフォルト値として扱う。 | `true` | 複雑なデフォルト式を使用する場合は、[input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) も有効にする必要がある。 | -| [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) | CSV フォーマットで挿入された Enum 値を Enum のインデックスとして扱う。 | `false` | | -| [input_format_csv_use_best_effort_in_schema_inference](/operations/settings/settings-formats.md/#input_format_csv_use_best_effort_in_schema_inference) | CSV フォーマットでスキーマ推論を行う際に、各種調整とヒューリスティックを用いて推論精度を高める。無効な場合、すべてのフィールドは String として推論される。 | `true` | | -| [input_format_csv_arrays_as_nested_csv](/operations/settings/settings-formats.md/#input_format_csv_arrays_as_nested_csv) | CSV から Array を読み取る際、その要素がネストされた CSV としてシリアライズされ、その結果が文字列に格納されていることを前提とする。 | `false` | | -| [output_format_csv_crlf_end_of_line](/operations/settings/settings-formats.md/#output_format_csv_crlf_end_of_line) | `true` に設定すると、CSV 出力の行末は `\n` ではなく `\r\n` になる。 | `false` | | -| [input_format_csv_skip_first_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_first_lines) | データの先頭から指定した行数をスキップする。 | `0` | | -| [input_format_csv_detect_header](/operations/settings/settings-formats.md/#input_format_csv_detect_header) | CSV フォーマットで、名前および型を含むヘッダー行を自動検出する。 | `true` | | -| [input_format_csv_skip_trailing_empty_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_trailing_empty_lines) | データ末尾の空行をスキップする。 | `false` | | -| [input_format_csv_trim_whitespaces](/operations/settings/settings-formats.md/#input_format_csv_trim_whitespaces) | 引用符で囲まれていない CSV 文字列内の空白およびタブをトリムする。 | `true` | | -| [input_format_csv_allow_whitespace_or_tab_as_delimiter](/operations/settings/settings-formats.md/#input_format_csv_allow_whitespace_or_tab_as_delimiter) | CSV 文字列でフィールド区切りとして空白またはタブを使用できるようにする。 | `false` | | -| [input_format_csv_allow_variable_number_of_columns](/operations/settings/settings-formats.md/#input_format_csv_allow_variable_number_of_columns) | CSV フォーマットで可変数のカラム数を許可し、余分なカラムは無視し、欠損しているカラムにはデフォルト値を使用する。 | `false` | | -| [input_format_csv_use_default_on_bad_values](/operations/settings/settings-formats.md/#input_format_csv_use_default_on_bad_values) | CSV フィールドのデシリアライズが不正な値で失敗した場合に、そのカラムにデフォルト値を設定できるようにする。 | `false` | | -| [input_format_csv_try_infer_numbers_from_strings](/operations/settings/settings-formats.md/#input_format_csv_try_infer_numbers_from_strings) | スキーマ推論時に、文字列フィールドから数値型を推論しようとする。 | `false` | | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md deleted file mode 100644 index 02278c79c52..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -alias: [] -description: 'CSV 形式のドキュメント' -input_format: true -keywords: ['CSVWithNames'] -output_format: true -slug: /interfaces/formats/CSVWithNames -title: 'CSVWithNames' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[TabSeparatedWithNames](/interfaces/formats/TabSeparatedWithNames) と同様に、列名を含むヘッダー行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -:::tip -[バージョン](https://github.com/ClickHouse/ClickHouse/releases) 23.1 以降、ClickHouse は `CSV` 形式を使用する際に CSV ファイル内のヘッダーを自動的に検出するため、`CSVWithNames` や `CSVWithNamesAndTypes` を使用する必要はありません。 -::: - -以下の内容の CSV ファイル `football.csv` を使用します。 - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -テーブルを作成する: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -`CSVWithNames` 形式を使用してデータを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.csv' FORMAT CSVWithNames; -``` - - -### データの読み込み {#reading-data} - -`CSVWithNames` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT CSVWithNames -``` - -出力は、先頭に 1 行のヘッダーのみを持つ CSV になります。 - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) の設定値が `1` の場合、 -入力データのカラムはその名前に基づいてテーブルのカラムにマッピングされます。また、[`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) の設定値が `1` の場合、テーブル側に存在しない名前のカラムはスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md deleted file mode 100644 index e6d359a6719..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -alias: [] -description: 'CSVWithNamesAndTypes 形式に関するドキュメント' -input_format: true -keywords: ['CSVWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CSVWithNamesAndTypes -title: 'CSVWithNamesAndTypes' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[TabSeparatedWithNamesAndTypes](../formats/TabSeparatedWithNamesAndTypes) と同様に、列名と型が記載されたヘッダー行を 2 行出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -:::tip -[バージョン](https://github.com/ClickHouse/ClickHouse/releases) 23.1 以降では、ClickHouse は `CSV` フォーマットを使用する際に CSV ファイル内のヘッダーを自動検出するため、`CSVWithNames` や `CSVWithNamesAndTypes` を使用する必要はありません。 -::: - -以下の内容の CSV ファイル(`football_types.csv` という名前)を使用します。 - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -Date,Int16,LowCardinality(String),LowCardinality(String),Int8,Int8 -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -テーブルを作成する: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -`CSVWithNamesAndTypes` 形式を使用してデータを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football_types.csv' FORMAT CSVWithNamesAndTypes; -``` - - -### データの読み込み {#reading-data} - -`CSVWithNamesAndTypes` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT CSVWithNamesAndTypes -``` - -出力は、列名と型を表す 2 行のヘッダー行を持つ CSV になります。 - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"Date","Int16","LowCardinality(String)","LowCardinality(String)","Int8","Int8" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## フォーマット設定 {#format-settings} - -:::note -[input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 設定が `1` の場合、 -入力データの列は名前に基づいてテーブルの列にマッピングされます。[input_format_skip_unknown_fields](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が `1` の場合、名前が不明な列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: - -:::note -[input_format_with_types_use_header](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) 設定が `1` の場合、 -入力データの型は、テーブル内の対応する列の型と比較されます。そうでない場合は、2 行目がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md deleted file mode 100644 index 4f6570392f1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -alias: [] -description: 'CapnProto に関するドキュメント' -input_format: true -keywords: ['CapnProto'] -output_format: true -slug: /interfaces/formats/CapnProto -title: 'CapnProto' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✔ | | - - -## 説明 {#description} - -`CapnProto` フォーマットは、[`Protocol Buffers`](https://developers.google.com/protocol-buffers/) フォーマットや [Thrift](https://en.wikipedia.org/wiki/Apache_Thrift) に似たバイナリメッセージフォーマットであり、[JSON](./JSON/JSON.md) や [MessagePack](https://msgpack.org/) とは異なります。 -CapnProto メッセージは厳密に型付けされており、自己記述的ではないため、外部のスキーマ記述が必要です。スキーマは動的に適用され、クエリごとにキャッシュされます。 - -[Format Schema](/interfaces/formats/#formatschema) も参照してください。 - -## データ型の対応 {#data_types-matching-capnproto} - -以下の表は、サポートされているデータ型と、それらが `INSERT` クエリおよび `SELECT` クエリで ClickHouse の [データ型](/sql-reference/data-types/index.md) とどのように対応するかを示します。 - -| CapnProto データ型 (`INSERT`) | ClickHouse データ型 | CapnProto データ型 (`SELECT`) | -|------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------| -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md), [Date](/sql-reference/data-types/date.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md), [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md), [Decimal32](/sql-reference/data-types/decimal.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md), [DateTime64](/sql-reference/data-types/datetime.md), [Decimal64](/sql-reference/data-types/decimal.md) | `INT64` | -| `FLOAT32` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `FLOAT64` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `TEXT, DATA` | [String](/sql-reference/data-types/string.md), [FixedString](/sql-reference/data-types/fixedstring.md) | `TEXT, DATA` | -| `union(T, Void), union(Void, T)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(T, Void), union(Void, T)` | -| `ENUM` | [Enum(8/16)](/sql-reference/data-types/enum.md) | `ENUM` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `DATA` | [IPv6](/sql-reference/data-types/ipv6.md) | `DATA` | -| `DATA` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `DATA` | -| `DATA` | [Decimal128/Decimal256](/sql-reference/data-types/decimal.md) | `DATA` | -| `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | [Map](/sql-reference/data-types/map.md) | `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | - -- 整数型は、入出力時に相互に変換できます。 -- CapnProto 形式で `Enum` を扱う場合は、[format_capn_proto_enum_comparising_mode](/operations/settings/settings-formats.md/#format_capn_proto_enum_comparising_mode) 設定を使用してください。 -- 配列は入れ子にでき、引数として `Nullable` 型の値を持つことができます。`Tuple` 型および `Map` 型も入れ子にできます。 - -## 使用例 {#example-usage} - -### データの挿入と選択 {#inserting-and-selecting-data-capnproto} - -次のコマンドを実行すると、ファイル内の CapnProto データを ClickHouse のテーブルに挿入できます。 - -```bash -$ cat capnproto_messages.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_schema = 'schema:Message' FORMAT CapnProto" -``` - -`schema.capnp` は次のような内容です: - -```capnp -struct Message { - SearchPhrase @0 :Text; - c @1 :Uint64; -} -``` - -次のコマンドを使用して、ClickHouse テーブルからデータを抽出し、`CapnProto` 形式のファイルに保存できます。 - -```bash -$ clickhouse-client --query = "SELECT * FROM test.hits FORMAT CapnProto SETTINGS format_schema = 'schema:Message'" -``` - - -### 自動生成スキーマの使用 {#using-autogenerated-capn-proto-schema} - -データ用の外部 `CapnProto` スキーマがない場合でも、自動生成されたスキーマを使用して、`CapnProto` 形式でデータを出力および入力できます。 - -例: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS format_capn_proto_use_autogenerated_schema=1 -``` - -この場合、ClickHouse はテーブル構造に基づいて関数 [structureToCapnProtoSchema](/sql-reference/functions/other-functions.md#structureToCapnProtoSchema) を使用して CapnProto スキーマを自動生成し、このスキーマを用いてデータを CapnProto 形式でシリアライズします。 - -自動生成されたスキーマを使って CapnProto ファイルを読み込むこともできます(この場合、そのファイルは同じスキーマから生成されている必要があります): - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_capn_proto_use_autogenerated_schema=1 FORMAT CapnProto" -``` - - -## フォーマット設定 {#format-settings} - -設定 [`format_capn_proto_use_autogenerated_schema`](../../operations/settings/settings-formats.md/#format_capn_proto_use_autogenerated_schema) はデフォルトで有効になっており、[`format_schema`](/interfaces/formats#formatschema) が設定されていない場合に適用されます。 - -また、[`output_format_schema`](/operations/settings/formats#output_format_schema) 設定を使用して、入出力時に自動生成されたスキーマをファイルに保存することもできます。 - -例: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS - format_capn_proto_use_autogenerated_schema=1, - output_format_schema='path/to/schema/schema.capnp' -``` - -この場合、自動生成された `CapnProto` スキーマは `path/to/schema/schema.capnp` というファイルに保存されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md deleted file mode 100644 index 1f9d5f8813f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: [] -description: 'CustomSeparated 形式に関するドキュメント' -input_format: true -keywords: ['CustomSeparated'] -output_format: true -slug: /interfaces/formats/CustomSeparated -title: 'CustomSeparated' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[Template](../Template/Template.md) と似ていますが、すべての列名と列の型を出力または読み取りを行い、[format_custom_escaping_rule](../../../operations/settings/settings-formats.md/#format_custom_escaping_rule) 設定で指定されたエスケープ規則と、次の設定で指定されたデリミタを使用します。 - -- [format_custom_field_delimiter](/operations/settings/settings-formats.md/#format_custom_field_delimiter) -- [format_custom_row_before_delimiter](/operations/settings/settings-formats.md/#format_custom_row_before_delimiter) -- [format_custom_row_after_delimiter](/operations/settings/settings-formats.md/#format_custom_row_after_delimiter) -- [format_custom_row_between_delimiter](/operations/settings/settings-formats.md/#format_custom_row_between_delimiter) -- [format_custom_result_before_delimiter](/operations/settings/settings-formats.md/#format_custom_result_before_delimiter) -- [format_custom_result_after_delimiter](/operations/settings/settings-formats.md/#format_custom_result_after_delimiter) - -:::note -フォーマット文字列に定義されたエスケープ規則やデリミタは使用しません。 -::: - -[`CustomSeparatedIgnoreSpaces`](../CustomSeparated/CustomSeparatedIgnoreSpaces.md) というフォーマットもあり、これは [TemplateIgnoreSpaces](../Template//TemplateIgnoreSpaces.md) と似ています。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下の内容の `football.txt` という txt ファイルを使用します。 - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -カスタムの区切り文字を設定します: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparated; -``` - - -### データの読み込み {#reading-data} - -カスタム区切り文字を設定します。 - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -`CustomSeparated` 形式を使用してデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT CustomSeparated -``` - -出力は、構成したカスタム形式で行われます。 - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## フォーマット設定 {#format-settings} - -追加の設定: - -| Setting | 説明 | 既定値 | -|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|---------| -| [input_format_custom_detect_header](../../../operations/settings/settings-formats.md/#input_format_custom_detect_header) | ヘッダーに名前および型がある場合に、そのヘッダーを自動検出できるようにします。 | `true` | -| [input_format_custom_skip_trailing_empty_lines](../../../operations/settings/settings-formats.md/#input_format_custom_skip_trailing_empty_lines) | ファイル末尾の空行をスキップします。 | `false` | -| [input_format_custom_allow_variable_number_of_columns](../../../operations/settings/settings-formats.md/#input_format_custom_allow_variable_number_of_columns) | CustomSeparated フォーマットで可変数のカラムを許可し、余分なカラムは無視し、欠損しているカラムにはデフォルト値を使用します。 | `false` | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md deleted file mode 100644 index a5e4ac512cd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpaces 形式のドキュメント' -keywords: ['CustomSeparatedIgnoreSpaces'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpaces -title: 'CustomSeparatedIgnoreSpaces' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | | | - -## 概要 {#description} - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のような内容の、`football.txt` という名前の txt ファイルを使用します。 - -```text -row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -カスタム区切り文字の設定を行います。 - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpaces; -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md deleted file mode 100644 index 73806b02e2a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpacesWithNames 形式のドキュメント' -keywords: ['CustomSeparatedIgnoreSpacesWithNames'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNames -title: 'CustomSeparatedIgnoreSpacesWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | | | - -## 概要 {#description} - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の txt ファイル `football.txt` を使用します。 - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -カスタム区切り文字設定を構成します: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNames; -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md deleted file mode 100644 index 34276cc9efb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpacesWithNamesAndTypes 形式に関するドキュメント' -keywords: ['CustomSeparatedIgnoreSpacesWithNamesAndTypes'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNamesAndTypes -title: 'CustomSeparatedIgnoreSpacesWithNamesAndTypes' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | | | - -## 説明 {#description} - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の内容の txt ファイル `football.txt` を使用します: - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('Date'; 'Int16'; 'LowCardinality(String)'; 'LowCardinality(String)'; 'Int8'; 'Int8'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -カスタム区切り文字を設定します: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNamesAndTypes; -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md deleted file mode 100644 index 7c69dad0b94..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -alias: [] -description: 'CustomSeparatedWithNames 形式に関するドキュメント' -input_format: true -keywords: ['CustomSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNames -title: 'CustomSeparatedWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md) と同様に、列名付きのヘッダー行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下の内容の `football.txt` テキストファイルを使用します。 - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -カスタム区切り文字の設定を行います: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNames; -``` - - -### データの読み取り {#reading-data} - -カスタム区切り文字の設定を行います。 - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -`CustomSeparatedWithNames` 形式でデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNames -``` - -出力は、設定したカスタム形式で行われます。 - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) が `1` に設定されている場合、 -入力データのカラムは、その名前に基づいてテーブルのカラムにマッピングされます。 -このとき、[`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が `1` に設定されている場合は、名前が不明なカラムはスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md deleted file mode 100644 index f1b89d25c67..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -alias: [] -description: 'CustomSeparatedWithNamesAndTypes フォーマットに関するドキュメント' -input_format: true -keywords: ['CustomSeparatedWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNamesAndTypes -title: 'CustomSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| Input | Output | エイリアス | -|-------|--------|------------| -| ✔ | ✔ | | - -## 説明 {#description} - -[TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md) と同様に、列名と型を示す 2 行のヘッダーも出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のような `football.txt` という名前の txt ファイルを使用します。 - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -カスタム区切り文字を設定します: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNamesAndTypes; -``` - - -### データの読み込み {#reading-data} - -カスタム区切り文字を設定します。 - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -`CustomSeparatedWithNamesAndTypes` フォーマットを使用してデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNamesAndTypes -``` - -出力は、設定したカスタム形式で行われます。 - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) の設定値が `1` の場合、 -入力データの列は名前に基づいてテーブルの列にマッピングされます。さらに [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) の設定値が `1` の場合、未知の名前を持つ列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: - -:::note -[`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) の設定値が `1` の場合、 -入力データの型はテーブル内の対応する列の型と比較されます。そうでない場合は、2 行目がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md deleted file mode 100644 index 3429b0c80d8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'DWARF 形式のドキュメント' -input_format: true -keywords: ['DWARF'] -output_format: false -slug: /interfaces/formats/DWARF -title: 'DWARF' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✗ | | - -## 説明 {#description} - -`DWARF` フォーマットは、ELF ファイル(実行ファイル、ライブラリ、またはオブジェクトファイル)から DWARF デバッグシンボルをパースします。 -`dwarfdump` に似ていますが、はるかに高速(数百 MB/s)で動作し、SQL をサポートします。 -`.debug_info` セクション内の各 Debug Information Entry (DIE) ごとに 1 行を出力し、さらに、DWARF エンコーディングがツリー内の子リストを終端するために使用する「null」エントリも含みます。 - -:::info -`.debug_info` は *unit* から構成されており、これはコンパイル単位に対応します: - -- 各 unit は *DIE* の木構造であり、ルートには `compile_unit` DIE が存在します。 -- 各 DIE は *tag* と *attribute* のリストを持ちます。 -- 各 attribute は *name* と *value*(および、その値がどのようにエンコードされているかを指定する *form*)を持ちます。 - -DIE はソースコード中のさまざまな要素を表し、その *tag* によって何を表しているかが分かります。例えば次のようなものがあります: - -- 関数(tag = `subprogram`) -- クラス / 構造体 / enum(`class_type` / `structure_type` / `enumeration_type`) -- 変数(`variable`) -- 関数引数(`formal_parameter`) - -ツリー構造は対応するソースコードを反映しています。例えば、`class_type` DIE は、そのクラスのメソッドを表す `subprogram` DIE を含むことができます。 -::: - -`DWARF` フォーマットは次の列を出力します: - -- `offset` - `.debug_info` セクション内での DIE の位置 -- `size` - エンコードされた DIE のバイト数(attribute を含む) -- `tag` - DIE の種類。慣例的な `"DW_TAG_"` プレフィックスは省略されています -- `unit_name` - この DIE を含むコンパイル単位の名前 -- `unit_offset` - この DIE を含むコンパイル単位の `.debug_info` セクション内での位置 -- `ancestor_tags` - 現在の DIE の祖先となるタグの配列(内側から外側の順) -- `ancestor_offsets` - 祖先のオフセットの配列で、`ancestor_tags` と並行 -- 利便性のために、attribute 配列から複製された、よく使われる attribute: - - `name` - - `linkage_name` - マングルされた完全修飾名。通常は関数のみに存在します(すべての関数にあるとは限りません) - - `decl_file` - このエンティティが宣言されたソースコードファイル名 - - `decl_line` - このエンティティが宣言されたソースコード中の行番号 -- attribute を表現する並行配列: - - `attr_name` - attribute の名前。慣例的な `"DW_AT_"` プレフィックスは省略されています - - `attr_form` - attribute がどのようにエンコードおよび解釈されるか。慣例的な `DW_FORM_` プレフィックスは省略されています - - `attr_int` - attribute の整数値。attribute に数値がない場合は 0 - - `attr_str` - attribute の文字列値。attribute に文字列がない場合は空文字列 - -## 使用例 {#example-usage} - -`DWARF` 形式を使うと、最も多くの関数定義(テンプレートのインスタンス化や、インクルードされたヘッダーファイル内の関数を含む)を持つコンパイル単位を特定できます。 - -```sql title="Query" -SELECT - unit_name, - count() AS c -FROM file('programs/clickhouse', DWARF) -WHERE tag = 'subprogram' AND NOT has(attr_name, 'declaration') -GROUP BY unit_name -ORDER BY c DESC -LIMIT 3 -``` - -```text title="Response" -┌─unit_name──────────────────────────────────────────────────┬─────c─┐ -│ ./src/Core/Settings.cpp │ 28939 │ -│ ./src/AggregateFunctions/AggregateFunctionSumMap.cpp │ 23327 │ -│ ./src/AggregateFunctions/AggregateFunctionUniqCombined.cpp │ 22649 │ -└────────────────────────────────────────────────────────────┴───────┘ - -3行のセット。経過時間: 1.487秒。処理済み: 1億3976万行、1.12 GB (9397万行/秒、752.77 MB/秒) -ピークメモリ使用量: 271.92 MiB。 -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md deleted file mode 100644 index 202772e221e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -alias: [] -description: 'Form フォーマットのドキュメント' -input_format: true -keywords: ['Form'] -output_format: false -slug: /interfaces/formats/Form -title: 'Form' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✗ | | - -## 説明 {#description} - -`Form` 形式は、`application/x-www-form-urlencoded` 形式の単一レコードを読み取るために使用できます。 -この形式では、データは `key1=value1&key2=value2` のようにフォーマットされます。 - -## 使用例 {#example-usage} - -URL エンコードされたデータを含む `data.tmp` というファイルが `user_files` パスに配置されているとします。 - -```text title="data.tmp" -t_page=116&c.e=ls7xfkpm&c.tti.m=raf&rt.start=navigation&rt.bmr=390%2C11%2C10 -``` - -```sql title="Query" -SELECT * FROM file(data.tmp, Form) FORMAT vertical; -``` - -```response title="Response" -行 1: -────── -t_page: 116 -c.e: ls7xfkpm -c.tti.m: raf -rt.start: navigation -rt.bmr: 390,11,10 -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md deleted file mode 100644 index def14744f15..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'Hash フォーマットに関するドキュメント' -input_format: false -keywords: ['hash', 'format'] -output_format: true -slug: /interfaces/formats/Hash -title: 'Hash' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✗ | ✔ | | - -## 説明 {#description} - -`Hash` 出力形式は、結果のすべての列および行を対象に、1つのハッシュ値を計算します。 -これは、たとえばデータ転送がボトルネックとなっている状況で、結果の「フィンガープリント」を算出する場合に役立ちます。 - -## 使用例 {#example-usage} - -### データの読み取り {#reading-data} - -次のデータを含む `football` テーブルを考えます。 - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -`Hash` フォーマットを使ってデータを読み込む: - -```sql -SELECT * -FROM football -FORMAT Hash -``` - -クエリはデータを処理しますが、出力は行われません。 - -```response -df2ec2f0669b000edff6adee264e7d68 - -1 rows in set. Elapsed: 0.154 sec. -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md deleted file mode 100644 index 868923c0e9c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: 'HiveText 形式に関するドキュメント' -keywords: ['HiveText'] -slug: /interfaces/formats/HiveText -title: 'HiveText' -doc_type: 'reference' ---- - -## 概要 {#description} - -## 使用例 {#example-usage} - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md deleted file mode 100644 index f3983564ce1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -alias: [] -description: 'JSON 形式に関するドキュメント' -input_format: true -keywords: ['JSON'] -output_format: true -slug: /interfaces/formats/JSON -title: 'JSON' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -`JSON` フォーマットは、JSON 形式でデータの読み取りおよび出力を行います。 - -`JSON` フォーマットは次の内容を返します。 - -| Parameter | Description | -|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `meta` | 列名と型。 | -| `data` | データテーブル。 | -| `rows` | 出力される行数の合計。 | -| `rows_before_limit_at_least` | `LIMIT` がなかった場合に存在し得る行数の下限推定値。クエリに `LIMIT` が含まれている場合にのみ出力されます。この推定値は、limit 変換の実行前にクエリパイプラインで処理されたデータブロックから計算されますが、その後 limit 変換によって破棄される可能性があります。クエリパイプラインでデータブロックが limit 変換に到達しなかった場合、それらは推定に含まれません。 | -| `statistics` | `elapsed`、`rows_read`、`bytes_read` などの統計情報。 | -| `totals` | (`WITH TOTALS` を使用している場合の)合計値。 | -| `extremes` | (`extremes` が 1 に設定されている場合の)極値。 | - -`JSON` 型は JavaScript と互換性があります。この互換性を確保するため、いくつかの文字は追加でエスケープされます。 - -- スラッシュ `/` は `\/` としてエスケープされます。 -- 一部のブラウザで問題を引き起こす代替改行文字 `U+2028` および `U+2029` は `\uXXXX` としてエスケープされます。 -- ASCII 制御文字はエスケープされます。バックスペース、フォームフィード、ラインフィード、キャリッジリターン、および水平タブはそれぞれ `\b`、`\f`、`\n`、`\r`、`\t` に置き換えられ、さらに 00-1F の範囲の残りのバイトは `\uXXXX` シーケンスで表現されます。 -- 無効な UTF-8 シーケンスは代替文字 � に置き換えられ、出力テキストが有効な UTF-8 シーケンスのみで構成されるようにします。 - -JavaScript との互換性のため、Int64 および UInt64 整数はデフォルトで二重引用符で囲まれます。 -引用符を削除するには、設定パラメータ [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) を `0` に設定します。 - -ClickHouse は [NULL](/sql-reference/syntax.md) をサポートしており、JSON 出力では `null` として表示されます。出力で `+nan`、`-nan`、`+inf`、`-inf` の値を有効にするには、[`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) を `1` に設定します。 - -## 使用例 {#example-usage} - -例: - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase WITH TOTALS ORDER BY c DESC LIMIT 5 FORMAT JSON -``` - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "hello", - "arr": [0,1] - }, - { - "num": 43, - "str": "hello", - "arr": [0,1,2] - }, - { - "num": 44, - "str": "hello", - "arr": [0,1,2,3] - } - ], - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.001137687, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - - -## フォーマット設定 {#format-settings} - -JSON 入力フォーマットの場合、[`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) 設定が `1` に設定されていると、入力データ内のメタデータに含まれる型が、テーブル内の対応する列の型と照合されます。 - -## 関連項目 {#see-also} - -- [JSONEachRow](/interfaces/formats/JSONEachRow) フォーマット -- [output_format_json_array_of_rows](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) 設定 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md deleted file mode 100644 index 8fba51853b2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'JSONAsObject 形式に関するドキュメント' -input_format: true -keywords: ['JSONAsObject'] -output_format: false -slug: /interfaces/formats/JSONAsObject -title: 'JSONAsObject' -doc_type: 'reference' ---- - -## 説明 {#description} - -この形式では、1 つの JSON オブジェクトが 1 つの [JSON](/sql-reference/data-types/newjson.md) 値として解釈されます。入力に複数の JSON オブジェクト(カンマ区切り)が含まれる場合、それぞれが個別の行として解釈されます。入力データが角括弧で囲まれている場合は、JSON の配列として解釈されます。 - -この形式をパースできるのは、[JSON](/sql-reference/data-types/newjson.md) 型のフィールドが 1 つだけのテーブルに対してのみです。残りのカラムは [`DEFAULT`](/sql-reference/statements/create/table.md/#default) または [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view) に設定されている必要があります。 - -## 使用例 {#example-usage} - -### 基本的な例 {#basic-example} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_object FORMAT JSONEachRow; -``` - -```response title="Response" -{"json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -{"json":{}} -{"json":{"any json stucture":"1"}} -``` - - -### JSON オブジェクトの配列 {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field JSON) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsObject [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; -SELECT * FROM json_square_brackets FORMAT JSONEachRow; -``` - -```response title="Response" -{"field":{"id":"1","name":"name1"}} -{"field":{"id":"2","name":"name2"}} -``` - - -### デフォルト値を持つ列 {#columns-with-default-values} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON, time DateTime MATERIALIZED now()) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"any json stucture":1} -SELECT time, json FROM json_as_object FORMAT JSONEachRow -``` - -```response title="Response" -{"time":"2024-09-16 12:18:10","json":{}} -{"time":"2024-09-16 12:18:13","json":{"any json stucture":"1"}} -{"time":"2024-09-16 12:18:08","json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md deleted file mode 100644 index dce25c787ab..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'JSONAsString フォーマットに関するドキュメント' -input_format: true -keywords: ['JSONAsString'] -output_format: false -slug: /interfaces/formats/JSONAsString -title: 'JSONAsString' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|---------|-------| -| ✔ | ✗ | | - -## 説明 {#description} - -この形式では、1つの JSON オブジェクトは1つの値として解釈されます。 -入力に複数の JSON オブジェクト(カンマ区切り)が含まれている場合、それぞれが個別の行として解釈されます。 -入力データが角かっこで囲まれている場合、それは JSON オブジェクトの配列として解釈されます。 - -:::note -この形式は、型が [String](/sql-reference/data-types/string.md) の1つのフィールドを持つテーブルに対してのみ解析できます。 -残りの列は [`DEFAULT`](/sql-reference/statements/create/table.md/#default) または [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view) のいずれかに設定するか、 -省略する必要があります。 -::: - -JSON オブジェクト全体を String にシリアライズしたら、[JSON 関数](/sql-reference/functions/json-functions.md) を使用して処理できます。 - -## 使用例 {#example-usage} - -### 基本的な例 {#basic-example} - -```sql title="Query" -DROP TABLE IF EXISTS json_as_string; -CREATE TABLE json_as_string (json String) ENGINE = Memory; -INSERT INTO json_as_string (json) FORMAT JSONAsString {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_string; -``` - -```response title="Response" -┌─json──────────────────────────────┐ -│ {"foo":{"bar":{"x":"y"},"baz":1}} │ -│ {} │ -│ {"any json stucture":1} │ -└───────────────────────────────────┘ -``` - - -### JSON オブジェクトの配列 {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field String) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsString [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; - -SELECT * FROM json_square_brackets; -``` - -```response title="Response" -┌─field──────────────────────┐ -│ {"id": 1, "name": "name1"} │ -│ {"id": 2, "name": "name2"} │ -└────────────────────────────┘ -``` - - -## フォーマットの設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md deleted file mode 100644 index ca280b3cf7e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -alias: [] -description: 'JSONColumns 形式のドキュメント' -input_format: true -keywords: ['JSONColumns'] -output_format: true -slug: /interfaces/formats/JSONColumns -title: 'JSONColumns' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -:::tip -`JSONColumns*` フォーマットの出力では、最初に ClickHouse のフィールド名が表示され、そのフィールドに対応するテーブル内の各行の内容が続きます。 -見た目としては、データが左に 90 度回転したような配置になります。 -::: - -このフォーマットでは、すべてのデータは 1 つの JSON オブジェクトとして表現されます。 - -:::note -`JSONColumns` フォーマットは、すべてのデータをメモリ上にバッファしてから 1 つのブロックとして出力するため、メモリ使用量が増大する可能性があります。 -::: - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む `football.json` という名前の JSON ファイルを使用します。 - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - -データを挿入します。 - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONColumns; -``` - - -### データの読み込み {#reading-data} - -`JSONColumns` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONColumns -``` - -出力は JSON 形式です。 - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - - -## フォーマット設定 {#format-settings} - -インポート時に、名前が不明な列は、設定 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が `1` に設定されている場合はスキップされます。 -ブロック内に存在しない列はデフォルト値で埋められます(この場合、設定 [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) を使用できます)。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md deleted file mode 100644 index a8cab2ae3ab..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -alias: [] -description: 'JSONColumnsWithMetadata 形式に関するドキュメント' -input_format: true -keywords: ['JSONColumnsWithMetadata'] -output_format: true -slug: /interfaces/formats/JSONColumnsWithMetadata -title: 'JSONColumnsWithMetadata' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONColumns`](./JSONColumns.md) フォーマットとは異なり、一部のメタデータおよび統計情報も含まれます([`JSON`](./JSON.md) フォーマットと同様)。 - -:::note -`JSONColumnsWithMetadata` フォーマットは、すべてのデータをメモリ内にバッファリングしてから単一のブロックとして出力するため、メモリ消費量が大きくなる可能性があります。 -::: - -## 使用例 {#example-usage} - -例: - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - { - "num": [42, 43, 44], - "str": ["hello", "hello", "hello"], - "arr": [[0,1], [0,1,2], [0,1,2,3]] - }, - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.000272376, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - -`JSONColumnsWithMetadata` 入力形式では、[`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) の設定が `1` の場合、 -入力データ内のメタデータに含まれる型が、テーブル内の対応するカラムの型と比較されます。 - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md deleted file mode 100644 index 21a6c69c23c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md +++ /dev/null @@ -1,160 +0,0 @@ ---- -alias: [] -description: 'JSONCompact 形式に関するドキュメント' -input_format: true -keywords: ['JSONCompact'] -output_format: true -slug: /interfaces/formats/JSONCompact -title: 'JSONCompact' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[JSON](./JSON.md) との違いは、データ行がオブジェクトではなく配列として出力される点のみです。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータが含まれた JSON ファイル `football.json` を用意します: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ] -} -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompact; -``` - - -### データの読み込み {#reading-data} - -`JSONCompact` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompact -``` - -出力は JSON 形式です: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.223690876, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md deleted file mode 100644 index 5f07d1832f7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -alias: [] -description: 'JSONCompactColumns フォーマットのドキュメント' -input_format: true -keywords: ['JSONCompactColumns'] -output_format: true -slug: /interfaces/formats/JSONCompactColumns -title: 'JSONCompactColumns' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -この形式では、すべてのデータは 1 つの JSON 配列として表現されます。 - -:::note -`JSONCompactColumns` 出力形式は、すべてのデータを 1 つのブロックとして出力するためにメモリ上にバッファリングするため、多くのメモリを消費する可能性があります。 -::: - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータが含まれた JSON ファイル `football.json` を使用します。 - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactColumns; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactColumns` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactColumns -``` - -出力はJSON形式です: - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -ブロック内に存在しないカラムにはデフォルト値が補われます(ここでは [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) 設定を使用できます) - - -## フォーマットの設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md deleted file mode 100644 index 24c2ee5d02e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRow フォーマットに関するドキュメント' -input_format: true -keywords: ['JSONCompactEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRow -title: 'JSONCompactEachRow' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONEachRow`](./JSONEachRow.md) と異なる点は、データ行がオブジェクトではなく配列として出力されることだけです。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下のデータを含む JSON ファイルを `football.json` という名前で用意します: - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRow; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactEachRow` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRow -``` - -出力は JSON 形式で行われます。 - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md deleted file mode 100644 index b18be220b9d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithNames フォーマットのドキュメント' -input_format: true -keywords: ['JSONCompactEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNames -title: 'JSONCompactEachRowWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONCompactEachRow`](./JSONCompactEachRow.md) 形式とは異なり、[`TabSeparatedWithNames`](../TabSeparated/TabSeparatedWithNames.md) 形式と同様に、列名を含むヘッダー行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下のデータを含む JSON ファイル `football.json` を作成します: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNames; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactEachRowWithNames` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNames -``` - -出力は JSON 形式です。 - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) が 1 の場合、入力データの列は名前に基づいてテーブルの列にマッピングされます。また、[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が 1 の場合は、名前が不明な列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md deleted file mode 100644 index 2ae3da830a7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithNamesAndTypes フォーマットに関するドキュメント' -input_format: true -keywords: ['JSONCompactEachRowWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNamesAndTypes -title: 'JSONCompactEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONCompactEachRow`](./JSONCompactEachRow.md) 形式とは異なり、[TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md) 形式と同様に、列名とデータ型を含むヘッダー 2 行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下のデータを含む JSON ファイル `football.json` を用意します。 - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNamesAndTypes; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactEachRowWithNamesAndTypes` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNamesAndTypes -``` - -出力は JSON 形式です: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) が `1` に設定されている場合、 -入力データのカラムは名前に基づいてテーブルのカラムにマッピングされます。[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が `1` に設定されている場合、不明な名前のカラムはスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -[`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header) が `1` に設定されている場合、 -入力データの型はテーブル内の対応するカラムの型と比較されます。そうでない場合は、2 行目がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md deleted file mode 100644 index 81df59f10d8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithProgress 形式のドキュメント' -input_format: false -keywords: ['JSONCompactEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithProgress -title: 'JSONCompactEachRowWithProgress' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | | - -## 説明 {#description} - -このフォーマットは、JSONCompactEachRow のコンパクトな行単位出力に、ストリーミングされる進行状況情報を組み合わせたものです。 -メタデータ、各行、進行状況の更新、合計値、例外を、それぞれ個別の JSON オブジェクトとして出力します。値はネイティブな型で表現されます。 - -主な特徴: - -- 最初に列名と型を含むメタデータを出力します -- 各行は、値の配列を含む「row」キーを持つ個別の JSON オブジェクトとして出力されます -- クエリ実行中の進行状況更新を含みます(`{"progress":...}` オブジェクト) -- totals と extremes をサポートします -- 値はネイティブな型を保持します(数値は数値、文字列は文字列として扱われます) - -## 使用例 {#example-usage} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":[[-8], 46848.5225, ["2064-06-11 14:00:36.578","b06f4fa1-22ff-f84f-a1b7-a5807d983ae6"]]} -{"row":[[-76], -85331.598, ["2038-06-16 04:10:27.271","2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c"]]} -{"row":[[-32], -31470.8994, ["2027-07-18 16:58:34.654","1cdbae4c-ceb2-1337-b954-b175f5efbef8"]]} -{"row":[[-116], 32104.097, ["1979-04-27 21:51:53.321","66903704-3c83-8f8a-648a-da4ac1ffa9fc"]]} -{"row":[[], 2427.6614, ["1980-04-24 11:30:35.487","fee19be8-0f46-149b-ed98-43e7455ce2b2"]]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"335771"}} -{"rows_before_limit_at_least":5} -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md deleted file mode 100644 index 74ee5ef6389..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStrings フォーマットのドキュメント' -input_format: false -keywords: ['JSONCompactStrings'] -output_format: true -slug: /interfaces/formats/JSONCompactStrings -title: 'JSONCompactStrings' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | | - -## 概要 {#description} - -`JSONCompactStrings` フォーマットは、データ行がオブジェクトではなく配列として出力されるという点だけが、[JSONStrings](./JSONStrings.md) と異なります。 - -## 使用例 {#example-usage} - -### データの読み込み {#reading-data} - -`JSONCompactStrings` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactStrings -``` - -出力結果は JSON 形式になります: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"], - ["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"], - ["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"], - ["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"], - ["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"], - ["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"], - ["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"], - ["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"], - ["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"], - ["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"], - ["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"], - ["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"], - ["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"], - ["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"], - ["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"], - ["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"], - ["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.112012501, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md deleted file mode 100644 index e192c242453..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRow フォーマットに関するドキュメント' -input_format: true -keywords: ['JSONCompactStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRow -title: 'JSONCompactStringsEachRow' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONCompactEachRow`](./JSONCompactEachRow.md) とは、データフィールドが型付きの JSON 値ではなく文字列として出力される点のみが異なります。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む JSON ファイル `football.json` を用意します。 - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRow; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactStringsEachRow` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRow -``` - -出力は JSON 形式です: - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## 形式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md deleted file mode 100644 index c84fa4fc158..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRowWithNames 形式に関するドキュメント' -input_format: true -keywords: ['JSONCompactStringsEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithNames -title: 'JSONCompactStringsEachRowWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[`JSONCompactEachRow`](./JSONCompactEachRow.md) 形式とは異なり、[TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md) 形式と同様に、列名を含むヘッダー行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを格納した JSON ファイルを `football.json` という名前で用意します。 - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNames; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactStringsEachRowWithNames` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNames -``` - -出力は JSON 形式です: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## フォーマット設定 {#format-settings} - -:::note -[`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) が `1` に設定されている場合、 -入力データの列は名前に基づいてテーブルの列にマッピングされ、[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が `1` に設定されている場合は、未知の名前を持つ列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md deleted file mode 100644 index 5404c63a740..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'JSONCompactStringsEachRowWithNamesAndTypes フォーマットに関するドキュメント' -keywords: ['JSONCompactStringsEachRowWithNamesAndTypes'] -slug: /interfaces/formats/JSONCompactStringsEachRowWithNamesAndTypes -title: 'JSONCompactStringsEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -`JSONCompactEachRow` フォーマットとは異なり、[TabSeparatedWithNamesAndTypes](/interfaces/formats/TabSeparatedRawWithNamesAndTypes) と同様に、列名と型を示す 2 行のヘッダー行も出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む JSON ファイルを `football.json` という名前で用意します: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNamesAndTypes; -``` - - -### データの読み込み {#reading-data} - -`JSONCompactStringsEachRowWithNamesAndTypes` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNamesAndTypes -``` - -出力は JSON 形式です。 - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## フォーマット設定 {#format-settings} - -:::note -[input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) の設定値が 1 の場合、 -入力データの列は名前に基づいてテーブルの列にマッピングされます。[input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) の設定値が 1 の場合、テーブル側に存在しない名前の列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -::: - -:::note -[input_format_with_types_use_header](/operations/settings/settings-formats.md/#input_format_with_types_use_header) の設定値が 1 の場合、 -入力データの型は、テーブル内の対応する列の型と比較されます。そうでない場合は、2 行目がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md deleted file mode 100644 index 52998ed4866..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRowWithProgress 形式に関するドキュメント' -input_format: true -keywords: ['JSONCompactStringsEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithProgress -title: 'JSONCompactStringsEachRowWithProgress' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|---------|--------| -| ✗ | ✔ | | - -## 説明 {#description} - -[`JSONCompactEachRowWithProgress`](/interfaces/formats/JSONCompactEachRowWithProgress) と同様ですが、すべての値が文字列に変換されます。 -これは、すべてのデータ型について一貫した文字列表現が必要な場合に役立ちます。 - -主な特徴: - -- `JSONCompactEachRowWithProgress` と同じ構造 -- すべての値が文字列として表現される(数値や配列なども、すべて引用符で囲まれた文字列) -- 進捗情報、合計値、例外処理を含む -- 文字列ベースのデータを好む、または必要とするクライアントに適している - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactStringsEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":["[-8]", "46848.5225", "('2064-06-11 14:00:36.578','b06f4fa1-22ff-f84f-a1b7-a5807d983ae6')"]} -{"row":["[-76]", "-85331.598", "('2038-06-16 04:10:27.271','2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c')"]} -{"row":["[-32]", "-31470.8994", "('2027-07-18 16:58:34.654','1cdbae4c-ceb2-1337-b954-b175f5efbef8')"]} -{"row":["[-116]", "32104.097", "('1979-04-27 21:51:53.321','66903704-3c83-8f8a-648a-da4ac1ffa9fc')"]} -{"row":["[]", "2427.6614", "('1980-04-24 11:30:35.487','fee19be8-0f46-149b-ed98-43e7455ce2b2')"]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"191151"}} -{"rows_before_limit_at_least":5} -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md deleted file mode 100644 index 562d9f91a18..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: ['JSONLines', 'NDJSON'] -description: 'JSONEachRow 形式に関するドキュメント' -keywords: ['JSONEachRow'] -slug: /interfaces/formats/JSONEachRow -title: 'JSONEachRow' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|------|------|-------------------------| -| ✔ | ✔ | `JSONLines`, `NDJSON` | - -## 説明 {#description} - -この形式では、ClickHouse は各行を改行区切りの個別の JSON オブジェクトとして出力します。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む JSON ファイル `football.json` を用意します: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONEachRow; -``` - - -### データの読み取り {#reading-data} - -`JSONEachRow` 形式を使ってデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONEachRow -``` - -出力は JSON 形式です: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -[input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が 1 に設定されている場合、不明な名前のデータ列のインポートはスキップされます。 - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md deleted file mode 100644 index f29536916ab..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -alias: [] -description: 'JSONEachRowWithProgress 形式のドキュメント' -input_format: false -keywords: ['JSONEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONEachRowWithProgress -title: 'JSONEachRowWithProgress' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✗ | ✔ | | - -## 説明 {#description} - -[`JSONEachRow`](./JSONEachRow.md)/[`JSONStringsEachRow`](./JSONStringsEachRow.md) とは異なり、ClickHouseは進捗情報もJSON値として出力します。 - -## 使用例 {#example-usage} - -```json -{"row":{"num":42,"str":"hello","arr":[0,1]}} -{"row":{"num":43,"str":"hello","arr":[0,1,2]}} -{"row":{"num":44,"str":"hello","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## 形式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md deleted file mode 100644 index 72bab9bb104..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -alias: ['JSONEachRow', 'JSONLines', 'NDJSON', 'JSONL'] -description: 'JSONLines フォーマットに関するドキュメント' -keywords: ['JSONLines'] -slug: /interfaces/formats/JSONLines -title: 'JSONLines' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|----------------------------------------------| -| ✔ | ✔ | `JSONEachRow`, `JSONLines`, `NDJSON`, `JSONL` | - -## 説明 {#description} - -この形式では、ClickHouse は各行を改行区切りの個別の JSON オブジェクトとして出力します。 - -この形式は `JSONEachRow`、`NDJSON`(Newline Delimited JSON)、または `JSONL`(`JSONLines`)としても知られています。これらの名称はすべて同じ形式の別名であり、区別なく使用できます。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む `football.json` という名前の JSON ファイルを使用します。 - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONLines; -``` - - -### データの読み取り {#reading-data} - -`JSONLines` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONLines -``` - -出力は JSON 形式です: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -列名が不明な列のインポートは、[input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が 1 に設定されている場合、スキップされます。 - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md deleted file mode 100644 index 600843b8af9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md +++ /dev/null @@ -1,245 +0,0 @@ ---- -alias: [] -description: 'JSONObjectEachRow フォーマットに関するドキュメント' -input_format: true -keywords: ['JSONObjectEachRow'] -output_format: true -slug: /interfaces/formats/JSONObjectEachRow -title: 'JSONObjectEachRow' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -この形式では、すべてのデータは 1 つの JSON オブジェクトとして表され、そのオブジェクト内で各行が個別のフィールドとして表現されます。これは [`JSONEachRow`](./JSONEachRow.md) 形式と同様です。 - - - -## 使用例 {#example-usage} - -### 基本的な例 {#basic-example} - -次のような JSON データがあるとします: - -```json -{ - "row_1": {"num": 42, "str": "hello", "arr": [0,1]}, - "row_2": {"num": 43, "str": "hello", "arr": [0,1,2]}, - "row_3": {"num": 44, "str": "hello", "arr": [0,1,2,3]} -} -``` - -オブジェクト名をカラム値として使用するには、専用の設定 [`format_json_object_each_row_column_for_object_name`](/operations/settings/settings-formats.md/#format_json_object_each_row_column_for_object_name) を利用できます。 -この設定には、結果オブジェクト内の各行に対して JSON のキーとして使用するカラム名を指定します。 - -#### 出力 {#output} - -テーブル `test` に 2 つのカラムがあるとします。 - -```text -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -`JSONObjectEachRow` 形式で出力し、`format_json_object_each_row_column_for_object_name` 設定を使用します。 - -```sql title="Query" -SELECT * FROM test SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```json title="Response" -{ - "first_obj": {"number": 1}, - "second_obj": {"number": 2}, - "third_obj": {"number": 3} -} -``` - -#### 入力 {#input} - -前の例の出力を `data.json` という名前のファイルに保存してあるとします。 - -```sql title="Query" -SELECT * FROM file('data.json', JSONObjectEachRow, 'object_name String, number UInt64') SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -これはスキーマ推論にも利用できます: - -```sql title="Query" -DESCRIBE file('data.json', JSONObjectEachRow) SETTING format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─name────────┬─type────────────┐ -│ object_name │ String │ -│ number │ Nullable(Int64) │ -└─────────────┴─────────────────┘ -``` - -### データの挿入 {#json-inserting-data} - -```sql title="Query" -INSERT INTO UserActivity FORMAT JSONEachRow {"PageViews":5, "UserID":"4324182021466249494", "Duration":146,"Sign":-1} {"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -ClickHouse では次のことができます: - -* オブジェクト内のキーと値のペアの順序は任意です。 -* 一部の値を省略できます。 - -ClickHouse は要素間の空白や、オブジェクトの後に続くカンマを無視します。すべてのオブジェクトを 1 行で渡すことができます。改行で区切る必要はありません。 - -#### 省略された値の処理 {#omitted-values-processing} - -ClickHouse は、省略された値を対応する[データ型](/sql-reference/data-types/index.md)のデフォルト値で補います。 - -`DEFAULT expr` が指定されている場合、ClickHouse は [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) 設定に応じて異なる補完ルールを使用します。 - -次のテーブルを考えてみます: - -```sql title="Query" -CREATE TABLE IF NOT EXISTS example_table -( - x UInt32, - a DEFAULT x * 2 -) ENGINE = Memory; -``` - -* `input_format_defaults_for_omitted_fields = 0` の場合、`x` と `a` のデフォルト値は `0`(`UInt32` データ型のデフォルト値)になります。 -* `input_format_defaults_for_omitted_fields = 1` の場合、`x` のデフォルト値は `0` ですが、`a` のデフォルト値は `x * 2` になります。 - -:::note -`input_format_defaults_for_omitted_fields = 1` を指定してデータを挿入すると、`input_format_defaults_for_omitted_fields = 0` の場合と比べて、ClickHouse はより多くの計算リソースを消費します。 -::: - -### データの選択 {#json-selecting-data} - -例として、`UserActivity` テーブルを使用します。 - - -```response -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -クエリ `SELECT * FROM UserActivity FORMAT JSONEachRow` は次のような結果を返します: - -```response -{"UserID":"4324182021466249494","PageViews":5,"Duration":146,"Sign":-1} -{"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -[JSON](/interfaces/formats/JSON) フォーマットとは異なり、不正な UTF-8 シーケンスの置換は行われません。値は `JSON` と同じ方法でエスケープされます。 - -:::info -任意のバイト列を文字列として出力できます。テーブル内のデータを、情報を失うことなく JSON として整形できると確信できる場合は、[`JSONEachRow`](./JSONEachRow.md) フォーマットを使用してください。 -::: - -### ネスト構造の使用 {#jsoneachrow-nested} - -[`Nested`](/sql-reference/data-types/nested-data-structures/index.md) データ型のカラムを持つテーブルがある場合、同じ構造を持つ JSON データを挿入できます。この機能は、[input_format_import_nested_json](/operations/settings/settings-formats.md/#input_format_import_nested_json) 設定を有効にすることで使用できます。 - -たとえば、次のテーブルを考えてみます。 - -```sql -CREATE TABLE json_each_row_nested (n Nested (s String, i Int32) ) ENGINE = Memory -``` - -`Nested` データ型の説明で確認できるように、ClickHouse はネストされた構造の各コンポーネントを個別のカラム(このテーブルでは `n.s` と `n.i`)として扱います。データは次のように挿入できます: - -```sql -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n.s": ["abc", "def"], "n.i": [1, 23]} -``` - -データを階層的なJSONオブジェクトとして挿入するには、[`input_format_import_nested_json=1`](/operations/settings/settings-formats.md/#input_format_import_nested_json) を設定します。 - -```json -{ - "n": { - "s": ["abc", "def"], - "i": [1, 23] - } -} -``` - -この設定がない場合、ClickHouse は例外をスローします。 - -```sql title="Query" -SELECT name, value FROM system.settings WHERE name = 'input_format_import_nested_json' -``` - -```response title="Response" -┌─name────────────────────────────┬─value─┐ -│ input_format_import_nested_json │ 0 │ -└─────────────────────────────────┴───────┘ -``` - -```sql title="Query" -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -``` - -```response title="Response" -コード: 117. DB::Exception: JSONEachRow形式の解析中に不明なフィールドが検出されました: n: (1行目) -``` - -```sql title="Query" -SET input_format_import_nested_json=1 -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -SELECT * FROM json_each_row_nested -``` - -```response title="Response" -┌─n.s───────────┬─n.i────┐ -│ ['abc','def'] │ [1,23] │ -└───────────────┴────────┘ -``` - - -## フォーマット設定 {#format-settings} - - - -| 設定 | 概要 | デフォルト | 注記 | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | ネストされた JSON データをネストされたテーブルにマッピングします(JSONEachRow フォーマットで動作します)。 | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | JSON 入力フォーマットでブール値を数値として解釈できるようにします。 | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | JSON入力フォーマットで、ブール値を文字列として解釈できるようにします。 | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | JSON入力フォーマットで数値を文字列として解析できるようにします。 | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | JSON入力フォーマットでJSON配列を文字列としてパースできるようにします。 | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | JSON入力フォーマットでJSONオブジェクトを文字列として解析できるようにします。 | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | NamedTuple 型の列を JSON オブジェクトとして解析します。 | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | スキーマ推論時に、文字列フィールドから数値型を推定しようとします。 | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | スキーマ推論時に JSON オブジェクトから名前付きタプル型を推論しようとします。 | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | JSON 入力フォーマットのスキーマ推論時には、NULL または空のオブジェクト/配列だけが含まれるキーには型 String を使用します。 | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | named tuple の解析時に、JSON オブジェクト内の不足している要素にデフォルト値を挿入する。 | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | 名前付きタプルの JSON オブジェクト内に存在する未知のキーを無視します。 | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | JSONCompact/JSONCompactEachRow 形式で列数を可変にし、余分な列は無視して、存在しない列にはデフォルト値を使用します。 | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | JSON 文字列に不正なエスケープシーケンスが含まれている場合は例外をスローします。無効にすると、不正なエスケープシーケンスはデータ内にそのまま残ります。 | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | JSON 入力で空のフィールドをデフォルト値として扱います。 | `false` | 複雑なデフォルト式を使用する場合は、[`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) も有効にする必要があります。 | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | JSON 出力形式で 64 ビット整数をクォート(文字列として出力)するかどうかを制御します。 | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | JSON 出力形式における 64 ビット浮動小数点数のクォート方法を制御します。 | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | JSON 出力形式において '+nan', '-nan', '+inf', '-inf' の出力を有効にします。 | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | JSON 出力形式での 10 進数のクォート方法を制御します。 | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | JSON 出力形式で、文字列中のスラッシュ (/) をエスケープするかどうかを制御します。 | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | NamedTuple 型の列を JSON オブジェクトとしてシリアル化します。 | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | すべての行を JSONEachRow(Compact) 形式の JSON 配列として出力します。 | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | JSON 出力フォーマットでの UTF-8 シーケンス検証を有効にします(ただし JSON/JSONCompact/JSONColumnsWithMetadata フォーマットには影響しません。これらは常に UTF-8 を検証します)。 | `false` | | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md deleted file mode 100644 index caa4ae546c5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md +++ /dev/null @@ -1,398 +0,0 @@ ---- -alias: [] -description: 'JSONStrings フォーマットのドキュメント' -input_format: true -keywords: ['JSONStrings'] -output_format: true -slug: /interfaces/formats/JSONStrings -title: 'JSONStrings' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -[JSON](./JSON.md) 形式との違いは、データ フィールドが型付きの JSON 値ではなく、文字列として出力される点だけです。 - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含んだ JSON ファイルを `football.json` という名前で用意します: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ] -} -``` - -データを挿入します。 - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStrings; -``` - - -### データの読み込み {#reading-data} - -`JSONStrings` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONStrings -``` - -出力は JSON 形式で表示されます。 - - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.173464376, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md deleted file mode 100644 index ac2230fd114..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -alias: [] -description: 'JSONStringsEachRow フォーマットに関するドキュメント' -input_format: false -keywords: ['JSONStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONStringsEachRow -title: 'JSONStringsEachRow' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -[`JSONEachRow`](./JSONEachRow.md) との違いは、データフィールドが型付きのJSON値ではなく文字列として出力される点だけです。 - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下のデータを含む JSON ファイルを `football.json` として用意します: - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStringsEachRow; -``` - -### データの読み込み {#reading-data} - -`JSONStringsEachRow` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT JSONStringsEachRow -``` - -出力は JSON 形式です: - - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - - -## フォーマット設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md deleted file mode 100644 index 0c48982985b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'JSONStringsEachRowWithProgress 形式のドキュメント' -keywords: ['JSONStringsEachRowWithProgress'] -slug: /interfaces/formats/JSONStringsEachRowWithProgress -title: 'JSONStringsEachRowWithProgress' -doc_type: 'reference' ---- - - - -## 説明 {#description} - -`JSONEachRow`/`JSONStringsEachRow` と異なり、ClickHouse は進捗情報も JSON 形式で出力します。 - - - -## 使用例 {#example-usage} - -```json -{"row":{"num":42,"str":"こんにちは","arr":[0,1]}} -{"row":{"num":43,"str":"こんにちは","arr":[0,1,2]}} -{"row":{"num":44,"str":"こんにちは","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## 書式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md deleted file mode 100644 index 130f9ece07f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -alias: ['PrettyJSONLines', 'PrettyNDJSON'] -description: 'PrettyJSONLines 形式に関するドキュメント' -input_format: false -keywords: ['PrettyJSONEachRow', 'PrettyJSONLines', 'PrettyNDJSON'] -output_format: true -slug: /interfaces/formats/PrettyJSONEachRow -title: 'PrettyJSONEachRow' -doc_type: 'guide' ---- - -| 入力 | 出力 | 別名 | -|------|------|-----------------------------------| -| ✗ | ✔ | `PrettyJSONLines`, `PrettyNDJSON` | - - - -## 説明 {#description} - -[JSONEachRow](./JSONEachRow.md) と異なる点は、JSON が改行区切りおよび 4 つのスペースによるインデントを付けて整形されていることだけです。 - - - -## 使用例 {#example-usage} -### データの挿入 {#inserting-data} - -次のデータを含む JSON ファイル `football.json` を用意します。 - - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT PrettyJSONEachRow; -``` - -### データの読み取り {#reading-data} - -`PrettyJSONEachRow` フォーマットを使用してデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT PrettyJSONEachRow -``` - -出力は JSON 形式です。 - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - - - - - -## 形式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md deleted file mode 100644 index 7b74844eef0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'JSON フォーマットの設定一覧' -keywords: ['フォーマット設定', 'JSON'] -slug: /interfaces/formats/JSON/format-settings -title: 'JSON 向けフォーマット設定' -doc_type: 'reference' ---- - -このページでは、すべての JSON フォーマットで共通して使用されるフォーマット設定を確認できます。 - -{/* TODO - 下のテーブルを自動生成する */ } - - -| 設定 | 概要 | デフォルト | 注記 | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | ネストされた JSON データをネストされたテーブルにマッピング(JSONEachRow フォーマットに対応)。 | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | JSON入力形式でブール値を数値として解析できるようにします。 | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | JSON入力形式でブール値を文字列として解析できるようにします。 | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | JSON入力形式で数値を文字列として解釈できるようにします。 | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | JSON入力フォーマットでJSON配列を文字列としてパースできるようにします。 | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | JSON入力フォーマットで、JSONオブジェクトを文字列として解析できるようにします。 | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | 名前付きタプル列を JSON オブジェクトとして解析します。 | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | スキーマ推論時に、文字列フィールドを数値として解釈できないか試みます。 | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | スキーマ推論時に、JSON オブジェクトから名前付きタプル型を推論しようと試みます。 | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | JSON 入力フォーマットでスキーマ推論を行う際、値が Null または空のオブジェクト/配列しかないキーには String 型を使用します。 | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | 名前付きタプルを解析する際、JSON オブジェクト内の不足している要素にデフォルト値を挿入します。 | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | 名前付きタプルでは、JSON オブジェクト内の未知のキーを無視します。 | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | JSONCompact/JSONCompactEachRow 形式で列数の可変を許可し、余分な列は無視し、存在しない列にはデフォルト値を使用します。 | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | JSON 文字列に不正なエスケープシーケンスが含まれている場合に例外をスローします。無効にした場合、不正なエスケープシーケンスはデータ内にそのまま残ります。 | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | JSON 入力内の空フィールドをデフォルト値として扱います。 | `false` | 複雑なデフォルト式を使用するには、[input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) も有効にしておく必要があります。 | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | JSON 出力形式における 64 ビット整数のクォート方法を制御します。 | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | JSON 出力形式における 64 ビット浮動小数点数の引用方法を制御します。 | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | JSON 出力形式で '+nan', '-nan', '+inf', '-inf' を出力できるようにします。 | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | JSON 出力形式における Decimal 型値のクオートを制御します。 | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | JSON 出力形式において、文字列出力中の正斜線(/)をエスケープするかどうかを制御します。 | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | 名前付きタプル型のカラムを JSON オブジェクトとしてシリアル化します。 | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | すべての行を JSONEachRow(Compact) 形式で JSON 配列として出力します。 | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | JSON 出力形式での UTF-8 シーケンス検証を有効にします | `false` | 形式 JSON/JSONCompact/JSONColumnsWithMetadata には影響しない点に注意してください。これらの形式では常に UTF-8 の検証が行われます。 | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md deleted file mode 100644 index a3ed834bfe5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -alias: [] -description: 'LineAsString フォーマットに関するドキュメント' -input_format: true -keywords: ['LineAsString'] -output_format: true -slug: /interfaces/formats/LineAsString -title: 'LineAsString' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -`LineAsString` フォーマットは、入力データの各行を 1 つの文字列値として解釈します。 -このフォーマットは、[String](/sql-reference/data-types/string.md) 型の単一フィールドだけを持つテーブルでのみ使用できます。 -残りのカラムは、[`DEFAULT`](/sql-reference/statements/create/table.md/#default)、[`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view) に設定するか、省略する必要があります。 - - - -## 使用例 {#example-usage} - -```sql title="Query" -DROP TABLE IF EXISTS line_as_string; -CREATE TABLE line_as_string (field String) ENGINE = Memory; -INSERT INTO line_as_string FORMAT LineAsString "私はリンゴが好きです", "私はバナナが好きです", "私はオレンジが好きです"; -SELECT * FROM line_as_string; -``` - -```text title="Response" -┌─field─────────────────────────────────────────────┐ -│ "I love apple", "I love banana", "I love orange"; │ -└───────────────────────────────────────────────────┘ -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md deleted file mode 100644 index 2575cd5c39b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -alias: [] -description: 'LineAsStringWithNames 形式に関するドキュメント' -input_format: true -keywords: ['LineAsStringWithNames'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNames -title: 'LineAsStringWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -`LineAsStringWithNames` フォーマットは、[`LineAsString`](./LineAsString.md) フォーマットに似ていますが、列名を含むヘッダー行を出力します。 - - - -## 使用例 {#example-usage} - -```sql title="Query" -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNames; -``` - -```response title="Response" -name value -John 30 -Jane 25 -Peter 35 -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md deleted file mode 100644 index c6a776acb80..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -alias: [] -description: 'LineAsStringWithNamesAndTypes 形式に関するドキュメント' -input_format: false -keywords: ['LineAsStringWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNamesAndTypes -title: 'LineAsStringWithNamesAndTypes' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -`LineAsStringWithNames` フォーマットは [`LineAsString`](./LineAsString.md) フォーマットに似ていますが、 -2 行のヘッダー行を出力します。1 行目は列名、2 行目は型です。 - - - -## 使用例 {#example-usage} - -```sql -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNamesAndTypes; -``` - -```response title="Response" -name value -String Int32 -John 30 -Jane 25 -Peter 35 -``` - - -## フォーマット設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md deleted file mode 100644 index 5e7e48c3b70..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: ['MD'] -description: 'Markdown 形式のドキュメント' -keywords: ['Markdown'] -slug: /interfaces/formats/Markdown -title: 'Markdown' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | `MD` | - - - -## 説明 {#description} - -結果を [Markdown](https://en.wikipedia.org/wiki/Markdown) 形式でエクスポートし、`.md` ファイルにそのまま貼り付けられる出力を生成できます。 - -Markdown 形式のテーブルは自動的に生成され、GitHub などの Markdown 対応プラットフォームで利用できます。この形式は出力専用です。 - - - -## 使用例 {#example-usage} - -```sql -SELECT - number, - number * 2 -FROM numbers(5) -FORMAT Markdown -``` - -```results -| number | multiply(number, 2) | -|-:|-:| -| 0 | 0 | -| 1 | 2 | -| 2 | 4 | -| 3 | 6 | -| 4 | 8 | -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md deleted file mode 100644 index 4caebd65c13..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'MsgPack フォーマットに関するドキュメント' -input_format: true -keywords: ['MsgPack'] -output_format: true -slug: /interfaces/formats/MsgPack -title: 'MsgPack' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -ClickHouse は [MessagePack](https://msgpack.org/) データファイルの読み書きをサポートしています。 - - - -## データ型の対応 {#data-types-matching} - -| MessagePack データ型(`INSERT`) | ClickHouse のデータ型 | MessagePack データ型(`SELECT`) | -|--------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|----------------------------------| -| `uint N`, `positive fixint` | [`UIntN`](/sql-reference/data-types/int-uint.md) | `uint N` | -| `int N`, `negative fixint` | [`IntN`](/sql-reference/data-types/int-uint.md) | `int N` | -| `bool` | [`UInt8`](/sql-reference/data-types/int-uint.md) | `uint 8` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`String`](/sql-reference/data-types/string.md) | `bin 8`, `bin 16`, `bin 32` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`FixedString`](/sql-reference/data-types/fixedstring.md) | `bin 8`, `bin 16`, `bin 32` | -| `float 32` | [`Float32`](/sql-reference/data-types/float.md) | `float 32` | -| `float 64` | [`Float64`](/sql-reference/data-types/float.md) | `float 64` | -| `uint 16` | [`Date`](/sql-reference/data-types/date.md) | `uint 16` | -| `int 32` | [`Date32`](/sql-reference/data-types/date32.md) | `int 32` | -| `uint 32` | [`DateTime`](/sql-reference/data-types/datetime.md) | `uint 32` | -| `uint 64` | [`DateTime64`](/sql-reference/data-types/datetime.md) | `uint 64` | -| `fixarray`, `array 16`, `array 32` | [`Array`](/sql-reference/data-types/array.md)/[`Tuple`](/sql-reference/data-types/tuple.md) | `fixarray`, `array 16`, `array 32` | -| `fixmap`, `map 16`, `map 32` | [`Map`](/sql-reference/data-types/map.md) | `fixmap`, `map 16`, `map 32` | -| `uint 32` | [`IPv4`](/sql-reference/data-types/ipv4.md) | `uint 32` | -| `bin 8` | [`String`](/sql-reference/data-types/string.md) | `bin 8` | -| `int 8` | [`Enum8`](/sql-reference/data-types/enum.md) | `int 8` | -| `bin 8` | [`(U)Int128`/`(U)Int256`](/sql-reference/data-types/int-uint.md) | `bin 8` | -| `int 32` | [`Decimal32`](/sql-reference/data-types/decimal.md) | `int 32` | -| `int 64` | [`Decimal64`](/sql-reference/data-types/decimal.md) | `int 64` | -| `bin 8` | [`Decimal128`/`Decimal256`](/sql-reference/data-types/decimal.md) | `bin 8` | - - - -## 使用例 {#example-usage} - -「.msgpk」ファイルへの書き込み: - -```sql -$ clickhouse-client --query="CREATE TABLE msgpack (array Array(UInt8)) ENGINE = Memory;" -$ clickhouse-client --query="INSERT INTO msgpack VALUES ([0, 1, 2, 3, 42, 253, 254, 255]), ([255, 254, 253, 42, 3, 2, 1, 0])"; -$ clickhouse-client --query="SELECT * FROM msgpack FORMAT MsgPack" > tmp_msgpack.msgpk; -``` - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | 既定値 | -|--------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|---------| -| [`input_format_msgpack_number_of_columns`](/operations/settings/settings-formats.md/#input_format_msgpack_number_of_columns) | 挿入される MsgPack データの列数。データからの自動スキーマ推論に使用されます。 | `0` | -| [`output_format_msgpack_uuid_representation`](/operations/settings/settings-formats.md/#output_format_msgpack_uuid_representation) | MsgPack 形式における UUID の出力方法を指定します。 | `EXT` | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md deleted file mode 100644 index feea1452ead..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -alias: [] -description: 'MySQLDump フォーマットに関するドキュメント' -input_format: true -keywords: ['MySQLDump'] -output_format: false -slug: /interfaces/formats/MySQLDump -title: 'MySQLDump' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|---------|-------| -| ✔ | ✗ | | - - - -## 説明 {#description} - -ClickHouse は MySQL の [ダンプ](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) の読み取りをサポートしています。 - -ダンプ内で、1 つのテーブルに対応する `INSERT` クエリからすべてのデータを読み取ります。 -テーブルが複数ある場合、既定では最初のテーブルからデータを読み取ります。 - -:::note -この形式はスキーマ推論をサポートします。ダンプに指定されたテーブルに対する `CREATE` クエリが含まれている場合は、そのクエリからテーブル構造を推論し、含まれていない場合は `INSERT` クエリのデータからスキーマを推論します。 -::: - - - -## 使用例 {#example-usage} - -次の SQL ダンプファイルがあるとします。 - -```sql title="dump.sql" -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test` ( - `x` int DEFAULT NULL, - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test` VALUES (1,NULL),(2,NULL),(3,NULL),(3,NULL),(4,NULL),(5,NULL),(6,7); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test 3` ( - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test 3` VALUES (1); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test2` ( - `x` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test2` VALUES (1),(2),(3); -``` - -次のクエリを実行します。 - -```sql title="Query" -DESCRIBE TABLE file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─name─┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ x │ Nullable(Int32) │ │ │ │ │ │ -└──────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql title="Query" -SELECT * -FROM file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─x─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - - -## フォーマット設定 {#format-settings} - -[`input_format_mysql_dump_table_name`](/operations/settings/settings-formats.md/#input_format_mysql_dump_table_name) 設定を使用して、データの読み取り元となるテーブル名を指定できます。 -`input_format_mysql_dump_map_columns` 設定が `1` に設定されており、ダンプに指定したテーブルの `CREATE` クエリ、または `INSERT` クエリ内でカラム名が指定されている場合、入力データのカラムは名前に基づいてテーブルのカラムにマッピングされます。 -[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が `1` に設定されている場合、不明な名前のカラムはスキップされます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md deleted file mode 100644 index 8e134ce6c43..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'MySQLWire 形式のドキュメント' -keywords: ['MySQLWire'] -slug: /interfaces/formats/MySQLWire -title: 'MySQLWire' -doc_type: 'reference' ---- - - - -## 概要 {#description} - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md deleted file mode 100644 index 621d4c01424..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -alias: [] -description: 'Native フォーマットのドキュメント' -input_format: true -keywords: ['Native'] -output_format: true -slug: /interfaces/formats/Native -title: 'Native' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -`Native` フォーマットは、カラムを行に変換しないという意味で真の「カラム指向」であり、ClickHouse において最も効率的なフォーマットです。 - -このフォーマットでは、データはバイナリ形式で [ブロック](/development/architecture#block) ごとに書き込みおよび読み取りが行われます。各ブロックについて、行数、カラム数、カラム名と型、およびブロック内の各カラムのデータが順番に記録されます。 - -このフォーマットは、サーバー間のやり取りに使われるネイティブインターフェイス、コマンドラインクライアント、および C++ クライアントで使用されます。 - -:::tip -このフォーマットを使用して、ClickHouse DBMS でのみ読み取ることができるダンプを高速に生成できます。 -ただし、このフォーマットを手作業で扱うのは現実的ではない場合があります。 -::: - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md deleted file mode 100644 index b9de218c80a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -alias: [] -description: 'Npy 形式に関するドキュメント' -input_format: true -keywords: ['Npy'] -output_format: true -slug: /interfaces/formats/Npy -title: 'Npy' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -`Npy` 形式は、`.npy` ファイルから NumPy 配列を ClickHouse に読み込むために設計されています。 -NumPy のファイル形式は、数値データの配列を効率的に保存するために使用されるバイナリ形式です。 -インポート時、ClickHouse は最上位の次元を、単一列を持つ行の配列として扱います。 - -下表は、サポートされている Npy データ型と、それに対応する ClickHouse の型を示します。 - - - -## データ型の対応 {#data_types-matching} - -| Npy データ型(`INSERT`) | ClickHouse データ型 | Npy データ型(`SELECT`) | -|--------------------------|-----------------------------------------------------------------|-------------------------| -| `i1` | [Int8](/sql-reference/data-types/int-uint.md) | `i1` | -| `i2` | [Int16](/sql-reference/data-types/int-uint.md) | `i2` | -| `i4` | [Int32](/sql-reference/data-types/int-uint.md) | `i4` | -| `i8` | [Int64](/sql-reference/data-types/int-uint.md) | `i8` | -| `u1`, `b1` | [UInt8](/sql-reference/data-types/int-uint.md) | `u1` | -| `u2` | [UInt16](/sql-reference/data-types/int-uint.md) | `u2` | -| `u4` | [UInt32](/sql-reference/data-types/int-uint.md) | `u4` | -| `u8` | [UInt64](/sql-reference/data-types/int-uint.md) | `u8` | -| `f2`, `f4` | [Float32](/sql-reference/data-types/float.md) | `f4` | -| `f8` | [Float64](/sql-reference/data-types/float.md) | `f8` | -| `S`, `U` | [String](/sql-reference/data-types/string.md) | `S` | -| | [FixedString](/sql-reference/data-types/fixedstring.md) | `S` | - - - -## 使用例 {#example-usage} - -### Python を使って配列を .npy 形式で保存する {#saving-an-array-in-npy-format-using-python} - -```Python -import numpy as np -arr = np.array([[[1],[2],[3]],[[4],[5],[6]]]) -np.save('example_array.npy', arr) -``` - -### ClickHouse で NumPy ファイルを読み込む {#reading-a-numpy-file-in-clickhouse} - -```sql title="Query" -SELECT * -FROM file('example_array.npy', Npy) -``` - -```response title="Response" -┌─array─────────┐ -│ [[1],[2],[3]] │ -│ [[4],[5],[6]] │ -└───────────────┘ -``` - -### データの選択 {#selecting-data} - -`clickhouse-client` で次のコマンドを実行すると、ClickHouse のテーブルからデータを抽出し、Npy 形式のファイルとして保存できます。 - -```bash -$ clickhouse-client --query="SELECT {column} FROM {some_table} FORMAT Npy" > {filename.npy} -``` - - -## 書式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md deleted file mode 100644 index dba8e60b819..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -alias: [] -description: 'Null フォーマットのドキュメント' -input_format: false -keywords: ['Null', 'format'] -output_format: true -slug: /interfaces/formats/Null -title: 'Null' -doc_type: 'reference' ---- - -| Input | Output | 別名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -`Null` フォーマットでは、何も出力されません。 -一見すると奇妙に思えるかもしれませんが、何も出力されない場合でもクエリ自体は処理されており、 -コマンドラインクライアントを使用している場合は、データがクライアントに送信される点に注意してください。 - -:::tip -`Null` フォーマットは、パフォーマンス測定や性能テストに役立ちます。 -::: - - - -## 使用例 {#example-usage} - -### データの読み取り {#reading-data} - -次のデータを含むテーブル `football` を例にします。 - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -`Null` フォーマットを使用してデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT Null -``` - -このクエリはデータを処理しますが、何も出力しません。 - -```response -0行のセット。経過時間: 0.154秒 -``` - - -## フォーマット設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md deleted file mode 100644 index ec9b045f243..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'ODBCDriver2 形式に関するドキュメント' -keywords: ['ODBCDriver2'] -slug: /interfaces/formats/ODBCDriver2 -title: 'ODBCDriver2' -doc_type: 'reference' ---- - - - -## 概要 {#description} - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md deleted file mode 100644 index 6d905be6edc..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -alias: [] -description: 'ORC 形式のドキュメント' -input_format: true -keywords: ['ORC'] -output_format: true -slug: /interfaces/formats/ORC -title: 'ORC' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -[Apache ORC](https://orc.apache.org/) は、[Hadoop](https://hadoop.apache.org/) エコシステムで広く使用されている列指向ストレージ形式です。 - - - -## データ型の対応関係 {#data-types-matching-orc} - -次の表は、`INSERT` および `SELECT` クエリにおいてサポートされる ORC データ型と、それに対応する ClickHouse の [データ型](/sql-reference/data-types/index.md) を比較したものです。 - -| ORC data type (`INSERT`) | ClickHouse data type | ORC data type (`SELECT`) | -|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------------------| -| `Boolean` | [UInt8](/sql-reference/data-types/int-uint.md) | `Boolean` | -| `Tinyint` | [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `Tinyint` | -| `Smallint` | [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `Smallint` | -| `Int` | [Int32/UInt32](/sql-reference/data-types/int-uint.md) | `Int` | -| `Bigint` | [Int64/UInt32](/sql-reference/data-types/int-uint.md) | `Bigint` | -| `Float` | [Float32](/sql-reference/data-types/float.md) | `Float` | -| `Double` | [Float64](/sql-reference/data-types/float.md) | `Double` | -| `Decimal` | [Decimal](/sql-reference/data-types/decimal.md) | `Decimal` | -| `Date` | [Date32](/sql-reference/data-types/date32.md) | `Date` | -| `Timestamp` | [DateTime64](/sql-reference/data-types/datetime64.md) | `Timestamp` | -| `String`, `Char`, `Varchar`, `Binary` | [String](/sql-reference/data-types/string.md) | `Binary` | -| `List` | [Array](/sql-reference/data-types/array.md) | `List` | -| `Struct` | [Tuple](/sql-reference/data-types/tuple.md) | `Struct` | -| `Map` | [Map](/sql-reference/data-types/map.md) | `Map` | -| `Int` | [IPv4](/sql-reference/data-types/int-uint.md) | `Int` | -| `Binary` | [IPv6](/sql-reference/data-types/ipv6.md) | `Binary` | -| `Binary` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `Binary` | -| `Binary` | [Decimal256](/sql-reference/data-types/decimal.md) | `Binary` | - -- 上記以外の型はサポートされていません。 -- 配列はネスト可能であり、要素として `Nullable` 型の値を取ることができます。`Tuple` および `Map` 型もネスト可能です。 -- ClickHouse テーブルの列のデータ型は、対応する ORC データフィールドと一致している必要はありません。データを挿入する際、ClickHouse は上記の表に従ってデータ型を解釈し、その後 ClickHouse テーブルの列に設定されているデータ型へデータを[キャスト](/sql-reference/functions/type-conversion-functions#cast)します。 - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータが入った ORC ファイル(名前は `football.orc`)を使用します。 - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -データを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.orc' FORMAT ORC; -``` - -### データの読み込み {#reading-data} - -`ORC` 形式でデータを読み込みます: - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.orc' -FORMAT ORC -``` - -:::tip -ORC はバイナリ形式のため、ターミナル上で人間が読める形で表示することはできません。`INTO OUTFILE` 句を使用して ORC ファイルとして出力してください。 -::: - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | デフォルト | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|------------| -| [`output_format_arrow_string_as_string`](/operations/settings/settings-formats.md/#output_format_arrow_string_as_string) | String 列に対して Binary 型ではなく Arrow の String 型を使用します。 | `false` | -| [`output_format_orc_compression_method`](/operations/settings/settings-formats.md/#output_format_orc_compression_method) | 出力 ORC フォーマットで使用される圧縮方式を指定します。 | `none` | -| [`input_format_arrow_case_insensitive_column_matching`](/operations/settings/settings-formats.md/#input_format_arrow_case_insensitive_column_matching) | Arrow の列を ClickHouse の列に対応付ける際に大文字小文字を区別しません。 | `false` | -| [`input_format_arrow_allow_missing_columns`](/operations/settings/settings-formats.md/#input_format_arrow_allow_missing_columns) | Arrow データの読み取り時に、欠落している列を許可します。 | `false` | -| [`input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference) | Arrow フォーマットのスキーマ推論時に、未サポート型を持つ列をスキップすることを許可します。 | `false` | - -Hadoop とデータをやり取りするには、[HDFS テーブルエンジン](/engines/table-engines/integrations/hdfs.md) を使用できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/One.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/One.md deleted file mode 100644 index 0e8f1400194..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/One.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -alias: [] -description: 'One フォーマットに関するドキュメント' -input_format: true -keywords: ['One'] -output_format: false -slug: /interfaces/formats/One -title: 'One' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 説明 {#description} - -`One` フォーマットは、ファイルから一切データを読み込まず、[`UInt8`](../../sql-reference/data-types/int-uint.md) 型の `dummy` という名前のカラムを 1 列だけ持つ 1 行(値は `0`)だけを返す、特別な入力フォーマットです(`system.one` テーブルと同様)。 -仮想カラム `_file/_path` と組み合わせることで、実際のデータを読み込まずにすべてのファイルを一覧表示するために使用できます。 - - - -## 使用例 {#example-usage} - -例: - -```sql title="Query" -SELECT _file FROM file('path/to/files/data*', One); -``` - -```text title="Response" -┌─_file────┐ -│ data.csv │ -└──────────┘ -┌─_file──────┐ -│ data.jsonl │ -└────────────┘ -┌─_file────┐ -│ data.tsv │ -└──────────┘ -┌─_file────────┐ -│ data.parquet │ -└──────────────┘ -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md deleted file mode 100644 index bbad1b51cf2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -alias: [] -description: 'Parquet 形式に関するドキュメント' -input_format: true -keywords: ['Parquet'] -output_format: true -slug: /interfaces/formats/Parquet -title: 'Parquet' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -[Apache Parquet](https://parquet.apache.org/) は、Hadoop エコシステムで広く利用されているカラムナストレージフォーマットです。ClickHouse は、このフォーマットの読み書きをサポートしています。 - - - -## データ型の対応 {#data-types-matching-parquet} - -以下の表は、Parquet のデータ型が ClickHouse の[データ型](/sql-reference/data-types/index.md)にどのように対応するかを示します。 - -| Parquet type (logical, converted, or physical) | ClickHouse data type | -|------------------------------------------------|----------------------| -| `BOOLEAN` | [Bool](/sql-reference/data-types/boolean.md) | -| `UINT_8` | [UInt8](/sql-reference/data-types/int-uint.md) | -| `INT_8` | [Int8](/sql-reference/data-types/int-uint.md) | -| `UINT_16` | [UInt16](/sql-reference/data-types/int-uint.md) | -| `INT_16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | -| `UINT_32` | [UInt32](/sql-reference/data-types/int-uint.md) | -| `INT_32` | [Int32](/sql-reference/data-types/int-uint.md) | -| `UINT_64` | [UInt64](/sql-reference/data-types/int-uint.md) | -| `INT_64` | [Int64](/sql-reference/data-types/int-uint.md) | -| `DATE` | [Date32](/sql-reference/data-types/date.md) | -| `TIMESTAMP`, `TIME` | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `FLOAT` | [Float32](/sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | -| `INT96` | [DateTime64(9, 'UTC')](/sql-reference/data-types/datetime64.md) | -| `BYTE_ARRAY`, `UTF8`, `ENUM`, `BSON` | [String](/sql-reference/data-types/string.md) | -| `JSON` | [JSON](/sql-reference/data-types/newjson.md) | -| `FIXED_LEN_BYTE_ARRAY` | [FixedString](/sql-reference/data-types/fixedstring.md) | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | -| `LIST` | [Array](/sql-reference/data-types/array.md) | -| `MAP` | [Map](/sql-reference/data-types/map.md) | -| struct | [Tuple](/sql-reference/data-types/tuple.md) | -| `FLOAT16` | [Float32](/sql-reference/data-types/float.md) | -| `UUID` | [FixedString(16)](/sql-reference/data-types/fixedstring.md) | -| `INTERVAL` | [FixedString(12)](/sql-reference/data-types/fixedstring.md) | - -Parquet ファイルを書き出す際、対応する Parquet 型が存在しないデータ型は、利用可能な最も近い型に変換されます。 - -| ClickHouse data type | Parquet type | -|----------------------|--------------| -| [IPv4](/sql-reference/data-types/ipv4.md) | `UINT_32` | -| [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_LEN_BYTE_ARRAY` (16 bytes) | -| [Date](/sql-reference/data-types/date.md) (16 bits) | `DATE` (32 bits) | -| [DateTime](/sql-reference/data-types/datetime.md) (32 bits, seconds) | `TIMESTAMP` (64 bits, milliseconds) | -| [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_LEN_BYTE_ARRAY` (16/32 bytes, little-endian) | - -`Array` は入れ子にでき、引数として `Nullable` 型の値を持つことができます。`Tuple` 型および `Map` 型も入れ子にできます。 - -ClickHouse テーブルのカラムのデータ型は、挿入される Parquet データ内の対応するフィールドの型と異なる場合があります。データ挿入時、ClickHouse は上記の表に従ってデータ型を解釈し、その後 ClickHouse テーブルのカラムに設定されているデータ型へ[キャスト](/sql-reference/functions/type-conversion-functions#cast)します。たとえば、`UINT_32` の Parquet カラムは [IPv4](/sql-reference/data-types/ipv4.md) 型の ClickHouse カラムとして読み取ることができます。 - - - -一部の Parquet 型には、対応する ClickHouse 型が存在しません。それらは次のように読み取られます。 -* `TIME`(時刻)は `timestamp` として読み取られます。例: `10:23:13.000` は `1970-01-01 10:23:13.000` になります。 -* `isAdjustedToUTC=false` の `TIMESTAMP`/`TIME` はローカルのウォールクロック時刻(どのタイムゾーンをローカルとみなすかにかかわらず、ローカルタイムゾーンにおける年・月・日・時・分・秒およびサブ秒フィールド)であり、SQL の `TIMESTAMP WITHOUT TIME ZONE` と同じです。ClickHouse はこれを、あたかも UTC の `timestamp` であるかのように読み取ります。例: `2025-09-29 18:42:13.000`(ローカルの時計の読み値を表す)は `2025-09-29 18:42:13.000`(ある時点を表す `DateTime64(3, 'UTC')`)になります。String に変換すると、年・月・日・時・分・秒およびサブ秒は正しい値として表示され、それを UTC ではなく何らかのローカルタイムゾーンの時刻として解釈できます。直感に反して、型を `DateTime64(3, 'UTC')` から `DateTime64(3)` に変更しても状況は改善しません。どちらの型も時計の読み値ではなくある時点を表すためですが、`DateTime64(3)` はローカルタイムゾーンを用いて誤ってフォーマットされてしまいます。 -* `INTERVAL` は現在、Parquet ファイル内でエンコードされているとおりの時間間隔の生のバイナリ表現を持つ `FixedString(12)` として読み取られます。 - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次のデータを含む `football.parquet` という名前の Parquet ファイルを使用します: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.parquet' FORMAT Parquet; -``` - -### データの読み込み {#reading-data} - -`Parquet` 形式でデータを読み込みます。 - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.parquet' -FORMAT Parquet -``` - -:::tip -Parquet はバイナリ形式であり、ターミナル上では人間が読める形で表示されません。Parquet ファイルを出力するには `INTO OUTFILE` を使用します。 -::: - -Hadoop とデータを交換するには、[`HDFS table engine`](/engines/table-engines/integrations/hdfs.md) を使用できます。 - - -## フォーマット設定 {#format-settings} - - - -| 設定 | 概要 | デフォルト | -| ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `input_format_parquet_case_insensitive_column_matching` | Parquet の列と CH の列を照合する際に、大文字と小文字を区別しないようにします。 | `0` | -| `input_format_parquet_preserve_order` | Parquet ファイルを読み取る際に行の並べ替えは行わないでください。通常、処理が大幅に遅くなります。 | `0` | -| `input_format_parquet_filter_push_down` | Parquet ファイルを読み込む際、WHERE/PREWHERE 句と Parquet メタデータ内の最小値/最大値の統計量に基づいて、行グループ全体をスキップします。 | `1` | -| `input_format_parquet_bloom_filter_push_down` | Parquet ファイルを読み取る際、WHERE 句の条件式と Parquet メタデータ内のブルームフィルターに基づいて、行グループ全体をスキップします。 | `0` | -| `input_format_parquet_use_native_reader` | Parquet ファイルの読み込み時に、Arrow リーダーではなくネイティブリーダーを使用します。 | `0` | -| `input_format_parquet_allow_missing_columns` | Parquet 入力フォーマット読み込み時に欠落列を許可する | `1` | -| `input_format_parquet_local_file_min_bytes_for_seek` | Parquet 入力形式で、無視しながら読み進めるのではなくシークを行うために必要なローカル読み取り(ファイル)の最小バイト数 | `8192` | -| `input_format_parquet_enable_row_group_prefetch` | Parquet のパース中に行グループのプリフェッチを有効にします。現在は、単一スレッドでのパース時にのみプリフェッチが可能です。 | `1` | -| `input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference` | Parquet 形式でのスキーマ推論時に、未サポートの型の列をスキップする | `0` | -| `input_format_parquet_max_block_size` | Parquet リーダーにおけるブロックサイズの最大値。 | `65409` | -| `input_format_parquet_prefer_block_bytes` | Parquet リーダーの出力ブロックの平均バイト数 | `16744704` | -| `input_format_parquet_enable_json_parsing` | Parquet ファイルを読み込む際は、JSON 列を ClickHouse の JSON カラムとしてパースします。 | `1` | -| `output_format_parquet_row_group_size` | 行数で指定する行グループの目標サイズ。 | `1000000` | -| `output_format_parquet_row_group_size_bytes` | 圧縮前の行グループの目標サイズ(バイト単位)。 | `536870912` | -| `output_format_parquet_string_as_string` | String 列には Binary ではなく Parquet の String 型を使用してください。 | `1` | -| `output_format_parquet_fixed_string_as_fixed_byte_array` | FixedString 列には Binary 型ではなく、Parquet の FIXED_LEN_BYTE_ARRAY 型を使用してください。 | `1` | -| `output_format_parquet_version` | 出力フォーマットに使用する Parquet フォーマットのバージョン。サポートされているバージョン: 1.0、2.4、2.6、および 2.latest(デフォルト)。 | `2.latest` | -| `output_format_parquet_compression_method` | Parquet 出力フォーマットの圧縮方式。サポートされるコーデック:snappy、lz4、brotli、zstd、gzip、none(非圧縮) | `zstd` | -| `output_format_parquet_compliant_nested_types` | Parquet ファイルのスキーマでは、リスト要素には 'item' ではなく 'element' という名前を使用します。これは Arrow ライブラリの実装に起因する歴史的な経緯です。一般的には互換性が向上しますが、一部の古いバージョンの Arrow とは互換性がない可能性があります。 | `1` | -| `output_format_parquet_use_custom_encoder` | より高速な Parquet エンコーダー実装を使用する。 | `1` | -| `output_format_parquet_parallel_encoding` | 複数スレッドで Parquet エンコードを行います。output_format_parquet_use_custom_encoder を有効にする必要があります。 | `1` | -| `output_format_parquet_data_page_size` | 圧縮前のターゲットページのサイズ(バイト単位)。 | `1048576` | -| `output_format_parquet_batch_size` | この行数ごとにページサイズをチェックします。値の平均サイズが数KBを超える列がある場合は、この値を小さくすることを検討してください。 | `1024` | -| `output_format_parquet_write_page_index` | Parquet ファイルにページインデックスを書き込む機能を追加します。 | `1` | -| `input_format_parquet_import_nested` | この設定は廃止されており、指定しても何の効果もありません。 | `0` | -| `input_format_parquet_local_time_as_utc` | true | isAdjustedToUTC=false の Parquet タイムスタンプに対して、スキーマ推論時に使用されるデータ型を決定します。true の場合は DateTime64(..., 'UTC')、false の場合は DateTime64(...) になります。ClickHouse にはローカルの壁時計時刻を表すデータ型がないため、どちらの動作も完全には正しくありません。直感に反して、'true' の方がまだ誤りが少ない選択肢と考えられます。これは、'UTC' タイムスタンプを String としてフォーマットすると、正しいローカル時刻を表す文字列表現が得られるためです。 | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md deleted file mode 100644 index f4ff2c9f6ff..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -description: 'ParquetMetadata 形式のドキュメント' -keywords: ['ParquetMetadata'] -slug: /interfaces/formats/ParquetMetadata -title: 'ParquetMetadata' -doc_type: 'reference' ---- - - - -## 説明 {#description} - -Parquet ファイルメタデータ (https://parquet.apache.org/docs/file-format/metadata/) を読み取るための特別なフォーマットです。常に次の構造/内容を持つ 1 行を出力します: -- `num_columns` - 列数 -- `num_rows` - 行の総数 -- `num_row_groups` - 行グループの総数 -- `format_version` - Parquet フォーマットバージョン。常に 1.0 または 2.6 -- `total_uncompressed_size` - すべての行グループの total_byte_size の合計として計算される、データの非圧縮バイトサイズの総量 -- `total_compressed_size` - すべての行グループの total_compressed_size の合計として計算される、データの圧縮バイトサイズの総量 -- `columns` - 次の構造を持つ列メタデータのリスト: - - `name` - 列名 - - `path` - 列パス (ネストされた列の場合は name と異なります) - - `max_definition_level` - 最大定義レベル - - `max_repetition_level` - 最大反復レベル - - `physical_type` - 列の物理型 - - `logical_type` - 列の論理型 - - `compression` - この列で使用される圧縮方式 - - `total_uncompressed_size` - すべての行グループにおける当該列の total_uncompressed_size の合計として計算される、列の非圧縮バイトサイズの総量 - - `total_compressed_size` - すべての行グループにおける当該列の total_compressed_size の合計として計算される、列の圧縮バイトサイズの総量 - - `space_saved` - 圧縮によって節約された容量の割合。(1 - total_compressed_size/total_uncompressed_size) として計算されます - - `encodings` - この列で使用されるエンコーディングのリスト -- `row_groups` - 次の構造を持つ行グループメタデータのリスト: - - `num_columns` - 行グループ内の列数 - - `num_rows` - 行グループ内の行数 - - `total_uncompressed_size` - 行グループの非圧縮バイトサイズの総量 - - `total_compressed_size` - 行グループの圧縮バイトサイズの総量 - - `columns` - 次の構造を持つカラムチャンクメタデータのリスト: - - `name` - 列名 - - `path` - 列パス - - `total_compressed_size` - 列の圧縮バイトサイズの総量 - - `total_uncompressed_size` - 行グループの非圧縮バイトサイズの総量 - - `have_statistics` - カラムチャンクメタデータに列統計が含まれるかどうかを示すブールフラグ - - `statistics` - カラムチャンク統計 (have_statistics = false の場合、すべてのフィールドは NULL) で、次の構造を持ちます: - - `num_values` - カラムチャンク内の非 NULL 値の数 - - `null_count` - カラムチャンク内の NULL 値の数 - - `distinct_count` - カラムチャンク内の異なる値の数 - - `min` - カラムチャンクの最小値 - - `max` - カラムチャンクの最大値 - - - -## 使用例 {#example-usage} - -例: - -```sql -SELECT * -FROM file(data.parquet, ParquetMetadata) -FORMAT PrettyJSONEachRow -``` - -```json -{ - "num_columns": "2", - "num_rows": "100000", - "num_row_groups": "2", - "format_version": "2.6", - "metadata_size": "577", - "total_uncompressed_size": "282436", - "total_compressed_size": "26633", - "columns": [ - { - "name": "number", - "path": "number", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "INT32", - "logical_type": "Int(bitWidth=16, isSigned=false)", - "compression": "LZ4", - "total_uncompressed_size": "133321", - "total_compressed_size": "13293", - "space_saved": "90.03%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "BYTE_ARRAY", - "logical_type": "None", - "compression": "LZ4", - "total_uncompressed_size": "149115", - "total_compressed_size": "13340", - "space_saved": "91.05%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - } - ], - "row_groups": [ - { - "num_columns": "2", - "num_rows": "65409", - "total_uncompressed_size": "179809", - "total_compressed_size": "14163", - "columns": [ - { - "name": "number", - "path": "number", - "total_compressed_size": "7070", - "total_uncompressed_size": "85956", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "0", - "max": "999" - } - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "total_compressed_size": "7093", - "total_uncompressed_size": "93853", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "Hello0", - "max": "Hello999" - } - } - ] - }, - ... - ] -} -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md deleted file mode 100644 index 39d62a8360a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'PostgreSQLWire 形式に関するドキュメント' -keywords: ['PostgreSQLWire'] -slug: /interfaces/formats/PostgreSQLWire -title: 'PostgreSQLWire' -doc_type: 'reference' ---- - - - -## 概要 {#description} - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md deleted file mode 100644 index f851d9b90f9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -alias: [] -description: 'Pretty 形式に関するドキュメント' -input_format: false -keywords: ['Pretty'] -output_format: true -slug: /interfaces/formats/Pretty -title: 'Pretty' -doc_type: 'リファレンス' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -`Pretty` フォーマットは、Unicode アートによるテーブルとしてデータを出力し、 -ターミナルで色を表示するために ANSI エスケープシーケンスを使用します。 -テーブルは完全なグリッドとして描画され、各行はターミナル上で 2 行分を占有します。 -各結果ブロックは個別のテーブルとして出力されます。 -これは、結果をバッファリングせずにブロックを出力できるようにするためです(すべての値の見た目上の幅を事前計算するにはバッファリングが必要になります)。 - -[NULL](/sql-reference/syntax.md) は `ᴺᵁᴸᴸ` として出力されます。 - - - -## 使用例 {#example-usage} - -例([`PrettyCompact`](./PrettyCompact.md) 形式の場合): - -```sql title="Query" -SELECT * FROM t_null -``` - -```response title="Response" -┌─x─┬────y─┐ -│ 1 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -`Pretty` 系のいずれのフォーマットでも、行はエスケープされません。次の例は、[`PrettyCompact`](./PrettyCompact.md) フォーマットの場合を示しています。 - -```sql title="Query" -SELECT 'String with \'quotes\' and \t character' AS Escaping_test -``` - -```response title="Response" -┌─Escaping_test────────────────────────┐ -│ String with 'quotes' and character │ -└──────────────────────────────────────┘ -``` - -ターミナルへの過剰なデータ出力を避けるため、最初の `10,000` 行のみが表示されます。 -行数が `10,000` 以上の場合は、メッセージ "Showed first 10 000" が出力されます。 - -:::note -このフォーマットはクエリ結果を出力する用途にのみ適しており、データのパースには適していません。 -::: - -Pretty フォーマットは、合計値(`WITH TOTALS` を使用する場合)および極値('extremes' が 1 に設定されている場合)の出力をサポートします。 -この場合、合計値と極値はメインデータの後に、別々のテーブルとして出力されます。 -これは、[`PrettyCompact`](./PrettyCompact.md) フォーマットを使用した次の例で示されています。 - -```sql title="Query" -SELECT EventDate, count() AS c -FROM test.hits -GROUP BY EventDate -WITH TOTALS -ORDER BY EventDate -FORMAT PrettyCompact -``` - -```response title="Response" -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1406958 │ -│ 2014-03-18 │ 1383658 │ -│ 2014-03-19 │ 1405797 │ -│ 2014-03-20 │ 1353623 │ -│ 2014-03-21 │ 1245779 │ -│ 2014-03-22 │ 1031592 │ -│ 2014-03-23 │ 1046491 │ -└────────────┴─────────┘ - -合計: -┌──EventDate─┬───────c─┐ -│ 1970-01-01 │ 8873898 │ -└────────────┴─────────┘ - -極値: -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1031592 │ -│ 2014-03-23 │ 1406958 │ -└────────────┴─────────┘ -``` - - -## 書式設定 {#format-settings} - - diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md deleted file mode 100644 index b5649023c35..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -alias: [] -description: 'PrettyCompact 形式に関するドキュメント' -input_format: false -keywords: ['PrettyCompact'] -output_format: true -slug: /interfaces/formats/PrettyCompact -title: 'PrettyCompact' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | 別名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`Pretty`](./Pretty.md) フォーマットとは異なり、行と行の間に罫線を表示してテーブルを描画します。 -そのため、結果はより省スペースな表示になります。 - -:::note -このフォーマットは、コマンドラインクライアントのインタラクティブモードで既定で使用されます。 -::: - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md deleted file mode 100644 index b31e4753ea4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactMonoBlock フォーマットに関するドキュメント' -input_format: false -keywords: ['PrettyCompactMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactMonoBlock -title: 'PrettyCompactMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettyCompact`](./PrettyCompact.md) 形式とは異なり、最大で `10,000` 行をバッファリングしてから 1 つのテーブルとしてまとめて出力し、[ブロック](/development/architecture#block) 単位では出力しません。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md deleted file mode 100644 index 7c006df8aa7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactNoEscapes 形式に関するドキュメント' -input_format: false -keywords: ['PrettyCompactNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapes -title: 'PrettyCompactNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettyCompact`](./PrettyCompact.md) 形式とは、[ANSI エスケープシーケンス](http://en.wikipedia.org/wiki/ANSI_escape_code) を使用しない点が異なります。 -これは、ブラウザでこの形式を表示する場合や、`watch` コマンドラインユーティリティで使用する場合に必要です。 - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} - - diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md deleted file mode 100644 index 653b2a6193b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactNoEscapesMonoBlock フォーマットに関するドキュメント' -input_format: false -keywords: ['PrettyCompactNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapesMonoBlock -title: 'PrettyCompactNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettyCompactNoEscapes`](./PrettyCompactNoEscapes.md) フォーマットとは異なり、最大 `10,000` 行をバッファリングし、その後 1 つのテーブルとして出力し、[ブロック](/development/architecture#block)単位では出力しません。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md deleted file mode 100644 index b056e5fb855..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettyMonoBlock 形式のドキュメント' -input_format: false -keywords: ['PrettyMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyMonoBlock -title: 'PrettyMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`Pretty`](/interfaces/formats/Pretty) フォーマットとは異なり、最大 `10,000` 行をバッファリングしてから 1 つのテーブルとしてまとめて出力し、[ブロック](/development/architecture#block) ごとには出力しません。 - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md deleted file mode 100644 index a510183028a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -alias: [] -description: 'PrettyNoEscapes フォーマットに関するドキュメント' -input_format: false -keywords: ['PrettyNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapes -title: 'PrettyNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | 別名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[Pretty](/interfaces/formats/Pretty) とは異なり、[ANSI エスケープシーケンス](http://en.wikipedia.org/wiki/ANSI_escape_code) を使用しません。 -これは、この形式をブラウザで表示したり、`watch` コマンドラインユーティリティで使用したりするために必要です。 - - - -## 使用例 {#example-usage} - -例: - -```bash -$ watch -n1 "clickhouse-client --query='SELECT event, value FROM system.events FORMAT PrettyCompactNoEscapes'" -``` - -:::note -[HTTP インターフェイス](../../../interfaces/http.md)を使用して、この形式をブラウザで表示できます。 -::: - - -## 書式設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md deleted file mode 100644 index fe2a0b68f19..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettyNoEscapesMonoBlock フォーマットに関するドキュメント' -input_format: false -keywords: ['PrettyNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapesMonoBlock -title: 'PrettyNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettyNoEscapes`](./PrettyNoEscapes.md) 形式とは異なり、最大で `10,000` 行をバッファリングしてから 1 つのテーブルとしてまとめて出力し、ブロックごとには出力しません。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md deleted file mode 100644 index d60cc500605..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettySpace 形式に関するドキュメント' -input_format: false -keywords: ['PrettySpace'] -output_format: true -slug: /interfaces/formats/PrettySpace -title: 'PrettySpace' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettyCompact`](./PrettyCompact.md) フォーマットとは異なり、テーブルの表示にグリッドではなく空白文字(スペース文字)が使用されます。 - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md deleted file mode 100644 index be493bc26e5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceMonoBlock フォーマットに関するドキュメント' -input_format: false -keywords: ['PrettySpaceMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceMonoBlock -title: 'PrettySpaceMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettySpace`](./PrettySpace.md) 形式とは異なり、最大 `10,000` 行をバッファリングしてから、[ブロック](/development/architecture#block)単位ではなく 1 つのテーブルとして出力します。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md deleted file mode 100644 index 0c5888f846e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceNoEscapes 形式に関するドキュメント' -input_format: false -keywords: ['PrettySpaceNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapes -title: 'PrettySpaceNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettySpace`](./PrettySpace.md) 形式とは、[ANSI エスケープシーケンス](http://en.wikipedia.org/wiki/ANSI_escape_code) を使用しない点が異なります。 -これは、この形式をブラウザで表示したり、`watch` コマンドラインユーティリティで使用したりするために必要です。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md deleted file mode 100644 index cbb24a7ae17..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceNoEscapesMonoBlock フォーマットに関するリファレンスドキュメント' -input_format: false -keywords: ['PrettySpaceNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapesMonoBlock -title: 'PrettySpaceNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✗ | ✔ | | - - -## 説明 {#description} - -[`PrettySpaceNoEscapes`](./PrettySpaceNoEscapes.md) フォーマットとは異なり、最大 `10,000` 行までをバッファリングし、その後 1 つのテーブルとしてまとめて出力し、[ブロック](/development/architecture#block)ごとには出力しません。 - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md deleted file mode 100644 index dedec588426..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md +++ /dev/null @@ -1,14 +0,0 @@ -{/* 注意: このファイルは、これをインポートするすべてのファイルでスニペットとして使用されます。 */ } - -以下の設定は、すべての `Pretty` 形式に共通です: - -| Setting | Description | Default | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| [`output_format_pretty_max_rows`](/operations/settings/settings-formats.md/#output_format_pretty_max_rows) | Pretty 形式における行数の上限。 | `10000` | -| [`output_format_pretty_max_column_pad_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_column_pad_width) | Pretty 形式で、列内のすべての値をパディングする際の最大幅。 | `250` | -| [`output_format_pretty_max_value_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_value_width) | Pretty 形式で表示する値の最大幅。この値を超える場合は切り詰められます。 | `10000` | -| [`output_format_pretty_color`](/operations/settings/settings-formats.md/#output_format_pretty_color) | Pretty 形式での色付けに ANSI エスケープシーケンスを使用します。 | `true` | -| [`output_format_pretty_grid_charset`](/operations/settings/settings-formats.md/#output_format_pretty_grid_charset) | グリッドの枠線を出力する際に使用する文字セット。使用可能な文字セットは ASCII と UTF-8 です。 | `UTF-8` | -| [`output_format_pretty_row_numbers`](/operations/settings/settings-formats.md/#output_format_pretty_row_numbers) | Pretty 形式の出力で、各行の前に行番号を追加します。 | `true` | -| [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names) | テーブルに多くの行が含まれる場合、フッターに列名を表示します。 | `true` | -| [`output_format_pretty_display_footer_column_names_min_rows`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names_min_rows) | [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names) が有効な場合に、フッターを表示するための最小行数を設定します。 | `50` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md deleted file mode 100644 index 5a277b0369e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -alias: [] -description: 'Prometheus 形式に関するドキュメント' -input_format: false -keywords: ['Prometheus'] -output_format: true -slug: /interfaces/formats/Prometheus -title: 'Prometheus' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | | - -## 説明 {#description} - -[Prometheus のテキストベースのエクスポジション形式](https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format)でメトリクスを公開します。 - -この形式では、出力テーブルが次のルールに従って正しく構造化されている必要があります。 - -- 列 `name` ([String](/sql-reference/data-types/string.md)) と `value` (数値) は必須です。 -- 行には任意で `help` ([String](/sql-reference/data-types/string.md)) と `timestamp` (数値) を含めることができます。 -- 列 `type` ([String](/sql-reference/data-types/string.md)) は、`counter`、`gauge`、`histogram`、`summary`、`untyped` のいずれか、または空である必要があります。 -- 各メトリクス値には、`labels` ([Map(String, String)](/sql-reference/data-types/map.md)) を持たせることもできます。 -- 連続する複数の行が、異なるラベルを持つ同一メトリクスを参照している場合があります。テーブルはメトリクス名でソートされている必要があります(例: `ORDER BY name` を使用)。 - -`histogram` と `summary` のラベルには特別な要件があります。詳細は [Prometheus のドキュメント](https://prometheus.io/docs/instrumenting/exposition_formats/#histograms-and-summaries)を参照してください。 -ラベル `{'count':''}` および `{'sum':''}` を持つ行には特別なルールが適用され、それぞれ `_count` および `_sum` に変換されます。 - -## 使用例 {#example-usage} - -```yaml -┌─name────────────────────────────────┬─type──────┬─help──────────────────────────────────────┬─labels─────────────────────────┬────value─┬─────timestamp─┐ -│ http_request_duration_seconds │ histogram │ リクエスト処理時間のヒストグラム。 │ {'le':'0.05'} │ 24054 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.1'} │ 33444 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.2'} │ 100392 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.5'} │ 129389 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'1'} │ 133988 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'+Inf'} │ 144320 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'sum':''} │ 53423 │ 0 │ -│ http_requests_total │ counter │ HTTPリクエストの総数 │ {'method':'post','code':'200'} │ 1027 │ 1395066363000 │ -│ http_requests_total │ counter │ │ {'method':'post','code':'400'} │ 3 │ 1395066363000 │ -│ metric_without_timestamp_and_labels │ │ │ {} │ 12.47 │ 0 │ -│ rpc_duration_seconds │ summary │ RPC処理時間(秒)のサマリー。 │ {'quantile':'0.01'} │ 3102 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.05'} │ 3272 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.5'} │ 4773 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.9'} │ 9001 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.99'} │ 76656 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'count':''} │ 2693 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'sum':''} │ 17560473 │ 0 │ -│ something_weird │ │ │ {'problem':'division by zero'} │ inf │ -3982045 │ -└─────────────────────────────────────┴───────────┴───────────────────────────────────────────┴────────────────────────────────┴──────────┴───────────────┘ -``` - -次の形式になります: - -```text -# HELP http_request_duration_seconds リクエスト処理時間のヒストグラム。 {#help-http_request_duration_seconds-a-histogram-of-the-request-duration} -# TYPE http_request_duration_seconds histogram {#type-http_request_duration_seconds-histogram} -http_request_duration_seconds_bucket{le="0.05"} 24054 -http_request_duration_seconds_bucket{le="0.1"} 33444 -http_request_duration_seconds_bucket{le="0.5"} 129389 -http_request_duration_seconds_bucket{le="1"} 133988 -http_request_duration_seconds_bucket{le="+Inf"} 144320 -http_request_duration_seconds_sum 53423 -http_request_duration_seconds_count 144320 - -# HELP http_requests_total HTTPリクエストの総数 {#help-http_requests_total-total-number-of-http-requests} -# TYPE http_requests_total counter {#type-http_requests_total-counter} -http_requests_total{code="200",method="post"} 1027 1395066363000 -http_requests_total{code="400",method="post"} 3 1395066363000 - -metric_without_timestamp_and_labels 12.47 - -# HELP rpc_duration_seconds RPC の処理時間(秒)のサマリー。 {#help-rpc_duration_seconds-a-summary-of-the-rpc-duration-in-seconds} -# TYPE rpc_duration_seconds summary {#type-rpc_duration_seconds-summary} -rpc_duration_seconds{quantile="0.01"} 3102 -rpc_duration_seconds{quantile="0.05"} 3272 -rpc_duration_seconds{quantile="0.5"} 4773 -rpc_duration_seconds{quantile="0.9"} 9001 -rpc_duration_seconds{quantile="0.99"} 76656 -rpc_duration_seconds_sum 17560473 -rpc_duration_seconds_count 2693 - -something_weird{problem="ゼロによる除算"} +Inf -3982045 -``` - -## 書式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md deleted file mode 100644 index 0cd43581f32..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md +++ /dev/null @@ -1,389 +0,0 @@ ---- -alias: [] -description: 'Protobuf 形式に関するドキュメント' -input_format: true -keywords: ['Protobuf'] -output_format: true -slug: /interfaces/formats/Protobuf -title: 'Protobuf' -doc_type: 'guide' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - -## 説明 {#description} - -`Protobuf` フォーマットは [Protocol Buffers](https://protobuf.dev/) フォーマットです。 - -このフォーマットでは外部のフォーマットスキーマが必要であり、そのスキーマはクエリ間でキャッシュされます。 - -ClickHouse は次をサポートしています: - -* `proto2` と `proto3` の両方の構文。 -* `Repeated` / `optional` / `required` フィールド。 - -テーブルのカラムと Protocol Buffers のメッセージ型のフィールドとの対応関係を決定するために、ClickHouse はそれらの名前を比較します。 -この比較では大文字・小文字は区別されず、文字 `_`(アンダースコア)と `.`(ドット)は同一と見なされます。 -カラムと Protocol Buffers のメッセージのフィールドの型が異なる場合は、必要な型変換が適用されます。 - -ネストされたメッセージもサポートされています。例えば、次のメッセージ型におけるフィールド `z` の場合: - -```capnp -message MessageType { - message XType { - message YType { - int32 z; - }; - repeated YType y; - }; - XType x; -}; -``` - -ClickHouse は `x.y.z`(または `x_y_z` や `X.y_Z` など)という名前の列を探索します。 - -ネストされたメッセージは、[ネストされたデータ構造](/sql-reference/data-types/nested-data-structures/index.md) の入力または出力として適しています。 - -次に示すような protobuf スキーマで定義されたデフォルト値は適用されず、代わりに [テーブルのデフォルト値](/sql-reference/statements/create/table#default_values) が使用されます。 - -```capnp -syntax = "proto2"; - -message MessageType { - optional int32 result_per_page = 3 [default = 10]; -} -``` - -メッセージに [oneof](https://protobuf.dev/programming-guides/proto3/#oneof) が含まれており、`input_format_protobuf_oneof_presence` が設定されている場合、ClickHouse は、どの oneof フィールドが見つかったかを示す列を設定します。 - -```capnp -syntax = "proto3"; - -message StringOrString { - oneof string_oneof { - string string1 = 1; - string string2 = 42; - } -} -``` - -```sql -CREATE TABLE string_or_string ( string1 String, string2 String, string_oneof Enum('no'=0, 'hello' = 1, 'world' = 42)) Engine=MergeTree ORDER BY tuple(); -INSERT INTO string_or_string from INFILE '$CURDIR/data_protobuf/String1' SETTINGS format_schema='$SCHEMADIR/string_or_string.proto:StringOrString' FORMAT ProtobufSingle; -SELECT * FROM string_or_string -``` - -```text - ┌─────────┬─────────┬──────────────┐ - │ string1 │ string2 │ string_oneof │ - ├─────────┼─────────┼──────────────┤ -1. │ │ string2 │ world │ - ├─────────┼─────────┼──────────────┤ -2. │ string1 │ │ hello │ - └─────────┴─────────┴──────────────┘ -``` - -存在を示す列の名前は、`oneof` の名前と同一でなければなりません。入れ子になったメッセージもサポートされています([basic-examples](#basic-examples) を参照)。 - -許可される型は、Int8、UInt8、Int16、UInt16、Int32、UInt32、Int64、UInt64、Enum、Enum8 または Enum16 です。 -Enum(および Enum8 または Enum16)は、`oneof` で取り得るすべてのタグに加えて、不在を示す 0 を含んでいる必要があります。文字列表現は任意です。 - -設定 [`input_format_protobuf_oneof_presence`](/operations/settings/settings-formats.md#input_format_protobuf_oneof_presence) はデフォルトで無効になっています。 - -ClickHouse は、`length-delimited` 形式で protobuf メッセージを入力および出力します。 -これは、各メッセージの前に、その長さを [可変長整数 (varint)](https://developers.google.com/protocol-buffers/docs/encoding#varints) として書き込む必要があることを意味します。 - - -## 使用例 {#example-usage} - -### データの読み取りと書き込み {#basic-examples} - -:::note Example files -この例で使用するファイルは [examples repository](https://github.com/ClickHouse/formats/ProtoBuf) から入手できます。 -::: - -この例では、ファイル `protobuf_message.bin` から ClickHouse テーブルにデータを読み込みます。その後、`Protobuf` フォーマットを使用して `protobuf_message_from_clickhouse.bin` というファイルに書き出します。 - -`schemafile.proto` というファイルを用意します。 - -```capnp -syntax = "proto3"; - -message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; -}; -``` - - -
- バイナリファイルの生成 - - すでに `Protobuf` 形式でデータをシリアライズおよびデシリアライズする方法をご存知の場合は、この手順はスキップできます。 - - ここでは Python を使用していくつかのデータを `protobuf_message.bin` にシリアライズし、それを ClickHouse に読み込みます。 - 別の言語を使用したい場合は、次も参照してください。「[How to read/write length-delimited Protobuf messages in popular languages](https://cwiki.apache.org/confluence/display/GEODE/Delimiting+Protobuf+Messages)」 - - 次のコマンドを実行して、`schemafile.proto` と同じディレクトリに `schemafile_pb2.py` という名前の Python ファイルを生成します。 - このファイルには、`UserData` Protobuf メッセージを表す Python クラスが含まれます。 - - ```bash - protoc --python_out=. schemafile.proto - ``` - - 次に、`schemafile_pb2.py` と同じディレクトリに `generate_protobuf_data.py` という名前の新しい Python ファイルを作成します。 - 次のコードをそのファイルに貼り付けてください。 - - ```python - import schemafile_pb2 # Module generated by 'protoc' - from google.protobuf import text_format - from google.protobuf.internal.encoder import _VarintBytes # Import the internal varint encoder - - def create_user_data_message(name, surname, birthDate, phoneNumbers): - """ - Creates and populates a UserData Protobuf message. - """ - message = schemafile_pb2.MessageType() - message.name = name - message.surname = surname - message.birthDate = birthDate - message.phoneNumbers.extend(phoneNumbers) - return message - - # The data for our example users - data_to_serialize = [ - {"name": "Aisha", "surname": "Khan", "birthDate": 19920815, "phoneNumbers": ["(555) 247-8903", "(555) 612-3457"]}, - {"name": "Javier", "surname": "Rodriguez", "birthDate": 20001015, "phoneNumbers": ["(555) 891-2046", "(555) 738-5129"]}, - {"name": "Mei", "surname": "Ling", "birthDate": 19980616, "phoneNumbers": ["(555) 956-1834", "(555) 403-7682"]}, - ] - - output_filename = "protobuf_messages.bin" - - # バイナリファイルを書き込みモード ('wb') で開く - with open(output_filename, "wb") as f: - for item in data_to_serialize: - # 現在のユーザー用の Protobuf メッセージインスタンスを作成する - message = create_user_data_message( - item["name"], - item["surname"], - item["birthDate"], - item["phoneNumbers"] - ) - - # メッセージをシリアライズする - serialized_data = message.SerializeToString() - - # シリアライズされたデータの長さを取得する - message_length = len(serialized_data) - - # Protobuf ライブラリの内部 _VarintBytes を使用して長さをエンコードする - length_prefix = _VarintBytes(message_length) - - # 長さのプレフィックスを書き込む - f.write(length_prefix) - # シリアライズされたメッセージデータを書き込む - f.write(serialized_data) - - print(f"Protobuf messages (length-delimited) written to {output_filename}") - - # --- オプション: 検証 (読み戻して表示する) --- - # 読み戻しには、varint 用の内部 Protobuf デコーダも使用します。 - from google.protobuf.internal.decoder import _DecodeVarint32 - - print("\n--- Verifying by reading back ---") - with open(output_filename, "rb") as f: - buf = f.read() # varint のデコードを簡単にするため、ファイル全体をバッファに読み込む - n = 0 - while n < len(buf): - # varint の長さプレフィックスをデコードする - msg_len, new_pos = _DecodeVarint32(buf, n) - n = new_pos - - # メッセージデータを取り出す - message_data = buf[n:n+msg_len] - n += msg_len - - # メッセージをパースする - decoded_message = schemafile_pb2.MessageType() - decoded_message.ParseFromString(message_data) - print(text_format.MessageToString(decoded_message, as_utf8=True)) - ``` - - ここで、コマンドラインからスクリプトを実行します。 - 例えば `uv` を使って、Python の仮想環境から実行することを推奨します。 - - ```bash - uv venv proto-venv - source proto-venv/bin/activate - ``` - - 次の Python ライブラリをインストールする必要があります。 - - ```bash - uv pip install --upgrade protobuf - ``` - - バイナリファイルを生成するために、スクリプトを実行します。 - - ```bash - python generate_protobuf_data.py - ``` -
- -スキーマに一致する ClickHouse テーブルを作成します。 - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - - -コマンドラインからテーブルにデータを挿入します: - -```bash -cat protobuf_messages.bin | clickhouse-client --query "INSERT INTO test.protobuf_messages SETTINGS format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -`Protobuf` 形式を使用して、データをバイナリファイルに書き出すこともできます。 - -```sql -SELECT * FROM test.protobuf_messages INTO OUTFILE 'protobuf_message_from_clickhouse.bin' FORMAT Protobuf SETTINGS format_schema = 'schemafile:MessageType' -``` - -Protobuf スキーマを使用して、ClickHouse からファイル `protobuf_message_from_clickhouse.bin` に書き出されたデータをデシリアライズできます。 - - -### ClickHouse Cloud を使用したデータの読み取りと書き込み - -ClickHouse Cloud では、Protobuf スキーマファイルをアップロードすることはできません。ただし、クエリ内でスキーマを指定するために `format_protobuf_schema` -設定を使用できます。この例では、ローカルマシン上のシリアル化されたデータを読み取り、ClickHouse Cloud のテーブルに挿入する方法を示します。 - -前の例と同様に、ClickHouse Cloud 上で Protobuf のスキーマに従ってテーブルを作成します。 - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -設定 `format_schema_source` は、設定 `format_schema` のソースを定義します。 - -指定可能な値: - -* 'file' (デフォルト): Cloud ではサポートされていません -* 'string': `format_schema` はスキーマ内容そのもの(リテラル)です。 -* 'query': `format_schema` はスキーマを取得するためのクエリです。 - - -### `format_schema_source='string'` - -スキーマを文字列として指定して ClickHouse Cloud にデータを挿入するには、次のコマンドを実行します: - -```bash -cat protobuf_messages.bin | clickhouse client --host --secure --password --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -テーブルに挿入されたデータを取得します: - -```sql -clickhouse client --host <ホスト名> --secure --password <パスワード> --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### `format_schema_source='query'` - -Protobuf スキーマをテーブルに保存することもできます。 - -データを挿入するためのテーブルを ClickHouse Cloud 上で作成します。 - -```sql -CREATE TABLE testing.protobuf_schema ( - schema String -) -ENGINE = MergeTree() -ORDER BY tuple(); -``` - -```sql -INSERT INTO testing.protobuf_schema VALUES ('syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};'); -``` - -実行するクエリ内でスキーマを指定して、データを ClickHouse Cloud に挿入します: - -```bash -cat protobuf_messages.bin | clickhouse client --host <ホスト名> --secure --password <パスワード> --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='SELECT schema FROM testing.protobuf_schema', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -テーブルに挿入したデータを選択します: - -```sql -clickhouse client --host <ホスト名> --secure --password <パスワード> --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### 自動生成スキーマの利用 - -データ用の外部 Protobuf スキーマがない場合でも、自動生成されたスキーマを利用することで、Protobuf 形式でデータを出力/入力できます。 -この場合は `format_protobuf_use_autogenerated_schema` 設定を使用します。 - -例: - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1 -``` - -この場合、ClickHouse はテーブル構造に従って関数 [`structureToProtobufSchema`](/sql-reference/functions/other-functions#structureToProtobufSchema) を使用し、Protobuf スキーマを自動生成します。その後、このスキーマを使ってデータを Protobuf 形式でシリアライズします。 - -自動生成されたスキーマを使って Protobuf ファイルを読み込むこともできます。この場合、そのファイルは同じスキーマを使用して作成されている必要があります。 - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_protobuf_use_autogenerated_schema=1 FORMAT Protobuf" -``` - -設定 [`format_protobuf_use_autogenerated_schema`](/operations/settings/settings-formats.md#format_protobuf_use_autogenerated_schema) はデフォルトで有効になっており、[`format_schema`](/operations/settings/formats#format_schema) が設定されていない場合に適用されます。 - -また、[`output_format_schema`](/operations/settings/formats#output_format_schema) 設定を使用して、入出力時に自動生成されたスキーマをファイルに保存することもできます。たとえば、次のようにします。 - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1, output_format_schema='path/to/schema/schema.proto' -``` - -この場合、自動生成された Protobuf スキーマは `path/to/schema/schema.capnp` というファイルに保存されます。 - - -### Protobuf キャッシュの削除 {#basic-examples-cloud} - -[`format_schema_path`](/operations/server-configuration-parameters/settings.md/#format_schema_path) から読み込まれた Protobuf スキーマを再読み込むには、[`SYSTEM DROP ... FORMAT CACHE`](/sql-reference/statements/system.md/#system-drop-schema-format) ステートメントを使用します。 - -```sql -SYSTEM DROP FORMAT SCHEMA CACHE FOR Protobuf -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md deleted file mode 100644 index f6c3c6f03df..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -alias: [] -description: 'ProtobufList フォーマットに関するドキュメント' -input_format: true -keywords: ['ProtobufList'] -output_format: true -slug: /interfaces/formats/ProtobufList -title: 'ProtobufList' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✔ | | - - -## 説明 {#description} - -`ProtobufList` フォーマットは [`Protobuf`](./Protobuf.md) フォーマットと似ていますが、行は固定名「Envelope」を持つメッセージ内に含まれるサブメッセージの列として表現されます。 - - - -## 使用例 {#example-usage} - -例えば、次のようにします。 - -```sql -SELECT * FROM test.table FORMAT ProtobufList SETTINGS format_schema = 'schemafile:MessageType' -``` - -```bash -cat protobuflist_messages.bin | clickhouse-client --query "INSERT INTO test.table FORMAT ProtobufList SETTINGS format_schema='schemafile:MessageType'" -``` - -ファイル `schemafile.proto` は次のような内容です: - -```capnp title="schemafile.proto" -syntax = "proto3"; -message Envelope { - message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; - }; - MessageType row = 1; -}; -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md deleted file mode 100644 index ae9b63553a8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'ProtobufSingle 形式のドキュメント' -input_format: true -keywords: ['ProtobufSingle'] -output_format: true -slug: /interfaces/formats/ProtobufSingle -title: 'ProtobufSingle' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✔ | | - - -## 説明 {#description} - -`ProtobufSingle` 形式は [`Protobuf`](./Protobuf.md) 形式と同じですが、長さデリミタなしで単一の Protobuf メッセージを保存およびパースするためのものです。 - - - -## 使用例 {#example-usage} - - - -## 書式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md deleted file mode 100644 index d4766e30eac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'RawBLOB フォーマットに関するドキュメント' -keywords: ['RawBLOB'] -slug: /interfaces/formats/RawBLOB -title: 'RawBLOB' -doc_type: 'reference' ---- - - - -## 説明 {#description} - -`RawBLOB` 形式は、すべての入力データを単一の値として読み取ります。これは、[`String`](/sql-reference/data-types/string.md) 型またはそれに類似した単一フィールドのみを持つテーブルだけをパースできます。 -結果は区切り文字やエスケープなしのバイナリ形式として出力されます。2 つ以上の値が出力される場合、この形式はあいまいになり、データを読み戻すことは不可能になります。 - -### Raw フォーマットの比較 {#raw-formats-comparison} - -以下は、`RawBLOB` と [`TabSeparatedRaw`](./TabSeparated/TabSeparatedRaw.md) フォーマットの比較です。 - -`RawBLOB`: - -* データはバイナリ形式で出力され、エスケープは行われません。 -* 値同士の間に区切り文字はありません。 -* 各値の末尾には改行がありません。 - -`TabSeparatedRaw`: - -* データはエスケープなしで出力されます。 -* 行にはタブで区切られた値が含まれます。 -* 各行の最後の値の後に改行文字があります。 - -以下は、`RawBLOB` と [RowBinary](./RowBinary/RowBinary.md) フォーマットの比較です。 - -`RawBLOB`: - -* String フィールドは長さを表すプレフィックスなしで出力されます。 - -`RowBinary`: - -* String フィールドは、可変長整数 (varint) 形式(符号なし [LEB128] ([https://en.wikipedia.org/wiki/LEB128](https://en.wikipedia.org/wiki/LEB128))) の長さで表現され、その後に文字列のバイト列が続きます。 - -`RawBLOB` 入力に空のデータが渡されると、ClickHouse は例外をスローします。 - -```text -コード: 108. DB::Exception: 挿入するデータがありません -``` - - -## 使用例 {#example-usage} - -```bash title="Query" -$ clickhouse-client --query "CREATE TABLE {some_table} (a String) ENGINE = Memory;" -$ cat {filename} | clickhouse-client --query="INSERT INTO {some_table} FORMAT RawBLOB" -$ clickhouse-client --query "SELECT * FROM {some_table} FORMAT RawBLOB" | md5sum -``` - -```text title="Response" -f9725a22f9191e064120d718e26862a9 - -``` - - -## フォーマットの設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md deleted file mode 100644 index e1fd0b9e41e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -alias: [] -description: 'Regexp フォーマットに関するドキュメント' -input_format: true -keywords: ['Regexp'] -output_format: false -slug: /interfaces/formats/Regexp -title: 'Regexp' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 説明 {#description} - -`Regex` フォーマットは、指定された正規表現に従って、インポートされたデータの各行をパースします。 - -**使用方法** - -[format_regexp](/operations/settings/settings-formats.md/#format_regexp) 設定で指定された正規表現が、インポートされたデータの各行に適用されます。正規表現内のサブパターンの数は、インポートされるデータセット内の列数と同じである必要があります。 - -インポートされるデータの各行は、改行文字 `'\n'` または DOS 形式の改行 `"\r\n"` で区切られている必要があります。 - -マッチした各サブパターンの内容は、[format_regexp_escaping_rule](/operations/settings/settings-formats.md/#format_regexp_escaping_rule) 設定に従い、対応するデータ型のパース方法で処理されます。 - -正規表現が行にマッチせず、かつ [format_regexp_skip_unmatched](/operations/settings/settings-formats.md/#format_regexp_escaping_rule) が 1 に設定されている場合、その行は何の通知もなくスキップされます。そうでない場合は、例外がスローされます。 - - - -## 使用例 {#example-usage} - -`data.tsv` というファイルがあるとします。 - -```text title="data.tsv" -id: 1 array: [1,2,3] string: str1 date: 2020-01-01 -id: 2 array: [1,2,3] string: str2 date: 2020-01-02 -id: 3 array: [1,2,3] string: str3 date: 2020-01-03 -``` - -および `imp_regex_table` テーブル: - -```sql -CREATE TABLE imp_regex_table (id UInt32, array Array(UInt32), string String, date Date) ENGINE = Memory; -``` - -先ほどのファイルのデータを、次のクエリで上記のテーブルに挿入します。 - -```bash -$ cat data.tsv | clickhouse-client --query "INSERT INTO imp_regex_table SETTINGS format_regexp='id: (.+?) array: (.+?) string: (.+?) date: (.+?)', format_regexp_escaping_rule='Escaped', format_regexp_skip_unmatched=0 FORMAT Regexp;" -``` - -これで、テーブルからデータを `SELECT` して、`Regex` フォーマットでファイル内のデータがどのように解析されたかを確認できます。 - -```sql title="Query" -SELECT * FROM imp_regex_table; -``` - -```text title="Response" -┌─id─┬─array───┬─string─┬───────date─┐ -│ 1 │ [1,2,3] │ str1 │ 2020-01-01 │ -│ 2 │ [1,2,3] │ str2 │ 2020-01-02 │ -│ 3 │ [1,2,3] │ str3 │ 2020-01-03 │ -└────┴─────────┴────────┴────────────┘ -``` - - -## フォーマット設定 {#format-settings} - -`Regexp` フォーマットを使用する場合、次の設定を使用できます。 - -- `format_regexp` — [String](/sql-reference/data-types/string.md)。[re2](https://github.com/google/re2/wiki/Syntax) 形式の正規表現を指定します。 -- `format_regexp_escaping_rule` — [String](/sql-reference/data-types/string.md)。次のエスケープ規則がサポートされています。 - - - CSV([CSV](/interfaces/formats/CSV) と同様) - - JSON([JSONEachRow](/interfaces/formats/JSONEachRow) と同様) - - Escaped([TSV](/interfaces/formats/TabSeparated) と同様) - - Quoted([Values](/interfaces/formats/Values) と同様) - - Raw(サブパターンを全体として抽出し、エスケープ規則は適用されません。[TSVRaw](/interfaces/formats/TabSeparated) と同様) - -- `format_regexp_skip_unmatched` — [UInt8](/sql-reference/data-types/int-uint.md)。`format_regexp` 式がインポートされたデータにマッチしない場合に例外をスローするかどうかを制御します。`0` または `1` に設定できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md deleted file mode 100644 index 876db78435a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -alias: [] -description: 'RowBinary 形式のドキュメント' -input_format: true -keywords: ['RowBinary'] -output_format: true -slug: /interfaces/formats/RowBinary -title: 'RowBinary' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✔ | | - - -## 説明 {#description} - -`RowBinary` フォーマットは、バイナリ形式で行ごとにデータをパースします。 -行および値は区切り文字なしで連続して並びます。 -データがバイナリ形式であるため、`FORMAT RowBinary` の後に続く区切り文字は次のように厳密に決められています。 - -- 任意数の空白文字: - - `' '` (スペース - コード `0x20`) - - `'\t'` (タブ - コード `0x09`) - - `'\f'` (フォームフィード - コード `0x0C`) -- 続いて、正確に 1 つの改行シーケンス: - - Windows スタイルの `"\r\n"` - - または Unix スタイルの `'\n'` -- その直後にバイナリデータが続きます。 - -:::note -このフォーマットは行ベースであるため、[Native](../Native.md) フォーマットより効率が劣ります。 -::: - -以下のデータ型については、次の点が重要です。 - -- [Integers](../../../sql-reference/data-types/int-uint.md) は固定長のリトルエンディアン表現を使用します。たとえば、`UInt64` は 8 バイトを使用します。 -- [DateTime](../../../sql-reference/data-types/datetime.md) は、値として Unix タイムスタンプを含む `UInt32` で表現されます。 -- [Date](../../../sql-reference/data-types/date.md) は、`1970-01-01` からの日数を値として含む `UInt16` として表現されます。 -- [String](../../../sql-reference/data-types/string.md) は可変長整数 (varint) (符号なし [`LEB128`](https://en.wikipedia.org/wiki/LEB128)) で長さを表し、その後に文字列のバイト列が続きます。 -- [FixedString](../../../sql-reference/data-types/fixedstring.md) は単にバイト列として表現されます。 -- [Arrays](../../../sql-reference/data-types/array.md) は可変長整数 (varint) (符号なし [LEB128](https://en.wikipedia.org/wiki/LEB128)) で要素数を表し、その後に配列要素が順に続きます。 - -[NULL](/sql-reference/syntax#null) をサポートするために、各 [Nullable](/sql-reference/data-types/nullable.md) 値の前に `1` または `0` を含む追加の 1 バイトが挿入されます。 -- `1` の場合、その値は `NULL` であり、このバイトは別個の値として解釈されます。 -- `0` の場合、そのバイトの後の値は `NULL` ではありません。 - -`RowBinary` フォーマットと `RawBlob` フォーマットの比較については、[Raw Formats Comparison](../RawBLOB.md/#raw-formats-comparison) を参照してください。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md deleted file mode 100644 index 2928d241dbb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: [] -description: 'RowBinaryWithDefaults フォーマットに関するドキュメント' -input_format: true -keywords: ['RowBinaryWithDefaults'] -output_format: false -slug: /interfaces/formats/RowBinaryWithDefaults -title: 'RowBinaryWithDefaults' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 入力 | 出力 | エイリアス | -| -- | -- | ----- | -| ✔ | ✗ | | - - -## 説明 {#description} - -[`RowBinary`](./RowBinary.md) 形式と似ていますが、各列の前に 1 バイトが追加され、そのバイトでデフォルト値を使用するかどうかを示します。 - - - -## 使用例 {#example-usage} - -例: - -```sql title="Query" -SELECT * FROM FORMAT('RowBinaryWithDefaults', 'x UInt32 default 42, y UInt32', x'010001000000') -``` - -```response title="Response" -┌──x─┬─y─┐ -│ 42 │ 1 │ -└────┴───┘ -``` - -* 列 `x` には、デフォルト値を使用すべきことを示す 1 バイトの `01` だけがあり、このバイト以降には一切データがありません。 -* 列 `y` では、データはバイト `00` から始まっており、これは列に実際の値が存在し、その値を後続のデータ `01000000` から読み取る必要があることを示します。 - - -## フォーマット設定 {#format-settings} - - diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md deleted file mode 100644 index 1e2c4f10e64..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'RowBinaryWithNames 形式のドキュメント' -input_format: true -keywords: ['RowBinaryWithNames'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNames -title: 'RowBinaryWithNames' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 入力 | 出力 | 別名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 説明 {#description} - -[`RowBinary`](./RowBinary.md) フォーマットと類似していますが、ヘッダーが追加されています。 - -- 列数 (N) を表す [`LEB128`](https://en.wikipedia.org/wiki/LEB128) でエンコードされた数値。 -- 列名を指定する N 個の `String`。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - - -:::note -- 設定 [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) が `1` に設定されている場合、 - 入力データの列は名前に基づいてテーブルの列にマッピングされます。 -- 設定 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が `1` に設定されている場合は、不明な名前の列はスキップされます。 - それ以外の場合、最初の行がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md deleted file mode 100644 index ad74fe81e05..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -alias: [] -description: 'RowBinaryWithNamesAndTypes 形式に関するドキュメント' -input_format: true -keywords: ['RowBinaryWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNamesAndTypes -title: 'RowBinaryWithNamesAndTypes' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 入力 | 出力 | 別名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 説明 {#description} - -[RowBinary](./RowBinary.md) 形式と類似していますが、ヘッダーが追加されています。 - -- 列数 (N) を表す [`LEB128`](https://en.wikipedia.org/wiki/LEB128) でエンコードされた数値。 -- 列名を指定する N 個の `String`。 -- 列の型を指定する N 個の `String`。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - - - -:::note -[`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) が 1 に設定されている場合、 -入力データの列は名前に基づいてテーブルの列に対応付けられ、[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) が 1 に設定されている場合は、名前が不明な列はスキップされます。 -それ以外の場合は、最初の行がスキップされます。 -[`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header) が `1` に設定されている場合、 -入力データの型はテーブル内の対応する列の型と比較されます。そうでない場合は、2 行目がスキップされます。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md deleted file mode 100644 index 07efad4908e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md +++ /dev/null @@ -1,11 +0,0 @@ -{/* 注意: このスニペットは、これをインポートするすべてのファイルで再利用されます */ } - -The following settings are common to all `RowBinary` type formats. - -| Setting | Description | Default | -| -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| [`format_binary_max_string_size`](/operations/settings/settings-formats.md/#format_binary_max_string_size) | RowBinary フォーマットにおける `String` の最大許容サイズ。 | `1GiB` | -| [`output_format_binary_encode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md) 出力フォーマットで、ヘッダー内の型を型名の文字列ではなく、[`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) を用いたバイナリ表現で書き出すことを許可します。 | `false` | -| [`input_format_binary_decode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md) 入力フォーマットで、ヘッダー内の型を型名の文字列ではなく、[`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) を用いたバイナリ表現として読み取ることを許可します。 | `false` | -| [`output_format_binary_write_json_as_string`](/operations/settings/settings-formats.md/#output_format_binary_write_json_as_string) | [`RowBinary`](../RowBinary.md) 出力フォーマットで、[`JSON`](/sql-reference/data-types/newjson.md) データ型の値を `JSON` の [String](/sql-reference/data-types/string.md) 値として書き出すことを許可します。 | `false` | -| [`input_format_binary_read_json_as_string`](/operations/settings/settings-formats.md/#input_format_binary_read_json_as_string) | [`RowBinary`](../RowBinary.md) 入力フォーマットで、[`JSON`](/sql-reference/data-types/newjson.md) データ型の値を `JSON` の [String](/sql-reference/data-types/string.md) 値として読み取ることを許可します。 | `false` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md deleted file mode 100644 index 931b6f31daf..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'SQLInsert フォーマットに関するドキュメント' -input_format: false -keywords: ['SQLInsert'] -output_format: true -slug: /interfaces/formats/SQLInsert -title: 'SQLInsert' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -データを `INSERT INTO table (columns...) VALUES (...), (...) ...;` 文の連続として出力します。 - - - -## 使用例 {#example-usage} - -例: - -```sql -SELECT number AS x, number + 1 AS y, 'Hello' AS z FROM numbers(10) FORMAT SQLInsert SETTINGS output_format_sql_insert_max_batch_size = 2 -``` - -```sql -INSERT INTO table (x, y, z) VALUES (0, 1, 'こんにちは'), (1, 2, 'こんにちは'); -INSERT INTO table (x, y, z) VALUES (2, 3, 'こんにちは'), (3, 4, 'こんにちは'); -INSERT INTO table (x, y, z) VALUES (4, 5, 'こんにちは'), (5, 6, 'こんにちは'); -INSERT INTO table (x, y, z) VALUES (6, 7, 'こんにちは'), (7, 8, 'こんにちは'); -INSERT INTO table (x, y, z) VALUES (8, 9, 'こんにちは'), (9, 10, 'こんにちは'); -``` - -このフォーマットで出力されたデータを読み取るには、[MySQLDump](../formats/MySQLDump.md) 入力フォーマットを使用できます。 - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | デフォルト | -|-------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|-----------| -| [`output_format_sql_insert_max_batch_size`](../../operations/settings/settings-formats.md/#output_format_sql_insert_max_batch_size) | 1つの INSERT 文内の最大行数。 | `65505` | -| [`output_format_sql_insert_table_name`](../../operations/settings/settings-formats.md/#output_format_sql_insert_table_name) | 出力される INSERT クエリ内のテーブル名。 | `'table'` | -| [`output_format_sql_insert_include_column_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_include_column_names) | INSERT クエリにカラム名を含めるかどうか。 | `true` | -| [`output_format_sql_insert_use_replace`](../../operations/settings/settings-formats.md/#output_format_sql_insert_use_replace) | INSERT の代わりに REPLACE 文を使用するかどうか。 | `false` | -| [`output_format_sql_insert_quote_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_quote_names) | カラム名を "\`"(バッククォート)で囲むかどうか。 | `true` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md deleted file mode 100644 index d51d4ef0291..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -alias: [] -description: 'TSKV 形式に関するドキュメント' -input_format: true -keywords: ['TSKV'] -output_format: true -slug: /interfaces/formats/TSKV -title: 'TSKV' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -[`TabSeparated`](./TabSeparated.md) フォーマットと似ていますが、値を `name=value` 形式で出力します。 -名前は [`TabSeparated`](./TabSeparated.md) フォーマットの場合と同じ方法でエスケープされ、`=` 記号も同様にエスケープされます。 - -```text -SearchPhrase= count()=8267016 -SearchPhrase=バスルームインテリアデザイン count()=2166 -SearchPhrase=clickhouse count()=1655 -SearchPhrase=2014年春ファッション count()=1549 -SearchPhrase=フリーフォーム写真 count()=1480 -SearchPhrase=アンジェリーナ・ジョリー count()=1245 -SearchPhrase=オムスク count()=1112 -SearchPhrase=犬種の写真 count()=1091 -SearchPhrase=カーテンデザイン count()=1064 -SearchPhrase=バクー count()=1000 -``` - -```sql title="Query" -SELECT * FROM t_null FORMAT TSKV -``` - -```text title="Response" -x=1 y=\N -``` - -:::note -小さなカラムが多数ある場合、このフォーマットは非効率であり、通常これを使用する理由はありません。 -とはいえ、効率の面では [`JSONEachRow`](../JSON/JSONEachRow.md) フォーマットと同等です。 -::: - -パース時には、異なるカラムの値の順序は任意で構いません。 -一部の値を省略しても許容され、省略された値はデフォルト値と等しいものとして扱われます。 -この場合、ゼロおよび空行がデフォルト値として使用されます。 -テーブルで指定可能な複雑な値は、デフォルト値としてはサポートされません。 - -パース時には、追加のフィールド `tskv` をイコール記号や値なしで追加しても構いません。このフィールドは無視されます。 - -インポート時、[`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が `1` に設定されている場合、名前が不明なカラムはスキップされます。 - -[NULL](/sql-reference/syntax.md) は `\N` としてフォーマットされます。 - - -## 利用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の tskv ファイル `football.tskv` を使用します。 - -```tsv -date=2022-04-30 season=2021 home_team=Sutton United away_team=Bradford City home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=Swindon Town away_team=Barrow home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=Tranmere Rovers away_team=Oldham Athletic home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=Port Vale away_team=Newport County home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=Salford City away_team=Mansfield Town home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Barrow away_team=Northampton Town home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Bradford City away_team=Carlisle United home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Bristol Rovers away_team=Scunthorpe United home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Exeter City away_team=Port Vale home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Harrogate Town A.F.C. away_team=Sutton United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Hartlepool United away_team=Colchester United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Leyton Orient away_team=Tranmere Rovers home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Mansfield Town away_team=Forest Green Rovers home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Newport County away_team=Rochdale home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Oldham Athletic away_team=Crawley Town home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Stevenage Borough away_team=Salford City home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Walsall away_team=Swindon Town home_team_goals=0 away_team_goals=3 -``` - -データを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.tskv' FORMAT TSKV; -``` - -### データの読み込み {#reading-data} - -`TSKV` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TSKV -``` - -出力は、列名と型を示す 2 行のヘッダー付きのタブ区切り形式になります。 - - -```tsv -date=2022-04-30 season=2021 home_team=Sutton United away_team=Bradford City home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=Swindon Town away_team=Barrow home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=Tranmere Rovers away_team=Oldham Athletic home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=Port Vale away_team=Newport County home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=Salford City away_team=Mansfield Town home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Barrow away_team=Northampton Town home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Bradford City away_team=Carlisle United home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Bristol Rovers away_team=Scunthorpe United home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Exeter City away_team=Port Vale home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Harrogate Town A.F.C. away_team=Sutton United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Hartlepool United away_team=Colchester United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Leyton Orient away_team=Tranmere Rovers home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Mansfield Town away_team=Forest Green Rovers home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Newport County away_team=Rochdale home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Oldham Athletic away_team=Crawley Town home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Stevenage Borough away_team=Salford City home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Walsall away_team=Swindon Town home_team_goals=0 away_team_goals=3 -``` - - -## フォーマットの設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md deleted file mode 100644 index 5db4351a2af..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md +++ /dev/null @@ -1,190 +0,0 @@ ---- -alias: ['TSV'] -description: 'TSV 形式のドキュメント' -input_format: true -keywords: ['TabSeparated', 'TSV'] -output_format: true -slug: /interfaces/formats/TabSeparated -title: 'TabSeparated' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|--------| -| ✔ | ✔ | `TSV` | - - - -## 説明 {#description} - -TabSeparated フォーマットでは、データは行単位で書き込まれます。各行には、タブで区切られた値が含まれます。各値の後にはタブが続きますが、行の最後の値の後にはタブではなく改行コードが続きます。改行コードはいずれも Unix スタイルであることを前提とします。最後の行の末尾にも改行コードが付いていなければなりません。値はテキスト形式で、引用符で囲まず、特殊文字はエスケープして書き込まれます。 - -このフォーマットは `TSV` という名前でも利用可能です。 - -`TabSeparated` フォーマットは、独自のプログラムやスクリプトでデータを処理するのに便利です。HTTP インターフェイスおよびコマンドラインクライアントのバッチモードでは、デフォルトでこのフォーマットが使用されます。また、このフォーマットを使用すると、異なる DBMS 間でデータを転送できます。たとえば、MySQL からダンプを取得して ClickHouse にインポートしたり、その逆も可能です。 - -`TabSeparated` フォーマットは、合計値(WITH TOTALS を使用する場合)および極値(`extremes` が 1 に設定されている場合)の出力をサポートします。この場合、合計値と極値はメイン結果の後に出力されます。メイン結果、合計値、および極値は、互いに空行で区切られます。例: - -```sql -SELECT EventDate, count() AS c FROM test.hits GROUP BY EventDate WITH TOTALS ORDER BY EventDate FORMAT TabSeparated - -2014-03-17 1406958 -2014-03-18 1383658 -2014-03-19 1405797 -2014-03-20 1353623 -2014-03-21 1245779 -2014-03-22 1031592 -2014-03-23 1046491 - -1970-01-01 8873898 - -2014-03-17 1031592 -2014-03-23 1406958 -``` - - -## データの書式設定 {#tabseparated-data-formatting} - -整数は 10 進数形式で記述されます。数値は先頭に追加の「+」記号を含むことができます(パース時には無視され、書式化時には出力されません)。非負の数値には負号を含めることはできません。読み取り時には、空文字列をゼロとしてパースすること、または(符号付き型の場合)マイナス記号だけから成る文字列をゼロとしてパースすることが許可されています。対応するデータ型に収まらない数値は、エラーを出さずに別の数値としてパースされる場合があります。 - -浮動小数点数は 10 進数形式で記述されます。小数点にはドットが使用されます。指数表記がサポートされており、`inf`、`+inf`、`-inf`、`nan` も使用できます。浮動小数点数の表記は、小数点で始まったり終わったりすることがあります。 -書式化時には、浮動小数点数の精度が失われる可能性があります。 -パース時には、機械で表現可能な最近接の数値を厳密に読み取ることは要求されません。 - -日付は `YYYY-MM-DD` 形式で記述され、同じ形式でパースされますが、区切り文字には任意の文字を使用できます。 -時刻付きの日付は `YYYY-MM-DD hh:mm:ss` 形式で記述され、同じ形式でパースされますが、区切り文字には任意の文字を使用できます。 -これらはすべて、クライアントまたはサーバー(どちらがデータをフォーマットするかによって異なる)が起動した時点のシステムタイムゾーンで処理されます。時刻付きの日付については、夏時間は指定されません。そのため、ダンプに夏時間中の時刻が含まれている場合、そのダンプはデータと一意に対応せず、パース時には 2 つの時刻のうちいずれかが選択されます。 -読み取り操作では、不正な日付や時刻付きの日付も、自然なオーバーフローとして、またはヌルの日付および時刻として、エラーなしでパースされる場合があります。 - -例外として、時刻付き日付のパースは、ちょうど 10 桁の 10 進数字から成る場合に Unix タイムスタンプ形式でのパースもサポートされます。この結果はタイムゾーンに依存しません。`YYYY-MM-DD hh:mm:ss` 形式と `NNNNNNNNNN` 形式は自動的に区別されます。 - -文字列は、バックスラッシュでエスケープされた特殊文字として出力されます。出力には次のエスケープシーケンスが使用されます: `\b`、`\f`、`\r`、`\n`、`\t`、`\0`、`\'`、`\\`。パースでは、これらに加えて `\a`、`\v`、`\xHH`(16 進エスケープシーケンス)、および任意の `\c` シーケンス(ここで `c` は任意の文字であり、これらのシーケンスは `c` に変換される)もサポートされます。したがって、データの読み取りでは、改行を `\n` として、`\` として、あるいは実際の改行として記述する形式をサポートします。たとえば、単語の間をスペースではなく改行で区切った文字列 `Hello world` は、次のいずれのバリエーションとしてもパースできます。 - -```text -こんにちは\n世界 - -こんにちは\ -世界 -``` - -2 番目のバリアントがサポートされているのは、MySQL がタブ区切りダンプを書き出す際にこれを使用するためです。 - -TabSeparated 形式でデータを渡すときにエスケープが必要となる文字の最小集合は、タブ、改行 (LF)、およびバックスラッシュです。 - -エスケープされる記号はごく一部だけです。そのため、端末での出力が乱れてしまうような文字列値に容易に遭遇する可能性があります。 - -配列は角かっこ内のカンマ区切り値のリストとして書き出されます。配列内の数値要素は通常どおりにフォーマットされます。`Date` および `DateTime` 型はシングルクォートで囲んで書き出されます。文字列は、上記と同じエスケープ規則を用いてシングルクォートで囲んで書き出されます。 - -[NULL](/sql-reference/syntax.md) は設定 [format_tsv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation) に従ってフォーマットされます (デフォルト値は `\N` です)。 - -入力データでは、ENUM 値は名前または id として表現できます。まず入力値を ENUM 名にマッチさせようとします。失敗し、かつ入力値が数値である場合は、この数値を ENUM id にマッチさせようとします。 -入力データが ENUM id のみを含む場合は、ENUM のパースを最適化するために、設定 [input_format_tsv_enum_as_number](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) を有効にすることを推奨します。 - -[Nested](/sql-reference/data-types/nested-data-structures/index.md) 構造の各要素は配列として表現されます。 - -例えば次のとおりです。 - -```sql -CREATE TABLE nestedt -( - `id` UInt8, - `aux` Nested( - a UInt8, - b String - ) -) -ENGINE = TinyLog -``` - -```sql -INSERT INTO nestedt VALUES ( 1, [1], ['a']) -``` - -```sql -SELECT * FROM nestedt FORMAT TSV -``` - -```response -1 [1] ['a'] -``` - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の `football.tsv` という名前の TSV ファイルを使用します: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparated; -``` - -### データの読み込み {#reading-data} - -`TabSeparated` 形式でデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TabSeparated -``` - -出力はタブ区切り形式です: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## フォーマット設定 {#format-settings} - -| Setting | Description | Default | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| -| [`format_tsv_null_representation`](/operations/settings/settings-formats.md/#format_tsv_null_representation) | TSV 形式における NULL のカスタム表現。 | `\N` | -| [`input_format_tsv_empty_as_default`](/operations/settings/settings-formats.md/#input_format_tsv_empty_as_default) | TSV 入力で空フィールドをデフォルト値として扱います。複雑なデフォルト式を使用する場合は、[input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) も有効化する必要があります。 | `false` | -| [`input_format_tsv_enum_as_number`](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) | TSV 形式で挿入される enum 値を enum のインデックスとして扱います。 | `false` | -| [`input_format_tsv_use_best_effort_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_tsv_use_best_effort_in_schema_inference) | TSV 形式でスキーマを推論する際に、いくつかの調整とヒューリスティクスを使用します。無効化すると、すべてのフィールドは String として推論されます。 | `true` | -| [`output_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#output_format_tsv_crlf_end_of_line) | `true` に設定すると、TSV 出力形式の行末は `\n` ではなく `\r\n` になります。 | `false` | -| [`input_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#input_format_tsv_crlf_end_of_line) | `true` に設定すると、TSV 入力形式の行末は `\n` ではなく `\r\n` になります。 | `false` | -| [`input_format_tsv_skip_first_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_first_lines) | データの先頭から指定された行数をスキップします。 | `0` | -| [`input_format_tsv_detect_header`](/operations/settings/settings-formats.md/#input_format_tsv_detect_header) | TSV 形式で名前および型を含むヘッダーを自動検出します。 | `true` | -| [`input_format_tsv_skip_trailing_empty_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_trailing_empty_lines) | データ末尾の空行をスキップします。 | `false` | -| [`input_format_tsv_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_tsv_allow_variable_number_of_columns) | TSV 形式で列数の変動を許可し、余分な列を無視し、欠損している列にはデフォルト値を使用します。 | `false` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md deleted file mode 100644 index ee81f7168da..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: ['TSVRaw', 'Raw'] -description: 'TabSeparatedRaw フォーマットのドキュメント' -input_format: true -keywords: ['TabSeparatedRaw'] -output_format: true -slug: /interfaces/formats/TabSeparatedRaw -title: 'TabSeparatedRaw' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|------|------|------------------| -| ✔ | ✔ | `TSVRaw`, `Raw` | - - - -## 説明 {#description} - -このフォーマットは [`TabSeparated`](/interfaces/formats/TabSeparated) フォーマットと異なり、行をエスケープせずに書き込みます。 - -:::note -このフォーマットで解析する場合、各フィールド内にタブ文字または改行文字を含めることはできません。 -::: - -`TabSeparatedRaw` フォーマットと `RawBlob` フォーマットの比較については、[Raw フォーマットの比較](../RawBLOB.md/#raw-formats-comparison) を参照してください。 - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の TSV ファイル `football.tsv` を使用します: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRaw; -``` - -### データの読み込み {#reading-data} - -`TabSeparatedRaw` 形式でデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRaw -``` - -出力はタブ区切り形式になります。 - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## フォーマット設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md deleted file mode 100644 index 7273321547e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: ['TSVRawWithNames', 'RawWithNames'] -description: 'TabSeparatedRawWithNames 形式に関するドキュメント' -input_format: true -keywords: ['TabSeparatedRawWithNames', 'TSVRawWithNames', 'RawWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNames -title: 'TabSeparatedRawWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|------|------|-----------------------------------| -| ✔ | ✔ | `TSVRawWithNames`, `RawWithNames` | - - - -## 説明 {#description} - -このフォーマットは、行がエスケープ処理されずに書き込まれるという点で、[`TabSeparatedWithNames`](./TabSeparatedWithNames.md) フォーマットとは異なります。 - -:::note -このフォーマットで解析する際、各フィールド内でタブや改行文字は使用できません。 -::: - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -`football.tsv` という名前の次の `tsv` ファイルを使用します。 - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNames; -``` - -### データの読み込み {#reading-data} - -`TabSeparatedRawWithNames` 形式でデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNames -``` - -出力は、1 行目がヘッダーのタブ区切り形式になります。 - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md deleted file mode 100644 index c7f7914dae3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: ['TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -description: 'TabSeparatedRawWithNamesAndTypes 形式のドキュメント' -input_format: true -keywords: ['TabSeparatedRawWithNamesAndTypes', 'TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNamesAndTypes -title: 'TabSeparatedRawWithNamesAndTypes' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|------|------|---------------------------------------------------| -| ✔ | ✔ | `TSVRawWithNamesAndNames`, `RawWithNamesAndNames` | - - - -## 説明 {#description} - -この形式は、行がエスケープ処理なしで書き出されるという点で、[`TabSeparatedWithNamesAndTypes`](./TabSeparatedWithNamesAndTypes.md) 形式と異なります。 - -:::note -この形式でパースする場合、各フィールド内にタブや改行文字を含めることはできません。 -::: - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -以下の内容の `football.tsv` という名前の TSV ファイルを使用します: - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNamesAndTypes; -``` - -### データの読み取り {#reading-data} - -`TabSeparatedRawWithNamesAndTypes` 形式を使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNamesAndTypes -``` - -出力は、列名と型を示す 2 行のヘッダー行を持つタブ区切り形式になります。 - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 書式設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md deleted file mode 100644 index c8ccdd9f720..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -alias: ['TSVWithNames'] -description: 'TabSeparatedWithNames 形式に関するドキュメント' -input_format: true -keywords: ['TabSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedWithNames -title: 'TabSeparatedWithNames' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|--------------------------------| -| ✔ | ✔ | `TSVWithNames`, `RawWithNames` | - - - -## 説明 {#description} - -最初の行に列名が記載されている点が、[`TabSeparated`](./TabSeparated.md) 形式と異なります。 - -解析時には、最初の行に列名が含まれていることが前提となります。列名を使用して列の位置を特定したり、正しさを検証したりできます。 - -:::note -[`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 設定が `1` に設定されている場合、 -入力データ中の列は名前に基づいてテーブルの列にマッピングされ、さらに [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が `1` に設定されている場合は、不明な名前の列はスキップされます。 -それ以外の場合、最初の行はスキップされます。 -::: - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の TSV ファイル(ファイル名: `football.tsv`)を使用します: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入します: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNames; -``` - -### データの読み取り {#reading-data} - -`TabSeparatedWithNames` 形式を使用してデータを読み込みます: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNames -``` - -出力はタブ区切り形式で表示されます: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 書式設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md deleted file mode 100644 index 697bf3dea2d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: 'TabSeparatedWithNamesAndTypes 形式に関するドキュメント' -keywords: ['TabSeparatedWithNamesAndTypes'] -slug: /interfaces/formats/TabSeparatedWithNamesAndTypes -title: 'TabSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| Input | Output | エイリアス | -|-------|--------|------------------------------------------------| -| ✔ | ✔ | `TSVWithNamesAndTypes`, `RawWithNamesAndTypes` | - - - -## 説明 {#description} - -[`TabSeparated`](./TabSeparated.md) 形式とは異なり、最初の行にはカラム名が、2 行目にはカラム型が書き込まれます。 - -:::note -- [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 設定が `1` の場合、 -入力データのカラムは名前によってテーブル内のカラムにマッピングされます。さらに [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 設定が `1` の場合、名前が不明なカラムはスキップされます。 -それ以外の場合、最初の行はスキップされます。 -- [`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) 設定が `1` の場合、 -入力データの型はテーブル内の対応するカラムの型と比較されます。それ以外の場合、2 行目はスキップされます。 -::: - - - -## 使用例 {#example-usage} - -### データの挿入 {#inserting-data} - -次の内容の `football.tsv` という tsv ファイルを使用します。 - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -データを挿入する: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNamesAndTypes; -``` - -### データの読み込み {#reading-data} - -`TabSeparatedWithNamesAndTypes` フォーマットを使用してデータを読み込みます。 - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNamesAndTypes -``` - -出力はタブ区切り形式となり、列名と型を表す 2 行のヘッダーが付きます。 - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md deleted file mode 100644 index e1bc070c87f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md +++ /dev/null @@ -1,246 +0,0 @@ ---- -alias: [] -description: 'Template フォーマットのドキュメント' -input_format: true -keywords: ['Template'] -output_format: true -slug: /interfaces/formats/Template -title: 'Template' -doc_type: 'guide' ---- - -| Input | Output | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -他の標準フォーマットでは対応できない、より高度なカスタマイズが必要な場合に、 -`Template` フォーマットを使用すると、値のプレースホルダーを含む独自のカスタムフォーマット文字列と、 -データに対するエスケープルールをユーザーが指定できます。 - -このフォーマットでは、次の設定を使用します: - -| 設定 | 説明 | -|----------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------| -| [`format_template_row`](#format_template_row) | 行のフォーマット文字列を含むファイルへのパスを指定します。 | -| [`format_template_resultset`](#format_template_resultset) | 結果セットのフォーマット文字列を含むファイルへのパスを指定します。 | -| [`format_template_rows_between_delimiter`](#format_template_rows_between_delimiter) | 行と行の間の区切り文字を指定します。これは最後の行を除く各行の後に出力(または入力として期待)されます(デフォルトは `\n`)。 | -| `format_template_row_format` | 行のフォーマット文字列を[インライン](#inline_specification)で指定します。 | -| `format_template_resultset_format` | 結果セットのフォーマット文字列を[インライン](#inline_specification)で指定します。 | -| 他のフォーマットの一部の設定(例: `JSON` エスケープを使用する場合の `output_format_json_quote_64bit_integers` | | - - - -## 設定とエスケープ規則 {#settings-and-escaping-rules} - -### format_template_row {#format_template_row} - -`format_template_row` 設定は、次の構文で行用のフォーマット文字列が記述されたファイルへのパスを指定します。 - -```text -delimiter_1${column_1:serializeAs_1}delimiter_2${column_2:serializeAs_2} ... delimiter_N -``` - -Where: - -| Part of syntax | Description | -| --------------- | ----------------------------------------------- | -| `delimiter_i` | 値同士の区切り文字(`$` 記号は `$$` としてエスケープ可能) | -| `column_i` | 値を選択または挿入する対象となる列の名前またはインデックス(空の場合、その列はスキップされる) | -| `serializeAs_i` | 列の値に対するエスケープ規則。 | - -サポートされているエスケープ規則は次のとおりです。 - -| Escaping Rule | Description | -| -------------------- | ---------------------- | -| `CSV`, `JSON`, `XML` | 同名のフォーマットと同様 | -| `Escaped` | `TSV` と同様 | -| `Quoted` | `Values` と同様 | -| `Raw` | エスケープなしで、`TSVRaw` と同様 | -| `None` | エスケープ規則なし(詳細は下記の注記を参照) | - -:::note -エスケープ規則が省略された場合、`None` が使用されます。`XML` は出力にのみ適しています。 -::: - -例を見てみましょう。次のフォーマット文字列が与えられているとします。 - -```text -検索フレーズ: ${s:Quoted}、件数: ${c:Escaped}、広告価格: $$${p:JSON}; -``` - -以下の値は(`SELECT` を使用している場合は)出力され、(`INPUT` を使用している場合は)入力として期待されます。 -それぞれ、カラム `Search phrase:`, `, count:`, `, ad price: $` と `;` の区切り文字の間に対応します。 - -* `s`(エスケープルール `Quoted`) -* `c`(エスケープルール `Escaped`) -* `p`(エスケープルール `JSON`) - -例: - -* `INSERT` を行う場合、下記の行は期待されるテンプレートに一致しており、カラム `Search phrase`, `count`, `ad price` にそれぞれ値 `bathroom interior design`, `2166`, `$3` を読み込みます。 -* `SELECT` を行う場合、下記の行は、値 `bathroom interior design`, `2166`, `$3` がすでにテーブル内のカラム `Search phrase`, `count`, `ad price` に保存されていると仮定したときの出力例です。 - -```yaml -検索フレーズ: 'bathroom interior design'、件数: 2166、広告単価: $3; -``` - -### format_template_rows_between_delimiter {#format_template_rows_between_delimiter} - -`format_template_rows_between_delimiter` 設定は、行と行の間に出力(または入力として期待)される区切り文字列を指定します。最後の行を除くすべての行の後に出力され、デフォルトは `\n` です。 - -### format_template_resultset {#format_template_resultset} - -`format_template_resultset` 設定は、結果セット用のフォーマット文字列を含むファイルへのパスを指定します。 - -結果セット用のフォーマット文字列は、行用のフォーマット文字列と同じ構文を持ちます。 -これにより、接頭辞や接尾辞、追加情報の出力方法を指定でき、列名の代わりに次のプレースホルダを使用します。 - -* `data` は、`format_template_row` フォーマットで表現されたデータ行で、`format_template_rows_between_delimiter` で区切られます。このプレースホルダは、フォーマット文字列内の先頭に配置する必要があります。 -* `totals` は、`format_template_row` フォーマットで表現された合計値の行です(WITH TOTALS 使用時)。 -* `min` は、`format_template_row` フォーマットで表現された最小値の行です(extremes が 1 に設定されている場合)。 -* `max` は、`format_template_row` フォーマットで表現された最大値の行です(extremes が 1 に設定されている場合)。 -* `rows` は、出力行の総数です。 -* `rows_before_limit` は、LIMIT がなかった場合に存在していたであろう行数の下限値です。LIMIT を含むクエリでのみ出力されます。クエリに GROUP BY が含まれている場合、rows_before_limit_at_least は LIMIT がなかった場合に存在していた行数の正確な値になります。 -* `time` は、リクエストの実行時間(秒)です。 -* `rows_read` は、読み取られた行数です。 -* `bytes_read` は、読み取られた(非圧縮の)バイト数です。 - -`data`、`totals`、`min`、`max` の各プレースホルダには、エスケープ規則を指定してはいけません(または明示的に `None` を指定する必要があります)。残りのプレースホルダには任意のエスケープ規則を指定できます。 - -:::note -`format_template_resultset` 設定が空文字列の場合、デフォルト値として `${data}` が使用されます。 -::: - - -挿入クエリでは、先頭または末尾を省略する場合(例を参照)、一部の列やフィールドをスキップできるフォーマットを利用できます。 - -### インライン指定 {#inline_specification} - -クラスター内のすべてのノード上のディレクトリに、テンプレートフォーマットの設定(`format_template_row`、`format_template_resultset` で設定)をデプロイすることが困難、あるいは不可能な場合がよくあります。 -さらに、そのフォーマットが非常に単純で、ファイルとして配置する必要がない場合もあります。 - -このような場合には、`format_template_row_format`(`format_template_row` 用)および `format_template_resultset_format`(`format_template_resultset` 用)を使用して、ファイルへのパスではなく、テンプレート文字列そのものをクエリ内で直接指定できます。 - -:::note -フォーマット文字列およびエスケープシーケンスに関するルールは、次と同じです。 -- `format_template_row_format` を使用する場合は [`format_template_row`](#format_template_row)。 -- `format_template_resultset_format` を使用する場合は [`format_template_resultset`](#format_template_resultset)。 -::: - - - -## 使用例 {#example-usage} - -まずは `Template` 形式の利用例として、データの選択と挿入の 2 つのケースを見ていきます。 - -### データの選択 {#selecting-data} - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase ORDER BY c DESC LIMIT 5 FORMAT Template SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format', format_template_rows_between_delimiter = '\n ' -``` - -```text title="/some/path/resultset.format" - - 検索フレーズ - - - - ${data} -
検索フレーズ
検索フレーズ 件数
- - ${max} -
最大値
- ${rows_read:XML} 行を ${time:XML} 秒で処理しました - - -``` - -```text title="/some/path/row.format" - ${0:XML} ${1:XML} -``` - -結果: - -```html - - 検索フレーズ - - - - - - - - -
検索フレーズ
検索フレーズ 件数
8267016
bathroom interior design 2166
clickhouse 1655
spring 2014 fashion 1549
freeform photos 1480
- - -
最大値
8873898
- 3095973行を0.1569913秒で処理しました - - -``` - -### データの挿入 {#inserting-data} - -```text -ヘッダー -ページビュー: 5、ユーザーID: 4324182021466249494、不要なフィールド: hello、期間: 146、符号: -1 -ページビュー: 6、ユーザーID: 4324182021466249494、不要なフィールド: world、期間: 185、符号: 1 -合計行数: 2 -``` - -```sql -INSERT INTO UserActivity SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format' -FORMAT Template -``` - -```text title="/some/path/resultset.format" -ヘッダー\n${data}\n総行数: ${:CSV}\n -``` - -```text title="/some/path/row.format" -ページビュー: ${PageViews:CSV}, ユーザーID: ${UserID:CSV}, 未使用フィールド: ${:CSV}, 期間: ${Duration:CSV}, 符号: ${Sign:CSV} -``` - -プレースホルダー内の `PageViews`、`UserID`、`Duration` および `Sign` は、テーブル内の列名です。行中の `Useless field` 以降の値と、サフィックス中の `\nTotal rows:` 以降の値は無視されます。 -入力データ内のすべての区切り文字は、指定されたフォーマット文字列内の区切り文字と厳密に一致している必要があります。 - -### インライン指定 {#in-line-specification} - -Markdown テーブルを手作業で整形するのにうんざりしていませんか?この例では、`Template` フォーマットとインライン指定の設定を使って、簡単なタスクをどのように実現できるかを見ていきます。ここでは、`system.formats` テーブルからいくつかの ClickHouse フォーマット名を `SELECT` し、それらを Markdown テーブルとして整形します。これは、`Template` フォーマットと `format_template_row_format` および `format_template_resultset_format` 設定を使うことで容易に実現できます。 - - -前の例では、結果セットおよび行フォーマットの文字列を別ファイルに記述し、それらファイルへのパスをそれぞれ `format_template_resultset` および `format_template_row` 設定で指定しました。ここではテンプレートがごく単純で、Markdown テーブルを作るためのいくつかの `|` と `-` だけで構成されるため、インラインで指定します。結果セットのテンプレート文字列は、`format_template_resultset_format` 設定を使って指定します。テーブルヘッダを作るために、`${data}` の前に `|ClickHouse Formats|\n|---|\n` を追加しています。行に対しては、`format_template_row_format` 設定を使用し、テンプレート文字列 ``|`{0:XML}`|`` を指定します。`Template` フォーマットは、指定したフォーマットで整形した行をプレースホルダ `${data}` に挿入します。この例ではカラムは 1 つだけですが、もし追加したい場合は、行テンプレート文字列に `{1:XML}`、`{2:XML}` ... のように追記し、適切なエスケープルールを選択すればかまいません。この例ではエスケープルールとして `XML` を使用しています。 - -```sql title="Query" -WITH formats AS -( - SELECT * FROM system.formats - ORDER BY rand() - LIMIT 5 -) -SELECT * FROM formats -FORMAT Template -SETTINGS - format_template_row_format='|`${0:XML}`|', - format_template_resultset_format='|ClickHouse フォーマット|\n|---|\n${data}\n' -``` - -ご覧のとおり、あの markdown テーブルを作るために必要な `|` や `-` を、手作業で一つずつ追加していく手間を省くことができました: - -```response title="Response" -|ClickHouse フォーマット| -|---| -|`BSONEachRow`| -|`CustomSeparatedWithNames`| -|`Prometheus`| -|`DWARF`| -|`Avro`| -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md deleted file mode 100644 index fb54b474462..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -alias: [] -description: 'TemplateIgnoreSpaces フォーマットのドキュメント' -input_format: true -keywords: ['TemplateIgnoreSpaces'] -output_format: false -slug: /interfaces/formats/TemplateIgnoreSpaces -title: 'TemplateIgnoreSpaces' -doc_type: 'reference' ---- - -| 入力 | 出力 | 別名 | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 説明 {#description} - -[`Template`] と似ていますが、入力ストリーム内のデリミタと値の間にある空白文字をスキップします。 -ただし、フォーマット文字列に空白文字が含まれている場合、その空白文字は入力ストリーム内にも存在している必要があります。 -また、空白を無視するために、あるデリミタを複数の部分に分割する目的で空のプレースホルダー(`${}` または `${:None}`)を指定することもできます。 -このようなプレースホルダーは、空白文字をスキップする場合にのみ使用されます。 -すべての行で列の値の順序が同じであれば、このフォーマットを使って `JSON` を読み込むことも可能です。 - -:::note -このフォーマットは入力専用です。 -::: - - - -## 使用例 {#example-usage} - -以下のリクエストを使用すると、[JSON](/interfaces/formats/JSON) 形式の出力例からデータを挿入できます。 - -```sql -INSERT INTO table_name -SETTINGS - format_template_resultset = '/some/path/resultset.format', - format_template_row = '/some/path/row.format', - format_template_rows_between_delimiter = ',' -FORMAT TemplateIgnoreSpaces -``` - -```text title="/some/path/resultset.format" -{${}"meta"${}:${:JSON},${}"data"${}:${}[${data}]${},${}"totals"${}:${:JSON},${}"extremes"${}:${:JSON},${}"rows"${}:${:JSON},${}"rows_before_limit_at_least"${}:${:JSON}${}} -``` - -```text title="/some/path/row.format" -{${}"SearchPhrase"${}:${}${phrase:JSON}${},${}"c"${}:${}${cnt:JSON}${}} -``` - - -## フォーマット設定 {#format-settings} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md deleted file mode 100644 index 8cd27b8d734..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -alias: [] -description: 'Values 形式に関するドキュメント' -input_format: true -keywords: ['Values'] -output_format: true -slug: /interfaces/formats/Values -title: 'Values' -doc_type: 'guide' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 説明 {#description} - -`Values` 形式は、各行を丸括弧で囲んで出力します。 - -- 行同士はカンマで区切られ、最後の行の後にはカンマは付きません。 -- 丸括弧内の値もカンマ区切りです。 -- 数値は、引用符なしの 10 進数形式で出力されます。 -- 配列は角括弧で出力されます。 -- 文字列、日付、および日時は引用符で囲んで出力されます。 -- エスケープ規則とパースは [TabSeparated](TabSeparated/TabSeparated.md) 形式と同様です。 - -フォーマット時には余分なスペースは挿入されませんが、パース時には(配列要素内のスペースを除き)スペースがあっても許可され、スキップされます。 -[`NULL`](/sql-reference/syntax.md) は `NULL` として表現されます。 - -`Values` 形式でデータを渡す際に、最低限エスケープが必要な文字は次のとおりです。 -- シングルクォート -- バックスラッシュ - -この形式は `INSERT INTO t VALUES ...` で使用されますが、クエリ結果のフォーマットにも使用できます。 - - - -## 使用例 {#example-usage} - - - -## フォーマット設定 {#format-settings} - -| 設定 | 説明 | 既定値 | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------| -| [`input_format_values_interpret_expressions`](../../operations/settings/settings-formats.md/#input_format_values_interpret_expressions) | フィールドをストリーミングパーサーで解析できなかった場合、SQL パーサーを実行し、SQL 式として解釈しようとします。 | `true` | -| [`input_format_values_deduce_templates_of_expressions`](../../operations/settings/settings-formats.md/#input_format_values_deduce_templates_of_expressions) | フィールドをストリーミングパーサーで解析できなかった場合、SQL パーサーを実行して SQL 式のテンプレートを推定し、そのテンプレートを使ってすべての行の解析を試みたうえで、全行について式を解釈します。 | `true` | -| [`input_format_values_accurate_types_of_literals`](../../operations/settings/settings-formats.md/#input_format_values_accurate_types_of_literals) | テンプレートを使用して式を解析および解釈する際に、リテラルの実際の型を確認し、オーバーフローや精度の問題を回避します。 | `true` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md deleted file mode 100644 index 7ca77cbe1df..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -alias: [] -description: 'Vertical 形式に関するドキュメント' -input_format: false -keywords: ['Vertical'] -output_format: true -slug: /interfaces/formats/Vertical -title: 'Vertical' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -各値を、指定された列名とともに個別の行に出力します。この形式は、各行が多数の列で構成されている場合に、1 行または少数の行だけを出力するのに便利です。 - -[`NULL`](/sql-reference/syntax.md) は、文字列値 `NULL` と値が存在しないことを区別しやすくするために `ᴺᵁᴸᴸ` として出力されることに注意してください。JSON 列は整形して表示され、`NULL` は `null` として出力されます。これは有効な JSON 値であり、`"null"` と容易に区別できるためです。 - - - -## 使用例 {#example-usage} - -例: - -```sql -SELECT * FROM t_null FORMAT Vertical -``` - -```response -行 1: -────── -x: 1 -y: ᴺᵁᴸᴸ -``` - -Vertical 形式では行はエスケープされません。 - -```sql -SELECT 'string with \'quotes\' and \t with some special \n characters' AS test FORMAT Vertical -``` - -```response -行 1: -────── -test: 'quotes' を含む文字列と いくつかの特殊 - 文字 -``` - -この形式はクエリ結果の出力にのみ適しており、パース(テーブルへの挿入用にデータを取得する処理)には適していません。 - - -## フォーマット設定 {#format-settings} diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md deleted file mode 100644 index f28824ee1a7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -alias: [] -description: 'XML 形式に関するドキュメント' -input_format: false -keywords: ['XML'] -output_format: true -slug: /interfaces/formats/XML -title: 'XML' -doc_type: 'reference' ---- - -| 入力 | 出力 | エイリアス | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 説明 {#description} - -`XML` 形式は出力専用であり、入力のパースには使用できません。 - -列名が妥当な形式でない場合、要素名として単に `field` が使用されます。一般的に、XML 構造は JSON 構造に従います。 -JSON と同様に、不正な UTF-8 シーケンスは置換文字 `�` に置き換えられるため、出力テキストは有効な UTF-8 シーケンスだけで構成されます。 - -文字列値では、文字 `<` と `&` はそれぞれ `<` および `&` としてエスケープされます。 - -配列は `HelloWorld...` のように、タプルは `HelloWorld...` のように出力されます。 - - - -## 使用例 {#example-usage} - -例: - -```xml - - - - - - SearchPhrase - String - - - count() - UInt64 - - - - - - - 8267016 - - - bathroom interior design - 2166 - - - clickhouse - 1655 - - - 2014 spring fashion - 1549 - - - freeform photos - 1480 - - - angelina jolie - 1245 - - - omsk - 1112 - - - photos of dog breeds - 1091 - - - curtain designs - 1064 - - - baku - 1000 - - - 10 - 141137 - -``` - - -## フォーマット設定 {#format-settings} - - - -## XML {#xml} \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/grpc.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/grpc.md deleted file mode 100644 index 1fb2a1a7197..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/grpc.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -description: 'ClickHouse の gRPC インターフェイスに関するドキュメント' -sidebar_label: 'gRPC インターフェイス' -sidebar_position: 25 -slug: /interfaces/grpc -title: 'gRPC インターフェイス' -doc_type: 'reference' ---- - - - -# gRPC インターフェース {#grpc-interface} - - - -## はじめに {#grpc-interface-introduction} - -ClickHouse は [gRPC](https://grpc.io/) インターフェースをサポートしています。gRPC は、HTTP/2 と [Protocol Buffers](https://en.wikipedia.org/wiki/Protocol_Buffers) を使用するオープンソースのリモートプロシージャコールシステムです。ClickHouse における gRPC の実装は、次の機能をサポートします。 - -- SSL -- 認証 -- セッション -- 圧縮 -- 同一チャネル経由での並列クエリ -- クエリのキャンセル -- 進捗およびログの取得 -- 外部テーブル - -このインターフェースの仕様は [clickhouse_grpc.proto](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto) に記載されています。 - - - -## gRPC 構成 {#grpc-interface-configuration} - -gRPC インターフェイスを使用するには、メインの[サーバー構成](../operations/configuration-files.md)で `grpc_port` を設定します。その他の構成オプションについては、以下の例を参照してください。 - -```xml -9100 - - false - - - /path/to/ssl_cert_file - /path/to/ssl_key_file - - - false - - - /path/to/ssl_ca_cert_file - - - deflate - - - medium - - - -1 - -1 - - - false - -``` - - -## 組み込みクライアント {#grpc-client} - -提供されている[仕様](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto)に基づき、gRPC がサポートしている任意のプログラミング言語でクライアントを実装できます。 -あるいは、組み込みの Python クライアントを使用することもできます。これはリポジトリ内の [utils/grpc-client/clickhouse-grpc-client.py](https://github.com/ClickHouse/ClickHouse/blob/master/utils/grpc-client/clickhouse-grpc-client.py) に配置されています。組み込みクライアントを使用するには、Python モジュール [grpcio および grpcio-tools](https://grpc.io/docs/languages/python/quickstart) が必要です。 - -クライアントは以下の引数をサポートします。 - -* `--help` – ヘルプメッセージを表示して終了します。 -* `--host HOST, -h HOST` – サーバー名。デフォルト値: `localhost`。IPv4 または IPv6 アドレスも使用できます。 -* `--port PORT` – 接続先ポート。このポートは ClickHouse サーバー設定で有効化されている必要があります(`grpc_port` を参照)。デフォルト値: `9100`。 -* `--user USER_NAME, -u USER_NAME` – ユーザー名。デフォルト値: `default`。 -* `--password PASSWORD` – パスワード。デフォルト値: 空文字列。 -* `--query QUERY, -q QUERY` – 非対話モードで実行するクエリ。 -* `--database DATABASE, -d DATABASE` – デフォルトデータベース。指定されていない場合は、サーバー設定で現在設定されているデータベースが使用されます(デフォルトは `default`)。 -* `--format OUTPUT_FORMAT, -f OUTPUT_FORMAT` – 結果の出力[フォーマット](formats.md)。対話モードでのデフォルト値: `PrettyCompact`。 -* `--debug` – デバッグ情報の表示を有効にします。 - -対話モードでクライアントを実行するには、`--query` 引数を付けずに実行します。 - -バッチモードでは、クエリデータを `stdin` 経由で渡すことができます。 - -**クライアントの使用例** - -次の例では、テーブルを作成し、CSV ファイルからデータをロードします。その後、そのテーブルの内容をクエリします。 - -```bash -./clickhouse-grpc-client.py -q "CREATE TABLE grpc_example_table (id UInt32, text String) ENGINE = MergeTree() ORDER BY id;" -echo -e "0,Input data for\n1,gRPC protocol example" > a.csv -cat a.csv | ./clickhouse-grpc-client.py -q "INSERT INTO grpc_example_table FORMAT CSV" - -./clickhouse-grpc-client.py --format PrettyCompact -q "SELECT * FROM grpc_example_table;" -``` - -結果: - -```text -┌─id─┬─text──────────────────┐ -│ 0 │ Input data for │ -│ 1 │ gRPCプロトコルの例 │ -└────┴───────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/http.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/http.md deleted file mode 100644 index c3953b24848..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/http.md +++ /dev/null @@ -1,1222 +0,0 @@ ---- -description: 'ClickHouse の HTTP インターフェイスに関するドキュメント。あらゆるプラットフォームおよびプログラミング言語から、REST - API を通じて ClickHouse にアクセスできます' -sidebar_label: 'HTTP インターフェイス' -sidebar_position: 15 -slug: /interfaces/http -title: 'HTTP インターフェイス' -doc_type: 'reference' ---- - -import PlayUI from '@site/static/images/play.png'; -import Image from '@theme/IdealImage'; - - -# HTTP インターフェース {#http-interface} - - - -## 前提条件 {#prerequisites} - -この記事の例を実行するには、次のものが必要です: -- 稼働中の ClickHouse サーバーインスタンス -- `curl` がインストールされていること。Ubuntu または Debian では `sudo apt install curl` を実行するか、インストール手順については[こちらのドキュメント](https://curl.se/download.html)を参照してください。 - - - -## 概要 {#overview} - -HTTP インターフェイスを使用すると、REST API の形であらゆるプラットフォームやプログラミング言語から ClickHouse を利用できます。HTTP インターフェイスはネイティブインターフェイスよりも機能面では制限がありますが、言語サポートは優れています。 - -デフォルトでは、`clickhouse-server` は次のポートで待ち受けます: - -* HTTP 用にポート 8123 -* HTTPS 用にポート 8443(有効化可能) - -パラメーターなしで `GET /` リクエストを送信すると、文字列 "Ok." とともにステータスコード 200 が返されます。 - -```bash -$ curl 'http://localhost:8123/' -Ok. -``` - -"Ok." は、[`http_server_default_response`](../operations/server-configuration-parameters/settings.md#http_server_default_response) で定義されている既定値であり、必要に応じて変更できます。 - -あわせて [HTTP 応答コードに関する注意事項](#http_response_codes_caveats) も参照してください。 - - -## Web ユーザーインターフェイス {#web-ui} - -ClickHouse には Web ユーザーインターフェイスが用意されており、以下のアドレスからアクセスできます。 - -```text -http://localhost:8123/play -``` - -Web UI は、クエリ実行中の進行状況の表示、クエリのキャンセル、結果のストリーミング表示をサポートしています。 -また、クエリパイプラインに対してチャートやグラフを表示する隠し機能も備えています。 - -クエリが正常に実行されると、ダウンロードボタンが表示され、CSV、TSV、JSON、JSONLines、Parquet、Markdown など、または ClickHouse がサポートする任意のカスタムフォーマットでクエリ結果をダウンロードできます。ダウンロード機能はクエリキャッシュを利用して、クエリを再実行することなく効率的に結果を取得します。UI 上では多数あるページのうち 1 ページ分しか表示されていない場合でも、ダウンロードされるのは結果セット全体です。 - -Web UI は、あなたのようなプロフェッショナル向けに設計されています。 - -ClickHouse Web UI のスクリーンショット - -ヘルスチェック用スクリプトでは `GET /ping` リクエストを使用してください。このハンドラーは常に "Ok."(末尾に改行付き)を返します。バージョン 18.12.13 以降で利用可能です。レプリカの遅延を確認するには `/replicas_status` も参照してください。 - -```bash -$ curl 'http://localhost:8123/ping' -Ok. -$ curl 'http://localhost:8123/replicas_status' -Ok. -``` - - -## HTTP/HTTPS でのクエリ実行 {#querying} - -HTTP/HTTPS 経由でクエリを実行する方法は次の 3 つです。 - -* リクエストを URL の `query` パラメータとして送信する -* POST メソッドを使用する -* クエリの先頭部分を `query` パラメータで送り、残りを POST で送信する - -:::note -URL のサイズはデフォルトで 1 MiB に制限されています。この値は `http_max_uri_size` 設定で変更できます。 -::: - -成功した場合は、ステータスコード 200 と、レスポンスボディ内に結果が返されます。 -エラーが発生した場合は、ステータスコード 500 と、レスポンスボディ内にエラー内容のテキストが返されます。 - -GET を使用するリクエストは「読み取り専用」です。つまり、データを変更するクエリには POST メソッドしか使用できません。 -クエリ自体は、POST ボディまたは URL パラメータのどちらかで送信できます。いくつか例を見てみましょう。 - -次の例では、curl を使用してクエリ `SELECT 1` を送信します。スペースを表す URL エンコード `%20` の使用に注意してください。 - -```bash title="command" -curl 'http://localhost:8123/?query=SELECT%201' -``` - -```response title="Response" -1 -``` - -この例では、`-nv`(非詳細)および `-O-` オプションを指定した wget を使用して、結果をターミナルに出力しています。 -この場合、スペース文字に対して URL エンコードを使用する必要はありません。 - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1' -``` - -```response -1 -``` - -この例では、生の HTTP リクエストを netcat にパイプで渡します。 - -```bash title="command" -echo -ne 'GET /?query=SELECT%201 HTTP/1.0\r\n\r\n' | nc localhost 8123 -``` - -```response title="response" -HTTP/1.0 200 OK -X-ClickHouse-Summary: {"read_rows":"1","read_bytes":"1","written_rows":"0","written_bytes":"0","total_rows_to_read":"1","result_rows":"0","result_bytes":"0","elapsed_ns":"4505959","memory_usage":"1111711"} -Date: Tue, 11 Nov 2025 18:16:01 GMT -Connection: Close -Content-Type: text/tab-separated-values; charset=UTF-8 -Access-Control-Expose-Headers: X-ClickHouse-Query-Id,X-ClickHouse-Summary,X-ClickHouse-Server-Display-Name,X-ClickHouse-Format,X-ClickHouse-Timezone,X-ClickHouse-Exception-Code,X-ClickHouse-Exception-Tag -X-ClickHouse-Server-Display-Name: MacBook-Pro.local -X-ClickHouse-Query-Id: ec0d8ec6-efc4-4e1d-a14f-b748e01f5294 -X-ClickHouse-Format: TabSeparated -X-ClickHouse-Timezone: Europe/London -X-ClickHouse-Exception-Tag: dngjzjnxkvlwkeua - -1 -``` - -ご覧のとおり、`curl` コマンドはスペース文字を URL エスケープしなければならないという点で、やや不便です。 -`wget` は自動的にすべてをエスケープしてくれますが、keep-alive と Transfer-Encoding: chunked を使用した場合に HTTP/1.1 上でうまく動作しないため、使用は推奨しません。 - -```bash -$ echo 'SELECT 1' | curl 'http://localhost:8123/' --data-binary @- -1 - -$ echo 'SELECT 1' | curl 'http://localhost:8123/?query=' --data-binary @- -1 - -$ echo '1' | curl 'http://localhost:8123/?query=SELECT' --data-binary @- -1 -``` - -クエリの一部がパラメータで送信され、別の一部が POST で送信される場合、これら 2 つのデータ部分の間に改行が挿入されます。 -例えば、次のようなものは動作しません。 - -```bash -$ echo 'ECT 1' | curl 'http://localhost:8123/?query=SEL' --data-binary @- -Code: 59, e.displayText() = DB::Exception: 構文エラー: 位置0で失敗: SEL -ECT 1 -, 期待される値: SHOW TABLES, SHOW DATABASES, SELECT, INSERT, CREATE, ATTACH, RENAME, DROP, DETACH, USE, SET, OPTIMIZE., e.what() = DB::Exception -``` - -デフォルトでは、データは [`TabSeparated`](/interfaces/formats/TabSeparated) 形式で返されます。 - -`FORMAT` 句は、クエリ内で別の形式での出力を指定するために使用します。例えば、次のように指定します。 - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1, 2, 3 FORMAT JSON' -``` - - -```response title="Response" -{ - "meta": - [ - { - "name": "1", - "type": "UInt8" - }, - { - "name": "2", - "type": "UInt8" - }, - { - "name": "3", - "type": "UInt8" - } - ], - - "data": - [ - { - "1": 1, - "2": 2, - "3": 3 - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.000515, - "rows_read": 1, - "bytes_read": 1 - } -} -``` - -`TabSeparated` 以外のデフォルトのフォーマットを指定するには、`default_format` URL パラメータまたは `X-ClickHouse-Format` ヘッダーを使用できます。 - -```bash -$ echo 'SELECT 1 FORMAT Pretty' | curl 'http://localhost:8123/?' --data-binary @- -┏━━━┓ -┃ 1 ┃ -┡━━━┩ -│ 1 │ -└───┘ -``` - -パラメータ化されたクエリには POST メソッドを使用できます。パラメータは、パラメータ名と型を波括弧で指定します。例えば `{name:Type}` のように記述します。パラメータ値は `param_name` として渡します。 - -```bash -$ curl -X POST -F 'query=select {p1:UInt8} + {p2:UInt8}' -F "param_p1=3" -F "param_p2=4" 'http://localhost:8123/' - -7 -``` - - -## HTTP/HTTPS 経由での INSERT クエリ {#insert-queries} - -`INSERT` クエリでは、データ送信に `POST` メソッドが必要です。この場合、クエリの先頭部分を URL パラメータに記述し、挿入するデータ本体を POST メソッドで送信できます。挿入するデータとしては、例えば MySQL からのタブ区切りダンプなどが利用できます。この方法では、`INSERT` クエリによって MySQL の `LOAD DATA LOCAL INFILE` と同等の処理を行えます。 - -### 例 {#examples} - -テーブルを作成するには: - -```bash -$ echo 'CREATE TABLE t (a UInt8) ENGINE = Memory' | curl 'http://localhost:8123/' --data-binary @- -``` - -使い慣れた `INSERT` クエリでデータを挿入するには、次のようにします。 - -```bash -$ echo 'INSERT INTO t VALUES (1),(2),(3)' | curl 'http://localhost:8123/' --data-binary @- -``` - -クエリとは別にデータを送信するには、次のようにします。 - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -任意のデータフォーマットを指定できます。たとえば、`INSERT INTO t VALUES` と記述するときに使用するものと同じ「Values」フォーマットを指定できます。 - -```bash -$ echo '(7),(8),(9)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20Values' --data-binary @- -``` - -タブ区切りダンプからデータを挿入するには、対応するフォーマットを指定してください。 - -```bash -$ echo -ne '10\n11\n12\n' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20TabSeparated' --data-binary @- -``` - -テーブルの内容を確認するには、次のようにします: - -```bash -$ curl 'http://localhost:8123/?query=SELECT%20a%20FROM%20t' -7 -8 -9 -10 -11 -12 -1 -2 -3 -4 -5 -6 -``` - -:::note -並列クエリ処理のため、データはランダムな順序で出力されます -::: - -テーブルを削除するには: - -```bash -$ echo 'DROP TABLE t' | curl 'http://localhost:8123/' --data-binary @- -``` - -データテーブルを返さない成功したリクエストでは、空のレスポンスボディが返されます。 - - -## 圧縮 {#compression} - -大量のデータを送信する際のネットワークトラフィックを削減したり、その場で圧縮済みのダンプを作成したりするために、圧縮を使用できます。 - -データ送信時に、ClickHouse の内部圧縮フォーマットを使用できます。圧縮されたデータは独自フォーマットであり、これを扱うには `clickhouse-compressor` プログラムが必要です。これは `clickhouse-client` パッケージと共にデフォルトでインストールされます。 - -データ挿入の効率を高めるには、[`http_native_compression_disable_checksumming_on_decompress`](../operations/settings/settings.md#http_native_compression_disable_checksumming_on_decompress) 設定を使用して、サーバー側のチェックサム検証を無効にします。 - -URL に `compress=1` を指定すると、サーバーは送信するデータを圧縮します。URL に `decompress=1` を指定すると、サーバーは `POST` メソッドで送信したデータを解凍します。 - -[HTTP 圧縮](https://en.wikipedia.org/wiki/HTTP_compression) を使用することもできます。ClickHouse は次の [圧縮方式](https://en.wikipedia.org/wiki/HTTP_compression#Content-Encoding_tokens) をサポートしています。 - -- `gzip` -- `br` -- `deflate` -- `xz` -- `zstd` -- `lz4` -- `bz2` -- `snappy` - -圧縮された `POST` リクエストを送信するには、リクエストヘッダーに `Content-Encoding: compression_method` を追加します。 - -ClickHouse にレスポンスを圧縮させるには、リクエストに `Accept-Encoding: compression_method` ヘッダーを追加します。 - -すべての圧縮方式に対して、[`http_zlib_compression_level`](../operations/settings/settings.md#http_zlib_compression_level) 設定を使用してデータ圧縮レベルを設定できます。 - -:::info -一部の HTTP クライアントは、デフォルトで(`gzip` および `deflate` を用いて)サーバーからのデータを自動的に解凍する場合があり、その場合は圧縮設定を正しく使用していても、解凍済みのデータを受け取ることがあります。 -::: - - - -## 例 {#examples-compression} - -圧縮データをサーバーに送信するには: - -```bash -echo "SELECT 1" | gzip -c | \ -curl -sS --data-binary @- -H 'Content-Encoding: gzip' 'http://localhost:8123/' -``` - -サーバーから圧縮されたデータアーカイブを受信するには: - -```bash -curl -vsS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' --output result.gz -d 'SELECT number FROM system.numbers LIMIT 3' - -zcat result.gz -0 -1 -2 -``` - -サーバーから圧縮データを受信し、展開されたデータを取得するには、gunzip を使用します。 - -```bash -curl -sS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' -d 'SELECT number FROM system.numbers LIMIT 3' | gunzip - -0 -1 -2 -``` - - -## デフォルトデータベース {#default-database} - -デフォルトデータベースを指定するには、`database` URL パラメータまたは `X-ClickHouse-Database` ヘッダーを使用できます。 - -```bash -echo 'SELECT number FROM numbers LIMIT 10' | curl 'http://localhost:8123/?database=system' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -既定では、サーバー設定で登録されているデータベースが既定のデータベースとして使用されます。インストール直後の状態では、これは `default` という名前のデータベースです。あるいは、テーブル名の前にドットを付けてデータベース名を明示的に指定することもできます。 - - -## 認証 {#authentication} - -ユーザー名とパスワードは、次の3つの方法のいずれかで指定できます。 - -1. HTTP Basic 認証を使用する。 - -例: - -```bash -echo 'SELECT 1' | curl 'http://user:password@localhost:8123/' -d @- -``` - -2. `user` および `password` を URL パラメータで指定する方法 - -:::warning -パラメータが Web プロキシでログに記録されたり、ブラウザにキャッシュされたりする可能性があるため、この方法は推奨しません -::: - -例: - -```bash -echo 'SELECT 1' | curl 'http://localhost:8123/?user=user&password=password' -d @- -``` - -3. 「X-ClickHouse-User」ヘッダーと「X-ClickHouse-Key」ヘッダーを使用する - -例: - -```bash -echo 'SELECT 1' | curl -H 'X-ClickHouse-User: user' -H 'X-ClickHouse-Key: password' 'http://localhost:8123/' -d @- -``` - -ユーザー名が指定されていない場合は、`default` というユーザー名が使用されます。パスワードが指定されていない場合は、空のパスワードが使用されます。 -単一のクエリや設定プロファイル全体に対する任意の設定を指定するために、URL パラメータを使用することもできます。 - -例: - -```text -http://localhost:8123/?profile=web&max_rows_to_read=1000000000&query=SELECT+1 -``` - -```bash -$ echo 'SELECT number FROM system.numbers LIMIT 10' | curl 'http://localhost:8123/?' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -詳細については、以下を参照してください。 - -* [設定](/operations/settings/settings) -* [SET](/sql-reference/statements/set) - - -## HTTP プロトコルでの ClickHouse セッションの使用 {#using-clickhouse-sessions-in-the-http-protocol} - -ClickHouse セッションは HTTP プロトコルでも使用できます。そのためには、リクエストに `session_id` の `GET` パラメータを追加する必要があります。セッション ID には任意の文字列を指定できます。 - -デフォルトでは、セッションは 60 秒間アクティビティがないと終了します。このタイムアウト値(秒)を変更するには、サーバー設定の `default_session_timeout` を変更するか、リクエストに `session_timeout` の `GET` パラメータを追加します。 - -セッションの状態を確認するには、`session_check=1` パラメータを指定します。1 つのセッション内で同時に実行できるクエリは 1 つだけです。 - -クエリの進行状況に関する情報は、`X-ClickHouse-Progress` レスポンスヘッダーで受け取ることができます。そのためには、[`send_progress_in_http_headers`](../operations/settings/settings.md#send_progress_in_http_headers) を有効にします。 - -以下にヘッダーのシーケンス例を示します。 - -```text -X-ClickHouse-Progress: {"read_rows":"261636","read_bytes":"2093088","total_rows_to_read":"1000000","elapsed_ns":"14050417","memory_usage":"22205975"} -X-ClickHouse-Progress: {"read_rows":"654090","read_bytes":"5232720","total_rows_to_read":"1000000","elapsed_ns":"27948667","memory_usage":"83400279"} -X-ClickHouse-Progress: {"read_rows":"1000000","read_bytes":"8000000","total_rows_to_read":"1000000","elapsed_ns":"38002417","memory_usage":"80715679"} -``` - -利用可能なヘッダーフィールドは次のとおりです。 - -| Header field | Description | -| -------------------- | ------------------- | -| `read_rows` | 読み込まれた行数。 | -| `read_bytes` | バイト単位での読み込みデータ量。 | -| `total_rows_to_read` | 読み取る予定の総行数。 | -| `written_rows` | 書き込まれた行数。 | -| `written_bytes` | バイト単位での書き込みデータ量。 | -| `elapsed_ns` | ナノ秒単位のクエリ実行時間。 | -| `memory_usage` | クエリで使用されたメモリ量(バイト)。 | - -実行中のリクエストは、HTTP 接続が失われても自動的には停止しません。パース処理とデータのフォーマットはサーバー側で実行されるため、ネットワークの利用が非効率になる場合があります。 - -次のオプションパラメータが存在します。 - -| Parameters | Description | -| ---------------------- | ----------------------------------------------------------------------------------------------------------- | -| `query_id` (optional) | クエリ ID として渡すことができます(任意の文字列)。 [`replace_running_query`](/operations/settings/settings#replace_running_query) | -| `quota_key` (optional) | クオータキーとして渡すことができます(任意の文字列)。 [「Quotas」](/operations/quotas) | - -HTTP インターフェイスでは、クエリ用に外部データ(外部一時テーブル)を渡すことができます。詳細は [「External data for query processing」](/engines/table-engines/special/external-data) を参照してください。 - - -## レスポンスのバッファリング {#response-buffering} - -レスポンスのバッファリングはサーバー側で有効化できます。このために、次の URL パラメータが用意されています。 - -* `buffer_size` -* `wait_end_of_query` - -次の設定を使用できます。 - -* [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size) -* [`http_wait_end_of_query`](/operations/settings/settings#http_wait_end_of_query) - -`buffer_size` は、サーバーメモリ内でバッファリングする結果のバイト数を指定します。レスポンスボディがこの閾値より大きい場合、バッファは HTTP チャネルに書き出され、残りのデータは直接 HTTP チャネルに送信されます。 - -レスポンス全体を確実にバッファリングするには、`wait_end_of_query=1` を設定します。この場合、メモリに保持されないデータはサーバー上の一時ファイルにバッファリングされます。 - -例: - -```bash -curl -sS 'http://localhost:8123/?max_result_bytes=4000000&buffer_size=3000000&wait_end_of_query=1' -d 'SELECT toUInt8(number) FROM system.numbers LIMIT 9000000 FORMAT RowBinary' -``` - -:::tip -バッファリングを使用して、レスポンスコードおよび HTTP ヘッダーがクライアントに送信された後にクエリの処理エラーが発生する状況を回避してください。このような場合、エラーメッセージはレスポンスボディの末尾に書き込まれ、クライアント側ではパース処理の段階になって初めてエラーを検知できます。 -::: - - -## クエリパラメーターを使用してロールを設定する {#setting-role-with-query-parameters} - -この機能は ClickHouse 24.4 で追加されました。 - -特定のケースでは、ステートメント自体を実行する前に、付与されたロールを先に設定する必要がある場合があります。 -ただし、マルチステートメントは許可されていないため、`SET ROLE` とステートメントをまとめて送信することはできません。 - -```bash -curl -sS "http://localhost:8123" --data-binary "SET ROLE my_role;SELECT * FROM my_table;" -``` - -上記のコマンドを実行すると、エラーが発生します。 - -```sql -コード: 62. DB::Exception: 構文エラー (複数のステートメントは許可されていません) -``` - -この制限を回避するには、代わりに `role` クエリパラメータを使用してください。 - -```bash -curl -sS "http://localhost:8123?role=my_role" --data-binary "SELECT * FROM my_table;" -``` - -これは、ステートメントの前に `SET ROLE my_role` を実行するのと同じ意味になります。 - -また、`role` クエリパラメータを複数指定することもできます。 - -```bash -curl -sS "http://localhost:8123?role=my_role&role=my_other_role" --data-binary "SELECT * FROM my_table;" -``` - -この場合、`?role=my_role&role=my_other_role` は、ステートメントを実行する前に `SET ROLE my_role, my_other_role` を実行した場合と同様に動作します。 - - -## HTTP レスポンスコードに関する注意点 {#http_response_codes_caveats} - -HTTP プロトコルの制約上、HTTP 200 のレスポンスコードであっても、クエリが成功したことは保証されません。 - -以下に例を示します。 - -```bash -curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)" -* Trying 127.0.0.1:8123... -... -< HTTP/1.1 200 OK -... -Code: 395. DB::Exception: 'throwIf' 関数に渡された値が非ゼロです: 'FUNCTION throwIf(equals(number, 2) :: 1) -> throwIf(equals(number, 2))' の実行中 -``` - -この動作が発生する理由は、HTTP プロトコルの性質によるものです。まず HTTP ヘッダーが HTTP ステータスコード 200 とともに送信され、その後に HTTP ボディが続き、そのボディの中にエラーがプレーンテキストとして差し込まれます。 - -この動作は使用されるフォーマット、つまり `Native`、`TSV`、`JSON` のいずれであっても変わらず、エラーメッセージは常にレスポンスストリームの途中に現れます。 - -この問題は、`wait_end_of_query=1`([Response Buffering](#response-buffering))を有効にすることで軽減できます。この場合、クエリ全体が完了するまで HTTP ヘッダーの送信が遅延されます。ただし、この方法でも問題は完全には解決されません。というのも、結果は依然として [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size) の範囲内に収まる必要があり、[`send_progress_in_http_headers`](/operations/settings/settings#send_progress_in_http_headers) などの他の設定がヘッダー送信の遅延と干渉し得るためです。 - -:::tip -すべてのエラーを確実に検出する唯一の方法は、必要なフォーマットでパースする前に HTTP ボディを解析することです。 -::: - -ClickHouse におけるこのような例外は、`http_write_exception_in_output_format=0`(デフォルト)の場合、使用されるフォーマット(`Native`、`TSV`、`JSON` など)に関係なく、以下のように一貫した例外フォーマットを持ちます。これにより、クライアント側でエラーメッセージをパースおよび抽出しやすくなります。 - -```text -\r\n -__exception__\r\n -\r\n -\r\n - \r\n -__exception__\r\n - -``` - -ここで `` は 16 バイトのランダムなタグであり、`X-ClickHouse-Exception-Tag` レスポンスヘッダーで送信されるタグと同じです。 -`` は実際の例外メッセージです(正確な長さは `` で確認できます)。上で説明した例外ブロック全体のサイズは最大 16 KiB です。 - -`JSON` 形式での例を次に示します。 - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+JSON" -... -{ - "meta": - [ - { - "name": "sleepEachRow(0.001)", - "type": "UInt8" - }, - { - "name": "throwIf(equals(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - }, - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - } -__exception__ -dmrdfnujjqvszhav -Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 25.11.1.1) -262 dmrdfnujjqvszhav -__exception__ -``` - -こちらは `CSV` 形式の同様の例です - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+CSV" -... -< -0,0 -0,0 -``` - - -**例外** -rumfyutuqkncbgau -Code: 395. DB::Exception: `throwIf` 関数に渡された値がゼロ以外です: `FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0` を実行中に発生しました。 (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 25.11.1.1) -262 rumfyutuqkncbgau -**例外** - -``` -``` - - -## パラメーター付きクエリ {#cli-queries-with-parameters} - -パラメーター付きのクエリを作成し、対応する HTTP リクエストのパラメーターから値を渡すことができます。詳細については、[CLI 向けパラメーター付きクエリ](../interfaces/cli.md#cli-queries-with-parameters)を参照してください。 - -### 例 {#example-3} - -```bash -$ curl -sS "
?param_id=2¶m_phrase=test" -d "SELECT * FROM table WHERE int_column = {id:UInt8} and string_column = {phrase:String}" -``` - -### URL パラメータ内のタブ文字 {#tabs-in-url-parameters} - -クエリパラメータは「エスケープ」形式から解析されます。これには、`\N` を null としてあいまいさなく解析できるといった利点があります。これは、タブ文字は `\t`(または `\` とタブ文字)としてエンコードする必要があることを意味します。たとえば、次の例では `abc` と `123` の間に実際のタブ文字が含まれており、入力文字列は 2 つの値に分割されます。 - -```bash -curl -sS "http://localhost:8123" -d "SELECT splitByChar('\t', 'abc 123')" -``` - -```response -['abc','123'] -``` - -しかし、URL パラメータで実際のタブ文字を `%09` としてエンコードしても、正しく解釈されません。 - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%09123" -d "SELECT splitByChar('\t', {arg1:String})" -Code: 457. DB::Exception: クエリパラメータ 'arg1' の値 abc 123 を String として解析できません。完全に解析されていないため: 7バイト中3バイトのみ解析されました: abc。(BAD_QUERY_PARAMETER) (version 23.4.1.869 (official build)) -``` - -URL パラメータを使用する場合は、`\t` を `%5C%09` にエンコードする必要があります。例: - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%5C%09123" -d "SELECT splitByChar('\t', {arg1:String})" -``` - -```response -['abc','123'] -``` - - -## あらかじめ定義された HTTP インターフェイス {#predefined_http_interface} - -ClickHouse は、HTTP インターフェイス経由で特定のクエリをサポートしています。たとえば、次のようにテーブルにデータを書き込むことができます。 - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -ClickHouse は、[Prometheus exporter](https://github.com/ClickHouse/clickhouse_exporter) のようなサードパーティツールとの連携を容易にする Predefined HTTP Interface もサポートしています。例を見てみましょう。 - -まず、このセクションをサーバー設定ファイルに追加します。 - -`http_handlers` には複数の `rule` を含めるように設定します。ClickHouse は受信した HTTP リクエストを `rule` で定義されたタイプと照合し、最初にマッチした `rule` のハンドラーが実行されます。その後、マッチに成功すると、ClickHouse は対応する事前定義クエリを実行します。 - -```yaml title="config.xml" - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.metrics LIMIT 5 FORMAT Template SETTINGS format_template_resultset = 'prometheus_template_output_format_resultset', format_template_row = 'prometheus_template_output_format_row', format_template_rows_between_delimiter = '\n' - - - ... - ... - -``` - -これで、Prometheus 形式のデータを取得するための URL を直接リクエストできます。 - - -```bash -$ curl -v 'http://localhost:8123/predefined_query' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /predefined_query HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> -< HTTP/1.1 200 OK -< Date: Tue, 28 Apr 2020 08:52:56 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< X-ClickHouse-Server-Display-Name: i-mloy5trc -< Transfer-Encoding: chunked -< X-ClickHouse-Query-Id: 96fe0052-01e6-43ce-b12a-6b7370de6e8a -< X-ClickHouse-Format: Template -< X-ClickHouse-Timezone: Asia/Shanghai -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -# HELP "Query" "Number of executing queries" {#help-query-number-of-executing-queries} -# TYPE "Query" counter {#type-query-counter} -"Query" 1 -``` - - -# HELP "Merge" "実行中のバックグラウンドマージ数" {#help-merge-number-of-executing-background-merges} -# TYPE "Merge" counter {#type-merge-counter} -"Merge" 0 - - - -# HELP "PartMutation" "ミューテーション数 (ALTER DELETE/UPDATE)" {#help-partmutation-number-of-mutations-alter-deleteupdate} -# TYPE "PartMutation" counter {#type-partmutation-counter} -"PartMutation" 0 - - - -# HELP "ReplicatedFetch" "レプリカから取得中のデータパーツ数" {#help-replicatedfetch-number-of-data-parts-being-fetched-from-replica} -# TYPE "ReplicatedFetch" counter {#type-replicatedfetch-counter} -"ReplicatedFetch" 0 - - - -# HELP "ReplicatedSend" "レプリカへ送信中のデータパーツ数" {#help-replicatedsend-number-of-data-parts-being-sent-to-replicas} - -# TYPE "ReplicatedSend" counter {#type-replicatedsend-counter} - -"ReplicatedSend" 0 - -* ホスト localhost への接続 #0 はそのまま維持されています - -* ホスト localhost への接続 #0 はそのまま維持されています - -``` - -`http_handlers`の設定オプションは以下のように動作します。 - -`rule`では以下のパラメータを設定できます: -- `method` -- `headers` -- `url` -- `full_url` -- `handler` - -各パラメータについて以下で説明します: - -- `method`はHTTPリクエストのメソッド部分のマッチングを担当します。`method`はHTTPプロトコルにおける[`method`] - (https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods)の定義に完全に準拠しています。これはオプションの設定です。設定ファイルで定義されていない場合、HTTPリクエストのメソッド部分とはマッチングしません。 - -- `url`はHTTPリクエストのURL部分(パスとクエリ文字列)のマッチングを担当します。 - `url`に`regex:`のプレフィックスが付いている場合、[RE2](https://github.com/google/re2)の正規表現を使用します。 - これはオプションの設定です。設定ファイルで定義されていない場合、HTTPリクエストのURL部分とはマッチングしません。 - -- `full_url`は`url`と同様ですが、完全なURL、すなわち`schema://host:port/path?query_string`を含みます。 - 注意: ClickHouseは「仮想ホスト」をサポートしていないため、`host`はIPアドレスです(`Host`ヘッダーの値ではありません)。 - -- `empty_query_string` - リクエストにクエリ文字列(`?query_string`)が存在しないことを保証します - -- `headers`はHTTPリクエストのヘッダー部分のマッチングを担当します。RE2の正規表現と互換性があります。これはオプションの設定です。設定ファイルで定義されていない場合、HTTPリクエストのヘッダー部分とはマッチングしません。 - -- `handler`はメイン処理部分を含みます。 - - 以下の`type`を指定できます: - - [`predefined_query_handler`](#predefined_query_handler) - - [`dynamic_query_handler`](#dynamic_query_handler) - - [`static`](#static) - - [`redirect`](#redirect) - - また、以下のパラメータを指定できます: - - `query` — `predefined_query_handler`タイプで使用し、ハンドラーが呼び出されたときにクエリを実行します。 - - `query_param_name` — `dynamic_query_handler`タイプで使用し、HTTPリクエストパラメータ内の`query_param_name`値に対応する値を抽出して実行します。 - - `status` — `static`タイプで使用し、レスポンスステータスコードを指定します。 - - `content_type` — 任意のタイプで使用し、レスポンスの[content-type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type)を指定します。 - - `http_response_headers` — 任意のタイプで使用し、レスポンスヘッダーマップを指定します。コンテンツタイプの設定にも使用できます。 - - `response_content` — `static`タイプで使用し、クライアントに送信されるレスポンスコンテンツを指定します。プレフィックス'file://'または'config://'を使用する場合、ファイルまたは設定からコンテンツを取得してクライアントに送信します。 - - `user` - クエリを実行するユーザー(デフォルトユーザーは`default`)。 - **注意**: このユーザーのパスワードを指定する必要はありません。 - -異なる`type`の設定方法について次に説明します。 - -### predefined_query_handler {#predefined_query_handler} - -`predefined_query_handler`は`Settings`と`query_params`の値の設定をサポートします。`predefined_query_handler`タイプで`query`を設定できます。 - -`query`値は`predefined_query_handler`の事前定義されたクエリであり、HTTPリクエストがマッチしたときにClickHouseによって実行され、クエリの結果が返されます。これは必須の設定です。 - -以下の例では、[`max_threads`](../operations/settings/settings.md#max_threads)と[`max_final_threads`](/operations/settings/settings#max_final_threads)設定の値を定義し、その後システムテーブルをクエリしてこれらの設定が正常に設定されたかを確認します。 - -:::note -`query`、`play`、`ping`などのデフォルトの`handlers`を保持するには、``ルールを追加してください。 -::: - -例: -``` - - -```yaml - - - [^/]+)]]> - GET - - TEST_HEADER_VALUE - [^/]+)]]> - - - predefined_query_handler - - SELECT name, value FROM system.settings - WHERE name IN ({name_1:String}, {name_2:String}) - - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE' -H 'PARAMS_XXX:max_final_threads' 'http://localhost:8123/query_param_with_url/max_threads?max_threads=1&max_final_threads=2' -max_final_threads 2 -max_threads 1 -``` - -:::note -1つの `predefined_query_handler` では、1つの `query` のみがサポートされます。 -::: - -### dynamic_query_handler {#dynamic_query_handler} - -`dynamic_query_handler` では、クエリは HTTP リクエストのパラメータとして記述されます。`predefined_query_handler` との違いは、後者ではクエリが設定ファイル内に記述される点です。`query_param_name` は `dynamic_query_handler` 内で設定できます。 - -ClickHouse は、HTTP リクエストの URL 内で `query_param_name` に対応する値を抽出して実行します。`query_param_name` のデフォルト値は `/query` です。これは省略可能な設定項目です。設定ファイル内に定義がない場合は、パラメータは渡されません。 - -この機能を試すために、次の例では [`max_threads`](../operations/settings/settings.md#max_threads) と `max_final_threads` の値を定義し、さらに設定が正しく反映されたかどうかを確認する `query` を実行します。 - -例: - -```yaml - - - - TEST_HEADER_VALUE_DYNAMIC - - dynamic_query_handler - query_param - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE_DYNAMIC' 'http://localhost:8123/own?max_threads=1&max_final_threads=2¶m_name_1=max_threads¶m_name_2=max_final_threads&query_param=SELECT%20name,value%20FROM%20system.settings%20where%20name%20=%20%7Bname_1:String%7D%20OR%20name%20=%20%7Bname_2:String%7D' -max_threads 1 -max_final_threads 2 -``` - -### static {#static} - -`static` は [`content_type`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type)、[status](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status)、および `response_content` を返すことができます。`response_content` で指定したコンテンツを返せます。 - -たとえば、"Say Hi!" というメッセージを返すには次のようにします。 - -```yaml - - - GET - xxx - /hi - - static - 402 - text/html; charset=UTF-8 - - en - 43 - - #highlight-next-line - Say Hi! - - - - -``` - -`content_type` の代わりに `http_response_headers` を使用して Content-Type を設定できます。 - - -```yaml - - - GET - xxx - /hi - - static - 402 - #begin-highlight - - text/html; charset=UTF-8 - en - 43 - - #end-highlight - こんにちは! - - - - -``` - -```bash -curl -vv -H 'XXX:xxx' 'http://localhost:8123/hi' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /hi HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 402 Payment Required -< Date: Wed, 29 Apr 2020 03:51:26 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -Say Hi!% -``` - -クライアントに送信される設定内容を特定します。 - -```yaml -
]]>
- - - - GET - xxx - /get_config_static_handler - - static - config://get_config_static_handler - - - -``` - -```bash -$ curl -v -H 'XXX:xxx' 'http://localhost:8123/get_config_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_config_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:01:24 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -
% -``` - -クライアントに送信したファイル内の内容を確認するには、次のようにします。 - - -```yaml - - - GET - xxx - /get_absolute_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file:///absolute_path_file.html - - - - GET - xxx - /get_relative_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file://./relative_path_file.html - - - -``` - -```bash -$ user_files_path='/var/lib/clickhouse/user_files' -$ sudo echo "相対パスファイル" > $user_files_path/relative_path_file.html -$ sudo echo "絶対パスファイル" > $user_files_path/absolute_path_file.html -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_absolute_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_absolute_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:16 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -絶対パスファイル -* Connection #0 to host localhost left intact -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_relative_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_relative_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:31 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -相対パスファイル -* Connection #0 to host localhost left intact -``` - -### redirect {#redirect} - -`redirect` は `location` へ `302` リダイレクトを行います。 - -例えば、ClickHouse play でユーザーを自動的に `play` に設定するには次のようにします。 - -```xml - - - - GET - /play - - redirect - /play?user=play - - - - -``` - - -## HTTP レスポンスヘッダー {#http-response-headers} - -ClickHouse では、設定可能なあらゆる種類のハンドラーに適用できるカスタム HTTP レスポンスヘッダーを設定できます。これらのヘッダーは、ヘッダー名とその値を表すキーと値のペアを指定する `http_response_headers` 設定を使用して設定します。この機能は、カスタムセキュリティヘッダーや CORS ポリシー、その他 ClickHouse の HTTP インターフェイス全体で必要となる HTTP ヘッダー要件を実装するのに特に有用です。 - -たとえば、次のような対象にヘッダーを設定できます: - -* 通常のクエリエンドポイント -* Web UI -* ヘルスチェック - -また、`common_http_response_headers` を指定することも可能です。これらは、設定で定義されたすべての HTTP ハンドラーに適用されます。 - -ヘッダーは、設定されたすべてのハンドラーに対する HTTP レスポンスに含まれます。 - -以下の例では、すべてのサーバーレスポンスに `X-My-Common-Header` と `X-My-Custom-Header` という 2 つのカスタムヘッダーが含まれます。 - -```xml - - - - 共通ヘッダー - - - GET - /ping - - ping - - カスタムヘッダー - - - - - -``` - - -## HTTP ストリーミング中の例外発生時における有効な JSON/XML レスポンス {#valid-output-on-exception-http-streaming} - -クエリが HTTP 経由で実行されている間に、データの一部がすでに送信された後で例外が発生することがあります。通常、例外はプレーンテキストとしてクライアントに送信されます。 -特定のデータフォーマットを使用してデータを出力している場合、そのフォーマットの観点から出力が不正になってしまう可能性があります。 -これを防ぐには、ClickHouse に例外を指定したフォーマットで書き出すよう指示する設定 [`http_write_exception_in_output_format`](/operations/settings/settings#http_write_exception_in_output_format)(デフォルトでは無効)を使用できます(現在は XML および JSON* フォーマットでサポートされています)。 - -例: - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>3)+from+system.numbers+format+JSON+settings+max_block_size=1&http_write_exception_in_output_format=1' -{ - "meta": - [ - { - "name": "number", - "type": "UInt64" - }, - { - "name": "throwIf(greater(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "number": "0", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "1", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "2", - "throwIf(greater(number, 2))": 0 - } - ], - - "rows": 3, - - "exception": "Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 23.8.1.1)" -} -``` - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>2)+from+system.numbers+format+XML+settings+max_block_size=1&http_write_exception_in_output_format=1' - - - - - - number - UInt64 - - - throwIf(greater(number, 2)) - UInt8 - - - - - - 0 - 0 - - - 1 - 0 - - - 2 - 0 - - - 3 - Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 23.8.1.1) - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/jdbc.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/jdbc.md deleted file mode 100644 index bc8b46d40c8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/jdbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Java アプリケーションから ClickHouse に接続するための JDBC ドライバー利用ガイド' -sidebar_label: 'JDBC ドライバー' -sidebar_position: 20 -slug: /interfaces/jdbc -title: 'JDBC ドライバー' -doc_type: 'guide' ---- - -# JDBC ドライバー {#jdbc-driver} - -Java アプリケーションから ClickHouse にアクセスするには、[公式 JDBC ドライバー](/docs/integrations/language-clients/java/jdbc)(および Java クライアント)を使用してください。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/mysql.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/mysql.md deleted file mode 100644 index 0ad531b1944..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/mysql.md +++ /dev/null @@ -1,181 +0,0 @@ ---- -description: 'ClickHouse の MySQL プロトコルインターフェイスに関するドキュメントです。MySQL クライアントから ClickHouse に接続できます' -sidebar_label: 'MySQL インターフェイス' -sidebar_position: 25 -slug: /interfaces/mysql -title: 'MySQL インターフェイス' -doc_type: 'guide' ---- - -import Image from '@theme/IdealImage'; -import mysql0 from '@site/static/images/interfaces/mysql0.png'; -import mysql1 from '@site/static/images/interfaces/mysql1.png'; -import mysql2 from '@site/static/images/interfaces/mysql2.png'; -import mysql3 from '@site/static/images/interfaces/mysql3.png'; - - -# MySQL インターフェイス {#mysql-interface} - -ClickHouse は MySQL ワイヤープロトコルをサポートしています。これにより、ネイティブな ClickHouse コネクタを持たない一部のクライアントでも MySQL プロトコルを代わりに利用でき、次の BI ツールで動作検証が行われています: - -- [Looker Studio](../integrations/data-visualization/looker-studio-and-clickhouse.md) -- [Tableau Online](../integrations/tableau-online) -- [QuickSight](../integrations/quicksight) - -他の未検証のクライアントやインテグレーションを試す場合は、次のような制限があり得ることに留意してください: - -- SSL 実装が完全には互換でない可能性があり、[TLS SNI](https://www.cloudflare.com/learning/ssl/what-is-sni/) に関連する問題が発生する場合があります。 -- 特定のツールが、まだ実装されていない方言機能(例: MySQL 固有の関数や設定)を必要とする可能性があります。 - -ネイティブドライバが利用可能な場合(例: [DBeaver](../integrations/dbeaver))は、MySQL インターフェイスではなく、常にネイティブドライバを使用することを推奨します。さらに、ほとんどの MySQL 言語クライアントは問題なく動作するはずですが、MySQL インターフェイスが既存の MySQL クエリを持つコードベースに対するドロップインの代替となることは保証されません。 - -特定のツールにネイティブな ClickHouse ドライバがなく、MySQL インターフェイス経由で利用したいものの、何らかの非互換性を見つけた場合は、ClickHouse リポジトリで[Issue を作成](https://github.com/ClickHouse/ClickHouse/issues)してください。 - -::::note -上記の BI ツールの SQL 方言をより適切にサポートするため、ClickHouse の MySQL インターフェイスは、設定 [prefer_column_name_to_alias = 1](/operations/settings/settings#prefer_column_name_to_alias) を有効にした状態で SELECT クエリを暗黙的に実行します。 -この設定は無効化できず、まれなエッジケースでは、ClickHouse の通常のクエリインターフェイスと MySQL クエリインターフェイスに送信されたクエリの間で挙動が異なる原因となる場合があります。 -:::: - - - -## ClickHouse Cloud での MySQL インターフェイスの有効化 {#enabling-the-mysql-interface-on-clickhouse-cloud} - -1. ClickHouse Cloud サービスを作成したら、`Connect` ボタンをクリックします。 - -
- -認証情報画面 - プロンプト - -2. `Connect with` のドロップダウンを `MySQL` に変更します。 - -
- -認証情報画面 - MySQL を選択 - -3. このサービスで MySQL インターフェイスを有効にするためにスイッチをオンにします。これにより、このサービスでポート `3306` が公開され、一意の MySQL ユーザー名を含む MySQL 接続画面が表示されます。パスワードはサービスのデフォルトユーザーのパスワードと同一です。 - -
- -認証情報画面 - MySQL を有効化 - -表示されている MySQL 接続文字列をコピーします。 - -認証情報画面 - 接続文字列 - - - -## ClickHouse Cloud で複数の MySQL ユーザーを作成する {#creating-multiple-mysql-users-in-clickhouse-cloud} - -デフォルトでは、組み込みの `mysql4` ユーザーが存在し、このユーザーは `default` ユーザーと同じパスワードを使用します。`` 部分は、ClickHouse Cloud ホスト名の先頭のセグメントです。この形式は、安全な接続を実装しているものの [TLS ハンドシェイクで SNI 情報を提供しない](https://www.cloudflare.com/learning/ssl/what-is-sni) ツールと連携するために必要であり、ユーザー名に追加のヒントがないと内部ルーティングができないためです(MySQL コンソールクライアントはそのようなツールの一例です)。 - -このため、MySQL インターフェイスで使用する新しいユーザーを作成する際は、`mysql4_` という形式に従うことを*強く推奨*します。ここで、`` は Cloud サービスを識別するためのヒントであり、`` は任意のサフィックスです。 - -:::tip -`foobar.us-east1.aws.clickhouse.cloud` のような ClickHouse Cloud ホスト名の場合、`` 部分は `foobar` に相当し、カスタム MySQL ユーザー名は `mysql4foobar_team1` のようになります。 -::: - -たとえば、追加の設定を適用する必要がある場合などに、MySQL インターフェイスで使用する追加ユーザーを作成できます。 - -1. (任意)カスタムユーザーに適用する [settings profile](/sql-reference/statements/create/settings-profile) を作成します。たとえば、後で作成するユーザーで接続した際にデフォルトで適用される追加設定を持つ `my_custom_profile` を作成します: - - ```sql - CREATE SETTINGS PROFILE my_custom_profile SETTINGS prefer_column_name_to_alias=1; - ``` - - `prefer_column_name_to_alias` はあくまで例であり、ここには他の設定を指定できます。 - -2. 次の形式で [ユーザーを作成](/sql-reference/statements/create/user) します: `mysql4_`([前述](#creating-multiple-mysql-users-in-clickhouse-cloud) 参照)。パスワードは double SHA1 形式である必要があります。例: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$'; - ``` - - または、このユーザーに対してカスタムプロファイルを使用したい場合: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$' SETTINGS PROFILE 'my_custom_profile'; - ``` - - ここで、`my_custom_profile` は先ほど作成したプロファイル名です。 - -3. 新しいユーザーに対して、対象とするテーブルやデータベースとやり取りするために必要な権限を [付与](/sql-reference/statements/grant) します。たとえば、`system.query_log` のみにアクセス権を与えたい場合: - - ```sql - GRANT SELECT ON system.query_log TO mysql4foobar_team1; - ``` - -4. 作成したユーザーを使用して、MySQL インターフェイス経由で ClickHouse Cloud サービスに接続します。 - -### ClickHouse Cloud での複数 MySQL ユーザーに関するトラブルシューティング {#troubleshooting-multiple-mysql-users-in-clickhouse-cloud} - -新しい MySQL ユーザーを作成し、MySQL CLI クライアント経由で接続する際に次のエラーが表示される場合があります: - -```sql -ERROR 2013 (HY000): MySQLサーバーへの接続が切断されました at 'reading authorization packet', system error: 54 -``` - -この場合は、ユーザー名が `mysql4<subdomain>_<username>` という形式([上記](#creating-multiple-mysql-users-in-clickhouse-cloud)で説明したとおり)になっていることを確認してください。 - - -## セルフマネージド ClickHouse での MySQL インターフェイスの有効化 {#enabling-the-mysql-interface-on-self-managed-clickhouse} - -サーバーの構成ファイルに [mysql_port](../operations/server-configuration-parameters/settings.md#mysql_port) 設定を追加します。たとえば、`config.d/` [ディレクトリ](../operations/configuration-files) 内の新しい XML ファイルでこのポートを定義できます。 - -```xml - - 9004 - -``` - -ClickHouse サーバーを起動し、`Listening for MySQL compatibility protocol` というメッセージを含む、次のようなログメッセージを探します。 - -```bash -{} Application: MySQL互換プロトコルでリッスン中: 127.0.0.1:9004 -``` - - -## MySQL を ClickHouse に接続する {#connect-mysql-to-clickhouse} - -次のコマンドは、MySQL クライアント `mysql` から ClickHouse へ接続する方法を示します。 - -```bash -mysql --protocol tcp -h [hostname] -u [username] -P [port_number] [database_name] -``` - -例えば: - -```bash -$ mysql --protocol tcp -h 127.0.0.1 -u default -P 9004 default -``` - -接続に成功した場合の出力: - -```text -MySQLモニターへようこそ。コマンドは ; または \g で終了します。 -MySQL接続IDは 4 です -サーババージョン: 20.2.1.1-ClickHouse - -Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. - -Oracleは、Oracle Corporationおよび/またはその関連会社の登録商標です。 -その他の名称は、それぞれの所有者の商標である可能性があります。 - -ヘルプを表示するには 'help;' または '\h' と入力してください。現在の入力文をクリアするには '\c' と入力してください。 - -mysql> -``` - -すべての MySQL クライアントとの互換性を確保するため、設定ファイルではユーザーのパスワードを [double SHA1](/operations/settings/settings-users#user-namepassword) で指定することを推奨します。 -ユーザーパスワードを [SHA256](/sql-reference/functions/hash-functions#SHA256) で指定した場合、一部のクライアント(mysqljs や、古いバージョンのコマンドラインツール MySQL および MariaDB)は認証に失敗します。 - -制限事項: - -* プリペアドステートメントはサポートされていません - -* 一部のデータ型は文字列として送信されます - -長時間実行されているクエリをキャンセルするには、`KILL QUERY connection_id` ステートメントを使用します(実行時には `KILL QUERY WHERE query_id = connection_id` に置き換えられます)。例: - -```bash -$ mysql --protocol tcp -h mysql_server -P 9004 default -u default --password=123 -e "KILL QUERY 123456;" -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md deleted file mode 100644 index 67e80421d94..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'ClickHouse 用のネイティブ クライアントとインターフェース' -keywords: ['クライアント', 'インターフェース', 'CLI', 'SQL コンソール', 'ドライバー'] -slug: /interfaces/natives-clients-and-interfaces -title: 'ネイティブ クライアントとインターフェース' -doc_type: 'landing-page' ---- - -# ネイティブクライアントとインターフェース {#native-clients-interfaces} - -ClickHouse には、接続のために利用できる複数のネイティブクライアントおよびインターフェースが用意されています。 - -詳細については、以下のページを参照してください。 - -| セクション | 概要 | -|--------------------------------------------------------------|-------------------------------------------------------------------------------------| -| [Command-Line Client](/interfaces/cli) | コマンドラインオプションおよび設定ファイルをサポートするネイティブなコマンドラインクライアント。 | -| [Drivers & Interfaces](/interfaces/overview) | 各種ネットワークインターフェース、ライブラリ、およびビジュアルインターフェース。 | -| [SQL Console](/integrations/sql-clients/sql-console) | ClickHouse Cloud のデータを操作するための、高速かつ簡便な方法。 | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/odbc.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/odbc.md deleted file mode 100644 index 356d7d28c67..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/odbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ClickHouse ODBC ドライバーに関するドキュメント' -sidebar_label: 'ODBC ドライバー' -sidebar_position: 35 -slug: /interfaces/odbc -title: 'ODBC ドライバー' -doc_type: 'reference' ---- - -# ODBC ドライバー {#odbc-driver} - -ClickHouse をデータソースとして利用するには、[公式 ODBC ドライバー](https://github.com/ClickHouse/clickhouse-odbc) を使用してください。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/overview.md deleted file mode 100644 index 35f66ed2393..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/overview.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'ClickHouse への接続用ネットワークインターフェース、ドライバー、およびツールの概要' -keywords: ['clickhouse', 'network', 'interfaces', 'http', 'tcp', 'grpc', 'command-line', - 'client', 'jdbc', 'odbc', 'driver'] -sidebar_label: '概要' -slug: /interfaces/overview -title: 'ドライバーとインターフェース' -doc_type: 'reference' ---- - -# ドライバとインターフェイス {#drivers-and-interfaces} - -ClickHouse は 2 つのネットワークインターフェイスを提供します(いずれも追加のセキュリティのためにオプションで TLS で保護できます)。 - -* [HTTP](http.md) — ドキュメントが整備されており、そのまま容易に利用できます。 -* [ネイティブ TCP](../interfaces/tcp.md) — オーバーヘッドが小さいインターフェイスです。 - -ほとんどの場合、これらのインターフェイスに直接アクセスするのではなく、適切なツールやライブラリを使用することを推奨します。ClickHouse により公式にサポートされているものは次のとおりです。 - -* [コマンドラインクライアント](../interfaces/cli.md) -* [JDBC ドライバ](../interfaces/jdbc.md) -* [ODBC ドライバ](../interfaces/odbc.md) -* [C++ クライアントライブラリ](../interfaces/cpp.md) - -ClickHouse は 2 つの RPC プロトコルもサポートしています。 - -* ClickHouse 向けに特別に設計された [gRPC プロトコル](grpc.md) -* [Apache Arrow Flight](arrowflight.md) - -ClickHouse サーバーは、パワーユーザー向けに組み込みのビジュアルインターフェイスも提供しています。 - -* Play UI: ブラウザで `/play` を開きます。 -* Advanced Dashboard: ブラウザで `/dashboard` を開きます。 -* ClickHouse エンジニア向けバイナリシンボルビューア: ブラウザで `/binary` を開きます。 - -また、ClickHouse と連携して動作するサードパーティライブラリも多数提供されています。 - -* [クライアントライブラリ](../interfaces/third-party/client-libraries.md) -* [インテグレーション](../interfaces/third-party/integrations.md) -* [ビジュアルインターフェイス](../interfaces/third-party/gui.md) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/postgresql.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/postgresql.md deleted file mode 100644 index 92bfcee7a65..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/postgresql.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'ClickHouse における PostgreSQL ワイヤプロトコル インターフェイスに関するドキュメント' -sidebar_label: 'PostgreSQL インターフェイス' -sidebar_position: 20 -slug: /interfaces/postgresql -title: 'PostgreSQL インターフェイス' -doc_type: 'リファレンス' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# PostgreSQL インターフェイス {#postgresql-interface} - - - -ClickHouse は PostgreSQL ワイヤープロトコルをサポートしており、Postgres クライアントを使用して ClickHouse に接続できます。言わば ClickHouse が PostgreSQL インスタンスとして振る舞うことで、ClickHouse がまだ直接サポートしていない PostgreSQL クライアントアプリケーション(たとえば Amazon Redshift)からも ClickHouse へ接続できるようになります。 - -PostgreSQL ワイヤープロトコルを有効化するには、サーバーの構成ファイルに [postgresql_port](../operations/server-configuration-parameters/settings.md#postgresql_port) 設定を追加します。たとえば、`config.d` ディレクトリ内の新しい XML ファイルでポートを定義できます。 - -```xml - - 9005 - -``` - -ClickHouse サーバーを起動し、**Listening for PostgreSQL compatibility protocol** という文言を含む、次のようなログメッセージを探します。 - -```response -{} Application: PostgreSQL互換プロトコルでリッスン中: 127.0.0.1:9005 -``` - -## psql を ClickHouse に接続する {#connect-psql-to-clickhouse} - -次のコマンドは、PostgreSQL クライアント `psql` を ClickHouse に接続する方法を示します。 - -```bash -psql -p [port] -h [hostname] -U [username] [database_name] -``` - -例えば: - -```bash -psql -p 9005 -h 127.0.0.1 -U alice default -``` - -:::note -`psql` クライアントではパスワード付きのログインが必須のため、パスワード未設定の `default` ユーザーでは接続できません。`default` ユーザーにパスワードを設定するか、別のユーザーとしてログインしてください。 -::: - -`psql` クライアントはパスワードの入力を求めます。 - -```response -ユーザー alice のパスワード: -psql (14.2, server 22.3.1.1) -警告: psql メジャーバージョン 14、サーバーメジャーバージョン 22。 - 一部の psql 機能が動作しない可能性があります。 -ヘルプを表示するには "help" と入力してください。 - -default=> -``` - -以上で完了です。これで PostgreSQL クライアントが ClickHouse に接続され、すべてのコマンドとクエリは ClickHouse 上で実行されます。 - -:::note -現在、PostgreSQL プロトコルはプレーンテキストのパスワードのみをサポートしています。 -::: - -## SSL の使用 {#using-ssl} - -ClickHouse インスタンスで SSL/TLS が構成されている場合、`postgresql_port` は同じ設定を使用します(このポートは、SSL を利用するクライアントと利用しないクライアントの両方で共有されます)。 - -各クライアントごとに、SSL を使用した接続方法は異なります。次のコマンドは、証明書と秘密鍵を指定して、`psql` を ClickHouse に安全に接続する方法を示しています。 - -```bash -psql "port=9005 host=127.0.0.1 user=alice dbname=default sslcert=/path/to/certificate.pem sslkey=/path/to/key.pem sslrootcert=/path/to/rootcert.pem sslmode=verify-ca" -``` - -## SCRAM-SHA-256 を使用した ClickHouse ユーザー認証の構成 {#using-scram-sha256} - -ClickHouse で安全なユーザー認証を行うためには、SCRAM-SHA-256 プロトコルの使用を推奨します。`users.xml` ファイルで `password_scram_sha256_hex` 要素を指定してユーザーを設定します。パスワードハッシュは num_iterations=4096 で生成する必要があります。 - -接続時に、psql クライアントが SCRAM-SHA-256 をサポートし、この方式で認証をネゴシエートできることを確認してください。 - -パスワード `abacaba` を持つユーザー `user_with_sha256` の設定例: - -```xml - - 04e7a70338d7af7bb6142fe7e19fef46d9b605f3e78b932a60e8200ef9154976 - -``` - -SSL 設定の詳細については、[PostgreSQL ドキュメント](https://jdbc.postgresql.org/documentation/head/ssl-client.html)を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/prometheus.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/prometheus.md deleted file mode 100644 index 6f645ede705..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/prometheus.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'ClickHouse の Prometheus プロトコルサポートに関するドキュメント' -sidebar_label: 'Prometheus プロトコル' -sidebar_position: 19 -slug: /interfaces/prometheus -title: 'Prometheus プロトコル' -doc_type: 'reference' ---- - -# Prometheus プロトコル {#prometheus-protocols} - -## メトリクスの公開 {#expose} - -:::note -ClickHouse Cloud を使用している場合、[Prometheus Integration](/integrations/prometheus) を利用して Prometheus にメトリクスを公開できます。 -::: - -ClickHouse は、自身のメトリクスを Prometheus からスクレイプできる形で公開できます。 - -````xml - - 9363 - /metrics - true - true - true - true - true - true - - -`` セクションを使用することで、より高度なハンドラーを作成できます。 -このセクションは [](/interfaces/http) と同様ですが、Prometheusプロトコルに対応します: - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - -```` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------- | -| `port` | none | メトリクス公開用プロトコルを提供するポート。 | -| `endpoint` | `/metrics` | Prometheus サーバーによるメトリクスのスクレイプ用 HTTP エンドポイント。`/` から開始します。`<handlers>` セクションと組み合わせて使用しないでください。 | -| `url` / `headers` / `method` | none | リクエストにマッチするハンドラーを特定するために使用されるフィルター。同名フィールドを持つ [`<http_handlers>`](/interfaces/http) セクションと同様です。 | -| `metrics` | true | [system.metrics](/operations/system-tables/metrics) テーブルからメトリクスを公開します。 | -| `asynchronous_metrics` | true | [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) テーブルから現在のメトリクス値を公開します。 | -| `events` | true | [system.events](/operations/system-tables/events) テーブルからメトリクスを公開します。 | -| `errors` | true | サーバーの最後の再起動以降に発生した、エラーコードごとのエラー数を公開します。この情報は [system.errors](/operations/system-tables/errors) からも取得できます。 | -| `histograms` | true | [system.histogram_metrics](/operations/system-tables/histogram_metrics) テーブルからヒストグラムメトリクスを公開します。 | -| `dimensional_metrics` | true | [system.dimensional_metrics](/operations/system-tables/dimensional_metrics) テーブルからディメンショナルメトリクスを公開します。 | - -確認(`127.0.0.1` を ClickHouse サーバーの IP アドレスまたはホスト名に置き換えてください): - -```bash -curl 127.0.0.1:9363/metrics -``` - -## Remote-write プロトコル {#remote-write} - -ClickHouse は [remote-write](https://prometheus.io/docs/specs/remote_write_spec/) プロトコルをサポートしています。 -このプロトコルを通じてデータを受信し、[TimeSeries](/engines/table-engines/special/time_series) テーブルに書き込みます -(テーブルは事前に作成しておく必要があります)。 - -```xml - - 9363 - - - /write - - remote_write - db_name - time_series_table
-
-
-
-
-``` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `port` | none | `remote-write` プロトコル用に待ち受けるポート。 | -| `url` / `headers` / `method` | none | リクエストに対して一致するハンドラーを見つけるために使用されるフィルター。[``](/interfaces/http) セクション内の同名フィールドと同様です。 | -| `table` | none | `remote-write` プロトコルで受信したデータを書き込む [TimeSeries](/engines/table-engines/special/time_series) テーブルの名前。この名前には、任意でデータベース名も含めることができます。 | -| `database` | none | `table` 設定で指定されたテーブル名にデータベース名が含まれていない場合に、そのテーブルが存在するデータベースの名前。 | - -## リモートリードプロトコル {#remote-read} - -ClickHouse は [remote-read](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/) プロトコルをサポートしています。 -データは [TimeSeries](/engines/table-engines/special/time_series) テーブルから読み出され、このプロトコル経由で送信されます。 - -```xml - - 9363 - - - /read - - remote_read - db_name - time_series_table
-
-
-
-
-``` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------- | -| `port` | none | `remote-read` プロトコルを待ち受けるためのポート。 | -| `url` / `headers` / `method` | none | リクエストに対して一致するハンドラーを見つけるために使用されるフィルタ。[``](/interfaces/http) セクション内の同名フィールドと同様です。 | -| `table` | none | `remote-read` プロトコルで送信するデータを読み取るための [TimeSeries](/engines/table-engines/special/time_series) テーブル名。この名前にはオプションでデータベース名も含めることができます。 | -| `database` | none | `table` 設定で指定されたテーブルが存在するデータベース名。テーブル名にデータベース名が含まれていない場合に使用されます。 | - -## 複数プロトコルの設定 {#multiple-protocols} - -複数のプロトコルを 1 か所でまとめて指定できます。 - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - /write - - remote_write - db_name.time_series_table
-
-
- - /read - - remote_read - db_name.time_series_table
-
-
-
-
-``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md deleted file mode 100644 index 39f53f45962..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md +++ /dev/null @@ -1,2374 +0,0 @@ ---- -description: 'ClickHouse における入力データからの自動スキーマ推論について説明するページ' -sidebar_label: 'スキーマ推論' -slug: /interfaces/schema-inference -title: '入力データからの自動スキーマ推論' -doc_type: 'reference' ---- - -ClickHouse は、サポートされているほぼすべての[入力フォーマット](formats.md)において、入力データの構造を自動的に判定できます。 -このドキュメントでは、スキーマ推論がいつ使用されるか、各種入力フォーマットでどのように動作するか、およびどの設定によって制御できるかについて説明します。 - - - -## 使用方法 {#usage} - -スキーマ推論は、ClickHouse が特定のデータ形式でデータを読み取る必要があるものの、その構造が不明な場合に使用されます。 - - - -## テーブル関数 [file](../sql-reference/table-functions/file.md)、[s3](../sql-reference/table-functions/s3.md)、[url](../sql-reference/table-functions/url.md)、[hdfs](../sql-reference/table-functions/hdfs.md)、[azureBlobStorage](../sql-reference/table-functions/azureBlobStorage.md)。 {#table-functions-file-s3-url-hdfs-azureblobstorage} - -これらのテーブル関数には、入力データの構造を表すオプションの引数 `structure` があります。この引数が指定されていないか、`auto` に設定されている場合は、構造がデータから自動的に推論されます。 - -**例:** - -`user_files` ディレクトリ内に、次の内容を持つ JSONEachRow 形式のファイル `hobbies.jsonl` があるとします: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["football", "cooking", "music"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["tennis", "art"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["fitness", "reading", "shopping"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["movies", "skydiving"]} -``` - -ClickHouse は、このデータの構造を事前に定義しなくても読み取ることができます。 - -```sql -SELECT * FROM file('hobbies.jsonl') -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -注記: フォーマット `JSONEachRow` は、ファイル拡張子 `.jsonl` によって自動的に認識されました。 - -`DESCRIBE` クエリを使用して、自動的に認識された構造を確認できます。 - -```sql -DESCRIBE file('hobbies.jsonl') -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## テーブルエンジン [File](../engines/table-engines/special/file.md)、[S3](../engines/table-engines/integrations/s3.md)、[URL](../engines/table-engines/special/url.md)、[HDFS](../engines/table-engines/integrations/hdfs.md)、[azureBlobStorage](../engines/table-engines/integrations/azureBlobStorage.md) {#table-engines-file-s3-url-hdfs-azureblobstorage} - -`CREATE TABLE` クエリでカラムのリストを指定しない場合、テーブルの構造はデータから自動的に推論されます。 - -**例:** - -ファイル `hobbies.jsonl` を使用するとします。このファイルのデータを使って、次のようにエンジン `File` のテーブルを作成できます。 - -```sql -CREATE TABLE hobbies ENGINE=File(JSONEachRow, 'hobbies.jsonl') -``` - -```response -OK -``` - -```sql -SELECT * FROM hobbies -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -```sql -DESCRIBE TABLE hobbies -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## clickhouse-local {#clickhouse-local} - -`clickhouse-local` には、入力データの構造を指定するためのオプションのパラメータ `-S/--structure` があります。このパラメータが指定されていないか、`auto` に設定されている場合、構造はデータから自動的に推論されます。 - -**例:** - -ファイル `hobbies.jsonl` を使ってみましょう。このファイルのデータは `clickhouse-local` を使ってクエリすることができます。 - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='DESCRIBE TABLE hobbies' -``` - -```response -id Nullable(Int64) -age Nullable(Int64) -name Nullable(String) -hobbies Array(Nullable(String)) -``` - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='SELECT * FROM hobbies' -``` - -```response -1 25 Josh ['football','cooking','music'] -2 19 Alan ['tennis','art'] -3 32 Lana ['fitness','reading','shopping'] -4 47 Brayan ['movies','skydiving'] -``` - - -## 挿入先テーブルの構造を使用する {#using-structure-from-insertion-table} - -テーブル関数 `file/s3/url/hdfs` を使用してテーブルにデータを挿入する場合、 -データから構造を推論する代わりに、挿入先テーブルの構造を使用するオプションがあります。 -スキーマ推論には時間がかかることがあるため、挿入性能を向上させることができます。また、テーブルに最適化されたスキーマが定義されている場合は、 -型間の変換が発生しないため有用です。 - -この挙動を制御する専用の設定 [use_structure_from_insertion_table_in_table_functions](/operations/settings/settings.md/#use_structure_from_insertion_table_in_table_functions) -があります。取り得る値は 3 つです: - -* 0 - テーブル関数はデータから構造を抽出します。 -* 1 - テーブル関数は挿入先テーブルの構造を使用します。 -* 2 - ClickHouse は、挿入先テーブルの構造を使用できるか、スキーマ推論を使用する必要があるかを自動的に判断します。デフォルト値です。 - -**例 1:** - -次の構造を持つテーブル `hobbies1` を作成します: - -```sql -CREATE TABLE hobbies1 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `name` String, - `hobbies` Array(String) -) -ENGINE = MergeTree -ORDER BY id; -``` - -次に、ファイル `hobbies.jsonl` からデータを挿入します: - -```sql -INSERT INTO hobbies1 SELECT * FROM file(hobbies.jsonl) -``` - -この場合、ファイルのすべての列が一切変更されることなくテーブルに挿入されるため、ClickHouse はスキーマ推論ではなく、挿入先テーブルの構造を使用します。 - -**例 2:** - -次の構造を持つテーブル `hobbies2` を作成してみましょう: - -```sql -CREATE TABLE hobbies2 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -`hobbies.jsonl` ファイルからデータを挿入します: - -```sql -INSERT INTO hobbies2 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -この場合、`SELECT` クエリ内のすべての列がテーブル内に存在するため、ClickHouse は挿入先テーブルの構造を使用します。 -なお、これは JSONEachRow、TSKV、Parquet などのように列のサブセットの読み取りをサポートする入力フォーマットでのみ機能します(したがって、たとえば TSV フォーマットでは動作しません)。 - -**例 3:** - -次の構造を持つテーブル `hobbies3` を作成してみましょう。 - -```sql -CREATE TABLE hobbies3 -( - `identifier` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY identifier; -``` - -次に、`hobbies.jsonl` ファイルからデータを挿入します: - -```sql -INSERT INTO hobbies3 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -この場合、`SELECT` クエリでは列 `id` が使用されていますが、テーブルにはこの列は存在せず(`identifier` という名前の列があります)、 -ClickHouse は挿入テーブルの構造を使用できないため、スキーマ推論が行われます。 - -**例 4:** - -次の構造でテーブル `hobbies4` を作成してみましょう: - -```sql -CREATE TABLE hobbies4 -( - `id` UInt64, - `any_hobby` Nullable(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -次に、ファイル `hobbies.jsonl` からデータを挿入します。 - -```sql -INSERT INTO hobbies4 SELECT id, empty(hobbies) ? NULL : hobbies[1] FROM file(hobbies.jsonl) -``` - -この場合、テーブルに挿入するための `SELECT` クエリ内でカラム `hobbies` に対していくつかの操作が実行されているため、ClickHouse は挿入元テーブルの構造を利用できず、スキーマ推論が行われます。 - - -## スキーマ推論キャッシュ {#schema-inference-cache} - -ほとんどの入力フォーマットでは、スキーマ推論のために一部のデータを読み取り、その構造を判定しますが、この処理には一定の時間がかかります。 -同じファイルからデータを読むたびに毎回同じスキーマを推論しないようにするため、推論されたスキーマはキャッシュされ、同じファイルへ再度アクセスする際には、ClickHouse はキャッシュからそのスキーマを利用します。 - -このキャッシュを制御するための専用設定があります: -- `schema_inference_cache_max_elements_for_{file/s3/hdfs/url/azure}` - 対応するテーブル関数ごとにキャッシュできるスキーマの最大数を指定します。デフォルト値は `4096` です。これらの設定はサーバー構成で指定する必要があります。 -- `schema_inference_use_cache_for_{file,s3,hdfs,url,azure}` - スキーマ推論でキャッシュを使用するかどうかを切り替えます。これらの設定はクエリ内で使用できます。 - -ファイルのスキーマは、データを変更したり、フォーマット設定を変更したりすることで変わる可能性があります。 -このため、スキーマ推論キャッシュは、ファイルのソース、フォーマット名、使用されたフォーマット設定、およびファイルの最終更新時刻によってスキーマを識別します。 - -注意: `url` テーブル関数で URL 経由でアクセスされる一部のファイルには、最終更新時刻に関する情報が含まれない場合があります。そのようなケースのために、特別な設定 -`schema_inference_cache_require_modification_time_for_url` が用意されています。この設定を無効にすると、そのようなファイルについて最終更新時刻がなくてもキャッシュされたスキーマを使用できるようになります。 - -さらに、現在キャッシュされている全スキーマを保持しているシステムテーブル [schema_inference_cache](../operations/system-tables/schema_inference_cache.md) と、全ソースまたは特定ソースのスキーマキャッシュを削除できるシステムクエリ `SYSTEM DROP SCHEMA CACHE [FOR File/S3/URL/HDFS]` があります。 - -**例:** - -S3 上のサンプルデータセット `github-2022.ndjson.gz` の構造推論を試し、スキーマ推論キャッシュがどのように動作するか確認してみます: - - - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -5行を取得しました。経過時間: 0.601秒 -``` - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -``` - -5 行の結果セット。経過時間: 0.059 秒。 - -```` - -ご覧のとおり、2回目のクエリはほぼ瞬時に成功しました。 - -推論されるスキーマに影響を与える可能性のある設定を変更してみましょう。 - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -SETTINGS input_format_json_try_infer_named_tuples_from_objects=0, input_format_json_read_objects_as_strings = 1 - -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ type │ Nullable(String) │ │ │ │ │ │ -│ actor │ Nullable(String) │ │ │ │ │ │ -│ repo │ Nullable(String) │ │ │ │ │ │ -│ created_at │ Nullable(String) │ │ │ │ │ │ -│ payload │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -5 rows in set. Elapsed: 0.611 sec -```` - -ご覧のとおり、同じファイルに対してキャッシュからのスキーマは使用されていません。これは、推論されるスキーマに影響を与える設定が変更されたためです。 - -`system.schema_inference_cache` テーブルの内容を確認してみましょう: - -```sql -SELECT schema, format, source FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─schema──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─format─┬─source───────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ type Nullable(String), actor Tuple(avatar_url Nullable(String), display_login Nullable(String), id Nullable(Int64), login Nullable(String), url Nullable(String)), repo Tuple(id Nullable(Int64), name Nullable(String), url Nullable(String)), created_at Nullable(String), payload Tuple(action Nullable(String), distinct_size Nullable(Int64), pull_request Tuple(author_association Nullable(String), base Tuple(ref Nullable(String), sha Nullable(String)), head Tuple(ref Nullable(String), sha Nullable(String)), number Nullable(Int64), state Nullable(String), title Nullable(String), updated_at Nullable(String), user Tuple(login Nullable(String))), ref Nullable(String), ref_type Nullable(String), size Nullable(Int64)) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -│ type Nullable(String), actor Nullable(String), repo Nullable(String), created_at Nullable(String), payload Nullable(String) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -ご覧のとおり、同じファイルに対して 2 つの異なるスキーマがあります。 - -システムクエリを使用してスキーマキャッシュをクリアできます。 - -```sql -SYSTEM DROP SCHEMA CACHE FOR S3 -``` - -```response -OK。 -``` - -```sql -SELECT count() FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─count()─┐ -│ 0 │ -└─────────┘ -``` - - -## テキスト形式 {#text-formats} - -テキスト形式では、ClickHouse はデータを 1 行ずつ読み取り、フォーマットに従ってカラム値を抽出し、その後、再帰的なパーサーとヒューリスティクスを用いて各値の型を判定します。スキーマ推論時にデータから読み取られる最大行数および最大バイト数は、設定 `input_format_max_rows_to_read_for_schema_inference`(デフォルト 25000)および `input_format_max_bytes_to_read_for_schema_inference`(デフォルト 32MB)によって制御されます。 -デフォルトでは、推論された型はすべて [Nullable](../sql-reference/data-types/nullable.md) になりますが、`schema_inference_make_columns_nullable` を設定することでこれを変更できます([settings](#settings-for-text-formats) セクションの例を参照)。 - -### JSON 形式 {#json-formats} - -JSON 形式では、ClickHouse は JSON 仕様に従って値をパースし、その後、それぞれに最も適切なデータ型を推論しようとします。 - -ここでは、その動作の仕組み、推論可能な型、および JSON 形式で使用できる特定の設定について説明します。 - -**例** - -この先の例では、[format](../sql-reference/table-functions/format.md) テーブル関数を使用します。 - -整数、浮動小数点数、ブール値、文字列: - -```sql -DESC format(JSONEachRow, '{"int" : 42, "float" : 42.42, "string" : "Hello, World!"}'); -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -日付・日時: - -```sql -DESC format(JSONEachRow, '{"date" : "2022-01-01", "datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"}') -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列: - -```sql -DESC format(JSONEachRow, '{"arr" : [1, 2, 3], "nested_arrays" : [[1, 2, 3], [4, 5, 6], []]}') -``` - -```response -┌─name──────────┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ nested_arrays │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└───────────────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列に `null` が含まれている場合、ClickHouse は他の配列要素から型を推論します。 - -```sql -DESC format(JSONEachRow, '{"arr" : [null, 42, null]}') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -配列に異なる型の値が含まれていて、`input_format_json_infer_array_of_dynamic_from_array_of_different_types` 設定が有効になっている場合(デフォルトで有効)、その配列は `Array(Dynamic)` 型になります。 - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=1; -DESC format(JSONEachRow, '{"arr" : [42, "hello", [1, 2, 3]]}'); -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Dynamic) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -名前付きタプル: - -`input_format_json_try_infer_named_tuples_from_objects` 設定が有効な場合、スキーマ推論時に ClickHouse は JSON オブジェクトから名前付きタプルを推論しようとします。 -生成される名前付きタプルには、サンプルデータ中の対応するすべての JSON オブジェクトに含まれるすべての要素が含まれます。 - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -名前なしタプル: - -`input_format_json_infer_array_of_dynamic_from_array_of_different_types` 設定が無効になっている場合、異なる型の要素を含む配列は、JSON 形式において名前なしタプルとして扱われます。 - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types = 0; -DESC format(JSONEachRow, '{"tuple" : [1, "Hello, World!", [1, 2, 3]]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -いくつかの値が `null` または空である場合は、他の行の対応する値の型を使用します。 - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=0; -DESC format(JSONEachRow, $$ - {"tuple" : [1, null, null]} - {"tuple" : [null, "Hello, World!", []]} - {"tuple" : [null, null, [1, 2, 3]]} - $$) -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -マップ: - -JSON から、値が Map 型の値と同じ型で揃っているオブジェクトを Map 型として読み取ることができます。 -注意: これは、設定 `input_format_json_read_objects_as_strings` と `input_format_json_try_infer_named_tuples_from_objects` が無効になっている場合にのみ有効です。 - - -```sql -SET input_format_json_read_objects_as_strings = 0, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, '{"map" : {"key1" : 42, "key2" : 24, "key3" : 4}}') -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ map │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ネストされた複合型: - -```sql -DESC format(JSONEachRow, '{"value" : [[[42, 24], []], {"key1" : 42, "key2" : 24}]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Tuple(Array(Array(Nullable(String))), Tuple(key1 Nullable(Int64), key2 Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -あるキーについて、データが null/空オブジェクト/空配列のみを含んでいて ClickHouse が型を決定できない場合、設定 `input_format_json_infer_incomplete_types_as_strings` が有効であれば型 `String` が使用され、無効であれば例外がスローされます。 - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 1; -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(String)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 0; -``` - -```response -Code: 652. DB::Exception: localhost:9000から受信。DB::Exception: -最初の1行のデータから列'arr'の型を判定できません。 -この列にはNullまたは空のArray/Mapのみが含まれている可能性があります。 -... -``` - -#### JSON 設定 {#json-settings} - -##### input_format_json_try_infer_numbers_from_strings {#input_format_json_try_infer_numbers_from_strings} - -この設定を有効にすると、文字列の値から数値を推論できるようになります。 - -この設定はデフォルトで無効です。 - -**例:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : "42"} - {"value" : "424242424242"} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_try_infer_named_tuples_from_objects {#input_format_json_try_infer_named_tuples_from_objects} - -この設定を有効にすると、JSON オブジェクトから名前付きタプルを推論できるようになります。生成される名前付きタプルには、サンプルデータ中の対応するすべての JSON オブジェクトに含まれる要素がすべて含まれます。 -JSON データがスパースでない場合には、サンプルデータにあらゆるオブジェクトキーが含まれるため、とくに有用です。 - -この設定はデフォルトで有効になっています。 - -**例** - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -結果: - - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"array" : [{"a" : 42, "b" : "Hello"}, {}, {"c" : [1,2,3]}, {"d" : "2020-01-01"}]}') -``` - -結果: - -```markdown -┌─name──┬─type────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ array │ Array(Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Nullable(Date))) │ │ │ │ │ │ -└───────┴─────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects {#input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects} - -この設定を有効にすると、`input_format_json_try_infer_named_tuples_from_objects` が有効な場合に、JSON オブジェクトから named Tuple を推論する際、あいまいなパスに対して例外をスローする代わりに String 型を使用できるようになります。 -これにより、パスにあいまいさがある JSON オブジェクトであっても、named Tuple として読み取ることができます。 - -デフォルトでは無効です。 - -**使用例** - -設定を無効にした場合: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 0; -DESC format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -結果: - -```response -コード: 636. DB::Exception: JSONEachRow形式ファイルからテーブル構造を抽出できません。エラー: -コード: 117. DB::Exception: JSONオブジェクトに曖昧なデータが含まれています: 一部のオブジェクトではパス 'a' の型が 'Int64' ですが、他のオブジェクトでは 'Tuple(b String)' です。パス 'a' にString型を使用するには、設定 input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects を有効化してください。(INCORRECT_DATA) (バージョン 24.3.1.1)。 -構造は手動で指定することができます。(CANNOT_EXTRACT_TABLE_STRUCTURE) -``` - -この設定を有効にしている場合: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : "a" : 42}, {"obj" : {"a" : {"b" : "Hello"}}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -結果: - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(String)) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -┌─obj─────────────────┐ -│ ('42') │ -│ ('{"b" : "Hello"}') │ -└─────────────────────┘ -``` - -##### input_format_json_read_objects_as_strings {#input_format_json_read_objects_as_strings} - -この設定を有効にすると、ネストされた JSON オブジェクトを文字列として読み取ることができます。 -この設定を使用すると、JSON オブジェクト型を使用せずにネストされた JSON オブジェクトを読み取ることができます。 - -この設定はデフォルトで有効になっています。 - -注意: この設定の有効化は、設定 `input_format_json_try_infer_named_tuples_from_objects` が無効になっている場合にのみ効果があります。 - - -```sql -SET input_format_json_read_objects_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, $$ - {"obj" : {"key1" : 42, "key2" : [1,2,3,4]}} - {"obj" : {"key3" : {"nested_key" : 1}}} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_numbers_as_strings {#input_format_json_read_numbers_as_strings} - -この設定を有効にすると、数値を文字列として読み取ります。 - -この設定はデフォルトで有効になっています。 - -**例** - -```sql -SET input_format_json_read_numbers_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : 1055} - {"value" : "unknown"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_numbers {#input_format_json_read_bools_as_numbers} - -この設定を有効にすると、Bool 型の値を数値として読み取れるようになります。 - -この設定はデフォルトで有効になっています。 - -**例:** - -```sql -SET input_format_json_read_bools_as_numbers = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : 42} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_strings {#input_format_json_read_bools_as_strings} - -この設定を有効にすると、Bool 型の値を文字列として読み取ることができます。 - -この設定は既定で有効になっています。 - -**例:** - -```sql -SET input_format_json_read_bools_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : "Hello, World"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_arrays_as_strings {#input_format_json_read_arrays_as_strings} - -この設定を有効にすると、JSON 配列の値を文字列として読み取ります。 - -この設定は既定で有効になっています。 - -**例** - -```sql -SET input_format_json_read_arrays_as_strings = 1; -SELECT arr, toTypeName(arr), JSONExtractArrayRaw(arr)[3] from format(JSONEachRow, 'arr String', '{"arr" : [1, "Hello", [1,2,3]]}'); -``` - -```response -┌─arr───────────────────┬─toTypeName(arr)─┬─arrayElement(JSONExtractArrayRaw(arr), 3)─┐ -│ [1, "Hello", [1,2,3]] │ String │ [1,2,3] │ -└───────────────────────┴─────────────────┴───────────────────────────────────────────┘ -``` - -##### input_format_json_infer_incomplete_types_as_strings {#input_format_json_infer_incomplete_types_as_strings} - - -この設定を有効にすると、スキーマ推論時に、サンプルデータ内で `Null`・`{}`・`[]` のみを含む JSON キーに `String` 型を使用できるようになります。 -JSON フォーマットでは、関連する設定がすべて有効になっている場合(既定でいずれも有効)には、任意の値を String として読み取ることができるため、スキーマ推論時に `Cannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps` のようなエラーが発生する状況でも、型が不明なキーに対して String 型を使用することでこれを回避できます。 - -例: - -```sql -SET input_format_json_infer_incomplete_types_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 1; -DESCRIBE format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -``` - -結果: - -```markdown -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Array(Nullable(Int64)), b Nullable(String), c Nullable(String), d Nullable(String), e Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -┌─obj────────────────────────────┐ -│ ([1,2,3],'hello',NULL,'{}',[]) │ -└────────────────────────────────┘ -``` - -### CSV {#csv} - -CSV 形式では、ClickHouse は区切り文字に従って行から列の値を抽出します。ClickHouse は、数値と文字列以外のすべての型が二重引用符で囲まれていることを前提とします。値が二重引用符で囲まれている場合、ClickHouse は再帰パーサーを使って引用符内のデータを解析し、その値に最も適したデータ型を判定しようとします。値が二重引用符で囲まれていない場合、ClickHouse はそれを数値として解析しようとし、その値が数値でない場合は文字列として扱います。 - -ClickHouse にパーサーやヒューリスティクスを使って複雑な型を推定させたくない場合は、`input_format_csv_use_best_effort_in_schema_inference` 設定を無効化すると、ClickHouse はすべての列を String 型として扱います。 - -`input_format_csv_detect_header` 設定が有効な場合、ClickHouse はスキーマ推論時に、列名(および場合によっては型)を含むヘッダー行を検出しようとします。この設定はデフォルトで有効になっています。 - -**例:** - -整数、浮動小数点数、ブール値、文字列: - -```sql -DESC format(CSV, '42,42.42,true,"Hello,World!"') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -引用符で囲まれていない文字列: - -```sql -DESC format(CSV, 'Hello world!,World hello!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -日付、日時: - - -```sql -DESC format(CSV, '"2020-01-01","2020-01-01 00:00:00","2022-01-01 00:00:00.000"') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列: - -```sql -DESC format(CSV, '"[1,2,3]","[[1, 2], [], [3, 4]]"') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(CSV, $$"['Hello', 'world']","[['Abc', 'Def'], []]"$$) -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列に null が含まれている場合、ClickHouse は他の配列要素の型に基づいて型を決定します。 - -```sql -DESC format(CSV, '"[NULL, 42, NULL]"') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -マップ: - -```sql -DESC format(CSV, $$"{'key1' : 42, 'key2' : 24}"$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ネストされた配列とマップ: - -```sql -DESC format(CSV, $$"[{'key1' : [[42, 42], []], 'key2' : [[null], [42]]}]"$$) -``` - -```response -┌─name─┬─type──────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Array(Nullable(Int64))))) │ │ │ │ │ │ -└──────┴───────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -データに null しか含まれず、引用符で囲まれた値の型を ClickHouse が判別できない場合、ClickHouse はそれを String 型として扱います。 - -```sql -DESC format(CSV, '"[NULL, NULL]"') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -`input_format_csv_use_best_effort_in_schema_inference` 設定を無効化した場合の例: - -```sql -SET input_format_csv_use_best_effort_in_schema_inference = 0 -DESC format(CSV, '"[1,2,3]",42.42,Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -`input_format_csv_detect_header` が有効な場合のヘッダー自動検出の例: - -列名のみ: - -```sql -SELECT * FROM format(CSV, -$$"number","string","array" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -名前と型: - -```sql -DESC format(CSV, -$$"number","string","array" -"UInt32","String","Array(UInt16)" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ヘッダーが検出されるのは、少なくとも1つの列が String 以外の型である場合のみです。すべての列が String 型の場合、ヘッダーは検出されません。 - -```sql -SELECT * FROM format(CSV, -$$"first_column","second_column" -"Hello","World" -"World","Hello" -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ first_column │ second_column │ -│ Hello │ World │ -│ World │ Hello │ -└──────────────┴───────────────┘ -``` - -#### CSV 設定 {#csv-settings} - -##### input_format_csv_try_infer_numbers_from_strings {#input_format_csv_try_infer_numbers_from_strings} - -この設定を有効にすると、文字列の値から数値を推論できるようになります。 - -この設定は既定では無効です。 - -**例:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(CSV, '42,42.42'); -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -### TSV/TSKV {#tsv-tskv} - -TSV/TSKV 形式では、ClickHouse はタブ区切りに従って行から列の値を抽出し、その後、再帰パーサーを使って抽出した値を解析し、最も適切な型を決定します。型を決定できない場合、ClickHouse はこの値を String として扱います。 - -ClickHouse に一部のパーサーやヒューリスティックを用いた複雑な型の推論を行わせたくない場合は、`input_format_tsv_use_best_effort_in_schema_inference` -設定を無効にしてください。そうすると ClickHouse はすべての列を String として扱います。 - -`input_format_tsv_detect_header` 設定が有効な場合、ClickHouse はスキーマ推論時に、列名(および場合によっては型)を含むヘッダーを検出しようとします。この設定はデフォルトで有効です。 - -**例:** - -整数、浮動小数点数、ブール値、文字列: - -```sql -DESC format(TSV, '42 42.42 true Hello,World!') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSKV, 'int=42 float=42.42 bool=true string=Hello,World!\n') -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date 型、DateTime 型: - -```sql -DESC format(TSV, '2020-01-01 2020-01-01 00:00:00 2022-01-01 00:00:00.000') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -配列: - -```sql -DESC format(TSV, '[1,2,3] [[1, 2], [], [3, 4]]') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSV, '[''Hello'', ''world''] [[''Abc'', ''Def''], []]') -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列に null が含まれている場合、ClickHouse は他の要素から型を決定します。 - -```sql -DESC format(TSV, '[NULL, 42, NULL]') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -タプル: - -```sql -DESC format(TSV, $$(42, 'Hello, world!')$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -マップ: - -```sql -DESC format(TSV, $${'key1' : 42, 'key2' : 24}$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ネストされた配列、タプル、マップ: - -```sql -DESC format(TSV, $$[{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}]$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -データに null しか含まれておらず、ClickHouse が型を決定できない場合、ClickHouse はそれを String として扱います。 - -```sql -DESC format(TSV, '[NULL, NULL]') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -設定 `input_format_tsv_use_best_effort_in_schema_inference` を無効にした状態での例: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ヘッダー自動検出の例(`input_format_tsv_detect_header` が有効な場合): - -列名のみの場合: - -```sql -SELECT * FROM format(TSV, -$$number string array -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$); -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -名前と型: - -```sql -DESC format(TSV, -$$number string array -UInt32 String Array(UInt16) -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ヘッダーが検出されるのは、少なくとも1つの列が String 型以外の場合のみです。すべての列が String 型の場合、ヘッダーは検出されません。 - -```sql -SELECT * FROM format(TSV, -$$first_column second_column -Hello World -World Hello -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ first_column │ second_column │ -│ Hello │ World │ -│ World │ Hello │ -└──────────────┴───────────────┘ -``` - -### 値 {#values} - -Values 形式では、ClickHouse は行から列の値を抽出し、その後リテラルを解析する場合と同様に、再帰パーサーを使って値を解析します。 - -**例:** - - -整数、浮動小数点数、ブール値、文字列: - -```sql -DESC format(Values, $$(42, 42.42, true, 'Hello,World!')$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date、DateTime: - -```sql - DESC format(Values, $$('2020-01-01', '2020-01-01 00:00:00', '2022-01-01 00:00:00.000')$$) -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列: - -```sql -DESC format(Values, '([1,2,3], [[1, 2], [], [3, 4]])') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -配列に null が含まれている場合、ClickHouse は他の配列要素の型を使用します。 - -```sql -DESC format(Values, '([NULL, 42, NULL])') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -タプル: - -```sql -DESC format(Values, $$((42, 'Hello, world!'))$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -マップ: - -```sql -DESC format(Values, $$({'key1' : 42, 'key2' : 24})$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -ネストされた配列、タプル、マップ: - -```sql -DESC format(Values, $$([{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}])$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -データに null 値しか含まれておらず、ClickHouse が型を判別できない場合は、例外がスローされます。 - -```sql -DESC format(Values, '([NULL, NULL])') -``` - -```response -コード: 652. DB::Exception: localhost:9000から受信。DB::Exception: -最初の1行のデータから列'c1'の型を判定できません。 -この列にはNullまたは空のArray/Mapのみが含まれている可能性があります。 -... -``` - -`input_format_tsv_use_best_effort_in_schema_inference` 設定を無効にした状態での例: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### CustomSeparated {#custom-separated} - -CustomSeparated フォーマットでは、ClickHouse はまず指定された区切り文字に従って行からすべての列の値を抽出し、その後、エスケープ規則に基づいて各値のデータ型を推論します。 - -設定 `input_format_custom_detect_header` が有効な場合、ClickHouse はスキーマ推論中に列名(および場合によっては型)を含むヘッダー行を検出しようとします。この設定はデフォルトで有効になっています。 - -**例** - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -ヘッダーの自動検出例(`input_format_custom_detect_header` が有効化されている場合): - - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -'number''string''array' - -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─number─┬─string────────┬─array──────┐ -│ 42.42 │ Some string 1 │ [1,NULL,3] │ -│ ᴺᵁᴸᴸ │ Some string 3 │ [1,2,NULL] │ -└────────┴───────────────┴────────────┘ -``` - -### テンプレート {#template} - -Template 形式では、ClickHouse はまず指定されたテンプレートに従って行からすべてのカラム値を抽出し、その後、それぞれの値に対してエスケープ規則に基づいてデータ型を推論します。 - -**例** - -次の内容を持つ `resultset` というファイルがあるとします。 - -```bash - -${data} -``` - -また、次の内容の `row_format` ファイルを用意します。 - -```text -${column_1:CSV}${column_2:Quoted}${column_3:JSON} -``` - -次に、以下のクエリを実行します。 - -```sql -SET format_template_rows_between_delimiter = '\n', - format_template_row = 'row_format', - format_template_resultset = 'resultset_format' - -DESC format(Template, $$ -42.42'Some string 1'[1, null, 2] - -\N'Some string 3'[1, 2, null] - -$$) -``` - -```response -┌─name─────┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ column_1 │ Nullable(Float64) │ │ │ │ │ │ -│ column_2 │ Nullable(String) │ │ │ │ │ │ -│ column_3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Regexp {#regexp} - -Template 形式と同様に、Regexp 形式では ClickHouse はまず、指定された正規表現に従って行からすべての列の値を抽出し、その後、指定されたエスケープ規則に基づいて各値のデータ型を自動的に推定します。 - -**例** - -```sql -SET format_regexp = '^Line: value_1=(.+?), value_2=(.+?), value_3=(.+?)', - format_regexp_escaping_rule = 'CSV' -``` - - -DESC format(Regexp, $$Line: value_1=42, value_2="Some string 1", value_3="[1, NULL, 3]" -Line: value_1=2, value_2="Some string 2", value_3="[4, 5, NULL]"$$) - -```` -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -```` - -### テキスト形式用の設定 {#settings-for-text-formats} - -#### input_format_max_rows_to_read_for_schema_inference/input_format_max_bytes_to_read_for_schema_inference {#input-format-max-rows-to-read-for-schema-inference} - -これらの設定は、スキーマ推論を行う際に読み取るデータ量を制御します。 -より多くの行やバイト数を読み取るほどスキーマ推論に時間はかかりますが、型を正しく判定できる -可能性が高くなります(特に、データに `null` が多く含まれる場合)。 - -デフォルト値: - -* `input_format_max_rows_to_read_for_schema_inference`: `25000` -* `input_format_max_bytes_to_read_for_schema_inference`: `33554432`(32 MB) - -#### column_names_for_schema_inference {#column-names-for-schema-inference} - -明示的なカラム名を持たないフォーマットに対して、スキーマ推論で使用するカラム名のリストです。指定した名前は、デフォルトの `c1,c2,c3,...` の代わりに使用されます。形式: `column1,column2,column3,...`。 - -**例** - -```sql -DESC format(TSV, 'Hello, World! 42 [1, 2, 3]') settings column_names_for_schema_inference = 'str,int,arr' -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ str │ Nullable(String) │ │ │ │ │ │ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_hints {#schema-inference-hints} - -自動的に決定される型の代わりに、スキーマ推論で使用する列名と型のリストです。形式は 'column_name1 column_type1, column_name2 column_type2, ...' のようになります。 -この設定は、自動的に型を決定できなかった列の型を指定する場合や、スキーマを最適化する目的で使用できます。 - -**例** - -```sql -DESC format(JSONEachRow, '{"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]}') SETTINGS schema_inference_hints = 'age LowCardinality(UInt8), status Nullable(String)', allow_suspicious_low_cardinality_types=1 -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ LowCardinality(UInt8) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_make_columns_nullable $ {#schema-inference-make-columns-nullable} - - -NULL 許容性に関する情報を持たないフォーマットに対して、スキーマ推論時に推論された型を `Nullable` にするかどうかを制御します。設定可能な値: - -* 0 - 推論された型は決して `Nullable` になりません。 -* 1 - すべての推論された型が `Nullable` になります。 -* 2 または 'auto' - テキストフォーマットでは、スキーマ推論中に解析されるサンプル内で列に `NULL` が含まれている場合にのみ、推論された型が `Nullable` になります。強く型付けされたフォーマット(Parquet、ORC、Arrow)では、NULL 許容性の情報はファイルメタデータから取得されます。 -* 3 - テキストフォーマットでは常に `Nullable` を使用し、強く型付けされたフォーマットではファイルメタデータを使用します。 - -デフォルト: 3。 - -**例** - -```sql -SET schema_inference_make_columns_nullable = 1; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 'auto'; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 0; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response - -┌─name────┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ String │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -#### input_format_try_infer_integers {#input-format-try-infer-integers} - -:::note -この設定は `JSON` データ型には適用されません。 -::: - -有効化されている場合、ClickHouse はテキストフォーマットのスキーマ推論時に、浮動小数点数ではなく整数として推論しようとします。 -サンプルデータの列内のすべての数値が整数であれば、結果の型は `Int64` になり、1 つでも浮動小数点数が含まれていれば、結果の型は `Float64` になります。 -サンプルデータが整数のみを含み、かつ少なくとも 1 つの整数が正で `Int64` をオーバーフローする場合、ClickHouse は `UInt64` として推論します。 - -デフォルトで有効です。 - -**例** - -```sql -SET input_format_try_infer_integers = 0 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_integers = 1 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Int64) │ │ │ │ │ │ -└────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 18446744073709551615} - $$) -``` - -```response -┌─name───┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(UInt64) │ │ │ │ │ │ -└────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2.2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes {#input-format-try-infer-datetimes} - -有効にすると、ClickHouse はテキスト形式のスキーマ推論を行う際に、文字列フィールドから型 `DateTime` または `DateTime64` を推定しようとします。 -サンプルデータのある列に含まれるすべてのフィールドが日時として正常にパースできた場合、結果の型は `DateTime` または `DateTime64(9)`(いずれかの日時に小数部が含まれていた場合)になり、 -少なくとも 1 つのフィールドが日時としてパースできなかった場合、結果の型は `String` になります。 - -デフォルトで有効です。 - -**例** - - -```sql -SET input_format_try_infer_datetimes = 0; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_datetimes = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "unknown", "datetime64" : "unknown"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes_only_datetime64 {#input-format-try-infer-datetimes-only-datetime64} - -この設定を有効にすると、`input_format_try_infer_datetimes` が有効な場合、日時の値に小数部が含まれていなくても、ClickHouse は常に `DateTime64(9)` 型として推論します。 - -既定では無効です。 - -**例** - -```sql -SET input_format_try_infer_datetimes = 1; -SET input_format_try_infer_datetimes_only_datetime64 = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime64(9)) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -注記: スキーマ推論時の日時のパースは、設定 [date_time_input_format](/operations/settings/settings-formats.md#date_time_input_format) に従います。 - - -#### input_format_try_infer_dates {#input-format-try-infer-dates} - -有効にすると、ClickHouse はテキストフォーマットに対するスキーマ推論時に、文字列フィールドから型 `Date` を推定しようとします。 -サンプルデータ内のあるカラムのすべてのフィールドが日付として正常にパースされた場合、その結果型は `Date` になります。 -少なくとも 1 つのフィールドが日付としてパースできなかった場合、その結果型は `String` になります。 - -デフォルトで有効です。 - -**例** - -```sql -SET input_format_try_infer_datetimes = 0, input_format_try_infer_dates = 0 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_dates = 1 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "unknown"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_exponent_floats {#input-format-try-infer-exponent-floats} - -有効にすると、ClickHouse はテキストフォーマットにおいて、指数表記された数値を浮動小数点数として解釈しようとします(指数表記の数値が常に浮動小数点数として解釈される JSON を除く)。 - -デフォルトでは無効です。 - -**例** - -```sql -SET input_format_try_infer_exponent_floats = 1; -DESC format(CSV, -$$1.1E10 -2.3e-12 -42E00 -$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## 自己記述フォーマット {#self-describing-formats} - -自己記述フォーマットは、データ自体の中にデータ構造に関する情報を含むフォーマットです。 -ヘッダーに記述が含まれていたり、バイナリの型ツリーや、ある種のテーブルとして表現されている場合があります。 -このようなフォーマットのファイルからスキーマを自動的に推論するために、ClickHouse は型に関する情報を含むデータの一部を読み取り、 -それを ClickHouse テーブルのスキーマに変換します。 - -### -WithNamesAndTypes サフィックス付きフォーマット {#formats-with-names-and-types} - -ClickHouse は、サフィックス -WithNamesAndTypes を持ついくつかのテキストフォーマットをサポートしています。このサフィックスは、実データの前にカラム名と型を含む追加の 2 行がデータに含まれていることを意味します。 -このようなフォーマットに対してスキーマ推論を行う際、ClickHouse は最初の 2 行を読み取り、カラム名と型を抽出します。 - -**例** - -```sql -DESC format(TSVWithNamesAndTypes, -$$num str arr -UInt8 String Array(UInt8) -42 Hello, World! [1,2,3] -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### メタデータを含む JSON フォーマット {#json-with-metadata} - -一部の JSON 入力フォーマット([JSON](/interfaces/formats/JSON)、[JSONCompact](/interfaces/formats/JSONCompact)、[JSONColumnsWithMetadata](/interfaces/formats/JSONColumnsWithMetadata))には、列名およびデータ型に関するメタデータが含まれます。 -そのようなフォーマットでスキーマ推論を行う際、ClickHouse はこのメタデータを参照します。 - -**例** - -```sql -DESC format(JSON, $$ -{ - "meta": - [ - { - "name": "num", - "type": "UInt8" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "Hello, World", - "arr": [1,2,3] - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.005723915, - "rows_read": 1, - "bytes_read": 1 - } -} -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Avro {#avro} - -Avro 形式では、ClickHouse はデータからスキーマを読み取り、以下の型の対応付けに基づいて ClickHouse のスキーマに変換します。 - - -| Avro データ型 | ClickHouse データ型 | -|------------------------------------|--------------------------------------------------------------------------------| -| `boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int (date)` \* | [Date32](../sql-reference/data-types/date32.md) | -| `long` | [Int64](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `bytes`, `string` | [String](../sql-reference/data-types/string.md) | -| `fixed` | [FixedString(N)](../sql-reference/data-types/fixedstring.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `array(T)` | [Array(T)](../sql-reference/data-types/array.md) | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](../sql-reference/data-types/date.md) | -| `null` | [Nullable(Nothing)](../sql-reference/data-types/special-data-types/nothing.md) | -| `string (uuid)` \* | [UUID](../sql-reference/data-types/uuid.md) | -| `binary (decimal)` \* | [Decimal(P, S)](../sql-reference/data-types/decimal.md) | - -\* [Avro logical types](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -その他の Avro 型はサポートされていません。 - -### Parquet {#parquet} - -Parquet フォーマットでは、ClickHouse はデータからスキーマを読み取り、次の型対応に従って ClickHouse のスキーマに変換します。 - -| Parquet データ型 | ClickHouse データ型 | -|------------------------------|---------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE` | [Date32](../sql-reference/data-types/date32.md) | -| `TIME (ms)` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME (us, ns)` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -その他の Parquet 型はサポートされていません。 - -### Arrow {#arrow} - -Arrow フォーマットでは、ClickHouse はデータからスキーマを読み取り、次の型対応に従って ClickHouse のスキーマに変換します。 - - - -| Arrow データ型 | ClickHouse データ型 | -|---------------------------------|---------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT`, `HALF_FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE32` | [Date32](../sql-reference/data-types/date32.md) | -| `DATE64` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL128`, `DECIMAL256` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -その他の Arrow 型はサポートされていません。 - -### ORC {#orc} - -ORC 形式では、ClickHouse はスキーマをデータから読み取り、以下の型対応に従って ClickHouse のスキーマに変換します。 - -| ORC データ型 | ClickHouse データ型 | -|--------------------------------------|---------------------------------------------------------| -| `Boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `Tinyint` | [Int8](../sql-reference/data-types/int-uint.md) | -| `Smallint` | [Int16](../sql-reference/data-types/int-uint.md) | -| `Int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `Bigint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `Float` | [Float32](../sql-reference/data-types/float.md) | -| `Double` | [Float64](../sql-reference/data-types/float.md) | -| `Date` | [Date32](../sql-reference/data-types/date32.md) | -| `Timestamp` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `String`, `Char`, `Varchar`,`BINARY` | [String](../sql-reference/data-types/string.md) | -| `Decimal` | [Decimal](../sql-reference/data-types/decimal.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `Struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `Map` | [Map](../sql-reference/data-types/map.md) | - -その他の ORC 型はサポートされていません。 - -### Native {#native} - -Native 形式は ClickHouse 内部で使用され、スキーマをデータ内に含みます。 -スキーマ推論では、ClickHouse は変換を一切行わずにデータからスキーマを読み取ります。 - - - -## 外部スキーマを用いるフォーマット {#formats-with-external-schema} - -この種のフォーマットでは、特定のスキーマ言語で記述された、データを表すスキーマを別ファイルで用意する必要があります。 -これらのフォーマットのファイルからスキーマを自動推論するために、ClickHouse は外部スキーマを別ファイルから読み込み、ClickHouse のテーブルスキーマに変換します。 - -### Protobuf {#protobuf} - -Protobuf フォーマットのスキーマ推論では、ClickHouse は次の型の対応関係を使用します。 - -| Protobuf data type | ClickHouse data type | -|-------------------------------|---------------------------------------------------| -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `int32`, `sint32`, `sfixed32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int64`, `sint64`, `sfixed64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `uint32`, `fixed32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `uint64`, `fixed64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `string`, `bytes` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `repeated T` | [Array(T)](../sql-reference/data-types/array.md) | -| `message`, `group` | [Tuple](../sql-reference/data-types/tuple.md) | - -### CapnProto {#capnproto} - -CapnProto フォーマットのスキーマ推論では、ClickHouse は次の型の対応関係を使用します。 - -| CapnProto data type | ClickHouse data type | -|------------------------------------|--------------------------------------------------------| -| `Bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UInt8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UInt16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `Int32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UInt32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `Int64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `UInt64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `Float32` | [Float32](../sql-reference/data-types/float.md) | -| `Float64` | [Float64](../sql-reference/data-types/float.md) | -| `Text`, `Data` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `union(T, Void)`, `union(Void, T)` | [Nullable(T)](../sql-reference/data-types/nullable.md) | - - - -## 強い型付けのバイナリ形式 {#strong-typed-binary-formats} - -この種の形式では、シリアル化された各値にはその型(および場合によっては名前)に関する情報が含まれますが、テーブル全体に関する情報は含まれません。 -このような形式に対するスキーマ推論では、ClickHouse は行ごとにデータを読み込み(最大 `input_format_max_rows_to_read_for_schema_inference` 行または `input_format_max_bytes_to_read_for_schema_inference` バイト)、データから各値の -型(および場合によっては名前)を抽出し、その後それらの型を ClickHouse の型に変換します。 - -### MsgPack {#msgpack} - -MsgPack 形式では行の区切りが存在しないため、この形式に対してスキーマ推論を行うには、設定 `input_format_msgpack_number_of_columns` を用いて -テーブル内の列数を指定する必要があります。ClickHouse は次の型対応を使用します。 - -| MessagePack data type (`INSERT`) | ClickHouse data type | -|--------------------------------------------------------------------|-----------------------------------------------------------| -| `int N`, `uint N`, `negative fixint`, `positive fixint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [String](../sql-reference/data-types/string.md) | -| `float 32` | [Float32](../sql-reference/data-types/float.md) | -| `float 64` | [Float64](../sql-reference/data-types/float.md) | -| `uint 16` | [Date](../sql-reference/data-types/date.md) | -| `uint 32` | [DateTime](../sql-reference/data-types/datetime.md) | -| `uint 64` | [DateTime64](../sql-reference/data-types/datetime.md) | -| `fixarray`, `array 16`, `array 32` | [Array](../sql-reference/data-types/array.md) | -| `fixmap`, `map 16`, `map 32` | [Map](../sql-reference/data-types/map.md) | - -デフォルトでは、推論されたすべての型は `Nullable` でラップされますが、これは設定 `schema_inference_make_columns_nullable` を使用して変更できます。 - -### BSONEachRow {#bsoneachrow} - -BSONEachRow では、各データ行は BSON ドキュメントとして表現されます。スキーマ推論では、ClickHouse は BSON ドキュメントを 1 つずつ読み込み、 -データから値、名前、および型を抽出し、その後次の型対応を使用してそれらの型を ClickHouse の型に変換します。 - -| BSON Type | ClickHouse type | -|-----------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------| -| `\x08` boolean | [Bool](../sql-reference/data-types/boolean.md) | -| `\x10` int32 | [Int32](../sql-reference/data-types/int-uint.md) | -| `\x12` int64 | [Int64](../sql-reference/data-types/int-uint.md) | -| `\x01` double | [Float64](../sql-reference/data-types/float.md) | -| `\x09` datetime | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `\x05` binary with `\x00` binary subtype, `\x02` string, `\x0E` symbol, `\x0D` JavaScript code | [String](../sql-reference/data-types/string.md) | -| `\x07` ObjectId, | [FixedString(12)](../sql-reference/data-types/fixedstring.md) | -| `\x05` binary with `\x04` UUID subtype, size = 16 | [UUID](../sql-reference/data-types/uuid.md) | -| `\x04` array | [Array](../sql-reference/data-types/array.md)/[Tuple](../sql-reference/data-types/tuple.md)(ネストされた型が異なる場合) | -| `\x03` document | [Named Tuple](../sql-reference/data-types/tuple.md)/[Map](../sql-reference/data-types/map.md)(キーは String) | - -デフォルトでは、推論されたすべての型は `Nullable` でラップされますが、これは設定 `schema_inference_make_columns_nullable` を使用して変更できます。 - - - -## 固定スキーマを持つフォーマット {#formats-with-constant-schema} - -このようなフォーマットのデータは、常に同じスキーマを持ちます。 - -### LineAsString {#line-as-string} - -このフォーマットでは、ClickHouse はデータから1行全体を `String` 型の1つのカラムとして読み込みます。このフォーマットで推論されるデータ型は常に `String` であり、カラム名は `line` です。 - -**例** - -```sql -DESC format(LineAsString, 'Hello\nworld!') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ line │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsString {#json-as-string} - -このフォーマットでは、ClickHouse はデータ内の JSON オブジェクト全体を `String` 型の単一の列として読み取ります。このフォーマットで推論される型は常に `String` で、列名は `json` です。 - -**例** - -```sql -DESC format(JSONAsString, '{"x" : 42, "y" : "Hello, World!"}') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsObject {#json-as-object} - -この形式では、ClickHouse はデータ内の JSON オブジェクト全体を `JSON` データ型の単一の列として読み取ります。この形式で推論されるデータ型は常に `JSON` であり、列名は `json` です。 - -**例** - -```sql -DESC format(JSONAsObject, '{"x" : 42, "y" : "Hello, World!"}'); -``` - -```response -┌─name─┬─type─┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ JSON │ │ │ │ │ │ -└──────┴──────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## スキーマ推論モード {#schema-inference-modes} - -データファイルの集合からのスキーマ推論は、`default` と `union` の 2 つのモードで動作します。 -モードは設定項目 `schema_inference_mode` で制御されます。 - -### `default` モード {#default-schema-inference-mode} - -`default` モードでは、ClickHouse はすべてのファイルが同一のスキーマを持つと仮定し、スキーマの推論に成功するまでファイルを 1 つずつ読み込みます。 - -例: - -`data1.jsonl`、`data2.jsonl`、`data3.jsonl` の 3 つのファイルがあり、その内容が次のようになっているとします。 - -`data1.jsonl`: - -```json -{"field1" : 1, "field2" : null} -{"field1" : 2, "field2" : null} -{"field1" : 3, "field2" : null} -``` - -`data2.jsonl`: - -```json -{"field1" : 4, "field2" : "Data4"} -{"field1" : 5, "field2" : "Data5"} -{"field1" : 6, "field2" : "Data5"} -``` - -`data3.jsonl`: - -```json -{"field1" : 7, "field2" : "Data7", "field3" : [1, 2, 3]} -{"field1" : 8, "field2" : "Data8", "field3" : [4, 5, 6]} -{"field1" : 9, "field2" : "Data9", "field3" : [7, 8, 9]} -``` - -これら3つのファイルに対してスキーマ推論を試してみましょう。 - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='default' -``` - -結果: - -```response -┌─name───┬─type─────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -└────────┴──────────────────┘ -``` - -ご覧のとおり、ファイル `data3.jsonl` には `field3` がありません。 -これは、ClickHouse がまずファイル `data1.jsonl` からスキーマの推論を行おうとしましたが、`field2` フィールドがすべて null だったために失敗し、 -その後 `data2.jsonl` からスキーマ推論を行って成功した結果、ファイル `data3.jsonl` のデータが読み込まれなかったためです。 - -### Union モード {#default-schema-inference-mode-1} - -Union モードでは、ClickHouse はファイルごとに異なるスキーマを持つ可能性があるとみなし、すべてのファイルのスキーマを推論してから、それらを共通スキーマに対して union します。 - -`data1.jsonl`、`data2.jsonl`、`data3.jsonl` という 3 つのファイルがあり、その内容が次のようであるとします。 - -`data1.jsonl`: - -```json -{"field1" : 1} -{"field1" : 2} -{"field1" : 3} -``` - -`data2.jsonl`: - -```json -{"field2" : "Data4"} -{"field2" : "Data5"} -{"field2" : "Data5"} -``` - -`data3.jsonl`: - -```json -{"field3" : [1, 2, 3]} -{"field3" : [4, 5, 6]} -{"field3" : [7, 8, 9]} -``` - -これら 3 つのファイルに対してスキーマ推論を試してみましょう。 - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='union' -``` - -結果: - -```response -┌─name───┬─type───────────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -│ field3 │ Array(Nullable(Int64)) │ -└────────┴────────────────────────┘ -``` - -ご覧のとおり、すべてのファイルからすべてのフィールドが取得されています。 - -注意: - -* 一部のファイルには結果のスキーマに含まれる特定のカラムが存在しない場合があるため、union モードはカラムのサブセット読み取りをサポートするフォーマット(JSONEachRow、Parquet、TSVWithNames など)でのみ使用でき、それ以外のフォーマット(CSV、TSV、JSONCompactEachRow など)では動作しません。 -* ClickHouse がファイルの 1 つからスキーマを推論できない場合は、例外がスローされます。 -* ファイル数が多い場合、すべてのファイルからスキーマを読み取る処理に多くの時間がかかる可能性があります。 - - -## 自動フォーマット検出 {#automatic-format-detection} - -データのフォーマットが指定されておらず、かつファイル拡張子からも判定できない場合、ClickHouse はファイルの内容に基づいてフォーマットの検出を試みます。 - -**例:** - -`data` というファイルに次のような内容があるとします: - -```csv -"a","b" -1,"Data1" -2,"Data2" -3,"Data3" -``` - -形式や構造を指定せずに、このファイルを確認しクエリを実行できます。 - -```sql -:) desc file(data); -``` - -```repsonse -┌─name─┬─type─────────────┐ -│ a │ Nullable(Int64) │ -│ b │ Nullable(String) │ -└──────┴──────────────────┘ -``` - -```sql -:) select * from file(data); -``` - -```response -┌─a─┬─b─────┐ -│ 1 │ Data1 │ -│ 2 │ Data2 │ -│ 3 │ Data3 │ -└───┴───────┘ -``` - -:::note -ClickHouse は一部のフォーマットしか自動検出できず、検出にも時間がかかるため、フォーマットは明示的に指定することをお勧めします。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/ssh.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/ssh.md deleted file mode 100644 index 1cf2c91c809..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/ssh.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'ClickHouse の SSH インターフェイスに関するドキュメント' -keywords: ['client', 'ssh', 'putty'] -sidebar_label: 'SSH インターフェイス' -sidebar_position: 60 -slug: /interfaces/ssh -title: 'SSH インターフェイス' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# PTY を使用する SSH インターフェース {#ssh-interface-with-pty} - - - - - -## 序文 {#preface} - -ClickHouse サーバーは、SSH プロトコルを使用して自身に直接接続できます。任意のクライアントから接続可能です。 - -[SSH キーで認証されるデータベースユーザー](/knowledgebase/how-to-connect-to-ch-cloud-using-ssh-keys) を作成したら、次のようにします。 - -```sql -CREATE USER abcuser IDENTIFIED WITH ssh_key BY KEY '' TYPE 'ssh-ed25519'; -``` - -このキーを使用して ClickHouse サーバーに接続できます。`clickhouse-client` の対話型セッション用の擬似端末 (PTY) が開きます。 - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 -ClickHouse embedded version 25.1.1.1. - -ip-10-1-13-116.us-west-2.compute.internal :) SELECT 1; - -SELECT 1 - -Query id: cdd91b7f-215b-4537-b7df-86d19bf63f64 - - ┌─1─┐ -1. │ 1 │ - └───┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -SSH 経由でのコマンド実行(非対話モード)にも対応しています。 - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "select 1" -1 -``` - -## サーバー設定 {#server-configuration} - -SSH サーバー機能を有効にするには、`config.xml` 内で次のセクションのコメントアウトを解除するか、追加する必要があります: - -```xml -9022 - - 鍵へのパス - - - -``` - -ホストキーは SSH プロトコルにおいて不可欠な要素です。このキーの公開鍵部分はクライアント側の `~/.ssh/known_hosts` ファイルに保存されており、通常は中間者攻撃を防ぐために利用されます。サーバーに初めて接続すると、次のようなメッセージが表示されます。 - -```shell -ホスト '[localhost]:9022 ([127.0.0.1]:9022)' の真正性を確立できません。 -RSA 鍵フィンガープリントは SHA256:3qxVlJKMr/PEKw/hfeg06HAK451Tt0eenhwqQvh58Do です。 -この鍵は他の名前では認識されていません -接続を続行しますか (yes/no/[fingerprint])? -``` - -これは実際には「このホストの公開鍵を記憶して、接続を続行しますか?」という意味です。 - -SSH クライアントにオプションを指定することで、ホストの検証を行わないように指示できます。 - -```bash -ssh -o "StrictHostKeyChecking no" user@host -``` - -## 組み込みクライアントの設定 {#configuring-embedded-client} - -組み込みクライアントには、通常の `clickhouse-client` と同様にオプションを指定できますが、いくつか制限があります。 -SSH プロトコルを使用しているため、接続先ホストにパラメータを渡す唯一の方法は環境変数を通すことです。 - -例えば、`format` を設定するには次のようにします。 - -```bash -> ssh -o SetEnv="format=Pretty" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` - -この方法で任意のユーザーレベルの設定を変更できるほか、ほとんどの一般的な `clickhouse-client` オプションも渡すことができます(このセットアップでは意味をなさないものを除きます)。 - -重要: - -`query` オプションと SSH コマンドの両方が指定された場合、後者は実行するクエリのリストに追加されます。 - -```bash -ubuntu ip-10-1-13-116@~$ ssh -o SetEnv="format=Pretty query=\"SELECT 2;\"" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 2 ┃ - ┡━━━┩ -1. │ 2 │ - └───┘ - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/tcp.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/tcp.md deleted file mode 100644 index 112c6308667..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/tcp.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ClickHouse のネイティブ TCP インターフェイスに関するドキュメント' -sidebar_label: 'ネイティブインターフェイス (TCP)' -sidebar_position: 18 -slug: /interfaces/tcp -title: 'ネイティブインターフェイス (TCP)' -doc_type: 'reference' ---- - -# ネイティブインターフェース (TCP) {#native-interface-tcp} - -ネイティブプロトコルは、[コマンドラインクライアント](../interfaces/cli.md)、分散クエリ処理中のサーバー間通信、およびその他の C++ プログラムで使用されます。残念ながら、ClickHouse のネイティブプロトコルにはまだ正式な仕様がありませんが、ClickHouse のソースコード([このあたり](https://github.com/ClickHouse/ClickHouse/tree/master/src/Client) から始まります)からリバースエンジニアリングすることや、TCP トラフィックを傍受して解析することで明らかにすることができます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md deleted file mode 100644 index 8cfb74f23b4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: 'さまざまなプログラミング言語向けに利用可能なサードパーティ製クライアントライブラリの概要' -sidebar_label: 'クライアントライブラリ' -sidebar_position: 26 -slug: /interfaces/third-party/client-libraries -title: 'サードパーティ製クライアントライブラリ' -doc_type: 'reference' ---- - -# サードパーティ開発者によるクライアントライブラリ {#client-libraries-from-third-party-developers} - -:::note -ClickHouse Inc は、以下に記載されているライブラリをメンテナンスしておらず、その品質を保証するための広範なテストも実施していません。 -::: - -### Python {#python} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm) -* [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) -* [clickhouse-client](https://github.com/yurial/clickhouse-client) -* [aiochclient](https://github.com/maximdanilchenko/aiochclient) -* [asynch](https://github.com/long2ice/asynch) - -### PHP {#php} - -* [smi2/phpclickhouse](https://packagist.org/packages/smi2/phpClickHouse) -* [8bitov/clickhouse-php-client](https://packagist.org/packages/8bitov/clickhouse-php-client) -* [bozerkins/clickhouse-client](https://packagist.org/packages/bozerkins/clickhouse-client) -* [simpod/clickhouse-client](https://packagist.org/packages/simpod/clickhouse-client) -* [seva-code/php-click-house-client](https://packagist.org/packages/seva-code/php-click-house-client) -* [SeasClick C++ クライアント](https://github.com/SeasX/SeasClick) -* [one-ck](https://github.com/lizhichao/one-ck) -* [glushkovds/phpclickhouse-laravel](https://packagist.org/packages/glushkovds/phpclickhouse-laravel) -* [glushkovds/php-clickhouse-schema-builder](https://packagist.org/packages/glushkovds/php-clickhouse-schema-builder) -* [kolya7k ClickHouse PHP 拡張モジュール](https://github.com//kolya7k/clickhouse-php) -* [hyvor/clickhouse-php](https://github.com/hyvor/clickhouse-php) - -### Go {#go} - -* [ClickHouse](https://github.com/kshvakov/clickhouse/) -* [go-clickhouse](https://github.com/roistat/go-clickhouse) -* [chconn](https://github.com/vahid-sohrabloo/chconn) -* [mailrugo-clickhouse](https://github.com/mailru/go-clickhouse) -* [golang-clickhouse](https://github.com/leprosus/golang-clickhouse) -* [uptrace/go-clickhouse](https://clickhouse.uptrace.dev/) - -### Swift {#swift} - -* [ClickHouseNIO](https://github.com/patrick-zippenfenig/ClickHouseNIO) -* [ClickHouseVapor ORM](https://github.com/patrick-zippenfenig/ClickHouseVapor) - -### Node.js {#nodejs} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [clickhouse (NodeJs)](https://github.com/TimonKK/clickhouse) -* [node-clickhouse](https://github.com/apla/node-clickhouse) -* [nestjs-clickhouse](https://github.com/depyronick/nestjs-clickhouse) -* [clickhouse-client](https://github.com/depyronick/clickhouse-client) -* [node-clickhouse-orm](https://github.com/zimv/node-clickhouse-orm) -* [clickhouse-ts](https://github.com/bytadaniel/clickhouse-ts) -* [clickcache](https://github.com/bytadaniel/clickcache) - -### Perl {#perl} - -* [perl-DBD-ClickHouse](https://github.com/elcamlost/perl-DBD-ClickHouse) -* [HTTP-ClickHouse](https://metacpan.org/release/HTTP-ClickHouse) -* [AnyEvent-ClickHouse](https://metacpan.org/release/AnyEvent-ClickHouse) - -### Ruby {#ruby} - -* [ClickHouse(Ruby)](https://github.com/shlima/click_house) -* [clickhouse-activerecord](https://github.com/PNixx/clickhouse-activerecord) - -### Rust {#rust} - -* [clickhouse.rs](https://github.com/loyd/clickhouse.rs) -* [clickhouse-rs](https://github.com/suharev7/clickhouse-rs) -* [Klickhouse](https://github.com/Protryon/klickhouse) - -### R {#r} - -* [RClickHouse](https://github.com/IMSMWU/RClickHouse) - -### Java {#java} - -* [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java) -* [clickhouse-client](https://github.com/Ecwid/clickhouse-client) - -### Scala {#scala} - -* [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) - -### Kotlin {#kotlin} - -* [AORM](https://github.com/TanVD/AORM) - -### C# {#c} - -* [Octonica.ClickHouseClient](https://github.com/Octonica/ClickHouseClient) -* [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) -* [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) -* [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - -### Elixir {#elixir} - -* [clickhousex](https://github.com/appodeal/clickhousex/) -* [pillar](https://github.com/sofakingworld/pillar) -* [ecto_ch](https://github.com/plausible/ecto_ch) -* [req_ch](https://github.com/livebook-dev/req_ch) - -### Nim {#nim} - -* [nim-clickhouse](https://github.com/leonardoce/nim-clickhouse) - -### Haskell {#haskell} - -* [hdbc-clickhouse](https://github.com/zaneli/hdbc-clickhouse) -* [ClickHaskell](https://clickhaskell.dev/) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md deleted file mode 100644 index d181f593088..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md +++ /dev/null @@ -1,445 +0,0 @@ ---- -description: 'ClickHouse を操作するためのサードパーティ製 GUI ツールおよびアプリケーションの一覧' -sidebar_label: 'ビジュアルインターフェイス' -sidebar_position: 28 -slug: /interfaces/third-party/gui -title: 'サードパーティ製ビジュアルインターフェイス' -doc_type: 'reference' ---- - - - -# サードパーティ開発のビジュアルインターフェース {#visual-interfaces-from-third-party-developers} - - - -## オープンソース {#open-source} - -### agx {#agx} - -[agx](https://github.com/agnosticeng/agx) は、Tauri と SvelteKit で構築されたデスクトップアプリケーションで、ClickHouse の組み込みデータベースエンジン (chdb) を利用してデータを探索およびクエリするためのモダンなインターフェイスを提供します。 - -- ネイティブアプリケーション実行時に chdb を活用可能。 -- Web 版として実行する場合、ClickHouse インスタンスに接続可能。 -- Monaco エディタにより、なじみのある操作感を提供。 -- 多様で拡張可能なデータ可視化機能。 - -### ch-ui {#ch-ui} - -[ch-ui](https://github.com/caioricciuti/ch-ui) は、ClickHouse データベース向けに設計されたシンプルな React.js アプリケーションインターフェイスで、クエリ実行とデータ可視化を行うことができます。React と Web 向け ClickHouse クライアントで構築されており、洗練された使いやすい UI により、データベースとのやり取りを容易にします。 - -Features: - -- ClickHouse 連携: 接続の管理やクエリ実行を容易に実施。 -- レスポンシブなタブ管理: クエリタブやテーブルタブなど、複数タブを動的に扱うことが可能。 -- パフォーマンス最適化: Indexed DB を利用した効率的なキャッシュおよび状態管理。 -- ローカルデータ保存: すべてのデータはブラウザ内にローカル保存され、外部には送信されません。 - -### ChartDB {#chartdb} - -[ChartDB](https://chartdb.io) は、ClickHouse を含むデータベーススキーマを単一のクエリで設計・可視化できる、無料かつオープンソースのツールです。React で構築されており、データベースの認証情報やサインアップを行うことなく、シームレスで使いやすい体験を提供します。 - -Features: - -- スキーマの可視化: ClickHouse スキーマを即座にインポートして可視化可能。マテリアライズドビューや標準ビューを含む ER 図を生成し、テーブルへの参照も表示。 -- AI ベースの DDL エクスポート: スキーマ管理とドキュメント化のために、DDL スクリプトを容易に生成。 -- 複数の SQL 方言をサポート: 多様なデータベース環境で利用可能。 -- サインアップや認証情報は不要: すべての機能はブラウザ上から直接利用可能で、摩擦のない安全な利用が可能。 - -[ChartDB Source Code](https://github.com/chartdb/chartdb). - -### DataPup {#datapup} - -[DataPup](https://github.com/DataPupOrg/DataPup) は、ネイティブな ClickHouse サポートを備えた、モダンで AI 支援のクロスプラットフォームデータベースクライアントです。 - -Features: - -- AI による SQL クエリ支援とインテリジェントなサジェスト機能 -- 安全な認証情報管理を備えたネイティブ ClickHouse 接続サポート -- 複数のテーマ (ライト、ダーク、カラフルなバリエーション) を備えた美しくアクセシブルなインターフェイス -- 高度なクエリ結果のフィルタリングと探索機能 -- クロスプラットフォームサポート (macOS, Windows, Linux) -- 高速かつレスポンシブなパフォーマンス -- オープンソースかつ MIT ライセンス - -### ClickHouse Schema Flow Visualizer {#clickhouse-schemaflow-visualizer} - -[ClickHouse Schema Flow Visualizer](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) は、Mermaid.js ダイアグラムを使用して ClickHouse のテーブル間リレーションシップを可視化する強力なオープンソース Web アプリケーションです。直感的なインターフェイスでデータベースやテーブルをブラウズし、オプションの行数やサイズ情報を含むテーブルメタデータを探索し、インタラクティブなスキーマダイアグラムをエクスポートできます。 - -Features: - -- 直感的なインターフェイスで ClickHouse のデータベースとテーブルをブラウズ -- Mermaid.js ダイアグラムでテーブル間のリレーションシップを可視化 -- テーブルタイプに対応した色分けアイコンによる視認性の向上 -- テーブル間のデータフロー方向を表示 -- 図をスタンドアロンの HTML ファイルとしてエクスポート -- メタデータ (テーブル行数およびサイズ情報) の表示切り替え -- TLS サポート付きの安全な ClickHouse 接続 -- あらゆるデバイス向けのレスポンシブ Web インターフェイス - -[ClickHouse Schema Flow Visualizer - source code](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) - -### Tabix {#tabix} - -[Tabix](https://github.com/tabixio/tabix) プロジェクトにおける ClickHouse 向け Web インターフェイス。 - -Features: - -- 追加ソフトウェアをインストールすることなく、ブラウザから直接 ClickHouse を操作可能。 -- 構文ハイライト付きクエリエディタ。 -- コマンドの自動補完。 -- クエリ実行のグラフィカル解析ツール。 -- 配色スキームの選択オプション。 - -[Tabix documentation](https://tabix.io/doc/). - -### HouseOps {#houseops} - -[HouseOps](https://github.com/HouseOps/HouseOps) は macOS、Linux、Windows 向けの UI/IDE です。 - -Features: - -- 構文ハイライト付きクエリビルダー。レスポンスをテーブルビューまたは JSON ビューで表示可能。 -- クエリ結果を CSV または JSON としてエクスポート可能。 -- 説明付きのプロセス一覧。書き込みモード。プロセスを停止する (`KILL`) 機能。 -- データベースグラフ。すべてのテーブルとそのカラム、および追加情報を表示。 -- カラムサイズのクイックビュー。 -- サーバー設定管理。 - -今後開発が予定されている機能: - -- データベース管理。 -- ユーザー管理。 -- リアルタイムデータ分析。 -- クラスター監視。 -- クラスター管理。 -- レプリケートテーブルおよび Kafka テーブルの監視。 - -### LightHouse {#lighthouse} - - - -[LightHouse](https://github.com/VKCOM/lighthouse) は、ClickHouse 向けの軽量な Web インターフェイスです。 - -特徴: - -- フィルタリングおよびメタデータ付きのテーブル一覧。 -- フィルタリングおよびソートが可能なテーブルプレビュー。 -- 読み取り専用クエリの実行。 - -### Redash {#redash} - -[Redash](https://github.com/getredash/redash) は、データ可視化のためのプラットフォームです。 - -ClickHouse を含む複数のデータソースをサポートしており、Redash は異なるデータソースからのクエリ結果を 1 つの最終データセットに結合できます。 - -特徴: - -- 強力なクエリエディタ。 -- データベースエクスプローラー。 -- データをさまざまな形式で表現できる可視化ツール。 - -### Grafana {#grafana} - -[Grafana](https://grafana.com/grafana/plugins/grafana-clickhouse-datasource/) は、モニタリングと可視化のためのプラットフォームです。 - -"Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data-driven culture. Trusted and loved by the community" — grafana.com. - -ClickHouse data source プラグインにより、ClickHouse をバックエンドデータベースとして利用できます。 - -### qryn {#qryn} - -[qryn](https://metrico.in) は、ClickHouse 向けの複数プロトコル対応・高性能なオブザーバビリティスタック _(旧称 cLoki)_ であり、ネイティブな Grafana 連携により、Loki/LogQL、Prometheus/PromQL、OTLP/Tempo、Elastic、InfluxDB などをサポートするあらゆるエージェントからログ、メトリクス、テレメトリトレースを取り込み、分析できます。 - -特徴: - -- データのクエリ、抽出、可視化のための組み込み Explore UI と LogQL CLI -- プラグイン不要で、クエリ、処理、インジェスト、トレース、アラートに対応するネイティブ Grafana API サポート -- ログやイベント、トレースなどから動的にデータを検索、フィルタ、抽出できる強力なパイプライン -- LogQL、PromQL、InfluxDB、Elastic などと透過的に互換性のあるインジェストおよび PUSH API -- Promtail、Grafana-Agent、Vector、Logstash、Telegraf などのエージェントですぐに利用可能 - -### DBeaver {#dbeaver} - -[DBeaver](https://dbeaver.io/) は、ClickHouse をサポートする汎用デスクトップデータベースクライアントです。 - -特徴: - -- 構文ハイライトとオートコンプリートによるクエリ開発。 -- フィルタおよびメタデータ検索付きのテーブル一覧。 -- テーブルデータのプレビュー。 -- フルテキスト検索。 - -デフォルトでは、DBeaver はセッションを使用して接続しません(CLI などは使用します)。セッションサポートが必要な場合(例: セッションに対して設定を行いたい場合)、ドライバーの接続プロパティを編集し、`session_id` をランダムな文字列に設定します(内部的には HTTP 接続を使用します)。その後、クエリウィンドウから任意の設定を使用できます。 - -### clickhouse-cli {#clickhouse-cli} - -[clickhouse-cli](https://github.com/hatarist/clickhouse-cli) は、Python 3 で実装された ClickHouse 用の代替コマンドラインクライアントです。 - -特徴: - -- オートコンプリート。 -- クエリおよびデータ出力の構文ハイライト。 -- データ出力のページャサポート。 -- PostgreSQL 風のカスタムコマンド。 - -### clickhouse-flamegraph {#clickhouse-flamegraph} - -[clickhouse-flamegraph](https://github.com/Slach/clickhouse-flamegraph) は、`system.trace_log` を [flamegraph](http://www.brendangregg.com/flamegraphs.html) として可視化するための専用ツールです。 - -### clickhouse-plantuml {#clickhouse-plantuml} - -[cickhouse-plantuml](https://pypi.org/project/clickhouse-plantuml/) は、テーブルスキーマの [PlantUML](https://plantuml.com/) 図を生成するためのスクリプトです。 - -### ClickHouse table graph {#clickhouse-table-graph} - -[ClickHouse table graph](https://github.com/mbaksheev/clickhouse-table-graph) は、ClickHouse テーブル間の依存関係を可視化するためのシンプルな CLI ツールです。このツールは `system.tables` テーブルからテーブル間の接続情報を取得し、[mermaid](https://mermaid.js.org/syntax/flowchart.html) 形式で依存関係のフローチャートを構築します。このツールを使用することで、テーブルの依存関係を簡単に可視化し、ClickHouse データベース内のデータフローを把握できます。mermaid により、生成されたフローチャートは見栄えが良く、Markdown ドキュメントにも容易に追加できます。 - -### xeus-clickhouse {#xeus-clickhouse} - -[xeus-clickhouse](https://github.com/wangfenjin/xeus-clickhouse) は ClickHouse 用の Jupyter カーネルであり、Jupyter 上で SQL を使用して ClickHouse のデータをクエリできます。 - -### MindsDB Studio {#mindsdb} - - - -[MindsDB](https://mindsdb.com/) は、ClickHouse を含むデータベース向けのオープンソースの AI レイヤーであり、最先端の機械学習モデルを容易に開発・学習・デプロイできるようにします。MindsDB Studio(GUI)を使用すると、データベースから新しいモデルを学習させ、モデルによる予測結果を解釈し、潜在的なデータバイアスを特定し、Explainable AI 機能を用いてモデル精度を評価および可視化することで、機械学習モデルをより迅速に適応・チューニングできます。 - -### DBM {#dbm} - -[DBM](https://github.com/devlive-community/dbm) DBM は ClickHouse 用のビジュアル管理ツールです。 - -機能: - -- クエリ履歴のサポート(ページネーション、すべてクリアなど) -- 選択した SQL 句のクエリをサポート -- クエリの強制終了をサポート -- テーブル管理をサポート(メタデータ、削除、プレビュー) -- データベース管理をサポート(削除、作成) -- カスタムクエリをサポート -- 複数のデータソース管理をサポート(接続テスト、モニタリング) -- モニタリングをサポート(プロセッサ、接続、クエリ) -- データ移行をサポート - -### Bytebase {#bytebase} - -[Bytebase](https://bytebase.com) は、チーム向けの Web ベースのオープンソースのスキーマ変更およびバージョン管理ツールです。ClickHouse を含む各種データベースをサポートします。 - -機能: - -- 開発者と DBA 間でのスキーマレビュー。 -- Database-as-Code による、GitLab などの VCS でのスキーマのバージョン管理と、コードコミット時のデプロイメントのトリガー。 -- 環境ごとのポリシーに基づく効率的なデプロイメント。 -- 完全なマイグレーション履歴。 -- スキーマドリフト検出。 -- バックアップおよびリストア。 -- RBAC。 - -### Zeppelin-Interpreter-for-ClickHouse {#zeppelin-interpreter-for-clickhouse} - -[Zeppelin-Interpreter-for-ClickHouse](https://github.com/SiderZhang/Zeppelin-Interpreter-for-ClickHouse) は、ClickHouse 向けの [Zeppelin](https://zeppelin.apache.org) インタプリタです。JDBC インタプリタと比較して、長時間実行されるクエリに対してより優れたタイムアウト制御を提供します。 - -### ClickCat {#clickcat} - -[ClickCat](https://github.com/clickcat-project/ClickCat) は、ClickHouse データを検索、探索、および可視化できる、使いやすいユーザーインターフェースです。 - -機能: - -- インストール不要で SQL コードを実行できるオンライン SQL エディタ。 -- すべてのプロセスおよびミューテーションを確認可能。未完了のプロセスについては、UI から強制終了できます。 -- メトリクスには、クラスター分析、データ分析、およびクエリ分析が含まれます。 - -### ClickVisual {#clickvisual} - -[ClickVisual](https://clickvisual.net/) ClickVisual は、軽量なオープンソースのログクエリ・分析・アラーム可視化プラットフォームです。 - -機能: - -- ワンクリックで分析ログライブラリを作成可能 -- ログ収集設定管理をサポート -- ユーザー定義インデックス設定をサポート -- アラーム設定をサポート -- ライブラリおよびテーブル単位のきめ細かな権限設定に対応 - -### ClickHouse-Mate {#clickmate} - -[ClickHouse-Mate](https://github.com/metrico/clickhouse-mate) は、ClickHouse 内のデータを検索および探索するための Angular 製 Web クライアント + ユーザーインターフェースです。 - -機能: - -- ClickHouse SQL クエリのオートコンプリート -- 高速なデータベースおよびテーブルツリーナビゲーション -- 高度な結果フィルタリングおよびソート -- インライン ClickHouse SQL ドキュメント -- クエリプリセットおよび履歴 -- 100% ブラウザベースで、サーバー/バックエンド不要 - -クライアントは GitHub Pages を通じてすぐに利用可能です: https://metrico.github.io/clickhouse-mate/ - -### Uptrace {#uptrace} - -[Uptrace](https://github.com/uptrace/uptrace) は、OpenTelemetry と ClickHouse を基盤とした分散トレーシングおよびメトリクスを提供する APM ツールです。 - -機能: - -- [OpenTelemetry トレーシング](https://uptrace.dev/opentelemetry/distributed-tracing.html)、メトリクス、およびログ。 -- AlertManager を使用した Email/Slack/PagerDuty 通知。 -- スパンを集約するための SQL ライクなクエリ言語。 -- メトリクスをクエリするための PromQL ライクな言語。 -- 事前構築済みのメトリクスダッシュボード。 -- YAML 設定による複数ユーザー/プロジェクトのサポート。 - -### clickhouse-monitoring {#clickhouse-monitoring} - -[clickhouse-monitoring](https://github.com/duyet/clickhouse-monitoring) は、`system.*` テーブルに依存して ClickHouse クラスターの監視および概要把握を支援する、シンプルな Next.js ダッシュボードです。 - -機能: - -- クエリモニター: 現在のクエリ、クエリ履歴、クエリリソース(メモリ、読み取られたパーツ、file_open など)、最も高コストなクエリ、最も使用されているテーブルやカラムなど。 -- クラスター監視: 合計メモリ/CPU 使用量、分散キュー、グローバル設定、MergeTree 設定、メトリクスなど。 -- テーブルおよびパーツ情報: サイズ、行数、圧縮、パーツサイズなど、カラムレベルの詳細。 -- 有用なツール: Zookeeper データ探索、クエリ EXPLAIN、クエリの強制終了など。 -- 可視化メトリクスチャート: クエリおよびリソース使用量、マージ/ミューテーション数、マージ性能、クエリ性能など。 - -### CKibana {#ckibana} - - - -[CKibana](https://github.com/TongchengOpenSource/ckibana) は、ネイティブな Kibana UI を使用して ClickHouse のデータを手軽に検索・探索・可視化できる軽量なサービスです。 - -機能: - -- ネイティブな Kibana UI からのチャートのリクエストを ClickHouse 用のクエリ構文に変換します。 -- サンプリングやキャッシュなど、クエリ性能を向上させる高度な機能をサポートします。 -- Elasticsearch から ClickHouse への移行後も、ユーザーの学習コストを最小限に抑えます。 - -### Telescope {#telescope} - -[Telescope](https://iamtelescope.net/) は、ClickHouse に保存されたログを探索するためのモダンな Web インターフェースです。きめ細かなアクセス制御を備えた、ログデータのクエリ、可視化、管理を行うためのユーザーフレンドリーな UI を提供します。 - -機能: - -- 強力なフィルタとカスタマイズ可能なフィールド選択を備えた、クリーンでレスポンシブな UI。 -- 直感的かつ表現力の高いログフィルタリングを可能にする FlyQL 構文。 -- ネストされた JSON、Map、Array フィールドを含む、グループ化に対応した時間ベースのグラフ。 -- 高度なフィルタリングのための任意指定の生の SQL `WHERE` クエリのサポート(権限チェック付き)。 -- Saved Views: クエリおよびレイアウトに対する UI のカスタム構成を保存・共有可能。 -- ロールベースのアクセス制御 (RBAC) および GitHub 認証との連携。 -- ClickHouse 側では追加のエージェントやコンポーネントは不要です。 - -[Telescope ソースコード](https://github.com/iamtelescope/telescope) · [ライブデモ](https://demo.iamtelescope.net) - - - -## 商用 {#commercial} - -### DataGrip {#datagrip} - -[DataGrip](https://www.jetbrains.com/datagrip/) は、JetBrains が提供する ClickHouse をネイティブにサポートするデータベース IDE です。PyCharm、IntelliJ IDEA、GoLand、PhpStorm など他の IntelliJ ベースのツールにも組み込まれています。 - -機能: - -- 非常に高速なコード補完。 -- ClickHouse 構文のシンタックスハイライト。 -- 入れ子カラム、テーブルエンジンなど、ClickHouse 固有機能のサポート。 -- データエディタ。 -- リファクタリング。 -- 検索とナビゲーション。 - -### Yandex DataLens {#yandex-datalens} - -[Yandex DataLens](https://cloud.yandex.ru/services/datalens) は、データの可視化および分析のためのサービスです。 - -機能: - -- シンプルな棒グラフから複雑なダッシュボードまで、多彩なビジュアライゼーション。 -- ダッシュボードを公開して一般に利用可能にすることが可能。 -- ClickHouse を含む複数のデータソースをサポート。 -- ClickHouse をベースにしたマテリアライズドデータ用ストレージ。 - -DataLens は、商用利用であっても、低負荷のプロジェクト向けには[無料で利用可能](https://cloud.yandex.com/docs/datalens/pricing)です。 - -- [DataLens のドキュメント](https://cloud.yandex.com/docs/datalens/)。 -- ClickHouse データベースのデータを可視化するための[チュートリアル](https://cloud.yandex.com/docs/solutions/datalens/data-from-ch-visualization)。 - -### Holistics Software {#holistics-software} - -[Holistics](https://www.holistics.io/) は、フルスタックのデータプラットフォーム兼ビジネスインテリジェンスツールです。 - -機能: - -- レポートの自動メール送信、Slack、Google Sheet へのスケジュール配信。 -- ビジュアライゼーション、バージョン管理、オートコンプリート、再利用可能なクエリコンポーネント、動的フィルタを備えた SQL エディタ。 -- iframe を利用したレポートおよびダッシュボードの埋め込み分析。 -- データ準備および ETL 機能。 -- データのリレーショナルマッピングを行う SQL データモデリングのサポート。 - -### Looker {#looker} - -[Looker](https://looker.com) は、ClickHouse を含む 50 以上のデータベースダイアレクトをサポートするデータプラットフォーム兼ビジネスインテリジェンスツールです。Looker は SaaS プラットフォームおよびセルフホストで利用できます。ユーザーはブラウザから Looker を利用してデータを探索し、ビジュアライゼーションやダッシュボードを作成し、レポートをスケジュールし、インサイトを同僚と共有できます。Looker はこれらの機能を他のアプリケーションに埋め込むための豊富なツールと、他のアプリケーションとデータ連携するための API を提供します。 - -機能: - -- LookML を用いた、容易かつアジャイルな開発。LookML はレポート作成者やエンドユーザーを支援するための精選された - [Data Modeling](https://looker.com/platform/data-modeling) をサポートする言語です。 -- Looker の [Data Actions](https://looker.com/platform/actions) による強力なワークフロー統合。 - -[Looker で ClickHouse を構成する方法。](https://docs.looker.com/setup-and-management/database-config/clickhouse) - -### SeekTable {#seektable} - -[SeekTable](https://www.seektable.com) は、データ探索および運用レポーティング向けのセルフサービス BI ツールです。クラウドサービスとセルフホスト版の両方が利用可能です。SeekTable のレポートは任意の Web アプリケーションに埋め込むことができます。 - -機能: - -- ビジネスユーザーにとって扱いやすいレポートビルダー。 -- SQL フィルタリングおよびレポート固有のクエリカスタマイズのための強力なレポートパラメータ。 -- ネイティブな TCP/IP エンドポイントおよび HTTP(S) インターフェイス(2 種類のドライバ)の両方で ClickHouse に接続可能。 -- ディメンション/メジャーの定義で ClickHouse の SQL 方言のすべての機能を利用可能。 -- 自動レポート生成のための [Web API](https://www.seektable.com/help/web-api-integration)。 -- アカウントデータの [backup/restore](https://www.seektable.com/help/self-hosted-backup-restore) によるレポート開発フローをサポート。データモデル(キューブ)/レポート設定は人間が読める XML で表現され、バージョン管理システムで管理できます。 - -SeekTable は、個人/個人用途での利用については[無料](https://www.seektable.com/help/cloud-pricing)です。 - -[SeekTable で ClickHouse 接続を構成する方法。](https://www.seektable.com/help/clickhouse-pivot-table) - -### Chadmin {#chadmin} - -[Chadmin](https://github.com/bun4uk/chadmin) は、ClickHouse クラスター上で現在実行中のクエリとその情報を可視化し、必要に応じてそれらを強制終了できるシンプルな UI です。 - -### TABLUM.IO {#tablum_io} - -[TABLUM.IO](https://tablum.io/) は、ETL と可視化のためのオンラインクエリおよび分析ツールです。ClickHouse へ接続し、柔軟な SQL コンソール経由でデータをクエリできるほか、静的ファイルやサードパーティサービスからデータをロードすることもできます。TABLUM.IO は、クエリ結果データをチャートやテーブルとして可視化できます。 - - - -機能: -- ETL: 一般的なデータベース、ローカルおよびリモートファイル、API 呼び出しからのデータのロード。 -- シンタックスハイライトとビジュアルクエリビルダーを備えた多機能 SQL コンソール。 -- チャートやテーブルによるデータの可視化。 -- データのマテリアライゼーションおよびサブクエリ。 -- Slack、Telegram、メールへのデータレポート。 -- 独自 API を利用したデータパイプライニング。 -- JSON、CSV、SQL、HTML 形式でのデータエクスポート。 -- Web ベースのインターフェース。 - -TABLUM.IO は、セルフホスト型ソリューション(Docker イメージ)としても、クラウド上でも実行できます。 -ライセンス: 3 か月の無料期間付きの[商用](https://tablum.io/pricing)製品です。 - -[クラウド上](https://tablum.io/try)で無料で試すことができます。 -製品の詳細は [TABLUM.IO](https://tablum.io/) を参照してください。 - -### CKMAN {#ckman} - -[CKMAN](https://www.github.com/housepower/ckman) は、ClickHouse クラスターの管理およびモニタリング用のツールです。 - -機能: - -- ブラウザーインターフェースによる、クラスターの迅速かつ手軽な自動デプロイメント -- クラスターをスケールインまたはスケールアウト可能 -- クラスター内のデータを負荷分散 -- クラスターをオンラインでアップグレード -- Web ページ上からクラスター設定を変更可能 -- クラスターノードおよび ZooKeeper のモニタリングを提供 -- テーブルおよびパーティションの状態、ならびに遅い SQL ステートメントを監視 -- 使いやすい SQL 実行ページを提供 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md deleted file mode 100644 index 5b6a21dae09..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: 'ClickHouse 向けに利用可能なサードパーティツール、ライブラリ、およびインテグレーションの概要' -sidebar_position: 24 -slug: /interfaces/third-party/ -toc_folder_title: 'サードパーティ' -title: 'サードパーティインターフェース' -doc_type: 'landing-page' ---- - -# サードパーティインターフェイス {#third-party-interfaces} - -これは、ClickHouse へのインターフェイスを提供するサードパーティ製ツールへのリンク集です。グラフィカルインターフェイス、コマンドラインインターフェイス、API などがあります。 - -* [クライアントライブラリ](../../interfaces/third-party/client-libraries.md) -* [インテグレーション](../../interfaces/third-party/integrations.md) -* [GUI](../../interfaces/third-party/gui.md) -* [プロキシ](../../interfaces/third-party/proxy.md) - -:::note -[ODBC](../../interfaces/odbc.md) や [JDBC](../../interfaces/jdbc.md) などの一般的な API をサポートする汎用ツールは、多くの場合 ClickHouse でも利用できますが、その数が非常に多いため、ここには掲載していません。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md deleted file mode 100644 index b087d046f97..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md +++ /dev/null @@ -1,192 +0,0 @@ ---- -description: 'ClickHouse をさまざまなサードパーティ製システムおよびツールと統合するためのドキュメント' -sidebar_label: '連携' -sidebar_position: 27 -slug: /interfaces/third-party/integrations -title: 'サードパーティ開発者による統合ライブラリ' -doc_type: 'reference' ---- - -# サードパーティ開発者による連携ライブラリ {#integration-libraries-from-third-party-developers} - -:::warning 免責事項 -ClickHouse, Inc. は、以下に掲載しているツールおよびライブラリを**保守しておらず**、その品質について十分なテストも行っていません。 -公式な連携については [Integrations ページ](/integrations) を参照してください。 -::: - -## インフラストラクチャ製品 {#infrastructure-products} - -
-リレーショナルデータベース管理システム - -- [MySQL](https://www.mysql.com) - - [mysql2ch](https://github.com/long2ice/mysql2ch) - - [ProxySQL](https://github.com/sysown/proxysql/wiki/ClickHouse-Support) - - [clickhouse-mysql-data-reader](https://github.com/Altinity/clickhouse-mysql-data-reader) - - [horgh-replicator](https://github.com/larsnovikov/horgh-replicator) -- [PostgreSQL](https://www.postgresql.org) - - [clickhousedb_fdw](https://github.com/Percona-Lab/clickhousedb_fdw) - - [infi.clickhouse_fdw](https://github.com/Infinidat/infi.clickhouse_fdw) ([infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm) を使用) - - [pg2ch](https://github.com/mkabilov/pg2ch) - - [clickhouse_fdw](https://github.com/adjust/clickhouse_fdw) -- [MSSQL](https://en.wikipedia.org/wiki/Microsoft_SQL_Server) - - [ClickHouseMigrator](https://github.com/zlzforever/ClickHouseMigrator) -
- -
-メッセージキュー - -- [Kafka](https://kafka.apache.org) - - [clickhouse_sinker](https://github.com/housepower/clickhouse_sinker) ([Go client](https://github.com/ClickHouse/clickhouse-go/) を使用) - - [stream-loader-clickhouse](https://github.com/adform/stream-loader) -
- -
-バッチ処理 - -- [Spark](https://spark.apache.org) - - [spark-clickhouse-connector](https://github.com/housepower/spark-clickhouse-connector) -
- -
-ストリーム処理 - -- [Flink](https://flink.apache.org) - - [flink-clickhouse-sink](https://github.com/ivi-ru/flink-clickhouse-sink) -
- -
-オブジェクトストレージ - -- [S3](https://en.wikipedia.org/wiki/Amazon_S3) - - [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) -
- -
-コンテナオーケストレーション - -- [Kubernetes](https://kubernetes.io) - - [clickhouse-operator](https://github.com/Altinity/clickhouse-operator) -
- -
-構成管理 -- [puppet](https://puppet.com) - - [innogames/clickhouse](https://forge.puppet.com/innogames/clickhouse) - - [mfedotov/clickhouse](https://forge.puppet.com/mfedotov/clickhouse) -
- -
-監視 - -- [Graphite](https://graphiteapp.org) - - [graphouse](https://github.com/ClickHouse/graphouse) - - [carbon-clickhouse](https://github.com/lomik/carbon-clickhouse) - - [graphite-clickhouse](https://github.com/lomik/graphite-clickhouse) - - [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) - [rollup configuration](../../engines/table-engines/mergetree-family/graphitemergetree.md#rollup-configuration) のルールを適用できる場合に、[\*GraphiteMergeTree](/engines/table-engines/mergetree-family/graphitemergetree) 内の古くなったパーティションを最適化します -- [Grafana](https://grafana.com/) - - [clickhouse-grafana](https://github.com/Altinity/clickhouse-grafana) -- [Prometheus](https://prometheus.io/) - - [clickhouse_exporter](https://github.com/f1yegor/clickhouse_exporter) - - [PromHouse](https://github.com/Percona-Lab/PromHouse) - - [clickhouse_exporter](https://github.com/hot-wifi/clickhouse_exporter) ([Go client](https://github.com/kshvakov/clickhouse/) を使用) -- [Nagios](https://www.nagios.org/) - - [check_clickhouse](https://github.com/exogroup/check_clickhouse/) - - [check_clickhouse.py](https://github.com/innogames/igmonplugins/blob/master/src/check_clickhouse.py) -- [Zabbix](https://www.zabbix.com) - - [clickhouse-zabbix-template](https://github.com/Altinity/clickhouse-zabbix-template) -- [Sematext](https://sematext.com/) - - [clickhouse integration](https://github.com/sematext/sematext-agent-integrations/tree/master/clickhouse) -
- -
-ログ - -- [rsyslog](https://www.rsyslog.com/) - - [omclickhouse](https://www.rsyslog.com/doc/master/configuration/modules/omclickhouse.html) -- [fluentd](https://www.fluentd.org) - - [loghouse](https://github.com/flant/loghouse) ([Kubernetes](https://kubernetes.io) 向け) -- [logagent](https://www.sematext.com/logagent) - - [logagent output-plugin-clickhouse](https://sematext.com/docs/logagent/output-plugin-clickhouse/) -
- -
-ジオロケーション - -- [MaxMind](https://dev.maxmind.com/geoip/) - - [clickhouse-maxmind-geoip](https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip) -
- -
-AutoML - -- [MindsDB](https://mindsdb.com/) - - [MindsDB](https://github.com/mindsdb/mindsdb) - ClickHouse と統合し、ClickHouse のデータを多様な AI/ML モデルから利用可能にします。 -
- -## プログラミング言語エコシステム {#programming-language-ecosystems} - -
-Python - -- [SQLAlchemy](https://www.sqlalchemy.org) - - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse)(内部で [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm) を使用) -- [PyArrow/Pandas](https://pandas.pydata.org) - - [Ibis](https://github.com/ibis-project/ibis) -
- -
-PHP - -- [Doctrine](https://www.doctrine-project.org/) - - [dbal-clickhouse](https://packagist.org/packages/friendsofdoctrine/dbal-clickhouse) -
- -
-R - -- [dplyr](https://db.rstudio.com/dplyr/) - - [RClickHouse](https://github.com/IMSMWU/RClickHouse)(内部で [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp) を使用) -
- -
-Java - -- [Hadoop](http://hadoop.apache.org) - - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader)(内部で [JDBC](../../sql-reference/table-functions/jdbc.md) を使用) -
- -
-Scala - -- [Akka](https://akka.io) - - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) -
- -
-C# - -- [ADO.NET](https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-overview) - - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) - - [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) - - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations) - - [Linq To DB](https://github.com/linq2db/linq2db) -
- -
-Elixir - -- [Ecto](https://github.com/elixir-ecto/ecto) - - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto) -
- -
-Ruby - -- [Ruby on Rails](https://rubyonrails.org/) - - [activecube](https://github.com/bitquery/activecube) - - [ActiveRecord](https://github.com/PNixx/clickhouse-activerecord) -- [GraphQL](https://github.com/graphql) - - [activecube-graphql](https://github.com/bitquery/activecube-graphql) -
\ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md b/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md deleted file mode 100644 index 9196ea27615..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: 'ClickHouse 向けに利用可能なサードパーティ製プロキシソリューションについて説明します' -sidebar_label: 'プロキシ' -sidebar_position: 29 -slug: /interfaces/third-party/proxy -title: 'サードパーティ製プロキシサーバー' -doc_type: 'reference' ---- - - - -# サードパーティ開発のプロキシサーバー {#proxy-servers-from-third-party-developers} - - - -## chproxy {#chproxy} - -[chproxy](https://github.com/Vertamedia/chproxy) は、ClickHouse データベース向けの HTTP プロキシ兼ロードバランサーです。 - -機能: - -- ユーザー単位のルーティングおよびレスポンスのキャッシュ。 -- 柔軟な制限設定。 -- SSL 証明書の自動更新。 - -Go で実装されています。 - - - -## KittenHouse {#kittenhouse} - -[KittenHouse](https://github.com/VKCOM/kittenhouse) は、アプリケーション側で INSERT データをバッファリングすることができない、あるいは不便な場合に、ClickHouse とアプリケーションサーバーとの間に位置するローカルプロキシとして設計されています。 - -機能: - -- メモリ内およびディスク上でのデータバッファリング -- テーブル単位のルーティング -- 負荷分散およびヘルスチェック - -Go で実装されています。 - - - -## ClickHouse-Bulk {#clickhouse-bulk} - -[ClickHouse-Bulk](https://github.com/nikepan/clickhouse-bulk) は、ClickHouse への INSERT をまとめて処理するシンプルなコレクターです。 - -機能: - -- リクエストをグループ化し、しきい値または一定間隔で送信。 -- 複数のリモートサーバーに対応。 -- ベーシック認証に対応。 - -Go 言語で実装されています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md deleted file mode 100644 index 42e4c7c578c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md +++ /dev/null @@ -1,228 +0,0 @@ - -[//]: # (This file is included in FAQ > Troubleshooting) - -- [インストール](#troubleshooting-installation-errors) -- [サーバーへの接続](#troubleshooting-accepts-no-connections) -- [クエリ処理](#troubleshooting-does-not-process-queries) -- [クエリ処理のパフォーマンス](#troubleshooting-too-slow) - - - -## インストール {#troubleshooting-installation-errors} - -### apt-get を使用して ClickHouse リポジトリから deb パッケージを取得できない {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get} - -* ファイアウォールの設定を確認してください。 -* 何らかの理由でリポジトリにアクセスできない場合は、[インストールガイド](../getting-started/install.md)の記事で説明されているようにパッケージをダウンロードし、`sudo dpkg -i ` コマンドを使用して手動でインストールしてください。`tzdata` パッケージも必要です。 - -### apt-get を使用して ClickHouse リポジトリから deb パッケージを更新できない {#you-cannot-update-deb-packages-from-clickhouse-repository-with-apt-get} - -* GPG キーが変更された場合にこの問題が発生することがあります。 - -リポジトリ設定を更新するには、[セットアップ](../getting-started/install.md#setup-the-debian-repository) ページの手順に従ってください。 - -### `apt-get update` でさまざまな警告が表示される {#you-get-different-warnings-with-apt-get-update} - -* 実際の警告メッセージは次のいずれかになります: - -```bash -N: リポジトリ 'https://packages.clickhouse.com/deb stable InRelease' はアーキテクチャ 'i386' に対応していないため、設定ファイル 'main/binary-i386/Packages' の取得をスキップします -``` - -```bash -E: https://packages.clickhouse.com/deb/dists/stable/main/binary-amd64/Packages.gz の取得に失敗しました ファイルサイズが想定と異なります (30451 != 28154)。ミラー同期中ですか? -``` - -```text -E: リポジトリ 'https://packages.clickhouse.com/deb stable InRelease' の 'Origin' 値が 'Artifactory' から 'ClickHouse' に変更されました -E: リポジトリ 'https://packages.clickhouse.com/deb stable InRelease' の 'Label' 値が 'Artifactory' から 'ClickHouse' に変更されました -N: リポジトリ 'https://packages.clickhouse.com/deb stable InRelease' の 'Suite' 値が 'stable' から '' に変更されました -N: このリポジトリの更新を適用するには、明示的に承認する必要があります。詳細は apt-secure(8) マニュアルページを参照してください。 -``` - -```bash -Err:11 https://packages.clickhouse.com/deb stable InRelease - 400 Bad Request [IP: 172.66.40.249 443] -``` - -上記の問題を解決するには、以下のスクリプトを使用してください。 - -```bash -sudo rm /var/lib/apt/lists/packages.clickhouse.com_* /var/lib/dpkg/arch /var/lib/apt/lists/partial/packages.clickhouse.com_* -sudo apt-get clean -sudo apt-get autoclean -``` - -### 誤った署名が原因で yum からパッケージを取得できない {#you-cant-get-packages-with-yum-because-of-wrong-signature} - -起こりうる原因: キャッシュが不正です。2022 年 9 月に GPG キーを更新した後に壊れた可能性があります。 - -解決策は、yum のキャッシュと lib ディレクトリをクリアすることです。 - -```bash -sudo find /var/lib/yum/repos/ /var/cache/yum/ -name 'clickhouse-*' -type d -exec rm -rf {} + -sudo rm -f /etc/yum.repos.d/clickhouse.repo -``` - -その後は、[インストールガイド](../getting-started/install.md#from-rpm-packages) に従ってください。 - -### Docker コンテナを実行できない {#you-cant-run-docker-container} - -単純に `docker run clickhouse/clickhouse-server` を実行すると、以下のようなスタックトレースが出力されてクラッシュします。 - -```bash -$ docker run -it clickhouse/clickhouse-server -........ -Poco::Exception. Code: 1000, e.code() = 0, System exception: cannot start thread, Stack trace (このメッセージをコピーする際は、以下の行を必ず含めてください): -``` - - -0. Poco::ThreadImpl::startImpl(Poco::SharedPtr>) @ 0x00000000157c7b34 -1. Poco::Thread::start(Poco::Runnable&) @ 0x00000000157c8a0e -2. BaseDaemon::initializeTerminationAndSignalProcessing() @ 0x000000000d267a14 -3. BaseDaemon::initialize(Poco::Util::Application&) @ 0x000000000d2652cb -4. DB::Server::initialize(Poco::Util::Application&) @ 0x000000000d128b38 -5. Poco::Util::Application::run() @ 0x000000001581cfda -6. DB::Server::run() @ 0x000000000d1288f0 -7. Poco::Util::ServerApplication::run(int, char\*\*) @ 0x0000000015825e27 -8. mainEntryClickHouseServer(int, char\*\*) @ 0x000000000d125b38 -9. main @ 0x0000000007ea4eee -10. ? @ 0x00007f67ff946d90 -11. ? @ 0x00007f67ff946e40 -12. \_start @ 0x00000000062e802e - (version 24.10.1.2812 (official build)) - -``` - -原因は、`20.10.10`より古いバージョンのDockerデーモンです。修正するには、アップグレードするか、`docker run [--privileged | --security-opt seccomp=unconfined]`を実行してください。後者にはセキュリティ上の影響があります。 - -``` - - -## サーバーへの接続 {#troubleshooting-accepts-no-connections} - -想定される問題: - -* サーバーが起動していない -* 想定外または誤った設定パラメータ - -### サーバーが起動していない {#server-is-not-running} - -**サーバーが起動しているか確認する** - -コマンド: - -```bash -$ sudo service clickhouse-server status -``` - -サーバーが起動していない場合は、次のコマンドで起動してください。 - -```bash -$ sudo service clickhouse-server start -``` - -**ログを確認する** - -`clickhouse-server` のメインログは、デフォルトで `/var/log/clickhouse-server/clickhouse-server.log` に出力されます。 - -サーバーが正常に起動していれば、次のような行が表示されます。 - -* ` Application: starting up.` — サーバーが起動しました。 -* ` Application: Ready for connections.` — サーバーが稼働中で、接続を受け付ける準備ができています。 - -`clickhouse-server` の起動が設定エラーで失敗した場合は、エラー内容の説明とともに `` という行が表示されます。例えば、次のようになります。 - -```text -2019.01.11 15:23:25.549505 [ 45 ] {} ExternalDictionaries: 外部ディクショナリ 'event2id' の再読み込みに失敗しました: Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused, e.what() = Connection refused -``` - -ファイルの末尾にエラーが表示されていない場合は、次の文字列以降を起点にファイル全体を確認してください。 - -```text - Application: 起動中。 -``` - -サーバー上で 2 つ目の `clickhouse-server` インスタンスを起動しようとすると、次のようなログが表示されます。 - -```text -2019.01.11 15:25:11.151730 [ 1 ] {} : Starting ClickHouse 19.1.0 with revision 54413 -2019.01.11 15:25:11.154578 [ 1 ] {} Application: starting up -2019.01.11 15:25:11.156361 [ 1 ] {} StatusFile: Status file ./status already exists - unclean restart. Contents: -PID: 8510 -Started at: 2019-01-11 15:24:23 -Revision: 54413 - -2019.01.11 15:25:11.156673 [ 1 ] {} Application: DB::Exception: Cannot lock file ./status. Another server instance in same directory is already running. -2019.01.11 15:25:11.156682 [ 1 ] {} Application: shutting down -2019.01.11 15:25:11.156686 [ 1 ] {} Application: Uninitializing subsystem: Logging Subsystem -2019.01.11 15:25:11.156716 [ 2 ] {} BaseDaemon: Stop SignalListener thread -``` - -**systemd のログを確認する** - -`clickhouse-server` のログに有用な情報が見つからない場合や、ログ自体が存在しない場合は、次のコマンドを実行して `systemd` のログを確認できます。 - -```bash -$ sudo journalctl -u clickhouse-server -``` - -**対話モードで clickhouse-server を起動する** - -```bash -$ sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml -``` - -このコマンドは、autostart スクリプトの標準パラメータを使用してサーバーを対話型アプリケーションとして起動します。このモードでは、`clickhouse-server` はすべてのイベントメッセージをコンソールに出力します。 - -### 設定パラメータ {#configuration-parameters} - -次の点を確認します。 - -* Docker 設定 - - IPv6 ネットワーク上の Docker で ClickHouse を実行している場合は、`network=host` が設定されていることを確認します。 - -* エンドポイント設定 - - [listen_host](../operations/server-configuration-parameters/settings.md#listen_host) と [tcp_port](../operations/server-configuration-parameters/settings.md#tcp_port) の設定を確認します。 - - ClickHouse サーバーは、デフォルトでは localhost からの接続のみを受け付けます。 - -* HTTP プロトコル設定 - - HTTP API のプロトコル設定を確認します。 - -* セキュア接続の設定 - - 次を確認します。 - - * [tcp_port_secure](../operations/server-configuration-parameters/settings.md#tcp_port_secure) の設定 - * [SSL certificates](../operations/server-configuration-parameters/settings.md#openssl) の設定 - - 接続時には適切なパラメータを使用します。たとえば、`clickhouse_client` では `port_secure` パラメータを使用します。 - -* ユーザー設定 - - 誤ったユーザー名またはパスワードを使用している可能性があります。 - - -## クエリ処理 {#troubleshooting-does-not-process-queries} - -ClickHouse がクエリを処理できない場合、エラー内容の説明をクライアントに送信します。`clickhouse-client` を使用している場合は、コンソール上でエラー内容を確認できます。HTTP インターフェイスを使用している場合は、ClickHouse はレスポンスボディ内にエラーの説明を送信します。例えば次のとおりです。 - -```bash -$ curl 'http://localhost:8123/' --data-binary "SELECT a" -Code: 47, e.displayText() = DB::Exception: Unknown identifier: a. Note that there are no tables (FROM clause) in your query, context: required_names: 'a' source_tables: table_aliases: private_aliases: column_aliases: public_columns: 'a' masked_columns: array_join_columns: source_columns: , e.what() = DB::Exception -``` - -`clickhouse-client` を `stack-trace` パラメータを指定して起動すると、ClickHouse はエラーの説明とともにサーバーのスタックトレースを返します。 - -接続が切断されたことを示すメッセージが表示される場合があります。この場合は、クエリを再実行してみてください。クエリを実行するたびに接続が切断される場合は、サーバーログにエラーが出力されていないか確認してください。 - - -## クエリ処理の効率 {#troubleshooting-too-slow} - -ClickHouse の動作が遅すぎると感じる場合は、クエリに対するサーバーリソースやネットワークへの負荷をプロファイリングする必要があります。 - -`clickhouse-benchmark` ユーティリティを使用してクエリをプロファイリングできます。1 秒あたりのクエリ処理数、1 秒あたりの行処理数、およびクエリ処理時間のパーセンタイルを表示します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md deleted file mode 100644 index 12930aa7971..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md +++ /dev/null @@ -1,224 +0,0 @@ ---- -description: 'ClickHouse におけるアロケーションプロファイリングの詳細ページ' -sidebar_label: 'バージョン 25.9 以前のアロケーションプロファイリング' -slug: /operations/allocation-profiling-old -title: 'バージョン 25.9 以前のアロケーションプロファイリング' -doc_type: 'reference' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# 25.9 以前のバージョン向けのアロケーションプロファイリング {#allocation-profiling-for-versions-before-259} - -ClickHouse はグローバルアロケータとして [jemalloc](https://github.com/jemalloc/jemalloc) を使用します。jemalloc には、アロケーションのサンプリングとプロファイリングのためのツールが付属しています。 -アロケーションプロファイリングをより便利に行えるように、Keeper では `SYSTEM` コマンドに加えて four letter word (4LW) コマンドも提供されています。 - - - -## アロケーションのサンプリングとヒーププロファイルのフラッシュ {#sampling-allocations-and-flushing-heap-profiles} - -`jemalloc` でアロケーションのサンプリングとプロファイリングを行う場合は、環境変数 `MALLOC_CONF` を使用してプロファイリングを有効にし、ClickHouse/Keeper を起動する必要があります。 - -```sh -MALLOC_CONF=background_thread:true,prof:true -``` - -`jemalloc` はアロケーションをサンプリングし、その情報を内部に保持します。 - -現在のプロファイルをフラッシュするように `jemalloc` に指示するには、次を実行します: - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -デフォルトでは、ヒーププロファイル用のファイルは `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap` に生成されます。ここで `_pid_` は ClickHouse の PID、`_seqnum_` は現在のヒーププロファイルに対するグローバルシーケンス番号です。\ -Keeper の場合、デフォルトファイルは `/tmp/jemalloc_keeper._pid_._seqnum_.heap` であり、同じルールに従います。 - -`MALLOC_CONF` 環境変数に `prof_prefix` オプションを追加することで、別の場所を指定できます。\ -たとえば、`/data` ディレクトリ内に、ファイル名のプレフィックスを `my_current_profile` としてプロファイルを生成したい場合は、ClickHouse/Keeper を次の環境変数を指定して実行します: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_prefix:/data/my_current_profile -``` - -生成されるファイル名は、接頭辞、PID、シーケンス番号を連結したものになります。 - - -## ヒーププロファイルの解析 {#analyzing-heap-profiles} - -ヒーププロファイルを生成したら、それを解析する必要があります。\ -そのために、`jemalloc` のツールである [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in) を使用できます。これは複数の方法でインストールできます: - -* システムのパッケージマネージャーを使用する -* [jemalloc リポジトリ](https://github.com/jemalloc/jemalloc) をクローンし、ルートディレクトリで `autogen.sh` を実行する。この方法では、`bin` ディレクトリ内で `jeprof` スクリプトが利用できるようになります - -:::note -`jeprof` はスタックトレースを生成するために `addr2line` を使用しますが、これは非常に遅くなる場合があります。\ -その場合は、このツールの[代替実装](https://github.com/gimli-rs/addr2line)をインストールすることを推奨します。 - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -::: - -`jeprof` を使用してヒーププロファイルから生成できる形式には、さまざまなものがあります。 -ツールの使い方および利用可能な各種オプションについては、`jeprof --help` を実行して確認することをお勧めします。 - -一般的に、`jeprof` コマンドは次のように使用します。 - -```sh -jeprof path/to/binary path/to/heap/profile --output_format [ > output_file] -``` - -2つのプロファイル間でどの割り当てが行われたかを比較したい場合は、`base` 引数を指定できます。 - -```sh -jeprof path/to/binary --base path/to/first/heap/profile path/to/second/heap/profile --output_format [ > output_file] -``` - -### 例 {#examples} - -* 各プロシージャを1行ごとに記述したテキストファイルを生成したい場合: - -```sh -jeprof path/to/binary path/to/heap/profile --text > result.txt -``` - -* コールグラフを含む PDF ファイルを生成したい場合: - -```sh -jeprof path/to/binary path/to/heap/profile --pdf > result.pdf -``` - -### フレームグラフの生成 {#generating-flame-graph} - -`jeprof` を使用すると、フレームグラフの作成に必要な折りたたみスタック(collapsed stack)を生成できます。 - -`--collapsed` 引数を指定する必要があります。 - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -その後、折り畳まれたスタックを可視化するために、さまざまなツールを利用できます。 - -最も一般的なのは [FlameGraph](https://github.com/brendangregg/FlameGraph) で、`flamegraph.pl` というスクリプトが付属しています。 - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg -``` - -もう 1 つ有用なツールとして [speedscope](https://www.speedscope.app/) があり、収集したスタック情報をよりインタラクティブに解析できます。 - - -## 実行時のアロケーションプロファイラの制御 {#controlling-allocation-profiler-during-runtime} - -ClickHouse/Keeper をプロファイラを有効にした状態で起動した場合、実行時にアロケーションプロファイリングを無効化/有効化するための追加コマンドを使用できます。 -これらのコマンドを使用すると、特定の時間区間のみをプロファイルしやすくなります。 - -プロファイラを無効にするには: - - - - ```sql - SYSTEM JEMALLOC DISABLE PROFILE - ``` - - - - ```sh - echo jmdp | nc localhost 9181 - ``` - - - -プロファイラを有効にするには: - - - - ```sql - SYSTEM JEMALLOC ENABLE PROFILE - ``` - - - - ```sh - echo jmep | nc localhost 9181 - ``` - - - -`prof_active` オプションを設定することで、プロファイラの初期状態を制御することも可能です。このオプションはデフォルトで有効になっています。\ -たとえば、起動時にはアロケーションをサンプリングせず、起動後のみサンプリングしたい場合は、その時点でプロファイラを有効にします。次の環境変数を指定して ClickHouse/Keeper を起動できます: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_active:false -``` - -プロファイラは後から有効化することもできます。 - - -## プロファイラの追加オプション {#additional-options-for-profiler} - -`jemalloc` にはプロファイラに関連する多数のオプションが用意されており、`MALLOC_CONF` 環境変数を変更して制御できます。 -たとえば、アロケーションサンプル間の間隔は `lg_prof_sample` で制御できます。 -ヒーププロファイルを N バイトごとにダンプしたい場合は、`lg_prof_interval` を有効化してください。 - -利用可能なオプションの一覧については、`jemalloc` の[リファレンスページ](https://jemalloc.net/jemalloc.3.html)を参照してください。 - - - -## その他のリソース {#other-resources} - -ClickHouse/Keeper は、`jemalloc` 関連のメトリクスをさまざまな方法で公開しています。 - -:::warning 注意 -これらのメトリクスは互いに同期されておらず、値がずれていく可能性があることを認識しておくことが重要です。 -::: - -### システムテーブル `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[リファレンス](/operations/system-tables/asynchronous_metrics) - -### システムテーブル `jemalloc_bins` {#system-table-jemalloc_bins} - -サイズクラス(bin)ごとに `jemalloc` アロケータ経由で行われたメモリ割り当てに関する情報を、すべてのアリーナから集約して格納します。 - -[リファレンス](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -`asynchronous_metrics` に含まれるすべての `jemalloc` 関連メトリクスは、ClickHouse と Keeper の両方で Prometheus エンドポイントを通じても公開されます。 - -[リファレンス](/operations/server-configuration-parameters/settings#prometheus) - -### Keeper における `jmst` 4LW コマンド {#jmst-4lw-command-in-keeper} - -Keeper は `jmst` 4LW コマンドをサポートしており、[基本的なアロケータ統計情報](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics) を返します。 - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md deleted file mode 100644 index b035ba7cb92..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md +++ /dev/null @@ -1,337 +0,0 @@ ---- -description: 'ClickHouse におけるアロケーションプロファイリングを詳しく説明するページ' -sidebar_label: 'アロケーションプロファイリング' -slug: /operations/allocation-profiling -title: 'アロケーションプロファイリング' -doc_type: 'guide' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# アロケーションプロファイリング {#allocation-profiling} - -ClickHouse はグローバルアロケータとして [jemalloc](https://github.com/jemalloc/jemalloc) を使用しています。jemalloc には、アロケーションのサンプリングおよびプロファイリング用のツールが付属しています。 -アロケーションプロファイリングをより手軽に行えるように、ClickHouse と Keeper では、設定ファイルやクエリ設定、`SYSTEM` コマンド、Keeper の four letter word (4LW) コマンドを使用してサンプリングを制御できます。 -さらに、サンプルは `JemallocSample` 型として `system.trace_log` テーブルに収集できます。 - -:::note - -このガイドはバージョン 25.9 以降に適用されます。 -それ以前のバージョンについては、[25.9 より前のバージョン向けアロケーションプロファイリング](/operations/allocation-profiling-old.md) を参照してください。 - -::: - - - -## アロケーションのサンプリング {#sampling-allocations} - -`jemalloc` でアロケーションのサンプリングおよびプロファイリングを行うには、`jemalloc_enable_global_profiler` 設定を有効にして ClickHouse/Keeper を起動する必要があります。 - -```xml - - 1 - -``` - -`jemalloc` はアロケーションをサンプリングし、その情報を内部に保持します。 - -`jemalloc_enable_profiler` 設定を使用することで、クエリ単位のアロケーションを有効にすることもできます。 - -:::warning 警告 -ClickHouse はアロケーションが多いアプリケーションであるため、jemalloc のサンプリングによりパフォーマンス上のオーバーヘッドが発生する可能性があります。 -::: - - -## `system.trace_log` に jemalloc サンプルを保存する {#storing-jemalloc-samples-in-system-trace-log} - -すべての jemalloc サンプルを `JemallocSample` 型として `system.trace_log` に格納できます。 -これをグローバルに有効化するには、設定項目 `jemalloc_collect_global_profile_samples_in_trace_log` を使用します。 - -```xml - - 1 - -``` - -:::warning 警告 -ClickHouse はメモリ割り当てを多用するアプリケーションであるため、`system.trace_log` ですべてのサンプルを収集すると高負荷になる可能性があります。 -::: - -`jemalloc_collect_profile_samples_in_trace_log` 設定を使用して、クエリごとに有効化することもできます。 - -### `system.trace_log` を使用してクエリのメモリ使用量を分析する例 {#example-analyzing-memory-usage-trace-log} - -まず、jemalloc プロファイラを有効にしてクエリを実行し、そのクエリのサンプルを `system.trace_log` に収集する必要があります。 - -```sql -SELECT * -FROM numbers(1000000) -ORDER BY number DESC -SETTINGS max_bytes_ratio_before_external_sort = 0 -FORMAT `Null` -SETTINGS jemalloc_enable_profiler = 1, jemalloc_collect_profile_samples_in_trace_log = 1 - -Query id: 8678d8fe-62c5-48b8-b0cd-26851c62dd75 - -Ok. - -0 rows in set. Elapsed: 0.009 sec. Processed 1.00 million rows, 8.00 MB (108.58 million rows/s., 868.61 MB/s.) -Peak memory usage: 12.65 MiB. -``` - -:::note -ClickHouse を起動するときに `jemalloc_enable_global_profiler` を有効にしている場合、`jemalloc_enable_profiler` を有効にする必要はありません。\ -`jemalloc_collect_global_profile_samples_in_trace_log` と `jemalloc_collect_profile_samples_in_trace_log` についても同様です。 -::: - -ここで `system.trace_log` をフラッシュします。 - -```sql -SYSTEM FLUSH LOGS trace_log -``` - -そして、各時点において実行したクエリのメモリ使用量を取得するためにクエリします。 - -```sql -WITH per_bucket AS -( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time -) -SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable -FROM per_bucket -ORDER BY bucket_time -``` - -メモリ使用量が最大だった時刻も確認できます: - -```sql -SELECT - argMax(bucket_time, cumulative_size), - max(cumulative_size) -FROM -( - WITH per_bucket AS - ( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time - ) - SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable - FROM per_bucket - ORDER BY bucket_time -) -``` - -その結果を使って、その時点でどこが最も活発に割り当てを行っていたかを確認できます。 - - -```sql -SELECT - concat( - '\n', - arrayStringConcat( - arrayMap( - (x, y) -> concat(x, ': ', y), - arrayMap(x -> addressToLine(x), allocation_trace), - arrayMap(x -> demangle(addressToSymbol(x)), allocation_trace) - ), - '\n' - ) - ) AS symbolized_trace, - sum(s) AS per_trace_sum -FROM -( - SELECT - ptr, - sum(size) AS s, - argMax(trace, event_time_microseconds) AS allocation_trace - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - AND event_time_microseconds <= '2025-09-04 11:56:21.737139' - GROUP BY ptr - HAVING s > 0 -) -GROUP BY ALL -ORDER BY per_trace_sum ASC -``` - - -## ヒーププロファイルのフラッシュ {#flushing-heap-profiles} - -デフォルトでは、ヒーププロファイル用ファイルは `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap` に生成されます。ここで `_pid_` は ClickHouse の PID、`_seqnum_` は現在のヒーププロファイルに対応するグローバルなシーケンス番号です。\ -Keeper のデフォルトファイルは `/tmp/jemalloc_keeper._pid_._seqnum_.heap` で、同じルールに従います。 - -`jemalloc` に現在のプロファイルをフラッシュさせるには、次を実行します。 - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - フラッシュされたプロファイルの保存先パスが返されます。 - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -`MALLOC_CONF` 環境変数に `prof_prefix` オプションを追加することで、別の保存場所を指定できます。\ -例えば、`/data` ディレクトリ内に、ファイル名のプレフィックスを `my_current_profile` としてプロファイルを生成したい場合は、次の環境変数を指定して ClickHouse/Keeper を実行します。 - -```sh -MALLOC_CONF=prof_prefix:/data/my_current_profile -``` - -生成されるファイル名には、プレフィックスに続いて PID とシーケンス番号が付加されます。 - - -## ヒーププロファイルの分析 {#analyzing-heap-profiles} - -ヒーププロファイルが生成されたら、それらを分析する必要があります。\ -そのために、`jemalloc` のツールである [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in) を使用できます。次のいずれかの方法でインストールできます。 - -* システムのパッケージマネージャーを使用する -* [jemalloc リポジトリ](https://github.com/jemalloc/jemalloc)をクローンし、ルートディレクトリで `autogen.sh` を実行する。これにより、`bin` ディレクトリ内に `jeprof` スクリプトが作成されます - -:::note -`jeprof` はスタックトレースを生成するために `addr2line` を使用しますが、処理が非常に遅くなる場合があります。\ -そのような場合には、このツールの[代替実装](https://github.com/gimli-rs/addr2line)をインストールすることを推奨します。 - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -また、`llvm-addr2line` も同様に問題なく動作します。 - -::: - -`jeprof` を使用して、ヒーププロファイルからさまざまな形式を生成できます。 -ツールの使い方や提供される各種オプションについては、`jeprof --help` を実行して確認することを推奨します。 - -一般的に、`jeprof` コマンドは次のように使用します。 - -```sh -jeprof path/to/binary path/to/heap/profile --output_format [ > output_file] -``` - -2つのプロファイルを比較して、その間にどの割り当てが発生したかを確認したい場合は、`base` 引数を指定します。 - -```sh -jeprof path/to/binary --base path/to/first/heap/profile path/to/second/heap/profile --output_format [ > output_file] -``` - -### 例 {#examples} - -* 各プロシージャを1行ごとに記述したテキストファイルを生成したい場合: - -```sh -jeprof path/to/binary path/to/heap/profile --text > result.txt -``` - -* コールグラフを含む PDF ファイルを生成したい場合: - -```sh -jeprof path/to/binary path/to/heap/profile --pdf > result.pdf -``` - -### フレームグラフの生成 {#generating-flame-graph} - -`jeprof` を使用すると、フレームグラフの作成に必要な折り畳みスタック(collapsed stacks)を生成できます。 - -`--collapsed` 引数を使用する必要があります。 - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -その後は、コラプスされたスタックを可視化するために利用できるツールが多数あります。 - -最も広く使われているのは [FlameGraph](https://github.com/brendangregg/FlameGraph) で、`flamegraph.pl` というスクリプトが含まれています。 - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="メモリ割り当てフレームグラフ" --width 2400 > result.svg -``` - -もう 1 つ便利なツールに [speedscope](https://www.speedscope.app/) があり、収集したスタックをよりインタラクティブに分析できます。 - - -## プロファイラ用の追加オプション {#additional-options-for-profiler} - -`jemalloc` にはプロファイラに関連する多くのオプションがあり、`MALLOC_CONF` 環境変数を変更することで制御できます。 -例えば、メモリ割り当てサンプル間の間隔は `lg_prof_sample` で制御できます。 -ヒーププロファイルを N バイトごとにダンプしたい場合は、`lg_prof_interval` を有効にします。 - -利用可能なオプションの完全な一覧については、`jemalloc` の [リファレンスページ](https://jemalloc.net/jemalloc.3.html) を参照してください。 - - - -## その他のリソース {#other-resources} - -ClickHouse/Keeper は、`jemalloc` 関連のメトリクスをさまざまな方法で公開します。 - -:::warning 警告 -これらのメトリクスは相互に同期されておらず、値がずれる可能性があることを認識しておくことが重要です。 -::: - -### システムテーブル `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[リファレンス](/operations/system-tables/asynchronous_metrics) - -### システムテーブル `jemalloc_bins` {#system-table-jemalloc_bins} - -すべてのアリーナから集約された、さまざまなサイズクラス(bin)における jemalloc アロケータによるメモリ割り当てに関する情報を含みます。 - -[リファレンス](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -`asynchronous_metrics` に含まれるすべての `jemalloc` 関連メトリクスは、ClickHouse と Keeper の両方で Prometheus エンドポイントからも公開されます。 - -[リファレンス](/operations/server-configuration-parameters/settings#prometheus) - -### Keeper における `jmst` 4LW コマンド {#jmst-4lw-command-in-keeper} - -Keeper は `jmst` 4LW コマンドをサポートしており、[基本的なアロケータ統計情報](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics)を返します。 - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/analyzer.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/analyzer.md deleted file mode 100644 index 7e385323b25..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/analyzer.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -description: 'ClickHouse クエリ アナライザーの詳細ページ' -keywords: ['analyzer'] -sidebar_label: 'アナライザー' -slug: /operations/analyzer -title: 'アナライザー' -doc_type: 'reference' ---- - -# Analyzer {#analyzer} - -ClickHouse バージョン `24.3` では、新しいクエリアナライザーがデフォルトで有効になりました。 -その動作の詳細については[こちら](/guides/developer/understanding-query-execution-with-the-analyzer#analyzer)を参照してください。 - -## 既知の非互換性 {#known-incompatibilities} - -多数のバグ修正と新しい最適化を導入した一方で、ClickHouse の動作には後方互換性を破る変更もいくつか含まれています。新しいアナライザ用にクエリを書き換える方法を判断するために、以下の変更点を確認してください。 - -### 無効なクエリはこれ以上最適化されない {#invalid-queries-are-no-longer-optimized} - -以前のクエリプランニング基盤では、クエリの検証ステップより前に AST レベルの最適化が適用されていました。 -この最適化により、元のクエリを書き換えて有効かつ実行可能にできる場合がありました。 - -新しいアナライザでは、最適化ステップより前にクエリ検証が行われます。 -これは、以前は実行可能だった無効なクエリが、現在はサポートされなくなったことを意味します。 -そのような場合、そのクエリは手動で修正する必要があります。 - -#### 例 1 {#example-1} - -次のクエリは、集約後に利用可能なのが `toString(number)` だけであるにもかかわらず、SELECT 句でカラム `number` を使用しています。 -旧アナライザでは、`GROUP BY toString(number)` は `GROUP BY number` に最適化され、その結果クエリは有効になっていました。 - -```sql -SELECT number -FROM numbers(1) -GROUP BY toString(number) -``` - -#### 例 2 {#example-2} - -このクエリでも同じ問題が発生します。列 `number` は、別のキーと一緒に集約した後に使用されています。 -以前のクエリアナライザーは、`number > 5` というフィルタを `HAVING` 句から `WHERE` 句に移動することで、このクエリを修正しました。 - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -GROUP BY n -HAVING number > 5 -``` - -このクエリを修正するには、標準的な SQL 構文に従うよう、集約されていない列に適用されるすべての条件を `WHERE` 句に移動する必要があります。 - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -WHERE number > 5 -GROUP BY n -``` - -### 無効なクエリを指定した `CREATE VIEW` {#create-view-with-invalid-query} - -新しいアナライザは常に型チェックを行います。 -以前は、無効な `SELECT` クエリを含む `VIEW` を作成できていました。 -その場合、最初の `SELECT` または(`MATERIALIZED VIEW` の場合は)`INSERT` で失敗していました。 - -このような形で `VIEW` を作成することは、もはやできません。 - -#### 例 {#example-view} - -```sql -CREATE TABLE source (data String) -ENGINE=MergeTree -ORDER BY tuple(); - -CREATE VIEW some_view -AS SELECT JSONExtract(data, 'test', 'DateTime64(3)') -FROM source; -``` - -### `JOIN` 句の既知の非互換性 {#known-incompatibilities-of-the-join-clause} - -#### プロジェクションの列を使用する `JOIN` {#join-using-column-from-projection} - -`SELECT` リスト内のエイリアスは、デフォルトでは `JOIN USING` のキーとして使用できません。 - -新しい設定 `analyzer_compatibility_join_using_top_level_identifier` を有効にすると、`JOIN USING` の動作が変更され、左側テーブルの列を直接使用するのではなく、`SELECT` クエリのプロジェクションリストに含まれる式を基に識別子を優先的に解決するようになります。 - -例: - -```sql -SELECT a + 1 AS b, t2.s -FROM VALUES('a UInt64, b UInt64', (1, 1)) AS t1 -JOIN VALUES('b UInt64, s String', (1, 'one'), (2, 'two')) t2 -USING (b); -``` - -`analyzer_compatibility_join_using_top_level_identifier` を `true` に設定すると、結合条件は `t1.a + 1 = t2.b` と解釈され、以前のバージョンと同じ動作になります。 -結果は `2, 'two'` になります。 -設定が `false` の場合、結合条件はデフォルトで `t1.b = t2.b` となり、クエリは `2, 'one'` を返します。 -`b` が `t1` に存在しない場合、クエリはエラーとなって失敗します。 - -#### `JOIN USING` と `ALIAS` / `MATERIALIZED` カラムにおける動作の変更 {#changes-in-behavior-with-join-using-and-aliasmaterialized-columns} - -新しいアナライザでは、`ALIAS` または `MATERIALIZED` カラムを含む `JOIN USING` クエリで `*` を使用すると、それらのカラムもデフォルトで結果セットに含まれます。 - -例えば次のようになります。 - -```sql -CREATE TABLE t1 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t1 VALUES (1), (2); - -CREATE TABLE t2 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t2 VALUES (2), (3); - -SELECT * FROM t1 -FULL JOIN t2 USING (payload); -``` - -新しいアナライザーでは、このクエリの結果には、両方のテーブルからの `id` 列に加えて `payload` 列が含まれます。 -これに対して、以前のアナライザーでは、特定の設定(`asterisk_include_alias_columns` または `asterisk_include_materialized_columns`)が有効になっている場合にのみ、これらの `ALIAS` 列が結果に含まれ、 -かつ列の並び順が異なる場合がありました。 - -特に古いクエリを新しいアナライザーに移行する際に、一貫性があり期待どおりの結果を得るためには、`*` を使うのではなく、`SELECT` 句で列を明示的に指定することを推奨します。 - -#### `USING` 句内の列に対する型修飾子の扱い {#handling-of-type-modifiers-for-columns-in-using-clause} - -新しいバージョンのアナライザーでは、`USING` 句で指定された列に対して共通スーパータイプを決定するための規則が標準化されており、 -特に `LowCardinality` や `Nullable` といった型修飾子を扱う際に、より予測しやすい結果が得られるようになっています。 - -* `LowCardinality(T)` と `T`: 型 `LowCardinality(T)` の列が型 `T` の列と結合される場合、結果の共通スーパータイプは `T` となり、`LowCardinality` 修飾子は実質的に破棄されます。 -* `Nullable(T)` と `T`: 型 `Nullable(T)` の列が型 `T` の列と結合される場合、結果の共通スーパータイプは `Nullable(T)` となり、NULL 許容であるという性質が保持されます。 - -例: - -```sql -SELECT id, toTypeName(id) -FROM VALUES('id LowCardinality(String)', ('a')) AS t1 -FULL OUTER JOIN VALUES('id String', ('b')) AS t2 -USING (id); -``` - -このクエリでは、`id` の共通スーパータイプは `String` として判定され、`t1` では `LowCardinality` 修飾子が除外されます。 - -### Projection 列名の変更 {#projection-column-names-changes} - -Projection 列名を計算する際、エイリアスは展開されません。 - -```sql -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 0 -FORMAT PrettyCompact - - ┌─x─┬─plus(plus(1, 1), 1)─┐ -1. │ 2 │ 3 │ - └───┴─────────────────────┘ - -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 1 -FORMAT PrettyCompact - - ┌─x─┬─plus(x, 1)─┐ -1. │ 2 │ 3 │ - └───┴────────────┘ -``` - -### 互換性のない関数引数の型 {#incompatible-function-arguments-types} - -新しいアナライザーでは、型推論はクエリの初期解析中に行われます。 -この変更により、短絡評価の前に型チェックが行われるようになり、その結果、`if` 関数の引数は常に共通の上位型(スーパータイプ)を持つ必要があります。 - -例えば、次のクエリは `There is no supertype for types Array(UInt8), String because some of them are Array and some of them are not` というエラーで失敗します。 - -```sql -SELECT toTypeName(if(0, [2, 3, 4], 'String')) -``` - -### 異種クラスタ {#heterogeneous-clusters} - -新しい analyzer は、クラスタ内のサーバー間の通信プロトコルを大きく変更します。\ -そのため、`enable_analyzer` の設定値が異なるサーバー間で分散クエリを実行することはできません。 - -### ミューテーションは旧 analyzer によって解釈される {#mutations-are-interpreted-by-previous-analyzer} - -ミューテーションは依然として古い analyzer を使用します。\ -これは、一部の新しい ClickHouse SQL 機能(たとえば `QUALIFY` 句)がミューテーションでは使用できないことを意味します。\ -対応状況は[こちら](https://github.com/ClickHouse/ClickHouse/issues/61563)で確認できます。 - -### 未サポートの機能 {#unsupported-features} - -新しい analyzer が現在サポートしていない機能の一覧は以下のとおりです。 - -* Annoy インデックス。 -* Hypothesis インデックス。作業状況は[こちら](https://github.com/ClickHouse/ClickHouse/pull/48381)。 -* Window view はサポートされていません。今後もサポートする予定はありません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/backup.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/backup.md deleted file mode 100644 index 0c2265e73fc..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/backup.md +++ /dev/null @@ -1,574 +0,0 @@ ---- -description: 'ClickHouse データベースおよびテーブルのバックアップおよび復元に関するガイド' -sidebar_label: 'バックアップと復元' -sidebar_position: 10 -slug: /operations/backup -title: 'バックアップと復元' -doc_type: 'guide' ---- - - - -# バックアップと復元 {#backup-and-restore} - -- [ローカルディスクへのバックアップ](#backup-to-a-local-disk) -- [S3 エンドポイントを使用したバックアップ/復元の設定](#configuring-backuprestore-to-use-an-s3-endpoint) -- [S3 ディスクを使用したバックアップ/復元](#backuprestore-using-an-s3-disk) -- [代替案](#alternatives) - - - -## コマンドの概要 {#command-summary} - -```bash - BACKUP|RESTORE - TABLE [db.]table_name [AS [db.]table_name_in_backup] - [PARTITION[S] partition_expr [, ...]] | - DICTIONARY [db.]dictionary_name [AS [db.]name_in_backup] | - DATABASE database_name [AS database_name_in_backup] - [EXCEPT TABLES ...] | - TEMPORARY TABLE table_name [AS table_name_in_backup] | - VIEW view_name [AS view_name_in_backup] | - ALL [EXCEPT {TABLES|DATABASES}...] } [, ...] - [ON CLUSTER 'cluster_name'] - TO|FROM File('/') | Disk('', '/') | S3('/', '', '') - [SETTINGS base_backup = File('/') | Disk(...) | S3('/', '', '')] - [SYNC|ASYNC] - -``` - -:::note ALL -バージョン 23.4 より前の ClickHouse では、`ALL` は `RESTORE` コマンドにのみ適用されていました。 -::: - - -## 背景 {#background} - -[レプリケーション](../engines/table-engines/mergetree-family/replication.md) はハードウェア障害からの保護を提供しますが、人為的なミスからは保護しません。たとえば、データの誤削除、誤ったテーブルや誤ったクラスタ上のテーブルの削除、不正なデータ処理やデータ破損を引き起こすソフトウェアバグなどです。多くの場合、このようなミスはすべてのレプリカに影響します。ClickHouse には、特定の種類のミスを防ぐための組み込みの安全機構があります。たとえばデフォルトでは、[50 GB を超えるデータを含む MergeTree 系エンジンのテーブルを、そのまま DROP することはできません](/operations/settings/settings#max_table_size_to_drop)。しかし、これらの安全機構はあらゆるケースを網羅しているわけではなく、回避されてしまう可能性もあります。 - -起こり得る人為的なミスの影響を効果的に軽減するためには、**事前に** データのバックアップおよび復元戦略を慎重に準備しておく必要があります。 - -各社で利用可能なリソースやビジネス要件は異なるため、あらゆる状況に適合するような ClickHouse のバックアップおよび復元の汎用的な解決策は存在しません。1 GB のデータで有効な方法が、数十 PB のデータでも有効とは限りません。長所と短所を併せ持つさまざまなアプローチが存在し、それらについては後述します。個々の欠点を補うために、1 つのアプローチだけではなく、複数のアプローチを併用することが望ましいです。 - -:::note -バックアップを取得しただけで一度も復元を試していない場合、いざというときに復元が正しく動作しない(少なくとも、ビジネスが許容できるよりも長い時間がかかる)可能性が高いことを忘れないでください。どのようなバックアップ手法を選択するにせよ、復元プロセスも必ず自動化し、予備の ClickHouse クラスタで定期的に復元演習を行ってください。 -::: - - - -## ローカルディスクへのバックアップ {#backup-to-a-local-disk} - -### バックアップ先の設定 {#configure-a-backup-destination} - -以下の例では、バックアップ先は `Disk('backups', '1.zip')` のように指定されます。バックアップ先を準備するには、バックアップ先を指定したファイルを `/etc/clickhouse-server/config.d/backup_disk.xml` に追加します。例えば、このファイルでは `backups` という名前のディスクを定義し、そのディスクを **backups > allowed_disk** リストに追加します。 - -```xml - - - - - - local - /backups/ - - - - - - backups - /backups/ - - - -``` - -### パラメーター {#parameters} - -バックアップはフルバックアップまたは増分バックアップとし、テーブル(マテリアライズドビュー、プロジェクション、ディクショナリを含む)やデータベースを対象にできます。バックアップは同期(デフォルト)または非同期で実行できます。圧縮することも可能で、パスワード保護を設定できます。 - -`BACKUP` および `RESTORE` ステートメントは、`DATABASE` および `TABLE` 名のリスト、宛先(またはソース)、オプション、および設定を受け取ります。 - -* バックアップの宛先、またはリストアのソースです。これは前に定義したディスクに基づきます。例: `Disk('backups', 'filename.zip')` -* ASYNC: バックアップまたはリストアを非同期で実行 -* PARTITIONS: リストアするパーティションのリスト -* SETTINGS: - * `id`: バックアップまたはリストア処理の識別子。設定されていないか空の場合は、ランダムに生成された UUID が使用されます。 - 明示的に空でない文字列に設定する場合は、毎回異なる値にする必要があります。この `id` は、特定のバックアップまたはリストア処理に関連する `system.backups` テーブル内の行を検索するために使用されます。 - * [`compression_method`](/sql-reference/statements/create/table#column_compression_codec) と `compression_level` - * ディスク上のファイルに対する `password` - * `base_backup`: このソースの前回のバックアップの宛先。例: `Disk('backups', '1.zip')` - * `use_same_s3_credentials_for_base_backup`: S3 へのベースバックアップがクエリで使用されている認証情報を継承するかどうか。`S3` でのみ動作します。 - * `use_same_password_for_base_backup`: ベースバックアップアーカイブがクエリからパスワードを継承するかどうか。 - * `structure_only`: 有効にすると、テーブルデータなしで `CREATE` ステートメントのみをバックアップまたはリストアできます。 - * `storage_policy`: リストアされるテーブルに対するストレージポリシー。[複数のブロックデバイスを使用したデータストレージ](../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) を参照してください。この設定は `RESTORE` コマンドにのみ適用されます。指定されたストレージポリシーは、`MergeTree` ファミリーのエンジンを持つテーブルにのみ適用されます。 - * `s3_storage_class`: S3 バックアップに使用されるストレージクラス。例: `STANDARD` - * `azure_attempt_to_create_container`: Azure Blob Storage を使用する場合、指定されたコンテナーが存在しないときに作成を試みるかどうか。デフォルト: true。 - * [コア設定](/operations/settings/settings) もここで使用できます - -### 使用例 {#usage-examples} - -テーブルをバックアップしてからリストアする例: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.zip') -``` - -対応するリストア: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -``` - -:::note -上記の RESTORE は、テーブル `test.table` にデータが含まれている場合は失敗します。RESTORE をテストする場合は、テーブルを削除するか、設定 `allow_non_empty_tables=true` を有効にする必要があります。 - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -SETTINGS allow_non_empty_tables=true -``` - -::: - -テーブルは、新しい名前を指定してリストアしたりバックアップを作成したりできます。 - -```sql -RESTORE TABLE test.table AS test.table2 FROM Disk('backups', '1.zip') -``` - -```sql -BACKUP TABLE test.table3 AS test.table4 TO Disk('backups', '2.zip') -``` - -### 増分バックアップ {#incremental-backups} - -`base_backup` を指定すると、増分バックアップを作成できます。 -:::note -増分バックアップはベースバックアップに依存します。増分バックアップからリストアできるようにするには、ベースバックアップを利用可能な状態で保持しておく必要があります。 -::: - - -新しいデータを増分的に保存します。設定 `base_backup` により、以前のバックアップ `Disk('backups', 'd.zip')` 以降に追加されたデータが `Disk('backups', 'incremental-a.zip')` に保存されます。 - -```sql -BACKUP TABLE test.table TO Disk('backups', 'incremental-a.zip') - SETTINGS base_backup = Disk('backups', 'd.zip') -``` - -インクリメンタルバックアップと `base_backup` からすべてのデータを、新しいテーブル `test.table2` に復元します: - -```sql -RESTORE TABLE test.table AS test.table2 - FROM Disk('backups', 'incremental-a.zip'); -``` - -### バックアップにパスワードを設定する {#assign-a-password-to-the-backup} - -ディスクに書き込まれるバックアップファイルには、パスワードを設定できます。 - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -復元: - -```sql -RESTORE TABLE test.table - FROM Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -### 圧縮設定 {#compression-settings} - -圧縮方式や圧縮レベルを指定したい場合は、次のように設定します。 - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'filename.zip') - SETTINGS compression_method='lzma', compression_level=3 -``` - -### 特定のパーティションを復元する {#restore-specific-partitions} - -テーブルに関連付けられた特定のパーティションのみを復元する必要がある場合は、それらを指定できます。バックアップからパーティション 1 と 4 を復元するには: - -```sql -RESTORE TABLE test.table PARTITIONS '2', '3' - FROM Disk('backups', 'filename.zip') -``` - -### tar アーカイブとしてのバックアップ {#backups-as-tar-archives} - -バックアップは tar アーカイブとして保存することもできます。パスワードがサポートされていない点を除き、機能は zip の場合と同じです。 - -バックアップを tar 形式で作成します: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar') -``` - -対応するリストア: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.tar') -``` - -圧縮方式を変更するには、バックアップ名に適切なファイル拡張子を付ける必要があります。例えば、tar アーカイブを gzip で圧縮するには次のようにします。 - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar.gz') -``` - -サポートされている圧縮ファイルの拡張子は、`tar.gz`、`.tgz`、`tar.bz2`、`tar.lzma`、`.tar.zst`、`.tzst`、`.tar.xz` です。 - -### バックアップのステータスを確認する {#check-the-status-of-backups} - -バックアップコマンドは `id` と `status` を返し、その `id` を使ってバックアップのステータスを取得できます。これは、時間のかかる ASYNC バックアップの進行状況を確認するのに非常に便利です。以下の例は、既存のバックアップファイルを上書きしようとしたときに発生した失敗を示しています。 - -```sql -BACKUP TABLE helloworld.my_first_table TO Disk('backups', '1.zip') ASYNC -``` - -```response -┌─id───────────────────────────────────┬─status──────────┐ -│ 7678b0b3-f519-4e6e-811f-5a0781a4eb52 │ CREATING_BACKUP │ -└──────────────────────────────────────┴─────────────────┘ - -1行が返されました。経過時間: 0.001秒 -``` - -```sql -SELECT - * -FROM system.backups -WHERE id='7678b0b3-f519-4e6e-811f-5a0781a4eb52' -FORMAT Vertical -``` - -```response -行 1: -────── -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -#highlight-next-line -status: BACKUP_FAILED -num_files: 0 -uncompressed_size: 0 -compressed_size: 0 -#highlight-next-line -error: Code: 598. DB::Exception: Backup Disk('backups', '1.zip') already exists. (BACKUP_ALREADY_EXISTS) (version 22.8.2.11 (official build)) -start_time: 2022-08-30 09:21:46 -end_time: 2022-08-30 09:21:46 - -1行のセット。経過時間: 0.002秒 -``` - - -`system.backups` テーブルに加えて、すべてのバックアップおよびリストア操作は、システムログテーブル [backup_log](../operations/system-tables/backup_log.md) にも記録されます。 - -```sql -SELECT * -FROM system.backup_log -WHERE id = '7678b0b3-f519-4e6e-811f-5a0781a4eb52' -ORDER BY event_time_microseconds ASC -FORMAT Vertical -``` - -```response -Row 1: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.097414 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-18 11:13:43 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Row 2: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.174782 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: BACKUP_FAILED -#highlight-next-line -error: Code: 598. DB::Exception: Backup Disk('backups', '1.zip') already exists. (BACKUP_ALREADY_EXISTS) (version 23.8.1.1) -start_time: 2023-08-18 11:13:43 -end_time: 2023-08-18 11:13:43 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -2 rows in set. Elapsed: 0.075 sec. -``` - - -## S3 エンドポイントを使用するように BACKUP/RESTORE を構成する {#configuring-backuprestore-to-use-an-s3-endpoint} - -バックアップを S3 バケットに書き込むには、次の 3 つの情報が必要です。 - -* S3 エンドポイント\ - 例: `https://mars-doc-test.s3.amazonaws.com/backup-S3/` -* アクセスキー ID\ - 例: `ABC123` -* シークレットアクセスキー\ - 例: `Abc+123` - -:::note -S3 バケットの作成手順については、[Use S3 Object Storage as a ClickHouse disk](/integrations/data-ingestion/s3/index.md#configuring-s3-for-clickhouse-use) で説明しています。ポリシーを保存したらこのドキュメントに戻ってください。S3 バケットを使用するように ClickHouse を設定する必要はありません。 -::: - -バックアップの保存先は次のように指定します。 - -```sql -S3('/<ディレクトリ>', '<アクセスキーID>', '<シークレットアクセスキー>') -``` - -```sql -CREATE TABLE data -( - `key` Int, - `value` String, - `array` Array(String) -) -ENGINE = MergeTree -ORDER BY tuple() -``` - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 1000 -``` - -### ベース(初期)バックアップを作成する {#create-a-base-initial-backup} - -増分バックアップを実行するには、開始点となる *ベース* バックアップが必要です。この例では、後でベースバックアップとして使用します。S3 の宛先の最初のパラメーターは S3 エンドポイントであり、その後に、このバックアップで使用するバケット内のディレクトリを指定します。この例では、ディレクトリ名は `my_backup` です。 - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ de442b75-a66c-4a3c-a193-f76f278c70f3 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### データをさらに追加する {#add-more-data} - -増分バックアップには、ベースバックアップとバックアップ対象テーブルの現在の内容との差分が格納されます。増分バックアップを作成する前に、テーブルにデータを追加します。 - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 100 -``` - -### 増分バックアップを作成する {#take-an-incremental-backup} - -このバックアップコマンドはベースバックアップと似ていますが、`SETTINGS base_backup` とベースバックアップの場所を指定する点が異なります。増分バックアップの保存先はベースバックアップと同じディレクトリではなく、同じエンドポイント上のバケット内にある別のターゲットディレクトリであることに注意してください。ベースバックアップは `my_backup` にあり、増分バックアップは `my_incremental` に書き込まれます。 - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') SETTINGS base_backup = S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ f6cd3900-850f-41c9-94f1-0c4df33ea528 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### 増分バックアップからのリストア {#restore-from-the-incremental-backup} - -このコマンドは、増分バックアップを新しいテーブル `data3` にリストアします。増分バックアップをリストアする場合、ベースバックアップも合わせて含まれることに注意してください。リストア時には、増分バックアップのみを指定してください。 - -```sql -RESTORE TABLE data AS data3 FROM S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ ff0c8c39-7dff-4324-a241-000796de11ca │ RESTORED │ -└──────────────────────────────────────┴──────────┘ -``` - -### 件数を確認する {#verify-the-count} - - -元のテーブル `data` には、1,000 行の挿入と 100 行の挿入の 2 回の INSERT が行われており、合計で 1,100 行になっています。復元されたテーブルに 1,100 行あることを確認します。 - -```sql -SELECT count() -FROM data3 -``` - -```response -┌─count()─┐ -│ 1100 │ -└─────────┘ -``` - - -### 内容を検証する {#verify-the-content} - -ここでは、元のテーブル `data` の内容を、復元したテーブル `data3` と比較します。 - -```sql -SELECT throwIf(( - SELECT groupArray(tuple(*)) - FROM data - ) != ( - SELECT groupArray(tuple(*)) - FROM data3 - ), 'BACKUP/RESTORE後にデータが一致しません') -``` - -## S3 ディスクを使用した BACKUP/RESTORE {#backuprestore-using-an-s3-disk} - -ClickHouse のストレージ設定で S3 ディスクを設定することで、`BACKUP`/`RESTORE` を S3 を対象として実行することもできます。`/etc/clickhouse-server/config.d` にファイルを追加して、次のようにディスクを設定します。 - -```xml - - - - - s3_plain - - - - - - - - -
- s3_plain -
-
-
-
-
- - - s3_plain - -
-``` - -あとは通常どおり `BACKUP`/`RESTORE` を行います。 - -```sql -BACKUP TABLE data TO Disk('s3_plain', 'cloud_backup'); -RESTORE TABLE data AS data_restored FROM Disk('s3_plain', 'cloud_backup'); -``` - -:::note -ただし、次の点に注意してください。 - -* このディスクは `MergeTree` 自体には使用せず、`BACKUP`/`RESTORE` のみに使用してください -* テーブルが S3 ストレージをバックエンドとしている場合、パーツを宛先バケットにコピーする際には、宛先側のクレデンシャルを用いて、`CopyObject` 呼び出しによる S3 のサーバーサイドコピーを試行します。認証エラーが発生した場合は、パーツを一度ダウンロードしてからアップロードするバッファ経由のコピー方式にフォールバックしますが、これは非常に非効率です。この場合、宛先バケットのクレデンシャルに、ソースバケットに対する `read` 権限が付与されていることを確認してください。 - ::: - - -## 名前付きコレクションの使用 {#using-named-collections} - -名前付きコレクションは `BACKUP/RESTORE` のパラメータとして使用できます。例については [こちら](./named-collections.md#named-collections-for-backups) を参照してください。 - - - -## 代替案 {#alternatives} - -ClickHouse はデータをディスク上に保存しますが、ディスクをバックアップする方法は多数あります。以下はこれまでに利用されてきた代替手段の一部であり、お使いの環境にも適合する可能性があります。 - -### ソースデータを別の場所に複製する {#duplicating-source-data-somewhere-else} - -ClickHouse に取り込まれるデータは、多くの場合、[Apache Kafka](https://kafka.apache.org) のような永続的なキューを通じて配信されます。この場合、ClickHouse に書き込まれているのと同じデータストリームを読み取り、どこかのコールドストレージに保存するための追加のサブスクライバーセットを構成することができます。ほとんどの企業には、オブジェクトストアや [HDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) のような分散ファイルシステムといった、推奨される標準的なコールドストレージが既に存在します。 - -### ファイルシステムスナップショット {#filesystem-snapshots} - -一部のローカルファイルシステムはスナップショット機能を提供します(たとえば [ZFS](https://en.wikipedia.org/wiki/ZFS))。しかし、それらはライブクエリを処理する用途には最適とは限りません。考えられる解決策としては、この種のファイルシステムを用いた追加レプリカを作成し、`SELECT` クエリに使用される [Distributed](../engines/table-engines/special/distributed.md) テーブルからそれらを除外する方法があります。このようなレプリカ上のスナップショットは、データを変更するクエリの影響を受けません。さらに、これらのレプリカは 1 サーバーあたりより多くのディスクを接続した特別なハードウェア構成にでき、コスト効率を高められます。 - -データ量が小さい場合には、リモートテーブルへの単純な `INSERT INTO ... SELECT ...` でも有効な場合があります。 - -### パーツの操作 {#manipulations-with-parts} - -ClickHouse では、`ALTER TABLE ... FREEZE PARTITION ...` クエリを使用してテーブルパーティションのローカルコピーを作成できます。これは `/var/lib/clickhouse/shadow/` フォルダへのハードリンクを使って実装されているため、通常は古いデータに対して追加のディスク容量を消費しません。作成されたファイルのコピーは ClickHouse サーバーによっては扱われないため、そのまま残しておくこともできます。これにより、追加の外部システムを必要としない単純なバックアップが得られますが、それでもハードウェア障害の影響は受けます。このため、別の場所にリモートコピーを行い、その後ローカルコピーを削除する方が望ましいです。この用途には分散ファイルシステムやオブジェクトストアが引き続き有効な選択肢ですが、十分な容量を持つ通常の接続ファイルサーバーも利用可能です(この場合、転送はネットワークファイルシステム経由、または [rsync](https://en.wikipedia.org/wiki/Rsync) によって行われる可能性があります)。 -バックアップからのデータ復元は `ALTER TABLE ... ATTACH PARTITION ...` を用いて実行できます。 - -パーティション操作に関連するクエリの詳細については、[ALTER のドキュメント](/sql-reference/statements/alter/partition)を参照してください。 - -このアプローチを自動化するためのサードパーティーツールが利用可能です: [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup)。 - - - -## バックアップとリストアの並行実行を禁止するための設定 {#settings-to-disallow-concurrent-backuprestore} - -バックアップとリストアの並行実行を禁止するには、それぞれ次の設定を使用します。 - -```xml - - - false - false - - -``` - -両方のデフォルト値は true なので、デフォルトではバックアップ/リストアの並行実行が許可されています。 -これらの設定がクラスターで false に設定されている場合、そのクラスター上で同時に実行できるバックアップ/リストアは 1 つだけになります。 - - -## AzureBlobStorage エンドポイントを使用するように BACKUP/RESTORE を構成する {#configuring-backuprestore-to-use-an-azureblobstorage-endpoint} - -バックアップを書き込む AzureBlobStorage コンテナには、以下の情報が必要です。 - -* AzureBlobStorage エンドポイントの接続文字列 / URL -* コンテナ -* パス -* アカウント名(URL が指定されている場合) -* アカウント キー(URL が指定されている場合) - -バックアップの保存先は次のように指定します。 - -```sql -AzureBlobStorage('/', '', '', '', '') -``` - -```sql -BACKUP TABLE data TO AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -RESTORE TABLE data AS data_restored FROM AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -``` - - -## システムテーブルのバックアップ {#backup-up-system-tables} - -システムテーブルもバックアップおよびリストアのワークフローに含めることができますが、含めるかどうかはユースケースによって異なります。 - -### ログテーブルのバックアップ {#backing-up-log-tables} - -`query_log` や `part_log` のように、`_log` サフィックスを持ち履歴データを保存するシステムテーブルは、他のテーブルと同様にバックアップおよびリストアできます。ユースケースとして履歴データの分析に依存している場合、たとえば `query_log` を使ってクエリパフォーマンスを追跡したり、問題のデバッグに利用したりしている場合には、これらのテーブルをバックアップ戦略に含めることを推奨します。一方で、これらのテーブルの履歴データが不要な場合は、バックアップストレージ容量を節約するために除外できます。 - -### アクセス管理テーブルのバックアップ {#backing-up-access-management-tables} - -`users`、`roles`、`row_policies`、`settings_profiles`、`quotas` など、アクセス管理に関連するシステムテーブルは、バックアップおよびリストア処理の際に特別な扱いを受けます。これらのテーブルがバックアップに含まれている場合、その内容は特別な `accessXX.txt` ファイルにエクスポートされます。このファイルには、アクセスエンティティを作成および構成するための同等の SQL ステートメントが格納されます。リストア時には、リストア処理がこれらのファイルを解釈し、ユーザー、ロール、およびその他の設定を再作成するための SQL コマンドが再適用されます。 - -この機能により、ClickHouse クラスターのアクセス制御構成を、クラスター全体のセットアップの一部としてバックアップおよびリストアできます。 - -Note: この機能は、SQL コマンドによって管理される構成(["SQL-driven Access Control and Account Management"](/operations/access-rights#enabling-access-control) と呼ばれます)に対してのみ動作します。ClickHouse サーバーの設定ファイル(例: `users.xml`)で定義されたアクセス構成はバックアップに含まれず、この方法でリストアすることはできません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/caches.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/caches.md deleted file mode 100644 index 94dab47a3f3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/caches.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'クエリの実行時に、ClickHouse はさまざまな種類のキャッシュを使用します。' -sidebar_label: 'キャッシュ' -sidebar_position: 65 -slug: /operations/caches -title: 'キャッシュの種類' -keywords: ['cache'] -doc_type: 'reference' ---- - -# キャッシュの種類 {#cache-types} - -クエリを実行する際には、ClickHouse はさまざまなキャッシュを利用してクエリ処理を高速化し、 -ディスクからの読み取りやディスクへの書き込みの必要性を減らします。 - -主なキャッシュの種類は次のとおりです: - -* `mark_cache` — [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md) ファミリーのテーブルエンジンで使用される [marks](/development/architecture#merge-tree) のキャッシュ。 -* `uncompressed_cache` — [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md) ファミリーのテーブルエンジンで使用される非圧縮データのキャッシュ。 -* オペレーティングシステムのページキャッシュ(実データを保持するファイルに対して間接的に利用される)。 - -このほかにも、多数の追加キャッシュがあります: - -* DNS キャッシュ。 -* [Regexp](/interfaces/formats/Regexp) キャッシュ。 -* コンパイル済み式のキャッシュ。 -* [ベクトル類似性インデックス](../engines/table-engines/mergetree-family/annindexes.md) キャッシュ。 -* [テキストインデックス](../engines/table-engines/mergetree-family/invertedindexes.md#caching) キャッシュ。 -* [Avro format](/interfaces/formats/Avro) スキーマキャッシュ。 -* [Dictionaries](../sql-reference/dictionaries/index.md) データキャッシュ。 -* スキーマ推論キャッシュ。 -* S3、Azure、ローカルおよびその他のディスクを対象とした [Filesystem cache](storing-data.md)。 -* [Userspace page cache](/operations/userspace-page-cache)。 -* [Query cache](query-cache.md)。 -* [Query condition cache](query-condition-cache.md)。 -* フォーマットスキーマキャッシュ。 - -パフォーマンスチューニング、トラブルシューティング、またはデータ整合性の観点から -これらのキャッシュのいずれかを削除したい場合は、 -[`SYSTEM DROP ... CACHE`](../sql-reference/statements/system.md) ステートメントを使用できます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md deleted file mode 100644 index c3a3aae87c8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -description: 'ClickHouse のクラスターディスカバリに関するドキュメント' -sidebar_label: 'クラスターディスカバリ' -slug: /operations/cluster-discovery -title: 'クラスターディスカバリ' -doc_type: 'ガイド' ---- - -# クラスターディスカバリ {#cluster-discovery} - -## 概要 {#overview} - -ClickHouse の Cluster Discovery 機能は、ノードを設定ファイル内で明示的に定義しなくても、自動的に検出して登録できるようにすることで、クラスタの構成を簡素化します。これは、各ノードを手動で定義することが負担になる場合に特に有用です。 - -:::note - -Cluster Discovery は実験的な機能であり、将来のバージョンで変更または削除される可能性があります。 -この機能を有効にするには、設定ファイルに `allow_experimental_cluster_discovery` 設定項目を追加してください。 - -```xml - - - 1 - - -``` - -::: - -## リモートサーバーの設定 {#remote-servers-configuration} - -### 従来の手動設定 {#traditional-manual-configuration} - -従来は ClickHouse では、クラスタ内の各シャードおよびレプリカを設定ファイルで手動指定する必要がありました。 - -```xml - - - - - node1 - 9000 - - - node2 - 9000 - - - - - node3 - 9000 - - - node4 - 9000 - - - - - -``` - -### クラスター検出を使用する {#using-cluster-discovery} - -Cluster Discovery を使用すると、各ノードを明示的に定義する代わりに、ZooKeeper 内のパスを 1 つ指定するだけで済みます。ZooKeeper のそのパス配下に登録されたすべてのノードは、自動的に検出されてクラスターに追加されます。 - -```xml - - - - /clickhouse/discovery/cluster_name - - - - - - - - - - - - - - - - - -``` - -特定のノードに対してシャード番号を指定したい場合は、`` セクション内に `` タグを記述できます。 - -`node1` および `node2` の場合: - -```xml - - /clickhouse/discovery/cluster_name - 1 - -``` - -`node3` および `node4` の場合: - -```xml - - /clickhouse/discovery/cluster_name - 2 - -``` - -### オブザーバーモード {#observer-mode} - -オブザーバーモードで構成されたノードは、自身をレプリカとして登録しません。 -これらのノードはクラスター内の他のアクティブなレプリカを監視・検出するだけで、能動的には参加しません。 -オブザーバーモードを有効にするには、`` セクション内に `` タグを追加します。 - -```xml - - /clickhouse/discovery/cluster_name - - -``` - -### クラスターの検出 {#discovery-of-clusters} - -場合によっては、クラスター内のホストだけでなく、クラスター自体を追加・削除する必要が生じることがあります。複数のクラスターのルートパスを指定するために `` ノードを使用できます。 - -```xml - - - - /clickhouse/discovery - - - - -``` - -この場合、別のホストがパス `/clickhouse/discovery/some_new_cluster` で自分自身を登録すると、名前が `some_new_cluster` のクラスタが追加されます。 - -両方の機能を同時に使用できます。ホストはクラスタ `my_cluster` に自分自身を登録しつつ、他の任意のクラスタを検出することもできます。 - -```xml - - - - /clickhouse/discovery/my_cluster - - - - - /clickhouse/discovery - - - - -``` - -制限事項: - -* 同じ `remote_servers` サブツリー内で `` と `` を同時に使用することはできません。 -* `` と併用できるのは `` のみです。 -* Keeper で指定されたパスの最後の部分がクラスタ名として使用されますが、登録時には XML タグに記載された名前が使用されます。 - -## ユースケースと制限事項 {#use-cases-and-limitations} - -指定された ZooKeeper パスにノードが追加または削除されると、構成変更やサーバーの再起動を行うことなく、それらのノードはクラスターに自動的に認識されて参加するか、クラスターから削除されます。 - -ただし、変更の対象はクラスター構成のみであり、データや既存のデータベースおよびテーブルには影響しません。 - -3 ノードのクラスターを用いた次の例を考えます。 - -```xml - - - - /clickhouse/discovery/default_cluster - - - -``` - -```sql -SELECT * EXCEPT (default_database, errors_count, slowdowns_count, estimated_recovery_time, database_shard_name, database_replica_name) -FROM system.clusters WHERE cluster = 'default'; - -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -```sql -CREATE TABLE event_table ON CLUSTER default (event_time DateTime, value String) -ENGINE = ReplicatedMergeTree('/clickhouse/tables/event_table', '{replica}') -ORDER BY event_time PARTITION BY toYYYYMM(event_time); - -INSERT INTO event_table ... -``` - -次に、クラスターに新しいノードを追加します。設定ファイルの `remote_servers` セクションに同じエントリを含めて、新しいノードを起動します。 - -```response -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 4 │ b0df3669b81f │ 172.26.0.6 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -4 番目のノードはクラスターに参加していますが、テーブル `event_table` は依然として最初の 3 つのノードにしか存在していません。 - -```sql -SELECT hostname(), database, table FROM clusterAllReplicas(default, system.tables) WHERE table = 'event_table' FORMAT PrettyCompactMonoBlock -``` - -┌─hostname()───┬─database─┬─table───────┐ -│ a6a68731c21b │ default │ event_table │ -│ 92d3c04025e8 │ default │ event_table │ -│ 8e62b9cb17a1 │ default │ event_table │ -└──────────────┴──────────┴─────────────┘ - -``` - -すべてのノードでテーブルをレプリケートする必要がある場合は、クラスタディスカバリの代替として[Replicated](../engines/database-engines/replicated.md)データベースエンジンを使用することができます。 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/configuration-files.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/configuration-files.md deleted file mode 100644 index ebdcbd365f1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/configuration-files.md +++ /dev/null @@ -1,472 +0,0 @@ ---- -description: 'このページでは、ClickHouse サーバーを XML または YAML 構文の設定ファイルで構成する方法を説明します。' -sidebar_label: '設定ファイル' -sidebar_position: 50 -slug: /operations/configuration-files -title: '設定ファイル' -doc_type: 'guide' ---- - -:::note -XML ベースの設定プロファイルおよび設定ファイルは ClickHouse Cloud ではサポートされていません。そのため、ClickHouse Cloud には config.xml ファイルが存在しません。代わりに、設定プロファイルを通じて設定を管理するために SQL コマンドを使用する必要があります。 - -詳細については、["Configuring Settings"](/manage/settings) を参照してください。 -::: - -ClickHouse サーバーは、XML または YAML 構文の設定ファイルで構成できます。 -ほとんどのインストール形態では、ClickHouse サーバーはデフォルトの設定ファイルとして `/etc/clickhouse-server/config.xml` を使用して動作します。ただし、サーバー起動時にコマンドラインオプション `--config-file` または `-C` を使用して、設定ファイルの場所を手動で指定することもできます。 -追加の設定ファイルは、メインの設定ファイルからの相対パスである `config.d/` ディレクトリ、例えば `/etc/clickhouse-server/config.d/` ディレクトリに配置できます。 -このディレクトリ内のファイルとメインの設定ファイルは、ClickHouse サーバーに設定が適用される前の前処理段階でマージされます。 -設定ファイルはアルファベット順にマージされます。 -更新を簡素化しモジュール化を改善するために、デフォルトの `config.xml` ファイルは変更せずに保持し、追加のカスタマイズは `config.d/` に配置することがベストプラクティスです。 -ClickHouse Keeper の設定は `/etc/clickhouse-keeper/keeper_config.xml` に格納されています。 -同様に、Keeper 用の追加の設定ファイルは `/etc/clickhouse-keeper/keeper_config.d/` に配置する必要があります。 - -XML と YAML の設定ファイルを混在させることができ、例えばメインの設定ファイルを `config.xml` とし、追加の設定ファイルとして `config.d/network.xml`、`config.d/timezone.yaml`、`config.d/keeper.yaml` を用意することができます。 -1 つの設定ファイル内で XML と YAML を混在させることはサポートされていません。 -XML 設定ファイルでは、トップレベルのタグとして `...` を使用する必要があります。 -YAML 設定ファイルでは、`clickhouse:` は省略可能であり、省略された場合はパーサーが自動的に挿入します。 - - - -## 設定のマージ {#merging} - -2 つの設定ファイル(通常はメインの設定ファイルと `config.d/` 内の別の設定ファイル)は、次のようにマージされます。 - -* あるノード(要素へのパス)が両方のファイルに存在し、かつ `replace` または `remove` 属性を持たない場合、そのノードはマージ後の設定ファイルに含まれ、両方のノード配下の子要素が含まれて再帰的にマージされます。 -* 2 つのノードの一方が `replace` 属性を持つ場合、そのノードはマージ後の設定ファイルに含まれますが、子要素は `replace` 属性を持つ側のノードのもののみが含まれます。 -* 2 つのノードの一方が `remove` 属性を持つ場合、そのノードはマージ後の設定ファイルには含まれません(すでに存在している場合は削除されます)。 - -例えば、2 つの設定ファイルが次のような場合を考えます。 - -```xml title="config.xml" - - - 1 - - - 2 - - - 3 - - -``` - -および - -```xml title="config.d/other_config.xml" - - - 4 - - - 5 - - - 6 - - -``` - -マージによって生成される設定ファイルは次のとおりです。 - -```xml - - - 1 - 4 - - - 5 - - -``` - -### 環境変数および ZooKeeper ノードによる置換 {#from_env_zk} - -要素の値を環境変数の値で置き換えることを指定するには、属性 `from_env` を使用できます。 - -たとえば、環境変数 `$MAX_QUERY_SIZE` に `150000` が設定されている場合: - -```xml - - - - - - - -``` - -最終的な設定は次のとおりです: - -```xml - - - - 150000 - - - -``` - -同様のことは `from_zk`(ZooKeeper ノード)を使用しても行えます。 - -```xml - - - -``` - - -```shell -# clickhouse-keeper-client {#clickhouse-keeper-client} -/ :) touch /zk_configs -/ :) create /zk_configs/postgresql_port "9005" -/ :) get /zk_configs/postgresql_port -9005 -``` - -その結果、設定は次のようになります。 - -```xml - - 9005 - -``` - -#### デフォルト値 {#default-values} - -`from_env` または `from_zk` 属性を持つ要素には、追加で `replace="1"` 属性を指定できます(この属性は `from_env` / `from_zk` より前に記述する必要があります)。 -この場合、その要素でデフォルト値を定義できます。 -環境変数または ZooKeeper ノードが設定されていればその値を使用し、設定されていなければデフォルト値を使用します。 - -前の例を繰り返しますが、ここでは `MAX_QUERY_SIZE` が設定されていないと仮定します: - -```xml - - - - 150000 - - - -``` - -すると、設定は次のようになります。 - -```xml - - - - 150000 - - - -``` - - -## ファイル内容による置換 {#substitution-with-file-content} - -設定の一部をファイルの内容で置き換えることも可能です。これは次の 2 つの方法で行えます。 - -* *値の置換*: 要素が `incl` 属性を持つ場合、その値は参照先ファイルの内容で置き換えられます。デフォルトでは、置換を定義したファイルへのパスは `/etc/metrika.xml` です。これはサーバー設定内の [`include_from`](../operations/server-configuration-parameters/settings.md#include_from) 要素で変更できます。置換値はこのファイル内の `/clickhouse/substitution_name` 要素に指定します。`incl` で指定した置換が存在しない場合、その情報はログに記録されます。欠落した置換について ClickHouse がログ出力しないようにするには、属性 `optional="true"` を指定します(例えば、[macros](../operations/server-configuration-parameters/settings.md#macros) 用の設定など)。 -* *要素の置換*: 要素全体を置換で差し替えたい場合は、要素名として `include` を使用します。要素名 `include` は属性 `from_zk = "/path/to/node"` と組み合わせることができます。この場合、要素の値は `/path/to/node` にある ZooKeeper ノードの内容で置き換えられます。これは、XML のサブツリー全体を 1 つの ZooKeeper ノードとして保存している場合にも機能し、そのサブツリーは元の要素に完全に挿入されます。 - -その例を以下に示します。 - -```xml - - - - - - - - - - -``` - -既存の設定に追記するのではなく、include で差し込む内容を既存の設定とマージしたい場合は、属性 `merge="true"` を使用できます。たとえば、`` のように指定します。この場合、既存の設定は include で読み込まれる内容とマージされ、既存の設定値は読み込まれた側の値で置き換えられます。 - - -## 設定の暗号化と秘匿 {#encryption} - -共通鍵暗号を使用して、平文のパスワードや秘密鍵などの設定要素を暗号化できます。 -そのためには、まず [encryption codec](../sql-reference/statements/create/table.md#encryption-codecs) を設定し、その後、暗号化する要素に対して属性 `encrypted_by` を追加し、その値として暗号化コーデックの名前を指定します。 - -属性 `from_zk`、`from_env`、`incl` や要素 `include` とは異なり、前処理後のファイルでは値の置換(つまり暗号化された値の復号)は行われません。 -復号はサーバープロセスの実行時にのみ行われます。 - -例: - -```xml - - - - - 00112233445566778899aabbccddeeff - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -属性 [`from_env`](#from_env_zk) および [`from_zk`](#from_env_zk) は、`encryption_codecs` に対しても適用できます。 - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -暗号鍵と暗号化された値は、どちらの設定ファイルでも定義できます。 - -`config.xml` の例を次に示します: - -```xml - - - - - - - - - -``` - -`users.xml` の例を次に示します。 - -```xml - - - - - 96280000000D000000000030D4632962295D46C6FA4ABF007CCEC9C1D0E19DA5AF719C1D9A46C446 - default - - - - -``` - -値を暗号化するには、例として `encrypt_decrypt` プログラムを使用できます。 - -```bash -./encrypt_decrypt /etc/clickhouse-server/config.xml -e AES_128_GCM_SIV abcd -``` - -```text -961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 -``` - -暗号化された設定要素を使用していても、前処理後の設定ファイルには暗号化された要素がそのまま表示されます。 -これが ClickHouse のデプロイメントにとって問題となる場合は、次の 2 つの代替手段があります。前処理後のファイルのファイルパーミッションを 600 に設定するか、属性 `hide_in_preprocessed` を使用します。 - -例: - -```xml - - - - admin - secret - - - -``` - - -## ユーザー設定 {#user-settings} - -`config.xml` ファイルでは、ユーザー設定、プロファイル、およびクォータを含む別の設定ファイルを指定できます。この設定ファイルへの相対パスは `users_config` 要素で設定します。デフォルトでは `users.xml` が使用されます。`users_config` が省略された場合、ユーザー設定、プロファイル、およびクォータは `config.xml` 内で直接指定されます。 - -ユーザー設定は、`config.xml` および `config.d/` と同様に、個別のファイルに分割できます。 -ディレクトリ名は、`.xml` 接尾辞を除いた `users_config` 設定値に `.d` を連結したものとして定義されます。 -`users_config` のデフォルトが `users.xml` であるため、デフォルトでは `users.d` ディレクトリが使用されます。 - -設定ファイルは、まず設定値を考慮して[マージ](#merging)され、その後に include が処理される点に注意してください。 - - - -## XML の例 {#example} - -例えば、各ユーザーごとにこのように個別の設定ファイルを用意できます: - -```bash -$ cat /etc/clickhouse-server/users.d/alice.xml -``` - -```xml - - - - analytics - - ::/0 - - ... - analytics - - - -``` - - -## YAML の例 {#example-1} - -ここでは、YAML で記述されたデフォルト設定を確認できます: [`config.yaml.example`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/server/config.yaml.example)。 - -ClickHouse の設定においては、YAML 形式と XML 形式ではいくつかの違いがあります。 -YAML 形式で設定を書く際のヒントを以下に示します。 - -テキスト値を持つ XML タグは、YAML のキーと値のペアで表現されます。 - -```yaml -key: value -``` - -対応する XML: - -```xml - -``` - -ネストされた XML ノードは YAML マップとして表現されます。 - -```yaml -map_key: - key1: val1 - key2: val2 - key3: val3 -``` - -対応するXML: - -```xml - - val1 - val2 - val3 - -``` - -同じ XML タグを複数回作成するには、YAML シーケンスを使用します。 - -```yaml -seq_key: - - val1 - - val2 - - key1: val3 - - map: - key2: val4 - key3: val5 -``` - -対応するXML: - -```xml -val1 -val2 - - val3 - - - - val4 - val5 - - -``` - -XML の属性を指定するには、`@` プレフィックス付きの属性キー名を使用できます。`@` は YAML 標準で予約済みなので、必ずダブルクォーテーションで囲む必要があります。 - -```yaml -map: - "@attr1": value1 - "@attr2": value2 - key: 123 -``` - -対応するXML: - -```xml - - 123 - -``` - -YAML のシーケンス内でも属性を使用できます: - -```yaml -seq: - - "@attr1": value1 - - "@attr2": value2 - - 123 - - abc -``` - -対応する XML: - -```xml -123 -abc -``` - -前述の構文では、XML 属性を持つ XML テキストノードを YAML として表現することはできません。この特殊なケースには、 -`#text` 属性キーを使用します。 - -```yaml -map_key: - "@attr1": value1 - "#text": value2 -``` - -対応するXML: - -```xml -value2 -``` - - -## 実装の詳細 {#implementation-details} - -各設定ファイルごとに、サーバーは起動時に `file-preprocessed.xml` ファイルも生成します。これらのファイルには、すべての置換と上書きが反映された結果が含まれており、参照用として利用されることを想定しています。設定ファイル内で ZooKeeper による置換が使われているにもかかわらず、サーバー起動時に ZooKeeper が利用できない場合、サーバーはこの前処理済みファイルから設定を読み込みます。 - -サーバーは、設定ファイルに加え、置換および上書きを行う際に使用されたファイルや ZooKeeper ノードの変更も追跡し、ユーザーおよびクラスタ用の設定を即時に再読み込みします。これにより、サーバーを再起動することなく、クラスタとユーザーおよびその設定を変更できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md deleted file mode 100644 index 5ca39ec4644..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -description: 'HTTP に関するドキュメント' -slug: /operations/external-authenticators/http -title: 'HTTP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -HTTP サーバーを使用して ClickHouse ユーザーを認証できます。HTTP 認証は、`users.xml` またはローカルのアクセス制御パスで定義された既存ユーザーに対する外部認証方式としてのみ使用できます。現在は、GET メソッドを使用する [Basic](https://datatracker.ietf.org/doc/html/rfc7617) 認証スキームがサポートされています。 - -## HTTP 認証サーバーの定義 {#http-auth-server-definition} - -HTTP認証サーバーを定義するには、`config.xml` ファイルに `http_authentication_servers` セクションを追加する必要があります。 - -**例** - -```xml - - - - - http://localhost:8000/auth - 1000 - 1000 - 1000 - 3 - 50 - 1000 - - Custom-Auth-Header-1 - Custom-Auth-Header-2 - - - - - - -``` - -`http_authentication_servers` セクション内には、異なる名前を用いることで複数の HTTP サーバーを定義できます。 - -**パラメータ** - -* `uri` - 認証リクエストを送信するための URI - -サーバーとの通信に使用されるソケットのタイムアウト(ミリ秒): - -* `connection_timeout_ms` - デフォルト: 1000 ms -* `receive_timeout_ms` - デフォルト: 1000 ms -* `send_timeout_ms` - デフォルト: 1000 ms - -再試行用パラメータ: - -* `max_tries` - 認証リクエストを試行する最大回数。デフォルト: 3 -* `retry_initial_backoff_ms` - 再試行時のバックオフ初期間隔。デフォルト: 50 ms -* `retry_max_backoff_ms` - バックオフの最大間隔。デフォルト: 1000 ms - -転送するヘッダー: - -ここでは、クライアントリクエストのヘッダーから外部 HTTP 認証サーバーへ転送されるヘッダーを定義します。ヘッダーは設定済みのものと大文字小文字を区別せずに照合されますが、転送時は元の形、すなわち変更されずに送信されます。 - -### `users.xml` での HTTP 認証の有効化 {#enabling-http-auth-in-users-xml} - -ユーザーに対して HTTP 認証を有効にするには、ユーザー定義内で `password` などのセクションの代わりに `http_authentication` セクションを指定します。 - -パラメータ: - -* `server` - 前述のとおり、メインの `config.xml` ファイル内で設定されている HTTP 認証サーバーの名前。 -* `scheme` - HTTP 認証スキーム。現在サポートされているのは `Basic` のみです。デフォルト: Basic - -例(`users.xml` に記述): - -```xml - - - - - - basic_server - basic - - - -``` - -:::note -HTTP 認証は、他の認証メカニズムと併用できない点に注意してください。`http_authentication` と一緒に `password` など他のセクションが存在する場合、ClickHouse は強制終了されます。 -::: - -### SQL を使用した HTTP 認証の有効化 {#enabling-http-auth-using-sql} - -ClickHouse で [SQL ベースのアクセス制御とアカウント管理](/operations/access-rights#access-control-usage) が有効になっている場合、HTTP 認証で認証されるユーザーは、SQL 文を使用して作成することもできます。 - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' SCHEME 'Basic' -``` - -…または、スキームを明示的に指定しない場合は `Basic` がデフォルトとして使用されます - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' -``` - -### セッション設定の受け渡し {#passing-session-settings} - -HTTP 認証サーバーからのレスポンスボディが JSON 形式で、`settings` サブオブジェクトを含んでいる場合、ClickHouse はそのキーと値のペアを文字列値として解析しようとし、認証済みユーザーの現在のセッションのセッション設定としてそれらを設定します。解析に失敗した場合、サーバーからのレスポンスボディは無視されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md deleted file mode 100644 index 0271dbcac34..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'ClickHouse がサポートする外部認証方式の概要' -pagination_next: operations/external-authenticators/kerberos -sidebar_label: '外部ユーザー認証およびディレクトリ' -sidebar_position: 48 -slug: /operations/external-authenticators/ -title: '外部ユーザー認証およびディレクトリ' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -ClickHouse は、外部サービスを利用したユーザー認証および管理をサポートしています。 - -サポートされている外部認証方式およびディレクトリは次のとおりです。 - -* [LDAP](/operations/external-authenticators/ldap#ldap-external-authenticator) [Authenticator](./ldap.md#ldap-external-authenticator) および [Directory](./ldap.md#ldap-external-user-directory) -* Kerberos [Authenticator](/operations/external-authenticators/kerberos#kerberos-as-an-external-authenticator-for-existing-users) -* [SSL X.509 authentication](/operations/external-authenticators/ssl-x509) -* HTTP [Authenticator](./http.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md deleted file mode 100644 index bb288768131..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: '既存の ClickHouse ユーザーが適切に構成されている場合、Kerberos 認証プロトコルで認証できます。' -slug: /operations/external-authenticators/kerberos -title: 'Kerberos' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# Kerberos {#kerberos} - - - -既存の適切に設定された ClickHouse ユーザーは、Kerberos 認証プロトコルを使用して認証できます。 - -現在、Kerberos は `users.xml` またはローカルのアクセス制御パスで定義されている既存ユーザーに対する外部認証器としてのみ使用できます。これらのユーザーは HTTP リクエストのみを使用でき、かつ GSS-SPNEGO メカニズムを用いて認証できる必要があります。 - -この方式を利用するには、システム側で Kerberos が設定されており、かつ ClickHouse の設定で有効化されている必要があります。 - -## ClickHouse で Kerberos を有効化する {#enabling-kerberos-in-clickhouse} - -Kerberos を有効化するには、`config.xml` に `kerberos` セクションを追加する必要があります。このセクションには追加のパラメータを含めることができます。 - -#### パラメータ {#parameters} - -* `principal` - セキュリティコンテキストを受け入れる際に取得・使用される正規のサービスプリンシパル名。 - * このパラメータは省略可能で、省略された場合はデフォルトのプリンシパルが使用されます。 - -* `realm` - 認証を、そのリクエストのイニシエータの realm がこの値と一致するもののみに制限するために使用される realm。 - * このパラメータは省略可能で、省略された場合は realm による追加のフィルタリングは行われません。 - -* `keytab` - サービスの keytab ファイルへのパス。 - * このパラメータは省略可能で、省略された場合はサービスの keytab ファイルへのパスを `KRB5_KTNAME` 環境変数で指定する必要があります。 - -例(`config.xml` に記述): - -```xml - - - - -``` - -プリンシパルを指定した場合: - -```xml - - - - HTTP/clickhouse.example.com@EXAMPLE.COM - - -``` - -レルムによるフィルタリング: - -```xml - - - - EXAMPLE.COM - - -``` - -:::note -`kerberos` セクションは 1 つだけ定義できます。複数の `kerberos` セクションが存在する場合、ClickHouse は Kerberos 認証を無効にします。 -::: - -:::note -`principal` セクションと `realm` セクションは同時に指定できません。`principal` と `realm` の両方のセクションが存在する場合、ClickHouse は Kerberos 認証を無効にします。 -::: - -## 既存ユーザー向けの外部認証方式としての Kerberos {#kerberos-as-an-external-authenticator-for-existing-users} - -Kerberos は、ローカルに定義されたユーザー(`users.xml` またはローカルのアクセス制御パスで定義されたユーザー)の認証方法として使用できます。現在のところ、**HTTP インターフェイス経由のリクエストのみ**が(GSS-SPNEGO メカニズムを通じて)*Kerberos 対応*とできます。 - -Kerberos プリンシパル名の形式は通常、次のパターンに従います。 - -* *primary/instance@REALM* - -*/instance* の部分は 0 回以上出現する可能性があります。**イニシエーターの正規(canonical)なプリンシパル名の *primary* 部分が Kerberos 対応ユーザー名と一致している必要があり、一致した場合にのみ認証が成功します。** - -### `users.xml` で Kerberos を有効化する {#enabling-kerberos-in-users-xml} - -ユーザーに対して Kerberos 認証を有効にするには、ユーザー定義内で `password` などのセクションの代わりに `kerberos` セクションを指定します。 - -パラメータ: - -* `realm` - このレルムと一致するレルムを持つイニシエーターからのリクエストにのみ認証を制限するために使用されるレルム。 - * このパラメータは省略可能で、省略された場合はレルムによる追加フィルタリングは行われません。 - -例(`users.xml` に記述): - -```xml - - - - - - - - EXAMPLE.COM - - - - -``` - -:::note -Kerberos 認証は、他の認証メカニズムと同時に使用できない点に注意してください。`kerberos` と同時に `password` などの他のセクションが存在すると、ClickHouse はシャットダウンします。 -::: - -:::info Reminder -ここで、ユーザー `my_user` が `kerberos` を使用するようになった場合、前述のとおり、メインの `config.xml` ファイルで Kerberos を有効化しておく必要がある点に注意してください。 -::: - -### SQL を使用した Kerberos の有効化 {#enabling-kerberos-using-sql} - -ClickHouse で [SQL-driven Access Control and Account Management](/operations/access-rights#access-control-usage) が有効化されている場合、Kerberos で識別されるユーザーも SQL 文を使用して作成できます。 - -```sql -CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' -``` - -…または、レルムでフィルタリングしない場合: - -```sql -CREATE USER my_user IDENTIFIED WITH kerberos -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md deleted file mode 100644 index 965b69fcedd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: 'ClickHouse の LDAP 認証を設定するためのガイド' -slug: /operations/external-authenticators/ldap -title: 'LDAP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -LDAP サーバーは ClickHouse ユーザーの認証に使用できます。これを行う方法は 2 つあります。 - -* `users.xml` やローカルのアクセス制御パスで定義されている既存ユーザーに対する外部認証手段として LDAP を使用する。 -* LDAP を外部ユーザーディレクトリとして使用し、LDAP サーバー上に存在する場合にはローカルで未定義のユーザーの認証を許可する。 - -これら 2 つのアプローチのいずれにおいても、ClickHouse の設定内で内部名を持つ LDAP サーバーを定義し、設定の他の箇所からそれを参照できるようにする必要があります。 - -## LDAP サーバーの定義 {#ldap-server-definition} - -LDAP サーバーを定義するには、`config.xml` に `ldap_servers` セクションを追加する必要があります。 - -**例** - -```xml - - - - - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - - - - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - - - -``` - -なお、`ldap_servers` セクション内には、異なる名前を付けることで複数の LDAP サーバーを定義できます。 - -**パラメータ** - -* `host` — LDAP サーバーのホスト名または IP。このパラメータは必須であり、空にはできません。 -* `port` — LDAP サーバーのポート。`enable_tls` が `true` に設定されている場合のデフォルトは `636`、それ以外の場合は `389` です。 -* `bind_dn` — バインドに使用する DN を構築するためのテンプレート。 - * 認証が試行されるたびに、このテンプレート内のすべての `{user_name}` 文字列が実際のユーザー名で置き換えられ、最終的な DN が構築されます。 -* `user_dn_detection` — バインドされたユーザーの実際のユーザー DN を検出するための LDAP 検索パラメータのセクション。 - * これは主にサーバーが Active Directory の場合に、追加のロールマッピングのための検索フィルタで使用されます。最終的なユーザー DN は、`{user_dn}` 文字列を許可されている場所で置き換える際に使用されます。デフォルトではユーザー DN は bind DN と同一に設定されますが、検索が実行されると、検出された実際のユーザー DN の値で更新されます。 - * `base_dn` — LDAP 検索用の base DN を構築するために使用されるテンプレート。 - * LDAP 検索中に、このテンプレート内のすべての `{user_name}` および `{bind_dn}` 文字列が実際のユーザー名および bind DN で置き換えられ、最終的な DN が構築されます。 - * `scope` — LDAP 検索のスコープ。 - * 受け付けられる値は `base`、`one_level`、`children`、`subtree`(デフォルト)です。 - * `search_filter` — LDAP 検索用の検索フィルタを構築するために使用されるテンプレート。 - * LDAP 検索中に、このテンプレート内のすべての `{user_name}`、`{bind_dn}`、`{base_dn}` 文字列が、実際のユーザー名、bind DN、base DN で置き換えられ、最終的なフィルタが構築されます。 - * 特殊文字は XML で正しくエスケープする必要がある点に注意してください。 -* `verification_cooldown` — バインドが成功した後、この秒数の間は LDAP サーバーに問い合わせることなく、連続するすべてのリクエストについてユーザーが認証済みであるとみなされる期間。 - * キャッシュを無効化し、認証リクエストごとに LDAP サーバーへの問い合わせを強制するには、`0`(デフォルト)を指定します。 -* `enable_tls` — LDAP サーバーへのセキュア接続を使用するかどうかを制御するフラグ。 - * プレーンテキストの `ldap://` プロトコル(推奨されません)を使用するには `no` を指定します。 - * SSL/TLS 上の LDAP `ldaps://` プロトコル(推奨、デフォルト)を使用するには `yes` を指定します。 - * レガシーな StartTLS プロトコル(TLS へアップグレードされるプレーンテキストの `ldap://` プロトコル)を使用するには `starttls` を指定します。 -* `tls_minimum_protocol_version` — SSL/TLS の最小プロトコルバージョン。 - * 受け付けられる値は `ssl2`、`ssl3`、`tls1.0`、`tls1.1`、`tls1.2`(デフォルト)です。 -* `tls_require_cert` — SSL/TLS ピア証明書の検証動作。 - * 受け付けられる値は `never`、`allow`、`try`、`demand`(デフォルト)です。 -* `tls_cert_file` — 証明書ファイルへのパス。 -* `tls_key_file` — 証明書鍵ファイルへのパス。 -* `tls_ca_cert_file` — CA 証明書ファイルへのパス。 -* `tls_ca_cert_dir` — CA 証明書を含むディレクトリへのパス。 -* `tls_cipher_suite` — 許可される暗号スイート(OpenSSL の表記)。 - -## LDAP 外部認証 {#ldap-external-authenticator} - -リモートの LDAP サーバーを、ローカルで定義されたユーザー(`users.xml` またはローカルのアクセス制御パスで定義されたユーザー)のパスワード検証方法として使用できます。これを行うには、ユーザー定義内で `password` などのセクションの代わりに、事前に定義した LDAP サーバー名を指定します。 - -ログインのたびに、ClickHouse は [LDAP サーバー定義](#ldap-server-definition) で `bind_dn` パラメータにより定義された DN に対して、提供された認証情報を用いて「バインド」を試み、成功した場合、そのユーザーは認証済みと見なされます。これは一般的に「シンプルバインド」方式と呼ばれます。 - -**例** - -```xml - - - - - - - - my_ldap_server - - - - -``` - -ユーザー `my_user` は `my_ldap_server` を参照している点に注意してください。この LDAP サーバーは、前述のとおりメインの `config.xml` ファイルで設定されている必要があります。 - -SQL 駆動の[アクセス制御とアカウント管理](/operations/access-rights#access-control-usage)が有効な場合、LDAP サーバーで認証されるユーザーも [CREATE USER](/sql-reference/statements/create/user) ステートメントを使用して作成できます。 - -クエリ: - -```sql -CREATE USER my_user IDENTIFIED WITH ldap SERVER 'my_ldap_server'; -``` - -## LDAP 外部ユーザーディレクトリ {#ldap-external-user-directory} - -ローカルで定義されたユーザーに加えて、リモートの LDAP サーバーをユーザー定義のソースとして使用できます。これを行うには、`config.xml` ファイルの `users_directories` セクション内の `ldap` セクションで、事前に定義した LDAP サーバー名([LDAP Server Definition](#ldap-server-definition) を参照)を指定します。 - -各ログイン時に、ClickHouse はまずローカルでユーザー定義を検索し、通常どおり認証を試みます。ユーザーが定義されていない場合、ClickHouse は外部 LDAP ディレクトリ内にその定義が存在するとみなし、指定された DN に対して、与えられた認証情報を用いて LDAP サーバーへの「バインド」を試行します。成功した場合、そのユーザーは存在し、認証済みであると見なされます。ユーザーには、`roles` セクションで指定されたリストからロールが割り当てられます。さらに、`role_mapping` セクションも構成されている場合、LDAP の「search」を実行し、その結果を変換してロール名として扱い、ユーザーに割り当てることができます。これらすべては、SQL 駆動の [Access Control and Account Management](/operations/access-rights#access-control-usage) が有効であり、ロールが [CREATE ROLE](/sql-reference/statements/create/role) ステートメントを使用して作成されていることを前提としています。 - -**例** - -`config.xml` に記述します。 - -```xml - - - - - - my_ldap_server - - - - - - ou=groups,dc=example,dc=com - subtree - (&(objectClass=groupOfNames)(member={bind_dn})) - cn - clickhouse_ - - - - - - my_ad_server - - CN=Users,DC=example,DC=com - CN - subtree - (&(objectClass=group)(member={user_dn})) - clickhouse_ - - - - -``` - -`user_directories` セクション内の `ldap` セクションで参照されている `my_ldap_server` は、事前に `config.xml` 内で定義・設定されている LDAP サーバーでなければなりません([LDAP Server Definition](#ldap-server-definition) を参照)。 - -**パラメータ** - -* `server` — 上記の `ldap_servers` 設定セクションで定義されている LDAP サーバー名の 1 つ。このパラメータは必須で、空にはできません。 -* `roles` — LDAP サーバーから取得した各ユーザーに割り当てられる、ローカルで定義されたロールの一覧を含むセクション。 - * ここでロールが指定されていない場合、または(下記の)ロールマッピング時にロールが割り当てられない場合、ユーザーは認証後に一切の操作を行うことができません。 -* `role_mapping` — LDAP 検索パラメータとマッピングルールを定義するセクション。 - * ユーザーが認証するとき、LDAP にバインドした状態のまま、`search_filter` とログインしたユーザー名を使って LDAP 検索が実行されます。その検索で見つかった各エントリについて、指定された属性の値が抽出されます。指定されたプレフィックスを持つ各属性値については、プレフィックスが削除され、その残りの値が ClickHouse 内で定義されたローカルロールの名前になります。このロールはあらかじめ [CREATE ROLE](/sql-reference/statements/create/role) ステートメントで作成されている必要があります。 - * 同じ `ldap` セクション内には複数の `role_mapping` セクションを定義できます。それらはすべて適用されます。 - * `base_dn` — LDAP 検索用のベース DN を構築するために使用されるテンプレート。 - * 実際の DN は、各 LDAP 検索時にテンプレート内の `{user_name}`、`{bind_dn}`、`{user_dn}` の各部分文字列を、それぞれ実際のユーザー名、バインド DN、ユーザー DN に置き換えることで構築されます。 - * `scope` — LDAP 検索のスコープ。 - * 指定可能な値は `base`、`one_level`、`children`、`subtree`(デフォルト)です。 - * `search_filter` — LDAP 検索用の検索フィルタを構築するために使用されるテンプレート。 - * 実際のフィルタは、各 LDAP 検索時にテンプレート内の `{user_name}`、`{bind_dn}`、`{user_dn}`、`{base_dn}` の各部分文字列を、実際のユーザー名、バインド DN、ユーザー DN、およびベース DN に置き換えることで構築されます。 - * 特殊文字は XML では正しくエスケープする必要があることに注意してください。 - * `attribute` — LDAP 検索によって返される値を持つ属性名。デフォルトは `cn` です。 - * `prefix` — LDAP 検索で返される元の文字列リストの各文字列の先頭に付いていることが期待されるプレフィックス。プレフィックスは元の文字列から削除され、その結果の文字列がローカルロール名として扱われます。デフォルトは空です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md deleted file mode 100644 index 7e9c30cfa63..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'SSL X.509 のドキュメント' -slug: /operations/external-authenticators/ssl-x509 -title: 'SSL X.509 証明書認証' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -[SSL 'strict' option](../server-configuration-parameters/settings.md#openssl) は、受信接続に対して証明書検証を必須とします。この場合、信頼された証明書を持つ接続のみが確立されます。信頼されていない証明書による接続は拒否されます。このように、証明書検証により受信接続を一意に識別して認証できます。接続ユーザーの識別には、証明書の `Common Name` フィールドまたは `subjectAltName extension` フィールドが使用されます。`subjectAltName extension` では、サーバー設定でワイルドカード '*' を 1 つだけ使用できます。これにより、複数の証明書を同一ユーザーに関連付けることができます。さらに、証明書の再発行や失効は ClickHouse の設定に影響しません。 - -SSL 証明書認証を有効にするには、各 ClickHouse ユーザーに対する `Common Name` または `Subject Alt Name` の一覧を設定ファイル `users.xml` で指定する必要があります。 - -**例** - -```xml - - - - - - host.domain.com:example_user - host.domain.com:example_user_dev - - - - - - - DNS:host.domain.com - - - - - - - - URI:spiffe://foo.com/*/bar - - - - -``` - -SSL の[`chain of trust`](https://en.wikipedia.org/wiki/Chain_of_trust)が正しく機能するためには、[`caConfig`](../server-configuration-parameters/settings.md#openssl)パラメータが適切に設定されていることを確認することも重要です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/monitoring.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/monitoring.md deleted file mode 100644 index f10c5de5447..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/monitoring.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'ハードウェアリソースの利用状況や ClickHouse サーバーメトリクスを監視できます。' -keywords: ['監視', 'オブザーバビリティ', '高度なダッシュボード', 'ダッシュボード', 'オブザーバビリティダッシュボード'] -sidebar_label: '監視' -sidebar_position: 45 -slug: /operations/monitoring -title: '監視' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; - - -# 監視 {#monitoring} - -:::note -このガイドで説明している監視データは、ClickHouse Cloud で参照できます。以下で説明する組み込みダッシュボードで表示できるだけでなく、基本および高度なパフォーマンスメトリクスの両方をメインのサービスコンソールで直接確認することもできます。 -::: - -次の内容を監視できます: - -- ハードウェアリソースの利用状況 -- ClickHouse サーバーのメトリクス - - - -## 組み込みの高度なオブザーバビリティダッシュボード {#built-in-advanced-observability-dashboard} - -Screenshot 2023-11-12 at 6 08 58 PM - -ClickHouse には、`$HOST:$PORT/dashboard`(ユーザー名とパスワードが必要)からアクセスできる、組み込みの高度なオブザーバビリティダッシュボード機能が備わっており、次のメトリクスを表示します: -- Queries/second -- CPU 使用率(コア) -- Queries running -- Merges running -- Selected bytes/second -- IO wait -- CPU wait -- OS CPU Usage (userspace) -- OS CPU Usage (kernel) -- Read from disk -- Read from filesystem -- Memory (tracked) -- Inserted rows/second -- Total MergeTree parts -- Max parts for partition - - - -## リソース使用状況 {#resource-utilization} - -ClickHouse は、次のようなハードウェアリソースの状態も自動的に監視します。 - -- プロセッサの負荷および温度 -- ストレージシステム、RAM、ネットワークの使用率 - -このデータは `system.asynchronous_metric_log` テーブルに蓄積されます。 - - - -## ClickHouse サーバーメトリクス {#clickhouse-server-metrics} - -ClickHouse サーバーには、自身の状態を監視するための組み込み機能があります。 - -サーバーイベントを追跡するには、サーバーログを使用します。設定ファイルの [logger](../operations/server-configuration-parameters/settings.md#logger) セクションを参照してください。 - -ClickHouse は次の情報を収集します: - -- サーバーが計算リソースをどのように使用しているかに関する各種メトリクス。 -- クエリ処理に関する一般的な統計情報。 - -メトリクスは [system.metrics](/operations/system-tables/metrics)、[system.events](/operations/system-tables/events)、[system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) テーブルで確認できます。 - -ClickHouse を構成することで、メトリクスを [Graphite](https://github.com/graphite-project) にエクスポートできます。ClickHouse サーバー設定ファイルの [Graphite セクション](../operations/server-configuration-parameters/settings.md#graphite) を参照してください。メトリクスのエクスポートを設定する前に、公式の [ガイド](https://graphite.readthedocs.io/en/latest/install.html) に従って Graphite をセットアップしておく必要があります。 - -ClickHouse を構成することで、メトリクスを [Prometheus](https://prometheus.io) にエクスポートできます。ClickHouse サーバー設定ファイルの [Prometheus セクション](../operations/server-configuration-parameters/settings.md#prometheus) を参照してください。メトリクスのエクスポートを設定する前に、公式の [ガイド](https://prometheus.io/docs/prometheus/latest/installation/) に従って Prometheus をセットアップしておく必要があります。 - -さらに、HTTP API を通じてサーバーの可用性を監視できます。`HTTP GET` リクエストを `/ping` に送信してください。サーバーが利用可能であれば、`200 OK` で応答します。 - -クラスタ構成のサーバーを監視するには、[max_replica_delay_for_distributed_queries](../operations/settings/settings.md#max_replica_delay_for_distributed_queries) パラメータを設定し、HTTP リソース `/replicas_status` を使用する必要があります。`/replicas_status` へのリクエストは、レプリカが利用可能で他のレプリカから遅延していない場合には `200 OK` を返します。レプリカが遅延している場合には、遅延量に関する情報とともに `503 HTTP_SERVICE_UNAVAILABLE` を返します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/named-collections.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/named-collections.md deleted file mode 100644 index 0489d7456aa..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/named-collections.md +++ /dev/null @@ -1,645 +0,0 @@ ---- -description: '名前付きコレクションに関するドキュメント' -sidebar_label: '名前付きコレクション' -sidebar_position: 69 -slug: /operations/named-collections -title: '名前付きコレクション' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -名前付きコレクションは、外部ソースとの連携設定に使用されるキーと値のペアの集合を保存するための仕組みです。名前付きコレクションは、dictionaries、テーブル、table functions、オブジェクトストレージで使用できます。 - -名前付きコレクションは DDL または設定ファイルで定義でき、ClickHouse の起動時に適用されます。これにより、オブジェクトの作成が簡素化され、管理権限を持たないユーザーから認証情報を隠すことができます。 - -名前付きコレクション内のキーは、対応する関数、テーブルエンジン、データベースなどのパラメータ名と一致している必要があります。以下の例では、各タイプごとにパラメータリストへのリンクを示しています。 - -名前付きコレクションで設定されたパラメータは SQL で上書き可能であり、その例を以下で示しています。この機能は、`[NOT] OVERRIDABLE` キーワードや XML 属性、および/または設定オプション `allow_named_collection_override_by_default` を使用して制限できます。 - -:::warning -上書きが許可されている場合、管理権限を持たないユーザーが、非表示にしようとしている認証情報を割り出せてしまう可能性があります。 -その目的で名前付きコレクションを使用している場合は、デフォルトで有効になっている `allow_named_collection_override_by_default` を無効化する必要があります。 -::: - -## system データベースに名前付きコレクションを保存する {#storing-named-collections-in-the-system-database} - -### DDLの例 {#ddl-example} - -```sql -CREATE NAMED COLLECTION name AS -key_1 = 'value' OVERRIDABLE, -key_2 = 'value2' NOT OVERRIDABLE, -url = 'https://connection.url/' -``` - -上記の例では次のようになります。 - -* `key_1` は常に上書きできます。 -* `key_2` は上書きすることはできません。 -* `url` は、`allow_named_collection_override_by_default` の値に応じて上書きできる場合とできない場合があります。 - -### DDL で名前付きコレクションを作成するための権限 {#permissions-to-create-named-collections-with-ddl} - -DDL で名前付きコレクションを管理するには、ユーザーは `named_collection_control` 権限を持っている必要があります。これは `/etc/clickhouse-server/users.d/` にファイルを追加することで付与できます。次の例では、ユーザー `default` に `access_management` と `named_collection_control` の両方の権限を付与しています。 - -```xml title='/etc/clickhouse-server/users.d/user_default.xml' - - - - 65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5 - 1 - - 1 - - - - -``` - -:::tip -上記の例では、`password_sha256_hex` の値は、パスワードの SHA256 ハッシュを 16 進数で表現したものです。ユーザー `default` 向けのこの設定では、デフォルト設定で平文の `password` が設定されているため、属性 `replace=true` を指定しています。同一ユーザーに対して、平文パスワードと SHA256 の 16 進数パスワードを同時に設定することはできません。 -::: - -### 名前付きコレクションのストレージ {#storage-for-named-collections} - -名前付きコレクションはローカルディスクまたは ZooKeeper/Keeper に保存できます。デフォルトではローカルストレージが使用されます。 -また、[ディスク暗号化](storing-data#encrypted-virtual-file-system) と同じアルゴリズムを使用して暗号化して保存することもでき、その際にはデフォルトで `aes_128_ctr` が使用されます。 - -名前付きコレクションのストレージを構成するには、`type` を指定する必要があります。これは `local` または `keeper`/`zookeeper` のいずれかです。暗号化ストレージの場合は、 -`local_encrypted` または `keeper_encrypted`/`zookeeper_encrypted` を使用できます。 - -ZooKeeper/Keeper を使用するには、構成ファイルの `named_collections_storage` セクションに `path`(名前付きコレクションを保存する ZooKeeper/Keeper 上のパス)も設定する必要があります。次の例は、暗号化と ZooKeeper/Keeper を併用しています。 - -```xml - - - zookeeper_encrypted - bebec0cabebec0cabebec0cabebec0ca - aes_128_ctr - /named_collections_path/ - 1000 - - -``` - -オプションの設定パラメーター `update_timeout_ms` のデフォルト値は `5000` です。 - -## 設定ファイルに名前付きコレクションを保存する {#storing-named-collections-in-configuration-files} - -### XML の例 {#xml-example} - -```xml title='/etc/clickhouse-server/config.d/named_collections.xml' - - - - value - value_2 - https://connection.url/ - - - -``` - -上記の例では: - -* `key_1` は常に上書きできます。 -* `key_2` は上書きすることはできません。 -* `url` は、`allow_named_collection_override_by_default` の値に応じて、上書きできる場合とできない場合があります。 - -## 名前付きコレクションの変更 {#modifying-named-collections} - -DDL クエリで作成された名前付きコレクションは、DDL によって変更または削除できます。XML ファイルで作成された名前付きコレクションは、対応する XML を編集または削除することで管理できます。 - -### DDL で作成された名前付きコレクションを変更する {#alter-a-ddl-named-collection} - -コレクション `collection2` のキー `key1` と `key3` を変更または追加します -(この操作では、それらのキーに対する `overridable` フラグの値は変更されません): - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, key3='value3' -``` - -キー `key1` を変更または追加し、常に上書き可能にします。 - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4 OVERRIDABLE -``` - -`collection2` からキー `key2` を削除します: - -```sql -ALTER NAMED COLLECTION collection2 DELETE key2 -``` - -コレクション `collection2` のキー `key1` を追加または変更し、キー `key3` を削除します。 - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, DELETE key3 -``` - -キーに対して `overridable` フラグのデフォルト設定を強制的に使用させるには、 -そのキーを一度削除してから再度追加する必要があります。 - -```sql -ALTER NAMED COLLECTION collection2 DELETE key1; -ALTER NAMED COLLECTION collection2 SET key1=4; -``` - -### DDL の名前付きコレクション `collection2` を削除: {#drop-the-ddl-named-collection-collection2} - -```sql -DROP NAMED COLLECTION collection2 -``` - -## S3 にアクセスするための名前付きコレクション {#named-collections-for-accessing-s3} - -パラメータの説明については、[s3 テーブル関数](../sql-reference/table-functions/s3.md)を参照してください。 - -### DDL の例 {#ddl-example-1} - -```sql -CREATE NAMED COLLECTION s3_mydata AS -access_key_id = 'AKIAIOSFODNN7EXAMPLE', -secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', -format = 'CSV', -url = 'https://s3.us-east-1.amazonaws.com/yourbucket/mydata/' -``` - -### XML の例 {#xml-example-1} - -```xml - - - - AKIAIOSFODNN7EXAMPLE - wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY - CSV - https://s3.us-east-1.amazonaws.com/yourbucket/mydata/ - - - -``` - -### s3() 関数と S3 テーブルの名前付きコレクションの例 {#s3-function-and-s3-table-named-collection-examples} - -次の 2 つの例では、同じ名前付きコレクション `s3_mydata` を使用します。 - -#### s3() 関数 {#s3-function} - -```sql -INSERT INTO FUNCTION s3(s3_mydata, filename = 'test_file.tsv.gz', - format = 'TSV', structure = 'number UInt64', compression_method = 'gzip') -SELECT * FROM numbers(10000); -``` - -:::tip -上記の `s3()` 関数の最初の引数には、コレクション名 `s3_mydata` を指定しています。名前付きコレクションを使用しない場合は、`s3()` 関数を呼び出すたびにアクセスキー ID、シークレット、フォーマット、URL をすべて渡す必要があります。 -::: - -#### S3 テーブル {#s3-table} - -```sql -CREATE TABLE s3_engine_table (number Int64) -ENGINE=S3(s3_mydata, url='https://s3.us-east-1.amazonaws.com/yourbucket/mydata/test_file.tsv.gz', format = 'TSV') -SETTINGS input_format_with_names_use_header = 0; - -SELECT * FROM s3_engine_table LIMIT 3; -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -└────────┘ -``` - -## MySQL データベースにアクセスするための名前付きコレクション {#named-collections-for-accessing-mysql-database} - -パラメータの説明については、[mysql](../sql-reference/table-functions/mysql.md) を参照してください。 - -### DDL の例 {#ddl-example-2} - -```sql -CREATE NAMED COLLECTION mymysql AS -user = 'myuser', -password = 'mypass', -host = '127.0.0.1', -port = 3306, -database = 'test', -connection_pool_size = 8, -replace_query = 1 -``` - -### XML の例 {#xml-example-2} - -```xml - - - - myuser - mypass - 127.0.0.1 - 3306 - test - 8 - 1 - - - -``` - -### mysql() 関数、MySQL テーブル、MySQL データベース、および Dictionary 名前付きコレクションの例 {#mysql-function-mysql-table-mysql-database-and-dictionary-named-collection-examples} - -以下の 4 つの例では、同じ名前付きコレクション `mymysql` を使用します。 - -#### mysql() 関数 {#mysql-function} - -```sql -SELECT count() FROM mysql(mymysql, table = 'test'); - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -この名前付きコレクションでは `table` パラメータが指定されていないため、関数呼び出し時に `table = 'test'` として指定します。 -::: - -#### MySQL テーブル {#mysql-table} - -```sql -CREATE TABLE mytable(A Int64) ENGINE = MySQL(mymysql, table = 'test', connection_pool_size=3, replace_query=0); -SELECT count() FROM mytable; - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -この DDL ステートメントは、`connection_pool_size` に対する名前付きコレクションの設定を上書きします。 -::: - -#### MySQL データベース {#mysql-database} - -```sql -CREATE DATABASE mydatabase ENGINE = MySQL(mymysql); - -SHOW TABLES FROM mydatabase; - -┌─name───┐ -│ source │ -│ test │ -└────────┘ -``` - -#### MySQL 辞書 {#mysql-dictionary} - -```sql -CREATE DICTIONARY dict (A Int64, B String) -PRIMARY KEY A -SOURCE(MYSQL(NAME mymysql TABLE 'source')) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'B', 2); - -┌─dictGet('dict', 'B', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## PostgreSQL データベースへのアクセス用名前付きコレクション {#named-collections-for-accessing-postgresql-database} - -パラメータの説明については [postgresql](../sql-reference/table-functions/postgresql.md) を参照してください。さらに、次のエイリアスがあります: - -* `user` のエイリアス: `username` -* `database` のエイリアス: `db` - -パラメータ `addresses_expr` は、コレクション内で `host:port` の代わりに使用されます。`host`、`hostname`、`port` といった他のパラメータが任意指定であるため、このパラメータも必須ではありません。以下の擬似コードは、その優先順位を説明しています: - -```sql -CASE - WHEN collection['addresses_expr'] != '' THEN collection['addresses_expr'] - WHEN collection['host'] != '' THEN collection['host'] || ':' || if(collection['port'] != '', collection['port'], '5432') - WHEN collection['hostname'] != '' THEN collection['hostname'] || ':' || if(collection['port'] != '', collection['port'], '5432') -END -``` - -作成例: - -```sql -CREATE NAMED COLLECTION mypg AS -user = 'pguser', -password = 'jw8s0F4', -host = '127.0.0.1', -port = 5432, -database = 'test', -schema = 'test_schema' -``` - -設定例: - -```xml - - - - pguser - jw8s0F4 - 127.0.0.1 - 5432 - test - test_schema - - - -``` - -### PostgreSQL 関数で名前付きコレクションを使用する例 {#example-of-using-named-collections-with-the-postgresql-function} - -```sql -SELECT * FROM postgresql(mypg, table = 'test'); - -┌─a─┬─b───┐ -│ 2 │ two │ -│ 1 │ one │ -└───┴─────┘ -SELECT * FROM postgresql(mypg, table = 'test', schema = 'public'); - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -### PostgreSQL エンジンを使用するデータベースで名前付きコレクションを利用する例 {#example-of-using-named-collections-with-database-with-engine-postgresql} - -```sql -CREATE TABLE mypgtable (a Int64) ENGINE = PostgreSQL(mypg, table = 'test', schema = 'public'); - -SELECT * FROM mypgtable; - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -:::note -PostgreSQL は、テーブル作成時に名前付きコレクションからデータをコピーします。コレクションが変更されても、既存のテーブルには影響しません。 -::: - -### PostgreSQL エンジンを使用するデータベースで名前付きコレクションを使用する例 {#example-of-using-named-collections-with-database-with-engine-postgresql-1} - -```sql -CREATE DATABASE mydatabase ENGINE = PostgreSQL(mypg); - -SHOW TABLES FROM mydatabase - -┌─name─┐ -│ test │ -└──────┘ -``` - -### ソースに POSTGRESQL を使用する辞書での名前付きコレクションの使用例 {#example-of-using-named-collections-with-a-dictionary-with-source-postgresql} - -```sql -CREATE DICTIONARY dict (a Int64, b String) -PRIMARY KEY a -SOURCE(POSTGRESQL(NAME mypg TABLE test)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## リモート ClickHouse データベースにアクセスするための名前付きコレクション {#named-collections-for-accessing-a-remote-clickhouse-database} - -パラメータの説明については、[remote](../sql-reference/table-functions/remote.md/#parameters) を参照してください。 - -設定例: - -```sql -CREATE NAMED COLLECTION remote1 AS -host = 'remote_host', -port = 9000, -database = 'system', -user = 'foo', -password = 'secret', -secure = 1 -``` - -```xml - - - - remote_host - 9000 - system - foo - secret - 1 - - - -``` - -接続では `remoteSecure` を使用するため `secure` は不要ですが、ディクショナリには使用できます。 - -### `remote` / `remoteSecure` 関数で名前付きコレクションを使用する例 {#example-of-using-named-collections-with-the-remoteremotesecure-functions} - -```sql -SELECT * FROM remote(remote1, table = one); -┌─dummy─┐ -│ 0 │ -└───────┘ - -SELECT * FROM remote(remote1, database = merge(system, '^one')); -┌─dummy─┐ -│ 0 │ -└───────┘ - -INSERT INTO FUNCTION remote(remote1, database = default, table = test) VALUES (1,'a'); - -SELECT * FROM remote(remote1, database = default, table = test); -┌─a─┬─b─┐ -│ 1 │ a │ -└───┴───┘ -``` - -### ClickHouse をソースとする辞書での名前付きコレクションの使用例 {#example-of-using-named-collections-with-a-dictionary-with-source-clickhouse} - -```sql -CREATE DICTIONARY dict(a Int64, b String) -PRIMARY KEY a -SOURCE(CLICKHOUSE(NAME remote1 TABLE test DB default)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 1); -┌─dictGet('dict', 'b', 1)─┐ -│ a │ -└─────────────────────────┘ -``` - -## Kafka へのアクセスに使用する名前付きコレクション {#named-collections-for-accessing-kafka} - -パラメータの説明については [Kafka](../engines/table-engines/integrations/kafka.md) を参照してください。 - -### DDL の例 {#ddl-example-3} - -```sql -CREATE NAMED COLLECTION my_kafka_cluster AS -kafka_broker_list = 'localhost:9092', -kafka_topic_list = 'kafka_topic', -kafka_group_name = 'consumer_group', -kafka_format = 'JSONEachRow', -kafka_max_block_size = '1048576'; - -``` - -### XML の例 {#xml-example-3} - -```xml - - - - localhost:9092 - kafka_topic - consumer_group - JSONEachRow - 1048576 - - - -``` - -### Kafka テーブルで名前付きコレクションを使用する例 {#example-of-using-named-collections-with-a-kafka-table} - -次の 2 つの例では、いずれも同じ名前付きコレクション `my_kafka_cluster` を使用します。 - -```sql -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) - -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) -SETTINGS kafka_num_consumers = 4, - kafka_thread_per_consumer = 1; -``` - -## バックアップ用の名前付きコレクション {#named-collections-for-backups} - -パラメータの説明については[バックアップとリストア](./backup.md)を参照してください。 - -### DDL の例 {#ddl-example-4} - -```sql -BACKUP TABLE default.test to S3(named_collection_s3_backups, 'directory') -``` - -### XML の例 {#xml-example-4} - -```xml - - - - https://my-s3-bucket.s3.amazonaws.com/backup-S3/ - ABC123 - Abc+123 - - - -``` - -## MongoDB テーブルおよび辞書にアクセスするための名前付きコレクション {#named-collections-for-accessing-mongodb-table-and-dictionary} - -パラメータの説明については [mongodb](../sql-reference/table-functions/mongodb.md) を参照してください。 - -### DDL の例 {#ddl-example-5} - -```sql -CREATE NAMED COLLECTION mymongo AS -user = '', -password = '', -host = '127.0.0.1', -port = 27017, -database = 'test', -collection = 'my_collection', -options = 'connectTimeoutMS=10000' -``` - -### XML の例 {#xml-example-5} - -```xml - - - - - - 127.0.0.1 - 27017 - test - my_collection - connectTimeoutMS=10000 - - - -``` - -#### MongoDB テーブル {#mongodb-table} - -```sql -CREATE TABLE mytable(log_type VARCHAR, host VARCHAR, command VARCHAR) ENGINE = MongoDB(mymongo, options='connectTimeoutMS=10000&compressors=zstd') -SELECT count() FROM mytable; - -┌─count()─┐ -│ 2 │ -└─────────┘ -``` - -:::note -DDL は、オプションで指定した名前付きコレクションの設定を上書きします。 -::: - -#### MongoDB 辞書 {#mongodb-dictionary} - -```sql -CREATE DICTIONARY dict -( - `a` Int64, - `b` String -) -PRIMARY KEY a -SOURCE(MONGODB(NAME mymongo COLLECTION my_dict)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()) - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -:::note -名前付きコレクションでは、コレクション名として `my_collection` を指定しています。関数呼び出しでは、別のコレクションを選択するために `collection = 'my_dict'` を指定してこれを上書きします。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/opentelemetry.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/opentelemetry.md deleted file mode 100644 index 77eb3eb8231..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/opentelemetry.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'ClickHouse における OpenTelemetry を用いた分散トレーシングおよびメトリクス収集のためのガイド' -sidebar_label: 'OpenTelemetry による ClickHouse のトレーシング' -sidebar_position: 62 -slug: /operations/opentelemetry -title: 'OpenTelemetry による ClickHouse のトレーシング' -doc_type: 'guide' ---- - -[OpenTelemetry](https://opentelemetry.io/) は、分散アプリケーションからトレースとメトリクスを収集するためのオープンな標準仕様です。ClickHouse は OpenTelemetry を一部サポートしています。 - - - -## ClickHouse へのトレースコンテキストの指定 {#supplying-trace-context-to-clickhouse} - -ClickHouse は、[W3C 勧告](https://www.w3.org/TR/trace-context/) で説明されているトレースコンテキスト HTTP ヘッダーを受け付けます。また、ClickHouse サーバー間やクライアントとサーバー間の通信に使用されるネイティブプロトコルを介して渡されるトレースコンテキストも受け付けます。手動でテストする場合、Trace Context 勧告に準拠したトレースコンテキストヘッダーは、`clickhouse-client` に対して `--opentelemetry-traceparent` および `--opentelemetry-tracestate` フラグを使用して指定できます。 - -親トレースコンテキストが指定されていない場合、または指定されたトレースコンテキストが上記の W3C 標準に準拠していない場合、ClickHouse は新しいトレースを開始します。その発生確率は、[opentelemetry_start_trace_probability](/operations/settings/settings#opentelemetry_start_trace_probability) 設定で制御されます。 - - - -## トレースコンテキストの伝播 {#propagating-the-trace-context} - -トレースコンテキストは、次の場合に下流のサービスへ伝播されます。 - -* [Distributed](../engines/table-engines/special/distributed.md) テーブルエンジンを使用する場合など、リモートの ClickHouse サーバーに対してクエリを実行する場合。 - -* [url](../sql-reference/table-functions/url.md) テーブル関数を使用する場合。この場合、トレースコンテキスト情報は HTTP ヘッダーで送信されます。 - - - -## ClickHouse 自体のトレース {#tracing-the-clickhouse-itself} - -ClickHouse は、各クエリおよびクエリ計画や分散クエリなど一部のクエリ実行ステージごとに `trace spans` を作成します。 - -トレース情報を有用にするには、[Jaeger](https://jaegertracing.io/) や [Prometheus](https://prometheus.io/) など、OpenTelemetry をサポートするモニタリングシステムにエクスポートする必要があります。ClickHouse は特定のモニタリングシステムへの依存を避け、代わりにシステムテーブルを通じてのみトレースデータを提供します。標準で[要求されている](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/overview.md#span) OpenTelemetry のトレーススパン情報は、[system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md) テーブルに保存されます。 - -このテーブルを利用するには、サーバー設定で有効化されている必要があります。デフォルトの設定ファイル `config.xml` 内の `opentelemetry_span_log` 要素を参照してください。デフォルトでは有効になっています。 - -タグまたは属性は、キーと値を含む 2 つの並列配列として保存されます。これらを扱うには [ARRAY JOIN](../sql-reference/statements/select/array-join.md) を使用します。 - - - -## Log-query-settings {#log-query-settings} - -[log_query_settings](settings/settings.md) 設定を有効にすると、クエリの実行中に行われたクエリ設定の変更内容をログとして記録できるようになります。有効化されている場合、クエリ設定に対して行われたあらゆる変更は OpenTelemetry のスパンログに記録されます。この機能は、本番環境においてクエリ性能に影響を与えうる設定変更を追跡するのに特に有用です。 - - - -## 監視システムとの統合 {#integration-with-monitoring-systems} - -現時点では、ClickHouse から監視システムへトレースデータをエクスポートするための既製ツールは用意されていません。 - -テスト用途として、[system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md) テーブルに対して [URL](../engines/table-engines/special/url.md) エンジンを使用したマテリアライズドビューをセットアップすることで、受信したログデータをトレースコレクターの HTTP エンドポイントに送信できます。例えば、`http://localhost:9411` で稼働している Zipkin インスタンスに、Zipkin v2 の JSON フォーマットで最小限の span データを送信するには、次のようにします。 - -```sql -CREATE MATERIALIZED VIEW default.zipkin_spans -ENGINE = URL('http://127.0.0.1:9411/api/v2/spans', 'JSONEachRow') -SETTINGS output_format_json_named_tuples_as_objects = 1, - output_format_json_array_of_rows = 1 AS -SELECT - lower(hex(trace_id)) AS traceId, - CASE WHEN parent_span_id = 0 THEN '' ELSE lower(hex(parent_span_id)) END AS parentId, - lower(hex(span_id)) AS id, - operation_name AS name, - start_time_us AS timestamp, - finish_time_us - start_time_us AS duration, - cast(tuple('clickhouse'), 'Tuple(serviceName text)') AS localEndpoint, - cast(tuple( - attribute.values[indexOf(attribute.names, 'db.statement')]), - 'Tuple("db.statement" text)') AS tags -FROM system.opentelemetry_span_log -``` - -エラーが発生した場合、そのエラーが発生した対象のログデータの一部は、エラーを通知することなく失われます。データが届かない場合は、サーバーログでエラーメッセージを確認してください。 - - -## 関連コンテンツ {#related-content} - -- ブログ: [ClickHouse を用いたオブザーバビリティソリューションの構築 - パート 2: トレース](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md deleted file mode 100644 index 2bd995f1b62..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'プロファイルガイド最適化のドキュメント' -sidebar_label: 'プロファイルガイド最適化 (PGO)' -sidebar_position: 54 -slug: /operations/optimizing-performance/profile-guided-optimization -title: 'プロファイルガイド最適化' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# プロファイルガイド最適化 {#profile-guided-optimization} - -Profile-Guided Optimization (PGO) は、実行時プロファイルに基づいてプログラムを最適化するコンパイラ最適化手法です。 - -テストの結果、PGO は ClickHouse のパフォーマンス向上に有効であることが分かっています。ClickBench テストスイートでは、QPS が最大 15% 向上することが確認されています。より詳細な結果は[こちら](https://pastebin.com/xbue3HMU)で確認できます。パフォーマンス向上の度合いは、典型的なワークロードに依存し、より良い結果が得られる場合もあれば、そうでない場合もあります。 - -ClickHouse における PGO の詳細については、該当する GitHub の[issue](https://github.com/ClickHouse/ClickHouse/issues/44567)を参照してください。 - -## PGO を使って ClickHouse をビルドする方法 {#how-to-build-clickhouse-with-pgo} - -PGO には大きく分けて 2 種類あります。[Instrumentation](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers) と [Sampling](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers)(AutoFDO とも呼ばれます)です。このガイドでは、ClickHouse における Instrumentation PGO について説明します。 - -1. Instrumentation モードで ClickHouse をビルドします。Clang では `CXXFLAGS` に `-fprofile-generate` オプションを渡すことで行えます。 -2. サンプルのワークロードで Instrumentation モードの ClickHouse を実行します。ここでは、通常運用しているワークロードを使用する必要があります。1 つの方法として、サンプルワークロードとして [ClickBench](https://github.com/ClickHouse/ClickBench) を使用できます。Instrumentation モードの ClickHouse は低速に動作する可能性があるため、その点を理解したうえで使用し、性能が重要な環境では Instrumentation モードの ClickHouse を実行しないでください。 -3. 前のステップで収集したプロファイルを使用し、コンパイラフラグ `-fprofile-use` を付与して ClickHouse を再コンパイルします。 - -PGO の適用方法についての、より詳細なガイドは Clang の[ドキュメント](https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization)に記載されています。 - -本番環境から直接サンプルワークロードを収集する場合は、Sampling PGO の利用を検討することをお勧めします。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md deleted file mode 100644 index b7e6b68bb79..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -description: 'ClickHouse のサンプリングクエリプロファイラーツールに関するドキュメント' -sidebar_label: 'クエリプロファイリング' -sidebar_position: 54 -slug: /operations/optimizing-performance/sampling-query-profiler -title: 'サンプリングクエリプロファイラー' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# サンプリングクエリプロファイラ {#sampling-query-profiler} - -ClickHouse にはクエリ実行を解析できるサンプリングプロファイラが搭載されています。プロファイラを使用すると、クエリ実行中に最も頻繁に使用されたソースコード内のルーチンを特定できます。CPU 時間と、アイドル時間を含むウォールクロック時間(実時間)を追跡できます。 - -クエリプロファイラは ClickHouse Cloud では自動的に有効化されており、次のようにサンプルクエリを実行できます。 - -:::note ClickHouse Cloud で以下のクエリを実行する場合は、クラスタ内のすべてのノードから取得するために、`FROM system.trace_log` を `FROM clusterAllReplicas(default, system.trace_log)` に変更してください。 -::: - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c' AND trace_type = 'CPU' AND event_date = today() -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -SETTINGS allow_introspection_functions = 1 -``` - -自己管理型デプロイメントでクエリプロファイラを使用するには、次の手順を実行します。 - -* サーバー設定の [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log) セクションを設定します。 - - このセクションでは、プロファイラの動作結果を格納する [trace_log](/operations/system-tables/trace_log) システムテーブルを構成します。デフォルトで有効になっています。このテーブル内のデータは、稼働中のサーバーに対してのみ有効であることに注意してください。サーバー再起動後、ClickHouse はこのテーブルをクリーンアップせず、保存されている仮想メモリアドレスはすべて無効になる可能性があります。 - -* [query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns) または [query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns) 設定を構成します。両方の設定を同時に使用できます。 - - これらの設定により、プロファイラのタイマーを構成できます。これはセッション設定であるため、サーバー全体、個々のユーザーやユーザープロファイル、対話型セッション、さらに各個別クエリごとに異なるサンプリング頻度を設定できます。 - -デフォルトのサンプリング頻度は 1 秒あたり 1 サンプルで、CPU タイマーと実時間タイマーの両方が有効になっています。この頻度により、ClickHouse クラスターに関する十分な情報を収集できます。同時に、この頻度で動作してもプロファイラは ClickHouse サーバーのパフォーマンスに影響を与えません。各個別クエリを詳細にプロファイルする必要がある場合は、より高いサンプリング頻度の使用を検討してください。 - -`trace_log` システムテーブルを分析するには、次の手順を実行します。 - -* `clickhouse-common-static-dbg` パッケージをインストールします。詳細は [Install from DEB Packages](../../getting-started/install/install.mdx) を参照してください。 - -* [allow_introspection_functions](../../operations/settings/settings.md#allow_introspection_functions) 設定で introspection 関数を許可します。 - - セキュリティ上の理由から、introspection 関数はデフォルトで無効になっています。 - -* `addressToLine`、`addressToLineWithInlines`、`addressToSymbol`、`demangle` などの [introspection 関数](../../sql-reference/functions/introspection.md) を使用して、ClickHouse コード内の関数名およびその位置を取得します。特定のクエリについてプロファイルを取得するには、`trace_log` テーブルからデータを集約する必要があります。個々の関数単位、またはスタックトレース全体単位でデータを集約できます。 - -`trace_log` の情報を可視化する必要がある場合は、[flamegraph](/interfaces/third-party/gui#clickhouse-flamegraph) や [speedscope](https://github.com/laplab/clickhouse-speedscope) の使用を検討してください。 - -## 例 {#example} - -この例では次のことを行います。 - -* クエリ識別子と現在の日付で `trace_log` データをフィルタリングします。 -* スタックトレース単位で集計します。 -* イントロスペクション用関数を使用して、次の内容を含むレポートを取得します。 - * シンボル名と、それに対応するソースコード上の関数名 - * これらの関数が記述されているソースコード上の位置 - -{/* */ } - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE (query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c') AND (event_date = today()) -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/performance-test.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/performance-test.md deleted file mode 100644 index 4076158f97b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/performance-test.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: 'ClickHouse を用いたハードウェア性能テストとベンチマークのガイド' -sidebar_label: 'ハードウェアテスト' -sidebar_position: 54 -slug: /operations/performance-test -title: 'ClickHouse でハードウェア性能をテストする方法' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -ClickHouse パッケージをインストールしなくても、どのサーバー上でも ClickHouse の基本的な性能テストを実行できます。 - -## 自動実行 {#automated-run} - -ベンチマークは 1 つのスクリプトだけで実行できます。 - -1. スクリプトをダウンロードします。 - -```bash -wget https://raw.githubusercontent.com/ClickHouse/ClickBench/main/hardware/hardware.sh -``` - -2. スクリプトを実行します。 - -```bash -chmod a+x ./hardware.sh -./hardware.sh -``` - -3. 出力結果をコピーして [feedback@clickhouse.com](mailto:feedback@clickhouse.com) 宛に送信してください - -すべての結果は以下で公開されています: [https://clickhouse.com/benchmark/hardware/](https://clickhouse.com/benchmark/hardware/) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-cache.md deleted file mode 100644 index 3819a9b8a80..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-cache.md +++ /dev/null @@ -1,190 +0,0 @@ ---- -description: 'ClickHouse のクエリキャッシュ機能の利用と設定に関するガイド' -sidebar_label: 'クエリキャッシュ' -sidebar_position: 65 -slug: /operations/query-cache -title: 'クエリキャッシュ' -doc_type: 'guide' ---- - -# クエリキャッシュ {#query-cache} - -クエリキャッシュを使用すると、`SELECT` クエリを一度だけ実行して結果を保存し、同じクエリの後続の実行にはキャッシュから直接結果を返すことができます。 -クエリの種類によっては、これにより ClickHouse サーバーのレイテンシとリソース消費を劇的に削減できます。 - -## 背景、設計、および制限事項 {#background-design-and-limitations} - -クエリキャッシュは一般的に、トランザクション一貫性があるものと、ないものに分類できます。 - -* トランザクション一貫性のあるキャッシュでは、`SELECT` クエリの結果が変化した場合、または変化する可能性がある場合に、データベースがキャッシュ済みのクエリ結果を無効化(破棄)します。ClickHouse においてデータを変更する操作には、テーブルへの挿入・更新・削除や、テーブルからの削除、あるいは collapsing merge が含まれます。トランザクション一貫性のあるキャッシュは、特に OLTP データベースに適しており、たとえば - [MySQL](https://dev.mysql.com/doc/refman/5.6/en/query-cache.html)(v8.0 でクエリキャッシュは削除)や - [Oracle](https://docs.oracle.com/database/121/TGDBA/tune_result_cache.htm) などがあります。 -* トランザクション一貫性のないキャッシュでは、すべてのキャッシュエントリに有効期間(例: 1 分)を割り当て、その期間経過後に期限切れになること、そしてその期間内には基盤となるデータの変更がごくわずかにとどまることを前提として、クエリ結果がわずかに不正確になることを許容します。この手法は、全体として OLAP データベースにより適しています。トランザクション一貫性のないキャッシュで十分な例としては、複数ユーザーが同時にアクセスするレポーティングツールにおける毎時の売上レポートが挙げられます。売上データの変化は通常十分に遅いため、データベースはレポートを一度だけ計算すればよく(最初の `SELECT` クエリ)、以降のクエリはすべてクエリキャッシュから直接提供できます。この例では、妥当な有効期間は 30 分程度と考えられます。 - -トランザクション一貫性のないキャッシュは、従来はデータベースと連携するクライアントツールやプロキシパッケージ(例: -[chproxy](https://www.chproxy.org/configuration/caching/))によって提供されてきました。その結果、同じキャッシュロジックや設定が重複しがちです。ClickHouse のクエリキャッシュでは、キャッシュロジックがサーバー側に移動します。これにより、保守作業が軽減され、冗長性を回避できます。 - -## 設定と使用方法 {#configuration-settings-and-usage} - -:::note -ClickHouse Cloud では、クエリキャッシュ設定を編集するには [クエリレベルの設定](/operations/settings/query-level) を使用する必要があります。[コンフィグレベルの設定](/operations/configuration-files) の編集は現在サポートされていません。 -::: - -:::note -[clickhouse-local](utilities/clickhouse-local.md) は一度に 1 つのクエリのみを実行します。クエリ結果キャッシュを利用する意味がないため、clickhouse-local ではクエリ結果キャッシュは無効になっています。 -::: - -[use_query_cache](/operations/settings/settings#use_query_cache) 設定を使用すると、特定のクエリ、または現在のセッション内のすべてのクエリでクエリキャッシュを利用するかどうかを制御できます。例えば、あるクエリを最初に実行したときは - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true; -``` - -クエリ結果はクエリキャッシュに保存されます。その後に同じクエリを実行した場合(パラメータ `use_query_cache = true` を指定した場合も同様)、計算済みの結果をキャッシュから読み取り、即座に返します。 - -:::note -`use_query_cache` およびその他すべてのクエリキャッシュ関連設定は、スタンドアロンの `SELECT` 文に対してのみ効果があります。とくに、 -`CREATE VIEW AS SELECT [...] SETTINGS use_query_cache = true` で作成されたビューに対する `SELECT` の結果は、その `SELECT` -文が `SETTINGS use_query_cache = true` を指定して実行されない限りキャッシュされません。 -::: - -キャッシュの利用方法は、設定 [enable_writes_to_query_cache](/operations/settings/settings#enable_writes_to_query_cache) -および [enable_reads_from_query_cache](/operations/settings/settings#enable_reads_from_query_cache)(どちらもデフォルトは `true`)を使って、より細かく設定できます。前者の設定はクエリ結果をキャッシュに保存するかどうかを制御し、後者の設定はデータベースがキャッシュからクエリ結果を取得しようとするかどうかを決定します。たとえば、次のクエリはキャッシュを受動的にのみ使用します。つまり、キャッシュからの読み取りは試みますが、自身の結果をキャッシュに保存しません。 - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, enable_writes_to_query_cache = false; -``` - -最大限の制御を行うため、一般的には `use_query_cache`、`enable_writes_to_query_cache`、`enable_reads_from_query_cache` の設定は特定のクエリに対してのみ指定することを推奨します。ユーザーまたはプロファイル単位(例:`SET -use_query_cache = true`)でキャッシュを有効化することも可能ですが、その場合、すべての `SELECT` クエリがキャッシュされた結果を返すようになることに注意してください。 - -クエリキャッシュは `SYSTEM DROP QUERY CACHE` ステートメントを使用してクリアできます。クエリキャッシュの内容はシステムテーブル -[system.query_cache](system-tables/query_cache.md) に表示されます。データベース起動以降のクエリキャッシュのヒット数とミス数は、システムテーブル [system.events](system-tables/events.md) 内のイベント -"QueryCacheHits" および "QueryCacheMisses" として表示されます。両方のカウンタは、設定 `use_query_cache = true` で実行される `SELECT` クエリに対してのみ更新され、それ以外のクエリは "QueryCacheMisses" には影響しません。システムテーブル [system.query_log](system-tables/query_log.md) 内のフィールド `query_cache_usage` は、各実行クエリについて、その結果がクエリキャッシュに書き込まれたか、あるいはクエリキャッシュから読み出されたかを示します。システムテーブル -[system.metrics](system-tables/metrics.md) 内のメトリクス `QueryCacheEntries` および `QueryCacheBytes` は、現在クエリキャッシュに含まれるエントリ数およびバイト数を示します。 - -クエリキャッシュは ClickHouse サーバープロセスごとに 1 つ存在します。ただし、キャッシュ結果はデフォルトではユーザー間で共有されません。これは(後述のように)変更可能ですが、セキュリティ上の理由から推奨されません。 - -クエリ結果は、そのクエリの [抽象構文木 (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) によってクエリキャッシュ内で参照されます。これは、キャッシュが大文字・小文字を区別しないことを意味し、たとえば `SELECT 1` と `select 1` は同一のクエリとして扱われます。より自然なマッチングを行うため、クエリキャッシュおよび[出力フォーマット](settings/settings-formats.md)に関連するクエリレベルのすべての設定は AST から削除されます。 - -クエリが例外またはユーザーによるキャンセルによって中断された場合、そのエントリはクエリキャッシュに書き込まれません。 - -クエリキャッシュのサイズ(バイト数)、キャッシュエントリの最大数、および個々のキャッシュエントリの最大サイズ(バイト数およびレコード数)は、さまざまな[サーバー構成オプション](/operations/server-configuration-parameters/settings#query_cache)を使用して設定できます。 - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - -[settings profiles](settings/settings-profiles.md) と [settings -constraints](settings/constraints-on-settings.md) を使用して、個々のユーザーのキャッシュ利用量を制限することもできます。具体的には、ユーザーがクエリキャッシュ内で割り当て可能なメモリの最大量(バイト単位)と、保存できるクエリ結果の最大数を制限できます。そのためには、まず `users.xml` 内のユーザープロファイルで -[query_cache_max_size_in_bytes](/operations/settings/settings#query_cache_max_size_in_bytes) と -[query_cache_max_entries](/operations/settings/settings#query_cache_max_entries) を設定し、その後両方の設定を読み取り専用(readonly)にします。 - -```xml - - - - 10000 - - 100 - - - - - - - - - - - -``` - -クエリ結果をキャッシュ対象とするための最小実行時間を定義するには、設定 -[query_cache_min_query_duration](/operations/settings/settings#query_cache_min_query_duration) を使用できます。たとえば、次のクエリの結果は - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, query_cache_min_query_duration = 5000; -``` - -クエリ結果は、クエリの実行時間が 5 秒より長い場合にのみキャッシュされます。クエリの結果がキャッシュされるまでに、そのクエリを何回実行する必要があるかも指定できます。その場合は設定 [query_cache_min_query_runs](/operations/settings/settings#query_cache_min_query_runs) を使用します。 - -クエリキャッシュ内のエントリは、一定時間が経過すると古くなります(time-to-live)。デフォルトではこの期間は 60 秒ですが、[query_cache_ttl](/operations/settings/settings#query_cache_ttl) 設定を使って、セッション、プロファイル、またはクエリ単位で別の値を指定できます。クエリキャッシュはエントリを「遅延的に」追い出します。つまり、エントリが古くなっても、キャッシュから即座には削除されません。代わりに、新しいエントリをクエリキャッシュに挿入しようとしたとき、データベースは新しいエントリ用の空き領域がキャッシュ内に十分あるかどうかを確認します。そうでない場合、データベースはすべての古いエントリを削除しようとします。それでもキャッシュに十分な空き領域がない場合は、新しいエントリは挿入されません。 - -クエリが HTTP 経由で実行される場合、ClickHouse はキャッシュされたエントリの経過時間(秒単位)と有効期限のタイムスタンプを示す `Age` および `Expires` ヘッダーを設定します。 - -クエリキャッシュ内のエントリは、デフォルトで圧縮されます。これは、クエリキャッシュへの書き込み/読み取りが遅くなる代わりに、全体的なメモリ使用量を削減するためです。圧縮を無効にするには、設定 [query_cache_compress_entries](/operations/settings/settings#query_cache_compress_entries) を使用します。 - -同じクエリに対して複数の結果をキャッシュしておきたい場合があります。これは、クエリキャッシュエントリのラベル(またはネームスペース)の役割を果たす設定 [query_cache_tag](/operations/settings/settings#query_cache_tag) を使用することで実現できます。クエリキャッシュは、同じクエリでもタグが異なれば、結果を別物として扱います。 - -同じクエリに対して 3 つの異なるクエリキャッシュエントリを作成する例: - -```sql -SELECT 1 SETTINGS use_query_cache = true; -- query_cache_tag は暗黙的に ''(空文字列)になります -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 1'; -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 2'; -``` - -クエリキャッシュからタグ `tag` が付いたエントリのみを削除するには、`SYSTEM DROP QUERY CACHE TAG 'tag'` 文を使用します。 - -ClickHouse は、テーブルデータを [max_block_size](/operations/settings/settings#max_block_size) 行のブロック単位で読み取ります。フィルタリングや集約などにより、 -結果ブロックは通常は `max_block_size` よりかなり小さくなりますが、逆にそれより大きくなる場合もあります。 -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results)(デフォルトで有効)を設定すると、 -クエリ結果キャッシュへの挿入前に、結果ブロックが(非常に小さい場合は)まとめられ、(大きい場合は)`max_block_size` の大きさのブロックに分割されるかどうかを制御できます。 -これによりクエリキャッシュへの書き込み性能は低下しますが、キャッシュエントリの圧縮率が向上し、 -後でクエリキャッシュからクエリ結果を提供する際に、より自然なブロック粒度が得られます。 - -その結果として、クエリキャッシュは各クエリに対して複数の(部分)結果ブロックを保存します。 -この挙動は有用なデフォルトですが、設定 -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results) によって無効化できます。 - -また、非決定的関数を含むクエリの結果は、デフォルトではキャッシュされません。これらの関数には次のようなものが含まれます。 - -* 辞書にアクセスする関数([`dictGet()`](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull) など) -* XML 定義内にタグ `true` を持たない - [ユーザー定義関数](../sql-reference/statements/create/function.md) -* 現在の日付または時刻を返す関数: - [`now()`](../sql-reference/functions/date-time-functions.md#now), - [`today()`](../sql-reference/functions/date-time-functions.md#today), - [`yesterday()`](../sql-reference/functions/date-time-functions.md#yesterday) など -* ランダムな値を返す関数: - [`randomString()`](../sql-reference/functions/random-functions.md#randomString), - [`fuzzBits()`](../sql-reference/functions/random-functions.md#fuzzBits) など -* クエリ処理に使用される内部チャンクのサイズと順序に応じて結果が変化する関数: - [`nowInBlock()`](../sql-reference/functions/date-time-functions.md#nowInBlock) など、 - [`rowNumberInBlock()`](../sql-reference/functions/other-functions.md#rowNumberInBlock), - [`runningDifference()`](../sql-reference/functions/other-functions.md#runningDifference), - [`blockSize()`](../sql-reference/functions/other-functions.md#blockSize) など -* 実行環境に依存する関数: - [`currentUser()`](../sql-reference/functions/other-functions.md#currentUser), - [`queryID()`](/sql-reference/functions/other-functions#queryID), - [`getMacro()`](../sql-reference/functions/other-functions.md#getMacro) など。 - -非決定的関数を含むクエリの結果であっても強制的にキャッシュしたい場合は、 -[query_cache_nondeterministic_function_handling](/operations/settings/settings#query_cache_nondeterministic_function_handling) を設定します。 - -system テーブル(例: [system.processes](system-tables/processes.md) や -[information_schema.tables](system-tables/information_schema.md))を含むクエリの結果は、デフォルトではキャッシュされません。 -system テーブルを含むクエリの結果を強制的にキャッシュしたい場合は、 -[query_cache_system_table_handling](/operations/settings/settings#query_cache_system_table_handling) を設定します。 - -最後に、セキュリティ上の理由から、クエリキャッシュ内のエントリはユーザー間で共有されません。たとえば、ユーザー A は、 -ユーザー B(そのようなポリシーが存在しないユーザー)と同じクエリを実行することで、テーブルに対する行ポリシーを回避できてはなりません。 -ただし、必要に応じて、設定 -[query_cache_share_between_users](/operations/settings/settings#query_cache_share_between_users) を指定することで、 -キャッシュエントリを他のユーザーからアクセス可能(すなわち共有)としてマークできます。 - -## 関連コンテンツ {#related-content} - -* ブログ記事: [Introducing the ClickHouse Query Cache](https://clickhouse.com/blog/introduction-to-the-clickhouse-query-cache-and-design) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md deleted file mode 100644 index e9d11898a1f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: 'ClickHouse のクエリ条件キャッシュ機能の利用と設定方法を解説するガイド' -sidebar_label: 'クエリ条件キャッシュ' -sidebar_position: 64 -slug: /operations/query-condition-cache -title: 'クエリ条件キャッシュ' -doc_type: 'guide' ---- - - - -# クエリ条件キャッシュ {#query-condition-cache} - -:::note -クエリ条件キャッシュは、[enable_analyzer](https://clickhouse.com/docs/operations/settings/settings#enable_analyzer) が true に設定されている場合にのみ動作します。これはデフォルト値です。 -::: - -多くの実運用ワークロードでは、同一またはほぼ同一のデータ(たとえば、既存データに新規データが追加されたもの)に対して、同じクエリが繰り返し実行されます。 -ClickHouse には、そのようなクエリパターンを最適化するためのさまざまな最適化手法があります。 -1 つの方法は、インデックス構造(例: プライマリキーインデックス、スキップインデックス、プロジェクション)や事前計算(マテリアライズドビュー)を用いて物理データ配置を調整することです。 -もう 1 つの方法は、ClickHouse の [query cache](query-cache.md) を使用して、クエリの再評価を回避することです。 -最初のアプローチの欠点は、データベース管理者による手動での調整と監視が必要になることです。 -2 つ目のアプローチは、古い結果を返す可能性があります(query cache はトランザクション整合性を保証しないため)が、これはユースケースによっては許容できない場合があります。 - -クエリ条件キャッシュは、これら 2 つの問題に対してエレガントな解決策を提供します。 -これは、同じデータに対してフィルタ条件(例: `WHERE col = 'xyz'`)を評価すると、常に同じ結果が返されるという考えに基づいています。 -より具体的には、クエリ条件キャッシュは、評価された各フィルタおよび各グラニュール(=デフォルトでは 8192 行のブロック)について、そのグラニュール内のいずれの行もフィルタ条件を満たさないかどうかを記録します。 -この情報は 1 ビットとして記録されます。ビット 0 は一致する行が 1 行もないことを表し、ビット 1 は少なくとも 1 行の一致する行が存在することを意味します。 -前者の場合、ClickHouse はフィルタ評価時に対応するグラニュールをスキップできますが、後者の場合、そのグラニュールは読み込んで評価する必要があります。 - -クエリ条件キャッシュが有効に機能するためには、次の 3 つの前提条件が満たされている必要があります: -- 1 つ目に、ワークロードが同じフィルタ条件を繰り返し評価している必要があります。これは、同じクエリが何度も繰り返し実行される場合には自然に起こりますが、2 つのクエリが同じフィルタを共有している場合にも起こり得ます。例: `SELECT product FROM products WHERE quality > 3` および `SELECT vendor, count() FROM products WHERE quality > 3`。 -- 2 つ目に、データの大部分が不変(クエリ間で変化しない)である必要があります。これは一般に ClickHouse では当てはまります。なぜなら、パーツは不変であり、INSERT によってのみ作成されるからです。 -- 3 つ目に、フィルタが選択的である、つまりフィルタ条件を満たす行が相対的に少ない必要があります。フィルタ条件に一致する行が少なければ少ないほど、ビット 0(該当行なし)で記録されるグラニュールが増え、その結果として後続のフィルタ評価から「プルーニング(prune)」できるデータ量が増加します。 - - - -## メモリ使用量 {#memory-consumption} - -クエリ条件キャッシュは、フィルタ条件とグラニュールごとに 1 ビットのみを保存するため、使用するメモリ量はごくわずかです。 -クエリ条件キャッシュの最大サイズは、サーバー設定 [`query_condition_cache_size`](server-configuration-parameters/settings.md#query_condition_cache_size)(デフォルト: 100 MB)で設定できます。 -キャッシュサイズが 100 MB の場合、100 * 1024 * 1024 * 8 = 838,860,800 エントリに相当します。 -各エントリは 1 つのマーク(デフォルトでは 8192 行)を表すため、キャッシュは 1 つのカラムについて最大 6,871,947,673,600(6.8 兆)行をカバーできます。 -実際には、フィルタは複数のカラムに対して評価されるため、この数値はフィルタ対象のカラム数で割る必要があります。 - - - -## 設定項目と使用方法 {#configuration-settings-and-usage} - -[use_query_condition_cache](settings/settings#use_query_condition_cache) 設定は、特定のクエリ、または現在のセッション内のすべてのクエリでクエリ条件キャッシュを利用するかどうかを制御します。 - -たとえば、このクエリを最初に実行すると - -```sql -SELECT col1, col2 -FROM table -WHERE col1 = 'x' -SETTINGS use_query_condition_cache = true; -``` - -クエリ条件キャッシュは、述語を満たさないテーブルの範囲を保存します。 -以降に同じクエリを、パラメータ `use_query_condition_cache = true` を指定して実行した場合、クエリ条件キャッシュを利用してスキャン対象のデータ量を減らします。 - - -## 管理 {#administration} - -クエリ条件キャッシュは、ClickHouse を再起動しても保持されません。 - -クエリ条件キャッシュをクリアするには、[`SYSTEM DROP QUERY CONDITION CACHE`](../sql-reference/statements/system.md#drop-query-condition-cache) を実行します。 - -キャッシュの内容は、システムテーブル [system.query_condition_cache](system-tables/query_condition_cache.md) に表示されます。 -現在のクエリ条件キャッシュのサイズを MB 単位で取得するには、`SELECT formatReadableSize(sum(entry_size)) FROM system.query_condition_cache` を実行します。 -個別のフィルター条件を調査したい場合は、`system.query_condition_cache` の `condition` フィールドを確認できます。 -このフィールドは、設定 [query_condition_cache_store_conditions_as_plaintext](settings/settings#query_condition_cache_store_conditions_as_plaintext) を有効にしてクエリを実行した場合にのみ値が設定される点に注意してください。 - -データベースの起動以降のクエリ条件キャッシュのヒット数とミス数は、システムテーブル [system.events](system-tables/events.md) において、イベント "QueryConditionCacheHits" および "QueryConditionCacheMisses" として表示されます。 -いずれのカウンタも、設定 `use_query_condition_cache = true` を有効にして実行された `SELECT` クエリに対してのみ更新され、その他のクエリは "QueryCacheMisses" に影響しません。 - - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Query Condition Cache の紹介](https://clickhouse.com/blog/introducing-the-clickhouse-query-condition-cache) -- [Predicate Caching: Query-Driven Secondary Indexing for Cloud Data Warehouses (Schmidt ほか, 2024)](https://doi.org/10.1145/3626246.3653395) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/quotas.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/quotas.md deleted file mode 100644 index cd3bc7f8a2e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/quotas.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -description: 'ClickHouse におけるリソース使用クォータの設定と管理ガイド' -sidebar_label: 'クォータ' -sidebar_position: 51 -slug: /operations/quotas -title: 'クォータ' -doc_type: 'guide' ---- - -:::note ClickHouse Cloud におけるクォータ -クォータは ClickHouse Cloud でサポートされていますが、[DDL 構文](/sql-reference/statements/create/quota) を使用して作成する必要があります。以下で説明する XML 設定方式は **サポートされていません**。 -::: - -クォータを使用すると、一定期間におけるリソース使用量を制限したり、リソースの使用状況を追跡したりできます。 -クォータはユーザー設定ファイルで行い、通常は「users.xml」に記述します。 - -システムには、単一クエリの複雑さを制限するための機能もあります。詳しくは、[クエリの複雑さに関する制限](../operations/settings/query-complexity.md)のセクションを参照してください。 - -クエリ複雑性の制限とは対照的に、クォータには次のような特徴があります。 - -* 単一クエリを制限するのではなく、一定期間に実行できるクエリの集合に制限をかけます。 -* 分散クエリ処理のために、すべてのリモートサーバーで消費されたリソースを計上します。 - -クォータを定義している「users.xml」ファイルの該当セクションを見てみましょう。 - -```xml - - - - - - - - 3600 - - - 0 - 0 - 0 - 0 - 0 - 0 - 0 - - -``` - -デフォルトでは、クォータは各時間ごとのリソース消費量を追跡しますが、使用量を制限はしません。 -各時間間隔で計算されたリソース消費量は、各リクエスト後にサーバーログへ出力されます。 - -```xml - - - - - 3600 - - 1000 - 100 - 100 - 5000000 - 100 - 1000000000 - 100000000000 - 900 - 5 - - - - 86400 - - 10000 - 10000 - 10000 - 1000 - 5000000000 - 160000000000 - 500000000000 - 16000000000000 - 7200 - - -``` - -'statbox' クォータでは、1時間ごとと 24時間ごと(86,400 秒)に制限が設定されます。時間間隔は、実装依存の固定された時点からの経過時間でカウントされます。つまり、24時間の間隔は必ずしも真夜中から始まるとは限りません。 - -間隔が終了すると、収集されたすべての値はクリアされます。次の1時間に対しては、クォータの計算が再びゼロから始まります。 - -制限を設定できる項目は次のとおりです。 - -`queries` – クエリの合計数。 - -`query_selects` – `select` クエリの合計数。 - -`query_inserts` – `insert` クエリの合計数。 - -`errors` – 例外をスローしたクエリの数。 - -`result_rows` – 結果として返された行数の合計。 - -`result_bytes` - 結果として返された行の合計サイズ。 - -`read_rows` – すべてのリモートサーバーでクエリを実行するためにテーブルから読み取られた元データ行数の合計。 - -`read_bytes` - すべてのリモートサーバーでクエリを実行するためにテーブルから読み取られたデータ量の合計。 - -`written_bytes` - 書き込み処理で出力されたデータ量の合計。 - -`execution_time` – クエリ実行時間の合計(秒、ウォールクロックタイム)。 - -`failed_sequential_authentications` - 連続して発生した認証エラーの合計回数。 - - -少なくとも 1 つの時間間隔で制限を超過した場合、どの制限がどの間隔で超過されたか、さらに新しい時間間隔(再びクエリを送信できるようになるタイミング)がいつ開始するかについてのメッセージを含む例外がスローされます。 - -クォータは「quota key」機能を使用して、複数のキーごとにリソースを独立して集計・報告できます。以下はその例です。 - -```xml - - - - -``` - -クォータは設定ファイルの'users'セクションでユーザーに割り当てられます。"Access rights" のセクションを参照してください。 - -分散クエリ処理では、累積値はリクエスト元サーバーに保存されます。そのため、ユーザーが別のサーバーへ移動した場合、そのサーバーでのクォータはゼロからカウントし直しになります。 - -サーバーを再起動すると、クォータはリセットされます。 - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [ClickHouse でシングルページアプリケーションを構築する](https://clickhouse.com/blog/building-single-page-applications-with-clickhouse-and-http) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md deleted file mode 100644 index af5fdc2b69a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md +++ /dev/null @@ -1,2682 +0,0 @@ -## asynchronous_metric_log {#asynchronous_metric_log} - -ClickHouse Cloud のデプロイメントでは、デフォルトで有効になっています。 - -この設定がデフォルトで有効になっていない環境では、ClickHouse のインストール方法に応じて、以下の手順で有効化または無効化できます。 - -**有効化** - -非同期メトリクスログ履歴の収集 [`system.asynchronous_metric_log`](../../operations/system-tables/asynchronous_metric_log.md) を手動で有効にするには、以下の内容で `/etc/clickhouse-server/config.d/asynchronous_metric_log.xml` を作成します。 - -```xml - - - system - asynchronous_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**無効化** - -`asynchronous_metric_log` 設定を無効化するには、次の内容でファイル `/etc/clickhouse-server/config.d/disable_asynchronous_metric_log.xml` を作成します。 - -```xml - -``` - - - - -## auth_use_forwarded_address {#auth_use_forwarded_address} - -プロキシ経由で接続しているクライアントに対して、認証に発信元アドレスを使用します。 - -:::note -この設定は、転送されたアドレスが容易になりすまし可能であるため、細心の注意を払って使用する必要があります。このような認証を受け付けるサーバーには、信頼できるプロキシ経由のみでアクセスし、直接アクセスしないようにしてください。 -::: - - - -## backups {#backups} - -[`BACKUP` および `RESTORE`](../backup.md) ステートメントの実行時に使用されるバックアップ関連の設定です。 - -以下の設定はサブタグを使用して構成できます。 - - - -{/* SQL - WITH settings AS ( - SELECT arrayJoin([ - ('allow_concurrent_backups', 'Bool','同一ホスト上で複数のバックアップ処理を同時に実行できるかどうかを決定します。', 'true'), - ('allow_concurrent_restores', 'Bool', '同一ホスト上で複数のリストア処理を同時に実行できるかどうかを決定します。', 'true'), - ('allowed_disk', 'String', '`File()` を使用する際のバックアップ先ディスク。この設定を指定しないと `File` は使用できません。', ''), - ('allowed_path', 'String', '`File()` を使用する際のバックアップ先パス。この設定を指定しないと `File` は使用できません。', ''), - ('attempts_to_collect_metadata_before_sleep', 'UInt', '収集済みメタデータを比較した結果に不整合があった場合、スリープに入る前にメタデータ収集を試行する回数。', '2'), - ('collect_metadata_timeout', 'UInt64', 'バックアップ中のメタデータ収集に対するタイムアウト(ミリ秒)。', '600000'), - ('compare_collected_metadata', 'Bool', 'true の場合、バックアップ中にメタデータが変更されていないことを保証するため、収集したメタデータと既存のメタデータを比較します。', 'true'), - ('create_table_timeout', 'UInt64', 'リストア中のテーブル作成に対するタイムアウト(ミリ秒)。', '300000'), - ('max_attempts_after_bad_version', 'UInt64', '協調バックアップ/リストア中に bad version エラーが発生した際に、リトライを行う最大試行回数。', '3'), - ('max_sleep_before_next_attempt_to_collect_metadata', 'UInt64', '次回のメタデータ収集を試行する前にスリープする最大時間(ミリ秒)。', '100'), - ('min_sleep_before_next_attempt_to_collect_metadata', 'UInt64', '次回のメタデータ収集を試行する前にスリープする最小時間(ミリ秒)。', '5000'), - ('remove_backup_files_after_failure', 'Bool', '`BACKUP` コマンドが失敗した場合、ClickHouse は失敗前にバックアップへコピー済みのファイルを削除しようとします。それ以外の場合は、コピー済みファイルはそのまま残されます。', 'true'), - ('sync_period_ms', 'UInt64', '協調バックアップ/リストアにおける同期間隔(ミリ秒)。', '5000'), - ('test_inject_sleep', 'Bool', 'テスト用のスリープを挿入します。', 'false'), - ('test_randomize_order', 'Bool', 'true の場合、テスト目的で特定の処理の順序をランダム化します。', 'false'), - ('zookeeper_path', 'String', '`ON CLUSTER` 句を使用する場合に、バックアップおよびリストア用メタデータを格納する ZooKeeper 上のパス。', '/clickhouse/backups') - ]) AS t ) - SELECT concat('`', t.1, '`') AS Setting, t.2 AS Type, t.3 AS Description, concat('`', t.4, '`') AS Default FROM settings FORMAT Markdown - */ } - -| 設定 | 型 | 説明 | デフォルト | -| :-------------------------------------------------- | :----- | :----------------------------------------------------------------------------------------------------- | :-------------------- | -| `allow_concurrent_backups` | Bool | 同一ホスト上で複数のバックアップ処理を同時に実行できるかどうかを制御します。 | `true` | -| `allow_concurrent_restores` | Bool | 同一ホスト上で複数のリストア処理を同時に実行できるかどうかを制御します。 | `true` | -| `allowed_disk` | String | `File()` を使用する際のバックアップ先ディスク。この設定を行わないと `File()` は使用できません。 | `` | -| `allowed_path` | 文字列 | `File()` を使用する場合のバックアップ先となるパス。`File` を使用するには、この設定を必ず指定する必要があります。 | `` | -| `attempts_to_collect_metadata_before_sleep` | UInt | 収集したメタデータを比較して不整合が検出された場合に、スリープに入る前にメタデータ収集を再試行する回数。 | `2` | -| `collect_metadata_timeout` | UInt64 | バックアップ時にメタデータを収集するためのタイムアウト(ミリ秒)。 | `600000` | -| `compare_collected_metadata` | Bool型 | true の場合、バックアップ中に変更されていないことを確認するために、収集したメタデータを既存のメタデータと比較します。 | `true` | -| `create_table_timeout` | UInt64 | リストア時のテーブル作成タイムアウト(ミリ秒単位)。 | `300000` | -| `max_attempts_after_bad_version` | UInt64 | 協調バックアップ/リストア中に不正バージョンエラーが発生した場合に再試行を行う最大回数。 | `3` | -| `max_sleep_before_next_attempt_to_collect_metadata` | UInt64 | 次回のメタデータ収集を試行するまでの最大スリープ時間(ミリ秒)。 | `100` | -| `min_sleep_before_next_attempt_to_collect_metadata` | UInt64 | 次のメタデータ収集試行までの最小スリープ時間(ミリ秒)。 | `5000` | -| `remove_backup_files_after_failure` | Bool | `BACKUP` コマンドが失敗した場合、ClickHouse は失敗が発生する前にバックアップにコピーされたファイルを削除しようとします。削除できなかった場合は、コピー済みのファイルはそのまま残ります。 | `true` | -| `sync_period_ms` | UInt64 | 協調バックアップ/リストア用の同期周期(ミリ秒単位)。 | `5000` | -| `test_inject_sleep` | Bool | テスト用のスリープ | `false` | -| `test_randomize_order` | Bool | true の場合、テスト目的で一部の操作の実行順序をランダムに入れ替えます。 | `false` | -| `zookeeper_path` | 文字列 | `ON CLUSTER` 句を使用する場合に、バックアップおよびリストアのメタデータが保存される ZooKeeper 上のパス。 | `/clickhouse/backups` | - -この設定はデフォルトで次のように設定されています。 - -```xml - - .... - -``` - - -## bcrypt_workfactor {#bcrypt_workfactor} - -[Bcrypt アルゴリズム](https://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/) を使用する `bcrypt_password` 認証タイプのワークファクターです。 -ワークファクターは、ハッシュを計算してパスワードを検証するのに必要な計算量と時間を決定します。 - -```xml -12 -``` - -:::warning -認証処理が高頻度で発生するアプリケーションでは、 -高いワークファクター設定時の bcrypt の計算コストの高さを考慮し、 -別の認証方式の採用を検討してください。 -::: - - -## table_engines_require_grant {#table_engines_require_grant} - -true に設定した場合、ユーザーが特定のエンジンを使用してテーブルを作成するには、対応する GRANT が必要になります(例: `GRANT TABLE ENGINE ON TinyLog to user`)。 - -:::note -デフォルトでは、後方互換性のため、特定のテーブルエンジンを指定してテーブルを作成する際の GRANT 要件は無視されますが、これを true に設定することでこの挙動を変更できます。 -::: - - - -## builtin_dictionaries_reload_interval {#builtin_dictionaries_reload_interval} - -組み込みディクショナリを再読み込みする間隔(秒)。 - -ClickHouse は、組み込みディクショナリを x 秒ごとに再読み込みします。これにより、サーバーを再起動せずに、ディクショナリを「オンザフライ」で編集できるようになります。 - -**例** - -```xml -3600 -``` - - -## 圧縮 {#compression} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) エンジンテーブル向けのデータ圧縮設定です。 - -:::note -ClickHouse を使い始めたばかりの場合は、この設定は変更しないことを推奨します。 -::: - -**設定テンプレート**: - -```xml - - - ... - ... - ... - ... - - ... - -``` - -**`` フィールド**: - -* `min_part_size` – データパートの最小サイズ。 -* `min_part_size_ratio` – データパートサイズとテーブルサイズの比率。 -* `method` – 圧縮方式。使用可能な値: `lz4`, `lz4hc`, `zstd`, `deflate_qpl`。 -* `level` – 圧縮レベル。[Codecs](/sql-reference/statements/create/table#general-purpose-codecs) を参照。 - -:::note -複数の `` セクションを設定できます。 -::: - -**条件が満たされたときの動作**: - -* データパートがある条件セットに一致した場合、ClickHouse は指定された圧縮方式を使用します。 -* データパートが複数の条件セットに一致した場合、ClickHouse は最初に一致した条件セットを使用します。 - -:::note -データパートがいずれの条件も満たさない場合、ClickHouse は `lz4` 圧縮を使用します。 -::: - -**例** - -```xml - - - 10000000000 - 0.01 - zstd - 1 - - -``` - - -## encryption {#encryption} - -[暗号化コーデック](/sql-reference/statements/create/table#encryption-codecs)で使用するキーを取得するためのコマンドを設定します。キー(複数可)は環境変数として指定するか、設定ファイルで設定する必要があります。 - -キーは、長さが 16 バイトの 16 進数、または文字列で指定できます。 - -**例** - -設定ファイルからの読み込み: - -```xml - - - 1234567812345678 - - -``` - -:::note -キーを設定ファイルに保存することは推奨されません。セキュアではありません。キーをセキュアなディスク上の別の設定ファイルに移動し、その設定ファイルへのシンボリックリンクを `config.d/` フォルダに配置できます。 -::: - -キーが 16 進数形式の場合に、設定から読み込むには次のようにします: - -```xml - - - 00112233445566778899aabbccddeeff - - -``` - -環境変数からキーを読み込む: - -```xml - - - - - -``` - -ここで `current_key_id` は暗号化に使用する現在のキーを指定し、指定されたすべてのキーを復号に使用できます。 - -これらの各方法は、複数のキーに対して適用できます。 - -```xml - - - 00112233445566778899aabbccddeeff - - 1 - - -``` - -ここで `current_key_id` は、暗号化に使用されている現在のキーを示します。 - -また、ユーザーはノンスを追加することもでき、その長さは 12 バイトである必要があります(デフォルトでは、暗号化および復号処理にはゼロバイトのみで構成されたノンスが使用されます): - -```xml - - - 012345678910 - - -``` - -または 16 進数表記で設定できます: - -```xml - - - abcdefabcdef - - -``` - -:::note -上記の内容はすべて `aes_256_gcm_siv` にも適用できます(ただし、キーは 32 バイトである必要があります)。 -::: - - -## error_log {#error_log} - -これはデフォルトでは無効です。 - -**有効化** - -エラー履歴の収集 [`system.error_log`](../../operations/system-tables/error_log.md) を手動で有効にするには、次の内容で `/etc/clickhouse-server/config.d/error_log.xml` を作成します。 - -```xml - - - system - error_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**無効化** - -`error_log` 設定を無効にするには、以下の内容で `/etc/clickhouse-server/config.d/disable_error_log.xml` ファイルを作成します。 - -```xml - - - -``` - - - - -## custom_settings_prefixes {#custom_settings_prefixes} - -[カスタム設定](/operations/settings/query-level#custom_settings) に使用するプレフィックスのリスト。プレフィックスはカンマ区切りで指定する必要があります。 - -**例** - -```xml -custom_ -``` - -**関連情報** - -* [カスタム設定](/operations/settings/query-level#custom_settings) - - -## core_dump {#core_dump} - -コアダンプファイルサイズのソフトリミットを設定します。 - -:::note -ハードリミットはシステムツールで設定します。 -::: - -**例** - -```xml - - 1073741824 - -``` - - -## default_profile {#default_profile} - -デフォルトの設定プロファイルです。設定プロファイルは、`user_config` 設定で指定されたファイル内に定義されます。 - -**例** - -```xml -default -``` - - -## dictionaries_config {#dictionaries_config} - -辞書の設定ファイルへのパス。 - -パス: - -* 絶対パス、またはサーバー設定ファイルからの相対パスを指定します。 -* パスにはワイルドカード * および ? を含めることができます。 - -参照: - -* 「[Dictionaries](../../sql-reference/dictionaries/index.md)」。 - -**例** - -```xml -*_dictionary.xml -``` - - -## user_defined_executable_functions_config {#user_defined_executable_functions_config} - -実行可能なユーザー定義関数用の設定ファイルのパスです。 - -Path: - -* 絶対パス、またはサーバー設定ファイルからの相対パスを指定します。 -* パスにはワイルドカードの * と ? を含めることができます。 - -See also: - -* "[Executable User Defined Functions](/sql-reference/functions/udf#executable-user-defined-functions)." - -**Example** - -```xml -*_function.xml -``` - - -## format_schema_path {#format_schema_path} - -入力データ用のスキーマ(例:[CapnProto](/interfaces/formats/CapnProto) フォーマットのスキーマ)が格納されているディレクトリへのパス。 - -**例** - -```xml - -format_schemas/ -``` - - -## graphite {#graphite} - -[Graphite](https://github.com/graphite-project) にデータを送信します。 - -設定: - -* `host` – Graphite サーバー。 -* `port` – Graphite サーバー上のポート。 -* `interval` – 送信間隔(秒)。 -* `timeout` – データ送信のタイムアウト(秒)。 -* `root_path` – キーのプレフィックス。 -* `metrics` – [system.metrics](/operations/system-tables/metrics) テーブルからデータを送信します。 -* `events` – [system.events](/operations/system-tables/events) テーブルから、一定期間に蓄積された差分データを送信します。 -* `events_cumulative` – [system.events](/operations/system-tables/events) テーブルから累積データを送信します。 -* `asynchronous_metrics` – [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) テーブルからデータを送信します。 - -`` 句は複数設定できます。たとえば、送信するデータごとに異なる送信間隔を設定できます。 - -**例** - -```xml - - localhost - 42000 - 0.1 - 60 - one_min - true - true - false - true - -``` - - -## graphite_rollup {#graphite_rollup} - -Graphite データを間引くための設定です。 - -詳細については、[GraphiteMergeTree](../../engines/table-engines/mergetree-family/graphitemergetree.md) を参照してください。 - -**例** - -```xml - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - - -## google_protos_path {#google_protos_path} - -Protobuf 型の proto ファイルを格納したディレクトリを指定します。 - -例: - -```xml -/usr/share/clickhouse/protos/ -``` - - -## http_handlers {#http_handlers} - -カスタム HTTP ハンドラーを使用できるようにします。 -新しい http ハンドラーを追加するには、新しい `` を追加するだけです。 -ルールは定義された順に上から下へ評価され、 -最初に一致したルールのハンドラーが実行されます。 - -以下の設定はサブタグで指定できます: - -| Sub-tags | Definition | -| -------------------- | ------------------------------------------------------------------------------------------------ | -| `url` | リクエスト URL をマッチさせるために使用します。正規表現によるマッチを行うには、プレフィックス 'regex:' を付けてください (オプション) | -| `methods` | リクエストメソッドをマッチさせるために使用します。複数メソッドを指定する場合はカンマで区切ります (オプション) | -| `headers` | リクエストヘッダーをマッチさせます。各子要素 (子要素名がヘッダー名) をマッチさせ、正規表現マッチを行うにはプレフィックス 'regex:' を付けてください (オプション) | -| `handler` | リクエストハンドラー | -| `empty_query_string` | URL にクエリ文字列が存在しないことを確認します | - -`handler` には以下の設定が含まれており、サブタグで指定できます: - -| Sub-tags | Definition | -| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | -| `url` | リダイレクト先の場所 | -| `type` | サポートされるタイプ: static, dynamic_query_handler, predefined_query_handler, redirect | -| `status` | static タイプで使用します。レスポンスステータスコード | -| `query_param_name` | dynamic_query_handler タイプで使用します。HTTP リクエストパラメータ内で `` に対応する値を抽出して実行します | -| `query` | predefined_query_handler タイプで使用します。ハンドラーが呼び出されたときにクエリを実行します | -| `content_type` | static タイプで使用します。レスポンスの content-type | -| `response_content` | static タイプで使用します。クライアントに送信されるレスポンスコンテンツです。'file://' または 'config://' プレフィックスを使用する場合、ファイルまたは設定から取得したコンテンツをクライアントに送信します | - -ルールのリストに加えて、すべてのデフォルトハンドラーを有効にする `` を指定できます。 - -例: - -```xml - - - / - POST,GET - no-cache - - dynamic_query_handler - query - - - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.settings - - - - - - static - 200 - text/plain; charset=UTF-8 - config://http_server_default_response - - - -``` - - -## http_server_default_response {#http_server_default_response} - -ClickHouse の HTTP(S) サーバーにアクセスしたときに、デフォルトで表示されるページです。 -デフォルト値は「Ok.」(末尾に改行文字付き)です。 - -**例** - -`http://localhost:http_port` にアクセスした際に `https://tabix.io/` が開かれます。 - -```xml - -
]]> -
-``` - - -## http_options_response {#http_options_response} - -`OPTIONS` HTTP リクエストに対するレスポンスにヘッダーを追加するために使用します。 -`OPTIONS` メソッドは、CORS プリフライトリクエストを行う際に使用されます。 - -詳細は [OPTIONS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS) を参照してください。 - -例: - -```xml - -
- Access-Control-Allow-Origin - * -
-
- Access-Control-Allow-Headers - origin, x-requested-with, x-clickhouse-format, x-clickhouse-user, x-clickhouse-key, Authorization -
-
- Access-Control-Allow-Methods - POST, GET, OPTIONS -
-
- Access-Control-Max-Age - 86400 -
-
-``` - - -## hsts_max_age {#hsts_max_age} - -HSTS の有効期限(秒単位)。 - -:::note -`0` の場合、ClickHouse は HSTS を無効化します。正の数値を設定すると HSTS が有効になり、`max-age` は指定した数値になります。 -::: - -**例** - -```xml -600000 -``` - - -## mlock_executable {#mlock_executable} - -起動後に `mlockall` を実行して、最初のクエリのレイテンシーを下げ、高い I/O 負荷時に ClickHouse の実行ファイルがページアウトされるのを防ぎます。 - -:::note -このオプションを有効にすることを推奨しますが、起動時間が数秒程度長くなります。 -また、この設定は「CAP_IPC_LOCK」ケーパビリティがないと動作しないことに注意してください。 -::: - -**例** - -```xml -false -``` - - -## include_from {#include_from} - -置換設定を含むファイルへのパスです。XML と YAML の両方の形式がサポートされています。 - -詳細については「[設定ファイル](/operations/configuration-files)」のセクションを参照してください。 - -**例** - -```xml -/etc/metrica.xml -``` - - -## interserver_listen_host {#interserver_listen_host} - -ClickHouse サーバー間でデータを交換できるホストを制限する設定。 -Keeper が使用されている場合は、異なる Keeper インスタンス間の通信にも同じ制限が適用されます。 - -:::note -デフォルトでは、この値は [`listen_host`](#listen_host) 設定と同じです。 -::: - -**例** - -```xml -::ffff:a00:1 -10.0.0.1 -``` - -型: - -デフォルト値: - - -## interserver_http_port {#interserver_http_port} - -ClickHouse サーバー間のデータ交換に使用するポート。 - -**例** - -```xml -9009 -``` - - -## interserver_http_host {#interserver_http_host} - -他のサーバーがこのサーバーにアクセスする際に使用されるホスト名です。 - -省略した場合は、`hostname -f` コマンドと同様に定義されます。 - -特定のネットワークインターフェイスに依存しないようにする場合に便利です。 - -**例** - -```xml -example.clickhouse.com -``` - - -## interserver_https_port {#interserver_https_port} - -`HTTPS` 経由で ClickHouse サーバー間のデータを交換するためのポート。 - -**例** - -```xml -9010 -``` - - -## interserver_https_host {#interserver_https_host} - -[`interserver_http_host`](#interserver_http_host) と同様ですが、このホスト名は他のサーバーが `HTTPS` 経由でこのサーバーにアクセスする際に使用されます。 - -**例** - -```xml -example.clickhouse.com -``` - - -## interserver_http_credentials {#interserver_http_credentials} - -[レプリケーション](../../engines/table-engines/mergetree-family/replication.md)中に他のサーバーへ接続するために使用されるユーザー名とパスワードです。さらに、サーバーはこれらの認証情報を使用して他のレプリカを認証します。 -そのため、`interserver_http_credentials` はクラスター内のすべてのレプリカで同一である必要があります。 - -:::note - -* 既定では、`interserver_http_credentials` セクションが省略された場合、レプリケーション時に認証は使用されません。 -* `interserver_http_credentials` 設定は ClickHouse クライアント認証情報の[設定](../../interfaces/cli.md#configuration_files)とは関係ありません。 -* これらの認証情報は、`HTTP` および `HTTPS` を介したレプリケーションで共通です。 - ::: - -以下の設定はサブタグで構成できます。 - -* `user` — ユーザー名。 -* `password` — パスワード。 -* `allow_empty` — `true` の場合、認証情報が設定されていても、他のレプリカは認証なしで接続することが許可されます。`false` の場合、認証なしの接続は拒否されます。既定値: `false`。 -* `old` — 認証情報ローテーション時に使用される、古い `user` と `password` を保持します。複数の `old` セクションを指定できます。 - -**認証情報のローテーション** - -ClickHouse は、すべてのレプリカを同時に停止して設定を更新することなく、interserver 用認証情報の動的なローテーションをサポートします。認証情報は複数の段階に分けて変更できます。 - -認証を有効にするには、`interserver_http_credentials.allow_empty` を `true` に設定し、認証情報を追加します。これにより、認証ありおよび認証なしの両方の接続が許可されます。 - -```xml - - admin - 111 - true - -``` - -すべてのレプリカの設定が完了したら、`allow_empty` を `false` に設定するか、この設定を削除してください。これにより、新しい認証情報での認証が必須になります。 - -既存の認証情報を変更するには、ユーザー名とパスワードを `interserver_http_credentials.old` セクションに移動し、`user` と `password` を新しい値に更新します。この時点で、サーバーは他のレプリカへの接続には新しい認証情報を使用し、他のレプリカからの接続については新旧どちらの認証情報も受け付けます。 - -```xml - - admin - 222 - - admin - 111 - - - temp - 000 - - -``` - -新しい認証情報がすべてのレプリカに適用されたら、古い認証情報は削除できます。 - - -## ldap_servers {#ldap_servers} - -接続パラメータ付きの LDAP サーバーをここに列挙します。これにより次のことができます: -- `'password'` の代わりに `'ldap'` 認証メカニズムが指定されたローカル専用ユーザーの認証器として使用する。 -- リモートユーザーディレクトリとして使用する。 - -以下の設定はサブタグで構成できます: - -| Setting | Description | -|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `host` | LDAP サーバーのホスト名または IP。必須パラメータであり、空にはできません。 | -| `port` | LDAP サーバーのポート。`enable_tls` が true の場合のデフォルトは 636、それ以外の場合は `389` です。 | -| `bind_dn` | バインドする DN を構成するために使用されるテンプレート。認証試行ごとに、テンプレート内の `\{user_name\}` のすべての部分文字列が実際のユーザー名に置き換えられて、最終的な DN が構成されます。 | -| `user_dn_detection` | バインドされたユーザーの実際のユーザー DN を検出するための LDAP 検索パラメータを含むセクション。これは主に、サーバーが Active Directory の場合に、後続のロールマッピングのための検索フィルターで使用されます。最終的なユーザー DN は、`\{user_dn\}` が許可されている場所の部分文字列を置き換える際に使用されます。デフォルトでは、ユーザー DN は bind DN と同じに設定されますが、一度検索が実行されると、実際に検出されたユーザー DN の値で更新されます。 | -| `verification_cooldown` | 正常にバインドされた後に、そのユーザーが LDAP サーバーへ問い合わせることなく、連続するすべてのリクエストに対して認証済みであるとみなされる時間(秒単位)。キャッシュを無効化し、各認証リクエストごとに LDAP サーバーへの問い合わせを強制するには、`0`(デフォルト)を指定します。 | -| `enable_tls` | LDAP サーバーへのセキュア接続を使用するかどうかを制御するフラグ。プレーンテキスト(`ldap://`)プロトコルを使用するには `no` を指定します(非推奨)。SSL/TLS 上の LDAP(`ldaps://`)プロトコルを使用するには `yes` を指定します(推奨、デフォルト)。レガシーな StartTLS プロトコル(プレーンテキストの `ldap://` プロトコルを TLS にアップグレードする方式)を使用するには `starttls` を指定します。 | -| `tls_minimum_protocol_version` | SSL/TLS の最小プロトコルバージョン。指定可能な値は: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2`(デフォルト)です。 | -| `tls_require_cert` | SSL/TLS ピア証明書の検証動作。指定可能な値は: `never`, `allow`, `try`, `demand`(デフォルト)です。 | -| `tls_cert_file` | 証明書ファイルへのパス。 | -| `tls_key_file` | 証明書鍵ファイルへのパス。 | -| `tls_ca_cert_file` | CA 証明書ファイルへのパス。 | -| `tls_ca_cert_dir` | CA 証明書を格納しているディレクトリへのパス。 | -| `tls_cipher_suite` | 許可される暗号スイート(OpenSSL 表記)。 | - -`user_dn_detection` 設定はサブタグで構成できます: - -| Setting | Description | -|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `base_dn` | LDAP 検索用のベース DN を構成するために使用されるテンプレート。LDAP 検索時に、テンプレート内の `\{user_name\}` および `\{bind_dn\}` のすべての部分文字列が、実際のユーザー名および bind DN に置き換えられて、最終的な DN が構成されます。 | -| `scope` | LDAP 検索のスコープ。指定可能な値は: `base`, `one_level`, `children`, `subtree`(デフォルト)です。 | -| `search_filter` | LDAP 検索用の検索フィルターを構成するために使用されるテンプレート。LDAP 検索時に、テンプレート内の `\{user_name\}`, `\{bind_dn\}`, `\{base_dn\}` のすべての部分文字列が、実際のユーザー名、bind DN、base DN に置き換えられて、最終的なフィルターが構成されます。なお、特殊文字は XML 内で正しくエスケープされている必要があります。 | - -例: - - - -```xml - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - -``` - -例(後続のロールマッピング用にユーザー DN 検出を設定した一般的な Active Directory 環境): - -```xml - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - -``` - - -## listen_host {#listen_host} - -リクエスト元ホストを制限します。サーバーがすべてのホストからのリクエストを受け付けるようにするには、`::` を指定します。 - -例: - -```xml -::1 -127.0.0.1 -``` - - -## listen_try {#listen_try} - -`listen` しようとした際に IPv6 または IPv4 ネットワークが利用できなくても、サーバーは終了しません。 - -**例** - -```xml -0 -``` - - -## listen_reuse_port {#listen_reuse_port} - -複数のサーバーが同じアドレスとポート番号で待ち受けできるようにします。リクエストはオペレーティングシステムによってランダムなサーバーにルーティングされます。この設定を有効にすることは推奨されていません。 - -**例** - -```xml -0 -``` - -型: - -デフォルト: - - -## listen_backlog {#listen_backlog} - -listen ソケットのバックログ(保留中の接続のキューサイズ)。デフォルト値 `4096` は Linux [5.4+](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19f92a030ca6d772ab44b22ee6a01378a8cb32d4)) と同じです。 - -通常、この値を変更する必要はありません。理由は次のとおりです: - -* デフォルト値が十分に大きいこと -* クライアント接続の accept 用にサーバー側で専用スレッドがあること - -そのため、`TcpExtListenOverflows`(`nstat` から取得)が 0 以外であり、かつ ClickHouse サーバーでこのカウンターが増加していても、この値を増やす必要があるとは限りません。理由は次のとおりです: - -* 通常、`4096` で足りない場合は ClickHouse 内部のスケーリング上の問題を示していることが多いため、Issue を作成して報告した方がよいです。 -* だからといって、サーバーが後からより多くの接続を処理できることを意味しません(たとえ処理できたとしても、その時点ではクライアントがすでに離脱または切断している可能性があります)。 - -**例** - -```xml -4096 -``` - - -## logger {#logger} - -ログメッセージの出力先とフォーマットを設定します。 - -**キー**: - -| Key | Description | -|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `level` | ログレベル。指定可能な値: `none` (ログ出力を無効化)、`fatal`、`critical`、`error`、`warning`、`notice`、`information`、`debug`、`trace`、`test` | -| `log` | ログファイルのパス。 | -| `errorlog` | エラーログファイルのパス。 | -| `size` | ローテーションポリシー: ログファイルの最大サイズ (バイト数)。ログファイルがこの閾値を超えると、名前が変更されてアーカイブされ、新しいログファイルが作成されます。 | -| `count` | ローテーションポリシー: 過去のログファイルを最大いくつまで ClickHouse に保持するかを指定します。 | -| `stream_compress` | LZ4 を使用してログメッセージを圧縮します。有効化するには `1` または `true` を設定します。 | -| `console` | コンソールへのログ出力を有効にします。有効化するには `1` または `true` を設定します。ClickHouse がデーモンモードで動作していない場合のデフォルトは `1`、それ以外は `0` です。 | -| `console_log_level` | コンソール出力用のログレベル。デフォルトは `level` と同じです。 | -| `formatting.type` | コンソール出力のログフォーマット。現在は `json` のみサポートされています。 | -| `use_syslog` | syslog にもログ出力を転送します。 | -| `syslog_level` | syslog へのログ出力に使用するログレベル。 | -| `async` | `true` (デフォルト) の場合、ログは非同期に出力されます (出力チャネルごとに 1 つのバックグラウンドスレッド)。それ以外の場合は、`LOG` を呼び出したスレッド内で出力されます。 | -| `async_queue_max_size` | 非同期ログを使用する場合に、フラッシュ待ちのメッセージをキューに保持しておける最大数。超過したメッセージは破棄されます。 | -| `startup_level` | サーバー起動時にルートロガーのレベルを設定するための起動時レベル。起動後は、ログレベルは `level` 設定の値に戻されます。 | -| `shutdown_level` | サーバー停止時にルートロガーのレベルを設定するための停止時レベル。 | - -**ログフォーマット指定子** - -`log` および `errorLog` パスに含まれるファイル名部分では、以下のフォーマット指定子を使用して生成されるファイル名を制御できます (ディレクトリ部分では使用できません)。 - -「Example」列では、`2023-07-06 18:32:07` のときの出力例を示しています。 - - - -| 指定子 | 概要 | 例 | -| ---- | ----------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| `%%` | リテラルの % 記号 | `%` | -| `%n` | 改行文字 | | -| `%t` | 水平タブ文字 | | -| `%Y` | 10進数で表した年(例: 2017) | `2023` | -| `%y` | 年を10進数で表した場合の下2桁(範囲 [00,99]) | `23` | -| `%C` | 年の上2桁を10進数で表した数値(範囲 [00,99]) | `20` | -| `%G` | 4桁の [ISO 8601 週を基準とする年](https://en.wikipedia.org/wiki/ISO_8601#Week_dates)、つまり指定された週を含む年。通常は `%V` と組み合わせて使用する場合にのみ有効です。 | `2023` | -| `%g` | [ISO 8601 週単位暦年](https://en.wikipedia.org/wiki/ISO_8601#Week_dates)の末尾2桁。すなわち、指定された週を含む年。 | `23` | -| `%b` | 省略形の月名。例: Oct(ロケールに依存) | `Jul` | -| `%h` | 「%b」の同義語 | `Jul` | -| `%B` | 月名をフル表記。例: October(ロケールに依存) | `7月` | -| `%m` | 月を表す10進数(範囲 [01,12]) | `07` | -| `%U` | 年内の週番号(10 進数。週の最初の曜日は日曜日)(範囲 [00,53]) | `27` | -| `%W` | 年内の週番号を 10 進数で表す(週の開始曜日は月曜日)(範囲 [00,53]) | `27` | -| `%V` | ISO 8601 の週番号(範囲 [01,53]) | `27` | -| `%j` | 年内通算日を10進数で表したもの(範囲 [001,366]) | `187` | -| `%d` | 月の日付をゼロ埋めした10進数で表します(範囲 [01,31])。1桁の場合は先頭にゼロを付けます。 | `06` | -| `%e` | 月の日を、先頭をスペースで桁埋めした 10 進数で表します(範囲 [1,31])。1 桁の場合は先頭にスペースが入ります。 | `  6` | -| `%a` | 曜日名の省略形。例: Fri(ロケールに依存) | `木` | -| `%A` | 曜日の完全な名称。例: Friday(ロケールに依存) | `木曜日` | -| `%w` | 日曜日を0とする整数値で表した曜日(範囲 [0-6]) | `4` | -| `%u` | 曜日を表す10進数で、月曜日を1とする(ISO 8601 形式)(範囲 [1-7]) | `4` | -| `%H` | 時を表す10進数、24時間制(範囲 [00-23]) | `18` | -| `%I` | 時を 10 進数で表した値(12 時間制、範囲 [01,12]) | `06` | -| `%M` | 分を表す 10 進数(範囲 [00,59]) | `32` | -| `%S` | 秒を表す 10 進数(範囲 [00,60]) | `07` | -| `%c` | 標準的な日付と時刻の文字列表現。例: Sun Oct 17 04:41:13 2010(ロケールに依存) | `Thu Jul 6 18:32:07 2023` | -| `%x` | ロケールに応じた日付表記(ロケール依存) | `07/06/23` | -| `%X` | ロケールに応じた時刻表現。例: 18:40:20 または 6:40:20 PM(ロケールに依存) | `18:32:07` | -| `%D` | 短い MM/DD/YY 形式の日付(%m/%d/%y と同等) | `07/06/23` | -| `%F` | 短い YYYY-MM-DD 形式の日付。%Y-%m-%d に相当 | `2023-07-06` | -| `%r` | ロケールに応じた12時間制の時刻表記 | `06:32:07 PM` | -| `%R` | 「%H:%M」と同じ | `18:32` | -| `%T` | "%H:%M:%S"(ISO 8601 の時刻形式)と同等 | `18:32:07` | -| `%p` | ローカライズされた午前/午後の表記(ロケールに依存) | `PM` | -| `%z` | ISO 8601 形式の UTC からのオフセット(例: -0430)、またはタイムゾーン情報が利用できない場合は空文字列 | `+0800` | -| `%Z` | ロケールに依存したタイムゾーン名またはその略称。タイムゾーン情報が利用できない場合は空文字列 | `Z AWST ` | - -**例** - -```xml - - trace - /var/log/clickhouse-server/clickhouse-server-%F-%T.log - /var/log/clickhouse-server/clickhouse-server-%F-%T.err.log - 1000M - 10 - true - -``` - -ログメッセージのみをコンソールに出力するには: - -```xml - - information - true - -``` - -**レベルごとのオーバーライド** - -特定のログ名ごとにログレベルを上書きできます。例えば、ロガー「Backup」と「RBAC」のすべてのメッセージを出力しないようにする場合です。 - -```xml - - - - Backup - none - - - RBAC - none - - - -``` - -**syslog** - -ログメッセージを syslog にも書き込むには、次のようにします。 - -```xml - - 1 - -
syslog.remote:10514
- myhost.local - LOG_LOCAL6 - syslog -
-
-``` - -`<syslog>` のキー: - -| Key | Description | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `address` | `host\[:port\]` 形式で指定する syslog のアドレス。省略した場合はローカルデーモンが使用されます。 | -| `hostname` | ログが送信されるホスト名(任意)。 | -| `facility` | syslog の[facility キーワード](https://en.wikipedia.org/wiki/Syslog#Facility)。必ず大文字で `"LOG_"` 接頭辞を付けて指定します(例: `LOG_USER`, `LOG_DAEMON`, `LOG_LOCAL3` など)。デフォルト: `address` が指定されている場合は `LOG_USER`、それ以外は `LOG_DAEMON`。 | -| `format` | ログメッセージの形式。指定可能な値: `bsd` および `syslog`。 | - -**Log formats** - -コンソールログに出力されるログ形式を指定できます。現在は JSON のみがサポートされています。 - -**Example** - -出力される JSON ログの例を次に示します: - -```json -{ - "date_time_utc": "2024-11-06T09:06:09Z", - "date_time": "1650918987.180175", - "thread_name": "#1", - "thread_id": "254545", - "level": "Trace", - "query_id": "", - "logger_name": "BaseDaemon", - "message": "シグナル2を受信", - "source_file": "../base/daemon/BaseDaemon.cpp; virtual void SignalListener::run()", - "source_line": "192" -} -``` - -JSON ログ出力を有効にするには、以下のスニペットを使用します。 - -```xml - - - json - - - - date_time - thread_name - thread_id - level - query_id - logger_name - message - source_file - source_line - - - -``` - -**JSON ログのキー名の変更** - -キー名は、`` タグ内のタグの値を変更することで変更できます。たとえば、`DATE_TIME` を `MY_DATE_TIME` に変更するには、`MY_DATE_TIME` を使用します。 - -**JSON ログのキーの省略** - -ログプロパティは、そのプロパティをコメントアウトすることで省略できます。たとえば、ログに `query_id` を出力したくない場合は、`` タグをコメントアウトします。 - - -## send_crash_reports {#send_crash_reports} - -ClickHouse コア開発チームへクラッシュレポートを送信するための設定です。 - -特に本番前の環境では有効化していただけると非常に助かります。 - -Keys: - -| Key | Description | -| --------------------- | ---------------------------------------------------------------------------------------------- | -| `enabled` | 機能を有効にするためのブールフラグ。デフォルトは `true`。クラッシュレポート送信を行いたくない場合は `false` に設定します。 | -| `send_logical_errors` | `LOGICAL_ERROR` は `assert` のようなもので、ClickHouse のバグです。このブールフラグは、これらの例外の送信を有効にします(デフォルト: `true`)。 | -| `endpoint` | クラッシュレポートの送信先エンドポイント URL を上書きできます。 | - -**推奨される使用方法** - -```xml - - true - -``` - - -## ssh_server {#ssh_server} - -ホスト鍵の公開鍵部分は、最初に接続した際に SSH クライアント側の known_hosts ファイルに書き込まれます。 - -Host Key Configurations はデフォルトでは無効です。 -Host Key Configurations のコメントアウトを解除し、それぞれに対応する ssh 鍵へのパスを指定して有効にします: - -例: - -```xml - - path_to_the_ssh_key - path_to_the_ssh_key - path_to_the_ssh_key - -``` - - -## tcp_ssh_port {#tcp_ssh_port} - -ユーザーが組み込みクライアントを使用して PTY 経由で接続し、対話的にクエリを実行できるようにする SSH サーバーのポートです。 - -例: - -```xml -9022 -``` - - -## storage_configuration {#storage_configuration} - -ストレージの複数ディスク構成を行うための設定です。 - -ストレージ構成は次のような構造になります。 - -```xml - - - - - - - - -``` - -### ディスクの構成 {#configuration-of-disks} - -`disks` の構成は、以下に示す構造に従います。 - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - ... - - -``` - -上記のサブタグでは、`disks` に対して次の設定を行います: - -| Setting | Description | -| ----------------------- | ------------------------------------------------------------- | -| `` | 一意である必要があるディスク名。 | -| `path` | サーバーデータ(`data` および `shadow` カタログ)が保存されるパス。末尾は `/` である必要があります。 | -| `keep_free_space_bytes` | ディスク上で予約される空き容量のサイズ。 | - -:::note -ディスクの順序は関係ありません。 -::: - -### ポリシーの設定 {#configuration-of-policies} - -上記のサブタグでは、`policies` に対して次の設定を行います: - - -| Setting | Description | -|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `policy_name_N` | ポリシー名。ポリシー名は一意でなければなりません。 | -| `volume_name_N` | ボリューム名。ボリューム名は一意でなければなりません。 | -| `disk` | ボリューム内にあるディスク。 | -| `max_data_part_size_bytes` | このボリューム内の任意のディスク上に存在できるデータパーツの最大サイズ。マージの結果として生成されるパーツサイズが `max_data_part_size_bytes` より大きくなると予想される場合、そのパーツは次のボリュームに書き込まれます。基本的に、この機能により、新規 / 小さいパーツをホット(SSD)ボリュームに保存し、サイズが大きくなった時点でコールド(HDD)ボリュームに移動できます。ポリシーにボリュームが 1 つしかない場合は、このオプションを使用しないでください。 | -| `move_factor` | ボリューム上の利用可能な空き容量の割合。この値を下回ると、(存在する場合は)データの転送が次のボリュームに開始されます。転送では、パーツはサイズの大きいものから小さいもの(降順)にソートされ、合計サイズが `move_factor` 条件を満たすのに十分なパーツが選択されます。すべてのパーツの合計サイズでも条件を満たせない場合は、すべてのパーツが移動されます。 | -| `perform_ttl_move_on_insert` | 挿入時の TTL 期限切れデータの移動を無効にします。デフォルト(有効な場合)では、TTL に基づく移動ルールに従ってすでに期限切れとなっているデータを挿入すると、そのデータは直ちに移動ルールで指定されたボリューム / ディスクに移動されます。ターゲットのボリューム / ディスクが遅い場合(例: S3)、これにより挿入が大幅に遅くなる可能性があります。無効にした場合、期限切れ部分のデータはまずデフォルトボリュームに書き込まれ、その後直ちに、期限切れ TTL 用のルールで指定されたボリュームに移動されます。 | -| `load_balancing` | ディスクのバランシングポリシー。`round_robin` または `least_used`。 | -| `least_used_ttl_ms` | すべてのディスク上の利用可能な空き容量を更新するためのタイムアウト(ミリ秒)を設定します(`0` - 常に更新、`-1` - 一切更新しない、デフォルト値は `60000`)。ディスクが ClickHouse のみによって使用され、ファイルシステムのオンラインでのリサイズが発生しない場合は、`-1` の値を使用できます。それ以外のすべての場合、この設定は推奨されません。最終的に不正確な空き容量の割り当てにつながるためです。 | -| `prefer_not_to_merge` | このボリューム上でのデータパーツのマージを無効にします。注意: これは潜在的に有害であり、パフォーマンス低下を引き起こす可能性があります。この設定が有効な場合(有効にしないでください)、このボリューム上でのデータマージは禁止されます(これは望ましくありません)。この設定により、ClickHouse が低速なディスクとどのようにやり取りするかを制御できますが、基本的には使用しないことを推奨します。 | -| `volume_priority` | ボリュームを埋めていく際の優先度(順序)を定義します。値が小さいほど優先度が高くなります。パラメーター値は自然数でなければならず、1 から N(N は指定されたパラメーター値の最大値)までの範囲を欠番なく網羅する必要があります。 | - -`volume_priority` について: -- すべてのボリュームにこのパラメーターが設定されている場合、指定された順序で優先されます。 -- _一部の_ ボリュームのみに設定されている場合、設定されていないボリュームは最も低い優先度になります。設定されているボリュームはパラメーター値に基づいて優先され、残りについては設定ファイル内での記述順によって互いの優先度が決まります。 -- _どの_ ボリュームにもこのパラメーターが設定されていない場合、その順序は設定ファイル内での記述順によって決まります。 -- ボリュームの優先度は同一である必要はありません。 - - - -## マクロ {#macros} - -レプリケーテッドテーブル向けのパラメータ置換です。 - -レプリケーテッドテーブルを使用しない場合は省略できます。 - -詳細については、[レプリケーテッドテーブルの作成](../../engines/table-engines/mergetree-family/replication.md#creating-replicated-tables)のセクションを参照してください。 - -**例** - -```xml - -``` - - -## replica_group_name {#replica_group_name} - -Replicated データベースのレプリカグループ名。 - -Replicated データベースで作成されるクラスタは、同一グループ内のレプリカで構成されます。 -DDL クエリは同一グループ内のレプリカに対してのみ待機します。 - -既定値は空です。 - -**例** - -```xml -バックアップ -``` - - -## remap_executable {#remap_executable} - -ヒュージページを使用して、マシンコード(「text」)用のメモリを再割り当てするための設定です。 - -:::note -この機能は非常に実験的な機能です。 -::: - -例: - -```xml -false -``` - - -## max_open_files {#max_open_files} - -同時に開くことができるファイルの最大数です。 - -:::note -`getrlimit()` 関数が誤った値を返すため、macOS ではこのオプションの使用を推奨します。 -::: - -**例** - -```xml -262144 -``` - - -## max_session_timeout {#max_session_timeout} - -セッションの最大タイムアウト時間(秒)。 - -例: - -```xml -3600 -``` - - -## merge_tree {#merge_tree} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブル向けの微調整。 - -詳細については、`MergeTreeSettings.h` ヘッダーファイルを参照してください。 - -**例** - -```xml - - 5 - -``` - - -## metric_log {#metric_log} - -デフォルトでは無効になっています。 - -**有効化** - -メトリクス履歴の収集を手動で有効化するには、次の内容で `/etc/clickhouse-server/config.d/metric_log.xml` を作成します:[`system.metric_log`](../../operations/system-tables/metric_log.md)。 - -```xml - - - system - metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**無効化** - -`metric_log` 設定を無効にするには、以下の内容でファイル `/etc/clickhouse-server/config.d/disable_metric_log.xml` を作成する必要があります。 - -```xml - - - -``` - - - - -## replicated_merge_tree {#replicated_merge_tree} - -[ReplicatedMergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブル向けの微調整用設定です。この設定はより高い優先度を持ちます。 - -詳細については、MergeTreeSettings.h ヘッダーファイルを参照してください。 - -**例** - -```xml - - 5 - -``` - - -## opentelemetry_span_log {#opentelemetry_span_log} - -[`opentelemetry_span_log`](../system-tables/opentelemetry_span_log.md) システムテーブルの設定。 - - - -例: - -```xml - - - engine MergeTree - partition by toYYYYMM(finish_date) - order by (finish_date, finish_time_us, trace_id) - - system - opentelemetry_span_log
- 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## openSSL {#openSSL} - -SSL クライアント/サーバーの設定。 - -SSL のサポートは `libpoco` ライブラリによって提供されます。利用可能な設定オプションについては、[SSLManager.h](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/SSLManager.h) を参照してください。デフォルト値は [SSLManager.cpp](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/src/SSLManager.cpp) で確認できます。 - -サーバー/クライアント設定用のキー: - - - -| オプション | 説明 | デフォルト値 | -| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | -| `privateKeyFile` | PEM 証明書の秘密鍵を含むファイルへのパスです。ファイルには鍵と証明書の両方を含めることができます。 | | -| `certificateFile` | PEM 形式のクライアント/サーバー証明書ファイルへのパス。`privateKeyFile` に証明書が含まれている場合は省略可能です。 | | -| `caConfig` | 信頼された CA 証明書を含むファイルまたはディレクトリへのパス。ファイルを指す場合、そのファイルは PEM 形式であり、複数の CA 証明書を含めることができます。ディレクトリを指す場合、そのディレクトリには CA 証明書ごとに 1 つの .pem ファイルが必要です。ファイル名は CA のサブジェクト名ハッシュ値に基づいて検索されます。詳細は [SSL_CTX_load_verify_locations](https://www.openssl.org/docs/man3.0/man3/SSL_CTX_load_verify_locations.html) の man ページを参照してください。 | | -| `verificationMode` | ノードの証明書を検証する方法を指定します。詳細は [Context](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/Context.h) クラスの説明を参照してください。指定可能な値は `none`, `relaxed`, `strict`, `once` です。 | `relaxed` | -| `verificationDepth` | 検証チェーンの最大長。証明書チェーンの長さが設定値を超えると、検証は失敗します。 | `9` | -| `loadDefaultCAFile` | OpenSSL の組み込み CA 証明書を使用するかどうかを指定します。ClickHouse は、組み込み CA 証明書がファイル `/etc/ssl/cert.pem`(またはディレクトリ `/etc/ssl/certs`)にあるか、環境変数 `SSL_CERT_FILE`(または `SSL_CERT_DIR`)で指定されたファイル(またはディレクトリ)にあると想定します。 | `true` | -| `cipherList` | サポートされている OpenSSL の暗号方式。 | `ALL:!ADH:!LOW:!EXP:!MD5:!3DES:@STRENGTH` | -| `cacheSessions` | セッションのキャッシュを有効または無効にします。`sessionIdContext` と組み合わせて使用する必要があります。設定可能な値: `true`、`false`。 | `false` | -| `sessionIdContext` | サーバーが生成する各識別子に付加される、一意のランダム文字列です。文字列の長さは `SSL_MAX_SSL_SESSION_ID_LENGTH` を超えてはなりません。サーバー側でセッションをキャッシュする場合にも、クライアントがキャッシュを要求した場合にも問題の発生を防ぐのに役立つため、このパラメータは常に指定することを推奨します。 | `$\{application.name\}` | -| `sessionCacheSize` | サーバーがキャッシュするセッションの最大数。値を `0` にすると、セッション数が無制限であることを意味します。 | [1024*20](https://github.com/ClickHouse/boringssl/blob/master/include/openssl/ssl.h#L1978) | -| `sessionTimeout` | サーバー側でセッションをキャッシュしておく時間(単位:時間)。 | `2` | -| `extendedVerification` | 有効にすると、証明書の CN または SAN がピアのホスト名と一致するか検証します。 | `false` | -| `requireTLSv1` | TLSv1 接続を必須とします。有効な値: `true`、`false`。 | `false` | -| `requireTLSv1_1` | TLSv1.1 接続を必須にします。設定可能な値: `true`, `false`。 | `false` | -| `requireTLSv1_2` | TLSv1.2 接続を必須とします。指定可能な値: `true`, `false`。 | `false` | -| `fips` | OpenSSL の FIPS モードを有効にします。ライブラリで使用している OpenSSL のバージョンが FIPS に対応している場合に有効です。 | `false` | -| `privateKeyPassphraseHandler` | 秘密鍵にアクセスするためのパスフレーズを要求するクラス(PrivateKeyPassphraseHandler のサブクラス)。例:``, `KeyFileHandler`, `test`, ``。 | `KeyConsoleHandler` | -| `invalidCertificateHandler` | 無効な証明書を検証するクラス(CertificateHandler のサブクラス)。例:` RejectCertificateHandler `。 | `RejectCertificateHandler` | -| `disableProtocols` | 使用禁止のプロトコル。 | | -| `preferServerCiphers` | クライアント優先のサーバー側暗号スイート。 | `false` | - -**設定例:** - -```xml - - - - /etc/clickhouse-server/server.crt - /etc/clickhouse-server/server.key - - /etc/clickhouse-server/dhparam.pem - none - true - true - sslv2,sslv3 - true - - - true - true - sslv2,sslv3 - true - - - - RejectCertificateHandler - - - -``` - - -## part_log {#part_log} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) に関連するイベントを記録するログです。たとえば、データの追加やマージなどです。ログを使用してマージアルゴリズムをシミュレートし、その特性を比較できます。マージ処理を可視化することもできます。 - -クエリは個別のファイルではなく、[system.part_log](/operations/system-tables/part_log) テーブルに記録されます。このテーブル名は `table` パラメータ(以下参照)で構成できます。 - - - -**例** - -```xml - - system - part_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## path {#path} - -データが格納されているディレクトリへのパス。 - -:::note -末尾のスラッシュは必須です。 -::: - -**例** - -```xml -/var/lib/clickhouse/ -``` - - -## processors_profile_log {#processors_profile_log} - -[`processors_profile_log`](../system-tables/processors_profile_log.md) システムテーブルの設定です。 - - - -既定の設定は次のとおりです。 - -```xml - - system - processors_profile_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## prometheus {#prometheus} - -[Prometheus](https://prometheus.io) からスクレイプできるようにメトリクスデータを公開します。 - -設定項目: - -* `endpoint` – Prometheus サーバーがメトリクスをスクレイプするための HTTP エンドポイント。必ず '/' から始めます。 -* `port` – `endpoint` のポート。 -* `metrics` – [system.metrics](/operations/system-tables/metrics) テーブルからメトリクスを公開します。 -* `events` – [system.events](/operations/system-tables/events) テーブルからメトリクスを公開します。 -* `asynchronous_metrics` – [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) テーブルから現在のメトリクス値を公開します。 -* `errors` - 最後にサーバーが再起動されてから発生したエラーコードごとのエラー数を公開します。この情報は [system.errors](/operations/system-tables/errors) からも取得できます。 - -**例** - -```xml - - 0.0.0.0 - 8123 - 9000 - - - /metrics - 9363 - true - true - true - true - - - -``` - -確認します(`127.0.0.1` を ClickHouse サーバーの IP アドレスまたはホスト名に置き換えてください): - -```bash -curl 127.0.0.1:9363/metrics -``` - - -## query_log {#query_log} - -[log_queries=1](../../operations/settings/settings.md) 設定が有効な場合に受信したクエリをログに記録するための設定です。 - -クエリは個別のファイルではなく、[system.query_log](/operations/system-tables/query_log) テーブルに記録されます。テーブル名は `table` パラメータで変更できます(後述)。 - - - -テーブルが存在しない場合、ClickHouse はテーブルを作成します。ClickHouse サーバーのアップデートによってクエリログの構造が変更された場合は、古い構造のテーブルの名前が変更され、新しいテーブルが自動的に作成されます。 - -**例** - -```xml - - system - query_log
- Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_metric_log {#query_metric_log} - -デフォルトでは無効です。 - -**有効化** - -メトリクス履歴収集 [`system.query_metric_log`](../../operations/system-tables/query_metric_log.md) を手動で有効にするには、次の内容で `/etc/clickhouse-server/config.d/query_metric_log.xml` を作成します。 - -```xml - - - system - query_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**無効化** - -`query_metric_log` 設定を無効にするには、以下の内容のファイル `/etc/clickhouse-server/config.d/disable_query_metric_log.xml` を作成してください。 - -```xml - - - -``` - - - - -## query_cache {#query_cache} - -[クエリキャッシュ](../query-cache.md)の設定です。 - -利用可能な設定は次のとおりです。 - -| Setting | Description | Default Value | -| ------------------------- | -------------------------------------------- | ------------- | -| `max_size_in_bytes` | キャッシュの最大サイズ(バイト単位)。`0` の場合、クエリキャッシュは無効になります。 | `1073741824` | -| `max_entries` | キャッシュに保存される `SELECT` クエリ結果の最大件数。 | `1024` | -| `max_entry_size_in_bytes` | キャッシュに保存できる `SELECT` クエリ結果の最大サイズ(バイト単位)。 | `1048576` | -| `max_entry_size_in_rows` | キャッシュに保存できる `SELECT` クエリ結果の最大行数。 | `30000000` | - -:::note - -* 設定変更は即座に有効になります。 -* クエリキャッシュ用のデータは DRAM に確保されます。メモリに余裕がない場合は、`max_size_in_bytes` を小さな値に設定するか、クエリキャッシュを無効化することを検討してください。 - ::: - -**例** - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - - -## query_thread_log {#query_thread_log} - -[log_query_threads=1](/operations/settings/settings#log_query_threads) が設定されたクエリのスレッドをログに記録するための設定です。 - -クエリは個別のファイルではなく、[system.query_thread_log](/operations/system-tables/query_thread_log) テーブルに記録されます。`table` パラメータでテーブル名を変更できます(後述)。 - - - -テーブルが存在しない場合、ClickHouse はテーブルを作成します。ClickHouse サーバーの更新時にクエリスレッドログの構造が変更された場合は、古い構造を持つテーブルの名前が変更され、新しいテーブルが自動的に作成されます。 - -**例** - -```xml - - system - query_thread_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_views_log {#query_views_log} - -[log_query_views=1](/operations/settings/settings#log_query_views) 設定を指定して受信したクエリに応じて、ビュー(live、materialized など)をログに記録するための設定です。 - -クエリは別ファイルではなく、[system.query_views_log](/operations/system-tables/query_views_log) テーブルに記録されます。テーブル名は(後述の)`table` パラメータで変更できます。 - - - -テーブルが存在しない場合、ClickHouse が作成します。ClickHouse サーバーの更新時にクエリビューのログ構造が変更された場合、古い構造のテーブルは名前が変更され、新しいテーブルが自動的に作成されます。 - -**例** - -```xml - - system - query_views_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## text_log {#text_log} - -テキストメッセージをログに記録するための [text_log](/operations/system-tables/text_log) システムテーブルの設定。 - - - -追加の設定: - -| Setting | Description | Default Value | -| ------- | -------------------------------------- | ------------- | -| `level` | テーブルに保存されるメッセージレベルの上限(デフォルトは `Trace`)。 | `Trace` | - -**例** - -```xml - - - notice - system - text_log
- 7500 - 1048576 - 8192 - 524288 - false - - Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day -
-
-``` - - -## trace_log {#trace_log} - -[trace_log](/operations/system-tables/trace_log) システムテーブルの動作設定。 - - - -デフォルトのサーバー設定ファイル `config.xml` には、次の設定セクションが含まれています。 - -```xml - - system - trace_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false - false -
-``` - - -## asynchronous_insert_log {#asynchronous_insert_log} - -非同期挿入をログとして記録するための [asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) システムテーブル用の設定です。 - - - -**例** - -```xml - - - system - asynchronous_insert_log
- 7500 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## crash_log {#crash_log} - -[crash_log](../../operations/system-tables/crash_log.md) システムテーブルの動作に関する設定です。 - -以下の設定はサブタグで構成できます。 - -| Setting | Description | Default | Note | -| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------- | ------------------- | --------------------------------------------------------------------------------------- | -| `database` | データベース名。 | | | -| `table` | システムテーブル名。 | | | -| `engine` | システムテーブル用の [MergeTree エンジン定義](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-creating-a-table)。 | | `partition_by` または `order_by` が定義されている場合には使用できません。指定されていない場合、デフォルトで `MergeTree` が選択されます | -| `partition_by` | システムテーブル用の [カスタムパーティショニングキー](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。 | | システムテーブルに対して `engine` が指定されている場合、`partition_by` パラメータは直接 `engine` の内部で指定する必要があります | -| `ttl` | テーブルの [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) を指定します。 | | システムテーブルに対して `engine` が指定されている場合、`ttl` パラメータは直接 `engine` の内部で指定する必要があります | -| `order_by` | システムテーブル用の [カスタムソートキー](/engines/table-engines/mergetree-family/mergetree#order_by)。`engine` が定義されている場合は使用できません。 | | システムテーブルに対して `engine` が指定されている場合、`order_by` パラメータは直接 `engine` の内部で指定する必要があります | -| `storage_policy` | テーブルに使用するストレージポリシー名 (オプション)。 | | システムテーブルに対して `engine` が指定されている場合、`storage_policy` パラメータは直接 `engine` の内部で指定する必要があります | -| `settings` | MergeTree の動作を制御する [追加パラメータ](/engines/table-engines/mergetree-family/mergetree/#settings) (オプション)。 | | システムテーブルに対して `engine` が指定されている場合、`settings` パラメータは直接 `engine` の内部で指定する必要があります | -| `flush_interval_milliseconds` | メモリ上のバッファからテーブルへデータをフラッシュする間隔。 | `7500` | | -| `max_size_rows` | ログの行数の最大値。フラッシュされていないログの量が `max_size_rows` に達すると、ログはディスクにダンプされます。 | `1024` | | -| `reserved_size_rows` | ログ用に事前確保されるメモリサイズ (行数)。 | `1024` | | -| `buffer_size_rows_flush_threshold` | 行数に対するしきい値。このしきい値に達すると、バックグラウンドでディスクへのログフラッシュが開始されます。 | `max_size_rows / 2` | | -| `flush_on_crash` | クラッシュ時にログをディスクへダンプするかどうかを設定します。 | `false` | | - -デフォルトのサーバー設定ファイル `config.xml` には、次の `settings` セクションが含まれます。 - -```xml - - system - crash_log
- toYYYYMM(event_date) - 7500 - 1024 - 1024 - 512 - false -
-``` - - -## custom_cached_disks_base_directory {#custom_cached_disks_base_directory} - -この設定は、カスタム(SQL から作成した)キャッシュディスクのキャッシュパスを指定します。 -`custom_cached_disks_base_directory` は、カスタムディスクに対しては `filesystem_caches_path`(`filesystem_caches_path.xml` に記載)より高い優先度を持ち、 -前者が未設定の場合にのみ後者が使用されます。 -ファイルシステムキャッシュの設定パスは、このディレクトリ配下に含まれている必要があり、 -そうでない場合はディスクの作成を防ぐために例外がスローされます。 - -:::note -これは、古いバージョンで作成され、その後サーバーをアップグレードしたディスクには影響しません。 -この場合、サーバーが正常に起動できるように、例外はスローされません。 -::: - -例: - -```xml -/var/lib/clickhouse/caches/ -``` - - -## backup_log {#backup_log} - -`BACKUP` および `RESTORE` 操作をログに記録するための [backup_log](../../operations/system-tables/backup_log.md) システムテーブル用の設定です。 - - - -**例** - -```xml - - - system - backup_log
- 1000 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## blob_storage_log {#blob_storage_log} - -[`blob_storage_log`](../system-tables/blob_storage_log.md) システムテーブルに関する設定です。 - - - -例: - -```xml - - systemblob_storage_logtoYYYYMM(event_date) - 7500event_date + INTERVAL 30 DAY - -``` - - -## query_masking_rules {#query_masking_rules} - -正規表現に基づくルールです。クエリおよびすべてのログメッセージに対して、サーバーログ -[`system.query_log`](/operations/system-tables/query_log)、[`system.text_log`](/operations/system-tables/text_log)、[`system.processes`](/operations/system-tables/processes) テーブルに保存される前と、クライアントに送信されるログの両方に適用されます。これにより、名前、メールアドレス、個人識別子、クレジットカード番号などの機密データが、SQL クエリからログへ漏洩するのを防ぐことができます。 - -**例** - -```xml - - - SSNを非表示 - (^|\D)\d{3}-\d{2}-\d{4}($|\D) - 000-00-0000 - - -``` - -**設定フィールド**: - -| Setting | Description | -| --------- | ----------------------------------- | -| `name` | ルール名(任意) | -| `regexp` | RE2 互換の正規表現(必須) | -| `replace` | 機微なデータを置換する文字列(任意、デフォルトはアスタリスク 6 個) | - -マスキングルールは、不正な形式または解析不能なクエリから機微なデータが漏洩するのを防ぐため、クエリ全体に適用されます。 - -[`system.events`](/operations/system-tables/events) テーブルには `QueryMaskingRulesMatch` というカウンタがあり、クエリマスキングルールがマッチした総回数を表します。 - -分散クエリの場合は、各サーバーを個別に設定する必要があります。そうしないと、他ノードに渡されるサブクエリはマスクされないまま保存されます。 - - -## remote_servers {#remote_servers} - -[Distributed](../../engines/table-engines/special/distributed.md) テーブルエンジンおよび `cluster` テーブル関数で使用されるクラスタの設定です。 - -**例** - -```xml - -``` - -`incl` 属性の値については、「[設定ファイル](/operations/configuration-files)」のセクションを参照してください。 - -**関連項目** - -* [skip_unavailable_shards](../../operations/settings/settings.md#skip_unavailable_shards) -* [クラスタディスカバリ](../../operations/cluster-discovery.md) -* [Replicatedデータベースエンジン](../../engines/database-engines/replicated.md) - - -## remote_url_allow_hosts {#remote_url_allow_hosts} - -URL 関連のストレージエンジンおよびテーブル関数で使用を許可するホストのリスト。 - -`\` XML タグでホストを追加する場合: - -* URL 内に記載されているものとまったく同じように指定する必要があります。名前は DNS 解決の前にチェックされます。例: `clickhouse.com` -* URL でポートが明示的に指定されている場合は、host:port の組み合わせとしてチェックされます。例: `clickhouse.com:80` -* ホストがポートなしで指定されている場合、そのホストの任意のポートが許可されます。例: `clickhouse.com` が指定されていると、`clickhouse.com:20` (FTP)、`clickhouse.com:80` (HTTP)、`clickhouse.com:443` (HTTPS) などが許可されます。 -* ホストが IP アドレスとして指定されている場合は、URL に記載されたとおりにチェックされます。例: `[2a02:6b8:a::a]`。 -* リダイレクトが存在し、リダイレクトのサポートが有効になっている場合は、すべてのリダイレクト (`Location` フィールド) がチェックされます。 - -例: - -```sql - - clickhouse.com - -``` - - -## timezone {#timezone} - -サーバーのタイムゾーンです。 - -UTC タイムゾーンまたは地理的位置を表す IANA 識別子として指定します(例: Africa/Abidjan)。 - -タイムゾーンは、DateTime 型のフィールドをテキスト形式(画面への表示やファイル出力)に変換する際の String 型との相互変換や、文字列から DateTime を取得する際に必要です。さらに、時間や日付を扱う関数が入力パラメータとしてタイムゾーンを受け取らない場合には、それらの関数の内部でもこのタイムゾーンが使用されます。 - -**例** - -```xml -Asia/Istanbul -``` - -**関連項目** - -* [session_timezone](../settings/settings.md#session_timezone) - - -## tcp_port {#tcp_port} - -TCP プロトコルでクライアントと通信するためのポート。 - -**例** - -```xml -9000 -``` - - -## tcp_port_secure {#tcp_port_secure} - -クライアントとの安全な通信に使用する TCP ポートです。[OpenSSL](#openssl) の設定と併用します。 - -**デフォルト値** - -```xml -9440 -``` - - -## mysql_port {#mysql_port} - -MySQL プロトコル経由でクライアントと通信するためのポート。 - -:::note - -* 正の整数は待ち受けるポート番号を指定する -* 空の値は、MySQL プロトコル経由でのクライアントとの通信を無効にするために使用される - ::: - -**例** - -```xml -9004 -``` - - -## postgresql_port {#postgresql_port} - -PostgreSQL プロトコル経由でクライアントと通信するためのポートです。 - -:::note - -* 正の整数は待ち受けるポート番号を指定します -* 空にすると、PostgreSQL プロトコル経由でのクライアントとの通信が無効になります - ::: - -**例** - -```xml -9005 -``` - - -## mysql_require_secure_transport {#mysql_require_secure_transport} - -true に設定した場合、[mysql_port](#mysql_port) 経由のクライアントとの通信にはセキュアな接続が必須になります。`--ssl-mode=none` オプションでの接続は拒否されます。[OpenSSL](#openssl) の設定と併用してください。 - - - -## postgresql_require_secure_transport {#postgresql_require_secure_transport} - -true に設定すると、[postgresql_port](#postgresql_port) を介したクライアントとのセキュアな通信が必須となります。`sslmode=disable` オプションでの接続は拒否されます。[OpenSSL](#openssl) の設定と併用してください。 - - - -## tmp_path {#tmp_path} - -大規模なクエリを処理するための一時データを保存する、ローカルファイルシステム上のパスです。 - -:::note - -* 一時データストレージを構成する際に使用できるオプションは、`tmp_path`、`tmp_policy`、`temporary_data_in_cache` のいずれか一つだけです。 -* 末尾のスラッシュは必須です。 - ::: - -**例** - -```xml -/var/lib/clickhouse/tmp/ -``` - - -## url_scheme_mappers {#url_scheme_mappers} - -短縮またはシンボリックな URL プレフィックスを完全な URL にマッピングするための設定です。 - -例: - -```xml - - - https://{bucket}.s3.amazonaws.com - - - https://storage.googleapis.com/{bucket} - - - https://{bucket}.oss.aliyuncs.com - - -``` - - -## user_files_path {#user_files_path} - -ユーザーファイルを格納するディレクトリです。テーブル関数 [file()](../../sql-reference/table-functions/file.md)、[fileCluster()](../../sql-reference/table-functions/fileCluster.md) で使用されます。 - -**例** - -```xml -/var/lib/clickhouse/user_files/ -``` - - -## user_scripts_path {#user_scripts_path} - -ユーザースクリプトファイルを格納するディレクトリです。[Executable User Defined Functions](/sql-reference/functions/udf#executable-user-defined-functions) で使用されます。 - -**例** - -```xml -/var/lib/clickhouse/user_scripts/ -``` - -型: - -デフォルト: - - -## user_defined_path {#user_defined_path} - -ユーザー定義ファイルを格納するディレクトリです。SQL のユーザー定義関数で使用されます。詳細は [SQL User Defined Functions](/sql-reference/functions/udf) を参照してください。 - -**例** - -```xml -/var/lib/clickhouse/user_defined/ -``` - - -## users_config {#users_config} - -次の内容を含むファイルへのパス: - -* ユーザー設定 -* アクセス権 -* 設定プロファイル -* クォータ設定 - -**例** - -```xml -users.xml -``` - - -## access_control_improvements {#access_control_improvements} - -アクセス制御システムにおける任意の改善用設定です。 - -| Setting | Description | Default | -| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| `users_without_row_policies_can_read_rows` | パーミッシブな行ポリシーを持たないユーザーが、`SELECT` クエリを使用して行を読み取れるかどうかを設定します。たとえば、ユーザー A と B がいて、行ポリシーが A に対してのみ定義されている場合、この設定が true であれば、ユーザー B はすべての行を閲覧できます。この設定が false の場合、ユーザー B はどの行も閲覧できません。 | `true` | -| `on_cluster_queries_require_cluster_grant` | `ON CLUSTER` クエリに `CLUSTER` 権限が必要かどうかを設定します。 | `true` | -| `select_from_system_db_requires_grant` | `SELECT * FROM system.` を実行する際に権限が必要かどうか(権限が不要な場合は任意のユーザーが実行可能かどうか)を設定します。true に設定した場合、このクエリには非 system テーブルと同様に `GRANT SELECT ON system.
` が必要です。例外として、いくつかの system テーブル(`tables`、`columns`、`databases`、および `one`、`contributors` のような一部の定数テーブル)は依然として全員がアクセス可能です。また、`SHOW` 権限(例: `SHOW USERS`)が付与されている場合、対応する system テーブル(つまり `system.users`)にはアクセスできます。 | `true` | -| `select_from_information_schema_requires_grant` | `SELECT * FROM information_schema.
` を実行する際に権限が必要かどうか(権限が不要な場合は任意のユーザーが実行可能かどうか)を設定します。true に設定した場合、このクエリには、通常のテーブルと同様に `GRANT SELECT ON information_schema.
` が必要です。 | `true` | -| `settings_constraints_replace_previous` | ある設定に対して設定プロファイル内で定義された制約が、その設定に対する以前の制約(他のプロファイルで定義されたもの)による動作を、新しい制約で設定されていないフィールドも含めて打ち消すかどうかを設定します。また、`changeable_in_readonly` 制約タイプを有効にします。 | `true` | -| `table_engines_require_grant` | 特定のテーブルエンジンを使用してテーブルを作成する際に、権限が必要かどうかを設定します。 | `false` | -| `role_cache_expiration_time_seconds` | ロールが最後にアクセスされてから、ロールキャッシュに保持される時間(秒)を設定します。 | `600` | - -例: - -```xml - - true - true - true - true - true - false - 600 - -``` - - -## s3queue_log {#s3queue_log} - -`s3queue_log` システムテーブルの設定です。 - - - -デフォルトの設定は次のとおりです。 - -```xml - - system -
s3queue_log
- toYYYYMM(event_date) - 7500 - -``` - - -## dead_letter_queue {#dead_letter_queue} - -'dead_letter_queue' システムテーブルの設定です。 - - - -既定の設定値は次のとおりです。 - -```xml - - system - dead_letter
- toYYYYMM(event_date) - 7500 -
-``` - - -## zookeeper {#zookeeper} - -ClickHouse が [ZooKeeper](http://zookeeper.apache.org/) クラスターと連携するための設定です。ClickHouse は、レプリケーテッドテーブルを使用する場合に、レプリカのメタデータを保存するために ZooKeeper を使用します。レプリケーテッドテーブルを使用しない場合は、このセクションのパラメーターは省略できます。 - -次の設定はサブタグで指定できます: - -| Setting | Description | -| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `node` | ZooKeeper のエンドポイント。複数のエンドポイントを設定できます。例: `example_host2181`。`index` 属性は、ZooKeeper クラスターへの接続を試行するときのノードの順序を指定します。 | -| `session_timeout_ms` | クライアントセッションの最大タイムアウト (ミリ秒単位)。 | -| `operation_timeout_ms` | 1 つの操作の最大タイムアウト (ミリ秒単位)。 | -| `root` (optional) | ClickHouse サーバーが使用する znode 群のルートとして使用される znode。 | -| `fallback_session_lifetime.min` (optional) | プライマリが利用できない場合に、フォールバックノードへの ZooKeeper セッションの存続期間に対する最小制限 (ロードバランシング)。秒単位で設定します。デフォルト: 3 時間。 | -| `fallback_session_lifetime.max` (optional) | プライマリが利用できない場合に、フォールバックノードへの ZooKeeper セッションの存続期間に対する最大制限 (ロードバランシング)。秒単位で設定します。デフォルト: 6 時間。 | -| `identity` (optional) | 要求された znode にアクセスするために ZooKeeper によって要求されるユーザーとパスワード。 | -| `use_compression` (optional) | `true` に設定すると、Keeper プロトコルでの圧縮を有効にします。 | - -また、ZooKeeper ノードの選択アルゴリズムを選択できる `zookeeper_load_balancing` 設定 (オプション) もあります: - -| Algorithm Name | Description | -| ------------------------------- | ----------------------------------------------------------------------- | -| `random` | ZooKeeper ノードの 1 つをランダムに選択します。 | -| `in_order` | 最初の ZooKeeper ノードを選択し、それが利用できない場合は 2 番目、その次へと順に選択します。 | -| `nearest_hostname` | サーバーのホスト名と最も似ているホスト名を持つ ZooKeeper ノードを選択します。ホスト名は名前プレフィックスで比較されます。 | -| `hostname_levenshtein_distance` | `nearest_hostname` と同様ですが、ホスト名をレーベンシュタイン距離で比較します。 | -| `first_or_random` | 最初の ZooKeeper ノードを選択し、それが利用できない場合は残りの ZooKeeper ノードの中からランダムに 1 つを選択します。 | -| `round_robin` | 最初の ZooKeeper ノードを選択し、再接続が発生した場合は次のノードを選択します。 | - -**設定例** - -```xml - - - example1 - 2181 - - - example2 - 2181 - - 30000 - 10000 - - /path/to/zookeeper/node - - user:password - - random - -``` - -**関連項目** - -* [レプリケーション](../../engines/table-engines/mergetree-family/replication.md) -* [ZooKeeper プログラマー向けガイド](http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html) -* [ClickHouse と ZooKeeper 間の通信をセキュアにする(オプション)](/operations/ssl-zookeeper) - - -## use_minimalistic_part_header_in_zookeeper {#use_minimalistic_part_header_in_zookeeper} - -ZooKeeper におけるデータパートヘッダーの保存方式を指定します。この設定は [`MergeTree`](/engines/table-engines/mergetree-family) ファミリーにのみ適用されます。次のいずれかで指定できます。 - -**`config.xml` ファイルの [merge_tree](#merge_tree) セクションでグローバルに指定** - -ClickHouse はサーバー上のすべてのテーブルに対してこの設定を使用します。設定はいつでも変更できます。既存のテーブルも、設定が変更されると動作が変わります。 - -**テーブルごとに指定** - -テーブル作成時に、対応する [エンジン設定](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)を指定します。この設定を持つ既存テーブルの動作は、グローバル設定が変わっても変化しません。 - -**指定可能な値** - -- `0` — 機能を無効にします。 -- `1` — 機能を有効にします。 - -[`use_minimalistic_part_header_in_zookeeper = 1`](#use_minimalistic_part_header_in_zookeeper) の場合、[replicated](../../engines/table-engines/mergetree-family/replication.md) テーブルは、データパートのヘッダーを 1 つの `znode` を用いてコンパクトに保存します。テーブルに多数のカラムが含まれる場合、この保存方式により ZooKeeper に保存されるデータ量を大幅に削減できます。 - -:::note -`use_minimalistic_part_header_in_zookeeper = 1` を適用した後は、この設定をサポートしないバージョンの ClickHouse サーバーにダウングレードすることはできません。クラスタ内のサーバーで ClickHouse をアップグレードする際は注意してください。すべてのサーバーを一度にアップグレードしないでください。ClickHouse の新バージョンは、テスト環境やクラスタ内の一部のサーバーだけで検証する方が安全です。 - -この設定で既に保存されたデータパートヘッダーは、以前の(非コンパクトな)表現に戻すことはできません。 -::: - - - -## distributed_ddl {#distributed_ddl} - -クラスタ上での[分散 DDL クエリ](../../sql-reference/distributed-ddl.md)(`CREATE`、`DROP`、`ALTER`、`RENAME`)の実行を管理します。 -[ZooKeeper](/operations/server-configuration-parameters/settings#zookeeper) が有効な場合にのみ動作します。 - -`` 内で設定可能な項目は次のとおりです: - -| Setting | Description | Default Value | -| ---------------------- | ------------------------------------------------------------------------------ | ----------------------------- | -| `path` | DDL クエリ用の `task_queue` に対して Keeper 上で使用するパス | | -| `profile` | DDL クエリの実行に使用するプロファイル | | -| `pool_size` | 同時に実行可能な `ON CLUSTER` クエリの数 | | -| `max_tasks_in_queue` | キュー内に存在できるタスクの最大数 | `1,000` | -| `task_max_lifetime` | ノードの経過時間がこの値を超えた場合にノードを削除します | `7 * 24 * 60 * 60`(秒単位で 1 週間) | -| `cleanup_delay_period` | 直近のクリーンアップから `cleanup_delay_period` 秒以上経過している場合に、新しいノードイベントを受信するとクリーンアップを開始します | `60` 秒 | - -**例** - -```xml - - - /clickhouse/task_queue/ddl - - - default - - - 1 - - - - - 604800 - - - 60 - - - 1000 - -``` - - -## access_control_path {#access_control_path} - -ClickHouse サーバーが SQL コマンドで作成したユーザーおよびロールの設定ファイルを保存するディレクトリへのパス。 - -**関連項目** - -- [アクセス制御とアカウント管理](/operations/access-rights#access-control-usage) - - - -## allow_plaintext_password {#allow_plaintext_password} - -平文パスワードタイプ(安全ではない)の使用を許可するかどうかを設定します。 - -```xml -1 -``` - - -## allow_no_password {#allow_no_password} - -安全ではないパスワード種別 `no_password` を許可するかどうかを設定します。 - -```xml -1 -``` - - -## allow_implicit_no_password {#allow_implicit_no_password} - -明示的に 'IDENTIFIED WITH no_password' が指定されていない限り、パスワードなしのユーザーは作成できません。 - -```xml -1 -``` - - -## default_session_timeout {#default_session_timeout} - -セッションのデフォルトのタイムアウト時間(秒)。 - -```xml -60 -``` - - -## default_password_type {#default_password_type} - -`CREATE USER u IDENTIFIED BY 'p'` のようなクエリにおいて、自動的に設定されるパスワードの種類を指定します。 - -指定可能な値は次のとおりです: - -* `plaintext_password` -* `sha256_password` -* `double_sha1_password` -* `bcrypt_password` - -```xml -sha256_password -``` - - -## user_directories {#user_directories} - -次の設定を含む設定ファイルのセクションです: - -* 事前定義されたユーザーを含む設定ファイルへのパス。 -* SQL コマンドで作成されたユーザーが保存されるフォルダへのパス。 -* SQL コマンドで作成されたユーザーが保存およびレプリケートされる ZooKeeper ノードパス。 - -このセクションが指定されている場合、[users_config](/operations/server-configuration-parameters/settings#users_config) および [access_control_path](../../operations/server-configuration-parameters/settings.md#access_control_path) のパスは使用されません。 - -`user_directories` セクションには任意の数の項目を含めることができ、項目の順序は優先順位を表します (上にある項目ほど優先されます)。 - -**例** - -```xml - - - /etc/clickhouse-server/users.xml - - - /var/lib/clickhouse/access/ - - -``` - -ユーザー、ロール、行ポリシー、クォータ、プロファイルは ZooKeeperに保存することもできます。 - -```xml - - - /etc/clickhouse-server/users.xml - - - /clickhouse/access/ - - -``` - -`memory` セクションも定義できます。これは情報をディスクに書き込まず、メモリ上にのみ保持することを意味します。また、`ldap` セクションは情報を LDAP サーバー上に保存することを意味します。 - -ローカルに定義されていないユーザーのリモートユーザーディレクトリとして LDAP サーバーを追加するには、以下の設定を持つ単一の `ldap` セクションを定義します。 - -| Setting | Description | -| -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `server` | `ldap_servers` 設定セクションで定義された LDAP サーバー名のいずれか。必須パラメータであり、空にすることはできません。 | -| `roles` | LDAP サーバーから取得された各ユーザーに割り当てられる、ローカルに定義されたロールの一覧を含むセクション。ロールが一切指定されていない場合、ユーザーは認証後も一切の操作を実行できません。認証時点で一覧内のロールのいずれかがローカルに定義されていない場合、その認証試行は指定されたパスワードが誤っている場合と同様に失敗します。 | - -**例** - -```xml - - my_ldap_server - - - - - -``` - - -## top_level_domains_list {#top_level_domains_list} - -追加で登録するカスタムトップレベルドメインのリストを定義します。各エントリは `/path/to/file` という形式です。 - -例えば: - -```xml - - /path/to/public_suffix_list.dat - -``` - -See also: - -* 関数 [`cutToFirstSignificantSubdomainCustom`](../../sql-reference/functions/url-functions.md/#cutToFirstSignificantSubdomainCustom) およびそのバリエーション。\ - カスタムの TLD リスト名を受け取り、最初の有意なサブドメインまでのトップレベルサブドメインを含むドメイン部分を返します。 - - -## proxy {#proxy} - -HTTP および HTTPS リクエスト向けのプロキシサーバーを定義します。現在、S3 ストレージ、S3 テーブル関数、および URL 関数でサポートされています。 - -プロキシサーバーを定義する方法は 3 通りあります。 - -* 環境変数 -* プロキシリスト -* リモートプロキシリゾルバー - -`no_proxy` を使用することで、特定のホストをプロキシサーバー経由の通信から除外することもできます。 - -**環境変数** - -`http_proxy` および `https_proxy` 環境変数を使用すると、 -プロトコルごとにプロキシサーバーを指定できます。システム上で設定されていれば、そのまま問題なく動作します。 - -特定のプロトコルに対してプロキシサーバーが 1 つだけ存在し、そのプロキシサーバーが変更されない場合には、これが最も簡単な方法です。 - -**プロキシリスト** - -この方法では、プロトコルごとに 1 つ以上の -プロキシサーバーを指定できます。複数のプロキシサーバーが定義されている場合、 -ClickHouse は異なるプロキシをラウンドロビン方式で使用し、サーバー間で -負荷を分散します。特定のプロトコルに複数のプロキシサーバーがあり、プロキシサーバーのリストが変わらない場合には、これが最も簡単な方法です。 - -**設定テンプレート** - -```xml - - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - -以下のタブで親フィールドを選択して、その子要素を確認してください: - - - - | フィールド | 説明 | - | --------- | --------------------- | - | `` | 1 つ以上の HTTP プロキシのリスト | - | `` | 1 つ以上の HTTPS プロキシのリスト | - - - - | フィールド | 説明 | - | ------- | --------- | - | `` | プロキシの URI | - - - -**リモートプロキシリゾルバー** - -プロキシサーバーが動的に変更される場合があります。 -その場合は、リゾルバーのエンドポイントを定義できます。ClickHouse は -そのエンドポイントに対して空の GET リクエストを送信し、リモートリゾルバーはプロキシホストを返す必要があります。 -ClickHouse はそれを使用し、次のテンプレートでプロキシ URI を構成します: `\{proxy_scheme\}://\{proxy_host\}:{proxy_port}` - -**設定テンプレート** - -```xml - - - - http://resolver:8080/hostname - http - 80 - 10 - - - - - - http://resolver:8080/hostname - http - 3128 - 10 - - - - -``` - -下のタブから親フィールドを選択すると、その子要素が表示されます: - - - - | Field | Description | - | --------- | ------------------------ | - | `` | 1 つ以上の resolver からなるリスト* | - | `` | 1 つ以上の resolver からなるリスト* | - - - - | Field | Description | - | ------------ | -------------------------- | - | `` | resolver のエンドポイントとその他の詳細情報 | - - :::note - 複数の `` 要素を指定できますが、特定のプロトコルに対して使用されるのは - 最初の `` のみです。そのプロトコルに対するその他の `` - 要素は無視されます。つまり、ロードバランシングが必要な場合は、 - リモート側の resolver で実装する必要があります。 - ::: - - - - | Field | Description | - | -------------------- | ---------------------------------------------------------------------------------------------------------------------- | - | `` | プロキシ resolver の URI | - | `` | 最終的なプロキシ URI のプロトコル。`http` または `https` のいずれかを指定できます。 | - | `` | プロキシ resolver のポート番号 | - | `` | resolver から取得した値を ClickHouse がキャッシュする秒数。この値を `0` に設定すると、ClickHouse はすべての HTTP または HTTPS リクエストごとに resolver に問い合わせを行います。 | - - - -**優先順位** - -プロキシ設定は次の順序で決定されます。 - - -| 順序 | 設定 | -|------|--------------------------| -| 1. | リモートプロキシリゾルバ | -| 2. | プロキシリスト | -| 3. | 環境変数 | - -ClickHouse は、リクエストプロトコルに対して最も優先度の高いリゾルバタイプを確認します。それが定義されていない場合は、 -環境リゾルバに到達するまで、次に優先度の高いリゾルバタイプを順に確認します。 -これにより、異なる種類のリゾルバタイプを組み合わせて使用することも可能になります。 - - - -## disable_tunneling_for_https_requests_over_http_proxy {#disable_tunneling_for_https_requests_over_http_proxy} - -デフォルトでは、`HTTP` プロキシ経由で `HTTPS` リクエストを行う際にトンネリング(`HTTP CONNECT`)が利用されます。この設定でトンネリングを無効化できます。 - -**no_proxy** - -デフォルトでは、すべてのリクエストがプロキシを経由します。特定のホストについてプロキシを無効化するには、`no_proxy` 変数を設定する必要があります。 -これは、list resolver と remote resolver では `` 句の中で、environment resolver では環境変数として設定できます。 -IP アドレス、ドメイン、サブドメイン、および完全なバイパス用の `'*'` ワイルドカードをサポートします。先頭のドットは、curl と同様に削除されます。 - -**Example** - -次の設定では、`clickhouse.cloud` とそのすべてのサブドメイン(例: `auth.clickhouse.cloud`)へのリクエストはプロキシをバイパスします。 -GitLab についても同様で、先頭にドットが付いていても同じ挙動になります。`gitlab.com` と `about.gitlab.com` の両方がプロキシをバイパスします。 - -```xml - - clickhouse.cloud,.gitlab.com - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - - -## workload_path {#workload_path} - -すべての `CREATE WORKLOAD` クエリおよび `CREATE RESOURCE` クエリの保存先として使用されるディレクトリです。デフォルトでは、サーバーの作業ディレクトリ配下の `/workload/` フォルダが使用されます。 - -**例** - -```xml -/var/lib/clickhouse/workload/ -``` - -**関連項目** - -* [ワークロード階層](/operations/workload-scheduling.md#workloads) -* [workload_zookeeper_path](#workload_zookeeper_path) - - -## workload_zookeeper_path {#workload_zookeeper_path} - -ZooKeeper ノードへのパスです。すべての `CREATE WORKLOAD` および `CREATE RESOURCE` クエリの保存先として使用されます。一貫性を保つため、すべての SQL 定義はこの単一の znode に値として保存されます。デフォルトでは ZooKeeper は使用されず、定義は [ディスク](#workload_path) 上に保存されます。 - -**例** - -```xml -/clickhouse/workload/definitions.sql -``` - -**関連項目** - -* [ワークロード階層](/operations/workload-scheduling.md#workloads) -* [workload_path](#workload_path) - - -## zookeeper_log {#zookeeper_log} - -[`zookeeper_log`](/operations/system-tables/zookeeper_log) システムテーブルに関する設定です。 - -次の設定はサブタグで指定できます: - - - -**例** - -```xml - - - system - zookeeper_log
- 7500 - event_date + INTERVAL 1 WEEK DELETE -
-
-``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md deleted file mode 100644 index 2a6906ffd81..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md +++ /dev/null @@ -1,17 +0,0 @@ -The following settings can be configured by sub-tags: - -| Setting | Description | Default | Note | -|------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|--------------------------------------------------------------------------------------------------------------------| -| `database` | データベース名。 | | | -| `table` | システムテーブル名。 | | | -| `engine` | システムテーブルに対する [MergeTree エンジン定義](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。 | | `partition_by` または `order_by` が定義されている場合は使用できません。指定しない場合はデフォルトで `MergeTree` が選択されます | -| `partition_by` | システムテーブルに対する[カスタムパーティショニングキー](../../../engines/table-engines/mergetree-family/custom-partitioning-key.md)。 | | システムテーブルに `engine` を指定する場合、`partition_by` パラメータは `engine` の設定内で直接指定する必要があります | -| `ttl` | テーブルの [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) を指定します。 | | システムテーブルに `engine` を指定する場合、`ttl` パラメータは `engine` の設定内で直接指定する必要があります | -| `order_by` | システムテーブルに対する[カスタムソートキー](../../../engines/table-engines/mergetree-family/mergetree.md#order_by)。`engine` が定義されている場合は使用できません。 | | システムテーブルに `engine` を指定する場合、`order_by` パラメータは `engine` の設定内で直接指定する必要があります | -| `storage_policy` | テーブルに使用するストレージポリシー名(任意)。 | | システムテーブルに `engine` を指定する場合、`storage_policy` パラメータは `engine` の設定内で直接指定する必要があります | -| `settings` | MergeTree の動作を制御する[追加パラメータ](../../../engines/table-engines/mergetree-family/mergetree.md/#settings)(任意)。 | | システムテーブルに `engine` を指定する場合、`settings` パラメータは `engine` の設定内で直接指定する必要があります | -| `flush_interval_milliseconds` | メモリ上のバッファからテーブルへデータをフラッシュする間隔。 | `7500` | | -| `max_size_rows` | ログの最大行数。未フラッシュのログ数が `max_size_rows` に達すると、ログがディスクにダンプされます。 | `1048576` | | -| `reserved_size_rows` | ログ用に事前確保されるメモリ容量(行数)。 | `8192` | | -| `buffer_size_rows_flush_threshold` | 行数のしきい値。このしきい値に達すると、バックグラウンドでログをディスクへフラッシュする処理が起動します。 | `max_size_rows / 2` | | -| `flush_on_crash` | クラッシュ発生時にログをディスクへダンプするかどうかを設定します。 | `false` | | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md deleted file mode 100644 index 240c2837cac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: 'このセクションでは、セッション単位やクエリ単位では変更できないサーバー設定について説明します。' -keywords: ['グローバルなサーバー設定'] -sidebar_label: 'サーバー設定' -sidebar_position: 57 -slug: /operations/server-configuration-parameters/settings -title: 'サーバー設定' -doc_type: 'reference' ---- - -{/* NOTE: このファイル内の設定は自動生成されています。 - 詳細については、次を参照してください: ["Generating documentation from source code"](https://github.com/ClickHouse/clickhouse-docs/blob/main/contribute/autogenerated-documentation-from-source.md) - */ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md deleted file mode 100644 index bac012c4d29..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md +++ /dev/null @@ -1,188 +0,0 @@ ---- -description: 'コンポーザブルプロトコルを使用すると、ClickHouse サーバーへの TCP アクセスの構成をより柔軟に行えます。' -sidebar_label: 'コンポーザブルプロトコル' -sidebar_position: 64 -slug: /operations/settings/composable-protocols -title: 'コンポーザブルプロトコル' -doc_type: 'reference' ---- - -# 合成可能なプロトコル {#composable-protocols} - -## 概要 {#overview} - -Composable プロトコルを使用すると、ClickHouse サーバーへの TCP アクセスをより柔軟に設定できます。この設定は、従来の設定と併用することも、置き換えることもできます。 - -## コンポーザブルプロトコルの設定 {#composable-protocols-section-is-denoted-as-protocols-in-configuration-xml} - -コンポーザブルプロトコルは XML 設定ファイルで設定できます。`protocols` セクションは、XML 設定ファイル内で `protocols` タグで表されます。 - -```xml - - - -``` - - -### プロトコルレイヤーの設定 {#basic-modules-define-protocol-layers} - -プロトコルレイヤーは基本モジュールを使用して定義できます。たとえば、HTTP レイヤーを定義するには、`protocols` セクションに新しい基本モジュールを追加します。 - -```xml - - - - - http - - - -``` - -モジュールは次の項目で構成できます: - -* `plain_http` - 別のレイヤーから参照される名前 -* `type` - データを処理するためにインスタンス化されるプロトコルハンドラを示します。 - あらかじめ定義されているプロトコルハンドラは次のとおりです: - * `tcp` - ネイティブな ClickHouse プロトコルハンドラ - * `http` - HTTP ClickHouse プロトコルハンドラ - * `tls` - TLS 暗号化レイヤー - * `proxy1` - PROXYv1 レイヤー - * `mysql` - MySQL 互換プロトコルハンドラ - * `postgres` - PostgreSQL 互換プロトコルハンドラ - * `prometheus` - Prometheus プロトコルハンドラ - * `interserver` - ClickHouse インターサーバープロトコルハンドラ - -:::note -`gRPC` プロトコルハンドラは `Composable protocols` では実装されていません。 -::: - - -### エンドポイントの設定 {#endpoint-ie-listening-port-is-denoted-by-port-and-optional-host-tags} - -エンドポイント(待ち受けポート)は `` と、任意の `` タグで指定します。 -例えば、先ほど追加した HTTP レイヤーに対してエンドポイントを設定するには、 -次のように設定を変更します。 - -```xml - - - - - http - - 127.0.0.1 - 8123 - - - - -``` - -`` タグが省略された場合は、ルート設定の `` が使用されます。 - - -### レイヤーシーケンスの設定 {#layers-sequence-is-defined-by-impl-tag-referencing-another-module} - -レイヤーシーケンスは `` タグを使用し、別のモジュールを参照することで定義します。例えば、`plain_http` モジュールの上に TLS レイヤーを構成するには、設定を次のようにさらに変更できます。 - -```xml - - - - - http - - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### レイヤーにエンドポイントを関連付ける {#endpoint-can-be-attached-to-any-layer} - -エンドポイントは任意のレイヤーに関連付けることができます。たとえば、HTTP(ポート 8123)および HTTPS(ポート 8443)向けのエンドポイントを定義できます。 - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### 追加のエンドポイントの定義 {#additional-endpoints-can-be-defined-by-referencing-any-module-and-omitting-type-tag} - -追加のエンドポイントは、任意のモジュールを参照し、`` タグを省略することで定義できます。たとえば、`plain_http` モジュールに対する `another_http` エンドポイントを次のように定義できます。 - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - - plain_http - 127.0.0.1 - 8223 - - - -``` - - -### 追加のレイヤーパラメータの指定 {#some-modules-can-contain-specific-for-its-layer-parameters} - -一部のモジュールには、追加のレイヤーパラメータが含まれる場合があります。たとえば、TLS レイヤーでは -秘密鍵ファイル(`privateKeyFile`)および証明書ファイル(`certificateFile`)を -次のように指定できます。 - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - another_server.key - another_server.crt - - - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md deleted file mode 100644 index 05a63ff4219..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md +++ /dev/null @@ -1,181 +0,0 @@ ---- -description: '`user.xml` 設定ファイルの `profiles` セクションで設定に対する制約を定義でき、これによりユーザーが `SET` クエリを使って一部の設定を変更することを禁止できます。' -sidebar_label: '設定への制約' -sidebar_position: 62 -slug: /operations/settings/constraints-on-settings -title: '設定への制約' -doc_type: 'reference' ---- - - - -# 設定上の制約事項 {#constraints-on-settings} - - - -## 概要 {#overview} - -ClickHouse における設定に対する「制約」とは、その設定に付与できる制限やルールを指します。これらの制約を適用することで、データベースの安定性、セキュリティ、および予測可能な動作を維持できます。 - - - -## 制約の定義 {#defining-constraints} - -設定に対する制約は、`user.xml` 設定ファイルの `profiles` セクションで定義できます。これにより、ユーザーが [`SET`](/sql-reference/statements/set) ステートメントを使用して一部の設定を変更できないようにします。 - -制約は次のように定義します。 - -```xml - - - - <設定名_1> - lower_boundary - - <設定名_2> - upper_boundary - - <設定名_3> - lower_boundary - upper_boundary - - <設定名_4> - - - <設定名_5> - lower_boundary - upper_boundary - - - <設定名_6> - lower_boundary - upper_boundary - value1 - value2 - value3 - - - - - -``` - -ユーザーが制約に違反しようとすると例外がスローされ、設定は変更されずにそのまま維持されます。 - - -## 制約の種類 {#types-of-constraints} - -ClickHouse でサポートされている制約には、いくつかの種類があります。 - -* `min` -* `max` -* `disallowed` -* `readonly`(エイリアス `const`) -* `changeable_in_readonly` - -`min` と `max` 制約は、数値設定に対する下限および上限を指定し、互いに組み合わせて使用できます。 - -`disallowed` 制約は、特定の設定に対して許可してはならない値を指定するために使用できます。 - -`readonly` または `const` 制約は、ユーザーが対応する設定を一切変更できないことを示します。 - -`changeable_in_readonly` 制約タイプを使用すると、`readonly` 設定が `1` に設定されている場合でも、`min`/`max` の範囲内であればその設定を変更できます。それ以外の設定は、`readonly=1` モードでは変更できません。 - -:::note -`changeable_in_readonly` は、`settings_constraints_replace_previous` -が有効化されている場合にのみサポートされます。 - -```xml - - true - -``` - -::: - - -## 複数の制約プロファイル {#multiple-constraint-profiles} - -ユーザーに対して複数のプロファイルがアクティブな場合、それらの制約はマージされます。 -マージ方法は `settings_constraints_replace_previous` によって決まります: -- **true** (推奨): 同じ設定に対する制約はマージ時に置き換えられ、最後に適用される制約のみが使用され、それ以前のものはすべて無視されます。 - これには、新しい制約で設定されていないフィールドも含まれます。 -- **false** (デフォルト): 同じ設定に対する制約は、未設定の種類の制約は前のプロファイルから引き継ぎ、設定されている種類の制約は新しいプロファイルの値で置き換える形でマージされます。 - - - -## 読み取り専用モード {#read-only} - -読み取り専用モードは `readonly` 設定によって有効になります。これは -`readonly` 制約タイプと混同しないでください。 - -* `readonly=0`: 読み取り専用に関する制限はありません。 -* `readonly=1`: 読み取りクエリのみ許可され、`changeable_in_readonly` が設定されていない限り、設定を変更できません。 -* `readonly=2`: 読み取りクエリのみ許可されますが、`readonly` 設定自体を除き、その他の設定を変更できます。 - -### 例 {#example-read-only} - -`users.xml` に次の行が含まれているとします。 - -```xml - - - 10000000000 - 0 - ... - - - 5000000000 - 20000000000 - - - - - - - -``` - -以下のクエリはすべて例外をスローします。 - -```sql -SET max_memory_usage=20000000001; -SET max_memory_usage=4999999999; -SET force_index_by_date=1; -``` - -```text -Code: 452, e.displayText() = DB::Exception: max_memory_usage は 20000000000 を超える値に設定してはいけません。 -Code: 452, e.displayText() = DB::Exception: max_memory_usage は 5000000000 未満の値に設定してはいけません。 -Code: 452, e.displayText() = DB::Exception: force_index_by_date は変更してはいけません。 -``` - -:::note -`default` プロファイルは特別に扱われます。`default` プロファイルに定義されたすべての制約はデフォルトの制約となり、各ユーザーに対して明示的に上書きされるまで、すべてのユーザーに対して制約として適用されます。 -::: - - -## MergeTree 設定に対する制約 {#constraints-on-merge-tree-settings} - -[MergeTree 設定](merge-tree-settings.md) に対して制約を設定できます。 -これらの制約は、MergeTree エンジンを使用するテーブルを作成するとき、 -またはそのストレージ設定を変更するときに適用されます。 - -`` セクション内で参照する場合は、 -MergeTree 設定名の前に `merge_tree_` プレフィックスを付ける必要があります。 - -### 例 {#example-mergetree} - -明示的に `storage_policy` を指定して新しいテーブルを作成できないようにすることができます。 - -```xml - - - - - - - - - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md deleted file mode 100644 index 347062ec98e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'クエリごとにより柔軟なメモリ制限を設定できるようにすることを目的とした実験的な手法。' -slug: /operations/settings/memory-overcommit -title: 'メモリオーバーコミット' -doc_type: 'reference' ---- - - - -# メモリオーバーコミット {#memory-overcommit} - -メモリオーバーコミットは、クエリに対してより柔軟にメモリ制限を設定できるようにすることを目的とした実験的な手法です。 - -この手法の考え方は、クエリが使用できるメモリの保証量を表現する設定を導入することにあります。 -メモリオーバーコミットが有効な状態でメモリ制限に達した場合、ClickHouse は最もオーバーコミットしているクエリを選択し、そのクエリを強制終了することでメモリの解放を試みます。 - -メモリ制限に達した場合、クエリは新しいメモリの割り当てを試みている間、一定時間待機します。 -タイムアウトまでにメモリが解放されれば、そのクエリは実行を継続します。 -そうでない場合は例外がスローされ、そのクエリは強制終了されます。 - -停止または強制終了するクエリの選択は、どのメモリ制限に達したかに応じて、グローバルまたはユーザーのオーバーコミットトラッカーのいずれかによって行われます。 -オーバーコミットトラッカーが停止させるクエリを選択できない場合、MEMORY_LIMIT_EXCEEDED 例外がスローされます。 - - - -## ユーザーオーバーコミットトラッカー {#user-overcommit-tracker} - -ユーザーオーバーコミットトラッカーは、ユーザーのクエリの中から、最もオーバーコミット率が高いクエリを特定します。 -クエリのオーバーコミット率は、割り当てられたバイト数を `memory_overcommit_ratio_denominator_for_user` 設定値で割ることで計算されます。 - -そのクエリに対する `memory_overcommit_ratio_denominator_for_user` がゼロの場合、オーバーコミットトラッカーはそのクエリを選択しません。 - -待機タイムアウトは、`memory_usage_overcommit_max_wait_microseconds` 設定で指定されます。 - -**例** - -```sql -SELECT number FROM numbers(1000) GROUP BY number SETTINGS memory_overcommit_ratio_denominator_for_user=4000, memory_usage_overcommit_max_wait_microseconds=500 -``` - - -## グローバルオーバーコミットトラッカー {#global-overcommit-tracker} - -グローバルオーバーコミットトラッカーは、すべてのクエリの一覧の中から、オーバーコミット率が最大のクエリを検出します。 -ここでのオーバーコミット率は、割り当てられたバイト数を設定値 `memory_overcommit_ratio_denominator` で割った値として計算されます。 - -そのクエリの `memory_overcommit_ratio_denominator` が 0 に設定されている場合、オーバーコミットトラッカーはそのクエリを選択しません。 - -待機タイムアウトは、設定ファイル内のパラメータ `memory_usage_overcommit_max_wait_microseconds` によって設定されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md deleted file mode 100644 index 8e2fabad026..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '`system.merge_tree_settings` に含まれる MergeTree の設定' -slug: /operations/settings/merge-tree-settings -title: 'MergeTree テーブルの設定' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import BetaBadge from '@theme/badges/BetaBadge'; -import SettingsInfoBlock from '@theme/SettingsInfoBlock/SettingsInfoBlock'; -import VersionHistory from '@theme/VersionHistory/VersionHistory'; - -システムテーブル `system.merge_tree_settings` には、グローバルに設定されている MergeTree の設定が表示されます。 - -MergeTree の設定は、サーバー設定ファイルの `merge_tree` セクションで設定するか、`CREATE TABLE` 文の `SETTINGS` 句で個々の `MergeTree` テーブルごとに指定できます。 - -設定 `max_suspicious_broken_parts` をカスタマイズする例: - -すべての `MergeTree` テーブルに対するデフォルト値をサーバー設定ファイルで設定します: - -```text - - 5 - -``` - -特定のテーブル用の設定: - -```sql -CREATE TABLE tab -( - `A` Int64 -) -ENGINE = MergeTree -ORDER BY tuple() -SETTINGS max_suspicious_broken_parts = 500; -``` - -特定のテーブルの設定は `ALTER TABLE ... MODIFY SETTING` を使用して変更します。 - -```sql -ALTER TABLE tab MODIFY SETTING max_suspicious_broken_parts = 100; - --- グローバルデフォルトにリセット(system.merge_tree_settings の値) -ALTER TABLE tab RESET SETTING max_suspicious_broken_parts; -``` - -## MergeTree の設定 {#mergetree-settings} - -{/* 以下の設定は、次のスクリプトによって自動生成されたものです - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/settings/autogenerate-settings.sh - */ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/overview.md deleted file mode 100644 index 4b2953cb1b6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/overview.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '設定の概要ページ。' -sidebar_position: 1 -slug: /operations/settings/overview -title: '設定の概要' -doc_type: 'reference' ---- - - - -# 設定の概要 {#settings-overview} - - - -## 概要 {#overview} - -:::note -XML ベースの設定プロファイルおよび [設定ファイル](/operations/configuration-files) は、現在 ClickHouse Cloud ではサポートされていません。ClickHouse Cloud サービスの設定を指定するには、[SQL ベースの設定プロファイル](/operations/access-rights#settings-profiles-management) を使用する必要があります。 -::: - -ClickHouse の設定には、大きく分けて 2 つのグループがあります。 - -- グローバルサーバー設定 -- セッション設定 - -両者の主な違いは、グローバルサーバー設定は ClickHouse サーバー全体に適用されるのに対し、セッション設定はユーザーセッションや個々のクエリに適用される点です。 - - - -## 既定値以外の設定の確認 {#see-non-default-settings} - -既定値から変更されている設定を表示するには、`system.settings` テーブルをクエリします。 - -```sql -SELECT name, value FROM system.settings WHERE changed -``` - -設定がデフォルト値からまったく変更されていない場合、ClickHouse は何も返しません。 - -特定の設定の値を確認するには、クエリ内でその設定の `name` を指定します。 - -```sql -SELECT name, value FROM system.settings WHERE name = 'max_threads' -``` - -次のような結果が得られます: - -```response -┌─name────────┬─value─────┐ -│ max_threads │ 'auto(8)' │ -└─────────────┴───────────┘ - -1 行の結果。経過時間: 0.002 秒。 -``` - - -## 関連情報 {#further-reading} - -- ClickHouse サーバーをグローバルレベルで構成する方法の詳細については、[グローバルサーバー設定](/operations/server-configuration-parameters/settings.md)を参照してください。 -- ClickHouse サーバーをセッションレベルで構成する方法の詳細については、[セッション設定](/operations/settings/settings-query-level.md)を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md deleted file mode 100644 index 8a0fa39d9b9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'クエリ実行権限の設定。' -sidebar_label: 'クエリ権限' -sidebar_position: 58 -slug: /operations/settings/permissions-for-queries -title: 'クエリ権限' -doc_type: 'reference' ---- - - - -# クエリの権限 {#permissions-for-queries} - -ClickHouse におけるクエリは、次のいくつかの種類に分類されます。 - -1. データ読み取りクエリ: `SELECT`, `SHOW`, `DESCRIBE`, `EXISTS`。 -2. データ書き込みクエリ: `INSERT`, `OPTIMIZE`。 -3. 設定変更クエリ: `SET`, `USE`。 -4. [DDL](https://en.wikipedia.org/wiki/Data_definition_language) クエリ: `CREATE`, `ALTER`, `RENAME`, `ATTACH`, `DETACH`, `DROP`, `TRUNCATE`。 -5. `KILL QUERY`。 - -次の設定によって、クエリ種別ごとにユーザー権限を制御します。 - - - -## readonly {#readonly} -データの読み取り・書き込みおよび設定変更を行うクエリに対する権限を制限します。 - -1 に設定した場合、以下が許可されます: - -- あらゆる種類の読み取りクエリ(`SELECT` およびそれと同等のクエリなど)。 -- セッションコンテキストのみを変更するクエリ(`USE` など)。 - -2 に設定した場合、上記に加えて以下が許可されます: -- `SET` および `CREATE TEMPORARY TABLE` - - :::tip - `EXISTS`、`DESCRIBE`、`EXPLAIN`、`SHOW PROCESSLIST` などのクエリは、システムテーブルに対して `SELECT` を実行しているだけなので、`SELECT` と同等です。 - ::: - -取りうる値: - -- 0 — 読み取り、書き込み、および設定変更クエリが許可されます。 -- 1 — データの読み取りクエリのみが許可されます。 -- 2 — データの読み取りおよび設定変更クエリが許可されます。 - -デフォルト値: 0 - -:::note -`readonly = 1` を設定すると、ユーザーは現在のセッションで `readonly` および `allow_ddl` の設定を変更できません。 - -[HTTP インターフェイス](../../interfaces/http.md)で `GET` メソッドを使用する場合、`readonly = 1` が自動的に設定されます。データを変更するには、`POST` メソッドを使用してください。 - -`readonly = 1` を設定すると、ユーザーは設定を変更できなくなります。特定の設定のみの変更を禁止することもできます。また、`readonly = 1` の制約下で特定の設定のみの変更を許可することもできます。詳細については、[設定に対する制約](../../operations/settings/constraints-on-settings.md) を参照してください。 -::: - - - -## allow_ddl {#allow_ddl} - -[DDL](https://en.wikipedia.org/wiki/Data_definition_language) クエリを許可するかどうかを制御します。 - -取り得る値: - -- 0 — DDL クエリは許可されません。 -- 1 — DDL クエリは許可されます。 - -デフォルト値: 1 - -:::note -現在のセッションで `allow_ddl = 0` の場合、`SET allow_ddl = 1` を実行することはできません。 -::: - -:::note KILL QUERY -`KILL QUERY` は、readonly と allow_ddl の設定値の組み合わせに関わらず実行できます。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md deleted file mode 100644 index 8c44f02a79c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -description: 'クエリの複雑さを制限するための設定。' -sidebar_label: 'クエリの複雑さの制限' -sidebar_position: 59 -slug: /operations/settings/query-complexity -title: 'クエリの複雑さの制限' -doc_type: 'reference' ---- - - - -# クエリの複雑さに関する制限 {#restrictions-on-query-complexity} - - - -## 概要 {#overview} - -[settings](/operations/settings/overview) の一部として、ClickHouse では -クエリの複雑さに関する制限を設けることができます。これにより、潜在的にリソース負荷の高いクエリからシステムを保護し、 -特にユーザーインターフェイスを使用する場合に、より安全で予測可能な -実行を実現できます。 - -ほとんどすべての制限は `SELECT` クエリにのみ適用され、分散クエリ処理の場合には、 -各サーバーごとに個別に制限が適用されます。 - -ClickHouse は一般的に、各行ごとに制限をチェックするのではなく、 -データパーツが完全に処理された後でのみ制限をチェックします。 -このため、パーツの処理中に制限違反が発生する状況が起こり得ます。 - - - -## `overflow_mode` 設定 {#overflow_mode_setting} - -ほとんどの制限には `overflow_mode` 設定もあり、制限を超過した場合に何が起こるかを定義します。`overflow_mode` には次の 2 つの値を設定できます: -- `throw`: 例外をスローする(デフォルト)。 -- `break`: クエリの実行を停止し、ソースデータが尽きたかのように途中までの結果を返します。 - - - -## `group_by_overflow_mode` 設定 {#group_by_overflow_mode_settings} - -`group_by_overflow_mode` 設定には `any` という値もあります: -- `any` : 集合にすでに含まれているキーは集計を継続しますが、新しいキーは集合に追加しません。 - - - -## 設定一覧 {#relevant-settings} - -以下の設定は、クエリの複雑さに対する制限を適用するために使用されます。 - -:::note -「〜の最大量」に対する制限は値として `0` を指定でき、 -その場合は「無制限」であることを意味します。 -::: - - - -| 設定 | 概要 | -| ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- | -| [`max_memory_usage`](/operations/settings/settings#max_memory_usage) | 単一サーバーでクエリを実行する際に使用できる RAM の最大量。 | -| [`max_memory_usage_for_user`](/operations/settings/settings#max_memory_usage_for_user) | 単一サーバーでユーザーのクエリを実行する際に使用できる RAM の最大容量。 | -| [`max_rows_to_read`](/operations/settings/settings#max_rows_to_read) | クエリ実行時にテーブルから読み取ることができる行数の上限。 | -| [`max_bytes_to_read`](/operations/settings/settings#max_bytes_to_read) | クエリ実行時にテーブルから読み取ることができる非圧縮データの最大バイト数。 | -| [`read_overflow_mode_leaf`](/operations/settings/settings#read_overflow_mode_leaf) | 読み取りデータ量がリーフのいずれかの制限値を超えた場合の挙動を設定します | -| [`max_rows_to_read_leaf`](/operations/settings/settings#max_rows_to_read_leaf) | 分散クエリ実行時にリーフノード上のローカルテーブルから読み出せる最大行数 | -| [`max_bytes_to_read_leaf`](/operations/settings/settings#max_bytes_to_read_leaf) | 分散クエリ実行時に、リーフノード上のローカルテーブルから読み取ることのできる非圧縮データの最大バイト数。 | -| [`read_overflow_mode_leaf`](/docs/operations/settings/settings#read_overflow_mode_leaf) | 読み取られるデータ量がいずれかのリーフ上限を超えた場合の動作を設定します。 | -| [`max_rows_to_group_by`](/operations/settings/settings#max_rows_to_group_by) | 集約で受信した一意キーの最大数。 | -| [`group_by_overflow_mode`](/operations/settings/settings#group_by_overflow_mode) | 集約における一意キーの数が制限を超えた場合の動作を設定します | -| [`max_bytes_before_external_group_by`](/operations/settings/settings#max_bytes_before_external_group_by) | `GROUP BY` 句の外部メモリでの実行を有効または無効にします。 | -| [`max_bytes_ratio_before_external_group_by`](/operations/settings/settings#max_bytes_ratio_before_external_group_by) | 使用可能メモリのうち、`GROUP BY` に使用を許可する割合です。この割合に達すると、集約には外部メモリが使用されます。 | -| [`max_bytes_before_external_sort`](/operations/settings/settings#max_bytes_before_external_sort) | 外部メモリでの `ORDER BY` 句の実行を有効化または無効化します。 | -| [`max_bytes_ratio_before_external_sort`](/operations/settings/settings#max_bytes_ratio_before_external_sort) | 利用可能メモリのうち、`ORDER BY` に使用を許可する比率。この比率に達すると、外部ソートが使用されます。 | -| [`max_rows_to_sort`](/operations/settings/settings#max_rows_to_sort) | ソート前の行数の上限。ソート時のメモリ使用量を制限するために利用します。 | -| [`max_bytes_to_sort`](/operations/settings/settings#max_rows_to_sort) | ソートを実行する前の最大バイト数。 | -| [`sort_overflow_mode`](/operations/settings/settings#sort_overflow_mode) | ソート前に受信した行数がいずれかの制限を超えた場合の挙動を設定します。 | -| [`max_result_rows`](/operations/settings/settings#max_result_rows) | 結果の行数を制限します。 | -| [`max_result_bytes`](/operations/settings/settings#max_result_bytes) | 結果サイズをバイト数(圧縮前)で制限します | -| [`result_overflow_mode`](/operations/settings/settings#result_overflow_mode) | 結果の量がいずれかの上限を超えた場合の動作を設定します。 | -| [`max_execution_time`](/operations/settings/settings#max_execution_time) | クエリの最大実行時間(秒)。 | -| [`timeout_overflow_mode`](/operations/settings/settings#timeout_overflow_mode) | クエリの実行時間が `max_execution_time` を超えた場合、または推定実行時間が `max_estimated_execution_time` を超える場合に実行する動作を設定します。 | -| [`max_execution_time_leaf`](/operations/settings/settings#max_execution_time_leaf) | 意味的には `max_execution_time` と同様ですが、分散クエリやリモートクエリではリーフノードにのみ適用されます。 | -| [`timeout_overflow_mode_leaf`](/operations/settings/settings#timeout_overflow_mode_leaf) | リーフノード上でのクエリ実行時間が `max_execution_time_leaf` を超えた場合の動作を設定します。 | -| [`min_execution_speed`](/operations/settings/settings#min_execution_speed) | 実行速度の最小値(行/秒)。 | -| [`min_execution_speed_bytes`](/operations/settings/settings#min_execution_speed_bytes) | 毎秒の最小実行バイト数。 | -| [`max_execution_speed`](/operations/settings/settings#max_execution_speed) | 1 秒あたりに実行される行数の最大値。 | -| [`max_execution_speed_bytes`](/operations/settings/settings#max_execution_speed_bytes) | 1 秒あたりの実行バイト数の上限。 | -| [`timeout_before_checking_execution_speed`](/operations/settings/settings#timeout_before_checking_execution_speed) | 指定された秒数が経過した後、実行速度が遅すぎないこと(`min_execution_speed` 以上であること)を確認します。 | -| [`max_estimated_execution_time`](/operations/settings/settings#max_estimated_execution_time) | クエリ実行時間の最大推定値(秒)。 | -| [`max_columns_to_read`](/operations/settings/settings#max_columns_to_read) | 1つのクエリでテーブルから読み取れる列の最大数。 | -| [`max_temporary_columns`](/operations/settings/settings#max_temporary_columns) | クエリ実行時に、定数列も含めて同時にRAM内に保持される必要がある一時列の最大数。 | -| [`max_temporary_non_const_columns`](/operations/settings/settings#max_temporary_non_const_columns) | クエリの実行時に同時にRAM内に保持しておく必要がある一時カラムの最大数を示します。ただし、定数カラムは含めません。 | -| [`max_subquery_depth`](/operations/settings/settings#max_subquery_depth) | クエリ内のネストされたサブクエリ数が指定した数を超えた場合の動作を設定します。 | -| [`max_ast_depth`](/operations/settings/settings#max_ast_depth) | クエリの構文木における最大のネスト深さ。 | -| [`max_ast_elements`](/operations/settings/settings#max_ast_elements) | クエリの構文木における要素数の上限。 | -| [`max_rows_in_set`](/operations/settings/settings#max_rows_in_set) | サブクエリから生成された IN 句のデータセットに含められる最大行数。 | -| [`max_bytes_in_set`](/operations/settings/settings#max_bytes_in_set) | サブクエリから作成された IN 句内の集合に対して使用される未圧縮データの最大バイト数。 | -| [`set_overflow_mode`](/operations/settings/settings#max_bytes_in_set) | データ量がいずれかの制限を超えた場合に実行される動作を設定します。 | -| [`max_rows_in_distinct`](/operations/settings/settings#max_rows_in_distinct) | DISTINCT 使用時の重複排除後の行数の最大値。 | -| [`max_bytes_in_distinct`](/operations/settings/settings#max_bytes_in_distinct) | DISTINCT を使用する際にハッシュテーブルがメモリ上に保持する状態の最大サイズ(非圧縮時のバイト数)。 | -| [`distinct_overflow_mode`](/operations/settings/settings#distinct_overflow_mode) | データ量がいずれかの制限を超えた場合の動作を設定します。 | -| [`max_rows_to_transfer`](/operations/settings/settings#max_rows_to_transfer) | GLOBAL IN/JOIN セクションの実行時に、リモートサーバーへ渡すか一時テーブルに保存できる最大サイズ(行数)。 | -| [`max_bytes_to_transfer`](/operations/settings/settings#max_bytes_to_transfer) | GLOBAL IN/JOIN セクションの実行時に、リモートサーバーへ送信するか一時テーブルに保存できる非圧縮データの最大バイト数。 | -| [`transfer_overflow_mode`](/operations/settings/settings#transfer_overflow_mode) | データ量がいずれかの制限を超えた場合の挙動を設定します。 | -| [`max_rows_in_join`](/operations/settings/settings#max_rows_in_join) | テーブル結合時に使用されるハッシュテーブルの行数を制限します。 | -| [`max_bytes_in_join`](/operations/settings/settings#max_bytes_in_join) | テーブル結合時に使用されるハッシュテーブルの最大サイズ(バイト単位)。 | -| [`join_overflow_mode`](/operations/settings/settings#join_overflow_mode) | 次のいずれかの結合制限に達した場合に ClickHouse が実行する動作を定義します。 | -| [`max_partitions_per_insert_block`](/operations/settings/settings#max_partitions_per_insert_block) | 単一の挿入ブロックに含められるパーティション数の最大値を制限し、ブロックに含まれるパーティションが多すぎる場合は例外をスローします。 | -| [`throw_on_max_partitions_per_insert_block`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | `max_partitions_per_insert_block` に到達した場合の動作を制御します。 | -| [`max_temporary_data_on_disk_size_for_user`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | 同時実行されているすべてのユーザークエリに対して、ディスク上の一時ファイルによって消費されるデータ量の最大値(バイト単位)。 | -| [`max_temporary_data_on_disk_size_for_query`](/operations/settings/settings#max_temporary_data_on_disk_size_for_query) | 同時実行中のすべてのクエリで、ディスク上の一時ファイルが使用するデータ量の最大値(バイト単位)。 | -| [`max_sessions_for_user`](/operations/settings/settings#max_sessions_for_user) | 認証済みユーザー1人あたりの ClickHouse サーバーへの同時セッション数の上限。 | -| [`max_partitions_to_read`](/operations/settings/settings#max_partitions_to_read) | 1つのクエリでアクセスできるパーティションの最大数を制限します。 | - - - - - -## 廃止された設定 {#obsolete-settings} - -:::note -次の設定は廃止されています -::: - -### max_pipeline_depth {#max-pipeline-depth} - -パイプラインの最大深さです。クエリ処理中に各データブロックが通過する変換の数に対応します。単一サーバー内でのみカウントされます。パイプラインの深さがこれより大きい場合は、例外が送出されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md deleted file mode 100644 index f3bbfed3db1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'サーバー CPU 過負荷時の挙動制御' -sidebar_label: 'サーバー過負荷' -slug: /operations/settings/server-overload -title: 'サーバー過負荷' -doc_type: 'reference' ---- - - - -# サーバー過負荷 {#server-overload} - - - -## 概要 {#overview} - -サーバーはさまざまな要因により過負荷状態になることがあります。現在の CPU の過負荷状況を判断するために、 -ClickHouse サーバーは CPU の待ち時間(`OSCPUWaitMicroseconds` メトリクス)とビジー時間 -(`OSCPUVirtualTimeMicroseconds` メトリクス)の比率を計算します。サーバーの負荷が一定の比率を超えて過負荷と判断された場合、 -負荷をこれ以上増やさないように、一部のクエリを破棄したり、接続要求自体を拒否したりすることが妥当です。 - -サーバー設定 `os_cpu_busy_time_threshold` により、CPU が有用な処理を行っていると見なすための最小ビジー時間が制御されます。 -現在の `OSCPUVirtualTimeMicroseconds` メトリクスの値がこの値を下回っている場合、 -CPU の過負荷率は 0 と見なされます。 - - - -## クエリの拒否 {#rejecting-queries} - -クエリを拒否する動作は、クエリレベルの設定である `min_os_cpu_wait_time_ratio_to_throw` と -`max_os_cpu_wait_time_ratio_to_throw` によって制御されます。これらの設定が指定されていて、かつ `min_os_cpu_wait_time_ratio_to_throw` が -`max_os_cpu_wait_time_ratio_to_throw` より小さい場合、CPU の過負荷比率が少なくとも `min_os_cpu_wait_time_ratio_to_throw` に -達しているときに、ある確率でクエリが拒否され、`SERVER_OVERLOADED` エラーがスローされます。この確率は、 -最小比と最大比の間の線形補間として決定されます。例えば、`min_os_cpu_wait_time_ratio_to_throw = 2`、 -`max_os_cpu_wait_time_ratio_to_throw = 6`、`cpu_overload = 4` の場合、クエリは `0.5` の確率で拒否されます。 - - - -## 接続のドロップ {#dropping-connections} - -接続のドロップは、サーバーレベルの設定である `min_os_cpu_wait_time_ratio_to_drop_connection` と -`max_os_cpu_wait_time_ratio_to_drop_connection` によって制御されます。これらの設定はサーバーを再起動せずに変更できます。これらの設定の意図は、クエリ拒否に関する設定と同様です。この場合に唯一異なる点は、サーバーが過負荷状態のときに、サーバー側で接続試行が拒否されることです。 - - - -## リソース過負荷警告 {#resource-overload-warnings} - -サーバーが過負荷になった際、ClickHouse は CPU とメモリの過負荷警告を `system.warnings` テーブルにも記録します。これらのしきい値はサーバー設定でカスタマイズできます。 - -**例** - -```xml - - - 0.9 - 0.8 - 600 - 0.9 - 0.8 - 600 - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md deleted file mode 100644 index ecec7626a19..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -description: 'フォーマット関連の設定' -sidebar_label: 'フォーマット設定' -slug: /operations/settings/formats -title: 'フォーマット設定' -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/*編集しないでください – このファイルは自動生成されたものです*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md deleted file mode 100644 index 78c75da264a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: '同じ名前でグループ化された設定の集合。' -sidebar_label: '設定プロファイル' -sidebar_position: 61 -slug: /operations/settings/settings-profiles -title: '設定プロファイル' -doc_type: 'reference' ---- - -# 設定プロファイル {#settings-profiles} - -設定プロファイルとは、同じ名前でグループ化された設定の集合です。 - -:::note -ClickHouse では、設定プロファイルを管理するための [SQL ベースのワークフロー](/operations/access-rights#access-control-usage) もサポートしています。こちらの利用を推奨します。 -::: - -プロファイル名は任意に設定できます。同じプロファイルを複数のユーザーに指定することもできます。設定プロファイルで最も重要な設定は `readonly=1` であり、これにより読み取り専用アクセスが保証されます。 - -設定プロファイルは互いに継承できます。継承を利用するには、プロファイル内で列挙される他の設定より前に、1 つまたは複数の `profile` 設定を指定します。ある設定が異なるプロファイルで定義されている場合は、最後に定義されたものが有効になります。 - -プロファイル内のすべての設定を適用するには、`profile` 設定を指定します。 - -例: - -`web` プロファイルを適用します。 - -```sql -SET profile = 'web' -``` - -設定プロファイルはユーザー設定ファイルで定義します。通常は `users.xml` です。 - -例: - -```xml - - - - - - 8 - - - - - 1000000000 - 100000000000 - - 1000000 - any - - 1000000 - 1000000000 - - 100000 - 100000000 - break - - 600 - 1000000 - 15 - - 25 - 100 - 50 - - 2 - 25 - 50 - 100 - - 4 - - 1 - - -``` - -この例では、`default` と `web` の 2 つのプロファイルを指定しています。 - -`default` プロファイルには特別な目的があります。必ず定義されている必要があり、サーバー起動時に適用されます。言い換えると、`default` プロファイルには既定の設定が含まれます。 - -`web` プロファイルは通常のプロファイルであり、`SET` クエリ、または HTTP クエリ内の URL パラメータを使って設定できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md deleted file mode 100644 index 494a3f85747..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md +++ /dev/null @@ -1,232 +0,0 @@ ---- -description: 'クエリレベルの設定' -sidebar_label: 'クエリレベルのセッション設定' -slug: /operations/settings/query-level -title: 'クエリレベルのセッション設定' -doc_type: 'reference' ---- - -## 概要 {#overview} - -特定の設定を指定してステートメントを実行する方法はいくつかあります。 -設定は階層的に構成されており、後のレイヤーほど前のレイヤーで指定された設定値を再定義します。 - -## 優先順位 {#order-of-priority} - -設定を定義する際の優先順位は次のとおりです。 - -1. 設定をユーザーに直接、または設定プロファイル内で適用する - - - SQL(推奨) - - 1 つ以上の XML または YAML ファイルを `/etc/clickhouse-server/users.d` に追加する - -2. セッション設定 - - - ClickHouse Cloud の SQL コンソール、または対話モードの - `clickhouse client` から `SET setting=value` を送信します。同様に、 - HTTP プロトコルで ClickHouse のセッションを使用できます。その場合は - `session_id` HTTP パラメータを指定する必要があります。 - -3. クエリ設定 - - - `clickhouse client` を非対話モードで起動する場合、起動時パラメータ - `--setting=value` を指定します。 - - HTTP API を使用する場合、CGI パラメータ(`URL?setting_1=value&setting_2=value...`)を渡します。 - - SELECT クエリの - [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select-query) - 句で設定を定義します。設定値はそのクエリにのみ適用され、クエリの実行後に - デフォルトまたは以前の値にリセットされます。 - -## 設定をデフォルト値に戻す {#converting-a-setting-to-its-default-value} - -設定を変更してデフォルト値に戻したい場合は、値を `DEFAULT` に設定します。構文は次のようになります。 - -```sql -SET setting_name = DEFAULT -``` - -たとえば、`async_insert` のデフォルト値は `0` です。これを `1` に変更するとします。 - -```sql -SET async_insert = 1; - -SELECT value FROM system.settings where name='async_insert'; -``` - -レスポンスは以下のとおりです: - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - -次のコマンドで値を 0 に戻します。 - -```sql -SET async_insert = DEFAULT; - -SELECT value FROM system.settings where name='async_insert'; -``` - -設定は既定値に戻りました。 - -```response -┌─value───┐ -│ 0 │ -└─────────┘ -``` - - -## カスタム設定 {#custom_settings} - -共通の[設定](/operations/settings/settings.md)に加えて、ユーザーはカスタム設定を定義できます。 - -カスタム設定名は、あらかじめ定義された接頭辞のいずれかで始まっていなければなりません。これらの接頭辞の一覧は、サーバー設定ファイル内の [custom_settings_prefixes](../../operations/server-configuration-parameters/settings.md#custom_settings_prefixes) パラメータで宣言しなければなりません。 - -```xml -custom_ -``` - -カスタム設定を定義するには、`SET` コマンドを使用します。 - -```sql -SET custom_a = 123; -``` - -カスタム設定の現在値を取得するには、`getSetting()` 関数を使用してください。 - -```sql -SELECT getSetting('custom_a'); -``` - - -## 例 {#examples} - -これらの例はすべて、`async_insert` 設定の値を `1` に設定し、 -稼働中のシステムで設定を確認する方法を示します。 - -### SQL を使用して設定をユーザーに直接適用する {#using-sql-to-apply-a-setting-to-a-user-directly} - -以下の SQL は、設定 `async_inset = 1` を持つユーザー `ingester` を作成します。 - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS async_insert = 1 -``` - - -#### 設定プロファイルおよび割り当てを確認する {#examine-the-settings-profile-and-assignment} - -```sql -SHOW ACCESS -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ ... │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS async_insert = true │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### SQL を使用して設定プロファイルを作成し、ユーザーに割り当てる {#using-sql-to-create-a-settings-profile-and-assign-to-a-user} - -次の SQL は、設定 `async_inset = 1` を持つプロファイル `log_ingest` を作成します。 - -```sql -CREATE -SETTINGS PROFILE log_ingest SETTINGS async_insert = 1 -``` - -これによりユーザー `ingester` が作成され、そのユーザーに設定プロファイル `log_ingest` が割り当てられます: - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS PROFILE log_ingest -``` - - -### XML を使用して設定プロファイルおよびユーザーを作成する {#using-xml-to-create-a-settings-profile-and-user} - -```xml title=/etc/clickhouse-server/users.d/users.xml - -# highlight-start {#highlight-start} - - - 1 - - -# highlight-end {#highlight-end} - - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 -# highlight-start {#highlight-start} - log_ingest -# highlight-end {#highlight-end} - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 - 1 - 1 - - - -``` - - -#### 設定プロファイルとその割り当てを確認する {#examine-the-settings-profile-and-assignment-1} - -```sql -アクセスを表示 -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ CREATE USER default IDENTIFIED WITH sha256_password │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS PROFILE log_ingest │ -│ CREATE SETTINGS PROFILE default │ -# highlight-next-line {#highlight-next-line} -│ CREATE SETTINGS PROFILE log_ingest SETTINGS async_insert = true │ -│ CREATE SETTINGS PROFILE readonly SETTINGS readonly = 1 │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### セッションに設定を適用する {#assign-a-setting-to-a-session} - -```sql -SET async_insert =1; -SELECT value FROM system.settings where name='async_insert'; -``` - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - - -### クエリ実行時に設定を指定する {#assign-a-setting-during-a-query} - -```sql -INSERT INTO YourTable --- highlight-next-line -SETTINGS async_insert=1 -VALUES (...) -``` - - -## 関連項目 {#see-also} - -- ClickHouse の設定の説明については、[Settings](/operations/settings/settings.md) ページを参照してください。 -- [グローバルサーバー設定](/operations/server-configuration-parameters/settings.md) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md deleted file mode 100644 index 4e9399953a0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md +++ /dev/null @@ -1,260 +0,0 @@ ---- -description: 'ユーザーおよびロールの構成設定。' -sidebar_label: 'ユーザー設定' -sidebar_position: 63 -slug: /operations/settings/settings-users -title: 'ユーザーおよびロールの設定' -doc_type: 'reference' ---- - -# ユーザーとロールの設定 {#users-and-roles-settings} - -`users.xml` 設定ファイルの `users` セクションには、ユーザーの設定が含まれます。 - -:::note -ClickHouse は、ユーザー管理のための [SQL 駆動のワークフロー](/operations/access-rights#access-control-usage) もサポートしています。こちらの利用を推奨します。 -::: - -`users` セクションの構造: - -```xml - - - - - - - - - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - - - ecdsa-sha2-nistp256 - AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxeV2uN5UY6CUbCzTA1rXfYimKQA5ivNIqxdax4bcMXz4D0nSk2l5E1TkR5mG8EBWtmExSPbcEPJ8V7lyWWbA8= - - - ssh-rsa - AAAAB3NzaC1yc2EAAAADAQABAAABgQCpgqL1SHhPVBOTFlOm0pu+cYBbADzC2jL41sPMawYCJHDyHuq7t+htaVVh2fRgpAPmSEnLEC2d4BEIKMtPK3bfR8plJqVXlLt6Q8t4b1oUlnjb3VPA9P6iGcW7CV1FBkZQEVx8ckOfJ3F+kI5VsrRlEDgiecm/C1VPl0/9M2llW/mPUMaD65cM9nlZgM/hUeBrfxOEqM11gDYxEZm1aRSbZoY4dfdm3vzvpSQ6lrCrkjn3X2aSmaCLcOWJhfBWMovNDB8uiPuw54g3ioZ++qEQMlfxVsqXDGYhXCrsArOVuW/5RbReO79BvXqdssiYShfwo+GhQ0+aLWMIW/jgBkkqx/n7uKLzCMX7b2F+aebRYFh+/QXEj7SnihdVfr9ud6NN3MWzZ1ltfIczlEcFLrLJ1Yq57wW6wXtviWh59WvTWFiPejGjeSjjJyqqB49tKdFVFuBnIU5u/bch2DXVgiAEdQwUrIp1ACoYPq22HFFAYUJrL32y7RxX3PGzuAv3LOc= - - - - 0|1 - - - - - profile_name - - default - default - - - - expression(式) - - - - - - GRANT SELECT ON system.* - - - - -``` - -### user_name/password {#user-namepassword} - -パスワードは平文または SHA256(16進数形式)で指定できます。 - -* パスワードを平文で設定する場合(**非推奨**)、`password` 要素に記述します。 - - 例: `qwerty`。パスワードは空のままにしておくこともできます。 - - - -* パスワードを SHA256 のハッシュ値で設定する場合、`password_sha256_hex` 要素に記述します。 - -たとえば、`65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5` のようになります。 - -シェルでパスワードを生成する例: - -```bash -PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-' -``` - -結果の1行目はパスワードです。2行目は対応する SHA256 ハッシュです。 - - - -* MySQL クライアントとの互換性のために、パスワードはダブル SHA1 ハッシュで指定できます。その場合は `password_double_sha1_hex` 要素に設定します。 - - 例えば、`08b4a0f1de6ad37da17359e592c8d74788a83eb0` のように指定します。 - - シェルでパスワードを生成する例: - - ```bash - PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-' - ``` - - 結果の1行目がパスワードです。2行目が対応するダブル SHA1 ハッシュです。 - -### username/ssh-key {#user-sshkey} - -この設定により、SSH 鍵を用いた認証を行えます。 - -`ssh-keygen` で生成された次のような SSH 鍵があるとします。 - -```text -ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj john@example.com -``` - -`ssh_key` 要素は次のようであることが想定されています - -```xml - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - -``` - -`ssh-ed25519` を、他のサポートされているアルゴリズムである `ssh-rsa` または `ecdsa-sha2-nistp256` に置き換えます。 - -### access_management {#access_management-user-setting} - -この設定は、ユーザーに対して SQL 駆動の[アクセス制御およびアカウント管理](/operations/access-rights#access-control-usage)を使用するかどうかを有効または無効にします。 - -取りうる値: - -* 0 — 無効。 -* 1 — 有効。 - -デフォルト値: 0。 - -### grants {#grants-user-setting} - -この設定により、指定したユーザーに任意の権限を付与できます。 -リストの各要素は、被付与者 (`grantees`) を指定していない `GRANT` クエリである必要があります。 - -例: - -```xml - - - GRANT SHOW ON *.* - GRANT CREATE ON *.* WITH GRANT OPTION - GRANT SELECT ON system.* - - -``` - -この設定は、`dictionaries`、`access_management`、`named_collection_control`、`show_named_collections_secrets` および `allow_databases` の各設定と同時に指定することはできません。 - -### user_name/networks {#user-namenetworks} - -ユーザーが ClickHouse サーバーに接続できるネットワークの一覧です。 - -リストの各要素は、次のいずれかの形式を取ることができます。 - -* `` — IP アドレスまたはネットワークマスク。 - - 例: `213.180.204.3`, `10.0.0.1/8`, `10.0.0.1/255.255.255.0`, `2a02:6b8::3`, `2a02:6b8::3/64`, `2a02:6b8::3/ffff:ffff:ffff:ffff::`。 - -* `` — ホスト名。 - - 例: `example01.host.ru`。 - - アクセスを確認するために DNS クエリが実行され、返されたすべての IP アドレスが接続元アドレスと照合されます。 - -* `` — ホスト名に対する正規表現。 - - 例: `^example\d\d-\d\d-\d\.host\.ru$` - -アクセスを確認するために、まずピアアドレスに対して [DNS PTR クエリ](https://en.wikipedia.org/wiki/Reverse_DNS_lookup) が実行され、その結果に指定された regexp が適用されます。次に、PTR クエリの結果に対して別の DNS クエリが実行され、取得したすべてのアドレスがピアアドレスと照合されます。regexp の末尾には必ず $ を付けることを強く推奨します。 - -DNS クエリのすべての結果は、サーバーが再起動するまでキャッシュされます。 - -**例** - -任意のネットワークからのユーザーへのアクセスを許可するには、次を指定します。 - -```xml -::/0 -``` - -:::note -ファイアウォールが適切に構成されている場合、またはサーバーがインターネットに直接接続されていない場合を除き、任意のネットワークからのアクセスを開放するのは安全ではありません。 -::: - -`localhost` からのみアクセスを許可するには、次のように指定します。 - -```xml -::1 -127.0.0.1 -``` - -### user_name/profile {#user-nameprofile} - -ユーザーに設定プロファイルを割り当てることができます。設定プロファイルは `users.xml` ファイル内の別セクションで定義します。詳細については、[Profiles of Settings](../../operations/settings/settings-profiles.md) を参照してください。 - -### user_name/quota {#user-namequota} - -クオータを使用すると、一定期間にわたるリソース使用量を追跡したり、制限したりできます。クオータは、`users.xml` 設定ファイルの `quotas` -セクションで設定します。 - -ユーザーに一連のクオータを割り当てることができます。クオータ設定の詳細な説明については、[Quotas](/operations/quotas) を参照してください。 - -### user_name/databases {#user-namedatabases} - -このセクションでは、現在のユーザーによって実行される `SELECT` クエリに対して ClickHouse が返す行を制限することで、基本的な行レベルセキュリティを実装できます。 - -**例** - -次の設定では、ユーザー `user1` は、`id` フィールドの値が 1000 である行のみを、`SELECT` クエリの結果として `table1` から確認できるように制限されます。 - -```xml - - - - - id = 1000 - - - - -``` - -`filter` には、[UInt8](../../sql-reference/data-types/int-uint.md) 型の値を返す任意の式を指定できます。通常は比較演算子や論理演算子を含みます。`database_name.table1` の行のうち、`filter` の結果が 0 を返すものは、このユーザーには返されません。このフィルタリングは `PREWHERE` 演算と互換性がなく、`WHERE→PREWHERE` 最適化を無効にします。 - -## ロール {#roles} - -`user.xml` 設定ファイルの `roles` セクションを使用して、あらかじめ定義されたロールを任意に作成できます。 - -`roles` セクションの構造は次のとおりです。 - -```xml - - - - GRANT SHOW ON *.* - REVOKE SHOW ON system.* - GRANT CREATE ON *.* WITH GRANT OPTION - - - -``` - -これらのロールは、`users` セクション内のユーザーに対して付与することもできます。 - -```xml - - - ... - - GRANT test_role - - - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings.md deleted file mode 100644 index 9cb420688f5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/settings.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: 'セッション設定' -description: 'system.settings から設定されるセッション用の設定' -sidebar_label: 'セッション設定' -slug: /operations/settings/settings -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/* 自動生成されたファイルのため、編集しないでください */ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md deleted file mode 100644 index 6774d87be1c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'TCP 接続制限' -sidebar_label: 'TCP 接続制限' -slug: /operations/settings/tcp-connection-limits -title: 'TCP 接続制限' -doc_type: 'reference' ---- - - - -# TCP 接続制限 {#tcp-connection-limits} - - - -## 概要 {#overview} - -ClickHouse の TCP 接続(たとえば [コマンドラインクライアント](https://clickhouse.com/docs/interfaces/cli) を通したもの)が、 -一定回数のクエリ実行または一定時間の経過後に自動的に切断されることがあります。 -切断された後は、(コマンドラインクライアントで別のクエリを送信するなど、 -他の要因によってトリガーされない限り)自動再接続は行われません。 - -接続制限は、サーバー設定 -`tcp_close_connection_after_queries_num`(クエリ回数の制限) -または `tcp_close_connection_after_queries_seconds`(時間の制限)を 0 より大きい値に設定することで有効になります。 -両方の制限が有効な場合、どちらか一方の制限に先に達した時点で接続がクローズされます。 - -制限に達して切断されると、クライアントは -`TCP_CONNECTION_LIMIT_REACHED` 例外を受け取り、**切断を引き起こしたクエリが処理されることは決してありません**。 - - - -## クエリ制限 {#query-limits} - -`tcp_close_connection_after_queries_num` が N に設定されていると仮定すると、その接続では -N 回のクエリが正常に実行できます。その後、N + 1 回目のクエリでクライアントが切断されます。 - -処理されたすべてのクエリはクエリ制限にカウントされます。したがって、コマンドラインクライアントで接続する場合には、 -自動的に実行される初期のシステム警告クエリがあり、これも制限に含まれます。 - -TCP 接続がアイドル状態(つまり、ある一定期間クエリを処理していない状態で、 -その期間はセッション設定 `poll_interval` によって指定される)の場合、これまでにカウントされたクエリ数は 0 にリセットされます。 -これは、アイドル状態が発生した場合、1 つの接続における合計クエリ数が -`tcp_close_connection_after_queries_num` を超える可能性があることを意味します。 - - - -## 接続時間の制限 {#duration-limits} - -接続時間は、クライアントが接続した直後から計測されます。 -`tcp_close_connection_after_queries_seconds` 秒が経過した後に実行された最初のクエリで、クライアントは切断されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md deleted file mode 100644 index 47510843779..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse と ZooKeeper 間の安全な SSL/TLS 通信を構成するためのガイド' -sidebar_label: 'ZooKeeper とのセキュア通信' -sidebar_position: 45 -slug: /operations/ssl-zookeeper -title: 'ClickHouse と ZooKeeper 間のセキュア通信(オプション)' -doc_type: 'guide' ---- - -# ClickHouse と Zookeeper 間のオプションのセキュア通信 {#optional-secured-communication-between-clickhouse-and-zookeeper} - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - -ClickHouse クライアントとの SSL 経由の通信のために、`ssl.keyStore.location`、`ssl.keyStore.password` および `ssl.trustStore.location`、`ssl.trustStore.password` を指定する必要があります。これらのオプションは Zookeeper バージョン 3.5.2 以降で利用可能です。 - -`zookeeper.crt` を信頼済み証明書に追加できます。 - -```bash -sudo cp zookeeper.crt /usr/local/share/ca-certificates/zookeeper.crt -sudo update-ca-certificates -``` - -`config.xml` の client セクションは次のようになります: - -```xml - - /etc/clickhouse-server/client.crt - /etc/clickhouse-server/client.key - true - true - sslv2,sslv3 - true - - RejectCertificateHandler - - -``` - -クラスタ定義とマクロを含めて、ClickHouse の設定に Zookeeper を追加します: - -```xml - - - - localhost - 2281 - 1 - - - -``` - -`clickhouse-server` を起動します。ログに次のような出力が表示されるはずです: - -```text - ZooKeeper: 初期化済み、ホスト: secure://localhost:2281 -``` - -`secure://` プレフィックスは、接続が SSL によって保護されていることを示します。 - -トラフィックが暗号化されていることを確認するには、セキュアなポート上で `tcpdump` を実行します。 - -```bash -tcpdump -i any dst port 2281 -nnXS -``` - -次に、`clickhouse-client` でクエリを実行します: - -```sql -SELECT * FROM system.zookeeper WHERE path = '/'; -``` - -暗号化されていない接続の場合、`tcpdump` の出力は次のようになります。 - -```text -..../zookeeper/quota. -``` - -暗号化された接続の場合、これは表示されないはずです。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/startup-scripts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/startup-scripts.md deleted file mode 100644 index 295b46ded9d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/startup-scripts.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'スキーマの自動作成とマイグレーションのために、ClickHouse で SQL 起動スクリプトを構成および利用するためのガイド' -sidebar_label: '起動スクリプト' -slug: /operations/startup-scripts -title: '起動スクリプト' -doc_type: 'guide' ---- - -# 起動スクリプト {#startup-scripts} - -ClickHouse は、起動時にサーバー設定に記述した任意の SQL クエリを実行できます。これは、マイグレーションやスキーマの自動作成などに役立ちます。 - -```xml - - - false - - CREATE ROLE OR REPLACE test_role - - - CREATE TABLE TestTable (id UInt64) ENGINE=TinyLog - SELECT 1; - - - CREATE DICTIONARY test_dict (...) SOURCE(CLICKHOUSE(...)) - default - - - -``` - -ClickHouse は、`startup_scripts` に含まれるすべてのクエリを指定された順序で順番に実行します。いずれかのクエリが失敗しても、その後のクエリの実行は中断されません。ただし、`throw_on_error` が true に設定されている場合は、 -スクリプト実行中にエラーが発生するとサーバーは起動しません。 - -設定で条件付きクエリを指定できます。その場合、対応するクエリは、条件クエリが値 `1` または `true` を返した場合にのみ実行されます。 - -:::note -条件クエリが `1` または `true` 以外の値を返した場合、その結果は `false` として解釈され、対応するクエリは実行されません。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/storing-data.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/storing-data.md deleted file mode 100644 index 1c5d89973e8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/storing-data.md +++ /dev/null @@ -1,1065 +0,0 @@ ---- -description: 'highlight-next-line のドキュメント' -sidebar_label: 'データ保存用の外部ディスク' -sidebar_position: 68 -slug: /operations/storing-data -title: 'データ保存用の外部ディスク' -doc_type: 'guide' ---- - -ClickHouse で処理されるデータは通常、ClickHouse サーバーが動作している -マシンのローカルファイルシステムに保存されます。これには大容量ディスクが -必要となり、コストが高くなる場合があります。ローカルにデータを保存しないために、さまざまなストレージオプションがサポートされています: -1. [Amazon S3](https://aws.amazon.com/s3/) オブジェクトストレージ。 -2. [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs)。 -3. サポート対象外: Hadoop Distributed File System ([HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)) - -
- -:::note -ClickHouse には外部テーブルエンジンのサポートもありますが、これはこのページで説明している外部ストレージオプションとは異なります。外部テーブルエンジンは、Parquet のような汎用的なファイル形式で保存されたデータを読み取ることができます。このページでは、ClickHouse の `MergeTree` ファミリーまたは `Log` ファミリーテーブル向けのストレージ設定について説明します。 - -1. `Amazon S3` ディスク上に保存されたデータを扱うには、[S3](/engines/table-engines/integrations/s3.md) テーブルエンジンを使用します。 -2. Azure Blob Storage に保存されたデータを扱うには、[AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage.md) テーブルエンジンを使用します。 -3. Hadoop Distributed File System(サポート対象外)上のデータを扱うには、[HDFS](/engines/table-engines/integrations/hdfs.md) テーブルエンジンを使用します。 -::: - - - -## 外部ストレージを構成する {#configuring-external-storage} - -[`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) および [`Log`](/engines/table-engines/log-family/log.md) -ファミリーのテーブルエンジンは、`S3`、`AzureBlobStorage`、`HDFS`(サポート対象外)に対して、それぞれディスクタイプ `s3`、 -`azure_blob_storage`、`hdfs`(サポート対象外)を使用してデータを保存できます。 - -ディスク構成には次が必要です: - -1. `type` セクション(値は `s3`、`azure_blob_storage`、`hdfs`(サポート対象外)、`local_blob_storage`、`web` のいずれか)。 -2. 対象となる外部ストレージタイプ固有の設定。 - -ClickHouse バージョン 24.1 以降では、新しい構成オプションを使用できます。 -この場合は次を指定する必要があります: - -1. `type` を `object_storage` に設定する。 -2. `object_storage_type` を `s3`、`azure_blob_storage`(または `24.3` 以降では単に `azure`)、`hdfs`(サポート対象外)、`local_blob_storage`(または `24.3` 以降では単に `local`)、`web` のいずれかに設定する。 - -
- -任意で `metadata_type` を指定できます(デフォルトは `local`)が、`plain`、`web`、さらに `24.4` 以降では `plain_rewritable` に設定することもできます。 -`plain` メタデータタイプの使用方法は [plain storage セクション](/operations/storing-data#plain-storage) で説明されています。`web` メタデータタイプは `web` オブジェクトストレージタイプでのみ使用でき、`local` メタデータタイプはメタデータファイルをローカルに保存します(各メタデータファイルには、オブジェクトストレージ内のファイルへのマッピングと、それらに関する追加のメタ情報が含まれます)。 - -例: - -```xml - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -は、次の設定(バージョン `24.1` 以降)に相当します。 - -```xml - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -次の設定は以下のとおりです。 - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -は次の値と等しい: - -```xml - - object_storage - s3 - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -ストレージ構成の完全な例は次のとおりです。 - -```xml - - - - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -バージョン 24.1 以降では、次のような形にもなります: - - -```xml - - - - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -すべての `MergeTree` テーブルで特定の種類のストレージをデフォルトとして使用するには、 -設定ファイルに次のセクションを追加します。 - -```xml - - - s3 - - -``` - -特定のテーブルに対して特定のストレージポリシーを設定したい場合は、 -テーブルを作成する際に `settings` で指定できます。 - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS storage_policy = 's3'; -``` - -`storage_policy` の代わりに `disk` を使用することもできます。この場合、設定ファイルに `storage_policy` セクションを含める必要はなく、`disk` セクションだけで十分です。 - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS disk = 's3'; -``` - - -## 動的設定 {#dynamic-configuration} - -ディスクを構成ファイルで事前定義しなくても、`CREATE` / `ATTACH` クエリの設定でストレージ設定を指定することもできます。 - -次のクエリ例は、上記の動的ディスク設定を基にしており、URL でホストされているテーブルのデータをローカルディスクにキャッシュする方法を示しています。 - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -以下の例では、外部ストレージにキャッシュを追加します。 - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) --- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); --- highlight-end -``` - -以下で強調されている設定では、`type=web` のディスクが -`type=cache` のディスクの中にネストされている点に注意してください。 - -:::note -この例では `type=web` を使用していますが、ローカルディスクを含め、任意のディスクタイプを動的ディスクとして構成できます。ローカルディスクでは、サーバー設定パラメータ `custom_local_disks_base_directory` で指定されたディレクトリ配下のパスを、パス引数として指定する必要があります。このパラメータにはデフォルト値がないため、ローカルディスクを使用する場合は合わせて設定してください。 -::: - -設定ファイルベースの設定と SQL で定義された設定を組み合わせて使用することもできます。 - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); - -- highlight-end -``` - -ここで `web` はサーバー設定ファイルで定義された値です。 - - -```xml - - - - web - 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - - - -``` - -### S3 ストレージの使用 {#s3-storage} - -#### 必須パラメータ {#required-parameters-s3} - -| Parameter | Description | -| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `endpoint` | `path` または `virtual hosted` [スタイル](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html) の S3 エンドポイント URL。データ格納用のバケットおよびルートパスを含める必要があります。 | -| `access_key_id` | 認証に使用される S3 アクセスキー ID。 | -| `secret_access_key` | 認証に使用される S3 シークレットアクセスキー。 | - -#### オプションパラメータ {#optional-parameters-s3} - - -| Parameter | Description | Default Value | -|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------| -| `region` | S3 のリージョン名。 | - | -| `support_batch_delete` | バッチ削除のサポートをチェックするかどうかを制御します。Google Cloud Storage (GCS) を使用する場合は、GCS はバッチ削除をサポートしないため `false` に設定します。 | `true` | -| `use_environment_credentials` | 存在する場合、環境変数 `AWS_ACCESS_KEY_ID`、`AWS_SECRET_ACCESS_KEY`、`AWS_SESSION_TOKEN` から AWS クレデンシャルを読み取ります。 | `false` | -| `use_insecure_imds_request` | `true` の場合、Amazon EC2 メタデータからクレデンシャルを取得する際に、非セキュアな IMDS リクエストを使用します。 | `false` | -| `expiration_window_seconds` | 有効期限付きクレデンシャルが失効しているかどうかを確認するための猶予期間(秒)。 | `120` | -| `proxy` | S3 エンドポイント用のプロキシ設定。`proxy` ブロック内の各 `uri` 要素にはプロキシ URL を指定する必要があります。 | - | -| `connect_timeout_ms` | ソケット接続のタイムアウト(ミリ秒)。 | `10000` (10 seconds) | -| `request_timeout_ms` | リクエストのタイムアウト(ミリ秒)。 | `5000` (5 seconds) | -| `retry_attempts` | 失敗したリクエストに対する再試行回数。 | `10` | -| `single_read_retries` | 読み取り中の接続断に対する再試行回数。 | `4` | -| `min_bytes_for_seek` | シーケンシャルリードではなく seek 操作を使用するための最小バイト数。 | `1 MB` | -| `metadata_path` | S3 メタデータファイルを保存するローカルファイルシステム上のパス。 | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | `true` の場合、起動時のディスクアクセスチェックをスキップします。 | `false` | -| `header` | リクエストに指定した HTTP ヘッダーを追加します。複数回指定できます。 | - | -| `server_side_encryption_customer_key_base64` | SSE-C で暗号化された S3 オブジェクトにアクセスするために必要なヘッダー。 | - | -| `server_side_encryption_kms_key_id` | [SSE-KMS 暗号化](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html) を使用した S3 オブジェクトにアクセスするために必要なヘッダー。空文字列の場合は AWS 管理の S3 キーが使用されます。 | - | -| `server_side_encryption_kms_encryption_context` | SSE-KMS 用の暗号化コンテキストヘッダー(`server_side_encryption_kms_key_id` と併用)。 | - | -| `server_side_encryption_kms_bucket_key_enabled` | SSE-KMS 用に S3 バケットキーを有効化します(`server_side_encryption_kms_key_id` と併用)。 | バケットレベルの設定に一致 | -| `s3_max_put_rps` | スロットリングが行われる前の 1 秒あたりの最大 PUT リクエスト数。 | `0` (unlimited) | -| `s3_max_put_burst` | RPS 制限に到達する前に許可される同時 PUT リクエストの最大数。 | `s3_max_put_rps` と同じ | -| `s3_max_get_rps` | スロットリングが行われる前の 1 秒あたりの最大 GET リクエスト数。 | `0` (unlimited) | -| `s3_max_get_burst` | RPS 制限に到達する前に許可される同時 GET リクエストの最大数。 | `s3_max_get_rps` と同じ | -| `read_resource` | 読み取りリクエスト用の [scheduling](/operations/workload-scheduling.md) リソース名。 | 空文字列(無効) | -| `write_resource` | 書き込みリクエスト用の [scheduling](/operations/workload-scheduling.md) リソース名。 | 空文字列(無効) | -| `key_template` | [re2](https://github.com/google/re2/wiki/Syntax) 構文を使用してオブジェクトキーの生成フォーマットを定義します。`storage_metadata_write_full_object_key` フラグが必要です。`endpoint` の `root path` とは非互換です。`key_compatibility_prefix` が必要です。 | - | -| `key_compatibility_prefix` | `key_template` と併用する必須設定。古いメタデータバージョンを読み取るために、以前 `endpoint` で使用していた `root path` を指定します。 | - | -| `read_only` | ディスクからの読み取りのみを許可します。 | - | -:::note -Google Cloud Storage (GCS) も `s3` タイプとしてサポートされています。詳細は [GCS backed MergeTree](/integrations/gcs) を参照してください。 -::: - -### プレーンストレージの使用 {#plain-storage} - - - -`22.10` で新しいディスクタイプ `s3_plain` が導入されました。これは一度だけ書き込めるストレージを提供します。 -このディスクタイプの設定パラメータは、`s3` ディスクタイプの場合と同じです。 -`s3` ディスクタイプと異なり、データをそのまま保存します。言い換えると、 -ランダムに生成された BLOB 名を使う代わりに通常のファイル名を使用し -(ClickHouse がローカルディスク上にファイルを保存するのと同じ方式)、 -ローカルには一切メタデータを保存しません。例えば、メタデータは `s3` 上のデータから導出されます。 - -このディスクタイプでは、既存データに対してマージを実行できず、新規データの挿入もできないため、 -テーブルの静的なバージョンを保持できます。 -このディスクタイプのユースケースとしては、その上にバックアップを作成することが挙げられます。 -これは `BACKUP TABLE data TO Disk('plain_disk_name', 'backup_name')` を実行することで行えます。 -その後、`RESTORE TABLE data AS data_restored FROM Disk('plain_disk_name', 'backup_name')` -を実行するか、`ATTACH TABLE data (...) ENGINE = MergeTree() SETTINGS disk = 'plain_disk_name'` -を使用できます。 - -設定: - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -`24.1` 以降では、任意のオブジェクトストレージディスク(`s3`、`azure`、`hdfs`(非サポート)、`local`)を -`plain` メタデータタイプを使用して設定できます。 - -設定: - -```xml - - object_storage - azure - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -### S3 Plain Rewritable Storage の使用 {#s3-plain-rewritable-storage} - -新しいディスクタイプ `s3_plain_rewritable` は `24.4` で導入されました。 -`s3_plain` ディスクタイプと同様に、メタデータファイル用の追加ストレージは必要ありません。 -メタデータは代わりに S3 に保存されます。 -`s3_plain` ディスクタイプと異なり、`s3_plain_rewritable` ではマージの実行が可能で、 -`INSERT` 操作をサポートします。 -[Mutations](/sql-reference/statements/alter#mutations) とテーブルのレプリケーションには対応していません。 - -このディスクタイプの主な利用例は、非レプリケートの `MergeTree` テーブルです。 -`s3` ディスクタイプも非レプリケートの `MergeTree` テーブルに適していますが、 -テーブルのローカルメタデータが不要で、利用できる操作が制限されても問題ない場合は、 -`s3_plain_rewritable` ディスクタイプを選択できます。 -これは、たとえば system テーブルに対して有用な場合があります。 - -設定: - -```xml - - s3_plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -に等しい - -```xml - - object_storage - s3 - plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -`24.5` 以降、`plain_rewritable` メタデータタイプを使用して、任意のオブジェクトストレージディスク -(`s3`、`azure`、`local`)を設定できるようになりました。 - -### Azure Blob Storage の使用 {#azure-blob-storage} - -`MergeTree` ファミリのテーブルエンジンは、`azure_blob_storage` タイプのディスクを使用して -データを [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/) に保存できます。 - -設定例: - - -```xml - - ... - - - azure_blob_storage - http://account.blob.core.windows.net - container - account - pass123 - /var/lib/clickhouse/disks/blob_storage_disk/ - /var/lib/clickhouse/disks/blob_storage_disk/cache/ - false - - - ... - -``` - -#### 接続パラメータ {#azure-blob-storage-connection-parameters} - -| Parameter | Description | Default Value | -| -------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `storage_account_url` (Required) | Azure Blob Storage アカウントの URL。例: `http://account.blob.core.windows.net` または `http://azurite1:10000/devstoreaccount1`。 | - | -| `container_name` | 対象となるコンテナ名。 | `default-container` | -| `container_already_exists` | コンテナ作成時の挙動を制御します:
- `false`: 新しいコンテナを作成する
- `true`: 既存のコンテナに直接接続する
- 未設定: コンテナの存在を確認し、存在しない場合は作成する | - | - -認証パラメータ(ディスクは利用可能なすべての方法 **および** Managed Identity Credential を順に試行します): - -| Parameter | Description | -| ------------------- | --------------------------------- | -| `connection_string` | 接続文字列を使用した認証に利用します。 | -| `account_name` | 共有キー認証(`account_key` と併用)に利用します。 | -| `account_key` | 共有キー認証(`account_name` と併用)に利用します。 | - -#### 制限パラメータ {#azure-blob-storage-limit-parameters} - -| Parameter | Description | -| ------------------------------------ | ----------------------------------------- | -| `s3_max_single_part_upload_size` | Blob Storage への単一ブロックアップロードの最大サイズ。 | -| `min_bytes_for_seek` | シーク可能領域の最小サイズ。 | -| `max_single_read_retries` | Blob Storage からデータチャンクを読み取る試行回数の上限。 | -| `max_single_download_retries` | Blob Storage から読み取り用バッファをダウンロードする試行回数の上限。 | -| `thread_pool_size` | `IDiskRemote` のインスタンス化に使用されるスレッド数の上限。 | -| `s3_max_inflight_parts_for_one_file` | 1 つのオブジェクトに対する同時 put リクエスト数の上限。 | - -#### その他のパラメータ {#azure-blob-storage-other-parameters} - -| Parameter | Description | Default Value | -| -------------------------------- | ---------------------------------------------------------------- | ---------------------------------------- | -| `metadata_path` | Blob Storage 用のメタデータファイルを保存するローカルファイルシステム上のパス。 | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | `true` の場合、起動時のディスクアクセスチェックをスキップします。 | `false` | -| `read_resource` | [スケジューリング](/operations/workload-scheduling.md) における読み取り要求のリソース名。 | 空文字列(無効) | -| `write_resource` | [スケジューリング](/operations/workload-scheduling.md) における書き込み要求のリソース名。 | 空文字列(無効) | -| `metadata_keep_free_space_bytes` | メタデータディスクに予約しておく空き容量(バイト数)。 | - | - -動作する構成の例は integration tests ディレクトリ内で確認できます(例: [test_merge_tree_azure_blob_storage](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_merge_tree_azure_blob_storage/configs/config.d/storage_conf.xml) や [test_azure_blob_storage_zero_copy_replication](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_azure_blob_storage_zero_copy_replication/configs/config.d/storage_conf.xml) など)。 - -:::note Zero-copy レプリケーションは本番環境利用の準備が整っていません -Zero-copy レプリケーションは ClickHouse バージョン 22.8 以降ではデフォルトで無効です。この機能の本番環境での利用は推奨されません。 -::: - - -## HDFS ストレージの使用(サポート対象外) {#using-hdfs-storage-unsupported} - -このサンプル構成では: - -* ディスクのタイプは `hdfs`(サポート対象外)です -* データは `hdfs://hdfs1:9000/clickhouse/` にホストされています - -なお、HDFS はサポート対象外であるため、使用時に問題が発生する可能性があります。問題が発生した場合は、その修正を含む Pull Request の送信を歓迎します。 - -```xml - - - - - hdfs - hdfs://hdfs1:9000/clickhouse/ - true - - - local - / - - - - - -
- hdfs -
- - hdd - -
-
-
-
-
-``` - -HDFS は一部のコーナーケースでは動作しない場合があることに注意してください。 - -### データ暗号化の使用 {#encrypted-virtual-file-system} - -[S3](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3) や [HDFS](#using-hdfs-storage-unsupported)(非サポート)の外部ディスク、あるいはローカルディスクに保存されるデータを暗号化できます。暗号化モードを有効にするには、設定ファイル内で `encrypted` 型のディスクを定義し、データを保存するディスクを選択する必要があります。`encrypted` ディスクは、書き込まれるすべてのファイルをリアルタイムに暗号化し、`encrypted` ディスクからファイルを読み取るときには自動的に復号します。そのため、通常のディスクと同様に `encrypted` ディスクを扱うことができます。 - -ディスク設定の例: - -```xml - - - local - /path1/ - - - encrypted - disk1 - path2/ - _16_ascii_chars_ - - -``` - -例えば、ClickHouse があるテーブルのデータをファイル `store/all_1_1_0/data.bin` として `disk1` に書き込む場合、実際にはこのファイルは物理ディスク上のパス `/path1/store/all_1_1_0/data.bin` に書き込まれます。 - -同じファイルを `disk2` に書き込む場合は、実際には暗号化モードで物理ディスク上のパス `/path1/path2/store/all_1_1_0/data.bin` に書き込まれます。 - -### 必須パラメータ {#required-parameters-encrypted-disk} - -| Parameter | Type | Description | -| --------- | ------ | ----------------------------------------------------------------------- | -| `type` | String | 暗号化ディスクを作成するには `encrypted` を指定する必要があります。 | -| `disk` | String | 基盤となるストレージとして使用するディスクの種類。 | -| `key` | Uint64 | 暗号化および復号に使用するキー。`key_hex` を使用して 16 進数で指定できます。複数のキーは `id` 属性を使用して指定できます。 | - -### オプションパラメータ {#optional-parameters-encrypted-disk} - -| Parameter | Type | Default | Description | -| ---------------- | ------ | -------------- | ------------------------------------------------------------------------------------------------------------------- | -| `path` | String | Root directory | データが保存されるディスク上の場所。 | -| `current_key_id` | String | - | 暗号化に使用されるキー ID。指定されたすべてのキーは復号に使用できます。 | -| `algorithm` | Enum | `AES_128_CTR` | 暗号化アルゴリズム。オプション:
- `AES_128_CTR` (16 バイトキー)
- `AES_192_CTR` (24 バイトキー)
- `AES_256_CTR` (32 バイトキー) | - -ディスク設定の例: - - -```xml - - - - - s3 - ... - - - encrypted - disk_s3 - AES_128_CTR - 00112233445566778899aabbccddeeff - ffeeddccbbaa99887766554433221100 - 1 - - - - -``` - -### ローカルキャッシュの使用 {#using-local-cache} - -バージョン 22.3 以降では、ストレージ設定でディスクに対してローカルキャッシュを設定できます。 -バージョン 22.3 ~ 22.7 では、キャッシュは `s3` ディスクタイプでのみサポートされます。バージョン >= 22.8 では、キャッシュは任意のディスクタイプ (S3、Azure、Local、Encrypted など) でサポートされます。 -バージョン >= 23.5 では、キャッシュはリモートディスクタイプ (S3、Azure、HDFS (非サポート)) に対してのみサポートされます。 -キャッシュは `LRU` キャッシュポリシーで管理されます。 - -22.8 以降のバージョンにおける設定例: - -```xml - - - - - s3 - ... - ... S3の設定 ... - - - cache - s3 - /s3_cache/ - 10Gi - - - - - -
- cache -
-
-
- -
-``` - -22.8 より前のバージョン向けの設定例: - -```xml - - - - - s3 - ... - ... S3の設定 ... - 1 - 10737418240 - - - - - -
- s3 -
-
-
- -
-``` - -ファイルキャッシュの**ディスク構成設定**: - -これらの設定は、ディスク構成セクション内で定義する必要があります。 - - -| Parameter | Type | Default | Description | -|---------------------------------------|---------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `path` | String | - | **必須**。キャッシュを保存するディレクトリへのパス。 | -| `max_size` | Size | - | **必須**。キャッシュの最大サイズ(バイト数または可読形式。例: `10Gi`)。上限に達すると、LRU ポリシーでファイルが削除されます。`ki`、`Mi`、`Gi` 形式をサポートします(v22.10 以降)。 | -| `cache_on_write_operations` | Boolean | `false` | `INSERT` クエリおよびバックグラウンドマージに対してライトスルーキャッシュを有効にします。クエリ単位で `enable_filesystem_cache_on_write_operations` により上書きできます。 | -| `enable_filesystem_query_cache_limit` | Boolean | `false` | `max_query_cache_size` に基づくクエリ単位のキャッシュサイズ上限を有効にします。 | -| `enable_cache_hits_threshold` | Boolean | `false` | 有効化すると、データが複数回読み取られた後にのみキャッシュされます。 | -| `cache_hits_threshold` | Integer | `0` | データをキャッシュするまでに必要な読み取り回数(`enable_cache_hits_threshold` が有効である必要があります)。 | -| `enable_bypass_cache_with_threshold` | Boolean | `false` | 大きな読み取り範囲の場合にキャッシュをバイパスします。 | -| `bypass_cache_threshold` | Size | `256Mi` | キャッシュのバイパスをトリガーする読み取り範囲サイズ(`enable_bypass_cache_with_threshold` が有効である必要があります)。 | -| `max_file_segment_size` | Size | `8Mi` | 1 つのキャッシュファイルの最大サイズ(バイト数または可読形式)。 | -| `max_elements` | Integer | `10000000` | キャッシュファイルの最大数。 | -| `load_metadata_threads` | Integer | `16` | 起動時にキャッシュメタデータを読み込むスレッド数。 | - -> **Note**: サイズ指定値は `ki`、`Mi`、`Gi` などの単位をサポートします(例: `10Gi`)。 - - - -## ファイルキャッシュのクエリ/プロファイル設定 {#file-cache-query-profile-settings} - -| Setting | Type | Default | Description | -| ----------------------------------------------------------------------- | ------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------- | -| `enable_filesystem_cache` | Boolean | `true` | `cache` ディスクタイプを使用している場合でも、クエリごとにキャッシュの使用を有効/無効にします。 | -| `read_from_filesystem_cache_if_exists_otherwise_bypass_cache` | Boolean | `false` | 有効にすると、データが存在するときにのみキャッシュを使用します。新しいデータはキャッシュされません。 | -| `enable_filesystem_cache_on_write_operations` | Boolean | `false` (Cloud: `true`) | ライトスルー型キャッシュを有効にします。キャッシュ構成で `cache_on_write_operations` を有効にする必要があります。 | -| `enable_filesystem_cache_log` | Boolean | `false` | `system.filesystem_cache_log` への詳細なキャッシュ利用状況のログ出力を有効にします。 | -| `filesystem_cache_allow_background_download` | Boolean | `true` | 部分的にダウンロードされたセグメントをバックグラウンドで完了させることを許可します。現在のクエリ/セッションではダウンロードをフォアグラウンドで行わせたい場合は無効にします。 | -| `max_query_cache_size` | Size | `false` | クエリごとの最大キャッシュサイズです。キャッシュ構成で `enable_filesystem_query_cache_limit` を有効にする必要があります。 | -| `filesystem_cache_skip_download_if_exceeds_per_query_cache_write_limit` | Boolean | `true` | `max_query_cache_size` に達したときの動作を制御します:
- `true`: 新しいデータのダウンロードを停止する
- `false`: 新しいデータのために古いデータを削除する | - -:::warning -キャッシュ構成設定およびキャッシュ関連のクエリ設定は最新の ClickHouse バージョンに対応しています。 -以前のバージョンでは一部がサポートされていない場合があります。 -::: - -#### キャッシュのシステムテーブル {#cache-system-tables-file-cache} - -| Table Name | Description | Requirements | -| ----------------------------- | -------------------------- | ------------------------------------------ | -| `system.filesystem_cache` | ファイルシステムキャッシュの現在の状態を表示します。 | なし | -| `system.filesystem_cache_log` | クエリごとの詳細なキャッシュ利用統計を提供します。 | `enable_filesystem_cache_log = true` が必要です | - -#### キャッシュコマンド {#cache-commands-file-cache} - -##### `SYSTEM DROP FILESYSTEM CACHE () (ON CLUSTER)` -- `ON CLUSTER` {#system-drop-filesystem-cache-on-cluster} - -このコマンドは `` が指定されていない場合にのみサポートされています。 - -##### `SHOW FILESYSTEM CACHES` {#show-filesystem-caches} - -サーバー上で構成されているファイルシステムキャッシュの一覧を表示します。 -(バージョン `22.8` 以下では、このコマンド名は `SHOW CACHES` です) - -```sql title="Query" -SHOW FILESYSTEM CACHES -``` - -```text title="Response" -┌─Caches────┐ -│ s3_cache │ -└───────────┘ -``` - -##### `DESCRIBE FILESYSTEM CACHE ''` {#describe-filesystem-cache} - -特定のキャッシュについて、キャッシュ設定といくつかの基本的な統計情報を表示します。 -キャッシュ名は `SHOW FILESYSTEM CACHES` コマンドから取得できます(バージョン `22.8` 以下では、コマンド名は `DESCRIBE CACHE` です)。 - -```sql title="Query" -DESCRIBE FILESYSTEM CACHE 's3_cache' -``` - -```text title="Response" -┌────max_size─┬─max_elements─┬─max_file_segment_size─┬─boundary_alignment─┬─cache_on_write_operations─┬─cache_hits_threshold─┬─current_size─┬─current_elements─┬─path───────┬─background_download_threads─┬─enable_bypass_cache_with_threshold─┐ -│ 10000000000 │ 1048576 │ 104857600 │ 4194304 │ 1 │ 0 │ 3276 │ 54 │ /s3_cache/ │ 2 │ 0 │ -└─────────────┴──────────────┴───────────────────────┴────────────────────┴───────────────────────────┴──────────────────────┴──────────────┴──────────────────┴────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - - -| キャッシュ現在メトリクス | キャッシュ非同期メトリクス | キャッシュプロファイルイベント | -| ------------------------- | ---------------------- | ----------------------------------------------------------------------------------------- | -| `FilesystemCacheSize` | `FilesystemCacheBytes` | `CachedReadBufferReadFromSourceBytes`, `CachedReadBufferReadFromCacheBytes` | -| `FilesystemCacheElements` | `FilesystemCacheFiles` | `CachedReadBufferReadFromSourceMicroseconds`, `CachedReadBufferReadFromCacheMicroseconds` | -| | | `CachedReadBufferCacheWriteBytes`, `CachedReadBufferCacheWriteMicroseconds` | -| | | `CachedWriteBufferCacheWriteBytes`, `CachedWriteBufferCacheWriteMicroseconds` | - -### 静的 Web ストレージの使用(読み取り専用) {#web-storage} - -これは読み取り専用ディスクです。データは読み出されるだけで、一切変更されません。新しいテーブルは -`ATTACH TABLE` クエリ(以下の例を参照)を介してこのディスクにロードされます。ローカルディスクは -実際には使用されず、各 `SELECT` クエリは必要なデータを取得するために `http` リクエストを発行します。 -テーブルデータを変更しようとするとすべて例外がスローされます。つまり、次の種類のクエリは許可されません: -[`CREATE TABLE`](/sql-reference/statements/create/table.md), -[`ALTER TABLE`](/sql-reference/statements/alter/index.md), [`RENAME TABLE`](/sql-reference/statements/rename#rename-table), -[`DETACH TABLE`](/sql-reference/statements/detach.md) および [`TRUNCATE TABLE`](/sql-reference/statements/truncate.md)。 -Web ストレージは読み取り専用用途に使用できます。典型的な用途の例としては、サンプルデータのホスティングや、 -データの移行があります。`clickhouse-static-files-uploader` というツールがあり、 -指定したテーブル用のデータディレクトリを準備します -(`SELECT data_paths FROM system.tables WHERE name = 'table_name'`)。 -必要な各テーブルごとに、ファイルが格納されたディレクトリを 1 つ取得できます。これらのファイルを、 -たとえば静的ファイルを配信する Web サーバーにアップロードできます。この準備が完了したら、 -`DiskWeb` を介して任意の ClickHouse サーバーにこのテーブルをロードできます。 - -このサンプル設定では: - -* ディスクのタイプは `web` -* データは `http://nginx:80/test1/` でホストされている -* ローカルストレージ上のキャッシュが使用される - -```xml - - - - - web - http://nginx:80/test1/ - - - cache - web - cached_web_cache/ - 100000000 - - - - - -
- web -
-
-
- - -
- cached_web -
-
-
-
-
-
-``` - -:::tip -Web データセットを日常的に使用しない想定であれば、ストレージはクエリ内で一時的に構成することもできます。[動的設定](#dynamic-configuration) を参照し、 -設定ファイルの編集は省略できます。 - -[デモデータセット](https://github.com/ClickHouse/web-tables-demo) が GitHub 上でホストされています。独自のテーブルを Web ストレージ用に準備するには、ツール [clickhouse-static-files-uploader](/operations/utilities/static-files-disk-uploader) を使用してください。 -::: - -この `ATTACH TABLE` クエリでは、指定された `UUID` がデータのディレクトリ名と一致し、エンドポイントには GitHub の Raw コンテンツの URL を指定します。 - - -```sql --- highlight-next-line -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -すぐに試せるテストケースです。この設定を `config` に追加する必要があります: - -```xml - - - - - web - https://clickhouse-datasets.s3.yandex.net/disk-with-static-files-tests/test-hits/ - - - - - -
- web -
-
-
-
-
-
-``` - -次のクエリを実行します。 - - -```sql -ATTACH TABLE test_hits UUID '1ae36516-d62d-4218-9ae3-6516d62da218' -( - WatchID UInt64, - JavaEnable UInt8, - Title String, - GoodEvent Int16, - EventTime DateTime, - EventDate Date, - CounterID UInt32, - ClientIP UInt32, - ClientIP6 FixedString(16), - RegionID UInt32, - UserID UInt64, - CounterClass Int8, - OS UInt8, - UserAgent UInt8, - URL String, - Referer String, - URLDomain String, - RefererDomain String, - Refresh UInt8, - IsRobot UInt8, - RefererCategories Array(UInt16), - URLCategories Array(UInt16), - URLRegions Array(UInt32), - RefererRegions Array(UInt32), - ResolutionWidth UInt16, - ResolutionHeight UInt16, - ResolutionDepth UInt8, - FlashMajor UInt8, - FlashMinor UInt8, - FlashMinor2 String, - NetMajor UInt8, - NetMinor UInt8, - UserAgentMajor UInt16, - UserAgentMinor FixedString(2), - CookieEnable UInt8, - JavascriptEnable UInt8, - IsMobile UInt8, - MobilePhone UInt8, - MobilePhoneModel String, - Params String, - IPNetworkID UInt32, - TraficSourceID Int8, - SearchEngineID UInt16, - SearchPhrase String, - AdvEngineID UInt8, - IsArtifical UInt8, - WindowClientWidth UInt16, - WindowClientHeight UInt16, - ClientTimeZone Int16, - ClientEventTime DateTime, - SilverlightVersion1 UInt8, - SilverlightVersion2 UInt8, - SilverlightVersion3 UInt32, - SilverlightVersion4 UInt16, - PageCharset String, - CodeVersion UInt32, - IsLink UInt8, - IsDownload UInt8, - IsNotBounce UInt8, - FUniqID UInt64, - HID UInt32, - IsOldCounter UInt8, - IsEvent UInt8, - IsParameter UInt8, - DontCountHits UInt8, - WithHash UInt8, - HitColor FixedString(1), - UTCEventTime DateTime, - Age UInt8, - Sex UInt8, - Income UInt8, - Interests UInt16, - Robotness UInt8, - GeneralInterests Array(UInt16), - RemoteIP UInt32, - RemoteIP6 FixedString(16), - WindowName Int32, - OpenerName Int32, - HistoryLength Int16, - BrowserLanguage FixedString(2), - BrowserCountry FixedString(2), - SocialNetwork String, - SocialAction String, - HTTPError UInt16, - SendTiming Int32, - DNSTiming Int32, - ConnectTiming Int32, - ResponseStartTiming Int32, - ResponseEndTiming Int32, - FetchTiming Int32, - RedirectTiming Int32, - DOMInteractiveTiming Int32, - DOMContentLoadedTiming Int32, - DOMCompleteTiming Int32, - LoadEventStartTiming Int32, - LoadEventEndTiming Int32, - NSToDOMContentLoadedTiming Int32, - FirstPaintTiming Int32, - RedirectCount Int8, - SocialSourceNetworkID UInt8, - SocialSourcePage String, - ParamPrice Int64, - ParamOrderID String, - ParamCurrency FixedString(3), - ParamCurrencyID UInt16, - GoalsReached Array(UInt32), - OpenstatServiceName String, - OpenstatCampaignID String, - OpenstatAdID String, - OpenstatSourceID String, - UTMSource String, - UTMMedium String, - UTMCampaign String, - UTMContent String, - UTMTerm String, - FromTag String, - HasGCLID UInt8, - RefererHash UInt64, - URLHash UInt64, - CLID UInt32, - YCLID UInt64, - ShareService String, - ShareURL String, - ShareTitle String, - ParsedParams Nested( - Key1 String, - Key2 String, - Key3 String, - Key4 String, - Key5 String, - ValueDouble Float64), - IslandID FixedString(16), - RequestNum UInt32, - RequestTry UInt8 -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID) -SETTINGS storage_policy='web'; -``` - -#### 必須パラメータ {#static-web-storage-required-parameters} - -| パラメータ | 説明 | -| ---------- | -------------------------------------------------------------------------- | -| `type` | `web`。それ以外の値の場合、ディスクは作成されません。 | -| `endpoint` | `path` 形式のエンドポイント URL。エンドポイント URL には、アップロードされたデータを保存するためのルートパスを含める必要があります。 | - -#### オプションパラメータ {#optional-parameters-web} - - -| Parameter | Description | Default Value | -|-------------------------------------|------------------------------------------------------------------------------|-----------------| -| `min_bytes_for_seek` | シーケンシャル読み取りではなくシーク操作を使用するための最小バイト数 | `1` MB | -| `remote_fs_read_backoff_threashold` | リモートディスクからデータを読み取ろうとする際に待機する最大時間 | `10000` seconds | -| `remote_fs_read_backoff_max_tries` | バックオフしながら読み取りを試行する最大回数 | `5` | - -クエリが例外 `DB:Exception Unreachable URL` で失敗する場合は、次の設定値の調整を試してください: [http_connection_timeout](/operations/settings/settings.md/#http_connection_timeout)、[http_receive_timeout](/operations/settings/settings.md/#http_receive_timeout)、[keep_alive_timeout](/operations/server-configuration-parameters/settings#keep_alive_timeout)。 - -アップロード用のファイルを取得するには次を実行します: -`clickhouse static-files-disk-uploader --metadata-path --output-dir
`(`--metadata-path` はクエリ `SELECT data_paths FROM system.tables WHERE name = 'table_name'` で確認できます)。 - -`endpoint` 経由でファイルをロードする場合、ファイルは `/store/` パスに配置する必要がありますが、設定には `endpoint` のみを含める必要があります。 - -サーバー起動時にテーブルをロードする際、ディスクのロード時に URL に到達できない場合でも、すべてのエラーは捕捉されます。この場合にエラーが発生していた場合は、`DETACH TABLE table_name` -> `ATTACH TABLE table_name` によってテーブルを再ロード(再度利用可能に)できます。サーバー起動時にメタデータが正常にロードされていれば、テーブルは直ちに利用可能です。 - -単一の HTTP 読み取り中のリトライ回数の上限を制限するには、[http_max_single_read_retries](/operations/storing-data#web-storage) 設定を使用します。 - -### Zero-copy Replication(本番利用には未対応) {#zero-copy} - -Zero-copy replication は、`S3` および `HDFS`(非サポート)のディスクで利用可能ですが、推奨されません。Zero-copy replication とは、データが複数マシン上のリモートに保存されており同期が必要な場合に、データ自体ではなくメタデータ(データパーツへのパス)のみをレプリケートすることを意味します。 - -:::note Zero-copy replication は本番利用には未対応です -Zero-copy replication は ClickHouse バージョン 22.8 以降ではデフォルトで無効化されています。この機能は本番環境での利用は推奨されません。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md deleted file mode 100644 index b7c9595ca63..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '非同期インサートに関する情報を含むシステムテーブル。各エントリは、非同期インサート用にバッファリングされた INSERT クエリを表します。' -keywords: ['system table', 'asynchronous_insert_log'] -slug: /operations/system-tables/asynchronous_insert_log -title: 'system.asynchronous_insert_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_insert_log {#systemasynchronous_insert_log} - - - -非同期インサートに関する情報を保持します。各エントリは、非同期インサート用にバッファリングされたインサートクエリを表します。 - -ロギングを開始するには、[asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log) セクションでパラメータを設定します。 - -データのフラッシュ周期は、[asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log) サーバー設定セクションの `flush_interval_milliseconds` パラメータで設定します。フラッシュを強制するには、[SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) クエリを使用します。 - -ClickHouse はテーブルからデータを自動的に削除しません。詳細は [Introduction](/operations/system-tables/overview#system-tables-introduction) を参照してください。 - -カラム: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 非同期インサートが発生した日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 非同期インサートの実行が完了した日時。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 非同期インサートの実行が完了した日時(マイクロ秒精度)。 -* `query` ([String](../../sql-reference/data-types/string.md)) — クエリ文字列。 -* `database` ([String](../../sql-reference/data-types/string.md)) — テーブルが属するデータベース名。 -* `table` ([String](../../sql-reference/data-types/string.md)) — テーブル名。 -* `format` ([String](/sql-reference/data-types/string.md)) — フォーマット名。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 元のクエリの ID。 -* `bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 挿入されたバイト数。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 例外メッセージ。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — ステータス。値: - * `'Ok' = 1` — インサート成功。 - * `'ParsingError' = 2` — データのパース時に発生した例外。 - * `'FlushError' = 3` — データのフラッシュ時に発生した例外。 -* `flush_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — フラッシュが発生した日時。 -* `flush_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — フラッシュが発生した日時(マイクロ秒精度)。 -* `flush_query_id` ([String](../../sql-reference/data-types/string.md)) — フラッシュクエリの ID。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.asynchronous_insert_log LIMIT 1 \G; -``` - -結果: - -```text -hostname: clickhouse.eu-central1.internal -event_date: 2023-06-08 -event_time: 2023-06-08 10:08:53 -event_time_microseconds: 2023-06-08 10:08:53.199516 -query: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -database: public -table: data_guess -format: CSV -query_id: b46cd4c4-0269-4d0b-99f5-d27668c6102e -bytes: 133223 -exception: -status: Ok -flush_time: 2023-06-08 10:08:55 -flush_time_microseconds: 2023-06-08 10:08:55.139676 -flush_query_id: cd2c1e43-83f5-49dc-92e4-2fbc7f8d3716 -``` - -**関連項目** - -* [system.query_log](../../operations/system-tables/query_log) — クエリ実行に関する一般的な情報を含む `query_log` システムテーブルの説明。 -* [system.asynchronous_inserts](/operations/system-tables/asynchronous_inserts) — キュー内の保留中の非同期挿入に関する情報を保持するテーブル。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md deleted file mode 100644 index a7889e28255..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'キュー内の保留中の非同期挿入に関する情報を保持するシステムテーブル。' -keywords: ['システムテーブル', 'asynchronous_inserts'] -slug: /operations/system-tables/asynchronous_inserts -title: 'system.asynchronous_inserts' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -キュー内で保留中の非同期挿入に関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -クエリ: - -```sql -SELECT * FROM system.asynchronous_inserts LIMIT 1 \G; -``` - -結果: - -```text -行 1: -────── -query: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -database: public -table: data_guess -format: CSV -first_update: 2023-06-08 10:08:54.199606 -total_bytes: 133223 -entries.query_id: ['b46cd4c4-0269-4d0b-99f5-d27668c6102e'] -entries.bytes: [133223] -``` - -**関連項目** - -* [system.query_log](/operations/system-tables/query_log) — クエリ実行に関する共通情報を含む `query_log` システムテーブルの説明。 -* [system.asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) — 実行された非同期インサートに関する情報を含むテーブル。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md deleted file mode 100644 index 08d2a2efe5c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: '最近の非同期ジョブ(例: 読み込み中のテーブルに対するジョブ)に関する情報と状態を含むシステムテーブル。このテーブルにはジョブごとに 1 行が格納されます。' -keywords: ['system テーブル', 'asynchronous_loader'] -slug: /operations/system-tables/asynchronous_loader -title: 'system.asynchronous_loader' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_loader {#systemasynchronous_loader} - - - -最近の非同期ジョブ(例: テーブルのロード)に関する情報およびステータスを含みます。このテーブルには、各ジョブごとに 1 行が格納されます。このテーブルの情報を可視化するためのツールとして `utils/async_loader_graph` が用意されています。 - -例: - -```sql -SELECT * -FROM system.asynchronous_loader -LIMIT 1 -FORMAT Vertical -``` - -カラム: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -保留中のジョブは、次のいずれかの状態になっている可能性があります: - -* `is_executing` (`UInt8`) - ジョブが現在ワーカーによって実行中である。 -* `is_blocked` (`UInt8`) - ジョブが依存関係の完了を待機している。 -* `is_ready` (`UInt8`) - ジョブが実行可能な状態で、ワーカーを待機している。 -* `elapsed` (`Float64`) - 実行開始から経過した秒数。ジョブが開始されていなければ 0。ジョブが完了していれば合計実行時間。 - -すべてのジョブには関連付けられたプールがあり、そのプール内で開始されます。各プールには固定の優先度と、変更可能な最大ワーカー数があります。優先度が高い(`priority` 値が低い)ジョブが先に実行されます。少なくとも 1 つの高優先度ジョブが実行中または実行可能である間は、低優先度ジョブは開始されません。ジョブの優先度は(下げることはできませんが)引き上げることができ、ジョブを優先順位付けすることで行います。たとえば、あるテーブルのロードや起動に関するジョブは、そのテーブルを必要とするクエリが到着した場合に優先されます。ジョブの実行中に優先順位付けを行うことも可能ですが、その場合でもジョブは `execution_pool` から新たに割り当てられた `pool` へは移動しません。ジョブは優先度逆転を避けるために、新しいジョブを作成する際に `pool` を使用します。すでに開始されたジョブは、高優先度ジョブによってプリエンプトされることはなく、開始後は常に完了まで実行されます。 - -* `pool_id` (`UInt64`) - 現在そのジョブに割り当てられているプールの ID。 - -* `pool` (`String`) - `pool_id` のプール名。 - -* `priority` (`Int64`) - `pool_id` のプールの優先度。 - -* `execution_pool_id` (`UInt64`) - ジョブが実行されているプールの ID。実行開始前に最初に割り当てられたプールと同一。 - -* `execution_pool` (`String`) - `execution_pool_id` のプール名。 - -* `execution_priority` (`Int64`) - `execution_pool_id` のプールの優先度。 - -* `ready_seqno` (`Nullable(UInt64)`) - 実行可能なジョブに対して非 Null。ワーカーは、そのプールの ready キューから次に実行するジョブを取得します。複数の実行可能ジョブがある場合、`ready_seqno` の値が最も小さいジョブが選択されます。 - -* `waiters` (`UInt64`) - このジョブを待機しているスレッド数。 - -* `exception` (`Nullable(String)`) - 失敗またはキャンセルされたジョブに対して非 Null。クエリ実行中に発生したエラーメッセージ、またはこのジョブのキャンセルの原因となったエラーと、それに至る依存関係の失敗によるジョブ名のチェーンを保持します。 - -ジョブのライフタイム中の時刻情報: - -* `schedule_time` (`DateTime64`) - ジョブが作成され、(通常はすべての依存関係とともに)実行するようスケジュールされた時刻。 -* `enqueue_time` (`Nullable(DateTime64)`) - ジョブが実行可能になり、そのプールの ready キューにエンキューされた時刻。ジョブがまだ実行可能でなければ Null。 -* `start_time` (`Nullable(DateTime64)`) - ワーカーが ready キューからジョブをデキューし、その実行を開始した時刻。ジョブがまだ開始されていなければ Null。 -* `finish_time` (`Nullable(DateTime64)`) - ジョブの実行が完了した時刻。ジョブがまだ完了していなければ Null。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md deleted file mode 100644 index 3e78776a81e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '`system.asynchronous_metrics` の履歴値を保持するシステムテーブル。これらの値は一定の時間間隔ごと(デフォルトでは 1 秒ごと)に保存されます' -keywords: ['system table', 'asynchronous_metric_log'] -slug: /operations/system-tables/asynchronous_metric_log -title: 'system.asynchronous_metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -`system.asynchronous_metrics` の過去の値を保持します。これらは一定の時間間隔ごと(デフォルトでは 1 秒ごと)に保存されます。デフォルトで有効です。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `metric` ([String](../../sql-reference/data-types/string.md)) — メトリクス名。 -* `value` ([Float64](../../sql-reference/data-types/float.md)) — メトリクス値。 - -**例** - -```sql -SELECT * FROM system.asynchronous_metric_log LIMIT 3 \G -``` - -```text -1 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:07 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0.001 - -2 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:08 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 - -3 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:09 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 -``` - -**関連項目** - -* [asynchronous_metric_log 設定](../../operations/server-configuration-parameters/settings.md#asynchronous_metric_log) — この設定の有効化と無効化の方法。 -* [system.asynchronous_metrics](../system-tables/asynchronous_metrics.md) — バックグラウンドで定期的に計算されるメトリクスを含みます。 -* [system.metric_log](../system-tables/metric_log.md) — テーブル `system.metrics` および `system.events` のメトリクス値の履歴を保持し、定期的にディスクへフラッシュされます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md deleted file mode 100644 index 11cbcdbca5d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md +++ /dev/null @@ -1,631 +0,0 @@ ---- -description: 'バックグラウンドで定期的に計算されるメトリクスを保持する system テーブル。例えば、使用中の RAM の量。' -keywords: ['system テーブル', 'asynchronous_metrics'] -slug: /operations/system-tables/asynchronous_metrics -title: 'system.asynchronous_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_metrics {#systemasynchronous_metrics} - - - -バックグラウンドで定期的に計算されるメトリクスを保持しています。たとえば、使用中の RAM の量などです。 - -列: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — メトリクス名。 -* `value` ([Float64](../../sql-reference/data-types/float.md)) — メトリクス値。 -* `description` ([String](../../sql-reference/data-types/string.md)) — メトリクスの説明 - -**例** - -```sql -SELECT * FROM system.asynchronous_metrics LIMIT 10 -``` - -```text -┌─metric──────────────────────────────────┬──────value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ AsynchronousMetricsCalculationTimeSpent │ 0.00179053 │ 非同期メトリクスの計算に費やされた時間(秒単位)。これは非同期メトリクスのオーバーヘッドです。 │ -│ NumberOfDetachedByUserParts │ 0 │ ユーザーが `ALTER TABLE DETACH` クエリでMergeTreeテーブルから切り離したパーツの総数(予期しないパーツ、破損したパーツ、無視されたパーツとは異なります)。サーバーは切り離されたパーツを管理しないため、削除可能です。 │ -│ NumberOfDetachedParts │ 0 │ MergeTreeテーブルから切り離されたパーツの総数。パーツは、ユーザーが `ALTER TABLE DETACH` クエリで切り離すか、パーツが破損、予期しない、または不要な場合にサーバー自体が切り離すことがあります。サーバーは切り離されたパーツを管理しないため、削除可能です。 │ -│ TotalRowsOfMergeTreeTables │ 2781309 │ MergeTreeファミリーのすべてのテーブルに格納されている行(レコード)の総数。 │ -│ TotalBytesOfMergeTreeTables │ 7741926 │ MergeTreeファミリーのすべてのテーブルに格納されているバイト数の総量(データとインデックスを含む圧縮済み)。 │ -│ NumberOfTables │ 93 │ サーバー上のデータベース全体で集計されたテーブルの総数。MergeTreeテーブルを含むことができないデータベースは除外されます。除外されるデータベースエンジンは、`Lazy`、`MySQL`、`PostgreSQL`、`SQlite`など、テーブルのセットを動的に生成するものです。 │ -│ NumberOfDatabases │ 6 │ サーバー上のデータベースの総数。 │ -│ MaxPartCountForPartition │ 6 │ MergeTreeファミリーのすべてのテーブルの全パーティションにおける、パーティションあたりのパーツの最大数。300を超える値は、設定ミス、過負荷、または大量のデータロードを示します。 │ -│ ReplicasSumMergesInQueue │ 0 │ Replicatedテーブル全体でキュー内のマージ操作(まだ適用されていないもの)の合計。 │ -│ ReplicasSumInsertsInQueue │ 0 │ Replicatedテーブル全体でキュー内のINSERT操作(まだ複製されていないもの)の合計。 │ -└─────────────────────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -{/*- system.events や system.metrics と異なり、非同期メトリクスはソースコードファイル内に単純な一覧として定義されているわけではなく、 - src/Interpreters/ServerAsynchronousMetrics.cpp 内のロジックと混在して記述されています。 - 読者の便宜のため、ここで明示的に列挙します。 -*/ } - -## メトリクスの説明 {#metric-descriptions} - -### AsynchronousHeavyMetricsCalculationTimeSpent {#asynchronousheavymetricscalculationtimespent} - -非同期の重い(テーブル関連の)メトリクスの計算に費やされた時間(秒単位)。これは非同期メトリクスのオーバーヘッドです。 - -### AsynchronousHeavyMetricsUpdateInterval {#asynchronousheavymetricsupdateinterval} - -重い(テーブル関連の)メトリクスの更新間隔 - -### AsynchronousMetricsCalculationTimeSpent {#asynchronousmetricscalculationtimespent} - -非同期メトリクスの計算に費やされた時間(秒単位)。これは非同期メトリクスのオーバーヘッドです。 - -### AsynchronousMetricsUpdateInterval {#asynchronousmetricsupdateinterval} - -メトリクスの更新間隔 - -### BlockActiveTime_*name* {#blockactivetime_name} - -ブロックデバイスで I/O リクエストがキューに積まれていた時間(秒単位)。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockDiscardBytes_*name* {#blockdiscardbytes_name} - -ブロックデバイス上で破棄されたバイト数。これらの操作は SSD に関連があります。破棄(discard)操作は ClickHouse では使用されませんが、システム上の他のプロセスで使用される可能性があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockDiscardMerges_*name* {#blockdiscardmerges_name} - -ブロックデバイスに対して要求され、OS の I/O スケジューラによってマージされた破棄(discard)操作の数。これらの操作は SSD に関連があります。破棄操作は ClickHouse では使用されませんが、システム上の他のプロセスで使用される可能性があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockDiscardOps_*name* {#blockdiscardops_name} - -ブロックデバイスに対して要求された破棄(discard)操作の数。これらの操作は SSD に関連があります。破棄操作は ClickHouse では使用されませんが、システム上の他のプロセスで使用される可能性があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockDiscardTime_*name* {#blockdiscardtime_name} - -ブロックデバイスに対して要求された破棄(discard)操作に費やされた時間(秒単位)の合計。すべての操作にわたって合算されます。これらの操作は SSD に関連があります。破棄操作は ClickHouse では使用されませんが、システム上の他のプロセスで使用される可能性があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockInFlightOps_*name* {#blockinflightops_name} - -デバイスドライバに発行されたものの、まだ完了していない I/O リクエストの数。この値には、キュー内にはあるがまだデバイスドライバに発行されていない I/O リクエストは含まれません。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockQueueTime_*name* {#blockqueuetime_name} - -この値は、I/O リクエストがこのブロックデバイス上で待機していたミリ秒数をカウントします。複数の I/O リクエストが待機している場合、この値は「ミリ秒数 × 待機しているリクエスト数」の積として増加します。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockReadBytes_*name* {#blockreadbytes_name} - -ブロックデバイスから読み取られたバイト数。OS のページキャッシュの利用により I/O が節約されるため、ファイルシステムから読み取られたバイト数より少なくなる場合があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockReadMerges_*name* {#blockreadmerges_name} - -ブロックデバイスに対して要求され、OS の I/O スケジューラによってマージされた読み取り操作の数。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけでなく)を含みます。ソース: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください。 - -### BlockReadOps_*name* {#blockreadops_name} - -ブロックデバイスに対して要求された読み取り処理の回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### BlockReadTime_*name* {#blockreadtime_name} - -ブロックデバイスに対して要求された読み取り処理に費やされた時間(秒)。すべての処理の合計です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### BlockWriteBytes_*name* {#blockwritebytes_name} - -ブロックデバイスに書き込まれたバイト数。OS のページキャッシュの利用によって IO が削減されるため、ファイルシステムに書き込まれたバイト数よりも小さくなる場合があります。ライトスルーキャッシュにより、対応するファイルシステムへの書き込みより後になってブロックデバイスへの書き込みが行われることがあります。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### BlockWriteMerges_*name* {#blockwritemerges_name} - -OS の IO スケジューラによってマージされた、ブロックデバイスへの書き込み要求の回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### BlockWriteOps_*name* {#blockwriteops_name} - -ブロックデバイスに対して要求された書き込み処理の回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### BlockWriteTime_*name* {#blockwritetime_name} - -ブロックデバイスに対して要求された書き込み処理に費やされた時間(秒)。すべての処理の合計です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。出典: `/sys/block`。https://www.kernel.org/doc/Documentation/block/stat.txt を参照してください - -### CPUFrequencyMHz_*name* {#cpufrequencymhz_name} - -CPU の現在の動作周波数(MHz 単位)。最近の CPU の多くは、省電力やターボブーストのために周波数を動的に調整します。 - -### DictionaryMaxUpdateDelay {#dictionarymaxlastsuccessfulupdatetime} - -辞書更新の最大遅延時間(秒)。 - -### DictionaryTotalFailedUpdates {#dictionaryloadfailed} - -直近の成功した読み込み以降、すべての辞書で発生したエラーの総数。 - -### DiskAvailable_*name* {#diskavailable_name} - -ディスク(仮想ファイルシステム)上で利用可能なバイト数。リモートファイルシステムでは 16 EiB のような大きな値を報告することがあります。 - -### DiskTotal_*name* {#disktotal_name} - -ディスク(仮想ファイルシステム)の総サイズ(バイト単位)。リモートファイルシステムでは 16 EiB のような大きな値を報告することがあります。 - -### DiskUnreserved_*name* {#diskunreserved_name} - -マージ、フェッチ、移動のための予約分を除いた、ディスク(仮想ファイルシステム)上で利用可能なバイト数。リモートファイルシステムでは 16 EiB のような大きな値を報告することがあります。 - -### DiskUsed_*name* {#diskused_name} - -ディスク(仮想ファイルシステム)上で使用中のバイト数。リモートファイルシステムでは、この情報が常に提供されるとは限りません。 - -### FilesystemCacheBytes {#filesystemcachebytes} - -`cache` 仮想ファイルシステム内の合計バイト数。このキャッシュはディスク上に保持されます。 - -### FilesystemCacheFiles {#filesystemcachefiles} - -`cache` 仮想ファイルシステム内のキャッシュされたファイルセグメントの総数。このキャッシュはディスク上に保持されます。 - -### FilesystemLogsPathAvailableBytes {#filesystemlogspathavailablebytes} - -ClickHouse のログパスがマウントされているボリューム上で利用可能なバイト数。この値がゼロに近づいてきた場合は、設定ファイルでログローテーションを調整する必要があります。 - -### FilesystemLogsPathAvailableINodes {#filesystemlogspathavailableinodes} - -ClickHouse のログパスがマウントされているボリューム上で利用可能な iノード数。 - -### FilesystemLogsPathTotalBytes {#filesystemlogspathtotalbytes} - -ClickHouse のログパスがマウントされているボリュームのサイズ(バイト単位)。ログ用に少なくとも 10 GB を確保することを推奨します。 - -### FilesystemLogsPathTotalINodes {#filesystemlogspathtotalinodes} - -ClickHouse のログパスがマウントされているボリューム上の iノードの総数。 - -### FilesystemLogsPathUsedBytes {#filesystemlogspathusedbytes} - -ClickHouse のログパスがマウントされているボリューム上で使用中のバイト数。 - -### FilesystemLogsPathUsedINodes {#filesystemlogspathusedinodes} - -ClickHouse のログパスがマウントされているボリューム上で使用中の iノード数。 - -### FilesystemMainPathAvailableBytes {#filesystemmainpathavailablebytes} - -メインの ClickHouse パスがマウントされているボリュームで利用可能なバイト数。 - -### FilesystemMainPathAvailableINodes {#filesystemmainpathavailableinodes} - -メインの ClickHouse パスがマウントされているボリュームで利用可能な inode の数。値が 0 に近い場合は設定ミスを示しており、ディスクが満杯でなくても「no space left on device」というエラーが発生します。 - -### FilesystemMainPathTotalBytes {#filesystemmainpathtotalbytes} - -メインの ClickHouse パスがマウントされているボリュームのサイズ(バイト単位)。 - -### FilesystemMainPathTotalINodes {#filesystemmainpathtotalinodes} - -メインの ClickHouse パスがマウントされているボリューム上の inode の総数。2,500 万未満の場合、設定ミスを示します。 - -### FilesystemMainPathUsedBytes {#filesystemmainpathusedbytes} - -メインの ClickHouse パスがマウントされているボリュームで使用中のバイト数。 - -### FilesystemMainPathUsedINodes {#filesystemmainpathusedinodes} - -メインの ClickHouse パスがマウントされているボリュームで使用中の inode の数。この値は主にファイル数に対応します。 - -### HTTPThreads {#httpthreads} - -HTTP インターフェイスのサーバーにおけるスレッド数(TLS なし)。 - -### InterserverThreads {#interserverthreads} - -レプリカ間通信プロトコルのサーバーにおけるスレッド数(TLS なし)。 - -### Jitter {#jitter} - -非同期メトリクスを計算するスレッドが起床するようにスケジュールされた時刻と、実際に起床した時刻との差分。システム全体のレイテンシーと応答性の代理指標です。 - -### LoadAverage*N* {#loadaveragen} - -1 分間の指数平滑化平均で算出したシステム全体のロード。ロードは、すべてのプロセス(OS カーネルのスケジューリング対象)において、現在 CPU 上で実行中、IO 待ち、または今はスケジュールされていないが実行可能状態にあるスレッド数を表します。この数値には、clickhouse-server だけでなくすべてのプロセスが含まれます。システムが過負荷であり、多数のプロセスが実行可能だが CPU または IO を待っている場合、この数値は CPU コア数を上回ることがあります。 - -### MaxPartCountForPartition {#maxpartcountforpartition} - -すべての MergeTree 系テーブルのすべてのパーティションにおける、パーティションごとのパーツ数の最大値。300 を超える値は、設定ミス、過負荷、または大量データのロードを示します。 - -### MemoryCode {#memorycode} - -サーバープロセスの機械語ページ向けにマッピングされた仮想メモリ量(バイト単位)。 - -### MemoryDataAndStack {#memorydataandstack} - -スタックおよび割り当てられたメモリの使用のためにマッピングされた仮想メモリ量(バイト単位)。スレッドごとのスタックや、`mmap` システムコールで割り当てられた大部分のメモリを含むかどうかは規定されていません。このメトリクスは完全性のために存在します。監視には `MemoryResident` メトリクスを使用することを推奨します。 - -### MemoryResidentMax {#memoryresidentmax} - -サーバープロセスによって使用された物理メモリ量の最大値(バイト単位)。 - -### MemoryResident {#memoryresident} - -サーバープロセスによって使用されている物理メモリ量(バイト単位)。 - -### MemoryShared {#memoryshared} - -サーバープロセスによって使用され、かつ他のプロセスと共有されているメモリ量(バイト単位)。ClickHouse は共有メモリを使用しませんが、一部のメモリは OS の都合により共有としてラベル付けされる場合があります。このメトリクスを監視してもあまり意味はなく、完全性のために存在しています。 - -### MemoryVirtual {#memoryvirtual} - -サーバープロセスによって割り当てられた仮想アドレス空間のサイズ(バイト単位)。仮想アドレス空間のサイズは通常、物理メモリ消費量よりもはるかに大きく、メモリ消費量の推定には使用すべきではありません。このメトリクスの値が大きくてもまったく正常であり、技術的な意味しか持ちません。 - -### MySQLThreads {#mysqlthreads} - -MySQL 互換プロトコルのサーバーにおけるスレッド数。 - -### NetworkReceiveBytes_*name* {#networkreceivebytes_name} - -ネットワークインターフェイス経由で受信したバイト数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。 - -### NetworkReceiveDrop_*name* {#networkreceivedrop_name} - -ネットワークインターフェイス経由で受信中にドロップされたパケットのバイト数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。 - -### NetworkReceiveErrors_*name* {#networkreceiveerrors_name} - -ネットワークインターフェイス経由での受信時にエラーが発生した回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。 - -### NetworkReceivePackets_*name* {#networkreceivepackets_name} - - ネットワークインターフェイス経由で受信したネットワークパケット数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### NetworkSendBytes_*name* {#networksendbytes_name} - - ネットワークインターフェイス経由で送信されたバイト数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### NetworkSendDrop_*name* {#networksenddrop_name} - - ネットワークインターフェイス経由で送信中にパケットがドロップされた回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### NetworkSendErrors_*name* {#networksenderrors_name} - - ネットワークインターフェイス経由で送信中にエラー(例: TCP 再送)が発生した回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### NetworkSendPackets_*name* {#networksendpackets_name} - - ネットワークインターフェイス経由で送信されたネットワークパケット数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### NumberOfDatabases {#numberofdatabases} - -サーバー上のデータベースの総数。 - -### NumberOfDetachedByUserParts {#numberofdetachedbyuserparts} - -ユーザーが `ALTER TABLE DETACH` クエリを使用して MergeTree テーブルから切り離した part の総数(予期しない、破損した、または無視された part とは対照的)。サーバーは切り離された part を管理対象とはみなさず、これらは削除可能です。 - -### NumberOfDetachedParts {#numberofdetachedparts} - -MergeTree テーブルから切り離された part の総数。part は、ユーザーが `ALTER TABLE DETACH` クエリで切り離すか、part が破損している、予期しない、または不要な場合にはサーバー自身によって切り離されることがあります。サーバーは切り離された part を管理対象とはみなさず、これらは削除可能です。 - -### NumberOfTables {#numberoftables} - -サーバー上のデータベースをまたいで合計したテーブルの総数。ただし、MergeTree テーブルを含まないデータベースは除外されます。除外されるデータベースエンジンは、`Lazy`、`MySQL`、`PostgreSQL`、`SQlite` のように、テーブル集合をオンザフライで(動的に)生成するものです。 - -### OSContextSwitches {#oscontextswitches} - -ホストマシン上でシステムに発生したコンテキストスイッチの回数。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。 - -### OSGuestNiceTime {#osguestnicetime} - -Linux カーネルの制御下で、ゲスト OS 用の仮想 CPU を、ゲストが高い優先度に設定されている状態で実行するのに費やした時間の比率(`man procfs` を参照)。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。このメトリクスは ClickHouse にとっては無関係ですが、完全性のために存在します。単一の CPU コアについての値は [0..1] の範囲になります。すべての CPU コアについての値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSGuestNiceTimeCPU_*N* {#osguestnicetimecpu_n} - -Linux カーネルの制御下で、ゲスト OS 用の仮想 CPU を、ゲストが高い優先度に設定されている状態で実行するのに費やした時間の比率(`man procfs` を参照)。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。このメトリクスは ClickHouse にとっては無関係ですが、完全性のために存在します。単一の CPU コアについての値は [0..1] の範囲になります。すべての CPU コアについての値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSGuestNiceTimeNormalized {#osguestnicetimenormalized} - -値は `OSGuestNiceTime` と同様ですが、CPU コア数で割られており、コア数に依存せず [0..1] の範囲で測定されます。これにより、コア数が一様でない場合でも、クラスタ内の複数サーバーにわたってこのメトリクスの値を平均し、平均的なリソース使用率メトリクスを得ることが可能になります。 - -### OSGuestTime {#osguesttime} - -Linux カーネルの制御下でゲスト OS 用の仮想 CPU を実行するのに費やした時間の比率(`man procfs` を参照)。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。このメトリクスは ClickHouse にとっては無関係ですが、完全性のために存在します。単一の CPU コアについての値は [0..1] の範囲になります。すべての CPU コアについての値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSGuestTimeCPU_*N* {#osguesttimecpu_n} - -Linux カーネルの制御下で、ゲスト OS 向けの仮想 CPU を実行していた時間の比率です(`man procfs` を参照)。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。このメトリクスは ClickHouse にとっては本質的ではありませんが、完全性のために用意されています。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSGuestTimeNormalized {#osguesttimenormalized} - -値は `OSGuestTime` と同様ですが、CPU コア数で割ることで、コア数に関係なく [0..1] の範囲で測定されます。これにより、クラスタ内でコア数が一様でない複数サーバー間でも、このメトリクスの値を平均化して、平均的なリソース使用率メトリクスを取得できます。 - -### OSIOWaitTime {#osiowaittime} - -CPU コアがコードを実行しておらず、かつプロセスが I/O を待機しているために OS カーネルがこの CPU 上で他のいかなるプロセスも実行していなかった時間の比率です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSIOWaitTimeCPU_*N* {#osiowaittimecpu_n} - -CPU コアがコードを実行しておらず、かつプロセスが I/O を待機しているために OS カーネルがこの CPU 上で他のいかなるプロセスも実行していなかった時間の比率です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSIOWaitTimeNormalized {#osiowaittimenormalized} - -値は `OSIOWaitTime` と同様ですが、CPU コア数で割ることで、コア数に関係なく [0..1] の範囲で測定されます。これにより、クラスタ内でコア数が一様でない複数サーバー間でも、このメトリクスの値を平均化して、平均的なリソース使用率メトリクスを取得できます。 - -### OSIdleTime {#osidletime} - -OS カーネルの観点から見た、CPU コアがアイドル状態(I/O 待ちのプロセスを実行する準備すらしていない状態)であった時間の比率です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。これは、CPU 内部要因(メモリロード、パイプラインストール、分岐予測ミス、別の SMT コアの実行)によって CPU が十分に活用されていなかった時間は含みません。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSIdleTimeCPU_*N* {#osidletimecpu_n} - -OS カーネルの観点から見た、CPU コアがアイドル状態(I/O 待ちのプロセスを実行する準備すらしていない状態)であった時間の比率です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。これは、CPU 内部要因(メモリロード、パイプラインストール、分岐予測ミス、別の SMT コアの実行)によって CPU が十分に活用されていなかった時間は含みません。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSIdleTimeNormalized {#osidletimenormalized} - -値は `OSIdleTime` と同様ですが、CPU コア数で割ることで、コア数に関係なく [0..1] の範囲で測定されます。これにより、クラスタ内でコア数が一様でない複数サーバー間でも、このメトリクスの値を平均化して、平均的なリソース使用率メトリクスを取得できます。 - -### OSInterrupts {#osinterrupts} - -ホストマシン上で発生した割り込み回数です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。 - -### OSIrqTime {#osirqtime} - -CPU 上でハードウェア割り込み処理を実行していた時間の比率です。これはシステム全体のメトリクスであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスが含まれます。このメトリクスの値が高い場合は、ハードウェアの誤った構成、または非常に高いネットワーク負荷を示している可能性があります。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、各コアの値を合計して [0..num cores] の範囲で算出されます。 - -### OSIrqTimeCPU_*N* {#osirqtimecpu_n} - -CPU 上でハードウェア割り込み要求を処理するのに費やされた時間の比率です。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。このメトリクスの値が高い場合、ハードウェアの誤設定や非常に高いネットワーク負荷を示している可能性があります。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらの合計として計算され、[0..num cores] の範囲になります。 - -### OSIrqTimeNormalized {#osirqtimenormalized} - -`OSIrqTime` と同様の値ですが、CPU コア数で割ることで、コア数にかかわらず [0..1] の範囲で測定されます。これにより、コア数が一様でないクラスター内の複数サーバー間でも、このメトリクスの値を平均して、平均的なリソース使用率メトリクスを得ることができます。 - -### OSMemoryAvailable {#osmemoryavailable} - -プログラムで使用可能なメモリ量(バイト単位)です。これは `OSMemoryFreePlusCached` メトリクスと非常によく似ています。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSMemoryBuffers {#osmemorybuffers} - -OS カーネルバッファによって使用されているメモリ量(バイト単位)です。通常は小さい値であるべきであり、大きな値は OS の誤設定を示している可能性があります。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSMemoryCached {#osmemorycached} - -OS のページキャッシュによって使用されているメモリ量(バイト単位)です。通常、利用可能なメモリのほとんどすべてが OS のページキャッシュによって使用されるため、このメトリクスの値が高いことは正常であり、期待される動作です。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSMemoryFreePlusCached {#osmemoryfreepluscached} - -ホストシステム上の空きメモリと OS ページキャッシュメモリを合わせた量(バイト単位)です。このメモリはプログラムで使用可能です。この値は `OSMemoryAvailable` と非常によく似たものになるはずです。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSMemoryFreeWithoutCached {#osmemoryfreewithoutcached} - -ホストシステム上の空きメモリ量(バイト単位)です。これは、OS のページキャッシュメモリによって使用されているメモリ量(バイト単位)を含みません。ページキャッシュメモリもプログラムで利用可能であるため、このメトリクスの値は混乱を招く場合があります。代わりに `OSMemoryAvailable` メトリクスを参照してください。便宜上、`OSMemoryFreePlusCached` メトリクスも提供しており、これは OSMemoryAvailable とある程度似た値になるはずです。併せて https://www.linuxatemyram.com/ も参照してください。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSMemoryTotal {#osmemorytotal} - -ホストシステム上のメモリの総量(バイト単位)です。 - -### OSNiceTime {#osnicetime} - -CPU コアがより高い優先度でユーザ空間コードを実行していた時間の比率です。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらの合計として計算され、[0..num cores] の範囲になります。 - -### OSNiceTimeCPU_*N* {#osnicetimecpu_n} - -CPU コアがより高い優先度でユーザ空間コードを実行していた時間の比率です。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。単一の CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらの合計として計算され、[0..num cores] の範囲になります。 - -### OSNiceTimeNormalized {#osnicetimenormalized} - -`OSNiceTime` と同様の値ですが、CPU コア数で割ることで、コア数にかかわらず [0..1] の範囲で測定されます。これにより、コア数が一様でないクラスター内の複数サーバー間でも、このメトリクスの値を平均して、平均的なリソース使用率メトリクスを得ることができます。 - -### OSOpenFiles {#osopenfiles} - -ホストマシン上で開かれているファイルの総数です。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSProcessesBlocked {#osprocessesblocked} - -I/O の完了待ちでブロックされているスレッド数です(`man procfs` を参照)。これはシステム全体のメトリクスであり、ホストマシン上のすべてのプロセス(clickhouse-server だけではありません)を含みます。 - -### OSProcessesCreated {#osprocessescreated} - -作成されたプロセスの数。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。 - -### OSProcessesRunning {#osprocessesrunning} - -オペレーティングシステムによって「実行可能」(実行中または実行待ち)とみなされているスレッドの数。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。 - -### OSSoftIrqTime {#ossoftirqtime} - -CPU 上でソフトウェア割り込み要求を処理するのに費やされた時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。このメトリクスの値が高い場合、システム上で非効率なソフトウェアが動作している可能性を示します。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSSoftIrqTimeCPU_*N* {#ossoftirqtimecpu_n} - -CPU 上でソフトウェア割り込み要求を処理するのに費やされた時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。このメトリクスの値が高い場合、システム上で非効率なソフトウェアが動作している可能性を示します。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSSoftIrqTimeNormalized {#ossoftirqtimenormalized} - -値の意味は `OSSoftIrqTime` と同様ですが、CPU コア数で割ることで、コア数に依存せず常に [0..1] の範囲で測定されます。これにより、コア数が不均一な場合でも、クラスタ内の複数サーバーにわたってこのメトリクスの値を平均化し、平均的なリソース使用率メトリクスを得ることができます。 - -### OSStealTime {#osstealtime} - -仮想化環境で動作している際に、CPU が他のオペレーティングシステムで費やした時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。このメトリクスを提供する仮想化環境はすべてではなく、多くは提供しません。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSStealTimeCPU_*N* {#osstealtimecpu_n} - -仮想化環境で動作している際に、CPU が他のオペレーティングシステムで費やした時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。このメトリクスを提供する仮想化環境はすべてではなく、多くは提供しません。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSStealTimeNormalized {#osstealtimenormalized} - -値の意味は `OSStealTime` と同様ですが、CPU コア数で割ることで、コア数に依存せず常に [0..1] の範囲で測定されます。これにより、コア数が不均一な場合でも、クラスタ内の複数サーバーにわたってこのメトリクスの値を平均化し、平均的なリソース使用率メトリクスを得ることができます。 - -### OSSystemTime {#ossystemtime} - -CPU コアが OS カーネル(システム)コードを実行していた時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSSystemTimeCPU_*N* {#ossystemtimecpu_n} - -CPU コアが OS カーネル(システム)コードを実行していた時間の割合。これはシステム全体のメトリクスであり、clickhouse-server だけでなく、ホストマシン上のすべてのプロセスが含まれます。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらを合計した [0..num cores] の範囲で計算されます。 - -### OSSystemTimeNormalized {#ossystemtimenormalized} - -値の意味は `OSSystemTime` と同様ですが、CPU コア数で割ることで、コア数に依存せず常に [0..1] の範囲で測定されます。これにより、コア数が不均一な場合でも、クラスタ内の複数サーバーにわたってこのメトリクスの値を平均化し、平均的なリソース使用率メトリクスを得ることができます。 - -### OSThreadsRunnable {#osthreadsrunnable} - -OS カーネルのスケジューラから見て「実行可能」となっているスレッドの総数。 - -### OSThreadsTotal {#osthreadstotal} - -OS カーネルのスケジューラから見たスレッドの総数。 - -### OSUptime {#osuptime} - -ClickHouse が稼働しているホストサーバー(マシン)の稼働時間(秒単位)。 - -### OSUserTime {#osusertime} - -CPU コアがユーザー空間コードを実行していた時間の比率。このメトリクスはシステム全体のものであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。また、CPU 内部の要因(メモリロード、パイプラインストール、分岐予測ミス、別の SMT コアの実行など)により CPU が過小利用されていた時間も含みます。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらの合計として計算され [0..num cores] の範囲になります。 - -### OSUserTimeCPU_*N* {#osusertimecpu_n} - -CPU コアがユーザー空間コードを実行していた時間の比率。このメトリクスはシステム全体のものであり、clickhouse-server だけでなくホストマシン上のすべてのプロセスを含みます。また、CPU 内部の要因(メモリロード、パイプラインストール、分岐予測ミス、別の SMT コアの実行など)により CPU が過小利用されていた時間も含みます。単一 CPU コアに対する値は [0..1] の範囲になります。すべての CPU コアに対する値は、それらの合計として計算され [0..num cores] の範囲になります。 - -### OSUserTimeNormalized {#osusertimenormalized} - -`OSUserTime` と同様の値ですが、CPU コア数で割ることで、コア数に依存せず常に [0..1] の範囲になるようにしたものです。これにより、コア数が不均一であってもクラスタ内の複数サーバー間でこのメトリクスの値を平均化し、平均的なリソース利用率メトリクスを得ることができます。 - -### PostgreSQLThreads {#postgresqlthreads} - -PostgreSQL 互換プロトコルのサーバー内のスレッド数。 - -### ReplicasMaxAbsoluteDelay {#replicasmaxabsolutedelay} - -すべての Replicated テーブルにおいて、最新のレプリケート済みパーツと、まだレプリケートされていない最新のデータパーツとの秒単位での最大差分。非常に大きな値は、データを持たないレプリカが存在することを示します。 - -### ReplicasMaxInsertsInQueue {#replicasmaxinsertsinqueue} - -すべての Replicated テーブルにおいて、キュー内(まだレプリケートされていない)にある INSERT 操作数の最大値。 - -### ReplicasMaxMergesInQueue {#replicasmaxmergesinqueue} - -すべての Replicated テーブルにおいて、キュー内(まだ適用されていない)にあるマージ操作数の最大値。 - -### ReplicasMaxQueueSize {#replicasmaxqueuesize} - -すべての Replicated テーブルにおいて、get や merge などの操作数として表したキューサイズの最大値。 - -### ReplicasMaxRelativeDelay {#replicasmaxrelativedelay} - -すべての Replicated テーブルにおいて、あるレプリカの遅延と、そのテーブルでもっとも最新なレプリカの遅延との差分の最大値。 - -### ReplicasSumInsertsInQueue {#replicassuminsertsinqueue} - -すべての Replicated テーブルにおいて、キュー内(まだレプリケートされていない)にある INSERT 操作数の合計。 - -### ReplicasSumMergesInQueue {#replicassummergesinqueue} - -すべての Replicated テーブルにおいて、キュー内(まだ適用されていない)にあるマージ操作数の合計。 - -### ReplicasSumQueueSize {#replicassumqueuesize} - -すべての Replicated テーブルにおいて、get や merge などの操作数として表したキューサイズの合計。 - -### TCPThreads {#tcpthreads} - -TCP プロトコル(TLS なし)のサーバー内のスレッド数。 - -### Temperature_*N* {#temperature_n} - -対応するデバイスの温度(℃)。センサーは非現実的な値を返す場合があります。取得元: `/sys/class/thermal` - -### Temperature_*name* {#temperature_name} - -対応するハードウェアモニターおよび対応するセンサーが報告する温度(℃)。センサーは非現実的な値を返す場合があります。取得元: `/sys/class/hwmon` - -### TotalBytesOfMergeTreeTables {#totalbytesofmergetreetables} - -すべての MergeTree ファミリーのテーブルに保存されているバイト数(圧縮後、データおよびインデックスを含む)の合計。 - -### TotalPartsOfMergeTreeTables {#totalpartsofmergetreetables} - -すべての MergeTree ファミリーのテーブルに存在するデータパーツの総数。10 000 を超える値はサーバーの起動時間に悪影響を与え、パーティションキーの選択が不適切であることを示している可能性があります。 - -### TotalPrimaryKeyBytesInMemory {#totalprimarykeybytesinmemory} - -プライマリキー値に使用されているメモリ量(バイト単位)の合計(アクティブなパーツのみを対象)。 - -### TotalPrimaryKeyBytesInMemoryAllocated {#totalprimarykeybytesinmemoryallocated} - -プライマリキー値のために予約されているメモリ量(バイト単位)の合計(アクティブなパーツのみを対象)。 - -### TotalRowsOfMergeTreeTables {#totalrowsofmergetreetables} - -MergeTree ファミリーのすべてのテーブルに保存されている行(レコード)の総数。 - -### Uptime {#uptime} - -サーバーの稼働時間(秒単位)。接続を受け付けるまでのサーバー初期化に要した時間も含まれます。 - -### jemalloc.active {#jemallocactive} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.allocated {#jemallocallocated} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.arenas.all.dirty_purged {#jemallocarenasalldirty_purged} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.arenas.all.muzzy_purged {#jemallocarenasallmuzzy_purged} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.arenas.all.pactive {#jemallocarenasallpactive} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.arenas.all.pdirty {#jemallocarenasallpdirty} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.arenas.all.pmuzzy {#jemallocarenasallpmuzzy} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.background_thread.num_runs {#jemallocbackground_threadnum_runs} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.background_thread.num_threads {#jemallocbackground_threadnum_threads} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.background_thread.run_intervals {#jemallocbackground_threadrun_intervals} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.epoch {#jemallocepoch} - -jemalloc(Jason Evans のメモリアロケータ)の統計情報に対する内部の増分更新番号で、他のすべての `jemalloc` メトリクスで使用されます。 - -### jemalloc.mapped {#jemallocmapped} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.metadata {#jemallocmetadata} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.metadata_thp {#jemallocmetadata_thp} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.resident {#jemallocresident} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.retained {#jemallocretained} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -### jemalloc.prof.active {#jemallocprofactive} - -低レベルメモリアロケータ(jemalloc)の内部メトリクス。https://jemalloc.net/jemalloc.3.html を参照してください。 - -**関連項目** - -- [Monitoring](../../operations/monitoring.md) — ClickHouse のモニタリングに関する基本概念。 -- [system.metrics](/operations/system-tables/metrics) — 即時計算されるメトリクスを含みます。 -- [system.events](/operations/system-tables/events) — 発生したイベント数を含みます。 -- [system.metric_log](/operations/system-tables/metric_log) — テーブル `system.metrics` および `system.events` のメトリクス値の履歴を含みます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md deleted file mode 100644 index a9231ae147f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'AzureQueue テーブルの設定に関する情報を含むシステムテーブル。 - サーバーのバージョン `24.10` から利用可能。' -keywords: ['system table', 'azure_queue_settings'] -slug: /operations/system-tables/azure_queue_settings -title: 'system.azure_queue_settings' -doc_type: 'reference' ---- - -[AzureQueue](../../engines/table-engines/integrations/azure-queue.md) テーブルの設定に関する情報を含みます。 -サーバーのバージョン `24.10` から利用可能です。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md deleted file mode 100644 index e141ae82d00..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -description: '`BACKUP` および `RESTORE` 操作に関する情報を含むログエントリを格納するシステムテーブル。' -keywords: ['system table', 'backup_log'] -slug: /operations/system-tables/backup_log -title: 'system.backup_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.backup_log {#systembackup_log} - - - -`BACKUP` および `RESTORE` 操作に関する情報を含むログエントリを保持します。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行したサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — エントリの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — エントリの日付と時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度でのエントリの時刻。 -* `id` ([String](../../sql-reference/data-types/string.md)) — バックアップまたはリストア操作の識別子。 -* `name` ([String](../../sql-reference/data-types/string.md)) — バックアップストレージの名前(`FROM` 句または `TO` 句の内容)。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — 操作のステータス。取りうる値は次のとおりです: - * `'CREATING_BACKUP'` - * `'BACKUP_CREATED'` - * `'BACKUP_FAILED'` - * `'RESTORING'` - * `'RESTORED'` - * `'RESTORE_FAILED'` -* `error` ([String](../../sql-reference/data-types/string.md)) — 失敗した操作のエラーメッセージ(成功した操作では空文字列)。 -* `start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 操作の開始時刻。 -* `end_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 操作の終了時刻。 -* `num_files` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — バックアップに保存されているファイル数。 -* `total_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — バックアップに保存されているファイルの合計サイズ。 -* `num_entries` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — バックアップ内のエントリ数。つまり、バックアップがフォルダとして保存されている場合はそのフォルダ内のファイル数、バックアップがアーカイブとして保存されている場合はそのアーカイブ内のファイル数。増分バックアップである場合や空ファイルや重複を含む場合、`num_files` と一致しないことがあります。常に次の関係が成り立ちます: `num_entries <= num_files`。 -* `uncompressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — バックアップの非圧縮サイズ。 -* `compressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — バックアップの圧縮サイズ。バックアップがアーカイブとして保存されていない場合は `uncompressed_size` と等しくなります。 -* `files_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — リストア操作中に読み取られたファイル数。 -* `bytes_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — リストア操作中に読み取られたファイルの合計サイズ。 - -**Example** - -```sql -バックアップ TABLE test_db.my_table TO Disk('backups_disk', '1.zip') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'e5b74ecb-f6f1-426a-80be-872f90043885' ORDER BY event_date, event_time_microseconds \G -``` - -```response -1 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:05:21.998566 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-19 11:05:21 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -2 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time: 2023-08-19 11:08:56 -event_time_microseconds: 2023-08-19 11:08:56.916192 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: BACKUP_CREATED -error: -start_time: 2023-08-19 11:05:21 -end_time: 2023-08-19 11:08:56 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 3525068304 -files_read: 0 -bytes_read: 0 -``` - -```sql -ディスク('backups_disk', '1.zip')から test_db.my_table テーブルを復元 -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ RESTORED │ -└──────────────────────────────────────┴──────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'cdf1f731-52ef-42da-bc65-2e1bfcd4ce90' ORDER BY event_date, event_time_microseconds \G -``` - -```response -行 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:19.718077 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORING -error: -start_time: 2023-08-19 11:09:19 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -行 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:29.334234 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORED -error: -start_time: 2023-08-19 11:09:19 -end_time: 2023-08-19 11:09:29 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 4290362365 -files_read: 57 -bytes_read: 4290364870 -``` - -これは、基本的にはシステムテーブル `system.backups` に記載されている内容と同じです。 - -```sql -SELECT * FROM system.backups ORDER BY start_time -``` - -```response -┌─id───────────────────────────────────┬─name──────────────────────────┬─status─────────┬─error─┬──────────start_time─┬────────────end_time─┬─num_files─┬─total_size─┬─num_entries─┬─uncompressed_size─┬─compressed_size─┬─files_read─┬─bytes_read─┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ Disk('backups_disk', '1.zip') │ バックアップ作成 │ │ 2023-08-19 11:05:21 │ 2023-08-19 11:08:56 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 3525068304 │ 0 │ 0 │ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ Disk('backups_disk', '1.zip') │ 復元済み │ │ 2023-08-19 11:09:19 │ 2023-08-19 11:09:29 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 4290362365 │ 57 │ 4290364870 │ -└──────────────────────────────────────┴───────────────────────────────┴────────────────┴───────┴─────────────────────┴─────────────────────┴───────────┴────────────┴─────────────┴───────────────────┴─────────────────┴────────────┴────────────┘ -``` - -**関連項目** - -* [バックアップと復元](../../operations/backup.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md deleted file mode 100644 index cf4cf4bcdac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: '`BACKUP` および `RESTORE` 操作に関する情報を含むログエントリを格納するシステムテーブル。' -keywords: ['システムテーブル', 'バックアップ'] -slug: /operations/system-tables/backups -title: 'system.backups' -doc_type: 'reference' ---- - -# system.backups {#systembackups} - -すべての `BACKUP` または `RESTORE` 操作と、その現在の状態およびその他のプロパティの一覧を含みます。なお、このテーブルは永続化されず、最後にサーバーを再起動して以降に実行された操作のみが表示されます。 - -以下は、name 列および comment 列を含む Markdown テーブルです。 - -| Column | Description | -|---------------------|----------------------------------------------------------------------------------------------------------------------| -| `id` | 操作 ID。`SETTINGS id=...` で明示的に指定するか、ランダムに生成される UUID です。 | -| `name` | 操作名。`Disk('backups', 'my_backup')` のような文字列です。 | -| `base_backup_name` | ベースバックアップの操作名。`Disk('backups', 'my_base_backup')` のような文字列です。 | -| `query_id` | バックアップを開始したクエリのクエリ ID。 | -| `status` | バックアップまたはリストア操作のステータス。 | -| `error` | エラーが発生した場合のエラーメッセージ。 | -| `start_time` | 操作が開始された時刻。 | -| `end_time` | 操作が終了した時刻。 | -| `num_files` | バックアップに格納されているファイル数。 | -| `total_size` | バックアップに格納されているファイルの合計サイズ。 | -| `num_entries` | バックアップ内のエントリ数。つまり、バックアップがフォルダとして保存されている場合、そのフォルダ内のファイル数。 | -| `uncompressed_size` | バックアップの非圧縮サイズ。 | -| `compressed_size` | バックアップの圧縮サイズ。 | -| `files_read` | このバックアップから RESTORE を実行する際に読み取られたファイル数。 | -| `bytes_read` | このバックアップから RESTORE を実行する際に読み取られたファイルの合計サイズ。 | -| `ProfileEvents` | この操作中に取得されたすべてのプロファイルイベント。 | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md deleted file mode 100644 index 6cf6a63e2bf..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'アップロードや削除など、各種 BLOB ストレージ操作に関する情報を含むログエントリを格納するシステムテーブル。' -keywords: ['system table', 'blob_storage_log'] -slug: /operations/system-tables/blob_storage_log -title: 'system.blob_storage_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -アップロードや削除など、さまざまな BLOB ストレージ操作に関する情報を含むログエントリを保持します。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度でのイベントの時刻。 -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — イベントの種類。取りうる値は次のとおり: - * `'Upload'` - * `'Delete'` - * `'MultiPartUploadCreate'` - * `'MultiPartUploadWrite'` - * `'MultiPartUploadComplete'` - * `'MultiPartUploadAbort'` -* `query_id` ([String](../../sql-reference/data-types/string.md)) — そのイベントに関連付けられたクエリの識別子(存在する場合)。 -* `thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 操作を実行しているスレッドの識別子。 -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — 操作を実行しているスレッドの名前。 -* `disk_name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — 関連付けられたディスク名。 -* `bucket` ([String](../../sql-reference/data-types/string.md)) — バケット名。 -* `remote_path` ([String](../../sql-reference/data-types/string.md)) — リモートリソースへのパス。 -* `local_path` ([String](../../sql-reference/data-types/string.md)) — リモートリソースを参照する、ローカルシステム上のメタデータファイルへのパス。 -* `data_size` ([UInt32](/sql-reference/data-types/int-uint#integer-ranges)) — アップロードイベントに関与するデータのサイズ。 -* `error` ([String](../../sql-reference/data-types/string.md)) — そのイベントに関連付けられたエラーメッセージ(存在する場合)。 - -**例** - -BLOB ストレージ操作でファイルをアップロードし、そのイベントがログに記録されるとします: - -```sql -SELECT * FROM system.blob_storage_log WHERE query_id = '7afe0450-504d-4e4b-9a80-cd9826047972' ORDER BY event_date, event_time_microseconds \G -``` - -```text -1 行目: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-10-31 -event_time: 2023-10-31 16:03:40 -event_time_microseconds: 2023-10-31 16:03:40.481437 -event_type: Upload -query_id: 7afe0450-504d-4e4b-9a80-cd9826047972 -thread_id: 2381740 -disk_name: disk_s3 -bucket: bucket1 -remote_path: rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe -local_path: store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt -data_size: 259 -error: -``` - -この例では、アップロード処理は ID `7afe0450-504d-4e4b-9a80-cd9826047972` の `INSERT` クエリに関連付けられていました。ローカルメタデータファイル `store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt` は、ディスク `disk_s3` 上のバケット `bucket1` 内にあるリモートパス `rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe` を参照しており、サイズは 259 バイトです。 - -**関連項目** - -* [データを保存するための外部ディスク](../../operations/storing-data.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md deleted file mode 100644 index 1a16333f483..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'ClickHouse サーバーのビルドオプションに関する情報を保持する system テーブル。' -slug: /operations/system-tables/build_options -title: 'system.build_options' -keywords: ['system テーブル', 'build_options'] -doc_type: 'reference' ---- - -ClickHouse サーバーのビルドオプションに関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.build_options LIMIT 5 -``` - -```text -┌─name─────────────┬─value─┐ -│ USE_BROTLI │ 1 │ -│ USE_BZIP2 │ 1 │ -│ USE_CAPNP │ 1 │ -│ USE_CASSANDRA │ 1 │ -│ USE_DATASKETCHES │ 1 │ -└──────────────────┴───────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md deleted file mode 100644 index dee3ed325b4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '設定ファイルで利用可能なクラスターと、それらで定義されているサーバーに関する情報を含むシステムテーブルです。' -keywords: ['system table', 'clusters'] -slug: /operations/system-tables/clusters -title: 'system.clusters' -doc_type: 'reference' ---- - -設定ファイルで利用可能なクラスターと、そのクラスター内のサーバーに関する情報を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -クエリ: - -```sql -SELECT * FROM system.clusters LIMIT 2 FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -cluster: test_cluster_two_shards -shard_num: 1 -shard_name: shard_01 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.1 -host_address: 127.0.0.1 -port: 9000 -is_local: 1 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL - -Row 2: -────── -cluster: test_cluster_two_shards -shard_num: 2 -shard_name: shard_02 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.2 -host_address: 127.0.0.2 -port: 9000 -is_local: 0 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL -``` - -**関連項目** - -* [Distributed テーブルエンジン](../../engines/table-engines/special/distributed.md) -* [distributed_replica_error_cap 設定](../../operations/settings/settings.md#distributed_replica_error_cap) -* [distributed_replica_error_half_life 設定](../../operations/settings/settings.md#distributed_replica_error_half_life) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md deleted file mode 100644 index add873509a8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'キュー内のコーデックに関する情報を含むシステムテーブル。' -keywords: ['system table', 'コーデック', '圧縮'] -slug: /operations/system-tables/codecs -title: 'system.codecs' -doc_type: 'reference' ---- - -圧縮および暗号化コーデックに関する情報を保持しています。 - -このテーブルを使用して、利用可能な圧縮および暗号化コーデックに関する情報を取得できます。 - -`system.codecs` テーブルには次の列が含まれます(括弧内は列のデータ型を示します)。 - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -クエリ: - -```sql -SELECT * FROM system.codecs WHERE name='LZ4' -``` - -結果: - -```text -Row 1: -────── -name: LZ4 -method_byte: 130 -is_compression: 1 -is_generic_compression: 1 -is_encryption: 0 -is_timeseries_codec: 0 -is_experimental: 0 -description: 極めて高速、優れた圧縮率、速度と効率のバランスが良好。 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md deleted file mode 100644 index e0a855162ab..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'すべてのテーブル内のカラムに関する情報を含むシステムテーブル' -keywords: ['system table', 'columns'] -slug: /operations/system-tables/columns -title: 'system.columns' -doc_type: 'reference' ---- - -すべてのテーブル内のカラムに関する情報を格納しています。 - -このテーブルを使用すると、複数のテーブルに対してまとめて、[DESCRIBE TABLE](../../sql-reference/statements/describe-table.md) クエリと同様の情報を取得できます。 - -[一時テーブル](../../sql-reference/statements/create/table.md#temporary-tables) のカラムは、それらが作成されたセッションでのみ `system.columns` に表示されます。これらは空の `database` フィールドで表示されます。 - -`system.columns` テーブルには、次のカラムが含まれます(カラム型は括弧内に示します): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.columns LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_catalog -type: String -position: 1 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ - -Row 2: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_schema -type: String -position: 2 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md deleted file mode 100644 index 3b2f57b0e65..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '貢献者に関する情報を格納するシステムテーブル。' -keywords: ['system table', 'contributors'] -slug: /operations/system-tables/contributors -title: 'system.contributors' -doc_type: 'reference' ---- - -貢献者に関する情報を格納します。クエリ実行時の並び順はランダムです。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.contributors LIMIT 10 -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -│ Max Vetrov │ -│ LiuYangkuan │ -│ svladykin │ -│ zamulla │ -│ Šimon Podlipský │ -│ BayoNet │ -│ Ilya Khomutov │ -│ Amy Krishnevsky │ -│ Loud_Scream │ -└──────────────────┘ -``` - -テーブル内から自分に該当するレコードを探すには、次のクエリを実行します。 - -```sql -SELECT * FROM system.contributors WHERE name = 'Olga Khvostikova' -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -└──────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md deleted file mode 100644 index 94e07a0e649..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '致命的なエラーのスタックトレース情報を含むシステムテーブル。' -keywords: ['system table', 'crash_log'] -slug: /operations/system-tables/crash_log -title: 'system.crash_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -致命的なエラーのスタックトレース情報を保持します。テーブルはデフォルトではデータベース内に存在せず、致命的なエラーが発生したときにのみ作成されます。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ナノ秒精度のイベントタイムスタンプ。 -* `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — シグナル番号。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — スレッド ID。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — クエリ ID。 -* `trace` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 障害発生時点のスタックトレース。各要素は ClickHouse サーバープロセス内の仮想メモリアドレスです。 -* `trace_full` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 障害発生時点のスタックトレース。各要素には ClickHouse サーバープロセス内で呼び出されたメソッドが含まれます。 -* `version` ([String](../../sql-reference/data-types/string.md)) — ClickHouse サーバーのバージョン。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse サーバーのリビジョン。 -* `build_id` ([String](../../sql-reference/data-types/string.md)) — コンパイラによって生成される Build ID。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.crash_log ORDER BY event_time DESC LIMIT 1; -``` - -結果(抜粋): - -```text -行 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-10-14 -event_time: 2020-10-14 15:47:40 -timestamp_ns: 1602679660271312710 -signal: 11 -thread_id: 23624 -query_id: 428aab7c-8f5c-44e9-9607-d16b44467e69 -trace: [188531193,...] -trace_full: ['3. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::__1::vector >&, std::__1::vector > const&, unsigned long, unsigned long) const @ 0xb3cc1f9 in /home/username/work/ClickHouse/build/programs/clickhouse',...] -version: ClickHouse 20.11.1.1 -revision: 54442 -build_id: -``` - -**関連項目** - -* [trace_log](../../operations/system-tables/trace_log.md) システムテーブル diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md deleted file mode 100644 index 18d999f5385..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: '現在のユーザーに対してアクティブなロールを含むシステムテーブル。' -keywords: ['system table', 'current_roles'] -slug: /operations/system-tables/current_roles -title: 'system.current_roles' -doc_type: 'reference' ---- - -現在のユーザーに対してアクティブなロールを含みます。`SET ROLE` によってこのテーブルの内容が変更されます。 - -列: - -- `role_name` ([String](../../sql-reference/data-types/string.md)) — ロール名。 -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `current_role` が `ADMIN OPTION` 権限を持つロールかどうかを示すフラグ。 -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `current_role` がデフォルトロールかどうかを示すフラグ。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md deleted file mode 100644 index 9945993595a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '`/dashboard` ページで使用されるクエリを含むテーブルで、HTTP インターフェイス経由でアクセスできます。監視やトラブルシューティングに有用です。' -keywords: ['system table', 'dashboards', 'monitoring', 'troubleshooting'] -slug: /operations/system-tables/dashboards -title: 'system.dashboards' -doc_type: 'reference' ---- - -`/dashboard` ページで使用されるクエリを含むテーブルで、[HTTP インターフェイス](/interfaces/http.md) 経由でアクセスできます。 -このテーブルは監視およびトラブルシューティングに役立ちます。テーブルには、ダッシュボード内の各チャートごとに 1 行が含まれます。 - -:::note -`/dashboard` ページは `system.dashboards` だけでなく、同じスキーマを持つ任意のテーブルに含まれるクエリも表示できます。 -これはカスタムダッシュボードを作成する際に役立ちます。 -::: - -例: - -```sql -SELECT * -FROM system.dashboards -WHERE title ILIKE '%CPU%' -``` - -```text -Row 1: -────── -dashboard: overview -title: CPU使用率(コア) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUVirtualTimeMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 2: -────── -dashboard: overview -title: CPU待機 -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUWaitMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 3: -────── -dashboard: overview -title: OS CPU使用率(ユーザー空間) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSUserTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 4: -────── -dashboard: overview -title: OS CPU使用率(カーネル) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSSystemTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md deleted file mode 100644 index 825453e8058..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'すべてのテーブルに存在するデータスキップインデックスに関する情報を含むシステムテーブル。' -keywords: ['system table', 'data_skipping_indices'] -slug: /operations/system-tables/data_skipping_indices -title: 'system.data_skipping_indices' -doc_type: 'reference' ---- - -すべてのテーブルに存在するデータスキップインデックスに関する情報を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.data_skipping_indices LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: user_actions -name: clicks_idx -type: minmax -type_full: minmax -expr: clicks -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 - -Row 2: -────── -database: default -table: users -name: contacts_null_idx -type: minmax -type_full: minmax -expr: assumeNotNull(contacts_null) -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md deleted file mode 100644 index 524767414cd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'サポートされているデータ型に関する情報を保持するシステムテーブル' -keywords: ['system table', 'data_type_families'] -slug: /operations/system-tables/data_type_families -title: 'system.data_type_families' -doc_type: 'reference' ---- - -サポートされている[データ型](../../sql-reference/data-types/index.md)に関する情報を保持しています。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.data_type_families WHERE alias_to = 'String' -``` - -```text -┌─name───────┬─case_insensitive─┬─alias_to─┐ -│ LONGBLOB │ 1 │ String │ -│ LONGTEXT │ 1 │ String │ -│ TINYTEXT │ 1 │ String │ -│ TEXT │ 1 │ String │ -│ VARCHAR │ 1 │ String │ -│ MEDIUMBLOB │ 1 │ String │ -│ BLOB │ 1 │ String │ -│ TINYBLOB │ 1 │ String │ -│ CHAR │ 1 │ String │ -│ MEDIUMTEXT │ 1 │ String │ -└────────────┴──────────────────┴──────────┘ -``` - -**関連項目** - -* [Syntax](../../sql-reference/syntax.md) — サポートされている構文に関する情報。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md deleted file mode 100644 index 438cfedc823..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'サーバーでサポートされているデータベースエンジンの一覧を保持するシステムテーブル。' -keywords: ['system table', 'database_engines'] -slug: /operations/system-tables/database_engines -title: 'system.database_engines' -doc_type: 'reference' ---- - -サーバーでサポートされているデータベースエンジンの一覧を保持します。 - -このテーブルには次の列が含まれます(列の型は括弧内に示します)。 - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -例: - -```sql -SELECT * -FROM system.database_engines -WHERE name IN ('Atomic', 'Lazy', 'Ordinary') -``` - -```text -┌─name─────┐ -│ Ordinary │ -│ Atomic │ -│ Lazy │ -└──────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md deleted file mode 100644 index 710f954e66b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: 'レプリケーテッドデータベースのレプリカに関する情報およびステータスを格納するシステムテーブル。' -keywords: ['system table', 'database_replicas'] -slug: /operations/system-tables/database_replicas -title: 'system.database_replicas' -doc_type: 'reference' ---- - -各 Replicated データベースのレプリカに関する情報およびステータスを保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.database_replicas FORMAT Vertical; -``` - -```text -Row 1: -────── -database: db_2 -is_readonly: 0 -max_log_ptr: 2 -replica_name: replica1 -replica_path: /test/db_2/replicas/shard1|replica1 -zookeeper_path: /test/db_2 -shard_name: shard1 -log_ptr: 2 -total_replicas: 1 -zookeeper_exception: -is_session_expired: 0 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md deleted file mode 100644 index ada1e584d1d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '現在のユーザーが利用可能なデータベースに関する情報を格納するシステムテーブル。' -keywords: ['system table', 'databases'] -slug: /operations/system-tables/databases -title: 'system.databases' -doc_type: 'reference' ---- - -現在のユーザーが利用可能なデータベースに関する情報を格納します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -このシステムテーブルの `name` 列は、`SHOW DATABASES` クエリの実装に使用されます。 - -**例** - -データベースを作成する。 - -```sql -CREATE DATABASE test; -``` - -ユーザーが利用可能なすべてのデータベースを確認する。 - -```sql -SELECT * FROM system.databases; -``` - -```text -┌─name────────────────┬─engine─────┬─data_path────────────────────┬─metadata_path─────────────────────────────────────────────────────────┬─uuid─────────────────────────────────┬─engine_full────────────────────────────────────────────┬─comment─┐ -│ INFORMATION_SCHEMA │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ default │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/f97/f97a3ceb-2e8a-4912-a043-c536e826a4d4/ │ f97a3ceb-2e8a-4912-a043-c536e826a4d4 │ Atomic │ │ -│ information_schema │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ replicated_database │ Replicated │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/da8/da85bb71-102b-4f69-9aad-f8d6c403905e/ │ da85bb71-102b-4f69-9aad-f8d6c403905e │ Replicated('some/path/database', 'shard1', 'replica1') │ │ -│ system │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/b57/b5770419-ac7a-4b67-8229-524122024076/ │ b5770419-ac7a-4b67-8229-524122024076 │ Atomic │ │ -└─────────────────────┴────────────┴──────────────────────────────┴───────────────────────────────────────────────────────────────────────┴──────────────────────────────────────┴────────────────────────────────────────────────────────┴─────────┘ - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md deleted file mode 100644 index b889c2d5be0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -description: 'ストリーミングエンジン経由で受信され、パース時にエラーが発生したメッセージに関する情報を保持するシステムテーブル。' -keywords: ['system table', 'dead_letter_queue'] -slug: /operations/system-tables/dead_letter_queue -title: 'system.dead_letter_queue' -doc_type: 'reference' ---- - -ストリーミングエンジン経由で受信され、パース時にエラーが発生したメッセージに関する情報を保持します。現在は Kafka と RabbitMQ 向けに実装されています。 - -エンジン固有の `handle_error_mode` 設定に `dead_letter_queue` を指定することで、ロギングが有効になります。 - -データのフラッシュ間隔は、サーバー設定セクション [dead_letter_queue](../../operations/server-configuration-parameters/settings.md#dead_letter_queue) の `flush_interval_milliseconds` パラメータで設定します。フラッシュを強制するには、[SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) クエリを使用します。 - -ClickHouse はテーブルからデータを自動的には削除しません。詳細は [Introduction](../../operations/system-tables/overview.md#system-tables-introduction) を参照してください。 - -列: - -* `table_engine` ([Enum8](../../sql-reference/data-types/enum.md)) - ストリームの種類。取りうる値: `Kafka` と `RabbitMQ`。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) - メッセージを消費した日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - メッセージを消費した日時。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) - マイクロ秒精度のメッセージ消費時刻。 -* `database` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) - ストリーミングテーブルが属する ClickHouse データベース。 -* `table` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) - ClickHouse テーブル名。 -* `error` ([String](../../sql-reference/data-types/string.md)) - エラーの内容。 -* `raw_message` ([String](../../sql-reference/data-types/string.md)) - メッセージ本文。 -* `kafka_topic_name` ([String](../../sql-reference/data-types/string.md)) - Kafka トピック名。 -* `kafka_partition` ([UInt64](../../sql-reference/data-types/int-uint.md)) - トピックの Kafka パーティション。 -* `kafka_offset` ([UInt64](../../sql-reference/data-types/int-uint.md)) - メッセージの Kafka オフセット。 -* `kafka_key` ([String](../../sql-reference/data-types/string.md)) - メッセージの Kafka キー。 -* `rabbitmq_exchange_name` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ Exchange 名。 -* `rabbitmq_message_id` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ メッセージ ID。 -* `rabbitmq_message_timestamp` ([DateTime](../../sql-reference/data-types/datetime.md)) - RabbitMQ メッセージのタイムスタンプ。 -* `rabbitmq_message_redelivered` ([UInt8](../../sql-reference/data-types/int-uint.md)) - RabbitMQ の再配信フラグ。 -* `rabbitmq_message_delivery_tag` ([UInt64](../../sql-reference/data-types/int-uint.md)) - RabbitMQ の delivery tag。 -* `rabbitmq_channel_id` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ チャネル ID。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.dead_letter_queue LIMIT 1 \G; -``` - -結果: - - -```text -Row 1: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910773 -database: default -table: kafka -error: 入力を解析できません: 次の文字の前に '\t' が必要です: 'qwertyuiop': (at row 1) -: -Row 1: -Column 0, name: key, type: UInt64, ERROR: テキスト "qwertyuiop" はUInt64形式ではありません -raw_message: qwertyuiop -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -Row 2: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910944 -database: default -table: kafka -error: 入力を解析できません: 次の文字の前に '\t' が必要です: 'asdfghjkl': (at row 1) -: -Row 1: -Column 0, name: key, type: UInt64, ERROR: テキスト "asdfghjkl" はUInt64形式ではありません -raw_message: asdfghjkl -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -Row 3: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.911092 -database: default -table: kafka -error: 入力を解析できません: 次の文字の前に '\t' が必要です: 'zxcvbnm': (at row 1) -: -Row 1: -Column 0, name: key, type: UInt64, ERROR: テキスト "zxcvbnm" はUInt64形式ではありません -raw_message: zxcvbnm -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - (test.py:78, dead_letter_queue_test) - -``` - -**関連項目** - -* [Kafka](/engines/table-engines/integrations/kafka.md) - Kafka エンジン -* [system.kafka_consumers](/operations/system-tables/kafka_consumers.md) — Kafka コンシューマに関する統計情報やエラーなどの情報を含む `kafka_consumers` システムテーブルの説明です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md deleted file mode 100644 index 09420938fb4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: 'Delta Lake テーブルから読み取られたメタデータファイルに関する情報を含むシステムテーブル。各エントリは 1 つのルートメタデータ JSON ファイルに対応します。' -keywords: ['システムテーブル', 'delta_lake_metadata_log'] -slug: /operations/system-tables/delta_lake_metadata_log -title: 'system.delta_lake_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.delta_lake_metadata_log {#systemdelta_lake_metadata_log} - -`system.delta_lake_metadata_log` テーブルは、ClickHouse によって読み取られた Delta Lake テーブルのメタデータへのアクセスおよび解析イベントを記録します。各メタデータファイルに関する詳細な情報を提供し、デバッグや監査、Delta テーブル構造の変化を理解する際に有用です。 - - - -## 目的 {#purpose} - -このテーブルは、Delta Lake テーブルから読み取られたすべてのメタデータファイルを記録します。これにより、ClickHouse が Delta テーブルのメタデータをどのように解釈しているかを追跡でき、スキーマの進化、スナップショットの解決、クエリプランニングに関連する問題の診断に役立ちます。 - -:::note -このテーブルは主にデバッグ用途を想定しています。 -::: - - - -## 列 {#columns} -| Name | Type | Description | -|----------------|-----------|----------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | ログファイルの日付。 | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | イベントのタイムスタンプ。 | -| `query_id` | [String](../../sql-reference/data-types/string.md) | メタデータ読み取りを開始したクエリ ID。 | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Delta Lake テーブルへのパス。 | -| `file_path` | [String](../../sql-reference/data-types/string.md) | ルートメタデータ JSON ファイルへのパス。 | -| `content` | [String](../../sql-reference/data-types/string.md) | JSON 形式の内容(.json ファイル由来の生のメタデータ)。 | - - - - - -## ログの詳細度の制御 {#controlling-log-verbosity} - -[`delta_lake_log_metadata`](../../operations/settings/settings.md#delta_lake_log_metadata) 設定を使用して、どのメタデータイベントをログに記録するかを制御できます。 - -現在のクエリで使用されるすべてのメタデータをログに記録するには: - -```sql -SELECT * FROM my_delta_table SETTINGS delta_lake_log_metadata = 1; - -SYSTEM FLUSH LOGS delta_lake_metadata_log; - -SELECT * -FROM system.delta_lake_metadata_log -WHERE query_id = '{previous_query_id}'; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md deleted file mode 100644 index d72eed94a69..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: 'MergeTree テーブルのデタッチされたパーツに関する情報を含むシステムテーブル' -keywords: ['system table', 'detached_parts'] -slug: /operations/system-tables/detached_parts -title: 'system.detached_parts' -doc_type: 'reference' ---- - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルのデタッチされたパーツに関する情報を含みます。`reason` 列は、そのパーツがデタッチされた理由を示します。 - -ユーザーによってデタッチされたパーツの場合、`reason` 列は空です。このようなパーツは [ALTER TABLE ATTACH PARTITION\|PART](/sql-reference/statements/alter/partition#attach-partitionpart) コマンドで再度アタッチできます。 - -他の列の説明については、[system.parts](../../operations/system-tables/parts.md) を参照してください。 - -パーツ名が不正な場合、いくつかの列の値は `NULL` になることがあります。このようなパーツは [ALTER TABLE DROP DETACHED PART](/sql-reference/statements/alter/partition#drop-detached-partitionpart) で削除できます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md deleted file mode 100644 index 1da6b580b3d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '切り離された各テーブルに関する情報を含むシステムテーブル。' -keywords: ['system table', 'detached_tables'] -slug: /operations/system-tables/detached_tables -title: 'system.detached_tables' -doc_type: 'reference' ---- - -切り離された各テーブルに関する情報を含むシステムテーブルです。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.detached_tables FORMAT Vertical; -``` - -```text -Row 1: -────── -database: base -table: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -is_permanently: 1 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md deleted file mode 100644 index ca5f28d2223..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '辞書に関する情報を保持するシステムテーブル' -keywords: ['システムテーブル', '辞書'] -slug: /operations/system-tables/dictionaries -title: 'system.dictionaries' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -[辞書](../../sql-reference/dictionaries/index.md)に関する情報を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -辞書を設定します: - -```sql -CREATE DICTIONARY dictionary_with_comment -( - id UInt64, - value String -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table')) -LAYOUT(FLAT()) -LIFETIME(MIN 0 MAX 1000) -COMMENT '一時ディクショナリ'; -``` - -辞書が読み込まれていることを確認してください。 - -```sql -SELECT * FROM system.dictionaries LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -name: dictionary_with_comment -uuid: 4654d460-0d03-433a-8654-d4600d03d33a -status: NOT_LOADED -origin: 4654d460-0d03-433a-8654-d4600d03d33a -type: -key.names: ['id'] -key.types: ['UInt64'] -attribute.names: ['value'] -attribute.types: ['String'] -bytes_allocated: 0 -query_count: 0 -hit_rate: 0 -found_rate: 0 -element_count: 0 -load_factor: 0 -source: -lifetime_min: 0 -lifetime_max: 0 -loading_start_time: 1970-01-01 00:00:00 -last_successful_update_time: 1970-01-01 00:00:00 -loading_duration: 0 -last_exception: -comment: 一時ディクショナリ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md deleted file mode 100644 index b795f82fbb6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'このテーブルには、その場で計算でき、Prometheus 形式でエクスポート可能なディメンションメトリクスが含まれています。常に最新の状態に保たれています。' -keywords: ['system table', 'dimensional_metrics'] -slug: /operations/system-tables/dimensional_metrics -title: 'system.dimensional_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# dimensional_metrics {#dimensional_metrics} - - - -このテーブルには、その場で計算でき、Prometheus 形式でエクスポート可能な次元メトリクスが含まれます。常に最新の状態に保たれます。 - -列: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — メトリクス名。 -* `value` ([Int64](../../sql-reference/data-types/int-uint.md)) — メトリクス値。 -* `description` ([String](../../sql-reference/data-types/string.md)) — メトリクスの説明。 -* `labels` ([Map(String, String)](../../sql-reference/data-types/map.md)) — メトリクスラベル。 -* `name` ([String](../../sql-reference/data-types/string.md)) — `metric` のエイリアス。 - -**Example** - -Prometheus 形式ですべての次元メトリクスをエクスポートするには、次のようなクエリを使用できます。 - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'gauge' AS type -FROM system.dimensional_metrics -FORMAT Prometheus -``` - - -## メトリクスの説明 {#metric_descriptions} - -### merge_failures {#merge_failures} -起動以降に失敗したマージの総数。 - -### startup_scripts_failure_reason {#startup_scripts_failure_reason} -エラー種別ごとに、起動スクリプトの失敗を示します。起動スクリプトが失敗した場合に 1 に設定され、エラー名でラベル付けされます。 - -### merge_tree_parts {#merge_tree_parts} -MergeTree のデータパーツ数。パーツの状態、パーツ種別、およびプロジェクションパーツかどうかでラベル付けされます。 - -**参照** -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 定期的に算出されるメトリクスを格納します。 -- [system.events](/operations/system-tables/events) — 発生したイベントの数を格納します。 -- [system.metric_log](/operations/system-tables/metric_log) — テーブル `system.metrics` および `system.events` に由来するメトリクス値の履歴を格納します。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse のモニタリングに関する基本的な概念。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md deleted file mode 100644 index e11816aca4e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: 'サーバーの設定で定義されたディスクに関する情報を保持するシステムテーブル' -keywords: ['システムテーブル', 'ディスク'] -slug: /operations/system-tables/disks -title: 'system.disks' -doc_type: 'リファレンス' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -[サーバー構成](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure)で定義されたディスクに関する情報が格納されています。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.disks; -``` - -```response -┌─name────┬─path─────────────────┬───free_space─┬──total_space─┬─keep_free_space─┐ -│ default │ /var/lib/clickhouse/ │ 276392587264 │ 490652508160 │ 0 │ -└─────────┴──────────────────────┴──────────────┴──────────────┴─────────────────┘ - -1行が返されました。経過時間: 0.001秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md deleted file mode 100644 index 511f3faa846..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'クラスタ上で実行された分散 DDL クエリ(ON CLUSTER 句を使用するクエリ)に関する情報を含むシステムテーブル。' -keywords: ['system table', 'distributed_ddl_queue'] -slug: /operations/system-tables/distributed_ddl_queue -title: 'system.distributed_ddl_queue' -doc_type: 'reference' ---- - -クラスタ上で実行された[分散 DDL クエリ(ON CLUSTER 句)](../../sql-reference/distributed-ddl.md)に関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * -FROM system.distributed_ddl_queue -WHERE cluster = 'test_cluster' -LIMIT 2 -FORMAT Vertical - -クエリ ID: f544e72a-6641-43f1-836b-24baa1c9632a - -行 1: -────── -entry: query-0000000000 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: Finished -exception_code: 0 -exception_text: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -行 2: -────── -entry: query-0000000001 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: Finished -exception_code: 630 -exception_text: Code: 630. DB::Exception: Cannot drop or rename test_db, because some tables depend on it: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -2 行を取得しました。経過時間: 0.025 秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md deleted file mode 100644 index 60ab842ddca..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'シャードに送信するためのキューに入っているローカルファイルに関する情報を格納するシステムテーブルです。' -keywords: ['システムテーブル', 'distribution_queue'] -slug: /operations/system-tables/distribution_queue -title: 'system.distribution_queue' -doc_type: 'reference' ---- - -シャードに送信するためのキューに入っているローカルファイルに関する情報を格納します。これらのローカルファイルには、非同期モードで Distributed テーブルに新しいデータを挿入した際に作成される新しいパーツが含まれます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.distribution_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: dist -data_path: ./store/268/268bc070-3aad-4b1a-9cf2-4987580161af/default@127%2E0%2E0%2E2:9000/ -is_blocked: 1 -error_count: 0 -data_files: 1 -data_compressed_bytes: 499 -last_exception: -``` - -**関連項目** - -* [分散テーブルエンジン](../../engines/table-engines/special/distributed.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md deleted file mode 100644 index 827a8af03d3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'DNS レコードのキャッシュ情報を保持するシステムテーブル。' -keywords: ['システムテーブル', 'dns_cache'] -slug: /operations/system-tables/dns_cache -title: 'system.dns_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -キャッシュされた DNS レコードに関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -クエリ: - -```sql -SELECT * FROM system.dns_cache; -``` - -結果: - -| hostname | ip_address | ip_family | cached_at | -| :-------- | :------------- | :------------ | :------------------ | -| localhost | ::1 | IPv6 | 2024-02-11 17:04:40 | -| localhost | 127.0.0.1 | IPv4 | 2024-02-11 17:04:40 | - -**関連項目** - -* [disable_internal_dns_cache 設定](../../operations/server-configuration-parameters/settings.md#disable_internal_dns_cache) -* [dns_cache_max_entries 設定](../../operations/server-configuration-parameters/settings.md#dns_cache_max_entries) -* [dns_cache_update_period 設定](../../operations/server-configuration-parameters/settings.md#dns_cache_update_period) -* [dns_max_consecutive_failures 設定](../../operations/server-configuration-parameters/settings.md#dns_max_consecutive_failures) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md deleted file mode 100644 index 594f88f7645..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'DROP TABLE が実行されたが、まだデータのクリーンアップが行われていないテーブルに関する情報を保持するシステムテーブル' -keywords: ['system table', 'dropped_tables'] -slug: /operations/system-tables/dropped_tables -title: 'system.dropped_tables' -doc_type: 'reference' ---- - -DROP TABLE が実行されたが、まだデータのクリーンアップが行われていないテーブルに関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -以下の例では、`dropped_tables` に関する情報を取得する方法を示します。 - -```sql -SELECT * -FROM system.dropped_tables\G -``` - -```text -Row 1: -────── -index: 0 -database: default -table: test -uuid: 03141bb2-e97a-4d7c-a172-95cc066bb3bd -engine: MergeTree -metadata_dropped_path: /data/ClickHouse/build/programs/data/metadata_dropped/default.test.03141bb2-e97a-4d7c-a172-95cc066bb3bd.sql -table_dropped_time: 2023-03-16 23:43:31 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md deleted file mode 100644 index 74f352d5bbd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '`system.dropped_tables` に含まれる削除済み MergeTree テーブルのパーツ情報を保持するシステムテーブル' -keywords: ['system table', 'dropped_tables_parts'] -slug: /operations/system-tables/dropped_tables_parts -title: 'system.dropped_tables_parts' -doc_type: 'reference' ---- - -[system.dropped_tables](./dropped_tables.md) に含まれる削除済みの [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルのパーツに関する情報を保持します。 - -このテーブルのスキーマは [system.parts](./parts.md) と同じです。 - -**関連項目** - -- [MergeTree ファミリー](../../engines/table-engines/mergetree-family/mergetree.md) -- [system.parts](./parts.md) -- [system.dropped_tables](./dropped_tables.md) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md deleted file mode 100644 index c5c440b9853..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: '現在有効なすべてのロールを含み、現在のユーザーの現在のロールおよびそのロールに付与されているロールを含むシステムテーブル' -keywords: ['system table', 'enabled_roles'] -slug: /operations/system-tables/enabled_roles -title: 'system.enabled_roles' -doc_type: 'reference' ---- - -現在有効なすべてのロールを含むシステムテーブルであり、現在のユーザーの現在のロールおよびそのロールに付与されているロールを含みます。 - -列: - -- `role_name` ([String](../../sql-reference/data-types/string.md)) — ロール名。 -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `enabled_role` が `ADMIN OPTION` 権限を持つロールかどうかを示すフラグ。 -- `is_current` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `enabled_role` が現在のユーザーの現在のロールかどうかを示すフラグ。 -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `enabled_role` がデフォルトのロールかどうかを示すフラグ。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md deleted file mode 100644 index 0cc45acf231..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '`system.errors` テーブルのエラー値の履歴を保持するシステムテーブルで、内容は定期的にディスクにフラッシュされます。' -keywords: ['システムテーブル', 'error_log'] -slug: /operations/system-tables/system-error-log -title: 'system.error_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -テーブル `system.errors` に含まれるエラー値の履歴を保持しており、定期的にディスクにフラッシュされます。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `code` ([Int32](../../sql-reference/data-types/int-uint.md)) — エラーコード番号。 -* `error` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — エラー名。 -* `value` ([UInt64](../../sql-reference/data-types/int-uint.md)) — このエラーが発生した回数。 -* `remote` ([UInt8](../../sql-reference/data-types/int-uint.md)) — リモート例外(分散クエリのいずれかの実行中に受信したもの)。 - -**例** - -```sql -SELECT * FROM system.error_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2024-06-18 -event_time: 2024-06-18 07:32:39 -code: 999 -error: KEEPER_EXCEPTION -value: 2 -remote: 0 -``` - -**関連項目** - -* [error_log 設定](../../operations/server-configuration-parameters/settings.md#error_log) — 設定の有効化および無効化について。 -* [system.errors](../../operations/system-tables/errors.md) — 発生回数とともにエラーコードを保持します。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse の監視に関する基本概念。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md deleted file mode 100644 index 52288d76f8e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'エラーコードとその発生回数を含むシステムテーブル。' -keywords: ['system table', 'エラー'] -slug: /operations/system-tables/errors -title: 'system.errors' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -エラーコードと、それぞれが発生した回数を含みます。 - -まだ発生していないものも含めて、想定されるすべてのエラーコードを表示するには、設定 [system_events_show_zero_values](../settings/settings.md#system_events_show_zero_values) を 1 に設定します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -:::note -一部のエラーについては、クエリが正常に実行された場合でもカウンターが増加することがあります。対応するエラーが誤検知となる可能性がないと確信できない限り、このテーブルをサーバーの監視目的で使用することは推奨されません。 -::: - -**例** - -```sql -SELECT name, code, value -FROM system.errors -WHERE value > 0 -ORDER BY code ASC -LIMIT 1 - -┌─name─────────────┬─code─┬─value─┐ -│ CANNOT_OPEN_FILE │ 76 │ 1 │ -└──────────────────┴──────┴───────┘ -``` - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), last_error_trace) AS all -SELECT name, arrayStringConcat(all, '\n') AS res -FROM system.errors -LIMIT 1 -SETTINGS allow_introspection_functions=1\G -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/events.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/events.md deleted file mode 100644 index 0a4127c2cb3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/events.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'システム内で発生したイベントの回数に関する情報を保持するシステムテーブル。' -keywords: ['システムテーブル', 'イベント'] -slug: /operations/system-tables/events -title: 'system.events' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -システム内で発生したイベント数に関する情報を含みます。たとえば、このテーブルでは、ClickHouse サーバーの起動以降に処理された `SELECT` クエリの件数を確認できます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -サポートされているすべてのイベントは、ソースファイル [src/Common/ProfileEvents.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/ProfileEvents.cpp) で確認できます。 - -**例** - -```sql -SELECT * FROM system.events LIMIT 5 -``` - -```text -┌─event─────────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ Query │ 12 │ 解釈および実行される可能性のあるクエリの数。パースに失敗したクエリ、ASTサイズ制限、クォータ制限、または同時実行クエリ数の制限により拒否されたクエリは含まれません。ClickHouse自体が開始した内部クエリを含む場合があります。サブクエリはカウントされません。 │ -│ SelectQuery │ 8 │ Queryと同様ですが、SELECTクエリのみが対象です。 │ -│ FileOpen │ 73 │ 開かれたファイルの数。 │ -│ ReadBufferFromFileDescriptorRead │ 155 │ ファイルディスクリプタからの読み取り回数(read/pread)。ソケットは含まれません。 │ -│ ReadBufferFromFileDescriptorReadBytes │ 9931 │ ファイルディスクリプタから読み取られたバイト数。ファイルが圧縮されている場合は、圧縮後のデータサイズが表示されます。 │ -└───────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 定期的に計算されるメトリクスを保持します。 -* [system.metrics](/operations/system-tables/metrics) — 即時に計算されるメトリクスを保持します。 -* [system.metric_log](/operations/system-tables/metric_log) — テーブル `system.metrics` および `system.events` のメトリクス値の履歴を保持します。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse のモニタリングに関する基本概念です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md deleted file mode 100644 index 4c5ad734266..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: '通常関数および集約関数に関する情報を含むシステムテーブルです。' -keywords: ['system table', 'functions'] -slug: /operations/system-tables/functions -title: 'system.functions' -doc_type: 'reference' ---- - -通常関数および集約関数に関する情報が含まれます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql - SELECT name, is_aggregate, is_deterministic, case_insensitive, alias_to FROM system.functions LIMIT 5; -``` - -```text -┌─name─────────────────────┬─is_aggregate─┬─is_deterministic─┬─case_insensitive─┬─alias_to─┐ -│ BLAKE3 │ 0 │ 1 │ 0 │ │ -│ sipHash128Reference │ 0 │ 1 │ 0 │ │ -│ mapExtractKeyLike │ 0 │ 1 │ 0 │ │ -│ sipHash128ReferenceKeyed │ 0 │ 1 │ 0 │ │ -│ mapPartialSort │ 0 │ 1 │ 0 │ │ -└──────────────────────────┴──────────────┴──────────────────┴──────────────────┴──────────┘ - -5行が設定されました。経過時間: 0.002秒。 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md deleted file mode 100644 index 2c7bae87df5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'ClickHouse ユーザーアカウントに付与されている権限を表示する system テーブル。' -keywords: ['system table', 'grants'] -slug: /operations/system-tables/grants -title: 'system.grants' -doc_type: 'reference' ---- - -ClickHouse ユーザーアカウントに付与されている権限。 - -Columns: - -- `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — ユーザー名。 - -- `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — ユーザーアカウントに割り当てられているロール。 - -- `access_type` ([Enum8](../../sql-reference/data-types/enum.md)) — ClickHouse ユーザーアカウントに対するアクセス種別。 - -- `database` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — データベース名。 - -- `table` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — テーブル名。 - -- `column` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — アクセスが許可されているカラム名。 - -- `is_partial_revoke` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 論理値。一部の権限が取り消されているかどうかを示します。取り得る値は次のとおりです: -- `0` — 行は付与(grant)を表します。 -- `1` — 行は部分的な取り消し(partial revoke)を表します。 - -- `grant_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 権限が `WITH GRANT OPTION` 付きで付与されていることを示します。詳細は [GRANT](../../sql-reference/statements/grant.md#granting-privilege-syntax) を参照してください。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md deleted file mode 100644 index 2ebd3560d8c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: '`GraphiteMergeTree` 型エンジンを使用するテーブルで利用されるパラメータ `graphite_rollup` に関する情報を含むシステムテーブルです。' -keywords: ['system table', 'graphite_retentions'] -slug: /operations/system-tables/graphite_retentions -title: 'system.graphite_retentions' -doc_type: 'reference' ---- - -[`GraphiteMergeTree`](../../engines/table-engines/mergetree-family/graphitemergetree.md) エンジンを使用するテーブルで利用されるパラメータ [graphite_rollup](../../operations/server-configuration-parameters/settings.md#graphite) に関する情報が含まれています。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md deleted file mode 100644 index a280d1d4c72..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: 'このテーブルには、即時に計算でき、Prometheus 形式でエクスポート可能なヒストグラムメトリクスが格納されており、常に最新の状態が保たれます。' -keywords: ['system table', 'histogram_metrics'] -slug: /operations/system-tables/histogram_metrics -title: 'system.histogram_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# histogram_metrics {#histogram_metrics} - - - -このテーブルには、即座に計算でき、Prometheus 形式でエクスポートできるヒストグラムメトリクスが含まれています。常に最新の状態です。非推奨となった `system.latency_log` を置き換えます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -すべてのヒストグラムメトリクスを Prometheus 形式でエクスポートするには、次のようなクエリを使用できます。 - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'histogram' AS type -FROM system.histogram_metrics -FORMAT Prometheus -``` - - -## メトリクスの説明 {#metric_descriptions} - -### keeper_response_time_ms_bucket {#keeper_response_time_ms_bucket} - -Keeper の応答時間(ミリ秒単位)。 - -**関連項目** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 定期的に計算されるメトリクスを含みます。 -- [system.events](/operations/system-tables/events) — 発生した各種イベントを含みます。 -- [system.metric_log](/operations/system-tables/metric_log) — テーブル `system.metrics` および `system.events` からのメトリクス値の履歴を含みます。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse モニタリングの基本的な概念。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md deleted file mode 100644 index 6080b7d18f0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'system iceberg スナップショットの履歴' -keywords: ['system iceberg_history'] -slug: /operations/system-tables/iceberg_history -title: 'system.iceberg_history' -doc_type: 'reference' ---- - -# system.iceberg_history {#systemiceberg_history} - -このシステムテーブルには、ClickHouse 内に存在する Iceberg テーブルのスナップショット履歴が格納されています。ClickHouse に Iceberg テーブルが存在しない場合、このテーブルは空になります。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md deleted file mode 100644 index 94671c62a16..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -description: 'Iceberg テーブルから読み取られたメタデータファイルに関する情報を保持するシステムテーブル。各エントリは、ルートメタデータファイル、Avro ファイルから抽出されたメタデータ、または Avro ファイル内のエントリのいずれかを表します。' -keywords: ['システムテーブル', 'iceberg_metadata_log'] -slug: /operations/system-tables/iceberg_metadata_log -title: 'system.iceberg_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.iceberg_metadata_log {#systemiceberg_metadata_log} - -`system.iceberg_metadata_log` テーブルは、ClickHouse が読み取り対象とする Iceberg テーブルに対するメタデータのアクセスおよび解析イベントを記録します。処理された各メタデータファイルやエントリに関する詳細情報を提供し、デバッグや監査、Iceberg テーブル構造の変遷を把握する際に有用です。 - - - -## 目的 {#purpose} - -このテーブルは、Iceberg テーブルから読み取ったすべてのメタデータファイルおよびエントリ(ルートメタデータファイル、マニフェストリスト、マニフェストエントリを含む)を記録します。これにより、ClickHouse が Iceberg テーブルのメタデータをどのように解釈しているかを追跡し、スキーマの進化、ファイルの解決、クエリプランニングに関連する問題の診断に役立ちます。 - -:::note -このテーブルは主にデバッグ目的で使用されます。 -::: - - - -## 列 {#columns} - -| Name | Type | Description | -|----------------|-----------|----------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | ログエントリの日付。 | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | イベントのタイムスタンプ。 | -| `query_id` | [String](../../sql-reference/data-types/string.md) | メタデータの読み取りをトリガーしたクエリ ID。 | -| `content_type` | [Enum8](../../sql-reference/data-types/enum.md) | メタデータコンテンツの種類(下記参照)。 | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Iceberg テーブルへのパス。 | -| `file_path` | [String](../../sql-reference/data-types/string.md) | ルートメタデータ JSON ファイル、Avro マニフェストリスト、またはマニフェストファイルへのパス。 | -| `content` | [String](../../sql-reference/data-types/string.md) | JSON 形式のコンテンツ(.json からの生メタデータ、Avro メタデータ、または Avro エントリ)。 | -| `row_in_file` | [Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md)) | ファイル内の行番号(該当する場合)。`ManifestListEntry` および `ManifestFileEntry` のコンテンツタイプに対してのみ設定されます。 | - - - -## `content_type` の値 {#content-type-values} - -- `None`: コンテンツなし。 -- `Metadata`: ルートメタデータファイル。 -- `ManifestListMetadata`: マニフェストリストのメタデータ。 -- `ManifestListEntry`: マニフェストリスト内のエントリ。 -- `ManifestFileMetadata`: マニフェストファイルのメタデータ。 -- `ManifestFileEntry`: マニフェストファイル内のエントリ。 - - - - - -## ログの詳細度の制御 {#controlling-log-verbosity} - -[`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level) 設定を使用して、どのメタデータイベントをログに出力するかを制御できます。 - -現在のクエリで使用されるすべてのメタデータをログに出力するには: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'manifest_file_entry'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -現在のクエリで使用されるルートメタデータ JSON ファイルだけをログに記録するには: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'metadata'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -[`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level) 設定の説明に、より詳しい情報があります。 - -### 補足事項 {#good-to-know} - -* Iceberg テーブルを詳細に調査する必要がある場合にのみ、クエリ単位で `iceberg_metadata_log_level` を使用してください。そうしないと、ログテーブルに不要なメタデータが大量に蓄積され、パフォーマンス低下を招く可能性があります。 -* このテーブルは主にデバッグ目的であり、エンティティごとの一意性は保証されないため、重複したエントリが含まれている場合があります。 -* `ManifestListMetadata` より冗長な `content_type` を使用すると、マニフェストリストに対する Iceberg メタデータキャッシュは無効化されます。 -* 同様に、`ManifestFileMetadata` より冗長な `content_type` を使用すると、マニフェストファイルに対する Iceberg メタデータキャッシュは無効化されます。 - - -## 関連項目 {#see-also} -- [Iceberg テーブルエンジン](../../engines/table-engines/integrations/iceberg.md) -- [Iceberg テーブル関数](../../sql-reference/table-functions/iceberg.md) -- [system.iceberg_history](./iceberg_history.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md deleted file mode 100644 index 2060d109a89..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md +++ /dev/null @@ -1,391 +0,0 @@ ---- -description: 'データベースオブジェクトのメタデータに対して、ほぼ標準化された DBMS 非依存ビューを提供するシステムデータベース。' -keywords: ['system database', 'information_schema'] -slug: /operations/system-tables/information_schema -title: 'INFORMATION_SCHEMA' -doc_type: 'reference' ---- - -`INFORMATION_SCHEMA`(または `information_schema`)は、データベースオブジェクトのメタデータに対して(ある程度)標準化された[DBMS 非依存ビュー](https://en.wikipedia.org/wiki/Information_schema)を提供するシステムデータベースです。`INFORMATION_SCHEMA` 内のビューは、一般に通常のシステムテーブルと比べて情報量や柔軟性の面で劣りますが、ツールはこれらを利用することで、複数の DBMS 間で共通した方法で基本的な情報を取得できます。`INFORMATION_SCHEMA` 内のビューの構造と内容は、後方互換性を維持する形で進化していくことになっています。つまり、新しい機能のみが追加され、既存の機能は変更や削除はされません。内部実装の観点では、`INFORMATION_SCHEMA` のビューは通常、[system.columns](../../operations/system-tables/columns.md)、[system.databases](../../operations/system-tables/databases.md)、[system.tables](../../operations/system-tables/tables.md) などの通常のシステムテーブルに対応付けられます。 - -```sql -SHOW TABLES FROM INFORMATION_SCHEMA; - --- または: -SHOW TABLES FROM information_schema; -``` - -```text -┌─name────────────────────┐ -│ COLUMNS │ -│ KEY_COLUMN_USAGE │ -│ REFERENTIAL_CONSTRAINTS │ -│ SCHEMATA │ -| STATISTICS | -│ TABLES │ -│ VIEWS │ -│ columns │ -│ key_column_usage │ -│ referential_constraints │ -│ schemata │ -| statistics | -│ tables │ -│ views │ -└─────────────────────────┘ -``` - -`INFORMATION_SCHEMA` には次のビューが含まれます。 - -* [COLUMNS](#columns) -* [KEY_COLUMN_USAGE](#key_column_usage) -* [REFERENTIAL_CONSTRAINTS](#referential_constraints) -* [SCHEMATA](#schemata) -* [STATISTICS](#statistics) -* [TABLES](#tables) -* [VIEWS](#views) - -他のデータベースとの互換性のために、大文字小文字を区別しない同等のビュー(例: `INFORMATION_SCHEMA.columns`)も用意されています。同様に、これらのビュー内のすべてのカラムについても、小文字(例: `table_name`)と大文字(`TABLE_NAME`)の両方の表記が提供されています。 - - -## COLUMNS {#columns} - -[system.columns](../../operations/system-tables/columns.md) システムテーブルから読み取られるカラムと、ClickHouse ではサポートされていないか意味を持たない(常に `NULL` となる)が、標準上存在している必要があるカラムを含みます。 - -カラム: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -クエリ: - -```sql -SELECT table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - column_default, - is_nullable, - data_type, - character_maximum_length, - character_octet_length, - numeric_precision, - numeric_precision_radix, - numeric_scale, - datetime_precision, - character_set_catalog, - character_set_schema, - character_set_name, - collation_catalog, - collation_schema, - collation_name, - domain_catalog, - domain_schema, - domain_name, - column_comment, - column_type -FROM INFORMATION_SCHEMA.COLUMNS -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -table_catalog: default -table_schema: default -table_name: describe_example -column_name: id -ordinal_position: 1 -column_default: -is_nullable: 0 -data_type: UInt64 -character_maximum_length: ᴺᵁᴸᴸ -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: 64 -numeric_precision_radix: 2 -numeric_scale: 0 -datetime_precision: ᴺᵁᴸᴸ -character_set_catalog: ᴺᵁᴸᴸ -character_set_schema: ᴺᵁᴸᴸ -character_set_name: ᴺᵁᴸᴸ -collation_catalog: ᴺᵁᴸᴸ -collation_schema: ᴺᵁᴸᴸ -collation_name: ᴺᵁᴸᴸ -domain_catalog: ᴺᵁᴸᴸ -domain_schema: ᴺᵁᴸᴸ -domain_name: ᴺᵁᴸᴸ -``` - - -## SCHEMATA {#schemata} - -[system.databases](../../operations/system-tables/databases.md) システムテーブルから読み取られるカラムと、ClickHouse ではサポートされていない、または意味を持たない(常に `NULL`)が、標準上は存在している必要があるカラムを含みます。 - -カラム: - -* `catalog_name` ([String](../../sql-reference/data-types/string.md)) — データベースの名前。 -* `schema_name` ([String](../../sql-reference/data-types/string.md)) — データベースの名前。 -* `schema_owner` ([String](../../sql-reference/data-types/string.md)) — スキーマの所有者名。常に `'default'`。 -* `default_character_set_catalog` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`。サポートされていません。 -* `default_character_set_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`。サポートされていません。 -* `default_character_set_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`。サポートされていません。 -* `sql_path` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`。サポートされていません。 - -**例** - -クエリ: - -```sql -SELECT catalog_name, - schema_name, - schema_owner, - default_character_set_catalog, - default_character_set_schema, - default_character_set_name, - sql_path -FROM information_schema.schemata -WHERE schema_name ILIKE 'information_schema' -LIMIT 1 -FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -catalog_name: INFORMATION_SCHEMA -schema_name: INFORMATION_SCHEMA -schema_owner: default -default_character_set_catalog: ᴺᵁᴸᴸ -default_character_set_schema: ᴺᵁᴸᴸ -default_character_set_name: ᴺᵁᴸᴸ -sql_path: ᴺᵁᴸᴸ -``` - - -## TABLES {#tables} - -[system.tables](../../operations/system-tables/tables.md) システムテーブルから読み取った列を含みます。 - -Columns: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — テーブルが存在するデータベース名。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — テーブルが存在するデータベース名。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — テーブル名。 -* `table_type` ([String](../../sql-reference/data-types/string.md)) — テーブル種別。取り得る値は次のとおりです。 - * `BASE TABLE` - * `VIEW` - * `FOREIGN TABLE` - * `LOCAL TEMPORARY` - * `SYSTEM VIEW` -* `table_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 総行数。特定できない場合は NULL。 -* `data_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — ディスク上のデータサイズ。特定できない場合は NULL。 -* `index_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — プライマリキー、セカンダリインデックス、およびすべてのマークの合計サイズ。 -* `table_collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — テーブルのデフォルト照合順序。常に `utf8mb4_0900_ai_ci`。 -* `table_comment` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — テーブル作成時に指定されたコメント。 - -**Example** - -Query: - -```sql -SELECT table_catalog, - table_schema, - table_name, - table_type, - table_collation, - table_comment -FROM INFORMATION_SCHEMA.TABLES -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -結果: - -```text -行 1: -────── -table_catalog: default -table_schema: default -table_name: describe_example -table_type: BASE TABLE -table_collation: utf8mb4_0900_ai_ci -table_comment: -``` - - -## VIEWS {#views} - -テーブルエンジン [View](../../engines/table-engines/special/view.md) が使用されている場合に、[system.tables](../../operations/system-tables/tables.md) システムテーブルから読み込まれるカラムが含まれます。 - -カラム: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — テーブルが存在するデータベースの名前。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — テーブルが存在するデータベースの名前。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — テーブル名。 -* `view_definition` ([String](../../sql-reference/data-types/string.md)) — ビュー用の `SELECT` クエリ。 -* `check_option` ([String](../../sql-reference/data-types/string.md)) — `NONE`、チェックを行わない。 -* `is_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`、ビューは更新されない。 -* `is_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — 作成されたビューが[マテリアライズドビュー](/sql-reference/statements/create/view#materialized-view)かどうかを示します。取り得る値: - * `NO` — 作成されたビューはマテリアライズドビューではない。 - * `YES` — 作成されたビューはマテリアライズドビューである。 -* `is_trigger_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`、トリガーは更新されない。 -* `is_trigger_deletable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`、トリガーは削除されない。 -* `is_trigger_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`、トリガーにはデータが挿入されない。 - -**例** - -クエリ: - -```sql -CREATE VIEW v (n Nullable(Int32), f Float64) AS SELECT n, f FROM t; -CREATE MATERIALIZED VIEW mv ENGINE = Null AS SELECT * FROM system.one; -SELECT table_catalog, - table_schema, - table_name, - view_definition, - check_option, - is_updatable, - is_insertable_into, - is_trigger_updatable, - is_trigger_deletable, - is_trigger_insertable_into -FROM information_schema.views -WHERE table_schema = currentDatabase() -LIMIT 1 -FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -table_catalog: default -table_schema: default -table_name: mv -view_definition: SELECT * FROM system.one -check_option: NONE -is_updatable: NO -is_insertable_into: YES -is_trigger_updatable: NO -is_trigger_deletable: NO -is_trigger_insertable_into: NO -``` - - -## KEY_COLUMN_USAGE {#key_column_usage} - -制約によって制限されている [system.tables](../../operations/system-tables/tables.md) システムテーブル内の列を含みます。 - -列: - -* `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。常に `def`。 -* `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 制約が属するスキーマ(データベース)の名前。 -* `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 制約の名前。 -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。常に `def`。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — テーブルが属するスキーマ(データベース)の名前。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — 制約を持つテーブルの名前。 -* `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 制約を持つ列の名前。 -* `ordinal_position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 現在は未使用。常に `1`。 -* `position_in_unique_constraint` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt32](../../sql-reference/data-types/int-uint.md))) — 現在は未使用。常に `NULL`。 -* `referenced_table_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。常に `NULL`。 -* `referenced_table_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。常に `NULL`。 -* `referenced_column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。常に `NULL`。 - -**例** - -```sql -CREATE TABLE test (i UInt32, s String) ENGINE MergeTree ORDER BY i; -SELECT constraint_catalog, - constraint_schema, - constraint_name, - table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - position_in_unique_constraint, - referenced_table_schema, - referenced_table_name, - referenced_column_name -FROM information_schema.key_column_usage -WHERE table_name = 'test' -FORMAT Vertical; -``` - -結果: - -```response -Row 1: -────── -constraint_catalog: def -constraint_schema: default -constraint_name: PRIMARY -table_catalog: def -table_schema: default -table_name: test -column_name: i -ordinal_position: 1 -position_in_unique_constraint: ᴺᵁᴸᴸ -referenced_table_schema: ᴺᵁᴸᴸ -referenced_table_name: ᴺᵁᴸᴸ -referenced_column_name: ᴺᵁᴸᴸ -``` - - -## REFERENTIAL_CONSTRAINTS {#referential_constraints} - -外部キーに関する情報を保持します。現在は常に空の結果(行なし)を返しますが、Tableau Online などのサードパーティツールとの互換性を確保するには十分です。 - -列: - -- `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `unique_constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `unique_constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `unique_constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `match_option` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `update_rule` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `delete_rule` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `table_name` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `referenced_table_name` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 - - - -## STATISTICS {#statistics} - -テーブルインデックスに関する情報を提供します。現在は空の結果セット(行なし)を返しますが、Tableau Online などのサードパーティツールとの互換性を確保するにはこれで十分です。 - -列: - -- `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `table_schema` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `table_name` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `non_unique` ([Int32](../../sql-reference/data-types/int-uint.md)) — 現在は未使用。 -- `index_schema` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `index_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `seq_in_index` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 現在は未使用。 -- `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `cardinality` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — 現在は未使用。 -- `sub_part` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — 現在は未使用。 -- `packed` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 -- `nullable` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `index_type` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `comment` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `index_comment` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `is_visible` ([String](../../sql-reference/data-types/string.md)) — 現在は未使用。 -- `expression` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 現在は未使用。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md deleted file mode 100644 index cbb63cf5e41..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'すべてのアリーナから集約した、jemalloc アロケータによるサイズクラス(ビン)別のメモリアロケーション情報を保持するシステムテーブル。' -keywords: ['system table', 'jemalloc_bins'] -slug: /operations/system-tables/jemalloc_bins -title: 'system.jemalloc_bins' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -jemalloc アロケーターによるサイズクラス(bin)ごとのメモリアロケーション情報を、すべてのアリーナから集約したものを含みます。 -jemalloc におけるスレッドローカルキャッシュの影響により、これらの統計は完全に正確ではない場合があります。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -現在のメモリ使用量のうち、特に大きく影響しているアロケーションのサイズを特定します。 - -```sql -SELECT - *, - allocations - deallocations AS active_allocations, - size * active_allocations AS allocated_bytes -FROM system.jemalloc_bins -WHERE allocated_bytes > 0 -ORDER BY allocated_bytes DESC -LIMIT 10 -``` - -```text -┌─index─┬─large─┬─────size─┬─allocactions─┬─deallocations─┬─active_allocations─┬─allocated_bytes─┐ -│ 82 │ 1 │ 50331648 │ 1 │ 0 │ 1 │ 50331648 │ -│ 10 │ 0 │ 192 │ 512336 │ 370710 │ 141626 │ 27192192 │ -│ 69 │ 1 │ 5242880 │ 6 │ 2 │ 4 │ 20971520 │ -│ 3 │ 0 │ 48 │ 16938224 │ 16559484 │ 378740 │ 18179520 │ -│ 28 │ 0 │ 4096 │ 122924 │ 119142 │ 3782 │ 15491072 │ -│ 61 │ 1 │ 1310720 │ 44569 │ 44558 │ 11 │ 14417920 │ -│ 39 │ 1 │ 28672 │ 1285 │ 913 │ 372 │ 10665984 │ -│ 4 │ 0 │ 64 │ 2837225 │ 2680568 │ 156657 │ 10026048 │ -│ 6 │ 0 │ 96 │ 2617803 │ 2531435 │ 86368 │ 8291328 │ -│ 36 │ 1 │ 16384 │ 22431 │ 21970 │ 461 │ 7553024 │ -└───────┴───────┴──────────┴──────────────┴───────────────┴────────────────────┴─────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md deleted file mode 100644 index 5115ac0d17d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: 'Kafka コンシューマーに関する情報を含む system テーブル。' -keywords: ['system テーブル', 'kafka_consumers'] -slug: /operations/system-tables/kafka_consumers -title: 'system.kafka_consumers' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Kafka コンシューマに関する情報を保持します。 -[Kafka table engine](../../engines/table-engines/integrations/kafka)(ネイティブな ClickHouse 統合機能)に適用されます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -例: - -```sql -SELECT * -FROM system.kafka_consumers -FORMAT Vertical -``` - -```text -行 1: -────── -database: test -table: kafka -consumer_id: ClickHouse-instance-test-kafka-1caddc7f-f917-4bb1-ac55-e28bd103a4a0 -assignments.topic: ['system_kafka_cons'] -assignments.partition_id: [0] -assignments.current_offset: [18446744073709550615] -exceptions.time: [] -exceptions.text: [] -last_poll_time: 2006-11-09 18:47:47 -num_messages_read: 4 -last_commit_time: 2006-11-10 04:39:40 -num_commits: 1 -last_rebalance_time: 1970-01-01 00:00:00 -num_rebalance_revocations: 0 -num_rebalance_assignments: 1 -is_currently_used: 1 -rdkafka_stat: {...} - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md deleted file mode 100644 index bd19c3f1f01..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: 'ClickHouse ソースコードの contrib ディレクトリにあるサードパーティライブラリのライセンスを格納しているシステムテーブル。' -keywords: ['システムテーブル', 'ライセンス'] -slug: /operations/system-tables/licenses -title: 'system.licenses' -doc_type: 'reference' ---- - -# system.licenses {#systemlicenses} - -ClickHouse のソースコード内にある [contrib](https://github.com/ClickHouse/ClickHouse/tree/master/contrib) ディレクトリに含まれるサードパーティライブラリのライセンスが格納されています。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT library_name, license_type, license_path FROM system.licenses LIMIT 15 -``` - -```text -┌─library_name───────┬─license_type─┬─license_path────────────────────────┐ -│ aws-c-common │ Apache │ /contrib/aws-c-common/LICENSE │ -│ base64 │ BSD 2-clause │ /contrib/aklomp-base64/LICENSE │ -│ brotli │ MIT │ /contrib/brotli/LICENSE │ -│ [...] │ [...] │ [...] │ -└────────────────────┴──────────────┴─────────────────────────────────────┘ - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md deleted file mode 100644 index ec16f8a713c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'MergeTree テーブルの設定に関する情報を格納する system テーブル。' -keywords: ['system table', 'merge_tree_settings'] -slug: /operations/system-tables/merge_tree_settings -title: 'system.merge_tree_settings' -doc_type: 'reference' ---- - -# system.merge_tree_settings {#systemmerge_tree_settings} - -`MergeTree` テーブルの設定に関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.merge_tree_settings LIMIT 3 FORMAT Vertical; -``` - -```response -SELECT * -FROM system.merge_tree_settings -LIMIT 3 -FORMAT Vertical - -Query id: 2580779c-776e-465f-a90c-4b7630d0bb70 - -Row 1: -────── -name: min_compress_block_size -value: 0 -default: 0 -changed: 0 -description: グラニュール書き込み時に、バッファ内の未圧縮データのサイズが指定された閾値以上の場合、データを圧縮します。この設定が未指定の場合は、対応するグローバル設定が使用されます。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 2: -────── -name: max_compress_block_size -value: 0 -default: 0 -changed: 0 -description: バッファ内の未圧縮データのサイズが指定された閾値以上の場合、データを圧縮します。現在のグラニュールが未完了の場合でも、データブロックは圧縮されます。この設定が未指定の場合は、対応するグローバル設定が使用されます。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 3: -────── -name: index_granularity -value: 8192 -default: 8192 -changed: 0 -description: 1つのプライマリキー値に対応する行数。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -3 rows in set. Elapsed: 0.001 sec. -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md deleted file mode 100644 index b44cadc705b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: 'MergeTree ファミリーに属するテーブルに対して現在実行中のマージおよびパーツのミューテーションに関する情報を含む system テーブル。' -keywords: ['system テーブル', 'マージ処理'] -slug: /operations/system-tables/merges -title: 'system.merges' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.merges {#systemmerges} - - - -MergeTree ファミリーのテーブルに対して現在実行中のマージおよびパーツのミューテーションに関する情報が含まれます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md deleted file mode 100644 index e0ac0e1e564..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'テーブル `system.metrics` と `system.events` に由来するメトリクス値の履歴を保持し、定期的にディスクにフラッシュされるシステムテーブル。' -keywords: ['システムテーブル', 'metric_log'] -slug: /operations/system-tables/metric_log -title: 'system.metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.metric_log {#systemmetric_log} - - - -`system.metrics` および `system.events` テーブルのメトリクス値の履歴を保持しており、定期的にディスクにフラッシュされます。 - -Columns: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行するサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度のイベント時刻。 - -**例** - -```sql -SELECT * FROM system.metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -milliseconds: 196 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -... -CurrentMetric_Revision: 54439 -CurrentMetric_VersionInteger: 20009001 -CurrentMetric_RWLockWaitingReaders: 0 -CurrentMetric_RWLockWaitingWriters: 0 -CurrentMetric_RWLockActiveReaders: 0 -CurrentMetric_RWLockActiveWriters: 0 -CurrentMetric_GlobalThread: 74 -CurrentMetric_GlobalThreadActive: 26 -CurrentMetric_LocalThread: 0 -CurrentMetric_LocalThreadActive: 0 -CurrentMetric_DistributedFilesToInsert: 0 -``` - -**スキーマ** -このテーブルは、XML タグ `` を使用して、異なるスキーマ種別に構成できます。デフォルトのスキーマ種別は `wide` であり、各メトリクスまたはプロファイルイベントが個別の列として保存されます。このスキーマは、単一列の読み取りに対して最も高いパフォーマンスと効率を発揮します。 - -`transposed` スキーマは、メトリクスやイベントが行として保存される `system.asynchronous_metric_log` と類似した形式でデータを保存します。このスキーマは、マージ時のリソース消費を削減するため、リソースの限られた環境での利用に有効です。 - -互換性のためのスキーマとして `transposed_with_wide_view` も存在します。これは、転置スキーマ(`system.transposed_metric_log`)を持つテーブルに実データを保存し、その上に wide スキーマを使用したビューを作成します。このビューは転置テーブルをクエリするため、`wide` スキーマから `transposed` スキーマへの移行に役立ちます。 - -**関連項目** - -* [metric_log 設定](../../operations/server-configuration-parameters/settings.md#metric_log) — 設定の有効化と無効化。 -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — 定期的に計算されるメトリクスを含みます。 -* [system.events](/operations/system-tables/events) — 発生した各種イベントを含みます。 -* [system.metrics](../../operations/system-tables/metrics.md) — 即時に計算されるメトリクスを含みます。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse モニタリングの基本概念。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md deleted file mode 100644 index 747bbd0ed1c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md +++ /dev/null @@ -1,789 +0,0 @@ ---- -description: '即時計算できる、または現在値を持つメトリクスを含むシステムテーブル。' -keywords: ['システムテーブル', 'メトリクス'] -slug: /operations/system-tables/metrics -title: 'system.metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.metrics {#systemmetrics} - - - -即座に計算できるメトリクス、または現在値を表すメトリクスを含みます。例えば、同時に処理されているクエリ数や、現在のレプリカ遅延などです。このテーブルの内容は常に最新です。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -サポートされているメトリクスの一覧は、ソースファイル [src/Common/CurrentMetrics.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/CurrentMetrics.cpp) で確認できます。 - -**例** - -```sql -SELECT * FROM system.metrics LIMIT 10 -``` - -```text -┌─metric───────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────┐ -│ Query │ 1 │ 実行中のクエリ数 │ -│ Merge │ 0 │ 実行中のバックグラウンドマージ数 │ -│ PartMutation │ 0 │ ミューテーション数(ALTER DELETE/UPDATE) │ -│ ReplicatedFetch │ 0 │ レプリカから取得中のデータパーツ数 │ -│ ReplicatedSend │ 0 │ レプリカへ送信中のデータパーツ数 │ -│ ReplicatedChecks │ 0 │ 整合性チェック中のデータパーツ数 │ -│ BackgroundMergesAndMutationsPoolTask │ 0 │ 関連するバックグラウンドプール内のアクティブなマージとミューテーション数│ -│ BackgroundFetchesPoolTask │ 0 │ 関連するバックグラウンドプール内のアクティブなフェッチ数 │ -│ BackgroundCommonPoolTask │ 0 │ 関連するバックグラウンドプール内のアクティブなタスク数 │ -│ BackgroundMovePoolTask │ 0 │ BackgroundProcessingPool内の移動用アクティブタスク数 │ -└──────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────┘ -``` - - -## メトリクスの説明 {#metric-descriptions} - -### AggregatorThreads {#aggregatorthreads} - -Aggregator スレッドプール内のスレッド数。 - -### AggregatorThreadsActive {#aggregatorthreadsactive} - -タスクを実行中の Aggregator スレッドプール内のスレッド数。 - -### TablesLoaderForegroundThreads {#tablesloaderforegroundthreads} - -非同期ローダーのフォアグラウンドスレッドプールで使用されるスレッド数。 - -### TablesLoaderForegroundThreadsActive {#tablesloaderforegroundthreadsactive} - -タスクを実行中の非同期ローダー用フォアグラウンドスレッドプール内のスレッド数。 - -### TablesLoaderBackgroundThreads {#tablesloaderbackgroundthreads} - -非同期ローダー用バックグラウンドスレッドプールのスレッド数。 - -### TablesLoaderBackgroundThreadsActive {#tablesloaderbackgroundthreadsactive} - -非同期ローダーのバックグラウンドスレッドプールでタスクを実行中のスレッド数。 - -### AsyncInsertCacheSize {#asyncinsertcachesize} - -キャッシュ内に保持されている非同期インサートのハッシュIDの数 - -### AsynchronousInsertThreads {#asynchronousinsertthreads} - -AsynchronousInsert スレッドプール内のスレッド数。 - -### AsynchronousInsertThreadsActive {#asynchronousinsertthreadsactive} - -タスクを実行している AsynchronousInsert スレッドプール内のスレッド数。 - -### AsynchronousReadWait {#asynchronousreadwait} - -非同期読み取りを待機中のスレッド数。 - -### BackgroundBufferFlushSchedulePoolSize {#backgroundbufferflushschedulepoolsize} - -BackgroundBufferFlushSchedulePool 内のタスク数の最大値 - -### BackgroundBufferFlushSchedulePoolTask {#backgroundbufferflushschedulepooltask} - -BackgroundBufferFlushSchedulePool 内のアクティブなタスク数。このプールは Buffer を定期的にフラッシュするために使用されます。 - -### BackgroundCommonPoolSize {#backgroundcommonpoolsize} - -関連するバックグラウンドプールのタスク数の上限 - -### BackgroundCommonPoolTask {#backgroundcommonpooltask} - -対応するバックグラウンドプール内のアクティブなタスク数 - -### BackgroundDistributedSchedulePoolSize {#backgrounddistributedschedulepoolsize} - -BackgroundDistributedSchedulePool 内のタスク数の上限 - -### BackgroundDistributedSchedulePoolTask {#backgrounddistributedschedulepooltask} - -BackgroundDistributedSchedulePool 内のアクティブなタスク数。このプールは、バックグラウンドで行われる分散送信処理に使用されます。 - -### BackgroundFetchesPoolSize {#backgroundfetchespoolsize} - -関連するバックグラウンドプールでの同時フェッチ数の上限 - -### BackgroundFetchesPoolTask {#backgroundfetchespooltask} - -関連するバックグラウンドプール内のアクティブなフェッチ数 - -### BackgroundMergesAndMutationsPoolSize {#backgroundmergesandmutationspoolsize} - -関連するバックグラウンドプール内でアクティブなマージおよびミューテーション数の上限 - -### BackgroundMergesAndMutationsPoolTask {#backgroundmergesandmutationspooltask} - -関連するバックグラウンドプール内のアクティブなマージおよびミューテーションの数 - -### BackgroundMessageBrokerSchedulePoolSize {#backgroundmessagebrokerschedulepoolsize} - -メッセージストリーミング用 BackgroundProcessingPool におけるタスク数の上限 - -### BackgroundMessageBrokerSchedulePoolTask {#backgroundmessagebrokerschedulepooltask} - -メッセージストリーミング向け BackgroundProcessingPool のアクティブタスク数 - -### BackgroundMovePoolSize {#backgroundmovepoolsize} - -移動処理用 BackgroundProcessingPool におけるタスク数の上限 - -### BackgroundMovePoolTask {#backgroundmovepooltask} - -移動処理用の BackgroundProcessingPool におけるアクティブなタスク数 - -### BackgroundSchedulePoolSize {#backgroundschedulepoolsize} - -BackgroundSchedulePool 内のタスク数の上限。このプールは、古いデータパーツのクリーンアップ、データパーツの ALTER、レプリカの再初期化など、`ReplicatedMergeTree` の定期的なタスクに使用されます。 - -### BackgroundSchedulePoolTask {#backgroundschedulepooltask} - -BackgroundSchedulePool 内のアクティブなタスク数です。このプールは、古いデータパーツの削除、データパーツに対する ALTER 操作、レプリカの再初期化など、ReplicatedMergeTree の定期タスクに使用されます。 - -### BackupsIOThreads {#backupsiothreads} - -BackupsIO スレッドプールのスレッド数。 - -### BackupsIOThreadsActive {#backupsiothreadsactive} - -BackupsIO スレッドプール内でタスクを実行中のスレッド数。 - -### BackupsThreads {#backupsthreads} - -BACKUP用のスレッドプール内のスレッド数。 - -### BackupsThreadsActive {#backupsthreadsactive} - -BACKUP タスクを実行中のスレッドプール内のスレッド数。 - -### BrokenDistributedFilesToInsert {#brokendistributedfilestoinsert} - -非同期で Distributed テーブルに挿入するためのファイルのうち、破損としてマークされたファイル数。このメトリクス値は起動時に 0 から開始します。各シャードごとのファイル数を合算した値です。 - -### CacheDetachedFileSegments {#cachedetachedfilesegments} - -既存の切り離されたキャッシュファイルセグメントの数 - -### CacheDictionaryThreads {#cachedictionarythreads} - -CacheDictionary のスレッドプール内のスレッド数。 - -### CacheDictionaryThreadsActive {#cachedictionarythreadsactive} - -タスクを実行中の CacheDictionary スレッドプール内のスレッド数。 - -### CacheDictionaryUpdateQueueBatches {#cachedictionaryupdatequeuebatches} - -CacheDictionary における更新キュー内の「バッチ」(キー集合)の数。 - -### CacheDictionaryUpdateQueueKeys {#cachedictionaryupdatequeuekeys} - -CacheDictionary の更新キュー内にあるキーの正確な数。 - -### CacheFileSegments {#cachefilesegments} - -既存のキャッシュファイルセグメント数 - -### ContextLockWait {#contextlockwait} - -Context 内でロック獲得待ちのスレッド数。これはグローバルロックです。 - -### DDLWorkerThreads {#ddlworkerthreads} - -ON CLUSTER クエリに対応する DDLWorker スレッドプールのスレッド数。 - -### DDLWorkerThreadsActive {#ddlworkerthreadsactive} - -`ON CLUSTER` クエリのタスクを実行している `DDLWORKER` スレッドプール内のスレッド数。 - -### DatabaseCatalogThreads {#databasecatalogthreads} - -DatabaseCatalog スレッドプール内のスレッド数。 - -### DatabaseCatalogThreadsActive {#databasecatalogthreadsactive} - -タスクを実行中の DatabaseCatalog スレッドプール内のスレッド数。 - -### DatabaseOnDiskThreads {#databaseondiskthreads} - -DatabaseOnDisk のスレッドプール内のスレッド数。 - -### DatabaseOnDiskThreadsActive {#databaseondiskthreadsactive} - -タスクを実行している DatabaseOnDisk スレッドプール内のスレッド数。 - -### DelayedInserts {#delayedinserts} - -MergeTree テーブルのパーティションでアクティブなデータパーツ数が多すぎるためにスロットルされた INSERT クエリの数。 - -### DestroyAggregatesThreads {#destroyaggregatesthreads} - -集約状態の破棄を行うスレッドプールのスレッド数。 - -### DestroyAggregatesThreadsActive {#destroyaggregatesthreadsactive} - -集約状態の破棄タスクを実行するスレッドプール内のスレッド数。 - -### DictCacheRequests {#dictcacherequests} - -キャッシュ型ディクショナリのデータソースに対する処理中のリクエスト数。 - -### DiskObjectStorageAsyncThreads {#diskobjectstorageasyncthreads} - -DiskObjectStorage の非同期スレッドプールのスレッド数。 - -### DiskObjectStorageAsyncThreadsActive {#diskobjectstorageasyncthreadsactive} - -DiskObjectStorage の非同期スレッドプールでタスクを実行中のスレッド数。 - -### DiskSpaceReservedForMerge {#diskspacereservedformerge} - -バックグラウンドで実行中のマージ処理のために予約されているディスクスペース。現在マージ中のパーツの合計サイズよりもわずかに多めに確保されます。 - -### DistributedFilesToInsert {#distributedfilestoinsert} - -Distributed テーブルへの非同期挿入で処理待ちとなっているファイル数。シャードごとのファイル数を合計した値。 - -### DistributedSend {#distributedsend} - -Distributed テーブルに対して INSERT されたデータを送信するリモートサーバーへの接続数。同期モードおよび非同期モードの両方を対象とする。 - -### EphemeralNode {#ephemeralnode} - -ZooKeeper に保持されているエフェメラルノードの数。 - -### FilesystemCacheElements {#filesystemcacheelements} - -ファイルシステムキャッシュ要素(ファイルセグメント) - -### FilesystemCacheReadBuffers {#filesystemcachereadbuffers} - -アクティブなキャッシュバッファの数 - -### FilesystemCacheSize {#filesystemcachesize} - -ファイルシステムキャッシュのサイズ(バイト単位) - -### QueryCacheBytes {#querycachebytes} - -クエリキャッシュの合計サイズ(バイト単位)。 - -### QueryCacheEntries {#querycacheentries} - -クエリキャッシュ内のエントリー総数。 - -### UncompressedCacheBytes {#uncompressedcachebytes} - -非圧縮キャッシュの合計サイズ(バイト単位)。非圧縮キャッシュは通常パフォーマンス向上には寄与しないため、使用は極力避けるべきです。 - -### UncompressedCacheCells {#uncompressedcachecells} - -### CompiledExpressionCacheBytes {#compiledexpressioncachebytes} - -JIT コンパイル済みコードのキャッシュに使用されている総バイト数。 - -### CompiledExpressionCacheCount {#compiledexpressioncachecount} - -JIT コンパイルされたコードのキャッシュ内のエントリ総数です。 - -### MMapCacheCells {#mmapcachecells} - -`mmap`(メモリマップ)で開かれているファイル数。これは、設定 `local_filesystem_read_method` が `mmap` に設定されているクエリで使用されます。`mmap` で開かれたファイルは、高コストな TLB フラッシュを回避するためにキャッシュに保持されます。 - -### MarkCacheBytes {#markcachebytes} - -マークキャッシュの合計サイズ(バイト単位) - -### MarkCacheFiles {#markcachefiles} - -マークキャッシュに格納されているマークファイルの総数 - -### GlobalThread {#globalthread} - -グローバルスレッドプールのスレッド数。 - -### GlobalThreadActive {#globalthreadactive} - -グローバルスレッドプールでタスクを実行しているスレッド数。 - -### HTTPConnection {#httpconnection} - -HTTP サーバーへの接続数 - -### HashedDictionaryThreads {#hasheddictionarythreads} - -HashedDictionary スレッドプール内のスレッド数。 - -### HashedDictionaryThreadsActive {#hasheddictionarythreadsactive} - -HashedDictionary のスレッドプールでタスクを実行中のスレッド数。 - -### IOPrefetchThreads {#ioprefetchthreads} - -IOプリフェッチ用スレッドプール内のスレッド数。 - -### IOPrefetchThreadsActive {#ioprefetchthreadsactive} - -IO プリフェッチスレッドプールで現在タスクを実行しているスレッド数。 - -### IOThreads {#iothreads} - -IOスレッドプール内のスレッド数。 - -### IOThreadsActive {#iothreadsactive} - -タスクを実行している IO スレッドプール内のスレッド数。 - -### IOUringInFlightEvents {#iouringinflightevents} - -処理中の io_uring SQEの数 - -### IOUringPendingEvents {#iouringpendingevents} - -送信待ちの io_uring SQE 数 - -### IOWriterThreads {#iowriterthreads} - -IO writer スレッドプール内のスレッド数。 - -### IOWriterThreadsActive {#iowriterthreadsactive} - -タスクを実行中の IO ライタースレッドプール内のスレッド数。 - -### InterserverConnection {#interserverconnection} - -他のレプリカがパーツを取得するために確立している接続数 - -### KafkaAssignedPartitions {#kafkaassignedpartitions} - -Kafka テーブルに現在割り当てられているパーティション数 - -### KafkaBackgroundReads {#kafkabackgroundreads} - -現在動作中のバックグラウンド読み取りの数(Kafka からマテリアライズドビューを更新している処理の数) - -### KafkaConsumers {#kafkaconsumers} - -アクティブな Kafka コンシューマーの数 - -### KafkaConsumersInUse {#kafkaconsumersinuse} - -直接またはバックグラウンドでの読み出しに現在使用されているコンシューマーの数 - -### KafkaConsumersWithAssignment {#kafkaconsumerswithassignment} - -パーティションが割り当てられているアクティブな Kafka コンシューマーの数。 - -### KafkaLibrdkafkaThreads {#kafkalibrdkafkathreads} - -アクティブな librdkafka スレッド数 - -### KafkaProducers {#kafkaproducers} - -作成済みのアクティブな Kafka プロデューサー数 - -### KafkaWrites {#kafkawrites} - -現在実行中の Kafka への INSERT の数 - -### KeeperAliveConnections {#keeperaliveconnections} - -有効な接続数 - -### KeeperOutstandingRequests {#keeperoutstandingrequests} - -未処理リクエスト数 - -### LocalThread {#localthread} - -ローカルスレッドプール内のスレッド数です。ローカルスレッドプール内のスレッドは、グローバルスレッドプールから供給されます。 - -### LocalThreadActive {#localthreadactive} - -ローカルのスレッドプールでタスクを実行しているスレッドの数。 - -### MMappedAllocBytes {#mmappedallocbytes} - -mmap によるアロケーションの合計バイト数 - -### MMappedAllocs {#mmappedallocs} - -mmap によるメモリ割り当ての総数 - -### MMappedFileBytes {#mmappedfilebytes} - -メモリマップされたファイル領域の合計サイズ。 - -### MMappedFiles {#mmappedfiles} - -メモリマップファイルの総数。 - -### MarksLoaderThreads {#marksloaderthreads} - -マーク読み込み用スレッドプールのスレッド数。 - -### MarksLoaderThreadsActive {#marksloaderthreadsactive} - -マークを読み込むためのスレッドプール内で、タスクを実行中のスレッド数。 - -### MaxDDLEntryID {#maxddlentryid} - -DDLWorker によって処理された DDL エントリの最大 ID。 - -### MaxPushedDDLEntryID {#maxpushedddlentryid} - -DDLWorker が ZooKeeper にプッシュした DDL エントリ ID の最大値。 - -### MemoryTracking {#memorytracking} - -サーバーによって割り当てられたメモリの総バイト数。 - -### Merge {#merge} - -実行中のバックグラウンドマージ数 - -### MergeTreeAllRangesAnnouncementsSent {#mergetreeallrangesannouncementssent} - -リモートサーバー側で、データパーツの集合(MergeTree テーブル用)についてイニシエータサーバー宛てに送信中のアナウンス数の現在値。計測はリモートサーバー側で行われます。 - -### MergeTreeBackgroundExecutorThreads {#mergetreebackgroundexecutorthreads} - -MergeTreeBackgroundExecutor のスレッドプールのスレッド数。 - -### MergeTreeBackgroundExecutorThreadsActive {#mergetreebackgroundexecutorthreadsactive} - -タスクを実行中の MergeTreeBackgroundExecutor スレッドプール内で稼働しているスレッド数。 - -### MergeTreeDataSelectExecutorThreads {#mergetreedataselectexecutorthreads} - -MergeTreeDataSelectExecutor スレッドプール内のスレッド数。 - -### MergeTreeDataSelectExecutorThreadsActive {#mergetreedataselectexecutorthreadsactive} - -タスクを実行中の MergeTreeDataSelectExecutor スレッドプール内のスレッド数。 - -### MergeTreePartsCleanerThreads {#mergetreepartscleanerthreads} - -MergeTree のパーツクリーナースレッドプール内のスレッド数。 - -### MergeTreePartsCleanerThreadsActive {#mergetreepartscleanerthreadsactive} - -タスクを実行している MergeTree パーツクリーナー用スレッドプール内のスレッド数。 - -### MergeTreePartsLoaderThreads {#mergetreepartsloaderthreads} - -MergeTree パーツローダースレッドプール内のスレッド数。 - -### MergeTreePartsLoaderThreadsActive {#mergetreepartsloaderthreadsactive} - -タスクを実行している MergeTree パーツローダー用スレッドプール内のスレッド数。 - -### MergeTreeReadTaskRequestsSent {#mergetreereadtaskrequestssent} - -リモートサーバーから開始側サーバーに対して、読み取りタスク(MergeTree テーブル用)を選択するために送信されるコールバックリクエストのうち、現在処理中のものの数。リモートサーバー側で計測されます。 - -### Move {#move} - -現在実行中の Move の数 - -### MySQLConnection {#mysqlconnection} - -MySQL プロトコルを使用しているクライアント接続数 - -### NetworkReceive {#networkreceive} - -ネットワークからデータを受信するスレッド数です。ClickHouse に関連するネットワーク処理のみが対象で、サードパーティライブラリによるものは含まれません。 - -### NetworkSend {#networksend} - -ネットワークへデータを送信しているスレッド数。ClickHouse 関連のネットワーク処理のみが対象で、サードパーティライブラリによるものは含まれません。 - -### OpenFileForRead {#openfileforread} - -読み取り用に開かれているファイル数 - -### OpenFileForWrite {#openfileforwrite} - -書き込みのために開かれているファイル数 - -### ParallelFormattingOutputFormatThreads {#parallelformattingoutputformatthreads} - -ParallelFormattingOutputFormatThreads のスレッドプール内のスレッド数。 - -### ParallelFormattingOutputFormatThreadsActive {#parallelformattingoutputformatthreadsactive} - -タスクを実行中の ParallelFormattingOutputFormatThreads スレッドプール内で稼働しているスレッド数。 - -### PartMutation {#partmutation} - -ミューテーション(ALTER DELETE/UPDATE)の実行回数 - -### PartsActive {#partsactive} - -現在および今後実行される SELECT で使用されるアクティブなデータパーツ。 - -### PartsCommitted {#partscommitted} - -非推奨です。PartsActive を参照してください。 - -### PartsCompact {#partscompact} - -Compact 形式のパーツ。 - -### PartsDeleteOnDestroy {#partsdeleteondestroy} - -パートは別のディスクに移動されており、自身のデストラクタで削除されるべきです。 - -### PartsDeleting {#partsdeleting} - -識別用参照カウンタを持つ非アクティブなデータパーツで、現在クリーンアップ処理により削除中の状態です。 - -### PartsOutdated {#partsoutdated} - -アクティブなデータパーツではありませんが、現在実行中の `SELECT` によってのみ使用される可能性があります。`SELECT` が完了すると削除できる場合があります。 - -### PartsPreActive {#partspreactive} - -パーツは `data_parts` 内に存在しますが、`SELECT` クエリでは使用されません。 - -### PartsPreCommitted {#partsprecommitted} - -非推奨です。PartsPreActive を参照してください。 - -### PartsTemporary {#partstemporary} - -パーツは現在生成中で、まだ `data_parts` リストには含まれていません。 - -### PartsWide {#partswide} - -Wide 形式のパーツです。 - -### PendingAsyncInsert {#pendingasyncinsert} - -フラッシュされるのを待っている非同期挿入の数。 - -### PostgreSQLConnection {#postgresqlconnection} - -PostgreSQL プロトコルを使用しているクライアント接続数 - -### クエリ {#query} - -実行中のクエリ数 - -### QueryPreempted {#querypreempted} - -`priority` 設定により停止されて待機状態となったクエリの数。 - -### QueryThread {#querythread} - -クエリ処理スレッド数 - -### RWLockActiveReaders {#rwlockactivereaders} - -テーブルの RWLock で読み取りロックを保持しているスレッド数。 - -### RWLockActiveWriters {#rwlockactivewriters} - -テーブルの RWLock で書き込みロックを保持しているスレッド数。 - -### RWLockWaitingReaders {#rwlockwaitingreaders} - -テーブルの RWLock に対して読み取りロックの取得を待機しているスレッド数。 - -### RWLockWaitingWriters {#rwlockwaitingwriters} - -テーブルの RWLock の書き込みロック獲得を待っているスレッド数。 - -### 読み取り {#read} - -実行中の read 系(`read`、`pread`、`io_getevents` など)システムコールの数 - -### ReadTaskRequestsSent {#readtaskrequestssent} - -リモートサーバーからイニシエーターサーバーへ、読み取りタスクを選択するために送信されているコールバックリクエストの現在の数(`s3Cluster` テーブル関数などにおいて)。リモートサーバー側で計測されます。 - -### ReadonlyReplica {#readonlyreplica} - -ZooKeeper セッションの喪失後の再初期化、または ZooKeeper が構成されていない状態での起動により、現在読み取り専用状態になっている Replicated テーブルの数。 - -### RemoteRead {#remoteread} - -リモートリーダーで実行中の読み取り数 - -### ReplicatedChecks {#replicatedchecks} - -一貫性を検証中のデータパーツ数 - -### ReplicatedFetch {#replicatedfetch} - -レプリカからフェッチ中のデータパーツ数 - -### ReplicatedSend {#replicatedsend} - -レプリカへ送信しているデータパーツの数 - -### RestartReplicaThreads {#restartreplicathreads} - -RESTART REPLICA のスレッドプールで使用されるスレッド数。 - -### RestartReplicaThreadsActive {#restartreplicathreadsactive} - -RESTART REPLICA スレッドプール内でタスクを実行中のスレッド数。 - -### RestoreThreads {#restorethreads} - -RESTORE 用スレッドプールのスレッド数。 - -### RestoreThreadsActive {#restorethreadsactive} - -RESTORE のタスクを実行するスレッドプール内のスレッド数。 - -### リビジョン {#revision} - -サーバーのリビジョンです。パッチリリースを除き、各リリースまたはリリース候補ごとにインクリメントされる番号です。 - -### S3Requests {#s3requests} - -S3 リクエスト - -### SendExternalTables {#sendexternaltables} - -リモートサーバーに外部テーブル用のデータを送信している接続の数です。外部テーブルは、分散サブクエリで `GLOBAL IN` および `GLOBAL JOIN` 演算子を実装するために使用されます。 - -### SendScalars {#sendscalars} - -スカラーデータをリモートサーバーに送信している接続数です。 - -### StorageBufferBytes {#storagebufferbytes} - -Buffer テーブルのバッファ内にあるバイト数 - -### StorageBufferRows {#storagebufferrows} - -Buffer テーブルのバッファにある行数 - -### StorageDistributedThreads {#storagedistributedthreads} - -StorageDistributed のスレッドプールのスレッド数。 - -### StorageDistributedThreadsActive {#storagedistributedthreadsactive} - -タスクを実行している StorageDistributed スレッドプール内のスレッド数。 - -### StorageHiveThreads {#storagehivethreads} - -StorageHive のスレッドプールにおけるスレッド数。 - -### StorageHiveThreadsActive {#storagehivethreadsactive} - -タスクを実行している StorageHive スレッドプール内のスレッド数。 - -### StorageS3Threads {#storages3threads} - -StorageS3 のスレッドプールで使用されるスレッド数。 - -### StorageS3ThreadsActive {#storages3threadsactive} - -タスクを実行中の StorageS3 スレッドプール内のスレッド数。 - -### SystemReplicasThreads {#systemreplicasthreads} - -system.replicas のスレッドプール内のスレッド数。 - -### SystemReplicasThreadsActive {#systemreplicasthreadsactive} - -system.replicas のスレッドプールでタスクを実行しているスレッド数。 - -### TCPConnection {#tcpconnection} - -TCP サーバー(ネイティブインターフェイスを使用するクライアント)への接続数。サーバー間の分散クエリ接続も含まれます。 - -### TablesToDropQueueSize {#tablestodropqueuesize} - -バックグラウンドでのデータ削除を待っている削除対象テーブルの数。 - -### TemporaryFilesForAggregation {#temporaryfilesforaggregation} - -外部集約で作成される一時ファイルの数 - -### TemporaryFilesForJoin {#temporaryfilesforjoin} - -JOIN 用に作成された一時ファイルの数 - -### TemporaryFilesForSort {#temporaryfilesforsort} - -外部ソートのために作成された一時ファイルの数 - -### TemporaryFilesUnknown {#temporaryfilesunknown} - -用途不明の一時ファイルの作成数 - -### ThreadPoolFSReaderThreads {#threadpoolfsreaderthreads} - -local_filesystem_read_method=threadpool 使用時のスレッドプールのスレッド数。 - -### ThreadPoolFSReaderThreadsActive {#threadpoolfsreaderthreadsactive} - -local_filesystem_read_method=threadpool のタスクを実行中のスレッドプール内のスレッド数。 - -### ThreadPoolRemoteFSReaderThreads {#threadpoolremotefsreaderthreads} - -remote_filesystem_read_method=threadpool の場合に使用されるスレッドプールのスレッド数。 - -### ThreadPoolRemoteFSReaderThreadsActive {#threadpoolremotefsreaderthreadsactive} - -remote_filesystem_read_method=threadpool で実行中のタスクを処理しているスレッドプール内のスレッド数。 - -### ThreadsInOvercommitTracker {#threadsinovercommittracker} - -OvercommitTracker 内で待機中のスレッド数 - -### TotalTemporaryFiles {#totaltemporaryfiles} - -作成された一時ファイル数 - -### VersionInteger {#versioninteger} - -サーバーのバージョンを、基数 1000(base-1000)の単一の整数値で表します。たとえば、バージョン 11.22.33 は 11022033 に変換されます。 - -### Write {#write} - -fly における write(write、pwrite、io_getevents など)システムコールの回数 - -### ZooKeeperRequest {#zookeeperrequest} - -処理中の ZooKeeper へのリクエスト数。 - -### ZooKeeperSession {#zookeepersession} - -ZooKeeper へのセッション(接続)の数。1 つを超えてはいけません。ZooKeeper の一貫性モデルは線形化可能性を満たさず(古い読み取りを許容している)ため、複数の接続を使用するとバグを引き起こす可能性があります。 - -### ZooKeeperWatch {#zookeeperwatch} - -ZooKeeper 内のウォッチ数(イベントサブスクリプション数)。 - -### ConcurrencyControlAcquired {#concurrencycontrolacquired} - -確保された CPU スロット数の合計。 - -### ConcurrencyControlSoftLimit {#concurrencycontrolsoftlimit} - -CPU スロット数に対するソフトリミット値。 - -**関連項目** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 定期的に計算されるメトリクスが含まれます。 -- [system.events](/operations/system-tables/events) — 発生したイベントの数が含まれます。 -- [system.metric_log](/operations/system-tables/metric_log) — テーブル `system.metrics` と `system.events` のメトリクス値の履歴が含まれます。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse モニタリングの基本概念について説明します。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md deleted file mode 100644 index 55132bfd32a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: '進行中のデータパーツの移動に関する情報を含む MergeTree テーブルのシステムテーブル。各データパーツの移動は 1 行として表されます。' -keywords: ['システムテーブル', '移動'] -slug: /operations/system-tables/moves -title: 'system.moves' -doc_type: 'reference' ---- - -# system.moves {#systemmoves} - -このテーブルには、[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) テーブルにおける進行中の [データパート移動](/sql-reference/statements/alter/partition#move-partitionpart) に関する情報が含まれます。各データパートの移動は 1 行で表されます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.moves -``` - -```response -┌─database─┬─table─┬─────elapsed─┬─target_disk_name─┬─target_disk_path─┬─part_name─┬─part_size─┬─thread_id─┐ -│ default │ test2 │ 1.668056039 │ s3 │ ./disks/s3/ │ all_3_3_0 │ 136 │ 296146 │ -└──────────┴───────┴─────────────┴──────────────────┴──────────────────┴───────────┴───────────┴───────────┘ -``` - -**関連項目** - -* [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) テーブルエンジン -* [データストレージ向けに複数のブロックデバイスを使用する](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-multiple-volumes) -* [ALTER TABLE ... MOVE PART](/sql-reference/statements/alter/partition#move-partitionpart) コマンド diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md deleted file mode 100644 index 5a7ea876b0c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'MergeTree テーブルのミューテーションおよびその進行状況に関する情報を保持する system テーブルです。各ミューテーションコマンドは 1 行で表されます。' -keywords: ['system テーブル', 'ミューテーション'] -slug: /operations/system-tables/mutations -title: 'system.mutations' -doc_type: 'reference' ---- - -# system.mutations {#systemmutations} - -このテーブルには、[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) テーブルに対する[ミューテーション](/sql-reference/statements/alter/index.md#mutations)と、その進行状況に関する情報が含まれます。各ミューテーションコマンドは 1 行で表現されます。 - -## Columns: {#columns} - -- `database` ([String](/sql-reference/data-types/string.md)) — ミューテーションが適用されたデータベースの名前。 -- `table` ([String](/sql-reference/data-types/string.md)) — ミューテーションが適用されたテーブルの名前。 -- `mutation_id` ([String](/sql-reference/data-types/string.md)) — ミューテーションのID。レプリケートされたテーブルの場合、これらのIDはClickHouse Keeperの`/mutations/`ディレクトリ内のznode名に対応します。レプリケートされていないテーブルの場合、IDはテーブルのデータディレクトリ内のファイル名に対応します。 -- `command` ([String](/sql-reference/data-types/string.md)) — ミューテーションコマンド文字列(`ALTER TABLE [db.]table`の後のクエリ部分)。 -- `create_time` ([DateTime](/sql-reference/data-types/datetime.md)) — ミューテーションコマンドが実行のために送信された日時。 -- `block_numbers.partition_id` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — レプリケートされたテーブルのミューテーションの場合、配列にはパーティションのID(各パーティションに1つのレコード)が含まれます。レプリケートされていないテーブルのミューテーションの場合、配列は空です。 -- `block_numbers.number` ([Array](/sql-reference/data-types/array.md)([Int64](/sql-reference/data-types/int-uint.md))) — レプリケートされたテーブルのミューテーションの場合、配列には各パーティションに対して1つのレコードが含まれ、ミューテーションによって取得されたブロック番号が格納されます。パーティション内では、この番号より小さい番号のブロックを含むパートのみがミューテーションされます。レプリケートされていないテーブルでは、すべてのパーティションのブロック番号が単一のシーケンスを形成します。つまり、レプリケートされていないテーブルのミューテーションの場合、この列にはミューテーションによって取得された単一のブロック番号を持つ1つのレコードが含まれます。 -- `parts_to_do_names` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — ミューテーションを完了するためにミューテーションする必要があるデータパートの名前の配列。 -- `parts_to_do` ([Int64](/sql-reference/data-types/int-uint.md)) — ミューテーションを完了するためにミューテーションする必要があるデータパートの数。 -- `is_killed` ([UInt8](/sql-reference/data-types/int-uint.md)) — ミューテーションが強制終了されたかどうかを示します。**ClickHouse Cloudでのみ利用可能です。** - -:::note -`is_killed=1`は、必ずしもミューテーションが完全に終了したことを意味するわけではありません。`is_killed=1`かつ`is_done=0`の状態が長期間続くことがあります。これは、別の長時間実行されているミューテーションが強制終了されたミューテーションをブロックしている場合に発生する可能性があります。これは正常な状況です。 -::: - -- `is_done` ([UInt8](/sql-reference/data-types/int-uint.md)) — ミューテーションが完了しているかどうかを示すフラグ。可能な値: - - `1` ミューテーションが完了している場合、 - - `0` ミューテーションがまだ処理中の場合。 - -:::note -`parts_to_do = 0`であっても、長時間実行されている`INSERT`クエリがミューテーションする必要がある新しいデータパートを作成するため、レプリケートされたテーブルのミューテーションがまだ完了していない可能性があります。 -::: - -一部のデータパートのミューテーションに問題があった場合、以下の列に追加情報が含まれます: - -- `latest_failed_part` ([String](/sql-reference/data-types/string.md)) — ミューテーションできなかった最新のパートの名前。 -- `latest_fail_time` ([DateTime](/sql-reference/data-types/datetime.md)) — 最新のパートミューテーション失敗の日時。 -- `latest_fail_reason` ([String](/sql-reference/data-types/string.md)) — 最新のパートミューテーション失敗の原因となった例外メッセージ。 - -## ミューテーションの監視 {#monitoring-mutations} - -`system.mutations`テーブルで進行状況を追跡するには、以下のクエリを使用します: - -```sql -SELECT * FROM clusterAllReplicas('cluster_name', 'system', 'mutations') -WHERE is_done = 0 AND table = 'tmp'; - --- or - -SELECT * FROM clusterAllReplicas('cluster_name', 'system.mutations') -WHERE is_done = 0 AND table = 'tmp'; -``` - -注:これには`system.*`テーブルに対する読み取り権限が必要です。 - -:::tip Cloudでの使用 -ClickHouse Cloudでは、各ノードの`system.mutations`テーブルにクラスタ内のすべてのミューテーションが含まれているため、`clusterAllReplicas`を使用する必要はありません。 -::: - -**関連項目** - -- [ミューテーション](/sql-reference/statements/alter/index.md#mutations) -- [MergeTree](/engines/table-engines/mergetree-family/mergetree.md)テーブルエンジン -- [ReplicatedMergeTree](/engines/table-engines/mergetree-family/replication.md)ファミリー diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md deleted file mode 100644 index 60da000d4a1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: '0 から始まるほとんどの自然数を格納する、`number` という名前の単一の UInt64 型列を持つシステムテーブル。' -keywords: ['システムテーブル', 'numbers'] -slug: /operations/system-tables/numbers -title: 'system.numbers' -doc_type: 'reference' ---- - -# system.numbers {#systemnumbers} - -このテーブルには、`number` という名前の単一の UInt64 列があり、0 から始まるほぼすべての自然数が含まれます。 - -このテーブルはテスト用途や、総当たり検索を行う必要がある場合に使用できます。 - -このテーブルからの読み取りは並列化されません。 - -**例** - -```sql -SELECT * FROM system.numbers LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 rows in set. Elapsed: 0.001 sec. -``` - -出力を条件で絞り込むこともできます。 - -```sql -SELECT * FROM system.numbers < 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 rows in set. Elapsed: 0.001 sec. -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md deleted file mode 100644 index 1d865684e47..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: '`system.numbers` と同様のシステムテーブルですが、読み取りが並列化され、 - 数値が任意の順序で返されます。' -keywords: ['system table', 'numbers_mt'] -slug: /operations/system-tables/numbers_mt -title: 'system.numbers_mt' -doc_type: 'reference' ---- - -[`system.numbers`](../../operations/system-tables/numbers.md) と同様ですが、読み取りが並列化されます。数値は任意の順序で返される可能性があります。 - -テスト用に使用されます。 - -**例** - -```sql -SELECT * FROM system.numbers_mt LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 rows in set. Elapsed: 0.001 sec. -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/one.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/one.md deleted file mode 100644 index d73db5a95aa..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/one.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'UInt8 型の `dummy` 列が 1 列だけあり、その列に値 0 が格納された行が 1 行だけ存在するシステムテーブル。他の DBMS にある `DUAL` テーブルに相当します。' -keywords: ['システムテーブル', 'one'] -slug: /operations/system-tables/one -title: 'system.one' -doc_type: 'reference' ---- - -# system.one {#systemone} - -このテーブルは 1 行 1 列のテーブルで、UInt8 型の `dummy` 列に値 0 が格納されています。 - -このテーブルは、`SELECT` クエリで `FROM` 句が指定されていない場合に使用されます。 - -これは、他の DBMS に存在する `DUAL` テーブルに相当します。 - -**例** - -```sql -SELECT * FROM system.one LIMIT 10; -``` - -```response -┌─dummy─┐ -│ 0 │ -└───────┘ - -1行のデータセット。経過時間: 0.001秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md deleted file mode 100644 index 6604a7123c9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '実行されたクエリのトレーススパンに関する情報を含むシステムテーブル。' -keywords: ['システムテーブル', 'opentelemetry_span_log'] -slug: /operations/system-tables/opentelemetry_span_log -title: 'system.opentelemetry_span_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.opentelemetry_span_log {#systemopentelemetry_span_log} - - - -実行されたクエリに対する [trace span](https://opentracing.io/docs/overview/spans/) に関する情報を含みます。 - -列: - -* `trace_id` ([UUID](../../sql-reference/data-types/uuid.md)) — 実行されたクエリのトレース ID。 -* `span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` の ID。 -* `parent_span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 親 `trace span` の ID。 -* `operation_name` ([String](../../sql-reference/data-types/string.md)) — オペレーション名。 -* `kind` ([Enum8](../../sql-reference/data-types/enum.md)) — スパンの [SpanKind](https://opentelemetry.io/docs/reference/specification/trace/api/#spankind)。 - * `INTERNAL` — スパンがアプリケーション内部のオペレーションを表すことを示します。 - * `SERVER` — スパンが同期 RPC またはその他のリモートリクエストのサーバー側処理をカバーしていることを示します。 - * `CLIENT` — スパンが何らかのリモートサービスへのリクエストを表していることを示します。 - * `PRODUCER` — スパンが非同期リクエストの起点となる処理を表していることを示します。この親スパンは、対応する子の CONSUMER スパンよりも前に終了することが多く、場合によっては子スパンが開始する前に終了します。 - * `CONSUMER` — スパンが非同期 PRODUCER リクエストの子を表していることを示します。 -* `start_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` の開始時刻(マイクロ秒)。 -* `finish_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` の終了時刻(マイクロ秒)。 -* `finish_date` ([Date](../../sql-reference/data-types/date.md)) — `trace span` の終了日。 -* `attribute.names` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — `trace span` に応じた [Attribute](https://opentelemetry.io/docs/go/instrumentation/#attributes) 名。 [OpenTelemetry](https://opentelemetry.io/) 標準の推奨事項に従って設定されます。 -* `attribute.values` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — `trace span` に応じた Attribute の値。`OpenTelemetry` 標準の推奨事項に従って設定されます。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.opentelemetry_span_log LIMIT 1 FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -trace_id: cdab0847-0d62-61d5-4d38-dd65b19a1914 -span_id: 701487461015578150 -parent_span_id: 2991972114672045096 -operation_name: DB::Block DB::InterpreterSelectQuery::getSampleBlockImpl() -kind: INTERNAL -start_time_us: 1612374594529090 -finish_time_us: 1612374594529108 -finish_date: 2021-02-03 -attribute.names: [] -attribute.values: [] -``` - -**関連項目** - -* [OpenTelemetry](../../operations/opentelemetry.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md deleted file mode 100644 index 791fa6dd1d6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md +++ /dev/null @@ -1,263 +0,0 @@ ---- -description: 'システムテーブルとは何かと、その有用性の概要。' -keywords: ['システムテーブル', '概要'] -sidebar_label: '概要' -sidebar_position: 52 -slug: /operations/system-tables/overview -title: 'システムテーブルの概要' -doc_type: 'reference' ---- - - - -## システムテーブルの概要 {#system-tables-introduction} - -システムテーブルは以下の情報を提供します: - -* サーバーの状態、プロセス、および環境。 -* サーバー内部のプロセス。 -* ClickHouse バイナリのビルド時に使用されたオプション。 - -システムテーブルの特性: - -* `system` データベース内に配置されている。 -* データの読み取り専用で利用できる。 -* 削除 (`DROP`) や変更 (`ALTER`) はできないが、切り離し (`DETACH`) は可能。 - -ほとんどのシステムテーブルは、データを RAM 上に保持します。ClickHouse サーバーは起動時にこのようなシステムテーブルを作成します。 - -他のシステムテーブルとは異なり、システムログテーブル [metric_log](../../operations/system-tables/metric_log.md)、[query_log](../../operations/system-tables/query_log.md)、[query_thread_log](../../operations/system-tables/query_thread_log.md)、[trace_log](../../operations/system-tables/trace_log.md)、[part_log](../../operations/system-tables/part_log.md)、[crash_log](../../operations/system-tables/crash_log.md)、[text_log](../../operations/system-tables/text_log.md)、および [backup_log](../../operations/system-tables/backup_log.md) は [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルエンジンで動作しており、デフォルトではファイルシステムにデータを保存します。ファイルシステムからテーブルを削除した場合、ClickHouse サーバーは次回のデータ書き込み時に空のテーブルを再作成します。新しいリリースでシステムテーブルのスキーマが変更された場合、ClickHouse は現在のテーブルの名前を変更し、新しいテーブルを作成します。 - -システムログテーブルは、テーブルと同じ名前の設定ファイルを `/etc/clickhouse-server/config.d/` 配下に作成するか、`/etc/clickhouse-server/config.xml` 内の対応する要素を設定することでカスタマイズできます。カスタマイズ可能な要素は次のとおりです: - -* `database`: システムログテーブルが属するデータベース。このオプションは現在非推奨です。すべてのシステムログテーブルは `system` データベース配下にあります。 -* `table`: データを挿入するテーブル。 -* `partition_by`: [PARTITION BY](../../engines/table-engines/mergetree-family/custom-partitioning-key.md) 句の式を指定。 -* `ttl`: テーブルの [TTL](../../sql-reference/statements/alter/ttl.md) 式を指定。 -* `flush_interval_milliseconds`: データをディスクへフラッシュする間隔。 -* `engine`: パラメータ付きで、`ENGINE =` で始まる完全なエンジン式を指定。このオプションは `partition_by` および `ttl` と競合します。同時に設定した場合、サーバーは例外をスローして終了します。 - -例: - -```xml - - - system - query_log
- toYYYYMM(event_date) - event_date + INTERVAL 30 DAY DELETE - - 7500 - 1048576 - 8192 - 524288 - false -
-
-``` - -デフォルトでは、テーブルの成長に上限はありません。テーブルサイズを制御するには、古くなったログレコードを削除するための [TTL](/sql-reference/statements/alter/ttl) 設定を使用できます。また、`MergeTree` エンジンを使用するテーブルのパーティション機能を利用することもできます。 - - -## システムメトリクスの取得元 {#system-tables-sources-of-system-metrics} - -システムメトリクスを収集するために、ClickHouse サーバーは次を使用します。 - -- `CAP_NET_ADMIN` ケーパビリティ -- [procfs](https://en.wikipedia.org/wiki/Procfs)(Linux のみ) - -**procfs** - -ClickHouse サーバーが `CAP_NET_ADMIN` ケーパビリティを持たない場合、`ProcfsMetricsProvider` へのフォールバックを試みます。`ProcfsMetricsProvider` により、クエリごとのシステムメトリクス(CPU および I/O)を収集できます。 - -procfs がシステムでサポートされていて有効化されている場合、ClickHouse サーバーは次のメトリクスを収集します。 - -- `OSCPUVirtualTimeMicroseconds` -- `OSCPUWaitMicroseconds` -- `OSIOWaitMicroseconds` -- `OSReadChars` -- `OSWriteChars` -- `OSReadBytes` -- `OSWriteBytes` - -:::note -`OSIOWaitMicroseconds` は、Linux カーネル 5.14.x 以降ではデフォルトで無効になっています。 -`sudo sysctl kernel.task_delayacct=1` を実行するか、`/etc/sysctl.d/` に `kernel.task_delayacct = 1` を含む `.conf` ファイルを作成することで有効にできます。 -::: - - - -## ClickHouse Cloud における system テーブル {#system-tables-in-clickhouse-cloud} - -ClickHouse Cloud では、system テーブルは自己管理型デプロイメントの場合と同様に、サービスの状態とパフォーマンスに関する重要な洞察を提供します。いくつかの system テーブルは、特に分散メタデータを管理する Keeper ノードからデータを取得するものについては、クラスタ全体で動作します。これらのテーブルはクラスタの集合的な状態を反映しており、個々のノードでクエリした場合にも結果が整合している必要があります。たとえば、[`parts`](/operations/system-tables/parts) は、どのノードからクエリしても一貫した結果が得られる必要があります。 - -```sql -SELECT hostname(), count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-vccsrty-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.005 sec. - -SELECT - hostname(), - count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-w59bfco-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.004 sec. -``` - -一方、他の system テーブルはノード固有です。例えば、メモリ上にのみ存在するものや、MergeTree テーブルエンジンを使ってデータを永続化しているものがあります。これは、ログやメトリクスといったデータで一般的です。この永続化により、過去のデータが分析のために利用可能な状態で保持されます。しかし、これらのノード固有テーブルは、本質的に各ノードに固有です。 - -一般に、ある system テーブルがノード固有かどうかを判断する際には、次のルールを適用できます。 - -* `_log` 接尾辞を持つ system テーブル。 -* メトリクスを公開する system テーブル。例: `metrics`、`asynchronous_metrics`、`events`。 -* 実行中のプロセスを公開する system テーブル。例: `processes`、`merges`。 - -さらに、アップグレードやスキーマ変更の結果として、新しいバージョンの system テーブルが作成される場合があります。これらのバージョンは数値の接尾辞を使って命名されます。 - -例えば、`system.query_log` テーブル群を考えてみましょう。これらのテーブルには、ノードで実行された各クエリごとに 1 行が格納されます。 - -```sql -SHOW TABLES FROM system LIKE 'query_log%' - -┌─name─────────┐ -│ query_log │ -│ query_log_1 │ -│ query_log_10 │ -│ query_log_2 │ -│ query_log_3 │ -│ query_log_4 │ -│ query_log_5 │ -│ query_log_6 │ -│ query_log_7 │ -│ query_log_8 │ -│ query_log_9 │ -└──────────────┘ - -11行が返されました。経過時間: 0.004秒。 -``` - -### 複数バージョンにまたがるクエリ {#querying-multiple-versions} - -[`merge`](/sql-reference/table-functions/merge) 関数を使用すると、これらのテーブルをまたいでクエリを実行できます。たとえば、次のクエリは、各 `query_log` テーブル内で対象ノードに対して発行された最新のクエリを特定します。 - -```sql -SELECT - _table, - max(event_time) AS most_recent -FROM merge('system', '^query_log') -GROUP BY _table -ORDER BY most_recent DESC - -┌─_table───────┬─────────most_recent─┐ -│ query_log │ 2025-04-13 10:59:29 │ -│ query_log_1 │ 2025-04-09 12:34:46 │ -│ query_log_2 │ 2025-04-09 12:33:45 │ -│ query_log_3 │ 2025-04-07 17:10:34 │ -│ query_log_5 │ 2025-03-24 09:39:39 │ -│ query_log_4 │ 2025-03-24 09:38:58 │ -│ query_log_6 │ 2025-03-19 16:07:41 │ -│ query_log_7 │ 2025-03-18 17:01:07 │ -│ query_log_8 │ 2025-03-18 14:36:07 │ -│ query_log_10 │ 2025-03-18 14:01:33 │ -│ query_log_9 │ 2025-03-18 14:01:32 │ -└──────────────┴─────────────────────┘ -``` - - -11 行が返されました。経過時間: 0.373 秒。処理行数: 644 万行、25.77 MB(毎秒 1,729 万行、69.17 MB)。 -ピークメモリ使用量: 28.45 MiB。 - -```` - -:::note 順序付けに数値接尾辞を使用しないでください -テーブルの数値接尾辞はデータの順序を示唆する場合がありますが、これに依存すべきではありません。このため、特定の日付範囲を対象とする場合は、必ず日付フィルタと組み合わせてmergeテーブル関数を使用してください。 -::: - -重要な点として、これらのテーブルは依然として**各ノードにローカル**です。 - -### ノード間でのクエリ実行 {#querying-across-nodes} - -クラスタ全体を包括的に表示するには、[`clusterAllReplicas`](/sql-reference/table-functions/cluster)関数を`merge`関数と組み合わせて使用できます。`clusterAllReplicas`関数は、「default」クラスタ内のすべてのレプリカにわたってシステムテーブルをクエリし、ノード固有のデータを統合された結果に集約します。`merge`関数と組み合わせることで、クラスタ内の特定のテーブルのすべてのシステムデータを対象とすることができます。 - -このアプローチは、クラスタ全体の操作の監視とデバッグに特に有用であり、ユーザーがClickHouse Cloudデプロイメントの健全性とパフォーマンスを効果的に分析できるようにします。 - -:::note -ClickHouse Cloudは、冗長性とフェイルオーバーのために複数のレプリカのクラスタを提供します。これにより、動的な自動スケーリングやゼロダウンタイムアップグレードなどの機能が実現されます。特定の時点で、新しいノードがクラスタに追加される過程にあるか、クラスタから削除される過程にある可能性があります。これらのノードをスキップするには、以下に示すように`clusterAllReplicas`を使用するクエリに`SETTINGS skip_unavailable_shards = 1`を追加してください。 -::: - -例えば、分析に不可欠な`query_log`テーブルをクエリする際の違いを考えてみましょう。 - -```sql -SELECT - hostname() AS host, - count() -FROM system.query_log -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.010 sec. Processed 17.87 thousand rows, 71.51 KB (1.75 million rows/s., 7.01 MB/s.) - -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', system.query_log) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 656029 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 641155 │ -└───────────────────────────────┴─────────┘ - -3 rows in set. Elapsed: 0.026 sec. Processed 1.97 million rows, 7.88 MB (75.51 million rows/s., 302.05 MB/s.) -```` - -### ノードおよびバージョンをまたいだクエリ実行 {#querying-across-nodes-and-versions} - -system テーブルのバージョニングにより、これだけではクラスタ全体のデータをすべて表しているわけではありません。上記に `merge` 関数を組み合わせることで、指定した日付範囲について正確な結果を得ることができます。 - -```sql -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', merge('system', '^query_log')) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 3008000 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 3659443 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 1078287 │ -└───────────────────────────────┴─────────┘ -``` - - -3 行のセット。経過時間: 0.462 秒。処理した行数: 7.94 百万行、31.75 MB (17.17 百万行/秒、68.67 MB/秒)。 - -``` -``` - - -## 関連コンテンツ {#related-content} - -- ブログ: [システムテーブルと ClickHouse の内部構造を覗く](https://clickhouse.com/blog/clickhouse-debugging-issues-with-system-tables) -- ブログ: [必須監視クエリ - パート 1 - INSERT クエリ](https://clickhouse.com/blog/monitoring-troubleshooting-insert-queries-clickhouse) -- ブログ: [必須監視クエリ - パート 2 - SELECT クエリ](https://clickhouse.com/blog/monitoring-troubleshooting-select-queries-clickhouse) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md deleted file mode 100644 index 6c21c8ed75e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: 'MergeTree ファミリーのテーブルで、データパーツに対して行われた追加やマージなどのイベントに関する情報を格納するシステムテーブル。' -keywords: ['システムテーブル', 'part_log'] -slug: /operations/system-tables/part_log -title: 'system.part_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.part_log {#systempart_log} - - - -[part_log](/operations/server-configuration-parameters/settings#part_log) サーバー設定が指定されている場合にのみ、`system.part_log` テーブルが作成されます。 - -このテーブルには、[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) ファミリーのテーブル内の [データパーツ](../../engines/table-engines/mergetree-family/custom-partitioning-key.md) に対して発生した、追加やマージなどのイベントに関する情報が含まれます。 - -`system.part_log` テーブルには次の列が含まれます。 - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — このデータパートを作成した `INSERT` クエリの識別子。 -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — データパーツに対して発生したイベントの種類。次のいずれかの値を取ります。 - * `NewPart` — 新しいデータパーツの挿入処理。 - * `MergePartsStart` — データパーツのマージ開始。 - * `MergeParts` — データパーツのマージ完了。 - * `DownloadPart` — データパーツのダウンロード処理。 - * `RemovePart` — [DETACH PARTITION](/sql-reference/statements/alter/partition#detach-partitionpart) を使用したデータパーツの削除またはデタッチ。 - * `MutatePartStart` — データパーツのミューテーション開始。 - * `MutatePart` — データパーツのミューテーション完了。 - * `MovePart` — データパーツをあるディスクから別のディスクへ移動する処理。 -* `merge_reason` ([Enum8](../../sql-reference/data-types/enum.md)) — 型 `MERGE_PARTS` のイベントが発生した理由。次のいずれかの値を取ります:` - * `NotAMerge` — 現在のイベントタイプが `MERGE_PARTS` 以外の場合。 - * `RegularMerge` — 通常のマージ。 - * `TTLDeleteMerge` — 有効期限切れデータの削除。 - * `TTLRecompressMerge` — データパーツの再圧縮。 -* `merge_algorithm` ([Enum8](../../sql-reference/data-types/enum.md)) — タイプが `MERGE_PARTS` のイベントに対して使用されるマージアルゴリズム。取り得る値は次のいずれかです: - * `未決定` - * `水平` - * `垂直` -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの発生時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度のイベントの時刻。 -* `duration_ms` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 継続時間。 -* `database` ([String](../../sql-reference/data-types/string.md)) — データパーツが属するデータベースの名前。 -* `table` ([String](../../sql-reference/data-types/string.md)) — データパーツが含まれているテーブルの名前。 -* `table_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — データパートが属するテーブルのUUID。 -* `part_name` ([String](../../sql-reference/data-types/string.md)) — データパーツ名。 -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — データパートが挿入されたパーティションの ID。パーティション分割が `tuple()` によって行われている場合、この列は `all` の値を取ります。 -* `partition` ([String](../../sql-reference/data-types/string.md)) - パーティションの名前。 -* `part_type` ([String](../../sql-reference/data-types/string.md)) - パーツのタイプ。取り得る値は Wide と Compact です。 -* `disk_name` ([String](../../sql-reference/data-types/string.md)) - データパーツが配置されているディスク名。 -* `path_on_disk` ([String](../../sql-reference/data-types/string.md)) — データパーツファイルが格納されているフォルダへの絶対パス。 -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — データパートに含まれる行数です。 -* `size_in_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — データパートのサイズ(バイト単位)。 -* `merged_from` ([Array(String)](../../sql-reference/data-types/array.md)) — 現在のパーツが(マージ後に)作成される際に元となったパーツ名の配列。 -* `bytes_uncompressed` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 非圧縮データのバイト数。 -* `read_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マージ中に読み込まれた行数。 -* `read_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マージ中に読み取られたバイト数。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — このスレッドのコンテキストにおいて、割り当てられたメモリ量と解放されたメモリ量の差の最大値。 -* `error` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 発生したエラーのエラーコード。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 発生したエラーの内容を示すテキストメッセージ。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/map.md)) — 各種メトリクスを計測する `ProfileEvents`。それぞれの説明はテーブル [system.events](/operations/system-tables/events) に記載されています。 - -`system.part_log` テーブルは、`MergeTree` テーブルにデータを初めて挿入した後に作成されます。 - -**例** - -```sql -SELECT * FROM system.part_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -query_id: -event_type: MergeParts -merge_reason: RegularMerge -merge_algorithm: Vertical -event_date: 2025-07-19 -event_time: 2025-07-19 23:54:19 -event_time_microseconds: 2025-07-19 23:54:19.710761 -duration_ms: 2158 -database: default -table: github_events -table_uuid: 1ad33424-f5f5-402b-ac03-ec82282634ab -part_name: all_1_7_1 -partition_id: all -partition: tuple() -part_type: Wide -disk_name: default -path_on_disk: ./data/store/1ad/1ad33424-f5f5-402b-ac03-ec82282634ab/all_1_7_1/ -rows: 3285726 -- 329万 -size_in_bytes: 438968542 -- 4億3897万 -merged_from: ['all_1_1_0','all_2_2_0','all_3_3_0','all_4_4_0','all_5_5_0','all_6_6_0','all_7_7_0'] -bytes_uncompressed: 1373137767 -- 13億7000万 -read_rows: 3285726 -- 329万 -read_bytes: 1429206946 -- 14億3000万 -peak_memory_usage: 303611887 -- 3億361万 -error: 0 -exception: -ProfileEvents: {'FileOpen':703,'ReadBufferFromFileDescriptorRead':3824,'ReadBufferFromFileDescriptorReadBytes':439601681,'WriteBufferFromFileDescriptorWrite':592,'WriteBufferFromFileDescriptorWriteBytes':438988500,'ReadCompressedBytes':439601681,'CompressedReadBufferBlocks':6314,'CompressedReadBufferBytes':1539835748,'OpenedFileCacheHits':50,'OpenedFileCacheMisses':484,'OpenedFileCacheMicroseconds':222,'IOBufferAllocs':1914,'IOBufferAllocBytes':319810140,'ArenaAllocChunks':8,'ArenaAllocBytes':131072,'MarkCacheMisses':7,'CreatedReadBufferOrdinary':534,'DiskReadElapsedMicroseconds':139058,'DiskWriteElapsedMicroseconds':51639,'AnalyzePatchRangesMicroseconds':28,'ExternalProcessingFilesTotal':1,'RowsReadByMainReader':170857759,'WaitMarksLoadMicroseconds':988,'LoadedMarksFiles':7,'LoadedMarksCount':14,'LoadedMarksMemoryBytes':728,'Merge':2,'MergeSourceParts':14,'MergedRows':3285733,'MergedColumns':4,'GatheredColumns':51,'MergedUncompressedBytes':1429207058,'MergeTotalMilliseconds':2158,'MergeExecuteMilliseconds':2155,'MergeHorizontalStageTotalMilliseconds':145,'MergeHorizontalStageExecuteMilliseconds':145,'MergeVerticalStageTotalMilliseconds':2008,'MergeVerticalStageExecuteMilliseconds':2006,'MergeProjectionStageTotalMilliseconds':5,'MergeProjectionStageExecuteMilliseconds':4,'MergingSortedMilliseconds':7,'GatheringColumnMilliseconds':56,'ContextLock':2091,'PartsLockHoldMicroseconds':77,'PartsLockWaitMicroseconds':1,'RealTimeMicroseconds':2157475,'CannotWriteToWriteBufferDiscard':36,'LogTrace':6,'LogDebug':59,'LoggerElapsedNanoseconds':514040,'ConcurrencyControlSlotsGranted':53,'ConcurrencyControlSlotsAcquired':53} -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md deleted file mode 100644 index 03e3e82411c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: 'MergeTree のパーツに関する情報を格納するシステムテーブル' -keywords: ['システムテーブル', 'パーツ'] -slug: /operations/system-tables/parts -title: 'system.parts' -doc_type: 'reference' ---- - -# system.parts {#systemparts} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルのパーツに関する情報を格納します。 - -各行は 1 つのデータパーツを表します。 - -カラム: - -* `partition` ([String](../../sql-reference/data-types/string.md)) – パーティション名。パーティションとは何かについては、[ALTER](/sql-reference/statements/alter) クエリの説明を参照してください。 - - 形式: - - * 月ごとの自動パーティションの場合は `YYYYMM`。 - * 手動パーティションの場合は `any_string`。 - -* `name` ([String](../../sql-reference/data-types/string.md)) – データパーツの名前。パーツ名の構造から、データ、取り込み、およびマージパターンに関する多くの側面を把握できます。パーツ名の形式は次のとおりです。 - -```text -____ -``` - -* 定義: - * `partition_id` - パーティションキーを識別します - * `minimum_block_number` - パーツ内の最小ブロック番号を識別します。ClickHouse は常に連続したブロックをマージします - * `maximum_block_number` - パーツ内の最大ブロック番号を識別します - * `level` - パーツに対するマージが 1 回行われるたびに 1 ずつ増加します。レベル 0 は、まだマージされていない新しいパーツであることを示します。ClickHouse のすべてのパーツは常に不変(イミュータブル)であることを理解しておくことが重要です - * `data_version` - オプションの値で、パーツに対してミューテーションが実行されると増加します(繰り返しになりますが、パーツは不変であるため、ミューテーションされたデータは常に新しいパーツにのみ書き込まれます) - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) - データパーツのUUID。 - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — データパーツの格納形式。 - - 指定可能な値: - - * `Wide` — 各列がファイルシステム内の別々のファイルに保存されます。 - * `Compact` — すべての列がファイルシステム内の1つのファイルに保存されます。 - - データの保存形式は、[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルの `min_bytes_for_wide_part` および `min_rows_for_wide_part` の設定によって制御されます。 - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) – データパーツがアクティブかどうかを示すフラグ。データパーツがアクティブな場合、そのパーツはテーブルで利用される。そうでない場合は削除される。非アクティブなデータパーツはマージ処理後も残る。 - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) – マーク数。データパート内のおおよその行数を求めるには、`marks` にインデックス粒度(通常は 8192)を掛けます(この目安はアダプティブ粒度を使用している場合には当てはまりません)。 - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 行数。 - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) – すべてのデータパーツファイルのサイズ合計(バイト単位)。 - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – データパート内の圧縮済みデータの合計サイズ。すべての補助ファイル(たとえばマークファイル)は含まれません。 - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – データパート内の非圧縮データの合計サイズ。すべての補助ファイル(たとえばマークファイル)は含まれません。 - -* `primary_key_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – ディスク上の primary.idx/cidx ファイルにおいて、プライマリキー値によって消費されているメモリ量(バイト数)。 - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – マークを含むファイルのサイズ。 - -* `secondary_indices_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – データパーツ内のセカンダリインデックスに対する圧縮データの合計サイズ。マークファイルなどの補助ファイルは含まれません。 - -* `secondary_indices_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – データパーツ内のセカンダリインデックスに対する非圧縮データの合計サイズです。すべての補助ファイル(たとえばマークのファイルなど)は含まれません。 - -* `secondary_indices_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – セカンダリインデックスのマークを含むファイルのサイズ。 - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – データパーツのディレクトリが更新された時刻。通常はデータパーツの作成時刻に対応します。 - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – データパーツが非アクティブになった時刻。 - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) – データパーツが使用されている箇所の数。値が 2 を超える場合、そのデータパーツがクエリまたはマージで使用されていることを示します。 - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) – データパートにおける日付キーの最小値。 - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) – データパーツ内における日付キーの最大値です。 - -* `min_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – データパート内の日付と時刻キーの最小値。 - -* `max_time`([DateTime](../../sql-reference/data-types/datetime.md)) – データパーツ内の日付時刻キーの最大値。 - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) – パーティション ID。 - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – マージ後に形成される現在のパートを構成するデータブロック番号の最小値。 - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – マージ後の現在のパーツを構成するデータブロック番号の最大値。 - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) – マージツリーの深さ。0 の場合、現在のパートは他のパートのマージではなく、挿入によって作成されたことを意味します。 - -* `data_version` ([UInt64](../../sql-reference/data-types/int-uint.md)) – データパーツに対してどのミューテーションを適用すべきかを判定するために使用される数値(`data_version` より大きいバージョンを持つミューテーションが適用される)。 - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) – プライマリキーの値によって使用されているメモリ量(バイト単位)。`primary_key_lazy_load=1` かつ `use_primary_key_cache=1` の場合は `0` になります。 - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) – プライマリキー値のために確保されているメモリ容量(バイト単位)。`primary_key_lazy_load=1` かつ `use_primary_key_cache=1` の場合は `0` になります。 - -* `is_frozen` ([UInt8](../../sql-reference/data-types/int-uint.md)) – パーティションデータのバックアップが存在するかを示すフラグ。1 の場合はバックアップが存在し、0 の場合はバックアップが存在しません。詳細は [FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) を参照してください。 - -* `database` ([String](../../sql-reference/data-types/string.md)) – データベースの名前。 - -* `table` ([String](../../sql-reference/data-types/string.md)) – テーブルの名前。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) – パラメータなしのテーブルエンジン名。 - -* `path` ([String](../../sql-reference/data-types/string.md)) – データパーツのファイルが格納されているフォルダへの絶対パス。 - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) – データパーツが保存されているディスクの名前。 - -* `hash_of_all_files` ([String](../../sql-reference/data-types/string.md)) – 圧縮ファイルの [sipHash128](/sql-reference/functions/hash-functions#sipHash128) 値。 - -* `hash_of_uncompressed_files` ([String](../../sql-reference/data-types/string.md)) – 非圧縮ファイル(マークファイル、インデックスファイルなど)に対する [sipHash128](/sql-reference/functions/hash-functions#sipHash128) のハッシュ値。 - -* `uncompressed_hash_of_compressed_files` ([String](../../sql-reference/data-types/string.md)) – 圧縮ファイル内のデータを、非圧縮データとして扱った場合の [sipHash128](/sql-reference/functions/hash-functions#sipHash128)。 - -* `delete_ttl_info_min` ([DateTime](../../sql-reference/data-types/datetime.md)) — [TTL DELETE ルール](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) における日付時刻キーの最小値。 - -* `delete_ttl_info_max` ([DateTime](../../sql-reference/data-types/datetime.md)) — [TTL DELETE ルール](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) における日付と時刻キーの最大値です。 - -* `move_ttl_info.expression` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — ルールを定義する式の配列です。各式は [TTL MOVE ルール](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) を定義します。 - -:::note -`move_ttl_info.expression` 配列は主に後方互換性のために残されています。現在では、`TTL MOVE` ルールを確認する最も簡単な方法は、`move_ttl_info.min` および `move_ttl_info.max` フィールドを使用することです。 -::: - -* `move_ttl_info.min` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — 日付と時刻の値の配列。各要素は、[TTL MOVE ルール](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)の最小キー値を表します。 - -* `move_ttl_info.max` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — 日付と時刻の値の配列。各要素は、[TTL MOVE ルール](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)の最大キー値を表します。 - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – `bytes_on_disk` のエイリアス。 - -* `marks_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – `marks_bytes` のエイリアス。 - -**例** - -```sql -SELECT * FROM system.parts LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_4_1_6 -part_type: Wide -active: 1 -marks: 2 -rows: 6 -bytes_on_disk: 310 -data_compressed_bytes: 157 -data_uncompressed_bytes: 91 -secondary_indices_compressed_bytes: 58 -secondary_indices_uncompressed_bytes: 6 -secondary_indices_marks_bytes: 48 -marks_bytes: 144 -modification_time: 2020-06-18 13:01:49 -remove_time: 1970-01-01 00:00:00 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -min_time: 1970-01-01 00:00:00 -max_time: 1970-01-01 00:00:00 -partition_id: all -min_block_number: 1 -max_block_number: 4 -level: 1 -data_version: 6 -primary_key_bytes_in_memory: 8 -primary_key_bytes_in_memory_allocated: 64 -is_frozen: 0 -database: default -table: months -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/months/all_1_4_1_6/ -hash_of_all_files: 2d0657a16d9430824d35e327fcbd87bf -hash_of_uncompressed_files: 84950cc30ba867c77a408ae21332ba29 -uncompressed_hash_of_compressed_files: 1ad78f1c6843bbfb99a2c931abe7df7d -delete_ttl_info_min: 1970-01-01 00:00:00 -delete_ttl_info_max: 1970-01-01 00:00:00 -move_ttl_info.expression: [] -move_ttl_info.min: [] -move_ttl_info.max: [] -``` - -**関連項目** - -* [MergeTree ファミリー](../../engines/table-engines/mergetree-family/mergetree.md) -* [列およびテーブルの TTL](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md deleted file mode 100644 index 18239e45d34..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -description: 'MergeTree テーブルのパーツおよび列に関する情報を保持する system テーブル。' -keywords: ['system テーブル', 'parts_columns'] -slug: /operations/system-tables/parts_columns -title: 'system.parts_columns' -doc_type: 'reference' ---- - -# system.parts_columns {#systemparts_columns} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルのパーツおよび列に関する情報を含んでいます。 - -各行は1つのデータパーツを表します。 - -列: - -* `partition` ([String](../../sql-reference/data-types/string.md)) — パーティション名。パーティションについては、[ALTER](/sql-reference/statements/alter) クエリの説明を参照してください。 - - フォーマット: - - * 月ごとの自動パーティション作成には `YYYYMM` を使用します。 - * 手動でパーティションを指定する場合は `any_string` を使用します。 - -* `name` ([String](../../sql-reference/data-types/string.md)) — データパーツの名称。 - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — データパーツの格納形式。 - - 指定可能な値: - - * `Wide` — 各列はファイルシステム内の個別のファイルに保存されます。 - * `Compact` — すべての列はファイルシステム内の1つのファイルに保存されます。 - - データの格納形式は、[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルの `min_bytes_for_wide_part` および `min_rows_for_wide_part` 設定によって制御されます。 - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) — データパーツがアクティブかどうかを示すフラグ。データパーツがアクティブな場合、そのパーツはテーブルで使用される。それ以外の場合は削除される。マージ後も非アクティブなデータパーツは残る。 - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マーク数。データパート内のおおよその行数を求めるには、`marks` にインデックスの粒度(通常は 8192)を掛けます(このヒントはアダプティブ粒度には使用できません)。 - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 行数。 - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — すべてのデータパーツのファイルサイズの合計(バイト単位)。 - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — データパーツ内の圧縮済みデータの合計サイズ。マークファイルなどの補助ファイルは含まれません。 - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — データパーツ内の非圧縮データの合計サイズ。すべての補助ファイル(たとえばマークを含むファイルなど)は含まれません。 - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マークを格納するファイルのサイズ。 - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — データパーツを含むディレクトリが更新された時刻。通常はデータパーツが作成された時刻に対応します。 - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — データパートが非アクティブになった時刻。 - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) — データパーツが使用されている場所の数。値が 2 より大きい場合、そのデータパーツがクエリまたはマージで使用されていることを示します。 - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) — データパートに含まれる日付キーの最小値。 - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) — データパート内の日付キーの最大値。 - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — パーティションのID。 - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マージ後の現在のパーツを構成するデータパーツの最小番号。 - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マージ後に現在のパーツを構成するデータパーツ数の最大値。 - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) — マージツリー内での深さ。0 は、現在のパーツが他のパーツをマージしてではなく、挿入によって作成されたことを意味します。 - -* `data_version` ([UInt64](../../sql-reference/data-types/int-uint.md)) — データパートにどのミューテーションを適用すべきかを判定するために使用される数値(`data_version` より大きいバージョンを持つミューテーションが適用される)。 - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プライマリキー値により使用されているメモリ量(バイト単位)。 - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プライマリキー値用に確保されているメモリ量(バイト単位)。 - -* `database` ([String](../../sql-reference/data-types/string.md)) — データベースの名前。 - -* `table` ([String](../../sql-reference/data-types/string.md)) — テーブルの名前。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) — パラメータを含まないテーブルエンジン名。 - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) — データパーツを格納しているディスクの名前。 - -* `path` ([String](../../sql-reference/data-types/string.md)) — データパートのファイルが格納されているフォルダへの絶対パス。 - -* `column` ([String](../../sql-reference/data-types/string.md)) — カラム名。 - -* `type` ([String](../../sql-reference/data-types/string.md)) — 列のデータ型。 - -* `column_position` ([UInt64](../../sql-reference/data-types/int-uint.md)) — テーブル内の列の位置(1 から開始)。 - -* `default_kind` ([String](../../sql-reference/data-types/string.md)) — デフォルト値に対する式タイプ(`DEFAULT`、`MATERIALIZED`、`ALIAS`)。定義されていない場合は空文字列となります。 - -* `default_expression` ([String](../../sql-reference/data-types/string.md)) — デフォルト値の式。定義されていない場合は空文字列。 - -* `column_bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — カラムの合計サイズ(バイト数)。 - -* `column_data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列内の圧縮されたデータの合計サイズ(バイト単位)。 - -* `column_data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列内の非圧縮データの合計サイズ(バイト単位)。 - -* `column_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — マークを含むカラムのサイズ(バイト単位)。 - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `bytes_on_disk` のエイリアス。 - -* `marks_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `marks_bytes` のエイリアスです。 - -**例** - -```sql -SELECT * FROM system.parts_columns LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_2_1 -part_type: Wide -active: 1 -marks: 2 -rows: 2 -bytes_on_disk: 155 -data_compressed_bytes: 56 -data_uncompressed_bytes: 4 -marks_bytes: 96 -modification_time: 2020-09-23 10:13:36 -remove_time: 2106-02-07 06:28:15 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -partition_id: all -min_block_number: 1 -max_block_number: 2 -level: 1 -data_version: 1 -primary_key_bytes_in_memory: 2 -primary_key_bytes_in_memory_allocated: 64 -database: default -table: 53r93yleapyears -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/53r93yleapyears/all_1_2_1/ -column: id -type: Int8 -column_position: 1 -default_kind: -default_expression: -column_bytes_on_disk: 76 -column_data_compressed_bytes: 28 -column_data_uncompressed_bytes: 2 -column_marks_bytes: 48 -``` - -**関連項目** - -* [MergeTree ファミリー](../../engines/table-engines/mergetree-family/mergetree.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md deleted file mode 100644 index 6aa5742aafd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: '`SHOW PROCESSLIST` クエリの実装に使用されるシステムテーブル。' -keywords: ['システムテーブル', 'プロセス'] -slug: /operations/system-tables/processes -title: 'system.processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processes {#systemprocesses} - - - -このシステムテーブルは、`SHOW PROCESSLIST` クエリを実装するために使用されます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.processes LIMIT 10 FORMAT Vertical; -``` - -```response -行 1: -────── -is_initial_query: 1 -user: default -query_id: 35a360fa-3743-441d-8e1f-228c938268da -address: ::ffff:172.23.0.1 -port: 47588 -initial_user: default -initial_query_id: 35a360fa-3743-441d-8e1f-228c938268da -initial_address: ::ffff:172.23.0.1 -initial_port: 47588 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -elapsed: 0.000582537 -is_cancelled: 0 -is_all_data_sent: 0 -read_rows: 0 -read_bytes: 0 -total_rows_approx: 0 -written_rows: 0 -written_bytes: 0 -memory_usage: 0 -peak_memory_usage: 0 -query: SELECT * from system.processes LIMIT 10 FORMAT Vertical; -thread_ids: [67] -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -Settings: {'background_pool_size':'32','load_balancing':'random','allow_suspicious_low_cardinality_types':'1','distributed_aggregation_memory_efficient':'1','skip_unavailable_shards':'1','log_queries':'1','max_bytes_before_external_group_by':'20000000000','max_bytes_before_external_sort':'20000000000','allow_introspection_functions':'1'} - -1行を取得しました。経過時間: 0.002秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md deleted file mode 100644 index 6e200d2c2d6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'プロセッサーレベルのプロファイリング情報を含むシステムテーブル(`EXPLAIN PIPELINE` の結果で確認可能)' -keywords: ['system table', 'processors_profile_log', 'EXPLAIN PIPELINE'] -slug: /operations/system-tables/processors_profile_log -title: 'system.processors_profile_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processors_profile_log {#systemprocessors_profile_log} - - - -このテーブルには、プロセッサーレベルのプロファイリング情報が含まれます([`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline) で確認できます)。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントが発生した日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントが発生した日時。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — イベントが発生した日時(マイクロ秒精度)。 -* `id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プロセッサーの ID。 -* `parent_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 親プロセッサーの ID の配列。 -* `plan_step` ([UInt64](../../sql-reference/data-types/int-uint.md)) — このプロセッサーを作成したクエリプランステップの ID。プロセッサーがどのステップからも追加されていない場合、この値は 0。 -* `plan_group` ([UInt64](../../sql-reference/data-types/int-uint.md)) — クエリプランステップによって作成された場合のプロセッサーのグループ。同一のクエリプランステップから追加されたプロセッサーを論理的にグループ化するためのものです。グループは `EXPLAIN PIPELINE` の結果を見やすくする目的にのみ使用されます。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初期クエリの ID(分散クエリ実行用)。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — クエリの ID。 -* `name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — プロセッサー名。 -* `elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — このプロセッサーが実行されていた時間(マイクロ秒単位)。 -* `input_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — このプロセッサーが(別のプロセッサーからの)データを待機していた時間(マイクロ秒単位)。 -* `output_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 出力ポートがフルだったためにこのプロセッサーが待機していた時間(マイクロ秒単位)。 -* `input_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プロセッサーが消費した行数。 -* `input_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プロセッサーが消費したバイト数。 -* `output_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プロセッサーが生成した行数。 -* `output_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — プロセッサーが生成したバイト数。 - -**例** - -クエリ: - -```sql -EXPLAIN PIPELINE -SELECT sleep(1) -┌─explain─────────────────────────┐ -│ (Expression) │ -│ ExpressionTransform │ -│ (SettingQuotaAndLimits) │ -│ (ReadFromStorage) │ -│ SourceFromSingleChunk 0 → 1 │ -└─────────────────────────────────┘ - -SELECT sleep(1) -SETTINGS log_processors_profiles = 1 -Query id: feb5ed16-1c24-4227-aa54-78c02b3b27d4 -┌─sleep(1)─┐ -│ 0 │ -└──────────┘ -1 rows in set. Elapsed: 1.018 sec. - -SELECT - name, - elapsed_us, - input_wait_elapsed_us, - output_wait_elapsed_us -FROM system.processors_profile_log -WHERE query_id = 'feb5ed16-1c24-4227-aa54-78c02b3b27d4' -ORDER BY name ASC -``` - -結果: - -```text -┌─name────────────────────┬─elapsed_us─┬─input_wait_elapsed_us─┬─output_wait_elapsed_us─┐ -│ ExpressionTransform │ 1000497 │ 2823 │ 197 │ -│ LazyOutputFormat │ 36 │ 1002188 │ 0 │ -│ LimitsCheckingTransform │ 10 │ 1002994 │ 106 │ -│ NullSource │ 5 │ 1002074 │ 0 │ -│ NullSource │ 1 │ 1002084 │ 0 │ -│ SourceFromSingleChunk │ 45 │ 4736 │ 1000819 │ -└─────────────────────────┴────────────┴───────────────────────┴────────────────────────┘ -``` - -ここでは次のことがわかります: - -* `ExpressionTransform` は `sleep(1)` 関数を実行していたため、その `work` に 1e6 us がかかり、その結果 `elapsed_us` > 1e6 となります。 -* `SourceFromSingleChunk` は待機する必要があります。これは、`ExpressionTransform` が `sleep(1)` の実行中はデータを一切受け付けないためで、その 1e6 us のあいだ `PortFull` 状態となり、結果として `output_wait_elapsed_us` > 1e6 となります。 -* `LimitsCheckingTransform`/`NullSource`/`LazyOutputFormat` は、結果を処理するために `ExpressionTransform` が `sleep(1)` の実行を完了するまで待機する必要があるため、`input_wait_elapsed_us` > 1e6 となります。 - -**関連項目** - -* [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md deleted file mode 100644 index 82e85f8e5d6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'MergeTree ファミリーのテーブル用のプロジェクションパーツに関する情報を格納するシステムテーブル。' -keywords: ['システムテーブル', 'projection_parts'] -slug: /operations/system-tables/projection_parts -title: 'system.projection_parts' -doc_type: 'reference' ---- - -# system.projection_parts {#systemprojection_parts} - -このテーブルには、MergeTree ファミリーのテーブルにおけるプロジェクションパーツに関する情報が格納されています。 - -## 列 {#columns} - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md deleted file mode 100644 index e72d6c095fe..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'MergeTree ファミリーのテーブルにおける projection パーツのカラム情報を含むシステムテーブル' -keywords: ['システムテーブル', 'projection_parts_columns'] -slug: /operations/system-tables/projection_parts_columns -title: 'system.projection_parts_columns' -doc_type: 'reference' ---- - -# system.projection_parts_columns {#systemprojection_parts_columns} - -このテーブルには、MergeTree ファミリーのテーブルにおけるプロジェクションパーツ内の列に関する情報が格納されています。 - -## カラム {#columns} - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md deleted file mode 100644 index adb5d1b1dd3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: 'すべてのテーブルに存在するプロジェクションに関する情報を含むシステムテーブル。' -keywords: ['システムテーブル', 'プロジェクション'] -slug: /operations/system-tables/projections -title: 'system.projections' -doc_type: 'reference' ---- - -# system.projections {#systemprojections} - -すべてのテーブルに存在するプロジェクションに関する情報を含みます。 - -カラム: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.projections LIMIT 2 FORMAT Vertical; -``` - -```text -行 1: -────── -database: default -table: landing -name: improved_sorting_key -type: Normal -sorting_key: ['user_id','date'] -query: SELECT * ORDER BY user_id, date - -行 2: -────── -database: default -table: landing -name: agg_no_key -type: Aggregate -sorting_key: [] -query: SELECT count() -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md deleted file mode 100644 index cb7470037c0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: 'クエリキャッシュの内容を表示するシステムテーブル。' -keywords: ['system table', 'query_cache'] -slug: /operations/system-tables/query_cache -title: 'system.query_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_cache {#systemquery_cache} - - - -[クエリキャッシュ](../query-cache.md) の内容を表示します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.query_cache FORMAT Vertical; -``` - -```text -Row 1: -────── -query: SELECT 1 SETTINGS use_query_cache = 1 -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -result_size: 128 -tag: -stale: 0 -shared: 0 -compressed: 1 -expires_at: 2023-10-13 13:35:45 -key_hash: 12188185624808016954 - -1行取得。経過時間: 0.004秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md deleted file mode 100644 index 1b3ea1f398f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'クエリ条件キャッシュの内容を表示するシステムテーブル。' -keywords: ['system table', 'query_condition_cache'] -slug: /operations/system-tables/query_condition_cache -title: 'system.query_condition_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_condition_cache {#systemquery_condition_cache} - - - -[クエリ条件キャッシュ](../query-condition-cache.md) の内容を表示します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.query_condition_cache FORMAT Vertical; -``` - -```text -行 1: -────── -table_uuid: 28270a24-ea27-49f6-99cd-97b9bee976ac -part_name: all_1_1_0 -condition: or(equals(b, 10000_UInt16), equals(c, 10000_UInt16)) -condition_hash: 5456494897146899690 -- 5.46 quintillion -entry_size: 40 -matching_marks: 111111110000000000000000000000000000000000000000000000000111111110000000000000000 - -1行のセット。経過時間: 0.004秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md deleted file mode 100644 index 107740c05c5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -description: '実行されたクエリに関する情報を含むシステムテーブルです。たとえば、開始時刻、処理時間、エラーメッセージなどが含まれます。' -keywords: ['システムテーブル', 'query_log'] -slug: /operations/system-tables/query_log -title: 'system.query_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.query_log {#systemquery_log} - - - -実行されたクエリのメタデータおよび統計情報(開始時刻、実行時間、エラーメッセージ、リソース使用量、その他の実行詳細など)を保存します。クエリ結果自体は保存しません。 - -クエリのログ記録に関する設定は、サーバー設定の [query_log](../../operations/server-configuration-parameters/settings.md#query_log) セクションで変更できます。 - -[log_queries = 0](/operations/settings/settings#log_queries) を設定することで、クエリのログ記録を無効にできます。ただし、このテーブルに含まれる情報は問題の解決に重要であるため、ログを無効にすることは推奨しません。 - -データのフラッシュ間隔は、サーバー設定の [query_log](../../operations/server-configuration-parameters/settings.md#query_log) セクションにある `flush_interval_milliseconds` パラメータで設定します。フラッシュを即時に実行するには、[SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) クエリを使用します。 - -ClickHouse はこのテーブルからデータを自動的に削除しません。詳細については [Introduction](/operations/system-tables/overview#system-tables-introduction) を参照してください。 - -`system.query_log` テーブルには、次の 2 種類のクエリが記録されます。 - -1. クライアントによって直接実行された初期クエリ。 -2. (分散クエリ実行のために)他のクエリによって開始された子クエリ。この種類のクエリについては、親クエリに関する情報が `initial_*` 列に表示されます。 - -各クエリは、そのステータス(`type` 列を参照)に応じて、`query_log` テーブルに 1 行または 2 行が作成されます。 - -1. クエリが正常に実行された場合、`QueryStart` と `QueryFinish` タイプの 2 行が作成されます。 -2. クエリ処理中にエラーが発生した場合、`QueryStart` と `ExceptionWhileProcessing` タイプの 2 つのイベントが作成されます。 -3. クエリを開始する前にエラーが発生した場合、`ExceptionBeforeStart` タイプの単一のイベントが作成されます。 - -[log_queries_probability](/operations/settings/settings#log_queries_probability) 設定を使用して、`query_log` テーブルに記録されるクエリ数を減らすことができます。 - -[log_formatted_queries](/operations/settings/settings#log_formatted_queries) 設定を使用して、フォーマット済みクエリを `formatted_query` 列に記録できます。 - - - -## カラム {#columns} - - - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — クエリの実行時に発生したイベントの種別。値: - * `'QueryStart' = 1` — クエリ実行が正常に開始されたことを示します。 - * `'QueryFinish' = 2` — クエリ実行が正常に終了したことを示します。 - * `'ExceptionBeforeStart' = 3` — クエリ実行の開始前に例外が発生したことを示します。 - * `'ExceptionWhileProcessing' = 4` — クエリ実行中に例外が発生したことを示します。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — クエリの開始日。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — クエリの開始時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度のクエリの開始時刻。 -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — クエリ実行の開始時刻。 -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度でのクエリ実行開始時刻。 -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリの実行時間(ミリ秒単位)。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリに参加したすべてのテーブルおよびテーブル関数から読み取られた行の総数です。通常のサブクエリに加え、`IN` や `JOIN` 用のサブクエリも含まれます。分散クエリの場合、`read_rows` にはすべてのレプリカで読み取られた行の総数が含まれます。各レプリカは自身の `read_rows` の値を送信し、クエリのイニシエータであるサーバーが、受信した値とローカルの値を合計します。キャッシュ量はこの値に影響しません。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリに関与したすべてのテーブルおよびテーブル関数から読み取られたバイト数の合計です。通常のサブクエリに加え、`IN` および `JOIN` のサブクエリも含まれます。分散クエリの場合、`read_bytes` にはすべてのレプリカで読み取られたバイト数の合計が含まれます。各レプリカはそれぞれの `read_bytes` の値を送信し、クエリのイニシエータであるサーバーが、受信した値とローカルの値を合算します。キャッシュの使用量はこの値に影響しません。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `INSERT` クエリでは書き込まれた行数です。それ以外のクエリでは、この列の値は 0 です。 -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `INSERT` クエリでは書き込まれたバイト数(非圧縮)を表します。それ以外のクエリでは、この列の値は 0 です。 -* `result_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `SELECT` クエリ結果の行数、または `INSERT` クエリで挿入される行数。 -* `result_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリ結果の保存に使用される RAM の容量(バイト単位)。 -* `memory_usage` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリによって消費されたメモリ量。 -* `current_database` ([String](../../sql-reference/data-types/string.md)) — 現在のデータベースの名前。 -* `query` ([String](../../sql-reference/data-types/string.md)) — クエリ文字列。 -* `formatted_query` ([String](../../sql-reference/data-types/string.md)) — 整形済みのクエリ文字列。 -* `normalized_query_hash` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — リテラルの値だけが異なるクエリであれば同一になるように計算された数値ハッシュ。 -* `query_kind` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — クエリの種別。 -* `databases` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — クエリ内で参照されているデータベース名。 -* `tables` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — クエリ内に含まれるテーブルの名前。 -* `columns` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — クエリ内に含まれる列名。 -* `partitions` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — クエリに含まれるパーティション名。 -* `projections` ([String](../../sql-reference/data-types/string.md)) — クエリ実行時に使用されるプロジェクション名。 -* `views` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — クエリに含まれる(マテリアライズドビューまたはライブビュー)の名前。 -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — 例外コード。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 例外メッセージ。 -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [スタックトレース](https://en.wikipedia.org/wiki/Stack_trace)。クエリが正常に完了した場合は空文字列となります。 -* `is_initial_query` ([UInt8](../../sql-reference/data-types/int-uint.md)) — クエリ種別。取りうる値: - * 1 — クエリがクライアントによって開始された。 - * 0 — クエリが、分散クエリ実行の一部として別のクエリから開始された。 -* `user` ([String](../../sql-reference/data-types/string.md)) — 現在のクエリを実行したユーザー名。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — クエリ ID。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — クエリの実行に使用されたIPアドレス。 -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — クエリの送信に使用されたクライアントポート。 -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — 初期クエリを実行したユーザー名(分散クエリ実行時)。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 分散クエリ実行における初期クエリの ID。 -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 親クエリが実行された送信元 IP アドレス。 -* `initial_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 親クエリの発行に使用されたクライアントポート。 -* `initial_query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 分散クエリ実行における初期クエリの開始時刻。 -* `initial_query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 分散クエリ実行時の初期クエリ開始時刻(マイクロ秒精度)。 -* `interface` ([UInt8](../../sql-reference/data-types/int-uint.md)) — クエリが発行されたインターフェイス。取りうる値: - * 1 — TCP。 - * 2 — HTTP。 -* `os_user` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) を実行している OS のユーザー名。 -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントが実行されているクライアントマシン上のホスト名。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) または別の TCP クライアントの名前。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントのリビジョン番号。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) または別の TCP クライアントのメジャーバージョン番号。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) または別の TCP クライアントのマイナーバージョン。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントのバージョン番号のパッチ部分。 -* `script_query_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 用の、複数のクエリを含むスクリプト内におけるクエリ番号。 -* `script_line_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 複数のクエリを含むスクリプト内で、[clickhouse-client](../../interfaces/cli.md) 用クエリの開始行番号。 -* `http_method` (UInt8) — クエリを実行した HTTP メソッド。取りうる値: - * 0 — クエリはTCPインターフェイスから実行されました。 - * 1 — `GET`メソッドが使用されました。 - * 2 — `POST`メソッドが使用されました。 -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — HTTP クエリで送信された HTTP ヘッダー `UserAgent`。 -* `http_referer` ([String](../../sql-reference/data-types/string.md)) — HTTP クエリで送信される HTTP ヘッダー `Referer`(クエリを発行したページの絶対 URL または一部の URL を含む)。 -* `forwarded_for` ([String](../../sql-reference/data-types/string.md)) — HTTP クエリで送信される HTTP ヘッダー `X-Forwarded-For`。 -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — [quotas](../../operations/quotas.md) 設定で指定される `quota key`(`keyed` を参照)。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse のリビジョン番号。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/map.md)) — さまざまな指標を計測する ProfileEvents。これらの説明はテーブル [system.events](/operations/system-tables/events) に記載されています。 -* `Settings` ([Map(String, String)](../../sql-reference/data-types/map.md)) — クライアントがクエリを実行したときに変更された設定。設定変更のログ記録を有効にするには、`log_query_settings` パラメータを 1 に設定します。 -* `log_comment` ([String](../../sql-reference/data-types/string.md)) — ログコメント。任意の文字列を設定できますが、その長さは [max_query_size](../../operations/settings/settings.md#max_query_size) を超えてはなりません。定義されていない場合は空文字列です。 -* `thread_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — クエリ実行に関与したスレッド ID。これらのスレッドが同時に実行されていたとは限りません。 -* `peak_threads_usage` ([UInt64)](../../sql-reference/data-types/int-uint.md)) — クエリの実行に使用された同時スレッド数の最大値。 -* `used_aggregate_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `aggregate functions` の正準名。 -* `used_aggregate_function_combinators` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `aggregate function combinators` の正規名。 -* `used_database_engines` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用されたデータベースエンジンのカノニカル名。 -* `used_data_type_families` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `data type families` の正準名。 -* `used_dictionaries` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `dictionaries` の正規名。XML ファイルで設定されたディクショナリの場合はそのディクショナリ名であり、SQL ステートメントで作成されたディクショナリの場合は、正規名は完全修飾オブジェクト名になります。 -* `used_formats` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `formats` の正規名。 -* `used_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行時に使用された `functions` の正規名。 -* `used_storages` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `storages` の正規名。 -* `used_table_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリの実行中に使用された `table functions` の正規名。 -* `used_executable_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリ実行中に使用された `executable user defined functions` の正規名。 -* `used_sql_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — クエリの実行中に使用された `sql user defined functions` の正規名。 -* `used_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) - クエリ実行中にアクセスチェックに成功した権限。 -* `missing_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) - クエリ実行時に不足している権限。 -* `query_cache_usage` ([Enum8](../../sql-reference/data-types/enum.md)) — クエリ実行中における [query cache](../query-cache.md) の使用状況。値: - * `'Unknown'` = ステータスは不明。 - * `'None'` = クエリ結果はクエリキャッシュへの書き込みも読み取りも行われなかった。 - * `'Write'` = クエリ結果がクエリキャッシュに書き込まれた。 - * `'Read'` = クエリ結果がクエリキャッシュから読み取られた。 - - - - - -## 例 {#examples} - -**基本的な例** - -```sql -SELECT * FROM system.query_log WHERE type = 'QueryFinish' ORDER BY query_start_time DESC LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: QueryFinish -event_date: 2021-11-03 -event_time: 2021-11-03 16:13:54 -event_time_microseconds: 2021-11-03 16:13:54.953024 -query_start_time: 2021-11-03 16:13:54 -query_start_time_microseconds: 2021-11-03 16:13:54.952325 -query_duration_ms: 0 -read_rows: 69 -read_bytes: 6187 -written_rows: 0 -written_bytes: 0 -result_rows: 69 -result_bytes: 48256 -memory_usage: 0 -current_database: default -query: DESCRIBE TABLE system.query_log -formatted_query: -normalized_query_hash: 8274064835331539124 -query_kind: -databases: [] -tables: [] -columns: [] -projections: [] -views: [] -exception_code: 0 -exception: -stack_trace: -is_initial_query: 1 -user: default -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -address: ::ffff:127.0.0.1 -port: 40452 -initial_user: default -initial_query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -initial_address: ::ffff:127.0.0.1 -initial_port: 40452 -initial_query_start_time: 2021-11-03 16:13:54 -initial_query_start_time_microseconds: 2021-11-03 16:13:54.952325 -interface: 1 -os_user: sevirov -client_hostname: clickhouse.eu-central1.internal -client_name: ClickHouse -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 1 -http_method: 0 -http_user_agent: -http_referer: -forwarded_for: -quota_key: -revision: 54456 -log_comment: -thread_ids: [30776,31174] -ProfileEvents: {'Query':1,'NetworkSendElapsedMicroseconds':59,'NetworkSendBytes':2643,'SelectedRows':69,'SelectedBytes':6187,'ContextLock':9,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':817,'UserTimeMicroseconds':427,'SystemTimeMicroseconds':212,'OSCPUVirtualTimeMicroseconds':639,'OSReadChars':894,'OSWriteChars':319} -Settings: {'load_balancing':'random','max_memory_usage':'10000000000'} -used_aggregate_functions: [] -used_aggregate_function_combinators: [] -used_database_engines: [] -used_data_type_families: [] -used_dictionaries: [] -used_formats: [] -used_functions: [] -used_storages: [] -used_table_functions: [] -used_executable_user_defined_functions:[] -used_sql_user_defined_functions: [] -used_privileges: [] -missing_privileges: [] -query_cache_usage: None -``` - -**クラウドでの例** - -ClickHouse Cloud では、`system.query_log` は各ノードにローカルであるため、すべてのエントリを確認するには [`clusterAllReplicas`](/sql-reference/table-functions/cluster) 経由でクエリする必要があります。 - -たとえば、「default」クラスタ内のすべてのレプリカから `query_log` の行を集約するには、次のように書けます。 - -```sql -SELECT * -FROM clusterAllReplicas('default', system.query_log) -WHERE event_time >= now() - toIntervalHour(1) -LIMIT 10 -SETTINGS skip_unavailable_shards = 1; -``` - -**関連項目** - -* [system.query_thread_log](/operations/system-tables/query_thread_log) — このテーブルには、各クエリ実行スレッドに関する情報が含まれています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md deleted file mode 100644 index b4162279ec7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '`system.events` テーブルのメモリおよびメトリック値について、クエリごとの履歴を保持し、定期的にディスクにフラッシュするシステムテーブル。' -keywords: ['システムテーブル', 'query_metric_log'] -slug: /operations/system-tables/query_metric_log -title: 'system.query_metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_metric_log {#systemquery_metric_log} - - - -個々のクエリに対して `system.events` テーブルから取得されたメモリおよびメトリック値の履歴を保持し、定期的にディスクへフラッシュします。 - -クエリが開始されると、`query_metric_log_interval` ミリ秒(デフォルトでは 1000 ミリ秒に設定)ごとの一定間隔でデータが収集されます。クエリの実行時間が `query_metric_log_interval` より長い場合は、クエリ終了時にもデータが収集されます。 - -カラム: - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — クエリの ID。 -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — イベントの時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度のイベント時刻。 - -**例** - -```sql -SELECT * FROM system.query_metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -行 1: -────── -query_id: 97c8ba04-b6d4-4bd7-b13e-6201c5c6e49d -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -memory_usage: 313434219 -peak_memory_usage: 598951986 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -``` - -**関連項目** - -* [query_metric_log 設定](../../operations/server-configuration-parameters/settings.md#query_metric_log) — この設定の有効化および無効化について。 -* [query_metric_log_interval](../../operations/settings/settings.md#query_metric_log_interval) -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — 定期的に計算されるメトリクスを格納します。 -* [system.events](/operations/system-tables/events) — 発生した各種イベントの回数を格納します。 -* [system.metrics](../../operations/system-tables/metrics.md) — 即時に計算されるメトリクスを格納します。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse における監視の基本概念。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md deleted file mode 100644 index c2954f7c7cd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: 'クエリを実行するスレッドに関する情報(スレッド名、スレッドの開始時刻、クエリ処理時間など)を含むシステムテーブル。' -keywords: ['system table', 'query_thread_log'] -slug: /operations/system-tables/query_thread_log -title: 'system.query_thread_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_thread_log {#systemquery_thread_log} - - - -クエリを実行するスレッドに関する情報を含みます。たとえば、スレッド名、スレッドの開始時刻、クエリ処理時間などです。 - -ログの記録を開始するには: - -1. [query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log) セクションでパラメータを設定します。 -2. [log_query_threads](/operations/settings/settings#log_query_threads) を 1 に設定します。 - -データのフラッシュ間隔は、[query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log) のサーバー設定セクション内にある `flush_interval_milliseconds` パラメータで設定します。フラッシュを強制的に実行するには、[SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) クエリを使用します。 - -ClickHouse はテーブルからデータを自動的には削除しません。詳細は [Introduction](/operations/system-tables/overview#system-tables-introduction) を参照してください。 - -`query_thread_log` テーブルに登録されるクエリ数を減らすには、[log_queries_probability](/operations/settings/settings#log_queries_probability) 設定を使用できます。 - -Columns: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行するサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — スレッドがクエリの実行を完了した日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — スレッドがクエリの実行を完了した日時。 -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — スレッドがクエリの実行を完了した日時(マイクロ秒精度まで)。 -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — クエリ実行開始時刻。 -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — クエリ実行の開始時刻(マイクロ秒単位の精度)。 -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — クエリの実行時間。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 読み込まれた行数。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 読み込まれたバイト数。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `INSERT` クエリでは書き込まれた行数を示します。それ以外のクエリでは、この列の値は 0 です。 -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `INSERT` クエリの場合は書き込まれたバイト数。それ以外のクエリの場合、この列の値は 0 になります。 -* `memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — このスレッドのコンテキストにおいて、割り当てられたメモリ量と解放されたメモリ量の差を表します。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — このスレッドのコンテキストにおいて、割り当てられたメモリ量と解放されたメモリ量の差が最大となったときの値。 -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — スレッドの名前。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — OS のスレッド ID。 -* `master_thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — OS によって割り当てられた最初のスレッドの ID。 -* `query` ([String](../../sql-reference/data-types/string.md)) — クエリを表す文字列。 -* `is_initial_query` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — クエリの種類。取りうる値: - * 1 — クエリがクライアントによって開始された場合。 - * 0 — 分散クエリ実行のために、別のクエリから開始された場合。 -* `user` ([String](../../sql-reference/data-types/string.md)) — 現在のクエリを実行したユーザー名。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — クエリ ID。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — クエリの実行に使用されたIPアドレス。 -* `port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — クエリの送信に使用されたクライアントのポート番号。 -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — 初回のクエリを実行したユーザー名(分散クエリ実行時)。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 分散クエリ実行における初期クエリの ID。 -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 親クエリが実行された元の IP アドレス。 -* `initial_port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — 親クエリの実行に使用されたクライアントポート。 -* `interface` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — クエリがどのインターフェイスから実行されたかを表します。取り得る値: - * 1 — TCP - * 2 — HTTP -* `os_user` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) を実行する OS のユーザー名。 -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) または他の TCP クライアントが実行されているクライアントマシンのホスト名です。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) または別の TCP クライアントの名前。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントのリビジョン番号。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントのメジャーバージョン。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) またはその他の TCP クライアントのマイナーバージョン。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) または他の TCP クライアントのバージョンのパッチ番号。 -* `http_method` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — クエリを発行した HTTP メソッド。取りうる値: - * 0 — クエリは TCPインターフェイスから実行されました。 - * 1 — `GET` メソッドが使用されました。 - * 2 — `POST` メソッドが使用されました。 -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — HTTP リクエストで渡された `UserAgent` ヘッダー。 -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — [quotas](../../operations/quotas.md) 設定で指定される「quota key」(`keyed` を参照)です。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse のリビジョン番号。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — このスレッドに関するさまざまなメトリクスを測定する `ProfileEvents`。各イベントの説明は、[system.events](/operations/system-tables/events) テーブルで確認できます。 - -**例** - -```sql - SELECT * FROM system.query_thread_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-11 -event_time: 2020-09-11 10:08:17 -event_time_microseconds: 2020-09-11 10:08:17.134042 -query_start_time: 2020-09-11 10:08:17 -query_start_time_microseconds: 2020-09-11 10:08:17.063150 -query_duration_ms: 70 -read_rows: 0 -read_bytes: 0 -written_rows: 1 -written_bytes: 12 -memory_usage: 4300844 -peak_memory_usage: 4300844 -thread_name: TCPHandler -thread_id: 638133 -master_thread_id: 638133 -query: INSERT INTO test1 VALUES -is_initial_query: 1 -user: default -query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -address: ::ffff:127.0.0.1 -port: 33452 -initial_user: default -initial_query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -initial_address: ::ffff:127.0.0.1 -initial_port: 33452 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -revision: 54440 -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -``` - -**関連項目** - -* [system.query_log](/operations/system-tables/query_log) — クエリの実行に関する共通情報を含む `query_log` システムテーブルの説明。 -* [system.query_views_log](/operations/system-tables/query_views_log) — クエリ中に実行された各ビューに関する情報を含むシステムテーブル。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md deleted file mode 100644 index 676697428ee..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: 'クエリ実行時に実行された依存ビューに関する情報(ビューの種類や実行時間など)を保持するシステムテーブル。' -keywords: ['system table', 'query_views_log'] -slug: /operations/system-tables/query_views_log -title: 'system.query_views_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_views_log {#systemquery_views_log} - - - -クエリ実行時に実行された依存ビューに関する情報を保持します。たとえば、ビューの種類や実行時間などです。 - -ログを有効化するには: - -1. [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log) セクションでパラメータを設定します。 -2. [log_query_views](/operations/settings/settings#log_query_views) を 1 に設定します。 - -データのフラッシュ周期は、サーバー設定セクション [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log) の `flush_interval_milliseconds` パラメータで設定します。フラッシュを強制するには、[SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) クエリを使用します。 - -ClickHouse はこのテーブルからデータを自動的には削除しません。詳細は [Introduction](/operations/system-tables/overview#system-tables-introduction) を参照してください。 - -`query_views_log` テーブルに登録されるクエリ数を減らすには、[log_queries_probability](/operations/settings/settings#log_queries_probability) 設定を使用できます。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行したサーバーのホスト名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — ビューで最後のイベントが発生した日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — ビューの実行が終了した日時。 -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — ビューの実行がマイクロ秒精度で終了した日時。 -* `view_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — ビュー実行時間(すべてのステージの合計)をミリ秒で表したもの。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初期クエリの ID(分散クエリ実行用)。 -* `view_name` ([String](../../sql-reference/data-types/string.md)) — ビュー名。 -* `view_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — ビューの UUID。 -* `view_type` ([Enum8](../../sql-reference/data-types/enum.md)) — ビューの種類。値: - * `'Default' = 1` — [Default views](/sql-reference/statements/create/view#normal-view)。このログには出力されないはずです。 - * `'Materialized' = 2` — [Materialized views](/sql-reference/statements/create/view#materialized-view)。 - * `'Live' = 3` — [Live views](../../sql-reference/statements/create/view.md#live-view)。 -* `view_query` ([String](../../sql-reference/data-types/string.md)) — ビューによって実行されたクエリ。 -* `view_target` ([String](../../sql-reference/data-types/string.md)) — ビューのターゲットテーブル名。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 読み取った行数。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 読み取ったバイト数。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 書き込まれた行数。 -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 書き込まれたバイト数。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — このビューのコンテキストにおける、割り当てられたメモリ量と解放されたメモリ量の差分の最大値。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — さまざまなメトリクスを計測する ProfileEvents。これらの説明はテーブル [system.events](/operations/system-tables/events) に記載されています。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — ビューのステータス。値: - * `'QueryStart' = 1` — ビュー実行の開始に成功。表示されないはずです。 - * `'QueryFinish' = 2` — ビュー実行が正常に終了。 - * `'ExceptionBeforeStart' = 3` — ビュー実行開始前の例外。 - * `'ExceptionWhileProcessing' = 4` — ビュー実行中の例外。 -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — 例外のコード。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 例外メッセージ。 -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [スタックトレース](https://en.wikipedia.org/wiki/Stack_trace)。クエリが正常に完了した場合は空文字列。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.query_views_log LIMIT 1 \G; -``` - -結果: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2021-06-22 -event_time: 2021-06-22 13:23:07 -event_time_microseconds: 2021-06-22 13:23:07.738221 -view_duration_ms: 0 -initial_query_id: c3a1ac02-9cad-479b-af54-9e9c0a7afd70 -view_name: default.matview_inner -view_uuid: 00000000-0000-0000-0000-000000000000 -view_type: Materialized -view_query: SELECT * FROM default.table_b -view_target: default.`.inner.matview_inner` -read_rows: 4 -read_bytes: 64 -written_rows: 2 -written_bytes: 32 -peak_memory_usage: 4196188 -ProfileEvents: {'FileOpen':2,'WriteBufferFromFileDescriptorWrite':2,'WriteBufferFromFileDescriptorWriteBytes':187,'IOBufferAllocs':3,'IOBufferAllocBytes':3145773,'FunctionExecute':3,'DiskWriteElapsedMicroseconds':13,'InsertedRows':2,'InsertedBytes':16,'SelectedRows':4,'SelectedBytes':48,'ContextLock':16,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':698,'SoftPageFaults':4,'OSReadChars':463} -status: QueryFinish -exception_code: 0 -exception: -stack_trace: -``` - -**関連情報** - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md deleted file mode 100644 index dcf651df948..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -description: 'すべてのクォータにおける各間隔ごとの最大値に関する情報を含む system テーブル。1 つのクォータに対して、任意の数(0 行を含む)の行が対応する場合があります。' -keywords: ['system table', 'quota_limits'] -slug: /operations/system-tables/quota_limits -title: 'system.quota_limits' -doc_type: 'reference' ---- - -# system.quota_limits {#systemquota_limits} - -すべてのクォータのすべての間隔に対する上限に関する情報を含みます。1 つのクォータに対して、対応する行数は任意(0 行も可)です。 - -カラム: - -* `quota_name` ([String](../../sql-reference/data-types/string.md)) — クォータ名。 -* `duration` ([UInt32](../../sql-reference/data-types/int-uint.md)) — リソース消費量を集計する時間間隔の長さ(秒単位)。 -* `is_randomized_interval` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 論理値。間隔がランダム化されているかどうかを示します。ランダム化されていない場合、間隔は常に同じ時刻から開始します。たとえば、1 分の間隔は常に分の整数値の時刻で開始します(つまり 11:20:00 で開始することはあっても、11:20:01 で開始することはありません)。1 日の間隔は常に UTC の真夜中から開始します。間隔がランダム化されている場合、最初の間隔はランダムな時刻から開始し、その後の間隔は連続して開始されます。値: -* `0` — 間隔はランダム化されていません。 -* `1` — 間隔はランダム化されています。 -* `max_queries` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — クエリの最大数。 -* `max_query_selects` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — SELECT クエリの最大数。 -* `max_query_inserts` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — INSERT クエリの最大数。 -* `max_errors` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — エラーの最大数。 -* `max_result_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 結果の行数の最大値。 -* `max_result_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — クエリ結果を保存するために使用される RAM 量の最大値(バイト単位)。 -* `max_read_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — クエリで使用されたすべてのテーブルおよびテーブル関数から読み取られた行数の最大値。 -* `max_read_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — クエリで使用されたすべてのテーブルおよびテーブル関数から読み取られたバイト数の最大値。 -* `max_execution_time` ([Nullable](../../sql-reference/data-types/nullable.md)([Float64](../../sql-reference/data-types/float.md))) — クエリ実行時間の最大値(秒単位)。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md deleted file mode 100644 index 7407f467ec5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -description: '現在のユーザーによるクォータ使用状況(使用済みクォータ量と残りのクォータ量など)に関する情報を保持する system テーブル。' -keywords: ['system テーブル', 'quota_usage'] -slug: /operations/system-tables/quota_usage -title: 'system.quota_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -現在のユーザーのクォータ使用状況:使用量と残量を示します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連項目 {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md deleted file mode 100644 index 7e89ba4d3ac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: 'クォータに関する情報を含むシステムテーブル。' -keywords: ['system テーブル', 'クォータ', 'quota'] -slug: /operations/system-tables/quotas -title: 'system.quotas' -doc_type: 'reference' ---- - - - -# system.quotas {#systemquotas} - -[クォータ](../../operations/system-tables/quotas.md)に関する情報を含みます。 - -Columns: -- `name` ([String](../../sql-reference/data-types/string.md)) — クォータ名。 -- `id` ([UUID](../../sql-reference/data-types/uuid.md)) — クォータID。 -- `storage`([String](../../sql-reference/data-types/string.md)) — クォータのストレージ。可能な値: クォータが users.xml ファイルで設定されている場合は "users.xml"、SQL クエリで設定されている場合は "disk"。 -- `keys` ([Array](../../sql-reference/data-types/array.md)([Enum8](../../sql-reference/data-types/enum.md))) — クォータをどのように共有するかを指定するキー。同じクォータとキーを使用する 2 つの接続は、同じ量のリソースを共有します。値: - - `[]` — すべてのユーザーが同じクォータを共有します。 - - `['user_name']` — 同じユーザー名の接続が同じクォータを共有します。 - - `['ip_address']` — 同一 IP からの接続が同じクォータを共有します。 - - `['client_key']` — 同じキーを持つ接続が同じクォータを共有します。キーはクライアントによって明示的に指定する必要があります。[clickhouse-client](../../interfaces/cli.md) を使用する場合は、`--quota_key` パラメータでキー値を渡すか、クライアント構成ファイル内で `quota_key` パラメータを使用します。HTTP インターフェイスを使用する場合は、`X-ClickHouse-Quota` ヘッダーを使用します。 - - `['user_name', 'client_key']` — 同じ `client_key` を持つ接続が同じクォータを共有します。キーがクライアントによって指定されない場合、クォータは `user_name` ごとに追跡されます。 - - `['client_key', 'ip_address']` — 同じ `client_key` を持つ接続が同じクォータを共有します。キーがクライアントによって指定されない場合、クォータは `ip_address` ごとに追跡されます。 -- `durations` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 秒単位の時間間隔の長さ。 -- `apply_to_all` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 論理値。クォータがどのユーザーに適用されるかを示します。値: - - `0` — クォータは `apply_to_list` で指定されたユーザーに適用されます。 - - `1` — クォータは `apply_to_except` に列挙されたユーザーを除くすべてのユーザーに適用されます。 -- `apply_to_list` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — クォータを適用するユーザー名/[ロール](../../guides/sre/user-management/index.md#role-management)のリスト。 -- `apply_to_except` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — クォータを適用しないユーザー名/ロールのリスト。 - - - -## 関連項目 {#see-also} - -- [SHOW QUOTAS](/sql-reference/statements/show#show-quotas) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md deleted file mode 100644 index 9e14c2d5b75..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'すべてのユーザーのクォータ使用状況に関する情報を含むシステムテーブル。' -keywords: ['system table', 'quotas_usage', 'quota'] -slug: /operations/system-tables/quotas_usage -title: 'system.quotas_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.quotas_usage {#systemquotas_usage} - - - -すべてのユーザーのクォータ使用状況。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連項目 {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md deleted file mode 100644 index 93cdb4db96f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'ローカルサーバー上に存在するレプリケートテーブルの情報およびステータスを含むシステムテーブル。監視に役立ちます。' -keywords: ['system table', 'replicas'] -slug: /operations/system-tables/replicas -title: 'system.replicas' -doc_type: 'reference' ---- - -# system.replicas {#systemreplicas} - -ローカルサーバー上に存在するレプリケートテーブルの情報とステータスを含みます。 -このテーブルは監視用途に使用できます。テーブルには、すべての `Replicated*` テーブルごとに 1 行が格納されます。 - -例: - -```sql -SELECT * -FROM system.replicas -WHERE table = 'test_table' -FORMAT Vertical -``` - -```text -Query id: dc6dcbcb-dc28-4df9-ae27-4354f5b3b13e - -Row 1: -─────── -database: db -table: test_table -engine: ReplicatedMergeTree -is_leader: 1 -can_become_leader: 1 -is_readonly: 0 -is_session_expired: 0 -future_parts: 0 -parts_to_check: 0 -zookeeper_path: /test/test_table -replica_name: r1 -replica_path: /test/test_table/replicas/r1 -columns_version: -1 -queue_size: 27 -inserts_in_queue: 27 -merges_in_queue: 0 -part_mutations_in_queue: 0 -queue_oldest_time: 2021-10-12 14:48:48 -inserts_oldest_time: 2021-10-12 14:48:48 -merges_oldest_time: 1970-01-01 03:00:00 -part_mutations_oldest_time: 1970-01-01 03:00:00 -oldest_part_to_get: 1_17_17_0 -oldest_part_to_merge_to: -oldest_part_to_mutate_to: -log_max_index: 206 -log_pointer: 207 -last_queue_update: 2021-10-12 14:50:08 -absolute_delay: 99 -total_replicas: 5 -active_replicas: 5 -lost_part_count: 0 -last_queue_update_exception: -zookeeper_exception: -replica_is_active: {'r1':1,'r2':1} -``` - -カラム: - -* `database` (`String`) - データベース名 -* `table` (`String`) - テーブル名 -* `engine` (`String`) - テーブルエンジン名 -* `is_leader` (`UInt8`) - レプリカがリーダーかどうか。 - 複数のレプリカが同時にリーダーになることがあります。`merge_tree` 設定の `replicated_can_become_leader` を使用して、レプリカがリーダーになることを防止できます。リーダーはバックグラウンドマージのスケジューリングを担当します。 - リーダーかどうかに関わらず、利用可能で ZK にセッションを持つ任意のレプリカに対して書き込みを実行できることに注意してください。 -* `can_become_leader` (`UInt8`) - レプリカがリーダーになれるかどうか。 -* `is_readonly` (`UInt8`) - レプリカが読み取り専用モードかどうか。 - このモードは、設定に ClickHouse Keeper のセクションが存在しない場合、ClickHouse Keeper でセッションを再初期化する際に不明なエラーが発生した場合、または ClickHouse Keeper でセッションを再初期化している間に有効になります。 -* `is_session_expired` (`UInt8`) - ClickHouse Keeper とのセッションが期限切れになっているかどうか。基本的には `is_readonly` と同じです。 -* `future_parts` (`UInt32`) - INSERT または、まだ実行されていないマージの結果として出現するデータパーツの数。 -* `parts_to_check` (`UInt32`) - 検証キュー内のデータパーツの数。パーツが破損している疑いがある場合、そのパーツは検証キューに入れられます。 -* `zookeeper_path` (`String`) - ClickHouse Keeper 内のテーブルデータへのパス。 -* `replica_name` (`String`) - ClickHouse Keeper 内でのレプリカ名。同じテーブルの異なるレプリカは異なる名前を持ちます。 -* `replica_path` (`String`) - ClickHouse Keeper 内のレプリカデータへのパス。'zookeeper_path/replicas/replica_path' を連結したものと同じです。 -* `columns_version` (`Int32`) - テーブル構造のバージョン番号。何回 ALTER が実行されたかを示します。レプリカ間でバージョンが異なる場合、一部のレプリカがまだすべての ALTER を実行していないことを意味します。 -* `queue_size` (`UInt32`) - 実行待ちの操作キューのサイズ。操作にはデータブロックの挿入、マージ、およびその他の特定のアクションが含まれます。通常は `future_parts` と一致します。 -* `inserts_in_queue` (`UInt32`) - 実行が必要なデータブロックの挿入数。挿入は通常かなり速くレプリケートされます。この数が大きい場合は、何か問題が発生していることを意味します。 -* `merges_in_queue` (`UInt32`) - 実行待ちのマージ数。マージには時間がかかることがあるため、この値が長時間ゼロより大きい場合があります。 -* `part_mutations_in_queue` (`UInt32`) - 実行待ちのミューテーションの数。 -* `queue_oldest_time` (`DateTime`) - `queue_size` が 0 より大きい場合、キューに追加された最も古い操作の時刻を示します。 -* `inserts_oldest_time` (`DateTime`) - `queue_oldest_time` を参照してください -* `merges_oldest_time` (`DateTime`) - `queue_oldest_time` を参照してください -* `part_mutations_oldest_time` (`DateTime`) - `queue_oldest_time` を参照してください - -次の 4 列は、ZK にアクティブなセッションがある場合にのみゼロ以外の値になります。 - -* `log_max_index` (`UInt64`) - 全体的なアクティビティログにおける最大エントリ番号。 -* `log_pointer` (`UInt64`) - レプリカがその実行キューへコピーした全体的なアクティビティログ中の最大エントリ番号に 1 を加えた値。`log_pointer` が `log_max_index` よりかなり小さい場合は、何か問題があります。 -* `last_queue_update` (`DateTime`) - キューが最後に更新された時刻。 -* `absolute_delay` (`UInt64`) - 現在のレプリカの遅延時間(秒単位)。 -* `total_replicas` (`UInt8`) - このテーブルの既知のレプリカの総数。 -* `active_replicas` (`UInt8`) - このテーブルのうち、ClickHouse Keeper にセッションを持つレプリカの数(つまり、稼働中のレプリカの数)。 -* `lost_part_count` (`UInt64`) - テーブル作成以降、すべてのレプリカで合計してテーブル内で失われたデータパーツの数。この値は ClickHouse Keeper に永続化され、増加するだけです。 -* `last_queue_update_exception` (`String`) - キューに壊れたエントリが含まれている場合の最後の例外メッセージ。特に、ClickHouse がバージョン間で下位互換性を破り、新しいバージョンによって書き込まれたログエントリを古いバージョンがパースできない場合に重要です。 -* `zookeeper_exception` (`String`) - ClickHouse Keeper から情報を取得するときにエラーが発生した場合の、最後の例外メッセージ。 -* `replica_is_active` ([Map(String, UInt8)](../../sql-reference/data-types/map.md)) — レプリカ名と、そのレプリカがアクティブかどうかの対応を表すマップ。 - -すべてのカラムを取得すると、各行ごとに複数回 ClickHouse Keeper から読み取る必要があるため、テーブルの動作がやや遅くなる場合があります。 -最後の 4 つのカラム(log_max_index、log_pointer、total_replicas、active_replicas)を取得しなければ、テーブルは高速に動作します。 - -例えば、次のようにしてすべてが正しく動作していることを確認できます。 - -```sql -SELECT - database, - table, - is_leader, - is_readonly, - is_session_expired, - future_parts, - parts_to_check, - columns_version, - queue_size, - inserts_in_queue, - merges_in_queue, - log_max_index, - log_pointer, - total_replicas, - active_replicas -FROM system.replicas -WHERE - is_readonly - OR is_session_expired - OR future_parts > 20 - OR parts_to_check > 10 - OR queue_size > 20 - OR inserts_in_queue > 10 - OR log_max_index - log_pointer > 10 - OR total_replicas < 2 - OR active_replicas < total_replicas -``` - -このクエリで何も返されない場合は、問題ないことを意味します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md deleted file mode 100644 index 45f02fe273c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'バックグラウンドで現在実行中のフェッチに関する情報を含むシステムテーブル。' -keywords: ['システムテーブル', 'replicated_fetches'] -slug: /operations/system-tables/replicated_fetches -title: 'system.replicated_fetches' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.replicated_fetches {#systemreplicated_fetches} - - - -現在実行中のバックグラウンドでのフェッチ処理に関する情報を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.replicated_fetches LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: t -elapsed: 7.243039876 -progress: 0.41832135995612835 -result_part_name: all_0_0_0 -result_part_path: /var/lib/clickhouse/store/700/70080a04-b2de-4adf-9fa5-9ea210e81766/all_0_0_0/ -partition_id: all -total_size_bytes_compressed: 1052783726 -bytes_read_compressed: 440401920 -source_replica_path: /clickhouse/test/t/replicas/1 -source_replica_hostname: node1 -source_replica_port: 9009 -interserver_scheme: http -URI: http://node1:9009/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftest%2Ft%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=4&compress=false -to_detached: 0 -thread_id: 54 -``` - -**関連項目** - -* [ReplicatedMergeTree テーブルの管理](../../sql-reference/statements/system.md/#managing-replicatedmergetree-tables) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md deleted file mode 100644 index 9461e79c341..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'ClickHouse Keeper または ZooKeeper に保存されている、`ReplicatedMergeTree` - ファミリーのテーブル用レプリケーションキュー内のタスクに関する情報を保持するシステムテーブル。' -keywords: ['system table', 'replication_queue'] -slug: /operations/system-tables/replication_queue -title: 'system.replication_queue' -doc_type: 'reference' ---- - -# system.replication_queue {#systemreplication_queue} - -ClickHouse Keeper または ZooKeeper に保存されている、`ReplicatedMergeTree` ファミリーのテーブル用のレプリケーションキュー内のタスクに関する情報が含まれます。 - -カラム: - -* `database` ([String](../../sql-reference/data-types/string.md)) — データベース名。 - -* `table` ([String](../../sql-reference/data-types/string.md)) — テーブル名。 - -* `replica_name` ([String](../../sql-reference/data-types/string.md)) — ClickHouse Keeper 内のレプリカ名。同じテーブルの異なるレプリカは、それぞれ異なる名前を持ちます。 - -* `position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — キュー内でのタスクの位置。 - -* `node_name` ([String](../../sql-reference/data-types/string.md)) — ClickHouse Keeper 内のノード名。 - -* `type` ([String](../../sql-reference/data-types/string.md)) — キュー内のタスクの種類。次のいずれか: - - * `GET_PART` — 他のレプリカからパーツを取得します。 - * `ATTACH_PART` — パーツをアタッチします。`detached` フォルダ内で見つかった場合は、同じレプリカからのパーツである可能性があります。`GET_PART` とほぼ同一で、一部が最適化されたものと考えることができます。 - * `MERGE_PARTS` — パーツをマージします。 - * `DROP_RANGE` — 指定されたパーティション内の、指定された範囲のパーツを削除します。 - * `CLEAR_COLUMN` — 注: 非推奨。指定したパーティションから特定のカラムを削除します。 - * `CLEAR_INDEX` — 注: 非推奨。指定したパーティションから特定のインデックスを削除します。 - * `REPLACE_RANGE` — 特定の範囲のパーツを削除し、新しいパーツで置き換えます。 - * `MUTATE_PART` — パーツに対して 1 つまたは複数のミューテーション (mutation) を適用します。 - * `ALTER_METADATA` — グローバルな /metadata および /columns パスに従って ALTER による変更を適用します。 - -* `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — タスクが実行のために投入された日時。 - -* `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — タスクの完了および完了確認を待っているレプリカの数。このカラムは `GET_PARTS` タスクにのみ関連します。 - -* `source_replica` ([String](../../sql-reference/data-types/string.md)) — ソースレプリカ名。 - -* `new_part_name` ([String](../../sql-reference/data-types/string.md)) — 新しいパーツ名。 - -* `parts_to_merge` ([Array](../../sql-reference/data-types/array.md) ([String](../../sql-reference/data-types/string.md))) — マージまたは更新の対象となるパーツ名。 - -* `is_detach` ([UInt8](../../sql-reference/data-types/int-uint.md)) — `DETACH_PARTS` タスクがキューにあるかどうかを示すフラグ。 - -* `is_currently_executing` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 特定のタスクが現在実行中かどうかを示すフラグ。 - -* `num_tries` ([UInt32](../../sql-reference/data-types/int-uint.md)) — タスクの実行に失敗した試行回数。 - -* `last_exception` ([String](../../sql-reference/data-types/string.md)) — 発生した最後のエラー (存在する場合) に関するテキストメッセージ。 - -* `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 最後にタスクの実行を試行した日時。 - -* `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — アクションが延期された回数。 - -* `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — タスクが延期された理由。 - -* `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 最後にタスクが延期された日時。 - -* `merge_type` ([String](../../sql-reference/data-types/string.md)) — 現在実行中のマージの種類。ミューテーションの場合は空になります。 - -**例** - -```sql -SELECT * FROM system.replication_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: merge -table: visits_v2 -replica_name: mtgiga001-1t -position: 15 -node_name: queue-0009325559 -type: MERGE_PARTS -create_time: 2020-12-07 14:04:21 -required_quorum: 0 -source_replica: mtgiga001-1t -new_part_name: 20201130_121373_121384_2 -parts_to_merge: ['20201130_121373_121378_1','20201130_121379_121379_0','20201130_121380_121380_0','20201130_121381_121381_0','20201130_121382_121382_0','20201130_121383_121383_0','20201130_121384_121384_0'] -is_detach: 0 -is_currently_executing: 0 -num_tries: 36 -last_exception: Code: 226, e.displayText() = DB::Exception: Marks file '/opt/clickhouse/data/merge/visits_v2/tmp_fetch_20201130_121373_121384_2/CounterID.mrk' does not exist (version 20.8.7.15 (official build)) -last_attempt_time: 2020-12-08 17:35:54 -num_postponed: 0 -postpone_reason: -last_postpone_time: 1970-01-01 03:00:00 -``` - -**関連情報** - -* [ReplicatedMergeTree テーブルの管理](/sql-reference/statements/system#managing-replicatedmergetree-tables) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md deleted file mode 100644 index 148cf63bf36..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'ローカルサーバー上に存在するリソースに関する情報を含むシステムテーブルであり、各リソースにつき 1 行を持ちます。' -keywords: ['システムテーブル', 'リソース'] -slug: /operations/system-tables/resources -title: 'system.resources' -doc_type: 'reference' ---- - -# system.resources {#systemresources} - -ローカルサーバー上に存在する[リソース](/operations/workload-scheduling.md#workload_entity_storage)に関する情報が含まれます。テーブルには、各リソースにつき 1 行が格納されます。 - -例: - -```sql -SELECT * -FROM system.resources -FORMAT Vertical -``` - -```text -行 1: -────── -name: io_read -read_disks: ['s3'] -write_disks: [] -create_query: CREATE RESOURCE io_read (READ DISK s3) - -行 2: -────── -name: io_write -read_disks: [] -write_disks: ['s3'] -create_query: CREATE RESOURCE io_write (WRITE DISK s3) -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md deleted file mode 100644 index 35fd071c001..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'ユーザーおよびロールに対するロール付与情報を保持するシステムテーブル。' -keywords: ['system table', 'role_grants'] -slug: /operations/system-tables/role_grants -title: 'system.role_grants' -doc_type: 'reference' ---- - -# system.role_grants {#systemrole_grants} - -ユーザーおよびロールに対するロール付与情報を保持します。このテーブルに行を追加するには、`GRANT role TO user` を使用します。 - -列: - -* `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — ユーザー名。 - -* `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — ロール名。 - -* `granted_role_name` ([String](../../sql-reference/data-types/string.md)) — `role_name` ロールに付与されたロール名。あるロールを別のロールに付与するには `GRANT role1 TO role2` を使用します。 - -* `granted_role_is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `granted_role` がデフォルトロールかどうかを示すフラグ。取りうる値: - * 1 — `granted_role` はデフォルトロール。 - * 0 — `granted_role` はデフォルトロールではない。 - -* `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — `granted_role` が [ADMIN OPTION](/sql-reference/statements/grant#admin-option) 権限を持つロールかどうかを示すフラグ。取りうる値: - * 1 — ロールは `ADMIN OPTION` 権限を持つ。 - * 0 — ロールは `ADMIN OPTION` 権限を持たない。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md deleted file mode 100644 index da58a3bfc01..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '設定されているロールに関する情報を含むシステムテーブル。' -keywords: ['システムテーブル', 'ロール'] -slug: /operations/system-tables/roles -title: 'system.roles' -doc_type: 'reference' ---- - -# system.roles {#systemroles} - -設定されている[ロール](../../guides/sre/user-management/index.md#role-management)に関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連情報 {#see-also} - -- [SHOW ROLES](/sql-reference/statements/show#show-roles) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md deleted file mode 100644 index 11f28d11245..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '特定のテーブルに対するフィルタと、その行ポリシーを使用するロールおよび/またはユーザーの一覧を含む system テーブル。' -keywords: ['system table', 'row_policies'] -slug: /operations/system-tables/row_policies -title: 'system.row_policies' -doc_type: 'reference' ---- - -# system.row_policies {#systemrow_policies} - -特定の 1 つのテーブルに対するフィルターと、この行ポリシーを使用するロールおよび/またはユーザーの一覧を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連項目 {#see-also} - -- [SHOW POLICIES](/sql-reference/statements/show#show-policies) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md deleted file mode 100644 index 3fa68c1f5e0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -description: 'S3Queue テーブルの設定に関する情報を格納する system テーブル。 - サーバーバージョン `24.10` 以降で利用可能。' -keywords: ['system テーブル', 's3_queue_settings'] -slug: /operations/system-tables/s3_queue_settings -title: 'system.s3_queue_settings' -doc_type: 'reference' ---- - -# system.s3_queue_settings {#systems3_queue_settings} - -[S3Queue](../../engines/table-engines/integrations/s3queue.md) テーブルの設定に関する情報を含みます。サーバーバージョン `24.10` 以降で利用可能です。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md deleted file mode 100644 index 693900298c0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'ローカルサーバー上に配置されているスケジューリングノードの情報および状態を含むシステムテーブル。' -keywords: ['システムテーブル', 'スケジューラ'] -slug: /operations/system-tables/scheduler -title: 'system.scheduler' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.scheduler {#systemscheduler} - - - -ローカルサーバー上に存在する[スケジューリングノード](/operations/workload-scheduling.md/#hierarchy)に関する情報とステータスを含みます。 -このテーブルは監視に使用できます。テーブルには各スケジューリングノードごとに1行が格納されます。 - -例: - -```sql -SELECT * -FROM system.scheduler -WHERE resource = 'network_read' AND path = '/prio/fair/prod' -FORMAT Vertical -``` - -```text -Row 1: -────── -resource: network_read -path: /prio/fair/prod -type: fifo -weight: 5 -priority: 0 -is_active: 0 -active_children: 0 -dequeued_requests: 67 -canceled_requests: 0 -dequeued_cost: 4692272 -canceled_cost: 0 -busy_periods: 63 -vruntime: 938454.1999999989 -system_vruntime: ᴺᵁᴸᴸ -queue_length: 0 -queue_cost: 0 -budget: -60524 -is_satisfied: ᴺᵁᴸᴸ -inflight_requests: ᴺᵁᴸᴸ -inflight_cost: ᴺᵁᴸᴸ -max_requests: ᴺᵁᴸᴸ -max_cost: ᴺᵁᴸᴸ -max_speed: ᴺᵁᴸᴸ -max_burst: ᴺᵁᴸᴸ -throttling_us: ᴺᵁᴸᴸ -tokens: ᴺᵁᴸᴸ -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md deleted file mode 100644 index 025680ceb80..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: 'キャッシュされたすべてのファイルスキーマに関する情報を含む system テーブル。' -keywords: ['system テーブル', 'schema_inference_cache'] -slug: /operations/system-tables/schema_inference_cache -title: 'system.schema_inference_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.schema_inference_cache {#systemschema_inference_cache} - - - -キャッシュされたすべてのファイルスキーマに関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -`data.jsonl` というファイルがあり、その内容が次のような内容だとします: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["football", "cooking", "music"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["tennis", "art"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["fitness", "reading", "shopping"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["movies", "skydiving"]} -``` - -:::tip -`data.jsonl` を `user_files_path` ディレクトリに配置します。これは ClickHouse の設定ファイルを確認することで特定できます。デフォルトは次のとおりです。 - -```sql -/var/lib/clickhouse/user_files/ -``` - -::: - -`clickhouse-client` を起動し、`DESCRIBE` クエリを実行します。 - -```sql -DESCRIBE file('data.jsonl') SETTINGS input_format_try_infer_integers=0; -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Float64) │ │ │ │ │ │ -│ age │ Nullable(Float64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -`system.schema_inference_cache` テーブルの内容を確認してみましょう。 - -```sql -SELECT * -FROM system.schema_inference_cache -FORMAT Vertical -``` - -```response -行 1: -────── -storage: File -source: /home/droscigno/user_files/data.jsonl -format: JSONEachRow -additional_format_info: schema_inference_hints=, max_rows_to_read_for_schema_inference=25000, schema_inference_make_columns_nullable=true, try_infer_integers=false, try_infer_dates=true, try_infer_datetimes=true, try_infer_numbers_from_strings=true, read_bools_as_numbers=true, try_infer_objects=false -registration_time: 2022-12-29 17:49:52 -schema: id Nullable(Float64), age Nullable(Float64), name Nullable(String), hobbies Array(Nullable(String)) -``` - -**関連項目** - -* [入力データからのスキーマ自動推論](/interfaces/schema-inference.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md deleted file mode 100644 index f387ef8f8ca..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'サーバーのグローバル設定に関する情報を格納するシステムテーブルで、これらの設定は `config.xml` に記述されています。' -keywords: ['system table', 'server_settings'] -slug: /operations/system-tables/server_settings -title: 'system.server_settings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.server_settings {#systemserver_settings} - - - -`config.xml` に指定されているサーバーのグローバル設定に関する情報を保持します。 -現在、このテーブルに表示されるのは `config.xml` の最上位階層にある設定のみで、入れ子になった設定(例: [logger](../../operations/server-configuration-parameters/settings.md#logger))には対応していません。 - -Columns: - -* `name` ([String](../../sql-reference/data-types/string.md)) — サーバー設定名。 -* `value` ([String](../../sql-reference/data-types/string.md)) — サーバー設定値。 -* `default` ([String](../../sql-reference/data-types/string.md)) — サーバー設定のデフォルト値。 -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 設定が `config.xml` で明示的に指定されているかどうかを示します。 -* `description` ([String](../../sql-reference/data-types/string.md)) — サーバー設定の簡潔な説明。 -* `type` ([String](../../sql-reference/data-types/string.md)) — サーバー設定値の型。 -* `changeable_without_restart` ([Enum8](../../sql-reference/data-types/enum.md)) — 設定をサーバーの再起動なしで変更できるかどうか。値: - * `'No' ` - * `'IncreaseOnly'` - * `'DecreaseOnly'` - * `'Yes'` -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) - 設定が廃止済みかどうかを示します。 - -**Example** - -次の例は、名前に `thread_pool` を含むサーバー設定に関する情報の取得方法を示します。 - -```sql -SELECT * -FROM system.server_settings -WHERE name LIKE '%thread_pool%' -``` - -```text -┌─name──────────────────────────────────────────┬─value─┬─default─┬─changed─┬─description─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─type───┬─changeable_without_restart─┬─is_obsolete─┐ -│ max_thread_pool_size │ 10000 │ 10000 │ 0 │ OSから割り当て可能で、クエリ実行およびバックグラウンド操作に使用できるスレッドの最大数。 │ UInt64 │ No │ 0 │ -│ max_thread_pool_free_size │ 1000 │ 1000 │ 0 │ 一度割り当てられた後、タスク数が不足している場合にグローバルスレッドプールに常駐しアイドル状態を維持するスレッドの最大数。 │ UInt64 │ No │ 0 │ -│ thread_pool_queue_size │ 10000 │ 10000 │ 0 │ キューに配置され実行を待機するタスクの最大数。 │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_size │ 100 │ 100 │ 0 │ IO操作に使用されるスレッドの最大数。 │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_free_size │ 0 │ 0 │ 0 │ IOスレッドプールの最大空きサイズ。 │ UInt64 │ No │ 0 │ -│ io_thread_pool_queue_size │ 10000 │ 10000 │ 0 │ IOスレッドプールのキューサイズ。 │ UInt64 │ No │ 0 │ -│ max_active_parts_loading_thread_pool_size │ 64 │ 64 │ 0 │ 起動時にアクティブなデータパートセット(Active)をロードするスレッド数。 │ UInt64 │ No │ 0 │ -│ max_outdated_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ 起動時に非アクティブなデータパートセット(Outdated)をロードするスレッド数。 │ UInt64 │ No │ 0 │ -│ max_unexpected_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ 起動時に非アクティブなデータパートセット(Unexpected)をロードするスレッド数。 │ UInt64 │ No │ 0 │ -│ max_parts_cleaning_thread_pool_size │ 128 │ 128 │ 0 │ 非アクティブなデータパートを並行削除するためのスレッド数。 │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_size │ 1000 │ 1000 │ 0 │ BACKUPクエリのIO操作に使用されるスレッドの最大数。 │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_free_size │ 0 │ 0 │ 0 │ バックアップIOスレッドプールの最大空きサイズ。 │ UInt64 │ No │ 0 │ -│ backups_io_thread_pool_queue_size │ 0 │ 0 │ 0 │ バックアップIOスレッドプールのキューサイズ。 │ UInt64 │ No │ 0 │ -└───────────────────────────────────────────────┴───────┴─────────┴─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴────────────────────────────┴─────────────┘ - -``` - -`WHERE changed` は、例えば設定ファイル内の設定が正しく読み込まれ、実際に利用されているかどうかを確認したい場合に有用です。 - -{/* */ } - -```sql -SELECT * FROM system.server_settings WHERE changed AND name='max_thread_pool_size' -``` - -**関連項目** - -* [設定](../../operations/system-tables/settings.md) -* [設定ファイル](../../operations/configuration-files.md) -* [サーバー設定](../../operations/server-configuration-parameters/settings.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md deleted file mode 100644 index 6dc5454aa96..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'すべての成功・失敗したログインおよびログアウトイベントに関する情報を含む system テーブル。' -keywords: ['system table', 'session_log'] -slug: /operations/system-tables/session_log -title: 'system.session_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.session_log {#systemsession_log} - - - -すべてのログインおよびログアウトイベント(成功・失敗)に関する情報を含みます。 - -Columns: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — ログイン/ログアウトの結果。取りうる値: - * `LoginFailure` — ログインエラー。 - * `LoginSuccess` — ログイン成功。 - * `Logout` — システムからのログアウト。 -* `auth_id` ([UUID](../../sql-reference/data-types/uuid.md)) — 認証 ID。ユーザーがログインするたびに自動生成される UUID。 -* `session_id` ([String](../../sql-reference/data-types/string.md)) — クライアントから [HTTP](../../interfaces/http.md) インターフェイス経由で渡されるセッション ID。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — ログイン/ログアウトの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — ログイン/ログアウトの時刻。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度でのログイン/ログアウト開始時刻。 -* `user` ([String](../../sql-reference/data-types/string.md)) — ユーザー名。 -* `auth_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 認証タイプ。取りうる値: - * `NO_PASSWORD` - * `PLAINTEXT_PASSWORD` - * `SHA256_PASSWORD` - * `DOUBLE_SHA1_PASSWORD` - * `LDAP` - * `KERBEROS` - * `SSL_CERTIFICATE` -* `profiles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — すべてのロールおよび/またはユーザーに設定されたプロファイルの一覧。 -* `roles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — プロファイルが適用されるロールの一覧。 -* `settings` ([Array](../../sql-reference/data-types/array.md)([Tuple](../../sql-reference/data-types/tuple.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md), [String](../../sql-reference/data-types/string.md)))) — クライアントがログイン/ログアウトした際に変更された設定。 -* `client_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — ログイン/ログアウトに使用された IP アドレス。 -* `client_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — ログイン/ログアウトに使用されたクライアントポート番号。 -* `interface` ([Enum8](../../sql-reference/data-types/enum.md)) — ログインが開始されたインターフェイス。取りうる値: - * `TCP` - * `HTTP` - * `gRPC` - * `MySQL` - * `PostgreSQL` -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) または他の TCP クライアントが実行されているクライアントマシンのホスト名。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — `clickhouse-client` または他の TCP クライアントの名前。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` または他の TCP クライアントのリビジョン。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` または他の TCP クライアントのメジャーバージョン。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` または他の TCP クライアントのマイナーバージョン。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` または他の TCP クライアントのパッチバージョン番号。 -* `failure_reason` ([String](../../sql-reference/data-types/string.md)) — ログイン/ログアウトの失敗理由を含む例外メッセージ。 - -**Example** - -Query: - -```sql -SELECT * FROM system.session_log LIMIT 1 FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: LoginSuccess -auth_id: 45e6bd83-b4aa-4a23-85e6-bd83b4aa1a23 -session_id: -event_date: 2021-10-14 -event_time: 2021-10-14 20:33:52 -event_time_microseconds: 2021-10-14 20:33:52.104247 -user: default -auth_type: PLAINTEXT_PASSWORD -profiles: ['default'] -roles: [] -settings: [('load_balancing','random'),('max_memory_usage','10000000000')] -client_address: ::ffff:127.0.0.1 -client_port: 38490 -interface: TCP -client_hostname: -client_name: ClickHouse client -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 0 -failure_reason: -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md deleted file mode 100644 index 87764406fca..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -description: '現在のユーザーのセッション設定情報を含む system テーブル。' -keywords: ['system テーブル', '設定'] -slug: /operations/system-tables/settings -title: 'system.settings' -doc_type: 'reference' ---- - -# system.settings {#systemsettings} - -現在のユーザーのセッション設定に関する情報を含みます。 - -列: - -* `name` ([String](../../sql-reference/data-types/string.md)) — 設定名。 -* `value` ([String](../../sql-reference/data-types/string.md)) — 設定値。 -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 設定が構成ファイルで明示的に定義されたか、または明示的に変更されたかどうかを示します。 -* `description` ([String](../../sql-reference/data-types/string.md)) — 設定の簡潔な説明。 -* `min` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — この設定に対して [constraints](/operations/settings/constraints-on-settings) によって最小値が設定されている場合、その最小値。設定に最小値がない場合は [NULL](/operations/settings/formats#input_format_null_as_default) を含みます。 -* `max` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — この設定に対して [constraints](/operations/settings/constraints-on-settings) によって最大値が設定されている場合、その最大値。設定に最大値がない場合は [NULL](/operations/settings/formats#input_format_null_as_default) を含みます。 -* `disallowed_values` ([Array](/sql-reference/data-types/array)([String](../../sql-reference/data-types/string.md))) — 許可されない値の一覧。 -* `readonly` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 現在のユーザーが設定を変更できるかどうかを示します: - * `0` — 現在のユーザーは設定を変更できます。 - * `1` — 現在のユーザーは設定を変更できません。 -* `default` ([String](../../sql-reference/data-types/string.md)) — 設定のデフォルト値。 -* `alias_for` ([String](../../sql-reference/data-types/string.md)) — この設定が他の設定のエイリアスである場合、その元の設定名。 -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) - 設定が廃止済みかどうかを示します。 -* `tier` ([Enum8](../../sql-reference/data-types/enum.md)) — この機能のサポートレベル。ClickHouse の機能は、その開発の現在の状況および使用時に期待できる安定性に応じて階層化されています。値: - * `'Production'` — 機能は安定しており、安全に使用でき、他の **本番** 機能との相互作用にも問題がありません。 - * `'Beta'` — 機能は安定しており安全です。他の機能と組み合わせて使用した場合の結果は不明であり、正しさは保証されません。テストおよびレポートを歓迎します。 - * `'Experimental'` — 機能は開発中です。開発者および ClickHouse の愛好家のみを対象としています。機能が動作するかどうかは不明であり、いつでも削除される可能性があります。 - * `'Obsolete'` — もはやサポートされていません。すでに削除されたか、今後のリリースで削除される予定です。 - -**例** - -次の例は、名前に `min_i` を含む設定に関する情報を取得する方法を示しています。 - -```sql -SELECT * -FROM system.settings -WHERE name LIKE '%min_insert_block_size_%' -FORMAT Vertical -``` - -```text -Row 1: -────── -name: min_insert_block_size_rows -value: 1048449 -changed: 0 -description: `INSERT`クエリでテーブルに挿入可能なブロック内の最小行数を設定します。小さいブロックは大きなブロックに統合されます。 - -設定可能な値: - -- 正の整数。 -- 0 — 統合無効。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 1048449 -alias_for: -is_obsolete: 0 -tier: Production - -Row 2: -────── -name: min_insert_block_size_bytes -value: 268402944 -changed: 0 -description: `INSERT`クエリでテーブルに挿入可能なブロック内の最小バイト数を設定します。小さいブロックは大きなブロックに統合されます。 - -設定可能な値: - -- 正の整数。 -- 0 — 統合無効。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 268402944 -alias_for: -is_obsolete: 0 -tier: Production -``` - -Row 3: -────── -name: min_insert_block_size_rows_for_materialized_views -value: 0 -changed: 0 -description: `INSERT` クエリでテーブルに挿入できるブロック内の最小行数を設定します。より小さいサイズのブロックは、より大きなブロックにまとめられます。この設定は、[マテリアライズドビュー](../../sql-reference/statements/create/view.md) に挿入されるブロックに対してのみ適用されます。この設定を調整することで、マテリアライズドビューへの書き込み時のブロックのまとめを制御し、過剰なメモリ使用を回避できます。 - -Possible values: - -* 任意の正の整数。 -* 0 — まとめ処理を無効にします。 - -**See Also** - -* [min_insert_block_size_rows](/operations/settings/settings#min_insert_block_size_rows) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -Row 4: -────── -name: min_insert_block_size_bytes_for_materialized_views -value: 0 -changed: 0 -description: `INSERT` クエリでテーブルに挿入できるブロック内の最小バイト数を設定します。より小さいサイズのブロックは、より大きなブロックにまとめられます。この設定は、[マテリアライズドビュー](../../sql-reference/statements/create/view.md) に挿入されるブロックに対してのみ適用されます。この設定を調整することで、マテリアライズドビューへの書き込み時のブロックのまとめを制御し、過剰なメモリ使用を回避できます。 - -Possible values: - -* 任意の正の正の整数。 -* 0 — まとめ処理を無効にします。 - -**See also** - -* [min_insert_block_size_bytes](/operations/settings/settings#min_insert_block_size_bytes) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -```` - -`WHERE changed` は、例えば以下を確認する際に有用です: - -- 設定ファイルの設定が正しく読み込まれ、使用されているか -- 現在のセッションで変更された設定 - - - -```sql -SELECT * FROM system.settings WHERE changed AND name='load_balancing' -```` - -**関連項目** - -* [設定](/operations/system-tables/overview#system-tables-introduction) -* [クエリの権限](/operations/settings/permissions-for-queries) -* [設定の制約](../../operations/settings/constraints-on-settings.md) -* [SHOW SETTINGS](../../sql-reference/statements/show.md#show-settings) ステートメント diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md deleted file mode 100644 index 9f1be6eec28..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: '過去の ClickHouse バージョンで行われた設定変更に関する情報を含む system テーブル。' -keywords: ['system テーブル', 'settings_changes'] -slug: /operations/system-tables/settings_changes -title: 'system.settings_changes' -doc_type: 'reference' ---- - -# system.settings_changes {#systemsettings_changes} - -以前の ClickHouse バージョンにおける設定変更に関する情報を保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * -FROM system.settings_changes -WHERE version = '23.5' -FORMAT Vertical -``` - -```text -Row 1: -────── -type: Core -version: 23.5 -changes: [('input_format_parquet_preserve_order','1','0','Parquetリーダーが並列性向上のために行を並べ替えることを許可します。'),('parallelize_output_from_storages','0','1','file/url/s3などから読み取るクエリ実行時の並列処理を許可します。これにより行が並べ替えられる可能性があります。'),('use_with_fill_by_sorting_prefix','0','1','ORDER BY句内のWITH FILL列より前の列がソートプレフィックスを形成します。ソートプレフィックスで異なる値を持つ行は独立して埋められます。'),('output_format_parquet_compliant_nested_types','0','1','出力Parquetファイルスキーマ内の内部フィールド名を変更します。')] -``` - -**関連項目** - -* [Settings](/operations/system-tables/overview#system-tables-introduction) -* [system.settings](settings.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md deleted file mode 100644 index 0b72fa620ee..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: 'SETTINGS PROFILE の内容、すなわち制約、この設定が適用されるロールおよびユーザー、親設定プロファイルを表す system テーブル。' -keywords: ['system テーブル', 'settings_profile_elements'] -slug: /operations/system-tables/settings_profile_elements -title: 'system.settings_profile_elements' -doc_type: 'reference' ---- - -# system.settings_profile_elements {#systemsettings_profile_elements} - -設定プロファイルの内容を表します: - -* 制約 -* 設定が適用されるロールとユーザー -* 親設定プロファイル - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md deleted file mode 100644 index 7c2b64913b2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '構成済み設定プロファイルのプロパティを含むシステムテーブル。' -keywords: ['システムテーブル', 'settings_profiles'] -slug: /operations/system-tables/settings_profiles -title: 'system.settings_profiles' -doc_type: 'reference' ---- - -# system.settings_profiles {#systemsettings_profiles} - -構成済みの設定プロファイルのプロパティを保持します。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連項目 {#see-also} - -- [SHOW PROFILES](/sql-reference/statements/show#show-profiles) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md deleted file mode 100644 index 3d61b82f74a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -description: 'すべてのサーバースレッドのスタックトレースを含むシステムテーブルです。開発者がサーバーの内部状態を調査できるようにします。' -keywords: ['system table', 'stack_trace'] -slug: /operations/system-tables/stack_trace -title: 'system.stack_trace' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.stack_trace {#systemstack_trace} - - - -すべてのサーバースレッドのスタックトレースを保持します。開発者がサーバーの状態を把握するために利用できます。 - -スタックフレームを解析するには、`addressToLine`、`addressToLineWithInlines`、`addressToSymbol`、`demangle` の各[イントロスペクション関数](../../sql-reference/functions/introspection.md)を使用します。 - -列: - -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — スレッド名。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — スレッド識別子。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — [query_log](../system-tables/query_log.md) システムテーブルから、実行されていたクエリの詳細を取得するために使用できるクエリ識別子。 -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 呼び出されたメソッドが格納されている物理アドレスの一覧を表す[スタックトレース](https://en.wikipedia.org/wiki/Stack_trace)。 - -:::tip -[現在実行中のスレッドを確認する方法](/knowledgebase/find-expensive-queries)や[トラブルシューティングに役立つクエリ](/knowledgebase/useful-queries-for-troubleshooting)など、便利なクエリが掲載されている Knowledge Base を参照してください。 -::: - -**例** - -イントロスペクション関数の有効化: - -```sql -SET allow_introspection_functions = 1; -``` - -ClickHouse のオブジェクトファイルからシンボルを取得する: - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), trace) AS all SELECT thread_name, thread_id, query_id, arrayStringConcat(all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Row 1: -────── -thread_name: QueryPipelineEx -thread_id: 743490 -query_id: dc55a564-febb-4e37-95bb-090ef182c6f1 -res: memcpy -large_ralloc -arena_ralloc -do_rallocx -Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) -HashTable, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>::resize(unsigned long, unsigned long) -void DB::Aggregator::executeImplBatch, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>>(DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>&, DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>::State&, DB::Arena*, unsigned long, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const -DB::Aggregator::executeImpl(DB::AggregatedDataVariants&, unsigned long, unsigned long, std::__1::vector>&, DB::Aggregator::AggregateFunctionInstruction*, bool, bool, char*) const -DB::Aggregator::executeOnBlock(std::__1::vector::immutable_ptr, std::__1::allocator::immutable_ptr>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::__1::vector>&, std::__1::vector>, std::__1::allocator>>>&, bool&) const -DB::AggregatingTransform::work() -DB::ExecutionThreadContext::executeTask() -DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic*) -void std::__1::__function::__policy_invoker::__call_impl>(std::__1::__function::__policy_storage const*) -ThreadPoolImpl>::worker(std::__1::__list_iterator, void*>) -void std::__1::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) -void* std::__1::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>>(void*) -``` - -ClickHouse ソースコード内のファイル名と行番号の取得: - -```sql -WITH arrayMap(x -> addressToLine(x), trace) AS all, arrayFilter(x -> x LIKE '%/dbms/%', all) AS dbms SELECT thread_name, thread_id, query_id, arrayStringConcat(notEmpty(dbms) ? dbms : all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Row 1: -────── -thread_name: clickhouse-serv - -thread_id: 686 -query_id: cad353e7-1c29-4b2e-949f-93e597ab7a54 -res: /lib/x86_64-linux-gnu/libc-2.27.so -/build/obj-x86_64-linux-gnu/../src/Storages/System/StorageSystemStackTrace.cpp:182 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:656 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:1338 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:751 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:224 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:192 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:384 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:643 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:251 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1197 -/build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223 -/lib/x86_64-linux-gnu/libpthread-2.27.so -/lib/x86_64-linux-gnu/libc-2.27.so -``` - -**関連項目** - -* [Introspection Functions](../../sql-reference/functions/introspection.md) — 利用可能なイントロスペクション関数とその使い方。 -* [system.trace_log](../system-tables/trace_log.md) — サンプリングクエリプロファイラで収集されたスタックトレースを含みます。 -* [arrayMap](/sql-reference/functions/array-functions#arrayMap) — `arrayMap` 関数の説明と使用例。 -* [arrayFilter](/sql-reference/functions/array-functions#arrayFilter) — `arrayFilter` 関数の説明と使用例。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md deleted file mode 100644 index 281497c4514..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'サーバー構成で定義されているストレージポリシーおよびボリュームに関する情報を格納する system テーブル。' -keywords: ['system table', 'storage_policies'] -slug: /operations/system-tables/storage_policies -title: 'system.storage_policies' -doc_type: 'reference' ---- - -# system.storage_policies {#systemstorage_policies} - -[サーバー構成](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure)で定義されたストレージポリシーおよびボリュームに関する情報を含みます。 - -Columns: - -* `policy_name` ([String](../../sql-reference/data-types/string.md)) — ストレージポリシーの名前。 -* `volume_name` ([String](../../sql-reference/data-types/string.md)) — ストレージポリシーで定義されているボリューム名。 -* `volume_priority` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 構成内でのボリュームの順序番号。データはこの優先度に従ってボリュームに格納されます。つまり、INSERT およびマージ時のデータは、より低い優先度のボリュームに書き込まれます(他のルール: TTL、`max_data_part_size`、`move_factor` を考慮)。 -* `disks` ([Array(String)](../../sql-reference/data-types/array.md)) — ストレージポリシーで定義されているディスク名。 -* `volume_type` ([Enum8](../../sql-reference/data-types/enum.md)) — ボリュームの種類。次のいずれかの値を取ります: - * `JBOD` - * `SINGLE_DISK` - * `UNKNOWN` -* `max_data_part_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ボリューム上のディスクに保存できるデータパーツの最大サイズ(0 — 制限なし)。 -* `move_factor` ([Float64](../../sql-reference/data-types/float.md)) — 空きディスク容量の比率。この比率が構成パラメータで設定された値を超えると、ClickHouse は順番に次のボリュームへデータの移動を開始します。 -* `prefer_not_to_merge` ([UInt8](../../sql-reference/data-types/int-uint.md)) — `prefer_not_to_merge` 設定の値。常に false であるべきです。この設定が有効になっている場合は、設定を誤っています。 -* `perform_ttl_move_on_insert` ([UInt8](../../sql-reference/data-types/int-uint.md)) — `perform_ttl_move_on_insert` 設定の値。データパーツの INSERT 時の TTL による移動を無効にします。デフォルトでは、TTL の移動ルールですでに期限切れになっているデータパーツを挿入した場合、そのパーツは直ちに移動ルールで指定されたボリューム/ディスクに移動されます。移動先のボリューム/ディスクが遅い(例: S3)場合、INSERT が大幅に遅くなる可能性があります。 -* `load_balancing` ([Enum8](../../sql-reference/data-types/enum.md)) — ディスクバランシングのポリシー。次のいずれかの値を取ります: - * `ROUND_ROBIN` - * `LEAST_USED` - -ストレージポリシーに複数のボリュームが含まれている場合、各ボリュームの情報はテーブルの個別の行として保存されます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md deleted file mode 100644 index 37b59bcc84e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'C++ のエキスパートおよび ClickHouse エンジニア向けの、`clickhouse` バイナリのイントロスペクション用情報を含む system テーブル。' -keywords: ['system table', 'symbols'] -slug: /operations/system-tables/symbols -title: 'system.symbols' -doc_type: 'reference' ---- - -`clickhouse` バイナリのイントロスペクション用の情報を含みます。アクセスには introspection 権限が必要です。 -このテーブルは C++ のエキスパートおよび ClickHouse エンジニアにのみ有用です。 - -カラム: - -* `symbol` ([String](../../sql-reference/data-types/string.md)) — バイナリ内のシンボル名。マングルされた形式です。可読な名前を取得するには `demangle(symbol)` を適用できます。 -* `address_begin` ([UInt64](../../sql-reference/data-types/int-uint.md)) — バイナリ内でのシンボルの開始アドレス。 -* `address_end` ([UInt64](../../sql-reference/data-types/int-uint.md)) — バイナリ内でのシンボルの終了アドレス。 -* `name` ([String](../../sql-reference/data-types/string.md)) — `event` のエイリアス。 - -**例** - -```sql -SELECT address_begin, address_end - address_begin AS size, demangle(symbol) FROM system.symbols ORDER BY size DESC LIMIT 10 -``` - -```text -┌─address_begin─┬─────size─┬─demangle(symbol)──────────────────────────────────────────────────────────────────┐ -│ 25000976 │ 29466000 │ icudt70_dat │ -│ 400605288 │ 2097272 │ arena_emap_global │ -│ 18760592 │ 1048576 │ CLD2::kQuadChrome1015_2 │ -│ 9807152 │ 884808 │ TopLevelDomainLookupHash::isValid(char const*, unsigned long)::wordlist │ -│ 57442432 │ 850608 │ llvm::X86Insts │ -│ 55682944 │ 681360 │ (anonymous namespace)::X86DAGToDAGISel::SelectCode(llvm::SDNode*)::MatcherTable │ -│ 55130368 │ 502840 │ (anonymous namespace)::X86InstructionSelector::getMatchTable() const::MatchTable0 │ -│ 402930616 │ 404032 │ qpl::ml::dispatcher::hw_dispatcher::get_instance()::instance │ -│ 274131872 │ 356795 │ DB::SettingsTraits::Accessor::instance()::$_0::operator()() const │ -│ 58293040 │ 249424 │ llvm::X86InstrNameData │ -└───────────────┴──────────┴───────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md deleted file mode 100644 index 1205c37bdd6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'このテーブルには、ClickHouse サーバーに関する警告メッセージが格納されています。' -keywords: [ 'システムテーブル', '警告' ] -slug: /operations/system-tables/system_warnings -title: 'system.warnings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.warnings {#systemwarnings} - - - -このテーブルは ClickHouse サーバーに関する警告を表示します。 -同種の警告は 1 つの警告にまとめられます。 -たとえば、アタッチされているデータベースの数 N が設定可能なしきい値 T を超えた場合、N 個の個別エントリではなく、現在の値 N を含む 1 つのエントリのみが表示されます。 -現在の値がしきい値を下回ると、そのエントリはテーブルから削除されます。 - -このテーブルは次の設定で調整できます: - -* [max_table_num_to_warn](../server-configuration-parameters/settings.md#max_table_num_to_warn) -* [max_database_num_to_warn](../server-configuration-parameters/settings.md#max_database_num_to_warn) -* [max_dictionary_num_to_warn](../server-configuration-parameters/settings.md#max_dictionary_num_to_warn) -* [max_view_num_to_warn](../server-configuration-parameters/settings.md#max_view_num_to_warn) -* [max_part_num_to_warn](../server-configuration-parameters/settings.md#max_part_num_to_warn) -* [max_pending_mutations_to_warn](../server-configuration-parameters/settings.md#max_pending_mutations_to_warn) -* [max_pending_mutations_execution_time_to_warn](/operations/server-configuration-parameters/settings#max_pending_mutations_execution_time_to_warn) -* [max_named_collection_num_to_warn](../server-configuration-parameters/settings.md#max_named_collection_num_to_warn) -* [resource_overload_warnings](/operations/settings/server-overload#resource-overload-warnings) - -列: - -* `message` ([String](../../sql-reference/data-types/string.md)) — 警告メッセージ。 -* `message_format_string` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — メッセージの整形に使用されるフォーマット文字列。 - -**例** - -クエリ: - -```sql - SELECT * FROM system.warnings LIMIT 2 \G; -``` - -結果: - -```text -Row 1: -────── -message: アクティブなパーツ数が10を超えています。 -message_format_string: アクティブなパーツ数が{}を超えています。 - -Row 2: -────── -message: アタッチされているデータベース数が2を超えています。 -message_format_string: アタッチされているデータベース数が{}を超えています。 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md deleted file mode 100644 index 791a64a9109..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'サーバーがサポートするテーブルエンジンと、その対応機能の説明を含むシステムテーブル。' -keywords: ['システムテーブル', 'table_engines'] -slug: /operations/system-tables/table_engines -title: 'system.table_engines' -doc_type: 'reference' ---- - -# system.table_engines {#systemtable_engines} - -サーバーがサポートするテーブルエンジンと、それぞれの機能サポート情報の説明を含みます。 - -このテーブルには次の列が含まれます(かっこ内は列の型を示します)。 - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -例: - -```sql -SELECT * -FROM system.table_engines -WHERE name IN ('Kafka', 'MergeTree', 'ReplicatedCollapsingMergeTree') -``` - -```text -┌─name──────────────────────────┬─supports_settings─┬─supports_skipping_indices─┬─supports_sort_order─┬─supports_ttl─┬─supports_replication─┬─supports_deduplication─┬─supports_parallel_insert─┐ -│ MergeTree │ 1 │ 1 │ 1 │ 1 │ 0 │ 0 │ 1 │ -│ Kafka │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ -│ ReplicatedCollapsingMergeTree │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ -└───────────────────────────────┴───────────────────┴───────────────────────────┴─────────────────────┴──────────────┴──────────────────────┴────────────────────────┴──────────────────────────┘ -``` - -**関連項目** - -* MergeTree ファミリーの[クエリ句](../../engines/table-engines/mergetree-family/mergetree.md#mergetree-query-clauses) -* Kafka の[設定](/engines/table-engines/integrations/kafka#creating-a-table) -* Join の[設定](../../engines/table-engines/special/join.md#join-limitations-and-settings) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md deleted file mode 100644 index 881eef11ecb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -description: 'サーバーが認識している各テーブルのメタデータを含む system テーブルです。' -keywords: ['system テーブル', 'テーブル'] -slug: /operations/system-tables/tables -title: 'system.tables' -doc_type: 'reference' ---- - -# system.tables {#systemtables} - -サーバーが把握している各テーブルのメタデータを含みます。 - -[Detached](../../sql-reference/statements/detach.md) テーブルは `system.tables` には表示されません。 - -[Temporary tables](../../sql-reference/statements/create/table.md#temporary-tables) は、それらが作成されたセッション内でのみ `system.tables` に表示されます。これらは、`database` フィールドが空で、`is_temporary` フラグが有効になっている状態で表示されます。 - -列: - -* `database` ([String](../../sql-reference/data-types/string.md)) — テーブルが属するデータベース名。 - -* `name` ([String](../../sql-reference/data-types/string.md)) — テーブルの名前。 - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — テーブル uuid(Atomic データベース)。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) — テーブルエンジンの名前(パラメータを含まない)。 - -* `is_temporary` ([UInt8](../../sql-reference/data-types/int-uint.md)) - テーブルが一時テーブルであるかどうかを示すフラグ。 - -* `data_paths` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - ファイルシステム上のテーブルデータへのパス。 - -* `metadata_path` ([String](../../sql-reference/data-types/string.md)) - ファイルシステム内のテーブルのメタデータへのパス。 - -* `metadata_modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - テーブルのメタデータが最後に変更された時刻。 - -* `metadata_version` ([Int32](../../sql-reference/data-types/int-uint.md)) - ReplicatedMergeTree テーブルに対するメタデータバージョン。ReplicatedMergeTree 以外のテーブルでは 0。 - -* `dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 依存しているデータベース。 - -* `dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - テーブルの依存関係(このテーブルをソースとする[マテリアライズドビュー](/sql-reference/statements/create/view#materialized-view))。 - -* `create_table_query` ([String](../../sql-reference/data-types/string.md)) - テーブルの作成に使用したクエリ。 - -* `engine_full` ([String](../../sql-reference/data-types/string.md)) - テーブルエンジンのパラメータ。 - -* `as_select` ([String](../../sql-reference/data-types/string.md)) - ビューを定義するための `SELECT` クエリ。 - -* `parameterized_view_parameters` ([Array](../../sql-reference/data-types/array.md) of [Tuple](../../sql-reference/data-types/tuple.md)) — パラメータ化されたビューのパラメータ。 - -* `partition_key` ([String](../../sql-reference/data-types/string.md)) - テーブルで指定されたパーティションキーの式。 - -* `sorting_key` ([String](../../sql-reference/data-types/string.md)) - テーブルに指定されたソートキー式。 - -* `primary_key` ([String](../../sql-reference/data-types/string.md)) - テーブルで指定された主キー式。 - -* `sampling_key` ([String](../../sql-reference/data-types/string.md)) - テーブルで指定されたサンプリングキーの式。 - -* `storage_policy` ([String](../../sql-reference/data-types/string.md)) - ストレージポリシー: - - * [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) - * [Distributed](/engines/table-engines/special/distributed) - -* `total_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - テーブル(基盤となる `Buffer` テーブルを含む)内の行数を正確かつ迅速に判定できる場合は、その総行数。できない場合は `NULL`。 - -* `total_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - テーブルのストレージ上に保存されているバイト数の正確な値をすばやく取得できる場合は、インデックスおよびプロジェクションを含む合計バイト数。それが不可能な場合は `NULL`(基盤となるストレージ自体のサイズは含まれない)。 - - * テーブルがディスク上にデータを保存している場合、ディスク上で使用されている領域(圧縮後)を返します。 - * テーブルがメモリ上にデータを保存している場合、メモリで使用されているバイト数のおおよその値を返します。 - -* `total_bytes_uncompressed` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 圧縮されていないバイト数の合計(インデックスおよびプロジェクションを含む)。ストレージ上のテーブルについて、パートのチェックサムから正確なバイト数をすばやく算出できる場合はその値、それ以外の場合は `NULL`(下層のストレージが存在する場合でも、それは考慮しない)。 - -* `lifetime_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - サーバー起動後に INSERT された行の合計数(`Buffer` テーブルに対してのみ)。 - -* `lifetime_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - サーバー起動後に INSERT されたバイト数の総計(`Buffer` テーブルに対してのみ)。 - -* `comment` ([String](../../sql-reference/data-types/string.md)) - テーブルに関するコメント。 - -* `has_own_data` ([UInt8](../../sql-reference/data-types/int-uint.md)) — テーブル自体がディスク上にデータを保存しているかどうか、または他のデータソースにのみアクセスしているかどうかを示すフラグです。 - -* `loading_dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - データベースの読み込み依存関係(現在のオブジェクトより前に読み込まれている必要のあるオブジェクトの一覧)。 - -* `loading_dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - テーブル読み込み時の依存関係(現在のオブジェクトより前に読み込む必要があるオブジェクトのリスト)。 - -* `loading_dependent_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 依存してロードされるデータベース。 - -* `loading_dependent_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 依存関係のあるロード先テーブル。 - -`system.tables` テーブルは、`SHOW TABLES` クエリを実装する際に使用されます。 - -**例** - -```sql -SELECT * FROM system.tables LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: base -name: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/store/81b/81b1c20a-b7c6-4116-a2ce-7583fb6b6736/'] -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -metadata_modification_time: 2021-01-25 19:14:32 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE base.t1 (`n` UInt64) ENGINE = MergeTree ORDER BY n SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY n SETTINGS index_granularity = 8192 -as_select: SELECT database AS table_catalog -partition_key: -sorting_key: n -primary_key: n -sampling_key: -storage_policy: default -total_rows: 1 -total_bytes: 99 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] - -Row 2: -────── -database: default -name: 53r93yleapyears -uuid: 00000000-0000-0000-0000-000000000000 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/data/default/53r93yleapyears/'] -metadata_path: /var/lib/clickhouse/metadata/default/53r93yleapyears.sql -metadata_modification_time: 2020-09-23 09:05:36 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE default.`53r93yleapyears` (`id` Int8, `febdays` Int8) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY id SETTINGS index_granularity = 8192 -as_select: SELECT name AS catalog_name -partition_key: -sorting_key: id -primary_key: id -sampling_key: -storage_policy: default -total_rows: 2 -total_bytes: 155 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md deleted file mode 100644 index cd631d7f468..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -description: 'ログエントリを保持するシステムテーブル。' -keywords: ['system table', 'text_log'] -slug: /operations/system-tables/text_log -title: 'system.text_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.text_log {#systemtext_log} - - - -ログエントリを含むテーブルです。このテーブルに出力されるログレベルは、`text_log.level` サーバー設定で制限できます。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `event_date` (Date) — エントリの日付。 -* `event_time` (DateTime) — エントリの時刻。 -* `event_time_microseconds` (DateTime64) — マイクロ秒精度でのエントリの時刻。 -* `microseconds` (UInt32) — エントリのマイクロ秒。 -* `thread_name` (String) — ログ出力を行ったスレッド名。 -* `thread_id` (UInt64) — OS のスレッド ID。 -* `level` (`Enum8`) — エントリのレベル。取りうる値: - * `1` または `'Fatal'`。 - * `2` または `'Critical'`。 - * `3` または `'Error'`。 - * `4` または `'Warning'`。 - * `5` または `'Notice'`。 - * `6` または `'Information'`。 - * `7` または `'Debug'`。 - * `8` または `'Trace'`。 -* `query_id` (String) — クエリの ID。 -* `logger_name` (LowCardinality(String)) — ロガー名(例: `DDLWorker`)。 -* `message` (String) — メッセージ本体。 -* `revision` (UInt32) — ClickHouse のリビジョン。 -* `source_file` (LowCardinality(String)) — ログ出力を行ったソースファイル。 -* `source_line` (UInt64) — ログ出力を行ったソースコード行番号。 -* `message_format_string` (LowCardinality(String)) — メッセージの整形に使用されたフォーマット文字列。 -* `value1` (String) - メッセージの整形に使用された引数 1。 -* `value2` (String) - メッセージの整形に使用された引数 2。 -* `value3` (String) - メッセージの整形に使用された引数 3。 -* `value4` (String) - メッセージの整形に使用された引数 4。 -* `value5` (String) - メッセージの整形に使用された引数 5。 -* `value6` (String) - メッセージの整形に使用された引数 6。 -* `value7` (String) - メッセージの整形に使用された引数 7。 -* `value8` (String) - メッセージの整形に使用された引数 8。 -* `value9` (String) - メッセージの整形に使用された引数 9。 -* `value10` (String) - メッセージの整形に使用された引数 10。 - -**例** - -```sql -SELECT * FROM system.text_log LIMIT 1 \G -``` - -```text -行 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:07 -event_time_microseconds: 2020-09-10 11:23:07.871397 -microseconds: 871397 -thread_name: clickhouse-serv -thread_id: 564917 -level: Information -query_id: -logger_name: DNSCacheUpdater -message: Update period 15 seconds -revision: 54440 -source_file: /ClickHouse/src/Interpreters/DNSCacheUpdater.cpp; void DB::DNSCacheUpdater::start() -source_line: 45 -message_format_string: Update period {} seconds -value1: 15 -value2: -value3: -value4: -value5: -value6: -value7: -value8: -value9: -value10: -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md deleted file mode 100644 index a04ddc7cbbd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'ClickHouse サーバーでサポートされているタイムゾーンの一覧を含む system テーブル。' -keywords: ['system テーブル', 'time_zones'] -slug: /operations/system-tables/time_zones -title: 'system.time_zones' -doc_type: 'reference' ---- - -# system.time_zones {#systemtime_zones} - -ClickHouse サーバーがサポートしているタイムゾーンの一覧を保持します。この一覧は ClickHouse のバージョンによって異なる場合があります。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT * FROM system.time_zones LIMIT 10 -``` - -```text -┌─time_zone──────────┐ -│ Africa/Abidjan │ -│ Africa/Accra │ -│ Africa/Addis_Ababa │ -│ Africa/Algiers │ -│ Africa/Asmara │ -│ Africa/Asmera │ -│ Africa/Bamako │ -│ Africa/Bangui │ -│ Africa/Banjul │ -│ Africa/Bissau │ -└────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md deleted file mode 100644 index ff9f80cfe96..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: 'サンプリングクエリプロファイラによって収集されたスタックトレースを含むシステムテーブル。' -keywords: ['システムテーブル', 'trace_log'] -slug: /operations/system-tables/trace_log -title: 'system.trace_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.trace_log {#systemtrace_log} - - - -[サンプリングクエリプロファイラ](../../operations/optimizing-performance/sampling-query-profiler.md)によって収集されたスタックトレースを含みます。 - -ClickHouse は、サーバー設定セクション [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log) が設定されている場合にこのテーブルを作成します。関連する設定も参照してください: [query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns)、[query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns)、[memory_profiler_step](../../operations/settings/settings.md#memory_profiler_step)、[memory_profiler_sample_probability](../../operations/settings/settings.md#memory_profiler_sample_probability)、[trace_profile_events](../../operations/settings/settings.md#trace_profile_events)。 - -ログを分析するには、`addressToLine`、`addressToLineWithInlines`、`addressToSymbol`、`demangle` といったイントロスペクション関数を使用します。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 - -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — サンプリング時点の日付。 - -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — サンプリング時点のタイムスタンプ。 - -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — マイクロ秒精度のサンプリング時点のタイムスタンプ。 - -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ナノ秒単位のサンプリング時点のタイムスタンプ。 - -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse サーバービルドのリビジョン。 - - `clickhouse-client` でサーバーに接続すると、`Connected to ClickHouse server version 19.18.1.` のような文字列が表示されます。このフィールドにはサーバーの `version` ではなく `revision` が格納されます。 - -* `trace_type` ([Enum8](../../sql-reference/data-types/enum.md)) — トレースの種類: - * `Real` はウォールクロック時間に基づくスタックトレース収集を表します。 - * `CPU` は CPU 時間に基づくスタックトレース収集を表します。 - * `Memory` はメモリアロケーションが次のウォーターマークを超えたときのアロケーションおよび解放の収集を表します。 - * `MemorySample` はランダムなアロケーションおよび解放の収集を表します。 - * `MemoryPeak` はピークメモリ使用量の更新の収集を表します。 - * `ProfileEvent` はプロファイルイベントのインクリメントの収集を表します。 - * `JemallocSample` は jemalloc サンプルの収集を表します。 - * `MemoryAllocatedWithoutCheck` は、任意のメモリ制限を無視して行われる大きなアロケーション (>16MiB) の収集を表します (ClickHouse 開発者専用)。 - -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — スレッド識別子。 - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — [query_log](/operations/system-tables/query_log) システムテーブルから、実行されていたクエリの詳細を取得するために使用できるクエリ識別子。 - -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — サンプリング時点のスタックトレース。各要素は ClickHouse サーバープロセス内の仮想メモリアドレスです。 - -* `size` ([Int64](../../sql-reference/data-types/int-uint.md)) - トレースタイプが `Memory`、`MemorySample`、`MemoryPeak` の場合は割り当てられたメモリ量、それ以外のトレースタイプの場合は 0。 - -* `event` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) - トレースタイプが `ProfileEvent` の場合は更新されたプロファイルイベント名、それ以外のトレースタイプの場合は空文字列。 - -* `increment` ([UInt64](../../sql-reference/data-types/int-uint.md)) - トレースタイプが `ProfileEvent` の場合はプロファイルイベントのインクリメント量、それ以外のトレースタイプの場合は 0。 - -* `symbols`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — シンボル化が有効な場合、`trace` に対応するデマングル済みシンボル名を含みます。 - -* `lines`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — シンボル化が有効な場合、`trace` に対応するファイル名と行番号を含む文字列を含みます。 - -シンボル化は、サーバーの設定ファイル内の `trace_log` 配下の `symbolize` で有効または無効にできます。 - -**例** - -```sql -SELECT * FROM system.trace_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:09 -event_time_microseconds: 2020-09-10 11:23:09.872924 -timestamp_ns: 1599762189872924510 -revision: 54440 -trace_type: Memory -thread_id: 564963 -query_id: -trace: [371912858,371912789,371798468,371799717,371801313,371790250,624462773,566365041,566440261,566445834,566460071,566459914,566459842,566459580,566459469,566459389,566459341,566455774,371993941,371988245,372158848,372187428,372187309,372187093,372185478,140222123165193,140222122205443] -size: 5244400 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md deleted file mode 100644 index 0ab9a7a3874..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md +++ /dev/null @@ -1,178 +0,0 @@ ---- -description: 'Unicode 文字とその属性の一覧を含むシステムテーブル' -keywords: ['system table', 'unicode'] -slug: /operations/system-tables/unicode -title: 'system.unicode' -doc_type: 'reference' ---- - -# system.unicode {#systemunicode} - -`system.unicode` テーブルは、Unicode 文字およびそのプロパティに関する情報を提供する仮想テーブルです([https://unicode-org.github.io/icu/userguide/strings/properties.html](https://unicode-org.github.io/icu/userguide/strings/properties.html) を参照)。このテーブルは要求時に動的に生成されます。 - -Columns - -:::note -ICU ドキュメントにおける Unicode コードポイントのプロパティ名は、スネークケースに変換されています。 -::: - -* `code_point` ([String](../../sql-reference/data-types/string.md)) — コードポイントの UTF-8 表現。 -* `code_point_value` ([Int32](../../sql-reference/data-types/int-uint.md)) — コードポイントの数値表現。 -* `notation` ([String](../../sql-reference/data-types/string.md)) — コードポイントの Unicode 表記。 -* Binary Properties ([UInt8](../../sql-reference/data-types/int-uint.md)) - コードポイントのバイナリプロパティ。 - * `alphabetic`, `ascii_hex_digit`, `case_ignorable`... -* Enumerated Properties ([Int32](../../sql-reference/data-types/int-uint.md)) - コードポイントの列挙プロパティ。 - * `bidi_class`, `bidi_paired_bracket_type`, `block`... -* String Properties ([String](../../sql-reference/data-types/string.md)) - コードポイントの文字列プロパティ(ASCII 文字列または Unicode 文字列、あるいはコードポイント) - * `case_folding`, `decomposition_mapping`, `name`... - -:::note -Mapping には多少特殊な点があるため、ICU のドキュメントを参照してください。たとえば、simple_uppercase_mapping と uppercase_mapping は完全に同一ではありません。また、言語固有の mapping は実装されていません(例:トルコ語では i の大文字は "İ" (U+0130) です)。 -::: - -* `numeric_value` ([Float64](../../sql-reference/data-types/float.md)) - コードポイントの数値表現。 -* `script_extensions` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) - コードポイントの script extensions。 -* `identifier_type` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) - コードポイントの identifier type。 -* `general_category_mask` ([Int32](../../sql-reference/data-types/int-uint.md)) - コードポイントの general category mask。 - -**Example** - -```sql -SELECT * FROM system.unicode WHERE code_point = 'a' LIMIT 1; -``` - -```text -Row 1: -────── -code_point: a -code_point_value: 97 -notation: U+0061 -alphabetic: 1 -ascii_hex_digit: 1 -bidi_control: 0 -bidi_mirrored: 0 -dash: 0 -default_ignorable_code_point: 0 -deprecated: 0 -diacritic: 0 -extender: 0 -full_composition_exclusion: 0 -grapheme_base: 1 -grapheme_extend: 0 -grapheme_link: 0 -hex_digit: 1 -hyphen: 0 -id_continue: 1 -id_start: 1 -ideographic: 0 -ids_binary_operator: 0 -ids_trinary_operator: 0 -join_control: 0 -logical_order_exception: 0 -lowercase: 1 -math: 0 -noncharacter_code_point: 0 -quotation_mark: 0 -radical: 0 -soft_dotted: 0 -terminal_punctuation: 0 -unified_ideograph: 0 -uppercase: 0 -white_space: 0 -xid_continue: 1 -xid_start: 1 -case_sensitive: 1 -sentence_terminal: 0 -variation_selector: 0 -nfd_inert: 1 -nfkd_inert: 1 -nfc_inert: 0 -nfkc_inert: 0 -segment_starter: 1 -pattern_syntax: 0 -pattern_white_space: 0 -alnum: 1 -blank: 0 -graph: 1 -print: 1 -xdigit: 1 -cased: 1 -case_ignorable: 0 -changes_when_lowercased: 0 -changes_when_uppercased: 1 -changes_when_titlecased: 1 -changes_when_casefolded: 0 -changes_when_casemapped: 1 -changes_when_nfkc_casefolded: 0 -emoji: 0 -emoji_presentation: 0 -emoji_modifier: 0 -emoji_modifier_base: 0 -emoji_component: 0 -regional_indicator: 0 -prepended_concatenation_mark: 0 -extended_pictographic: 0 -basic_emoji: 0 -emoji_keycap_sequence: 0 -rgi_emoji_modifier_sequence: 0 -rgi_emoji_flag_sequence: 0 -rgi_emoji_tag_sequence: 0 -rgi_emoji_zwj_sequence: 0 -rgi_emoji: 0 -ids_unary_operator: 0 -id_compat_math_start: 0 -id_compat_math_continue: 0 -bidi_class: 0 -block: 1 -canonical_combining_class: 0 -decomposition_type: 0 -east_asian_width: 4 -general_category: 2 -joining_group: 0 -joining_type: 0 -line_break: 2 -numeric_type: 0 -script: 25 -hangul_syllable_type: 0 -nfd_quick_check: 1 -nfkd_quick_check: 1 -nfc_quick_check: 1 -nfkc_quick_check: 1 -lead_canonical_combining_class: 0 -trail_canonical_combining_class: 0 -grapheme_cluster_break: 0 -sentence_break: 4 -word_break: 1 -bidi_paired_bracket_type: 0 -indic_positional_category: 0 -indic_syllabic_category: 0 -vertical_orientation: 0 -identifier_status: 1 -general_category_mask: 4 -numeric_value: 0 -age: 1.1 -bidi_mirroring_glyph: a -case_folding: a -lowercase_mapping: a -name: LATIN SMALL LETTER A -simple_case_folding: a -simple_lowercase_mapping: a -simple_titlecase_mapping: A -simple_uppercase_mapping: A -titlecase_mapping: A -uppercase_mapping: A -bidi_paired_bracket: a -script_extensions: ['Latin'] -identifier_type: ['Recommended'] - -``` - -```sql -SELECT code_point, code_point_value, notation FROM system.unicode WHERE code_point = '😂'; -``` - -```text - ┌─code_point─┬─code_point_value─┬─notation─┐ -1. │ 😂 │ 128514 │ U+1F602 │ - └────────────┴──────────────────┴──────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md deleted file mode 100644 index 37f178d8d6c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'メモリ使用量およびユーザーの ProfileEvents の概要を把握するのに役立つ情報を含むシステムテーブル。' -keywords: ['system table', 'user_processes'] -slug: /operations/system-tables/user_processes -title: 'system.user_processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.user_processes {#systemuser_processes} - - - -このシステムテーブルを使用すると、ユーザーごとのメモリ使用量と ProfileEvents の概要を確認できます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.user_processes LIMIT 10 FORMAT Vertical; -``` - -```response -行 1: -────── -user: default -memory_usage: 9832 -peak_memory_usage: 9832 -ProfileEvents: {'Query':5,'SelectQuery':5,'QueriesWithSubqueries':38,'SelectQueriesWithSubqueries':38,'QueryTimeMicroseconds':842048,'SelectQueryTimeMicroseconds':842048,'ReadBufferFromFileDescriptorRead':6,'ReadBufferFromFileDescriptorReadBytes':234,'IOBufferAllocs':3,'IOBufferAllocBytes':98493,'ArenaAllocChunks':283,'ArenaAllocBytes':1482752,'FunctionExecute':670,'TableFunctionExecute':16,'DiskReadElapsedMicroseconds':19,'NetworkSendElapsedMicroseconds':684,'NetworkSendBytes':139498,'SelectedRows':6076,'SelectedBytes':685802,'ContextLock':1140,'RWLockAcquiredReadLocks':193,'RWLockReadersWaitMilliseconds':4,'RealTimeMicroseconds':1585163,'UserTimeMicroseconds':889767,'SystemTimeMicroseconds':13630,'SoftPageFaults':1947,'OSCPUWaitMicroseconds':6,'OSCPUVirtualTimeMicroseconds':903251,'OSReadChars':28631,'OSWriteChars':28888,'QueryProfilerRuns':3,'LogTrace':79,'LogDebug':24} - -1行のセット。経過時間: 0.010秒 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/users.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/users.md deleted file mode 100644 index f369f34317b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/users.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'サーバー上で構成されているユーザーアカウントの一覧を含む system テーブル。' -keywords: ['system テーブル', 'ユーザー'] -slug: /operations/system-tables/users -title: 'system.users' -doc_type: 'reference' ---- - -# system.users {#systemusers} - -サーバー上で構成されている[ユーザーアカウント](../../guides/sre/user-management/index.md#user-account-management)の一覧を含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 関連項目 {#see-also} - -- [SHOW USERS](/sql-reference/statements/show#show-users) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md deleted file mode 100644 index 22d1aa43107..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'Refreshable マテリアライズドビューに関する情報を保持する system テーブル。' -keywords: ['system table', 'view_refreshes'] -slug: /operations/system-tables/view_refreshes -title: 'system.view_refreshes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.view_refreshes {#systemview_refreshes} - - - -[Refreshable Materialized Views](../../sql-reference/statements/create/view.md#refreshable-materialized-view) に関する情報。リフレッシュ処理の進行中かどうかに関係なく、すべてのリフレッシュ可能なマテリアライズドビューを含みます。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**例** - -```sql -SELECT - database, - view, - status, - last_refresh_result, - last_refresh_time, - next_refresh_time -FROM system.view_refreshes - -┌─database─┬─view───────────────────────┬─status────┬─last_refresh_result─┬───last_refresh_time─┬───next_refresh_time─┐ -│ default │ hello_documentation_reader │ Scheduled │ Finished │ 2023-12-01 01:24:00 │ 2023-12-01 01:25:00 │ -└──────────┴────────────────────────────┴───────────┴─────────────────────┴─────────────────────┴─────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md deleted file mode 100644 index 3cec4c7046d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: 'ローカルサーバー上で動作するワークロードに関する情報を格納するシステムテーブル。' -keywords: ['system table', 'ワークロード'] -slug: /operations/system-tables/workloads -title: 'system.workloads' -doc_type: 'reference' ---- - -# system.workloads {#systemworkloads} - -ローカルサーバー上に存在する [workloads](/operations/workload-scheduling.md#workload_entity_storage) に関する情報を含みます。テーブルには各 workload ごとに 1 行が格納されます。 - -例: - -```sql -SELECT * -FROM system.workloads -FORMAT Vertical -``` - -```text -Row 1: -────── -name: production -parent: all -create_query: CREATE WORKLOAD production IN `all` SETTINGS weight = 9 - -Row 2: -────── -name: development -parent: all -create_query: CREATE WORKLOAD development IN `all` - -Row 3: -────── -name: all -parent: -create_query: CREATE WORKLOAD `all` -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md deleted file mode 100644 index fe2991605ef..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: 'ClickHouse Keeper または ZooKeeper が構成されている場合にのみ存在するシステムテーブルです。設定ファイルで定義された Keeper クラスターのデータを公開します。' -keywords: ['システムテーブル', 'ZooKeeper'] -slug: /operations/system-tables/zookeeper -title: 'system.zookeeper' -doc_type: 'reference' ---- - -# system.zookeeper {#systemzookeeper} - -このテーブルは、ClickHouse Keeper または ZooKeeper が構成されている場合にのみ存在します。`system.zookeeper` テーブルでは、設定ファイルで定義された Keeper クラスターのデータを参照できます。 -クエリには、以下のように `WHERE` 句で `path =` 条件、または `path IN` 条件のいずれかを必ず指定する必要があります。これは、データを取得したい子ノードのパスに対応します。 - -クエリ `SELECT * FROM system.zookeeper WHERE path = '/clickhouse'` は、`/clickhouse` ノード直下のすべての子ノードのデータを出力します。 -すべてのルートノードのデータを出力するには、path = '/' と記述します。 -'path' で指定したパスが存在しない場合、例外がスローされます。 - -クエリ `SELECT * FROM system.zookeeper WHERE path IN ('/', '/clickhouse')` は、`/` および `/clickhouse` ノード直下のすべての子ノードのデータを出力します。 -指定した 'path' コレクションの中に存在しないパスが含まれている場合、例外がスローされます。 -これは、複数の Keeper パスに対するクエリを一括で実行する用途に使用できます。 - -クエリ `SELECT * FROM system.zookeeper WHERE path = '/clickhouse' AND zookeeperName = 'auxiliary_cluster'` は、ZooKeeper クラスター `auxiliary_cluster` 内のデータを出力します。 -指定した 'auxiliary_cluster' が存在しない場合、例外がスローされます。 - -列: - -* `name` (String) — ノード名。 -* `path` (String) — ノードへのパス。 -* `value` (String) — ノードの値。 -* `zookeeperName` (String) — デフォルトまたは補助 ZooKeeper クラスターのうちいずれかの名前。 -* `dataLength` (Int32) — 値のサイズ。 -* `numChildren` (Int32) — 子孫ノードの数。 -* `czxid` (Int64) — ノードを作成したトランザクションの ID。 -* `mzxid` (Int64) — 直近でノードを変更したトランザクションの ID。 -* `pzxid` (Int64) — 直近で子孫ノードを削除または追加したトランザクションの ID。 -* `ctime` (DateTime) — ノード作成時刻。 -* `mtime` (DateTime) — ノードが最後に変更された時刻。 -* `version` (Int32) — ノードのバージョン(ノードが変更された回数)。 -* `cversion` (Int32) — 追加または削除された子孫ノードの数。 -* `aversion` (Int32) — ACL の変更回数。 -* `ephemeralOwner` (Int64) — 一時ノードの場合、このノードを所有するセッションの ID。 - -例: - -```sql -SELECT * -FROM system.zookeeper -WHERE path = '/clickhouse/tables/01-08/visits/replicas' -FORMAT Vertical -``` - -```text -Row 1: -────── -name: example01-08-1 -value: -czxid: 932998691229 -mzxid: 932998691229 -ctime: 2015-03-27 16:49:51 -mtime: 2015-03-27 16:49:51 -version: 0 -cversion: 47 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021031383 -path: /clickhouse/tables/01-08/visits/replicas - -Row 2: -────── -name: example01-08-2 -value: -czxid: 933002738135 -mzxid: 933002738135 -ctime: 2015-03-27 16:57:01 -mtime: 2015-03-27 16:57:01 -version: 0 -cversion: 37 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021252247 -path: /clickhouse/tables/01-08/visits/replicas -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md deleted file mode 100644 index 90e82de9300..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'ZooKeeper が設定されている場合にのみ存在するシステムテーブル。現在の ZooKeeper への接続状況(補助的な ZooKeeper を含む)を表示します。' -keywords: ['システムテーブル', 'zookeeper_connection'] -slug: /operations/system-tables/zookeeper_connection -title: 'system.zookeeper_connection' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection {#systemzookeeper_connection} - - - -ZooKeeper が構成されていない場合、このテーブルは存在しません。`system.zookeeper_connection` テーブルは、ZooKeeper(補助 ZooKeeper を含む)への現在の接続を表示します。各行は 1 つの接続に関する情報を表します。 - -列: - -* `name` ([String](../../sql-reference/data-types/string.md)) — ZooKeeper クラスター名。 -* `host` ([String](../../sql-reference/data-types/string.md)) — ClickHouse が接続した ZooKeeper ノードのホスト名 / IP。 -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — ClickHouse が接続した ZooKeeper ノードのポート。 -* `index` ([Nullable(UInt8)](../../sql-reference/data-types/int-uint.md)) — ClickHouse が接続した ZooKeeper ノードのインデックス。インデックスは ZooKeeper の設定に基づきます。接続されていない場合、このカラムは NULL です。 -* `connected_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 接続が確立された時刻。 -* `session_uptime_elapsed_seconds` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 接続が確立されてから経過した秒数。 -* `is_expired` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 現在の接続が期限切れかどうか。 -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Keeper API バージョン。 -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — 接続のセッション ID。 -* `xid` ([Int64](../../sql-reference/data-types/int-uint.md)) — 現在のセッションの XID。 -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — 有効になっている機能フラグ。ClickHouse Keeper にのみ適用されます。取りうる値は `FILTERED_LIST`, `MULTI_READ`, `CHECK_NOT_EXISTS`, `CREATE_IF_NOT_EXISTS`, `REMOVE_RECURSIVE` です。 -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — アベイラビリティゾーン。 - -例: - -```sql -SELECT * FROM system.zookeeper_connection; -``` - -```text -┌─name────┬─host──────┬─port─┬─index─┬──────connected_time─┬─session_uptime_elapsed_seconds─┬─is_expired─┬─keeper_api_version─┬─client_id─┬─xid─┬─enabled_feature_flags────────────────────────────────────────────────────┬─availability_zone─┐ -│ default │ 127.0.0.1 │ 2181 │ 0 │ 2025-04-10 14:30:00 │ 943 │ 0 │ 0 │ 420 │ 69 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS'] │ eu-west-1b │ -└─────────┴───────────┴──────┴───────┴─────────────────────┴────────────────────────────────┴────────────┴────────────────────┴───────────┴─────┴──────────────────────────────────────────────────────────────────────────┴───────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md deleted file mode 100644 index 58d47f8c4d1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'ZooKeeper への接続履歴(補助的な ZooKeeper を含む)を表示します。' -keywords: ['system table', 'zookeeper_connection_log'] -slug: /operations/system-tables/zookeeper_connection_log -title: 'system.zookeeper_connection_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection_log {#systemzookeeper_connection_log} - - - -'system.zookeeper_connection_log' テーブルは、ZooKeeper への接続履歴(補助 ZooKeeper を含む)を示します。各行は、接続に関する 1 件のイベント情報を表します。 - -:::note -このテーブルには、サーバーのシャットダウンによって発生した切断イベントは含まれません。 -::: - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — ZooKeeper に接続または切断したサーバーのホスト名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) - イベントの種類。取りうる値: `Connected`, `Disconnected`。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) - エントリの日付。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - エントリの時刻。 -* `event_time_microseconds` ([Date](../../sql-reference/data-types/datetime64.md)) - マイクロ秒精度でのエントリの時刻。 -* `name` ([String](../../sql-reference/data-types/string.md)) — ZooKeeper クラスター名。 -* `host` ([String](../../sql-reference/data-types/string.md)) — ClickHouse が接続した ZooKeeper ノードのホスト名/IP。 -* `port` ([UIn16](../../sql-reference/data-types/int-uint.md)) — ClickHouse が接続した ZooKeeper ノードのポート。 -* `index` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ClickHouse が接続または切断した ZooKeeper ノードのインデックス。インデックスは ZooKeeper の設定に由来します。 -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — 接続のセッション ID。 -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Keeper API バージョン。 -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — 有効化されているフィーチャーフラグ。ClickHouse Keeper にのみ適用されます。取りうる値は `FILTERED_LIST`, `MULTI_READ`, `CHECK_NOT_EXISTS`, `CREATE_IF_NOT_EXISTS`, `REMOVE_RECURSIVE` です。 -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — アベイラビリティーゾーン。 -* `reason` ([String](../../sql-reference/data-types/string.md)) — 接続または切断の理由。 - -例: - -```sql -SELECT * FROM system.zookeeper_connection_log; -``` - -```text -┌─hostname─┬─type─────────┬─event_date─┬──────────event_time─┬────event_time_microseconds─┬─name───────────────┬─host─┬─port─┬─index─┬─client_id─┬─keeper_api_version─┬─enabled_feature_flags───────────────────────────────────────────────────────────────────────┬─availability_zone─┬─reason──────────────┐ - 1. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:35 │ 2025-05-12 19:49:35.713067 │ zk_conn_log_test_4 │ zoo2 │ 2181 │ 0 │ 10 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初期化 │ - 2. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:23 │ 2025-05-12 19:49:23.981570 │ default │ zoo1 │ 2181 │ 0 │ 4 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初期化 │ - 3. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:28 │ 2025-05-12 19:49:28.104021 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初期化 │ - 4. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.459251 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初期化 │ - 5. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.574312 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初期化 │ - 6. │ node │ 切断済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909890 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 設定変更 │ - 7. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909895 │ default │ zoo2 │ 2181 │ 0 │ 8 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 設定変更 │ - 8. │ node │ 切断済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912010 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 設定変更 │ - 9. │ node │ 接続済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912014 │ zk_conn_log_test_2 │ zoo3 │ 2181 │ 0 │ 9 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 設定変更 │ -10. │ node │ 切断済み │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912061 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 設定から削除 │ - └──────────┴──────────────┴────────────┴─────────────────────┴────────────────────────────┴────────────────────┴──────┴──────┴───────┴───────────┴────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────┴─────────────────────┘ -``` \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md deleted file mode 100644 index 9213232c87a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'ZooKeeper サーバーへのリクエストのパラメータおよびレスポンスに関する情報を含むシステムテーブル。' -keywords: ['システムテーブル', 'zookeeper_log'] -slug: /operations/system-tables/zookeeper_log -title: 'system.zookeeper_log' -doc_type: 'reference' ---- - -# system.zookeeper_log {#systemzookeeper_log} - -このテーブルには、ZooKeeper サーバーへのリクエストのパラメータと、そのレスポンスに関する情報が含まれます。 - -リクエスト時には、リクエストパラメータを持つカラムのみが埋められ、残りのカラムはデフォルト値(`0` または `NULL`)で埋められます。レスポンスが到着すると、そのレスポンスのデータが他のカラムに追加されます。 - -リクエストパラメータを持つカラム: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — クエリを実行しているサーバーのホスト名。 -* `type` ([Enum](../../sql-reference/data-types/enum.md)) — ZooKeeper クライアントにおけるイベント種別。次のいずれかの値を取ります: - * `Request` — リクエストが送信された。 - * `Response` — レスポンスを受信した。 - * `Finalize` — 接続が失われ、レスポンスを受信しなかった。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — イベントが発生した日付。 -* `event_time` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — イベントが発生した日時。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — リクエストに使用された ZooKeeper サーバーの IP アドレス。 -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — リクエストに使用された ZooKeeper サーバーのポート。 -* `session_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — 各接続に対して ZooKeeper サーバーが設定するセッション ID。 -* `xid` ([Int32](../../sql-reference/data-types/int-uint.md)) — セッション内でのリクエスト ID。通常は連番のリクエスト番号です。リクエスト行と、それに対応する `response` / `finalize` 行で同じ値になります。 -* `has_watch` ([UInt8](../../sql-reference/data-types/int-uint.md)) — [watch](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkWatches) が設定されているかどうかを示すフラグ。 -* `op_num` ([Enum](../../sql-reference/data-types/enum.md)) — リクエストまたはレスポンスの種別。 -* `path` ([String](../../sql-reference/data-types/string.md)) — リクエストで指定された ZooKeeper ノードへのパス。パスの指定を必要としないリクエストの場合は空文字列。 -* `data` ([String](../../sql-reference/data-types/string.md)) — ZooKeeper ノードに書き込まれるデータ(`SET` および `CREATE` リクエストではリクエストが書き込もうとした内容、`GET` リクエストのレスポンスでは読み取られた内容)、または空文字列。 -* `is_ephemeral` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ZooKeeper ノードが[エフェメラル](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Ephemeral+Nodes)として作成されているかどうか。 -* `is_sequential` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ZooKeeper ノードが[シーケンシャル](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Sequence+Nodes+--+Unique+Naming)として作成されているかどうか。 -* `version` ([Nullable(Int32)](../../sql-reference/data-types/nullable.md)) — 実行時にリクエストが期待する ZooKeeper ノードのバージョン。これは `CHECK`、`SET`、`REMOVE` リクエストでサポートされています(リクエストがバージョンをチェックしない場合は `-1` が設定され、バージョンチェックをサポートしない他のリクエストでは `NULL`)。 -* `requests_size` ([UInt32](../../sql-reference/data-types/int-uint.md)) — マルチリクエストに含まれるリクエストの数(これは複数の連続した通常リクエストから構成され、それらをアトミックに実行する特別なリクエストです)。マルチリクエストに含まれるすべてのリクエストは同じ `xid` を持ちます。 -* `request_idx` ([UInt32](../../sql-reference/data-types/int-uint.md)) — マルチリクエストに含まれるリクエストの番号(マルチリクエスト本体は `0`、その後 `1` から順番)。 - -レスポンスパラメータを持つカラム: - -* `zxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — ZooKeeper トランザクション ID。ZooKeeper サーバーが、正常に実行されたリクエストに対する応答として発行する通し番号(リクエストが実行されなかった/エラーを返した/クライアントがリクエストが実行されたかどうかを認識していない場合は `0`)。 -* `error` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — エラーコード。取りうる値は多数あるが、以下はその一部: - * `ZOK` — リクエストは正常に実行された。 - * `ZCONNECTIONLOSS` — 接続が失われた。 - * `ZOPERATIONTIMEOUT` — リクエスト実行のタイムアウトが発生した。 - * `ZSESSIONEXPIRED` — セッションが期限切れになった。 - * `NULL` — リクエストは完了している。 -* `watch_type` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — `watch` イベントの種別(`op_num` = `Watch` のレスポンスの場合)。それ以外のレスポンスでは `NULL`。 -* `watch_state` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — `watch` イベントのステータス(`op_num` = `Watch` のレスポンスの場合)。それ以外のレスポンスでは `NULL`。 -* `path_created` ([String](../../sql-reference/data-types/string.md)) — 作成された ZooKeeper ノードへのパス(`CREATE` リクエストへのレスポンスの場合)。ノードが `sequential` として作成された場合は、`path` と異なることがある。 -* `stat_czxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードが作成される原因となった変更の `zxid`。 -* `stat_mzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードを最後に変更したときの `zxid`。 -* `stat_pzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードの子ノードを最後に変更したときのトランザクション ID。 -* `stat_version` ([Int32](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードのデータに対する変更回数。 -* `stat_cversion` ([Int32](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードの子ノードに対する変更回数。 -* `stat_dataLength` ([Int32](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードのデータフィールドの長さ。 -* `stat_numChildren` ([Int32](../../sql-reference/data-types/int-uint.md)) — この ZooKeeper ノードの子ノード数。 -* `children` ([Array(String)](../../sql-reference/data-types/array.md)) — 子 ZooKeeper ノードのリスト(`LIST` リクエストへのレスポンスの場合)。 - -**例** - -クエリ: - -```sql -SELECT * FROM system.zookeeper_log WHERE (session_id = '106662742089334927') AND (xid = '10858') FORMAT Vertical; -``` - -結果: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: Request -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.291792 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 0 -error: ᴺᵁᴸᴸ -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 0 -stat_mzxid: 0 -stat_pzxid: 0 -stat_version: 0 -stat_cversion: 0 -stat_dataLength: 0 -stat_numChildren: 0 -children: [] - -Row 2: -────── -type: Response -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.292086 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 16926267 -error: ZOK -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 16925469 -stat_mzxid: 16925469 -stat_pzxid: 16926179 -stat_version: 0 -stat_cversion: 7 -stat_dataLength: 0 -stat_numChildren: 7 -children: ['query-0000000006','query-0000000005','query-0000000004','query-0000000003','query-0000000002','query-0000000001','query-0000000000'] -``` - -**関連項目** - -* [ZooKeeper](../../operations/tips.md#zookeeper) -* [ZooKeeper ガイド](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/tips.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/tips.md deleted file mode 100644 index 826df4b1a85..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/tips.md +++ /dev/null @@ -1,334 +0,0 @@ ---- -description: 'オープンソース版 ClickHouse の利用に関する推奨事項をまとめたページ' -sidebar_label: 'OSS 利用の推奨事項' -sidebar_position: 58 -slug: /operations/tips -title: 'OSS 利用の推奨事項' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - - -## CPU スケーリングガバナー {#cpu-scaling-governor} - -常に `performance` スケーリングガバナーを使用してください。`on-demand` スケーリングガバナーは、常時高負荷のワークロードではパフォーマンスが大きく低下します。 - -```bash -$ echo 'performance' | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor -``` - - -## CPU の制限事項 {#cpu-limitations} - -プロセッサは過熱する場合があります。CPU のクロック周波数が過熱によって制限されたかどうかを確認するには、`dmesg` を使用します。 -この制限はデータセンター側の設定として外部から課されている場合もあります。負荷をかけた状態で監視するには、`turbostat` を使用できます。 - -## RAM {#ram} - -少量のデータ(圧縮後で最大約 200 GB)の場合は、データ量と同程度のメモリを搭載するのが理想的です。 -大量のデータを扱い、かつインタラクティブ(オンライン)クエリを処理する場合は、ページキャッシュにホットデータのサブセットが収まるよう、妥当な量の RAM(128 GB 以上)を使用することを推奨します。 -1 サーバーあたり約 50 TB のデータ量であっても、128 GB の RAM を使用することで、64 GB と比較してクエリ性能が大幅に向上します。 - -`overcommit` を無効化しないでください。`cat /proc/sys/vm/overcommit_memory` の値は 0 または 1 である必要があります。次を実行します - -```bash -$ echo 0 | sudo tee /proc/sys/vm/overcommit_memory -``` - -メモリ管理に費やされるカーネル時間を観測するには、`perf top` を使用します。 -恒久的なヒュージページを割り当てる必要もありません。 - - -### 16GB 未満の RAM を使用する場合 {#using-less-than-16gb-of-ram} - -推奨される RAM 容量は 32 GB 以上です。 - -システムの RAM が 16 GB 未満の場合、デフォルト設定がこのメモリ容量に合っていないため、さまざまなメモリ関連の例外が発生する可能性があります。少量の RAM(最小で 2 GB)しかないシステムでも ClickHouse を使用できますが、そのような構成では追加のチューニングが必要となり、取り込みレートも低くなります。 - -16GB 未満の RAM で ClickHouse を使用する場合、次の設定を推奨します: - -- `config.xml` 内の mark cache のサイズを小さくします。500 MB まで下げられますが、0 に設定することはできません。 -- クエリ処理スレッド数を `1` まで減らします。 -- `max_block_size` を `8192` まで下げます。`1024` まで下げても実用的です。 -- `max_download_threads` を `1` に下げます。 -- `input_format_parallel_parsing` と `output_format_parallel_formatting` を `0` に設定します。 -- ログテーブルへの書き込みを無効化します。ログテーブルのマージを行うバックグラウンドのマージタスクが、マージ処理のために RAM を予約し続けるためです。`asynchronous_metric_log`、`metric_log`、`text_log`、`trace_log` を無効化します。 - -補足事項: - -- メモリアロケータによりキャッシュされたメモリを解放するには、`SYSTEM JEMALLOC PURGE` -コマンドを実行します。 -- バッファ用に多くのメモリを必要とするため、メモリ量の少ないマシンでは S3 や Kafka との連携機能の使用は推奨しません。 - -## ストレージサブシステム {#storage-subsystem} - -予算が許して SSD を使用できるなら、SSD を使用してください。 -そうでなければ HDD を使用してください。7200RPM の SATA HDD で十分です。 - -ディスクシェルフを接続した少数のサーバーよりも、ローカルハードディスクを搭載した多数のサーバー構成を優先してください。 -ただし、滅多にクエリされないアーカイブを保存する用途であれば、ディスクシェルフでも問題ありません。 - -## RAID {#raid} - -HDD を使用する場合は、RAID-10、RAID-5、RAID-6、RAID-50 のいずれかの RAID 構成を組むことができます。 -Linux では、ソフトウェア RAID(`mdadm` を使用)が望ましいです。 -RAID-10 を作成する際は、`far` レイアウトを選択してください。 -予算に余裕がある場合は、RAID-10 を選択してください。 - -LVM 単体(RAID や `mdadm` なし)での利用は問題ありませんが、LVM で RAID を組んだり、`mdadm` と組み合わせたりする構成はあまり検証されておらず、 -チャンクサイズの誤設定、チャンクのアライメントずれ、不適切な RAID タイプの選択、ディスクのクリーンアップ忘れなど、ミスが発生しやすくなります。 -LVM の使用に十分自信があるのであれば、利用しても差し支えありません。 - -ディスクが 4 台を超える場合は、RAID-5 ではなく RAID-6(推奨)か RAID-50 を使用してください。 -RAID-5、RAID-6、RAID-50 を使用する場合は、常に `stripe_cache_size` を増やしてください。デフォルト値は通常最適な選択ではありません。 - -```bash -$ echo 4096 | sudo tee /sys/block/md2/md/stripe_cache_size -``` - -`2 * num_devices * chunk_size_in_bytes / 4096` という式を使って、デバイス数とブロックサイズから正確な値を算出します。 - -ほとんどの RAID 構成では、ブロックサイズとして 64 KB で十分です。平均的な clickhouse-server の書き込みサイズはおよそ 1 MB (1024 KB) であり、そのため推奨されるストライプサイズも 1 MB です。RAID アレイ内の非パリティディスクの本数で 1 MB を割った値を設定することで、必要に応じてブロックサイズを最適化できます。このように設定することで、各書き込みが利用可能なすべての非パリティディスクにまたがって並列化されます。 -ブロックサイズを小さくしすぎたり、大きくしすぎたりしないでください。 - -SSD では RAID-0 を使用できます。 -RAID を使用するかどうかに関わらず、データ保護のために常にレプリケーションを有効にしてください。 - -十分に長いキュー長で NCQ を有効にします。HDD には mq-deadline または CFQ スケジューラを、SSD には noop を選択します。`readahead` 設定を減らさないでください。 -HDD ではライトキャッシュを有効にします。 - -OS で NVMe および SSD ディスクに対して [`fstrim`](https://en.wikipedia.org/wiki/Trim_\(computing\)) が有効になっていることを確認してください (通常は cron ジョブまたは systemd サービスとして実装されています)。 - - -## ファイルシステム {#file-system} - -Ext4 が最も信頼性の高い選択肢です。マウントオプションとして `noatime` を設定してください。XFS も同様に問題なく動作します。 -その他のほとんどのファイルシステムも概ね正常に動作するはずです。 - -FAT-32 および exFAT はハードリンクをサポートしていないため使用できません。 - -圧縮ファイルシステムは使用しないでください。ClickHouse 自身が、より優れた方法で圧縮を行うためです。 -暗号化ファイルシステムの利用も推奨されません。より優れた ClickHouse の組み込み暗号化機能を利用できます。 - -ClickHouse は NFS 上でも動作しますが、最適な選択肢とは言えません。 - -## Linux Kernel {#linux-kernel} - -古い Linux カーネルの使用は避けてください。 - -## ネットワーク {#network} - -IPv6 を使用している場合は、ルートキャッシュのサイズを増やしてください。 -Linux カーネル 3.2 以前には、IPv6 実装に多くの問題がありました。 - -可能であれば、少なくとも 10 GB のネットワークを使用してください。1 Gb でも動作しますが、数十テラバイトのデータを持つレプリカへのパッチ適用や、大量の中間データを伴う分散クエリの処理では、性能が大きく低下します。 - -## ヒュージページ {#huge-pages} - -古い Linux カーネルを使用している場合は、Transparent Huge Pages を無効化してください。メモリアロケータと干渉し、著しいパフォーマンス低下を招きます。 -より新しい Linux カーネルでは、Transparent Huge Pages を有効にしたままで問題ありません。 - -```bash -$ echo 'madvise' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled -``` - -Transparent Huge Pages (THP) の設定を永続的に変更したい場合は、`/etc/default/grub` を編集し、`GRUB_CMDLINE_LINUX_DEFAULT` オプションに `transparent_hugepage=madvise` を追加します。 - -```bash -$ GRUB_CMDLINE_LINUX_DEFAULT="transparent_hugepage=madvise ..." -``` - -その後、`sudo update-grub` コマンドを実行し、反映させるためにシステムを再起動してください。 - - -## ハイパーバイザーの構成 {#hypervisor-configuration} - -OpenStack を使用している場合は、以下を設定します。 - -```ini -cpu_mode=host-passthrough -``` - -`nova.conf` 内で行います。 - -libvirt を使用している場合は、次を設定します。 - -```xml - -``` - -XML 設定で指定します。 - -これは、ClickHouse が `cpuid` 命令によって正しい情報を取得できるようにするために重要です。 -そうしていない場合、古い CPU モデル上でハイパーバイザーを実行していると `Illegal instruction` エラーによりクラッシュする可能性があります。 - - -## ClickHouse Keeper と ZooKeeper {#zookeeper} - -ClickHouse Keeper は、ClickHouse クラスターにおいて ZooKeeper を置き換えることが推奨されます。[ClickHouse Keeper](../guides/sre/keeper/index.md) のドキュメントを参照してください。 - -ZooKeeper を引き続き使用したい場合は、新しいバージョンの ZooKeeper(3.4.9 以降)を使用するのが最善です。安定版 Linux ディストリビューションに含まれるバージョンは古くなっている可能性があります。 - -異なる ZooKeeper クラスター間でデータを転送するために、自前のスクリプトを決して使用してはいけません。シーケンシャルノードに対して結果が不正になるためです。同じ理由で "zkcopy" ユーティリティも決して使用しないでください: [https://github.com/ksprojects/zkcopy/issues/15](https://github.com/ksprojects/zkcopy/issues/15) - -既存の ZooKeeper クラスターを 2 つに分割したい場合、正しい方法は、まずレプリカ数を増やしてから、それを 2 つの独立したクラスターとして再構成することです。 - -ClickHouse Keeper は、テスト環境やインジェストレートが低い環境では ClickHouse と同じサーバー上で実行できます。 -本番環境では、ClickHouse と ZooKeeper/Keeper 用に別々のサーバーを使用するか、ClickHouse のファイルと Keeper のファイルを別々のディスクに配置することを推奨します。ZooKeeper/Keeper はディスクレイテンシに非常に敏感であり、ClickHouse は利用可能なシステムリソースをすべて使い切る可能性があるためです。 - -アンサンブル内に ZooKeeper オブザーバーを配置することはできますが、ClickHouse サーバーはオブザーバーと通信してはいけません。 - -`minSessionTimeout` 設定は変更しないでください。大きな値は ClickHouse の再起動の安定性に影響を与える可能性があります。 - -デフォルト設定のままでは、ZooKeeper は時限爆弾のようなものです: - -> ZooKeeper サーバーは、デフォルト設定のままでは(`autopurge` を参照)、古いスナップショットやログからファイルを削除しません。これはオペレーターの責任となります。 - -この爆弾は必ず解除しなければなりません。 - -以下の ZooKeeper (3.5.1) の設定は、大規模な本番環境で使用されているものです。 - -zoo.cfg: - -```bash -# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html {#httphadoopapacheorgzookeeperdocscurrentzookeeperadminhtml} - -# 各ティックのミリ秒数 {#the-number-of-milliseconds-of-each-tick} -tickTime=2000 -# 初期同期フェーズで {#the-number-of-ticks-that-the-initial} -# 許容されるティック数 {#synchronization-phase-can-take} -# この値は十分に検証されていない {#this-value-is-not-quite-motivated} -initLimit=300 -# リクエスト送信から確認応答受信までに {#the-number-of-ticks-that-can-pass-between} -# 経過可能なティック数 {#sending-a-request-and-getting-an-acknowledgement} -syncLimit=10 - -maxClientCnxns=2000 - -# クライアントが要求でき、サーバーが受け入れる最大値。 {#it-is-the-maximum-value-that-client-may-request-and-the-server-will-accept} -# クライアントが高いセッションタイムアウトで動作できるよう、サーバー側でmaxSessionTimeoutを高く設定しても問題ない。 {#it-is-ok-to-have-high-maxsessiontimeout-on-server-to-allow-clients-to-work-with-high-session-timeout-if-they-want} -# ただし、デフォルトでは30秒のセッションタイムアウトを要求する(ClickHouse設定のsession_timeout_msで変更可能)。 {#but-we-request-session-timeout-of-30-seconds-by-default-you-can-change-it-with-session_timeout_ms-in-clickhouse-config} -maxSessionTimeout=60000000 -# スナップショットが保存されるディレクトリ。 {#the-directory-where-the-snapshot-is-stored} -dataDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/data -# パフォーマンス向上のため、dataLogDirは別の物理ディスクに配置すること {#place-the-datalogdir-to-a-separate-physical-disc-for-better-performance} -dataLogDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/logs - -autopurge.snapRetainCount=10 -autopurge.purgeInterval=1 - - -# シークを回避するため、ZooKeeperはトランザクションログファイル内の領域を {#to-avoid-seeks-zookeeper-allocates-space-in-the-transaction-log-file-in} -# preAllocSizeキロバイト単位のブロックで割り当てる。デフォルトのブロックサイズは64M。 {#blocks-of-preallocsize-kilobytes-the-default-block-size-is-64m-one-reason} -# ブロックサイズを変更する理由の一つは、スナップショットをより頻繁に取得する場合に {#for-changing-the-size-of-the-blocks-is-to-reduce-the-block-size-if-snapshots} -# ブロックサイズを削減することである(snapCountも参照)。 {#are-taken-more-often-also-see-snapcount} -preAllocSize=131072 - -# クライアントはZooKeeperが処理できる速度よりも速くリクエストを送信できる。 {#clients-can-submit-requests-faster-than-zookeeper-can-process-them} -# 特に多数のクライアントが存在する場合に顕著である。キューに入ったリクエストによって {#especially-if-there-are-a-lot-of-clients-to-prevent-zookeeper-from-running} -# ZooKeeperがメモリ不足に陥るのを防ぐため、ZooKeeperはクライアントをスロットルし、 {#out-of-memory-due-to-queued-requests-zookeeper-will-throttle-clients-so-that} -# システム内の未処理リクエスト数がglobalOutstandingLimitを超えないようにする。 {#there-is-no-more-than-globaloutstandinglimit-outstanding-requests-in-the} -# デフォルトの制限値は1000。 {#system-the-default-limit-is-1000} -# globalOutstandingLimit=1000 {#globaloutstandinglimit1000} - -# ZooKeeperはトランザクションをトランザクションログに記録する。snapCount件のトランザクションが {#zookeeper-logs-transactions-to-a-transaction-log-after-snapcount-transactions} -# ログファイルに書き込まれた後、スナップショットが開始され、新しいトランザクションログファイルが {#are-written-to-a-log-file-a-snapshot-is-started-and-a-new-transaction-log-file} -# 開始される。デフォルトのsnapCountは100000。 {#is-started-the-default-snapcount-is-100000} -snapCount=3000000 - -# このオプションが定義されている場合、リクエストは {#if-this-option-is-defined-requests-will-be-will-logged-to-a-trace-file-named} -# traceFile.year.month.dayという名前のトレースファイルに記録される。 {#tracefileyearmonthday} -#traceFile= - -# リーダーはクライアント接続を受け入れる。デフォルト値は"yes"。リーダーマシンは {#leader-accepts-client-connections-default-value-is-yes-the-leader-machine} -# 更新を調整する。読み取りスループットをわずかに犠牲にして更新スループットを向上させるため、 {#coordinates-updates-for-higher-update-throughput-at-thes-slight-expense-of} -# リーダーをクライアント接続を受け入れず調整に専念するよう設定できる。 {#read-throughput-the-leader-can-be-configured-to-not-accept-clients-and-focus} -leaderServes=yes - -standaloneEnabled=false -dynamicConfigFile=/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/zoo.cfg.dynamic -``` - -Java 版: - - -```text -openjdk 11.0.5-shenandoah 2019-10-15 -OpenJDK Runtime Environment (build 11.0.5-shenandoah+10-adhoc.heretic.src) -OpenJDK 64-Bit Server VM (build 11.0.5-shenandoah+10-adhoc.heretic.src, mixed mode) -``` - -JVM パラメータ: - - -```bash -NAME=zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} -ZOOCFGDIR=/etc/$NAME/conf - -# TODO this is really ugly {#on-coordination} -# How to find out, which jars are needed? {#todo-this-is-really-ugly} -# seems, that log4j requires the log4j.properties file to be in the classpath {#how-to-find-out-which-jars-are-needed} -CLASSPATH="$ZOOCFGDIR:/usr/build/classes:/usr/build/lib/*.jar:/usr/share/zookeeper-3.6.2/lib/audience-annotations-0.5.0.jar:/usr/share/zookeeper-3.6.2/lib/commons-cli-1.2.jar:/usr/share/zookeeper-3.6.2/lib/commons-lang-2.6.jar:/usr/share/zookeeper-3.6.2/lib/jackson-annotations-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-core-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-databind-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/javax.servlet-api-3.1.0.jar:/usr/share/zookeeper-3.6.2/lib/jetty-http-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-io-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-security-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-server-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-servlet-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-util-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jline-2.14.6.jar:/usr/share/zookeeper-3.6.2/lib/json-simple-1.1.1.jar:/usr/share/zookeeper-3.6.2/lib/log4j-1.2.17.jar:/usr/share/zookeeper-3.6.2/lib/metrics-core-3.2.5.jar:/usr/share/zookeeper-3.6.2/lib/netty-buffer-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-codec-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-handler-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-resolver-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-epoll-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-unix-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_common-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_hotspot-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_servlet-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-api-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-log4j12-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/snappy-java-1.1.7.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-jute-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-prometheus-metrics-3.6.2.jar:/usr/share/zookeeper-3.6.2/etc" - -ZOOCFG="$ZOOCFGDIR/zoo.cfg" -ZOO_LOG_DIR=/var/log/$NAME -USER=zookeeper -GROUP=zookeeper -PIDDIR=/var/run/$NAME -PIDFILE=$PIDDIR/$NAME.pid -SCRIPTNAME=/etc/init.d/$NAME -JAVA=/usr/local/jdk-11/bin/java -ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain" -ZOO_LOG4J_PROP="INFO,ROLLINGFILE" -JMXLOCALONLY=false -JAVA_OPTS="-Xms{{ '{{' }} cluster.get('xms','128M') {{ '}}' }} \ - -Xmx{{ '{{' }} cluster.get('xmx','1G') {{ '}}' }} \ - -Xlog:safepoint,gc*=info,age*=debug:file=/var/log/$NAME/zookeeper-gc.log:time,level,tags:filecount=16,filesize=16M - -verbose:gc \ - -XX:+UseG1GC \ - -Djute.maxbuffer=8388608 \ - -XX:MaxGCPauseMillis=50" -``` - -ソルトの初期化 - -```text -description "zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} 集中コーディネーションサービス" - -start on runlevel [2345] -stop on runlevel [!2345] - -respawn - -limit nofile 8192 8192 - -pre-start script - [ -r "/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment" ] || exit 0 - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -d $ZOO_LOG_DIR ] || mkdir -p $ZOO_LOG_DIR - chown $USER:$GROUP $ZOO_LOG_DIR -end script - -script - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -r /etc/default/zookeeper ] && . /etc/default/zookeeper - if [ -z "$JMXDISABLE" ]; then - JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY" - fi - exec start-stop-daemon --start -c $USER --exec $JAVA --name zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} \ - -- -cp $CLASSPATH $JAVA_OPTS -Dzookeeper.log.dir=${ZOO_LOG_DIR} \ - -Dzookeeper.root.logger=${ZOO_LOG4J_PROP} $ZOOMAIN $ZOOCFG -end script -``` - - -## アンチウイルスソフトウェア {#antivirus-software} - -アンチウイルスソフトウェアを使用している場合は、ClickHouse のデータファイル(`/var/lib/clickhouse`)が格納されているディレクトリをスキャン対象から除外するように設定してください。そうしない場合、パフォーマンスが低下し、データのインジェストやバックグラウンドマージ中に予期しないエラーが発生する可能性があります。 - -## 関連コンテンツ {#related-content} - -- [ClickHouse を使い始めるときの 13 の「大罪」とその回避方法](https://clickhouse.com/blog/common-getting-started-issues-with-clickhouse) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/update.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/update.md deleted file mode 100644 index 3634419c1ac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/update.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -description: 'セルフマネージド環境のアップグレードに関するドキュメント' -sidebar_title: 'セルフマネージド環境のアップグレード' -slug: /operations/update -title: 'セルフマネージド環境のアップグレード' -doc_type: 'guide' ---- - - - -## ClickHouse アップグレードの概要 {#clickhouse-upgrade-overview} - -このドキュメントでは、次の内容を説明します: -- 一般的なガイドライン -- 推奨される計画 -- システム上のバイナリをアップグレードするための詳細 - - - -## 一般的なガイドライン {#general-guidelines} - -ここでの注意事項は、計画を立てる際の助けになるとともに、本ドキュメントの後半で行っている推奨事項の理由を理解するのに役立ちます。 - -### ClickHouse Keeper または ZooKeeper とは別に ClickHouse サーバーをアップグレードする {#upgrade-clickhouse-server-separately-from-clickhouse-keeper-or-zookeeper} -ClickHouse Keeper または Apache ZooKeeper に対するセキュリティ修正が必要な場合を除き、ClickHouse サーバーをアップグレードする際に Keeper を同時にアップグレードする必要はありません。アップグレード処理中は Keeper の安定性が必要となるため、まず ClickHouse サーバーのアップグレードを完了してから、Keeper のアップグレードを検討してください。 - -### マイナーバージョンのアップグレードは頻繁に行う {#minor-version-upgrades-should-be-adopted-often} -新しいマイナーバージョンがリリースされたら、可能な限り速やかにその最新版へアップグレードすることを強く推奨します。マイナーリリースには互換性を壊す変更は含まれませんが、重要なバグ修正(およびセキュリティ修正が含まれる場合もあります)が含まれます。 - -### 対象バージョンで動作する別の ClickHouse サーバー上で実験的機能をテストする {#test-experimental-features-on-a-separate-clickhouse-server-running-the-target-version} - -実験的機能の互換性は、いつでもどのような形でも壊れる可能性があります。実験的機能を利用している場合は、変更履歴を確認し、対象バージョンをインストールした別の ClickHouse サーバーを用意して、そのサーバー上で実験的機能の利用方法をテストすることを検討してください。 - -### ダウングレード {#downgrades} -アップグレード後に、新しいバージョンが依存している機能と互換性がないことに気付いた場合で、かつ新機能をまだ一切使用していなければ、比較的最近の(1年以内にリリースされた)バージョンにダウングレードできる可能性があります。一度でも新機能を使用してしまうと、ダウングレードは行えません。 - -### クラスター内の複数の ClickHouse サーバーバージョン {#multiple-clickhouse-server-versions-in-a-cluster} - -ClickHouse では 1年間の互換性ウィンドウ(2つの LTS バージョンを含む)を維持するよう努めています。これは、2つのバージョン間の差が1年未満(またはその間に存在する LTS バージョンが2つ未満)であれば、クラスター内で一緒に動作できるべきであることを意味します。ただし、分散クエリの低速化や、ReplicatedMergeTree における一部バックグラウンド処理でのリトライ可能なエラーなど、軽微な問題が発生する可能性があるため、クラスターのすべてのメンバーをできるだけ早く同じバージョンにアップグレードすることを推奨します。 - -リリース日の差が1年を超える異なるバージョンを同一クラスターで動作させることは決して推奨しません。データ損失は想定していないものの、クラスターが利用不能になる可能性があります。バージョン間の差が1年を超える場合に想定される問題には、以下が含まれます: - -- クラスターが動作しない可能性がある -- 一部(あるいはすべて)のクエリが予期しないエラーで失敗する可能性がある -- 予期しないエラーや警告がログに出力される可能性がある -- ダウングレードが不可能になる場合がある - -### 段階的なアップグレード {#incremental-upgrades} - -現在のバージョンと対象バージョンの差が1年を超える場合は、次のいずれかを推奨します: -- ダウンタイムを伴うアップグレード(すべてのサーバーを停止し、全サーバーをアップグレードしてから、すべてのサーバーを起動する)。 -- もしくは、中間バージョン(現在のバージョンから見て1年以内にリリースされた、より新しいバージョン)を経由してアップグレードする。 - - - -## 推奨プラン {#recommended-plan} - -以下は、ダウンタイムなしで ClickHouse をアップグレードするための推奨手順です。 - -1. 設定変更がデフォルトの `/etc/clickhouse-server/config.xml` ファイル内ではなく、`/etc/clickhouse-server/config.d/` 内にあることを確認します。`/etc/clickhouse-server/config.xml` はアップグレード時に上書きされる可能性があります。 -2. 対象リリースから現在利用しているリリースまでさかのぼって、非互換な変更点について [changelogs](/whats-new/changelog/index.md) を確認します。 -3. 非互換な変更点として特定されたうち、アップグレード前に実施できる更新を適用し、アップグレード後に実施が必要な変更点の一覧を作成します。 -4. 各シャードについて、他のレプリカをアップグレードしている間も稼働させておく 1 つ以上のレプリカを特定します。 -5. アップグレード対象のレプリカごとに、1 台ずつ以下を実施します。 - -* ClickHouse サーバーをシャットダウンする -* サーバーを対象バージョンにアップグレードする -* ClickHouse サーバーを起動する -* Keeper のメッセージを確認し、システムが安定した状態になるまで待つ -* 次のレプリカに進む - -6. Keeper ログおよび ClickHouse ログにエラーがないか確認します。 -7. 手順 4 で特定したレプリカを新しいバージョンにアップグレードします。 -8. 手順 1 ~ 3 で作成した変更点の一覧を参照し、アップグレード後に実施が必要な変更を適用します。 - -:::note -このエラーメッセージは、レプリケーション環境内で複数バージョンの ClickHouse が動作している場合に想定されるものです。すべてのレプリカが同一バージョンにアップグレードされると、このメッセージは表示されなくなります。 - -```text -MergeFromLogEntryTask: Code: 40. DB::Exception: Checksums of parts don't match: -hash of uncompressed files doesn't match. (CHECKSUM_DOESNT_MATCH) マージ後のデータが他のレプリカのデータとバイト単位で同一ではありません。 -``` - -::: - - -## ClickHouse サーバーバイナリのアップグレード手順 {#clickhouse-server-binary-upgrade-process} - -ClickHouse を `deb` パッケージからインストールした場合は、サーバー上で次のコマンドを実行します。 - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-client clickhouse-server -$ sudo service clickhouse-server restart -``` - -推奨されている `deb` パッケージ以外の方法で ClickHouse をインストールした場合は、適切な更新方法を使用してください。 - -:::note -1 つのシャードに属するすべてのレプリカが同時にオフラインになる瞬間さえなければ、複数のサーバーを同時に更新できます。 -::: - -古いバージョンの ClickHouse を特定のバージョンにアップグレードするには: - -例として: - -`xx.yy.a.b` は現在の安定版です。最新の安定版は[こちら](https://github.com/ClickHouse/ClickHouse/releases)で確認できます。 - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-server=xx.yy.a.b clickhouse-client=xx.yy.a.b clickhouse-common-static=xx.yy.a.b -$ sudo service clickhouse-server restart -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md deleted file mode 100644 index 74b8c157d53..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -description: 'OS のページキャッシュに依存せず、プロセス内メモリ上にデータをキャッシュするユーザースペースのページキャッシュ機構。' -sidebar_label: 'ユーザースペースページキャッシュ' -sidebar_position: 65 -slug: /operations/userspace-page-cache -title: 'ユーザースペースページキャッシュ' -doc_type: 'reference' ---- - - - -# ユーザー空間ページキャッシュ {#userspace-page-cache} - - - -## 概要 {#overview} - -> ユーザースペースページキャッシュは、新しいキャッシュ機構であり、OS のページキャッシュに依存するのではなく、プロセス内メモリ上にデータをキャッシュできるようにします。 - -ClickHouse にはすでに、Amazon S3、Google Cloud Storage (GCS)、Azure Blob Storage などのリモートオブジェクトストレージ上にキャッシュ層を設ける方法として、[Filesystem cache](/docs/operations/storing-data) が用意されています。ユーザースペースページキャッシュは、通常の OS キャッシュが十分に機能しない場合に、リモートデータへのアクセスを高速化することを目的として設計されています。 - -これは、ファイルシステムキャッシュと次の点で異なります。 - -| Filesystem Cache | ユーザースペースページキャッシュ | -|---------------------------------------------------------|---------------------------------------| -| データをローカルファイルシステムに書き込む | メモリ上にのみ存在する | -| ディスク容量を消費する(tmpfs 上でも構成可能) | ファイルシステムに依存しない | -| サーバーの再起動をまたいで保持される | サーバー再起動後は保持されない | -| サーバーのメモリ使用量としては表示されない | サーバーのメモリ使用量として表示される | -| ディスク上のデータおよびインメモリ(OS ページキャッシュ)の両方に適している | **ディスクレスサーバーに適している** | - - - -## 設定と使用方法 {#configuration-settings-and-usage} - -### 利用方法 {#usage} - -ユーザースペースページキャッシュを有効にするには、まずサーバー側で設定を行います。 - -```bash -cat config.d/page_cache.yaml -page_cache_max_size: 100G -``` - -:::note -ユーザー空間ページキャッシュは、指定された量までメモリを使用しますが、 -このメモリが予約されるわけではありません。サーバーの他の用途で -必要になった場合には、メモリは追い出されます。 -::: - -次に、クエリレベルでの利用を有効にします。 - -```sql -SET use_page_cache_for_disks_without_file_cache=1; -``` - -### 設定 {#settings} - -| Setting | Description | Default | -| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- | -| `use_page_cache_for_disks_without_file_cache` | ファイルシステムキャッシュが有効になっていないリモートディスクに対して、ユーザー空間ページキャッシュを使用します。 | `0` | -| `use_page_cache_with_distributed_cache` | 分散キャッシュが使用されている場合に、ユーザー空間ページキャッシュを使用します。 | `0` | -| `read_from_page_cache_if_exists_otherwise_bypass_cache` | パッシブモードでユーザー空間ページキャッシュを使用します。[`read_from_filesystem_cache_if_exists_otherwise_bypass_cache`](/docs/operations/settings/settings#read_from_filesystem_cache_if_exists_otherwise_bypass_cache) と同様の動作です。 | `0` | -| `page_cache_inject_eviction` | ユーザー空間ページキャッシュが、ランダムにいくつかのページを無効化することがあります。テスト用途を想定しています。 | `0` | -| `page_cache_block_size` | ユーザー空間ページキャッシュ内に保存するファイルチャンクのサイズ(バイト単位)です。キャッシュ経由で行われるすべての読み取りは、このサイズの倍数に切り上げられます。 | `1048576` | -| `page_cache_history_window_ms` | 解放されたメモリがユーザー空間ページキャッシュで再利用可能になるまでの遅延時間です。 | `1000` | -| `page_cache_policy` | ユーザー空間ページキャッシュのポリシー名です。 | `SLRU` | -| `page_cache_size_ratio` | ユーザー空間ページキャッシュにおける、保護キューのサイズがキャッシュ全体のサイズに対して占める比率です。 | `0.5` | -| `page_cache_min_size` | ユーザー空間ページキャッシュの最小サイズです。 | `104857600` | -| `page_cache_max_size` | ユーザー空間ページキャッシュの最大サイズです。0 に設定するとキャッシュを無効化します。`page_cache_min_size` より大きい場合、利用可能なメモリの大部分を使用しつつ、合計メモリ使用量が制限値(`max_server_memory_usage`[`_to_ram_ratio`]) を下回るよう、この範囲内でキャッシュサイズが継続的に調整されます。 | `0` | -| `page_cache_free_memory_ratio` | ユーザー空間ページキャッシュで確保せずに残しておくメモリ制限値の割合です。Linux の `min_free_kbytes` 設定に相当します。 | `0.15` | -| `page_cache_lookahead_blocks` | ユーザー空間ページキャッシュでキャッシュミスが発生し、かつそれらがキャッシュ内に存在しない場合に、基盤ストレージから一度にまとめて読み取る連続ブロック数の上限です。各ブロックのサイズは `page_cache_block_size` バイトです。 | `16` | -| `page_cache_shards` | ミューテックス競合を減らすために、ユーザー空間ページキャッシュを指定した数のシャードにストライプ化します。実験的な機能であり、性能向上はあまり期待できません。 | `4` | - - -## 関連コンテンツ {#related-content} -- [ファイルシステムキャッシュ](/docs/operations/storing-data) -- [ClickHouse v25.3 リリースウェビナー](https://www.youtube.com/live/iCKEzp0_Z2Q?feature=shared&t=1320) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md deleted file mode 100644 index b3da32316b6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'clickhouse_backupview のリファレンスドキュメント {#clickhouse_backupview}' -slug: /operations/utilities/backupview -title: 'clickhouse_backupview' -doc_type: 'reference' ---- - - - -# clickhouse_backupview {#clickhouse_backupview} - -[BACKUP](/operations/backup) コマンドによって作成されたバックアップを分析するための Python モジュールです。 -主な目的は、バックアップを実際にリストアすることなく、そのバックアップから情報を取得できるようにすることです。 - -このモジュールは、次の機能を提供します。 -- バックアップに含まれるファイルを列挙する -- バックアップからファイルを読み取る -- バックアップに含まれるデータベース、テーブル、パーツに関する有用な情報を可読な形式で取得する -- バックアップの整合性をチェックする - - - -## 例 {#example} - -```python -from clickhouse_backupview import open_backup, S3, FileInfo -``` - - -# バックアップを開きます。ローカルパスを利用することもできます: {#open-a-backup-we-could-also-use-a-local-path} -# backup = open_backup("/backups/my_backup_1/") {#backup-open_backupbackupsmy_backup_1} -backup = open_backup(S3("uri", "access_key_id", "secret_access_key")) - - - -# バックアップ内のデータベースの一覧を取得する {#get-a-list-of-databasess-inside-the-backup} -print(backup.get_databases())) - - - -# バックアップ内のテーブル一覧を取得し、 {#get-a-list-of-tables-inside-the-backup} -# 各テーブルについて、その作成クエリとパーツおよびパーティションの一覧を表示します。 {#and-for-each-table-its-create-query-and-a-list-of-parts-and-partitions} -for db in backup.get_databases(): - for tbl in backup.get_tables(database=db): - print(backup.get_create_query(database=db, table=tbl)) - print(backup.get_partitions(database=db, table=tbl)) - print(backup.get_parts(database=db, table=tbl)) - - - -# バックアップからすべてを抽出する {#extract-everything-from-the-backup} -backup.extract_all(table="mydb.mytable", out='/tmp/my_backup_1/all/') - - - -# 特定のテーブルのデータを抽出します。 {#extract-the-data-of-a-specific-table} -backup.extract_table_data(table="mydb.mytable", out='/tmp/my_backup_1/mytable/') - - - -# 1 つのパーティションを抽出する。 {#extract-a-single-partition} -backup.extract_table_data(table="mydb.mytable", partition="202201", out='/tmp/my_backup_1/202201/') - - - -# 単一のパートを抽出する。 {#extract-a-single-part} - -backup.extract_table_data(table="mydb.mytable", part="202201_100_200_3", out='/tmp/my_backup_1/202201_100_200_3/') - -``` - -その他の例については、[test](https://github.com/ClickHouse/ClickHouse/blob/master/utils/backupview/test/test.py)を参照してください。 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md deleted file mode 100644 index 7cada4f1465..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'clickhouse-benchmark のドキュメント' -sidebar_label: 'clickhouse-benchmark' -sidebar_position: 61 -slug: /operations/utilities/clickhouse-benchmark -title: 'clickhouse-benchmark' -doc_type: 'reference' ---- - - - -# clickhouse-benchmark {#clickhouse-benchmark} - -ClickHouse サーバーに接続し、指定したクエリを繰り返し送信します。 - -**構文** - -```bash -$ clickhouse-benchmark --query ["single query"] [keys] -``` - -または - -```bash -$ echo "single query" | clickhouse-benchmark [keys] -``` - -または - -```bash -$ clickhouse-benchmark [keys] <<< "single query" -``` - -クエリのセットを送信したい場合は、テキストファイルを作成し、このファイル内の各行に個々のクエリを1つずつ記述します。例: - -```sql -SELECT * FROM system.numbers LIMIT 10000000; -SELECT 1; -``` - -次に、このファイルを `clickhouse-benchmark` の標準入力に渡します。 - -```bash -clickhouse-benchmark [keys] < queries_file; -``` - - -## コマンドラインオプション {#clickhouse-benchmark-command-line-options} - -- `--query=QUERY` — 実行するクエリ。このパラメータが渡されない場合、`clickhouse-benchmark` は標準入力からクエリを読み込みます。 -- `--query_id=ID` — クエリ ID。 -- `--query_id_prefix=ID_PREFIX` — クエリ ID のプレフィックス。 -- `-c N`, `--concurrency=N` — `clickhouse-benchmark` が同時に送信するクエリ数。デフォルト値: 1。 -- `-C N`, `--max_concurrency=N` — 並列クエリ数を指定した値まで段階的に増やし、各並列度ごとにレポートを 1 つ作成します。 -- `--precise` — 重み付きメトリクスを用いた、インターバルごとの精密なレポートを有効にします。 -- `-d N`, `--delay=N` — 中間レポート間の間隔(秒)(レポートを無効にするには 0 を指定)。デフォルト値: 1。 -- `-h HOST`, `--host=HOST` — サーバーホスト。デフォルト値: `localhost`。[比較モード](#clickhouse-benchmark-comparison-mode) では複数の `-h` オプションを使用できます。 -- `-i N`, `--iterations=N` — クエリの総数。デフォルト値: 0(無限に繰り返す)。 -- `-r`, `--randomize` — 複数の入力クエリがある場合、クエリ実行順序をランダムにします。 -- `-s`, `--secure` — `TLS` 接続を使用します。 -- `-t N`, `--timelimit=N` — 時間制限(秒)。指定した時間制限に達すると、`clickhouse-benchmark` はクエリ送信を停止します。デフォルト値: 0(時間制限なし)。 -- `--port=N` — サーバーポート。デフォルト値: 9000。[比較モード](#clickhouse-benchmark-comparison-mode) では複数の `--port` オプションを使用できます。 -- `--confidence=N` — t 検定の信頼水準。指定可能な値: 0 (80%), 1 (90%), 2 (95%), 3 (98%), 4 (99%), 5 (99.5%)。デフォルト値: 5。[比較モード](#clickhouse-benchmark-comparison-mode) では、`clickhouse-benchmark` は選択された信頼水準で 2 つの分布に差がないかを判定するために [独立 2 標本スチューデントの t 検定](https://en.wikipedia.org/wiki/Student%27s_t-test#Independent_two-sample_t-test) を実行します。 -- `--cumulative` — インターバルごとのデータではなく累積データを出力します。 -- `--database=DATABASE_NAME` — ClickHouse データベース名。デフォルト値: `default`。 -- `--user=USERNAME` — ClickHouse ユーザー名。デフォルト値: `default`。 -- `--password=PSWD` — ClickHouse ユーザーパスワード。デフォルト値: 空文字列。 -- `--stacktrace` — スタックトレースを出力します。このオプションが指定されている場合、`clickhouse-bencmark` は例外のスタックトレースを出力します。 -- `--stage=WORD` — サーバー側でのクエリ処理ステージ。ClickHouse は指定されたステージでクエリ処理を停止し、その時点の結果を `clickhouse-benchmark` に返します。指定可能な値: `complete`, `fetch_columns`, `with_mergeable_state`。デフォルト値: `complete`。 -- `--roundrobin` — 複数の `--host`/`--port` を比較する代わりに、クエリごとにランダムに 1 つの `--host`/`--port` を選択して、そのホストにクエリを送信します。 -- `--reconnect=N` — 再接続の動作を制御します。指定可能な値: 0(再接続しない)、1(クエリごとに再接続)、N(N クエリごとに再接続)。デフォルト値: 0。 -- `--max-consecutive-errors=N` — 許容される連続エラー数。デフォルト値: 0。 -- `--ignore-error`,`--continue_on_errors` — クエリが失敗してもテストを継続します。 -- `--client-side-time` — サーバー側の時間ではなく、ネットワーク通信を含むクライアント側の時間を表示します。サーバーバージョン 22.8 より前では、常にクライアント側の時間が表示される点に注意してください。 -- `--proto-caps` — データ転送時のチャンク化を有効/無効にします。指定可能な値(カンマ区切りで複数指定可): `chunked_optional`, `notchunked`, `notchunked_optional`, `send_chunked`, `send_chunked_optional`, `send_notchunked`, `send_notchunked_optional`, `recv_chunked`, `recv_chunked_optional`, `recv_notchunked`, `recv_notchunked_optional`。デフォルト値: `notchunked`。 -- `--help` — ヘルプメッセージを表示します。 -- `--verbose` — ヘルプメッセージの詳細度を上げます。 - -クエリに対していくつかの[設定](/operations/settings/overview)を適用したい場合は、`--= SETTING_VALUE` というオプションとして渡します。たとえば、`--max_memory_usage=1048576` のようになります。 - - - -## 環境変数オプション {#clickhouse-benchmark-environment-variable-options} - -ユーザー名、パスワード、およびホストは、環境変数 `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD`、`CLICKHOUSE_HOST` を使って設定できます。 -コマンドライン引数 `--user`、`--password`、`--host` が、環境変数よりも優先されます。 - - - -## 出力 {#clickhouse-benchmark-output} - -デフォルトでは、`clickhouse-benchmark` は各 `--delay` 間隔ごとにレポートを出力します。 - -レポート例: - -```text -Queries executed: 10. - -localhost:9000, queries 10, QPS: 6.772, RPS: 67904487.440, MiB/s: 518.070, result RPS: 67721584.984, result MiB/s: 516.675. - -0.000% 0.145 sec. -10.000% 0.146 sec. -20.000% 0.146 sec. -30.000% 0.146 sec. -40.000% 0.147 sec. -50.000% 0.148 sec. -60.000% 0.148 sec. -70.000% 0.148 sec. -80.000% 0.149 sec. -90.000% 0.150 sec. -95.000% 0.150 sec. -99.000% 0.150 sec. -99.900% 0.150 sec. -99.990% 0.150 sec. -``` - -レポートでは次の情報を確認できます: - -* `Queries executed:` フィールドにおけるクエリ数。 - -* 次の内容をこの順序で含むステータス文字列: - - * ClickHouse サーバーのエンドポイント。 - * 処理されたクエリ数。 - * QPS: `--delay` 引数で指定された期間中に、サーバーが 1 秒あたりに実行したクエリ数。 - * RPS: `--delay` 引数で指定された期間中に、サーバーが 1 秒あたりに読み取った行数。 - * MiB/s: `--delay` 引数で指定された期間中に、サーバーが 1 秒あたりに読み取ったメビバイト数 (MiB)。 - * result RPS: `--delay` 引数で指定された期間中に、サーバーがクエリ結果に 1 秒あたりに出力した行数。 - * result MiB/s: `--delay` 引数で指定された期間中に、サーバーがクエリ結果に 1 秒あたりに出力したメビバイト数 (MiB)。 - -* クエリ実行時間のパーセンタイル値。 - - -## 比較モード {#clickhouse-benchmark-comparison-mode} - -`clickhouse-benchmark` は、稼働中の 2 つの ClickHouse サーバーのパフォーマンスを比較できます。 - -比較モードを使用するには、両方のサーバーのエンドポイントを、2 組の `--host` と `--port` キーを使って指定します。キーは引数リスト内での位置によって対応付けられ、最初の `--host` は最初の `--port` に対応し、以降も同様です。`clickhouse-benchmark` は両方のサーバーへの接続を確立した後、クエリを送信します。各クエリはランダムに選択されたどちらかのサーバーに送られます。結果はテーブル形式で表示されます。 - - - -## 例 {#clickhouse-benchmark-example} - -```bash -$ echo "SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000" | clickhouse-benchmark --host=localhost --port=9001 --host=localhost --port=9000 -i 10 -``` - -```text -1件のクエリを読み込みました。 - -実行されたクエリ: 5件 - -localhost:9001, queries 2, QPS: 3.764, RPS: 75446929.370, MiB/s: 575.614, result RPS: 37639659.982, result MiB/s: 287.168. -localhost:9000, queries 3, QPS: 3.815, RPS: 76466659.385, MiB/s: 583.394, result RPS: 38148392.297, result MiB/s: 291.049. - -0.000% 0.258 sec. 0.250 sec. -10.000% 0.258 sec. 0.250 sec. -20.000% 0.258 sec. 0.250 sec. -30.000% 0.258 sec. 0.267 sec. -40.000% 0.258 sec. 0.267 sec. -50.000% 0.273 sec. 0.267 sec. -60.000% 0.273 sec. 0.267 sec. -70.000% 0.273 sec. 0.267 sec. -80.000% 0.273 sec. 0.269 sec. -90.000% 0.273 sec. 0.269 sec. -95.000% 0.273 sec. 0.269 sec. -99.000% 0.273 sec. 0.269 sec. -99.900% 0.273 sec. 0.269 sec. -99.990% 0.273 sec. 0.269 sec. - -99.5%信頼水準で有意差なし -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md deleted file mode 100644 index 076d00e3eba..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'ClickHouse Compressor のドキュメント' -slug: /operations/utilities/clickhouse-compressor -title: 'clickhouse-compressor' -doc_type: 'reference' ---- - -データの圧縮および解凍を行うシンプルなプログラムです。 - -### 例 {#examples} - -LZ4 でデータを圧縮します: - -```bash -$ ./clickhouse-compressor < input_file > output_file -``` - -LZ4 形式のデータを展開します: - -```bash -$ ./clickhouse-compressor --decompress < input_file > output_file -``` - -ZSTD のレベル 5 でデータを圧縮します: - -```bash -$ ./clickhouse-compressor --codec 'ZSTD(5)' < input_file > output_file -``` - -4 バイトのデルタ圧縮と ZSTD レベル 10 を用いてデータを圧縮します。 - -```bash -$ ./clickhouse-compressor --codec 'Delta(4)' --codec 'ZSTD(10)' < input_file > output_file -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md deleted file mode 100644 index 04b59508b5d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse-disks のリファレンス' -sidebar_label: 'clickhouse-disks' -sidebar_position: 59 -slug: /operations/utilities/clickhouse-disks -title: 'ClickHouse-disks' -doc_type: 'reference' ---- - - - -# Clickhouse-disks {#clickhouse-disks} - -ClickHouse ディスクに対して、ファイルシステムのような操作を提供するユーティリティです。インタラクティブおよび非インタラクティブの両モードで動作します。 - - - -## プログラム全体のオプション {#program-wide-options} - -* `--config-file, -C` -- 使用する ClickHouse の設定ファイルへのパス。既定値は `/etc/clickhouse-server/config.xml`。 -* `--save-logs` -- 実行したコマンドの進行状況を `/var/log/clickhouse-server/clickhouse-disks.log` にログ出力します。 -* `--log-level` -- ログ出力するイベントの[種類](../server-configuration-parameters/settings#logger)。既定値は `none`。 -* `--disk` -- `mkdir, move, read, write, remove` コマンドで使用するディスク。既定値は `default`。 -* `--query, -q` -- 対話モードを起動せずに実行できる単一のクエリ。 -* `--help, -h` -- すべてのオプションとコマンド、およびその説明を表示します。 - - - -## 遅延初期化 {#lazy-initialization} -設定に記載されているすべてのディスクは、遅延初期化されます。つまり、あるディスクに対応するオブジェクトは、そのディスクが何らかのコマンドで実際に使用されたときにのみ初期化されます。これは、ユーティリティをより堅牢にし、設定には記述されているもののユーザーが使用しておらず、初期化時に失敗する可能性のあるディスクに触れないようにするためです。ただし、`clickhouse-disks` の起動時に初期化されるディスクが 1 つ存在している必要があります。このディスクは、コマンドラインからパラメータ `--disk` を使用して指定します(デフォルト値は `default` です)。 - - - -## デフォルトディスク {#default-disks} -起動後、設定には明示されていませんが、初期化に利用可能なディスクが 2 つあります。 - -1. **`local` ディスク**: このディスクは、`clickhouse-disks` ユーティリティが起動された元のローカルファイルシステムを模倣するように設計されています。初期パスは `clickhouse-disks` が開始されたディレクトリであり、ファイルシステムのルートディレクトリにマウントされます。 - -2. **`default` ディスク**: このディスクは、設定内の `clickhouse/path` パラメータで指定されたディレクトリ(デフォルト値は `/var/lib/clickhouse`)に、ローカルファイルシステム上のディレクトリとしてマウントされます。初期パスは `/` に設定されています。 - - - -## Clickhouse-disks の状態 {#clickhouse-disks-state} -追加された各ディスクについて、このユーティリティは現在のディレクトリ(通常のファイルシステムと同様)を記録します。ユーザーは現在のディレクトリを変更したり、ディスク間を切り替えたりできます。 - -状態はプロンプト "`disk_name`:`path_name`" に反映されます。 - - - -## コマンド {#commands} - -このドキュメントでは、必須の位置引数は ``、名前付き引数は `[--parameter value]` の形式で表記します。すべての位置引数は、対応する名前を用いた名前付き引数として指定することもできます。 - -* `cd (change-dir, change_dir) [--disk disk] ` - ディスク `disk` 上のパス `path` をカレントディレクトリに変更します(デフォルト値は現在のディスク)。ディスクの切り替えは行われません。 -* `copy (cp) [--disk-from disk_1] [--disk-to disk_2] `. - ディスク `disk_1` 上の `path-from` からデータを再帰的にコピーし(デフォルト値は現在のディスク(非対話モードではパラメータ `disk`))、 - ディスク `disk_2` 上の `path-to` にコピーします(デフォルト値は現在のディスク(非対話モードではパラメータ `disk`))。 -* `current_disk_with_path (current, current_disk, current_path)` - 現在の状態を次の形式で出力します: - `Disk: "current_disk" Path: "current path on current disk"` -* `help []` - コマンド `command` に関するヘルプメッセージを出力します。`command` が指定されていない場合は、すべてのコマンドに関する情報を出力します。 -* `move (mv) `. - 現在のディスク内で、`path-from` から `path-to` へファイルまたはディレクトリを移動します。 -* `remove (rm, delete) `. - 現在のディスク上で `path` を再帰的に削除します。 -* `link (ln) `. - 現在のディスク上で、`path-from` から `path-to` へのハードリンクを作成します。 -* `list (ls) [--recursive] ` - 現在のディスク上の `path` にあるファイルを一覧表示します。デフォルトでは再帰的に一覧表示しません。 -* `list-disks (list_disks, ls-disks, ls_disks)`. - ディスク名を一覧表示します。 -* `mkdir [--recursive] ` 現在のディスク上で実行。 - ディレクトリを作成します。デフォルトでは再帰的に作成しません。 -* `read (r) [--path-to path]` - `path-from` からファイルを読み取り、`path` に出力します(指定されていない場合は `stdout` に出力します)。 -* `switch-disk [--path path] ` - パス `path` 上のディスク `disk` に切り替えます(`path` が指定されていない場合のデフォルト値は、ディスク `disk` 上の直前のパスです)。 -* `write (w) [--path-from path] `. - `path` からファイルを書き込み、`path-to` に出力します(`path` が指定されていない場合は `stdin` を使用し、入力は Ctrl+D で終了する必要があります)。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md deleted file mode 100644 index 5e4ed4892ec..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: 'ClickHouse のデータフォーマットを扱うための `clickhouse-format` ユーティリティの使用ガイド' -slug: /operations/utilities/clickhouse-format -title: 'clickhouse-format' -doc_type: 'reference' ---- - -# clickhouse-format ユーティリティ {#clickhouse-format-utility} - -入力クエリを整形します。 - -オプション: - -* `--help` または `-h` — ヘルプメッセージを出力します。 -* `--query` — 任意の長さや複雑さのクエリを整形します。 -* `--hilite` または `--highlight` — ANSI ターミナルのエスケープシーケンスを使用して構文ハイライトを追加します。 -* `--oneline` — 1 行で整形します。 -* `--max_line_length` — 指定した長さ未満のクエリを 1 行で整形します。 -* `--comments` — 出力にコメントを保持します。 -* `--quiet` または `-q` — 構文のみをチェックし、成功時は出力しません。 -* `--multiquery` または `-n` — 同一ファイル内で複数のクエリを許可します。 -* `--obfuscate` — 整形の代わりに難読化を行います。 -* `--seed ` — 難読化の結果を決定する任意のシード文字列を指定します。 -* `--backslash` — 整形されたクエリの各行末にバックスラッシュを追加します。複数行のクエリを Web などからコピーしてコマンドラインで実行したい場合に便利です。 -* `--semicolons_inline` — multiquery モードで、クエリの末尾行では改行せず同じ行にセミコロンを書きます。 - -## 例 {#examples} - -1. クエリのフォーマット: - -```bash -$ clickhouse-format --query "select number from numbers(10) where number%2 order by number desc;" -``` - -結果: - -```bash -SELECT number -FROM numbers(10) -WHERE number % 2 -ORDER BY number DESC -``` - -2. ハイライトと1行表示: - -```bash -$ clickhouse-format --oneline --hilite <<< "SELECT sum(number) FROM numbers(5);" -``` - -結果: - -```sql -SELECT sum(number) FROM numbers(5) -``` - -3. マルチクエリ: - -```bash -$ clickhouse-format -n <<< "SELECT min(number) FROM numbers(5); SELECT max(number) FROM numbers(5);" -``` - -結果: - -```sql -SELECT min(number) -FROM numbers(5) -; - -SELECT max(number) -FROM numbers(5) -; - -``` - -4. 難読化: - -```bash -$ clickhouse-format --seed Hello --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -結果: - -```sql -SELECT treasury_mammoth_hazelnut BETWEEN nutmeg AND span, CASE WHEN chive >= 116 THEN switching ELSE ANYTHING END; -``` - -同じクエリで別のシード文字列を使用した例: - -```bash -$ clickhouse-format --seed World --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -結果: - -```sql -SELECT horse_tape_summer BETWEEN folklore AND moccasins, CASE WHEN intestine >= 116 THEN nonconformist ELSE FORESTRY END; -``` - -5. バックスラッシュの追加: - -```bash -$ clickhouse-format --backslash <<< "SELECT * FROM (SELECT 1 AS x UNION ALL SELECT 1 UNION DISTINCT SELECT 3);" -``` - -結果: - -```sql -SELECT * \ -FROM \ -( \ - SELECT 1 AS x \ - UNION ALL \ - SELECT 1 \ - UNION DISTINCT \ - SELECT 3 \ -) -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md deleted file mode 100644 index dd3e4962eee..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'ClickHouse Keeper クライアント ユーティリティのドキュメント' -sidebar_label: 'clickhouse-keeper-client' -slug: /operations/utilities/clickhouse-keeper-client -title: 'clickhouse-keeper-client ユーティリティ' -doc_type: 'reference' ---- - - - -# clickhouse-keeper-client ユーティリティ {#clickhouse-keeper-client-utility} - -ネイティブプロトコルを使用して clickhouse-keeper と通信するためのクライアントアプリケーションです。 - - - -## オプション {#clickhouse-keeper-client} - -- `-q QUERY`, `--query=QUERY` — 実行するクエリ。 このパラメータが指定されない場合、`clickhouse-keeper-client` はインタラクティブモードで起動します。 -- `-h HOST`, `--host=HOST` — サーバーのホスト名。 デフォルト値: `localhost`。 -- `-p N`, `--port=N` — サーバーのポート番号。 デフォルト値: 9181。 -- `-c FILE_PATH`, `--config-file=FILE_PATH` — 接続文字列を取得するための設定ファイルのパスを指定します。 デフォルト値: `config.xml`。 -- `--connection-timeout=TIMEOUT` — 接続タイムアウトを秒単位で指定します。 デフォルト値: 10s。 -- `--session-timeout=TIMEOUT` — セッションタイムアウトを秒単位で指定します。 デフォルト値: 10s。 -- `--operation-timeout=TIMEOUT` — オペレーションタイムアウトを秒単位で指定します。 デフォルト値: 10s。 -- `--history-file=FILE_PATH` — 履歴ファイルのパスを指定します。 デフォルト値: `~/.keeper-client-history`。 -- `--log-level=LEVEL` — ログレベルを指定します。 デフォルト値: `information`。 -- `--no-confirmation` — 指定した場合、いくつかのコマンドで確認を求めません。 インタラクティブモードではデフォルト値は `false`、クエリでは `true` です。 -- `--help` — ヘルプメッセージを表示します。 - - - -## 例 {#clickhouse-keeper-client-example} - -```bash -./clickhouse-keeper-client -h localhost -p 9181 --connection-timeout 30 --session-timeout 30 --operation-timeout 30 -ZooKeeperに接続しました [::1]:9181 session_id 137 -/ :) ls -keeper foo bar -/ :) cd 'keeper' -/keeper :) ls -api_version -/keeper :) cd 'api_version' -/keeper/api_version :) ls - -/keeper/api_version :) cd 'xyz' -パス /keeper/api_version/xyz は存在しません -/keeper/api_version :) cd ../../ -/ :) ls -keeper foo bar -/ :) get 'keeper/api_version' -2 -``` - - -## コマンド {#clickhouse-keeper-client-commands} - -- `ls '[path]'` -- 指定されたパスのノードを一覧表示します(デフォルト: カレントディレクトリ) -- `cd '[path]'` -- 作業パスを変更します(デフォルト: `.`) -- `cp '' ''` -- `src` ノードを `dest` パスにコピーします -- `cpr '' ''` -- `src` ノードのサブツリーを `dest` パスにコピーします -- `mv '' ''` -- `src` ノードを `dest` パスに移動します -- `mvr '' ''` -- `src` ノードのサブツリーを `dest` パスに移動します -- `exists ''` -- ノードが存在する場合は `1`、それ以外は `0` を返します -- `set '' [version]` -- ノードの値を更新します。バージョンが一致する場合にのみ更新します(デフォルト: -1) -- `create '' [mode]` -- 指定した値で新しいノードを作成します -- `touch ''` -- 値が空文字列の新しいノードを作成します。ノードがすでに存在している場合でも例外はスローされません -- `get ''` -- ノードの値を返します -- `rm '' [version]` -- バージョンが一致する場合にのみノードを削除します(デフォルト: -1) -- `rmr '' [limit]` -- サブツリーのサイズが上限より小さい場合に、パスを再帰的に削除します。確認が必要です(デフォルトの上限 = 100) -- `flwc ` -- four-letter-word コマンドを実行します -- `help` -- このヘルプメッセージを表示します -- `get_direct_children_number '[path]'` -- 特定のパス直下の子ノード数を取得します -- `get_all_children_number '[path]'` -- 特定のパス配下のすべての子ノード数を取得します -- `get_stat '[path]'` -- ノードの stat を返します(デフォルト: `.`) -- `find_super_nodes '[path]'` -- 指定されたパスに対して、子ノード数がしきい値より大きいノードを検索します(デフォルト: `.`) -- `delete_stale_backups` -- 現在は非アクティブなバックアップ用の ClickHouse ノードを削除します -- `find_big_family [path] [n]` -- サブツリー内で子ノードが最も多い上位 n 個のノードを返します(デフォルト: path = `.`、n = 10) -- `sync ''` -- プロセスとリーダー間でノードを同期します -- `reconfig "" [version]` -- Keeper クラスターを再構成します。/docs/en/guides/sre/keeper/clickhouse-keeper#reconfiguration を参照してください diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md deleted file mode 100644 index 5cd21043565..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md +++ /dev/null @@ -1,323 +0,0 @@ ---- -description: 'サーバーを使わずにデータを処理するための clickhouse-local 利用ガイド' -sidebar_label: 'clickhouse-local' -sidebar_position: 60 -slug: /operations/utilities/clickhouse-local -title: 'clickhouse-local' -doc_type: 'reference' ---- - - - -# clickhouse-local {#clickhouse-local} - - - -## clickhouse-local を使うときと ClickHouse を使うとき {#when-to-use-clickhouse-local-vs-clickhouse} - -`clickhouse-local` は、フル機能のデータベースサーバーをインストールすることなく、SQL を使ってローカルおよびリモートファイルに対して高速な処理を行いたい開発者に最適な、使いやすい ClickHouse のバージョンです。`clickhouse-local` を使用すると、開発者はコマンドラインから直接 [ClickHouse SQL dialect](../../sql-reference/index.md) を用いた SQL コマンドを実行でき、フルの ClickHouse をインストールすることなく ClickHouse の機能にシンプルかつ効率的にアクセスできます。`clickhouse-local` の主な利点の 1 つは、[clickhouse-client](/operations/utilities/clickhouse-local) をインストールする際に同梱されていることです。これにより、複雑なインストール手順なしに、開発者はすぐに `clickhouse-local` を使い始めることができます。 - -`clickhouse-local` は、開発およびテスト用途、ならびにファイル処理に非常に有用なツールですが、エンドユーザーやアプリケーションに対するサービス提供には適していません。これらのシナリオでは、オープンソースの [ClickHouse](/install) を使用することを推奨します。ClickHouse は、大規模な分析ワークロードを処理するように設計された強力な OLAP データベースです。大規模なデータセットに対する複雑なクエリを高速かつ効率的に処理できるため、高パフォーマンスが重要となる本番環境に最適です。加えて ClickHouse は、レプリケーション、シャーディング、高可用性など、大規模データセットの処理やアプリケーション提供に必要となるスケールアウトのための幅広い機能を提供します。より大きなデータセットを扱う必要がある場合や、エンドユーザーまたはアプリケーションに提供する必要がある場合は、`clickhouse-local` ではなくオープンソースの ClickHouse を使用することを推奨します。 - -以下のドキュメントでは、[ローカルファイルのクエリ](#query_data_in_file) や [S3 上の Parquet ファイルの読み取り](#query-data-in-a-parquet-file-in-aws-s3) など、`clickhouse-local` の代表的なユースケースを示していますので、参照してください。 - - - -## clickhouse-local をダウンロードする {#download-clickhouse-local} - -`clickhouse-local` は、ClickHouse サーバーや `clickhouse-client` と同じ `clickhouse` バイナリで実行されます。最新バージョンをダウンロードする最も簡単な方法は、次のコマンドを使用することです。 - -```bash -curl https://clickhouse.com/ | sh -``` - -:::note -ダウンロードしたばかりのバイナリは、さまざまな ClickHouse ツールやユーティリティを実行できます。ClickHouse をデータベースサーバーとして実行したい場合は、[クイックスタート](/get-started/quick-start)を参照してください。 -::: - - -## SQL を使用してファイル内のデータをクエリする {#query_data_in_file} - -`clickhouse-local` の一般的な用途は、データをテーブルに挿入することなく、ファイルに対してアドホックなクエリを実行することです。`clickhouse-local` はファイルから一時テーブルへデータをストリーミングし、その一時テーブルに対して SQL を実行できます。 - -ファイルが `clickhouse-local` と同じマシン上にある場合は、読み込むファイルを指定するだけで構いません。次の `reviews.tsv` ファイルには、Amazon の商品レビューのサンプルが含まれています。 - -```bash -./clickhouse local -q "SELECT * FROM 'reviews.tsv'" -``` - -このコマンドは、次のコマンドのショートカットです: - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv')" -``` - -ClickHouse は、ファイル名の拡張子からそのファイルがタブ区切り形式であることを認識します。形式を明示的に指定する必要がある場合は、単に [多様な ClickHouse の入力フォーマット](../../interfaces/formats.md) のいずれかを指定してください。 - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv', 'TabSeparated')" -``` - -`file` テーブル関数はテーブルを作成し、`DESCRIBE` を使って推論されたスキーマを確認できます。 - -```bash -./clickhouse local -q "DESCRIBE file('reviews.tsv')" -``` - -:::tip -ファイル名にはグロブを使用できます([グロブ置換](/sql-reference/table-functions/file.md/#globs-in-path)を参照してください)。 - -例: - -```bash -./clickhouse local -q "SELECT * FROM 'reviews*.jsonl'" -./clickhouse local -q "SELECT * FROM 'review_?.csv'" -./clickhouse local -q "SELECT * FROM 'review_{1..3}.csv'" -``` - -::: - -```response -marketplace Nullable(String) -customer_id Nullable(Int64) -review_id Nullable(String) -product_id Nullable(String) -product_parent Nullable(Int64) -product_title Nullable(String) -product_category Nullable(String) -star_rating Nullable(Int64) -helpful_votes Nullable(Int64) -total_votes Nullable(Int64) -vine Nullable(String) -verified_purchase Nullable(String) -review_headline Nullable(String) -review_body Nullable(String) -review_date Nullable(Date) -``` - -評価が最も高い製品を探してみましょう。 - -```bash -./clickhouse local -q "SELECT - argMax(product_title,star_rating), - max(star_rating) -FROM file('reviews.tsv')" -``` - -```response -Monopoly Junior Board Game 5 -``` - - -## AWS S3 内の Parquet ファイルをクエリする {#query-data-in-a-parquet-file-in-aws-s3} - -S3 にファイルがある場合は、`clickhouse-local` と `s3` テーブル関数を使用して、データを ClickHouse のテーブルに挿入せずに、そのファイルをその場でクエリできます。ここでは、英国で売却された不動産の住宅価格を含む `house_0.parquet` という名前のファイルが、パブリックなバケット内にあります。このファイルに何行含まれているかを確認してみましょう。 - -```bash -./clickhouse local -q " -SELECT count() -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -このファイルには 270万行あります: - -```response -2772030 -``` - -ClickHouse がファイルからどのようなスキーマを推論したかを確認しておくと便利です。 - -```bash -./clickhouse local -q "DESCRIBE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -```response -price Nullable(Int64) -date Nullable(UInt16) -postcode1 Nullable(String) -postcode2 Nullable(String) -type Nullable(String) -is_new Nullable(UInt8) -duration Nullable(String) -addr1 Nullable(String) -addr2 Nullable(String) -street Nullable(String) -locality Nullable(String) -town Nullable(String) -district Nullable(String) -county Nullable(String) -``` - -最も高額な地域を見てみましょう。 - -```bash -./clickhouse local -q " -SELECT - town, - district, - count() AS c, - round(avg(price)) AS price, - bar(price, 0, 5000000, 100) -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet') -GROUP BY - town, - district -HAVING c >= 100 -ORDER BY price DESC -LIMIT 10" -``` - -```response -LONDON CITY OF LONDON 886 2271305 █████████████████████████████████████████████▍ -LEATHERHEAD ELMBRIDGE 206 1176680 ███████████████████████▌ -LONDON CITY OF WESTMINSTER 12577 1108221 ██████████████████████▏ -LONDON KENSINGTON AND CHELSEA 8728 1094496 █████████████████████▉ -HYTHE FOLKESTONE AND HYTHE 130 1023980 ████████████████████▍ -CHALFONT ST GILES CHILTERN 113 835754 ████████████████▋ -AMERSHAM BUCKINGHAMSHIRE 113 799596 ███████████████▉ -VIRGINIA WATER RUNNYMEDE 356 789301 ███████████████▊ -BARNET ENFIELD 282 740514 ██████████████▊ -NORTHWOOD THREE RIVERS 184 731609 ██████████████▋ -``` - -:::tip -ファイルを ClickHouse に取り込む準備ができたら、ClickHouse サーバーを起動して、`file` および `s3` テーブル関数の結果を `MergeTree` テーブルに挿入します。詳細については [Quick Start](/get-started/quick-start) を参照してください。 -::: - - -## フォーマット変換 {#format-conversions} - -`clickhouse-local` を使用して、異なるフォーマット間でデータを変換できます。例: - -```bash -$ clickhouse-local --input-format JSONLines --output-format CSV --query "SELECT * FROM table" < data.json > data.csv -``` - -形式はファイル拡張子から自動的に判別されます。 - -```bash -$ clickhouse-local --query "SELECT * FROM table" < data.json > data.csv -``` - -簡単に書くには、`--copy` 引数を指定して記述することもできます: - -```bash -$ clickhouse-local --copy < data.json > data.csv -``` - - -## 使用方法 {#usage} - -デフォルトでは、`clickhouse-local` は同一ホスト上の ClickHouse サーバーのデータにアクセスでき、サーバーの設定には依存しません。`--config-file` 引数を使用してサーバーの設定を読み込むこともできます。一時データ用には、デフォルトで一意の一時データディレクトリが作成されます。 - -基本的な使用方法(Linux): - -```bash -$ clickhouse-local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -基本的な使い方(Mac): - -```bash -$ ./clickhouse local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -:::note -`clickhouse-local` は、Windows では WSL2 経由でも利用できます。 -::: - -引数: - -* `-S`, `--structure` — 入力データのテーブル構造。 -* `--input-format` — 入力フォーマット。デフォルトは `TSV`。 -* `-F`, `--file` — データのパス。デフォルトは `stdin`。 -* `-q`, `--query` — 実行するクエリ(区切りは `;`)。`--query` は複数回指定可能です(例:`--query "SELECT 1" --query "SELECT 2"`)。`--queries-file` と同時には使用できません。 -* `--queries-file` - 実行するクエリを含むファイルパス。`--queries-file` は複数回指定可能です(例:`--query queries1.sql --query queries2.sql`)。`--query` と同時には使用できません。 -* `--multiquery, -n` – 指定した場合、セミコロン区切りの複数クエリを `--query` オプションの後に列挙できます。利便性のため、`--query` を省略して `--multiquery` の後にクエリを直接渡すことも可能です。 -* `-N`, `--table` — 出力データを書き込むテーブル名。デフォルトは `table`。 -* `-f`, `--format`, `--output-format` — 出力フォーマット。デフォルトは `TSV`。 -* `-d`, `--database` — デフォルトデータベース。デフォルトは `_local`。 -* `--stacktrace` — 例外発生時にデバッグ出力をダンプするかどうか。 -* `--echo` — 実行前にクエリを表示します。 -* `--verbose` — クエリ実行の詳細をより多く出力します。 -* `--logger.console` — コンソールにログを出力します。 -* `--logger.log` — ログファイル名。 -* `--logger.level` — ログレベル。 -* `--ignore-error` — クエリが失敗しても処理を停止しません。 -* `-c`, `--config-file` — ClickHouse サーバーと同じ形式の設定ファイルへのパス。デフォルトでは設定は空です。 -* `--no-system-tables` — system テーブルをアタッチしません。 -* `--help` — `clickhouse-local` の引数リファレンスを表示します。 -* `-V`, `--version` — バージョン情報を表示して終了します。 - -また、`--config-file` の代わりによく用いられる、各 ClickHouse 設定変数に対応する引数も用意されています。 - - -## 例 {#examples} - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local --structure "a Int64, b Int64" \ - --input-format "CSV" --query "SELECT * FROM table" -2行読み込み、32.00 B、0.000秒、5182行/秒、80.97 KiB/秒 -1 2 -3 4 -``` - -先ほどの例は次と同じです。 - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -n --query " - CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); - SELECT a, b FROM table; - DROP TABLE table;" -2行読み込み、32.00 B、0.000秒、4987行/秒、77.93 KiB/秒。 -1 2 -3 4 -``` - -`stdin` や `--file` 引数を使う必要はなく、[`file` テーブル関数](../../sql-reference/table-functions/file.md) を使えば任意の数のファイルを開けます。 - -```bash -$ echo 1 | tee 1.tsv -1 - -$ echo 2 | tee 2.tsv -2 - -$ clickhouse-local --query " - select * from file('1.tsv', TSV, 'a int') t1 - cross join file('2.tsv', TSV, 'b int') t2" -1 2 -``` - -では、各 Unix ユーザーごとのメモリ使用量を出力してみましょう。 - -クエリ: - -```bash -$ ps aux | tail -n +2 | awk '{ printf("%s\t%s\n", $1, $4) }' \ - | clickhouse-local --structure "user String, mem Float64" \ - --query "SELECT user, round(sum(mem), 2) as memTotal - FROM table GROUP BY user ORDER BY memTotal DESC FORMAT Pretty" -``` - -結果: - -```text -186行、4.15 KiBを0.035秒で読み込み、5302行/秒、118.34 KiB/秒 -┏━━━━━━━━━━┳━━━━━━━━━━┓ -┃ user ┃ memTotal ┃ -┡━━━━━━━━━━╇━━━━━━━━━━┩ -│ bayonet │ 113.5 │ -├──────────┼──────────┤ -│ root │ 8.8 │ -├──────────┼──────────┤ -... -``` - - -## 関連コンテンツ {#related-content-1} - -- [clickhouse-local を使用してローカルファイル内のデータを抽出、変換、クエリする](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) -- [ClickHouse へのデータ取り込み - パート 1](https://clickhouse.com/blog/getting-data-into-clickhouse-part-1) -- [大規模な実世界データセットを探索する:ClickHouse で 100 年以上の気象記録を扱う](https://clickhouse.com/blog/real-world-data-noaa-climate-data) -- ブログ:[clickhouse-local を使用してローカルファイル内のデータを抽出、変換、クエリする](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md deleted file mode 100644 index e73d067af38..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'ClickHouse Obfuscator のドキュメント' -slug: /operations/utilities/clickhouse-obfuscator -title: 'clickhouse-obfuscator' -doc_type: 'reference' ---- - -テーブルデータを難読化するためのシンプルなツールです。 - -入力テーブルを読み込み、入力テーブルのいくつかの特性は保持しつつも異なるデータを含む出力テーブルを生成します。 -これにより、ベンチマークで利用する目的で、ほぼ実運用データに近いデータを公開できるようになります。 - -このツールは、以下のデータ特性を保持するように設計されています。 - -- 各カラムおよびカラムの組の「値のカーディナリティ(異なる値の個数)」; -- 条件付きカーディナリティ:あるカラムの値に条件を付けたときの、別のカラムの異なる値の個数; -- 整数の絶対値の確率分布、有符号整数の符号、浮動小数点数の指数と符号; -- 文字列長の確率分布; -- 数値のゼロ値、空文字列と空配列、`NULL` の出現確率; - -- LZ77 やエントロピー系コーデックで圧縮したときのデータ圧縮率; -- テーブル全体にわたる時刻値の連続性(差分の大きさ)、浮動小数点値の連続性; -- `DateTime` の日付成分; - -- 文字列値の UTF-8 妥当性; -- 文字列値が自然な見た目であること。 - -上記の特性の大部分は、性能テストに有用です。 - -カーディナリティ、値の大きさ、圧縮率などが保持されているため、 -データの読み取り、フィルタリング、集約、ソートは元のデータとほぼ同じ速度で動作します。 - -このツールは決定論的に動作します。シード値を指定すると、変換は入力データとシード値によって一意に決まります。 -一対一で逆変換可能な変換もあるため、大きなシード値を使い、それを秘密にしておく必要があります。 - -データ変換にはいくつかの暗号プリミティブを使用していますが、暗号学的な観点から見ると正しいやり方ではありません。そのため、ほかに根拠がない限り、結果を安全なものとは見なさないでください。結果には、公開したくないデータが一部残る可能性があります。 - -このツールは、0、1、-1 の数値、日付、配列の長さ、null フラグを常に元データとまったく同じ状態で残します。 -例えば、テーブルに 0 と 1 の値を持つ `IsMobile` カラムがある場合、変換後のデータでも同じ値のままになります。 - -したがって、ユーザーはモバイルトラフィックの正確な比率を算出できます。 - -もう一つ例を挙げます。テーブル内にユーザーのメールアドレスのような機微なデータがあり、単一のメールアドレスも公開したくないとします。 -テーブルが十分大きく、異なるメールアドレスが多数含まれていて、特定のメールアドレスだけが他より極端に高頻度で出現していないのであれば、すべてのデータは匿名化された状態になります。しかし、あるカラムに含まれる異なる値の種類が少ない場合には、その一部が再現されてしまう可能性があります。 -このツールの動作アルゴリズムを確認し、コマンドラインパラメータを適切にチューニングしてください。 - -このツールがうまく機能するのは、少なくとも中程度以上のデータ量(少なくとも数千行)がある場合のみです。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/index.md deleted file mode 100644 index ed37c1d7e78..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: 'さまざまな有用な ClickHouse ツールおよびユーティリティの一覧ページ。' -keywords: ['tools', 'utilities'] -sidebar_label: 'ツールおよびユーティリティ一覧' -sidebar_position: 56 -slug: /operations/utilities/ -title: 'ツールおよびユーティリティ一覧' -doc_type: 'landing-page' ---- - -| Tool/Utility | Description | -|------|-------------| -|[clickhouse-local](../../operations/utilities/clickhouse-local.md) | ClickHouse サーバーを起動せずにデータに対して SQL クエリを実行できるツールで、`awk` のように動作します。| -|[clickhouse-benchmark](../../operations/utilities/clickhouse-benchmark.md) | カスタムクエリと設定を使用してサーバーに負荷をかけます。| -| [clickhouse-format](../../operations/utilities/clickhouse-format.md) | 入力クエリを整形します。| -|[ClickHouse obfuscator](../../operations/utilities/clickhouse-obfuscator.md) | データを難読化します。| -|[ClickHouse compressor](../../operations/utilities/clickhouse-compressor.md) | データを圧縮および解凍します。| -| [clickhouse-disks](../../operations/utilities/clickhouse-disks.md) | 複数の ClickHouse ディスク間で、ファイルに対するファイルシステムのような操作を提供します。| -| [clickhouse-odbc-bridge](../../operations/utilities/odbc-bridge.md) | ODBC ドライバー用のプロキシサーバーです。| -| [clickhouse_backupview](../../operations/utilities/backupview.md) | ClickHouse のバックアップを解析するための Python モジュールです。| \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md deleted file mode 100644 index 7e9ca039d63..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: 'Odbc Bridge のドキュメント' -slug: /operations/utilities/odbc-bridge -title: 'clickhouse-odbc-bridge' -doc_type: 'reference' ---- - -ODBC ドライバーのプロキシとして動作するシンプルな HTTP サーバーです。主な目的は、 -ODBC 実装におけるセグメンテーションフォルトやその他の不具合によって -clickhouse-server のプロセス全体がクラッシュしてしまう可能性を回避することです。 - -このツールは、パイプ、共有メモリ、TCP ではなく HTTP 経由で動作します。その理由は次のとおりです。 -- 実装がより簡単である -- デバッグがより簡単である -- jdbc-bridge も同様の方法で実装できる - - - -## 使用方法 {#usage} - -`clickhouse-server` は、このツールを ODBC テーブル関数および StorageODBC 内で使用します。 -ただし、POST リクエストの URL に以下のパラメータを指定することで、コマンドラインからスタンドアロンのツールとして使用することもできます: -- `connection_string` -- ODBC 接続文字列。 -- `sample_block` -- カラムの定義を ClickHouse の NamesAndTypesList 形式で指定します。名前はバッククォートで囲み、 - 型は文字列とします。名前と型はスペース区切りで、行は改行区切りです。 -- `max_block_size` -- オプションのパラメータで、1 ブロックあたりの最大サイズを設定します。 -クエリは POST リクエストのボディで送信されます。応答は RowBinary 形式で返されます。 - - - -## 例: {#example} - -```bash -$ clickhouse-odbc-bridge --http-port 9018 --daemon - -$ curl -d "query=SELECT PageID, ImpID, AdType FROM Keys ORDER BY PageID, ImpID" --data-urlencode "connection_string=DSN=ClickHouse;DATABASE=stat" --data-urlencode "sample_block=columns format version: 1 -3 columns: -\`PageID\` String -\`ImpID\` String -\`AdType\` String -" "http://localhost:9018/" > result.txt - -$ cat result.txt -12246623837185725195925621517 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md b/i18n/jp/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md deleted file mode 100644 index e7baf7f68e3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md +++ /dev/null @@ -1,420 +0,0 @@ ---- -description: 'ワークロードスケジューリングに関するドキュメント' -sidebar_label: 'ワークロードスケジューリング' -sidebar_position: 69 -slug: /operations/workload-scheduling -title: 'ワークロードスケジューリング' -doc_type: 'reference' ---- - -ClickHouse が複数のクエリを同時に実行する場合、それらは共有リソース(例: ディスクや CPU コア)を共有して利用することになります。スケジューリングの制約とポリシーを適用することで、リソースの利用方法や、異なるワークロード間での共有方法を制御できます。すべてのリソースに対して共通のスケジューリング階層を構成できます。階層のルートは共有リソースを表し、葉ノードは特定のワークロードを表します。葉ノードには、リソース容量を超過したリクエストが保持されます。 - -:::note -現在、[リモートディスク IO](#disk_config) と [CPU](#cpu_scheduling) は、ここで説明する方法を用いてスケジューリングできます。柔軟なメモリ制限については [Memory overcommit](settings/memory-overcommit.md) を参照してください。 -::: - - - -## ディスク構成 {#disk_config} - -特定のディスクに対して IO ワークロードのスケジューリングを有効にするには、WRITE および READ アクセス用の読み取りリソースと書き込みリソースを作成する必要があります。 - -```sql -CREATE RESOURCE resource_name (WRITE DISK disk_name, READ DISK disk_name) --- または -CREATE RESOURCE read_resource_name (WRITE DISK write_disk_name) -CREATE RESOURCE write_resource_name (READ DISK read_disk_name) -``` - -リソースは、任意の数のディスクに対して、読み取り専用、書き込み専用、または読み書き両用として使用できます。すべてのディスクに対してこのリソースを使用するための構文も用意されています。 - -```sql -CREATE RESOURCE all_io (READ ANY DISK, WRITE ANY DISK); -``` - -リソースがどのディスクを使用するかを指定する別の方法として、サーバーの `storage_configuration` を利用するものがあります。 - -:::warning -ClickHouse の設定によるワークロードのスケジューリングは非推奨です。代わりに SQL 構文を使用してください。 -::: - -特定のディスクに対して I/O スケジューリングを有効にするには、`storage_configuration` 内で `read_resource` および/または `write_resource` を指定する必要があります。これにより、指定したディスクに対する各読み取りおよび書き込みリクエストに対して、どのリソースを使用するかを ClickHouse に指示します。読み取りリソースと書き込みリソースは同じリソース名を参照することができ、これはローカル SSD や HDD に対して有用です。複数の異なるディスクを同じリソースに割り当てることもでき、これはリモートディスクに対して有用です。例えば、「production」ワークロードと「development」ワークロード間でネットワーク帯域幅を公平に分配できるようにしたい場合などです。 - -例: - -```xml - - - ... - - - s3 - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - network_read - network_write - - - - - -
- s3 -
-
-
-
-
-
-``` - -サーバー側の設定オプションのほうが、リソースを SQL で定義する方法よりも優先されることに注意してください。 - - -## ワークロードのマーキング {#workload_markup} - -クエリには、異なるワークロードを区別するために設定 `workload` を指定できます。`workload` が設定されていない場合は、値「default」が使用されます。設定プロファイルを使用して別の値を指定することも可能です。設定の制約機能を利用すると、特定のユーザーからのすべてのクエリが `workload` 設定の固定値でマークされるように、`workload` を一定の値に固定できます。 - -バックグラウンド処理に対しても `workload` 設定を割り当てることができます。マージとミューテーションは、それぞれ `merge_workload` および `mutation_workload` サーバー設定を使用します。これらの値は、`merge_workload` と `mutation_workload` の MergeTree 設定を使用して、特定のテーブルごとに上書きすることもできます。 - -「production」と「development」という 2 つの異なるワークロードを持つシステムの例を考えてみましょう。 - -```sql -SELECT count() FROM my_table WHERE value = 42 SETTINGS workload = 'production' -SELECT count() FROM my_table WHERE value = 13 SETTINGS workload = 'development' -``` - - -## リソーススケジューリング階層 {#hierarchy} - -スケジューリングサブシステムの観点では、リソースはスケジューリングノードの階層として扱われます。 - -```mermaid -graph TD - subgraph network_read - nr_root(("/")) - -->|同時リクエスト100件| nr_fair("fair") - -->|帯域幅75%| nr_prod["prod"] - nr_fair - -->|帯域幅25%| nr_dev["dev"] - end - - subgraph network_write - nw_root(("/")) - -->|同時リクエスト100件| nw_fair("fair") - -->|帯域幅75%| nw_prod["prod"] - nw_fair - -->|帯域幅25%| nw_dev["dev"] - end -``` - -:::warning -ClickHouse の設定を用いたワークロードスケジューリングは非推奨です。代わりに SQL 構文を使用してください。SQL 構文は必要なすべてのスケジューリングノードを自動的に作成するため、以下のスケジューリングノードの説明は、[system.scheduler](/operations/system-tables/scheduler.md) テーブルから参照できる、より低レベルな実装詳細として扱うべきです。 -::: - -**利用可能なノードタイプ:** - -* `inflight_limit` (制約) - 同時に処理中のリクエスト数が `max_requests` を超えるか、またはそれらの合計コストが `max_cost` を超えた場合にブロックします。子ノードは 1 つだけでなければなりません。 -* `bandwidth_limit` (制約) - 現在の帯域幅が `max_speed` を超えた場合 (0 は無制限を意味します)、またはバーストが `max_burst` を超えた場合 (デフォルトは `max_speed` と同じ) にブロックします。子ノードは 1 つだけでなければなりません。 -* `fair` (ポリシー) - max-min フェアネスに従って、子ノードのいずれかから次に処理するリクエストを選択します。子ノードは `weight` を指定できます (デフォルトは 1)。 -* `priority` (ポリシー) - 静的な優先度に従って、子ノードのいずれかから次に処理するリクエストを選択します (値が小さいほど優先度が高くなります)。子ノードは `priority` を指定できます (デフォルトは 0)。 -* `fifo` (キュー) - リソース容量を超えたリクエストを保持できる、階層の葉ノードです。 - -基盤となるリソースの全容量を活用できるようにするには、`inflight_limit` を使用する必要があります。`max_requests` または `max_cost` の値が小さすぎると、リソースが十分に活用されない可能性があります。一方で、値が大きすぎるとスケジューラ内部のキューが空になり、その結果、サブツリー内でポリシーが無視される (不公平なスケジューリングや優先度の無視) ことにつながる可能性があります。逆に、リソースを過度な利用から保護したい場合は、`bandwidth_limit` を使用する必要があります。これは、`duration` 秒間に消費されるリソース量が `max_burst + max_speed * duration` バイトを超えたときにスロットルします。同一リソース上に 2 つの `bandwidth_limit` ノードを配置することで、短い時間間隔におけるピーク帯域幅と、より長い時間における平均帯域幅の双方を制限できます。 - -次の例は、図に示されている IO スケジューリング階層をどのように定義するかを示しています。 - -```xml - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - -``` - - -## ワークロード分類子 {#workload_classifiers} - -:::warning -ClickHouse の設定を用いたワークロードスケジューリングは非推奨です。代わりに SQL 構文を使用してください。SQL 構文を使用する場合、分類子は自動的に作成されます。 -::: - -ワークロード分類子は、クエリで指定された `workload` と、特定のリソースに対して使用されるリーフキューとの対応付けを定義するために使用されます。現時点では、ワークロードの分類は単純で、静的なマッピングのみが利用可能です。 - -例: - -```xml - - - - /fair/prod - /fair/prod - - - /fair/dev - /fair/dev - - - /fair/dev - /fair/dev - - - -``` - - -## ワークロード階層 {#workloads} - -ClickHouse は、スケジューリング階層を定義するための便利な SQL 構文を提供します。`CREATE RESOURCE` で作成されたすべてのリソースは同じ階層構造を共有しますが、一部の点では異なる場合があります。`CREATE WORKLOAD` で作成された各ワークロードは、各リソースごとに自動的に作成される複数のスケジューリングノードを保持します。子ワークロードは、別の親ワークロードの配下に作成できます。以下は、上記の XML 構成とまったく同じ階層を定義する例です。 - -```sql -CREATE RESOURCE network_write (WRITE DISK s3) -CREATE RESOURCE network_read (READ DISK s3) -CREATE WORKLOAD all SETTINGS max_io_requests = 100 -CREATE WORKLOAD development IN all -CREATE WORKLOAD production IN all SETTINGS weight = 3 -``` - -子を持たないリーフワークロードの名前は、クエリ設定 `SETTINGS workload = 'name'` で使用できます。 - -ワークロードをカスタマイズするには、次の設定を使用できます。 - -* `priority` - 兄弟ワークロードは静的な優先度値に従って処理されます(値が小さいほど優先度が高くなります)。 -* `weight` - 同じ静的優先度を持つ兄弟ワークロードは、重みに応じてリソースを共有します。 -* `max_io_requests` - このワークロードで同時に実行できる IO リクエスト数の上限です。 -* `max_bytes_inflight` - このワークロードにおける同時リクエストで送信中(in-flight)の合計バイト数の上限です。 -* `max_bytes_per_second` - このワークロードの読み取りまたは書き込みレート(バイト/秒)の上限です。 -* `max_burst_bytes` - ワークロードがスロットリングされることなく処理できる最大バイト数です(各リソースごとに独立)。 -* `max_concurrent_threads` - このワークロード内のクエリに対して許可されるスレッド数の上限です。 -* `max_concurrent_threads_ratio_to_cores` - `max_concurrent_threads` と同様ですが、利用可能な CPU コア数に対して正規化された値です。 -* `max_cpus` - このワークロードのクエリ処理に使用できる CPU コア数の上限です。 -* `max_cpu_share` - `max_cpus` と同様ですが、利用可能な CPU コア数に対して正規化された値です。 -* `max_burst_cpu_seconds` - `max_cpus` によるスロットリングを受けることなく、このワークロードが消費できる最大 CPU 秒数です。 - -ワークロード設定で指定されたすべての制限は、各リソースごとに独立しています。たとえば、`max_bytes_per_second = 10485760` を持つワークロードは、読み取りおよび書き込みの各リソースごとに独立して 10 MB/s の帯域幅制限を持ちます。読み取りと書き込みに共通の制限が必要な場合は、READ と WRITE アクセスに同じリソースを使用することを検討してください。 - -異なるリソースごとに異なるワークロード階層を指定する方法はありません。ただし、特定のリソースに対して異なるワークロード設定値を指定する方法はあります。 - -```sql -CREATE OR REPLACE WORKLOAD all SETTINGS max_io_requests = 100, max_bytes_per_second = 1000000 FOR network_read, max_bytes_per_second = 2000000 FOR network_write -``` - -また、あるワークロードが別のワークロードから参照されている場合、そのワークロードやリソースは DROP できないことにも留意してください。ワークロードの定義を更新するには、`CREATE OR REPLACE WORKLOAD` クエリを使用します。 - -:::note -Workload の設定は、適切な一連のスケジューリングノードに変換されます。より低レイヤーの詳細については、スケジューリングノードの[種類とオプション](#hierarchy)の説明を参照してください。 -::: - - -## CPU scheduling {#cpu_scheduling} - -ワークロードに対して CPU スケジューリングを有効にするには、CPU リソースを設定し、同時スレッド数の上限を指定します。 - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 -``` - -ClickHouse サーバーが [複数スレッド](/operations/settings/settings.md#max_threads)で多数のクエリを同時に実行しており、利用可能な CPU スロットがすべて使用中になると、オーバーロード状態に達します。オーバーロード状態では、解放された各 CPU スロットはスケジューリングポリシーに従って適切なワークロードに再割り当てされます。同一のワークロードを共有するクエリに対しては、スロットはラウンドロビンで割り当てられます。別々のワークロードに属するクエリに対しては、スロットはワークロードに対して指定された重み、優先度、および制限に従って割り当てられます。 - -CPU 時間は、スレッドがブロックされておらず、CPU 集約的なタスクを実行しているときに消費されます。スケジューリングの目的上、スレッドには 2 種類があります: - -* マスタースレッド — クエリや、マージ/ミューテーションといったバックグラウンド処理で最初に動作を開始するスレッド。 -* ワーカースレッド — マスターが CPU 集約的なタスクを処理するために生成する追加のスレッド。 - -応答性を高めるために、マスターとワーカースレッドに別々のリソースを使用したい場合があります。`max_threads` クエリ設定の値が高い場合、大量のワーカースレッドが CPU リソースを容易に独占してしまう可能性があります。その場合、到着したクエリはマスタースレッドが実行を開始するための CPU スロットを待ってブロックされることになります。これを回避するには、次のような設定を使用します: - -```sql -CREATE RESOURCE worker_cpu (WORKER THREAD) -CREATE RESOURCE master_cpu (MASTER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 FOR worker_cpu, max_concurrent_threads = 1000 FOR master_cpu -``` - -これにより、master スレッドと worker スレッドに対して、それぞれ個別の制限が設定されます。たとえ 100 個すべての worker CPU スロットが使用中でも、master CPU スロットに空きがある限り、新しいクエリはブロックされません。これらのクエリは 1 本のスレッドで実行を開始します。後から worker CPU スロットに空きができれば、そのようなクエリはスケールアップして worker スレッドを生成できます。一方で、このようなアプローチでは、スロットの総数が CPU プロセッサ数に結び付けられていないため、同時実行スレッド数が多すぎるとパフォーマンスに影響します。 - -master スレッドの同時実行数を制限しても、同時実行クエリ数そのものは制限されません。CPU スロットはクエリ実行の途中で解放され、他のスレッドによって再取得される可能性があります。たとえば、master スレッドの同時実行制限が 2 の場合でも、4 つの同時クエリをすべて並列に実行できます。この場合、各クエリは CPU プロセッサの 50% を利用します。同時実行クエリ数を制限するには別のロジックを使用する必要がありますが、ワークロードに対しては現在サポートされていません。 - -個別のスレッド同時実行制限は、ワークロードに対して利用できます。 - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all -CREATE WORKLOAD admin IN all SETTINGS max_concurrent_threads = 10 -CREATE WORKLOAD 本番環境 IN all SETTINGS max_concurrent_threads = 100 -CREATE WORKLOAD analytics IN 本番環境 SETTINGS max_concurrent_threads = 60, weight = 9 -CREATE WORKLOAD ingestion IN 本番環境 -``` - -この構成例では、admin と production 用に独立した CPU スロットプールが用意されています。production プールは analytics とインジェストで共有されます。さらに、production プールが過負荷になった場合、解放されたスロットのうち 10 個中 9 個は、必要に応じて分析クエリに再スケジューリングされます。インジェストクエリは、過負荷期間中は 10 個中 1 個のスロットしか割り当てを受けません。これにより、ユーザー向けクエリのレイテンシが改善される可能性があります。Analytics ワークロードには同時実行スレッド数 60 個という独自の上限があり、インジェストを支えるために少なくとも 40 個のスレッドが常に確保されます。過負荷がない場合、インジェストは 100 個すべてのスレッドを使用できます。 - -クエリを CPU スケジューリングの対象外にするには、クエリ設定 [use_concurrency_control](/operations/settings/settings.md/#use_concurrency_control) を 0 に設定します。 - -CPU スケジューリングは、マージおよびミューテーションにはまだ対応していません。 - -ワークロード間で公平な割り当てを提供するには、クエリ実行中にプリエンプションおよびダウンスケーリングを行う必要があります。プリエンプションはサーバー設定 `cpu_slot_preemption` で有効化できます。有効化されている場合、すべてのスレッドは一定間隔で(サーバー設定 `cpu_slot_quantum_ns` に従って)自分の CPU スロットを更新します。この更新処理は、CPU が過負荷の場合には実行をブロックすることがあります。実行が長時間ブロックされた場合(サーバー設定 `cpu_slot_preemption_timeout_ms` を参照)、クエリはスケールダウンし、同時に実行されるスレッド数が動的に減少します。CPU 時間の公平性はワークロード間では保証されますが、同じワークロード内のクエリ同士については、いくつかのコーナーケースでは満たされない可能性があります。 - -:::warning -スロットスケジューリングは、[クエリの同時実行数](/operations/settings/settings.md#max_threads) を制御する手段を提供しますが、サーバー設定 `cpu_slot_preemption` が `true` に設定されていない限り、公平な CPU 時間割り当ては保証されません。その場合、公平性は競合するワークロード間の CPU スロット割り当て数に基づいてのみ提供されます。これは CPU 秒数が等しくなることを意味しません。プリエンプションがない場合、CPU スロットは無期限に保持される可能性があるためです。スレッドは開始時にスロットを取得し、処理が完了したときにスロットを解放します。 -::: - - -:::note -CPU リソースを宣言すると、[`concurrent_threads_soft_limit_num`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_num) および [`concurrent_threads_soft_limit_ratio_to_cores`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_ratio_to_cores) 設定の効果は無効になります。代わりに、特定のワークロードに割り当てられる CPU 数を制限するために、ワークロード設定 `max_concurrent_threads` が使用されます。従来の動作を再現するには、WORKER THREAD リソースのみを定義し、ワークロード `all` に対して `max_concurrent_threads` を `concurrent_threads_soft_limit_num` と同じ値に設定し、`workload = "all"` クエリ設定を使用してください。この構成は、[`concurrent_threads_scheduler`](server-configuration-parameters/settings.md#concurrent_threads_scheduler) 設定に "fair_round_robin" が指定されている状態に相当します。 -::: - - - -## スレッドと CPU {#threads_vs_cpus} - -ワークロードの CPU 消費を制御する方法は 2 つあります。 - -* スレッド数の制限: `max_concurrent_threads` および `max_concurrent_threads_ratio_to_cores` -* CPU スロットリング: `max_cpus`、`max_cpu_share` および `max_burst_cpu_seconds` - -前者は、現在のサーバー負荷に応じてクエリごとに生成されるスレッド数を動的に制御します。これは実質的に、クエリ設定 `max_threads` が規定する上限を引き下げます。後者は、トークンバケットアルゴリズムを用いてワークロードの CPU 消費をスロットリングします。スレッド数そのものには直接影響しませんが、ワークロード内のすべてのスレッドによる合計 CPU 消費をスロットリングします。 - -`max_cpus` と `max_burst_cpu_seconds` を用いたトークンバケット方式のスロットリングは、次のような意味になります。任意の `delta` 秒の間隔において、ワークロード内のすべてのクエリによる合計 CPU 消費量は `max_cpus * delta + max_burst_cpu_seconds` CPU 秒を超えることはできません。これにより長期的には平均消費量が `max_cpus` によって制限されますが、短期的にはこの制限を超えることがあります。たとえば、`max_burst_cpu_seconds = 60` および `max_cpus=0.001` の場合、スロットリングされることなく 1 スレッドを 60 秒、2 スレッドを 30 秒、あるいは 60 スレッドを 1 秒実行することができます。`max_burst_cpu_seconds` のデフォルト値は 1 秒です。多くのスレッドが同時に実行される場合、この値を小さくしすぎると、許可されている `max_cpus` コアを十分に活用できない可能性があります。 - -:::warning -CPU スロットリング設定は、サーバー設定 `cpu_slot_preemption` が有効な場合にのみ有効であり、それ以外の場合は無視されます。 -::: - -CPU スロットを保持している間、スレッドは次の 3 つの主な状態のいずれかになります。 - -* **Running:** 実際に CPU リソースを消費している状態。この状態で費やされた時間は CPU スロットリングで計測されます。 -* **Ready:** 利用可能な CPU を待機している状態。CPU スロットリングでは計測されません。 -* **Blocked:** IO 操作やその他のブロッキングシステムコール(例: ミューテックス待ち)を実行している状態。CPU スロットリングでは計測されません。 - -CPU スロットリングとスレッド数制限の両方を組み合わせた構成例を考えてみましょう。 - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads_ratio_to_cores = 2 -CREATE WORKLOAD admin IN all SETTINGS max_concurrent_threads = 2, priority = -1 -CREATE WORKLOAD 本番 IN all SETTINGS weight = 4 -CREATE WORKLOAD analytics IN 本番 SETTINGS max_cpu_share = 0.7, weight = 3 -CREATE WORKLOAD ingestion IN 本番 -CREATE WORKLOAD 開発 IN all SETTINGS max_cpu_share = 0.3 -``` - -ここでは、すべてのクエリに対するスレッドの総数を、利用可能な CPU 数の 2 倍に制限しています。管理(admin)ワークロードは、利用可能な CPU 数に関係なく、最大でも 2 スレッドに制限されます。Admin の優先度は -1(デフォルトの 0 より低い)であり、必要な場合にはどの CPU スロットよりも優先して割り当てを受けます。Admin がクエリを実行していないときは、CPU リソースは本番(production)と開発(development)のワークロードの間で分配されます。CPU 時間の保証された持分は重み(4 対 1)に基づいており、少なくとも 80% が本番に(必要な場合)、少なくとも 20% が開発に(必要な場合)割り当てられます。重みが下限(保証)を定める一方で、CPU スロットルが上限(制限)を定めます。本番には上限がなく 100% まで消費できますが、開発には 30% の上限があり、これは他のワークロードからクエリがまったくない場合でも適用されます。本番ワークロードはリーフではないため、そのリソースは分析(analytics)とインジェスト(ingestion)の間で重み(3 対 1)に従って分割されます。つまり、分析には少なくとも 0.8 * 0.75 = 60% の保証があり、`max_cpu_share` に基づいて、合計 CPU リソースの 70% の上限があります。一方で、インジェストには少なくとも 0.8 * 0.25 = 20% の保証が与えられますが、上限はありません。 - -:::note -ClickHouse サーバー上で CPU 利用率を最大化したい場合は、ルートワークロード `all` に対して `max_cpus` と `max_cpu_share` を使用するのは避けてください。その代わり、`max_concurrent_threads` をより高い値に設定します。例えば、CPU が 8 個あるシステムでは、`max_concurrent_threads = 16` と設定します。これにより、8 スレッドが CPU タスクを実行し、残りの 8 スレッドが I/O 処理を担当できます。追加のスレッドによって CPU プレッシャーが生じ、スケジューリングルールが確実に適用されるようになります。対照的に、`max_cpus = 8` を設定すると、サーバーは利用可能な 8 個の CPU を超えて使用できないため、CPU プレッシャーが発生することはありません。 -::: - - -## クエリスロットスケジューリング {#query_scheduling} - -ワークロードに対してクエリスロットスケジューリングを有効化するには、QUERY リソースを作成し、同時実行されるクエリ数または 1 秒あたりのクエリ数の上限を設定します。 - -```sql -CREATE RESOURCE query (QUERY) -CREATE WORKLOAD all SETTINGS max_concurrent_queries = 100, max_queries_per_second = 10, max_burst_queries = 20 -``` - -ワークロード設定 `max_concurrent_queries` は、特定のワークロードに対して同時に実行できるクエリの数を制限します。これは、クエリ設定 [`max_concurrent_queries_for_all_users`](/operations/settings/settings#max_concurrent_queries_for_all_users) およびサーバー設定 [max_concurrent_queries](/operations/server-configuration-parameters/settings#max_concurrent_queries) に相当する設定です。非同期インサートクエリや KILL のような一部の特定のクエリは、この上限には含まれません。 - -ワークロード設定 `max_queries_per_second` と `max_burst_queries` は、トークンバケット方式のスロットラーによって、そのワークロードに対するクエリ数を制限します。これにより、任意の時間区間 `T` において、新規に実行を開始するクエリ数は `max_queries_per_second * T + max_burst_queries` を超えないことが保証されます。 - -ワークロード設定 `max_waiting_queries` は、そのワークロードに対して待機状態にできるクエリの数を制限します。上限に達すると、サーバーはエラー `SERVER_OVERLOADED` を返します。 - -:::note -ブロックされたクエリは、すべての制約が満たされるまで無期限に待機し、その間は `SHOW PROCESSLIST` に表示されません。 -::: - - -## ワークロードおよびリソースのストレージ {#workload_entity_storage} - -`CREATE WORKLOAD` および `CREATE RESOURCE` クエリの形式で定義されたすべてのワークロードとリソースは、ディスク上の `workload_path`、または ZooKeeper 上の `workload_zookeeper_path` のいずれかに永続的に保存されます。ノード間の整合性を確保するには、ZooKeeper でのストレージを推奨します。代わりに、ディスクストレージと組み合わせて `ON CLUSTER` 句を使用することもできます。 - - - -## 設定ベースのワークロードとリソース {#config_based_workloads} - -SQL ベースの定義に加えて、ワークロードとリソースはサーバーの設定ファイルであらかじめ定義しておくこともできます。これは、一部の制限はインフラストラクチャによって決まっている一方で、その他の制限は顧客が変更できるようなクラウド環境で有用です。設定ベースのエンティティは SQL で定義されたものより優先され、SQL コマンドを使用して変更や削除を行うことはできません。 - -### 設定形式 {#config_based_workloads_format} - -```xml - - - RESOURCE s3disk_read (READ DISK s3); - RESOURCE s3disk_write (WRITE DISK s3); - WORKLOAD all SETTINGS max_io_requests = 500 FOR s3disk_read, max_io_requests = 1000 FOR s3disk_write, max_bytes_per_second = 1342177280 FOR s3disk_read, max_bytes_per_second = 3355443200 FOR s3disk_write; - WORKLOAD production IN all SETTINGS weight = 3; - - -``` - -この設定は、`CREATE WORKLOAD` および `CREATE RESOURCE` ステートメントと同じ SQL 構文を使用します。すべてのクエリは有効でなければなりません。 - -### 利用に関する推奨事項 {#config_based_workloads_usage_recommendations} - -クラウド環境では、一般的な構成例として次のようなものが考えられます。 - -1. ルートワークロードおよびネットワーク IO リソースを設定で定義し、インフラストラクチャの上限を設定する -2. これらの制限を強制するために `throw_on_unknown_workload` を有効にする -3. すべてのクエリに対して制限を自動的に適用するため、`CREATE WORKLOAD default IN all` を作成する(`workload` クエリ設定のデフォルト値が 'default' であるため) -4. ユーザーが、定義済みの階層構造内で追加のワークロードを作成できるようにする - -これにより、すべてのバックグラウンド処理およびクエリがインフラストラクチャ上の制約を遵守しつつ、ユーザー固有のスケジューリングポリシーに対する柔軟性も維持できます。 - -別のユースケースとしては、異種混在クラスタ内でノードごとに異なる設定を行うことが挙げられます。 - - -## 厳格なリソースアクセス {#strict_resource_access} - -すべてのクエリにリソーススケジューリングポリシーの適用を強制するために、サーバー設定 `throw_on_unknown_workload` を使用できます。これが `true` に設定されている場合、すべてのクエリは有効な `workload` クエリ設定を使用する必要があり、そうでない場合は `RESOURCE_ACCESS_DENIED` 例外がスローされます。`false` に設定されている場合、そのようなクエリはリソーススケジューラを使用せず、つまり任意の `RESOURCE` に対して無制限にアクセスできます。クエリ設定 `use_concurrency_control = 0` により、クエリは CPU スケジューラをバイパスし、CPU への無制限アクセスを得ることができます。CPU スケジューリングを強制するには、`use_concurrency_control` を読み取り専用の定数値として固定する設定制約を作成します。 - -:::note -`CREATE WORKLOAD default` を実行していない限り、`throw_on_unknown_workload` を `true` に設定しないでください。起動時に `workload` を明示的に設定していないクエリが実行されると、サーバーの起動に問題が発生する可能性があります。 -::: - - - -## 関連項目 {#see-also} -- [system.scheduler](/operations/system-tables/scheduler.md) -- [system.workloads](/operations/system-tables/workloads.md) -- [system.resources](/operations/system-tables/resources.md) -- [merge_workload](/operations/settings/merge-tree-settings.md#merge_workload) MergeTree 設定 -- [merge_workload](/operations/server-configuration-parameters/settings.md#merge_workload) グローバルサーバー設定 -- [mutation_workload](/operations/settings/merge-tree-settings.md#mutation_workload) MergeTree 設定 -- [mutation_workload](/operations/server-configuration-parameters/settings.md#mutation_workload) グローバルサーバー設定 -- [workload_path](/operations/server-configuration-parameters/settings.md#workload_path) グローバルサーバー設定 -- [workload_zookeeper_path](/operations/server-configuration-parameters/settings.md#workload_zookeeper_path) グローバルサーバー設定 -- [cpu_slot_preemption](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption) グローバルサーバー設定 -- [cpu_slot_quantum_ns](/operations/server-configuration-parameters/settings.md#cpu_slot_quantum_ns) グローバルサーバー設定 -- [cpu_slot_preemption_timeout_ms](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption_timeout_ms) グローバルサーバー設定 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md deleted file mode 100644 index 224e1b0071f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md +++ /dev/null @@ -1,362 +0,0 @@ ---- -description: '集約関数コンビネータに関するリファレンス' -sidebar_label: 'コンビネータ' -sidebar_position: 37 -slug: /sql-reference/aggregate-functions/combinators -title: '集約関数コンビネータ' -doc_type: 'reference' ---- - - - -# 集約関数のコンビネーター {#aggregate-function-combinators} - -集約関数の名前には、接尾辞を付けることができます。これにより、その集約関数の挙動が変化します。 - - - -## -If {#-if} - -サフィックス -If は、任意の集約関数名に付与できます。この場合、集約関数は追加の引数として条件(`UInt8` 型)を受け取ります。集約関数は、その条件が真となる行だけを処理します。条件が一度も真とならなかった場合、デフォルト値(通常はゼロまたは空文字列)を返します。 - -例: `sumIf(column, cond)`, `countIf(cond)`, `avgIf(x, cond)`, `quantilesTimingIf(level1, level2)(x, cond)`, `argMinIf(arg, val, cond)` など。 - -条件付き集約関数を使用すると、サブクエリや `JOIN` を使わずに、複数の条件に対する集約値を同時に計算できます。たとえば、条件付き集約関数を用いてセグメント比較機能を実装できます。 - - - -## -Array {#-array} - --Array サフィックスは、任意の集約関数に付加できます。この場合、集約関数は引数として型 'T' ではなく、型 'Array(T)'(配列)を取ります。集約関数が複数の引数を受け取る場合、これらの引数はすべて長さが等しい配列でなければなりません。配列を処理する際、集約関数は、すべての配列要素に対して元の集約関数と同様に動作します。 - -例 1: `sumArray(arr)` - すべての 'arr' 配列に含まれるすべての要素を合計します。この例では、より単純に `sum(arraySum(arr))` と書くこともできます。 - -例 2: `uniqArray(arr)` – すべての 'arr' 配列に含まれる一意な要素の数を数えます。これは、より簡単な方法として `uniq(arrayJoin(arr))` でも実行できますが、常にクエリに 'arrayJoin' を追加できるとは限りません。 - --If と -Array は組み合わせて使用できます。ただし、'Array' を先に、次に 'If' を付ける必要があります。例: `uniqArrayIf(arr, cond)`, `quantilesTimingArrayIf(level1, level2)(arr, cond)`。この順序により、'cond' 引数は配列型の引数にはなりません。 - - - -## -Map {#-map} - -`-Map` サフィックスは、任意の集約関数に付加して使用できます。これにより、引数として `Map` 型を受け取り、指定した集約関数を用いてマップ内の各キーに対応する値を個別に集約する集約関数が作成されます。結果も `Map` 型となります。 - -**例** - -```sql -CREATE TABLE map_map( - date Date, - timeslot DateTime, - status Map(String, UInt64) -) ENGINE = Log; - -INSERT INTO map_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', (['a', 'b', 'c'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', (['c', 'd', 'e'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['d', 'e', 'f'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['f', 'g', 'g'], [10, 10, 10])); - -SELECT - timeslot, - sumMap(status), - avgMap(status), - minMap(status) -FROM map_map -GROUP BY timeslot; - -┌────────────timeslot─┬─sumMap(status)───────────────────────┬─avgMap(status)───────────────────────┬─minMap(status)───────────────────────┐ -│ 2000-01-01 00:00:00 │ {'a':10,'b':10,'c':20,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ -│ 2000-01-01 00:01:00 │ {'d':10,'e':10,'f':20,'g':20} │ {'d':10,'e':10,'f':10,'g':10} │ {'d':10,'e':10,'f':10,'g':10} │ -└─────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┘ -``` - - -## -SimpleState {#-simplestate} - -このコンビネータを適用すると、集約関数は同じ値を返しますが、型が異なるようになります。これは、テーブルに保存して [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) テーブルで使用できる [SimpleAggregateFunction(...)](../../sql-reference/data-types/simpleaggregatefunction.md) 型です。 - -**構文** - -```sql -SimpleState(x) -``` - -**引数** - -* `x` — 集約関数のパラメータ。 - -**戻り値** - -`SimpleAggregateFunction(...)` 型の集約関数の値。 - -**例** - -クエリ: - -```sql -WITH anySimpleState(number) AS c SELECT toTypeName(c), c FROM numbers(1); -``` - -結果: - -```text -┌─toTypeName(c)────────────────────────┬─c─┐ -│ SimpleAggregateFunction(any, UInt64) │ 0 │ -└──────────────────────────────────────┴───┘ -``` - - -## -State {#-state} - -このコンビネータを適用すると、集約関数は結果の値([uniq](/sql-reference/aggregate-functions/reference/uniq) 関数における一意な値の個数など)ではなく、集約の中間状態(`uniq` では、一意な値の数を計算するためのハッシュテーブル)を返します。これは `AggregateFunction(...)` 型であり、さらなる処理に利用したり、テーブルに保存して後から集約処理を完了させたりできます。 - -:::note -同一のデータに対しても、-MapState は中間状態におけるデータの順序が変化しうるため不変ではないことに注意してください。ただし、これはこのデータのインジェストには影響しません。 -::: - -これらの状態を扱うには、次を使用します。 - -- [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) テーブルエンジン -- [finalizeAggregation](/sql-reference/functions/other-functions#finalizeAggregation) 関数 -- [runningAccumulate](../../sql-reference/functions/other-functions.md#runningAccumulate) 関数 -- [-Merge](#-merge) コンビネータ -- [-MergeState](#-mergestate) コンビネータ - - - -## -Merge {#-merge} - -このコンビネータを適用すると、集約関数は引数として中間集約状態を受け取り、それらを結合して集約を完了し、その結果の値を返します。 - - - -## -MergeState {#-mergestate} - -`-Merge` コンビネータと同様に中間集約状態をマージします。ただし、結果の値は返さず、`-State` コンビネータと同様に中間集約状態を返します。 - - - -## -ForEach {#-foreach} - -テーブルに対する集約関数を、対応する配列要素ごとに集約を行い、その結果を配列で返す配列向けの集約関数に変換します。たとえば、配列 `[1, 2]`、`[3, 4, 5]`、`[6, 7]` に対する `sumForEach` は、対応する配列要素を加算した結果として `[10, 13, 5]` を返します。 - - - -## -Distinct {#-distinct} - -引数の一意な組み合わせごとに、集約は 1 回だけ行われます。重複する値は無視されます。 -例: `sum(DISTINCT x)`(または `sumDistinct(x)`)、`groupArray(DISTINCT x)`(または `groupArrayDistinct(x)`)、`corrStable(DISTINCT x, y)`(または `corrStableDistinct(x, y)`)など。 - - - -## -OrDefault {#-ordefault} - -集約関数の動作を変更します。 - -集約関数に入力値がまったくない場合、このコンビネータを使用すると、その戻り値のデータ型に対するデフォルト値を返します。空の入力データを取り得る集約関数に適用できます。 - -`-OrDefault` は他のコンビネータと併用できます。 - -**構文** - -```sql -OrDefault(x) -``` - -**引数** - -* `x` — 集約関数のパラメータ。 - -**戻り値** - -集約する対象が何もない場合、集約関数の戻り値の型に対するデフォルト値を返します。 - -型は使用する集約関数に依存します。 - -**例** - -クエリ: - -```sql -SELECT avg(number), avgOrDefault(number) FROM numbers(0) -``` - -結果: - -```text -┌─avg(number)─┬─avgOrDefault(number)─┐ -│ nan │ 0 │ -└─────────────┴──────────────────────┘ -``` - -また `-OrDefault` は他のコンビネータと組み合わせて使用できます。これは、集約関数が空の入力を受け付けない場合に有用です。 - -クエリ: - -```sql -SELECT avgOrDefaultIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -結果: - -```text -┌─avgOrDefaultIf(x, greater(x, 10))─┐ -│ 0.00 │ -└───────────────────────────────────┘ -``` - - -## -OrNull {#-ornull} - -集約関数の動作を変更します。 - -このコンビネータは、集約関数の結果を [Nullable](../../sql-reference/data-types/nullable.md) データ型に変換します。集約関数に集計対象の値が存在しない場合は、[NULL](/operations/settings/formats#input_format_null_as_default) を返します。 - -`-OrNull` は他のコンビネータと組み合わせて使用できます。 - -**構文** - -```sql -OrNull(x) -``` - -**引数** - -* `x` — 集約関数のパラメーター。 - -**戻り値** - -* 集約関数の結果を `Nullable` データ型に変換した値。 -* 集約対象の値が存在しない場合は `NULL`。 - -型: `Nullable(集約関数の戻り値の型)`。 - -**例** - -集約関数名の末尾に `-orNull` を追加します。 - -クエリ: - -```sql -SELECT sumOrNull(number), toTypeName(sumOrNull(number)) FROM numbers(10) WHERE number > 10 -``` - -結果: - -```text -┌─sumOrNull(number)─┬─toTypeName(sumOrNull(number))─┐ -│ ᴺᵁᴸᴸ │ Nullable(UInt64) │ -└───────────────────┴───────────────────────────────┘ -``` - -`-OrNull` は他のコンビネータと組み合わせて使用することもできます。これは、集約関数が空の入力を許容しない場合に有用です。 - -クエリ: - -```sql -SELECT avgOrNullIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -結果: - -```text -┌─avgOrNullIf(x, greater(x, 10))─┐ -│ ᴺᵁᴸᴸ │ -└────────────────────────────────┘ -``` - - -## -Resample {#-resample} - -データをグループに分割し、その各グループ内で個別にデータを集計できるようにします。グループは、1 列の値を区間ごとに分割することで作成されます。 - -```sql -Resample(start, end, step)(, resampling_key) -``` - -**引数** - -* `start` — `resampling_key` の値に対する対象となる全区間の開始値。 -* `stop` — `resampling_key` の値に対する対象となる全区間の終了値。この全区間には `stop` の値は含まれず、`[start, stop)` となります。 -* `step` — 全区間をサブ区間に分割する際のステップ幅。`aggFunction` はそれぞれのサブ区間ごとに独立して実行されます。 -* `resampling_key` — データを区間に分割するために使用される値を持つカラム。 -* `aggFunction_params` — `aggFunction` のパラメータ。 - -**戻り値** - -* 各サブ区間に対する `aggFunction` の結果の配列。 - -**例** - -次のデータを持つ `people` テーブルを想定します。 - -```text -┌─名前───┬─年齢─┬─賃金─┐ -│ John │ 16 │ 10 │ -│ Alice │ 30 │ 15 │ -│ Mary │ 35 │ 8 │ -│ Evelyn │ 48 │ 11.5 │ -│ David │ 62 │ 9.9 │ -│ Brian │ 60 │ 16 │ -└────────┴─────┴──────┘ -``` - -`[30,60)` および `[60,75)` の区間に年齢が含まれる人の名前を取得しましょう。年齢は整数で表現しているため、実際には `[30, 59]` および `[60,74]` の区間の年齢が対象になります。 - -名前を配列に集約するには、集約関数 [groupArray](/sql-reference/aggregate-functions/reference/grouparray) を使用します。この関数は 1 つの引数を取ります。ここでは `name` 列を渡します。`groupArrayResample` 関数では、`age` 列を使用して年齢ごとに名前を集約します。必要な区間を定義するために、`groupArrayResample` 関数には引数として `30, 75, 30` を渡します。 - -```sql -SELECT groupArrayResample(30, 75, 30)(name, age) FROM people -``` - -```text -┌─groupArrayResample(30, 75, 30)(name, age)─────┐ -│ [['Alice','Mary','Evelyn'],['David','Brian']] │ -└───────────────────────────────────────────────┘ -``` - -結果を確認します。 - -`John` は年齢が若すぎるため、サンプルには含まれていません。他の人たちは、指定した年齢区間に従って分布しています。 - -次に、指定した年齢区間ごとに、総人数と平均賃金を求めましょう。 - -```sql -SELECT - countResample(30, 75, 30)(name, age) AS amount, - avgResample(30, 75, 30)(wage, age) AS avg_wage -FROM people -``` - -```text -┌─amount─┬─avg_wage──────────────────┐ -│ [3,2] │ [11.5,12.949999809265137] │ -└────────┴───────────────────────────┘ -``` - - -## -ArgMin {#-argmin} - -接尾辞 -ArgMin は、任意の集約関数の名前に付加できます。この場合、その集約関数は追加の引数を 1 つ受け取り、この引数には任意の比較可能な式を指定できます。集約関数は、指定された追加の式が最小値となる行だけを処理します。 - -例: `sumArgMin(column, expr)`, `countArgMin(expr)`, `avgArgMin(x, expr)` など。 - - - -## -ArgMax {#-argmax} - -サフィックス -ArgMin と同様ですが、指定された追加の式に対して最大値を持つ行だけを処理します。 - - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Using Aggregate Combinators in ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md deleted file mode 100644 index 8e965757e03..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md +++ /dev/null @@ -1,369 +0,0 @@ ---- -description: 'GROUPING 集約関数に関するドキュメント。' -slug: /sql-reference/aggregate-functions/grouping_function -title: 'GROUPING' -doc_type: 'リファレンス' ---- - - - -# グループ化 {#grouping} - - - -## GROUPING {#grouping} - -[ROLLUP](../statements/select/group-by.md/#rollup-modifier) と [CUBE](../statements/select/group-by.md/#cube-modifier) は GROUP BY の修飾子です。これらはいずれも小計を計算します。ROLLUP は `(day, month, year)` のような順序付きのカラムリストを受け取り、集約の各レベルで小計を計算し、最後に総計を計算します。CUBE は、指定されたカラムのあらゆる組み合わせに対して小計を計算します。GROUPING は、ROLLUP または CUBE によって返される行のうち、どれがスーパー集約(より上位レベルの集約行)であり、どれが修飾されていない GROUP BY によって返される行に相当するかを識別します。 - -GROUPING 関数は複数のカラムを引数として取り、ビットマスクを返します。 -- `1` は、`GROUP BY` に対する `ROLLUP` または `CUBE` 修飾子によって返された行が小計であることを示します -- `0` は、`GROUP BY` に対する `ROLLUP` または `CUBE` 修飾子によって返された行が小計ではないことを示します - - - -## GROUPING SETS {#grouping-sets} - -デフォルトでは、`CUBE` 修飾子は、`CUBE` に渡された列のあらゆる組み合わせに対して小計を計算します。`GROUPING SETS` を使用すると、計算する組み合わせを明示的に指定できます。 - -階層データの分析は、`ROLLUP`、`CUBE`、`GROUPING SETS` 修飾子の代表的なユースケースの 1 つです。ここでのサンプルは、2 つのデータセンターにインストールされている Linux ディストリビューションとそのバージョンに関するデータを含むテーブルです。ディストリビューション別、バージョン別、ロケーション別にデータを確認することが有用な場合があります。 - -### サンプルデータの読み込み {#load-sample-data} - -```sql -CREATE TABLE servers ( datacenter VARCHAR(255), - distro VARCHAR(255) NOT NULL, - version VARCHAR(50) NOT NULL, - quantity INT - ) - ORDER BY (datacenter, distro, version) -``` - -```sql -INSERT INTO servers(datacenter, distro, version, quantity) -VALUES ('Schenectady', 'Arch','2022.08.05',50), - ('Westport', 'Arch','2022.08.05',40), - ('Schenectady','Arch','2021.09.01',30), - ('Westport', 'Arch','2021.09.01',20), - ('Schenectady','Arch','2020.05.01',10), - ('Westport', 'Arch','2020.05.01',5), - ('Schenectady','RHEL','9',60), - ('Westport','RHEL','9',70), - ('Westport','RHEL','7',80), - ('Schenectady','RHEL','7',80) -``` - -```sql -SELECT - * -FROM - servers; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─quantity─┐ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ 9 │ 70 │ -└─────────────┴────────┴────────────┴──────────┘ - -10行のセット。経過時間: 0.409秒。 -``` - -### 簡単なクエリ {#simple-queries} - -分布別に各データセンター内のサーバー数を取得します。 - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ - -4行のセット。経過時間: 0.212秒。 -``` - -```sql -SELECT - datacenter, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter; -``` - -```response -┌─datacenter──┬─qty─┐ -│ Westport │ 215 │ -│ Schenectady │ 230 │ -└─────────────┴─────┘ - -2行のデータセット。経過時間: 0.277秒 -``` - -```sql -SELECT - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro; -``` - -```response - -┌─distro─┬─qty─┐ -│ Arch │ 155 │ -│ RHEL │ 290 │ -└────────┴─────┘ - -2行が設定されています。経過時間: 0.352秒。 -``` - - -```sql -SELECT - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─qty─┐ -│ 445 │ -└─────┘ - -1 行。経過時間: 0.244 秒 -``` - -### 複数の GROUP BY と GROUPING SETS の比較 {#comparing-multiple-group-by-statements-with-grouping-sets} - -CUBE、ROLLUP、GROUPING SETS を使わずにデータを集計する場合: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro -UNION ALL -SELECT - datacenter, - null, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter -UNION ALL -SELECT - null, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro -UNION ALL -SELECT - null, - null, - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ ᴺᵁᴸᴸ │ 215 │ -│ Schenectady │ ᴺᵁᴸᴸ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ Arch │ 155 │ -│ ᴺᵁᴸᴸ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -9行のセット。経過時間: 0.527秒。 -``` - -GROUPING SETS を使って同じ情報を取得する場合: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - GROUPING SETS( - (datacenter,distro), - (datacenter), - (distro), - () - ) -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ │ 215 │ -│ Schenectady │ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ Arch │ 155 │ -│ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -9 rows in set. Elapsed: 0.427 sec. -``` - -### GROUPING SETS との比較 {#comparing-cube-with-grouping-sets} - -次のクエリにおける CUBE `CUBE(datacenter,distro,version)` は、意味のある階層にはなりません。Arch と RHEL ではリリースサイクルやバージョン命名規則が異なるため、2 つのディストリビューションをまたいでバージョンを見ることには意味がありません。この後に続く GROUPING SETS の例のほうが適切であり、`distro` と `version` を同じセット内でグループ化しています。 - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM - servers -GROUP BY - CUBE(datacenter,distro,version) -ORDER BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ │ │ 7 │ 160 │ -│ │ │ 2020.05.01 │ 15 │ -│ │ │ 2021.09.01 │ 50 │ -│ │ │ 2022.08.05 │ 90 │ -│ │ │ 9 │ 130 │ -│ │ │ │ 445 │ -│ │ Arch │ 2021.09.01 │ 50 │ -│ │ Arch │ 2022.08.05 │ 90 │ -│ │ Arch │ 2020.05.01 │ 15 │ -│ │ Arch │ │ 155 │ -│ │ RHEL │ 9 │ 130 │ -│ │ RHEL │ 7 │ 160 │ -│ │ RHEL │ │ 290 │ -│ Schenectady │ │ 9 │ 60 │ -│ Schenectady │ │ 2021.09.01 │ 30 │ -│ Schenectady │ │ 7 │ 80 │ -│ Schenectady │ │ 2022.08.05 │ 50 │ -│ Schenectady │ │ 2020.05.01 │ 10 │ -│ Schenectady │ │ │ 230 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ │ 90 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ │ 9 │ 70 │ -│ Westport │ │ 2020.05.01 │ 5 │ -│ Westport │ │ 2022.08.05 │ 40 │ -│ Westport │ │ 7 │ 80 │ -│ Westport │ │ 2021.09.01 │ 20 │ -│ Westport │ │ │ 215 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ Arch │ │ 65 │ -│ Westport │ RHEL │ 9 │ 70 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴────────────┴───────────────┘ - -39 rows in set. Elapsed: 0.355 sec. -``` - -:::note -上記の例におけるバージョンは、ディストリビューションと結び付いていない場合には、あまり意味をなさないかもしれません。カーネルバージョンを追跡しているのであれば、カーネルバージョンはどちらのディストリビューションにも結び付けられるため、意味を持つと言えるでしょう。次の例で示すように `GROUPING SETS` を使用する方が、より適切な選択肢となる場合があります。 -::: - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM servers -GROUP BY - GROUPING SETS ( - (datacenter, distro, version), - (datacenter, distro)) -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ Westport │ RHEL │ 9 │ 70 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -└─────────────┴────────┴────────────┴───────────────┘ -┌─datacenter──┬─distro─┬─version─┬─sum(quantity)─┐ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ Arch │ │ 65 │ -│ Schenectady │ Arch │ │ 90 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴─────────┴───────────────┘ - -14 rows in set. Elapsed: 1.036 sec. -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md deleted file mode 100644 index 62bdb1b0162..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -description: '集約関数に関するリファレンス' -sidebar_label: '集約関数' -sidebar_position: 33 -slug: /sql-reference/aggregate-functions/ -title: '集約関数' -doc_type: 'reference' ---- - -# 集約関数 {#aggregate-functions} - -集約関数は、データベースの専門家にとって[一般的な](http://www.sql-tutorial.com/sql-aggregate-functions-sql-tutorial)方法で動作します。 - -ClickHouse は次の機能もサポートしています: - -* 列に加えて他のパラメータも受け取る [パラメトリック集約関数](/sql-reference/aggregate-functions/parametric-functions) -* 集約関数の動作を変更する [コンビネータ](/sql-reference/aggregate-functions/combinators) - -## NULL の処理 {#null-processing} - -集約処理の際、すべての `NULL` 引数はスキップされます。集約に複数の引数がある場合、それらのうち 1 つでも NULL が含まれる行は無視されます。 - -このルールには例外があり、`RESPECT NULLS` 修飾子を伴う関数 [`first_value`](../../sql-reference/aggregate-functions/reference/first_value.md)、[`last_value`](../../sql-reference/aggregate-functions/reference/last_value.md) と、それぞれのエイリアス(`any` と `anyLast`)が該当します。たとえば、`FIRST_VALUE(b) RESPECT NULLS` のように指定します。 - -**例:** - -次のテーブルを考えてみます: - -```text -┌─x─┬────y─┐ -│ 1 │ 2 │ -│ 2 │ ᴺᵁᴸᴸ │ -│ 3 │ 2 │ -│ 3 │ 3 │ -│ 3 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -たとえば、`y` 列の値を合計する必要があるとしましょう。 - -```sql -SELECT sum(y) FROM t_null_big -``` - -```text -┌─sum(y)─┐ -│ 7 │ -└────────┘ -``` - -ここで `groupArray` 関数を使用して、`y` 列から配列を作成します。 - -```sql -SELECT groupArray(y) FROM t_null_big -``` - -```text -┌─groupArray(y)─┐ -│ [2,2,3] │ -└───────────────┘ -``` - -`groupArray` は、結果の配列に `NULL` を含めません。 - -[COALESCE](../../sql-reference/functions/functions-for-nulls.md#coalesce) を使用して、`NULL` をユースケースに応じて意味のある値に変換できます。たとえば、`avg(COALESCE(column, 0))` は、集約時に列の値を使用し、`NULL` の場合は 0 を使用します。 - -```sql -SELECT - avg(y), - avg(coalesce(y, 0)) -FROM t_null_big -``` - -```text -┌─────────────avg(y)─┬─avg(coalesce(y, 0))─┐ -│ 2.3333333333333335 │ 1.4 │ -└────────────────────┴─────────────────────┘ -``` - -また、NULL がスキップされる挙動を回避するために [Tuple](sql-reference/data-types/tuple.md) を使用することもできます。`NULL` 値だけを含む `Tuple` は `NULL` ではないため、集約関数はその `NULL` 値を理由にその行をスキップしません。 - -```sql -SELECT - groupArray(y), - groupArray(tuple(y)).1 -FROM t_null_big; - -┌─groupArray(y)─┬─tupleElement(groupArray(tuple(y)), 1)─┐ -│ [2,2,3] │ [2,NULL,2,3,NULL] │ -└───────────────┴───────────────────────────────────────┘ -``` - -列が集約関数の引数として使用される場合、その列に対する集約はスキップされることに注意してください。たとえば、引数なしの [`count`](../../sql-reference/aggregate-functions/reference/count.md)(`count()`)や定数引数付きのもの(`count(1)`)は、(それが引数ではないため GROUP BY 列の値に依存せずに)ブロック内のすべての行をカウントしますが、`count(column)` は `column` が NULL でない行のみの数を返します。 - -```sql -SELECT - v, - count(1), - count(v) -FROM -( - SELECT if(number < 10, NULL, number % 3) AS v - FROM numbers(15) -) -GROUP BY v - -┌────v─┬─count()─┬─count(v)─┐ -│ ᴺᵁᴸᴸ │ 10 │ 0 │ -│ 0 │ 1 │ 1 │ -│ 1 │ 2 │ 2 │ -│ 2 │ 2 │ 2 │ -└──────┴─────────┴──────────┘ -``` - -以下は、`RESPECT NULLS` を使用した first_value の例です。NULL の入力値が尊重され、読み取られた最初の値が NULL かどうかに関係なく返されることがわかります。 - -```sql -SELECT - col || '_' || ((col + 1) * 5 - 1) AS range, - first_value(odd_or_null) AS first, - first_value(odd_or_null) IGNORE NULLS as first_ignore_null, - first_value(odd_or_null) RESPECT NULLS as first_respect_nulls -FROM -( - SELECT - intDiv(number, 5) AS col, - if(number % 2 == 0, NULL, number) AS odd_or_null - FROM numbers(15) -) -GROUP BY col -ORDER BY col - -┌─range─┬─first─┬─first_ignore_null─┬─first_respect_nulls─┐ -│ 0_4 │ 1 │ 1 │ ᴺᵁᴸᴸ │ -│ 1_9 │ 5 │ 5 │ 5 │ -│ 2_14 │ 11 │ 11 │ ᴺᵁᴸᴸ │ -└───────┴───────┴───────────────────┴─────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md deleted file mode 100644 index 69fc147f45d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md +++ /dev/null @@ -1,941 +0,0 @@ ---- -description: 'パラメトリック集約関数のドキュメント' -sidebar_label: 'パラメトリック' -sidebar_position: 38 -slug: /sql-reference/aggregate-functions/parametric-functions -title: 'パラメトリック集約関数' -doc_type: 'reference' ---- - -# パラメトリック集約関数 {#parametric-aggregate-functions} - -一部の集約関数は、(圧縮に使用される)引数列だけでなく、初期化に用いる定数パラメータの集合も受け取ることができます。構文としては、1 組ではなく 2 組の括弧を使用します。最初の括弧はパラメータ用で、2 番目の括弧は引数用です。 - -## histogram {#histogram} - -適応型ヒストグラムを計算します。厳密な結果が得られることは保証されません。 - -```sql -histogram(ビンの数)(値) -``` - -この関数は [A Streaming Parallel Decision Tree Algorithm](http://jmlr.org/papers/volume11/ben-haim10a/ben-haim10a.pdf) に基づいています。ヒストグラムのビンの境界は、新しいデータが関数に入力されるたびに調整されます。一般的には、ビンの幅は等しくありません。 - -**引数** - -`values` — 入力値を生成する[式](/sql-reference/syntax#expressions)。 - -**パラメータ** - -`number_of_bins` — ヒストグラム内のビン数の上限。関数はビン数を自動的に計算します。指定されたビン数に到達するよう試行しますが、失敗した場合はより少ないビン数を使用します。 - -**戻り値** - -* 次の形式の [Tuple](../../sql-reference/data-types/tuple.md) の [Array](../../sql-reference/data-types/array.md): - - ``` - [(lower_1, upper_1, height_1), ... (lower_N, upper_N, height_N)] - ``` - - * `lower` — ビンの下限。 - * `upper` — ビンの上限。 - * `height` — ビンの高さ(計算結果)。 - -**例** - -```sql -SELECT histogram(5)(number + 1) -FROM ( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─histogram(5)(plus(number, 1))───────────────────────────────────────────┐ -│ [(1,4.5,4),(4.5,8.5,4),(8.5,12.75,4.125),(12.75,17,4.625),(17,20,3.25)] │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -例えば、[bar](/sql-reference/functions/other-functions#bar) 関数を使ってヒストグラムを可視化できます。 - -```sql -WITH histogram(5)(rand() % 100) AS hist -SELECT - arrayJoin(hist).3 AS height, - bar(height, 0, 6, 5) AS bar -FROM -( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─height─┬─bar───┐ -│ 2.125 │ █▋ │ -│ 3.25 │ ██▌ │ -│ 5.625 │ ████▏ │ -│ 5.625 │ ████▏ │ -│ 3.375 │ ██▌ │ -└────────┴───────┘ -``` - -この場合、ヒストグラムのビン境界は分かっていないことを念頭に置いてください。 - -## sequenceMatch {#sequencematch} - -シーケンスにパターンに一致するイベントチェーンが含まれているかどうかを判定します。 - -**構文** - -```sql -sequenceMatch(pattern)(timestamp, cond1, cond2, ...) -``` - -:::note -同じ秒に発生したイベントは、シーケンス内での並び順が未定義となる場合があり、結果に影響する可能性があります。 -::: - -**引数** - -* `timestamp` — 時刻データを含むとみなされるカラム。代表的なデータ型は `Date` および `DateTime` です。サポートされている任意の [UInt](../../sql-reference/data-types/int-uint.md) 型も使用できます。 - -* `cond1`, `cond2` — イベントのシーケンスを表す条件。データ型: `UInt8`。条件引数は最大 32 個まで指定できます。関数は、これらの条件で記述されたイベントのみを考慮します。シーケンスに条件で記述されていないデータが含まれている場合、関数はそれらをスキップします。 - -**パラメータ** - -* `pattern` — パターン文字列。[パターン構文](#pattern-syntax) を参照してください。 - -**返される値** - -* パターンに一致した場合は 1。 -* パターンに一致しない場合は 0。 - -型: `UInt8`。 - -#### パターン構文 {#pattern-syntax} - -* `(?N)` — 位置 `N` の条件引数に一致します。条件は `[1, 32]` の範囲で番号付けされます。たとえば、`(?1)` は `cond1` パラメータに渡された引数に一致します。 - -* `.*` — 任意個数のイベントに一致します。このパターン要素に一致させるために条件引数を使用する必要はありません。 - -* `(?t operator value)` — 2 つのイベントを隔てる時間(秒数)を指定します。たとえば、パターン `(?1)(?t>1800)(?2)` は、互いに 1800 秒より長い間隔で発生したイベントに一致します。これらのイベントの間には、任意の数の任意のイベントが存在し得ます。`>=`、`>`、`<`、`<=`、`==` 演算子を使用できます。 - -**例** - -`t` テーブル内の次のデータを考えます。 - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -└──────┴────────┘ -``` - -次のクエリを実行してください: - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 1 │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -この関数は、数値1の後に数値2が続くイベントチェーンを見つけました。その間にある3は、イベントとして記述されていないためスキップされました。もし例で示したイベントチェーンを検索する際にこの数値も考慮したい場合は、この数値に対する条件を追加する必要があります。 - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 3) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 3))─┐ -│ 0 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -この場合、この関数はパターンに一致するイベントチェーンを見つけられませんでした。番号 3 のイベントが 1 と 2 の間に発生していたためです。同じ状況で番号 4 について条件を確認した場合は、そのシーケンスはパターンに一致します。 - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ 1 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [sequenceCount](#sequencecount) - -## sequenceCount {#sequencecount} - -パターンに一致したイベントチェーンの個数をカウントします。関数は、互いに重複しないイベントチェーンを検索します。現在のチェーンが一致した後に、次のチェーンの検索を開始します。 - -:::note -同じ秒に発生したイベントは、シーケンス内で順序が未定義となる場合があり、結果に影響を与える可能性があります。 -::: - -**構文** - -```sql -sequenceCount(pattern)(timestamp, cond1, cond2, ...) -``` - -**引数** - -* `timestamp` — 時刻データを含むと見なされる列。一般的なデータ型は `Date` と `DateTime` です。サポートされているいずれかの [UInt](../../sql-reference/data-types/int-uint.md) 型も使用できます。 - -* `cond1`, `cond2` — 一連のイベントを表す条件。データ型: `UInt8`。条件引数は最大 32 個まで指定できます。関数は、これらの条件で表現されるイベントのみを対象とします。シーケンス内に条件で表現されていないデータが含まれている場合、そのデータは関数によってスキップされます。 - -**パラメータ** - -* `pattern` — パターン文字列。[パターン構文](#pattern-syntax) を参照してください。 - -**戻り値** - -* 一致した、互いに重ならないイベントチェーンの数。 - -型: `UInt64`。 - -**例** - -`t` テーブル内のデータを考えます。 - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -「1」の後に、その間にいくつでも他の数値が入ってよいものとして、「2」が現れる回数を数えます。 - -```sql -SELECT sequenceCount('(?1).*(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceCount('(?1).*(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 2 │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -## sequenceMatchEvents {#sequencematchevents} - -パターンに一致した最長のイベントチェーン内のイベントタイムスタンプを返します。 - -:::note -同じ秒に発生したイベントは、シーケンス内で順序が未定義となる場合があり、結果に影響する可能性があります。 -::: - -**構文** - -```sql -sequenceMatchEvents(pattern)(timestamp, cond1, cond2, ...) -``` - -**引数** - -* `timestamp` — 時刻データを含むと見なされるカラム。代表的なデータ型は `Date` および `DateTime` です。サポートされている任意の [UInt](../../sql-reference/data-types/int-uint.md) 型も使用できます。 - -* `cond1`, `cond2` — イベントチェーンを表す条件。データ型: `UInt8`。条件引数は最大 32 個まで渡すことができます。関数は、これらの条件で記述されたイベントのみを考慮します。シーケンスに条件で記述されていないデータが含まれている場合、関数はそれらをスキップします。 - -**パラメータ** - -* `pattern` — パターン文字列。[パターン構文](#pattern-syntax) を参照。 - -**返される値** - -* イベントチェーンから、一致した条件引数 (?N) のタイムスタンプの配列。配列内の位置は、パターン内の条件引数の位置と対応します。 - -型: Array。 - -**例** - -次のような `t` テーブル内のデータを考えます。 - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -最長チェーンのイベントのタイムスタンプを返す - -```sql -SELECT sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ [1,3,4] │ -└────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [sequenceMatch](#sequencematch) - -## windowFunnel {#windowfunnel} - -スライディング時間ウィンドウ内でイベントチェーンを探索し、そのチェーンから発生したイベント数の最大値を計算します。 - -この関数は次のアルゴリズムに従って動作します。 - -* まず、チェーン内の最初の条件を満たすデータを検索し、イベントカウンターを 1 に設定します。この時点がスライディングウィンドウの開始時刻になります。 - -* ウィンドウ内でチェーンに属するイベントが順番どおりに発生した場合、カウンターをインクリメントします。イベントの順序が崩れた場合、カウンターはインクリメントされません。 - -* データに複数のイベントチェーンがあり、それぞれ進行度合いが異なる場合、関数は最も長いチェーンの長さのみを出力します。 - -**構文** - -```sql -windowFunnel(window, [mode, [mode, ... ]])(timestamp, cond1, cond2, ..., condN) -``` - -**引数** - -* `timestamp` — タイムスタンプを含む列の名前。サポートされるデータ型: [Date](../../sql-reference/data-types/date.md)、[DateTime](/sql-reference/data-types/datetime) およびその他の符号なし整数型(`timestamp` 列は `UInt64` 型をサポートしますが、その値は Int64 の最大値である 2^63 - 1 を超えることはできません)。 -* `cond` — 事象の連鎖を表す条件またはデータ。[UInt8](../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `window` — スライディングウィンドウの長さ。最初と最後の条件の間の時間間隔です。`window` の単位は `timestamp` 自体に依存し、状況によって異なります。`timestamp of cond1 <= timestamp of cond2 <= ... <= timestamp of condN <= timestamp of cond1 + window` という式により決定されます。 -* `mode` — 省略可能な引数です。1 つ以上のモードを指定できます。 - * `'strict_deduplication'` — 同じ条件がイベントのシーケンスに対して成り立つ場合、そのような繰り返しイベントは以降の処理を打ち切ります。注: 同じイベントに対して複数の条件が成り立つ場合、想定と異なる動作になる可能性があります。 - * `'strict_order'` — 他のイベントの介在を許可しません。例えば `A->B->D->C` の場合、`A->B->C` の検出は `D` の時点で停止し、最大イベントレベルは 2 になります。 - * `'strict_increase'` — タイムスタンプが厳密に増加しているイベントにのみ条件を適用します。 - * `'strict_once'` — 条件を複数回満たす場合でも、チェーン内で各イベントは 1 回だけカウントします。 - -**戻り値** - -スライディング時間ウィンドウ内で、チェーン内で連続して満たされた条件の最大数。 -選択されたすべてのチェーンが解析されます。 - -型: `Integer`. - -**例** - -オンラインストアにおいて、ユーザーが電話を選択して 2 回購入するのに、ある一定時間が十分かどうかを判定します。 - -次のイベントチェーンを設定します: - -1. ユーザーがストアのアカウントにログインした(`eventID = 1003`)。 -2. ユーザーが電話を検索した(`eventID = 1007, product = 'phone'`)。 -3. ユーザーが注文を行った(`eventID = 1009`)。 -4. ユーザーがその注文を再度行った(`eventID = 1010`)。 - -入力テーブル: - -```text -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-28 │ 1 │ 2019-01-29 10:00:00 │ 1003 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-31 │ 1 │ 2019-01-31 09:00:00 │ 1007 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-30 │ 1 │ 2019-01-30 08:00:00 │ 1009 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-02-01 │ 1 │ 2019-02-01 08:00:00 │ 1010 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -``` - -2019年1月から2月の期間に、ユーザー `user_id` がチェーンをどこまで進んだかを確認します。 - -クエリ: - -```sql -SELECT - level, - count() AS c -FROM -( - SELECT - user_id, - windowFunnel(6048000000000000)(timestamp, eventID = 1003, eventID = 1009, eventID = 1007, eventID = 1010) AS level - FROM trend - WHERE (event_date >= '2019-01-01') AND (event_date <= '2019-02-02') - GROUP BY user_id -) -GROUP BY level -ORDER BY level ASC; -``` - -結果: - -```text -┌─level─┬─c─┐ -│ 4 │ 1 │ -└───────┴───┘ -``` - -## retention {#retention} - -この関数は、イベントで特定の条件が満たされたかどうかを示す `UInt8` 型の引数を 1〜32 個受け取ります。 -どの条件も([WHERE](/sql-reference/statements/select/where) と同様に)引数として指定できます。 - -最初の条件を除き、それ以外の条件はペアで評価されます。2 番目の戻り値は 1 番目と 2 番目の条件がともに真のときに真となり、3 番目の戻り値は 1 番目と 3 番目の条件がともに真のときに真となる、という具合です。 - -**構文** - -```sql -retention(cond1, cond2, ..., cond32); -``` - -**引数** - -* `cond` — `UInt8` の結果(1 または 0)を返す式。 - -**戻り値** - -1 または 0 の配列。 - -* 1 — イベントで条件が満たされた。 -* 0 — イベントで条件が満たされなかった。 - -型: `UInt8`。 - -**例** - -サイトトラフィックを把握するために `retention` 関数を計算する例を考えます。 - -**1.** 例を示すためのテーブルを作成します。 - -```sql -CREATE TABLE retention_test(date Date, uid Int32) ENGINE = Memory; - -INSERT INTO retention_test SELECT '2020-01-01', number FROM numbers(5); -INSERT INTO retention_test SELECT '2020-01-02', number FROM numbers(10); -INSERT INTO retention_test SELECT '2020-01-03', number FROM numbers(15); -``` - -入力テーブル: - -クエリ: - -```sql -SELECT * FROM retention_test -``` - -結果: - -```text -┌───────date─┬─uid─┐ -│ 2020-01-01 │ 0 │ -│ 2020-01-01 │ 1 │ -│ 2020-01-01 │ 2 │ -│ 2020-01-01 │ 3 │ -│ 2020-01-01 │ 4 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-02 │ 0 │ -│ 2020-01-02 │ 1 │ -│ 2020-01-02 │ 2 │ -│ 2020-01-02 │ 3 │ -│ 2020-01-02 │ 4 │ -│ 2020-01-02 │ 5 │ -│ 2020-01-02 │ 6 │ -│ 2020-01-02 │ 7 │ -│ 2020-01-02 │ 8 │ -│ 2020-01-02 │ 9 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-03 │ 0 │ -│ 2020-01-03 │ 1 │ -│ 2020-01-03 │ 2 │ -│ 2020-01-03 │ 3 │ -│ 2020-01-03 │ 4 │ -│ 2020-01-03 │ 5 │ -│ 2020-01-03 │ 6 │ -│ 2020-01-03 │ 7 │ -│ 2020-01-03 │ 8 │ -│ 2020-01-03 │ 9 │ -│ 2020-01-03 │ 10 │ -│ 2020-01-03 │ 11 │ -│ 2020-01-03 │ 12 │ -│ 2020-01-03 │ 13 │ -│ 2020-01-03 │ 14 │ -└────────────┴─────┘ -``` - -**2.** `retention` 関数を使用して、ユーザーを一意の ID `uid` ごとにグループ化します。 - -クエリ: - -```sql -SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r -FROM retention_test -WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') -GROUP BY uid -ORDER BY uid ASC -``` - -結果: - -```text -┌─uid─┬─r───────┐ -│ 0 │ [1,1,1] │ -│ 1 │ [1,1,1] │ -│ 2 │ [1,1,1] │ -│ 3 │ [1,1,1] │ -│ 4 │ [1,1,1] │ -│ 5 │ [0,0,0] │ -│ 6 │ [0,0,0] │ -│ 7 │ [0,0,0] │ -│ 8 │ [0,0,0] │ -│ 9 │ [0,0,0] │ -│ 10 │ [0,0,0] │ -│ 11 │ [0,0,0] │ -│ 12 │ [0,0,0] │ -│ 13 │ [0,0,0] │ -│ 14 │ [0,0,0] │ -└─────┴─────────┘ -``` - -**3.** 1 日あたりのサイト訪問数の合計を計算します。 - -クエリ: - -```sql -SELECT - sum(r[1]) AS r1, - sum(r[2]) AS r2, - sum(r[3]) AS r3 -FROM -( - SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r - FROM retention_test - WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') - GROUP BY uid -) -``` - -結果: - -```text -┌─r1─┬─r2─┬─r3─┐ -│ 5 │ 5 │ 5 │ -└────┴────┴────┘ -``` - -ここで: - -* `r1` - 2020-01-01 の一日を通してサイトを訪問したユニーク訪問者数(`cond1` 条件)。 -* `r2` - 2020-01-01 から 2020-01-02 の間の特定の期間にサイトを訪問したユニーク訪問者数(`cond1` および `cond2` 条件)。 -* `r3` - 2020-01-01 および 2020-01-03 の特定の期間にサイトを訪問したユニーク訪問者数(`cond1` および `cond3` 条件)。 - -## uniqUpTo(N)(x) {#uniquptonx} - -引数の異なる値の個数を、指定された上限 `N` まで数えます。異なる値の個数が `N` より大きい場合、この関数は `N` + 1 を返し、それ以外の場合は正確な値を返します。 - -小さい `N`(最大 10 程度)での利用を推奨します。`N` の最大値は 100 です。 - -集約関数の状態に対して、この関数は `1 + N * (1 値あたりのサイズ(バイト))` に等しい量のメモリを使用します。 -文字列を扱う場合、この関数は 8 バイトの非暗号学的ハッシュを保存します。文字列に対する計算は近似となります。 - -例えば、ウェブサイトでユーザーが行ったすべての検索クエリを記録するテーブルがあるとします。テーブルの各行は 1 件の検索クエリを表し、ユーザー ID、検索クエリ、クエリのタイムスタンプの列を持ちます。`uniqUpTo` を使用すると、少なくとも 5 人の一意のユーザーによって検索されたキーワードのみを表示するレポートを生成できます。 - -```sql -SELECT SearchPhrase -FROM SearchLog -GROUP BY SearchPhrase -HAVING uniqUpTo(4)(UserID) >= 5 -``` - -`uniqUpTo(4)(UserID)` は、各 `SearchPhrase` ごとの一意な `UserID` の数を計算しますが、数えるのは一意な値を最大 4 個までに制限します。ある `SearchPhrase` に対して一意な `UserID` が 4 個を超えて存在する場合、この関数は 5(4 + 1)を返します。その後、`HAVING` 句で、一意な `UserID` の数が 5 未満である `SearchPhrase` を除外します。これにより、少なくとも 5 人の異なるユーザーによって使用された検索キーワードの一覧を取得できます。 - -## sumMapFiltered {#summapfiltered} - -この関数は [sumMap](/sql-reference/aggregate-functions/reference/summap) と同様に動作しますが、追加でフィルタリングに使用するキーの配列をパラメータとして受け取ります。これはキーのカーディナリティが高いキー集合を扱う場合に特に有用です。 - -**構文** - -`sumMapFiltered(keys_to_keep)(keys, values)` - -**パラメータ** - -* `keys_to_keep`: フィルタリングに使用するキーの [Array](../data-types/array.md)。 -* `keys`: キーの [Array](../data-types/array.md)。 -* `values`: 値の [Array](../data-types/array.md)。 - -**戻り値** - -* 2 つの配列からなるタプルを返します。キーをソートした配列と、それぞれのキーに対応する値の合計からなる配列です。 - -**例** - -クエリ: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt16, requests UInt64) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) FROM sum_map; -``` - -結果: - -```response - ┌─sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests)─┐ -1. │ ([1,4,8],[10,20,10]) │ - └─────────────────────────────────────────────────────────────────┘ -``` - -## sumMapFilteredWithOverflow {#summapfilteredwithoverflow} - -この関数は [sumMap](/sql-reference/aggregate-functions/reference/summap) と同様に動作しますが、パラメータとしてフィルタリングに使用するキー配列も受け取る点が異なります。これは、キーのカーディナリティが高い場合に特に有用です。また、[sumMapFiltered](#summapfiltered) 関数とは、オーバーフローを許容して合計を行う点が異なります。つまり、合計結果のデータ型が引数のデータ型と同じになります。 - -**構文** - -`sumMapFilteredWithOverflow(keys_to_keep)(keys, values)` - -**パラメータ** - -* `keys_to_keep`: フィルタリングに使用するキーの [Array](../data-types/array.md)。 -* `keys`: キーの [Array](../data-types/array.md)。 -* `values`: 値の [Array](../data-types/array.md)。 - -**戻り値** - -* 2 つの配列からなるタプルを返します。ソート済みのキー配列と、それぞれのキーに対応して合計された値の配列です。 - -**例** - -この例では、まずテーブル `sum_map` を作成し、いくつかのデータを挿入した後、`sumMapFilteredWithOverflow` と `sumMapFiltered` の両方および `toTypeName` 関数を使用し、結果を比較します。作成したテーブルでは `requests` は型 `UInt8` ですが、`sumMapFiltered` はオーバーフローを避けるために合計後の値の型を `UInt64` に昇格させている一方、`sumMapFilteredWithOverflow` は型を `UInt8` のまま保持しており、結果を格納するには十分な大きさではありません。このため、オーバーフローが発生します。 - -クエリ: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt8, requests UInt8) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFilteredWithOverflow([1, 4, 8])(statusMap.status, statusMap.requests) as summap_overflow, toTypeName(summap_overflow) FROM sum_map; -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) as summap, toTypeName(summap) FROM sum_map; -``` - -結果: - -```response - ┌─sum──────────────────┬─toTypeName(sum)───────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt8)) │ - └──────────────────────┴───────────────────────────────────┘ -``` - -```response - ┌─summap───────────────┬─toTypeName(summap)─────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt64)) │ - └──────────────────────┴────────────────────────────────────┘ -``` - -## sequenceNextNode {#sequencenextnode} - -イベントチェーンにマッチした次のイベントの値を返します。 - -*試験的な関数です。有効化するには `SET allow_experimental_funnel_functions = 1` を実行します。* - -**構文** - -```sql -sequenceNextNode(direction, base)(timestamp, event_column, base_condition, event1, event2, event3, ...) -``` - -**パラメーター** - -* `direction` — 方向を指定して移動に使用します。 - * forward — 前方に移動します。 - * backward — 後方に移動します。 - -* `base` — 基準点を設定するために使用します。 - * head — 基準点を最初のイベントに設定します。 - * tail — 基準点を最後のイベントに設定します。 - * first_match — 基準点を最初に一致した `event1` に設定します。 - * last_match — 基準点を最後に一致した `event1` に設定します。 - -**引数** - -* `timestamp` — タイムスタンプを含む列名。サポートされるデータ型: [Date](../../sql-reference/data-types/date.md)、[DateTime](/sql-reference/data-types/datetime) およびその他の符号なし整数型。 -* `event_column` — 次に返すイベントの値を含む列名。サポートされるデータ型: [String](../../sql-reference/data-types/string.md) および [Nullable(String)](../../sql-reference/data-types/nullable.md)。 -* `base_condition` — 基準点が満たすべき条件。 -* `event1`, `event2`, ... — イベントの連鎖を表す条件。[UInt8](../../sql-reference/data-types/int-uint.md)。 - -**戻り値** - -* `event_column[next_index]` — パターンが一致し、かつ次の値が存在する場合。 -* `NULL` - パターンが一致しない、または次の値が存在しない場合。 - -型: [Nullable(String)](../../sql-reference/data-types/nullable.md)。 - -**例** - -イベントが A->B->C->D->E のように並んでおり、B->C に続くイベント D を知りたい場合に使用できます。 - -A->B に続くイベントを検索するクエリ: - -```sql -CREATE TABLE test_flow ( - dt DateTime, - id int, - page String) -ENGINE = MergeTree() -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow VALUES (1, 1, 'A') (2, 1, 'B') (3, 1, 'C') (4, 1, 'D') (5, 1, 'E'); - -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'A', page = 'A', page = 'B') as next_flow FROM test_flow GROUP BY id; -``` - -結果: - -```text -┌─id─┬─next_flow─┐ -│ 1 │ C │ -└────┴───────────┘ -``` - -**`forward` と `head` の挙動** - -```sql -ALTER TABLE test_flow DELETE WHERE 1 = 1 settings mutations_sync = 1; - -INSERT INTO test_flow VALUES (1, 1, 'Home') (2, 1, 'Gift') (3, 1, 'Exit'); -INSERT INTO test_flow VALUES (1, 2, 'Home') (2, 2, 'Home') (3, 2, 'Gift') (4, 2, 'Basket'); -INSERT INTO test_flow VALUES (1, 3, 'Gift') (2, 3, 'Home') (3, 3, 'Gift') (4, 3, 'Basket'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'Home', page = 'Home', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page - 1970-01-01 09:00:01 1 Home // 基準点、Homeに一致 - 1970-01-01 09:00:02 1 Gift // Giftに一致 - 1970-01-01 09:00:03 1 Exit // 結果 - - 1970-01-01 09:00:01 2 Home // 基準点、Homeに一致 - 1970-01-01 09:00:02 2 Home // Giftに不一致 - 1970-01-01 09:00:03 2 Gift - 1970-01-01 09:00:04 2 Basket -``` - -1970-01-01 09:00:01 3 Gift // 基準値、Home とは一致しない -1970-01-01 09:00:02 3 Home -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket - -```` - -**`backward` と `tail` の動作** - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, page = 'Basket', page = 'Basket', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift -1970-01-01 09:00:03 1 Exit // 基準点、Basketに不一致 - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // 結果 -1970-01-01 09:00:03 2 Gift // Giftに一致 -1970-01-01 09:00:04 2 Basket // 基準点、Basketに一致 - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // 結果 -1970-01-01 09:00:03 3 Gift // 基準点、Giftに一致 -1970-01-01 09:00:04 3 Basket // 基準点、Basketに一致 -```` - -**`forward` および `first_match` の挙動** - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // 基準点 -1970-01-01 09:00:03 1 Exit // 結果 - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // 基準点 -1970-01-01 09:00:04 2 Basket // 結果 - -1970-01-01 09:00:01 3 Gift // 基準点 -1970-01-01 09:00:02 3 Home // 結果 -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // 基準点 -1970-01-01 09:00:03 1 Exit // Home と一致しない - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // 基準点 -1970-01-01 09:00:04 2 Basket // Home と一致しない - -1970-01-01 09:00:01 3 Gift // 基準点 -1970-01-01 09:00:02 3 Home // Home と一致 -1970-01-01 09:00:03 3 Gift // 結果 -1970-01-01 09:00:04 3 Basket -``` - -**`backward` と `last_match` の動作** - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; -``` - -dt id page -1970-01-01 09:00:01 1 Home // 結果 -1970-01-01 09:00:02 1 Gift // 起点 -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // 結果 -1970-01-01 09:00:03 2 Gift // 起点 -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // 結果 -1970-01-01 09:00:03 3 Gift // 起点 -1970-01-01 09:00:04 3 Basket - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home // Homeに一致、結果はnull -1970-01-01 09:00:02 1 Gift // 基準点 -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home // 結果 -1970-01-01 09:00:02 2 Home // Homeに一致 -1970-01-01 09:00:03 2 Gift // 基準点 -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift // 結果 -1970-01-01 09:00:02 3 Home // Homeに一致 -1970-01-01 09:00:03 3 Gift // 基準点 -1970-01-01 09:00:04 3 Basket -```` - -**`base_condition` の挙動** - -```sql -CREATE TABLE test_flow_basecond -( - `dt` DateTime, - `id` int, - `page` String, - `ref` String -) -ENGINE = MergeTree -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow_basecond VALUES (1, 1, 'A', 'ref4') (2, 1, 'A', 'ref3') (3, 1, 'B', 'ref2') (4, 1, 'B', 'ref1'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, ref = 'ref1', page = 'A') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 // 先頭行のref列が'ref1'と一致しないため、先頭を基準点として使用できません。 - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 -``` - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, ref = 'ref4', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 // 末尾の ref 列が 'ref4' と一致しないため、この末尾は基準点になりません。 -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, ref = 'ref3', page = 'A') FROM test_flow_basecond GROUP BY id; -``` - -dt id page ref -1970-01-01 09:00:01 1 A ref4 // ref 列が 'ref3' と一致しないため、この行は基準行にはできません。 -1970-01-01 09:00:02 1 A ref3 // 基準行 -1970-01-01 09:00:03 1 B ref2 // 結果行 -1970-01-01 09:00:04 1 B ref1 - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, ref = 'ref2', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 // 結果 - 1970-01-01 09:00:03 1 B ref2 // 基準点 - 1970-01-01 09:00:04 1 B ref1 // この行は ref カラムが 'ref2' と一致しないため、基準点になりません。 -```` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md deleted file mode 100644 index 1fc46ff0bc0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'この関数は、例外安全性をテストするために使用できます。 - 指定した確率で生成時に例外をスローします。' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/aggthrow -title: 'aggThrow' -doc_type: 'reference' ---- - -# aggThrow {#aggthrow} - -この関数は、例外安全性をテストするために使用できます。指定した確率で、生成時に例外をスローします。 - -**構文** - -```sql -aggThrow(throw_prob) -``` - -**引数** - -* `throw_prob` — 作成時に例外をスローする確率。[Float64](../../data-types/float.md)。 - -**戻り値** - -* 例外: `Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully`。 - -**例** - -クエリ: - -```sql -SELECT number % 2 AS even, aggThrow(number) FROM numbers(10) GROUP BY even; -``` - -結果: - -```response -例外を受信しました: -コード: 503. DB::Exception: 集約関数 aggThrow が正常に例外をスローしました: AggregatingTransform の実行中。(AGGREGATE_FUNCTION_THROW) -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md deleted file mode 100644 index f7129f46d56..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '一元配置分散分析(ANOVA 検定)のための統計的手法を提供します。正規分布に従う複数の観測値グループについて、すべてのグループの平均値が等しいかどうかを検定します。' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/analysis_of_variance -title: 'analysisOfVariance' -doc_type: 'reference' ---- - -# analysisOfVariance {#analysisofvariance} - -一元分散分析(ANOVA 検定)を行うための統計的検定です。正規分布に従う複数のグループの観測値に対して、すべてのグループの平均が同一かどうかを判定します。 - -**構文** - -```sql -analysisOfVariance(val, group_no) -``` - -エイリアス: `anova` - -**パラメータ** - -* `val`: 値。 -* `group_no` : `val` が属するグループ番号。 - -:::note -グループは 0 から番号付けされ、検定を実行するには少なくとも 2 つのグループが必要です。 -観測値の数が 1 を超えるグループが少なくとも 1 つ必要です。 -::: - -**戻り値** - -* `(f_statistic, p_value)`。[Tuple](../../data-types/tuple.md)([Float64](../../data-types/float.md), [Float64](../../data-types/float.md))。 - -**例** - -クエリ: - -```sql -SELECT analysisOfVariance(number, number % 2) FROM numbers(1048575); -``` - -結果: - -```response -┌─analysisOfVariance(number, modulo(number, 2))─┐ -│ (0,1) │ -└───────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md deleted file mode 100644 index 29c54468316..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: '列で最初に出現した値を選択します。' -sidebar_position: 102 -slug: /sql-reference/aggregate-functions/reference/any -title: 'any' -doc_type: 'reference' ---- - -# any {#any} - -列内で最初に見つかった値を選択します。 - -:::warning -クエリは任意の順序で実行される可能性があるため、この関数が返す結果は非決定的です。 -どれでもよいが決定的な結果が必要な場合は、[`min`](../reference/min.md) または [`max`](../reference/max.md) 関数を使用してください。 -::: - -デフォルトでは、この関数が NULL を返すことはなく、入力列内の NULL 値は無視されます。 -ただし、`RESPECT NULLS` 修飾子とともにこの関数を使用した場合は、NULL かどうかに関係なく、読み取った最初の値を返します。 - -**構文** - -```sql -any(column) [RESPECT NULLS] -``` - -エイリアス `any(column)`(`RESPECT NULLS` なし) - -* `any_value` -* [`first_value`](../reference/first_value.md) - -`any(column) RESPECT NULLS` のエイリアス - -* `anyRespectNulls`, `any_respect_nulls` -* `firstValueRespectNulls`, `first_value_respect_nulls` -* `anyValueRespectNulls`, `any_value_respect_nulls` - -**パラメータ** - -* `column`: カラム名。 - -**戻り値** - -最初に出現した値。 - -:::note -この関数の戻り値の型は、LowCardinality が取り除かれる点を除き、入力と同じです。 -つまり、入力として 1 行もない場合、その型のデフォルト値(整数なら 0、Nullable() カラムなら Null)が返されます。 -この挙動を変更するには、`-OrNull` [combinator](../../../sql-reference/aggregate-functions/combinators.md) を使用できます。 -::: - -**実装の詳細** - -場合によっては、実行順序に依存することができます。 -これは、`ORDER BY` を使用するサブクエリから `SELECT` されるケースに当てはまります。 - -`SELECT` クエリに `GROUP BY` 句、または少なくとも 1 つの集計関数が含まれている場合、ClickHouse(MySQL とは対照的に)は、`SELECT`、`HAVING`、`ORDER BY` 各句内のすべての式がキーまたは集計関数から計算されることを要求します。 -言い換えると、テーブルから選択される各カラムは、キーとして、または集計関数の内部のいずれかで使用されている必要があります。 -MySQL と同様の動作を得るには、その他のカラムを `any` 集計関数に渡すことができます。 - -**例** - -クエリ: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES (NULL), ('Amsterdam'), ('New York'), ('Tokyo'), ('Valencia'), (NULL); - -SELECT any(city), anyRespectNulls(city) FROM tab; -``` - -```response -┌─any(city)─┬─anyRespectNulls(city)─┐ -│ Amsterdam │ ᴺᵁᴸᴸ │ -└───────────┴───────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md deleted file mode 100644 index 1de5d7f955b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'heavy hitters アルゴリズムを使用して、頻繁に出現する値を選択します。 - 各クエリ実行スレッドで、ケースの半数を超える頻度で出現する値がある場合は、その値が返されます。 - 通常、この結果は非決定的です。' -sidebar_position: 104 -slug: /sql-reference/aggregate-functions/reference/anyheavy -title: 'anyHeavy' -doc_type: 'reference' ---- - -# anyHeavy {#anyheavy} - -[heavy hitters](https://doi.org/10.1145/762471.762473) アルゴリズムを使用して、頻繁に出現する値を 1 つ選択します。クエリの各実行スレッドにおいて、全ケースの半数を超える頻度で出現する値がある場合、その値が返されます。通常、この結果は非決定的です。 - -```sql -anyHeavy(column) -``` - -**引数** - -* `column` – 列名です。 - -**例** - -[OnTime](../../../getting-started/example-datasets/ontime.md) データセットを使用し、`AirlineID` 列で頻繁に現れる値のいずれかを選択します。 - -```sql -SELECT anyHeavy(AirlineID) AS res -FROM ontime -``` - -```text -┌───res─┐ -│ 19690 │ -└───────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md deleted file mode 100644 index e0aa1d22756..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '列で最後に現れた値を選択します。' -sidebar_position: 105 -slug: /sql-reference/aggregate-functions/reference/anylast -title: 'anyLast' -doc_type: 'reference' ---- - -# anyLast {#anylast} - -列で最後に出現した値を選択します。 - -:::warning -クエリは任意の順序で実行される可能性があるため、この関数の結果は決定的ではありません。 -任意の値でよいが決定的な結果が必要な場合は、[`min`](../reference/min.md) または [`max`](../reference/max.md) 関数を使用してください。 -::: - -デフォルトでは、この関数は NULL を返しません。つまり、入力列中の NULL 値は無視されます。 -ただし、`RESPECT NULLS` 修飾子とともに使用された場合、NULL かどうかに関わらず、読み取った最初の値を返します。 - -**構文** - -```sql -anyLast(カラム) [RESPECT NULLS] -``` - -別名 `anyLast(column)`(`RESPECT NULLS` なし) - -* [`last_value`](../reference/last_value.md) - -`anyLast(column) RESPECT NULLS` の別名 - -* `anyLastRespectNulls`, `anyLast_respect_nulls` -* `lastValueRespectNulls`, `last_value_respect_nulls` - -**パラメータ** - -* `column`: 列名。 - -**返り値** - -* 最後に出現した値。 - -**例** - -クエリ: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES ('Amsterdam'),(NULL),('New York'),('Tokyo'),('Valencia'),(NULL); - -SELECT anyLast(city), anyLastRespectNulls(city) FROM tab; -``` - -```response -┌─anyLast(city)─┬─anyLastRespectNulls(city)─┐ -│ Valencia │ ᴺᵁᴸᴸ │ -└───────────────┴───────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md deleted file mode 100644 index b7345f9f227..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -description: '`val` の最大値に対応する `arg` と `val` の値を計算します。最大値となる `val` を持つ行が複数ある場合、どの行に対応する `arg` と `val` が返されるかは決定的ではありません。' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmax -title: 'argAndMax' -doc_type: 'reference' ---- - -# argAndMax {#argandmax} - -最大の`val`値に対応する`arg`と`val`の値を計算します。最大値となる`val`が等しい行が複数存在する場合、どの`arg`と`val`の組み合わせが返されるかは非決定的です。 -`arg`と`max`の両方の部分は[集約関数](/sql-reference/aggregate-functions/index.md)として動作し、処理中に[`Null`をスキップ](/sql-reference/aggregate-functions/index.md#null-processing)し、`Null`でない値が利用可能な場合は`Null`でない値を返します。 - -:::note -`argMax`との唯一の違いは、`argAndMax`が引数と値の両方を返す点です。 -::: - -**構文** - -```sql -argAndMax(arg, val) -``` - -**引数** - -* `arg` — 引数。 -* `val` — 値 - -**戻り値** - -* 最大の `val` に対応する `arg` の値。 -* `val` の最大値 - -型: `arg`、`val` の各型に対応するタプル。 - -**例** - -入力テーブル: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -クエリ: - -```sql -SELECT argAndMax(user, salary) FROM salary; -``` - -結果: - -```text -┌─argAndMax(user, salary)─┐ -│ ('director',5000) │ -└─────────────────────────┘ -``` - -**拡張例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), argAndMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─argAndMax(a, b)─┬─max(b)─┐ -│ b │ ('b',2) │ 3 │ -- argMax = b は最初の非NULL値であるため、max(b)は別の行から取得されます! -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMax(tuple(a), b) FROM test; -┌─argAndMax((a), b)─┐ -│ ((NULL),3) │-- `NULL`値のみを含む`Tuple`は`NULL`ではないため、集約関数はその`NULL`値によってその行をスキップしません -└───────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA──┬─argMaxB─┐ -│ (NULL,3) │ 3 │ -- Tupleを使用して、対応するmax(b)の両方の列(すべて - tuple(*))を取得できます -└──────────┴─────────┘ - -SELECT argAndMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argAndMax(a, b)─┬─max(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │-- フィルタにより集約されたすべての行に少なくとも1つの`NULL`値が含まれるため、すべての行がスキップされ、結果は`NULL`になります -└─────────────────┴────────┘ - -SELECT argAndMax(a, (b,a)) FROM test; -┌─argAndMax(a, (b, a))─┐ -│ ('c',(2,'c')) │ -- b=2の行が2つあり、`Max`内の`Tuple`を使用することで最初の`arg`以外を取得できます -└──────────────────────┘ - -SELECT argAndMax(a, tuple(b)) FROM test; -┌─argAndMax(a, (b))─┐ -│ ('b',(2)) │ -- `Max`内で`Tuple`を使用することで、`Max`でNULLをスキップしないようにできます -└───────────────────┘ -``` - -**関連項目** - -* [argMax](/sql-reference/aggregate-functions/reference/argmax.md) -* [Tuple](/sql-reference/data-types/tuple.md) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md deleted file mode 100644 index e6b44adab8a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: '`val` の最小値に対応する `arg` と `val` の値を計算します。最小となる同じ `val` を持つ行が複数存在する場合、どの行の `arg` と `val` が返されるかは非決定的です。' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmin -title: 'argAndMin' -doc_type: 'reference' ---- - -# argAndMin {#argandmin} - -最小の `val` 値に対応する `arg` と `val` の値を計算します。最小値となる同じ `val` を持つ行が複数ある場合、どの行に対応する `arg` と `val` が返されるかは決定されていません。 -`arg` および `min` の両方は[集約関数](/sql-reference/aggregate-functions/index.md)として動作し、処理中にどちらも[`Null` をスキップ](/sql-reference/aggregate-functions/index.md#null-processing)し、`Null` でない値が存在する場合には `Null` でない値を返します。 - -:::note -`argMin` との唯一の違いは、`argAndMin` が引数と値の両方を返す点です。 -::: - -**構文** - -```sql -argAndMin(arg, val) -``` - -**引数** - -* `arg` — 引数。 -* `val` — 値。 - -**戻り値** - -* 最小の `val` に対応する `arg` の値。 -* `val` の最小値。 - -型: `arg`、`val` の型にそれぞれ対応するタプル。 - -**例** - -入力テーブル: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -クエリ: - -```sql -SELECT argAndMin(user, salary) FROM salary -``` - -結果: - -```text -┌─argAndMin(user, salary)─┐ -│ ('worker',1000) │ -└─────────────────────────┘ -``` - -**詳細な例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a,b), argAndMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─argAndMin(a, b)─┬─min(b)─┐ -│ a │ ('a',1) │ 0 │ -- argMin = a は最初の非 `NULL` 値であるため。min(b) は別の行から取得されています -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMin(tuple(a), b) FROM test; -┌─argAndMin((a), b)─┐ -│ ((NULL),0) │ -- `NULL` 値のみを含む 'a' `Tuple` 自体は `NULL` ではないため、集約関数はその `NULL` 値を理由にその行をスキップしません -└───────────────────┘ - -SELECT (argAndMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA──┬─argMinB─┐ -│ (NULL,0) │ 0 │ -- `Tuple` を使用することで、対応する min(b) の両方(すべて - tuple(*))の列を取得できます -└──────────┴─────────┘ - -SELECT argAndMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argAndMin(a, b)─┬─min(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │ -- フィルタにより集約対象のすべての行に少なくとも1つの `NULL` 値が含まれるため、すべての行がスキップされ、結果は `NULL` になります -└─────────────────┴────────┘ - -SELECT argAndMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin(a, (b, a))─┬─min((b, a))─┐ -│ ('a',(1,'a')) │ (0,NULL) │ -- 'a' は min における最初の非 `NULL` 値です -└──────────────────────┴─────────────┘ - -SELECT argAndMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin((a, b), (b, a))─┬─min((b, a))─┐ -│ ((NULL,0),(0,NULL)) │ (0,NULL) │ -- `Tuple` は `NULL` をスキップしないため、argAndMin はここで ((NULL,0),(0,NULL)) を返します。この場合の min(tuple(b, a)) はこのデータセットの最小値です -└───────────────────────────┴─────────────┘ - -SELECT argAndMin(a, tuple(b)) FROM test; -┌─argAndMin(a, (b))─┐ -│ ('a',(1)) │ -- `Tuple` を `min` で使用することで、b に `NULL` 値を持つ行をスキップしないようにできます -└───────────────────┘ -``` - -**関連項目** - -* [argMin](/sql-reference/aggregate-functions/reference/argmin.md) -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md deleted file mode 100644 index 50bd9d487a7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -description: '最大の `val` に対応する `arg` の値を計算します。' -sidebar_position: 109 -slug: /sql-reference/aggregate-functions/reference/argmax -title: 'argMax' -doc_type: 'reference' ---- - -# argMax {#argmax} - -最大の `val` 値に対応する `arg` の値を計算します。最大値の `val` を持つ行が複数ある場合、どの行の `arg` が返されるかは決定的ではありません。 -`arg` 側と `max` 側はどちらも[集約関数](/sql-reference/aggregate-functions/index.md)として動作し、処理中はいずれも[`Null` をスキップ](/sql-reference/aggregate-functions/index.md#null-processing)し、`Null` 以外の値が存在する場合には `Null` 以外の値を返します。 - -**構文** - -```sql -argMax(arg, val) -``` - -**引数** - -* `arg` — 引数。 -* `val` — 値。 - -**戻り値** - -* `val` の最大値に対応する `arg` の値。 - -型: `arg` と同じ型。 - -**例** - -入力テーブル: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -クエリ: - -```sql -SELECT argMax(user, salary) FROM salary; -``` - -結果: - -```text -┌─argMax(user, salary)─┐ -│ director │ -└──────────────────────┘ -``` - -**詳細な例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─max(b)─┐ -│ b │ 3 │ -- argMax = 'b' は最初の非NULL値であるため。max(b) は別の行から取得されます! -└──────────────┴────────┘ - -SELECT argMax(tuple(a), b) FROM test; -┌─argMax(tuple(a), b)─┐ -│ (NULL) │ -- `NULL` 値のみを含む `Tuple` は `NULL` ではないため、集約関数はその `NULL` 値によってその行をスキップしません -└─────────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA─┬─argMaxB─┐ -│ ᴺᵁᴸᴸ │ 3 │ -- Tuple を使用して、対応する max(b) の両方(すべて - tuple(*))の列を取得できます -└─────────┴─────────┘ - -SELECT argMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argMax(a, b)─┬─max(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- フィルタにより集約されたすべての行に少なくとも1つの `NULL` 値が含まれるため、すべての行がスキップされ、結果は `NULL` になります -└──────────────┴────────┘ - -SELECT argMax(a, (b,a)) FROM test; -┌─argMax(a, tuple(b, a))─┐ -│ c │ -- b=2 の行が2つあります。`Max` 内の `Tuple` を使用すると、最初の `arg` 以外を取得できます -└────────────────────────┘ - -SELECT argMax(a, tuple(b)) FROM test; -┌─argMax(a, tuple(b))─┐ -│ b │ -- `Max` 内で `Tuple` を使用することで、`Max` で NULL をスキップしないようにできます -└─────────────────────┘ -``` - -**関連項目** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md deleted file mode 100644 index 5d29b694868..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: '`val` の最小値に対応する `arg` の値を計算します。`val` が同じ最小値を持つ行が複数存在する場合、どの行に対応する `arg` が返されるかは規定されていません。' -sidebar_position: 110 -slug: /sql-reference/aggregate-functions/reference/argmin -title: 'argMin' -doc_type: 'reference' ---- - -# argMin {#argmin} - -最小の `val` 値に対応する `arg` の値を計算します。`val` が最小となる行が複数ある場合、どの行に対応する `arg` が返されるかは非決定的です。 -`arg` 部分および `min` 部分の両方は[集約関数](/sql-reference/aggregate-functions/index.md)として動作し、処理中はどちらも[`Null` をスキップ](/sql-reference/aggregate-functions/index.md#null-processing)し、`Null` ではない値が存在する場合には `Null` ではない値を返します。 - -**構文** - -```sql -argMin(arg, val) -``` - -**引数** - -* `arg` — 引数。 -* `val` — 値。 - -**返される値** - -* `val` が最小となる行の `arg` の値。 - -型: `arg` と同じ型。 - -**例** - -入力テーブル: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -クエリ: - -```sql -SELECT argMin(user, salary) FROM salary -``` - -結果: - -```text -┌─argMin(user, salary)─┐ -│ worker │ -└──────────────────────┘ -``` - -**詳細な例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─min(b)─┐ -│ a │ 0 │ -- argMin = a となるのは、最初の非 `NULL` 値であるため。min(b) は別の行から取得される -└──────────────┴────────┘ - -SELECT argMin(tuple(a), b) FROM test; -┌─argMin(tuple(a), b)─┐ -│ (NULL) │ -- `NULL` 値のみを含む `Tuple` 自体は `NULL` ではないため、集約関数はその `NULL` 値を理由にその行をスキップしない -└─────────────────────┘ - -SELECT (argMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA─┬─argMinB─┐ -│ ᴺᵁᴸᴸ │ 0 │ -- `Tuple` を使用することで、対応する max(b) の両方(すべて - tuple(*))の列を取得できる -└─────────┴─────────┘ - -SELECT argMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argMin(a, b)─┬─min(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- フィルタにより集約対象のすべての行に少なくとも1つの `NULL` 値が含まれるため、すべての行がスキップされ、結果は `NULL` となる -└──────────────┴────────┘ - -SELECT argMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(a, tuple(b, a))─┬─min(tuple(b, a))─┐ -│ d │ (NULL,NULL) │ -- 'd' は min に対する最初の非 `NULL` 値である -└────────────────────────┴──────────────────┘ - -SELECT argMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(tuple(a, b), tuple(b, a))─┬─min(tuple(b, a))─┐ -│ (NULL,NULL) │ (NULL,NULL) │ -- `Tuple` は `NULL` をスキップしないため argMin はここで (NULL,NULL) を返し、この場合の min(tuple(b, a)) はこのデータセットの最小値となる -└──────────────────────────────────┴──────────────────┘ - -SELECT argMin(a, tuple(b)) FROM test; -┌─argMin(a, tuple(b))─┐ -│ d │ -- `Tuple` を `min` で使用することで、b に `NULL` 値を持つ行をスキップしないようにできる -└─────────────────────┘ -``` - -**関連項目** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md deleted file mode 100644 index a502ad13b47..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: '算術平均を計算します。' -sidebar_position: 112 -slug: /sql-reference/aggregate-functions/reference/avg -title: 'avg' -doc_type: 'reference' ---- - -# avg {#avg} - -算術平均値を計算します。 - -**構文** - -```sql -avg(x) -``` - -**引数** - -* `x` — 入力値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md) である必要があります。 - -**戻り値** - -* 算術平均値。常に [Float64](../../../sql-reference/data-types/float.md) として返されます。 -* 入力パラメーター `x` が空の場合は `NaN` を返します。 - -**例** - -クエリ: - -```sql -SELECT avg(x) FROM VALUES('x Int8', 0, 1, 2, 3, 4, 5); -``` - -結果: - -```text -┌─avg(x)─┐ -│ 2.5 │ -└────────┘ -``` - -**例** - -一時テーブルを作成します。 - -クエリ: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -``` - -算術平均を求めます: - -クエリ: - -```sql -SELECT avg(t) FROM test; -``` - -結果: - -```text -┌─avg(x)─┐ -│ nan │ -└────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md deleted file mode 100644 index 480373f57a9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -description: '重み付き算術平均を計算します。' -sidebar_position: 113 -slug: /sql-reference/aggregate-functions/reference/avgweighted -title: 'avgWeighted' -doc_type: 'reference' ---- - -# avgWeighted {#avgweighted} - -[加重算術平均](https://en.wikipedia.org/wiki/Weighted_arithmetic_mean)を計算します。 - -**構文** - -```sql -加重平均(x, weight) -``` - -**引数** - -* `x` — 値。 -* `weight` — 値に対応する重み。 - -`x` と `weight` はどちらも -[Integer](../../../sql-reference/data-types/int-uint.md) または [floating-point](../../../sql-reference/data-types/float.md) -である必要がありますが、型が異なっていてもかまいません。 - -**戻り値** - -* すべての重みが 0 の場合、または指定された `weight` パラメータが空の場合は `NaN`。 -* それ以外の場合は加重平均。 - -**戻り値の型** は常に [Float64](../../../sql-reference/data-types/float.md) です。 - -**例** - -クエリ: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (4, 1), (1, 0), (10, 2)) -``` - -結果: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**例** - -クエリ: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Float64', (4, 1), (1, 0), (10, 2)) -``` - -結果: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**例** - -クエリ: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (0, 0), (1, 0), (10, 0)) -``` - -結果: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` - -**例** - -クエリ: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -SELECT avgWeighted(t) FROM test -``` - -結果: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md deleted file mode 100644 index 1fd4b1cc172..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '値のグループ内で、最も左端と最も右端の点の間の傾きを計算する集約関数。' -sidebar_position: 114 -slug: /sql-reference/aggregate-functions/reference/boundingRatio -title: 'boundingRatio' -doc_type: 'reference' ---- - -値のグループ内で、最も左端と最も右端の点の間の傾きを計算する集約関数。 - -例: - -サンプルデータ: - -```sql -SELECT - number, - number * 1.5 -FROM numbers(10) -``` - -```response -┌─number─┬─multiply(number, 1.5)─┐ -│ 0 │ 0 │ -│ 1 │ 1.5 │ -│ 2 │ 3 │ -│ 3 │ 4.5 │ -│ 4 │ 6 │ -│ 5 │ 7.5 │ -│ 6 │ 9 │ -│ 7 │ 10.5 │ -│ 8 │ 12 │ -│ 9 │ 13.5 │ -└────────┴───────────────────────┘ -``` - -`boundingRatio()` 関数は、左端の点と右端の点を結ぶ直線の傾きを返します。上記のデータでは、これらの点は `(0,0)` と `(9,13.5)` です。 - -```sql -SELECT boundingRatio(number, number * 1.5) -FROM numbers(10) -``` - -```response -┌─boundingRatio(number, multiply(number, 1.5))─┐ -│ 1.5 │ -└──────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md deleted file mode 100644 index 35d17f632e8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: 'カテゴリごとに `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` の値を計算します。' -sidebar_position: 115 -slug: /sql-reference/aggregate-functions/reference/categoricalinformationvalue -title: 'categoricalInformationValue' -doc_type: 'reference' ---- - -カテゴリごとに `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` の値を計算します。 - -```sql -categoricalInformationValue(category1, category2, ..., tag) -``` - -この結果は、離散(カテゴリカル)特徴量 `[category1, category2, ...]` が、`tag` の値を予測する学習モデルにどのように寄与しているかを示します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md deleted file mode 100644 index b6d88a12280..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '`contingency` 関数は連関係数(contingency coefficient)を計算します。これはテーブル内の 2 つの列間の関連性を測定する値です。計算方法は `cramersV` 関数と似ていますが、平方根内の分母が異なります。' -sidebar_position: 116 -slug: /sql-reference/aggregate-functions/reference/contingency -title: 'contingency' -doc_type: 'reference' ---- - -# contingency {#contingency} - -`contingency` 関数は、テーブル内の 2 つの列間の関連性を測定する値である [コンティンジェンシー係数 (contingency coefficient)](https://en.wikipedia.org/wiki/Contingency_table#Cram%C3%A9r's_V_and_the_contingency_coefficient_C) を計算します。計算方法は [`cramersV` 関数](./cramersv.md) と類似していますが、平方根の中の分母が異なります。 - -**構文** - -```sql -contingency(column1, column2) -``` - -**引数** - -* `column1` と `column2` は比較対象となる列です - -**戻り値** - -* 0 から 1 の間の値。結果が大きいほど、2 つの列の関連性が高くなります。 - -**戻り値の型** は常に [Float64](../../../sql-reference/data-types/float.md) です。 - -**例** - -以下で比較している 2 つの列は、お互いの関連性が低いことが分かります。比較のために、`cramersV` の結果も併記しています。 - -```sql -SELECT - cramersV(a, b), - contingency(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -結果: - -```response -┌─────cramersV(a, b)─┬──contingency(a, b)─┐ -│ 0.5798088336225178 │ 0.0817230766271248 │ -└────────────────────┴────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md deleted file mode 100644 index 0b78dd8e55e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: 'ピアソン相関係数を計算します。' -sidebar_position: 117 -slug: /sql-reference/aggregate-functions/reference/corr -title: 'corr' -doc_type: 'reference' ---- - - - -# corr {#corr} - -[ピアソンの相関係数](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)を計算します。 - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -
-:::note この関数は数値的に不安定なアルゴリズムを使用します。計算において[数値的安定性](https://en.wikipedia.org/wiki/Numerical_stability)が必要な場合は、[`corrStable`](../reference/corrstable.md) 関数を使用してください。処理は遅くなりますが、より高精度な結果が得られます。 ::: - -**構文** - -```sql -corr(x, y) -``` - -**引数** - -- `x` — 1 番目の変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)。 -- `y` — 2 番目の変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)。 - -**戻り値** - -- ピアソンの相関係数。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corr(x_value, y_value) -FROM series; -``` - -結果: - -```response -┌─corr(x_value, y_value)─┐ -│ 0.1730265755453256 │ -└────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md deleted file mode 100644 index 9cd5a24f8e9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'N個の変数の相関行列を計算します。' -sidebar_position: 118 -slug: /sql-reference/aggregate-functions/reference/corrmatrix -title: 'corrMatrix' -doc_type: 'reference' ---- - -# corrMatrix {#corrmatrix} - -N 個の変数にわたる相関行列を計算します。 - -**構文** - -```sql -corrMatrix(x[, ...]) -``` - -**引数** - -* `x` — 可変個のパラメーター。[`(U)Int8/16/32/64`](../../data-types/int-uint.md), [`Float*`](../../data-types/float.md)。 - -**返り値** - -* 相関行列。[`Array`](../../data-types/array.md)([`Array`](../../data-types/array.md)([`Float64`](../../data-types/float.md)))。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(corrMatrix(a, b, c, d))) AS corrMatrix -FROM test; -``` - -結果: - -```response - ┌─corrMatrix─────────────┐ -1. │ [1,-0.096,0.243,0.746] │ -2. │ [-0.096,1,0.173,0.106] │ -3. │ [0.243,0.173,1,0.258] │ -4. │ [0.746,0.106,0.258,1] │ - └────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md deleted file mode 100644 index 27d0feecc5e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: 'ピアソンの相関係数を、数値的に安定なアルゴリズムで計算します。' -sidebar_position: 119 -slug: /sql-reference/aggregate-functions/reference/corrstable -title: 'corrStable' -doc_type: 'reference' ---- - - - -# corrStable {#corrstable} - -[ピアソン相関係数](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient)を計算します。 - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -[`corr`](../reference/corr.md) 関数と同様ですが、数値的に安定なアルゴリズムを使用します。その結果、`corrStable` は `corr` よりも遅くなりますが、より高い精度の結果を返します。 - -**構文** - -```sql -corrStable(x, y) -``` - -**引数** - -- `x` — 1 番目の変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 2 番目の変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -- ピアソン相関係数。[Float64](../../data-types/float.md)。 - -**\*例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corrStable(x_value, y_value) -FROM series; -``` - -結果: - -```response -┌─corrStable(x_value, y_value)─┐ -│ 0.17302657554532558 │ -└──────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md deleted file mode 100644 index f5d028ea20e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '行数または NULL 以外の値の数をカウントします。' -sidebar_position: 120 -slug: /sql-reference/aggregate-functions/reference/count -title: 'count' -doc_type: 'reference' ---- - -# count {#count} - -行数または NULL ではない値の数をカウントします。 - -ClickHouse は `count` に対して次の構文をサポートしています: - -* `count(expr)` または `COUNT(DISTINCT expr)`。 -* `count()` または `COUNT(*)`。`count()` 構文は ClickHouse 固有です。 - -**引数** - -この関数は次を受け取ることができます: - -* パラメータなし。 -* 1 つの[式](/sql-reference/syntax#expressions)。 - -**返される値** - -* 関数がパラメータなしで呼び出された場合、行数をカウントします。 -* [式](/sql-reference/syntax#expressions)が渡された場合、その式が NULL ではない値を返した回数をカウントします。式が [Nullable](../../../sql-reference/data-types/nullable.md) 型の値を返す場合でも、`count` の結果は `Nullable` にはなりません。すべての行で式が `NULL` を返した場合、関数は 0 を返します。 - -どちらの場合も、返される値の型は [UInt64](../../../sql-reference/data-types/int-uint.md) です。 - -**詳細** - -ClickHouse は `COUNT(DISTINCT ...)` 構文をサポートします。この構文の挙動は [count_distinct_implementation](../../../operations/settings/settings.md#count_distinct_implementation) 設定に依存します。この設定は、処理を行う際にどの [uniq*](/sql-reference/aggregate-functions/reference/uniq) 関数を使用するかを定義します。デフォルトは [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) 関数です。 - -`SELECT count() FROM table` クエリは、デフォルトで MergeTree のメタデータを使用して最適化されます。行レベルセキュリティを使用する必要がある場合は、[optimize_trivial_count_query](/operations/settings/settings#optimize_trivial_count_query) 設定を使用してこの最適化を無効にしてください。 - -一方、`SELECT count(nullable_column) FROM table` クエリは [optimize_functions_to_subcolumns](/operations/settings/settings#optimize_functions_to_subcolumns) 設定を有効にすることで最適化できます。`optimize_functions_to_subcolumns = 1` の場合、関数はカラム全体のデータを読み取って処理する代わりに、[null](../../../sql-reference/data-types/nullable.md#finding-null) サブカラムのみを読み取ります。クエリ `SELECT count(n) FROM table` は `SELECT sum(NOT n.null) FROM table` に変換されます。 - -**COUNT(DISTINCT expr) のパフォーマンス改善** - -`COUNT(DISTINCT expr)` クエリが遅い場合は、並列化が向上するため [`GROUP BY`](/sql-reference/statements/select/group-by) 句の追加を検討してください。また、`COUNT(DISTINCT target_col)` で使用される対象カラムにインデックスを作成するために、[プロジェクション](../../../sql-reference/statements/alter/projection.md)を使用することもできます。 - -**例** - -例 1: - -```sql -SELECT count() FROM t -``` - -```text -┌─count()─┐ -│ 5 │ -└─────────┘ -``` - -例 2: - -```sql -SELECT name, value FROM system.settings WHERE name = 'count_distinct_implementation' -``` - -```text -┌─name──────────────────────────┬─value─────┐ -│ count_distinct_implementation │ uniqExact │ -└───────────────────────────────┴───────────┘ -``` - -```sql -SELECT count(DISTINCT num) FROM t -``` - -```text -┌─uniqExact(num)─┐ -│ 3 │ -└────────────────┘ -``` - -この例は、`count_distinct_implementation` の設定値に応じて、`count(DISTINCT num)` が `uniqExact` 関数によって実行されることを示しています。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md deleted file mode 100644 index 0c815497706..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '母集団の共分散を計算します' -sidebar_position: 121 -slug: /sql-reference/aggregate-functions/reference/covarpop -title: 'covarPop' -doc_type: 'reference' ---- - - - -# covarPop {#covarpop} - -母集団共分散を計算します: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -:::note -この関数は数値的に不安定なアルゴリズムを使用しています。計算において[数値安定性](https://en.wikipedia.org/wiki/Numerical_stability)が必要な場合は、[`covarPopStable`](../reference/covarpopstable.md)関数を使用してください。処理速度は遅くなりますが、計算誤差を低く抑えることができます。 -::: - -**構文** - -```sql -covarPop(x, y) -``` - -**引数** - -- `x` — 第1変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 第2変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -- `x`と`y`の母集団共分散。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT covarPop(x_value, y_value) -FROM series; -``` - -結果: - -```reference -┌─covarPop(x_value, y_value)─┐ -│ 6.485648 │ -└────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md deleted file mode 100644 index ba76452bcb0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'N 個の変数の母集団共分散行列を返します。' -sidebar_position: 122 -slug: /sql-reference/aggregate-functions/reference/covarpopmatrix -title: 'covarPopMatrix' -doc_type: 'reference' ---- - -# covarPopMatrix {#covarpopmatrix} - -N 個の変数に対する母共分散行列を返します。 - -**構文** - -```sql -covarPopMatrix(x[, ...]) -``` - -**引数** - -* `x` — 可変長の引数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -* 母集団共分散行列。[Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md)))。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarPopMatrix(a, b, c, d))) AS covarPopMatrix -FROM test; -``` - -結果: - -```reference - ┌─covarPopMatrix────────────┐ -1. │ [8.25,-1.76,4.08,6.748] │ -2. │ [-1.76,41.07,6.486,2.132] │ -3. │ [4.08,6.486,34.21,4.755] │ -4. │ [6.748,2.132,4.755,9.93] │ - └───────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md deleted file mode 100644 index c5d3956deb5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: '母集団共分散を計算します' -sidebar_position: 123 -slug: /sql-reference/aggregate-functions/reference/covarpopstable -title: 'covarPopStable' -doc_type: 'reference' ---- - - - -# covarPopStable {#covarpopstable} - -母集団共分散の値を計算します: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -[covarPop](../reference/covarpop.md)関数と似ていますが、数値的に安定したアルゴリズムを使用します。その結果、`covarPopStable`は`covarPop`よりも処理速度は遅くなりますが、より正確な結果が得られます。 - -**構文** - -```sql -covarPop(x, y) -``` - -**引数** - -- `x` — 第1変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 第2変数。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -- `x`と`y`の母集団共分散。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarPopStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -結果: - -```reference -┌─covarPopStable(x_value, y_value)─┐ -│ 6.485648 │ -└──────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md deleted file mode 100644 index 61e90e83efb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: '`Σ((x - x̅)(y - y̅)) / (n - 1)` を計算します' -sidebar_position: 124 -slug: /sql-reference/aggregate-functions/reference/covarsamp -title: 'covarSamp' -doc_type: 'reference' ---- - -# covarSamp {#covarsamp} - -`Σ((x - x̅)(y - y̅)) / (n - 1)` の値を計算します。 - -:::note -この関数は数値的に不安定なアルゴリズムを使用します。計算で[数値安定性](https://en.wikipedia.org/wiki/Numerical_stability)が必要な場合は、[`covarSampStable`](../reference/covarsamp.md) 関数を使用してください。`covarSampStable` は処理速度は遅くなりますが、計算誤差をより小さく抑えられます。 -::: - -**構文** - -```sql -covarSamp(x, y) -``` - -**引数** - -* `x` — 1つ目の変数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -* `y` — 2つ目の変数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -* `x` と `y` の標本共分散。`n <= 1` の場合は `nan` を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -結果: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ 7.206275555555556 │ -└─────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); - -``` - -結果: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ nan │ -└─────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md deleted file mode 100644 index aaf880a91a0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'N個の変数の標本共分散行列を返します。' -sidebar_position: 125 -slug: /sql-reference/aggregate-functions/reference/covarsampmatrix -title: 'covarSampMatrix' -doc_type: 'reference' ---- - -# covarSampMatrix {#covarsampmatrix} - -N 個の変数からなる標本共分散行列を返します。 - -**構文** - -```sql -covarSampMatrix(x[, ...]) -``` - -**引数** - -* `x` — 可変個のパラメータ。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -* 標本共分散行列。[Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md)))。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarSampMatrix(a, b, c, d))) AS covarSampMatrix -FROM test; -``` - -結果: - -```reference - ┌─covarSampMatrix─────────────┐ -1. │ [9.167,-1.956,4.534,7.498] │ -2. │ [-1.956,45.634,7.206,2.369] │ -3. │ [4.534,7.206,38.011,5.283] │ -4. │ [7.498,2.369,5.283,11.034] │ - └─────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md deleted file mode 100644 index d01f716dfcb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'covarSamp と同様ですが、計算誤差を小さく抑える代わりに処理が遅くなります。' -sidebar_position: 126 -slug: /sql-reference/aggregate-functions/reference/covarsampstable -title: 'covarSampStable' -doc_type: 'reference' ---- - -# covarSampStable {#covarsampstable} - -`Σ((x - x̅)(y - y̅)) / (n - 1)` の値を計算します。[covarSamp](../reference/covarsamp.md) と同様ですが、計算誤差をより小さく抑えられる一方で、処理速度は低下します。 - -**構文** - -```sql -covarSampStable(x, y) -``` - -**引数** - -* `x` — 1 番目の変数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -* `y` — 2 番目の変数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**戻り値** - -* `x` と `y` の標本共分散。`n <= 1` の場合は `inf` が返されます。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -結果: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ 7.206275555555556 │ -└───────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); -``` - -結果: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ inf │ -└───────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md deleted file mode 100644 index e9bcf670ca6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -description: '`cramersV` 関数の結果は 0(変数間に連関がないことに対応)から 1 の範囲を取り、一方の変数の各値が他方の変数の値によって完全に決定される場合にのみ 1 に達します。これは、2 つの変数間の連関の強さを、それらが取りうる理論上の最大変動に対する割合として解釈できます。' -sidebar_position: 127 -slug: /sql-reference/aggregate-functions/reference/cramersv -title: 'cramersV' -doc_type: 'reference' ---- - -# cramersV {#cramersv} - -[Cramer's V](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V)(Cramer's phi と呼ばれることもあります)は、テーブル内の 2 つの列間の連関を測る指標です。`cramersV` 関数の結果は 0(変数間に連関がないことに対応)から 1 の範囲を取り、各値が他方によって完全に決定される場合にのみ 1 になります。これは、2 つの変数間の連関を、それらが取りうる最大の変動に対する割合として捉えることができます。 - -:::note -バイアス補正版の Cramer's V については次を参照してください: [cramersVBiasCorrected](./cramersvbiascorrected.md) -::: - -**構文** - -```sql -cramersV(列1, 列2) -``` - -**パラメータ** - -* `column1`: 比較対象となる 1 つ目の列。 -* `column2`: 比較対象となる 2 つ目の列。 - -**戻り値** - -* 0(列の値同士にまったく連関がないことに対応)から 1(完全な連関)までの値。 - -型: 常に [Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -以下で比較している 2 つの列には互いに連関がないため、`cramersV` の結果は 0 になります: - -クエリ: - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 3 AS a, - number % 5 AS b - FROM - numbers(150) - ); -``` - -結果: - -```response -┌─cramersV(a, b)─┐ -│ 0 │ -└────────────────┘ -``` - -以下の 2 つの列同士には比較的強い関連性があるため、`cramersV` の結果は高い値になります。 - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 10 AS a, - if(number % 12 = 0, (number + 1) % 5, number % 5) AS b - FROM - numbers(150) - ); -``` - -結果: - -```response -┌─────cramersV(a, b)─┐ -│ 0.9066801892162646 │ -└────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md deleted file mode 100644 index 7662422de68..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: 'Cramer の V を計算し、バイアス補正を適用します。' -sidebar_position: 128 -slug: /sql-reference/aggregate-functions/reference/cramersvbiascorrected -title: 'cramersVBiasCorrected' -doc_type: 'reference' ---- - -# cramersVBiasCorrected {#cramersvbiascorrected} - -Cramer's V は、テーブル内の 2 つの列間の関連の強さを表す指標です。[`cramersV` 関数](./cramersv.md) の結果は 0(変数間に関連がないことに対応)から 1 の範囲の値を取り、一方の値が他方の値によって一意に決まる場合にのみ 1 に達します。この関数は強いバイアスがかかる可能性があるため、このバージョンの Cramer's V では [バイアス補正](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V#Bias_correction) を使用します。 - -**構文** - -```sql -cramersVBiasCorrected(column1, column2) -``` - -**パラメータ** - -* `column1`: 比較対象となる最初の列。 -* `column2`: 比較対象となる2番目の列。 - -**戻り値** - -* 0(列の値同士に連関がまったくない状態)から 1(完全な連関)までの値。 - -型: 常に [Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -以下で比較されている2つの列には、互いに中程度の連関があります。`cramersVBiasCorrected` の結果が `cramersV` の結果より小さいことに注目してください。 - -クエリ: - -```sql -SELECT - cramersV(a, b), - cramersVBiasCorrected(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -結果: - -```response -┌─────cramersV(a, b)─┬─cramersVBiasCorrected(a, b)─┐ -│ 0.5798088336225178 │ 0.5305112825189074 │ -└────────────────────┴─────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md deleted file mode 100644 index af92f3ea158..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '連続する行の値の差を合計します。' -sidebar_position: 129 -slug: /sql-reference/aggregate-functions/reference/deltasum -title: 'deltaSum' -doc_type: 'reference' ---- - - - -# deltaSum {#deltasum} - -連続する行間の算術的な差分を合計します。差分が負の値の場合は無視されます。 - -:::note -この関数が正しく動作するには、基になるデータがソートされている必要があります。この関数を[マテリアライズドビュー](/sql-reference/statements/create/view#materialized-view)で使用したい場合は、代わりに [deltaSumTimestamp](/sql-reference/aggregate-functions/reference/deltasumtimestamp) メソッドを使用することをお勧めします。 -::: - -**構文** - -```sql -deltaSum(value) -``` - -**引数** - -* `value` — 入力値。[Integer](../../data-types/int-uint.md) 型または [Float](../../data-types/float.md) 型である必要があります。 - -**戻り値** - -* `Integer` または `Float` 型の算術差分の累積値。 - -**使用例** - -クエリ: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3])); -``` - -結果: - -```text -┌─deltaSum(arrayJoin([1, 2, 3]))─┐ -│ 2 │ -└────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3])); -``` - -結果: - -```text -┌─deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3]))─┐ -│ 7 │ -└───────────────────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT deltaSum(arrayJoin([2.25, 3, 4.5])); -``` - -結果: - -```text -┌─deltaSum(arrayJoin([2.25, 3, 4.5]))─┐ -│ 2.25 │ -└─────────────────────────────────────┘ -``` - - -## 関連項目 {#see-also} - -- [runningDifference](/sql-reference/functions/other-functions#runningDifference) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md deleted file mode 100644 index 93c8913e080..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '連続する行間の差分を加算します。差分が負の場合は無視されます。' -sidebar_position: 130 -slug: /sql-reference/aggregate-functions/reference/deltasumtimestamp -title: 'deltaSumTimestamp' -doc_type: 'reference' ---- - -連続する行間の差分を加算します。差分が負の場合は無視されます。 - -この関数は主に、`toStartOfMinute` バケットのような時間バケットに揃えたタイムスタンプでソートされたデータを保持する[マテリアライズドビュー](/sql-reference/statements/create/view#materialized-view)向けです。このようなマテリアライズドビューでは、行はすべて同じタイムスタンプを持つため、元の丸められていないタイムスタンプ値を保持していないと、正しい順序でマージすることはできません。`deltaSumTimestamp` 関数は、これまでに見た値の元の `timestamp` を追跡することで、パーツのマージ時に関数の値(状態)が正しく計算されるようにします。 - -順序付けされたコレクション全体でデルタの合計を計算するには、単に [deltaSum](/sql-reference/aggregate-functions/reference/deltasum) 関数を使用できます。 - -**構文** - -```sql -deltaSumTimestamp(value, timestamp) -``` - -**引数** - -* `value` — 入力値。いずれかの [Integer](../../data-types/int-uint.md) 型、[Float](../../data-types/float.md) 型、または [Date](../../data-types/date.md)、[DateTime](../../data-types/datetime.md) である必要があります。 -* `timestamp` — 値の順序付けに使用するパラメーター。いずれかの [Integer](../../data-types/int-uint.md) 型、[Float](../../data-types/float.md) 型、または [Date](../../data-types/date.md)、[DateTime](../../data-types/datetime.md) である必要があります。 - -**返される値** - -* `timestamp` パラメーターで指定された順序で並ぶ連続する値同士の差分を累積したもの。 - -型: [Integer](../../data-types/int-uint.md)、[Float](../../data-types/float.md)、[Date](../../data-types/date.md)、または [DateTime](../../data-types/datetime.md) のいずれか。 - -**例** - -クエリ: - -```sql -SELECT deltaSumTimestamp(value, timestamp) -FROM (SELECT number AS timestamp, [0, 4, 8, 3, 0, 0, 0, 1, 3, 5][number] AS value FROM numbers(1, 10)); -``` - -結果: - -```text -┌─deltaSumTimestamp(value, timestamp)─┐ -│ 13 │ -└─────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md deleted file mode 100644 index 1238833a262..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Dynamic カラムに格納されている重複のないデータ型の一覧を計算します。' -sidebar_position: 215 -slug: /sql-reference/aggregate-functions/reference/distinctdynamictypes -title: 'distinctDynamicTypes' -doc_type: 'reference' ---- - -# distinctDynamicTypes {#distinctdynamictypes} - -[Dynamic](../../data-types/dynamic.md) カラムに格納されている異なるデータ型の一覧を返します。 - -**構文** - -```sql -distinctDynamicTypes(dynamic) -``` - -**引数** - -* `dynamic` — [Dynamic](../../data-types/dynamic.md) 列。 - -**戻り値** - -* データ型名のソートされたリスト [Array(String)](../../data-types/array.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_dynamic; -CREATE TABLE test_dynamic(d Dynamic) ENGINE = Memory; -INSERT INTO test_dynamic VALUES (42), (NULL), ('Hello'), ([1, 2, 3]), ('2020-01-01'), (map(1, 2)), (43), ([4, 5]), (NULL), ('World'), (map(3, 4)) -``` - -```sql -SELECT distinctDynamicTypes(d) FROM test_dynamic; -``` - -結果: - -```reference -┌─distinctDynamicTypes(d)──────────────────────────────────────┐ -│ ['Array(Int64)','Date','Int64','Map(UInt8, UInt8)','String'] │ -└──────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md deleted file mode 100644 index 615b5289f86..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -description: 'JSON 列に格納された一意なパスの一覧を算出します。' -sidebar_position: 216 -slug: /sql-reference/aggregate-functions/reference/distinctjsonpaths -title: 'distinctJSONPaths' -doc_type: 'reference' ---- - -# distinctJSONPaths {#distinctjsonpaths} - -[JSON](../../data-types/newjson.md) カラムに保存されているパスのうち、一意なもののリストを返します。 - -**構文** - -```sql -distinctJSONPaths(json) -``` - -**引数** - -* `json` — [JSON](../../data-types/newjson.md) 列。 - -**返される値** - -* ソート済みのパスのリスト [Array(String)](../../data-types/array.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "こんにちは"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -結果: - -```reference -┌─distinctJSONPaths(json)───┐ -│ ['a','b','c.d.e','c.d.f'] │ -└───────────────────────────┘ -``` - -# distinctJSONPathsAndTypes {#distinctjsonpathsandtypes} - -[JSON](../../data-types/newjson.md) 列に保存されている一意なパスとその型の一覧を取得します。 - -**構文** - -```sql -distinctJSONPathsAndTypes(json) -``` - -**引数** - -* `json` — [JSON](../../data-types/newjson.md) 型の列。 - -**戻り値** - -* パスと型の対応を表すソート済みマップ [Map(String, Array(String))](../../data-types/map.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "Hello"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -結果: - -```reference -┌─distinctJSONPathsAndTypes(json)───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {'a':['Int64'],'b':['Array(Nullable(Int64))','String'],'c.d.e':['Date'],'c.d.f':['Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))']} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**注記** - -JSON 宣言に型が指定されているパスが含まれている場合、入力データにそのパスの値が存在しない場合でも、これらのパスは常に `distinctJSONPaths/distinctJSONPathsAndTypes` 関数の結果に含まれます。 - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON(a UInt32)) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"b" : "Hello"}'), ('{"b" : "World", "c" : [1, 2, 3]}'); -``` - -```sql -SELECT json FROM test_json; -``` - -```text -┌─json──────────────────────────────────┐ -│ {"a":0,"b":"Hello"} │ -│ {"a":0,"b":"World","c":["1","2","3"]} │ -└───────────────────────────────────────┘ -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -```text -┌─distinctJSONPaths(json)─┐ -│ ['a','b','c'] │ -└─────────────────────────┘ -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -```text -┌─distinctJSONPathsAndTypes(json)────────────────────────────────┐ -│ {'a':['UInt32'],'b':['String'],'c':['Array(Nullable(Int64))']} │ -└────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md deleted file mode 100644 index 8f7116dc468..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '値のカラムのシャノンエントロピーを計算します。' -sidebar_position: 131 -slug: /sql-reference/aggregate-functions/reference/entropy -title: 'entropy' -doc_type: 'reference' ---- - -# entropy {#entropy} - -列内の値に対して[シャノンエントロピー](https://en.wikipedia.org/wiki/Entropy_\(information_theory\))を計算します。 - -**構文** - -```sql -entropy(val) -``` - -**引数** - -* `val` — 任意の型の値を持つ列。 - -**返される値** - -* シャノンのエントロピー。 - -型: [Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE entropy (`vals` UInt32,`strings` String) ENGINE = Memory; - -INSERT INTO entropy VALUES (1, 'A'), (1, 'A'), (1,'A'), (1,'A'), (2,'B'), (2,'B'), (2,'C'), (2,'D'); - -SELECT entropy(vals), entropy(strings) FROM entropy; -``` - -結果: - -```text -┌─entropy(vals)─┬─entropy(strings)─┐ -│ 1 │ 1.75 │ -└───────────────┴──────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md deleted file mode 100644 index 02693dc8fc2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: '指定された列を実際に圧縮することなく、その圧縮率を推定します。' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/estimateCompressionRatio -title: 'estimateCompressionRatio' -doc_type: 'reference' ---- - -## estimateCompressionRatio {#estimatecompressionration} - -指定されたカラムを圧縮処理を行わずに、その圧縮率を推定します。 - -**構文** - -```sql -estimateCompressionRatio(codec, block_size_bytes)(column) -``` - -**引数** - -* `column` - 任意の型の列 - -**パラメータ** - -* `codec` - [圧縮コーデック](/sql-reference/statements/create/table#column_compression_codec) を指定する [String](../../../sql-reference/data-types/string.md)。1 つのコーデック、または複数のコーデックをカンマ区切りで 1 つの文字列として指定します。 -* `block_size_bytes` - 圧縮データのブロックサイズ。これは [`max_compress_block_size`](../../../operations/settings/merge-tree-settings.md#max_compress_block_size) と [`min_compress_block_size`](../../../operations/settings/merge-tree-settings.md#min_compress_block_size) の両方を設定することに類似します。デフォルト値は 1 MiB (1048576 バイト) です。 - -どちらのパラメータも省略可能です。 - -**戻り値** - -* 指定された列に対する圧縮率の推定値を返します。 - -型: [Float64](/sql-reference/data-types/float)。 - -**例** - -```sql title="Input table" -CREATE TABLE compression_estimate_example -( - `number` UInt64 -) -ENGINE = MergeTree() -ORDER BY number -SETTINGS min_bytes_for_wide_part = 0; - -INSERT INTO compression_estimate_example -SELECT number FROM system.numbers LIMIT 100_000; -``` - -```sql title="Query" -SELECT estimateCompressionRatio(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌───────────estimate─┐ -│ 1.9988506608699999 │ -└────────────────────┘ -``` - -:::note -上記の結果は、サーバーのデフォルト圧縮コーデックによって異なります。詳細については、[列圧縮コーデック](/sql-reference/statements/create/table#column_compression_codec)を参照してください。 -::: - -```sql title="Query" -SELECT estimateCompressionRatio('T64')(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌──────────estimate─┐ -│ 3.762758101688538 │ -└───────────────────┘ -``` - -関数では複数のコーデックを指定することもできます。 - -```sql title="Query" -SELECT estimateCompressionRatio('T64, ZSTD')(number) AS estimate FROM compression_estimate_example; -``` - -```response title="Response" -┌───────────estimate─┐ -│ 143.60078980434392 │ -└────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md deleted file mode 100644 index 8fc548245fc..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md +++ /dev/null @@ -1,203 +0,0 @@ ---- -description: '指定された期間における値の指数移動平均を計算します。' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/exponentialMovingAverage -title: 'exponentialMovingAverage' -doc_type: 'reference' ---- - -## exponentialMovingAverage {#exponentialmovingaverage} - -指定された期間における値の指数移動平均を計算します。 - -**構文** - -```sql -exponentialMovingAverage(x)(value, timeunit) -``` - -各 `value` は、与えられた `timeunit` に対応します。半減期 `x` は、指数重みが半分に減衰する時間差を表します。関数は加重平均を返します。時点が古くなるほど、その値に与えられる重みは小さくなります。 - -**引数** - -* `value` — 値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `timeunit` — 時間単位。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。`timeunit` はタイムスタンプ(秒)ではなく、時間間隔のインデックスです。[intDiv](/sql-reference/functions/arithmetic-functions#intDiv) を使って計算できます。 - -**パラメータ** - -* `x` — 半減期。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**戻り値** - -* 直近の時点において、過去 `x` の期間にわたる値に対する[指数平滑移動平均](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average)を返します。 - -型: [Float64](/sql-reference/data-types/float)。 - -**使用例** - -入力テーブル: - -```text -┌──temperature─┬─timestamp──┐ -│ 95 │ 1 │ -│ 95 │ 2 │ -│ 95 │ 3 │ -│ 96 │ 4 │ -│ 96 │ 5 │ -│ 96 │ 6 │ -│ 96 │ 7 │ -│ 97 │ 8 │ -│ 97 │ 9 │ -│ 97 │ 10 │ -│ 97 │ 11 │ -│ 98 │ 12 │ -│ 98 │ 13 │ -│ 98 │ 14 │ -│ 98 │ 15 │ -│ 99 │ 16 │ -│ 99 │ 17 │ -│ 99 │ 18 │ -│ 100 │ 19 │ -│ 100 │ 20 │ -└──────────────┴────────────┘ -``` - -クエリ: - -```sql -SELECT exponentialMovingAverage(5)(temperature, timestamp); -``` - -結果: - -```text -┌──exponentialMovingAverage(5)(temperature, timestamp)──┐ -│ 92.25779635374204 │ -└───────────────────────────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 1, 50) AS bar -FROM -( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialMovingAverage(10)(value, time) OVER (Rows BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -) -``` - -結果: - -```text -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────────────────────────────────────┐ -│ 1 │ 0 │ 0.067 │ ███▎ │ -│ 0 │ 1 │ 0.062 │ ███ │ -│ 0 │ 2 │ 0.058 │ ██▊ │ -│ 0 │ 3 │ 0.054 │ ██▋ │ -│ 0 │ 4 │ 0.051 │ ██▌ │ -│ 0 │ 5 │ 0.047 │ ██▎ │ -│ 0 │ 6 │ 0.044 │ ██▏ │ -│ 0 │ 7 │ 0.041 │ ██ │ -│ 0 │ 8 │ 0.038 │ █▊ │ -│ 0 │ 9 │ 0.036 │ █▋ │ -│ 0 │ 10 │ 0.033 │ █▋ │ -│ 0 │ 11 │ 0.031 │ █▌ │ -│ 0 │ 12 │ 0.029 │ █▍ │ -│ 0 │ 13 │ 0.027 │ █▎ │ -│ 0 │ 14 │ 0.025 │ █▎ │ -│ 0 │ 15 │ 0.024 │ █▏ │ -│ 0 │ 16 │ 0.022 │ █ │ -│ 0 │ 17 │ 0.021 │ █ │ -│ 0 │ 18 │ 0.019 │ ▊ │ -│ 0 │ 19 │ 0.018 │ ▊ │ -│ 0 │ 20 │ 0.017 │ ▋ │ -│ 0 │ 21 │ 0.016 │ ▋ │ -│ 0 │ 22 │ 0.015 │ ▋ │ -│ 0 │ 23 │ 0.014 │ ▋ │ -│ 0 │ 24 │ 0.013 │ ▋ │ -│ 1 │ 25 │ 0.079 │ ███▊ │ -│ 1 │ 26 │ 0.14 │ ███████ │ -│ 1 │ 27 │ 0.198 │ █████████▊ │ -│ 1 │ 28 │ 0.252 │ ████████████▌ │ -│ 1 │ 29 │ 0.302 │ ███████████████ │ -│ 1 │ 30 │ 0.349 │ █████████████████▍ │ -│ 1 │ 31 │ 0.392 │ ███████████████████▌ │ -│ 1 │ 32 │ 0.433 │ █████████████████████▋ │ -│ 1 │ 33 │ 0.471 │ ███████████████████████▌ │ -│ 1 │ 34 │ 0.506 │ █████████████████████████▎ │ -│ 1 │ 35 │ 0.539 │ ██████████████████████████▊ │ -│ 1 │ 36 │ 0.57 │ ████████████████████████████▌ │ -│ 1 │ 37 │ 0.599 │ █████████████████████████████▊ │ -│ 1 │ 38 │ 0.626 │ ███████████████████████████████▎ │ -│ 1 │ 39 │ 0.651 │ ████████████████████████████████▌ │ -│ 1 │ 40 │ 0.674 │ █████████████████████████████████▋ │ -│ 1 │ 41 │ 0.696 │ ██████████████████████████████████▋ │ -│ 1 │ 42 │ 0.716 │ ███████████████████████████████████▋ │ -│ 1 │ 43 │ 0.735 │ ████████████████████████████████████▋ │ -│ 1 │ 44 │ 0.753 │ █████████████████████████████████████▋ │ -│ 1 │ 45 │ 0.77 │ ██████████████████████████████████████▍ │ -│ 1 │ 46 │ 0.785 │ ███████████████████████████████████████▎ │ -│ 1 │ 47 │ 0.8 │ ███████████████████████████████████████▊ │ -│ 1 │ 48 │ 0.813 │ ████████████████████████████████████████▋ │ -│ 1 │ 49 │ 0.825 │ █████████████████████████████████████████▎ │ -└───────┴──────┴──────────────────────┴────────────────────────────────────────────┘ -``` - -```sql -CREATE TABLE data -ENGINE = Memory AS -SELECT - 10 AS value, - toDateTime('2020-01-01') + (3600 * number) AS time -FROM numbers_mt(10); --- intDivを使用してtimeunitを計算 -SELECT - value, - time, - exponentialMovingAverage(1)(value, intDiv(toUInt32(time), 3600)) OVER (ORDER BY time ASC) AS res, - intDiv(toUInt32(time), 3600) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ --- toRelativeHourNumを使用してtimeunitを計算 -SELECT - value, - time, - exponentialMovingAverage(1)(value, toRelativeHourNum(time)) OVER (ORDER BY time ASC) AS res, - toRelativeHourNum(time) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md deleted file mode 100644 index ea5f836bbf0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '時点 `t` における時系列データの値に対する、指数平滑化された加重移動平均を返します。' -sidebar_position: 133 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedAvg -title: 'exponentialTimeDecayedAvg' -doc_type: 'reference' ---- - -## exponentialTimeDecayedAvg {#exponentialtimedecayedavg} - -時刻 `t` における時系列データの値に対する、指数平滑化された加重移動平均を返します。 - -**構文** - -```sql -exponentialTimeDecayedAvg(x)(v, t) -``` - -**引数** - -* `v` — 値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `t` — 時刻。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**パラメータ** - -* `x` — 半減期の期間。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**戻り値** - -* 時刻 `t` における指数平滑化された加重移動平均を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedAvg(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -レスポンス: - -```sql -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ -1. │ 1 │ 0 │ 1 │ ██████████ │ -2. │ 0 │ 1 │ 0.475 │ ████▊ │ -3. │ 0 │ 2 │ 0.301 │ ███ │ -4. │ 0 │ 3 │ 0.214 │ ██▏ │ -5. │ 0 │ 4 │ 0.162 │ █▌ │ -6. │ 0 │ 5 │ 0.128 │ █▎ │ -7. │ 0 │ 6 │ 0.104 │ █ │ -8. │ 0 │ 7 │ 0.086 │ ▊ │ -9. │ 0 │ 8 │ 0.072 │ ▋ │ -0. │ 0 │ 9 │ 0.061 │ ▌ │ -1. │ 0 │ 10 │ 0.052 │ ▌ │ -2. │ 0 │ 11 │ 0.045 │ ▍ │ -3. │ 0 │ 12 │ 0.039 │ ▍ │ -4. │ 0 │ 13 │ 0.034 │ ▎ │ -5. │ 0 │ 14 │ 0.03 │ ▎ │ -6. │ 0 │ 15 │ 0.027 │ ▎ │ -7. │ 0 │ 16 │ 0.024 │ ▏ │ -8. │ 0 │ 17 │ 0.021 │ ▏ │ -9. │ 0 │ 18 │ 0.018 │ ▏ │ -0. │ 0 │ 19 │ 0.016 │ ▏ │ -1. │ 0 │ 20 │ 0.015 │ ▏ │ -2. │ 0 │ 21 │ 0.013 │ ▏ │ -3. │ 0 │ 22 │ 0.012 │ │ -4. │ 0 │ 23 │ 0.01 │ │ -5. │ 0 │ 24 │ 0.009 │ │ -6. │ 1 │ 25 │ 0.111 │ █ │ -7. │ 1 │ 26 │ 0.202 │ ██ │ -8. │ 1 │ 27 │ 0.283 │ ██▊ │ -9. │ 1 │ 28 │ 0.355 │ ███▌ │ -0. │ 1 │ 29 │ 0.42 │ ████▏ │ -1. │ 1 │ 30 │ 0.477 │ ████▊ │ -2. │ 1 │ 31 │ 0.529 │ █████▎ │ -3. │ 1 │ 32 │ 0.576 │ █████▊ │ -4. │ 1 │ 33 │ 0.618 │ ██████▏ │ -5. │ 1 │ 34 │ 0.655 │ ██████▌ │ -6. │ 1 │ 35 │ 0.689 │ ██████▉ │ -7. │ 1 │ 36 │ 0.719 │ ███████▏ │ -8. │ 1 │ 37 │ 0.747 │ ███████▍ │ -9. │ 1 │ 38 │ 0.771 │ ███████▋ │ -0. │ 1 │ 39 │ 0.793 │ ███████▉ │ -1. │ 1 │ 40 │ 0.813 │ ████████▏ │ -2. │ 1 │ 41 │ 0.831 │ ████████▎ │ -3. │ 1 │ 42 │ 0.848 │ ████████▍ │ -4. │ 1 │ 43 │ 0.862 │ ████████▌ │ -5. │ 1 │ 44 │ 0.876 │ ████████▊ │ -6. │ 1 │ 45 │ 0.888 │ ████████▉ │ -7. │ 1 │ 46 │ 0.898 │ ████████▉ │ -8. │ 1 │ 47 │ 0.908 │ █████████ │ -9. │ 1 │ 48 │ 0.917 │ █████████▏ │ -0. │ 1 │ 49 │ 0.925 │ █████████▏ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md deleted file mode 100644 index afaa56c4abd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: '時系列に対し、時間インデックス `t` における累積の指数減衰値を返します。' -sidebar_position: 134 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedCount -title: 'exponentialTimeDecayedCount' -doc_type: 'reference' ---- - -## exponentialTimeDecayedCount {#exponentialtimedecayedcount} - -時系列において、時刻 `t` における指数減衰の累積値を返します。 - -**構文** - -```sql -exponentialTimeDecayedCount(x)(t) -``` - -**引数** - -* `t` — 時刻。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、[Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**パラメータ** - -* `x` — 半減期を表す値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**戻り値** - -* 指定した時点における累積指数減衰を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 20, 50) AS bar -FROM -( - SELECT - (number % 5) = 0 AS value, - number AS time, - exponentialTimeDecayedCount(10)(time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -); -``` - -結果: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ ██▌ │ - 2. │ 0 │ 1 │ 1.905 │ ████▊ │ - 3. │ 0 │ 2 │ 2.724 │ ██████▊ │ - 4. │ 0 │ 3 │ 3.464 │ ████████▋ │ - 5. │ 0 │ 4 │ 4.135 │ ██████████▎ │ - 6. │ 1 │ 5 │ 4.741 │ ███████████▊ │ - 7. │ 0 │ 6 │ 5.29 │ █████████████▏ │ - 8. │ 0 │ 7 │ 5.787 │ ██████████████▍ │ - 9. │ 0 │ 8 │ 6.236 │ ███████████████▌ │ -10. │ 0 │ 9 │ 6.643 │ ████████████████▌ │ -11. │ 1 │ 10 │ 7.01 │ █████████████████▌ │ -12. │ 0 │ 11 │ 7.343 │ ██████████████████▎ │ -13. │ 0 │ 12 │ 7.644 │ ███████████████████ │ -14. │ 0 │ 13 │ 7.917 │ ███████████████████▊ │ -15. │ 0 │ 14 │ 8.164 │ ████████████████████▍ │ -16. │ 1 │ 15 │ 8.387 │ ████████████████████▉ │ -17. │ 0 │ 16 │ 8.589 │ █████████████████████▍ │ -18. │ 0 │ 17 │ 8.771 │ █████████████████████▉ │ -19. │ 0 │ 18 │ 8.937 │ ██████████████████████▎ │ -20. │ 0 │ 19 │ 9.086 │ ██████████████████████▋ │ -21. │ 1 │ 20 │ 9.222 │ ███████████████████████ │ -22. │ 0 │ 21 │ 9.344 │ ███████████████████████▎ │ -23. │ 0 │ 22 │ 9.455 │ ███████████████████████▋ │ -24. │ 0 │ 23 │ 9.555 │ ███████████████████████▉ │ -25. │ 0 │ 24 │ 9.646 │ ████████████████████████ │ -26. │ 1 │ 25 │ 9.728 │ ████████████████████████▎ │ -27. │ 0 │ 26 │ 9.802 │ ████████████████████████▌ │ -28. │ 0 │ 27 │ 9.869 │ ████████████████████████▋ │ -29. │ 0 │ 28 │ 9.93 │ ████████████████████████▊ │ -30. │ 0 │ 29 │ 9.985 │ ████████████████████████▉ │ -31. │ 1 │ 30 │ 10.035 │ █████████████████████████ │ -32. │ 0 │ 31 │ 10.08 │ █████████████████████████▏ │ -33. │ 0 │ 32 │ 10.121 │ █████████████████████████▎ │ -34. │ 0 │ 33 │ 10.158 │ █████████████████████████▍ │ -35. │ 0 │ 34 │ 10.191 │ █████████████████████████▍ │ -36. │ 1 │ 35 │ 10.221 │ █████████████████████████▌ │ -37. │ 0 │ 36 │ 10.249 │ █████████████████████████▌ │ -38. │ 0 │ 37 │ 10.273 │ █████████████████████████▋ │ -39. │ 0 │ 38 │ 10.296 │ █████████████████████████▋ │ -40. │ 0 │ 39 │ 10.316 │ █████████████████████████▊ │ -41. │ 1 │ 40 │ 10.334 │ █████████████████████████▊ │ -42. │ 0 │ 41 │ 10.351 │ █████████████████████████▉ │ -43. │ 0 │ 42 │ 10.366 │ █████████████████████████▉ │ -44. │ 0 │ 43 │ 10.379 │ █████████████████████████▉ │ -45. │ 0 │ 44 │ 10.392 │ █████████████████████████▉ │ -46. │ 1 │ 45 │ 10.403 │ ██████████████████████████ │ -47. │ 0 │ 46 │ 10.413 │ ██████████████████████████ │ -48. │ 0 │ 47 │ 10.422 │ ██████████████████████████ │ -49. │ 0 │ 48 │ 10.43 │ ██████████████████████████ │ -50. │ 0 │ 49 │ 10.438 │ ██████████████████████████ │ - └───────┴──────┴──────────────────────┴────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md deleted file mode 100644 index a73a8f1abf2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '時間インデックス `t` における指数平滑移動平均と、時間インデックス `t-1` における指数平滑移動平均との最大値を返します。 ' -sidebar_position: 135 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedMax -title: 'exponentialTimeDecayedMax' -doc_type: 'reference' ---- - -## exponentialTimeDecayedMax {#exponentialtimedecayedmax} - -時刻インデックス `t` における指数平滑移動平均の値と、その一つ前の時刻インデックス `t-1` における値のうち、大きい方を返します。 - -**構文** - -```sql -exponentialTimeDecayedMax(x)(value, timeunit) -``` - -**引数** - -* `value` — 値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `timeunit` — 時間単位。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**パラメータ** - -* `x` — 半減期の期間。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**戻り値** - -* `t` と `t-1` における指数平滑化された加重移動平均の最大値を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedMax(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -結果: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ - 1. │ 1 │ 0 │ 1 │ ██████████ │ - 2. │ 0 │ 1 │ 0.905 │ █████████ │ - 3. │ 0 │ 2 │ 0.819 │ ████████▏ │ - 4. │ 0 │ 3 │ 0.741 │ ███████▍ │ - 5. │ 0 │ 4 │ 0.67 │ ██████▋ │ - 6. │ 0 │ 5 │ 0.607 │ ██████ │ - 7. │ 0 │ 6 │ 0.549 │ █████▍ │ - 8. │ 0 │ 7 │ 0.497 │ ████▉ │ - 9. │ 0 │ 8 │ 0.449 │ ████▍ │ -10. │ 0 │ 9 │ 0.407 │ ████ │ -11. │ 0 │ 10 │ 0.368 │ ███▋ │ -12. │ 0 │ 11 │ 0.333 │ ███▎ │ -13. │ 0 │ 12 │ 0.301 │ ███ │ -14. │ 0 │ 13 │ 0.273 │ ██▋ │ -15. │ 0 │ 14 │ 0.247 │ ██▍ │ -16. │ 0 │ 15 │ 0.223 │ ██▏ │ -17. │ 0 │ 16 │ 0.202 │ ██ │ -18. │ 0 │ 17 │ 0.183 │ █▊ │ -19. │ 0 │ 18 │ 0.165 │ █▋ │ -20. │ 0 │ 19 │ 0.15 │ █▍ │ -21. │ 0 │ 20 │ 0.135 │ █▎ │ -22. │ 0 │ 21 │ 0.122 │ █▏ │ -23. │ 0 │ 22 │ 0.111 │ █ │ -24. │ 0 │ 23 │ 0.1 │ █ │ -25. │ 0 │ 24 │ 0.091 │ ▉ │ -26. │ 1 │ 25 │ 1 │ ██████████ │ -27. │ 1 │ 26 │ 1 │ ██████████ │ -28. │ 1 │ 27 │ 1 │ ██████████ │ -29. │ 1 │ 28 │ 1 │ ██████████ │ -30. │ 1 │ 29 │ 1 │ ██████████ │ -31. │ 1 │ 30 │ 1 │ ██████████ │ -32. │ 1 │ 31 │ 1 │ ██████████ │ -33. │ 1 │ 32 │ 1 │ ██████████ │ -34. │ 1 │ 33 │ 1 │ ██████████ │ -35. │ 1 │ 34 │ 1 │ ██████████ │ -36. │ 1 │ 35 │ 1 │ ██████████ │ -37. │ 1 │ 36 │ 1 │ ██████████ │ -38. │ 1 │ 37 │ 1 │ ██████████ │ -39. │ 1 │ 38 │ 1 │ ██████████ │ -40. │ 1 │ 39 │ 1 │ ██████████ │ -41. │ 1 │ 40 │ 1 │ ██████████ │ -42. │ 1 │ 41 │ 1 │ ██████████ │ -43. │ 1 │ 42 │ 1 │ ██████████ │ -44. │ 1 │ 43 │ 1 │ ██████████ │ -45. │ 1 │ 44 │ 1 │ ██████████ │ -46. │ 1 │ 45 │ 1 │ ██████████ │ -47. │ 1 │ 46 │ 1 │ ██████████ │ -48. │ 1 │ 47 │ 1 │ ██████████ │ -49. │ 1 │ 48 │ 1 │ ██████████ │ -50. │ 1 │ 49 │ 1 │ ██████████ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md deleted file mode 100644 index 31022833a9b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '時系列に対して、時刻インデックス `t` における指数平滑移動平均値の合計を返します。' -sidebar_position: 136 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedSum -title: 'exponentialTimeDecayedSum' -doc_type: 'reference' ---- - -## exponentialTimeDecayedSum {#exponentialtimedecayedsum} - -時系列データのインデックス `t` における、指数平滑移動平均値の合計を返します。 - -**構文** - -```sql -exponentialTimeDecayedSum(x)(v, t) -``` - -**引数** - -* `v` — 値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `t` — 時刻。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**パラメータ** - -* `x` — 値の重みが 1/e まで減衰するのに必要な時間差。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**戻り値** - -* 指定した時点における指数平滑移動平均の合計を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 10, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedSum(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -結果: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar───────────────────────────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ █████ │ - 2. │ 0 │ 1 │ 0.905 │ ████▌ │ - 3. │ 0 │ 2 │ 0.819 │ ████ │ - 4. │ 0 │ 3 │ 0.741 │ ███▋ │ - 5. │ 0 │ 4 │ 0.67 │ ███▎ │ - 6. │ 0 │ 5 │ 0.607 │ ███ │ - 7. │ 0 │ 6 │ 0.549 │ ██▋ │ - 8. │ 0 │ 7 │ 0.497 │ ██▍ │ - 9. │ 0 │ 8 │ 0.449 │ ██▏ │ -10. │ 0 │ 9 │ 0.407 │ ██ │ -11. │ 0 │ 10 │ 0.368 │ █▊ │ -12. │ 0 │ 11 │ 0.333 │ █▋ │ -13. │ 0 │ 12 │ 0.301 │ █▌ │ -14. │ 0 │ 13 │ 0.273 │ █▎ │ -15. │ 0 │ 14 │ 0.247 │ █▏ │ -16. │ 0 │ 15 │ 0.223 │ █ │ -17. │ 0 │ 16 │ 0.202 │ █ │ -18. │ 0 │ 17 │ 0.183 │ ▉ │ -19. │ 0 │ 18 │ 0.165 │ ▊ │ -20. │ 0 │ 19 │ 0.15 │ ▋ │ -21. │ 0 │ 20 │ 0.135 │ ▋ │ -22. │ 0 │ 21 │ 0.122 │ ▌ │ -23. │ 0 │ 22 │ 0.111 │ ▌ │ -24. │ 0 │ 23 │ 0.1 │ ▌ │ -25. │ 0 │ 24 │ 0.091 │ ▍ │ -26. │ 1 │ 25 │ 1.082 │ █████▍ │ -27. │ 1 │ 26 │ 1.979 │ █████████▉ │ -28. │ 1 │ 27 │ 2.791 │ █████████████▉ │ -29. │ 1 │ 28 │ 3.525 │ █████████████████▋ │ -30. │ 1 │ 29 │ 4.19 │ ████████████████████▉ │ -31. │ 1 │ 30 │ 4.791 │ ███████████████████████▉ │ -32. │ 1 │ 31 │ 5.335 │ ██████████████████████████▋ │ -33. │ 1 │ 32 │ 5.827 │ █████████████████████████████▏ │ -34. │ 1 │ 33 │ 6.273 │ ███████████████████████████████▎ │ -35. │ 1 │ 34 │ 6.676 │ █████████████████████████████████▍ │ -36. │ 1 │ 35 │ 7.041 │ ███████████████████████████████████▏ │ -37. │ 1 │ 36 │ 7.371 │ ████████████████████████████████████▊ │ -38. │ 1 │ 37 │ 7.669 │ ██████████████████████████████████████▎ │ -39. │ 1 │ 38 │ 7.939 │ ███████████████████████████████████████▋ │ -40. │ 1 │ 39 │ 8.184 │ ████████████████████████████████████████▉ │ -41. │ 1 │ 40 │ 8.405 │ ██████████████████████████████████████████ │ -42. │ 1 │ 41 │ 8.605 │ ███████████████████████████████████████████ │ -43. │ 1 │ 42 │ 8.786 │ ███████████████████████████████████████████▉ │ -44. │ 1 │ 43 │ 8.95 │ ████████████████████████████████████████████▊ │ -45. │ 1 │ 44 │ 9.098 │ █████████████████████████████████████████████▍ │ -46. │ 1 │ 45 │ 9.233 │ ██████████████████████████████████████████████▏ │ -47. │ 1 │ 46 │ 9.354 │ ██████████████████████████████████████████████▊ │ -48. │ 1 │ 47 │ 9.464 │ ███████████████████████████████████████████████▎ │ -49. │ 1 │ 48 │ 9.563 │ ███████████████████████████████████████████████▊ │ -50. │ 1 │ 49 │ 9.653 │ ████████████████████████████████████████████████▎ │ - └───────┴──────┴──────────────────────┴───────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md deleted file mode 100644 index 72b3dbf439e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'これは any のエイリアスですが、ウィンドウ関数との互換性のために導入されました。ウィンドウ関数では、`NULL` 値を処理する必要がある場合があります(デフォルトでは、すべての ClickHouse 集約関数は `NULL` 値を無視します)。' -sidebar_position: 137 -slug: /sql-reference/aggregate-functions/reference/first_value -title: 'first_value' -doc_type: 'reference' ---- - -# first_value {#first_value} - -これは[`any`](../../../sql-reference/aggregate-functions/reference/any.md) のエイリアスですが、[Window Functions](../../window-functions/index.md) との互換性のために導入されたものです。Window Functions では `NULL` 値を処理する必要が生じることがあります(デフォルトでは、すべての ClickHouse 集約関数は NULL 値を無視します)。 - -この関数は、NULL を考慮する修飾子(`RESPECT NULLS`)の指定をサポートしており、[Window Functions](../../window-functions/index.md) および通常の集約処理の両方で使用できます。 - -`any` と同様に、Window Functions を使用しない場合、ソースストリームがソートされていなければ結果は不定となり、戻り値の型は入力の型と一致します(NULL が返されるのは、入力が Nullable 型である場合、または -OrNull コンビネータが追加されている場合のみです)。 - -## 使用例 {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null); -``` - -### 例 1 {#example1} - -デフォルトでは、NULL値は無視されます。 - -```sql -SELECT first_value(b) FROM test_data; -``` - -```text -┌─any(b)─┐ -│ 3 │ -└────────┘ -``` - -### 例 2 {#example2} - -NULL 値は無視されます。 - -```sql -SELECT first_value(b) ignore nulls FROM test_data -``` - -```text -┌─any(b) IGNORE NULLS ─┐ -│ 3 │ -└──────────────────────┘ -``` - -### 例 3 {#example3} - -NULL 値が許容されます。 - -```sql -SELECT first_value(b) respect nulls FROM test_data -``` - -```text -┌─any(b) RESPECT NULLS ─┐ -│ ᴺᵁᴸᴸ │ -└───────────────────────┘ -``` - -### 例 4 {#example4} - -`ORDER BY` を含むサブクエリを使用して結果を安定させた例。 - -```sql -SELECT - first_value_respect_nulls(b), - first_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─any_respect_nulls(b)─┬─any(b)─┐ -│ ᴺᵁᴸᴸ │ 3 │ -└──────────────────────┴────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md deleted file mode 100644 index 893d56e01a6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: 'スタックトレースのリストからフレームグラフを構築する集約関数。' -sidebar_position: 138 -slug: /sql-reference/aggregate-functions/reference/flame_graph -title: 'flameGraph' -doc_type: 'reference' ---- - - - -# flameGraph {#flamegraph} - -スタックトレースのリストを使用して[フレームグラフ](https://www.brendangregg.com/flamegraphs.html)を構築する集約関数です。[flamegraph.pl ユーティリティ](https://github.com/brendangregg/FlameGraph)でフレームグラフの SVG をレンダリングする際に利用できる文字列配列を出力します。 - - - -## 構文 {#syntax} - -```sql -flameGraph(traces, [size], [ptr]) -``` - - -## パラメータ {#parameters} - -- `traces` — スタックトレース。[Array](../../data-types/array.md)([UInt64](../../data-types/int-uint.md))。 -- `size` — メモリプロファイリング用のアロケーションサイズ(省略可、既定値は `1`)。[UInt64](../../data-types/int-uint.md)。 -- `ptr` — アロケーションアドレス(省略可、既定値は `0`)。[UInt64](../../data-types/int-uint.md)。 - -:::note -`ptr != 0` の場合、FlameGraph は同じ size と ptr を持つアロケーション(size > 0)とデアロケーション(size < 0)を対応付けます。 -解放されていないアロケーションのみが表示されます。対応付けられないデアロケーションは無視されます。 -::: - - - -## 戻り値 {#returned-value} - -- [flamegraph.pl ユーティリティ](https://github.com/brendangregg/FlameGraph) で使用する文字列の配列。[Array](../../data-types/array.md)([String](../../data-types/string.md))。 - - - -## 例 {#examples} - -### CPU クエリプロファイラに基づくフレームグラフの作成 {#building-a-flamegraph-based-on-a-cpu-query-profiler} - -```sql -SET query_profiler_cpu_time_period_ns=10000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(arrayReverse(trace))) from system.trace_log where trace_type = 'CPU' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl > flame_cpu.svg -``` - -### メモリクエリプロファイラに基づいて、すべてのアロケーションを可視化するフレームグラフの作成 {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-all-allocations} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(trace, size)) from system.trace_log where trace_type = 'MemorySample' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem.svg -``` - -### メモリクエリプロファイラに基づいて、クエリコンテキスト内で解放されなかったメモリアロケーションを示すフレームグラフを作成する {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-allocations-which-were-not-deallocated-in-query-context} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1, use_uncompressed_cache=1, merge_tree_max_rows_to_use_cache=100000000000, merge_tree_max_bytes_to_use_cache=1000000000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_untracked.svg -``` - -### メモリクエリプロファイラに基づいてフレームグラフを作成し、ある時点における有効なメモリ割り当てを表示する {#build-a-flamegraph-based-on-memory-query-profiler-showing-active-allocations-at-the-fixed-point-of-time} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -* 1 - メモリ使用量(毎秒) - -```sql -SELECT event_time, m, formatReadableSize(max(s) AS m) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample') GROUP BY event_time ORDER BY event_time; -``` - -* 2 - メモリ使用量が最大となる時刻を特定する - -```sql -SELECT argMax(event_time, s), max(s) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample'); -``` - -* 3 - 特定時点におけるアクティブな割り当てを固定する - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time <= 'yyy' ORDER BY event_time)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_pos.svg -``` - -* 4 - 固定時刻におけるメモリ解放を検出する - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, -size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time > 'yyy' ORDER BY event_time desc)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_neg.svg -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md deleted file mode 100644 index 4d9b6d45ea5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '引数値の配列を作成します。値は任意の(順不同の)順序で配列に追加されます。' -sidebar_position: 139 -slug: /sql-reference/aggregate-functions/reference/grouparray -title: 'groupArray' -doc_type: 'reference' ---- - -# groupArray {#grouparray} - -構文: `groupArray(x)` または `groupArray(max_size)(x)` - -引数の値を要素とする配列を作成します。 -配列に値が追加される順序は任意(非決定的)です。 - -2 番目の形式(`max_size` パラメータ付き)は、結果の配列サイズを `max_size` 要素に制限します。たとえば、`groupArray(1)(x)` は `[any (x)]` と等価です。 - -場合によっては、実行順序に依存しても問題ないことがあります。これは、`SELECT` が `ORDER BY` を使用するサブクエリからのものであり、そのサブクエリの結果が十分に小さい場合に当てはまります。 - -**例** - -```text -SELECT * FROM default.ck; - -┌─id─┬─name─────┐ -│ 1 │ zhangsan │ -│ 1 │ ᴺᵁᴸᴸ │ -│ 1 │ lisi │ -│ 2 │ wangwu │ -└────┴──────────┘ - -``` - -クエリ: - -```sql -SELECT id, groupArray(10)(name) FROM default.ck GROUP BY id; -``` - -結果: - -```text -┌─id─┬─groupArray(10)(name)─┐ -│ 1 │ ['zhangsan','lisi'] │ -│ 2 │ ['wangwu'] │ -└────┴──────────────────────┘ -``` - -`groupArray` 関数は、上記の結果に基づいて ᴺᵁᴸᴸ 値を除外します。 - -* 別名: `array_agg`。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md deleted file mode 100644 index 14ea728c88e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '複数の配列を集約し、それらを要素とするより大きな配列を生成します。' -keywords: ['groupArrayArray', 'array_concat_agg'] -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/grouparrayarray -title: 'groupArrayArray' -doc_type: 'reference' ---- - -# groupArrayArray {#grouparrayarray} - -複数の配列を、それらの配列を要素とするより大きな配列に集約します。 -[`groupArray`](/sql-reference/aggregate-functions/reference/grouparray) 関数と [Array](/sql-reference/aggregate-functions/combinators#-array) コンビネータを組み合わせたものです。 - -エイリアス: `array_concat_agg` - -**例** - -ユーザーのブラウジングセッションを記録したデータがあるとします。各セッションには、特定のユーザーが訪問したページの遷移順が記録されています。 -`groupArrayArray` 関数を使用して、ユーザーごとのページ訪問パターンを分析できます。 - -```sql title="Setup" -CREATE TABLE website_visits ( - user_id UInt32, - session_id UInt32, - page_visits Array(String) -) ENGINE = Memory; - -INSERT INTO website_visits VALUES -(101, 1, ['homepage', 'products', 'checkout']), -(101, 2, ['search', 'product_details', 'contact']), -(102, 1, ['homepage', 'about_us']), -(101, 3, ['blog', 'homepage']), -(102, 2, ['products', 'product_details', 'add_to_cart', 'checkout']); -``` - -```sql title="Query" -SELECT - user_id, - groupArrayArray(page_visits) AS user_session_page_sequences -FROM website_visits -GROUP BY user_id; -``` - -```sql title="Response" - ┌─user_id─┬─user_session_page_sequences───────────────────────────────────────────────────────────────┐ -1. │ 101 │ ['homepage','products','checkout','search','product_details','contact','blog','homepage'] │ -2. │ 102 │ ['homepage','about_us','products','product_details','add_to_cart','checkout'] │ - └─────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md deleted file mode 100644 index 9f5bfe464f7..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: '配列の指定した位置に値を挿入します。' -sidebar_position: 140 -slug: /sql-reference/aggregate-functions/reference/grouparrayinsertat -title: 'groupArrayInsertAt' -doc_type: 'reference' ---- - -# groupArrayInsertAt {#grouparrayinsertat} - -配列の指定した位置に値を挿入します。 - -**構文** - -```sql -groupArrayInsertAt(default_x, size)(x, pos) -``` - -1 つのクエリ内で同じ位置に複数の値が挿入される場合、この関数は次のように動作します。 - -* クエリが単一スレッドで実行される場合、挿入された値のうち最初のものが使用されます。 -* クエリがマルチスレッドで実行される場合、結果の値が挿入された値のどれになるかは不定です。 - -**引数** - -* `x` — 挿入する値。[式](/sql-reference/syntax#expressions) であり、[サポートされているデータ型](../../../sql-reference/data-types/index.md) のいずれかになります。 -* `pos` — 要素 `x` を挿入する位置。配列のインデックス番号は 0 から始まります。[UInt32](/sql-reference/data-types/int-uint#integer-ranges)。 -* `default_x` — 空の位置を埋めるためのデフォルト値。省略可能なパラメータ。[式](/sql-reference/syntax#expressions) であり、パラメータ `x` に設定されたデータ型になります。`default_x` が定義されていない場合は、[デフォルト値](/sql-reference/statements/create/table) が使用されます。 -* `size` — 結果の配列の長さ。省略可能なパラメータ。このパラメータを使用する場合、デフォルト値 `default_x` を指定する必要があります。[UInt32](/sql-reference/data-types/int-uint#integer-ranges)。 - -**返される値** - -* 値が挿入された配列。 - -型: [Array](/sql-reference/data-types/array)。 - -**例** - -クエリ: - -```sql -SELECT groupArrayInsertAt(toString(number), number * 2) FROM numbers(5); -``` - -結果: - -```text -┌─groupArrayInsertAt(toString(number), multiply(number, 2))─┐ -│ ['0','','1','','2','','3','','4'] │ -└───────────────────────────────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT groupArrayInsertAt('-')(toString(number), number * 2) FROM numbers(5); -``` - -結果: - -```text -┌─groupArrayInsertAt('-')(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2','-','3','-','4'] │ -└────────────────────────────────────────────────────────────────┘ -``` - -クエリ: - -```sql -SELECT groupArrayInsertAt('-', 5)(toString(number), number * 2) FROM numbers(5); -``` - -結果: - -```text -┌─groupArrayInsertAt('-', 5)(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2'] │ -└───────────────────────────────────────────────────────────────────┘ -``` - -1 つの位置に要素をマルチスレッドで挿入。 - -クエリ: - -```sql -SELECT groupArrayInsertAt(number, 0) FROM numbers_mt(10) SETTINGS max_block_size = 1; -``` - -このクエリの結果として、`[0,9]` の範囲のランダムな整数が得られます。たとえば次のようになります。 - -```text -┌─groupArrayInsertAt(number, 0)─┐ -│ [7] │ -└───────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md deleted file mode 100644 index 854d2096043..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '指定された配列の共通部分(すべての配列に共通して含まれる要素)を返します。' -sidebar_position: 141 -slug: /sql-reference/aggregate-functions/reference/grouparrayintersect -title: 'groupArrayIntersect' -doc_type: 'reference' ---- - -# groupArrayIntersect {#grouparrayintersect} - -指定された配列同士の共通部分(指定されたすべての配列に共通して含まれるすべての要素)を返します。 - -**構文** - -```sql -groupArrayIntersect(x) -``` - -**引数** - -* `x` — 引数(列名または式)。 - -**返される値** - -* すべての配列に共通して含まれる要素を集めた配列。 - -型: [Array](../../data-types/array.md)。 - -**使用例** - -`numbers` テーブルを考えます。 - -```text -┌─a──────────────┐ -│ [1,2,4] │ -│ [1,5,2,8,-1,0] │ -│ [1,5,7,5,8,2] │ -└────────────────┘ -``` - -列名を引数に取るクエリ: - -```sql -SELECT groupArrayIntersect(a) AS intersection FROM numbers; -``` - -結果: - -```text -┌─intersection──────┐ -│ [1, 2] │ -└───────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md deleted file mode 100644 index e3e6fc761cf..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: '最後の引数の値の配列を作成します。' -sidebar_position: 142 -slug: /sql-reference/aggregate-functions/reference/grouparraylast -title: 'groupArrayLast' -doc_type: 'reference' ---- - -# groupArrayLast {#grouparraylast} - -構文: `groupArrayLast(max_size)(x)` - -引数のうち最後の値からなる配列を作成します。 -たとえば、`groupArrayLast(1)(x)` は `[anyLast (x)]` と同等です。 - -場合によっては、依然として実行順序に依存できます。これは、サブクエリの結果が十分に小さい場合に、`ORDER BY` を使用するサブクエリに対する `SELECT` のケースに適用されます。 - -**例** - -クエリ: - -```sql -SELECT groupArrayLast(2)(number+1) numbers FROM numbers(10) -``` - -結果: - -```text -┌─numbers─┐ -│ [9,10] │ -└─────────┘ -``` - -`groupArray` と比較すると、 - -```sql -SELECT groupArray(2)(number+1) numbers FROM numbers(10) -``` - -```text -┌─numbers─┐ -│ [1,2] │ -└─────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md deleted file mode 100644 index ad2ca4c3c20..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -description: '入力値の移動平均を計算します。' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingavg -title: 'groupArrayMovingAvg' -doc_type: 'reference' ---- - -# groupArrayMovingAvg {#grouparraymovingavg} - -入力値の移動平均を計算します。 - -```sql -groupArrayMovingAvg(numbers_for_summing) -groupArrayMovingAvg(window_size)(numbers_for_summing) -``` - -この関数はウィンドウサイズをパラメータとして受け取ることができます。省略した場合、関数はカラム内の行数と同じ値をウィンドウサイズとして使用します。 - -**引数** - -* `numbers_for_summing` — 数値データ型の値を返す[式](/sql-reference/syntax#expressions)。 -* `window_size` — 計算ウィンドウのサイズ。 - -**戻り値** - -* 入力データと同じサイズおよび型の配列。 - -この関数は[ゼロ方向への丸め](https://en.wikipedia.org/wiki/Rounding#Rounding_towards_zero)を使用します。結果のデータ型で表現できない桁の小数部を切り捨てます。 - -**例** - -サンプルテーブル `b`: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -クエリは次のとおりです。 - -```sql -SELECT - groupArrayMovingAvg(int) AS I, - groupArrayMovingAvg(float) AS F, - groupArrayMovingAvg(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F───────────────────────────────────┬─D─────────────────────┐ -│ [0,0,1,3] │ [0.275,0.82500005,1.9250001,3.8675] │ [0.27,0.82,1.92,3.86] │ -└───────────┴─────────────────────────────────────┴───────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingAvg(2)(int) AS I, - groupArrayMovingAvg(2)(float) AS F, - groupArrayMovingAvg(2)(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F────────────────────────────────┬─D─────────────────────┐ -│ [0,1,3,5] │ [0.55,1.6500001,3.3000002,6.085] │ [0.55,1.65,3.30,6.08] │ -└───────────┴──────────────────────────────────┴───────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md deleted file mode 100644 index 09e839c6cbe..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '入力値の移動合計を計算します。' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingsum -title: 'groupArrayMovingSum' -doc_type: 'reference' ---- - -# groupArrayMovingSum {#grouparraymovingsum} - -入力値の移動和を計算します。 - -```sql -groupArrayMovingSum(numbers_for_summing) -groupArrayMovingSum(window_size)(numbers_for_summing) -``` - -この関数はウィンドウサイズをパラメータとして受け取れます。指定しない場合は、列内の行数と同じウィンドウサイズが使用されます。 - -**引数** - -* `numbers_for_summing` — 数値データ型の値を返す[式](/sql-reference/syntax#expressions)。 -* `window_size` — 計算に用いるウィンドウのサイズ。 - -**戻り値** - -* 入力データと同じサイズおよび型の配列。 - -**例** - -サンプルテーブル: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -クエリ: - -```sql -SELECT - groupArrayMovingSum(int) AS I, - groupArrayMovingSum(float) AS F, - groupArrayMovingSum(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,7,14] │ [1.1,3.3000002,7.7000003,15.47] │ [1.10,3.30,7.70,15.47] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingSum(2)(int) AS I, - groupArrayMovingSum(2)(float) AS F, - groupArrayMovingSum(2)(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,6,11] │ [1.1,3.3000002,6.6000004,12.17] │ [1.10,3.30,6.60,12.17] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md deleted file mode 100644 index d4620fcdd62..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: '引数値のサンプル配列を作成します。結果の配列のサイズは `max_size` 要素に制限されます。引数値はランダムに選択され、配列に追加されます。' -sidebar_position: 145 -slug: /sql-reference/aggregate-functions/reference/grouparraysample -title: 'groupArraySample' -doc_type: 'reference' ---- - -# groupArraySample {#grouparraysample} - -引数値のサンプル配列を作成します。結果として得られる配列のサイズは `max_size` 個の要素に制限されます。引数値はランダムに選択され、配列に追加されます。 - -**構文** - -```sql -groupArraySample(max_size[, seed])(x) -``` - -**引数** - -* `max_size` — 結果となる配列の最大サイズ。[UInt64](../../data-types/int-uint.md)。 -* `seed` — 乱数生成器のシード値。省略可能。[UInt64](../../data-types/int-uint.md)。デフォルト値: `123456`。 -* `x` — 引数(列名または式)。 - -**返される値** - -* 引数 `x` からランダムに選択された要素の配列。 - -型: [Array](../../data-types/array.md)。 - -**例** - -テーブル `colors` を考えます: - -```text -┌─id─┬─color──┐ -│ 1 │ red │ -│ 2 │ blue │ -│ 3 │ green │ -│ 4 │ white │ -│ 5 │ orange │ -└────┴────────┘ -``` - -列名を引数に指定したクエリ: - -```sql -SELECT groupArraySample(3)(color) as newcolors FROM colors; -``` - -結果: - -```text -┌─newcolors──────────────────┐ -│ ['white','blue','green'] │ -└────────────────────────────┘ -``` - -列名と異なるシード値を指定したクエリ: - -```sql -SELECT groupArraySample(3, 987654321)(color) as newcolors FROM colors; -``` - -結果: - -```text -┌─newcolors──────────────────┐ -│ ['red','orange','green'] │ -└────────────────────────────┘ -``` - -引数に式を指定するクエリ: - -```sql -SELECT groupArraySample(3)(concat('light-', color)) as newcolors FROM colors; -``` - -結果: - -```text -┌─newcolors───────────────────────────────────┐ -│ ['light-blue','light-orange','light-green'] │ -└─────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md deleted file mode 100644 index 73250e1c019..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '昇順で先頭 N 個の要素を含む配列を返します。' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/grouparraysorted -title: 'groupArraySorted' -doc_type: 'reference' ---- - -# groupArraySorted {#grouparraysorted} - -先頭 N 個の要素を昇順に並べた配列を返します。 - -```sql -groupArraySorted(N)(column) -``` - -**引数** - -* `N` – 返す要素数。 - -* `column` – 値(Integer、String、Float などの汎用的な型)。 - -**例** - -先頭の 10 個の数値を取得します。 - -```sql -SELECT groupArraySorted(10)(number) FROM numbers(100) -``` - -```text -┌─groupArraySorted(10)(number)─┐ -│ [0,1,2,3,4,5,6,7,8,9] │ -└──────────────────────────────┘ -``` - -列内のすべての数値を文字列として取得します。 - -```sql -SELECT groupArraySorted(5)(str) FROM (SELECT toString(number) AS str FROM numbers(5)); -``` - -```text -┌─groupArraySorted(5)(str)─┐ -│ ['0','1','2','3','4'] │ -└──────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md deleted file mode 100644 index 61c9d03be87..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '数値列に対してビットごとの `AND` を適用します。' -sidebar_position: 147 -slug: /sql-reference/aggregate-functions/reference/groupbitand -title: 'groupBitAnd' -doc_type: 'reference' ---- - -# groupBitAnd {#groupbitand} - -数値の集合に対してビット単位の `AND` 演算を行います。 - -```sql -groupBitAnd(expr) -``` - -**引数** - -`expr` – `UInt*` または `Int*` 型を結果とする式。 - -**戻り値** - -`UInt*` または `Int*` 型の値。 - -**例** - -テストデータ: - -```text -2進数 10進数 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -クエリ: - -```sql -SELECT groupBitAnd(num) FROM t -``` - -ここで、`num` はテストデータが入っている列です。 - -結果: - -```text -2進数 10進数 -00000100 = 4 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md deleted file mode 100644 index 5da7a8ceb0a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '符号なし整数列に対するビットマップまたは集約計算を行い、UInt64 型の基数(要素数)を返します。サフィックスに -State を付けた場合は、ビットマップオブジェクトを返します。' -sidebar_position: 148 -slug: /sql-reference/aggregate-functions/reference/groupbitmap -title: 'groupBitmap' -doc_type: 'reference' ---- - -# groupBitmap {#groupbitmap} - -符号なし整数列に対してビットマップまたは集計計算を実行し、`UInt64` 型のカーディナリティ値を返します。サフィックスとして `-State` を付けた場合は、[ビットマップオブジェクト](../../../sql-reference/functions/bitmap-functions.md) を返します。 - -```sql -groupBitmap(expr) -``` - -**引数** - -`expr` – 評価結果が `UInt*` 型となる式。 - -**戻り値** - -`UInt64` 型の値。 - -**例** - -テストデータ: - -```text -UserID -1 -1 -2 -3 -``` - -クエリ: - -```sql -SELECT groupBitmap(UserID) AS num FROM t -``` - -結果: - -```text -num -3 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md deleted file mode 100644 index 68ecc808928..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'ビットマップ列に対して AND 演算を行い、型 UInt64 の基数を返します。-State サフィックスを付与すると、ビットマップオブジェクトを返します。' -sidebar_position: 149 -slug: /sql-reference/aggregate-functions/reference/groupbitmapand -title: 'groupBitmapAnd' -doc_type: 'reference' ---- - -ビットマップ列に対して AND 演算を行い、型 UInt64 の基数を返します。-State サフィックスを付与すると、[ビットマップオブジェクト](../../../sql-reference/functions/bitmap-functions.md) を返します。 - -```sql -groupBitmapAnd(expr) -``` - -**引数** - -`expr` – `AggregateFunction(groupBitmap, UInt*)` 型の式。 - -**戻り値** - -`UInt64` 型の値。 - -**例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapAnd(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapAnd(z)─┐ -│ 3 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapAndState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapAndState(z)))─┐ -│ [6,8,10] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md deleted file mode 100644 index ca9f41ddadd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'ビットマップ列の OR 演算を行い、型 UInt64 の要素数(カーディナリティ)を返します。-State サフィックスを付与すると、ビットマップオブジェクトを返します。これは `groupBitmapMerge` と同等です。' -sidebar_position: 150 -slug: /sql-reference/aggregate-functions/reference/groupbitmapor -title: 'groupBitmapOr' -doc_type: 'reference' ---- - -# groupBitmapOr {#groupbitmapor} - -ビットマップ列の OR を計算し、`UInt64` 型の基数(要素数)を返します。末尾に `-State` のサフィックスを付けると、[ビットマップオブジェクト](../../../sql-reference/functions/bitmap-functions.md) を返します。これは `groupBitmapMerge` と同等です。 - -```sql -groupBitmapOr(expr) -``` - -**引数** - -`expr` – 評価結果が `AggregateFunction(groupBitmap, UInt*)` 型となる式。 - -**戻り値** - -`UInt64` 型の値。 - -**例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapOr(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapOr(z)─┐ -│ 15 │ -└──────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapOrState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapOrState(z)))─┐ -│ [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] │ -└─────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md deleted file mode 100644 index 804032870c4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'ビットマップ列の XOR を計算し、そのカーディナリティを UInt64 型で返します。サフィックス「-State」と併用した場合は、ビットマップオブジェクトを返します' -sidebar_position: 151 -slug: /sql-reference/aggregate-functions/reference/groupbitmapxor -title: 'groupBitmapXor' -doc_type: 'reference' ---- - -# groupBitmapXor {#groupbitmapxor} - -`groupBitmapXor` はビットマップ列の XOR を計算し、その結果の基数(cardinality)を `UInt64` 型で返します。`-State` 接尾辞付きで使用した場合は、[bitmap オブジェクト](../../../sql-reference/functions/bitmap-functions.md) を返します。 - -```sql -groupBitmapXor(expr) -``` - -**引数** - -`expr` – 評価結果が `AggregateFunction(groupBitmap, UInt*)` 型となる式。 - -**戻り値** - -`UInt64` 型の値。 - -**例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapXor(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapXor(z)─┐ -│ 10 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapXorState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapXorState(z)))─┐ -│ [1,3,5,6,8,10,11,13,14,15] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md deleted file mode 100644 index 9df445ccac5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '数値の系列に対してビット単位の `OR` 演算を適用します。' -sidebar_position: 152 -slug: /sql-reference/aggregate-functions/reference/groupbitor -title: 'groupBitOr' -doc_type: 'reference' ---- - -# groupBitOr {#groupbitor} - -数値の系列にビット単位の `OR` 演算を適用します。 - -```sql -groupBitOr(expr) -``` - -**引数** - -`expr` – 結果が `UInt*` または `Int*` 型となる式。 - -**戻り値** - -`UInt*` または `Int*` 型の値。 - -**例** - -テストデータ: - -```text -2進数 10進数 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -クエリ: - -```sql -SELECT groupBitOr(num) FROM t -``` - -ここで、`num` はテストデータが入っている列です。 - -結果: - -```text -2進数 10進数 -01111101 = 125 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md deleted file mode 100644 index fffd76a01d0..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '一連の数値に対してビット単位の `XOR` を適用します。' -sidebar_position: 153 -slug: /sql-reference/aggregate-functions/reference/groupbitxor -title: 'groupBitXor' -doc_type: 'reference' ---- - -# groupBitXor {#groupbitxor} - -一連の数値に対してビット単位の `XOR` 演算を適用します。 - -```sql -groupBitXor(expr) -``` - -**引数** - -`expr` – 評価結果が `UInt*` または `Int*` 型となる式。 - -**戻り値** - -`UInt*` または `Int*` 型の値。 - -**例** - -テストデータ: - -```text -2進数 10進数 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -クエリ: - -```sql -SELECT groupBitXor(num) FROM t -``` - -ここで `num` はテストデータが入っている列です。 - -結果: - -```text -2進数 10進数 -01101000 = 104 -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md deleted file mode 100644 index 2115d36c577..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: '文字列のグループを連結して 1 つの文字列を生成します。必要に応じて区切り文字を挟み、要素数の上限を指定できます。' -sidebar_label: 'groupConcat' -sidebar_position: 363 -slug: /sql-reference/aggregate-functions/reference/groupconcat -title: 'groupConcat' -doc_type: 'reference' ---- - -文字列のグループを連結して 1 つの文字列を生成します。必要に応じて区切り文字を挟み、要素数の上限を指定できます。 - -**構文** - -```sql -groupConcat[(delimiter [, limit])](expression); -``` - -Alias: `group_concat` - -**引数** - -* `expression` — 連結する文字列を出力する式または列名。 -* `delimiter` — 連結された値を区切るために使用される[文字列](../../../sql-reference/data-types/string.md)。この引数は省略可能で、指定されていない場合は空文字列、またはパラメータで指定された区切り文字が既定値として使用されます。 - -**パラメータ** - -* `delimiter` — 連結された値を区切るために使用される[文字列](../../../sql-reference/data-types/string.md)。このパラメータは省略可能で、指定されていない場合は空文字列が既定値として使用されます。 -* `limit` — 連結する要素数の最大値を指定する正の[整数](../../../sql-reference/data-types/int-uint.md)。より多くの要素が存在する場合、超過分の要素は無視されます。このパラメータは省略可能です。 - -:::note -`limit` を指定せずに `delimiter` のみを指定する場合、`delimiter` は最初のパラメータでなければなりません。`delimiter` と `limit` の両方を指定する場合は、`delimiter` を `limit` より前に指定する必要があります。 - -また、パラメータと引数の両方で異なる区切り文字が指定された場合は、引数で指定された区切り文字のみが使用されます。 -::: - -**戻り値** - -* 列または式の値を連結した[文字列](../../../sql-reference/data-types/string.md)を返します。グループに要素が存在しない場合、または null 要素のみで構成されており、かつ関数で「null のみの値」の扱いが指定されていない場合、結果は値が null の Nullable 型の文字列になります。 - -**例** - -入力テーブル: - -```text -┌─id─┬─name─┐ -│ 1 │ John │ -│ 2 │ Jane │ -│ 3 │ Bob │ -└────┴──────┘ -``` - -1. 区切り文字なしでの基本的な使用方法: - -クエリ: - -```sql -SELECT groupConcat(Name) FROM Employees; -``` - -結果: - -```text -JohnJaneBob -``` - -これは、すべての name を区切りなしで 1 つの連続した文字列に連結します。 - -2. 区切り文字としてカンマを使用する場合: - -クエリ: - -```sql -SELECT groupConcat(', ')(Name) FROM Employees; -``` - -または - -```sql -SELECT groupConcat(Name, ', ') FROM Employees; -``` - -結果: - -```text -John, Jane, Bob -``` - -この出力は、名前がカンマとその後のスペースで区切られていることを示しています。 - -3. 連結する要素数の制限 - -クエリ: - -```sql -SELECT groupConcat(', ', 2)(Name) FROM Employees; -``` - -結果: - -```text -John, Jane -``` - -このクエリでは、テーブルにさらに多くの名前が存在していても、出力は先頭2件の名前に制限されます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md deleted file mode 100644 index e595128e633..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: '異なる引数値から配列を作成します。' -sidebar_position: 154 -slug: /sql-reference/aggregate-functions/reference/groupuniqarray -title: 'groupUniqArray' -doc_type: 'reference' ---- - -# groupUniqArray {#groupuniqarray} - -構文: `groupUniqArray(x)` または `groupUniqArray(max_size)(x)` - -異なる引数の値から配列を作成します。メモリ消費量は [uniqExact](../../../sql-reference/aggregate-functions/reference/uniqexact.md) 関数と同じです。 - -2つ目の形式(`max_size` パラメータを指定するもの)は、結果配列のサイズを `max_size` 要素に制限します。 -例えば、`groupUniqArray(1)(x)` は `[any(x)]` と同等です。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md deleted file mode 100644 index f65bd120aa1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'すべての範囲(数値軸上の区間)の和集合の全長を計算します。' -sidebar_label: 'intervalLengthSum' -sidebar_position: 155 -slug: /sql-reference/aggregate-functions/reference/intervalLengthSum -title: 'intervalLengthSum' -doc_type: 'reference' ---- - -すべての範囲(数値軸上の区間)の和集合の全長を計算します。 - -**構文** - -```sql -intervalLengthSum(start, end) -``` - -**引数** - -* `start` — 区間の開始値。[Int32](/sql-reference/data-types/int-uint#integer-ranges)、[Int64](/sql-reference/data-types/int-uint#integer-ranges)、[UInt32](/sql-reference/data-types/int-uint#integer-ranges)、[UInt64](/sql-reference/data-types/int-uint#integer-ranges)、[Float32](/sql-reference/data-types/float)、[Float64](/sql-reference/data-types/float)、[DateTime](/sql-reference/data-types/datetime) または [Date](/sql-reference/data-types/date)。 -* `end` — 区間の終了値。[Int32](/sql-reference/data-types/int-uint#integer-ranges)、[Int64](/sql-reference/data-types/int-uint#integer-ranges)、[UInt32](/sql-reference/data-types/int-uint#integer-ranges)、[UInt64](/sql-reference/data-types/int-uint#integer-ranges)、[Float32](/sql-reference/data-types/float)、[Float64](/sql-reference/data-types/float)、[DateTime](/sql-reference/data-types/datetime) または [Date](/sql-reference/data-types/date)。 - -:::note -引数は同一のデータ型でなければなりません。そうでない場合は、例外がスローされます。 -::: - -**戻り値** - -* すべての範囲(数直線上の区間)の和集合の全体の長さ。引数の型に応じて、戻り値は [UInt64](/sql-reference/data-types/int-uint#integer-ranges) 型または [Float64](/sql-reference/data-types/float) 型になります。 - -**例** - -1. 入力テーブル: - -```text -┌─id─┬─start─┬─end─┐ -│ a │ 1.1 │ 2.9 │ -│ a │ 2.5 │ 3.2 │ -│ a │ 4 │ 5 │ -└────┴───────┴─────┘ -``` - -この例では、`Float32` 型の引数が使用されています。関数は `Float64` 型の値を返します。 - -結果は、区間 `[1.1, 3.2]`(`[1.1, 2.9]` と `[2.5, 3.2]` の和集合)と `[4, 5]` の長さの合計です。 - -クエリ: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM fl_interval GROUP BY id ORDER BY id; -``` - -結果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 3.1 │ Float64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -2. 入力テーブル: - -```text -┌─id─┬───────────────start─┬─────────────────end─┐ -│ a │ 2020-01-01 01:12:30 │ 2020-01-01 02:10:10 │ -│ a │ 2020-01-01 02:05:30 │ 2020-01-01 02:50:31 │ -│ a │ 2020-01-01 03:11:22 │ 2020-01-01 03:23:31 │ -└────┴─────────────────────┴─────────────────────┘ -``` - -この例では、`DateTime` 型の引数を使用します。関数は秒単位の値を返します。 - -クエリ: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM dt_interval GROUP BY id ORDER BY id; -``` - -結果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 6610 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -3. 入力テーブル: - - -```text -┌─id─┬──────start─┬────────end─┐ -│ a │ 2020-01-01 │ 2020-01-04 │ -│ a │ 2020-01-12 │ 2020-01-18 │ -└────┴────────────┴────────────┘ -``` - -この例では、`Date` 型の引数が使用されています。関数は日単位の値を返します。 - -クエリ: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM date_interval GROUP BY id ORDER BY id; -``` - -結果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 9 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md deleted file mode 100644 index c65b42c4032..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: '2つの母集団から得られた標本に Kolmogorov-Smirnov 検定を適用します。' -sidebar_label: 'kolmogorovSmirnovTest' -sidebar_position: 156 -slug: /sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest -title: 'kolmogorovSmirnovTest' -doc_type: 'reference' ---- - -# kolmogorovSmirnovTest {#kolmogorovsmirnovtest} - -2つの母集団から得られた標本に対してコルモゴロフ–スミルノフ検定を適用します。 - -**構文** - -```sql -kolmogorovSmirnovTest([alternative, computation_method])(sample_data, sample_index) -``` - -両方のサンプルの値は `sample_data` カラムにあります。`sample_index` が 0 の場合、その行の値は第 1 集団からのサンプルに属します。そうでない場合は第 2 集団からのサンプルに属します。 -サンプルは連続な一次元の確率分布に属している必要があります。 - -**引数** - -* `sample_data` — サンプルデータ。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — サンプルインデックス。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `alternative` — 対立仮説。(省略可、デフォルト: `'two-sided'`。)[String](../../../sql-reference/data-types/string.md)。\ - F(x) および G(x) を、それぞれ第 1 および第 2 の分布の CDF とします。 - * `'two-sided'`\ - 帰無仮説は、サンプルが同じ分布に由来する、すなわちすべての x について `F(x) = G(x)` であるというものです。\ - 対立仮説は、分布が同一ではないというものです。 - * `'greater'`\ - 帰無仮説は、第 1 サンプルの値が第 2 サンプルの値よりも *確率的に小さい* というものです。\ - 例えば、第 1 分布の CDF が第 2 分布の CDF より上(したがって左側)に位置する場合です。\ - これは実質的に、すべての x について `F(x) >= G(x)` であることを意味します。この場合の対立仮説は、少なくとも 1 つの x について `F(x) < G(x)` であるというものです。 - * `'less'`\ - 帰無仮説は、第 1 サンプルの値が第 2 サンプルの値よりも *確率的に大きい* というものです。\ - 例えば、第 1 分布の CDF が第 2 分布の CDF より下(したがって右側)に位置する場合です。\ - これは実質的に、すべての x について `F(x) <= G(x)` であることを意味します。この場合の対立仮説は、少なくとも 1 つの x について `F(x) > G(x)` であるというものです。 -* `computation_method` — p 値を計算する際に使用される手法。(省略可、デフォルト: `'auto'`。)[String](../../../sql-reference/data-types/string.md)。 - * `'exact'` - 検定統計量の正確な確率分布を用いて計算を行います。小さなサンプル以外では計算コストが高く非効率です。 - * `'asymp'` (`'asymptotic'`) - 近似を用いて計算を行います。サンプルサイズが大きい場合、`'exact'` と漸近的な p 値は非常に近くなります。 - * `'auto'` - サンプル数の最大値が 10'000 未満の場合に `'exact'` 手法が使用されます。 - -**戻り値** - -2 要素からなる [Tuple](../../../sql-reference/data-types/tuple.md): - -* 計算された統計量。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された p 値。[Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -クエリ: - -```sql -SELECT kolmogorovSmirnovTest('less', 'exact')(value, num) -FROM -( - SELECT - randNormal(0, 10) AS value, - 0 AS num - FROM numbers(10000) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(10000) -) -``` - -結果: - -```text -┌─kolmogorovSmirnovTest('less', 'exact')(value, num)─┐ -│ (0.009899999999999996,0.37528595205132287) │ -└────────────────────────────────────────────────────┘ -``` - -Note: -P値が 0.05 より大きい(信頼水準 95% において)ため、帰無仮説は棄却されません。 - -Query: - -```sql -SELECT kolmogorovSmirnovTest('two-sided', 'exact')(value, num) -FROM -( - SELECT - randStudentT(10) AS value, - 0 AS num - FROM numbers(100) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(100) -) -``` - -結果: - -```text -┌─kolmogorovSmirnovTest('two-sided', 'exact')(value, num)─┐ -│ (0.4100000000000002,6.61735760482795e-8) │ -└─────────────────────────────────────────────────────────┘ -``` - -注記: -P 値が 0.05 未満(信頼水準 95% において)であるため、帰無仮説は棄却されます。 - -**関連項目** - -* [Kolmogorov-Smirnov test](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test) \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md deleted file mode 100644 index d4c2c08de55..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: '系列の尖度を計算します。' -sidebar_position: 157 -slug: /sql-reference/aggregate-functions/reference/kurtpop -title: 'kurtPop' -doc_type: 'reference' ---- - -# kurtPop {#kurtpop} - -系列の[尖度 (kurtosis)](https://en.wikipedia.org/wiki/Kurtosis)を計算します。 - -```sql -kurtPop(expr) -``` - -**引数** - -`expr` — 数値を返す[式](/sql-reference/syntax#expressions)。 - -**戻り値** - -与えられた分布の尖度。型 — [Float64](../../../sql-reference/data-types/float.md) - -**例** - -```sql -SELECT kurtPop(value) FROM series_with_value_column; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md deleted file mode 100644 index 1f0dd99f6c8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'シーケンスの標本尖度を計算します。' -sidebar_position: 158 -slug: /sql-reference/aggregate-functions/reference/kurtsamp -title: 'kurtSamp' -doc_type: 'reference' ---- - -# kurtSamp {#kurtsamp} - -シーケンスの[標本尖度](https://en.wikipedia.org/wiki/Kurtosis)を計算します。 - -渡された値がその確率変数の標本を構成している場合、その確率変数の尖度の不偏推定量を表します。 - -```sql -kurtSamp(expr) -``` - -**引数** - -`expr` — 数値を返す[式](/sql-reference/syntax#expressions)。 - -**戻り値** - -与えられた分布の尖度(kurtosis)。型 — [Float64](../../../sql-reference/data-types/float.md)。`n <= 1`(`n` は標本のサイズ)の場合、関数は `nan` を返します。 - -**例** - -```sql -SELECT kurtSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md deleted file mode 100644 index 71c64074263..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '入力データに対して Largest-Triangle-Three-Buckets アルゴリズムを適用します。' -sidebar_label: 'largestTriangleThreeBuckets' -sidebar_position: 159 -slug: /sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets -title: 'largestTriangleThreeBuckets' -doc_type: 'reference' ---- - -# largestTriangleThreeBuckets {#largesttrianglethreebuckets} - -入力データに対して [Largest-Triangle-Three-Buckets](https://skemman.is/bitstream/1946/15343/3/SS_MSthesis.pdf) アルゴリズムを適用します。 -このアルゴリズムは、可視化のために時系列データをダウンサンプリングする際に使用されます。x 座標でソートされた系列を前提として設計されています。 -ソート済み系列を複数のバケットに分割し、各バケット内で最大の三角形を見つけることで動作します。バケット数は、ダウンサンプリング後の系列に含まれる点の数と同じです。 -この関数は、まずデータを `x` でソートし、その後ソート済みデータに対してダウンサンプリングアルゴリズムを適用します。 - -**構文** - -```sql -largestTriangleThreeBuckets(n)(x, y) -``` - -エイリアス: `lttb`. - -**引数** - -* `x` — x 座標。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、[Decimal](../../../sql-reference/data-types/decimal.md)、[Date](../../../sql-reference/data-types/date.md)、[Date32](../../../sql-reference/data-types/date32.md)、[DateTime](../../../sql-reference/data-types/datetime.md)、[DateTime64](../../../sql-reference/data-types/datetime64.md)。 -* `y` — y 座標。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、[Decimal](../../../sql-reference/data-types/decimal.md)、[Date](../../../sql-reference/data-types/date.md)、[Date32](../../../sql-reference/data-types/date32.md)、[DateTime](../../../sql-reference/data-types/datetime.md)、[DateTime64](../../../sql-reference/data-types/datetime64.md)。 - -与えられた系列中の NaN は無視され、これらの NaN 値は分析から除外されます。これにより、この関数は有効な数値データのみに対して動作します。 - -**パラメータ** - -* `n` — 結果の系列に含まれる点の数。[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返される値** - -2 要素を持つ [Tuple](../../../sql-reference/data-types/tuple.md) の [Array](../../../sql-reference/data-types/array.md): - -**例** - -入力テーブル: - -```text -┌─────x───────┬───────y──────┐ -│ 1.000000000 │ 10.000000000 │ -│ 2.000000000 │ 20.000000000 │ -│ 3.000000000 │ 15.000000000 │ -│ 8.000000000 │ 60.000000000 │ -│ 9.000000000 │ 55.000000000 │ -│ 10.00000000 │ 70.000000000 │ -│ 4.000000000 │ 30.000000000 │ -│ 5.000000000 │ 40.000000000 │ -│ 6.000000000 │ 35.000000000 │ -│ 7.000000000 │ 50.000000000 │ -└─────────────┴──────────────┘ -``` - -クエリ: - -```sql -SELECT largestTriangleThreeBuckets(4)(x, y) FROM largestTriangleThreeBuckets_test; -``` - -結果: - -```text -┌────────largestTriangleThreeBuckets(4)(x, y)───────────┐ -│ [(1,10),(3,15),(9,55),(10,70)] │ -└───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md deleted file mode 100644 index 6a48ca43f2f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: '最後に出現した値を選択します。`anyLast` と類似していますが、NULL を許容します。' -sidebar_position: 160 -slug: /sql-reference/aggregate-functions/reference/last_value -title: 'last_value' -doc_type: 'reference' ---- - -# last_value {#last_value} - -`anyLast` と同様に、最後に出現した値を選択しますが、NULL も許容します。 -主に [Window Functions](../../window-functions/index.md)(ウィンドウ関数)と組み合わせて使用します。 -Window Functions を使用しない場合、入力ストリームに順序付けがされていないと、結果はランダムになります。 - -## 使用例 {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null) -``` - -### 例 1 {#example1} - -`NULL` 値はデフォルトで無視されます。 - -```sql -SELECT last_value(b) FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### 例 2 {#example2} - -NULL 値は無視されます。 - -```sql -SELECT last_value(b) ignore nulls FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### 例 3 {#example3} - -NULL 値が許可されます。 - -```sql -SELECT last_value(b) respect nulls FROM test_data -``` - -```text -┌─last_value_respect_nulls(b)─┐ -│ ᴺᵁᴸᴸ │ -└─────────────────────────────┘ -``` - -### 例 4 {#example4} - -`ORDER BY` を含むサブクエリで結果を安定化。 - -```sql -SELECT - last_value_respect_nulls(b), - last_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─last_value_respect_nulls(b)─┬─last_value(b)─┐ -│ ᴺᵁᴸᴸ │ 5 │ -└─────────────────────────────┴───────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md deleted file mode 100644 index 533e1125840..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '2つの母集団からの標本に対してマン–ホイットニーのU検定を適用します。' -sidebar_label: 'mannWhitneyUTest' -sidebar_position: 161 -slug: /sql-reference/aggregate-functions/reference/mannwhitneyutest -title: 'mannWhitneyUTest' -doc_type: 'reference' ---- - -# mannWhitneyUTest {#mannwhitneyutest} - -2つの母集団からのサンプルに対して、Mann-Whitney の順位検定を適用します。 - -**構文** - -```sql -mannWhitneyUTest[(alternative[, continuity_correction])](sample_data, sample_index) -``` - -両方のサンプルの値は `sample_data` 列にあります。`sample_index` が 0 の場合、その行の値は第 1 集団のサンプルに属します。そうでない場合は第 2 集団のサンプルに属します。 -帰無仮説は、2 つの集団が確率的に等しいというものです。片側検定も実行できます。この検定では、データが正規分布に従うという仮定は置きません。 - -**引数** - -* `sample_data` — サンプルデータ。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — サンプルインデックス。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `alternative` — 対立仮説。(省略可、デフォルト: `'two-sided'`。)[String](../../../sql-reference/data-types/string.md)。 - * `'two-sided'`; - * `'greater'`; - * `'less'`。 -* `continuity_correction` — 0 以外の場合、p 値の正規近似において連続性補正を適用します。(省略可、デフォルト: 1。)[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返される値** - -2 要素の [Tuple](../../../sql-reference/data-types/tuple.md): - -* 計算された U 統計量。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された p 値。[Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -入力テーブル: - -```text -┌─sample_data─┬─sample_index─┐ -│ 10 │ 0 │ -│ 11 │ 0 │ -│ 12 │ 0 │ -│ 1 │ 1 │ -│ 2 │ 1 │ -│ 3 │ 1 │ -└─────────────┴──────────────┘ -``` - -クエリ: - -```sql -SELECT mannWhitneyUTest('greater')(sample_data, sample_index) FROM mww_ttest; -``` - -結果: - -```text -┌─mannWhitneyUTest('greater')(sample_data, sample_index)─┐ -│ (9,0.04042779918503192) │ -└────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [マン・ホイットニーのU検定](https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test) -* [確率順序](https://en.wikipedia.org/wiki/Stochastic_ordering) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md deleted file mode 100644 index 288fcb097b1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: '値のグループに対して最大値を計算する集約関数。' -sidebar_position: 162 -slug: /sql-reference/aggregate-functions/reference/max -title: 'max' -doc_type: 'reference' ---- - -値のグループに対して最大値を計算する集約関数。 - -例: - -```sql -SELECT max(salary) FROM employees; -``` - -```sql -SELECT department, max(salary) FROM employees GROUP BY department; -``` - -非集約関数で 2 つの値のうち大きい方を選択したい場合は、`greatest` を使用してください。 - -```sql -SELECT greatest(a, b) FROM table; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md deleted file mode 100644 index b119fa26b2a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: '区間のグループにおいて、区間同士が互いに交差する回数の最大値を計算する集約関数(ただし、すべての区間が少なくとも 1 回は交差している場合)。' -sidebar_position: 163 -slug: /sql-reference/aggregate-functions/reference/maxintersections -title: 'maxIntersections' -doc_type: 'reference' ---- - -# maxIntersections {#maxintersections} - -区間のグループ内で、すべての区間が少なくとも一度は互いに交差する場合に、その交差回数の最大値を計算する集約関数です。 - -構文は次のとおりです。 - -```sql -maxIntersections(start_column, end_column) -``` - -**引数** - -* `start_column` – 各インターバルの開始を表す数値カラム。`start_column` が `NULL` または 0 の場合、そのインターバルはスキップされます。 - -* `end_column` - 各インターバルの終了を表す数値カラム。`end_column` が `NULL` または 0 の場合、そのインターバルはスキップされます。 - -**戻り値** - -交差するインターバルの最大数を返します。 - -**例** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -間隔は次のようになります。 - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -これらの区間のうち 3 つは共通の値を取ります(値は `4` ですが、どの値が共通かは重要ではなく、交差している区間の数を数えています)。区間 `(1,3)` と `(3,7)` は端点を共有していますが、`maxIntersections` 関数では交差しているとはみなされません。 - -```sql -SELECT maxIntersections(start, end) FROM my_events; -``` - -レスポンス: - -```response -3 -``` - -最大区間が複数回存在する場合は、[`maxIntersectionsPosition` 関数](./maxintersectionsposition.md)を使用して、その発生回数と位置を特定できます。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md deleted file mode 100644 index 4f0e76495ac..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: 'maxIntersections 関数の結果が出現する位置を計算する集約関数。' -sidebar_position: 164 -slug: /sql-reference/aggregate-functions/reference/maxintersectionsposition -title: 'maxIntersectionsPosition' -doc_type: 'reference' ---- - -# maxIntersectionsPosition {#maxintersectionsposition} - -[`maxIntersections` 関数](./maxintersections.md) が出現する位置を計算する集計関数です。 - -構文は次のとおりです。 - -```sql -maxIntersectionsPosition(start_column, end_column) -``` - -**引数** - -* `start_column` – 各インターバルの開始を表す数値型の列。`start_column` が `NULL` または 0 の場合、そのインターバルはスキップされます。 - -* `end_column` - 各インターバルの終了を表す数値型の列。`end_column` が `NULL` または 0 の場合、そのインターバルはスキップされます。 - -**戻り値** - -最大数のインターバルが互いに重なり合う区間の開始位置を返します。 - -**例** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -インターバルは以下のようになります。 - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -これらの区間のうち 3 つはいずれも値 4 を含んでおり、それが 2 番目の区間から始まっていることに注目してください。 - -```sql -SELECT maxIntersectionsPosition(start, end) FROM my_events; -``` - -レスポンス: - -```response -2 -``` - -言い換えると、`(1,6)` の行は互いに交差する 3 つの区間が始まる位置であり、3 は交差する区間の最大本数です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md deleted file mode 100644 index 0c05460a64f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '`key` 配列で指定されたキーごとに、`value` 配列の最大値を計算します。' -sidebar_position: 165 -slug: /sql-reference/aggregate-functions/reference/maxmap -title: 'maxMap' -doc_type: 'reference' ---- - -# maxMap {#maxmap} - -`key` 配列で指定されたキーに基づいて、`value` 配列から最大値を計算します。 - -**構文** - -```sql -maxMap(key, value) -``` - -または - -```sql -maxMap(Tuple(key, value)) -``` - -Alias: `maxMappedArrays` - -:::note - -* キー配列と値配列のタプルを渡すことは、キー配列と値配列の 2 つを渡すことと同じです。 -* 合計される各行について、`key` と `value` の要素数は同じでなければなりません。 - ::: - -**パラメータ** - -* `key` — キーの配列。[Array](../../data-types/array.md)。 -* `value` — 値の配列。[Array](../../data-types/array.md)。 - -**戻り値** - -* 2 つの配列からなるタプルを返します。ソート順のキー配列と、それぞれのキーに対応して計算された値の配列です。[Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md))。 - -**例** - -クエリ: - -```sql -SELECT maxMap(a, b) -FROM VALUES('a Array(Char), b Array(Int64)', (['x', 'y'], [2, 2]), (['y', 'z'], [3, 1])) -``` - -結果: - -```text -┌─maxMap(a, b)───────────┐ -│ [['x','y','z'],[2,3,1]]│ -└────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md deleted file mode 100644 index 6b8e9325419..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -description: '2つの母集団からの標本に対して平均のZ検定を適用します。' -sidebar_label: 'meanZTest' -sidebar_position: 166 -slug: /sql-reference/aggregate-functions/reference/meanztest -title: 'meanZTest' -doc_type: 'reference' ---- - -# meanZTest {#meanztest} - -2つの母集団からの標本に対して、平均に対するZ検定を適用します。 - -**構文** - -```sql -meanZTest(population_variance_x, population_variance_y, confidence_level)(sample_data, sample_index) -``` - -両方のサンプルの値は `sample_data` 列にあります。`sample_index` が 0 の場合、その行の値は第 1 母集団からのサンプルに属します。0 でない場合は第 2 母集団からのサンプルに属します。 -帰無仮説は、母集団の平均が等しいというものです。正規分布を仮定します。母集団は不等分散であってもよく、分散は既知と仮定します。 - -**引数** - -* `sample_data` — サンプルデータ。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — サンプルのインデックス。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `population_variance_x` — 母集団 x の分散。[Float](../../../sql-reference/data-types/float.md)。 -* `population_variance_y` — 母集団 y の分散。[Float](../../../sql-reference/data-types/float.md)。 -* `confidence_level` — 信頼区間を計算するための信頼水準。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -4 要素からなる [Tuple](../../../sql-reference/data-types/tuple.md): - -* 計算された t 統計量。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された p 値。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された信頼区間の下限。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された信頼区間の上限。[Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -入力テーブル: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.9 │ 0 │ -│ 22.1 │ 0 │ -│ 18.9 │ 1 │ -│ 19 │ 1 │ -│ 20.3 │ 1 │ -└─────────────┴──────────────┘ -``` - -クエリ: - -```sql -SELECT meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index) FROM mean_ztest -``` - -結果: - -```text -┌─meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index)────────────────────────────┐ -│ (3.2841296025548123,0.0010229786769086013,0.8198428246768334,3.2468238419898365) │ -└──────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md deleted file mode 100644 index 30051fe0ad8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '`median*` 関数は、対応する `quantile*` 関数のエイリアスです。数値データサンプルの中央値を計算します。' -sidebar_position: 167 -slug: /sql-reference/aggregate-functions/reference/median -title: 'median' -doc_type: 'reference' ---- - -# median {#median} - -`median*` 関数は、対応する `quantile*` 関数のエイリアスです。数値データのサンプルに対する中央値を計算します。 - -関数: - -* `median` — [quantile](/sql-reference/aggregate-functions/reference/quantile) のエイリアス。 -* `medianDeterministic` — [quantileDeterministic](/sql-reference/aggregate-functions/reference/quantiledeterministic) のエイリアス。 -* `medianExact` — [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) のエイリアス。 -* `medianExactWeighted` — [quantileExactWeighted](/sql-reference/aggregate-functions/reference/quantileexactweighted) のエイリアス。 -* `medianTiming` — [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) のエイリアス。 -* `medianTimingWeighted` — [quantileTimingWeighted](/sql-reference/aggregate-functions/reference/quantiletimingweighted) のエイリアス。 -* `medianTDigest` — [quantileTDigest](/sql-reference/aggregate-functions/reference/quantiletdigest) のエイリアス。 -* `medianTDigestWeighted` — [quantileTDigestWeighted](/sql-reference/aggregate-functions/reference/quantiletdigestweighted) のエイリアス。 -* `medianBFloat16` — [quantileBFloat16](/sql-reference/aggregate-functions/reference/quantilebfloat16) のエイリアス。 -* `medianDD` — [quantileDD](/sql-reference/aggregate-functions/reference/quantileddsketch) のエイリアス。 - -**例** - -入力テーブル: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -クエリ: - -```sql -SELECT medianDeterministic(val, 1) FROM t; -``` - -結果: - -```text -┌─medianDeterministic(val, 1)─┐ -│ 1.5 │ -└─────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md deleted file mode 100644 index 427bdc76d82..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: '値のグループの最小値を計算する集約関数。' -sidebar_position: 168 -slug: /sql-reference/aggregate-functions/reference/min -title: 'min' -doc_type: 'reference' ---- - -値のグループの最小値を計算する集約関数。 - -例: - -```sql -SELECT min(salary) FROM employees; -``` - -```sql -SELECT department, min(salary) FROM employees GROUP BY department; -``` - -非集約関数で 2 つの値の小さい方を選択する必要がある場合は、`least` を参照してください。 - -```sql -SELECT least(a, b) FROM table; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md deleted file mode 100644 index 3a8710d5353..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '`key` 配列で指定されたキーごとに、`value` 配列から最小値を計算します。' -sidebar_position: 169 -slug: /sql-reference/aggregate-functions/reference/minmap -title: 'minMap' -doc_type: 'reference' ---- - -# minMap {#minmap} - -`key` 配列で指定されたキーに基づいて、`value` 配列の最小値を計算します。 - -**構文** - -```sql -`minMap(key, value)` -``` - -または - -```sql -minMap(Tuple(key, value)) -``` - -Alias: `minMappedArrays` - -:::note - -* キー配列と値配列のタプルを渡すことは、キーの配列と値の配列をそれぞれ渡すことと同等です。 -* 集計対象となる各行について、`key` と `value` の要素数は同じでなければなりません。 - ::: - -**パラメータ** - -* `key` — キーの配列。[Array](../../data-types/array.md)。 -* `value` — 値の配列。[Array](../../data-types/array.md)。 - -**返り値** - -* 2 つの配列からなるタプルを返します。キーはソート済みで、値は対応するキーごとに計算されたものです。[Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md))。 - -**例** - -クエリ: - -```sql -SELECT minMap(a, b) -FROM VALUES('a Array(Int32), b Array(Int64)', ([1, 2], [2, 2]), ([2, 3], [1, 1])) -``` - -結果: - -```text -┌─minMap(a, b)──────┐ -│ ([1,2,3],[2,1,1]) │ -└───────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md deleted file mode 100644 index a2b9adb3d45..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '数値データ列の近似的な分位数を計算します。' -sidebar_position: 170 -slug: /sql-reference/aggregate-functions/reference/quantile -title: 'quantile' -doc_type: 'reference' ---- - -# quantile {#quantile} - -数値データシーケンスの近似的な[分位点](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -この関数は、最大サイズ 8192 の[リザーバサンプリング](https://en.wikipedia.org/wiki/Reservoir_sampling)と乱数生成器を用いてサンプリングを行います。結果は非決定的です。厳密な分位点を取得するには、[quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 関数を使用してください。 - -1 つのクエリ内で、異なるレベルを持つ複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、クエリは本来の性能より非効率になります)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -数値シーケンスが空の場合、`quantile` は NaN を返しますが、`quantile*` の各バリアントは、そのバリアントに応じて NaN またはシーケンス型のデフォルト値のいずれかを返すことに注意してください。 - -**構文** - -```sql -quantile(level)(expr) -``` - -別名: `median`。 - -**引数** - -* `level` — 分位点のレベル。省略可能なパラメーター。0 から 1 の間の定数浮動小数点数。`level` の値は `[0.01, 0.99]` の範囲で使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 数値[データ型](/sql-reference/data-types)、[Date](/sql-reference/data-types/date) または [DateTime](/sql-reference/data-types/datetime) を結果とする、列値に対する式。 - -**戻り値** - -* 指定されたレベルの近似的な分位点。 - -型: - -* 数値データ型入力に対しては [Float64](/sql-reference/data-types/float)。 -* 入力値が `Date` 型の場合は [Date](/sql-reference/data-types/date)。 -* 入力値が `DateTime` 型の場合は [DateTime](/sql-reference/data-types/datetime)。 - -**例** - -入力テーブル: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -クエリ: - -```sql -SELECT quantile(val) FROM t -``` - -結果: - -```text -┌─quantile(val)─┐ -│ 1.5 │ -└───────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md deleted file mode 100644 index f09a19d58a1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'Greenwald-Khanna アルゴリズムを使用して、数値データ系列の分位数を計算します。' -sidebar_position: 175 -slug: /sql-reference/aggregate-functions/reference/quantileGK -title: 'quantileGK' -doc_type: 'reference' ---- - -# quantileGK {#quantilegk} - -数値データ系列の[分位数](https://en.wikipedia.org/wiki/Quantile)を、[Greenwald-Khanna](http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf) アルゴリズムを用いて計算します。Greenwald-Khanna アルゴリズムは、データストリームに対する分位数を高効率に計算するためのアルゴリズムです。2001 年に Michael Greenwald と Sanjeev Khanna によって提案されました。大量のデータストリームに対してリアルタイムに正確な分位数を計算する必要があるデータベースやビッグデータシステムで広く利用されています。このアルゴリズムは非常に効率的で、必要な空間計算量は O(log n)、1 要素あたりの時間計算量は O(log log n) です(ここで n は入力のサイズ)。また、精度も非常に高く、高い確率で分位数の近似値を提供します。 - -`quantileGK` は、近似分位数結果の精度をユーザーが制御できる点で、ClickHouse の他の分位数関数とは異なります。 - -**構文** - -```sql -quantileGK(accuracy, level)(expr) -``` - -Alias: `medianGK`. - -**引数** - -* `accuracy` — 分位数の精度を表します。正の整数の定数です。値が大きいほど誤差は小さくなります。例えば、`accuracy` 引数を 100 に設定した場合、計算される分位数の誤差は高い確率で 1% 以下になります。計算される分位数の精度とアルゴリズムの計算量との間にはトレードオフがあります。`accuracy` を大きくすると分位数を正確に計算するためにより多くのメモリと計算リソースが必要になりますが、`accuracy` を小さくすると計算はより高速かつメモリ効率が良くなる一方で、精度はやや低くなります。 - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 の範囲の浮動小数点数の定数です。デフォルト値: 0.5。`level=0.5` の場合、この関数は [median](https://en.wikipedia.org/wiki/Median) を計算します。 - -* `expr` — 数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) のいずれかを結果とする、列の値に対する式。 - -**戻り値** - -* 指定されたレベルと精度の分位数。 - -Type: - -* 数値データ型入力に対しては [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -```sql -SELECT quantileGK(1, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1, 0.25)(plus(number, 1))─┐ -│ 1 │ -└──────────────────────────────────────┘ - -SELECT quantileGK(10, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(10, 0.25)(plus(number, 1))─┐ -│ 156 │ -└───────────────────────────────────────┘ - -SELECT quantileGK(100, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(100, 0.25)(plus(number, 1))─┐ -│ 251 │ -└────────────────────────────────────────┘ - -SELECT quantileGK(1000, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1000, 0.25)(plus(number, 1))─┐ -│ 249 │ -└─────────────────────────────────────────┘ -``` - -**関連項目** - -* [median]/sql-reference/aggregate-functions/reference/median -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md deleted file mode 100644 index ab188c7030d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'bfloat16 数値で構成されるサンプルの近似分位数を計算します。' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantilebfloat16 -title: 'quantileBFloat16' -doc_type: 'reference' ---- - -# quantileBFloat16Weighted {#quantilebfloat16weighted} - -`quantileBFloat16` と同様ですが、系列内の各要素の重みを考慮します。 - -[bfloat16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format) 形式の数値からなる標本について、おおよその [分位数](https://en.wikipedia.org/wiki/Quantile) を計算します。`bfloat16` は、1 ビットの符号ビット、8 ビットの指数部、7 ビットの仮数部を持つ浮動小数点データ型です。 -この関数は、入力値を 32 ビット浮動小数点数に変換し、その最上位 16 ビットを取得します。次に `bfloat16` の分位数値を計算し、ゼロビットを付加することで結果を 64 ビット浮動小数点数に変換します。 -この関数は高速な分位数推定器であり、相対誤差は 0.390625% を超えません。 - -**構文** - -```sql -quantileBFloat16[(level)](expr) -``` - -Alias: `medianBFloat16` - -**引数** - -* `expr` — 数値データを持つ列。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)。 - -**パラメータ** - -* `level` — 分位数のレベル。省略可能。指定可能な値の範囲は 0 から 1。デフォルト値: 0.5。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -* 指定したレベルの近似分位数。 - -型: [Float64](/sql-reference/data-types/float). - -**例** - -入力テーブルには整数列と浮動小数点数列があります。 - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -0.75 分位数(第3四分位数)を計算するクエリ: - -```sql -SELECT quantileBFloat16(0.75)(a), quantileBFloat16(0.75)(b) FROM example_table; -``` - -結果: - -```text -┌─quantileBFloat16(0.75)(a)─┬─quantileBFloat16(0.75)(b)─┐ -│ 3 │ 1 │ -└───────────────────────────┴───────────────────────────┘ -``` - -この例では、`bfloat16` へ変換すると、すべての浮動小数点値は 1.0 に切り捨てられることに注意してください。 - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md deleted file mode 100644 index c37d4de1b8a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: '相対誤差保証つきでサンプルの近似分位数を計算します。' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantileddsketch -title: 'quantileDD' -doc_type: 'reference' ---- - -相対誤差保証つきでサンプルの近似[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。[DD](https://www.vldb.org/pvldb/vol12/p2195-masson.pdf) を構築することで実現します。 - -**構文** - -```sql -quantileDD(relative_accuracy, [level])(expr) -``` - -**引数** - -* `expr` — 数値データを含む列。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)。 - -**パラメータ** - -* `relative_accuracy` — 分位数の相対精度。取りうる値は 0 から 1 の範囲。[Float](../../../sql-reference/data-types/float.md)。スケッチのサイズは、データの範囲と相対精度に依存します。範囲が大きく相対精度が小さいほど、スケッチは大きくなります。スケッチのおおよそのメモリ使用量は `log(max_value/min_value)/relative_accuracy` です。推奨値は 0.001 以上です。 - -* `level` — 分位数レベル。省略可能。取りうる値は 0 から 1 の範囲。デフォルト値: 0.5。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -* 指定されたレベルの近似分位数。 - -型: [Float64](/sql-reference/data-types/float). - -**例** - -入力テーブルには、整数型列と浮動小数点数型列があります。 - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -0.75 分位数(第 3 四分位数)を計算するためのクエリ: - -```sql -SELECT quantileDD(0.01, 0.75)(a), quantileDD(0.01, 0.75)(b) FROM example_table; -``` - -結果: - -```text -┌─quantileDD(0.01, 0.75)(a)─┬─quantileDD(0.01, 0.75)(b)─┐ -│ 2.974233423476717 │ 1.01 │ -└─────────────────────────────────┴─────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md deleted file mode 100644 index a278457767b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '数値データの並びに対する近似分位数を計算します。' -sidebar_position: 172 -slug: /sql-reference/aggregate-functions/reference/quantiledeterministic -title: 'quantileDeterministic' -doc_type: 'reference' ---- - -# quantileDeterministic {#quantiledeterministic} - -数値データ系列の概算[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -この関数は、最大 8192 のリザーバーサイズを持つ[リザーバサンプリング](https://en.wikipedia.org/wiki/Reservoir_sampling)と決定論的なサンプリングアルゴリズムを適用します。結果は決定論的です。厳密な分位数を取得するには、[quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 関数を使用します。 - -1 つのクエリ内で異なるレベルの複数の `quantile*` 関数を使用する場合、内部状態はマージされません(つまり、そのクエリは本来よりも非効率になります)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用します。 - -**構文** - -```sql -quantileDeterministic(level)(expr, determinator) -``` - -別名: `medianDeterministic`. - -**引数** - -* `level` — 分位点のレベル。省略可能な引数。0 から 1 までの定数の浮動小数点数。`level` の値として `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は [median](https://en.wikipedia.org/wiki/Median) を計算します。 -* `expr` — 列値に対する式で、その結果が数値の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) となるもの。 -* `determinator` — サンプリング結果を決定的にするために、リザーバサンプリングアルゴリズムで乱数生成器の代わりに、そのハッシュ値が使用される数値。`determinator` としては、ユーザー ID やイベント ID など、任意の決定的な正の数を使用できます。同じ `determinator` の値があまりに頻繁に出現する場合、この関数は正しく動作しません。 - -**返される値** - -* 指定されたレベルの近似分位点。 - -型: - -* 数値データ型の入力に対しては [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -入力テーブル: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -クエリ: - -```sql -SELECT quantileDeterministic(val, 1) FROM t -``` - -結果: - -```text -┌─quantileDeterministic(val, 1)─┐ -│ 1.5 │ -└───────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md deleted file mode 100644 index b0db250764a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md +++ /dev/null @@ -1,292 +0,0 @@ ---- -description: 'quantileExact、quantileExactLow、quantileExactHigh、quantileExactExclusive、quantileExactInclusive 関数' -sidebar_position: 173 -slug: /sql-reference/aggregate-functions/reference/quantileexact -title: 'quantileExact 関数' -doc_type: 'reference' ---- - -# quantileExact 関数 {#quantileexact-functions} - -## quantileExact {#quantileexact} - -数値データ列の[分位点 (quantile)](https://en.wikipedia.org/wiki/Quantile) を厳密に計算します。 - -厳密な値を得るために、渡されたすべての値を配列にまとめ、その後で部分的にソートします。したがって、この関数は `O(n)` のメモリを消費します。ここで `n` は渡された値の個数です。ただし、値の個数が少ない場合には、この関数は非常に効率的です。 - -1つのクエリ内で異なるレベルの複数の `quantile*` 関数を使用する場合、内部状態はマージされません(つまり、そのクエリは本来よりも非効率に動作します)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileExact(level)(expr) -``` - -Alias: `medianExact`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメーター。0 から 1 の間の定数の浮動小数点数です。`level` の値には `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値は 0.5 です。`level=0.5` のとき、この関数は [中央値](https://en.wikipedia.org/wiki/Median) を計算します。 -* `expr` — 列の値に対する式で、その結果が数値の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md)、または [DateTime](../../../sql-reference/data-types/datetime.md) になります。 - -**戻り値** - -* 指定したレベルの分位数。 - -型: - -* 数値データ型に対しては、出力形式は入力形式と同じになります。例: - -```sql - -SELECT - toTypeName(quantileExact(number)) AS `quantile`, - toTypeName(quantileExact(number::Int32)) AS `quantile_int32`, - toTypeName(quantileExact(number::Float32)) AS `quantile_float32`, - toTypeName(quantileExact(number::Float64)) AS `quantile_float64`, - toTypeName(quantileExact(number::Int64)) AS `quantile_int64` -FROM numbers(1) - ┌─quantile─┬─quantile_int32─┬─quantile_float32─┬─quantile_float64─┬─quantile_int64─┐ -1. │ UInt64 │ Int32 │ Float32 │ Float64 │ Int64 │ - └──────────┴────────────────┴──────────────────┴──────────────────┴────────────────┘ - -1行が結果セットに含まれています。経過時間: 0.002秒。 -``` - -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantileExact(number) FROM numbers(10) -``` - -結果: - -```text -┌─quantileExact(number)─┐ -│ 5 │ -└───────────────────────┘ -``` - -## quantileExactLow {#quantileexactlow} - -`quantileExact` と同様に、数値データ列の正確な[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -正確な値を取得するために、渡されたすべての値を配列にまとめ、それを完全にソートします。ソート[アルゴリズム](https://en.cppreference.com/w/cpp/algorithm/sort)の計算量は `O(N·log(N))` であり、ここで `N = std::distance(first, last)` 回の比較が必要です。 - -戻り値は分位数レベルと選択された要素数に依存します。例えばレベルが 0.5 の場合、要素数が偶数のときは低い方の中央値を、要素数が奇数のときは中央値を返します。中央値は、Python で使用されている [median_low](https://docs.python.org/3/library/statistics.html#statistics.median_low) の実装と同様の方法で計算されます。 - -それ以外のすべてのレベルでは、`level * size_of_array` の値に対応するインデックスの要素が返されます。例えば次のとおりです。 - -```sql -SELECT quantileExactLow(0.1)(number) FROM numbers(10) - -┌─quantileExactLow(0.1)(number)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -同一クエリ内でレベルの異なる `quantile*` 関数を複数使用すると、内部状態が統合されず(つまり、本来よりもクエリの効率が低下します)。このような場合は、[quantiles](/sql-reference/aggregate-functions/reference/quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileExactLow(level)(expr) -``` - -エイリアス: `medianExactLow`。 - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 の範囲の定数の浮動小数点数。`level` の値として `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 数値の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) を結果とする、列値に対する式。 - -**戻り値** - -* 指定されたレベルの分位数。 - -型: - -* 数値データ型を入力とする場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantileExactLow(number) FROM numbers(10) -``` - -結果: - -```text -┌─quantileExactLow(number)─┐ -│ 4 │ -└──────────────────────────┘ -``` - -## quantileExactHigh {#quantileexacthigh} - -`quantileExact` と同様に、数値データ系列の正確な[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -渡されたすべての値は配列にまとめられ、その配列を完全にソートしてから正確な値を求めます。ソート[アルゴリズム](https://en.cppreference.com/w/cpp/algorithm/sort)の計算量は `O(N·log(N))` であり、ここで `N = std::distance(first, last)` は比較回数です。 - -戻り値は分位レベルと選択された要素数に依存します。たとえばレベルが 0.5 の場合、要素数が偶数のときは高い方の中央値を、要素数が奇数のときは中央の中央値を返します。中央値は、Python で使用されている [median_high](https://docs.python.org/3/library/statistics.html#statistics.median_high) 実装と同様の方法で計算されます。その他のレベルでは、`level * size_of_array` の値に対応するインデックスの要素が返されます。 - -この実装は、現行の `quantileExact` 実装とまったく同じ動作をします。 - -1 つのクエリ内で、異なるレベルを持つ複数の `quantile*` 関数を使用する場合、内部状態は統合されません(つまり、そのクエリは本来よりも非効率に動作します)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileExactHigh(level)(expr) -``` - -別名: `medianExactHigh`。 - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 以上 1 以下の定数の浮動小数点数。`level` の値として `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は [median](https://en.wikipedia.org/wiki/Median) を計算します。 -* `expr` — 列の値に対して評価され、その結果が数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) となる式。 - -**戻り値** - -* 指定されたレベルの分位数。 - -型: - -* 数値データ型の入力に対しては [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantileExactHigh(number) FROM numbers(10) -``` - -結果: - -```text -┌─quantileExactHigh(number)─┐ -│ 5 │ -└───────────────────────────┘ -``` - -## quantileExactExclusive {#quantileexactexclusive} - -数値データシーケンスの[分位点 (quantile)](https://en.wikipedia.org/wiki/Quantile) を厳密に計算します。 - -厳密な値を得るために、渡されたすべての値を配列にまとめ、その配列を部分的にソートします。したがって、この関数は、`n` を渡された値の個数とすると `O(n)` のメモリを消費します。ただし、値の個数が少ない場合には、この関数は非常に効率的です。 - -この関数は、Excel の [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) 関数([タイプ R6](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))と同等です。 - -1 つのクエリ内で異なるレベルの複数の `quantileExactExclusive` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来より非効率になります)。このような場合は、[quantilesExactExclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactexclusive) 関数を使用してください。 - -**構文** - -```sql -quantileExactExclusive(level)(expr) -``` - -**引数** - -* `expr` — カラム値に対する式で、その結果が数値の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md)、または [DateTime](../../../sql-reference/data-types/datetime.md) になるもの。 - -**パラメータ** - -* `level` — 分位数のレベル。省略可能。取りうる値の範囲: (0, 1) — 端点は含みません。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -* 指定したレベルの分位数。 - -型: - -* 数値データ型の入力に対しては [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactExclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -結果: - -```text -┌─quantileExactExclusive(0.6)(x)─┐ -│ 599.6 │ -└────────────────────────────────┘ -``` - -## quantileExactInclusive {#quantileexactinclusive} - -数値データ系列の[分位数](https://en.wikipedia.org/wiki/Quantile)を厳密に計算します。 - -厳密な値を取得するために、渡されたすべての値は配列に連結され、その配列が部分的にソートされます。そのため、この関数は `O(n)` のメモリを消費します。ここで `n` は渡された値の個数です。ただし、値の個数が少ない場合、この関数は非常に効率的です。 - -この関数は Excel の [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed) 関数([タイプ R7](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))と同等です。 - -1つのクエリ内で異なるレベルを持つ複数の `quantileExactInclusive` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来より非効率に動作します)。このような場合は、[quantilesExactInclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactinclusive) 関数を使用してください。 - -**構文** - -```sql -quantileExactInclusive(level)(expr) -``` - -**引数** - -* `expr` — 列の値に対する式で、結果として数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md)、または [DateTime](../../../sql-reference/data-types/datetime.md) を返します。 - -**パラメータ** - -* `level` — 分位数のレベル。省略可能。取り得る値は [0, 1](端点を含む)です。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。[Float](../../../sql-reference/data-types/float.md)。 - -**返される値** - -* 指定されたレベルの分位数。 - -型: - -* 数値データ型を入力とする場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactInclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -結果: - -```text -┌─quantileExactInclusive(0.6)(x)─┐ -│ 599.4 │ -└────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md deleted file mode 100644 index 4c49d83e5f9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '各要素の重みを考慮して、数値データ列の分位点を厳密に計算します。' -sidebar_position: 174 -slug: /sql-reference/aggregate-functions/reference/quantileexactweighted -title: 'quantileExactWeighted' -doc_type: 'reference' ---- - -# quantileExactWeighted {#quantileexactweighted} - -数値データ列について、各要素の重みを考慮して[分位数](https://en.wikipedia.org/wiki/Quantile)を厳密に計算します。 - -厳密な値を取得するために、渡されたすべての値は配列にまとめられ、その後部分的にソートされます。各値は、あたかも `weight` 回出現しているかのように、その重みを考慮してカウントされます。アルゴリズムではハッシュテーブルが使用されます。そのため、渡された値に同じ値が頻出する場合、この関数は [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) よりも RAM 消費量が少なくなります。この関数を `quantileExact` の代わりに使用し、重みとして 1 を指定することもできます。 - -1 つのクエリ内で、異なるレベルを持つ複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来よりも非効率に動作します)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileExactWeighted(level)(expr, weight) -``` - -Alias: `medianExactWeighted`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 までの定数の浮動小数点値です。`level` の値には `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — カラム値に対する式で、その結果が数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) になります。 -* `weight` — シーケンス要素の重みを格納するカラム。重みは、その値の出現回数を表す [Unsigned 整数型](../../../sql-reference/data-types/int-uint.md) の数値です。 - -**戻り値** - -* 指定されたレベルの分位数。 - -型: - -* 数値データ型入力の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -入力テーブル: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -クエリ: - -```sql -SELECT quantileExactWeighted(n, val) FROM t -``` - -結果: - -```text -┌─quantileExactWeighted(n, val)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md deleted file mode 100644 index a87b854b2d8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -description: '各要素の重みを考慮した線形補間により、数値データ列の分位数を計算します。' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated -title: 'quantileExactWeightedInterpolated' -doc_type: 'reference' ---- - -# quantileExactWeightedInterpolated {#quantileexactweightedinterpolated} - -数値データ列の各要素の重みを考慮し、線形補間を用いて[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -補間値を求めるために、すべての入力値を配列にまとめ、それぞれに対応する重みに基づいてソートします。次に、重みに基づいて累積分布を構築し、その上で[重み付きパーセンタイル法](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method)を用いて分位数の補間を行います。このとき、重みと値を用いて線形補間を実行し、分位数を計算します。 - -クエリ内で異なるレベルを持つ複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来可能な場合よりも非効率に動作します)。この場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -`quantileInterpolatedWeighted` よりも `quantileExactWeightedInterpolated` の使用を強く推奨します。`quantileExactWeightedInterpolated` の方が `quantileInterpolatedWeighted` よりも高い精度を持つためです。以下に例を示します。 - -```sql -SELECT - quantileExactWeightedInterpolated(0.99)(number, 1), - quantile(0.99)(number), - quantileInterpolatedWeighted(0.99)(number, 1) -FROM numbers(9) -┌─quantileExactWeightedInterpolated(0.99)(number, 1)─┬─quantile(0.99)(number)─┬─quantileInterpolatedWeighted(0.99)(number, 1)─┐ -│ 7.92 │ 7.92 │ 8 │ -└────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────────────────────────┘ -``` - -**構文** - -```sql -quantileExactWeightedInterpolated(level)(expr, weight) -``` - -エイリアス: `medianExactWeightedInterpolated`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 の間の定数の浮動小数点数値。`level` の値として `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 列の値に対して適用され、結果として数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) を返す式。 -* `weight` — シーケンス要素の重みを格納する列。重みは、その値の出現回数を表す[符号なし整数型](../../../sql-reference/data-types/int-uint.md)の数値です。 - -**戻り値** - -* 指定したレベルの分位数。 - -型: - -* 数値データ型入力の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -入力テーブル: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -結果: - -```text -┌─quantileExactWeightedInterpolated(n, val)─┐ -│ 1.5 │ -└───────────────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md deleted file mode 100644 index 62a1c8e54ef..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '各要素の重みを考慮し、線形補間を用いて数値データシーケンスの分位数を計算します。' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileInterpolatedWeighted -title: 'quantileInterpolatedWeighted' -doc_type: 'reference' ---- - -# quantileInterpolatedWeighted {#quantileinterpolatedweighted} - -数値データ列の[分位数](https://en.wikipedia.org/wiki/Quantile)を、各要素の重みを考慮した線形補間により計算します。 - -補間値を求めるために、渡されたすべての値を1つの配列にまとめ、その対応する重みに基づいてソートします。次に、重みに基づいて累積分布を構築し、その上で[重み付きパーセンタイル法](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method)を用いて分位数の補間を行います。このとき、重みと値を用いて線形補間を実行し、分位数を計算します。 - -1つのクエリ内で複数の `quantile*` 関数を異なるレベル(分位点)で使用すると、内部状態は結合されません(つまり、そのクエリは本来よりも効率が低くなります)。この場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileInterpolatedWeighted(level)(expr, weight) -``` - -Alias: `medianInterpolatedWeighted`. - -**引数** - -* `level` — 分位数レベル。省略可能なパラメータ。0 以上 1 以下の定数の浮動小数点数です。`level` の値には `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 列の値に対して評価され、数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md)、または [DateTime](../../../sql-reference/data-types/datetime.md) のいずれかになる式。 -* `weight` — シーケンス要素の重みを持つ列。重みは値の出現回数を表す数値です。 - -**返される値** - -* 指定したレベルの分位数。 - -型: - -* 数値データ型の入力に対しては [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -入力テーブル: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -クエリ: - -```sql -SELECT quantileInterpolatedWeighted(n, val) FROM t -``` - -結果: - -```text -┌─quantileInterpolatedWeighted(n, val)─┐ -│ 1 │ -└──────────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md deleted file mode 100644 index ca7d1f101b9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '線形補間を使用してヒストグラムの分位数を計算します。' -sidebar_position: 364 -slug: /sql-reference/aggregate-functions/reference/quantilePrometheusHistogram -title: 'quantilePrometheusHistogram' -doc_type: 'reference' ---- - -# quantilePrometheusHistogram {#quantileprometheushistogram} - -線形補間を用いてヒストグラムの[分位数 (quantile)](https://en.wikipedia.org/wiki/Quantile) を計算します。各ヒストグラムバケットの累積値と上限値を考慮します。 - -補間された値を取得するために、渡されたすべての値を 1 つの配列に結合し、その配列を対応するバケットの上限値でソートします。その後、従来型のヒストグラムに対する PromQL の [histogram_quantile()](https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile) 関数と同様に分位数の補間を行い、分位数の位置が属するバケットの下限値と上限値を用いて線形補間を実行します。 - -**構文** - -```sql -quantilePrometheusHistogram(level)(bucket_upper_bound, cumulative_bucket_value) -``` - -**引数** - -* `level` — 分位点のレベル。省略可能なパラメータ。0 から 1 の範囲の定数の浮動小数点数。`level` の値として `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値:`0.5`。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 - -* `bucket_upper_bound` — ヒストグラムバケットの上限値。 - - * 最も高いバケットは上限値として `+Inf` を持たなければなりません。 - -* `cumulative_bucket_value` — ヒストグラムバケットの累積 [UInt](../../../sql-reference/data-types/int-uint) または [Float64](../../../sql-reference/data-types/float.md) の値。 - - * バケットの上限値が増加するにつれて、値は単調増加になっている必要があります。 - -**返される値** - -* 指定されたレベルの分位点。 - -型: - -* `Float64`。 - -**例** - -入力テーブル: - -```text - ┌─bucket_upper_bound─┬─cumulative_bucket_value─┐ -1. │ 0 │ 6 │ -2. │ 0.5 │ 11 │ -3. │ 1 │ 14 │ -4. │ inf │ 19 │ - └────────────────────┴─────────────────────────┘ -``` - -結果: - -```text - ┌─quantilePrometheusHistogram(bucket_upper_bound, cumulative_bucket_value)─┐ -1. │ 0.35 │ - └──────────────────────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md deleted file mode 100644 index c7900731a79..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -description: 'quantiles、quantilesExactExclusive、quantilesExactInclusive、quantilesGK' -sidebar_position: 177 -slug: /sql-reference/aggregate-functions/reference/quantiles -title: 'quantiles 関数' -doc_type: 'reference' ---- - - - -# 分位数関数 {#quantiles-functions} - - - -## quantiles {#quantiles} - -構文: `quantiles(level1, level2, ...)(x)` - -すべての分位数関数には、対応する `quantiles` 系関数があります: `quantiles`, `quantilesDeterministic`, `quantilesTiming`, `quantilesTimingWeighted`, `quantilesExact`, `quantilesExactWeighted`, `quantileExactWeightedInterpolated`, `quantileInterpolatedWeighted`, `quantilesTDigest`, `quantilesBFloat16`, `quantilesDD`。これらの関数は、列挙されたレベルのすべての分位数を一度の処理で計算し、結果の値を配列で返します。 - - - -## quantilesExactExclusive {#quantilesexactexclusive} - -数値データシーケンスの[分位数](https://en.wikipedia.org/wiki/Quantile)を厳密に計算します。 - -厳密な値を取得するために、渡されたすべての値を配列にまとめ、その配列に対して部分ソートを実行します。したがって、この関数は `O(n)` のメモリを消費します。ここで `n` は渡された値の個数です。ただし、値の個数が少ない場合、この関数は非常に効率的です。 - -この関数は、Excel の [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) 関数([type R6](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))と同等です。 - -[quantileExactExclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactexclusive) と比較して、複数のレベル(分位点)の集合を扱う場合に、より効率的に動作します。 - -**構文** - -```sql -quantilesExactExclusive(level1, level2, ...)(expr) -``` - -**引数** - -* `expr` — 数値型の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) を結果とする、列の値に対して適用される式。 - -**パラメータ** - -* `level` — 分位点のレベル。取りうる値: (0, 1) — 両端点は含まれません。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -* 指定されたレベルの分位点の[Array](../../../sql-reference/data-types/array.md)。 - -配列要素の型: - -* 数値データ型入力の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -結果: - -```text -┌─quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.25,499.5,749.75,899.9,949.9499999999999,989.99,998.999] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesExactInclusive {#quantilesexactinclusive} - -数値データ系列の[分位数](https://en.wikipedia.org/wiki/Quantile)を厳密に計算します。 - -正確な値を取得するために、渡されたすべての値を配列にまとめ、その配列を部分的にソートします。そのため、この関数は `O(n)` のメモリを消費します。ここで `n` は渡された値の個数です。ただし、値の個数が少ない場合には、この関数は非常に効率的です。 - -この関数は、Excel の [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed) 関数([type R7](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))と同等です。 - -[quantileExactInclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactinclusive) よりも、複数のレベルに対して効率的に動作します。 - -**構文** - -```sql -quantilesExactInclusive(level1, level2, ...)(expr) -``` - -**引数** - -* `expr` — 列の値に対する式で、結果が数値の[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md)、または [DateTime](../../../sql-reference/data-types/datetime.md) になります。 - -**パラメータ** - -* `level` — 求める分位数のレベル。指定可能な値: [0, 1] — 両端を含む範囲。[Float](../../../sql-reference/data-types/float.md)。 - -**戻り値** - -* 指定したレベルの分位数を要素とする [Array](../../../sql-reference/data-types/array.md)。 - -配列要素の型: - -* 数値データ型を入力とした場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -結果: - -```text -┌─quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.75,499.5,749.25,899.1,949.05,989.01,998.001] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesGK {#quantilesgk} - -`quantilesGK` は `quantileGK` と同様に動作しますが、複数のレベルの分位数を同時に計算でき、配列を返します。 - -**構文** - -```sql -quantilesGK(accuracy, level1, level2, ...)(expr) -``` - -**返される値** - -* 指定したレベルの分位数の [Array](../../../sql-reference/data-types/array.md)。 - -配列要素の型: - -* 入力が数値データ型の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantilesGK(1, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [1,1,1] │ -└──────────────────────────────────────────────────┘ - -SELECT quantilesGK(10, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(10, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [156,413,659] │ -└───────────────────────────────────────────────────┘ -SELECT quantilesGK(100, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(100, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [251,498,741] │ -└────────────────────────────────────────────────────┘ - -SELECT quantilesGK(1000, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1000, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [249,499,749] │ -└─────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md deleted file mode 100644 index 90944114930..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: 't-digest アルゴリズムを使用して数値データ系列の近似的な分位数を計算します。' -sidebar_position: 178 -slug: /sql-reference/aggregate-functions/reference/quantiletdigest -title: 'quantileTDigest' -doc_type: 'reference' ---- - -# quantileTDigest {#quantiletdigest} - -[t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf) アルゴリズムを使用して、数値データ列の近似的な[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -メモリ消費量は `log(n)` であり、`n` は値の個数です。結果はクエリの実行順序に依存し、決定論的ではありません。 - -この関数のパフォーマンスは、[quantile](/sql-reference/aggregate-functions/reference/quantile) や [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) のパフォーマンスより劣ります。状態サイズと精度の比率という観点では、この関数は `quantile` よりも優れています。 - -異なるレベルを持つ複数の `quantile*` 関数を 1 つのクエリで使用する場合、内部状態は結合されません(つまり、そのクエリは本来可能なものよりも効率が低くなります)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileTDigest(level)(expr) -``` - -エイリアス: `medianTDigest`。 - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメーター。0 から 1 までの定数の浮動小数点数。`level` の値には `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` のとき、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) を返す、列値に対して適用される式。 - -**戻り値** - -* 指定したレベルの近似分位数。 - -型: - -* 数値データ型入力の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantileTDigest(number) FROM numbers(10) -``` - -結果: - -```text -┌─quantileTDigest(number)─┐ -│ 4.5 │ -└─────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md deleted file mode 100644 index 322cc97a176..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 't-digest アルゴリズムを使用して、数値データのシーケンスに対する近似分位数を計算します。' -sidebar_position: 179 -slug: /sql-reference/aggregate-functions/reference/quantiletdigestweighted -title: 'quantileTDigestWeighted' -doc_type: 'reference' ---- - -# quantileTDigestWeighted {#quantiletdigestweighted} - -数値データ列に対して、[t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf) アルゴリズムを用いて近似的な [分位点](https://en.wikipedia.org/wiki/Quantile) を計算します。各要素の重みを考慮します。最大誤差は 1% です。メモリ使用量は `log(n)` で、`n` は値の数です。 - -この関数の性能は [quantile](/sql-reference/aggregate-functions/reference/quantile) や [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) よりも劣ります。State サイズと精度の比率という観点では、この関数は `quantile` よりも優れています。 - -結果はクエリの実行順序に依存し、非決定的です。 - -同一クエリ内でレベルの異なる複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来よりも効率が低下します)。この場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数の使用を検討してください。 - -:::note\ -`quantileTDigestWeighted` の使用は、[ごく小さいデータセットには推奨されておらず](https://github.com/tdunning/t-digest/issues/167#issuecomment-828650275)、大きな誤差につながる可能性があります。この場合は、代わりに [`quantileTDigest`](../../../sql-reference/aggregate-functions/reference/quantiletdigest.md) の使用を検討してください。 -::: - -**構文** - -```sql -quantileTDigestWeighted(level)(expr, weight) -``` - -エイリアス: `medianTDigestWeighted`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 までの定数の浮動小数点数。`level` の値は `[0.01, 0.99]` の範囲で使用することを推奨します。既定値: 0.5。`level=0.5` の場合、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 -* `expr` — 数値[データ型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) または [DateTime](../../../sql-reference/data-types/datetime.md) を結果とする、カラム値に対する式。 -* `weight` — シーケンスの各要素の重みを持つカラム。重みは値の出現回数です。 - -**戻り値** - -* 指定されたレベルの近似分位数。 - -型: - -* 数値データ型入力の場合は [Float64](../../../sql-reference/data-types/float.md)。 -* 入力値が `Date` 型の場合は [Date](../../../sql-reference/data-types/date.md)。 -* 入力値が `DateTime` 型の場合は [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**例** - -クエリ: - -```sql -SELECT quantileTDigestWeighted(number, 1) FROM numbers(10) -``` - -結果: - -```text -┌─quantileTDigestWeighted(number, 1)─┐ -│ 4.5 │ -└────────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md deleted file mode 100644 index c783d59d1a3..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: '指定された精度で数値データシーケンスの分位数を計算します。' -sidebar_position: 180 -slug: /sql-reference/aggregate-functions/reference/quantiletiming -title: 'quantileTiming' -doc_type: 'reference' ---- - -# quantileTiming {#quantiletiming} - -指定された精度で数値データシーケンスの[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -結果は決定的であり(クエリの処理順序に依存しません)、Webページの読み込み時間やバックエンドのレスポンス時間のような分布を表すシーケンスで動作するよう最適化されています。 - -1 つのクエリ内で異なるレベルを持つ複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来可能な場合よりも効率が低下します)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileTiming(level)(expr) -``` - -Alias: `medianTiming`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメーター。0 から 1 までの定数の浮動小数点数値。`level` の値には `[0.01, 0.99]` の範囲を使用することを推奨します。デフォルト値: 0.5。`level=0.5` のとき、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 - -* `expr` — カラム値に対する[式](/sql-reference/syntax#expressions)で、[Float*](../../../sql-reference/data-types/float.md) 型の数値を返します。 - - * 負の値が関数に渡された場合、その動作は未定義です。 - * 値が 30,000(ページ読み込み時間が 30 秒超)より大きい場合、30,000 とみなされます。 - -**精度** - -計算は次の場合に正確です: - -* 値の総数が 5670 を超えない。 -* 値の総数が 5670 を超えるが、ページ読み込み時間が 1024 ms 未満である。 - -それ以外の場合、計算結果は 16 ms の最も近い倍数に丸められます。 - -:::note\ -ページ読み込み時間の分位数を計算する場合、この関数は [quantile](/sql-reference/aggregate-functions/reference/quantile) よりも効率的かつ高精度です。 -::: - -**戻り値** - -* 指定されたレベルの分位数。 - -型: `Float32`。 - -:::note\ -(`quantileTimingIf` を使用していて)関数に値が 1 つも渡されない場合、[NaN](/sql-reference/data-types/float#nan-and-inf) が返されます。これは、結果が 0 になるケースと区別することを目的としています。`NaN` 値のソートに関する注意事項については、[ORDER BY 句](/sql-reference/statements/select/order-by)を参照してください。 -::: - -**例** - -入力テーブル: - -```text -┌─response_time─┐ -│ 72 │ -│ 112 │ -│ 126 │ -│ 145 │ -│ 104 │ -│ 242 │ -│ 313 │ -│ 168 │ -│ 108 │ -└───────────────┘ -``` - -クエリ: - -```sql -SELECT quantileTiming(response_time) FROM t -``` - -結果: - -```text -┌─quantileTiming(response_time)─┐ -│ 126 │ -└───────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md deleted file mode 100644 index af14fd30107..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md +++ /dev/null @@ -1,124 +0,0 @@ ---- -description: '指定された精度で、各要素の重みに基づいて数値データ列の分位数を計算します。' -sidebar_position: 181 -slug: /sql-reference/aggregate-functions/reference/quantiletimingweighted -title: 'quantileTimingWeighted' -doc_type: 'reference' ---- - -# quantileTimingWeighted {#quantiletimingweighted} - -指定した精度で、数値データのシーケンスに対して、各要素の重みを考慮した[分位数](https://en.wikipedia.org/wiki/Quantile)を計算します。 - -結果は決定論的であり(クエリの処理順序には依存しません)、Web ページの読み込み時間やバックエンドの応答時間のような分布を表すシーケンスの処理に最適化されています。 - -1 つのクエリ内で、異なるレベルを指定した複数の `quantile*` 関数を使用する場合、内部状態は結合されません(つまり、そのクエリは本来より非効率に動作します)。このような場合は、[quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 関数を使用してください。 - -**構文** - -```sql -quantileTimingWeighted(level)(expr, weight) -``` - -Alias: `medianTimingWeighted`. - -**引数** - -* `level` — 分位数のレベル。省略可能なパラメータ。0 から 1 の間の定数の浮動小数点数。`level` の値は `[0.01, 0.99]` の範囲で使用することを推奨します。デフォルト値: 0.5。`level=0.5` のとき、この関数は[中央値](https://en.wikipedia.org/wiki/Median)を計算します。 - -* `expr` — 列の値に対する[式](/sql-reference/syntax#expressions)で、[Float*](../../../sql-reference/data-types/float.md) 型の数値を返します。 - - * 負の値が関数に渡された場合、その動作は未定義です。 - * 値が 30,000(30 秒を超えるページ読み込み時間)より大きい場合、30,000 であるとみなされます。 - -* `weight` — シーケンスの要素の重みを格納する列。重みは値の出現回数です。 - -**精度** - -次の場合、計算は正確です: - -* 値の総数が 5670 を超えない場合。 -* 値の総数が 5670 を超える場合でも、ページ読み込み時間が 1024 ms 未満の場合。 - -それ以外の場合、計算結果は 16 ms の倍数のうち最も近い値に丸められます。 - -:::note\ -ページ読み込み時間の分位数を計算する場合、この関数は [quantile](/sql-reference/aggregate-functions/reference/quantile) よりも効率的かつ高精度です。 -::: - -**戻り値** - -* 指定したレベルの分位数。 - -型: `Float32`。 - -:::note\ -関数に値が一つも渡されない場合(`quantileTimingIf` を使用する場合)、[NaN](/sql-reference/data-types/float#nan-and-inf) が返されます。これは、このようなケースを結果がゼロになるケースと区別することを目的としています。`NaN` 値のソートに関する注意点については、[ORDER BY 句](/sql-reference/statements/select/order-by)を参照してください。 -::: - -**例** - -入力テーブル: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -クエリ: - -```sql -SELECT quantileTimingWeighted(response_time, weight) FROM t -``` - -結果: - -```text -┌─quantileTimingWeighted(response_time, weight)─┐ -│ 112 │ -└───────────────────────────────────────────────┘ -``` - -# quantilesTimingWeighted {#quantilestimingweighted} - -`quantileTimingWeighted` と同様ですが、複数の分位レベルをパラメータとして受け取り、それらの分位数に対応する値を要素とする Array を返します。 - -**例** - -入力テーブル: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -クエリ: - -```sql -SELECT quantilesTimingWeighted(0,5, 0.99)(response_time, weight) FROM t -``` - -結果: - -```text -┌─quantilesTimingWeighted(0.5, 0.99)(response_time, weight)─┐ -│ [112,162] │ -└───────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md deleted file mode 100644 index 927b23af122..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '順位相関係数を計算します。' -sidebar_position: 182 -slug: /sql-reference/aggregate-functions/reference/rankCorr -title: 'rankCorr' -doc_type: 'reference' ---- - -# rankCorr {#rankcorr} - -順位相関係数を計算します。 - -**構文** - -```sql -rankCorr(x, y) -``` - -**引数** - -* `x` — 任意の値。[Float32](/sql-reference/data-types/float) または [Float64](/sql-reference/data-types/float)。 -* `y` — 任意の値。[Float32](/sql-reference/data-types/float) または [Float64](/sql-reference/data-types/float)。 - -**返り値** - -* `x` と `y` の順位に対する順位相関係数を返します。相関係数の値は -1 から +1 の範囲です。2 つ未満の引数が渡された場合、関数は例外をスローします。+1 に近い値は強い線形的な関係を示し、一方の確率変数が増加すると、もう一方の確率変数も増加します。-1 に近い値は強い線形的な関係を示し、一方の確率変数が増加すると、もう一方の確率変数は減少します。0 付近または 0 に等しい値は、2 つの確率変数の間に関係がないことを示します。 - -型: [Float64](/sql-reference/data-types/float)。 - -**例** - -クエリ: - -```sql -SELECT rankCorr(number, number) FROM numbers(100); -``` - -結果: - -```text -┌─rankCorr(number, number)─┐ -│ 1 │ -└──────────────────────────┘ -``` - -クエリ: - -```sql -SELECT roundBankers(rankCorr(exp(number), sin(number)), 3) FROM numbers(100); -``` - -結果: - -```text -┌─roundBankers(rankCorr(exp(number), sin(number)), 3)─┐ -│ -0.037 │ -└─────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [スピアマンの順位相関係数](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md deleted file mode 100644 index e5605049f07..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: '単回帰(1次元の線形回帰)を行います。' -sidebar_position: 183 -slug: /sql-reference/aggregate-functions/reference/simplelinearregression -title: 'simpleLinearRegression' -doc_type: 'reference' ---- - -# simpleLinearRegression {#simplelinearregression} - -単回帰(1次元の線形回帰)を実行します。 - -```sql -simpleLinearRegression(x, y) -``` - -パラメータ: - -* `x` — 説明変数の値を含む列。 -* `y` — 従属変数の値を含む列。 - -戻り値: - -得られた直線 `y = k*x + b` の定数 `(k, b)`。 - -**例** - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3])─┐ -│ (1,0) │ -└───────────────────────────────────────────────────────────────────┘ -``` - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6])─┐ -│ (1,3) │ -└───────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md deleted file mode 100644 index a382de01719..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '集約関数 `singleValueOrNull` は、`x = ALL (SELECT ...)` のようなサブクエリ演算子を実装するために使用されます。データ内に一意な非 NULL の値がちょうど 1 つだけ存在するかどうかを判定します。' -sidebar_position: 184 -slug: /sql-reference/aggregate-functions/reference/singlevalueornull -title: 'singleValueOrNull' -doc_type: 'reference' ---- - -# singleValueOrNull {#singlevalueornull} - -集約関数 `singleValueOrNull` は、`x = ALL (SELECT ...)` のようなサブクエリの演算子を実装するために使用されます。データ内にただ 1 つだけ、一意な非 NULL 値が存在するかどうかを判定します。 -一意な値が 1 つだけ存在する場合、その値を返します。値が 0 個、または相異なる値が 2 個以上存在する場合は、NULL を返します。 - -**構文** - -```sql -singleValueOrNull(x) -``` - -**パラメーター** - -* `x` — 任意の [データ型](../../data-types/index.md) の列(ただし [Nullable](../../data-types/nullable.md) 型にできない [Map](../../data-types/map.md)、[Array](../../data-types/array.md)、[Tuple](../../data-types/tuple) 型は除く)。 - -**戻り値** - -* `x` の中に一意な非 NULL 値がちょうど 1 つだけ存在する場合、その一意な値。 -* 0 個、または 2 個以上の異なる値が存在する場合は `NULL`。 - -**例** - -クエリ: - -```sql -CREATE TABLE test (x UInt8 NULL) ENGINE=Log; -INSERT INTO test (x) VALUES (NULL), (NULL), (5), (NULL), (NULL); -SELECT singleValueOrNull(x) FROM test; -``` - -結果: - -```response -┌─singleValueOrNull(x)─┐ -│ 5 │ -└──────────────────────┘ -``` - -クエリ: - -```sql -INSERT INTO test (x) VALUES (10); -SELECT singleValueOrNull(x) FROM test; -``` - -結果: - -```response -┌─singleValueOrNull(x)─┐ -│ ᴺᵁᴸᴸ │ -└──────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md deleted file mode 100644 index b866ca2a0cb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'データ系列の歪度を計算します。' -sidebar_position: 185 -slug: /sql-reference/aggregate-functions/reference/skewpop -title: 'skewPop' -doc_type: 'reference' ---- - -# skewPop {#skewpop} - -シーケンスの[歪度](https://en.wikipedia.org/wiki/Skewness)を計算します。 - -```sql -skewPop(expr) -``` - -**引数** - -`expr` — 数値を返す[式](/sql-reference/syntax#expressions)。 - -**戻り値** - -指定された分布の歪度。型 — [Float64](../../../sql-reference/data-types/float.md) - -**例** - -```sql -SELECT skewPop(value) FROM series_with_value_column; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md deleted file mode 100644 index 88462f9a8cb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'シーケンスの標本歪度を計算します。' -sidebar_position: 186 -slug: /sql-reference/aggregate-functions/reference/skewsamp -title: 'skewSamp' -doc_type: 'reference' ---- - -# skewSamp {#skewsamp} - -一連の値の[標本歪度](https://en.wikipedia.org/wiki/Skewness)を計算します。 - -渡された値がある確率変数の標本を構成している場合、その確率変数の歪度に対する不偏推定量となります。 - -```sql -skewSamp(expr) -``` - -**引数** - -`expr` — 数値を返す[式](/sql-reference/syntax#expressions)。 - -**返り値** - -与えられた分布の歪度。型は [Float64](../../../sql-reference/data-types/float.md)。`n <= 1`(`n` は標本サイズ)の場合、関数は `nan` を返します。 - -**例** - -```sql -SELECT skewSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md deleted file mode 100644 index ffee955c5a9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 'この関数は、値 `x` とそれらの繰り返し回数 `y` に基づき、区間 `[min_x, max_x]` 上の度数ヒストグラムをプロットします。' -sidebar_label: 'sparkbar' -sidebar_position: 187 -slug: /sql-reference/aggregate-functions/reference/sparkbar -title: 'sparkbar' -doc_type: 'reference' ---- - -# sparkbar {#sparkbar} - -この関数は、値 `x` と、その値の出現回数(頻度)を表す `y` に対して、区間 `[min_x, max_x]` 上で度数ヒストグラムをプロットします。 -同じバケットに入るすべての `x` の出現回数(頻度)`y` は平均化されるため、データは事前に集約されている必要があります。 -`y` の負の値は無視されます。 - -区間が指定されていない場合、最小の `x` が区間の開始として、最大の `x` が区間の終了として使用されます。 -区間が指定されている場合は、その区間外の値は無視されます。 - -**構文** - -```sql -sparkbar(buckets[, min_x, max_x])(x, y) -``` - -**パラメータ** - -* `buckets` — セグメント数。型: [Integer](../../../sql-reference/data-types/int-uint.md)。 -* `min_x` — 区間の開始値。省略可能なパラメータ。 -* `max_x` — 区間の終了値。省略可能なパラメータ。 - -**引数** - -* `x` — 値を持つフィールド。 -* `y` — 値の頻度を示すフィールド。 - -**返される値** - -* 度数ヒストグラム。 - -**例** - -クエリ: - -```sql -CREATE TABLE spark_bar_data (`value` Int64, `event_date` Date) ENGINE = MergeTree ORDER BY event_date; - -INSERT INTO spark_bar_data VALUES (1,'2020-01-01'), (3,'2020-01-02'), (4,'2020-01-02'), (-3,'2020-01-02'), (5,'2020-01-03'), (2,'2020-01-04'), (3,'2020-01-05'), (7,'2020-01-06'), (6,'2020-01-07'), (8,'2020-01-08'), (2,'2020-01-11'); - -SELECT sparkbar(9)(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); - -SELECT sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); -``` - -結果: - -```text -┌─sparkbar(9)(event_date, cnt)─┐ -│ ▂▅▂▃▆█ ▂ │ -└──────────────────────────────┘ - -┌─sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date, cnt)─┐ -│ ▂▅▂▃▇▆█ │ -└──────────────────────────────────────────────────────────────────────────┘ -``` - -この関数の別名(エイリアス)は sparkBar です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md deleted file mode 100644 index 7daaeb0512b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '結果は varPop の平方根になります。' -sidebar_position: 188 -slug: /sql-reference/aggregate-functions/reference/stddevpop -title: 'stddevPop' -doc_type: 'reference' ---- - -# stddevPop {#stddevpop} - -結果は [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md) の平方根と等しくなります。 - -別名: `STD`, `STDDEV_POP`. - -:::note -この関数は数値的に安定でないアルゴリズムを使用します。計算で[数値安定性](https://en.wikipedia.org/wiki/Numerical_stability)が必要な場合は、[`stddevPopStable`](../reference/stddevpopstable.md) 関数を使用してください。動作は遅くなりますが、計算誤差が小さくなります。 -::: - -**構文** - -```sql -stddevPop(x) -``` - -**パラメータ** - -* `x`: 標準偏差を求める対象となる値の集合。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返り値** - -* `x` の標準偏差の平方根。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevPop(population) AS stddev -FROM test_data; -``` - -結果: - -```response -┌────────────stddev─┐ -│ 3.794733192202055 │ -└───────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md deleted file mode 100644 index 1130d45fd11..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '結果は varPop の平方根と等しくなります。stddevPop とは異なり、 - この関数は数値的に安定なアルゴリズムを使用します。' -sidebar_position: 189 -slug: /sql-reference/aggregate-functions/reference/stddevpopstable -title: 'stddevPopStable' -doc_type: 'reference' ---- - -# stddevPopStable {#stddevpopstable} - -結果は [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md) の平方根と等しくなります。[`stddevPop`](../reference/stddevpop.md) とは異なり、この関数は数値的に安定したアルゴリズムを使用します。処理は遅くなりますが、計算誤差をより小さく抑えることができます。 - -**構文** - -```sql -stddevPopStable(x) -``` - -**パラメーター** - -* `x`: 標準偏差を求める対象となる値の集合。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返される値** - -`x` の分散の平方根。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population Float64, -) -ENGINE = Log; - -INSERT INTO test_data SELECT randUniform(5.5, 10) FROM numbers(1000000) - -SELECT - stddevPopStable(population) AS stddev -FROM test_data; -``` - -結果: - -```response -┌─────────────stddev─┐ -│ 1.2999977786592576 │ -└────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md deleted file mode 100644 index b74682794bf..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '結果は varSamp の平方根です' -sidebar_position: 190 -slug: /sql-reference/aggregate-functions/reference/stddevsamp -title: 'stddevSamp' -doc_type: 'reference' ---- - -# stddevSamp {#stddevsamp} - -結果は [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md) の平方根と等しくなります。 - -エイリアス: `STDDEV_SAMP`。 - -:::note -この関数は数値的に不安定なアルゴリズムを使用します。計算で[数値安定性](https://en.wikipedia.org/wiki/Numerical_stability)が必要な場合は、[`stddevSampStable`](../reference/stddevsampstable.md) 関数を使用してください。こちらは動作が遅くなりますが、計算誤差をより小さく抑えることができます。 -::: - -**構文** - -```sql -stddevSamp(x) -``` - -**パラメータ** - -* `x`: 標本分散の平方根を求める対象の値。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**戻り値** - -`x` の標本分散の平方根。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSamp(population) -FROM test_data; -``` - -結果: - -```response -┌─stddevSamp(population)─┐ -│ 4 │ -└────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md deleted file mode 100644 index 9f2c55b7539..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '結果は varSamp の平方根と等しくなります。この関数では、数値的に安定したアルゴリズムを使用します。' -sidebar_position: 191 -slug: /sql-reference/aggregate-functions/reference/stddevsampstable -title: 'stddevSampStable' -doc_type: 'reference' ---- - -# stddevSampStable {#stddevsampstable} - -結果は [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md) の平方根と等しくなります。[`stddevSamp`](../reference/stddevsamp.md) と異なり、この関数は数値的に安定したアルゴリズムを使用します。動作は遅くなりますが、計算誤差を小さく抑えることができます。 - -**構文** - -```sql -stddevSampStable(x) -``` - -**パラメータ** - -* `x`: 標本分散の平方根を計算する対象となる値。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**戻り値** - -`x` の標本分散の平方根。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSampStable(population) -FROM test_data; -``` - -結果: - -```response -┌─stddevSampStable(population)─┐ -│ 4 │ -└──────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md deleted file mode 100644 index 61644470085..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: 'この関数は確率的線形回帰を実装します。学習率、L2 正則化係数、ミニバッチサイズのカスタムパラメーターをサポートし、さらに複数の重み更新手法(Adam、単純 SGD、Momentum、Nesterov)に対応しています。' -sidebar_position: 192 -slug: /sql-reference/aggregate-functions/reference/stochasticlinearregression -title: 'stochasticLinearRegression' -doc_type: 'reference' ---- - -# stochasticLinearRegression {#agg_functions_stochasticlinearregression_parameters} - -この関数は確率的線形回帰を実装します。学習率、L2 正則化係数、ミニバッチサイズのカスタムパラメータをサポートし、重みを更新するためのいくつかの手法(デフォルトで使用される [Adam](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam)、[simple SGD](https://en.wikipedia.org/wiki/Stochastic_gradient_descent)、[Momentum](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum)、[Nesterov](https://mipt.ru/upload/medialibrary/d7e/41-91.pdf))を備えています。 - -### Parameters {#parameters} - -カスタマイズ可能なパラメータは 4 つあります。これらは関数に順番に渡されますが、4 つすべてを渡す必要はありません。指定されなかったものにはデフォルト値が使用されますが、良いモデルを得るには一部のパラメータ調整が必要になる場合があります。 - -```text -stochasticLinearRegression(0.00001, 0.1, 15, 'Adam') -``` - -1. `learning rate` は、勾配降下ステップを実行する際のステップ長に掛かる係数です。学習率が大きすぎると、モデルの重みが発散して無限大になる可能性があります。デフォルトは `0.00001` です。 -2. `l2 regularization coefficient` は、過学習の防止に役立つ係数です。デフォルトは `0.1` です。 -3. `mini-batch size` は、1 回の勾配降下ステップを行うために勾配を計算して加算する要素数を指定します。純粋な確率的降下では 1 要素のみを使用しますが、小さなバッチ(約 10 要素)にすることで、勾配ステップがより安定します。デフォルトは `15` です。 -4. `method for updating weights` には次のものがあります: `Adam`(デフォルト)、`SGD`、`Momentum`、`Nesterov`。`Momentum` と `Nesterov` は、やや多くの計算とメモリを必要としますが、収束速度および確率的勾配法の安定性の観点から有用な場合があります。 - -### Usage {#usage} - -`stochasticLinearRegression` は 2 段階で利用します。まずモデルをフィット(学習)し、その後新しいデータに対して予測を行います。モデルをフィットして、その状態を後で利用できるように保存するには、状態(例: モデルの重み)を保存する `-State` コンビネータを使用します。 -予測には、状態と予測対象の特徴量を引数として受け取る関数 [evalMLMethod](/sql-reference/functions/machine-learning-functions#evalmlmethod) を使用します。 - -
- -**1.** Fitting - -次のようなクエリを使用します。 - -```sql -CREATE TABLE IF NOT EXISTS train_data -( - param1 Float64, - param2 Float64, - target Float64 -) ENGINE = Memory; - -CREATE TABLE your_model ENGINE = Memory AS SELECT -stochasticLinearRegressionState(0.1, 0.0, 5, 'SGD')(target, param1, param2) -AS state FROM train_data; -``` - -ここでは、`train_data` テーブルにデータを挿入する必要もあります。パラメータの数は固定ではなく、`linearRegressionState` に渡される引数の数のみに依存します。これらはすべて数値である必要があります。 -予測したいターゲット値(学習したい値)の列は、最初の引数として挿入されることに注意してください。 - -**2.** 予測 - -状態をテーブルに保存した後は、その状態を予測に複数回使用したり、他の状態とマージして、より新しく高性能なモデルを作成することもできます。 - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -クエリは予測値の列を返します。`evalMLMethod` の最初の引数は `AggregateFunctionState` オブジェクトであり、その後に特徴量の列が続くことに注意してください。 - -`test_data` は `train_data` と同様のテーブルですが、目的変数を含まない場合があります。 - -### Notes {#notes} - -1. 2 つのモデルを統合するには、次のようなクエリを作成できます: - `sql SELECT state1 + state2 FROM your_models` - ここで、`your_models` テーブルには両方のモデルが含まれています。このクエリは新しい `AggregateFunctionState` オブジェクトを返します。 - -2. ユーザーは、`-State` コンビネータを使用しない場合、モデルを保存しなくても、作成したモデルの重みを任意の用途で取得できます。 - `sql SELECT stochasticLinearRegression(0.01)(target, param1, param2) FROM train_data` - このようなクエリはモデルをフィットし、その重みを返します。先頭の値はモデルの各パラメータに対応する重みで、最後の 1 つがバイアスです。したがって上記の例では、クエリは 3 つの値を持つ列を返します。 - -**See Also** - -* [stochasticLogisticRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [線形回帰とロジスティック回帰の違い](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md deleted file mode 100644 index f32d264a28a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: 'この関数は確率的ロジスティック回帰を実装します。二値分類タスクに使用でき、stochasticLinearRegression と同じカスタムパラメータをサポートし、同様に動作します。' -sidebar_position: 193 -slug: /sql-reference/aggregate-functions/reference/stochasticlogisticregression -title: 'stochasticLogisticRegression' -doc_type: 'reference' ---- - -# stochasticLogisticRegression {#stochasticlogisticregression} - -この関数は確率的ロジスティック回帰を実装します。二値分類問題に使用でき、stochasticLinearRegression と同じカスタムパラメータをサポートし、動作も同様です。 - -### Parameters {#parameters} - -パラメータは stochasticLinearRegression とまったく同じです: -`learning rate`、`l2 regularization coefficient`、`mini-batch size`、`method for updating weights`。 -詳細については [parameters](../reference/stochasticlinearregression.md/#parameters) を参照してください。 - -```text -stochasticLogisticRegression(1.0, 1.0, 10, 'SGD') -``` - -**1.** 適合 - -{/* */ } - -[stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlinearregression) の説明にある `Fitting` セクションを参照してください。 - -予測ラベルの値は [-1, 1] の範囲内である必要があります。 - -**2.** 予測 - -{/* */ } - -保存済みの状態を使って、対象オブジェクトにラベル `1` が付与される確率を予測できます。 - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -クエリは確率の列を返します。`evalMLMethod` の最初の引数は `AggregateFunctionState` オブジェクトであり、それ以降の引数には特徴量のカラムを指定します。 - -また、確率の閾値を設定して、要素を異なるラベルに割り当てることもできます。 - -```sql -SELECT ans < 1.1 AND ans > 0.5 FROM -(WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) AS ans FROM test_data) -``` - -その結果はラベルとして出力されます。 - -`test_data` は `train_data` と同様のテーブルですが、目的変数の値を含まない場合があります。 - -**関連項目** - -* [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [線形回帰とロジスティック回帰の違い](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md deleted file mode 100644 index 4f6fcd241c2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '2つの母集団から得られた標本に対してスチューデントの t 検定を適用します。' -sidebar_label: 'studentTTest' -sidebar_position: 194 -slug: /sql-reference/aggregate-functions/reference/studentttest -title: 'studentTTest' -doc_type: 'reference' ---- - -# studentTTest {#studentttest} - -2つの母集団から得られた標本にスチューデントの t 検定を適用します。 - -**構文** - -```sql -studentTTest([confidence_level])(sample_data, sample_index) -``` - -両方のサンプルの値は `sample_data` 列にあります。`sample_index` が 0 の場合、その行の値は第1母集団からのサンプルに属します。そうでない場合は第2母集団からのサンプルに属します。 -帰無仮説は、母集団の平均が等しいというものです。等分散の正規分布に従うと仮定します。 - -**引数** - -* `sample_data` — サンプルデータ。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — サンプルのインデックス。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `confidence_level` — 信頼区間を計算するための信頼水準。[Float](../../../sql-reference/data-types/float.md)。 - -**返される値** - -2 要素または 4 要素(オプションの `confidence_level` が指定された場合)の [Tuple](../../../sql-reference/data-types/tuple.md): - -* 計算された t統計量。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された p値。[Float64](../../../sql-reference/data-types/float.md)。 -* [計算された信頼区間の下限。[Float64](../../../sql-reference/data-types/float.md)。] -* [計算された信頼区間の上限。[Float64](../../../sql-reference/data-types/float.md)。] - -**例** - -入力テーブル: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.1 │ 0 │ -│ 21.9 │ 1 │ -│ 21.7 │ 0 │ -│ 19.9 │ 1 │ -│ 21.8 │ 1 │ -└─────────────┴──────────────┘ -``` - -クエリ: - -```sql -SELECT studentTTest(sample_data, sample_index) FROM student_ttest; -``` - -結果: - -```text -┌─studentTTest(sample_data, sample_index)───┐ -│ (-0.21739130434783777,0.8385421208415731) │ -└───────────────────────────────────────────┘ -``` - -**関連項目** - -* [スチューデントのt検定](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [welchTTest 関数](/sql-reference/aggregate-functions/reference/welchttest) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md deleted file mode 100644 index 5d2dba8da90..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '既知の母平均に対して、標本に 1 標本 Student の t 検定を適用します。' -sidebar_label: 'studentTTestOneSample' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/studentttestonesample -title: 'studentTTestOneSample' -doc_type: 'reference' ---- - -# studentTTestOneSample {#studentttestonesample} - -既知の母平均と比較して標本の平均値が異なるかどうかを判定するために、1標本の Student の t 検定を適用します。 - -標本が正規分布に従うと仮定します。帰無仮説は、標本平均が母平均に等しいというものです。 - -**構文** - -```sql -studentTTestOneSample([confidence_level])(sample_data, population_mean) -``` - -オプションの `confidence_level` により、信頼区間の計算が有効になります。 - -**引数** - -* `sample_data` — サンプルデータ。Integer、Float または Decimal。 -* `population_mean` — 検定対象となる既知の母平均。Integer、Float または Decimal(通常は定数)。 - -**パラメータ** - -* `confidence_level` — 信頼区間に用いる信頼水準。(0, 1) の Float。 - -注意: - -* 観測値が少なくとも 2 つ必要です。それ未満の場合、結果は `(nan, nan)` となり(要求された区間も `nan` となります)。 -* 入力が定数またはほぼ定数の場合、標準誤差が 0(もしくは実質的に 0)であるため、`nan` が返されます。 - -**返される値** - -[Tuple](../../../sql-reference/data-types/tuple.md) 型で、2 要素または 4 要素(`confidence_level` が指定された場合)です: - -* 計算された t 統計量。Float64。 -* 計算された p 値(両側検定)。Float64。 -* 計算された信頼区間の下限。Float64。(オプション) -* 計算された信頼区間の上限。Float64。(オプション) - -信頼区間は、指定された信頼水準における標本平均に対するものです。 - -**例** - -入力テーブル: - -```text -┌─value─┐ -│ 20.3 │ -│ 21.1 │ -│ 21.7 │ -│ 19.9 │ -│ 21.8 │ -└───────┘ -``` - -信頼区間なし: - -```sql -SELECT studentTTestOneSample()(value, 20.0) FROM t; --- or simply -SELECT studentTTestOneSample(value, 20.0) FROM t; -``` - -95% 信頼区間: - -```sql -SELECT studentTTestOneSample(0.95)(value, 20.0) FROM t; -``` - -**関連項目** - -* [スチューデントのt検定](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [studentTTest 関数](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md deleted file mode 100644 index ba80301252e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '合計を計算します。数値に対してのみ使用できます。' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/sum -title: 'sum' -doc_type: 'reference' ---- - -# sum {#sum} - -合計を計算します。数値に対してのみ使用できます。 - -**構文** - -```sql -sum(num) -``` - -**パラメータ** - -* `num`: 数値の列。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**戻り値** - -* 値の合計。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**例** - -まずテーブル `employees` を作成し、架空の従業員データを挿入します。 - -クエリ: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `salary` UInt32 -) -ENGINE = Log -``` - -```sql -INSERT INTO employees VALUES - (87432, 'John Smith', 45680), - (59018, 'Jane Smith', 72350), - (20376, 'Ivan Ivanovich', 58900), - (71245, 'Anastasia Ivanovna', 89210); -``` - -`sum` 関数を使用して、従業員の給与総額を取得します。 - -クエリ: - -```sql -SELECT sum(salary) FROM employees; -``` - -結果: - -```response - ┌─sum(salary)─┐ -1. │ 266140 │ - └─────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md deleted file mode 100644 index b8e2e2a6b87..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '数値の合計を計算すると同時に、行数もカウントします。ClickHouse のクエリオプティマイザによって使用されます。クエリ内に複数の `sum`、`count`、`avg` 関数がある場合、計算結果を再利用するために、それらを 1 つの `sumCount` 関数に置き換えることができます。この関数が明示的に必要になることはまれです。' -sidebar_position: 196 -slug: /sql-reference/aggregate-functions/reference/sumcount -title: 'sumCount' -doc_type: 'reference' ---- - -数値の合計を計算すると同時に、行数もカウントします。関数は ClickHouse のクエリオプティマイザによって使用されます。クエリ内に複数の `sum`、`count`、`avg` 関数がある場合、計算結果を再利用するために、それらを 1 つの `sumCount` 関数に置き換えることができます。この関数が明示的に必要になることはまれです。 - -**構文** - -```sql -sumCount(x) -``` - -**引数** - -* `x` — 入力値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md) のいずれかである必要があります。 - -**戻り値** - -* タプル `(sum, count)`。`sum` は数値の合計、`count` は NULL 以外の値が設定されている行数です。 - -型: [Tuple](../../../sql-reference/data-types/tuple.md)。 - -**例** - -クエリ: - -```sql -CREATE TABLE s_table (x Int8) ENGINE = Log; -INSERT INTO s_table SELECT number FROM numbers(0, 20); -INSERT INTO s_table VALUES (NULL); -SELECT sumCount(x) FROM s_table; -``` - -結果: - -```text -┌─sumCount(x)─┐ -│ (190,20) │ -└─────────────┘ -``` - -**関連項目** - -* [optimize_syntax_fuse_functions](../../../operations/settings/settings.md#optimize_syntax_fuse_functions) 設定を参照してください。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md deleted file mode 100644 index 650cc5f68aa..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Kahan 補償和アルゴリズムを用いて数値の合計を計算します' -sidebar_position: 197 -slug: /sql-reference/aggregate-functions/reference/sumkahan -title: 'sumKahan' -doc_type: 'reference' ---- - -[Kahan 補償和アルゴリズム](https://en.wikipedia.org/wiki/Kahan_summation_algorithm) を用いて数値の合計を計算します。 -[sum](./sum.md) 関数よりも処理が遅くなります。 -補償は [Float](../../../sql-reference/data-types/float.md) 型に対してのみ有効です。 - -**構文** - -```sql -sumKahan(x) -``` - -**引数** - -* `x` — 入力値。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md) である必要があります。 - -**戻り値** - -* 数値の合計。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、または [Decimal](../../../sql-reference/data-types/decimal.md) 型で、入力引数の型に依存します。 - -**例** - -クエリ: - -```sql -SELECT sum(0.1), sumKahan(0.1) FROM numbers(10); -``` - -結果: - -```text -┌───────────sum(0.1)─┬─sumKahan(0.1)─┐ -│ 0.9999999999999999 │ 1 │ -└────────────────────┴───────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md deleted file mode 100644 index f096e554f41..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: '`key` 配列で指定されたキーに従って、1つ以上の `value` 配列を集計します。キーをソート順に並べた配列と、それぞれのキーに対応する値をオーバーフローなしで合計した配列から成るタプルを返します。' -sidebar_position: 198 -slug: /sql-reference/aggregate-functions/reference/summap -title: 'sumMap' -doc_type: 'reference' ---- - -# sumMap {#summap} - -`key` 配列で指定されたキーに従って、1 つ以上の `value` 配列を合計します。ソート済みのキー配列と、それぞれのキーに対応する値をオーバーフローなしで合計した配列からなるタプルを返します。 - -**構文** - -* `sumMap(key , value1 [, value2 , ...])` [Array 型](../../data-types/array.md)。 -* `sumMap(Tuple(key [, value1 , value2 , ...]))` [Tuple 型](../../data-types/tuple.md)。 - -エイリアス: `sumMappedArrays`。 - -**引数** - -* `key`: キーの [Array](../../data-types/array.md)。 -* `value1`, `value2`, ...: 各キーごとに合計する値の [Array](../../data-types/array.md)。 - -キー配列と値配列からなるタプルを渡すことは、キー配列と値配列を個別に渡すことと同義です。 - -:::note -合計対象となる各行において、`key` とすべての `value` 配列の要素数は同じでなければなりません。 -::: - -**戻り値** - -* 配列のタプルを返します。最初の配列にはソート済みのキーが含まれ、それに続く配列にはそれぞれ対応するキーごとに合計された値が含まれます。 - -**例** - -まず `sum_map` というテーブルを作成し、いくつかのデータを挿入します。キーと値の配列は [Nested](../../data-types/nested-data-structures/index.md) 型の `statusMap` という列として個別に保存されており、上で説明したこの関数の 2 つの異なる構文の利用例を示すために、それらをまとめた [Tuple](../../data-types/tuple.md) 型の `statusMapTuple` という列としても保存されています。 - -クエリ: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt16, - requests UInt64 - ), - statusMapTuple Tuple(Array(Int32), Array(Int32)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -次に、`sumMap` 関数を使用してテーブルにクエリを実行し、配列型とタプル型の両方の構文を利用します。 - -クエリ: - -```sql -SELECT - timeslot, - sumMap(statusMap.status, statusMap.requests), - sumMap(statusMapTuple) -FROM sum_map -GROUP BY timeslot -``` - -結果: - -```text -┌────────────timeslot─┬─sumMap(statusMap.status, statusMap.requests)─┬─sumMap(statusMapTuple)─────────┐ -│ 2000-01-01 00:00:00 │ ([1,2,3,4,5],[10,10,20,10,10]) │ ([1,2,3,4,5],[10,10,20,10,10]) │ -│ 2000-01-01 00:01:00 │ ([4,5,6,7,8],[10,10,20,10,10]) │ ([4,5,6,7,8],[10,10,20,10,10]) │ -└─────────────────────┴──────────────────────────────────────────────┴────────────────────────────────┘ -``` - -**複数の値配列を扱う例** - -`sumMap` は、複数の値配列を一度に集約することもできます。 -これは、同じキーを共有する関連するメトリクスがある場合に有用です。 - -```sql title="Query" -CREATE TABLE multi_metrics( - date Date, - browser_metrics Nested( - browser String, - impressions UInt32, - clicks UInt32 - ) -) -ENGINE = MergeTree() -ORDER BY tuple(); - -INSERT INTO multi_metrics VALUES - ('2000-01-01', ['Firefox', 'Chrome'], [100, 200], [10, 25]), - ('2000-01-01', ['Chrome', 'Safari'], [150, 50], [20, 5]), - ('2000-01-01', ['Firefox', 'Edge'], [80, 40], [8, 4]); - -SELECT - sumMap(browser_metrics.browser, browser_metrics.impressions, browser_metrics.clicks) AS result -FROM multi_metrics; -``` - -```text title="Response" -┌─result────────────────────────────────────────────────────────────────────────┐ -│ (['Chrome', 'Edge', 'Firefox', 'Safari'], [350, 40, 180, 50], [45, 4, 18, 5]) │ -└───────────────────────────────────────────────────────────────────────────────┘ -``` - -この例では: - -* 結果のタプルには 3 つの配列が含まれます -* 1 つ目の配列: ソート済みのキー(ブラウザー名) -* 2 つ目の配列: 各ブラウザーの合計インプレッション数 -* 3 つ目の配列: 各ブラウザーの合計クリック数 - -**関連項目** - -* [Map データ型向けの Map コンビネーター](../combinators.md#-map) -* [sumMapWithOverflow](../reference/summapwithoverflow.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md deleted file mode 100644 index 87fce7a02fe..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -description: '`key` 配列で指定されたキーに基づいて `value` 配列を合計します。キーを昇順に並べた配列と、それぞれのキーに対応する値を合計した配列の 2 つからなるタプルを返します。sumMap 関数とは異なり、オーバーフローを許容して加算を行います。' -sidebar_position: 199 -slug: /sql-reference/aggregate-functions/reference/summapwithoverflow -title: 'sumMapWithOverflow' -doc_type: 'reference' ---- - -# sumMapWithOverflow {#summapwithoverflow} - -`key` 配列で指定されたキーに従って、`value` 配列の値を集計します。結果として 2 つの配列からなるタプルを返します。1 つ目はソートされたキーの配列、2 つ目は対応するキーごとに合計された値の配列です。 -この関数は [sumMap](../reference/summap.md) 関数と異なり、オーバーフローを許容する形で加算を行います。すなわち、集計結果のデータ型は引数のデータ型と同じになります。 - -**構文** - -* `sumMapWithOverflow(key , value )` [Array 型](../../data-types/array.md)。 -* `sumMapWithOverflow(Tuple(key , value ))` [Tuple 型](../../data-types/tuple.md)。 - -**引数** - -* `key`: キーの [Array](../../data-types/array.md)。 -* `value`: 値の [Array](../../data-types/array.md)。 - -キー配列と値配列のタプルを渡すことは、キー配列と値配列をそれぞれ個別に渡すことと同等です。 - -:::note -合計対象となる各行において、`key` と `value` の要素数は同じでなければなりません。 -::: - -**返される値** - -* 2 つの配列からなるタプルを返します。1 つ目はソートされたキーの配列、2 つ目は対応するキーごとに合計された値の配列です。 - -**例** - -まず `sum_map` というテーブルを作成し、いくつかデータを挿入します。キーと値の配列は、それぞれ [Nested](../../data-types/nested-data-structures/index.md) 型の `statusMap` というカラムとして個別に保存し、さらに両者をまとめて [tuple](../../data-types/tuple.md) 型の `statusMapTuple` というカラムとしても保存して、この関数で前述した 2 種類の構文の使い方を示します。 - -クエリ: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt8, - requests UInt8 - ), - statusMapTuple Tuple(Array(Int8), Array(Int8)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -`sumMap`、配列型の構文を使った `sumMapWithOverflow`、および `toTypeName` 関数を用いてテーブルをクエリすると、 -`sumMapWithOverflow` 関数では、合計された値の配列のデータ型が引数の型と同じく `UInt8` であることが分かります(つまり、オーバーフローを許容したまま加算が行われています)。一方、`sumMap` では合計された値の配列のデータ型が `UInt8` から `UInt64` に変更され、オーバーフローが発生しないようになっています。 - -Query: - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMap.status, statusMap.requests)), - toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests)), -FROM sum_map -GROUP BY timeslot -``` - -同様に、タプル構文を使っても同じ結果が得られます。 - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMapTuple)), - toTypeName(sumMapWithOverflow(statusMapTuple)), -FROM sum_map -GROUP BY timeslot -``` - -結果: - -```text - ┌────────────timeslot─┬─toTypeName(sumMap(statusMap.status, statusMap.requests))─┬─toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests))─┐ -1. │ 2000-01-01 00:01:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ -2. │ 2000-01-01 00:00:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ - └─────────────────────┴──────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [sumMap](../reference/summap.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md deleted file mode 100644 index 584e3333f1e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '入力パラメータと同じデータ型で数値の合計を計算します。合計がこのデータ型の最大値を超えた場合は、オーバーフローした値として計算されます。' -sidebar_position: 200 -slug: /sql-reference/aggregate-functions/reference/sumwithoverflow -title: 'sumWithOverflow' -doc_type: 'reference' ---- - -# sumWithOverflow {#sumwithoverflow} - -入力パラメータと同じデータ型を結果にも使用して数値の合計を計算します。このデータ型で表現できる最大値を超えた場合は、オーバーフローさせて計算します。 - -数値型に対してのみ使用できます。 - -**構文** - -```sql -sumWithOverflow(num) -``` - -**パラメーター** - -* `num`: 数値カラム。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**戻り値** - -* 値の合計。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**例** - -まず `employees` というテーブルを作成し、架空の従業員データを挿入します。この例では、これらの値の合計でオーバーフローが発生し得るように、`salary` を `UInt16` として選択します。 - -クエリ: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `monthly_salary` UInt16 -) -ENGINE = Log -``` - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow) -FROM employees -``` - -`sum` 関数と `sumWithOverflow` 関数を使って従業員の給与の合計を計算し、`toTypeName` 関数でそれぞれの型を表示します。 -`sum` 関数では結果の型は `UInt64` となり、合計値を格納するのに十分な大きさですが、`sumWithOverflow` では結果の型は `UInt16` のままです。 - -クエリ: - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow), -FROM employees; -``` - -結果: - -```response - ┌─no_overflow─┬─overflow─┬─toTypeName(no_overflow)─┬─toTypeName(overflow)─┐ -1. │ 118700 │ 53164 │ UInt64 │ UInt16 │ - └─────────────┴──────────┴─────────────────────────┴──────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md deleted file mode 100644 index d814683b08f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '`theilsU` 関数は、テーブル内の 2 つの列間の関連性を測定する指標である Theil の U 不確実性係数を計算します。' -sidebar_position: 201 -slug: /sql-reference/aggregate-functions/reference/theilsu -title: 'theilsU' -doc_type: 'reference' ---- - -# theilsU {#theilsu} - -`theilsU` 関数は、[Theil's U 不確実性係数](https://en.wikipedia.org/wiki/Contingency_table#Uncertainty_coefficient)を計算します。これは、テーブル内の 2 つの列間の関連性を測定する値です。値の範囲は −1.0(負の関連が 100%、または完全な反転)から +1.0(正の関連が 100%、または完全な一致)までです。値が 0.0 の場合は、関連が存在しないことを示します。 - -**構文** - -```sql -theilsU(column1, column2) -``` - -**引数** - -* `column1` と `column2` は比較対象となる列です - -**戻り値** - -* -1 から 1 の間の値 - -**戻り値の型** は常に [Float64](../../../sql-reference/data-types/float.md) です。 - -**例** - -以下で比較している 2 つの列は互いの関連性が低いため、`theilsU` の値は負になります。 - -```sql -SELECT - theilsU(a, b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -結果: - -```response -┌────────theilsU(a, b)─┐ -│ -0.30195720557678846 │ -└──────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md deleted file mode 100644 index e7a74171fd9..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL 風の変化量を計算する集約関数。' -sidebar_position: 229 -slug: /sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid -title: 'timeSeriesChangesToGrid' -doc_type: 'reference' ---- - -タイムスタンプと値のペアとして与えられる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される規則的な時間グリッド上で、そのデータから[PromQL 風の変化量](https://prometheus.io/docs/prometheus/latest/querying/functions/#changes)を計算する集約関数です。グリッド上の各ポイントに対して、`changes` を計算するためのサンプルは、指定された時間ウィンドウ内のものが対象になります。 - -Parameters: - -* `start timestamp` - グリッドの開始時刻を指定します -* `end timestamp` - グリッドの終了時刻を指定します -* `grid step` - グリッドのステップ(秒)を指定します -* `staleness` - 対象とするサンプルの最大の「staleness」(秒)を指定します - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `changes` の値を `Array(Nullable(Float64))` として返します。返される配列には、時間グリッドの各ポイントに対して 1 つの値が含まれます。特定のグリッドポイントについて、変化量を計算するためのサンプルがウィンドウ内に存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210, 225] 上の `changes` の値を計算します: - -```sql -WITH - -- 注記: 130と190の間隔は、windowパラメータに基づいてts = 180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 135 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドの間隔(秒) - 45 AS window_seconds -- "staleness"ウィンドウ(秒) -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -タイムスタンプと値の複数のサンプルを、同じ長さの配列として渡すこともできます。配列引数を使った同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。有効化するには `allow_experimental_ts_to_grid_aggregate_function=true` を設定してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md deleted file mode 100644 index ed5c679e6f4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL 風の delta を計算する集約関数。' -sidebar_position: 221 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid -title: 'timeSeriesDeltaToGrid' -doc_type: 'reference' ---- - -この集約関数は、タイムスタンプと値のペアからなる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される等間隔の時間グリッド上で、このデータから [PromQL 風の delta](https://prometheus.io/docs/prometheus/latest/querying/functions/#delta) を計算します。グリッド上の各ポイントに対して、`delta` の計算に用いるサンプルは、指定された時間ウィンドウ内のものが対象になります。 - -Parameters: - -* `start timestamp` - グリッドの開始を指定します。 -* `end timestamp` - グリッドの終了を指定します。 -* `grid step` - グリッドのステップを秒単位で指定します。 -* `staleness` - 対象とするサンプルの最大の「古さ(staleness)」を秒単位で指定します。staleness ウィンドウは左開・右閉の区間です。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `delta` の値を `Array(Nullable(Float64))` として返します。返される配列には、各時間グリッドポイントに対して 1 つの値が含まれます。特定のグリッドポイントについて、ウィンドウ内に delta 値を計算するのに十分なサンプルが存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `delta` 値を計算します。 - -```sql -WITH - -- 注記: 140と190の間隔は、windowパラメータに基づいてts = 150、165、180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ幅 - 45 AS window_seconds -- 「staleness」ウィンドウ -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesDeltaToGr⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,3,4.5,3.75,NULL,NULL,3.75] │ - └─────────────────────────────────────────┘ -``` - -また、同じ長さの配列として複数のタイムスタンプと値のサンプルを渡すことも可能です。配列引数を使った同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効化してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md deleted file mode 100644 index c02b7dbaba1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL 風の導関数を計算する集約関数。' -sidebar_position: 227 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid -title: 'timeSeriesDerivToGrid' -doc_type: 'reference' ---- - -タイムスタンプと値のペアとして与えられる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される規則的な時間グリッド上で、そのデータから [PromQL 風の導関数 (derivative)](https://prometheus.io/docs/prometheus/latest/querying/functions/#deriv) を計算する集約関数です。グリッド上の各ポイントについて、`deriv` を計算するためのサンプルは、指定された時間ウィンドウ内のものが対象となります。 - -Parameters: - -* `start timestamp` - グリッドの開始を指定します。 -* `end timestamp` - グリッドの終了を指定します。 -* `grid step` - グリッドのステップを秒単位で指定します。 -* `staleness` - 対象とするサンプルに許容される最大の「staleness」を秒単位で指定します。staleness ウィンドウは左開・右閉区間です。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `deriv` の値を `Array(Nullable(Float64))` として返します。返される配列には、時間グリッド上の各ポイントごとに 1 つの値が含まれます。特定のグリッドポイントについて、そのウィンドウ内に導関数の値を計算するのに十分なサンプルが存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `deriv` の値を計算します: - -```sql -WITH - -- 注記: 140と190の間隔は、windowパラメータに基づいてts = 150、165、180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記タイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドの間隔(秒) - 45 AS window_seconds -- "staleness"ウィンドウ(秒) -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,0.1,0.11,0.15,NULL,NULL,0.15] │ - └─────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -また、同じ長さの配列としてタイムスタンプと値の複数サンプルを渡すこともできます。配列引数を用いた同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効化してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md deleted file mode 100644 index 23fbf54684c..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'タイムスタンプで時系列データを昇順にソートします。' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/timeSeriesGroupArray -title: 'timeSeriesGroupArray' -doc_type: 'reference' ---- - -# timeSeriesGroupArray {#timeseriesgrouparray} - -タイムスタンプ順に時系列データを昇順で並べ替えます。 - -**構文** - -```sql -timeSeriesGroupArray(timestamp, value) -``` - -**引数** - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -**戻り値** - -この関数は、`timestamp` を昇順にソートしたタプル (`timestamp`, `value`) の配列を返します。 -同じ `timestamp` に複数の値がある場合、関数はそれらの中で最大の値を選択します。 - -**例** - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- 上記のタイムスタンプに対応する値の配列 -SELECT timeSeriesGroupArray(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を `timestamp`、`value` の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesGroupArray(timestamp, value)───────┐ -1. │ [(100,5),(110,1),(120,6),(130,8),(140,19)] │ - └──────────────────────────────────────────────┘ -``` - -複数のタイムスタンプと値のサンプルを、同じ長さの配列として渡すこともできます。配列引数を用いた同じクエリは次のとおりです: - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- 上記のタイムスタンプに対応する値の配列 -SELECT timeSeriesGroupArray(timestamps, values); -``` - -:::note -この関数は実験的機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効にしてください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md deleted file mode 100644 index 365dad404c8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL ライクな idelta を計算する集約関数。' -sidebar_position: 222 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid -title: 'timeSeriesInstantDeltaToGrid' -doc_type: 'reference' ---- - -この集約関数は、タイムスタンプと値のペアとして与えられた時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される規則的な時間グリッド上で [PromQL ライクな idelta](https://prometheus.io/docs/prometheus/latest/querying/functions/#idelta) を計算します。グリッド上の各ポイントについて、`idelta` を計算するためのサンプルは、指定された時間ウィンドウ内のものが考慮されます。 - -Parameters: - -* `start timestamp` - グリッドの開始を指定します。 -* `end timestamp` - グリッドの終了を指定します。 -* `grid step` - グリッドのステップを秒単位で指定します。 -* `staleness` - 対象とするサンプルの最大「staleness(古さ)」を秒単位で指定します。staleness ウィンドウは左開・右閉区間です。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `idelta` の値を `Array(Nullable(Float64))` として返します。返される配列には、時間グリッドの各ポイントごとに 1 つの値が含まれます。特定のグリッドポイントについて、インスタントデルタ値を計算するのに十分なサンプルがウィンドウ内に存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `idelta` 値を計算します: - -```sql -WITH - -- 注記: 140と190の間隔は、windowパラメータに基づいてts = 150、165、180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドの間隔(秒) - 45 AS window_seconds -- "staleness"ウィンドウ(秒) -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesInsta⋯stamps, values)─┐ -1. │ [NULL,NULL,0,2,1,1,NULL,NULL,3] │ - └─────────────────────────────────┘ -``` - -また、タイムスタンプと値のサンプルを、それぞれ同じ長さの配列として複数渡すこともできます。配列引数を用いた同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効化してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md deleted file mode 100644 index f39a82d474e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL ライクな irate を計算する集約関数。' -sidebar_position: 223 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid -title: 'timeSeriesInstantRateToGrid' -doc_type: 'reference' ---- - -タイムスタンプと値のペアとして与えられる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される規則的な時間グリッド上で、このデータから [PromQL ライクな irate](https://prometheus.io/docs/prometheus/latest/querying/functions/#irate) を計算する集約関数です。グリッド上の各点について、`irate` を計算するために使用するサンプルは、指定された時間ウィンドウ内のものが考慮されます。 - -Parameters: - -* `start timestamp` - グリッドの開始を指定します。 -* `end timestamp` - グリッドの終了を指定します。 -* `grid step` - グリッドのステップを秒単位で指定します。 -* `staleness` - 対象とするサンプルの最大の「古さ(staleness)」を秒で指定します。staleness ウィンドウは左開・右閉区間です。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `irate` の値を `Array(Nullable(Float64))` として返します。返される配列には、時間グリッドの各ポイントごとに 1 つの値が含まれます。特定のグリッドポイントに対して瞬時レート値を計算するのに十分なサンプルがウィンドウ内に存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `irate` の値を計算します。 - -```sql -WITH - -- 注記: 140と190の間隔は、windowパラメータに基づいてts = 150、165、180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドの間隔(秒) - 45 AS window_seconds -- 「staleness」ウィンドウ(秒) -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesInstantRa⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,0.2,0.1,0.1,NULL,NULL,0.3] │ - └─────────────────────────────────────────┘ -``` - -また、同じ長さの配列として複数のタイムスタンプと値のサンプルを渡すことも可能です。配列引数を使用した同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効にしてください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md deleted file mode 100644 index f82ae2dfa30..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -description: 'PromQL風の irate および idelta 計算のために時系列データを再サンプリングする集約関数' -sidebar_position: 224 -slug: /sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples -title: 'timeSeriesLastTwoSamples' -doc_type: 'reference' ---- - -タイムスタンプと値のペアとして渡された時系列データから、最大で直近2件のサンプルのみを保持する集約関数です。 - -引数: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値\ - また、同じサイズの配列として、複数のタイムスタンプと値のサンプルを渡すこともできます。 - -戻り値: -`Tuple(Array(DateTime), Array(Float64))` - 長さが 0 から 2 の範囲の、同じ長さを持つ 2 つの配列から成るペア。1 つ目の配列にはサンプリングされた時系列のタイムスタンプが含まれ、2 つ目の配列にはそれに対応する時系列の値が含まれます。 - -例: -この集約関数は、グリッドに揃えたタイムスタンプに対して再サンプリングされた時系列データを保存するマテリアライズドビューと集約テーブルでの利用を想定しています。 -以下の生データ用テーブルと、再サンプリングされたデータを保存するテーブルの例を考えます。 - -```sql --- 生データ用テーブル -CREATE TABLE t_raw_timeseries -( - metric_id UInt64, - timestamp DateTime64(3, 'UTC') CODEC(DoubleDelta, ZSTD), - value Float64 CODEC(DoubleDelta) -) -ENGINE = MergeTree() -ORDER BY (metric_id, timestamp); - --- より大きな時間間隔(15秒)にリサンプリングされたデータ用テーブル -CREATE TABLE t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), -- 15秒に整列されたタイムスタンプ - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -ENGINE = AggregatingMergeTree() -ORDER BY (metric_id, grid_timestamp); - --- リサンプリングテーブルにデータを投入するためのマテリアライズドビュー -CREATE MATERIALIZED VIEW mv_resampled_timeseries TO t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -AS SELECT - metric_id, - ceil(toUnixTimestamp(timestamp + interval 999 millisecond) / 15, 0) * 15 AS grid_timestamp, -- タイムスタンプを次のグリッドポイントに切り上げ - initializeAggregation('timeSeriesLastTwoSamplesState', timestamp, value) AS samples -FROM t_raw_timeseries -ORDER BY metric_id, grid_timestamp; -``` - -いくつかテストデータを挿入し、'2024-12-12 12:00:12' から '2024-12-12 12:00:30' の間のデータを読み取ります - -```sql --- データを挿入 -INSERT INTO t_raw_timeseries(metric_id, timestamp, value) SELECT number%10 AS metric_id, '2024-12-12 12:00:00'::DateTime64(3, 'UTC') + interval ((number/10)%100)*900 millisecond as timestamp, number%3+number%29 AS value FROM numbers(1000); - --- 生データを確認 -SELECT * -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN '2024-12-12 12:00:12' AND '2024-12-12 12:00:31' -ORDER BY metric_id, timestamp; -``` - - -```response -3 2024-12-12 12:00:12.870 29 -3 2024-12-12 12:00:13.770 8 -3 2024-12-12 12:00:14.670 19 -3 2024-12-12 12:00:15.570 30 -3 2024-12-12 12:00:16.470 9 -3 2024-12-12 12:00:17.370 20 -3 2024-12-12 12:00:18.270 2 -3 2024-12-12 12:00:19.170 10 -3 2024-12-12 12:00:20.070 21 -3 2024-12-12 12:00:20.970 3 -3 2024-12-12 12:00:21.870 11 -3 2024-12-12 12:00:22.770 22 -3 2024-12-12 12:00:23.670 4 -3 2024-12-12 12:00:24.570 12 -3 2024-12-12 12:00:25.470 23 -3 2024-12-12 12:00:26.370 5 -3 2024-12-12 12:00:27.270 13 -3 2024-12-12 12:00:28.170 24 -3 2024-12-12 12:00:29.069 6 -3 2024-12-12 12:00:29.969 14 -3 2024-12-12 12:00:30.869 25 -``` - -タイムスタンプ '2024-12-12 12:00:15' および '2024-12-12 12:00:30' の直近 2 件のサンプルをクエリします: - -```sql --- リサンプリングされたデータを確認 -SELECT metric_id, grid_timestamp, (finalizeAggregation(samples).1 as timestamp, finalizeAggregation(samples).2 as value) -FROM t_resampled_timeseries_15_sec -WHERE metric_id = 3 AND grid_timestamp BETWEEN '2024-12-12 12:00:15' AND '2024-12-12 12:00:30' -ORDER BY metric_id, grid_timestamp; -``` - -```response -3 2024-12-12 12:00:15 (['2024-12-12 12:00:14.670','2024-12-12 12:00:13.770'],[19,8]) -3 2024-12-12 12:00:30 (['2024-12-12 12:00:29.969','2024-12-12 12:00:29.069'],[14,6]) -``` - -集約テーブルは、15 秒間隔に揃えられた各タイムスタンプについて、最新 2 つの値のみを保存します。これにより、生データのテーブルに保存されているデータ量よりはるかに少ないデータを読み込むだけで、PromQL の `irate` や `idelta` と同様の計算を行うことができます。 - -```sql --- 生データからideltaとirateを計算 -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- タイムスタンプグリッドの開始時刻 - start_ts + INTERVAL 60 SECOND AS end_ts, -- タイムスタンプグリッドの終了時刻 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ間隔 - 45 AS window_seconds -- 「staleness」ウィンドウ -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - - -```sql --- 再サンプリングされたデータからideltaとirateを計算 -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- タイムスタンプグリッドの開始時刻 - start_ts + INTERVAL 60 SECOND AS end_ts, -- タイムスタンプグリッドの終了時刻 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ間隔 - 45 AS window_seconds -- 「staleness」ウィンドウ -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values) -FROM ( - SELECT - metric_id, - finalizeAggregation(samples).1 AS timestamps, - finalizeAggregation(samples).2 AS values - FROM t_resampled_timeseries_15_sec - WHERE metric_id = 3 AND grid_timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -) -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - -:::note -この関数は実験的機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効化してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md deleted file mode 100644 index 554c10f2e5b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL に類似した線形予測を計算する集約関数。' -sidebar_position: 228 -slug: /sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid -title: 'timeSeriesPredictLinearToGrid' -doc_type: 'reference' ---- - -この集約関数は、タイムスタンプと値のペアからなる時系列データを受け取り、開始タイムスタンプ、終了タイムスタンプ、およびステップで記述される規則的な時間グリッド上で、このデータに基づき指定された予測タイムスタンプのオフセットを用いた[PromQL に類似した線形予測](https://prometheus.io/docs/prometheus/latest/querying/functions/#predict_linear)を計算します。グリッド上の各ポイントについて、`predict_linear` を計算するためのサンプルは、指定された時間ウィンドウ内のものが考慮されます。 - -Parameters: - -* `start timestamp` - グリッドの開始を指定します。 -* `end timestamp` - グリッドの終了を指定します。 -* `grid step` - グリッドのステップ(秒)を指定します。 -* `staleness` - 対象とするサンプルの最大の「古さ」(秒)を指定します。staleness ウィンドウは左開・右閉の区間です。 -* `predict_offset` - 予測時刻に加算するオフセット秒数を指定します。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `predict_linear` の値を `Array(Nullable(Float64))` として返します。返される配列には、時間グリッドの各ポイントに 1 つの値が含まれます。特定のグリッドポイントについて、そのウィンドウ内に予測値を計算するのに十分なサンプルが存在しない場合、その値は NULL になります。 - -Example: -次のクエリは、60 秒のオフセット付きで、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `predict_linear` の値を計算します。 - -```sql -WITH - -- 注記: 140と190の間の間隔は、windowパラメータに基づいてts = 150、165、180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ幅 - 45 AS window_seconds, -- "staleness"ウィンドウ - 60 AS predict_offset -- 予測時間オフセット -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -応答: - -```response - ┌─timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value)─┐ -1. │ [NULL,NULL,1,9.166667,11.6,16.916666,NULL,NULL,16.5] │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -同じ長さの配列として、複数のタイムスタンプと値を渡すこともできます。配列引数を使った同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds, - 60 AS predict_offset -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効化してください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md deleted file mode 100644 index 8dda3ff4d5f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL 風の rate を計算する集約関数。' -sidebar_position: 225 -slug: /sql-reference/aggregate-functions/reference/timeSeriesRateToGrid -title: 'timeSeriesRateToGrid' -doc_type: 'reference' ---- - -タイムスタンプと値のペアからなる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップによって定義される等間隔の時間グリッド上で、このデータから [PromQL 風の rate](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) を計算する集約関数です。グリッド上の各ポイントについて、`rate` を計算するために使用するサンプルは、指定された時間ウィンドウ内のもののみが考慮されます。 - -Parameters: - -* `start timestamp` - グリッドの開始時刻を指定します。 -* `end timestamp` - グリッドの終了時刻を指定します。 -* `grid step` - グリッドのステップを秒単位で指定します。 -* `staleness` - 対象とするサンプルの最大の「staleness」(古さ)を秒単位で指定します。staleness ウィンドウは左開・右閉区間です。 - -Arguments: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -Return value: -指定されたグリッド上の `rate` の値を `Array(Nullable(Float64))` として返します。返される配列には、各時間グリッドポイントごとに 1 つの値が含まれます。特定のグリッドポイントに対して、そのウィンドウ内に rate を計算するのに十分なサンプルがない場合、その値は NULL になります。 - -Example: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] 上の `rate` の値を計算します。 - -```sql -WITH - -- NOTE: 140 と 190 の間のギャップは、ウィンドウパラメータに従って ts = 150, 165, 180 の値がどのように埋められるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始時刻 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了時刻 - 15 AS step_seconds, -- タイムスタンプグリッドの刻み幅(秒) - 45 AS window_seconds -- 「staleness」ウィンドウ -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を `timestamp`, `value` の行に展開します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesRateToGrid(start_ts, ⋯w_seconds)(timestamps, values)─┐ -1. │ [NULL,NULL,0,0.06666667,0.1,0.083333336,NULL,NULL,0.083333336] │ - └────────────────────────────────────────────────────────────────┘ -``` - -また、同じ長さの配列として複数のタイムスタンプと値のサンプルを渡すことも可能です。配列引数を用いた同じクエリは次のとおりです: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効にしてください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md deleted file mode 100644 index d019cb4061f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '指定されたグリッドにタイムシリーズデータを再サンプリングする集約関数。' -sidebar_position: 226 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness -title: 'timeSeriesResampleToGridWithStaleness' -doc_type: 'reference' ---- - -タイムスタンプと値のペアとして与えられたタイムシリーズデータを、開始タイムスタンプ・終了タイムスタンプ・ステップによって定義される等間隔の時間グリッドに再サンプリングする集約関数です。グリッド上の各ポイントについて、(指定された時間ウィンドウ内で)最も新しいサンプルが選択されます。 - -エイリアス: `timeSeriesLastToGrid`。 - -パラメータ: - -* `start timestamp` - グリッドの開始時刻を指定します -* `end timestamp` - グリッドの終了時刻を指定します -* `grid step` - グリッドのステップ(秒)を指定します -* `staleness window` - 最新サンプルに許容される最大の「staleness」(古さ)を秒で指定します - -引数: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応するタイムシリーズの値 - -戻り値: -指定されたグリッドに再サンプリングされたタイムシリーズの値を `Array(Nullable(Float64))` として返します。返される配列には、各時間グリッドポイントに1つの値が含まれます。特定のグリッドポイントに対応するサンプルが存在しない場合、その値は NULL になります。 - -例: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210] に対して、それぞれのグリッドポイントについて 30 秒より古くない値を選択することでタイムシリーズデータを再サンプリングします: - -```sql -WITH - -- 注意: 140 と 190 の間のギャップは、staleness window パラメータに基づいて ts = 150, 165, 180 の値がどのように補間されるかを示すためのものです - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始時刻 - 90 + 120 AS end_ts, -- タイムスタンプグリッドの終了時刻 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ幅(秒) - 30 AS window_seconds -- 「staleness」ウィンドウ -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を `timestamp`, `value` の行に展開します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesResa⋯stamp, value)─┐ -1. │ [NULL,NULL,1,3,4,4,NULL,5,8] │ - └──────────────────────────────┘ -``` - -また、タイムスタンプと値のサンプルを、同じ長さの配列として複数渡すことも可能です。配列引数を用いた同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 30 AS window_seconds -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効にしてください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md deleted file mode 100644 index 908d5caa4f8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定されたグリッド上の時系列データに対して、PromQL 風の resets を計算する集約関数。' -sidebar_position: 230 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid -title: 'timeSeriesResetsToGrid' -doc_type: 'reference' ---- - -この集約関数は、タイムスタンプと値のペアからなる時系列データを受け取り、開始タイムスタンプ・終了タイムスタンプ・ステップで定義される一定間隔の時間グリッド上で、このデータから[PromQL 風の resets](https://prometheus.io/docs/prometheus/latest/querying/functions/#resets) を計算します。グリッド上の各ポイントについて、`resets` を計算するために使用するサンプルは、指定された時間ウィンドウ内のものが対象となります。 - -パラメータ: - -* `start timestamp` - グリッドの開始時刻を指定 -* `end timestamp` - グリッドの終了時刻を指定 -* `grid step` - グリッドのステップ幅を秒単位で指定 -* `staleness` - 対象とするサンプルの許容される最大の「staleness」(古さ)を秒単位で指定 - -引数: - -* `timestamp` - サンプルのタイムスタンプ -* `value` - `timestamp` に対応する時系列の値 - -戻り値: -指定されたグリッド上の `resets` の値を `Array(Nullable(Float64))` として返します。返される配列には、各時間グリッドポイントごとに 1 つの値が含まれます。特定のグリッドポイントについて、そのウィンドウ内に resets の値を計算するためのサンプルが存在しない場合、その要素は NULL になります。 - -例: -次のクエリは、グリッド [90, 105, 120, 135, 150, 165, 180, 195, 210, 225] 上の `resets` の値を計算します。 - -```sql -WITH - -- 注記: 130と190の間隔は、windowパラメータに基づいてts = 180の値がどのように補完されるかを示すためのものです - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, -- 上記のタイムスタンプに対応する値の配列 - 90 AS start_ts, -- タイムスタンプグリッドの開始位置 - 90 + 135 AS end_ts, -- タイムスタンプグリッドの終了位置 - 15 AS step_seconds, -- タイムスタンプグリッドのステップ幅 - 45 AS window_seconds -- 「staleness」ウィンドウ -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- このサブクエリは、タイムスタンプと値の配列を`timestamp`、`value`の行形式に変換します - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -レスポンス: - -```response - ┌─timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -また、複数のタイムスタンプと値のサンプルを、同じ長さの配列として渡すこともできます。配列引数を用いた同じクエリは次のとおりです。 - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -この関数は実験的な機能です。`allow_experimental_ts_to_grid_aggregate_function=true` を設定して有効にしてください。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md deleted file mode 100644 index 4091467984a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '指定された列内で、おおよそ最も頻度の高い値の配列を返します。結果の配列は、値そのものではなく、値のおおよその出現頻度の降順でソートされます。さらに、値の重みも頻度の算出時に考慮されます。' -sidebar_position: 203 -slug: /sql-reference/aggregate-functions/reference/topkweighted -title: 'topKWeighted' -doc_type: 'reference' ---- - -# topKWeighted {#topkweighted} - -指定した列内で、概ね最も頻出する値を要素とする配列を返します。返される配列は、値そのものではなく、値のおおよその出現頻度が高い順にソートされます。加えて、値の重みも考慮されます。 - -**構文** - -```sql -topKWeighted(N)(column, weight) -topKWeighted(N, load_factor)(column, weight) -topKWeighted(N, load_factor, 'counts')(column, weight) -``` - -**パラメーター** - -* `N` — 返す要素数。省略可能。デフォルト値: 10。 -* `load_factor` — 値のために予約するセルの数を定義します。uniq(column) > N * load_factor の場合、topK 関数の結果は近似値になります。省略可能。デフォルト値: 3。 -* `counts` — 結果に近似的なカウント値と誤差値を含めるかどうかを定義します。 - -**引数** - -* `column` — 値。 -* `weight` — 重み。頻度計算において、各値は `weight` 回分として計上されます。[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返される値** - -重みの近似的な合計が最大となる値の配列を返します。 - -**例** - -クエリ: - -```sql -SELECT topKWeighted(2)(k, w) FROM -VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -結果: - -```text -┌─topKWeighted(2)(k, w)──┐ -│ ['z','x'] │ -└────────────────────────┘ -``` - -クエリ: - -```sql -SELECT topKWeighted(2, 10, 'counts')(k, w) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -結果: - -```text -┌─topKWeighted(2, 10, 'counts')(k, w)─┐ -│ [('z',10,0),('x',5,0)] │ -└─────────────────────────────────────┘ -``` - -**関連項目** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md deleted file mode 100644 index e404bb0b44d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: '引数の異なる値の個数を近似的に算出します。' -sidebar_position: 204 -slug: /sql-reference/aggregate-functions/reference/uniq -title: 'uniq' -doc_type: 'reference' ---- - -# uniq {#uniq} - -引数に含まれる異なる値のおおよその個数を計算します。 - -```sql -uniq(x[, ...]) -``` - -**引数** - -この関数は可変個のパラメーターを受け取ります。パラメーターには `Tuple`、`Array`、`Date`、`DateTime`、`String`、または数値型を指定できます。 - -**戻り値** - -* [UInt64](../../../sql-reference/data-types/int-uint.md) 型の数値。 - -**実装の詳細** - -関数は次の処理を行います: - -* 集約内のすべてのパラメーターに対してハッシュ値を計算し、その値を内部計算に使用します。 - -* アダプティブサンプリングアルゴリズムを使用します。計算状態として、要素のハッシュ値のサンプルを最大 65536 個まで保持します。このアルゴリズムは非常に高精度であり、CPU 上で非常に効率的です。クエリにこれらの関数が複数含まれている場合でも、`uniq` の使用は他の集約関数とほぼ同等の速度です。 - -* 決定論的な結果を返します(クエリの処理順序には依存しません)。 - -この関数はほぼすべてのケースでの使用を推奨します。 - -**関連項目** - -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md deleted file mode 100644 index 770813bf4b2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '引数の異なる値のおおよその個数を計算します。' -sidebar_position: 205 -slug: /sql-reference/aggregate-functions/reference/uniqcombined -title: 'uniqCombined' -doc_type: 'reference' ---- - -# uniqCombined {#uniqcombined} - -異なる引数値のおおよその個数を計算します。 - -```sql -uniqCombined(HLL_precision)(x[, ...]) -``` - -`uniqCombined` 関数は、異なる値の個数を計算するのに適した関数です。 - -**Arguments** - -* `HLL_precision`: [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) におけるセル数の 2 を底とする対数値。省略可能で、`uniqCombined(x[, ...])` のように関数を使用できます。`HLL_precision` のデフォルト値は 17 で、これはおおよそ 96 KiB の領域(2^17 個のセル、各 6 ビット)に相当します。 -* `X`: 可変長のパラメーター。パラメーターには `Tuple`、`Array`、`Date`、`DateTime`、`String`、または数値型を指定できます。 - -**Returned value** - -* [UInt64](../../../sql-reference/data-types/int-uint.md) 型の数値。 - -**Implementation details** - -`uniqCombined` 関数は次のように動作します。 - -* 集計対象のすべてのパラメーターに対してハッシュ(`String` には 64 ビットハッシュ、それ以外には 32 ビットハッシュ)を計算し、そのハッシュ値を用いて計算を行います。 -* 3 つのアルゴリズム(配列、ハッシュテーブル、誤差補正テーブル付き HyperLogLog)を組み合わせて使用します。 - * 相異なる要素数が少ない場合は配列を使用します。 - * 集合のサイズがより大きくなるとハッシュテーブルを使用します。 - * 要素数がさらに大きい場合は HyperLogLog を使用し、一定量のメモリのみを使用します。 -* 決定的な結果を返します(クエリの処理順序には依存しません)。 - -:::note\ -非 `String` 型には 32 ビットハッシュを使用するため、`UINT_MAX` を大きく超えるカーディナリティに対しては誤差が非常に大きくなります(数百億件程度の相異なる値を超えると急速に誤差が増加します)。したがって、このような場合は [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) を使用する必要があります。 -::: - -[uniq](/sql-reference/aggregate-functions/reference/uniq) 関数と比較すると、`uniqCombined` 関数は次の特性を持ちます。 - -* 使用メモリ量が数倍少ない。 -* 計算精度が数倍高い。 -* 通常は若干性能が低くなります。一部のシナリオでは、たとえば多くの集約状態をネットワーク越しに送信する分散クエリでは、`uniqCombined` の方が `uniq` より高速になることがあります。 - -**Example** - -Query: - -```sql -SELECT uniqCombined(number) FROM numbers(1e6); -``` - -結果: - -```response -┌─uniqCombined(number)─┐ -│ 1001148 │ -- 100万 -└──────────────────────┘ -``` - -はるかに大きな入力に対する `uniqCombined` と `uniqCombined64` の違いの例については、[uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) の例のセクションを参照してください。 - -**関連項目** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md deleted file mode 100644 index d594a6adee8..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: '異なる引数値のおおよその個数を計算します。uniqCombined と同様ですが、String データ型だけでなく、すべてのデータ型に対して 64 ビットハッシュを使用します。' -sidebar_position: 206 -slug: /sql-reference/aggregate-functions/reference/uniqcombined64 -title: 'uniqCombined64' -doc_type: 'reference' ---- - -# uniqCombined64 {#uniqcombined64} - -異なる引数値のおおよその個数を計算します。[uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) と同様ですが、`String` データ型だけでなく、すべてのデータ型に対して 64 ビットのハッシュを使用します。 - -```sql -uniqCombined64(HLL_precision)(x[, ...]) -``` - -**パラメーター** - -* `HLL_precision`: [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) におけるセル数の 2 を底とする対数。オプションとして、関数を `uniqCombined64(x[, ...])` のように使用できます。`HLL_precision` のデフォルト値は 17 で、これは実質的に 96 KiB のメモリを使用します(2^17 個のセル、各セル 6 ビット)。 -* `X`: 可変個のパラメーター。パラメーターには `Tuple`、`Array`、`Date`、`DateTime`、`String`、数値型を指定できます。 - -**戻り値** - -* [UInt64](../../../sql-reference/data-types/int-uint.md) 型の数値。 - -**実装の詳細** - -`uniqCombined64` 関数は次のように動作します。 - -* 集約内のすべてのパラメーターに対してハッシュ(すべてのデータ型に対する 64 ビットハッシュ)を計算し、その値を用いて計算を行います。 -* 3 つのアルゴリズム(配列、ハッシュテーブル、誤差補正テーブル付き HyperLogLog)を組み合わせて使用します。 - * 相異なる要素数が少ない場合は、配列を使用します。 - * 集合のサイズが大きくなると、ハッシュテーブルを使用します。 - * さらに要素数が多い場合は、一定量のメモリを使用する HyperLogLog を使用します。 -* 結果は決定的であり(クエリの処理順序に依存しません)、常に同じ値を返します。 - -:::note -すべての型に対して 64 ビットハッシュを使用するため、非 `String` 型に対して 32 ビットハッシュを使用する [uniqCombined](../../../sql-reference/aggregate-functions/reference/uniqcombined.md) とは異なり、`UINT_MAX` を大きく超えるカーディナリティに対しても非常に大きな誤差は生じません。 -::: - -[uniq](/sql-reference/aggregate-functions/reference/uniq) 関数と比較して、`uniqCombined64` 関数は次の特徴があります。 - -* 使用するメモリ量が数倍少ない。 -* 計算精度が数倍高い。 - -**例** - -以下の例では、`uniqCombined64` を `1e10` 個の異なる数値に対して実行し、異なる引数値の個数に非常に近い近似値を返します。 - -クエリ: - -```sql -SELECT uniqCombined64(number) FROM numbers(1e10); -``` - -結果: - -```response -┌─uniqCombined64(number)─┐ -│ 9998568925 │ -- 100億 -└────────────────────────┘ -``` - -比較すると、この程度のサイズの入力に対しては、`uniqCombined` 関数はあまり精度の高くない近似結果を返します。 - -クエリ: - -```sql -SELECT uniqCombined(number) FROM numbers(1e10); -``` - -結果: - -```response -┌─uniqCombined(number)─┐ -│ 5545308725 │ -- 55億4530万 -└──────────────────────┘ -``` - -**関連項目** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md deleted file mode 100644 index 92c9e516627..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '引数の異なる値の個数を正確に計算します。' -sidebar_position: 207 -slug: /sql-reference/aggregate-functions/reference/uniqexact -title: 'uniqExact' -doc_type: 'reference' ---- - -# uniqExact {#uniqexact} - -引数の異なる値の数を正確に計算します。 - -```sql -uniqExact(x[, ...]) -``` - -厳密な結果がどうしても必要な場合は `uniqExact` 関数を使用してください。それ以外の場合は [uniq](/sql-reference/aggregate-functions/reference/uniq) 関数を使用します。 - -`uniqExact` 関数は `uniq` よりも多くのメモリを使用します。これは、異なる値の数が増えるにつれて状態のサイズが際限なく増加するためです。 - -**引数** - -この関数は任意個のパラメータを受け取ります。パラメータには `Tuple`、`Array`、`Date`、`DateTime`、`String`、または数値型を指定できます。 - -**例** - -この例では、`uniqExact` 関数を使用して、[OpenSky データセット](https://sql.clickhouse.com?query=U0VMRUNUIHVuaXFFeGFjdCh0eXBlY29kZSkgRlJPTSBvcGVuc2t5Lm9wZW5za3k&) 内の一意なタイプコード(航空機の型式を表す短い識別子)の数を数えます。 - -```sql title="Query" -SELECT uniqExact(typecode) FROM opensky.opensky -``` - -```response title="Response" -1106 -``` - -**関連項目** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md deleted file mode 100644 index 985a0703a0d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'HyperLogLog アルゴリズムを使用して、異なる引数値の個数を近似的に計算します。' -sidebar_position: 208 -slug: /sql-reference/aggregate-functions/reference/uniqhll12 -title: 'uniqHLL12' -doc_type: 'reference' ---- - -# uniqHLL12 {#uniqhll12} - -[HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) アルゴリズムを使用して、引数の異なる値のおおよその個数を計算します。 - -```sql -uniqHLL12(x[, ...]) -``` - -**引数** - -この関数は可変個の引数を取ります。引数には `Tuple`、`Array`、`Date`、`DateTime`、`String`、または数値型を指定できます。 - -**戻り値** - -* [UInt64](../../../sql-reference/data-types/int-uint.md) 型の数値。 - -**実装の詳細** - -この関数は次のように動作します: - -* 集約内のすべての引数に対してハッシュ値を計算し、それを内部計算に使用します。 - -* HyperLogLog アルゴリズムを使用して、異なる引数値の個数を近似します。 - - 2^12 個の 5 ビットセルが使用されます。状態のサイズは 2.5 KB をわずかに上回ります。小さいデータセット(<10K 要素)では結果の精度はあまり高くなく(誤差は最大で約 10%)、高カーディナリティのデータセット(10K〜100M)では結果は比較的正確で、最大誤差は約 1.6% です。100M を超えると推定誤差は増加し、極めて高いカーディナリティ(1B+ 要素)のデータセットに対しては非常に不正確な結果を返します。 - -* 決定論的な結果を返します(クエリ処理の順序に依存しません)。 - -この関数の使用は推奨しません。ほとんどの場合、代わりに [uniq](/sql-reference/aggregate-functions/reference/uniq) 関数または [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) 関数を使用してください。 - -**関連項目** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md deleted file mode 100644 index d04e9c3c395..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Theta Sketch Framework を使用して、異なる引数値のおおよその個数を計算します。' -sidebar_position: 209 -slug: /sql-reference/aggregate-functions/reference/uniqthetasketch -title: 'uniqTheta' -doc_type: 'reference' ---- - -[Theta Sketch Framework](https://datasketches.apache.org/docs/Theta/ThetaSketches.html#theta-sketch-framework) を使用して、異なる引数値のおおよその個数を計算します。 - -```sql -uniqTheta(x[, ...]) -``` - -**引数** - -この関数は可変個のパラメータを受け取ります。パラメータには `Tuple`、`Array`、`Date`、`DateTime`、`String`、または数値型を指定できます。 - -**戻り値** - -* [UInt64](../../../sql-reference/data-types/int-uint.md) 型の数値。 - -**実装の詳細** - -関数は次のように動作します: - -* 集約内のすべてのパラメータに対してハッシュ値を計算し、そのハッシュを計算に利用します。 - -* [KMV](https://datasketches.apache.org/docs/Theta/InverseEstimate.html) アルゴリズムを使用して、異なる引数値の個数を近似します。 - - 4096 (2^12) 個の 64 ビットスケッチが使用されます。状態のサイズは約 41 KB です。 - -* 相対誤差は 3.125%(95% 信頼水準)です。詳細は [relative error table](https://datasketches.apache.org/docs/Theta/ThetaErrorTable.html) を参照してください。 - -**関連項目** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md deleted file mode 100644 index da10a31cffa..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '母分散を計算します。' -sidebar_position: 210 -slug: /sql-reference/aggregate-functions/reference/varPop -title: 'varPop' -doc_type: 'reference' ---- - - - -## varPop {#varpop} - -母集団分散を計算します: - -$$ -\frac{\Sigma{(x - \bar{x})^2}}{n} -$$ - -**構文** - -```sql -varPop(x) -``` - -エイリアス: `VAR_POP` - -**パラメータ** - -- `x`: 母集団分散を求める値の母集団。[(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal\*](../../data-types/decimal.md)。 - -**戻り値** - -- `x` の母集団分散を返します。[`Float64`](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3), (3), (3), (4), (4), (5), (5), (7), (11), (15); - -SELECT - varPop(x) AS var_pop -FROM test_data; -``` - -結果: - -```response -┌─var_pop─┐ -│ 14.4 │ -└─────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md deleted file mode 100644 index 8c15c1cbd8d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: '母分散を返します。varPop と異なり、この関数は数値的に安定したアルゴリズムを使用します。処理速度は低下しますが、計算誤差をより小さく抑えられます。' -sidebar_position: 211 -slug: /sql-reference/aggregate-functions/reference/varpopstable -title: 'varPopStable' -doc_type: 'reference' ---- - -## varPopStable {#varpopstable} - -母分散を返します。[`varPop`](../reference/varpop.md) と異なり、この関数は[数値的に安定な](https://en.wikipedia.org/wiki/Numerical_stability)アルゴリズムを使用します。処理は遅くなりますが、計算誤差を小さく抑えることができます。 - -**構文** - -```sql -varPopStable(x) -``` - -別名: `VAR_POP_STABLE`. - -**パラメーター** - -* `x`: 分散を計算する対象となる値の母集団。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返り値** - -* `x` の母分散を返します。[Float64](../../data-types/float.md)。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - varPopStable(x) AS var_pop_stable -FROM test_data; -``` - -結果: - -```response -┌─var_pop_stable─┐ -│ 14.4 │ -└────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md deleted file mode 100644 index ed491d54302..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -description: 'データセットの標本分散を計算します。' -sidebar_position: 212 -slug: /sql-reference/aggregate-functions/reference/varSamp -title: 'varSamp' -doc_type: 'reference' ---- - - - -## varSamp {#varsamp} - -データセットの標本分散を計算します。 - -**構文** - -```sql -varSamp(x) -``` - -エイリアス: `VAR_SAMP` - -**パラメータ** - -- `x`: 標本分散を計算する対象のデータ。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal\*](../../data-types/decimal.md) - -**戻り値** - -- 入力データセット `x` の標本分散を返します。[Float64](../../data-types/float.md) - -**実装の詳細** - -`varSamp` 関数は、以下の式を使用して標本分散を計算します: - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -ここで: - -- `x` はデータセット内の各データポイント -- `mean(x)` はデータセットの算術平均 -- `n` はデータセット内のデータポイント数 - -この関数は、入力データセットがより大きな母集団からの標本であることを前提としています。母集団全体の分散を計算する場合(完全なデータセットがある場合)は、代わりに [`varPop`](../reference/varpop.md) を使用してください。 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSamp(x),3) AS var_samp FROM test_data; -``` - -レスポンス: - -```response -┌─var_samp─┐ -│ 0.865 │ -└──────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md deleted file mode 100644 index c403ae95087..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'データセットの標本分散を計算します。`varSamp` とは異なり、この関数は数値的に安定したアルゴリズムを使用します。動作は遅くなりますが、計算誤差をより小さく抑えられます。' -sidebar_position: 213 -slug: /sql-reference/aggregate-functions/reference/varsampstable -title: 'varSampStable' -doc_type: 'reference' ---- - - - -## varSampStable {#varsampstable} - -データセットの標本分散を計算します。[`varSamp`](../reference/varsamp.md)とは異なり、この関数は数値的に安定したアルゴリズムを使用します。処理速度は遅くなりますが、計算誤差が小さくなります。 - -**構文** - -```sql -varSampStable(x) -``` - -別名: `VAR_SAMP_STABLE` - -**パラメータ** - -- `x`: 標本分散を計算する対象のデータセット。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal\*](../../data-types/decimal.md)。 - -**戻り値** - -- 入力データセットの標本分散を返します。[Float64](../../data-types/float.md)。 - -**実装の詳細** - -`varSampStable`関数は、[`varSamp`](../reference/varsamp.md)と同じ式を使用して標本分散を計算します: - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -ここで: - -- `x`はデータセット内の各データポイント -- `mean(x)`はデータセットの算術平均 -- `n`はデータセット内のデータポイント数 - -**例** - -クエリ: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSampStable(x),3) AS var_samp_stable FROM test_data; -``` - -レスポンス: - -```response -┌─var_samp_stable─┐ -│ 0.865 │ -└─────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md deleted file mode 100644 index bea639f0f24..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '2つの母集団から得られた標本に Welch の t 検定を適用します。' -sidebar_label: 'welchTTest' -sidebar_position: 214 -slug: /sql-reference/aggregate-functions/reference/welchttest -title: 'welchTTest' -doc_type: 'reference' ---- - -# welchTTest {#welchttest} - -2つの母集団からの標本に Welch の t 検定を適用します。 - -**構文** - -```sql -welchTTest([confidence_level])(sample_data, sample_index) -``` - -両方のサンプルの値は `sample_data` カラムにあります。`sample_index` が 0 の場合、その行の値は第1母集団からのサンプルに属します。それ以外の場合は第2母集団からのサンプルに属します。 -帰無仮説は、母集団の平均が等しいというものです。母集団は正規分布に従うと仮定します。母集団の分散は等しくない場合があります。 - -**引数** - -* `sample_data` — サンプルデータ。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) または [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — サンプルのインデックス。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**パラメータ** - -* `confidence_level` — 信頼区間を計算するための信頼水準。[Float](../../../sql-reference/data-types/float.md)。 - -**返される値** - -[Tuple](../../../sql-reference/data-types/tuple.md)。要素数は 2 つまたは 4 つ(オプションの `confidence_level` が指定されている場合は 4 つ) - -* 計算された t統計量。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された p値。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された信頼区間の下限。[Float64](../../../sql-reference/data-types/float.md)。 -* 計算された信頼区間の上限。[Float64](../../../sql-reference/data-types/float.md)。 - -**例** - -入力テーブル: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 22.1 │ 0 │ -│ 21.9 │ 0 │ -│ 18.9 │ 1 │ -│ 20.3 │ 1 │ -│ 19 │ 1 │ -└─────────────┴──────────────┘ -``` - -クエリ: - -```sql -SELECT welchTTest(sample_data, sample_index) FROM welch_ttest; -``` - -結果: - -```text -┌─welchTTest(sample_data, sample_index)─────┐ -│ (2.7988719532211235,0.051807360348581945) │ -└───────────────────────────────────────────┘ -``` - -**関連項目** - -* [Welch's t-test](https://en.wikipedia.org/wiki/Welch%27s_t-test) -* [studentTTest 関数](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md deleted file mode 100644 index 202d55ef2f2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -description: '集約関数の中間状態を格納する ClickHouse の AggregateFunction データ型に関するドキュメント' -keywords: ['AggregateFunction', 'Type'] -sidebar_label: 'AggregateFunction' -sidebar_position: 46 -slug: /sql-reference/data-types/aggregatefunction -title: 'AggregateFunction 型' -doc_type: 'reference' ---- - -# AggregateFunction 型 {#aggregatefunction-type} - -## 説明 {#description} - -ClickHouse のすべての [集約関数](/sql-reference/aggregate-functions) には、 -実装固有の中間状態があり、それを `AggregateFunction` データ型としてシリアル化して -テーブルに保存できます。これは通常、[マテリアライズドビュー](../../sql-reference/statements/create/view.md) -を用いて行うのが一般的です。 - -`AggregateFunction` 型とともに一般的に用いられる集約関数の -[コンビネータ](/sql-reference/aggregate-functions/combinators) が 2 つあります: - -- 集約関数名に付けると `AggregateFunction` の中間状態を生成する - [`-State`](/sql-reference/aggregate-functions/combinators#-state) 集約関数コンビネータ。 -- 中間状態から集約の最終結果を取得するために使用される - [`-Merge`](/sql-reference/aggregate-functions/combinators#-merge) 集約関数コンビネータ。 - -## 構文 {#syntax} - -```sql -AggregateFunction(集約関数名, 引数の型...) -``` - -**パラメータ** - -* `aggregate_function_name` - 集約関数の名前。関数がパラメータ付きである場合は、そのパラメータも指定する必要があります。 -* `types_of_arguments` - 集約関数の引数の型。 - -例: - -```sql -CREATE TABLE t -( - column1 AggregateFunction(uniq, UInt64), - column2 AggregateFunction(anyIf, String, UInt8), - column3 AggregateFunction(quantiles(0.5, 0.9), UInt64) -) ENGINE = ... -``` - - -## 使用方法 {#usage} - -### データ挿入 {#data-insertion} - -`AggregateFunction` 型のカラムを持つテーブルにデータを挿入するには、 -集約関数と -[`-State`](/sql-reference/aggregate-functions/combinators#-state) -という集約関数コンビネータを用いた `INSERT SELECT` を使用できます。 - -例えば、型が `AggregateFunction(uniq, UInt64)` および -`AggregateFunction(quantiles(0.5, 0.9), UInt64)` のカラムにデータを挿入する場合は、 -次のようなコンビネータ付きの集約関数を使用します。 - -```sql -uniqState(UserID) -quantilesState(0.5, 0.9)(SendTiming) -``` - -`uniq` および `quantiles` 関数とは対照的に、`uniqState` と `quantilesState` -(`-State` コンビネータが付与されたもの)は、最終値ではなく状態を返します。 -言い換えると、これらは `AggregateFunction` 型の値を返します。 - -`SELECT` クエリの結果では、`AggregateFunction` 型の値は、 -すべての ClickHouse の出力フォーマットにおいて、実装依存のバイナリ表現を持ちます。 - -入力値から状態を構築できるようにする、セッションレベルの特別な設定 `aggregate_function_input_format` があります。 -これは次のフォーマットをサポートします。 - -* `state` - シリアライズされた状態を含むバイナリ文字列(デフォルト)。 - たとえば、`SELECT` クエリで `TabSeparated` フォーマットにデータをダンプした場合、 - このダンプは `INSERT` クエリを使って再読み込みできます。 -* `value` - フォーマットは集約関数の引数の単一の値、もしくは複数引数の場合はそれらのタプルを受け取り、それをデシリアライズして対応する状態を構成します。 -* `array` - フォーマットは上記の `value` オプションで説明したような値の Array を受け取り、その配列内のすべての要素を集約して状態を構成します。 - - -### データの選択 {#data-selection} - -`AggregatingMergeTree` テーブルからデータを選択する場合は、データを挿入したときと同じ集約関数を `GROUP BY` 句とともに使用しますが、[`-Merge`](/sql-reference/aggregate-functions/combinators#-merge) コンビネータを付けて使用します。 - -`-Merge` コンビネータが付いた集約関数は、一連の状態を受け取り、それらを結合して、完全なデータ集約の結果を返します。 - -例えば、次の 2 つのクエリは同じ結果を返します。 - -```sql -SELECT uniq(UserID) FROM table - -SELECT uniqMerge(state) FROM (SELECT uniqState(UserID) AS state FROM table GROUP BY RegionID) -``` - - -## 使用例 {#usage-example} - -[AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) エンジンの説明を参照してください。 - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Using Aggregate Combinators in ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -- [MergeState](/sql-reference/aggregate-functions/combinators#-mergestate) - コンビネータ -- [State](/sql-reference/aggregate-functions/combinators#-state) コンビネータ \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md deleted file mode 100644 index 2def2a3045b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'ClickHouse の Array データ型に関するドキュメント' -sidebar_label: 'Array(T)' -sidebar_position: 32 -slug: /sql-reference/data-types/array -title: 'Array(T)' -doc_type: 'reference' ---- - -# Array(T) {#arrayt} - -`T` 型の要素からなる配列で、配列の先頭インデックスは 1 です。`T` は配列を含む任意のデータ型になり得ます。 - -## 配列の作成 {#creating-an-array} - -関数を使用して配列を作成できます: - -```sql -array(T) -``` - -角括弧([])を使用することもできます。 - -```sql -[] -``` - -配列の作成例: - -```sql -SELECT array(1, 2) AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName(array(1, 2))─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴─────────────────────────┘ -``` - -```sql -SELECT [1, 2] AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName([1, 2])─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴────────────────────┘ -``` - -## データ型の扱い {#working-with-data-types} - -配列をその場で作成する場合、ClickHouse は、指定されたすべての引数を格納できる中で最も狭いデータ型を自動的に選択します。[Nullable](/sql-reference/data-types/nullable) やリテラルの [NULL](/operations/settings/formats#input_format_null_as_default) 値が含まれている場合、配列要素の型も [Nullable](../../sql-reference/data-types/nullable.md) になります。 - -ClickHouse がデータ型を決定できない場合は、例外をスローします。例えば、文字列と数値を同時に含む配列を作成しようとした場合(`SELECT array(1, 'a')`)にこのような状況が発生します。 - -自動データ型推定の例: - -```sql -SELECT array(1, 2, NULL) AS x, toTypeName(x) -``` - -```text -┌─x──────────┬─toTypeName(array(1, 2, NULL))─┐ -│ [1,2,NULL] │ Array(Nullable(UInt8)) │ -└────────────┴───────────────────────────────┘ -``` - -互換性のないデータ型の配列を作成しようとすると、ClickHouse は例外を発生させます。 - -```sql -SELECT array(1, 'a') -``` - -```text -サーバーから例外を受信しました (バージョン 1.1.54388): -Code: 386. DB::Exception: Received from localhost:9000, 127.0.0.1. DB::Exception: UInt8 型と String 型に共通のスーパータイプが存在しません。一部が String/FixedString 型であり、一部がそうでないためです。 -``` - -## 配列サイズ {#array-size} - -`size0` サブカラムを使用すると、列全体を読み込むことなく配列のサイズを取得できます。多次元配列の場合は `sizeN-1` を使用できます。ここで `N` は取得したい次元の番号です。 - -**例** - -クエリ: - -```sql -CREATE TABLE t_arr (`arr` Array(Array(Array(UInt32)))) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO t_arr VALUES ([[[12, 13, 0, 1],[12]]]); - -SELECT arr.size0, arr.size1, arr.size2 FROM t_arr; -``` - -結果: - -```text -┌─arr.size0─┬─arr.size1─┬─arr.size2─┐ -│ 1 │ [2] │ [[4,1]] │ -└───────────┴───────────┴───────────┘ -``` - -## Array からのネストされたサブカラムの読み取り {#reading-nested-subcolumns-from-array} - -`Array` 内のネストされた型 `T` がサブカラムを持つ場合(たとえば [named tuple](./tuple.md) である場合など)、`Array(T)` 型から同じサブカラム名を使ってサブカラムを読み取ることができます。サブカラムの型は、元のサブカラムの型を要素とする `Array` 型になります。 - -**例** - -```sql -CREATE TABLE t_arr (arr Array(Tuple(field1 UInt32, field2 String))) ENGINE = MergeTree ORDER BY tuple(); -INSERT INTO t_arr VALUES ([(1, 'Hello'), (2, 'World')]), ([(3, 'This'), (4, 'is'), (5, 'subcolumn')]); -SELECT arr.field1, toTypeName(arr.field1), arr.field2, toTypeName(arr.field2) from t_arr; -``` - -```test -┌─arr.field1─┬─toTypeName(arr.field1)─┬─arr.field2────────────────┬─toTypeName(arr.field2)─┐ -│ [1,2] │ Array(UInt32) │ ['Hello','World'] │ Array(String) │ -│ [3,4,5] │ Array(UInt32) │ ['This','is','subcolumn'] │ Array(String) │ -└────────────┴────────────────────────┴───────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md deleted file mode 100644 index f7b9c1a30db..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'ClickHouse の Boolean データ型に関するドキュメント' -sidebar_label: 'Boolean' -sidebar_position: 33 -slug: /sql-reference/data-types/boolean -title: 'Bool' -doc_type: 'reference' ---- - -# Bool {#bool} - -型 `bool` は内部的に UInt8 型として保存されます。取りうる値は `true` (1) と `false` (0) です。 - -```sql -SELECT true AS col, toTypeName(col); -┌─col──┬─toTypeName(true)─┐ -│ true │ Bool │ -└──────┴──────────────────┘ - -select true == 1 as col, toTypeName(col); -┌─col─┬─toTypeName(equals(true, 1))─┐ -│ 1 │ UInt8 │ -└─────┴─────────────────────────────┘ -``` - -```sql -CREATE TABLE test_bool -( - `A` Int64, - `B` Bool -) -ENGINE = Memory; - -INSERT INTO test_bool VALUES (1, true),(2,0); - -SELECT * FROM test_bool; -┌─A─┬─B─────┐ -│ 1 │ true │ -│ 2 │ false │ -└───┴───────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md deleted file mode 100644 index 9557aefad49..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: 'データ型のバイナリエンコーディング仕様に関するドキュメント' -sidebar_label: 'データ型のバイナリエンコーディング仕様' -sidebar_position: 56 -slug: /sql-reference/data-types/data-types-binary-encoding -title: 'データ型のバイナリエンコーディング仕様' -doc_type: 'reference' ---- - -# データ型のバイナリエンコーディング仕様 {#data-types-binary-encoding-specification} - -この仕様では、ClickHouse のデータ型のバイナリエンコードおよびデコードに使用できるバイナリ形式について説明します。この形式は `Dynamic` カラムの[バイナリシリアル化](dynamic.md#binary-output-format)で使用され、対応する設定の下で入出力フォーマット [RowBinaryWithNamesAndTypes](/interfaces/formats/RowBinaryWithNamesAndTypes) および [Native](/interfaces/formats/Native) でも使用できます。 - -次の表では、各データ型がバイナリ形式でどのように表現されるかを説明します。各データ型のエンコードは、型を示す 1 バイトと、必要に応じて追加される情報で構成されます。 -バイナリエンコーディングにおける `var_uint` は、サイズが Variable-Length Quantity 圧縮を用いてエンコードされていることを意味します。 - -| ClickHouse のデータ型 | バイナリ符号化 | -| --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Nothing` | `0x00` | -| `UInt8` | `0x01` | -| `UInt16` | `0x02` | -| `UInt32` | `0x03` | -| `UInt64` | `0x04` | -| `UInt128` | `0x05` | -| `UInt256` | `0x06` | -| `Int8` | `0x07` | -| `Int16` | `0x08` | -| `Int32` | `0x09` | -| `Int64` | `0x0A` | -| `Int128` | `0x0B` | -| `Int256` | `0x0C` | -| `Float32` | `0x0D` | -| `Float64` | `0x0E` | -| `Date` | `0x0F` | -| `Date32` | `0x10` | -| `DateTime` | `0x11` | -| `DateTime(time_zone)` | `0x12` | -| `DateTime64(P)` | `0x13` | -| `DateTime64(P, time_zone)` | `0x14` | -| `String` | `0x15` | -| `FixedString(N)` | `0x16` | -| `Enum8` | `0x17...` | -| `Enum16` | `0x18...>` | -| `Decimal32(P, S)` | `0x19` | -| `Decimal64(P, S)` | `0x1A` | -| `Decimal128(P, S)` | `0x1B` | -| `Decimal256(P, S)` | `0x1C` | -| `UUID` | `0x1D` | -| `Array(T)` | `0x1E` | -| `Tuple(T1, ..., TN)` | `0x1F...` | -| `Tuple(name1 T1, ..., nameN TN)` | `0x20...` | -| `Set` | `0x21` | -| `Interval` | `0x22`([interval kind のバイナリ表現](#interval-kind-binary-encoding)を参照) | -| `Nullable(T)` | `0x23` | -| `Function` | `0x24...` | -| `AggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x25......` ([集約関数パラメータのバイナリ符号化](#aggregate-function-parameter-binary-encoding)を参照) | -| `LowCardinality(T)` | `0x26` | -| `Map(K, V)` | `0x27` | -| `IPv4` | `0x28` | -| `IPv6` | `0x29` | -| `Variant(T1, ..., TN)` | `0x2A...` | -| `Dynamic(max_types=N)` | `0x2B` | -| `カスタム型` (`Ring`, `Polygon` など) | `0x2C` | -| `Bool` | `0x2D` | -| `SimpleAggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x2E......`([集約関数パラメータのバイナリエンコーディング](#aggregate-function-parameter-binary-encoding)を参照) | -| `Nested(name1 T1, ..., nameN TN)` | `0x2F...` | -| `JSON(max_dynamic_paths=N, max_dynamic_types=M, path Type, SKIP skip_path, SKIP REGEXP skip_path_regexp)` | `0x30.........` | -| `BFloat16` | `0x31` | -| `時刻` | `0x32` | -| `Time64(P)` | `0x34` | -| `QBit(T, N)` | `0x36` | - -`JSON` 型の場合、バイト `uint8_serialization_version` はシリアル化のバージョンを表します。現在のところバージョンは常に 0 ですが、今後 `JSON` 型に新しい引数が導入された場合には変更される可能性があります。 - -### Interval 種別のバイナリ エンコーディング {#interval-kind-binary-encoding} - -次の表は、`Interval` データ型のさまざまな種別がどのようにバイナリ形式でエンコードされるかを示します。 - -| Interval の種別 | バイナリ エンコーディング | -|-----------------|----------------------------| -| `Nanosecond` | `0x00` | -| `Microsecond` | `0x01` | -| `Millisecond` | `0x02` | -| `Second` | `0x03` | -| `Minute` | `0x04` | -| `Hour` | `0x05` | -| `Day` | `0x06` | -| `Week` | `0x07` | -| `Month` | `0x08` | -| `Quarter` | `0x09` | -| `Year` | `0x1A` | - -### 集約関数パラメータのバイナリエンコーディング {#aggregate-function-parameter-binary-encoding} - -次の表は、`AggregateFunction` および `SimpleAggregateFunction` のパラメータがどのようにエンコードされるかを示します。 -パラメータのエンコードは、パラメータの型を示す 1 バイトと、その値そのものから構成されます。 - -| パラメータ型 | バイナリエンコード形式 | -|--------------------------|--------------------------------------------------------------------------------------------------------------------------------| -| `Null` | `0x00` | -| `UInt64` | `0x01` | -| `Int64` | `0x02` | -| `UInt128` | `0x03` | -| `Int128` | `0x04` | -| `UInt256` | `0x05` | -| `Int256` | `0x06` | -| `Float64` | `0x07` | -| `Decimal32` | `0x08` | -| `Decimal64` | `0x09` | -| `Decimal128` | `0x0A` | -| `Decimal256` | `0x0B` | -| `String` | `0x0C` | -| `Array` | `0x0D...` | -| `Tuple` | `0x0E...` | -| `Map` | `0x0F...` | -| `IPv4` | `0x10` | -| `IPv6` | `0x11` | -| `UUID` | `0x12` | -| `Bool` | `0x13` | -| `Object` | `0x14...` | -| `AggregateFunctionState` | `0x15` | -| `Negative infinity` | `0xFE` | -| `Positive infinity` | `0xFF` | \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md deleted file mode 100644 index d71bf60de84..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'ClickHouse における Date データ型に関するドキュメント' -sidebar_label: 'Date' -sidebar_position: 12 -slug: /sql-reference/data-types/date -title: 'Date' -doc_type: 'reference' ---- - -# Date {#date} - -日付。2バイトの符号なし整数値として、1970-01-01 からの経過日数の形で保存されます。Unixエポックの開始直後から、コンパイル時に定数として定義される上限までの値を保存できます(現在は 2149 年までですが、完全にサポートされる最終年は 2148 年です)。 - -サポートされる値の範囲: [1970-01-01, 2149-06-06]。 - -日付値はタイムゾーン情報なしで保存されます。 - -**例** - -`Date` 型のカラムを持つテーブルを作成し、データを挿入する例: - -```sql -CREATE TABLE dt -( - `timestamp` Date, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 日付の解析 --- - 文字列から --- - 1970-01-01からの日数として解釈される「小さい」整数から --- - 1970-01-01からの秒数として解釈される「大きい」整数から -INSERT INTO dt VALUES ('2019-01-01', 1), (17897, 2), (1546300800, 3); - -SELECT * FROM dt; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2019-01-01 │ 1 │ -│ 2019-01-01 │ 2 │ -│ 2019-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**関連項目** - -* [日付と時刻を扱う関数](../../sql-reference/functions/date-time-functions.md) -* [日付と時刻を扱う演算子](../../sql-reference/operators#operators-for-working-with-dates-and-times) -* [`DateTime` データ型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md deleted file mode 100644 index 0400ca8adc4..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'ClickHouse における Date32 データ型に関するドキュメント。Date 型と比べて、より広い範囲の日付を保存できます' -sidebar_label: 'Date32' -sidebar_position: 14 -slug: /sql-reference/data-types/date32 -title: 'Date32' -doc_type: 'reference' ---- - -# Date32 {#date32} - -日付型です。[DateTime64](../../sql-reference/data-types/datetime64.md) と同じ日付範囲をサポートします。ネイティブバイトオーダーの符号付き 32 ビット整数として保存され、その値は `1900-01-01` からの経過日数を表します。**重要!** 0 は `1970-01-01` を表し、負の値は `1970-01-01` より前の日付を表します。 - -**例** - -`Date32` 型のカラムを持つテーブルを作成し、データを挿入する例: - -```sql -CREATE TABLE dt32 -( - `timestamp` Date32, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 日付の解析 --- - 文字列から --- - 1970-01-01からの日数として解釈される「小さい」整数から --- - 1970-01-01からの秒数として解釈される「大きい」整数から -INSERT INTO dt32 VALUES ('2100-01-01', 1), (47482, 2), (4102444800, 3); - -SELECT * FROM dt32; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2100-01-01 │ 1 │ -│ 2100-01-01 │ 2 │ -│ 2100-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**関連項目** - -* [toDate32](../../sql-reference/functions/type-conversion-functions.md#todate32) -* [toDate32OrZero](/sql-reference/functions/type-conversion-functions#todate32orzero) -* [toDate32OrNull](/sql-reference/functions/type-conversion-functions#todate32ornull) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md deleted file mode 100644 index deb98f27ab5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -description: 'ClickHouse の DateTime データ型に関するドキュメント。秒単位の精度でタイムスタンプを格納します' -sidebar_label: 'DateTime' -sidebar_position: 16 -slug: /sql-reference/data-types/datetime -title: 'DateTime' -doc_type: 'reference' ---- - - - -# DateTime {#datetime} - -カレンダー形式の日付と一日の時刻で表現できる、時間上の瞬間を保存します。 - -構文: - -```sql -DateTime([timezone]) -``` - -サポートされる値の範囲: [1970-01-01 00:00:00, 2106-02-07 06:28:15]。 - -精度: 1秒。 - - -## 速度 {#speed} - -`Date` データ型は、_ほとんど_ の場合において `DateTime` より高速です。 - -`Date` 型は 2 バイトのストレージを必要としますが、`DateTime` 型は 4 バイトを必要とします。ただし、圧縮時には、`Date` と `DateTime` のサイズ差はさらに大きくなります。これは、`DateTime` に含まれる分と秒が圧縮されにくいためです。`DateTime` ではなく `Date` でフィルタリングおよび集計を行う方が、高速です。 - - - -## 使用上の注意 {#usage-remarks} - -時刻は、タイムゾーンや夏時間に関係なく [Unix タイムスタンプ](https://en.wikipedia.org/wiki/Unix_time) として保存されます。タイムゾーンは、`DateTime` 型の値がテキスト形式でどのように表示されるか、および文字列として指定された値(`'2020-01-01 05:00:01'`)がどのようにパースされるかに影響します。 - -テーブルにはタイムゾーンに依存しない Unix タイムスタンプが保存され、タイムゾーンはデータのインポート/エクスポート時にそれをテキスト形式へ、またはその逆へ変換したり、値に対して暦に基づく計算(例: `toDate`, `toHour` 関数など)を行うために使用されます。タイムゾーンはテーブルの行(または結果セット)には保存されず、カラムのメタデータに保存されます。 - -サポートされているタイムゾーンの一覧は [IANA Time Zone Database](https://www.iana.org/time-zones) で確認でき、`SELECT * FROM system.time_zones` によって問い合わせることもできます。[一覧](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) は Wikipedia にも掲載されています。 - -テーブル作成時に、`DateTime` 型カラムに対して明示的にタイムゾーンを設定できます。例: `DateTime('UTC')`。タイムゾーンが設定されていない場合、ClickHouse はサーバー設定における [timezone](../../operations/server-configuration-parameters/settings.md#timezone) パラメータの値、もしくは ClickHouse サーバー起動時点のオペレーティングシステムの設定値を使用します。 - -[clickhouse-client](../../interfaces/cli.md) は、データ型の初期化時にタイムゾーンが明示的に設定されていない場合、デフォルトでサーバーのタイムゾーンを適用します。クライアント側のタイムゾーンを使用するには、`--use_client_time_zone` パラメータを付けて `clickhouse-client` を実行します。 - -ClickHouse は、[date_time_output_format](../../operations/settings/settings-formats.md#date_time_output_format) 設定の値に応じて値を出力します。デフォルトでは `YYYY-MM-DD hh:mm:ss` 形式のテキストで出力されます。さらに、[formatDateTime](../../sql-reference/functions/date-time-functions.md#formatDateTime) 関数を使用して出力形式を変更できます。 - -ClickHouse にデータを挿入する際には、[date_time_input_format](../../operations/settings/settings-formats.md#date_time_input_format) 設定の値に応じて、さまざまな形式の日付および時刻文字列を使用できます。 - - - -## 例 {#examples} - -**1.** `DateTime` 型の列を持つテーブルを作成し、そのテーブルにデータを挿入する: - -```sql -CREATE TABLE dt -( - `timestamp` DateTime('Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- DateTimeの解析 --- - 文字列から --- - 1970-01-01からの経過秒数として解釈される整数から -INSERT INTO dt VALUES ('2019-01-01 00:00:00', 1), (1546300800, 2); - -SELECT * FROM dt; -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -│ 2019-01-01 03:00:00 │ 2 │ -└─────────────────────┴──────────┘ -``` - -* datetime を整数として挿入する場合、それは Unix タイムスタンプ (UTC) として扱われます。`1546300800` は UTC の `'2019-01-01 00:00:00'` を表します。ただし、`timestamp` 列には `Asia/Istanbul` (UTC+3) のタイムゾーンが指定されているため、文字列として出力すると値は `'2019-01-01 03:00:00'` と表示されます。 -* 文字列値を datetime として挿入する場合、その値は列に指定されているタイムゾーンの時刻として解釈されます。`'2019-01-01 00:00:00'` は `Asia/Istanbul` タイムゾーンの時刻として扱われ、`1546290000` として保存されます。 - -**2.** `DateTime` 値でのフィルタリング - -```sql -SELECT * FROM dt WHERE timestamp = toDateTime('2019-01-01 00:00:00', 'Asia/Istanbul') -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -`DateTime` 列の値は、`WHERE` 句で文字列値を使ってフィルタリングできます。文字列は自動的に `DateTime` に変換されます。 - -```sql -SELECT * FROM dt WHERE timestamp = '2019-01-01 00:00:00' -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -**3.** `DateTime` 型列のタイムゾーンを取得する: - -```sql -SELECT toDateTime(now(), 'Asia/Istanbul') AS column, toTypeName(column) AS x -``` - -```text -┌──────────────column─┬─x─────────────────────────┐ -│ 2019-10-16 04:12:04 │ DateTime('Asia/Istanbul') │ -└─────────────────────┴───────────────────────────┘ -``` - -**4.** タイムゾーンの変換 - -```sql -SELECT -toDateTime(timestamp, 'Europe/London') AS lon_time, -toDateTime(timestamp, 'Asia/Istanbul') AS mos_time -FROM dt -``` - -```text -┌───────────lon_time──┬────────────mos_time─┐ -│ 2019-01-01 00:00:00 │ 2019-01-01 03:00:00 │ -│ 2018-12-31 21:00:00 │ 2019-01-01 00:00:00 │ -└─────────────────────┴─────────────────────┘ -``` - -タイムゾーン変換はメタデータのみを変更するため、この操作に計算コストは発生しません。 - - -## タイムゾーンサポートの制限事項 {#limitations-on-time-zones-support} - -一部のタイムゾーンは完全にはサポートされていない場合があります。次のようなケースがあります。 - -UTC からのオフセットが 15 分の倍数ではない場合、時と分の計算が正しくないことがあります。たとえば、リベリアのモンロビアのタイムゾーンは、1972 年 1 月 7 日以前は UTC -0:44:30 のオフセットを持っていました。モンロビアのタイムゾーンにおける過去時刻の計算を行うと、時刻処理関数が誤った結果を返す可能性があります。ただし、1972 年 1 月 7 日以降の結果は正しくなります。 - -時間の切り替え(夏時間やその他の理由による)が、15 分の倍数ではない時刻で行われた場合、その特定の日の計算結果が誤ってしまうことがあります。 - -暦日が単調に増加しないケース。たとえば、Happy Valley - Goose Bay では、2010 年 11 月 7 日 00:01:00(真夜中の 1 分後)に時間が 1 時間戻されました。その結果、11 月 6 日が終わった後に 11 月 7 日を 1 分間だけ迎え、その後時間が 11 月 6 日 23:01 に戻され、さらに 59 分経過した後に再び 11 月 7 日が始まりました。ClickHouse は(まだ)このような楽しいケースをサポートしていません。これらの日付においては、時刻処理関数の結果がわずかに不正確になる可能性があります。 - -同様の問題が、2010 年の Casey 南極基地にもあります。3 月 5 日 02:00 に時間を 3 時間戻しました。もしあなたが南極基地で作業している場合でも、安心して ClickHouse を使用できます。ただし、タイムゾーンを UTC に設定するか、若干の不正確さが生じうることを理解しておいてください。 - -複数日にわたる時間のシフト。一部の太平洋の島々は、タイムゾーンのオフセットを UTC+14 から UTC-12 に変更しました。これは問題ありませんが、そのタイムゾーンを用いて、切り替えが行われた日の過去時点の計算を行うと、いくらかの不正確さが生じる可能性があります。 - - - -## 夏時間(DST)の扱い {#handling-daylight-saving-time-dst} - -タイムゾーン付きの ClickHouse の `DateTime` 型は、夏時間(Daylight Saving Time, DST)の切り替え時に、特に次のような場合に予期しない動作を示すことがあります。 - -* [`date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) が `simple` に設定されている場合 -* 時計が戻され(「Fall Back」)、1 時間分の重複が発生する場合 -* 時計が進められ(「Spring Forward」)、1 時間分の欠損が発生する場合 - -デフォルトでは、ClickHouse は常に重複する時刻のうち先に出現する方を選択し、時計が進められる際に存在しない時刻を有効な時刻として解釈してしまうことがあります。 - -例として、夏時間(DST)から標準時への次のような切り替えを考えてみます。 - -* 2023 年 10 月 29 日の 02:00:00 に、時計が 01:00:00 に戻されます(BST → GMT)。 -* 01:00:00 ~ 01:59:59 の 1 時間が(BST と GMT のそれぞれで)2 回出現します。 -* ClickHouse は常に最初の出現(BST)を選択するため、時間間隔を加算する際に予期しない結果を引き起こす可能性があります。 - -```sql -SELECT '2023-10-29 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-10-29 01:30:00 │ 2023-10-29 01:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -同様に、標準時から夏時間への切り替え時には、1時間分の時刻が飛ばされたように見えることがあります。 - -例: - -* 2023年3月26日の `00:59:59` に、時計は `02:00:00` に進みます(GMT → BST)。 -* `01:00:00` ~ `01:59:59` の1時間は存在しません。 - -```sql -SELECT '2023-03-26 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-03-26 00:30:00 │ 2023-03-26 02:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -この場合、ClickHouse は存在しない時刻 `2023-03-26 01:30:00` を、ひとつ前の時刻である `2023-03-26 00:30:00` にずらします。 - - -## 関連項目 {#see-also} - -- [型変換関数](../../sql-reference/functions/type-conversion-functions.md) -- [日付と時刻を扱う関数](../../sql-reference/functions/date-time-functions.md) -- [配列を扱う関数](../../sql-reference/functions/array-functions.md) -- [`date_time_input_format` 設定](../../operations/settings/settings-formats.md#date_time_input_format) -- [`date_time_output_format` 設定](../../operations/settings/settings-formats.md#date_time_output_format) -- [`timezone` サーバー構成パラメータ](../../operations/server-configuration-parameters/settings.md#timezone) -- [`session_timezone` 設定](../../operations/settings/settings.md#session_timezone) -- [日付と時刻を扱う演算子](../../sql-reference/operators#operators-for-working-with-dates-and-times) -- [`Date` データ型](../../sql-reference/data-types/date.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md deleted file mode 100644 index bdce7493a12..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -description: 'サブ秒精度のタイムスタンプを格納する ClickHouse の DateTime64 データ型に関するドキュメント' -sidebar_label: 'DateTime64' -sidebar_position: 18 -slug: /sql-reference/data-types/datetime64 -title: 'DateTime64' -doc_type: 'reference' ---- - -# DateTime64 {#datetime64} - -カレンダー日付と一日の時刻で表現できる時点を、サブ秒精度を指定して保存できるデータ型です。 - -ティックサイズ(精度)は 10-precision 秒です。指定可能な範囲: [ 0 : 9 ]。\ -通常は 3(ミリ秒)、6(マイクロ秒)、9(ナノ秒)が使用されます。 - -**構文:** - -```sql -DateTime64(精度, [タイムゾーン]) -``` - -内部的には、エポック開始(1970-01-01 00:00:00 UTC)からの「ティック」数としてデータを Int64 で格納します。ティックの分解能は precision パラメータによって決まります。さらに、`DateTime64` 型では列全体で共通のタイムゾーンを保持でき、このタイムゾーンが `DateTime64` 型の値のテキスト形式での表示方法や、文字列として指定された値('2020-01-01 05:00:01.000')のパース方法に影響します。タイムゾーンはテーブルの行(または結果セット)には保存されず、列メタデータとして保存されます。詳細は [DateTime](../../sql-reference/data-types/datetime.md) を参照してください。 - -サポートされる値の範囲: [1900-01-01 00:00:00, 2299-12-31 23:59:59.999999999] - -小数点以下の桁数は precision パラメータに依存します。 - -注記: 最大値に対する precision は 8 です。最大の 9 桁(ナノ秒)の precision を使用する場合、UTC におけるサポートされる最大値は `2262-04-11 23:47:16` です。 - -## 例 {#examples} - -1. `DateTime64` 型の列を持つテーブルを作成し、データを挿入する: - -```sql -CREATE TABLE dt64 -( - `timestamp` DateTime64(3, 'Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- DateTime を解析する --- - 整数値を(精度 3 のため)1970-01-01 からのマイクロ秒数として解釈する --- - 小数値を、小数点より前の部分を秒数として、小数点以下の桁数に基づいて解釈する --- - 文字列から解釈する -INSERT INTO dt64 VALUES (1546300800123, 1), (1546300800.123, 2), ('2019-01-01 00:00:00', 3); - -SELECT * FROM dt64; -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -* datetime を整数として挿入する場合、それは適切にスケーリングされた Unix タイムスタンプ (UTC) として扱われます。精度 3 の `1546300800000` は UTC の `'2019-01-01 00:00:00'` を表します。ただし、`timestamp` 列にはタイムゾーンとして `Asia/Istanbul` (UTC+3) が指定されているため、文字列として出力すると、値は `'2019-01-01 03:00:00'` と表示されます。datetime を小数として挿入する場合も整数と同様に扱われますが、小数点より前の値は秒までを含む Unix タイムスタンプであり、小数点より後ろの値は精度として扱われます。 -* 文字列値を datetime として挿入する場合、それは当該列のタイムゾーンの時刻として解釈されます。`'2019-01-01 00:00:00'` は `Asia/Istanbul` タイムゾーンの時刻として扱われ、`1546290000000` として保存されます。 - -2. `DateTime64` 値でのフィルタリング - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64('2019-01-01 00:00:00', 3, 'Asia/Istanbul'); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -`DateTime` と異なり、`DateTime64` の値は `String` 型から自動的には変換されません。 - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64(1546300800.123, 3); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -└─────────────────────────┴──────────┘ -``` - -挿入する場合とは異なり、`toDateTime64` 関数はすべての値を小数形式として扱うため、小数点以下の精度を指定する必要があります。 - -3. `DateTime64` 型の値に対してタイムゾーンを取得する方法: - -```sql -SELECT toDateTime64(now(), 3, 'Asia/Istanbul') AS column, toTypeName(column) AS x; -``` - -```text -┌──────────────────column─┬─x──────────────────────────────┐ -│ 2023-06-05 00:09:52.000 │ DateTime64(3, 'Asia/Istanbul') │ -└─────────────────────────┴────────────────────────────────┘ -``` - -4. タイムゾーンの変換 - -```sql -SELECT -toDateTime64(timestamp, 3, 'Europe/London') AS lon_time, -toDateTime64(timestamp, 3, 'Asia/Istanbul') AS istanbul_time -FROM dt64; -``` - -```text -┌────────────────ロンドン_time─┬───────────イスタンブール_time─┐ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2018-12-31 21:00:00.000 │ 2019-01-01 00:00:00.000 │ -└─────────────────────────┴─────────────────────────┘ -``` - -**関連項目** - -* [型変換関数](../../sql-reference/functions/type-conversion-functions.md) -* [日付と時刻を操作する関数](../../sql-reference/functions/date-time-functions.md) -* [`date_time_input_format` 設定](../../operations/settings/settings-formats.md#date_time_input_format) -* [`date_time_output_format` 設定](../../operations/settings/settings-formats.md#date_time_output_format) -* [`timezone` サーバー構成パラメータ](../../operations/server-configuration-parameters/settings.md#timezone) -* [`session_timezone` 設定](../../operations/settings/settings.md#session_timezone) -* [日付と時刻を操作する演算子](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [`Date` データ型](../../sql-reference/data-types/date.md) -* [`DateTime` データ型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md deleted file mode 100644 index 5a387ebddec..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -description: 'ClickHouse の Decimal データ型に関するドキュメント。Decimal は精度を構成可能な固定小数点演算を提供します' -sidebar_label: 'Decimal' -sidebar_position: 6 -slug: /sql-reference/data-types/decimal -title: 'Decimal, Decimal(P), Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), - Decimal256(S)' -doc_type: 'reference' ---- - - - -# Decimal, Decimal(P), Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), Decimal256(S) {#decimal-decimalp-decimalp-s-decimal32s-decimal64s-decimal128s-decimal256s} - -加算、減算、乗算の演算で精度を維持する符号付き固定小数点数です。除算では、小数点以下の末尾の桁は丸めずに切り捨てられます。 - - - -## パラメータ {#parameters} - -- P - 精度。指定可能な範囲: \[ 1 : 76 \]。数値が持つことのできる 10 進数の桁数(小数部分を含む)を決定します。デフォルトの精度は 10 です。 -- S - スケール。指定可能な範囲: \[ 0 : P \]。小数部分が持つことのできる 10 進数の桁数を決定します。 - -Decimal(P) は Decimal(P, 0) と同等です。同様に、構文 Decimal は Decimal(10, 0) と同等です。 - -P パラメータの値に応じて、Decimal(P, S) は次の型の別名になります: -- P が \[ 1 : 9 \] の場合 - Decimal32(S) -- P が \[ 10 : 18 \] の場合 - Decimal64(S) -- P が \[ 19 : 38 \] の場合 - Decimal128(S) -- P が \[ 39 : 76 \] の場合 - Decimal256(S) - - - -## 10 進数値の範囲 {#decimal-value-ranges} - -- Decimal(P, S) - ( -1 \* 10^(P - S), 1 \* 10^(P - S) ) -- Decimal32(S) - ( -1 \* 10^(9 - S), 1 \* 10^(9 - S) ) -- Decimal64(S) - ( -1 \* 10^(18 - S), 1 \* 10^(18 - S) ) -- Decimal128(S) - ( -1 \* 10^(38 - S), 1 \* 10^(38 - S) ) -- Decimal256(S) - ( -1 \* 10^(76 - S), 1 \* 10^(76 - S) ) - -たとえば、Decimal32(4) では、-99999.9999 から 99999.9999 までの数値を 0.0001 刻みで表現できます。 - - - -## 内部表現 {#internal-representation} - -内部的には、データは対応するビット幅を持つ通常の符号付き整数として表現されます。メモリに格納可能な実際の値の範囲は、上記で指定したものよりわずかに広くなっていますが、この範囲は文字列からの変換時にのみ検査されます。 - -現代の CPU は 128 ビットおよび 256 ビット整数をネイティブにはサポートしていないため、Decimal128 と Decimal256 に対する演算はエミュレートされます。その結果、Decimal128 および Decimal256 は Decimal32/Decimal64 と比べて大幅に低速に動作します。 - - - -## 演算と結果の型 {#operations-and-result-type} - -Decimal 同士の二項演算の結果は(引数の順序に関係なく)より大きな桁幅の結果型になります。 - -- `Decimal64(S1) Decimal32(S2) -> Decimal64(S)` -- `Decimal128(S1) Decimal32(S2) -> Decimal128(S)` -- `Decimal128(S1) Decimal64(S2) -> Decimal128(S)` -- `Decimal256(S1) Decimal<32|64|128>(S2) -> Decimal256(S)` - -スケールに関する規則: - -- 加算、減算: S = max(S1, S2)。 -- 乗算: S = S1 + S2。 -- 除算: S = S1。 - -Decimal と整数との同様の演算では、結果は引数と同じサイズの Decimal になります。 - -Decimal と Float32/Float64 の間の演算は定義されていません。これらが必要な場合は、一方の引数を `toDecimal32`、`toDecimal64`、`toDecimal128` もしくは `toFloat32`、`toFloat64` のビルトインを使って明示的にキャストしてください。結果の精度が失われること、および型変換は計算コストの高い操作であることに注意してください。 - -Decimal に対する一部の関数は、結果を Float64 として返します(例えば `var` や `stddev`)。中間計算は依然として Decimal で実行される場合があり、そのため同じ値を持つ Float64 入力と Decimal 入力の間で結果が異なる可能性があります。 - - - -## オーバーフローのチェック {#overflow-checks} - -Decimal で計算を行う際には、整数オーバーフローが発生する可能性があります。小数部の桁数が多すぎる場合は、余分な桁は切り捨てられます(丸めは行われません)。整数部の桁数が多すぎる場合は、例外がスローされます。 - -:::warning -Decimal128 および Decimal256 ではオーバーフローのチェックは実装されていません。オーバーフローが発生した場合は、例外はスローされず、不正な結果が返されます。 -::: - -```sql -SELECT toDecimal32(2, 4) AS x, x / 3 -``` - -```text -┌──────x─┬─divide(toDecimal32(2, 4), 3)─┐ -│ 2.0000 │ 0.6666 │ -└────────┴──────────────────────────────┘ -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, x * x -``` - -```text -DB::Exception: スケールが範囲外です。 -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -DB::Exception: Decimal演算オーバーフロー。 -``` - -オーバーフローのチェックは演算を遅くします。オーバーフローが発生しないことが分かっている場合は、`decimal_check_overflow` 設定を使用してチェックを無効化するのが有効です。チェックを無効化した状態でオーバーフローが発生すると、結果は正しくなくなります。 - -```sql -SET decimal_check_overflow = 0; -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -┌──────────x─┬─multiply(6, toDecimal32(4.2, 8))─┐ -│ 4.20000000 │ -17.74967296 │ -└────────────┴──────────────────────────────────┘ -``` - -オーバーフロー検査は、算術演算だけでなく値の比較時にも行われます。 - -```sql -SELECT toDecimal32(1, 8) < 100 -``` - -```text -DB::Exception: 比較できません。 -``` - -**関連項目** - -* [isDecimalOverflow](/sql-reference/functions/other-functions#isDecimalOverflow) -* [countDigits](/sql-reference/functions/other-functions#countDigits) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md deleted file mode 100644 index 7fa1ec98e21..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'ClickHouse におけるドメイン型の概要。ドメイン型は基本型を拡張し、追加機能を提供します。' -sidebar_label: 'ドメイン' -sidebar_position: 56 -slug: /sql-reference/data-types/domains/ -title: 'ドメイン' -doc_type: 'reference' ---- - - - -# ドメイン {#domains} - -ドメインは、既存の基本型の上に追加機能を付与するための特別な型であり、基になるデータ型のオンワイヤ形式およびオンディスク形式はそのまま保持されます。現在、ClickHouse はユーザー定義ドメインをサポートしていません。 - -ドメインは、対応する基本型が使用できるあらゆる場所で使用できます。例えば次のような用途があります。 - -- ドメイン型のカラムを作成する -- ドメインカラムから/へ値を読み書きする -- 基本型をインデックスとして使用できる場合、インデックスとして使用する -- ドメインカラムの値を引数として関数を呼び出す - -### ドメインの追加機能 {#extra-features-of-domains} - -- `SHOW CREATE TABLE` または `DESCRIBE TABLE` における明示的なカラム型名 -- `INSERT INTO domain_table(domain_column) VALUES(...)` による、人間が読みやすい形式からの入力 -- `SELECT domain_column FROM domain_table` に対する、人間が読みやすい形式での出力 -- 外部ソースから人間が読みやすい形式のデータをロードする: `INSERT INTO domain_table FORMAT CSV ...` - -### 制限事項 {#limitations} - -- `ALTER TABLE` によって、基本型のインデックスカラムをドメイン型に変換することはできません。 -- 他のカラムやテーブルからデータを挿入する際に、文字列値をドメイン値へ暗黙的に変換することはできません。 -- ドメインは、保存される値に対して追加の制約を課しません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md deleted file mode 100644 index 39766c811ec..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md +++ /dev/null @@ -1,717 +0,0 @@ ---- -description: '1 つの列に異なる型の値を格納できる ClickHouse の Dynamic データ型のドキュメント' -sidebar_label: 'Dynamic' -sidebar_position: 62 -slug: /sql-reference/data-types/dynamic -title: 'Dynamic' -doc_type: 'guide' ---- - -# Dynamic {#dynamic} - -この型は、あらかじめすべての型を把握していなくても、任意の型の値を格納できます。 - -`Dynamic` 型のカラムを宣言するには、次の構文を使用します。 - -```sql -<カラム名> Dynamic(max_types=N) -``` - -ここで `N` はオプションのパラメータで、`0` から `254` の範囲の値を取り、個別に保存される 1 つのデータブロック(たとえば MergeTree テーブルの 1 つのデータパート)内で、型が `Dynamic` のカラムに個別のサブカラムとして保存できる異なるデータ型の数を指定します。この上限を超えると、新しい型を持つすべての値は、バイナリ形式で特別な共有データ構造の中にまとめて保存されます。`max_types` のデフォルト値は `32` です。 - -## Dynamic の作成 {#creating-dynamic} - -テーブルの列定義で `Dynamic` 型を使用します: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ Hello, World! │ String │ -│ [1,2,3] │ Array(Int64) │ -└───────────────┴────────────────┘ -``` - -通常の列に対する CAST の使用: - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -`Variant` 列の値に対して CAST を使用する: - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 3) = 0, number, (number % 3) = 1, range(number + 1), NULL)::Dynamic AS d, dynamicType(d) FROM numbers(3) -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 0 │ UInt64 │ -│ [0,1] │ Array(UInt64) │ -│ ᴺᵁᴸᴸ │ None │ -└───────┴────────────────┘ -``` - -## Dynamic のネストされた型をサブカラムとして読み取る {#reading-dynamic-nested-types-as-subcolumns} - -`Dynamic` 型では、型名をサブカラム名として指定することで、`Dynamic` カラムから単一のネストされた型を読み取ることができます。 -そのため、`d Dynamic` というカラムがある場合、有効な任意の型 `T` のサブカラムを構文 `d.T` を使って読み取ることができます。 -このサブカラムは、`T` が `Nullable` の中に入ることができる場合は型 `Nullable(T)` になり、それ以外の場合は `T` になります。 -このサブカラムの行数は元の `Dynamic` カラムと同じであり、元の `Dynamic` カラムに型 `T` が存在しないすべての行では、 -`NULL` 値(あるいは、`T` が `Nullable` の中に入ることができない場合は空の値)を含みます。 - -`Dynamic` のサブカラムは、関数 `dynamicElement(dynamic_column, type_name)` を使って読み取ることもできます。 - -例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d), d.String, d.Int64, d.`Array(Int64)`, d.Date, d.`Array(String)` FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─d.String──────┬─d.Int64─┬─d.Array(Int64)─┬─d.Date─┬─d.Array(String)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴───────────────┴─────────┴────────────────┴────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(d.String), toTypeName(d.Int64), toTypeName(d.`Array(Int64)`), toTypeName(d.Date), toTypeName(d.`Array(String)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(d.String)─┬─toTypeName(d.Int64)─┬─toTypeName(d.Array(Int64))─┬─toTypeName(d.Date)─┬─toTypeName(d.Array(String))─┐ -│ Nullable(String) │ Nullable(Int64) │ Array(Int64) │ Nullable(Date) │ Array(String) │ -└──────────────────────┴─────────────────────┴────────────────────────────┴────────────────────┴─────────────────────────────┘ -``` - -````sql -SELECT d, dynamicType(d), dynamicElement(d, 'String'), dynamicElement(d, 'Int64'), dynamicElement(d, 'Array(Int64)'), dynamicElement(d, 'Date'), dynamicElement(d, 'Array(String)') FROM test;``` -```` - -```text -┌─d─────────────┬─dynamicType(d)─┬─dynamicElement(d, 'String')─┬─dynamicElement(d, 'Int64')─┬─dynamicElement(d, 'Array(Int64)')─┬─dynamicElement(d, 'Date')─┬─dynamicElement(d, 'Array(String)')─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴─────────────────────────────┴────────────────────────────┴───────────────────────────────────┴───────────────────────────┴────────────────────────────────────┘ -``` - -各行に格納されているバリアントを確認するには、関数 `dynamicType(dynamic_column)` を使用できます。各行ごとに値の型名を表す `String` を返します(行が `NULL` の場合は `'None'` を返します)。 - -例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT dynamicType(d) FROM test; -``` - -```text -┌─dynamicType(d)─┐ -│ None │ -│ Int64 │ -│ String │ -│ Array(Int64) │ -└────────────────┘ -``` - -## Dynamic 列と他の列との変換 {#conversion-between-dynamic-column-and-other-columns} - -`Dynamic` 列に対して行える変換は 4 種類あります。 - -### 通常の列を `Dynamic` 列に変換する {#converting-an-ordinary-column-to-a-dynamic-column} - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -### パースによる String 列から Dynamic 列への変換 {#converting-a-string-column-to-a-dynamic-column-through-parsing} - -`String` 列に含まれる値を `Dynamic` 型としてパースするには、設定 `cast_string_to_dynamic_use_inference` を有効にします。 - -```sql -SET cast_string_to_dynamic_use_inference = 1; -SELECT CAST(materialize(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01')), 'Map(String, Dynamic)') as map_of_dynamic, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamic) as map_of_dynamic_types; -``` - -```text -┌─map_of_dynamic──────────────────────────────┬─map_of_dynamic_types─────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'Int64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴──────────────────────────────────────────────┘ -``` - -### Dynamicカラムを通常カラムに変換する {#converting-a-dynamic-column-to-an-ordinary-column} - -`Dynamic` カラムを通常カラムに変換できます。この場合、すべてのネストされた型は宛先の型に変換されます。 - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'), (true), ('e10'); -SELECT d::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(d, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -│ 1 │ -│ 0 │ -└──────────────────────────────┘ -``` - -### Variant 型の列を Dynamic 型の列に変換する {#converting-a-variant-column-to-dynamic-column} - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'), ([1, 2, 3]); -SELECT v::Dynamic AS d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ UInt64 │ -│ String │ String │ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴────────────────┘ -``` - -### Dynamic(max_types=N) 列を別の Dynamic(max_types=K) に変換する {#converting-a-dynamicmax_typesn-column-to-another-dynamicmax_typesk} - -`K >= N` の場合、変換してもデータは変わりません: - -```sql -CREATE TABLE test (d Dynamic(max_types=3)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true); -SELECT d::Dynamic(max_types=5) as d2, dynamicType(d2) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ 42.42 │ String │ -│ true │ Bool │ -└───────┴────────────────┘ -``` - -`K < N` の場合、最も出現頻度の低い型を持つ値は 1 つの特別なサブカラムに挿入されますが、引き続きアクセス可能です。 - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=2) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ false │ -│ 43 │ Int64 │ 43 │ Int64 │ false │ -│ 42.42 │ String │ 42.42 │ String │ false │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -関数 `isDynamicElementInSharedData` は、`Dynamic` 内の特別な共有データ構造に保存されている行に対して `true` を返します。ここからわかるように、結果のカラムには共有データ構造に保存されていない 2 種類の型のみが含まれます。 - -`K=0` の場合、すべての型は 1 つの特別なサブカラムに挿入されます。 - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=0) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ true │ -│ 43 │ Int64 │ 43 │ Int64 │ true │ -│ 42.42 │ String │ 42.42 │ String │ true │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -## データからの Dynamic 型の読み取り {#reading-dynamic-type-from-the-data} - -すべてのテキスト形式(TSV、CSV、CustomSeparated、Values、JSONEachRow など)は `Dynamic` 型の読み取りをサポートします。データの解析中に、ClickHouse は各値の型を推論し、その型情報を `Dynamic` 列への挿入時に使用します。 - -例: - -```sql -SELECT - d, - dynamicType(d), - dynamicElement(d, 'String') AS str, - dynamicElement(d, 'Int64') AS num, - dynamicElement(d, 'Float64') AS float, - dynamicElement(d, 'Date') AS date, - dynamicElement(d, 'Array(Int64)') AS arr -FROM format(JSONEachRow, 'd Dynamic', $$ -{"d" : "Hello, World!"}, -{"d" : 42}, -{"d" : 42.42}, -{"d" : "2020-01-01"}, -{"d" : [1, 2, 3]} -$$) -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─str───────────┬──num─┬─float─┬───────date─┬─arr─────┐ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ Float64 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 │ Date │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴────────────────┴───────────────┴──────┴───────┴────────────┴─────────┘ -``` - -## 関数での Dynamic 型の使用 {#using-dynamic-type-in-functions} - -ほとんどの関数は、`Dynamic` 型の引数を受け付けます。この場合、関数は `Dynamic` 列の内部に格納されている各データ型ごとに個別に実行されます。 -関数の結果型が引数の型に依存する場合、`Dynamic` 引数で実行されたその関数の結果型は `Dynamic` になります。関数の結果型が引数の型に依存しない場合は、その結果型は `Nullable(T)` となり、ここで `T` はその関数の通常の結果型です。 - -例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (NULL), (1::Int8), (2::Int16), (3::Int32), (4::Int64); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 1 │ Int8 │ -│ 2 │ Int16 │ -│ 3 │ Int32 │ -│ 4 │ Int64 │ -└──────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 3 │ Dynamic │ Int32 │ -│ 3 │ 4 │ Dynamic │ Int64 │ -│ 4 │ 5 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d + d AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 4 │ Dynamic │ Int32 │ -│ 3 │ 6 │ Dynamic │ Int64 │ -│ 4 │ 8 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d < 3 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(UInt8) │ -│ 1 │ 1 │ Nullable(UInt8) │ -│ 2 │ 1 │ Nullable(UInt8) │ -│ 3 │ 0 │ Nullable(UInt8) │ -│ 4 │ 0 │ Nullable(UInt8) │ -└──────┴──────┴─────────────────┘ -``` - -```sql -SELECT d, exp2(d) AS res, toTypeName(res) FROM test; -``` - -```sql -┌─d────┬──res─┬─toTypeName(res)───┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Float64) │ -│ 1 │ 2 │ Nullable(Float64) │ -│ 2 │ 4 │ Nullable(Float64) │ -│ 3 │ 8 │ Nullable(Float64) │ -│ 4 │ 16 │ Nullable(Float64) │ -└──────┴──────┴───────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ('str_1'), ('str_2'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ str_1 │ String │ -│ str_2 │ String │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, upper(d) AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res───┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ STR_1 │ Nullable(String) │ -│ str_2 │ STR_2 │ Nullable(String) │ -└───────┴───────┴──────────────────┘ -``` - -```sql -SELECT d, extract(d, '([0-3])') AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ 1 │ Nullable(String) │ -│ str_2 │ 2 │ Nullable(String) │ -└───────┴──────┴──────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ([1, 2]), ([3, 4]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d[1] AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ [1,2] │ 1 │ Dynamic │ Int64 │ -│ [3,4] │ 3 │ Dynamic │ Int64 │ -└───────┴──────┴─────────────────┴──────────────────┘ -``` - -`Dynamic` 列内の一部の型に対して関数を実行できない場合は、例外がスローされます。 - -```sql -INSERT INTO test VALUES (42), (43), ('str_1'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ str_1 │ String │ -└───────┴────────────────┘ -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(d) FROM test; -``` - -```text -例外が発生しました: -Code: 43. DB::Exception: 関数plusの引数の型が不正です Array(Int64) と UInt8: 'FUNCTION plus(__table1.d : 3, 1_UInt8 :: 1) -> plus(__table1.d, 1_UInt8) Dynamic : 0' の実行中。(ILLEGAL_TYPE_OF_ARGUMENT) -``` - -不要な型を除外できます: - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test WHERE dynamicType(d) NOT IN ('String', 'Array(Int64)', 'None') -``` - -```text -┌─d──┬─res─┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ 42 │ 43 │ Dynamic │ Int64 │ -│ 43 │ 44 │ Dynamic │ Int64 │ -└────┴─────┴─────────────────┴──────────────────┘ -``` - -または、必要な型のみをサブカラムとして抽出します: - -```sql -SELECT d, d.Int64 + 1 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ 42 │ 43 │ Nullable(Int64) │ -│ 43 │ 44 │ Nullable(Int64) │ -│ str_1 │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [1,2] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [3,4] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -``` - -## ORDER BY と GROUP BY における Dynamic 型の使用 {#using-dynamic-type-in-order-by-and-group-by} - -`ORDER BY` および `GROUP BY` において `Dynamic` 型の値は、`Variant` 型の値と同様に比較されます。 -型 `Dynamic` の値 `d1`(基底型 `T1`)と `d2`(基底型 `T2`)に対する演算子 `<` の結果は、次のように定義されます。 - -* `T1 = T2 = T` の場合、結果は `d1.T < d2.T` になります(基底となる値同士が比較されます)。 -* `T1 != T2` の場合、結果は `T1 < T2` になります(型名同士が比較されます)。 - -デフォルトでは、`Dynamic` 型は `GROUP BY`/`ORDER BY` のキーとしては許可されていません。使用したい場合は、この特別な比較ルールを考慮しつつ、`allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by` 設定を有効にしてください。 - -例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (42), (43), ('abc'), ('abd'), ([1, 2, 3]), ([]), (NULL); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ [1,2,3] │ Array(Int64) │ -│ [] │ Array(Int64) │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```sql -┌─d───────┬─dynamicType(d)─┐ -│ [] │ Array(Int64) │ -│ [1,2,3] │ Array(Int64) │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -**注記:** 異なる数値型を持つ Dynamic 型の値は、別個の値として扱われ、相互に比較されません。その代わりに、型名が比較されます。 - -例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```text -┌─v───┬─dynamicType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test GROUP BY d SETTINGS allow_suspicious_types_in_group_by=1; -``` - -```text -┌─d───┬─dynamicType(d)─┐ -│ 1 │ Int64 │ -│ 100 │ UInt32 │ -│ 1 │ UInt32 │ -│ 100 │ Int64 │ -└─────┴────────────────┘ -``` - -**注:** ここで説明した比較ルールは、`<` / `>` / `=` などの比較演算の実行時には適用されません。これは、`Dynamic` 型に対する関数の[特殊な動作](#using-dynamic-type-in-functions)によるものです。 - -## Dynamic 内部に格納される異なるデータ型数の上限に達した場合 {#reaching-the-limit-in-number-of-different-data-types-stored-inside-dynamic} - -`Dynamic` データ型は、異なるデータ型を個別のサブカラムとして格納できる数に上限があります。デフォルトではこの上限は 32 ですが、型宣言で `Dynamic(max_types=N)` という構文を使うことで変更できます。このときの N は 0 から 254 の間の値です(実装上の制約により、Dynamic 内に個別のサブカラムとして格納可能な異なるデータ型の数は 254 を超えることはできません)。 -上限に達すると、`Dynamic` カラムに挿入される新しいデータ型はすべて、異なるデータ型の値をバイナリ形式で格納する単一の共有データ構造に挿入されます。 - -さまざまなケースでこの上限に達した場合に何が起こるかを見ていきます。 - -### データのパース中に上限に達した場合 {#reaching-the-limit-during-data-parsing} - -データから `Dynamic` の値をパースしている際に、現在のデータブロックで上限に達した場合は、新しい値はすべて共有データ構造に挿入されます。 - -```sql -SELECT d, dynamicType(d), isDynamicElementInSharedData(d) FROM format(JSONEachRow, 'd Dynamic(max_types=3)', ' -{"d" : 42} -{"d" : [1, 2, 3]} -{"d" : "Hello, World!"} -{"d" : "2020-01-01"} -{"d" : ["str1", "str2", "str3"]} -{"d" : {"a" : 1, "b" : [1, 2, 3]}} -') -``` - -```text -┌─d──────────────────────┬─dynamicType(d)─────────────────┬─isDynamicElementInSharedData(d)─┐ -│ 42 │ Int64 │ false │ -│ [1,2,3] │ Array(Int64) │ false │ -│ Hello, World! │ String │ false │ -│ 2020-01-01 │ Date │ true │ -│ ['str1','str2','str3'] │ Array(String) │ true │ -│ (1,[1,2,3]) │ Tuple(a Int64, b Array(Int64)) │ true │ -└────────────────────────┴────────────────────────────────┴─────────────────────────────────┘ -``` - -ご覧のとおり、3 つの異なるデータ型 `Int64`、`Array(Int64)`、`String` を挿入すると、すべての新しい型が特殊な共有データ構造に格納されました。 - -### MergeTree テーブルエンジンでのデータパーツのマージ中 {#during-merges-of-data-parts-in-mergetree-table-engines} - -MergeTree テーブル内で複数のデータパーツをマージする際、結果のデータパーツにおける `Dynamic` 列は、内部で別個のサブカラムとして保持できる異なるデータ型の上限に達してしまい、元のパーツに含まれるすべての型をサブカラムとして保持できなくなる可能性があります。 -この場合、ClickHouse は、マージ後も別個のサブカラムとして残す型と、共有データ構造に格納する型を選択します。多くの場合、ClickHouse は最も出現頻度の高い型をサブカラムとして保持し、出現頻度の低い型を共有データ構造に保存しようとしますが、これは実装によって異なります。 - -このようなマージの例を見てみましょう。まず、`Dynamic` 列を持つテーブルを作成し、異なるデータ型数の上限を `3` に設定してから、`5` 種類の異なる型の値を挿入します。 - -```sql -CREATE TABLE test (id UInt64, d Dynamic(max_types=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, number FROM numbers(5); -INSERT INTO test SELECT number, range(number) FROM numbers(4); -INSERT INTO test SELECT number, toDate(number) FROM numbers(3); -INSERT INTO test SELECT number, map(number, number) FROM numbers(2); -INSERT INTO test SELECT number, 'str_' || toString(number) FROM numbers(1); -``` - -各 INSERT によって、単一の型のみを含む `Dynamic` 列を持つ個別のデータパートが作成されます。 - -```sql -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count(); -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_1_0 │ -│ 4 │ Array(UInt64) │ false │ all_2_2_0 │ -│ 3 │ Date │ false │ all_3_3_0 │ -│ 2 │ Map(UInt64, UInt64) │ false │ all_4_4_0 │ -│ 1 │ String │ false │ all_5_5_0 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -では、これらをすべてひとつにまとめて、どうなるか確認してみましょう。 - -```sql -SYSTEM START MERGES test; -OPTIMIZE TABLE test FINAL; -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count() desc; -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_5_2 │ -│ 4 │ Array(UInt64) │ false │ all_1_5_2 │ -│ 3 │ Date │ false │ all_1_5_2 │ -│ 2 │ Map(UInt64, UInt64) │ true │ all_1_5_2 │ -│ 1 │ String │ true │ all_1_5_2 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -ここからわかるように、ClickHouse は最も頻繁に現れる型である `UInt64` と `Array(UInt64)` をサブカラムとして保持し、それ以外のすべての型は共有データに格納しました。 - -## Dynamic 型での JSONExtract 関数 {#jsonextract-functions-with-dynamic} - -すべての `JSONExtract*` 関数は `Dynamic` 型をサポートします。 - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Dynamic') AS dynamic, dynamicType(dynamic) AS dynamic_type; -``` - -```text -┌─dynamic─┬─dynamic_type───────────┐ -│ [1,2,3] │ Array(Nullable(Int64)) │ -└─────────┴────────────────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Dynamic)') AS map_of_dynamics, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamics) AS map_of_dynamic_types -``` - -```text -┌─map_of_dynamics──────────────────┬─map_of_dynamic_types────────────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'Int64','b':'String','c':'Array(Nullable(Int64))'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────────────┘ -``` - -````sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Dynamic') AS dynamics, arrayMap(x -> (x.1, dynamicType(x.2)), dynamics) AS dynamic_types``` -```` - -```text -┌─dynamics───────────────────────────────┬─dynamic_types─────────────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','Int64'),('b','String'),('c','Array(Nullable(Int64))')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘ -``` - -### バイナリ出力形式 {#binary-output-format} - -RowBinary 形式では、`Dynamic` 型の値は次のフォーマットでシリアライズされます。 - -```text - -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md deleted file mode 100644 index 24e3de0b6cb..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -description: 'ClickHouse における Enum データ型に関するドキュメント。名前付き定数値の集合を表します' -sidebar_label: 'Enum' -sidebar_position: 20 -slug: /sql-reference/data-types/enum -title: 'Enum' -doc_type: 'reference' ---- - -# Enum {#enum} - -名前付きの値から構成される列挙型です。 - -名前付きの値は、`'string' = integer` のペア、または `'string'` の名前として宣言できます。ClickHouse では数値のみを保存しますが、名前を使って値に対する操作を行えます。 - -ClickHouse は次の Enum 型をサポートします。 - -* 8 ビットの `Enum`。`[-128, 127]` の範囲で列挙される最大 256 個の値を含めることができます。 -* 16 ビットの `Enum`。`[-32768, 32767]` の範囲で列挙される最大 65536 個の値を含めることができます。 - -ClickHouse は、データ挿入時に自動的に `Enum` の型を選択します。格納に必要なサイズを明確にするために、`Enum8` や `Enum16` 型を使用することもできます。 - -## 使用例 {#usage-examples} - -ここでは、`Enum8('hello' = 1, 'world' = 2)` 型の列を持つテーブルを作成します。 - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world' = 2) -) -ENGINE = TinyLog -``` - -同様に、番号は省略できます。ClickHouse が自動的に連番を割り当てます。デフォルトでは 1 から割り当てられます。 - -```sql -CREATE TABLE t_enum -( - x Enum('hello', 'world') -) -ENGINE = TinyLog -``` - -ファーストネームに対して有効な開始番号を指定することもできます。 - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world') -) -ENGINE = TinyLog -``` - -```sql -CREATE TABLE t_enum -( - x Enum8('hello' = -129, 'world') -) -ENGINE = TinyLog -``` - -```text -サーバーでの例外: -コード: 69. DB::Exception: 要素 'hello' の値 -129 が Enum8 の範囲を超えています。 -``` - -カラム `x` には、型定義で列挙された値である `'hello'` または `'world'` しか保存できません。これ以外の値を保存しようとすると、ClickHouse は例外を発生させます。この `Enum` のサイズとして 8 ビットが自動的に選択されます。 - -```sql -INSERT INTO t_enum VALUES ('hello'), ('world'), ('hello') -``` - -```text -Ok. -``` - -```sql -INSERT INTO t_enum VALUES('a') -``` - -```text -クライアントでの例外: -Code: 49. DB::Exception: 型 Enum('hello' = 1, 'world' = 2) に対して不明な要素 'a' です -``` - -テーブルをクエリすると、ClickHouse は `Enum` の文字列値を返します。 - -```sql -SELECT * FROM t_enum -``` - -```text -┌─x─────┐ -│ hello │ -│ world │ -│ hello │ -└───────┘ -``` - -行に対応する数値を確認する必要がある場合は、`Enum` 型の値を整数型にキャストする必要があります。 - -```sql -SELECT CAST(x, 'Int8') FROM t_enum -``` - -```text -┌─CAST(x, 'Int8')─┐ -│ 1 │ -│ 2 │ -│ 1 │ -└─────────────────┘ -``` - -クエリ内で Enum 型の値を作成するには、`CAST` も使用する必要があります。 - -```sql -SELECT toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)')) -``` - -```text -┌─toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)'))─┐ -│ Enum8('a' = 1, 'b' = 2) │ -└─────────────────────────────────────────────────────┘ -``` - -## 一般的な規則と使用方法 {#general-rules-and-usage} - -各値には、`Enum8` の場合は `-128 ... 127` の範囲、`Enum16` の場合は `-32768 ... 32767` の範囲の数値が割り当てられます。すべての文字列および数値は相互に異なっていなければなりません。空文字列も許可されます。この型が(テーブル定義で)指定されている場合、数値は任意の順序でかまいません。ただし、その順序に意味はありません。 - -`Enum` 内の文字列値および数値値はどちらも [NULL](../../sql-reference/syntax.md) にすることはできません。 - -`Enum` は [Nullable](../../sql-reference/data-types/nullable.md) 型に含めることができます。そのため、次のクエリを使用してテーブルを作成する場合 - -```sql -CREATE TABLE t_enum_nullable -( - x Nullable( Enum8('hello' = 1, 'world' = 2) ) -) -ENGINE = TinyLog -``` - -`'hello'` と `'world'` だけでなく、`NULL` も格納できます。 - -```sql -INSERT INTO t_enum_nullable VALUES('hello'),('world'),(NULL) -``` - -RAM 上では、`Enum` カラムは対応する数値表現の `Int8` または `Int16` と同じ方法で格納されます。 - -テキスト形式で読み込むとき、ClickHouse は値を文字列としてパースし、Enum の値集合から対応する文字列を検索します。見つからない場合は例外がスローされます。テキスト形式で出力された文字列を読み込む際には、その文字列を読み取り、対応する数値表現を検索します。見つからなければ例外がスローされます。 -テキスト形式で書き込むときは、対応する文字列として値を書き込みます。カラムデータに不正な値(有効な集合に含まれない数値)が含まれている場合は、例外がスローされます。バイナリ形式での読み書きでは、`Int8` および `Int16` データ型の場合と同様に動作します。 -暗黙のデフォルト値は、最も小さい数値を持つ値です。 - -`ORDER BY`、`GROUP BY`、`IN`、`DISTINCT` などの処理時には、Enum は対応する数値と同じように振る舞います。たとえば、`ORDER BY` はそれらを数値としてソートします。等価演算子および比較演算子は、Enum に対しても基になる数値に対するときと同じように動作します。 - -Enum 値は数値と比較することはできません。Enum は定数文字列と比較することができます。比較対象の文字列が Enum にとって有効な値でない場合は、例外がスローされます。`IN` 演算子は、左辺が Enum、右辺が文字列の集合である場合にサポートされます。これらの文字列は対応する Enum の値です。 - -ほとんどの数値演算および文字列演算は Enum 値に対しては定義されていません。たとえば Enum に数値を加算する、あるいは Enum に文字列を連結するといった操作です。 -しかし、Enum には自然な形で `toString` 関数が定義されており、その文字列値を返します。 - -Enum 値は、`toT` 関数を使って数値型に変換することもできます。ここで T は数値型です。T が Enum の基になる数値型に対応する場合、この変換はゼロコストです。 -Enum 型は、値の集合だけが変更される場合には、ALTER を使ってコストなしに変更できます。ALTER を用いて Enum のメンバーを追加および削除することが可能です(削除は、その値がテーブル内で一度も使用されていない場合にのみ安全です)。安全策として、すでに定義済みの Enum メンバーの数値を変更しようとすると例外がスローされます。 - -ALTER を使用することで、`Enum8` を `Enum16` に、あるいはその逆に変更することができます。これは `Int8` を `Int16` に変更する場合と同様です。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md deleted file mode 100644 index 9080e320514..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'ClickHouse の FixedString データ型に関するドキュメント' -sidebar_label: 'FixedString(N)' -sidebar_position: 10 -slug: /sql-reference/data-types/fixedstring -title: 'FixedString(N)' -doc_type: 'reference' ---- - -# FixedString(N) {#fixedstringn} - -`N` バイトの固定長文字列(文字数でもコードポイント数でもありません)。 - -`FixedString` 型の列を宣言するには、次の構文を使用します。 - -```sql - FixedString(N) -``` - -ここで、`N` は自然数です。 - -`FixedString` 型は、データの長さがちょうど `N` バイトである場合に効率的です。それ以外の場合は、かえって非効率になる可能性があります。 - -`FixedString` 型のカラムに効率的に保存できる値の例は次のとおりです。 - -* IP アドレスのバイナリ表現(IPv6 では `FixedString(16)`)。 -* 言語コード(ru_RU, en_US など)。 -* 通貨コード(USD, RUB など)。 -* ハッシュのバイナリ表現(MD5 では `FixedString(16)`、SHA256 では `FixedString(32)`)。 - -UUID の値を保存するには、[UUID](../../sql-reference/data-types/uuid.md) データ型を使用します。 - -データを挿入する際、ClickHouse は次のように動作します。 - -* 文字列が `N` バイト未満の場合、ヌルバイトで文字列を埋めます。 -* 文字列が `N` バイトを超える場合、`Too large value for FixedString(N)` 例外をスローします。 - -次のような、単一の `FixedString(2)` カラムをもつテーブルを考えます。 - -```sql - - -INSERT INTO FixedStringTable VALUES ('a'), ('ab'), (''); -``` - -```sql -SELECT - name, - toTypeName(name), - length(name), - empty(name) -FROM FixedStringTable; -``` - -```text -┌─name─┬─toTypeName(name)─┬─length(name)─┬─empty(name)─┐ -│ a │ FixedString(2) │ 2 │ 0 │ -│ ab │ FixedString(2) │ 2 │ 0 │ -│ │ FixedString(2) │ 2 │ 1 │ -└──────┴──────────────────┴──────────────┴─────────────┘ -``` - -`FixedString(N)` の値の長さは一定であることに注意してください。[length](/sql-reference/functions/array-functions#length) 関数は、`FixedString(N)` の値がヌルバイトのみで埋められている場合でも `N` を返しますが、この場合 [empty](/sql-reference/functions/array-functions#empty) 関数は `1` を返します。 - -`WHERE` 句でデータを選択する場合、条件の指定方法によって返される結果が変わります。 - -* 等価演算子 `=` または `==`、あるいは `equals` 関数が使用される場合、ClickHouse は `\0` 文字を考慮 *しません*。つまり、`SELECT * FROM FixedStringTable WHERE name = 'a';` と `SELECT * FROM FixedStringTable WHERE name = 'a\0';` というクエリは同じ結果を返します。 -* `LIKE` 句が使用される場合、ClickHouse は `\0` 文字を考慮 *します*。このため、フィルタ条件で明示的に `\0` 文字を指定する必要が生じる場合があります。 - -```sql -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -Query id: c32cec28-bb9e-4650-86ce-d74a1694d79e - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a' -FORMAT JSONStringsEachRow - -0 rows in set. - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md deleted file mode 100644 index ab40b2373f5..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -description: 'ClickHouse における浮動小数点データ型: Float32、Float64、BFloat16 のドキュメント' -sidebar_label: 'Float32 | Float64 | BFloat16' -sidebar_position: 4 -slug: /sql-reference/data-types/float -title: 'Float32 | Float64 | BFloat16 型' -doc_type: 'reference' ---- - -:::note -特に高い精度が求められる金融データや業務データを扱うなど、正確な計算が必要な場合は、代わりに [Decimal](../data-types/decimal.md) の利用を検討してください。 - -[浮動小数点数](https://en.wikipedia.org/wiki/IEEE_754)は、以下の例で示すように不正確な結果になる可能性があります。 - -```sql -CREATE TABLE IF NOT EXISTS float_vs_decimal -( - my_float Float64, - my_decimal Decimal64(3) -) -ENGINE=MergeTree -ORDER BY tuple(); -``` - - -# 小数点以下2桁の乱数を 1 000 000 個生成し、float 型および decimal 型として保存する {#generate-1-000-000-random-numbers-with-2-decimal-places-and-store-them-as-a-float-and-as-a-decimal} - -INSERT INTO float_vs_decimal SELECT round(randCanonical(), 3) AS res, res FROM system.numbers LIMIT 1000000; - -```` -```sql -SELECT sum(my_float), sum(my_decimal) FROM float_vs_decimal; - -┌──────sum(my_float)─┬─sum(my_decimal)─┐ -│ 499693.60500000004 │ 499693.605 │ -└────────────────────┴─────────────────┘ - -SELECT sumKahan(my_float), sumKahan(my_decimal) FROM float_vs_decimal; - -┌─sumKahan(my_float)─┬─sumKahan(my_decimal)─┐ -│ 499693.605 │ 499693.605 │ -└────────────────────┴──────────────────────┘ -```` - -::: - -ClickHouse と C における対応する型は次のとおりです。 - -* `Float32` — `float` -* `Float64` — `double` - -ClickHouse における浮動小数点数型には次のエイリアスがあります。 - -* `Float32` — `FLOAT`、`REAL`、`SINGLE` -* `Float64` — `DOUBLE`、`DOUBLE PRECISION` - -テーブルを作成する際、浮動小数点数に対して数値パラメータを指定できます(例: `FLOAT(12)`、`FLOAT(15, 22)`、`DOUBLE(12)`、`DOUBLE(4, 18)`)が、ClickHouse はこれらを無視します。 - - -## 浮動小数点数を使用する {#using-floating-point-numbers} - -* 浮動小数点数での計算では、丸め誤差が発生することがあります。 - -{/* */ } - -```sql -SELECT 1 - 0.9 - -┌───────minus(1, 0.9)─┐ -│ 0.09999999999999998 │ -└─────────────────────┘ -``` - -* 計算結果は、計算方法(コンピュータシステムのプロセッサの種類およびアーキテクチャ)によって異なります。 -* 浮動小数点計算では、無限大(`Inf`)や "not-a-number"(`NaN`)といった値が結果として得られる場合があります。計算結果を処理する際には、この点を考慮する必要があります。 -* 文字列から浮動小数点数を解析する場合、結果が最も近い機械表現可能な数値にならないことがあります。 - - -## NaN と Inf {#nan-and-inf} - -標準的な SQL とは異なり、ClickHouse は次のカテゴリの浮動小数点数をサポートしています。 - -* `Inf` – 無限大。 - -{/* */ } - -```sql -SELECT 0.5 / 0 - -┌─divide(0.5, 0)─┐ -│ inf │ -└────────────────┘ -``` - -* `-Inf` — 負の無限大。 - -{/* */ } - -```sql -SELECT -0.5 / 0 - -┌─divide(-0.5, 0)─┐ -│ -inf │ -└─────────────────┘ -``` - -* `NaN` — 「Not a Number」(数値ではないことを示す値)。 - -{/* */ } - -```sql -SELECT 0 / 0 - -┌─divide(0, 0)─┐ -│ nan │ -└──────────────┘ -``` - -`NaN` のソート規則については、[ORDER BY 句](../../sql-reference/statements/select/order-by.md) を参照してください。 - - -## BFloat16 {#bfloat16} - -`BFloat16` は、8 ビットの指数部、符号、7 ビットの仮数部を持つ 16 ビット浮動小数点データ型です。 -機械学習や AI アプリケーションに有用です。 - -ClickHouse は `Float32` と `BFloat16` 間の変換をサポートしており、 -[`toFloat32()`](../functions/type-conversion-functions.md/#tofloat32) または [`toBFloat16`](../functions/type-conversion-functions.md/#tobfloat16) 関数で行えます。 - -:::note -その他のほとんどの演算はサポートされていません。 -::: diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md deleted file mode 100644 index f202e0fb60f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md +++ /dev/null @@ -1,206 +0,0 @@ ---- -description: '地理的なオブジェクトや位置を表現するために ClickHouse で使用されるジオメトリ(幾何)データ型のドキュメント' -sidebar_label: 'ジオ' -sidebar_position: 54 -slug: /sql-reference/data-types/geo -title: 'ジオメトリ' -doc_type: 'reference' ---- - -ClickHouse は、位置情報や領域などの地理的オブジェクトを表現するためのデータ型をサポートします。 - -**関連項目** -- [単純な地理的地物の表現](https://en.wikipedia.org/wiki/GeoJSON)。 - - - -## Point {#point} - -`Point` は X 座標と Y 座標で表現され、[Tuple](tuple.md) 型 ([Float64](float.md), [Float64](float.md)) として格納されます。 - -**例** - -クエリ: - -```sql -CREATE TABLE geo_point (p Point) ENGINE = Memory(); -INSERT INTO geo_point VALUES((10, 10)); -SELECT p, toTypeName(p) FROM geo_point; -``` - -結果: - -```text -┌─p───────┬─toTypeName(p)─┐ -│ (10,10) │ Point │ -└─────────┴───────────────┘ -``` - - -## Ring {#ring} - -`Ring` は、穴を持たない単純多角形であり、点の配列として保存されます: [Array](array.md)([Point](#point))。 - -**例** - -クエリ: - -```sql -CREATE TABLE geo_ring (r Ring) ENGINE = Memory(); -INSERT INTO geo_ring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT r, toTypeName(r) FROM geo_ring; -``` - -結果: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ Ring │ -└───────────────────────────────┴───────────────┘ -``` - - -## LineString {#linestring} - -`LineString` は、点の配列として保存される線です: [Array](array.md)([Point](#point))。 - -**例** - -クエリ: - -```sql -CREATE TABLE geo_linestring (l LineString) ENGINE = Memory(); -INSERT INTO geo_linestring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT l, toTypeName(l) FROM geo_linestring; -``` - -結果: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ LineString │ -└───────────────────────────────┴───────────────┘ -``` - - -## MultiLineString {#multilinestring} - -`MultiLineString` は、複数の線分を `LineString` の配列として格納したものです: [Array](array.md)([LineString](#linestring))。 - -**例** - -クエリ: - -```sql -CREATE TABLE geo_multilinestring (l MultiLineString) ENGINE = Memory(); -INSERT INTO geo_multilinestring VALUES([[(0, 0), (10, 0), (10, 10), (0, 10)], [(1, 1), (2, 2), (3, 3)]]); -SELECT l, toTypeName(l) FROM geo_multilinestring; -``` - -結果: - -```text -┌─l───────────────────────────────────────────────────┬─toTypeName(l)───┐ -│ [[(0,0),(10,0),(10,10),(0,10)],[(1,1),(2,2),(3,3)]] │ MultiLineString │ -└─────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Polygon {#polygon} - -`Polygon` は、[Array](array.md)([Ring](#ring)) として保存される、穴を含むポリゴンです。外側の配列の最初の要素がポリゴンの外形で、それ以降のすべての要素が穴を表します。 - -**例** - -これは 1 つの穴を持つポリゴンです: - -```sql -CREATE TABLE geo_polygon (pg Polygon) ENGINE = Memory(); -INSERT INTO geo_polygon VALUES([[(20, 20), (50, 20), (50, 50), (20, 50)], [(30, 30), (50, 50), (50, 30)]]); -SELECT pg, toTypeName(pg) FROM geo_polygon; -``` - -結果: - -```text -┌─pg────────────────────────────────────────────────────────────┬─toTypeName(pg)─┐ -│ [[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]] │ Polygon │ -└───────────────────────────────────────────────────────────────┴────────────────┘ -``` - - -## MultiPolygon {#multipolygon} - -`MultiPolygon` は複数のポリゴンで構成されており、ポリゴンの配列として格納されます: [Array](array.md)([Polygon](#polygon))。 - -**例** - -このマルチポリゴンは 2 つの別々のポリゴンで構成されています。1 つ目には穴がなく、2 つ目には 1 つの穴があります。 - -```sql -CREATE TABLE geo_multipolygon (mpg MultiPolygon) ENGINE = Memory(); -INSERT INTO geo_multipolygon VALUES([[[(0, 0), (10, 0), (10, 10), (0, 10)]], [[(20, 20), (50, 20), (50, 50), (20, 50)],[(30, 30), (50, 50), (50, 30)]]]); -SELECT mpg, toTypeName(mpg) FROM geo_multipolygon; -``` - -結果: - -```text -┌─mpg─────────────────────────────────────────────────────────────────────────────────────────────┬─toTypeName(mpg)─┐ -│ [[[(0,0),(10,0),(10,10),(0,10)]],[[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]]] │ MultiPolygon │ -└─────────────────────────────────────────────────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Geometry {#geometry} - -`Geometry` は、上記のすべての型に共通する型です。これらの型の `Variant` 型と同等です。 - -**例** - -```sql -CREATE TABLE IF NOT EXISTS geo (geom Geometry) ENGINE = Memory(); -INSERT INTO geo VALUES ((1, 2)); -SELECT * FROM geo; -``` - -結果: - -```text - ┌─geom──┐ -1. │ (1,2) │ - └───────┘ -``` - -{/* */ } - -```sql -CREATE TABLE IF NOT EXISTS geo_dst (geom Geometry) ENGINE = Memory(); - -CREATE TABLE IF NOT EXISTS geo (geom String, id Int) ENGINE = Memory(); -INSERT INTO geo VALUES ('POLYGON((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 1); -INSERT INTO geo VALUES ('POINT(0 0)', 2); -INSERT INTO geo VALUES ('MULTIPOLYGON(((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4)),((-10 -10,-10 -9,-9 10,-10 -10)))', 3); -INSERT INTO geo VALUES ('LINESTRING(1 0,10 0,10 10,0 10,1 0)', 4); -INSERT INTO geo VALUES ('MULTILINESTRING((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 5); -INSERT INTO geo_dst SELECT readWkt(geom) FROM geo ORDER BY id; - -SELECT * FROM geo_dst; -``` - -結果: - -```text - ┌─geom─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -1. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ -2. │ (0,0) │ -3. │ [[[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]],[[(-10,-10),(-10,-9),(-9,10),(-10,-10)]]] │ -4. │ [(1,0),(10,0),(10,10),(0,10),(1,0)] │ -5. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ - └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -## 関連コンテンツ {#related-content} - -- [大規模な実データセットの活用:ClickHouse で扱う 100 年以上の気象記録](https://clickhouse.com/blog/real-world-data-noaa-climate-data) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md deleted file mode 100644 index 83054fa4491..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: 'ClickHouse のデータ型に関するドキュメント' -sidebar_label: 'データ型一覧' -sidebar_position: 1 -slug: /sql-reference/data-types/ -title: 'ClickHouse のデータ型' -doc_type: 'reference' ---- - -# ClickHouse におけるデータ型 {#data-types-in-clickhouse} - -このセクションでは、ClickHouse でサポートされているデータ型について説明します。たとえば、[整数](int-uint.md)、[浮動小数点数](float.md)、[文字列](string.md) などです。 - -システムテーブル [system.data_type_families](/operations/system-tables/data_type_families) では、利用可能なすべてのデータ型の概要を確認できます。 -また、あるデータ型が別のデータ型のエイリアスかどうか、名前が大文字と小文字を区別するかどうか(例: `bool` と `BOOL`)も示します。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md deleted file mode 100644 index b5e039023b2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'ClickHouse における符号付きおよび符号なし整数データ型のドキュメントで、 - 8 ビットから 256 ビットまでを扱います。' -sidebar_label: 'Int | UInt' -sidebar_position: 2 -slug: /sql-reference/data-types/int-uint -title: 'Int | UInt 型' -doc_type: 'reference' ---- - -ClickHouse では、1 バイトから 32 バイトまでの固定長整数型を、 -符号付き(`Int`)および符号なし(`UInt`、unsigned)として提供しています。 - -テーブルを作成する際、整数型に対して数値パラメータを指定することはできます(例: `TINYINT(8)`、`SMALLINT(16)`、`INT(32)`、`BIGINT(64)`)が、ClickHouse はこれらのパラメータを無視します。 - - - -## 整数の範囲 {#integer-ranges} - -整数型の取り得る値の範囲は次のとおりです。 - -| Type | Range | -|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `Int8` | \[-128 : 127\] | -| `Int16` | \[-32768 : 32767\] | -| `Int32` | \[-2147483648 : 2147483647\] | -| `Int64` | \[-9223372036854775808 : 9223372036854775807\] | -| `Int128` | \[-170141183460469231731687303715884105728 : 170141183460469231731687303715884105727\] | -| `Int256` | \[-57896044618658097711785492504343953926634992332820282019728792003956564819968 : 57896044618658097711785492504343953926634992332820282019728792003956564819967\] | - -符号なし整数型の取り得る値の範囲は次のとおりです。 - -| Type | Range | -|-----------|----------------------------------------------------------------------------------------| -| `UInt8` | \[0 : 255\] | -| `UInt16` | \[0 : 65535\] | -| `UInt32` | \[0 : 4294967295\] | -| `UInt64` | \[0 : 18446744073709551615\] | -| `UInt128` | \[0 : 340282366920938463463374607431768211455\] | -| `UInt256` | \[0 : 115792089237316195423570985008687907853269984665640564039457584007913129639935\] | - - - -## 整数型のエイリアス {#integer-aliases} - -整数型には次のエイリアスがあります。 - -| Type | Alias | -|---------|-----------------------------------------------------------------------------------| -| `Int8` | `TINYINT`, `INT1`, `BYTE`, `TINYINT SIGNED`, `INT1 SIGNED` | -| `Int16` | `SMALLINT`, `SMALLINT SIGNED` | -| `Int32` | `INT`, `INTEGER`, `MEDIUMINT`, `MEDIUMINT SIGNED`, `INT SIGNED`, `INTEGER SIGNED` | -| `Int64` | `BIGINT`, `SIGNED`, `BIGINT SIGNED`, `TIME` | - -符号なし整数型には次のエイリアスがあります。 - -| Type | Alias | -|----------|----------------------------------------------------------| -| `UInt8` | `TINYINT UNSIGNED`, `INT1 UNSIGNED` | -| `UInt16` | `SMALLINT UNSIGNED` | -| `UInt32` | `MEDIUMINT UNSIGNED`, `INT UNSIGNED`, `INTEGER UNSIGNED` | -| `UInt64` | `UNSIGNED`, `BIGINT UNSIGNED`, `BIT`, `SET` | diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md deleted file mode 100644 index f0ace832104..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'ClickHouse の IPv4 データ型に関するドキュメント' -sidebar_label: 'IPv4' -sidebar_position: 28 -slug: /sql-reference/data-types/ipv4 -title: 'IPv4' -doc_type: 'reference' ---- - -## IPv4 {#ipv4} - -IPv4 アドレス。4 バイトの UInt32 型として格納されます。 - -### 基本的な使い方 {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv4 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -または IPv4 ドメインをキーとして使用できます: - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY from; -``` - -`IPv4` ドメインは、IPv4 文字列形式でのカスタム入力をサポートします。 - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '116.253.40.133')('https://clickhouse.com', '183.247.232.58')('https://clickhouse.com/docs/en/', '116.106.34.242'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬───────────from─┐ -│ https://clickhouse.com/docs/en/ │ 116.106.34.242 │ -│ https://wikipedia.org │ 116.253.40.133 │ -│ https://clickhouse.com │ 183.247.232.58 │ -└────────────────────────────────────┴────────────────┘ -``` - -値はコンパクトなバイナリ形式で格納されます。 - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)─┐ -│ IPv4 │ B7F7E83A │ -└──────────────────┴───────────┘ -``` - -IPv4 アドレスは IPv6 アドレスと直接比較できます。 - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [IPv4 および IPv6 アドレスを扱う関数](../functions/ip-address-functions.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md deleted file mode 100644 index ba1eed24c83..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'ClickHouse における IPv6 データ型のドキュメント。IPv6 アドレスを 16 バイトの値として保存します' -sidebar_label: 'IPv6' -sidebar_position: 30 -slug: /sql-reference/data-types/ipv6 -title: 'IPv6' -doc_type: 'reference' ---- - -## IPv6 {#ipv6} - -IPv6 アドレス。16 バイトの UInt128 型(ビッグエンディアン)として格納されます。 - -### 基本的な使い方 {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv6 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -または、`IPv6` ドメイン名をキーとして使用することもできます。 - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY from; -``` - -`IPv6` ドメインは、IPv6 文字列表現としてのカスタム入力をサポートします。 - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '2a02:aa08:e000:3100::2')('https://clickhouse.com', '2001:44c8:129:2632:33:0:252:2')('https://clickhouse.com/docs/en/', '2a02:e980:1e::1'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬─from──────────────────────────┐ -│ https://clickhouse.com │ 2001:44c8:129:2632:33:0:252:2 │ -│ https://clickhouse.com/docs/en/ │ 2a02:e980:1e::1 │ -│ https://wikipedia.org │ 2a02:aa08:e000:3100::2 │ -└────────────────────────────────────┴───────────────────────────────┘ -``` - -値はコンパクトなバイナリ形式で格納されます。 - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)────────────────────────┐ -│ IPv6 │ 200144C8012926320033000002520002 │ -└──────────────────┴──────────────────────────────────┘ -``` - -IPv6 アドレスは IPv4 アドレスと直接比較できます。 - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**関連項目** - -* [IPv4 および IPv6 アドレスを扱うための関数](../functions/ip-address-functions.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md deleted file mode 100644 index 6483aad5d0b..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '文字列カラム向け LowCardinality 最適化に関するドキュメント' -sidebar_label: 'LowCardinality(T)' -sidebar_position: 42 -slug: /sql-reference/data-types/lowcardinality -title: 'LowCardinality(T)' -doc_type: 'reference' ---- - - - -# LowCardinality(T) {#lowcardinalityt} - -他のデータ型の内部表現を辞書エンコードされた形式に変更します。 - - - -## 構文 {#syntax} - -```sql -LowCardinality(data_type) -``` - -**パラメータ** - -* `data_type` — [String](../../sql-reference/data-types/string.md)、[FixedString](../../sql-reference/data-types/fixedstring.md)、[Date](../../sql-reference/data-types/date.md)、[DateTime](../../sql-reference/data-types/datetime.md)、および [Decimal](../../sql-reference/data-types/decimal.md) 以外の数値型。`LowCardinality` は一部のデータ型では効率的ではありません。詳細は [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types) 設定の説明を参照してください。 - - -## 説明 {#description} - -`LowCardinality` は、データの格納方式とデータ処理規則を変更するための上位構造です。ClickHouse は `LowCardinality` 列に対して [dictionary coding](https://en.wikipedia.org/wiki/Dictionary_coder) を適用します。辞書エンコードされたデータを扱うことで、多くのアプリケーションにおいて [SELECT](../../sql-reference/statements/select/index.md) クエリのパフォーマンスが大幅に向上します。 - -`LowCardinality` データ型の有効性は、データの多様性に依存します。辞書に含まれる異なる値が 10,000 未満である場合、ClickHouse は一般的にデータの読み取りおよび保存の効率が高くなります。辞書に 100,000 を超える異なる値が含まれている場合、通常のデータ型を使用した場合と比較して、ClickHouse のパフォーマンスが低下することがあります。 - -文字列を扱う際は、[Enum](../../sql-reference/data-types/enum.md) の代わりに `LowCardinality` の使用を検討してください。`LowCardinality` は利用時の柔軟性が高く、多くの場合、同等またはそれ以上の効率を発揮します。 - - - -## 例 {#example} - -`LowCardinality` 列を持つテーブルを作成します。 - -```sql -CREATE TABLE lc_t -( - `id` UInt16, - `strings` LowCardinality(String) -) -ENGINE = MergeTree() -ORDER BY id -``` - - -## 関連設定と関数 {#related-settings-and-functions} - -設定: - -- [low_cardinality_max_dictionary_size](../../operations/settings/settings.md#low_cardinality_max_dictionary_size) -- [low_cardinality_use_single_dictionary_for_part](../../operations/settings/settings.md#low_cardinality_use_single_dictionary_for_part) -- [low_cardinality_allow_in_native_format](../../operations/settings/settings.md#low_cardinality_allow_in_native_format) -- [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types) -- [output_format_arrow_low_cardinality_as_dictionary](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary) - -関数: - -- [toLowCardinality](../../sql-reference/functions/type-conversion-functions.md#tolowcardinality) - - - -## 関連コンテンツ {#related-content} - -- ブログ: [Schemas と Codecs を用いた ClickHouse の最適化](https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema) -- ブログ: [ClickHouse における時系列データの扱い方](https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse) -- [String Optimization(ロシア語によるビデオプレゼンテーション)](https://youtu.be/rqf-ILRgBdY?list=PL0Z2YDlm0b3iwXCpEFiOOYmwXzVmjJfEt)。[英語スライド](https://github.com/ClickHouse/clickhouse-presentations/raw/master/meetup19/string_optimization.pdf) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md deleted file mode 100644 index 1bce230781d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -description: 'ClickHouse の Map データ型に関するドキュメント' -sidebar_label: 'Map(K, V)' -sidebar_position: 36 -slug: /sql-reference/data-types/map -title: 'Map(K, V)' -doc_type: 'reference' ---- - - - -# Map(K, V) {#mapk-v} - -データ型 `Map(K, V)` はキーと値のペアを格納します。 - -他のデータベースと異なり、ClickHouse における map ではキーは一意である必要はありません。つまり、同じキーを持つ要素を 2 つ含むことができます。 -(これは、map が内部的には `Array(Tuple(K, V))` として実装されているためです。) - -map `m` からキー `k` に対応する値を取得するには、構文 `m[k]` を使用できます。 -また、`m[k]` は map を走査するため、この操作の実行時間は map のサイズに比例します。 - -**パラメータ** - -* `K` — Map のキーの型。[Nullable](../../sql-reference/data-types/nullable.md) および [Nullable](../../sql-reference/data-types/nullable.md) 型をネストした [LowCardinality](../../sql-reference/data-types/lowcardinality.md) を除く任意の型。 -* `V` — Map の値の型。任意の型。 - -**例** - -map 型のカラムを持つテーブルを作成します。 - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':1, 'key2':10}), ({'key1':2,'key2':20}), ({'key1':3,'key2':30}); -``` - -`key2` の値を選択するには: - -```sql -SELECT m['key2'] FROM tab; -``` - -結果: - -```text -┌─arrayElement(m, 'key2')─┐ -│ 10 │ -│ 20 │ -│ 30 │ -└─────────────────────────┘ -``` - -指定したキー `k` がマップ内に含まれていない場合、`m[k]` は値型のデフォルト値を返します。例えば、整数型なら `0`、文字列型なら `''` です。 -マップ内にキーが存在するかどうかを確認するには、[mapContains](../../sql-reference/functions/tuple-map-functions#mapcontains) 関数を使用します。 - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':100}), ({}); -SELECT m['key1'] FROM tab; -``` - -結果: - -```text -┌─arrayElement(m, 'key1')─┐ -│ 100 │ -│ 0 │ -└─────────────────────────┘ -``` - - -## Tuple から Map への変換 {#converting-tuple-to-map} - -`Tuple()` 型の値は、[CAST](/sql-reference/functions/type-conversion-functions#cast) 関数を使用して `Map()` 型にキャストできます。 - -**例** - -クエリ: - -```sql -SELECT CAST(([1, 2, 3], ['Ready', 'Steady', 'Go']), 'Map(UInt8, String)') AS map; -``` - -結果: - -```text -┌─map───────────────────────────┐ -│ {1:'Ready',2:'Steady',3:'Go'} │ -└───────────────────────────────┘ -``` - - -## Map のサブカラムの読み取り {#reading-subcolumns-of-map} - -Map 全体を読み出さずに済むように、場合によってはサブカラム `keys` と `values` を使用できます。 - -**例** - -クエリ: - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE = Memory; -INSERT INTO tab VALUES (map('key1', 1, 'key2', 2, 'key3', 3)); - -SELECT m.keys FROM tab; -- mapKeys(m)と同じ -SELECT m.values FROM tab; -- mapValues(m)と同じ -``` - -結果: - -```text -┌─m.keys─────────────────┐ -│ ['key1','key2','key3'] │ -└────────────────────────┘ - -┌─m.values─┐ -│ [1,2,3] │ -└──────────┘ -``` - -**関連項目** - -* [map()](/sql-reference/functions/tuple-map-functions#map) 関数 -* [CAST()](/sql-reference/functions/type-conversion-functions#cast) 関数 -* [Map データ型用 -Map コンビネータ](../aggregate-functions/combinators.md#-map) - - -## 関連コンテンツ {#related-content} - -- ブログ記事: [Building an Observability Solution with ClickHouse - Part 2 - Traces](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md deleted file mode 100644 index 19a72c2e318..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -description: 'ClickHouse におけるネストされたデータ構造の概要' -sidebar_label: 'Nested(Name1 Type1, Name2 Type2, ...)' -sidebar_position: 57 -slug: /sql-reference/data-types/nested-data-structures/nested -title: 'Nested' -doc_type: 'guide' ---- - -# ネスト {#nested} - -## Nested(name1 Type1, Name2 Type2, ...) {#nestedname1-type1-name2-type2-} - -ネストされたデータ構造は、セルの中にあるテーブルのようなものです。ネストされたデータ構造のパラメータ(列名や型)は、[CREATE TABLE](../../../sql-reference/statements/create/table.md) クエリの場合と同様に指定します。各テーブル行は、ネストされたデータ構造内の任意の数の行に対応できます。 - -例: - -```sql -CREATE TABLE test.visits -( - CounterID UInt32, - StartDate Date, - Sign Int8, - IsNew UInt8, - VisitID UInt64, - UserID UInt64, - ... - Goals Nested - ( - ID UInt32, - Serial UInt32, - EventTime DateTime, - Price Int64, - OrderID String, - CurrencyID UInt32 - ), - ... -) ENGINE = CollapsingMergeTree(StartDate, intHash32(UserID), (CounterID, StartDate, intHash32(UserID), VisitID), 8192, Sign) -``` - -この例では、コンバージョン(達成されたゴール)に関するデータを含むネストされたデータ構造 `Goals` を宣言しています。`visits` テーブルの各行は、0 個または任意の数のコンバージョンに対応できます。 - -[flatten_nested](/operations/settings/settings#flatten_nested) が `0` に設定されている場合(デフォルト値は 0 ではありません)、任意のレベルのネストがサポートされます。 - -ほとんどの場合、ネストされたデータ構造を扱う際には、そのカラムはドットで区切られたカラム名で指定されます。これらのカラムは、対応する型の配列になります。単一のネストされたデータ構造に属するすべてのカラム配列は同じ長さを持ちます。 - -例: - -```sql -SELECT - Goals.ID, - Goals.EventTime -FROM test.visits -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goals.ID───────────────────────┬─Goals.EventTime───────────────────────────────────────────────────────────────────────────┐ -│ [1073752,591325,591325] │ ['2014-03-17 16:38:10','2014-03-17 16:38:48','2014-03-17 16:42:27'] │ -│ [1073752] │ ['2014-03-17 00:28:25'] │ -│ [1073752] │ ['2014-03-17 10:46:20'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:59:20','2014-03-17 22:17:55','2014-03-17 22:18:07','2014-03-17 22:18:51'] │ -│ [] │ [] │ -│ [1073752,591325,591325] │ ['2014-03-17 11:37:06','2014-03-17 14:07:47','2014-03-17 14:36:21'] │ -│ [] │ [] │ -│ [] │ [] │ -│ [591325,1073752] │ ['2014-03-17 00:46:05','2014-03-17 00:46:05'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:28:33','2014-03-17 13:30:26','2014-03-17 18:51:21','2014-03-17 18:51:45'] │ -└────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -ネストされたデータ構造は、同じ長さの複数のカラム配列の集合として考えるのが最も分かりやすいです。 - -`SELECT` クエリで、個々のカラムではなくネストされたデータ構造全体の名前を指定できる唯一の場所は、`ARRAY JOIN` 句です。詳細については「ARRAY JOIN 句」を参照してください。例: - -```sql -SELECT - Goal.ID, - Goal.EventTime -FROM test.visits -ARRAY JOIN Goals AS Goal -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goal.ID─┬──────Goal.EventTime─┐ -│ 1073752 │ 2014-03-17 16:38:10 │ -│ 591325 │ 2014-03-17 16:38:48 │ -│ 591325 │ 2014-03-17 16:42:27 │ -│ 1073752 │ 2014-03-17 00:28:25 │ -│ 1073752 │ 2014-03-17 10:46:20 │ -│ 1073752 │ 2014-03-17 13:59:20 │ -│ 591325 │ 2014-03-17 22:17:55 │ -│ 591325 │ 2014-03-17 22:18:07 │ -│ 591325 │ 2014-03-17 22:18:51 │ -│ 1073752 │ 2014-03-17 11:37:06 │ -└─────────┴─────────────────────┘ -``` - -ネストされたデータ構造全体に対して `SELECT` を実行することはできません。その一部である個々のカラムのみを明示的に指定できます。 - -`INSERT` クエリの場合、ネストされたデータ構造を構成するすべてのカラム配列を(それぞれが個別のカラム配列であるかのように)個別に渡す必要があります。挿入時に、システムはそれらの長さが同じであることをチェックします。 - -`DESCRIBE` クエリでは、ネストされたデータ構造内のカラムは、同様に個別に列挙されます。 - -ネストされたデータ構造内の要素に対する `ALTER` クエリには、いくつかの制限があります。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md deleted file mode 100644 index ca8ab3a9794..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md +++ /dev/null @@ -1,1065 +0,0 @@ ---- -description: 'ClickHouse における JSON データ型のドキュメントで、JSON データを扱うためのネイティブサポートについて説明します' -keywords: ['json', 'data type'] -sidebar_label: 'JSON' -sidebar_position: 63 -slug: /sql-reference/data-types/newjson -title: 'JSON データ型' -doc_type: 'reference' ---- - -import {CardSecondary} from '@clickhouse/click-ui/bundled'; -import Link from '@docusaurus/Link' - - - - - -
- -`JSON` 型は、JavaScript Object Notation (JSON) ドキュメントを 1 つの列に保存します。 - -:::note -ClickHouse オープンソース版では、バージョン 25.3 で JSON データ型が本番利用可能 (production ready) としてマークされています。以前のバージョンでこの型を本番環境で使用することは推奨されません。 -::: - -`JSON` 型の列を宣言するには、次の構文を使用できます。 - -```sql -<カラム名> JSON -( - max_dynamic_paths=N, - max_dynamic_types=M, - some.path 型名, - SKIP path.to.skip, - SKIP REGEXP 'paths_regexp' -) -``` - -上記の構文内の各パラメータは、次のように定義されます。 - -| Parameter | Description | Default Value | -| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| `max_dynamic_paths` | 個別に保存される 1 つのデータブロック内で(例えば、MergeTree テーブルの 1 つのデータパーツ内で)、いくつのパスをサブカラムとして個別に保持できるかを示すオプションのパラメータです。

この制限を超えた場合、それ以外のすべてのパスは 1 つの共通の構造体としてまとめて保存されます。 | `1024` | -| `max_dynamic_types` | 個別に保存される 1 つのデータブロック内で(例えば、MergeTree テーブルの 1 つのデータパーツ内で)、型 `Dynamic` を持つ 1 つのパスカラム内にいくつの異なるデータ型を保存できるかを示す、`1` から `255` の範囲のオプションのパラメータです。

この制限を超えた場合、新しい型はすべて `String` 型に変換されます。 | `32` | -| `some.path TypeName` | JSON 内の特定のパスに対するオプションの型ヒントです。そのようなパスは、常に指定された型のサブカラムとして保存されます。 | | -| `SKIP path.to.skip` | JSON パース時にスキップすべき特定のパスを指定するオプションのヒントです。そのようなパスは JSON カラム内に保存されることはありません。指定されたパスがネストされた JSON オブジェクトである場合、そのネストされたオブジェクト全体がスキップされます。 | | -| `SKIP REGEXP 'path_regexp'` | JSON パース中にパスをスキップするために使用される正規表現を指定するオプションのヒントです。この正規表現にマッチするすべてのパスは、JSON カラム内に保存されることはありません。 | | - - -## JSON の生成 {#creating-json} - -このセクションでは、`JSON` を生成するさまざまな方法を確認します。 - -### テーブルの列定義で `JSON` を使用する {#using-json-in-a-table-column-definition} - -```sql title="Query (Example 1)" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 1)" -┌─json────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ -│ {"f":"こんにちは、世界!"} │ -│ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │ -└─────────────────────────────────────────────┘ -``` - -```sql title="Query (Example 2)" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 2)" -┌─json──────────────────────────────┐ -│ {"a":{"b":42},"c":["1","2","3"]} │ -│ {"a":{"b":0},"f":"こんにちは、世界!"} │ -│ {"a":{"b":43},"c":["4","5","6"]} │ -└───────────────────────────────────┘ -``` - -### `::JSON` を使用した CAST {#using-cast-with-json} - -特別な構文 `::JSON` を使用して、さまざまな型の値を `JSON` 型にキャストできます。 - -#### `String` から `JSON` への CAST {#cast-from-string-to-json} - -```sql title="Query" -SELECT '{"a" : {"b" : 42},"c" : [1, 2, 3], "d" : "Hello, World!"}'::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"こんにちは、世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### `Tuple` から `JSON` への CAST {#cast-from-tuple-to-json} - -```sql title="Query" -SET enable_named_columns_in_function_tuple = 1; -SELECT (tuple(42 AS b) AS a, [1, 2, 3] AS c, 'Hello, World!' AS d)::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"こんにちは、世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### `Map` を `JSON` に CAST する {#cast-from-map-to-json} - -```sql title="Query" -SET use_variant_as_common_type=1; -SELECT map('a', map('b', 42), 'c', [1,2,3], 'd', 'Hello, World!')::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"こんにちは、世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -:::note -JSON のパスはフラット化されたパスとして保存されます。これは、`a.b.c` のようなパスから JSON オブジェクトを整形する際に、 -そのオブジェクトを `{ "a.b.c" : ... }` として構築すべきなのか、あるいは `{ "a": { "b": { "c": ... } } }` として構築すべきなのかを判別できないことを意味します。 -本実装では常に後者として解釈します。 - -例えば次のとおりです。 - -```sql -SELECT CAST('{"a.b.c" : 42}', 'JSON') AS json -``` - -は以下を返します: - -```response - ┌─json───────────────────┐ -1. │ {"a":{"b":{"c":"42"}}} │ - └────────────────────────┘ -``` - -そして、**次のようにしてはいけません**: - - -```sql - ┌─json───────────┐ -1. │ {"a.b.c":"42"} │ - └────────────────┘ -``` - -::: - - -## JSON パスをサブカラムとして読み取る {#reading-json-paths-as-sub-columns} - -`JSON` 型では、各パスを個別のサブカラムとして読み取ることができます。 -要求されたパスの型が JSON 型の宣言で指定されていない場合、 -そのパスのサブカラムは常に型 [Dynamic](/sql-reference/data-types/dynamic.md) になります。 - -例: - -```sql title="Query" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42, "g" : 42.42}, "c" : [1, 2, 3], "d" : "2020-01-01"}'), ('{"f" : "Hello, World!", "d" : "2020-01-02"}'), ('{"a" : {"b" : 43, "e" : 10, "g" : 43.43}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────┐ -│ {"a":{"b":42,"g":42.42},"c":["1","2","3"],"d":"2020-01-01"} │ -│ {"a":{"b":0},"d":"2020-01-02","f":"Hello, World!"} │ -│ {"a":{"b":43,"g":43.43},"c":["4","5","6"]} │ -└─────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query (Reading JSON paths as sub-columns)" -SELECT json.a.b, json.a.g, json.c, json.d FROM test; -``` - -```text title="Response (Reading JSON paths as sub-columns)" -┌─json.a.b─┬─json.a.g─┬─json.c──┬─json.d─────┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└──────────┴──────────┴─────────┴────────────┘ -``` - -`getSubcolumn` 関数を使用して、`JSON` 型からサブカラムを読み取ることもできます。 - -```sql title="Query" -SELECT getSubcolumn(json, 'a.b'), getSubcolumn(json, 'a.g'), getSubcolumn(json, 'c'), getSubcolumn(json, 'd') FROM test; -``` - -```text title="Response" -┌─getSubcolumn(json, 'a.b')─┬─getSubcolumn(json, 'a.g')─┬─getSubcolumn(json, 'c')─┬─getSubcolumn(json, 'd')─┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└───────────────────────────┴───────────────────────────┴─────────────────────────┴─────────────────────────┘ -``` - -要求されたパスがデータ内に存在しなかった場合、そのパスは `NULL` 値で埋められます: - -```sql title="Query" -SELECT json.non.existing.path FROM test; -``` - -```text title="Response" -┌─json.non.existing.path─┐ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -└────────────────────────┘ -``` - -返されたサブカラムのデータ型を確認してみましょう。 - -```sql title="Query" -SELECT toTypeName(json.a.b), toTypeName(json.a.g), toTypeName(json.c), toTypeName(json.d) FROM test; -``` - - -```text title="Response" -┌─toTypeName(json.a.b)─┬─toTypeName(json.a.g)─┬─toTypeName(json.c)─┬─toTypeName(json.d)─┐ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -└──────────────────────┴──────────────────────┴────────────────────┴────────────────────┘ -``` - -上記のとおり、`a.b` については JSON の型宣言で指定したとおり型は `UInt32` になっており、 -それ以外のすべてのサブカラムの型は `Dynamic` になっています。 - -また、特別な構文 `json.some.path.:TypeName` を使用して、`Dynamic` 型のサブカラムを読み取ることもできます。 - -```sql title="Query" -SELECT - json.a.g.:Float64, - dynamicType(json.a.g), - json.d.:Date, - dynamicType(json.d) -FROM test -``` - -```text title="Response" -┌─json.a.g.:`Float64`─┬─dynamicType(json.a.g)─┬─json.d.:`Date`─┬─dynamicType(json.d)─┐ -│ 42.42 │ Float64 │ 2020-01-01 │ Date │ -│ ᴺᵁᴸᴸ │ None │ 2020-01-02 │ Date │ -│ 43.43 │ Float64 │ ᴺᵁᴸᴸ │ None │ -└─────────────────────┴───────────────────────┴────────────────┴─────────────────────┘ -``` - -`Dynamic` のサブカラムは任意のデータ型にキャストできます。このとき、`Dynamic` の内部型を要求された型にキャストできない場合には、例外がスローされます。 - -```sql title="Query" -SELECT json.a.g::UInt64 AS uint -FROM test; -``` - -```text title="Response" -┌─uint─┐ -│ 42 │ -│ 0 │ -│ 43 │ -└──────┘ -``` - -```sql title="Query" -SELECT json.a.g::UUID AS float -FROM test; -``` - -```text title="Response" -サーバーから例外を受信しました: -Code: 48. DB::Exception: Received from localhost:9000. DB::Exception: -数値型とUUID間の変換はサポートされていません。 -渡されたUUIDが引用符で囲まれていない可能性があります: -'FUNCTION CAST(__table1.json.a.g :: 2, 'UUID'_String :: 1) -> CAST(__table1.json.a.g, 'UUID'_String) UUID : 0'の実行中。 -(NOT_IMPLEMENTED) -``` - -:::note -Compact MergeTree のパーツからサブカラムを効率的に読み込むには、MergeTree 設定 [write_marks_for_substreams_in_compact_parts](../../operations/settings/merge-tree-settings.md#write_marks_for_substreams_in_compact_parts) が有効になっていることを確認してください。 -::: - - -## JSON サブオブジェクトをサブカラムとして読み取る {#reading-json-sub-objects-as-sub-columns} - -`JSON` 型では、特別な構文 `json.^some.path` を使うことで、ネストされたオブジェクトを型 `JSON` のサブカラムとして読み取ることができます。 - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : {"c" : 42, "g" : 42.42}}, "c" : [1, 2, 3], "d" : {"e" : {"f" : {"g" : "Hello, World", "h" : [1, 2, 3]}}}}'), ('{"f" : "Hello, World!", "d" : {"e" : {"f" : {"h" : [4, 5, 6]}}}}'), ('{"a" : {"b" : {"c" : 43, "e" : 10, "g" : 43.43}}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":"42","g":42.42}},"c":["1","2","3"],"d":{"e":{"f":{"g":"Hello, World","h":["1","2","3"]}}}} │ -│ {"d":{"e":{"f":{"h":["4","5","6"]}}},"f":"Hello, World!"} │ -│ {"a":{"b":{"c":"43","e":"10","g":43.43}},"c":["4","5","6"]} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.^a.b, json.^d.e.f FROM test; -``` - -```text title="Response" -┌─json.^`a`.b───────────────────┬─json.^`d`.e.f──────────────────────────┐ -│ {"c":"42","g":42.42} │ {"g":"Hello, World","h":["1","2","3"]} │ -│ {} │ {"h":["4","5","6"]} │ -│ {"c":"43","e":"10","g":43.43} │ {} │ -└───────────────────────────────┴────────────────────────────────────────┘ -``` - -:::note -サブオブジェクトをサブカラムとして読み出すことは非効率になる可能性があります。JSON データをほぼ全件スキャンする必要が生じる場合があるためです。 -::: - - -## パスの型推論 {#type-inference-for-paths} - -`JSON` のパース中、ClickHouse は各 JSON パスに対して最も適切なデータ型を推定しようとします。 -これは [入力データからの自動スキーマ推論](/interfaces/schema-inference.md) と同様に動作し、 -同じ設定によって制御されます: - -* [input_format_try_infer_dates](/operations/settings/formats#input_format_try_infer_dates) -* [input_format_try_infer_datetimes](/operations/settings/formats#input_format_try_infer_datetimes) -* [schema_inference_make_columns_nullable](/operations/settings/formats#schema_inference_make_columns_nullable) -* [input_format_json_try_infer_numbers_from_strings](/operations/settings/formats#input_format_json_try_infer_numbers_from_strings) -* [input_format_json_infer_incomplete_types_as_strings](/operations/settings/formats#input_format_json_infer_incomplete_types_as_strings) -* [input_format_json_read_numbers_as_strings](/operations/settings/formats#input_format_json_read_numbers_as_strings) -* [input_format_json_read_bools_as_strings](/operations/settings/formats#input_format_json_read_bools_as_strings) -* [input_format_json_read_bools_as_numbers](/operations/settings/formats#input_format_json_read_bools_as_numbers) -* [input_format_json_read_arrays_as_strings](/operations/settings/formats#input_format_json_read_arrays_as_strings) -* [input_format_json_infer_array_of_dynamic_from_array_of_different_types](/operations/settings/formats#input_format_json_infer_array_of_dynamic_from_array_of_different_types) - -いくつかの例を見てみましょう。 - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=1, input_format_try_infer_datetimes=1; -``` - -```text title="Response" -┌─paths_with_types─────────────────┐ -│ {'a':'Date','b':'DateTime64(9)'} │ -└──────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=0, input_format_try_infer_datetimes=0; -``` - -```text title="Response" -┌─paths_with_types────────────┐ -│ {'a':'String','b':'String'} │ -└─────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types settings schema_inference_make_columns_nullable=1; -``` - -```text title="Response" -┌─paths_with_types───────────────┐ -│ {'a':'Array(Nullable(Int64))'} │ -└────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types settings schema_inference_make_columns_nullable=0; -``` - -```text title="Response" -┌─paths_with_types─────┐ -│ {'a':'Array(Int64)'} │ -└──────────────────────┘ -``` - - -## JSON オブジェクト配列の扱い方 {#handling-arrays-of-json-objects} - -JSON オブジェクトの配列を含む JSON パスは、型 `Array(JSON)` として解釈され、そのパスに対応する `Dynamic` 列に挿入されます。 -オブジェクト配列を読み取るには、`Dynamic` 列からサブカラムとして抽出します。 - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES -('{"a" : {"b" : [{"c" : 42, "d" : "Hello", "f" : [[{"g" : 42.42}]], "k" : {"j" : 1000}}, {"c" : 43}, {"e" : [1, 2, 3], "d" : "My", "f" : [[{"g" : 43.43, "h" : "2020-01-01"}]], "k" : {"j" : 2000}}]}}'), -('{"a" : {"b" : [1, 2, 3]}}'), -('{"a" : {"b" : [{"c" : 44, "f" : [[{"h" : "2020-01-02"}]]}, {"e" : [4, 5, 6], "d" : "World", "f" : [[{"g" : 44.44}]], "k" : {"j" : 3000}}]}}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":[{"c":"42","d":"こんにちは","f":[[{"g":42.42}]],"k":{"j":"1000"}},{"c":"43"},{"d":"私の","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}]}} │ -│ {"a":{"b":["1","2","3"]}} │ -│ {"a":{"b":[{"c":"44","f":[[{"h":"2020-01-02"}]]},{"d":"世界","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}]}} │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.a.b, dynamicType(json.a.b) FROM test; -``` - -```text title="Response" -┌─json.a.b──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─dynamicType(json.a.b)────────────────────────────────────┐ -│ ['{"c":"42","d":"Hello","f":[[{"g":42.42}]],"k":{"j":"1000"}}','{"c":"43"}','{"d":"My","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -│ [1,2,3] │ Array(Nullable(Int64)) │ -│ ['{"c":"44","f":[[{"h":"2020-01-02"}]]}','{"d":"World","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────┘ -``` - -ご覧のとおり、ネストされた `JSON` 型の `max_dynamic_types` と `max_dynamic_paths` パラメータは、デフォルト値よりも小さい値に減らされています。 -これは、JSON オブジェクトの配列が入れ子になっている場合に、サブカラムの数が際限なく増加するのを防ぐために必要です。 - -ネストされた `JSON` カラムからサブカラムを読み取ってみましょう。 - -```sql title="Query" -SELECT json.a.b.:`Array(JSON)`.c, json.a.b.:`Array(JSON)`.f, json.a.b.:`Array(JSON)`.d FROM test; -``` - - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -特別な構文を使うことで、`Array(JSON)` サブカラムの名前を記述せずに済みます。 - -```sql title="Query" -SELECT json.a.b[].c, json.a.b[].f, json.a.b[].d FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -パスの後に続く `[]` の数は配列のネストレベルを示します。例えば、`json.path[][]` は `json.path.:Array(Array(JSON))` に変換されます。 - -`Array(JSON)` 内のパスと型を確認してみましょう。 - -```sql title="Query" -SELECT DISTINCT arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b[]))) FROM test; -``` - -```text title="Response" -┌─arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b.:`Array(JSON)`)))──┐ -│ ('c','Int64') │ -│ ('d','String') │ -│ ('f','Array(Array(JSON(max_dynamic_types=8, max_dynamic_paths=64)))') │ -│ ('k.j','Int64') │ -│ ('e','Array(Nullable(Int64))') │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -`Array(JSON)` 型の列からサブカラムを読み取ってみましょう: - -```sql title="Query" -SELECT json.a.b[].c.:Int64, json.a.b[].f[][].g.:Float64, json.a.b[].f[][].h.:Date FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c.:`Int64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.g.:`Float64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.h.:`Date`─┐ -│ [42,43,NULL] │ [[[42.42]],[],[[43.43]]] │ [[[NULL]],[],[['2020-01-01']]] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[[NULL]],[[44.44]]] │ [[['2020-01-02']],[[NULL]]] │ -└────────────────────────────────────┴──────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────┘ -``` - -ネストされた `JSON` カラムからサブオブジェクト内のサブカラムを読み取ることもできます。 - -```sql title="Query" -SELECT json.a.b[].^k FROM test -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.^`k`─────────┐ -│ ['{"j":"1000"}','{}','{"j":"2000"}'] │ -│ [] │ -│ ['{}','{"j":"3000"}'] │ -└──────────────────────────────────────┘ -``` - - -## NULL を含む JSON キーの扱い {#handling-json-keys-with-nulls} - -本製品の JSON 実装では、`null` と値が存在しない状態は同等と見なされます。 - -```sql title="Query" -SELECT '{}'::JSON AS json1, '{"a" : null}'::JSON AS json2, json1 = json2 -``` - -```text title="Response" -┌─json1─┬─json2─┬─equals(json1, json2)─┐ -│ {} │ {} │ 1 │ -└───────┴───────┴──────────────────────┘ -``` - -これは、元の JSON データに NULL 値を持つパスが含まれていたのか、それともそのようなパスがそもそも含まれていなかったのかを判定することが不可能であることを意味します。 - - -## ドットを含む JSON キーの扱い {#handling-json-keys-with-dots} - -内部的に、JSON カラムではすべてのパスと値がフラットな形式で保存されます。つまり、デフォルトでは次の 2 つのオブジェクトは同一のものとして扱われます。 - -```json -{"a" : {"b" : 42}} -{"a.b" : 42} -``` - -どちらも内部的には、パス `a.b` と値 `42` の組として保存されます。JSON を整形する際には、ドットで区切られたパスの各要素に基づいて、常に入れ子のオブジェクトを構築します。 - -```sql title="Query" -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a":{"b":"42"}} │ ['a.b'] │ ['a.b'] │ -└──────────────────┴──────────────────┴─────────────────────┴─────────────────────┘ -``` - -ご覧のとおり、元の JSON `{"a.b" : 42}` は、`{"a" : {"b" : 42}}` のような形式に変換されています。 - -この制約により、次のような有効な JSON オブジェクトも正しく解析できません。 - -```sql title="Query" -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json; -``` - -```text title="Response" -コード: 117. DB::Exception: JSONカラムへのデータ挿入ができません: JSONオブジェクトの解析中に重複パスが検出されました: a.b。挿入時に重複パスをスキップするには、type_json_skip_duplicated_paths 設定を有効にしてください: スコープ SELECT CAST('{"a.b" : 42, "a" : {"b" : "Hello, World"}}', 'JSON') AS json 内。(INCORRECT_DATA) -``` - -ドットを含むキーをそのまま保持し、それらをネストされたオブジェクトとして解釈させたくない場合は、`25.8` バージョン以降で利用可能な設定 [json_type_escape_dots_in_keys](/operations/settings/formats#json_type_escape_dots_in_keys) を有効にしてください。この場合、パース時に JSON キー内のすべてのドットは `%2E` にエスケープされ、フォーマット時に元に戻されます。 - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a.b":"42"} │ ['a.b'] │ ['a%2Eb'] │ -└──────────────────┴──────────────┴─────────────────────┴─────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, JSONAllPaths(json); -``` - -```text title="Response" -┌─json──────────────────────────────────┬─JSONAllPaths(json)─┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ ['a%2Eb','a.b'] │ -└───────────────────────────────────────┴────────────────────┘ -``` - -エスケープされたドットを含むキーをサブカラムとして読み取るには、サブカラム名にもエスケープされたドットを使用する必要があります。 - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┘ -``` - -注意: 識別子パーサーおよびアナライザーの制限により、サブカラム `` json.`a.b` `` はサブカラム `json.a.b` と同等とみなされ、エスケープされたドットを含むパスとしては解釈されません: - - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.`a.b`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┴──────────────┘ -``` - -また、キー名にドットを含む JSON パスに対してヒントを指定する場合(あるいはそれを `SKIP` / `SKIP REGEX` セクションで使用する場合)は、ヒント内ではドットをエスケープして記述する必要があります。 - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(`a%2Eb` UInt8) as json, json.`a%2Eb`, toTypeName(json.`a%2Eb`); -``` - -```text title="Response" -┌─json────────────────────────────────┬─json.a%2Eb─┬─toTypeName(json.a%2Eb)─┐ -│ {"a.b":42,"a":{"b":"Hello World!"}} │ 42 │ UInt8 │ -└─────────────────────────────────────┴────────────┴────────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(SKIP `a%2Eb`) as json, json.`a%2Eb`; -``` - -```text title="Response" -┌─json───────────────────────┬─json.a%2Eb─┐ -│ {"a":{"b":"こんにちは世界!"}} │ ᴺᵁᴸᴸ │ -└────────────────────────────┴────────────┘ -``` - - -## データから JSON 型を読み込む {#reading-json-type-from-data} - -すべてのテキストフォーマット -([`JSONEachRow`](/interfaces/formats/JSONEachRow), -[`TSV`](/interfaces/formats/TabSeparated), -[`CSV`](/interfaces/formats/CSV), -[`CustomSeparated`](/interfaces/formats/CustomSeparated), -[`Values`](/interfaces/formats/Values) など) は、`JSON` 型のデータの読み取りをサポートしています。 - -例: - -```sql title="Query" -SELECT json FROM format(JSONEachRow, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP d.e, SKIP REGEXP \'b.*\')', ' -{"json" : {"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}}} -{"json" : {"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}}} -{"json" : {"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"}} -{"json" : {"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43}} -{"json" : {"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}} -') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"こんにちは、世界!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - -`CSV` や `TSV` などのテキスト形式では、`JSON` は JSON オブジェクトを含む文字列から解析されます。 - - -```sql title="Query" -SELECT json FROM format(TSV, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP REGEXP \'b.*\')', -'{"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}} -{"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}} -{"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"} -{"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43} -{"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"こんにちは、世界!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - - -## JSON 内部の動的パス数の上限に到達する場合 {#reaching-the-limit-of-dynamic-paths-inside-json} - -`JSON` データ型は、内部的にサブカラムとして保持できるパスの数に上限があります。 -デフォルトではこの上限は `1024` ですが、型宣言時にパラメータ `max_dynamic_paths` を指定して変更できます。 - -上限に到達すると、その後に `JSON` カラムへ挿入される新しいパスは、1 つの共有データ構造に格納されます。 -そのようなパスも引き続きサブカラムとして読み取ることは可能ですが、 -効率は低下する可能性があります([共有データに関するセクション](#shared-data-structure) を参照)。 -この上限は、テーブルが使用不能になるほど膨大な数のサブカラムが作成されるのを防ぐために必要です。 - -いくつかのシナリオで、この上限に到達した場合に何が起こるかを見ていきます。 - -### データのパース中に上限へ到達する場合 {#reaching-the-limit-during-data-parsing} - -データから `JSON` オブジェクトをパースしている際に、現在のデータブロックで上限に達した場合、 -それ以降のすべての新しいパスは共有データ構造に格納されます。次の 2 つのイントロスペクション関数 `JSONDynamicPaths`、`JSONSharedDataPaths` を使用できます。 - -```sql title="Query" -SELECT json, JSONDynamicPaths(json), JSONSharedDataPaths(json) FROM format(JSONEachRow, 'json JSON(max_dynamic_paths=3)', ' -{"json" : {"a" : {"b" : 42}, "c" : [1, 2, 3]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-01"}} -{"json" : {"a" : {"b" : 44}, "c" : [4, 5, 6]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-02", "e" : "Hello", "f" : {"g" : 42.42}}} -{"json" : {"a" : {"b" : 43}, "c" : [7, 8, 9], "f" : {"g" : 43.43}, "h" : "World"}} -') -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────────────┬─JSONDynamicPaths(json)─┬─JSONSharedDataPaths(json)─┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-01"} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"44"},"c":["4","5","6"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-02","e":"Hello","f":{"g":42.42}} │ ['a.b','c','d'] │ ['e','f.g'] │ -│ {"a":{"b":"43"},"c":["7","8","9"],"f":{"g":43.43},"h":"World"} │ ['a.b','c','d'] │ ['f.g','h'] │ -└────────────────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────┘ -``` - -ご覧のとおり、パス `e` と `f.g` を挿入した後に上限に達し、 -それらは共有データ構造に格納されました。 - -### MergeTree テーブルエンジンにおけるデータパーツのマージ中 {#during-merges-of-data-parts-in-mergetree-table-engines} - -`MergeTree` テーブルで複数のデータパーツをマージする際、結果として得られるデータパーツ内の `JSON` 列が動的パスの上限に達し、 -元のパーツに含まれるすべてのパスをサブカラムとして保持できなくなる場合があります。 -この場合、ClickHouse はマージ後もサブカラムとして残すパスと、 -共有データ構造に格納するパスを選択します。 -多くの場合、ClickHouse は非 NULL 値の数が最も多いパスを保持し、 -出現頻度の低いパスを共有データ構造に移動しようとします。 -ただし、これは実装に依存します。 - -このようなマージの例を見てみましょう。 -まず、`JSON` 列を持つテーブルを作成し、動的パスの上限を `3` に設定してから、 -`5` つの異なるパスを持つ値を挿入してみます。 - - -```sql title="Query" -CREATE TABLE test (id UInt64, json JSON(max_dynamic_paths=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as a) FROM numbers(5); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as b) FROM numbers(4); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as c) FROM numbers(3); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as d) FROM numbers(2); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as e) FROM numbers(1); -``` - -各 `INSERT` は、1 つのパスのみを含む `JSON` 列を持つ、別個のデータパーツを作成します。 - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 5 │ ['a'] │ [] │ all_1_1_0 │ -│ 4 │ ['b'] │ [] │ all_2_2_0 │ -│ 3 │ ['c'] │ [] │ all_3_3_0 │ -│ 2 │ ['d'] │ [] │ all_4_4_0 │ -│ 1 │ ['e'] │ [] │ all_5_5_0 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -では、すべてのパーツをひとつにまとめて、どうなるか確認してみましょう。 - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 15 │ ['a','b','c'] │ ['d','e'] │ all_1_5_2 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -ご覧のとおり、ClickHouse は最も頻出するパスである `a`、`b`、`c` を保持し、パス `d` と `e` を共有のデータ構造に移しました。 - - -## 共有データ構造 {#shared-data-structure} - -前のセクションで説明したように、`max_dynamic_paths` の制限に達すると、すべての新しいパスは 1 つの共有データ構造に保存されます。 -このセクションでは、その共有データ構造の詳細と、そこからパスのサブカラムをどのように読み取るかを見ていきます。 - -JSON カラムの内容を調査するために使用される関数の詳細については、「["introspection functions"](/sql-reference/data-types/newjson#introspection-functions)」セクションを参照してください。 - -### メモリ上の共有データ構造 {#shared-data-structure-in-memory} - -メモリ上では、共有データ構造は単に `Map(String, String)` 型のサブカラムであり、フラット化された JSON パスからバイナリエンコードされた値へのマッピングを保持します。 -そこから特定のパスのサブカラムを抽出するには、この `Map` カラムのすべての行を走査し、要求されたパスとその値を探します。 - -### MergeTree パーツ内の共有データ構造 {#shared-data-structure-in-merge-tree-parts} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) テーブルでは、ローカルまたはリモートのディスク上にあるすべてのデータを格納するデータパーツにデータを保存します。ディスク上のデータは、メモリとは異なる方式で保存される場合があります。 -現在、MergeTree のデータパーツには 3 種類の共有データ構造シリアライゼーションがあります: `map`、`map_with_buckets`、 -および `advanced` です。 - -シリアライゼーションのバージョンは MergeTree の -設定 [object_shared_data_serialization_version](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version) -と [object_shared_data_serialization_version_for_zero_level_parts](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version_for_zero_level_parts) -によって制御されます(ゼロレベルパーツはテーブルへのデータ挿入時に作成されるパーツであり、マージ中に作成されるパーツはより高いレベルを持ちます)。 - -注記: 共有データ構造のシリアライゼーションの変更がサポートされるのは、 -`v3` の [object serialization version](../../operations/settings/merge-tree-settings.md#object_serialization_version) の場合のみです。 - -#### Map {#shared-data-map} - -`map` シリアライゼーションバージョンでは、共有データはメモリ上と同様に、`Map(String, String)` 型の単一カラムとしてシリアライズされます。 -この形式のシリアライゼーションからパスのサブカラムを読み取るために、ClickHouse は `Map` カラム全体を読み込み、 -メモリ上で要求されたパスを抽出します。 - -このシリアライゼーションはデータの書き込みや `JSON` カラム全体の読み取りには効率的ですが、パスのサブカラムの読み取りには効率的ではありません。 - -#### Map with buckets {#shared-data-map-with-buckets} - -`map_with_buckets` シリアライゼーションバージョンでは、共有データは `Map(String, String)` 型の `N` 個のカラム(「バケット」)としてシリアライズされます。 -各バケットにはパスのサブセットのみが含まれます。この形式のシリアライゼーションからパスのサブカラムを読み取るために、ClickHouse は -単一のバケットから `Map` カラム全体を読み込み、メモリ上で要求されたパスを抽出します。 - -このシリアライゼーションはデータの書き込みや `JSON` カラム全体の読み取りについては効率が低くなりますが、必要なバケットからのみデータを読み取るため、 -パスのサブカラムの読み取りにはより効率的です。 - -バケット数 `N` は、MergeTree 設定 [object_shared_data_buckets_for_compact_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_compact_part)(デフォルト 8) -および [object_shared_data_buckets_for_wide_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_wide_part)(デフォルト 32)によって制御されます。 - -#### Advanced {#shared-data-advanced} - -`advanced` シリアライゼーションバージョンでは、共有データはパスのサブカラムの読み取り性能を最大化するための特殊なデータ構造としてシリアライズされます。このデータ構造では、要求されたパスのデータのみを読み取れるようにする追加情報を保持します。 -このシリアライゼーションもバケットをサポートしており、各バケットにはパスのサブセットのみが含まれます。 - -このシリアライゼーションはデータの書き込みにはかなり非効率的であるため(ゼロレベルパーツでこのシリアライゼーションを使用することは推奨されません)、`JSON` カラム全体の読み取りは `map` シリアライゼーションと比較してわずかに効率が低下しますが、パスのサブカラムの読み取りには非常に効率的です。 - -注記: データ構造内部に追加情報を保存するため、このシリアライゼーションでは `map` および `map_with_buckets` シリアライゼーションと比較してディスクのストレージサイズが大きくなります。 - -新しい共有データシリアライゼーションのより詳細な概要と実装の詳細については、[ブログ記事](https://clickhouse.com/blog/json-data-type-gets-even-better) を参照してください。 - - - -## イントロスペクション関数 {#introspection-functions} - -JSON 列の内容を調査するのに役立つ関数がいくつかあります: - -* [`JSONAllPaths`](../functions/json-functions.md#JSONAllPaths) -* [`JSONAllPathsWithTypes`](../functions/json-functions.md#JSONAllPathsWithTypes) -* [`JSONDynamicPaths`](../functions/json-functions.md#JSONDynamicPaths) -* [`JSONDynamicPathsWithTypes`](../functions/json-functions.md#JSONDynamicPathsWithTypes) -* [`JSONSharedDataPaths`](../functions/json-functions.md#JSONSharedDataPaths) -* [`JSONSharedDataPathsWithTypes`](../functions/json-functions.md#JSONSharedDataPathsWithTypes) -* [`distinctDynamicTypes`](../aggregate-functions/reference/distinctdynamictypes.md) -* [`distinctJSONPaths and distinctJSONPathsAndTypes`](../aggregate-functions/reference/distinctjsonpaths.md) - -**例** - -`2020-01-01` の [GH Archive](https://www.gharchive.org/) データセットの内容を調査してみましょう。 - -```sql title="Query" -SELECT arrayJoin(distinctJSONPaths(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -``` - -```text title="Response" -┌─arrayJoin(distinctJSONPaths(json))─────────────────────────┐ -│ actor.avatar_url │ -│ actor.display_login │ -│ actor.gravatar_id │ -│ actor.id │ -│ actor.login │ -│ actor.url │ -│ created_at │ -│ id │ -│ org.avatar_url │ -│ org.gravatar_id │ -│ org.id │ -│ org.login │ -│ org.url │ -│ payload.action │ -│ payload.before │ -│ payload.comment._links.html.href │ -│ payload.comment._links.pull_request.href │ -│ payload.comment._links.self.href │ -│ payload.comment.author_association │ -│ payload.comment.body │ -│ payload.comment.commit_id │ -│ payload.comment.created_at │ -│ payload.comment.diff_hunk │ -│ payload.comment.html_url │ -│ payload.comment.id │ -│ payload.comment.in_reply_to_id │ -│ payload.comment.issue_url │ -│ payload.comment.line │ -│ payload.comment.node_id │ -│ payload.comment.original_commit_id │ -│ payload.comment.original_position │ -│ payload.comment.path │ -│ payload.comment.position │ -│ payload.comment.pull_request_review_id │ -... -│ payload.release.node_id │ -│ payload.release.prerelease │ -│ payload.release.published_at │ -│ payload.release.tag_name │ -│ payload.release.tarball_url │ -│ payload.release.target_commitish │ -│ payload.release.upload_url │ -│ payload.release.url │ -│ payload.release.zipball_url │ -│ payload.size │ -│ public │ -│ repo.id │ -│ repo.name │ -│ repo.url │ -│ type │ -└─arrayJoin(distinctJSONPaths(json))─────────────────────────┘ -``` - -```sql -SELECT arrayJoin(distinctJSONPathsAndTypes(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -SETTINGS date_time_input_format = 'best_effort' -``` - - -```text -┌─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┐ -│ ('actor.avatar_url',['String']) │ -│ ('actor.display_login',['String']) │ -│ ('actor.gravatar_id',['String']) │ -│ ('actor.id',['Int64']) │ -│ ('actor.login',['String']) │ -│ ('actor.url',['String']) │ -│ ('created_at',['DateTime']) │ -│ ('id',['String']) │ -│ ('org.avatar_url',['String']) │ -│ ('org.gravatar_id',['String']) │ -│ ('org.id',['Int64']) │ -│ ('org.login',['String']) │ -│ ('org.url',['String']) │ -│ ('payload.action',['String']) │ -│ ('payload.before',['String']) │ -│ ('payload.comment._links.html.href',['String']) │ -│ ('payload.comment._links.pull_request.href',['String']) │ -│ ('payload.comment._links.self.href',['String']) │ -│ ('payload.comment.author_association',['String']) │ -│ ('payload.comment.body',['String']) │ -│ ('payload.comment.commit_id',['String']) │ -│ ('payload.comment.created_at',['DateTime']) │ -│ ('payload.comment.diff_hunk',['String']) │ -│ ('payload.comment.html_url',['String']) │ -│ ('payload.comment.id',['Int64']) │ -│ ('payload.comment.in_reply_to_id',['Int64']) │ -│ ('payload.comment.issue_url',['String']) │ -│ ('payload.comment.line',['Int64']) │ -│ ('payload.comment.node_id',['String']) │ -│ ('payload.comment.original_commit_id',['String']) │ -│ ('payload.comment.original_position',['Int64']) │ -│ ('payload.comment.path',['String']) │ -│ ('payload.comment.position',['Int64']) │ -│ ('payload.comment.pull_request_review_id',['Int64']) │ -... -│ ('payload.release.node_id',['String']) │ -│ ('payload.release.prerelease',['Bool']) │ -│ ('payload.release.published_at',['DateTime']) │ -│ ('payload.release.tag_name',['String']) │ -│ ('payload.release.tarball_url',['String']) │ -│ ('payload.release.target_commitish',['String']) │ -│ ('payload.release.upload_url',['String']) │ -│ ('payload.release.url',['String']) │ -│ ('payload.release.zipball_url',['String']) │ -│ ('payload.size',['Int64']) │ -│ ('public',['Bool']) │ -│ ('repo.id',['Int64']) │ -│ ('repo.name',['String']) │ -│ ('repo.url',['String']) │ -│ ('type',['String']) │ -└─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┘ -``` - - -## ALTER MODIFY COLUMN で JSON 型に変更する {#alter-modify-column-to-json-type} - -既存のテーブルに対して `ALTER` を実行し、列の型を新しい `JSON` 型に変更できます。現時点では、`String` 型からの `ALTER` のみがサポートされています。 - -**例** - -```sql title="Query" -CREATE TABLE test (json String) ENGINE=MergeTree ORDER BY tuple(); -INSERT INTO test VALUES ('{"a" : 42}'), ('{"a" : 43, "b" : "Hello"}'), ('{"a" : 44, "b" : [1, 2, 3]}'), ('{"c" : "2020-01-01"}'); -ALTER TABLE test MODIFY COLUMN json JSON; -SELECT json, json.a, json.b, json.c FROM test; -``` - -```text title="Response" -┌─json─────────────────────────┬─json.a─┬─json.b──┬─json.c─────┐ -│ {"a":"42"} │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ {"a":"43","b":"Hello"} │ 43 │ Hello │ ᴺᵁᴸᴸ │ -│ {"a":"44","b":["1","2","3"]} │ 44 │ [1,2,3] │ ᴺᵁᴸᴸ │ -│ {"c":"2020-01-01"} │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ -└──────────────────────────────┴────────┴─────────┴────────────┘ -``` - - -## JSON 型の値の比較 {#comparison-between-values-of-the-json-type} - -JSON オブジェクトは Map 型と同様に比較されます。 - -例: - -```sql title="Query" -CREATE TABLE test (json1 JSON, json2 JSON) ENGINE=Memory; -INSERT INTO test FORMAT JSONEachRow -{"json1" : {}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : [1, 2, 3]}} -{"json1" : {"a" : 42}, "json2" : {"a" : "Hello"}} -{"json1" : {"a" : 42}, "json2" : {"b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42, "b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41, "b" : 42}} - -SELECT json1, json2, json1 < json2, json1 = json2, json1 > json2 FROM test; -``` - -```text title="Response" -┌─json1──────┬─json2───────────────┬─less(json1, json2)─┬─equals(json1, json2)─┬─greater(json1, json2)─┐ -│ {} │ {} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"41"} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"42"} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {"a":["1","2","3"]} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"Hello"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"42","b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"41","b":"42"} │ 0 │ 0 │ 1 │ -└────────────┴─────────────────────┴────────────────────┴──────────────────────┴───────────────────────┘ -``` - -**注:** 2 つのパスに含まれる値のデータ型が異なる場合、それらは `Variant` データ型の[比較ルール](/sql-reference/data-types/variant#comparing-values-of-variant-data)に従って比較されます。 - - -## JSON 型をより効果的に利用するためのヒント {#tips-for-better-usage-of-the-json-type} - -`JSON` カラムを作成してデータを読み込む前に、次の点を検討してください。 - -- データを調査し、可能な限り多くのパスヒントとその型を指定してください。これにより、保存および読み取りがより効率的になります。 -- 必要となるパスと、今後も使用しないパスを検討してください。不要なパスは `SKIP` セクションに、必要に応じて `SKIP REGEXP` セクションに指定します。これによりストレージ効率が向上します。 -- `max_dynamic_paths` パラメータを過度に大きな値に設定しないでください。保存および読み取りが非効率になる可能性があります。 - メモリ、CPU などのシステムパラメータに大きく依存しますが、一般的な目安として、ローカルファイルシステムストレージでは `max_dynamic_paths` を 10 000 を超える値に設定せず、リモートファイルシステムストレージでは 1024 を超える値に設定しないことを推奨します。 - - - -## 関連ドキュメント {#further-reading} - -- [ClickHouse 向けに新しい強力な JSON データ型を構築した方法](https://clickhouse.com/blog/a-new-powerful-json-data-type-for-clickhouse) -- [10 億件 JSON ドキュメントチャレンジ: ClickHouse、MongoDB、Elasticsearch などの比較](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md deleted file mode 100644 index 34a5d30f39e..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse における Nullable データ型修飾子に関するリファレンス' -sidebar_label: 'Nullable(T)' -sidebar_position: 44 -slug: /sql-reference/data-types/nullable -title: 'Nullable(T)' -doc_type: 'reference' ---- - - - -# Nullable(T) {#nullablet} - -`T` で許可されている通常の値に加えて、「値が存在しない」ことを示す特別なマーカー([NULL](../../sql-reference/syntax.md))を保存できます。例えば、`Nullable(Int8)` 型のカラムには `Int8` 型の値を保存でき、値を持たない行には `NULL` が保存されます。 - -`T` としては、複合データ型である [Array](../../sql-reference/data-types/array.md)、[Map](../../sql-reference/data-types/map.md)、[Tuple](../../sql-reference/data-types/tuple.md) のいずれも指定できませんが、複合データ型の要素として `Nullable` 型の値を含めることはできます(例: `Array(Nullable(Int8))`)。 - -`Nullable` 型のフィールドはテーブルのインデックスに含めることはできません。 - -ClickHouse サーバーの設定で別途指定しない限り、任意の `Nullable` 型のデフォルト値は `NULL` です。 - - - -## ストレージ機能 {#storage-features} - -テーブル列に `Nullable` 型の値を格納するために、ClickHouse は値を格納する通常のファイルとは別に、`NULL` マスクを含むファイルを使用します。マスクファイル内のエントリによって、ClickHouse は各テーブル行について、`NULL` と、そのデータ型に対応するデフォルト値とを区別できます。追加のファイルが必要になるため、`Nullable` 列は、同等の通常列と比較して、より多くのストレージ容量を消費します。 - -:::note -`Nullable` の使用は、ほとんど常にパフォーマンスを低下させます。この点を念頭に置いてデータベースを設計してください。 -::: - - - -## NULL の検索 {#finding-null} - -列全体を読み取ることなく、`null` サブカラムを使って列内の `NULL` 値を特定できます。対応する値が `NULL` の場合は `1` を、それ以外の場合は `0` を返します。 - -**例** - -クエリ: - -```sql -CREATE TABLE nullable (`n` Nullable(UInt32)) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO nullable VALUES (1) (NULL) (2) (NULL); - -SELECT n.null FROM nullable; -``` - -結果: - -```text -┌─n.null─┐ -│ 0 │ -│ 1 │ -│ 0 │ -│ 1 │ -└────────┘ -``` - - -## 使用例 {#usage-example} - -```sql -CREATE TABLE t_null(x Int8, y Nullable(Int8)) ENGINE TinyLog -``` - -```sql -INSERT INTO t_null VALUES (1, NULL), (2, 3) -``` - -```sql -SELECT x + y FROM t_null -``` - -```text -┌─plus(x, y)─┐ -│ ᴺᵁᴸᴸ │ -│ 5 │ -└────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md deleted file mode 100644 index 90d04f09a50..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse における QBit データ型のドキュメントです。QBit データ型は、近似ベクトル検索のための細粒度な量子化を可能にします' -keywords: ['qbit', 'data type'] -sidebar_label: 'QBit' -sidebar_position: 64 -slug: /sql-reference/data-types/qbit -title: 'QBit データ型' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - - -`QBit` データ型は、近似検索を高速化するためにベクトルの格納方式を再構成します。各ベクトルの要素をまとめて保存する代わりに、すべてのベクトルにわたって同じビット位置をグループ化して格納します。 -これにより、ベクトルはフル精度のまま保持しつつ、検索時にきめ細かな量子化レベルを選択できます。読み込むビット数を少なくすれば I/O が減って計算が高速になり、多く読めば精度が向上します。量子化によるデータ転送量および計算量削減の高速化メリットを得ながら、必要に応じて元のデータをすべて参照できます。 - -:::note -`QBit` データ型とそれに関連する距離関数は、現在は実験的機能です。 -これらを有効にするには、まず `SET allow_experimental_qbit_type = 1` を実行してください。 -問題が発生した場合は、[ClickHouse repository](https://github.com/clickhouse/clickhouse/issues) に issue を作成してください。 -::: - -`QBit` 型のカラムを宣言するには、次の構文を使用します。 - -```sql -column_name QBit(element_type, dimension) -``` - -* `element_type` – 各ベクトル要素の型。利用可能な型は `BFloat16`、`Float32`、`Float64` です -* `dimension` – 各ベクトル内の要素数。 - - -## QBit の作成 {#creating-qbit} - -テーブルの列を定義する際に `QBit` 型を使用します: - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [1, 2, 3, 4, 5, 6, 7, 8]), (2, [9, 10, 11, 12, 13, 14, 15, 16]); -SELECT vec FROM test ORDER BY id; -``` - -```text -┌─vec──────────────────────┐ -│ [1,2,3,4,5,6,7,8] │ -│ [9,10,11,12,13,14,15,16] │ -└──────────────────────────┘ -``` - - -## QBit サブカラム {#qbit-subcolumns} - -`QBit` は、格納されたベクトルの個々のビットプレーンにアクセスできるサブカラムアクセスパターンを実装しています。各ビット位置には `.N` 構文を使用してアクセスでき、`N` はビット位置を表します。 - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [0, 0, 0, 0, 0, 0, 0, 0]); -INSERT INTO test VALUES (1, [-0, -0, -0, -0, -0, -0, -0, -0]); -SELECT bin(vec.1) FROM test; -``` - -```text -┌─bin(tupleElement(vec, 1))─┐ -│ 00000000 │ -│ 11111111 │ -└───────────────────────────┘ -``` - -アクセス可能なサブカラムの数は要素型に依存します。 - -* `BFloat16`: サブカラム 16 個 (1〜16) -* `Float32`: サブカラム 32 個 (1〜32) -* `Float64`: サブカラム 64 個 (1〜64) - - -## ベクトル検索関数 {#vector-search-functions} - -`QBit` データ型を使用するベクトル類似度検索向けの距離関数は次のとおりです。 - -* [`L2DistanceTransposed`](../functions/distance-functions.md#L2DistanceTransposed) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md deleted file mode 100644 index e6adb8fde60..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: 'SimpleAggregateFunction データ型に関するドキュメント' -sidebar_label: 'SimpleAggregateFunction' -sidebar_position: 48 -slug: /sql-reference/data-types/simpleaggregatefunction -title: 'SimpleAggregateFunction 型' -doc_type: 'reference' ---- - - - -# SimpleAggregateFunction 型 {#simpleaggregatefunction-type} - - - -## 説明 {#description} - -`SimpleAggregateFunction` データ型は、[`AggregateFunction`](../../sql-reference/data-types/aggregatefunction.md) 型が保持するような集約関数の完全な状態ではなく、集約関数の中間状態のみを格納します。 - -この最適化は、次の性質を満たす関数に適用できます。 - -> 行集合 `S1 UNION ALL S2` に関数 `f` を適用した結果が、行集合の各部分に個別に `f` を適用し、その結果に対して再度 `f` を適用することで得られる場合: -> `f(S1 UNION ALL S2) = f(f(S1) UNION ALL f(S2))`。 - -この性質により、結合された集約結果を計算するには部分集約結果だけで十分であり、余分なデータを保存および処理する必要がないことが保証されます。たとえば、`min` や `max` 関数の結果は、中間ステップから最終結果を計算するために追加の処理を必要としません。一方、`avg` 関数では、最終的に中間状態を結合する `Merge` ステップで平均値を求めるために、合計値と件数を保持しておく必要があります。これら 2 つを割り算することで平均値を求めます。 - -集約関数の値は通常、関数名に [`-SimpleState`](/sql-reference/aggregate-functions/combinators#-simplestate) コンビネータを付与した集約関数を呼び出すことで生成されます。 - - - -## 構文 {#syntax} - -```sql -SimpleAggregateFunction(aggregate_function_name, types_of_arguments...) -``` - -**パラメータ** - -* `aggregate_function_name` - 集約関数の名前。 -* `Type` - 集約関数の引数の型。 - - -## サポートされている関数 {#supported-functions} - -次の集約関数がサポートされています。 - -- [`any`](/sql-reference/aggregate-functions/reference/any) -- [`any_respect_nulls`](/sql-reference/aggregate-functions/reference/any) -- [`anyLast`](/sql-reference/aggregate-functions/reference/anylast) -- [`anyLast_respect_nulls`](/sql-reference/aggregate-functions/reference/anylast) -- [`min`](/sql-reference/aggregate-functions/reference/min) -- [`max`](/sql-reference/aggregate-functions/reference/max) -- [`sum`](/sql-reference/aggregate-functions/reference/sum) -- [`sumWithOverflow`](/sql-reference/aggregate-functions/reference/sumwithoverflow) -- [`groupBitAnd`](/sql-reference/aggregate-functions/reference/groupbitand) -- [`groupBitOr`](/sql-reference/aggregate-functions/reference/groupbitor) -- [`groupBitXor`](/sql-reference/aggregate-functions/reference/groupbitxor) -- [`groupArrayArray`](/sql-reference/aggregate-functions/reference/grouparray) -- [`groupUniqArrayArray`](../../sql-reference/aggregate-functions/reference/groupuniqarray.md) -- [`groupUniqArrayArrayMap`](../../sql-reference/aggregate-functions/combinators#-map) -- [`sumMap`](/sql-reference/aggregate-functions/reference/summap) -- [`minMap`](/sql-reference/aggregate-functions/reference/minmap) -- [`maxMap`](/sql-reference/aggregate-functions/reference/maxmap) - -:::note -`SimpleAggregateFunction(func, Type)` の値は、すべて同じ `Type` を持ちます。 -そのため、`AggregateFunction` 型とは異なり、`-Merge` / `-State` コンビネータを適用する必要はありません。 - -`SimpleAggregateFunction` 型は、同じ集約関数に対して `AggregateFunction` 型よりも高いパフォーマンスを発揮します。 -::: - - - -## 例 {#example} - - - -```sql -CREATE TABLE simple (id UInt64, val SimpleAggregateFunction(sum, Double)) ENGINE=AggregatingMergeTree ORDER BY id; -``` - -## 関連コンテンツ {#related-content} - -* ブログ: [ClickHouse で集約関数コンビネータを使用する](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) - ブログ: [ClickHouse で集約関数コンビネータを使用する](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -* [AggregateFunction](/sql-reference/data-types/aggregatefunction) 型。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md deleted file mode 100644 index 57eec54961d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Expression 特殊データ型に関するリファレンス' -sidebar_label: 'Expression' -sidebar_position: 58 -slug: /sql-reference/data-types/special-data-types/expression -title: 'Expression' -doc_type: 'reference' ---- - -# 式 {#expression} - -式は高階関数においてラムダを表すために用いられます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md deleted file mode 100644 index 3e86a6788bd..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'クエリ実行中の中間結果を表現するために使用される ClickHouse の特殊なデータ型の概要' -sidebar_label: '特殊データ型' -sidebar_position: 55 -slug: /sql-reference/data-types/special-data-types/ -title: '特殊データ型' -doc_type: 'reference' ---- - -# 特殊なデータ型 {#special-data-types} - -特殊なデータ型の値は、シリアライズしてテーブルに保存したり、クエリ結果として出力したりすることはできませんが、クエリ実行中の中間結果として使用できます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md deleted file mode 100644 index 92d11273fb1..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: 'Interval 特殊データ型に関するドキュメント' -sidebar_label: 'Interval' -sidebar_position: 61 -slug: /sql-reference/data-types/special-data-types/interval -title: 'Interval' -doc_type: 'reference' ---- - - - -# Interval {#interval} - -日時の間隔を表すデータ型のファミリーです。[INTERVAL](/sql-reference/operators#interval) 演算子の結果として得られる型です。 - -構造: - -* 符号なし整数値として表される時間間隔。 -* 間隔の型。 - -サポートされている間隔の種類: - -* `NANOSECOND` -* `MICROSECOND` -* `MILLISECOND` -* `SECOND` -* `MINUTE` -* `HOUR` -* `DAY` -* `WEEK` -* `MONTH` -* `QUARTER` -* `YEAR` - -各間隔の種類ごとに、個別のデータ型が定義されています。たとえば、`DAY` 間隔は `IntervalDay` データ型に対応します。 - -```sql -SELECT toTypeName(INTERVAL 4 DAY) -``` - -```text -┌─toTypeName(toIntervalDay(4))─┐ -│ IntervalDay │ -└──────────────────────────────┘ -``` - - -## 使用上の注意 {#usage-remarks} - -`Interval` 型の値は、[Date](../../../sql-reference/data-types/date.md) 型および [DateTime](../../../sql-reference/data-types/datetime.md) 型の値との算術演算に使用できます。たとえば、現在時刻に 4 日を足すことができます。 - -```sql -SELECT now() AS current_date_time, current_date_time + INTERVAL 4 DAY -``` - -```text -┌───current_date_time─┬─plus(now(), toIntervalDay(4))─┐ -│ 2019-10-23 10:58:45 │ 2019-10-27 10:58:45 │ -└─────────────────────┴───────────────────────────────┘ -``` - -また、複数のインターバルを同時に指定することもできます。 - -```sql -SELECT now() AS current_date_time, current_date_time + (INTERVAL 4 DAY + INTERVAL 3 HOUR) -``` - -```text -┌───current_date_time─┬─plus(current_date_time, plus(toIntervalDay(4), toIntervalHour(3)))─┐ -│ 2024-08-08 18:31:39 │ 2024-08-12 21:31:39 │ -└─────────────────────┴────────────────────────────────────────────────────────────────────┘ -``` - -また、異なる間隔の値を比較するには、次のようにします: - -```sql -SELECT toIntervalMicrosecond(3600000000) = toIntervalHour(1); -``` - -```text -┌─less(toIntervalMicrosecond(179999999), toIntervalMinute(3))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────────┘ -``` - - -## 関連項目 {#see-also} - -- [INTERVAL](/sql-reference/operators#interval) 演算子 -- [toInterval](/sql-reference/functions/type-conversion-functions#tointervalyear) 型変換関数 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md deleted file mode 100644 index e5cec5d41a6..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -description: 'Nothing 特殊データ型に関するドキュメント' -sidebar_label: 'Nothing' -sidebar_position: 60 -slug: /sql-reference/data-types/special-data-types/nothing -title: 'Nothing' -doc_type: 'reference' ---- - -# Nothing {#nothing} - -このデータ型の唯一の目的は、値が存在することが想定されていないケースを表すことです。そのため、`Nothing` 型の値を作成することはできません。 - -たとえば、リテラル [NULL](/sql-reference/syntax#null) は `Nullable(Nothing)` 型を持ちます。詳細は [Nullable](../../../sql-reference/data-types/nullable.md) を参照してください。 - -`Nothing` 型は空の配列を表すためにも使用することができます。 - -```sql -SELECT toTypeName(array()) -``` - -```text -┌─toTypeName(array())─┐ -│ Array(Nothing) │ -└─────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md deleted file mode 100644 index 9b5dbee8528..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'IN 式で使用される Set 特殊データ型に関するドキュメント' -sidebar_label: 'Set' -sidebar_position: 59 -slug: /sql-reference/data-types/special-data-types/set -title: 'Set' -doc_type: 'reference' ---- - -# Set {#set} - -[IN](/sql-reference/operators/in) 式の右側で使用されます。 \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md deleted file mode 100644 index 0e74930de7a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'ClickHouse の String データ型に関するドキュメント' -sidebar_label: 'String' -sidebar_position: 8 -slug: /sql-reference/data-types/string -title: 'String' -doc_type: 'reference' ---- - - - -# String {#string} - -任意の長さの文字列型です。長さに制限はありません。値には、ヌルバイトを含む任意のバイト列を格納できます。 -String 型は、他の DBMS の VARCHAR、BLOB、CLOB などの型を置き換えるものです。 - -テーブルを作成する際、文字列フィールドに対して数値パラメータ(例: `VARCHAR(255)`)を指定できますが、ClickHouse はこれらを無視します。 - -エイリアス: - -- `String` — `LONGTEXT`, `MEDIUMTEXT`, `TINYTEXT`, `TEXT`, `LONGBLOB`, `MEDIUMBLOB`, `TINYBLOB`, `BLOB`, `VARCHAR`, `CHAR`, `CHAR LARGE OBJECT`, `CHAR VARYING`, `CHARACTER LARGE OBJECT`, `CHARACTER VARYING`, `NCHAR LARGE OBJECT`, `NCHAR VARYING`, `NATIONAL CHARACTER LARGE OBJECT`, `NATIONAL CHARACTER VARYING`, `NATIONAL CHAR VARYING`, `NATIONAL CHARACTER`, `NATIONAL CHAR`, `BINARY LARGE OBJECT`, `BINARY VARYING`, - - - -## エンコーディング {#encodings} - -ClickHouse にはエンコーディングという概念がありません。文字列は任意のバイト列を含むことができ、それらはそのままの形で保存および出力されます。 -テキストを保存する必要がある場合は、UTF-8 エンコーディングの使用を推奨します。少なくとも、端末が(推奨どおり)UTF-8 を使用している場合は、値を変換することなく読み書きできます。 -同様に、文字列を扱う一部の関数には、その文字列が UTF-8 でエンコードされたテキストを表すバイト列であることを前提として動作する別バージョンがあります。 -たとえば、[length](/sql-reference/functions/array-functions#length) 関数は文字列の長さをバイト数で計算し、[lengthUTF8](../functions/string-functions.md#lengthUTF8) 関数は値が UTF-8 でエンコードされていると仮定して、文字列の長さを Unicode コードポイント数で計算します。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md deleted file mode 100644 index f2d76d24e4f..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: '秒精度で時刻範囲を保存する ClickHouse の Time データ型に関するドキュメント' -slug: /sql-reference/data-types/time -sidebar_position: 15 -sidebar_label: 'Time' -title: 'Time' -doc_type: 'reference' ---- - - - -# Time {#time} - -データ型 `Time` は、時・分・秒の要素から成る時刻を表します。 -これはカレンダーの日付とは独立しており、日・月・年の要素を必要としない値に適しています。 - -構文: - -```sql -時間 -``` - -テキスト表現可能な範囲: [-999:59:59, 999:59:59]。 - -精度: 1秒。 - - -## 実装の詳細 {#implementation-details} - -**表現とパフォーマンス** -データ型 `Time` は内部的に、秒数を符号付き 32 ビット整数として保持します。 -`Time` 型と `DateTime` 型の値は同じバイトサイズであり、そのためパフォーマンスも同程度です。 - -**正規化** -文字列を `Time` に解析する際、時刻成分は正規化されますが、妥当性検証は行われません。 -たとえば、`25:70:70` は `26:11:10` として解釈されます。 - -**負の値** -先頭のマイナス記号はサポートされ、そのまま保持されます。 -負の値は通常、`Time` 値に対する算術演算から生じます。 -`Time` 型では、負の入力はテキスト入力(例: `'-01:02:03'`)および数値入力(例: `-3723`)の両方で保持されます。 - -**サチュレーション(飽和)** -時刻の成分は [-999:59:59, 999:59:59] の範囲に制限されます。 -時間が 999 を超える(または -999 未満の)値は、テキストでの表現および往復変換の際には `999:59:59`(または `-999:59:59`)として扱われます。 - -**タイムゾーン** -`Time` はタイムゾーンをサポートしません。すなわち、`Time` 値は地域的な文脈なしに解釈されます。 -型パラメータとして、または値の作成時に `Time` に対してタイムゾーンを指定するとエラーが発生します。 -同様に、`Time` 列に対してタイムゾーンを適用したり変更しようとする操作はサポートされず、エラーとなります。 -`Time` 値が異なるタイムゾーンの下で暗黙的に再解釈されることはありません。 - - - -## 例 {#examples} - -**1.** `Time` 型の列を持つテーブルを作成し、そのテーブルにデータを挿入する: - -```sql -CREATE TABLE tab -( - `event_id` UInt8, - `time` Time -) -ENGINE = TinyLog; -``` - -```sql --- 時刻を解析 --- - 文字列から --- - 00:00:00 からの経過秒数と解釈される整数から -INSERT INTO tab VALUES (1, '14:30:25'), (2, 52225); - -SELECT * FROM tab ORDER BY event_id; -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**2.** `Time` 値によるフィルタリング - -```sql -SELECT * FROM tab WHERE time = toTime('14:30:25') -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -`Time` 列の値は、`WHERE` 述語で文字列値を使ってフィルタできます。文字列値は自動的に `Time` 型に変換されます。 - -```sql -SELECT * FROM tab WHERE time = '14:30:25' -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**3.** 結果の型を確認する: - -```sql -SELECT CAST('14:30:25' AS Time) AS column, toTypeName(column) AS type -``` - -```text - ┌────列────┬─型──┐ -1. │ 14:30:25 │ Time │ - └───────────┴──────┘ -``` - - -## 関連項目 {#see-also} - -- [型変換関数](../functions/type-conversion-functions.md) -- [日付と時刻を扱う関数](../functions/date-time-functions.md) -- [配列を扱う関数](../functions/array-functions.md) -- [`date_time_input_format` 設定](../../operations/settings/settings-formats.md#date_time_input_format) -- [`date_time_output_format` 設定](../../operations/settings/settings-formats.md#date_time_output_format) -- [`timezone` サーバー構成パラメータ](../../operations/server-configuration-parameters/settings.md#timezone) -- [`session_timezone` 設定](../../operations/settings/settings.md#session_timezone) -- [`DateTime` データ型](datetime.md) -- [`Date` データ型](date.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md deleted file mode 100644 index 40013f54d0a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'サブ秒精度の時間範囲を保持する ClickHouse の Time64 データ型に関するドキュメント' -slug: /sql-reference/data-types/time64 -sidebar_position: 17 -sidebar_label: 'Time64' -title: 'Time64' -doc_type: 'reference' ---- - - - -# Time64 {#time64} - -データ型 `Time64` は、小数秒を含む一日の時刻を表します。 -日・月・年といった暦の要素は持ちません。 -`precision` パラメータは小数部の桁数、すなわちティックサイズを定義します。 - -ティックサイズ(精度): 10-precision 秒。有効範囲: 0..9。よく使われる値は 3(ミリ秒)、6(マイクロ秒)、9(ナノ秒)です。 - -**構文:** - -```sql -Time64(精度) -``` - -内部的には、`Time64` は符号付き 64 ビットの 10 進数 (Decimal64) で秒の小数部分を保持します。 -刻み幅は `precision` パラメータによって決まります。 -タイムゾーンはサポートされていません。`Time64` にタイムゾーンを指定するとエラーが発生します。 - -`DateTime64` と異なり、`Time64` は日付成分を保持しません。 -[`Time`](../../sql-reference/data-types/time.md) も参照してください。 - -テキスト表現の範囲: `precision = 3` の場合は [-999:59:59.000, 999:59:59.999] です。一般に、最小値は `-999:59:59`、最大値は `999:59:59` で、`precision` で指定された桁数までの小数を持つことができます (`precision = 9` の場合、最小値は `-999:59:59.999999999` です)。 - - -## 実装の詳細 {#implementation-details} - -**表現**。 -小数点以下 `precision` 桁で表される小数秒をカウントする符号付き `Decimal64` 値。 - -**正規化**。 -文字列を `Time64` にパースする際、時刻コンポーネントは正規化されますが、検証は行われません。 -たとえば、`25:70:70` は `26:11:10` と解釈されます。 - -**負の値**。 -先頭のマイナス記号はサポートされ、そのまま保持されます。 -負の値は通常、`Time64` 値に対する算術演算から生じます。 -`Time64` では、テキスト入力(例: `'-01:02:03.123'`)および数値入力(例: `-3723.123`)のいずれの場合も、負の入力はそのまま保持されます。 - -**飽和**。 -時刻(time-of-day)コンポーネントは、コンポーネントへの変換やテキストへのシリアル化時に [-999:59:59.xxx, 999:59:59.xxx] の範囲に制限されます。 -保存されている数値自体はこの範囲を超える場合がありますが、コンポーネントの抽出(時、分、秒)およびテキスト表現では、飽和させた値が使用されます。 - -**タイムゾーン**。 -`Time64` はタイムゾーンをサポートしません。 -`Time64` 型または値を作成する際にタイムゾーンを指定するとエラーが発生します。 -同様に、`Time64` カラムに対してタイムゾーンを適用したり変更したりする試みもサポートされず、エラーになります。 - - - -## 例 {#examples} - -1. `Time64` 型のカラムを持つテーブルを作成し、データを挿入する: - -```sql -CREATE TABLE tab64 -( - `event_id` UInt8, - `time` Time64(3) -) -ENGINE = TinyLog; -``` - -```sql --- Time64を解析 --- - 文字列から --- - 00:00:00からの秒数(小数部分は精度に従う)から -INSERT INTO tab64 VALUES (1, '14:30:25'), (2, 52225.123), (3, '14:30:25'); - -SELECT * FROM tab64 ORDER BY event_id; -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 2 │ 14:30:25.123 │ -3. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -2. `Time64` 値によるフィルタリング - -```sql -SELECT * FROM tab64 WHERE time = toTime64('14:30:25', 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -```sql -SELECT * FROM tab64 WHERE time = toTime64(52225.123, 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 2 │ 14:30:25.123 │ - └──────────┴──────────────┘ -``` - -Note: `toTime64` は、指定された精度に従って数値リテラルを小数部付きの秒として解釈するため、意図する小数桁数を明示的に指定してください。 - -3. 結果の型を確認する: - -```sql -SELECT CAST('14:30:25.250' AS Time64(3)) AS column, toTypeName(column) AS type; -``` - -```text - ┌────────column─┬─type──────┐ -1. │ 14:30:25.250 │ Time64(3) │ - └───────────────┴───────────┘ -``` - -**関連項目** - -* [型変換関数](../../sql-reference/functions/type-conversion-functions.md) -* [日付と時刻を扱う関数](../../sql-reference/functions/date-time-functions.md) -* [`date_time_input_format` 設定](../../operations/settings/settings-formats.md#date_time_input_format) -* [`date_time_output_format` 設定](../../operations/settings/settings-formats.md#date_time_output_format) -* [`timezone` サーバー構成パラメータ](../../operations/server-configuration-parameters/settings.md#timezone) -* [`session_timezone` 設定](../../operations/settings/settings.md#session_timezone) -* [日付と時刻を扱うための演算子](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [`Date` データ型](../../sql-reference/data-types/date.md) -* [`Time` データ型](../../sql-reference/data-types/time.md) -* [`DateTime` データ型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md deleted file mode 100644 index a6b2f31030d..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -description: 'ClickHouse の Tuple データ型に関するドキュメント' -sidebar_label: 'Tuple(T1, T2, ...)' -sidebar_position: 34 -slug: /sql-reference/data-types/tuple -title: 'Tuple(T1, T2, ...)' -doc_type: 'reference' ---- - -# Tuple(T1, T2, ...) {#tuplet1-t2} - -それぞれが個別の[型](/sql-reference/data-types)を持つ要素からなるタプルです。Tuple には少なくとも 1 つの要素が含まれている必要があります。 - -タプルは一時的な列のグループ化に使用されます。クエリで `IN` 式を使用する場合や、ラムダ関数の特定の仮引数を指定する場合に、列をグループ化できます。詳細は、[IN 演算子](../../sql-reference/operators/in.md) と [高階関数](/sql-reference/functions/overview#higher-order-functions) のセクションを参照してください。 - -タプルはクエリ結果として返されることがあります。この場合、JSON 以外のテキスト形式では、値は丸かっこ内でカンマ区切りになります。JSON 形式では、タプルは配列(角かっこ内)として出力されます。 - -## タプルの作成 {#creating-tuples} - -関数を使用してタプルを作成できます。 - -```sql -tuple(T1, T2, ...) -``` - -タプルの作成例: - -```sql -SELECT tuple(1, 'a') AS x, toTypeName(x) -``` - -```text -┌─x───────┬─toTypeName(tuple(1, 'a'))─┐ -│ (1,'a') │ Tuple(UInt8, String) │ -└─────────┴───────────────────────────┘ -``` - -タプルは 1 つの要素だけを含むこともできます - -例: - -```sql -SELECT tuple('a') AS x; -``` - -```text -┌─x─────┐ -│ ('a') │ -└───────┘ -``` - -構文 `(tuple_element1, tuple_element2)` を使うと、`tuple()` 関数を呼び出さずに複数の要素から成るタプルを作成できます。 - -例: - -```sql -SELECT (1, 'a') AS x, (today(), rand(), 'someString') AS y, ('a') AS not_a_tuple; -``` - -```text -┌─x───────┬─y──────────────────────────────────────┬─not_a_tuple─┐ -│ (1,'a') │ ('2022-09-21',2006973416,'someString') │ a │ -└─────────┴────────────────────────────────────────┴─────────────┘ -``` - -## データ型の自動判定 {#data-type-detection} - -タプルをその場で作成する場合、ClickHouse はタプルの引数の値を保持できる最小の型として、その引数の型を推論します。値が [NULL](/operations/settings/formats#input_format_null_as_default) の場合、推論される型は [Nullable](../../sql-reference/data-types/nullable.md) になります。 - -自動的なデータ型判定の例: - -```sql -SELECT tuple(1, NULL) AS x, toTypeName(x) -``` - -```text -┌─x─────────┬─toTypeName(tuple(1, NULL))──────┐ -│ (1, NULL) │ Tuple(UInt8, Nullable(Nothing)) │ -└───────────┴─────────────────────────────────┘ -``` - -## タプル要素の参照 {#referring-to-tuple-elements} - -タプル要素は名前またはインデックスで参照できます。 - -```sql -CREATE TABLE named_tuples (`a` Tuple(s String, i Int64)) ENGINE = Memory; -INSERT INTO named_tuples VALUES (('y', 10)), (('x',-10)); - -SELECT a.s FROM named_tuples; -- 名前による参照 -SELECT a.2 FROM named_tuples; -- インデックスによる参照 -``` - -結果: - -```text -┌─a.s─┐ -│ y │ -│ x │ -└─────┘ - -┌─tupleElement(a, 2)─┐ -│ 10 │ -│ -10 │ -└────────────────────┘ -``` - -## Tuple による比較演算 {#comparison-operations-with-tuple} - -2 つのタプルは、左から右へ順に要素を比較していきます。最初のタプルの要素が 2 番目のタプルの対応する要素より大きい(または小さい)場合、最初のタプルは 2 番目のタプルより大きい(または小さい)とみなされます。そうでない場合(両方の要素が等しい場合)は、次の要素を比較します。 - -例: - -```sql -SELECT (1, 'z') > (1, 'a') c1, (2022, 01, 02) > (2023, 04, 02) c2, (1,2,3) = (3,2,1) c3; -``` - -```text -┌─c1─┬─c2─┬─c3─┐ -│ 1 │ 0 │ 0 │ -└────┴────┴────┘ -``` - -実際の使用例: - -```sql -CREATE TABLE test -( - `year` Int16, - `month` Int8, - `day` Int8 -) -ENGINE = Memory AS -SELECT * -FROM values((2022, 12, 31), (2000, 1, 1)); - -SELECT * FROM test; - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -│ 2000 │ 1 │ 1 │ -└──────┴───────┴─────┘ - -SELECT * -FROM test -WHERE (year, month, day) > (2010, 1, 1); - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -└──────┴───────┴─────┘ -CREATE TABLE test -( - `key` Int64, - `duration` UInt32, - `value` Float64 -) -ENGINE = Memory AS -SELECT * -FROM values((1, 42, 66.5), (1, 42, 70), (2, 1, 10), (2, 2, 0)); - -SELECT * FROM test; - -┌─key─┬─duration─┬─value─┐ -│ 1 │ 42 │ 66.5 │ -│ 1 │ 42 │ 70 │ -│ 2 │ 1 │ 10 │ -│ 2 │ 2 │ 0 │ -└─────┴──────────┴───────┘ - --- 各キーについて最大のdurationを持つvalueを検索します。durationが同じ場合は最大のvalueを選択します - -SELECT - key, - max(duration), - argMax(value, (duration, value)) -FROM test -GROUP BY key -ORDER BY key ASC; - -┌─key─┬─max(duration)─┬─argMax(value, tuple(duration, value))─┐ -│ 1 │ 42 │ 70 │ -│ 2 │ 2 │ 0 │ -└─────┴───────────────┴───────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md deleted file mode 100644 index 02a213eab38..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'ClickHouse における UUID データ型に関するドキュメント' -sidebar_label: 'UUID' -sidebar_position: 24 -slug: /sql-reference/data-types/uuid -title: 'UUID' -doc_type: 'reference' ---- - - - -# UUID {#uuid} - -Universally Unique Identifier (UUID、汎用一意識別子) は、レコードを識別するために使用される 16 バイトの値です。UUID の詳細については、[Wikipedia](https://en.wikipedia.org/wiki/Universally_unique_identifier) を参照してください。 - -異なる UUID バリアントが存在しますが([こちら](https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis) を参照)、ClickHouse は挿入された UUID が特定のバリアントに準拠しているかどうかを検証しません。 -UUID は内部的には 16 バイトのランダムなバイト列として扱われ、SQL レベルでは [8-4-4-4-12 表記](https://en.wikipedia.org/wiki/Universally_unique_identifier#Textual_representation) で表現されます。 - -UUID 値の例: - -```text -61f0c404-5cb3-11e7-907b-a6006ad3dba0 -``` - -デフォルトの UUID はすべて 0 です。これは、たとえば新しいレコードを挿入する際に、UUID 列の値が指定されていない場合などに使用されます。 - -```text -00000000-0000-0000-0000-000000000000 -``` - -歴史的経緯により、UUID は後半部分によってソートされます。 -したがって、UUID をテーブルの主キー、ソートキー、またはパーティションキーとして直接使用すべきではありません。 - -例: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY uuid; -``` - -結果: - -```text -┌─uuid─────────────────────────────────┐ -│ 36a0b67c-b74a-4640-803b-e44bb4547e3c │ -│ 3a00aeb8-2605-4eec-8215-08c0ecb51112 │ -│ 3fda7c49-282e-421a-85ab-c5684ef1d350 │ -│ 16ab55a7-45f6-44a8-873c-7a0b44346b3e │ -│ e3776711-6359-4f22-878d-bf290d052c85 │ -│ [...] │ -│ 9eceda2f-6946-40e3-b725-16f2709ca41a │ -│ 03644f74-47ba-4020-b865-be5fd4c8c7ff │ -│ ce3bc93d-ab19-4c74-b8cc-737cb9212099 │ -│ b7ad6c91-23d6-4b5e-b8e4-a52297490b56 │ -│ 06892f64-cc2d-45f3-bf86-f5c5af5768a9 │ -└──────────────────────────────────────┘ -``` - -回避策として、UUID を直感的に分かりやすい並び順を持つ型に変換できます。 - -UInt128 に変換する例: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY toUInt128(uuid); -``` - -結果: - -```sql -┌─uuid─────────────────────────────────┐ -│ 018b81cd-aca1-4e9c-9e56-a84a074dc1a8 │ -│ 02380033-c96a-438e-913f-a2c67e341def │ -│ 057cf435-7044-456a-893b-9183a4475cea │ -│ 0a3c1d4c-f57d-44cc-8567-60cb0c46f76e │ -│ 0c15bf1c-8633-4414-a084-7017eead9e41 │ -│ [...] │ -│ f808cf05-ea57-4e81-8add-29a195bde63d │ -│ f859fb5d-764b-4a33-81e6-9e4239dae083 │ -│ fb1b7e37-ab7b-421a-910b-80e60e2bf9eb │ -│ fc3174ff-517b-49b5-bfe2-9b369a5c506d │ -│ fece9bf6-3832-449a-b058-cd1d70a02c8b │ -└──────────────────────────────────────┘ -``` - - -## UUID の生成 {#generating-uuids} - -ClickHouse は、ランダムな UUID バージョン 4 の値を生成するための関数 [generateUUIDv4](../../sql-reference/functions/uuid-functions.md) を提供します。 - - - -## 使用例 {#usage-example} - -**例 1** - -この例では、UUID 列を持つテーブルを作成し、そのテーブルに値を挿入する方法を示します。 - -```sql -CREATE TABLE t_uuid (x UUID, y String) ENGINE=TinyLog - -INSERT INTO t_uuid SELECT generateUUIDv4(), '例 1' - -SELECT * FROM t_uuid -``` - -結果: - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ 例1 │ -└──────────────────────────────────────┴───────────┘ -``` - -**例 2** - -この例では、レコード挿入時に UUID 列の値を指定しないため、つまりデフォルトの UUID 値が挿入されます。 - -```sql -INSERT INTO t_uuid (y) VALUES ('例2') - -SELECT * FROM t_uuid -``` - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ 例1 │ -│ 00000000-0000-0000-0000-000000000000 │ 例2 │ -└──────────────────────────────────────┴───────────┘ -``` - - -## 制限事項 {#restrictions} - -`UUID` データ型は、[String](../../sql-reference/data-types/string.md) データ型がサポートする関数のみをサポートします(たとえば [min](/sql-reference/aggregate-functions/reference/min)、[max](/sql-reference/aggregate-functions/reference/max)、[count](/sql-reference/aggregate-functions/reference/count) など)。 - -`UUID` データ型は、[abs](/sql-reference/functions/arithmetic-functions#abs) などの算術演算や、[sum](/sql-reference/aggregate-functions/reference/sum)、[avg](/sql-reference/aggregate-functions/reference/avg) などの集約関数はサポートされません。 diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md deleted file mode 100644 index 628c6586bc2..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md +++ /dev/null @@ -1,490 +0,0 @@ ---- -description: 'ClickHouse の Variant データ型に関するドキュメント' -sidebar_label: 'Variant(T1, T2, ...)' -sidebar_position: 40 -slug: /sql-reference/data-types/variant -title: 'Variant(T1, T2, ...)' -doc_type: 'reference' ---- - -# Variant(T1, T2, ...) {#variantt1-t2} - -この型は、他のデータ型のユニオン(共用体)を表します。型 `Variant(T1, T2, ..., TN)` は、この型の各行が -`T1`、`T2`、…、`TN` のいずれか、またはいずれでもない(`NULL` 値)であることを意味します。 - -ネストされた型の順序は問いません: Variant(T1, T2) = Variant(T2, T1) です。 -ネストされた型には、Nullable(...)、LowCardinality(Nullable(...))、Variant(...) 型以外の任意の型を指定できます。 - -:::note -似たような型をバリアントとして使用すること(たとえば、`Variant(UInt32, Int64)` のような異なる数値型や、`Variant(Date, DateTime)` のような異なる日付型)は推奨されません。 -そのような型の値を扱うと曖昧さを招く可能性があるためです。デフォルトでは、このような `Variant` 型を作成しようとすると例外がスローされますが、設定 `allow_suspicious_variant_types` を使用して有効化できます。 -::: - -## Variant の作成 {#creating-variant} - -テーブル列を定義する際に `Variant` 型を使用する: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v FROM test; -``` - -```text -┌─v─────────────┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ Hello, World! │ -│ [1,2,3] │ -└───────────────┘ -``` - -通常のカラムからの CAST の使用: - -```sql -SELECT toTypeName(variant) AS type_name, 'Hello, World!'::Variant(UInt64, String, Array(UInt64)) as variant; -``` - -```text -┌─type_name──────────────────────────────┬─variant───────┐ -│ Variant(Array(UInt64), String, UInt64) │ Hello, World! │ -└────────────────────────────────────────┴───────────────┘ -``` - -引数に共通の型がない場合に関数 `if` / `multiIf` を使用するには(設定 `use_variant_as_common_type` を有効にしておく必要があります): - -```sql -SET use_variant_as_common_type = 1; -SELECT if(number % 2, number, range(number)) as variant FROM numbers(5); -``` - -```text -┌─variant───┐ -│ [] │ -│ 1 │ -│ [0,1] │ -│ 3 │ -│ [0,1,2,3] │ -└───────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 4) = 0, 42, (number % 4) = 1, [1, 2, 3], (number % 4) = 2, 'Hello, World!', NULL) AS variant FROM numbers(4); -``` - -```text -┌─variant───────┐ -│ 42 │ -│ [1,2,3] │ -│ Hello, World! │ -│ ᴺᵁᴸᴸ │ -└───────────────┘ -``` - -配列要素やマップ値に共通の型がない場合は、関数 'array/map' を使用します(このとき設定 `use_variant_as_common_type` を有効にしておく必要があります): - -```sql -SET use_variant_as_common_type = 1; -SELECT array(range(number), number, 'str_' || toString(number)) as array_of_variants FROM numbers(3); -``` - -```text -┌─array_of_variants─┐ -│ [[],0,'str_0'] │ -│ [[0],1,'str_1'] │ -│ [[0,1],2,'str_2'] │ -└───────────────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT map('a', range(number), 'b', number, 'c', 'str_' || toString(number)) as map_of_variants FROM numbers(3); -``` - -```text -┌─map_of_variants───────────────┐ -│ {'a':[],'b':0,'c':'str_0'} │ -│ {'a':[0],'b':1,'c':'str_1'} │ -│ {'a':[0,1],'b':2,'c':'str_2'} │ -└───────────────────────────────┘ -``` - -## Variant のネストされた型をサブカラムとして読み取る {#reading-variant-nested-types-as-subcolumns} - -`Variant` 型は、`Variant` 列から型名をサブカラムとして指定することで、単一のネストされた型を読み取ることをサポートします。 -そのため、列 `variant Variant(T1, T2, T3)` がある場合、`variant.T2` という構文を使って型 `T2` のサブカラムを読み取ることができます。 -このサブカラムの型は、`T2` が `Nullable` でラップ可能な場合は `Nullable(T2)` になり、そうでない場合は `T2` になります。 -このサブカラムは元の `Variant` 列と同じサイズとなり、元の `Variant` 列に型 `T2` が存在しないすべての行では、`NULL` 値(または `T2` を `Nullable` でラップできない場合は空値)を含みます。 - -Variant のサブカラムは、関数 `variantElement(variant_column, type_name)` を使って読み取ることもできます。 - -例: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v, v.String, v.UInt64, v.`Array(UInt64)` FROM test; -``` - -```text -┌─v─────────────┬─v.String──────┬─v.UInt64─┬─v.Array(UInt64)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴───────────────┴──────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(v.String), toTypeName(v.UInt64), toTypeName(v.`Array(UInt64)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(v.String)─┬─toTypeName(v.UInt64)─┬─toTypeName(v.Array(UInt64))─┐ -│ Nullable(String) │ Nullable(UInt64) │ Array(UInt64) │ -└──────────────────────┴──────────────────────┴─────────────────────────────┘ -``` - -```sql -SELECT v, variantElement(v, 'String'), variantElement(v, 'UInt64'), variantElement(v, 'Array(UInt64)') FROM test; -``` - -```text -┌─v─────────────┬─variantElement(v, 'String')─┬─variantElement(v, 'UInt64')─┬─variantElement(v, 'Array(UInt64)')─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴─────────────────────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - -各行にどのバリアントの種類が格納されているかを確認するには、関数 `variantType(variant_column)` を使用できます。これは各行について、そのバリアント型名を表す `Enum` を返します(行が `NULL` の場合は `'None'` を返します)。 - -例: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT variantType(v) FROM test; -``` - -```text -┌─variantType(v)─┐ -│ None │ -│ UInt64 │ -│ String │ -│ Array(UInt64) │ -└────────────────┘ -``` - -```sql -SELECT toTypeName(variantType(v)) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(variantType(v))──────────────────────────────────────────┐ -│ Enum8('None' = -1, 'Array(UInt64)' = 0, 'String' = 1, 'UInt64' = 2) │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -## Variant 列と他の列との変換 {#conversion-between-a-variant-column-and-other-columns} - -`Variant` 型の列では、4 種類の変換を行うことができます。 - -### String 列を Variant 列に変換する {#converting-a-string-column-to-a-variant-column} - -`String` から `Variant` への変換は、文字列値を解析して `Variant` 型の値を生成することで行われます。 - -```sql -SELECT '42'::Variant(String, UInt64) AS variant, variantType(variant) AS variant_type -``` - -```text -┌─variant─┬─variant_type─┐ -│ 42 │ UInt64 │ -└─────────┴──────────────┘ -``` - -```sql -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴───────────────┘ -``` - -````sql -SELECT CAST(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01'), 'Map(String, Variant(UInt64, Bool, Date))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types``` -```` - -```text -┌─map_of_variants─────────────────────────────┬─map_of_variant_types──────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'UInt64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴───────────────────────────────────────────────┘ -``` - -`String` から `Variant` への変換時のパースを無効にするには、設定項目 `cast_string_to_dynamic_use_inference` を無効にします。 - -```sql -SET cast_string_to_variant_use_inference = 0; -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### 通常のカラムを Variant カラムに変換する {#converting-an-ordinary-column-to-a-variant-column} - -型 `T` を持つ通常のカラムを、その型を含む `Variant` カラムに変換できます。 - -```sql -SELECT toTypeName(variant) AS type_name, [1,2,3]::Array(UInt64)::Variant(UInt64, String, Array(UInt64)) as variant, variantType(variant) as variant_name -``` - -```text -┌─type_name──────────────────────────────┬─variant─┬─variant_name──┐ -│ Variant(Array(UInt64), String, UInt64) │ [1,2,3] │ Array(UInt64) │ -└────────────────────────────────────────┴─────────┴───────────────┘ -``` - -注意: `String` 型からの変換は常にパースを通じて行われます。パースを行わずに `String` 列を `Variant` の `String` バリアントに変換する必要がある場合は、次のようにします: - -```sql -SELECT '[1, 2, 3]'::Variant(String)::Variant(String, Array(UInt64), UInt64) as variant, variantType(variant) as variant_type -``` - -```sql -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### Variant列を通常の列に変換する {#converting-a-variant-column-to-an-ordinary-column} - -`Variant` 列を通常の列に変換できます。この場合、すべてのネストされた Variant は指定した変換先の型に変換されます。 - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'); -SELECT v::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(v, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -└──────────────────────────────┘ -``` - -### ある Variant を別の Variant に変換する {#converting-a-variant-to-another-variant} - -`Variant` 列を別の `Variant` 列に変換することも可能ですが、変換先の `Variant` 列が元の `Variant` 列に含まれるすべてのネストされた型を含んでいる場合に限られます。 - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'); -SELECT v::Variant(UInt64, String, Array(UInt64)) FROM test; -``` - -```text -┌─CAST(v, 'Variant(UInt64, String, Array(UInt64))')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ String │ -└───────────────────────────────────────────────────┘ -``` - -## データからの Variant 型の読み取り {#reading-variant-type-from-the-data} - -すべてのテキスト形式 (TSV、CSV、CustomSeparated、Values、JSONEachRow など) は `Variant` 型での読み取りをサポートしています。データの解析時に、ClickHouse は値を最も適切な Variant 型の要素に挿入しようとします。 - -例: - -```sql -SELECT - v, - variantElement(v, 'String') AS str, - variantElement(v, 'UInt64') AS num, - variantElement(v, 'Float64') AS float, - variantElement(v, 'DateTime') AS date, - variantElement(v, 'Array(UInt64)') AS arr -FROM format(JSONEachRow, 'v Variant(String, UInt64, Float64, DateTime, Array(UInt64))', $$ -{"v" : "Hello, World!"}, -{"v" : 42}, -{"v" : 42.42}, -{"v" : "2020-01-01 00:00:00"}, -{"v" : [1, 2, 3]} -$$) -``` - -```text -┌─v───────────────────┬─str───────────┬──num─┬─float─┬────────────────date─┬─arr─────┐ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 00:00:00 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 00:00:00 │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└─────────────────────┴───────────────┴──────┴───────┴─────────────────────┴─────────┘ -``` - -## Variant 型の値の比較 {#comparing-values-of-variant-data} - -`Variant` 型の値は、同じ `Variant` 型の値とのみ比較できます。 - -型 `Variant(..., T1, ... T2, ...)` に属し、それぞれの基底型が `T1` と `T2` である値 `v1` と `v2` に対する演算子 `<` の結果は、次のように定義されます。 - -* `T1 = T2 = T` の場合、結果は `v1.T < v2.T` になります(基底値どうしが比較されます)。 -* `T1 != T2` の場合、結果は `T1 < T2` になります(型名どうしが比較されます)。 - -例: - -```sql -CREATE TABLE test (v1 Variant(String, UInt64, Array(UInt32)), v2 Variant(String, UInt64, Array(UInt32))) ENGINE=Memory; -INSERT INTO test VALUES (42, 42), (42, 43), (42, 'abc'), (42, [1, 2, 3]), (42, []), (42, NULL); -``` - -```sql -SELECT v2, variantType(v2) AS v2_type FROM test ORDER BY v2; -``` - -```text -┌─v2──────┬─v2_type───────┐ -│ [] │ Array(UInt32) │ -│ [1,2,3] │ Array(UInt32) │ -│ abc │ String │ -│ 42 │ UInt64 │ -│ 43 │ UInt64 │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴───────────────┘ -``` - -```sql -SELECT v1, variantType(v1) AS v1_type, v2, variantType(v2) AS v2_type, v1 = v2, v1 < v2, v1 > v2 FROM test; -``` - -```text -┌─v1─┬─v1_type─┬─v2──────┬─v2_type───────┬─equals(v1, v2)─┬─less(v1, v2)─┬─greater(v1, v2)─┐ -│ 42 │ UInt64 │ 42 │ UInt64 │ 1 │ 0 │ 0 │ -│ 42 │ UInt64 │ 43 │ UInt64 │ 0 │ 1 │ 0 │ -│ 42 │ UInt64 │ abc │ String │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [1,2,3] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ ᴺᵁᴸᴸ │ None │ 0 │ 1 │ 0 │ -└────┴─────────┴─────────┴───────────────┴────────────────┴──────────────┴─────────────────┘ - -``` - -特定の `Variant` 値を持つ行を見つける必要がある場合、次のいずれかの方法を使用できます。 - -* 値を対応する `Variant` 型にキャストする: - -```sql -SELECT * FROM test WHERE v2 == [1,2,3]::Array(UInt32)::Variant(String, UInt64, Array(UInt32)); -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -* `Variant` サブカラムを必要な型と比較します。 - -```sql -SELECT * FROM test WHERE v2.`Array(UInt32)` == [1,2,3] -- もしくは variantElement(v2, 'Array(UInt32)') を使用 -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -`Array/Map/Tuple` のような複合型を持つサブカラムは `Nullable` の中に含めることができず、異なる型を持つ行では `NULL` ではなくデフォルト値になるため、Variant 型に対して追加のチェックを行っておくと便利な場合があります。 - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE v2.`Array(UInt32)` == []; -``` - -```text -┌─v2───┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ 42 │ [] │ UInt64 │ -│ 43 │ [] │ UInt64 │ -│ abc │ [] │ String │ -│ [] │ [] │ Array(UInt32) │ -│ ᴺᵁᴸᴸ │ [] │ None │ -└──────┴──────────────────┴─────────────────┘ -``` - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE variantType(v2) == 'Array(UInt32)' AND v2.`Array(UInt32)` == []; -``` - -```text -┌─v2─┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ [] │ [] │ Array(UInt32) │ -└────┴──────────────────┴─────────────────┘ -``` - -**注:** 数値型が異なる `Variant` の値は、別の `Variant` とみなされ、値同士は比較されません。代わりに、その型名同士が比較されます。 - -例: - -```sql -SET allow_suspicious_variant_types = 1; -CREATE TABLE test (v Variant(UInt32, Int64)) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT v, variantType(v) FROM test ORDER by v; -``` - -```text -┌─v───┬─variantType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -**注記** デフォルトでは `Variant` 型は `GROUP BY`/`ORDER BY` のキーとしては許可されていません。使用したい場合は、その特殊な比較ルールを考慮した上で、`allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by` 設定を有効にしてください。 - -## Variant における JSONExtract 関数 {#jsonextract-functions-with-variant} - -すべての `JSONExtract*` 関数は `Variant` 型をサポートします。 - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Variant(UInt32, String, Array(UInt32))') AS variant, variantType(variant) AS variant_type; -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt32) │ -└─────────┴───────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Variant(UInt32, String, Array(UInt32)))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types -``` - -```text -┌─map_of_variants──────────────────┬─map_of_variant_types────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'UInt32','b':'String','c':'Array(UInt32)'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────┘ -``` - -```sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Variant(UInt32, String, Array(UInt32))') AS variants, arrayMap(x -> (x.1, variantType(x.2)), variants) AS variant_types -``` - -```text -┌─variants───────────────────────────────┬─variant_types─────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','UInt32'),('b','String'),('c','Array(UInt32)')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md deleted file mode 100644 index 3eeca95271a..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md +++ /dev/null @@ -1,4 +0,0 @@ -:::tip -ClickHouse Cloud でディクショナリを使用している場合は、ディクショナリの作成に DDL クエリオプションを使用し、ユーザー `default` でディクショナリを作成してください。 -また、[Cloud 互換性ガイド](/whats-new/cloud-compatibility) でサポートされているディクショナリソースの一覧を確認してください。 -::: \ No newline at end of file diff --git a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md b/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md deleted file mode 100644 index 71bfe16a509..00000000000 --- a/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md +++ /dev/null @@ -1,2529 +0,0 @@ ---- -description: 'ClickHouse における外部辞書機能の概要' -sidebar_label: '辞書の定義' -sidebar_position: 35 -slug: /sql-reference/dictionaries -title: '辞書' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/jp/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; -import CloudDetails from '@site/i18n/jp/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# 辞書 {#dictionaries} - -辞書は、さまざまな種類の参照リストに便利なマッピング(`key -> attributes`)です。 - -ClickHouse は、クエリで使用できる辞書を扱うための特殊な関数をサポートしています。参照テーブルとの `JOIN` を使うよりも、関数と辞書を組み合わせて使用する方が簡単かつ効率的です。 - -ClickHouse は次の機能をサポートしています: - -- [一連の関数](../../sql-reference/functions/ext-dict-functions.md)を備えた辞書。 -- 特定の[一連の関数](../../sql-reference/functions/embedded-dict-functions.md)を持つ[組み込み辞書](#embedded-dictionaries)。 - -:::tip チュートリアル -ClickHouse の辞書の使い方を始める場合は、このトピックを扱ったチュートリアルがあります。[こちら](tutorial.md)をご覧ください。 -::: - -さまざまなデータソースから独自の辞書を追加できます。辞書のソースには、ClickHouse テーブル、ローカルのテキストファイルまたは実行可能ファイル、HTTP(s) リソース、別の DBMS などを使用できます。詳細については、「[Dictionary Sources](#dictionary-sources)」を参照してください。 - -ClickHouse では次のことが可能です: - -- 辞書を RAM に全体または一部保存します。 -- 辞書を定期的に更新し、欠落している値を動的にロードします。言い換えると、辞書は動的にロード可能です。 -- XML ファイルまたは [DDL クエリ](../../sql-reference/statements/create/dictionary.md)で辞書を作成できます。 - -辞書の設定は 1 つ以上の XML ファイルに配置できます。設定ファイルへのパスは、[dictionaries_config](../../operations/server-configuration-parameters/settings.md#dictionaries_config) パラメータで指定します。 - -辞書は、[dictionaries_lazy_load](../../operations/server-configuration-parameters/settings.md#dictionaries_lazy_load) 設定に応じて、サーバー起動時または初回利用時にロードできます。 - -[dictionaries](/operations/system-tables/dictionaries) システムテーブルには、サーバーで構成されている辞書に関する情報が含まれています。各辞書について、次の情報を確認できます: - -- 辞書のステータス。 -- 設定パラメータ。 -- 辞書に割り当てられた RAM の使用量や、辞書が正常にロードされてからのクエリ数などのメトリクス。 - - - -## DDL クエリによる辞書の作成 {#creating-a-dictionary-with-a-ddl-query} - -辞書は [DDL クエリ](../../sql-reference/statements/create/dictionary.md) で作成でき、この方法が推奨されます。DDL で作成された辞書には次の利点があります。 -- サーバーの設定ファイルに追加のレコードを追記する必要がありません。 -- テーブルやビューと同様に、第一級オブジェクトとして辞書を扱うことができます。 -- 辞書テーブル関数ではなく、慣れ親しんだ SELECT を用いてデータを直接読み取ることができます。SELECT 文で辞書に直接アクセスする場合、キャッシュ型の辞書はキャッシュされているデータのみを返し、非キャッシュ型の辞書は格納しているすべてのデータを返す点に注意してください。 -- 辞書は簡単に名前を変更できます。 - -## 設定ファイルによる辞書の作成 {#creating-a-dictionary-with-a-configuration-file} - - - -:::note -設定ファイルによる辞書の作成は ClickHouse Cloud ではサポートされていません。DDL(上記参照)を使用し、ユーザー `default` で辞書を作成してください。 -::: - -辞書の設定ファイルは次の形式です。 - -```xml - - 任意の要素。任意の内容を含めることができます。ClickHouseサーバーによって無視されます。 - - - /etc/metrika.xml - - - - - - - - -``` - -同じファイル内で任意の数の辞書を[設定](#configuring-a-dictionary)できます。 - -:::note -`SELECT` クエリで辞書を記述することで、小さな辞書の値を変換できます([transform](../../sql-reference/functions/other-functions.md) 関数を参照)。この機能は辞書機能とは別物です。 -::: - -## ディクショナリの設定 {#configuring-a-dictionary} - - - -ディクショナリを XML ファイルで構成する場合、ディクショナリ設定は次のような構造になります。 - -```xml - - dict_name - - - - - - - - - - - - - - - - - -``` - -対応する [DDL クエリ](../../sql-reference/statements/create/dictionary.md) の構造は次のとおりです。 - -```sql -CREATE DICTIONARY dict_name -( - ... -- 属性 -) -PRIMARY KEY ... -- 複合キーまたは単一キーの構成 -SOURCE(...) -- ソースの構成 -LAYOUT(...) -- メモリレイアウトの構成 -LIFETIME(...) -- メモリ内のディクショナリのライフタイム -``` - -## メモリ内にディクショナリを保存する {#storing-dictionaries-in-memory} - -メモリ上にディクショナリを保存する方法はいくつかあります。 - -最適な処理速度が得られるため、[flat](#flat)、[hashed](#hashed)、および [complex_key_hashed](#complex_key_hashed) を推奨します。 - -パフォーマンスが低下する可能性があり、最適なパラメータの選定も難しいため、キャッシュ方式は推奨されません。詳細は [cache](#cache) セクションを参照してください。 - -ディクショナリのパフォーマンスを改善する方法はいくつかあります。 - -* `GROUP BY` の後に、ディクショナリを扱う関数を呼び出します。 -* 抽出する属性を単射としてマークします。異なるキーが異なる属性値に対応する場合、その属性は単射と呼ばれます。このため、`GROUP BY` でキーから属性値を取得する関数が使われている場合、この関数は自動的に `GROUP BY` の外に出されます。 - -ClickHouse は、ディクショナリに関連するエラーに対して例外をスローします。代表的なエラーは次のとおりです。 - -* アクセスしようとしているディクショナリをロードできない。 -* `cached` ディクショナリへのクエリでエラーが発生した。 - -[system.dictionaries](../../operations/system-tables/dictionaries.md) テーブルで、ディクショナリの一覧とそのステータスを確認できます。 - - - -設定は次のようになります。 - -```xml - - - ... - - - - - - ... - - -``` - -対応する [DDL クエリ](../../sql-reference/statements/create/dictionary.md): - -```sql -CREATE DICTIONARY (...) -... -LAYOUT(LAYOUT_TYPE(param value)) -- レイアウト設定 -... -``` - -レイアウトで `complex-key*` が指定されていない辞書はキーに [UInt64](../../sql-reference/data-types/int-uint.md) 型を持ち、`complex-key*` 辞書は複合キー(任意の型を組み合わせた複雑なキー)を持ちます。 - -XML 辞書における [UInt64](../../sql-reference/data-types/int-uint.md) キーは `` タグで定義されます。 - -設定例(列 `key_column` の型が UInt64 の場合): - -```xml -... - - - key_column - -... -``` - -複合(`complex`)キーを持つ XML 辞書は、`` タグで定義されます。 - -複合キーの構成例(キーは [String](../../sql-reference/data-types/string.md) 型の要素を 1 つ持ちます): - -```xml -... - - - - country_code - String - - -... -``` - -## メモリ内にディクショナリを格納する方法 {#ways-to-store-dictionaries-in-memory} - -ディクショナリデータをメモリに格納するさまざまな方法には、CPU および RAM 使用量とのトレードオフがあります。どのレイアウトを使用するかを決定するための出発点として、ディクショナリに関する [ブログ記事](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse) の [Choosing a Layout](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout) 節に掲載されている意思決定ツリーが有用です。 - -* [flat](#flat) -* [hashed](#hashed) -* [sparse_hashed](#sparse_hashed) -* [complex_key_hashed](#complex_key_hashed) -* [complex_key_sparse_hashed](#complex_key_sparse_hashed) -* [hashed_array](#hashed_array) -* [complex_key_hashed_array](#complex_key_hashed_array) -* [range_hashed](#range_hashed) -* [complex_key_range_hashed](#complex_key_range_hashed) -* [cache](#cache) -* [complex_key_cache](#complex_key_cache) -* [ssd_cache](#ssd_cache) -* [complex_key_ssd_cache](#complex_key_ssd_cache) -* [direct](#direct) -* [complex_key_direct](#complex_key_direct) -* [ip_trie](#ip_trie) - -### flat {#flat} - -ディクショナリはフラットな配列形式でメモリ内に完全に格納されます。ディクショナリはどの程度のメモリを使用するでしょうか。使用量は(メモリ空間上での)最大キーの大きさに比例します。 - -ディクショナリキーは [UInt64](../../sql-reference/data-types/int-uint.md) 型であり、値(配列の要素数)は `max_array_size`(デフォルトでは 500,000)に制限されます。ディクショナリを作成する際に、これより大きなキーが検出された場合、ClickHouse は例外をスローし、そのディクショナリは作成されません。ディクショナリのフラット配列の初期サイズは、`initial_array_size` 設定(デフォルトでは 1024)によって制御されます。 - -すべての種類のソースがサポートされています。更新時には、データ(ファイルまたはテーブルから)はすべて読み込まれます。 - -この方法は、利用可能なディクショナリ格納方法の中で最も高いパフォーマンスを提供します。 - -設定例: - -```xml - - - 50000 - 5000000 - - -``` - -または - -```sql -LAYOUT(FLAT(INITIAL_ARRAY_SIZE 50000 MAX_ARRAY_SIZE 5000000)) -``` - -### hashed {#hashed} - -辞書はハッシュテーブル形式で、すべてメモリ上に格納されます。辞書には任意の識別子を持つ要素をいくつでも含めることができます。実際には、キーの数が数千万件に達することもあります。 - -辞書キーの型は [UInt64](../../sql-reference/data-types/int-uint.md) です。 - -あらゆる種類のソースがサポートされています。更新時には、ファイルまたはテーブルからのデータがすべて読み込まれます。 - -構成例: - -```xml - - - -``` - -または - -```sql -LAYOUT(HASHED()) -``` - -設定例: - -```xml - - - - 10 - - - 10000 - - - 0.5 - - -``` - -または - -```sql -LAYOUT(HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### sparse_hashed {#sparse_hashed} - -`hashed` に似ていますが、より多くの CPU 資源を消費する代わりに、メモリ使用量を抑えます。 - -辞書キーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 - -設定例: - -```xml - - - - - - - -``` - -または - -```sql -LAYOUT(SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -この種の辞書では `shards` を使用することも可能です。`sparse_hashed` は `hashed` よりも遅いため、`shards` の使用は `hashed` の場合よりも `sparse_hashed` の場合のほうが重要です。 - -### complex_key_hashed {#complex_key_hashed} - -このストレージタイプは複合[キー](#dictionary-key-and-fields)用です。`hashed` と同様です。 - -設定例: - -```xml - - - - - - - -``` - -または - -```sql -LAYOUT(COMPLEX_KEY_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### complex_key_sparse_hashed {#complex_key_sparse_hashed} - -このストレージタイプは、複合 [キー](#dictionary-key-and-fields) を持つ辞書で使用します。[sparse_hashed](#sparse_hashed) と類似しています。 - -設定例: - -```xml - - - - - - - -``` - -または - -```sql -LAYOUT(COMPLEX_KEY_SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### hashed_array {#hashed_array} - -辞書は全体がメモリ上に保持されます。各属性は配列として格納されます。キー属性は、値が属性配列内のインデックスとなるハッシュテーブルの形式で保存されます。辞書には、任意の識別子を持つ任意数の要素を含めることができます。実運用では、キーの数が数千万件に達することがあります。 - -辞書キーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 - -あらゆる種類のソースをサポートします。更新時には、(ファイルまたはテーブルからの)データ全体が読み込まれます。 - -設定例: - -```xml - - - - -``` - -または - -```sql -LAYOUT(HASHED_ARRAY([SHARDS 1])) -``` - -### complex_key_hashed_array {#complex_key_hashed_array} - -このストレージタイプは、複合[キー](#dictionary-key-and-fields)で使用するものです。[hashed_array](#hashed_array)と同様です。 - -設定例: - -```xml - - - -``` - -または - -```sql -LAYOUT(COMPLEX_KEY_HASHED_ARRAY([SHARDS 1])) -``` - -### range_hashed {#range_hashed} - -辞書は、範囲とそれに対応する値の順序付き配列を持つハッシュテーブルの形式でメモリ内に保存されます。 - -辞書キーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 -このストレージ方式は `hashed` と同様に動作し、キーに加えて日付/時刻(任意の数値型)の範囲を使用することができます。 - -例: テーブルには、各広告主ごとの割引が次の形式で含まれています。 - -```text -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 123 │ 2015-01-16 │ 2015-01-31 │ 0.25 │ -│ 123 │ 2015-01-01 │ 2015-01-15 │ 0.15 │ -│ 456 │ 2015-01-01 │ 2015-01-15 │ 0.05 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ -``` - -日付範囲を対象としたサンプルを使用するには、[構造](#dictionary-key-and-fields)内で `range_min` と `range_max` 要素を定義します。これらの要素には `name` と `type` フィールドを含める必要があります(`type` が指定されていない場合、デフォルトの型である Date が使用されます)。`type` には任意の数値型(Date / DateTime / UInt64 / Int32 / その他)を指定できます。 - -:::note -`range_min` と `range_max` の値は `Int64` 型に収まる必要があります。 -::: - -例: - -```xml - - - - min - - - - - advertiser_id - - - discount_start_date - Date - - - discount_end_date - Date - - ... -``` - -または - -```sql -CREATE DICTIONARY discounts_dict ( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Date, - amount Float64 -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(TABLE 'discounts')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(RANGE_HASHED(range_lookup_strategy 'max')) -RANGE(MIN discount_start_date MAX discount_end_date) -``` - -これらのディクショナリを使用するには、範囲を指定するための追加の引数を `dictGet` 関数に渡す必要があります。 - -```sql -dictGet('dict_name', 'attr_name', id, date) -``` - -クエリ例: - -```sql -SELECT dictGet('discounts_dict', 'amount', 1, '2022-10-20'::Date); -``` - -この関数は、指定された `id` に対して、渡された日付を含む日付範囲に対応する値を返します。 - -アルゴリズムの詳細: - -* `id` が見つからない場合、またはその `id` に対する範囲が見つからない場合、属性型のデフォルト値を返します。 -* 範囲が重複していて `range_lookup_strategy=min` の場合、一致する範囲のうち `range_min` が最小のものを返し、さらに複数の範囲が見つかった場合は、その中から `range_max` が最小の範囲を返し、それでもなお複数の範囲がある場合(複数の範囲が同じ `range_min` と `range_max` を持つ場合)は、それらの中からランダムに 1 つの範囲を返します。 -* 範囲が重複していて `range_lookup_strategy=max` の場合、一致する範囲のうち `range_min` が最大のものを返し、さらに複数の範囲が見つかった場合は、その中から `range_max` が最大の範囲を返し、それでもなお複数の範囲がある場合(複数の範囲が同じ `range_min` と `range_max` を持つ場合)は、それらの中からランダムに 1 つの範囲を返します。 -* `range_max` が `NULL` の場合、その範囲は開区間です。`NULL` は取り得る最大値として扱われます。`range_min` については、開区間値として `1970-01-01` または `0` (-MAX_INT) を使用できます。 - -設定例: - -```xml - - - ... - - - - - - - - Abcdef - - - StartTimeStamp - UInt64 - - - EndTimeStamp - UInt64 - - - XXXType - String - - - - - - -``` - -または - -```sql -CREATE DICTIONARY somedict( - Abcdef UInt64, - StartTimeStamp UInt64, - EndTimeStamp UInt64, - XXXType String DEFAULT '' -) -PRIMARY KEY Abcdef -RANGE(MIN StartTimeStamp MAX EndTimeStamp) -``` - -重複する範囲と開区間を含む設定例: - -```sql -CREATE TABLE discounts -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -ENGINE = Memory; -``` - -INSERT INTO discounts VALUES (1, '2015-01-01', Null, 0.1); -INSERT INTO discounts VALUES (1, '2015-01-15', Null, 0.2); -INSERT INTO discounts VALUES (2, '2015-01-01', '2015-01-15', 0.3); -INSERT INTO discounts VALUES (2, '2015-01-04', '2015-01-10', 0.4); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-15', 0.5); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-10', 0.6); - -SELECT * FROM discounts ORDER BY advertiser_id, discount_start_date; -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 1 │ 2015-01-01 │ ᴺᵁᴸᴸ │ 0.1 │ -│ 1 │ 2015-01-15 │ ᴺᵁᴸᴸ │ 0.2 │ -│ 2 │ 2015-01-01 │ 2015-01-15 │ 0.3 │ -│ 2 │ 2015-01-04 │ 2015-01-10 │ 0.4 │ -│ 3 │ 1970-01-01 │ 2015-01-15 │ 0.5 │ -│ 3 │ 1970-01-01 │ 2015-01-10 │ 0.6 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ - --- RANGE_LOOKUP_STRATEGY 'max' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'max')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- 一致する範囲は1つだけ: 2015-01-01 - Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.2 │ -- 2つの範囲がマッチしており、range_min 2015-01-15 (0.2) は 2015-01-01 (0.1) より大きい -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.4 │ -- 2つの範囲がマッチしており、range_min 2015-01-04 (0.4) は 2015-01-01 (0.3) より大きい -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.5 │ -- 2つの範囲がマッチしており、range_min は同じで、2015-01-15 (0.5) は 2015-01-10 (0.6) より大きい -└─────┘ - -DROP DICTIONARY discounts_dict; - --- RANGE_LOOKUP_STRATEGY 'min' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'min')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- 一致する範囲は1つだけ: 2015-01-01 - Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.1 │ -- 2つの範囲が一致しており、range_min 2015-01-01 (0.1) は 2015-01-15 (0.2) より小さい -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.3 │ -- 2つの範囲が一致しており、range_min 2015-01-01 (0.3) は 2015-01-04 (0.4) より小さい -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.6 │ -- 2つの範囲が一致しており、range_min は等しく、2015-01-10 (0.6) は 2015-01-15 (0.5) より小さい -└─────┘ - -```` - -### complex_key_range_hashed {#complex_key_range_hashed} - -ディクショナリは、範囲の順序付き配列とそれに対応する値を持つハッシュテーブルの形式でメモリに格納されます([range_hashed](#range_hashed)を参照)。このストレージタイプは、複合[キー](#dictionary-key-and-fields)で使用するためのものです。 - -設定例: - -```sql -CREATE DICTIONARY range_dictionary -( - CountryID UInt64, - CountryKey String, - StartDate Date, - EndDate Date, - Tax Float64 DEFAULT 0.2 -) -PRIMARY KEY CountryID, CountryKey -SOURCE(CLICKHOUSE(TABLE 'date_table')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(COMPLEX_KEY_RANGE_HASHED()) -RANGE(MIN StartDate MAX EndDate); -```` - -### cache {#cache} - -辞書は固定数のセルを持つキャッシュ内に格納されます。これらのセルには頻繁に使用される要素が含まれます。 - -辞書キーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 - -辞書を参照する際には、まずキャッシュが検索されます。各データブロックに対して、キャッシュに存在しない、または期限切れのすべてのキーが、`SELECT attrs... FROM db.table WHERE id IN (k1, k2, ...)` を使ってソースから要求されます。受信したデータはその後キャッシュに書き込まれます。 - -辞書内にキーが見つからない場合、キャッシュ更新タスクが作成され、更新キューに追加されます。更新キューのプロパティは、`max_update_queue_size`、`update_queue_push_timeout_milliseconds`、`query_wait_timeout_milliseconds`、`max_threads_for_updates` の設定で制御できます。 - -キャッシュ辞書では、キャッシュ内データの有効期限 [lifetime](#refreshing-dictionary-data-using-lifetime) を設定できます。あるセルにデータを読み込んでから `lifetime` を超える時間が経過している場合、そのセルの値は使用されず、キーは期限切れと見なされます。そのキーは次回使用が必要になったときに再要求されます。この動作は `allow_read_expired_keys` 設定で構成できます。 - -これは、辞書を保存するすべての方法の中で最も非効率的な方法です。キャッシュの速度は、適切な設定と使用シナリオに大きく依存します。キャッシュ型辞書は、ヒット率が十分に高い場合(推奨は 99% 以上)にのみ高い性能を発揮します。平均ヒット率は [system.dictionaries](../../operations/system-tables/dictionaries.md) テーブルで確認できます。 - -`allow_read_expired_keys` 設定が 1(デフォルトは 0)に設定されている場合、辞書は非同期更新をサポートできます。クライアントがキーを要求し、そのすべてがキャッシュ内に存在するが一部が期限切れである場合、辞書はクライアントに対して期限切れのキーを返し、ソースからそれらを非同期に要求します。 - -キャッシュ性能を向上させるには、`LIMIT` を伴うサブクエリを使用し、辞書を参照する関数はサブクエリの外側で呼び出してください。 - -すべての種類のソースがサポートされています。 - -設定例: - -```xml - - - - 1000000000 - - 0 - - 100000 - - 10 - - 60000 - - 4 - - -``` - -または - -```sql -LAYOUT(CACHE(SIZE_IN_CELLS 1000000000)) -``` - -十分に大きなキャッシュサイズを設定します。セル数は実際に試しながら決定する必要があります。 - -1. ある値を設定する。 -2. キャッシュが完全に埋まるまでクエリを実行する。 -3. `system.dictionaries` テーブルを使用してメモリ使用量を確認する。 -4. 目的のメモリ使用量に達するまでセル数を増減する。 - -:::note -ランダムリードを伴うクエリの処理が遅いため、ClickHouse をソースとして使用しないでください。 -::: - -### complex_key_cache {#complex_key_cache} - -この種類のストレージは、複合[キー](#dictionary-key-and-fields)用に使用します。`cache` と同様です。 - -### ssd_cache {#ssd_cache} - -`cache` と同様ですが、データを SSD に、インデックスを RAM に保存します。更新キューに関連するすべてのキャッシュディクショナリ設定は、SSD キャッシュディクショナリにも適用できます。 - -ディクショナリキーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 - -```xml - - - - 4096 - - 16777216 - - 131072 - - 1048576 - - /var/lib/clickhouse/user_files/test_dict - - -``` - -または - -```sql -LAYOUT(SSD_CACHE(BLOCK_SIZE 4096 FILE_SIZE 16777216 READ_BUFFER_SIZE 1048576 - PATH '/var/lib/clickhouse/user_files/test_dict')) -``` - -### complex_key_ssd_cache {#complex_key_ssd_cache} - -このストレージタイプは、複合[キー](#dictionary-key-and-fields)用に使用します。`ssd_cache` と同様です。 - -### direct {#direct} - -このディクショナリはメモリ上には保持されず、リクエストの処理中にソースへ直接アクセスします。 - -ディクショナリキーは [UInt64](../../sql-reference/data-types/int-uint.md) 型です。 - -ローカルファイルを除くすべての種類の[ソース](#dictionary-sources)がサポートされています。 - -構成例: - -```xml - - - -``` - -または - -```sql -LAYOUT(DIRECT()) -``` - -### complex_key_direct {#complex_key_direct} - -この種類のストレージは、複合[キー](#dictionary-key-and-fields)で使用するためのものです。`direct` と同様です。 - -### ip_trie {#ip_trie} - -このディクショナリは、ネットワークプレフィックスによる IP アドレス検索向けに設計されています。IP アドレスの範囲を CIDR 表記で保持し、ある IP がどのプレフィックス(例: サブネットまたは ASN 範囲)に属するかを高速に判定できるため、ジオロケーションやネットワーク分類などの IP ベースの検索に最適です。 - - - - -Движок обрабатывает все столбцы со следующими типами: - -- [`AggregateFunction`](../../../sql-reference/data-types/aggregatefunction.md) -- [`SimpleAggregateFunction`](../../../sql-reference/data-types/simpleaggregatefunction.md) - -Имеет смысл использовать `AggregatingMergeTree`, если он уменьшает число строк на несколько порядков. - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = AggregatingMergeTree() -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[TTL expr] -[SETTINGS name=value, ...] -``` - -Для описания параметров запроса см. [описание запроса](../../../sql-reference/statements/create/table.md). - -**Части запроса** - -При создании таблицы `AggregatingMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. - -
- Устаревший способ создания таблицы - - :::note - Не используйте этот способ в новых проектах и по возможности переведите старые проекты на метод, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] AggregatingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - Все параметры имеют то же значение, что и в `MergeTree`. -
- - -## SELECT и INSERT {#select-and-insert} - -Для вставки данных используйте запрос [INSERT SELECT](../../../sql-reference/statements/insert-into.md) с агрегирующими функциями с суффиксом `-State`. -При выборке данных из таблицы `AggregatingMergeTree` используйте предложение `GROUP BY` и те же агрегирующие функции, что и при вставке данных, но с суффиксом `-Merge`. - -В результатах запроса `SELECT` значения типа `AggregateFunction` имеют двоичное представление, зависящее от реализации, для всех форматов вывода ClickHouse. Например, если вы выгружаете данные в формате `TabSeparated` с помощью запроса `SELECT`, то этот дамп можно загрузить обратно с помощью запроса `INSERT`. - - - -## Пример агрегированного материализованного представления {#example-of-an-aggregated-materialized-view} - -В этом примере предполагается, что у вас есть база данных под названием `test`. Создайте её, если она ещё не существует, с помощью приведённой ниже команды: - -```sql -CREATE DATABASE test; -``` - -Теперь создайте таблицу `test.visits`, которая содержит сырые данные: - -```sql -CREATE TABLE test.visits - ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Sign Nullable(Int32), - UserID Nullable(Int32) -) ENGINE = MergeTree ORDER BY (StartDate, CounterID); -``` - -Далее необходимо создать таблицу `AggregatingMergeTree`, которая будет хранить агрегирующие функции `AggregationFunction`, отслеживающие общее количество посещений и количество уникальных пользователей. - -Создайте материализованное представление с движком `AggregatingMergeTree`, которое отслеживает таблицу `test.visits` и использует тип [`AggregateFunction`](/sql-reference/data-types/aggregatefunction): - -```sql -CREATE TABLE test.agg_visits ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Visits AggregateFunction(sum, Nullable(Int32)), - Users AggregateFunction(uniq, Nullable(Int32)) -) -ENGINE = AggregatingMergeTree() ORDER BY (StartDate, CounterID); -``` - -Создайте материализованное представление, которое заполняет таблицу `test.agg_visits` данными из `test.visits`: - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - sumState(Sign) AS Visits, - uniqState(UserID) AS Users -FROM test.visits -GROUP BY StartDate, CounterID; -``` - -Добавьте данные в таблицу `test.visits`: - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1667446031000, 1, 3, 4), (1667446031000, 1, 6, 3); -``` - -Данные вставляются как в `test.visits`, так и в `test.agg_visits`. - -Чтобы получить агрегированные данные, выполните запрос вида `SELECT ... GROUP BY ...` к материализованному представлению `test.visits_mv`: - -```sql -SELECT - StartDate, - sumMerge(Visits) AS Visits, - uniqMerge(Users) AS Users -FROM test.visits_mv -GROUP BY StartDate -ORDER BY StartDate; -``` - -```text -┌───────────────StartDate─┬─Visits─┬─Users─┐ -│ 2022-11-03 03:27:11.000 │ 9 │ 2 │ -└─────────────────────────┴────────┴───────┘ -``` - -Добавьте ещё пару записей в `test.visits`, но на этот раз попробуйте использовать другое значение временной метки для одной из записей: - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1669446031000, 2, 5, 10), (1667446031000, 3, 7, 5); -``` - -Выполните запрос `SELECT` ещё раз — будет выведен следующий результат: - -```text -┌───────────────ДатаНачала─┬─Посещения─┬─Пользователи─┐ -│ 2022-11-03 03:27:11.000 │ 16 │ 3 │ -│ 2022-11-26 07:00:31.000 │ 5 │ 1 │ -└─────────────────────────┴────────┴───────┘ -``` - -В некоторых случаях вы можете захотеть избежать предварительной агрегации строк во время вставки, чтобы перенести нагрузку агрегации с момента вставки -на момент слияния. Обычно необходимо включать столбцы, которые не участвуют в агрегации, в оператор `GROUP BY` -в определении материализованного представления, чтобы избежать ошибки. Однако вы можете воспользоваться функцией [`initializeAggregation`](/sql-reference/functions/other-functions#initializeAggregation) -с настройкой `optimize_on_insert = 0` (по умолчанию она включена), чтобы добиться этого. Использование `GROUP BY` -в этом случае больше не требуется: - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - initializeAggregation('sumState', Sign) AS Visits, - initializeAggregation('uniqState', UserID) AS Users -FROM test.visits; -``` - - -:::note -При использовании `initializeAggregation` агрегатное состояние создаётся для каждой отдельной строки без группировки. -Каждая исходная строка даёт одну строку в материализованном представлении, а фактическая агрегация происходит позже, когда -`AggregatingMergeTree` объединяет части. Это верно только в том случае, если `optimize_on_insert = 0`. -::: - - - -## Связанные материалы {#related-content} - -- Блог: [Использование комбинаторов агрегатных функций в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md deleted file mode 100644 index 0c49b86fed3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ /dev/null @@ -1,736 +0,0 @@ ---- -description: 'Документация по точному и приближённому векторному поиску' -keywords: ['векторный поиск по сходству', 'ann', 'knn', 'hnsw', 'индексы', 'индекс', 'поиск ближайших соседей', 'векторный поиск'] -sidebar_label: 'Точный и приближённый векторный поиск' -slug: /engines/table-engines/mergetree-family/annindexes -title: 'Точный и приближённый векторный поиск' -doc_type: 'guide' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# Точный и приближённый векторный поиск {#exact-and-approximate-vector-search} - -Задача нахождения N ближайших точек в многомерном (векторном) пространстве для заданной точки известна как [поиск ближайших соседей](https://en.wikipedia.org/wiki/Nearest_neighbor_search) или, кратко, векторный поиск. -Существуют два общих подхода к решению задачи векторного поиска: - -* Точный векторный поиск вычисляет расстояние между заданной точкой и всеми точками в векторном пространстве. Это обеспечивает максимально возможную точность, то есть возвращаемые точки гарантированно являются фактическими ближайшими соседями. Поскольку векторное пространство просматривается полностью, точный векторный поиск может быть слишком медленным для практического использования. -* Приближённый векторный поиск относится к группе методов (например, специальные структуры данных, такие как графы и случайные леса), которые вычисляют результаты намного быстрее, чем точный векторный поиск. Точность результата, как правило, «достаточно хороша» для практического применения. Многие приближённые методы предоставляют параметры для настройки компромисса между точностью результата и временем поиска. - -Векторный поиск (точный или приближённый) может быть записан на SQL следующим образом: - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- условие WHERE необязательно -ORDER BY (vectors, reference_vector) -LIMIT -``` - -Точки в векторном пространстве хранятся в столбце `vectors` типа массива, например [Array(Float64)](../../../sql-reference/data-types/array.md), [Array(Float32)](../../../sql-reference/data-types/array.md) или [Array(BFloat16)](../../../sql-reference/data-types/array.md). -Эталонный вектор — это константный массив, задаваемый в виде общего табличного выражения. -`<DistanceFunction>` вычисляет расстояние между эталонной точкой и всеми сохранёнными точками. -Для этого может быть использована любая из доступных [функций расстояния](/sql-reference/functions/distance-functions). -`<N>` задаёт, сколько соседей нужно вернуть. - - -## Точный поиск по векторам {#exact-nearest-neighbor-search} - -Точный поиск по векторам можно выполнить с использованием приведённого выше запроса SELECT без изменений. -Время выполнения таких запросов, как правило, пропорционально количеству сохранённых векторов и их размерности, то есть количеству элементов массива. -Кроме того, поскольку ClickHouse выполняет полный перебор всех векторов, время выполнения таких запросов также зависит от количества потоков, используемых запросом (см. настройку [max_threads](../../../operations/settings/settings.md#max_threads)). - -### Пример {#exact-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -возвращает - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - - -## Приблизительный векторный поиск {#approximate-nearest-neighbor-search} - -### Индексы сходства векторов {#vector-similarity-index} - -ClickHouse предоставляет специальный индекс сходства векторов («vector similarity») для выполнения приблизительного векторного поиска. - -:::note -Индексы сходства векторов доступны в ClickHouse версиях 25.8 и новее. -Если вы столкнётесь с проблемами, пожалуйста, создайте issue в [репозитории ClickHouse](https://github.com/clickhouse/clickhouse/issues). -::: - -#### Создание индекса сходства векторов {#creating-a-vector-similarity-index} - -Индекс сходства векторов можно создать на новой таблице следующим образом: - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -В качестве альтернативы можно добавить индекс векторного сходства к уже существующей таблице: - -```sql -ALTER TABLE table ADD INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ]; -``` - -Индексы векторного сходства — это особый вид пропускающих индексов (см. [здесь](mergetree.md#table_engine-mergetree-data_skipping-indexes) и [здесь](../../../optimize/skipping-indexes)). -Соответственно, приведенный выше оператор `ALTER TABLE` приводит к тому, что индекс строится только для новых данных, которые будут вставляться в таблицу. -Чтобы построить индекс и для уже существующих данных, его необходимо материализовать: - -```sql -ALTER TABLE table MATERIALIZE INDEX SETTINGS mutations_sync = 2; -``` - -Функция `` должна быть - -* `L2Distance`, [евклидовым расстоянием](https://en.wikipedia.org/wiki/Euclidean_distance), представляющим длину отрезка между двумя точками в евклидовом пространстве, или -* `cosineDistance`, [косинусным расстоянием](https://en.wikipedia.org/wiki/Cosine_similarity#Cosine_distance), представляющим угол между двумя ненулевыми векторами. - -Для нормализованных данных обычно лучше всего подходит `L2Distance`, в противном случае рекомендуется `cosineDistance`, чтобы компенсировать масштаб. - -`` задаёт размер массива (количество элементов) в базовом столбце. -Если при создании индекса ClickHouse обнаружит массив с другим размером, индекс не будет создан, а будет возвращена ошибка. - -Необязательный параметр GRANULARITY `` относится к размеру гранул индекса (см. [здесь](../../../optimize/skipping-indexes)). -Значение по умолчанию — 100 миллионов — должно хорошо подходить для большинства вариантов использования, но его также можно настроить. -Мы рекомендуем выполнять настройку только продвинутым пользователям, которые понимают последствия своих действий (см. [ниже](#differences-to-regular-skipping-indexes)). - -Индексы векторного сходства являются обобщёнными в том смысле, что могут использовать различные методы приблизительного поиска. -Фактически используемый метод задаётся параметром ``. -На данный момент доступен только метод HNSW ([академическая статья](https://arxiv.org/abs/1603.09320)) — популярная и современная техника приблизительного поиска по векторам на основе иерархических графов близости. -Если в качестве типа используется HNSW, пользователи могут дополнительно указать специфичные для HNSW параметры: - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX index_name vectors TYPE vector_similarity('hnsw', , [, , , ]) [GRANULARITY N] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -Доступны следующие параметры, специфичные для HNSW: - -* `` управляет квантизацией векторов в графе близости. Возможные значения: `f64`, `f32`, `f16`, `bf16`, `i8` или `b1`. Значение по умолчанию — `bf16`. Обратите внимание, что этот параметр не влияет на представление векторов во внутреннем столбце. -* `` управляет количеством соседей у каждой вершины графа, также известным как гиперпараметр HNSW `M`. Значение по умолчанию — `32`. Значение `0` означает использование значения по умолчанию. -* `` управляет размером динамического списка кандидатов во время построения графа HNSW, также известным как гиперпараметр HNSW `ef_construction`. Значение по умолчанию — `128`. Значение `0` означает использование значения по умолчанию. - -Значения по умолчанию всех параметров, специфичных для HNSW, достаточно хорошо подходят для большинства сценариев использования. -Поэтому мы не рекомендуем изменять эти параметры. - - -Дополнительно действуют следующие ограничения: - -* Индексы векторного сходства могут быть построены только по столбцам типов [Array(Float32)](../../../sql-reference/data-types/array.md), [Array(Float64)](../../../sql-reference/data-types/array.md) или [Array(BFloat16)](../../../sql-reference/data-types/array.md). Массивы допускающих `NULL` и чисел с плавающей запятой с низкой кардинальностью, такие как `Array(Nullable(Float32))` и `Array(LowCardinality(Float32))`, не поддерживаются. -* Индексы векторного сходства должны строиться по отдельным столбцам. -* Индексы векторного сходства могут быть построены по вычисляемым выражениям (например, `INDEX index_name arraySort(vectors) TYPE vector_similarity([...])`), но такие индексы не могут быть использованы для последующего приближённого поиска ближайших соседей. -* Для индексов векторного сходства требуется, чтобы все массивы в базовом столбце содержали `` элементов — это проверяется при создании индекса. Чтобы как можно раньше обнаружить нарушения этого требования, пользователи могут добавить [ограничение](/sql-reference/statements/create/table.md#constraints) для векторного столбца, например, `CONSTRAINT same_length CHECK length(vectors) = 256`. -* Аналогично, значения массивов в базовом столбце не должны быть пустыми (`[]`) и не должны иметь пустое значение по умолчанию (также `[]`). - -**Оценка потребления хранилища и памяти** - -Вектор, сгенерированный для использования с типичной моделью ИИ (например, большой языковой моделью, [LLMs](https://en.wikipedia.org/wiki/Large_language_model)), состоит из сотен или тысяч значений с плавающей запятой. -Таким образом, одно векторное значение может потреблять несколько килобайт памяти. -Пользователи, которые хотят оценить объём хранилища, необходимый для базового векторного столбца в таблице, а также объём оперативной памяти, необходимый для индекса векторного сходства, могут использовать две приведённые ниже формулы: - -Потребление хранилища векторным столбцом в таблице (без сжатия): - -```text -Потребление хранилища = Количество векторов × Размерность × Размер типа данных столбца -``` - -Пример для [датасета DBpedia](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M): - -```text -Потребление хранилища = 1 миллион * 1536 * 4 (для Float32) = 6,1 ГБ -``` - -Индекс сходства векторов должен быть полностью загружен с диска в основную память для выполнения поиска. -Аналогично, векторный индекс также полностью строится в памяти, а затем сохраняется на диск. - -Объём памяти, необходимый для загрузки векторного индекса: - -```text -Память для векторов в индексе (mv) = Количество векторов * Размерность * Размер квантованного типа данных -Память для графа в оперативной памяти (mg) = Количество векторов * hnsw_max_connections_per_layer * Байт_на_идентификатор_узла (= 4) * Коэффициент_повторения_узлов_по_слоям (= 2) - -Потребление памяти: mv + mg -``` - -Пример для [набора данных DBpedia](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M): - -```text -Память для векторов в индексе (mv) = 1 млн * 1536 * 2 (для BFloat16) = 3072 МБ -Память для графа в памяти (mg) = 1 млн * 64 * 2 * 4 = 512 МБ - -Потребление памяти = 3072 + 512 = 3584 МБ -``` - -Приведенная выше формула не учитывает дополнительную память, необходимую индексам векторного сходства для выделения структур данных, используемых во время выполнения, таких как заранее выделенные буферы и кэши. - -#### Использование индекса векторного сходства {#using-a-vector-similarity-index} - -:::note -Чтобы использовать индексы векторного сходства, настройка [compatibility](../../../operations/settings/settings.md) должна быть равна `''` (значение по умолчанию) или `'25.1'` либо новее. -::: - -Индексы векторного сходства поддерживают запросы SELECT следующего вида: - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- предложение WHERE необязательно -ORDER BY (vectors, reference_vector) -LIMIT -``` - -Оптимизатор запросов ClickHouse пытается сопоставить запрос с приведённым выше шаблоном и использовать доступные индексы векторного сходства. -Запрос может использовать индекс векторного сходства только в том случае, если функция расстояния в запросе SELECT совпадает с функцией расстояния в определении индекса. - -Продвинутые пользователи могут задать собственное значение настройки [hnsw_candidate_list_size_for_search](../../../operations/settings/settings.md#hnsw_candidate_list_size_for_search) (также известной как гиперпараметр HNSW «ef_search»), чтобы настраивать размер списка кандидатов при выполнении поиска (например, `SELECT [...] SETTINGS hnsw_candidate_list_size_for_search = `). -Значение настройки по умолчанию, равное 256, хорошо работает в большинстве сценариев использования. -Более высокие значения настройки обеспечивают лучшую точность ценой более низкой производительности. - - -Если запрос может использовать индекс векторного сходства, ClickHouse проверяет, что значение LIMIT ``, указанное в запросах SELECT, находится в разумных пределах. -Более точно, будет возвращена ошибка, если `` больше значения настройки [max_limit_for_vector_search_queries](../../../operations/settings/settings.md#max_limit_for_vector_search_queries), по умолчанию равного 100. -Слишком большие значения LIMIT могут замедлять поиск и обычно указывают на ошибку в использовании. - -Чтобы проверить, использует ли запрос SELECT индекс векторного сходства, вы можете добавить к запросу префикс `EXPLAIN indexes = 1`. - -В качестве примера, запрос - -```sql -EXPLAIN indexes = 1 -WITH [0.462, 0.084, ..., -0.110] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 10; -``` - -может вернуть - -```result - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (Проекция имён) │ - 2. │ Limit (предварительный LIMIT (без OFFSET)) │ - 3. │ Sorting (Сортировка для ORDER BY) │ - 4. │ Expression ((Перед ORDER BY + (Проекция + Преобразование имён столбцов в идентификаторы))) │ - 5. │ ReadFromMergeTree (default.tab) │ - 6. │ Индексы: │ - 7. │ PrimaryKey │ - 8. │ Условие: true │ - 9. │ Части: 1/1 │ -10. │ Гранулы: 575/575 │ -11. │ Skip │ -12. │ Имя: idx │ -13. │ Описание: vector_similarity GRANULARITY 100000000 │ -14. │ Части: 1/1 │ -15. │ Гранулы: 10/575 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -В этом примере 1 миллион векторов из [набора данных dbpedia](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M), каждый размерностью 1536, хранятся в 575 гранулах, т.е. примерно по 1,7 тыс. строк на гранулу. -В запросе запрашиваются 10 соседей, и индекс векторного сходства находит этих 10 соседей в 10 отдельных гранулах. -Эти 10 гранул будут прочитаны во время выполнения запроса. - -Индексы векторного сходства используются, если вывод содержит `Skip`, а также имя и тип векторного индекса (в примере `idx` и `vector_similarity`). -В этом случае индекс векторного сходства отбросил две из четырёх гранул, т.е. 50% данных. -Чем больше гранул может быть отброшено, тем эффективнее используется индекс. - -:::tip -Чтобы принудительно использовать индекс, вы можете выполнить запрос SELECT с параметром [force_data_skipping_indexes](../../../operations/settings/settings#force_data_skipping_indices) (укажите имя индекса в качестве значения параметра). -::: - -**Постфильтрация и префильтрация** - -Пользователи могут дополнительно указать предложение `WHERE` с дополнительными условиями фильтрации для запроса SELECT. -ClickHouse будет применять эти условия фильтрации, используя стратегию постфильтрации или префильтрации. -Кратко, обе стратегии определяют порядок, в котором применяются фильтры: - -* Постфильтрация означает, что индекс векторного сходства применяется первым, после чего ClickHouse применяет дополнительные фильтры, указанные в предложении `WHERE`. -* Префильтрация означает, что порядок применения фильтров будет обратным. - -У стратегий разные компромиссы: - -* У постфильтрации есть типичная проблема: она может вернуть меньше строк, чем запрошено в предложении `LIMIT `. Такая ситуация возникает, когда одна или несколько строк результата, возвращённых индексом векторного сходства, не удовлетворяют дополнительным фильтрам. -* Префильтрация в целом остаётся нерешённой задачей. Некоторые специализированные векторные базы данных предоставляют алгоритмы префильтрации, но большинство реляционных баз данных (включая ClickHouse) будут переходить к точному поиску соседей, т.е. к полному перебору без индекса. - -Используемая стратегия зависит от условия фильтрации. - -*Дополнительные фильтры являются частью ключа партиционирования* - -Если дополнительное условие фильтрации является частью ключа партиционирования, то ClickHouse применит отсечение партиций. -В качестве примера, таблица разбита на диапазонные партиции по столбцу `year`, и выполняется следующий запрос: - -```sql -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -WHERE year = 2025 -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -ClickHouse отбросит все партиции, кроме партиции за 2025 год. - -*Дополнительные фильтры, которые не могут быть выполнены по индексам* - -Если дополнительные условия фильтрации не могут быть выполнены по индексам (индекс по первичному ключу, пропускающий индекс), ClickHouse применит последующую фильтрацию. - - -*Дополнительные фильтры могут оцениваться с использованием индекса первичного ключа* - -Если дополнительные условия фильтрации могут быть оценены с использованием [первичного ключа](mergetree.md#primary-key) (т. е. они образуют префикс первичного ключа) и - -* условие фильтрации отбрасывает как минимум одну строку внутри части, ClickHouse будет использовать предварительную фильтрацию для «выживших» диапазонов внутри части, -* условие фильтрации не отбрасывает ни одной строки внутри части, ClickHouse будет выполнять постфильтрацию для этой части. - -На практике второй вариант встречается довольно редко. - -*Дополнительные фильтры могут оцениваться с использованием skipping index* - -Если дополнительные условия фильтрации могут быть оценены с использованием [skipping indexes](mergetree.md#table_engine-mergetree-data_skipping-indexes) (minmax index, set index и т. д.), ClickHouse выполняет постфильтрацию. -В таких случаях индекс векторного сходства вычисляется первым, так как ожидается, что он отбросит наибольшее количество строк по сравнению с другими skipping indexes. - -Для более тонкого управления постфильтрацией и предварительной фильтрацией можно использовать два параметра: - -Параметр [vector_search_filter_strategy](../../../operations/settings/settings#vector_search_filter_strategy) (по умолчанию: `auto`, который реализует приведённые выше эвристики) может быть установлен в значение `prefilter`. -Это полезно для принудительной предварительной фильтрации в случаях, когда дополнительные условия фильтрации обладают крайне высокой селективностью. -Например, следующий запрос может выиграть от предварительной фильтрации: - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('Книги о древних азиатских империях')) -LIMIT 10 -``` - -Предположим, что только очень небольшое количество книг стоит меньше 2 долларов. В этом случае постфильтрация может вернуть ноль строк, потому что все 10 лучших совпадений, возвращённых векторным индексом, могут иметь цену выше 2 долларов. -Принудительно включив префильтрацию (добавьте `SETTINGS vector_search_filter_strategy = 'prefilter'` к запросу), ClickHouse сначала находит все книги с ценой менее 2 долларов, а затем выполняет переборный (brute-force) векторный поиск по найденным книгам. - -В качестве альтернативного подхода к решению указанной выше проблемы можно задать настройку [vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier) (по умолчанию: `1.0`, максимум: `1000.0`) значением > `1.0` (например, `2.0`). -Количество ближайших соседей, выбираемых из векторного индекса, умножается на значение этой настройки, после чего к этим строкам применяется дополнительный фильтр, чтобы вернуть количество строк, не превышающее значение LIMIT. -Например, мы можем выполнить запрос ещё раз, но уже с множителем `3.0`: - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('Books on ancient Asian empires')) -LIMIT 10 -SETTING vector_search_index_fetch_multiplier = 3.0; -``` - -ClickHouse извлечёт 3,0 x 10 = 30 ближайших соседей из векторного индекса в каждой части и затем применит дополнительные фильтры. -Будут возвращены только десять ближайших соседей. -Отметим, что настройка `vector_search_index_fetch_multiplier` может смягчить проблему, но в крайних случаях (при очень селективном условии WHERE) всё ещё возможно, что будет возвращено меньше N запрошенных строк. - -**Пересчёт оценок (rescoring)** - -Skip-индексы в ClickHouse обычно фильтруют данные на уровне гранул, то есть запрос к skip-индексу (внутренне) возвращает список потенциально подходящих гранул, что сокращает объём читаемых данных при последующем сканировании. -Это хорошо работает для skip-индексов в целом, но в случае индексов векторного сходства создаётся «несоответствие гранулярности». -Подробнее: индекс векторного сходства определяет номера строк N наиболее похожих векторов для заданного опорного вектора, но затем ему нужно сопоставить эти номера строк с номерами гранул. -ClickHouse затем загрузит эти гранулы с диска и повторит вычисление расстояний для всех векторов в этих гранулах. -Этот шаг называется пересчётом оценок (rescoring), и хотя теоретически он может повысить точность — помните, что индекс векторного сходства возвращает только *приближённый* результат, — очевидно, что с точки зрения производительности он не оптимален. - -Поэтому ClickHouse предоставляет оптимизацию, которая отключает пересчёт оценок и возвращает наиболее похожие векторы и их расстояния напрямую из индекса. -Эта оптимизация включена по умолчанию, см. настройку [vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring). -В общих чертах она работает так: ClickHouse делает доступными наиболее похожие векторы и их расстояния в виде виртуального столбца `_distances`. -Чтобы увидеть это, выполните запрос векторного поиска с `EXPLAIN header = 1`: - - -```sql -EXPLAIN header = 1 -WITH [0., 2.] AS reference_vec -SELECT id -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3 -SETTINGS vector_search_with_rescoring = 0 -``` - -```result -Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 - - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (Проекция имён) │ - 2. │ Header: id Int32 │ - 3. │ Limit (предварительный LIMIT (без OFFSET)) │ - 4. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 5. │ __table1.id Int32 │ - 6. │ Sorting (Сортировка для ORDER BY) │ - 7. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 8. │ __table1.id Int32 │ - 9. │ Expression ((До ORDER BY + (Проекция + Замена имён столбцов на идентификаторы столбцов))) │ -10. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ -11. │ __table1.id Int32 │ -12. │ ReadFromMergeTree (default.tab) │ -13. │ Header: id Int32 │ -14. │ _distance Float32 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -:::note -Запрос, выполняемый без повторной оценки (`vector_search_with_rescoring = 0`) и с включёнными параллельными репликами, может всё равно перейти к повторной оценке результатов. -::: - -#### Оптимизация производительности {#performance-tuning} - -**Настройка сжатия** - -Практически во всех вариантах использования векторы в соответствующем столбце являются плотными и плохо сжимаются. -В результате [сжатие](/sql-reference/statements/create/table.md#column_compression_codec) замедляет операции вставки данных во векторный столбец и чтения из него. -Поэтому мы рекомендуем отключить сжатие. -Для этого укажите `CODEC(NONE)` для векторного столбца следующим образом: - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32) CODEC(NONE), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; -``` - -**Настройка создания индексов** - -Жизненный цикл индексов векторного сходства привязан к жизненному циклу частей. -Другими словами, всякий раз, когда создаётся новая часть с определённым индексом векторного сходства, создаётся и сам индекс. -Обычно это происходит при [вставке данных](https://clickhouse.com/docs/guides/inserting-data) или во время [слияний](https://clickhouse.com/docs/merges). -К сожалению, HNSW известен длительным временем создания индексов, что может значительно замедлять вставки и слияния. -Индексы векторного сходства оптимальны, когда данные неизменяемы или изменяются редко. - -Для ускорения создания индексов можно использовать следующие приёмы: - -Во-первых, создание индексов можно распараллелить. -Максимальное количество потоков создания индексов настраивается с помощью серверного параметра [max_build_vector_similarity_index_thread_pool_size](/operations/server-configuration-parameters/settings#max_build_vector_similarity_index_thread_pool_size). -Для оптимальной производительности значение этого параметра следует устанавливать равным числу ядер CPU. - -Во-вторых, для ускорения операторов INSERT пользователи могут отключить создание пропускающих индексов для вновь вставляемых частей при помощи параметра сессии [materialize_skip_indexes_on_insert](../../../operations/settings/settings.md#materialize_skip_indexes_on_insert). -SELECT-запросы к таким частям будут выполнять точный поиск. -Поскольку вставляемые части, как правило, малы по сравнению с общим размером таблицы, ожидается, что влияние этого на производительность будет незначительным. - -В-третьих, для ускорения слияний пользователи могут отключить создание пропускающих индексов на слитых частях с помощью параметра сессии [materialize_skip_indexes_on_merge](../../../operations/settings/merge-tree-settings.md#materialize_skip_indexes_on_merge). -В сочетании с оператором [ALTER TABLE [...] MATERIALIZE INDEX [...]](../../../sql-reference/statements/alter/skipping-index.md#materialize-index) это обеспечивает явный контроль над жизненным циклом индексов векторного сходства. -Например, создание индексов можно отложить до момента, когда весь объём данных уже принят, или до периода низкой нагрузки на систему, например на выходные. - -**Настройка использования индексов** - - -Запросы SELECT должны загружать индексы векторного сходства в оперативную память, чтобы использовать их. -Чтобы один и тот же индекс векторного сходства не загружался в оперативную память многократно, ClickHouse предоставляет специализированный кэш в оперативной памяти для таких индексов. -Чем больше этот кэш, тем меньше будет лишних загрузок. -Максимальный размер кэша можно настроить с помощью серверной настройки [vector_similarity_index_cache_size](../../../operations/server-configuration-parameters/settings.md#vector_similarity_index_cache_size). -По умолчанию кэш может увеличиваться до 5 ГБ. - -:::note -Кэш индекса векторного сходства хранит гранулы векторного индекса. -Если отдельные гранулы векторного индекса больше размера кэша, они не будут кэшироваться. -Поэтому необходимо вычислить размер векторного индекса (на основе формулы из раздела «Оценка потребления хранилища и памяти» или [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices)) и задать размер кэша соответствующим образом. -::: - -Текущий размер кэша индекса векторного сходства отображается в [system.metrics](../../../operations/system-tables/metrics.md): - -```sql -SELECT metric, value -FROM system.metrics -WHERE metric = 'VectorSimilarityIndexCacheBytes' -``` - -Информацию о попаданиях и промахах кэша для запроса с заданным идентификатором можно получить из [system.query_log](../../../operations/system-tables/query_log.md): - -```sql -SYSTEM FLUSH LOGS query_log; - -SELECT ProfileEvents['VectorSimilarityIndexCacheHits'], ProfileEvents['VectorSimilarityIndexCacheMisses'] -FROM system.query_log -WHERE type = 'QueryFinish' AND query_id = '<...>' -ORDER BY event_time_microseconds; -``` - -Для production-сценариев мы рекомендуем выбирать размер кэша таким образом, чтобы все векторные индексы постоянно помещались в память. - -**Настройка квантования** - -[Квантование](https://huggingface.co/blog/embedding-quantization) — это метод уменьшения объёма памяти, занимаемой векторами, и вычислительных затрат на построение и обход векторных индексов. -Векторные индексы ClickHouse поддерживают следующие варианты квантования: - -| Quantization | Name | Storage per dimension | -| -------------- | ---------------------------- | --------------------- | -| f32 | Single precision | 4 bytes | -| f16 | Half precision | 2 bytes | -| bf16 (default) | Half precision (brain float) | 2 bytes | -| i8 | Quarter precision | 1 byte | -| b1 | Binary | 1 bit | - -Квантование снижает точность векторного поиска по сравнению с поиском по исходным значениям с полной точностью с плавающей запятой (`f32`). -Однако на большинстве наборов данных квантование в формате half-precision brain float (`bf16`) приводит к пренебрежимо малой потере точности, поэтому векторные индексы похожести по умолчанию используют именно этот метод квантования. -Квантование до четверной точности (`i8`) и бинарное квантование (`b1`) приводят к заметной потере точности в векторном поиске. -Мы рекомендуем эти варианты квантования только в том случае, если размер векторного индекса похожести значительно превышает доступный объём DRAM. -В этом случае мы также рекомендуем включить пересчёт скоринга ([vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier), [vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring)) для повышения точности. -Бинарное квантование рекомендуется только для 1) нормализованных эмбеддингов (т.е. длина вектора = 1, модели OpenAI обычно нормализованы) и 2) если в качестве функции расстояния используется косинусное расстояние. -Бинарное квантование на внутреннем уровне использует расстояние Хэмминга для построения и обхода графа близости. -На этапе пересчёта скоринга используются исходные векторы с полной точностью, хранящиеся в таблице, для определения ближайших соседей по косинусному расстоянию. - -**Настройка передачи данных** - -Опорный вектор в запросе векторного поиска предоставляется пользователем и, как правило, получается путём вызова большой языковой модели (LLM). -Типичный Python-код, выполняющий векторный поиск в ClickHouse, может выглядеть так: - -```python -search_v = openai_client.embeddings.create(input = "[Good Books]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'search_v': search_v} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, %(search_v)s) - LIMIT 10", - parameters = params) -``` - - -Векторы встраивания (`search_v` в приведённом выше фрагменте) могут иметь очень большую размерность. -Например, OpenAI предоставляет модели, которые генерируют векторы встраивания с размерностью 1536 или даже 3072. -В приведённом выше коде драйвер ClickHouse для Python подставляет вектор встраивания, преобразуя его в человекочитаемую строку, и затем целиком отправляет запрос SELECT в виде строки. -Предположим, что вектор встраивания состоит из 1536 значений с плавающей запятой одинарной точности, тогда длина отправляемой строки достигает 20 кБ. -Это приводит к высокому использованию CPU на токенизацию, разбор и выполнение тысяч преобразований строки в число с плавающей запятой. -Кроме того, требуется значительный объём места в файле журнала сервера ClickHouse, что также вызывает разрастание `system.query_log`. - -Обратите внимание, что большинство LLM‑моделей возвращают вектор встраивания в виде списка или массива NumPy из нативных чисел с плавающей запятой. -Поэтому мы рекомендуем Python‑приложениям привязывать параметр опорного вектора в бинарной форме, используя следующий стиль: - -```python -search_v = openai_client.embeddings.create(input = "[Хорошие книги]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'$search_v_binary$': np.array(search_v, dtype=np.float32).tobytes()} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, (SELECT reinterpret($search_v_binary$, 'Array(Float32)'))) - LIMIT 10", - parameters = params) -``` - -В этом примере опорный вектор отправляется как есть в бинарном виде и на сервере интерпретируется как массив чисел с плавающей запятой. -Это экономит процессорное время на стороне сервера и предотвращает избыточный рост серверных логов и `system.query_log`. - -#### Администрирование и мониторинг {#administration} - -Объём на диске индексов векторного сходства можно получить из [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices): - -```sql -SELECT database, table, name, formatReadableSize(data_compressed_bytes) -FROM system.data_skipping_indices -WHERE type = 'vector_similarity'; -``` - -Пример результата: - -```result -┌─database─┬─table─┬─name─┬─formatReadab⋯ssed_bytes)─┐ -│ default │ tab │ idx │ 348.00 MB │ -└──────────┴───────┴──────┴──────────────────────────┘ -``` - -#### Отличия от обычных пропускающих индексов {#differences-to-regular-skipping-indexes} - -Как и все обычные [пропускающие индексы](/optimize/skipping-indexes), индексы векторного сходства строятся поверх гранул, и каждый индексируемый блок состоит из `GRANULARITY = [N]` гранул (`[N]` = 1 по умолчанию для обычных пропускающих индексов). -Например, если гранулярность первичного индекса таблицы равна 8192 (параметр `index_granularity = 8192`) и `GRANULARITY = 2`, то каждый индексируемый блок будет содержать 16384 строки. -Однако структуры данных и алгоритмы для поиска приблизительных соседей по своей природе ориентированы на строки. -Они хранят компактное представление набора строк, а также возвращают строки в ответ на запросы векторного поиска. -Это приводит к довольно неочевидным отличиям в поведении индексов векторного сходства по сравнению с обычными пропускающими индексами. - -Когда пользователь определяет индекс векторного сходства по столбцу, ClickHouse внутренне создает для каждого индексного блока отдельный «подиндекс» векторного сходства. -Подиндекс является «локальным» в том смысле, что он знает только о строках своего индексного блока. -В предыдущем примере, при условии, что столбец содержит 65536 строк, мы получаем четыре индексных блока (охватывающих восемь гранул) и подиндекс векторного сходства для каждого индексного блока. -Теоретически подиндекс может напрямую возвращать строки с N ближайшими точками в пределах своего индексного блока. -Однако, поскольку ClickHouse загружает данные с диска в память на уровне гранул, подиндексы расширяют найденные строки до границ гранул. -Это отличается от обычных пропускающих индексов, которые пропускают данные на уровне индексных блоков. - - -Параметр `GRANULARITY` определяет, сколько подиндексов векторного сходства создаётся. -Большие значения `GRANULARITY` означают меньшее количество, но более крупные подиндексы векторного сходства, вплоть до ситуации, когда столбец (или часть данных столбца) имеет только один подиндекс. -В этом случае подиндекс имеет «глобальное» представление обо всех строках столбца и может напрямую вернуть все гранулы столбца (части) с релевантными строками (таких гранул не более `LIMIT [N]`). -На втором шаге ClickHouse загрузит эти гранулы и определит действительно лучшие строки, выполнив расчёт расстояний полным перебором (brute-force) по всем строкам гранул. -При небольшом значении `GRANULARITY` каждый из подиндексов возвращает до `LIMIT N` гранул. -В результате требуется загрузить и дополнительно отфильтровать больше гранул. -Обратите внимание, что точность поиска в обоих случаях одинаково высока, различается только производительность обработки. -Обычно рекомендуется использовать большое значение `GRANULARITY` для индексов векторного сходства и переходить к меньшим значениям `GRANULARITY` только в случае проблем, например чрезмерного потребления памяти структурами векторного сходства. -Если `GRANULARITY` для индексов векторного сходства не задан, значение по умолчанию — 100 миллионов. - -#### Пример {#approximate-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -возвращает - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - -Дополнительные наборы данных, использующие приблизительный векторный поиск: - -* [LAION-400M](../../../getting-started/example-datasets/laion-400m-dataset) -* [LAION-5B](../../../getting-started/example-datasets/laion-5b-dataset) -* [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) -* [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) - -### Квантованный бит (QBit) {#approximate-nearest-neighbor-search-qbit} - - - -Один из распространённых подходов к ускорению точного векторного поиска — использовать менее точный [тип данных float](../../../sql-reference/data-types/float.md). -Например, если векторы хранятся как `Array(BFloat16)` вместо `Array(Float32)`, размер данных уменьшается вдвое, и ожидается, что время выполнения запросов сократится пропорционально. -Этот метод называется квантизацией. Хотя он ускоряет вычисления, точность результатов может снижаться, даже при полном переборе всех векторов. - -При традиционной квантизации точность теряется как во время поиска, так и при хранении данных. В приведённом выше примере мы бы хранили `BFloat16` вместо `Float32`, что означает невозможность выполнения более точного поиска в будущем, даже если это потребуется. Один из альтернативных подходов — хранить две копии данных: квантованную и с полной точностью. Хотя это работает, такой подход требует избыточного объёма хранилища. Рассмотрим сценарий, когда исходные данные имеют тип `Float64`, и нам нужно выполнять поиск с различной точностью (16-битной, 32-битной или полной 64-битной). В этом случае нам пришлось бы хранить три отдельные копии данных. - -ClickHouse предлагает тип данных Quantized Bit (`QBit`), который устраняет эти ограничения за счёт: - -1. Хранения исходных данных с полной точностью. -2. Возможности указания точности квантизации на этапе выполнения запроса. - - -Это достигается за счёт хранения данных в формате с побитовой группировкой (то есть все i-е биты всех векторов хранятся вместе), что позволяет выполнять чтение только с запрошенным уровнем точности. Вы получаете выигрыш в скорости за счёт сокращения объёма операций ввода-вывода и вычислений благодаря квантизации, при этом все исходные данные остаются доступными при необходимости. При выборе максимальной точности поиск становится точным. - -:::note -Тип данных `QBit` и связанные с ним функции вычисления расстояния в настоящее время являются экспериментальными. Чтобы их включить, выполните `SET allow_experimental_qbit_type = 1`. -Если вы столкнулись с проблемами, пожалуйста, создайте issue в [репозитории ClickHouse](https://github.com/clickhouse/clickhouse/issues). -::: - -Чтобы объявить столбец типа `QBit`, используйте следующий синтаксис: - -```sql -column_name QBit(element_type, dimension) -``` - -Где: - -* `element_type` – тип каждого элемента вектора. Поддерживаемые типы: `BFloat16`, `Float32` и `Float64` -* `dimension` – количество элементов в каждом векторе - -#### Создание таблицы `QBit` и добавление данных {#qbit-create} - -```sql -CREATE TABLE fruit_animal ( - word String, - vec QBit(Float64, 5) -) ENGINE = MergeTree -ORDER BY word; - -INSERT INTO fruit_animal VALUES - ('apple', [-0.99105519, 1.28887844, -0.43526649, -0.98520696, 0.66154391]), - ('banana', [-0.69372815, 0.25587061, -0.88226235, -2.54593015, 0.05300475]), - ('orange', [0.93338752, 2.06571317, -0.54612565, -1.51625717, 0.69775337]), - ('dog', [0.72138876, 1.55757105, 2.10953259, -0.33961248, -0.62217325]), - ('cat', [-0.56611276, 0.52267331, 1.27839863, -0.59809804, -1.26721048]), - ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); -``` - -#### Векторный поиск с `QBit` {#qbit-search} - -Найдём ближайших соседей к вектору, соответствующему слову «lemon», используя L2-расстояние. Третий параметр в функции расстояния задаёт точность в битах: более высокие значения обеспечивают большую точность, но требуют больше вычислительных ресурсов. - -Все доступные функции расстояния для `QBit` можно найти [здесь](../../../sql-reference/data-types/qbit.md#vector-search-functions). - -**Поиск с полной точностью (64-битной):** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 64) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬────────────distance─┐ -1. │ apple │ 0.14639757188169716 │ -2. │ banana │ 1.998961369007679 │ -3. │ orange │ 2.039041552613732 │ -4. │ cat │ 2.752802631487914 │ -5. │ horse │ 2.7555776805484813 │ -6. │ dog │ 3.382295083120104 │ - └────────┴─────────────────────┘ -``` - -**Поиск с уменьшенной точностью:** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 12) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬───────────distance─┐ -1. │ apple │ 0.757668703053566 │ -2. │ orange │ 1.5499475034938677 │ -3. │ banana │ 1.6168396735102937 │ -4. │ cat │ 2.429752230904804 │ -5. │ horse │ 2.524650475528617 │ -6. │ dog │ 3.17766975527459 │ - └────────┴────────────────────┘ -``` - - -Обратите внимание, что с 12-битной квантизацией мы получаем хорошее приближение расстояний при более быстром выполнении запросов. Относительный порядок в целом сохраняется: 'apple' по‑прежнему является ближайшим совпадением. - -:::note -В текущей реализации ускорение достигается за счёт уменьшения I/O, так как мы читаем меньше данных. Если исходные данные были «широкими», например `Float64`, выбор меньшей точности всё равно приведёт к вычислению расстояния по данным той же ширины — только с меньшей точностью. -::: - -#### Соображения по производительности {#qbit-performance} - -Производительность `QBit` повышается за счёт сокращения операций I/O, поскольку при использовании меньшей точности из хранилища нужно читать меньше данных. Кроме того, когда `QBit` содержит данные типа `Float32` и параметр точности равен 16 или меньше, появляется дополнительное преимущество за счёт уменьшения объёма вычислений. Параметр точности напрямую управляет компромиссом между точностью и скоростью: - -- **Более высокая точность** (ближе к исходной разрядности данных): более точные результаты, более медленные запросы -- **Низкая точность**: более быстрые запросы с приближёнными результатами, уменьшенное использование памяти - -### Ссылки {#references} - -Блоги: -- [Vector Search with ClickHouse - Part 1](https://clickhouse.com/blog/vector-search-clickhouse-p1) -- [Vector Search with ClickHouse - Part 2](https://clickhouse.com/blog/vector-search-clickhouse-p2) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md deleted file mode 100644 index d552544b967..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -description: 'CoalescingMergeTree наследует от движка MergeTree. Его ключевая особенность — возможность автоматически сохранять последнее ненулевое значение каждого столбца при слиянии частей.' -sidebar_label: 'CoalescingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/coalescingmergetree -title: 'Движок таблицы CoalescingMergeTree' -keywords: ['CoalescingMergeTree'] -show_related_blogs: true -doc_type: 'reference' ---- - -# Движок таблицы CoalescingMergeTree {#coalescingmergetree-table-engine} - -:::note Доступно начиная с версии 25.6 -Этот движок таблицы доступен начиная с версии 25.6 как в OSS, так и в Cloud. -::: - -Этот движок наследуется от [MergeTree](/engines/table-engines/mergetree-family/mergetree). Ключевое отличие заключается в том, как сливаются части данных: для таблиц `CoalescingMergeTree` ClickHouse заменяет все строки с одинаковым первичным ключом (или, точнее, с одинаковым [ключом сортировки](../../../engines/table-engines/mergetree-family/mergetree.md)) одной строкой, содержащей последние значения, отличные от NULL, для каждого столбца. - -Это позволяет выполнять upsert-операции на уровне отдельных столбцов, то есть вы можете обновлять только отдельные столбцы, а не целые строки. - -`CoalescingMergeTree` предназначен для использования с типами Nullable в неклю́чевых столбцах. Если столбцы не являются Nullable, поведение такое же, как у [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree). - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = CoalescingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -Описание параметров запроса см. в разделе [описание запроса](../../../sql-reference/statements/create/table.md). - -### Параметры движка CoalescingMergeTree {#parameters-of-coalescingmergetree} - -#### Столбцы {#columns} - -`columns` — кортеж имен столбцов, значения в которых будут объединены. Необязательный параметр. -Столбцы должны иметь числовой тип и не должны входить в ключ партиционирования или сортировки. - -Если `columns` не указан, ClickHouse объединяет значения во всех столбцах, которые не входят в ключ сортировки. - -### Части запроса {#query-clauses} - -При создании таблицы `CoalescingMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. - -
- Устаревший метод создания таблицы - - :::note - Не используйте этот метод в новых проектах и, по возможности, переведите старые проекты на метод, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] CoalescingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - Все параметры, за исключением `columns`, имеют то же значение, что и в `MergeTree`. - - * `columns` — кортеж имен столбцов, значения которых будут суммироваться. Необязательный параметр. Описание см. в тексте выше. -
- -## Пример использования {#usage-example} - -Рассмотрим следующую таблицу: - -```sql -CREATE TABLE test_table -( - key UInt64, - value_int Nullable(UInt32), - value_string Nullable(String), - value_date Nullable(Date) -) -ENGINE = CoalescingMergeTree() -ORDER BY key -``` - -Добавьте в неё данные: - -```sql -INSERT INTO test_table VALUES(1, NULL, NULL, '2025-01-01'), (2, 10, 'test', NULL); -INSERT INTO test_table VALUES(1, 42, 'win', '2025-02-01'); -INSERT INTO test_table(key, value_date) VALUES(2, '2025-02-01'); -``` - -Результат будет выглядеть так: - -```sql -SELECT * FROM test_table ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 1 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-01-01 │ -│ 2 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-02-01 │ -│ 2 │ 10 │ test │ ᴺᵁᴸᴸ │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -Рекомендуемый запрос для получения корректного итогового результата: - -```sql -SELECT * FROM test_table FINAL ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 2 │ 10 │ test │ 2025-02-01 │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -Использование модификатора `FINAL` указывает ClickHouse применять логику слияния на этапе выполнения запроса, гарантируя получение корректного, объединённого «последнего» значения для каждого столбца. Это самый безопасный и точный метод при выполнении запросов к таблице CoalescingMergeTree. - -:::note - -Подход с использованием `GROUP BY` может возвращать некорректные результаты, если части данных в таблице ещё не были полностью слиты. - -```sql -SELECT key, last_value(value_int), last_value(value_string), last_value(value_date) FROM test_table GROUP BY key; -- Не рекомендуется. -``` - -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md deleted file mode 100644 index 9604373a546..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ /dev/null @@ -1,380 +0,0 @@ ---- -description: 'Наследуется от MergeTree, но добавляет логику свёртки строк в процессе слияния.' -keywords: ['обновления', 'свёртка'] -sidebar_label: 'CollapsingMergeTree' -sidebar_position: 70 -slug: /engines/table-engines/mergetree-family/collapsingmergetree -title: 'Движок таблицы CollapsingMergeTree' -doc_type: 'guide' ---- - - - -# Движок таблицы CollapsingMergeTree {#collapsingmergetree-table-engine} - - - -## Описание {#description} - -Движок `CollapsingMergeTree` наследуется от [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) -и добавляет логику схлопывания строк в процессе слияния. -Движок таблицы `CollapsingMergeTree` асинхронно удаляет (схлопывает) -пары строк, если все поля в ключе сортировки (`ORDER BY`) совпадают, за исключением специального поля `Sign`, -которое может иметь значения `1` или `-1`. -Строки без пары с противоположным значением `Sign` сохраняются. - -Более подробную информацию см. в разделе [Collapsing](#table_engine-collapsingmergetree-collapsing) этого документа. - -:::note -Этот движок может значительно сократить объём хранимых данных, -что, как следствие, повышает эффективность запросов `SELECT`. -::: - - - -## Параметры {#parameters} - -Все параметры этого табличного движка, за исключением параметра `Sign`, -имеют то же значение, что и в [`MergeTree`](/engines/table-engines/mergetree-family/mergetree). - -- `Sign` — Имя столбца, указывающего тип строки: `1` — строка «состояния», `-1` — строка «отмены». Тип: [Int8](/sql-reference/data-types/int-uint). - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) -ENGINE = CollapsingMergeTree(Sign) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -
- Устаревший способ создания таблицы - - :::note - Приведённый ниже метод не рекомендуется использовать в новых проектах. - По возможности мы советуем обновить старые проекты и перейти на новый метод. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) - ENGINE [=] CollapsingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, Sign) - ``` - - `Sign` — имя столбца типа [Int8](/sql-reference/data-types/int-uint), в котором значение `1` обозначает строку состояния, а `-1` — строку отмены. -
- -* Описание параметров запроса см. в разделе [описание запроса](../../../sql-reference/statements/create/table.md). -* При создании таблицы `CollapsingMergeTree` используются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table), что и при создании таблицы `MergeTree`. - - -## Схлопывание {#table_engine-collapsingmergetree-collapsing} - -### Данные {#data} - -Рассмотрим ситуацию, когда необходимо сохранять постоянно меняющиеся данные для некоторого объекта. -Может показаться логичным иметь по одной строке на объект и обновлять её каждый раз при изменении данных, -однако операции обновления являются дорогостоящими и медленными для СУБД, потому что требуют перезаписи данных в хранилище. -Если нам нужно быстро записывать данные, выполнение большого количества обновлений — неприемлемый подход, -но мы всегда можем записывать изменения объекта последовательно. -Для этого мы используем специальный столбец `Sign`. - -* Если `Sign` = `1`, это означает, что строка является строкой "состояния": *строка, содержащая поля, представляющие текущее актуальное состояние*. -* Если `Sign` = `-1`, это означает, что строка является строкой "отмены": *строка, используемая для отмены состояния объекта с теми же атрибутами*. - -Например, мы хотим посчитать, сколько страниц пользователи просмотрели на каком‑то сайте и как долго они их посещали. -В некоторый момент времени мы записываем следующую строку с состоянием активности пользователя: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Позже мы фиксируем изменение активности пользователя и записываем его с помощью следующих двух строк: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Первая строка отменяет предыдущее состояние объекта (в данном случае представляющего пользователя). -Она должна копировать все поля сортировочного ключа для строки «canceled», за исключением `Sign`. -Вторая строка выше содержит текущее состояние. - -Поскольку нам нужно только последнее состояние активности пользователя, исходную строку «state» и строку -«cancel», которую мы вставили, можно удалить, как показано ниже, таким образом сворачивая недействительное (старое) состояние объекта: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -- старая строка состояния может быть удалена -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -- строка отмены может быть удалена -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -- новая строка состояния остаётся -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -`CollapsingMergeTree` выполняет именно такое *свёртывание* во время слияния частей данных. - -:::note -Причина, по которой для каждого изменения нужны две строки, -подробнее рассматривается в разделе [Algorithm](#table_engine-collapsingmergetree-collapsing-algorithm). -::: - -**Особенности такого подхода** - -1. Программа, которая записывает данные, должна запоминать состояние объекта, чтобы впоследствии иметь возможность его отменить. Строка "cancel" должна содержать копии полей ключа сортировки строки "state" и противоположный `Sign`. Это увеличивает первоначальный объём хранилища, но позволяет быстро записывать данные. -2. Длинные растущие массивы в столбцах снижают эффективность движка из-за увеличенной нагрузки на запись. Чем проще данные, тем выше эффективность. -3. Результаты `SELECT` в значительной степени зависят от согласованности истории изменений объекта. Будьте аккуратны при подготовке данных для вставки. При несогласованных данных можно получить непредсказуемые результаты. Например, отрицательные значения для неотрицательных метрик, таких как глубина сессии. - -### Algorithm {#table_engine-collapsingmergetree-collapsing-algorithm} - -Когда ClickHouse сливает [части](/concepts/glossary#parts) данных, -каждая группа последовательных строк с одинаковым ключом сортировки (`ORDER BY`) сокращается до не более чем двух строк: -строки "state" с `Sign` = `1` и строки "cancel" с `Sign` = `-1`. -Другими словами, в ClickHouse записи сворачиваются. - - -Для каждой результирующей части данных ClickHouse сохраняет: - -| | | -|--|-------------------------------------------------------------------------------------------------------------------------------------| -|1.| Первую строку с `cancel` и последнюю строку с `state`, если количество строк с `state` и `cancel` совпадает и последняя строка является строкой с `state`. | -|2.| Последнюю строку с `state`, если строк с `state` больше, чем строк с `cancel`. | -|3.| Первую строку с `cancel`, если строк с `cancel` больше, чем строк с `state`. | -|4.| Ни одной строки во всех остальных случаях. | - -Кроме того, когда строк с `state` как минимум на две больше, чем строк с `cancel`, -или строк с `cancel` как минимум на две больше, чем строк с `state`, слияние продолжается. -Однако ClickHouse рассматривает эту ситуацию как логическую ошибку и записывает её в журнал сервера. -Эта ошибка может возникнуть, если одни и те же данные вставляются более одного раза. -Таким образом, схлопывание не должно изменять результаты вычисления статистики. -Изменения постепенно схлопываются так, что в итоге остаётся только последнее состояние почти каждого объекта. - -Столбец `Sign` обязателен, потому что алгоритм слияния не гарантирует, -что все строки с одинаковым ключом сортировки окажутся в одной и той же результирующей части данных и даже на одном и том же физическом сервере. -ClickHouse обрабатывает запросы `SELECT` в несколько потоков и не может предсказать порядок строк в результате. - -Агрегация необходима, если нужно получить полностью схлопнутые данные из таблицы `CollapsingMergeTree`. -Чтобы завершить схлопывание, напишите запрос с предложением `GROUP BY` и агрегатными функциями, учитывающими знак. -Например, для вычисления количества используйте `sum(Sign)` вместо `count()`. -Для вычисления суммы используйте `sum(Sign * x)` вместе с `HAVING sum(Sign) > 0` вместо `sum(x)`, -как в [примере](#example-of-use) ниже. - -Агрегаты `count`, `sum` и `avg` можно вычислять таким образом. -Агрегат `uniq` можно вычислить, если объект имеет хотя бы одно несхлопнутое состояние. -Агрегаты `min` и `max` вычислить нельзя, -потому что `CollapsingMergeTree` не сохраняет историю схлопнутых состояний. - -:::note -Если вам нужно извлечь данные без агрегации -(например, чтобы проверить, присутствуют ли строки, чьи самые новые значения удовлетворяют определённым условиям), -вы можете использовать модификатор [`FINAL`](../../../sql-reference/statements/select/from.md#final-modifier) для предложения `FROM`. Он выполнит слияние данных перед возвратом результата. -Для `CollapsingMergeTree` возвращается только последняя строка состояния для каждого ключа. -::: - - - -## Примеры {#examples} - -### Пример использования {#example-of-use} - -Даны следующие исходные данные: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Давайте создадим таблицу `UAct` с движком `CollapsingMergeTree`: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -Теперь вставим немного данных: - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1),(4324182021466249494, 6, 185, 1) -``` - -Мы используем два запроса `INSERT`, чтобы создать две отдельные части данных. - -:::note -Если мы вставляем данные одним запросом, ClickHouse создаёт только одну часть данных и никогда не будет выполнять слияние. -::: - -Мы можем получить данные с помощью: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Давайте взглянем на возвращённые выше данные и посмотрим, произошло ли схлопывание... -С помощью двух запросов `INSERT` мы создали две части данных. -Запрос `SELECT` был выполнен в двух потоках, и мы получили строки в случайном порядке. -Однако схлопывание **не произошло**, потому что слияния частей данных ещё не было, -а ClickHouse объединяет части данных в фоновом режиме в непредсказуемый момент времени, который мы не можем предсказать. - -Поэтому нам нужна агрегация, -которую мы выполняем с помощью агрегатной функции [`sum`](/sql-reference/aggregate-functions/reference/sum) -и предложения [`HAVING`](/sql-reference/statements/select/having): - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration -FROM UAct -GROUP BY UserID -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -Если нам не нужна агрегация и мы хотим принудительно выполнить схлопывание, мы также можем использовать модификатор `FINAL` для секции `FROM`. - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -:::note -Этот способ выборки данных менее эффективен и не рекомендуется при работе с большими объемами сканируемых данных (миллионы строк). -::: - -### Пример другого подхода {#example-of-another-approach} - -Идея этого подхода состоит в том, что при слияниях учитываются только ключевые поля. -В строке "cancel" мы, соответственно, можем указать отрицательные значения, -которые при суммировании без использования столбца `Sign` компенсируют предыдущую версию строки. - -В этом примере мы воспользуемся приведенными ниже демонстрационными данными: - - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ -5 │ -146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Для этого подхода необходимо изменить типы данных столбцов `PageViews` и `Duration`, чтобы можно было хранить отрицательные значения. -Поэтому при создании таблицы `UAct` с использованием `collapsingMergeTree` мы изменяем типы этих столбцов с `UInt8` на `Int16`: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews Int16, - Duration Int16, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -Давайте протестируем этот подход, вставив данные в нашу таблицу. - -Однако для примеров или небольших таблиц это приемлемо: - -```sql -INSERT INTO UAct VALUES(4324182021466249494, 5, 146, 1); -INSERT INTO UAct VALUES(4324182021466249494, -5, -146, -1); -INSERT INTO UAct VALUES(4324182021466249494, 6, 185, 1); - -SELECT * FROM UAct FINAL; -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -```sql -SELECT - UserID, - sum(PageViews) AS PageViews, - sum(Duration) AS Duration -FROM UAct -GROUP BY UserID -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -```sql -SELECT COUNT() FROM UAct -``` - -```text -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -```sql -OPTIMIZE TABLE UAct FINAL; - -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md deleted file mode 100644 index 5464b210ba1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ /dev/null @@ -1,183 +0,0 @@ ---- -description: 'Узнайте, как добавить пользовательский ключ партиционирования для таблиц MergeTree.' -sidebar_label: 'Пользовательский ключ партиционирования' -sidebar_position: 30 -slug: /engines/table-engines/mergetree-family/custom-partitioning-key -title: 'Пользовательский ключ партиционирования' -doc_type: 'guide' ---- - -# Пользовательский ключ партиционирования {#custom-partitioning-key} - -:::note -В большинстве случаев ключ партиционирования не нужен, а в большинстве остальных случаев нет необходимости в ключе партиционирования с более мелкой детализацией, чем по месяцам, за исключением сценариев наблюдаемости (observability), где часто используется партиционирование по дням. - -Не следует использовать чрезмерно детализированное партиционирование. Не партиционируйте данные по идентификаторам или именам клиентов. Вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY. -::: - -Партиционирование доступно для [таблиц семейства MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md), включая [реплицируемые таблицы](../../../engines/table-engines/mergetree-family/replication.md) и [материализованные представления](/sql-reference/statements/create/view#materialized-view). - -Партиция — это логическое объединение записей в таблице по заданному критерию. Вы можете задать партицию по произвольному критерию, например по месяцу, по дню или по типу события. Каждая партиция хранится отдельно, что упрощает операции с этими данными. При доступе к данным ClickHouse использует минимально возможное подмножество партиций. Партиции повышают производительность запросов, содержащих ключ партиционирования, потому что ClickHouse будет фильтровать по этой партиции до выбора партиций (parts) и гранул внутри партиции. - -Партиция задаётся в предложении `PARTITION BY expr` при [создании таблицы](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table). Ключ партиционирования может быть любым выражением над столбцами таблицы. Например, чтобы указать партиционирование по месяцам, используйте выражение `toYYYYMM(date_column)`: - -```sql -CREATE TABLE visits -( - VisitDate Date, - Hour UInt8, - ClientID UUID -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(VisitDate) -ORDER BY Hour; -``` - -Ключ партиционирования может также представлять собой кортеж выражений (аналогично [первичному ключу](../../../engines/table-engines/mergetree-family/mergetree.md#primary-keys-and-indexes-in-queries)). Например: - -```sql -ENGINE = ReplicatedCollapsingMergeTree('/clickhouse/tables/name', 'replica1', Sign) -PARTITION BY (toMonday(StartDate), EventType) -ORDER BY (CounterID, StartDate, intHash32(UserID)); -``` - -В этом примере мы задаём секционирование по типам событий, произошедших в течение текущей недели. - -По умолчанию тип данных с плавающей запятой не поддерживается в качестве ключа секционирования. Чтобы его использовать, включите настройку [allow_floating_point_partition_key](../../../operations/settings/merge-tree-settings.md#allow_floating_point_partition_key). - -При вставке новых данных в таблицу эти данные сохраняются как отдельная часть (chunk), отсортированная по первичному ключу. Через 10–15 минут после вставки части одной и той же секции объединяются в единую часть. - -:::info -Слияние работает только для частей данных, у которых одинаковое значение выражения секционирования. Это означает, что **не следует делать секционирование слишком детализированным** (более чем примерно тысяча секций). В противном случае запрос `SELECT` будет выполняться неэффективно из‑за неоправданно большого количества файлов в файловой системе и открытых файловых дескрипторов. -::: - -Используйте таблицу [system.parts](../../../operations/system-tables/parts.md) для просмотра частей таблицы и её секций. Например, предположим, что у нас есть таблица `visits` с секционированием по месяцам. Выполним запрос `SELECT` к таблице `system.parts`: - -```sql -SELECT - partition, - name, - active -FROM system.parts -WHERE table = 'visits' -``` - -```text -┌─partition─┬─name──────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1_11 │ 1 │ -│ 201902 │ 201902_10_10_0_11 │ 1 │ -│ 201902 │ 201902_11_11_0_11 │ 1 │ -└───────────┴───────────────────┴────────┘ -``` - -Столбец `partition` содержит имена партиций. В этом примере есть две партиции: `201901` и `201902`. Вы можете использовать значение этого столбца, чтобы указать имя партиции в запросах [ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md). - -Столбец `name` содержит имена частей данных партиции. Вы можете использовать этот столбец, чтобы указать имя части в запросе [ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart). - -Разберём имя части: `201901_1_9_2_11`: - -* `201901` — имя партиции. -* `1` — минимальный номер блока данных. -* `9` — максимальный номер блока данных. -* `2` — уровень части (глубина дерева MergeTree, из которого она сформирована). -* `11` — версия мутации (если часть была изменена мутацией). - -:::info -Части таблиц старого типа имеют имена вида: `20190117_20190123_2_2_0` (минимальная дата — максимальная дата — минимальный номер блока — максимальный номер блока — уровень). -::: - -Столбец `active` показывает статус части. `1` — активна; `0` — неактивна. Неактивные части — это, например, исходные части, оставшиеся после слияния в более крупную часть. Повреждённые части данных также помечаются как неактивные. - -Как видно из примера, существует несколько отдельных частей одной и той же партиции (например, `201901_1_3_1` и `201901_1_9_2`). Это означает, что эти части ещё не были слиты. ClickHouse периодически сливает вставленные части данных, примерно через 15 минут после вставки. Кроме того, вы можете выполнить внеплановое слияние с помощью запроса [OPTIMIZE](../../../sql-reference/statements/optimize.md). Пример: - -```sql -OPTIMIZE TABLE visits PARTITION 201902; -``` - -```text -┌─partition─┬─name─────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1 │ 0 │ -│ 201902 │ 201902_4_11_2_11 │ 1 │ -│ 201902 │ 201902_10_10_0 │ 0 │ -│ 201902 │ 201902_11_11_0 │ 0 │ -└───────────┴──────────────────┴────────┘ -``` - -Неактивные части будут удалены примерно через 10 минут после слияния. - -Другой способ просмотреть набор частей и партиций — перейти в каталог таблицы: `/var/lib/clickhouse/data///`. Например: - -```bash -/var/lib/clickhouse/data/default/visits$ ls -l -total 40 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 201901_1_3_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201901_1_9_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_8_8_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_9_9_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_10_10_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_11_11_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:19 201902_4_11_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 12:09 201902_4_6_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached -``` - -Папки '201901_1_1_0', '201901_1_7_1' и так далее являются каталогами частей. Каждая часть относится к соответствующему разделу (partition) и содержит данные только за определённый месяц (в этом примере таблица разбита на разделы по месяцам). - -Каталог `detached` содержит части, которые были отсоединены от таблицы с помощью запроса [DETACH](/sql-reference/statements/detach). Повреждённые части также перемещаются в этот каталог вместо удаления. Сервер не использует части из каталога `detached`. Вы можете добавлять, удалять или изменять данные в этом каталоге в любое время — сервер не узнает об этом, пока вы не выполните запрос [ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart). - -Обратите внимание, что на рабочем сервере вы не можете вручную изменять набор частей или их данные в файловой системе, так как сервер не узнает об этом. Для нереплицируемых таблиц вы можете делать это, когда сервер остановлен, но это не рекомендуется. Для реплицируемых таблиц набор частей не может быть изменён ни при каких обстоятельствах. - -ClickHouse позволяет выполнять операции с партициями: удалять их, копировать из одной таблицы в другую или создавать резервную копию. См. полный список операций в разделе [Manipulations With Partitions and Parts](/sql-reference/statements/alter/partition). - -## Оптимизация Group By с использованием ключа партиционирования {#group-by-optimisation-using-partition-key} - -Для некоторых комбинаций ключа партиционирования таблицы и ключа группировки запроса (`GROUP BY`) агрегацию можно выполнять независимо для каждой партиции. -Тогда нам не придется в конце объединять частично агрегированные данные со всех потоков выполнения, -поскольку у нас есть гарантия, что каждое значение ключа группировки не может встречаться в рабочих наборах данных двух разных потоков. - -Типичный пример: - -```sql -CREATE TABLE session_log -( - UserID UInt64, - SessionID UUID -) -ENGINE = MergeTree -PARTITION BY sipHash64(UserID) % 16 -ORDER BY tuple(); - -SELECT - UserID, - COUNT() -FROM session_log -GROUP BY UserID; -``` - -:::note -Производительность такого запроса в значительной степени зависит от структуры таблицы. Поэтому эта оптимизация по умолчанию не включена. -::: - -Ключевые факторы высокой производительности: - -* количество секций, задействованных в запросе, должно быть достаточно большим (больше, чем `max_threads / 2`), иначе запрос будет недостаточно эффективно использовать ресурсы сервера -* секции не должны быть слишком маленькими, чтобы пакетная обработка не вырождалась в построчную обработку -* секции должны быть сопоставимы по размеру, чтобы все потоки выполняли примерно одинаковый объём работы - -:::info -Рекомендуется применить некоторую хеш-функцию к столбцам в выражении `partition by`, чтобы равномерно распределить данные между секциями. -::: - -Соответствующие настройки: - -* `allow_aggregate_partitions_independently` — управляет тем, включено ли использование оптимизации -* `force_aggregate_partitions_independently` — принудительно включает её использование, когда это допустимо с точки зрения корректности, но может быть отключено внутренней логикой, оценивающей её целесообразность -* `max_number_of_partitions_for_independent_aggregation` — жёсткое ограничение на максимальное количество секций, которое может иметь таблица diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md deleted file mode 100644 index 8a705ad5061..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ /dev/null @@ -1,272 +0,0 @@ ---- -description: 'Предназначен для прореживания и агрегации/усреднения (rollup) данных Graphite.' -sidebar_label: 'GraphiteMergeTree' -sidebar_position: 90 -slug: /engines/table-engines/mergetree-family/graphitemergetree -title: 'Движок таблицы GraphiteMergeTree' -doc_type: 'guide' ---- - -# Движок таблицы GraphiteMergeTree {#graphitemergetree-table-engine} - -Этот движок предназначен для прореживания и агрегирования/усреднения (rollup) данных [Graphite](http://graphite.readthedocs.io/en/latest/index.html). Он может быть полезен разработчикам, которые хотят использовать ClickHouse в качестве хранилища данных для Graphite. - -Вы можете использовать любой движок таблицы ClickHouse для хранения данных Graphite, если вам не нужен rollup, но если rollup необходим, используйте `GraphiteMergeTree`. Движок уменьшает объем хранимых данных и повышает эффективность выполнения запросов Graphite. - -Движок наследует свойства от [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md). - -## Создание таблицы {#creating-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - Path String, - Time DateTime, - Value Float64, - Version - ... -) ENGINE = GraphiteMergeTree(config_section) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -См. подробное описание запроса [CREATE TABLE](/sql-reference/statements/create/table). - -Таблица для данных Graphite должна иметь следующие столбцы: - -* Имя метрики (датчика Graphite). Тип данных: `String`. - -* Время измерения метрики. Тип данных: `DateTime`. - -* Значение метрики. Тип данных: `Float64`. - -* Версия метрики. Тип данных: любой числовой тип (ClickHouse сохраняет строки с наибольшей версией или последнюю записанную, если версии совпадают. Остальные строки удаляются при слиянии частей данных). - -Имена этих столбцов должны быть указаны в конфигурации rollup. - -**Параметры GraphiteMergeTree** - -* `config_section` — Имя раздела в файле конфигурации, где заданы правила rollup. - -**Части запроса** - -При создании таблицы `GraphiteMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table), что и при создании таблицы `MergeTree`. - -
- Устаревший способ создания таблицы - - :::note - Не используйте этот способ в новых проектах и, по возможности, переведите старые проекты на способ, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - EventDate Date, - Path String, - Time DateTime, - Value Float64, - Version - ... - ) ENGINE [=] GraphiteMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, config_section) - ``` - - Все параметры, кроме `config_section`, имеют то же значение, что и в `MergeTree`. - - * `config_section` — Имя раздела в файле конфигурации, где заданы правила rollup. -
- -## Конфигурация rollup {#rollup-configuration} - -Настройки rollup задаются параметром [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) в конфигурации сервера. Имя параметра может быть любым. Вы можете создать несколько конфигураций и использовать их для разных таблиц. - -Структура конфигурации rollup: - -required-columns -patterns - -### Обязательные столбцы {#required-columns} - -#### `path_column_name` {#path_column_name} - -`path_column_name` — имя столбца, в котором хранится имя метрики (датчик Graphite). Значение по умолчанию: `Path`. - -#### `time_column_name` {#time_column_name} - -`time_column_name` — имя столбца, в котором хранится время измерения метрики. Значение по умолчанию: `Time`. - -#### `value_column_name` {#value_column_name} - -`value_column_name` — имя столбца, в котором хранится значение метрики в момент времени, указанной в `time_column_name`. Значение по умолчанию: `Value`. - -#### `version_column_name` {#version_column_name} - -`version_column_name` — имя столбца, в котором хранится версия метрики. Значение по умолчанию: `Timestamp`. - -### Шаблоны {#patterns} - -Структура раздела `patterns`: - -```text -pattern - rule_type - regexp - function -pattern - rule_type - regexp - age + precision - ... -pattern - rule_type - regexp - function - age + precision - ... -pattern - ... -default - function - age + precision - ... -``` - -:::important -Паттерны должны быть строго упорядочены: - -1. Паттерны без `function` или `retention`. -2. Паттерны с обоими параметрами `function` и `retention`. -3. Паттерн `default`. - ::: - -При обработке строки ClickHouse проверяет правила в секциях `pattern`. Каждая из секций `pattern` (включая `default`) может содержать параметр `function` для агрегации, параметры `retention` или оба. Если имя метрики соответствует `regexp`, применяются правила из секции (или секций) `pattern`; в противном случае используются правила из секции `default`. - -Поля для секций `pattern` и `default`: - -* `rule_type` - тип правила. Применяется только к конкретным метрикам. Движок использует его для разделения обычных и помеченных (tagged) метрик. Необязательный параметр. Значение по умолчанию: `all`. - Он не нужен, когда производительность не критична или используется только один тип метрик, например обычные метрики. По умолчанию создаётся только один набор правил. В противном случае, если определён любой из специальных типов, создаются два разных набора. Один для обычных метрик (root.branch.leaf) и один для помеченных метрик (root.branch.leaf;tag1=value1). - Правила по умолчанию попадают в оба набора. - Допустимые значения: - * `all` (по умолчанию) - универсальное правило, используется, когда `rule_type` опущен. - * `plain` - правило для обычных метрик. Поле `regexp` обрабатывается как регулярное выражение. - * `tagged` - правило для помеченных метрик (метрики хранятся в БД в формате `someName?tag1=value1&tag2=value2&tag3=value3`). Регулярное выражение должно быть отсортировано по именам тегов, первый тег должен быть `__name__`, если он существует. Поле `regexp` обрабатывается как регулярное выражение. - * `tag_list` - правило для помеченных метрик, простой DSL для более удобного описания метрики в формате graphite `someName;tag1=value1;tag2=value2`, `someName` или `tag1=value1;tag2=value2`. Поле `regexp` преобразуется в правило `tagged`. Сортировка по именам тегов не нужна, она будет выполнена автоматически. Значение тега (но не имя) может задаваться как регулярное выражение, например `env=(dev|staging)`. -* `regexp` – шаблон для имени метрики (регулярное выражение или DSL). -* `age` – минимальный возраст данных в секундах. -* `precision` – точность определения возраста данных в секундах. Должно быть делителем 86400 (количество секунд в сутках). -* `function` – имя агрегирующей функции, применяемой к данным, возраст которых попадает в диапазон `[age, age + precision]`. Допустимые функции: min / max / any / avg. Среднее значение рассчитывается неточно, как среднее от средних значений. - -### Пример конфигурации без типов правил {#configuration-example} - -```xml - - Version - - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -### Пример конфигурации с типами правил {#configuration-typed-example} - -```xml - - Version - - plain - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - tagged - ^((.*)|.)min\? - min - - 0 - 5 - - - 86400 - 60 - - - - tagged - - min - - 0 - 5 - - - 86400 - 60 - - - - tag_list - someName;tag2=value2 - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -:::note -Агрегация (rollup) данных выполняется во время слияний. Обычно для старых партиций слияния не запускаются, поэтому для выполнения rollup необходимо инициировать внеплановое слияние с помощью команды [OPTIMIZE](../../../sql-reference/statements/optimize.md). Также можно использовать дополнительные инструменты, например [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer). -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md deleted file mode 100644 index 08f8de9d12d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: 'Документация по семейству движков MergeTree' -sidebar_label: 'Семейство MergeTree' -sidebar_position: 10 -slug: /engines/table-engines/mergetree-family/ -title: 'Семейство движков MergeTree' -doc_type: 'reference' ---- - -# Семейство движков MergeTree {#mergetree-engine-family} - -Табличные движки из семейства MergeTree являются ядром возможностей ClickHouse по хранению данных. Они предоставляют большинство функций для обеспечения отказоустойчивости и высокопроизводительного чтения данных: колоночное хранение, настраиваемое разбиение на партиции, разрежённый первичный индекс, вторичные индексы пропуска данных и т. д. - -Базовый табличный движок [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) можно считать движком по умолчанию для одиночных экземпляров ClickHouse, поскольку он универсален и практически применим для широкого спектра сценариев. - -Для промышленной эксплуатации предпочтительно использовать [ReplicatedMergeTree](../../../engines/table-engines/mergetree-family/replication.md), поскольку он добавляет отказоустойчивость ко всем возможностям обычного движка MergeTree. Дополнительным преимуществом является автоматическое дедуплицирование данных при ингестии, поэтому ПО может безопасно повторять попытку, если при вставке возникла сетевая ошибка. - -Все остальные движки семейства MergeTree добавляют дополнительную функциональность для некоторых специфических сценариев использования. Обычно это реализуется как дополнительная обработка данных в фоновом режиме. - -Основной недостаток движков MergeTree состоит в том, что они достаточно «тяжёлые». Поэтому типичный подход — не создавать их слишком много. Если вам нужно множество небольших таблиц, например для временных данных, рассмотрите использование [семейства движков Log](../../../engines/table-engines/log-family/index.md). - -{/* Таблица оглавления для этой страницы автоматически генерируется скриптом - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - из полей YAML front matter: slug, description, title. - - Если вы обнаружили ошибку, отредактируйте YAML front matter на соответствующих страницах. - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [MergeTree table engine](/engines/table-engines/mergetree-family/mergetree) | Движки таблиц семейства `MergeTree` спроектированы для высокой скорости приёма и обработки очень больших объёмов данных. | -| [Replicated* table engines](/engines/table-engines/mergetree-family/replication) | Обзор репликации данных с использованием семейства движков таблиц Replicated* в ClickHouse. | -| [Custom Partitioning Key](/engines/table-engines/mergetree-family/custom-partitioning-key) | Как добавить пользовательский ключ партиционирования в таблицы MergeTree. | -| [ReplacingMergeTree table engine](/engines/table-engines/mergetree-family/replacingmergetree) | Отличается от MergeTree тем, что удаляет дублирующиеся записи с одинаковым значением ключа сортировки (секция таблицы `ORDER BY`, а не `PRIMARY KEY`). | -| [CoalescingMergeTree table engine](/engines/table-engines/mergetree-family/coalescingmergetree) | CoalescingMergeTree наследует возможности движка MergeTree. Его ключевая особенность — автоматическое сохранение последнего ненулевого значения каждого столбца при слиянии частей данных. | -| [SummingMergeTree table engine](/engines/table-engines/mergetree-family/summingmergetree) | SummingMergeTree наследует возможности движка MergeTree. Его ключевая особенность — автоматическое суммирование числовых данных при слиянии частей данных. | -| [AggregatingMergeTree table engine](/engines/table-engines/mergetree-family/aggregatingmergetree) | Заменяет все строки с одинаковым первичным ключом (или, точнее, с одинаковым [ключом сортировки](../../../engines/table-engines/mergetree-family/mergetree.md)) одной строкой (в пределах одной части данных), которая хранит комбинацию состояний агрегатных функций. | -| [CollapsingMergeTree table engine](/engines/table-engines/mergetree-family/collapsingmergetree) | Наследует возможности MergeTree, но добавляет логику схлопывания строк в процессе слияния. | -| [VersionedCollapsingMergeTree table engine](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) | Обеспечивает быструю запись постоянно изменяющихся состояний объектов и фоновое удаление устаревших состояний объектов. | -| [GraphiteMergeTree table engine](/engines/table-engines/mergetree-family/graphitemergetree) | Предназначен для прореживания и агрегирования/усреднения (rollup) данных Graphite. | -| [Exact and Approximate Vector Search](/engines/table-engines/mergetree-family/annindexes) | Документация по точному и приближенному векторному поиску. | -| [Full-text Search using Text Indexes](/engines/table-engines/mergetree-family/invertedindexes) | Позволяет быстро находить искомые слова и фразы в тексте. | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md deleted file mode 100644 index 5516126eae6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ /dev/null @@ -1,816 +0,0 @@ ---- -description: 'Быстро находите нужные термины в тексте.' -keywords: ['полнотекстовый поиск', 'текстовый индекс', 'индекс', 'индексы'] -sidebar_label: 'Полнотекстовый поиск с использованием текстовых индексов' -slug: /engines/table-engines/mergetree-family/invertedindexes -title: 'Полнотекстовый поиск с использованием текстовых индексов' -doc_type: 'reference' ---- - -import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; - - -# Полнотекстовый поиск с использованием текстовых индексов {#full-text-search-using-text-indexes} - - - -Текстовые индексы в ClickHouse (также известные как ["обратные индексы"](https://en.wikipedia.org/wiki/Inverted_index)) обеспечивают быстрый полнотекстовый поиск по строковым данным. -Индекс сопоставляет каждый токен в столбце со строками, которые содержат этот токен. -Токены генерируются процессом, называемым токенизацией. -Например, ClickHouse по умолчанию токенизирует английское предложение "All cat like mice." как ["All", "cat", "like", "mice"] (обратите внимание, что завершающая точка игнорируется). -Доступны более продвинутые токенизаторы, например для данных журналов (логов). - - - -## Создание текстового индекса {#creating-a-text-index} - -Чтобы создать текстовый индекс, сначала включите соответствующий экспериментальный параметр: - -```sql -SET allow_experimental_full_text_index = true; -``` - -Текстовый индекс может быть определён для столбцов типов [String](/sql-reference/data-types/string.md), [FixedString](/sql-reference/data-types/fixedstring.md), [Array(String)](/sql-reference/data-types/array.md), [Array(FixedString)](/sql-reference/data-types/array.md) и [Map](/sql-reference/data-types/map.md) (через функции работы с map [mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) и [mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues)) с использованием следующего синтаксиса: - -```sql -CREATE TABLE tab -( - `key` UInt64, - `str` String, - INDEX text_idx(str) TYPE text( - -- Обязательные параметры: - tokenizer = splitByNonAlpha - | splitByString[(S)] - | ngrams[(N)] - | sparseGrams[(min_length[, max_length[, min_cutoff_length]])] - | array - -- Необязательные параметры: - [, preprocessor = expression(str)] - -- Дополнительные необязательные параметры: - [, dictionary_block_size = D] - [, dictionary_block_frontcoding_compression = B] - [, max_cardinality_for_embedded_postings = M] - [, bloom_filter_false_positive_rate = R] - ) [GRANULARITY 64] -) -ENGINE = MergeTree -ORDER BY key -``` - -**Аргумент tokenizer (обязательный)**. Аргумент `tokenizer` задаёт токенизатор: - -* `splitByNonAlpha` разбивает строки по небуквенно-цифровым ASCII-символам (см. также функцию [splitByNonAlpha](/sql-reference/functions/splitting-merging-functions.md/#splitByNonAlpha)). -* `splitByString(S)` разбивает строки по определённым пользовательским строкам-разделителям `S` (см. также функцию [splitByString](/sql-reference/functions/splitting-merging-functions.md/#splitByString)). - Разделители можно задать с помощью необязательного параметра, например, `tokenizer = splitByString([', ', '; ', '\n', '\\'])`. - Обратите внимание, что каждая строка может состоять из нескольких символов (в примере — `', '`). - Список разделителей по умолчанию, если он не указан явно (например, `tokenizer = splitByString`), содержит один пробел `[' ']`. -* `ngrams(N)` разбивает строки на `N`-граммы одинакового размера (см. также функцию [ngrams](/sql-reference/functions/splitting-merging-functions.md/#ngrams)). - Длину n-граммы можно задать с помощью необязательного целочисленного параметра от 2 до 8, например, `tokenizer = ngrams(3)`. - Размер n-граммы по умолчанию, если он не указан явно (например, `tokenizer = ngrams`), равен 3. -* `sparseGrams(min_length, max_length, min_cutoff_length)` разбивает строки на n-граммы переменной длины как минимум из `min_length` и не более чем из `max_length` (включительно) символов (см. также функцию [sparseGrams](/sql-reference/functions/string-functions#sparseGrams)). - Если не указано явно, значения `min_length` и `max_length` по умолчанию равны 3 и 100. - Если задан параметр `min_cutoff_length`, в индекс сохраняются только n-граммы с длиной не меньше `min_cutoff_length`. - По сравнению с `ngrams(N)` токенизатор `sparseGrams` генерирует n-граммы переменной длины, что позволяет более гибко представлять исходный текст. - Например, `tokenizer = sparseGrams(3, 5, 4)` генерирует из входной строки 3-, 4- и 5-граммы, но в индекс сохраняются только 4- и 5-граммы. -* `array` не выполняет токенизацию, т. е. каждое значение в строке является токеном (см. также функцию [array](/sql-reference/functions/array-functions.md/#array)). - -:::note -Токенизатор `splitByString` применяет строки-разделители слева направо. -Это может приводить к неоднозначной токенизации. -Например, строки-разделители `['%21', '%']` приведут к тому, что `%21abc` будет токенизировано как `['abc']`, тогда как при перестановке строк-разделителей `['%', '%21']` результатом будет `['21abc']`. -В большинстве случаев нужно, чтобы при сопоставлении более длинные разделители имели приоритет. -Обычно это можно обеспечить, передавая строки-разделители в порядке убывания длины. -Если строки-разделители образуют [префиксный код](https://en.wikipedia.org/wiki/Prefix_code), их можно передавать в произвольном порядке. -::: - - -:::warning -В настоящее время не рекомендуется создавать текстовые индексы для текста на незападных языках, например китайском. -Поддерживаемые в данный момент токенизаторы могут привести к чрезмерно большим размерам индексов и длительному времени выполнения запросов. -В будущем планируется добавить специализированные языково-специфичные токенизаторы, которые будут лучше обрабатывать такие случаи. -::: - -Чтобы проверить, как токенизаторы разбивают входную строку, можно использовать функцию ClickHouse [tokens](/sql-reference/functions/splitting-merging-functions.md/#tokens): - -Пример: - -```sql -SELECT tokens('abc def', 'ngrams', 3); -``` - -Результат: - -```result -['abc','bc ','c d',' de','def'] -``` - -**Аргумент препроцессора (необязательный)**. Аргумент `preprocessor` представляет собой выражение, которое применяется к входной строке перед токенизацией. - -Типичные варианты использования аргумента препроцессора включают: - -1. Приведение к нижнему или верхнему регистру для обеспечения регистронезависимого сопоставления, например [lower](/sql-reference/functions/string-functions.md/#lower), [lowerUTF8](/sql-reference/functions/string-functions.md/#lowerUTF8) — см. первый пример ниже. -2. Нормализация UTF-8, например [normalizeUTF8NFC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFC), [normalizeUTF8NFD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFD), [normalizeUTF8NFKC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKC), [normalizeUTF8NFKD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKD), [toValidUTF8](/sql-reference/functions/string-functions.md/#toValidUTF8). -3. Удаление или преобразование нежелательных символов или подстрок, например [extractTextFromHTML](/sql-reference/functions/string-functions.md/#extractTextFromHTML), [substring](/sql-reference/functions/string-functions.md/#substring), [idnaEncode](/sql-reference/functions/string-functions.md/#idnaEncode). - -Выражение препроцессора должно преобразовывать входное значение типа [String](/sql-reference/data-types/string.md) или [FixedString](/sql-reference/data-types/fixedstring.md) в значение того же типа. - -Примеры: - -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(col))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = substringIndex(col, '\n', 1))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(extractTextFromHTML(col))` - -Кроме того, выражение препроцессора должно ссылаться только на столбец, для которого определён текстовый индекс. -Использование недетерминированных функций не допускается. - -Функции [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken), [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) и [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) используют препроцессор для предварительного преобразования поискового термина перед его токенизацией. - -Например: - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(str) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(str)) -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, 'Foo'); -``` - -эквивалентно: - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(lower(str)) TYPE text(tokenizer = 'splitByNonAlpha') -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, lower('Foo')); -``` - -**Другие аргументы (необязательные)**. Текстовые индексы в ClickHouse реализованы как [вторичные индексы](/engines/table-engines/mergetree-family/mergetree.md/#skip-index-types). -Однако, в отличие от других индексов с пропуском, текстовые индексы имеют значение GRANULARITY по умолчанию, равное 64. -Это значение выбрано эмпирически и обеспечивает хороший баланс между скоростью и размером индекса для большинства случаев использования. -Опытные пользователи могут указать другую гранулярность индекса (мы не рекомендуем этого делать). - -
- -Необязательные расширенные параметры - -Значения по умолчанию следующих расширенных параметров будут хорошо работать практически во всех ситуациях. -Мы не рекомендуем их изменять. - -Необязательный параметр `dictionary_block_size` (по умолчанию: 128) задаёт размер блоков словаря в строках. - -Необязательный параметр `dictionary_block_frontcoding_compression` (по умолчанию: 1) указывает, используют ли блоки словаря фронтальное кодирование в качестве сжатия. - -Необязательный параметр `max_cardinality_for_embedded_postings` (по умолчанию: 16) задаёт порог кардинальности, ниже которого списки постингов должны быть встроены в блоки словаря. - - -Необязательный параметр `bloom_filter_false_positive_rate` (по умолчанию: 0.1) задаёт вероятность ложноположительных срабатываний фильтра Блума словаря. - -
- -Текстовые индексы можно добавлять в столбец или удалять из него после создания таблицы: - -```sql -ALTER TABLE tab DROP INDEX text_idx; -ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); -``` - - -## Использование текстового индекса {#using-a-text-index} - -Использовать текстовый индекс в запросах SELECT просто, так как стандартные функции строкового поиска автоматически используют индекс. -Если индекс отсутствует, приведённые ниже функции строкового поиска будут выполнять медленный полный перебор по всем данным. - -### Поддерживаемые функции {#functions-support} - -Текстовый индекс может быть использован, если текстовые функции применяются в условии `WHERE` запроса SELECT: - -```sql -SELECT [...] -FROM [...] -WHERE функция_поиска_по_строке(столбец_с_текстовым_индексом) -``` - -#### `=` и `!=` {#functions-example-equals-notequals} - -`=` ([equals](/sql-reference/functions/comparison-functions.md/#equals)) и `!=` ([notEquals](/sql-reference/functions/comparison-functions.md/#notEquals)) совпадают с заданным поисковым выражением целиком. - -Пример: - -```sql -SELECT * from tab WHERE str = 'Привет'; -``` - -Индекс по тексту поддерживает операторы `=` и `!=`, однако поиск по равенству и неравенству имеет смысл только с токенизатором `array` (он приводит к тому, что индекс хранит значения целых строк). - -#### `IN` и `NOT IN` {#functions-example-in-notin} - -`IN` ([in](/sql-reference/functions/in-functions)) и `NOT IN` ([notIn](/sql-reference/functions/in-functions)) похожи на функции `equals` и `notEquals`, но они выбирают все (`IN`) или ни одного (`NOT IN`) из искомых значений. - -Пример: - -```sql -SELECT * from tab WHERE str IN ('Привет', 'Мир'); -``` - -Те же ограничения, что и для `=` и `!=`, действуют и здесь: `IN` и `NOT IN` имеют смысл только в сочетании с токенизатором `array`. - -#### `LIKE`, `NOT LIKE` и `match` {#functions-example-like-notlike-match} - -:::note -В настоящее время эти функции используют текстовый индекс для фильтрации только в том случае, если токенизатор индекса — `splitByNonAlpha` или `ngrams`. -::: - -Чтобы использовать `LIKE` ([like](/sql-reference/functions/string-search-functions.md/#like)), `NOT LIKE` ([notLike](/sql-reference/functions/string-search-functions.md/#notLike)) и функцию [match](/sql-reference/functions/string-search-functions.md/#match) с текстовыми индексами, ClickHouse должен иметь возможность извлечь полные токены из поискового термина. - -Пример: - -```sql -SELECT count() FROM tab WHERE comment LIKE 'support%'; -``` - -`support` в примере может соответствовать `support`, `supports`, `supporting` и т. д. -Такой запрос является запросом по подстроке, и его нельзя ускорить с помощью текстового индекса. - -Чтобы использовать текстовый индекс для запросов с LIKE, шаблон LIKE нужно изменить следующим образом: - -```sql -SELECT count() FROM tab WHERE comment LIKE ' support %'; -- или `% support %` -``` - -Пробелы слева и справа от `support` гарантируют, что термин может быть извлечён как токен. - -#### `startsWith` и `endsWith` {#functions-example-startswith-endswith} - -Аналогично оператору `LIKE`, функции [startsWith](/sql-reference/functions/string-functions.md/#startsWith) и [endsWith](/sql-reference/functions/string-functions.md/#endsWith) могут использовать текстовый индекс только в том случае, если из поискового выражения могут быть извлечены полные токены. - -Пример: - -```sql -SELECT count() FROM tab WHERE startsWith(comment, 'поддержка clickhouse'); -``` - -В этом примере только `clickhouse` считается токеном. -`support` не является токеном, так как он может соответствовать `support`, `supports`, `supporting` и т.д. - -Чтобы найти все строки, которые начинаются с `clickhouse supports`, завершите шаблон поиска пробелом в конце: - -```sql -startsWith(comment, 'clickhouse supports ')` -``` - -Аналогично, `endsWith` следует использовать с пробелом в начале строки: - -```sql -SELECT count() FROM tab WHERE endsWith(comment, ' olap engine'); -``` - -#### `hasToken` и `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} - -Функции [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) и [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) сопоставляют значение с одним заданным токеном. - -В отличие от упомянутых выше функций, они не токенизируют искомое значение (предполагается, что на вход подаётся один токен). - -Пример: - -```sql -SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); -``` - -Функции `hasToken` и `hasTokenOrNull` являются наиболее эффективными при использовании с индексом `text`. - -#### `hasAnyTokens` и `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} - - -Функции [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) и [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) сопоставляют строку соответственно с одним или со всеми указанными токенами. - -Обе функции принимают поисковые токены либо в виде строки, которая будет разбита на токены с использованием того же токенизатора, что и для столбца, по которому построен индекс, либо в виде массива уже подготовленных токенов, к которым перед поиском токенизация применяться не будет. -См. документацию по функциям для получения дополнительной информации. - -Пример: - -```sql --- Поисковые токены передаются как строковый аргумент -SELECT count() FROM tab WHERE hasAnyTokens(comment, 'clickhouse olap'); -SELECT count() FROM tab WHERE hasAllTokens(comment, 'clickhouse olap'); - --- Поисковые токены передаются как Array(String) -SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); -SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); -``` - -#### `has` {#functions-example-has} - -Функция для работы с массивами [`has`](/sql-reference/functions/array-functions#has) проверяет наличие отдельного токена в массиве строк. - -Пример: - -```sql -SELECT count() FROM tab WHERE has(array, 'clickhouse'); -``` - -#### `mapContains` {#functions-example-mapcontains} - -Функция [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) (псевдоним `mapContainsKey`) проверяет, содержится ли заданный токен среди ключей отображения. - -Пример: - -```sql -SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); --- OR -SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); -``` - -#### `operator[]` {#functions-example-access-operator} - -Оператор доступа [`operator[]`](/sql-reference/operators#access-operators) можно использовать с текстовым индексом для фильтрации по ключам и значениям. - -Пример: - -```sql -SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; -``` - -Рассмотрим следующие примеры использования столбцов типов `Array(T)` и `Map(K, V)` с текстовым индексом. - -### Примеры для столбцов `Array` и `Map` с текстовыми индексами {#text-index-array-and-map-examples} - -#### Индексирование столбцов `Array(String)` {#text-index-example-array} - -Представьте платформу для ведения блогов, где авторы классифицируют свои записи с помощью ключевых слов. -Мы хотим, чтобы пользователи могли находить похожие материалы, выполняя поиск или переходя по темам. - -Рассмотрим следующее определение таблицы: - -```sql -CREATE TABLE posts ( - post_id UInt64, - title String, - content String, - keywords Array(String) -) -ENGINE = MergeTree -ORDER BY (post_id); -``` - -Без текстового индекса поиск постов с определённым ключевым словом (например, `clickhouse`) требует сканирования всех постов: - -```sql -SELECT count() FROM posts WHERE has(keywords, 'clickhouse'); -- медленное полное сканирование таблицы - проверяет каждое ключевое слово в каждой записи -``` - -По мере роста платформы выполнение запроса становится всё более медленным, поскольку ему приходится просматривать каждый массив `keywords` в каждой строке. -Чтобы решить эту проблему с производительностью, мы создаём текстовый индекс для столбца `keywords`: - -```sql -ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitByNonAlpha); -ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- Не забудьте материализовать индекс для существующих данных -``` - -#### Индексация столбцов типа Map {#text-index-example-map} - -Во многих сценариях систем наблюдаемости сообщения логов разбиваются на отдельные «компоненты» и сохраняются в виде соответствующих типов данных, например тип `DateTime` для временной метки, `Enum` для уровня логирования и т. д. -Поля метрик оптимально хранить в виде пар «ключ–значение». -Операционным командам нужно эффективно искать по логам для отладки, расследования инцидентов информационной безопасности и мониторинга. - -Рассмотрим следующую таблицу логов: - -```sql -CREATE TABLE logs ( - id UInt64, - timestamp DateTime, - message String, - attributes Map(String, String) -) -ENGINE = MergeTree -ORDER BY (timestamp); -``` - -Без текстового индекса поиск по данным типа [Map](/sql-reference/data-types/map.md) требует полного сканирования таблицы: - -```sql --- Находит все логи с данными об ограничении скорости: -SELECT count() FROM logs WHERE has(mapKeys(attributes), 'rate_limit'); -- медленное полное сканирование таблицы - --- Находит все логи с конкретного IP-адреса: -SELECT count() FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- медленное полное сканирование таблицы -``` - -По мере роста объёма логов эти запросы начинают работать медленнее. - -Решение — создать текстовый индекс для ключей и значений типа [Map](/sql-reference/data-types/map.md). -Используйте [mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) для создания текстового индекса, когда нужно находить логи по именам полей или типам атрибутов: - - -```sql -ALTER TABLE logs ADD INDEX attributes_keys_idx mapKeys(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_keys_idx; -``` - -Используйте [mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues), чтобы создать текстовый индекс, когда вам нужно выполнять поиск по самому содержимому атрибутов: - -```sql -ALTER TABLE logs ADD INDEX attributes_vals_idx mapValues(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_vals_idx; -``` - -Примеры запросов: - -```sql --- Найти все запросы с ограничением скорости: -SELECT * FROM logs WHERE mapContainsKey(attributes, 'rate_limit'); -- быстро - --- Найти все логи с конкретного IP-адреса: -SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- быстро -``` - - -## Оптимизация производительности {#performance-tuning} - -### Прямое чтение {#direct-read} - -Некоторые типы текстовых запросов можно значительно ускорить с помощью оптимизации, называемой «прямое чтение». -Более точно, эту оптимизацию можно применить, если в запросе SELECT текстовый столбец *не* выбирается. - -Пример: - -```sql -SELECT column_a, column_b, ... -- не: column_with_text_index -FROM [...] -WHERE string_search_function(column_with_text_index) -``` - -Оптимизация прямого чтения в ClickHouse обрабатывает запрос, используя исключительно текстовый индекс (т. е. обращения к текстовому индексу) без доступа к исходному текстовому столбцу. -Обращения к текстовому индексу читают относительно небольшой объём данных и поэтому значительно быстрее, чем обычные skip-индексы в ClickHouse (которые выполняют поиск по skip-индексу, а затем загружают и фильтруют отобранные гранулы). - -Прямое чтение управляется двумя настройками: - -* Настройка [query_plan_direct_read_from_text_index](../../../operations/settings/settings#query_plan_direct_read_from_text_index), которая определяет, включено ли прямое чтение в целом. -* Настройка [use_skip_indexes_on_data_read](../../../operations/settings/settings#use_skip_indexes_on_data_read), которая является ещё одним обязательным условием для прямого чтения. Обратите внимание, что в базах данных ClickHouse с [compatibility](../../../operations/settings/settings#compatibility) < 25.10 `use_skip_indexes_on_data_read` отключена, поэтому вам нужно либо повысить значение настройки compatibility, либо явно выполнить `SET use_skip_indexes_on_data_read = 1`. - -Кроме того, текстовый индекс должен быть полностью материализован, чтобы использовать прямое чтение (для этого используйте `ALTER TABLE ... MATERIALIZE INDEX`). - -**Поддерживаемые функции** -Оптимизация прямого чтения поддерживает функции `hasToken`, `hasAllTokens` и `hasAnyTokens`. -Эти функции также могут комбинироваться операторами AND, OR и NOT. -Условие WHERE также может содержать дополнительные фильтры, не являющиеся функциями полнотекстового поиска (как для текстовых, так и для других столбцов) — в этом случае оптимизация прямого чтения всё равно будет использоваться, но менее эффективно (она применяется только к поддерживаемым функциям текстового поиска). - -Чтобы убедиться, что запрос использует прямое чтение, выполните его с `EXPLAIN PLAN actions = 1`. -В качестве примера — запрос с отключённым прямым чтением - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 0, -- отключить прямое чтение - use_skip_indexes_on_data_read = 1; -``` - -возвращает - -```text -[...] -Filter ((WHERE + Change column names to column identifiers)) -Filter column: hasToken(__table1.col, 'some_token'_String) (removed) -Actions: INPUT : 0 -> col String : 0 - COLUMN Const(String) -> 'some_token'_String String : 1 - FUNCTION hasToken(col :: 0, 'some_token'_String :: 1) -> hasToken(__table1.col, 'some_token'_String) UInt8 : 2 -[...] -``` - -тогда как тот же запрос, выполненный при `query_plan_direct_read_from_text_index = 1` - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 1, -- включить прямое чтение - use_skip_indexes_on_data_read = 1; -``` - -возвращает - -```text -[...] -Expression (перед GROUP BY) -Позиции: - Фильтр - Столбец фильтра: __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 (удалён) - Actions: INPUT :: 0 -> __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 UInt8 : 0 -[...] -``` - -Вывод второго EXPLAIN PLAN содержит виртуальный столбец `__text_index___`. -Если этот столбец присутствует, используется прямое чтение. - -### Кэширование {#caching} - -Доступны различные кэши для буферизации частей текстового индекса в памяти (см. раздел [Сведения о реализации](#implementation)): -В настоящее время доступны кэши для десериализованных блоков словаря, заголовков и списков постингов текстового индекса для снижения объема операций ввода-вывода (I/O). -Их можно включить с помощью настроек [use_text_index_dictionary_cache](/operations/settings/settings#use_text_index_dictionary_cache), [use_text_index_header_cache](/operations/settings/settings#use_text_index_header_cache) и [use_text_index_postings_cache](/operations/settings/settings#use_text_index_postings_cache). -По умолчанию все кэши отключены. - -Для настройки кэшей используйте следующие параметры сервера. - -#### Настройки кэша блоков словаря {#caching-dictionary} - - -| Параметр | Описание | -|----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| -| [text_index_dictionary_block_cache_policy](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_policy) | Название политики кэша блоков словаря текстового индекса. | -| [text_index_dictionary_block_cache_size](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size) | Максимальный размер кэша в байтах. | -| [text_index_dictionary_block_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_max_entries) | Максимальное количество десериализованных блоков словаря в кэше. | -| [text_index_dictionary_block_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size_ratio) | Размер защищённой очереди в кэше блоков словаря текстового индекса относительно общего размера кэша. | - -#### Настройки кэша заголовков {#caching-header} - -| Параметр | Описание | -|--------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| -| [text_index_header_cache_policy](/operations/server-configuration-parameters/settings#text_index_header_cache_policy) | Название политики кэша заголовков текстового индекса. | -| [text_index_header_cache_size](/operations/server-configuration-parameters/settings#text_index_header_cache_size) | Максимальный размер кэша в байтах. | -| [text_index_header_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_header_cache_max_entries) | Максимальное количество десериализованных заголовков в кэше. | -| [text_index_header_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_header_cache_size_ratio) | Размер защищённой очереди в кэше заголовков текстового индекса относительно общего размера кэша. | - -#### Настройки кэша списков вхождений {#caching-posting-lists} - -| Параметр | Описание | -|---------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------| -| [text_index_postings_cache_policy](/operations/server-configuration-parameters/settings#text_index_postings_cache_policy) | Название политики кэша списков вхождений текстового индекса. | -| [text_index_postings_cache_size](/operations/server-configuration-parameters/settings#text_index_postings_cache_size) | Максимальный размер кэша в байтах. | -| [text_index_postings_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_postings_cache_max_entries) | Максимальное количество десериализованных списков вхождений в кэше. | -| [text_index_postings_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_postings_cache_size_ratio) | Размер защищённой очереди в кэше списков вхождений текстового индекса относительно общего размера кэша. | - - - -## Детали реализации {#implementation} - -Каждый текстовый индекс состоит из двух (абстрактных) структур данных: -- словаря, который отображает каждый токен на список вхождений, и -- набора списков вхождений, каждый из которых представляет собой набор номеров строк. - -Поскольку текстовый индекс является skip-индексом, эти структуры данных логически существуют для каждой гранулы индекса. - -Во время создания индекса создаются три файла (на каждую part): - -**Файл блоков словаря (.dct)** - -Токены в грануле индекса сортируются и сохраняются в блоках словаря по 128 токенов в каждом (размер блока настраивается параметром `dictionary_block_size`). -Файл блоков словаря (.dct) содержит все блоки словаря всех гранул индекса в одной part. - -**Файл гранул индекса (.idx)** - -Файл гранул индекса содержит для каждого блока словаря первый токен блока, его относительное смещение в файле блоков словаря и фильтр Блума для всех токенов в блоке. -Эта разреженная структура индекса аналогична [разреженному индексу первичного ключа](https://clickhouse.com/docs/guides/best-practices/sparse-primary-indexes) в ClickHouse. -Фильтр Блума позволяет заранее отбрасывать блоки словаря, если разыскиваемый токен не содержится в блоке словаря. - -**Файл списков вхождений (.pst)** - -Списки вхождений для всех токенов располагаются последовательно в файле списков вхождений. -Чтобы экономить место и при этом обеспечивать быстрые операции пересечения и объединения, списки вхождений хранятся в виде [roaring bitmaps](https://roaringbitmap.org/). -Если кардинальность списка вхождений меньше 16 (настраивается параметром `max_cardinality_for_embedded_postings`), он встраивается в словарь. - - - -## Пример: датасет Hacker News {#hacker-news-dataset} - -Рассмотрим, как текстовые индексы улучшают производительность на большом текстовом наборе данных. -Мы будем использовать 28,7 млн строк комментариев с популярного сайта Hacker News. -Вот таблица без текстового индекса: - -```sql -CREATE TABLE hackernews ( - id UInt64, - deleted UInt8, - type String, - author String, - timestamp DateTime, - comment String, - dead UInt8, - parent UInt64, - poll UInt64, - children Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32 -) -ENGINE = MergeTree -ORDER BY (type, author); -``` - -28,7 млн строк хранятся в Parquet-файле в S3 — давайте загрузим их в таблицу `hackernews`: - -```sql -INSERT INTO hackernews - SELECT * FROM s3Cluster( - 'default', - 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/hackernews/hacknernews.parquet', - 'Parquet', - ' - id UInt64, - deleted UInt8, - type String, - by String, - time DateTime, - text String, - dead UInt8, - parent UInt64, - poll UInt64, - kids Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32'); -``` - -Используем `ALTER TABLE`, создадим текстовый индекс по столбцу comment, затем материализуем его: - -```sql --- Добавить индекс -ALTER TABLE hackernews ADD INDEX comment_idx(comment) TYPE text(tokenizer = splitByNonAlpha); - --- Материализовать индекс для существующих данных -ALTER TABLE hackernews MATERIALIZE INDEX comment_idx SETTINGS mutations_sync = 2; -``` - -Теперь давайте выполним запросы с использованием функций `hasToken`, `hasAnyTokens` и `hasAllTokens`. -Следующие примеры покажут существенную разницу в производительности между стандартным сканированием индекса и оптимизацией прямого чтения. - -### 1. Использование `hasToken` {#using-hasToken} - -`hasToken` проверяет, содержит ли текст указанный одиночный токен. -Мы будем искать регистрозависимый токен `ClickHouse`. - -**Прямое чтение отключено (стандартное сканирование)** -По умолчанию ClickHouse использует skip-индекс для фильтрации гранул, а затем читает данные столбца для этих гранул. -Мы можем смоделировать это поведение, отключив прямое чтение. - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -1 строка в наборе. Прошло: 0,362 сек. Обработано 24,90 млн строк, 9,51 ГБ -``` - -**Включено прямое чтение (Fast index read)** -Теперь запустим тот же запрос с включённым режимом прямого чтения (это значение по умолчанию). - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -Обработана 1 строка. Затрачено: 0.008 сек. Обработано 3.15 млн строк, 3.15 МБ -``` - -Запрос прямого чтения более чем в 45 раз быстрее (0,362 с против 0,008 с) и обрабатывает значительно меньше данных (9,51 ГБ против 3,15 МБ), поскольку читает только из индекса. - -### 2. Использование `hasAnyTokens` {#using-hasAnyTokens} - -`hasAnyTokens` проверяет, содержит ли текст хотя бы один из заданных токенов. -Мы будем искать комментарии, содержащие либо «love», либо «ClickHouse». - -**Прямое чтение отключено (стандартное сканирование)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 408426 │ -└─────────┘ - -1 row in set. Elapsed: 1.329 sec. Processed 28.74 million rows, 9.72 GB -``` - -**Включено прямое чтение (быстрое чтение индекса)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 408426 │ -└─────────┘ -``` - - -1 строка в наборе. Прошло: 0.015 сек. Обработано 27.99 млн строк, 27.99 МБ - -```` -Ускорение ещё более впечатляющее для этого распространённого поиска с условием "ИЛИ". -Запрос выполняется почти в 89 раз быстрее (1.329 с против 0.015 с) благодаря отсутствию полного сканирования столбца. - -### 3. Использование `hasAllTokens` {#using-hasAllTokens} - -`hasAllTokens` проверяет, содержит ли текст все указанные токены. -Выполним поиск комментариев, содержащих одновременно 'love' и 'ClickHouse'. - -**Прямое чтение отключено (стандартное сканирование)** -Даже при отключённом прямом чтении стандартный skip-индекс остаётся эффективным. -Он сокращает 28,7 млн строк до 147,46 тыс. строк, но всё равно должен прочитать 57,03 МБ из столбца. - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -1 row in set. Elapsed: 0.184 sec. Processed 147.46 thousand rows, 57.03 MB -```` - -**Прямое чтение (Fast index read) включено** -Прямое чтение обрабатывает запрос, работая только с данными индекса и считывая всего 147,46 КБ. - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -1 строка в наборе. Затрачено: 0.007 сек. Обработано 147.46 тыс. строк, 147.46 КБ -``` - -Для этого поиска с оператором "AND" оптимизация прямого чтения более чем в 26 раз быстрее (0.184 s против 0.007 s), чем стандартное сканирование с использованием skip-индекса. - -### 4. Составной поиск: OR, AND, NOT, ... {#compound-search} - -Оптимизация прямого чтения также применима к составным булевым выражениям. -Здесь мы выполним поиск без учета регистра для 'ClickHouse' OR 'clickhouse'. - -**Прямое чтение отключено (стандартное сканирование)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -1 строка в результате. Время выполнения: 0.450 сек. Обработано 25.87 млн строк, 9.58 ГБ -``` - -**Прямое чтение включено (быстрый доступ к индексу)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -Обработана 1 строка. Затрачено: 0.013 сек. Обработано 25.87 млн строк, 51.73 МБ -``` - -Благодаря объединению результатов из индекса прямой запрос на чтение выполняется в 34 раза быстрее (0,450 s против 0,013 s) и позволяет избежать чтения 9,58 GB данных столбцов. -В этом конкретном случае предпочтительнее использовать более эффективный синтаксис `hasAnyTokens(comment, ['ClickHouse', 'clickhouse'])`. - - -## Связанные материалы {#related-content} - -- Статья в блоге: [Introducing Inverted Indices in ClickHouse](https://clickhouse.com/blog/clickhouse-search-with-inverted-indices) -- Статья в блоге: [Inside ClickHouse full-text search: fast, native, and columnar](https://clickhouse.com/blog/clickhouse-full-text-search) -- Видео: [Full-Text Indices: Design and Experiments](https://www.youtube.com/watch?v=O_MnyUkrIq8) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md deleted file mode 100644 index 5b32f798d56..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md +++ /dev/null @@ -1,1163 +0,0 @@ ---- -description: 'Семейство табличных движков `MergeTree` предназначено для высоких скоростей приёма данных и работы с очень большими объёмами данных.' -sidebar_label: 'MergeTree' -sidebar_position: 11 -slug: /engines/table-engines/mergetree-family/mergetree -title: 'Табличный движок MergeTree' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# Движок таблицы MergeTree {#mergetree-table-engine} - -Движок `MergeTree` и другие движки семейства `MergeTree` (например, `ReplacingMergeTree`, `AggregatingMergeTree`) являются наиболее часто используемыми и наиболее надёжными движками таблиц в ClickHouse. - -Движки таблиц семейства `MergeTree` спроектированы для высокой скорости приёма данных и работы с очень большими объёмами. -Операции вставки создают части таблицы, которые затем объединяются фоновым процессом с другими частями таблицы. - -Основные особенности движков таблиц семейства `MergeTree`. - -* Первичный ключ таблицы определяет порядок сортировки внутри каждой части таблицы (кластерный индекс). При этом первичный ключ указывает не на отдельные строки, а на блоки по 8192 строки, которые называются гранулами. Это делает первичные ключи для очень больших наборов данных достаточно компактными, чтобы оставаться загруженными в основную память, при этом обеспечивая быстрый доступ к данным на диске. - -* Таблицы могут быть разбиты на разделы (партиции) с использованием произвольного выражения секционирования. Исключение разделов (partition pruning) гарантирует, что такие разделы пропускаются при чтении, когда это допускает запрос. - -* Данные могут реплицироваться между несколькими узлами кластера для обеспечения высокой доступности, отказоустойчивости и обновлений без простоя. См. раздел [Репликация данных](/engines/table-engines/mergetree-family/replication.md). - -* Движки таблиц `MergeTree` поддерживают различные виды статистики и методы выборочного чтения (sampling), помогающие оптимизировать запросы. - -:::note -Несмотря на похожее название, движок [Merge](/engines/table-engines/special/merge) отличается от движков `*MergeTree`. -::: - -## Создание таблиц {#table_engine-mergetree-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr1] [COMMENT ...] [CODEC(codec1)] [STATISTICS(stat1)] [TTL expr1] [PRIMARY KEY] [SETTINGS (name = value, ...)], - name2 [type2] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr2] [COMMENT ...] [CODEC(codec2)] [STATISTICS(stat2)] [TTL expr2] [PRIMARY KEY] [SETTINGS (name = value, ...)], - ... - INDEX index_name1 expr1 TYPE type1(...) [GRANULARITY value1], - INDEX index_name2 expr2 TYPE type2(...) [GRANULARITY value2], - ... - PROJECTION projection_name_1 (SELECT [GROUP BY] [ORDER BY]), - PROJECTION projection_name_2 (SELECT [GROUP BY] [ORDER BY]) -) ENGINE = MergeTree() -ORDER BY expr -[PARTITION BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[TTL expr - [DELETE|TO DISK 'xxx'|TO VOLUME 'xxx' [, ...] ] - [WHERE conditions] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] ] -[SETTINGS name = value, ...] -``` - -Подробное описание параметров см. в описании оператора [CREATE TABLE](/sql-reference/statements/create/table.md) - -### Части запроса {#mergetree-query-clauses} - -#### ENGINE {#engine} - -`ENGINE` — имя и параметры движка таблицы. `ENGINE = MergeTree()`. Движок таблицы `MergeTree` не имеет параметров. - -#### ORDER BY {#order_by} - -`ORDER BY` — ключ сортировки. - -Кортеж имён столбцов или произвольных выражений. Пример: `ORDER BY (CounterID + 1, EventDate)`. - -Если первичный ключ не определён (то есть `PRIMARY KEY` не был указан), ClickHouse использует ключ сортировки в качестве первичного ключа. - -Если сортировка не требуется, можно использовать синтаксис `ORDER BY tuple()`. -Либо, если включена настройка `create_table_empty_primary_key_by_default`, `ORDER BY ()` неявно добавляется к операторам `CREATE TABLE`. См. раздел [Выбор первичного ключа](#selecting-a-primary-key). - -#### PARTITION BY {#partition-by} - -`PARTITION BY` — [ключ партиционирования](/engines/table-engines/mergetree-family/custom-partitioning-key.md). Необязателен. В большинстве случаев ключ партиционирования не нужен, а если и требуется партиционирование, как правило, нет необходимости использовать ключ с более высокой детализацией, чем по месяцам. Партиционирование не ускоряет выполнение запросов (в отличие от выражения ORDER BY). Никогда не используйте слишком мелкое партиционирование. Не разбивайте данные по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). - -Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — это столбец с датой типа [Date](/sql-reference/data-types/date.md). Имена партиций в этом случае имеют формат `"YYYYMM"`. - -#### PRIMARY KEY {#primary-key} - -`PRIMARY KEY` — первичный ключ, если он [отличается от сортировочного ключа](#choosing-a-primary-key-that-differs-from-the-sorting-key). Необязательный параметр. - -Указание сортировочного ключа (с помощью клаузы `ORDER BY`) неявно задаёт первичный ключ. -Обычно нет необходимости указывать первичный ключ дополнительно к сортировочному ключу. - -#### SAMPLE BY {#sample-by} - -`SAMPLE BY` — выражение для семплирования (sampling expression). Необязательное выражение. - -Если указано, оно должно входить в первичный ключ. -Результат этого выражения должен быть беззнаковым целым числом. - -Пример: `SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`. - -#### TTL {#ttl} - -`TTL` — список правил, которые задают срок хранения строк и логику автоматического перемещения частей [между дисками и томами](#table_engine-mergetree-multiple-volumes). Необязательный параметр. - -Выражение должно возвращать `Date` или `DateTime`, например, `TTL date + INTERVAL 1 DAY`. - -Тип правила `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` определяет действие, которое выполняется с частью, если выражение удовлетворяется (достигает текущего времени): удаление истёкших строк, перемещение части (если выражение выполняется для всех строк в части) на указанный диск (`TO DISK 'xxx'`) или на том (`TO VOLUME 'xxx'`), либо агрегация значений в истёкших строках. Тип правила по умолчанию — удаление (`DELETE`). Можно задать список из нескольких правил, но не более одного правила `DELETE`. - -Подробнее см. [TTL для столбцов и таблиц](#table_engine-mergetree-ttl) - -#### ПАРАМЕТРЫ {#settings} - -См. [настройки MergeTree](../../../operations/settings/merge-tree-settings.md). - -**Пример настройки параметра sections** - -```sql -ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity=8192 -``` - -В этом примере мы задаём секционирование по месяцам. - -Мы также задаём выражение для выборочного чтения данных в виде хэша по ID пользователя. Это позволяет псевдослучайно распределить данные в таблице для каждого `CounterID` и `EventDate`. Если вы укажете предложение [SAMPLE](/sql-reference/statements/select/sample) при выборке данных, ClickHouse вернёт равномерную псевдослучайную выборку данных для подмножества пользователей. - -Параметр `index_granularity` можно опустить, так как 8192 — это значение по умолчанию. - -
- Устаревший метод создания таблицы - - :::note - Не используйте этот метод в новых проектах. По возможности переведите старые проекты на метод, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - **Параметры MergeTree()** - - * `date-column` — Имя столбца типа [Date](/sql-reference/data-types/date.md). ClickHouse автоматически создаёт партиции по месяцам на основе этого столбца. Имена партиций имеют формат `"YYYYMM"`. - * `sampling_expression` — Выражение для выборочного чтения данных. - * `(primary, key)` — Первичный ключ. Тип: [Tuple()](/sql-reference/data-types/tuple.md) - * `index_granularity` — Гранулярность индекса. Количество строк данных между «метками» индекса. Значение 8192 подходит для большинства задач. - - **Пример** - - ```sql - MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) - ``` - - Движок `MergeTree` настраивается так же, как в примере выше для основного метода конфигурации движка. -
- -## Хранение данных {#mergetree-data-storage} - -Таблица состоит из частей данных, отсортированных по первичному ключу. - -При вставке данных в таблицу создаются отдельные части данных, и каждая из них лексикографически сортируется по первичному ключу. Например, если первичный ключ — `(CounterID, Date)`, данные в части сортируются по `CounterID`, а внутри каждого `CounterID` упорядочиваются по `Date`. - -Данные, принадлежащие разным партициям, разделяются на отдельные части. В фоновом режиме ClickHouse сливает части данных для более эффективного хранения. Части, принадлежащие разным партициям, не сливаются. Механизм слияния не гарантирует, что все строки с одинаковым первичным ключом окажутся в одной и той же части. - -Части данных могут храниться в форматах `Wide` или `Compact`. В формате `Wide` каждый столбец хранится в отдельном файле в файловой системе, в формате `Compact` все столбцы хранятся в одном файле. Формат `Compact` может использоваться для повышения производительности при небольших и частых вставках. - -Формат хранения данных контролируется параметрами движка таблицы `min_bytes_for_wide_part` и `min_rows_for_wide_part`. Если количество байт или строк в части данных меньше соответствующего значения параметра, часть хранится в формате `Compact`. В противном случае она хранится в формате `Wide`. Если ни один из этих параметров не задан, части данных хранятся в формате `Wide`. - -Каждая часть данных логически разделена на гранулы. Гранула — это наименьший неделимый набор данных, который ClickHouse читает при выборке. ClickHouse не разбивает строки или значения, поэтому каждая гранула всегда содержит целое число строк. Первая строка гранулы помечается значением первичного ключа для этой строки. Для каждой части данных ClickHouse создает файл индекса, в котором хранятся эти метки. Для каждого столбца, независимо от того, входит он в первичный ключ или нет, ClickHouse также хранит те же метки. Эти метки позволяют находить данные непосредственно в файлах столбцов. - -Размер гранулы ограничивается параметрами движка таблицы `index_granularity` и `index_granularity_bytes`. Число строк в грануле находится в диапазоне `[1, index_granularity]` и зависит от размера строк. Размер гранулы может превышать `index_granularity_bytes`, если размер одной строки больше значения этого параметра. В этом случае размер гранулы равен размеру строки. - -## Первичные ключи и индексы в запросах {#primary-keys-and-indexes-in-queries} - -Рассмотрим в качестве примера первичный ключ `(CounterID, Date)`. В этом случае сортировку и индекс можно представить следующим образом: - -```text -Все данные: [---------------------------------------------] -CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] -Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] -Метки: | | | | | | | | | | | - a,1 a,2 a,3 b,3 e,2 e,3 g,1 h,2 i,1 i,3 l,3 -Номера меток: 0 1 2 3 4 5 6 7 8 9 10 -``` - -Если в запросе к данным указано: - -* `CounterID in ('a', 'h')`, сервер читает данные в диапазонах меток `[0, 3)` и `[6, 8)`. -* `CounterID IN ('a', 'h') AND Date = 3`, сервер читает данные в диапазонах меток `[1, 3)` и `[7, 8)`. -* `Date = 3`, сервер читает данные в диапазоне меток `[1, 10]`. - -Приведённые выше примеры показывают, что использование индекса всегда эффективнее, чем полное сканирование. - -Разреженный индекс допускает чтение лишних данных. При чтении одного диапазона первичного ключа в каждом блоке данных может быть прочитано до `index_granularity * 2` дополнительных строк. - -Разреженные индексы позволяют работать с очень большим числом строк в таблице, потому что в большинстве случаев такие индексы помещаются в оперативную память. - -ClickHouse не требует уникального первичного ключа. Вы можете вставлять несколько строк с одинаковым первичным ключом. - -Вы можете использовать выражения типа `Nullable` в выражениях `PRIMARY KEY` и `ORDER BY`, но это настоятельно не рекомендуется. Чтобы включить эту возможность, активируйте настройку [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key). Принцип [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) применяется к значениям `NULL` в выражении `ORDER BY`. - -### Выбор первичного ключа {#selecting-a-primary-key} - -Количество столбцов в первичном ключе явно не ограничено. В зависимости от структуры данных вы можете включать больше или меньше столбцов в первичный ключ. Это может: - -* Повысить производительность индекса. - - Если первичный ключ — `(a, b)`, то добавление дополнительного столбца `c` улучшит производительность, если выполняются следующие условия: - - * Есть запросы с условием по столбцу `c`. - * Длинные диапазоны данных (в несколько раз длиннее, чем `index_granularity`) с одинаковыми значениями для `(a, b)` встречаются часто. Другими словами, добавление еще одного столбца позволяет пропускать достаточно длинные диапазоны данных. - -* Улучшить сжатие данных. - - ClickHouse сортирует данные по первичному ключу, поэтому чем выше упорядоченность, тем лучше сжатие. - -* Обеспечить дополнительную логику при слиянии частей данных в движках [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) и [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md). - - В этом случае имеет смысл указать *ключ сортировки*, отличающийся от первичного ключа. - -Длинный первичный ключ негативно влияет на производительность операций вставки и потребление памяти, но дополнительные столбцы в первичном ключе не влияют на производительность ClickHouse при выполнении `SELECT`‑запросов. - -Вы можете создать таблицу без первичного ключа, используя синтаксис `ORDER BY tuple()`. В этом случае ClickHouse хранит данные в порядке вставки. Если вы хотите сохранить порядок данных при вставке через запросы `INSERT ... SELECT`, установите [max_insert_threads = 1](/operations/settings/settings#max_insert_threads). - -Чтобы выбирать данные в исходном порядке, используйте [однопоточные](/operations/settings/settings.md/#max_threads) `SELECT`‑запросы. - -### Выбор первичного ключа, отличного от ключа сортировки {#choosing-a-primary-key-that-differs-from-the-sorting-key} - -Можно задать первичный ключ (выражение со значениями, которые записываются в файл индекса для каждой метки), отличающийся от ключа сортировки (выражение для сортировки строк в частях данных). В этом случае кортеж выражений первичного ключа должен быть префиксом кортежа выражений ключа сортировки. - -Эта возможность полезна при использовании движков таблиц [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) и -[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md). В типичном случае при использовании этих движков таблица содержит два типа столбцов: *измерения* и *показатели*. Типичные запросы агрегируют значения столбцов-показателей с произвольным `GROUP BY` и фильтрацией по измерениям. Поскольку SummingMergeTree и AggregatingMergeTree агрегируют строки с одинаковым значением ключа сортировки, естественно включить в него все измерения. В результате выражение ключа состоит из длинного списка столбцов, и этот список необходимо часто обновлять при добавлении новых измерений. - -В этом случае имеет смысл оставить в первичном ключе только несколько столбцов, которые обеспечат эффективное диапазонное сканирование, а оставшиеся столбцы-измерения добавить в кортеж ключа сортировки. - -[ALTER](/sql-reference/statements/alter/index.md) ключа сортировки — это лёгкая операция, потому что когда новый столбец одновременно добавляется в таблицу и в ключ сортировки, существующие части данных не нужно изменять. Поскольку старый ключ сортировки является префиксом нового ключа сортировки и в только что добавленном столбце ещё нет данных, данные на момент изменения таблицы отсортированы как по старому, так и по новому ключам сортировки. - -### Использование индексов и партиций в запросах {#use-of-indexes-and-partitions-in-queries} - -Для запросов `SELECT` ClickHouse анализирует, может ли быть использован индекс. Индекс может быть использован, если предложение `WHERE/PREWHERE` содержит выражение (как один из элементов конъюнкции или целиком), представляющее собой операцию сравнения на равенство или неравенство, или если оно содержит `IN` или `LIKE` с фиксированным префиксом по столбцам или выражениям, входящим в первичный ключ или ключ партиционирования, или по определённым частично повторяющимся функциям этих столбцов, или логические комбинации этих выражений. - -Таким образом, можно быстро выполнять запросы по одному или нескольким диапазонам первичного ключа. В этом примере запросы будут выполняться быстро при выборке по конкретному тегу отслеживания, по конкретному тегу и диапазону дат, по конкретному тегу и дате, по нескольким тегам с диапазоном дат и так далее. - -Рассмотрим движок, настроенный следующим образом: - -```sql -ENGINE MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate) -SETTINGS index_granularity=8192 -``` - -В таком случае в запросах: - -```sql -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND CounterID = 34 - -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND (CounterID = 34 OR CounterID = 42) - -SELECT count() FROM table -WHERE ((EventDate >= toDate('2014-01-01') -AND EventDate <= toDate('2014-01-31')) OR EventDate = toDate('2014-05-01')) -AND CounterID IN (101500, 731962, 160656) -AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) -``` - -ClickHouse будет использовать индекс по первичному ключу для отсечения нерелевантных данных и ежемесячный ключ партиционирования для отсечения партиций, попадающих в неподходящие диапазоны дат. - -Приведённые выше запросы показывают, что индекс используется даже для сложных выражений. Чтение из таблицы организовано так, что использование индекса не может быть медленнее полного сканирования. - -В приведённом ниже примере индекс использоваться не будет. - -```sql -SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' -``` - -Чтобы проверить, может ли ClickHouse использовать индекс при выполнении запроса, используйте настройки [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) и [force_primary_key](/operations/settings/settings#force_primary_key). - -Ключ партиционирования по месяцам позволяет читать только те блоки данных, которые содержат даты из нужного диапазона. В этом случае блок данных может содержать данные за множество дат (вплоть до целого месяца). Внутри блока данные отсортированы по первичному ключу, который может не содержать дату в качестве первого столбца. Из-за этого использование запроса только с условием по дате, без указания префикса первичного ключа, приведёт к чтению большего объёма данных, чем при выборке за одну дату. - -### Использование индекса для частично-монотонных первичных ключей {#use-of-index-for-partially-monotonic-primary-keys} - -Рассмотрим, например, дни месяца. Они образуют [монотонную последовательность](https://en.wikipedia.org/wiki/Monotonic_function) в пределах одного месяца, но не являются монотонными на более длительных промежутках времени. Это частично-монотонная последовательность. Если пользователь создаёт таблицу с частично-монотонным первичным ключом, ClickHouse создаёт разреженный индекс как обычно. Когда пользователь выбирает данные из такой таблицы, ClickHouse анализирует условия запроса. Если пользователь хочет получить данные между двумя метками индекса и обе эти метки попадают в один месяц, ClickHouse может использовать индекс в этом конкретном случае, потому что он может вычислить расстояние между параметрами запроса и метками индекса. - -ClickHouse не может использовать индекс, если значения первичного ключа в заданном в параметрах запроса диапазоне не образуют монотонную последовательность. В этом случае ClickHouse использует полное сканирование. - -ClickHouse применяет эту логику не только к последовательностям дней месяца, но и к любому первичному ключу, который представляет частично-монотонную последовательность. - -### Индексы пропуска данных {#table_engine-mergetree-data_skipping-indexes} - -Объявление индекса указывается в разделе `COLUMNS` оператора `CREATE`. - -```sql -INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] -``` - -Для таблиц из семейства `*MergeTree` можно задать индексы пропуска данных (data skipping indices). - -Эти индексы агрегируют некоторую информацию об указанном выражении по блокам, которые состоят из гранул размера `granularity_value` (размер гранулы задаётся с помощью настройки `index_granularity` в движке таблицы). Затем эти агрегаты используются в запросах `SELECT` для уменьшения объёма данных, считываемых с диска, за счёт пропуска крупных блоков данных, в которых условие секции `WHERE` не может быть выполнено. - -Секцию `GRANULARITY` можно опустить, значение `granularity_value` по умолчанию равно 1. - -**Пример** - -```sql -CREATE TABLE table_name -( - u64 UInt64, - i32 Int32, - s String, - ... - INDEX idx1 u64 TYPE bloom_filter GRANULARITY 3, - INDEX idx2 u64 * i32 TYPE minmax GRANULARITY 3, - INDEX idx3 u64 * length(s) TYPE set(1000) GRANULARITY 4 -) ENGINE = MergeTree() -... -``` - -ClickHouse может использовать индексы из примера, чтобы сократить объём данных, считываемых с диска, в следующих запросах: - -```sql -SELECT count() FROM table WHERE u64 == 10; -SELECT count() FROM table WHERE u64 * i32 >= 1234 -SELECT count() FROM table WHERE u64 * length(s) == 1234 -``` - -Индексы пропуска данных также могут создаваться для составных столбцов: - -```sql --- для столбцов типа Map: -INDEX map_key_index mapKeys(map_column) TYPE bloom_filter -INDEX map_value_index mapValues(map_column) TYPE bloom_filter - --- для столбцов типа Tuple: -INDEX tuple_1_index tuple_column.1 TYPE bloom_filter -INDEX tuple_2_index tuple_column.2 TYPE bloom_filter - --- для столбцов типа Nested: -INDEX nested_1_index col.nested_col1 TYPE bloom_filter -INDEX nested_2_index col.nested_col2 TYPE bloom_filter -``` - -### Типы пропускающих индексов {#skip-index-types} - -Движок таблицы `MergeTree` поддерживает следующие типы пропускающих индексов. -Подробнее об использовании пропускающих индексов для оптимизации производительности -см. в разделе ["Понимание пропускающих индексов данных в ClickHouse"](/optimize/skipping-indexes). - -* индекс [`MinMax`](#minmax) -* индекс [`Set`](#set) -* индекс [`bloom_filter`](#bloom-filter) -* индекс [`ngrambf_v1`](#n-gram-bloom-filter) -* индекс [`tokenbf_v1`](#token-bloom-filter) - -#### Индекс MinMax {#minmax} - -Для каждой гранулы индекса сохраняются минимальные и максимальные значения выражения. -(Если выражение имеет тип `tuple`, сохраняются минимальные и максимальные значения для каждого элемента кортежа.) - -```text title="Syntax" -minmax -``` - -#### Set {#set} - -Для каждой гранулы индекса сохраняется не более `max_rows` уникальных значений указанного выражения. -`max_rows = 0` означает «хранить все уникальные значения». - -```text title="Syntax" -set(max_rows) -``` - -#### Фильтр Блума {#bloom-filter} - -Для каждой гранулы индекса хранится [фильтр Блума](https://en.wikipedia.org/wiki/Bloom_filter) по указанным столбцам. - -```text title="Syntax" -bloom_filter([false_positive_rate]) -``` - -Параметр `false_positive_rate` может принимать значение от 0 до 1 (по умолчанию: `0.025`) и задаёт вероятность положительного срабатывания (что увеличивает объём считываемых данных). - -Поддерживаются следующие типы данных: - -* `(U)Int*` -* `Float*` -* `Enum` -* `Date` -* `DateTime` -* `String` -* `FixedString` -* `Array` -* `LowCardinality` -* `Nullable` -* `UUID` -* `Map` - -:::note Тип данных Map: указание создания индекса по ключам или значениям -Для типа данных `Map` клиент может указать, должен ли индекс создаваться по ключам или по значениям, с помощью функций [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) или [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues). -::: - -#### N-граммовый Bloom-фильтр {#n-gram-bloom-filter} - -Для каждой гранулы индекса хранится [Bloom-фильтр](https://en.wikipedia.org/wiki/Bloom_filter) по [n-граммам](https://en.wikipedia.org/wiki/N-gram) указанных столбцов. - -```text title="Syntax" -ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -| Parameter | Description | -| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `n` | размер n-граммы | -| `size_of_bloom_filter_in_bytes` | Размер фильтра Блума в байтах. Здесь можно использовать большое значение, например `256` или `512`, поскольку оно хорошо сжимается. | -| `number_of_hash_functions` | Количество хеш-функций, используемых в фильтре Блума. | -| `random_seed` | Начальное значение (seed) для хеш-функций фильтра Блума. | - -Этот индекс работает только со следующими типами данных: - -* [`String`](/sql-reference/data-types/string.md) -* [`FixedString`](/sql-reference/data-types/fixedstring.md) -* [`Map`](/sql-reference/data-types/map.md) - -Чтобы оценить параметры `ngrambf_v1`, вы можете использовать следующие [пользовательские функции (UDF)](/sql-reference/statements/create/function.md). - -```sql title="UDFs for ngrambf_v1" -CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] -AS -(total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); - -CREATE FUNCTION bfEstimateBmSize [ON CLUSTER cluster] -AS -(total_number_of_all_grams, probability_of_false_positives) -> ceil((total_number_of_all_grams * log(probability_of_false_positives)) / log(1 / pow(2, log(2)))); - -CREATE FUNCTION bfEstimateFalsePositive [ON CLUSTER cluster] -AS -(total_number_of_all_grams, number_of_hash_functions, size_of_bloom_filter_in_bytes) -> pow(1 - exp(-number_of_hash_functions/ (size_of_bloom_filter_in_bytes / total_number_of_all_grams)), number_of_hash_functions); - -CREATE FUNCTION bfEstimateGramNumber [ON CLUSTER cluster] -AS -(number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) -``` - -Чтобы использовать эти функции, необходимо указать не менее двух параметров: - -* `total_number_of_all_grams` -* `probability_of_false_positives` - -Например, в грануле есть `4300` n-грамм, и вы ожидаете, что вероятность ложных срабатываний будет меньше `0.0001`. -Остальные параметры можно затем оценить, выполнив следующие запросы: - -```sql ---- оценить количество битов в фильтре -SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; - -┌─size_of_bloom_filter_in_bytes─┐ -│ 10304 │ -└───────────────────────────────┘ - ---- оценить количество хеш-функций -SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_hash_functions - -┌─number_of_hash_functions─┐ -│ 13 │ -└──────────────────────────┘ -``` - -Разумеется, вы также можете использовать эти функции для оценки параметров и в других условиях. -Приведённые выше функции соответствуют калькулятору фильтра Блума, доступному по адресу [здесь](https://hur.st/bloomfilter). - -#### Фильтр Блума по токенам {#token-bloom-filter} - -Фильтр Блума по токенам аналогичен `ngrambf_v1`, но вместо n-грамм хранит токены (последовательности, разделённые небуквенно-цифровыми символами). - -```text title="Syntax" -tokenbf_v1(size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -#### Разрежённый n-граммный фильтр Блума {#sparse-grams-bloom-filter} - -Разрежённый n-граммный фильтр Блума аналогичен `ngrambf_v1`, но использует [токены разрежённых n-грамм](/sql-reference/functions/string-functions.md/#sparseGrams) вместо n-грамм. - -```text title="Syntax" -sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -### Текстовый индекс {#text} - -Поддерживает полнотекстовый поиск; подробности см. [здесь](invertedindexes.md). - -#### Сходство векторов {#vector-similarity} - -Поддерживает приближённый поиск ближайших соседей, подробнее см. [здесь](annindexes.md). - -### Поддержка функций {#functions-support} - -Условия в предложении `WHERE` содержат вызовы функций, которые работают со столбцами. Если столбец является частью индекса, ClickHouse пытается использовать этот индекс при вычислении этих функций. ClickHouse поддерживает различные подмножества функций для работы с индексами. - -Индексы типа `set` могут использоваться всеми функциями. Остальные типы индексов поддерживаются следующим образом: - -| Функция (оператор) / Индекс | первичный ключ | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | текст | -| ------------------------------------------------------------------------------------------------------------------------- | -------------- | ------ | -------------- | -------------- | ---------------- | ---------------- | ----- | -| [равно (=, ==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [меньше (`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greater (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [greaterOrEquals (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenCaseInsensitive (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | - -Функции с константным аргументом, значение которого меньше размера n-граммы, не могут использоваться индексом `ngrambf_v1` для оптимизации запросов. - -(*) Чтобы `hasTokenCaseInsensitive` и `hasTokenCaseInsensitiveOrNull` были эффективны, индекс `tokenbf_v1` должен быть создан по данным в нижнем регистре, например: `INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`. - -:::note -У фильтров Блума возможны ложноположительные срабатывания, поэтому индексы `ngrambf_v1`, `tokenbf_v1`, `sparse_grams` и `bloom_filter` не могут использоваться для оптимизации запросов, в которых ожидается, что результат функции будет ложным. - -Например: - -* Может быть оптимизировано: - * `s LIKE '%test%'` - * `NOT s NOT LIKE '%test%'` - * `s = 1` - * `NOT s != 1` - * `startsWith(s, 'test')` -* Не может быть оптимизировано: - * `NOT s LIKE '%test%'` - * `s NOT LIKE '%test%'` - * `NOT s = 1` - * `s != 1` - * `NOT startsWith(s, 'test')` - ::: - -## Проекции {#projections} - -Проекции похожи на [materialized views](/sql-reference/statements/create/view), но определяются на уровне частей таблицы (parts). Они обеспечивают гарантии согласованности, а также автоматическое использование в запросах. - -:::note -При использовании проекций следует также учитывать настройку [force_optimize_projection](/operations/settings/settings#force_optimize_projection). -::: - -Проекции не поддерживаются в операторах `SELECT` с модификатором [FINAL](/sql-reference/statements/select/from#final-modifier). - -### Запрос проекции {#projection-query} - -Запрос проекции определяет проекцию. Он неявно выбирает данные из родительской таблицы. -**Синтаксис** - -```sql -SELECT <выражение списка столбцов> [GROUP BY] <выражение ключей группировки> [ORDER BY] <выражение> -``` - -Проекции можно изменять или удалять с помощью оператора [ALTER](/sql-reference/statements/alter/projection.md). - -### Хранение проекций {#projection-storage} - -Проекции хранятся внутри каталога части. По сути это похоже на индекс, но включает подкаталог, в котором хранится часть анонимной таблицы `MergeTree`. Таблица задаётся запросом, определяющим проекцию. Если в определении есть предложение `GROUP BY`, базовый движок хранения становится [AggregatingMergeTree](aggregatingmergetree.md), и все агрегатные функции приводятся к типу `AggregateFunction`. Если есть предложение `ORDER BY`, таблица `MergeTree` использует его как выражение первичного ключа. Во время процесса слияния часть проекции объединяется с использованием процедуры слияния её движка хранения. Контрольная сумма части родительской таблицы объединяется с частью проекции. Остальные операции обслуживания аналогичны операциям для skip-индексов. - -### Анализ запросов {#projection-query-analysis} - -1. Проверьте, может ли проекция быть использована для ответа на данный запрос, то есть даёт ли она тот же результат, что и запрос к базовой таблице. -2. Выберите оптимальное соответствие, для которого нужно прочитать наименьшее количество гранул. -3. Конвейер обработки запроса, использующий проекции, будет отличаться от конвейера, работающего с исходными частями. Если в некоторых частях проекция отсутствует, можно добавить конвейер, чтобы «спроецировать» её на лету. - -## Одновременный доступ к данным {#concurrent-data-access} - -Для одновременного доступа к таблице используется многоверсионность. Иными словами, когда таблица одновременно читается и обновляется, данные читаются из набора частей, актуального на момент выполнения запроса. Длительные блокировки отсутствуют. Вставки не мешают операциям чтения. - -Чтение из таблицы автоматически распараллеливается. - -## TTL для столбцов и таблиц {#table_engine-mergetree-ttl} - -Определяет время жизни значений. - -Выражение `TTL` может быть задано как для всей таблицы, так и для каждого отдельного столбца. `TTL` на уровне таблицы также может задавать логику автоматического перемещения данных между дисками и томами, а также перекомпрессии частей, в которых срок жизни всех данных истёк. - -Выражения должны вычисляться в значение типа данных [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md), [DateTime](/sql-reference/data-types/datetime.md) или [DateTime64](/sql-reference/data-types/datetime64.md). - -**Синтаксис** - -Установка времени жизни для столбца: - -```sql -TTL time_column -TTL time_column + interval -``` - -Чтобы задать `interval`, используйте операторы [интервалов времени](/sql-reference/operators#operators-for-working-with-dates-and-times), например: - -```sql -TTL date_time + INTERVAL 1 MONTH -TTL date_time + INTERVAL 15 HOUR -``` - -### TTL столбца {#mergetree-column-ttl} - -Когда срок жизни значений в столбце истекает, ClickHouse заменяет их значениями по умолчанию для типа данных столбца. Если срок жизни всех значений столбца в части данных истекает, ClickHouse удаляет этот столбец из соответствующей части данных в файловой системе. - -Предложение `TTL` нельзя использовать для ключевых столбцов. - -**Примеры** - -#### Создание таблицы с параметром `TTL`: {#creating-a-table-with-ttl} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int TTL d + INTERVAL 1 MONTH, - b Int TTL d + INTERVAL 1 MONTH, - c String -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d; -``` - -#### Добавление TTL для столбца существующей таблицы {#adding-ttl-to-a-column-of-an-existing-table} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 DAY; -``` - -#### Изменение TTL для столбца {#altering-ttl-of-the-column} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 MONTH; -``` - -### TTL таблицы {#mergetree-table-ttl} - -Для таблицы может быть задано выражение для удаления строк с истекшим сроком жизни и несколько выражений для автоматического перемещения частей между [дисками или томами](#table_engine-mergetree-multiple-volumes). Когда срок жизни строк в таблице истекает, ClickHouse удаляет все соответствующие строки. Для перемещения или перекомпрессии частей все строки части должны удовлетворять критериям выражения `TTL`. - -```sql -TTL expr - [DELETE|RECOMPRESS codec_name1|НА ДИСК 'xxx'|НА ТОМ 'xxx'][, DELETE|RECOMPRESS codec_name2|НА ДИСК 'aaa'|НА ТОМ 'bbb'] ... - [WHERE условия] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] -``` - -Тип правила TTL может следовать за каждым выражением TTL. Он определяет действие, которое будет выполнено, когда выражение будет выполнено (достигнет текущего времени): - -* `DELETE` — удалить истекшие строки (действие по умолчанию); -* `RECOMPRESS codec_name` — перекомпрессировать часть данных с использованием `codec_name`; -* `TO DISK 'aaa'` — перенести часть на диск `aaa`; -* `TO VOLUME 'bbb'` — перенести часть в том `bbb`; -* `GROUP BY` — агрегировать истекшие строки. - -Действие `DELETE` может использоваться вместе с предложением `WHERE`, чтобы удалять только часть истекших строк на основе условия фильтрации: - -```sql -TTL time_column + INTERVAL 1 MONTH УДАЛИТЬ ГДЕ column = 'value' -``` - -Выражение `GROUP BY` должно быть префиксом первичного ключа таблицы. - -Если столбец не входит в выражение `GROUP BY` и явно не задан в предложении `SET`, в результирующей строке он будет содержать произвольное значение из сгруппированных строк (как если бы к нему была применена агрегатная функция `any`). - -**Примеры** - -#### Создание таблицы с `TTL`: {#creating-a-table-with-ttl-1} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE, - d + INTERVAL 1 WEEK TO VOLUME 'aaa', - d + INTERVAL 2 WEEK TO DISK 'bbb'; -``` - -#### Изменение `TTL` для таблицы: {#altering-ttl-of-the-table} - -```sql -ALTER TABLE tab - MODIFY TTL d + INTERVAL 1 DAY; -``` - -Создание таблицы, в которой строки автоматически удаляются через один месяц. Просроченные строки с датами, приходящимися на понедельник, удаляются: - -```sql -CREATE TABLE table_with_where -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE WHERE toDayOfWeek(d) = 1; -``` - -#### Создание таблицы, в которой строки с истёкшим сроком хранения повторно сжимаются: {#creating-a-table-where-expired-rows-are-recompressed} - -```sql -CREATE TABLE table_for_recompression -( - d DateTime, - key UInt64, - value String -) ENGINE MergeTree() -ORDER BY tuple() -PARTITION BY key -TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPRESS CODEC(LZ4HC(10)) -SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; -``` - -Создание таблицы, в которой агрегируются просроченные строки. В результате в столбце `x` содержится максимальное значение по сгруппированным строкам, в `y` — минимальное значение, а в `d` — произвольное значение из сгруппированных строк. - -```sql -CREATE TABLE table_for_aggregation -( - d DateTime, - k1 Int, - k2 Int, - x Int, - y Int -) -ENGINE = MergeTree -ORDER BY (k1, k2) -TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); -``` - -### Удаление просроченных данных {#mergetree-removing-expired-data} - -Данные с истёкшим `TTL` удаляются, когда ClickHouse объединяет части данных. - -Когда ClickHouse обнаруживает, что данные просрочены, он выполняет внеплановое слияние. Чтобы контролировать частоту таких слияний, вы можете задать `merge_with_ttl_timeout`. Если значение слишком мало, будет выполняться много внеплановых слияний, которые могут потреблять значительный объём ресурсов. - -Если вы выполняете запрос `SELECT` между слияниями, вы можете получить просроченные данные. Чтобы этого избежать, используйте запрос [OPTIMIZE](/sql-reference/statements/optimize.md) перед `SELECT`. - -**См. также** - -* настройка [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) - -## Типы дисков {#disk-types} - -Помимо локальных блочных устройств, ClickHouse поддерживает следующие типы хранилищ: - -* [`s3` для S3 и MinIO](#table_engine-mergetree-s3) -* [`gcs` для GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -* [`blob_storage_disk` для Azure Blob Storage](/operations/storing-data#azure-blob-storage) -* [`hdfs` для HDFS](/engines/table-engines/integrations/hdfs) -* [`web` для режима только чтения с веба](/operations/storing-data#web-storage) -* [`cache` для локального кэширования](/operations/storing-data#using-local-cache) -* [`s3_plain` для резервных копий в S3](/operations/backup#backuprestore-using-an-s3-disk) -* [`s3_plain_rewritable` для неизменяемых, нереплицируемых таблиц в S3](/operations/storing-data.md#s3-plain-rewritable-storage) - -## Использование нескольких блочных устройств для хранения данных {#table_engine-mergetree-multiple-volumes} - -### Введение {#introduction} - -Семейство движков таблиц `MergeTree` может хранить данные на нескольких блочных устройствах. Например, это может быть полезно, когда данные определённой таблицы фактически разделены на «горячие» и «холодные». Самые свежие данные запрашиваются регулярно, но занимают небольшой объём. Напротив, большой «хвост» исторических данных запрашивается редко. Если доступно несколько дисков, «горячие» данные могут располагаться на быстрых дисках (например, NVMe SSD или в памяти), а «холодные» — на относительно медленных (например, HDD). - -Часть данных (data part) — минимальная единица, которую можно перемещать, для таблиц на движке `MergeTree`. Данные, принадлежащие одной части, хранятся на одном диске. Части данных могут перемещаться между дисками в фоновом режиме (в соответствии с пользовательскими настройками), а также с помощью запросов [ALTER](/sql-reference/statements/alter/partition). - -### Термины {#terms} - -* Диск — блочное устройство, смонтированное к файловой системе. -* Диск по умолчанию — диск, на котором расположен путь, указанный в серверной настройке [path](/operations/server-configuration-parameters/settings.md/#path). -* Том — упорядоченный набор одинаковых дисков (аналогично [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures)). -* Политика хранения — набор томов и правил перемещения данных между ними. - -Названия описанных сущностей можно найти в системных таблицах [system.storage_policies](/operations/system-tables/storage_policies) и [system.disks](/operations/system-tables/disks). Чтобы применить одну из настроенных политик хранения к таблице, используйте настройку `storage_policy` для таблиц семейства движков `MergeTree`. - -### Конфигурация {#table_engine-mergetree-multiple-volumes_configure} - -Диски, тома и политики хранения должны быть объявлены внутри тега `` или в файле в каталоге `config.d`. - -:::tip -Диски также могут быть объявлены в секции `SETTINGS` запроса. Это полезно -для разового анализа, когда нужно временно подключить диск, который, например, доступен по URL-адресу. -См. раздел [dynamic storage](/operations/storing-data#dynamic-configuration) для получения дополнительной информации. -::: - -Структура конфигурации: - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - - ... - - - ... - -``` - -Теги: - -* `` — имя диска. Имена должны быть разными для всех дисков. -* `path` — путь, по которому сервер будет хранить данные (каталоги `data` и `shadow`); должен заканчиваться символом '/'. -* `keep_free_space_bytes` — объем свободного дискового пространства, который необходимо зарезервировать. - -Порядок определения дисков не имеет значения. - -Разметка конфигурации политик хранения: - -```xml - - ... - - - - - disk_name_from_disks_configuration - 1073741824 - round_robin - - - - - - - 0.2 - - - - - - - - ... - -``` - -Теги: - -* `policy_name_N` — Имя политики. Имена политик должны быть уникальными. -* `volume_name_N` — Имя тома. Имена томов должны быть уникальными. -* `disk` — диск внутри тома. -* `max_data_part_size_bytes` — максимальный размер части данных, которая может быть сохранена на любом из дисков тома. Если оценочный размер сливаемой части будет больше, чем `max_data_part_size_bytes`, то эта часть будет записана на следующий том. По сути эта возможность позволяет держать новые/маленькие части на «горячем» томе (SSD) и перемещать их на «холодный» том (HDD), когда они достигают большого размера. Не используйте этот параметр, если в вашей политике только один том. -* `move_factor` — когда доступное пространство становится меньше этого коэффициента, данные автоматически начинают перемещаться на следующий том, если он есть (по умолчанию 0.1). ClickHouse сортирует существующие части данных по размеру от наибольшей к наименьшей (по убыванию) и выбирает части с суммарным размером, достаточным для выполнения условия `move_factor`. Если суммарный размер всех частей недостаточен, будут перемещены все части. -* `perform_ttl_move_on_insert` — Отключает перемещение по TTL при INSERT части данных. По умолчанию (если включено), если мы вставляем часть данных, которая уже просрочена по правилу перемещения TTL, она сразу попадает на том/диск, указанный в правиле перемещения. Это может существенно замедлить вставку, если целевой том/диск медленный (например, S3). Если выключено, то уже просроченная часть данных записывается на том по умолчанию, а затем сразу перемещается на том, указанный в правиле TTL. -* `load_balancing` — политика балансировки дисков: `round_robin` или `least_used`. -* `least_used_ttl_ms` — настройка таймаута (в миллисекундах) для обновления информации о доступном пространстве на всех дисках (`0` — всегда обновлять, `-1` — никогда не обновлять, по умолчанию `60000`). Обратите внимание: если диск может использоваться только ClickHouse и не подвержен онлайн-изменению размера файловой системы (расширению/сжатию), вы можете использовать `-1`; во всех остальных случаях это не рекомендуется, так как в итоге это приведёт к некорректному распределению пространства. -* `prefer_not_to_merge` — Не следует использовать этот параметр. Отключает слияние частей данных на этом томе (это вредно и приводит к деградации производительности). При включённом параметре (не делайте этого) слияние данных на этом томе не допускается (что плохо). Это позволяет (но вам это не нужно) управлять (если вы хотите что‑то контролировать, вы совершаете ошибку) тем, как ClickHouse работает с медленными дисками (но ClickHouse знает лучше, поэтому, пожалуйста, не используйте этот параметр). -* `volume_priority` — Определяет приоритет (порядок), в котором заполняются тома. Меньшее значение означает более высокий приоритет. Значения параметра должны быть натуральными числами и совместно покрывать диапазон от 1 до N (для самого низкого приоритета) без пропусков. - * Если *все* тома помечены, они получают приоритет в указанном порядке. - * Если помечены только *некоторые* тома, те, у которых нет метки, имеют самый низкий приоритет и получают приоритет в порядке, в котором они определены в конфигурации. - * Если *ни один* том не помечен, их приоритет задаётся в соответствии с порядком, в котором они объявлены в конфигурации. - * Два тома не могут иметь одинаковое значение приоритета. - -Примеры конфигурации: - -```xml - - ... - - - - - disk1 - disk2 - - - - - - - - fast_ssd - 1073741824 - - - disk1 - - - 0.2 - - - - -
- jbod1 -
- - external - -
-
-
- ... -
-``` - -В приведённом примере политика `hdd_in_order` реализует стратегию [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling). Поэтому эта политика определяет только один том (`single`), а части данных хранятся на всех его дисках по кругу. Такая политика может быть весьма полезна, если в системе подключено несколько однотипных дисков, но RAID не настроен. Имейте в виду, что каждый отдельный диск ненадёжен, и может потребоваться компенсировать это фактором репликации 3 или более. - -Если в системе доступны разные типы дисков, вместо этого можно использовать политику `moving_from_ssd_to_hdd`. Том `hot` состоит из SSD-диска (`fast_ssd`), и максимальный размер части, которая может храниться на этом томе, составляет 1 ГБ. Все части размером более 1 ГБ будут храниться непосредственно на томе `cold`, который содержит HDD-диск `disk1`. -Кроме того, как только диск `fast_ssd` будет заполнен более чем на 80%, данные будут перенесены на `disk1` фоновым процессом. - -Порядок перечисления томов в политике хранения важен в случае, если хотя бы один из перечисленных томов не имеет явно заданного параметра `volume_priority`. -Когда том переполнен, данные переносятся на следующий. Порядок перечисления дисков также важен, поскольку данные записываются на них по очереди. - -При создании таблицы к ней можно применить одну из настроенных политик хранения: - -```sql -CREATE TABLE table_with_non_default_policy ( - EventDate Date, - OrderID UInt64, - BannerID UInt64, - SearchPhrase String -) ENGINE = MergeTree -ORDER BY (OrderID, BannerID) -PARTITION BY toYYYYMM(EventDate) -SETTINGS storage_policy = 'moving_from_ssd_to_hdd' -``` - -Политика хранения `default` подразумевает использование только одного тома, который включает один диск, заданный в ``. -Вы можете изменить политику хранения после создания таблицы с помощью запроса [ALTER TABLE ... MODIFY SETTING]; при этом новая политика должна включать все старые диски и тома с теми же именами. - -Количество потоков, выполняющих фоновое перемещение частей данных, можно изменить с помощью настройки [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size). - -### Подробности {#details} - -В случае таблиц `MergeTree` данные попадают на диск разными способами: - -* В результате вставки (запрос `INSERT`). -* Во время фоновых слияний и [мутаций](/sql-reference/statements/alter#mutations). -* При загрузке с другой реплики. -* В результате заморозки партиции [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition). - -Во всех этих случаях, за исключением мутаций и заморозки партиций, часть данных сохраняется на томе и диске в соответствии с заданной политикой хранения: - -1. Выбирается первый том (в порядке объявления), у которого достаточно свободного дискового пространства для хранения части (`unreserved_space > current_part_size`) и который допускает хранение частей заданного размера (`max_data_part_size_bytes > current_part_size`). -2. Внутри этого тома выбирается тот диск, который следует за диском, использованным для хранения предыдущей части данных, и у которого свободное пространство больше размера части (`unreserved_space - keep_free_space_bytes > current_part_size`). - -Внутренним образом мутации и заморозка партиций используют [жёсткие ссылки](https://en.wikipedia.org/wiki/Hard_link). Жёсткие ссылки между разными дисками не поддерживаются, поэтому в таких случаях результирующие части сохраняются на тех же дисках, что и исходные. - -В фоновом режиме части перемещаются между томами на основе количества свободного места (параметр `move_factor`) в соответствии с порядком, в котором тома объявлены в конфигурационном файле. -Данные никогда не переносятся ни с последнего тома, ни на первый том. Для мониторинга фоновых перемещений можно использовать системные таблицы [system.part_log](/operations/system-tables/part_log) (поле `type = MOVE_PART`) и [system.parts](/operations/system-tables/parts.md) (поля `path` и `disk`). Также подробная информация может быть найдена в логах сервера. - -Пользователь может принудительно переместить часть или партицию с одного тома на другой с помощью запроса [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition); при этом учитываются все ограничения для фоновых операций. Запрос самостоятельно инициирует перемещение и не ждёт завершения фоновых операций. Пользователь получит сообщение об ошибке, если недостаточно свободного места или не выполнено какое-либо из требуемых условий. - -Перемещение данных не мешает репликации данных. Поэтому для одной и той же таблицы на разных репликах могут быть заданы разные политики хранения. - -После завершения фоновых слияний и мутаций старые части удаляются только спустя определённый промежуток времени (`old_parts_lifetime`). -В течение этого времени они не перемещаются на другие тома или диски. Поэтому до окончательного удаления части по-прежнему учитываются при оценке занятого дискового пространства. - -Пользователь может более равномерно распределять новые большие части по разным дискам тома [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) с помощью настройки [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod). - -## Использование внешнего хранилища для хранения данных {#table_engine-mergetree-s3} - -Движки таблиц семейства [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) могут хранить данные в `S3`, `AzureBlobStorage`, `HDFS`, используя диск с типом `s3`, `azure_blob_storage`, `hdfs` соответственно. Для получения дополнительной информации см. раздел [настройка параметров внешнего хранилища](/operations/storing-data.md/#configuring-external-storage). - -Пример использования [S3](https://aws.amazon.com/s3/) в качестве внешнего хранилища с диском типа `s3`. - -Фрагмент конфигурации: - -```xml - - ... - - - s3 - true - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - -
Authorization: Bearer SOME-TOKEN
- your_base64_encoded_customer_key - your_kms_key_id - your_kms_encryption_context - true - - http://proxy1 - http://proxy2 - - 10000 - 5000 - 10 - 4 - 1000 - /var/lib/clickhouse/disks/s3/ - false -
- - cache - s3 - /var/lib/clickhouse/disks/s3_cache/ - 10Gi - -
- ... -
-``` - -См. также [настройку вариантов внешних хранилищ](/operations/storing-data.md/#configuring-external-storage). - -:::note конфигурация кэша -Версии ClickHouse с 22.3 по 22.7 используют другую конфигурацию кэша, см. [использование локального кэша](/operations/storing-data.md/#using-local-cache), если вы используете одну из этих версий. -::: - -## Виртуальные столбцы {#virtual-columns} - -* `_part` — Имя парта. -* `_part_index` — Последовательный индекс парта в результате запроса. -* `_part_starting_offset` — Кумулятивный номер первой строки парта в результате запроса. -* `_part_offset` — Номер строки в парте. -* `_part_granule_offset` — Номер гранулы в парте. -* `_partition_id` — Имя партиции. -* `_part_uuid` — Уникальный идентификатор парта (если включена настройка MergeTree `assign_part_uuids`). -* `_part_data_version` — Версия данных парта (либо минимальный номер блока, либо версия мутации). -* `_partition_value` — Значения (кортеж) выражения `PARTITION BY`. -* `_sample_factor` — Коэффициент выборки (из запроса). -* `_block_number` — Исходный номер блока для строки, который был присвоен при вставке; сохраняется при слияниях, когда включена настройка `enable_block_number_column`. -* `_block_offset` — Исходный номер строки в блоке, который был присвоен при вставке; сохраняется при слияниях, когда включена настройка `enable_block_offset_column`. -* `_disk_name` — Имя диска, используемого для хранения. - -## Статистика по столбцам {#column-statistics} - - - - - -Объявление статистики задаётся в секции `COLUMNS` запроса `CREATE` для таблиц из семейства `*MergeTree*` при включённой настройке `set allow_experimental_statistics = 1`. - -```sql -CREATE TABLE tab -( - a Int64 STATISTICS(TDigest, Uniq), - b Float64 -) -ENGINE = MergeTree -ORDER BY a -``` - -Мы также можем изменять статистику с помощью операторов `ALTER`. - -```sql -ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; -ALTER TABLE tab DROP STATISTICS a; -``` - -Эта легковесная статистика агрегирует информацию о распределении значений в столбцах. Статистика хранится в каждой части и обновляется при каждой вставке. -Её можно использовать для оптимизации `PREWHERE` только при включённом параметре `set allow_statistics_optimize = 1`. - -### Доступные типы статистики по столбцам {#available-types-of-column-statistics} - -* `MinMax` - - Минимальное и максимальное значения столбца, что позволяет оценивать селективность диапазонных фильтров по числовым столбцам. - - Синтаксис: `minmax` - -* `TDigest` - - Скетчи [TDigest](https://github.com/tdunning/t-digest), которые позволяют вычислять приблизительные перцентили (например, 90-й перцентиль) для числовых столбцов. - - Синтаксис: `tdigest` - -* `Uniq` - - Скетчи [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog), которые позволяют оценить, сколько различных значений содержит столбец. - - Синтаксис: `uniq` - -* `CountMin` - - Скетчи [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch), которые предоставляют приблизительный подсчёт частоты каждого значения в столбце. - - Синтаксис: `countmin` - -### Поддерживаемые типы данных {#supported-data-types} - -| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String или FixedString | -|-----------|----------------------------------------------------|------------------------| -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | - -### Поддерживаемые операции {#supported-operations} - -| | Фильтры равенства (==) | Фильтры по диапазону (`>, >=, <, <=`) | -|-----------|------------------------|---------------------------------------| -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | - -## Параметры на уровне столбцов {#column-level-settings} - -Некоторые настройки MergeTree можно переопределять на уровне столбцов: - -* `max_compress_block_size` — максимальный размер блоков несжатых данных перед их сжатием при записи в таблицу. -* `min_compress_block_size` — минимальный размер блоков несжатых данных, необходимый для сжатия при записи следующей метки. - -Пример: - -```sql -CREATE TABLE tab -( - id Int64, - document String SETTINGS (min_compress_block_size = 16777216, max_compress_block_size = 16777216) -) -ENGINE = MergeTree -ORDER BY id -``` - -Настройки для отдельных столбцов можно изменять или удалять с помощью [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md), например: - -* Удалить `SETTINGS` из определения столбца: - -```sql -ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; -``` - -* Измените параметр: - -```sql -ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; -``` - -* Сбрасывает одну или несколько настроек, а также удаляет объявление настройки в определении столбца в запросе CREATE для таблицы. - -```sql -ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md deleted file mode 100644 index e920e05b138..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ /dev/null @@ -1,240 +0,0 @@ ---- -description: 'отличается от MergeTree тем, что удаляет дублирующиеся записи с - одинаковым значением сортировочного ключа (секция `ORDER BY` таблицы, а не `PRIMARY KEY`).' -sidebar_label: 'ReplacingMergeTree' -sidebar_position: 40 -slug: /engines/table-engines/mergetree-family/replacingmergetree -title: 'Движок таблицы ReplacingMergeTree' -doc_type: 'reference' ---- - - - -# Движок таблиц ReplacingMergeTree {#replacingmergetree-table-engine} - -Этот движок отличается от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) тем, что удаляет дублирующиеся записи с одинаковым значением [ключа сортировки](../../../engines/table-engines/mergetree-family/mergetree.md) (раздел `ORDER BY` в определении таблицы, а не `PRIMARY KEY`). - -Дедупликация данных происходит только во время слияния. Слияния выполняются в фоновом режиме в неизвестный момент времени, поэтому вы не можете планировать их выполнение. Часть данных может остаться необработанной. Хотя вы можете запустить внеплановое слияние с помощью запроса `OPTIMIZE`, не рассчитывайте на это, потому что запрос `OPTIMIZE` будет считывать и записывать большой объем данных. - -Таким образом, `ReplacingMergeTree` подходит для фоновой очистки дублирующихся данных с целью экономии места, но не гарантирует отсутствие дубликатов. - -:::note -Подробное руководство по ReplacingMergeTree, включая лучшие практики и способы оптимизации производительности, доступно [здесь](/guides/replacing-merge-tree). -::: - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = ReplacingMergeTree([ver [, is_deleted]]) -[PARTITION BY expr] -[ORDER BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -Описание параметров запроса см. в [описании оператора](../../../sql-reference/statements/create/table.md). - -:::note -Уникальность строк определяется разделом таблицы `ORDER BY`, а не `PRIMARY KEY`. -::: - - -## Параметры ReplacingMergeTree {#replacingmergetree-parameters} - -### `ver` {#ver} - -`ver` — столбец с номером версии. Тип `UInt*`, `Date`, `DateTime` или `DateTime64`. Необязательный параметр. - -При слиянии `ReplacingMergeTree` из всех строк с одинаковым сортировочным ключом оставляет только одну: - -* Последнюю в выборке, если `ver` не указан. Выборка — это набор строк в наборе кусков, участвующих в слиянии. Самый недавно созданный кусок (последняя вставка) будет последним в выборке. Таким образом, после дедупликации для каждого уникального сортировочного ключа останется самая последняя строка из самой свежей вставки. -* С максимальной версией, если `ver` указан. Если `ver` одинаков для нескольких строк, для них используется правило «если `ver` не указан», то есть останется самая недавно вставленная строка. - -Пример: - -```sql --- без ver - «побеждает» последняя вставленная запись -CREATE TABLE myFirstReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree -ORDER BY key; - -INSERT INTO myFirstReplacingMT Values (1, 'первая', '2020-01-01 01:01:01'); -INSERT INTO myFirstReplacingMT Values (1, 'вторая', '2020-01-01 00:00:00'); - -SELECT * FROM myFirstReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ вторая │ 2020-01-01 00:00:00 │ -└─────┴─────────┴─────────────────────┘ - - --- с ver - «побеждает» запись с наибольшим значением ver -CREATE TABLE mySecondReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree(eventTime) -ORDER BY key; - -INSERT INTO mySecondReplacingMT Values (1, 'первая', '2020-01-01 01:01:01'); -INSERT INTO mySecondReplacingMT Values (1, 'вторая', '2020-01-01 00:00:00'); - -SELECT * FROM mySecondReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ первая │ 2020-01-01 01:01:01 │ -└─────┴─────────┴─────────────────────┘ -``` - -### `is_deleted` {#is_deleted} - -`is_deleted` — имя столбца, используемого во время слияния для определения, представляет ли строка состояние или подлежит удалению; `1` — строка-удаление, `0` — строка-состояние. - -Тип данных столбца — `UInt8`. - -:::note -`is_deleted` может быть включён только при использовании `ver`. - -Независимо от выполняемой над данными операции, версия должна увеличиваться. Если две вставленные строки имеют одинаковый номер версии, сохраняется последняя вставленная строка. - -По умолчанию ClickHouse сохраняет последнюю строку для ключа, даже если эта строка является строкой удаления. Это нужно для того, чтобы любые будущие строки с более низкими версиями могли быть безопасно вставлены, и строка удаления всё равно применялась. - -Чтобы навсегда удалить такие строки удаления, включите настройку таблицы `allow_experimental_replacing_merge_with_cleanup` и выполните одно из следующих действий: - -1. Задайте настройки таблицы `enable_replacing_merge_with_cleanup_for_min_age_to_force_merge`, `min_age_to_force_merge_on_partition_only` и `min_age_to_force_merge_seconds`. Если все части в партиции старше, чем `min_age_to_force_merge_seconds`, ClickHouse выполнит их слияние - в одну часть и удалит все строки удаления. - -2. Вручную выполните `OPTIMIZE TABLE table [PARTITION partition | PARTITION ID 'partition_id'] FINAL CLEANUP`. - ::: - -Пример: - -```sql --- с ver и is_deleted -CREATE OR REPLACE TABLE myThirdReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime, - `is_deleted` UInt8 -) -ENGINE = ReplacingMergeTree(eventTime, is_deleted) -ORDER BY key -SETTINGS allow_experimental_replacing_merge_with_cleanup = 1; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 0); -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 1); -``` - - -select * from myThirdReplacingMT final; - -0 строк в наборе. Прошло: 0.003 сек. - --- удалить строки с is_deleted -OPTIMIZE TABLE myThirdReplacingMT FINAL CLEANUP; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 00:00:00', 0); - -select * from myThirdReplacingMT final; - -┌─key─┬─someCol─┬───────────eventTime─┬─is_deleted─┐ -│ 1 │ first │ 2020-01-01 00:00:00 │ 0 │ -└─────┴─────────┴─────────────────────┴────────────┘ - -``` -``` - - -## Части запроса {#query-clauses} - -При создании таблицы `ReplacingMergeTree` необходимо указывать те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. - -
- -Устаревший способ создания таблицы - -:::note -Не используйте этот способ в новых проектах и, по возможности, переведите старые проекты на способ, описанный выше. -::: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] ReplacingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [ver]) -``` - -Все параметры, за исключением `ver`, имеют тот же смысл, что и в `MergeTree`. - -- `ver` — столбец с версией. Необязательный параметр. Описание см. в тексте выше. - -
- - - -## Дедупликация при выполнении запроса & FINAL {#query-time-de-duplication--final} - -Во время слияния ReplacingMergeTree определяет дублирующиеся строки, используя значения столбцов `ORDER BY` (указанных при создании таблицы) в качестве уникального идентификатора и сохраняя только самую позднюю версию. Однако это обеспечивает лишь корректность «в конечном счёте» — нет гарантии, что строки будут дедуплицированы, и полагаться на это не следует. Поэтому запросы могут возвращать некорректные результаты, так как строки с обновлениями и удалениями учитываются в запросах. - -Для получения корректных результатов пользователям необходимо дополнять фоновые слияния дедупликацией и удалением строк при выполнении запроса. Это можно сделать с помощью оператора `FINAL`. Например, рассмотрим следующий пример: - -```sql -CREATE TABLE rmt_example -( - `number` UInt16 -) -ENGINE = ReplacingMergeTree -ORDER BY number - -INSERT INTO rmt_example SELECT floor(randUniform(0, 100)) AS number -FROM numbers(1000000000) - -0 строк в наборе. Затрачено: 19.958 сек. Обработано 1.00 миллиард строк, 8.00 ГБ (50.11 миллионов строк/с., 400.84 МБ/с.) -``` - -Запрос без `FINAL` возвращает некорректный результат подсчёта (точное значение будет отличаться в зависимости от выполняемых слияний): - -```sql -SELECT count() -FROM rmt_example - -┌─count()─┐ -│ 200 │ -└─────────┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -Добавление FINAL даёт правильный результат: - -```sql -SELECT count() -FROM rmt_example -FINAL - -┌─count()─┐ -│ 100 │ -└─────────┘ - -1 строка в наборе. Затрачено: 0.002 сек. -``` - -Для получения дополнительных сведений о `FINAL`, включая рекомендации по оптимизации его производительности, мы рекомендуем ознакомиться с [подробным руководством по ReplacingMergeTree](/guides/replacing-merge-tree). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md deleted file mode 100644 index a469af6ea44..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md +++ /dev/null @@ -1,349 +0,0 @@ ---- -description: 'Обзор репликации данных с использованием семейства движков таблиц Replicated* в ClickHouse' -sidebar_label: 'Replicated*' -sidebar_position: 20 -slug: /engines/table-engines/mergetree-family/replication -title: 'Движки таблиц Replicated*' -doc_type: 'reference' ---- - -# Движки таблиц Replicated* {#replicated-table-engines} - -:::note -В ClickHouse Cloud репликацией управляет платформа. Пожалуйста, создавайте таблицы без дополнительных аргументов. Например, в тексте ниже вы бы заменили: - -```sql -ENGINE = ReplicatedMergeTree( - '/clickhouse/tables/{shard}/table_name', - '{replica}' -) -``` - -с помощью: - -```sql -ENGINE = ReplicatedMergeTree -``` - -::: - -Репликация поддерживается только для таблиц семейства MergeTree: - -* ReplicatedMergeTree -* ReplicatedSummingMergeTree -* ReplicatedReplacingMergeTree -* ReplicatedAggregatingMergeTree -* ReplicatedCollapsingMergeTree -* ReplicatedVersionedCollapsingMergeTree -* ReplicatedGraphiteMergeTree - -Репликация работает на уровне отдельной таблицы, а не всего сервера. Один сервер может одновременно хранить как реплицируемые, так и нереплицируемые таблицы. - -Репликация не зависит от разбиения на сегменты. Каждый сегмент имеет собственную независимую репликацию. - -Сжатые данные для запросов `INSERT` и `ALTER` реплицируются (дополнительную информацию см. в документации по [ALTER](/sql-reference/statements/alter)). - -Запросы `CREATE`, `DROP`, `ATTACH`, `DETACH` и `RENAME` выполняются на одном сервере и не реплицируются: - -* Запрос `CREATE TABLE` создаёт новую реплицируемую таблицу на сервере, на котором выполняется запрос. Если такая таблица уже существует на других серверах, создаётся новая реплика. -* Запрос `DROP TABLE` удаляет реплику, расположенную на сервере, на котором выполняется запрос. -* Запрос `RENAME` переименовывает таблицу на одной из реплик. Другими словами, реплицируемые таблицы могут иметь разные имена на разных репликах. - -ClickHouse использует [ClickHouse Keeper](/guides/sre/keeper/index.md) для хранения метаинформации о репликах. Можно использовать ZooKeeper версии 3.4.5 или новее, но рекомендуется ClickHouse Keeper. - -Чтобы использовать репликацию, задайте параметры в разделе конфигурации сервера [zookeeper](/operations/server-configuration-parameters/settings#zookeeper). - -:::note -Не пренебрегайте настройками безопасности. ClickHouse поддерживает схему `digest` [ACL](https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#sc_ZooKeeperAccessControl) подсистемы безопасности ZooKeeper. -::: - -Пример настройки адресов кластера ClickHouse Keeper: - -```xml - - - example1 - 2181 - - - example2 - 2181 - - - example3 - 2181 - - -``` - -ClickHouse также поддерживает хранение метаинформации о репликах во вспомогательном кластере ZooKeeper. Для этого укажите имя кластера ZooKeeper и путь в параметрах движка. -Другими словами, поддерживается хранение метаданных разных таблиц в разных кластерах ZooKeeper. - -Пример задания адресов вспомогательного кластера ZooKeeper: - -```xml - - - - example_2_1 - 2181 - - - example_2_2 - 2181 - - - example_2_3 - 2181 - - - - - example_3_1 - 2181 - - - -``` - -Чтобы хранить метаданные таблицы в дополнительном кластере ZooKeeper вместо кластера ZooKeeper по умолчанию, можно создать таблицу с -движком ReplicatedMergeTree с помощью следующего SQL-запроса: - -```sql -CREATE TABLE table_name ( ... ) ENGINE = ReplicatedMergeTree('zookeeper_name_configured_in_auxiliary_zookeepers:path', 'replica_name') ... -``` - -Вы можете указать любой существующий кластер ZooKeeper, и система будет использовать на нём каталог для собственных данных (каталог задаётся при создании реплицируемой таблицы). - -Если ZooKeeper не указан в конфигурационном файле, вы не сможете создавать реплицируемые таблицы, а любые существующие реплицируемые таблицы будут доступны только для чтения. - - -ZooKeeper не используется в запросах `SELECT`, потому что репликация не влияет на производительность `SELECT`, и запросы выполняются так же быстро, как и для нереплицируемых таблиц. При выполнении запросов к распределённым реплицируемым таблицам поведение ClickHouse управляется настройками [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) и [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries). - -Для каждого запроса `INSERT` в ZooKeeper добавляется примерно десять записей через несколько транзакций. (Более точно: для каждого вставленного блока данных; один запрос `INSERT` содержит один блок или один блок на `max_insert_block_size = 1048576` строк.) Это приводит к немного большему времени ожидания для `INSERT` по сравнению с нереплицируемыми таблицами. Но если вы следуете рекомендациям и вставляете данные пакетами не чаще одного `INSERT` в секунду, это не создаёт никаких проблем. Весь кластер ClickHouse, используемый для координации одного кластера ZooKeeper, в сумме обрабатывает несколько сотен запросов `INSERT` в секунду. Пропускная способность по вставке данных (количество строк в секунду) столь же высока, как и для нереплицируемых данных. - -Для очень больших кластеров вы можете использовать разные кластеры ZooKeeper для разных сегментов. Однако, по нашему опыту, в боевых кластерах примерно из 300 серверов в этом не было необходимости. - -Репликация асинхронная и multi-master. Запросы `INSERT` (а также `ALTER`) могут быть отправлены на любой доступный сервер. Данные вставляются на сервере, где выполняется запрос, а затем копируются на другие серверы. Поскольку это происходит асинхронно, недавно вставленные данные появляются на других репликах с некоторой задержкой. Если часть реплик недоступна, данные будут записаны, когда они станут доступными. Если реплика доступна, задержка равна времени передачи блока сжатых данных по сети. Количество потоков, выполняющих фоновые задачи для реплицируемых таблиц, может быть задано настройкой [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size). - -Движок `ReplicatedMergeTree` использует отдельный пул потоков для реплицируемых загрузок данных (fetches). Размер пула ограничен настройкой [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size), которую можно изменить с перезапуском сервера. - -По умолчанию запрос `INSERT` ожидает подтверждения записи данных только от одной реплики. Если данные были успешно записаны только на одну реплику и сервер с этой репликой перестаёт существовать, сохранённые данные будут утеряны. Чтобы включить получение подтверждения о записи данных от нескольких реплик, используйте опцию `insert_quorum`. - -Каждый блок данных записывается атомарно. Запрос `INSERT` разбивается на блоки до `max_insert_block_size = 1048576` строк. Другими словами, если запрос `INSERT` содержит меньше 1048576 строк, он выполняется атомарно. - -Блоки данных дедуплицируются. При многократной записи одного и того же блока данных (блоки данных одного размера, содержащие одни и те же строки в одном и том же порядке) блок записывается только один раз. Это сделано для случая сбоев сети, когда клиентское приложение не знает, были ли данные записаны в БД, поэтому запрос `INSERT` можно просто повторить. Неважно, на какие реплики отправлялись запросы `INSERT` с идентичными данными. Запросы `INSERT` являются идемпотентными. Параметры дедупликации управляются серверными настройками [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree). - -Во время репликации по сети передаются только исходные данные для вставки. Дальнейшее преобразование данных (слияние) координируется и выполняется на всех репликах одинаковым образом. Это минимизирует сетевой трафик, что означает, что репликация хорошо работает, когда реплики находятся в разных дата-центрах. (Обратите внимание, что дублирование данных в разных дата-центрах является основной целью репликации.) - -Вы можете иметь любое количество реплик одних и тех же данных. По нашему опыту, относительно надёжным и удобным решением может быть двойная репликация в продакшене, когда каждый сервер использует RAID-5 или RAID-6 (и RAID-10 в некоторых случаях). - -Система отслеживает синхронность данных на репликах и способна восстанавливаться после сбоя. Переключение на резерв (failover) происходит автоматически (при небольших различиях в данных) или полуавтоматически (когда данные различаются слишком сильно, что может указывать на ошибку конфигурации). - -## Создание реплицируемых таблиц {#creating-replicated-tables} - -:::note -В ClickHouse Cloud репликация осуществляется автоматически. - -Создавайте таблицы, используя [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) без аргументов репликации. Система внутренне преобразует [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) в [`SharedMergeTree`](/cloud/reference/shared-merge-tree) для репликации и распределения данных. - -Не используйте `ReplicatedMergeTree` и не задавайте параметры репликации, так как репликацией управляет платформа. - -::: - -### Параметры Replicated*MergeTree {#replicatedmergetree-parameters} - -| Параметр | Описание | -| ------------------ | --------------------------------------------------------------------------------------------------------------- | -| `zoo_path` | Путь к таблице в ClickHouse Keeper. | -| `replica_name` | Имя реплики в ClickHouse Keeper. | -| `other_parameters` | Параметры движка, используемого для создания реплицированной версии, например `version` в `ReplacingMergeTree`. | - -Пример: - -```sql -CREATE TABLE table_name -( - EventDate DateTime, - CounterID UInt32, - UserID UInt32, - ver UInt16 -) -ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{layer}-{shard}/table_name', '{replica}', ver) -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID); -``` - -
- Пример с устаревшим синтаксисом - - ```sql - CREATE TABLE table_name - ( - EventDate DateTime, - CounterID UInt32, - UserID UInt32 - ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/table_name', '{replica}', EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID), EventTime), 8192); - ``` -
- -Как видно из примера, эти параметры могут содержать подстановки в фигурных скобках. Значения для подстановки берутся из раздела [macros](/operations/server-configuration-parameters/settings.md/#macros) файла конфигурации. - -Пример: - -```xml - - 02 - example05-02-1 - -``` - -Путь к таблице в ClickHouse Keeper должен быть уникальным для каждой реплицируемой таблицы. Таблицы на разных сегментах должны иметь разные пути. -В данном случае путь состоит из следующих частей: - -`/clickhouse/tables/` — это общий префикс. Мы рекомендуем использовать именно его. - -`{shard}` будет подставлен как идентификатор сегмента. - -`table_name` — это имя узла для таблицы в ClickHouse Keeper. Хорошей идеей будет сделать его таким же, как имя таблицы. Оно задаётся явно, потому что, в отличие от имени таблицы, не изменяется после запроса RENAME. -*ПОДСКАЗКА*: вы также можете добавить имя базы данных перед `table_name`. Например, `db_name.table_name` - -Можно использовать две встроенные подстановки `{database}` и `{table}`, которые подставляются как имя таблицы и имя базы данных соответственно (если только эти макросы не определены в секции `macros`). Таким образом, путь в ClickHouse Keeper можно задать как `'/clickhouse/tables/{shard}/{database}/{table}'`. -Будьте осторожны с переименованием таблиц при использовании этих встроенных подстановок. Путь в ClickHouse Keeper нельзя изменить, и когда таблица переименовывается, макросы будут подставляться в другой путь, таблица будет ссылаться на путь, который не существует в ClickHouse Keeper, и перейдёт в режим только для чтения. - -Имя реплики идентифицирует разные реплики одной и той же таблицы. Для этого вы можете использовать имя сервера, как в примере. Имя должно быть уникальным только внутри каждого сегмента. - -Вы можете задать параметры явно вместо использования подстановок. Это может быть удобно для тестирования и настройки небольших кластеров. Однако в этом случае вы не можете использовать распределённые DDL-запросы (`ON CLUSTER`). - -При работе с большими кластерами мы рекомендуем использовать подстановки, так как они снижают вероятность ошибок. - -Вы можете задать аргументы по умолчанию для движка таблиц `Replicated` в конфигурационном файле сервера. Например: - -```xml -/clickhouse/tables/{shard}/{database}/{table} -{replica} -``` - -В этом случае при создании таблиц можно не указывать аргументы: - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree -ORDER BY x; -``` - -Это эквивалентно следующему: - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/{database}/table_name', '{replica}') -ORDER BY x; -``` - -Выполните запрос `CREATE TABLE` на каждой реплике. Этот запрос создает новую реплицируемую таблицу или добавляет новую реплику к уже существующей. - -Если вы добавляете новую реплику после того, как таблица уже содержит какие‑то данные на других репликах, данные будут скопированы с других реплик на новую реплику после выполнения этого запроса. Другими словами, новая реплика синхронизируется с остальными. - -Чтобы удалить реплику, выполните `DROP TABLE`. Однако удаляется только одна реплика — та, которая находится на сервере, где вы выполняете запрос. - - -## Восстановление после сбоев {#recovery-after-failures} - -Если ClickHouse Keeper недоступен при запуске сервера, реплицируемые таблицы переключаются в режим только для чтения. Система периодически пытается подключиться к ClickHouse Keeper. - -Если ClickHouse Keeper недоступен во время выполнения `INSERT` или при взаимодействии с ClickHouse Keeper возникает ошибка, выбрасывается исключение. - -После подключения к ClickHouse Keeper система проверяет, соответствует ли набор данных в локальной файловой системе ожидаемому набору данных (ClickHouse Keeper хранит эту информацию). При незначительных несоответствиях система устраняет их, синхронизируя данные с репликами. - -Если система обнаруживает поврежденные части данных (с неправильным размером файлов) или нераспознанные части (части, записанные в файловую систему, но не зарегистрированные в ClickHouse Keeper), она перемещает их в подкаталог `detached` (они не удаляются). Любые отсутствующие части копируются с реплик. - -Обратите внимание, что ClickHouse не выполняет никаких разрушающих действий, таких как автоматическое удаление большого объема данных. - -При запуске сервера (или установлении новой сессии с ClickHouse Keeper) он проверяет только количество и размеры всех файлов. Если размеры файлов совпадают, но байты были изменены где-то в середине файла, это обнаруживается не сразу, а только при попытке чтения данных для запроса `SELECT`. В этом случае запрос завершится исключением о несовпадающей контрольной сумме или размере сжатого блока. Части данных добавляются в очередь проверки и при необходимости копируются с реплик. - -Если локальный набор данных существенно отличается от ожидаемого, срабатывает защитный механизм. Сервер записывает это в лог и отказывается запускаться. Причина в том, что такая ситуация может указывать на ошибку конфигурации, например, если реплика на одном сегменте была по ошибке настроена как реплика на другом сегменте. Однако пороговые значения для этого механизма установлены достаточно низко, и такая ситуация может возникнуть при обычном восстановлении после сбоя. В этом случае данные восстанавливаются полуавтоматически — «нажатием кнопки». - -Чтобы начать восстановление, создайте в ClickHouse Keeper узел `/path_to_table/replica_name/flags/force_restore_data` с любым содержимым или выполните команду для восстановления всех реплицируемых таблиц: - -```bash -sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data -``` - -Затем перезапустите сервер. При запуске сервер удаляет эти флаги и начинает восстановление. - - -## Восстановление после полной потери данных {#recovery-after-complete-data-loss} - -Если все данные и метаданные исчезли с одного из серверов, выполните следующие шаги для восстановления: - -1. Установите ClickHouse на сервер. Корректно задайте подстановки в конфигурационном файле, который содержит идентификатор сегмента и реплик, если вы их используете. -2. Если у вас были нереплицированные таблицы, которые необходимо вручную продублировать на серверах, скопируйте их данные с реплики (из каталога `/var/lib/clickhouse/data/db_name/table_name/`). -3. Скопируйте определения таблиц, расположенные в `/var/lib/clickhouse/metadata/`, с реплики. Если в определениях таблиц явно задан идентификатор сегмента или реплики, скорректируйте его так, чтобы он соответствовал этой реплике. (Либо запустите сервер и выполните все запросы `ATTACH TABLE`, которые должны были быть в .sql-файлах в `/var/lib/clickhouse/metadata/`.) -4. Чтобы запустить восстановление, создайте узел ClickHouse Keeper `/path_to_table/replica_name/flags/force_restore_data` с любым содержимым или выполните команду для восстановления всех реплицированных таблиц: `sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` - -Затем запустите сервер (или перезапустите, если он уже работает). Данные будут загружены с реплик. - -Альтернативный вариант восстановления — удалить информацию о потерянной реплике из ClickHouse Keeper (`/path_to_table/replica_name`), затем создать реплику заново, как описано в разделе "[Создание реплицированных таблиц](#creating-replicated-tables)". - -Во время восстановления нет ограничений на пропускную способность сети. Учитывайте это, если вы восстанавливаете большое количество реплик одновременно. - -## Преобразование из MergeTree в ReplicatedMergeTree {#converting-from-mergetree-to-replicatedmergetree} - -Мы используем термин `MergeTree` для обозначения всех движков таблиц семейства `MergeTree`, аналогично и для `ReplicatedMergeTree`. - -Если у вас была таблица `MergeTree`, которая реплицировалась вручную, вы можете преобразовать её в реплицируемую таблицу. Это может понадобиться, если вы уже накопили большой объём данных в таблице `MergeTree` и теперь хотите включить репликацию. - -Команда [ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) позволяет прикрепить отсоединённую таблицу `MergeTree` как таблицу `ReplicatedMergeTree`. - -Таблица `MergeTree` может быть автоматически преобразована при перезапуске сервера, если флаг `convert_to_replicated` установлен в каталоге с данными таблицы (`/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/` для базы данных `Atomic`). -Создайте пустой файл `convert_to_replicated`, и при следующем перезапуске сервера таблица будет загружена как реплицируемая. - -Этот запрос можно использовать, чтобы получить путь к данным таблицы. Если у таблицы несколько путей к данным, необходимо использовать первый. - -```sql -SELECT data_paths FROM system.tables WHERE table = 'table_name' AND database = 'database_name'; -``` - -Обратите внимание, что таблица ReplicatedMergeTree будет создана со значениями настроек `default_replica_path` и `default_replica_name`. -Чтобы создать преобразованную таблицу на других репликах, вам нужно явно указать ее путь в первом аргументе движка таблицы `ReplicatedMergeTree`. Для получения этого пути можно выполнить следующий запрос. - -```sql -SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; -``` - -Есть и способ сделать это вручную. - -Если данные различаются на разных репликах, сначала синхронизируйте их или удалите эти данные на всех репликах, кроме одной. - -Переименуйте существующую таблицу MergeTree, затем создайте таблицу `ReplicatedMergeTree` со старым именем. -Переместите данные из старой таблицы в подкаталог `detached` внутри каталога с данными новой таблицы (`/var/lib/clickhouse/data/db_name/table_name/`). -Затем выполните `ALTER TABLE ATTACH PARTITION` на одной из реплик, чтобы добавить эти части в рабочий набор. - - -## Преобразование из ReplicatedMergeTree в MergeTree {#converting-from-replicatedmergetree-to-mergetree} - -Используйте команду [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree), чтобы подключить отсоединённую таблицу `ReplicatedMergeTree` как `MergeTree` на одном сервере. - -Другой способ сделать это требует перезапуска сервера. Создайте таблицу MergeTree с другим именем. Переместите все данные из каталога с данными таблицы `ReplicatedMergeTree` в каталог данных новой таблицы. Затем удалите таблицу `ReplicatedMergeTree` и перезапустите сервер. - -Если вы хотите избавиться от таблицы `ReplicatedMergeTree`, не запуская сервер: - -- Удалите соответствующий файл `.sql` в каталоге метаданных (`/var/lib/clickhouse/metadata/`). -- Удалите соответствующий путь в ClickHouse Keeper (`/path_to_table/replica_name`). - -После этого вы можете запустить сервер, создать таблицу `MergeTree`, переместить данные в её каталог, а затем перезапустить сервер. - -## Восстановление после потери или повреждения метаданных в кластере ClickHouse Keeper {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} - -Если данные в ClickHouse Keeper были утеряны или повреждены, вы можете сохранить их, переместив в нереплицируемую таблицу, как описано выше. - -**См. также** - -- [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) -- [background_fetches_pool_size](/operations/server-configuration-parameters/settings.md/#background_fetches_pool_size) -- [execute_merges_on_single_replica_time_threshold](/operations/settings/merge-tree-settings#execute_merges_on_single_replica_time_threshold) -- [max_replicated_fetches_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_fetches_network_bandwidth) -- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md deleted file mode 100644 index 56e740f3701..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -description: 'SummingMergeTree наследуется от движка MergeTree. Его ключевая особенность — возможность автоматически суммировать числовые данные при слиянии частей.' -sidebar_label: 'SummingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/summingmergetree -title: 'Табличный движок SummingMergeTree' -doc_type: 'reference' ---- - - - -# Движок таблиц SummingMergeTree {#summingmergetree-table-engine} - -Этот движок наследуется от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree). Разница в том, что при слиянии частей данных для таблиц `SummingMergeTree` ClickHouse заменяет все строки с одинаковым первичным ключом (или, точнее, с одинаковым [ключом сортировки](../../../engines/table-engines/mergetree-family/mergetree.md)) одной строкой, которая содержит суммы значений для столбцов с числовым типом данных. Если ключ сортировки построен таким образом, что одному значению ключа соответствует большое количество строк, это существенно уменьшает объем хранимых данных и ускоряет выборку. - -Мы рекомендуем использовать этот движок совместно с `MergeTree`. Храните полные данные в таблице `MergeTree`, а `SummingMergeTree` используйте для хранения агрегированных данных, например при подготовке отчетов. Такой подход поможет избежать потери ценных данных из-за некорректно составленного первичного ключа. - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = SummingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -Описание параметров запроса см. в [описании запроса](../../../sql-reference/statements/create/table.md). - -### Параметры SummingMergeTree {#parameters-of-summingmergetree} - -#### Столбцы {#columns} - -`columns` — кортеж с именами столбцов, значения в которых будут суммироваться. Необязательный параметр. -Столбцы должны иметь числовой тип и не должны входить в ключ партиционирования или сортировки. - -Если `columns` не указан, ClickHouse суммирует значения во всех столбцах с числовым типом данных, которые не входят в ключ сортировки. - -### Части запроса {#query-clauses} - -При создании таблицы `SummingMergeTree` требуются те же [части запроса](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. - -
- Устаревший метод создания таблицы - - :::note - Не используйте этот метод в новых проектах и, по возможности, переведите старые проекты на метод, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] SummingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - Все параметры, кроме `columns`, имеют то же значение, что и в `MergeTree`. - - * `columns` — кортеж с именами столбцов, значения которых будут суммироваться. Необязательный параметр. Описание см. в тексте выше. -
- - -## Пример использования {#usage-example} - -Рассмотрим следующую таблицу: - -```sql -CREATE TABLE summtt -( - key UInt32, - value UInt32 -) -ENGINE = SummingMergeTree() -ORDER BY key -``` - -Запишите в неё данные: - -```sql -INSERT INTO summtt VALUES(1,1),(1,2),(2,1) -``` - -ClickHouse может суммировать строки не полностью ([см. ниже](#data-processing)), поэтому в запросе мы используем агрегатную функцию `sum` и предложение `GROUP BY`. - -```sql -SELECT key, sum(value) FROM summtt GROUP BY key -``` - -```text -┌─key─┬─sum(value)─┐ -│ 2 │ 1 │ -│ 1 │ 3 │ -└─────┴────────────┘ -``` - - -## Обработка данных {#data-processing} - -Когда данные вставляются в таблицу, они сохраняются как есть. ClickHouse периодически сливает вставленные части данных, и именно в этот момент строки с одинаковым первичным ключом суммируются и заменяются одной строкой для каждой получившейся части данных. - -ClickHouse может сливать части данных таким образом, что разные получившиеся части данных могут содержать строки с одинаковым первичным ключом, т. е. суммирование будет неполным. Поэтому при выполнении запроса (`SELECT`) следует использовать агрегатную функцию [sum()](/sql-reference/aggregate-functions/reference/sum) и предложение `GROUP BY`, как описано в примере выше. - -### Общие правила суммирования {#common-rules-for-summation} - -Значения в столбцах с числовым типом данных суммируются. Набор столбцов определяется параметром `columns`. - -Если значения были равны 0 во всех столбцах для суммирования, строка удаляется. - -Если столбец не входит в первичный ключ и не суммируется, из существующих значений выбирается произвольное. - -Значения не суммируются для столбцов, входящих в первичный ключ. - -### Суммирование в столбцах AggregateFunction {#the-summation-in-the-aggregatefunction-columns} - -Для столбцов типа [AggregateFunction](../../../sql-reference/data-types/aggregatefunction.md) ClickHouse ведёт себя как движок [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md), агрегируя в соответствии с функцией. - -### Вложенные структуры {#nested-structures} - -Таблица может содержать вложенные структуры данных, которые обрабатываются особым образом. - -Если имя вложенной таблицы оканчивается на `Map` и она содержит как минимум два столбца, удовлетворяющих следующим критериям: - -* первый столбец — числовой `(*Int*, Date, DateTime)` или строковый `(String, FixedString)`, назовём его `key`, -* остальные столбцы — арифметические `(*Int*, Float32/64)`, назовём их `(values...)`, - -то такая вложенная таблица интерпретируется как отображение `key => (values...)`, и при слиянии её строк элементы двух наборов данных объединяются по `key` с суммированием соответствующих `(values...)`. - -Примеры: - -```text -DROP TABLE IF EXISTS nested_sum; -CREATE TABLE nested_sum -( - date Date, - site UInt32, - hitsMap Nested( - browser String, - imps UInt32, - clicks UInt32 - ) -) ENGINE = SummingMergeTree -PRIMARY KEY (date, site); - -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Firefox', 'Opera'], [10, 5], [2, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Chrome', 'Firefox'], [20, 1], [1, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['IE'], [22], [0]); -INSERT INTO nested_sum VALUES ('2020-01-01', 10, ['Chrome'], [4], [3]); - -OPTIMIZE TABLE nested_sum FINAL; -- эмулировать слияние - -SELECT * FROM nested_sum; -┌───────date─┬─site─┬─hitsMap.browser───────────────────┬─hitsMap.imps─┬─hitsMap.clicks─┐ -│ 2020-01-01 │ 10 │ ['Chrome'] │ [4] │ [3] │ -│ 2020-01-01 │ 12 │ ['Chrome','Firefox','IE','Opera'] │ [20,11,22,5] │ [1,3,0,1] │ -└────────────┴──────┴───────────────────────────────────┴──────────────┴────────────────┘ - -SELECT - site, - browser, - impressions, - clicks -FROM -( - SELECT - site, - sumMap(hitsMap.browser, hitsMap.imps, hitsMap.clicks) AS imps_map - FROM nested_sum - GROUP BY site -) -ARRAY JOIN - imps_map.1 AS browser, - imps_map.2 AS impressions, - imps_map.3 AS clicks; - -┌─site─┬─browser─┬─impressions─┬─clicks─┐ -│ 12 │ Chrome │ 20 │ 1 │ -│ 12 │ Firefox │ 11 │ 3 │ -│ 12 │ IE │ 22 │ 0 │ -│ 12 │ Opera │ 5 │ 1 │ -│ 10 │ Chrome │ 4 │ 3 │ -└──────┴─────────┴─────────────┴────────┘ -``` - - -При запросе данных используйте функцию [sumMap(key, value)](../../../sql-reference/aggregate-functions/reference/summap.md) для агрегации значений типа `Map`. - -Для вложенных структур данных не нужно указывать их столбцы в кортеже столбцов для суммирования. - - - -## Связанные материалы {#related-content} - -- Блог: [Использование агрегатных комбинаторов в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md deleted file mode 100644 index d9462a14649..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ /dev/null @@ -1,243 +0,0 @@ ---- -description: 'Обеспечивает быструю запись состояний объектов, которые постоянно изменяются, - и фоновое удаление старых состояний объектов.' -sidebar_label: 'VersionedCollapsingMergeTree' -sidebar_position: 80 -slug: /engines/table-engines/mergetree-family/versionedcollapsingmergetree -title: 'Движок таблицы VersionedCollapsingMergeTree' -doc_type: 'reference' ---- - - - -# Движок таблицы VersionedCollapsingMergeTree {#versionedcollapsingmergetree-table-engine} - -Этот движок: - -- Обеспечивает быструю запись состояний объектов, которые постоянно изменяются. -- Удаляет старые состояния объектов в фоновом режиме, что значительно сокращает объём занимаемого места. - -См. раздел [Collapsing](#table_engines_versionedcollapsingmergetree) для получения дополнительной информации. - -Движок наследуется от [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) и добавляет к алгоритму слияния кусков данных логику схлопывания строк. `VersionedCollapsingMergeTree` служит той же цели, что и [CollapsingMergeTree](../../../engines/table-engines/mergetree-family/collapsingmergetree.md), но использует другой алгоритм схлопывания, который позволяет вставлять данные в произвольном порядке несколькими потоками. В частности, столбец `Version` помогает корректно схлопывать строки, даже если они вставляются в неверном порядке. В отличие от этого, `CollapsingMergeTree` допускает только строго последовательную вставку. - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = VersionedCollapsingMergeTree(sign, version) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -Описание параметров запроса см. в разделе [описание запроса](../../../sql-reference/statements/create/table.md). - -### Параметры движка {#engine-parameters} - -```sql -VersionedCollapsingMergeTree(sign, version) -``` - -| Параметр | Описание | Тип | -| --------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sign` | Имя столбца с типом записи: `1` — запись «state», `-1` — запись «cancel». | [`Int8`](/sql-reference/data-types/int-uint) | -| `version` | Имя столбца с версией состояния объекта. | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) или [`DateTime64`](/sql-reference/data-types/datetime64) | - -### Клаузы запроса {#query-clauses} - -При создании таблицы `VersionedCollapsingMergeTree` требуются те же [клаузы](../../../engines/table-engines/mergetree-family/mergetree.md), что и при создании таблицы `MergeTree`. - -
- Устаревший метод создания таблицы - - :::note - Не используйте этот метод в новых проектах. По возможности переведите старые проекты на метод, описанный выше. - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] VersionedCollapsingMergeTree(date-column [, samp#table_engines_versionedcollapsingmergetreeling_expression], (primary, key), index_granularity, sign, version) - ``` - - Все параметры, кроме `sign` и `version`, имеют то же значение, что и в `MergeTree`. - - * `sign` — имя столбца с типом записи: `1` — запись «state», `-1` — запись «cancel». - - Тип данных столбца — `Int8`. - - * `version` — имя столбца с версией состояния объекта. - - Тип данных столбца должен быть `UInt*`. -
- - -## Коллапсирование {#table_engines_versionedcollapsingmergetree} - -### Данные {#data} - -Рассмотрим ситуацию, когда нужно сохранять постоянно изменяющиеся данные для некоторого объекта. Разумно иметь одну строку на объект и обновлять эту строку при каждом изменении. Однако операция UPDATE для СУБД дорогая и медленная, поскольку требует перезаписи данных в хранилище. Обновление неприемлемо, если нужно быстро записывать данные, но вы можете последовательно записывать изменения объекта следующим образом. - -При записи строки используйте столбец `Sign`. Если `Sign = 1`, это означает, что строка представляет состояние объекта (назовём её строкой «state»). Если `Sign = -1`, это указывает на отмену состояния объекта с теми же атрибутами (назовём её строкой «cancel»). Также используйте столбец `Version`, который должен идентифицировать каждое состояние объекта отдельным номером. - -Например, мы хотим посчитать, сколько страниц посетили пользователи на некотором сайте и как долго они там находились. В некоторый момент времени мы записываем следующую строку с состоянием активности пользователя: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -Позже, на одном из этапов, мы фиксируем изменение активности пользователя и записываем это с помощью следующих двух строк. - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -Первая строка аннулирует предыдущее состояние объекта (пользователя). В ней должны быть скопированы все поля аннулируемого состояния, кроме `Sign`. - -Вторая строка содержит текущее состояние. - -Поскольку нам нужно только последнее состояние активности пользователя, строки - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -может быть удалён, схлопывая некорректное (старое) состояние объекта. `VersionedCollapsingMergeTree` делает это при слиянии частей данных. - -Чтобы понять, зачем нужны две строки для каждого изменения, см. раздел [Algorithm](#table_engines-versionedcollapsingmergetree-algorithm). - -**Примечания по использованию** - -1. Программа, которая записывает данные, должна запоминать состояние объекта, чтобы иметь возможность его отменить. Строка "Cancel" должна содержать копии полей первичного ключа, версию строки "state" и противоположный `Sign`. Это увеличивает начальный размер хранилища, но позволяет быстро записывать данные. -2. Длинные постоянно растущие массивы в столбцах снижают эффективность движка из‑за нагрузки на запись. Чем проще данные, тем выше эффективность. -3. Результаты `SELECT` сильно зависят от согласованности истории изменений объекта. Будьте внимательны при подготовке данных для вставки. При несогласованных данных вы можете получить непредсказуемые результаты, например отрицательные значения для неотрицательных метрик, таких как глубина сессии. - -### Algorithm {#table_engines-versionedcollapsingmergetree-algorithm} - -Когда ClickHouse сливает части данных, он удаляет каждую пару строк с одинаковым первичным ключом и версией и разным `Sign`. Порядок строк не имеет значения. - -Когда ClickHouse вставляет данные, он упорядочивает строки по первичному ключу. Если столбец `Version` не входит в первичный ключ, ClickHouse неявно добавляет его в первичный ключ как последнее поле и использует для сортировки. - - -## Выборка данных {#selecting-data} - -ClickHouse не гарантирует, что все строки с одинаковым первичным ключом окажутся в одной и той же результирующей части данных или даже на одном и том же физическом сервере. Это верно как при записи данных, так и при последующем объединении частей данных. Кроме того, ClickHouse обрабатывает запросы `SELECT` в несколько потоков и не может предсказать порядок строк в результате. Это означает, что агрегирование обязательно, если необходимо получить полностью «схлопнутые» данные из таблицы `VersionedCollapsingMergeTree`. - -Чтобы завершить схлопывание, сформируйте запрос с оператором `GROUP BY` и агрегатными функциями, которые учитывают знак. Например, для вычисления количества используйте `sum(Sign)` вместо `count()`. Для вычисления суммы некоторой величины используйте `sum(Sign * x)` вместо `sum(x)` и добавьте `HAVING sum(Sign) > 0`. - -Такие агрегаты, как `count`, `sum` и `avg`, можно вычислять таким образом. Агрегат `uniq` можно вычислить, если объект имеет хотя бы одно несхлопнутое состояние. Агрегаты `min` и `max` вычислить нельзя, потому что `VersionedCollapsingMergeTree` не сохраняет историю значений схлопнутых состояний. - -Если нужно извлечь данные со «схлопыванием», но без агрегирования (например, чтобы проверить, существуют ли строки, последние значения которых удовлетворяют определённым условиям), можно использовать модификатор `FINAL` в секции `FROM`. Такой подход неэффективен и не должен применяться для больших таблиц. - - - -## Пример использования {#example-of-use} - -Пример данных: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -Создание таблицы: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8, - Version UInt8 -) -ENGINE = VersionedCollapsingMergeTree(Sign, Version) -ORDER BY UserID -``` - -Добавление данных: - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1, 1),(4324182021466249494, 6, 185, 1, 2) -``` - -Выполним два запроса `INSERT`, чтобы создать две разные части данных. Если вставить данные одним запросом, ClickHouse создаст одну часть данных, и слияние никогда не будет выполнено. - -Получение данных: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -Что мы здесь видим и куда делись схлопнувшиеся части? -Мы создали две части данных с помощью двух запросов `INSERT`. Запрос `SELECT` был выполнен в двух потоках, и результат — строки в случайном порядке. -Схлопывание не произошло, потому что части данных ещё не были объединены. ClickHouse объединяет части данных в неизвестный момент времени, который мы не можем предсказать. - -Именно поэтому нам нужно агрегирование: - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration, - Version -FROM UAct -GROUP BY UserID, Version -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Version─┐ -│ 4324182021466249494 │ 6 │ 185 │ 2 │ -└─────────────────────┴───────────┴──────────┴─────────┘ -``` - -Если нам не нужна агрегация и мы хотим принудительно выполнить схлопывание, мы можем использовать модификатор `FINAL` в предложении `FROM`. - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -Это крайне неэффективный способ выборки данных. Не используйте его для больших таблиц. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md deleted file mode 100644 index fdd588065d7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ /dev/null @@ -1,333 +0,0 @@ ---- -description: 'Табличный движок Alias создает прозрачный прокси к другой таблице. Все операции перенаправляются в целевую таблицу, при этом сам алиас не хранит данные.' -sidebar_label: 'Alias' -sidebar_position: 5 -slug: /engines/table-engines/special/alias -title: 'Табличный движок Alias' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# Движок таблицы Alias {#alias-table-engine} - - - -Движок `Alias` создаёт прокси для другой таблицы. Все операции чтения и записи перенаправляются в целевую таблицу, при этом сама таблица-алиас не хранит данных и только поддерживает ссылку на целевую таблицу. - -:::info -Это экспериментальная функция, которая может измениться в будущих релизах с нарушением обратной совместимости. -Включите использование движка таблицы Alias -с помощью настройки [allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine). -Введите команду `set allow_experimental_alias_table_engine = 1`. -::: - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_table) -``` - -Или с указанием имени базы данных: - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_db, target_table) -``` - -:::note -Таблица `Alias` не поддерживает явное определение столбцов. Столбцы автоматически наследуются от целевой таблицы. Это гарантирует, что таблица `Alias` всегда соответствует схеме целевой таблицы. -::: - - -## Параметры движка {#engine-parameters} - -- **`target_db (optional)`** — Имя базы данных, содержащей целевую таблицу. -- **`target_table`** — Имя целевой таблицы. - -## Поддерживаемые операции {#supported-operations} - -Движок таблицы `Alias` поддерживает все основные операции. - -### Операции с целевой таблицей {#operations-on-target} - -Эти операции проксируются на целевую таблицу: - -| Операция | Поддержка | Описание | -|-----------|-----------|----------| -| `SELECT` | ✅ | Чтение данных из целевой таблицы | -| `INSERT` | ✅ | Запись данных в целевую таблицу | -| `INSERT SELECT` | ✅ | Пакетная вставка в целевую таблицу | -| `ALTER TABLE ADD COLUMN` | ✅ | Добавление столбцов в целевую таблицу | -| `ALTER TABLE MODIFY SETTING` | ✅ | Изменение настроек целевой таблицы | -| `ALTER TABLE PARTITION` | ✅ | Операции с партициями (DETACH/ATTACH/DROP) для целевой таблицы | -| `ALTER TABLE UPDATE` | ✅ | Обновление строк в целевой таблице (мутация) | -| `ALTER TABLE DELETE` | ✅ | Удаление строк из целевой таблицы (мутация) | -| `OPTIMIZE TABLE` | ✅ | Оптимизация целевой таблицы (слияние частей) | -| `TRUNCATE TABLE` | ✅ | Очистка целевой таблицы | - -### Операции с самим алиасом {#operations-on-alias} - -Эти операции применяются только к алиасу, **а не** к целевой таблице: - -| Операция | Поддержка | Описание | -|----------|-----------|----------| -| `DROP TABLE` | ✅ | Удаляет только алиас, целевая таблица остаётся без изменений | -| `RENAME TABLE` | ✅ | Переименовывает только алиас, целевая таблица остаётся без изменений | - -## Примеры использования {#usage-examples} - -### Создание простого алиаса {#basic-alias-creation} - -Создайте простой алиас в этой же базе данных: - -```sql --- Создать исходную таблицу -CREATE TABLE source_data ( - id UInt32, - name String, - value Float64 -) ENGINE = MergeTree -ORDER BY id; - --- Вставить данные -INSERT INTO source_data VALUES (1, 'one', 10.1), (2, 'two', 20.2); - --- Создать псевдоним -CREATE TABLE data_alias ENGINE = Alias('source_data'); - --- Выполнить запрос через псевдоним -SELECT * FROM data_alias; -``` - -```text -┌─id─┬─name─┬─value─┐ -│ 1 │ one │ 10.1 │ -│ 2 │ two │ 20.2 │ -└────┴──────┴───────┘ -``` - - -### Межбазовый псевдоним {#cross-database-alias} - -Создайте псевдоним, ссылающийся на таблицу в другой базе данных: - -```sql --- Создать базы данных -CREATE DATABASE db1; -CREATE DATABASE db2; - --- Создать исходную таблицу в db1 -CREATE TABLE db1.events ( - timestamp DateTime, - event_type String, - user_id UInt32 -) ENGINE = MergeTree -ORDER BY timestamp; - --- Создать псевдоним в db2, указывающий на db1.events -CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); - --- Или используя формат database.table -CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); - --- Оба псевдонима работают одинаково -INSERT INTO db2.events_alias VALUES (now(), 'click', 100); -SELECT * FROM db2.events_alias2; -``` - - -### Операции записи через алиас {#write-operations} - -Все операции записи перенаправляются в целевую таблицу: - -```sql -CREATE TABLE metrics ( - ts DateTime, - metric_name String, - value Float64 -) ENGINE = MergeTree -ORDER BY ts; - -CREATE TABLE metrics_alias ENGINE = Alias('metrics'); - --- Вставка через псевдоним -INSERT INTO metrics_alias VALUES - (now(), 'cpu_usage', 45.2), - (now(), 'memory_usage', 78.5); - --- Вставка с SELECT -INSERT INTO metrics_alias -SELECT now(), 'disk_usage', number * 10 -FROM system.numbers -LIMIT 5; - --- Проверка данных в целевой таблице -SELECT count() FROM metrics; -- Возвращает 7 -SELECT count() FROM metrics_alias; -- Возвращает 7 -``` - - -### Изменение схемы {#schema-modification} - -Операции ALTER изменяют схему целевой таблицы: - -```sql -CREATE TABLE users ( - id UInt32, - name String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE users_alias ENGINE = Alias('users'); - --- Добавление столбца через псевдоним -ALTER TABLE users_alias ADD COLUMN email String DEFAULT ''; - --- Столбец добавляется в целевую таблицу -DESCRIBE users; -``` - -```text -┌─name──┬─type───┬─default_type─┬─default_expression─┐ -│ id │ UInt32 │ │ │ -│ name │ String │ │ │ -│ email │ String │ DEFAULT │ '' │ -└───────┴────────┴──────────────┴────────────────────┘ -``` - - -### Мутации данных {#data-mutations} - -Поддерживаются операции UPDATE и DELETE: - -```sql -CREATE TABLE products ( - id UInt32, - name String, - price Float64, - status String DEFAULT 'active' -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE products_alias ENGINE = Alias('products'); - -INSERT INTO products_alias VALUES - (1, 'item_one', 100.0, 'active'), - (2, 'item_two', 200.0, 'active'), - (3, 'item_three', 300.0, 'inactive'); - --- Обновление через алиас -ALTER TABLE products_alias UPDATE price = price * 1.1 WHERE status = 'active'; - --- Удаление через алиас -ALTER TABLE products_alias DELETE WHERE status = 'inactive'; - --- Изменения применяются к целевой таблице -SELECT * FROM products ORDER BY id; -``` - -```text -┌─id─┬─name─────┬─price─┬─status─┐ -│ 1 │ item_one │ 110.0 │ активный │ -│ 2 │ item_two │ 220.0 │ активный │ -└────┴──────────┴───────┴────────┘ -``` - - -### Операции с партициями {#partition-operations} - -Для секционированных таблиц операции с партициями передаются далее: - -```sql -CREATE TABLE logs ( - date Date, - level String, - message String -) ENGINE = MergeTree -PARTITION BY toYYYYMM(date) -ORDER BY date; - -CREATE TABLE logs_alias ENGINE = Alias('logs'); - -INSERT INTO logs_alias VALUES - ('2024-01-15', 'INFO', 'message1'), - ('2024-02-15', 'ERROR', 'message2'), - ('2024-03-15', 'INFO', 'message3'); - --- Отсоединить партицию через псевдоним -ALTER TABLE logs_alias DETACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- Возвращает 2 (партиция 202402 отсоединена) - --- Присоединить партицию обратно -ALTER TABLE logs_alias ATTACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- Возвращает 3 -``` - - -### Оптимизация таблицы {#table-optimization} - -Оптимизируйте операции по слиянию частей в целевой таблице: - -```sql -CREATE TABLE events ( - id UInt32, - data String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE events_alias ENGINE = Alias('events'); - --- Множественные вставки создают несколько частей -INSERT INTO events_alias VALUES (1, 'data1'); -INSERT INTO events_alias VALUES (2, 'data2'); -INSERT INTO events_alias VALUES (3, 'data3'); - --- Проверка количества частей -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; - --- Оптимизация через алиас -OPTIMIZE TABLE events_alias FINAL; - --- Части объединяются в целевой таблице -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; -- Возвращает 1 -``` - - -### Управление алиасами {#alias-management} - -Алиасы можно переименовывать или удалять независимо: - -```sql -CREATE TABLE important_data ( - id UInt32, - value String -) ENGINE = MergeTree -ORDER BY id; - -INSERT INTO important_data VALUES (1, 'critical'), (2, 'important'); - -CREATE TABLE old_alias ENGINE = Alias('important_data'); - --- Переименование псевдонима (целевая таблица не изменяется) -RENAME TABLE old_alias TO new_alias; - --- Создание ещё одного псевдонима для той же таблицы -CREATE TABLE another_alias ENGINE = Alias('important_data'); - --- Удаление одного псевдонима (целевая таблица и другие псевдонимы не изменяются) -DROP TABLE new_alias; - -SELECT * FROM another_alias; -- По-прежнему работает -SELECT count() FROM important_data; -- Данные не повреждены, возвращается 2 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md deleted file mode 100644 index ac3ddf72127..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'Буферизует данные для записи в оперативной памяти (RAM), периодически сбрасывая их в другую таблицу. При чтении данные считываются одновременно из буфера и из другой таблицы.' -sidebar_label: 'Buffer' -sidebar_position: 120 -slug: /engines/table-engines/special/buffer -title: 'Движок таблицы Buffer' -doc_type: 'reference' ---- - -# Движок таблицы Buffer {#buffer-table-engine} - -Буферизует данные, подлежащие записи, в оперативной памяти и периодически сбрасывает их в другую таблицу. При чтении данные одновременно считываются из буфера и из другой таблицы. - -:::note -Рекомендуемая альтернатива движку таблицы Buffer — включение [асинхронных вставок](/guides/best-practices/asyncinserts.md). -::: - -```sql -Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) -``` - -### Параметры движка {#engine-parameters} - -#### `database` {#database} - -`database` – Имя базы данных. Можно использовать `currentDatabase()` или другое константное выражение, возвращающее строку. - -#### `table` {#table} - -`table` – Таблица, в которую сбрасываются данные. - -#### `num_layers` {#num_layers} - -`num_layers` – Уровень параллелизма. Физически таблица будет представлена как `num_layers` независимых буферов. - -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes` и `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} - -Условия для сброса данных из буфера. - -### Необязательные параметры движка {#optional-engine-parameters} - -#### `flush_time`, `flush_rows` и `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} - -Условия для фонового сброса данных из буфера (отсутствие параметра или значение 0 означает отсутствие параметров `flush*`). - -Данные сбрасываются из буфера и записываются в целевую таблицу, если выполнены все условия `min*` или хотя бы одно условие `max*`. - -Также, если выполнено хотя бы одно условие `flush*`, в фоновом режиме инициируется сброс. В отличие от параметров `max*`, параметры `flush*` позволяют настраивать фоновые сбросы отдельно, чтобы избежать добавления задержки для запросов `INSERT` в таблицы Buffer. - -#### `min_time`, `max_time` и `flush_time` {#min_time-max_time-and-flush_time} - -Условие по времени в секундах с момента первой записи в буфер. - -#### `min_rows`, `max_rows` и `flush_rows` {#min_rows-max_rows-and-flush_rows} - -Условие по количеству строк в буфере. - -#### `min_bytes`, `max_bytes` и `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} - -Условие по количеству байт в буфере. - -Во время операции записи данные вставляются в один или несколько случайных буферов (настраивается с помощью `num_layers`). Или, если объём вставляемых данных достаточно велик (больше `max_rows` или `max_bytes`), они записываются непосредственно в целевую таблицу, минуя буфер. - -Условия для сброса данных вычисляются отдельно для каждого из буферов `num_layers`. Например, если `num_layers = 16` и `max_bytes = 100000000`, максимальное потребление оперативной памяти составляет 1,6 ГБ. - -Пример: - -```sql -CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, 100, 10000, 1000000, 10000000, 100000000) -``` - -Создание таблицы `merge.hits_buffer` с той же структурой, что и `merge.hits`, с использованием движка Buffer. При записи в эту таблицу данные буферизуются в оперативной памяти и позже записываются в таблицу `merge.hits`. Создаётся единый буфер, и данные сбрасываются, если выполняется одно из условий: - -* прошло 100 секунд с момента последнего сброса (`max_time`) или -* было записано 1 миллион строк (`max_rows`) или -* было записано 100 МБ данных (`max_bytes`) или -* прошло 10 секунд (`min_time`) и были записаны 10 000 строк (`min_rows`) и 10 МБ (`min_bytes`) данных - -Например, если была записана всего одна строка, то через 100 секунд она будет сброшена в любом случае. Но если было записано много строк, данные будут сброшены раньше. - -Когда сервер останавливается или выполняется `DROP TABLE` либо `DETACH TABLE`, буферизованные данные также сбрасываются в целевую таблицу. - -Вы можете задать пустые строковые литералы в одинарных кавычках для имени базы данных и имени таблицы. Это указывает на отсутствие целевой таблицы. В этом случае, когда условия сброса данных выполняются, буфер просто очищается. Это может быть полезно для хранения окна данных в памяти. - -При чтении из таблицы Buffer данные обрабатываются как из буфера, так и из целевой таблицы (если она существует). -Обратите внимание, что таблица Buffer не поддерживает индекс. Иными словами, данные в буфере полностью сканируются, что может быть медленно для больших буферов. (Для данных в подчинённой таблице будет использоваться индекс, который она поддерживает.) - -Если набор столбцов в таблице Buffer не совпадает с набором столбцов в подчинённой таблице, вставляется подмножество столбцов, существующих в обеих таблицах. - -Если типы не совпадают для одного из столбцов в таблице Buffer и подчинённой таблице, в журнал сервера записывается сообщение об ошибке и буфер очищается. -То же самое происходит, если подчинённая таблица не существует в момент сброса буфера. - -:::note -Выполнение ALTER для таблицы Buffer в релизах, выпущенных до 26 октября 2021 года, приведёт к ошибке `Block structure mismatch` (см. [#15117](https://github.com/ClickHouse/ClickHouse/issues/15117) и [#30565](https://github.com/ClickHouse/ClickHouse/pull/30565)), поэтому единственный вариант — удалить таблицу Buffer и затем создать её заново. Перед тем как выполнять ALTER для таблицы Buffer, проверьте, что эта ошибка исправлена в вашем релизе. -::: - -Если сервер аварийно перезапускается, данные в буфере теряются. - -`FINAL` и `SAMPLE` работают некорректно для таблиц Buffer. Эти условия передаются в целевую таблицу, но не используются при обработке данных в буфере. Если эти возможности необходимы, рекомендуется использовать таблицу Buffer только для записи, а читать данные из целевой таблицы. - -При добавлении данных в таблицу Buffer один из буферов блокируется. Это вызывает задержки, если одновременно с этим из таблицы выполняется операция чтения. - -Данные, вставленные в таблицу Buffer, могут попасть в подчинённую таблицу в другом порядке и в других блоках. Из‑за этого таблицу Buffer сложно корректно использовать для записи в CollapsingMergeTree. Чтобы избежать проблем, можно установить `num_layers` в 1. - -Если целевая таблица реплицируется, при записи в таблицу Buffer теряются некоторые ожидаемые свойства реплицируемых таблиц. Случайные изменения порядка строк и размеров кусков данных приводят к тому, что дедупликация данных перестаёт работать, а это означает, что невозможно гарантировать надёжную запись в реплицируемые таблицы строго один раз. - -Из‑за этих недостатков мы можем рекомендовать использовать таблицу Buffer лишь в редких случаях. - -Таблица Buffer используется, когда за единицу времени поступает слишком много INSERT-запросов от большого количества серверов и данные нельзя буферизовать до вставки, что приводит к тому, что операции INSERT выполняются недостаточно быстро. - -Обратите внимание, что вставка данных по одной строке не имеет смысла даже для таблиц Buffer. Это даст скорость всего в несколько тысяч строк в секунду, тогда как вставка больших блоков данных может обеспечить более миллиона строк в секунду. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md deleted file mode 100644 index 883a540fbfb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -description: 'Движок `Dictionary` отображает данные словаря как таблицу в ClickHouse.' -sidebar_label: 'Dictionary' -sidebar_position: 20 -slug: /engines/table-engines/special/dictionary -title: 'Движок таблицы Dictionary' -doc_type: 'reference' ---- - -# Движок таблицы Dictionary {#dictionary-table-engine} - -Движок `Dictionary` представляет данные [словаря](../../../sql-reference/dictionaries/index.md) в виде таблицы ClickHouse. - -## Пример {#example} - -В качестве примера рассмотрим словарь `products` со следующей конфигурацией: - -```xml - - - products - - -
products
- DSN=some-db-server - - - - 300 - 360 - - - - - - - product_id - - - title - String - - - - - -``` - -Запросите данные словаря: - -```sql -SELECT - name, - type, - key, - attribute.names, - attribute.types, - bytes_allocated, - element_count, - source -FROM system.dictionaries -WHERE name = 'products' -``` - -```text -┌─name─────┬─type─┬─key────┬─attribute.names─┬─attribute.types─┬─bytes_allocated─┬─element_count─┬─source──────────┐ -│ products │ Flat │ UInt64 │ ['title'] │ ['String'] │ 23065376 │ 175032 │ ODBC: .products │ -└──────────┴──────┴────────┴─────────────────┴─────────────────┴─────────────────┴───────────────┴─────────────────┘ -``` - -Вы можете использовать функцию [dictGet*](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull), чтобы получить данные словаря в этом формате. - -Это представление неудобно, когда нужно получить сырые (raw) данные или выполнить операцию `JOIN`. В таких случаях можно использовать движок `Dictionary`, который отображает данные словаря в виде таблицы. - -Синтаксис: - -```sql -CREATE TABLE %table_name% (%fields%) engine = Dictionary(%dictionary_name%)` -``` - -Пример использования: - -```sql -CREATE TABLE products (product_id UInt64, title String) ENGINE = Dictionary(products); -``` - -Хорошо - -Посмотрите, что содержится в таблице. - -```sql -SELECT * FROM products LIMIT 1; -``` - -```text -┌────product_id─┬─title───────────┐ -│ 152689 │ Некий товар │ -└───────────────┴─────────────────┘ -``` - -**См. также** - -* [Функция dictionary](/sql-reference/table-functions/dictionary) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md deleted file mode 100644 index 13e629e2e7b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ /dev/null @@ -1,251 +0,0 @@ ---- -description: 'Таблицы с движком Distributed не хранят собственные данные, а позволяют выполнять распределённую обработку запросов на нескольких серверах. Чтение автоматически распараллеливается. При чтении используются индексы таблиц на удалённых серверах, если они есть.' -sidebar_label: 'Distributed' -sidebar_position: 10 -slug: /engines/table-engines/special/distributed -title: 'Движок таблицы Distributed' -doc_type: 'reference' ---- - - - -# Движок распределённой таблицы {#distributed-table-engine} - -:::warning Распределённый движок в Cloud -Чтобы создать движок распределённой таблицы в ClickHouse Cloud, можно использовать табличные функции [`remote` и `remoteSecure`](../../../sql-reference/table-functions/remote). -Синтаксис `Distributed(...)` в ClickHouse Cloud использовать нельзя. -::: - -Таблицы с движком Distributed не хранят собственные данные, но позволяют выполнять распределённую обработку запросов на нескольких серверах. -Чтение автоматически распараллеливается. Во время чтения используются индексы таблиц на удалённых серверах, если они есть. - - - -## Создание таблицы {#distributed-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) -[SETTINGS name=value, ...] -``` - -### Из таблицы {#distributed-from-a-table} - -Когда таблица `Distributed` указывает на таблицу на текущем сервере, вы можете заимствовать её схему: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] -``` - -### Параметры движка Distributed {#distributed-parameters} - -| Параметр | Описание | -| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `cluster` | Имя кластера в конфигурационном файле сервера | -| `database` | Имя удалённой базы данных | -| `table` | Имя удалённой таблицы | -| `sharding_key` (необязательно) | Ключ шардинга.
Указание `sharding_key` необходимо в следующих случаях:
  • Для операций `INSERT` в распределённую таблицу (так как движку таблицы нужен `sharding_key`, чтобы определить, как распределить данные по шардам). Однако, если включена настройка `insert_distributed_one_random_shard`, то для `INSERT` ключ шардинга не требуется.
  • Для использования с `optimize_skip_unused_shards`, так как `sharding_key` необходим для определения, какие шарды должны запрашиваться
| -| `policy_name` (необязательно) | Имя политики, которое будет использоваться для хранения временных файлов при фоновой отправке | - -**См. также** - -* настройка [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) -* [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) для примеров - -### Настройки движка Distributed {#distributed-settings} - - -| Параметр | Описание | Значение по умолчанию | -| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| `fsync_after_insert` | Выполнять `fsync` для данных файла после фоновой вставки в Distributed. Гарантирует, что ОС сбросила все вставленные данные в файл **на диске инициирующего узла**. | `false` | -| `fsync_directories` | Выполнять `fsync` для каталогов. Гарантирует, что ОС обновила метаданные каталога после операций, связанных с фоновыми вставками в таблицу Distributed (после вставки, после отправки данных на шард и т. д.). | `false` | -| `skip_unavailable_shards` | Если `true`, ClickHouse молча пропускает недоступные шарды. Шард помечается как недоступный, когда: 1) к нему нет доступа из‑за ошибки соединения; 2) шард не может быть разрешён через DNS; 3) таблица не существует на шарде. | `false` | -| `bytes_to_throw_insert` | Если количество сжатых байт, ожидающих фоновой операции `INSERT`, превысит это значение, будет выброшено исключение. `0` — не выбрасывать. | `0` | -| `bytes_to_delay_insert` | Если количество сжатых байт, ожидающих фоновой операции `INSERT`, превысит это значение, выполнение запроса будет задержано. `0` — не задерживать. | `0` | -| `max_delay_to_insert` | Максимальная задержка вставки данных в таблицу Distributed в секундах, если имеется много данных (байт), ожидающих фоновой отправки. | `60` | -| `background_insert_batch` | То же, что и [`distributed_background_insert_batch`](../../../operations/settings/settings.md#distributed_background_insert_batch). | `0` | -| `background_insert_split_batch_on_failure` | То же, что и [`distributed_background_insert_split_batch_on_failure`](../../../operations/settings/settings.md#distributed_background_insert_split_batch_on_failure). | `0` | -| `background_insert_sleep_time_ms` | То же, что и [`distributed_background_insert_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms). | `0` | -| `background_insert_max_sleep_time_ms` | То же, что и [`distributed_background_insert_max_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms). | `0` | -| `flush_on_detach` | Сбрасывать данные на удалённые узлы при `DETACH`/`DROP`/остановке сервера. | `true` | - -:::note -**Параметры надёжности** (`fsync_...`): - -* Влияют только на фоновые `INSERT` (то есть при `distributed_foreground_insert=false`), когда данные сначала сохраняются на диск инициирующего узла, а затем в фоне отправляются на шарды. -* Могут существенно снизить производительность `INSERT`. -* Влияют на запись данных, хранящихся внутри каталога распределённой таблицы, на **узел, который принял вашу вставку**. Если вам нужны гарантии записи данных в базовые таблицы MergeTree, см. параметры надёжности (`...fsync...`) в `system.merge_tree_settings`. - -Для **параметров ограничений вставки** (`..._insert`) см. также: - -* параметр [`distributed_foreground_insert`](../../../operations/settings/settings.md#distributed_foreground_insert) -* параметр [`prefer_localhost_replica`](/operations/settings/settings#prefer_localhost_replica) -* `bytes_to_throw_insert` обрабатывается раньше, чем `bytes_to_delay_insert`, поэтому его не следует устанавливать в значение меньше значения `bytes_to_delay_insert`. - ::: - -**Пример** - -```sql -CREATE TABLE hits_all AS hits -ENGINE = Distributed(logs, default, hits[, sharding_key[, policy_name]]) -SETTINGS - fsync_after_insert=0, - fsync_directories=0; -``` - -Данные будут читаться со всех серверов в кластере `logs` из таблицы `default.hits`, расположенной на каждом сервере кластера. Данные не только читаются, но и частично обрабатываются на удалённых серверах (насколько это возможно). Например, для запроса с `GROUP BY` данные будут агрегироваться на удалённых серверах, а промежуточные состояния агрегатных функций будут отправлены на сервер, выполняющий запрос. Затем данные будут дополнительно агрегированы. - -Вместо имени базы данных вы можете использовать константное выражение, которое возвращает строку. Например: `currentDatabase()`. - - -## Кластеры {#distributed-clusters} - -Кластеры настраиваются в [конфигурационном файле сервера](../../../operations/configuration-files.md): - -```xml - - - - - - - - - - - 1 - - shard_01 - - false - - - 1 - example01-01-1 - 9000 - - - example01-01-2 - 9000 - - - - 2 - shard_02 - false - - example01-02-1 - 9000 - - - example01-02-2 - 1 - 9440 - - - - -``` - -Здесь определяется кластер с именем `logs`, который состоит из двух шардов, каждый из которых содержит по две реплики. Шарды — это серверы, в которых хранятся разные части данных (чтобы прочитать все данные, необходимо обратиться ко всем шардам). Реплики — это серверы-дубликаты (чтобы прочитать все данные, можно обратиться к данным на любой из реплик). - -Имена кластеров не должны содержать точек. - -Для каждого сервера указываются параметры `host`, `port`, а также, при необходимости, `user`, `password`, `secure`, `compression`, `bind_host`: - - -| Parameter | Description | Default Value | -|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| -| `host` | Адрес удалённого сервера. Можно использовать либо домен, либо IPv4- или IPv6-адрес. Если вы указываете домен, сервер выполняет DNS-запрос при запуске и сохраняет результат на всё время своей работы. Если DNS-запрос завершился с ошибкой, сервер не запускается. Если вы изменили DNS-запись, перезапустите сервер. | - | -| `port` | TCP-порт для работы сервера (`tcp_port` в конфигурации, обычно имеет значение 9000). Не путать с `http_port`. | - | -| `user` | Имя пользователя для подключения к удалённому серверу. У этого пользователя должны быть права на подключение к указанному серверу. Доступ настраивается в файле `users.xml`. Дополнительные сведения см. в разделе [Access rights](../../../guides/sre/user-management/index.md). | `default` | -| `password` | Пароль для подключения к удалённому серверу (не маскируется). | '' | -| `secure` | Использовать ли защищённое SSL/TLS-подключение. Обычно также требуется указать порт (порт по умолчанию для защищённого подключения — `9440`). Сервер должен слушать на `9440` и быть настроен с корректными сертификатами. | `false` | -| `compression` | Использовать сжатие данных. | `true` | -| `bind_host` | Исходный адрес, используемый при подключении к удалённому серверу с этого узла. Поддерживается только IPv4-адрес. Предназначен для продвинутых сценариев развертывания, когда требуется задать исходный IP-адрес, используемый распределёнными запросами ClickHouse. | - | - -При указании реплик одна из доступных реплик будет выбрана для каждого шарда при чтении. Можно настроить алгоритм балансировки нагрузки (предпочтения при выборе реплики для доступа) — см. настройку [load_balancing](../../../operations/settings/settings.md#load_balancing). Если не удалось установить подключение с сервером, выполняется попытка подключения с коротким таймаутом. Если подключиться не удалось, выбирается следующая реплика и так далее для всех реплик. Если попытка подключения не удалась для всех реплик, попытка будет повторена тем же образом несколько раз. Это повышает устойчивость, но не обеспечивает полной отказоустойчивости: удалённый сервер может принять подключение, но не работать или работать некорректно. - -Можно указать только один шард (в этом случае обработку запроса корректнее называть удалённой, а не распределённой) или любое количество шардов. В каждом шарде можно указать от одной до любого количества реплик. Можно задать разное количество реплик для каждого шарда. - -В конфигурации можно указать произвольное количество кластеров. - -Для просмотра ваших кластеров используйте таблицу `system.clusters`. - -Движок `Distributed` позволяет работать с кластером как с локальным сервером. Однако конфигурацию кластера нельзя задавать динамически — её необходимо настроить в конфигурационном файле сервера. Обычно все серверы в кластере имеют одинаковую конфигурацию кластера (хотя это и не обязательно). Кластеры из конфигурационного файла обновляются на лету, без перезапуска сервера. - -Если при каждом выполнении запроса нужно отправлять его в неизвестный заранее набор шардов и реплик, нет необходимости создавать таблицу `Distributed` — вместо этого используйте табличную функцию `remote`. См. раздел [Table functions](../../../sql-reference/table-functions/index.md). - - - -## Запись данных {#distributed-writing-data} - -Существует два способа записи данных в кластер: - -Во-первых, вы можете определить, на какие серверы записывать какие данные, и выполнять запись непосредственно на каждый шард. Другими словами, выполнять прямые запросы `INSERT` в удалённые таблицы в кластере, на которые указывает таблица `Distributed`. Это наиболее гибкое решение, поскольку вы можете использовать любую схему шардинга, даже нетривиальную, в силу требований предметной области. Это также наиболее оптимальное решение, так как данные могут записываться в разные шарды полностью независимо. - -Во-вторых, вы можете выполнять запросы `INSERT` в таблицу `Distributed`. В этом случае таблица будет самостоятельно распределять вставленные данные по серверам. Чтобы выполнять запись в таблицу `Distributed`, у неё должен быть настроен параметр `sharding_key` (кроме случая, когда существует только один шард). - -Для каждого шарда в конфигурационном файле может быть задан ``. По умолчанию вес равен `1`. Данные распределяются по шардам в объёме, пропорциональном весу шарда. Все веса шардов суммируются, затем вес каждого шарда делится на эту сумму для определения доли каждого шарда. Например, если есть два шарда и первый имеет вес 1, а второй — вес 2, первому будет отправляться одна треть (1 / 3) вставляемых строк, а второму — две трети (2 / 3). - -Для каждого шарда в конфигурационном файле может быть задан параметр `internal_replication`. Если этот параметр установлен в `true`, операция записи выбирает первую «здоровую» реплику и записывает данные в неё. Используйте это, если таблицы, лежащие в основе таблицы `Distributed`, являются реплицируемыми таблицами (например, любые движки таблиц `Replicated*MergeTree`). Одна из реплик таблицы получит запись, и она будет автоматически реплицирована на остальные реплики. - -Если `internal_replication` установлено в `false` (значение по умолчанию), данные записываются во все реплики. В этом случае таблица `Distributed` сама реплицирует данные. Это хуже, чем использование реплицируемых таблиц, потому что согласованность реплик не проверяется, и со временем они будут содержать немного отличающиеся данные. - -Для выбора шарда, на который будет отправлена строка данных, анализируется выражение шардинга и берётся остаток от его деления на суммарный вес шардов. Строка отправляется на шард, соответствующий полуинтервалу остатков от `prev_weights` до `prev_weights + weight`, где `prev_weights` — суммарный вес шардов с наименьшими номерами, а `weight` — вес данного шарда. Например, если есть два шарда, и первый имеет вес 9, а второй — вес 10, строка будет отправлена на первый шард для остатков из диапазона \[0, 9), и на второй — для остатков из диапазона \[9, 19). - -Выражение шардинга может быть любым выражением на основе констант и столбцов таблицы, которое возвращает целое число. Например, вы можете использовать выражение `rand()` для случайного распределения данных или `UserID` для распределения по остатку от деления идентификатора пользователя (тогда данные одного пользователя будут находиться на одном шарде, что упрощает выполнение `IN` и `JOIN` по пользователям). Если один из столбцов распределяется недостаточно равномерно, вы можете обернуть его в хеш-функцию, например `intHash64(UserID)`. - -Простой остаток от деления — ограниченное решение для шардинга и подходит не всегда. Он работает для средних и больших объёмов данных (десятки серверов), но не для очень больших объёмов данных (сотни серверов и более). В последнем случае используйте схему шардинга, требуемую предметной областью, вместо использования таблиц `Distributed`. - -О схеме шардинга следует задуматься в следующих случаях: - - - -- Используются запросы, которые выполняют объединение данных (`IN` или `JOIN`) по определённому ключу. Если данные разбиты на шарды по этому ключу, можно использовать локальные `IN` или `JOIN` вместо `GLOBAL IN` или `GLOBAL JOIN`, что значительно эффективнее. -- Используется большое количество серверов (сотни и более) с большим количеством небольших запросов, например запросов по данным отдельных клиентов (например, веб‑сайтов, рекламодателей или партнёров). Чтобы небольшие запросы не влияли на весь кластер, имеет смысл размещать данные одного клиента на одном шарде. В качестве альтернативы можно настроить двухуровневый шардинг: разделить весь кластер на «слои», где слой может состоять из нескольких шардов. Данные одного клиента располагаются в одном слое, но шарды можно добавлять в слой по мере необходимости, а данные распределяются между ними случайным образом. Для каждого слоя создаются таблицы `Distributed`, а для глобальных запросов создаётся одна общая распределённая таблица. - -Данные записываются в фоновом режиме. При вставке в таблицу блок данных просто записывается в локальную файловую систему. Данные отправляются на удалённые серверы в фоновом режиме как можно скорее. Периодичность отправки данных управляется настройками [distributed_background_insert_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms) и [distributed_background_insert_max_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms). Движок `Distributed` отправляет каждый файл с вставленными данными отдельно, но можно включить пакетную отправку файлов с помощью настройки [distributed_background_insert_batch](../../../operations/settings/settings.md#distributed_background_insert_batch). Эта настройка повышает производительность кластера за счёт более эффективного использования ресурсов локального сервера и сети. Следует проверять, были ли данные успешно отправлены, просматривая список файлов (данных, ожидающих отправки) в каталоге таблицы: `/var/lib/clickhouse/data/database/table/`. Количество потоков, выполняющих фоновые задачи, можно задать настройкой [background_distributed_schedule_pool_size](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size). - -Если сервер был аварийно остановлен или пережил жёсткую перезагрузку (например, из‑за сбоя оборудования) после выполнения `INSERT` в таблицу `Distributed`, вставленные данные могут быть потеряны. Если в каталоге таблицы обнаруживается повреждённая часть данных, она переносится в подкаталог `broken` и больше не используется. - - - -## Чтение данных {#distributed-reading-data} - -При выполнении запроса к таблице `Distributed` запросы `SELECT` отправляются на все шарды и выполняются независимо от того, как распределены данные по шардам (они могут быть распределены полностью случайным образом). При добавлении нового шарда нет необходимости переносить в него старые данные. Вместо этого можно записывать в него новые данные, используя более высокий вес — данные будут распределены немного неравномерно, но запросы по-прежнему будут выполняться корректно и эффективно. - -Когда включён параметр `max_parallel_replicas`, обработка запросов параллелизуется по всем репликам внутри одного шарда. Дополнительную информацию см. в разделе [max_parallel_replicas](../../../operations/settings/settings.md#max_parallel_replicas). - -Чтобы узнать больше о том, как обрабатываются распределённые запросы `in` и `global in`, см. [документацию](/sql-reference/operators/in#distributed-subqueries). - - - -## Виртуальные столбцы {#virtual-columns} - -#### _shard_num {#_shard_num} - -`_shard_num` — содержит значение `shard_num` из таблицы `system.clusters`. Тип: [UInt32](../../../sql-reference/data-types/int-uint.md). - -:::note -Поскольку табличные функции [`remote`](../../../sql-reference/table-functions/remote.md) и [`cluster`](../../../sql-reference/table-functions/cluster.md) внутренне создают временную таблицу `Distributed`, `_shard_num` доступен и в них. -::: - -**См. также** - -- Описание [виртуальных столбцов](../../../engines/table-engines/index.md#table_engines-virtual_columns) -- Настройка [`background_distributed_schedule_pool_size`](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size) -- Функции [`shardNum()`](../../../sql-reference/functions/other-functions.md#shardNum) и [`shardCount()`](../../../sql-reference/functions/other-functions.md#shardCount) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md deleted file mode 100644 index 32bd946ad29..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -description: 'Движки таблиц `Executable` и `ExecutablePool` позволяют создать - таблицу, строки которой генерируются скриптом, который вы задаёте (записывая строки - в **stdout**).' -sidebar_label: 'Executable/ExecutablePool' -sidebar_position: 40 -slug: /engines/table-engines/special/executable -title: 'Движки таблиц Executable и ExecutablePool' -doc_type: 'reference' ---- - -# Движки таблиц Executable и ExecutablePool {#executable-and-executablepool-table-engines} - -Движки таблиц `Executable` и `ExecutablePool` позволяют задать таблицу, строки которой генерируются скриптом, который вы пишете (путём вывода строк в **stdout**). Исполняемый скрипт хранится в каталоге `users_scripts` и может читать данные из любого источника. - -* Таблицы `Executable`: скрипт выполняется при каждом запросе -* Таблицы `ExecutablePool`: поддерживается пул долгоживущих процессов, и для чтения процессы берутся из этого пула - -Дополнительно вы можете указать один или несколько входных запросов, результаты которых будут передаваться в **stdin** для чтения скриптом. - -## Создание таблицы `Executable` {#creating-an-executable-table} - -Движку таблицы `Executable` требуются два параметра: имя скрипта и формат входящих данных. При необходимости можно передать один или несколько входных запросов: - -```sql -Executable(имя_скрипта, формат, [входной_запрос...]) -``` - -Ниже приведены соответствующие параметры для таблицы `Executable`: - -* `send_chunk_header` - * Description: Отправлять количество строк в каждом чанке перед отправкой чанка на обработку. Этот параметр позволяет писать скрипты более эффективно, заранее выделяя необходимые ресурсы - * Default value: false -* `command_termination_timeout` - * Description: Таймаут завершения команды в секундах - * Default value: 10 -* `command_read_timeout` - * Description: Таймаут чтения данных из stdout команды в миллисекундах - * Default value: 10000 -* `command_write_timeout` - * Description: Таймаут записи данных в stdin команды в миллисекундах - * Default value: 10000 - -Рассмотрим пример. Следующий скрипт на Python называется `my_script.py` и сохранён в папке `user_scripts`. Он считывает число `i` и выводит `i` случайных строк, при этом каждая строка предваряется числом, отделённым символом табуляции: - -```python -#!/usr/bin/python3 - -import sys -import string -import random - -def main(): - - # Чтение входного значения - for number in sys.stdin: - i = int(number) - - # Генерация случайных строк - for id in range(0, i): - letters = string.ascii_letters - random_string = ''.join(random.choices(letters ,k=10)) - print(str(id) + '\t' + random_string + '\n', end='') - - # Сброс результатов в stdout - sys.stdout.flush() - -if __name__ == "__main__": - main() -``` - -Таблица `my_executable_table` ниже создаётся на основе вывода `my_script.py`, который будет генерировать 10 случайных строк каждый раз, когда вы выполняете `SELECT` из `my_executable_table`: - -```sql -CREATE TABLE my_executable_table ( - x UInt32, - y String -) -ENGINE = Executable('my_script.py', TabSeparated, (SELECT 10)) -``` - -Создание таблицы завершается немедленно и не запускает скрипт. Выполнение запроса к `my_executable_table` приводит к запуску скрипта: - -```sql -SELECT * FROM my_executable_table -``` - -```response -┌─x─┬─y──────────┐ -│ 0 │ BsnKBsNGNH │ -│ 1 │ mgHfBCUrWM │ -│ 2 │ iDQAVhlygr │ -│ 3 │ uNGwDuXyCk │ -│ 4 │ GcFdQWvoLB │ -│ 5 │ UkciuuOTVO │ -│ 6 │ HoKeCdHkbs │ -│ 7 │ xRvySxqAcR │ -│ 8 │ LKbXPHpyDI │ -│ 9 │ zxogHTzEVV │ -└───┴────────────┘ -``` - -## Передача результатов запроса в скрипт {#passing-query-results-to-a-script} - -Пользователи веб‑сайта Hacker News оставляют комментарии. В Python есть набор инструментов для обработки естественного языка (`nltk`) с `SentimentIntensityAnalyzer` для определения, являются ли комментарии положительными, отрицательными или нейтральными, — в том числе для присвоения значения от -1 (очень негативный комментарий) до 1 (очень позитивный комментарий). Давайте создадим таблицу `Executable`, которая вычисляет тональность комментариев Hacker News с помощью `nltk`. - -В этом примере используется таблица `hackernews`, описанная [здесь](/engines/table-engines/mergetree-family/invertedindexes/#hacker-news-dataset). Таблица `hackernews` включает столбец `id` типа `UInt64` и столбец типа `String` с именем `comment`. Начнём с определения таблицы `Executable`: - -```sql -CREATE TABLE sentiment ( - id UInt64, - sentiment Float32 -) -ENGINE = Executable( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20) -); -``` - -Некоторые комментарии о таблице `sentiment`: - -* Файл `sentiment.py` сохранён в каталоге `user_scripts` (каталог по умолчанию для настройки `user_scripts_path`) -* Формат `TabSeparated` означает, что наш Python-скрипт должен генерировать строки сырых данных, содержащие значения, разделённые символом табуляции -* Запрос выбирает два столбца из `hackernews`. Python-скрипту потребуется извлекать значения этих столбцов из входящих строк - -Ниже приведено определение `sentiment.py`: - -```python -#!/usr/local/bin/python3.9 - -import sys -import nltk -from nltk.sentiment import SentimentIntensityAnalyzer - -def main(): - sentiment_analyzer = SentimentIntensityAnalyzer() - - while True: - try: - row = sys.stdin.readline() - if row == '': - break - - split_line = row.split("\t") - - id = str(split_line[0]) - comment = split_line[1] - - score = sentiment_analyzer.polarity_scores(comment)['compound'] - print(id + '\t' + str(score) + '\n', end='') - sys.stdout.flush() - except BaseException as x: - break - -if __name__ == "__main__": - main() -``` - -Несколько комментариев к нашему Python-скрипту: - -* Чтобы это работало, вам нужно выполнить `nltk.downloader.download('vader_lexicon')`. Это можно было бы поместить в сам скрипт, но тогда загрузка выполнялась бы при каждом выполнении запроса к таблице `sentiment`, что неэффективно -* Каждое значение `row` будет строкой в результирующем наборе запроса `SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20` -* Входящая строка имеет табличный формат (значения разделены табуляцией), поэтому мы извлекаем `id` и `comment`, используя Python-функцию `split` -* Результат `polarity_scores` — это JSON-объект с несколькими значениями. Мы решили просто взять значение `compound` из этого JSON-объекта -* Напомним, что таблица `sentiment` в ClickHouse использует формат `TabSeparated` и содержит два столбца, поэтому наша функция `print` разделяет эти столбцы символом табуляции - -Каждый раз, когда вы пишете запрос, выбирающий строки из таблицы `sentiment`, выполняется запрос `SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20`, а его результат передаётся в `sentiment.py`. Давайте протестируем это: - -```sql -SELECT * -FROM sentiment -``` - -Ответ будет выглядеть следующим образом: - -```response -┌───────id─┬─sentiment─┐ -│ 7398199 │ 0.4404 │ -│ 21640317 │ 0.1779 │ -│ 21462000 │ 0 │ -│ 25168863 │ 0 │ -│ 25168978 │ -0.1531 │ -│ 25169359 │ 0 │ -│ 25169394 │ -0.9231 │ -│ 25169766 │ 0.4137 │ -│ 25172570 │ 0.7469 │ -│ 25173687 │ 0.6249 │ -│ 28291534 │ 0 │ -│ 28291669 │ -0.4767 │ -│ 28291731 │ 0 │ -│ 28291949 │ -0.4767 │ -│ 28292004 │ 0.3612 │ -│ 28292050 │ -0.296 │ -│ 28292322 │ 0 │ -│ 28295172 │ 0.7717 │ -│ 28295288 │ 0.4404 │ -│ 21465723 │ -0.6956 │ -└──────────┴───────────┘ -``` - -## Создание таблицы `ExecutablePool` {#creating-an-executablepool-table} - -Синтаксис `ExecutablePool` похож на `Executable`, но у таблицы `ExecutablePool` есть несколько параметров, характерных именно для нее: - -* `pool_size` - * Описание: Размер пула процессов. Если размер равен 0, ограничения по размеру отсутствуют. - * Значение по умолчанию: 16 -* `max_command_execution_time` - * Описание: Максимальное время выполнения команды в секундах. - * Значение по умолчанию: 10 - -Мы можем легко преобразовать таблицу `sentiment`, показанную выше, для использования `ExecutablePool` вместо `Executable`: - -```sql -CREATE TABLE sentiment_pooled ( - id UInt64, - sentiment Float32 -) -ENGINE = ExecutablePool( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20000) -) -SETTINGS - pool_size = 4; -``` - -ClickHouse будет по требованию поддерживать в работе 4 процесса, когда клиент выполняет запрос к таблице `sentiment_pooled`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md deleted file mode 100644 index bec15dea888..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'ClickHouse позволяет отправлять на сервер вместе с запросом `SELECT` данные, необходимые для обработки запроса. Эти данные помещаются во временную таблицу и могут использоваться в запросе (например, в операторах `IN`).' -sidebar_label: 'Внешние данные для обработки запросов' -sidebar_position: 130 -slug: /engines/table-engines/special/external-data -title: 'Внешние данные для обработки запросов' -doc_type: 'reference' ---- - -# Внешние данные для обработки запросов {#external-data-for-query-processing} - -ClickHouse позволяет отправлять на сервер данные, которые необходимы для обработки запроса, вместе с запросом `SELECT`. Эти данные помещаются во временную таблицу (см. раздел «Временные таблицы») и могут использоваться в запросе (например, в операторах `IN`). - -Например, если у вас есть текстовый файл с важными идентификаторами пользователей, вы можете загрузить его на сервер вместе с запросом, который использует фильтрацию по этому списку. - -Если вам нужно выполнить более одного запроса с большим объёмом внешних данных, не используйте эту возможность. Лучше заранее загрузить данные в БД. - -Внешние данные могут быть загружены с помощью клиента командной строки (в неинтерактивном режиме) или с использованием HTTP-интерфейса. - -В клиенте командной строки вы можете указать секцию параметров в формате - -```bash ---external --file=... [--name=...] [--format=...] [--types=...|--structure=...] -``` - -У вас может быть несколько таких секций — по числу передаваемых таблиц. - -**–external** – Обозначает начало секции. -**–file** – Путь к файлу с дампом таблицы или `-`, что означает stdin. -Из stdin может быть получена только одна таблица. - -Следующие параметры являются необязательными: **–name** – Имя таблицы. Если параметр не указан, используется _data. -**–format** – Формат данных в файле. Если параметр не указан, используется TabSeparated. - -Один из следующих параметров является обязательным: **–types** – Список типов столбцов, разделённых запятыми. Например: `UInt64,String`. Столбцы будут иметь имена _1, _2, ... -**–structure** – Структура таблицы в формате `UserID UInt64`, `URL String`. Определяет имена и типы столбцов. - -Файлы, указанные в параметре 'file', будут разобраны с использованием формата, указанного в 'format', и типов данных, указанных в 'types' или 'structure'. Таблица будет загружена на сервер и доступна там как временная таблица с именем из параметра 'name'. - -Примеры: - -```bash -$ echo -ne "1\n2\n3\n" | clickhouse-client --query="SELECT count() FROM test.visits WHERE TraficSourceID IN _data" --external --file=- --types=Int8 -849897 -$ cat /etc/passwd | sed 's/:/\t/g' | clickhouse-client --query="SELECT shell, count() AS c FROM passwd GROUP BY shell ORDER BY c DESC" --external --file=- --name=passwd --structure='login String, unused String, uid UInt16, gid UInt16, comment String, home String, shell String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -При использовании HTTP-интерфейса внешние данные передаются в формате multipart/form-data. Каждая таблица передаётся как отдельный файл. Имя таблицы берётся из имени файла. В `query_string` передаются параметры `name_format`, `name_types` и `name_structure`, где `name` — это имя таблицы, к которой относятся эти параметры. Значения параметров такие же, как при использовании клиентской программы командной строки. - -Пример: - -```bash -$ cat /etc/passwd | sed 's/:/\t/g' > passwd.tsv - -$ curl -F 'passwd=@passwd.tsv;' 'http://localhost:8123/?query=SELECT+shell,+count()+AS+c+FROM+passwd+GROUP+BY+shell+ORDER+BY+c+DESC&passwd_structure=login+String,+unused+String,+uid+UInt16,+gid+UInt16,+comment+String,+home+String,+shell+String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -Для распределённой обработки запросов временные таблицы отправляются на все удалённые серверы. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md deleted file mode 100644 index e2fe7d77234..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: 'Табличный движок File хранит данные в файле в одном из поддерживаемых - форматов (`TabSeparated`, `Native` и т. д.).' -sidebar_label: 'File' -sidebar_position: 40 -slug: /engines/table-engines/special/file -title: 'Табличный движок File' -doc_type: 'reference' ---- - - - -# Движок таблицы File {#file-table-engine} - -Движок таблицы File хранит данные в файле в одном из поддерживаемых [форматов файлов](/interfaces/formats#formats-overview) (`TabSeparated`, `Native` и т. д.). - -Сценарии использования: - -- Экспорт данных из ClickHouse в файл. -- Преобразование данных из одного формата в другой. -- Обновление данных в ClickHouse путём редактирования файла на диске. - -:::note -Этот движок в настоящее время недоступен в ClickHouse Cloud, пожалуйста, [используйте вместо него табличную функцию S3](/sql-reference/table-functions/s3.md). -::: - - - -## Использование на сервере ClickHouse {#usage-in-clickhouse-server} - -```sql -File(Format) -``` - -Параметр `Format` задаёт один из доступных форматов файлов. Для выполнения -запросов `SELECT` формат должен поддерживать чтение (input), а для выполнения -запросов `INSERT` — запись (output). Доступные форматы перечислены в разделе -[Formats](/interfaces/formats#formats-overview). - -ClickHouse не позволяет указывать путь в файловой системе для `File`. Будет использована папка, определённая настройкой [path](../../../operations/server-configuration-parameters/settings.md) в конфигурации сервера. - -При создании таблицы с использованием `File(Format)` в этой папке создаётся пустая поддиректория. Когда данные записываются в эту таблицу, они помещаются в файл `data.Format` в этой поддиректории. - -Вы можете вручную создать эту поддиректорию и файл в файловой системе сервера, а затем [ATTACH](../../../sql-reference/statements/attach.md) их к информации о таблице с совпадающим именем, чтобы иметь возможность выполнять запросы к данным в этом файле. - -:::note -Будьте осторожны с этой функциональностью, так как ClickHouse не отслеживает внешние изменения таких файлов. Результат одновременной записи через ClickHouse и вне ClickHouse неопределён. -::: - - -## Пример {#example} - -**1.** Настройте таблицу `file_engine_table`: - -```sql -CREATE TABLE file_engine_table (name String, value UInt32) ENGINE=File(TabSeparated) -``` - -По умолчанию ClickHouse создаст папку `/var/lib/clickhouse/data/default/file_engine_table`. - -**2.** Вручную создайте файл `/var/lib/clickhouse/data/default/file_engine_table/data.TabSeparated` со следующим содержимым: - -```bash -$ cat data.TabSeparated -one 1 -two 2 -``` - -**3.** Выполните запрос к данным: - -```sql -SELECT * FROM file_engine_table -``` - -```text -┌─name─┬─value─┐ -│ one │ 1 │ -│ two │ 2 │ -└──────┴───────┘ -``` - - -## Использование в clickhouse-local {#usage-in-clickhouse-local} - -В [clickhouse-local](../../../operations/utilities/clickhouse-local.md) движок File, помимо параметра `Format`, принимает путь к файлу. Потоки ввода/вывода по умолчанию можно указывать с помощью числовых или понятных имён, таких как `0` или `stdin`, `1` или `stdout`. Можно читать и записывать сжатые файлы, исходя из дополнительного параметра движка или расширения файла (`gz`, `br` или `xz`). - -**Пример:** - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -q "CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); SELECT a, b FROM table; DROP TABLE table" -``` - - -## Подробности реализации {#details-of-implementation} - -- Несколько запросов `SELECT` могут выполняться одновременно, но запросы `INSERT` выполняются последовательно. -- Поддерживается создание нового файла с помощью запроса `INSERT`. -- Если файл существует, `INSERT` будет дописывать в него новые значения. -- Не поддерживаются: - - `ALTER` - - `SELECT ... SAMPLE` - - Индексы - - Репликация - - - -## PARTITION BY {#partition-by} - -`PARTITION BY` — необязательное выражение. Можно создавать отдельные файлы, разбивая данные на партиции по ключу партиционирования (partition key). В большинстве случаев ключ партиционирования не нужен, а если он и требуется, как правило, нет необходимости делать его более детализированным, чем до уровня месяца. Партиционирование не ускоряет выполнение запросов (в отличие от выражения ORDER BY). Никогда не используйте слишком мелкое партиционирование. Не разделяйте данные на партиции по идентификаторам или именам клиентов (вместо этого сделайте идентификатор или имя клиента первым столбцом в выражении ORDER BY). - -Для партиционирования по месяцам используйте выражение `toYYYYMM(date_column)`, где `date_column` — это столбец с датой типа данных [Date](/sql-reference/data-types/date.md). Имена партиций в этом случае имеют формат `"YYYYMM"`. - - - -## Виртуальные столбцы {#virtual-columns} - -- `_path` — Путь к файлу. Тип: `LowCardinality(String)`. -- `_file` — Имя файла. Тип: `LowCardinality(String)`. -- `_size` — Размер файла в байтах. Тип: `Nullable(UInt64)`. Если размер неизвестен, значение — `NULL`. -- `_time` — Время последнего изменения файла. Тип: `Nullable(DateTime)`. Если время неизвестно, значение — `NULL`. - - - -## Настройки {#settings} - -- [engine_file_empty_if_not_exists](/operations/settings/settings#engine_file_empty_if_not_exists) — позволяет выполнять выборку из несуществующего файла, возвращая пустой набор данных. По умолчанию отключена. -- [engine_file_truncate_on_insert](/operations/settings/settings#engine_file_truncate_on_insert) — позволяет усекать файл перед вставкой в него данных. По умолчанию отключена. -- [engine_file_allow_create_multiple_files](/operations/settings/settings.md#engine_file_allow_create_multiple_files) — позволяет создавать новый файл при каждой вставке, если формат имеет суффикс. По умолчанию отключена. -- [engine_file_skip_empty_files](/operations/settings/settings.md#engine_file_skip_empty_files) — позволяет пропускать пустые файлы при чтении. По умолчанию отключена. -- [storage_file_read_method](/operations/settings/settings#engine_file_empty_if_not_exists) — метод чтения данных из файла хранилища, один из: `read`, `pread`, `mmap`. Метод `mmap` не применяется к clickhouse-server (предназначен для clickhouse-local). Значение по умолчанию: `pread` для clickhouse-server, `mmap` для clickhouse-local. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md deleted file mode 100644 index cc1006fcc4f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -description: 'Этот движок позволяет обрабатывать файлы журналов приложений как поток - записей.' -sidebar_label: 'FileLog' -sidebar_position: 160 -slug: /engines/table-engines/special/filelog -title: 'Движок таблиц FileLog' -doc_type: 'reference' ---- - - - -# Движок таблицы FileLog {#filelog-engine} - -Этот движок позволяет обрабатывать файлы журналов приложений как поток записей. - -`FileLog` позволяет: - -- Подписываться на файлы журналов. -- Обрабатывать новые записи по мере их добавления в подписанные файлы журналов. - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = FileLog('path_to_logs', 'format_name') SETTINGS - [poll_timeout_ms = 0,] - [poll_max_batch_size = 0,] - [max_block_size = 0,] - [max_threads = 0,] - [poll_directory_watch_events_backoff_init = 500,] - [poll_directory_watch_events_backoff_max = 32000,] - [poll_directory_watch_events_backoff_factor = 2,] - [handle_error_mode = 'default'] -``` - -Аргументы движка: - -* `path_to_logs` – Путь к лог-файлам для чтения. Это может быть путь к каталогу с лог-файлами или к отдельному лог-файлу. Обратите внимание, что ClickHouse допускает только пути внутри каталога `user_files`. -* `format_name` - Формат записей. Учтите, что FileLog обрабатывает каждую строку в файле как отдельную запись, и не все форматы данных подходят для этого. - -Необязательные параметры: - -* `poll_timeout_ms` - Тайм-аут для одного опроса лог-файла. Значение по умолчанию: [stream_poll_timeout_ms](../../../operations/settings/settings.md#stream_poll_timeout_ms). -* `poll_max_batch_size` — Максимальное количество записей, считываемых за один опрос. Значение по умолчанию: [max_block_size](/operations/settings/settings#max_block_size). -* `max_block_size` — Максимальный размер пакета (в записях) для одного опроса. Значение по умолчанию: [max_insert_block_size](../../../operations/settings/settings.md#max_insert_block_size). -* `max_threads` - Максимальное количество потоков для разбора файлов, по умолчанию `0`, что означает значение max(1, physical_cpu_cores / 4). -* `poll_directory_watch_events_backoff_init` - Начальное значение паузы для потока, отслеживающего каталог. Значение по умолчанию: `500`. -* `poll_directory_watch_events_backoff_max` - Максимальное значение паузы для потока, отслеживающего каталог. Значение по умолчанию: `32000`. -* `poll_directory_watch_events_backoff_factor` - Коэффициент увеличения паузы (backoff), по умолчанию экспоненциальный. Значение по умолчанию: `2`. -* `handle_error_mode` — Как обрабатывать ошибки в движке FileLog. Возможные значения: default (будет сгенерировано исключение, если не удалось разобрать сообщение), stream (сообщение об исключении и исходное сообщение будут сохранены во виртуальных столбцах `_error` и `_raw_message`). - - -## Описание {#description} - -Поступившие записи отслеживаются автоматически, поэтому каждая запись в файле журнала учитывается только один раз. - -`SELECT` не особенно полезен для чтения записей (кроме отладки), потому что каждую запись можно прочитать только один раз. Гораздо практичнее создавать потоки в реальном времени, используя [материализованные представления](../../../sql-reference/statements/create/view.md). Для этого: - -1. Используйте движок для создания таблицы FileLog и рассматривайте её как поток данных. -2. Создайте таблицу с требуемой структурой. -3. Создайте материализованное представление, которое преобразует данные из движка и помещает их в ранее созданную таблицу. - -Когда `MATERIALIZED VIEW` подключается к движку, оно начинает собирать данные в фоновом режиме. Это позволяет постоянно получать записи из файлов журналов и преобразовывать их в нужный формат с помощью `SELECT`. -Одна таблица FileLog может иметь сколько угодно материализованных представлений; они не читают данные из таблицы напрямую, а получают новые записи (блоками), таким образом можно записывать данные в несколько таблиц с разным уровнем детализации (с группировкой — агрегацией и без неё). - -Пример: - -```sql - CREATE TABLE logs ( - timestamp UInt64, - level String, - message String - ) ENGINE = FileLog('user_files/my_app/app.log', 'JSONEachRow'); - - CREATE TABLE daily ( - day Date, - level String, - total UInt64 - ) ENGINE = SummingMergeTree(day, (day, level), 8192); - - CREATE MATERIALIZED VIEW consumer TO daily - AS SELECT toDate(toDateTime(timestamp)) AS day, level, count() AS total - FROM queue GROUP BY day, level; - - SELECT level, sum(total) FROM daily GROUP BY level; -``` - -Чтобы прекратить приём потоковых данных или изменить логику преобразования, отсоедините материализованное представление: - -```sql - DETACH TABLE consumer; - ATTACH TABLE consumer; -``` - -Если вы хотите изменить целевую таблицу с помощью команды `ALTER`, мы рекомендуем предварительно отключить материализованное представление, чтобы избежать расхождений между целевой таблицей и данными из представления. - - -## Виртуальные столбцы {#virtual-columns} - -- `_filename` — Имя файла журнала. Тип данных: `LowCardinality(String)`. -- `_offset` — Смещение в файле журнала. Тип данных: `UInt64`. - -Дополнительные виртуальные столбцы при `handle_error_mode='stream'`: - -- `_raw_record` — Исходная запись, которую не удалось разобрать. Тип данных: `Nullable(String)`. -- `_error` — Сообщение об исключении, возникшем при неудачном разборе. Тип данных: `Nullable(String)`. - -Примечание: виртуальные столбцы `_raw_record` и `_error` заполняются только в случае исключения во время разбора; при успешном разборе сообщения они всегда имеют значение `NULL`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md deleted file mode 100644 index 9d9543fe694..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: 'Движок таблицы GenerateRandom генерирует случайные данные для заданной - схемы таблицы.' -sidebar_label: 'GenerateRandom' -sidebar_position: 140 -slug: /engines/table-engines/special/generate -title: 'Движок таблицы GenerateRandom' -doc_type: 'reference' ---- - - - -# Движок таблицы GenerateRandom {#generaterandom-table-engine} - -Движок таблицы GenerateRandom генерирует случайные данные в соответствии с заданной схемой таблицы. - -Примеры использования: - -- Используйте в тестах для заполнения больших таблиц воспроизводимыми данными. -- Генерируйте случайные входные данные для фаззинговых тестов. - - - -## Использование в ClickHouse Server {#usage-in-clickhouse-server} - -```sql -ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) -``` - -Параметры `max_array_length` и `max_string_length` задают соответственно максимальную длину всех столбцов типов Array или Map и строк в генерируемых данных. - -Движок таблицы `Generate` поддерживает только запросы `SELECT`. - -Он поддерживает все [типы данных](../../../sql-reference/data-types/index.md), которые могут храниться в таблице, за исключением `AggregateFunction`. - - -## Пример {#example} - -**1.** Создайте таблицу `generate_engine_table`: - -```sql -CREATE TABLE generate_engine_table (name String, value UInt32) ENGINE = GenerateRandom(1, 5, 3) -``` - -**2.** Выполните запрос: - -```sql -SELECT * FROM generate_engine_table LIMIT 3 -``` - -```text -┌─name─┬──────value─┐ -│ c4xJ │ 1412771199 │ -│ r │ 1791099446 │ -│ 7#$ │ 124312908 │ -└──────┴────────────┘ -``` - - -## Подробности реализации {#details-of-implementation} - -- Не поддерживаются: - - `ALTER` - - `SELECT ... SAMPLE` - - `INSERT` - - Индексы - - Репликация diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md deleted file mode 100644 index ec76dffe7d2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'Документация по специальным движкам таблиц' -sidebar_label: 'Специальные' -sidebar_position: 50 -slug: /engines/table-engines/special/ -title: 'Специальные движки таблиц' -doc_type: 'reference' ---- - -# Специальные движки таблиц {#special-table-engines} - -Существует три основные категории движков таблиц: - -* [Семейство движков MergeTree](../../../engines/table-engines/mergetree-family/index.md) для основного использования в продакшене. -* [Семейство движков Log](../../../engines/table-engines/log-family/index.md) для небольших временных данных. -* [Движки таблиц для интеграций](../../../engines/table-engines/integrations/index.md). - -Оставшиеся движки уникальны по своему назначению и пока не объединены в семейства, поэтому они отнесены к этой «специальной» категории. - -{/* Оглавление для этой страницы генерируется автоматически с помощью - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - на основе полей YAML front matter: slug, description, title. - - Если вы заметили ошибку, отредактируйте YAML front matter соответствующих страниц. - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [Alias table engine](/engines/table-engines/special/alias) | Движок таблиц Alias создает прозрачный прокси для другой таблицы. Все операции перенаправляются в целевую таблицу, при этом алиас сам по себе не хранит данные. | -| [Distributed table engine](/engines/table-engines/special/distributed) | Таблицы с движком Distributed не хранят собственные данные, но позволяют выполнять распределенную обработку запросов на нескольких серверах. Чтение автоматически параллелизуется. При чтении используются индексы таблиц на удаленных серверах, если они есть. | -| [Dictionary table engine](/engines/table-engines/special/dictionary) | Движок `Dictionary` отображает данные словаря в виде таблицы ClickHouse. | -| [Merge table engine](/engines/table-engines/special/merge) | Движок `Merge` (не путать с `MergeTree`) сам по себе не хранит данные, но позволяет одновременно читать из любого количества других таблиц. | -| [Executable and ExecutablePool table engines](/engines/table-engines/special/executable) | Движки таблиц `Executable` и `ExecutablePool` позволяют определить таблицу, строки которой генерируются заданным вами скриптом (путем записи строк в **stdout**). | -| [File table engine](/engines/table-engines/special/file) | Движок таблиц File хранит данные в файле в одном из поддерживаемых форматов (`TabSeparated`, `Native` и т. д.). | -| [Null table engine](/engines/table-engines/special/null) | При записи в таблицу `Null` данные игнорируются. При чтении из таблицы `Null` результат пуст. | -| [Set table engine](/engines/table-engines/special/set) | Набор данных, который всегда находится в оперативной памяти (RAM). Предназначен для использования в правой части оператора `IN`. | -| [Join table engine](/engines/table-engines/special/join) | Опциональная предварительно подготовленная структура данных для использования в операциях JOIN. | -| [URL table engine](/engines/table-engines/special/url) | Выполняет запросы данных к удаленному HTTP/HTTPS-серверу и с него. Этот движок похож на движок File. | -| [View table engine](/engines/table-engines/special/view) | Используется для реализации представлений (подробнее см. запрос `CREATE VIEW`). Он не хранит данные, а только сохраняет указанный запрос `SELECT`. При чтении из таблицы выполняется этот запрос (и из него удаляются все ненужные столбцы). | -| [Memory table engine](/engines/table-engines/special/memory) | Движок Memory хранит данные в оперативной памяти (RAM) в несжатом виде. Данные хранятся в точности в том виде, в каком они получены при чтении. Другими словами, чтение из этой таблицы практически бесплатное. | -| [Buffer table engine](/engines/table-engines/special/buffer) | Буферизует данные для записи в оперативной памяти, периодически сбрасывая их в другую таблицу. При чтении данные считываются одновременно из буфера и из другой таблицы. | -| [External data for query processing](/engines/table-engines/special/external-data) | ClickHouse позволяет отправлять на сервер данные, необходимые для обработки запроса, вместе с запросом `SELECT`. Эти данные помещаются во временную таблицу и могут использоваться в запросе (например, в операторах `IN`). | -| [GenerateRandom table engine](/engines/table-engines/special/generate) | Движок таблиц GenerateRandom генерирует случайные данные для заданной схемы таблицы. | -| [KeeperMap table engine](/engines/table-engines/special/keeper-map) | Этот движок позволяет использовать кластер Keeper/ZooKeeper как согласованное key-value-хранилище с линеаризуемыми записями и последовательно согласованными чтениями. | -| [FileLog table engine](/engines/table-engines/special/filelog) | Этот движок позволяет обрабатывать файлы журналов приложений как поток записей. | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md deleted file mode 100644 index ad165544a38..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ /dev/null @@ -1,176 +0,0 @@ ---- -description: 'Необязательная подготовленная структура данных для использования в операциях JOIN.' -sidebar_label: 'Join' -sidebar_position: 70 -slug: /engines/table-engines/special/join -title: 'Движок таблицы Join' -doc_type: 'reference' ---- - - - -# Табличный движок Join {#join-table-engine} - -Дополнительная подготовленная структура данных для использования в операциях [JOIN](/sql-reference/statements/select/join). - -:::note -В ClickHouse Cloud, если ваш сервис был создан на версии раньше 25.4, необходимо установить параметр `compatibility` не ниже 25.4, выполнив команду `SET compatibility=25.4`. -::: - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1] [TTL expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2] [TTL expr2], -) ENGINE = Join(join_strictness, join_type, k1[, k2, ...]) -``` - -См. подробное описание запроса [CREATE TABLE](/sql-reference/statements/create/table). - - -## Параметры движка {#engine-parameters} - -### `join_strictness` {#join_strictness} - -`join_strictness` – [строгость JOIN](/sql-reference/statements/select/join#supported-types-of-join). - -### `join_type` {#join_type} - -`join_type` – [тип JOIN](/sql-reference/statements/select/join#supported-types-of-join). - -### Ключевые столбцы {#key-columns} - -`k1[, k2, ...]` – ключевые столбцы из предложения `USING`, по которым выполняется операция `JOIN`. - -Задавайте параметры `join_strictness` и `join_type` без кавычек, например `Join(ANY, LEFT, col1)`. Они должны соответствовать операции `JOIN`, для которой будет использоваться таблица. Если параметры не соответствуют, ClickHouse не выбрасывает исключение и может вернуть некорректные данные. - - - -## Особенности и рекомендации {#specifics-and-recommendations} - -### Хранение данных {#data-storage} - -Данные таблицы `Join` всегда находятся в оперативной памяти. При вставке строк в таблицу ClickHouse записывает блоки данных в каталог на диске, чтобы их можно было восстановить при перезапуске сервера. - -Если сервер перезапускается некорректно, блок данных на диске может быть потерян или повреждён. В этом случае может потребоваться вручную удалить файл с повреждёнными данными. - -### Выборка и вставка данных {#selecting-and-inserting-data} - -Вы можете использовать запросы `INSERT` для добавления данных в таблицы с движком `Join`. Если таблица была создана с режимом строгости `ANY`, данные для дублирующихся ключей игнорируются. При режиме строгости `ALL` добавляются все строки. - -Основные варианты использования таблиц с движком `Join`: - -- Использовать таблицу справа в выражении `JOIN`. -- Вызывать функцию [joinGet](/sql-reference/functions/other-functions.md/#joinGet), которая позволяет извлекать данные из таблицы так же, как из словаря. - -### Удаление данных {#deleting-data} - -Запросы `ALTER DELETE` для таблиц с движком `Join` реализованы как [мутации](/sql-reference/statements/alter/index.md#mutations). Мутация `DELETE` считывает отфильтрованные данные и перезаписывает данные в памяти и на диске. - -### Ограничения и настройки {#join-limitations-and-settings} - -При создании таблицы применяются следующие настройки: - -#### `join_use_nulls` {#join_use_nulls} - -[join_use_nulls](/operations/settings/settings.md/#join_use_nulls) - -#### `max_rows_in_join` {#max_rows_in_join} - -[max_rows_in_join](/operations/settings/settings#max_rows_in_join) - -#### `max_bytes_in_join` {#max_bytes_in_join} - -[max_bytes_in_join](/operations/settings/settings#max_bytes_in_join) - -#### `join_overflow_mode` {#join_overflow_mode} - -[join_overflow_mode](/operations/settings/settings#join_overflow_mode) - -#### `join_any_take_last_row` {#join_any_take_last_row} - -[join_any_take_last_row](/operations/settings/settings.md/#join_any_take_last_row) -#### `join_use_nulls` {#join_use_nulls-1} - -#### Persistent {#persistent} - -Отключает персистентность для движков таблиц Join и [Set](/engines/table-engines/special/set.md). - -Снижает нагрузку на подсистему ввода-вывода. Подходит для сценариев, ориентированных на производительность и не требующих персистентности. - -Возможные значения: - -- 1 — включено. -- 0 — выключено. - -Значение по умолчанию: `1`. - -Таблицы с движком `Join` не могут использоваться в операциях `GLOBAL JOIN`. - -Движок `Join` позволяет указать настройку [join_use_nulls](/operations/settings/settings.md/#join_use_nulls) в операторе `CREATE TABLE`. Запрос [SELECT](/sql-reference/statements/select/index.md) должен иметь то же значение `join_use_nulls`. - - - -## Примеры использования {#example} - -Создание левой таблицы: - -```sql -CREATE TABLE id_val(`id` UInt32, `val` UInt32) ENGINE = TinyLog; -``` - -```sql -INSERT INTO id_val VALUES (1,11)(2,12)(3,13); -``` - -Создание правой таблицы для операции `JOIN`: - -```sql -CREATE TABLE id_val_join(`id` UInt32, `val` UInt8) ENGINE = Join(ANY, LEFT, id); -``` - -```sql -INSERT INTO id_val_join VALUES (1,21)(1,22)(3,23); -``` - -Объединение таблиц: - -```sql -SELECT * FROM id_val ANY LEFT JOIN id_val_join USING (id); -``` - -```text -┌─id─┬─val─┬─id_val_join.val─┐ -│ 1 │ 11 │ 21 │ -│ 2 │ 12 │ 0 │ -│ 3 │ 13 │ 23 │ -└────┴─────┴─────────────────┘ -``` - -В качестве альтернативы вы можете получить данные из таблицы `Join`, указав значение ключа соединения: - -```sql -SELECT joinGet('id_val_join', 'val', toUInt32(1)); -``` - -```text -┌─joinGet('id_val_join', 'val', toUInt32(1))─┐ -│ 21 │ -└────────────────────────────────────────────┘ -``` - -Удаление строки из таблицы `Join`: - -```sql -ALTER TABLE id_val_join DELETE WHERE id = 3; -``` - -```text -┌─id─┬─val─┐ -│ 1 │ 21 │ -└────┴─────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md deleted file mode 100644 index 9dc42339bda..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: 'Этот движок позволяет использовать кластер Keeper/ZooKeeper как согласованное - хранилище пар ключ–значение с линеаризуемыми записями и последовательной согласованностью при чтении.' -sidebar_label: 'KeeperMap' -sidebar_position: 150 -slug: /engines/table-engines/special/keeper-map -title: 'Табличный движок KeeperMap' -doc_type: 'reference' ---- - - - -# Табличный движок KeeperMap {#keepermap-table-engine} - -Этот движок позволяет использовать кластер Keeper/ZooKeeper как согласованное хранилище ключ–значение с линеаризуемыми записями и последовательно согласованными чтениями. - -Чтобы включить табличный движок KeeperMap, необходимо задать в конфигурации параметр ``, определяющий путь в ZooKeeper, по которому будут храниться таблицы. - -Например: - -```xml - - /keeper_map_tables - -``` - -где path может быть любым другим допустимым путем в ZooKeeper. - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = KeeperMap(root_path, [keys_limit]) PRIMARY KEY(primary_key_name) -``` - -Параметры движка: - -* `root_path` — путь в ZooKeeper, по которому будет храниться `table_name`.\ - Этот путь не должен содержать префикс, определённый в конфигурации ``, поскольку префикс будет автоматически добавлен к `root_path`.\ - Также поддерживается формат `auxiliary_zookeeper_cluster_name:/some/path`, где `auxiliary_zookeeper_cluster` — это кластер ZooKeeper, определённый в конфигурации ``.\ - По умолчанию используется кластер ZooKeeper, определённый в конфигурации ``. -* `keys_limit` — максимальное количество ключей, разрешённых в таблице.\ - Это мягкое ограничение, и в некоторых крайних случаях в таблице может оказаться большее количество ключей. -* `primary_key_name` — любое имя столбца из списка столбцов. -* `primary key` должен быть указан; он может содержать только один столбец. Первичный ключ будет сериализован в бинарном виде как `node name` в ZooKeeper. -* столбцы, отличные от первичного ключа, будут сериализованы в бинарном виде в соответствующем порядке и сохранены как значение результирующего узла, определяемого сериализованным ключом. -* запросы с фильтрацией по ключу с операторами `equals` или `in` будут оптимизированы до поиска по нескольким ключам в `Keeper`, в противном случае будут извлечены все значения. - -Пример: - -```sql -CREATE TABLE keeper_map_table -( - `key` String, - `v1` UInt32, - `v2` String, - `v3` Float32 -) -ENGINE = KeeperMap('/keeper_map_table', 4) -PRIMARY KEY key -``` - -с - -```xml - - /keeper_map_tables - -``` - -Каждое значение — бинарная сериализация `(v1, v2, v3)` — будет храниться в `/keeper_map_tables/keeper_map_table/data/serialized_key` в `Keeper`. -Кроме того, на количество ключей будет действовать мягкий лимит в 4 ключа. - -Если несколько таблиц создаются на одном и том же пути в ZooKeeper, значения сохраняются до тех пор, пока существует как минимум одна таблица, использующая их.\ -В результате можно использовать предложение `ON CLUSTER` при создании таблицы и разделять данные между несколькими экземплярами ClickHouse.\ -Разумеется, можно вручную выполнить `CREATE TABLE` с тем же путём на несвязанных экземплярах ClickHouse, чтобы получить тот же эффект совместного использования данных. - - -## Поддерживаемые операции {#supported-operations} - -### Вставки {#inserts} - -Когда в `KeeperMap` вставляются новые строки, если ключ не существует, создаётся новая запись с этим ключом. -Если ключ существует и настройка `keeper_map_strict_mode` имеет значение `true`, выбрасывается исключение, в противном случае значение для ключа перезаписывается. - -Пример: - -```sql -INSERT INTO keeper_map_table VALUES ('какой-то ключ', 1, 'значение', 3.2); -``` - -### Удаления {#deletes} - -Строки можно удалять с помощью запроса `DELETE` или команды `TRUNCATE`. -Если ключ существует и параметр `keeper_map_strict_mode` установлен в значение `true`, операции выборки и удаления данных завершатся успешно только в том случае, если их можно выполнить атомарно. - -```sql -DELETE FROM keeper_map_table WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -TRUNCATE TABLE keeper_map_table; -``` - -### Обновления {#updates} - -Значения можно изменять с помощью запроса `ALTER TABLE`. Первичный ключ изменить нельзя. -Если параметр `keeper_map_strict_mode` имеет значение `true`, выборка и обновление данных выполнятся успешно только при атомарном выполнении операции. - -```sql -ALTER TABLE keeper_map_table UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; -``` - - -## Связанные материалы {#related-content} - -- Блог: [Создание аналитических приложений реального времени с ClickHouse и Hex](https://clickhouse.com/blog/building-real-time-applications-with-clickhouse-and-hex-notebook-keeper-engine) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md deleted file mode 100644 index 1c3c7e69810..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'Движок Memory хранит данные в оперативной памяти в несжатом виде. Данные - сохраняются в точно том же виде, в котором они были получены при чтении. Другими словами, чтение - из этой таблицы не требует затрат.' -sidebar_label: 'Memory' -sidebar_position: 110 -slug: /engines/table-engines/special/memory -title: 'Табличный движок Memory' -doc_type: 'reference' ---- - - - -# Движок таблиц Memory {#memory-table-engine} - -:::note -При использовании движка таблиц Memory в ClickHouse Cloud данные по проекту не реплицируются на все узлы (так задумано). Чтобы гарантировать, что все запросы направляются на один и тот же узел и движок Memory работает предсказуемо, вы можете: -- Выполнять все операции в одном и том же сеансе -- Использовать клиент, работающий по TCP или через нативный интерфейс (что позволяет использовать «липкие» соединения), например [clickhouse-client](/interfaces/cli) -::: - -Движок Memory хранит данные в оперативной памяти (RAM) в несжатом виде. Данные хранятся в точности в том виде, в котором они были получены при чтении. Другими словами, чтение из этой таблицы полностью бесплатно с точки зрения затрат на обработку. -Одновременный доступ к данным синхронизирован. Блокировки кратковременные: операции чтения и записи не блокируют друг друга. -Индексы не поддерживаются. Чтение выполняется параллельно. - -Максимальная производительность (более 10 ГБ/сек) достигается на простых запросах, поскольку отсутствуют чтение с диска, декомпрессия и десериализация данных. (Следует отметить, что во многих случаях производительность движка MergeTree почти столь же высока.) -При перезапуске сервера данные из таблицы исчезают, и таблица становится пустой. -Как правило, использование этого движка таблиц не оправдано. Однако его можно применять для тестов и для задач, где требуется максимальная скорость на относительно небольшом количестве строк (до примерно 100 000 000). - -Движок Memory используется системой для временных таблиц с внешними данными запроса (см. раздел «Внешние данные для обработки запроса»), а также для реализации `GLOBAL IN` (см. раздел «Операторы IN»). - -Для ограничения размера таблицы с движком Memory можно задать верхнюю и нижнюю границы, фактически позволяя использовать её как кольцевой буфер (см. [Параметры движка](#engine-parameters)). - - - -## Параметры движка {#engine-parameters} - -- `min_bytes_to_keep` — Минимальное количество байт, которое необходимо сохранять, когда размер таблицы в памяти ограничен. - - Значение по умолчанию: `0` - - Требует `max_bytes_to_keep` -- `max_bytes_to_keep` — Максимальное количество байт, которое необходимо сохранять в таблице в памяти, где самые старые строки удаляются при каждой вставке (то есть используется кольцевой буфер). Максимальное количество байт может превышать указанный лимит, если при добавлении большого блока самая старая партия строк, подлежащих удалению, попадает под порог `min_bytes_to_keep`. - - Значение по умолчанию: `0` -- `min_rows_to_keep` — Минимальное количество строк, которое необходимо сохранять, когда размер таблицы в памяти ограничен. - - Значение по умолчанию: `0` - - Требует `max_rows_to_keep` -- `max_rows_to_keep` — Максимальное количество строк, которое необходимо сохранять в таблице в памяти, где самые старые строки удаляются при каждой вставке (то есть используется кольцевой буфер). Максимальное количество строк может превышать указанный лимит, если при добавлении большого блока самая старая партия строк, подлежащих удалению, попадает под порог `min_rows_to_keep`. - - Значение по умолчанию: `0` -- `compress` — Определяет, нужно ли сжимать данные в памяти. - - Значение по умолчанию: `false` - - - -## Использование {#usage} - -**Инициализация настроек** - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**Изменить настройки** - -```sql -ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**Примечание:** Параметры ограничения `bytes` и `rows` могут быть заданы одновременно, однако будет использоваться меньшее из значений `max` и `min`. - - -## Примеры {#examples} - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; - -/* 1. проверка, что самый старый блок не удаляется из-за минимального порога — 3000 строк */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 8'192 bytes - -/* 2. добавление блока, который не удаляется */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 1'024 bytes - -/* 3. проверка, что самый старый блок удаляется — 9216 байт — 1100 строк */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 8'192 bytes - -/* 4. проверка, что очень большой блок вытесняет все остальные */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 65'536 bytes - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` - -а также для строк: - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 4000, max_rows_to_keep = 10000; - -/* 1. проверяем, что самый старый блок не удаляется из‑за минимального порога — 3000 строк */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 1'600 строк - -/* 2. добавляем блок, который не удаляется */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 100 строк - -/* 3. проверяем, что самый старый блок удаляется — 9216 байт — 1100 строк */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 1'000 строк - -/* 4. проверяем, что очень большой блок вытесняет все остальные */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 10'000 строк - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md deleted file mode 100644 index 0455e83c946..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: 'Табличный движок `Merge` (не путать с `MergeTree`) сам не хранит данные, а позволяет одновременно читать из любого количества других таблиц.' -sidebar_label: 'Merge' -sidebar_position: 30 -slug: /engines/table-engines/special/merge -title: 'Табличный движок Merge' -doc_type: 'reference' ---- - - - -# Движок таблицы Merge {#merge-table-engine} - -Движок `Merge` (не путать с `MergeTree`) сам не хранит данные, но позволяет одновременно читать из любого количества других таблиц. - -Чтение автоматически распараллеливается. Запись в таблицу не поддерживается. При чтении используются индексы таблиц, из которых фактически производится чтение, если они есть. - - - -## Создание таблицы {#creating-a-table} - -```sql -CREATE TABLE ... Engine=Merge(db_name, tables_regexp) -``` - - -## Параметры движка {#engine-parameters} - -### `db_name` {#db_name} - -`db_name` — возможные значения: - - имя базы данных, - - константное выражение, которое возвращает строку с именем базы данных, например `currentDatabase()`, - - `REGEXP(expression)`, где `expression` — регулярное выражение для сопоставления имен баз данных. - -### `tables_regexp` {#tables_regexp} - -`tables_regexp` — регулярное выражение для сопоставления имен таблиц в указанной БД или нескольких БД. - -Регулярные выражения обрабатываются библиотекой [re2](https://github.com/google/re2) (поддерживает подмножество PCRE) и чувствительны к регистру. -См. примечания об экранировании символов в регулярных выражениях в разделе «match». - - - -## Использование {#usage} - -При выборе таблиц для чтения сама таблица `Merge` не выбирается, даже если она подходит под регулярное выражение. Это сделано, чтобы избежать циклов. -Можно создать две таблицы `Merge`, которые будут бесконечно пытаться читать данные друг друга, но это не лучшая идея. - -Типичный сценарий использования движка `Merge` — работа с большим количеством таблиц `TinyLog` так, как будто это одна таблица. - - - -## Примеры {#examples} - -**Пример 1** - -Рассмотрим две базы данных `ABC_corporate_site` и `ABC_store`. Таблица `all_visitors` будет содержать идентификаторы из таблиц `visitors` обеих баз данных. - -```sql -CREATE TABLE all_visitors (id UInt32) ENGINE=Merge(REGEXP('ABC_*'), 'visitors'); -``` - -**Пример 2** - -Предположим, у вас есть старая таблица `WatchLog_old`, вы решили изменить способ секционирования, при этом не перенося данные в новую таблицу `WatchLog_new`, и вам нужно просматривать данные из обеих таблиц. - -```sql -CREATE TABLE WatchLog_old( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -ORDER BY (date, UserId, EventType); - -INSERT INTO WatchLog_old VALUES ('2018-01-01', 1, 'hit', 3); - -CREATE TABLE WatchLog_new( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -PARTITION BY date -ORDER BY (UserId, EventType) -SETTINGS index_granularity=8192; - -INSERT INTO WatchLog_new VALUES ('2018-01-02', 2, 'hit', 3); - -CREATE TABLE WatchLog AS WatchLog_old ENGINE=Merge(currentDatabase(), '^WatchLog'); - -SELECT * FROM WatchLog; -``` - -```text -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-01 │ 1 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-02 │ 2 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -``` - - -## Виртуальные столбцы {#virtual-columns} - -- `_table` — имя таблицы, из которой были прочитаны данные. Тип: [String](../../../sql-reference/data-types/string.md). - - Если отфильтровать по `_table` (например, `WHERE _table='xyz'`), будут прочитаны только те таблицы, которые удовлетворяют условию фильтрации. - -- `_database` — имя базы данных, из которой были прочитаны данные. Тип: [String](../../../sql-reference/data-types/string.md). - -**См. также** - -- [Виртуальные столбцы](../../../engines/table-engines/index.md#table_engines-virtual_columns) -- Табличная функция [merge](../../../sql-reference/table-functions/merge.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md deleted file mode 100644 index 245785fc0ee..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -description: 'При записи в таблицу `Null` данные игнорируются. При чтении из таблицы - `Null` ответ пустой.' -sidebar_label: 'Null' -sidebar_position: 50 -slug: /engines/table-engines/special/null -title: 'Движок таблицы Null' -doc_type: 'reference' ---- - -# Движок таблицы Null {#null-table-engine} - -При записи данных в таблицу `Null` они игнорируются. -При чтении из таблицы `Null` результат будет пустым. - -Движок таблицы `Null` полезен для преобразования данных, когда исходные данные больше не нужны после преобразования. -Для этого вы можете создать материализованное представление поверх таблицы `Null`. -Данные, записываемые в таблицу, будут считываться представлением, но исходные сырые данные будут отбрасываться. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md deleted file mode 100644 index 0e9166dbaa5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Набор данных, который всегда находится в оперативной памяти. Предназначен для использования в правой части оператора `IN`.' -sidebar_label: 'Set' -sidebar_position: 60 -slug: /engines/table-engines/special/set -title: 'Табличный движок Set' -doc_type: 'reference' ---- - - - -# Движок таблицы Set {#set-table-engine} - -:::note -В ClickHouse Cloud, если ваш сервис был создан с версией ранее 25.4, вам необходимо установить совместимость как минимум с 25.4 с помощью `SET compatibility=25.4`. -::: - -Это набор данных, который всегда находится в оперативной памяти. Предназначен для использования в правой части оператора `IN` (см. раздел «Операторы IN»). - -Вы можете использовать `INSERT` для вставки данных в таблицу. Новые элементы будут добавлены в набор данных, а дубликаты будут игнорироваться. -Но вы не можете выполнять `SELECT` из этой таблицы. Единственный способ получить данные — использовать её в правой части оператора `IN`. - -Данные всегда расположены в оперативной памяти. При выполнении `INSERT` блоки вставленных данных также записываются в каталог таблиц на диске. При запуске сервера эти данные загружаются в оперативную память. Другими словами, после перезапуска данные сохраняются. - -При некорректном (жёстком) перезапуске сервера блок данных на диске может быть потерян или повреждён. В последнем случае вам может понадобиться вручную удалить файл с повреждёнными данными. - -### Ограничения и настройки {#join-limitations-and-settings} - -При создании таблицы применяются следующие настройки: - -#### persistent {#persistent} - -Отключает персистентность для движков таблиц Set и [Join](/engines/table-engines/special/join). - -Уменьшает нагрузку на подсистему ввода-вывода. Подходит для сценариев, где приоритетом является производительность и не требуется сохранность данных. - -Возможные значения: - -- 1 — включено. -- 0 — отключено. - -Значение по умолчанию: `1`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md deleted file mode 100644 index 0766455d5c8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: 'Позволяет запрашивать и отправлять данные на удалённый HTTP/HTTPS-сервер. Этот движок аналогичен движку File.' -sidebar_label: 'URL' -sidebar_position: 80 -slug: /engines/table-engines/special/url -title: 'Табличный движок URL' -doc_type: 'reference' ---- - - - -# Движок таблицы URL {#url-table-engine} - -Выполняет чтение и запись данных на удалённый HTTP/HTTPS-сервер. Этот движок похож на движок [File](../../../engines/table-engines/special/file.md). - -Синтаксис: `URL(URL [,Format] [,CompressionMethod])` - -- Параметр `URL` должен соответствовать структуре Uniform Resource Locator. Указанный URL должен указывать на сервер, использующий HTTP или HTTPS. Для получения ответа от сервера не требуются дополнительные заголовки. - -- `Format` должен быть форматом, который ClickHouse может использовать в запросах `SELECT` и, при необходимости, в запросах `INSERT`. Полный список поддерживаемых форматов см. в разделе [Formats](/interfaces/formats#formats-overview). - - Если этот аргумент не указан, ClickHouse автоматически определяет формат по суффиксу параметра `URL`. Если суффикс параметра `URL` не соответствует ни одному из поддерживаемых форматов, создание таблицы завершится с ошибкой. Например, для выражения движка `URL('http://localhost/test.json')` будет применён формат `JSON`. - -- `CompressionMethod` указывает, нужно ли сжимать тело HTTP-запроса. Если сжатие включено, HTTP-пакеты, отправляемые движком URL, содержат заголовок `Content-Encoding`, который указывает, какой метод сжатия используется. - -Чтобы включить сжатие, сначала убедитесь, что удалённая HTTP‑конечная точка, указанная параметром `URL`, поддерживает соответствующий алгоритм сжатия. - -Поддерживаемое значение `CompressionMethod` должно быть одним из следующих: -- gzip или gz -- deflate -- brotli или br -- lzma или xz -- zstd или zst -- lz4 -- bz2 -- snappy -- none -- auto - -Если `CompressionMethod` не указан, по умолчанию используется значение `auto`. Это означает, что ClickHouse автоматически определяет метод сжатия по суффиксу параметра `URL`. Если суффикс совпадает с каким-либо из перечисленных выше методов сжатия, применяется соответствующее сжатие, в противном случае сжатие не включается. - -Например, для выражения движка `URL('http://localhost/test.gzip')` применяется метод сжатия `gzip`, но для `URL('http://localhost/test.fr')` сжатие не включается, поскольку суффикс `fr` не соответствует ни одному из указанных выше методов сжатия. - - - -## Использование {#using-the-engine-in-the-clickhouse-server} - -Запросы `INSERT` и `SELECT` преобразуются соответственно в HTTP-запросы `POST` и `GET`. Для обработки `POST`-запросов удалённый сервер должен поддерживать [передачу с кодированием фрагментами (Chunked transfer encoding)](https://en.wikipedia.org/wiki/Chunked_transfer_encoding). - -Вы можете ограничить максимальное количество переходов по перенаправлениям для HTTP-запросов GET с помощью настройки [max_http_get_redirects](/operations/settings/settings#max_http_get_redirects). - - - -## Пример {#example} - -**1.** Создайте таблицу `url_engine_table` на сервере: - -```sql -CREATE TABLE url_engine_table (word String, value UInt64) -ENGINE=URL('http://127.0.0.1:12345/', CSV) -``` - -**2.** Создайте базовый HTTP-сервер, используя стандартные средства Python 3 -и запустите его: - -```python3 -from http.server import BaseHTTPRequestHandler, HTTPServer - -class CSVHTTPServer(BaseHTTPRequestHandler): - def do_GET(self): - self.send_response(200) - self.send_header('Content-type', 'text/csv') - self.end_headers() - - self.wfile.write(bytes('Hello,1\nWorld,2\n', "utf-8")) - -if __name__ == "__main__": - server_address = ('127.0.0.1', 12345) - HTTPServer(server_address, CSVHTTPServer).serve_forever() -``` - -```bash -$ python3 server.py -``` - -**3.** Выполните запрос данных: - -```sql -SELECT * FROM url_engine_table -``` - -```text -┌─word──┬─value─┐ -│ Привет │ 1 │ -│ Мир │ 2 │ -└───────┴───────┘ -``` - - -## Подробности реализации {#details-of-implementation} - -- Возможны параллельные операции чтения и записи -- Не поддерживаются: - - Операции `ALTER` и `SELECT...SAMPLE`. - - Индексы. - - Репликация. - - - -## Виртуальные столбцы {#virtual-columns} - -- `_path` — Путь к URL-ресурсу. Тип: `LowCardinality(String)`. -- `_file` — Имя URL-ресурса. Тип: `LowCardinality(String)`. -- `_size` — Размер ресурса в байтах. Тип: `Nullable(UInt64)`. Если размер неизвестен, значение — `NULL`. -- `_time` — Время последнего изменения файла. Тип: `Nullable(DateTime)`. Если время неизвестно, значение — `NULL`. -- `_headers` — Заголовки HTTP-ответа. Тип: `Map(LowCardinality(String), LowCardinality(String))`. - - - -## Настройки хранения {#storage-settings} - -- [engine_url_skip_empty_files](/operations/settings/settings.md#engine_url_skip_empty_files) — позволяет пропускать пустые файлы при чтении. По умолчанию отключена. -- [enable_url_encoding](/operations/settings/settings.md#enable_url_encoding) — позволяет включать и отключать кодирование и декодирование пути в URI. По умолчанию включена. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md deleted file mode 100644 index e39fb505ef2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Используется для создания представлений (подробнее см. запрос `CREATE VIEW`). Движок не хранит данные, а только сохраняет указанный запрос `SELECT`. При чтении из таблицы этот запрос выполняется (из него удаляются все ненужные столбцы).' -sidebar_label: 'View' -sidebar_position: 90 -slug: /engines/table-engines/special/view -title: 'Движок таблицы View' -doc_type: 'reference' ---- - -# Движок таблицы View {#view-table-engine} - -Используется для реализации представлений (подробнее см. запрос `CREATE VIEW`). Он не хранит данные, а только сохраняет указанный запрос `SELECT`. При чтении из таблицы выполняется этот запрос (при этом из него удаляются все ненужные столбцы). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md deleted file mode 100644 index c70df9afa37..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: 'Документация по интерфейсу Apache Arrow Flight в ClickHouse, который позволяет клиентам Flight SQL подключаться к ClickHouse' -sidebar_label: 'Интерфейс Arrow Flight' -sidebar_position: 26 -slug: /interfaces/arrowflight -title: 'Интерфейс Arrow Flight' -doc_type: 'reference' ---- - - - -# Интерфейс Apache Arrow Flight {#apache-arrow-flight-interface} - -ClickHouse поддерживает интеграцию с протоколом [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) — высокопроизводительным RPC‑фреймворком, предназначенным для эффективной передачи колоночных данных с использованием формата Arrow IPC поверх gRPC. - -Этот интерфейс позволяет клиентам Flight SQL выполнять запросы к ClickHouse и получать результаты в формате Arrow, обеспечивая высокую пропускную способность и низкую задержку для аналитических нагрузок. - - - -## Возможности {#features} - -* Выполнять SQL‑запросы по протоколу Arrow Flight SQL -* Передавать результаты запросов в потоковом режиме в формате Apache Arrow -* Интегрироваться с BI‑инструментами и прикладными решениями для работы с данными, поддерживающими Arrow Flight -* Обеспечивать легковесный и высокопроизводительный обмен данными по gRPC - - - -## Ограничения {#limitations} - -Интерфейс Arrow Flight в данный момент является экспериментальным и активно дорабатывается. Известные ограничения: - -* Ограниченная поддержка сложных SQL-возможностей, специфичных для ClickHouse -* Еще не реализованы все операции с метаданными Arrow Flight SQL -* В эталонной реализации отсутствуют встроенная аутентификация и настройка TLS - -Если вы столкнетесь с проблемами совместимости или хотите внести вклад, пожалуйста, [создайте issue](https://github.com/ClickHouse/ClickHouse/issues) в репозитории ClickHouse. - - - -## Запуск сервера Arrow Flight {#running-server} - -Чтобы включить сервер Arrow Flight в самоуправляемом экземпляре ClickHouse, добавьте следующую конфигурацию в конфигурационный файл сервера: - -```xml - - 9005 - -``` - -Перезапустите сервер ClickHouse. После успешного запуска вы должны увидеть в логах сообщение, похожее на следующее: - -```bash -{} Application: Протокол совместимости Arrow Flight: 0.0.0.0:9005 -``` - - -## Подключение к ClickHouse через Arrow Flight SQL {#connecting-to-clickhouse} - -Вы можете использовать любой клиент, который поддерживает Arrow Flight SQL. Например, с помощью `pyarrow`: - -```python -import pyarrow.flight - -client = pyarrow.flight.FlightClient("grpc://localhost:9005") -ticket = pyarrow.flight.Ticket(b"SELECT number FROM system.numbers LIMIT 10") -reader = client.do_get(ticket) - -for batch in reader: - print(batch.to_pandas()) -``` - - -## Совместимость {#compatibility} - -Интерфейс Arrow Flight совместим с инструментами, которые поддерживают Arrow Flight SQL, включая собственные приложения, реализованные на: - -* Python (`pyarrow`) -* Java (`arrow-flight`) -* C++ и других языках, совместимых с gRPC - -Если для вашего инструмента доступен нативный коннектор ClickHouse (например, JDBC, ODBC), предпочтительнее использовать именно его, если только Arrow Flight не требуется специально для достижения нужной производительности или совместимости форматов. - - - -## Отмена запросов {#query-cancellation} - -Долго выполняющиеся запросы можно отменить, закрыв gRPC-соединение на стороне клиента. Поддержка более продвинутых возможностей отмены запланирована. - ---- - -Для получения дополнительной информации см.: - -* [Спецификацию Apache Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html) -* [Задачу ClickHouse на GitHub №7554](https://github.com/ClickHouse/ClickHouse/issues/7554) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cli.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cli.md deleted file mode 100644 index 9c5db6bfe33..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cli.md +++ /dev/null @@ -1,918 +0,0 @@ ---- -description: 'Документация по клиенту командной строки ClickHouse' -sidebar_label: 'Клиент ClickHouse' -sidebar_position: 17 -slug: /interfaces/cli -title: 'Клиент ClickHouse' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; -import cloud_connect_button from '@site/static/images/_snippets/cloud-connect-button.png'; -import connection_details_native from '@site/static/images/_snippets/connection-details-native.png'; -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -ClickHouse предоставляет штатный клиент командной строки для выполнения SQL‑запросов напрямую к серверу ClickHouse. -Он поддерживает как интерактивный режим (для выполнения запросов в реальном времени), так и пакетный режим (для скриптов и автоматизации). -Результаты запросов могут отображаться в терминале или экспортироваться в файл; поддерживаются все [форматы](formats.md) вывода ClickHouse, такие как Pretty, CSV, JSON и другие. - -Клиент предоставляет информацию о выполнении запросов в реальном времени с индикатором прогресса, количеством прочитанных строк, объёмом обработанных данных (в байтах) и временем выполнения запроса. -Он поддерживает как [параметры командной строки](#command-line-options), так и [файлы конфигурации](#configuration_files). - - -## Установка {#install} - -Чтобы загрузить ClickHouse, выполните команду: - -```bash -curl https://clickhouse.com/ | sh -``` - -Чтобы установить его, выполните: - -```bash -sudo ./clickhouse install -``` - -Дополнительные варианты установки см. в разделе [Install ClickHouse](../getting-started/install/install.mdx). - -Различные версии клиента и сервера совместимы между собой, но некоторые функции могут быть недоступны в более старых клиентах. Рекомендуется использовать одну и ту же версию для клиента и сервера. - - -## Запуск {#run} - -:::note -Если вы только скачали, но ещё не установили ClickHouse, используйте `./clickhouse client` вместо `clickhouse-client`. -::: - -Чтобы подключиться к серверу ClickHouse, выполните следующую команду: - -```bash -$ clickhouse-client --host server - -ClickHouse client version 24.12.2.29 (official build). -Подключение к server:9000 как пользователь default. -Подключено к серверу ClickHouse версии 24.12.2. - -:) -``` - -Укажите при необходимости дополнительные параметры подключения: - -| Option | Description | -| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--port <port>` | Порт, на котором сервер ClickHouse принимает подключения. Порты по умолчанию: 9440 (TLS) и 9000 (без TLS). Обратите внимание, что ClickHouse Client использует нативный протокол, а не HTTP(S). | -| `-s [ --secure ]` | Использовать ли TLS (обычно определяется автоматически). | -| `-u [ --user ] <username>` | Имя пользователя базы данных, под которым выполняется подключение. По умолчанию выполняется подключение от имени пользователя `default`. | -| `--password <password>` | Пароль пользователя базы данных. Пароль для подключения также можно указать в конфигурационном файле. Если вы не укажете пароль, клиент запросит его. | -| `-c [ --config ] <path-to-file>` | Расположение конфигурационного файла для ClickHouse Client, если он не находится в одном из стандартных мест. См. раздел [Configuration Files](#configuration_files). | -| `--connection <name>` | Имя преднастроенного подключения из [configuration file](#connection-credentials). | - -Полный список параметров командной строки см. в разделе [Command Line Options](#command-line-options). - - -### Подключение к ClickHouse Cloud {#connecting-cloud} - -Информация о вашем сервисе ClickHouse Cloud доступна в консоли ClickHouse Cloud. Выберите сервис, к которому нужно подключиться, и нажмите **Connect**: - -Кнопка подключения к сервису ClickHouse Cloud - -
- -
- -Выберите **Native** — после этого будут показаны детали подключения вместе с примером команды `clickhouse-client`: - -Детали подключения к ClickHouse Cloud по протоколу Native TCP - -### Хранение подключений в файле конфигурации {#connection-credentials} - -Вы можете хранить параметры подключения для одного или нескольких серверов ClickHouse в [файле конфигурации](#configuration_files). - -Формат имеет следующий вид: - -```xml - - - - default - hostname - 9440 - 1 - default - password - - - - - - - -``` - -См. [раздел о файлах конфигурации](#configuration_files) для получения дополнительной информации. - -:::note -Чтобы сосредоточиться на синтаксисе запросов, в последующих примерах опущены параметры подключения (`--host`, `--port` и т. д.). Не забудьте добавить их, когда будете использовать команды. -::: - - -## Интерактивный режим {#interactive-mode} - -### Использование интерактивного режима {#using-interactive-mode} - -Чтобы запустить ClickHouse в интерактивном режиме, достаточно выполнить: - -```bash -clickhouse-client -``` - -Это откроет цикл Read-Eval-Print (REPL), в котором вы сможете интерактивно вводить SQL-запросы. -После подключения вы увидите приглашение командной строки, где сможете вводить запросы: - -```bash -Клиент ClickHouse версии 25.x.x.x -Подключение к localhost:9000 от имени пользователя default. -Подключено к серверу ClickHouse версии 25.x.x.x - -hostname :) -``` - -В интерактивном режиме формат вывода по умолчанию — `PrettyCompact`. -Вы можете изменить формат в секции `FORMAT` запроса или указав параметр командной строки `--format`. -Чтобы использовать формат `Vertical`, вы можете указать `--vertical` или добавить `\G` в конце запроса. -В этом формате каждое значение выводится на отдельной строке, что удобно для широких таблиц. - -В интерактивном режиме по умолчанию любой введённый запрос выполняется при нажатии `Enter`. -Точка с запятой в конце запроса не обязательна. - -Вы можете запустить клиент с параметром `-m, --multiline`. -Чтобы ввести многострочный запрос, введите обратную косую черту `\` перед переводом строки. -После нажатия `Enter` вам будет предложено ввести следующую строку запроса. -Чтобы выполнить запрос, завершите его точкой с запятой и нажмите `Enter`. - -ClickHouse Client основан на `replxx` (аналог `readline`), поэтому он использует знакомые сочетания клавиш и ведёт историю. -История по умолчанию записывается в `~/.clickhouse-client-history`. - -Чтобы выйти из клиента, нажмите `Ctrl+D` или введите одну из следующих команд вместо запроса: - -* `exit` или `exit;` -* `quit` или `quit;` -* `q`, `Q` или `:q` -* `logout` или `logout;` - - -### Информация об обработке запроса {#processing-info} - -Во время обработки запроса клиент показывает: - -1. Прогресс выполнения, который по умолчанию обновляется не чаще 10 раз в секунду. - Для быстрых запросов прогресс может не успеть отобразиться. -2. Отформатированный запрос после разбора, для отладки. -3. Результат в указанном формате. -4. Количество строк в результате, прошедшее время и среднюю скорость обработки запроса. - Все объёмы относятся к несжатым данным. - -Вы можете отменить длительный запрос, нажав `Ctrl+C`. -Однако вам всё равно придётся немного подождать, пока сервер прервёт запрос. -На некоторых этапах отменить запрос невозможно. -Если вы не будете ждать и нажмёте `Ctrl+C` во второй раз, клиент завершит работу. - -ClickHouse Client позволяет передавать внешние данные (внешние временные таблицы) для выполнения запросов. -Дополнительные сведения см. в разделе [Внешние данные для обработки запросов](../engines/table-engines/special/external-data.md). - -### Псевдонимы {#cli_aliases} - -В интерактивной оболочке (REPL) можно использовать следующие псевдонимы: - -- `\l` — SHOW DATABASES -- `\d` — SHOW TABLES -- `\c ` — USE DATABASE -- `.` — повторить последний запрос - -### Сочетания клавиш {#keyboard_shortcuts} - -- `Alt (Option) + Shift + e` — открыть редактор с текущим запросом. Можно указать редактор с помощью переменной окружения `EDITOR`. По умолчанию используется `vim`. -- `Alt (Option) + #` — закомментировать строку. -- `Ctrl + r` — нечеткий поиск по истории. - -Полный список всех доступных сочетаний клавиш приведён в [replxx](https://github.com/AmokHuginnsson/replxx/blob/1f149bf/src/replxx_impl.cxx#L262). - -:::tip -Чтобы настроить корректную работу клавиши Meta (Option) в macOS: - -iTerm2: перейдите в Preferences -> Profiles -> Keys -> Left Option key и нажмите Esc+ -::: - -## Пакетный режим {#batch-mode} - -### Использование пакетного режима {#using-batch-mode} - -Вместо интерактивного использования ClickHouse Client вы можете запускать его в пакетном режиме. -В пакетном режиме ClickHouse выполняет один запрос и сразу завершается — без интерактивного приглашения или цикла. - -Вы можете задать один запрос следующим образом: - -```bash -$ clickhouse-client "SELECT sum(number) FROM numbers(10)" -45 -``` - -Вы также можете использовать параметр командной строки `--query`: - -```bash -$ clickhouse-client --query "SELECT uniq(number) FROM numbers(10)" -10 -``` - -Вы можете передать запрос в `stdin`: - -```bash -$ echo "SELECT avg(number) FROM numbers(10)" | clickhouse-client -4.5 -``` - -Если таблица `messages` уже существует, вы также можете вставлять данные из командной строки: - -```bash -$ echo "Hello\nGoodbye" | clickhouse-client --query "INSERT INTO messages FORMAT CSV" -``` - -Когда указана опция `--query`, любой ввод данных добавляется к запросу после символа перевода строки. - - -### Загрузка CSV-файла в удалённый сервис ClickHouse {#cloud-example} - -В этом примере демонстрируется загрузка примерного набора данных из CSV-файла `cell_towers.csv` в существующую таблицу `cell_towers` в базе данных `default`: - -```bash -clickhouse-client --host HOSTNAME.clickhouse.cloud \ - --port 9440 \ - --user default \ - --password PASSWORD \ - --query "INSERT INTO cell_towers FORMAT CSVWithNames" \ - < cell_towers.csv -``` - - -### Примеры вставки данных из командной строки {#more-examples} - -Существует несколько способов вставить данные из командной строки. -Следующий пример вставляет две строки данных в формате CSV в таблицу ClickHouse в пакетном режиме: - -```bash -echo -ne "1, 'some text', '2016-08-14 00:00:00'\n2, 'some more text', '2016-08-14 00:00:01'" | \ - clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -В приведённом ниже примере `cat <<_EOF` открывает here-документ, который будет считывать всё до тех пор, пока снова не встретит `_EOF`, после чего выведет это: - -```bash -cat <<_EOF | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -3, 'some text', '2016-08-14 00:00:00' -4, 'some more text', '2016-08-14 00:00:01' -_EOF -``` - -В примере ниже содержимое файла file.csv выводится на стандартный вывод (stdout) с помощью команды `cat` и через конвейер (pipe) передаётся в `clickhouse-client` как входные данные: - -```bash -cat file.csv | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -В пакетном режиме формат данных по умолчанию — `TabSeparated` (см. [форматы](formats.md)). -Вы можете указать формат в предложении `FORMAT` запроса, как показано в примере выше. - - -## Запросы с параметрами {#cli-queries-with-parameters} - -Вы можете указать параметры в запросе и передать им значения с помощью опций командной строки. -Это позволяет избежать необходимости формировать запрос с конкретными динамическими значениями на стороне клиента. -Например: - -```bash -$ clickhouse-client --param_parName="[1, 2]" --query "SELECT {parName: Array(UInt16)}" -[1,2] -``` - -Параметры можно задавать и из [интерактивного сеанса](#interactive-mode): - -```text -$ clickhouse-client -Клиент ClickHouse версии 25.X.X.XXX (официальная сборка). - -#highlight-next-line -:) SET param_parName='[1, 2]'; - -SET param_parName = '[1, 2]' - -Query id: 7ac1f84e-e89a-4eeb-a4bb-d24b8f9fd977 - -ОК. - -0 строк в наборе. Прошло: 0.000 сек. - -#highlight-next-line -:) SELECT {parName:Array(UInt16)} - -SELECT {parName:Array(UInt16)} - -Query id: 0358a729-7bbe-4191-bb48-29b063c548a7 - - ┌─_CAST([1, 2]⋯y(UInt16)')─┐ -1. │ [1,2] │ - └──────────────────────────┘ - -1 строка в наборе. Прошло: 0.006 сек. -``` - - -### Синтаксис запроса {#cli-queries-with-parameters-syntax} - -В запросе указывайте значения, которые хотите подставлять с помощью параметров командной строки, заключая их в фигурные скобки в следующем формате: - -```sql -{:} -``` - -| Parameter | Description | -| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | Идентификатор подстановочного параметра. Соответствующая опция командной строки — `--param_ = value`. | -| `data type` | [Тип данных](../sql-reference/data-types/index.md) параметра.

Например, структура данных вида `(integer, ('string', integer))` может иметь тип данных `Tuple(UInt8, Tuple(String, UInt8))` (можно также использовать другие [целочисленные](../sql-reference/data-types/int-uint.md) типы).

Также можно передавать в качестве параметров имя таблицы, имя базы данных и имена столбцов; в этом случае следует использовать тип данных `Identifier`. | - - -### Примеры {#cli-queries-with-parameters-examples} - -```bash -$ clickhouse-client --param_tuple_in_tuple="(10, ('dt', 10))" \ - --query "SELECT * FROM table WHERE val = {tuple_in_tuple:Tuple(UInt8, Tuple(String, UInt8))}" - -$ clickhouse-client --param_tbl="numbers" --param_db="system" --param_col="number" --param_alias="top_ten" \ - --query "SELECT {col:Identifier} as {alias:Identifier} FROM {db:Identifier}.{tbl:Identifier} LIMIT 10" -``` - - -## Генерация SQL с помощью ИИ {#ai-sql-generation} - -ClickHouse Client включает встроенную поддержку ИИ для генерации SQL-запросов по описаниям на естественном языке. Эта функция помогает пользователям составлять сложные запросы без глубоких знаний SQL. - -Помощь ИИ работает «из коробки», если у вас задана переменная среды `OPENAI_API_KEY` или `ANTHROPIC_API_KEY`. Для более продвинутой настройки см. раздел [Конфигурация](#ai-sql-generation-configuration). - -### Использование {#ai-sql-generation-usage} - -Чтобы использовать генерацию SQL с ИИ, добавьте префикс `??` к запросу на естественном языке: - -```bash -:) ?? показать всех пользователей, совершивших покупки за последние 30 дней -``` - -ИИ будет: - -1. Автоматически анализировать структуру вашей базы данных -2. Генерировать соответствующий SQL‑запрос на основе обнаруженных таблиц и столбцов -3. Сразу выполнять сгенерированный запрос - - -### Пример {#ai-sql-generation-example} - -```bash -:) ?? подсчитать заказы по категориям продуктов - -Запуск генерации SQL с обнаружением схемы... -────────────────────────────────────────────────── - -🔍 list_databases - ➜ system, default, sales_db - -🔍 list_tables_in_database - database: sales_db - ➜ orders, products, categories - -🔍 get_schema_for_table - database: sales_db - table: orders - ➜ CREATE TABLE orders (order_id UInt64, product_id UInt64, quantity UInt32, ...) - -✨ SQL-запрос успешно сгенерирован! -────────────────────────────────────────────────── - -SELECT - c.name AS category, - COUNT(DISTINCT o.order_id) AS order_count -FROM sales_db.orders o -JOIN sales_db.products p ON o.product_id = p.product_id -JOIN sales_db.categories c ON p.category_id = c.category_id -GROUP BY c.name -ORDER BY order_count DESC -``` - - -### Конфигурация {#ai-sql-generation-configuration} - -Для генерации SQL-запросов с помощью ИИ необходимо настроить поставщика ИИ в конфигурационном файле клиента ClickHouse. Вы можете использовать OpenAI, Anthropic или любой совместимый с OpenAI API-сервис. - -#### Резервный механизм на основе переменных окружения {#ai-sql-generation-fallback} - -Если конфигурация ИИ не указана в конфигурационном файле, ClickHouse Client автоматически попытается использовать переменные окружения: - -1. Сначала проверяется переменная окружения `OPENAI_API_KEY` -2. Если она не найдена, проверяется переменная окружения `ANTHROPIC_API_KEY` -3. Если ни одна из них не найдена, функции ИИ будут отключены - -Это позволяет быстро выполнить настройку без конфигурационных файлов: - -```bash -# Использование OpenAI {#using-openai} -export OPENAI_API_KEY=your-openai-key -clickhouse-client - -# Использование Anthropic {#using-anthropic} -export ANTHROPIC_API_KEY=your-anthropic-key -clickhouse-client -``` - - -#### Файл конфигурации {#ai-sql-generation-configuration-file} - -Для более тонкого управления настройками ИИ задайте их в файле конфигурации ClickHouse Client, который находится по одному из следующих путей: - -* `$XDG_CONFIG_HOME/clickhouse/config.xml` (или `~/.config/clickhouse/config.xml`, если `XDG_CONFIG_HOME` не задан) (формат XML) -* `$XDG_CONFIG_HOME/clickhouse/config.yaml` (или `~/.config/clickhouse/config.yaml`, если `XDG_CONFIG_HOME` не задан) (формат YAML) -* `~/.clickhouse-client/config.xml` (формат XML, устаревший путь) -* `~/.clickhouse-client/config.yaml` (формат YAML, устаревший путь) -* Или укажите произвольный путь с помощью `--config-file` - - - - ```xml - - - - your-api-key-here - - - openai - - - gpt-4o - - - - - - true - - - 0.0 - 1000 - 30 - 10 - - - - - - ``` - - - - ```yaml - ai: - # Обязательно: ваш API-ключ (или задайте через переменную окружения) - api_key: your-api-key-here - - # Обязательно: тип провайдера (openai, anthropic) - provider: openai - - # Используемая модель - model: gpt-4o - - # Необязательно: пользовательская конечная точка API для сервисов, совместимых с OpenAI - # base_url: https://openrouter.ai/api - - # Включить доступ к схеме — позволяет ИИ запрашивать информацию о базах данных и таблицах - enable_schema_access: true - - # Параметры генерации - temperature: 0.0 # Управляет степенью случайности (0.0 = детерминированно) - max_tokens: 1000 # Максимальная длина ответа - timeout_seconds: 30 # Тайм-аут запроса - max_steps: 10 # Максимальное количество шагов исследования схемы - - # Необязательно: пользовательский системный промпт - # system_prompt: | - # You are an expert ClickHouse SQL assistant. Convert natural language to SQL. - # Focus on performance and use ClickHouse-specific optimizations. - # Always return executable SQL without explanations. - ``` - - - -
- -**Использование API, совместимых с OpenAI (например, OpenRouter):** - -```yaml -ai: - provider: openai # Используйте 'openai' для обеспечения совместимости - api_key: your-openrouter-api-key - base_url: https://openrouter.ai/api/v1 - model: anthropic/claude-3.5-sonnet # Используйте схему именования моделей OpenRouter -``` - -**Примеры минимальных конфигураций:** - -```yaml -# Минимальная конфигурация — использует переменную окружения для API-ключа {#minimal-config-uses-environment-variable-for-api-key} -ai: - provider: openai # Будет использована переменная окружения OPENAI_API_KEY - -# Без конфигурации — автоматический резервный вариант {#no-config-at-all-automatic-fallback} -# (Пустая или отсутствующая секция ai — будет использована OPENAI_API_KEY, затем ANTHROPIC_API_KEY) {#empty-or-no-ai-section-will-try-openai_api_key-then-anthropic_api_key} - -# Переопределение только модели — использует переменную окружения для API-ключа {#only-override-model-uses-env-var-for-api-key} -ai: - provider: openai - model: gpt-3.5-turbo -``` - - -### Параметры {#ai-sql-generation-parameters} - -
-Обязательные параметры - -- `api_key` - Ваш API-ключ для сервиса ИИ. Можно не указывать, если он задан через переменную окружения: - - OpenAI: `OPENAI_API_KEY` - - Anthropic: `ANTHROPIC_API_KEY` - - Примечание: API-ключ в конфигурационном файле имеет приоритет над переменной окружения -- `provider` - Провайдер ИИ: `openai` или `anthropic` - - Если не указан, используется автоматический выбор на основе доступных переменных окружения - -
- -
-Конфигурация модели - -- `model` - Используемая модель (по умолчанию: провайдер-зависимая) - - OpenAI: `gpt-4o`, `gpt-4`, `gpt-3.5-turbo` и т.д. - - Anthropic: `claude-3-5-sonnet-20241022`, `claude-3-opus-20240229` и т.д. - - OpenRouter: используйте их схему наименования моделей, например `anthropic/claude-3.5-sonnet` - -
- -
-Настройки подключения - -- `base_url` - Пользовательский API-эндпоинт для сервисов, совместимых с OpenAI (необязательно) -- `timeout_seconds` - Тайм-аут запроса в секундах (по умолчанию: `30`) - -
- -
-Исследование схемы - -- `enable_schema_access` - Разрешить ИИ исследовать схемы баз данных (по умолчанию: `true`) -- `max_steps` - Максимальное число шагов вызова инструментов для исследования схемы (по умолчанию: `10`) - -
- -
-Параметры генерации - -- `temperature` - Контролирует степень случайности: 0.0 = детерминированно, 1.0 = более творчески (по умолчанию: `0.0`) -- `max_tokens` - Максимальная длина ответа в токенах (по умолчанию: `1000`) -- `system_prompt` - Пользовательские инструкции для ИИ (необязательно) - -
- -### Как это работает {#ai-sql-generation-how-it-works} - -AI-генератор SQL использует многошаговый процесс: - - - -1. **Анализ схемы** - -AI использует встроенные инструменты для исследования вашей базы данных -- Составляет список доступных баз данных -- Обнаруживает таблицы в соответствующих базах данных -- Изучает структуры таблиц с помощью операторов `CREATE TABLE` - -2. **Генерация запроса** - -На основе обнаруженной схемы AI генерирует SQL, который: -- Соответствует вашему запросу на естественном языке -- Использует корректные имена таблиц и столбцов -- Применяет подходящие соединения и агрегации - -3. **Выполнение** - -Сгенерированный SQL автоматически выполняется, и результаты отображаются - - - -### Ограничения {#ai-sql-generation-limitations} - -- Требуется активное подключение к Интернету -- Использование API подчиняется ограничениям по частоте и стоимости со стороны поставщика ИИ -- Сложные запросы могут потребовать нескольких уточнений -- ИИ имеет доступ только для чтения к информации о схеме, но не к самим данным - -### Безопасность {#ai-sql-generation-security} - -- API-ключи никогда не передаются на серверы ClickHouse -- ИИ видит только информацию о схеме (имена таблиц/столбцов и их типы), но не реальные данные -- Все сгенерированные запросы учитывают ваши текущие права доступа к базе данных - -## Строка подключения {#connection_string} - -### Использование {#connection-string-usage} - -ClickHouse Client также поддерживает альтернативный способ подключения к серверу ClickHouse — с помощью строки подключения, аналогичной [MongoDB](https://www.mongodb.com/docs/manual/reference/connection-string/), [PostgreSQL](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING), [MySQL](https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html#connecting-using-uri). Строка подключения имеет следующий синтаксис: - -```text -clickhouse:[//[user[:password]@][hosts_and_ports]][/database][?query_parameters] -``` - -| Компонент (все необязательные) | Описание | Значение по умолчанию | -| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| `user` | Имя пользователя базы данных. | `default` | -| `password` | Пароль пользователя базы данных. Если указан символ `:` и пароль не задан, клиент запросит пароль у пользователя. | - | -| `hosts_and_ports` | Список хостов и необязательных портов `host[:port] [, host:[port]], ...`. | `localhost:9000` | -| `database` | Имя базы данных. | `default` | -| `query_parameters` | Список пар «ключ–значение» `param1=value1[,¶m2=value2], ...`. Для некоторых параметров значение не требуется. Имена параметров и значений чувствительны к регистру. | - | - - -### Примечания {#connection-string-notes} - -Если имя пользователя, пароль или база данных указаны в строке подключения, их нельзя указывать с помощью `--user`, `--password` или `--database` (и наоборот). - -Компонент host может быть либо именем хоста, либо адресом IPv4 или IPv6. -Адреса IPv6 должны указываться в квадратных скобках: - -```text -clickhouse://[2001:db8::1234] -``` - -Строки подключения могут содержать несколько хостов. -ClickHouse Client будет пытаться подключиться к этим хостам по порядку (слева направо). -После установления соединения попытки подключиться к оставшимся хостам не выполняются. - -Строка подключения должна быть указана как первый аргумент `clickHouse-client`. -Строку подключения можно комбинировать с произвольным количеством других [параметров командной строки](#command-line-options), за исключением `--host` и `--port`. - -Для `query_parameters` поддерживаются следующие ключи: - -| Ключ | Описание | -| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `secure` (или `s`) | Если указан, клиент будет подключаться к серверу по защищённому соединению (TLS). См. `--secure` в [параметрах командной строки](#command-line-options). | - -**Процентное кодирование** - -Символы, отличные от US-ASCII, пробелы и специальные символы в следующих параметрах должны быть [закодированы с использованием процентного кодирования](https://en.wikipedia.org/wiki/URL_encoding): - -* `user` -* `password` -* `hosts` -* `database` -* `query parameters` - - -### Примеры {#connection_string_examples} - -Подключитесь к `localhost` через порт 9000 и выполните запрос `SELECT 1`. - -```bash -clickhouse-client clickhouse://localhost:9000 --query "SELECT 1" -``` - -Подключитесь к `localhost` под пользователем `john` с паролем `secret`, используя хост `127.0.0.1` и порт `9000` - -```bash -clickhouse-client clickhouse://john:secret@127.0.0.1:9000 -``` - -Подключитесь к `localhost` под пользователем `default` (хост с IPv6-адресом `[::1]`, портом `9000`). - -```bash -clickhouse-client clickhouse://[::1]:9000 -``` - -Подключитесь к `localhost` на порту 9000 в многострочном режиме. - -```bash -clickhouse-client clickhouse://localhost:9000 '-m' -``` - -Подключитесь к `localhost` на порту 9000 от имени пользователя `default`. - -```bash -clickhouse-client clickhouse://default@localhost:9000 - -# эквивалентно: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --user default -``` - -Подключитесь к `localhost` на порту 9000 и используйте в качестве базы данных по умолчанию `my_database`. - -```bash -clickhouse-client clickhouse://localhost:9000/my_database - -# эквивалентно: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --database my_database -``` - -Подключитесь к `localhost` на порту 9000, по умолчанию используя базу данных `my_database`, указанную в строке подключения, и защищённое соединение с помощью сокращённого параметра `s`. - -```bash -clickhouse-client clickhouse://localhost/my_database?s - -# эквивалентно: {#equivalent-to} -clickhouse-client clickhouse://localhost/my_database -s -``` - -Подключитесь к хосту, порту, пользователю и базе данных по умолчанию. - -```bash -clickhouse-client clickhouse: -``` - -Подключитесь к хосту по умолчанию, используя порт по умолчанию, под пользователем `my_user` и без пароля. - -```bash -clickhouse-client clickhouse://my_user@ - -# Пустой пароль между : и @ означает, что пользователю будет предложено ввести пароль перед установкой соединения. {#using-a-blank-password-between-and-means-to-asking-the-user-to-enter-the-password-before-starting-the-connection} -clickhouse-client clickhouse://my_user:@ -``` - -Подключитесь к `localhost`, используя адрес электронной почты в качестве имени пользователя. Символ `@` кодируется в `%40`. - -```bash -clickhouse-client clickhouse://some_user%40some_mail.com@localhost:9000 -``` - -Подключитесь к одному из следующих хостов: `192.168.1.15`, `192.168.1.25`. - -```bash -clickhouse-client clickhouse://192.168.1.15,192.168.1.25 -``` - - -## Формат ID запроса {#query-id-format} - -В интерактивном режиме ClickHouse Client показывает ID для каждого запроса. По умолчанию ID имеет следующий формат: - -```sql -ID запроса: 927f137d-00f1-4175-8914-0dd066365e96 -``` - -Пользовательский формат можно задать в конфигурационном файле внутри тега `query_id_formats`. Заполнитель `{query_id}` в строке формата заменяется на идентификатор запроса. Внутри тега можно указать несколько строк формата. -Эту возможность можно использовать для генерации URL-адресов, чтобы упростить профилирование запросов. - -**Пример** - -```xml - - - http://speedscope-host/#profileURL=qp%3Fid%3D{query_id} - - -``` - -При указанной выше конфигурации идентификатор запроса отображается в следующем формате: - -```response -speedscope:http://speedscope-host/#profileURL=qp%3Fid%3Dc8ecc783-e753-4b38-97f1-42cddfb98b7d -``` - - -## Файлы конфигурации {#configuration_files} - -ClickHouse Client использует первый найденный файл из следующего списка: - -- Файл, заданный параметром `-c [ -C, --config, --config-file ]`. -- `./clickhouse-client.[xml|yaml|yml]` -- `$XDG_CONFIG_HOME/clickhouse/config.[xml|yaml|yml]` (или `~/.config/clickhouse/config.[xml|yaml|yml]`, если переменная окружения `XDG_CONFIG_HOME` не установлена) -- `~/.clickhouse-client/config.[xml|yaml|yml]` -- `/etc/clickhouse-client/config.[xml|yaml|yml]` - -См. пример файла конфигурации в репозитории ClickHouse: [`clickhouse-client.xml`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/client/clickhouse-client.xml) - - - - ```xml - - username - password - true - - - /etc/ssl/cert.pem - - - - ``` - - - ```yaml - user: username - password: 'password' - secure: true - openSSL: - client: - caConfig: '/etc/ssl/cert.pem' - ``` - - - -## Параметры переменных окружения {#environment-variable-options} - -Имя пользователя, пароль и хост могут быть заданы через переменные окружения `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD` и `CLICKHOUSE_HOST`. -Аргументы командной строки `--user`, `--password` или `--host`, а также [строка подключения](#connection_string) (если указана) имеют приоритет над переменными окружения. - -## Параметры командной строки {#command-line-options} - -Все параметры командной строки можно указать непосредственно в командной строке или задать как значения по умолчанию в [конфигурационном файле](#configuration_files). - -### Общие параметры {#command-line-options-general} - -| Параметр | Описание | Значение по умолчанию | -|----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|------------------------------| -| `-c [ -C, --config, --config-file ] ` | Путь к конфигурационному файлу клиента, если он не находится в одном из стандартных расположений. См. [Configuration Files](#configuration_files). | - | -| `--help` | Вывести краткую справку по использованию и завершить работу. Используйте совместно с `--verbose`, чтобы отобразить все возможные параметры, включая настройки запроса. | - | -| `--history_file ` | Путь к файлу, содержащему историю команд. | - | -| `--history_max_entries` | Максимальное количество записей в файле истории. | `1000000` (1 миллион) | -| `--prompt ` | Указать пользовательскую строку приглашения. | `display_name` сервера | -| `--verbose` | Увеличить детализацию вывода. | - | -| `-V [ --version ]` | Вывести версию и завершить работу. | - | - -### Параметры подключения {#command-line-options-connection} - -| Параметр | Описание | Значение по умолчанию | -|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------| -| `--connection ` | Имя предварительно настроенных параметров подключения из конфигурационного файла. См. [Учетные данные подключения](#connection-credentials). | - | -| `-d [ --database ] ` | База данных, используемая по умолчанию для этого подключения. | Текущая база данных из настроек сервера (по умолчанию `default`) | -| `-h [ --host ] ` | Имя хоста сервера ClickHouse, к которому выполняется подключение. Может быть именем хоста или IPv4/IPv6-адресом. Можно передать несколько хостов, указав параметр несколько раз. | `localhost` | -| `--jwt ` | Использовать JSON Web Token (JWT) для аутентификации.

Авторизация JWT на стороне сервера доступна только в ClickHouse Cloud. | - | -| `--no-warnings` | Отключить показ предупреждений из `system.warnings` при подключении клиента к серверу. | - | -| `--no-server-client-version-message` | Скрывать сообщение о несовпадении версий сервера и клиента при подключении клиента к серверу. | - | -| `--password ` | Пароль пользователя базы данных. Пароль для подключения также можно указать в конфигурационном файле. Если пароль не указан, клиент запросит его интерактивно. | - | -| `--port ` | Порт, на котором сервер принимает подключения. Порты по умолчанию: 9440 (TLS) и 9000 (без TLS).

Примечание: клиент использует нативный протокол, а не HTTP(S). | `9440`, если указан `--secure`, иначе `9000`. Всегда по умолчанию `9440`, если имя хоста оканчивается на `.clickhouse.cloud`. | -| `-s [ --secure ]` | Использовать ли TLS.

Включается автоматически при подключении к порту 9440 (безопасный порт по умолчанию) или к ClickHouse Cloud.

Вам может потребоваться настроить ваши CA-сертификаты в [конфигурационном файле](#configuration_files). Доступные параметры конфигурации такие же, как для [настройки TLS на стороне сервера](../operations/server-configuration-parameters/settings.md#openssl). | Автоматически включается при подключении к порту 9440 или ClickHouse Cloud | -| `--ssh-key-file ` | Файл, содержащий приватный SSH-ключ для аутентификации на сервере. | - | -| `--ssh-key-passphrase ` | Парольная фраза для приватного SSH-ключа, указанного в `--ssh-key-file`. | - | -| `-u [ --user ] ` | Пользователь базы данных, от имени которого выполняется подключение. | `default` | - -:::note -Вместо параметров `--host`, `--port`, `--user` и `--password` клиент также поддерживает [строки подключения](#connection_string). -::: - -### Параметры запросов {#command-line-options-query} - -| Опция | Описание | -|---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--param_=` | Значение подстановки для параметра [запроса с параметрами](#cli-queries-with-parameters). | -| `-q [ --query ] ` | Запрос для выполнения в пакетном режиме. Может быть указан несколько раз (`--query "SELECT 1" --query "SELECT 2"`) или один раз с несколькими запросами, разделёнными точкой с запятой (`--query "SELECT 1; SELECT 2;"`). В последнем случае `INSERT`‑запросы с форматами, отличными от `VALUES`, должны быть разделены пустыми строками.

Один запрос также можно передать без параметра `--query`: `clickhouse-client "SELECT 1"`

Нельзя использовать вместе с `--queries-file`. | -| `--queries-file ` | Путь к файлу, содержащему запросы. `--queries-file` может быть указан несколько раз, например: `--queries-file queries1.sql --queries-file queries2.sql`.

Нельзя использовать вместе с `--query`. | -| `-m [ --multiline ]` | Если указана опция, разрешает многострочные запросы (запрос не отправляется при нажатии Enter). Запросы будут отправляться только когда они заканчиваются точкой с запятой. | - -### Настройки запроса {#command-line-options-query-settings} - -Настройки запроса можно задать в клиенте как параметры командной строки, например: - -```bash -$ clickhouse-client --max_threads 1 -``` - -Список настроек см. в разделе [Settings](../operations/settings/settings.md). - - -### Параметры форматирования {#command-line-options-formatting} - -| Параметр | Описание | По умолчанию | -|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------| -| `-f [ --format ] ` | Использовать указанный формат для вывода результата.

Список поддерживаемых форматов см. в разделе [Форматы входных и выходных данных](formats.md). | `TabSeparated` | -| `--pager ` | Передавать весь вывод этой команде через конвейер. Обычно используется `less` (например, `less -S` для отображения широких результирующих наборов) или аналогичная программа. | - | -| `-E [ --vertical ]` | Использовать формат [Vertical](/interfaces/formats/Vertical) для вывода результата. То же самое, что и `--format Vertical`. В этом формате каждое значение выводится на отдельной строке, что удобно при отображении широких таблиц. | - | - -### Подробности выполнения {#command-line-options-execution-details} - -| Опция | Описание | Значение по умолчанию | -|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------| -| `--enable-progress-table-toggle` | Включить переключение таблицы прогресса нажатием сочетания клавиш Ctrl+Space. Применимо только в интерактивном режиме при включённой печати таблицы прогресса. | `enabled` | -| `--hardware-utilization` | Выводить информацию об использовании аппаратных ресурсов в индикаторе прогресса. | - | -| `--memory-usage` | Если указано, выводить использование памяти в `stderr` в неинтерактивном режиме.

Возможные значения:
• `none` — не выводить использование памяти
• `default` — выводить количество байт
• `readable` — выводить использование памяти в удобочитаемом формате | - | -| `--print-profile-events` | Выводить пакеты `ProfileEvents`. | - | -| `--progress` | Выводить прогресс выполнения запроса.

Возможные значения:
• `tty\|on\|1\|true\|yes` — вывод в терминал в интерактивном режиме
• `err` — вывод в `stderr` в неинтерактивном режиме
• `off\|0\|false\|no` — отключить вывод прогресса | `tty` в интерактивном режиме, `off` в неинтерактивном (пакетном) режиме | -| `--progress-table` | Выводить таблицу прогресса с изменяющимися метриками во время выполнения запроса.

Возможные значения:
• `tty\|on\|1\|true\|yes` — вывод в терминал в интерактивном режиме
• `err` — вывод в `stderr` в неинтерактивном режиме
• `off\|0\|false\|no` — отключить таблицу прогресса | `tty` в интерактивном режиме, `off` в неинтерактивном (пакетном) режиме | -| `--stacktrace` | Выводить трассировки стека исключений. | - | -| `-t [ --time ]` | Выводить время выполнения запроса в `stderr` в неинтерактивном режиме (для бенчмарков). | - | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cpp.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cpp.md deleted file mode 100644 index 9caf34a857f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/cpp.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'Документация по клиентской библиотеке ClickHouse на C++ и интеграции - с фреймворком u-server' -sidebar_label: 'Клиентская библиотека C++' -sidebar_position: 24 -slug: /interfaces/cpp -title: 'Клиентская библиотека C++' -doc_type: 'reference' ---- - -# Клиентская библиотека на C++ {#c-client-library} - -См. файл README в репозитории [clickhouse-cpp](https://github.com/ClickHouse/clickhouse-cpp). - -# Асинхронный фреймворк userver {#userver-asynchronous-framework} - -[userver (beta)](https://github.com/userver-framework/userver) имеет встроенную поддержку ClickHouse. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats.md deleted file mode 100644 index 3a210c21ffb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'Обзор поддерживаемых форматов входных и выходных данных в ClickHouse' -sidebar_label: 'Просмотреть все форматы...' -sidebar_position: 21 -slug: /interfaces/formats -title: 'Форматы входных и выходных данных' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - -# Форматы входных и выходных данных {#formats-for-input-and-output-data} - -ClickHouse поддерживает большинство известных текстовых и бинарных форматов данных. Это обеспечивает простую интеграцию практически в любой существующий конвейер данных и позволяет в полной мере использовать преимущества ClickHouse. - - - -## Форматы ввода {#input-formats} - -Форматы ввода используются для: -- Разбора данных, передаваемых в операторы `INSERT` -- Выполнения запросов `SELECT` к табличным данным с файловым хранилищем, таким как `File`, `URL` или `HDFS` -- Чтения словарей - -Выбор подходящего формата ввода критически важен для эффективной ингестии данных в ClickHouse. При наличии более чем 70 поддерживаемых форматов выбор наиболее производительного варианта может существенно повлиять на скорость вставки, использование CPU и памяти, а также общую эффективность системы. Чтобы упростить выбор, мы провели бенчмарк производительности ингестии для различных форматов, что позволило выявить ключевые выводы: - -- **Формат [Native](formats/Native.md) является наиболее эффективным форматом ввода**, обеспечивая наилучшее сжатие, минимальное использование ресурсов и минимальные накладные расходы на обработку на стороне сервера. -- **Сжатие имеет ключевое значение** — LZ4 уменьшает размер данных при минимальных затратах CPU, тогда как ZSTD обеспечивает более высокий уровень сжатия за счёт дополнительной нагрузки на CPU. -- **Предварительная сортировка оказывает умеренное влияние**, так как ClickHouse уже эффективно выполняет сортировку. -- **Объединение данных в батчи (batching) значительно повышает эффективность** — крупные батчи уменьшают накладные расходы на вставку и повышают пропускную способность. - -Для детального разбора результатов и рекомендаций по лучшим практикам ознакомьтесь с полной [аналитикой бенчмарка](https://www.clickhouse.com/blog/clickhouse-input-format-matchup-which-is-fastest-most-efficient). -Все результаты тестов доступны в онлайн-дашборде [FastFormats](https://fastformats.clickhouse.com/). - - - -## Форматы вывода {#output-formats} - -Поддерживаемые форматы вывода используются для: -- Представления результатов запроса `SELECT` -- Выполнения операций `INSERT` в таблицы с файловой поддержкой - - - -## Обзор форматов {#formats-overview} - -Поддерживаемые форматы: - - - -| Формат | Ввод | Вывод | -| ---------------------------------------------------------------------------------------------------------- | ---- | ----- | -| [TabSeparated](./formats/TabSeparated/TabSeparated.md) | ✔ | ✔ | -| [TabSeparatedRaw](./formats/TabSeparated/TabSeparatedRaw.md) | ✔ | ✔ | -| [TabSeparatedWithNames](./formats/TabSeparated/TabSeparatedWithNames.md) | ✔ | ✔ | -| [TabSeparatedWithNamesAndTypes](./formats/TabSeparated/TabSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [TabSeparatedRawWithNames](./formats/TabSeparated/TabSeparatedRawWithNames.md) | ✔ | ✔ | -| [TabSeparatedRawWithNamesAndTypes](./formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md) | ✔ | ✔ | -| [Template](./formats/Template/Template.md) | ✔ | ✔ | -| [TemplateIgnoreSpaces](./formats/Template/TemplateIgnoreSpaces.md) | ✔ | ✗ | -| [CSV](./formats/CSV/CSV.md) | ✔ | ✔ | -| [CSVWithNames](./formats/CSV/CSVWithNames.md) | ✔ | ✔ | -| [CSVWithNamesAndTypes](./formats/CSV/CSVWithNamesAndTypes.md) | ✔ | ✔ | -| [CustomSeparated](./formats/CustomSeparated/CustomSeparated.md) | ✔ | ✔ | -| [CustomSeparatedWithNames](./formats/CustomSeparated/CustomSeparatedWithNames.md) | ✔ | ✔ | -| [CustomSeparatedWithNamesAndTypes](./formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [SQLInsert](./formats/SQLInsert.md) | ✗ | ✔ | -| [Значения](./formats/Values.md) | ✔ | ✔ | -| [Vertical](./formats/Vertical.md) | ✗ | ✔ | -| [JSON](./formats/JSON/JSON.md) | ✔ | ✔ | -| [JSONAsString](./formats/JSON/JSONAsString.md) | ✔ | ✗ | -| [JSONAsObject](./formats/JSON/JSONAsObject.md) | ✔ | ✗ | -| [JSONStrings](./formats/JSON/JSONStrings.md) | ✔ | ✔ | -| [JSONColumns](./formats/JSON/JSONColumns.md) | ✔ | ✔ | -| [JSONColumnsWithMetadata](./formats/JSON/JSONColumnsWithMetadata.md) | ✔ | ✔ | -| [JSONCompact](./formats/JSON/JSONCompact.md) | ✔ | ✔ | -| [JSONCompactStrings](./formats/JSON/JSONCompactStrings.md) | ✗ | ✔ | -| [JSONCompactColumns](./formats/JSON/JSONCompactColumns.md) | ✔ | ✔ | -| [JSONEachRow](./formats/JSON/JSONEachRow.md) | ✔ | ✔ | -| [PrettyJSONEachRow](./formats/JSON/PrettyJSONEachRow.md) | ✗ | ✔ | -| [JSONEachRowWithProgress](./formats/JSON/JSONEachRowWithProgress.md) | ✗ | ✔ | -| [JSONStringsEachRow](./formats/JSON/JSONStringsEachRow.md) | ✔ | ✔ | -| [JSONStringsEachRowWithProgress](./formats/JSON/JSONStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactEachRow](./formats/JSON/JSONCompactEachRow.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNames](./formats/JSON/JSONCompactEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNamesAndTypes](./formats/JSON/JSONCompactEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactEachRowWithProgress](./formats/JSON/JSONCompactEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactStringsEachRow](./formats/JSON/JSONCompactStringsEachRow.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNames](./formats/JSON/JSONCompactStringsEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNamesAndTypes](./formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithProgress](./formats/JSON/JSONCompactStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONObjectEachRow](./formats/JSON/JSONObjectEachRow.md) | ✔ | ✔ | -| [BSONEachRow](./formats/BSONEachRow.md) | ✔ | ✔ | -| [TSKV](./formats/TabSeparated/TSKV.md) | ✔ | ✔ | -| [Pretty](./formats/Pretty/Pretty.md) | ✗ | ✔ | -| [PrettyNoEscapes](./formats/Pretty/PrettyNoEscapes.md) | ✗ | ✔ | -| [PrettyMonoBlock](./formats/Pretty/PrettyMonoBlock.md) | ✗ | ✔ | -| [PrettyNoEscapesMonoBlock](./formats/Pretty/PrettyNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettyCompact](./formats/Pretty/PrettyCompact.md) | ✗ | ✔ | -| [PrettyCompactNoEscapes](./formats/Pretty/PrettyCompactNoEscapes.md) | ✗ | ✔ | -| [PrettyCompactMonoBlock](./formats/Pretty/PrettyCompactMonoBlock.md) | ✗ | ✔ | -| [PrettyCompactNoEscapesMonoBlock](./formats/Pretty/PrettyCompactNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettySpace](./formats/Pretty/PrettySpace.md) | ✗ | ✔ | -| [PrettySpaceNoEscapes](./formats/Pretty/PrettySpaceNoEscapes.md) | ✗ | ✔ | -| [PrettySpaceMonoBlock](./formats/Pretty/PrettySpaceMonoBlock.md) | ✗ | ✔ | -| [PrettySpaceNoEscapesMonoBlock](./formats/Pretty/PrettySpaceNoEscapesMonoBlock.md) | ✗ | ✔ | -| [Prometheus](./formats/Prometheus.md) | ✗ | ✔ | -| [Protobuf](./formats/Protobuf/Protobuf.md) | ✔ | ✔ | -| [ProtobufSingle](./formats/Protobuf/ProtobufSingle.md) | ✔ | ✔ | -| [ProtobufList](./formats/Protobuf/ProtobufList.md) | ✔ | ✔ | -| [Avro](./formats/Avro/Avro.md) | ✔ | ✔ | -| [AvroConfluent](./formats/Avro/AvroConfluent.md) | ✔ | ✗ | -| [Parquet](./formats/Parquet/Parquet.md) | ✔ | ✔ | -| [ParquetMetadata](./formats/Parquet/ParquetMetadata.md) | ✔ | ✗ | -| [Arrow](./formats/Arrow/Arrow.md) | ✔ | ✔ | -| [ArrowStream](./formats/Arrow/ArrowStream.md) | ✔ | ✔ | -| [ORC](./formats/ORC.md) | ✔ | ✔ | -| [Один](./formats/One.md) | ✔ | ✗ | -| [Npy](./formats/Npy.md) | ✔ | ✔ | -| [RowBinary](./formats/RowBinary/RowBinary.md) | ✔ | ✔ | -| [RowBinaryWithNames](./formats/RowBinary/RowBinaryWithNames.md) | ✔ | ✔ | -| [RowBinaryWithNamesAndTypes](./formats/RowBinary/RowBinaryWithNamesAndTypes.md) | ✔ | ✔ | -| [RowBinaryWithDefaults](./formats/RowBinary/RowBinaryWithDefaults.md) | ✔ | ✗ | -| [Native](./formats/Native.md) | ✔ | ✔ | -| [Null](./formats/Null.md) | ✗ | ✔ | -| [Hash](./formats/Hash.md) | ✗ | ✔ | -| [XML](./formats/XML.md) | ✗ | ✔ | -| [CapnProto](./formats/CapnProto.md) | ✔ | ✔ | -| [LineAsString](./formats/LineAsString/LineAsString.md) | ✔ | ✔ | -| [LineAsStringWithNames](./formats/LineAsString/LineAsStringWithNames.md) | ✔ | ✔ | -| [LineAsStringWithNamesAndTypes](./formats/LineAsString/LineAsStringWithNamesAndTypes.md) | ✔ | ✔ | -| [Регулярные выражения](./formats/Regexp.md) | ✔ | ✗ | -| [RawBLOB](./formats/RawBLOB.md) | ✔ | ✔ | -| [MsgPack](./formats/MsgPack.md) | ✔ | ✔ | -| [MySQLDump](./formats/MySQLDump.md) | ✔ | ✗ | -| [DWARF](./formats/DWARF.md) | ✔ | ✗ | -| [Markdown](./formats/Markdown.md) | ✗ | ✔ | -| [Форма](./formats/Form.md) | ✔ | ✗ | - - - -Вы можете управлять некоторыми параметрами обработки форматов с помощью настроек ClickHouse. Подробнее см. раздел [Настройки](/operations/settings/settings-formats.md). - - - -## Схема формата {#formatschema} - -Имя файла, содержащего схему формата, задаётся настройкой `format_schema`. -Эту настройку необходимо задать при использовании одного из форматов `Cap'n Proto` или `Protobuf`. -Схема формата — это комбинация имени файла и имени типа сообщения в этом файле, разделённых двоеточием, -например, `schemafile.proto:MessageType`. -Если файл имеет стандартное расширение для формата (например, `.proto` для `Protobuf`), -его можно опустить, и в этом случае схема формата будет выглядеть как `schemafile:MessageType`. - -Если вы вводите или выводите данные через [клиент](/interfaces/cli.md) в интерактивном режиме, имя файла, указанное в схеме формата, -может содержать абсолютный путь или путь относительно текущего каталога на клиенте. -Если вы используете клиент в [пакетном режиме](/interfaces/cli.md/#batch-mode), путь к схеме должен быть относительным — по соображениям безопасности. - -Если вы вводите или выводите данные через [HTTP-интерфейс](/interfaces/http.md), имя файла, указанное в схеме формата, -должно находиться в каталоге, указанном в [format_schema_path](/operations/server-configuration-parameters/settings.md/#format_schema_path) -в конфигурации сервера. - - - -## Пропуск ошибок {#skippingerrors} - -Некоторые форматы, такие как `CSV`, `TabSeparated`, `TSKV`, `JSONEachRow`, `Template`, `CustomSeparated` и `Protobuf`, могут пропускать некорректную строку при возникновении ошибки парсинга и продолжать разбор начиная со следующей строки. См. настройки [input_format_allow_errors_num](/operations/settings/settings-formats.md/#input_format_allow_errors_num) и -[input_format_allow_errors_ratio](/operations/settings/settings-formats.md/#input_format_allow_errors_ratio). -Ограничения: -- В случае ошибки парсинга `JSONEachRow` пропускает все данные до новой строки (или EOF), поэтому строки должны быть разделены символом `\n`, чтобы корректно подсчитывать ошибки. -- `Template` и `CustomSeparated` используют разделитель после последнего столбца и разделитель между строками, чтобы найти начало следующей строки, поэтому пропуск ошибок работает только в том случае, если хотя бы один из этих разделителей не пуст. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md deleted file mode 100644 index 5336729e38a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Документация формата Arrow' -input_format: true -keywords: ['Arrow'] -output_format: true -slug: /interfaces/formats/Arrow -title: 'Arrow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -[Apache Arrow](https://arrow.apache.org/) включает два встроенных формата колоночного хранения данных. ClickHouse поддерживает операции чтения и записи для этих форматов. -`Arrow` — это формат Apache Arrow в режиме «file mode». Он предназначен для произвольного доступа к данным в оперативной памяти. - -## Соответствие типов данных {#data-types-matching} - -В таблице ниже показаны поддерживаемые типы данных и их соответствие [типам данных](/sql-reference/data-types/index.md) ClickHouse в запросах `INSERT` и `SELECT`. - -| Тип данных Arrow (`INSERT`) | Тип данных ClickHouse | Тип данных Arrow (`SELECT`) | -|-----------------------------------------|------------------------------------------------------------------------------------------------------------|-----------------------------| -| `BOOL` | [Bool](/sql-reference/data-types/boolean.md) | `BOOL` | -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md) | `INT64` | -| `FLOAT`, `HALF_FLOAT` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `DATE32` | [Date32](/sql-reference/data-types/date32.md) | `UINT16` | -| `DATE64` | [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](/sql-reference/data-types/datetime64.md) | `TIMESTAMP` | -| `STRING`, `BINARY` | [String](/sql-reference/data-types/string.md) | `BINARY` | -| `STRING`, `BINARY`, `FIXED_SIZE_BINARY` | [FixedString](/sql-reference/data-types/fixedstring.md) | `FIXED_SIZE_BINARY` | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | `DECIMAL` | -| `DECIMAL256` | [Decimal256](/sql-reference/data-types/decimal.md) | `DECIMAL256` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `MAP` | [Map](/sql-reference/data-types/map.md) | `MAP` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `FIXED_SIZE_BINARY`, `BINARY` | [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_SIZE_BINARY` | -| `FIXED_SIZE_BINARY`, `BINARY` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_SIZE_BINARY` | - -Массивы могут быть вложенными и в качестве типа элементов могут использовать значение типа `Nullable`. Типы `Tuple` и `Map` также могут быть вложенными. - -Тип `DICTIONARY` поддерживается для запросов `INSERT`, а для запросов `SELECT` доступна настройка [`output_format_arrow_low_cardinality_as_dictionary`](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary), которая позволяет выводить тип [LowCardinality](/sql-reference/data-types/lowcardinality.md) как тип `DICTIONARY`. - -Неподдерживаемые типы данных Arrow: - -- `FIXED_SIZE_BINARY` -- `JSON` -- `UUID` -- `ENUM`. - -Типы данных столбцов таблицы ClickHouse не обязаны совпадать с соответствующими полями Arrow. При вставке данных ClickHouse интерпретирует типы данных в соответствии с таблицей выше, а затем [приводит](/sql-reference/functions/type-conversion-functions#cast) данные к типу, заданному для столбца таблицы ClickHouse. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Вы можете вставить данные Arrow из файла в таблицу ClickHouse с помощью следующей команды: - -```bash -$ cat filename.arrow | clickhouse-client --query="INSERT INTO some_table FORMAT Arrow" -``` - - -### Выбор данных {#selecting-data} - -Вы можете выбрать данные из таблицы ClickHouse и сохранить их в файл формата Arrow с помощью следующей команды: - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Arrow" > {filename.arrow} -``` - - -## Настройки формата {#format-settings} - -| Параметр | Описание | Значение по умолчанию | -|--------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|------------------------| -| `input_format_arrow_allow_missing_columns` | Разрешать отсутствие столбцов при чтении входных форматов Arrow | `1` | -| `input_format_arrow_case_insensitive_column_matching` | Игнорировать регистр при сопоставлении столбцов Arrow со столбцами ClickHouse | `0` | -| `input_format_arrow_import_nested` | Устаревшая настройка, ничего не делает. | `0` | -| `input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference` | Пропускать столбцы с неподдерживаемыми типами при выводе (определении) схемы для формата Arrow | `0` | -| `output_format_arrow_compression_method` | Метод сжатия для формата вывода Arrow. Поддерживаемые кодеки: lz4_frame, zstd, none (без сжатия) | `lz4_frame` | -| `output_format_arrow_fixed_string_as_fixed_byte_array` | Использовать тип Arrow FIXED_SIZE_BINARY вместо Binary для столбцов FixedString. | `1` | -| `output_format_arrow_low_cardinality_as_dictionary` | Включить вывод типа LowCardinality как типа Arrow Dictionary | `0` | -| `output_format_arrow_string_as_string` | Использовать тип Arrow String вместо Binary для столбцов String | `1` | -| `output_format_arrow_use_64_bit_indexes_for_dictionary` | Всегда использовать 64-битные целые числа для индексов словаря в формате Arrow | `0` | -| `output_format_arrow_use_signed_indexes_for_dictionary` | Использовать знаковые целые числа для индексов словаря в формате Arrow | `1` | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md deleted file mode 100644 index aba13fbdf84..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -alias: [] -description: 'Документация по формату ArrowStream' -input_format: true -keywords: ['ArrowStream'] -output_format: true -slug: /interfaces/formats/ArrowStream -title: 'ArrowStream' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -`ArrowStream` — это формат «потокового режима» (stream mode) Apache Arrow. Он предназначен для потоковой обработки данных в памяти. - -## Пример использования {#example-usage} - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md deleted file mode 100644 index 41b3eb8273e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Avro' -input_format: true -keywords: ['Avro'] -output_format: true -slug: /interfaces/formats/Avro -title: 'Avro' -doc_type: 'reference' ---- - -import DataTypeMapping from './_snippets/data-types-matching.md' - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -[Apache Avro](https://avro.apache.org/) — это строчно-ориентированный формат сериализации данных, который использует двоичное кодирование для эффективной обработки. Формат `Avro` поддерживает чтение и запись [файлов данных Avro](https://avro.apache.org/docs/++version++/specification/#object-container-files). Этот формат рассчитан на самоописательные сообщения со встроенной схемой. Если вы используете Avro с реестром схем, обратитесь к формату [`AvroConfluent`](./AvroConfluent.md). - -## Сопоставление типов данных {#data-type-mapping} - - - -## Настройки формата {#format-settings} - -| Параметр | Описание | Значение по умолчанию | -|--------------------------------------------|----------------------------------------------------------------------------------------------------|------------------------| -| `input_format_avro_allow_missing_fields` | Использовать ли значение по умолчанию вместо генерации ошибки, если поле не найдено в схеме. | `0` | -| `input_format_avro_null_as_default` | Использовать ли значение по умолчанию вместо генерации ошибки при вставке значения `null` в столбец, не допускающий `null`. | `0` | -| `output_format_avro_codec` | Алгоритм сжатия для выходных файлов Avro. Возможные значения: `null`, `deflate`, `snappy`, `zstd`. | | -| `output_format_avro_sync_interval` | Интервал записи маркеров синхронизации в файлах Avro (в байтах). | `16384` | -| `output_format_avro_string_column_pattern` | Регулярное выражение для определения столбцов типа `String` для отображения в строковый тип Avro. По умолчанию столбцы ClickHouse типа `String` записываются как тип Avro `bytes`. | | -| `output_format_avro_rows_in_file` | Максимальное количество строк в одном выходном файле Avro. При достижении этого предела создаётся новый файл (если система хранения поддерживает разбиение файлов). | `1` | - -## Примеры {#examples} - -### Чтение данных в формате Avro {#reading-avro-data} - -Чтобы прочитать данные из файла в формате Avro в таблицу ClickHouse: - -```bash -$ cat file.avro | clickhouse-client --query="INSERT INTO {some_table} FORMAT Avro" -``` - -Корневая схема загружаемого файла Avro должна иметь тип `record`. - -Чтобы сопоставить столбцы таблицы с полями схемы Avro, ClickHouse сравнивает их имена. -Сравнение чувствительно к регистру, а неиспользуемые поля пропускаются. - -Типы данных столбцов таблицы ClickHouse могут отличаться от соответствующих полей вставляемых данных Avro. При вставке данных ClickHouse интерпретирует типы данных в соответствии с таблицей выше, а затем [приводит](/sql-reference/functions/type-conversion-functions#cast) данные к соответствующему типу столбца. - -При импорте данных, если поле не найдено в схеме и включена настройка [`input_format_avro_allow_missing_fields`](/operations/settings/settings-formats.md/#input_format_avro_allow_missing_fields), вместо генерации ошибки будет использовано значение по умолчанию. - - -### Запись данных в формате Avro {#writing-avro-data} - -Чтобы записать данные из таблицы ClickHouse в файл формата Avro: - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Avro" > file.avro -``` - -Имена столбцов должны: - -* Начинаться с `[A-Za-z_]` -* Далее могут включать только `[A-Za-z0-9_]` - -Сжатие выходных данных и интервал синхронизации для файлов Avro можно настроить с помощью параметров [`output_format_avro_codec`](/operations/settings/settings-formats.md/#output_format_avro_codec) и [`output_format_avro_sync_interval`](/operations/settings/settings-formats.md/#output_format_avro_sync_interval) соответственно. - - -### Определение схемы Avro {#inferring-the-avro-schema} - -С помощью функции ClickHouse [`DESCRIBE`](/sql-reference/statements/describe-table) вы можете быстро просмотреть выводимую (выведенную) схему файла Avro, как показано в следующем примере. -В этом примере используется URL общедоступного файла Avro в публичном бакете S3 ClickHouse: - -```sql -DESCRIBE url('https://clickhouse-public-datasets.s3.eu-central-1.amazonaws.com/hits.avro','Avro); - -┌─name───────────────────────┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ WatchID │ Int64 │ │ │ │ │ │ -│ JavaEnable │ Int32 │ │ │ │ │ │ -│ Title │ String │ │ │ │ │ │ -│ GoodEvent │ Int32 │ │ │ │ │ │ -│ EventTime │ Int32 │ │ │ │ │ │ -│ EventDate │ Date32 │ │ │ │ │ │ -│ CounterID │ Int32 │ │ │ │ │ │ -│ ClientIP │ Int32 │ │ │ │ │ │ -│ ClientIP6 │ FixedString(16) │ │ │ │ │ │ -│ RegionID │ Int32 │ │ │ │ │ │ -... -│ IslandID │ FixedString(16) │ │ │ │ │ │ -│ RequestNum │ Int32 │ │ │ │ │ │ -│ RequestTry │ Int32 │ │ │ │ │ │ -└────────────────────────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md deleted file mode 100644 index 206f5dd5ba7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -alias: [] -description: 'Документация о формате AvroConfluent' -input_format: true -keywords: ['AvroConfluent'] -output_format: false -slug: /interfaces/formats/AvroConfluent -title: 'AvroConfluent' -doc_type: 'reference' ---- - -import DataTypesMatching from './_snippets/data-types-matching.md' - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✗ | | - - -## Описание {#description} - -[Apache Avro](https://avro.apache.org/) — это строчно-ориентированный формат сериализации, который использует двоичное кодирование для эффективной обработки данных. Формат `AvroConfluent` поддерживает декодирование отдельных объектов — сообщений Kafka, закодированных в Avro и сериализованных с использованием [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/index.html) (или API-совместимых сервисов). - -Каждое Avro-сообщение содержит идентификатор схемы, по которому ClickHouse автоматически получает схему, обращаясь к настроенному реестру схем. После этого схемы кэшируются для оптимальной производительности. - -
- -## Сопоставление типов данных {#data-type-mapping} - - - -## Настройки формата {#format-settings} - -[//]: # "NOTE Эти настройки могут быть заданы на уровне сессии, но это используется нечасто, и слишком подробная документация по этому поводу может запутать пользователей." - -| Параметр | Описание | Значение по умолчанию | -|--------------------------------------------|----------------------------------------------------------------------------------------------------|------------------------| -| `input_format_avro_allow_missing_fields` | Использовать ли значение по умолчанию вместо возникновения ошибки, когда поле не найдено в схеме. | `0` | -| `input_format_avro_null_as_default` | Использовать ли значение по умолчанию вместо возникновения ошибки при вставке значения `null` в столбец, не допускающий значения `null`. | `0` | -| `format_avro_schema_registry_url` | URL Confluent Schema Registry. Для базовой аутентификации URL-кодированные учетные данные могут быть включены непосредственно в путь URL. | | - -## Примеры {#examples} - -### Использование реестра схем {#using-a-schema-registry} - -Чтобы прочитать Kafka-топик, закодированный в Avro, с помощью [движка таблиц Kafka](/engines/table-engines/integrations/kafka.md), используйте настройку `format_avro_schema_registry_url` для указания URL-адреса реестра схем. - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'http://schema-registry-url'; - -SELECT * FROM topic1_stream; -``` - - -#### Использование базовой аутентификации {#using-basic-authentication} - -Если для вашего реестра схем требуется базовая аутентификация (например, при использовании Confluent Cloud), вы можете указать URL-кодированные учетные данные в настройке `format_avro_schema_registry_url`. - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'https://:@schema-registry-url'; -``` - - -## Диагностика неполадок {#troubleshooting} - -Чтобы отслеживать ход ингестии и отлаживать ошибки потребителя Kafka, вы можете выполнить запрос к [системной таблице `system.kafka_consumers`](../../../operations/system-tables/kafka_consumers.md). Если в вашем развертывании несколько реплик (например, ClickHouse Cloud), необходимо использовать табличную функцию [`clusterAllReplicas`](../../../sql-reference/table-functions/cluster.md). - -```sql -SELECT * FROM clusterAllReplicas('default',system.kafka_consumers) -ORDER BY assignments.partition_id ASC; -``` - -Если вы столкнулись с проблемами с разрешением схемы, вы можете использовать [kafkacat](https://github.com/edenhill/kafkacat) вместе с [clickhouse-local](/operations/utilities/clickhouse-local.md) для диагностики: - -```bash -$ kafkacat -b kafka-broker -C -t topic1 -o beginning -f '%s' -c 3 | clickhouse-local --input-format AvroConfluent --format_avro_schema_registry_url 'http://schema-registry' -S "field1 Int64, field2 String" -q 'select * from table' -1 a -2 b -3 c -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md deleted file mode 100644 index a14d3e83a09..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md +++ /dev/null @@ -1,40 +0,0 @@ -В таблице ниже приведены все типы данных, поддерживаемые форматом Apache Avro, и их соответствующие [типы данных](/sql-reference/data-types/index.md) ClickHouse в запросах `INSERT` и `SELECT`. - -| Тип данных Avro `INSERT` | Тип данных ClickHouse | Тип данных Avro `SELECT` | -|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|---------------------------------| -| `boolean`, `int`, `long`, `float`, `double` | [Int(8\16\32)](/sql-reference/data-types/int-uint.md), [UInt(8\16\32)](/sql-reference/data-types/int-uint.md) | `int` | -| `boolean`, `int`, `long`, `float`, `double` | [Int64](/sql-reference/data-types/int-uint.md), [UInt64](/sql-reference/data-types/int-uint.md) | `long` | -| `boolean`, `int`, `long`, `float`, `double` | [Float32](/sql-reference/data-types/float.md) | `float` | -| `boolean`, `int`, `long`, `float`, `double` | [Float64](/sql-reference/data-types/float.md) | `double` | -| `bytes`, `string`, `fixed`, `enum` | [String](/sql-reference/data-types/string.md) | `bytes` или `string` \* | -| `bytes`, `string`, `fixed` | [FixedString(N)](/sql-reference/data-types/fixedstring.md) | `fixed(N)` | -| `enum` | [Enum(8\16)](/sql-reference/data-types/enum.md) | `enum` | -| `array(T)` | [Array(T)](/sql-reference/data-types/array.md) | `array(T)` | -| `map(V, K)` | [Map(V, K)](/sql-reference/data-types/map.md) | `map(string, K)` | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(null, T)` | -| `union(T1, T2, …)` \** | [Variant(T1, T2, …)](/sql-reference/data-types/variant.md) | `union(T1, T2, …)` \** | -| `null` | [Nullable(Nothing)](/sql-reference/data-types/special-data-types/nothing.md) | `null` | -| `int (date)` \**\* | [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md) | `int (date)` \**\* | -| `long (timestamp-millis)` \**\* | [DateTime64(3)](/sql-reference/data-types/datetime.md) | `long (timestamp-millis)` \**\* | -| `long (timestamp-micros)` \**\* | [DateTime64(6)](/sql-reference/data-types/datetime.md) | `long (timestamp-micros)` \**\* | -| `bytes (decimal)` \**\* | [DateTime64(N)](/sql-reference/data-types/datetime.md) | `bytes (decimal)` \**\* | -| `int` | [IPv4](/sql-reference/data-types/ipv4.md) | `int` | -| `fixed(16)` | [IPv6](/sql-reference/data-types/ipv6.md) | `fixed(16)` | -| `bytes (decimal)` \**\* | [Decimal(P, S)](/sql-reference/data-types/decimal.md) | `bytes (decimal)` \**\* | -| `string (uuid)` \**\* | [UUID](/sql-reference/data-types/uuid.md) | `string (uuid)` \**\* | -| `fixed(16)` | [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `fixed(16)` | -| `fixed(32)` | [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `fixed(32)` | -| `record` | [Tuple](/sql-reference/data-types/tuple.md) | `record` | - -\* По умолчанию используется `bytes`; поведение определяется настройкой [`output_format_avro_string_column_pattern`](/operations/settings/settings-formats.md/#output_format_avro_string_column_pattern) - -\** Тип [Variant](/sql-reference/data-types/variant) неявно допускает `null` в качестве значения поля, поэтому, например, Avro `union(T1, T2, null)` будет преобразован в `Variant(T1, T2)`. -В результате при формировании Avro из ClickHouse мы всегда должны включать тип `null` в набор типов Avro `union`, так как при выводе схемы мы не знаем, является ли какое-либо значение фактически `null`. - -\**\* [Логические типы Avro](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -Неподдерживаемые логические типы данных Avro: - -- `time-millis` -- `time-micros` -- `duration` \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md deleted file mode 100644 index 812f0e1b7a9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -alias: [] -description: 'Документация по формату BSONEachRow' -input_format: true -keywords: ['BSONEachRow'] -output_format: true -slug: /interfaces/formats/BSONEachRow -title: 'BSONEachRow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Формат `BSONEachRow` разбирает данные как последовательность документов в формате Binary JSON (BSON) без каких-либо разделителей между ними. -Каждая строка представлена одним документом, а каждый столбец — одним полем BSON-документа, где в качестве ключа используется имя столбца. - -## Соответствие типов данных {#data-types-matching} - -При выводе используется следующее соответствие между типами ClickHouse и типами BSON: - -| Тип ClickHouse | Тип BSON | -|----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| -| [Bool](/sql-reference/data-types/boolean.md) | `\x08` boolean | -| [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int32](/sql-reference/data-types/int-uint.md) | `\x10` int32 | -| [UInt32](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Int64/UInt64](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Float32/Float64](/sql-reference/data-types/float.md) | `\x01` double | -| [Date](/sql-reference/data-types/date.md)/[Date32](/sql-reference/data-types/date32.md) | `\x10` int32 | -| [DateTime](/sql-reference/data-types/datetime.md) | `\x12` int64 | -| [DateTime64](/sql-reference/data-types/datetime64.md) | `\x09` datetime | -| [Decimal32](/sql-reference/data-types/decimal.md) | `\x10` int32 | -| [Decimal64](/sql-reference/data-types/decimal.md) | `\x12` int64 | -| [Decimal128](/sql-reference/data-types/decimal.md) | `\x05` binary, `\x00` подтип binary, размер = 16 | -| [Decimal256](/sql-reference/data-types/decimal.md) | `\x05` binary, `\x00` подтип binary, размер = 32 | -| [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `\x05` binary, `\x00` подтип binary, размер = 16 | -| [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `\x05` binary, `\x00` подтип binary, размер = 32 | -| [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | `\x05` binary, `\x00` подтип binary или \x02 string, если включена настройка output_format_bson_string_as_string | -| [UUID](/sql-reference/data-types/uuid.md) | `\x05` binary, `\x04` подтип uuid, размер = 16 | -| [Array](/sql-reference/data-types/array.md) | `\x04` array | -| [Tuple](/sql-reference/data-types/tuple.md) | `\x04` array | -| [Named Tuple](/sql-reference/data-types/tuple.md) | `\x03` document | -| [Map](/sql-reference/data-types/map.md) | `\x03` document | -| [IPv4](/sql-reference/data-types/ipv4.md) | `\x10` int32 | -| [IPv6](/sql-reference/data-types/ipv6.md) | `\x05` binary, `\x00` подтип binary | - -При вводе используется следующее соответствие между типами BSON и типами ClickHouse: - -| Тип BSON | Тип ClickHouse | -|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `\x01` double | [Float32/Float64](/sql-reference/data-types/float.md) | -| `\x02` string | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x03` document | [Map](/sql-reference/data-types/map.md)/[Named Tuple](/sql-reference/data-types/tuple.md) | -| `\x04` array | [Array](/sql-reference/data-types/array.md)/[Tuple](/sql-reference/data-types/tuple.md) | -| `\x05` binary, `\x00` двоичный подтип | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md)/[IPv6](/sql-reference/data-types/ipv6.md) | -| `\x05` binary, `\x02` старый двоичный подтип | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x05` binary, `\x03` старый подтип uuid | [UUID](/sql-reference/data-types/uuid.md) | -| `\x05` binary, `\x04` подтип uuid | [UUID](/sql-reference/data-types/uuid.md) | -| `\x07` ObjectId | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x08` boolean | [Bool](/sql-reference/data-types/boolean.md) | -| `\x09` datetime | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `\x0A` null value | [NULL](/sql-reference/data-types/nullable.md) | -| `\x0D` JavaScript code | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x0E` symbol | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x10` int32 | [Int32/UInt32](/sql-reference/data-types/int-uint.md)/[Decimal32](/sql-reference/data-types/decimal.md)/[IPv4](/sql-reference/data-types/ipv4.md)/[Enum8/Enum16](/sql-reference/data-types/enum.md) | -| `\x12` int64 | [Int64/UInt64](/sql-reference/data-types/int-uint.md)/[Decimal64](/sql-reference/data-types/decimal.md)/[DateTime64](/sql-reference/data-types/datetime64.md) | - -Другие типы BSON не поддерживаются. Кроме того, выполняется преобразование между различными целочисленными типами. -Например, можно вставить значение BSON типа `int32` в ClickHouse как [`UInt8`](../../sql-reference/data-types/int-uint.md). - -Большие целые и десятичные числа, такие как `Int128`/`UInt128`/`Int256`/`UInt256`/`Decimal128`/`Decimal256`, могут быть получены при разборе значения BSON типа Binary с двоичным подтипом `\x00`. -В этом случае формат проверяет, что размер двоичных данных равен размеру ожидаемого значения. - -:::note -Этот формат работает некорректно на платформах с порядком байтов Big-Endian. -::: - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем BSON-файл со следующими данными с именем `football.bson`: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.bson' FORMAT BSONEachRow; -``` - - -### Чтение данных {#reading-data} - -Считывайте данные в формате `BSONEachRow`: - -```sql -SELECT * -FROM football INTO OUTFILE 'docs_data/bson/football.bson' -FORMAT BSONEachRow -``` - -:::tip -BSON — это двоичный формат, который не отображается в человекочитаемом виде в терминале. Используйте `INTO OUTFILE` для вывода файлов BSON. -::: - - -## Настройки формата {#format-settings} - -| Параметр | Описание | Значение по умолчанию | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------|------------------------| -| [`output_format_bson_string_as_string`](../../operations/settings/settings-formats.md/#output_format_bson_string_as_string) | Использовать тип BSON String вместо Binary для строковых столбцов. | `false` | -| [`input_format_bson_skip_fields_with_unsupported_types_in_schema_inference`](../../operations/settings/settings-formats.md/#input_format_bson_skip_fields_with_unsupported_types_in_schema_inference) | Разрешить пропуск столбцов с неподдерживаемыми типами при определении схемы для формата BSONEachRow. | `false` | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md deleted file mode 100644 index 897bc2efb2b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -alias: [] -description: 'Документация по формату CSV' -input_format: true -keywords: ['CSV'] -output_format: true -slug: /interfaces/formats/CSV -title: 'CSV' -doc_type: 'reference' ---- - -## Описание {#description} - -Формат значений, разделённых запятыми (Comma Separated Values, [RFC](https://tools.ietf.org/html/rfc4180)). -При форматировании строки заключаются в двойные кавычки. Двойная кавычка внутри строки выводится как две двойные кавычки подряд. -Других правил экранирования символов нет. - -* Даты и дата-время заключаются в двойные кавычки. -* Числа выводятся без кавычек. -* Значения разделяются символом-разделителем, который по умолчанию — `,`. Символ-разделитель задаётся настройкой [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter). -* Строки разделяются с использованием перевода строки в стиле Unix (LF). -* Массивы сериализуются в CSV следующим образом: - * сначала массив сериализуется в строку так же, как в формате TabSeparated, - * Полученная строка выводится в CSV в двойных кавычках. -* Кортежи в формате CSV сериализуются как отдельные столбцы (то есть их вложенность в кортеже теряется). - -```bash -$ clickhouse-client --format_csv_delimiter="|" --query="INSERT INTO test.csv FORMAT CSV" < data.csv -``` - -:::note -По умолчанию разделителем является `,`. -Дополнительные сведения см. в настройке [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter). -::: - -При разборе все значения могут обрабатываться как с кавычками, так и без них. Поддерживаются как двойные, так и одинарные кавычки. - -Строки также могут быть без кавычек. В этом случае они разбираются до символа-разделителя или символа новой строки (CR или LF). -Однако в нарушение RFC при разборе строк без кавычек начальные и конечные пробелы и табуляции игнорируются. -Переход на новую строку поддерживает форматы: Unix (LF), Windows (CR LF) и Mac OS Classic (CR LF). - -`NULL` форматируется в соответствии с настройкой [format_csv_null_representation](/operations/settings/settings-formats.md/#format_csv_null_representation) (значение по умолчанию — `\N`). - -Во входных данных значения `ENUM` могут быть представлены как именами, так и идентификаторами. -Сначала выполняется попытка сопоставить входное значение с именем ENUM. -Если это не удаётся и входное значение является числом, выполняется попытка сопоставить это число с идентификатором ENUM. -Если входные данные содержат только идентификаторы ENUM, рекомендуется включить настройку [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) для оптимизации разбора `ENUM`. - - -## Пример использования {#example-usage} - -## Настройки формата {#format-settings} - -| Параметр | Описание | Значение по умолчанию | Примечания | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) | символ, который рассматривается как разделитель в данных CSV. | `,` | | -| [format_csv_allow_single_quotes](/operations/settings/settings-formats.md/#format_csv_allow_single_quotes) | разрешать строки в одинарных кавычках. | `true` | | -| [format_csv_allow_double_quotes](/operations/settings/settings-formats.md/#format_csv_allow_double_quotes) | разрешать строки в двойных кавычках. | `true` | | -| [format_csv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation) | пользовательское представление NULL в формате CSV. | `\N` | | -| [input_format_csv_empty_as_default](/operations/settings/settings-formats.md/#input_format_csv_empty_as_default) | рассматривать пустые поля во входном CSV как значения по умолчанию. | `true` | Для сложных выражений по умолчанию также должен быть включён [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields). | -| [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) | рассматривать вставленные значения Enum в CSV как индексы Enum. | `false` | | -| [input_format_csv_use_best_effort_in_schema_inference](/operations/settings/settings-formats.md/#input_format_csv_use_best_effort_in_schema_inference) | использовать дополнительные приёмы и эвристики для определения схемы в формате CSV. Если отключено, все поля будут интерпретироваться как строки. | `true` | | -| [input_format_csv_arrays_as_nested_csv](/operations/settings/settings-formats.md/#input_format_csv_arrays_as_nested_csv) | при чтении Array из CSV ожидать, что его элементы были сериализованы во вложенный CSV и затем помещены в строку. | `false` | | -| [output_format_csv_crlf_end_of_line](/operations/settings/settings-formats.md/#output_format_csv_crlf_end_of_line) | если установлено значение `true`, конец строки в выходном формате CSV будет `\r\n` вместо `\n`. | `false` | | -| [input_format_csv_skip_first_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_first_lines) | пропускать указанное количество строк в начале данных. | `0` | | -| [input_format_csv_detect_header](/operations/settings/settings-formats.md/#input_format_csv_detect_header) | автоматически определять заголовок с именами и типами в формате CSV. | `true` | | -| [input_format_csv_skip_trailing_empty_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_trailing_empty_lines) | пропускать завершающие пустые строки в конце данных. | `false` | | -| [input_format_csv_trim_whitespaces](/operations/settings/settings-formats.md/#input_format_csv_trim_whitespaces) | обрезать пробелы и табуляции в строках CSV вне кавычек. | `true` | | -| [input_format_csv_allow_whitespace_or_tab_as_delimiter](/operations/settings/settings-formats.md/#input_format_csv_allow_whitespace_or_tab_as_delimiter) | разрешать использовать пробел или табуляцию как разделитель полей в строках CSV. | `false` | | -| [input_format_csv_allow_variable_number_of_columns](/operations/settings/settings-formats.md/#input_format_csv_allow_variable_number_of_columns) | разрешать переменное количество столбцов в формате CSV, игнорировать лишние столбцы и использовать значения по умолчанию для отсутствующих столбцов. | `false` | | -| [input_format_csv_use_default_on_bad_values](/operations/settings/settings-formats.md/#input_format_csv_use_default_on_bad_values) | разрешать устанавливать значение по умолчанию для столбца, когда десериализация поля CSV не удалась из‑за некорректного значения. | `false` | | -| [input_format_csv_try_infer_numbers_from_strings](/operations/settings/settings-formats.md/#input_format_csv_try_infer_numbers_from_strings) | пытаться выводить числовые значения из строковых полей при определении схемы. | `false` | | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md deleted file mode 100644 index 62b81aca022..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -alias: [] -description: 'Документация о формате CSV' -input_format: true -keywords: ['CSVWithNames'] -output_format: true -slug: /interfaces/formats/CSVWithNames -title: 'CSVWithNames' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Также выводит строку заголовка с названиями столбцов, аналогично [TabSeparatedWithNames](/interfaces/formats/TabSeparatedWithNames). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -:::tip -Начиная с [версии](https://github.com/ClickHouse/ClickHouse/releases) 23.1, ClickHouse автоматически распознаёт заголовки в файлах CSV при использовании формата `CSV`, поэтому нет необходимости использовать `CSVWithNames` или `CSVWithNamesAndTypes`. -::: - -Возьмём следующий CSV-файл с именем `football.csv`: - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -Создайте таблицу: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -Вставьте данные в формате `CSVWithNames`: - -```sql -INSERT INTO football FROM INFILE 'football.csv' FORMAT CSVWithNames; -``` - - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `CSVWithNames`: - -```sql -SELECT * -FROM football -FORMAT CSVWithNames -``` - -На выходе получится CSV‑файл с одной строкой заголовка: - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) установлена в значение `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если настройка [input_format_skip_unknown_fields](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) также установлена в значение `1`. -В противном случае первая строка входных данных будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md deleted file mode 100644 index c4d757c869e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -alias: [] -description: 'Документация по формату CSVWithNamesAndTypes' -input_format: true -keywords: ['CSVWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CSVWithNamesAndTypes -title: 'CSVWithNamesAndTypes' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Также выводит две заголовочные строки с именами и типами столбцов, аналогично формату [TabSeparatedWithNamesAndTypes](../formats/TabSeparatedWithNamesAndTypes). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -:::tip -Начиная с [версии](https://github.com/ClickHouse/ClickHouse/releases) 23.1, ClickHouse автоматически распознаёт заголовки в CSV-файлах при использовании формата `CSV`, поэтому нет необходимости использовать `CSVWithNames` или `CSVWithNamesAndTypes`. -::: - -Используем следующий CSV-файл с именем `football_types.csv`: - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -Date,Int16,LowCardinality(String),LowCardinality(String),Int8,Int8 -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -Создайте таблицу: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -Вставьте данные, используя формат `CSVWithNamesAndTypes`: - -```sql -INSERT INTO football FROM INFILE 'football_types.csv' FORMAT CSVWithNamesAndTypes; -``` - - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `CSVWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT CSVWithNamesAndTypes -``` - -Результатом будет CSV с двумя строками заголовков: одна для названий столбцов, другая — для их типов: - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"Date","Int16","LowCardinality(String)","LowCardinality(String)","Int8","Int8" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## Настройки формата {#format-settings} - -:::note -Если параметр [input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) установлен в значение `1`, -столбцы из входных данных будут сопоставляться со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если параметр [input_format_skip_unknown_fields](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлен в значение `1`. -В противном случае первая строка будет пропущена. -::: - -:::note -Если параметр [input_format_with_types_use_header](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) установлен в значение `1`, -типы из входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md deleted file mode 100644 index 8ca0f16af83..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -alias: [] -description: 'Документация по CapnProto' -input_format: true -keywords: ['CapnProto'] -output_format: true -slug: /interfaces/formats/CapnProto -title: 'CapnProto' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -Формат `CapnProto` — это двоичный формат сообщений, похожий на формат [`Protocol Buffers`](https://developers.google.com/protocol-buffers/) и [Thrift](https://en.wikipedia.org/wiki/Apache_Thrift), но, в отличие от [JSON](./JSON/JSON.md) или [MessagePack](https://msgpack.org/), не имеет с ними общего. -Сообщения CapnProto строго типизированы и не являются самоописательными, то есть им требуется внешнее описание схемы. Схема применяется на лету и кэшируется для каждого запроса. - -См. также [Схема формата](/interfaces/formats/#formatschema). - -## Соответствие типов данных {#data_types-matching-capnproto} - -Приведённая ниже таблица показывает поддерживаемые типы данных и то, как они сопоставляются с [типами данных](/sql-reference/data-types/index.md) ClickHouse в запросах `INSERT` и `SELECT`. - -| Тип данных CapnProto (`INSERT`) | Тип данных ClickHouse | Тип данных CapnProto (`SELECT`) | -|-------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------| -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md), [Date](/sql-reference/data-types/date.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md), [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md), [Decimal32](/sql-reference/data-types/decimal.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md), [DateTime64](/sql-reference/data-types/datetime.md), [Decimal64](/sql-reference/data-types/decimal.md) | `INT64` | -| `FLOAT32` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `FLOAT64` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `TEXT, DATA` | [String](/sql-reference/data-types/string.md), [FixedString](/sql-reference/data-types/fixedstring.md) | `TEXT, DATA` | -| `union(T, Void), union(Void, T)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(T, Void), union(Void, T)` | -| `ENUM` | [Enum(8/16)](/sql-reference/data-types/enum.md) | `ENUM` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `DATA` | [IPv6](/sql-reference/data-types/ipv6.md) | `DATA` | -| `DATA` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `DATA` | -| `DATA` | [Decimal128/Decimal256](/sql-reference/data-types/decimal.md) | `DATA` | -| `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | [Map](/sql-reference/data-types/map.md) | `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | - -- Целочисленные типы могут быть преобразованы друг в друга при вводе и выводе данных. -- Для работы с `Enum` в формате CapnProto используйте настройку [format_capn_proto_enum_comparising_mode](/operations/settings/settings-formats.md/#format_capn_proto_enum_comparising_mode). -- Массивы могут быть вложенными и могут иметь значение типа `Nullable` в качестве аргумента. Типы `Tuple` и `Map` также могут быть вложенными. - -## Пример использования {#example-usage} - -### Вставка и выборка данных {#inserting-and-selecting-data-capnproto} - -Вы можете вставить данные формата CapnProto из файла в таблицу ClickHouse с помощью следующей команды: - -```bash -$ cat capnproto_messages.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_schema = 'schema:Message' FORMAT CapnProto" -``` - -где файл `schema.capnp` выглядит следующим образом: - -```capnp -struct Message { - SearchPhrase @0 :Text; - c @1 :Uint64; -} -``` - -Вы можете выбрать данные из таблицы ClickHouse и сохранить их в файл в формате `CapnProto`, используя следующую команду: - -```bash -$ clickhouse-client --query = "SELECT * FROM test.hits FORMAT CapnProto SETTINGS format_schema = 'schema:Message'" -``` - - -### Использование автоматически сгенерированной схемы {#using-autogenerated-capn-proto-schema} - -Если у вас нет внешней схемы `CapnProto` для ваших данных, вы все равно можете считывать и выводить данные в формате `CapnProto`, используя автоматически сгенерированную схему. - -Например: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS format_capn_proto_use_autogenerated_schema=1 -``` - -В этом случае ClickHouse автоматически сгенерирует схему CapnProto в соответствии со структурой таблицы с помощью функции [structureToCapnProtoSchema](/sql-reference/functions/other-functions.md#structureToCapnProtoSchema) и будет использовать эту схему для сериализации данных в формате CapnProto. - -Вы также можете прочитать файл CapnProto с автоматически сгенерированной схемой (в этом случае файл должен быть создан с использованием той же схемы): - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_capn_proto_use_autogenerated_schema=1 FORMAT CapnProto" -``` - - -## Настройки формата {#format-settings} - -Настройка [`format_capn_proto_use_autogenerated_schema`](../../operations/settings/settings-formats.md/#format_capn_proto_use_autogenerated_schema) включена по умолчанию и применяется, если параметр [`format_schema`](/interfaces/formats#formatschema) не задан. - -Вы также можете сохранить автоматически сгенерированную схему в файл при операциях ввода/вывода, используя настройку [`output_format_schema`](/operations/settings/formats#output_format_schema). - -Например: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS - format_capn_proto_use_autogenerated_schema=1, - output_format_schema='path/to/schema/schema.capnp' -``` - -В этом случае автоматически сгенерированная схема `CapnProto` будет сохранена в файле `path/to/schema/schema.capnp`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md deleted file mode 100644 index b430b1db54e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: [] -description: 'Документация по формату CustomSeparated' -input_format: true -keywords: ['CustomSeparated'] -output_format: true -slug: /interfaces/formats/CustomSeparated -title: 'CustomSeparated' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Аналогично формату [Template](../Template/Template.md), но выводит или читает все имена и типы столбцов и использует правило экранирования из настройки [format_custom_escaping_rule](../../../operations/settings/settings-formats.md/#format_custom_escaping_rule), а также разделители из следующих настроек: - -- [format_custom_field_delimiter](/operations/settings/settings-formats.md/#format_custom_field_delimiter) -- [format_custom_row_before_delimiter](/operations/settings/settings-formats.md/#format_custom_row_before_delimiter) -- [format_custom_row_after_delimiter](/operations/settings/settings-formats.md/#format_custom_row_after_delimiter) -- [format_custom_row_between_delimiter](/operations/settings/settings-formats.md/#format_custom_row_between_delimiter) -- [format_custom_result_before_delimiter](/operations/settings/settings-formats.md/#format_custom_result_before_delimiter) -- [format_custom_result_after_delimiter](/operations/settings/settings-formats.md/#format_custom_result_after_delimiter) - -:::note -Не использует настройки правил экранирования и разделители, определённые в строках формата. -::: - -Также существует формат [`CustomSeparatedIgnoreSpaces`](../CustomSeparated/CustomSeparatedIgnoreSpaces.md), аналогичный [TemplateIgnoreSpaces](../Template//TemplateIgnoreSpaces.md). - -## Примеры использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте следующий текстовый файл `football.txt`: - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparated; -``` - - -### Чтение данных {#reading-data} - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Считайте данные, используя формат `CustomSeparated`: - -```sql -SELECT * -FROM football -FORMAT CustomSeparated -``` - -Результат будет выведен в настроенном пользовательском формате: - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## Настройки формата {#format-settings} - -Дополнительные настройки: - -| Настройка | Описание | Значение по умолчанию | -|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|------------------------| -| [input_format_custom_detect_header](../../../operations/settings/settings-formats.md/#input_format_custom_detect_header) | включает автоматическое определение заголовка с именами и типами, если он присутствует. | `true` | -| [input_format_custom_skip_trailing_empty_lines](../../../operations/settings/settings-formats.md/#input_format_custom_skip_trailing_empty_lines) | пропускает завершающие пустые строки в конце файла. | `false` | -| [input_format_custom_allow_variable_number_of_columns](../../../operations/settings/settings-formats.md/#input_format_custom_allow_variable_number_of_columns) | допускает переменное число столбцов в формате CustomSeparated, игнорирует лишние столбцы и использует значения по умолчанию для отсутствующих столбцов. | `false` | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md deleted file mode 100644 index daa10513193..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Документация по формату CustomSeparatedIgnoreSpaces' -keywords: ['CustomSeparatedIgnoreSpaces'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpaces -title: 'CustomSeparatedIgnoreSpaces' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | | | - -## Описание {#description} - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используя следующий текстовый файл с именем `football.txt`: - -```text -row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -Выполните настройку параметров пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpaces; -``` - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md deleted file mode 100644 index f45cc616c3f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Документация по формату CustomSeparatedIgnoreSpacesWithNames' -keywords: ['CustomSeparatedIgnoreSpacesWithNames'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNames -title: 'CustomSeparatedIgnoreSpacesWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | | | - -## Описание {#description} - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте следующий текстовый файл `football.txt`: - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNames; -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md deleted file mode 100644 index 61a12524168..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Документация по формату CustomSeparatedIgnoreSpacesWithNamesAndTypes' -keywords: ['CustomSeparatedIgnoreSpacesWithNamesAndTypes'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNamesAndTypes -title: 'CustomSeparatedIgnoreSpacesWithNamesAndTypes' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✔ | | | - -## Описание {#description} - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Для вставки данных используйте следующий текстовый файл `football.txt`: - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('Date'; 'Int16'; 'LowCardinality(String)'; 'LowCardinality(String)'; 'Int8'; 'Int8'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNamesAndTypes; -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md deleted file mode 100644 index bf7f6be7097..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -alias: [] -description: 'Документация по формату CustomSeparatedWithNames' -input_format: true -keywords: ['CustomSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNames -title: 'CustomSeparatedWithNames' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|------|-------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -Также выводит строку заголовков с именами столбцов, аналогично [TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий текстовый файл `football.txt`: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -Настройте параметры настраиваемого разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNames; -``` - - -### Чтение данных {#reading-data} - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Прочитайте данные в формате `CustomSeparatedWithNames`: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNames -``` - -Результат будет в настроенном пользовательском формате: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) имеет значение `1`, -то столбцы из входных данных будут сопоставляться со столбцами таблицы по их именам, -а столбцы с неизвестными именами будут пропускаться, если настройка [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) имеет значение `1`. -В противном случае первая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md deleted file mode 100644 index cd1215b63fe..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -alias: [] -description: 'Документация по формату CustomSeparatedWithNamesAndTypes' -input_format: true -keywords: ['CustomSeparatedWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNamesAndTypes -title: 'CustomSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Также выводит две строки заголовков с названиями и типами столбцов, аналогично формату [TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте следующий текстовый файл `football.txt`: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNamesAndTypes; -``` - - -### Чтение данных {#reading-data} - -Настройте параметры пользовательского разделителя: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -Считайте данные в формате `CustomSeparatedWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNamesAndTypes -``` - -Результат будет выведен в настроенном пользовательском формате: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) равна `1`, -столбцы во входных данных будут сопоставляться со столбцами таблицы по имени, а столбцы с неизвестными именами будут пропускаться, если настройка [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) также равна `1`. -В противном случае первая строка будет пропущена. -::: - -:::note -Если настройка [`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) равна `1`, -типы во входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md deleted file mode 100644 index 9869037fcbb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -alias: [] -description: 'Документация по формату DWARF' -input_format: true -keywords: ['DWARF'] -output_format: false -slug: /interfaces/formats/DWARF -title: 'DWARF' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✗ | | - -## Описание {#description} - -Формат `DWARF` разбирает отладочные символы DWARF из ELF-файла (исполняемого файла, библиотеки или объектного файла). -Он аналогичен `dwarfdump`, но гораздо быстрее (сотни МБ/с) и поддерживает SQL. -Он формирует одну строку для каждой Debug Information Entry (DIE) в секции `.debug_info` -и включает «нулевые» записи, которые кодировка DWARF использует для завершения списков дочерних элементов в дереве. - -:::info -`.debug_info` состоит из *unit*-ов, которые соответствуют единицам компиляции: - -- Каждый unit — это дерево *DIE* с `compile_unit` DIE в качестве корня. -- Каждый DIE имеет *tag* и список *attributes*. -- Каждый attribute имеет *name* и *value* (а также *form*, который определяет, как закодировано значение). - -Записи DIE представляют сущности из исходного кода, и их *tag* указывает, что это за сущность. Например, есть: - -- функции (tag = `subprogram`) -- классы/структуры/перечисления (`class_type`/`structure_type`/`enumeration_type`) -- переменные (`variable`) -- аргументы функций (`formal_parameter`). - -Древовидная структура отражает соответствующий исходный код. Например, `class_type` DIE может содержать `subprogram` DIE, представляющие методы класса. -::: - -Формат `DWARF` выводит следующие столбцы: - -- `offset` — позиция DIE в секции `.debug_info` -- `size` — количество байт в закодированном DIE (включая атрибуты) -- `tag` — тип DIE; традиционный префикс `DW_TAG_` опущен -- `unit_name` — имя единицы компиляции, содержащей данный DIE -- `unit_offset` — позиция единицы компиляции, содержащей этот DIE, в секции `.debug_info` -- `ancestor_tags` — массив тегов предков текущего DIE в дереве, в порядке от самого внутреннего к самому внешнему -- `ancestor_offsets` — смещения предков, параллельные `ancestor_tags` -- несколько распространённых атрибутов, продублированных из массива атрибутов для удобства: - - `name` - - `linkage_name` — преобразованное (mangled) полностью квалифицированное имя; обычно оно есть только у функций (и не у всех) - - `decl_file` — имя файла исходного кода, в котором эта сущность была объявлена - - `decl_line` — номер строки в исходном коде, в которой эта сущность была объявлена -- параллельные массивы, описывающие атрибуты: - - `attr_name` — имя атрибута; традиционный префикс `DW_AT_` опущен - - `attr_form` — то, как атрибут закодирован и интерпретируется; традиционный префикс `DW_FORM_` опущен - - `attr_int` — целочисленное значение атрибута; 0, если атрибут не имеет числового значения - - `attr_str` — строковое значение атрибута; пустая строка, если атрибут не имеет строкового значения - -## Пример использования {#example-usage} - -Формат `DWARF` можно использовать, чтобы найти единицы компиляции, которые содержат наибольшее число определений функций (включая инстанцирования шаблонов и функции из подключаемых заголовочных файлов): - -```sql title="Query" -SELECT - unit_name, - count() AS c -FROM file('programs/clickhouse', DWARF) -WHERE tag = 'subprogram' AND NOT has(attr_name, 'declaration') -GROUP BY unit_name -ORDER BY c DESC -LIMIT 3 -``` - -```text title="Response" -┌─unit_name──────────────────────────────────────────────────┬─────c─┐ -│ ./src/Core/Settings.cpp │ 28939 │ -│ ./src/AggregateFunctions/AggregateFunctionSumMap.cpp │ 23327 │ -│ ./src/AggregateFunctions/AggregateFunctionUniqCombined.cpp │ 22649 │ -└────────────────────────────────────────────────────────────┴───────┘ - -3 строки в наборе. Затрачено: 1.487 сек. Обработано 139.76 млн строк, 1.12 ГБ (93.97 млн строк/с., 752.77 МБ/с.) -Пиковое использование памяти: 271.92 МиБ. -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md deleted file mode 100644 index 9e6cd1a2c89..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Form' -input_format: true -keywords: ['Form'] -output_format: false -slug: /interfaces/formats/Form -title: 'Form' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✗ | | - -## Описание {#description} - -Формат `Form` можно использовать для чтения одной записи в формате application/x-www-form-urlencoded, -в котором данные представлены в виде `key1=value1&key2=value2`. - -## Пример использования {#example-usage} - -Предположим, что файл `data.tmp` находится в каталоге `user_files` и содержит некоторые данные в URL-кодировке: - -```text title="data.tmp" -t_page=116&c.e=ls7xfkpm&c.tti.m=raf&rt.start=navigation&rt.bmr=390%2C11%2C10 -``` - -```sql title="Query" -SELECT * FROM file(data.tmp, Form) FORMAT vertical; -``` - -```response title="Response" -Строка 1: -────── -t_page: 116 -c.e: ls7xfkpm -c.tti.m: raf -rt.start: navigation -rt.bmr: 390,11,10 -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md deleted file mode 100644 index 1faf9e37364..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Hash' -input_format: false -keywords: ['hash', 'format'] -output_format: true -slug: /interfaces/formats/Hash -title: 'Hash' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - -## Описание {#description} - -Формат вывода `Hash` вычисляет единое хеш-значение по всем столбцам и строкам результата. -Это полезно для вычисления «отпечатка» результата, например в ситуациях, когда узким местом является передача данных. - -## Пример использования {#example-usage} - -### Чтение данных {#reading-data} - -Рассмотрим таблицу `football` со следующими данными: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -Прочитать данные в формате `Hash`: - -```sql -SELECT * -FROM football -FORMAT Hash -``` - -Запрос обработает данные, но ничего не выведет. - -```response -df2ec2f0669b000edff6adee264e7d68 - -Получена 1 строка. Время выполнения: 0,154 сек. -``` - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md deleted file mode 100644 index 3ad85329763..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: 'Документация по формату HiveText' -keywords: ['HiveText'] -slug: /interfaces/formats/HiveText -title: 'HiveText' -doc_type: 'reference' ---- - -## Описание {#description} - -## Примеры использования {#example-usage} - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md deleted file mode 100644 index 8cba95a1c4f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSON' -input_format: true -keywords: ['JSON'] -output_format: true -slug: /interfaces/formats/JSON -title: 'JSON' -doc_type: 'reference' ---- - -| Входной формат | Выходной формат | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -Формат `JSON` считывает и выводит данные в формате JSON. - -Формат `JSON` возвращает следующее: - -| Parameter | Description | -|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `meta` | Имена и типы столбцов. | -| `data` | Таблицы с данными. | -| `rows` | Общее количество выводимых строк. | -| `rows_before_limit_at_least` | Нижняя оценка количества строк, которое было бы без LIMIT. Выводится только в том случае, если запрос содержит LIMIT. Эта оценка рассчитывается по блокам данных, обработанным в конвейере запроса до преобразования LIMIT, но затем может быть отброшена этим преобразованием. Если блоки даже не дошли до преобразования LIMIT в конвейере запроса, они не участвуют в оценке. | -| `statistics` | Статистика, такая как `elapsed`, `rows_read`, `bytes_read`. | -| `totals` | Итоговые значения (при использовании WITH TOTALS). | -| `extremes` | Минимальные и максимальные значения (когда extremes установлено в 1). | - -Тип `JSON` совместим с JavaScript. Для обеспечения совместимости некоторые символы дополнительно экранируются: - -- косая черта `/` экранируется как `\/`; -- альтернативные переносы строки `U+2028` и `U+2029`, которые вызывают проблемы в некоторых браузерах, экранируются как `\uXXXX`; -- управляющие символы ASCII экранируются: backspace, form feed, line feed, carriage return и горизонтальная табуляция заменяются на `\b`, `\f`, `\n`, `\r`, `\t`, а оставшиеся байты в диапазоне 00-1F — с помощью последовательностей `\uXXXX`; -- некорректные последовательности UTF-8 заменяются символом замены �, так что выходной текст состоит из корректных последовательностей UTF-8. - -Для совместимости с JavaScript целые числа Int64 и UInt64 по умолчанию заключаются в двойные кавычки. -Чтобы убрать кавычки, можно установить параметр конфигурации [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) в значение `0`. - -ClickHouse поддерживает [NULL](/sql-reference/syntax.md), который отображается как `null` в выводе JSON. Чтобы включить вывод значений `+nan`, `-nan`, `+inf`, `-inf`, установите параметр [output_format_json_quote_denormals](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) в значение `1`. - -## Пример использования {#example-usage} - -Пример: - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase WITH TOTALS ORDER BY c DESC LIMIT 5 FORMAT JSON -``` - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "привет", - "arr": [0,1] - }, - { - "num": 43, - "str": "привет", - "arr": [0,1,2] - }, - { - "num": 44, - "str": "привет", - "arr": [0,1,2,3] - } - ], - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.001137687, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - - -## Настройки формата {#format-settings} - -Для формата ввода JSON, если настройка [`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) установлена в значение `1`, -типы из метаданных во входных данных будут сравниваться с типами соответствующих столбцов таблицы. - -## См. также {#see-also} - -- формат [JSONEachRow](/interfaces/formats/JSONEachRow) -- настройка [output_format_json_array_of_rows](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md deleted file mode 100644 index 7ccbca236ea..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'Документация о формате JSONAsObject' -input_format: true -keywords: ['JSONAsObject'] -output_format: false -slug: /interfaces/formats/JSONAsObject -title: 'JSONAsObject' -doc_type: 'reference' ---- - -## Описание {#description} - -В этом формате один JSON-объект интерпретируется как одно значение типа [JSON](/sql-reference/data-types/newjson.md). Если на входе — несколько JSON-объектов (через запятую), они интерпретируются как отдельные строки. Если входные данные заключены в квадратные скобки, они интерпретируются как массив значений JSON. - -Этот формат может быть разобран только для таблицы с одним полем типа [JSON](/sql-reference/data-types/newjson.md). Остальные столбцы должны быть заданы как [`DEFAULT`](/sql-reference/statements/create/table.md/#default) или [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view). - -## Пример использования {#example-usage} - -### Простой пример {#basic-example} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_object FORMAT JSONEachRow; -``` - -```response title="Response" -{"json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -{"json":{}} -{"json":{"any json stucture":"1"}} -``` - - -### Массив объектов JSON {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field JSON) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsObject [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; -SELECT * FROM json_square_brackets FORMAT JSONEachRow; -``` - -```response title="Response" -{"field":{"id":"1","name":"name1"}} -{"field":{"id":"2","name":"name2"}} -``` - - -### Столбцы со значениями по умолчанию {#columns-with-default-values} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON, time DateTime MATERIALIZED now()) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"any json stucture":1} -SELECT time, json FROM json_as_object FORMAT JSONEachRow -``` - -```response title="Response" -{"time":"2024-09-16 12:18:10","json":{}} -{"time":"2024-09-16 12:18:13","json":{"any json stucture":"1"}} -{"time":"2024-09-16 12:18:08","json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md deleted file mode 100644 index 8f39e7095d4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONAsString' -input_format: true -keywords: ['JSONAsString'] -output_format: false -slug: /interfaces/formats/JSONAsString -title: 'JSONAsString' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|------|--------|-----------| -| ✔ | ✗ | | - -## Описание {#description} - -В этом формате один JSON-объект интерпретируется как одно значение. -Если на вход передано несколько JSON-объектов (разделённых запятыми), они интерпретируются как отдельные строки. -Если входные данные заключены в квадратные скобки, они интерпретируются как массив JSON-объектов. - -:::note -Этот формат может быть разобран только для таблицы с одним полем типа [String](/sql-reference/data-types/string.md). -Остальные столбцы должны быть заданы с помощью [`DEFAULT`](/sql-reference/statements/create/table.md/#default) или [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view), -либо опущены. -::: - -После того как вы сериализуете весь JSON-объект в значение типа String, вы можете использовать [JSON-функции](/sql-reference/functions/json-functions.md) для его обработки. - -## Пример использования {#example-usage} - -### Простой пример {#basic-example} - -```sql title="Query" -DROP TABLE IF EXISTS json_as_string; -CREATE TABLE json_as_string (json String) ENGINE = Memory; -INSERT INTO json_as_string (json) FORMAT JSONAsString {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_string; -``` - -```response title="Response" -┌─json──────────────────────────────┐ -│ {"foo":{"bar":{"x":"y"},"baz":1}} │ -│ {} │ -│ {"any json stucture":1} │ -└───────────────────────────────────┘ -``` - - -### Массив объектов JSON {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field String) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsString [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; - -SELECT * FROM json_square_brackets; -``` - -```response title="Response" -┌─field──────────────────────┐ -│ {"id": 1, "name": "name1"} │ -│ {"id": 2, "name": "name2"} │ -└────────────────────────────┘ -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md deleted file mode 100644 index d6bde8bef51..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONColumns' -input_format: true -keywords: ['JSONColumns'] -output_format: true -slug: /interfaces/formats/JSONColumns -title: 'JSONColumns' -doc_type: 'reference' ---- - -| Входной формат | Выходной формат | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -:::tip -Вывод форматов JSONColumns* содержит имя поля ClickHouse, а затем содержимое каждой строки таблицы для этого поля; -визуально данные повернуты на 90 градусов влево. -::: - -В этом формате все данные представлены в виде одного объекта JSON. - -:::note -Формат `JSONColumns` буферизует все данные в памяти и затем выводит их одним блоком, поэтому это может приводить к высокому потреблению памяти. -::: - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными под названием `football.json`: - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - -Введите данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONColumns; -``` - - -### Чтение данных {#reading-data} - -Считывайте данные в формате `JSONColumns`: - -```sql -SELECT * -FROM football -FORMAT JSONColumns -``` - -Вывод будет в формате JSON: - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - - -## Настройки формата {#format-settings} - -При импорте столбцы с неизвестными именами будут пропущены, если настройка [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в значение `1`. -Столбцы, которые отсутствуют в блоке, будут заполнены значениями по умолчанию (для этого можно использовать настройку [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md deleted file mode 100644 index 09c5411938e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONColumnsWithMetadata' -input_format: true -keywords: ['JSONColumnsWithMetadata'] -output_format: true -slug: /interfaces/formats/JSONColumnsWithMetadata -title: 'JSONColumnsWithMetadata' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата [`JSONColumns`](./JSONColumns.md) тем, что также содержит некоторые метаданные и статистику (аналогично формату [`JSON`](./JSON.md)). - -:::note -Формат `JSONColumnsWithMetadata` буферизует все данные в памяти и затем выводит их одним блоком, поэтому может приводить к высокому потреблению памяти. -::: - -## Пример использования {#example-usage} - -Пример: - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - { - "num": [42, 43, 44], - "str": ["hello", "hello", "hello"], - "arr": [[0,1], [0,1,2], [0,1,2,3]] - }, - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.000272376, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - -Для формата ввода `JSONColumnsWithMetadata`, если параметр [`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) имеет значение `1`, -типы, указанные в метаданных во входных данных, будут сравниваться с типами соответствующих столбцов таблицы. - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md deleted file mode 100644 index 90f482f4327..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md +++ /dev/null @@ -1,160 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompact' -input_format: true -keywords: ['JSONCompact'] -output_format: true -slug: /interfaces/formats/JSONCompact -title: 'JSONCompact' -doc_type: 'reference' ---- - -| Входной | Выходной | Псевдонимы | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от [JSON](./JSON.md) только тем, что строки данных выводятся в виде массивов, а не объектов. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными в файле с именем `football.json`: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ] -} -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompact; -``` - - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONCompact`: - -```sql -SELECT * -FROM football -FORMAT JSONCompact -``` - -Вывод будет в формате JSON: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.223690876, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md deleted file mode 100644 index 0b3107181e7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactColumns' -input_format: true -keywords: ['JSONCompactColumns'] -output_format: true -slug: /interfaces/formats/JSONCompactColumns -title: 'JSONCompactColumns' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -В этом формате все данные представлены в виде одного массива JSON. - -:::note -Формат вывода `JSONCompactColumns` буферизует все данные в памяти и выводит их одним блоком, что может привести к высокому потреблению памяти. -::: - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -JSON-файл со следующими данными, сохранённый как `football.json`: - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactColumns; -``` - - -### Чтение данных {#reading-data} - -Прочитайте данные, используя формат `JSONCompactColumns`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactColumns -``` - -Результат будет в формате JSON: - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -Столбцы, которые отсутствуют в блоке, будут заполнены значениями по умолчанию (здесь можно использовать настройку [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)) - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md deleted file mode 100644 index cd011244a94..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactEachRow' -input_format: true -keywords: ['JSONCompactEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRow -title: 'JSONCompactEachRow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от [`JSONEachRow`](./JSONEachRow.md) только тем, что строки данных выводятся в виде массивов, а не объектов. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON‑файл со следующими данными под названием `football.json`: - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRow; -``` - - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONCompactEachRow`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRow -``` - -Вывод будет в формате JSON: - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md deleted file mode 100644 index e21d272af56..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactEachRowWithNames' -input_format: true -keywords: ['JSONCompactEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNames -title: 'JSONCompactEachRowWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата [`JSONCompactEachRow`](./JSONCompactEachRow.md) тем, что также выводит строку заголовка с именами столбцов, аналогично формату [`TabSeparatedWithNames`](../TabSeparated/TabSeparatedWithNames.md). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными с именем `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNames; -``` - - -### Чтение данных {#reading-data} - -Считывайте данные в формате `JSONCompactEachRowWithNames`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNames -``` - -Результат будет в формате JSON: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) равна 1, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропускаться, если настройка [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) равна 1. -В противном случае первая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md deleted file mode 100644 index acc1add1b2b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactEachRowWithNamesAndTypes' -input_format: true -keywords: ['JSONCompactEachRowWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNamesAndTypes -title: 'JSONCompactEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата [`JSONCompactEachRow`](./JSONCompactEachRow.md) тем, что также выводит две заголовочные строки с именами и типами столбцов, аналогично формату [TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными под именем `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNamesAndTypes; -``` - - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONCompactEachRowWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNamesAndTypes -``` - -Результат будет в формате JSON: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) равна `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если настройка [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) равна `1`. -В противном случае первая строка будет пропущена. -Если настройка [`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header) равна `1`, -типы из входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md deleted file mode 100644 index 4a815e7c3b9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -alias: [] -description: 'Документация о формате JSONCompactEachRowWithProgress' -input_format: false -keywords: ['JSONCompactEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithProgress -title: 'JSONCompactEachRowWithProgress' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - -## Описание {#description} - -Этот формат объединяет компактный построчный вывод JSONCompactEachRow с потоковой -информацией о ходе выполнения. -Он выводит данные в виде отдельных JSON-объектов для метаданных, отдельных строк, обновлений прогресса, -итогов и исключений. Значения представлены в их исходных типах. - -Основные особенности: - -- Сначала выводит метаданные с названиями и типами столбцов -- Каждая строка — отдельный JSON-объект с ключом "row", содержащим массив значений -- Включает обновления прогресса во время выполнения запроса (в виде объектов `{"progress":...}`) -- Поддерживает `totals` и `extremes` -- Значения сохраняют свои исходные типы (числа — как числа, строки — как строки) - -## Пример использования {#example-usage} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":[[-8], 46848.5225, ["2064-06-11 14:00:36.578","b06f4fa1-22ff-f84f-a1b7-a5807d983ae6"]]} -{"row":[[-76], -85331.598, ["2038-06-16 04:10:27.271","2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c"]]} -{"row":[[-32], -31470.8994, ["2027-07-18 16:58:34.654","1cdbae4c-ceb2-1337-b954-b175f5efbef8"]]} -{"row":[[-116], 32104.097, ["1979-04-27 21:51:53.321","66903704-3c83-8f8a-648a-da4ac1ffa9fc"]]} -{"row":[[], 2427.6614, ["1980-04-24 11:30:35.487","fee19be8-0f46-149b-ed98-43e7455ce2b2"]]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"335771"}} -{"rows_before_limit_at_least":5} -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md deleted file mode 100644 index 1ebe4a2a3d2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactStrings' -input_format: false -keywords: ['JSONCompactStrings'] -output_format: true -slug: /interfaces/formats/JSONCompactStrings -title: 'JSONCompactStrings' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - -## Описание {#description} - -Формат `JSONCompactStrings` отличается от [JSONStrings](./JSONStrings.md) лишь тем, что строки данных выводятся как массивы, а не как объекты. - -## Пример использования {#example-usage} - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONCompactStrings`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStrings -``` - -Результат будет в формате JSON: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"], - ["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"], - ["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"], - ["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"], - ["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"], - ["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"], - ["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"], - ["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"], - ["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"], - ["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"], - ["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"], - ["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"], - ["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"], - ["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"], - ["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"], - ["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"], - ["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.112012501, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## Настройки форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md deleted file mode 100644 index 2cbd83b83b4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactStringsEachRow' -input_format: true -keywords: ['JSONCompactStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRow -title: 'JSONCompactStringsEachRow' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от [`JSONCompactEachRow`](./JSONCompactEachRow.md) только тем, что поля данных выводятся в виде строк, а не типизированных значений JSON. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON‑файл со следующими данными под названием `football.json`: - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRow; -``` - - -### Чтение данных {#reading-data} - -Считывайте данные в формате `JSONCompactStringsEachRow`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRow -``` - -Результат будет в формате JSON: - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md deleted file mode 100644 index f9b1b3bc632..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -alias: [] -description: 'Документация о формате JSONCompactStringsEachRowWithNames' -input_format: true -keywords: ['JSONCompactStringsEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithNames -title: 'JSONCompactStringsEachRowWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Алиас | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата [`JSONCompactEachRow`](./JSONCompactEachRow.md) тем, что также выводит заголовочную строку с названиями столбцов, аналогично формату [TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными с именем `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNames; -``` - - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONCompactStringsEachRowWithNames`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNames -``` - -Вывод будет в формате JSON: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) установлена в значение `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам. Столбцы с неизвестными именами будут пропущены, если настройка [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в значение `1`. -В противном случае первая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md deleted file mode 100644 index ff5b2e532b0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'Документация по формату JSONCompactStringsEachRowWithNamesAndTypes' -keywords: ['JSONCompactStringsEachRowWithNamesAndTypes'] -slug: /interfaces/formats/JSONCompactStringsEachRowWithNamesAndTypes -title: 'JSONCompactStringsEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата `JSONCompactEachRow` тем, что дополнительно выводит две строки заголовка с именами и типами столбцов, аналогично [TabSeparatedWithNamesAndTypes](/interfaces/formats/TabSeparatedRawWithNamesAndTypes). - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используя JSON‑файл со следующими данными с именем `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNamesAndTypes; -``` - - -### Чтение данных {#reading-data} - -Считывайте данные в формате `JSONCompactStringsEachRowWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNamesAndTypes -``` - -Выходные данные будут в формате JSON: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## Настройки формата {#format-settings} - -:::note -Если настройка [input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) равна 1, -столбцы во входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если настройка [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) также равна 1. -В противном случае первая строка будет пропущена. -::: - -:::note -Если настройка [input_format_with_types_use_header](/operations/settings/settings-formats.md/#input_format_with_types_use_header) равна 1, -типы во входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md deleted file mode 100644 index 5c86576ae9f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONCompactStringsEachRowWithProgress' -input_format: true -keywords: ['JSONCompactStringsEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithProgress -title: 'JSONCompactStringsEachRowWithProgress' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✗ | ✔ | | - -## Описание {#description} - -Аналогичен [`JSONCompactEachRowWithProgress`](/interfaces/formats/JSONCompactEachRowWithProgress), но все значения преобразуются в строки. -Это полезно, когда требуется единообразное строковое представление всех типов данных. - -Ключевые особенности: - -- Та же структура, что и у `JSONCompactEachRowWithProgress` -- Все значения представлены в виде строк (числа, массивы и т. д. — это строковые значения в кавычках) -- Включает обновления прогресса, итоговые значения и обработку исключений -- Полезен для клиентов, которым предпочтителен или необходим строковый формат данных - -## Пример использования {#example-usage} - -### Добавление данных {#inserting-data} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactStringsEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":["[-8]", "46848.5225", "('2064-06-11 14:00:36.578','b06f4fa1-22ff-f84f-a1b7-a5807d983ae6')"]} -{"row":["[-76]", "-85331.598", "('2038-06-16 04:10:27.271','2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c')"]} -{"row":["[-32]", "-31470.8994", "('2027-07-18 16:58:34.654','1cdbae4c-ceb2-1337-b954-b175f5efbef8')"]} -{"row":["[-116]", "32104.097", "('1979-04-27 21:51:53.321','66903704-3c83-8f8a-648a-da4ac1ffa9fc')"]} -{"row":["[]", "2427.6614", "('1980-04-24 11:30:35.487','fee19be8-0f46-149b-ed98-43e7455ce2b2')"]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"191151"}} -{"rows_before_limit_at_least":5} -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md deleted file mode 100644 index 0c5b815fb9f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: ['JSONLines', 'NDJSON'] -description: 'Документация по формату JSONEachRow' -keywords: ['JSONEachRow'] -slug: /interfaces/formats/JSONEachRow -title: 'JSONEachRow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|------|-------|----------------------| -| ✔ | ✔ | `JSONLines`, `NDJSON` | - -## Описание {#description} - -В этом формате ClickHouse выводит каждую строку в виде отдельного JSON-объекта, отделённого символом новой строки. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте JSON-файл со следующими данными с именем `football.json`: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONEachRow; -``` - - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `JSONEachRow`: - -```sql -SELECT * -FROM football -FORMAT JSONEachRow -``` - -Результат будет в формате JSON: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -Импорт столбцов с неизвестными именами будет пропускаться, если параметр [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлен в 1. - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md deleted file mode 100644 index 61e46f04654..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONEachRowWithProgress' -input_format: false -keywords: ['JSONEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONEachRowWithProgress -title: 'JSONEachRowWithProgress' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - -## Описание {#description} - -Отличается от [`JSONEachRow`](./JSONEachRow.md)/[`JSONStringsEachRow`](./JSONStringsEachRow.md) тем, что ClickHouse также возвращает информацию о прогрессе в формате JSON. - -## Пример использования {#example-usage} - -```json -{"row":{"num":42,"str":"hello","arr":[0,1]}} -{"row":{"num":43,"str":"hello","arr":[0,1,2]}} -{"row":{"num":44,"str":"hello","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md deleted file mode 100644 index ceecb72e5e5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -alias: ['JSONEachRow', 'JSONLines', 'NDJSON', 'JSONL'] -description: 'Документация по формату JSONLines' -keywords: ['JSONLines'] -slug: /interfaces/formats/JSONLines -title: 'JSONLines' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-------------------------------------------| -| ✔ | ✔ | `JSONEachRow`, `JSONLines`, `NDJSON`, `JSONL` | - -## Описание {#description} - -В этом формате ClickHouse выводит каждую строку в виде отдельного JSON-объекта, по одному объекту на строку. - -Этот формат также известен как `JSONEachRow`, `NDJSON` (Newline Delimited JSON) или `JSONL` (`JSONLines`). Все эти названия являются синонимами одного и того же формата и могут использоваться взаимозаменяемо. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем JSON-файл со следующими данными, сохранённый под именем `football.json`: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONLines; -``` - - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `JSONLines`: - -```sql -SELECT * -FROM football -FORMAT JSONLines -``` - -Результат будет в формате JSON: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -Столбцы данных с неизвестными именами будут пропущены при импорте, если настройка [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в 1. - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md deleted file mode 100644 index 5de1ea8c011..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md +++ /dev/null @@ -1,245 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONObjectEachRow' -input_format: true -keywords: ['JSONObjectEachRow'] -output_format: true -slug: /interfaces/formats/JSONObjectEachRow -title: 'JSONObjectEachRow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -В этом формате все данные представлены одним JSON-объектом, где каждая строка является отдельным полем этого объекта, аналогично формату [`JSONEachRow`](./JSONEachRow.md). - - - -## Пример использования {#example-usage} - -### Базовый пример {#basic-example} - -Пусть у нас есть следующий JSON: - -```json -{ - "row_1": {"num": 42, "str": "hello", "arr": [0,1]}, - "row_2": {"num": 43, "str": "hello", "arr": [0,1,2]}, - "row_3": {"num": 44, "str": "hello", "arr": [0,1,2,3]} -} -``` - -Чтобы использовать имя объекта в качестве значения в столбце, вы можете воспользоваться специальной настройкой [`format_json_object_each_row_column_for_object_name`](/operations/settings/settings-formats.md/#format_json_object_each_row_column_for_object_name). -В качестве значения этой настройки задаётся имя столбца, который используется как JSON-ключ для строки в результирующем объекте. - -#### Вывод {#output} - -Допустим, у нас есть таблица `test` с двумя столбцами: - -```text -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -Выведем его в формате `JSONObjectEachRow` и воспользуемся настройкой `format_json_object_each_row_column_for_object_name`: - -```sql title="Query" -SELECT * FROM test SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```json title="Response" -{ - "first_obj": {"number": 1}, - "second_obj": {"number": 2}, - "third_obj": {"number": 3} -} -``` - -#### Ввод {#input} - -Допустим, мы сохранили результат из предыдущего примера в файл с именем `data.json`: - -```sql title="Query" -SELECT * FROM file('data.json', JSONObjectEachRow, 'object_name String, number UInt64') SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -Также работает для автоматического определения схемы: - -```sql title="Query" -DESCRIBE file('data.json', JSONObjectEachRow) SETTING format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─name────────┬─type────────────┐ -│ object_name │ String │ -│ number │ Nullable(Int64) │ -└─────────────┴─────────────────┘ -``` - -### Вставка данных {#json-inserting-data} - -```sql title="Query" -INSERT INTO UserActivity FORMAT JSONEachRow {"PageViews":5, "UserID":"4324182021466249494", "Duration":146,"Sign":-1} {"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -ClickHouse позволяет: - -* Любой порядок пар ключ–значение в объекте. -* Пропуск некоторых значений. - -ClickHouse игнорирует пробелы между элементами и запятые после объектов. Вы можете передавать все объекты в одной строке. Не требуется разделять их переводами строки. - -#### Обработка пропущенных значений {#omitted-values-processing} - -ClickHouse подставляет пропущенные значения значениями по умолчанию для соответствующих [типов данных](/sql-reference/data-types/index.md). - -Если указано `DEFAULT expr`, ClickHouse использует разные правила подстановки в зависимости от настройки [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields). - -Рассмотрим следующую таблицу: - -```sql title="Query" -CREATE TABLE IF NOT EXISTS example_table -( - x UInt32, - a DEFAULT x * 2 -) ENGINE = Memory; -``` - -* Если `input_format_defaults_for_omitted_fields = 0`, то значение по умолчанию для `x` и `a` равно `0` (как и значение по умолчанию для типа данных `UInt32`). -* Если `input_format_defaults_for_omitted_fields = 1`, то значение по умолчанию для `x` равно `0`, но значение по умолчанию для `a` равно `x * 2`. - -:::note -При вставке данных при `input_format_defaults_for_omitted_fields = 1` ClickHouse потребляет больше вычислительных ресурсов по сравнению со вставкой при `input_format_defaults_for_omitted_fields = 0`. -::: - -### Выборка данных {#json-selecting-data} - -Рассмотрим в качестве примера таблицу `UserActivity`: - - -```response -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -Запрос `SELECT * FROM UserActivity FORMAT JSONEachRow` возвращает: - -```response -{"UserID":"4324182021466249494","PageViews":5,"Duration":146,"Sign":-1} -{"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -В отличие от формата [JSON](/interfaces/formats/JSON), некорректные последовательности UTF-8 не подставляются. Значения экранируются так же, как для `JSON`. - -:::info -Любой набор байт может выводиться в строковых значениях. Используйте формат [`JSONEachRow`](./JSONEachRow.md), если вы уверены, что данные в таблице могут быть отформатированы как JSON без потери какой-либо информации. -::: - -### Использование вложенных структур {#jsoneachrow-nested} - -Если у вас есть таблица со столбцами типа данных [`Nested`](/sql-reference/data-types/nested-data-structures/index.md), вы можете вставлять JSON-данные с той же структурой. Включите эту возможность с помощью настройки [input_format_import_nested_json](/operations/settings/settings-formats.md/#input_format_import_nested_json). - -Например, рассмотрим следующую таблицу: - -```sql -CREATE TABLE json_each_row_nested (n Nested (s String, i Int32) ) ENGINE = Memory -``` - -Как видно из описания типа данных `Nested`, ClickHouse обрабатывает каждый компонент вложенной структуры как отдельный столбец (`n.s` и `n.i` для нашей таблицы). Данные можно вставлять следующим образом: - -```sql -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n.s": ["abc", "def"], "n.i": [1, 23]} -``` - -Для вставки данных в виде иерархического объекта JSON установите [`input_format_import_nested_json=1`](/operations/settings/settings-formats.md/#input_format_import_nested_json). - -```json -{ - "n": { - "s": ["abc", "def"], - "i": [1, 23] - } -} -``` - -Без этой настройки ClickHouse выдаёт исключение. - -```sql title="Query" -SELECT name, value FROM system.settings WHERE name = 'input_format_import_nested_json' -``` - -```response title="Response" -┌─name────────────────────────────┬─value─┐ -│ input_format_import_nested_json │ 0 │ -└─────────────────────────────────┴───────┘ -``` - -```sql title="Query" -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -``` - -```response title="Response" -Код: 117. DB::Exception: Обнаружено неизвестное поле при парсинге формата JSONEachRow: n: (в строке 1) -``` - -```sql title="Query" -SET input_format_import_nested_json=1 -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -SELECT * FROM json_each_row_nested -``` - -```response title="Response" -┌─n.s───────────┬─n.i────┐ -│ ['abc','def'] │ [1,23] │ -└───────────────┴────────┘ -``` - - -## Параметры форматирования {#format-settings} - - - -| Настройка | Описание | По умолчанию | Примечания | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | сопоставлять вложенные данные JSON вложенным таблицам (работает для формата JSONEachRow). | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | позволяет интерпретировать логические значения как числа во входных форматах JSON. | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | позволяет интерпретировать логические значения как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | позволяет интерпретировать числа как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | позволяет интерпретировать массивы JSON как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | позволяет интерпретировать JSON‑объекты как строки во входных форматах JSON. | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | разбирать столбцы типа NamedTuple как JSON-объекты. | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | пытаться распознавать числа в строковых полях при автоматическом определении схемы. | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | пытаться выводить тип NamedTuple из JSON-объектов при определении схемы. | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | используйте тип String для ключей, которые содержат только значения Null или пустые объекты/массивы при выводе схемы во входных форматах JSON. | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | вставлять значения по умолчанию для отсутствующих полей объекта JSON при разборе именованного кортежа. | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | игнорировать неизвестные ключи в JSON-объекте для именованных кортежей. | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | разрешить переменное количество столбцов в формате JSONCompact/JSONCompactEachRow, игнорировать лишние столбцы и использовать значения по умолчанию для отсутствующих столбцов. | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | выбрасывать исключение, если JSON-строка содержит некорректную escape-последовательность. Если параметр отключен, некорректные escape-последовательности останутся в данных как есть. | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | обрабатывать пустые поля во входном JSON-документе как значения по умолчанию. | `false`. | Для сложных выражений по умолчанию необходимо также включить [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields). | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | управляет заключением 64-битных целых чисел в кавычки в выходном формате JSON. | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | определяет, заключать ли 64-битные числа с плавающей запятой в кавычки в формате вывода JSON. | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | разрешает вывод значений '+nan', '-nan', '+inf', '-inf' в формате JSON. | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | управляет тем, будут ли десятичные числа заключаться в кавычки в выводе в формате JSON. | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | управляет экранированием косых черт в строковых значениях при выводе в формате JSON. | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | сериализует столбцы именованных кортежей в виде JSON-объектов. | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | выводит JSON-массив всех строк в формате JSONEachRow(Compact). | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | включает проверку корректности последовательностей UTF-8 в форматах вывода JSON (учтите, что это не влияет на форматы JSON/JSONCompact/JSONColumnsWithMetadata — они всегда проверяют UTF-8). | `false` | | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md deleted file mode 100644 index 70989a5888e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md +++ /dev/null @@ -1,398 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONStrings' -input_format: true -keywords: ['JSONStrings'] -output_format: true -slug: /interfaces/formats/JSONStrings -title: 'JSONStrings' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Отличается от формата [JSON](./JSON.md) только тем, что поля данных выводятся в виде строк, а не типизированных значений JSON. - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем JSON-файл со следующими данными, сохранённый как `football.json`: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ] -} -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStrings; -``` - - -### Чтение данных {#reading-data} - -Считайте данные в формате `JSONStrings`: - -```sql -SELECT * -FROM football -FORMAT JSONStrings -``` - -Результат будет в формате JSON: - - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.173464376, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md deleted file mode 100644 index 1ca778f6f3a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -alias: [] -description: 'Документация по формату JSONStringsEachRow' -input_format: false -keywords: ['JSONStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONStringsEachRow -title: 'JSONStringsEachRow' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Отличается от [`JSONEachRow`](./JSONEachRow.md) только тем, что поля данных выводятся как строки, а не как типизированные JSON-значения. - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем JSON-файл со следующими данными с именем `football.json`: - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStringsEachRow; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `JSONStringsEachRow`: - -```sql -SELECT * -FROM football -FORMAT JSONStringsEachRow -``` - -Вывод будет в формате JSON: - - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md deleted file mode 100644 index 577cad940a9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'Документация по формату JSONStringsEachRowWithProgress' -keywords: ['JSONStringsEachRowWithProgress'] -slug: /interfaces/formats/JSONStringsEachRowWithProgress -title: 'JSONStringsEachRowWithProgress' -doc_type: 'reference' ---- - - - -## Описание {#description} - -Отличается от `JSONEachRow`/`JSONStringsEachRow` тем, что ClickHouse также возвращает сведения о прогрессе в формате JSON. - - - -## Пример использования {#example-usage} - -```json -{"row":{"num":42,"str":"привет","arr":[0,1]}} -{"row":{"num":43,"str":"привет","arr":[0,1,2]}} -{"row":{"num":44,"str":"привет","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md deleted file mode 100644 index 5fda8d0b4ba..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -alias: ['PrettyJSONLines', 'PrettyNDJSON'] -description: 'Документация по формату PrettyJSONLines' -input_format: false -keywords: ['PrettyJSONEachRow', 'PrettyJSONLines', 'PrettyNDJSON'] -output_format: true -slug: /interfaces/formats/PrettyJSONEachRow -title: 'PrettyJSONEachRow' -doc_type: 'guide' ---- - -| Вход | Выход | Псевдонимы | -|------|-------|-----------------------------------| -| ✗ | ✔ | `PrettyJSONLines`, `PrettyNDJSON` | - - - -## Описание {#description} - -Отличается от [JSONEachRow](./JSONEachRow.md) только тем, что JSON форматируется в читабельном виде с разделением по строкам и отступом в 4 пробела. - - - -## Пример использования {#example-usage} -### Вставка данных {#inserting-data} - -Используем JSON-файл `football.json` со следующими данными: - - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT PrettyJSONEachRow; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные, используя формат `PrettyJSONEachRow`: - -```sql -SELECT * -FROM football -FORMAT PrettyJSONEachRow -``` - -Результат будет в формате JSON: - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - - - - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md deleted file mode 100644 index 8ec33e1c071..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'Список настроек формата для JSON' -keywords: ['Настройки формата', 'JSON'] -slug: /interfaces/formats/JSON/format-settings -title: 'Настройки формата для JSON' -doc_type: 'reference' ---- - -На этой странице представлены настройки формата, общие для всех JSON-форматов. - -{/* TODO: автоматически сгенерировать таблицу ниже */ } - - -| Параметр | Описание | По умолчанию | Примечание | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | Отображает вложенные данные JSON во вложенные таблицы (работает для формата JSONEachRow). | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | Разрешает интерпретировать булевы значения как числа во входных форматах JSON. | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | Позволяет интерпретировать логические значения как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | Разрешает интерпретировать числовые значения как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | Позволяет разбирать JSON‑массивы как строки во входных форматах JSON. | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | Позволяет разбирать JSON-объекты как строки во входных форматах JSON. | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | Интерпретировать столбцы типа NamedTuple как объекты JSON. | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | При выводе схемы пытаться распознавать числовые значения в строковых полях. | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | Пытаться выводить тип NamedTuple из объектов JSON при выводе схемы. | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | Используйте тип String для ключей, которые при выводе схемы в форматах ввода JSON содержат только значения Null или пустые объекты/массивы. | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | Вставлять значения по умолчанию для отсутствующих полей JSON-объекта при разборе именованного кортежа. | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | Игнорировать неизвестные ключи JSON-объекта для именованных кортежей. | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | Разрешить переменное количество столбцов в формате JSONCompact/JSONCompactEachRow, игнорировать лишние столбцы и использовать значения по умолчанию для отсутствующих столбцов. | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | Выбрасывать исключение, если строка JSON содержит некорректную escape-последовательность. Если опция отключена, некорректные escape-последовательности останутся в данных без изменений. | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | Считать пустые поля во входных данных JSON значениями по умолчанию. | `false` | Для сложных выражений по умолчанию необходимо также включить настройку [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields). | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | Определяет, заключать ли 64-битные целые числа в кавычки в формате вывода JSON. | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | Управляет заключением 64-битных чисел с плавающей запятой в кавычки в выходном формате JSON. | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | Включает вывод '+nan', '-nan', '+inf', '-inf' в формате JSON. | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | Управляет заключением значений типа Decimal в кавычки при выводе в формате JSON. | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | Определяет, нужно ли экранировать прямые слэши в строковых значениях в JSON-формате. | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | Сериализовать столбцы именованных кортежей как объекты JSON. | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | Выведите JSON-массив со всеми строками в формате JSONEachRow(Compact). | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | Включает проверку корректности последовательностей UTF-8 в выходных форматах JSON | `false` | Обратите внимание, что это не влияет на форматы JSON/JSONCompact/JSONColumnsWithMetadata — в них всегда выполняется проверка UTF-8. | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md deleted file mode 100644 index 950e402183c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -alias: [] -description: 'Документация по формату LineAsString' -input_format: true -keywords: ['LineAsString'] -output_format: true -slug: /interfaces/formats/LineAsString -title: 'LineAsString' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -Формат `LineAsString` интерпретирует каждую строку входных данных как одно строковое значение. -Этот формат может быть использован только для таблицы с одним полем типа [String](/sql-reference/data-types/string.md). -Остальные столбцы должны иметь типы [`DEFAULT`](/sql-reference/statements/create/table.md/#default), [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view) или быть опущены. - - - -## Пример использования {#example-usage} - -```sql title="Query" -DROP TABLE IF EXISTS line_as_string; -CREATE TABLE line_as_string (field String) ENGINE = Memory; -INSERT INTO line_as_string FORMAT LineAsString "I love apple", "I love banana", "I love orange"; -SELECT * FROM line_as_string; -``` - -```text title="Response" -┌─field─────────────────────────────────────────────┐ -│ "I love apple", "I love banana", "I love orange"; │ -└───────────────────────────────────────────────────┘ -``` - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md deleted file mode 100644 index 25691de4423..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -alias: [] -description: 'Документация по формату LineAsStringWithNames' -input_format: true -keywords: ['LineAsStringWithNames'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNames -title: 'LineAsStringWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|------|-------|-----------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Формат `LineAsStringWithNames` похож на формат [`LineAsString`](./LineAsString.md), но выводит строку заголовков с именами столбцов. - - - -## Пример использования {#example-usage} - -```sql title="Query" -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNames; -``` - -```response title="Response" -name value -John 30 -Jane 25 -Peter 35 -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md deleted file mode 100644 index c33967054a4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -alias: [] -description: 'Документация по формату LineAsStringWithNamesAndTypes' -input_format: false -keywords: ['LineAsStringWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNamesAndTypes -title: 'LineAsStringWithNamesAndTypes' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Формат `LineAsStringWithNames` похож на формат [`LineAsString`](./LineAsString.md), -но выводит две строки заголовков: одну с именами столбцов, другую — с их типами. - - - -## Пример использования {#example-usage} - -```sql -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNamesAndTypes; -``` - -```response title="Response" -name value -String Int32 -John 30 -Jane 25 -Peter 35 -``` - - -## Параметры формата {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md deleted file mode 100644 index 6530214650d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: ['MD'] -description: 'Документация по формату Markdown' -keywords: ['Markdown'] -slug: /interfaces/formats/Markdown -title: 'Markdown' -doc_type: 'reference' ---- - -| Вход | Выход | Сокращение | -|-------|--------|-------| -| ✗ | ✔ | `MD` | - - - -## Описание {#description} - -Вы можете экспортировать результаты в формате [Markdown](https://en.wikipedia.org/wiki/Markdown), чтобы получить данные, готовые для вставки в ваши файлы `.md`: - -Таблица в формате Markdown будет сгенерирована автоматически и может использоваться на платформах с поддержкой Markdown, таких как GitHub. Этот формат используется только для представления результатов. - - - -## Пример использования {#example-usage} - -```sql -SELECT - number, - number * 2 -FROM numbers(5) -FORMAT Markdown -``` - -```results -| number | multiply(number, 2) | -|-:|-:| -| 0 | 0 | -| 1 | 2 | -| 2 | 4 | -| 3 | 6 | -| 4 | 8 | -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md deleted file mode 100644 index d0596278fe0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'Документация по формату MsgPack' -input_format: true -keywords: ['MsgPack'] -output_format: true -slug: /interfaces/formats/MsgPack -title: 'MsgPack' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -ClickHouse поддерживает чтение и запись файлов данных в формате [MessagePack](https://msgpack.org/). - - - -## Соответствие типов данных {#data-types-matching} - -| Тип данных MessagePack (`INSERT`) | Тип данных ClickHouse | Тип данных MessagePack (`SELECT`) | -|--------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|-----------------------------------| -| `uint N`, `positive fixint` | [`UIntN`](/sql-reference/data-types/int-uint.md) | `uint N` | -| `int N`, `negative fixint` | [`IntN`](/sql-reference/data-types/int-uint.md) | `int N` | -| `bool` | [`UInt8`](/sql-reference/data-types/int-uint.md) | `uint 8` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`String`](/sql-reference/data-types/string.md) | `bin 8`, `bin 16`, `bin 32` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`FixedString`](/sql-reference/data-types/fixedstring.md) | `bin 8`, `bin 16`, `bin 32` | -| `float 32` | [`Float32`](/sql-reference/data-types/float.md) | `float 32` | -| `float 64` | [`Float64`](/sql-reference/data-types/float.md) | `float 64` | -| `uint 16` | [`Date`](/sql-reference/data-types/date.md) | `uint 16` | -| `int 32` | [`Date32`](/sql-reference/data-types/date32.md) | `int 32` | -| `uint 32` | [`DateTime`](/sql-reference/data-types/datetime.md) | `uint 32` | -| `uint 64` | [`DateTime64`](/sql-reference/data-types/datetime.md) | `uint 64` | -| `fixarray`, `array 16`, `array 32` | [`Array`](/sql-reference/data-types/array.md)/[`Tuple`](/sql-reference/data-types/tuple.md) | `fixarray`, `array 16`, `array 32` | -| `fixmap`, `map 16`, `map 32` | [`Map`](/sql-reference/data-types/map.md) | `fixmap`, `map 16`, `map 32` | -| `uint 32` | [`IPv4`](/sql-reference/data-types/ipv4.md) | `uint 32` | -| `bin 8` | [`String`](/sql-reference/data-types/string.md) | `bin 8` | -| `int 8` | [`Enum8`](/sql-reference/data-types/enum.md) | `int 8` | -| `bin 8` | [`(U)Int128`/`(U)Int256`](/sql-reference/data-types/int-uint.md) | `bin 8` | -| `int 32` | [`Decimal32`](/sql-reference/data-types/decimal.md) | `int 32` | -| `int 64` | [`Decimal64`](/sql-reference/data-types/decimal.md) | `int 64` | -| `bin 8` | [`Decimal128`/`Decimal256`](/sql-reference/data-types/decimal.md) | `bin 8 ` | - - - -## Пример использования {#example-usage} - -Запись в файл «.msgpk»: - -```sql -$ clickhouse-client --query="CREATE TABLE msgpack (array Array(UInt8)) ENGINE = Memory;" -$ clickhouse-client --query="INSERT INTO msgpack VALUES ([0, 1, 2, 3, 42, 253, 254, 255]), ([255, 254, 253, 42, 3, 2, 1, 0])"; -$ clickhouse-client --query="SELECT * FROM msgpack FORMAT MsgPack" > tmp_msgpack.msgpk; -``` - - -## Настройки формата {#format-settings} - -| Настройка | Описание | По умолчанию | -|-----------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------|--------------| -| [`input_format_msgpack_number_of_columns`](/operations/settings/settings-formats.md/#input_format_msgpack_number_of_columns) | количество столбцов во вставляемых MsgPack‑данных. Используется для автоматического определения схемы по данным. | `0` | -| [`output_format_msgpack_uuid_representation`](/operations/settings/settings-formats.md/#output_format_msgpack_uuid_representation) | способ вывода UUID в формате MsgPack. | `EXT` | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md deleted file mode 100644 index a9e4657b9d8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -alias: [] -description: 'Документация по формату MySQLDump' -input_format: true -keywords: ['MySQLDump'] -output_format: false -slug: /interfaces/formats/MySQLDump -title: 'MySQLDump' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|-------|-----------| -| ✔ | ✗ | | - - - -## Описание {#description} - -ClickHouse поддерживает чтение [дампов](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html) MySQL. - -Он считывает все данные из запросов `INSERT`, относящихся к одной таблице в дампе. -Если таблиц больше одной, по умолчанию считываются данные из первой. - -:::note -Этот формат поддерживает автоматическое определение схемы: если дамп содержит запрос `CREATE` для указанной таблицы, структура определяется по нему, в противном случае схема определяется по данным запросов `INSERT`. -::: - - - -## Пример использования {#example-usage} - -Предположим, у нас есть следующий файл дампа SQL: - -```sql title="dump.sql" -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test` ( - `x` int DEFAULT NULL, - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test` VALUES (1,NULL),(2,NULL),(3,NULL),(3,NULL),(4,NULL),(5,NULL),(6,7); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test 3` ( - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test 3` VALUES (1); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test2` ( - `x` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test2` VALUES (1),(2),(3); -``` - -Выполните следующие запросы: - -```sql title="Query" -DESCRIBE TABLE file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─name─┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ x │ Nullable(Int32) │ │ │ │ │ │ -└──────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql title="Query" -SELECT * -FROM file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─x─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - - -## Настройки формата {#format-settings} - -Вы можете указать имя таблицы, из которой нужно читать данные, с помощью настройки [`input_format_mysql_dump_table_name`](/operations/settings/settings-formats.md/#input_format_mysql_dump_table_name). -Если настройка `input_format_mysql_dump_map_columns` установлена в `1` и дамп содержит запрос `CREATE` для указанной таблицы или содержит имена столбцов в запросе `INSERT`, то столбцы из входных данных будут сопоставлены со столбцами таблицы по имени. -Столбцы с неизвестными именами будут пропущены, если настройка [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в `1`. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md deleted file mode 100644 index 695e5290468..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'Документация по формату MySQLWire' -keywords: ['MySQLWire'] -slug: /interfaces/formats/MySQLWire -title: 'MySQLWire' -doc_type: 'reference' ---- - - - -## Описание {#description} - - - -## Пример использования {#example-usage} - - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md deleted file mode 100644 index 7729fea843d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Native' -input_format: true -keywords: ['Native'] -output_format: true -slug: /interfaces/formats/Native -title: 'Native' -doc_type: 'reference' ---- - -| Входной | Выходной | Псевдоним | -|---------|----------|-----------| -| ✔ | ✔ | | - - - -## Описание {#description} - -Формат `Native` является самым эффективным форматом ClickHouse, поскольку он по-настоящему «колоночный» -и не преобразует столбцы в строки. - -В этом формате данные записываются и читаются [блоками](/development/architecture#block) в двоичном виде. -Для каждого блока последовательно записываются количество строк, количество столбцов, имена и типы столбцов, а также части столбцов, входящие в этот блок. - -Этот формат используется во встроенном интерфейсе для взаимодействия между серверами, в клиенте командной строки и в C++-клиентах. - -:::tip -Вы можете использовать этот формат для быстрого создания дампов, которые могут быть прочитаны только СУБД ClickHouse. -Работать с этим форматом напрямую может быть не слишком практично. -::: - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md deleted file mode 100644 index baa8687e572..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -alias: [] -description: 'Документация формата Npy' -input_format: true -keywords: ['Npy'] -output_format: true -slug: /interfaces/formats/Npy -title: 'Npy' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -Формат `Npy` предназначен для загрузки массива NumPy из файла `.npy` в ClickHouse. -Формат файла NumPy — это бинарный формат, используемый для эффективного хранения массивов числовых данных. -Во время импорта ClickHouse рассматривает внешнюю размерность массива как массив строк с одним столбцом. - -В таблице ниже приведены поддерживаемые типы данных Npy и соответствующие им типы в ClickHouse: - - - -## Соответствие типов данных {#data_types-matching} - -| Тип данных Npy (`INSERT`) | Тип данных ClickHouse | Тип данных Npy (`SELECT`) | -|---------------------------|----------------------------------------------------------------|---------------------------| -| `i1` | [Int8](/sql-reference/data-types/int-uint.md) | `i1` | -| `i2` | [Int16](/sql-reference/data-types/int-uint.md) | `i2` | -| `i4` | [Int32](/sql-reference/data-types/int-uint.md) | `i4` | -| `i8` | [Int64](/sql-reference/data-types/int-uint.md) | `i8` | -| `u1`, `b1` | [UInt8](/sql-reference/data-types/int-uint.md) | `u1` | -| `u2` | [UInt16](/sql-reference/data-types/int-uint.md) | `u2` | -| `u4` | [UInt32](/sql-reference/data-types/int-uint.md) | `u4` | -| `u8` | [UInt64](/sql-reference/data-types/int-uint.md) | `u8` | -| `f2`, `f4` | [Float32](/sql-reference/data-types/float.md) | `f4` | -| `f8` | [Float64](/sql-reference/data-types/float.md) | `f8` | -| `S`, `U` | [String](/sql-reference/data-types/string.md) | `S` | -| | [FixedString](/sql-reference/data-types/fixedstring.md) | `S` | - - - -## Пример использования {#example-usage} - -### Сохранение массива в формате .npy на Python {#saving-an-array-in-npy-format-using-python} - -```Python -import numpy as np -arr = np.array([[[1],[2],[3]],[[4],[5],[6]]]) -np.save('example_array.npy', arr) -``` - -### Чтение файлов NumPy в ClickHouse {#reading-a-numpy-file-in-clickhouse} - -```sql title="Query" -SELECT * -FROM file('example_array.npy', Npy) -``` - -```response title="Response" -┌─array─────────┐ -│ [[1],[2],[3]] │ -│ [[4],[5],[6]] │ -└───────────────┘ -``` - -### Выбор данных {#selecting-data} - -Вы можете выбрать данные из таблицы ClickHouse и сохранить их в файл формата Npy с помощью следующей команды clickhouse-client: - -```bash -$ clickhouse-client --query="SELECT {column} FROM {some_table} FORMAT Npy" > {filename.npy} -``` - - -## Настройки формата {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md deleted file mode 100644 index 496aa71e7a4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Null' -input_format: false -keywords: ['Null', 'format'] -output_format: true -slug: /interfaces/formats/Null -title: 'Null' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✗ | ✔ | | - - - -## Описание {#description} - -В формате `Null` ничего не выводится. -Это может поначалу показаться странным, но важно отметить, что, несмотря на отсутствие вывода, запрос всё равно обрабатывается, -и при использовании клиентского приложения командной строки данные передаются клиенту. - -:::tip -Формат `Null` может быть полезен для тестирования производительности. -::: - - - -## Пример использования {#example-usage} - -### Чтение данных {#reading-data} - -Рассмотрим таблицу `football` со следующими данными: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -Считайте данные в формате `Null`: - -```sql -SELECT * -FROM football -FORMAT Null -``` - -Запрос обработает данные, но ничего не выведет. - -```response -Получено 0 строк. Время выполнения: 0.154 сек. -``` - - -## Настройки формата {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md deleted file mode 100644 index 90ce52a59d6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'Документация по формату ODBCDriver2' -keywords: ['ODBCDriver2'] -slug: /interfaces/formats/ODBCDriver2 -title: 'ODBCDriver2' -doc_type: 'reference' ---- - - - -## Описание {#description} - - - -## Пример использования {#example-usage} - - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md deleted file mode 100644 index c92fa766eba..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -alias: [] -description: 'Документация по формату ORC' -input_format: true -keywords: ['ORC'] -output_format: true -slug: /interfaces/formats/ORC -title: 'ORC' -doc_type: 'reference' ---- - -| Входной | Выходной | Псевдоним | -|--------|----------|-----------| -| ✔ | ✔ | | - - - -## Описание {#description} - -[Apache ORC](https://orc.apache.org/) — это колоночный формат хранения, широко используемый в экосистеме [Hadoop](https://hadoop.apache.org/). - - - -## Соответствие типов данных {#data-types-matching-orc} - -В таблице ниже приведено сравнение поддерживаемых типов данных ORC и соответствующих типов данных [ClickHouse](/sql-reference/data-types/index.md) в запросах `INSERT` и `SELECT`. - -| ORC data type (`INSERT`) | ClickHouse data type | ORC data type (`SELECT`) | -|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------------------| -| `Boolean` | [UInt8](/sql-reference/data-types/int-uint.md) | `Boolean` | -| `Tinyint` | [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `Tinyint` | -| `Smallint` | [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `Smallint` | -| `Int` | [Int32/UInt32](/sql-reference/data-types/int-uint.md) | `Int` | -| `Bigint` | [Int64/UInt32](/sql-reference/data-types/int-uint.md) | `Bigint` | -| `Float` | [Float32](/sql-reference/data-types/float.md) | `Float` | -| `Double` | [Float64](/sql-reference/data-types/float.md) | `Double` | -| `Decimal` | [Decimal](/sql-reference/data-types/decimal.md) | `Decimal` | -| `Date` | [Date32](/sql-reference/data-types/date32.md) | `Date` | -| `Timestamp` | [DateTime64](/sql-reference/data-types/datetime64.md) | `Timestamp` | -| `String`, `Char`, `Varchar`, `Binary` | [String](/sql-reference/data-types/string.md) | `Binary` | -| `List` | [Array](/sql-reference/data-types/array.md) | `List` | -| `Struct` | [Tuple](/sql-reference/data-types/tuple.md) | `Struct` | -| `Map` | [Map](/sql-reference/data-types/map.md) | `Map` | -| `Int` | [IPv4](/sql-reference/data-types/int-uint.md) | `Int` | -| `Binary` | [IPv6](/sql-reference/data-types/ipv6.md) | `Binary` | -| `Binary` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `Binary` | -| `Binary` | [Decimal256](/sql-reference/data-types/decimal.md) | `Binary` | - -- Другие типы не поддерживаются. -- Массивы могут быть вложенными и иметь в качестве аргумента значение типа `Nullable`. Типы `Tuple` и `Map` также могут быть вложенными. -- Типы данных столбцов таблицы ClickHouse не обязаны совпадать с соответствующими полями ORC. При вставке данных ClickHouse интерпретирует типы данных согласно таблице выше, а затем [приводит](/sql-reference/functions/type-conversion-functions#cast) данные к типу, заданному для столбца таблицы ClickHouse. - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем ORC-файл с именем `football.orc` со следующими данными: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -Введите данные: - -```sql -INSERT INTO football FROM INFILE 'football.orc' FORMAT ORC; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `ORC`: - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.orc' -FORMAT ORC -``` - -:::tip -ORC — это бинарный формат, который не отображается в человекочитаемом виде в терминале. Используйте оператор `INTO OUTFILE` для вывода данных в файлы ORC. -::: - - -## Настройки формата {#format-settings} - -| Setting | Description | Default | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------|---------| -| [`output_format_arrow_string_as_string`](/operations/settings/settings-formats.md/#output_format_arrow_string_as_string) | Использовать тип Arrow String вместо Binary для столбцов типа String. | `false` | -| [`output_format_orc_compression_method`](/operations/settings/settings-formats.md/#output_format_orc_compression_method) | Метод сжатия, используемый в выходном формате ORC. Значение по умолчанию — | `none` | -| [`input_format_arrow_case_insensitive_column_matching`](/operations/settings/settings-formats.md/#input_format_arrow_case_insensitive_column_matching) | Игнорировать регистр при сопоставлении столбцов Arrow со столбцами ClickHouse. | `false` | -| [`input_format_arrow_allow_missing_columns`](/operations/settings/settings-formats.md/#input_format_arrow_allow_missing_columns) | Разрешить отсутствующие столбцы при чтении данных в формате Arrow. | `false` | -| [`input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference) | Разрешить пропуск столбцов с неподдерживаемыми типами при определении схемы для формата Arrow. | `false` | - -Для обмена данными с Hadoop вы можете использовать [движок таблиц HDFS](/engines/table-engines/integrations/hdfs.md). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/One.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/One.md deleted file mode 100644 index ea208815fe9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/One.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -alias: [] -description: 'Документация формата One' -input_format: true -keywords: ['One'] -output_format: false -slug: /interfaces/formats/One -title: 'One' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## Описание {#description} - -Формат `One` — это специальный входной формат, который не читает данные из файла и возвращает только одну строку со столбцом типа [`UInt8`](../../sql-reference/data-types/int-uint.md) с именем `dummy` и значением `0` (как таблица `system.one`). -Может использоваться с виртуальными столбцами `_file/_path` для получения списка всех файлов без чтения реальных данных. - - - -## Пример использования {#example-usage} - -Пример: - -```sql title="Query" -SELECT _file FROM file('path/to/files/data*', One); -``` - -```text title="Response" -┌─_file────┐ -│ data.csv │ -└──────────┘ -┌─_file──────┐ -│ data.jsonl │ -└────────────┘ -┌─_file────┐ -│ data.tsv │ -└──────────┘ -┌─_file────────┐ -│ data.parquet │ -└──────────────┘ -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md deleted file mode 100644 index 3929b0aee9d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Parquet' -input_format: true -keywords: ['Parquet'] -output_format: true -slug: /interfaces/formats/Parquet -title: 'Parquet' -doc_type: 'reference' ---- - -| Входной формат | Выходной формат | Синоним | -|----------------|-----------------|---------| -| ✔ | ✔ | | - - - -## Описание {#description} - -[Apache Parquet](https://parquet.apache.org/) — это колоночный формат хранения данных, широко распространённый в экосистеме Hadoop. ClickHouse поддерживает чтение и запись данных в этом формате. - - - -## Соответствие типов данных {#data-types-matching-parquet} - -В таблице ниже показано, как типы данных Parquet сопоставляются с [типами данных](/sql-reference/data-types/index.md) ClickHouse. - -| Тип Parquet (логический, преобразованный или физический) | Тип данных ClickHouse | -|-----------------------------------------------------------|------------------------| -| `BOOLEAN` | [Bool](/sql-reference/data-types/boolean.md) | -| `UINT_8` | [UInt8](/sql-reference/data-types/int-uint.md) | -| `INT_8` | [Int8](/sql-reference/data-types/int-uint.md) | -| `UINT_16` | [UInt16](/sql-reference/data-types/int-uint.md) | -| `INT_16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | -| `UINT_32` | [UInt32](/sql-reference/data-types/int-uint.md) | -| `INT_32` | [Int32](/sql-reference/data-types/int-uint.md) | -| `UINT_64` | [UInt64](/sql-reference/data-types/int-uint.md) | -| `INT_64` | [Int64](/sql-reference/data-types/int-uint.md) | -| `DATE` | [Date32](/sql-reference/data-types/date.md) | -| `TIMESTAMP`, `TIME` | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `FLOAT` | [Float32](/sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | -| `INT96` | [DateTime64(9, 'UTC')](/sql-reference/data-types/datetime64.md) | -| `BYTE_ARRAY`, `UTF8`, `ENUM`, `BSON` | [String](/sql-reference/data-types/string.md) | -| `JSON` | [JSON](/sql-reference/data-types/newjson.md) | -| `FIXED_LEN_BYTE_ARRAY` | [FixedString](/sql-reference/data-types/fixedstring.md) | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | -| `LIST` | [Array](/sql-reference/data-types/array.md) | -| `MAP` | [Map](/sql-reference/data-types/map.md) | -| struct | [Tuple](/sql-reference/data-types/tuple.md) | -| `FLOAT16` | [Float32](/sql-reference/data-types/float.md) | -| `UUID` | [FixedString(16)](/sql-reference/data-types/fixedstring.md) | -| `INTERVAL` | [FixedString(12)](/sql-reference/data-types/fixedstring.md) | - -При записи в файл Parquet типы данных, для которых нет соответствующего типа Parquet, преобразуются в ближайший доступный тип: - -| Тип данных ClickHouse | Тип Parquet | -|------------------------|-------------| -| [IPv4](/sql-reference/data-types/ipv4.md) | `UINT_32` | -| [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_LEN_BYTE_ARRAY` (16 байт) | -| [Date](/sql-reference/data-types/date.md) (16 бит) | `DATE` (32 бита) | -| [DateTime](/sql-reference/data-types/datetime.md) (32 бита, секунды) | `TIMESTAMP` (64 бита, миллисекунды) | -| [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_LEN_BYTE_ARRAY` (16/32 байта, порядок little-endian) | - -Массивы могут быть вложенными и могут иметь значение типа `Nullable` в качестве аргумента. Типы `Tuple` и `Map` также могут быть вложенными. - -Типы данных столбцов таблицы ClickHouse могут отличаться от соответствующих полей вставляемых данных Parquet. При вставке данных ClickHouse интерпретирует типы данных согласно приведённой выше таблице, а затем [приводит](/sql-reference/functions/type-conversion-functions#cast) данные к тому типу данных, который установлен для столбца таблицы ClickHouse. Например, столбец Parquet `UINT_32` может быть прочитан в столбец ClickHouse [IPv4](/sql-reference/data-types/ipv4.md). - - - -Для некоторых типов Parquet не существует близкого по смыслу типа ClickHouse. Мы читаем их следующим образом: -* `TIME` (время суток) читается как метка времени. Например, `10:23:13.000` становится `1970-01-01 10:23:13.000`. -* `TIMESTAMP`/`TIME` с `isAdjustedToUTC=false` — это локальное время по настенным часам (поля год, месяц, день, час, минута, секунда и доля секунды в локальном часовом поясе, независимо от того, какой конкретный часовой пояс считается локальным), аналогично SQL `TIMESTAMP WITHOUT TIME ZONE`. ClickHouse читает его так, как если бы это была метка времени в UTC. Например, `2025-09-29 18:42:13.000` (представляющее показания локальных настенных часов) становится `2025-09-29 18:42:13.000` (`DateTime64(3, 'UTC')`, представляя точку во времени). При преобразовании к String выводятся корректные год, месяц, день, час, минута, секунда и доля секунды, которые затем могут интерпретироваться как относящиеся к какому-либо локальному часовому поясу, а не к UTC. Противоинтуитивно, смена типа с `DateTime64(3, 'UTC')` на `DateTime64(3)` не поможет, так как оба типа представляют точку во времени, а не показание часов, но `DateTime64(3)` будет некорректно форматироваться с использованием локального часового пояса. -* `INTERVAL` в настоящий момент читается как `FixedString(12)` с сырым двоичным представлением временного интервала в том виде, в котором он закодирован в файле Parquet. - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используйте файл Parquet со следующими данными, сохранённый под именем `football.parquet`: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -Введите данные: - -```sql -INSERT INTO football FROM INFILE 'football.parquet' FORMAT Parquet; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `Parquet`: - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.parquet' -FORMAT Parquet -``` - -:::tip -Parquet — это двоичный формат, который не отображается в человекочитаемом виде в терминале. Используйте `INTO OUTFILE` для записи файлов в формате Parquet. -::: - -Для обмена данными с Hadoop вы можете использовать [`движок таблиц HDFS`](/engines/table-engines/integrations/hdfs.md). - - -## Параметры форматирования {#format-settings} - - - -| Настройка | Описание | По умолчанию | -| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `input_format_parquet_case_insensitive_column_matching` | Не учитывать регистр при сопоставлении столбцов Parquet со столбцами CH. | `0` | -| `input_format_parquet_preserve_order` | Избегайте изменения порядка строк при чтении из файлов Parquet. Обычно это значительно замедляет чтение. | `0` | -| `input_format_parquet_filter_push_down` | При чтении файлов Parquet пропускать целые группы строк на основе выражений WHERE/PREWHERE и статистики min/max в метаданных Parquet. | `1` | -| `input_format_parquet_bloom_filter_push_down` | При чтении файлов Parquet пропускать целые группы строк на основе выражений WHERE и фильтра Блума в метаданных файлов Parquet. | `0` | -| `input_format_parquet_use_native_reader` | При чтении файлов в формате Parquet использовать нативный считыватель вместо считывателя Arrow. | `0` | -| `input_format_parquet_allow_missing_columns` | Допускать отсутствующие столбцы при чтении входных форматов Parquet | `1` | -| `input_format_parquet_local_file_min_bytes_for_seek` | Минимальное количество байт при локальном чтении файла, начиная с которого используется seek вместо чтения с пропуском (ignore) во входном формате Parquet | `8192` | -| `input_format_parquet_enable_row_group_prefetch` | Включить предварительную выборку групп строк при разборе файлов Parquet. В настоящее время предварительную выборку может выполнять только однопоточный парсер. | `1` | -| `input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference` | Пропускать столбцы с неподдерживаемыми типами при определении схемы для формата Parquet | `0` | -| `input_format_parquet_max_block_size` | Максимальный размер блока для читателя Parquet. | `65409` | -| `input_format_parquet_prefer_block_bytes` | Средний размер блока в байтах, выдаваемого читателем Parquet | `16744704` | -| `input_format_parquet_enable_json_parsing` | При чтении файлов Parquet разбирайте столбцы JSON как столбцы ClickHouse JSON Column. | `1` | -| `output_format_parquet_row_group_size` | Целевое количество строк в группе. | `1000000` | -| `output_format_parquet_row_group_size_bytes` | Целевой размер группы строк в байтах до сжатия. | `536870912` | -| `output_format_parquet_string_as_string` | Используйте тип Parquet String вместо Binary для столбцов типа String. | `1` | -| `output_format_parquet_fixed_string_as_fixed_byte_array` | Используйте тип Parquet FIXED_LEN_BYTE_ARRAY вместо Binary для столбцов FixedString. | `1` | -| `output_format_parquet_version` | Версия формата Parquet для формата вывода. Поддерживаемые версии: 1.0, 2.4, 2.6 и 2.latest (по умолчанию) | `2.latest` | -| `output_format_parquet_compression_method` | Метод сжатия для формата вывода Parquet. Поддерживаемые кодеки: snappy, lz4, brotli, zstd, gzip, none (без сжатия) | `zstd` | -| `output_format_parquet_compliant_nested_types` | В схеме файла Parquet используйте имя 'element' вместо 'item' для элементов списка. Это исторический артефакт реализации библиотеки Arrow. Как правило, это повышает совместимость, за исключением, возможно, некоторых старых версий Arrow. | `1` | -| `output_format_parquet_use_custom_encoder` | Используйте более быструю реализацию кодировщика Parquet. | `1` | -| `output_format_parquet_parallel_encoding` | Выполнять кодирование в Parquet в нескольких потоках. Требует включённой настройки output_format_parquet_use_custom_encoder. | `1` | -| `output_format_parquet_data_page_size` | Целевой размер страницы в байтах перед сжатием. | `1048576` | -| `output_format_parquet_batch_size` | Проверять размер страницы каждые указанное количество строк. Рассмотрите возможность уменьшения значения, если в столбцах средний размер значений превышает несколько КБ. | `1024` | -| `output_format_parquet_write_page_index` | Добавить возможность записывать страничный индекс в файлы Parquet. | `1` | -| `input_format_parquet_import_nested` | Устаревший параметр, ни на что не влияет. | `0` | -| `input_format_parquet_local_time_as_utc` | true | Определяет тип данных, используемый при выводе схемы (schema inference) для временных меток Parquet с isAdjustedToUTC=false. Если true: DateTime64(..., 'UTC'), если false: DateTime64(...). Ни один из вариантов не является полностью корректным, так как в ClickHouse нет типа данных для локального времени по настенным часам (wall-clock time). Как ни парадоксально, значение 'true', вероятно, является менее некорректным вариантом, поскольку форматирование временной метки 'UTC' как String даст представление правильного локального времени. | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md deleted file mode 100644 index abcd72138f0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -description: 'Документация по формату ParquetMetadata' -keywords: ['ParquetMetadata'] -slug: /interfaces/formats/ParquetMetadata -title: 'ParquetMetadata' -doc_type: 'reference' ---- - - - -## Описание {#description} - -Специальный формат для чтения метаданных файлов Parquet (https://parquet.apache.org/docs/file-format/metadata/). Всегда выводит одну строку со следующей структурой/содержимым: -- `num_columns` - количество столбцов -- `num_rows` - общее количество строк -- `num_row_groups` - общее количество групп строк -- `format_version` - версия формата Parquet, всегда 1.0 или 2.6 -- `total_uncompressed_size` - общий размер данных в байтах в несжатом виде, вычисляемый как сумма total_byte_size из всех групп строк -- `total_compressed_size` - общий размер данных в байтах в сжатом виде, вычисляемый как сумма total_compressed_size из всех групп строк -- `columns` - список метаданных столбцов со следующей структурой: - - `name` - имя столбца - - `path` - путь столбца (отличается от имени для вложенного столбца) - - `max_definition_level` - максимальный уровень определения - - `max_repetition_level` - максимальный уровень повторения - - `physical_type` - физический тип столбца - - `logical_type` - логический тип столбца - - `compression` - тип сжатия, используемый для этого столбца - - `total_uncompressed_size` - общий размер столбца в байтах в несжатом виде, вычисляемый как сумма total_uncompressed_size для столбца из всех групп строк - - `total_compressed_size` - общий размер столбца в байтах в сжатом виде, вычисляемый как сумма total_compressed_size для столбца из всех групп строк - - `space_saved` - процент пространства, сэкономленного за счет сжатия, вычисляемый как (1 - total_compressed_size/total_uncompressed_size) - - `encodings` - список кодировок (encodings), используемых для этого столбца -- `row_groups` - список метаданных групп строк со следующей структурой: - - `num_columns` - количество столбцов в группе строк - - `num_rows` - количество строк в группе строк - - `total_uncompressed_size` - общий размер группы строк в байтах в несжатом виде - - `total_compressed_size` - общий размер группы строк в байтах в сжатом виде - - `columns` - список метаданных чанков столбцов со следующей структурой: - - `name` - имя столбца - - `path` - путь столбца - - `total_compressed_size` - общий размер столбца в байтах в сжатом виде - - `total_uncompressed_size` - общий размер группы строк в байтах в несжатом виде - - `have_statistics` - булевый флаг, который указывает, содержат ли метаданные чанка столбца статистику по столбцу - - `statistics` - статистика чанка столбца (все поля равны NULL, если have_statistics = false) со следующей структурой: - - `num_values` - количество значений, отличных от NULL (non-null), в чанке столбца - - `null_count` - количество значений NULL в чанке столбца - - `distinct_count` - количество различных значений в чанке столбца - - `min` - минимальное значение в чанке столбца - - `max` - максимальное значение в чанке столбца - - - -## Пример использования {#example-usage} - -Пример: - -```sql -SELECT * -FROM file(data.parquet, ParquetMetadata) -FORMAT PrettyJSONEachRow -``` - -```json -{ - "num_columns": "2", - "num_rows": "100000", - "num_row_groups": "2", - "format_version": "2.6", - "metadata_size": "577", - "total_uncompressed_size": "282436", - "total_compressed_size": "26633", - "columns": [ - { - "name": "number", - "path": "number", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "INT32", - "logical_type": "Int(bitWidth=16, isSigned=false)", - "compression": "LZ4", - "total_uncompressed_size": "133321", - "total_compressed_size": "13293", - "space_saved": "90.03%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "BYTE_ARRAY", - "logical_type": "None", - "compression": "LZ4", - "total_uncompressed_size": "149115", - "total_compressed_size": "13340", - "space_saved": "91.05%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - } - ], - "row_groups": [ - { - "num_columns": "2", - "num_rows": "65409", - "total_uncompressed_size": "179809", - "total_compressed_size": "14163", - "columns": [ - { - "name": "number", - "path": "number", - "total_compressed_size": "7070", - "total_uncompressed_size": "85956", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "0", - "max": "999" - } - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "total_compressed_size": "7093", - "total_uncompressed_size": "93853", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "Hello0", - "max": "Hello999" - } - } - ] - }, - ... - ] -} -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md deleted file mode 100644 index 039c11a47ea..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'Документация по формату PostgreSQLWire' -keywords: ['PostgreSQLWire'] -slug: /interfaces/formats/PostgreSQLWire -title: 'PostgreSQLWire' -doc_type: 'reference' ---- - - - -## Описание {#description} - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md deleted file mode 100644 index 78a8c4448ef..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Pretty' -input_format: false -keywords: ['Pretty'] -output_format: true -slug: /interfaces/formats/Pretty -title: 'Pretty' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Формат `Pretty` выводит данные в виде таблиц с использованием символов Unicode, -применяя ANSI escape-последовательности для отображения цветов в терминале. -Рисуется полная сетка таблицы, и каждая строка занимает две строки в терминале. -Каждый блок результатов выводится как отдельная таблица. -Это необходимо для того, чтобы блоки можно было выводить без буферизации результатов (буферизация потребовалась бы для предварительного расчёта видимой ширины всех значений). - -[NULL](/sql-reference/syntax.md) выводится как `ᴺᵁᴸᴸ`. - - - -## Пример использования {#example-usage} - -Пример (для формата [`PrettyCompact`](./PrettyCompact.md)): - -```sql title="Query" -SELECT * FROM t_null -``` - -```response title="Response" -┌─x─┬────y─┐ -│ 1 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -Строки не экранируются ни в одном из форматов семейства `Pretty`. Следующий пример показан для формата [`PrettyCompact`](./PrettyCompact.md): - -```sql title="Query" -SELECT 'Строка с \'кавычками\' и символом табуляции \t' AS Escaping_test -``` - -```response title="Response" -┌─Escaping_test────────────────────────┐ -│ Строка с 'кавычками' и символом │ -└──────────────────────────────────────┘ -``` - -Чтобы не выводить в терминал слишком большой объём данных, печатаются только первые `10,000` строк. -Если количество строк больше либо равно `10,000`, выводится сообщение «Showed first 10 000». - -:::note -Этот формат подходит только для вывода результатов запроса, но не для разбора данных. -::: - -Формат Pretty поддерживает вывод итоговых значений (при использовании `WITH TOTALS`) и экстремумов (когда параметр «extremes» установлен в 1). -В этих случаях итоговые значения и экстремумы выводятся после основных данных в отдельных таблицах. -Это показано в следующем примере, который использует формат [`PrettyCompact`](./PrettyCompact.md): - -```sql title="Query" -SELECT EventDate, count() AS c -FROM test.hits -GROUP BY EventDate -WITH TOTALS -ORDER BY EventDate -FORMAT PrettyCompact -``` - -```response title="Response" -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1406958 │ -│ 2014-03-18 │ 1383658 │ -│ 2014-03-19 │ 1405797 │ -│ 2014-03-20 │ 1353623 │ -│ 2014-03-21 │ 1245779 │ -│ 2014-03-22 │ 1031592 │ -│ 2014-03-23 │ 1046491 │ -└────────────┴─────────┘ - -Итого: -┌──EventDate─┬───────c─┐ -│ 1970-01-01 │ 8873898 │ -└────────────┴─────────┘ - -Минимальные и максимальные значения: -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1031592 │ -│ 2014-03-23 │ 1406958 │ -└────────────┴─────────┘ -``` - - -## Параметры форматирования {#format-settings} - - diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md deleted file mode 100644 index 8a5469c8382..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettyCompact' -input_format: false -keywords: ['PrettyCompact'] -output_format: true -slug: /interfaces/formats/PrettyCompact -title: 'PrettyCompact' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`Pretty`](./Pretty.md) тем, что таблица отображается с сеткой между строками. -Благодаря этому результат получается более компактным. - -:::note -Этот формат по умолчанию используется в клиенте командной строки в интерактивном режиме. -::: - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md deleted file mode 100644 index 0c2cb3b28c1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация о формате PrettyCompactMonoBlock' -input_format: false -keywords: ['PrettyCompactMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactMonoBlock -title: 'PrettyCompactMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettyCompact`](./PrettyCompact.md) тем, что до `10 000` строк накапливаются в буфере, -а затем выводятся в виде одной таблицы, а не по [блокам](/development/architecture#block). - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md deleted file mode 100644 index 8ba858601f3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация для формата PrettyCompactNoEscapes' -input_format: false -keywords: ['PrettyCompactNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapes -title: 'PrettyCompactNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettyCompact`](./PrettyCompact.md) тем, что [escape-последовательности ANSI](http://en.wikipedia.org/wiki/ANSI_escape_code) не используются. -Это необходимо для отображения формата в браузере, а также для использования утилиты командной строки `watch`. - - - -## Пример использования {#example-usage} - - - -## Настройки форматирования {#format-settings} - - diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md deleted file mode 100644 index f29330239c3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettyCompactNoEscapesMonoBlock' -input_format: false -keywords: ['PrettyCompactNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapesMonoBlock -title: 'PrettyCompactNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettyCompactNoEscapes`](./PrettyCompactNoEscapes.md) тем, что до `10,000` строк накапливаются в буфере, а затем выводятся как одна таблица, а не по [блокам](/development/architecture#block). - - - -## Примеры использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md deleted file mode 100644 index 25da2d2f848..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettyMonoBlock' -input_format: false -keywords: ['PrettyMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyMonoBlock -title: 'PrettyMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`Pretty`](/interfaces/formats/Pretty) тем, что до `10 000` строк буферизуются, после чего результат выводится в виде одной таблицы, а не по [блокам](/development/architecture#block). - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md deleted file mode 100644 index 4463d69f94f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettyNoEscapes' -input_format: false -keywords: ['PrettyNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapes -title: 'PrettyNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от [Pretty](/interfaces/formats/Pretty) тем, что не используются [последовательности управляющих кодов ANSI](http://en.wikipedia.org/wiki/ANSI_escape_code). -Это необходимо для отображения этого формата в браузере, а также для использования с утилитой командной строки `watch`. - - - -## Пример использования {#example-usage} - -Пример: - -```bash -$ watch -n1 "clickhouse-client --query='SELECT event, value FROM system.events FORMAT PrettyCompactNoEscapes'" -``` - -:::note -[HTTP-интерфейс](../../../interfaces/http.md) можно использовать для отображения данного формата в браузере. -::: - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md deleted file mode 100644 index 921350d5152..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация о формате PrettyNoEscapesMonoBlock' -input_format: false -keywords: ['PrettyNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapesMonoBlock -title: 'PrettyNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettyNoEscapes`](./PrettyNoEscapes.md) тем, что до `10 000` строк сохраняются в буфере, -а затем выводятся как одна таблица, а не блоками. - - - -## Пример использования {#example-usage} - - - -## Настройки форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md deleted file mode 100644 index 827c5a0fc23..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettySpace' -input_format: false -keywords: ['PrettySpace'] -output_format: true -slug: /interfaces/formats/PrettySpace -title: 'PrettySpace' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettyCompact`](./PrettyCompact.md) тем, что для отображения таблицы вместо сетки используются пробелы (пробельные символы). - - - -## Пример использования {#example-usage} - - - -## Настройки форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md deleted file mode 100644 index ee14a889c18..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация о формате PrettySpaceMonoBlock' -input_format: false -keywords: ['PrettySpaceMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceMonoBlock -title: 'PrettySpaceMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettySpace`](./PrettySpace.md) тем, что до `10,000` строк накапливаются в буфере, -а затем выводятся в виде одной таблицы, а не по [блокам](/development/architecture#block). - - - -## Пример использования {#example-usage} - - - -## Настройки форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md deleted file mode 100644 index 16c68c8acb5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация о формате PrettySpaceNoEscapes' -input_format: false -keywords: ['PrettySpaceNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapes -title: 'PrettySpaceNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettySpace`](./PrettySpace.md) тем, что в нём не используются [последовательности управляющих кодов ANSI (ANSI escape sequences)](http://en.wikipedia.org/wiki/ANSI_escape_code). -Это необходимо для корректного отображения этого формата в браузере, а также для использования консольной утилиты `watch`. - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md deleted file mode 100644 index 2d6384281d6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'Документация по формату PrettySpaceNoEscapesMonoBlock' -input_format: false -keywords: ['PrettySpaceNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapesMonoBlock -title: 'PrettySpaceNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✗ | ✔ | | - - -## Описание {#description} - -Отличается от формата [`PrettySpaceNoEscapes`](./PrettySpaceNoEscapes.md) тем, что до `10 000` строк буферизуются, -а затем выводятся в виде одной таблицы, а не по [блокам](/development/architecture#block). - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md deleted file mode 100644 index 265a0d64512..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md +++ /dev/null @@ -1,14 +0,0 @@ -{/* Примечание: этот файл используется как фрагмент во всех файлах, которые его импортируют */ } - -Следующие настройки общие для всех форматов `Pretty`: - -| Настройка | Описание | Значение по умолчанию | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| [`output_format_pretty_max_rows`](/operations/settings/settings-formats.md/#output_format_pretty_max_rows) | Максимальное количество строк для форматов Pretty. | `10000` | -| [`output_format_pretty_max_column_pad_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_column_pad_width) | Максимальная ширина дополнения (выравнивания) всех значений в столбце для форматов Pretty. | `250` | -| [`output_format_pretty_max_value_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_value_width) | Максимальная ширина отображаемого значения в форматах Pretty. Если значение больше — оно будет обрезано. | `10000` | -| [`output_format_pretty_color`](/operations/settings/settings-formats.md/#output_format_pretty_color) | Использовать управляющие последовательности ANSI для цветного вывода в форматах Pretty. | `true` | -| [`output_format_pretty_grid_charset`](/operations/settings/settings-formats.md/#output_format_pretty_grid_charset) | Набор символов для печати границ таблицы. Доступные наборы: ASCII, UTF-8. | `UTF-8` | -| [`output_format_pretty_row_numbers`](/operations/settings/settings-formats.md/#output_format_pretty_row_numbers) | Добавлять номера строк перед каждой строкой для форматов Pretty. | `true` | -| [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names) | Отображать имена столбцов в нижнем колонтитуле, если таблица содержит много строк. | `true` | -| [`output_format_pretty_display_footer_column_names_min_rows`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names_min_rows) | Минимальное количество строк, при котором отображается нижний колонтитул, если включена настройка [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names). | `50` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md deleted file mode 100644 index 93d029e238c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -alias: [] -description: 'Документация для формата Prometheus' -input_format: false -keywords: ['Prometheus'] -output_format: true -slug: /interfaces/formats/Prometheus -title: 'Prometheus' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|-------|-----------| -| ✗ | ✔ | | - -## Описание {#description} - -Выводит метрики в [текстовом формате экспозиции Prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format). - -Для этого формата требуется, чтобы выходная таблица была структурирована правильно в соответствии со следующими правилами: - -- Столбцы `name` ([String](/sql-reference/data-types/string.md)) и `value` (число) являются обязательными. -- Строки могут опционально содержать `help` ([String](/sql-reference/data-types/string.md)) и `timestamp` (число). -- Столбец `type` ([String](/sql-reference/data-types/string.md)) должен быть одним из `counter`, `gauge`, `histogram`, `summary`, `untyped` или пустым. -- Каждое значение метрики также может иметь некоторые `labels` ([Map(String, String)](/sql-reference/data-types/map.md)). -- Несколько последовательных строк могут относиться к одной метрике с различными метками. Таблица должна быть отсортирована по имени метрики (например, с помощью `ORDER BY name`). - -Существуют особые требования к меткам `histogram` и `summary` - см. [документацию Prometheus](https://prometheus.io/docs/instrumenting/exposition_formats/#histograms-and-summaries) для получения подробностей. -Специальные правила применяются к строкам с метками `{'count':''}` и `{'sum':''}`, которые преобразуются в `_count` и `_sum` соответственно. - -## Пример использования {#example-usage} - -```yaml -┌─name────────────────────────────────┬─type──────┬─help──────────────────────────────────────┬─labels─────────────────────────┬────value─┬─────timestamp─┐ -│ http_request_duration_seconds │ histogram │ A histogram of the request duration. │ {'le':'0.05'} │ 24054 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.1'} │ 33444 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.2'} │ 100392 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.5'} │ 129389 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'1'} │ 133988 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'+Inf'} │ 144320 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'sum':''} │ 53423 │ 0 │ -│ http_requests_total │ counter │ Total number of HTTP requests │ {'method':'post','code':'200'} │ 1027 │ 1395066363000 │ -│ http_requests_total │ counter │ │ {'method':'post','code':'400'} │ 3 │ 1395066363000 │ -│ metric_without_timestamp_and_labels │ │ │ {} │ 12.47 │ 0 │ -│ rpc_duration_seconds │ summary │ A summary of the RPC duration in seconds. │ {'quantile':'0.01'} │ 3102 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.05'} │ 3272 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.5'} │ 4773 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.9'} │ 9001 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.99'} │ 76656 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'count':''} │ 2693 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'sum':''} │ 17560473 │ 0 │ -│ something_weird │ │ │ {'problem':'division by zero'} │ inf │ -3982045 │ -└─────────────────────────────────────┴───────────┴───────────────────────────────────────────┴────────────────────────────────┴──────────┴───────────────┘ -``` - -Будет отформатировано как: - -```text -# HELP http_request_duration_seconds A histogram of the request duration. {#help-http_request_duration_seconds-a-histogram-of-the-request-duration} -# TYPE http_request_duration_seconds histogram {#type-http_request_duration_seconds-histogram} -http_request_duration_seconds_bucket{le="0.05"} 24054 -http_request_duration_seconds_bucket{le="0.1"} 33444 -http_request_duration_seconds_bucket{le="0.5"} 129389 -http_request_duration_seconds_bucket{le="1"} 133988 -http_request_duration_seconds_bucket{le="+Inf"} 144320 -http_request_duration_seconds_sum 53423 -http_request_duration_seconds_count 144320 - -# HELP http_requests_total Total number of HTTP requests {#help-http_requests_total-total-number-of-http-requests} -# TYPE http_requests_total counter {#type-http_requests_total-counter} -http_requests_total{code="200",method="post"} 1027 1395066363000 -http_requests_total{code="400",method="post"} 3 1395066363000 - -metric_without_timestamp_and_labels 12.47 - -# HELP rpc_duration_seconds A summary of the RPC duration in seconds. {#help-rpc_duration_seconds-a-summary-of-the-rpc-duration-in-seconds} -# TYPE rpc_duration_seconds summary {#type-rpc_duration_seconds-summary} -rpc_duration_seconds{quantile="0.01"} 3102 -rpc_duration_seconds{quantile="0.05"} 3272 -rpc_duration_seconds{quantile="0.5"} 4773 -rpc_duration_seconds{quantile="0.9"} 9001 -rpc_duration_seconds{quantile="0.99"} 76656 -rpc_duration_seconds_sum 17560473 -rpc_duration_seconds_count 2693 - -something_weird{problem="division by zero"} +Inf -3982045 -``` - -## Настройки формата {#format-settings} - diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md deleted file mode 100644 index e81cb90cd69..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md +++ /dev/null @@ -1,391 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Protobuf' -input_format: true -keywords: ['Protobuf'] -output_format: true -slug: /interfaces/formats/Protobuf -title: 'Protobuf' -doc_type: 'guide' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - -## Описание {#description} - -Формат `Protobuf` — это формат [Protocol Buffers](https://protobuf.dev/). - -Этот формат требует внешней схемы формата, которая кэшируется между запросами. - -ClickHouse поддерживает: - -* синтаксис как `proto2`, так и `proto3`. -* поля `Repeated`/`optional`/`required`. - -Чтобы определить соответствие между столбцами таблицы и полями типа сообщения в Protocol Buffers, ClickHouse сравнивает их имена. -Сравнение нечувствительно к регистру, а символы `_` (подчёркивание) и `.` (точка) считаются равными. -Если типы столбца и поля сообщения Protocol Buffers отличаются, выполняется необходимое преобразование. - -Поддерживаются вложенные сообщения. Например, для поля `z` в следующем типе сообщения: - -```capnp -message MessageType { - message XType { - message YType { - int32 z; - }; - repeated YType y; - }; - XType x; -}; -``` - -ClickHouse пытается найти столбец с именем `x.y.z` (или `x_y_z`, или `X.y_Z` и так далее). - -Вложенные сообщения подходят в качестве входных и выходных данных для [вложенных структур данных](/sql-reference/data-types/nested-data-structures/index.md). - -Значения по умолчанию, определённые в protobuf-схеме, подобной приведённой ниже, не применяются — вместо них используются [значения по умолчанию таблицы](/sql-reference/statements/create/table#default_values): - -```capnp -syntax = "proto2"; - -message MessageType { - optional int32 result_per_page = 3 [default = 10]; -} -``` - -Если сообщение содержит [oneof](https://protobuf.dev/programming-guides/proto3/#oneof) и параметр `input_format_protobuf_oneof_presence` установлен, ClickHouse заполняет столбец, указывающий, какое поле oneof было найдено. - -```capnp -syntax = "proto3"; - -message StringOrString { - oneof string_oneof { - string string1 = 1; - string string2 = 42; - } -} -``` - -```sql -CREATE TABLE string_or_string ( string1 String, string2 String, string_oneof Enum('no'=0, 'hello' = 1, 'world' = 42)) Engine=MergeTree ORDER BY tuple(); -INSERT INTO string_or_string from INFILE '$CURDIR/data_protobuf/String1' SETTINGS format_schema='$SCHEMADIR/string_or_string.proto:StringOrString' FORMAT ProtobufSingle; -SELECT * FROM string_or_string -``` - -```text - ┌─────────┬─────────┬──────────────┐ - │ строка1 │ строка2 │ строка_один_из │ - ├─────────┼─────────┼──────────────┤ -1. │ │ строка2 │ мир │ - ├─────────┼─────────┼──────────────┤ -2. │ строка1 │ │ привет │ - └─────────┴─────────┴──────────────┘ -``` - -Имя столбца, указывающего на наличие, должно совпадать с именем `oneof`. Поддерживаются вложенные сообщения (см. [basic-examples](#basic-examples)). -Допустимые типы: Int8, UInt8, Int16, UInt16, Int32, UInt32, Int64, UInt64, Enum, Enum8 или Enum16. -Enum (так же, как Enum8 или Enum16) должен содержать все возможные теги `oneof` плюс 0, обозначающий отсутствие; строковые представления при этом неважны. - -Настройка [`input_format_protobuf_oneof_presence`](/operations/settings/settings-formats.md#input_format_protobuf_oneof_presence) по умолчанию отключена. - -ClickHouse считывает и выводит сообщения protobuf в формате `length-delimited`. -Это означает, что перед каждым сообщением его длина должна быть записана как [целое число переменной длины (varint)](https://developers.google.com/protocol-buffers/docs/encoding#varints). - - -## Пример использования {#example-usage} - -### Чтение и запись данных {#basic-examples} - -:::note Example files -Файлы, используемые в этом примере, можно найти в [репозитории с примерами](https://github.com/ClickHouse/formats/ProtoBuf) -::: - -В этом примере мы прочитаем данные из файла `protobuf_message.bin` в таблицу ClickHouse. Затем запишем их -обратно в файл `protobuf_message_from_clickhouse.bin`, используя формат `Protobuf`. - -Имеется файл `schemafile.proto`: - -```capnp -syntax = "proto3"; - -message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; -}; -``` - - -
- Генерация бинарного файла - - Если вы уже знаете, как сериализовать и десериализовать данные в формате `Protobuf`, можете пропустить этот шаг. - - Мы будем использовать Python, чтобы сериализовать некоторые данные в файл `protobuf_message.bin` и затем прочитать их в ClickHouse. - Если вы хотите использовать другой язык, см. также: [«How to read/write length-delimited Protobuf messages in popular languages»](https://cwiki.apache.org/confluence/display/GEODE/Delimiting+Protobuf+Messages). - - Выполните следующую команду, чтобы сгенерировать Python‑файл с именем `schemafile_pb2.py` - в том же каталоге, что и `schemafile.proto`. Этот файл содержит Python‑классы, - представляющие Protobuf‑сообщение `UserData`: - - ```bash - protoc --python_out=. schemafile.proto - ``` - - Теперь создайте новый Python‑файл с именем `generate_protobuf_data.py` в том же - каталоге, что и `schemafile_pb2.py`. Вставьте в него следующий код: - - ```python - import schemafile_pb2 # Модуль, сгенерированный с помощью 'protoc' - from google.protobuf import text_format - from google.protobuf.internal.encoder import _VarintBytes # Импорт внутреннего кодировщика varint - - def create_user_data_message(name, surname, birthDate, phoneNumbers): - """ - Creates and populates a UserData Protobuf message. - """ - message = schemafile_pb2.MessageType() - message.name = name - message.surname = surname - message.birthDate = birthDate - message.phoneNumbers.extend(phoneNumbers) - return message - - # Данные для пользователей в нашем примере - data_to_serialize = [ - {"name": "Aisha", "surname": "Khan", "birthDate": 19920815, "phoneNumbers": ["(555) 247-8903", "(555) 612-3457"]}, - {"name": "Javier", "surname": "Rodriguez", "birthDate": 20001015, "phoneNumbers": ["(555) 891-2046", "(555) 738-5129"]}, - {"name": "Mei", "surname": "Ling", "birthDate": 19980616, "phoneNumbers": ["(555) 956-1834", "(555) 403-7682"]}, - ] - - output_filename = "protobuf_messages.bin" - - # Откройте бинарный файл в двоичном режиме записи ('wb') - with open(output_filename, "wb") as f: - for item in data_to_serialize: - # Создаем экземпляр Protobuf-сообщения для текущего пользователя - message = create_user_data_message( - item["name"], - item["surname"], - item["birthDate"], - item["phoneNumbers"] - ) - - # Сериализуем сообщение - serialized_data = message.SerializeToString() - - # Получаем длину сериализованных данных - message_length = len(serialized_data) - - # Используем внутреннюю функцию _VarintBytes из библиотеки Protobuf для кодирования длины - length_prefix = _VarintBytes(message_length) - - # Записываем префикс длины - f.write(length_prefix) - # Записываем сериализованные данные сообщения - f.write(serialized_data) - - print(f"Protobuf messages (length-delimited) written to {output_filename}") - - # --- Необязательно: проверка (чтение и вывод) --- - # Для обратного чтения мы также используем внутренний декодер varint из Protobuf. - from google.protobuf.internal.decoder import _DecodeVarint32 - - print("\n--- Verifying by reading back ---") - with open(output_filename, "rb") as f: - buf = f.read() # Читаем весь файл в буфер для упрощения декодирования varint - n = 0 - while n < len(buf): - # Декодируем префикс длины в формате varint - msg_len, new_pos = _DecodeVarint32(buf, n) - n = new_pos - - # Извлекаем данные сообщения - message_data = buf[n:n+msg_len] - n += msg_len - - # Разбираем сообщение - decoded_message = schemafile_pb2.MessageType() - decoded_message.ParseFromString(message_data) - print(text_format.MessageToString(decoded_message, as_utf8=True)) - ``` - - Теперь запустите скрипт из командной строки. Рекомендуется запускать его из - виртуального окружения Python, например с использованием `uv`: - - ```bash - uv venv proto-venv - source proto-venv/bin/activate - ``` - - Вам потребуется установить следующую библиотеку Python: - - ```bash - uv pip install --upgrade protobuf - ``` - - Запустите скрипт для генерации бинарного файла: - - ```bash - python generate_protobuf_data.py - ``` -
- -Создайте таблицу ClickHouse, соответствующую этой схеме: - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - - -Вставьте данные в таблицу из командной строки: - -```bash -cat protobuf_messages.bin | clickhouse-client --query "INSERT INTO test.protobuf_messages SETTINGS format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -Также можно записать данные в двоичный файл в формате `Protobuf`: - -```sql -SELECT * FROM test.protobuf_messages INTO OUTFILE 'protobuf_message_from_clickhouse.bin' FORMAT Protobuf SETTINGS format_schema = 'schemafile:MessageType' -``` - -Имея Protobuf-схему, вы теперь можете десериализовать данные, которые ClickHouse записал в файл `protobuf_message_from_clickhouse.bin`. - - -### Чтение и запись данных с использованием ClickHouse Cloud - -В ClickHouse Cloud нельзя загрузить файл схемы Protobuf. Однако вы можете использовать параметр `format_protobuf_schema`, -чтобы указать схему прямо в запросе. В этом примере показано, как читать сериализованные данные с вашей локальной -машины и вставлять их в таблицу в ClickHouse Cloud. - -Как и в предыдущем примере, создайте таблицу в ClickHouse Cloud в соответствии с вашей Protobuf-схемой: - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -Параметр `format_schema_source` определяет источник значения параметра `format_schema`. - -Возможные значения: - -* 'file' (по умолчанию): не поддерживается в Cloud -* 'string': `format_schema` содержит буквальное содержимое схемы. -* 'query': `format_schema` представляет собой запрос для получения схемы. - - -### `format_schema_source='string'` - -Чтобы вставить данные в ClickHouse Cloud, указав схему в виде строки, выполните команду: - -```bash -cat protobuf_messages.bin | clickhouse client --host <имя_хоста> --secure --password <пароль> --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -Выберите данные, добавленные в таблицу: - -```sql -clickhouse client --host --secure --password --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### `format_schema_source='query'` - -Вы также можете хранить Protobuf-схему в таблице. - -Создайте в ClickHouse Cloud таблицу для вставки данных: - -```sql -CREATE TABLE testing.protobuf_schema ( - schema String -) -ENGINE = MergeTree() -ORDER BY tuple(); -``` - -```sql -INSERT INTO testing.protobuf_schema VALUES ('syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};'); -``` - -Вставьте данные в ClickHouse Cloud, указав схему в виде выполняемого запроса: - -```bash -cat protobuf_messages.bin | clickhouse client --host --secure --password --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='SELECT schema FROM testing.protobuf_schema', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -Выведите данные, вставленные в таблицу: - -```sql -clickhouse client --host <имя_хоста> --secure --password <пароль> --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### Использование автоматически сгенерированной схемы - -Если у вас нет внешней Protobuf-схемы для ваших данных, вы всё равно можете выводить и считывать данные в формате Protobuf, используя автоматически сгенерированную схему. Для этого используйте настройку `format_protobuf_use_autogenerated_schema`. - -Например: - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1 -``` - -В этом случае ClickHouse автоматически сгенерирует Protobuf-схему в соответствии со структурой таблицы с помощью функции -[`structureToProtobufSchema`](/sql-reference/functions/other-functions#structureToProtobufSchema). Затем ClickHouse будет использовать эту схему для сериализации данных в формате Protobuf. - -Вы также можете прочитать файл в формате Protobuf с автоматически сгенерированной схемой. В этом случае необходимо, чтобы файл был создан с использованием той же схемы: - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_protobuf_use_autogenerated_schema=1 FORMAT Protobuf" -``` - -Настройка [`format_protobuf_use_autogenerated_schema`](/operations/settings/settings-formats.md#format_protobuf_use_autogenerated_schema) включена по умолчанию и применяется, если [`format_schema`](/operations/settings/formats#format_schema) не задан. - -Вы также можете сохранить автоматически сгенерированную схему в файл при вводе/выводе данных с помощью настройки [`output_format_schema`](/operations/settings/formats#output_format_schema). Например: - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1, output_format_schema='path/to/schema/schema.proto' -``` - -В этом случае автоматически сгенерированная схема Protobuf будет сохранена в файле `path/to/schema/schema.capnp`. - - -### Сброс кэша Protobuf {#basic-examples-cloud} - -Чтобы перезагрузить схему Protobuf, загруженную из [`format_schema_path`](/operations/server-configuration-parameters/settings.md/#format_schema_path), используйте оператор [`SYSTEM DROP ... FORMAT CACHE`](/sql-reference/statements/system.md/#system-drop-schema-format). - -```sql -SYSTEM DROP FORMAT SCHEMA CACHE FOR Protobuf -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md deleted file mode 100644 index 466f1fb3c80..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -alias: [] -description: 'Документация о формате ProtobufList' -input_format: true -keywords: ['ProtobufList'] -output_format: true -slug: /interfaces/formats/ProtobufList -title: 'ProtobufList' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| Ввод | Вывод | Алиас | -| ---- | ----- | ----- | -| ✔ | ✔ | | - - -## Описание {#description} - -Формат `ProtobufList` похож на формат [`Protobuf`](./Protobuf.md), но строки представлены в виде последовательности подсообщений, содержащихся в сообщении с фиксированным именем «Envelope». - - - -## Пример использования {#example-usage} - -Например: - -```sql -SELECT * FROM test.table FORMAT ProtobufList SETTINGS format_schema = 'schemafile:MessageType' -``` - -```bash -cat protobuflist_messages.bin | clickhouse-client --query "INSERT INTO test.table FORMAT ProtobufList SETTINGS format_schema='schemafile:MessageType'" -``` - -Файл `schemafile.proto` имеет следующий вид: - -```capnp title="schemafile.proto" -syntax = "proto3"; -message Envelope { - message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; - }; - MessageType row = 1; -}; -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md deleted file mode 100644 index d7a25bf9db3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'Документация о формате ProtobufSingle' -input_format: true -keywords: ['ProtobufSingle'] -output_format: true -slug: /interfaces/formats/ProtobufSingle -title: 'ProtobufSingle' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -Формат `ProtobufSingle` аналогичен формату [`Protobuf`](./Protobuf.md), но предназначен для хранения и парсинга одиночных Protobuf‑сообщений без префикса длины. - - - -## Пример использования {#example-usage} - - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md deleted file mode 100644 index 10d1de65b9f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'Документация по формату RawBLOB' -keywords: ['RawBLOB'] -slug: /interfaces/formats/RawBLOB -title: 'RawBLOB' -doc_type: 'reference' ---- - - - -## Описание {#description} - -Формат `RawBLOB` считывает все входные данные в одно значение. Можно разобрать только таблицу с одним полем типа [`String`](/sql-reference/data-types/string.md) или аналогичного типа. -Результат выводится в двоичном формате без разделителей и экранирования. Если выводится более одного значения, формат становится неоднозначным, и прочитать данные обратно будет невозможно. - -### Сравнение форматов Raw {#raw-formats-comparison} - -Ниже приведено сравнение форматов `RawBLOB` и [`TabSeparatedRaw`](./TabSeparated/TabSeparatedRaw.md). - -`RawBLOB`: - -* данные выводятся в двоичном формате, без экранирования; -* между значениями нет разделителей; -* в конце каждого значения нет символа новой строки. - -`TabSeparatedRaw`: - -* данные выводятся без экранирования; -* строки содержат значения, разделённые символами табуляции; -* после последнего значения в каждой строке есть перевод строки. - -Ниже приведено сравнение форматов `RawBLOB` и [RowBinary](./RowBinary/RowBinary.md). - -`RawBLOB`: - -* поля типа String выводятся без префикса длины. - -`RowBinary`: - -* поля типа String представляются как длина в формате varint (беззнаковый [LEB128] ([https://en.wikipedia.org/wiki/LEB128](https://en.wikipedia.org/wiki/LEB128))), за которой следуют байты строки. - -Когда на вход `RawBLOB` подаются пустые данные, ClickHouse генерирует исключение: - -```text -Код: 108. DB::Exception: Отсутствуют данные для вставки -``` - - -## Пример использования {#example-usage} - -```bash title="Query" -$ clickhouse-client --query "CREATE TABLE {some_table} (a String) ENGINE = Memory;" -$ cat {filename} | clickhouse-client --query="INSERT INTO {some_table} FORMAT RawBLOB" -$ clickhouse-client --query "SELECT * FROM {some_table} FORMAT RawBLOB" | md5sum -``` - -```text title="Response" -f9725a22f9191e064120d718e26862a9 - -``` - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md deleted file mode 100644 index 41cbed34be1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Regexp' -input_format: true -keywords: ['Regexp'] -output_format: false -slug: /interfaces/formats/Regexp -title: 'Regexp' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## Описание {#description} - -Формат `Regex` разбирает каждую строку импортируемых данных в соответствии с заданным регулярным выражением. - -**Использование** - -Регулярное выражение из настройки [format_regexp](/operations/settings/settings-formats.md/#format_regexp) применяется к каждой строке импортируемых данных. Количество подвыражений (групп) в регулярном выражении должно быть равно количеству столбцов в импортируемом наборе данных. - -Строки импортируемых данных должны быть разделены символом новой строки `'\n'` или переводом строки в стиле DOS `"\r\n"`. - -Содержимое каждого совпавшего подвыражения разбирается способом, соответствующим его типу данных, в соответствии с настройкой [format_regexp_escaping_rule](/operations/settings/settings-formats.md/#format_regexp_escaping_rule). - -Если регулярное выражение не соответствует строке и параметр [format_regexp_skip_unmatched](/operations/settings/settings-formats.md/#format_regexp_escaping_rule) равен 1, строка просто пропускается без ошибки. В противном случае генерируется исключение. - - - -## Пример использования {#example-usage} - -Рассмотрим файл `data.tsv`: - -```text title="data.tsv" -id: 1 array: [1,2,3] string: str1 date: 2020-01-01 -id: 2 array: [1,2,3] string: str2 date: 2020-01-02 -id: 3 array: [1,2,3] string: str3 date: 2020-01-03 -``` - -и таблица `imp_regex_table`: - -```sql -CREATE TABLE imp_regex_table (id UInt32, array Array(UInt32), string String, date Date) ENGINE = Memory; -``` - -Мы вставим данные из вышеупомянутого файла в таблицу, созданную выше, с помощью следующего запроса: - -```bash -$ cat data.tsv | clickhouse-client --query "INSERT INTO imp_regex_table SETTINGS format_regexp='id: (.+?) array: (.+?) string: (.+?) date: (.+?)', format_regexp_escaping_rule='Escaped', format_regexp_skip_unmatched=0 FORMAT Regexp;" -``` - -Теперь мы можем выполнить запрос `SELECT` к таблице, чтобы посмотреть, как формат `Regex` разобрал данные из файла: - -```sql title="Query" -SELECT * FROM imp_regex_table; -``` - -```text title="Response" -┌─id─┬─array───┬─string─┬───────date─┐ -│ 1 │ [1,2,3] │ str1 │ 2020-01-01 │ -│ 2 │ [1,2,3] │ str2 │ 2020-01-02 │ -│ 3 │ [1,2,3] │ str3 │ 2020-01-03 │ -└────┴─────────┴────────┴────────────┘ -``` - - -## Настройки формата {#format-settings} - -При работе с форматом `Regexp` можно использовать следующие настройки: - -- `format_regexp` — [String](/sql-reference/data-types/string.md). Содержит регулярное выражение в синтаксисе [re2](https://github.com/google/re2/wiki/Syntax). -- `format_regexp_escaping_rule` — [String](/sql-reference/data-types/string.md). Поддерживаются следующие правила экранирования: - - - CSV (аналогично [CSV](/interfaces/formats/CSV) - - JSON (аналогично [JSONEachRow](/interfaces/formats/JSONEachRow) - - Escaped (аналогично [TSV](/interfaces/formats/TabSeparated) - - Quoted (аналогично [Values](/interfaces/formats/Values) - - Raw (извлекает подвыражения целиком, без правил экранирования, аналогично [TSVRaw](/interfaces/formats/TabSeparated) - -- `format_regexp_skip_unmatched` — [UInt8](/sql-reference/data-types/int-uint.md). Определяет, нужно ли выбрасывать исключение, если выражение `format_regexp` не соответствует импортируемым данным. Может быть `0` или `1`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md deleted file mode 100644 index 405557f21b8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -alias: [] -description: 'Документация по формату RowBinary' -input_format: true -keywords: ['RowBinary'] -output_format: true -slug: /interfaces/formats/RowBinary -title: 'RowBinary' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -Формат `RowBinary` разбирает данные по строкам в двоичном виде. -Строки и значения идут последовательно, без разделителей. -Поскольку данные представлены в двоичном формате, разделитель после `FORMAT RowBinary` строго задан следующим образом: - -- Произвольное количество пробельных символов: - - `' '` (пробел — код `0x20`) - - `'\t'` (табуляция — код `0x09`) - - `'\f'` (form feed — код `0x0C`) -- После чего следует ровно одна последовательность перевода строки: - - в стиле Windows `"\r\n"` - - или в стиле Unix `'\n'` -- Сразу после этого идут двоичные данные. - -:::note -Этот формат менее эффективен, чем формат [Native](../Native.md), поскольку он построчный. -::: - -Для следующих типов данных важно отметить, что: - -- [Integers](../../../sql-reference/data-types/int-uint.md) используют фиксированное представление в формате little-endian. Например, `UInt64` использует 8 байт. -- [DateTime](../../../sql-reference/data-types/datetime.md) представляется как `UInt32`, содержащее в качестве значения Unix timestamp. -- [Date](../../../sql-reference/data-types/date.md) представляется как значение типа `UInt16`, которое содержит количество дней, прошедших с `1970-01-01`. -- [String](../../../sql-reference/data-types/string.md) представляется как целое число переменной длины (varint) (беззнаковый [`LEB128`](https://en.wikipedia.org/wiki/LEB128)), за которым следуют байты строки. -- [FixedString](../../../sql-reference/data-types/fixedstring.md) представляется просто как последовательность байт. -- [Arrays](../../../sql-reference/data-types/array.md) представляются как целое число переменной длины (varint) (беззнаковый [LEB128](https://en.wikipedia.org/wiki/LEB128)), за которым следуют элементы массива по порядку. - -Для поддержки [NULL](/sql-reference/syntax#null) перед каждым значением типа [Nullable](/sql-reference/data-types/nullable.md) добавляется дополнительный байт, содержащий `1` или `0`. -- Если `1`, то значение — `NULL`, и этот байт интерпретируется как отдельное значение. -- Если `0`, значение после байта не является `NULL`. - -Для сравнения форматов `RowBinary` и `RawBlob` см. раздел: [Raw Formats Comparison](../RawBLOB.md/#raw-formats-comparison) - - - -## Пример использования {#example-usage} - - - -## Параметры формата {#format-settings} - - \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md deleted file mode 100644 index 9be534dffd9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: [] -description: 'Документация по формату RowBinaryWithDefaults' -input_format: true -keywords: ['RowBinaryWithDefaults'] -output_format: false -slug: /interfaces/formats/RowBinaryWithDefaults -title: 'RowBinaryWithDefaults' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✗ | | - - -## Описание {#description} - -Аналогичен формату [`RowBinary`](./RowBinary.md), но с дополнительным байтом перед каждым столбцом, который указывает, следует ли использовать значение по умолчанию. - - - -## Примеры использования {#example-usage} - -Примеры: - -```sql title="Query" -SELECT * FROM FORMAT('RowBinaryWithDefaults', 'x UInt32 default 42, y UInt32', x'010001000000') -``` - -```response title="Response" -┌──x─┬─y─┐ -│ 42 │ 1 │ -└────┴───┘ -``` - -* Для столбца `x` есть только один байт `01`, который указывает, что должно быть использовано значение по умолчанию, и после этого байта не передаётся никаких других данных. -* Для столбца `y` данные начинаются с байта `00`, который указывает, что у столбца есть реальное значение, которое нужно прочитать из следующих данных `01000000`. - - -## Настройки формата {#format-settings} - - diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md deleted file mode 100644 index cf0a78b0730..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'Документация по формату RowBinaryWithNames' -input_format: true -keywords: ['RowBinaryWithNames'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNames -title: 'RowBinaryWithNames' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| Ввод | Вывод | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -Аналогично формату [`RowBinary`](./RowBinary.md), но с добавленным заголовком: - -- Количество столбцов (N), закодированное в формате [`LEB128`](https://en.wikipedia.org/wiki/LEB128). -- N строк (`String`), задающих имена столбцов. - - - -## Пример использования {#example-usage} - - - -## Настройки формата {#format-settings} - - - -:::note -- Если настройка [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) установлена в значение `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены. -- Если настройка [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в значение `1`, в противном случае первая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md deleted file mode 100644 index 8bf8607f078..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -alias: [] -description: 'Документация о формате RowBinaryWithNamesAndTypes' -input_format: true -keywords: ['RowBinaryWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNamesAndTypes -title: 'RowBinaryWithNamesAndTypes' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| Вход | Выход | Псевдоним | -| ---- | ----- | --------- | -| ✔ | ✔ | | - - -## Описание {#description} - -Аналогичен формату [RowBinary](./RowBinary.md), но с добавленным заголовком: - -- Число столбцов (N), закодированное в формате [`LEB128`](https://en.wikipedia.org/wiki/LEB128). -- N значений типа `String` с именами столбцов. -- N значений типа `String` с типами столбцов. - - - -## Пример использования {#example-usage} - - - -## Настройки формата {#format-settings} - - - -:::note -Если настройка [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) установлена в значение `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если настройка [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в значение `1`. -В противном случае первая строка будет пропущена. -Если настройка [`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header) установлена в значение `1`, -типы из входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md deleted file mode 100644 index 4d20eba7daa..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md +++ /dev/null @@ -1,11 +0,0 @@ -{/* Примечание: этот фрагмент повторно используется во всех файлах, в которые он импортируется */ } - -Следующие настройки общие для всех форматов типа `RowBinary`. - -| Setting | Description | Default | -| -------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| [`format_binary_max_string_size`](/operations/settings/settings-formats.md/#format_binary_max_string_size) | Максимально допустимый размер значения типа String в формате RowBinary. | `1GiB` | -| [`output_format_binary_encode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | Позволяет записывать типы в заголовке с использованием [`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) вместо строк с именами типов в формате вывода [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md). | `false` | -| [`input_format_binary_decode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | Позволяет читать типы в заголовке с использованием [`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) вместо строк с именами типов в формате ввода [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md). | `false` | -| [`output_format_binary_write_json_as_string`](/operations/settings/settings-formats.md/#output_format_binary_write_json_as_string) | Позволяет записывать значения типа данных [`JSON`](/sql-reference/data-types/newjson.md) как строковые значения JSON (типа [String](/sql-reference/data-types/string.md)) в формате вывода [`RowBinary`](../RowBinary.md). | `false` | -| [`input_format_binary_read_json_as_string`](/operations/settings/settings-formats.md/#input_format_binary_read_json_as_string) | Позволяет читать значения типа данных [`JSON`](/sql-reference/data-types/newjson.md) как строковые значения JSON (типа [String](/sql-reference/data-types/string.md)) в формате ввода [`RowBinary`](../RowBinary.md). | `false` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md deleted file mode 100644 index a9a66917156..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'Документация по формату SQLInsert' -input_format: false -keywords: ['SQLInsert'] -output_format: true -slug: /interfaces/formats/SQLInsert -title: 'SQLInsert' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Выводит данные в виде последовательности операторов вида `INSERT INTO table (columns...) VALUES (...), (...) ...;`. - - - -## Пример использования {#example-usage} - -Пример: - -```sql -SELECT number AS x, number + 1 AS y, 'Hello' AS z FROM numbers(10) FORMAT SQLInsert SETTINGS output_format_sql_insert_max_batch_size = 2 -``` - -```sql -INSERT INTO table (x, y, z) VALUES (0, 1, 'Привет'), (1, 2, 'Привет'); -INSERT INTO table (x, y, z) VALUES (2, 3, 'Привет'), (3, 4, 'Привет'); -INSERT INTO table (x, y, z) VALUES (4, 5, 'Привет'), (5, 6, 'Привет'); -INSERT INTO table (x, y, z) VALUES (6, 7, 'Привет'), (7, 8, 'Привет'); -INSERT INTO table (x, y, z) VALUES (8, 9, 'Привет'), (9, 10, 'Привет'); -``` - -Для чтения данных, выводимых этим форматом, можно использовать входной формат [MySQLDump](../formats/MySQLDump.md). - - -## Настройки формата {#format-settings} - -| Setting | Description | Default | -|----------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------|-----------| -| [`output_format_sql_insert_max_batch_size`](../../operations/settings/settings-formats.md/#output_format_sql_insert_max_batch_size) | Максимальное количество строк в одном операторе INSERT. | `65505` | -| [`output_format_sql_insert_table_name`](../../operations/settings/settings-formats.md/#output_format_sql_insert_table_name) | Имя таблицы в результирующем операторе INSERT. | `'table'` | -| [`output_format_sql_insert_include_column_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_include_column_names) | Включать имена столбцов в оператор INSERT. | `true` | -| [`output_format_sql_insert_use_replace`](../../operations/settings/settings-formats.md/#output_format_sql_insert_use_replace) | Использовать оператор REPLACE вместо INSERT. | `false` | -| [`output_format_sql_insert_quote_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_quote_names) | Заключать имена столбцов в символы «\`». | `true` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md deleted file mode 100644 index cbb4332f4b9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -alias: [] -description: 'Документация по формату TSKV' -input_format: true -keywords: ['TSKV'] -output_format: true -slug: /interfaces/formats/TSKV -title: 'TSKV' -doc_type: 'reference' ---- - -| Входной формат | Выходной формат | Псевдоним | -|----------------|-----------------|-----------| -| ✔ | ✔ | | - - - -## Описание {#description} - -Аналогичен формату [`TabSeparated`](./TabSeparated.md), но выводит значение в формате `name=value`. -Имена экранируются так же, как в формате [`TabSeparated`](./TabSeparated.md), и символ `=` также экранируется. - -```text -SearchPhrase= count()=8267016 -SearchPhrase=дизайн интерьера ванной комнаты count()=2166 -SearchPhrase=clickhouse count()=1655 -SearchPhrase=весенняя мода 2014 count()=1549 -SearchPhrase=фотографии в свободном стиле count()=1480 -SearchPhrase=анджелина джоли count()=1245 -SearchPhrase=омск count()=1112 -SearchPhrase=фотографии пород собак count()=1091 -SearchPhrase=дизайн штор count()=1064 -SearchPhrase=баку count()=1000 -``` - -```sql title="Query" -SELECT * FROM t_null FORMAT TSKV -``` - -```text title="Response" -x=1 y=\N -``` - -:::note -При большом количестве небольших столбцов этот формат неэффективен и, как правило, не имеет смысла его использовать. -Тем не менее по эффективности он не уступает формату [`JSONEachRow`](../JSON/JSONEachRow.md). -::: - -При разборе поддерживается любой порядок значений различных столбцов. -Допускается опускать некоторые значения, так как они считаются эквивалентными значениям по умолчанию. -В этом случае в качестве значений по умолчанию используются нули и пустые строки. -Сложные значения, которые могли бы быть заданы в таблице, не поддерживаются в качестве значений по умолчанию. - -При разборе допускается добавление дополнительного поля `tskv` без знака равенства или значения. Это поле игнорируется. - -При импорте столбцы с неизвестными именами будут пропущены, -если параметр [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлен в `1`. - -[NULL](/sql-reference/syntax.md) форматируется как `\N`. - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий файл в формате TSKV с именем `football.tskv`: - -```tsv -date=2022-04-30 season=2021 home_team=Саттон Юнайтед away_team=Брэдфорд Сити home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=Свиндон Таун away_team=Барроу home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=Транмер Роверс away_team=Олдэм Атлетик home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=Порт Вэйл away_team=Ньюпорт Каунти home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=Сэлфорд Сити away_team=Мансфилд Таун home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Барроу away_team=Нортгемптон Таун home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Брэдфорд Сити away_team=Карлайл Юнайтед home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Бристоль Роверс away_team=Сканторп Юнайтед home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Эксетер Сити away_team=Порт Вэйл home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Харрогейт Таун A.F.C. away_team=Саттон Юнайтед home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Хартлпул Юнайтед away_team=Колчестер Юнайтед home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Лейтон Орриент away_team=Транмер Роверс home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Мансфилд Таун away_team=Форест Грин Роверс home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Ньюпорт Каунти away_team=Рочдейл home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Олдэм Атлетик away_team=Кроули Таун home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Стивенэйдж Боро away_team=Сэлфорд Сити home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Уолсолл away_team=Свиндон Таун home_team_goals=0 away_team_goals=3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tskv' FORMAT TSKV; -``` - -### Чтение данных {#reading-data} - -Считайте данные в формате `TSKV`: - -```sql -SELECT * -FROM football -FORMAT TSKV -``` - -Результат будет в табличном формате с разделителем табуляцией и двумя строками заголовков для названий столбцов и их типов: - - -```tsv -date=2022-04-30 season=2021 home_team=Sutton United away_team=Bradford City home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=Swindon Town away_team=Barrow home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=Tranmere Rovers away_team=Oldham Athletic home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=Port Vale away_team=Newport County home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=Salford City away_team=Mansfield Town home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Barrow away_team=Northampton Town home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Bradford City away_team=Carlisle United home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Bristol Rovers away_team=Scunthorpe United home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=Exeter City away_team=Port Vale home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Harrogate Town A.F.C. away_team=Sutton United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Hartlepool United away_team=Colchester United home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Leyton Orient away_team=Tranmere Rovers home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=Mansfield Town away_team=Forest Green Rovers home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Newport County away_team=Rochdale home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Oldham Athletic away_team=Crawley Town home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=Stevenage Borough away_team=Salford City home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=Walsall away_team=Swindon Town home_team_goals=0 away_team_goals=3 -``` - - -## Настройки форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md deleted file mode 100644 index 51c9261eb3b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md +++ /dev/null @@ -1,190 +0,0 @@ ---- -alias: ['TSV'] -description: 'Документация по формату TSV' -input_format: true -keywords: ['TabSeparated', 'TSV'] -output_format: true -slug: /interfaces/formats/TabSeparated -title: 'TabSeparated' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|--------| -| ✔ | ✔ | `TSV` | - - - -## Описание {#description} - -В формате TabSeparated данные записываются построчно. Каждая строка содержит значения, разделённые символами табуляции. После каждого значения следует символ табуляции, за исключением последнего значения в строке, за которым следует символ перевода строки. Везде предполагается использование перевода строки в формате Unix. Последняя строка также должна заканчиваться переводом строки. Значения записываются в текстовом формате, без заключения в кавычки и с экранированием специальных символов. - -Этот формат также доступен под названием `TSV`. - -Формат `TabSeparated` удобен для обработки данных пользовательскими программами и скриптами. Он используется по умолчанию в HTTP-интерфейсе и в пакетном режиме командного клиента. Этот формат также позволяет переносить данные между различными СУБД. Например, можно сделать дамп из MySQL и загрузить его в ClickHouse или наоборот. - -Формат `TabSeparated` поддерживает вывод итоговых значений (при использовании WITH TOTALS) и экстремальных значений (когда параметр `extremes` установлен в 1). В этих случаях итоговые значения и экстремальные значения выводятся после основных данных. Основной результат, итоговые значения и экстремальные значения разделяются между собой пустой строкой. Пример: - -```sql -SELECT EventDate, count() AS c FROM test.hits GROUP BY EventDate WITH TOTALS ORDER BY EventDate FORMAT TabSeparated - -2014-03-17 1406958 -2014-03-18 1383658 -2014-03-19 1405797 -2014-03-20 1353623 -2014-03-21 1245779 -2014-03-22 1031592 -2014-03-23 1046491 - -1970-01-01 8873898 - -2014-03-17 1031592 -2014-03-23 1406958 -``` - - -## Форматирование данных {#tabseparated-data-formatting} - -Целые числа записываются в десятичной форме. Числа могут содержать дополнительный символ "+" в начале (он игнорируется при разборе и не записывается при форматировании). Неотрицательные числа не могут содержать знак минус. При чтении допускается интерпретировать пустую строку как ноль или (для знаковых типов) строку, состоящую только из знака минус, как ноль. Числа, которые не помещаются в соответствующий тип данных, могут быть разобраны как другое число, без сообщения об ошибке. - -Числа с плавающей запятой записываются в десятичной форме. В качестве десятичного разделителя используется точка. Поддерживается экспоненциальная форма записи, а также значения 'inf', '+inf', '-inf' и 'nan'. Запись числа с плавающей запятой может начинаться или заканчиваться десятичной точкой. -При форматировании точность чисел с плавающей запятой может быть потеряна. -При разборе не обязательно считывать ближайшее к значению число, представимое в формате машины. - -Даты записываются в формате YYYY-MM-DD и разбираются в том же формате, но с любыми символами в качестве разделителей. -Дата и время записываются в формате `YYYY-MM-DD hh:mm:ss` и разбираются в том же формате, но с любыми символами в качестве разделителей. -Все это происходит в системной временной зоне на момент запуска клиента или сервера (в зависимости от того, кто форматирует данные). Для дат со временем переход на летнее время не указывается. Поэтому, если дамп содержит время в период действия летнего времени, дамп однозначно не соответствует данным, и при разборе будет выбран один из двух возможных моментов времени. -При чтении некорректные даты и даты со временем могут быть разобраны с естественным переполнением или как нулевые дата и время, без сообщения об ошибке. - -В качестве исключения разбор дат со временем также поддерживается в формате Unix timestamp, если он состоит ровно из 10 десятичных цифр. Результат не зависит от часового пояса. Форматы `YYYY-MM-DD hh:mm:ss` и `NNNNNNNNNN` различаются автоматически. - -Строки выводятся с экранированием специальных символов обратной косой чертой. Для вывода используются следующие escape-последовательности: `\b`, `\f`, `\r`, `\n`, `\t`, `\0`, `\'`, `\\`. При разборе также поддерживаются последовательности `\a`, `\v` и `\xHH` (шестнадцатеричные escape-последовательности), а также любые последовательности `\c`, где `c` — любой символ (эти последовательности преобразуются в `c`). Таким образом, при чтении данных поддерживаются форматы, в которых перевод строки может быть записан как `\n`, как `\` или как собственно символ перевода строки. Например, строка `Hello world` с переводом строки между словами вместо пробела может быть разобрана в любом из следующих вариантов: - -```text -Привет\nмир - -Привет\ -мир -``` - -Второй вариант поддерживается, потому что MySQL использует его при записи дампов с разделителем табуляции. - -Минимальный набор символов, которые необходимо экранировать при передаче данных в формате TabSeparated: табуляция, перевод строки (LF) и обратная косая черта. - -Экранируется только небольшой набор символов. Вы легко можете наткнуться на строковое значение, которое ваш терминал исказит при выводе. - -Массивы записываются как список значений, разделённых запятыми, в квадратных скобках. Числовые элементы в массиве форматируются как обычно. Типы `Date` и `DateTime` записываются в одинарных кавычках. Строки записываются в одинарных кавычках с теми же правилами экранирования, что и выше. - -[NULL](/sql-reference/syntax.md) форматируется в соответствии с настройкой [format_tsv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation) (значение по умолчанию — `\N`). - -Во входных данных значения ENUM могут быть представлены как именами, так и идентификаторами. Сначала предпринимается попытка сопоставить входное значение с именем ENUM. Если это не удаётся и входное значение является числом, выполняется попытка сопоставить это число идентификатору ENUM. -Если входные данные содержат только идентификаторы ENUM, рекомендуется включить настройку [input_format_tsv_enum_as_number](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) для оптимизации разбора ENUM. - -Каждый элемент структур [Nested](/sql-reference/data-types/nested-data-structures/index.md) представляется в виде массива. - -Например: - -```sql -CREATE TABLE nestedt -( - `id` UInt8, - `aux` Nested( - a UInt8, - b String - ) -) -ENGINE = TinyLog -``` - -```sql -INSERT INTO nestedt VALUES ( 1, [1], ['a']) -``` - -```sql -SELECT * FROM nestedt FORMAT TSV -``` - -```response -1 [1] ['a'] -``` - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используя следующий TSV-файл с именем `football.tsv`: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparated; -``` - -### Чтение данных {#reading-data} - -Считывайте данные в формате `TabSeparated`: - -```sql -SELECT * -FROM football -FORMAT TabSeparated -``` - -Результат будет выведен в формате с разделителями табуляции: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Настройки формата {#format-settings} - -| Setting | Description | Default | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| -| [`format_tsv_null_representation`](/operations/settings/settings-formats.md/#format_tsv_null_representation) | Пользовательское представление значения NULL в формате TSV. | `\N` | -| [`input_format_tsv_empty_as_default`](/operations/settings/settings-formats.md/#input_format_tsv_empty_as_default) | трактовать пустые поля во входных данных TSV как значения по умолчанию. Для сложных выражений по умолчанию параметр [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) также должен быть включён. | `false` | -| [`input_format_tsv_enum_as_number`](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) | трактовать вставляемые значения enum в форматах TSV как индексы enum. | `false` | -| [`input_format_tsv_use_best_effort_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_tsv_use_best_effort_in_schema_inference) | использовать дополнительные эвристики для определения схемы в формате TSV. Если параметр отключён, все поля будут интерпретированы как String. | `true` | -| [`output_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#output_format_tsv_crlf_end_of_line) | если значение параметра равно true, конец строки в выходном формате TSV будет `\r\n` вместо `\n`. | `false` | -| [`input_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#input_format_tsv_crlf_end_of_line) | если значение параметра равно true, конец строки во входном формате TSV будет `\r\n` вместо `\n`. | `false` | -| [`input_format_tsv_skip_first_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_first_lines) | пропускать указанное количество строк в начале данных. | `0` | -| [`input_format_tsv_detect_header`](/operations/settings/settings-formats.md/#input_format_tsv_detect_header) | автоматически определять заголовок с именами и типами в формате TSV. | `true` | -| [`input_format_tsv_skip_trailing_empty_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_trailing_empty_lines) | пропускать завершающие пустые строки в конце данных. | `false` | -| [`input_format_tsv_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_tsv_allow_variable_number_of_columns) | разрешить переменное количество столбцов в формате TSV, игнорировать лишние столбцы и использовать значения по умолчанию для отсутствующих столбцов. | `false` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md deleted file mode 100644 index c921daf384d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: ['TSVRaw', 'Raw'] -description: 'Документация по формату TabSeparatedRaw' -input_format: true -keywords: ['TabSeparatedRaw'] -output_format: true -slug: /interfaces/formats/TabSeparatedRaw -title: 'TabSeparatedRaw' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------------| -| ✔ | ✔ | `TSVRaw`, `Raw` | - - - -## Описание {#description} - -Отличается от формата [`TabSeparated`](/interfaces/formats/TabSeparated) тем, что строки записываются без экранирования. - -:::note -При разборе этого формата символы табуляции и перевода строки внутри каждого поля не допускаются. -::: - -Сравнение форматов `TabSeparatedRaw` и `RawBlob` см. в разделе [Сравнение форматов Raw](../RawBLOB.md/#raw-formats-comparison). - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий TSV-файл с именем `football.tsv`: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRaw; -``` - -### Чтение данных {#reading-data} - -Считайте данные в формате `TabSeparatedRaw`: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRaw -``` - -Вывод будет в формате с разделителями-табуляциями: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Настройки формата {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md deleted file mode 100644 index b8d3a48fa05..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -alias: ['TSVRawWithNames', 'RawWithNames'] -description: 'Документация по формату TabSeparatedRawWithNames' -input_format: true -keywords: ['TabSeparatedRawWithNames', 'TSVRawWithNames', 'RawWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNames -title: 'TabSeparatedRawWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдонимы | -|------|-------|-----------------------------------| -| ✔ | ✔ | `TSVRawWithNames`, `RawWithNames` | - - - -## Описание {#description} - -Отличается от формата [`TabSeparatedWithNames`](./TabSeparatedWithNames.md) тем, -что строки записываются без экранирования. - -:::note -При разборе данных в этом формате символы табуляции или перевода строки внутри отдельных полей не допускаются. -::: - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий TSV-файл с именем `football.tsv`: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Добавьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNames; -``` - -### Чтение данных {#reading-data} - -Считайте данные в формате `TabSeparatedRawWithNames`: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNames -``` - -Вывод будет в табличном формате с разделителем-символом табуляции и одной строкой заголовков: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md deleted file mode 100644 index 0b66aa7b6da..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -alias: ['TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -description: 'Документация по формату TabSeparatedRawWithNamesAndTypes' -input_format: true -keywords: ['TabSeparatedRawWithNamesAndTypes', 'TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNamesAndTypes -title: 'TabSeparatedRawWithNamesAndTypes' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|------|-------|---------------------------------------------------| -| ✔ | ✔ | `TSVRawWithNamesAndNames`, `RawWithNamesAndNames` | - - - -## Описание {#description} - -Отличается от формата [`TabSeparatedWithNamesAndTypes`](./TabSeparatedWithNamesAndTypes.md) -тем, что строки записываются без экранирования. - -:::note -При разборе этого формата табуляция и перевод строки внутри каждого поля не допускаются. -::: - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий TSV‑файл с именем `football.tsv`: - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNamesAndTypes; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные, используя формат `TabSeparatedRawWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNamesAndTypes -``` - -Вывод будет в формате с разделителем табуляции и двумя строками заголовка: первая содержит имена столбцов, вторая — их типы: - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md deleted file mode 100644 index 17ad9b2e978..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -alias: ['TSVWithNames'] -description: 'Документация по формату TabSeparatedWithNames' -input_format: true -keywords: ['TabSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedWithNames -title: 'TabSeparatedWithNames' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдонимы | -|-------|--------|--------------------------------| -| ✔ | ✔ | `TSVWithNames`, `RawWithNames` | - - - -## Описание {#description} - -Отличается от формата [`TabSeparated`](./TabSeparated.md) тем, что имена столбцов записаны в первой строке. - -При разборе ожидается, что первая строка содержит имена столбцов. Вы можете использовать имена столбцов, чтобы определить их положение и проверить их корректность. - -:::note -Если параметр [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) установлен в значение `1`, -столбцы из входных данных будут сопоставлены со столбцами таблицы по их именам, а столбцы с неизвестными именами будут пропущены, если параметр [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлен в значение `1`. -В противном случае первая строка будет пропущена. -::: - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий TSV-файл с именем `football.tsv`: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNames; -``` - -### Чтение данных {#reading-data} - -Считайте данные в формате `TabSeparatedWithNames`: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNames -``` - -Вывод будет в табличном формате с разделителем табуляцией: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Настройки формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md deleted file mode 100644 index 31bb1ad9a45..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: 'Документация по формату TabSeparatedWithNamesAndTypes' -keywords: ['TabSeparatedWithNamesAndTypes'] -slug: /interfaces/formats/TabSeparatedWithNamesAndTypes -title: 'TabSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| Входные данные | Выходные данные | Псевдоним | -|----------------|-----------------|-----------------------------------------------| -| ✔ | ✔ | `TSVWithNamesAndTypes`, `RawWithNamesAndTypes` | - - - -## Описание {#description} - -Отличается от формата [`TabSeparated`](./TabSeparated.md) тем, что имена столбцов записываются в первую строку, а типы столбцов — во вторую. - -:::note -- Если настройка [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) установлена в значение `1`, -столбцы из входных данных будут сопоставляться со столбцами в таблице по их именам, а столбцы с неизвестными именами будут пропущены, если настройка [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) установлена в значение `1`. -В противном случае первая строка будет пропущена. -- Если настройка [`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) установлена в значение `1`, -типы из входных данных будут сравниваться с типами соответствующих столбцов таблицы. В противном случае вторая строка будет пропущена. -::: - - - -## Пример использования {#example-usage} - -### Вставка данных {#inserting-data} - -Используем следующий TSV-файл с именем `football.tsv`: - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -Вставьте данные: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNamesAndTypes; -``` - -### Чтение данных {#reading-data} - -Прочитайте данные в формате `TabSeparatedWithNamesAndTypes`: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNamesAndTypes -``` - -Вывод будет в формате с разделителями табуляции и двумя строками заголовков для имен столбцов и их типов: - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## Параметры формата {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md deleted file mode 100644 index 912a34bfe14..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md +++ /dev/null @@ -1,248 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Template' -input_format: true -keywords: ['Template'] -output_format: true -slug: /interfaces/formats/Template -title: 'Template' -doc_type: 'guide' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -В случаях, когда вам требуется больше возможностей для настройки, чем предоставляют другие стандартные форматы, -формат `Template` позволяет задать собственную строку формата с заполнителями для значений -и указать правила экранирования данных. - -Используются следующие настройки: - -| Setting | Description | -|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------| -| [`format_template_row`](#format_template_row) | Указывает путь к файлу, который содержит строки формата для строк. | -| [`format_template_resultset`](#format_template_resultset) | Указывает путь к файлу, который содержит строки формата для наборов результатов. | -| [`format_template_rows_between_delimiter`](#format_template_rows_between_delimiter) | Указывает разделитель между строками, который выводится (или ожидается) после каждой строки, кроме последней (`\n` по умолчанию). | -| `format_template_row_format` | Указывает строку формата для строк [во встроенной спецификации](#inline_specification). | -| `format_template_resultset_format` | Указывает строку формата для набора результатов [во встроенной спецификации](#inline_specification). | -| Некоторые настройки других форматов (например, `output_format_json_quote_64bit_integers` при использовании экранирования `JSON` | | - - - -## Настройки и правила экранирования {#settings-and-escaping-rules} - -### format_template_row {#format_template_row} - -Настройка `format_template_row` задаёт путь к файлу, который содержит шаблоны формата для строк со следующим синтаксисом: - -```text -разделитель_1${столбец_1:формат_сериализации_1}разделитель_2${столбец_2:формат_сериализации_2} ... разделитель_N -``` - -Где: - -| Часть синтаксиса | Описание | -| ---------------- | ---------------------------------------------------------------------------------------------------------------- | -| `delimiter_i` | Разделитель между значениями (символ `$` можно экранировать как `$$`) | -| `column_i` | Имя или индекс столбца, значения которого должны быть выбраны или вставлены (если пусто, столбец будет пропущен) | -| `serializeAs_i` | Правило экранирования для значений столбца. | - -Поддерживаются следующие правила экранирования: - -| Правило экранирования | Описание | -| --------------------- | ----------------------------------------------- | -| `CSV`, `JSON`, `XML` | Аналогично форматам с такими же названиями | -| `Escaped` | Аналогично `TSV` | -| `Quoted` | Аналогично `Values` | -| `Raw` | Без экранирования, аналогично `TSVRaw` | -| `None` | Без правила экранирования — см. примечание ниже | - -:::note -Если правило экранирования не указано, будет использовано `None`. `XML` подходит только для вывода. -::: - -Рассмотрим пример. Пусть задана следующая строка формата: - -```text -Поисковая фраза: ${s:Quoted}, количество: ${c:Escaped}, цена рекламы: $$${p:JSON}; -``` - -Следующие значения будут выведены (при использовании `SELECT`) или ожидаются (при использовании `INPUT`), -расположенные соответственно между разделителями `Search phrase:`, `, count:`, `, ad price: $` и `;`: - -* `s` (с правилом экранирования `Quoted`) -* `c` (с правилом экранирования `Escaped`) -* `p` (с правилом экранирования `JSON`) - -Например: - -* При выполнении `INSERT` строка ниже соответствует ожидаемому шаблону, и значения `bathroom interior design`, `2166`, `$3` будут записаны в столбцы `Search phrase`, `count`, `ad price`. -* При выполнении `SELECT` строка ниже является результатом вывода, при условии, что значения `bathroom interior design`, `2166`, `$3` уже сохранены в таблице в столбцах `Search phrase`, `count`, `ad price`. - -```yaml -Поисковая фраза: 'дизайн интерьера ванной комнаты', количество: 2166, цена объявления: $3; -``` - -### format_template_rows_between_delimiter {#format_template_rows_between_delimiter} - -Параметр `format_template_rows_between_delimiter` задаёт разделитель между строками, который выводится (или ожидается) после каждой строки, кроме последней (`\n` по умолчанию). - -### format_template_resultset {#format_template_resultset} - -Параметр `format_template_resultset` указывает путь к файлу, содержащему форматную строку для результирующего набора данных. - -Форматная строка для результирующего набора данных имеет тот же синтаксис, что и форматная строка для строк. -Она позволяет задать префикс, суффикс и способ вывода дополнительной информации и содержит следующие плейсхолдеры вместо имён столбцов: - -* `data` — строки с данными в формате `format_template_row`, разделённые `format_template_rows_between_delimiter`. Этот плейсхолдер должен быть первым плейсхолдером в форматной строке. -* `totals` — строка с итоговыми значениями в формате `format_template_row` (при использовании WITH TOTALS). -* `min` — строка с минимальными значениями в формате `format_template_row` (когда `extremes` установлено в 1). -* `max` — строка с максимальными значениями в формате `format_template_row` (когда `extremes` установлено в 1). -* `rows` — общее количество выводимых строк. -* `rows_before_limit` — минимальное количество строк, которое было бы без LIMIT. Выводится только если запрос содержит LIMIT. Если запрос содержит GROUP BY, `rows_before_limit_at_least` — это точное количество строк, которое было бы без LIMIT. -* `time` — время выполнения запроса в секундах. -* `rows_read` — количество прочитанных строк. -* `bytes_read` — количество прочитанных байт (в несжатом виде). - -Для плейсхолдеров `data`, `totals`, `min` и `max` не должно быть задано правило экранирования (или явно должно быть указано `None`). Для остальных плейсхолдеров может быть задано любое правило экранирования. - -:::note -Если параметр `format_template_resultset` является пустой строкой, по умолчанию используется `${data}`. -::: - - -Для запросов INSERT формат позволяет пропускать некоторые столбцы или поля, если задан префикс или суффикс (см. пример). - -### Встроенная спецификация {#inline_specification} - -Часто бывает сложно или невозможно развернуть конфигурации формата -(заданные `format_template_row`, `format_template_resultset`) для шаблонного формата в каталог на всех узлах кластера. -Кроме того, формат может быть настолько тривиальным, что его не требуется выносить в отдельный файл. - -Для таких случаев `format_template_row_format` (для `format_template_row`) и `format_template_resultset_format` (для `format_template_resultset`) можно использовать для задания строки шаблона непосредственно в запросе, -а не как путь к файлу, в котором она содержится. - -:::note -Правила для строк формата и управляющих последовательностей такие же, как и для: -- [`format_template_row`](#format_template_row) при использовании `format_template_row_format`. -- [`format_template_resultset`](#format_template_resultset) при использовании `format_template_resultset_format`. -::: - - - -## Пример использования {#example-usage} - -Рассмотрим два примера того, как можно использовать формат `Template`: сначала для выборки данных, а затем для вставки данных. - -### Выборка данных {#selecting-data} - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase ORDER BY c DESC LIMIT 5 FORMAT Template SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format', format_template_rows_between_delimiter = '\n ' -``` - -```text title="/some/path/resultset.format" - - Поисковые фразы - - - - ${data} -
Поисковые фразы
Поисковая фраза Количество
- - ${max} -
Максимум
- Обработано ${rows_read:XML} строк за ${time:XML} сек. - - -``` - -```text title="/some/path/row.format" - ${0:XML} ${1:XML} -``` - -Результат: - -```html - - Поисковые фразы - - - - - - - - -
Поисковые фразы
Поисковая фраза Количество
8267016
дизайн интерьера ванной 2166
clickhouse 1655
мода весна 2014 1549
произвольные фотографии 1480
- - -
Максимум
8873898
- Обработано 3095973 строк за 0,1569913 сек. - - -``` - -### Вставка данных {#inserting-data} - -```text -Заголовок -Просмотры страниц: 5, ID пользователя: 4324182021466249494, Бесполезное поле: hello, Длительность: 146, Знак: -1 -Просмотры страниц: 6, ID пользователя: 4324182021466249494, Бесполезное поле: world, Длительность: 185, Знак: 1 -Всего строк: 2 -``` - -```sql -INSERT INTO UserActivity SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format' -FORMAT Template -``` - -```text title="/some/path/resultset.format" -Некоторый заголовок\n${data}\nВсего строк: ${:CSV}\n -``` - -```text title="/some/path/row.format" -Просмотры страниц: ${PageViews:CSV}, Идентификатор пользователя: ${UserID:CSV}, Неиспользуемое поле: ${:CSV}, Продолжительность: ${Duration:CSV}, Знак: ${Sign:CSV} -``` - -`PageViews`, `UserID`, `Duration` и `Sign` внутри плейсхолдеров — это имена столбцов в таблице. Значения после `Useless field` в строках и после `\nTotal rows:` в суффиксе будут игнорироваться. -Все разделители во входных данных должны в точности совпадать с разделителями в указанных строках формата. - -### Встроенная спецификация {#in-line-specification} - -Устали вручную форматировать таблицы Markdown? В этом примере мы рассмотрим, как можно использовать формат `Template` и настройки встроенной спецификации, чтобы решить простую задачу — выполнить `SELECT` по именам некоторых форматов ClickHouse из таблицы `system.formats` и отформатировать их как таблицу в формате Markdown. Это можно легко сделать, используя формат `Template` и настройки `format_template_row_format` и `format_template_resultset_format`. - - -В предыдущих примерах мы указывали строки шаблонов для результирующего набора и строк в отдельных файлах, а пути к этим файлам задавали с помощью настроек `format_template_resultset` и `format_template_row` соответственно. Здесь мы сделаем это прямо в запросе, потому что наш шаблон тривиален и состоит лишь из нескольких символов `|` и `-` для создания таблицы в формате Markdown. Шаблонную строку для результирующего набора мы зададим с помощью настройки `format_template_resultset_format`. Чтобы сделать заголовок таблицы, мы добавили `|ClickHouse Formats|\n|---|\n` перед `${data}`. Настройку `format_template_row_format` мы используем, чтобы задать шаблонную строку ``|`{0:XML}`|`` для наших строк. Формат `Template` вставит наши строки с заданным форматом в плейсхолдер `${data}`. В этом примере у нас только один столбец, но при необходимости вы можете добавить больше, добавив `{1:XML}`, `{2:XML}` и т. д. в шаблон строки, выбирая правило экранирования по необходимости. В этом примере мы используем правило экранирования `XML`. - -```sql title="Query" -WITH formats AS -( - SELECT * FROM system.formats - ORDER BY rand() - LIMIT 5 -) -SELECT * FROM formats -FORMAT Template -SETTINGS - format_template_row_format='|`${0:XML}`|', - format_template_resultset_format='|Форматы ClickHouse|\n|---|\n${data}\n' -``` - -Посмотрите-ка! Мы избавили себя от необходимости вручную добавлять все эти `|` и `-`, чтобы сделать эту markdown-таблицу: - -```response title="Response" -|Форматы ClickHouse| -|---| -|`BSONEachRow`| -|`CustomSeparatedWithNames`| -|`Prometheus`| -|`DWARF`| -|`Avro`| -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md deleted file mode 100644 index 0193294a52d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -alias: [] -description: 'Документация по формату TemplateIgnoreSpaces' -input_format: true -keywords: ['TemplateIgnoreSpaces'] -output_format: false -slug: /interfaces/formats/TemplateIgnoreSpaces -title: 'TemplateIgnoreSpaces' -doc_type: 'reference' ---- - -| Вход | Выход | Алиас | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## Описание {#description} - -Аналогично формату [`Template`], но пропускает пробельные символы между разделителями и значениями во входном потоке. -Однако если в строках формата присутствуют пробельные символы, эти символы будут ожидаться во входном потоке. -Также позволяет указывать пустые плейсхолдеры (`${}` или `${:None}`), чтобы разделить один разделитель на отдельные части и игнорировать пробелы между ними. -Такие плейсхолдеры используются только для пропуска пробельных символов. -Можно читать данные в формате `JSON` с использованием этого формата, если значения столбцов имеют один и тот же порядок во всех строках. - -:::note -Этот формат предназначен только для ввода. -::: - - - -## Пример использования {#example-usage} - -Следующий запрос можно использовать для вставки данных из приведённого выше примера вывода в формате [JSON](/interfaces/formats/JSON): - -```sql -INSERT INTO table_name -SETTINGS - format_template_resultset = '/some/path/resultset.format', - format_template_row = '/some/path/row.format', - format_template_rows_between_delimiter = ',' -FORMAT TemplateIgnoreSpaces -``` - -```text title="/some/path/resultset.format" -{${}"meta"${}:${:JSON},${}"data"${}:${}[${data}]${},${}"totals"${}:${:JSON},${}"extremes"${}:${:JSON},${}"rows"${}:${:JSON},${}"rows_before_limit_at_least"${}:${:JSON}${}} -``` - -```text title="/some/path/row.format" -{${}"SearchPhrase"${}:${}${phrase:JSON}${},${}"c"${}:${}${cnt:JSON}${}} -``` - - -## Параметры форматирования {#format-settings} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md deleted file mode 100644 index e7616aa0243..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Values' -input_format: true -keywords: ['Values'] -output_format: true -slug: /interfaces/formats/Values -title: 'Values' -doc_type: 'guide' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Описание {#description} - -Формат `Values` выводит каждую строку в скобках. - -- Строки разделяются запятыми, без запятой после последней строки. -- Значения внутри скобок также разделяются запятыми. -- Числа выводятся в десятичном формате без кавычек. -- Массивы выводятся в квадратных скобках. -- Строки, даты и даты со временем выводятся в кавычках. -- Правила экранирования и разбора аналогичны формату [TabSeparated](TabSeparated/TabSeparated.md). - -При форматировании лишние пробелы не вставляются, но при разборе они допускаются и пропускаются (за исключением пробелов внутри значений массивов, которые не допускаются). -[`NULL`](/sql-reference/syntax.md) обозначается как `NULL`. - -Минимальный набор символов, которые необходимо экранировать при передаче данных в формате `Values`: -- одинарные кавычки -- обратные косые черты - -Это формат, который используется в `INSERT INTO t VALUES ...`, но вы также можете использовать его для форматирования результатов запроса. - - - -## Пример использования {#example-usage} - - - -## Настройки формата {#format-settings} - -| Настройка | Описание | Значение по умолчанию | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------| -| [`input_format_values_interpret_expressions`](../../operations/settings/settings-formats.md/#input_format_values_interpret_expressions) | если поле не удалось разобрать потоковым парсером, запустить SQL-парсер и попытаться интерпретировать его как SQL-выражение. | `true` | -| [`input_format_values_deduce_templates_of_expressions`](../../operations/settings/settings-formats.md/#input_format_values_deduce_templates_of_expressions) | если поле не удалось разобрать потоковым парсером, запустить SQL-парсер, определить шаблон SQL-выражения, попытаться разобрать все строки, используя этот шаблон, а затем интерпретировать выражение для всех строк. | `true` | -| [`input_format_values_accurate_types_of_literals`](../../operations/settings/settings-formats.md/#input_format_values_accurate_types_of_literals) | при разборе и интерпретации выражений с использованием шаблона проверять фактический тип литерала, чтобы избежать возможных переполнений и проблем с точностью. | `true` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md deleted file mode 100644 index 9af798e26ee..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -alias: [] -description: 'Документация по формату Vertical' -input_format: false -keywords: ['Vertical'] -output_format: true -slug: /interfaces/formats/Vertical -title: 'Vertical' -doc_type: 'reference' ---- - -| Ввод | Вывод | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Выводит каждое значение на отдельной строке с указанием имени столбца. Этот формат удобен для вывода одной или нескольких строк, если каждая строка содержит большое количество столбцов. - -Обратите внимание, что [`NULL`](/sql-reference/syntax.md) выводится как `ᴺᵁᴸᴸ`, чтобы было проще отличать строковое значение `NULL` от отсутствия значения. Столбцы JSON выводятся в удобочитаемом формате, а `NULL` выводится как `null`, поскольку это корректное значение JSON и его легко отличить от `"null"`. - - - -## Пример использования {#example-usage} - -Пример: - -```sql -SELECT * FROM t_null FORMAT Vertical -``` - -```response -Строка 1: -────── -x: 1 -y: ᴺᵁᴸᴸ -``` - -Строки не экранируются в вертикальном формате (Vertical): - -```sql -SELECT 'строка с \'кавычками\', символом табуляции \t и некоторыми специальными \n символами' AS test FORMAT Vertical -``` - -```response -Строка 1: -────── -test: строка с «кавычками» и с особыми - символами -``` - -Этот формат подходит только для вывода результата запроса, но не для разбора (извлечения данных для вставки в таблицу). - - -## Параметры форматирования {#format-settings} diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md deleted file mode 100644 index 5e104b99bf4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -alias: [] -description: 'Документация по формату XML' -input_format: false -keywords: ['XML'] -output_format: true -slug: /interfaces/formats/XML -title: 'XML' -doc_type: 'reference' ---- - -| Вход | Выход | Псевдоним | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## Описание {#description} - -Формат `XML` предназначен только для вывода и не подходит для парсинга. - -Если имя столбца имеет недопустимый формат, в качестве имени элемента используется просто `field`. В целом структура XML соответствует структуре JSON. -Как и в случае с JSON, недопустимые последовательности UTF-8 заменяются символом подстановки `�`, чтобы выходной текст состоял из корректных последовательностей UTF-8. - -В строковых значениях символы `<` и `&` экранируются как `<` и `&`. - -Массивы выводятся как `HelloWorld...`, а кортежи — как `HelloWorld...`. - - - -## Пример использования {#example-usage} - -Пример: - -```xml - - - - - - SearchPhrase - String - - - count() - UInt64 - - - - - - - 8267016 - - - дизайн интерьера ванной - 2166 - - - clickhouse - 1655 - - - весенняя мода 2014 - 1549 - - - фото в свободной форме - 1480 - - - анджелина джоли - 1245 - - - омск - 1112 - - - фото пород собак - 1091 - - - дизайн штор - 1064 - - - баку - 1000 - - - 10 - 141137 - -``` - - -## Параметры форматирования {#format-settings} - - - -## XML {#xml} \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/grpc.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/grpc.md deleted file mode 100644 index 3ab5c3dd57c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/grpc.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -description: 'Документация по интерфейсу gRPC в ClickHouse' -sidebar_label: 'Интерфейс gRPC' -sidebar_position: 25 -slug: /interfaces/grpc -title: 'Интерфейс gRPC' -doc_type: 'reference' ---- - - - -# Интерфейс gRPC {#grpc-interface} - - - -## Введение {#grpc-interface-introduction} - -ClickHouse поддерживает интерфейс [gRPC](https://grpc.io/). Это система удалённого вызова процедур с открытым исходным кодом, использующая HTTP/2 и [Protocol Buffers](https://en.wikipedia.org/wiki/Protocol_Buffers). Реализация gRPC в ClickHouse поддерживает: - -- SSL; -- аутентификацию; -- сессии; -- сжатие; -- параллельные запросы по одному и тому же каналу; -- отмену запросов; -- получение информации о прогрессе и логов; -- внешние таблицы. - -Спецификация интерфейса приведена в файле [clickhouse_grpc.proto](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto). - - - -## Настройка gRPC {#grpc-interface-configuration} - -Чтобы использовать интерфейс gRPC, задайте `grpc_port` в основном [конфигурационном файле сервера](../operations/configuration-files.md). Дополнительные параметры конфигурации приведены в следующем примере: - -```xml -9100 - - false - - - /path/to/ssl_cert_file - /path/to/ssl_key_file - - - false - - - /path/to/ssl_ca_cert_file - - - deflate - - - medium - - - -1 - -1 - - - false - -``` - - -## Встроенный клиент {#grpc-client} - -Вы можете написать клиент на любом из языков программирования, поддерживаемых gRPC, используя предоставленную [спецификацию](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto). -Либо вы можете использовать встроенный клиент на Python. Он находится в репозитории по пути [utils/grpc-client/clickhouse-grpc-client.py](https://github.com/ClickHouse/ClickHouse/blob/master/utils/grpc-client/clickhouse-grpc-client.py). Для встроенного клиента требуются Python‑модули [grpcio и grpcio-tools](https://grpc.io/docs/languages/python/quickstart). - -Клиент поддерживает следующие аргументы: - -* `--help` – Показывает справочное сообщение и завершает работу. -* `--host HOST, -h HOST` – Имя сервера. Значение по умолчанию: `localhost`. Можно также использовать адреса IPv4 или IPv6. -* `--port PORT` – Порт для подключения. Этот порт должен быть разрешён в конфигурации сервера ClickHouse (см. `grpc_port`). Значение по умолчанию: `9100`. -* `--user USER_NAME, -u USER_NAME` – Имя пользователя. Значение по умолчанию: `default`. -* `--password PASSWORD` – Пароль. Значение по умолчанию: пустая строка. -* `--query QUERY, -q QUERY` – Запрос для выполнения в неинтерактивном режиме. -* `--database DATABASE, -d DATABASE` – База данных по умолчанию. Если не указано, используется текущая база данных, заданная в настройках сервера (`default` по умолчанию). -* `--format OUTPUT_FORMAT, -f OUTPUT_FORMAT` – [Формат](formats.md) вывода результата. Значение по умолчанию для интерактивного режима: `PrettyCompact`. -* `--debug` – Включает вывод отладочной информации. - -Чтобы запустить клиент в интерактивном режиме, вызовите его без аргумента `--query`. - -В пакетном режиме данные запроса могут быть переданы через `stdin`. - -**Пример использования клиента** - -В следующем примере создаётся таблица и загружается данными из CSV‑файла. Затем выполняется запрос содержимого таблицы. - -```bash -./clickhouse-grpc-client.py -q "CREATE TABLE grpc_example_table (id UInt32, text String) ENGINE = MergeTree() ORDER BY id;" -echo -e "0,Входные данные для\n1,примера протокола gRPC" > a.csv -cat a.csv | ./clickhouse-grpc-client.py -q "INSERT INTO grpc_example_table FORMAT CSV" - -./clickhouse-grpc-client.py --format PrettyCompact -q "SELECT * FROM grpc_example_table;" -``` - -Результат: - -```text -┌─id─┬─text──────────────────┐ -│ 0 │ Входные данные для │ -│ 1 │ примера протокола gRPC│ -└────┴───────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/http.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/http.md deleted file mode 100644 index 33fe0476d49..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/http.md +++ /dev/null @@ -1,1224 +0,0 @@ ---- -description: 'Документация по HTTP-интерфейсу ClickHouse, предоставляющему доступ к ClickHouse через REST API с любой платформы и на любом языке программирования' -sidebar_label: 'Интерфейс HTTP' -sidebar_position: 15 -slug: /interfaces/http -title: 'Интерфейс HTTP' -doc_type: 'reference' ---- - -import PlayUI from '@site/static/images/play.png'; -import Image from '@theme/IdealImage'; - - -# HTTP-интерфейс {#http-interface} - - - -## Предварительные требования {#prerequisites} - -Для примеров в этой статье вам понадобится: -- запущенный сервер ClickHouse -- установленный `curl`. В Ubuntu или Debian выполните `sudo apt install curl` или обратитесь к этой [документации](https://curl.se/download.html) за инструкциями по установке. - - - -## Обзор {#overview} - -HTTP-интерфейс позволяет использовать ClickHouse на любой платформе и с любого языка программирования в виде REST API. HTTP-интерфейс более ограничен, чем нативный интерфейс, но обладает лучшей поддержкой языков. - -По умолчанию `clickhouse-server` слушает следующие порты: - -* порт 8123 для HTTP -* порт 8443 для HTTPS, который при необходимости можно включить - -Если вы выполняете запрос `GET /` без каких-либо параметров, возвращается код ответа 200 и строка "Ok.": - -```bash -$ curl 'http://localhost:8123/' -Ok. -``` - -"Ok." является значением по умолчанию, определённым в [`http_server_default_response`](../operations/server-configuration-parameters/settings.md#http_server_default_response), и при желании его можно изменить. - -См. также: [Особенности кодов ответа HTTP](#http_response_codes_caveats). - - -## Веб-интерфейс пользователя {#web-ui} - -ClickHouse включает веб-интерфейс пользователя, доступ к которому можно получить по следующему адресу: - -```text -http://localhost:8123/play -``` - -Веб-интерфейс поддерживает отображение прогресса во время выполнения запроса, отмену запроса и потоковую передачу результатов. -В нём есть скрытая функция для отображения диаграмм и графиков для пайплайнов запросов. - -После успешного выполнения запроса появляется кнопка загрузки, которая позволяет скачать результаты запроса в различных форматах, включая CSV, TSV, JSON, JSONLines, Parquet, Markdown или любой пользовательский формат, поддерживаемый ClickHouse. Функция загрузки использует кэш запросов для эффективного получения результатов без повторного выполнения запроса. Будут загружены все результаты, даже если в интерфейсе была показана только одна страница из многих. - -Веб-интерфейс разработан для таких профессионалов, как вы. - -Скриншот веб-интерфейса ClickHouse - -В скриптах проверки состояния используйте запрос `GET /ping`. Этот обработчик всегда возвращает «Ok.» (с символом перевода строки в конце). Доступно начиная с версии 18.12.13. См. также `/replicas_status` для проверки задержки реплики. - -```bash -$ curl 'http://localhost:8123/ping' -Ok. -$ curl 'http://localhost:8123/replicas_status' -Ok. -``` - - -## Выполнение запросов по HTTP/HTTPS {#querying} - -Для выполнения запросов по HTTP/HTTPS есть три варианта: - -* отправить запрос как параметр URL `query` -* использовать метод POST -* отправить начало запроса в параметре `query`, а остальную часть — в теле POST - -:::note -Размер URL по умолчанию ограничен 1 MiB, это можно изменить с помощью настройки `http_max_uri_size`. -::: - -В случае успешного выполнения вы получаете код ответа 200 и результат в теле ответа. -В случае ошибки вы получаете код ответа 500 и текстовое описание ошибки в теле ответа. - -Запросы с использованием GET являются «только для чтения» (`readonly`). Это означает, что для запросов, модифицирующих данные, можно использовать только метод POST. -Сам запрос можно отправить либо в теле POST, либо в параметре URL. Рассмотрим несколько примеров. - -В приведённом ниже примере для отправки запроса `SELECT 1` используется curl. Обратите внимание на использование URL-кодирования для пробела: `%20`. - -```bash title="command" -curl 'http://localhost:8123/?query=SELECT%201' -``` - -```response title="Response" -1 -``` - -В этом примере wget используется с параметрами `-nv` (минимальный вывод) и `-O-` для вывода результата в терминал. -В данном случае нет необходимости использовать URL-кодирование для пробела: - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1' -``` - -```response -1 -``` - -В этом примере мы передаём сырой HTTP-запрос в netcat через конвейер (pipe): - -```bash title="command" -echo -ne 'GET /?query=SELECT%201 HTTP/1.0\r\n\r\n' | nc localhost 8123 -``` - -```response title="response" -HTTP/1.0 200 OK -X-ClickHouse-Summary: {"read_rows":"1","read_bytes":"1","written_rows":"0","written_bytes":"0","total_rows_to_read":"1","result_rows":"0","result_bytes":"0","elapsed_ns":"4505959","memory_usage":"1111711"} -Date: Tue, 11 Nov 2025 18:16:01 GMT -Connection: Close -Content-Type: text/tab-separated-values; charset=UTF-8 -Access-Control-Expose-Headers: X-ClickHouse-Query-Id,X-ClickHouse-Summary,X-ClickHouse-Server-Display-Name,X-ClickHouse-Format,X-ClickHouse-Timezone,X-ClickHouse-Exception-Code,X-ClickHouse-Exception-Tag -X-ClickHouse-Server-Display-Name: MacBook-Pro.local -X-ClickHouse-Query-Id: ec0d8ec6-efc4-4e1d-a14f-b748e01f5294 -X-ClickHouse-Format: TabSeparated -X-ClickHouse-Timezone: Europe/London -X-ClickHouse-Exception-Tag: dngjzjnxkvlwkeua - -1 -``` - -Как видно, команда `curl` несколько неудобна тем, что пробелы в URL приходится экранировать. -Хотя `wget` выполняет экранирование автоматически, мы не рекомендуем его использовать, поскольку он некорректно работает по HTTP/1.1 при использовании keep-alive и Transfer-Encoding: chunked. - -```bash -$ echo 'SELECT 1' | curl 'http://localhost:8123/' --data-binary @- -1 - -$ echo 'SELECT 1' | curl 'http://localhost:8123/?query=' --data-binary @- -1 - -$ echo '1' | curl 'http://localhost:8123/?query=SELECT' --data-binary @- -1 -``` - -Если часть запроса отправляется в параметре, а часть в POST, между этими двумя частями данных вставляется символ новой строки. -Например, это не будет работать: - -```bash -$ echo 'ECT 1' | curl 'http://localhost:8123/?query=SEL' --data-binary @- -Code: 59, e.displayText() = DB::Exception: Синтаксическая ошибка: ошибка в позиции 0: SEL -ECT 1 -, ожидалось одно из: SHOW TABLES, SHOW DATABASES, SELECT, INSERT, CREATE, ATTACH, RENAME, DROP, DETACH, USE, SET, OPTIMIZE., e.what() = DB::Exception -``` - -По умолчанию данные возвращаются в формате [`TabSeparated`](/interfaces/formats/TabSeparated). - -Для запроса другого формата в запросе используется предложение `FORMAT`. Например: - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1, 2, 3 FORMAT JSON' -``` - - -```response title="Response" -{ - "meta": - [ - { - "name": "1", - "type": "UInt8" - }, - { - "name": "2", - "type": "UInt8" - }, - { - "name": "3", - "type": "UInt8" - } - ], - - "data": - [ - { - "1": 1, - "2": 2, - "3": 3 - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.000515, - "rows_read": 1, - "bytes_read": 1 - } -} -``` - -Вы можете использовать параметр `default_format` в URL или заголовок `X-ClickHouse-Format`, чтобы указать формат по умолчанию, отличный от `TabSeparated`. - -```bash -$ echo 'SELECT 1 FORMAT Pretty' | curl 'http://localhost:8123/?' --data-binary @- -┏━━━┓ -┃ 1 ┃ -┡━━━┩ -│ 1 │ -└───┘ -``` - -Вы можете использовать метод POST для выполнения параметризованных запросов. Параметры задаются в фигурных скобках с указанием имени и типа параметра, например `{name:Type}`. Значения параметров передаются через `param_name`: - -```bash -$ curl -X POST -F 'query=select {p1:UInt8} + {p2:UInt8}' -F "param_p1=3" -F "param_p2=4" 'http://localhost:8123/' - -7 -``` - - -## Запросы INSERT по HTTP/HTTPS {#insert-queries} - -Метод передачи данных `POST` используется для запросов `INSERT`. В этом случае вы можете указать начало запроса в параметре URL и использовать POST для передачи данных, которые нужно вставить. Вставляемыми данными может быть, например, дамп из MySQL в формате с разделителями табуляции. Таким образом, запрос `INSERT` заменяет `LOAD DATA LOCAL INFILE` из MySQL. - -### Примеры {#examples} - -Чтобы создать таблицу: - -```bash -$ echo 'CREATE TABLE t (a UInt8) ENGINE = Memory' | curl 'http://localhost:8123/' --data-binary @- -``` - -Чтобы вставлять данные с помощью привычного запроса `INSERT`: - -```bash -$ echo 'INSERT INTO t VALUES (1),(2),(3)' | curl 'http://localhost:8123/' --data-binary @- -``` - -Чтобы отправить данные отдельно от запроса: - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -Можно указать любой формат данных. Например, можно указать формат «Values» — тот же, что используется при выполнении `INSERT INTO t VALUES`: - -```bash -$ echo '(7),(8),(9)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20Values' --data-binary @- -``` - -Чтобы вставить данные из дампа с разделителями-табуляциями, укажите соответствующий формат: - -```bash -$ echo -ne '10\n11\n12\n' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20TabSeparated' --data-binary @- -``` - -Чтобы просмотреть содержимое таблицы: - -```bash -$ curl 'http://localhost:8123/?query=SELECT%20a%20FROM%20t' -7 -8 -9 -10 -11 -12 -1 -2 -3 -4 -5 -6 -``` - -:::note -Данные выводятся в произвольном порядке из‑за параллельной обработки запроса. -::: - -Чтобы удалить таблицу: - -```bash -$ echo 'DROP TABLE t' | curl 'http://localhost:8123/' --data-binary @- -``` - -Для успешных запросов, не возвращающих таблицу данных, возвращается пустое тело ответа. - - -## Сжатие {#compression} - -Сжатие можно использовать для уменьшения объема сетевого трафика при передаче больших объемов данных или для создания дампов, которые сразу сохраняются в сжатом виде. - -Вы можете использовать внутренний формат сжатия ClickHouse при передаче данных. Сжатые данные имеют нестандартный формат, и для работы с ними требуется программа `clickhouse-compressor`. Она устанавливается по умолчанию с пакетом `clickhouse-client`. - -Чтобы повысить эффективность вставки данных, отключите проверку контрольных сумм на стороне сервера с помощью настройки [`http_native_compression_disable_checksumming_on_decompress`](../operations/settings/settings.md#http_native_compression_disable_checksumming_on_decompress). - -Если вы укажете `compress=1` в URL, сервер будет сжимать данные, которые отправляет клиенту. Если вы укажете `decompress=1` в URL, сервер будет распаковывать данные, которые вы передаёте методом `POST`. - -Вы также можете использовать [HTTP-сжатие](https://en.wikipedia.org/wiki/HTTP_compression). ClickHouse поддерживает следующие [методы сжатия](https://en.wikipedia.org/wiki/HTTP_compression#Content-Encoding_tokens): - -- `gzip` -- `br` -- `deflate` -- `xz` -- `zstd` -- `lz4` -- `bz2` -- `snappy` - -Чтобы отправить сжатый `POST`-запрос, добавьте заголовок запроса `Content-Encoding: compression_method`. - -Чтобы ClickHouse сжал ответ, добавьте к запросу заголовок `Accept-Encoding: compression_method`. - -Вы можете настроить уровень сжатия данных с помощью настройки [`http_zlib_compression_level`](../operations/settings/settings.md#http_zlib_compression_level) для всех методов сжатия. - -:::info -Некоторые HTTP-клиенты могут по умолчанию распаковывать данные от сервера (для `gzip` и `deflate`), и вы можете получить уже распакованные данные, даже если правильно используете настройки сжатия. -::: - - - -## Примеры {#examples-compression} - -Чтобы отправить сжатые данные на сервер: - -```bash -echo "SELECT 1" | gzip -c | \ -curl -sS --data-binary @- -H 'Content-Encoding: gzip' 'http://localhost:8123/' -``` - -Чтобы получить с сервера архив сжатых данных: - -```bash -curl -vsS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' --output result.gz -d 'SELECT number FROM system.numbers LIMIT 3' - -zcat result.gz -0 -1 -2 -``` - -Для получения сжатых данных с сервера и их распаковки используйте gunzip: - -```bash -curl -sS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' -d 'SELECT number FROM system.numbers LIMIT 3' | gunzip - -0 -1 -2 -``` - - -## База данных по умолчанию {#default-database} - -Вы можете использовать параметр `database` в URL или заголовок `X-ClickHouse-Database`, чтобы указать базу данных по умолчанию. - -```bash -echo 'SELECT number FROM numbers LIMIT 10' | curl 'http://localhost:8123/?database=system' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -По умолчанию в качестве базы данных по умолчанию используется та, которая указана в настройках сервера. Изначально это база данных с именем `default`. При необходимости вы всегда можете указать базу данных, добавив её имя и точку перед именем таблицы. - - -## Аутентификация {#authentication} - -Имя пользователя и пароль можно указать одним из трёх способов: - -1. С помощью базовой HTTP-аутентификации (HTTP Basic Authentication). - -Например: - -```bash -echo 'SELECT 1' | curl 'http://user:password@localhost:8123/' -d @- -``` - -2. В URL-параметрах `user` и `password` - -:::warning -Мы не рекомендуем использовать этот метод, так как параметр может протоколироваться веб-прокси и кэшироваться в браузере -::: - -Например: - -```bash -echo 'SELECT 1' | curl 'http://localhost:8123/?user=user&password=password' -d @- -``` - -3. Использование заголовков 'X-ClickHouse-User' и 'X-ClickHouse-Key' - -Например: - -```bash -echo 'SELECT 1' | curl -H 'X-ClickHouse-User: user' -H 'X-ClickHouse-Key: password' 'http://localhost:8123/' -d @- -``` - -Если имя пользователя не указано, используется имя `default`. Если пароль не указан, используется пустой пароль. -Вы также можете использовать параметры URL-адреса, чтобы указать любые настройки для обработки одного запроса или целого профиля настроек. - -Например: - -```text -http://localhost:8123/?profile=web&max_rows_to_read=1000000000&query=SELECT+1 -``` - -```bash -$ echo 'SELECT number FROM system.numbers LIMIT 10' | curl 'http://localhost:8123/?' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -Дополнительную информацию см. в разделах: - -* [Settings](/operations/settings/settings) -* [SET](/sql-reference/statements/set) - - -## Использование сессий ClickHouse в протоколе HTTP {#using-clickhouse-sessions-in-the-http-protocol} - -Вы также можете использовать сессии ClickHouse в протоколе HTTP. Для этого необходимо добавить к запросу `GET`-параметр `session_id`. В качестве идентификатора сессии можно использовать любую строку. - -По умолчанию сессия завершается после 60 секунд бездействия. Чтобы изменить этот таймаут (в секундах), измените настройку `default_session_timeout` в конфигурации сервера или добавьте к запросу `GET`-параметр `session_timeout`. - -Чтобы проверить состояние сессии, используйте параметр `session_check=1`. В рамках одной сессии одновременно может выполняться только один запрос. - -Вы можете получать информацию о ходе выполнения запроса в заголовках ответа `X-ClickHouse-Progress`. Для этого включите [`send_progress_in_http_headers`](../operations/settings/settings.md#send_progress_in_http_headers). - -Ниже приведён пример последовательности заголовков: - -```text -X-ClickHouse-Progress: {"read_rows":"261636","read_bytes":"2093088","total_rows_to_read":"1000000","elapsed_ns":"14050417","memory_usage":"22205975"} -X-ClickHouse-Progress: {"read_rows":"654090","read_bytes":"5232720","total_rows_to_read":"1000000","elapsed_ns":"27948667","memory_usage":"83400279"} -X-ClickHouse-Progress: {"read_rows":"1000000","read_bytes":"8000000","total_rows_to_read":"1000000","elapsed_ns":"38002417","memory_usage":"80715679"} -``` - -Возможные поля заголовка: - -| Header field | Description | -| -------------------- | ------------------------------------------------------------- | -| `read_rows` | Количество прочитанных строк. | -| `read_bytes` | Объём прочитанных данных в байтах. | -| `total_rows_to_read` | Общее количество строк, подлежащих чтению. | -| `written_rows` | Количество записанных строк. | -| `written_bytes` | Объём записанных данных в байтах. | -| `elapsed_ns` | Время выполнения запроса в наносекундах. | -| `memory_usage` | Объём памяти в байтах, использованный для выполнения запроса. | - -Выполняющиеся запросы не останавливаются автоматически, если HTTP‑соединение потеряно. Разбор и форматирование данных выполняются на стороне сервера, и в таком случае использование сети может быть неэффективным. - -Существуют следующие необязательные параметры: - -| Parameters | Description | -| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | -| `query_id` (optional) | Может быть передан как идентификатор запроса (произвольная строка). [`replace_running_query`](/operations/settings/settings#replace_running_query) | -| `quota_key` (optional) | Может быть передан как ключ квоты (произвольная строка). [«Quotas»](/operations/quotas) | - -HTTP‑интерфейс позволяет передавать внешние данные (внешние временные таблицы) для выполнения запросов. Дополнительные сведения см. в разделе [«External data for query processing»](/engines/table-engines/special/external-data). - - -## Буферизация ответа {#response-buffering} - -Буферизацию ответа можно включить на стороне сервера. Для этого предусмотрены следующие параметры URL: - -* `buffer_size` -* `wait_end_of_query` - -Можно использовать следующие настройки: - -* [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size) -* [`http_wait_end_of_query`](/operations/settings/settings#http_wait_end_of_query) - -`buffer_size` определяет объём результата в байтах, который будет буферизован в памяти сервера. Если тело результата превышает этот порог, буфер записывается в HTTP-канал, а оставшиеся данные отправляются напрямую в HTTP-канал. - -Чтобы гарантировать буферизацию всего ответа, установите `wait_end_of_query=1`. В этом случае данные, не помещающиеся в память, будут буферизованы во временном файле на сервере. - -Например: - -```bash -curl -sS 'http://localhost:8123/?max_result_bytes=4000000&buffer_size=3000000&wait_end_of_query=1' -d 'SELECT toUInt8(number) FROM system.numbers LIMIT 9000000 FORMAT RowBinary' -``` - -:::tip -Используйте буферизацию, чтобы избежать ситуаций, когда ошибка обработки запроса возникает после того, как код ответа и HTTP-заголовки уже были отправлены клиенту. В таком случае сообщение об ошибке записывается в конце тела ответа, и на стороне клиента ошибка может быть обнаружена только при разборе ответа. -::: - - -## Установка роли с помощью параметров запроса {#setting-role-with-query-parameters} - -Эта возможность была добавлена в ClickHouse 24.4. - -В некоторых сценариях перед выполнением самого запроса может потребоваться сначала задать выданную роль. Однако отправить `SET ROLE` и сам запрос вместе нельзя, так как выполнение нескольких операторов в одном запросе не поддерживается: - -```bash -curl -sS "http://localhost:8123" --data-binary "SET ROLE my_role;SELECT * FROM my_table;" -``` - -Приведённая выше команда вызывает ошибку: - -```sql -Код: 62. DB::Exception: Синтаксическая ошибка (Выполнение нескольких операторов не разрешено) -``` - -Чтобы обойти это ограничение, используйте параметр запроса `role`: - -```bash -curl -sS "http://localhost:8123?role=my_role" --data-binary "SELECT * FROM my_table;" -``` - -Это эквивалентно выполнению `SET ROLE my_role` перед выполнением запроса. - -Кроме того, можно указать несколько параметров запроса `role`: - -```bash -curl -sS "http://localhost:8123?role=my_role&role=my_other_role" --data-binary "SELECT * FROM my_table;" -``` - -В этом случае `?role=my_role&role=my_other_role` работает аналогично выполнению `SET ROLE my_role, my_other_role` перед выполнением запроса. - - -## Особенности кодов ответа HTTP {#http_response_codes_caveats} - -Из-за ограничений протокола HTTP код ответа 200 не гарантирует, что запрос был успешно выполнен. - -Вот пример: - -```bash -curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)" -* Попытка подключения 127.0.0.1:8123... -... -< HTTP/1.1 200 OK -... -Код: 395. DB::Exception: Значение, переданное в функцию 'throwIf', не равно нулю: при выполнении 'FUNCTION throwIf(equals(number, 2) :: 1) -> throwIf(equals(number, 2)) -``` - -Причина такого поведения связана с природой протокола HTTP. Сначала отправляется HTTP‑заголовок с кодом 200, затем тело HTTP‑ответа, и уже после этого ошибка внедряется в это тело в виде обычного текста. - -Такое поведение не зависит от используемого формата — будь то `Native`, `TSV` или `JSON`: сообщение об ошибке всегда будет находиться посередине потока ответа. - -Вы можете частично смягчить эту проблему, включив `wait_end_of_query=1` ([Буферизация ответа](#response-buffering)). В этом случае отправка HTTP‑заголовка откладывается до тех пор, пока весь запрос не будет обработан. Однако это не полностью решает проблему, потому что результат по‑прежнему должен помещаться в [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size), а другие настройки, такие как [`send_progress_in_http_headers`](/operations/settings/settings#send_progress_in_http_headers), могут нарушать задержку отправки заголовка. - -:::tip -Единственный способ отловить все ошибки — проанализировать тело HTTP‑ответа до того, как вы начнёте разбирать его в требуемом формате. -::: - -Такие исключения в ClickHouse имеют единый формат, приведённый ниже, независимо от используемого формата (например, `Native`, `TSV`, `JSON` и т. д.), если `http_write_exception_in_output_format=0` (значение по умолчанию). Это упрощает разбор и извлечение сообщений об ошибках на стороне клиента. - -```text -\r\n -__exception__\r\n -\r\n -<сообщение_об_ошибке>\r\n -<длина_сообщения> \r\n -__exception__\r\n - -``` - -Где `` — это 16-байтовый случайный тег, совпадающий с тегом, отправленным в заголовке ответа `X-ClickHouse-Exception-Tag`. -`` — это собственно текст сообщения об исключении (его точную длину можно узнать из ``). Весь блок исключения, описанный выше, может иметь размер до 16 КиБ. - -Ниже приведён пример в формате `JSON` - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+JSON" -... -{ - "meta": - [ - { - "name": "sleepEachRow(0.001)", - "type": "UInt8" - }, - { - "name": "throwIf(equals(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - }, - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - } -__exception__ -dmrdfnujjqvszhav -Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 25.11.1.1) -262 dmrdfnujjqvszhav -__exception__ -``` - -Вот аналогичный пример, но в формате `CSV` - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+CSV" -... -< -0,0 -0,0 -``` - - -**исключение** -rumfyutuqkncbgau -Код: 395. DB::Exception: Значение, переданное в функцию 'throwIf', является ненулевым: при выполнении выражения 'FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (версия 25.11.1.1) -262 rumfyutuqkncbgau -**исключение** - -``` -``` - - -## Запросы с параметрами {#cli-queries-with-parameters} - -Вы можете создать запрос с параметрами и передавать им значения из соответствующих параметров HTTP-запроса. Для получения дополнительной информации см. раздел [Запросы с параметрами для CLI](../interfaces/cli.md#cli-queries-with-parameters). - -### Пример {#example-3} - -```bash -$ curl -sS "
?param_id=2¶m_phrase=test" -d "SELECT * FROM table WHERE int_column = {id:UInt8} and string_column = {phrase:String}" -``` - -### Табуляции в параметрах URL {#tabs-in-url-parameters} - -Параметры запроса разбираются из «экранированного» формата. У этого есть некоторые преимущества, например возможность однозначно интерпретировать значения `NULL` как `\N`. Это означает, что символ табуляции должен быть закодирован как `\t` (или как `\` и табуляция). Например, в следующем примере между `abc` и `123` содержится реальный символ табуляции, и входная строка разбивается на два значения: - -```bash -curl -sS "http://localhost:8123" -d "SELECT splitByChar('\t', 'abc 123')" -``` - -```response -['abc','123'] -``` - -Однако если вы попытаетесь закодировать символ табуляции, используя `%09` в параметре URL, он не будет корректно обработан: - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%09123" -d "SELECT splitByChar('\t', {arg1:String})" -Код: 457. DB::Exception: Значение abc 123 не может быть разобрано как String для параметра запроса 'arg1', так как оно разобрано не полностью: разобрано только 3 из 7 байтов: abc. (BAD_QUERY_PARAMETER) (версия 23.4.1.869 (официальная сборка)) -``` - -Если вы используете параметры URL, вам необходимо закодировать `\t` как `%5C%09`. Например: - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%5C%09123" -d "SELECT splitByChar('\t', {arg1:String})" -``` - -```response -['abc','123'] -``` - - -## Предопределённый HTTP-интерфейс {#predefined_http_interface} - -ClickHouse поддерживает выполнение специальных запросов через HTTP-интерфейс. Например, вы можете записать данные в таблицу следующим образом: - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -ClickHouse также поддерживает предопределённый HTTP‑интерфейс, который упрощает интеграцию со сторонними инструментами, такими как [Prometheus exporter](https://github.com/ClickHouse/clickhouse_exporter). Рассмотрим пример. - -Прежде всего добавьте этот раздел в файл конфигурации сервера. - -`http_handlers` настроен так, чтобы содержать несколько правил `rule`. ClickHouse будет сопоставлять входящие HTTP‑запросы с предопределённым типом, указанным в `rule`, и обработчик будет запущен для первого совпавшего правила. Затем ClickHouse выполнит соответствующий предопределённый запрос, если сопоставление прошло успешно. - -```yaml title="config.xml" - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.metrics LIMIT 5 FORMAT Template SETTINGS format_template_resultset = 'prometheus_template_output_format_resultset', format_template_row = 'prometheus_template_output_format_row', format_template_rows_between_delimiter = '\n' - - - ... - ... - -``` - -Теперь вы можете получать данные в формате Prometheus, обращаясь непосредственно по URL: - - -```bash -$ curl -v 'http://localhost:8123/predefined_query' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /predefined_query HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> -< HTTP/1.1 200 OK -< Date: Tue, 28 Apr 2020 08:52:56 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< X-ClickHouse-Server-Display-Name: i-mloy5trc -< Transfer-Encoding: chunked -< X-ClickHouse-Query-Id: 96fe0052-01e6-43ce-b12a-6b7370de6e8a -< X-ClickHouse-Format: Template -< X-ClickHouse-Timezone: Asia/Shanghai -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -# HELP "Query" "Number of executing queries" {#help-query-number-of-executing-queries} -# TYPE "Query" counter {#type-query-counter} -"Query" 1 -``` - - -# HELP "Merge" "Количество выполняемых фоновых слияний" {#help-merge-number-of-executing-background-merges} -# TYPE "Merge" counter {#type-merge-counter} -"Merge" 0 - - - -# HELP "PartMutation" "Количество мутаций (ALTER DELETE/UPDATE)" {#help-partmutation-number-of-mutations-alter-deleteupdate} -# TYPE "PartMutation" counter {#type-partmutation-counter} -"PartMutation" 0 - - - -# HELP "ReplicatedFetch" "Количество частей данных, получаемых из реплики" {#help-replicatedfetch-number-of-data-parts-being-fetched-from-replica} -# TYPE "ReplicatedFetch" counter {#type-replicatedfetch-counter} -"ReplicatedFetch" 0 - - - -# HELP "ReplicatedSend" "Количество частей данных, отправляемых на реплики" {#help-replicatedsend-number-of-data-parts-being-sent-to-replicas} - -# TYPE "ReplicatedSend" counter {#type-replicatedsend-counter} - -"ReplicatedSend" 0 - -* Соединение №0 с хостом localhost оставлено открытым - -* Соединение №0 с хостом localhost оставлено открытым - -``` - -Параметры конфигурации `http_handlers` работают следующим образом. - -`rule` может настраивать следующие параметры: -- `method` -- `headers` -- `url` -- `full_url` -- `handler` - -Каждый из них описан ниже: - -- `method` отвечает за сопоставление метода HTTP-запроса. `method` полностью соответствует определению [`method`] - (https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) в протоколе HTTP. Это необязательный параметр. Если он не определён в - конфигурационном файле, сопоставление метода HTTP-запроса не выполняется. - -- `url` отвечает за сопоставление части URL (пути и строки запроса) HTTP-запроса. - Если `url` имеет префикс `regex:`, ожидаются регулярные выражения [RE2](https://github.com/google/re2). - Это необязательный параметр. Если он не определён в конфигурационном файле, сопоставление части URL HTTP-запроса не выполняется. - -- `full_url` аналогичен `url`, но включает полный URL, т. е. `schema://host:port/path?query_string`. - Обратите внимание, что ClickHouse не поддерживает «виртуальные хосты», поэтому `host` является IP-адресом (а не значением заголовка `Host`). - -- `empty_query_string` — гарантирует отсутствие строки запроса (`?query_string`) в запросе - -- `headers` отвечает за сопоставление заголовков HTTP-запроса. Совместим с регулярными выражениями RE2. Это необязательный - параметр. Если он не определён в конфигурационном файле, сопоставление заголовков HTTP-запроса не выполняется. - -- `handler` содержит основную часть обработки. - - Может иметь следующие значения `type`: - - [`predefined_query_handler`](#predefined_query_handler) - - [`dynamic_query_handler`](#dynamic_query_handler) - - [`static`](#static) - - [`redirect`](#redirect) - - И следующие параметры: - - `query` — используется с типом `predefined_query_handler`, выполняет запрос при вызове обработчика. - - `query_param_name` — используется с типом `dynamic_query_handler`, извлекает и выполняет значение, соответствующее `query_param_name` в - параметрах HTTP-запроса. - - `status` — используется с типом `static`, код состояния ответа. - - `content_type` — используется с любым типом, [content-type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type) ответа. - - `http_response_headers` — используется с любым типом, карта заголовков ответа. Может также использоваться для установки типа содержимого. - - `response_content` — используется с типом `static`, содержимое ответа, отправляемое клиенту; при использовании префикса 'file://' или 'config://' содержимое - извлекается из файла или конфигурации и отправляется клиенту. - - `user` — пользователь, от имени которого выполняется запрос (пользователь по умолчанию — `default`). - **Примечание**: не требуется указывать пароль для этого пользователя. - -Методы конфигурации для различных значений `type` описаны далее. - -### predefined_query_handler {#predefined_query_handler} - -`predefined_query_handler` поддерживает установку значений `Settings` и `query_params`. Вы можете настроить `query` в типе `predefined_query_handler`. - -Значение `query` представляет собой предопределённый запрос `predefined_query_handler`, который выполняется ClickHouse при совпадении HTTP-запроса, и возвращается результат запроса. Это обязательный параметр. - -Следующий пример определяет значения настроек [`max_threads`](../operations/settings/settings.md#max_threads) и [`max_final_threads`](/operations/settings/settings#max_final_threads), затем выполняет запрос к системной таблице для проверки успешной установки этих настроек. - -:::note -Чтобы сохранить обработчики по умолчанию, такие как `query`, `play`, `ping`, добавьте правило ``. -::: - -Например: -``` - - -```yaml - - - [^/]+)]]> - GET - - TEST_HEADER_VALUE - [^/]+)]]> - - - predefined_query_handler - - SELECT name, value FROM system.settings - WHERE name IN ({name_1:String}, {name_2:String}) - - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE' -H 'PARAMS_XXX:max_final_threads' 'http://localhost:8123/query_param_with_url/max_threads?max_threads=1&max_final_threads=2' -max_final_threads 2 -max_threads 1 -``` - -:::note -В одном `predefined_query_handler` поддерживается только один `query`. -::: - -### dynamic_query_handler {#dynamic_query_handler} - -В `dynamic_query_handler` запрос передаётся в виде параметра HTTP‑запроса. В отличие от него, в `predefined_query_handler` запрос задаётся в конфигурационном файле. Параметр `query_param_name` может быть настроен в `dynamic_query_handler`. - -ClickHouse извлекает и выполняет значение, соответствующее `query_param_name`, из URL HTTP‑запроса. Значение `query_param_name` по умолчанию — `/query`. Это необязательная настройка. Если параметр не определён в конфигурационном файле, он не передаётся. - -Чтобы поэкспериментировать с этой функциональностью, в следующем примере задаются значения [`max_threads`](../operations/settings/settings.md#max_threads) и `max_final_threads`, а также выполняются запросы, чтобы проверить, были ли настройки успешно применены. - -Пример: - -```yaml - - - - TEST_HEADER_VALUE_DYNAMIC - - dynamic_query_handler - query_param - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE_DYNAMIC' 'http://localhost:8123/own?max_threads=1&max_final_threads=2¶m_name_1=max_threads¶m_name_2=max_final_threads&query_param=SELECT%20name,value%20FROM%20system.settings%20where%20name%20=%20%7Bname_1:String%7D%20OR%20name%20=%20%7Bname_2:String%7D' -max_threads 1 -max_final_threads 2 -``` - -### static {#static} - -`static` может возвращать [`content_type`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type), [status](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) и `response_content`. `response_content` задаёт возвращаемое содержимое. - -Например, чтобы вернуть сообщение «Say Hi!»: - -```yaml - - - GET - xxx - /hi - - static - 402 - text/html; charset=UTF-8 - - en - 43 - - #highlight-next-line - Say Hi! - - - - -``` - -`http_response_headers` можно использовать для указания типа контента вместо `content_type`. - - -```yaml - - - GET - xxx - /hi - - static - 402 - #begin-highlight - - text/html; charset=UTF-8 - en - 43 - - #end-highlight - Привет! - - - - -``` - -```bash -curl -vv -H 'XXX:xxx' 'http://localhost:8123/hi' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /hi HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 402 Payment Required -< Date: Wed, 29 Apr 2020 03:51:26 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -Say Hi!% -``` - -Найдите содержимое конфигурации, отправленной клиенту. - -```yaml -
]]>
- - - - GET - xxx - /get_config_static_handler - - static - config://get_config_static_handler - - - -``` - -```bash -$ curl -v -H 'XXX:xxx' 'http://localhost:8123/get_config_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_config_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:01:24 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -
% -``` - -Чтобы найти содержимое файла, отправленного клиенту: - - -```yaml - - - GET - xxx - /get_absolute_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file:///absolute_path_file.html - - - - GET - xxx - /get_relative_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file://./relative_path_file.html - - - -``` - -```bash -$ user_files_path='/var/lib/clickhouse/user_files' -$ sudo echo "Файл с относительным путем" > $user_files_path/relative_path_file.html -$ sudo echo "Файл с абсолютным путем" > $user_files_path/absolute_path_file.html -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_absolute_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_absolute_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:16 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -Файл с абсолютным путем -* Connection #0 to host localhost left intact -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_relative_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_relative_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:31 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -Файл с относительным путем -* Connection #0 to host localhost left intact -``` - -### redirect {#redirect} - -`redirect` выполнит перенаправление `302` на `location` - -Например, так вы можете автоматически добавить параметр `set user` в `play` для ClickHouse Play: - -```xml - - - - GET - /play - - redirect - /play?user=play - - - - -``` - - -## HTTP заголовки ответа {#http-response-headers} - -ClickHouse позволяет настраивать пользовательские HTTP-заголовки ответа, которые могут применяться к любому настраиваемому обработчику. Эти заголовки можно задать с помощью настройки `http_response_headers`, которая принимает пары ключ-значение, представляющие имена заголовков и их значения. Эта возможность особенно полезна для реализации пользовательских заголовков безопасности, политик CORS или любых других требований к HTTP-заголовкам для всего HTTP-интерфейса ClickHouse. - -Например, вы можете настроить заголовки для: - -* Обычных эндпоинтов выполнения запросов -* Web UI -* Проверки работоспособности. - -Также можно указать `common_http_response_headers`. Они будут применены ко всем HTTP-обработчикам, определённым в конфигурации. - -Заголовки будут включены в HTTP-ответ для каждого настроенного обработчика. - -В примере ниже каждый ответ сервера будет содержать два пользовательских заголовка: `X-My-Common-Header` и `X-My-Custom-Header`. - -```xml - - - - Общий заголовок - - - GET - /ping - - ping - - Пользовательский заголовок - - - - - -``` - - -## Корректный JSON/XML-ответ при исключении во время HTTP‑стриминга {#valid-output-on-exception-http-streaming} - -Во время выполнения запроса по HTTP может произойти исключение, когда часть данных уже была отправлена. Обычно исключение отправляется клиенту в виде обычного текста. -При этом, даже если для вывода использовался определённый формат данных, результат может стать некорректным с точки зрения этого формата. -Чтобы избежать этого, вы можете использовать настройку [`http_write_exception_in_output_format`](/operations/settings/settings#http_write_exception_in_output_format) (по умолчанию отключена), которая заставляет ClickHouse записывать исключение в заданном формате (в настоящее время поддерживаются форматы XML и JSON*). - -Примеры: - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>3)+from+system.numbers+format+JSON+settings+max_block_size=1&http_write_exception_in_output_format=1' -{ - "meta": - [ - { - "name": "number", - "type": "UInt64" - }, - { - "name": "throwIf(greater(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "number": "0", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "1", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "2", - "throwIf(greater(number, 2))": 0 - } - ], - - "rows": 3, - - "exception": "Code: 395. DB::Exception: Значение, переданное в функцию 'throwIf', не равно нулю: при выполнении 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (версия 23.8.1.1)" -} -``` - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>2)+from+system.numbers+format+XML+settings+max_block_size=1&http_write_exception_in_output_format=1' - - - - - - number - UInt64 - - - throwIf(greater(number, 2)) - UInt8 - - - - - - 0 - 0 - - - 1 - 0 - - - 2 - 0 - - - 3 - Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 23.8.1.1) - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/jdbc.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/jdbc.md deleted file mode 100644 index 8118f257ca2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/jdbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Руководство по использованию JDBC-драйвера для подключения к ClickHouse из Java-приложений' -sidebar_label: 'JDBC-драйвер' -sidebar_position: 20 -slug: /interfaces/jdbc -title: 'JDBC-драйвер' -doc_type: 'guide' ---- - -# JDBC-драйвер {#jdbc-driver} - -Используйте [официальный JDBC-драйвер](/docs/integrations/language-clients/java/jdbc) (и Java-клиент) для доступа к ClickHouse из Java-приложений. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/mysql.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/mysql.md deleted file mode 100644 index 32a5be2c707..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/mysql.md +++ /dev/null @@ -1,183 +0,0 @@ ---- -description: 'Документация по интерфейсу протокола MySQL в ClickHouse, который позволяет - клиентам MySQL подключаться к ClickHouse' -sidebar_label: 'Интерфейс MySQL' -sidebar_position: 25 -slug: /interfaces/mysql -title: 'Интерфейс MySQL' -doc_type: 'guide' ---- - -import Image from '@theme/IdealImage'; -import mysql0 from '@site/static/images/interfaces/mysql0.png'; -import mysql1 from '@site/static/images/interfaces/mysql1.png'; -import mysql2 from '@site/static/images/interfaces/mysql2.png'; -import mysql3 from '@site/static/images/interfaces/mysql3.png'; - - -# Интерфейс MySQL {#mysql-interface} - -ClickHouse поддерживает сетевой протокол MySQL (MySQL wire protocol). Это позволяет отдельным клиентам, у которых нет нативных коннекторов для ClickHouse, использовать вместо них протокол MySQL. Работа была проверена со следующими BI-инструментами: - -- [Looker Studio](../integrations/data-visualization/looker-studio-and-clickhouse.md) -- [Tableau Online](../integrations/tableau-online) -- [QuickSight](../integrations/quicksight) - -Если вы пробуете другие, ещё не протестированные клиенты или интеграции, имейте в виду, что возможны следующие ограничения: - -- Реализация SSL может быть не полностью совместима; возможны проблемы с [TLS SNI](https://www.cloudflare.com/learning/ssl/what-is-sni/). -- Конкретному инструменту могут требоваться особенности диалекта (например, функции или настройки, специфичные для MySQL), которые ещё не реализованы. - -Если доступен нативный драйвер (например, [DBeaver](../integrations/dbeaver)), всегда предпочтительнее использовать его вместо интерфейса MySQL. Кроме того, хотя большинство клиентов MySQL должны работать корректно, интерфейс MySQL не гарантируется как полностью прозрачная замена для существующей кодовой базы с запросами MySQL. - -Если ваш сценарий использования подразумевает конкретный инструмент, для которого нет нативного драйвера ClickHouse, и вы хотите использовать его через интерфейс MySQL, но обнаружили определённые несовместимости — пожалуйста, [создайте issue](https://github.com/ClickHouse/ClickHouse/issues) в репозитории ClickHouse. - -::::note -Чтобы лучше поддерживать SQL-диалект указанных выше BI-инструментов, интерфейс MySQL в ClickHouse неявно выполняет запросы SELECT с настройкой [prefer_column_name_to_alias = 1](/operations/settings/settings#prefer_column_name_to_alias). -Эту настройку нельзя отключить, и в редких пограничных случаях она может приводить к отличиям в поведении между запросами, отправленными в обычный интерфейс запросов ClickHouse и интерфейс запросов MySQL. -:::: - - - -## Включение интерфейса MySQL в ClickHouse Cloud {#enabling-the-mysql-interface-on-clickhouse-cloud} - -1. После создания сервиса ClickHouse Cloud нажмите кнопку `Connect`. - -
- -Экран учетных данных — запрос - -2. Измените значение выпадающего списка `Connect with` на `MySQL`. - -
- -Экран учетных данных — выбран MySQL - -3. Переключите тумблер, чтобы включить интерфейс MySQL для этого сервиса. Это откроет порт `3306` для данного сервиса и отобразит экран подключения к MySQL, содержащий ваше уникальное имя пользователя MySQL. Пароль будет таким же, как пароль пользователя по умолчанию для сервиса. - -
- -Экран учетных данных — MySQL включен - -Скопируйте показанную строку подключения к MySQL. - -Экран учетных данных — строка подключения - - - -## Создание нескольких пользователей MySQL в ClickHouse Cloud {#creating-multiple-mysql-users-in-clickhouse-cloud} - -По умолчанию существует встроенный пользователь `mysql4`, который использует тот же пароль, что и пользователь `default`. Часть `` — это первый сегмент имени хоста вашего ClickHouse Cloud. Такой формат необходим для работы с инструментами, которые реализуют безопасное подключение, но не передают [SNI-информацию в своем TLS-рукопожатии](https://www.cloudflare.com/learning/ssl/what-is-sni), из-за чего невозможно выполнить внутреннюю маршрутизацию без дополнительной подсказки в имени пользователя (консольный клиент MySQL является одним из таких инструментов). - -По этой причине мы *настоятельно рекомендуем* использовать формат `mysql4_` при создании нового пользователя, предназначенного для работы через интерфейс MySQL, где `` — это подсказка для идентификации вашего облачного сервиса, а `` — произвольный суффикс по вашему выбору. - -:::tip -Для ClickHouse Cloud с именем хоста вида `foobar.us-east1.aws.clickhouse.cloud` часть `` равна `foobar`, и пользователь MySQL с произвольным именем может выглядеть как `mysql4foobar_team1`. -::: - -Вы можете создать дополнительных пользователей для работы через интерфейс MySQL, если, например, вам нужно применить дополнительные настройки. - -1. Необязательный шаг — создайте [профиль настроек](/sql-reference/statements/create/settings-profile), который будет применяться к вашему кастомному пользователю. Например, `my_custom_profile` с дополнительной настройкой, которая будет применяться по умолчанию при подключении с использованием пользователя, которого мы создадим позже: - - ```sql - CREATE SETTINGS PROFILE my_custom_profile SETTINGS prefer_column_name_to_alias=1; - ``` - - `prefer_column_name_to_alias` используется здесь только в качестве примера, вы можете использовать другие настройки. - -2. [Создайте пользователя](/sql-reference/statements/create/user), используя следующий формат: `mysql4_` ([см. выше](#creating-multiple-mysql-users-in-clickhouse-cloud)). Пароль должен быть в формате double SHA1. Например: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$'; - ``` - - или, если вы хотите использовать пользовательский профиль для этого пользователя: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$' SETTINGS PROFILE 'my_custom_profile'; - ``` - - где `my_custom_profile` — это имя профиля, созданного ранее. - -3. С помощью [GRANT](/sql-reference/statements/grant) выдайте новому пользователю необходимые разрешения для работы с нужными таблицами или базами данных. Например, если вы хотите предоставить доступ только к `system.query_log`: - - ```sql - GRANT SELECT ON system.query_log TO mysql4foobar_team1; - ``` - -4. Используйте созданного пользователя для подключения к вашему сервису ClickHouse Cloud через интерфейс MySQL. - -### Устранение неполадок при работе с несколькими пользователями MySQL в ClickHouse Cloud {#troubleshooting-multiple-mysql-users-in-clickhouse-cloud} - -Если вы создали нового пользователя MySQL и видите следующую ошибку при подключении через консольный клиент MySQL (CLI): - -```sql -ERROR 2013 (HY000): Потеряно соединение с сервером MySQL при 'чтении пакета авторизации', системная ошибка: 54 -``` - -В этом случае убедитесь, что имя пользователя имеет формат `mysql4_`, как описано ([выше](#creating-multiple-mysql-users-in-clickhouse-cloud)). - - -## Включение интерфейса MySQL в самостоятельно управляемом ClickHouse {#enabling-the-mysql-interface-on-self-managed-clickhouse} - -Добавьте параметр [mysql_port](../operations/server-configuration-parameters/settings.md#mysql_port) в файл конфигурации сервера. Например, вы можете указать порт в новом XML-файле в папке `config.d/` [folder](../operations/configuration-files): - -```xml - - 9004 - -``` - -Запустите сервер ClickHouse и найдите в журнале сообщение, похожее на следующее, в котором упоминается «Listening for MySQL compatibility protocol»: - -```bash -{} Application: Прослушивается протокол совместимости MySQL: 127.0.0.1:9004 -``` - - -## Подключение MySQL к ClickHouse {#connect-mysql-to-clickhouse} - -Следующая команда демонстрирует, как подключить клиент MySQL `mysql` к ClickHouse: - -```bash -mysql --protocol tcp -h [hostname] -u [username] -P [port_number] [database_name] -``` - -Например: - -```bash -$ mysql --protocol tcp -h 127.0.0.1 -u default -P 9004 default -``` - -Вывод в случае успешного подключения: - -```text -Добро пожаловать в монитор MySQL. Команды завершаются символом ; или \g. -Идентификатор вашего MySQL-подключения: 4 -Версия сервера: 20.2.1.1-ClickHouse - -Copyright (c) 2000, 2019, Oracle и/или аффилированные лица. Все права защищены. - -Oracle является зарегистрированным товарным знаком Oracle Corporation и/или -аффилированных лиц. Другие названия могут являться товарными знаками -соответствующих владельцев. - -Введите 'help;' или '\h' для получения справки. Введите '\c' для очистки текущей команды. - -mysql> -``` - -Для совместимости со всеми клиентами MySQL рекомендуется указывать пароль пользователя в конфигурационном файле с использованием [двойного SHA1](/operations/settings/settings-users#user-namepassword). -Если пароль пользователя указан с использованием [SHA256](/sql-reference/functions/hash-functions#SHA256), некоторые клиенты не смогут пройти аутентификацию (mysqljs и старые версии утилит командной строки MySQL и MariaDB). - -Ограничения: - -* подготовленные запросы не поддерживаются - -* некоторые типы данных отправляются как строки - -Чтобы отменить долгий запрос, используйте оператор `KILL QUERY connection_id` (во время выполнения он заменяется на `KILL QUERY WHERE query_id = connection_id`). Например: - -```bash -$ mysql --protocol tcp -h mysql_server -P 9004 default -u default --password=123 -e "KILL QUERY 123456;" -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md deleted file mode 100644 index 0e1252647c8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'Нативные клиенты и интерфейсы для ClickHouse' -keywords: ['клиенты', 'интерфейсы', 'CLI', 'SQL-консоль', 'драйверы'] -slug: /interfaces/natives-clients-and-interfaces -title: 'Нативные клиенты и интерфейсы' -doc_type: 'landing-page' ---- - -# Нативные клиенты и интерфейсы {#native-clients-interfaces} - -ClickHouse предоставляет несколько нативных клиентов и интерфейсов, которые позволяют подключаться к ClickHouse. - -Для получения дополнительной информации см. разделы ниже: - -| Раздел | Краткое описание | -|-------------------------------------------------------------|--------------------------------------------------------------------------------------| -| [Command-Line Client](/interfaces/cli) | Нативный клиент командной строки с поддержкой параметров командной строки и файлов конфигурации. | -| [Drivers & Interfaces](/interfaces/overview) | Набор сетевых интерфейсов, библиотек и графических интерфейсов. | -| [SQL Console](/integrations/sql-clients/sql-console) | Быстрый и простой способ работать с вашими данными в ClickHouse Cloud. | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/odbc.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/odbc.md deleted file mode 100644 index 6f9755b0633..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/odbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Документация по драйверу ODBC ClickHouse' -sidebar_label: 'Драйвер ODBC' -sidebar_position: 35 -slug: /interfaces/odbc -title: 'Драйвер ODBC' -doc_type: 'reference' ---- - -# Драйвер ODBC {#odbc-driver} - -Используйте [официальный драйвер ODBC](https://github.com/ClickHouse/clickhouse-odbc) для доступа к ClickHouse как к источнику данных. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/overview.md deleted file mode 100644 index 608a897d0a7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/overview.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Обзор сетевых интерфейсов, драйверов и инструментов для подключения к - ClickHouse' -keywords: ['clickhouse', 'network', 'interfaces', 'http', 'tcp', 'grpc', 'command-line', - 'client', 'jdbc', 'odbc', 'driver'] -sidebar_label: 'Обзор' -slug: /interfaces/overview -title: 'Драйверы и интерфейсы' -doc_type: 'reference' ---- - -# Драйверы и интерфейсы {#drivers-and-interfaces} - -ClickHouse предоставляет два сетевых интерфейса (их при необходимости можно использовать поверх TLS для дополнительной безопасности): - -* [HTTP](http.md), который хорошо документирован и прост для прямого использования. -* [Нативный TCP](../interfaces/tcp.md), который имеет меньшие накладные расходы. - -В большинстве случаев рекомендуется использовать соответствующий инструмент или библиотеку вместо прямого взаимодействия с этими интерфейсами. В ClickHouse официально поддерживаются следующие инструменты: - -* [Клиент командной строки](../interfaces/cli.md) -* [JDBC-драйвер](../interfaces/jdbc.md) -* [ODBC-драйвер](../interfaces/odbc.md) -* [Клиентская библиотека C++](../interfaces/cpp.md) - -ClickHouse также поддерживает два RPC-протокола: - -* [Протокол gRPC](grpc.md), специально разработанный для ClickHouse. -* [Apache Arrow Flight](arrowflight.md). - -Сервер ClickHouse предоставляет встроенные визуальные интерфейсы для опытных пользователей: - -* Play UI: откройте `/play` в браузере; -* Advanced Dashboard: откройте `/dashboard` в браузере; -* Просмотрщик бинарных символов для инженеров ClickHouse: откройте `/binary` в браузере; - -Существует также широкий спектр сторонних библиотек для работы с ClickHouse: - -* [Клиентские библиотеки](../interfaces/third-party/client-libraries.md) -* [Интеграции](../interfaces/third-party/integrations.md) -* [Визуальные интерфейсы](../interfaces/third-party/gui.md) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/postgresql.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/postgresql.md deleted file mode 100644 index 1cac5e384cd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/postgresql.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'Документация по интерфейсу сетевого протокола PostgreSQL в ClickHouse' -sidebar_label: 'Интерфейс PostgreSQL' -sidebar_position: 20 -slug: /interfaces/postgresql -title: 'Интерфейс PostgreSQL' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# Интерфейс PostgreSQL {#postgresql-interface} - - - -ClickHouse поддерживает сетевой протокол PostgreSQL (PostgreSQL wire protocol), что позволяет использовать клиентские приложения PostgreSQL для подключения к ClickHouse. В определённом смысле ClickHouse может притворяться экземпляром PostgreSQL, позволяя подключать к ClickHouse клиентские приложения PostgreSQL, которые ещё не поддерживаются ClickHouse напрямую (например, Amazon Redshift). - -Чтобы включить сетевой протокол PostgreSQL, добавьте настройку [postgresql_port](../operations/server-configuration-parameters/settings.md#postgresql_port) в конфигурационный файл сервера. Например, вы можете задать порт в новом XML-файле в папке `config.d`: - -```xml - - 9005 - -``` - -Запустите сервер ClickHouse и найдите в журнале сообщение, аналогичное следующему, в котором упоминается **Listening for PostgreSQL compatibility protocol**: - -```response -{} Application: Прослушивание протокола совместимости PostgreSQL: 127.0.0.1:9005 -``` - -## Подключение psql к ClickHouse {#connect-psql-to-clickhouse} - -Следующая команда демонстрирует, как подключиться к ClickHouse с помощью клиента PostgreSQL `psql`: - -```bash -psql -p [port] -h [hostname] -U [username] [database_name] -``` - -Например: - -```bash -psql -p 9005 -h 127.0.0.1 -U alice default -``` - -:::note -Клиент `psql` требует аутентификации с паролем, поэтому вы не сможете подключиться, используя пользователя `default` без пароля. Либо задайте пароль пользователю `default`, либо войдите под другим пользователем. -::: - -Клиент `psql` запрашивает ввод пароля: - -```response -Пароль для пользователя alice: -psql (14.2, server 22.3.1.1) -ПРЕДУПРЕЖДЕНИЕ: мажорная версия psql 14, мажорная версия сервера 22. - Некоторые функции psql могут работать некорректно. -Введите "help" для вызова справки. - -default=> -``` - -И готово! Теперь у вас есть клиент PostgreSQL, подключённый к ClickHouse, и все команды и запросы выполняются на стороне ClickHouse. - -:::note -В настоящее время протокол PostgreSQL поддерживает только пароли в открытом виде (plain-text). -::: - -## Использование SSL {#using-ssl} - -Если в вашем инстансе ClickHouse настроен SSL/TLS, то `postgresql_port` будет использовать те же настройки (порт общий как для защищённых, так и для незащищённых клиентов). - -У каждого клиента свой способ подключения по SSL. Следующая команда демонстрирует, как передать сертификаты и ключ для безопасного подключения `psql` к ClickHouse: - -```bash -psql "port=9005 host=127.0.0.1 user=alice dbname=default sslcert=/path/to/certificate.pem sslkey=/path/to/key.pem sslrootcert=/path/to/rootcert.pem sslmode=verify-ca" -``` - -## Настройка аутентификации пользователей ClickHouse с использованием SCRAM-SHA-256 {#using-scram-sha256} - -Чтобы обеспечить безопасную аутентификацию пользователей в ClickHouse, рекомендуется использовать протокол SCRAM-SHA-256. Настройте пользователя, указав элемент `password_scram_sha256_hex` в файле users.xml. Хеш пароля должен быть сгенерирован с параметром num_iterations=4096. - -Убедитесь, что клиент psql поддерживает SCRAM-SHA-256 и использует его при согласовании параметров подключения. - -Пример конфигурации для пользователя `user_with_sha256` с паролем `abacaba`: - -```xml - - 04e7a70338d7af7bb6142fe7e19fef46d9b605f3e78b932a60e8200ef9154976 - -``` - -См. [документацию PostgreSQL](https://jdbc.postgresql.org/documentation/head/ssl-client.html) для получения дополнительной информации о настройках SSL. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/prometheus.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/prometheus.md deleted file mode 100644 index fa881cb6f52..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/prometheus.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'Документация по поддержке протокола Prometheus в ClickHouse' -sidebar_label: 'Протоколы Prometheus' -sidebar_position: 19 -slug: /interfaces/prometheus -title: 'Протоколы Prometheus' -doc_type: 'reference' ---- - -# Протоколы Prometheus {#prometheus-protocols} - -## Предоставление метрик {#expose} - -:::note -Если вы используете ClickHouse Cloud, вы можете передавать метрики в Prometheus с помощью [Prometheus Integration](/integrations/prometheus). -::: - -ClickHouse может предоставлять собственные метрики для сбора Prometheus: - -````xml - - 9363 - /metrics - true - true - true - true - true - true - - -Секция `` может использоваться для создания расширенных обработчиков. -Эта секция аналогична [](/interfaces/http), но работает с протоколами prometheus: - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - -```` - -Настройки: - -| Name | Default | Description | -| ---------------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `port` | none | Порт для публикации метрик. | -| `endpoint` | `/metrics` | HTTP-эндпоинт для сбора метрик сервером Prometheus. Должен начинаться с `/`. Не должен использоваться совместно с разделом ``. | -| `url` / `headers` / `method` | none | Фильтры, используемые для поиска обработчика, соответствующего запросу. Аналогичны полям с теми же именами в разделе [``](/interfaces/http). | -| `metrics` | true | Экспортировать метрики из таблицы [system.metrics](/operations/system-tables/metrics). | -| `asynchronous_metrics` | true | Экспортировать текущие значения метрик из таблицы [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics). | -| `events` | true | Экспортировать метрики из таблицы [system.events](/operations/system-tables/events). | -| `errors` | true | Экспортировать количество ошибок по кодам ошибок, произошедших с момента последнего перезапуска сервера. Эту информацию также можно получить из таблицы [system.errors](/operations/system-tables/errors). | -| `histograms` | true | Экспортировать гистограммные метрики из [system.histogram_metrics](/operations/system-tables/histogram_metrics). | -| `dimensional_metrics` | true | Экспортировать размерные метрики из [system.dimensional_metrics](/operations/system-tables/dimensional_metrics). | - -Проверьте (замените `127.0.0.1` на IP-адрес или имя хоста вашего сервера ClickHouse): - -```bash -curl 127.0.0.1:9363/metrics -``` - -## Протокол remote-write {#remote-write} - -ClickHouse поддерживает протокол [remote-write](https://prometheus.io/docs/specs/remote_write_spec/). -Данные принимаются с использованием этого протокола и записываются в таблицу [TimeSeries](/engines/table-engines/special/time_series) -(которую необходимо создать заранее). - -```xml - - 9363 - - - /write - - remote_write - db_name - time_series_table
-
-
-
-
-``` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `port` | none | Порт для обработки протокола `remote-write`. | -| `url` / `headers` / `method` | none | Фильтры, используемые для поиска подходящего обработчика запроса. Аналогичны полям с теми же именами в разделе [``](/interfaces/http). | -| `table` | none | Имя таблицы [TimeSeries](/engines/table-engines/special/time_series), в которую записываются данные, полученные по протоколу `remote-write`. Это имя при необходимости также может включать имя базы данных. | -| `database` | none | Имя базы данных, в которой находится таблица, указанная в настройке `table`, если оно не указано в самой настройке `table`. | - -## Протокол remote-read {#remote-read} - -ClickHouse поддерживает протокол [remote-read](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/). -Данные читаются из таблицы [TimeSeries](/engines/table-engines/special/time_series) и передаются по этому протоколу. - -```xml - - 9363 - - - /read - - remote_read - db_name - time_series_table
-
-
-
-
-``` - -Параметры: - -| Name | Default | Description | -| ---------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `port` | none | Порт для обработки протокола `remote-read`. | -| `url` / `headers` / `method` | none | Фильтры, используемые для поиска обработчика, соответствующего запросу. Аналогичны полям с теми же именами в разделе [``](/interfaces/http). | -| `table` | none | Имя таблицы [TimeSeries](/engines/table-engines/special/time_series), из которой читаются данные для отправки по протоколу `remote-read`. При необходимости это имя может включать имя базы данных. | -| `database` | none | Имя базы данных, в которой находится таблица, указанная в параметре `table`, если оно не указано в значении параметра `table`. | - -## Конфигурация нескольких протоколов {#multiple-protocols} - -Несколько протоколов можно задать вместе в одном конфигурационном блоке: - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - /write - - remote_write - db_name.time_series_table
-
-
- - /read - - remote_read - db_name.time_series_table
-
-
-
-
-``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md deleted file mode 100644 index 4a059e5217e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md +++ /dev/null @@ -1,2384 +0,0 @@ ---- -description: 'Страница, описывающая автоматическое определение схемы по входным данным в ClickHouse' -sidebar_label: 'Определение схемы' -slug: /interfaces/schema-inference -title: 'Автоматическое определение схемы по входным данным' -doc_type: 'reference' ---- - -ClickHouse может автоматически определять структуру входных данных практически во всех поддерживаемых [входных форматах](formats.md). -В этом документе описано, когда используется автоматическое определение схемы, как оно работает с различными входными форматами и какие настройки -его контролируют. - - - -## Использование {#usage} - -Автоматическое определение схемы используется, когда ClickHouse должен прочитать данные в определённом формате, но их структура неизвестна. - - - -## Табличные функции [file](../sql-reference/table-functions/file.md), [s3](../sql-reference/table-functions/s3.md), [url](../sql-reference/table-functions/url.md), [hdfs](../sql-reference/table-functions/hdfs.md), [azureBlobStorage](../sql-reference/table-functions/azureBlobStorage.md). {#table-functions-file-s3-url-hdfs-azureblobstorage} - -Эти табличные функции имеют необязательный аргумент `structure`, задающий структуру входных данных. Если этот аргумент не указан или имеет значение `auto`, структура будет выведена из данных. - -**Пример:** - -Предположим, у нас есть файл `hobbies.jsonl` в формате JSONEachRow в каталоге `user_files` со следующим содержимым: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["футбол", "кулинария", "музыка"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["теннис", "искусство"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["фитнес", "чтение", "шопинг"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["кино", "прыжки с парашютом"]} -``` - -ClickHouse может читать эти данные без необходимости задавать их структуру: - -```sql -SELECT * FROM file('hobbies.jsonl') -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -Примечание: формат `JSONEachRow` был автоматически определён на основе расширения файла `.jsonl`. - -Вы можете увидеть автоматически определённую структуру с помощью запроса `DESCRIBE`: - -```sql -DESCRIBE file('hobbies.jsonl') -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## Движки таблиц [File](../engines/table-engines/special/file.md), [S3](../engines/table-engines/integrations/s3.md), [URL](../engines/table-engines/special/url.md), [HDFS](../engines/table-engines/integrations/hdfs.md), [azureBlobStorage](../engines/table-engines/integrations/azureBlobStorage.md) {#table-engines-file-s3-url-hdfs-azureblobstorage} - -Если в запросе `CREATE TABLE` не указан список столбцов, структура таблицы будет автоматически определена по данным. - -**Пример:** - -Используем файл `hobbies.jsonl`. Мы можем создать таблицу с движком `File` на основе данных из этого файла: - -```sql -CREATE TABLE hobbies ENGINE=File(JSONEachRow, 'hobbies.jsonl') -``` - -```response -Ок. -``` - -```sql -SELECT * FROM hobbies -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -```sql -DESCRIBE TABLE hobbies -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## clickhouse-local {#clickhouse-local} - -У `clickhouse-local` есть необязательный параметр `-S/--structure`, задающий структуру входных данных. Если этот параметр не указан или имеет значение `auto`, структура будет определена по данным. - -**Пример:** - -Возьмём файл `hobbies.jsonl`. Мы можем выполнить запрос к данным из этого файла с помощью `clickhouse-local`: - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='DESCRIBE TABLE hobbies' -``` - -```response -id Nullable(Int64) -age Nullable(Int64) -name Nullable(String) -hobbies Array(Nullable(String)) -``` - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='SELECT * FROM hobbies' -``` - -```response -1 25 Josh ['football','cooking','music'] -2 19 Alan ['tennis','art'] -3 32 Lana ['fitness','reading','shopping'] -4 47 Brayan ['movies','skydiving'] -``` - - -## Использование структуры из таблицы-вставки {#using-structure-from-insertion-table} - -Когда табличные функции `file/s3/url/hdfs` используются для вставки данных в таблицу, -можно использовать структуру из таблицы-вставки вместо извлечения её из данных. -Это может улучшить производительность вставки, поскольку определение схемы может занимать некоторое время. Кроме того, это будет полезно, когда у таблицы оптимизированная схема, так что -не будут выполняться преобразования между типами. - -Существует специальная настройка [use_structure_from_insertion_table_in_table_functions](/operations/settings/settings.md/#use_structure_from_insertion_table_in_table_functions), -которая управляет этим поведением. Она может принимать три значения: - -* 0 — табличная функция будет извлекать структуру из данных. -* 1 — табличная функция будет использовать структуру из таблицы-вставки. -* 2 — ClickHouse автоматически определит, возможно ли использовать структуру из таблицы-вставки, или необходимо определять схему по данным. Значение по умолчанию. - -**Пример 1:** - -Создадим таблицу `hobbies1` со следующей структурой: - -```sql -CREATE TABLE hobbies1 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `name` String, - `hobbies` Array(String) -) -ENGINE = MergeTree -ORDER BY id; -``` - -И загрузите данные из файла `hobbies.jsonl`: - -```sql -INSERT INTO hobbies1 SELECT * FROM file(hobbies.jsonl) -``` - -В этом случае все столбцы из файла вставляются в таблицу без изменений, поэтому ClickHouse будет использовать структуру таблицы, в которую выполняется вставка, вместо автоматического определения схемы. - -**Пример 2:** - -Создадим таблицу `hobbies2` со следующей структурой: - -```sql -CREATE TABLE hobbies2 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -А затем вставьте данные из файла `hobbies.jsonl`: - -```sql -INSERT INTO hobbies2 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -В этом случае все столбцы в запросе `SELECT` присутствуют в таблице, поэтому ClickHouse будет использовать структуру из таблицы, в которую выполняется вставка. -Обратите внимание, что это будет работать только для входных форматов, поддерживающих чтение подмножества столбцов, таких как JSONEachRow, TSKV, Parquet и т. д. (то есть, например, это не будет работать для формата TSV). - -**Пример 3:** - -Создадим таблицу `hobbies3` со следующей структурой: - -```sql -CREATE TABLE hobbies3 -( - `identifier` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY identifier; -``` - -Теперь вставьте данные из файла `hobbies.jsonl`: - -```sql -INSERT INTO hobbies3 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -В этом случае столбец `id` используется в запросе `SELECT`, но в таблице нет этого столбца (в ней есть столбец с именем `identifier`), -поэтому ClickHouse не может использовать структуру таблицы, в которую выполняется вставка, и будет применено автоматическое определение схемы. - -**Пример 4:** - -Создадим таблицу `hobbies4` со следующей структурой: - -```sql -CREATE TABLE hobbies4 -( - `id` UInt64, - `any_hobby` Nullable(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -И вставьте данные из файла `hobbies.jsonl`: - -```sql -INSERT INTO hobbies4 SELECT id, empty(hobbies) ? NULL : hobbies[1] FROM file(hobbies.jsonl) -``` - -В этом случае в запросе `SELECT` выполняются некоторые операции со столбцом `hobbies` перед его вставкой в таблицу, поэтому ClickHouse не может использовать структуру целевой таблицы и будет использовано автоматическое определение схемы. - - -## Кэш автоопределения схемы {#schema-inference-cache} - -Для большинства форматов ввода автоопределение схемы читает часть данных, чтобы определить их структуру, и этот процесс может занять некоторое время. -Чтобы не определять одну и ту же схему каждый раз, когда ClickHouse читает данные из одного и того же файла, полученная схема кэшируется, и при повторном обращении к тому же файлу ClickHouse использует схему из кэша. - -Существуют специальные настройки, управляющие этим кэшем: -- `schema_inference_cache_max_elements_for_{file/s3/hdfs/url/azure}` — максимальное количество кэшируемых схем для соответствующей табличной функции. Значение по умолчанию — `4096`. Эти настройки следует задавать в конфигурации сервера. -- `schema_inference_use_cache_for_{file,s3,hdfs,url,azure}` — позволяет включать/отключать использование кэша для автоопределения схемы. Эти настройки можно указывать в запросах. - -Схема файла может быть изменена путём модификации данных или изменением настроек формата. -По этой причине кэш автоопределения схемы идентифицирует схему по источнику файла, имени формата, используемым настройкам формата и времени последней модификации файла. - -Примечание: некоторые файлы, к которым осуществляется доступ по URL в табличной функции `url`, могут не содержать информации о времени последней модификации; для этого случая существует специальная настройка -`schema_inference_cache_require_modification_time_for_url`. Отключение этой настройки позволяет использовать схему из кэша без учёта времени последней модификации для таких файлов. - -Также существует системная таблица [schema_inference_cache](../operations/system-tables/schema_inference_cache.md) со всеми текущими схемами в кэше и системный запрос `SYSTEM DROP SCHEMA CACHE [FOR File/S3/URL/HDFS]`, -который позволяет очистить кэш схем для всех источников или для конкретного источника. - -**Примеры:** - -Попробуем определить структуру примерного набора данных из S3 `github-2022.ndjson.gz` и посмотрим, как работает кэш автоопределения схемы: - - - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -5 строк в наборе. Затрачено: 0.601 сек. -``` - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -``` - -5 строк в наборе. Затрачено: 0,059 сек. - -```` - -Как видите, второй запрос выполнился практически мгновенно. - -Попробуем изменить некоторые настройки, которые могут повлиять на автоматически определённую схему: - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -SETTINGS input_format_json_try_infer_named_tuples_from_objects=0, input_format_json_read_objects_as_strings = 1 - -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ type │ Nullable(String) │ │ │ │ │ │ -│ actor │ Nullable(String) │ │ │ │ │ │ -│ repo │ Nullable(String) │ │ │ │ │ │ -│ created_at │ Nullable(String) │ │ │ │ │ │ -│ payload │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -Получено 5 строк. Затрачено: 0.611 сек -```` - -Как видите, схема из кэша не была использована для того же файла, потому что была изменена настройка, которая может влиять на автоматически определяемую схему. - -Проверим содержимое таблицы `system.schema_inference_cache`: - -```sql -SELECT schema, format, source FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─schema──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─format─┬─source───────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ type Nullable(String), actor Tuple(avatar_url Nullable(String), display_login Nullable(String), id Nullable(Int64), login Nullable(String), url Nullable(String)), repo Tuple(id Nullable(Int64), name Nullable(String), url Nullable(String)), created_at Nullable(String), payload Tuple(action Nullable(String), distinct_size Nullable(Int64), pull_request Tuple(author_association Nullable(String), base Tuple(ref Nullable(String), sha Nullable(String)), head Tuple(ref Nullable(String), sha Nullable(String)), number Nullable(Int64), state Nullable(String), title Nullable(String), updated_at Nullable(String), user Tuple(login Nullable(String))), ref Nullable(String), ref_type Nullable(String), size Nullable(Int64)) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -│ type Nullable(String), actor Nullable(String), repo Nullable(String), created_at Nullable(String), payload Nullable(String) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Как видите, для одного и того же файла существуют две разные схемы. - -Мы можем очистить кэш схем с помощью системного запроса: - -```sql -SYSTEM DROP SCHEMA CACHE FOR S3 -``` - -```response -Готово. -``` - -```sql -SELECT count() FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─count()─┐ -│ 0 │ -└─────────┘ -``` - - -## Текстовые форматы {#text-formats} - -Для текстовых форматов ClickHouse читает данные построчно, извлекает значения столбцов в соответствии с форматом, -а затем использует рекурсивные парсеры и эвристики, чтобы определить тип для каждого значения. Максимальное количество строк и байт, читаемых из данных при определении схемы, -управляется настройками `input_format_max_rows_to_read_for_schema_inference` (по умолчанию 25000) и `input_format_max_bytes_to_read_for_schema_inference` (по умолчанию 32 МБ). -По умолчанию все определённые типы являются [Nullable](../sql-reference/data-types/nullable.md), но вы можете изменить это, настроив `schema_inference_make_columns_nullable` (см. примеры в разделе [настройках](#settings-for-text-formats)). - -### Форматы JSON {#json-formats} - -В форматах JSON ClickHouse разбирает значения в соответствии со спецификацией JSON, а затем пытается подобрать для них наиболее подходящий тип данных. - -Рассмотрим, как это работает, какие типы могут быть выведены и какие отдельные настройки могут использоваться в форматах JSON. - -**Примеры** - -Здесь и далее в примерах будет использоваться табличная функция [format](../sql-reference/table-functions/format.md). - -Целые числа, числа с плавающей запятой, логические значения, строки: - -```sql -DESC format(JSONEachRow, '{"int" : 42, "float" : 42.42, "string" : "Hello, World!"}'); -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date и DateTime: - -```sql -DESC format(JSONEachRow, '{"date" : "2022-01-01", "datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"}') -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Массивы: - -```sql -DESC format(JSONEachRow, '{"arr" : [1, 2, 3], "nested_arrays" : [[1, 2, 3], [4, 5, 6], []]}') -``` - -```response -┌─name──────────┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ nested_arrays │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└───────────────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если массив содержит `null`, ClickHouse определит тип по другим элементам массива: - -```sql -DESC format(JSONEachRow, '{"arr" : [null, 42, null]}') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -Если массив содержит значения разных типов и параметр `input_format_json_infer_array_of_dynamic_from_array_of_different_types` включён (по умолчанию он включён), то его тип будет `Array(Dynamic)`: - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=1; -DESC format(JSONEachRow, '{"arr" : [42, "hello", [1, 2, 3]]}'); -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Dynamic) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Именованные кортежи: - -Когда настройка `input_format_json_try_infer_named_tuples_from_objects` включена, во время определения схемы ClickHouse пытается вывести именованный Tuple из объектов JSON. -Получившийся именованный Tuple будет содержать все элементы из всех соответствующих объектов JSON во входной выборке данных. - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Неименованные кортежи: - -Если параметр `input_format_json_infer_array_of_dynamic_from_array_of_different_types` отключён, массивы с элементами разных типов рассматриваются как неименованные кортежи в форматах JSON. - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types = 0; -DESC format(JSONEachRow, '{"tuple" : [1, "Hello, World!", [1, 2, 3]]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если некоторые значения равны `null` или пусты, мы используем типы соответствующих значений из других строк: - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=0; -DESC format(JSONEachRow, $$ - {"tuple" : [1, null, null]} - {"tuple" : [null, "Hello, World!", []]} - {"tuple" : [null, null, [1, 2, 3]]} - $$) -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Map: - -В JSON можно читать объекты, значения которых имеют один и тот же тип, как значения типа Map. -Примечание: это будет работать только в том случае, если настройки `input_format_json_read_objects_as_strings` и `input_format_json_try_infer_named_tuples_from_objects` отключены. - - -```sql -SET input_format_json_read_objects_as_strings = 0, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, '{"map" : {"key1" : 42, "key2" : 24, "key3" : 4}}') -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ map │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Вложенные сложные типы данных: - -```sql -DESC format(JSONEachRow, '{"value" : [[[42, 24], []], {"key1" : 42, "key2" : 24}]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Tuple(Array(Array(Nullable(String))), Tuple(key1 Nullable(Int64), key2 Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если ClickHouse не может определить тип для некоторого ключа, потому что данные содержат только значения null/пустые объекты/пустые массивы, будет использован тип `String`, если включена настройка `input_format_json_infer_incomplete_types_as_strings`, в противном случае будет сгенерировано исключение: - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 1; -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(String)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 0; -``` - -```response -Код: 652. DB::Exception: Получено от localhost:9000. DB::Exception: -Невозможно определить тип для столбца 'arr' по первой строке данных, -вероятно, этот столбец содержит только значения Null или пустые массивы/словари. -... -``` - -#### Настройки JSON {#json-settings} - -##### input_format_json_try_infer_numbers_from_strings {#input_format_json_try_infer_numbers_from_strings} - -Включение этого параметра позволяет распознавать числа в строковых значениях. - -По умолчанию этот параметр отключен. - -**Пример:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : "42"} - {"value" : "424242424242"} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_try_infer_named_tuples_from_objects {#input_format_json_try_infer_named_tuples_from_objects} - -Включение этого параметра позволяет выводить именованные `Tuple` из JSON-объектов. Получившийся именованный `Tuple` будет содержать все элементы из всех соответствующих JSON-объектов из выборки данных. -Это может быть полезно, когда JSON-данные не разрежены, и выборка данных содержит все возможные ключи объектов. - -Этот параметр включен по умолчанию. - -**Пример** - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -Результат: - - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"array" : [{"a" : 42, "b" : "Hello"}, {}, {"c" : [1,2,3]}, {"d" : "2020-01-01"}]}') -``` - -Результат: - -```markdown -┌─name──┬─type────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ array │ Array(Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Nullable(Date))) │ │ │ │ │ │ -└───────┴─────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects {#input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects} - -Включение этого параметра позволяет использовать тип String для неоднозначных путей при выводе именованных кортежей из JSON-объектов (когда `input_format_json_try_infer_named_tuples_from_objects` включён) вместо выбрасывания исключения. -Это позволяет читать JSON-объекты как именованные кортежи, даже если присутствуют неоднозначные пути. - -По умолчанию параметр отключён. - -**Примеры** - -При отключённом параметре: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 0; -DESC format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -Результат: - -```response -Код: 636. DB::Exception: Не удалось извлечь структуру таблицы из файла формата JSONEachRow. Ошибка: -Код: 117. DB::Exception: JSON-объекты содержат неоднозначные данные: в некоторых объектах путь 'a' имеет тип 'Int64', а в других — 'Tuple(b String)'. Вы можете включить параметр input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects, чтобы использовать тип String для пути 'a'. (INCORRECT_DATA) (версия 24.3.1.1). -Структуру можно указать вручную. (CANNOT_EXTRACT_TABLE_STRUCTURE) -``` - -При включённой настройке: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : "a" : 42}, {"obj" : {"a" : {"b" : "Hello"}}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -Результат: - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(String)) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -┌─obj─────────────────┐ -│ ('42') │ -│ ('{"b" : "Hello"}') │ -└─────────────────────┘ -``` - -##### input_format_json_read_objects_as_strings {#input_format_json_read_objects_as_strings} - -Включение этой настройки позволяет читать вложенные объекты JSON как строки. -Эту настройку можно использовать для чтения вложенных объектов JSON без использования типа JSON Object. - -Эта настройка включена по умолчанию. - -Примечание: включение этой настройки будет иметь эффект только в том случае, если настройка `input_format_json_try_infer_named_tuples_from_objects` отключена. - - -```sql -SET input_format_json_read_objects_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, $$ - {"obj" : {"key1" : 42, "key2" : [1,2,3,4]}} - {"obj" : {"key3" : {"nested_key" : 1}}} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_numbers_as_strings {#input_format_json_read_numbers_as_strings} - -Включение этого параметра позволяет считывать числовые значения в виде строк. - -Этот параметр включен по умолчанию. - -**Пример** - -```sql -SET input_format_json_read_numbers_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : 1055} - {"value" : "unknown"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_numbers {#input_format_json_read_bools_as_numbers} - -Включение этого параметра позволяет считывать логические значения типа Bool как числа. - -Этот параметр включён по умолчанию. - -**Пример:** - -```sql -SET input_format_json_read_bools_as_numbers = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : 42} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_strings {#input_format_json_read_bools_as_strings} - -Включение этого параметра позволяет считывать логические значения типа Bool как строки. - -Этот параметр включён по умолчанию. - -**Пример:** - -```sql -SET input_format_json_read_bools_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : "Hello, World"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_arrays_as_strings {#input_format_json_read_arrays_as_strings} - -Включение этого параметра позволяет считывать значения JSON-массивов как строки. - -Этот параметр включён по умолчанию. - -**Пример** - -```sql -SET input_format_json_read_arrays_as_strings = 1; -SELECT arr, toTypeName(arr), JSONExtractArrayRaw(arr)[3] from format(JSONEachRow, 'arr String', '{"arr" : [1, "Hello", [1,2,3]]}'); -``` - -```response -┌─arr───────────────────┬─toTypeName(arr)─┬─arrayElement(JSONExtractArrayRaw(arr), 3)─┐ -│ [1, "Hello", [1,2,3]] │ String │ [1,2,3] │ -└───────────────────────┴─────────────────┴───────────────────────────────────────────┘ -``` - -##### input_format_json_infer_incomplete_types_as_strings {#input_format_json_infer_incomplete_types_as_strings} - - -Включение этого параметра позволяет использовать тип данных String для JSON-ключей, которые в выборке данных при определении схемы содержат только `Null`/`{}`/`[]`. -В JSON-форматах любое значение может быть считано как String, если включены все соответствующие настройки (по умолчанию они включены), и мы можем избежать ошибок вида `Cannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps` при определении схемы, используя тип String для ключей с неизвестными типами. - -Пример: - -```sql -SET input_format_json_infer_incomplete_types_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 1; -DESCRIBE format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -``` - -Результат: - -```markdown -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Array(Nullable(Int64)), b Nullable(String), c Nullable(String), d Nullable(String), e Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -┌─obj────────────────────────────┐ -│ ([1,2,3],'hello',NULL,'{}',[]) │ -└────────────────────────────────┘ -``` - -### CSV {#csv} - -В формате CSV ClickHouse извлекает значения столбцов из строки в соответствии с разделителями. ClickHouse ожидает, что все типы, кроме чисел и строк, будут заключены в двойные кавычки. Если значение находится в двойных кавычках, ClickHouse пытается разобрать -данные внутри кавычек с помощью рекурсивного парсера, а затем определить для них наиболее подходящий тип данных. Если значение не в двойных кавычках, ClickHouse пытается разобрать его как число, -и если значение не является числом, ClickHouse рассматривает его как строку. - -Если вы не хотите, чтобы ClickHouse пытался определять сложные типы с помощью различных парсеров и эвристик, вы можете отключить настройку `input_format_csv_use_best_effort_in_schema_inference`, -и ClickHouse будет рассматривать все столбцы как строки (String). - -Если настройка `input_format_csv_detect_header` включена, ClickHouse попытается обнаружить заголовок с именами столбцов (и, возможно, типами) при определении схемы. Эта настройка включена по умолчанию. - -**Примеры:** - -Целые числа (Integers), числа с плавающей запятой (Floats), булевы значения (Bools), строки (Strings): - -```sql -DESC format(CSV, '42,42.42,true,"Hello,World!"') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Строки без кавычек: - -```sql -DESC format(CSV, 'Hello world!,World hello!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date и DateTime: - - -```sql -DESC format(CSV, '"2020-01-01","2020-01-01 00:00:00","2022-01-01 00:00:00.000"') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Массивы: - -```sql -DESC format(CSV, '"[1,2,3]","[[1, 2], [], [3, 4]]"') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(CSV, $$"['Hello', 'world']","[['Abc', 'Def'], []]"$$) -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если массив содержит null, ClickHouse будет определять тип по другим элементам массива: - -```sql -DESC format(CSV, '"[NULL, 42, NULL]"') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Карты: - -```sql -DESC format(CSV, $$"{'key1' : 42, 'key2' : 24}"$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Вложенные массивы и отображения (Map): - -```sql -DESC format(CSV, $$"[{'key1' : [[42, 42], []], 'key2' : [[null], [42]]}]"$$) -``` - -```response -┌─name─┬─type──────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Array(Nullable(Int64))))) │ │ │ │ │ │ -└──────┴───────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -Если ClickHouse не может определить тип значения в кавычках, потому что данные содержат только значения NULL, он будет интерпретировать его как String: - -```sql -DESC format(CSV, '"[NULL, NULL]"') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Пример с отключённым параметром `input_format_csv_use_best_effort_in_schema_inference`: - -```sql -SET input_format_csv_use_best_effort_in_schema_inference = 0 -DESC format(CSV, '"[1,2,3]",42.42,Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Примеры автоматического определения заголовков (при включённом `input_format_csv_detect_header`): - -Только имена столбцов: - -```sql -SELECT * FROM format(CSV, -$$"number","string","array" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -Имена и типы: - -```sql -DESC format(CSV, -$$"number","string","array" -"UInt32","String","Array(UInt16)" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Обратите внимание, что заголовок может быть распознан только в том случае, если есть хотя бы один столбец с типом, отличным от String. Если все столбцы имеют тип String, заголовок не распознаётся: - -```sql -SELECT * FROM format(CSV, -$$"first_column","second_column" -"Hello","World" -"World","Hello" -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ first_column │ second_column │ -│ Hello │ World │ -│ World │ Hello │ -└──────────────┴───────────────┘ -``` - -#### Настройки CSV {#csv-settings} - -##### input_format_csv_try_infer_numbers_from_strings {#input_format_csv_try_infer_numbers_from_strings} - -Включение этого параметра позволяет выводить числовые значения из строковых. - -По умолчанию этот параметр отключён. - -**Пример:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(CSV, '42,42.42'); -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -### TSV/TSKV {#tsv-tskv} - -В форматах TSV/TSKV ClickHouse извлекает значение столбца из строки в соответствии с табуляцией как разделителем, а затем разбирает извлечённое значение с помощью -рекурсивного парсера, чтобы определить наиболее подходящий тип. Если тип не удаётся определить, ClickHouse рассматривает это значение как String. - -Если вы не хотите, чтобы ClickHouse пытался определять сложные типы с использованием парсеров и эвристик, вы можете отключить настройку `input_format_tsv_use_best_effort_in_schema_inference`, -и ClickHouse будет рассматривать все столбцы как строки (String). - -Если настройка `input_format_tsv_detect_header` включена, ClickHouse попытается обнаружить заголовок с именами столбцов (и, возможно, типами) при определении схемы. Эта настройка включена по умолчанию. - -**Примеры:** - -Integers, Floats, Bools, Strings: - -```sql -DESC format(TSV, '42 42.42 true Hello,World!') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSKV, 'int=42 float=42.42 bool=true string=Hello,World!\n') -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Даты и время: - -```sql -DESC format(TSV, '2020-01-01 2020-01-01 00:00:00 2022-01-01 00:00:00.000') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -Массивы: - -```sql -DESC format(TSV, '[1,2,3] [[1, 2], [], [3, 4]]') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSV, '[''Hello'', ''world''] [[''Abc'', ''Def''], []]') -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если массив содержит значение null, ClickHouse определит тип по другим элементам массива: - -```sql -DESC format(TSV, '[NULL, 42, NULL]') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Кортежи: - -```sql -DESC format(TSV, $$(42, 'Hello, world!')$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Карты: - -```sql -DESC format(TSV, $${'key1' : 42, 'key2' : 24}$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Вложенные массивы, кортежи и отображения: - -```sql -DESC format(TSV, $$[{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}]$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -Если ClickHouse не может определить тип данных, потому что данные состоят только из значений NULL, он будет интерпретировать их как String: - -```sql -DESC format(TSV, '[NULL, NULL]') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Пример с отключённой настройкой `input_format_tsv_use_best_effort_in_schema_inference`: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Привет, мир!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Примеры автоматического определения заголовка (когда включён `input_format_tsv_detect_header`): - -Только имена: - -```sql -SELECT * FROM format(TSV, -$$number string array -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$); -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -Названия и типы: - -```sql -DESC format(TSV, -$$number string array -UInt32 String Array(UInt16) -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Обратите внимание, что заголовок может быть распознан только в том случае, если есть хотя бы один столбец с типом, отличным от String. Если все столбцы имеют тип String, заголовок не распознается: - -```sql -SELECT * FROM format(TSV, -$$first_column second_column -Hello World -World Hello -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ first_column │ second_column │ -│ Hello │ World │ -│ World │ Hello │ -└──────────────┴───────────────┘ -``` - -### Значения {#values} - -В формате Values ClickHouse извлекает значение столбца из строки, а затем разбирает его рекурсивным парсером, аналогично разбору литералов. - -**Примеры:** - - -Целые числа, вещественные числа, логические значения, строки: - -```sql -DESC format(Values, $$(42, 42.42, true, 'Hello,World!')$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Типы Date и DateTime: - -```sql - DESC format(Values, $$('2020-01-01', '2020-01-01 00:00:00', '2022-01-01 00:00:00.000')$$) -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Массивы: - -```sql -DESC format(Values, '([1,2,3], [[1, 2], [], [3, 4]])') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если массив содержит null, ClickHouse использует типы остальных элементов массива: - -```sql -DESC format(Values, '([NULL, 42, NULL])') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Кортежи: - -```sql -DESC format(Values, $$((42, 'Hello, world!'))$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Карты: - -```sql -DESC format(Values, $$({'key1' : 42, 'key2' : 24})$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -Вложенные массивы, кортежи и словари: - -```sql -DESC format(Values, $$([{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}])$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Если ClickHouse не удаётся определить тип, так как данные содержат только значения NULL, будет сгенерировано исключение: - -```sql -DESC format(Values, '([NULL, NULL])') -``` - -```response -Код: 652. DB::Exception: Получено от localhost:9000. DB::Exception: -Невозможно определить тип столбца 'c1' по первой строке данных, -вероятно, этот столбец содержит только значения Null или пустые массивы/словари. -... -``` - -Пример с отключённой настройкой `input_format_tsv_use_best_effort_in_schema_inference`: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### CustomSeparated {#custom-separated} - -В формате CustomSeparated ClickHouse сначала извлекает все значения столбцов из строки в соответствии с заданными разделителями, а затем пытается определить тип данных для каждого значения в соответствии с правилами экранирования. - -Если настройка `input_format_custom_detect_header` включена, ClickHouse попытается обнаружить заголовок с именами столбцов (и, возможно, типами) при определении схемы. Эта настройка включена по умолчанию. - -**Пример** - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Пример автоопределения заголовка (когда включён `input_format_custom_detect_header`): - - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -'number''string''array' - -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─number─┬─string────────┬─array──────┐ -│ 42.42 │ Some string 1 │ [1,NULL,3] │ -│ ᴺᵁᴸᴸ │ Some string 3 │ [1,2,NULL] │ -└────────┴───────────────┴────────────┘ -``` - -### Template {#template} - -В формате Template ClickHouse сначала извлекает все значения столбцов из строки в соответствии с указанным шаблоном, а затем пытается определить -тип данных для каждого значения в соответствии с соответствующим правилом экранирования. - -**Пример** - -Предположим, у нас есть файл `resultset` со следующим содержимым: - -```bash - -${data} -``` - -И файл `row_format` со следующим содержимым: - -```text -${column_1:CSV}${column_2:Quoted}${column_3:JSON} -``` - -Далее можно выполнить следующие запросы: - -```sql -SET format_template_rows_between_delimiter = '\n', - format_template_row = 'row_format', - format_template_resultset = 'resultset_format' - -DESC format(Template, $$ -42.42'Some string 1'[1, null, 2] - -\N'Some string 3'[1, 2, null] - -$$) -``` - -```response -┌─name─────┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ column_1 │ Nullable(Float64) │ │ │ │ │ │ -│ column_2 │ Nullable(String) │ │ │ │ │ │ -│ column_3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Regexp {#regexp} - -Аналогично формату Template, в формате Regexp ClickHouse сначала извлекает все значения столбцов из строки в соответствии с заданным регулярным выражением, а затем пытается определить -тип данных для каждого значения согласно указанному правилу экранирования. - -**Пример** - -```sql -SET format_regexp = '^Line: value_1=(.+?), value_2=(.+?), value_3=(.+?)', - format_regexp_escaping_rule = 'CSV' -``` - - -DESC format(Regexp, $$Line: value_1=42, value_2="Some string 1", value_3="[1, NULL, 3]" -Line: value_1=2, value_2="Some string 2", value_3="[4, 5, NULL]"$$) - -```` -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -```` - -### Настройки для текстовых форматов {#settings-for-text-formats} - -#### input_format_max_rows_to_read_for_schema_inference/input_format_max_bytes_to_read_for_schema_inference {#input-format-max-rows-to-read-for-schema-inference} - -Эти настройки управляют объёмом данных, считываемых при автоматическом определении схемы. -Чем больше строк/байт считывается, тем больше времени уходит на определение схемы, но тем выше шанс -корректно определить типы (особенно когда данные содержат много null-значений). - -Значения по умолчанию: - -* `25000` для `input_format_max_rows_to_read_for_schema_inference`. -* `33554432` (32 МБ) для `input_format_max_bytes_to_read_for_schema_inference`. - -#### column_names_for_schema_inference {#column-names-for-schema-inference} - -Список имён столбцов, используемых при определении схемы для форматов без явных имён столбцов. Указанные имена будут использованы вместо значений по умолчанию `c1,c2,c3,...`. Формат: `column1,column2,column3,...`. - -**Пример** - -```sql -DESC format(TSV, 'Hello, World! 42 [1, 2, 3]') settings column_names_for_schema_inference = 'str,int,arr' -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ str │ Nullable(String) │ │ │ │ │ │ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_hints {#schema-inference-hints} - -Список имен столбцов и их типов, используемых при определении схемы вместо автоматически определенных типов. Формат: «column_name1 column_type1, column_name2 column_type2, ...». -Этот параметр можно использовать для указания типов столбцов, которые не удалось определить автоматически, или для оптимизации схемы. - -**Пример** - -```sql -DESC format(JSONEachRow, '{"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]}') SETTINGS schema_inference_hints = 'age LowCardinality(UInt8), status Nullable(String)', allow_suspicious_low_cardinality_types=1 -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ LowCardinality(UInt8) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_make_columns_nullable $ {#schema-inference-make-columns-nullable} - - -Управляет приведением выводимых типов к `Nullable` при выводе схемы для форматов без информации о nullability. Возможные значения: - -* 0 — выводимый тип никогда не будет `Nullable`, -* 1 — все выводимые типы будут `Nullable`, -* 2 или 'auto' — для текстовых форматов выводимый тип будет `Nullable` только если столбец содержит `NULL` в образце данных, который разбирается при выводе схемы; для строго типизированных форматов (Parquet, ORC, Arrow) информация о nullability берётся из метаданных файла, -* 3 — для текстовых форматов использовать `Nullable`; для строго типизированных форматов использовать метаданные файла. - -По умолчанию: 3. - -**Примеры** - -```sql -SET schema_inference_make_columns_nullable = 1; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 'auto'; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 0; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response - -┌─name────┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ String │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -#### input_format_try_infer_integers {#input-format-try-infer-integers} - -:::note -Этот параметр не применяется к типу данных `JSON`. -::: - -Если параметр включён, ClickHouse будет пытаться определять целые числа вместо чисел с плавающей запятой при выводе схемы для текстовых форматов. -Если все числа в столбце в образце данных являются целыми, результирующим типом будет `Int64`, а если хотя бы одно число является числом с плавающей запятой, результирующим типом будет `Float64`. -Если в образце данных содержатся только целые числа и хотя бы одно положительное целое число превышает диапазон `Int64`, ClickHouse определит тип как `UInt64`. - -По умолчанию параметр включён. - -**Примеры** - -```sql -SET input_format_try_infer_integers = 0 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_integers = 1 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Int64) │ │ │ │ │ │ -└────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 18446744073709551615} - $$) -``` - -```response -┌─name───┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(UInt64) │ │ │ │ │ │ -└────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2.2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes {#input-format-try-infer-datetimes} - -Если параметр включён, ClickHouse будет пытаться определять тип `DateTime` или `DateTime64` по строковым полям при автоматическом выводе схемы для текстовых форматов. -Если все значения столбца в образце данных были успешно распознаны как значения даты и времени, результирующим типом будет `DateTime` или `DateTime64(9)` (если хотя бы одно значение даты и времени содержало дробную часть), -если хотя бы одно значение не удалось распознать как дату и время, результирующим типом будет `String`. - -По умолчанию параметр включён. - -**Примеры** - - -```sql -SET input_format_try_infer_datetimes = 0; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_datetimes = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "unknown", "datetime64" : "unknown"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes_only_datetime64 {#input-format-try-infer-datetimes-only-datetime64} - -Если включено, ClickHouse будет всегда определять тип `DateTime64(9)`, когда параметр `input_format_try_infer_datetimes` включён, даже если значения даты и времени не содержат дробной части. - -По умолчанию отключено. - -**Примеры** - -```sql -SET input_format_try_infer_datetimes = 1; -SET input_format_try_infer_datetimes_only_datetime64 = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime64(9)) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Примечание: При разборе значений типов DateTime при выводе схемы учитывается настройка [date_time_input_format](/operations/settings/settings-formats.md#date_time_input_format) - - -#### input_format_try_infer_dates {#input-format-try-infer-dates} - -Если параметр включён, ClickHouse будет пытаться определить тип `Date` из строковых полей при автоматическом определении схемы для текстовых форматов. -Если все значения в столбце в образце данных были успешно интерпретированы как даты, результирующим типом будет `Date`, -если хотя бы одно значение не было интерпретировано как дата, результирующим типом будет `String`. - -Включено по умолчанию. - -**Примеры** - -```sql -SET input_format_try_infer_datetimes = 0, input_format_try_infer_dates = 0 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_dates = 1 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "unknown"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_exponent_floats {#input-format-try-infer-exponent-floats} - -Если включено, ClickHouse будет пытаться распознавать числа с плавающей запятой в экспоненциальной форме в текстовых форматах (кроме JSON, где числа в экспоненциальной форме всегда распознаются). - -По умолчанию отключено. - -**Пример** - -```sql -SET input_format_try_infer_exponent_floats = 1; -DESC format(CSV, -$$1.1E10 -2.3e-12 -42E00 -$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## Самоописывающиеся форматы {#self-describing-formats} - -Самоописывающиеся форматы содержат информацию о структуре данных в самих данных: -это может быть заголовок с описанием, бинарное дерево типов или таблица. -Чтобы автоматически вывести схему по файлам в таких форматах, ClickHouse читает часть данных, содержащую -информацию о типах, и преобразует её в схему таблицы ClickHouse. - -### Форматы с суффиксом -WithNamesAndTypes {#formats-with-names-and-types} - -ClickHouse поддерживает некоторые текстовые форматы с суффиксом -WithNamesAndTypes. Этот суффикс означает, что данные содержат две дополнительные строки с именами и типами столбцов перед основными данными. -При автоматическом выводе схемы для таких форматов ClickHouse читает первые две строки и извлекает имена и типы столбцов. - -**Пример** - -```sql -DESC format(TSVWithNamesAndTypes, -$$num str arr -UInt8 String Array(UInt8) -42 Hello, World! [1,2,3] -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Форматы JSON с метаданными {#json-with-metadata} - -Некоторые входные форматы JSON ([JSON](/interfaces/formats/JSON), [JSONCompact](/interfaces/formats/JSONCompact), [JSONColumnsWithMetadata](/interfaces/formats/JSONColumnsWithMetadata)) содержат метаданные с именами и типами столбцов. -При определении схемы для таких форматов ClickHouse использует эти метаданные. - -**Пример** - -```sql -DESC format(JSON, $$ -{ - "meta": - [ - { - "name": "num", - "type": "UInt8" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "Hello, World", - "arr": [1,2,3] - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.005723915, - "rows_read": 1, - "bytes_read": 1 - } -} -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Avro {#avro} - -В формате Avro ClickHouse считывает схему из данных и преобразует её в схему ClickHouse, используя следующие соответствия типов: - - -| Тип данных Avro | Тип данных ClickHouse | -|-------------------------------------|---------------------------------------------------------------------------------| -| `boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int (date)` \* | [Date32](../sql-reference/data-types/date32.md) | -| `long` | [Int64](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `bytes`, `string` | [String](../sql-reference/data-types/string.md) | -| `fixed` | [FixedString(N)](../sql-reference/data-types/fixedstring.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `array(T)` | [Array(T)](../sql-reference/data-types/array.md) | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](../sql-reference/data-types/date.md) | -| `null` | [Nullable(Nothing)](../sql-reference/data-types/special-data-types/nothing.md) | -| `string (uuid)` \* | [UUID](../sql-reference/data-types/uuid.md) | -| `binary (decimal)` \* | [Decimal(P, S)](../sql-reference/data-types/decimal.md) | - -\* [Логические типы Avro](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -Другие типы Avro не поддерживаются. - -### Parquet {#parquet} - -В формате Parquet ClickHouse считывает схему из данных и преобразует её в схему ClickHouse, используя следующие соответствия типов: - -| Тип данных Parquet | Тип данных ClickHouse | -|--------------------------------|--------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE` | [Date32](../sql-reference/data-types/date32.md) | -| `TIME (ms)` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME (us, ns)` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -Другие типы Parquet не поддерживаются. - -### Arrow {#arrow} - -В формате Arrow ClickHouse считывает схему из данных и преобразует её в схему ClickHouse, используя следующие соответствия типов: - - - -| Тип данных Arrow | Тип данных ClickHouse | -|----------------------------------|----------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT`, `HALF_FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE32` | [Date32](../sql-reference/data-types/date32.md) | -| `DATE64` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL128`, `DECIMAL256` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -Другие типы Arrow не поддерживаются. - -### ORC {#orc} - -В формате ORC ClickHouse считывает схему из данных и преобразует её в схему ClickHouse, используя следующие соответствия типов: - -| Тип данных ORC | Тип данных ClickHouse | -|---------------------------------------|----------------------------------------------------------| -| `Boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `Tinyint` | [Int8](../sql-reference/data-types/int-uint.md) | -| `Smallint` | [Int16](../sql-reference/data-types/int-uint.md) | -| `Int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `Bigint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `Float` | [Float32](../sql-reference/data-types/float.md) | -| `Double` | [Float64](../sql-reference/data-types/float.md) | -| `Date` | [Date32](../sql-reference/data-types/date32.md) | -| `Timestamp` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `String`, `Char`, `Varchar`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `Decimal` | [Decimal](../sql-reference/data-types/decimal.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `Struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `Map` | [Map](../sql-reference/data-types/map.md) | - -Другие типы ORC не поддерживаются. - -### Native {#native} - -Формат Native используется внутри ClickHouse и содержит схему непосредственно в данных. -При определении схемы ClickHouse считывает её из данных без каких-либо преобразований. - - - -## Форматы с внешней схемой {#formats-with-external-schema} - -Такие форматы требуют наличия схемы, описывающей данные, в отдельном файле на определённом языке описания схем. -Чтобы автоматически определить схему по файлам в таких форматах, ClickHouse считывает внешнюю схему из отдельного файла и преобразует её в схему таблицы ClickHouse. - -### Protobuf {#protobuf} - -При определении схемы для формата Protobuf ClickHouse использует следующие соответствия типов: - -| Тип данных Protobuf | Тип данных ClickHouse | -|----------------------------------|------------------------------------------------------| -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `int32`, `sint32`, `sfixed32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int64`, `sint64`, `sfixed64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `uint32`, `fixed32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `uint64`, `fixed64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `string`, `bytes` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `repeated T` | [Array(T)](../sql-reference/data-types/array.md) | -| `message`, `group` | [Tuple](../sql-reference/data-types/tuple.md) | - -### CapnProto {#capnproto} - -При определении схемы для формата CapnProto ClickHouse использует следующие соответствия типов: - -| Тип данных CapnProto | Тип данных ClickHouse | -|--------------------------------------|-----------------------------------------------------------| -| `Bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UInt8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UInt16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `Int32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UInt32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `Int64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `UInt64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `Float32` | [Float32](../sql-reference/data-types/float.md) | -| `Float64` | [Float64](../sql-reference/data-types/float.md) | -| `Text`, `Data` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `union(T, Void)`, `union(Void, T)` | [Nullable(T)](../sql-reference/data-types/nullable.md) | - - - -## Строго типизированные бинарные форматы {#strong-typed-binary-formats} - -В таких форматах каждое сериализованное значение содержит информацию о своём типе (и, возможно, о своём имени), но нет информации о всей таблице. -При выводе схемы для таких форматов ClickHouse считывает данные построчно (до `input_format_max_rows_to_read_for_schema_inference` строк или `input_format_max_bytes_to_read_for_schema_inference` байт) и извлекает -тип (и, возможно, имя) каждого значения из данных, после чего преобразует эти типы в типы ClickHouse. - -### MsgPack {#msgpack} - -В формате MsgPack нет разделителя между строками; чтобы использовать вывод схемы для этого формата, необходимо указать количество столбцов в таблице -с помощью настройки `input_format_msgpack_number_of_columns`. ClickHouse использует следующие соответствия типов: - -| Тип данных MessagePack (`INSERT`) | Тип данных ClickHouse | -|---------------------------------------------------------------------|-----------------------------------------------------------| -| `int N`, `uint N`, `negative fixint`, `positive fixint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [String](../sql-reference/data-types/string.md) | -| `float 32` | [Float32](../sql-reference/data-types/float.md) | -| `float 64` | [Float64](../sql-reference/data-types/float.md) | -| `uint 16` | [Date](../sql-reference/data-types/date.md) | -| `uint 32` | [DateTime](../sql-reference/data-types/datetime.md) | -| `uint 64` | [DateTime64](../sql-reference/data-types/datetime.md) | -| `fixarray`, `array 16`, `array 32` | [Array](../sql-reference/data-types/array.md) | -| `fixmap`, `map 16`, `map 32` | [Map](../sql-reference/data-types/map.md) | - -По умолчанию все выведенные типы заключаются в `Nullable`, но это можно изменить с помощью настройки `schema_inference_make_columns_nullable`. - -### BSONEachRow {#bsoneachrow} - -В BSONEachRow каждая строка данных представлена как BSON-документ. При выводе схемы ClickHouse считывает BSON-документы один за другим и извлекает -значения, имена и типы из данных, после чего преобразует эти типы в типы ClickHouse, используя следующие соответствия типов: - -| Тип BSON | Тип ClickHouse | -|-----------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------| -| `\x08` boolean | [Bool](../sql-reference/data-types/boolean.md) | -| `\x10` int32 | [Int32](../sql-reference/data-types/int-uint.md) | -| `\x12` int64 | [Int64](../sql-reference/data-types/int-uint.md) | -| `\x01` double | [Float64](../sql-reference/data-types/float.md) | -| `\x09` datetime | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `\x05` тип binary с подтипом `\x00` (binary), `\x02` string, `\x0E` symbol, `\x0D` JavaScript code | [String](../sql-reference/data-types/string.md) | -| `\x07` ObjectId | [FixedString(12)](../sql-reference/data-types/fixedstring.md) | -| `\x05` тип binary с подтипом `\x04` (uuid), размером 16 | [UUID](../sql-reference/data-types/uuid.md) | -| `\x04` array | [Array](../sql-reference/data-types/array.md)/[Tuple](../sql-reference/data-types/tuple.md) (если вложенные типы различаются) | -| `\x03` document | [Named Tuple](../sql-reference/data-types/tuple.md)/[Map](../sql-reference/data-types/map.md) (с ключами типа String) | - -По умолчанию все выведенные типы заключаются в `Nullable`, но это можно изменить с помощью настройки `schema_inference_make_columns_nullable`. - - - -## Форматы с фиксированной схемой {#formats-with-constant-schema} - -Данные в таких форматах всегда имеют одну и ту же схему. - -### LineAsString {#line-as-string} - -В этом формате ClickHouse считывает всю строку данных в один столбец с типом данных `String`. Выводимый тип для этого формата всегда `String`, а имя столбца — `line`. - -**Пример** - -```sql -DESC format(LineAsString, 'Hello\nworld!') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ line │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsString {#json-as-string} - -В этом формате ClickHouse считывает весь JSON-объект из входных данных в один столбец с типом данных `String`. Определяемый для этого формата тип всегда `String`, а имя столбца — `json`. - -**Пример** - -```sql -DESC format(JSONAsString, '{"x" : 42, "y" : "Hello, World!"}') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsObject {#json-as-object} - -В этом формате ClickHouse считывает весь JSON-объект из входных данных в один столбец с типом данных `JSON`. Определяемый для этого формата тип данных всегда `JSON`, а имя столбца — `json`. - -**Пример** - -```sql -DESC format(JSONAsObject, '{"x" : 42, "y" : "Hello, World!"}'); -``` - -```response -┌─name─┬─type─┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ JSON │ │ │ │ │ │ -└──────┴──────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## Режимы определения схемы {#schema-inference-modes} - -Определение схемы по набору файлов данных может работать в двух разных режимах: `default` и `union`. -Режим задаётся настройкой `schema_inference_mode`. - -### Режим по умолчанию {#default-schema-inference-mode} - -В режиме по умолчанию ClickHouse предполагает, что все файлы имеют одинаковую схему, и пытается определить её, последовательно читая файлы до тех пор, пока это не удастся. - -Пример: - -Допустим, у нас есть 3 файла `data1.jsonl`, `data2.jsonl` и `data3.jsonl` со следующим содержимым: - -`data1.jsonl`: - -```json -{"field1" : 1, "field2" : null} -{"field1" : 2, "field2" : null} -{"field1" : 3, "field2" : null} -``` - -`data2.jsonl`: - -```json -{"field1" : 4, "field2" : "Data4"} -{"field1" : 5, "field2" : "Data5"} -{"field1" : 6, "field2" : "Data5"} -``` - -`data3.jsonl`: - -```json -{"field1" : 7, "field2" : "Data7", "field3" : [1, 2, 3]} -{"field1" : 8, "field2" : "Data8", "field3" : [4, 5, 6]} -{"field1" : 9, "field2" : "Data9", "field3" : [7, 8, 9]} -``` - -Давайте попробуем применить автоматическое определение схемы к этим трём файлам: - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='default' -``` - -Результат: - -```response -┌─name───┬─type─────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -└────────┴──────────────────┘ -``` - -Как мы видим, у нас нет `field3` из файла `data3.jsonl`. -Это происходит потому, что ClickHouse сначала попытался определить схему по файлу `data1.jsonl`, но не смог, так как для поля `field2` были только значения null, -затем попытался определить схему по `data2.jsonl` и успешно это сделал, поэтому данные из файла `data3.jsonl` не были прочитаны. - -### Режим union {#default-schema-inference-mode-1} - -В режиме union ClickHouse предполагает, что файлы могут иметь разные схемы, поэтому он определяет схемы всех файлов, а затем объединяет их в общую схему. - -Допустим, у нас есть 3 файла `data1.jsonl`, `data2.jsonl` и `data3.jsonl` со следующим содержимым: - -`data1.jsonl`: - -```json -{"field1" : 1} -{"field1" : 2} -{"field1" : 3} -``` - -`data2.jsonl`: - -```json -{"field2" : "Данные4"} -{"field2" : "Данные5"} -{"field2" : "Данные5"} -``` - -`data3.jsonl`: - -```json -{"field3" : [1, 2, 3]} -{"field3" : [4, 5, 6]} -{"field3" : [7, 8, 9]} -``` - -Давайте попробуем использовать автоматическое определение схемы для этих трёх файлов: - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='union' -``` - -Результат: - -```response -┌─name───┬─type───────────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -│ field3 │ Array(Nullable(Int64)) │ -└────────┴────────────────────────┘ -``` - -Как мы видим, у нас есть все поля из всех файлов. - -Примечание: - -* Поскольку некоторые файлы могут не содержать некоторые столбцы из результирующей схемы, режим объединения (union) поддерживается только для форматов, которые умеют читать подмножество столбцов (таких как JSONEachRow, Parquet, TSVWithNames и т. д.) и не будет работать для других форматов (таких как CSV, TSV, JSONCompactEachRow и т. д.). -* Если ClickHouse не может определить схему по одному из файлов, будет сгенерировано исключение. -* Если у вас много файлов, чтение схемы из всех них может занять много времени. - - -## Автоматическое определение формата {#automatic-format-detection} - -Если формат данных не указан и его нельзя определить по расширению файла, ClickHouse попытается определить формат файла по его содержимому. - -**Примеры:** - -Предположим, у нас есть файл `data` со следующим содержимым: - -```csv -"a","b" -1,"Data1" -2,"Data2" -3,"Data3" -``` - -Мы можем изучать и выполнять запросы к этому файлу, не указывая формат или структуру: - -```sql -:) desc file(data); -``` - -```repsonse -┌─name─┬─type─────────────┐ -│ a │ Nullable(Int64) │ -│ b │ Nullable(String) │ -└──────┴──────────────────┘ -``` - -```sql -:) select * from file(data); -``` - -```response -┌─a─┬─b─────┐ -│ 1 │ Data1 │ -│ 2 │ Data2 │ -│ 3 │ Data3 │ -└───┴───────┘ -``` - -:::note -ClickHouse может распознавать лишь некоторые форматы, и это распознавание занимает время. Поэтому всегда лучше явно указывать формат. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/ssh.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/ssh.md deleted file mode 100644 index 0463b1b7b1d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/ssh.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'Документация по SSH-интерфейсу в ClickHouse' -keywords: ['client', 'ssh', 'putty'] -sidebar_label: 'SSH-интерфейс' -sidebar_position: 60 -slug: /interfaces/ssh -title: 'SSH-интерфейс' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# SSH-интерфейс с псевдотерминалом (PTY) {#ssh-interface-with-pty} - - - - - -## Предисловие {#preface} - -Сервер ClickHouse позволяет устанавливать прямое подключение по протоколу SSH. Можно использовать любой клиент. - -После создания [пользователя базы данных, аутентифицируемого с помощью SSH-ключа](/knowledgebase/how-to-connect-to-ch-cloud-using-ssh-keys): - -```sql -CREATE USER abcuser IDENTIFIED WITH ssh_key BY KEY '<СКРЫТО>' TYPE 'ssh-ed25519'; -``` - -Вы можете использовать этот ключ для подключения к серверу ClickHouse. При этом будет открыт псевдотерминал (PTY) с интерактивной сессией clickhouse-client. - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 -ClickHouse embedded version 25.1.1.1. - -ip-10-1-13-116.us-west-2.compute.internal :) SELECT 1; - -SELECT 1 - -Query id: cdd91b7f-215b-4537-b7df-86d19bf63f64 - - ┌─1─┐ -1. │ 1 │ - └───┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -Выполнение команд через SSH в неинтерактивном режиме также поддерживается: - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "select 1" -1 -``` - -## Конфигурация сервера {#server-configuration} - -Чтобы включить функцию SSH-сервера, необходимо раскомментировать или добавить следующий раздел в файл `config.xml`: - -```xml -9022 - - путь-к-ключу - - - -``` - -Ключ хоста является неотъемлемой частью протокола SSH. Его открытая часть хранится в файле `~/.ssh/known_hosts` на стороне клиента и, как правило, используется для предотвращения атак типа «man-in-the-middle». При первом подключении к серверу вы увидите сообщение, приведённое ниже: - -```shell -Подлинность хоста '[localhost]:9022 ([127.0.0.1]:9022)' не может быть установлена. -Отпечаток RSA-ключа: SHA256:3qxVlJKMr/PEKw/hfeg06HAK451Tt0eenhwqQvh58Do. -Этот ключ не известен под другими именами -Вы уверены, что хотите продолжить подключение (yes/no/[fingerprint])? -``` - -Это по сути означает: «Вы хотите сохранить открытый ключ этого хоста и продолжить подключение?». - -Вы можете указать своему SSH‑клиенту не проверять хост, передав параметр: - -```bash -ssh -o "StrictHostKeyChecking no" user@host -``` - -## Настройка встроенного клиента {#configuring-embedded-client} - -Вы можете передавать параметры встроенному клиенту аналогично тому, как это делается для обычного `clickhouse-client`, но с некоторыми ограничениями. -Поскольку используется протокол SSH, единственный способ передать параметры целевому хосту — через переменные окружения. - -Например, задать `format` можно следующим образом: - -```bash -> ssh -o SetEnv="format=Pretty" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` - -Вы можете изменять любые пользовательские настройки таким образом и дополнительно передавать большинство обычных опций `clickhouse-client` (за исключением тех, которые не имеют смысла в этой конфигурации). - -Важно: - -Если одновременно переданы опция `query` и SSH-команда, SSH-команда добавляется в список запросов на выполнение: - -```bash -ubuntu ip-10-1-13-116@~$ ssh -o SetEnv="format=Pretty query=\"SELECT 2;\"" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 2 ┃ - ┡━━━┩ -1. │ 2 │ - └───┘ - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/tcp.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/tcp.md deleted file mode 100644 index 3362371ba19..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/tcp.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Документация по родному TCP-интерфейсу ClickHouse' -sidebar_label: 'Родной интерфейс (TCP)' -sidebar_position: 18 -slug: /interfaces/tcp -title: 'Родной интерфейс (TCP)' -doc_type: 'reference' ---- - -# Нативный интерфейс (TCP) {#native-interface-tcp} - -Нативный протокол используется в [клиенте командной строки](../interfaces/cli.md), для межсерверного взаимодействия при распределённой обработке запросов, а также в других C++‑программах. К сожалению, у нативного протокола ClickHouse пока нет формальной спецификации, но его можно восстановить методом реверс‑инжиниринга по исходному коду ClickHouse (начиная [примерно отсюда](https://github.com/ClickHouse/ClickHouse/tree/master/src/Client)) и/или перехватывая и анализируя TCP‑трафик. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md deleted file mode 100644 index a01f1d4cb2e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: 'Обзор доступных клиентских библиотек сторонних разработчиков для различных языков программирования' -sidebar_label: 'Клиентские библиотеки' -sidebar_position: 26 -slug: /interfaces/third-party/client-libraries -title: 'Клиентские библиотеки от сторонних разработчиков' -doc_type: 'reference' ---- - -# Клиентские библиотеки от сторонних разработчиков {#client-libraries-from-third-party-developers} - -:::note -ClickHouse Inc **не** поддерживает перечисленные ниже библиотеки и не проводила их всестороннего тестирования для оценки их качества. -::: - -### Python {#python} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm) -* [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) -* [clickhouse-client](https://github.com/yurial/clickhouse-client) -* [aiochclient](https://github.com/maximdanilchenko/aiochclient) -* [asynch](https://github.com/long2ice/asynch) - -### PHP {#php} - -* [smi2/phpclickhouse](https://packagist.org/packages/smi2/phpClickHouse) -* [8bitov/clickhouse-php-client](https://packagist.org/packages/8bitov/clickhouse-php-client) -* [bozerkins/clickhouse-client](https://packagist.org/packages/bozerkins/clickhouse-client) -* [simpod/clickhouse-client](https://packagist.org/packages/simpod/clickhouse-client) -* [seva-code/php-click-house-client](https://packagist.org/packages/seva-code/php-click-house-client) -* [C++-клиент SeasClick](https://github.com/SeasX/SeasClick) -* [one-ck](https://github.com/lizhichao/one-ck) -* [glushkovds/phpclickhouse-laravel](https://packagist.org/packages/glushkovds/phpclickhouse-laravel) -* [glushkovds/php-clickhouse-schema-builder](https://packagist.org/packages/glushkovds/php-clickhouse-schema-builder) -* [PHP-расширение ClickHouse от kolya7k](https://github.com//kolya7k/clickhouse-php) -* [hyvor/clickhouse-php](https://github.com/hyvor/clickhouse-php) - -### Go {#go} - -* [clickhouse](https://github.com/kshvakov/clickhouse/) -* [go-clickhouse](https://github.com/roistat/go-clickhouse) -* [chconn](https://github.com/vahid-sohrabloo/chconn) -* [mailrugo-clickhouse](https://github.com/mailru/go-clickhouse) -* [golang-clickhouse](https://github.com/leprosus/golang-clickhouse) -* [uptrace/go-clickhouse](https://clickhouse.uptrace.dev/) - -### Swift {#swift} - -* [ClickHouseNIO](https://github.com/patrick-zippenfenig/ClickHouseNIO) -* [ClickHouseVapor ORM](https://github.com/patrick-zippenfenig/ClickHouseVapor) - -### Node.js {#nodejs} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [clickhouse (NodeJs)](https://github.com/TimonKK/clickhouse) -* [node-clickhouse](https://github.com/apla/node-clickhouse) -* [nestjs-clickhouse](https://github.com/depyronick/nestjs-clickhouse) -* [clickhouse-client](https://github.com/depyronick/clickhouse-client) -* [node-clickhouse-orm](https://github.com/zimv/node-clickhouse-orm) -* [clickhouse-ts](https://github.com/bytadaniel/clickhouse-ts) -* [clickcache](https://github.com/bytadaniel/clickcache) - -### Perl {#perl} - -* [perl-DBD-ClickHouse](https://github.com/elcamlost/perl-DBD-ClickHouse) -* [HTTP-ClickHouse](https://metacpan.org/release/HTTP-ClickHouse) -* [AnyEvent-ClickHouse](https://metacpan.org/release/AnyEvent-ClickHouse) - -### Ruby {#ruby} - -* [ClickHouse (Ruby)](https://github.com/shlima/click_house) -* [clickhouse-activerecord](https://github.com/PNixx/clickhouse-activerecord) - -### Rust {#rust} - -* [clickhouse.rs](https://github.com/loyd/clickhouse.rs) -* [clickhouse-rs](https://github.com/suharev7/clickhouse-rs) -* [Klickhouse](https://github.com/Protryon/klickhouse) - -### R {#r} - -* [RClickHouse](https://github.com/IMSMWU/RClickHouse) - -### Java {#java} - -* [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java) -* [clickhouse-client](https://github.com/Ecwid/clickhouse-client) - -### Scala {#scala} - -* [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) - -### Kotlin {#kotlin} - -* [AORM](https://github.com/TanVD/AORM) - -### C# {#c} - -* [Octonica.ClickHouseClient](https://github.com/Octonica/ClickHouseClient) -* [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) -* [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) -* [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - -### Elixir {#elixir} - -* [clickhousex](https://github.com/appodeal/clickhousex/) -* [pillar](https://github.com/sofakingworld/pillar) -* [ecto_ch](https://github.com/plausible/ecto_ch) -* [req_ch](https://github.com/livebook-dev/req_ch) - -### Ним {#nim} - -* [nim-clickhouse](https://github.com/leonardoce/nim-clickhouse) - -### Haskell {#haskell} - -* [hdbc-clickhouse](https://github.com/zaneli/hdbc-clickhouse) -* [ClickHaskell](https://clickhaskell.dev/) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md deleted file mode 100644 index 206e892c193..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md +++ /dev/null @@ -1,446 +0,0 @@ ---- -description: 'Список сторонних графических интерфейсов (GUI) и приложений для работы с ClickHouse' -sidebar_label: 'Графические интерфейсы' -sidebar_position: 28 -slug: /interfaces/third-party/gui -title: 'Графические интерфейсы сторонних разработчиков' -doc_type: 'reference' ---- - - - -# Визуальные интерфейсы сторонних разработчиков {#visual-interfaces-from-third-party-developers} - - - -## Open-source {#open-source} - -### agx {#agx} - -[agx](https://github.com/agnosticeng/agx) — это десктопное приложение, созданное с использованием Tauri и SvelteKit, которое предоставляет современный интерфейс для исследования данных и выполнения запросов с использованием встроенного in-memory-движка базы данных ClickHouse (chdb). - -- Используйте ch-db при запуске нативного приложения. -- Может подключаться к экземпляру ClickHouse при запуске веб-версии. -- Редактор Monaco, чтобы вы чувствовали себя как дома. -- Несколько вариантов визуализации данных, которые постоянно развиваются. - -### ch-ui {#ch-ui} - -[ch-ui](https://github.com/caioricciuti/ch-ui) — это простой интерфейс приложения на React.js для баз данных ClickHouse, предназначенный для выполнения запросов и визуализации данных. Построен на основе React и веб-клиента ClickHouse, предлагает современный и удобный UI для простого взаимодействия с базой данных. - -Особенности: - -- Интеграция с ClickHouse: простое управление подключениями и выполнение запросов. -- Отзывчивое управление вкладками: динамическая работа с несколькими вкладками, такими как вкладки запросов и таблиц. -- Оптимизация производительности: использование IndexedDB для эффективного кэширования и управления состоянием. -- Локальное хранение данных: все данные хранятся локально в браузере, что гарантирует, что данные не отправляются куда-либо еще. - -### ChartDB {#chartdb} - -[ChartDB](https://chartdb.io) — это бесплатный инструмент с открытым исходным кодом для визуализации и проектирования схем баз данных, включая ClickHouse, с помощью одного запроса. Построен на React и обеспечивает плавный и удобный пользовательский опыт, не требующий учетных данных базы данных или регистрации для начала работы. - -Особенности: - -- Визуализация схемы: мгновенный импорт и визуализация вашей схемы ClickHouse, включая ER-диаграммы с материализованными представлениями и стандартными представлениями, с отображением ссылок на таблицы. -- Экспорт DDL на базе ИИ: легкая генерация DDL-скриптов для улучшенного управления схемой и документирования. -- Поддержка нескольких диалектов SQL: совместим с рядом диалектов SQL, что делает его универсальным для различных сред баз данных. -- Без регистрации и учетных данных: весь функционал доступен напрямую в браузере, что делает работу простой и безопасной. - -[ChartDB Source Code](https://github.com/chartdb/chartdb). - -### DataPup {#datapup} - -[DataPup](https://github.com/DataPupOrg/DataPup) — это современный, кроссплатформенный клиент для баз данных с ИИ-поддержкой и нативной поддержкой ClickHouse. - -Особенности: - -- Помощь в написании SQL-запросов на базе ИИ с интеллектуальными подсказками -- Нативная поддержка подключений к ClickHouse с безопасной обработкой учетных данных -- Красивый, доступный интерфейс с несколькими темами (Light, Dark и цветные варианты) -- Расширенная фильтрация и исследование результатов запросов -- Кроссплатформенная поддержка (macOS, Windows, Linux) -- Быстрая и отзывчивая работа -- Открытый исходный код и лицензия MIT - -### ClickHouse Schema Flow Visualizer {#clickhouse-schemaflow-visualizer} - -[ClickHouse Schema Flow Visualizer](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) — это мощное веб-приложение с открытым исходным кодом для визуализации связей таблиц ClickHouse с использованием диаграмм Mermaid.js. Просматривайте базы данных и таблицы через интуитивный интерфейс, изучайте метаданные таблиц с дополнительной информацией о количестве строк и размере, а также экспортируйте интерактивные диаграммы схем. - -Особенности: - -- Просмотр баз данных и таблиц ClickHouse через интуитивно понятный интерфейс -- Визуализация связей между таблицами с помощью диаграмм Mermaid.js -- Цветовые значки, соответствующие типам таблиц, для лучшей визуализации -- Просмотр направления потока данных между таблицами -- Экспорт диаграмм как автономных HTML-файлов -- Переключение видимости метаданных (строки таблицы и информация о размере) -- Безопасное подключение к ClickHouse с поддержкой TLS -- Отзывчивый веб-интерфейс для всех устройств - -[ClickHouse Schema Flow Visualizer - source code](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) - -### Tabix {#tabix} - -Веб-интерфейс к ClickHouse из проекта [Tabix](https://github.com/tabixio/tabix). - -Особенности: - -- Работает с ClickHouse напрямую из браузера без необходимости установки дополнительного ПО. -- Редактор запросов с подсветкой синтаксиса. -- Автодополнение команд. -- Инструменты для графического анализа выполнения запросов. -- Настраиваемые цветовые схемы. - -[Tabix documentation](https://tabix.io/doc/). - -### HouseOps {#houseops} - -[HouseOps](https://github.com/HouseOps/HouseOps) — это UI/IDE для OSX, Linux и Windows. - -Особенности: - -- Конструктор запросов с подсветкой синтаксиса. Просмотр ответа в виде таблицы или JSON-представления. -- Экспорт результатов запросов в CSV или JSON. -- Список процессов с описаниями. Режим записи. Возможность остановить процесс (`KILL`). -- Граф базы данных. Показывает все таблицы и их столбцы с дополнительной информацией. -- Быстрый просмотр размера столбца. -- Конфигурация сервера. - -Следующие возможности запланированы к разработке: - -- Управление базами данных. -- Управление пользователями. -- Анализ данных в реальном времени. -- Мониторинг кластера. -- Управление кластером. -- Мониторинг реплицируемых и Kafka-таблиц. - -### LightHouse {#lighthouse} - - - -[LightHouse](https://github.com/VKCOM/lighthouse) — это легковесный веб-интерфейс для ClickHouse. - -Возможности: - -- Список таблиц с фильтрацией и отображением метаданных. -- Просмотр таблиц с фильтрацией и сортировкой. -- Выполнение запросов в режиме только для чтения. - -### Redash {#redash} - -[Redash](https://github.com/getredash/redash) — это платформа для визуализации данных. - -Поддерживая несколько источников данных, включая ClickHouse, Redash может объединять результаты запросов из разных источников данных в один итоговый набор данных. - -Возможности: - -- Мощный редактор запросов. -- Обозреватель баз данных. -- Инструмент визуализации, позволяющий представлять данные в разных формах. - -### Grafana {#grafana} - -[Grafana](https://grafana.com/grafana/plugins/grafana-clickhouse-datasource/) — это платформа для мониторинга и визуализации. - -«Grafana позволяет запрашивать, визуализировать, настраивать оповещения и понимать ваши метрики независимо от того, где они хранятся. Создавайте, изучайте и делитесь дашбордами со своей командой и формируйте культуру работы с данными. Ей доверяет и её любит сообщество» — grafana.com. - -Плагин источника данных ClickHouse обеспечивает поддержку ClickHouse в качестве базовой базы данных. - -### qryn {#qryn} - -[qryn](https://metrico.in) — это многопротокольный высокопроизводительный стек наблюдаемости для ClickHouse _(ранее cLoki)_ с нативными интеграциями Grafana, позволяющий выполнять приём и анализ логов, метрик и телеметрических трассировок от любых агентов, поддерживающих Loki/LogQL, Prometheus/PromQL, OTLP/Tempo, Elastic, InfluxDB и многие другие. - -Возможности: - -- Встроенный Explore UI и LogQL CLI для запроса, извлечения и визуализации данных. -- Нативная поддержка Grafana API для запроса, обработки, приёма, трассировки и оповещений без плагинов. -- Мощный конвейер для динамического поиска, фильтрации и извлечения данных из логов, событий, трассировок и не только. -- Ингестия и PUSH API, прозрачно совместимые с LogQL, PromQL, InfluxDB, Elastic и многими другими. -- Готов к использованию с агентами, такими как Promtail, Grafana-Agent, Vector, Logstash, Telegraf и многие другие. - -### DBeaver {#dbeaver} - -[DBeaver](https://dbeaver.io/) — универсальный настольный клиент для работы с базами данных с поддержкой ClickHouse. - -Возможности: - -- Разработка запросов с подсветкой синтаксиса и автодополнением. -- Список таблиц с фильтрами и поиском по метаданным. -- Просмотр данных таблиц. -- Полнотекстовый поиск. - -По умолчанию DBeaver не подключается с использованием сессии (CLI, например, это делает). Если вам нужна поддержка сессий (например, чтобы задавать настройки для своей сессии), отредактируйте свойства подключения драйвера и установите `session_id` в случайную строку (под капотом используется HTTP-подключение). После этого вы сможете использовать любые настройки из окна запросов. - -### clickhouse-cli {#clickhouse-cli} - -[clickhouse-cli](https://github.com/hatarist/clickhouse-cli) — это альтернативный клиент командной строки для ClickHouse, написанный на Python 3. - -Возможности: - -- Автодополнение. -- Подсветка синтаксиса для запросов и выводимых данных. -- Поддержка пейджера для вывода данных. -- Пользовательские команды в стиле PostgreSQL. - -### clickhouse-flamegraph {#clickhouse-flamegraph} - -[clickhouse-flamegraph](https://github.com/Slach/clickhouse-flamegraph) — специализированный инструмент для визуализации `system.trace_log` в виде [flamegraph](http://www.brendangregg.com/flamegraphs.html). - -### clickhouse-plantuml {#clickhouse-plantuml} - -[cickhouse-plantuml](https://pypi.org/project/clickhouse-plantuml/) — это скрипт для генерации диаграммы схем таблиц [PlantUML](https://plantuml.com/). - -### ClickHouse table graph {#clickhouse-table-graph} - -[ClickHouse table graph](https://github.com/mbaksheev/clickhouse-table-graph) — это простой CLI-инструмент для визуализации зависимостей между таблицами ClickHouse. Этот инструмент извлекает связи между таблицами из таблицы `system.tables` и строит диаграмму потоков зависимостей в формате [mermaid](https://mermaid.js.org/syntax/flowchart.html). С помощью этого инструмента вы можете легко визуализировать зависимости таблиц и понять поток данных в вашей базе данных ClickHouse. Благодаря mermaid итоговая диаграмма выглядит наглядно и может быть легко добавлена в вашу документацию в формате Markdown. - -### xeus-clickhouse {#xeus-clickhouse} - -[xeus-clickhouse](https://github.com/wangfenjin/xeus-clickhouse) — это ядро Jupyter для ClickHouse, которое поддерживает выполнение SQL-запросов к данным ClickHouse в Jupyter. - -### MindsDB Studio {#mindsdb} - - - -[MindsDB](https://mindsdb.com/) — это открытый AI-слой для баз данных, включая ClickHouse, который позволяет без лишних усилий разрабатывать, обучать и развертывать передовые модели машинного обучения. MindsDB Studio (GUI) позволяет обучать новые модели на данных из базы, интерпретировать предсказания модели, выявлять потенциальные смещения в данных, а также оценивать и визуализировать точность модели с помощью функции Explainable AI, чтобы быстрее адаптировать и настраивать ваши модели машинного обучения. - -### DBM {#dbm} - -[DBM](https://github.com/devlive-community/dbm) — это инструмент визуального управления для ClickHouse! - -Возможности: - -- Поддержка истории запросов (пагинация, очистка и т. д.) -- Поддержка выполнения выбранных частей SQL-запроса -- Поддержка принудительного завершения запросов -- Поддержка управления таблицами (метаданные, удаление, предварительный просмотр) -- Поддержка управления базами данных (удаление, создание) -- Поддержка произвольных запросов -- Поддержка управления несколькими источниками данных (проверка подключения, мониторинг) -- Поддержка мониторинга (процессор, соединения, запросы) -- Поддержка миграции данных - -### Bytebase {#bytebase} - -[Bytebase](https://bytebase.com) — это веб-инструмент с открытым исходным кодом для управления изменениями схемы и контроля версий, предназначенный для команд. Он поддерживает различные базы данных, включая ClickHouse. - -Возможности: - -- Ревью схемы между разработчиками и DBA. -- Database-as-Code: контроль версий схемы в VCS (например, GitLab) и запуск развертывания при коммите кода. -- Упрощенное развертывание с политиками на уровне окружений. -- Полная история миграций. -- Обнаружение дрейфа схемы. -- Резервное копирование и восстановление. -- RBAC. - -### Zeppelin-Interpreter-for-ClickHouse {#zeppelin-interpreter-for-clickhouse} - -[Zeppelin-Interpreter-for-ClickHouse](https://github.com/SiderZhang/Zeppelin-Interpreter-for-ClickHouse) — это интерпретатор [Zeppelin](https://zeppelin.apache.org) для ClickHouse. По сравнению с JDBC-интерпретатором, он обеспечивает более тонкий контроль таймаутов для долго выполняющихся запросов. - -### ClickCat {#clickcat} - -[ClickCat](https://github.com/clickcat-project/ClickCat) — это удобный пользовательский интерфейс, который позволяет искать, исследовать и визуализировать ваши данные в ClickHouse. - -Возможности: - -- Онлайн SQL-редактор, который может выполнять ваш SQL-код без какой-либо установки. -- Вы можете наблюдать все процессы и мутации. Незавершенные процессы можно завершать через UI. -- Метрики включают «Cluster Analysis», «Data Analysis» и «Query Analysis». - -### ClickVisual {#clickvisual} - -[ClickVisual](https://clickvisual.net/) ClickVisual — это легковесная платформа с открытым исходным кодом для запросов к логам, их анализа и визуализации, а также настройки оповещений. - -Возможности: - -- Поддержка создания аналитических хранилищ логов в один клик -- Поддержка управления конфигурацией сбора логов -- Поддержка пользовательской настройки индексов -- Поддержка настройки оповещений -- Поддержка настройки прав доступа на уровне библиотек и таблиц - -### ClickHouse-Mate {#clickmate} - -[ClickHouse-Mate](https://github.com/metrico/clickhouse-mate) — это веб-клиент на Angular и пользовательский интерфейс для поиска и исследования данных в ClickHouse. - -Возможности: - -- Автодополнение ClickHouse SQL-запросов -- Быстрая навигация по дереву баз данных и таблиц -- Расширенная фильтрация и сортировка результатов -- Встроенная документация по ClickHouse SQL -- Предустановленные запросы и история запросов -- Полностью браузерное решение, без сервера/бэкенда - -Клиент доступен для мгновенного использования через GitHub Pages: https://metrico.github.io/clickhouse-mate/ - -### Uptrace {#uptrace} - -[Uptrace](https://github.com/uptrace/uptrace) — это APM-инструмент, предоставляющий распределенный трейсинг и метрики на базе OpenTelemetry и ClickHouse. - -Возможности: - -- [OpenTelemetry tracing](https://uptrace.dev/opentelemetry/distributed-tracing.html), метрики и логи. -- Уведомления по Email/Slack/PagerDuty с использованием AlertManager. -- SQL-подобный язык запросов для агрегации спанов. -- Язык, похожий на PromQL, для запросов метрик. -- Готовые дашборды метрик. -- Поддержка нескольких пользователей и проектов через конфигурацию YAML. - -### clickhouse-monitoring {#clickhouse-monitoring} - -[clickhouse-monitoring](https://github.com/duyet/clickhouse-monitoring) — это простой дашборд на Next.js, который опирается на таблицы `system.*`, чтобы помогать в мониторинге и предоставлять обзор состояния вашего кластера ClickHouse. - -Возможности: - -- Мониторинг запросов: текущие запросы, история запросов, потребление ресурсов запросами (память, читаемые части, file_open, ...), самые «дорогие» запросы, наиболее используемые таблицы или столбцы и т. д. -- Мониторинг кластера: суммарное использование памяти/CPU, распределенная очередь, глобальные настройки, настройки MergeTree, метрики и т. д. -- Информация о таблицах и частях: размер, количество строк, сжатие, размер части и т. д. с детализацией до уровня столбцов. -- Полезные инструменты: исследование данных ZooKeeper, EXPLAIN для запросов, принудительное завершение запросов и т. д. -- Графики визуализации метрик: запросы и использование ресурсов, количество слияний/мутаций, производительность слияний, производительность запросов и т. д. - -### CKibana {#ckibana} - - - -[CKibana](https://github.com/TongchengOpenSource/ckibana) — это легковесный сервис, который позволяет легко искать, исследовать и визуализировать данные ClickHouse с использованием нативного интерфейса Kibana. - -Возможности: - -- Транслирует запросы построения графиков из нативного интерфейса Kibana в синтаксис запросов ClickHouse. -- Поддерживает продвинутые функции, такие как семплирование и кэширование, для повышения производительности запросов. -- Минимизирует затраты на обучение для пользователей после миграции с ElasticSearch на ClickHouse. - -### Telescope {#telescope} - -[Telescope](https://iamtelescope.net/) — это современный веб-интерфейс для исследования логов, хранящихся в ClickHouse. Он предоставляет удобный UI для выполнения запросов, визуализации и управления логами с тонкой настройкой контроля доступа. - -Возможности: - -- Чистый, адаптивный UI с мощными фильтрами и настраиваемым выбором полей. -- Синтаксис FlyQL для интуитивной и выразительной фильтрации логов. -- Временной график с поддержкой group-by, включая вложенные поля JSON, Map и Array. -- Необязательная поддержка `WHERE`-запросов на чистом SQL для продвинутой фильтрации (с проверкой прав доступа). -- Saved Views: сохранение и совместное использование пользовательских конфигураций UI для запросов и макета. -- Ролевой контроль доступа (RBAC) и интеграция с аутентификацией через GitHub. -- Не требуются дополнительные агенты или компоненты на стороне ClickHouse. - -[Исходный код Telescope](https://github.com/iamtelescope/telescope) · [Демо-версия](https://demo.iamtelescope.net) - - - -## Коммерческие решения {#commercial} - -### DataGrip {#datagrip} - -[DataGrip](https://www.jetbrains.com/datagrip/) — это IDE для баз данных от JetBrains с полноценной поддержкой ClickHouse. Также она встроена в другие инструменты на базе IntelliJ: PyCharm, IntelliJ IDEA, GoLand, PhpStorm и другие. - -Возможности: - -- Очень быстрое автодополнение кода. -- Подсветка синтаксиса ClickHouse. -- Поддержка возможностей, специфичных для ClickHouse, например вложенные столбцы, движки таблиц. -- Редактор данных. -- Рефакторинги. -- Поиск и навигация. - -### Yandex DataLens {#yandex-datalens} - -[Yandex DataLens](https://cloud.yandex.ru/services/datalens) — это сервис визуализации и аналитики данных. - -Возможности: - -- Широкий набор доступных визуализаций — от простых столбчатых диаграмм до сложных дашбордов. -- Дашборды могут быть опубликованы в открытом доступе. -- Поддержка множества источников данных, включая ClickHouse. -- Хранилище материализованных данных на базе ClickHouse. - -DataLens [доступен бесплатно](https://cloud.yandex.com/docs/datalens/pricing) для проектов с низкой нагрузкой, в том числе для коммерческого использования. - -- [Документация DataLens](https://cloud.yandex.com/docs/datalens/). -- [Пошаговое руководство](https://cloud.yandex.com/docs/solutions/datalens/data-from-ch-visualization) по визуализации данных из базы данных ClickHouse. - -### Holistics Software {#holistics-software} - -[Holistics](https://www.holistics.io/) — это полнофункциональная платформа для работы с данными и инструмент бизнес-аналитики. - -Возможности: - -- Автоматическая рассылка отчётов по email, в Slack и Google Sheets по расписанию. -- SQL-редактор с визуализациями, контролем версий, автодополнением, многократно используемыми компонентами запросов и динамическими фильтрами. -- Встраиваемая аналитика отчётов и дашбордов через iframe. -- Возможности подготовки данных и ETL. -- Поддержка моделирования данных на SQL для реляционного отображения данных. - -### Looker {#looker} - -[Looker](https://looker.com) — это платформа для работы с данными и бизнес-аналитики с поддержкой более чем 50 диалектов баз данных, включая ClickHouse. Looker доступен как SaaS-платформа и в виде решения для самостоятельного развёртывания. Пользователи могут работать с Looker через браузер для исследования данных, построения визуализаций и дашбордов, планирования рассылки отчётов и обмена полученными инсайтами с коллегами. Looker предоставляет богатый набор инструментов для встраивания этих возможностей в другие приложения и API -для интеграции данных с другими приложениями. - -Возможности: - -- Удобная и гибкая разработка с использованием LookML — языка, который поддерживает курируемое - [Data Modeling](https://looker.com/platform/data-modeling) для авторов отчётов и конечных пользователей. -- Мощная интеграция в рабочие процессы через [Data Actions](https://looker.com/platform/actions) в Looker. - -[Как настроить ClickHouse в Looker.](https://docs.looker.com/setup-and-management/database-config/clickhouse) - -### SeekTable {#seektable} - -[SeekTable](https://www.seektable.com) — это BI-инструмент самообслуживания для исследования данных и операционной отчётности. Доступен как в виде облачного сервиса, так и в варианте для самостоятельного развёртывания. Отчёты из SeekTable можно встраивать в любые веб-приложения. - -Возможности: - -- Удобный конструктор отчётов для бизнес-пользователей. -- Мощные параметры отчётов для фильтрации на уровне SQL и настройки запросов для конкретного отчёта. -- Возможность подключения к ClickHouse как по нативному TCP/IP-эндпоинту, так и по HTTP(S)-интерфейсу (2 разных драйвера). -- Можно использовать весь функционал SQL-диалекта ClickHouse в определениях измерений и показателей. -- [Web API](https://www.seektable.com/help/web-api-integration) для автоматической генерации отчётов. -- Поддержка процесса разработки отчётов с [backup/restore](https://www.seektable.com/help/self-hosted-backup-restore) данных учётной записи; конфигурация моделей данных (кубов) и отчётов представлена в человеко-читаемом XML и может храниться в системе контроля версий. - -SeekTable [бесплатен](https://www.seektable.com/help/cloud-pricing) для персонального/индивидуального использования. - -[Как настроить подключение к ClickHouse в SeekTable.](https://www.seektable.com/help/clickhouse-pivot-table) - -### Chadmin {#chadmin} - -[Chadmin](https://github.com/bun4uk/chadmin) — это простой пользовательский интерфейс, в котором вы можете просматривать выполняющиеся в данный момент запросы в вашем кластере ClickHouse, получать информацию о них, а при необходимости завершать эти запросы. - -### TABLUM.IO {#tablum_io} - -[TABLUM.IO](https://tablum.io/) — онлайн-инструмент для запросов и аналитики, предназначенный для ETL и визуализации. Он позволяет подключаться к ClickHouse, выполнять запросы к данным через гибкую SQL-консоль, а также загружать данные из статических файлов и сторонних сервисов. TABLUM.IO может визуализировать результаты в виде графиков и таблиц. - - - -Возможности: -- ETL: загрузка данных из популярных баз данных, локальных и удалённых файлов, вызовы API. -- Универсальная SQL-консоль с подсветкой синтаксиса и визуальным конструктором запросов. -- Визуализация данных в виде графиков и таблиц. -- Материализация данных и подзапросы. -- Формирование отчётов с отправкой в Slack, Telegram или на email. -- Построение конвейеров обработки данных через проприетарный API. -- Экспорт данных в форматах JSON, CSV, SQL, HTML. -- Веб-интерфейс. - -TABLUM.IO может быть развёрнут как self-hosted‑решение (в виде Docker-образа) или в облаке. -Лицензия: [коммерческий](https://tablum.io/pricing) продукт с 3-месячным бесплатным периодом. - -Попробуйте бесплатно [в облаке](https://tablum.io/try). -Узнайте больше о продукте на [TABLUM.IO](https://tablum.io/) - -### CKMAN {#ckman} - -[CKMAN](https://www.github.com/housepower/ckman) — это инструмент для управления и мониторинга кластеров ClickHouse! - -Возможности: - -- Быстрое и удобное автоматическое развертывание кластеров через браузерный интерфейс -- Возможность масштабировать кластеры (увеличивать или уменьшать масштаб) -- Балансировка нагрузки по данным кластера -- Обновление кластера онлайн, без остановки -- Изменение конфигурации кластера через веб-интерфейс -- Мониторинг узлов кластера и ZooKeeper -- Мониторинг состояния таблиц и партиций, а также медленных SQL-запросов -- Удобная страница для выполнения SQL-запросов diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md deleted file mode 100644 index 086b5509e8e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: 'Обзор сторонних инструментов, библиотек и интеграций для ClickHouse' -sidebar_position: 24 -slug: /interfaces/third-party/ -toc_folder_title: 'Сторонние решения' -title: 'Сторонние интерфейсы' -doc_type: 'landing-page' ---- - -# Сторонние интерфейсы {#third-party-interfaces} - -Это подборка ссылок на сторонние инструменты, которые предоставляют тот или иной интерфейс к ClickHouse. Это может быть графический интерфейс, интерфейс командной строки или API: - -* [Клиентские библиотеки](../../interfaces/third-party/client-libraries.md) -* [Интеграции](../../interfaces/third-party/integrations.md) -* [GUI](../../interfaces/third-party/gui.md) -* [Прокси](../../interfaces/third-party/proxy.md) - -:::note -Универсальные инструменты, которые поддерживают распространённые API, такие как [ODBC](../../interfaces/odbc.md) или [JDBC](../../interfaces/jdbc.md), обычно также могут работать с ClickHouse, но здесь не перечислены, так как их слишком много. -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md deleted file mode 100644 index 6ef9025f3cf..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -description: 'Документация по интеграции ClickHouse с различными сторонними системами - и инструментами' -sidebar_label: 'Интеграции' -sidebar_position: 27 -slug: /interfaces/third-party/integrations -title: 'Интеграционные библиотеки сторонних разработчиков' -doc_type: 'reference' ---- - -# Библиотеки интеграции от сторонних разработчиков {#integration-libraries-from-third-party-developers} - -:::warning Отказ от ответственности -ClickHouse, Inc. **не** поддерживает перечисленные ниже инструменты и библиотеки и не проводила их всестороннего тестирования для оценки качества. -Ознакомиться с официальными интеграциями можно на [странице интеграций](/integrations). -::: - -## Инфраструктурные продукты {#infrastructure-products} - -
-Системы управления реляционными базами данных - -- [MySQL](https://www.mysql.com) - - [mysql2ch](https://github.com/long2ice/mysql2ch) - - [ProxySQL](https://github.com/sysown/proxysql/wiki/ClickHouse-Support) - - [clickhouse-mysql-data-reader](https://github.com/Altinity/clickhouse-mysql-data-reader) - - [horgh-replicator](https://github.com/larsnovikov/horgh-replicator) -- [PostgreSQL](https://www.postgresql.org) - - [clickhousedb_fdw](https://github.com/Percona-Lab/clickhousedb_fdw) - - [infi.clickhouse_fdw](https://github.com/Infinidat/infi.clickhouse_fdw) (использует [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)) - - [pg2ch](https://github.com/mkabilov/pg2ch) - - [clickhouse_fdw](https://github.com/adjust/clickhouse_fdw) -- [MSSQL](https://en.wikipedia.org/wiki/Microsoft_SQL_Server) - - [ClickHouseMigrator](https://github.com/zlzforever/ClickHouseMigrator) -
- -
-Очереди сообщений - -- [Kafka](https://kafka.apache.org) - - [clickhouse_sinker](https://github.com/housepower/clickhouse_sinker) (использует клиент [Go](https://github.com/ClickHouse/clickhouse-go/)) - - [stream-loader-clickhouse](https://github.com/adform/stream-loader) -
- -
-Пакетная обработка - -- [Spark](https://spark.apache.org) - - [spark-clickhouse-connector](https://github.com/housepower/spark-clickhouse-connector) -
- -
-Потоковая обработка - -- [Flink](https://flink.apache.org) - - [flink-clickhouse-sink](https://github.com/ivi-ru/flink-clickhouse-sink) -
- -
-Объектные хранилища - -- [S3](https://en.wikipedia.org/wiki/Amazon_S3) - - [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) -
- -
-Оркестрация контейнеров - -- [Kubernetes](https://kubernetes.io) - - [clickhouse-operator](https://github.com/Altinity/clickhouse-operator) -
- -
-Управление конфигурациями -- [puppet](https://puppet.com) - - [innogames/clickhouse](https://forge.puppet.com/innogames/clickhouse) - - [mfedotov/clickhouse](https://forge.puppet.com/mfedotov/clickhouse) -
- -
-Мониторинг - -- [Graphite](https://graphiteapp.org) - - [graphouse](https://github.com/ClickHouse/graphouse) - - [carbon-clickhouse](https://github.com/lomik/carbon-clickhouse) - - [graphite-clickhouse](https://github.com/lomik/graphite-clickhouse) - - [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) — оптимизирует устаревшие партиции в [\*GraphiteMergeTree](/engines/table-engines/mergetree-family/graphitemergetree), если к ним можно применить правила из [конфигурации rollup](../../engines/table-engines/mergetree-family/graphitemergetree.md#rollup-configuration) -- [Grafana](https://grafana.com/) - - [clickhouse-grafana](https://github.com/Altinity/clickhouse-grafana) -- [Prometheus](https://prometheus.io/) - - [clickhouse_exporter](https://github.com/f1yegor/clickhouse_exporter) - - [PromHouse](https://github.com/Percona-Lab/PromHouse) - - [clickhouse_exporter](https://github.com/hot-wifi/clickhouse_exporter) (использует [клиент Go](https://github.com/kshvakov/clickhouse/)) -- [Nagios](https://www.nagios.org/) - - [check_clickhouse](https://github.com/exogroup/check_clickhouse/) - - [check_clickhouse.py](https://github.com/innogames/igmonplugins/blob/master/src/check_clickhouse.py) -- [Zabbix](https://www.zabbix.com) - - [clickhouse-zabbix-template](https://github.com/Altinity/clickhouse-zabbix-template) -- [Sematext](https://sematext.com/) - - [интеграция с ClickHouse](https://github.com/sematext/sematext-agent-integrations/tree/master/clickhouse) -
- -
-Логирование - -- [rsyslog](https://www.rsyslog.com/) - - [omclickhouse](https://www.rsyslog.com/doc/master/configuration/modules/omclickhouse.html) -- [fluentd](https://www.fluentd.org) - - [loghouse](https://github.com/flant/loghouse) (для [Kubernetes](https://kubernetes.io)) -- [logagent](https://www.sematext.com/logagent) - - [logagent output-plugin-clickhouse](https://sematext.com/docs/logagent/output-plugin-clickhouse/) -
- -
-Гео - -- [MaxMind](https://dev.maxmind.com/geoip/) - - [clickhouse-maxmind-geoip](https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip) -
- -
-AutoML - -- [MindsDB](https://mindsdb.com/) - - [MindsDB](https://github.com/mindsdb/mindsdb) — интегрируется с ClickHouse, обеспечивая доступ к данным из ClickHouse для широкого спектра моделей AI/ML. -
- -## Экосистемы языков программирования {#programming-language-ecosystems} - -
-Python - -- [SQLAlchemy](https://www.sqlalchemy.org) - - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse) (использует [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)) -- [PyArrow/Pandas](https://pandas.pydata.org) - - [Ibis](https://github.com/ibis-project/ibis) -
- -
-PHP - -- [Doctrine](https://www.doctrine-project.org/) - - [dbal-clickhouse](https://packagist.org/packages/friendsofdoctrine/dbal-clickhouse) -
- -
-R - -- [dplyr](https://db.rstudio.com/dplyr/) - - [RClickHouse](https://github.com/IMSMWU/RClickHouse) (использует [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp)) -
- -
-Java - -- [Hadoop](http://hadoop.apache.org) - - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader) (использует [JDBC](../../sql-reference/table-functions/jdbc.md)) -
- -
-Scala - -- [Akka](https://akka.io) - - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) -
- -
-C# - -- [ADO.NET](https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-overview) - - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) - - [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) - - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations) - - [Linq To DB](https://github.com/linq2db/linq2db) -
- -
-Elixir - -- [Ecto](https://github.com/elixir-ecto/ecto) - - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto) -
- -
-Ruby - -- [Ruby on Rails](https://rubyonrails.org/) - - [activecube](https://github.com/bitquery/activecube) - - [ActiveRecord](https://github.com/PNixx/clickhouse-activerecord) -- [GraphQL](https://github.com/graphql) - - [activecube-graphql](https://github.com/bitquery/activecube-graphql) -
\ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md b/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md deleted file mode 100644 index 5bff9f19768..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: 'Описывает доступные сторонние прокси-решения для ClickHouse' -sidebar_label: 'Прокси' -sidebar_position: 29 -slug: /interfaces/third-party/proxy -title: 'Прокси-серверы от сторонних разработчиков' -doc_type: 'reference' ---- - - - -# Прокси-серверы сторонних разработчиков {#proxy-servers-from-third-party-developers} - - - -## chproxy {#chproxy} - -[chproxy](https://github.com/Vertamedia/chproxy) — это HTTP‑прокси и балансировщик нагрузки для базы данных ClickHouse. - -Возможности: - -- Маршрутизация по пользователям и кэширование ответов. -- Гибкая система ограничений. -- Автоматическое обновление SSL‑сертификатов. - -Реализован на Go. - - - -## KittenHouse {#kittenhouse} - -[KittenHouse](https://github.com/VKCOM/kittenhouse) предназначен для использования в качестве локального прокси между ClickHouse и сервером приложения в тех случаях, когда буферизация данных INSERT на стороне приложения невозможна или неудобна. - -Возможности: - -- Буферизация данных в памяти и на диске. -- Маршрутизация по таблицам. -- Балансировка нагрузки и проверка состояния. - -Реализован на Go. - - - -## ClickHouse-Bulk {#clickhouse-bulk} - -[ClickHouse-Bulk](https://github.com/nikepan/clickhouse-bulk) — это простой коллектор для вставки данных в ClickHouse. - -Возможности: - -- Группирует запросы и отправляет их при достижении порога или через заданные интервалы. -- Поддержка нескольких удалённых серверов. -- Базовая аутентификация. - -Написан на Go. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md deleted file mode 100644 index 740e15fd535..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md +++ /dev/null @@ -1,228 +0,0 @@ - -[//]: # (Этот файл включён в раздел FAQ > Устранение неполадок) - -- [Установка](#troubleshooting-installation-errors) -- [Подключение к серверу](#troubleshooting-accepts-no-connections) -- [Обработка запросов](#troubleshooting-does-not-process-queries) -- [Эффективность обработки запросов](#troubleshooting-too-slow) - - - -## Установка {#troubleshooting-installation-errors} - -### Невозможно получить пакеты .deb из репозитория ClickHouse с помощью apt-get {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get} - -* Проверьте настройки брандмауэра. -* Если вы по какой-либо причине не можете получить доступ к репозиторию, скачайте пакеты, как описано в статье [install guide](../getting-started/install.md), и установите их вручную с помощью команды `sudo dpkg -i `. Вам также понадобится пакет `tzdata`. - -### Невозможно обновить пакеты .deb из репозитория ClickHouse с помощью apt-get {#you-cannot-update-deb-packages-from-clickhouse-repository-with-apt-get} - -* Проблема может возникать, если ключ GPG был изменён. - -Используйте руководство со страницы [setup](../getting-started/install.md#setup-the-debian-repository) для обновления конфигурации репозитория. - -### При выполнении `apt-get update` появляются различные предупреждения {#you-get-different-warnings-with-apt-get-update} - -* Полные тексты предупреждений могут быть следующими: - -```bash -N: Пропущено получение настроенного файла 'main/binary-i386/Packages', так как репозиторий 'https://packages.clickhouse.com/deb stable InRelease' не поддерживает архитектуру 'i386' -``` - -```bash -E: Не удалось получить https://packages.clickhouse.com/deb/dists/stable/main/binary-amd64/Packages.gz Файл имеет неожиданный размер (30451 != 28154). Идёт синхронизация зеркала? -``` - -```text -E: Репозиторий 'https://packages.clickhouse.com/deb stable InRelease' изменил значение параметра 'Origin' с 'Artifactory' на 'ClickHouse' -E: Репозиторий 'https://packages.clickhouse.com/deb stable InRelease' изменил значение параметра 'Label' с 'Artifactory' на 'ClickHouse' -N: Репозиторий 'https://packages.clickhouse.com/deb stable InRelease' изменил значение параметра 'Suite' с 'stable' на '' -N: Это изменение должно быть явно подтверждено, прежде чем можно будет применять обновления для этого репозитория. Подробности см. в руководстве apt-secure(8). -``` - -```bash -Err:11 https://packages.clickhouse.com/deb stable InRelease - 400 Некорректный запрос [IP: 172.66.40.249 443] -``` - -Чтобы устранить описанную выше проблему, используйте следующий скрипт: - -```bash -sudo rm /var/lib/apt/lists/packages.clickhouse.com_* /var/lib/dpkg/arch /var/lib/apt/lists/partial/packages.clickhouse.com_* -sudo apt-get clean -sudo apt-get autoclean -``` - -### Не удаётся установить пакеты через yum из‑за неверной подписи {#you-cant-get-packages-with-yum-because-of-wrong-signature} - -Возможная причина: кэш повреждён, возможно, он сломался после обновления ключа GPG в сентябре 2022 года. - -Решение — очистить кэш и каталог lib, используемые yum: - -```bash -sudo find /var/lib/yum/repos/ /var/cache/yum/ -name 'clickhouse-*' -type d -exec rm -rf {} + -sudo rm -f /etc/yum.repos.d/clickhouse.repo -``` - -После этого следуйте [руководству по установке](../getting-started/install.md#from-rpm-packages) - -### Не удаётся запустить Docker-контейнер {#you-cant-run-docker-container} - -Вы выполняете простой `docker run clickhouse/clickhouse-server`, и он аварийно завершается, выводя трассировку стека, похожую на следующую: - -```bash -$ docker run -it clickhouse/clickhouse-server -........ -Poco::Exception. Code: 1000, e.code() = 0, системное исключение: невозможно создать поток, трассировка стека (при копировании этого сообщения всегда включайте строки, приведённые ниже): -``` - - -0. Poco::ThreadImpl::startImpl(Poco::SharedPtr>) @ 0x00000000157c7b34 -1. Poco::Thread::start(Poco::Runnable&) @ 0x00000000157c8a0e -2. BaseDaemon::initializeTerminationAndSignalProcessing() @ 0x000000000d267a14 -3. BaseDaemon::initialize(Poco::Util::Application&) @ 0x000000000d2652cb -4. DB::Server::initialize(Poco::Util::Application&) @ 0x000000000d128b38 -5. Poco::Util::Application::run() @ 0x000000001581cfda -6. DB::Server::run() @ 0x000000000d1288f0 -7. Poco::Util::ServerApplication::run(int, char\*\*) @ 0x0000000015825e27 -8. mainEntryClickHouseServer(int, char\*\*) @ 0x000000000d125b38 -9. main @ 0x0000000007ea4eee -10. ? @ 0x00007f67ff946d90 -11. ? @ 0x00007f67ff946e40 -12. \_start @ 0x00000000062e802e - (version 24.10.1.2812 (official build)) - -``` - -Причина — устаревший демон Docker версии ниже `20.10.10`. Исправить это можно либо обновив Docker, либо запустив `docker run [--privileged | --security-opt seccomp=unconfined]`. Второй вариант несет риски для безопасности. - -``` - - -## Подключение к серверу {#troubleshooting-accepts-no-connections} - -Возможные проблемы: - -* Сервер не запущен. -* Некорректные или неожиданные параметры конфигурации. - -### Сервер не запущен {#server-is-not-running} - -**Проверьте, запущен ли сервер** - -Команда: - -```bash -$ sudo service clickhouse-server status -``` - -Если сервер не запущен, запустите его командой: - -```bash -$ sudo service clickhouse-server start -``` - -**Проверьте логи** - -Основной лог `clickhouse-server` по умолчанию расположен в `/var/log/clickhouse-server/clickhouse-server.log`. - -Если сервер успешно запустился, вы увидите строки: - -* ` Application: starting up.` — Сервер запущен. -* ` Application: Ready for connections.` — Сервер работает и готов принимать подключения. - -Если запуск `clickhouse-server` завершился ошибкой конфигурации, вы увидите строку `` с описанием ошибки. Например: - -```text -2019.01.11 15:23:25.549505 [ 45 ] {} ExternalDictionaries: Не удалось перезагрузить внешний словарь 'event2id': Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Соединение отклонено, e.what() = Соединение отклонено -``` - -Если в конце файла нет сообщения об ошибке, просмотрите весь файл, начиная со строки: - -```text - Приложение: запускается. -``` - -Если вы попытаетесь запустить второй экземпляр `clickhouse-server` на сервере, вы увидите в логе следующее: - -```text -2019.01.11 15:25:11.151730 [ 1 ] {} : Запуск ClickHouse 19.1.0, ревизия 54413 -2019.01.11 15:25:11.154578 [ 1 ] {} Приложение: запускается -2019.01.11 15:25:11.156361 [ 1 ] {} StatusFile: файл состояния ./status уже существует — некорректный перезапуск. Содержимое: -PID: 8510 -Запущен: 2019-01-11 15:24:23 -Ревизия: 54413 - -2019.01.11 15:25:11.156673 [ 1 ] {} Приложение: DB::Exception: невозможно заблокировать файл ./status. Другой экземпляр сервера в этом же каталоге уже запущен. -2019.01.11 15:25:11.156682 [ 1 ] {} Приложение: завершает работу -2019.01.11 15:25:11.156686 [ 1 ] {} Приложение: деинициализация подсистемы: подсистема журналирования -2019.01.11 15:25:11.156716 [ 2 ] {} BaseDaemon: остановка потока SignalListener -``` - -**Просмотр логов systemd** - -Если в логах `clickhouse-server` нет полезной информации или они отсутствуют вовсе, вы можете просмотреть логи `systemd` с помощью команды: - -```bash -$ sudo journalctl -u clickhouse-server -``` - -**Запустите clickhouse-server в интерактивном режиме** - -```bash -$ sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml -``` - -Эта команда запускает сервер как интерактивное приложение со стандартными параметрами скрипта автозапуска. В этом режиме `clickhouse-server` выводит все сообщения о событиях в консоль. - -### Параметры конфигурации {#configuration-parameters} - -Проверьте: - -* Настройки Docker. - - Если вы запускаете ClickHouse в Docker в сети IPv6, убедитесь, что задано `network=host`. - -* Настройки конечной точки. - - Проверьте настройки [listen_host](../operations/server-configuration-parameters/settings.md#listen_host) и [tcp_port](../operations/server-configuration-parameters/settings.md#tcp_port). - - По умолчанию сервер ClickHouse принимает подключения только с localhost. - -* Настройки протокола HTTP. - - Проверьте настройки протокола для HTTP API. - -* Настройки защищённого соединения. - - Проверьте: - - * Настройку [tcp_port_secure](../operations/server-configuration-parameters/settings.md#tcp_port_secure). - * Настройки [SSL-сертификатов](../operations/server-configuration-parameters/settings.md#openssl). - - Используйте корректные параметры при подключении. Например, используйте параметр `port_secure` с `clickhouse_client`. - -* Настройки пользователя. - - Возможно, вы используете неверное имя пользователя или пароль. - - -## Обработка запросов {#troubleshooting-does-not-process-queries} - -Если ClickHouse не может выполнить запрос, он отправляет описание ошибки клиенту. В `clickhouse-client` вы получаете описание ошибки в консоли. Если вы используете HTTP-интерфейс, ClickHouse отправляет описание ошибки в теле ответа. Например: - -```bash -$ curl 'http://localhost:8123/' --data-binary "SELECT a" -Code: 47, e.displayText() = DB::Exception: Неизвестный идентификатор: a. Обратите внимание, что в вашем запросе нет таблиц (секция FROM), контекст: required_names: 'a' source_tables: table_aliases: private_aliases: column_aliases: public_columns: 'a' masked_columns: array_join_columns: source_columns: , e.what() = DB::Exception -``` - -Если запустить `clickhouse-client` с параметром `stack-trace`, ClickHouse вернёт стек вызовов сервера с описанием ошибки. - -Вы можете увидеть сообщение об обрыве соединения. В этом случае повторите запрос. Если соединение обрывается каждый раз при выполнении запроса, проверьте журналы сервера на наличие ошибок. - - -## Эффективность обработки запросов {#troubleshooting-too-slow} - -Если вы замечаете, что ClickHouse работает слишком медленно, необходимо выполнить профилирование нагрузки на ресурсы сервера и сеть, создаваемой вашими запросами. - -Вы можете использовать утилиту clickhouse-benchmark для профилирования запросов. Она показывает количество обработанных запросов в секунду, количество обработанных строк в секунду и перцентили времени обработки запросов. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md deleted file mode 100644 index a1d29b17ef5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md +++ /dev/null @@ -1,224 +0,0 @@ ---- -description: 'Страница о профилировании выделения памяти в ClickHouse' -sidebar_label: 'Профилирование выделения памяти для версий до 25.9' -slug: /operations/allocation-profiling-old -title: 'Профилирование выделения памяти для версий до 25.9' -doc_type: 'reference' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# Профилирование выделения памяти для версий до 25.9 {#allocation-profiling-for-versions-before-259} - -ClickHouse использует [jemalloc](https://github.com/jemalloc/jemalloc) в качестве глобального аллокатора. Jemalloc предоставляет инструменты для выборочного отслеживания и профилирования выделения памяти. -Для удобства профилирования выделения памяти предусмотрены команды `SYSTEM`, а также четырёхбуквенные (4LW) команды в Keeper. - - - -## Сэмплирование выделений памяти и сброс профилей кучи {#sampling-allocations-and-flushing-heap-profiles} - -Если вы хотите выполнять сэмплирование и профилирование выделений памяти в `jemalloc`, необходимо запустить ClickHouse/Keeper с включённым профилированием, задав переменную окружения `MALLOC_CONF`: - -```sh -MALLOC_CONF=background_thread:true,prof:true -``` - -`jemalloc` будет выборочно отслеживать выделения памяти и сохранять информацию во внутренних структурах. - -Вы можете принудительно сбросить текущий профиль `jemalloc`, выполнив: - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -По умолчанию файл профиля кучи будет создан в `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap`, где `_pid_` — это PID ClickHouse, а `_seqnum_` — глобальный порядковый номер текущего профиля кучи.\ -Для Keeper файл по умолчанию — `/tmp/jemalloc_keeper._pid_._seqnum_.heap` и подчиняется тем же правилам. - -Другое место хранения может быть задано путём добавления к переменной окружения `MALLOC_CONF` опции `prof_prefix`.\ -Например, если вы хотите генерировать профили в каталоге `/data`, при этом префиксом имени файла будет `my_current_profile`, вы можете запускать ClickHouse/Keeper со следующей переменной окружения: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_prefix:/data/my_current_profile -``` - -К имени сгенерированного файла будет добавлен префикс PID и порядковый номер. - - -## Анализ профилей кучи {#analyzing-heap-profiles} - -После генерации профилей кучи их необходимо проанализировать.\ -Для этого можно использовать инструмент `jemalloc` под названием [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in). Его можно установить несколькими способами: - -* С помощью пакетного менеджера операционной системы -* Клонировав [репозиторий jemalloc](https://github.com/jemalloc/jemalloc) и выполнив `autogen.sh` в корневом каталоге. В результате в каталоге `bin` появится скрипт `jeprof` - -:::note -`jeprof` использует `addr2line` для генерации стек-трейсов, что может быть очень медленным.\ -В таком случае рекомендуется установить [альтернативную реализацию](https://github.com/gimli-rs/addr2line) этого инструмента. - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -::: - -Существует множество форматов вывода, которые можно получить из профиля кучи с помощью `jeprof`. -Рекомендуется запустить `jeprof --help`, чтобы получить информацию об использовании и различных опциях, которые предоставляет этот инструмент. - -В общем случае команда `jeprof` используется следующим образом: - -```sh -jeprof path/to/binary path/to/heap/profile --output_format [ > output_file] -``` - -Если вы хотите сравнить, какие выделения памяти произошли между двумя профилями, задайте аргумент `base`: - -```sh -jeprof path/to/binary --base path/to/first/heap/profile path/to/second/heap/profile --output_format [ > output_file] -``` - -### Примеры {#examples} - -* если вы хотите создать текстовый файл, в котором каждая процедура записана в отдельной строке: - -```sh -jeprof path/to/binary path/to/heap/profile --text > result.txt -``` - -* если вы хотите создать PDF-файл с графом вызовов: - -```sh -jeprof путь/к/исполняемому/файлу путь/к/профилю/heap --pdf > result.pdf -``` - -### Генерация flame-графа {#generating-flame-graph} - -`jeprof` позволяет создавать свернутые стеки для построения flame-графов. - -Необходимо использовать аргумент `--collapsed`: - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -Затем вы можете использовать множество инструментов для визуализации свернутых стеков. - -Самый популярный — [FlameGraph](https://github.com/brendangregg/FlameGraph), который содержит скрипт под названием `flamegraph.pl`: - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg -``` - -Еще один полезный инструмент — [speedscope](https://www.speedscope.app/), который позволяет анализировать собранные стеки в более интерактивном режиме. - - -## Управление профилировщиком выделений во время работы {#controlling-allocation-profiler-during-runtime} - -Если ClickHouse/Keeper запущен с включённым профилировщиком, становятся доступны дополнительные команды для отключения и включения профилирования выделений во время работы. -С их помощью проще профилировать только отдельные интервалы. - -Чтобы отключить профилировщик: - - - - ```sql - SYSTEM JEMALLOC DISABLE PROFILE - ``` - - - - ```sh - echo jmdp | nc localhost 9181 - ``` - - - -Чтобы включить профилировщик: - - - - ```sql - SYSTEM JEMALLOC ENABLE PROFILE - ``` - - - - ```sh - echo jmep | nc localhost 9181 - ``` - - - -Также можно управлять начальным состоянием профилировщика, установив опцию `prof_active`, которая по умолчанию включена.\ -Например, если вы не хотите сэмплировать выделения памяти во время запуска, а только после, вы можете включить профилировщик позже. Для этого запустите ClickHouse/Keeper со следующей переменной окружения: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_active:false -``` - -Профилировщик можно будет включить позже. - - -## Дополнительные параметры профилировщика {#additional-options-for-profiler} - -В `jemalloc` доступно множество различных параметров, связанных с профилировщиком. Ими можно управлять через переменную окружения `MALLOC_CONF`. -Например, интервал между выборками выделений памяти можно контролировать с помощью `lg_prof_sample`. -Если вы хотите сбрасывать профиль кучи каждые N байт, вы можете включить это с помощью `lg_prof_interval`. - -Для получения полного списка параметров рекомендуется ознакомиться со [справочной страницей](https://jemalloc.net/jemalloc.3.html) `jemalloc`. - - - -## Другие ресурсы {#other-resources} - -ClickHouse/Keeper предоставляют метрики, связанные с `jemalloc`, множеством разных способов. - -:::warning Предупреждение -Важно понимать, что ни одна из этих метрик не синхронизирована с другими, и значения могут расходиться. -::: - -### Системная таблица `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[Справочник](/operations/system-tables/asynchronous_metrics) - -### Системная таблица `jemalloc_bins` {#system-table-jemalloc_bins} - -Содержит информацию о выделении памяти с помощью аллокатора `jemalloc` по различным классам размеров (bins), агрегированную по всем аренам. - -[Справочник](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -Все метрики, связанные с `jemalloc` и доступные в `asynchronous_metrics`, также публикуются через endpoint Prometheus как в ClickHouse, так и в Keeper. - -[Справочник](/operations/server-configuration-parameters/settings#prometheus) - -### 4LW-команда `jmst` в Keeper {#jmst-4lw-command-in-keeper} - -Keeper поддерживает 4LW-команду `jmst`, которая возвращает [базовую статистику аллокатора](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics): - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md deleted file mode 100644 index 9f15f8f9b09..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md +++ /dev/null @@ -1,337 +0,0 @@ ---- -description: 'Страница, описывающая профилирование выделения памяти в ClickHouse' -sidebar_label: 'Профилирование выделения памяти' -slug: /operations/allocation-profiling -title: 'Профилирование выделения памяти' -doc_type: 'guide' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# Профилирование выделений памяти {#allocation-profiling} - -ClickHouse использует [jemalloc](https://github.com/jemalloc/jemalloc) в качестве глобального аллокатора. Jemalloc предоставляет инструменты для сэмплирования и профилирования выделений памяти. -Чтобы сделать профилирование выделений более удобным, ClickHouse и Keeper позволяют управлять сэмплированием с помощью конфигурационных файлов, настроек запроса, команд `SYSTEM` и четырёхбуквенных (4LW) команд в Keeper. -Кроме того, сэмплы могут записываться в таблицу `system.trace_log` с типом `JemallocSample`. - -:::note - -Это руководство применимо к версиям 25.9 и новее. -Для более старых версий см. раздел [профилирование выделений для версий до 25.9](/operations/allocation-profiling-old.md). - -::: - - - -## Сэмплирование выделений памяти {#sampling-allocations} - -Если вы хотите выполнять сэмплирование и профилирование выделений памяти в `jemalloc`, необходимо запускать ClickHouse/Keeper с включённой настройкой конфигурации `jemalloc_enable_global_profiler`. - -```xml - - 1 - -``` - -`jemalloc` будет выборочно отслеживать выделения памяти и сохранять эту информацию во внутренних структурах. - -Вы также можете включить профилирование выделений памяти по каждому запросу с помощью настройки `jemalloc_enable_profiler`. - -:::warning Предупреждение -Поскольку ClickHouse — приложение с интенсивным использованием выделения памяти, выборочное отслеживание jemalloc может привести к дополнительным накладным расходам и снижению производительности. -::: - - -## Хранение выборок jemalloc в `system.trace_log` {#storing-jemalloc-samples-in-system-trace-log} - -Вы можете хранить все выборки jemalloc в `system.trace_log` с типом записи `JemallocSample`. -Чтобы включить это глобально, используйте параметр конфигурации `jemalloc_collect_global_profile_samples_in_trace_log`. - -```xml - - 1 - -``` - -:::warning Предупреждение -Поскольку ClickHouse — приложение с большим количеством операций выделения памяти, сбор всех сэмплов в system.trace_log может привести к значительной нагрузке. -::: - -Вы также можете включить это для отдельного запроса, используя настройку `jemalloc_collect_profile_samples_in_trace_log`. - -### Пример анализа использования памяти запросом с помощью `system.trace_log` {#example-analyzing-memory-usage-trace-log} - -Сначала нам нужно выполнить запрос с включённым профилировщиком памяти jemalloc и собрать для него сэмплы в `system.trace_log`: - -```sql -SELECT * -FROM numbers(1000000) -ORDER BY number DESC -SETTINGS max_bytes_ratio_before_external_sort = 0 -FORMAT `Null` -SETTINGS jemalloc_enable_profiler = 1, jemalloc_collect_profile_samples_in_trace_log = 1 - -Query id: 8678d8fe-62c5-48b8-b0cd-26851c62dd75 - -Ok. - -0 rows in set. Elapsed: 0.009 sec. Processed 1.00 million rows, 8.00 MB (108.58 million rows/s., 868.61 MB/s.) -Peak memory usage: 12.65 MiB. -``` - -:::note -Если ClickHouse был запущен с `jemalloc_enable_global_profiler`, вам не нужно включать `jemalloc_enable_profiler`.\ -То же самое относится к `jemalloc_collect_global_profile_samples_in_trace_log` и `jemalloc_collect_profile_samples_in_trace_log`. -::: - -Очистим `system.trace_log`: - -```sql -SYSTEM FLUSH LOGS trace_log -``` - -и выполнять к нему запрос, чтобы получить потребление памяти нашим запросом для каждого момента времени: - -```sql -WITH per_bucket AS -( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time -) -SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable -FROM per_bucket -ORDER BY bucket_time -``` - -Мы также можем найти момент времени, когда использование памяти достигало максимума: - -```sql -SELECT - argMax(bucket_time, cumulative_size), - max(cumulative_size) -FROM -( - WITH per_bucket AS - ( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time - ) - SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable - FROM per_bucket - ORDER BY bucket_time -) -``` - -Мы можем использовать этот результат, чтобы увидеть, откуда шло больше всего активных выделений памяти в тот момент времени: - - -```sql -SELECT - concat( - '\n', - arrayStringConcat( - arrayMap( - (x, y) -> concat(x, ': ', y), - arrayMap(x -> addressToLine(x), allocation_trace), - arrayMap(x -> demangle(addressToSymbol(x)), allocation_trace) - ), - '\n' - ) - ) AS symbolized_trace, - sum(s) AS per_trace_sum -FROM -( - SELECT - ptr, - sum(size) AS s, - argMax(trace, event_time_microseconds) AS allocation_trace - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - AND event_time_microseconds <= '2025-09-04 11:56:21.737139' - GROUP BY ptr - HAVING s > 0 -) -GROUP BY ALL -ORDER BY per_trace_sum ASC -``` - - -## Сброс профилей кучи {#flushing-heap-profiles} - -По умолчанию файл профиля кучи создаётся в `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap`, где `_pid_` — это PID ClickHouse, а `_seqnum_` — глобальный порядковый номер для текущего профиля кучи.\ -Для Keeper файл по умолчанию — `/tmp/jemalloc_keeper._pid_._seqnum_.heap` и подчиняется тем же правилам. - -Вы можете попросить `jemalloc` сбросить текущий профиль, выполнив: - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - Команда вернёт путь к сброшенному профилю. - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -Другое местоположение можно задать, дополнив переменную окружения `MALLOC_CONF` опцией `prof_prefix`.\ -Например, если вы хотите генерировать профили в каталоге `/data`, где префиксом имени файла будет `my_current_profile`, вы можете запустить ClickHouse/Keeper со следующей переменной окружения: - -```sh -MALLOC_CONF=prof_prefix:/data/my_current_profile -``` - -К имени сгенерированного файла будет добавлен префикс с PID и порядковым номером. - - -## Анализ профилей кучи {#analyzing-heap-profiles} - -После того как профили кучи были сгенерированы, их необходимо проанализировать.\ -Для этого можно использовать инструмент `jemalloc` под названием [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in). Его можно установить несколькими способами: - -* С помощью системного менеджера пакетов -* Клонировать [репозиторий jemalloc](https://github.com/jemalloc/jemalloc) и запустить `autogen.sh` из корневого каталога. В результате в каталоге `bin` появится скрипт `jeprof`. - -:::note -`jeprof` использует `addr2line` для генерации стек-трейсов, что может работать очень медленно.\ -В таком случае рекомендуется установить [альтернативную реализацию](https://github.com/gimli-rs/addr2line) этого инструмента. - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -В качестве альтернативы `llvm-addr2line` работает так же эффективно. - -::: - -Существует множество различных форматов, которые можно получить из профиля кучи с помощью `jeprof`. -Рекомендуется запустить `jeprof --help`, чтобы получить информацию об использовании и о различных опциях, которые предоставляет этот инструмент. - -В целом команду `jeprof` обычно используют следующим образом: - -```sh -jeprof path/to/binary path/to/heap/profile --output_format [ > output_file] -``` - -Если вы хотите сравнить, какие выделения памяти произошли между двумя профилями, укажите аргумент `base`: - -```sh -jeprof path/to/binary --base path/to/first/heap/profile path/to/second/heap/profile --output_format [ > output_file] -``` - -### Примеры {#examples} - -* если вы хотите сгенерировать текстовый файл, в котором каждая процедура будет записана в отдельной строке: - -```sh -jeprof path/to/binary path/to/heap/profile --text > result.txt -``` - -* если вы хотите сгенерировать PDF-файл с графом вызовов: - -```sh -jeprof path/to/binary path/to/heap/profile --pdf > result.pdf -``` - -### Построение flame-графа {#generating-flame-graph} - -`jeprof` позволяет получать свернутые стеки вызовов для построения flame-графов. - -Для этого следует использовать аргумент `--collapsed`: - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -После этого вы можете использовать различные инструменты для визуализации свернутых стеков. - -Самый популярный из них — [FlameGraph](https://github.com/brendangregg/FlameGraph), который содержит скрипт `flamegraph.pl`: - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg -``` - -Еще один полезный инструмент — [speedscope](https://www.speedscope.app/), который позволяет анализировать собранные стеки в более интерактивном режиме. - - -## Дополнительные параметры профилировщика {#additional-options-for-profiler} - -У `jemalloc` есть множество параметров, относящихся к профилировщику. Ими можно управлять, изменяя переменную окружения `MALLOC_CONF`. -Например, интервал между выборками операций выделения памяти можно контролировать с помощью `lg_prof_sample`. -Если вы хотите создавать дамп профиля кучи каждые N байт, вы можете включить это с помощью `lg_prof_interval`. - -Рекомендуется ознакомиться со [справочной страницей](https://jemalloc.net/jemalloc.3.html) `jemalloc` для получения полного перечня параметров. - - - -## Другие ресурсы {#other-resources} - -ClickHouse/Keeper предоставляют метрики, связанные с `jemalloc`, множеством разных способов. - -:::warning Warning -Важно понимать, что ни одна из этих метрик не синхронизирована с другими, и значения со временем могут расходиться. -::: - -### Системная таблица `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[Справочник](/operations/system-tables/asynchronous_metrics) - -### Системная таблица `jemalloc_bins` {#system-table-jemalloc_bins} - -Содержит информацию о выделении памяти, выполненном через аллокатор jemalloc в разных классах размеров (bins), агрегированную по всем аренам. - -[Справочник](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -Все метрики, связанные с `jemalloc` из `asynchronous_metrics`, также доступны через конечную точку Prometheus как в ClickHouse, так и в Keeper. - -[Справочник](/operations/server-configuration-parameters/settings#prometheus) - -### Команда `jmst` 4LW в Keeper {#jmst-4lw-command-in-keeper} - -Keeper поддерживает команду `jmst` 4LW, которая возвращает [базовую статистику аллокатора](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics): - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/analyzer.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/analyzer.md deleted file mode 100644 index 7312ee5ff01..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/analyzer.md +++ /dev/null @@ -1,201 +0,0 @@ ---- -description: 'Страница, посвящённая анализатору запросов ClickHouse' -keywords: ['анализатор'] -sidebar_label: 'Анализатор' -slug: /operations/analyzer -title: 'Анализатор' -doc_type: 'reference' ---- - -# Анализатор {#analyzer} - -В версии ClickHouse `24.3` новый анализатор запросов включён по умолчанию. -Подробнее о том, как он работает, можно прочитать [здесь](/guides/developer/understanding-query-execution-with-the-analyzer#analyzer). - -## Известные несовместимости {#known-incompatibilities} - -Несмотря на исправление большого числа ошибок и внедрение новых оптимизаций, также были внесены некоторые изменения в поведении ClickHouse, нарушающие обратную совместимость. Пожалуйста, ознакомьтесь со следующими изменениями, чтобы понять, как переписать ваши запросы для нового анализатора. - -### Некорректные запросы больше не оптимизируются {#invalid-queries-are-no-longer-optimized} - -Предыдущая инфраструктура планирования запросов применяла оптимизации на уровне AST до шага проверки запроса. -Оптимизации могли переписать исходный запрос так, чтобы он стал корректным и исполнимым. - -В новом анализаторе проверка запроса выполняется до шага оптимизации. -Это означает, что некорректные запросы, которые ранее можно было выполнить, больше не поддерживаются. -В таких случаях запрос должен быть исправлен вручную. - -#### Пример 1 {#example-1} - -Следующий запрос использует столбец `number` в списке проекции, хотя после агрегации доступен только `toString(number)`. -В старом анализаторе `GROUP BY toString(number)` оптимизировался в `GROUP BY number,` что делало запрос корректным. - -```sql -SELECT number -FROM numbers(1) -GROUP BY toString(number) -``` - -#### Пример 2 {#example-2} - -Та же проблема возникает в этом запросе. Столбец `number` используется после агрегации с другим ключом. -Предыдущий анализатор запросов исправил этот запрос, переместив фильтр `number > 5` из условия `HAVING` в условие `WHERE`. - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -GROUP BY n -HAVING number > 5 -``` - -Чтобы исправить запрос, перенесите все условия, относящиеся к неагрегированным столбцам, в предложение `WHERE`, чтобы он соответствовал стандартному синтаксису SQL: - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -WHERE number > 5 -GROUP BY n -``` - -### `CREATE VIEW` с некорректным запросом {#create-view-with-invalid-query} - -Новый анализатор всегда выполняет проверку типов. -Ранее можно было создать `VIEW` с некорректным запросом `SELECT`. -В этом случае первое обращение к представлению с `SELECT` или `INSERT` (для `MATERIALIZED VIEW`) завершалось с ошибкой. - -Теперь создать `VIEW` таким способом невозможно. - -#### Пример {#example-view} - -```sql -CREATE TABLE source (data String) -ENGINE=MergeTree -ORDER BY tuple(); - -CREATE VIEW some_view -AS SELECT JSONExtract(data, 'test', 'DateTime64(3)') -FROM source; -``` - -### Известные несовместимости предложения `JOIN` {#known-incompatibilities-of-the-join-clause} - -#### `JOIN` с использованием столбца из проекции {#join-using-column-from-projection} - -Псевдоним из списка `SELECT` по умолчанию не может использоваться как ключ `JOIN USING`. - -Новая настройка `analyzer_compatibility_join_using_top_level_identifier` при включении изменяет поведение `JOIN USING`, так что при разрешении идентификаторов приоритет отдается выражениям из списка проекций запроса `SELECT`, а не столбцам из левой таблицы напрямую. - -Например: - -```sql -SELECT a + 1 AS b, t2.s -FROM VALUES('a UInt64, b UInt64', (1, 1)) AS t1 -JOIN VALUES('b UInt64, s String', (1, 'один'), (2, 'два')) t2 -USING (b); -``` - -При установке параметра `analyzer_compatibility_join_using_top_level_identifier` в значение `true` условие соединения интерпретируется как `t1.a + 1 = t2.b`, что соответствует поведению более ранних версий. -Результатом будет `2, 'two'`. -Если настройка установлена в значение `false`, по умолчанию используется условие соединения `t1.b = t2.b`, и запрос вернет `2, 'one'`. -Если `b` отсутствует в `t1`, запрос завершится с ошибкой. - -#### Изменения в поведении с `JOIN USING` и столбцами `ALIAS`/`MATERIALIZED` {#changes-in-behavior-with-join-using-and-aliasmaterialized-columns} - -В новом анализаторе использование `*` в запросе `JOIN USING`, который включает столбцы `ALIAS` или `MATERIALIZED`, по умолчанию будет включать эти столбцы в результирующий набор. - -Например: - -```sql -CREATE TABLE t1 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t1 VALUES (1), (2); - -CREATE TABLE t2 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t2 VALUES (2), (3); - -SELECT * FROM t1 -FULL JOIN t2 USING (payload); -``` - -В новом анализаторе результат этого запроса будет включать столбец `payload` вместе со столбцом `id` из обеих таблиц. -Напротив, предыдущий анализатор добавлял бы эти столбцы-`ALIAS` только в том случае, если были включены определённые настройки (`asterisk_include_alias_columns` или `asterisk_include_materialized_columns`), -и столбцы могли бы идти в другом порядке. - -Чтобы обеспечить стабильные и предсказуемые результаты, особенно при миграции старых запросов на новый анализатор, рекомендуется явно указывать столбцы в предложении `SELECT` вместо использования `*`. - -#### Обработка модификаторов типов для столбцов в предложении `USING` {#handling-of-type-modifiers-for-columns-in-using-clause} - -В новой версии анализатора правила определения общего супертипа для столбцов, указанных в предложении `USING`, были унифицированы, чтобы давать более предсказуемые результаты, -особенно при работе с модификаторами типов, такими как `LowCardinality` и `Nullable`. - -* `LowCardinality(T)` и `T`: когда столбец типа `LowCardinality(T)` соединяется со столбцом типа `T`, результирующим общим супертипом будет `T`, то есть модификатор `LowCardinality` фактически отбрасывается. -* `Nullable(T)` и `T`: когда столбец типа `Nullable(T)` соединяется со столбцом типа `T`, результирующим общим супертипом будет `Nullable(T)`, что гарантирует сохранение возможности значений `NULL`. - -Например: - -```sql -SELECT id, toTypeName(id) -FROM VALUES('id LowCardinality(String)', ('a')) AS t1 -FULL OUTER JOIN VALUES('id String', ('b')) AS t2 -USING (id); -``` - -В этом запросе общий надтип для `id` определяется как `String`, при этом модификатор `LowCardinality` из `t1` отбрасывается. - -### Изменения имён столбцов проекций {#projection-column-names-changes} - -При вычислении имён столбцов проекций алиасы не подставляются. - -```sql -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 0 -FORMAT PrettyCompact - - ┌─x─┬─plus(plus(1, 1), 1)─┐ -1. │ 2 │ 3 │ - └───┴─────────────────────┘ - -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 1 -FORMAT PrettyCompact - - ┌─x─┬─plus(x, 1)─┐ -1. │ 2 │ 3 │ - └───┴────────────┘ -``` - -### Несовместимые типы аргументов функций {#incompatible-function-arguments-types} - -В новом анализаторе вывод типов происходит на этапе первоначального анализа запроса. -Это изменение означает, что проверки типов выполняются до вычисления с коротким замыканием; поэтому аргументы функции `if` всегда должны иметь общий супертип. - -Например, следующий запрос завершится с ошибкой `There is no supertype for types Array(UInt8), String because some of them are Array and some of them are not`: - -```sql -SELECT toTypeName(if(0, [2, 3, 4], 'String')) -``` - -### Гетерогенные кластеры {#heterogeneous-clusters} - -Новый анализатор существенно изменяет протокол взаимодействия между серверами в кластере. Поэтому выполнение распределённых запросов на серверах с разными значениями настройки `enable_analyzer` невозможно. - -### Мутации интерпретируются предыдущим анализатором {#mutations-are-interpreted-by-previous-analyzer} - -Мутации по-прежнему используют старый анализатор. -Это означает, что некоторые новые возможности ClickHouse SQL нельзя использовать в мутациях. Например, конструкцию `QUALIFY`. -Статус можно проверить [здесь](https://github.com/ClickHouse/ClickHouse/issues/61563). - -### Неподдерживаемые возможности {#unsupported-features} - -Список возможностей, которые новый анализатор в данный момент не поддерживает, приведён ниже: - -* Индекс Annoy. -* Индекс Hypothesis. Работа ведётся [здесь](https://github.com/ClickHouse/ClickHouse/pull/48381). -* Window view не поддерживается. В будущем поддержка не планируется. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/backup.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/backup.md deleted file mode 100644 index 51d731a7f13..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/backup.md +++ /dev/null @@ -1,575 +0,0 @@ ---- -description: 'Руководство по резервному копированию и восстановлению баз данных и таблиц ClickHouse' -sidebar_label: 'Резервное копирование и восстановление' -sidebar_position: 10 -slug: /operations/backup -title: 'Резервное копирование и восстановление' -doc_type: 'guide' ---- - - - -# Резервное копирование и восстановление {#backup-and-restore} - -- [Резервное копирование на локальный диск](#backup-to-a-local-disk) -- [Настройка резервного копирования и восстановления с использованием конечной точки S3](#configuring-backuprestore-to-use-an-s3-endpoint) -- [Резервное копирование и восстановление с использованием диска S3](#backuprestore-using-an-s3-disk) -- [Альтернативы](#alternatives) - - - -## Сводка команд {#command-summary} - -```bash - BACKUP|RESTORE - TABLE [db.]table_name [AS [db.]table_name_in_backup] - [PARTITION[S] partition_expr [, ...]] | - DICTIONARY [db.]dictionary_name [AS [db.]name_in_backup] | - DATABASE database_name [AS database_name_in_backup] - [EXCEPT TABLES ...] | - TEMPORARY TABLE table_name [AS table_name_in_backup] | - VIEW view_name [AS view_name_in_backup] | - ALL [EXCEPT {TABLES|DATABASES}...] } [, ...] - [ON CLUSTER 'cluster_name'] - TO|FROM File('/') | Disk('', '/') | S3('/', '', '') - [SETTINGS base_backup = File('/') | Disk(...) | S3('/', '', '')] - [SYNC|ASYNC] - -``` - -:::note ALL -До версии ClickHouse 23.4 `ALL` можно было использовать только с командой `RESTORE`. -::: - - -## Общие сведения {#background} - -Хотя [репликация](../engines/table-engines/mergetree-family/replication.md) обеспечивает защиту от отказов оборудования, она не защищает от человеческих ошибок: случайного удаления данных, удаления не той таблицы или таблицы не в том кластере, а также ошибок в программном обеспечении, приводящих к некорректной обработке данных или их порче. Во многих случаях подобные ошибки затронут все реплики. В ClickHouse есть встроенные механизмы защиты от некоторых типов ошибок — например, по умолчанию [нельзя просто так удалить таблицы с движком типа MergeTree, содержащие более 50 ГБ данных](/operations/settings/settings#max_table_size_to_drop). Однако эти механизмы не покрывают все возможные случаи и могут быть обойдены. - -Чтобы эффективно снизить риск последствий возможных человеческих ошибок, вам следует заранее тщательно продумать стратегию резервного копирования и восстановления данных. - -У каждой компании свои доступные ресурсы и бизнес‑требования, поэтому не существует универсального решения для резервного копирования и восстановления ClickHouse, которое подходило бы для любой ситуации. То, что работает для одного гигабайта данных, скорее всего, не будет работать для десятков петабайт. Существует множество возможных подходов с собственными достоинствами и недостатками, которые будут рассмотрены ниже. Имеет смысл использовать несколько подходов, а не полагаться только на один, чтобы компенсировать их недостатки. - -:::note -Имейте в виду, что если вы сделали резервную копию и ни разу не пытались выполнить восстановление, велика вероятность, что восстановление не сработает должным образом, когда оно действительно потребуется (или, по крайней мере, займет больше времени, чем бизнес может себе позволить). Поэтому, какой бы подход к резервному копированию вы ни выбрали, обязательно автоматизируйте и процесс восстановления и регулярно отрабатывайте его на отдельном кластере ClickHouse. -::: - - - -## Резервное копирование на локальный диск {#backup-to-a-local-disk} - -### Настройка хранилища резервных копий {#configure-a-backup-destination} - -В примерах ниже вы увидите, что хранилище резервной копии указывается как `Disk('backups', '1.zip')`. Чтобы подготовить хранилище, создайте файл `/etc/clickhouse-server/config.d/backup_disk.xml`, в котором будет задано место размещения резервных копий. Например, этот файл определяет диск с именем `backups`, а затем добавляет этот диск в список **backups > allowed_disk**: - -```xml - - - - - - local - /backups/ - - - - - - backups - /backups/ - - - -``` - -### Параметры {#parameters} - -Резервные копии могут быть полными или инкрементальными и могут включать таблицы (включая материализованные представления, проекции и словари) и базы данных. Резервное копирование может выполняться синхронно (по умолчанию) или асинхронно. Резервные копии могут быть сжаты. Резервные копии могут быть защищены паролем. - -Операторы BACKUP и RESTORE принимают список имен баз данных (DATABASE) и таблиц (TABLE), место назначения (или источник), опции и настройки: - -* Место назначения для резервной копии или источник для восстановления. Оно задаётся на основе диска, определенного ранее. Например, `Disk('backups', 'filename.zip')` -* ASYNC: выполнять резервное копирование или восстановление асинхронно -* PARTITIONS: список партиций для восстановления -* SETTINGS: - * `id`: идентификатор операции резервного копирования или восстановления. Если он не задан или пустой, будет использован случайным образом сгенерированный UUID. - Если он явно установлен в непустую строку, то должен отличаться каждый раз. Этот `id` используется для поиска строк в таблице `system.backups`, относящихся к конкретной операции резервного копирования или восстановления. - * [`compression_method`](/sql-reference/statements/create/table#column_compression_codec) и compression_level - * `password` для файла на диске - * `base_backup`: место назначения предыдущей резервной копии этого источника. Например, `Disk('backups', '1.zip')` - * `use_same_s3_credentials_for_base_backup`: должны ли базовые резервные копии в S3 наследовать учетные данные из запроса. Работает только с `S3`. - * `use_same_password_for_base_backup`: должен ли архив базовой резервной копии наследовать пароль из запроса. - * `structure_only`: если включено, позволяет выполнять резервное копирование или восстановление только операторов CREATE без данных таблиц - * `storage_policy`: политика хранения для восстанавливаемых таблиц. См. [Использование нескольких блочных устройств для хранения данных](../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes). Эта настройка применима только к команде `RESTORE`. Указанная политика хранения применяется только к таблицам с движком из семейства `MergeTree`. - * `s3_storage_class`: класс хранения, используемый для резервной копии в S3. Например, `STANDARD` - * `azure_attempt_to_create_container`: при использовании Azure Blob Storage — пытаться ли создать указанный контейнер, если он не существует. Значение по умолчанию: true. - * [основные настройки](/operations/settings/settings) также можно использовать здесь - -### Примеры использования {#usage-examples} - -Создание резервной копии таблицы и последующее восстановление: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.zip') -``` - -Соответствующая команда восстановления: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -``` - -:::note -Выполнение указанной выше команды RESTORE завершится с ошибкой, если таблица `test.table` содержит данные. Чтобы протестировать RESTORE, вам потребуется удалить таблицу или использовать настройку `allow_non_empty_tables=true`: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -SETTINGS allow_non_empty_tables=true -``` - -::: - -Таблицы можно восстанавливать или создавать их резервные копии под новыми именами: - -```sql -RESTORE TABLE test.table AS test.table2 FROM Disk('backups', '1.zip') -``` - -```sql -BACKUP TABLE test.table3 AS test.table4 TO Disk('backups', '2.zip') -``` - -### Инкрементные бэкапы {#incremental-backups} - -Инкрементные бэкапы можно создавать, указывая `base_backup`. -:::note -Инкрементные бэкапы зависят от базового бэкапа. Базовый бэкап должен оставаться доступным, чтобы можно было восстановиться из инкрементного бэкапа. -::: - - -Сохраняйте новые данные инкрементально. Настройка `base_backup` приводит к тому, что данные, появившиеся после предыдущей резервной копии в `Disk('backups', 'd.zip')`, сохраняются в `Disk('backups', 'incremental-a.zip')`: - -```sql -BACKUP TABLE test.table TO Disk('backups', 'incremental-a.zip') - SETTINGS base_backup = Disk('backups', 'd.zip') -``` - -Восстановите все данные из инкрементной резервной копии и `base_backup` в новую таблицу `test.table2`: - -```sql -RESTORE TABLE test.table AS test.table2 - FROM Disk('backups', 'incremental-a.zip'); -``` - -### Назначить пароль для резервной копии {#assign-a-password-to-the-backup} - -Резервные копии, записанные на диск, можно защитить паролем: - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -Восстановление: - -```sql -RESTORE TABLE test.table - FROM Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -### Настройки сжатия {#compression-settings} - -Если вы хотите указать метод или уровень сжатия: - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'filename.zip') - SETTINGS compression_method='lzma', compression_level=3 -``` - -### Восстановление отдельных партиций {#restore-specific-partitions} - -Если необходимо восстановить определённые партиции, связанные с таблицей, их можно указать. Чтобы восстановить партиции 1 и 4 из резервной копии: - -```sql -RESTORE TABLE test.table PARTITIONS '2', '3' - FROM Disk('backups', 'filename.zip') -``` - -### Резервные копии в виде tar-архивов {#backups-as-tar-archives} - -Резервные копии также могут храниться в виде tar-архивов. Функциональность такая же, как для zip, за исключением того, что пароли не поддерживаются. - -Создайте резервную копию в формате tar: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar') -``` - -Соответствующая операция восстановления: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.tar') -``` - -Чтобы изменить метод сжатия, к имени резервной копии нужно добавить соответствующее расширение файла. Например, чтобы сжать tar-архив с помощью gzip: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar.gz') -``` - -Поддерживаемые суффиксы файлов архивов: `tar.gz`, `.tgz`, `tar.bz2`, `tar.lzma`, `.tar.zst`, `.tzst` и `.tar.xz`. - -### Проверка статуса резервных копий {#check-the-status-of-backups} - -Команда резервного копирования возвращает `id` и `status`; этот `id` можно использовать, чтобы получить статус резервной копии. Это удобно для контроля прогресса длительных асинхронных (ASYNC) резервных копий. В примере ниже показан сбой, который произошёл при попытке перезаписать существующий файл резервной копии: - -```sql -BACKUP TABLE helloworld.my_first_table TO Disk('backups', '1.zip') ASYNC -``` - -```response -┌─id───────────────────────────────────┬─status──────────┐ -│ 7678b0b3-f519-4e6e-811f-5a0781a4eb52 │ CREATING_BACKUP │ -└──────────────────────────────────────┴─────────────────┘ - -1 строка в наборе. Затрачено: 0.001 сек. -``` - -```sql -SELECT - * -FROM system.backups -WHERE id='7678b0b3-f519-4e6e-811f-5a0781a4eb52' -FORMAT Vertical -``` - -```response -Row 1: -────── -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -#highlight-next-line -status: BACKUP_FAILED -num_files: 0 -uncompressed_size: 0 -compressed_size: 0 -#highlight-next-line -error: Code: 598. DB::Exception: Резервная копия Disk('backups', '1.zip') уже существует. (BACKUP_ALREADY_EXISTS) (версия 22.8.2.11 (официальная сборка)) -start_time: 2022-08-30 09:21:46 -end_time: 2022-08-30 09:21:46 - -Получена 1 строка. Прошло: 0.002 сек. -``` - - -Помимо таблицы `system.backups`, все операции резервного копирования и восстановления также отслеживаются в системной таблице журнала [backup_log](../operations/system-tables/backup_log.md): - -```sql -SELECT * -FROM system.backup_log -WHERE id = '7678b0b3-f519-4e6e-811f-5a0781a4eb52' -ORDER BY event_time_microseconds ASC -FORMAT Vertical -``` - -```response -Строка 1: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.097414 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-18 11:13:43 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Строка 2: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.174782 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: BACKUP_FAILED -#highlight-next-line -error: Code: 598. DB::Exception: Backup Disk('backups', '1.zip') already exists. (BACKUP_ALREADY_EXISTS) (version 23.8.1.1) -start_time: 2023-08-18 11:13:43 -end_time: 2023-08-18 11:13:43 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -2 строки в наборе. Затрачено: 0.075 сек. -``` - - -## Настройка BACKUP/RESTORE для использования S3 Endpoint {#configuring-backuprestore-to-use-an-s3-endpoint} - -Чтобы записывать резервные копии в бакет S3, вам нужны три параметра: - -* S3 endpoint, - например `https://mars-doc-test.s3.amazonaws.com/backup-S3/` -* Access key ID, - например `ABC123` -* Secret access key, - например `Abc+123` - -:::note -Создание бакета S3 рассматривается в разделе [Use S3 Object Storage as a ClickHouse disk](/integrations/data-ingestion/s3/index.md#configuring-s3-for-clickhouse-use); после сохранения политики просто вернитесь к этому документу — настраивать ClickHouse для использования этого бакета S3 не требуется. -::: - -Назначение для резервной копии указывается следующим образом: - -```sql -S3('<конечная точка S3>/<каталог>', '', '<секретный ключ доступа>') -``` - -```sql -CREATE TABLE data -( - `key` Int, - `value` String, - `array` Array(String) -) -ENGINE = MergeTree -ORDER BY tuple() -``` - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 1000 -``` - -### Создание базового (начального) бэкапа {#create-a-base-initial-backup} - -Инкрементные бэкапы требуют *базового* бэкапа, от которого они будут выполняться; этот пример будет -использован позже как базовый бэкап. Первый параметр назначения S3 — это endpoint S3, после которого указывается каталог внутри бакета, который будет использован для этого бэкапа. В этом примере каталог называется `my_backup`. - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ de442b75-a66c-4a3c-a193-f76f278c70f3 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### Добавьте больше данных {#add-more-data} - -Инкрементные резервные копии содержат разницу между базовой резервной копией и текущим содержимым резервируемой таблицы. Добавьте больше данных перед созданием инкрементной резервной копии: - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 100 -``` - -### Выполнение инкрементного бэкапа {#take-an-incremental-backup} - -Эта команда бэкапа аналогична команде базового бэкапа, но дополнительно задаёт `SETTINGS base_backup` и местоположение базового бэкапа. Обратите внимание, что назначение для инкрементного бэкапа — это не тот же каталог, что и для базового; используется тот же endpoint, но с другим целевым каталогом внутри бакета. Базовый бэкап находится в `my_backup`, а инкрементный будет записан в `my_incremental`: - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') SETTINGS base_backup = S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ f6cd3900-850f-41c9-94f1-0c4df33ea528 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### Восстановление из инкрементного бэкапа {#restore-from-the-incremental-backup} - -Эта команда восстанавливает инкрементный бэкап в новую таблицу `data3`. Обратите внимание, что при восстановлении инкрементного бэкапа также восстанавливается базовый бэкап. При восстановлении указывайте только инкрементный бэкап: - -```sql -RESTORE TABLE data AS data3 FROM S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ ff0c8c39-7dff-4324-a241-000796de11ca │ RESTORED │ -└──────────────────────────────────────┴──────────┘ -``` - -### Проверьте количество {#verify-the-count} - - -В исходную таблицу `data` было выполнено две операции вставки: одна на 1 000 строк, другая на 100 строк, всего 1 100 строк. Проверьте, что восстановленная таблица содержит 1 100 строк: - -```sql -SELECT count() -FROM data3 -``` - -```response -┌─count()─┐ -│ 1100 │ -└─────────┘ -``` - - -### Проверьте содержимое {#verify-the-content} - -Здесь сравнивается содержимое исходной таблицы `data` с восстановленной таблицей `data3`: - -```sql -SELECT throwIf(( - SELECT groupArray(tuple(*)) - FROM data - ) != ( - SELECT groupArray(tuple(*)) - FROM data3 - ), 'Данные не совпадают после операций BACKUP/RESTORE') -``` - -## BACKUP/RESTORE с использованием диска S3 {#backuprestore-using-an-s3-disk} - -Также можно выполнять `BACKUP`/`RESTORE` в S3, настроив диск S3 в конфигурации хранилища ClickHouse. Настройте диск следующим образом, добавив файл в каталог `/etc/clickhouse-server/config.d`: - -```xml - - - - - s3_plain - - - - - - - - -
- s3_plain -
-
-
-
-
- - - s3_plain - -
-``` - -А затем выполните `BACKUP`/`RESTORE` как обычно: - -```sql -BACKUP TABLE data TO Disk('s3_plain', 'cloud_backup'); -RESTORE TABLE data AS data_restored FROM Disk('s3_plain', 'cloud_backup'); -``` - -:::note -Но имейте в виду, что: - -* Этот диск не должен использоваться для самого `MergeTree`, только для `BACKUP`/`RESTORE` -* Если ваши таблицы используют хранилище S3, то будет предпринята попытка использовать серверное копирование на стороне S3 с помощью вызовов `CopyObject` для копирования частей в целевой бакет, используя его учетные данные. Если произойдет ошибка аутентификации, будет использован резервный вариант — метод копирования через буфер (скачивание частей и их последующая загрузка), который крайне неэффективен. В этом случае имеет смысл убедиться, что у вас есть права `read` на исходный бакет при использовании учетных данных целевого бакета. - ::: - - -## Использование именованных коллекций {#using-named-collections} - -Именованные коллекции можно использовать в параметрах команд `BACKUP/RESTORE`. Пример см. [здесь](./named-collections.md#named-collections-for-backups). - - - -## Альтернативы {#alternatives} - -ClickHouse хранит данные на диске, и существует множество способов резервного копирования дисков. Ниже перечислены некоторые альтернативы, которые уже применялись на практике и могут хорошо подойти для вашей среды. - -### Дублирование исходных данных в другом месте {#duplicating-source-data-somewhere-else} - -Часто данные, которые поступают в ClickHouse, доставляются через некоторую форму постоянной очереди, такую как [Apache Kafka](https://kafka.apache.org). В этом случае можно настроить дополнительный набор подписчиков, которые будут читать тот же поток данных в момент его записи в ClickHouse и сохранять его в холодном хранилище. У большинства компаний уже есть некоторое стандартное рекомендованное холодное хранилище, которым может быть объектное хранилище или распределённая файловая система, такая как [HDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html). - -### Снимки файловой системы {#filesystem-snapshots} - -Некоторые локальные файловые системы предоставляют функциональность создания снимков (например, [ZFS](https://en.wikipedia.org/wiki/ZFS)), но они могут быть не лучшим выбором для обработки онлайн‑запросов. Возможным решением является создание дополнительных реплик с такой файловой системой и их исключение из таблиц [Distributed](../engines/table-engines/special/distributed.md), которые используются для запросов `SELECT`. Снимки на таких репликах будут недоступны для любых запросов, изменяющих данные. В качестве бонуса эти реплики могут иметь специальные аппаратные конфигурации с большим числом дисков на сервер, что будет экономически выгодно. - -Для небольших объёмов данных может также подойти простой `INSERT INTO ... SELECT ...` в удалённые таблицы. - -### Операции с частями {#manipulations-with-parts} - -ClickHouse позволяет использовать запрос `ALTER TABLE ... FREEZE PARTITION ...` для создания локальной копии разделов таблицы. Это реализовано с использованием жёстких ссылок в папку `/var/lib/clickhouse/shadow/`, поэтому обычно это не потребляет дополнительное дисковое пространство для старых данных. Созданные копии файлов не обрабатываются сервером ClickHouse, поэтому вы можете просто оставить их там: у вас будет простой бэкап, не требующий какой‑либо дополнительной внешней системы, но он по‑прежнему будет подвержен аппаратным сбоям. По этой причине лучше скопировать их на удалённый ресурс, а затем удалить локальные копии. Распределённые файловые системы и объектные хранилища по‑прежнему являются хорошими вариантами для этого, но могут подойти и обычные подключённые файловые серверы с достаточной ёмкостью (в этом случае передача будет происходить по сетевой файловой системе или, возможно, с помощью [rsync](https://en.wikipedia.org/wiki/Rsync)). -Данные могут быть восстановлены из резервной копии с помощью `ALTER TABLE ... ATTACH PARTITION ...`. - -Для получения дополнительной информации о запросах, связанных с операциями с разделами, смотрите [документацию по ALTER](/sql-reference/statements/alter/partition). - -Для автоматизации этого подхода доступен сторонний инструмент: [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup). - - - -## Настройки для запрета одновременного выполнения резервного копирования и восстановления {#settings-to-disallow-concurrent-backuprestore} - -Чтобы запретить одновременное выполнение резервного копирования и восстановления, используйте соответственно следующие настройки. - -```xml - - - false - false - - -``` - -Значение по умолчанию для обоих параметров — `true`, поэтому по умолчанию допускается одновременное выполнение операций резервного копирования и восстановления. -Если на кластере эти настройки имеют значение `false`, в каждый момент времени на кластере может выполняться только одна операция резервного копирования или восстановления. - - -## Настройка BACKUP/RESTORE для использования конечной точки AzureBlobStorage {#configuring-backuprestore-to-use-an-azureblobstorage-endpoint} - -Чтобы сохранять резервные копии в контейнер AzureBlobStorage, вам необходима следующая информация: - -* Строка подключения / URL конечной точки AzureBlobStorage, -* Контейнер, -* Путь, -* Имя учетной записи (если указан URL), -* Ключ учетной записи (если указан URL). - -Назначение резервной копии задаётся следующим образом: - -```sql -AzureBlobStorage('/', '', '', '', '') -``` - -```sql -BACKUP TABLE data TO AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -RESTORE TABLE data AS data_restored FROM AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -``` - - -## Резервное копирование системных таблиц {#backup-up-system-tables} - -Системные таблицы также могут быть включены в процессы резервного копирования и восстановления, но необходимость этого зависит от конкретного сценария использования. - -### Резервное копирование лог-таблиц {#backing-up-log-tables} - -Системные таблицы, хранящие исторические данные, такие как таблицы с суффиксом _log (например, `query_log`, `part_log`), можно сохранять в резервных копиях и восстанавливать как любые другие таблицы. Если ваш сценарий использования предполагает анализ исторических данных — например, использование `query_log` для отслеживания производительности запросов или отладки проблем — рекомендуется включить эти таблицы в стратегию резервного копирования. Однако, если исторические данные из этих таблиц не требуются, их можно исключить для экономии дискового пространства, используемого под резервные копии. - -### Резервное копирование таблиц управления доступом {#backing-up-access-management-tables} - -Системные таблицы, связанные с управлением доступом, такие как users, roles, row_policies, settings_profiles и quotas, обрабатываются особым образом при операциях резервного копирования и восстановления. Когда эти таблицы включены в резервную копию, их содержимое экспортируется в специальный файл `accessXX.txt`, который содержит эквивалентные SQL-операторы для создания и настройки объектов управления доступом. При восстановлении процесс восстановления интерпретирует эти файлы и повторно применяет SQL-команды для воссоздания пользователей, ролей и других конфигураций. - -Эта возможность позволяет сохранять и восстанавливать конфигурацию управления доступом к кластеру ClickHouse как часть общей конфигурации кластера. - -Примечание: эта функциональность работает только для конфигураций, управляемых через SQL-команды (см. ["SQL-driven Access Control and Account Management"](/operations/access-rights#enabling-access-control)). Конфигурации доступа, определённые в конфигурационных файлах сервера ClickHouse (например, `users.xml`), не включаются в резервные копии и не могут быть восстановлены этим способом. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/caches.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/caches.md deleted file mode 100644 index 3bd85897a8d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/caches.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'При выполнении запросов ClickHouse использует различные кэши.' -sidebar_label: 'Кэши' -sidebar_position: 65 -slug: /operations/caches -title: 'Типы кэша' -keywords: ['cache'] -doc_type: 'reference' ---- - -# Типы кэша {#cache-types} - -При выполнении запросов ClickHouse использует различные кэши, чтобы ускорить выполнение запросов -и сократить необходимость чтения с диска или записи на диск. - -Основные типы кэша: - -* `mark_cache` — кэш [меток](/development/architecture#merge-tree), используемый движками таблиц семейства [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md). -* `uncompressed_cache` — кэш несжатых данных, используемый движками таблиц семейства [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md). -* Кэш страниц операционной системы (используется косвенно, для файлов с данными). - -Также существует множество дополнительных типов кэша: - -* DNS-кэш. -* Кэш [Regexp](/interfaces/formats/Regexp). -* Кэш скомпилированных выражений. -* Кэш [индекса векторного сходства](../engines/table-engines/mergetree-family/annindexes.md). -* Кэш [текстового индекса](../engines/table-engines/mergetree-family/invertedindexes.md#caching). -* Кэш схем [формата Avro](/interfaces/formats/Avro). -* Кэш данных [словарей](../sql-reference/dictionaries/index.md). -* Кэш инференса схем. -* [Кэш файловой системы](storing-data.md) поверх S3, Azure, локальных и других дисков. -* [Пользовательский (userspace) кэш страниц](/operations/userspace-page-cache). -* [Кэш запросов](query-cache.md). -* [Кэш условий запросов](query-condition-cache.md). -* Кэш схем форматов. - -Если требуется сбросить один из кэшей для настройки производительности, устранения неполадок или по причинам, -связанным с согласованностью данных, вы можете использовать оператор [`SYSTEM DROP ... CACHE`](../sql-reference/statements/system.md). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md deleted file mode 100644 index ce7ec1f4480..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -description: 'Руководство по обнаружению кластеров в ClickHouse' -sidebar_label: 'Обнаружение кластеров' -slug: /operations/cluster-discovery -title: 'Обнаружение кластеров' -doc_type: 'guide' ---- - -# Обнаружение кластера {#cluster-discovery} - -## Обзор {#overview} - -Функция Cluster Discovery в ClickHouse упрощает конфигурацию кластера, позволяя узлам автоматически обнаруживать и регистрировать себя без необходимости явного задания в конфигурационных файлах. Это особенно полезно в случаях, когда ручное описание каждого узла становится затруднительным. - -:::note - -Cluster Discovery — экспериментальная функция, и в будущих версиях она может быть изменена или удалена. -Чтобы включить её, добавьте настройку `allow_experimental_cluster_discovery` в конфигурационный файл: - -```xml - - - 1 - - -``` - -::: - -## Конфигурация удалённых серверов {#remote-servers-configuration} - -### Традиционная ручная конфигурация {#traditional-manual-configuration} - -Традиционно в ClickHouse каждый шард и каждая реплика в кластере должны были указываться вручную в конфигурации: - -```xml - - - - - node1 - 9000 - - - node2 - 9000 - - - - - node3 - 9000 - - - node4 - 9000 - - - - - -``` - -### Использование обнаружения кластера {#using-cluster-discovery} - -При использовании Cluster Discovery вместо явного указания каждого узла вы просто задаёте путь в ZooKeeper. Все узлы, которые зарегистрируются по этому пути в ZooKeeper, будут автоматически обнаружены и добавлены в кластер. - -```xml - - - - /clickhouse/discovery/cluster_name - - - - - - - - - - - - - - - - - -``` - -Если вы хотите задать номер шарда для конкретного узла, включите тег `` в раздел ``: - -для `node1` и `node2`: - -```xml - - /clickhouse/discovery/cluster_name - 1 - -``` - -для узлов `node3` и `node4`: - -```xml - - /clickhouse/discovery/cluster_name - 2 - -``` - -### Режим наблюдателя {#observer-mode} - -Узлы, настроенные в режиме наблюдателя, не будут регистрировать себя как реплики. -Они лишь будут наблюдать за другими активными репликами в кластере и обнаруживать их, не участвуя в работе. -Чтобы включить режим наблюдателя, добавьте тег `` в секцию ``: - -```xml - - /clickhouse/discovery/cluster_name - - -``` - -### Обнаружение кластеров {#discovery-of-clusters} - -Иногда может понадобиться добавлять и удалять не только хосты в кластерах, но и сами кластеры. Для этого можно использовать узел `` с корневым путем, общим для нескольких кластеров: - -```xml - - - - /clickhouse/discovery - - - - -``` - -В этом случае, когда какой-то другой хост зарегистрируется по пути `/clickhouse/discovery/some_new_cluster`, будет добавлен кластер с именем `some_new_cluster`. - -Вы можете использовать обе возможности одновременно: хост может зарегистрироваться в кластере `my_cluster` и при этом обнаруживать любые другие кластеры: - -```xml - - - - /clickhouse/discovery/my_cluster - - - - - /clickhouse/discovery - - - - -``` - -Ограничения: - -* Нельзя одновременно использовать `` и `` в одном поддереве `remote_servers`. -* `` может использоваться только совместно с ``. -* Последняя часть пути из Keeper используется в качестве имени кластера, в то время как при регистрации имя берётся из XML-тега. - -## Сценарии использования и ограничения {#use-cases-and-limitations} - -При добавлении или удалении узлов по указанному пути в ZooKeeper они автоматически обнаруживаются или удаляются из кластера без необходимости изменять конфигурацию или перезапускать серверы. - -Однако изменения затрагивают только конфигурацию кластера, а не данные и не существующие базы данных и таблицы. - -Рассмотрим следующий пример с кластером из 3 узлов: - -```xml - - - - /clickhouse/discovery/default_cluster - - - -``` - -```sql -SELECT * EXCEPT (default_database, errors_count, slowdowns_count, estimated_recovery_time, database_shard_name, database_replica_name) -FROM system.clusters WHERE cluster = 'default'; - -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -```sql -CREATE TABLE event_table ON CLUSTER default (event_time DateTime, value String) -ENGINE = ReplicatedMergeTree('/clickhouse/tables/event_table', '{replica}') -ORDER BY event_time PARTITION BY toYYYYMM(event_time); - -INSERT INTO event_table ... -``` - -Затем мы добавляем в кластер новый узел, запуская его с тем же элементом в разделе `remote_servers` в конфигурационном файле: - -```response -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 4 │ b0df3669b81f │ 172.26.0.6 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -Четвёртый узел участвует в кластере, но таблица `event_table` по-прежнему присутствует только на первых трёх узлах: - -```sql -SELECT hostname(), database, table FROM clusterAllReplicas(default, system.tables) WHERE table = 'event_table' FORMAT PrettyCompactMonoBlock -``` - -┌─hostname()───┬─database─┬─table───────┐ -│ a6a68731c21b │ default │ event_table │ -│ 92d3c04025e8 │ default │ event_table │ -│ 8e62b9cb17a1 │ default │ event_table │ -└──────────────┴──────────┴─────────────┘ - -``` - -Если требуется реплицировать таблицы на всех узлах, можно использовать движок базы данных [Replicated](../engines/database-engines/replicated.md) вместо механизма обнаружения кластера. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/configuration-files.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/configuration-files.md deleted file mode 100644 index 90ca24619f1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/configuration-files.md +++ /dev/null @@ -1,472 +0,0 @@ ---- -description: 'На этой странице объясняется, как сервер ClickHouse может настраиваться с помощью конфигурационных файлов в синтаксисе XML или YAML.' -sidebar_label: 'Конфигурационные файлы' -sidebar_position: 50 -slug: /operations/configuration-files -title: 'Конфигурационные файлы' -doc_type: 'guide' ---- - -:::note -Профили настроек и конфигурационные файлы на основе XML не поддерживаются в ClickHouse Cloud. Поэтому в ClickHouse Cloud вы не найдете файл config.xml. Вместо этого следует использовать SQL-команды для управления настройками через профили настроек. - -Для получения дополнительной информации см. ["Configuring Settings"](/manage/settings) -::: - -Сервер ClickHouse может настраиваться с помощью конфигурационных файлов в синтаксисе XML или YAML. -В большинстве типов установки сервер ClickHouse запускается с `/etc/clickhouse-server/config.xml` в качестве файла конфигурации по умолчанию, но также возможно указать расположение файла конфигурации вручную при запуске сервера, используя параметр командной строки `--config-file` или `-C`. -Дополнительные конфигурационные файлы могут быть помещены в каталог `config.d/` относительно основного файла конфигурации, например в каталог `/etc/clickhouse-server/config.d/`. -Файлы в этом каталоге и основной конфигурационный файл объединяются на этапе предварительной обработки до применения конфигурации на сервере ClickHouse. -Конфигурационные файлы объединяются в алфавитном порядке. -Чтобы упростить обновления и улучшить модульность, рекомендуется оставлять файл `config.xml` по умолчанию без изменений и размещать дополнительную настройку в `config.d/`. -Конфигурация ClickHouse Keeper находится в `/etc/clickhouse-keeper/keeper_config.xml`. -Аналогично, дополнительные конфигурационные файлы для Keeper необходимо помещать в `/etc/clickhouse-keeper/keeper_config.d/`. - -Допускается сочетать конфигурационные файлы XML и YAML, например, у вас может быть основной файл конфигурации `config.xml` и дополнительные файлы конфигурации `config.d/network.xml`, `config.d/timezone.yaml` и `config.d/keeper.yaml`. -Смешивание XML и YAML в одном конфигурационном файле не поддерживается. -Файлы конфигурации XML должны использовать `...` в качестве тега верхнего уровня. -В конфигурационных файлах YAML тег `clickhouse:` является необязательным; если он отсутствует, парсер вставляет его автоматически. - - - -## Объединение конфигураций {#merging} - -Два конфигурационных файла (обычно основной конфигурационный файл и дополнительный конфигурационный файл из `config.d/`) объединяются следующим образом: - -* Если узел (т. е. путь, ведущий к элементу) присутствует в обоих файлах и не имеет атрибутов `replace` или `remove`, он включается в объединённый конфигурационный файл, а дочерние элементы из обоих узлов включаются и рекурсивно объединяются. -* Если один из двух узлов содержит атрибут `replace`, он включается в объединённый конфигурационный файл, но при этом включаются только дочерние элементы из узла с атрибутом `replace`. -* Если один из двух узлов содержит атрибут `remove`, узел не включается в объединённый конфигурационный файл (если он уже существует, он удаляется). - -Например, если заданы два конфигурационных файла: - -```xml title="config.xml" - - - 1 - - - 2 - - - 3 - - -``` - -и - -```xml title="config.d/other_config.xml" - - - 4 - - - 5 - - - 6 - - -``` - -В результате получится объединённый файл конфигурации: - -```xml - - - 1 - 4 - - - 5 - - -``` - -### Подстановка значений из переменных окружения и узлов ZooKeeper {#from_env_zk} - -Чтобы указать, что значение элемента должно быть заменено значением переменной окружения, используйте атрибут `from_env`. - -Например, с переменной окружения `$MAX_QUERY_SIZE = 150000`: - -```xml - - - - - - - -``` - -Итоговая конфигурация будет следующей: - -```xml - - - - 150000 - - - -``` - -То же самое можно сделать с помощью `from_zk` (узла ZooKeeper): - -```xml - - - -``` - - -```shell -# clickhouse-keeper-client {#clickhouse-keeper-client} -/ :) touch /zk_configs -/ :) create /zk_configs/postgresql_port "9005" -/ :) get /zk_configs/postgresql_port -9005 -``` - -В итоге вы получите следующую конфигурацию: - -```xml - - 9005 - -``` - -#### Значения по умолчанию {#default-values} - -Элемент с атрибутом `from_env` или `from_zk` может дополнительно иметь атрибут `replace="1"` (этот атрибут должен указываться перед `from_env`/`from_zk`). -В этом случае элемент может задавать значение по умолчанию. -Элемент принимает значение переменной окружения или узла ZooKeeper, если оно задано, в противном случае используется значение по умолчанию. - -Повторим предыдущий пример, но будем считать, что `MAX_QUERY_SIZE` не задана: - -```xml - - - - 150000 - - - -``` - -В результате конфигурация будет выглядеть следующим образом: - -```xml - - - - 150000 - - - -``` - - -## Подстановка содержимого из файла {#substitution-with-file-content} - -Также можно заменять части конфигурации содержимым файлов. Это можно сделать двумя способами: - -* *Подстановка значений*: Если элемент имеет атрибут `incl`, его значение будет заменено содержимым указанного файла. По умолчанию путь к файлу с подстановками — `/etc/metrika.xml`. Это можно изменить в элементе [`include_from`](../operations/server-configuration-parameters/settings.md#include_from) в конфигурации сервера. Значения подстановок задаются в элементах `/clickhouse/substitution_name` в этом файле. Если подстановка, указанная в `incl`, не существует, об этом делается запись в журнал. Чтобы ClickHouse не записывал отсутствующие подстановки в журнал, укажите атрибут `optional="true"` (например, для настроек [macros](../operations/server-configuration-parameters/settings.md#macros)). -* *Подстановка элементов*: Если необходимо заменить целый элемент подстановкой, используйте `include` в качестве имени элемента. Имя элемента `include` можно комбинировать с атрибутом `from_zk = "/path/to/node"`. В этом случае значение элемента будет заменено содержимым узла ZooKeeper по пути `/path/to/node`. Это также работает, если вы храните целое поддерево XML в узле ZooKeeper — оно будет полностью вставлено в исходный элемент. - -Пример этого показан ниже: - -```xml - - - - - - - - - - -``` - -Если вы хотите объединить подставляемое содержимое с существующей конфигурацией вместо простого добавления в конец, вы можете использовать атрибут `merge="true"`. Например: ``. В этом случае существующая конфигурация будет объединена с подставляемым содержимым, а текущие настройки конфигурации будут заменены значениями из подстановки. - - -## Шифрование и скрытие конфигурации {#encryption} - -Вы можете использовать симметричное шифрование для шифрования элемента конфигурации, например пароля в открытом виде или закрытого ключа. -Для этого сначала настройте [кодек шифрования](../sql-reference/statements/create/table.md#encryption-codecs), затем добавьте атрибут `encrypted_by` со значением — именем кодека шифрования — к элементу, который нужно зашифровать. - -В отличие от атрибутов `from_zk`, `from_env` и `incl` или элемента `include`, никакая подстановка (то есть расшифровка зашифрованного значения) в предварительно обработанном файле не выполняется. -Расшифровка выполняется только во время работы процесса сервера. - -Например: - -```xml - - - - - 00112233445566778899aabbccddeeff - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -Атрибуты [`from_env`](#from_env_zk) и [`from_zk`](#from_env_zk) также могут применяться к `encryption_codecs`: - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -Ключи шифрования и зашифрованные значения могут быть определены в любом конфигурационном файле. - -Пример файла `config.xml`: - -```xml - - - - - - - - - -``` - -Пример файла `users.xml`: - -```xml - - - - - 96280000000D000000000030D4632962295D46C6FA4ABF007CCEC9C1D0E19DA5AF719C1D9A46C446 - default - - - - -``` - -Чтобы зашифровать значение, можно, например, использовать программу `encrypt_decrypt`: - -```bash -./encrypt_decrypt /etc/clickhouse-server/config.xml -e AES_128_GCM_SIV abcd -``` - -```text -961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 -``` - -Даже при использовании зашифрованных элементов конфигурации зашифрованные элементы по‑прежнему видны в предварительно обработанном конфигурационном файле. -Если это создаёт проблему для вашего развертывания ClickHouse, есть два варианта: либо установите права доступа к предварительно обработанному файлу 600, либо используйте атрибут `hide_in_preprocessed`. - -Например: - -```xml - - - - admin - secret - - - -``` - - -## Настройки пользователя {#user-settings} - -Файл `config.xml` может задавать отдельный конфигурационный файл с пользовательскими настройками, профилями и квотами. Относительный путь к этому файлу задаётся в элементе `users_config`. По умолчанию это `users.xml`. Если `users_config` не указан, пользовательские настройки, профили и квоты задаются непосредственно в `config.xml`. - -Пользовательскую конфигурацию можно разбить на отдельные файлы аналогично `config.xml` и `config.d/`. -Имя каталога определяется как значение настройки `users_config` без суффикса `.xml`, к которому добавляется `.d`. -Каталог `users.d` используется по умолчанию, так как `users_config` по умолчанию равен `users.xml`. - -Обратите внимание, что файлы конфигурации сначала [объединяются](#merging) с учётом настроек, а include-директивы обрабатываются после этого. - - - -## Пример XML {#example} - -Например, вы можете использовать отдельный файл конфигурации для каждого пользователя следующим образом: - -```bash -$ cat /etc/clickhouse-server/users.d/alice.xml -``` - -```xml - - - - analytics - - ::/0 - - ... - analytics - - - -``` - - -## Примеры YAML {#example-1} - -Здесь вы можете увидеть конфигурацию по умолчанию в формате YAML: [`config.yaml.example`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/server/config.yaml.example). - -Между форматами YAML и XML есть некоторые отличия с точки зрения конфигурации ClickHouse. -Ниже приведены рекомендации по написанию конфигурации в формате YAML. - -XML‑тег с текстовым значением представляется в виде пары ключ–значение YAML - -```yaml -key: value -``` - -Соответствующий XML-код: - -```xml -значение -``` - -Вложенный XML-узел представляется в виде YAML-отображения: - -```yaml -map_key: - key1: val1 - key2: val2 - key3: val3 -``` - -Соответствующий XML-код: - -```xml - - val1 - val2 - val3 - -``` - -Чтобы создать один и тот же XML‑тег несколько раз, используйте последовательность YAML: - -```yaml -seq_key: - - val1 - - val2 - - key1: val3 - - map: - key2: val4 - key3: val5 -``` - -Соответствующий XML-файл: - -```xml -val1 -val2 - - val3 - - - - val4 - val5 - - -``` - -Чтобы задать атрибут XML, можно использовать ключ атрибута с префиксом `@`. Учтите, что по стандарту YAML символ `@` является зарезервированным, поэтому его необходимо заключать в двойные кавычки: - -```yaml -map: - "@attr1": value1 - "@attr2": value2 - key: 123 -``` - -Соответствующий XML: - -```xml - - 123 - -``` - -Можно также использовать атрибуты в последовательности YAML: - -```yaml -seq: - - "@attr1": value1 - - "@attr2": value2 - - 123 - - abc -``` - -Соответствующий XML: - -```xml -123 -abc -``` - -Упомянутый выше синтаксис не позволяет представить XML-текстовые узлы с XML-атрибутами в виде YAML. Этот особый случай можно реализовать, используя ключ атрибута -`#text`: - -```yaml -map_key: - "@attr1": value1 - "#text": value2 -``` - -Соответствующий XML: - -```xml -value2 -``` - - -## Детали реализации {#implementation-details} - -Для каждого файла конфигурации сервер при запуске также генерирует файлы `file-preprocessed.xml`. Эти файлы содержат все выполненные подстановки и переопределения и предназначены только для ознакомления. Если в конфигурационных файлах использовались подстановки через ZooKeeper, но ZooKeeper недоступен при старте сервера, сервер загружает конфигурацию из предварительно обработанного файла. - -Сервер отслеживает изменения в конфигурационных файлах, а также в файлах и узлах ZooKeeper, которые использовались при выполнении подстановок и переопределений, и «на лету» перезагружает настройки пользователей и кластеров. Это означает, что вы можете изменять кластер, пользователей и их настройки без перезапуска сервера. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md deleted file mode 100644 index 32102b313c0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -description: 'Документация по HTTP' -slug: /operations/external-authenticators/http -title: 'HTTP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -HTTP-сервер можно использовать для аутентификации пользователей ClickHouse. HTTP-аутентификация может применяться только в качестве внешнего средства аутентификации для уже существующих пользователей, заданных в `users.xml` или в локальных путях управления доступом. В настоящее время поддерживается схема аутентификации [Basic](https://datatracker.ietf.org/doc/html/rfc7617), использующая метод GET. - -## Определение сервера HTTP-аутентификации {#http-auth-server-definition} - -Чтобы определить сервер HTTP-аутентификации, необходимо добавить раздел `http_authentication_servers` в файл `config.xml`. - -**Пример** - -```xml - - - - - http://localhost:8000/auth - 1000 - 1000 - 1000 - 3 - 50 - 1000 - - Custom-Auth-Header-1 - Custom-Auth-Header-2 - - - - - - -``` - -Обратите внимание, что вы можете определить несколько HTTP-серверов в секции `http_authentication_servers`, используя разные имена. - -**Параметры** - -* `uri` - URI для выполнения запроса аутентификации. - -Тайм-ауты в миллисекундах на сокете, используемом для взаимодействия с сервером: - -* `connection_timeout_ms` - По умолчанию: 1000 мс. -* `receive_timeout_ms` - По умолчанию: 1000 мс. -* `send_timeout_ms` - По умолчанию: 1000 мс. - -Параметры повторных попыток: - -* `max_tries` - Максимальное количество попыток выполнить запрос аутентификации. По умолчанию: 3. -* `retry_initial_backoff_ms` - Начальный интервал ожидания перед повторной попыткой. По умолчанию: 50 мс. -* `retry_max_backoff_ms` - Максимальный интервал ожидания. По умолчанию: 1000 мс. - -Проброс заголовков: - -Эта часть конфигурации определяет, какие заголовки будут проброшены из заголовков клиентского запроса во внешний HTTP-аутентификатор. Обратите внимание, что заголовки будут сопоставляться с указанными в конфигурации без учета регистра, но пробрасываться как есть, т.е. без изменений. - -### Включение HTTP-аутентификации в `users.xml` {#enabling-http-auth-in-users-xml} - -Чтобы включить HTTP-аутентификацию для пользователя, укажите секцию `http_authentication` вместо `password` или аналогичных секций в определении пользователя. - -Параметры: - -* `server` - Имя HTTP-сервера аутентификации, настроенного в основном файле `config.xml`, как описано выше. -* `scheme` - Схема HTTP-аутентификации. В настоящее время поддерживается только `Basic`. По умолчанию: Basic. - -Пример (добавляется в `users.xml`): - -```xml - - - - - - basic_server - basic - - - -``` - -:::note -Обратите внимание, что HTTP-аутентификация не может использоваться одновременно с любым другим механизмом аутентификации. Наличие любых других разделов, таких как `password` вместе с `http_authentication`, приведёт к остановке ClickHouse. -::: - -### Включение HTTP-аутентификации с использованием SQL {#enabling-http-auth-using-sql} - -Когда в ClickHouse включён [SQL-управляемый контроль доступа и управление аккаунтами](/operations/access-rights#access-control-usage), пользователи, идентифицируемые с помощью HTTP-аутентификации, также могут быть созданы с использованием SQL-операторов. - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' SCHEME 'Basic' -``` - -...или используется `Basic` по умолчанию при отсутствии явного указания схемы - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' -``` - -### Передача настроек сессии {#passing-session-settings} - -Если тело ответа от HTTP-сервера аутентификации имеет формат JSON и содержит подобъект `settings`, ClickHouse попытается разобрать его пары ключ–значение как строковые и установить их в качестве настроек текущего сеанса аутентифицированного пользователя. Если разбор не удался, тело ответа сервера будет проигнорировано. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md deleted file mode 100644 index 252390501ca..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Обзор методов внешней аутентификации, поддерживаемых в ClickHouse' -pagination_next: operations/external-authenticators/kerberos -sidebar_label: 'Внешние аутентификаторы пользователей и каталоги' -sidebar_position: 48 -slug: /operations/external-authenticators/ -title: 'Внешние аутентификаторы пользователей и каталоги' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -ClickHouse поддерживает аутентификацию и управление пользователями с использованием внешних сервисов. - -Поддерживаются следующие внешние аутентификаторы и каталоги пользователей: - -* [LDAP](/operations/external-authenticators/ldap#ldap-external-authenticator) [аутентификатор](./ldap.md#ldap-external-authenticator) и [каталог пользователей](./ldap.md#ldap-external-user-directory) -* Kerberos [аутентификатор](/operations/external-authenticators/kerberos#kerberos-as-an-external-authenticator-for-existing-users) -* [Аутентификация по SSL X.509](/operations/external-authenticators/ssl-x509) -* HTTP [аутентификатор](./http.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md deleted file mode 100644 index d174a835add..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: 'Уже созданные и корректно настроенные пользователи ClickHouse могут проходить аутентификацию по протоколу Kerberos.' -slug: /operations/external-authenticators/kerberos -title: 'Kerberos' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# Kerberos {#kerberos} - - - -Существующие и корректно настроенные пользователи ClickHouse могут аутентифицироваться через протокол Kerberos. - -В настоящее время Kerberos может использоваться только как внешний механизм аутентификации для существующих пользователей, которые определены в `users.xml` или в локальных путях управления доступом. Эти пользователи могут использовать только HTTP-запросы и должны иметь возможность аутентифицироваться с использованием механизма GSS-SPNEGO. - -Для этого подхода Kerberos должен быть настроен в системе и включен в конфигурации ClickHouse. - -## Включение Kerberos в ClickHouse {#enabling-kerberos-in-clickhouse} - -Чтобы включить Kerberos, необходимо добавить секцию `kerberos` в `config.xml`. Эта секция может содержать дополнительные параметры. - -#### Параметры {#parameters} - -* `principal` - каноническое имя сервисного principal, которое будет использоваться при приёме контекстов безопасности. - * Этот параметр является необязательным, если он опущен, будет использован principal по умолчанию. - -* `realm` - realm, который будет использоваться для ограничения аутентификации только запросами, у которых realm инициатора совпадает с ним. - * Этот параметр является необязательным, если он опущен, дополнительная фильтрация по realm применяться не будет. - -* `keytab` - путь к сервисному keytab-файлу. - * Этот параметр является необязательным, если он опущен, путь к сервисному keytab-файлу должен быть задан в переменной окружения `KRB5_KTNAME`. - -Пример (добавляется в `config.xml`): - -```xml - - - - -``` - -С указанием субъекта (principal): - -```xml - - - - HTTP/clickhouse.example.com@EXAMPLE.COM - - -``` - -С фильтром по realm: - -```xml - - - - EXAMPLE.COM - - -``` - -:::note -Можно задать только один раздел `kerberos`. Наличие нескольких разделов `kerberos` приведёт к отключению аутентификации Kerberos в ClickHouse. -::: - -:::note -Разделы `principal` и `realm` не могут указываться одновременно. Наличие одновременно и `principal`, и `realm` приведёт к отключению аутентификации Kerberos в ClickHouse. -::: - -## Kerberos в качестве внешнего аутентификатора для существующих пользователей {#kerberos-as-an-external-authenticator-for-existing-users} - -Kerberos может использоваться как метод проверки подлинности локально определённых пользователей (пользователей, определённых в `users.xml` или в локальных путях управления доступом). В настоящий момент **только** запросы через HTTP-интерфейс могут проходить аутентификацию по Kerberos (через механизм GSS-SPNEGO). - -Формат имени принципала Kerberos обычно имеет следующий вид: - -* *primary/instance@REALM* - -Часть */instance* может встречаться ноль или более раз. **Ожидается, что *primary*-часть канонического имени принципала инициатора будет совпадать с именем пользователя для Kerberos-аутентификации, чтобы аутентификация прошла успешно**. - -### Включение Kerberos в `users.xml` {#enabling-kerberos-in-users-xml} - -Чтобы включить аутентификацию Kerberos для пользователя, укажите секцию `kerberos` вместо `password` или аналогичных секций в определении пользователя. - -Параметры: - -* `realm` — область (realm), которая будет использоваться для ограничения аутентификации только запросами, у которых область инициатора совпадает с ней. - * Этот параметр является необязательным: если он опущен, дополнительная фильтрация по области применяться не будет. - -Пример (помещается в `users.xml`): - -```xml - - - - - - - - EXAMPLE.COM - - - - -``` - -:::note -Обратите внимание, что аутентификация Kerberos не может использоваться одновременно с другими механизмами аутентификации. Наличие любых других разделов конфигурации, таких как `password` вместе с `kerberos`, приведёт к принудительному завершению работы ClickHouse. -::: - -:::info Reminder -Обратите внимание, что теперь, когда пользователь `my_user` использует `kerberos`, Kerberos должен быть включён в основном файле `config.xml`, как описано ранее. -::: - -### Включение Kerberos через SQL {#enabling-kerberos-using-sql} - -Когда в ClickHouse включена [SQL-управляемая система контроля доступа и управления учётными записями](/operations/access-rights#access-control-usage), пользователей, аутентифицируемых через Kerberos, также можно создавать с помощью SQL-выражений. - -```sql -СОЗДАТЬ ПОЛЬЗОВАТЕЛЯ my_user ИДЕНТИФИЦИРОВАН С kerberos РЕАЛМ 'EXAMPLE.COM' -``` - -...или без фильтрации по реалму: - -```sql -CREATE USER my_user AUTHENTIFICIROVAN S POMOSHCH'YU kerberos -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md deleted file mode 100644 index 86a9a5c55c8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: 'Руководство по настройке LDAP-аутентификации для ClickHouse' -slug: /operations/external-authenticators/ldap -title: 'LDAP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -Сервер LDAP можно использовать для аутентификации пользователей ClickHouse. Для этого есть два подхода: - -* Использовать LDAP в качестве внешнего аутентификатора для существующих пользователей, которые определены в `users.xml` или в локальных путях управления доступом. -* Использовать LDAP в качестве внешнего пользовательского каталога и разрешить аутентификацию локально не определённых пользователей, если они существуют на сервере LDAP. - -В обоих этих случаях в конфигурации ClickHouse должен быть определён сервер LDAP с внутренним именем, чтобы на него можно было ссылаться из других частей конфигурации. - -## Определение сервера LDAP {#ldap-server-definition} - -Чтобы задать сервер LDAP, необходимо добавить раздел `ldap_servers` в файл `config.xml`. - -**Пример** - -```xml - - - - - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - - - - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - - - -``` - -Обратите внимание, что в секции `ldap_servers` можно указать несколько LDAP-серверов с разными именами. - -**Параметры** - -* `host` — имя хоста или IP‑адрес LDAP‑сервера, этот параметр обязателен и не может быть пустым. -* `port` — порт LDAP‑сервера, по умолчанию `636`, если `enable_tls` имеет значение `true`, иначе `389`. -* `bind_dn` — шаблон, используемый для формирования DN для привязки (bind). - * Итоговый DN будет сформирован путем замены всех подстрок `{user_name}` в шаблоне на фактическое имя пользователя при каждой попытке аутентификации. -* `user_dn_detection` — раздел с параметрами поиска в LDAP для определения фактического DN пользователя, к которому выполнена привязка. - * В основном используется в фильтрах поиска для последующего сопоставления ролей, когда сервером является Active Directory. Полученный DN пользователя будет использоваться при замене подстрок `{user_dn}` везде, где это разрешено. По умолчанию DN пользователя устанавливается равным bind DN, но после выполнения поиска он будет обновлен фактическим обнаруженным значением DN пользователя. - * `base_dn` — шаблон, используемый для формирования базового DN для поиска в LDAP. - * Итоговый DN будет сформирован путем замены всех подстрок `{user_name}` и `{bind_dn}` в шаблоне на фактическое имя пользователя и bind DN во время поиска в LDAP. - * `scope` — область поиска в LDAP. - * Допустимые значения: `base`, `one_level`, `children`, `subtree` (значение по умолчанию). - * `search_filter` — шаблон, используемый для формирования фильтра поиска в LDAP. - * Итоговый фильтр будет сформирован путем замены всех подстрок `{user_name}`, `{bind_dn}` и `{base_dn}` в шаблоне на фактическое имя пользователя, bind DN и base DN во время поиска в LDAP. - * Обратите внимание, что специальные символы должны быть корректно экранированы в XML. -* `verification_cooldown` — период времени в секундах после успешной попытки привязки, в течение которого пользователь считается успешно аутентифицированным для всех последующих запросов без обращения к LDAP‑серверу. - * Укажите `0` (значение по умолчанию), чтобы отключить кеширование и принудительно обращаться к LDAP‑серверу для каждого запроса аутентификации. -* `enable_tls` — флаг, включающий использование защищенного соединения с LDAP‑сервером. - * Укажите `no` для незашифрованного протокола `ldap://` (не рекомендуется). - * Укажите `yes` для протокола LDAP поверх SSL/TLS `ldaps://` (рекомендуется, используется по умолчанию). - * Укажите `starttls` для устаревшего протокола StartTLS (незашифрованный протокол `ldap://`, повышаемый до TLS). -* `tls_minimum_protocol_version` — минимальная версия протокола SSL/TLS. - * Допустимые значения: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2` (значение по умолчанию). -* `tls_require_cert` — способ проверки сертификата удаленного узла (peer) SSL/TLS. - * Допустимые значения: `never`, `allow`, `try`, `demand` (значение по умолчанию). -* `tls_cert_file` — путь к файлу сертификата. -* `tls_key_file` — путь к файлу ключа сертификата. -* `tls_ca_cert_file` — путь к файлу сертификата CA. -* `tls_ca_cert_dir` — путь к каталогу, содержащему сертификаты CA. -* `tls_cipher_suite` — разрешенный набор шифров (в нотации OpenSSL). - -## Внешний аутентификатор LDAP {#ldap-external-authenticator} - -Удалённый сервер LDAP может использоваться в качестве метода проверки паролей для локально определённых пользователей (пользователей, заданных в `users.xml` или в локальных путях управления доступом). Для этого укажите ранее определённое имя сервера LDAP вместо разделов `password` или аналогичных в определении пользователя. - -При каждой попытке входа в систему ClickHouse пытается «привязаться» (bind) к DN, указанному параметром `bind_dn` в [определении сервера LDAP](#ldap-server-definition), используя предоставленные учётные данные, и в случае успеха пользователь считается аутентифицированным. Это часто называют методом «simple bind». - -**Пример** - -```xml - - - - - - - - my_ldap_server - - - - -``` - -Обратите внимание, что пользователь `my_user` относится к `my_ldap_server`. Этот LDAP-сервер должен быть настроен в основном файле `config.xml`, как описано ранее. - -Когда включено основанное на SQL [управление доступом и учетными записями](/operations/access-rights#access-control-usage), пользователи, аутентифицируемые LDAP-серверами, также могут быть созданы с помощью оператора [CREATE USER](/sql-reference/statements/create/user). - -Запрос: - -```sql -CREATE USER my_user IDENTIFIED WITH ldap SERVER 'my_ldap_server'; -``` - -## Внешний пользовательский каталог LDAP {#ldap-external-user-directory} - -Помимо локально заданных пользователей, в качестве источника описаний пользователей может использоваться удалённый сервер LDAP. Для этого укажите ранее определённое имя сервера LDAP (см. [Определение сервера LDAP](#ldap-server-definition)) в секции `ldap` внутри секции `users_directories` файла `config.xml`. - -При каждой попытке входа ClickHouse сначала пытается найти описание пользователя локально и аутентифицировать его как обычно. Если пользователь не определён, ClickHouse предполагает, что его описание существует во внешнем каталоге LDAP и пытается выполнить операцию `bind` к указанному DN на сервере LDAP, используя предоставленные учётные данные. В случае успеха пользователь считается существующим и аутентифицированным. Пользователю будут назначены роли из списка, указанного в секции `roles`. Дополнительно может быть выполнен LDAP‑поиск (`search`), результаты которого могут быть преобразованы и интерпретированы как имена ролей и затем назначены пользователю, если также настроена секция `role_mapping`. Всё это подразумевает, что SQL‑управляемая [Система управления доступом и учётными записями](/operations/access-rights#access-control-usage) включена и роли созданы с помощью оператора [CREATE ROLE](/sql-reference/statements/create/role). - -**Пример** - -Размещается в файле `config.xml`. - -```xml - - - - - - my_ldap_server - - - - - - ou=groups,dc=example,dc=com - subtree - (&(objectClass=groupOfNames)(member={bind_dn})) - cn - clickhouse_ - - - - - - my_ad_server - - CN=Users,DC=example,DC=com - CN - subtree - (&(objectClass=group)(member={user_dn})) - clickhouse_ - - - - -``` - -Обратите внимание, что `my_ldap_server`, указанный в разделе `ldap` внутри секции `user_directories`, должен быть ранее определённым LDAP-сервером, описанным в `config.xml` (см. [Определение LDAP-сервера](#ldap-server-definition)). - -**Параметры** - -* `server` — Одно из имён LDAP-серверов, определённых в конфигурационном разделе `ldap_servers` выше. Этот параметр обязателен и не может быть пустым. -* `roles` — Раздел со списком локально определённых ролей, которые будут назначены каждому пользователю, полученному из LDAP-сервера. - * Если роли здесь не указаны или не назначены в процессе сопоставления ролей (см. ниже), пользователь не сможет выполнять никакие действия после аутентификации. -* `role_mapping` — Раздел с параметрами поиска LDAP и правилами сопоставления. - * Когда пользователь аутентифицируется, выполняется поиск в LDAP с использованием `search_filter` и имени пользователя, выполнившего вход. Для каждой записи, найденной в ходе этого поиска, извлекается значение указанного атрибута. Для каждого значения атрибута, которое имеет указанный префикс, префикс удаляется, и оставшаяся часть значения становится именем локальной роли, определённой в ClickHouse, которую предполагается создать заранее с помощью оператора [CREATE ROLE](/sql-reference/statements/create/role). - * В одном и том же разделе `ldap` может быть определено несколько разделов `role_mapping`. Все они будут применяться. - * `base_dn` — Шаблон, используемый для построения базового DN для поиска в LDAP. - * Итоговый DN будет построен путём замены всех подстрок `{user_name}`, `{bind_dn}` и `{user_dn}` в шаблоне фактическим именем пользователя и значениями bind DN и user DN при каждом поиске в LDAP. - * `scope` — Область поиска в LDAP. - * Допустимые значения: `base`, `one_level`, `children`, `subtree` (значение по умолчанию). - * `search_filter` — Шаблон, используемый для построения фильтра поиска для запроса к LDAP. - * Итоговый фильтр будет построен путём замены всех подстрок `{user_name}`, `{bind_dn}`, `{user_dn}` и `{base_dn}` в шаблоне фактическим именем пользователя и значениями bind DN, user DN и base DN при каждом поиске в LDAP. - * Обратите внимание, что специальные символы должны быть корректно экранированы в XML. - * `attribute` — Имя атрибута, значения которого будут возвращены поиском LDAP. По умолчанию — `cn`. - * `prefix` — Префикс, который ожидается в начале каждой строки в исходном списке строк, возвращённом поиском LDAP. Префикс будет удалён из исходных строк, а полученные строки будут интерпретированы как имена локальных ролей. По умолчанию — пустой. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md deleted file mode 100644 index 3e9f7f67c58..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Документация по SSL X.509' -slug: /operations/external-authenticators/ssl-x509 -title: 'Аутентификация по SSL-сертификату X.509' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -Параметр [SSL 'strict'](../server-configuration-parameters/settings.md#openssl) включает обязательную проверку сертификатов для входящих подключений. В этом случае могут быть установлены только соединения с доверенными сертификатами. Подключения с недоверенными сертификатами будут отклонены. Таким образом, проверка сертификатов позволяет однозначно аутентифицировать входящее подключение. Для идентификации подключенного пользователя используется поле сертификата `Common Name` или расширение `subjectAltName`. Расширение `subjectAltName` поддерживает использование одного подстановочного символа '*' в конфигурации сервера. Это позволяет связать несколько сертификатов с одним и тем же пользователем. Кроме того, перевыпуск и отзыв сертификатов не влияет на конфигурацию ClickHouse. - -Чтобы включить аутентификацию по SSL-сертификату, в файле настроек `users.xml` должен быть указан список значений полей `Common Name` или `Subject Alt Name` для каждого пользователя ClickHouse: - -**Пример** - -```xml - - - - - - host.domain.com:example_user - host.domain.com:example_user_dev - - - - - - - DNS:host.domain.com - - - - - - - - URI:spiffe://foo.com/*/bar - - - - -``` - -Чтобы SSL-цепочка доверия ([`chain of trust`](https://en.wikipedia.org/wiki/Chain_of_trust)) работала корректно, также важно убедиться, что параметр [`caConfig`](../server-configuration-parameters/settings.md#openssl) настроен правильно. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/monitoring.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/monitoring.md deleted file mode 100644 index 023609794ce..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/monitoring.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Вы можете отслеживать использование аппаратных ресурсов и метрики сервера ClickHouse.' -keywords: ['мониторинг', 'наблюдаемость', 'расширенный дашборд', 'дашборд', 'дашборд наблюдаемости'] -sidebar_label: 'Мониторинг' -sidebar_position: 45 -slug: /operations/monitoring -title: 'Мониторинг' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; - - -# Мониторинг {#monitoring} - -:::note -Данные мониторинга, описанные в данном руководстве, доступны в ClickHouse Cloud. Помимо отображения во встроенной панели, описанной ниже, как базовые, так и расширенные метрики производительности можно просматривать непосредственно в основной консоли сервиса. -::: - -Вы можете отслеживать: - -- Использование аппаратных ресурсов. -- Метрики сервера ClickHouse. - - - -## Встроенная расширенная панель наблюдаемости {#built-in-advanced-observability-dashboard} - -Screenshot 2023-11-12 at 6 08 58 PM - -ClickHouse включает встроенную расширенную панель наблюдаемости, доступную по адресу `$HOST:$PORT/dashboard` (требуются имя пользователя и пароль). На ней отображаются следующие метрики: -- Число запросов в секунду -- Использование CPU (ядра) -- Количество выполняющихся запросов -- Количество выполняющихся слияний -- Выборка байт в секунду -- Ожидание I/O -- Ожидание CPU -- Использование CPU ОС (userspace) -- Использование CPU ОС (kernel) -- Объём чтения с диска -- Объём чтения из файловой системы -- Память (отслеживаемая) -- Число вставленных строк в секунду -- Общее количество частей MergeTree -- Максимальное количество частей на раздел - - - -## Использование ресурсов {#resource-utilization} - -ClickHouse также самостоятельно отслеживает состояние аппаратных ресурсов, таких как: - -- Нагрузка и температура процессоров. -- Использование системы хранения, оперативной памяти и сети. - -Эти данные собираются в таблице `system.asynchronous_metric_log`. - - - -## Метрики сервера ClickHouse {#clickhouse-server-metrics} - -Сервер ClickHouse имеет встроенные средства для мониторинга собственного состояния. - -Для отслеживания событий сервера используйте журналы сервера. См. раздел [logger](../operations/server-configuration-parameters/settings.md#logger) в файле конфигурации. - -ClickHouse собирает: - -- Различные метрики использования сервером вычислительных ресурсов. -- Общую статистику по обработке запросов. - -Вы можете найти метрики в таблицах [system.metrics](/operations/system-tables/metrics), [system.events](/operations/system-tables/events) и [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics). - -Вы можете настроить ClickHouse на экспорт метрик в [Graphite](https://github.com/graphite-project). См. раздел [Graphite](../operations/server-configuration-parameters/settings.md#graphite) в файле конфигурации сервера ClickHouse. Перед настройкой экспорта метрик необходимо развернуть Graphite, следуя их официальному [руководству](https://graphite.readthedocs.io/en/latest/install.html). - -Вы можете настроить ClickHouse на экспорт метрик в [Prometheus](https://prometheus.io). См. раздел [Prometheus](../operations/server-configuration-parameters/settings.md#prometheus) в файле конфигурации сервера ClickHouse. Перед настройкой экспорта метрик необходимо развернуть Prometheus, следуя их официальному [руководству](https://prometheus.io/docs/prometheus/latest/installation/). - -Кроме того, вы можете мониторить доступность сервера через HTTP API. Отправьте запрос `HTTP GET` к `/ping`. Если сервер доступен, он отвечает `200 OK`. - -Для мониторинга серверов в конфигурации кластера необходимо задать параметр [max_replica_delay_for_distributed_queries](../operations/settings/settings.md#max_replica_delay_for_distributed_queries) и использовать HTTP-ресурс `/replicas_status`. Запрос к `/replicas_status` возвращает `200 OK`, если реплика доступна и не отстаёт от других реплик. Если реплика отстаёт, возвращается `503 HTTP_SERVICE_UNAVAILABLE` с информацией о величине отставания. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/named-collections.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/named-collections.md deleted file mode 100644 index 93df977466f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/named-collections.md +++ /dev/null @@ -1,656 +0,0 @@ ---- -description: 'Документация по именованным коллекциям' -sidebar_label: 'Именованные коллекции' -sidebar_position: 69 -slug: /operations/named-collections -title: 'Именованные коллекции' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -Именованные коллекции предоставляют способ хранения наборов пар ключ-значение, -используемых для настройки интеграций с внешними источниками. Вы можете использовать именованные коллекции -со словарями, таблицами, табличными функциями и объектным хранилищем. - -Именованные коллекции можно настраивать с помощью DDL или в конфигурационных файлах; они применяются -при запуске ClickHouse. Они упрощают создание объектов и скрытие учетных данных -от пользователей без административного доступа. - -Ключи в именованной коллекции должны соответствовать именам параметров соответствующей -функции, движка таблиц, базы данных и т. д. В примерах ниже для каждого типа -приведена ссылка на список параметров. - -Параметры, заданные в именованной коллекции, могут быть переопределены в SQL — это показано в примерах -ниже. Эту возможность можно ограничить с помощью ключевых слов `[NOT] OVERRIDABLE` и XML-атрибутов -и/или параметра конфигурации `allow_named_collection_override_by_default`. - -:::warning -Если переопределение разрешено, пользователи без административного доступа могут -получить возможность вычислить учетные данные, которые вы пытаетесь скрыть. -Если вы используете именованные коллекции с этой целью, следует отключить -`allow_named_collection_override_by_default` (по умолчанию он включен). -::: - -## Хранение именованных коллекций в системной базе данных {#storing-named-collections-in-the-system-database} - -### Пример DDL {#ddl-example} - -```sql -CREATE NAMED COLLECTION name AS -key_1 = 'value' OVERRIDABLE, -key_2 = 'value2' NOT OVERRIDABLE, -url = 'https://connection.url/' -``` - -В приведённом выше примере: - -* `key_1` всегда может быть переопределён. -* `key_2` никогда не может быть переопределён. -* Возможность переопределения `url` зависит от значения `allow_named_collection_override_by_default`. - -### Права на создание именованных коллекций с помощью DDL {#permissions-to-create-named-collections-with-ddl} - -Чтобы управлять именованными коллекциями с помощью DDL, пользователь должен иметь привилегию `named_collection_control`. Её можно назначить, добавив файл в `/etc/clickhouse-server/users.d/`. В примере пользователю `default` назначаются привилегии `access_management` и `named_collection_control`: - -```xml title='/etc/clickhouse-server/users.d/user_default.xml' - - - - 65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5 - 1 - - 1 - - - - -``` - -:::tip -В приведённом выше примере значение `password_sha256_hex` является шестнадцатеричным представлением SHA256-хеша пароля. В этой конфигурации для пользователя `default` указан атрибут `replace=true`, так как в конфигурации по умолчанию этому пользователю задан пароль в открытом виде `password`, и для одного пользователя нельзя одновременно задать пароль в открытом виде и пароль в формате SHA256 hex. -::: - -### Хранилище для именованных коллекций {#storage-for-named-collections} - -Именованные коллекции могут храниться либо на локальном диске, либо в ZooKeeper/Keeper. По умолчанию используется локальное хранилище. -Их также можно хранить с использованием шифрования с теми же алгоритмами, что применяются для [шифрования диска](storing-data#encrypted-virtual-file-system), -где по умолчанию используется `aes_128_ctr`. - -Чтобы настроить хранилище именованных коллекций, нужно задать `type`. Это может быть либо `local`, либо `keeper`/`zookeeper`. Для зашифрованного хранилища -можно использовать `local_encrypted` или `keeper_encrypted`/`zookeeper_encrypted`. - -Чтобы использовать ZooKeeper/Keeper, также необходимо задать `path` (путь в ZooKeeper/Keeper, где будут храниться именованные коллекции) в -секции `named_collections_storage` в конфигурационном файле. В следующем примере используется шифрование и ZooKeeper/Keeper: - -```xml - - - zookeeper_encrypted - bebec0cabebec0cabebec0cabebec0ca - aes_128_ctr - /named_collections_path/ - 1000 - - -``` - -Необязательный конфигурационный параметр `update_timeout_ms` по умолчанию равен `5000`. - -## Хранение именованных коллекций в конфигурационных файлах {#storing-named-collections-in-configuration-files} - -### Пример на XML {#xml-example} - -```xml title='/etc/clickhouse-server/config.d/named_collections.xml' - - - - value - value_2 - https://connection.url/ - - - -``` - -В приведённом выше примере: - -* `key_1` всегда может быть переопределён. -* `key_2` никогда не может быть переопределён. -* `url` может быть как переопределён, так и нет, в зависимости от значения `allow_named_collection_override_by_default`. - -## Изменение именованных коллекций {#modifying-named-collections} - -Именованные коллекции, созданные с помощью DDL-запросов, можно изменять или удалять с помощью DDL. Именованными коллекциями, созданными из XML-файлов, можно управлять, редактируя или удаляя соответствующие XML-файлы. - -### Изменение именованной коллекции, созданной через DDL {#alter-a-ddl-named-collection} - -Измените или добавьте ключи `key1` и `key3` коллекции `collection2` -(это не изменит значение флага `overridable` для этих ключей): - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, key3='value3' -``` - -Измените или добавьте ключ `key1` и разрешите всегда переопределять его значение: - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4 OVERRIDABLE -``` - -Удалите ключ `key2` из `collection2`: - -```sql -ALTER NAMED COLLECTION collection2 DELETE key2 -``` - -Измените или добавьте в коллекцию `collection2` ключ `key1` и удалите ключ `key3`: - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, DELETE key3 -``` - -Чтобы вернуть ключ к использованию значений по умолчанию флага `overridable`, необходимо удалить этот ключ и добавить его заново. - -```sql -ALTER NAMED COLLECTION collection2 DELETE key1; -ALTER NAMED COLLECTION collection2 SET key1=4; -``` - -### Удалите именованную коллекцию DDL `collection2`: {#drop-the-ddl-named-collection-collection2} - -```sql -DROP NAMED COLLECTION collection2 -``` - -## Именованные коллекции для доступа к S3 {#named-collections-for-accessing-s3} - -Описание параметров см. в разделе [табличной функции s3](../sql-reference/table-functions/s3.md). - -### Пример DDL {#ddl-example-1} - -```sql -CREATE NAMED COLLECTION s3_mydata AS -access_key_id = 'AKIAIOSFODNN7EXAMPLE', -secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', -format = 'CSV', -url = 'https://s3.us-east-1.amazonaws.com/yourbucket/mydata/' -``` - -### Пример XML {#xml-example-1} - -```xml - - - - AKIAIOSFODNN7EXAMPLE - wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY - CSV - https://s3.us-east-1.amazonaws.com/yourbucket/mydata/ - - - -``` - -### Примеры функции s3() и именованной коллекции таблицы S3 {#s3-function-and-s3-table-named-collection-examples} - -Оба следующих примера используют одну и ту же именованную коллекцию `s3_mydata`: - -#### Функция s3() {#s3-function} - -```sql -INSERT INTO FUNCTION s3(s3_mydata, filename = 'test_file.tsv.gz', - format = 'TSV', structure = 'number UInt64', compression_method = 'gzip') -SELECT * FROM numbers(10000); -``` - -:::tip -Первый аргумент функции `s3()` — это имя коллекции `s3_mydata`. Без именованных коллекций идентификатор ключа доступа, секретный ключ, формат и URL пришлось бы передавать при каждом вызове функции `s3()`. -::: - -#### Таблица S3 {#s3-table} - -```sql -CREATE TABLE s3_engine_table (number Int64) -ENGINE=S3(s3_mydata, url='https://s3.us-east-1.amazonaws.com/yourbucket/mydata/test_file.tsv.gz', format = 'TSV') -SETTINGS input_format_with_names_use_header = 0; - -SELECT * FROM s3_engine_table LIMIT 3; -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -└────────┘ -``` - -## Именованные коллекции для доступа к базе данных MySQL {#named-collections-for-accessing-mysql-database} - -См. описание параметров в разделе [mysql](../sql-reference/table-functions/mysql.md). - -### Пример DDL {#ddl-example-2} - -```sql -CREATE NAMED COLLECTION mymysql AS -user = 'myuser', -password = 'mypass', -host = '127.0.0.1', -port = 3306, -database = 'test', -connection_pool_size = 8, -replace_query = 1 -``` - -### Пример XML {#xml-example-2} - -```xml - - - - myuser - mypass - 127.0.0.1 - 3306 - test - 8 - 1 - - - -``` - -### Примеры для функции mysql(), таблицы MySQL, базы данных MySQL и именованной коллекции Dictionary {#mysql-function-mysql-table-mysql-database-and-dictionary-named-collection-examples} - -Следующие четыре примера используют одну и ту же именованную коллекцию `mymysql`: - -#### Функция mysql() {#mysql-function} - -```sql -SELECT count() FROM mysql(mymysql, table = 'test'); - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -Именованная коллекция не задаёт параметр `table`, поэтому он передаётся в вызове функции как `table = 'test'`. -::: - -#### Таблица MySQL {#mysql-table} - -```sql -CREATE TABLE mytable(A Int64) ENGINE = MySQL(mymysql, table = 'test', connection_pool_size=3, replace_query=0); -SELECT count() FROM mytable; - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -Оператор DDL переопределяет настройку connection_pool_size, заданную в named collection. -::: - -#### База данных MySQL {#mysql-database} - -```sql -CREATE DATABASE mydatabase ENGINE = MySQL(mymysql); - -SHOW TABLES FROM mydatabase; - -┌─name───┐ -│ source │ -│ test │ -└────────┘ -``` - -#### Справочник MySQL {#mysql-dictionary} - -```sql -CREATE DICTIONARY dict (A Int64, B String) -PRIMARY KEY A -SOURCE(MYSQL(NAME mymysql TABLE 'source')) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'B', 2); - -┌─dictGet('dict', 'B', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## Именованные коллекции для доступа к базе данных PostgreSQL {#named-collections-for-accessing-postgresql-database} - -Описание параметров см. в разделе [postgresql](../sql-reference/table-functions/postgresql.md). Дополнительно доступны следующие синонимы: - -* `username` для `user` -* `db` для `database`. - -Параметр `addresses_expr` используется в коллекции вместо `host:port`. Параметр необязательный, так как есть и другие необязательные параметры: `host`, `hostname`, `port`. Следующий псевдокод показывает приоритет: - -```sql -CASE - WHEN collection['addresses_expr'] != '' THEN collection['addresses_expr'] - WHEN collection['host'] != '' THEN collection['host'] || ':' || if(collection['port'] != '', collection['port'], '5432') - WHEN collection['hostname'] != '' THEN collection['hostname'] || ':' || if(collection['port'] != '', collection['port'], '5432') -END -``` - -Пример создания: - -```sql -CREATE NAMED COLLECTION mypg AS -user = 'pguser', -password = 'jw8s0F4', -host = '127.0.0.1', -port = 5432, -database = 'test', -schema = 'test_schema' -``` - -Пример конфигурации: - -```xml - - - - pguser - jw8s0F4 - 127.0.0.1 - 5432 - test - test_schema - - - -``` - -### Пример использования именованных коллекций с табличной функцией `postgresql` {#example-of-using-named-collections-with-the-postgresql-function} - -```sql -SELECT * FROM postgresql(mypg, table = 'test'); - -┌─a─┬─b───┐ -│ 2 │ two │ -│ 1 │ one │ -└───┴─────┘ -SELECT * FROM postgresql(mypg, table = 'test', schema = 'public'); - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -### Пример использования именованных коллекций с базой данных на движке PostgreSQL {#example-of-using-named-collections-with-database-with-engine-postgresql} - -```sql -CREATE TABLE mypgtable (a Int64) ENGINE = PostgreSQL(mypg, table = 'test', schema = 'public'); - -SELECT * FROM mypgtable; - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -:::note -PostgreSQL копирует данные из именованной коллекции при создании таблицы. Изменения в коллекции не влияют на уже существующие таблицы. -::: - -### Пример использования именованных коллекций с базой данных на движке PostgreSQL {#example-of-using-named-collections-with-database-with-engine-postgresql-1} - -```sql -CREATE DATABASE mydatabase ENGINE = PostgreSQL(mypg); - -SHOW TABLES FROM mydatabase - -┌─name─┐ -│ test │ -└──────┘ -``` - -### Пример использования именованных коллекций со словарём, использующим POSTGRESQL в качестве источника {#example-of-using-named-collections-with-a-dictionary-with-source-postgresql} - -```sql -CREATE DICTIONARY dict (a Int64, b String) -PRIMARY KEY a -SOURCE(POSTGRESQL(NAME mypg TABLE test)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## Именованные коллекции для доступа к удалённой базе данных ClickHouse {#named-collections-for-accessing-a-remote-clickhouse-database} - -См. описание параметров в разделе [remote](../sql-reference/table-functions/remote.md/#parameters). - -Пример конфигурации: - -```sql -CREATE NAMED COLLECTION remote1 AS -host = 'remote_host', -port = 9000, -database = 'system', -user = 'foo', -password = 'secret', -secure = 1 -``` - -```xml - - - - remote_host - 9000 - system - foo - secret - 1 - - - -``` - -`secure` не требуется для подключения, так как используется `remoteSecure`, но может применяться для словарей. - -### Пример использования именованных коллекций с функциями `remote`/`remoteSecure` {#example-of-using-named-collections-with-the-remoteremotesecure-functions} - -```sql -SELECT * FROM remote(remote1, table = one); -┌─dummy─┐ -│ 0 │ -└───────┘ - -SELECT * FROM remote(remote1, database = merge(system, '^one')); -┌─dummy─┐ -│ 0 │ -└───────┘ - -INSERT INTO FUNCTION remote(remote1, database = default, table = test) VALUES (1,'a'); - -SELECT * FROM remote(remote1, database = default, table = test); -┌─a─┬─b─┐ -│ 1 │ a │ -└───┴───┘ -``` - -### Пример использования именованных коллекций со словарём, использующим ClickHouse в качестве источника {#example-of-using-named-collections-with-a-dictionary-with-source-clickhouse} - -```sql -CREATE DICTIONARY dict(a Int64, b String) -PRIMARY KEY a -SOURCE(CLICKHOUSE(NAME remote1 TABLE test DB default)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 1); -┌─dictGet('dict', 'b', 1)─┐ -│ a │ -└─────────────────────────┘ -``` - -## Именованные коллекции для доступа к Kafka {#named-collections-for-accessing-kafka} - -См. описание параметров в разделе [Kafka](../engines/table-engines/integrations/kafka.md). - -### Пример DDL {#ddl-example-3} - -```sql -CREATE NAMED COLLECTION my_kafka_cluster AS -kafka_broker_list = 'localhost:9092', -kafka_topic_list = 'kafka_topic', -kafka_group_name = 'consumer_group', -kafka_format = 'JSONEachRow', -kafka_max_block_size = '1048576'; - -``` - -### Пример XML {#xml-example-3} - -```xml - - - - localhost:9092 - kafka_topic - consumer_group - JSONEachRow - 1048576 - - - -``` - -### Пример использования именованных коллекций с таблицей Kafka {#example-of-using-named-collections-with-a-kafka-table} - -Оба следующих примера используют одну и ту же именованную коллекцию `my_kafka_cluster`: - -```sql -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) - -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) -SETTINGS kafka_num_consumers = 4, - kafka_thread_per_consumer = 1; -``` - -## Именованные коллекции для резервных копий {#named-collections-for-backups} - -Описание параметров см. в разделе [Резервное копирование и восстановление](./backup.md). - -### Пример DDL {#ddl-example-4} - -```sql -BACKUP TABLE default.test to S3(named_collection_s3_backups, 'directory') -``` - -### Пример XML {#xml-example-4} - -```xml - - - - https://my-s3-bucket.s3.amazonaws.com/backup-S3/ - ABC123 - Abc+123 - - - -``` - -## Именованные коллекции для доступа к таблице и словарю MongoDB {#named-collections-for-accessing-mongodb-table-and-dictionary} - -Описание параметров см. в разделе [mongodb](../sql-reference/table-functions/mongodb.md). - -### Пример DDL {#ddl-example-5} - -```sql -CREATE NAMED COLLECTION mymongo AS -user = '', -password = '', -host = '127.0.0.1', -port = 27017, -database = 'test', -collection = 'my_collection', -options = 'connectTimeoutMS=10000' -``` - -### Пример XML {#xml-example-5} - -```xml - - - - - - 127.0.0.1 - 27017 - test - my_collection - connectTimeoutMS=10000 - - - -``` - -#### Таблица MongoDB {#mongodb-table} - -```sql -CREATE TABLE mytable(log_type VARCHAR, host VARCHAR, command VARCHAR) ENGINE = MongoDB(mymongo, options='connectTimeoutMS=10000&compressors=zstd') -SELECT count() FROM mytable; - -┌─count()─┐ -│ 2 │ -└─────────┘ -``` - -:::note -DDL переопределяет настройку именованной коллекции для опций. -::: - -#### Словарь MongoDB {#mongodb-dictionary} - -```sql -CREATE DICTIONARY dict -( - `a` Int64, - `b` String -) -PRIMARY KEY a -SOURCE(MONGODB(NAME mymongo COLLECTION my_dict)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()) - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -:::note -Именованная коллекция задаёт имя коллекции `my_collection`. В вызове функции это значение переопределяется параметром `collection = 'my_dict'`, чтобы выбрать другую коллекцию. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/opentelemetry.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/opentelemetry.md deleted file mode 100644 index 59f74a17b73..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/opentelemetry.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Руководство по использованию OpenTelemetry для распределённой трассировки и - сбора метрик в ClickHouse' -sidebar_label: 'Трассировка ClickHouse с помощью OpenTelemetry' -sidebar_position: 62 -slug: /operations/opentelemetry -title: 'Трассировка ClickHouse с помощью OpenTelemetry' -doc_type: 'guide' ---- - -[OpenTelemetry](https://opentelemetry.io/) — это открытый стандарт для сбора трасс и метрик из распределённых приложений. ClickHouse частично поддерживает OpenTelemetry. - - - -## Передача контекста трассировки в ClickHouse {#supplying-trace-context-to-clickhouse} - -ClickHouse принимает HTTP-заголовки контекста трассировки, как описано в [рекомендации W3C](https://www.w3.org/TR/trace-context/). Он также принимает контекст трассировки по нативному протоколу, который используется для обмена данными между серверами ClickHouse или между клиентом и сервером. Для ручного тестирования заголовки контекста трассировки, соответствующие спецификации Trace Context, можно передать в `clickhouse-client` с помощью флагов `--opentelemetry-traceparent` и `--opentelemetry-tracestate`. - -Если родительский контекст трассировки не передан или переданный контекст трассировки не соответствует указанному выше стандарту W3C, ClickHouse может начать новую трассировку с вероятностью, задаваемой настройкой [opentelemetry_start_trace_probability](/operations/settings/settings#opentelemetry_start_trace_probability). - - - -## Распространение контекста трассировки {#propagating-the-trace-context} - -Контекст трассировки распространяется в последующие сервисы в следующих случаях: - -* Запросы к удалённым серверам ClickHouse, например, при использовании движка таблиц [Distributed](../engines/table-engines/special/distributed.md). - -* Табличная функция [url](../sql-reference/table-functions/url.md). Информация о контексте трассировки передаётся в HTTP-заголовках. - - - -## Трассировка самого ClickHouse {#tracing-the-clickhouse-itself} - -ClickHouse создаёт `trace spans` для каждого запроса и некоторых этапов его выполнения, таких как планирование запроса или распределённые запросы. - -Чтобы эта информация была полезной, данные трассировки должны быть экспортированы в систему мониторинга, поддерживающую OpenTelemetry, такую как [Jaeger](https://jaegertracing.io/) или [Prometheus](https://prometheus.io/). ClickHouse избегает зависимости от конкретной системы мониторинга и вместо этого предоставляет данные трассировки через системную таблицу. Информация о span'ах трассировки OpenTelemetry, [требуемая стандартом](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/overview.md#span), хранится в таблице [system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md). - -Таблица должна быть включена в конфигурации сервера, см. элемент `opentelemetry_span_log` в файле конфигурации по умолчанию `config.xml`. По умолчанию она включена. - -Теги или атрибуты сохраняются в виде двух параллельных массивов, содержащих ключи и значения. Для работы с ними используйте [ARRAY JOIN](../sql-reference/statements/select/array-join.md). - - - -## Log-query-settings {#log-query-settings} - -Настройка [log_query_settings](settings/settings.md) позволяет логировать изменения параметров запроса во время его выполнения. При включении любые изменения настроек запроса будут записываться в журнал спанов OpenTelemetry. Эта функция особенно полезна в продуктивной среде для отслеживания изменений конфигурации, которые могут повлиять на производительность запросов. - - - -## Интеграция с системами мониторинга {#integration-with-monitoring-systems} - -На данный момент нет готового инструмента, позволяющего экспортировать данные трассировки из ClickHouse в систему мониторинга. - -Для тестирования можно настроить экспорт с помощью материализованного представления с движком [URL](../engines/table-engines/special/url.md) поверх таблицы [system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md), которое будет отправлять поступающие лог-записи на HTTP-эндпоинт коллектора трассировок. Например, чтобы отправлять минимальные данные о спане в экземпляр Zipkin, запущенный по адресу `http://localhost:9411`, в формате Zipkin v2 JSON: - -```sql -CREATE MATERIALIZED VIEW default.zipkin_spans -ENGINE = URL('http://127.0.0.1:9411/api/v2/spans', 'JSONEachRow') -SETTINGS output_format_json_named_tuples_as_objects = 1, - output_format_json_array_of_rows = 1 AS -SELECT - lower(hex(trace_id)) AS traceId, - CASE WHEN parent_span_id = 0 THEN '' ELSE lower(hex(parent_span_id)) END AS parentId, - lower(hex(span_id)) AS id, - operation_name AS name, - start_time_us AS timestamp, - finish_time_us - start_time_us AS duration, - cast(tuple('clickhouse'), 'Tuple(serviceName text)') AS localEndpoint, - cast(tuple( - attribute.values[indexOf(attribute.names, 'db.statement')]), - 'Tuple("db.statement" text)') AS tags -FROM system.opentelemetry_span_log -``` - -В случае возникновения ошибок та часть данных журнала, для которой произошла ошибка, будет незаметно потеряна. Если данные не поступают, проверьте журнал сервера на наличие сообщений об ошибках. - - -## См. также {#related-content} - -- Блог: [Построение решения для наблюдаемости с ClickHouse — часть 2. Трейсы](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md deleted file mode 100644 index 856a7811d4d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'Документация по оптимизации на основе профилирования' -sidebar_label: 'Оптимизация на основе профилирования (PGO)' -sidebar_position: 54 -slug: /operations/optimizing-performance/profile-guided-optimization -title: 'Оптимизация на основе профилирования' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# Оптимизация, управляемая профилированием {#profile-guided-optimization} - -Profile-Guided Optimization (PGO) — это техника оптимизации компилятора, при которой программа оптимизируется на основе профиля выполнения во время работы. - -Согласно тестам, PGO помогает достичь более высокой производительности в ClickHouse. Мы наблюдаем улучшение до 15% по показателю QPS в тестовом наборе ClickBench. Более подробные результаты доступны [здесь](https://pastebin.com/xbue3HMU). Прирост производительности зависит от вашей типичной нагрузки — вы можете получить как лучшие, так и худшие результаты. - -Подробнее о PGO в ClickHouse можно прочитать в соответствующем [issue](https://github.com/ClickHouse/ClickHouse/issues/44567) на GitHub. - -## Как собрать ClickHouse с PGO? {#how-to-build-clickhouse-with-pgo} - -Существует два основных вида PGO: [Instrumentation](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers) и [Sampling](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers) (также известный как AutoFDO). В этом руководстве описывается использование Instrumentation PGO с ClickHouse. - -1. Соберите ClickHouse в инструментированном режиме. В Clang это можно сделать, передав опцию `-fprofile-generate` в `CXXFLAGS`. -2. Запустите инструментированный ClickHouse на образцовой нагрузке. Здесь следует воспроизвести вашу обычную рабочую нагрузку. Один из подходов — использовать [ClickBench](https://github.com/ClickHouse/ClickBench) в качестве примерной нагрузки. ClickHouse в режиме инструментирования может работать медленно, поэтому будьте к этому готовы и не запускайте инструментированный ClickHouse в производственных средах, критичных к производительности. -3. Пересоберите ClickHouse ещё раз с флагами компилятора `-fprofile-use` и профилями, которые были собраны на предыдущем шаге. - -Более подробное руководство по применению PGO приведено в [документации](https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization) Clang. - -Если вы собираетесь собирать образцовую нагрузку непосредственно в продукционной среде, мы рекомендуем попробовать использовать Sampling PGO. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md deleted file mode 100644 index a42ac5b9418..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: 'Документация по инструменту выборочного профилирования запросов в ClickHouse' -sidebar_label: 'Профилирование запросов' -sidebar_position: 54 -slug: /operations/optimizing-performance/sampling-query-profiler -title: 'Профилировщик запросов с семплированием' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# Профилировщик запросов с выборочным сбором данных {#sampling-query-profiler} - -ClickHouse запускает профилировщик с выборочным сбором данных, который позволяет анализировать выполнение запросов. С помощью профилировщика вы можете найти участки исходного кода, которые использовались чаще всего во время выполнения запроса. Вы можете отслеживать затраченное процессорное время (CPU time) и реальное время (wall-clock time), включая периоды простоя. - -Профилировщик запросов автоматически включен в ClickHouse Cloud, и вы можете выполнить пример запроса следующим образом: - -:::note Если вы выполняете следующий запрос в ClickHouse Cloud, обязательно измените `FROM system.trace_log` на `FROM clusterAllReplicas(default, system.trace_log)`, чтобы выбирать данные со всех узлов кластера -::: - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c' AND trace_type = 'CPU' AND event_date = today() -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -SETTINGS allow_introspection_functions = 1 -``` - -В самостоятельных (self-managed) развертываниях, чтобы использовать профилировщик запросов: - -* Настройте раздел [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log) конфигурации сервера. - - Этот раздел настраивает системную таблицу [trace_log](/operations/system-tables/trace_log), в которой содержатся результаты работы профилировщика. Она настроена по умолчанию. Помните, что данные в этой таблице корректны только для работающего сервера. После перезапуска сервера ClickHouse не очищает таблицу, и все сохранённые виртуальные адреса памяти могут стать недействительными. - -* Настройте параметры [query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns) или [query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns). Оба параметра можно использовать одновременно. - - Эти параметры позволяют настроить таймеры профилировщика. Поскольку это сессионные настройки, вы можете задать разную частоту семплирования для всего сервера, отдельных пользователей или профилей пользователей, для вашей интерактивной сессии и для каждого отдельного запроса. - -Частота семплирования по умолчанию — один сэмпл в секунду, при этом включены оба таймера — CPU и реального времени. Эта частота позволяет собрать достаточно информации о кластере ClickHouse. Одновременно при такой частоте работы профилировщик не влияет на производительность сервера ClickHouse. Если вам нужно профилировать каждый отдельный запрос, попробуйте использовать более высокую частоту семплирования. - -Чтобы проанализировать системную таблицу `trace_log`: - -* Установите пакет `clickhouse-common-static-dbg`. См. раздел [Install from DEB Packages](../../getting-started/install/install.mdx). - -* Разрешите функции интроспекции с помощью настройки [allow_introspection_functions](../../operations/settings/settings.md#allow_introspection_functions). - - В целях безопасности функции интроспекции по умолчанию отключены. - -* Используйте функции интроспекции `addressToLine`, `addressToLineWithInlines`, `addressToSymbol` и `demangle` ([introspection functions](../../sql-reference/functions/introspection.md)), чтобы получить имена функций и их позиции в коде ClickHouse. Чтобы получить профиль для какого-либо запроса, вам нужно агрегировать данные из таблицы `trace_log`. Вы можете агрегировать данные по отдельным функциям или по целым стекам трассировки. - -Если вам нужно визуализировать информацию из `trace_log`, попробуйте [flamegraph](/interfaces/third-party/gui#clickhouse-flamegraph) и [speedscope](https://github.com/laplab/clickhouse-speedscope). - -## Пример {#example} - -В этом примере мы: - -* Фильтруем данные `trace_log` по идентификатору запроса и текущей дате. -* Агрегируем по стек-трейсу. -* Используя функции интроспекции, получим отчет о следующем: - - * Именах символов и соответствующих им функциях исходного кода. - * Местоположениях этих функций в исходном коде. - -{/* */ } - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE (query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c') AND (event_date = today()) -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/performance-test.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/performance-test.md deleted file mode 100644 index 5a5a9d541cf..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/performance-test.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: 'Руководство по тестированию и оценке производительности оборудования с ClickHouse' -sidebar_label: 'Тестирование оборудования' -sidebar_position: 54 -slug: /operations/performance-test -title: 'Как протестировать оборудование с помощью ClickHouse' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -Вы можете запустить базовый тест производительности ClickHouse на любом сервере без установки пакетов ClickHouse. - -## Автоматизированный запуск {#automated-run} - -Вы можете запустить бенчмарк одним скриптом. - -1. Скачайте скрипт. - -```bash -wget https://raw.githubusercontent.com/ClickHouse/ClickBench/main/hardware/hardware.sh -``` - -2. Запустите скрипт. - -```bash -chmod a+x ./hardware.sh -./hardware.sh -``` - -3. Скопируйте полученный вывод и отправьте его на [feedback@clickhouse.com](mailto:feedback@clickhouse.com) - -Все результаты публикуются здесь: [https://clickhouse.com/benchmark/hardware/](https://clickhouse.com/benchmark/hardware/) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-cache.md deleted file mode 100644 index 0918abf2e94..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-cache.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -description: 'Руководство по использованию и настройке функции кэша запросов в ClickHouse' -sidebar_label: 'Кэш запросов' -sidebar_position: 65 -slug: /operations/query-cache -title: 'Кэш запросов' -doc_type: 'guide' ---- - -# Кэш запросов {#query-cache} - -Кэш запросов позволяет выполнять запросы `SELECT` только один раз и обслуживать последующие выполнения того же запроса напрямую из кэша. -В зависимости от типа запросов это может существенно снизить время отклика и потребление ресурсов сервера ClickHouse. - -## Предпосылки, дизайн и ограничения {#background-design-and-limitations} - -Кэши запросов в общем случае можно рассматривать как транзакционно согласованные или транзакционно несогласованные. - -* В транзакционно согласованных кэшах база данных аннулирует (отбрасывает) кэшированные результаты запроса, если результат запроса `SELECT` изменяется - или может измениться. В ClickHouse к операциям, изменяющим данные, относятся вставки/обновления/удаления в/из таблиц или схлопывающие - слияния (collapsing merges). Транзакционно согласованное кэширование особенно подходит для OLTP-СУБД, например - [MySQL](https://dev.mysql.com/doc/refman/5.6/en/query-cache.html) (в которой кэш запросов был удалён, начиная с версии v8.0) и - [Oracle](https://docs.oracle.com/database/121/TGDBA/tune_result_cache.htm). -* В транзакционно несогласованных кэшах допускаются небольшие неточности в результатах запросов при предположении, что всем записям в кэше - назначается срок действия, по истечении которого они становятся недействительными (например, 1 минута), и что исходные данные за этот период меняются незначительно. - В целом такой подход больше подходит для OLAP-СУБД. В качестве примера, когда транзакционно несогласованного кэширования достаточно, - рассмотрим почасовой отчёт по продажам в отчётном инструменте, к которому одновременно обращаются несколько пользователей. Данные о продажах - обычно меняются достаточно медленно, чтобы базе данных нужно было вычислить отчёт только один раз (что соответствует первому запросу `SELECT`). Последующие запросы могут - обслуживаться напрямую из кэша запросов. В этом примере разумным сроком действия кэша может быть 30 минут. - -Транзакционно несогласованное кэширование традиционно реализуется клиентскими инструментами или прокси-пакетами (например, -[chproxy](https://www.chproxy.org/configuration/caching/)), взаимодействующими с базой данных. В результате одна и та же логика кэширования и -конфигурация часто дублируются. С появлением кэша запросов в ClickHouse логика кэширования переносится на сторону сервера. Это снижает затраты на сопровождение -и устраняет дублирование. - -## Параметры конфигурации и их использование {#configuration-settings-and-usage} - -:::note -В ClickHouse Cloud для редактирования настроек кэша запросов необходимо использовать [настройки на уровне запроса](/operations/settings/query-level). Редактирование [настроек на уровне конфигурации](/operations/configuration-files) в данный момент не поддерживается. -::: - -:::note -[clickhouse-local](utilities/clickhouse-local.md) выполняет один запрос за раз. Поскольку кэширование результатов запроса в этом случае не имеет смысла, кэш результатов запросов в clickhouse-local отключен. -::: - -Параметр [use_query_cache](/operations/settings/settings#use_query_cache) можно использовать для управления тем, будет ли использовать кэш запросов конкретный запрос или все запросы текущего сеанса. Например, при первом выполнении запроса - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true; -``` - -будет сохранять результат запроса в кэше запросов. Последующие выполнения того же запроса (также с параметром `use_query_cache = true`) -будут считывать вычисленный результат из кэша и возвращать его немедленно. - -:::note -Настройка `use_query_cache` и всех остальных параметров, связанных с кэшем запросов, влияет только на отдельные операторы `SELECT`. В частности, -результаты `SELECT` в представления, созданные через `CREATE VIEW AS SELECT [...] SETTINGS use_query_cache = true`, не кэшируются, если только оператор `SELECT` -не выполняется с `SETTINGS use_query_cache = true`. -::: - -Способ использования кэша можно подробнее настроить с помощью параметров [enable_writes_to_query_cache](/operations/settings/settings#enable_writes_to_query_cache) -и [enable_reads_from_query_cache](/operations/settings/settings#enable_reads_from_query_cache) (по умолчанию обе имеют значение `true`). Первый параметр -определяет, сохраняются ли результаты запросов в кэше, а второй — должен ли сервер пытаться получать результаты запросов -из кэша. Например, следующий запрос будет использовать кэш только пассивно, то есть пытаться читать из него, но не сохранять в него свой -результат: - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, enable_writes_to_query_cache = false; -``` - -Для максимального контроля обычно рекомендуется задавать настройки `use_query_cache`, `enable_writes_to_query_cache` и -`enable_reads_from_query_cache` только для конкретных запросов. Также можно включить кэширование на уровне пользователя или профиля (например, через `SET -use_query_cache = true`), но следует иметь в виду, что тогда все запросы `SELECT` могут возвращать результаты из кэша. - -Кэш запросов можно очистить командой `SYSTEM DROP QUERY CACHE`. Содержимое кэша запросов отображается в системной таблице -[system.query_cache](system-tables/query_cache.md). Количество попаданий и промахов в кэш запросов с момента запуска базы данных показывается как события -«QueryCacheHits» и «QueryCacheMisses» в системной таблице [system.events](system-tables/events.md). Оба счётчика обновляются только для -запросов `SELECT`, которые выполняются с настройкой `use_query_cache = true`; другие запросы не влияют на «QueryCacheMisses». Поле `query_cache_usage` -в системной таблице [system.query_log](system-tables/query_log.md) показывает для каждого выполненного запроса, был ли результат запроса записан в кэш -или прочитан из кэша запросов. Метрики `QueryCacheEntries` и `QueryCacheBytes` в системной таблице -[system.metrics](system-tables/metrics.md) показывают, сколько записей и байт в данный момент содержит кэш запросов. - -Кэш запросов существует в одном экземпляре для каждого серверного процесса ClickHouse. Однако результаты кэша по умолчанию не являются общими между пользователями. Это можно -изменить (см. ниже), но делать так не рекомендуется по соображениям безопасности. - -Результаты запросов в кэше запросов идентифицируются по [абстрактному синтаксическому дереву (AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) -их запроса. Это означает, что кэширование не зависит от регистра, например `SELECT 1` и `select 1` рассматриваются как один и тот же запрос. Чтобы -сделать сопоставление более естественным, все настройки на уровне запроса, относящиеся к кэшу запросов и [форматированию вывода](settings/settings-formats.md), -удаляются из AST. - -Если запрос был прерван из-за исключения или отмены пользователем, запись в кэш запросов не производится. - -Размер кэша запросов в байтах, максимальное количество записей в кэше и максимальный размер отдельных записей кэша (в байтах и в -строках) можно настроить с помощью различных [параметров конфигурации сервера](/operations/server-configuration-parameters/settings#query_cache). - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - -Также можно ограничить использование кэша отдельными пользователями с помощью [профилей настроек](settings/settings-profiles.md) и [ограничений настроек](settings/constraints-on-settings.md). В частности, вы можете ограничить максимальный объём памяти (в байтах), который пользователь может выделить в кэше запросов, и максимальное количество сохранённых результатов запросов. Для этого сначала задайте значения параметров -[query_cache_max_size_in_bytes](/operations/settings/settings#query_cache_max_size_in_bytes) и -[query_cache_max_entries](/operations/settings/settings#query_cache_max_entries) в профиле пользователя в `users.xml`, затем сделайте обе настройки доступными только для чтения: - -```xml - - - - 10000 - - 100 - - - - - - - - - - - -``` - -Чтобы задать минимальную длительность выполнения запроса, начиная с которой его результат может кэшироваться, вы можете использовать настройку -[query_cache_min_query_duration](/operations/settings/settings#query_cache_min_query_duration). Например, результат запроса - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, query_cache_min_query_duration = 5000; -``` - -кэшируется только если выполнение запроса длится дольше 5 секунд. Также можно задать, сколько раз запрос должен быть выполнен, прежде чем его результат будет -закэширован — для этого используйте настройку [query_cache_min_query_runs](/operations/settings/settings#query_cache_min_query_runs). - -Записи в кэше запросов становятся устаревшими через определенный период времени (time-to-live). По умолчанию этот период составляет 60 секунд, но другое -значение можно задать на уровне сессии, профиля или отдельного запроса, используя настройку [query_cache_ttl](/operations/settings/settings#query_cache_ttl). Кэш запросов -удаляет записи «лениво», то есть когда запись становится устаревшей, она не удаляется из кэша немедленно. Вместо этого, когда в кэш запросов нужно вставить новую запись, -база данных проверяет, достаточно ли в кэше свободного места для новой записи. Если это не так, -база данных пытается удалить все устаревшие записи. Если в кэше по-прежнему недостаточно свободного места, новая запись не добавляется. - -Если запрос выполняется через HTTP, то ClickHouse устанавливает заголовки `Age` и `Expires` с временем жизни (в секундах) и временной меткой истечения срока действия -закэшированной записи. - -Записи в кэше запросов по умолчанию сжимаются. Это уменьшает общее потребление памяти ценой более медленной записи в кэш и чтения -из кэша запросов. Чтобы отключить сжатие, используйте настройку [query_cache_compress_entries](/operations/settings/settings#query_cache_compress_entries). - -Иногда полезно хранить в кэше несколько результатов для одного и того же запроса. Это можно сделать с помощью настройки -[query_cache_tag](/operations/settings/settings#query_cache_tag), которая выступает в роли метки (или пространства имен) для записей кэша запросов. Кэш запросов -считает результаты одного и того же запроса с разными тегами разными. - -Пример создания трех разных записей в кэше запросов для одного и того же запроса: - -```sql -SELECT 1 SETTINGS use_query_cache = true; -- query_cache_tag неявно '' (пустая строка) -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 1'; -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 2'; -``` - -Чтобы удалить из кэша запросов только записи с тегом `tag`, можно использовать оператор `SYSTEM DROP QUERY CACHE TAG 'tag'`. - -ClickHouse читает данные таблицы блоками по [max_block_size](/operations/settings/settings#max_block_size) строк. Из-за фильтрации, агрегации -и т.п. блоки результатов обычно значительно меньше, чем `max_block_size`, но бывают случаи, когда они существенно больше. Настройка -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results) (включена по умолчанию) управляет тем, -будут ли блоки результатов объединяться (если они маленькие) или разбиваться (если они большие) на блоки размера `max_block_size` перед -записью в кэш результатов запросов. Это снижает производительность записи в кэш запросов, но улучшает степень сжатия элементов кэша и -обеспечивает более естественную зернистость блоков при последующей выдаче результатов запросов из кэша. - -В результате кэш запросов хранит для каждого запроса несколько (частичных) -блоков результатов. Хотя такое поведение является разумным значением по умолчанию, его можно отключить с помощью настройки -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results). - -Также результаты запросов с недетерминированными функциями по умолчанию не кэшируются. К таким функциям относятся: - -* функции для доступа к словарям: [`dictGet()`](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull) и т.п.; -* [пользовательские функции](../sql-reference/statements/create/function.md) без тега `true` в их XML- - определении; -* функции, возвращающие текущие дату или время: [`now()`](../sql-reference/functions/date-time-functions.md#now), - [`today()`](../sql-reference/functions/date-time-functions.md#today), - [`yesterday()`](../sql-reference/functions/date-time-functions.md#yesterday) и т.п.; -* функции, возвращающие случайные значения: [`randomString()`](../sql-reference/functions/random-functions.md#randomString), - [`fuzzBits()`](../sql-reference/functions/random-functions.md#fuzzBits) и т.п.; -* функции, результат которых зависит от размера и порядка внутренних фрагментов (чанков), используемых при обработке запроса: - [`nowInBlock()`](../sql-reference/functions/date-time-functions.md#nowInBlock) и т.п., - [`rowNumberInBlock()`](../sql-reference/functions/other-functions.md#rowNumberInBlock), - [`runningDifference()`](../sql-reference/functions/other-functions.md#runningDifference), - [`blockSize()`](../sql-reference/functions/other-functions.md#blockSize) и т.п.; -* функции, зависящие от окружения: [`currentUser()`](../sql-reference/functions/other-functions.md#currentUser), - [`queryID()`](/sql-reference/functions/other-functions#queryID), - [`getMacro()`](../sql-reference/functions/other-functions.md#getMacro) и т.п. - -Чтобы принудительно кэшировать результаты запросов с недетерминированными функциями, независимо от этого поведения, используйте настройку -[query_cache_nondeterministic_function_handling](/operations/settings/settings#query_cache_nondeterministic_function_handling). - -Результаты запросов, которые обращаются к системным таблицам (например, [system.processes](system-tables/processes.md) или -[information_schema.tables](system-tables/information_schema.md)), по умолчанию не кэшируются. Чтобы принудительно кэшировать результаты -запросов с системными таблицами, используйте настройку [query_cache_system_table_handling](/operations/settings/settings#query_cache_system_table_handling). - -Наконец, элементы в кэше запросов не разделяются между пользователями по соображениям безопасности. Например, пользователь A не должен -иметь возможность обойти политику по строкам таблицы, запустив тот же запрос, что и другой пользователь B, для которого такая политика не -задана. Однако при необходимости элементы кэша могут быть помечены как доступные другим пользователям (т.е. общими) с помощью настройки -[query_cache_share_between_users](/operations/settings/settings#query_cache_share_between_users). - -## Связанные материалы {#related-content} - -* Блог: [Introducing the ClickHouse Query Cache](https://clickhouse.com/blog/introduction-to-the-clickhouse-query-cache-and-design) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md deleted file mode 100644 index d817c442242..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: 'Руководство по использованию и настройке механизма кэша условий запроса в ClickHouse' -sidebar_label: 'Кэш условий запроса' -sidebar_position: 64 -slug: /operations/query-condition-cache -title: 'Кэш условий запроса' -doc_type: 'guide' ---- - - - -# Кэш условий запроса {#query-condition-cache} - -:::note -Кэш условий запроса работает только, когда [enable_analyzer](https://clickhouse.com/docs/operations/settings/settings#enable_analyzer) установлен в значение true, что является значением по умолчанию. -::: - -Во многих реальных рабочих нагрузках выполняются повторяющиеся запросы к одним и тем же или почти тем же данным (например, к уже существующим данным плюс новым данным). -ClickHouse предоставляет различные техники оптимизации для таких шаблонов запросов. -Один из вариантов — настроить физическую организацию данных с помощью индексных структур (например, индексы первичного ключа, пропускающие индексы, проекции) или предварительных вычислений (материализованные представления). -Другой вариант — использовать [кэш запросов](query-cache.md) ClickHouse, чтобы избежать повторной обработки запросов. -Недостаток первого подхода в том, что он требует ручного вмешательства и мониторинга со стороны администратора базы данных. -Второй подход может возвращать устаревшие результаты (так как кэш запросов транзакционно не согласован), что может быть как приемлемо, так и неприемлемо в зависимости от сценария использования. - -Кэш условий запроса предоставляет элегантное решение обеих этих проблем. -Он основан на идее, что вычисление фильтра (например, `WHERE col = 'xyz'`) над одними и теми же данными всегда возвращает один и тот же результат. -Более конкретно, кэш условий запроса запоминает для каждого вычисленного фильтра и каждой гранулы (= блок 8192 строк по умолчанию), существует ли в грануле хотя бы одна строка, удовлетворяющая условию фильтра. -Эта информация записывается в виде одного бита: бит 0 означает, что ни одна строка не подходит под фильтр, тогда как бит 1 означает, что существует как минимум одна подходящая строка. -В первом случае ClickHouse может пропустить соответствующую гранулу при вычислении фильтра, во втором случае гранулу необходимо загрузить и обработать. - -Кэш условий запроса эффективен, если выполняются три предпосылки: -- Во‑первых, нагрузка должна многократно вычислять одни и те же условия фильтрации. Это естественным образом происходит, если один и тот же запрос выполняется несколько раз, но это также возможно, если два запроса используют одни и те же фильтры, например `SELECT product FROM products WHERE quality > 3` и `SELECT vendor, count() FROM products WHERE quality > 3`. -- Во‑вторых, большая часть данных должна быть неизменяемой, то есть не изменяться между запросами. В ClickHouse это обычно так, поскольку части (parts) неизменяемы и создаются только с помощью INSERT. -- В‑третьих, фильтры должны быть селективными, то есть лишь относительно небольшое число строк удовлетворяет условию фильтра. Чем меньше строк соответствует условию фильтра, тем больше гранул будет записано с битом 0 (нет подходящих строк), и тем больше данных можно будет «отсечь» при последующих вычислениях фильтра. - - - -## Потребление памяти {#memory-consumption} - -Поскольку кэш условий запроса хранит только один бит на условие фильтрации и гранулу, он потребляет совсем немного памяти. -Максимальный размер кэша условий запроса можно настроить с помощью серверного параметра [`query_condition_cache_size`](server-configuration-parameters/settings.md#query_condition_cache_size) (по умолчанию: 100 МБ). -Размер кэша 100 МБ соответствует 100 * 1024 * 1024 * 8 = 838 860 800 записей. -Поскольку каждая запись представляет собой метку (по умолчанию 8192 строки), кэш может покрыть до 6 871 947 673 600 (6,8 триллиона) строк одного столбца. -На практике фильтры применяются более чем к одному столбцу, поэтому это число нужно разделить на количество столбцов, по которым выполняется фильтрация. - - - -## Параметры конфигурации и использование {#configuration-settings-and-usage} - -Параметр [use_query_condition_cache](settings/settings#use_query_condition_cache) определяет, должен ли конкретный запрос или все запросы текущего сеанса использовать кэш условий запроса. - -Например, первое выполнение запроса - -```sql -SELECT col1, col2 -FROM table -WHERE col1 = 'x' -SETTINGS use_query_condition_cache = true; -``` - -кэш будет сохранять диапазоны таблицы, которые не удовлетворяют предикату. -Последующие выполнения того же запроса, также с параметром `use_query_condition_cache = true`, будут использовать кэш условий запроса для сканирования меньшего объёма данных. - - -## Администрирование {#administration} - -Кэш условий запросов не сохраняется между перезапусками ClickHouse. - -Чтобы очистить кэш условий запросов, выполните команду [`SYSTEM DROP QUERY CONDITION CACHE`](../sql-reference/statements/system.md#drop-query-condition-cache). - -Содержимое кэша отображается в системной таблице [system.query_condition_cache](system-tables/query_condition_cache.md). -Чтобы вычислить текущий размер кэша условий запросов в МБ, выполните `SELECT formatReadableSize(sum(entry_size)) FROM system.query_condition_cache`. -Если требуется исследовать отдельные фильтрующие условия, проверьте поле `condition` в `system.query_condition_cache`. -Обратите внимание, что это поле заполняется только в том случае, если запрос выполняется с включённой настройкой [query_condition_cache_store_conditions_as_plaintext](settings/settings#query_condition_cache_store_conditions_as_plaintext). - -Количество попаданий и промахов кэша условий запросов с момента запуска базы данных отображается как события «QueryConditionCacheHits» и «QueryConditionCacheMisses» в системной таблице [system.events](system-tables/events.md). -Оба счётчика обновляются только для запросов `SELECT`, которые выполняются с настройкой `use_query_condition_cache = true`, другие запросы не влияют на счётчик «QueryCacheMisses». - - - -## Связанные материалы {#related-content} - -- Публикация в блоге: [Introducing the Query Condition Cache](https://clickhouse.com/blog/introducing-the-clickhouse-query-condition-cache) -- [Predicate Caching: Query-Driven Secondary Indexing for Cloud Data Warehouses (Schmidt et. al., 2024)](https://doi.org/10.1145/3626246.3653395) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/quotas.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/quotas.md deleted file mode 100644 index fae92d56208..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/quotas.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -description: 'Руководство по настройке и управлению квотами использования ресурсов в ClickHouse' -sidebar_label: 'Квоты' -sidebar_position: 51 -slug: /operations/quotas -title: 'Квоты' -doc_type: 'guide' ---- - -:::note Квоты в ClickHouse Cloud -Квоты поддерживаются в ClickHouse Cloud, но должны создаваться с использованием [DDL-синтаксиса](/sql-reference/statements/create/quota). Подход с XML-конфигурацией, описанный ниже, **не поддерживается**. -::: - -Квоты позволяют ограничивать или отслеживать использование ресурсов за определённый период времени. -Квоты настраиваются в конфигурации пользователей, которая обычно называется 'users.xml'. - -В системе также есть возможность ограничивать сложность одного запроса. См. раздел [Ограничения на сложность запросов](../operations/settings/query-complexity.md). - -В отличие от ограничений на сложность запросов, квоты: - -* Ограничивают набор запросов, которые можно выполнить за период времени, вместо ограничения одного запроса. -* Учитывают ресурсы, затраченные на всех удалённых серверах при распределённой обработке запросов. - -Рассмотрим раздел файла 'users.xml', который определяет квоты. - -```xml - - - - - - - - 3600 - - - 0 - 0 - 0 - 0 - 0 - 0 - 0 - - -``` - -По умолчанию квота отслеживает потребление ресурсов почасово, не ограничивая использование. -Потребление ресурсов, рассчитанное для каждого интервала, выводится в журнал сервера после каждого запроса. - -```xml - - - - - 3600 - - 1000 - 100 - 100 - 5000000 - 100 - 1000000000 - 100000000000 - 900 - 5 - - - - 86400 - - 10000 - 10000 - 10000 - 1000 - 5000000000 - 160000000000 - 500000000000 - 16000000000000 - 7200 - - -``` - -Для квоты 'statbox' ограничения задаются на каждый час и на каждые 24 часа (86 400 секунд). Отсчет интервала ведется от некоторого фиксированного, зависящего от реализации момента времени. Другими словами, 24-часовой интервал не обязательно начинается в полночь. - -Когда интервал заканчивается, все накопленные значения обнуляются. Для следующего часа вычисление квоты начинается заново. - -Вот значения, для которых могут быть заданы ограничения: - -`queries` – Общее количество запросов. - -`query_selects` – Общее количество запросов SELECT. - -`query_inserts` – Общее количество запросов INSERT. - -`errors` – Количество запросов, которые завершились с исключением. - -`result_rows` – Общее количество строк, возвращенных в результате. - -`result_bytes` - Общий размер строк, возвращенных в результате. - -`read_rows` – Общее количество исходных строк, прочитанных из таблиц для выполнения запроса на всех удаленных серверах. - -`read_bytes` - Общий размер данных, прочитанных из таблиц для выполнения запроса на всех удаленных серверах. - -`written_bytes` - Общий объем записанных данных. - -`execution_time` – Общее время выполнения запроса в секундах (реальное "настенное" время, wall time). - -`failed_sequential_authentications` - Общее количество последовательных ошибок аутентификации. - - -Если лимит превышен хотя бы для одного временного интервала, генерируется исключение с текстом о том, какое ограничение было превышено, для какого интервала и когда начинается новый интервал (когда запросы можно отправлять снова). - -Квоты могут использовать механизм «quota key» для раздельного учета использования ресурсов по нескольким ключам. Вот пример этого: - -```xml - - - - -``` - -Квота назначается пользователям в разделе конфигурации 'users'. См. раздел "Права доступа". - -При распределённой обработке запросов накопленные значения хранятся на сервере, с которого был отправлен запрос. Поэтому, если пользователь перейдёт на другой сервер, квота там будет "начинаться заново". - -При перезапуске сервера квоты сбрасываются. - - -## Связанные материалы {#related-content} - -- Статья в блоге: [Создание одностраничных веб-приложений с ClickHouse](https://clickhouse.com/blog/building-single-page-applications-with-clickhouse-and-http) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md deleted file mode 100644 index 1d0cc6738ee..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md +++ /dev/null @@ -1,2685 +0,0 @@ -## asynchronous_metric_log {#asynchronous_metric_log} - -По умолчанию включено в развертываниях ClickHouse Cloud. - -Если этот параметр не включён по умолчанию в вашей среде, в зависимости от того, как был установлен ClickHouse, вы можете воспользоваться приведённой ниже инструкцией, чтобы включить или отключить его. - -**Включение** - -Чтобы вручную включить ведение истории в асинхронном журнале метрик [`system.asynchronous_metric_log`](../../operations/system-tables/asynchronous_metric_log.md), создайте файл `/etc/clickhouse-server/config.d/asynchronous_metric_log.xml` со следующим содержимым: - -```xml - - - system - asynchronous_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**Отключение** - -Чтобы отключить параметр `asynchronous_metric_log`, необходимо создать файл `/etc/clickhouse-server/config.d/disable_asynchronous_metric_log.xml` со следующим содержимым: - -```xml - -``` - - - - -## auth_use_forwarded_address {#auth_use_forwarded_address} - -Использовать исходный IP-адрес для аутентификации клиентов, подключённых через прокси. - -:::note -Эта настройка должна использоваться с особой осторожностью, так как адреса, передаваемые прокси, легко подделать — серверы, принимающие такую аутентификацию, не должны быть доступны напрямую, а только через доверенный прокси. -::: - - - -## резервные копии {#backups} - -Настройки для резервного копирования, используемые при выполнении операторов [`BACKUP` и `RESTORE`](../backup.md). - -Следующие настройки можно задать с помощью под-тегов: - - - -{/* SQL - WITH settings AS ( - SELECT arrayJoin([ - ('allow_concurrent_backups', 'Bool','Определяет, могут ли несколько операций резервного копирования выполняться параллельно на одном хосте.', 'true'), - ('allow_concurrent_restores', 'Bool', 'Определяет, могут ли несколько операций восстановления выполняться параллельно на одном хосте.', 'true'), - ('allowed_disk', 'String', 'Диск для резервного копирования при использовании `File()`. Этот параметр должен быть задан для использования `File`.', ''), - ('allowed_path', 'String', 'Путь для резервного копирования при использовании `File()`. Этот параметр должен быть задан для использования `File`.', ''), - ('attempts_to_collect_metadata_before_sleep', 'UInt', 'Количество попыток сбора метаданных перед переходом в режим ожидания в случае несогласованности после сравнения собранных метаданных.', '2'), - ('collect_metadata_timeout', 'UInt64', 'Тайм-аут в миллисекундах на сбор метаданных во время резервного копирования.', '600000'), - ('compare_collected_metadata', 'Bool', 'Если true, сравнивает собранные метаданные с существующими, чтобы убедиться, что они не изменяются во время резервного копирования.', 'true'), - ('create_table_timeout', 'UInt64', 'Тайм-аут в миллисекундах на создание таблиц во время восстановления.', '300000'), - ('max_attempts_after_bad_version', 'UInt64', 'Максимальное количество попыток повторить операцию после ошибки «неверная версия» при координированном резервном копировании/восстановлении.', '3'), - ('max_sleep_before_next_attempt_to_collect_metadata', 'UInt64', 'Максимальное время ожидания в миллисекундах перед следующей попыткой сбора метаданных.', '100'), - ('min_sleep_before_next_attempt_to_collect_metadata', 'UInt64', 'Минимальное время ожидания в миллисекундах перед следующей попыткой сбора метаданных.', '5000'), - ('remove_backup_files_after_failure', 'Bool', 'Если команда `BACKUP` завершилась с ошибкой, ClickHouse попытается удалить файлы, уже скопированные в резервную копию до сбоя, иначе скопированные файлы будут оставлены как есть.', 'true'), - ('sync_period_ms', 'UInt64', 'Период синхронизации в миллисекундах для координированного резервного копирования/восстановления.', '5000'), - ('test_inject_sleep', 'Bool', 'Тестовая задержка (sleep), используемая при тестировании.', 'false'), - ('test_randomize_order', 'Bool', 'Если true, случайным образом изменяет порядок некоторых операций в целях тестирования.', 'false'), - ('zookeeper_path', 'String', 'Путь в ZooKeeper, по которому хранятся метаданные резервного копирования и восстановления при использовании предложения `ON CLUSTER`.', '/clickhouse/backups') - ]) AS t ) - SELECT concat('`', t.1, '`') AS Setting, t.2 AS Type, t.3 AS Description, concat('`', t.4, '`') AS Default FROM settings FORMAT Markdown - */ } - -| Настройка | Тип | Описание | По умолчанию | -| :-------------------------------------------------- | :----- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------- | -| `allow_concurrent_backups` | Булево | Определяет, могут ли несколько операций резервного копирования выполняться одновременно на одном и том же хосте. | `true` | -| `allow_concurrent_restores` | Bool | Определяет, можно ли выполнять несколько операций восстановления одновременно на одном хосте. | `true` | -| `allowed_disk` | Строка | Диск, на который выполняется резервное копирование при использовании `File()`. Этот параметр необходимо задать, чтобы использовать `File()`. | `` | -| `allowed_path` | Строка | Путь для сохранения резервной копии при использовании `File()`. Этот параметр необходимо задать, чтобы использовать `File`. | `` | -| `attempts_to_collect_metadata_before_sleep` | UInt | Количество попыток сбора метаданных, прежде чем сделать паузу при обнаружении несоответствий после сравнения собранных метаданных. | `2` | -| `collect_metadata_timeout` | UInt64 | Тайм-аут (в миллисекундах) на сбор метаданных при создании резервной копии. | `600000` | -| `compare_collected_metadata` | Bool | Если значение — `true`, сравнивает собранные метаданные с существующими, чтобы убедиться, что они не изменялись во время резервного копирования. | `true` | -| `create_table_timeout` | UInt64 | Таймаут в миллисекундах на создание таблиц при восстановлении. | `300000` | -| `max_attempts_after_bad_version` | UInt64 | Максимальное число повторных попыток при возникновении ошибки некорректной версии во время координированного резервного копирования/восстановления. | `3` | -| `max_sleep_before_next_attempt_to_collect_metadata` | UInt64 | Максимальное время ожидания (в миллисекундах) перед следующей попыткой сбора метаданных. | `100` | -| `min_sleep_before_next_attempt_to_collect_metadata` | UInt64 | Минимальная пауза (в миллисекундах) перед следующей попыткой сбора метаданных. | `5000` | -| `remove_backup_files_after_failure` | Bool | Если команда `BACKUP` завершается с ошибкой, ClickHouse попытается удалить файлы, уже скопированные в резервную копию к моменту сбоя, в противном случае оставит скопированные файлы без изменений. | `true` | -| `sync_period_ms` | UInt64 | Период синхронизации в миллисекундах для согласованного резервного копирования и восстановления. | `5000` | -| `test_inject_sleep` | Bool | Пауза для тестирования | `false` | -| `test_randomize_order` | Bool | Если имеет значение `true`, случайным образом изменяет порядок некоторых операций для целей тестирования. | `false` | -| `zookeeper_path` | String | Путь в ZooKeeper, где хранятся метаданные резервного копирования и восстановления при использовании выражения `ON CLUSTER`. | `/clickhouse/backups` | - -По умолчанию этот параметр имеет следующее значение: - -```xml - - .... - -``` - - -## bcrypt_workfactor {#bcrypt_workfactor} - -Коэффициент сложности для типа аутентификации `bcrypt_password`, который использует [алгоритм Bcrypt](https://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/). -Он определяет объём вычислений и время, необходимые для вычисления хеша и проверки пароля. - -```xml -12 -``` - -:::warning -Для приложений с частыми операциями аутентификации -из-за вычислительных затрат bcrypt при более высоких параметрах сложности -рассмотрите альтернативные методы аутентификации. -::: - - -## table_engines_require_grant {#table_engines_require_grant} - -Если установлено в значение `true`, пользователям требуется привилегия для создания таблицы с определённым движком, например: `GRANT TABLE ENGINE ON TinyLog TO user`. - -:::note -По умолчанию, для сохранения обратной совместимости, при создании таблицы с конкретным движком требование привилегии игнорируется, однако вы можете изменить это поведение, установив данный параметр в `true`. -::: - - - -## builtin_dictionaries_reload_interval {#builtin_dictionaries_reload_interval} - -Интервал в секундах между перезагрузками встроенных словарей. - -ClickHouse перезагружает встроенные словари каждые x секунд. Это позволяет редактировать словари «на лету» без перезапуска сервера. - -**Пример** - -```xml -3600 -``` - - -## compression {#compression} - -Настройки сжатия данных для таблиц с движком [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -:::note -Мы рекомендуем не изменять этот параметр, если вы только начинаете работать с ClickHouse. -::: - -**Шаблон конфигурации**: - -```xml - - - ... - ... - ... - ... - - ... - -``` - -**Поля ``**: - -* `min_part_size` – Минимальный размер части данных. -* `min_part_size_ratio` – Отношение размера части данных к размеру таблицы. -* `method` – Метод сжатия. Допустимые значения: `lz4`, `lz4hc`, `zstd`,`deflate_qpl`. -* `level` – Уровень сжатия. См. [Codecs](/sql-reference/statements/create/table#general-purpose-codecs). - -:::note -Вы можете сконфигурировать несколько секций ``. -::: - -**Действия при выполнении условий**: - -* Если часть данных соответствует набору условий, ClickHouse использует указанный метод сжатия. -* Если часть данных соответствует нескольким наборам условий, ClickHouse использует первый подходящий набор. - -:::note -Если ни одно условие не выполнено для части данных, ClickHouse использует сжатие `lz4`. -::: - -**Пример** - -```xml - - - 10000000000 - 0.01 - zstd - 1 - - -``` - - -## encryption {#encryption} - -Настраивает команду для получения ключа, который будет использоваться [кодеками шифрования](/sql-reference/statements/create/table#encryption-codecs). Ключ (или ключи) должен быть передан через переменные окружения или задан в конфигурационном файле. - -Ключи могут быть в шестнадцатеричном формате или строкой длиной 16 байт. - -**Пример** - -Загрузка из конфигурации: - -```xml - - - 1234567812345678 - - -``` - -:::note -Хранить ключи в конфигурационном файле не рекомендуется — это небезопасно. Вы можете вынести ключи в отдельный конфигурационный файл на защищённом диске и поместить символическую ссылку на этот файл в папку `config.d/`. -::: - -Загрузка из конфигурации, когда ключ задан в шестнадцатеричном формате: - -```xml - - - 00112233445566778899aabbccddeeff - - -``` - -Загрузка ключа из переменной окружения: - -```xml - - - - - -``` - -Здесь `current_key_id` задаёт текущий ключ для шифрования, а все перечисленные ключи могут использоваться для расшифровки. - -Каждый из этих методов может быть применён для нескольких ключей: - -```xml - - - 00112233445566778899aabbccddeeff - - 1 - - -``` - -Здесь `current_key_id` указывает текущий ключ шифрования. - -Также пользователь может задать nonce длиной 12 байт (по умолчанию в процессах шифрования и расшифровки используется nonce, состоящий из нулевых байт): - -```xml - - - 012345678910 - - -``` - -Или его можно задать в шестнадцатеричном формате: - -```xml - - - abcdefabcdef - - -``` - -:::note -Все вышесказанное применимо и к `aes_256_gcm_siv` (но длина ключа должна быть 32 байта). -::: - - -## error_log {#error_log} - -По умолчанию он отключён. - -**Включение** - -Чтобы вручную включить сбор истории ошибок [`system.error_log`](../../operations/system-tables/error_log.md), создайте `/etc/clickhouse-server/config.d/error_log.xml` со следующим содержимым: - -```xml - - - system - error_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**Отключение** - -Чтобы отключить параметр `error_log`, нужно создать файл `/etc/clickhouse-server/config.d/disable_error_log.xml` со следующим содержимым: - -```xml - - - -``` - - - - -## custom_settings_prefixes {#custom_settings_prefixes} - -Список префиксов для [пользовательских настроек](/operations/settings/query-level#custom_settings). Префиксы должны быть разделены запятыми. - -**Пример** - -```xml -custom_ -``` - -**См. также** - -* [Пользовательские настройки](/operations/settings/query-level#custom_settings) - - -## core_dump {#core_dump} - -Настраивает мягкое ограничение на размер файла дампа памяти (core dump). - -:::note -Жёсткое ограничение настраивается с помощью системных инструментов -::: - -**Пример** - -```xml - - 1073741824 - -``` - - -## default_profile {#default_profile} - -Профиль настроек по умолчанию. Профили настроек находятся в файле, указанном в параметре `user_config`. - -**Пример** - -```xml -default -``` - - -## dictionaries_config {#dictionaries_config} - -Путь к конфигурационному файлу словарей. - -Путь: - -* Укажите абсолютный путь или путь, относительный к файлу конфигурации сервера. -* Путь может содержать подстановочные знаки * и ?. - -См. также: - -* "[Словари](../../sql-reference/dictionaries/index.md)". - -**Пример** - -```xml -*_dictionary.xml -``` - - -## user_defined_executable_functions_config {#user_defined_executable_functions_config} - -Путь к конфигурационному файлу исполняемых пользовательских функций. - -Путь: - -* Укажите абсолютный путь или путь относительно конфигурационного файла сервера. -* Путь может содержать подстановочные символы * и ?. - -См. также: - -* "[Исполняемые пользовательские функции](/sql-reference/functions/udf#executable-user-defined-functions).". - -**Пример** - -```xml -*_function.xml -``` - - -## format_schema_path {#format_schema_path} - -Путь к каталогу со схемами для входных данных, например, схем для формата [CapnProto](/interfaces/formats/CapnProto). - -**Пример** - -```xml - -format_schemas/ -``` - - -## graphite {#graphite} - -Отправка данных в [Graphite](https://github.com/graphite-project). - -Настройки: - -* `host` – сервер Graphite. -* `port` – порт на сервере Graphite. -* `interval` – интервал отправки данных, в секундах. -* `timeout` – тайм-аут при отправке данных, в секундах. -* `root_path` – префикс для ключей. -* `metrics` – отправка данных из таблицы [system.metrics](/operations/system-tables/metrics). -* `events` – отправка дельта-данных, накопленных за период времени, из таблицы [system.events](/operations/system-tables/events). -* `events_cumulative` – отправка накопительных (кумулятивных) данных из таблицы [system.events](/operations/system-tables/events). -* `asynchronous_metrics` – отправка данных из таблицы [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics). - -Можно настроить несколько блоков ``. Например, это можно использовать для отправки разных данных с разными интервалами. - -**Пример** - -```xml - - localhost - 42000 - 0.1 - 60 - one_min - true - true - false - true - -``` - - -## graphite_rollup {#graphite_rollup} - -Настройки прореживания данных для Graphite. - -Для получения дополнительной информации см. [GraphiteMergeTree](../../engines/table-engines/mergetree-family/graphitemergetree.md). - -**Пример** - -```xml - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - - -## google_protos_path {#google_protos_path} - -Задает каталог, содержащий proto-файлы для типов Protobuf. - -Пример: - -```xml -/usr/share/clickhouse/protos/ -``` - - -## http_handlers {#http_handlers} - -Позволяет использовать пользовательские HTTP-обработчики. -Чтобы добавить новый http-обработчик, просто добавьте новый ``. -Правила проверяются сверху вниз в указанном порядке, -и первый совпавший запускает обработчик. - -Следующие настройки могут быть сконфигурированы с помощью подтегов: - -| Sub-tags | Definition | -| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `url` | Для сопоставления URL запроса можно использовать префикс 'regex:' для сопоставления по регулярному выражению (необязательно) | -| `methods` | Для сопоставления HTTP-методов запроса можно использовать запятые для разделения нескольких методов (необязательно) | -| `headers` | Для сопоставления заголовков запроса сопоставляйте каждый дочерний элемент (имя дочернего элемента — это имя заголовка), можно использовать префикс 'regex:' для сопоставления по регулярному выражению (необязательно) | -| `handler` | Обработчик запроса | -| `empty_query_string` | Проверяет, что в URL отсутствует строка запроса | - -`handler` содержит следующие настройки, которые могут быть сконфигурированы с помощью подтегов: - -| Sub-tags | Definition | -| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `url` | URL-адрес для перенаправления | -| `type` | Поддерживаемые типы: static, dynamic_query_handler, predefined_query_handler, redirect | -| `status` | Используется с типом static, код статуса ответа | -| `query_param_name` | Используется с типом dynamic_query_handler, извлекает и выполняет значение параметра HTTP-запроса, соответствующее `` | -| `query` | Используется с типом predefined_query_handler, выполняет запрос при вызове обработчика | -| `content_type` | Используется с типом static, content-type ответа | -| `response_content` | Используется с типом static, содержимое ответа, отправляемое клиенту; при использовании префикса 'file://' или 'config://' содержимое берётся из файла или конфигурации и отправляется клиенту | - -Помимо списка правил, вы можете указать ``, который включает все обработчики по умолчанию. - -Пример: - -```xml - - - / - POST,GET - no-cache - - dynamic_query_handler - query - - - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.settings - - - - - - static - 200 - text/plain; charset=UTF-8 - config://http_server_default_response - - - -``` - - -## http_server_default_response {#http_server_default_response} - -Страница, которая показывается по умолчанию при обращении к HTTP(S)-серверу ClickHouse. -Значение по умолчанию — "Ok." (с символом новой строки в конце). - -**Пример** - -При обращении к `http://localhost:http_port` открывается `https://tabix.io/`. - -```xml - -
]]> -
-``` - - -## http_options_response {#http_options_response} - -Используется для добавления заголовков к ответу на HTTP-запрос `OPTIONS`. -Метод `OPTIONS` используется при выполнении предварительных CORS-запросов (preflight). - -Для получения дополнительной информации см. [OPTIONS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS). - -Пример: - -```xml - -
- Access-Control-Allow-Origin - * -
-
- Access-Control-Allow-Headers - origin, x-requested-with, x-clickhouse-format, x-clickhouse-user, x-clickhouse-key, Authorization -
-
- Access-Control-Allow-Methods - POST, GET, OPTIONS -
-
- Access-Control-Max-Age - 86400 -
-
-``` - - -## hsts_max_age {#hsts_max_age} - -Срок действия HSTS в секундах. - -:::note -Значение `0` означает, что ClickHouse отключает HSTS. Если указать положительное число, HSTS будет включён, а max-age будет равен этому числу. -::: - -**Пример** - -```xml -600000 -``` - - -## mlock_executable {#mlock_executable} - -Выполнить `mlockall` после запуска, чтобы уменьшить задержку первых запросов и предотвратить выгрузку исполняемого файла ClickHouse в своп при высокой нагрузке на подсистему ввода-вывода (IO). - -:::note -Рекомендуется включить этот параметр, но это приведёт к увеличению времени запуска на несколько секунд. -Имейте в виду, что этот параметр не будет работать без capability "CAP_IPC_LOCK". -::: - -**Пример** - -```xml -false -``` - - -## include_from {#include_from} - -Путь к файлу с подстановками. Поддерживаются форматы XML и YAML. - -Подробнее см. в разделе "[Configuration files](/operations/configuration-files)". - -**Пример** - -```xml -/etc/metrica.xml -``` - - -## interserver_listen_host {#interserver_listen_host} - -Ограничение на хосты, которые могут обмениваться данными между серверами ClickHouse. -Если используется Keeper, то такое же ограничение будет применяться к взаимодействию между различными экземплярами Keeper. - -:::note -По умолчанию значение совпадает с настройкой [`listen_host`](#listen_host). -::: - -**Пример** - -```xml -::ffff:a00:1 -10.0.0.1 -``` - -Тип: - -Значение по умолчанию: - - -## interserver_http_port {#interserver_http_port} - -Порт, используемый для обмена данными между серверами ClickHouse. - -**Пример** - -```xml -9009 -``` - - -## interserver_http_host {#interserver_http_host} - -Имя хоста, которое могут использовать другие серверы для доступа к этому серверу. - -Если параметр не задан, его значение определяется так же, как командой `hostname -f`. - -Полезен, чтобы не зависеть от конкретного сетевого интерфейса. - -**Пример** - -```xml -example.clickhouse.com -``` - - -## interserver_https_port {#interserver_https_port} - -Порт для обмена данными между серверами ClickHouse по `HTTPS`. - -**Пример** - -```xml -9010 -``` - - -## interserver_https_host {#interserver_https_host} - -Аналогично [`interserver_http_host`](#interserver_http_host), за исключением того, что это имя хоста может использоваться другими серверами для доступа к этому серверу по `HTTPS`. - -**Пример** - -```xml -example.clickhouse.com -``` - - -## interserver_http_credentials {#interserver_http_credentials} - -Имя пользователя и пароль, используемые для подключения к другим серверам во время [репликации](../../engines/table-engines/mergetree-family/replication.md). Кроме того, сервер аутентифицирует другие реплики, используя эти учетные данные. -Поэтому `interserver_http_credentials` должны быть одинаковыми для всех реплик в кластере. - -:::note - -* По умолчанию, если секция `interserver_http_credentials` опущена, аутентификация при репликации не используется. -* Настройки `interserver_http_credentials` не связаны с учетными данными клиента ClickHouse в [конфигурации](../../interfaces/cli.md#configuration_files). -* Эти учетные данные являются общими для репликации через `HTTP` и `HTTPS`. - ::: - -Следующие настройки могут быть настроены с помощью подтегов: - -* `user` — Имя пользователя. -* `password` — Пароль. -* `allow_empty` — Если `true`, то другим репликам разрешено подключаться без аутентификации, даже если заданы учетные данные. Если `false`, то подключения без аутентификации отклоняются. Значение по умолчанию: `false`. -* `old` — Содержит старые `user` и `password`, использовавшиеся во время ротации учетных данных. Можно указать несколько секций `old`. - -**Ротация учетных данных** - -ClickHouse поддерживает динамическую ротацию межсерверных учетных данных без одновременной остановки всех реплик для обновления их конфигурации. Учетные данные можно изменить в несколько шагов. - -Чтобы включить аутентификацию, установите `interserver_http_credentials.allow_empty` в `true` и добавьте учетные данные. Это позволит выполнять подключения как с аутентификацией, так и без нее. - -```xml - - admin - 111 - true - -``` - -После того как настроите все реплики, установите параметр `allow_empty` в значение `false` или удалите его. Это сделает аутентификацию с использованием новых учетных данных обязательной. - -Чтобы изменить существующие учетные данные, переместите имя пользователя и пароль в раздел `interserver_http_credentials.old` и задайте новые значения параметрам `user` и `password`. На этом этапе сервер использует новые учетные данные для подключения к другим репликам и принимает подключения как с новыми, так и со старыми учетными данными. - -```xml - - admin - 222 - - admin - 111 - - - temp - 000 - - -``` - -После применения новых учетных данных ко всем репликам старые учетные данные можно удалить. - - -## ldap_servers {#ldap_servers} - -Перечислите здесь LDAP‑серверы с их параметрами подключения, чтобы: -- использовать их как аутентификаторы для отдельных локальных пользователей, у которых вместо механизма аутентификации `password` указан `ldap` -- использовать их как удалённые директории пользователей. - -Следующие настройки могут быть настроены с помощью подтегов: - -| Setting | Description | -|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `host` | Имя хоста или IP‑адрес LDAP‑сервера; этот параметр является обязательным и не может быть пустым. | -| `port` | Порт LDAP‑сервера, по умолчанию `636`, если `enable_tls` установлен в `true`, иначе `389`. | -| `bind_dn` | Шаблон, используемый для построения DN, к которому выполняется привязка. Итоговый DN будет построен путём замены всех подстрок `\{user_name\}` в шаблоне на фактическое имя пользователя при каждой попытке аутентификации. | -| `user_dn_detection` | Раздел с параметрами LDAP‑поиска для определения фактического пользовательского DN привязанного пользователя. В основном используется в фильтрах поиска для последующего сопоставления ролей, когда сервером является Active Directory. Полученный пользовательский DN будет использоваться при замене подстрок `\{user_dn\}` везде, где это разрешено. По умолчанию пользовательский DN устанавливается равным bind DN, но после выполнения поиска он будет обновлён фактически обнаруженным значением пользовательского DN. | -| `verification_cooldown` | Период времени в секундах после успешной попытки привязки, в течение которого пользователь считается успешно аутентифицированным для всех последующих запросов без обращения к LDAP‑серверу. Укажите `0` (значение по умолчанию) для отключения кэширования и принудительного обращения к LDAP‑серверу для каждого запроса аутентификации. | -| `enable_tls` | Флаг, включающий использование защищённого соединения с LDAP‑сервером. Укажите `no` для протокола в открытом виде (`ldap://`) (не рекомендуется). Укажите `yes` для LDAP поверх SSL/TLS (`ldaps://`) (рекомендуется, значение по умолчанию). Укажите `starttls` для устаревшего протокола StartTLS (протокол в открытом виде (`ldap://`), с последующим повышением до TLS). | -| `tls_minimum_protocol_version` | Минимальная версия протокола SSL/TLS. Допустимые значения: `ssl2`, `ssl3`, `tls1.0`, `tls1.1`, `tls1.2` (значение по умолчанию). | -| `tls_require_cert` | Поведение проверки сертификата удалённого узла SSL/TLS. Допустимые значения: `never`, `allow`, `try`, `demand` (значение по умолчанию). | -| `tls_cert_file` | Путь к файлу сертификата. | -| `tls_key_file` | Путь к файлу ключа сертификата. | -| `tls_ca_cert_file` | Путь к файлу сертификата CA (центра сертификации). | -| `tls_ca_cert_dir` | Путь к каталогу, содержащему сертификаты CA. | -| `tls_cipher_suite` | Допустимый набор шифров (в нотации OpenSSL). | - -Настройка `user_dn_detection` может быть настроена с помощью подтегов: - -| Setting | Description | -|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `base_dn` | Шаблон, используемый для построения базового DN для LDAP‑поиска. Итоговый DN будет построен путём замены всех подстрок `\{user_name\}` и `\{bind_dn\}` в шаблоне на фактическое имя пользователя и bind DN во время LDAP‑поиска. | -| `scope` | Область LDAP‑поиска. Допустимые значения: `base`, `one_level`, `children`, `subtree` (значение по умолчанию). | -| `search_filter` | Шаблон, используемый для построения фильтра для LDAP‑поиска. Итоговый фильтр будет построен путём замены всех подстрок `\{user_name\}`, `\{bind_dn\}` и `\{base_dn\}` в шаблоне на фактическое имя пользователя, bind DN и base DN во время LDAP‑поиска. Обратите внимание, что специальные символы должны быть корректно экранированы в XML. | - -Пример: - - - -```xml - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - -``` - -Пример (типичная среда Active Directory с настроенным определением DN пользователя для дальнейшего сопоставления ролей): - -```xml - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - -``` - - -## listen_host {#listen_host} - -Ограничение на хосты, с которых могут поступать запросы. Если вы хотите, чтобы сервер отвечал на запросы со всех хостов, укажите `::`. - -Примеры: - -```xml -::1 -127.0.0.1 -``` - - -## listen_try {#listen_try} - -Сервер не будет завершать работу, если сети IPv6 или IPv4 недоступны при попытке начать прослушивание. - -**Пример** - -```xml -0 -``` - - -## listen_reuse_port {#listen_reuse_port} - -Позволяет нескольким серверам прослушивать один и тот же адрес:порт. Запросы будут направляться операционной системой на случайный сервер. Включать этот параметр не рекомендуется. - -**Пример** - -```xml -0 -``` - -Тип: - -Значение по умолчанию: - - -## listen_backlog {#listen_backlog} - -Backlog (размер очереди ожидающих подключений) listen-сокета. Значение по умолчанию `4096` совпадает со значением для Linux [5.4+](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19f92a030ca6d772ab44b22ee6a01378a8cb32d4)). - -Обычно это значение не требуется менять, поскольку: - -* значение по умолчанию достаточно велико; -* для принятия клиентских подключений у сервера есть отдельный поток. - -Поэтому даже если у вас `TcpExtListenOverflows` (из `nstat`) имеет ненулевое значение и этот счётчик растёт для сервера ClickHouse, это не означает, что это значение нужно увеличивать, поскольку: - -* обычно, если `4096` недостаточно, это указывает на внутреннюю проблему масштабирования ClickHouse, поэтому лучше сообщить о проблеме; -* это не означает, что сервер сможет обработать больше подключений позже (и даже если сможет, к тому времени клиенты уже могут пропасть или отключиться). - -**Пример** - -```xml -4096 -``` - - -## logger {#logger} - -Расположение и формат сообщений журнала. - -**Ключи**: - -| Key | Description | -|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `level` | Уровень логирования. Допустимые значения: `none` (отключить логирование), `fatal`, `critical`, `error`, `warning`, `notice`, `information`, `debug`, `trace`, `test` | -| `log` | Путь к файлу журнала. | -| `errorlog` | Путь к файлу журнала ошибок. | -| `size` | Политика ротации: максимальный размер файлов журнала в байтах. После превышения этого порога файл журнала переименовывается и архивируется, а затем создаётся новый файл журнала. | -| `count` | Политика ротации: максимальное количество исторических файлов журнала ClickHouse, которое будет сохраняться. | -| `stream_compress` | Сжимать сообщения журнала с использованием LZ4. Установите `1` или `true`, чтобы включить. | -| `console` | Включить логирование в консоль. Установите `1` или `true`, чтобы включить. По умолчанию — `1`, если ClickHouse не запущен в режиме демона, иначе — `0`. | -| `console_log_level` | Уровень логирования для вывода в консоль. По умолчанию используется значение `level`. | -| `formatting.type` | Формат логов для вывода в консоль. В настоящее время поддерживается только `json`. | -| `use_syslog` | Дополнительно перенаправлять вывод журнала в syslog. | -| `syslog_level` | Уровень логирования для записи в syslog. | -| `async` | При значении `true` (по умолчанию) логирование выполняется асинхронно (по одному фоновому потоку на каждый канал вывода). В противном случае запись идёт в потоке, вызывающем LOG. | -| `async_queue_max_size` | При использовании асинхронного логирования — максимальное количество сообщений, которые будут храниться в очереди в ожидании сброса. Лишние сообщения будут отброшены. | -| `startup_level` | Уровень при запуске используется для установки уровня корневого логгера при старте сервера. После запуска уровень логирования возвращается к значению параметра `level`. | -| `shutdown_level` | Уровень при завершении работы используется для установки уровня корневого логгера при остановке сервера. | - -**Спецификаторы формата журнала** - -Имена файлов в путях `log` и `errorLog` поддерживают приведённые ниже спецификаторы формата для результирующего имени файла (часть пути, соответствующая каталогу, их не поддерживает). - -Столбец «Example» показывает вывод для `2023-07-06 18:32:07`. - - - -| Спецификатор | Описание | Пример | -| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| `%%` | Символ % как литерал | `%` | -| `%n` | Символ новой строки | | -| `%t` | Символ горизонтальной табуляции | | -| `%Y` | Год в десятичном формате, например 2017 | `2023` | -| `%y` | Последние 2 цифры года как десятичное число (диапазон [00, 99]) | `23` | -| `%C` | Первые две цифры года в виде десятичного числа (диапазон [00, 99]) | `20` | -| `%G` | Четырёхзначный [год по ISO 8601, основанный на неделях](https://en.wikipedia.org/wiki/ISO_8601#Week_dates), то есть год, к которому относится указанная неделя. Обычно используется только совместно с `%V` | `2023` | -| `%g` | Последние две цифры [недельного года по ISO 8601](https://en.wikipedia.org/wiki/ISO_8601#Week_dates), то есть года, к которому относится указанная неделя. | `23` | -| `%b` | Сокращённое название месяца, например Oct (в зависимости от локали) | `Июл` | -| `%h` | Синоним для %b | `июл` | -| `%B` | Полное название месяца, например Октябрь (зависит от локали) | `Июль` | -| `%m` | Месяц в десятичном формате (диапазон [01, 12]) | `07` | -| `%U` | Номер недели года как десятичное число (воскресенье — первый день недели) (диапазон [00,53]) | `27` | -| `%W` | Номер недели года в виде десятичного числа (понедельник — первый день недели) (диапазон [00,53]) | `27` | -| `%V` | Номер недели по ISO 8601 (диапазон [01,53]) | `27` | -| `%j` | День года в виде десятичного числа (диапазон [001,366]) | `187` | -| `%d` | День месяца в виде десятичного числа с ведущим нулём (диапазон [01, 31]). Однозначные значения дополняются ведущим нулём. | `06` | -| `%e` | День месяца в виде десятичного числа с добавлением ведущего пробела (диапазон [1, 31]). Однозначные значения предваряются пробелом. | `  6` | -| `%a` | Сокращённое название дня недели, например «Fri» (зависит от локали) | `Чт` | -| `%A` | Полное название дня недели, например «Friday» (зависит от локали) | `четверг` | -| `%w` | День недели как целое число, где воскресенье — 0 (диапазон от 0 до 6) | `4` | -| `%u` | День недели как десятичное число, где понедельник — 1 (формат ISO 8601) (диапазон [1–7]) | `4` | -| `%H` | Часы в виде десятичного числа, 24-часовой формат (диапазон [00–23]) | `18` | -| `%I` | Час в виде десятичного числа в 12-часовом формате (диапазон [01,12]) | `06` | -| `%M` | Минута в виде десятичного числа (в диапазоне [00, 59]) | `32` | -| `%S` | Секунда в десятичном формате (диапазон [00,60]) | `07` | -| `%c` | Стандартная строка даты и времени, например, Sun Oct 17 04:41:13 2010 (формат зависит от локали) | `Thu Jul 6 18:32:07 2023` | -| `%x` | Локализованный формат даты (зависит от локали) | `06.07.23` | -| `%X` | Локализованный формат времени, например 18:40:20 или 6:40:20 PM (в зависимости от локали) | `18:32:07` | -| `%D` | Краткая дата в формате MM/DD/YY, эквивалентная %m/%d/%y | `07.06.23` | -| `%F` | Краткий формат даты YYYY-MM-DD, эквивалентный %Y-%m-%d | `2023-07-06` | -| `%r` | Локализованное время в 12‑часовом формате (зависит от настроек локали) | `06:32:07 PM` | -| `%R` | Эквивалентно "%H:%M" | `18:32` | -| `%T` | Эквивалентно «%H:%M:%S» (формат времени ISO 8601) | `18:32:07` | -| `%p` | Локализованное обозначение a.m./p.m. (зависит от локали) | `PM` | -| `%z` | Смещение от UTC в формате ISO 8601 (например, -0430) или пустая строка, если сведения о часовом поясе недоступны | `+0800` | -| `%Z` | Локализованное название или аббревиатура часового пояса, либо пустая строка, если информация о часовом поясе недоступна | `Z AWST ` | - -**Пример** - -```xml - - trace - /var/log/clickhouse-server/clickhouse-server-%F-%T.log - /var/log/clickhouse-server/clickhouse-server-%F-%T.err.log - 1000M - 10 - true - -``` - -Чтобы выводить лог‑сообщения только в консоль: - -```xml - - information - true - -``` - -**Переопределения по уровням** - -Можно переопределить уровень логирования для отдельных логгеров. Например, чтобы отключить все сообщения логгеров "Backup" и "RBAC". - -```xml - - - - Backup - none - - - RBAC - none - - - -``` - -**syslog** - -Чтобы дополнительно отправлять журнальные сообщения в syslog: - -```xml - - 1 - -
syslog.remote:10514
- myhost.local - LOG_LOCAL6 - syslog -
-
-``` - -Keys for ``: - -| Key | Description | -| ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `address` | Адрес сервера syslog в формате `host\[:port\]`. Если не указан, используется локальный демон. | -| `hostname` | Имя хоста, с которого отправляются логи (необязательный параметр). | -| `facility` | [Ключевое слово facility](https://en.wikipedia.org/wiki/Syslog#Facility) для syslog. Должно быть указано в верхнем регистре с префиксом `LOG_`, например: `LOG_USER`, `LOG_DAEMON`, `LOG_LOCAL3` и т. д. По умолчанию: `LOG_USER`, если указан `address`, иначе `LOG_DAEMON`. | -| `format` | Формат лог-сообщений. Возможные значения: `bsd` и `syslog`. | - -**Форматы логов** - -Вы можете указать формат логов, который будет выводиться в консоль. В настоящее время поддерживается только JSON. - -**Пример** - -Ниже приведён пример выходного JSON-лога: - -```json -{ - "date_time_utc": "2024-11-06T09:06:09Z", - "date_time": "1650918987.180175", - "thread_name": "#1", - "thread_id": "254545", - "level": "Trace", - "query_id": "", - "logger_name": "BaseDaemon", - "message": "Получен сигнал 2", - "source_file": "../base/daemon/BaseDaemon.cpp; virtual void SignalListener::run()", - "source_line": "192" -} -``` - -Чтобы включить логирование в формате JSON, используйте следующий фрагмент: - -```xml - - - json - - - - date_time - thread_name - thread_id - level - query_id - logger_name - message - source_file - source_line - - - -``` - -**Переименование ключей JSON‑логов** - -Имена ключей можно изменить, задав другие значения тегов внутри тега ``. Например, чтобы заменить `DATE_TIME` на `MY_DATE_TIME`, можно использовать `MY_DATE_TIME`. - -**Пропуск ключей JSON‑логов** - -Свойства лога можно опустить, закомментировав соответствующее свойство. Например, если вы не хотите, чтобы в логе выводился `query_id`, вы можете закомментировать тег ``. - - -## send_crash_reports {#send_crash_reports} - -Настройки отправки отчётов о сбоях команде разработчиков ядра ClickHouse. - -Включение этой функции, особенно в предпродакшн-средах, крайне приветствуется. - -Ключи: - -| Key | Description | -| --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | -| `enabled` | Логический флаг для включения функции, по умолчанию `true`. Установите `false`, чтобы запретить отправку отчётов о сбоях. | -| `send_logical_errors` | `LOGICAL_ERROR` похож на `assert`: это ошибка (баг) в ClickHouse. Этот логический флаг включает отправку таких исключений (по умолчанию: `true`). | -| `endpoint` | Позволяет переопределить URL конечной точки для отправки отчётов о сбоях. | - -**Рекомендуемое использование** - -```xml - - true - -``` - - -## ssh_server {#ssh_server} - -Публичная часть ключа хоста будет записана в файл known_hosts -на стороне SSH-клиента при первом подключении. - -Параметры ключа хоста по умолчанию отключены. -Раскомментируйте параметры ключа хоста и укажите путь к соответствующему ssh-ключу, чтобы их активировать: - -Пример: - -```xml - - path_to_the_ssh_key - path_to_the_ssh_key - path_to_the_ssh_key - -``` - - -## tcp_ssh_port {#tcp_ssh_port} - -Порт SSH-сервера, позволяющий пользователю подключаться и выполнять запросы в интерактивном режиме с помощью встроенного клиента через PTY. - -Пример: - -```xml -9022 -``` - - -## storage_configuration {#storage_configuration} - -Поддерживает многодисковую конфигурацию хранилища. - -Конфигурация хранилища имеет следующую структуру: - -```xml - - - - - - - - -``` - -### Конфигурация дисков {#configuration-of-disks} - -Конфигурация раздела `disks` имеет следующую структуру: - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - ... - - -``` - -Вложенные теги выше определяют следующие настройки для `disks`: - -| Параметр | Описание | -| ----------------------- | --------------------------------------------------------------------------------------------------------------- | -| `` | Имя диска, которое должно быть уникальным. | -| `path` | Путь, по которому будут храниться данные сервера (каталоги `data` и `shadow`). Должен оканчиваться символом `/` | -| `keep_free_space_bytes` | Размер зарезервированного свободного пространства на диске. | - -:::note -Порядок дисков не имеет значения. -::: - -### Конфигурация политик {#configuration-of-policies} - -Вложенные теги выше определяют следующие настройки для `policies`: - - -| Setting | Description | -|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `policy_name_N` | Имя политики. Имена политик должны быть уникальными. | -| `volume_name_N` | Имя тома. Имена томов должны быть уникальными. | -| `disk` | Диск, расположенный внутри тома. | -| `max_data_part_size_bytes` | Максимальный размер части данных, которая может находиться на любом из дисков в этом томе. Если результат слияния приводит к тому, что ожидаемый размер части превышает `max_data_part_size_bytes`, эта часть будет записана в следующий том. По сути, эта возможность позволяет хранить новые / небольшие части на «горячем» томе (SSD) и перемещать их на «холодный» том (HDD), когда они достигают большого размера. Не используйте эту опцию, если в политике только один том. | -| `move_factor` | Доля доступного свободного пространства на томе. Если свободного места становится меньше, данные начинают переноситься на следующий том, если он существует. Для переноса части сортируются по размеру от большей к меньшей (по убыванию) и выбираются те части, суммарный размер которых достаточен для выполнения условия `move_factor`; если суммарный размер всех частей недостаточен, будут перенесены все части. | -| `perform_ttl_move_on_insert` | Отключает перенос данных с истекшим TTL при вставке. По умолчанию (если включено), если вставляется фрагмент данных, срок жизни которого уже истёк согласно правилу переноса по времени жизни, он немедленно переносится в том / на диск, указанный в правиле переноса. Это может существенно замедлить вставку, если целевой том / диск медленный (например, S3). Если отключено, просроченная часть данных записывается в том по умолчанию и затем сразу переносится в том, указанный в правиле для истекшего TTL. | -| `load_balancing` | Политика балансировки дисков: `round_robin` или `least_used`. | -| `least_used_ttl_ms` | Устанавливает тайм-аут (в миллисекундах) обновления доступного пространства на всех дисках (`0` — всегда обновлять, `-1` — никогда не обновлять, значение по умолчанию — `60000`). Обратите внимание: если диск используется только ClickHouse и не будет подвергаться «на лету» изменению размера файловой системы, вы можете использовать значение `-1`. Во всех остальных случаях это не рекомендуется, так как в итоге приведёт к некорректному распределению пространства. | -| `prefer_not_to_merge` | Отключает слияние частей данных на этом томе. Примечание: это потенциально вредно и может вызывать замедление. При включении этого параметра (не делайте так) слияние данных на этом томе запрещено (что плохо). Это позволяет управлять тем, как ClickHouse работает с медленными дисками. Мы рекомендуем вообще не использовать этот параметр. | -| `volume_priority` | Определяет приоритет (порядок), в котором заполняются тома. Чем меньше значение, тем выше приоритет. Значения параметра должны быть натуральными числами и непрерывно покрывать диапазон от 1 до N (N — наибольшее указанное значение параметра) без пропусков. | - -Для `volume_priority`: -- Если у всех томов задан этот параметр, они получают приоритет в указанном порядке. -- Если он есть только у _некоторых_ томов, тома без него имеют наименьший приоритет. Тома, у которых параметр задан, получают приоритет в соответствии со значением параметра, приоритет остальных определяется их порядком описания в конфигурационном файле относительно друг друга. -- Если _ни одному_ тому этот параметр не задан, их порядок определяется порядком описания в конфигурационном файле. -- Приоритеты томов могут отличаться друг от друга. - - - -## macros {#macros} - -Подстановки параметров для реплицируемых таблиц. - -Можно опустить, если реплицируемые таблицы не используются. - -Подробнее см. раздел [Creating replicated tables](../../engines/table-engines/mergetree-family/replication.md#creating-replicated-tables). - -**Пример** - -```xml - -``` - - -## replica_group_name {#replica_group_name} - -Имя группы реплик для базы данных Replicated. - -Кластер, созданный базой данных Replicated, будет состоять из реплик одной и той же группы. -DDL-запросы будут ожидать только реплики в той же группе. - -По умолчанию — пустое значение. - -**Пример** - -```xml -backups -``` - - -## remap_executable {#remap_executable} - -Настройка для перераспределения памяти под машинный код («text») с использованием больших страниц. - -:::note -Эта возможность является экспериментальной. -::: - -Пример: - -```xml -false -``` - - -## max_open_files {#max_open_files} - -Максимальное количество открытых файлов. - -:::note -Рекомендуем использовать этот параметр в macOS, поскольку функция `getrlimit()` возвращает некорректное значение. -::: - -**Пример** - -```xml -262144 -``` - - -## max_session_timeout {#max_session_timeout} - -Максимальное время сеанса, в секундах. - -Пример: - -```xml -3600 -``` - - -## merge_tree {#merge_tree} - -Тонкая настройка таблиц [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -Для получения более подробной информации см. заголовочный файл MergeTreeSettings.h. - -**Пример** - -```xml - - 5 - -``` - - -## metric_log {#metric_log} - -По умолчанию он отключён. - -**Включение** - -Чтобы вручную включить сбор истории метрик в [`system.metric_log`](../../operations/system-tables/metric_log.md), создайте файл `/etc/clickhouse-server/config.d/metric_log.xml` со следующим содержимым: - -```xml - - - system - metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**Отключение** - -Чтобы отключить параметр `metric_log`, необходимо создать следующий файл `/etc/clickhouse-server/config.d/disable_metric_log.xml` со следующим содержимым: - -```xml - - - -``` - - - - -## replicated_merge_tree {#replicated_merge_tree} - -Тонкая настройка для таблиц в [ReplicatedMergeTree](../../engines/table-engines/mergetree-family/mergetree.md). Этот параметр имеет более высокий приоритет. - -Для получения дополнительной информации см. заголовочный файл MergeTreeSettings.h. - -**Пример** - -```xml - - 5 - -``` - - -## opentelemetry_span_log {#opentelemetry_span_log} - -Настройки системной таблицы [`opentelemetry_span_log`](../system-tables/opentelemetry_span_log.md). - - - -Пример: - -```xml - - - engine MergeTree - partition by toYYYYMM(finish_date) - order by (finish_date, finish_time_us, trace_id) - - system - opentelemetry_span_log
- 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## openSSL {#openSSL} - -Конфигурация SSL для клиента и сервера. - -Поддержка SSL реализована с использованием библиотеки `libpoco`. Доступные параметры конфигурации описаны в [SSLManager.h](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/SSLManager.h). Значения по умолчанию можно найти в [SSLManager.cpp](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/src/SSLManager.cpp). - -Ключи настроек сервера и клиента: - - - -| Опция | Описание | Значение по умолчанию | -| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------ | -| `privateKeyFile` | Путь к файлу с закрытым ключом PEM-сертификата. В одном файле могут одновременно находиться и ключ, и сертификат. | | -| `certificateFile` | Путь к файлу клиентского/серверного сертификата в формате PEM. Можно не указывать, если `privateKeyFile` содержит сертификат. | | -| `caConfig` | Путь к файлу или каталогу, содержащему доверенные сертификаты УЦ (CA). Если указан файл, он должен быть в PEM-формате и может содержать несколько сертификатов УЦ. Если указан каталог, он должен содержать по одному файлу с расширением .pem на каждый сертификат УЦ. Имена файлов определяются по хэшу имени субъекта УЦ. Подробности можно найти на странице руководства man для [SSL_CTX_load_verify_locations](https://www.openssl.org/docs/man3.0/man3/SSL_CTX_load_verify_locations.html). | | -| `verificationMode` | Метод проверки сертификатов узла. Подробности см. в описании класса [Context](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/Context.h). Возможные значения: `none`, `relaxed`, `strict`, `once`. | `relaxed` | -| `verificationDepth` | Максимально допустимая длина цепочки проверки сертификатов. Проверка завершится с ошибкой, если длина цепочки сертификатов превышает заданное значение. | `9` | -| `loadDefaultCAFile` | Определяет, будут ли использоваться встроенные сертификаты удостоверяющих центров (CA) для OpenSSL. ClickHouse предполагает, что встроенные сертификаты CA находятся в файле `/etc/ssl/cert.pem` (или в каталоге `/etc/ssl/certs`), либо в файле (или каталоге), заданном переменной окружения `SSL_CERT_FILE` (соответственно, `SSL_CERT_DIR`). | `true` | -| `cipherList` | Поддерживаемые алгоритмы шифрования OpenSSL. | `ALL:!ADH:!LOW:!EXP:!MD5:!3DES:@STRENGTH` | -| `cacheSessions` | Включает или отключает кэширование сеансов. Используется совместно с `sessionIdContext`. Допустимые значения: `true`, `false`. | `false` | -| `sessionIdContext` | Уникальный набор случайных символов, который сервер добавляет к каждому сгенерированному идентификатору. Длина строки не должна превышать `SSL_MAX_SSL_SESSION_ID_LENGTH`. Всегда рекомендуется задавать этот параметр, так как он помогает избежать проблем как в случае, когда сервер кэширует сеанс, так и если клиент запросил кэширование. | `$\{application.name\}` | -| `sessionCacheSize` | Максимальное количество сеансов, которые кеширует сервер. Значение `0` означает неограниченное количество сеансов. | [1024*20](https://github.com/ClickHouse/boringssl/blob/master/include/openssl/ssl.h#L1978) | -| `sessionTimeout` | Время хранения сеанса в кэше на сервере (в часах). | `2` | -| `extendedVerification` | Если параметр включён, убедитесь, что CN или SAN сертификата совпадает с именем узла-пира. | `false` | -| `requireTLSv1` | Требовать соединение по протоколу TLSv1. Допустимые значения: `true`, `false`. | `false` | -| `requireTLSv1_1` | Требовать установления соединения по TLSv1.1. Допустимые значения: `true`, `false`. | `false` | -| `requireTLSv1_2` | Требовать соединение по TLSv1.2. Допустимые значения: `true`, `false`. | `false` | -| `fips` | Активирует режим OpenSSL FIPS. Поддерживается, если версия OpenSSL, используемая библиотекой, поддерживает FIPS. | `false` | -| `privateKeyPassphraseHandler` | Класс (подкласс PrivateKeyPassphraseHandler), предназначенный для запроса парольной фразы для доступа к закрытому ключу. Например: ``, `KeyFileHandler`, `test`, ``. | `KeyConsoleHandler` | -| `invalidCertificateHandler` | Класс (подкласс CertificateHandler) для обработки недействительных сертификатов. Например: ` RejectCertificateHandler ` . | `RejectCertificateHandler` | -| `disableProtocols` | Протоколы, которые запрещено использовать. | | -| `preferServerCiphers` | Серверные шифры по выбору клиента. | `false` | - -**Пример настроек:** - -```xml - - - - /etc/clickhouse-server/server.crt - /etc/clickhouse-server/server.key - - /etc/clickhouse-server/dhparam.pem - none - true - true - sslv2,sslv3 - true - - - true - true - sslv2,sslv3 - true - - - - RejectCertificateHandler - - - -``` - - -## part_log {#part_log} - -Логирование событий, связанных с [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md), например добавления или слияния данных. Вы можете использовать журнал для моделирования алгоритмов слияния и сравнения их характеристик, а также для визуализации процесса слияния. - -Запросы логируются в таблицу [system.part_log](/operations/system-tables/part_log), а не в отдельный файл. Имя этой таблицы можно настроить с помощью параметра `table` (см. ниже). - - - -**Пример** - -```xml - - system - part_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## path {#path} - -Путь к каталогу, содержащему данные. - -:::note -Обязателен завершающий слэш. -::: - -**Пример** - -```xml -/var/lib/clickhouse/ -``` - - -## processors_profile_log {#processors_profile_log} - -Настройки для системной таблицы [`processors_profile_log`](../system-tables/processors_profile_log.md). - - - -Параметры по умолчанию: - -```xml - - system - processors_profile_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## prometheus {#prometheus} - -Экспорт метрик для опроса из [Prometheus](https://prometheus.io). - -Параметры: - -* `endpoint` – HTTP-эндпоинт для сбора метрик сервером Prometheus. Должен начинаться с '/'. -* `port` – Порт для `endpoint`. -* `metrics` – Экспорт метрик из таблицы [system.metrics](/operations/system-tables/metrics). -* `events` – Экспорт метрик из таблицы [system.events](/operations/system-tables/events). -* `asynchronous_metrics` – Экспорт текущих значений метрик из таблицы [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics). -* `errors` – Экспорт количества ошибок по кодам ошибок, произошедших с момента последнего перезапуска сервера. Эту информацию можно также получить из таблицы [system.errors](/operations/system-tables/errors). - -**Пример** - -```xml - - 0.0.0.0 - 8123 - 9000 - - - /metrics - 9363 - true - true - true - true - - - -``` - -Проверьте (замените `127.0.0.1` на IP-адрес или имя хоста вашего сервера ClickHouse): - -```bash -curl 127.0.0.1:9363/metrics -``` - - -## query_log {#query_log} - -Настройка для логирования запросов при включённой опции [log_queries=1](../../operations/settings/settings.md). - -Запросы логируются в таблицу [system.query_log](/operations/system-tables/query_log), а не в отдельный файл. Имя таблицы можно изменить с помощью параметра `table` (см. ниже). - - - -Если таблица не существует, ClickHouse создаст её. Если структура журнала запросов изменилась при обновлении сервера ClickHouse, таблица со старой структурой будет переименована, а новая таблица будет создана автоматически. - -**Пример** - -```xml - - system - query_log
- Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_metric_log {#query_metric_log} - -По умолчанию он отключён. - -**Включение** - -Чтобы вручную включить сбор истории метрик [`system.query_metric_log`](../../operations/system-tables/query_metric_log.md), создайте файл `/etc/clickhouse-server/config.d/query_metric_log.xml` со следующим содержимым: - -```xml - - - system - query_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**Отключение** - -Чтобы отключить настройку `query_metric_log`, необходимо создать файл `/etc/clickhouse-server/config.d/disable_query_metric_log.xml` со следующим содержимым: - -```xml - - - -``` - - - - -## query_cache {#query_cache} - -Конфигурация [кеша запросов](../query-cache.md). - -Доступны следующие настройки: - -| Setting | Description | Default Value | -| ------------------------- | --------------------------------------------------------------------------------------------------- | ------------- | -| `max_size_in_bytes` | Максимальный размер кеша в байтах. Значение `0` означает, что кеш запросов отключён. | `1073741824` | -| `max_entries` | Максимальное количество результатов запросов `SELECT`, хранящихся в кеше. | `1024` | -| `max_entry_size_in_bytes` | Максимальный размер в байтах результатов запросов `SELECT`, которые могут быть сохранены в кеше. | `1048576` | -| `max_entry_size_in_rows` | Максимальное количество строк в результатах запросов `SELECT`, которые могут быть сохранены в кеше. | `30000000` | - -:::note - -* Изменённые настройки вступают в силу немедленно. -* Данные для кеша запросов размещаются в DRAM. Если объём памяти ограничен, установите небольшое значение `max_size_in_bytes` или полностью отключите кеш запросов. - ::: - -**Пример** - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - - -## query_thread_log {#query_thread_log} - -Настройка параметров логирования потоков запросов, включаемого параметром [log_query_threads=1](/operations/settings/settings#log_query_threads). - -Запросы логируются в таблицу [system.query_thread_log](/operations/system-tables/query_thread_log), а не в отдельный файл. Имя таблицы можно изменить в параметре `table` (см. ниже). - - - -Если таблица не существует, ClickHouse создаст её. Если структура журнала потоков запросов изменилась при обновлении сервера ClickHouse, таблица со старой структурой переименовывается, и новая таблица создаётся автоматически. - -**Пример** - -```xml - - system - query_thread_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_views_log {#query_views_log} - -Настройка для логирования представлений (live, materialized и т. д.), зависящая от запросов, выполняемых при включённой настройке [log_query_views=1](/operations/settings/settings#log_query_views). - -Запросы записываются в таблицу [system.query_views_log](/operations/system-tables/query_views_log), а не в отдельный файл. Вы можете изменить имя таблицы с помощью параметра `table` (см. ниже). - - - -Если таблицы не существует, ClickHouse создаст её. Если структура журнала представлений запросов изменилась после обновления сервера ClickHouse, таблица со старой структурой будет переименована, а новая таблица будет создана автоматически. - -**Пример** - -```xml - - system - query_views_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## text_log {#text_log} - -Настройки системной таблицы [text_log](/operations/system-tables/text_log) для журналирования текстовых сообщений. - - - -Дополнительно: - -| Параметр | Описание | Значение по умолчанию | -| -------- | ------------------------------------------------------------------------------------------- | --------------------- | -| `level` | Максимальный уровень сообщений (по умолчанию `Trace`), которые будут сохраняться в таблице. | `Trace` | - -**Пример** - -```xml - - - notice - system - text_log
- 7500 - 1048576 - 8192 - 524288 - false - - Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day -
-
-``` - - -## trace_log {#trace_log} - -Параметры системной таблицы [trace_log](/operations/system-tables/trace_log). - - - -Конфигурационный файл сервера по умолчанию `config.xml` содержит следующий раздел настроек: - -```xml - - system - trace_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false - false -
-``` - - -## asynchronous_insert_log {#asynchronous_insert_log} - -Параметры системной таблицы [asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log), отвечающей за журналирование асинхронных вставок. - - - -**Пример** - -```xml - - - system - asynchronous_insert_log
- 7500 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## crash_log {#crash_log} - -Настройки для работы системной таблицы [crash_log](../../operations/system-tables/crash_log.md). - -Следующие настройки могут быть заданы с помощью подтегов: - -| Setting | Description | Default | Note | -| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------- | -| `database` | Имя базы данных. | | | -| `table` | Имя системной таблицы. | | | -| `engine` | [Определение движка MergeTree](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-creating-a-table) для системной таблицы. | | Нельзя использовать, если определены `partition_by` или `order_by`. Если не указано, по умолчанию выбирается `MergeTree`. | -| `partition_by` | [Пользовательский ключ партиционирования](/engines/table-engines/mergetree-family/custom-partitioning-key.md) для системной таблицы. | | Если для системной таблицы указан `engine`, параметр `partition_by` должен быть указан непосредственно внутри `engine`. | -| `ttl` | Задает [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) таблицы. | | Если для системной таблицы указан `engine`, параметр `ttl` должен быть указан непосредственно внутри `engine`. | -| `order_by` | [Пользовательский ключ сортировки](/engines/table-engines/mergetree-family/mergetree#order_by) для системной таблицы. Нельзя использовать, если определен `engine`. | | Если для системной таблицы указан `engine`, параметр `order_by` должен быть указан непосредственно внутри `engine`. | -| `storage_policy` | Имя политики хранения, используемой для таблицы (необязательно). | | Если для системной таблицы указан `engine`, параметр `storage_policy` должен быть указан непосредственно внутри `engine`. | -| `settings` | [Дополнительные параметры](/engines/table-engines/mergetree-family/mergetree/#settings), управляющие поведением MergeTree (необязательно). | | Если для системной таблицы указан `engine`, параметр `settings` должен быть указан непосредственно внутри `engine`. | -| `flush_interval_milliseconds` | Интервал сброса данных из буфера в памяти в таблицу. | `7500` | | -| `max_size_rows` | Максимальный размер логов в строках. Когда количество несброшенных логов достигает `max_size_rows`, логи сбрасываются на диск. | `1024` | | -| `reserved_size_rows` | Предварительно выделенный объем памяти в строках для логов. | `1024` | | -| `buffer_size_rows_flush_threshold` | Порог по количеству строк. Если порог достигнут, в фоновом режиме запускается сброс логов на диск. | `max_size_rows / 2` | | -| `flush_on_crash` | Определяет, должны ли логи сбрасываться на диск в случае сбоя. | `false` | | - -Файл конфигурации сервера по умолчанию `config.xml` содержит следующий раздел настроек: - -```xml - - system - crash_log
- toYYYYMM(event_date) - 7500 - 1024 - 1024 - 512 - false -
-``` - - -## custom_cached_disks_base_directory {#custom_cached_disks_base_directory} - -Этот параметр задает путь к кэшу для пользовательских дисков (созданных через SQL). -`custom_cached_disks_base_directory` имеет более высокий приоритет для пользовательских дисков по сравнению с `filesystem_caches_path` (указанным в `filesystem_caches_path.xml`), -который используется, если первая настройка отсутствует. -Путь, заданный в настройке файлового кэша, должен находиться внутри этого каталога, -в противном случае будет выброшено исключение, предотвращающее создание диска. - -:::note -Это не влияет на диски, созданные в более старой версии, с которой был обновлен сервер. -В этом случае исключение выброшено не будет, чтобы сервер смог успешно запуститься. -::: - -Пример: - -```xml -/var/lib/clickhouse/caches/ -``` - - -## backup_log {#backup_log} - -Настройки системной таблицы [backup_log](../../operations/system-tables/backup_log.md) для ведения журнала операций `BACKUP` и `RESTORE`. - - - -**Пример** - -```xml - - - system - backup_log
- 1000 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## blob_storage_log {#blob_storage_log} - -Параметры системной таблицы [`blob_storage_log`](../system-tables/blob_storage_log.md). - - - -Пример: - -```xml - - systemblob_storage_logtoYYYYMM(event_date) - 7500event_date + INTERVAL 30 DAY - -``` - - -## query_masking_rules {#query_masking_rules} - -Правила на основе регулярных выражений, которые применяются к запросам, а также ко всем сообщениям журнала перед их сохранением в серверные логи, -таблицы [`system.query_log`](/operations/system-tables/query_log), [`system.text_log`](/operations/system-tables/text_log), [`system.processes`](/operations/system-tables/processes) и в логи, отправляемые клиенту. Это позволяет предотвратить утечку конфиденциальных данных из SQL-запросов, таких как имена, адреса электронной почты, персональные идентификаторы или номера кредитных карт, в журналы. - -**Пример** - -```xml - - - скрыть номер SSN - (^|\D)\d{3}-\d{2}-\d{4}($|\D) - 000-00-0000 - - -``` - -**Поля конфигурации**: - -| Setting | Description | -| --------- | -------------------------------------------------------------------------------------------- | -| `name` | имя правила (необязательно) | -| `regexp` | регулярное выражение, совместимое с RE2 (обязательно) | -| `replace` | строка подстановки для чувствительных данных (необязательно, по умолчанию — шесть звездочек) | - -Правила маскирования применяются ко всему запросу (чтобы предотвратить утечки чувствительных данных из некорректных / неразбираемых запросов). - -В таблице [`system.events`](/operations/system-tables/events) есть счётчик `QueryMaskingRulesMatch`, который показывает общее количество срабатываний правил маскирования запросов. - -Для распределённых запросов каждый сервер должен быть настроен отдельно, иначе подзапросы, передаваемые на другие узлы, будут сохраняться без маскирования. - - -## remote_servers {#remote_servers} - -Конфигурация кластеров, используемых движком таблиц [Distributed](../../engines/table-engines/special/distributed.md) и табличной функцией `cluster`. - -**Пример** - -```xml - -``` - -Сведения о значении атрибута `incl` см. в разделе «[Файлы конфигурации](/operations/configuration-files)». - -**См. также** - -* [skip_unavailable_shards](../../operations/settings/settings.md#skip_unavailable_shards) -* [Обнаружение кластеров](../../operations/cluster-discovery.md) -* [Движок реплицируемой базы данных](../../engines/database-engines/replicated.md) - - -## remote_url_allow_hosts {#remote_url_allow_hosts} - -Список хостов, которые разрешено использовать в движках хранения и табличных функциях, работающих с URL. - -При добавлении хоста с помощью XML-тега `\`: - -* он должен быть указан в точности так же, как в URL, поскольку имя проверяется до разрешения DNS-имени. Например: `clickhouse.com` -* если порт явно указан в URL, то host:port проверяется как единое целое. Например: `clickhouse.com:80` -* если хост указан без порта, то разрешён любой порт этого хоста. Например: если указан `clickhouse.com`, то `clickhouse.com:20` (FTP), `clickhouse.com:80` (HTTP), `clickhouse.com:443` (HTTPS) и т. д. разрешены. -* если хост указан как IP-адрес, то он проверяется так, как указан в URL. Например: `[2a02:6b8:a::a]`. -* если есть перенаправления и поддержка перенаправлений включена, то каждое перенаправление (заголовок Location) проверяется. - -Например: - -```sql - - clickhouse.com - -``` - - -## timezone {#timezone} - -Часовой пояс сервера. - -Указывается как идентификатор IANA для часового пояса UTC или географического региона (например, Africa/Abidjan). - -Часовой пояс необходим для преобразований между форматами String и DateTime, когда поля DateTime выводятся в текстовый формат (на экран или в файл), а также при получении значения DateTime из строки. Кроме того, часовой пояс используется в функциях, которые работают с датой и временем, если часовой пояс не был передан во входных параметрах. - -**Пример** - -```xml -Asia/Istanbul -``` - -**См. также** - -* [session_timezone](../settings/settings.md#session_timezone) - - -## tcp_port {#tcp_port} - -Порт для взаимодействия с клиентами по протоколу TCP. - -**Пример** - -```xml -9000 -``` - - -## tcp_port_secure {#tcp_port_secure} - -TCP-порт для защищённого взаимодействия с клиентами. Используйте его вместе с настройками [OpenSSL](#openssl). - -**Значение по умолчанию** - -```xml -9440 -``` - - -## mysql_port {#mysql_port} - -Порт для взаимодействия с клиентами по протоколу MySQL. - -:::note - -* Положительные целые значения задают номер порта для прослушивания. -* Пустое значение используется для отключения взаимодействия с клиентами по протоколу MySQL. - ::: - -**Пример** - -```xml -9004 -``` - - -## postgresql_port {#postgresql_port} - -Порт для взаимодействия с клиентами по протоколу PostgreSQL. - -:::note - -* Положительные целые числа задают номер порта, который необходимо прослушивать. -* Пустые значения используются, чтобы отключить взаимодействие с клиентами по протоколу PostgreSQL. - ::: - -**Пример** - -```xml -9005 -``` - - -## mysql_require_secure_transport {#mysql_require_secure_transport} - -Если параметр установлен в значение true, при взаимодействии с клиентами через [mysql_port](#mysql_port) требуется защищённое соединение. Подключения с опцией `--ssl-mode=none` будут отклоняться. Используйте вместе с настройками [OpenSSL](#openssl). - - - -## postgresql_require_secure_transport {#postgresql_require_secure_transport} - -Если установлено значение true, для клиентов через [postgresql_port](#postgresql_port) требуется защищённое соединение. Соединения с опцией `sslmode=disable` будут отклоняться. Используйте вместе с настройками [OpenSSL](#openssl). - - - -## tmp_path {#tmp_path} - -Путь в локальной файловой системе для хранения временных данных при обработке больших запросов. - -:::note - -* Для настройки хранилища временных данных можно использовать только один из параметров: `tmp_path`, `tmp_policy`, `temporary_data_in_cache`. -* Конечный слэш (/) обязателен. - ::: - -**Пример** - -```xml -/var/lib/clickhouse/tmp/ -``` - - -## url_scheme_mappers {#url_scheme_mappers} - -Настройка преобразования сокращённых или символических URL-префиксов в полные URL. - -Пример: - -```xml - - - https://{bucket}.s3.amazonaws.com - - - https://storage.googleapis.com/{bucket} - - - https://{bucket}.oss.aliyuncs.com - - -``` - - -## user_files_path {#user_files_path} - -Каталог пользовательских файлов. Используется в табличных функциях [file()](../../sql-reference/table-functions/file.md) и [fileCluster()](../../sql-reference/table-functions/fileCluster.md). - -**Пример** - -```xml -/var/lib/clickhouse/user_files/ -``` - - -## user_scripts_path {#user_scripts_path} - -Каталог, содержащий файлы пользовательских скриптов. Используется для исполняемых пользовательских функций [Executable User Defined Functions](/sql-reference/functions/udf#executable-user-defined-functions). - -**Пример** - -```xml -/var/lib/clickhouse/user_scripts/ -``` - -Тип: - -Значение по умолчанию: - - -## user_defined_path {#user_defined_path} - -Каталог с пользовательскими файлами. Используется для пользовательских SQL-функций [SQL User Defined Functions](/sql-reference/functions/udf). - -**Пример** - -```xml -/var/lib/clickhouse/user_defined/ -``` - - -## users_config {#users_config} - -Путь к файлу, содержащему: - -* Конфигурации пользователей. -* Права доступа. -* Профили настроек. -* Настройки квот. - -**Пример** - -```xml -users.xml -``` - - -## access_control_improvements {#access_control_improvements} - -Настройки для дополнительных (необязательных) улучшений в системе управления доступом. - -| Setting | Description | Default | -| ----------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| `users_without_row_policies_can_read_rows` | Определяет, могут ли пользователи без разрешающих политик по строкам читать строки с помощью запроса `SELECT`. Например, если есть два пользователя A и B, и политика по строкам определена только для A, то при значении параметра `true` пользователь B увидит все строки. При значении `false` пользователь B не увидит ни одной строки. | `true` | -| `on_cluster_queries_require_cluster_grant` | Определяет, требуется ли для запросов с `ON CLUSTER` привилегия `CLUSTER`. | `true` | -| `select_from_system_db_requires_grant` | Определяет, требует ли запрос `SELECT * FROM system.` каких-либо прав и может ли он выполняться любым пользователем. Если установлено значение `true`, этот запрос требует `GRANT SELECT ON system.
` так же, как и для несистемных таблиц. Исключения: несколько системных таблиц (`tables`, `columns`, `databases` и некоторые константные таблицы, такие как `one`, `contributors`) остаются доступными для всех; кроме того, если выдана привилегия `SHOW` (например, `SHOW USERS`), соответствующая системная таблица (то есть `system.users`) будет доступна. | `true` | -| `select_from_information_schema_requires_grant` | Определяет, требует ли запрос `SELECT * FROM information_schema.
` каких-либо прав и может ли он выполняться любым пользователем. Если установлено значение `true`, этот запрос требует `GRANT SELECT ON information_schema.
` так же, как и для обычных таблиц. | `true` | -| `settings_constraints_replace_previous` | Определяет, будет ли ограничение в профиле настроек для некоторой настройки отменять действие предыдущего ограничения (определённого в других профилях) для этой настройки, включая поля, которые не заданы новым ограничением. Также включает тип ограничения `changeable_in_readonly`. | `true` | -| `table_engines_require_grant` | Определяет, требуется ли привилегия для создания таблицы с конкретным движком таблицы. | `false` | -| `role_cache_expiration_time_seconds` | Определяет количество секунд с момента последнего доступа, в течение которых роль хранится в кэше ролей (Role Cache). | `600` | - -Пример: - -```xml - - true - true - true - true - true - false - 600 - -``` - - -## s3queue_log {#s3queue_log} - -Настройки системной таблицы `s3queue_log`. - - - -Настройки по умолчанию следующие: - -```xml - - system -
s3queue_log
- toYYYYMM(event_date) - 7500 - -``` - - -## dead_letter_queue {#dead_letter_queue} - -Настройка системной таблицы 'dead_letter_queue'. - - - -Параметры по умолчанию: - -```xml - - system - dead_letter
- toYYYYMM(event_date) - 7500 -
-``` - - -## zookeeper {#zookeeper} - -Содержит настройки, которые позволяют ClickHouse взаимодействовать с кластером [ZooKeeper](http://zookeeper.apache.org/). ClickHouse использует ZooKeeper для хранения метаданных реплик при использовании реплицируемых таблиц. Если реплицируемые таблицы не используются, этот раздел параметров можно опустить. - -Следующие настройки могут быть заданы с помощью подтегов: - -| Setting | Description | -| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `node` | Конечная точка ZooKeeper. Можно задать несколько endpoints. Например, `example_host2181`. Атрибут `index` задаёт порядок обхода узлов при попытке подключиться к кластеру ZooKeeper. | -| `session_timeout_ms` | Максимальное время ожидания для клиентской сессии в миллисекундах. | -| `operation_timeout_ms` | Максимальное время ожидания для одной операции в миллисекундах. | -| `root` (optional) | znode, который используется как корневой для znode, используемых сервером ClickHouse. | -| `fallback_session_lifetime.min` (optional) | Минимальное ограничение времени жизни сессии ZooKeeper к резервному узлу, когда основной недоступен (балансировка нагрузки). Указывается в секундах. Значение по умолчанию: 3 часа. | -| `fallback_session_lifetime.max` (optional) | Максимальное ограничение времени жизни сессии ZooKeeper к резервному узлу, когда основной недоступен (балансировка нагрузки). Указывается в секундах. Значение по умолчанию: 6 часов. | -| `identity` (optional) | Имя пользователя и пароль, требуемые ZooKeeper для доступа к запрашиваемым znode. | -| `use_compression` (optional) | Включает сжатие в протоколе Keeper, если установлено значение true. | - -Также существует настройка `zookeeper_load_balancing` (необязательная), которая позволяет выбрать алгоритм выбора узла ZooKeeper: - -| Algorithm Name | Description | -| ------------------------------- | ------------------------------------------------------------------------------------------------------------------- | -| `random` | случайным образом выбирает один из узлов ZooKeeper. | -| `in_order` | выбирает первый узел ZooKeeper, если он недоступен — второй и так далее. | -| `nearest_hostname` | выбирает узел ZooKeeper с именем хоста, наиболее похожим на имя хоста сервера; имена сравниваются по префиксу. | -| `hostname_levenshtein_distance` | так же, как `nearest_hostname`, но сравнивает имена хостов по расстоянию Левенштейна. | -| `first_or_random` | выбирает первый узел ZooKeeper, если он недоступен — случайным образом выбирает один из оставшихся узлов ZooKeeper. | -| `round_robin` | выбирает первый узел ZooKeeper, при переподключении выбирает следующий. | - -**Пример конфигурации** - -```xml - - - example1 - 2181 - - - example2 - 2181 - - 30000 - 10000 - - /path/to/zookeeper/node - - user:password - - random - -``` - -**См. также** - -* [Репликация](../../engines/table-engines/mergetree-family/replication.md) -* [Руководство программиста ZooKeeper](http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html) -* [Необязательное защищённое взаимодействие между ClickHouse и ZooKeeper](/operations/ssl-zookeeper) - - -## use_minimalistic_part_header_in_zookeeper {#use_minimalistic_part_header_in_zookeeper} - -Способ хранения заголовков частей данных в ZooKeeper. Этот параметр применяется только к семейству [`MergeTree`](/engines/table-engines/mergetree-family). Его можно задать: - -**Глобально в разделе [merge_tree](#merge_tree) файла `config.xml`** - -ClickHouse использует этот параметр для всех таблиц на сервере. Вы можете изменить его в любой момент. Поведение существующих таблиц изменится при изменении значения параметра. - -**Для каждой таблицы** - -При создании таблицы укажите соответствующую [настройку движка](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table). Поведение существующей таблицы с уже заданным значением этого параметра не меняется, даже если глобальная настройка изменена. - -**Возможные значения** - -- `0` — Функция отключена. -- `1` — Функция включена. - -Если [`use_minimalistic_part_header_in_zookeeper = 1`](#use_minimalistic_part_header_in_zookeeper), то [реплицируемые](../../engines/table-engines/mergetree-family/replication.md) таблицы хранят заголовки частей данных компактно, используя один `znode`. Если таблица содержит много столбцов, этот способ хранения значительно уменьшает объём данных, хранимых в ZooKeeper. - -:::note -После применения `use_minimalistic_part_header_in_zookeeper = 1` вы не сможете понизить версию сервера ClickHouse до версии, которая не поддерживает этот параметр. Будьте осторожны при обновлении ClickHouse на серверах в кластере. Не обновляйте все серверы одновременно. Безопаснее тестировать новые версии ClickHouse в тестовой среде или только на нескольких серверах кластера. - -Заголовки частей данных, уже сохранённые с включённой настройкой, нельзя восстановить в их прежнее (некомпактное) представление. -::: - - - -## distributed_ddl {#distributed_ddl} - -Управляет выполнением [распределённых DDL-запросов](../../sql-reference/distributed-ddl.md) (`CREATE`, `DROP`, `ALTER`, `RENAME`) на кластере. -Работает только при включённом [ZooKeeper](/operations/server-configuration-parameters/settings#zookeeper). - -Настраиваемые параметры в разделе `` включают: - -| Setting | Description | Default Value | -| ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | -| `path` | путь в Keeper к `task_queue` для DDL-запросов | | -| `profile` | профиль, используемый для выполнения DDL-запросов | | -| `pool_size` | сколько запросов `ON CLUSTER` может выполняться одновременно | | -| `max_tasks_in_queue` | максимальное количество задач, которое может находиться в очереди. | `1,000` | -| `task_max_lifetime` | удалять узел, если его возраст превышает это значение. | `7 * 24 * 60 * 60` (неделя в секундах) | -| `cleanup_delay_period` | очистка запускается после получения события о новом узле, если с момента последней очистки прошло не менее `cleanup_delay_period` секунд. | `60` секунд | - -**Пример** - -```xml - - - /clickhouse/task_queue/ddl - - - default - - - 1 - - - - - 604800 - - - 60 - - - 1000 - -``` - - -## access_control_path {#access_control_path} - -Путь к папке, в которой сервер ClickHouse хранит конфигурации пользователей и ролей, созданные SQL-командами. - -**См. также** - -- [Управление доступом и учетными записями](/operations/access-rights#access-control-usage) - - - -## allow_plaintext_password {#allow_plaintext_password} - -Задает, разрешено ли использование небезопасных паролей в открытом виде (plaintext-password). - -```xml -1 -``` - - -## allow_no_password {#allow_no_password} - -Определяет, допускается ли использование небезопасного типа пароля `no_password`. - -```xml -1 -``` - - -## allow_implicit_no_password {#allow_implicit_no_password} - -Запрещает создание пользователя без пароля, если явно не указано 'IDENTIFIED WITH no_password'. - -```xml -1 -``` - - -## default_session_timeout {#default_session_timeout} - -Тайм-аут сеанса по умолчанию, в секундах. - -```xml -60 -``` - - -## default_password_type {#default_password_type} - -Задает тип пароля, который будет автоматически устанавливаться в запросах вида `CREATE USER u IDENTIFIED BY 'p'`. - -Допустимые значения: - -* `plaintext_password` -* `sha256_password` -* `double_sha1_password` -* `bcrypt_password` - -```xml -sha256_password -``` - - -## user_directories {#user_directories} - -Раздел файла конфигурации, содержащий настройки: - -* Путь к файлу конфигурации с предопределёнными пользователями. -* Путь к папке, где хранятся пользователи, создаваемые с помощью SQL-команд. -* Путь к узлу ZooKeeper, где хранятся и реплицируются пользователи, создаваемые с помощью SQL-команд. - -Если этот раздел задан, путь из [users_config](/operations/server-configuration-parameters/settings#users_config) и [access_control_path](../../operations/server-configuration-parameters/settings.md#access_control_path) не используется. - -Раздел `user_directories` может содержать любое количество элементов, порядок элементов определяет их приоритет (чем выше элемент, тем выше приоритет). - -**Примеры** - -```xml - - - /etc/clickhouse-server/users.xml - - - /var/lib/clickhouse/access/ - - -``` - -Пользователей, роли, политики на уровне строк, квоты и профили также можно хранить в ZooKeeper: - -```xml - - - /etc/clickhouse-server/users.xml - - - /clickhouse/access/ - - -``` - -Вы также можете определить секции `memory` — это означает, что информация хранится только в памяти, без записи на диск, и `ldap` — это означает хранение информации на сервере LDAP. - -Чтобы добавить сервер LDAP в качестве удалённого каталога для пользователей, не определённых локально, задайте единственную секцию `ldap` со следующими настройками: - -| Setting | Description | -| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `server` | одно из имён серверов LDAP, определённых в секции конфигурации `ldap_servers`. Этот параметр является обязательным и не может быть пустым. | -| `roles` | секция со списком локально определённых ролей, которые будут назначаться каждому пользователю, полученному с сервера LDAP. Если роли не указаны, пользователь не сможет выполнять какие-либо действия после аутентификации. Если какая-либо из перечисленных ролей не определена локально на момент аутентификации, попытка аутентификации завершится неудачей так, как если бы был указан неверный пароль. | - -**Пример** - -```xml - - my_ldap_server - - - - - -``` - - -## top_level_domains_list {#top_level_domains_list} - -Определяет список пользовательских доменов верхнего уровня, в котором каждая запись имеет формат `/path/to/file`. - -Например: - -```xml - - /path/to/public_suffix_list.dat - -``` - -См. также: - -* функцию [`cutToFirstSignificantSubdomainCustom`](../../sql-reference/functions/url-functions.md/#cutToFirstSignificantSubdomainCustom) и её варианты, - которая принимает имя пользовательского списка TLD и возвращает часть домена, включающую домены верхнего уровня и поддомены вплоть до первого значимого поддомена. - - -## proxy {#proxy} - -Определите прокси‑серверы для HTTP‑ и HTTPS‑запросов, которые в настоящее время поддерживаются хранилищем S3, табличными функциями S3 и функциями URL. - -Существуют три способа определения прокси‑серверов: - -* переменные окружения -* списки прокси -* удалённые резолверы прокси. - -Обход прокси‑серверов для конкретных хостов также поддерживается с помощью `no_proxy`. - -**Переменные окружения** - -Переменные окружения `http_proxy` и `https_proxy` позволяют указать -прокси‑сервер для заданного протокола. Если они настроены в вашей системе, всё должно работать без дополнительной настройки. - -Это самый простой подход, если для заданного протокола используется -только один прокси‑сервер и этот прокси‑сервер не меняется. - -**Списки прокси** - -Этот подход позволяет указать один или несколько -прокси‑серверов для протокола. Если определено более одного прокси‑сервера, -ClickHouse использует разные прокси‑серверы по круговой схеме (round-robin), распределяя -нагрузку между серверами. Это самый простой подход, если для протокола больше -одного прокси‑сервера и список прокси‑серверов не меняется. - -**Шаблон конфигурации** - -```xml - - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - -Выберите родительское поле на вкладках ниже, чтобы просмотреть их дочерние поля: - - - - | Поле | Описание | - | --------- | -------------------------------------------- | - | `` | Список из одного или нескольких HTTP-прокси | - | `` | Список из одного или нескольких HTTPS-прокси | - - - - | Поле | Описание | - | ------- | ------------------ | - | `` | URI прокси-сервера | - - - -**Удалённые резолверы прокси** - -Прокси-серверы могут изменяться динамически. В этом -случае вы можете определить конечную точку резолвера. ClickHouse отправляет -пустой GET-запрос на эту конечную точку, удалённый резолвер должен вернуть хост прокси. -ClickHouse будет использовать его для формирования URI прокси по следующему шаблону: `\{proxy_scheme\}://\{proxy_host\}:{proxy_port}` - -**Шаблон конфигурации** - -```xml - - - - http://resolver:8080/hostname - http - 80 - 10 - - - - - - http://resolver:8080/hostname - http - 3128 - 10 - - - - -``` - -Выберите родительское поле во вкладках ниже, чтобы просмотреть соответствующие дочерние поля: - - - - | Поле | Описание | - | --------- | ------------------------------------------- | - | `` | Список из одного или нескольких резолверов* | - | `` | Список из одного или нескольких резолверов* | - - - - | Поле | Описание | - | ------------ | ------------------------------------------------------- | - | `` | Конечная точка (endpoint) и другие сведения о резолвере | - - :::note - Можно указать несколько элементов ``, но используется только первый - `` для данного протокола. Любые другие элементы `` - для этого протокола игнорируются. Это означает, что балансировка нагрузки - (если она требуется) должна быть реализована на стороне удалённого резолвера. - ::: - - - - | Поле | Описание | - | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | - | `` | URI прокси-резолвера | - | `` | Протокол итогового URI прокси. Может быть `http` или `https`. | - | `` | Номер порта прокси-резолвера | - | `` | Время в секундах, в течение которого значения от резолвера должны кэшироваться в ClickHouse. Установка значения `0` заставляет ClickHouse обращаться к резолверу для каждого HTTP- или HTTPS-запроса. | - - - -**Приоритет** - -Параметры прокси определяются в следующем порядке: - - -| Порядок | Параметр | -|---------|-----------------------------| -| 1. | Удалённые прокси‑резолверы | -| 2. | Списки прокси‑серверов | -| 3. | Переменные окружения | - -ClickHouse проверит тип резолвера с наивысшим приоритетом для протокола запроса. Если он не определён, -будет проверен следующий по приоритету тип резолвера, пока не будет достигнут резолвер на основе переменных окружения. -Это также позволяет одновременно использовать несколько типов резолверов. - - - -## отключение_туннелирования_HTTPS_запросов_через_HTTP_прокси {#disable_tunneling_for_https_requests_over_http_proxy} - -По умолчанию туннелирование (т. е. `HTTP CONNECT`) используется для выполнения `HTTPS`‑запросов через `HTTP`‑прокси. Этот параметр можно использовать, чтобы отключить его. - -**no_proxy** - -По умолчанию все запросы проходят через прокси. Чтобы отключить его для отдельных хостов, необходимо задать переменную `no_proxy`. -Её можно задать внутри блока `` для резолверов типов list и remote, а также в виде переменной окружения для резолвера типа environment. -Поддерживаются IP-адреса, домены, поддомены и шаблон `'*'` для полного обхода. Ведущие точки удаляются так же, как это делает curl. - -**Example** - -Ниже приведена конфигурация, которая обходит прокси для запросов к `clickhouse.cloud` и ко всем его поддоменам (например, `auth.clickhouse.cloud`). -То же самое относится к GitLab, даже если домен указан с ведущей точкой: и `gitlab.com`, и `about.gitlab.com` будут обходить прокси. - -```xml - - clickhouse.cloud,.gitlab.com - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - - -## workload_path {#workload_path} - -Каталог, используемый для хранения всех запросов `CREATE WORKLOAD` и `CREATE RESOURCE`. По умолчанию используется каталог `/workload/` в рабочем каталоге сервера. - -**Пример** - -```xml -/var/lib/clickhouse/workload/ -``` - -**См. также** - -* [Иерархия рабочих нагрузок](/operations/workload-scheduling.md#workloads) -* [workload_zookeeper_path](#workload_zookeeper_path) - - -## workload_zookeeper_path {#workload_zookeeper_path} - -Путь к узлу ZooKeeper, который используется в качестве хранилища для всех запросов `CREATE WORKLOAD` и `CREATE RESOURCE`. Для единообразия все SQL-определения хранятся в виде значения этого единственного `znode`. По умолчанию ZooKeeper не используется, и определения сохраняются на [диск](#workload_path). - -**Пример** - -```xml -/clickhouse/workload/definitions.sql -``` - -**См. также** - -* [Иерархия рабочих нагрузок](/operations/workload-scheduling.md#workloads) -* [workload_path](#workload_path) - - -## zookeeper_log {#zookeeper_log} - -Настройки для системной таблицы [`zookeeper_log`](/operations/system-tables/zookeeper_log). - -Следующие настройки можно настроить с помощью подтегов: - - - -**Пример** - -```xml - - - system - zookeeper_log
- 7500 - event_date + INTERVAL 1 WEEK DELETE -
-
-``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md deleted file mode 100644 index fc5f351beeb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md +++ /dev/null @@ -1,17 +0,0 @@ -Следующие параметры можно настроить с помощью подтегов: - -| Setting | Description | Default | Note | -|------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|--------------------------------------------------------------------------------------------------------------------| -| `database` | Имя базы данных. | | | -| `table` | Имя системной таблицы. | | | -| `engine` | [Определение движка MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table) для системной таблицы. | | Не может использоваться, если определены `partition_by` или `order_by`. Если не указан, по умолчанию выбирается `MergeTree` | -| `partition_by` | [Пользовательский ключ партиционирования](../../../engines/table-engines/mergetree-family/custom-partitioning-key.md) для системной таблицы. | | Если для системной таблицы указан `engine`, параметр `partition_by` должен быть указан непосредственно внутри `engine` | -| `ttl` | Задает [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl) таблицы. | | Если для системной таблицы указан `engine`, параметр `ttl` должен быть указан непосредственно внутри `engine` | -| `order_by` | [Пользовательский ключ сортировки](../../../engines/table-engines/mergetree-family/mergetree.md#order_by) для системной таблицы. Не может использоваться, если определен `engine`. | | Если для системной таблицы указан `engine`, параметр `order_by` должен быть указан непосредственно внутри `engine` | -| `storage_policy` | Имя политики хранения, используемой для таблицы (необязательно). | | Если для системной таблицы указан `engine`, параметр `storage_policy` должен быть указан непосредственно внутри `engine` | -| `settings` | [Дополнительные параметры](../../../engines/table-engines/mergetree-family/mergetree.md/#settings), управляющие поведением MergeTree (необязательно). | | Если для системной таблицы указан `engine`, параметр `settings` должен быть указан непосредственно внутри `engine` | -| `flush_interval_milliseconds` | Интервал сброса данных из буфера в памяти в таблицу. | `7500` | | -| `max_size_rows` | Максимальный размер логов в строках. Когда объем несброшенных логов достигает `max_size_rows`, логи сбрасываются на диск. | `1048576` | | -| `reserved_size_rows` | Предварительно выделенный размер памяти в строках для логов. | `8192` | | -| `buffer_size_rows_flush_threshold` | Порог по количеству строк. Если порог достигнут, в фоновом режиме запускается сброс логов на диск. | `max_size_rows / 2` | | -| `flush_on_crash` | Определяет, должны ли логи сбрасываться на диск в случае сбоя. | `false` | | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md deleted file mode 100644 index 64f11455481..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md +++ /dev/null @@ -1,14 +0,0 @@ ---- -description: 'Этот раздел содержит описания настроек сервера, то есть настроек, - которые не могут быть изменены на уровне сеанса или запроса.' -keywords: ['глобальные настройки сервера'] -sidebar_label: 'Настройки сервера' -sidebar_position: 57 -slug: /operations/server-configuration-parameters/settings -title: 'Настройки сервера' -doc_type: 'reference' ---- - -{/* ПРИМЕЧАНИЕ: настройки в этом файле сгенерированы автоматически. - Для получения дополнительной информации см. ["Generating documentation from source code"](https://github.com/ClickHouse/clickhouse-docs/blob/main/contribute/autogenerated-documentation-from-source.md) - */ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md deleted file mode 100644 index df489a64bf4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md +++ /dev/null @@ -1,199 +0,0 @@ ---- -description: 'Составные протоколы позволяют более гибко настраивать TCP-доступ - к серверу ClickHouse.' -sidebar_label: 'Составные протоколы' -sidebar_position: 64 -slug: /operations/settings/composable-protocols -title: 'Составные протоколы' -doc_type: 'reference' ---- - -# Компонуемые протоколы {#composable-protocols} - -## Обзор {#overview} - -Составные протоколы позволяют более гибко настраивать TCP‑доступ к -серверу ClickHouse. Эта конфигурация может использоваться параллельно с традиционной -конфигурацией или полностью её заменять. - -## Настройка компонуемых протоколов {#composable-protocols-section-is-denoted-as-protocols-in-configuration-xml} - -Компонуемые протоколы настраиваются в конфигурационном XML-файле. Раздел протоколов -обозначается тегами `protocols` в XML-файле конфигурации: - -```xml - - - -``` - - -### Настройка уровней протокола {#basic-modules-define-protocol-layers} - -Вы можете задавать уровни протокола с помощью базовых модулей. Например, чтобы задать -уровень HTTP, вы можете добавить новый базовый модуль в раздел `protocols`: - -```xml - - - - - http - - - -``` - -Модули можно настраивать по следующим параметрам: - -* `plain_http` - имя, на которое может ссылаться другой слой -* `type` - обозначает обработчик протокола, который будет создан для обработки данных. - Поддерживаются следующие предопределённые обработчики протоколов: - * `tcp` - собственный обработчик протокола ClickHouse - * `http` - HTTP-обработчик протокола ClickHouse - * `tls` - слой шифрования TLS - * `proxy1` - слой PROXYv1 - * `mysql` - обработчик протокола совместимости с MySQL - * `postgres` - обработчик протокола совместимости с PostgreSQL - * `prometheus` - обработчик протокола Prometheus - * `interserver` - межсерверный обработчик ClickHouse - -:::note -Обработчик протокола `gRPC` не реализован для `Composable protocols`. -::: - - -### Настройка конечных точек {#endpoint-ie-listening-port-is-denoted-by-port-and-optional-host-tags} - -Конечные точки (прослушивающие порты) задаются с помощью тегов `` и -необязательного ``. -Например, чтобы настроить конечную точку на ранее добавленном HTTP-слое, мы -можем изменить нашу конфигурацию следующим образом: - -```xml - - - - - http - - 127.0.0.1 - 8123 - - - - -``` - -Если тег `` опущен, используется `` из корневой конфигурации. - - -### Настройка последовательностей слоёв {#layers-sequence-is-defined-by-impl-tag-referencing-another-module} - -Последовательности слоёв задаются с помощью тега ``, ссылающегося на другой -модуль. Например, чтобы настроить слой TLS поверх нашего модуля plain_http, -мы могли бы дополнительно изменить конфигурацию следующим образом: - -```xml - - - - - http - - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### Привязка конечных точек к слоям {#endpoint-can-be-attached-to-any-layer} - -Конечные точки можно привязывать к любому слою. Например, мы можем определить конечные точки для -HTTP (порт 8123) и HTTPS (порт 8443): - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### Определение дополнительных конечных точек {#additional-endpoints-can-be-defined-by-referencing-any-module-and-omitting-type-tag} - -Дополнительные конечные точки можно задать, ссылаясь на любой модуль и опуская -тег ``. Например, мы можем задать конечную точку `another_http` для -модуля `plain_http` следующим образом: - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - - plain_http - 127.0.0.1 - 8223 - - - -``` - - -### Указание дополнительных параметров слоя {#some-modules-can-contain-specific-for-its-layer-parameters} - -Некоторые модули могут иметь дополнительные параметры слоя. Например, слой TLS -позволяет задать закрытый ключ (`privateKeyFile`) и файлы сертификатов (`certificateFile`) -следующим образом: - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - another_server.key - another_server.crt - - - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md deleted file mode 100644 index 04c74c3ce75..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md +++ /dev/null @@ -1,201 +0,0 @@ ---- -description: 'Ограничения для настроек можно задать в разделе `profiles` конфигурационного файла `user.xml`, чтобы запретить пользователям изменять некоторые из параметров с помощью запроса `SET`.' -sidebar_label: 'Ограничения для настроек' -sidebar_position: 62 -slug: /operations/settings/constraints-on-settings -title: 'Ограничения для настроек' -doc_type: 'reference' ---- - - - -# Ограничения настроек {#constraints-on-settings} - - - -## Обзор {#overview} - -В ClickHouse под «ограничениями настроек» понимаются ограничения и правила, -которые можно назначать этим настройкам. Эти ограничения можно применять для -поддержания стабильности, безопасности и предсказуемого поведения вашей базы данных. - - - -## Определение ограничений {#defining-constraints} - -Ограничения для настроек можно задать в разделе `profiles` файла конфигурации `user.xml`. -Они запрещают пользователям изменять некоторые настройки с помощью оператора -[`SET`](/sql-reference/statements/set). - -Ограничения задаются следующим образом: - -```xml - - - - - нижняя_граница - - - верхняя_граница - - - нижняя_граница - верхняя_граница - - - - - - нижняя_граница - верхняя_граница - - - - нижняя_граница - верхняя_граница - значение1 - значение2 - значение3 - - - - - -``` - -Если пользователь пытается нарушить ограничения, выбрасывается исключение, а -настройка остаётся без изменений. - - -## Типы ограничений {#types-of-constraints} - -В ClickHouse поддерживается несколько типов ограничений: - -* `min` -* `max` -* `disallowed` -* `readonly` (с псевдонимом `const`) -* `changeable_in_readonly` - -Ограничения `min` и `max` задают нижнюю и верхнюю границы для числовой -настройки и могут использоваться совместно. - -Ограничение `disallowed` можно использовать для указания конкретных значений, -которые являются недопустимыми для определённой настройки. - -Ограничение `readonly` или `const` указывает, что пользователь не может -изменять соответствующую настройку вообще. - -Тип ограничения `changeable_in_readonly` позволяет пользователям изменять -настройку в пределах диапазона `min`/`max`, даже если настройка `readonly` -установлена в значение `1`; в противном случае настройки нельзя изменять в режиме -`readonly=1`. - -:::note -`changeable_in_readonly` поддерживается только если включён -`settings_constraints_replace_previous`: - -```xml - - true - -``` - -::: - - -## Несколько профилей ограничений {#multiple-constraint-profiles} - -Если для пользователя активно несколько профилей, ограничения объединяются. -Процесс объединения зависит от `settings_constraints_replace_previous`: -- **true** (рекомендуется): ограничения для одной и той же настройки заменяются - при объединении так, что используется последнее ограничение, а все предыдущие - игнорируются. Это касается также полей, которые не заданы в новом - ограничении. -- **false** (по умолчанию): ограничения для одной и той же настройки - объединяются так, что каждый тип ограничения, который не задан, берётся из - предыдущего профиля, а каждый заданный тип ограничения заменяется значением - из нового профиля. - - - -## Режим только для чтения {#read-only} - -Режим только для чтения включается настройкой `readonly`, которую не следует путать -с типом ограничения `readonly`: - -* `readonly=0`: Нет ограничений: разрешены и чтение, и запись. -* `readonly=1`: Разрешены только запросы на чтение, и настройки нельзя изменять, - если только не установлен `changeable_in_readonly`. -* `readonly=2`: Разрешены только запросы на чтение, но настройки можно изменять, - за исключением самой настройки `readonly`. - -### Пример {#example-read-only} - -Пусть `users.xml` содержит следующие строки: - -```xml - - - 10000000000 - 0 - ... - - - 5000000000 - 20000000000 - - - - - - - -``` - -Все приведённые ниже запросы вызовут исключения: - -```sql -SET max_memory_usage=20000000001; -SET max_memory_usage=4999999999; -SET force_index_by_date=1; -``` - -```text -Code: 452, e.displayText() = DB::Exception: Значение настройки max_memory_usage не должно превышать 20000000000. -Code: 452, e.displayText() = DB::Exception: Значение настройки max_memory_usage не должно быть меньше 5000000000. -Code: 452, e.displayText() = DB::Exception: Настройка force_index_by_date не должна изменяться. -``` - -:::note -Профиль `default` обрабатывается особым образом: все ограничения, заданные для -профиля `default`, становятся ограничениями по умолчанию и, соответственно, применяются ко всем пользователям, -пока для конкретных пользователей они явно не будут переопределены. -::: - - -## Ограничения для настроек MergeTree {#constraints-on-merge-tree-settings} - -Можно задать ограничения для [настроек MergeTree](merge-tree-settings.md). -Эти ограничения применяются при создании таблицы с движком MergeTree -или при изменении её настроек хранения. - -Имя настройки MergeTree должно начинаться с префикса `merge_tree_` при -ссылке на неё в разделе ``. - -### Пример {#example-mergetree} - -Вы можете запретить создание новых таблиц, в которых явно задан `storage_policy`. - -```xml - - - - - - - - - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/index.md deleted file mode 100644 index 44237b4cf34..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: 'Страница оглавления для раздела «Settings»' -sidebar_position: 1 -slug: /operations/settings/ -title: 'Settings' -doc_type: 'landing-page' ---- - -{/* Таблица оглавления для этой страницы автоматически генерируется скриптом - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - из полей YAML frontmatter: slug, description, title. - - Если вы заметили ошибку, отредактируйте фронтматтер YML на самих страницах. - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [Обзор настроек](/operations/settings/overview) | Обзорная страница настроек. | -| [Права доступа для запросов](/operations/settings/permissions-for-queries) | Настройки прав доступа для запросов. | -| [Ограничения на сложность запросов](/operations/settings/query-complexity) | Настройки, ограничивающие сложность запросов. | -| [Профили настроек](/operations/settings/settings-profiles) | Набор настроек, объединённых общим именем. | -| [Ограничения на настройки](/operations/settings/constraints-on-settings) | Ограничения на настройки могут быть заданы в секции `profiles` файла конфигурации `user.xml` и запрещают пользователям изменять некоторые настройки с помощью запроса `SET`. | -| [Настройки пользователей и ролей](/operations/settings/settings-users) | Настройки пользователей и ролей. | -| [Композиционные протоколы](/operations/settings/composable-protocols) | Композиционные протоколы позволяют более гибко настраивать TCP-доступ к серверу ClickHouse. | -| [Настройки форматов](/operations/settings/formats) | Настройки, связанные с форматами. | -| [Память с перераспределением (overcommit)](/operations/settings/memory-overcommit) | Экспериментальный механизм, позволяющий задавать более гибкие лимиты памяти для запросов. | -| [Настройки таблиц MergeTree](/operations/settings/merge-tree-settings) | Настройки MergeTree, которые хранятся в `system.merge_tree_settings`. | -| [Сессионные настройки на уровне запроса](/operations/settings/query-level) | Настройки на уровне запроса. | -| [Перегрузка сервера](/operations/settings/server-overload) | Управление поведением сервера при перегрузке CPU. | -| [Сессионные настройки](/operations/settings/settings) | Сессионные настройки, задаваемые через `system.settings`. | -| [Ограничения на TCP-подключения](/operations/settings/tcp-connection-limits) | Ограничения на TCP-подключения. | - -{/*AUTOGENERATED_START*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md deleted file mode 100644 index 11229acf100..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'Экспериментальный механизм, позволяющий более гибко настраивать ограничения памяти при выполнении запросов.' -slug: /operations/settings/memory-overcommit -title: 'Оверкоммит памяти' -doc_type: 'reference' ---- - - - -# Оверкоммит памяти (memory overcommit) {#memory-overcommit} - -Оверкоммит памяти — это экспериментальный механизм, предназначенный для более гибкой настройки лимитов памяти для запросов. - -Идея этого механизма состоит во введении настроек, которые задают гарантированный объём памяти, доступный запросу. -Когда оверкоммит памяти включён и лимит памяти достигнут, ClickHouse выберет запрос с наибольшим перерасходом памяти и попытается освободить память, принудительно завершив этот запрос. - -Когда лимит памяти достигнут, любой запрос будет ждать некоторое время при попытке выделить новую память. -Если по истечении тайм‑аута память будет освобождена, запрос продолжит выполнение. -В противном случае будет сгенерировано исключение, и запрос будет принудительно завершён. - -Выбор запроса для остановки или завершения выполняется либо глобальным, либо пользовательским трекером оверкоммита, в зависимости от того, какой лимит памяти был достигнут. -Если трекер оверкоммита не может выбрать запрос для остановки, генерируется исключение MEMORY_LIMIT_EXCEEDED. - - - -## Отслеживание перерасхода памяти пользователем {#user-overcommit-tracker} - -Модуль отслеживания перерасхода памяти пользователем находит запрос с наибольшим коэффициентом перерасхода в списке запросов пользователя. -Коэффициент перерасхода для запроса вычисляется как количество выделенных байт, делённое на значение настройки `memory_overcommit_ratio_denominator_for_user`. - -Если значение `memory_overcommit_ratio_denominator_for_user` для запроса равно нулю, модуль отслеживания перерасхода не будет выбирать этот запрос. - -Время ожидания задаётся настройкой `memory_usage_overcommit_max_wait_microseconds`. - -**Пример** - -```sql -SELECT number FROM numbers(1000) GROUP BY number SETTINGS memory_overcommit_ratio_denominator_for_user=4000, memory_usage_overcommit_max_wait_microseconds=500 -``` - - -## Глобальный трекер перерасхода памяти {#global-overcommit-tracker} - -Глобальный трекер перерасхода памяти находит запрос с наибольшим коэффициентом перерасхода в списке всех запросов. -В этом случае коэффициент перерасхода вычисляется как количество выделенных байт, делённое на значение настройки `memory_overcommit_ratio_denominator`. - -Если значение `memory_overcommit_ratio_denominator` для запроса равно нулю, трекер перерасхода памяти не будет выбирать этот запрос. - -Таймаут ожидания задаётся параметром `memory_usage_overcommit_max_wait_microseconds` в конфигурационном файле. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md deleted file mode 100644 index ab65da655f4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'Настройки таблиц MergeTree, определённые в `system.merge_tree_settings`' -slug: /operations/settings/merge-tree-settings -title: 'Настройки таблиц MergeTree' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import BetaBadge from '@theme/badges/BetaBadge'; -import SettingsInfoBlock from '@theme/SettingsInfoBlock/SettingsInfoBlock'; -import VersionHistory from '@theme/VersionHistory/VersionHistory'; - -Системная таблица `system.merge_tree_settings` показывает глобальные настройки MergeTree. - -Настройки MergeTree могут быть заданы в секции `merge_tree` конфигурационного файла сервера или указаны отдельно для каждой таблицы `MergeTree` в разделе `SETTINGS` запроса `CREATE TABLE`. - -Пример настройки параметра `max_suspicious_broken_parts`: - -Настройте значение по умолчанию для всех таблиц `MergeTree` в конфигурационном файле сервера: - -```text - - 5 - -``` - -Задаётся для отдельной таблицы: - -```sql -CREATE TABLE tab -( - `A` Int64 -) -ENGINE = MergeTree -ORDER BY tuple() -SETTINGS max_suspicious_broken_parts = 500; -``` - -Измените настройки для конкретной таблицы с помощью `ALTER TABLE ... MODIFY SETTING`: - -```sql -ALTER TABLE tab MODIFY SETTING max_suspicious_broken_parts = 100; - --- сброс до глобального значения по умолчанию (значение из system.merge_tree_settings) -ALTER TABLE tab RESET SETTING max_suspicious_broken_parts; -``` - -## Настройки MergeTree {#mergetree-settings} - -{/* Нижеуказанные настройки автоматически создаются скриптом по адресу - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/settings/autogenerate-settings.sh - */ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/overview.md deleted file mode 100644 index dba7ffdcaa0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/overview.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Страница обзора настроек.' -sidebar_position: 1 -slug: /operations/settings/overview -title: 'Обзор настроек' -doc_type: 'reference' ---- - - - -# Обзор настроек {#settings-overview} - - - -## Обзор {#overview} - -:::note -Профили настроек на основе XML и [конфигурационные файлы](/operations/configuration-files) в настоящее время не поддерживаются в ClickHouse Cloud. Чтобы задать настройки для вашего сервиса ClickHouse Cloud, необходимо использовать [профили настроек на основе SQL](/operations/access-rights#settings-profiles-management). -::: - -Существуют две основные группы настроек ClickHouse: - -- Глобальные настройки сервера -- Сессионные настройки - -Основное различие между ними заключается в том, что глобальные настройки сервера применяются ко всему серверу ClickHouse, тогда как сессионные настройки применяются к пользовательским сеансам или даже к отдельным запросам. - - - -## Просмотр настроек, отличающихся от значений по умолчанию {#see-non-default-settings} - -Чтобы посмотреть, какие настройки были изменены по сравнению со значениями по умолчанию, выполните запрос к таблице -`system.settings`: - -```sql -SELECT name, value FROM system.settings WHERE changed -``` - -Если все настройки оставлены со значениями по умолчанию, ClickHouse -ничего не вернёт. - -Чтобы проверить значение конкретной настройки, вы можете указать имя -этой настройки (`name`) в запросе: - -```sql -SELECT name, value FROM system.settings WHERE name = 'max_threads' -``` - -В результате вы увидите примерно следующее: - -```response -┌─name────────┬─value─────┐ -│ max_threads │ 'auto(8)' │ -└─────────────┴───────────┘ - -1 row in set. Elapsed: 0.002 sec. -``` - - -## Дополнительные материалы {#further-reading} - -- См. раздел [глобальные настройки сервера](/operations/server-configuration-parameters/settings.md), чтобы узнать больше о настройке - сервера ClickHouse на глобальном уровне. -- См. раздел [настройки сеанса](/operations/settings/settings-query-level.md), чтобы узнать больше о настройке сервера ClickHouse - на уровне сеанса. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md deleted file mode 100644 index 9a2bc8b2e5d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'Настройки прав на запросы.' -sidebar_label: 'Права на запросы' -sidebar_position: 58 -slug: /operations/settings/permissions-for-queries -title: 'Права на запросы' -doc_type: 'reference' ---- - - - -# Права для запросов {#permissions-for-queries} - -Запросы в ClickHouse можно разделить на несколько типов: - -1. Запросы на чтение данных: `SELECT`, `SHOW`, `DESCRIBE`, `EXISTS`. -2. Запросы на запись данных: `INSERT`, `OPTIMIZE`. -3. Запросы на изменение настроек: `SET`, `USE`. -4. Запросы [DDL](https://en.wikipedia.org/wiki/Data_definition_language): `CREATE`, `ALTER`, `RENAME`, `ATTACH`, `DETACH`, `DROP`, `TRUNCATE`. -5. `KILL QUERY`. - -Следующие настройки регулируют права пользователей в зависимости от типа запроса: - - - -## readonly {#readonly} -Ограничивает права доступа для запросов на чтение данных, запись данных и изменение настроек. - -При значении 1 разрешаются: - -- Все типы запросов на чтение (такие как SELECT и эквивалентные запросы). -- Запросы, которые изменяют только контекст сессии (например, USE). - -При значении 2 разрешается всё вышеперечисленное, а также: -- SET и CREATE TEMPORARY TABLE - - :::tip - Такие запросы, как EXISTS, DESCRIBE, EXPLAIN, SHOW PROCESSLIST и т. д., эквивалентны SELECT, потому что они просто выполняют чтение из системных таблиц. - ::: - -Возможные значения: - -- 0 — Разрешены запросы на чтение, запись и изменение настроек. -- 1 — Разрешены только запросы на чтение данных. -- 2 — Разрешены запросы на чтение данных и изменение настроек. - -Значение по умолчанию: 0 - -:::note -После установки `readonly = 1` пользователь не может изменить настройки `readonly` и `allow_ddl` в текущей сессии. - -При использовании метода `GET` в [HTTP-интерфейсе](../../interfaces/http.md) значение `readonly = 1` устанавливается автоматически. Для изменения данных используйте метод `POST`. - -Установка `readonly = 1` запрещает пользователю менять настройки. Существует способ запретить пользователю изменение только отдельных настроек. Также существует способ разрешить изменение только отдельных настроек при ограничении `readonly = 1`. Подробности см. в разделе [ограничения на настройки](../../operations/settings/constraints-on-settings.md). -::: - - - -## allow_ddl {#allow_ddl} - -Разрешает или запрещает выполнение [DDL](https://en.wikipedia.org/wiki/Data_definition_language)-запросов. - -Возможные значения: - -- 0 — выполнение DDL-запросов запрещено. -- 1 — выполнение DDL-запросов разрешено. - -Значение по умолчанию: 1 - -:::note -Нельзя выполнить `SET allow_ddl = 1`, если для текущего сеанса `allow_ddl = 0`. -::: - -:::note KILL QUERY -Команда `KILL QUERY` может выполняться при любой комбинации значений параметров readonly и allow_ddl. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md deleted file mode 100644 index 1195ca942a9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: 'Настройки, ограничивающие сложность запросов.' -sidebar_label: 'Ограничения сложности запросов' -sidebar_position: 59 -slug: /operations/settings/query-complexity -title: 'Ограничения сложности запросов' -doc_type: 'reference' ---- - - - -# Ограничения сложности запросов {#restrictions-on-query-complexity} - - - -## Обзор {#overview} - -В рамках [настроек](/operations/settings/overview) ClickHouse предоставляет -возможность накладывать ограничения на сложность запросов. Это помогает защититься -от потенциально ресурсоёмких запросов, обеспечивая более безопасное и предсказуемое -выполнение, особенно при использовании пользовательского интерфейса. - -Почти все ограничения применяются только к запросам `SELECT`, а при распределённой -обработке запросов ограничения применяются на каждом сервере отдельно. - -Как правило, ClickHouse проверяет ограничения только после того, как части данных -были полностью обработаны, а не проверяет их для каждой строки. Это может приводить к ситуации, -когда ограничения нарушаются в процессе обработки части данных. - - - -## Настройки `overflow_mode` {#overflow_mode_setting} - -У большинства ограничений также есть настройка `overflow_mode`, которая определяет, что происходит -при превышении лимита, и может принимать одно из двух значений: -- `throw`: сгенерировать исключение (по умолчанию). -- `break`: остановить выполнение запроса и вернуть частичный результат — как если бы - исходные данные закончились. - - - -## Настройки `group_by_overflow_mode` {#group_by_overflow_mode_settings} - -Параметр `group_by_overflow_mode` также может принимать -значение `any`: -- `any` : продолжать агрегацию для ключей, которые попали в множество, но не - добавлять в множество новые ключи. - - - -## Список настроек {#relevant-settings} - -Следующие настройки используются для ограничения сложности запросов. - -:::note -Ограничения вида «максимальное количество чего-либо» могут принимать значение `0`, -что означает «без ограничений». -::: - - - -| Настройка | Краткое описание | -| ---------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`max_memory_usage`](/operations/settings/settings#max_memory_usage) | Максимальный объём оперативной памяти, используемый при выполнении запроса на одном сервере. | -| [`max_memory_usage_for_user`](/operations/settings/settings#max_memory_usage_for_user) | Максимальный объём оперативной памяти, используемый для выполнения запросов пользователя на одном сервере. | -| [`max_rows_to_read`](/operations/settings/settings#max_rows_to_read) | Максимальное число строк, которые могут быть прочитаны из таблицы при выполнении запроса. | -| [`max_bytes_to_read`](/operations/settings/settings#max_bytes_to_read) | Максимальное число байт несжатых данных, которое может быть прочитано из таблицы при выполнении запроса. | -| [`read_overflow_mode_leaf`](/operations/settings/settings#read_overflow_mode_leaf) | Определяет поведение при превышении объёма читаемых данных одного из листовых лимитов | -| [`max_rows_to_read_leaf`](/operations/settings/settings#max_rows_to_read_leaf) | Максимальное число строк, которое можно прочитать из локальной таблицы на листовом узле при выполнении распределённого запроса | -| [`max_bytes_to_read_leaf`](/operations/settings/settings#max_bytes_to_read_leaf) | Максимальное количество байт (несжатых данных), которое можно прочитать из локальной таблицы на листовом узле при выполнении распределённого запроса. | -| [`read_overflow_mode_leaf`](/docs/operations/settings/settings#read_overflow_mode_leaf) | Задаёт поведение при превышении объёма читаемых данных над одним из листовых ограничений. | -| [`max_rows_to_group_by`](/operations/settings/settings#max_rows_to_group_by) | Максимальное количество уникальных ключей, полученных при агрегации. | -| [`group_by_overflow_mode`](/operations/settings/settings#group_by_overflow_mode) | Задаёт, что происходит, когда число уникальных ключей агрегации превышает предел | -| [`max_bytes_before_external_group_by`](/operations/settings/settings#max_bytes_before_external_group_by) | Включает или отключает выполнение конструкций `GROUP BY` во внешней памяти. | -| [`max_bytes_ratio_before_external_group_by`](/operations/settings/settings#max_bytes_ratio_before_external_group_by) | Доля доступной памяти, которую может использовать `GROUP BY`. При достижении этого порога для агрегации используется внешняя память. | -| [`max_bytes_before_external_sort`](/operations/settings/settings#max_bytes_before_external_sort) | Включает или отключает выполнение выражений `ORDER BY` во внешней памяти. | -| [`max_bytes_ratio_before_external_sort`](/operations/settings/settings#max_bytes_ratio_before_external_sort) | Часть доступной памяти, которую можно использовать для `ORDER BY`. При достижении этого порога используется внешняя сортировка. | -| [`max_rows_to_sort`](/operations/settings/settings#max_rows_to_sort) | Максимальное количество строк до сортировки. Позволяет ограничить потребление памяти при сортировке. | -| [`max_bytes_to_sort`](/operations/settings/settings#max_rows_to_sort) | Максимальный объём в байтах до сортировки. | -| [`sort_overflow_mode`](/operations/settings/settings#sort_overflow_mode) | Определяет поведение при превышении одного из лимитов на число строк, полученных до сортировки. | -| [`max_result_rows`](/operations/settings/settings#max_result_rows) | Ограничивает количество строк в результате. | -| [`max_result_bytes`](/operations/settings/settings#max_result_bytes) | Ограничивает размер результата в байтах (без сжатия) | -| [`result_overflow_mode`](/operations/settings/settings#result_overflow_mode) | Определяет, что делать, если объем результата превышает одно из ограничений. | -| [`max_execution_time`](/operations/settings/settings#max_execution_time) | Максимальное время выполнения запроса в секундах. | -| [`timeout_overflow_mode`](/operations/settings/settings#timeout_overflow_mode) | Определяет, что делать, если запрос выполняется дольше, чем `max_execution_time`, или расчетное время выполнения превышает `max_estimated_execution_time`. | -| [`max_execution_time_leaf`](/operations/settings/settings#max_execution_time_leaf) | По смыслу аналогичен параметру `max_execution_time`, но применяется только на листовых узлах при выполнении распределённых или удалённых запросов. | -| [`timeout_overflow_mode_leaf`](/operations/settings/settings#timeout_overflow_mode_leaf) | Определяет, что происходит, когда выполнение запроса на листовом узле превышает `max_execution_time_leaf`. | -| [`min_execution_speed`](/operations/settings/settings#min_execution_speed) | Минимальная скорость выполнения (строк в секунду). | -| [`min_execution_speed_bytes`](/operations/settings/settings#min_execution_speed_bytes) | Минимальное количество байт, обрабатываемых в секунду. | -| [`max_execution_speed`](/operations/settings/settings#max_execution_speed) | Максимальное количество строк, обрабатываемых в секунду. | -| [`max_execution_speed_bytes`](/operations/settings/settings#max_execution_speed_bytes) | Максимальное число байт, обрабатываемых в секунду. | -| [`timeout_before_checking_execution_speed`](/operations/settings/settings#timeout_before_checking_execution_speed) | Проверяет, что скорость выполнения не слишком низкая (не меньше `min_execution_speed`) после истечения указанного количества секунд. | -| [`max_estimated_execution_time`](/operations/settings/settings#max_estimated_execution_time) | Максимальная оценка времени выполнения запроса в секундах. | -| [`max_columns_to_read`](/operations/settings/settings#max_columns_to_read) | Максимальное количество столбцов, которые можно прочитать из таблицы за один запрос. | -| [`max_temporary_columns`](/operations/settings/settings#max_temporary_columns) | Максимальное количество временных столбцов, одновременно хранимых в оперативной памяти при выполнении запроса, включая константные столбцы. | -| [`max_temporary_non_const_columns`](/operations/settings/settings#max_temporary_non_const_columns) | Максимальное количество временных столбцов, которые необходимо одновременно хранить в оперативной памяти при выполнении запроса, но без учета константных столбцов. | -| [`max_subquery_depth`](/operations/settings/settings#max_subquery_depth) | Определяет, что происходит, если запрос содержит больше, чем указанное число вложенных подзапросов. | -| [`max_ast_depth`](/operations/settings/settings#max_ast_depth) | Максимальная глубина вложенности синтаксического дерева запроса. | -| [`max_ast_elements`](/operations/settings/settings#max_ast_elements) | Максимальное число элементов в синтаксическом дереве запроса. | -| [`max_rows_in_set`](/operations/settings/settings#max_rows_in_set) | Максимальное количество строк в наборе данных для условия IN, сформированного подзапросом. | -| [`max_bytes_in_set`](/operations/settings/settings#max_bytes_in_set) | Максимальное количество байт (несжатых данных), которое может занимать множество в условии IN, построенном на основе подзапроса. | -| [`set_overflow_mode`](/operations/settings/settings#max_bytes_in_set) | Задаёт поведение при превышении объёма данных по одному из ограничений. | -| [`max_rows_in_distinct`](/operations/settings/settings#max_rows_in_distinct) | Максимальное количество уникальных строк при использовании DISTINCT. | -| [`max_bytes_in_distinct`](/operations/settings/settings#max_bytes_in_distinct) | Максимальный размер состояния в памяти в байтах (в несжатом виде), используемый хеш-таблицей при выполнении DISTINCT. | -| [`distinct_overflow_mode`](/operations/settings/settings#distinct_overflow_mode) | Задаёт поведение при превышении объёма данных по одному из лимитов. | -| [`max_rows_to_transfer`](/operations/settings/settings#max_rows_to_transfer) | Максимальный размер (в строках), который может быть передан на удалённый сервер или сохранён во временной таблице при выполнении конструкции GLOBAL IN/JOIN. | -| [`max_bytes_to_transfer`](/operations/settings/settings#max_bytes_to_transfer) | Максимальное количество байт несжатых данных, которое может быть передано на удалённый сервер или сохранено во временной таблице при выполнении секции GLOBAL IN/JOIN. | -| [`transfer_overflow_mode`](/operations/settings/settings#transfer_overflow_mode) | Определяет поведение, когда объём данных превышает один из лимитов. | -| [`max_rows_in_join`](/operations/settings/settings#max_rows_in_join) | Ограничивает количество строк в хеш-таблице, используемой при выполнении операции соединения таблиц. | -| [`max_bytes_in_join`](/operations/settings/settings#max_bytes_in_join) | Максимальный размер (в байтах) хэш-таблицы, используемой при соединении таблиц. | -| [`join_overflow_mode`](/operations/settings/settings#join_overflow_mode) | Определяет, какое действие выполняет ClickHouse при достижении любого из следующих ограничений для JOIN. | -| [`max_partitions_per_insert_block`](/operations/settings/settings#max_partitions_per_insert_block) | Ограничивает максимальное количество партиций в одном вставляемом блоке, и если блок содержит слишком много партиций, генерируется исключение. | -| [`throw_on_max_partitions_per_insert_block`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | Позволяет контролировать поведение при достижении предела `max_partitions_per_insert_block`. | -| [`max_temporary_data_on_disk_size_for_user`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | Максимальный объём данных во временных файлах на диске (в байтах) для всех одновременно выполняющихся пользовательских запросов. | -| [`max_temporary_data_on_disk_size_for_query`](/operations/settings/settings#max_temporary_data_on_disk_size_for_query) | Максимальный объём данных в байтах, занимаемый временными файлами на диске для всех одновременно выполняемых запросов. | -| [`max_sessions_for_user`](/operations/settings/settings#max_sessions_for_user) | Максимальное количество одновременных сессий на сервере ClickHouse для одного аутентифицированного пользователя. | -| [`max_partitions_to_read`](/operations/settings/settings#max_partitions_to_read) | Ограничивает максимальное количество разделов, к которым можно получить доступ в одном запросе. | - - - - - -## Устаревшие настройки {#obsolete-settings} - -:::note -Следующие настройки устарели -::: - -### max_pipeline_depth {#max-pipeline-depth} - -Максимальная глубина конвейера. Соответствует количеству преобразований, которые проходит каждый -блок данных во время обработки запроса. Считается в пределах -одного сервера. Если глубина конвейера превышает это значение, выбрасывается исключение. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md deleted file mode 100644 index 7ded82ab423..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Управление поведением при перегрузке процессора сервера.' -sidebar_label: 'Перегрузка сервера' -slug: /operations/settings/server-overload -title: 'Перегрузка сервера' -doc_type: 'reference' ---- - - - -# Перегрузка сервера {#server-overload} - - - -## Обзор {#overview} - -Иногда сервер может оказаться перегружен по разным причинам. Чтобы определить текущую перегрузку CPU, -сервер ClickHouse вычисляет отношение времени ожидания CPU (метрика `OSCPUWaitMicroseconds`) ко времени его занятости -(метрика `OSCPUVirtualTimeMicroseconds`). Когда полученное соотношение превышает определённый порог, -имеет смысл отбросить часть запросов или даже отклонять новые подключения, чтобы не увеличивать нагрузку ещё больше. - -Существует серверная настройка `os_cpu_busy_time_threshold`, которая задаёт минимальное время занятости, чтобы считать, -что CPU выполняет полезную работу. Если текущее значение метрики `OSCPUVirtualTimeMicroseconds` ниже этого значения, -считается, что перегрузка CPU равна нулю. - - - -## Отклонение запросов {#rejecting-queries} - -Поведение при отклонении запросов контролируется настройками на уровне запроса `min_os_cpu_wait_time_ratio_to_throw` и -`max_os_cpu_wait_time_ratio_to_throw`. Если эти настройки заданы и `min_os_cpu_wait_time_ratio_to_throw` меньше, -чем `max_os_cpu_wait_time_ratio_to_throw`, то запрос отклоняется и ошибка `SERVER_OVERLOADED` генерируется -с некоторой вероятностью, если коэффициент перегрузки не менее `min_os_cpu_wait_time_ratio_to_throw`. Вероятность -определяется как линейная интерполяция между минимальным и максимальным значениями коэффициента. Например, если `min_os_cpu_wait_time_ratio_to_throw = 2`, -`max_os_cpu_wait_time_ratio_to_throw = 6` и `cpu_overload = 4`, то запрос будет отклонён с вероятностью `0.5`. - - - -## Сброс соединений {#dropping-connections} - -Сброс соединений контролируется серверными параметрами `min_os_cpu_wait_time_ratio_to_drop_connection` и -`max_os_cpu_wait_time_ratio_to_drop_connection`. Эти параметры можно изменять без перезапуска сервера. Идея этих -параметров аналогична параметрам для отклонения запросов. Единственное отличие в данном случае состоит в том, что если сервер перегружен, -попытка установления соединения будет отклонена сервером. - - - -## Предупреждения о перегрузке ресурсов {#resource-overload-warnings} - -ClickHouse также записывает в таблицу `system.warnings` предупреждения о перегрузке CPU и памяти при перегрузке сервера. Вы можете -настроить эти пороги в конфигурации сервера. - -**Пример** - -```xml - - - 0.9 - 0.8 - 600 - 0.9 - 0.8 - 600 - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md deleted file mode 100644 index 3f1870f0c9f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -description: 'Настройки, связанные с форматами' -sidebar_label: 'Настройки форматов' -slug: /operations/settings/formats -title: 'Настройки форматов' -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/*Не редактируйте этот файл — он сгенерирован автоматически*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md deleted file mode 100644 index fa2bbe3a1a0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: 'Набор настроек, объединённых общим именем.' -sidebar_label: 'Профили настроек' -sidebar_position: 61 -slug: /operations/settings/settings-profiles -title: 'Профили настроек' -doc_type: 'reference' ---- - -# Профили настроек {#settings-profiles} - -Профиль настроек — это набор настроек, объединённых под одним именем. - -:::note -ClickHouse также поддерживает [SQL-ориентированный рабочий процесс](/operations/access-rights#access-control-usage) для управления профилями настроек. Мы рекомендуем использовать именно его. -::: - -Профиль может иметь любое имя. Вы можете указать один и тот же профиль для разных пользователей. Самое важное, что вы можете указать в профиле настроек, — это `readonly=1`, что обеспечивает доступ только для чтения. - -Профили настроек могут наследоваться друг от друга. Чтобы использовать наследование, укажите одну или несколько настроек `profile` перед другими настройками, перечисленными в профиле. Если одна и та же настройка определена в разных профилях, используется последняя определённая. - -Чтобы применить все настройки из профиля, задайте настройку `profile`. - -Пример: - -Установите профиль `web`. - -```sql -SET profile = 'web' -``` - -Профили настроек задаются в пользовательском конфигурационном файле. Обычно это `users.xml`. - -Пример: - -```xml - - - - - - 8 - - - - - 1000000000 - 100000000000 - - 1000000 - any - - 1000000 - 1000000000 - - 100000 - 100000000 - break - - 600 - 1000000 - 15 - - 25 - 100 - 50 - - 2 - 25 - 50 - 100 - - 4 - - 1 - - -``` - -В примере заданы два профиля: `default` и `web`. - -Профиль `default` имеет особое назначение: он всегда должен присутствовать и применяется при запуске сервера. Другими словами, профиль `default` содержит настройки по умолчанию. - -Профиль `web` — это обычный профиль, который можно задать с помощью запроса `SET` или параметра URL в HTTP-запросе. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md deleted file mode 100644 index 254b47bbb99..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md +++ /dev/null @@ -1,231 +0,0 @@ ---- -description: 'Настройки сеанса на уровне запроса' -sidebar_label: 'Сессионные настройки на уровне запроса' -slug: /operations/settings/query-level -title: 'Сессионные настройки на уровне запроса' -doc_type: 'reference' ---- - -## Обзор {#overview} - -Существует несколько способов выполнения запросов с заданными настройками. -Настройки задаются на нескольких уровнях, и каждый последующий уровень переопределяет предыдущие значения параметра. - -## Порядок приоритета {#order-of-priority} - -Порядок приоритета для задания настройки: - -1. Применение настройки непосредственно к пользователю или внутри профиля настроек - - - SQL (рекомендуется) - - добавление одного или нескольких XML- или YAML-файлов в `/etc/clickhouse-server/users.d` - -2. Настройки сессии - - - Отправьте `SET setting=value` из SQL-консоли ClickHouse Cloud или - `clickhouse client` в интерактивном режиме. Аналогично, вы можете - использовать сессии ClickHouse по протоколу HTTP. Для этого необходимо - указать HTTP-параметр `session_id`. - -3. Настройки запроса - - - При запуске `clickhouse client` в неинтерактивном режиме установите - параметр запуска `--setting=value`. - - При использовании HTTP API передавайте CGI-параметры (`URL?setting_1=value&setting_2=value...`). - - Определите настройки в разделе - [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select-query) - запроса SELECT. Значение настройки применяется только к этому запросу и - после выполнения запроса сбрасывается к значению по умолчанию или предыдущему значению. - -## Возврат настройки к значению по умолчанию {#converting-a-setting-to-its-default-value} - -Если вы изменили настройку и хотите вернуть её к значению по умолчанию, укажите значение `DEFAULT`. Синтаксис следующий: - -```sql -SET имя_настройки = DEFAULT -``` - -Например, по умолчанию `async_insert` имеет значение `0`. Предположим, вы измените его на `1`: - -```sql -SET async_insert = 1; - -SELECT value FROM system.settings where name='async_insert'; -``` - -Ответ: - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - -Следующая команда снова устанавливает его значение в 0: - -```sql -SET async_insert = DEFAULT; - -SELECT value FROM system.settings where name='async_insert'; -``` - -Параметр снова установлен в значение по умолчанию: - -```response -┌─value───┐ -│ 0 │ -└─────────┘ -``` - - -## Пользовательские настройки {#custom_settings} - -В дополнение к общим [настройкам](/operations/settings/settings.md) пользователи могут задавать собственные настройки. - -Имя пользовательской настройки должно начинаться с одного из предопределённых префиксов. Список этих префиксов задаётся в параметре [custom_settings_prefixes](../../operations/server-configuration-parameters/settings.md#custom_settings_prefixes) в файле конфигурации сервера. - -```xml -custom_ -``` - -Чтобы задать пользовательскую настройку, используйте команду `SET`: - -```sql -SET custom_a = 123; -``` - -Чтобы получить текущее значение пользовательской настройки, используйте функцию `getSetting()`: - -```sql -SELECT getSetting('custom_a'); -``` - - -## Примеры {#examples} - -Во всех этих примерах значение настройки `async_insert` устанавливается в `1` и демонстрируется, как просматривать настройки в работающей системе. - -### Применение настройки к пользователю напрямую с помощью SQL {#using-sql-to-apply-a-setting-to-a-user-directly} - -Это создаёт пользователя `ingester` с настройкой `async_inset = 1`: - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS async_insert = 1 -``` - - -#### Просмотрите профиль настроек и его назначение {#examine-the-settings-profile-and-assignment} - -```sql -ПОКАЗАТЬ ДОСТУП -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ ... │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS async_insert = true │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### Использование SQL для создания профиля настроек и назначения его пользователю {#using-sql-to-create-a-settings-profile-and-assign-to-a-user} - -Создаётся профиль `log_ingest` с настройкой `async_inset = 1`: - -```sql -CREATE -SETTINGS PROFILE log_ingest SETTINGS async_insert = 1 -``` - -Это создаёт пользователя `ingester` и назначает этому пользователю профиль настроек `log_ingest`: - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS PROFILE log_ingest -``` - - -### Создание профиля настроек и пользователя с помощью XML {#using-xml-to-create-a-settings-profile-and-user} - -```xml title=/etc/clickhouse-server/users.d/users.xml - -# highlight-start {#highlight-start} - - - 1 - - -# highlight-end {#highlight-end} - - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 -# highlight-start {#highlight-start} - log_ingest -# highlight-end {#highlight-end} - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 - 1 - 1 - - - -``` - - -#### Просмотрите профиль настроек и его назначение {#examine-the-settings-profile-and-assignment-1} - -```sql -ПОКАЗАТЬ ДОСТУП -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ CREATE USER default IDENTIFIED WITH sha256_password │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS PROFILE log_ingest │ -│ CREATE SETTINGS PROFILE default │ -# highlight-next-line {#highlight-next-line} -│ CREATE SETTINGS PROFILE log_ingest SETTINGS async_insert = true │ -│ CREATE SETTINGS PROFILE readonly SETTINGS readonly = 1 │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### Назначьте настройку сеансу {#assign-a-setting-to-a-session} - -```sql -SET async_insert =1; -SELECT value FROM system.settings where name='async_insert'; -``` - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - - -### Назначение настройки в запросе {#assign-a-setting-during-a-query} - -```sql -INSERT INTO YourTable --- highlight-next-line -SETTINGS async_insert=1 -VALUES (...) -``` - - -## См. также {#see-also} - -- См. страницу [Settings](/operations/settings/settings.md) с описанием настроек ClickHouse. -- [Глобальные настройки сервера](/operations/server-configuration-parameters/settings.md) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md deleted file mode 100644 index e2ba913be0c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md +++ /dev/null @@ -1,261 +0,0 @@ ---- -description: 'Параметры настройки пользователей и ролей.' -sidebar_label: 'Настройки пользователей' -sidebar_position: 63 -slug: /operations/settings/settings-users -title: 'Настройки пользователей и ролей' -doc_type: 'reference' ---- - -# Настройки пользователей и ролей {#users-and-roles-settings} - -Раздел `users` конфигурационного файла `users.xml` содержит настройки пользователей. - -:::note -ClickHouse также поддерживает [SQL-управляемый подход](/operations/access-rights#access-control-usage) к управлению пользователями. Мы рекомендуем использовать его. -::: - -Структура раздела `users`: - -```xml - - - - - - - - - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - - - ecdsa-sha2-nistp256 - AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxeV2uN5UY6CUbCzTA1rXfYimKQA5ivNIqxdax4bcMXz4D0nSk2l5E1TkR5mG8EBWtmExSPbcEPJ8V7lyWWbA8= - - - ssh-rsa - AAAAB3NzaC1yc2EAAAADAQABAAABgQCpgqL1SHhPVBOTFlOm0pu+cYBbADzC2jL41sPMawYCJHDyHuq7t+htaVVh2fRgpAPmSEnLEC2d4BEIKMtPK3bfR8plJqVXlLt6Q8t4b1oUlnjb3VPA9P6iGcW7CV1FBkZQEVx8ckOfJ3F+kI5VsrRlEDgiecm/C1VPl0/9M2llW/mPUMaD65cM9nlZgM/hUeBrfxOEqM11gDYxEZm1aRSbZoY4dfdm3vzvpSQ6lrCrkjn3X2aSmaCLcOWJhfBWMovNDB8uiPuw54g3ioZ++qEQMlfxVsqXDGYhXCrsArOVuW/5RbReO79BvXqdssiYShfwo+GhQ0+aLWMIW/jgBkkqx/n7uKLzCMX7b2F+aebRYFh+/QXEj7SnihdVfr9ud6NN3MWzZ1ltfIczlEcFLrLJ1Yq57wW6wXtviWh59WvTWFiPejGjeSjjJyqqB49tKdFVFuBnIU5u/bch2DXVgiAEdQwUrIp1ACoYPq22HFFAYUJrL32y7RxX3PGzuAv3LOc= - - - - 0|1 - - - - - profile_name - - default - default - - - - expression - - - - - - GRANT SELECT ON system.* - - - - -``` - -### user_name/password {#user-namepassword} - -Пароль может быть задан в открытом виде или в формате SHA256 (в шестнадцатеричном представлении). - -* Чтобы задать пароль в открытом виде (**не рекомендуется**), поместите его в элемент `password`. - - Например, `qwerty`. Пароль может быть пустым. - - - -* Чтобы задать пароль с использованием его SHA256-хэша, поместите его в элемент `password_sha256_hex`. - -Например, `65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5`. - -Пример генерации пароля в командной оболочке: - -```bash -PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-' -``` - -Первая строка результата — это пароль. Вторая строка — соответствующий хэш SHA256. - - - -* Для совместимости с клиентами MySQL пароль может быть указан в виде двойного хэша SHA1. Поместите его в элемент `password_double_sha1_hex`. - - Например, `08b4a0f1de6ad37da17359e592c8d74788a83eb0`. - - Пример генерации пароля в командной оболочке: - - ```bash - PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-' - ``` - - Первая строка результата — это пароль. Вторая строка — соответствующий двойной хэш SHA1. - -### username/ssh-key {#user-sshkey} - -Эта настройка позволяет аутентифицироваться с помощью SSH-ключей. - -Имея SSH-ключ (сгенерированный с помощью `ssh-keygen`) следующего вида - -```text -ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj john@example.com -``` - -Ожидается, что элемент `ssh_key` будет - -```xml - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - -``` - -Замените `ssh-ed25519` на `ssh-rsa` или `ecdsa-sha2-nistp256` для использования других поддерживаемых алгоритмов. - -### access_management {#access_management-user-setting} - -Этот параметр включает или отключает использование управляемого с помощью SQL [контроля доступа и управления учетными записями](/operations/access-rights#access-control-usage) для пользователя. - -Возможные значения: - -* 0 — Отключено. -* 1 — Включено. - -Значение по умолчанию: 0. - -### grants {#grants-user-setting} - -Этот параметр позволяет назначать любые права выбранному пользователю. -Каждый элемент списка должен представлять собой запрос `GRANT` без указания получателей прав. - -Пример: - -```xml - - - GRANT SHOW ON *.* - GRANT CREATE ON *.* WITH GRANT OPTION - GRANT SELECT ON system.* - - -``` - -Этот параметр не может быть указан одновременно с параметрами -`dictionaries`, `access_management`, `named_collection_control`, `show_named_collections_secrets` -и `allow_databases`. - -### user_name/networks {#user-namenetworks} - -Список сетей, из которых пользователь может подключаться к серверу ClickHouse. - -Каждый элемент списка может иметь одну из следующих форм: - -* `` — IP-адрес или сетевая маска. - - Примеры: `213.180.204.3`, `10.0.0.1/8`, `10.0.0.1/255.255.255.0`, `2a02:6b8::3`, `2a02:6b8::3/64`, `2a02:6b8::3/ffff:ffff:ffff:ffff::`. - -* `` — имя хоста. - - Пример: `example01.host.ru`. - - Для проверки доступа выполняется DNS-запрос, и все возвращённые IP-адреса сравниваются с адресом удалённой стороны. - -* `` — регулярное выражение для имён хостов. - - Пример: `^example\d\d-\d\d-\d\.host\.ru$` - -Чтобы проверить доступ, выполняется [DNS PTR‑запрос](https://en.wikipedia.org/wiki/Reverse_DNS_lookup) для адреса пира, после чего к результату применяется указанное регулярное выражение. Затем выполняется ещё один DNS‑запрос по результатам PTR‑запроса, и все полученные адреса сравниваются с адресом пира. Настоятельно рекомендуем, чтобы регулярное выражение заканчивалось символом $. - -Все результаты DNS‑запросов кэшируются до перезапуска сервера. - -**Примеры** - -Чтобы открыть доступ пользователю из любой сети, укажите: - -```xml -::/0 -``` - -:::note -Открывать доступ из любой сети небезопасно, если только у вас не настроен должным образом брандмауэр или сервер не подключён напрямую к интернету. -::: - -Чтобы открыть доступ только с localhost, укажите: - -```xml -::1 -127.0.0.1 -``` - -### user_name/profile {#user-nameprofile} - -Вы можете назначить пользователю профиль настроек. Профили настроек конфигурируются в отдельном разделе файла `users.xml`. Для получения дополнительной информации см. [Профили настроек](../../operations/settings/settings-profiles.md). - -### user_name/quota {#user-namequota} - -Квоты позволяют отслеживать или ограничивать использование ресурсов за определённый период времени. Квоты настраиваются в разделе `quotas` конфигурационного файла `users.xml`. - -Вы можете назначить пользователю набор квот. Подробное описание настройки квот см. в разделе [Quotas](/operations/quotas). - -### user_name/databases {#user-namedatabases} - -В этом разделе вы можете ограничить строки, возвращаемые ClickHouse для запросов `SELECT`, выполняемых текущим пользователем, тем самым реализуя базовую построчную безопасность (row-level security). - -**Пример** - -Следующая конфигурация гарантирует, что пользователь `user1` в результате запросов `SELECT` может видеть только строки таблицы `table1`, в которых значение поля `id` равно 1000. - -```xml - - - - - id = 1000 - - - - -``` - -`filter` может быть любым выражением, которое возвращает значение типа [UInt8](../../sql-reference/data-types/int-uint.md). Обычно оно содержит операторы сравнения и логические операторы. Строки из `database_name.table1`, для которых результат фильтра равен 0, этому пользователю не возвращаются. Фильтрация несовместима с операциями `PREWHERE` и отключает оптимизацию `WHERE→PREWHERE`. - -## Роли {#roles} - -Вы можете создавать любые предопределённые роли, используя раздел `roles` в конфигурационном файле `user.xml`. - -Структура раздела `roles`: - -```xml - - - - GRANT SHOW ON *.* - REVOKE SHOW ON system.* - GRANT CREATE ON *.* WITH GRANT OPTION - - - -``` - -Эти роли также могут быть назначены пользователям из раздела `users`: - -```xml - - - ... - - GRANT test_role - - - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings.md deleted file mode 100644 index f97997428c6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/settings.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: 'Параметры сессии' -description: 'Настройки, задаваемые в system.settings' -sidebar_label: 'Параметры сессии' -slug: /operations/settings/settings -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/*Не редактируйте — этот файл создан автоматически*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md deleted file mode 100644 index 3562bff31bb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: 'Ограничения на TCP-соединения.' -sidebar_label: 'Ограничения на TCP-соединения' -slug: /operations/settings/tcp-connection-limits -title: 'Ограничения на TCP-соединения' -doc_type: 'reference' ---- - - - -# Ограничения на количество TCP-соединений {#tcp-connection-limits} - - - -## Обзор {#overview} - -TCP-подключение к ClickHouse (например, при использовании [клиента командной строки](https://clickhouse.com/docs/interfaces/cli)) -может автоматически разорваться после определённого количества запросов или по истечении некоторого времени. -После разрыва соединения автоматическое переподключение не происходит (если только оно не инициировано чем-то ещё, -например, отправкой нового запроса в клиенте командной строки). - -Ограничения на соединения включаются настройками сервера -`tcp_close_connection_after_queries_num` (для ограничения по количеству запросов) -или `tcp_close_connection_after_queries_seconds` (для ограничения по длительности), заданными со значением, большим нуля. -Если включены оба ограничения, соединение закрывается при первом достигнутом лимите. - -При достижении лимита и разрыве соединения клиент получает -исключение `TCP_CONNECTION_LIMIT_REACHED`, и **запрос, который приводит к разрыву соединения, никогда не обрабатывается**. - - - -## Ограничения на количество запросов {#query-limits} - -Предположим, что `tcp_close_connection_after_queries_num` установлен в значение N — тогда соединение допускает -N успешных запросов. Затем, при запросе N + 1, клиент отключается. - -Каждый обработанный запрос учитывается в лимите запросов. Поэтому при подключении -клиента командной строки может выполняться автоматический начальный запрос -системных предупреждений, который также учитывается в лимите. - -Когда TCP‑соединение простаивает (то есть не обрабатывает запросы в течение некоторого периода -времени, задаваемого параметром сессии `poll_interval`), количество уже учтённых запросов -сбрасывается в 0. Это означает, что общее число запросов в рамках одного соединения может -превышать `tcp_close_connection_after_queries_num`, если происходит простой. - - - -## Ограничения по длительности {#duration-limits} - -Длительность соединения измеряется с момента подключения клиента. -Клиент отключается при выполнении первого запроса после истечения `tcp_close_connection_after_queries_seconds` секунд. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md deleted file mode 100644 index eba7128f494..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Руководство по настройке защищённого взаимодействия по SSL/TLS между ClickHouse и ZooKeeper' -sidebar_label: 'Защищённое взаимодействие с Zookeeper' -sidebar_position: 45 -slug: /operations/ssl-zookeeper -title: 'Опциональное защищённое взаимодействие между ClickHouse и Zookeeper' -doc_type: 'guide' ---- - -# Опциональное защищённое взаимодействие между ClickHouse и ZooKeeper {#optional-secured-communication-between-clickhouse-and-zookeeper} - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - -Необходимо указать `ssl.keyStore.location`, `ssl.keyStore.password` и `ssl.trustStore.location`, `ssl.trustStore.password` для взаимодействия с клиентом ClickHouse через SSL. Эти параметры доступны, начиная с версии Zookeeper 3.5.2. - -Вы можете добавить `zookeeper.crt` в список доверенных сертификатов. - -```bash -sudo cp zookeeper.crt /usr/local/share/ca-certificates/zookeeper.crt -sudo update-ca-certificates -``` - -Раздел `client` в файле `config.xml` будет выглядеть следующим образом: - -```xml - - /etc/clickhouse-server/client.crt - /etc/clickhouse-server/client.key - true - true - sslv2,sslv3 - true - - RejectCertificateHandler - - -``` - -Добавьте Zookeeper в конфигурацию ClickHouse, указав кластер и макросы: - -```xml - - - - localhost - 2281 - 1 - - - -``` - -Запустите `clickhouse-server`. В логах вы увидите: - -```text - ZooKeeper: инициализирован, хосты: secure://localhost:2281 -``` - -Префикс `secure://` указывает на то, что соединение защищено с помощью SSL. - -Чтобы убедиться, что трафик шифруется, запустите `tcpdump` на защищённом порту: - -```bash -tcpdump -i any dst port 2281 -nnXS -``` - -И выполните запрос в `clickhouse-client`: - -```sql -SELECT * FROM system.zookeeper WHERE path = '/'; -``` - -При незашифрованном соединении в выводе команды `tcpdump` вы увидите что-то вроде этого: - -```text -..../zookeeper/quota. -``` - -При защищённом соединении этого быть не должно. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/startup-scripts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/startup-scripts.md deleted file mode 100644 index 7d2a2baf23e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/startup-scripts.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'Руководство по настройке и использованию стартовых SQL-скриптов в ClickHouse для - автоматического создания схем и выполнения миграций' -sidebar_label: 'Стартовые скрипты' -slug: /operations/startup-scripts -title: 'Стартовые скрипты' -doc_type: 'guide' ---- - -# Скрипты инициализации {#startup-scripts} - -ClickHouse может выполнять произвольные SQL‑запросы из конфигурации сервера при запуске. Это может быть полезно для миграций или автоматического создания схемы. - -```xml - - - false - - CREATE ROLE OR REPLACE test_role - - - CREATE TABLE TestTable (id UInt64) ENGINE=TinyLog - SELECT 1; - - - CREATE DICTIONARY test_dict (...) SOURCE(CLICKHOUSE(...)) - default - - - -``` - -ClickHouse выполняет все запросы из `startup_scripts` последовательно в указанном порядке. Если какой-либо из запросов завершится с ошибкой, выполнение последующих запросов не будет прервано. Однако при `throw_on_error = true` -сервер не запустится, если во время выполнения скрипта произойдёт ошибка. - -Вы можете задать условный запрос в конфигурации. В этом случае соответствующий запрос будет выполнен только тогда, когда условный запрос вернёт значение `1` или `true`. - -:::note -Если условный запрос вернёт любое другое значение, отличное от `1` или `true`, результат будет интерпретирован как `false`, и соответствующий запрос выполнен не будет. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/storing-data.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/storing-data.md deleted file mode 100644 index fbc6391a467..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/storing-data.md +++ /dev/null @@ -1,1073 +0,0 @@ ---- -description: 'Документация по highlight-next-line' -sidebar_label: 'Внешние диски для хранения данных' -sidebar_position: 68 -slug: /operations/storing-data -title: 'Внешние диски для хранения данных' -doc_type: 'guide' ---- - -Данные, обрабатываемые в ClickHouse, обычно хранятся в локальной файловой системе -машины, на которой запущен сервер ClickHouse. Для этого требуются диски большой ёмкости, -которые могут быть дорогими. Чтобы отказаться от локального хранения данных, поддерживаются различные варианты хранилищ: -1. Объектное хранилище [Amazon S3](https://aws.amazon.com/s3/). -2. [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs). -3. Не поддерживается: Hadoop Distributed File System ([HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)) - -
- -:::note -ClickHouse также поддерживает внешние движки таблиц, которые отличаются от -вариантов внешнего хранилища, описанных на этой странице, так как позволяют читать данные, -хранящиеся в распространённых файловых форматах (например, Parquet). На этой странице мы описываем -конфигурацию хранилища для семейств таблиц `MergeTree` или `Log`. - -1. для работы с данными, хранящимися на дисках `Amazon S3`, используйте движок таблицы [S3](/engines/table-engines/integrations/s3.md). -2. для работы с данными, хранящимися в Azure Blob Storage, используйте движок таблицы [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage.md). -3. для работы с данными в Hadoop Distributed File System (не поддерживается), используйте движок таблицы [HDFS](/engines/table-engines/integrations/hdfs.md). -::: - - - -## Настройка внешнего хранилища {#configuring-external-storage} - -Движки таблиц семейств [`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) и [`Log`](/engines/table-engines/log-family/log.md) -могут сохранять данные в `S3`, `AzureBlobStorage`, `HDFS` (не поддерживается), используя диски типов `s3`, -`azure_blob_storage`, `hdfs` (не поддерживается) соответственно. - -Конфигурация диска требует: - -1. Раздела `type` со значением одним из `s3`, `azure_blob_storage`, `hdfs` (не поддерживается), `local_blob_storage`, `web`. -2. Настроек конкретного типа внешнего хранилища. - -Начиная с версии ClickHouse 24.1, можно использовать новый вариант конфигурации. -Он требует указания: - -1. `type`, равного `object_storage` -2. `object_storage_type`, равного одному из `s3`, `azure_blob_storage` (или просто `azure`, начиная с `24.3`), `hdfs` (не поддерживается), `local_blob_storage` (или просто `local`, начиная с `24.3`), `web`. - -
- -Дополнительно может быть указан `metadata_type` (по умолчанию он равен `local`), но его также можно установить в `plain`, `web` и, начиная с `24.4`, `plain_rewritable`. -Использование типа метаданных `plain` описано в [разделе plain‑хранилища](/operations/storing-data#plain-storage), тип метаданных `web` может использоваться только с типом объектного хранилища `web`, тип метаданных `local` хранит файлы метаданных локально (каждый файл метаданных содержит отображение файлов в объектном хранилище и дополнительную информацию о них). - -Например: - -```xml - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -эквивалентно следующей конфигурации (начиная с версии `24.1`): - -```xml - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -Следующая конфигурация: - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -равно: - -```xml - - object_storage - s3 - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -Пример полной конфигурации хранилища выглядит следующим образом: - -```xml - - - - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -Начиная с версии 24.1, это может выглядеть, например, так: - - -```xml - - - - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -Чтобы сделать определённый тип хранилища типом по умолчанию для всех таблиц `MergeTree`, -добавьте следующий раздел в файл конфигурации: - -```xml - - - s3 - - -``` - -Если вы хотите настроить политику хранения для отдельной таблицы, -вы можете указать её в `SETTINGS` при создании таблицы: - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS storage_policy = 's3'; -``` - -Можно также использовать `disk` вместо `storage_policy`. В этом случае -раздел `storage_policy` в конфигурационном файле не обязателен, достаточно -указать раздел `disk`. - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS disk = 's3'; -``` - - -## Динамическая конфигурация {#dynamic-configuration} - -Также есть возможность задать конфигурацию хранилища без заранее определённого -диска в конфигурационном файле, а настроить её в параметрах запросов `CREATE`/`ATTACH`. - -Следующий пример запроса основывается на приведённой выше конфигурации динамического диска и -показывает, как использовать локальный диск для кэширования данных из таблицы, размещённой по URL-адресу. - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -В следующем примере к внешнему хранилищу добавляется кэш. - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) --- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); --- highlight-end -``` - -В настройках, показанных ниже, обратите внимание, что диск с `type=web` вложен в -диск с `type=cache`. - -:::note -В примере используется `type=web`, но любой тип диска может быть настроен как динамический, -включая локальный диск. Для локальных дисков требуется, чтобы аргумент path находился внутри -каталога, заданного в параметре конфигурации сервера `custom_local_disks_base_directory`, который не имеет -значения по умолчанию, поэтому при использовании локального диска задайте и его. -::: - -Также возможна комбинация конфигурации, задаваемой в config-файлах, и конфигурации, определяемой в SQL: - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); - -- highlight-end -``` - -где `web` берётся из файла конфигурации сервера: - - -```xml - - - - web - 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - - - -``` - -### Использование хранилища S3 {#s3-storage} - -#### Обязательные параметры {#required-parameters-s3} - -| Параметр | Описание | -| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `endpoint` | URL-адрес конечной точки S3 в стиле `path` или `virtual hosted` ([styles](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html)). Должен включать bucket и корневой путь для хранения данных. | -| `access_key_id` | Идентификатор ключа доступа S3, используемый для аутентификации. | -| `secret_access_key` | Секретный ключ доступа S3, используемый для аутентификации. | - -#### Необязательные параметры {#optional-parameters-s3} - - -| Parameter | Description | Default Value | -|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------| -| `region` | Имя региона S3. | - | -| `support_batch_delete` | Управляет проверкой поддержки пакетного удаления. Установите значение `false` при использовании Google Cloud Storage (GCS), так как GCS не поддерживает пакетное удаление. | `true` | -| `use_environment_credentials` | Считывает учетные данные AWS из переменных окружения: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` и `AWS_SESSION_TOKEN`, если они существуют. | `false` | -| `use_insecure_imds_request` | Если `true`, использует небезопасный запрос IMDS при получении учетных данных из метаданных Amazon EC2. | `false` | -| `expiration_window_seconds` | Льготный интервал (в секундах), используемый при проверке, истекли ли учетные данные с ограниченным сроком действия. | `120` | -| `proxy` | Конфигурация прокси для конечной точки S3. Каждый элемент `uri` внутри блока `proxy` должен содержать URL прокси. | - | -| `connect_timeout_ms` | Таймаут установления сокет-соединения в миллисекундах. | `10000` (10 секунд) | -| `request_timeout_ms` | Таймаут запроса в миллисекундах. | `5000` (5 секунд) | -| `retry_attempts` | Количество попыток повтора для неуспешных запросов. | `10` | -| `single_read_retries` | Количество попыток повторного подключения при обрыве соединения во время чтения. | `4` | -| `min_bytes_for_seek` | Минимальное количество байт, при котором используется операция `seek` вместо последовательного чтения. | `1 MB` | -| `metadata_path` | Локальный путь файловой системы для хранения файлов метаданных S3. | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | Если `true`, пропускает проверку доступа к диску во время запуска. | `false` | -| `header` | Добавляет указанный HTTP-заголовок к запросам. Может быть указан несколько раз. | - | -| `server_side_encryption_customer_key_base64` | Необходимые заголовки для доступа к объектам S3 с шифрованием SSE-C. | - | -| `server_side_encryption_kms_key_id` | Необходимые заголовки для доступа к объектам S3 с [шифрованием SSE-KMS](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). Пустая строка означает использование управляемого AWS ключа для S3. | - | -| `server_side_encryption_kms_encryption_context` | Заголовок контекста шифрования для SSE-KMS (используется с `server_side_encryption_kms_key_id`). | - | -| `server_side_encryption_kms_bucket_key_enabled` | Включает ключи бакета S3 для SSE-KMS (используется с `server_side_encryption_kms_key_id`). | Соответствует настройке на уровне бакета | -| `s3_max_put_rps` | Максимальное количество запросов PUT в секунду до начала ограничения пропускной способности. | `0` (без ограничений) | -| `s3_max_put_burst` | Максимальное количество одновременных запросов PUT до достижения лимита RPS. | То же, что и `s3_max_put_rps` | -| `s3_max_get_rps` | Максимальное количество запросов GET в секунду до начала ограничения пропускной способности. | `0` (без ограничений) | -| `s3_max_get_burst` | Максимальное количество одновременных запросов GET до достижения лимита RPS. | То же, что и `s3_max_get_rps` | -| `read_resource` | Имя ресурса для [планирования](/operations/workload-scheduling.md) запросов чтения. | Пустая строка (отключено) | -| `write_resource` | Имя ресурса для [планирования](/operations/workload-scheduling.md) запросов записи. | Пустая строка (отключено) | -| `key_template` | Определяет формат генерации ключа объекта, используя синтаксис [re2](https://github.com/google/re2/wiki/Syntax). Требует флага `storage_metadata_write_full_object_key`. Несовместим с `root path` в `endpoint`. Требует `key_compatibility_prefix`. | - | -| `key_compatibility_prefix` | Обязателен при использовании `key_template`. Указывает предыдущий `root path` из `endpoint` для чтения более старых версий метаданных. | - | -| `read_only` | Разрешает только чтение с диска. | - | -:::note -Google Cloud Storage (GCS) также поддерживается с использованием типа `s3`. См. [MergeTree с хранилищем GCS](/integrations/gcs). -::: - -### Использование Plain Storage {#plain-storage} - - - -В версии `22.10` был введён новый тип диска `s3_plain`, который предоставляет хранилище с однократной записью. -Параметры его конфигурации такие же, как у типа диска `s3`. -В отличие от типа диска `s3`, он хранит данные как есть. Другими словами, -вместо случайно сгенерированных имён блобов он использует обычные имена файлов -(так же, как ClickHouse хранит файлы на локальном диске) и не хранит никаких -метаданных локально. Например, метаданные восстанавливаются из данных в `s3`. - -Этот тип диска позволяет сохранять статическую версию таблицы, так как он не -позволяет выполнять слияния существующих данных и не позволяет вставлять новые -данные. Один из вариантов использования этого типа диска — создание на нём резервных копий, что можно сделать -через `BACKUP TABLE data TO Disk('plain_disk_name', 'backup_name')`. После этого -можно выполнить `RESTORE TABLE data AS data_restored FROM Disk('plain_disk_name', 'backup_name')` -или использовать `ATTACH TABLE data (...) ENGINE = MergeTree() SETTINGS disk = 'plain_disk_name'`. - -Конфигурация: - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -Начиная с версии `24.1` можно настраивать любой диск объектного хранилища (`s3`, `azure`, `hdfs` (не поддерживается), `local`), используя тип метаданных `plain`. - -Конфигурация: - -```xml - - object_storage - azure - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -### Использование S3 Plain Rewritable Storage {#s3-plain-rewritable-storage} - -Новый тип диска `s3_plain_rewritable` был представлен в версии `24.4`. -Аналогично типу диска `s3_plain`, он не требует дополнительного хранилища для -файлов метаданных. Вместо этого метаданные хранятся в S3. -В отличие от типа диска `s3_plain`, `s3_plain_rewritable` позволяет выполнять -слияния и поддерживает операции `INSERT`. -[Мутации](/sql-reference/statements/alter#mutations) и репликация таблиц не поддерживаются. - -Один из вариантов использования этого типа диска — нереплицируемые таблицы `MergeTree`. Хотя -тип диска `s3` подходит для нереплицируемых таблиц `MergeTree`, вы можете выбрать -тип диска `s3_plain_rewritable`, если вам не требуются локальные метаданные -для таблицы и вы готовы принять ограниченный набор операций. Это может -быть полезно, например, для системных таблиц. - -Конфигурация: - -```xml - - s3_plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -равно - -```xml - - object_storage - s3 - plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -Начиная с версии `24.5` можно настроить любой диск объектного хранилища -(`s3`, `azure`, `local`) с использованием типа метаданных `plain_rewritable`. - -### Использование Azure Blob Storage {#azure-blob-storage} - -Семейство движков таблиц `MergeTree` может сохранять данные в [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/) -с использованием диска типа `azure_blob_storage`. - -Фрагмент конфигурации: - - -```xml - - ... - - - azure_blob_storage - http://account.blob.core.windows.net - container - account - pass123 - /var/lib/clickhouse/disks/blob_storage_disk/ - /var/lib/clickhouse/disks/blob_storage_disk/cache/ - false - - - ... - -``` - -#### Параметры подключения {#azure-blob-storage-connection-parameters} - -| Parameter | Description | Default Value | -| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------- | -| `storage_account_url` (Required) | URL аккаунта Azure Blob Storage. Примеры: `http://account.blob.core.windows.net` или `http://azurite1:10000/devstoreaccount1`. | - | -| `container_name` | Имя целевого контейнера. | `default-container` | -| `container_already_exists` | Управляет поведением при создании контейнера:
- `false`: создаёт новый контейнер
- `true`: подключается напрямую к существующему контейнеру
- не задан: проверяет существование контейнера и создаёт его при необходимости | - | - -Параметры аутентификации (диск попробует все доступные методы **и** Managed Identity Credential): - -| Parameter | Description | -| ------------------- | ------------------------------------------------------------------------------- | -| `connection_string` | Для аутентификации с использованием строки подключения. | -| `account_name` | Для аутентификации с использованием Shared Key (используется с `account_key`). | -| `account_key` | Для аутентификации с использованием Shared Key (используется с `account_name`). | - -#### Параметры ограничения {#azure-blob-storage-limit-parameters} - -| Parameter | Description | -| ------------------------------------ | -------------------------------------------------------------------------------- | -| `s3_max_single_part_upload_size` | Максимальный размер одиночной загрузки блока в Blob Storage. | -| `min_bytes_for_seek` | Минимальный размер области, доступной для произвольного чтения (seek). | -| `max_single_read_retries` | Максимальное число попыток чтения фрагмента данных из Blob Storage. | -| `max_single_download_retries` | Максимальное число попыток загрузки буфера данных из Blob Storage. | -| `thread_pool_size` | Максимальное количество потоков для создания экземпляров `IDiskRemote`. | -| `s3_max_inflight_parts_for_one_file` | Максимальное количество параллельных запросов на загрузку частей одного объекта. | - -#### Прочие параметры {#azure-blob-storage-other-parameters} - -| Parameter | Description | Default Value | -| -------------------------------- | ------------------------------------------------------------------------------------------- | ---------------------------------------- | -| `metadata_path` | Локальный путь в файловой системе для хранения файлов метаданных для Blob Storage. | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | Если `true`, пропускает проверку доступа к диску при запуске. | `false` | -| `read_resource` | Имя ресурса для [планирования](/operations/workload-scheduling.md) запросов на чтение. | Пустая строка (отключено) | -| `write_resource` | Имя ресурса для [планирования](/operations/workload-scheduling.md) запросов на запись. | Пустая строка (отключено) | -| `metadata_keep_free_space_bytes` | Объём свободного дискового пространства под метаданные, который необходимо зарезервировать. | - | - -Примеры рабочих конфигураций можно найти в каталоге интеграционных тестов (см., например, [test_merge_tree_azure_blob_storage](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_merge_tree_azure_blob_storage/configs/config.d/storage_conf.xml) или [test_azure_blob_storage_zero_copy_replication](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_azure_blob_storage_zero_copy_replication/configs/config.d/storage_conf.xml)). - -:::note Репликация без копирования (zero-copy) не готова к промышленной эксплуатации -Репликация без копирования (zero-copy) по умолчанию отключена в ClickHouse версии 22.8 и выше. Эта функция не рекомендуется для использования в production. -::: - - -## Использование хранилища HDFS (не поддерживается) {#using-hdfs-storage-unsupported} - -В этой примерной конфигурации: - -* диск имеет тип `hdfs` (не поддерживается) -* данные размещены по адресу `hdfs://hdfs1:9000/clickhouse/` - -Обратите внимание, что HDFS не поддерживается, поэтому при его использовании могут возникать проблемы. При возникновении каких-либо проблем вы можете отправить pull request с исправлением. - -```xml - - - - - hdfs - hdfs://hdfs1:9000/clickhouse/ - true - - - local - / - - - - - -
- hdfs -
- - hdd - -
-
-
-
-
-``` - -Имейте в виду, что HDFS может не работать в некоторых крайних случаях. - -### Использование шифрования данных {#encrypted-virtual-file-system} - -Вы можете шифровать данные, хранящиеся на внешних дисках [S3](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3) или [HDFS](#using-hdfs-storage-unsupported) (не поддерживается), а также на локальном диске. Чтобы включить режим шифрования, в конфигурационном файле нужно определить диск с типом `encrypted` и выбрать диск, на котором будут сохраняться данные. Диск типа `encrypted` шифрует все записываемые файлы на лету, а при чтении файлов с диска `encrypted` автоматически расшифровывает их. Таким образом, вы можете работать с диском `encrypted` так же, как с обычным диском. - -Пример конфигурации диска: - -```xml - - - local - /path1/ - - - encrypted - disk1 - path2/ - _16_ascii_chars_ - - -``` - -Например, когда ClickHouse записывает данные из некоторой таблицы в файл `store/all_1_1_0/data.bin` на `disk1`, на самом деле этот файл будет записан на физический диск по пути `/path1/store/all_1_1_0/data.bin`. - -При записи того же файла на `disk2` он фактически будет записан на физический диск по пути `/path1/path2/store/all_1_1_0/data.bin` в зашифрованном виде. - -### Обязательные параметры {#required-parameters-encrypted-disk} - -| Параметр | Тип | Описание | -| -------- | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `type` | String | Должен быть установлен в `encrypted` для создания зашифрованного диска. | -| `disk` | String | Тип диска, используемого в качестве базового хранилища. | -| `key` | Uint64 | Ключ для шифрования и расшифровки. Может быть указан в шестнадцатеричном формате с использованием `key_hex`. Несколько ключей можно задать с помощью атрибута `id`. | - -### Необязательные параметры {#optional-parameters-encrypted-disk} - -| Параметр | Тип | Значение по умолчанию | Описание | -| ---------------- | ------ | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `path` | String | Корневой каталог | Расположение на диске, где будут сохраняться данные. | -| `current_key_id` | String | - | Идентификатор ключа, используемого для шифрования. Все указанные ключи могут использоваться для расшифровки. | -| `algorithm` | Enum | `AES_128_CTR` | Алгоритм шифрования. Варианты:
- `AES_128_CTR` (16-байтный ключ)
- `AES_192_CTR` (24-байтный ключ)
- `AES_256_CTR` (32-байтный ключ) | - -Пример конфигурации диска: - - -```xml - - - - - s3 - ... - - - encrypted - disk_s3 - AES_128_CTR - 00112233445566778899aabbccddeeff - ffeeddccbbaa99887766554433221100 - 1 - - - - -``` - -### Использование локального кэша {#using-local-cache} - -Начиная с версии 22.3 можно настроить локальный кэш поверх дисков в конфигурации хранилища. -Для версий 22.3–22.7 кэш поддерживается только для дисков типа `s3`. Для версий >= 22.8 кэш поддерживается для любого типа диска: S3, Azure, Local, Encrypted и т. д. -Для версий >= 23.5 кэш поддерживается только для удалённых типов дисков: S3, Azure, HDFS (не поддерживается). -Кэш использует политику кэширования `LRU`. - -Пример конфигурации для версий 22.8 и выше: - -```xml - - - - - s3 - ... - ... конфигурация S3 ... - - - cache - s3 - /s3_cache/ - 10Gi - - - - - -
- cache -
-
-
- -
-``` - -Пример конфигурации для версий до 22.8: - -```xml - - - - - s3 - ... - ... конфигурация S3 ... - 1 - 10737418240 - - - - - -
- s3 -
-
-
- -
-``` - -Параметры конфигурации **диска файлового кэша**: - -Эти параметры следует задавать в разделе конфигурации диска. - - -| Параметр | Тип | Значение по умолчанию | Описание | -|--------------------------------------|---------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `path` | String | - | **Обязательный параметр**. Путь к каталогу, где будет храниться кэш. | -| `max_size` | Size | - | **Обязательный параметр**. Максимальный размер кэша в байтах или удобочитаемом формате (например, `10Gi`). При достижении лимита файлы удаляются по политике LRU. Поддерживаются форматы `ki`, `Mi`, `Gi` (начиная с v22.10). | -| `cache_on_write_operations` | Boolean | `false` | Включает кэширование при записи (write-through cache) для запросов `INSERT` и фоновых слияний. Может быть переопределён на уровне запроса с помощью `enable_filesystem_cache_on_write_operations`. | -| `enable_filesystem_query_cache_limit`| Boolean | `false` | Включает ограничение размера кэша для каждого запроса на основе `max_query_cache_size`. | -| `enable_cache_hits_threshold` | Boolean | `false` | Если включено, данные кэшируются только после многократного чтения. | -| `cache_hits_threshold` | Integer | `0` | Количество чтений, необходимое перед кэшированием данных (требует `enable_cache_hits_threshold`). | -| `enable_bypass_cache_with_threshold` | Boolean | `false` | Не использует кэш для больших диапазонов чтения. | -| `bypass_cache_threshold` | Size | `256Mi` | Размер диапазона чтения, при превышении которого кэш не используется (требует `enable_bypass_cache_with_threshold`). | -| `max_file_segment_size` | Size | `8Mi` | Максимальный размер одного файла кэша в байтах или удобочитаемом формате. | -| `max_elements` | Integer | `10000000` | Максимальное количество файлов кэша. | -| `load_metadata_threads` | Integer | `16` | Количество потоков для загрузки метаданных кэша при запуске. | - -> **Примечание**: Значения размера поддерживают единицы, такие как `ki`, `Mi`, `Gi` и т. д. (например, `10Gi`). - - - -## Параметры файлового кэша для запросов/профилей {#file-cache-query-profile-settings} - -| Параметр | Тип | По умолчанию | Описание | -| ----------------------------------------------------------------------- | ------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `enable_filesystem_cache` | Boolean | `true` | Включает/выключает использование кэша для конкретного запроса, даже если используется диск типа `cache`. | -| `read_from_filesystem_cache_if_exists_otherwise_bypass_cache` | Boolean | `false` | При включении читает данные только из кэша, если они уже там есть; новые данные не будут кэшироваться. | -| `enable_filesystem_cache_on_write_operations` | Boolean | `false` (Cloud: `true`) | Включает сквозное кэширование при операциях записи (write-through cache). Требует параметра `cache_on_write_operations` в конфигурации кэша. | -| `enable_filesystem_cache_log` | Boolean | `false` | Включает детализированное логирование использования кэша в `system.filesystem_cache_log`. | -| `filesystem_cache_allow_background_download` | Boolean | `true` | Разрешает завершать частично скачанные сегменты в фоновом режиме. Отключите, чтобы все скачивания выполнялись в основном потоке для текущего запроса/сессии. | -| `max_query_cache_size` | Size | `false` | Максимальный размер кэша на один запрос. Требует параметра `enable_filesystem_query_cache_limit` в конфигурации кэша. | -| `filesystem_cache_skip_download_if_exceeds_per_query_cache_write_limit` | Boolean | `true` | Управляет поведением при достижении `max_query_cache_size`:
- `true`: останавливает скачивание новых данных
- `false`: вытесняет старые данные, чтобы освободить место | - -:::warning -Параметры конфигурации кэша и параметры кэша на уровне запроса соответствуют последней версии ClickHouse; -в более ранних версиях часть параметров может не поддерживаться. -::: - -#### Системные таблицы кэша {#cache-system-tables-file-cache} - -| Имя таблицы | Описание | Требования | -| ----------------------------- | ------------------------------------------------------------------ | ---------------------------------------------- | -| `system.filesystem_cache` | Отображает текущее состояние файлового кэша. | Нет | -| `system.filesystem_cache_log` | Предоставляет детальную статистику использования кэша по запросам. | Требуется `enable_filesystem_cache_log = true` | - -#### Команды работы с кэшем {#cache-commands-file-cache} - -##### `SYSTEM DROP FILESYSTEM CACHE () (ON CLUSTER)` -- `ON CLUSTER` {#system-drop-filesystem-cache-on-cluster} - -Эта команда поддерживается только если `` не указан. - -##### `SHOW FILESYSTEM CACHES` {#show-filesystem-caches} - -Показывает список файловых кэшей, настроенных на сервере. -(Для версий меньше либо равных `22.8` команда называется `SHOW CACHES`) - -```sql title="Query" -SHOW FILESYSTEM CACHES -``` - -```text title="Response" -┌─Кеши──────┐ -│ s3_cache │ -└───────────┘ -``` - -##### `DESCRIBE FILESYSTEM CACHE ''` {#describe-filesystem-cache} - -Показывает конфигурацию кэша и некоторые общие статистические сведения для конкретного кэша. -Имя кэша можно получить из команды `SHOW FILESYSTEM CACHES`. (Для версий меньше или равных `22.8` команда называется `DESCRIBE CACHE`) - -```sql title="Query" -DESCRIBE FILESYSTEM CACHE 's3_cache' -``` - -```text title="Response" -┌────max_size─┬─max_elements─┬─max_file_segment_size─┬─boundary_alignment─┬─cache_on_write_operations─┬─cache_hits_threshold─┬─current_size─┬─current_elements─┬─path───────┬─background_download_threads─┬─enable_bypass_cache_with_threshold─┐ -│ 10000000000 │ 1048576 │ 104857600 │ 4194304 │ 1 │ 0 │ 3276 │ 54 │ /s3_cache/ │ 2 │ 0 │ -└─────────────┴──────────────┴───────────────────────┴────────────────────┴───────────────────────────┴──────────────────────┴──────────────┴──────────────────┴────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - - -| Кэш текущих метрик | Кэш асинхронных метрик | Кэш событий профиля | -| ------------------------- | ---------------------- | ----------------------------------------------------------------------------------------- | -| `FilesystemCacheSize` | `FilesystemCacheBytes` | `CachedReadBufferReadFromSourceBytes`, `CachedReadBufferReadFromCacheBytes` | -| `FilesystemCacheElements` | `FilesystemCacheFiles` | `CachedReadBufferReadFromSourceMicroseconds`, `CachedReadBufferReadFromCacheMicroseconds` | -| | | `CachedReadBufferCacheWriteBytes`, `CachedReadBufferCacheWriteMicroseconds` | -| | | `CachedWriteBufferCacheWriteBytes`, `CachedWriteBufferCacheWriteMicroseconds` | - -### Использование статического Web-хранилища (только для чтения) {#web-storage} - -Это диск только для чтения. Данные на нём только читаются и никогда не изменяются. Новая таблица -загружается на этот диск с помощью запроса `ATTACH TABLE` (см. пример ниже). Локальный диск -фактически не используется, каждый запрос `SELECT` будет приводить к HTTP-запросу для -получения необходимых данных. Любое изменение данных таблицы приведёт к -исключению, то есть следующие типы запросов не допускаются: [`CREATE TABLE`](/sql-reference/statements/create/table.md), -[`ALTER TABLE`](/sql-reference/statements/alter/index.md), [`RENAME TABLE`](/sql-reference/statements/rename#rename-table), -[`DETACH TABLE`](/sql-reference/statements/detach.md) и [`TRUNCATE TABLE`](/sql-reference/statements/truncate.md). -Web-хранилище может использоваться только для чтения. Пример использования — публикация -примеров данных или миграция данных. Существует утилита `clickhouse-static-files-uploader`, -которая подготавливает каталог данных для заданной таблицы (`SELECT data_paths FROM system.tables WHERE name = 'table_name'`). -Для каждой необходимой таблицы вы получаете каталог файлов. Эти файлы можно загрузить, -например, на веб-сервер со статическими файлами. После такой подготовки -вы можете загрузить эту таблицу на любой сервер ClickHouse через `DiskWeb`. - -В этой примерной конфигурации: - -* диск имеет тип `web` -* данные размещены по адресу `http://nginx:80/test1/` -* используется кэш на локальном хранилище - -```xml - - - - - web - http://nginx:80/test1/ - - - cache - web - cached_web_cache/ - 100000000 - - - - - -
- web -
-
-
- - -
- cached_web -
-
-
-
-
-
-``` - -:::tip -Хранилище также можно временно задать в самом запросе, если веб-набор данных -не планируется использовать регулярно. См. [динамическую конфигурацию](#dynamic-configuration) и -пропустите редактирование файла конфигурации. - -[Демонстрационный набор данных](https://github.com/ClickHouse/web-tables-demo) размещён в GitHub. Чтобы подготовить собственные таблицы для веб-хранилища, используйте инструмент [clickhouse-static-files-uploader](/operations/utilities/static-files-disk-uploader) -::: - -В этом запросе `ATTACH TABLE` указанное `UUID` совпадает с именем каталога с данными, а endpoint — это URL для необработанного содержимого на GitHub. - - -```sql --- highlight-next-line -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('прочее' = 0, 'рядная застройка' = 1, 'двухквартирный дом' = 2, 'отдельный дом' = 3, 'квартира' = 4), - is_new UInt8, - duration Enum8('неизвестно' = 0, 'собственность' = 1, 'аренда' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -Готовый тестовый пример. Нужно добавить эту конфигурацию в config: - -```xml - - - - - web - https://clickhouse-datasets.s3.yandex.net/disk-with-static-files-tests/test-hits/ - - - - - -
- web -
-
-
-
-
-
-``` - -Затем выполните этот запрос: - - -```sql -ATTACH TABLE test_hits UUID '1ae36516-d62d-4218-9ae3-6516d62da218' -( - WatchID UInt64, - JavaEnable UInt8, - Title String, - GoodEvent Int16, - EventTime DateTime, - EventDate Date, - CounterID UInt32, - ClientIP UInt32, - ClientIP6 FixedString(16), - RegionID UInt32, - UserID UInt64, - CounterClass Int8, - OS UInt8, - UserAgent UInt8, - URL String, - Referer String, - URLDomain String, - RefererDomain String, - Refresh UInt8, - IsRobot UInt8, - RefererCategories Array(UInt16), - URLCategories Array(UInt16), - URLRegions Array(UInt32), - RefererRegions Array(UInt32), - ResolutionWidth UInt16, - ResolutionHeight UInt16, - ResolutionDepth UInt8, - FlashMajor UInt8, - FlashMinor UInt8, - FlashMinor2 String, - NetMajor UInt8, - NetMinor UInt8, - UserAgentMajor UInt16, - UserAgentMinor FixedString(2), - CookieEnable UInt8, - JavascriptEnable UInt8, - IsMobile UInt8, - MobilePhone UInt8, - MobilePhoneModel String, - Params String, - IPNetworkID UInt32, - TraficSourceID Int8, - SearchEngineID UInt16, - SearchPhrase String, - AdvEngineID UInt8, - IsArtifical UInt8, - WindowClientWidth UInt16, - WindowClientHeight UInt16, - ClientTimeZone Int16, - ClientEventTime DateTime, - SilverlightVersion1 UInt8, - SilverlightVersion2 UInt8, - SilverlightVersion3 UInt32, - SilverlightVersion4 UInt16, - PageCharset String, - CodeVersion UInt32, - IsLink UInt8, - IsDownload UInt8, - IsNotBounce UInt8, - FUniqID UInt64, - HID UInt32, - IsOldCounter UInt8, - IsEvent UInt8, - IsParameter UInt8, - DontCountHits UInt8, - WithHash UInt8, - HitColor FixedString(1), - UTCEventTime DateTime, - Age UInt8, - Sex UInt8, - Income UInt8, - Interests UInt16, - Robotness UInt8, - GeneralInterests Array(UInt16), - RemoteIP UInt32, - RemoteIP6 FixedString(16), - WindowName Int32, - OpenerName Int32, - HistoryLength Int16, - BrowserLanguage FixedString(2), - BrowserCountry FixedString(2), - SocialNetwork String, - SocialAction String, - HTTPError UInt16, - SendTiming Int32, - DNSTiming Int32, - ConnectTiming Int32, - ResponseStartTiming Int32, - ResponseEndTiming Int32, - FetchTiming Int32, - RedirectTiming Int32, - DOMInteractiveTiming Int32, - DOMContentLoadedTiming Int32, - DOMCompleteTiming Int32, - LoadEventStartTiming Int32, - LoadEventEndTiming Int32, - NSToDOMContentLoadedTiming Int32, - FirstPaintTiming Int32, - RedirectCount Int8, - SocialSourceNetworkID UInt8, - SocialSourcePage String, - ParamPrice Int64, - ParamOrderID String, - ParamCurrency FixedString(3), - ParamCurrencyID UInt16, - GoalsReached Array(UInt32), - OpenstatServiceName String, - OpenstatCampaignID String, - OpenstatAdID String, - OpenstatSourceID String, - UTMSource String, - UTMMedium String, - UTMCampaign String, - UTMContent String, - UTMTerm String, - FromTag String, - HasGCLID UInt8, - RefererHash UInt64, - URLHash UInt64, - CLID UInt32, - YCLID UInt64, - ShareService String, - ShareURL String, - ShareTitle String, - ParsedParams Nested( - Key1 String, - Key2 String, - Key3 String, - Key4 String, - Key5 String, - ValueDouble Float64), - IslandID FixedString(16), - RequestNum UInt32, - RequestTry UInt8 -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID) -SETTINGS storage_policy='web'; -``` - -#### Обязательные параметры {#static-web-storage-required-parameters} - -| Параметр | Описание | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------- | -| `type` | `web`. В противном случае диск не создаётся. | -| `endpoint` | URL эндпоинта в формате `path`. URL эндпоинта должен содержать корневой путь к каталогу хранения данных, в который они были загружены. | - -#### Необязательные параметры {#optional-parameters-web} - - -| Параметр | Описание | Значение по умолчанию | -|-------------------------------------|-------------------------------------------------------------------------------|------------------------| -| `min_bytes_for_seek` | Минимальное количество байт, при котором используется операция seek вместо последовательного чтения | `1` МБ | -| `remote_fs_read_backoff_threashold` | Максимальное время ожидания при попытке чтения данных с удалённого диска | `10000` секунд | -| `remote_fs_read_backoff_max_tries` | Максимальное количество попыток чтения с использованием backoff | `5` | - -Если запрос завершается с исключением `DB:Exception Unreachable URL`, можно попробовать отрегулировать настройки: [http_connection_timeout](/operations/settings/settings.md/#http_connection_timeout), [http_receive_timeout](/operations/settings/settings.md/#http_receive_timeout), [keep_alive_timeout](/operations/server-configuration-parameters/settings#keep_alive_timeout). - -Чтобы получить файлы для загрузки, выполните: -`clickhouse static-files-disk-uploader --metadata-path --output-dir
` (`--metadata-path` можно получить запросом `SELECT data_paths FROM system.tables WHERE name = 'table_name'`). - -При загрузке файлов по `endpoint` их необходимо загружать в путь `/store/`, но в конфигурации должен быть указан только `endpoint`. - -Если URL недоступен при загрузке диска во время старта сервера и инициализации таблиц, то все ошибки перехватываются. Если в этом случае возникли ошибки, таблицы можно перезагрузить (они станут видимыми) с помощью `DETACH TABLE table_name` -> `ATTACH TABLE table_name`. Если метаданные были успешно загружены при старте сервера, таблицы сразу становятся доступными. - -Используйте настройку [http_max_single_read_retries](/operations/storing-data#web-storage) для ограничения максимального количества повторных попыток во время одного HTTP-чтения. - -### Репликация без копирования данных (Zero-copy Replication, не готово для production) {#zero-copy} - -Репликация без копирования данных (zero-copy) возможна, но не рекомендуется, для дисков `S3` и `HDFS` (для `HDFS` не поддерживается). Репликация без копирования данных означает, что если данные хранятся удалённо на нескольких машинах и должны быть синхронизированы, то реплицируются только метаданные (пути к частям данных), а не сами данные. - -:::note Репликация без копирования данных не готова для production -Репликация без копирования данных по умолчанию отключена в ClickHouse версии 22.8 и выше. Эта функция не рекомендуется для использования в production. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md deleted file mode 100644 index f1aa10ae2f1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об асинхронных вставках. Каждая запись представляет запрос INSERT, помещённый в буфер асинхронных вставок.' -keywords: ['системная таблица', 'asynchronous_insert_log'] -slug: /operations/system-tables/asynchronous_insert_log -title: 'system.asynchronous_insert_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_insert_log {#systemasynchronous_insert_log} - - - -Содержит информацию об асинхронных вставках. Каждая запись соответствует запросу INSERT, поставленному в буфер для асинхронной вставки. - -Чтобы начать вести журнал, настройте параметры в разделе [asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log). - -Период сброса данных задаётся параметром `flush_interval_milliseconds` в разделе настроек сервера [asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log). Чтобы принудительно выполнить сброс, используйте запрос [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs). - -ClickHouse не удаляет данные из таблицы автоматически. Подробнее см. в разделе [Введение](/operations/system-tables/overview#system-tables-introduction). - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата, когда была выполнена асинхронная вставка. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время завершения выполнения асинхронной вставки. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Дата и время завершения выполнения асинхронной вставки с точностью до микросекунд. -* `query` ([String](../../sql-reference/data-types/string.md)) — Строка запроса. -* `database` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой находится таблица. -* `table` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы. -* `format` ([String](/sql-reference/data-types/string.md)) — Имя формата. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — ID исходного запроса. -* `bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество вставленных байт. -* `exception` ([String](../../sql-reference/data-types/string.md)) — Сообщение об исключении. -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — Статус представления. Значения: - * `'Ok' = 1` — Успешная вставка. - * `'ParsingError' = 2` — Исключение при разборе данных. - * `'FlushError' = 3` — Исключение при сбросе данных. -* `flush_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время, когда был выполнен сброс. -* `flush_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Дата и время, когда был выполнен сброс, с точностью до микросекунд. -* `flush_query_id` ([String](../../sql-reference/data-types/string.md)) — ID запроса сброса. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.asynchronous_insert_log LIMIT 1 \G; -``` - -Результат: - -```text -hostname: clickhouse.eu-central1.internal -event_date: 2023-06-08 -event_time: 2023-06-08 10:08:53 -event_time_microseconds: 2023-06-08 10:08:53.199516 -query: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -database: public -table: data_guess -format: CSV -query_id: b46cd4c4-0269-4d0b-99f5-d27668c6102e -bytes: 133223 -exception: -status: Ok -flush_time: 2023-06-08 10:08:55 -flush_time_microseconds: 2023-06-08 10:08:55.139676 -flush_query_id: cd2c1e43-83f5-49dc-92e4-2fbc7f8d3716 -``` - -**См. также** - -* [system.query_log](../../operations/system-tables/query_log) — Описание системной таблицы `query_log`, которая содержит общую информацию о выполнении запросов. -* [system.asynchronous_inserts](/operations/system-tables/asynchronous_inserts) — Эта таблица содержит информацию о находящихся в очереди асинхронных вставках. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md deleted file mode 100644 index a0be1cbd24b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об асинхронных вставках, ожидающих выполнения в очереди.' -keywords: ['system table', 'asynchronous_inserts'] -slug: /operations/system-tables/asynchronous_inserts -title: 'system.asynchronous_inserts' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию об асинхронных вставках, ожидающих обработки в очереди. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.asynchronous_inserts LIMIT 1 \G; -``` - -Результат: - -```text -Строка 1: -────── -query: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -database: public -table: data_guess -format: CSV -first_update: 2023-06-08 10:08:54.199606 -total_bytes: 133223 -entries.query_id: ['b46cd4c4-0269-4d0b-99f5-d27668c6102e'] -entries.bytes: [133223] -``` - -**См. также** - -* [system.query_log](/operations/system-tables/query_log) — Описание системной таблицы `query_log`, которая содержит общую информацию о выполнении запросов. -* [system.asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) — Эта таблица содержит информацию о выполненных асинхронных вставках. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md deleted file mode 100644 index 31cea42ba06..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о последних асинхронных задачах и их состоянии (например, для загружаемых таблиц). Таблица содержит строку для каждой задачи.' -keywords: ['system table', 'asynchronous_loader'] -slug: /operations/system-tables/asynchronous_loader -title: 'system.asynchronous_loader' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_loader {#systemasynchronous_loader} - - - -Содержит информацию и статус недавних асинхронных задач (например, по загрузке таблиц). В таблице содержится по одной строке на каждую задачу. Для визуализации информации из этой таблицы существует утилита `utils/async_loader_graph`. - -Пример: - -```sql -SELECT * -FROM system.asynchronous_loader -LIMIT 1 -FORMAT Vertical -``` - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Ожидающая задача может находиться в одном из следующих состояний: - -* `is_executing` (`UInt8`) — Задача в данный момент выполняется воркером. -* `is_blocked` (`UInt8`) — Задача ожидает завершения своих зависимостей. -* `is_ready` (`UInt8`) — Задача готова к выполнению и ожидает воркера. -* `elapsed` (`Float64`) — Количество секунд, прошедших с начала выполнения. Ноль, если задача не запущена. Полное время выполнения, если задача завершена. - -У каждой задачи есть связанный с ней пул, и она запускается в этом пуле. У каждого пула есть фиксированный приоритет и изменяемое максимальное число воркеров. Сначала выполняются задачи с более высоким приоритетом (меньшим значением `priority`). Ни одна задача с более низким приоритетом не запускается, пока существует хотя бы одна задача с более высоким приоритетом, которая готова или выполняется. Приоритет задачи может быть повышен (но не понижен) за счёт её приоритизации. Например, задачи для загрузки и инициализации таблицы будут приоритизированы, если входящий запрос требует эту таблицу. Можно приоритизировать задачу во время её выполнения, но задача не переносится из её `execution_pool` во вновь назначенный `pool`. Задача использует `pool` для создания новых задач, чтобы избежать инверсии приоритетов. Уже запущенные задачи не вытесняются задачами с более высоким приоритетом и всегда выполняются до завершения после старта. - -* `pool_id` (`UInt64`) — ID пула, который в настоящий момент назначен задаче. - -* `pool` (`String`) — Имя пула `pool_id`. - -* `priority` (`Int64`) — Приоритет пула `pool_id`. - -* `execution_pool_id` (`UInt64`) — ID пула, в котором выполняется задача. Совпадает с изначально назначенным пулом до начала выполнения. - -* `execution_pool` (`String`) — Имя пула `execution_pool_id`. - -* `execution_priority` (`Int64`) — Приоритет пула `execution_pool_id`. - -* `ready_seqno` (`Nullable(UInt64)`) — Не равно `NULL` для готовых задач. Воркер забирает следующую задачу для выполнения из очереди готовых задач своего пула. Если есть несколько готовых задач, выбирается задача с наименьшим значением `ready_seqno`. - -* `waiters` (`UInt64`) — Количество потоков, ожидающих эту задачу. - -* `exception` (`Nullable(String)`) — Не равно `NULL` для задач, завершившихся с ошибкой и отменённых задач. Содержит сообщение об ошибке, возникшей во время выполнения запроса, или об ошибке, которая привела к отмене этой задачи, а также цепочку зависимостей с именами задач, завершившихся с ошибкой. - -Моменты времени в течение жизненного цикла задачи: - -* `schedule_time` (`DateTime64`) — Время, когда задача была создана и запланирована к выполнению (обычно вместе со всеми её зависимостями). -* `enqueue_time` (`Nullable(DateTime64)`) — Время, когда задача стала готовой и была помещена в очередь готовых задач своего пула. `NULL`, если задача ещё не готова. -* `start_time` (`Nullable(DateTime64)`) — Время, когда воркер извлекает задачу из очереди готовых задач и начинает её выполнение. `NULL`, если задача ещё не запущена. -* `finish_time` (`Nullable(DateTime64)`) — Время завершения выполнения задачи. `NULL`, если задача ещё не завершена. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md deleted file mode 100644 index 62cd0cfe776..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: 'Системная таблица, содержащая исторические значения для `system.asynchronous_metrics`, - которые сохраняются один раз за временной интервал (по умолчанию — одну секунду)' -keywords: ['системная таблица', 'asynchronous_metric_log'] -slug: /operations/system-tables/asynchronous_metric_log -title: 'system.asynchronous_metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит исторические значения таблицы `system.asynchronous_metrics`, которые сохраняются один раз за интервал времени (по умолчанию — каждую секунду). Включена по умолчанию. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, на котором выполняется запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время события. -* `metric` ([String](../../sql-reference/data-types/string.md)) — имя метрики. -* `value` ([Float64](../../sql-reference/data-types/float.md)) — значение метрики. - -**Пример** - -```sql -SELECT * FROM system.asynchronous_metric_log LIMIT 3 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:07 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0.001 - -Row 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:08 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 - -Row 3: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:09 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 -``` - -**См. также** - -* [параметр asynchronous_metric_log](../../operations/server-configuration-parameters/settings.md#asynchronous_metric_log) — включение и отключение параметра. -* [system.asynchronous_metrics](../system-tables/asynchronous_metrics.md) — содержит метрики, периодически вычисляемые в фоновом режиме. -* [system.metric_log](../system-tables/metric_log.md) — содержит историю значений метрик из таблиц `system.metrics` и `system.events`, которая периодически сбрасывается на диск. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md deleted file mode 100644 index bbcc0cfc139..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md +++ /dev/null @@ -1,632 +0,0 @@ ---- -description: 'Системная таблица, содержащая метрики, которые периодически рассчитываются - в фоновом режиме. Например, объем используемой оперативной памяти.' -keywords: ['системная таблица', 'asynchronous_metrics'] -slug: /operations/system-tables/asynchronous_metrics -title: 'system.asynchronous_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_metrics {#systemasynchronous_metrics} - - - -Содержит метрики, которые периодически вычисляются в фоновом режиме. Например, объём используемой оперативной памяти. - -Столбцы: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — имя метрики. -* `value` ([Float64](../../sql-reference/data-types/float.md)) — значение метрики. -* `description` ([String](../../sql-reference/data-types/string.md)) — описание метрики. - -**Пример** - -```sql -SELECT * FROM system.asynchronous_metrics LIMIT 10 -``` - -```text -┌─metric──────────────────────────────────┬──────value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ AsynchronousMetricsCalculationTimeSpent │ 0.00179053 │ Время в секундах, затраченное на вычисление асинхронных метрик (накладные расходы асинхронных метрик). │ -│ NumberOfDetachedByUserParts │ 0 │ Общее количество кусков, отсоединённых от таблиц MergeTree пользователями с помощью запроса `ALTER TABLE DETACH` (в отличие от неожиданных, повреждённых или игнорируемых кусков). Сервер не учитывает отсоединённые куски, и они могут быть удалены. │ -│ NumberOfDetachedParts │ 0 │ Общее количество кусков, отсоединённых от таблиц MergeTree. Кусок может быть отсоединён пользователем с помощью запроса `ALTER TABLE DETACH` или самим сервером, если кусок повреждён, неожиданный или не нужен. Сервер не учитывает отсоединённые куски, и они могут быть удалены. │ -│ TotalRowsOfMergeTreeTables │ 2781309 │ Общее количество строк (записей), хранящихся во всех таблицах семейства MergeTree. │ -│ TotalBytesOfMergeTreeTables │ 7741926 │ Общий объём данных в байтах (сжатых, включая данные и индексы), хранящихся во всех таблицах семейства MergeTree. │ -│ NumberOfTables │ 93 │ Общее количество таблиц по всем базам данных на сервере, за исключением баз данных, которые не могут содержать таблицы MergeTree. Исключаемые движки баз данных — это те, которые генерируют набор таблиц динамически, такие как `Lazy`, `MySQL`, `PostgreSQL`, `SQlite`. │ -│ NumberOfDatabases │ 6 │ Общее количество баз данных на сервере. │ -│ MaxPartCountForPartition │ 6 │ Максимальное количество кусков на партицию среди всех партиций всех таблиц семейства MergeTree. Значения больше 300 указывают на неправильную конфигурацию, перегрузку или массовую загрузку данных. │ -│ ReplicasSumMergesInQueue │ 0 │ Сумма операций слияния в очереди (ещё не применённых) по всем реплицируемым таблицам. │ -│ ReplicasSumInsertsInQueue │ 0 │ Сумма операций INSERT в очереди (ещё не реплицированных) по всем реплицируемым таблицам. │ -└─────────────────────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -{/*- В отличие от system.events и system.metrics, асинхронные метрики не собраны в простой список в файле исходного кода — они - смешаны с логикой в src/Interpreters/ServerAsynchronousMetrics.cpp. - Для удобства читателя мы явно перечислили их здесь. -*/ } - -## Описание метрик {#metric-descriptions} - -### AsynchronousHeavyMetricsCalculationTimeSpent {#asynchronousheavymetricscalculationtimespent} - -Время в секундах, затраченное на вычисление асинхронных тяжёлых метрик (связанных с таблицами) (это накладные расходы на асинхронные метрики). - -### AsynchronousHeavyMetricsUpdateInterval {#asynchronousheavymetricsupdateinterval} - -Интервал обновления тяжёлых метрик (связанных с таблицами). - -### AsynchronousMetricsCalculationTimeSpent {#asynchronousmetricscalculationtimespent} - -Время в секундах, затраченное на вычисление асинхронных метрик (это накладные расходы на асинхронные метрики). - -### AsynchronousMetricsUpdateInterval {#asynchronousmetricsupdateinterval} - -Интервал обновления метрик. - -### BlockActiveTime_*name* {#blockactivetime_name} - -Время в секундах, в течение которого на блочном устройстве находились в очереди I/O‑запросы. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardBytes_*name* {#blockdiscardbytes_name} - -Количество отброшенных байт на блочном устройстве. Эти операции актуальны для SSD. Операции discard не используются ClickHouse, но могут использоваться другими процессами в системе. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardMerges_*name* {#blockdiscardmerges_name} - -Количество операций discard для блочного устройства, которые были объединены планировщиком ввода-вывода ОС. Эти операции актуальны для SSD. Операции discard не используются ClickHouse, но могут использоваться другими процессами в системе. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardOps_*name* {#blockdiscardops_name} - -Количество операций discard для блочного устройства. Эти операции актуальны для SSD. Операции discard не используются ClickHouse, но могут использоваться другими процессами в системе. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardTime_*name* {#blockdiscardtime_name} - -Время в секундах, затраченное на операции discard для блочного устройства, суммарно по всем операциям. Эти операции актуальны для SSD. Операции discard не используются ClickHouse, но могут использоваться другими процессами в системе. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockInFlightOps_*name* {#blockinflightops_name} - -Это значение показывает количество запросов ввода-вывода, которые были отправлены драйверу устройства, но ещё не завершены. Оно не включает I/O‑запросы, которые находятся в очереди, но ещё не отправлены драйверу устройства. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockQueueTime_*name* {#blockqueuetime_name} - -Это значение показывает количество миллисекунд, в течение которых I/O‑запросы ожидали на этом блочном устройстве. Если одновременно ожидает несколько I/O‑запросов, это значение увеличивается как произведение количества миллисекунд на количество ожидающих запросов. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadBytes_*name* {#blockreadbytes_name} - -Количество байт, прочитанных с блочного устройства. Оно может быть меньше, чем количество байт, прочитанных из файловой системы, из-за использования страничного кэша ОС, который сокращает обращения к I/O. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadMerges_*name* {#blockreadmerges_name} - -Количество операций чтения для блочного устройства, которые были объединены планировщиком ввода-вывода ОС. Это метрика на уровне всей системы, она включает все процессы на хосте, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadOps_*name* {#blockreadops_name} - -Количество операций чтения, запрошенных у блочного устройства. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadTime_*name* {#blockreadtime_name} - -Время в секундах, затраченное на операции чтения, запрошенные у блочного устройства, суммарно по всем операциям. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteBytes_*name* {#blockwritebytes_name} - -Количество байт, записанных на блочное устройство. Оно может быть меньше количества байт, записанных в файловую систему, из‑за использования страничного кэша ОС, который сокращает операции ввода‑вывода. Запись на блочное устройство может произойти позже соответствующей записи в файловую систему из‑за write-through‑кэширования. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteMerges_*name* {#blockwritemerges_name} - -Количество операций записи, запрошенных у блочного устройства и объединённых планировщиком ввода‑вывода ОС. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteOps_*name* {#blockwriteops_name} - -Количество операций записи, запрошенных у блочного устройства. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteTime_*name* {#blockwritetime_name} - -Время в секундах, затраченное на операции записи, запрошенные у блочного устройства, суммарно по всем операциям. Это системная метрика: она учитывает все процессы на хостовой машине, а не только clickhouse-server. Источник: `/sys/block`. См. https://www.kernel.org/doc/Documentation/block/stat.txt - -### CPUFrequencyMHz_*name* {#cpufrequencymhz_name} - -Текущая частота CPU в МГц. Большинство современных CPU динамически регулируют частоту для энергосбережения и Turbo Boost. - -### DictionaryMaxUpdateDelay {#dictionarymaxlastsuccessfulupdatetime} - -Максимальная задержка (в секундах) обновления словаря. - -### DictionaryTotalFailedUpdates {#dictionaryloadfailed} - -Количество ошибок с момента последней успешной загрузки во всех словарях. - -### DiskAvailable_*name* {#diskavailable_name} - -Доступное количество байт на диске (виртуальная файловая система). Удалённые файловые системы могут показывать большое значение, например 16 EiB. - -### DiskTotal_*name* {#disktotal_name} - -Общий размер диска (виртуальной файловой системы) в байтах. Удалённые файловые системы могут показывать большое значение, например 16 EiB. - -### DiskUnreserved_*name* {#diskunreserved_name} - -Доступное количество байт на диске (виртуальная файловая система) без учёта резервов под слияния, выборки и перемещения. Удалённые файловые системы могут показывать большое значение, например 16 EiB. - -### DiskUsed_*name* {#diskused_name} - -Количество использованных байт на диске (виртуальная файловая система). Удалённые файловые системы не всегда предоставляют эту информацию. - -### FilesystemCacheBytes {#filesystemcachebytes} - -Общее количество байт в виртуальной файловой системе `cache`. Этот кэш хранится на диске. - -### FilesystemCacheFiles {#filesystemcachefiles} - -Общее количество кэшированных сегментов файлов в виртуальной файловой системе `cache`. Этот кэш хранится на диске. - -### FilesystemLogsPathAvailableBytes {#filesystemlogspathavailablebytes} - -Доступное количество байт на томе, где смонтирован путь к логам ClickHouse. Если это значение приближается к нулю, следует настроить ротацию логов в конфигурационном файле. - -### FilesystemLogsPathAvailableINodes {#filesystemlogspathavailableinodes} - -Количество доступных inode на томе, где смонтирован путь к логам ClickHouse. - -### FilesystemLogsPathTotalBytes {#filesystemlogspathtotalbytes} - -Размер тома, где смонтирован путь к логам ClickHouse, в байтах. Рекомендуется иметь не менее 10 ГБ под логи. - -### FilesystemLogsPathTotalINodes {#filesystemlogspathtotalinodes} - -Общее количество inode на томе, где смонтирован путь к логам ClickHouse. - -### FilesystemLogsPathUsedBytes {#filesystemlogspathusedbytes} - -Количество использованных байт на томе, где смонтирован путь к логам ClickHouse. - -### FilesystemLogsPathUsedINodes {#filesystemlogspathusedinodes} - -Количество использованных inode на томе, где смонтирован путь к логам ClickHouse. - -### FilesystemMainPathAvailableBytes {#filesystemmainpathavailablebytes} - -Доступное количество байт на томе, где смонтирован основной путь ClickHouse. - -### FilesystemMainPathAvailableINodes {#filesystemmainpathavailableinodes} - -Количество доступных inode на томе, где смонтирован основной путь ClickHouse. Если значение близко к нулю, это указывает на ошибочную конфигурацию, и вы получите ошибку `no space left on device`, даже если диск не заполнен. - -### FilesystemMainPathTotalBytes {#filesystemmainpathtotalbytes} - -Размер тома, где смонтирован основной путь ClickHouse, в байтах. - -### FilesystemMainPathTotalINodes {#filesystemmainpathtotalinodes} - -Общее количество inode на томе, где смонтирован основной путь ClickHouse. Если оно меньше 25 миллионов, это указывает на ошибочную конфигурацию. - -### FilesystemMainPathUsedBytes {#filesystemmainpathusedbytes} - -Используемое количество байт на томе, где смонтирован основной путь ClickHouse. - -### FilesystemMainPathUsedINodes {#filesystemmainpathusedinodes} - -Количество использованных inode на томе, где смонтирован основной путь ClickHouse. Это значение в основном соответствует количеству файлов. - -### HTTPThreads {#httpthreads} - -Количество потоков в сервере HTTP-интерфейса (без TLS). - -### InterserverThreads {#interserverthreads} - -Количество потоков в сервере протокола взаимодействия реплик (без TLS). - -### Jitter {#jitter} - -Разница между моментом времени, когда поток для расчета асинхронных метрик должен был быть запланирован к пробуждению, и моментом времени, когда он фактически был пробуждён. Косвенный индикатор общей задержки и отзывчивости системы. - -### LoadAverage*N* {#loadaveragen} - -Нагрузка на всю систему, усреднённая с экспоненциальным сглаживанием за 1 минуту. Нагрузка представляет собой количество потоков во всех процессах (сущностей, планируемых ядром ОС), которые в данный момент выполняются на CPU, ожидают IO или готовы к выполнению, но в данный момент не запланированы. Это число включает в себя все процессы, а не только clickhouse-server. Значение может быть больше количества ядер CPU, если система перегружена и многие процессы готовы к выполнению, но ожидают CPU или IO. - -### MaxPartCountForPartition {#maxpartcountforpartition} - -Максимальное количество кусков (parts) на партицию среди всех партиций всех таблиц семейства MergeTree. Значения больше 300 указывают на ошибочную конфигурацию, перегрузку или массовую загрузку данных. - -### MemoryCode {#memorycode} - -Объём виртуальной памяти, отображённой на страницы машинного кода процесса сервера, в байтах. - -### MemoryDataAndStack {#memorydataandstack} - -Объём виртуальной памяти, отображённой для использования стека и выделенной памяти, в байтах. Не указано, включает ли он стеки отдельных потоков и большую часть выделенной памяти, которая выделяется с помощью системного вызова `mmap`. Эта метрика существует только для полноты. Рекомендуется использовать метрику `MemoryResident` для мониторинга. - -### MemoryResidentMax {#memoryresidentmax} - -Максимальный объём физической памяти, используемой процессом сервера, в байтах. - -### MemoryResident {#memoryresident} - -Объём физической памяти, используемой процессом сервера, в байтах. - -### MemoryShared {#memoryshared} - -Объём памяти, используемой процессом сервера и одновременно разделяемой с другими процессами, в байтах. ClickHouse не использует разделяемую память, но часть памяти может быть помечена ОС как разделяемая по её собственным причинам. За этой метрикой нет большого смысла наблюдать, она существует только для полноты. - -### MemoryVirtual {#memoryvirtual} - -Размер виртуального адресного пространства, выделенного процессом сервера, в байтах. Размер виртуального адресного пространства обычно значительно больше фактического потребления физической памяти и не должен использоваться как оценка потребления памяти. Большие значения этой метрики являются совершенно нормальными и имеют лишь технический смысл. - -### MySQLThreads {#mysqlthreads} - -Количество потоков в сервере протокола совместимости с MySQL. - -### NetworkReceiveBytes_*name* {#networkreceivebytes_name} - -Количество байт, полученных через сетевой интерфейс. Это системная метрика, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkReceiveDrop_*name* {#networkreceivedrop_name} - -Количество байт, потерянных при приёме пакетов через сетевой интерфейс. Это системная метрика, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkReceiveErrors_*name* {#networkreceiveerrors_name} - -Количество случаев возникновения ошибок при приёме через сетевой интерфейс. Это системная метрика, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkReceivePackets_*name* {#networkreceivepackets_name} - - Количество сетевых пакетов, полученных через сетевой интерфейс. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkSendBytes_*name* {#networksendbytes_name} - - Количество байт, отправленных через сетевой интерфейс. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkSendDrop_*name* {#networksenddrop_name} - - Количество случаев, когда пакет был отброшен при отправке через сетевой интерфейс. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkSendErrors_*name* {#networksenderrors_name} - - Количество случаев возникновения ошибки (например, повторная передача TCP) при отправке через сетевой интерфейс. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NetworkSendPackets_*name* {#networksendpackets_name} - - Количество сетевых пакетов, отправленных через сетевой интерфейс. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### NumberOfDatabases {#numberofdatabases} - -Общее количество баз данных на сервере. - -### NumberOfDetachedByUserParts {#numberofdetachedbyuserparts} - -Общее количество частей, отделённых от таблиц MergeTree пользователями с помощью запроса `ALTER TABLE DETACH` (в отличие от неожиданных, повреждённых или игнорируемых частей). Сервер не отслеживает отделённые части, и их можно удалить. - -### NumberOfDetachedParts {#numberofdetachedparts} - -Общее количество частей, отделённых от таблиц MergeTree. Часть может быть отделена пользователем с помощью запроса `ALTER TABLE DETACH` или самим сервером, если часть повреждена, неожиданна или не нужна. Сервер не отслеживает отделённые части, и их можно удалить. - -### NumberOfTables {#numberoftables} - -Общее количество таблиц, суммированное по всем базам данных на сервере, за исключением баз данных, которые не могут содержать таблицы MergeTree. Исключены движки баз данных, которые формируют набор таблиц «на лету», такие как `Lazy`, `MySQL`, `PostgreSQL`, `SQlite`. - -### OSContextSwitches {#oscontextswitches} - -Количество переключений контекста, которые произошли в системе на хостовой машине. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### OSGuestNiceTime {#osguestnicetime} - -Доля времени, затраченного на работу виртуального CPU для гостевых операционных систем под управлением ядра Linux, когда для гостя был установлен более высокий приоритет (см. `man procfs`). Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Эта метрика не имеет значения для ClickHouse, но присутствует для полноты. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU рассчитывается как сумма по ним [0..количество ядер]. - -### OSGuestNiceTimeCPU_*N* {#osguestnicetimecpu_n} - -Доля времени, затраченного на работу виртуального CPU для гостевых операционных систем под управлением ядра Linux, когда для гостя был установлен более высокий приоритет (см. `man procfs`). Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Эта метрика не имеет значения для ClickHouse, но присутствует для полноты. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU рассчитывается как сумма по ним [0..количество ядер]. - -### OSGuestNiceTimeNormalized {#osguestnicetimenormalized} - -Значение аналогично `OSGuestNiceTime`, но поделено на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от числа ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если число ядер неоднородно, и всё равно получать среднюю метрику использования ресурсов. - -### OSGuestTime {#osguesttime} - -Доля времени, затраченного на работу виртуального CPU для гостевых операционных систем под управлением ядра Linux (см. `man procfs`). Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Эта метрика не имеет значения для ClickHouse, но присутствует для полноты. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU рассчитывается как сумма по ним [0..количество ядер]. - -### OSGuestTimeCPU_*N* {#osguesttimecpu_n} - -Отношение времени, затраченного на выполнение виртуального CPU для гостевых операционных систем под управлением ядра Linux (см. `man procfs`). Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Эта метрика не имеет значения для ClickHouse, но приведена для полноты. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSGuestTimeNormalized {#osguesttimenormalized} - -Значение аналогично `OSGuestTime`, но делится на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от их числа. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере даже при неодинаковом количестве ядер и при этом получать среднюю метрику использования ресурсов. - -### OSIOWaitTime {#osiowaittime} - -Отношение времени, в течение которого ядро CPU не выполняло код, но при этом ядро ОС не запускало на этом CPU никакой другой процесс, так как процессы ожидали I/O. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSIOWaitTimeCPU_*N* {#osiowaittimecpu_n} - -Отношение времени, в течение которого ядро CPU не выполняло код, но при этом ядро ОС не запускало на этом CPU никакой другой процесс, так как процессы ожидали I/O. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSIOWaitTimeNormalized {#osiowaittimenormalized} - -Значение аналогично `OSIOWaitTime`, но делится на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от их числа. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере даже при неодинаковом количестве ядер и при этом получать среднюю метрику использования ресурсов. - -### OSIdleTime {#osidletime} - -Отношение времени, в течение которого ядро CPU простаивало (даже не было готово выполнить процесс, ожидающий I/O) с точки зрения ядра ОС. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Она не включает время, когда CPU был недоиспользован по причинам, внутренним для самого CPU (загрузки памяти, простои конвейера, ошибочные предсказания переходов, выполнение другого SMT-ядра). Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSIdleTimeCPU_*N* {#osidletimecpu_n} - -Отношение времени, в течение которого ядро CPU простаивало (даже не было готово выполнить процесс, ожидающий I/O) с точки зрения ядра ОС. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Она не включает время, когда CPU был недоиспользован по причинам, внутренним для самого CPU (загрузки памяти, простои конвейера, ошибочные предсказания переходов, выполнение другого SMT-ядра). Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSIdleTimeNormalized {#osidletimenormalized} - -Значение аналогично `OSIdleTime`, но делится на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от их числа. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере даже при неодинаковом количестве ядер и при этом получать среднюю метрику использования ресурсов. - -### OSInterrupts {#osinterrupts} - -Количество прерываний на хостовой машине. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. - -### OSIrqTime {#osirqtime} - -Отношение времени, затраченного на обработку аппаратных прерываний на CPU. Это системная метрика: она включает все процессы на хостовой машине, а не только clickhouse-server. Высокое значение этой метрики может указывать на некорректную конфигурацию оборудования или очень высокую сетевую нагрузку. Значение для одного ядра CPU находится в интервале [0..1]. Значение для всех ядер CPU вычисляется как их сумма [0..num cores]. - -### OSIrqTimeCPU_*N* {#osirqtimecpu_n} - -Доля времени, затраченного на обработку аппаратных прерываний на CPU. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. Высокое значение этой метрики может указывать на некорректную конфигурацию оборудования или очень высокую сетевую нагрузку. Значение для одного ядра CPU будет в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по ним [0..num cores]. - -### OSIrqTimeNormalized {#osirqtimenormalized} - -Значение аналогично `OSIrqTime`, но поделено на количество ядер CPU, чтобы укладываться в интервал [0..1] независимо от количества ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если количество ядер различается, и при этом получать усреднённую метрику использования ресурсов. - -### OSMemoryAvailable {#osmemoryavailable} - -Объём памяти, доступной для использования программами, в байтах. Эта метрика очень похожа на `OSMemoryFreePlusCached`. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSMemoryBuffers {#osmemorybuffers} - -Объём памяти, используемой буферами ядра ОС, в байтах. Обычно это значение должно быть небольшим, а большие значения могут указывать на некорректную конфигурацию ОС. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSMemoryCached {#osmemorycached} - -Объём памяти, используемой страничным кэшем ОС, в байтах. Как правило, почти вся доступная память используется страничным кэшем ОС — высокие значения этой метрики являются нормальными и ожидаемыми. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSMemoryFreePlusCached {#osmemoryfreepluscached} - -Объём свободной памяти плюс память страничного кэша ОС на хост-системе, в байтах. Эта память доступна для использования программами. Значение должно быть очень близко к `OSMemoryAvailable`. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSMemoryFreeWithoutCached {#osmemoryfreewithoutcached} - -Объём свободной памяти на хост-системе, в байтах. Это значение не включает память, используемую страничным кэшем ОС. Память страничного кэша также доступна для использования программами, поэтому значение этой метрики может быть неоднозначным. Вместо неё см. метрику `OSMemoryAvailable`. Для удобства также предоставляется метрика `OSMemoryFreePlusCached`, которая должна быть примерно аналогична `OSMemoryAvailable`. См. также https://www.linuxatemyram.com/. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSMemoryTotal {#osmemorytotal} - -Общий объём памяти на хост-системе, в байтах. - -### OSNiceTime {#osnicetime} - -Доля времени, когда ядро CPU выполняло пользовательский код с повышенным приоритетом. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. Значение для одного ядра CPU будет в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по ним [0..num cores]. - -### OSNiceTimeCPU_*N* {#osnicetimecpu_n} - -Доля времени, когда указанное ядро CPU выполняло пользовательский код с повышенным приоритетом. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. Значение для одного ядра CPU будет в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по ним [0..num cores]. - -### OSNiceTimeNormalized {#osnicetimenormalized} - -Значение аналогично `OSNiceTime`, но поделено на количество ядер CPU, чтобы укладываться в интервал [0..1] независимо от количества ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если количество ядер различается, и при этом получать усреднённую метрику использования ресурсов. - -### OSOpenFiles {#osopenfiles} - -Общее количество открытых файлов на хост-системе. Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSProcessesBlocked {#osprocessesblocked} - -Количество потоков, заблокированных в ожидании завершения операций ввода-вывода (`man procfs`). Это системная метрика: она включает все процессы на хост-системе, а не только clickhouse-server. - -### OSProcessesCreated {#osprocessescreated} - -Количество созданных процессов. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### OSProcessesRunning {#osprocessesrunning} - -Количество потоков, которые могут выполняться (запущены или готовы к запуску), по данным операционной системы. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. - -### OSSoftIrqTime {#ossoftirqtime} - -Доля времени, затраченного на обработку программных запросов прерываний на CPU. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Высокое значение этой метрики может указывать на неэффективно работающее программное обеспечение в системе. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSSoftIrqTimeCPU_*N* {#ossoftirqtimecpu_n} - -Доля времени, затраченного на обработку программных запросов прерываний на CPU. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Высокое значение этой метрики может указывать на неэффективно работающее программное обеспечение в системе. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSSoftIrqTimeNormalized {#ossoftirqtimenormalized} - -Значение аналогично `OSSoftIrqTime`, но разделено на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от количества ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если количество ядер отличается, и при этом получать усреднённую метрику использования ресурсов. - -### OSStealTime {#osstealtime} - -Доля времени, которое CPU проводит в других операционных системах при работе в виртуализованной среде. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Не все виртуализованные среды предоставляют эту метрику, и большинство — нет. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSStealTimeCPU_*N* {#osstealtimecpu_n} - -Доля времени, которое CPU проводит в других операционных системах при работе в виртуализованной среде. Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Не все виртуализованные среды предоставляют эту метрику, и большинство — нет. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSStealTimeNormalized {#osstealtimenormalized} - -Значение аналогично `OSStealTime`, но разделено на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от количества ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если количество ядер отличается, и при этом получать усреднённую метрику использования ресурсов. - -### OSSystemTime {#ossystemtime} - -Доля времени, в течение которого ядро CPU выполняло код ядра ОС (системный код). Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSSystemTimeCPU_*N* {#ossystemtimecpu_n} - -Доля времени, в течение которого ядро CPU выполняло код ядра ОС (системный код). Это метрика на уровне всей системы, она включает все процессы на хостовой машине, а не только clickhouse-server. Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..количество ядер]. - -### OSSystemTimeNormalized {#ossystemtimenormalized} - -Значение аналогично `OSSystemTime`, но разделено на количество ядер CPU, чтобы измеряться в интервале [0..1] независимо от количества ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если количество ядер отличается, и при этом получать усреднённую метрику использования ресурсов. - -### OSThreadsRunnable {#osthreadsrunnable} - -Общее количество потоков в состоянии «runnable», как его видит планировщик ядра ОС. - -### OSThreadsTotal {#osthreadstotal} - -Общее количество потоков, с точки зрения планировщика ядра ОС. - -### OSUptime {#osuptime} - -Время непрерывной работы хост-сервера (машины, на которой запущен ClickHouse), в секундах. - -### OSUserTime {#osusertime} - -Доля времени, когда ядро CPU выполняло пользовательский код (userspace). Это метрика для всей системы, она включает все процессы на хост-машине, а не только clickhouse-server. Сюда также входит время, когда CPU был недоиспользован по причинам, внутренним для самого CPU (загрузки из памяти, остановки конвейера, ошибочные предсказания переходов, выполнение другого SMT-ядра). Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..число ядер]. - -### OSUserTimeCPU_*N* {#osusertimecpu_n} - -Доля времени, когда ядро CPU выполняло пользовательский код (userspace). Это метрика для всей системы, она включает все процессы на хост-машине, а не только clickhouse-server. Сюда также входит время, когда CPU был недоиспользован по причинам, внутренним для самого CPU (загрузки из памяти, остановки конвейера, ошибочные предсказания переходов, выполнение другого SMT-ядра). Значение для одного ядра CPU лежит в интервале [0..1]. Значение для всех ядер CPU вычисляется как сумма по всем ядрам [0..число ядер]. - -### OSUserTimeNormalized {#osusertimenormalized} - -Значение аналогично `OSUserTime`, но делится на количество ядер CPU, чтобы оставаться в интервале [0..1] независимо от числа ядер. Это позволяет усреднять значения этой метрики по нескольким серверам в кластере, даже если число ядер неоднородно, и при этом получать корректную усреднённую метрику использования ресурсов. - -### PostgreSQLThreads {#postgresqlthreads} - -Количество потоков в сервере протокола совместимости с PostgreSQL. - -### ReplicasMaxAbsoluteDelay {#replicasmaxabsolutedelay} - -Максимальная разница в секундах между самой свежей реплицированной частью и самой свежей частью данных, ещё не реплицированной, по всем Replicated-таблицам. Очень большое значение указывает на реплику без данных. - -### ReplicasMaxInsertsInQueue {#replicasmaxinsertsinqueue} - -Максимальное количество операций INSERT в очереди (ещё не реплицированных) по всем Replicated-таблицам. - -### ReplicasMaxMergesInQueue {#replicasmaxmergesinqueue} - -Максимальное количество операций слияния в очереди (ещё не применённых) по всем Replicated-таблицам. - -### ReplicasMaxQueueSize {#replicasmaxqueuesize} - -Максимальный размер очереди (по количеству операций типа get, merge) по всем Replicated-таблицам. - -### ReplicasMaxRelativeDelay {#replicasmaxrelativedelay} - -Максимальная разница между задержкой реплики и задержкой самой актуальной реплики той же таблицы по всем Replicated-таблицам. - -### ReplicasSumInsertsInQueue {#replicassuminsertsinqueue} - -Суммарное количество операций INSERT в очереди (ещё не реплицированных) по всем Replicated-таблицам. - -### ReplicasSumMergesInQueue {#replicassummergesinqueue} - -Суммарное количество операций слияния в очереди (ещё не применённых) по всем Replicated-таблицам. - -### ReplicasSumQueueSize {#replicassumqueuesize} - -Суммарный размер очереди (по количеству операций типа get, merge) по всем Replicated-таблицам. - -### TCPThreads {#tcpthreads} - -Количество потоков в сервере протокола TCP (без TLS). - -### Temperature_*N* {#temperature_n} - -Температура соответствующего устройства в ℃. Датчик может возвращать нереалистичное значение. Источник: `/sys/class/thermal` - -### Temperature_*name* {#temperature_name} - -Температура, сообщаемая соответствующим аппаратным монитором и соответствующим датчиком, в ℃. Датчик может возвращать нереалистичное значение. Источник: `/sys/class/hwmon` - -### TotalBytesOfMergeTreeTables {#totalbytesofmergetreetables} - -Общее количество байт (сжатых, включая данные и индексы), хранящихся во всех таблицах семейства MergeTree. - -### TotalPartsOfMergeTreeTables {#totalpartsofmergetreetables} - -Общее количество частей данных во всех таблицах семейства MergeTree. Значения больше 10 000 будут отрицательно влиять на время запуска сервера и могут указывать на неудачный выбор ключа партиционирования. - -### TotalPrimaryKeyBytesInMemory {#totalprimarykeybytesinmemory} - -Объём памяти (в байтах), используемой значениями первичного ключа (учитываются только активные части). - -### TotalPrimaryKeyBytesInMemoryAllocated {#totalprimarykeybytesinmemoryallocated} - -Объём памяти (в байтах), зарезервированной под значения первичного ключа (учитываются только активные части). - -### TotalRowsOfMergeTreeTables {#totalrowsofmergetreetables} - -Общее количество строк (записей), хранимых во всех таблицах семейства MergeTree. - -### Uptime {#uptime} - -Время работы сервера в секундах. Включает время, затраченное на инициализацию сервера до начала приёма подключений. - -### jemalloc.active {#jemallocactive} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.allocated {#jemallocallocated} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.dirty_purged {#jemallocarenasalldirty_purged} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.muzzy_purged {#jemallocarenasallmuzzy_purged} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pactive {#jemallocarenasallpactive} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pdirty {#jemallocarenasallpdirty} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pmuzzy {#jemallocarenasallpmuzzy} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.num_runs {#jemallocbackground_threadnum_runs} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.num_threads {#jemallocbackground_threadnum_threads} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.run_intervals {#jemallocbackground_threadrun_intervals} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.epoch {#jemallocepoch} - -Внутренний инкрементальный номер обновления статистики jemalloc (распределителя памяти Джейсона Эванса), используемый во всех остальных метриках `jemalloc`. - -### jemalloc.mapped {#jemallocmapped} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.metadata {#jemallocmetadata} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.metadata_thp {#jemallocmetadata_thp} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.resident {#jemallocresident} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.retained {#jemallocretained} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -### jemalloc.prof.active {#jemallocprofactive} - -Внутренняя метрика низкоуровневого распределителя памяти (jemalloc). См. https://jemalloc.net/jemalloc.3.html - -**См. также** - -- [Monitoring](../../operations/monitoring.md) — Базовые концепции мониторинга ClickHouse. -- [system.metrics](/operations/system-tables/metrics) — Содержит мгновенно вычисляемые метрики. -- [system.events](/operations/system-tables/events) — Содержит количество произошедших событий. -- [system.metric_log](/operations/system-tables/metric_log) — Содержит историю значений метрик из таблиц `system.metrics` и `system.events`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md deleted file mode 100644 index 7d888744c7e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о настройках таблиц AzureQueue. - Доступна начиная с версии сервера `24.10`.' -keywords: ['system table', 'azure_queue_settings'] -slug: /operations/system-tables/azure_queue_settings -title: 'system.azure_queue_settings' -doc_type: 'reference' ---- - -Содержит информацию о настройках таблиц [AzureQueue](../../engines/table-engines/integrations/azure-queue.md). -Доступна начиная с версии сервера `24.10`. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md deleted file mode 100644 index c01b3ff4d9b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md +++ /dev/null @@ -1,170 +0,0 @@ ---- -description: 'Системная таблица, содержащая записи журнала с информацией об операциях `BACKUP` - и `RESTORE`.' -keywords: ['системная таблица', 'backup_log'] -slug: /operations/system-tables/backup_log -title: 'system.backup_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.backup_log {#systembackup_log} - - - -Содержит записи журнала с информацией о выполнении операций `BACKUP` и `RESTORE`. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата записи. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время записи. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время записи с точностью до микросекунд. -* `id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор операции резервного копирования или восстановления. -* `name` ([String](../../sql-reference/data-types/string.md)) — Имя хранилища резервной копии (содержимое секции `FROM` или `TO`). -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — Статус операции. Возможные значения: - * `'CREATING_BACKUP'` - * `'BACKUP_CREATED'` - * `'BACKUP_FAILED'` - * `'RESTORING'` - * `'RESTORED'` - * `'RESTORE_FAILED'` -* `error` ([String](../../sql-reference/data-types/string.md)) — Сообщение об ошибке для неуспешной операции (пустая строка для успешных операций). -* `start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время начала операции. -* `end_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время окончания операции. -* `num_files` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество файлов, сохранённых в резервной копии. -* `total_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Общий размер файлов, сохранённых в резервной копии. -* `num_entries` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество записей в резервной копии, то есть количество файлов внутри папки, если резервная копия хранится как папка, или количество файлов внутри архива, если резервная копия хранится как архив. Это не то же самое, что `num_files`, если это инкрементная резервная копия или если она содержит пустые файлы или дубликаты. Всегда верно следующее: `num_entries <= num_files`. -* `uncompressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Несжатый размер резервной копии. -* `compressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Сжатый размер резервной копии. Если резервная копия не хранится как архив, его значение равно `uncompressed_size`. -* `files_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество файлов, прочитанных во время операции восстановления. -* `bytes_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Общий размер файлов, прочитанных во время операции восстановления. - -**Пример** - -```sql -BACKUP TABLE test_db.my_table TO Disk('backups_disk', '1.zip') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'e5b74ecb-f6f1-426a-80be-872f90043885' ORDER BY event_date, event_time_microseconds \G -``` - -```response -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:05:21.998566 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-19 11:05:21 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Row 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time: 2023-08-19 11:08:56 -event_time_microseconds: 2023-08-19 11:08:56.916192 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: BACKUP_CREATED -error: -start_time: 2023-08-19 11:05:21 -end_time: 2023-08-19 11:08:56 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 3525068304 -files_read: 0 -bytes_read: 0 -``` - -```sql -RESTORE TABLE test_db.my_table FROM Disk('backups_disk', '1.zip') -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ ВОССТАНОВЛЕН │ -└──────────────────────────────────────┴──────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'cdf1f731-52ef-42da-bc65-2e1bfcd4ce90' ORDER BY event_date, event_time_microseconds \G -``` - -```response -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:19.718077 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORING -error: -start_time: 2023-08-19 11:09:19 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Строка 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:29.334234 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORED -error: -start_time: 2023-08-19 11:09:19 -end_time: 2023-08-19 11:09:29 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 4290362365 -files_read: 57 -bytes_read: 4290364870 -``` - -По сути, это та же информация, которая содержится в системной таблице `system.backups`: - -```sql -SELECT * FROM system.backups ORDER BY start_time -``` - -```response -┌─id───────────────────────────────────┬─name──────────────────────────┬─status─────────┬─error─┬──────────start_time─┬────────────end_time─┬─num_files─┬─total_size─┬─num_entries─┬─uncompressed_size─┬─compressed_size─┬─files_read─┬─bytes_read─┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ Disk('backups_disk', '1.zip') │ BACKUP_CREATED │ │ 2023-08-19 11:05:21 │ 2023-08-19 11:08:56 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 3525068304 │ 0 │ 0 │ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ Disk('backups_disk', '1.zip') │ RESTORED │ │ 2023-08-19 11:09:19 │ 2023-08-19 11:09:29 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 4290362365 │ 57 │ 4290364870 │ -└──────────────────────────────────────┴───────────────────────────────┴────────────────┴───────┴─────────────────────┴─────────────────────┴───────────┴────────────┴─────────────┴───────────────────┴─────────────────┴────────────┴────────────┘ -``` - -**См. также** - -* [Резервное копирование и восстановление](../../operations/backup.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md deleted file mode 100644 index a2d7a1bfc08..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: 'Системная таблица, содержащая записи журнала с информацией об операциях `BACKUP` и `RESTORE`.' -keywords: ['системная таблица', 'резервные копии'] -slug: /operations/system-tables/backups -title: 'system.backups' -doc_type: 'reference' ---- - -# system.backups {#systembackups} - -Содержит список всех операций `BACKUP` или `RESTORE` с их текущими состояниями и другими свойствами. Обратите внимание, что эта таблица не является персистентной и отображает только операции, выполненные после последнего перезапуска сервера. - -Ниже приведена таблица в формате Markdown со столбцами name и comment: - -| Column | Description | -|---------------------|----------------------------------------------------------------------------------------------------------------------| -| `id` | ID операции, может быть либо передан через SETTINGS id=..., либо случайно сгенерирован как UUID. | -| `name` | Имя операции, строка вида `Disk('backups', 'my_backup')` | -| `base_backup_name` | Имя базовой операции резервного копирования, строка вида `Disk('backups', 'my_base_backup')` | -| `query_id` | ID запроса, который запустил создание резервной копии. | -| `status` | Статус операции BACKUP или RESTORE. | -| `error` | Сообщение об ошибке, если она была. | -| `start_time` | Время начала операции. | -| `end_time` | Время завершения операции. | -| `num_files` | Количество файлов, сохранённых в резервной копии. | -| `total_size` | Общий размер файлов, сохранённых в резервной копии. | -| `num_entries` | Количество записей в резервной копии, то есть количество файлов внутри каталога, если резервная копия хранится как каталог. | -| `uncompressed_size` | Несжатый размер резервной копии. | -| `compressed_size` | Сжатый размер резервной копии. | -| `files_read` | Количество файлов, прочитанных при выполнении RESTORE из этой резервной копии. | -| `bytes_read` | Общий размер файлов, прочитанных при выполнении RESTORE из этой резервной копии. | -| `ProfileEvents` | Все события профилирования, зафиксированные во время этой операции. | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md deleted file mode 100644 index 7c07c29fd31..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'Системная таблица, содержащая записи журнала с информацией о различных операциях с блоб-хранилищем, таких как загрузка и удаление данных.' -keywords: ['системная таблица', 'blob_storage_log'] -slug: /operations/system-tables/blob_storage_log -title: 'system.blob_storage_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит записи журнала с информацией о различных операциях с блоб‑хранилищем, таких как загрузка и удаление объектов. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время события. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время события с точностью до микросекунд. -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип события. Возможные значения: - * `'Upload'` - * `'Delete'` - * `'MultiPartUploadCreate'` - * `'MultiPartUploadWrite'` - * `'MultiPartUploadComplete'` - * `'MultiPartUploadAbort'` -* `query_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор запроса, связанного с событием, если он есть. -* `thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Идентификатор потока, выполняющего операцию. -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — Имя потока, выполняющего операцию. -* `disk_name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — Имя связанного диска. -* `bucket` ([String](../../sql-reference/data-types/string.md)) — Имя бакета. -* `remote_path` ([String](../../sql-reference/data-types/string.md)) — Путь к удалённому ресурсу. -* `local_path` ([String](../../sql-reference/data-types/string.md)) — Путь к файлу метаданных в локальной системе, который ссылается на удалённый ресурс. -* `data_size` ([UInt32](/sql-reference/data-types/int-uint#integer-ranges)) — Размер данных, участвующих в событии загрузки. -* `error` ([String](../../sql-reference/data-types/string.md)) — Сообщение об ошибке, связанной с событием, если оно есть. - -**Пример** - -Предположим, что в результате операции с блоб‑хранилищем загружается файл и соответствующее событие записывается в журнал: - -```sql -SELECT * FROM system.blob_storage_log WHERE query_id = '7afe0450-504d-4e4b-9a80-cd9826047972' ORDER BY event_date, event_time_microseconds \G -``` - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-10-31 -event_time: 2023-10-31 16:03:40 -event_time_microseconds: 2023-10-31 16:03:40.481437 -event_type: Загрузка -query_id: 7afe0450-504d-4e4b-9a80-cd9826047972 -thread_id: 2381740 -disk_name: disk_s3 -bucket: bucket1 -remote_path: rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe -local_path: store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt -data_size: 259 -error: -``` - -В этом примере операция загрузки была связана с запросом `INSERT` с идентификатором `7afe0450-504d-4e4b-9a80-cd9826047972`. Локальный файл метаданных `store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt` ссылается на удалённый путь `rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe` в бакете `bucket1` на диске `disk_s3` и имеет размер 259 байт. - -**См. также** - -* [Внешние диски для хранения данных](../../operations/storing-data.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md deleted file mode 100644 index d401a602afd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о параметрах сборки сервера ClickHouse.' -slug: /operations/system-tables/build_options -title: 'system.build_options' -keywords: ['system table', 'build_options'] -doc_type: 'reference' ---- - -Содержит сведения о параметрах сборки сервера ClickHouse. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.build_options LIMIT 5 -``` - -```text -┌─name─────────────┬─value─┐ -│ USE_BROTLI │ 1 │ -│ USE_BZIP2 │ 1 │ -│ USE_CAPNP │ 1 │ -│ USE_CASSANDRA │ 1 │ -│ USE_DATASKETCHES │ 1 │ -└──────────────────┴───────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md deleted file mode 100644 index c51184996bc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о кластерах, определённых в - файле конфигурации, и входящих в них серверах.' -keywords: ['системная таблица', 'кластеры'] -slug: /operations/system-tables/clusters -title: 'system.clusters' -doc_type: 'reference' ---- - -Содержит информацию о кластерах, определённых в файле конфигурации, и входящих в них серверах. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.clusters LIMIT 2 FORMAT Vertical; -``` - -Результат: - -```text -Row 1: -────── -cluster: test_cluster_two_shards -shard_num: 1 -shard_name: shard_01 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.1 -host_address: 127.0.0.1 -port: 9000 -is_local: 1 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL - -Row 2: -────── -cluster: test_cluster_two_shards -shard_num: 2 -shard_name: shard_02 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.2 -host_address: 127.0.0.2 -port: 9000 -is_local: 0 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL -``` - -**См. также** - -* [Табличный движок Distributed](../../engines/table-engines/special/distributed.md) -* [Настройка distributed_replica_error_cap](../../operations/settings/settings.md#distributed_replica_error_cap) -* [Настройка distributed_replica_error_half_life](../../operations/settings/settings.md#distributed_replica_error_half_life) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md deleted file mode 100644 index 1fc57ac4f31..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о кодеках.' -keywords: ['system table', 'codecs', 'compression'] -slug: /operations/system-tables/codecs -title: 'system.codecs' -doc_type: 'reference' ---- - -Содержит информацию о кодеках сжатия и шифрования. - -Эту таблицу можно использовать для получения информации о доступных кодеках сжатия и шифрования. - -Таблица `system.codecs` содержит следующие столбцы (тип столбца указан в скобках): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.codecs WHERE name='LZ4' -``` - -Результат: - -```text -Row 1: -────── -name: LZ4 -method_byte: 130 -is_compression: 1 -is_generic_compression: 1 -is_encryption: 0 -is_timeseries_codec: 0 -is_experimental: 0 -description: Очень быстрый; хорошее сжатие; сбалансированное соотношение скорости и эффективности. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md deleted file mode 100644 index 54d500e2848..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о столбцах во всех таблицах' -keywords: ['системная таблица', 'столбцы'] -slug: /operations/system-tables/columns -title: 'system.columns' -doc_type: 'reference' ---- - -Содержит информацию о столбцах во всех таблицах. - -Эту таблицу можно использовать для получения информации, аналогичной запросу [DESCRIBE TABLE](../../sql-reference/statements/describe-table.md), но сразу для нескольких таблиц. - -Столбцы из [временных таблиц](../../sql-reference/statements/create/table.md#temporary-tables) видны в таблице `system.columns` только в тех сессиях, в которых они были созданы. Они отображаются с пустым полем `database`. - -Таблица `system.columns` содержит следующие столбцы (тип столбца указан в скобках): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.columns LIMIT 2 FORMAT Vertical; -``` - -```text -Строка 1: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_catalog -type: String -position: 1 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ - -Строка 2: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_schema -type: String -position: 2 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md deleted file mode 100644 index 92871db49d8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об участниках.' -keywords: ['system table', 'contributors'] -slug: /operations/system-tables/contributors -title: 'system.contributors' -doc_type: 'reference' ---- - -Содержит информацию об участниках. Порядок является случайным при выполнении запроса. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.contributors LIMIT 10 -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -│ Max Vetrov │ -│ LiuYangkuan │ -│ svladykin │ -│ zamulla │ -│ Šimon Podlipský │ -│ BayoNet │ -│ Ilya Khomutov │ -│ Amy Krishnevsky │ -│ Loud_Scream │ -└──────────────────┘ -``` - -Чтобы найти свои данные в таблице, выполните запрос: - -```sql -SELECT * FROM system.contributors WHERE name = 'Olga Khvostikova' -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -└──────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md deleted file mode 100644 index ee17d7ca7d7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о трассировках стека при фатальных ошибках.' -keywords: ['system table', 'crash_log'] -slug: /operations/system-tables/crash_log -title: 'system.crash_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о трассировках стека при фатальных ошибках. Таблица по умолчанию в базе данных отсутствует, она создается только при возникновении фатальных ошибок. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время события. -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Метка времени события с точностью до наносекунд. -* `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — Номер сигнала. -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ID потока. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — ID запроса. -* `trace` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Трассировка стека на момент сбоя. Каждый элемент — это адрес виртуальной памяти внутри серверного процесса ClickHouse. -* `trace_full` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — Трассировка стека на момент сбоя. Каждый элемент содержит вызванный метод внутри серверного процесса ClickHouse. -* `version` ([String](../../sql-reference/data-types/string.md)) — Версия сервера ClickHouse. -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Ревизия сервера ClickHouse. -* `build_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор сборки (BuildID), генерируемый компилятором. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.crash_log ORDER BY event_time DESC LIMIT 1; -``` - -Результат (неполный): - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-10-14 -event_time: 2020-10-14 15:47:40 -timestamp_ns: 1602679660271312710 -signal: 11 -thread_id: 23624 -query_id: 428aab7c-8f5c-44e9-9607-d16b44467e69 -trace: [188531193,...] -trace_full: ['3. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::__1::vector >&, std::__1::vector > const&, unsigned long, unsigned long) const @ 0xb3cc1f9 in /home/username/work/ClickHouse/build/programs/clickhouse',...] -version: ClickHouse 20.11.1.1 -revision: 54442 -build_id: -``` - -**См. также** - -* системная таблица [trace_log](../../operations/system-tables/trace_log.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md deleted file mode 100644 index bff9d6a3623..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: 'Системная таблица, содержащая активные роли текущего пользователя.' -keywords: ['system table', 'current_roles'] -slug: /operations/system-tables/current_roles -title: 'system.current_roles' -doc_type: 'reference' ---- - -Содержит активные роли текущего пользователя. Команда `SET ROLE` изменяет содержимое этой таблицы. - -Столбцы: - -- `role_name` ([String](../../sql-reference/data-types/string.md))) — Имя роли. -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, имеет ли `current_role` привилегию `ADMIN OPTION`. -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, является ли `current_role` ролью по умолчанию. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md deleted file mode 100644 index e3d2de36c24..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Содержит запросы, используемые на странице `/dashboard`, доступной через HTTP-интерфейс. Полезно для мониторинга и устранения неполадок.' -keywords: ['system table', 'dashboards', 'monitoring', 'troubleshooting'] -slug: /operations/system-tables/dashboards -title: 'system.dashboards' -doc_type: 'reference' ---- - -Содержит запросы, используемые страницей `/dashboard`, доступной через [HTTP-интерфейс](/interfaces/http.md). -Эта таблица может быть полезна для мониторинга и устранения неполадок. Таблица содержит по одной строке для каждого графика на дашборде. - -:::note -Страница `/dashboard` может отображать запросы не только из `system.dashboards`, но и из любой таблицы с той же схемой. -Это может быть полезно для создания пользовательских дашбордов. -::: - -Пример: - -```sql -SELECT * -FROM system.dashboards -WHERE title ILIKE '%CPU%' -``` - -```text -Row 1: -────── -dashboard: overview -title: Использование ЦП (ядра) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUVirtualTimeMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 2: -────── -dashboard: overview -title: Ожидание ЦП -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUWaitMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 3: -────── -dashboard: overview -title: Использование ЦП ОС (пользовательское пространство) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSUserTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 4: -────── -dashboard: overview -title: Использование ЦП ОС (ядро) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSSystemTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} -``` - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md deleted file mode 100644 index a65359f1f6d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об имеющихся индексах пропуска данных - во всех таблицах.' -keywords: ['system table', 'data_skipping_indices'] -slug: /operations/system-tables/data_skipping_indices -title: 'system.data_skipping_indices' -doc_type: 'reference' ---- - -Содержит информацию об имеющихся индексах пропуска данных во всех таблицах. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*КОНЕЦ_АВТОГЕНЕРАЦИИ*/ } - -**Пример** - -```sql -SELECT * FROM system.data_skipping_indices LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: user_actions -name: clicks_idx -type: minmax -type_full: minmax -expr: clicks -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 - -Row 2: -────── -database: default -table: users -name: contacts_null_idx -type: minmax -type_full: minmax -expr: assumeNotNull(contacts_null) -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md deleted file mode 100644 index f867b278e5e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о поддерживаемых типах данных' -keywords: ['system table', 'data_type_families'] -slug: /operations/system-tables/data_type_families -title: 'system.data_type_families' -doc_type: 'reference' ---- - -Содержит информацию о поддерживаемых [типах данных](../../sql-reference/data-types/index.md). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.data_type_families WHERE alias_to = 'String' -``` - -```text -┌─name───────┬─case_insensitive─┬─alias_to─┐ -│ LONGBLOB │ 1 │ String │ -│ LONGTEXT │ 1 │ String │ -│ TINYTEXT │ 1 │ String │ -│ TEXT │ 1 │ String │ -│ VARCHAR │ 1 │ String │ -│ MEDIUMBLOB │ 1 │ String │ -│ BLOB │ 1 │ String │ -│ TINYBLOB │ 1 │ String │ -│ CHAR │ 1 │ String │ -│ MEDIUMTEXT │ 1 │ String │ -└────────────┴──────────────────┴──────────┘ -``` - -**См. также** - -* [Синтаксис](../../sql-reference/syntax.md) — сведения о поддерживаемом синтаксисе. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md deleted file mode 100644 index ceff7c09866..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: 'Системная таблица со списком движков баз данных, поддерживаемых - сервером.' -keywords: ['system table', 'database_engines'] -slug: /operations/system-tables/database_engines -title: 'system.database_engines' -doc_type: 'reference' ---- - -Содержит список движков баз данных, поддерживаемых сервером. - -Таблица содержит следующие столбцы (тип столбца указан в скобках): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Пример: - -```sql -SELECT * -FROM system.database_engines -WHERE name IN ('Atomic', 'Lazy', 'Ordinary') -``` - -```text -┌─name─────┐ -│ Ordinary │ -│ Atomic │ -│ Lazy │ -└──────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md deleted file mode 100644 index ffc9540e1cf..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о реплицируемой базе данных и состоянии её реплик.' -keywords: ['system table', 'database_replicas'] -slug: /operations/system-tables/database_replicas -title: 'system.database_replicas' -doc_type: 'reference' ---- - -Содержит сведения о каждой реплике базы данных с движком Replicated. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.database_replicas FORMAT Vertical; -``` - -```text -Row 1: -────── -database: db_2 -is_readonly: 0 -max_log_ptr: 2 -replica_name: replica1 -replica_path: /test/db_2/replicas/shard1|replica1 -zookeeper_path: /test/db_2 -shard_name: shard1 -log_ptr: 2 -total_replicas: 1 -zookeeper_exception: -is_session_expired: 0 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md deleted file mode 100644 index 1b70c8a0076..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о базах данных, доступных текущему пользователю.' -keywords: ['системная таблица', 'базы данных'] -slug: /operations/system-tables/databases -title: 'system.databases' -doc_type: 'reference' ---- - -Содержит информацию о базах данных, доступных текущему пользователю. - -Столбцы: - -{/*AUTOGENERGED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Столбец `name` в этой системной таблице используется для реализации запроса `SHOW DATABASES`. - -**Пример** - -Создайте базу данных. - -```sql -CREATE DATABASE test; -``` - -Просмотрите все базы данных, доступные пользователю. - -```sql -SELECT * FROM system.databases; -``` - -```text -┌─name────────────────┬─engine─────┬─data_path────────────────────┬─metadata_path─────────────────────────────────────────────────────────┬─uuid─────────────────────────────────┬─engine_full────────────────────────────────────────────┬─comment─┐ -│ INFORMATION_SCHEMA │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ default │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/f97/f97a3ceb-2e8a-4912-a043-c536e826a4d4/ │ f97a3ceb-2e8a-4912-a043-c536e826a4d4 │ Atomic │ │ -│ information_schema │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ replicated_database │ Replicated │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/da8/da85bb71-102b-4f69-9aad-f8d6c403905e/ │ da85bb71-102b-4f69-9aad-f8d6c403905e │ Replicated('some/path/database', 'shard1', 'replica1') │ │ -│ system │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/b57/b5770419-ac7a-4b67-8229-524122024076/ │ b5770419-ac7a-4b67-8229-524122024076 │ Atomic │ │ -└─────────────────────┴────────────┴──────────────────────────────┴───────────────────────────────────────────────────────────────────────┴──────────────────────────────────────┴────────────────────────────────────────────────────────┴─────────┘ - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md deleted file mode 100644 index 038d96cbd0e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о сообщениях, - полученных через стриминговый движок и разобранных с ошибками.' -keywords: ['system table', 'dead_letter_queue'] -slug: /operations/system-tables/dead_letter_queue -title: 'system.dead_letter_queue' -doc_type: 'reference' ---- - -Содержит информацию о сообщениях, полученных через стриминговый движок и разобранных с ошибками. В настоящее время поддерживается для Kafka и RabbitMQ. - -Логирование включается указанием `dead_letter_queue` для параметра `handle_error_mode`, специфичного для движка. - -Период сброса данных задаётся параметром `flush_interval_milliseconds` в разделе настроек сервера [dead_letter_queue](../../operations/server-configuration-parameters/settings.md#dead_letter_queue). Для принудительного сброса используйте запрос [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs). - -ClickHouse не удаляет данные из таблицы автоматически. Дополнительные сведения см. в разделе [Введение](../../operations/system-tables/overview.md#system-tables-introduction). - -Столбцы: - -* `table_engine` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип стриминга. Возможные значения: `Kafka` и `RabbitMQ`. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата получения сообщения. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время получения сообщения. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время получения сообщения с точностью до микросекунд. -* `database` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — База данных ClickHouse, к которой относится стриминговая таблица. -* `table` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя таблицы ClickHouse. -* `error` ([String](../../sql-reference/data-types/string.md)) — Текст ошибки. -* `raw_message` ([String](../../sql-reference/data-types/string.md)) — Тело сообщения. -* `kafka_topic_name` ([String](../../sql-reference/data-types/string.md)) — Имя топика Kafka. -* `kafka_partition` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Раздел (partition) топика Kafka. -* `kafka_offset` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Смещение сообщения в Kafka. -* `kafka_key` ([String](../../sql-reference/data-types/string.md)) — Ключ сообщения в Kafka. -* `rabbitmq_exchange_name` ([String](../../sql-reference/data-types/string.md)) — Имя exchange в RabbitMQ. -* `rabbitmq_message_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор сообщения в RabbitMQ. -* `rabbitmq_message_timestamp` ([DateTime](../../sql-reference/data-types/datetime.md)) — Отметка времени сообщения в RabbitMQ. -* `rabbitmq_message_redelivered` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Флаг повторной доставки в RabbitMQ. -* `rabbitmq_message_delivery_tag` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Метка доставки сообщения в RabbitMQ. -* `rabbitmq_channel_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор канала RabbitMQ. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.dead_letter_queue LIMIT 1 \G; -``` - -Результат: - - -```text -Строка 1: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910773 -database: default -table: kafka -error: Не удаётся разобрать входные данные: ожидался символ '\t' перед 'qwertyuiop' (в строке 1) -: -Строка 1: -Столбец 0, имя: key, тип: UInt64, ОШИБКА: текст "qwertyuiop" не соответствует типу UInt64 -raw_message: qwertyuiop -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -Строка 2: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910944 -database: default -table: kafka -error: Не удаётся разобрать входные данные: ожидался символ '\t' перед 'asdfghjkl' (в строке 1) -: -Строка 1: -Столбец 0, имя: key, тип: UInt64, ОШИБКА: текст "asdfghjkl" не соответствует типу UInt64 -raw_message: asdfghjkl -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -Строка 3: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.911092 -database: default -table: kafka -error: Не удаётся разобрать входные данные: ожидался символ '\t' перед 'zxcvbnm' (в строке 1) -: -Строка 1: -Столбец 0, имя: key, тип: UInt64, ОШИБКА: текст "zxcvbnm" не соответствует типу UInt64 -raw_message: zxcvbnm -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - (test.py:78, dead_letter_queue_test) - -``` - -**См. также** - -* [Kafka](/engines/table-engines/integrations/kafka.md) — табличный движок Kafka -* [system.kafka_consumers](/operations/system-tables/kafka_consumers.md) — описание системной таблицы `kafka_consumers`, содержащей статистику, ошибки и другую информацию о потребителях Kafka. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md deleted file mode 100644 index 957245b34ad..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о файлах метаданных, считываемых из таблиц Delta Lake. Каждая запись - соответствует корневому JSON-файлу метаданных.' -keywords: ['системная таблица', 'delta_lake_metadata_log'] -slug: /operations/system-tables/delta_lake_metadata_log -title: 'system.delta_lake_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.delta_lake_metadata_log {#systemdelta_lake_metadata_log} - -Таблица `system.delta_lake_metadata_log` фиксирует события доступа к метаданным и их разбора для таблиц Delta Lake, которые читает ClickHouse. Она предоставляет подробную информацию о каждом файле метаданных, что полезно для отладки, аудита и понимания эволюции структуры таблиц Delta Lake. - - - -## Назначение {#purpose} - -Эта таблица регистрирует каждый файл метаданных, прочитанный из таблиц Delta Lake. Она помогает пользователям отследить, как ClickHouse интерпретирует метаданные таблиц Delta, и диагностировать проблемы, связанные с эволюцией схемы, выбором снимка (snapshot) или планированием выполнения запросов. - -:::note -Эта таблица предназначена в первую очередь для целей отладки. -::: - - - -## Столбцы {#columns} -| Имя | Тип | Описание | -|----------------|-----------|----------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | Дата файла журнала. | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | Временная метка события. | -| `query_id` | [String](../../sql-reference/data-types/string.md) | Идентификатор запроса, инициировавшего чтение метаданных. | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Путь к таблице Delta Lake. | -| `file_path` | [String](../../sql-reference/data-types/string.md) | Путь к корневому JSON-файлу метаданных. | -| `content` | [String](../../sql-reference/data-types/string.md) | Содержимое в формате JSON (исходные метаданные из файла .json). | - - - - - -## Управление уровнем детализации журналирования {#controlling-log-verbosity} - -Вы можете управлять тем, какие события, связанные с метаданными, записываются в журнал с помощью настройки [`delta_lake_log_metadata`](../../operations/settings/settings.md#delta_lake_log_metadata). - -Чтобы записывать в журнал все метаданные, используемые в текущем запросе: - -```sql -SELECT * FROM my_delta_table SETTINGS delta_lake_log_metadata = 1; - -SYSTEM FLUSH LOGS delta_lake_metadata_log; - -SELECT * -FROM system.delta_lake_metadata_log -WHERE query_id = '{previous_query_id}'; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md deleted file mode 100644 index 32c913ee9e2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об отсоединённых частях таблиц - MergeTree' -keywords: ['system table', 'detached_parts'] -slug: /operations/system-tables/detached_parts -title: 'system.detached_parts' -doc_type: 'reference' ---- - -Содержит информацию об отсоединённых частях таблиц [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). Столбец `reason` указывает, по какой причине часть была отсоединена. - -Для частей, отсоединённых пользователем, причина не указана. Такие части можно снова присоединить командой [ALTER TABLE ATTACH PARTITION\|PART](/sql-reference/statements/alter/partition#attach-partitionpart). - -Описание остальных столбцов см. в [system.parts](../../operations/system-tables/parts.md). - -Если имя части некорректно, значения некоторых столбцов могут быть `NULL`. Такие части можно удалить командой [ALTER TABLE DROP DETACHED PART](/sql-reference/statements/alter/partition#drop-detached-partitionpart). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md deleted file mode 100644 index 9c1834e75c9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию обо всех отсоединённых таблицах.' -keywords: ['system table', 'detached_tables'] -slug: /operations/system-tables/detached_tables -title: 'system.detached_tables' -doc_type: 'reference' ---- - -Содержит информацию обо всех отсоединённых таблицах. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.detached_tables FORMAT Vertical; -``` - -```text -Строка 1: -────── -database: base -table: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -is_permanently: 1 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md deleted file mode 100644 index 5b7df44ca7a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о словарях' -keywords: ['системная таблица', 'словари'] -slug: /operations/system-tables/dictionaries -title: 'system.dictionaries' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о [словарях](../../sql-reference/dictionaries/index.md). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Настройте словарь: - -```sql -CREATE DICTIONARY dictionary_with_comment -( - id UInt64, - value String -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table')) -LAYOUT(FLAT()) -LIFETIME(MIN 0 MAX 1000) -COMMENT 'Временный словарь'; -``` - -Убедитесь, что словарь загружен. - -```sql -SELECT * FROM system.dictionaries LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -name: dictionary_with_comment -uuid: 4654d460-0d03-433a-8654-d4600d03d33a -status: NOT_LOADED -origin: 4654d460-0d03-433a-8654-d4600d03d33a -type: -key.names: ['id'] -key.types: ['UInt64'] -attribute.names: ['value'] -attribute.types: ['String'] -bytes_allocated: 0 -query_count: 0 -hit_rate: 0 -found_rate: 0 -element_count: 0 -load_factor: 0 -source: -lifetime_min: 0 -lifetime_max: 0 -loading_start_time: 1970-01-01 00:00:00 -last_successful_update_time: 1970-01-01 00:00:00 -loading_duration: 0 -last_exception: -comment: Временный словарь -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md deleted file mode 100644 index 1baaeefea8c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Эта таблица содержит многомерные метрики, которые могут быть мгновенно вычислены и экспортированы в формате Prometheus. Она всегда содержит актуальные данные.' -keywords: ['system table', 'dimensional_metrics'] -slug: /operations/system-tables/dimensional_metrics -title: 'system.dimensional_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# dimensional_metrics {#dimensional_metrics} - - - -Эта таблица содержит размерные метрики, которые могут быть мгновенно вычислены и экспортированы в формате Prometheus. Она всегда содержит актуальные данные. - -Столбцы: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — Имя метрики. -* `value` ([Int64](../../sql-reference/data-types/int-uint.md)) — Значение метрики. -* `description` ([String](../../sql-reference/data-types/string.md)) — Описание метрики. -* `labels` ([Map(String, String)](../../sql-reference/data-types/map.md)) — Метки метрики. -* `name` ([String](../../sql-reference/data-types/string.md)) — Псевдоним для `metric`. - -**Пример** - -Вы можете использовать следующий запрос, чтобы экспортировать все размерные метрики в формате Prometheus. - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'gauge' AS type -FROM system.dimensional_metrics -FORMAT Prometheus -``` - - -## Описание метрик {#metric_descriptions} - -### merge_failures {#merge_failures} -Общее количество неудачных слияний с момента запуска. - -### startup_scripts_failure_reason {#startup_scripts_failure_reason} -Отражает причины ошибок стартовых скриптов по типу ошибки. При неудачном выполнении стартового скрипта устанавливается в 1, при этом в метке указывается имя ошибки. - -### merge_tree_parts {#merge_tree_parts} -Количество частей данных MergeTree с метками, указывающими состояние части, тип части и то, является ли она частью проекции. - -**См. также** -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — Содержит периодически вычисляемые метрики. -- [system.events](/operations/system-tables/events) — Содержит количество произошедших событий. -- [system.metric_log](/operations/system-tables/metric_log) — Содержит историю значений метрик таблиц `system.metrics` и `system.events`. -- [Monitoring](../../operations/monitoring.md) — Основные концепции мониторинга ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md deleted file mode 100644 index d9f09e9a581..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о дисках, заданных в конфигурации сервера' -keywords: ['системная таблица', 'диски'] -slug: /operations/system-tables/disks -title: 'system.disks' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о дисках, заданных в [конфигурации сервера](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.disks; -``` - -```response -┌─name────┬─path─────────────────┬───free_space─┬──total_space─┬─keep_free_space─┐ -│ default │ /var/lib/clickhouse/ │ 276392587264 │ 490652508160 │ 0 │ -└─────────┴──────────────────────┴──────────────┴──────────────┴─────────────────┘ - -1 строка в наборе. Затрачено: 0.001 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md deleted file mode 100644 index d59b287cc5d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о распределённых DDL-запросах (запросах с оператором ON CLUSTER), которые были выполнены на кластере.' -keywords: ['system table', 'distributed_ddl_queue'] -slug: /operations/system-tables/distributed_ddl_queue -title: 'system.distributed_ddl_queue' -doc_type: 'reference' ---- - -Содержит информацию о [распределённых DDL-запросах (с оператором ON CLUSTER)](../../sql-reference/distributed-ddl.md), которые были выполнены на кластере. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * -FROM system.distributed_ddl_queue -WHERE cluster = 'test_cluster' -LIMIT 2 -FORMAT Vertical - -ID запроса: f544e72a-6641-43f1-836b-24baa1c9632a - -Строка 1: -────── -entry: query-0000000000 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: Finished -exception_code: 0 -exception_text: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -Строка 2: -────── -entry: query-0000000001 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: Finished -exception_code: 630 -exception_text: Code: 630. DB::Exception: Cannot drop or rename test_db, because some tables depend on it: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -2 строки в наборе. Затрачено: 0.025 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md deleted file mode 100644 index afdecb45608..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о локальных файлах, находящихся - в очереди на отправку сегментам.' -keywords: ['system table', 'distribution_queue'] -slug: /operations/system-tables/distribution_queue -title: 'system.distribution_queue' -doc_type: 'reference' ---- - -Содержит информацию о локальных файлах, которые находятся в очереди на отправку сегментам. Эти локальные файлы содержат новые части, создаваемые при асинхронной вставке новых данных в распределённую таблицу Distributed. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.distribution_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Строка 1: -────── -база_данных: default -таблица: dist -путь_данных: ./store/268/268bc070-3aad-4b1a-9cf2-4987580161af/default@127%2E0%2E0%2E2:9000/ -заблокирована: 1 -количество_ошибок: 0 -файлов_данных: 1 -сжатых_байт_данных: 499 -последнее_исключение: -``` - -**См. также** - -* [Движок таблиц Distributed](../../engines/table-engines/special/distributed.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md deleted file mode 100644 index dbc571ded93..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о кэшированных DNS-записях.' -keywords: ['system table', 'dns_cache'] -slug: /operations/system-tables/dns_cache -title: 'system.dns_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о кэшированных DNS-записях. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.dns_cache; -``` - -Результат: - -| hostname | ip_address | ip_family | cached_at | -| :-------- | :------------- | :------------ | :------------------ | -| localhost | ::1 | IPv6 | 2024-02-11 17:04:40 | -| localhost | 127.0.0.1 | IPv4 | 2024-02-11 17:04:40 | - -**См. также** - -* [параметр disable_internal_dns_cache](../../operations/server-configuration-parameters/settings.md#disable_internal_dns_cache) -* [параметр dns_cache_max_entries](../../operations/server-configuration-parameters/settings.md#dns_cache_max_entries) -* [параметр dns_cache_update_period](../../operations/server-configuration-parameters/settings.md#dns_cache_update_period) -* [параметр dns_max_consecutive_failures](../../operations/server-configuration-parameters/settings.md#dns_max_consecutive_failures) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md deleted file mode 100644 index a9daf39aaee..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о таблицах, для которых был выполнен оператор DROP TABLE, но очистка данных ещё не выполнена' -keywords: ['system table', 'dropped_tables'] -slug: /operations/system-tables/dropped_tables -title: 'system.dropped_tables' -doc_type: 'reference' ---- - -Содержит информацию о таблицах, для которых был выполнен оператор DROP TABLE, но очистка данных ещё не выполнена. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Следующий пример демонстрирует, как получить информацию о `dropped_tables`. - -```sql -SELECT * -FROM system.dropped_tables\G -``` - -```text -Строка 1: -────── -index: 0 -database: default -table: test -uuid: 03141bb2-e97a-4d7c-a172-95cc066bb3bd -engine: MergeTree -metadata_dropped_path: /data/ClickHouse/build/programs/data/metadata_dropped/default.test.03141bb2-e97a-4d7c-a172-95cc066bb3bd.sql -table_dropped_time: 2023-03-16 23:43:31 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md deleted file mode 100644 index 498e57741a4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о частях удалённых таблиц - MergeTree из `system.dropped_tables`' -keywords: ['system table', 'dropped_tables_parts'] -slug: /operations/system-tables/dropped_tables_parts -title: 'system.dropped_tables_parts' -doc_type: 'reference' ---- - -Содержит информацию о частях удалённых таблиц [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) из [system.dropped_tables](./dropped_tables.md) - -Схема этой таблицы совпадает со схемой [system.parts](./parts.md) - -**См. также** - -- [Семейство MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) -- [system.parts](./parts.md) -- [system.dropped_tables](./dropped_tables.md) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md deleted file mode 100644 index 782533c5596..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Системная таблица, содержащая все активные роли на текущий момент, включая текущую роль текущего пользователя и предоставленные роли для текущей роли' -keywords: ['system table', 'enabled_roles'] -slug: /operations/system-tables/enabled_roles -title: 'system.enabled_roles' -doc_type: 'reference' ---- - -Содержит все активные роли на текущий момент, включая текущую роль текущего пользователя и предоставленные роли для текущей роли. - -Столбцы: - -- `role_name` ([String](../../sql-reference/data-types/string.md)) — Имя роли. -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, что `enabled_role` является ролью с привилегией `ADMIN OPTION`. -- `is_current` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, что `enabled_role` является текущей ролью текущего пользователя. -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, что `enabled_role` является ролью по умолчанию. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md deleted file mode 100644 index 7102e0905f4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: 'Системная таблица, содержащая историю значений ошибок из таблицы `system.errors`, - периодически сохраняемую на диск.' -keywords: ['системная таблица', 'error_log'] -slug: /operations/system-tables/system-error-log -title: 'system.error_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит историю значений ошибок из таблицы `system.errors`, периодически сбрасываемую на диск. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время события. -* `code` ([Int32](../../sql-reference/data-types/int-uint.md)) — Код ошибки. -* `error` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя ошибки. -* `value` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество возникновений этой ошибки. -* `remote` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Удалённое исключение (т. е. полученное во время одного из распределённых запросов). -* `last_error_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время, когда произошла последняя ошибка. -* `last_error_message` ([String](../../sql-reference/data-types/string.md)) — Сообщение последней ошибки. -* `last_error_query_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор запроса, который вызвал последнюю ошибку (если доступен). -* `last_error_trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — Стек вызовов, представляющий собой список физических адресов, по которым расположены вызываемые методы. - -**Пример** - -```sql -SELECT * FROM system.error_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.testing.internal -event_date: 2025-11-11 -event_time: 2025-11-11 11:35:28 -code: 60 -error: UNKNOWN_TABLE -value: 1 -remote: 0 -last_error_time: 2025-11-11 11:35:28 -last_error_message: Неизвестный идентификатор табличного выражения 'system.table_not_exist' в области SELECT * FROM system.table_not_exist -last_error_query_id: 77ad9ece-3db7-4236-9b5a-f789bce4aa2e -last_error_trace: [100506790044914,100506534488542,100506409937998,100506409936517,100506425182891,100506618154123,100506617994473,100506617990486,100506617988112,100506618341386,100506630272160,100506630266232,100506630276900,100506629795243,100506633519500,100506633495783,100506692143858,100506692248921,100506790779783,100506790781278,100506790390399,100506790380047,123814948752036,123814949330028] -``` - -**См. также** - -* [параметр error_log](../../operations/server-configuration-parameters/settings.md#error_log) — Включение и отключение этого параметра. -* [system.errors](../../operations/system-tables/errors.md) — Содержит коды ошибок с числом срабатываний каждой из них. -* [Мониторинг](../../operations/monitoring.md) — Базовые концепции мониторинга ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md deleted file mode 100644 index 25d93183d79..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Системная таблица, содержащая коды ошибок и количество их срабатываний.' -keywords: ['системная таблица', 'ошибки'] -slug: /operations/system-tables/errors -title: 'system.errors' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит коды ошибок с числом их срабатываний. - -Чтобы показать все возможные коды ошибок, включая те, которые ни разу не сработали, установите настройку [system_events_show_zero_values](../settings/settings.md#system_events_show_zero_values) в 1. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -:::note -Счётчики некоторых ошибок могут увеличиваться при успешном выполнении запроса. Не рекомендуется использовать эту таблицу для мониторинга сервера, если только вы не уверены, что соответствующая ошибка не может быть ложным срабатыванием. -::: - -**Пример** - -```sql -SELECT name, code, value -FROM system.errors -WHERE value > 0 -ORDER BY code ASC -LIMIT 1 - -┌─name─────────────┬─code─┬─value─┐ -│ CANNOT_OPEN_FILE │ 76 │ 1 │ -└──────────────────┴──────┴───────┘ -``` - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), last_error_trace) AS all -SELECT name, arrayStringConcat(all, '\n') AS res -FROM system.errors -LIMIT 1 -SETTINGS allow_introspection_functions=1\G -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/events.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/events.md deleted file mode 100644 index edfc372faf9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/events.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о количестве событий, произошедших в системе.' -keywords: ['системная таблица', 'события'] -slug: /operations/system-tables/events -title: 'system.events' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о числе событий, произошедших в системе. Например, в этой таблице можно узнать, сколько запросов `SELECT` было обработано с момента запуска сервера ClickHouse. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Вы можете найти все поддерживаемые события в файле исходного кода [src/Common/ProfileEvents.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/ProfileEvents.cpp). - -**Пример** - -```sql -SELECT * FROM system.events LIMIT 5 -``` - -```text -┌─event─────────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ Query │ 12 │ Количество запросов для интерпретации и возможного выполнения. Не включает запросы, которые не удалось разобрать или которые были отклонены из-за ограничений размера AST, квот или ограничений на количество одновременно выполняемых запросов. Может включать внутренние запросы, инициированные самим ClickHouse. Не учитывает подзапросы. │ -│ SelectQuery │ 8 │ То же, что и Query, но только для SELECT-запросов. │ -│ FileOpen │ 73 │ Количество открытых файлов. │ -│ ReadBufferFromFileDescriptorRead │ 155 │ Количество операций чтения (read/pread) из файлового дескриптора. Не включает сокеты. │ -│ ReadBufferFromFileDescriptorReadBytes │ 9931 │ Количество байтов, прочитанных из файловых дескрипторов. Если файл сжат, отображается размер сжатых данных. │ -└───────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — содержит периодически вычисляемые метрики. -* [system.metrics](/operations/system-tables/metrics) — содержит мгновенно вычисляемые метрики. -* [system.metric_log](/operations/system-tables/metric_log) — содержит историю значений метрик из таблиц `system.metrics` и `system.events`. -* [Monitoring](../../operations/monitoring.md) — основные концепции мониторинга ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md deleted file mode 100644 index a7f5ef73040..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об обычных и агрегатных функциях.' -keywords: ['системная таблица', 'функции'] -slug: /operations/system-tables/functions -title: 'system.functions' -doc_type: 'reference' ---- - -Содержит информацию об обычных и агрегатных функциях. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql - SELECT name, is_aggregate, is_deterministic, case_insensitive, alias_to FROM system.functions LIMIT 5; -``` - -```text -┌─name─────────────────────┬─is_aggregate─┬─is_deterministic─┬─case_insensitive─┬─alias_to─┐ -│ BLAKE3 │ 0 │ 1 │ 0 │ │ -│ sipHash128Reference │ 0 │ 1 │ 0 │ │ -│ mapExtractKeyLike │ 0 │ 1 │ 0 │ │ -│ sipHash128ReferenceKeyed │ 0 │ 1 │ 0 │ │ -│ mapPartialSort │ 0 │ 1 │ 0 │ │ -└──────────────────────────┴──────────────┴──────────────────┴──────────────────┴──────────┘ - -Выбрано 5 строк. Затрачено: 0.002 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md deleted file mode 100644 index 7bf9f8caf54..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о привилегиях, предоставленных учетным записям пользователей ClickHouse.' -keywords: ['system table', 'grants'] -slug: /operations/system-tables/grants -title: 'system.grants' -doc_type: 'reference' ---- - -Привилегии, предоставленные учетным записям пользователей ClickHouse. - -Столбцы: - -- `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя пользователя. - -- `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Роль, назначенная учетной записи пользователя. - -- `access_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип привилегии доступа для учетной записи пользователя ClickHouse. - -- `database` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя базы данных. - -- `table` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя таблицы. - -- `column` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя столбца, к которому предоставлен доступ. - -- `is_partial_revoke` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Логическое значение. Показывает, были ли отозваны некоторые привилегии. Возможные значения: -- `0` — Строка описывает операцию выдачи привилегий (grant). -- `1` — Строка описывает операцию частичного отзыва привилегий (partial revoke). - -- `grant_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Разрешение выдано с опцией `WITH GRANT OPTION`, см. [GRANT](../../sql-reference/statements/grant.md#granting-privilege-syntax). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md deleted file mode 100644 index 9e85ad2afdc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о параметрах `graphite_rollup`, - которые используются в таблицах с движками типа `GraphiteMergeTree`.' -keywords: ['системная таблица', 'graphite_retentions'] -slug: /operations/system-tables/graphite_retentions -title: 'system.graphite_retentions' -doc_type: 'reference' ---- - -Содержит информацию о параметрах [graphite_rollup](../../operations/server-configuration-parameters/settings.md#graphite), которые используются в таблицах с движками типа [*GraphiteMergeTree](../../engines/table-engines/mergetree-family/graphitemergetree.md). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md deleted file mode 100644 index 3da9252389e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'Эта таблица содержит гистограммные метрики, которые можно мгновенно - вычислять и экспортировать в формате Prometheus. Она всегда актуальна.' -keywords: ['системная таблица', 'histogram_metrics'] -slug: /operations/system-tables/histogram_metrics -title: 'system.histogram_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# histogram_metrics {#histogram_metrics} - - - -Эта таблица содержит гистограммные метрики, которые можно мгновенно рассчитать и экспортировать в формате Prometheus. Она всегда находится в актуальном состоянии. Заменяет устаревшую таблицу `system.latency_log`. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Вы можете использовать такой запрос, чтобы экспортировать все метрики типа histogram в формате Prometheus. - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'histogram' AS type -FROM system.histogram_metrics -FORMAT Prometheus -``` - - -## Описание метрик {#metric_descriptions} - -### keeper_response_time_ms_bucket {#keeper_response_time_ms_bucket} - -Время ответа Keeper в миллисекундах. - -**См. также** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — Содержит периодически вычисляемые метрики. -- [system.events](/operations/system-tables/events) — Содержит информацию о произошедших событиях. -- [system.metric_log](/operations/system-tables/metric_log) — Содержит историю значений метрик таблиц `system.metrics` и `system.events`. -- [Monitoring](../../operations/monitoring.md) — Базовые концепции мониторинга ClickHouse. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md deleted file mode 100644 index 288c9ab9894..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'История снимков Iceberg в системной таблице' -keywords: ['system iceberg_history'] -slug: /operations/system-tables/iceberg_history -title: 'system.iceberg_history' -doc_type: 'reference' ---- - -# system.iceberg_history {#systemiceberg_history} - -Эта системная таблица содержит историю снимков таблиц Iceberg в ClickHouse. Она будет пустой, если в ClickHouse нет ни одной таблицы Iceberg. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md deleted file mode 100644 index a92fa6adc72..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о файлах метаданных, прочитанных из таблиц Iceberg. Каждая запись - представляет либо корневой файл метаданных, метаданные, извлечённые из файла Avro, либо запись из файла Avro.' -keywords: ['системная таблица', 'iceberg_metadata_log'] -slug: /operations/system-tables/iceberg_metadata_log -title: 'system.iceberg_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.iceberg_metadata_log {#systemiceberg_metadata_log} - -Таблица `system.iceberg_metadata_log` фиксирует события доступа к метаданным и их разбора для таблиц Iceberg, прочитанных ClickHouse. Она предоставляет подробную информацию о каждом обработанном файле или записи метаданных, что полезно для отладки, аудита и анализа эволюции структуры таблиц Iceberg. - - - -## Назначение {#purpose} - -Эта таблица фиксирует каждый файл и каждую запись метаданных, прочитанные из таблиц Iceberg, включая корневые файлы метаданных, списки манифестов и записи манифестов. Она помогает пользователям отследить, как ClickHouse интерпретирует метаданные таблиц Iceberg, и диагностировать проблемы, связанные с эволюцией схемы, поиском и разрешением файлов или планированием выполнения запросов. - -:::note -Эта таблица в первую очередь предназначена для отладки. -::: - - - -## Столбцы {#columns} - -| Имя | Тип | Описание | -|---------------|----------|------------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | Дата записи лога. | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | Временная метка события. | -| `query_id` | [String](../../sql-reference/data-types/string.md) | Идентификатор запроса, который инициировал чтение метаданных. | -| `content_type` | [Enum8](../../sql-reference/data-types/enum.md) | Тип содержимого метаданных (см. ниже). | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Путь к таблице Iceberg. | -| `file_path` | [String](../../sql-reference/data-types/string.md) | Путь к корневому файлу метаданных в формате JSON, списку манифестов Avro или файлу манифеста. | -| `content` | [String](../../sql-reference/data-types/string.md) | Содержимое в формате JSON (исходные метаданные из файла .json, метаданные Avro или запись Avro). | -| `row_in_file` | [Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md)) | Номер строки в файле, если применимо. Заполняется для типов содержимого `ManifestListEntry` и `ManifestFileEntry`. | - - - -## Значения `content_type` {#content-type-values} - -- `None`: Нет содержимого. -- `Metadata`: Корневой файл метаданных. -- `ManifestListMetadata`: Метаданные списка манифестов. -- `ManifestListEntry`: Запись в списке манифестов. -- `ManifestFileMetadata`: Метаданные файла манифеста. -- `ManifestFileEntry`: Запись в файле манифеста. - - - - - -## Управление подробностью журналирования {#controlling-log-verbosity} - -Вы можете управлять тем, какие события, связанные с метаданными, записываются в журнал, с помощью настройки [`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level). - -Чтобы записывать в журнал все метаданные, используемые в текущем запросе: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'manifest_file_entry'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -Чтобы логировать только корневой JSON‑файл метаданных, используемый в текущем запросе: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'metadata'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -См. дополнительную информацию в описании настройки [`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level). - -### Полезная информация {#good-to-know} - -* Используйте `iceberg_metadata_log_level` на уровне запроса только тогда, когда вам нужно детально исследовать таблицу Iceberg. В противном случае вы можете заполнить таблицу логов избыточными метаданными и столкнуться с ухудшением производительности. -* Таблица может содержать дублирующиеся записи, так как она предназначена в первую очередь для отладки и не гарантирует уникальность записей для каждой сущности. -* Если вы используете `content_type`, более подробный, чем `ManifestListMetadata`, кэш метаданных Iceberg для списков манифестов отключается. -* Аналогично, если вы используете `content_type`, более подробный, чем `ManifestFileMetadata`, кэш метаданных Iceberg для файлов манифестов отключается. - - -## См. также {#see-also} -- [Движок таблиц Iceberg](../../engines/table-engines/integrations/iceberg.md) -- [Табличная функция Iceberg](../../sql-reference/table-functions/iceberg.md) -- [system.iceberg_history](./iceberg_history.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/index.md deleted file mode 100644 index 8f1f2ea2a37..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/index.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -description: 'Обзор того, что такое системные таблицы и почему они полезны.' -keywords: ['системные таблицы', 'обзор'] -pagination_next: operations/system-tables/asynchronous_metric_log -sidebar_label: 'Обзор' -sidebar_position: 52 -slug: /operations/system-tables/ -title: 'Системные таблицы' -doc_type: 'reference' ---- - -{/* Таблица оглавления для этой страницы автоматически генерируется скриптом - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - на основе полей YAML front matter: slug, description, title. - - Если вы заметили ошибку, отредактируйте YML front matter самих страниц. - */ } - -{/*AUTOGENERATED_START*/ } - - -| Страница | Описание | -| ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| [Обзор системных таблиц](/operations/system-tables/overview) | Обзор системных таблиц и причин, по которым они полезны. | -| [INFORMATION_SCHEMA](/operations/system-tables/information_schema) | Системная база данных, обеспечивающая почти стандартизированное представление метаданных объектов базы данных, не зависящее от конкретной СУБД. | -| [system.asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) | Системная таблица, содержащая информацию об асинхронных вставках. Каждая запись соответствует запросу на вставку, помещённому в буфер асинхронных вставок. | -| [system.asynchronous_inserts](/operations/system-tables/asynchronous_inserts) | Системная таблица, содержащая информацию об ожидающих выполнения в очереди асинхронных вставках. | -| [system.asynchronous_loader](/operations/system-tables/asynchronous_loader) | Системная таблица, содержащая информацию и статус недавних асинхронных заданий (например, для таблиц, которые загружаются). Таблица содержит по одной строке для каждого задания. | -| [system.asynchronous_metric_log](/operations/system-tables/asynchronous_metric_log) | Системная таблица, содержащая исторические значения из `system.asynchronous_metrics`, которые сохраняются один раз за интервал времени (по умолчанию — раз в секунду) | -| [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) | Системная таблица, содержащая метрики, которые периодически рассчитываются в фоновом режиме. Например, объем используемой оперативной памяти. | -| [system.azure_queue_settings](/operations/system-tables/azure_queue_settings) | Системная таблица, содержащая информацию о настройках таблиц AzureQueue. Доступна, начиная с версии сервера `24.10`. | -| [system.backup_log](/operations/system-tables/backup_log) | Системная таблица, содержащая записи журнала с информацией об операциях `BACKUP` и `RESTORE`. | -| [system.backups](/operations/system-tables/backups) | Системная таблица с записями журнала, содержащими информацию об операциях `BACKUP` и `RESTORE`. | -| [system.blob_storage_log](/operations/system-tables/blob_storage_log) | Системная таблица, содержащая журнальные записи с информацией о различных операциях с blob‑хранилищем, таких как загрузка и удаление. | -| [system.build_options](/operations/system-tables/build_options) | Системная таблица, содержащая информацию об опциях сборки сервера ClickHouse. | -| [system.clusters](/operations/system-tables/clusters) | Системная таблица, содержащая информацию о кластерах, доступных в конфигурационном файле, и серверах, описанных в них. | -| [system.codecs](/operations/system-tables/codecs) | Системная таблица с информацией о кодеках в очереди. | -| [system.columns](/operations/system-tables/columns) | Системная таблица, содержащая сведения о столбцах всех таблиц | -| [system.contributors](/operations/system-tables/contributors) | Системная таблица с информацией об участниках. | -| [system.crash_log](/operations/system-tables/crash_log) | Системная таблица, содержащая информацию о стек-трейсах при фатальных ошибках. | -| [system.current_roles](/operations/system-tables/current_roles) | Системная таблица, содержащая активные роли для текущего пользователя. | -| [system.dashboards](/operations/system-tables/dashboards) | Содержит запросы, используемые страницей `/dashboard`, доступной по HTTP-интерфейсу. Полезно для мониторинга и устранения неполадок. | -| [system.data_skipping_indices](/operations/system-tables/data_skipping_indices) | Системная таблица, содержащая информацию о существующих индексах пропуска данных во всех таблицах. | -| [system.data_type_families](/operations/system-tables/data_type_families) | Системная таблица, содержащая сведения о поддерживаемых типах данных | -| [system.database_engines](/operations/system-tables/database_engines) | Системная таблица со списком движков баз данных, поддерживаемых сервером. | -| [system.database_replicas](/operations/system-tables/database_replicas) | Системная таблица, содержащая сведения о реплицируемой базе данных и её состоянии. | -| [system.databases](/operations/system-tables/databases) | Системная таблица, содержащая информацию о базах данных, доступных для текущего пользователя. | -| [system.dead_letter_queue](/operations/system-tables/dead_letter_queue) | Системная таблица, содержащая сведения о сообщениях, полученных из движка потоковой обработки и разобранных с ошибками. | -| [system.delta_lake_metadata_log](/operations/system-tables/delta_lake_metadata_log) | Системная таблица, содержащая информацию о файлах метаданных, прочитанных из таблиц Delta Lake. Каждая запись представляет корневой JSON‑файл метаданных. | -| [system.detached_parts](/operations/system-tables/detached_parts) | Системная таблица с информацией об отсоединённых частях таблиц MergeTree | -| [system.detached_tables](/operations/system-tables/detached_tables) | Системная таблица, содержащая информацию о каждой отсоединённой таблице. | -| [system.dictionaries](/operations/system-tables/dictionaries) | Системная таблица с информацией о словарях | -| [system.dimensional_metrics](/operations/system-tables/dimensional_metrics) | Эта таблица содержит метрики с измерениями, которые можно мгновенно вычислить и экспортировать в формате Prometheus. Данные в ней всегда актуальны. | -| [system.disks](/operations/system-tables/disks) | Системная таблица, содержащая информацию о дисках, определённых в конфигурации сервера | -| [system.distributed_ddl_queue](/operations/system-tables/distributed_ddl_queue) | Системная таблица, содержащая информацию о распределённых DDL-запросах (запросах с использованием предложения ON CLUSTER), которые были выполнены на кластере. | -| [system.distribution_queue](/operations/system-tables/distribution_queue) | Системная таблица, содержащая информацию о локальных файлах, находящихся в очереди к отправке в сегменты. | -| [system.dns_cache](/operations/system-tables/dns_cache) | Системная таблица, содержащая информацию о кэшированных DNS-записях. | -| [system.dropped_tables](/operations/system-tables/dropped_tables) | Системная таблица, содержащая информацию о таблицах, для которых была выполнена операция DROP TABLE, но очистка данных ещё не произведена | -| [system.dropped_tables_parts](/operations/system-tables/dropped_tables_parts) | Системная таблица, содержащая информацию о частях удалённых таблиц MergeTree из `system.dropped_tables` | -| [system.enabled_roles](/operations/system-tables/enabled_roles) | Системная таблица, содержащая все активные роли на данный момент, включая текущую роль текущего пользователя и роли, назначенные текущей роли | -| [system.error_log](/operations/system-tables/system-error-log) | Системная таблица, содержащая историю значений ошибок из таблицы `system.errors` и периодически записываемая на диск. | -| [system.errors](/operations/system-tables/errors) | Системная таблица, содержащая коды ошибок и число их срабатываний. | -| [system.events](/operations/system-tables/events) | Системная таблица, содержащая сведения о количестве событий, произошедших в системе. | -| [system.functions](/operations/system-tables/functions) | Системная таблица, содержащая информацию об обычных и агрегатных функциях. | -| [system.grants](/operations/system-tables/grants) | Системная таблица, в которой указано, какие привилегии предоставлены учетным записям пользователей ClickHouse. | -| [system.graphite_retentions](/operations/system-tables/graphite_retentions) | Системная таблица, содержащая информацию о параметрах `graphite_rollup`, которые используются в таблицах с движком `GraphiteMergeTree`. | -| [system.histogram_metrics](/operations/system-tables/histogram_metrics) | Эта таблица содержит гистограммные метрики, которые можно мгновенно вычислить и экспортировать в формате Prometheus. Она всегда содержит актуальные данные. | -| [system.iceberg_history](/operations/system-tables/iceberg_history) | История системных снимков Iceberg | -| [system.iceberg_metadata_log](/operations/system-tables/iceberg_metadata_log) | Системная таблица, содержащая информацию о файлах метаданных, прочитанных из таблиц Iceberg. Каждая запись представляет либо корневой файл метаданных, либо метаданные, извлечённые из файла Avro, либо отдельную запись файла Avro. | -| [system.jemalloc_bins](/operations/system-tables/jemalloc_bins) | Системная таблица, содержащая информацию о выделениях памяти, выполняемых аллокатором jemalloc в различных классах размеров (bins) и агрегируемых по всем аренам. | -| [system.kafka_consumers](/operations/system-tables/kafka_consumers) | Системная таблица, содержащая информацию о консьюмерах Kafka. | -| [system.licenses](/operations/system-tables/licenses) | Системная таблица, содержащая лицензии сторонних библиотек, которые расположены в каталоге contrib исходного кода ClickHouse. | -| [system.merge_tree_settings](/operations/system-tables/merge_tree_settings) | Системная таблица, содержащая информацию о настройках для таблиц MergeTree. | -| [system.merges](/operations/system-tables/merges) | Системная таблица, содержащая информацию о выполняющихся в данный момент слияниях и мутациях частей таблиц семейства MergeTree. | -| [system.metric_log](/operations/system-tables/metric_log) | Системная таблица, содержащая историю значений метрик из таблиц `system.metrics` и `system.events`, данные которой периодически сбрасываются на диск. | -| [system.metrics](/operations/system-tables/metrics) | Системная таблица, содержащая метрики, которые могут вычисляться на лету или отражают текущее значение. | -| [system.moves](/operations/system-tables/moves) | Системная таблица, содержащая информацию о текущих перемещениях частей данных таблиц MergeTree. Каждое перемещение части данных отображается одной строкой. | -| [system.mutations](/operations/system-tables/mutations) | Системная таблица, содержащая информацию о мутациях таблиц MergeTree и ходе их выполнения. Каждая операция мутации представлена одной строкой. | -| [system.numbers](/operations/system-tables/numbers) | Системная таблица, содержащая единственный столбец типа UInt64 с именем `number`, в котором хранятся почти все натуральные числа, начиная с нуля. | -| [system.numbers_mt](/operations/system-tables/numbers_mt) | Системная таблица, аналогичная `system.numbers`, но чтение выполняется в параллельном режиме, и числа могут возвращаться в произвольном порядке. | -| [system.one](/operations/system-tables/one) | Системная таблица, содержащая единственную строку с единственным столбцом `dummy` типа UInt8 со значением 0. Аналогична таблице `DUAL`, используемой в других СУБД. | -| [system.opentelemetry_span_log](/operations/system-tables/opentelemetry_span_log) | Системная таблица, содержащая информацию о спанах трассировки для выполненных запросов. | -| [system.part_log](/operations/system-tables/part_log) | Системная таблица, содержащая информацию о событиях, происходивших с частями данных в таблицах семейства MergeTree, таких как добавление или слияние частей данных. | -| [system.parts](/operations/system-tables/parts) | Системная таблица, содержащая информацию о частях таблиц семейства MergeTree | -| [system.parts_columns](/operations/system-tables/parts_columns) | Системная таблица, содержащая информацию о частях и столбцах таблиц MergeTree. | -| [system.processes](/operations/system-tables/processes) | Системная таблица, используемая для реализации запроса `SHOW PROCESSLIST`. | -| [system.processors_profile_log](/operations/system-tables/processors_profile_log) | Системная таблица, содержащая данные профилирования на уровне процессоров (их можно получить с помощью `EXPLAIN PIPELINE`) | -| [system.projection_parts](/operations/system-tables/projection_parts) | Системная таблица, содержащая информацию о частях проекций таблиц семейства MergeTree. | -| [system.projection_parts_columns](/operations/system-tables/projection_parts_columns) | Системная таблица, содержащая информацию о столбцах частей проекций для таблиц семейства MergeTree | -| [system.projections](/operations/system-tables/projections) | Системная таблица, содержащая информацию о существующих проекциях во всех таблицах. | -| [system.query_cache](/operations/system-tables/query_cache) | Системная таблица, отображающая содержимое кэша запросов. | -| [system.query_condition_cache](/operations/system-tables/query_condition_cache) | Системная таблица, показывающая содержимое кэша условий запроса. | -| [system.query_log](/operations/system-tables/query_log) | Системная таблица, содержащая информацию о выполненных запросах, таких как время начала, длительность обработки и сообщения об ошибках. | -| [system.query_metric_log](/operations/system-tables/query_metric_log) | Системная таблица, содержащая историю значений использования памяти и метрик из таблицы `system.events` для отдельных запросов, которая периодически сбрасывается на диск. | -| [system.query_thread_log](/operations/system-tables/query_thread_log) | Системная таблица, содержащая информацию о потоках, которые выполняют запросы, например, имя потока, время его запуска, длительность обработки запроса. | -| [system.query_views_log](/operations/system-tables/query_views_log) | Системная таблица, содержащая информацию о зависимых представлениях, выполненных при выполнении запроса, например о типе представления или времени выполнения. | -| [system.quota_limits](/operations/system-tables/quota_limits) | Системная таблица, содержащая информацию о максимальных значениях для всех интервалов всех квот. Одной квоте может соответствовать любое количество строк или ни одной. | -| [system.quota_usage](/operations/system-tables/quota_usage) | Системная таблица, содержащая информацию об использовании QUOTA текущим USER, в частности о том, какая часть QUOTA уже использована и сколько ещё доступно. | -| [system.quotas](/operations/system-tables/quotas) | Системная таблица с информацией о QUOTA. | -| [system.quotas_usage](/operations/system-tables/quotas_usage) | Системная таблица, содержащая информацию об использовании QUOTA всеми пользователями. | -| [system.replicas](/operations/system-tables/replicas) | Системная таблица, содержащая информацию о реплицированных таблицах, размещённых на локальном сервере, и их состоянии. Полезна для мониторинга. | -| [system.replicated_fetches](/operations/system-tables/replicated_fetches) | Системная таблица, содержащая информацию о выполняющихся в данный момент фоновых операциях выборки данных. | -| [system.replication_queue](/operations/system-tables/replication_queue) | Системная таблица, содержащая информацию о задачах из очередей репликации в ClickHouse Keeper или ZooKeeper для таблиц семейства `ReplicatedMergeTree`. | -| [system.resources](/operations/system-tables/resources) | Системная таблица, содержащая информацию о ресурсах, находящихся на локальном сервере, по одной строке для каждого ресурса. | -| [system.role_grants](/operations/system-tables/role_grants) | Системная таблица, содержащая назначения ролей для пользователей и ролей. | -| [system.roles](/operations/system-tables/roles) | Системная таблица, содержащая информацию о сконфигурированных ролях. | -| [system.row_policies](/operations/system-tables/row_policies) | Системная таблица, содержащая фильтры для одной таблицы, а также список ролей и/или пользователей, для которых применяется эта ROW POLICY. | -| [system.s3_queue_settings](/operations/system-tables/s3_queue_settings) | Системная таблица, содержащая информацию о настройках таблиц S3Queue. Доступна начиная с версии сервера `24.10`. | -| [system.scheduler](/operations/system-tables/scheduler) | Системная таблица, содержащая сведения об узлах планировщика, находящихся на локальном сервере, и их состоянии. | -| [system.schema_inference_cache](/operations/system-tables/schema_inference_cache) | Системная таблица, содержащая информацию обо всех кэшированных схемах файлов. | -| [system.server_settings](/operations/system-tables/server_settings) | Системная таблица, содержащая информацию о глобальных настройках сервера, задаваемых в `config.xml`. | -| [system.session_log](/operations/system-tables/session_log) | Системная таблица с информацией обо всех успешных и неуспешных входах в систему и выходах из неё. | -| [system.settings](/operations/system-tables/settings) | Системная таблица, содержащая информацию о настройках сессии текущего пользователя. | -| [system.settings_changes](/operations/system-tables/settings_changes) | Системная таблица, содержащая информацию об изменениях настроек в более ранних версиях ClickHouse. | -| [system.settings_profile_elements](/operations/system-tables/settings_profile_elements) | Системная таблица, описывающая содержимое профиля настроек: ограничения, роли и пользователей, к которым применяется настройка, а также родительские профили настроек. | -| [system.settings_profiles](/operations/system-tables/settings_profiles) | Системная таблица, содержащая свойства сконфигурированных профилей настроек. | -| [system.stack_trace](/operations/system-tables/stack_trace) | Системная таблица, содержащая стек-трейсы всех серверных потоков. Позволяет разработчикам исследовать состояние сервера. | -| [system.storage_policies](/operations/system-tables/storage_policies) | Системная таблица, содержащая информацию о политиках хранения и томах, заданных в конфигурации сервера. | -| [system.symbols](/operations/system-tables/symbols) | Системная таблица, полезная для специалистов по C++ и инженеров ClickHouse, содержащая сведения для детального анализа двоичного файла `clickhouse`. | -| [system.table_engines](/operations/system-tables/table_engines) | Системная таблица, содержащая описания движков таблиц, поддерживаемых сервером, и их возможностей. | -| [system.tables](/operations/system-tables/tables) | Системная таблица, содержащая метаданные о каждой таблице, известной серверу. | -| [system.text_log](/operations/system-tables/text_log) | Системная таблица, содержащая записи логирования. | -| [system.time_zones](/operations/system-tables/time_zones) | Системная таблица, содержащая список часовых поясов, поддерживаемых сервером ClickHouse. | -| [system.trace_log](/operations/system-tables/trace_log) | Системная таблица, содержащая трассировки стека, собранные профилировщиком запросов с выборочным профилированием. | -| [system.unicode](/operations/system-tables/unicode) | Системная таблица со списком символов Unicode и их свойств. | -| [system.user_processes](/operations/system-tables/user_processes) | Системная таблица, содержащая сведения для общего представления об использовании памяти и ProfileEvents пользователей. | -| [system.users](/operations/system-tables/users) | Системная таблица со списком учетных записей пользователей, настроенных на сервере. | -| [system.view_refreshes](/operations/system-tables/view_refreshes) | Системная таблица, содержащая информацию о refreshable materialized views. | -| [system.warnings](/operations/system-tables/system_warnings) | Эта таблица содержит предупреждения о сервере ClickHouse. | -| [system.workloads](/operations/system-tables/workloads) | Системная таблица, содержащая информацию о рабочих нагрузках, выполняемых на локальном сервере. | -| [system.zookeeper](/operations/system-tables/zookeeper) | Системная таблица, которая существует только если настроены ClickHouse Keeper или ZooKeeper. Предоставляет данные из кластера Keeper, определённого в конфигурации. | -| [system.zookeeper_connection](/operations/system-tables/zookeeper_connection) | Системная таблица, которая существует только при настроенном ZooKeeper. Показывает текущие подключения к ZooKeeper (включая вспомогательные экземпляры ZooKeeper). | -| [system.zookeeper_connection_log](/operations/system-tables/zookeeper_connection_log) | Показывает историю соединений с ZooKeeper (включая дополнительные серверы ZooKeeper). | -| [system.zookeeper_log](/operations/system-tables/zookeeper_log) | Системная таблица, содержащая информацию о параметрах запроса к серверу ZooKeeper и полученном от него ответе. | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md deleted file mode 100644 index 9f641cf5a51..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md +++ /dev/null @@ -1,392 +0,0 @@ ---- -description: 'Системная база данных, предоставляющая почти стандартизированное, - не зависяющее от СУБД представление о метаданных объектов базы данных.' -keywords: ['системная база данных', 'information_schema'] -slug: /operations/system-tables/information_schema -title: 'INFORMATION_SCHEMA' -doc_type: 'reference' ---- - -`INFORMATION_SCHEMA` (или: `information_schema`) — это системная база данных, которая предоставляет (в некоторой степени) стандартизированное, [независимое от СУБД представление](https://en.wikipedia.org/wiki/Information_schema) о метаданных объектов базы данных. Представления в `INFORMATION_SCHEMA` в целом уступают обычным системным таблицам, но инструменты могут использовать их для получения базовой информации в едином для разных СУБД формате. Структура и содержимое представлений в `INFORMATION_SCHEMA` должны развиваться с сохранением обратной совместимости, т.е. добавляется только новая функциональность, а существующая функциональность не изменяется и не удаляется. С точки зрения внутренней реализации представления в `INFORMATION_SCHEMA` обычно сопоставляются с обычными системными таблицами, такими как [system.columns](../../operations/system-tables/columns.md), [system.databases](../../operations/system-tables/databases.md) и [system.tables](../../operations/system-tables/tables.md). - -```sql -SHOW TABLES FROM INFORMATION_SCHEMA; - --- или: -SHOW TABLES FROM information_schema; -``` - -```text -┌─name────────────────────┐ -│ COLUMNS │ -│ KEY_COLUMN_USAGE │ -│ REFERENTIAL_CONSTRAINTS │ -│ SCHEMATA │ -| STATISTICS | -│ TABLES │ -│ VIEWS │ -│ columns │ -│ key_column_usage │ -│ referential_constraints │ -│ schemata │ -| statistics | -│ tables │ -│ views │ -└─────────────────────────┘ -``` - -`INFORMATION_SCHEMA` содержит следующие представления: - -* [COLUMNS](#columns) -* [KEY_COLUMN_USAGE](#key_column_usage) -* [REFERENTIAL_CONSTRAINTS](#referential_constraints) -* [SCHEMATA](#schemata) -* [STATISTICS](#statistics) -* [TABLES](#tables) -* [VIEWS](#views) - -Эквивалентные, нечувствительные к регистру представления, например `INFORMATION_SCHEMA.columns`, доступны для обеспечения совместимости с другими системами управления базами данных. То же самое относится ко всем столбцам в этих представлениях — доступны варианты как в нижнем регистре (например, `table_name`), так и в верхнем (`TABLE_NAME`). - - -## СТОЛБЦЫ {#columns} - -Содержит столбцы, считанные из системной таблицы [system.columns](../../operations/system-tables/columns.md), а также столбцы, которые не поддерживаются в ClickHouse или не имеют смысла (всегда `NULL`), но должны присутствовать согласно стандарту. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Запрос: - -```sql -SELECT table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - column_default, - is_nullable, - data_type, - character_maximum_length, - character_octet_length, - numeric_precision, - numeric_precision_radix, - numeric_scale, - datetime_precision, - character_set_catalog, - character_set_schema, - character_set_name, - collation_catalog, - collation_schema, - collation_name, - domain_catalog, - domain_schema, - domain_name, - column_comment, - column_type -FROM INFORMATION_SCHEMA.COLUMNS -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -Результат: - -```text -Строка 1: -────── -table_catalog: default -table_schema: default -table_name: describe_example -column_name: id -ordinal_position: 1 -column_default: -is_nullable: 0 -data_type: UInt64 -character_maximum_length: ᴺᵁᴸᴸ -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: 64 -numeric_precision_radix: 2 -numeric_scale: 0 -datetime_precision: ᴺᵁᴸᴸ -character_set_catalog: ᴺᵁᴸᴸ -character_set_schema: ᴺᵁᴸᴸ -character_set_name: ᴺᵁᴸᴸ -collation_catalog: ᴺᵁᴸᴸ -collation_schema: ᴺᵁᴸᴸ -collation_name: ᴺᵁᴸᴸ -domain_catalog: ᴺᵁᴸᴸ -domain_schema: ᴺᵁᴸᴸ -domain_name: ᴺᵁᴸᴸ -``` - - -## SCHEMATA {#schemata} - -Содержит столбцы, получаемые из системной таблицы [system.databases](../../operations/system-tables/databases.md), а также столбцы, которые не поддерживаются в ClickHouse или не имеют смысла (всегда `NULL`), но предусмотрены стандартом. - -Столбцы: - -* `catalog_name` ([String](../../sql-reference/data-types/string.md)) — имя базы данных. -* `schema_name` ([String](../../sql-reference/data-types/string.md)) — имя базы данных. -* `schema_owner` ([String](../../sql-reference/data-types/string.md)) — имя владельца схемы, всегда `'default'`. -* `default_character_set_catalog` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`, не поддерживается. -* `default_character_set_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`, не поддерживается. -* `default_character_set_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`, не поддерживается. -* `sql_path` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`, не поддерживается. - -**Пример** - -Запрос: - -```sql -SELECT catalog_name, - schema_name, - schema_owner, - default_character_set_catalog, - default_character_set_schema, - default_character_set_name, - sql_path -FROM information_schema.schemata -WHERE schema_name ILIKE 'information_schema' -LIMIT 1 -FORMAT Vertical; -``` - -Результат: - -```text -Row 1: -────── -catalog_name: INFORMATION_SCHEMA -schema_name: INFORMATION_SCHEMA -schema_owner: default -default_character_set_catalog: ᴺᵁᴸᴸ -default_character_set_schema: ᴺᵁᴸᴸ -default_character_set_name: ᴺᵁᴸᴸ -sql_path: ᴺᵁᴸᴸ -``` - - -## TABLES {#tables} - -Содержит столбцы, прочитанные из системной таблицы [system.tables](../../operations/system-tables/tables.md). - -Столбцы: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой находится таблица. -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой находится таблица. -* `table_name` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы. -* `table_type` ([String](../../sql-reference/data-types/string.md)) — Тип таблицы. Возможные значения: - * `BASE TABLE` - * `VIEW` - * `FOREIGN TABLE` - * `LOCAL TEMPORARY` - * `SYSTEM VIEW` -* `table_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Общее число строк. NULL, если значение не удалось определить. -* `data_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Размер данных на диске. NULL, если значение не удалось определить. -* `index_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Общий размер первичного ключа, вторичных индексов и всех меток. -* `table_collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Коллация по умолчанию для таблицы. Всегда `utf8mb4_0900_ai_ci`. -* `table_comment` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Комментарий, использованный при создании таблицы. - -**Пример** - -Запрос: - -```sql -SELECT table_catalog, - table_schema, - table_name, - table_type, - table_collation, - table_comment -FROM INFORMATION_SCHEMA.TABLES -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -Результат: - -```text -Строка 1: -────── -table_catalog: default -table_schema: default -table_name: describe_example -table_type: BASE TABLE -table_collation: utf8mb4_0900_ai_ci -table_comment: -``` - - -## ПРЕДСТАВЛЕНИЯ {#views} - -Содержит столбцы, считываемые из системной таблицы [system.tables](../../operations/system-tables/tables.md), когда используется движок таблицы [View](../../engines/table-engines/special/view.md). - -Столбцы: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой расположена таблица. -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой расположена таблица. -* `table_name` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы. -* `view_definition` ([String](../../sql-reference/data-types/string.md)) — `SELECT`-запрос для представления. -* `check_option` ([String](../../sql-reference/data-types/string.md)) — `NONE`, проверка не выполняется. -* `is_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`, представление не обновляется. -* `is_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — Указывает, является ли созданное представление [материализованным](/sql-reference/statements/create/view#materialized-view). Возможные значения: - * `NO` — Созданное представление не является материализованным. - * `YES` — Созданное представление является материализованным. -* `is_trigger_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`, триггер не обновляется. -* `is_trigger_deletable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`, триггер не удаляется. -* `is_trigger_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`, данные в триггер не вставляются. - -**Пример** - -Запрос: - -```sql -CREATE VIEW v (n Nullable(Int32), f Float64) AS SELECT n, f FROM t; -CREATE MATERIALIZED VIEW mv ENGINE = Null AS SELECT * FROM system.one; -SELECT table_catalog, - table_schema, - table_name, - view_definition, - check_option, - is_updatable, - is_insertable_into, - is_trigger_updatable, - is_trigger_deletable, - is_trigger_insertable_into -FROM information_schema.views -WHERE table_schema = currentDatabase() -LIMIT 1 -FORMAT Vertical; -``` - -Результат: - -```text -Row 1: -────── -table_catalog: default -table_schema: default -table_name: mv -view_definition: SELECT * FROM system.one -check_option: NONE -is_updatable: NO -is_insertable_into: YES -is_trigger_updatable: NO -is_trigger_deletable: NO -is_trigger_insertable_into: NO -``` - - -## KEY_COLUMN_USAGE {#key_column_usage} - -Содержит столбцы из системной таблицы [system.tables](../../operations/system-tables/tables.md), для которых заданы ограничения. - -Столбцы: - -* `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. Всегда `def`. -* `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — Имя схемы (базы данных), к которой относится ограничение. -* `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя ограничения. -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. Всегда `def`. -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — Имя схемы (базы данных), к которой относится таблица. -* `table_name` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы, к которой относится ограничение. -* `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя столбца, к которому относится ограничение. -* `ordinal_position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — В настоящее время не используется. Всегда `1`. -* `position_in_unique_constraint` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt32](../../sql-reference/data-types/int-uint.md))) — В настоящее время не используется. Всегда `NULL`. -* `referenced_table_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. Всегда NULL. -* `referenced_table_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. Всегда NULL. -* `referenced_column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. Всегда NULL. - -**Пример** - -```sql -CREATE TABLE test (i UInt32, s String) ENGINE MergeTree ORDER BY i; -SELECT constraint_catalog, - constraint_schema, - constraint_name, - table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - position_in_unique_constraint, - referenced_table_schema, - referenced_table_name, - referenced_column_name -FROM information_schema.key_column_usage -WHERE table_name = 'test' -FORMAT Vertical; -``` - -Результат: - -```response -Row 1: -────── -constraint_catalog: def -constraint_schema: default -constraint_name: PRIMARY -table_catalog: def -table_schema: default -table_name: test -column_name: i -ordinal_position: 1 -position_in_unique_constraint: ᴺᵁᴸᴸ -referenced_table_schema: ᴺᵁᴸᴸ -referenced_table_name: ᴺᵁᴸᴸ -referenced_column_name: ᴺᵁᴸᴸ -``` - - -## REFERENTIAL_CONSTRAINTS {#referential_constraints} - -Содержит информацию о внешних ключах. В настоящее время возвращает пустой результат (без строк), что достаточно для обеспечения совместимости со сторонними инструментами, такими как Tableau Online. - -Столбцы: - -- `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — в настоящее время не используется. -- `unique_constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `unique_constraint_schema` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `unique_constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — в настоящее время не используется. -- `match_option` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `update_rule` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `delete_rule` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `table_name` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. -- `referenced_table_name` ([String](../../sql-reference/data-types/string.md)) — в настоящее время не используется. - - - -## STATISTICS {#statistics} - -Содержит сведения об индексах таблиц. Сейчас возвращает пустой результат (без строк), чего достаточно для обеспечения совместимости со сторонними инструментами, такими как Tableau Online. - -Столбцы: - -- `table_catalog` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `table_schema` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `table_name` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `non_unique` ([Int32](../../sql-reference/data-types/int-uint.md)) — В настоящее время не используется. -- `index_schema` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `index_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. -- `seq_in_index` ([UInt32](../../sql-reference/data-types/int-uint.md)) — В настоящее время не используется. -- `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. -- `collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. -- `cardinality` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — В настоящее время не используется. -- `sub_part` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — В настоящее время не используется. -- `packed` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. -- `nullable` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `index_type` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `comment` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `index_comment` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `is_visible` ([String](../../sql-reference/data-types/string.md)) — В настоящее время не используется. -- `expression` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — В настоящее время не используется. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md deleted file mode 100644 index 4f4094b57d9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о выделении памяти, выполненном через аллокатор jemalloc по различным классам размеров (bins), агрегированную со всех арен.' -keywords: ['системная таблица', 'jemalloc_bins'] -slug: /operations/system-tables/jemalloc_bins -title: 'system.jemalloc_bins' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит сведения о выделениях памяти через аллокатор jemalloc в различных классах размеров (bins), агрегированные по всем аренам. -Эта статистика может быть не полностью точной из-за кэширования на уровне потоков в jemalloc. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Найдите размеры выделений памяти, которые вносят наибольший вклад в текущее суммарное использование памяти. - -```sql -SELECT - *, - allocations - deallocations AS active_allocations, - size * active_allocations AS allocated_bytes -FROM system.jemalloc_bins -WHERE allocated_bytes > 0 -ORDER BY allocated_bytes DESC -LIMIT 10 -``` - -```text -┌─index─┬─large─┬─────size─┬─allocactions─┬─deallocations─┬─active_allocations─┬─allocated_bytes─┐ -│ 82 │ 1 │ 50331648 │ 1 │ 0 │ 1 │ 50331648 │ -│ 10 │ 0 │ 192 │ 512336 │ 370710 │ 141626 │ 27192192 │ -│ 69 │ 1 │ 5242880 │ 6 │ 2 │ 4 │ 20971520 │ -│ 3 │ 0 │ 48 │ 16938224 │ 16559484 │ 378740 │ 18179520 │ -│ 28 │ 0 │ 4096 │ 122924 │ 119142 │ 3782 │ 15491072 │ -│ 61 │ 1 │ 1310720 │ 44569 │ 44558 │ 11 │ 14417920 │ -│ 39 │ 1 │ 28672 │ 1285 │ 913 │ 372 │ 10665984 │ -│ 4 │ 0 │ 64 │ 2837225 │ 2680568 │ 156657 │ 10026048 │ -│ 6 │ 0 │ 96 │ 2617803 │ 2531435 │ 86368 │ 8291328 │ -│ 36 │ 1 │ 16384 │ 22431 │ 21970 │ 461 │ 7553024 │ -└───────┴───────┴──────────┴──────────────┴───────────────┴────────────────────┴─────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md deleted file mode 100644 index ebab3b3bddc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о потребителях Kafka.' -keywords: ['системная таблица', 'kafka_consumers'] -slug: /operations/system-tables/kafka_consumers -title: 'system.kafka_consumers' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Содержит информацию о потребителях Kafka. -Применимо для [движка таблицы Kafka](../../engines/table-engines/integrations/kafka) (встроенная интеграция ClickHouse) - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Пример: - -```sql -SELECT * -FROM system.kafka_consumers -FORMAT Vertical -``` - -```text -Строка 1: -────── -database: test -table: kafka -consumer_id: ClickHouse-instance-test-kafka-1caddc7f-f917-4bb1-ac55-e28bd103a4a0 -assignments.topic: ['system_kafka_cons'] -assignments.partition_id: [0] -assignments.current_offset: [18446744073709550615] -exceptions.time: [] -exceptions.text: [] -last_poll_time: 2006-11-09 18:47:47 -num_messages_read: 4 -last_commit_time: 2006-11-10 04:39:40 -num_commits: 1 -last_rebalance_time: 1970-01-01 00:00:00 -num_rebalance_revocations: 0 -num_rebalance_assignments: 1 -is_currently_used: 1 -rdkafka_stat: {...} - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md deleted file mode 100644 index 68eb95ade71..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'Системная таблица, содержащая лицензии сторонних библиотек, которые находятся - в каталоге contrib в исходных кодах ClickHouse.' -keywords: ['системная таблица', 'лицензии'] -slug: /operations/system-tables/licenses -title: 'system.licenses' -doc_type: 'reference' ---- - -# system.licenses {#systemlicenses} - -Содержит лицензии сторонних библиотек, которые находятся в каталоге [contrib](https://github.com/ClickHouse/ClickHouse/tree/master/contrib) исходного кода ClickHouse. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT library_name, license_type, license_path FROM system.licenses LIMIT 15 -``` - -```text -┌─library_name───────┬─license_type─┬─license_path────────────────────────┐ -│ aws-c-common │ Apache │ /contrib/aws-c-common/LICENSE │ -│ base64 │ BSD 2-clause │ /contrib/aklomp-base64/LICENSE │ -│ brotli │ MIT │ /contrib/brotli/LICENSE │ -│ [...] │ [...] │ [...] │ -└────────────────────┴──────────────┴─────────────────────────────────────┘ - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md deleted file mode 100644 index bc44479414d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о настройках таблиц MergeTree.' -keywords: ['system table', 'merge_tree_settings'] -slug: /operations/system-tables/merge_tree_settings -title: 'system.merge_tree_settings' -doc_type: 'reference' ---- - -# system.merge_tree_settings {#systemmerge_tree_settings} - -Содержит сведения о настройках таблиц `MergeTree`. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.merge_tree_settings LIMIT 3 FORMAT Vertical; -``` - -```response -SELECT * -FROM system.merge_tree_settings -LIMIT 3 -FORMAT Vertical - -Query id: 2580779c-776e-465f-a90c-4b7630d0bb70 - -Row 1: -────── -name: min_compress_block_size -value: 0 -default: 0 -changed: 0 -description: При записи гранулы сжимать данные в буфере, если размер ожидающих несжатых данных больше или равен указанному порогу. Если эта настройка не задана, используется соответствующая глобальная настройка. -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 2: -────── -name: max_compress_block_size -value: 0 -default: 0 -changed: 0 -description: Сжимать ожидающие несжатые данные в буфере, если их размер больше или равен указанному порогу. Блок данных будет сжат, даже если текущая гранула не завершена. Если эта настройка не задана, используется соответствующая глобальная настройка. -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 3: -────── -name: index_granularity -value: 8192 -default: 8192 -changed: 0 -description: Количество строк, соответствующих одному значению первичного ключа. -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Получено 3 строки. Затрачено: 0,001 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md deleted file mode 100644 index 8b25e089b77..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о слияниях и мутациях партий, - выполняемых в данный момент для таблиц семейства MergeTree.' -keywords: ['системная таблица', 'слияния'] -slug: /operations/system-tables/merges -title: 'system.merges' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.merges {#systemmerges} - - - -Содержит информацию о слияниях и мутациях частей, которые в данный момент выполняются для таблиц семейства MergeTree. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md deleted file mode 100644 index ea85639fc78..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Системная таблица, содержащая историю значений метрик таблиц `system.metrics` - и `system.events`, которые периодически сбрасываются на диск.' -keywords: ['системная таблица', 'metric_log'] -slug: /operations/system-tables/metric_log -title: 'system.metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.metric_log {#systemmetric_log} - - - -Содержит историю значений метрик из таблиц `system.metrics` и `system.events`, которые периодически сбрасываются на диск. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, на котором выполняется запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время события. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — время события с точностью до микросекунд. - -**Пример** - -```sql -SELECT * FROM system.metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -milliseconds: 196 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -... -CurrentMetric_Revision: 54439 -CurrentMetric_VersionInteger: 20009001 -CurrentMetric_RWLockWaitingReaders: 0 -CurrentMetric_RWLockWaitingWriters: 0 -CurrentMetric_RWLockActiveReaders: 0 -CurrentMetric_RWLockActiveWriters: 0 -CurrentMetric_GlobalThread: 74 -CurrentMetric_GlobalThreadActive: 26 -CurrentMetric_LocalThread: 0 -CurrentMetric_LocalThreadActive: 0 -CurrentMetric_DistributedFilesToInsert: 0 -``` - -**Схема** -Эта таблица может быть настроена с разными типами схем с помощью XML-тега ``. Тип схемы по умолчанию — `wide`, при котором каждая метрика или событие профилирования хранятся в отдельном столбце. Такая схема является наиболее производительной и эффективной для операций чтения отдельных столбцов. - -Схема `transposed` хранит данные в формате, аналогичном `system.asynchronous_metric_log`, где метрики и события хранятся в строках. Эта схема полезна для конфигураций с ограниченными ресурсами, так как снижает потребление ресурсов во время слияний. - -Существует также схема совместимости `transposed_with_wide_view`, которая хранит фактические данные в таблице с транспонированной схемой (`system.transposed_metric_log`) и создает поверх нее представление с использованием широкой схемы. Это представление запрашивает транспонированную таблицу, что делает его полезным для миграции со схемы `wide` на схему `transposed`. - -**См. также** - -* [настройка metric_log](../../operations/server-configuration-parameters/settings.md#metric_log) — Включение и отключение настройки. -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — Содержит периодически вычисляемые метрики. -* [system.events](/operations/system-tables/events) — Содержит счетчики произошедших событий. -* [system.metrics](../../operations/system-tables/metrics.md) — Содержит моментально вычисляемые метрики. -* [Мониторинг](../../operations/monitoring.md) — Базовые концепции мониторинга ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md deleted file mode 100644 index d51002bb955..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md +++ /dev/null @@ -1,790 +0,0 @@ ---- -description: 'Системная таблица, содержащая метрики, которые могут быть вычислены - мгновенно или имеют текущее значение.' -keywords: ['системная таблица', 'метрики'] -slug: /operations/system-tables/metrics -title: 'system.metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.metrics {#systemmetrics} - - - -Содержит метрики, которые можно вычислить мгновенно или которые имеют текущее значение. Например, число одновременно обрабатываемых запросов или текущая задержка реплики. Эта таблица всегда содержит актуальные данные. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Все поддерживаемые метрики перечислены в файле исходного кода [src/Common/CurrentMetrics.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/CurrentMetrics.cpp). - -**Пример** - -```sql -SELECT * FROM system.metrics LIMIT 10 -``` - -```text -┌─metric───────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────┐ -│ Query │ 1 │ Количество выполняющихся запросов │ -│ Merge │ 0 │ Количество выполняющихся фоновых слияний │ -│ PartMutation │ 0 │ Количество мутаций (ALTER DELETE/UPDATE) │ -│ ReplicatedFetch │ 0 │ Количество частей данных, получаемых из реплик │ -│ ReplicatedSend │ 0 │ Количество частей данных, отправляемых в реплики │ -│ ReplicatedChecks │ 0 │ Количество частей данных, проверяемых на согласованность │ -│ BackgroundMergesAndMutationsPoolTask │ 0 │ Количество активных слияний и мутаций в связанном фоновом пуле │ -│ BackgroundFetchesPoolTask │ 0 │ Количество активных операций получения в связанном фоновом пуле │ -│ BackgroundCommonPoolTask │ 0 │ Количество активных задач в связанном фоновом пуле │ -│ BackgroundMovePoolTask │ 0 │ Количество активных задач в BackgroundProcessingPool для перемещений │ -└──────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────┘ -``` - - -## Описание метрик {#metric-descriptions} - -### AggregatorThreads {#aggregatorthreads} - -Количество потоков в пуле потоков Aggregator. - -### AggregatorThreadsActive {#aggregatorthreadsactive} - -Количество потоков в пуле потоков Aggregator, выполняющих задачи. - -### TablesLoaderForegroundThreads {#tablesloaderforegroundthreads} - -Количество потоков в пуле потоков переднего плана асинхронного загрузчика. - -### TablesLoaderForegroundThreadsActive {#tablesloaderforegroundthreadsactive} - -Количество потоков в пуле foreground-потоков асинхронного загрузчика, которые в данный момент выполняют задачи. - -### TablesLoaderBackgroundThreads {#tablesloaderbackgroundthreads} - -Количество потоков в пуле фоновых потоков асинхронного загрузчика. - -### TablesLoaderBackgroundThreadsActive {#tablesloaderbackgroundthreadsactive} - -Количество потоков в пуле фоновых потоков асинхронного загрузчика, выполняющих задания. - -### AsyncInsertCacheSize {#asyncinsertcachesize} - -Количество hash ID асинхронных вставок в кэше - -### AsynchronousInsertThreads {#asynchronousinsertthreads} - -Количество потоков в пуле AsynchronousInsert. - -### AsynchronousInsertThreadsActive {#asynchronousinsertthreadsactive} - -Количество потоков в пуле потоков AsynchronousInsert, выполняющих задания. - -### AsynchronousReadWait {#asynchronousreadwait} - -Количество потоков, ожидающих асинхронного чтения. - -### BackgroundBufferFlushSchedulePoolSize {#backgroundbufferflushschedulepoolsize} - -Ограничение количества задач в BackgroundBufferFlushSchedulePool - -### BackgroundBufferFlushSchedulePoolTask {#backgroundbufferflushschedulepooltask} - -Количество активных задач в BackgroundBufferFlushSchedulePool. Этот пул используется для периодического сброса буферов. - -### BackgroundCommonPoolSize {#backgroundcommonpoolsize} - -Ограничение количества задач в соответствующем фоновом пуле - -### BackgroundCommonPoolTask {#backgroundcommonpooltask} - -Количество активных задач в соответствующем фоновом пуле - -### BackgroundDistributedSchedulePoolSize {#backgrounddistributedschedulepoolsize} - -Ограничение на количество задач в пуле BackgroundDistributedSchedulePool - -### BackgroundDistributedSchedulePoolTask {#backgrounddistributedschedulepooltask} - -Количество активных задач в BackgroundDistributedSchedulePool. Этот пул используется для фоновой распределённой отправки данных. - -### BackgroundFetchesPoolSize {#backgroundfetchespoolsize} - -Лимит количества одновременных операций выборки в связанном фоновом пуле - -### BackgroundFetchesPoolTask {#backgroundfetchespooltask} - -Количество активных операций выборки в связанном фоновом пуле - -### BackgroundMergesAndMutationsPoolSize {#backgroundmergesandmutationspoolsize} - -Ограничение числа активных слияний и мутаций в соответствующем фоновом пуле - -### BackgroundMergesAndMutationsPoolTask {#backgroundmergesandmutationspooltask} - -Количество активных слияний и мутаций в связанном фоновом пуле - -### BackgroundMessageBrokerSchedulePoolSize {#backgroundmessagebrokerschedulepoolsize} - -Ограничение числа задач в BackgroundProcessingPool для стриминга сообщений - -### BackgroundMessageBrokerSchedulePoolTask {#backgroundmessagebrokerschedulepooltask} - -Количество активных задач в BackgroundProcessingPool для стриминга сообщений - -### BackgroundMovePoolSize {#backgroundmovepoolsize} - -Ограничение на число задач в BackgroundProcessingPool для операций перемещения - -### BackgroundMovePoolTask {#backgroundmovepooltask} - -Количество активных задач в BackgroundProcessingPool для операций MOVE - -### BackgroundSchedulePoolSize {#backgroundschedulepoolsize} - -Лимит количества задач в BackgroundSchedulePool. Этот пул используется для периодических задач ReplicatedMergeTree, таких как очистка старых частей данных, изменение частей данных, повторная инициализация реплики и т. д. - -### BackgroundSchedulePoolTask {#backgroundschedulepooltask} - -Количество активных задач в BackgroundSchedulePool. Этот пул используется для периодического выполнения задач ReplicatedMergeTree, таких как очистка старых частей данных, изменение частей данных, повторная инициализация реплик и т. д. - -### BackupsIOThreads {#backupsiothreads} - -Количество потоков в пуле потоков BackupsIO. - -### BackupsIOThreadsActive {#backupsiothreadsactive} - -Количество потоков в пуле потоков BackupsIO, выполняющих задачу. - -### BackupsThreads {#backupsthreads} - -Количество потоков в пуле потоков, используемом для выполнения BACKUP. - -### BackupsThreadsActive {#backupsthreadsactive} - -Количество потоков в пуле потоков операции BACKUP, выполняющих задачу. - -### BrokenDistributedFilesToInsert {#brokendistributedfilestoinsert} - -Количество файлов для асинхронной вставки в таблицы типа Distributed, которые были помечены как повреждённые. Эта метрика при запуске начинается с 0. Суммируется количество файлов по всем сегментам. - -### CacheDetachedFileSegments {#cachedetachedfilesegments} - -Количество существующих отсоединённых сегментов файлового кэша - -### CacheDictionaryThreads {#cachedictionarythreads} - -Число потоков в пуле потоков CacheDictionary. - -### CacheDictionaryThreadsActive {#cachedictionarythreadsactive} - -Количество потоков в пуле потоков CacheDictionary, выполняющих задачи. - -### CacheDictionaryUpdateQueueBatches {#cachedictionaryupdatequeuebatches} - -Количество «пакетов» (наборов ключей) в очереди обновления CacheDictionaries. - -### CacheDictionaryUpdateQueueKeys {#cachedictionaryupdatequeuekeys} - -Точное количество ключей в очереди обновления CacheDictionaries. - -### CacheFileSegments {#cachefilesegments} - -Количество существующих файловых сегментов кэша - -### ContextLockWait {#contextlockwait} - -Количество потоков, ожидающих получения блокировки в Context. Это глобальная блокировка. - -### DDLWorkerThreads {#ddlworkerthreads} - -Число потоков в пуле потоков DDLWorker, используемом для запросов ON CLUSTER. - -### DDLWorkerThreadsActive {#ddlworkerthreadsactive} - -Количество потоков в пуле потоков DDLWORKER для запросов ON CLUSTER, выполняющих задачи. - -### DatabaseCatalogThreads {#databasecatalogthreads} - -Количество потоков в пуле потоков DatabaseCatalog. - -### DatabaseCatalogThreadsActive {#databasecatalogthreadsactive} - -Количество потоков в пуле потоков DatabaseCatalog, выполняющих задачи. - -### DatabaseOnDiskThreads {#databaseondiskthreads} - -Количество потоков в пуле потоков DatabaseOnDisk. - -### DatabaseOnDiskThreadsActive {#databaseondiskthreadsactive} - -Количество потоков в пуле потоков DatabaseOnDisk, выполняющих задачи. - -### DelayedInserts {#delayedinserts} - -Количество запросов INSERT, скорость выполнения которых ограничивается из‑за большого числа активных частей данных партиции в таблице MergeTree. - -### DestroyAggregatesThreads {#destroyaggregatesthreads} - -Количество потоков в пуле потоков, используемом для уничтожения агрегатных состояний. - -### DestroyAggregatesThreadsActive {#destroyaggregatesthreadsactive} - -Количество потоков в пуле потоков для выполнения задач по уничтожению состояний агрегатов. - -### DictCacheRequests {#dictcacherequests} - -Количество одновременно выполняющихся запросов к источникам данных словарей кеширующего типа. - -### DiskObjectStorageAsyncThreads {#diskobjectstorageasyncthreads} - -Количество потоков в асинхронном пуле потоков для DiskObjectStorage. - -### DiskObjectStorageAsyncThreadsActive {#diskobjectstorageasyncthreadsactive} - -Количество потоков в асинхронном пуле потоков DiskObjectStorage, выполняющих задачи. - -### DiskSpaceReservedForMerge {#diskspacereservedformerge} - -Дисковое пространство, зарезервированное для выполняющихся в данный момент фоновых слияний. Оно немного больше суммарного размера частей, участвующих в слиянии. - -### DistributedFilesToInsert {#distributedfilestoinsert} - -Количество файлов, ожидающих обработки для асинхронной вставки в Distributed-таблицы. Суммируется количество файлов по всем сегментам. - -### DistributedSend {#distributedsend} - -Количество подключений к удалённым серверам для отправки данных, вставленных оператором INSERT в таблицы Distributed. Поддерживает синхронный и асинхронный режимы. - -### EphemeralNode {#ephemeralnode} - -Количество эфемерных узлов, хранящихся в ZooKeeper. - -### FilesystemCacheElements {#filesystemcacheelements} - -Элементы кэша файловой системы (сегменты файлов) - -### FilesystemCacheReadBuffers {#filesystemcachereadbuffers} - -Количество активных буферов файлового кэша - -### FilesystemCacheSize {#filesystemcachesize} - -Размер кеша файловой системы в байтах - -### QueryCacheBytes {#querycachebytes} - -Общий объём кэша запросов в байтах. - -### QueryCacheEntries {#querycacheentries} - -Общее количество записей в кэше запросов. - -### UncompressedCacheBytes {#uncompressedcachebytes} - -Общий размер несжатого кэша в байтах. Несжатый кэш обычно не улучшает производительность, и его, как правило, следует избегать. - -### UncompressedCacheCells {#uncompressedcachecells} - -### CompiledExpressionCacheBytes {#compiledexpressioncachebytes} - -Общий объём памяти в байтах, занимаемый кэшем JIT-сомпилированного кода. - -### CompiledExpressionCacheCount {#compiledexpressioncachecount} - -Общее количество записей в кэше JIT-компилированного кода. - -### MMapCacheCells {#mmapcachecells} - -Количество файлов, открытых с помощью `mmap` (отображённых в память). Используется для запросов с параметром `local_filesystem_read_method`, имеющим значение `mmap`. Файлы, открытые с помощью `mmap`, сохраняются в кэше, чтобы избежать дорогостоящих операций сброса TLB. - -### MarkCacheBytes {#markcachebytes} - -Общий размер кэша меток в байтах - -### MarkCacheFiles {#markcachefiles} - -Общее число файлов меток, находящихся в кэше меток - -### GlobalThread {#globalthread} - -Число потоков в глобальном пуле потоков. - -### GlobalThreadActive {#globalthreadactive} - -Количество потоков в глобальном пуле потоков, которые выполняют задачу. - -### HTTPConnection {#httpconnection} - -Количество подключений к HTTP-серверу - -### HashedDictionaryThreads {#hasheddictionarythreads} - -Количество потоков в пуле потоков HashedDictionary. - -### HashedDictionaryThreadsActive {#hasheddictionarythreadsactive} - -Количество потоков в пуле потоков HashedDictionary, выполняющих задачи. - -### IOPrefetchThreads {#ioprefetchthreads} - -Количество потоков в пуле предварительной выборки ввода-вывода. - -### IOPrefetchThreadsActive {#ioprefetchthreadsactive} - -Количество потоков в пуле потоков предварительной выборки операций ввода-вывода, выполняющих задачу. - -### IOThreads {#iothreads} - -Количество потоков в пуле потоков ввода/вывода. - -### IOThreadsActive {#iothreadsactive} - -Количество потоков в пуле потоков ввода-вывода, выполняющих задачи. - -### IOUringInFlightEvents {#iouringinflightevents} - -Количество активных SQE io_uring - -### IOUringPendingEvents {#iouringpendingevents} - -Количество SQE io_uring, ожидающих отправки - -### IOWriterThreads {#iowriterthreads} - -Количество потоков в пуле потоков записи ввода-вывода (IO writer). - -### IOWriterThreadsActive {#iowriterthreadsactive} - -Количество потоков в пуле потоков записи ввода-вывода, которые в данный момент выполняют задачу. - -### InterserverConnection {#interserverconnection} - -Количество подключений от других реплик для получения частей данных - -### KafkaAssignedPartitions {#kafkaassignedpartitions} - -Количество разделов, к которым в настоящий момент привязаны таблицы Kafka - -### KafkaBackgroundReads {#kafkabackgroundreads} - -Количество фоновых операций чтения, которые сейчас выполняются (для заполнения материализованных представлений из Kafka) - -### KafkaConsumers {#kafkaconsumers} - -Количество активных консьюмеров Kafka - -### KafkaConsumersInUse {#kafkaconsumersinuse} - -Количество консьюмеров, в настоящий момент задействованных для прямых или фоновых операций чтения - -### KafkaConsumersWithAssignment {#kafkaconsumerswithassignment} - -Количество активных Kafka-консьюмеров с назначенными партициями. - -### KafkaLibrdkafkaThreads {#kafkalibrdkafkathreads} - -Количество активных потоков librdkafka - -### KafkaProducers {#kafkaproducers} - -Количество активных продюсеров Kafka - -### KafkaWrites {#kafkawrites} - -Количество текущих выполняемых вставок в Kafka - -### KeeperAliveConnections {#keeperaliveconnections} - -Количество активных соединений - -### KeeperOutstandingRequests {#keeperoutstandingrequests} - -Количество необработанных запросов - -### LocalThread {#localthread} - -Количество потоков в локальных пулах потоков. Эти потоки выделяются из глобального пула потоков. - -### LocalThreadActive {#localthreadactive} - -Количество потоков в локальных пулах потоков, выполняющих задачи. - -### MMappedAllocBytes {#mmappedallocbytes} - -Сумма байт, выделенных через mmap - -### MMappedAllocs {#mmappedallocs} - -Общее число выделений памяти через mmap - -### MMappedFileBytes {#mmappedfilebytes} - -Суммарный размер участков файлов, отображённых в память (mmapped). - -### MMappedFiles {#mmappedfiles} - -Общее количество файлов, отображённых в память (mmap). - -### MarksLoaderThreads {#marksloaderthreads} - -Число потоков в пуле для загрузки меток. - -### MarksLoaderThreadsActive {#marksloaderthreadsactive} - -Количество потоков в пуле потоков загрузки меток, выполняющих задачу. - -### MaxDDLEntryID {#maxddlentryid} - -Идентификатор последней обработанной записи DDL для DDLWorker. - -### MaxPushedDDLEntryID {#maxpushedddlentryid} - -Максимальный идентификатор записи DDL, отправленной `DDLWorker` в ZooKeeper. - -### MemoryTracking {#memorytracking} - -Общий объём памяти (в байтах), выделенной сервером. - -### Слияние {#merge} - -Количество выполняющихся фоновых слияний - -### MergeTreeAllRangesAnnouncementsSent {#mergetreeallrangesannouncementssent} - -Текущее количество уведомлений, находящихся в процессе отправки с удалённого сервера на сервер-инициатор о наборе частей данных (для таблиц MergeTree). Измеряется на стороне удалённого сервера. - -### MergeTreeBackgroundExecutorThreads {#mergetreebackgroundexecutorthreads} - -Количество потоков в пуле потоков MergeTreeBackgroundExecutor. - -### MergeTreeBackgroundExecutorThreadsActive {#mergetreebackgroundexecutorthreadsactive} - -Количество потоков в пуле потоков MergeTreeBackgroundExecutor, выполняющих задания. - -### MergeTreeDataSelectExecutorThreads {#mergetreedataselectexecutorthreads} - -Количество потоков в пуле потоков MergeTreeDataSelectExecutor. - -### MergeTreeDataSelectExecutorThreadsActive {#mergetreedataselectexecutorthreadsactive} - -Количество потоков в пуле потоков MergeTreeDataSelectExecutor, выполняющих задания. - -### MergeTreePartsCleanerThreads {#mergetreepartscleanerthreads} - -Количество потоков в пуле потоков очистки частей MergeTree. - -### MergeTreePartsCleanerThreadsActive {#mergetreepartscleanerthreadsactive} - -Количество потоков в пуле потоков очистки частей MergeTree, выполняющих задачу. - -### MergeTreePartsLoaderThreads {#mergetreepartsloaderthreads} - -Количество потоков в пуле потоков загрузчика частей таблиц MergeTree. - -### MergeTreePartsLoaderThreadsActive {#mergetreepartsloaderthreadsactive} - -Количество потоков в пуле потоков загрузчика частей MergeTree, выполняющих задачи. - -### MergeTreeReadTaskRequestsSent {#mergetreereadtaskrequestssent} - -Текущее количество активных callback-запросов, выполняемых с удалённого сервера на сервер-инициатор для выбора задачи чтения (для таблиц MergeTree). Измеряется на стороне удалённого сервера. - -### Перемещения {#move} - -Количество операций перемещения, выполняющихся в данный момент - -### MySQLConnection {#mysqlconnection} - -Количество клиентских подключений по протоколу MySQL - -### NetworkReceive {#networkreceive} - -Количество потоков, принимающих данные по сети. Учитывается только сетевое взаимодействие, связанное с ClickHouse, а не со сторонними библиотеками. - -### NetworkSend {#networksend} - -Количество потоков, отправляющих данные в сеть. При этом учитывается только сетевое взаимодействие, связанное с ClickHouse, без участия сторонних библиотек. - -### OpenFileForRead {#openfileforread} - -Число файлов, открытых для чтения - -### OpenFileForWrite {#openfileforwrite} - -Количество файлов, открытых для записи - -### ParallelFormattingOutputFormatThreads {#parallelformattingoutputformatthreads} - -Количество потоков в пуле ParallelFormattingOutputFormatThreads. - -### ParallelFormattingOutputFormatThreadsActive {#parallelformattingoutputformatthreadsactive} - -Количество потоков в пуле потоков ParallelFormattingOutputFormatThreads, которые в данный момент выполняют задачу. - -### PartMutation {#partmutation} - -Количество мутаций (ALTER DELETE/UPDATE) - -### PartsActive {#partsactive} - -Активная часть данных, используемая текущими и последующими запросами SELECT. - -### PartsCommitted {#partscommitted} - -Устарело. См. PartsActive. - -### PartsCompact {#partscompact} - -Компактные части. - -### PartsDeleteOnDestroy {#partsdeleteondestroy} - -Часть была перемещена на другой диск и должна быть удалена в своём деструкторе. - -### PartsDeleting {#partsdeleting} - -Неактивная часть данных с собственным счётчиком ссылок (refcounter), в данный момент удаляется процессом очистки. - -### PartsOutdated {#partsoutdated} - -Неактивная часть данных; может использоваться только выполняющимися в данный момент запросами SELECT и может быть удалена после их завершения. - -### PartsPreActive {#partspreactive} - -Часть находится в data_parts, но не используется в запросах SELECT. - -### PartsPreCommitted {#partsprecommitted} - -Устарело. См. PartsPreActive. - -### PartsTemporary {#partstemporary} - -Часть находится в процессе создания, её нет в списке data_parts. - -### PartsWide {#partswide} - -Широкие части. - -### PendingAsyncInsert {#pendingasyncinsert} - -Количество асинхронных вставок, ожидающих сброса. - -### PostgreSQLConnection {#postgresqlconnection} - -Количество клиентских подключений по протоколу PostgreSQL - -### Query {#query} - -Количество выполняющихся запросов - -### QueryPreempted {#querypreempted} - -Количество запросов, которые приостановлены и ожидают выполнения из-за параметра `priority`. - -### QueryThread {#querythread} - -Количество потоков обработки запросов - -### RWLockActiveReaders {#rwlockactivereaders} - -Количество потоков, удерживающих блокировку чтения RWLock таблицы. - -### RWLockActiveWriters {#rwlockactivewriters} - -Количество потоков, удерживающих блокировку на запись (write lock) в RWLock таблицы. - -### RWLockWaitingReaders {#rwlockwaitingreaders} - -Количество потоков, ожидающих получения блокировки на чтение (RWLock) для таблицы. - -### RWLockWaitingWriters {#rwlockwaitingwriters} - -Количество потоков, ожидающих возможности записи на RWLock таблицы. - -### Read {#read} - -Количество системных вызовов чтения (read, pread, io_getevents и т. д.), находящихся в полёте - -### ReadTaskRequestsSent {#readtaskrequestssent} - -Текущее количество активных callback‑запросов от удалённого сервера к серверу‑инициатору для выбора задачи чтения (для табличной функции s3Cluster и аналогичных). Измеряется на стороне удалённого сервера. - -### ReadonlyReplica {#readonlyreplica} - -Количество реплицируемых таблиц, которые в данный момент находятся в режиме `readonly` из‑за повторной инициализации после потери сессии ZooKeeper или из‑за запуска без настроенного ZooKeeper. - -### RemoteRead {#remoteread} - -Количество выполняющихся в данный момент операций чтения с удалённым читателем - -### ReplicatedChecks {#replicatedchecks} - -Количество кусков данных, проверяемых на согласованность - -### ReplicatedFetch {#replicatedfetch} - -Количество частей данных, получаемых с реплики - -### ReplicatedSend {#replicatedsend} - -Количество частей данных, отправляемых репликам - -### RestartReplicaThreads {#restartreplicathreads} - -Количество потоков в пуле потоков для RESTART REPLICA. - -### RestartReplicaThreadsActive {#restartreplicathreadsactive} - -Количество потоков в пуле потоков RESTART REPLICA, в данный момент выполняющих задачу. - -### RestoreThreads {#restorethreads} - -Число потоков в пуле потоков, используемом для RESTORE. - -### RestoreThreadsActive {#restorethreadsactive} - -Количество потоков в пуле потоков, используемых RESTORE для выполнения задачи. - -### Ревизия {#revision} - -Ревизия сервера. Это число, которое увеличивается с каждым релизом или релиз-кандидатом, за исключением патч-релизов. - -### S3Requests {#s3requests} - -S3‑запросы - -### SendExternalTables {#sendexternaltables} - -Число подключений, которые отправляют данные для внешних таблиц на удалённые серверы. Внешние таблицы используются для реализации операторов GLOBAL IN и GLOBAL JOIN в распределённых подзапросах. - -### SendScalars {#sendscalars} - -Количество соединений, которые отправляют данные скаляров на удалённые серверы. - -### StorageBufferBytes {#storagebufferbytes} - -Количество байт в буферах таблиц Buffer - -### StorageBufferRows {#storagebufferrows} - -Количество строк в буферах таблиц типа Buffer - -### StorageDistributedThreads {#storagedistributedthreads} - -Количество потоков в пуле потоков хранилища StorageDistributed. - -### StorageDistributedThreadsActive {#storagedistributedthreadsactive} - -Количество потоков в пуле потоков StorageDistributed, которые в данный момент выполняют задачи. - -### StorageHiveThreads {#storagehivethreads} - -Количество потоков в пуле потоков StorageHive. - -### StorageHiveThreadsActive {#storagehivethreadsactive} - -Количество потоков в пуле потоков StorageHive, выполняющих задачи. - -### StorageS3Threads {#storages3threads} - -Число потоков в пуле потоков StorageS3. - -### StorageS3ThreadsActive {#storages3threadsactive} - -Количество потоков в пуле потоков StorageS3, которые выполняют задачи. - -### SystemReplicasThreads {#systemreplicasthreads} - -Количество потоков в пуле потоков `system.replicas`. - -### SystemReplicasThreadsActive {#systemreplicasthreadsactive} - -Количество потоков в пуле потоков system.replicas, в настоящее время выполняющих задачу. - -### TCPConnection {#tcpconnection} - -Количество подключений к TCP-серверу (включая клиентов с нативным интерфейсом), а также соединений «сервер–сервер» для распределённых запросов. - -### TablesToDropQueueSize {#tablestodropqueuesize} - -Количество удалённых таблиц, которые ожидают фонового удаления данных. - -### TemporaryFilesForAggregation {#temporaryfilesforaggregation} - -Количество временных файлов, создаваемых при внешней агрегации - -### TemporaryFilesForJoin {#temporaryfilesforjoin} - -Количество временных файлов, созданных для операций JOIN - -### TemporaryFilesForSort {#temporaryfilesforsort} - -Количество временных файлов, созданных для внешней сортировки - -### TemporaryFilesUnknown {#temporaryfilesunknown} - -Количество временных файлов, назначение которых неизвестно - -### ThreadPoolFSReaderThreads {#threadpoolfsreaderthreads} - -Количество потоков в пуле для local_filesystem_read_method=threadpool. - -### ThreadPoolFSReaderThreadsActive {#threadpoolfsreaderthreadsactive} - -Количество потоков в пуле потоков, выполняющих задачу, при local_filesystem_read_method=threadpool. - -### ThreadPoolRemoteFSReaderThreads {#threadpoolremotefsreaderthreads} - -Число потоков в пуле потоков для remote_filesystem_read_method=threadpool. - -### ThreadPoolRemoteFSReaderThreadsActive {#threadpoolremotefsreaderthreadsactive} - -Количество потоков в пуле потоков для remote_filesystem_read_method=threadpool, которые в данный момент выполняют задачу. - -### ThreadsInOvercommitTracker {#threadsinovercommittracker} - -Количество ожидающих потоков в OvercommitTracker - -### TotalTemporaryFiles {#totaltemporaryfiles} - -Число созданных временных файлов - -### VersionInteger {#versioninteger} - -Версия сервера в виде одного целого числа в системе счисления с основанием 1000. Например, версия 11.22.33 преобразуется в 11022033. - -### Write {#write} - -Количество выполняющихся системных вызовов записи (write, pwrite, io_getevents и т. д.) - -### ZooKeeperRequest {#zookeeperrequest} - -Количество выполняющихся запросов к ZooKeeper. - -### ZooKeeperSession {#zookeepersession} - -Количество сессий (соединений) с ZooKeeper не должно превышать одной, поскольку использование более чем одного соединения с ZooKeeper может приводить к ошибкам из‑за отсутствия линеаризуемости (устаревших чтений), допускаемого моделью согласованности ZooKeeper. - -### ZooKeeperWatch {#zookeeperwatch} - -Количество наблюдений (watch, подписок на события) в ZooKeeper. - -### ConcurrencyControlAcquired {#concurrencycontrolacquired} - -Общее количество выделенных слотов CPU. - -### ConcurrencyControlSoftLimit {#concurrencycontrolsoftlimit} - -Значение мягкого ограничения на количество CPU-слотов. - -**См. также** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — Содержит периодически вычисляемые метрики. -- [system.events](/operations/system-tables/events) — Содержит количество произошедших событий. -- [system.metric_log](/operations/system-tables/metric_log) — Содержит историю значений метрик из таблиц `system.metrics` и `system.events`. -- [Monitoring](../../operations/monitoring.md) — Базовые концепции мониторинга ClickHouse. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md deleted file mode 100644 index faf0646ebca..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о текущих перемещениях частей данных таблиц MergeTree. Каждое перемещение части данных представлено одной строкой.' -keywords: ['системная таблица', 'перемещения'] -slug: /operations/system-tables/moves -title: 'system.moves' -doc_type: 'reference' ---- - -# system.moves {#systemmoves} - -Таблица содержит информацию о выполняющихся [перемещениях частей данных](/sql-reference/statements/alter/partition#move-partitionpart) таблиц [MergeTree](/engines/table-engines/mergetree-family/mergetree.md). Каждое перемещение части данных представлено одной строкой. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.moves -``` - -```response -┌─database─┬─table─┬─────elapsed─┬─target_disk_name─┬─target_disk_path─┬─part_name─┬─part_size─┬─thread_id─┐ -│ default │ test2 │ 1.668056039 │ s3 │ ./disks/s3/ │ all_3_3_0 │ 136 │ 296146 │ -└──────────┴───────┴─────────────┴──────────────────┴──────────────────┴───────────┴───────────┴───────────┘ -``` - -**См. также** - -* Движок таблицы [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) -* [Использование нескольких блочных устройств для хранения данных](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-multiple-volumes) -* Команда [ALTER TABLE ... MOVE PART](/sql-reference/statements/alter/partition#move-partitionpart) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md deleted file mode 100644 index 069caaa656b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о мутациях таблиц MergeTree - и ходе их выполнения. Каждая команда мутации представлена одной строкой.' -keywords: ['system table', 'mutations'] -slug: /operations/system-tables/mutations -title: 'system.mutations' -doc_type: 'reference' ---- - -# system.mutations {#systemmutations} - -Эта таблица содержит информацию о [мутациях](/sql-reference/statements/alter/index.md#mutations) таблиц семейства [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) и прогрессе их выполнения. Каждая операция мутации представлена одной строкой. - -## Столбцы {#columns} - -- `database` ([String](/sql-reference/data-types/string.md)) — Имя базы данных, к которой применена мутация. -- `table` ([String](/sql-reference/data-types/string.md)) — Имя таблицы, к которой применена мутация. -- `mutation_id` ([String](/sql-reference/data-types/string.md)) — Идентификатор мутации. Для реплицируемых таблиц эти идентификаторы соответствуют именам znode в каталоге `/mutations/` в ClickHouse Keeper. Для нереплицируемых таблиц идентификаторы соответствуют именам файлов в каталоге данных таблицы. -- `command` ([String](/sql-reference/data-types/string.md)) — Строка команды мутации (часть запроса после `ALTER TABLE [db.]table`). -- `create_time` ([DateTime](/sql-reference/data-types/datetime.md)) — Дата и время отправки команды мутации на выполнение. -- `block_numbers.partition_id` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — Для мутаций реплицируемых таблиц массив содержит идентификаторы партиций (одна запись для каждой партиции). Для мутаций нереплицируемых таблиц массив пуст. -- `block_numbers.number` ([Array](/sql-reference/data-types/array.md)([Int64](/sql-reference/data-types/int-uint.md))) — Для мутаций реплицируемых таблиц массив содержит одну запись для каждой партиции с номером блока, полученным мутацией. В партиции будут мутированы только куски, содержащие блоки с номерами меньше этого номера. В нереплицируемых таблицах номера блоков во всех партициях образуют единую последовательность. Это означает, что для мутаций нереплицируемых таблиц столбец будет содержать одну запись с единственным номером блока, полученным мутацией. -- `parts_to_do_names` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — Массив имён кусков данных, которые необходимо мутировать для завершения мутации. -- `parts_to_do` ([Int64](/sql-reference/data-types/int-uint.md)) — Количество кусков данных, которые необходимо мутировать для завершения мутации. -- `is_killed` ([UInt8](/sql-reference/data-types/int-uint.md)) — Указывает, была ли мутация прервана. **Доступно только в ClickHouse Cloud.** - -:::note -`is_killed=1` не обязательно означает, что мутация полностью завершена. Мутация может оставаться в состоянии, где `is_killed=1` и `is_done=0` в течение длительного времени. Это может произойти, если другая долго выполняющаяся мутация блокирует прерванную мутацию. Это нормальная ситуация. -::: - -- `is_done` ([UInt8](/sql-reference/data-types/int-uint.md)) — Флаг, указывающий, завершена ли мутация. Возможные значения: - - `1` — мутация завершена, - - `0` — мутация всё ещё выполняется. - -:::note -Даже если `parts_to_do = 0`, возможно, что мутация реплицируемой таблицы ещё не завершена из-за долго выполняющегося запроса `INSERT`, который создаст новый кусок данных, требующий мутации. -::: - -Если возникли проблемы с мутацией некоторых кусков данных, следующие столбцы содержат дополнительную информацию: - -- `latest_failed_part` ([String](/sql-reference/data-types/string.md)) — Имя последнего куска, который не удалось мутировать. -- `latest_fail_time` ([DateTime](/sql-reference/data-types/datetime.md)) — Дата и время последнего сбоя мутации куска. -- `latest_fail_reason` ([String](/sql-reference/data-types/string.md)) — Сообщение об исключении, вызвавшем последний сбой мутации куска. - -## Мониторинг мутаций {#monitoring-mutations} - -Для отслеживания прогресса мутаций в таблице `system.mutations` используйте следующий запрос: - -```sql -SELECT * FROM clusterAllReplicas('cluster_name', 'system', 'mutations') -WHERE is_done = 0 AND table = 'tmp'; - --- или - -SELECT * FROM clusterAllReplicas('cluster_name', 'system.mutations') -WHERE is_done = 0 AND table = 'tmp'; -``` - -Примечание: для этого требуются права на чтение таблиц `system.*`. - -:::tip Использование в Cloud -В ClickHouse Cloud таблица `system.mutations` на каждом узле содержит все мутации кластера, поэтому использование `clusterAllReplicas` не требуется. -::: - -**См. также** - -- [Мутации](/sql-reference/statements/alter/index.md#mutations) -- Движок таблиц [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) -- Семейство [ReplicatedMergeTree](/engines/table-engines/mergetree-family/replication.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md deleted file mode 100644 index 4ff857362f3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: 'Системная таблица с одним столбцом типа UInt64 с именем `number`, в котором содержатся почти все натуральные числа, начиная с нуля.' -keywords: ['системная таблица', 'числа'] -slug: /operations/system-tables/numbers -title: 'system.numbers' -doc_type: 'reference' ---- - -# system.numbers {#systemnumbers} - -Эта таблица содержит единственный столбец типа UInt64 с именем `number`, в котором находятся почти все натуральные числа, начиная с нуля. - -Вы можете использовать эту таблицу для тестов или если вам нужно выполнить поиск методом полного перебора. - -Чтение из этой таблицы не выполняется параллельно. - -**Пример** - -```sql -SELECT * FROM system.numbers LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 строк в наборе. Затрачено: 0.001 сек. -``` - -Вы также можете ограничить вывод по предикатам. - -```sql -SELECT * FROM system.numbers < 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 строк в наборе. Затрачено: 0.001 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md deleted file mode 100644 index 9498eb3d548..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: 'Системная таблица, аналогичная `system.numbers`, но чтение из неё выполняется параллельно, - а числа могут возвращаться в произвольном порядке.' -keywords: ['system table', 'numbers_mt'] -slug: /operations/system-tables/numbers_mt -title: 'system.numbers_mt' -doc_type: 'reference' ---- - -То же, что и [`system.numbers`](../../operations/system-tables/numbers.md), но чтение из неё выполняется параллельно. Числа могут возвращаться в произвольном порядке. - -Используется для тестирования. - -**Пример** - -```sql -SELECT * FROM system.numbers_mt LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -10 строк в наборе. Затрачено: 0.001 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/one.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/one.md deleted file mode 100644 index d610ab4afb1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/one.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'Системная таблица, содержащая одну строку с одним столбцом `dummy` - типа UInt8, содержащим значение 0. Аналог таблицы `DUAL` в других СУБД.' -keywords: ['системная таблица', 'one'] -slug: /operations/system-tables/one -title: 'system.one' -doc_type: 'reference' ---- - -# system.one {#systemone} - -Эта таблица содержит одну строку с единственным столбцом `dummy` типа UInt8 со значением 0. - -Эта таблица используется, если в запросе `SELECT` не указано предложение `FROM`. - -Она аналогична таблице `DUAL`, используемой в других СУБД. - -**Пример** - -```sql -SELECT * FROM system.one LIMIT 10; -``` - -```response -┌─dummy─┐ -│ 0 │ -└───────┘ - -Получена 1 строка. Время выполнения: 0.001 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md deleted file mode 100644 index c9c4fb2af49..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о спанах трассировки для выполненных запросов.' -keywords: ['system table', 'opentelemetry_span_log'] -slug: /operations/system-tables/opentelemetry_span_log -title: 'system.opentelemetry_span_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.opentelemetry_span_log {#systemopentelemetry_span_log} - - - -Содержит информацию о [спанах трассировки](https://opentracing.io/docs/overview/spans/) для выполненных запросов. - -Столбцы: - -* `trace_id` ([UUID](../../sql-reference/data-types/uuid.md)) — идентификатор трассы для выполненного запроса. -* `span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — идентификатор `trace span`. -* `parent_span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — идентификатор родительского `trace span`. -* `operation_name` ([String](../../sql-reference/data-types/string.md)) — имя операции. -* `kind` ([Enum8](../../sql-reference/data-types/enum.md)) — [SpanKind](https://opentelemetry.io/docs/reference/specification/trace/api/#spankind) спана. - * `INTERNAL` — указывает, что спан представляет внутреннюю операцию в приложении. - * `SERVER` — указывает, что спан охватывает обработку на стороне сервера синхронного RPC или другого удалённого запроса. - * `CLIENT` — указывает, что спан описывает запрос к некоторому удалённому сервису. - * `PRODUCER` — указывает, что спан описывает инициаторов асинхронного запроса. Этот родительский спан часто заканчивается раньше соответствующего дочернего спана CONSUMER, возможно, даже до начала дочернего спана. - * `CONSUMER` — указывает, что спан описывает дочерний спан асинхронного запроса PRODUCER. -* `start_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — время начала `trace span` (в микросекундах). -* `finish_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — время окончания `trace span` (в микросекундах). -* `finish_date` ([Date](../../sql-reference/data-types/date.md)) — дата окончания `trace span`. -* `attribute.names` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — имена [атрибутов](https://opentelemetry.io/docs/go/instrumentation/#attributes) для данного `trace span`. Они заполняются в соответствии с рекомендациями стандарта [OpenTelemetry](https://opentelemetry.io/). -* `attribute.values` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — значения атрибутов для данного `trace span`. Они заполняются в соответствии с рекомендациями стандарта OpenTelemetry. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.opentelemetry_span_log LIMIT 1 FORMAT Vertical; -``` - -Результат: - -```text -Строка 1: -────── -trace_id: cdab0847-0d62-61d5-4d38-dd65b19a1914 -span_id: 701487461015578150 -parent_span_id: 2991972114672045096 -operation_name: DB::Block DB::InterpreterSelectQuery::getSampleBlockImpl() -kind: INTERNAL -start_time_us: 1612374594529090 -finish_time_us: 1612374594529108 -finish_date: 2021-02-03 -attribute.names: [] -attribute.values: [] -``` - -**См. также** - -* [OpenTelemetry](../../operations/opentelemetry.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md deleted file mode 100644 index f4856238126..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md +++ /dev/null @@ -1,263 +0,0 @@ ---- -description: 'Обзор того, что такое системные таблицы и зачем они нужны.' -keywords: ['system tables', 'overview'] -sidebar_label: 'Обзор' -sidebar_position: 52 -slug: /operations/system-tables/overview -title: 'Обзор системных таблиц' -doc_type: 'reference' ---- - - - -## Обзор системных таблиц {#system-tables-introduction} - -Системные таблицы предоставляют информацию о: - -* Состоянии сервера, процессах и окружении. -* Внутренних процессах сервера. -* Опциях, использованных при сборке бинарного файла ClickHouse. - -Системные таблицы: - -* Находятся в базе данных `system`. -* Доступны только для чтения данных. -* Не могут быть удалены или изменены, но могут быть отсоединены. - -Большинство системных таблиц хранят свои данные в оперативной памяти. Сервер ClickHouse создает такие системные таблицы при запуске. - -В отличие от других системных таблиц, системные журнальные таблицы [metric_log](../../operations/system-tables/metric_log.md), [query_log](../../operations/system-tables/query_log.md), [query_thread_log](../../operations/system-tables/query_thread_log.md), [trace_log](../../operations/system-tables/trace_log.md), [part_log](../../operations/system-tables/part_log.md), [crash_log](../../operations/system-tables/crash_log.md), [text_log](../../operations/system-tables/text_log.md) и [backup_log](../../operations/system-tables/backup_log.md) обслуживаются движком таблицы [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) и по умолчанию хранят свои данные в файловой системе. Если вы удалите таблицу из файловой системы, сервер ClickHouse создаст пустую таблицу заново при следующей записи данных. Если схема системной таблицы изменилась в новом релизе, ClickHouse переименовывает текущую таблицу и создает новую. - -Таблицы системных журналов можно настраивать, создавая конфигурационный файл с тем же именем, что и таблица, в каталоге `/etc/clickhouse-server/config.d/`, или задавая соответствующие элементы в `/etc/clickhouse-server/config.xml`. Настраиваемые элементы: - -* `database`: база данных, к которой относится системная журнальная таблица. Этот параметр сейчас устарел. Все системные журнальные таблицы находятся в базе данных `system`. -* `table`: таблица для записи данных. -* `partition_by`: выражение [PARTITION BY](../../engines/table-engines/mergetree-family/custom-partitioning-key.md). -* `ttl`: выражение [TTL](../../sql-reference/statements/alter/ttl.md) таблицы. -* `flush_interval_milliseconds`: интервал сброса данных на диск. -* `engine`: полное выражение движка (начиная с `ENGINE =` ) с параметрами. Этот параметр конфликтует с `partition_by` и `ttl`. При одновременной установке сервер сгенерирует исключение и завершит работу. - -Пример: - -```xml - - - system - query_log
- toYYYYMM(event_date) - event_date + INTERVAL 30 DAY DELETE - - 7500 - 1048576 - 8192 - 524288 - false -
-
-``` - -По умолчанию рост таблицы не ограничен. Чтобы контролировать размер таблицы, вы можете использовать настройки [TTL](/sql-reference/statements/alter/ttl) для удаления устаревших записей логов. Также вы можете использовать механизм партиционирования таблиц движка `MergeTree`. - - -## Источники системных метрик {#system-tables-sources-of-system-metrics} - -Для сбора системных метрик сервер ClickHouse использует: - -- привилегию `CAP_NET_ADMIN`. -- [procfs](https://en.wikipedia.org/wiki/Procfs) (только в Linux). - -**procfs** - -Если сервер ClickHouse не имеет привилегии `CAP_NET_ADMIN`, он пытается использовать `ProcfsMetricsProvider`. `ProcfsMetricsProvider` позволяет собирать системные метрики на уровне отдельных запросов (по CPU и I/O). - -Если procfs поддерживается и включён в системе, сервер ClickHouse собирает следующие метрики: - -- `OSCPUVirtualTimeMicroseconds` -- `OSCPUWaitMicroseconds` -- `OSIOWaitMicroseconds` -- `OSReadChars` -- `OSWriteChars` -- `OSReadBytes` -- `OSWriteBytes` - -:::note -`OSIOWaitMicroseconds` по умолчанию отключена в ядрах Linux, начиная с версии 5.14.x. -Вы можете включить её с помощью `sudo sysctl kernel.task_delayacct=1` или создав файл `.conf` в `/etc/sysctl.d/` с содержимым `kernel.task_delayacct = 1` -::: - - - -## Системные таблицы в ClickHouse Cloud {#system-tables-in-clickhouse-cloud} - -В ClickHouse Cloud системные таблицы предоставляют критически важную информацию о состоянии и производительности сервиса, так же, как и в развертываниях с самостоятельным управлением. Некоторые системные таблицы работают на уровне всего кластера, особенно те, которые получают данные с узлов Keeper, управляющих распределёнными метаданными. Эти таблицы отражают совокупное состояние кластера и должны возвращать согласованные данные при выполнении запросов с отдельных узлов. Например, таблица [`parts`](/operations/system-tables/parts) должна возвращать согласованные данные независимо от узла, с которого выполняется запрос: - -```sql -SELECT hostname(), count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-vccsrty-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.005 sec. - -SELECT - hostname(), - count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-w59bfco-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.004 sec. -``` - -Напротив, другие системные таблицы являются специфичными для узла, например, хранятся в памяти или сохраняют свои данные с использованием движка таблиц MergeTree. Это характерно для данных, таких как журналы (логи) и метрики. Такое постоянное хранение обеспечивает доступность исторических данных для анализа. Однако эти специфичные для узла таблицы по своей сути уникальны для каждого отдельного узла. - -В общем случае при определении того, является ли системная таблица специфичной для узла, можно руководствоваться следующими правилами: - -* Системные таблицы с суффиксом `_log`. -* Системные таблицы, предоставляющие метрики, например `metrics`, `asynchronous_metrics`, `events`. -* Системные таблицы, отражающие выполняющиеся процессы, например `processes`, `merges`. - -Кроме того, новые версии системных таблиц могут создаваться в результате обновлений или изменений их схемы. Эти версии именуются с использованием числового суффикса. - -Например, рассмотрим таблицы `system.query_log`, которые содержат строку для каждого запроса, выполненного на узле: - -```sql -SHOW TABLES FROM system LIKE 'query_log%' - -┌─name─────────┐ -│ query_log │ -│ query_log_1 │ -│ query_log_10 │ -│ query_log_2 │ -│ query_log_3 │ -│ query_log_4 │ -│ query_log_5 │ -│ query_log_6 │ -│ query_log_7 │ -│ query_log_8 │ -│ query_log_9 │ -└──────────────┘ - -11 строк в наборе. Затрачено: 0.004 сек. -``` - -### Выполнение запросов к нескольким версиям {#querying-multiple-versions} - -Мы можем выполнять запросы к этим таблицам, используя функцию [`merge`](/sql-reference/table-functions/merge). Например, запрос ниже определяет последний запрос, отправленный на целевой узел, в каждой таблице `query_log`: - -```sql -SELECT - _table, - max(event_time) AS most_recent -FROM merge('system', '^query_log') -GROUP BY _table -ORDER BY most_recent DESC - -┌─_table───────┬─────────most_recent─┐ -│ query_log │ 2025-04-13 10:59:29 │ -│ query_log_1 │ 2025-04-09 12:34:46 │ -│ query_log_2 │ 2025-04-09 12:33:45 │ -│ query_log_3 │ 2025-04-07 17:10:34 │ -│ query_log_5 │ 2025-03-24 09:39:39 │ -│ query_log_4 │ 2025-03-24 09:38:58 │ -│ query_log_6 │ 2025-03-19 16:07:41 │ -│ query_log_7 │ 2025-03-18 17:01:07 │ -│ query_log_8 │ 2025-03-18 14:36:07 │ -│ query_log_10 │ 2025-03-18 14:01:33 │ -│ query_log_9 │ 2025-03-18 14:01:32 │ -└──────────────┴─────────────────────┘ -``` - - -11 строк в наборе. Прошло: 0.373 сек. Обработано 6.44 миллиона строк, 25.77 МБ (17.29 миллиона строк/с, 69.17 МБ/с). -Пиковое потребление памяти: 28.45 МиБ. - -```` - -:::note Не полагайтесь на числовой суффикс для упорядочивания -Хотя числовой суффикс в таблицах может указывать на порядок данных, на него никогда не следует полагаться. По этой причине всегда используйте табличную функцию merge в сочетании с фильтром по дате при работе с конкретными диапазонами дат. -::: - -Важно отметить, что эти таблицы по-прежнему являются **локальными для каждого узла**. - -### Запросы к нескольким узлам {#querying-across-nodes} - -Для комплексного просмотра всего кластера пользователи могут использовать функцию [`clusterAllReplicas`](/sql-reference/table-functions/cluster) в сочетании с функцией `merge`. Функция `clusterAllReplicas` позволяет выполнять запросы к системным таблицам на всех репликах в кластере «default», объединяя данные отдельных узлов в единый результат. В сочетании с функцией `merge` это можно использовать для получения всех системных данных для конкретной таблицы в кластере. - -Этот подход особенно ценен для мониторинга и отладки операций на уровне кластера, позволяя пользователям эффективно анализировать работоспособность и производительность развертывания ClickHouse Cloud. - -:::note -ClickHouse Cloud предоставляет кластеры с несколькими репликами для обеспечения избыточности и отказоустойчивости. Это обеспечивает такие возможности, как динамическое автомасштабирование и обновления без простоя. В определенный момент времени новые узлы могут находиться в процессе добавления в кластер или удаления из кластера. Чтобы пропустить эти узлы, добавьте `SETTINGS skip_unavailable_shards = 1` к запросам, использующим `clusterAllReplicas`, как показано ниже. -::: - -Например, рассмотрим разницу при запросе к таблице `query_log` — часто необходимой для анализа. - -```sql -SELECT - hostname() AS host, - count() -FROM system.query_log -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.010 sec. Processed 17.87 thousand rows, 71.51 KB (1.75 million rows/s., 7.01 MB/s.) - -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', system.query_log) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 656029 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 641155 │ -└───────────────────────────────┴─────────┘ - -3 rows in set. Elapsed: 0.026 sec. Processed 1.97 million rows, 7.88 MB (75.51 million rows/s., 302.05 MB/s.) -```` - -### Запросы по узлам и версиям {#querying-across-nodes-and-versions} - -Из‑за версионности системных таблиц это по‑прежнему не отражает все данные в кластере. Если совместить приведённое выше с функцией `merge`, мы получим точный результат для нашего диапазона дат: - -```sql -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', merge('system', '^query_log')) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 3008000 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 3659443 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 1078287 │ -└───────────────────────────────┴─────────┘ -``` - - -3 строки в наборе. Прошло: 0.462 сек. Обработано 7.94 млн строк, 31.75 MB (17.17 млн строк/с, 68.67 MB/s.) - -``` -``` - - -## Связанные материалы {#related-content} - -- Блог: [Системные таблицы и окно во внутреннюю работу ClickHouse](https://clickhouse.com/blog/clickhouse-debugging-issues-with-system-tables) -- Блог: [Основные запросы для мониторинга — часть 1 — запросы INSERT](https://clickhouse.com/blog/monitoring-troubleshooting-insert-queries-clickhouse) -- Блог: [Основные запросы для мониторинга — часть 2 — запросы SELECT](https://clickhouse.com/blog/monitoring-troubleshooting-select-queries-clickhouse) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md deleted file mode 100644 index cadd08b00d1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о событиях, произошедших с частями данных в таблицах семейства MergeTree, таких, как добавление или слияние данных.' -keywords: ['системная таблица', 'part_log'] -slug: /operations/system-tables/part_log -title: 'system.part_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.part_log {#systempart_log} - - - -Таблица `system.part_log` создаётся только в том случае, если задан параметр сервера [part_log](/operations/server-configuration-parameters/settings#part_log). - -Эта таблица содержит информацию о событиях, произошедших с [частями данных](../../engines/table-engines/mergetree-family/custom-partitioning-key.md) в таблицах семейства [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md), таких как добавление или слияние данных. - -Таблица `system.part_log` содержит следующие столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, выполняющего запрос. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор запроса `INSERT`, создавшего эту часть данных. -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — тип события, произошедшего с частью данных. Может принимать одно из следующих значений: - * `NewPart` — Вставка новой части данных. - * `MergePartsStart` — Начало слияния частей данных. - * `MergeParts` — Завершение слияния частей данных. - * `DownloadPart` — Загрузка части данных. - * `RemovePart` — Удаление или отсоединение части данных с помощью [DETACH PARTITION](/sql-reference/statements/alter/partition#detach-partitionpart). - * `MutatePartStart` — Начало мутации части данных. - * `MutatePart` — Завершение мутации части данных. - * `MovePart` — Перемещение части данных с одного диска на другой. -* `merge_reason` ([Enum8](../../sql-reference/data-types/enum.md)) — Причина возникновения события типа `MERGE_PARTS`. Может иметь одно из следующих значений: - * `NotAMerge` — Текущее событие имеет тип, отличный от `MERGE_PARTS`. - * `RegularMerge` — Обычное слияние. - * `TTLDeleteMerge` — Удаление просроченных данных. - * `TTLRecompressMerge` — Повторное сжатие части данных с. -* `merge_algorithm` ([Enum8](../../sql-reference/data-types/enum.md)) — алгоритм слияния события типа `MERGE_PARTS`. Может принимать одно из следующих значений: - * `Не выбрано` - * `Горизонтальная` - * `Вертикальная` -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время события. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время события с точностью до микросекунд. -* `duration_ms` ([UInt64](../../sql-reference/data-types/int-uint.md)) — продолжительность. -* `database` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой хранится часть данных. -* `table` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы, в которой находится часть данных. -* `table_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — UUID таблицы, которой принадлежит часть данных. -* `part_name` ([String](../../sql-reference/data-types/string.md)) — Имя части данных. -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор раздела, в который была вставлена часть данных. Столбец принимает значение `all`, если разбиение выполняется по `tuple()`. -* `partition` ([String](../../sql-reference/data-types/string.md)) - Имя раздела. -* `part_type` ([String](../../sql-reference/data-types/string.md)) - Тип части. Возможные значения: Wide и Compact. -* `disk_name` ([String](../../sql-reference/data-types/string.md)) - Имя диска, на котором находится часть данных. -* `path_on_disk` ([String](../../sql-reference/data-types/string.md)) — Абсолютный путь к каталогу с файлами кусков данных. -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Число строк в части данных. -* `size_in_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — размер части данных в байтах. -* `merged_from` ([Array(String)](../../sql-reference/data-types/array.md)) — массив имен частей, из которых была сформирована текущая часть (после слияния). -* `bytes_uncompressed` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Объём несжатых данных (в байтах). -* `read_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество строк, прочитанных при слиянии. -* `read_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — количество байт, прочитанных во время слияния. -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — максимальная разница между объёмом выделенной и освобождённой памяти в контексте этого потока. -* `error` ([UInt16](../../sql-reference/data-types/int-uint.md)) — Код возникшей ошибки. -* `exception` ([String](../../sql-reference/data-types/string.md)) — Текст сообщения о возникшей ошибке. -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/map.md)) — счётчики ProfileEvents, измеряющие различные метрики. Их описание приведено в таблице [system.events](/operations/system-tables/events). - -Таблица `system.part_log` создаётся после первой вставки данных в таблицу `MergeTree`. - -**Пример** - -```sql -SELECT * FROM system.part_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -query_id: -event_type: MergeParts -merge_reason: RegularMerge -merge_algorithm: Vertical -event_date: 2025-07-19 -event_time: 2025-07-19 23:54:19 -event_time_microseconds: 2025-07-19 23:54:19.710761 -duration_ms: 2158 -database: default -table: github_events -table_uuid: 1ad33424-f5f5-402b-ac03-ec82282634ab -part_name: all_1_7_1 -partition_id: all -partition: tuple() -part_type: Wide -disk_name: default -path_on_disk: ./data/store/1ad/1ad33424-f5f5-402b-ac03-ec82282634ab/all_1_7_1/ -rows: 3285726 -- 3,29 млн -size_in_bytes: 438968542 -- 438,97 млн -merged_from: ['all_1_1_0','all_2_2_0','all_3_3_0','all_4_4_0','all_5_5_0','all_6_6_0','all_7_7_0'] -bytes_uncompressed: 1373137767 -- 1,37 млрд -read_rows: 3285726 -- 3,29 млн -read_bytes: 1429206946 -- 1,43 млрд -peak_memory_usage: 303611887 -- 303,61 млн -error: 0 -exception: -ProfileEvents: {'FileOpen':703,'ReadBufferFromFileDescriptorRead':3824,'ReadBufferFromFileDescriptorReadBytes':439601681,'WriteBufferFromFileDescriptorWrite':592,'WriteBufferFromFileDescriptorWriteBytes':438988500,'ReadCompressedBytes':439601681,'CompressedReadBufferBlocks':6314,'CompressedReadBufferBytes':1539835748,'OpenedFileCacheHits':50,'OpenedFileCacheMisses':484,'OpenedFileCacheMicroseconds':222,'IOBufferAllocs':1914,'IOBufferAllocBytes':319810140,'ArenaAllocChunks':8,'ArenaAllocBytes':131072,'MarkCacheMisses':7,'CreatedReadBufferOrdinary':534,'DiskReadElapsedMicroseconds':139058,'DiskWriteElapsedMicroseconds':51639,'AnalyzePatchRangesMicroseconds':28,'ExternalProcessingFilesTotal':1,'RowsReadByMainReader':170857759,'WaitMarksLoadMicroseconds':988,'LoadedMarksFiles':7,'LoadedMarksCount':14,'LoadedMarksMemoryBytes':728,'Merge':2,'MergeSourceParts':14,'MergedRows':3285733,'MergedColumns':4,'GatheredColumns':51,'MergedUncompressedBytes':1429207058,'MergeTotalMilliseconds':2158,'MergeExecuteMilliseconds':2155,'MergeHorizontalStageTotalMilliseconds':145,'MergeHorizontalStageExecuteMilliseconds':145,'MergeVerticalStageTotalMilliseconds':2008,'MergeVerticalStageExecuteMilliseconds':2006,'MergeProjectionStageTotalMilliseconds':5,'MergeProjectionStageExecuteMilliseconds':4,'MergingSortedMilliseconds':7,'GatheringColumnMilliseconds':56,'ContextLock':2091,'PartsLockHoldMicroseconds':77,'PartsLockWaitMicroseconds':1,'RealTimeMicroseconds':2157475,'CannotWriteToWriteBufferDiscard':36,'LogTrace':6,'LogDebug':59,'LoggerElapsedNanoseconds':514040,'ConcurrencyControlSlotsGranted':53,'ConcurrencyControlSlotsAcquired':53} -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md deleted file mode 100644 index 09a5af55740..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о частях таблиц MergeTree' -keywords: ['системная таблица', 'части'] -slug: /operations/system-tables/parts -title: 'system.parts' -doc_type: 'reference' ---- - -# system.parts {#systemparts} - -Содержит информацию о частях таблиц [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -Каждая строка описывает одну часть данных. - -Столбцы: - -* `partition` ([String](../../sql-reference/data-types/string.md)) – Имя партиции. Чтобы узнать, что такое партиция, см. описание запроса [ALTER](/sql-reference/statements/alter). - - Форматы: - - * `YYYYMM` для автоматического партиционирования по месяцам. - * `any_string` при ручном партиционировании. - -* `name` ([String](../../sql-reference/data-types/string.md)) – Имя части данных. Структура имени части может использоваться для определения многих аспектов данных, приёма и шаблонов слияния. Формат имени части следующий: - -```text -_<мин_номер_блока>_<макс_номер_блока>_<уровень>_<версия_данных> -``` - -* Определения: - * `partition_id` — идентификатор ключа партиционирования - * `minimum_block_number` — минимальный номер блока в части. ClickHouse всегда объединяет последовательные блоки - * `maximum_block_number` — максимальный номер блока в части - * `level` — увеличивается на единицу при каждом дополнительном объединении части. Уровень 0 означает, что это новая часть, которая ещё не была объединена. Важно помнить, что все части в ClickHouse всегда неизменяемы - * `data_version` — необязательное значение, увеличивается, когда часть изменяется (при этом изменённые данные всегда записываются только в новую часть, так как части неизменяемы) - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — UUID части данных. - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — Формат хранения части данных. - - Допустимые значения: - - * `Wide` — Каждый столбец хранится в отдельном файле в файловой системе. - * `Compact` — Все столбцы хранятся в одном файле в файловой системе. - - Формат хранения данных определяется настройками `min_bytes_for_wide_part` и `min_rows_for_wide_part` для таблицы [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) – Флаг, показывающий, активна ли часть данных. Если часть данных активна, она используется в таблице. В противном случае она удаляется. Неактивные части данных остаются после слияния. - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) – количество меток. Чтобы получить примерное количество строк в части данных, умножьте `marks` на гранулярность индекса (обычно 8192) (эта оценка не работает для адаптивной гранулярности). - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Количество строк. - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Общий размер всех файлов частей данных в байтах. - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Общий размер сжатых данных в части данных. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Общий размер несжатых данных в части данных. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `primary_key_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Объем памяти (в байтах), используемый значениями первичного ключа в файле primary.idx/cidx на диске. - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – размер файла меток. - -* `secondary_indices_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – общий размер сжатых данных вторичных индексов в части данных. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `secondary_indices_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Общий размер несжатых данных для вторичных индексов в части данных. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `secondary_indices_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Размер файла с метками для вторичных индексов. - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – Время изменения каталога с частью данных. Обычно соответствует времени создания части данных. - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – время, когда часть данных стала неактивной. - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) – Количество мест, в которых используется часть данных. Значение больше 2 указывает, что часть данных используется в запросах или слияниях. - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) – минимальное значение ключа даты в части данных. - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) – максимальное значение ключа даты в части данных. - -* `min_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – минимальное значение ключа даты и времени в части данных. - -* `max_time`([DateTime](../../sql-reference/data-types/datetime.md)) – максимальное значение ключа даты и времени в части данных. - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) – идентификатор партиции. - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – минимальный номер блока данных, который входит в состав текущей части после слияния. - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Максимальный номер блока данных, входящего в состав текущей части после слияния. - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) – Глубина дерева MergeTree. Ноль означает, что текущая часть была создана операцией INSERT, а не в результате слияния других частей. - -* `data_version` ([UInt64](../../sql-reference/data-types/int-uint.md)) – число, которое используется для определения, какие мутации следует применить к части данных (мутации с версией, большей, чем `data_version`). - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Объём памяти (в байтах), занимаемый значениями первичного ключа (будет равен `0` при `primary_key_lazy_load=1` и `use_primary_key_cache=1`). - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Объем памяти (в байтах), зарезервированный для значений первичного ключа (будет `0` при `primary_key_lazy_load=1` и `use_primary_key_cache=1`). - -* `is_frozen` ([UInt8](../../sql-reference/data-types/int-uint.md)) – Флаг, показывающий, что существует резервная копия данных партиции. 1 — резервная копия существует. 0 — резервная копия не существует. Подробнее см. в разделе [FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) - -* `database` ([String](../../sql-reference/data-types/string.md)) – имя базы данных. - -* `table` ([String](../../sql-reference/data-types/string.md)) – имя таблицы. - -* `engine` ([String](../../sql-reference/data-types/string.md)) – Имя движка таблицы без параметров. - -* `path` ([String](../../sql-reference/data-types/string.md)) – Абсолютный путь к каталогу с файлами частей данных. - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) – Имя диска, на котором хранится часть данных. - -* `hash_of_all_files` ([String](../../sql-reference/data-types/string.md)) – значение [sipHash128](/sql-reference/functions/hash-functions#sipHash128) от сжатых файлов. - -* `hash_of_uncompressed_files` ([String](../../sql-reference/data-types/string.md)) – [sipHash128](/sql-reference/functions/hash-functions#sipHash128) от несжатых файлов (файлов с метками, файла индекса и т. д.). - -* `uncompressed_hash_of_compressed_files` ([String](../../sql-reference/data-types/string.md)) – значение [sipHash128](/sql-reference/functions/hash-functions#sipHash128) данных в сжатых файлах, вычисленное так, как если бы они были несжаты. - -* `delete_ttl_info_min` ([DateTime](../../sql-reference/data-types/datetime.md)) — минимальное значение ключа даты и времени для [правила TTL DELETE](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl). - -* `delete_ttl_info_max` ([DateTime](../../sql-reference/data-types/datetime.md)) — максимальное значение ключа даты и времени для [правила TTL DELETE](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl). - -* `move_ttl_info.expression` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — массив выражений. Каждое выражение определяет [правило TTL MOVE](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl). - -:::note -Массив `move_ttl_info.expression` в основном сохраняется для обратной совместимости, а сейчас самый простой способ проверить правило `TTL MOVE` — использовать поля `move_ttl_info.min` и `move_ttl_info.max`. -::: - -* `move_ttl_info.min` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — Массив значений даты и времени. Каждый элемент задаёт минимальное значение ключа для [правила TTL MOVE](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl). - -* `move_ttl_info.max` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — Массив значений даты и времени. Каждый элемент задаёт максимальное значение ключа для [правила TTL MOVE](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl). - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Псевдоним для `bytes_on_disk`. - -* `marks_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – Псевдоним для `marks_bytes`. - -**Пример** - -```sql -SELECT * FROM system.parts LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_4_1_6 -part_type: Wide -active: 1 -marks: 2 -rows: 6 -bytes_on_disk: 310 -data_compressed_bytes: 157 -data_uncompressed_bytes: 91 -secondary_indices_compressed_bytes: 58 -secondary_indices_uncompressed_bytes: 6 -secondary_indices_marks_bytes: 48 -marks_bytes: 144 -modification_time: 2020-06-18 13:01:49 -remove_time: 1970-01-01 00:00:00 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -min_time: 1970-01-01 00:00:00 -max_time: 1970-01-01 00:00:00 -partition_id: all -min_block_number: 1 -max_block_number: 4 -level: 1 -data_version: 6 -primary_key_bytes_in_memory: 8 -primary_key_bytes_in_memory_allocated: 64 -is_frozen: 0 -database: default -table: months -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/months/all_1_4_1_6/ -hash_of_all_files: 2d0657a16d9430824d35e327fcbd87bf -hash_of_uncompressed_files: 84950cc30ba867c77a408ae21332ba29 -uncompressed_hash_of_compressed_files: 1ad78f1c6843bbfb99a2c931abe7df7d -delete_ttl_info_min: 1970-01-01 00:00:00 -delete_ttl_info_max: 1970-01-01 00:00:00 -move_ttl_info.expression: [] -move_ttl_info.min: [] -move_ttl_info.max: [] -``` - -**См. также** - -* [Семейство движков MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) -* [TTL для столбцов и таблиц](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md deleted file mode 100644 index bcf6791dfe7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о партах и столбцах таблиц MergeTree.' -keywords: ['системная таблица', 'parts_columns'] -slug: /operations/system-tables/parts_columns -title: 'system.parts_columns' -doc_type: 'reference' ---- - -# system.parts_columns {#systemparts_columns} - -Содержит информацию о партах и столбцах таблиц [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -Каждая строка описывает один парт данных. - -Столбцы: - -* `partition` ([String](../../sql-reference/data-types/string.md)) — Имя раздела. Чтобы узнать, что такое раздел, см. описание запроса [ALTER](/sql-reference/statements/alter). - - Форматы: - - * `YYYYMM` для автоматического разбиения на партиции по месяцам. - * `any_string` при ручном разбиении на партиции. - -* `name` ([String](../../sql-reference/data-types/string.md)) — Название части данных. - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — Формат хранения части данных. - - Допустимые значения: - - * `Wide` — каждый столбец хранится в отдельном файле в файловой системе. - * `Compact` — все столбцы хранятся в одном файле в файловой системе. - - Формат хранения данных задаётся настройками `min_bytes_for_wide_part` и `min_rows_for_wide_part` таблицы [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md). - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) — флаг, показывающий, активна ли часть данных. Если часть данных активна, она используется в таблице. В противном случае она удаляется. Неактивные части данных остаются после слияния. - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество меток. Чтобы получить примерное число строк в части данных, умножьте `marks` на гранулярность индекса (обычно 8192) (эта оценка не применима при адаптивной гранулярности). - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — количество строк. - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Общий размер всех файлов частей данных в байтах. - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Общий размер сжатых данных в части. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Общий размер несжатых данных в части данных. Все вспомогательные файлы (например, файлы с метками) не учитываются. - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — размер файла меток. - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время изменения каталога, содержащего часть данных. Обычно соответствует времени создания части данных. - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время, когда часть данных стала неактивной. - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) — количество мест, где используется часть данных. Значение больше 2 означает, что часть данных используется в запросах или операциях слияния. - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) — минимальное значение ключа по дате в части данных. - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) — максимальное значение по ключу даты в части данных. - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор раздела. - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Минимальное количество частей данных, из которых после слияния состоит текущая часть. - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Максимальное число частей данных, образующих текущую часть после слияния.` - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Глубина дерева слияний. Ноль означает, что текущая часть была создана вставкой, а не в результате слияния других частей. - -* `data_version` ([UInt64](../../sql-reference/data-types/int-uint.md)) — число, которое используется для определения, какие мутации нужно применить к части данных (мутации с версией больше, чем `data_version`). - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Объём памяти (в байтах), занимаемый значениями первичного ключа. - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Объём памяти (в байтах), зарезервированный для значений первичного ключа. - -* `database` ([String](../../sql-reference/data-types/string.md)) — имя базы данных. - -* `table` ([String](../../sql-reference/data-types/string.md)) — имя таблицы. - -* `engine` ([String](../../sql-reference/data-types/string.md)) — Название движка таблицы без указания параметров. - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) — Имя диска, на котором хранится часть данных. - -* `path` ([String](../../sql-reference/data-types/string.md)) — Абсолютный путь к папке с файлами частей данных. - -* `column` ([String](../../sql-reference/data-types/string.md)) — имя столбца. - -* `type` ([String](../../sql-reference/data-types/string.md)) — тип столбца. - -* `column_position` ([UInt64](../../sql-reference/data-types/int-uint.md)) — порядковый номер столбца в таблице, начиная с 1. - -* `default_kind` ([String](../../sql-reference/data-types/string.md)) — тип выражения (`DEFAULT`, `MATERIALIZED`, `ALIAS`) для значения по умолчанию или пустая строка, если значение не определено. - -* `default_expression` ([String](../../sql-reference/data-types/string.md)) — выражение, задающее значение по умолчанию, или пустая строка, если оно не определено. - -* `column_bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — общий размер столбца на диске в байтах. - -* `column_data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — общий размер сжатых данных в столбце, в байтах. - -* `column_data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Общий размер несжатых данных в столбце в байтах. - -* `column_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Размер столбца с метками в байтах. - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — псевдоним для `bytes_on_disk`. - -* `marks_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) — синоним для `marks_bytes`. - -**Пример** - -```sql -SELECT * FROM system.parts_columns LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_2_1 -part_type: Wide -active: 1 -marks: 2 -rows: 2 -bytes_on_disk: 155 -data_compressed_bytes: 56 -data_uncompressed_bytes: 4 -marks_bytes: 96 -modification_time: 2020-09-23 10:13:36 -remove_time: 2106-02-07 06:28:15 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -partition_id: all -min_block_number: 1 -max_block_number: 2 -level: 1 -data_version: 1 -primary_key_bytes_in_memory: 2 -primary_key_bytes_in_memory_allocated: 64 -database: default -table: 53r93yleapyears -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/53r93yleapyears/all_1_2_1/ -column: id -type: Int8 -column_position: 1 -default_kind: -default_expression: -column_bytes_on_disk: 76 -column_data_compressed_bytes: 28 -column_data_uncompressed_bytes: 2 -column_marks_bytes: 48 -``` - -**См. также** - -* [Семейство MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md deleted file mode 100644 index ad7251d12fc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 'Системная таблица, используемая для реализации запроса `SHOW PROCESSLIST`.' -keywords: ['системная таблица', 'процессы'] -slug: /operations/system-tables/processes -title: 'system.processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processes {#systemprocesses} - - - -Эта системная таблица предназначена для реализации запроса `SHOW PROCESSLIST`. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.processes LIMIT 10 FORMAT Vertical; -``` - -```response -Row 1: -────── -is_initial_query: 1 -user: default -query_id: 35a360fa-3743-441d-8e1f-228c938268da -address: ::ffff:172.23.0.1 -port: 47588 -initial_user: default -initial_query_id: 35a360fa-3743-441d-8e1f-228c938268da -initial_address: ::ffff:172.23.0.1 -initial_port: 47588 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -elapsed: 0.000582537 -is_cancelled: 0 -is_all_data_sent: 0 -read_rows: 0 -read_bytes: 0 -total_rows_approx: 0 -written_rows: 0 -written_bytes: 0 -memory_usage: 0 -peak_memory_usage: 0 -query: SELECT * from system.processes LIMIT 10 FORMAT Vertical; -thread_ids: [67] -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -Settings: {'background_pool_size':'32','load_balancing':'random','allow_suspicious_low_cardinality_types':'1','distributed_aggregation_memory_efficient':'1','skip_unavailable_shards':'1','log_queries':'1','max_bytes_before_external_group_by':'20000000000','max_bytes_before_external_sort':'20000000000','allow_introspection_functions':'1'} - -Получена 1 строка. Затрачено: 0.002 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md deleted file mode 100644 index d92dfaf7d9d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'Системная таблица, содержащая данные профилирования на уровне процессоров - (которые можно увидеть в `EXPLAIN PIPELINE`)' -keywords: ['system table', 'processors_profile_log', 'EXPLAIN PIPELINE'] -slug: /operations/system-tables/processors_profile_log -title: 'system.processors_profile_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processors_profile_log {#systemprocessors_profile_log} - - - -Эта таблица содержит профилирующие данные на уровне процессоров (которые можно увидеть в [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline)). - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата, когда произошло событие. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время, когда произошло событие. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Дата и время с точностью до микросекунд, когда произошло событие. -* `id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ID процессора. -* `parent_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — ID родительских процессоров. -* `plan_step` ([UInt64](../../sql-reference/data-types/int-uint.md)) — ID шага плана запроса, который создал этот процессор. Значение равно нулю, если процессор не был добавлен ни одним шагом. -* `plan_group` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Группа процессора, если он был создан шагом плана запроса. Группа — это логическое разбиение процессоров, добавленных одним и тем же шагом плана запроса. Группа используется только для улучшения читаемости результата EXPLAIN PIPELINE. -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — ID исходного запроса (для распределённого выполнения запросов). -* `query_id` ([String](../../sql-reference/data-types/string.md)) — ID запроса. -* `name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — Имя процессора. -* `elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество микросекунд, в течение которых выполнялся этот процессор. -* `input_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество микросекунд, в течение которых этот процессор ожидал данные (от другого процессора). -* `output_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество микросекунд, в течение которых этот процессор ожидал, потому что выходной порт был заполнен. -* `input_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество строк, обработанных процессором. -* `input_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество байт, обработанных процессором. -* `output_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество строк, сгенерированных процессором. -* `output_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество байт, сгенерированных процессором. - **Пример** - -Запрос: - -```sql -EXPLAIN PIPELINE -SELECT sleep(1) -┌─explain─────────────────────────┐ -│ (Expression) │ -│ ExpressionTransform │ -│ (SettingQuotaAndLimits) │ -│ (ReadFromStorage) │ -│ SourceFromSingleChunk 0 → 1 │ -└─────────────────────────────────┘ - -SELECT sleep(1) -SETTINGS log_processors_profiles = 1 -Query id: feb5ed16-1c24-4227-aa54-78c02b3b27d4 -┌─sleep(1)─┐ -│ 0 │ -└──────────┘ -1 rows in set. Elapsed: 1.018 sec. - -SELECT - name, - elapsed_us, - input_wait_elapsed_us, - output_wait_elapsed_us -FROM system.processors_profile_log -WHERE query_id = 'feb5ed16-1c24-4227-aa54-78c02b3b27d4' -ORDER BY name ASC -``` - -Результат: - -```text -┌─name────────────────────┬─elapsed_us─┬─input_wait_elapsed_us─┬─output_wait_elapsed_us─┐ -│ ExpressionTransform │ 1000497 │ 2823 │ 197 │ -│ LazyOutputFormat │ 36 │ 1002188 │ 0 │ -│ LimitsCheckingTransform │ 10 │ 1002994 │ 106 │ -│ NullSource │ 5 │ 1002074 │ 0 │ -│ NullSource │ 1 │ 1002084 │ 0 │ -│ SourceFromSingleChunk │ 45 │ 4736 │ 1000819 │ -└─────────────────────────┴────────────┴───────────────────────┴────────────────────────┘ -``` - -Здесь видно, что: - -* `ExpressionTransform` выполнял функцию `sleep(1)`, поэтому выполнение его `work` займет 1e6 мкс, и, следовательно, `elapsed_us` > 1e6. -* `SourceFromSingleChunk` должен ждать, потому что `ExpressionTransform` не принимает данные во время выполнения `sleep(1)`, поэтому он будет в состоянии `PortFull` в течение 1e6 мкс, и, следовательно, `output_wait_elapsed_us` > 1e6. -* `LimitsCheckingTransform`/`NullSource`/`LazyOutputFormat` должны ждать, пока `ExpressionTransform` выполнит `sleep(1)`, чтобы обработать результат, поэтому `input_wait_elapsed_us` > 1e6. - -**См. также** - -* [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md deleted file mode 100644 index 2c4dae432db..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о частях проекций таблиц семейства MergeTree.' -keywords: ['system table', 'projection_parts'] -slug: /operations/system-tables/projection_parts -title: 'system.projection_parts' -doc_type: 'reference' ---- - -# system.projection_parts {#systemprojection_parts} - -Эта таблица содержит информацию о частях проекций таблиц семейства MergeTree. - -## Столбцы {#columns} - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md deleted file mode 100644 index 1a9ef58d210..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о столбцах в частях проекций таблиц семейства MergeTree' -keywords: ['system table', 'projection_parts_columns'] -slug: /operations/system-tables/projection_parts_columns -title: 'system.projection_parts_columns' -doc_type: 'reference' ---- - -# system.projection_parts_columns {#systemprojection_parts_columns} - -Эта таблица содержит информацию о столбцах частей проекций таблиц семейства MergeTree. - -## Столбцы {#columns} - -{/*AUTOGENERGED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md deleted file mode 100644 index e091b768f1f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о существующих проекциях всех таблиц.' -keywords: ['системная таблица', 'проекции'] -slug: /operations/system-tables/projections -title: 'system.projections' -doc_type: 'reference' ---- - -# system.projections {#systemprojections} - -Содержит информацию о существующих проекциях во всех таблицах. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.projections LIMIT 2 FORMAT Vertical; -``` - -```text -Строка 1: -───────── -database: default -table: landing -name: improved_sorting_key -type: Normal -sorting_key: ['user_id','date'] -query: SELECT * ORDER BY user_id, date - -Строка 2: -───────── -database: default -table: landing -name: agg_no_key -type: Aggregate -sorting_key: [] -query: SELECT count() -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md deleted file mode 100644 index a9d9ab3230f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: 'Системная таблица, показывающая содержимое кэша запросов.' -keywords: ['system table', 'query_cache'] -slug: /operations/system-tables/query_cache -title: 'system.query_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_cache {#systemquery_cache} - - - -Показывает содержимое [кэша запросов](../query-cache.md). - -Столбцы: - -{/*AUTOGENERGED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.query_cache FORMAT Vertical; -``` - -```text -Строка 1: -────── -query: SELECT 1 SETTINGS use_query_cache = 1 -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -result_size: 128 -tag: -stale: 0 -shared: 0 -compressed: 1 -expires_at: 2023-10-13 13:35:45 -key_hash: 12188185624808016954 - -Получена 1 строка. Затрачено: 0.004 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md deleted file mode 100644 index 9333c1e6afe..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'Системная таблица, которая отображает содержимое кэша условий запроса.' -keywords: ['системная таблица', 'query_condition_cache'] -slug: /operations/system-tables/query_condition_cache -title: 'system.query_condition_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_condition_cache {#systemquery_condition_cache} - - - -Отображает содержимое [кэша условий запросов](../query-condition-cache.md). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.query_condition_cache FORMAT Vertical; -``` - -```text -Строка 1: -────── -table_uuid: 28270a24-ea27-49f6-99cd-97b9bee976ac -part_name: all_1_1_0 -condition: or(equals(b, 10000_UInt16), equals(c, 10000_UInt16)) -condition_hash: 5456494897146899690 -- 5.46 quintillion -entry_size: 40 -matching_marks: 111111110000000000000000000000000000000000000000000000000111111110000000000000000 - -Получена 1 строка. Затрачено: 0.004 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md deleted file mode 100644 index 930889c6567..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md +++ /dev/null @@ -1,243 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о выполненных запросах, например, - время начала, длительность выполнения, сообщения об ошибках.' -keywords: ['system table', 'query_log'] -slug: /operations/system-tables/query_log -title: 'system.query_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.query_log {#systemquery_log} - - - -Хранит метаданные и статистику о выполненных запросах: время начала, длительность, сообщения об ошибках, использование ресурсов и другие детали выполнения. Эта таблица не хранит результаты запросов. - -Вы можете изменить настройки логирования запросов в разделе конфигурации сервера [query_log](../../operations/server-configuration-parameters/settings.md#query_log). - -Вы можете отключить логирование запросов, установив [log_queries = 0](/operations/settings/settings#log_queries). Мы не рекомендуем отключать логирование, так как информация в этой таблице важна для устранения неполадок. - -Период сброса данных задаётся параметром `flush_interval_milliseconds` в разделе настроек сервера [query_log](../../operations/server-configuration-parameters/settings.md#query_log). Чтобы принудительно выполнить сброс, используйте запрос [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs). - -ClickHouse не удаляет данные из этой таблицы автоматически. См. раздел [Introduction](/operations/system-tables/overview#system-tables-introduction) для получения дополнительной информации. - -Таблица `system.query_log` регистрирует два типа запросов: - -1. Первичные запросы, которые были запущены непосредственно клиентом. -2. Дочерние запросы, которые были инициированы другими запросами (для распределённого выполнения запросов). Для таких запросов информация о родительских запросах отображается в столбцах `initial_*`. - -Каждый запрос создаёт одну или две строки в таблице `query_log` в зависимости от статуса запроса (см. столбец `type`): - -1. Если выполнение запроса прошло успешно, создаются две строки с типами `QueryStart` и `QueryFinish`. -2. Если при обработке запроса произошла ошибка, создаются два события с типами `QueryStart` и `ExceptionWhileProcessing`. -3. Если ошибка произошла до запуска запроса, создаётся одно событие с типом `ExceptionBeforeStart`. - -Вы можете использовать настройку [log_queries_probability](/operations/settings/settings#log_queries_probability), чтобы уменьшить количество запросов, регистрируемых в таблице `query_log`. - -Вы можете использовать настройку [log_formatted_queries](/operations/settings/settings#log_formatted_queries), чтобы логировать форматированные запросы в столбец `formatted_query`. - - - -## Столбцы {#columns} - - - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, на котором выполняется запрос. -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип события, произошедшего при выполнении запроса. Значения:` - * `'QueryStart' = 1` — Успешное начало выполнения запроса. - * `'QueryFinish' = 2` — Успешное завершение выполнения запроса. - * `'ExceptionBeforeStart' = 3` — Исключение до начала выполнения запроса. - * `'ExceptionWhileProcessing' = 4` — Исключение во время выполнения запроса. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата начала запроса. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время начала запроса. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — время начала запроса с точностью до микросекунд. -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время начала выполнения запроса. -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — время начала выполнения запроса с микросекундной точностью. -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — время выполнения запроса в миллисекундах. -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Общее количество строк, прочитанных из всех таблиц и табличных функций, участвующих в запросе. Включает обычные подзапросы, подзапросы в `IN` и `JOIN`. Для распределённых запросов `read_rows` включает общее количество строк, прочитанных на всех репликах. Каждая реплика отправляет своё значение `read_rows`, а сервер-инициатор запроса суммирует все полученные и локальные значения. Объём кэша не влияет на это значение. -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — общее количество байт, прочитанных из всех таблиц и табличных функций, участвовавших в запросе. Включает обычные подзапросы, подзапросы для `IN` и `JOIN`. Для распределённых запросов `read_bytes` включает общее количество байт, прочитанных на всех репликах. Каждая реплика отправляет своё значение `read_bytes`, а сервер-инициатор запроса суммирует все полученные и локальные значения. Объём кэша не влияет на это значение. -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — для запросов `INSERT` количество записанных строк. Для остальных запросов значение в этом столбце равно 0. -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Для запросов `INSERT` количество записанных байт в несжатом виде. Для других запросов значение столбца равно 0. -* `result_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество строк в результате выполнения запроса `SELECT` или количество строк в запросе `INSERT`. -* `result_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — объём оперативной памяти в байтах, используемый для хранения результата запроса. -* `memory_usage` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Использование памяти запросом. -* `current_database` ([String](../../sql-reference/data-types/string.md)) — имя текущей базы данных. -* `query` ([String](../../sql-reference/data-types/string.md)) — строка запроса. -* `formatted_query` ([String](../../sql-reference/data-types/string.md)) — Форматированная строка запроса. -* `normalized_query_hash` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — числовой хэш, одинаковый для запросов, которые отличаются только значениями литералов. -* `query_kind` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — тип запроса. -* `databases` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Имена баз данных, которые присутствуют в запросе. -* `tables` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Имена таблиц, используемых в запросе. -* `columns` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Имена столбцов, имеющихся в запросе. -* `partitions` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Имена партиций, задействованных в запросе. -* `projections` ([String](../../sql-reference/data-types/string.md)) — Имена проекций, использованных при выполнении запроса. -* `views` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Имена материализованных или живых представлений (Live View), используемых в запросе. -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — Код исключения. -* `exception` ([String](../../sql-reference/data-types/string.md)) — сообщение об исключении. -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [трассировка стека](https://en.wikipedia.org/wiki/Stack_trace). Пустая строка, если запрос был успешно выполнен. -* `is_initial_query` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Тип запроса. Возможные значения: - * 1 — Запрос был инициирован клиентом. - * 0 — Запрос был инициирован другим запросом в рамках выполнения распределённого запроса. -* `user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя, который инициировал текущий запрос. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор запроса. -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес, с которого был выполнен запрос. -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — Порт клиента, использованный для выполнения запроса. -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя, который выполнил исходный запрос (при распределённом выполнении запросов). -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор исходного запроса (для распределённого выполнения запросов). -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес, с которого был инициирован родительский запрос. -* `initial_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — клиентский порт, который использовался при выполнении родительского запроса. -* `initial_query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время начала исходного запроса (для распределённого выполнения запроса). -* `initial_query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время начала исходного запроса с точностью до микросекунд (при распределённом выполнении запросов). -* `interface` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Интерфейс, через который был инициирован запрос. Возможные значения: - * 1 — TCP. - * 2 — HTTP. -* `os_user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя операционной системы, от имени которого запускается [clickhouse-client](../../interfaces/cli.md). -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — имя хоста клиентской машины, на которой запущен [clickhouse-client](../../interfaces/cli.md) или другой TCP‑клиент. -* `client_name` ([String](../../sql-reference/data-types/string.md)) — Имя [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ревизия клиента [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Старший номер версии [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Минорная версия клиента [clickhouse-client](../../interfaces/cli.md) или другого TCP‑клиента. -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — номер патча в версии [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `script_query_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Номер запроса в скрипте с несколькими запросами для использования в [clickhouse-client](../../interfaces/cli.md). -* `script_line_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Номер строки, на которой начинается запрос, в скрипте с несколькими запросами для [clickhouse-client](../../interfaces/cli.md). -* `http_method` (UInt8) — HTTP-метод, которым был инициирован запрос. Возможные значения:` - * 0 — Запрос был отправлен через TCP-интерфейс. - * 1 — Использовался метод `GET`. - * 2 — Использовался метод `POST`. -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — HTTP-заголовок `User-Agent`, переданный в HTTP-запросе. -* `http_referer` ([String](../../sql-reference/data-types/string.md)) — HTTP-заголовок `Referer`, переданный в HTTP-запросе (содержит абсолютный или частичный адрес страницы, отправляющей запрос). -* `forwarded_for` ([String](../../sql-reference/data-types/string.md)) — HTTP-заголовок `X-Forwarded-For`, переданный в HTTP-запросе. -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — «ключ квоты», указанный в настройке [quotas](../../operations/quotas.md) (см. `keyed`). -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ревизия ClickHouse. -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/map.md)) — события ProfileEvents, которые измеряют различные метрики. Их описание приведено в таблице [system.events](/operations/system-tables/events) -* `Settings` ([Map(String, String)](../../sql-reference/data-types/map.md)) — настройки, которые были изменены при выполнении запроса клиентом. Чтобы включить логирование изменений настроек, установите значение параметра `log_query_settings` равным 1. -* `log_comment` ([String](../../sql-reference/data-types/string.md)) — комментарий для лога. Может быть установлен в произвольную строку длиной не более [max_query_size](../../operations/settings/settings.md#max_query_size). Пустая строка, если не задан. -* `thread_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — Идентификаторы потоков, участвующих в выполнении запроса. Эти потоки могли выполняться не параллельно.` -* `peak_threads_usage` ([UInt64)](../../sql-reference/data-types/int-uint.md)) — максимальное число потоков, одновременно выполняющих запрос. -* `used_aggregate_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена `aggregate functions`, использованных при выполнении запроса. -* `used_aggregate_function_combinators` ([Array(String)](../../sql-reference/data-types/array.md)) — канонические имена комбинаторов агрегатных функций, использованных при выполнении запроса. -* `used_database_engines` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена `движков баз данных`, использованных при выполнении запроса. -* `used_data_type_families` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена семейств типов данных, которые использовались во время выполнения запроса. -* `used_dictionaries` ([Array(String)](../../sql-reference/data-types/array.md)) — канонические имена `dictionaries`, которые были использованы при выполнении запроса. Для словарей, настроенных с использованием XML-файла, это имя словаря, а для словарей, созданных с помощью SQL-оператора, каноническим именем является полное квалифицированное имя объекта. -* `used_formats` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические названия `formats`, которые были использованы во время выполнения запроса. -* `used_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — канонические имена функций, использованных при выполнении запроса. -* `used_storages` ([Array(String)](../../sql-reference/data-types/array.md)) — канонические имена хранилищ (`storages`), которые использовались при выполнении запроса. -* `used_table_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена `table functions`, использованных при выполнении запроса. -* `used_executable_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена `executable user defined functions`, которые были использованы при выполнении запроса. -* `used_sql_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — Канонические имена пользовательских SQL-функций, которые были использованы во время выполнения запроса. -* `used_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) - Права доступа, которые были успешно проверены при выполнении запроса. -* `missing_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) — Недостающие привилегии при выполнении запроса. -* `query_cache_usage` ([Enum8](../../sql-reference/data-types/enum.md)) — использование [кэша запросов](../query-cache.md) при выполнении запроса. Значения: - * `'Unknown'` = Статус неизвестен. - * `'None'` = Результат запроса не был ни записан в кэш запросов, ни прочитан из него. - * `'Write'` = Результат запроса был записан в кэш запросов. - * `'Read'` = Результат запроса был прочитан из кэша запросов. - - - - - -## Примеры {#examples} - -**Простой пример** - -```sql -SELECT * FROM system.query_log WHERE type = 'QueryFinish' ORDER BY query_start_time DESC LIMIT 1 FORMAT Vertical; -``` - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -type: QueryFinish -event_date: 2021-11-03 -event_time: 2021-11-03 16:13:54 -event_time_microseconds: 2021-11-03 16:13:54.953024 -query_start_time: 2021-11-03 16:13:54 -query_start_time_microseconds: 2021-11-03 16:13:54.952325 -query_duration_ms: 0 -read_rows: 69 -read_bytes: 6187 -written_rows: 0 -written_bytes: 0 -result_rows: 69 -result_bytes: 48256 -memory_usage: 0 -current_database: default -query: DESCRIBE TABLE system.query_log -formatted_query: -normalized_query_hash: 8274064835331539124 -query_kind: -databases: [] -tables: [] -columns: [] -projections: [] -views: [] -exception_code: 0 -exception: -stack_trace: -is_initial_query: 1 -user: default -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -address: ::ffff:127.0.0.1 -port: 40452 -initial_user: default -initial_query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -initial_address: ::ffff:127.0.0.1 -initial_port: 40452 -initial_query_start_time: 2021-11-03 16:13:54 -initial_query_start_time_microseconds: 2021-11-03 16:13:54.952325 -interface: 1 -os_user: sevirov -client_hostname: clickhouse.eu-central1.internal -client_name: ClickHouse -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 1 -http_method: 0 -http_user_agent: -http_referer: -forwarded_for: -quota_key: -revision: 54456 -log_comment: -thread_ids: [30776,31174] -ProfileEvents: {'Query':1,'NetworkSendElapsedMicroseconds':59,'NetworkSendBytes':2643,'SelectedRows':69,'SelectedBytes':6187,'ContextLock':9,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':817,'UserTimeMicroseconds':427,'SystemTimeMicroseconds':212,'OSCPUVirtualTimeMicroseconds':639,'OSReadChars':894,'OSWriteChars':319} -Settings: {'load_balancing':'random','max_memory_usage':'10000000000'} -used_aggregate_functions: [] -used_aggregate_function_combinators: [] -used_database_engines: [] -used_data_type_families: [] -used_dictionaries: [] -used_formats: [] -used_functions: [] -used_storages: [] -used_table_functions: [] -used_executable_user_defined_functions:[] -used_sql_user_defined_functions: [] -used_privileges: [] -missing_privileges: [] -query_cache_usage: None -``` - -**Пример для облака** - -В ClickHouse Cloud `system.query_log` локален для каждого узла; чтобы увидеть все записи, необходимо выполнять запрос через [`clusterAllReplicas`](/sql-reference/table-functions/cluster). - -Например, чтобы агрегировать строки `query_log` со всех реплик в кластере «default», можно выполнить: - -```sql -SELECT * -FROM clusterAllReplicas('default', system.query_log) -WHERE event_time >= now() - toIntervalHour(1) -LIMIT 10 -SETTINGS skip_unavailable_shards = 1; -``` - -**См. также** - -* [system.query_thread_log](/operations/system-tables/query_thread_log) — Эта таблица содержит информацию о каждом потоке выполнения запроса. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md deleted file mode 100644 index e69e381325c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Системная таблица, содержащая историю значений использования памяти и метрик из таблицы `system.events` для отдельных запросов, периодически записываемую на диск.' -keywords: ['системная таблица', 'query_metric_log'] -slug: /operations/system-tables/query_metric_log -title: 'system.query_metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_metric_log {#systemquery_metric_log} - - - -Содержит историю значений использования памяти и метрик из таблицы `system.events` для отдельных запросов, периодически сбрасываемую на диск. - -После запуска запроса данные собираются с периодичностью `query_metric_log_interval` миллисекунд (по умолчанию — 1000). Данные также собираются при завершении запроса, если он выполняется дольше, чем `query_metric_log_interval`. - -Столбцы: - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор запроса. -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — дата события. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — время события. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — время события с точностью до микросекунд. - -**Пример** - -```sql -SELECT * FROM system.query_metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -Строка 1: -────── -query_id: 97c8ba04-b6d4-4bd7-b13e-6201c5c6e49d -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -memory_usage: 313434219 -peak_memory_usage: 598951986 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -``` - -**См. также** - -* [параметр query_metric_log](../../operations/server-configuration-parameters/settings.md#query_metric_log) — включение и выключение параметра. -* [query_metric_log_interval](../../operations/settings/settings.md#query_metric_log_interval) -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — содержит периодически вычисляемые метрики. -* [system.events](/operations/system-tables/events) — содержит ряд произошедших событий. -* [system.metrics](../../operations/system-tables/metrics.md) — содержит мгновенно вычисляемые метрики. -* [Мониторинг](../../operations/monitoring.md) — основные принципы мониторинга ClickHouse. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md deleted file mode 100644 index a1927b5492c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о потоках, которые выполняют запросы, - например, имя потока, время его запуска, длительность обработки запроса.' -keywords: ['системная таблица', 'query_thread_log'] -slug: /operations/system-tables/query_thread_log -title: 'system.query_thread_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_thread_log {#systemquery_thread_log} - - - -Содержит информацию о потоках, которые выполняют запросы, например имя потока, время его запуска, длительность обработки запроса. - -Чтобы включить логирование: - -1. Настройте параметры в разделе [query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log). -2. Установите [log_query_threads](/operations/settings/settings#log_query_threads) в значение 1. - -Период сброса данных задаётся параметром `flush_interval_milliseconds` в разделе настроек сервера [query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log). Для принудительного сброса используйте запрос [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs). - -ClickHouse не удаляет данные из таблицы автоматически. См. раздел [Введение](/operations/system-tables/overview#system-tables-introduction) для получения дополнительной информации. - -Вы можете использовать настройку [log_queries_probability](/operations/settings/settings#log_queries_probability) для уменьшения количества запросов, регистрируемых в таблице `query_thread_log`. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, на котором выполняется запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — дата, когда поток завершил выполнение запроса. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время, когда поток завершил выполнение запроса. -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время, когда поток завершил выполнение запроса, с точностью до микросекунд. -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время начала выполнения запроса. -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — время начала выполнения запроса с микросекундной точностью. -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Длительность выполнения запроса. -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество прочитанных строк. -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — количество прочитанных байт. -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — для запросов `INSERT` количество записанных строк. Для остальных запросов значение столбца равно 0. -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — для запросов `INSERT` количество записанных байт. Для других запросов значение столбца равно 0. -* `memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — разность между объемом выделенной и освобожденной памяти в контексте этого потока. -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — Максимальная разница между объёмами выделенной и освобождённой памяти в контексте этого потока. -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — Имя потока. -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — идентификатор потока операционной системы. -* `master_thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — исходный идентификатор основного потока в ОС. -* `query` ([String](../../sql-reference/data-types/string.md)) — строка запроса. -* `is_initial_query` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — тип запроса. Возможные значения: - * 1 — Запрос был инициирован клиентом. - * 0 — Запрос был инициирован другим запросом при распределенном выполнении. -* `user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя, инициировавшего текущий запрос. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор запроса. -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес, который использовался при выполнении запроса. -* `port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — Клиентский порт, использованный для выполнения запроса. -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя, который выполнил исходный запрос (для распределённого выполнения запроса). -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — идентификатор исходного запроса (для распределённого выполнения запросов). -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес, с которого был запущен родительский запрос. -* `initial_port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — Клиентский порт, который был использован для выполнения родительского запроса. -* `interface` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Интерфейс, из которого был инициирован запрос. Возможные значения: - * 1 — TCP. - * 2 — HTTP. -* `os_user` ([String](../../sql-reference/data-types/string.md)) — имя пользователя операционной системы, под которым запущен [clickhouse-client](../../interfaces/cli.md). -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — имя хоста клиентского компьютера, на котором запущен [clickhouse-client](../../interfaces/cli.md) или другой TCP-клиент. -* `client_name` ([String](../../sql-reference/data-types/string.md)) — имя [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ревизия клиента [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Мажорная версия [clickhouse-client](../../interfaces/cli.md) или другого TCP‑клиента. -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — номер минорной версии [clickhouse-client](../../interfaces/cli.md) или другого TCP-клиента. -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Патч-компонент версии [clickhouse-client](../../interfaces/cli.md) или другого TCP‑клиента. -* `http_method` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — HTTP-метод, инициировавший запрос. Возможные значения: - * 0 — Запрос был выполнен через TCP-интерфейс. - * 1 — Использовался метод `GET`. - * 2 — Использовался метод `POST`. -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — заголовок `UserAgent`, переданный в HTTP-запросе. -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — «ключ квоты», заданный в настройке [quotas](../../operations/quotas.md) (см. `keyed`). -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ревизия ClickHouse. -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — события профилирования ProfileEvents, которые измеряют различные метрики для этого потока. Их описание можно найти в таблице [system.events](/operations/system-tables/events). - -**Пример** - -```sql - SELECT * FROM system.query_thread_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-11 -event_time: 2020-09-11 10:08:17 -event_time_microseconds: 2020-09-11 10:08:17.134042 -query_start_time: 2020-09-11 10:08:17 -query_start_time_microseconds: 2020-09-11 10:08:17.063150 -query_duration_ms: 70 -read_rows: 0 -read_bytes: 0 -written_rows: 1 -written_bytes: 12 -memory_usage: 4300844 -peak_memory_usage: 4300844 -thread_name: TCPHandler -thread_id: 638133 -master_thread_id: 638133 -query: INSERT INTO test1 VALUES -is_initial_query: 1 -user: default -query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -address: ::ffff:127.0.0.1 -port: 33452 -initial_user: default -initial_query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -initial_address: ::ffff:127.0.0.1 -initial_port: 33452 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -revision: 54440 -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -``` - -**См. также** - -* [system.query_log](/operations/system-tables/query_log) — Описание системной таблицы `query_log`, которая содержит общую информацию о выполнении запросов. -* [system.query_views_log](/operations/system-tables/query_views_log) — Эта таблица содержит информацию о каждом представлении, задействованном при выполнении запроса. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md deleted file mode 100644 index 430e56274c2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о зависимых представлениях, выполняемых при выполнении запроса, например о типе представления или времени выполнения.' -keywords: ['system table', 'query_views_log'] -slug: /operations/system-tables/query_views_log -title: 'system.query_views_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_views_log {#systemquery_views_log} - - - -Содержит информацию о зависимых представлениях, выполняемых при выполнении запроса, например о типе представления или времени выполнения. - -Чтобы начать логирование: - -1. Настройте параметры в разделе [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log). -2. Установите [log_query_views](/operations/settings/settings#log_query_views) в 1. - -Период сброса данных задаётся параметром `flush_interval_milliseconds` в разделе серверных настроек [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log). Чтобы принудительно выполнить сброс, используйте запрос [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs). - -ClickHouse не удаляет данные из таблицы автоматически. Подробности см. во вводном разделе [Introduction](/operations/system-tables/overview#system-tables-introduction). - -Вы можете использовать настройку [log_queries_probability](/operations/settings/settings#log_queries_probability)) для уменьшения количества запросов, регистрируемых в таблице `query_views_log`. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата, когда произошло последнее событие представления. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время окончания выполнения представления. -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время окончания выполнения представления с точностью до микросекунд. -* `view_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Длительность выполнения представления (сумма его стадий) в миллисекундах. -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — ID исходного запроса (для распределённого выполнения запросов). -* `view_name` ([String](../../sql-reference/data-types/string.md)) — Имя представления. -* `view_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — UUID представления. -* `view_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип представления. Возможные значения: - * `'Default' = 1` — [Обычные представления](/sql-reference/statements/create/view#normal-view). Не должны появляться в этом журнале. - * `'Materialized' = 2` — [Материализованные представления](/sql-reference/statements/create/view#materialized-view). - * `'Live' = 3` — [Live-представления](../../sql-reference/statements/create/view.md#live-view). -* `view_query` ([String](../../sql-reference/data-types/string.md)) — Запрос, выполняемый представлением. -* `view_target` ([String](../../sql-reference/data-types/string.md)) — Имя целевой таблицы представления. -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество прочитанных строк. -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество прочитанных байт. -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество записанных строк. -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — Количество записанных байт. -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — Максимальная разница между объёмом выделенной и освобождённой памяти в контексте этого представления. -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — ProfileEvents, измеряющие различные метрики. Их описание можно найти в таблице [system.events](/operations/system-tables/events). -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — Статус представления. Возможные значения: - * `'QueryStart' = 1` — Успешный запуск выполнения представления. Не должен появляться. - * `'QueryFinish' = 2` — Успешное завершение выполнения представления. - * `'ExceptionBeforeStart' = 3` — Исключение до начала выполнения представления. - * `'ExceptionWhileProcessing' = 4` — Исключение во время выполнения представления. -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — Код исключения. -* `exception` ([String](../../sql-reference/data-types/string.md)) — Сообщение об исключении. -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [Трассировка стека](https://en.wikipedia.org/wiki/Stack_trace). Пустая строка, если запрос был успешно завершён. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.query_views_log LIMIT 1 \G; -``` - -Результат: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2021-06-22 -event_time: 2021-06-22 13:23:07 -event_time_microseconds: 2021-06-22 13:23:07.738221 -view_duration_ms: 0 -initial_query_id: c3a1ac02-9cad-479b-af54-9e9c0a7afd70 -view_name: default.matview_inner -view_uuid: 00000000-0000-0000-0000-000000000000 -view_type: Materialized -view_query: SELECT * FROM default.table_b -view_target: default.`.inner.matview_inner` -read_rows: 4 -read_bytes: 64 -written_rows: 2 -written_bytes: 32 -peak_memory_usage: 4196188 -ProfileEvents: {'FileOpen':2,'WriteBufferFromFileDescriptorWrite':2,'WriteBufferFromFileDescriptorWriteBytes':187,'IOBufferAllocs':3,'IOBufferAllocBytes':3145773,'FunctionExecute':3,'DiskWriteElapsedMicroseconds':13,'InsertedRows':2,'InsertedBytes':16,'SelectedRows':4,'SelectedBytes':48,'ContextLock':16,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':698,'SoftPageFaults':4,'OSReadChars':463} -status: QueryFinish -exception_code: 0 -exception: -stack_trace: -``` - -**См. также** - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md deleted file mode 100644 index a7cdfdd7310..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о максимальных значениях для всех интервалов всех квот. Одной квоте может соответствовать любое количество строк, включая ноль.' -keywords: ['системная таблица', 'quota_limits'] -slug: /operations/system-tables/quota_limits -title: 'system.quota_limits' -doc_type: 'reference' ---- - -# system.quota_limits {#systemquota_limits} - -Содержит информацию о максимальных значениях для всех интервалов всех квот. Одной квоте может соответствовать любое количество строк или ни одной. - -Столбцы: - -* `quota_name` ([String](../../sql-reference/data-types/string.md)) — Имя квоты. -* `duration` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Длительность временного интервала для расчёта потребления ресурсов, в секундах. -* `is_randomized_interval` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Логическое значение. Показывает, является ли интервал рандомизированным. Если интервал не рандомизирован, он всегда начинается в одно и то же время. Например, интервал в 1 минуту всегда начинается в целое количество минут (т. е. он может начаться в 11:20:00, но никогда не начинается в 11:20:01), интервал в один день всегда начинается в полночь по UTC. Если интервал рандомизирован, первый интервал начинается в случайный момент времени, а последующие интервалы идут один за другим. Значения: -* `0` — Интервал не рандомизирован. -* `1` — Интервал рандомизирован. -* `max_queries` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество запросов. -* `max_query_selects` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество запросов `SELECT`. -* `max_query_inserts` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество запросов `INSERT`. -* `max_errors` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество ошибок. -* `max_result_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество строк в результате. -* `max_result_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальный объём оперативной памяти в байтах, используемой для хранения результата запроса. -* `max_read_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество строк, прочитанных из всех таблиц и табличных функций, участвующих в запросах. -* `max_read_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Максимальное количество байт, прочитанных из всех таблиц и табличных функций, участвующих в запросах. -* `max_execution_time` ([Nullable](../../sql-reference/data-types/nullable.md)([Float64](../../sql-reference/data-types/float.md))) — Максимальное время выполнения запроса, в секундах. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md deleted file mode 100644 index 728c4334989..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об использовании квоты текущим пользователем, например о том, сколько квоты уже использовано и сколько осталось.' -keywords: ['системная таблица', 'quota_usage'] -slug: /operations/system-tables/quota_usage -title: 'system.quota_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -Использование квоты для текущего пользователя: сколько уже использовано и сколько осталось. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md deleted file mode 100644 index e16d1ebf037..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о квотах.' -keywords: ['системная таблица', 'квоты', 'квота'] -slug: /operations/system-tables/quotas -title: 'system.quotas' -doc_type: 'reference' ---- - - - -# system.quotas {#systemquotas} - -Содержит информацию о [квотах](../../operations/system-tables/quotas.md). - -Столбцы: -- `name` ([String](../../sql-reference/data-types/string.md)) — Имя квоты. -- `id` ([UUID](../../sql-reference/data-types/uuid.md)) — Идентификатор квоты. -- `storage`([String](../../sql-reference/data-types/string.md)) — Хранилище квот. Возможные значения: "users.xml", если квота настроена в файле users.xml; "disk", если квота настроена SQL-запросом. -- `keys` ([Array](../../sql-reference/data-types/array.md)([Enum8](../../sql-reference/data-types/enum.md))) — Ключ, определяющий, как должна распределяться квота. Если два соединения используют одну и ту же квоту и ключ, они делят один и тот же объём ресурсов. Значения: - - `[]` — Все пользователи используют одну и ту же квоту. - - `['user_name']` — Соединения с одинаковым именем пользователя используют одну и ту же квоту. - - `['ip_address']` — Соединения с одного и того же IP-адреса используют одну и ту же квоту. - - `['client_key']` — Соединения с одинаковым ключом используют одну и ту же квоту. Ключ должен быть явно передан клиентом. При использовании [clickhouse-client](../../interfaces/cli.md) передайте значение ключа в параметре `--quota_key` или используйте параметр `quota_key` в конфигурационном файле клиента. При использовании HTTP-интерфейса используйте заголовок `X-ClickHouse-Quota`. - - `['user_name', 'client_key']` — Соединения с одинаковым `client_key` используют одну и ту же квоту. Если ключ не передан клиентом, квота отслеживается по `user_name`. - - `['client_key', 'ip_address']` — Соединения с одинаковым `client_key` используют одну и ту же квоту. Если ключ не передан клиентом, квота отслеживается по `ip_address`. -- `durations` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Продолжительности временных интервалов в секундах. -- `apply_to_all` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Логическое значение. Показывает, к каким пользователям применяется квота. Значения: - - `0` — Квота применяется к пользователям, указанным в `apply_to_list`. - - `1` — Квота применяется ко всем пользователям, кроме перечисленных в `apply_to_except`. -- `apply_to_list` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — Список имён пользователей/[ролей](../../guides/sre/user-management/index.md#role-management), к которым должна применяться квота. -- `apply_to_except` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — Список имён пользователей/ролей, к которым квота не должна применяться. - - - -## Смотрите также {#see-also} - -- [SHOW QUOTAS](/sql-reference/statements/show#show-quotas) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md deleted file mode 100644 index b6ab61bb770..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об использовании квот всеми пользователями.' -keywords: ['system table', 'quotas_usage', 'quota'] -slug: /operations/system-tables/quotas_usage -title: 'system.quotas_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.quotas_usage {#systemquotas_usage} - - - -Использование квот по всем пользователям. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md deleted file mode 100644 index 28928034f09..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о реплицируемых таблицах и их статусе на локальном сервере. Полезна для мониторинга.' -keywords: ['системная таблица', 'реплики'] -slug: /operations/system-tables/replicas -title: 'system.replicas' -doc_type: 'reference' ---- - -# system.replicas {#systemreplicas} - -Содержит информацию и состояние реплицируемых таблиц, расположенных на локальном сервере. -Эту таблицу можно использовать для мониторинга. Таблица содержит по одной строке для каждой таблицы Replicated*. - -Пример: - -```sql -SELECT * -FROM system.replicas -WHERE table = 'test_table' -FORMAT Vertical -``` - -```text -Идентификатор запроса: dc6dcbcb-dc28-4df9-ae27-4354f5b3b13e - -Строка 1: -─────── -database: db -table: test_table -engine: ReplicatedMergeTree -is_leader: 1 -can_become_leader: 1 -is_readonly: 0 -is_session_expired: 0 -future_parts: 0 -parts_to_check: 0 -zookeeper_path: /test/test_table -replica_name: r1 -replica_path: /test/test_table/replicas/r1 -columns_version: -1 -queue_size: 27 -inserts_in_queue: 27 -merges_in_queue: 0 -part_mutations_in_queue: 0 -queue_oldest_time: 2021-10-12 14:48:48 -inserts_oldest_time: 2021-10-12 14:48:48 -merges_oldest_time: 1970-01-01 03:00:00 -part_mutations_oldest_time: 1970-01-01 03:00:00 -oldest_part_to_get: 1_17_17_0 -oldest_part_to_merge_to: -oldest_part_to_mutate_to: -log_max_index: 206 -log_pointer: 207 -last_queue_update: 2021-10-12 14:50:08 -absolute_delay: 99 -total_replicas: 5 -active_replicas: 5 -lost_part_count: 0 -last_queue_update_exception: -zookeeper_exception: -replica_is_active: {'r1':1,'r2':1} -``` - -Столбцы: - -* `database` (`String`) - Имя базы данных -* `table` (`String`) - Имя таблицы -* `engine` (`String`) - Имя движка таблицы -* `is_leader` (`UInt8`) - Является ли реплика лидером. - Одновременно лидерами могут быть несколько реплик. Реплике можно запретить становиться лидером с помощью настройки `merge_tree` `replicated_can_become_leader`. Лидеры отвечают за планирование фоновых слияний. - Обратите внимание, что запись может выполняться в любую реплику, которая доступна и имеет сессию в ZK, независимо от того, является ли она лидером. -* `can_become_leader` (`UInt8`) - Может ли реплика быть лидером. -* `is_readonly` (`UInt8`) - Находится ли реплика в режиме только для чтения. - Этот режим включается, если в конфигурации нет секций с ClickHouse Keeper, если при переинициализации сессий в ClickHouse Keeper произошла неизвестная ошибка, а также во время переинициализации сессий в ClickHouse Keeper. -* `is_session_expired` (`UInt8`) - Сессия с ClickHouse Keeper истекла. По сути то же самое, что `is_readonly`. -* `future_parts` (`UInt32`) - Количество частей данных, которые появятся в результате операций INSERT или слияний, которые еще не выполнены. -* `parts_to_check` (`UInt32`) - Количество частей данных в очереди на проверку. Часть помещается в очередь на проверку, если есть подозрение, что она может быть повреждена. -* `zookeeper_path` (`String`) - Путь к данным таблицы в ClickHouse Keeper. -* `replica_name` (`String`) - Имя реплики в ClickHouse Keeper. У разных реплик одной и той же таблицы разные имена. -* `replica_path` (`String`) - Путь к данным реплики в ClickHouse Keeper. То же, что и конкатенация 'zookeeper_path/replicas/replica_path'. -* `columns_version` (`Int32`) - Номер версии структуры таблицы. Показывает, сколько раз выполнялась операция ALTER. Если у реплик разные версии, это означает, что некоторые реплики еще не выполнили все операции ALTER. -* `queue_size` (`UInt32`) - Размер очереди операций, ожидающих выполнения. Операции включают вставку блоков данных, слияния и некоторые другие действия. Обычно совпадает со значением `future_parts`. -* `inserts_in_queue` (`UInt32`) - Количество вставок блоков данных, которые нужно выполнить. Вставки обычно реплицируются достаточно быстро. Если это число велико, значит что-то не так. -* `merges_in_queue` (`UInt32`) - Количество слияний, ожидающих выполнения. Иногда слияния длятся долго, поэтому это значение может оставаться больше нуля в течение длительного времени. -* `part_mutations_in_queue` (`UInt32`) - Количество мутаций, ожидающих выполнения. -* `queue_oldest_time` (`DateTime`) - Если `queue_size` больше 0, показывает, когда в очередь была добавлена самая старая операция. -* `inserts_oldest_time` (`DateTime`) - См. `queue_oldest_time`. -* `merges_oldest_time` (`DateTime`) - См. `queue_oldest_time`. -* `part_mutations_oldest_time` (`DateTime`) - См. `queue_oldest_time`. - -Следующие 4 столбца имеют ненулевое значение только там, где есть активная сессия с ZK. - -* `log_max_index` (`UInt64`) - Максимальный номер записи в логе общей активности. -* `log_pointer` (`UInt64`) - Максимальный номер записи в логе общей активности, который реплика скопировала в свою очередь выполнения, плюс один. Если `log_pointer` значительно меньше, чем `log_max_index`, это указывает на проблему. -* `last_queue_update` (`DateTime`) - Время последнего обновления очереди. -* `absolute_delay` (`UInt64`) - Величина задержки текущей реплики в секундах. -* `total_replicas` (`UInt8`) - Общее количество известных реплик этой таблицы. -* `active_replicas` (`UInt8`) - Количество реплик этой таблицы, которые имеют сессию в ClickHouse Keeper (т.е. количество работающих реплик). -* `lost_part_count` (`UInt64`) - Общее количество частей данных, потерянных всеми репликами таблицы с момента ее создания. Значение сохраняется в ClickHouse Keeper и может только увеличиваться. -* `last_queue_update_exception` (`String`) - Сообщение об исключении, возникающем, когда в очереди содержатся поврежденные записи. Особенно важно, когда ClickHouse нарушает обратную совместимость между версиями и записи лога, сделанные более новыми версиями, не могут быть разобраны старыми версиями. -* `zookeeper_exception` (`String`) - Сообщение о последнем исключении, полученное при возникновении ошибки во время получения информации из ClickHouse Keeper. -* `replica_is_active` ([Map(String, UInt8)](../../sql-reference/data-types/map.md)) — Отображение между именем реплики и признаком того, активна ли эта реплика. - -Если запросить все столбцы, таблица может работать немного медленнее, поскольку для каждой строки выполняется несколько чтений из ClickHouse Keeper. -Если не запрашивать последние 4 столбца (log_max_index, log_pointer, total_replicas, active_replicas), таблица работает быстро. - -Например, вы можете проверить, что всё работает корректно, следующим образом: - -```sql -SELECT - database, - table, - is_leader, - is_readonly, - is_session_expired, - future_parts, - parts_to_check, - columns_version, - queue_size, - inserts_in_queue, - merges_in_queue, - log_max_index, - log_pointer, - total_replicas, - active_replicas -FROM system.replicas -WHERE - is_readonly - OR is_session_expired - OR future_parts > 20 - OR parts_to_check > 10 - OR queue_size > 20 - OR inserts_in_queue > 10 - OR log_max_index - log_pointer > 10 - OR total_replicas < 2 - OR active_replicas < total_replicas -``` - -Если этот запрос не возвращает результатов, значит всё в порядке. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md deleted file mode 100644 index 45d366e40eb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о текущих фоновых операциях выборки данных.' -keywords: ['системная таблица', 'replicated_fetches'] -slug: /operations/system-tables/replicated_fetches -title: 'system.replicated_fetches' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.replicated_fetches {#systemreplicated_fetches} - - - -Содержит информацию о текущих фоновых операциях выборки данных. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.replicated_fetches LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: t -elapsed: 7.243039876 -progress: 0.41832135995612835 -result_part_name: all_0_0_0 -result_part_path: /var/lib/clickhouse/store/700/70080a04-b2de-4adf-9fa5-9ea210e81766/all_0_0_0/ -partition_id: all -total_size_bytes_compressed: 1052783726 -bytes_read_compressed: 440401920 -source_replica_path: /clickhouse/test/t/replicas/1 -source_replica_hostname: node1 -source_replica_port: 9009 -interserver_scheme: http -URI: http://node1:9009/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftest%2Ft%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=4&compress=false -to_detached: 0 -thread_id: 54 -``` - -**См. также** - -* [Управление таблицами ReplicatedMergeTree](../../sql-reference/statements/system.md/#managing-replicatedmergetree-tables) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md deleted file mode 100644 index dc0be20444a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о заданиях в очередях репликации, - хранящихся в ClickHouse Keeper или ZooKeeper для таблиц семейства `ReplicatedMergeTree`.' -keywords: ['системная таблица', 'replication_queue'] -slug: /operations/system-tables/replication_queue -title: 'system.replication_queue' -doc_type: 'reference' ---- - -# system.replication_queue {#systemreplication_queue} - -Содержит информацию о задачах из очередей репликации, хранящихся в ClickHouse Keeper или ZooKeeper, для таблиц семейства `ReplicatedMergeTree`. - -Столбцы: - -* `database` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных. - -* `table` ([String](../../sql-reference/data-types/string.md)) — Имя таблицы. - -* `replica_name` ([String](../../sql-reference/data-types/string.md)) — Имя реплики в ClickHouse Keeper. У разных реплик одной и той же таблицы разные имена. - -* `position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Позиция задачи в очереди. - -* `node_name` ([String](../../sql-reference/data-types/string.md)) — Имя узла в ClickHouse Keeper. - -* `type` ([String](../../sql-reference/data-types/string.md)) — Тип задачи в очереди, один из: - - * `GET_PART` — Получить часть с другой реплики. - * `ATTACH_PART` — Подключить часть, возможно, с собственной реплики (если найдена в папке `detached`). Можно рассматривать это как `GET_PART` с некоторыми оптимизациями, так как они почти идентичны. - * `MERGE_PARTS` — Слить части. - * `DROP_RANGE` — Удалить части в указанном разделе в заданном диапазоне номеров. - * `CLEAR_COLUMN` — ПРИМЕЧАНИЕ: Устарело. Удалить указанный столбец из заданного раздела. - * `CLEAR_INDEX` — ПРИМЕЧАНИЕ: Устарело. Удалить указанный индекс из заданного раздела. - * `REPLACE_RANGE` — Удалить определённый диапазон частей и заменить их новыми. - * `MUTATE_PART` — Применить одну или несколько мутаций к части. - * `ALTER_METADATA` — Применить изменение в соответствии с глобальными путями /metadata и /columns. - -* `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время, когда задача была отправлена на выполнение. - -* `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Число реплик, ожидающих завершения задачи с подтверждением выполнения. Этот столбец имеет смысл только для задачи `GET_PARTS`. - -* `source_replica` ([String](../../sql-reference/data-types/string.md)) — Имя исходной реплики. - -* `new_part_name` ([String](../../sql-reference/data-types/string.md)) — Имя новой части. - -* `parts_to_merge` ([Array](../../sql-reference/data-types/array.md) ([String](../../sql-reference/data-types/string.md))) — Имена частей для слияния или обновления. - -* `is_detach` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Флаг, указывающий, находится ли задача `DETACH_PARTS` в очереди. - -* `is_currently_executing` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Флаг, указывающий, выполняется ли данная задача в текущий момент. - -* `num_tries` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Число неудачных попыток выполнить задачу. - -* `last_exception` ([String](../../sql-reference/data-types/string.md)) — Текстовое сообщение о последней возникшей ошибке (если была). - -* `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время последней попытки выполнения задачи. - -* `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Количество раз, когда действие откладывалось. - -* `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — Причина, по которой задача была отложена. - -* `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Дата и время последнего откладывания задачи. - -* `merge_type` ([String](../../sql-reference/data-types/string.md)) — Тип текущего слияния. Пусто, если это мутация. - -**Пример** - -```sql -SELECT * FROM system.replication_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: merge -table: visits_v2 -replica_name: mtgiga001-1t -position: 15 -node_name: queue-0009325559 -type: MERGE_PARTS -create_time: 2020-12-07 14:04:21 -required_quorum: 0 -source_replica: mtgiga001-1t -new_part_name: 20201130_121373_121384_2 -parts_to_merge: ['20201130_121373_121378_1','20201130_121379_121379_0','20201130_121380_121380_0','20201130_121381_121381_0','20201130_121382_121382_0','20201130_121383_121383_0','20201130_121384_121384_0'] -is_detach: 0 -is_currently_executing: 0 -num_tries: 36 -last_exception: Code: 226, e.displayText() = DB::Exception: Файл отметок '/opt/clickhouse/data/merge/visits_v2/tmp_fetch_20201130_121373_121384_2/CounterID.mrk' не существует (version 20.8.7.15 (official build)) -last_attempt_time: 2020-12-08 17:35:54 -num_postponed: 0 -postpone_reason: -last_postpone_time: 1970-01-01 03:00:00 -``` - -**См. также** - -* [Управление таблицами ReplicatedMergeTree](/sql-reference/statements/system#managing-replicatedmergetree-tables) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md deleted file mode 100644 index 49ed815e933..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о ресурсах, находящихся на - локальном сервере; по одной строке на каждый ресурс.' -keywords: ['системная таблица', 'ресурсы'] -slug: /operations/system-tables/resources -title: 'system.resources' -doc_type: 'reference' ---- - -# system.resources {#systemresources} - -Содержит информацию о [ресурсах](/operations/workload-scheduling.md#workload_entity_storage), размещённых на локальном сервере. В таблице — по одной строке на каждый ресурс. - -Пример: - -```sql -SELECT * -FROM system.resources -FORMAT Vertical -``` - -```text -Строка 1: -────── -name: io_read -read_disks: ['s3'] -write_disks: [] -create_query: CREATE RESOURCE io_read (READ DISK s3) - -Строка 2: -────── -name: io_write -read_disks: [] -write_disks: ['s3'] -create_query: CREATE RESOURCE io_write (WRITE DISK s3) -``` - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md deleted file mode 100644 index 5ae55fda554..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о назначенных пользователям и ролям ролях.' -keywords: ['системная таблица', 'role_grants'] -slug: /operations/system-tables/role_grants -title: 'system.role_grants' -doc_type: 'reference' ---- - -# system.role_grants {#systemrole_grants} - -Содержит назначения ролей для пользователей и ролей. Чтобы добавить записи в эту таблицу, используйте `GRANT role TO user`. - -Столбцы: - -* `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя пользователя. - -* `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Имя роли. - -* `granted_role_name` ([String](../../sql-reference/data-types/string.md)) — Имя роли, назначенной роли `role_name`. Чтобы выдать одну роль другой, используйте `GRANT role1 TO role2`. - -* `granted_role_is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, является ли `granted_role` ролью по умолчанию. Возможные значения: - * 1 — `granted_role` является ролью по умолчанию. - * 0 — `granted_role` не является ролью по умолчанию. - -* `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Флаг, показывающий, является ли `granted_role` ролью с привилегией [ADMIN OPTION](/sql-reference/statements/grant#admin-option). Возможные значения: - * 1 — У роли есть привилегия `ADMIN OPTION`. - * 0 — У роли нет привилегии `ADMIN OPTION`. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md deleted file mode 100644 index ccc2d968a95..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о сконфигурированных ролях.' -keywords: ['системная таблица', 'роли'] -slug: /operations/system-tables/roles -title: 'system.roles' -doc_type: 'reference' ---- - -# system.roles {#systemroles} - -Содержит информацию о настроенных [ролях](../../guides/sre/user-management/index.md#role-management). - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW ROLES](/sql-reference/statements/show#show-roles) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md deleted file mode 100644 index ab1d4ba13a2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -description: 'Системная таблица, содержащая фильтры для одной таблицы, а также - список ролей и/или пользователей, для которых применяется эта политика на уровне строк.' -keywords: ['системная таблица', 'row_policies'] -slug: /operations/system-tables/row_policies -title: 'system.row_policies' -doc_type: 'reference' ---- - -# system.row_policies {#systemrow_policies} - -Содержит фильтры для одной таблицы, а также список ролей и/или пользователей, для которых применяется эта политика строк. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW POLICIES](/sql-reference/statements/show#show-policies) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md deleted file mode 100644 index 08da56a066d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md +++ /dev/null @@ -1,18 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о настройках таблиц S3Queue. - Доступна, начиная с версии сервера `24.10`.' -keywords: ['системная таблица', 's3_queue_settings'] -slug: /operations/system-tables/s3_queue_settings -title: 'system.s3_queue_settings' -doc_type: 'reference' ---- - -# system.s3_queue_settings {#systems3_queue_settings} - -Содержит сведения о настройках таблиц [S3Queue](../../engines/table-engines/integrations/s3queue.md). Эта системная таблица доступна начиная с версии сервера `24.10`. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md deleted file mode 100644 index c3588d6d2fe..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о состоянии узлов планировщика, находящихся на локальном сервере.' -keywords: ['системная таблица', 'планировщик'] -slug: /operations/system-tables/scheduler -title: 'system.scheduler' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.scheduler {#systemscheduler} - - - -Содержит сведения об [узлах планировщика](/operations/workload-scheduling.md/#hierarchy) на локальном сервере и их состоянии. -Эту таблицу можно использовать для мониторинга. Таблица содержит по одной строке на каждый узел планировщика. - -Пример: - -```sql -SELECT * -FROM system.scheduler -WHERE resource = 'network_read' AND path = '/prio/fair/prod' -FORMAT Vertical -``` - -```text -Row 1: -────── -resource: network_read -path: /prio/fair/prod -type: fifo -weight: 5 -priority: 0 -is_active: 0 -active_children: 0 -dequeued_requests: 67 -canceled_requests: 0 -dequeued_cost: 4692272 -canceled_cost: 0 -busy_periods: 63 -vruntime: 938454.1999999989 -system_vruntime: ᴺᵁᴸᴸ -queue_length: 0 -queue_cost: 0 -budget: -60524 -is_satisfied: ᴺᵁᴸᴸ -inflight_requests: ᴺᵁᴸᴸ -inflight_cost: ᴺᵁᴸᴸ -max_requests: ᴺᵁᴸᴸ -max_cost: ᴺᵁᴸᴸ -max_speed: ᴺᵁᴸᴸ -max_burst: ᴺᵁᴸᴸ -throttling_us: ᴺᵁᴸᴸ -tokens: ᴺᵁᴸᴸ -``` - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md deleted file mode 100644 index 92a4ff92e32..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию обо всех кэшированных схемах файлов.' -keywords: ['системная таблица', 'schema_inference_cache'] -slug: /operations/system-tables/schema_inference_cache -title: 'system.schema_inference_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.schema_inference_cache {#systemschema_inference_cache} - - - -Содержит информацию обо всех кэшированных схемах файлов. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -Предположим, у нас есть файл `data.jsonl` со следующим содержимым: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["футбол", "кулинария", "музыка"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["теннис", "искусство"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["фитнес", "чтение", "шопинг"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["кино", "прыжки с парашютом"]} -``` - -:::tip -Поместите `data.jsonl` в каталог, указанный в параметре `user_files_path`. Найти его можно в файлах конфигурации ClickHouse. Значение по умолчанию: - -```sql -/var/lib/clickhouse/user_files/ -``` - -::: - -Откройте `clickhouse-client` и выполните запрос `DESCRIBE`: - -```sql -DESCRIBE file('data.jsonl') SETTINGS input_format_try_infer_integers=0; -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Float64) │ │ │ │ │ │ -│ age │ Nullable(Float64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Давайте посмотрим на содержимое таблицы `system.schema_inference_cache`: - -```sql -SELECT * -FROM system.schema_inference_cache -FORMAT Vertical -``` - -```response -Строка 1: -────── -storage: File -source: /home/droscigno/user_files/data.jsonl -format: JSONEachRow -additional_format_info: schema_inference_hints=, max_rows_to_read_for_schema_inference=25000, schema_inference_make_columns_nullable=true, try_infer_integers=false, try_infer_dates=true, try_infer_datetimes=true, try_infer_numbers_from_strings=true, read_bools_as_numbers=true, try_infer_objects=false -registration_time: 2022-12-29 17:49:52 -schema: id Nullable(Float64), age Nullable(Float64), name Nullable(String), hobbies Array(Nullable(String)) -``` - -**См. также** - -* [Автоматический вывод схемы по входным данным](/interfaces/schema-inference.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md deleted file mode 100644 index d2d7861d558..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о глобальных настройках сервера, заданных в `config.xml`.' -keywords: ['system table', 'server_settings'] -slug: /operations/system-tables/server_settings -title: 'system.server_settings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.server_settings {#systemserver_settings} - - - -Содержит информацию о глобальных настройках сервера, которые задаются в `config.xml`. -В настоящее время таблица показывает только настройки из первого уровня `config.xml` и не поддерживает вложенные конфигурации (например, [logger](../../operations/server-configuration-parameters/settings.md#logger)). - -Столбцы: - -* `name` ([String](../../sql-reference/data-types/string.md)) — Имя настройки сервера. -* `value` ([String](../../sql-reference/data-types/string.md)) — Значение настройки сервера. -* `default` ([String](../../sql-reference/data-types/string.md)) — Значение настройки сервера по умолчанию. -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Показывает, была ли настройка задана в `config.xml`. -* `description` ([String](../../sql-reference/data-types/string.md)) — Краткое описание настройки сервера. -* `type` ([String](../../sql-reference/data-types/string.md)) — Тип значения настройки сервера. -* `changeable_without_restart` ([Enum8](../../sql-reference/data-types/enum.md)) — Можно ли изменить настройку во время работы сервера. Возможные значения: - * `'No' ` - * `'IncreaseOnly'` - * `'DecreaseOnly'` - * `'Yes'` -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Показывает, является ли настройка устаревшей. - -**Пример** - -Следующий пример показывает, как получить информацию о настройках сервера, имя которых содержит `thread_pool`. - -```sql -SELECT * -FROM system.server_settings -WHERE name LIKE '%thread_pool%' -``` - -```text -┌─name──────────────────────────────────────────┬─value─┬─default─┬─changed─┬─description─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─type───┬─changeable_without_restart─┬─is_obsolete─┐ -│ max_thread_pool_size │ 10000 │ 10000 │ 0 │ Максимальное количество потоков, которые могут быть выделены ОС и использованы для выполнения запросов и фоновых операций. │ UInt64 │ No │ 0 │ -│ max_thread_pool_free_size │ 1000 │ 1000 │ 0 │ Максимальное количество потоков, которые после выделения всегда остаются в глобальном пуле потоков и простаивают при недостаточном количестве задач. │ UInt64 │ No │ 0 │ -│ thread_pool_queue_size │ 10000 │ 10000 │ 0 │ Максимальное количество задач, которые будут помещены в очередь и ожидать выполнения. │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_size │ 100 │ 100 │ 0 │ Максимальное количество потоков, которые будут использоваться для операций ввода-вывода │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_free_size │ 0 │ 0 │ 0 │ Максимальное количество свободных потоков в пуле потоков ввода-вывода. │ UInt64 │ No │ 0 │ -│ io_thread_pool_queue_size │ 10000 │ 10000 │ 0 │ Размер очереди пула потоков ввода-вывода. │ UInt64 │ No │ 0 │ -│ max_active_parts_loading_thread_pool_size │ 64 │ 64 │ 0 │ Количество потоков для загрузки активного набора частей данных при запуске. │ UInt64 │ No │ 0 │ -│ max_outdated_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ Количество потоков для загрузки неактивного набора частей данных (устаревших) при запуске. │ UInt64 │ No │ 0 │ -│ max_unexpected_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ Количество потоков для загрузки неактивного набора частей данных (неожиданных) при запуске. │ UInt64 │ No │ 0 │ -│ max_parts_cleaning_thread_pool_size │ 128 │ 128 │ 0 │ Количество потоков для параллельного удаления неактивных частей данных. │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_size │ 1000 │ 1000 │ 0 │ Максимальное количество потоков, которые будут использоваться для операций ввода-вывода при выполнении запросов BACKUP │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_free_size │ 0 │ 0 │ 0 │ Максимальное количество свободных потоков в пуле потоков ввода-вывода для резервных копий. │ UInt64 │ No │ 0 │ -│ backups_io_thread_pool_queue_size │ 0 │ 0 │ 0 │ Размер очереди пула потоков ввода-вывода для резервных копий. │ UInt64 │ No │ 0 │ -└───────────────────────────────────────────────┴───────┴─────────┴─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴────────────────────────────┴─────────────┘ - -``` - -Использование `WHERE changed` может быть полезно, например, когда вы хотите проверить, -правильно ли загружены настройки из конфигурационных файлов и действительно ли они применяются. - -{/* */ } - -```sql -SELECT * FROM system.server_settings WHERE changed AND name='max_thread_pool_size' -``` - -**См. также** - -* [Настройки](../../operations/system-tables/settings.md) -* [Файлы конфигурации](../../operations/configuration-files.md) -* [Настройки сервера](../../operations/server-configuration-parameters/settings.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md deleted file mode 100644 index b7d122c587e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию обо всех успешных и неуспешных входах в систему и выходах из неё.' -keywords: ['system table', 'session_log'] -slug: /operations/system-tables/session_log -title: 'system.session_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.session_log {#systemsession_log} - - - -Содержит информацию обо всех успешных и неуспешных событиях входа в систему и выхода из неё. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — Результат входа/выхода. Возможные значения: - * `LoginFailure` — Ошибка входа. - * `LoginSuccess` — Успешный вход. - * `Logout` — Выход из системы. -* `auth_id` ([UUID](../../sql-reference/data-types/uuid.md)) — Идентификатор аутентификации (UUID), который автоматически генерируется при каждом входе пользователя в систему. -* `session_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор сеанса, который передаётся клиентом через интерфейс [HTTP](../../interfaces/http.md). -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата входа/выхода. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время входа/выхода. -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Время начала входа/выхода с точностью до микросекунд. -* `user` ([String](../../sql-reference/data-types/string.md)) — Имя пользователя. -* `auth_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип аутентификации. Возможные значения: - * `NO_PASSWORD` - * `PLAINTEXT_PASSWORD` - * `SHA256_PASSWORD` - * `DOUBLE_SHA1_PASSWORD` - * `LDAP` - * `KERBEROS` - * `SSL_CERTIFICATE` -* `profiles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Список профилей, заданных для всех ролей и/или пользователей. -* `roles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — Список ролей, для которых применяется профиль. -* `settings` ([Array](../../sql-reference/data-types/array.md)([Tuple](../../sql-reference/data-types/tuple.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md), [String](../../sql-reference/data-types/string.md)))) — Настройки, которые были изменены при входе/выходе клиента. -* `client_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес, с которого выполнялся вход/выход. -* `client_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — Порт клиента, с которого выполнялся вход/выход. -* `interface` ([Enum8](../../sql-reference/data-types/enum.md)) — Интерфейс, через который был инициирован вход. Возможные значения: - * `TCP` - * `HTTP` - * `gRPC` - * `MySQL` - * `PostgreSQL` -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — Имя хоста клиентской машины, на которой запущен [clickhouse-client](../../interfaces/cli.md) или другой TCP‑клиент. -* `client_name` ([String](../../sql-reference/data-types/string.md)) — Имя `clickhouse-client` или другого TCP‑клиента. -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Ревизия `clickhouse-client` или другого TCP‑клиента. -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Мажорная версия `clickhouse-client` или другого TCP‑клиента. -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Минорная версия `clickhouse-client` или другого TCP‑клиента. -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Патч‑версия `clickhouse-client` или другого TCP‑клиента. -* `failure_reason` ([String](../../sql-reference/data-types/string.md)) — Сообщение исключения, содержащее причину ошибки входа/выхода. - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.session_log LIMIT 1 FORMAT Vertical; -``` - -Результат: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: LoginSuccess -auth_id: 45e6bd83-b4aa-4a23-85e6-bd83b4aa1a23 -session_id: -event_date: 2021-10-14 -event_time: 2021-10-14 20:33:52 -event_time_microseconds: 2021-10-14 20:33:52.104247 -user: default -auth_type: PLAINTEXT_PASSWORD -profiles: ['default'] -roles: [] -settings: [('load_balancing','random'),('max_memory_usage','10000000000')] -client_address: ::ffff:127.0.0.1 -client_port: 38490 -interface: TCP -client_hostname: -client_name: ClickHouse client -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 0 -failure_reason: -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md deleted file mode 100644 index 250a7a438bd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о настройках сеанса для текущего пользователя.' -keywords: ['системная таблица', 'настройки'] -slug: /operations/system-tables/settings -title: 'system.settings' -doc_type: 'reference' ---- - -# system.settings {#systemsettings} - -Содержит информацию о настройках сессии для текущего пользователя. - -Столбцы: - -* `name` ([String](../../sql-reference/data-types/string.md)) — Имя настройки. -* `value` ([String](../../sql-reference/data-types/string.md)) — Значение настройки. -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Показывает, была ли настройка явно задана в конфигурации или явно изменена. -* `description` ([String](../../sql-reference/data-types/string.md)) — Краткое описание настройки. -* `min` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Минимальное значение настройки, если оно задано через [constraints](/operations/settings/constraints-on-settings). Если для настройки не задано минимальное значение, содержит [NULL](/operations/settings/formats#input_format_null_as_default). -* `max` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — Максимальное значение настройки, если оно задано через [constraints](/operations/settings/constraints-on-settings). Если для настройки не задано максимальное значение, содержит [NULL](/operations/settings/formats#input_format_null_as_default). -* `disallowed_values` ([Array](/sql-reference/data-types/array)([String](../../sql-reference/data-types/string.md))) — Список недопустимых значений. -* `readonly` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Показывает, может ли текущий пользователь изменять настройку: - * `0` — Текущий пользователь может изменить настройку. - * `1` — Текущий пользователь не может изменить настройку. -* `default` ([String](../../sql-reference/data-types/string.md)) — Значение настройки по умолчанию. -* `alias_for` ([String](../../sql-reference/data-types/string.md)) — Имя исходной настройки, если данная настройка является псевдонимом другой настройки. -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — Показывает, является ли настройка устаревшей. -* `tier` ([Enum8](../../sql-reference/data-types/enum.md)) — Уровень поддержки этой возможности. Возможности ClickHouse организованы по уровням, которые различаются в зависимости от текущего статуса их разработки и ожидаемого поведения при их использовании. Значения: - * `'Production'` — Функция стабильна, безопасна в использовании и не имеет проблем во взаимодействии с другими **production**‑функциями. - * `'Beta'` — Функция стабильна и безопасна. Результат её совместного использования с другими функциями неизвестен и корректность не гарантируется. Тестирование и отчёты приветствуются. - * `'Experimental'` — Функция находится в разработке. Предназначена только для разработчиков и энтузиастов ClickHouse. Функция может как работать, так и не работать и может быть удалена в любой момент. - * `'Obsolete'` — Больше не поддерживается. Либо уже удалена, либо будет удалена в будущих релизах. - -**Пример** - -В следующем примере показано, как получить информацию о настройках, имя которых содержит `min_i`. - -```sql -SELECT * -FROM system.settings -WHERE name LIKE '%min_insert_block_size_%' -FORMAT Vertical -``` - -```text -Row 1: -────── -name: min_insert_block_size_rows -value: 1048449 -changed: 0 -description: Задаёт минимальное количество строк в блоке, который может быть вставлен в таблицу запросом `INSERT`. Блоки меньшего размера объединяются в более крупные. - -Возможные значения: - -- Положительное целое число. -- 0 — объединение отключено. -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 1048449 -alias_for: -is_obsolete: 0 -tier: Production - -Row 2: -────── -name: min_insert_block_size_bytes -value: 268402944 -changed: 0 -description: Задаёт минимальное количество байтов в блоке, который может быть вставлен в таблицу запросом `INSERT`. Блоки меньшего размера объединяются в более крупные. - -Возможные значения: - -- Положительное целое число. -- 0 — объединение отключено. -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 268402944 -alias_for: -is_obsolete: 0 -tier: Production -``` - -Row 3: -────── -name: min_insert_block_size_rows_for_materialized_views -value: 0 -changed: 0 -description: Задает минимальное количество строк в блоке, которое может быть вставлено в таблицу запросом `INSERT`. Блоки меньшего размера объединяются в более крупные. Этот параметр применяется только к блокам, вставляемым в [материализованное представление](../../sql-reference/statements/create/view.md). Настраивая этот параметр, вы управляете объединением блоков при записи в материализованное представление и избегаете чрезмерного потребления памяти. - -Possible values: - -* Любое положительное целое число. -* 0 — объединение отключено. - -**See Also** - -* [min_insert_block_size_rows](/operations/settings/settings#min_insert_block_size_rows) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -Row 4: -────── -name: min_insert_block_size_bytes_for_materialized_views -value: 0 -changed: 0 -description: Задает минимальное количество байт в блоке, которое может быть вставлено в таблицу запросом `INSERT`. Блоки меньшего размера объединяются в более крупные. Этот параметр применяется только к блокам, вставляемым в [материализованное представление](../../sql-reference/statements/create/view.md). Настраивая этот параметр, вы управляете объединением блоков при записи в материализованное представление и избегаете чрезмерного потребления памяти. - -Possible values: - -* Любое положительное целое число. -* 0 — объединение отключено. - -**See also** - -* [min_insert_block_size_bytes](/operations/settings/settings#min_insert_block_size_bytes) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -```` - -Использование `WHERE changed` может быть полезно, например, для проверки: - -- Корректно ли загружены настройки из конфигурационных файлов и применяются ли они. -- Настроек, которые изменились в текущей сессии. - - - -```sql -SELECT * FROM system.settings WHERE changed AND name='load_balancing' -```` - -**См. также** - -* [Настройки](/operations/system-tables/overview#system-tables-introduction) -* [Права на выполнение запросов](/operations/settings/permissions-for-queries) -* [Ограничения для настроек](../../operations/settings/constraints-on-settings.md) -* Оператор [SHOW SETTINGS](../../sql-reference/statements/show.md#show-settings) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md deleted file mode 100644 index 14f4d3de521..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об изменениях настроек в предыдущих версиях ClickHouse.' -keywords: ['системная таблица', 'settings_changes'] -slug: /operations/system-tables/settings_changes -title: 'system.settings_changes' -doc_type: 'reference' ---- - -# system.settings_changes {#systemsettings_changes} - -Содержит информацию об изменениях настроек в предыдущих версиях ClickHouse. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * -FROM system.settings_changes -WHERE version = '23.5' -FORMAT Vertical -``` - -```text -Row 1: -────── -type: Core -version: 23.5 -changes: [('input_format_parquet_preserve_order','1','0','Позволяет читателю Parquet переупорядочивать строки для улучшения параллелизма.'),('parallelize_output_from_storages','0','1','Позволяет использовать параллелизм при выполнении запросов, читающих данные из file/url/s3/и т. д. Может привести к переупорядочиванию строк.'),('use_with_fill_by_sorting_prefix','0','1','Столбцы, предшествующие столбцам WITH FILL в предложении ORDER BY, формируют префикс сортировки. Строки с различными значениями в префиксе сортировки заполняются независимо'),('output_format_parquet_compliant_nested_types','0','1','Изменяет внутреннее имя поля в схеме выходного файла Parquet.')] -``` - -**См. также** - -* [Настройки](/operations/system-tables/overview#system-tables-introduction) -* [system.settings](settings.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md deleted file mode 100644 index a9d3c600a86..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Системная таблица, описывающая содержимое профиля настроек: ограничения, - роли и пользователей, к которым применяются эти настройки, а также родительские профили настроек.' -keywords: ['системная таблица', 'settings_profile_elements'] -slug: /operations/system-tables/settings_profile_elements -title: 'system.settings_profile_elements' -doc_type: 'reference' ---- - -# system.settings_profile_elements {#systemsettings_profile_elements} - -Описывает содержимое профиля настроек: - -* Ограничения. -* Роли и пользователи, к которым применяется настройка. -* Родительские профили настроек. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md deleted file mode 100644 index 75721e0512e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Системная таблица, содержащая свойства настроенных профилей настроек.' -keywords: ['системная таблица', 'settings_profiles'] -slug: /operations/system-tables/settings_profiles -title: 'system.settings_profiles' -doc_type: 'reference' ---- - -# system.settings_profiles {#systemsettings_profiles} - -Содержит свойства сконфигурированных профилей настроек. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW PROFILES](/sql-reference/statements/show#show-profiles) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md deleted file mode 100644 index 8bd11035a7b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -description: 'Системная таблица, содержащая стеки вызовов всех потоков сервера. Позволяет разработчикам анализировать состояние сервера.' -keywords: ['system table', 'stack_trace'] -slug: /operations/system-tables/stack_trace -title: 'system.stack_trace' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.stack_trace {#systemstack_trace} - - - -Содержит стек-трейсы всех потоков сервера. Позволяет разработчикам исследовать состояние сервера. - -Для анализа кадров стека используйте функции интроспекции `addressToLine`, `addressToLineWithInlines`, `addressToSymbol` и `demangle` ([функции интроспекции](../../sql-reference/functions/introspection.md)). - -Столбцы: - -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — Имя потока. -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Идентификатор потока. -* `query_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор запроса, который можно использовать для получения подробной информации о выполнявшемся запросе из системной таблицы [query_log](../system-tables/query_log.md). -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — [Стек-трейс](https://en.wikipedia.org/wiki/Stack_trace), представляющий собой список физических адресов, по которым расположены вызываемые методы. - -:::tip -Ознакомьтесь с базой знаний для примеров полезных запросов, включая [как увидеть, какие потоки выполняются в данный момент](/knowledgebase/find-expensive-queries) и [полезные запросы для устранения неполадок](/knowledgebase/useful-queries-for-troubleshooting). -::: - -**Пример** - -Включение функций интроспекции: - -```sql -SET allow_introspection_functions = 1; -``` - -Получение символов из объектных файлов ClickHouse: - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), trace) AS all SELECT thread_name, thread_id, query_id, arrayStringConcat(all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Строка 1: -────── -thread_name: QueryPipelineEx -thread_id: 743490 -query_id: dc55a564-febb-4e37-95bb-090ef182c6f1 -res: memcpy -large_ralloc -arena_ralloc -do_rallocx -Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) -HashTable, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>::resize(unsigned long, unsigned long) -void DB::Aggregator::executeImplBatch, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>>(DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>&, DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>::State&, DB::Arena*, unsigned long, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const -DB::Aggregator::executeImpl(DB::AggregatedDataVariants&, unsigned long, unsigned long, std::__1::vector>&, DB::Aggregator::AggregateFunctionInstruction*, bool, bool, char*) const -DB::Aggregator::executeOnBlock(std::__1::vector::immutable_ptr, std::__1::allocator::immutable_ptr>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::__1::vector>&, std::__1::vector>, std::__1::allocator>>>&, bool&) const -DB::AggregatingTransform::work() -DB::ExecutionThreadContext::executeTask() -DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic*) -void std::__1::__function::__policy_invoker::__call_impl>(std::__1::__function::__policy_storage const*) -ThreadPoolImpl>::worker(std::__1::__list_iterator, void*>) -void std::__1::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) -void* std::__1::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>>(void*) -``` - -Получение имён файлов и номеров строк в исходном коде ClickHouse: - -```sql -WITH arrayMap(x -> addressToLine(x), trace) AS all, arrayFilter(x -> x LIKE '%/dbms/%', all) AS dbms SELECT thread_name, thread_id, query_id, arrayStringConcat(notEmpty(dbms) ? dbms : all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Строка 1: -────── -thread_name: clickhouse-serv - -thread_id: 686 -query_id: cad353e7-1c29-4b2e-949f-93e597ab7a54 -res: /lib/x86_64-linux-gnu/libc-2.27.so -/build/obj-x86_64-linux-gnu/../src/Storages/System/StorageSystemStackTrace.cpp:182 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:656 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:1338 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:751 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:224 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:192 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:384 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:643 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:251 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1197 -/build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223 -/lib/x86_64-linux-gnu/libpthread-2.27.so -/lib/x86_64-linux-gnu/libc-2.27.so -``` - -**См. также** - -* [Introspection Functions](../../sql-reference/functions/introspection.md) — Какие функции интроспекции доступны и как ими пользоваться. -* [system.trace_log](../system-tables/trace_log.md) — Содержит трассировки стека, собранные профилировщиком запросов с выборкой. -* [arrayMap](/sql-reference/functions/array-functions#arrayMap) — Описание и пример использования функции `arrayMap`. -* [arrayFilter](/sql-reference/functions/array-functions#arrayFilter) — Описание и пример использования функции `arrayFilter`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md deleted file mode 100644 index 8275a14a205..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о политиках хранения и томах, - определённых в конфигурации сервера.' -keywords: ['системная таблица', 'storage_policies'] -slug: /operations/system-tables/storage_policies -title: 'system.storage_policies' -doc_type: 'reference' ---- - -# system.storage_policies {#systemstorage_policies} - -Содержит информацию о политиках хранения и томах, которые определены в [конфигурации сервера](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure). - -Столбцы: - -* `policy_name` ([String](../../sql-reference/data-types/string.md)) — Имя политики хранения. -* `volume_name` ([String](../../sql-reference/data-types/string.md)) — Имя тома, определённого в политике хранения. -* `volume_priority` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Порядковый номер тома в конфигурации, данные заполняют тома в соответствии с этим приоритетом, т.е. данные во время вставок и слияний записываются на тома с более низким значением приоритета (с учётом других правил: TTL, `max_data_part_size`, `move_factor`). -* `disks` ([Array(String)](../../sql-reference/data-types/array.md)) — Имена дисков, определённых в политике хранения. -* `volume_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип тома. Может принимать одно из следующих значений: - * `JBOD` - * `SINGLE_DISK` - * `UNKNOWN` -* `max_data_part_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Максимальный размер части данных, которая может быть сохранена на дисках тома (0 — без ограничений). -* `move_factor` ([Float64](../../sql-reference/data-types/float.md)) — Доля свободного дискового пространства. Когда эта доля превышает значение параметра конфигурации, ClickHouse начинает перемещать данные на следующий том по порядку. -* `prefer_not_to_merge` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Значение настройки `prefer_not_to_merge`. Всегда должно быть `false`. Если эта настройка включена, вы допустили ошибку. -* `perform_ttl_move_on_insert` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Значение настройки `perform_ttl_move_on_insert`. Отключает перемещение по TTL при вставке части данных (INSERT). По умолчанию, если вставляется часть данных, срок жизни которой уже истёк по правилу перемещения TTL, она немедленно попадает на том/диск, указанный в правиле перемещения. Это может существенно замедлить вставку, если целевой том/диск медленный (например, S3). -* `load_balancing` ([Enum8](../../sql-reference/data-types/enum.md)) — Политика балансировки нагрузки по дискам. Может принимать одно из следующих значений: - * `ROUND_ROBIN` - * `LEAST_USED` - -Если политика хранения содержит более одного тома, то информация по каждому тому хранится в отдельной строке таблицы. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md deleted file mode 100644 index b24e7589da8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: 'Системная таблица, полезная для специалистов по C++ и инженеров ClickHouse, содержащая - информацию для интроспекции бинарного файла `clickhouse`.' -keywords: ['system table', 'symbols'] -slug: /operations/system-tables/symbols -title: 'system.symbols' -doc_type: 'reference' ---- - -Содержит информацию для интроспекции бинарного файла `clickhouse`. Для доступа требуется привилегия `introspection`. -Эта таблица полезна только для специалистов по C++ и инженеров ClickHouse. - -Столбцы: - -* `symbol` ([String](../../sql-reference/data-types/string.md)) — Имя символа в бинарном файле. Оно задекорировано (mangled). Можно применить `demangle(symbol)`, чтобы получить читаемое имя. -* `address_begin` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Начальный адрес символа в бинарном файле. -* `address_end` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Конечный адрес символа в бинарном файле. -* `name` ([String](../../sql-reference/data-types/string.md)) — Псевдоним для `event`. - -**Пример** - -```sql -SELECT address_begin, address_end - address_begin AS size, demangle(symbol) FROM system.symbols ORDER BY size DESC LIMIT 10 -``` - -```text -┌─address_begin─┬─────size─┬─demangle(symbol)──────────────────────────────────────────────────────────────────┐ -│ 25000976 │ 29466000 │ icudt70_dat │ -│ 400605288 │ 2097272 │ arena_emap_global │ -│ 18760592 │ 1048576 │ CLD2::kQuadChrome1015_2 │ -│ 9807152 │ 884808 │ TopLevelDomainLookupHash::isValid(char const*, unsigned long)::wordlist │ -│ 57442432 │ 850608 │ llvm::X86Insts │ -│ 55682944 │ 681360 │ (anonymous namespace)::X86DAGToDAGISel::SelectCode(llvm::SDNode*)::MatcherTable │ -│ 55130368 │ 502840 │ (anonymous namespace)::X86InstructionSelector::getMatchTable() const::MatchTable0 │ -│ 402930616 │ 404032 │ qpl::ml::dispatcher::hw_dispatcher::get_instance()::instance │ -│ 274131872 │ 356795 │ DB::SettingsTraits::Accessor::instance()::$_0::operator()() const │ -│ 58293040 │ 249424 │ llvm::X86InstrNameData │ -└───────────────┴──────────┴───────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md deleted file mode 100644 index c5cbf7114e2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Эта таблица содержит предупреждения о сервере ClickHouse.' -keywords: [ 'системная таблица', 'предупреждения' ] -slug: /operations/system-tables/system_warnings -title: 'system.warnings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.warnings {#systemwarnings} - - - -Эта таблица показывает предупреждения, связанные с сервером ClickHouse. -Предупреждения одного и того же типа объединяются в одно предупреждение. -Например, если число N подключенных баз данных превышает настраиваемый порог T, вместо N отдельных записей отображается одна запись, содержащая текущее значение N. -Если текущее значение становится ниже порога, запись удаляется из таблицы. - -Таблицу можно настроить с помощью следующих настроек: - -* [max_table_num_to_warn](../server-configuration-parameters/settings.md#max_table_num_to_warn) -* [max_database_num_to_warn](../server-configuration-parameters/settings.md#max_database_num_to_warn) -* [max_dictionary_num_to_warn](../server-configuration-parameters/settings.md#max_dictionary_num_to_warn) -* [max_view_num_to_warn](../server-configuration-parameters/settings.md#max_view_num_to_warn) -* [max_part_num_to_warn](../server-configuration-parameters/settings.md#max_part_num_to_warn) -* [max_pending_mutations_to_warn](../server-configuration-parameters/settings.md#max_pending_mutations_to_warn) -* [max_pending_mutations_execution_time_to_warn](/operations/server-configuration-parameters/settings#max_pending_mutations_execution_time_to_warn) -* [max_named_collection_num_to_warn](../server-configuration-parameters/settings.md#max_named_collection_num_to_warn) -* [resource_overload_warnings](/operations/settings/server-overload#resource-overload-warnings) - -Столбцы: - -* `message` ([String](../../sql-reference/data-types/string.md)) — Текст предупреждения. -* `message_format_string` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Форматная строка, используемая для форматирования сообщения. - -**Пример** - -Запрос: - -```sql - SELECT * FROM system.warnings LIMIT 2 \G; -``` - -Результат: - -```text -Row 1: -────── -message: Количество активных частей больше 10. -message_format_string: Количество активных частей больше {}. - -Row 2: -────── -message: Количество подключенных баз данных больше 2. -message_format_string: Количество подключенных баз данных больше {}. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md deleted file mode 100644 index e60e770101d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: 'Системная таблица, содержащая описания табличных движков, поддерживаемых - сервером, и их возможностей.' -keywords: ['system table', 'table_engines'] -slug: /operations/system-tables/table_engines -title: 'system.table_engines' -doc_type: 'reference' ---- - -# system.table_engines {#systemtable_engines} - -Содержит описание движков таблиц, поддерживаемых сервером, а также информацию об их возможностях. - -Эта таблица содержит следующие столбцы (тип столбца указан в скобках): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -Пример: - -```sql -SELECT * -FROM system.table_engines -WHERE name IN ('Kafka', 'MergeTree', 'ReplicatedCollapsingMergeTree') -``` - -```text -┌─name──────────────────────────┬─supports_settings─┬─supports_skipping_indices─┬─supports_sort_order─┬─supports_ttl─┬─supports_replication─┬─supports_deduplication─┬─supports_parallel_insert─┐ -│ MergeTree │ 1 │ 1 │ 1 │ 1 │ 0 │ 0 │ 1 │ -│ Kafka │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ -│ ReplicatedCollapsingMergeTree │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ -└───────────────────────────────┴───────────────────┴───────────────────────────┴─────────────────────┴──────────────┴──────────────────────┴────────────────────────┴──────────────────────────┘ -``` - -**См. также** - -* [Части запроса](../../engines/table-engines/mergetree-family/mergetree.md#mergetree-query-clauses) семейства MergeTree -* [Настройки](/engines/table-engines/integrations/kafka#creating-a-table) Kafka -* [Настройки](../../engines/table-engines/special/join.md#join-limitations-and-settings) Join diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md deleted file mode 100644 index 7c9010ba515..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -description: 'Системная таблица, содержащая метаданные каждой таблицы, известной серверу.' -keywords: ['системная таблица', 'таблицы'] -slug: /operations/system-tables/tables -title: 'system.tables' -doc_type: 'reference' ---- - -# system.tables {#systemtables} - -Содержит метаданные для каждой таблицы, известной серверу. - -[Отсоединённые](../../sql-reference/statements/detach.md) таблицы не отображаются в `system.tables`. - -[Временные таблицы](../../sql-reference/statements/create/table.md#temporary-tables) видны в `system.tables` только в тех сессиях, в которых они были созданы. Они отображаются с пустым полем `database` и с включённым флагом `is_temporary`. - -Столбцы: - -* `database` ([String](../../sql-reference/data-types/string.md)) — Имя базы данных, в которой находится таблица. - -* `name` ([String](../../sql-reference/data-types/string.md)) — имя таблицы. - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — UUID таблицы (база данных Atomic). - -* `engine` ([String](../../sql-reference/data-types/string.md)) — Имя движка таблицы (без параметров). - -* `is_temporary` ([UInt8](../../sql-reference/data-types/int-uint.md)) — флаг, указывающий, является ли таблица временной. - -* `data_paths` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Пути к данным таблицы в файловой системе. - -* `metadata_path` ([String](../../sql-reference/data-types/string.md)) - Путь к метаданным таблицы в файловой системе. - -* `metadata_modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - время последнего изменения метаданных таблицы. - -* `metadata_version` ([Int32](../../sql-reference/data-types/int-uint.md)) — версия метаданных для таблиц ReplicatedMergeTree, 0 для таблиц других типов. - -* `dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Зависимости от баз данных. - -* `dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Зависимости таблицы (материализованные [представления](/sql-reference/statements/create/view#materialized-view), зависящие от текущей таблицы). - -* `create_table_query` ([String](../../sql-reference/data-types/string.md)) - Запрос, использованный для создания таблицы. - -* `engine_full` ([String](../../sql-reference/data-types/string.md)) - Параметры движка таблицы. - -* `as_select` ([String](../../sql-reference/data-types/string.md)) - запрос `SELECT` для представления. - -* `parameterized_view_parameters` ([Array](../../sql-reference/data-types/array.md) типа [Tuple](../../sql-reference/data-types/tuple.md)) — параметры параметризованного представления. - -* `partition_key` ([String](../../sql-reference/data-types/string.md)) — выражение ключа партиционирования, заданное в таблице. - -* `sorting_key` ([String](../../sql-reference/data-types/string.md)) — выражение ключа сортировки, заданное для таблицы. - -* `primary_key` ([String](../../sql-reference/data-types/string.md)) — выражение первичного ключа, указанное для таблицы. - -* `sampling_key` ([String](../../sql-reference/data-types/string.md)) — выражение ключа выборки, заданное в таблице. - -* `storage_policy` ([String](../../sql-reference/data-types/string.md)) - Политика хранения: - - * [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) - * [Distributed](/engines/table-engines/special/distributed) - -* `total_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - Общее число строк, если можно быстро определить точное количество строк в таблице, иначе `NULL` (включая строки в базовой таблице `Buffer`). - -* `total_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - Общее число байт (включая индексы и проекции), если можно быстро определить точное количество байт для таблицы в хранилище, иначе `NULL` (не включает данные в каком-либо базовом хранилище). - - * Если таблица хранит данные на диске, возвращает используемое место на диске (в сжатом виде). - * Если таблица хранит данные в памяти, возвращает приблизительный объём используемой памяти в байтах. - -* `total_bytes_uncompressed` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — Общее количество несжатых байт (включая индексы и проекции), если можно быстро определить точное количество байт по контрольным суммам кусков таблицы в хранилище, иначе `NULL` (не учитывает базовое хранилище, если оно есть). - -* `lifetime_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — общее количество строк, вставленных с момента запуска сервера (только для таблиц `Buffer`). - -* `lifetime_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - Общее количество байт, записанных операциями INSERT с момента запуска сервера (только для таблиц типа `Buffer`). - -* `comment` ([String](../../sql-reference/data-types/string.md)) - Комментарий к таблице. - -* `has_own_data` ([UInt8](../../sql-reference/data-types/int-uint.md)) — флаг, указывающий, хранит ли сама таблица данные на диске или только обращается к другому источнику. - -* `loading_dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Зависимости при загрузке базы данных (список объектов, которые должны быть загружены до текущего объекта). - -* `loading_dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Зависимости загрузки таблицы (список объектов, которые должны быть загружены до текущего объекта). - -* `loading_dependent_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Базы данных, от загрузки которых зависит текущая. - -* `loading_dependent_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - Таблица, от которой зависит загрузка. - -Таблица `system.tables` используется при реализации запроса `SHOW TABLES`. - -**Пример** - -```sql -SELECT * FROM system.tables LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: base -name: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/store/81b/81b1c20a-b7c6-4116-a2ce-7583fb6b6736/'] -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -metadata_modification_time: 2021-01-25 19:14:32 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE base.t1 (`n` UInt64) ENGINE = MergeTree ORDER BY n SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY n SETTINGS index_granularity = 8192 -as_select: SELECT database AS table_catalog -partition_key: -sorting_key: n -primary_key: n -sampling_key: -storage_policy: default -total_rows: 1 -total_bytes: 99 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] - -Row 2: -────── -database: default -name: 53r93yleapyears -uuid: 00000000-0000-0000-0000-000000000000 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/data/default/53r93yleapyears/'] -metadata_path: /var/lib/clickhouse/metadata/default/53r93yleapyears.sql -metadata_modification_time: 2020-09-23 09:05:36 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE default.`53r93yleapyears` (`id` Int8, `febdays` Int8) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY id SETTINGS index_granularity = 8192 -as_select: SELECT name AS catalog_name -partition_key: -sorting_key: id -primary_key: id -sampling_key: -storage_policy: default -total_rows: 2 -total_bytes: 155 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md deleted file mode 100644 index 4d30728c675..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -description: 'Системная таблица, содержащая записи журнала.' -keywords: ['system table', 'text_log'] -slug: /operations/system-tables/text_log -title: 'system.text_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.text_log {#systemtext_log} - - - -Содержит записи логов. Уровень логирования, записи с которым попадают в эту таблицу, можно ограничить с помощью серверной настройки `text_log.level`. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, выполняющего запрос. -* `event_date` (Date) — дата записи. -* `event_time` (DateTime) — время записи. -* `event_time_microseconds` (DateTime64) — время записи с точностью до микросекунд. -* `microseconds` (UInt32) — микросекунды записи. -* `thread_name` (String) — имя потока, из которого производилось логирование. -* `thread_id` (UInt64) — ID потока ОС. -* `level` (`Enum8`) — уровень записи. Возможные значения: - * `1` или `'Fatal'`. - * `2` или `'Critical'`. - * `3` или `'Error'`. - * `4` или `'Warning'`. - * `5` или `'Notice'`. - * `6` или `'Information'`. - * `7` или `'Debug'`. - * `8` или `'Trace'`. -* `query_id` (String) — ID запроса. -* `logger_name` (LowCardinality(String)) — имя логгера (например, `DDLWorker`). -* `message` (String) — само сообщение. -* `revision` (UInt32) — ревизия ClickHouse. -* `source_file` (LowCardinality(String)) — исходный файл, из которого производилось логирование. -* `source_line` (UInt64) — строка исходного кода, из которой производилось логирование. -* `message_format_string` (LowCardinality(String)) — строка формата, использовавшаяся для форматирования сообщения. -* `value1` (String) — аргумент 1, использованный для форматирования сообщения. -* `value2` (String) — аргумент 2, использованный для форматирования сообщения. -* `value3` (String) — аргумент 3, использованный для форматирования сообщения. -* `value4` (String) — аргумент 4, использованный для форматирования сообщения. -* `value5` (String) — аргумент 5, использованный для форматирования сообщения. -* `value6` (String) — аргумент 6, использованный для форматирования сообщения. -* `value7` (String) — аргумент 7, использованный для форматирования сообщения. -* `value8` (String) — аргумент 8, использованный для форматирования сообщения. -* `value9` (String) — аргумент 9, использованный для форматирования сообщения. -* `value10` (String) — аргумент 10, использованный для форматирования сообщения. - -**Пример** - -```sql -SELECT * FROM system.text_log LIMIT 1 \G -``` - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:07 -event_time_microseconds: 2020-09-10 11:23:07.871397 -microseconds: 871397 -thread_name: clickhouse-serv -thread_id: 564917 -level: Информация -query_id: -logger_name: DNSCacheUpdater -message: Период обновления 15 секунд -revision: 54440 -source_file: /ClickHouse/src/Interpreters/DNSCacheUpdater.cpp; void DB::DNSCacheUpdater::start() -source_line: 45 -message_format_string: Период обновления {} секунд -value1: 15 -value2: -value3: -value4: -value5: -value6: -value7: -value8: -value9: -value10: -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md deleted file mode 100644 index b42fac05aeb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'Системная таблица, содержащая список часовых поясов, поддерживаемых сервером ClickHouse.' -keywords: ['system table', 'time_zones'] -slug: /operations/system-tables/time_zones -title: 'system.time_zones' -doc_type: 'reference' ---- - -# system.time_zones {#systemtime_zones} - -Содержит перечень часовых поясов, поддерживаемых сервером ClickHouse. Этот список может меняться в зависимости от версии ClickHouse. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT * FROM system.time_zones LIMIT 10 -``` - -```text -┌─time_zone──────────┐ -│ Africa/Abidjan │ -│ Africa/Accra │ -│ Africa/Addis_Ababa │ -│ Africa/Algiers │ -│ Africa/Asmara │ -│ Africa/Asmera │ -│ Africa/Bamako │ -│ Africa/Bangui │ -│ Africa/Banjul │ -│ Africa/Bissau │ -└────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md deleted file mode 100644 index 66b874237e7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: 'Системная таблица, содержащая трассировки стека, собранные профилировщиком запросов с выборочным профилированием.' -keywords: ['системная таблица', 'trace_log'] -slug: /operations/system-tables/trace_log -title: 'system.trace_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.trace_log {#systemtrace_log} - - - -Содержит трассировки стека, собранные [профилировщиком запросов с выборкой](../../operations/optimizing-performance/sampling-query-profiler.md). - -ClickHouse создаёт эту таблицу, когда задан раздел конфигурации сервера [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log). Также см. настройки: [query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns), [query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns), [memory_profiler_step](../../operations/settings/settings.md#memory_profiler_step), -[memory_profiler_sample_probability](../../operations/settings/settings.md#memory_profiler_sample_probability), [trace_profile_events](../../operations/settings/settings.md#trace_profile_events). - -Для анализа логов используйте функции интроспекции `addressToLine`, `addressToLineWithInlines`, `addressToSymbol` и `demangle`. - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, выполняющего запрос. - -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата момента выборки. - -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Временная метка момента выборки. - -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Временная метка момента выборки с точностью до микросекунд. - -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Временная метка момента выборки в наносекундах. - -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Ревизия сборки сервера ClickHouse. - - При подключении к серверу через `clickhouse-client` вы видите строку вида `Connected to ClickHouse server version 19.18.1.`. Это поле содержит `revision`, а не `version` сервера. - -* `trace_type` ([Enum8](../../sql-reference/data-types/enum.md)) — Тип трассировки: - * `Real` — сбор трассировок стека по реальному времени (времени настенных часов). - * `CPU` — сбор трассировок стека по времени CPU. - * `Memory` — сбор выделений и освобождений памяти при превышении очередного порогового значения по объёму выделенной памяти. - * `MemorySample` — сбор случайных выделений и освобождений памяти. - * `MemoryPeak` — сбор обновлений пикового потребления памяти. - * `ProfileEvent` — сбор увеличений счётчиков событий профиля. - * `JemallocSample` — сбор выборок jemalloc. - * `MemoryAllocatedWithoutCheck` — сбор значительных выделений (>16MiB), выполняемый с игнорированием любых лимитов по памяти (только для разработчиков ClickHouse). - -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Идентификатор потока. - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — Идентификатор запроса, который можно использовать для получения подробной информации о выполнявшемся запросе из системной таблицы [query_log](/operations/system-tables/query_log). - -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — Трассировка стека в момент выборки. Каждый элемент — виртуальный адрес памяти внутри процесса сервера ClickHouse. - -* `size` ([Int64](../../sql-reference/data-types/int-uint.md)) — Для типов трассировок `Memory`, `MemorySample` или `MemoryPeak` — объём выделенной памяти, для остальных типов трассировок — 0. - -* `event` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — Для типа трассировки `ProfileEvent` — имя обновлённого события профиля, для остальных типов трассировок — пустая строка. - -* `increment` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Для типа трассировки `ProfileEvent` — величина увеличения счётчика события профиля, для остальных типов трассировок — 0. - -* `symbols`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — если символизация включена, содержит деманглированные имена символов, соответствующие `trace`. - -* `lines`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — если символизация включена, содержит строки с именами файлов и номерами строк, соответствующие `trace`. - -Символизацию можно включить или отключить в параметре `symbolize` в разделе `trace_log` в конфигурационном файле сервера. - -**Пример** - -```sql -SELECT * FROM system.trace_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:09 -event_time_microseconds: 2020-09-10 11:23:09.872924 -timestamp_ns: 1599762189872924510 -revision: 54440 -trace_type: Memory -thread_id: 564963 -query_id: -trace: [371912858,371912789,371798468,371799717,371801313,371790250,624462773,566365041,566440261,566445834,566460071,566459914,566459842,566459580,566459469,566459389,566459341,566455774,371993941,371988245,372158848,372187428,372187309,372187093,372185478,140222123165193,140222122205443] -size: 5244400 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md deleted file mode 100644 index 0103c61b0bf..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md +++ /dev/null @@ -1,178 +0,0 @@ ---- -description: 'Системная таблица, содержащая список символов Unicode и их свойств.' -keywords: ['системная таблица', 'Unicode'] -slug: /operations/system-tables/unicode -title: 'system.unicode' -doc_type: 'reference' ---- - -# system.unicode {#systemunicode} - -Таблица `system.unicode` — это виртуальная таблица, которая предоставляет информацию о символах Unicode и их свойствах ([https://unicode-org.github.io/icu/userguide/strings/properties.html](https://unicode-org.github.io/icu/userguide/strings/properties.html)). Эта таблица генерируется на лету. - -Столбцы - -:::note -Имена свойств кодовых точек Unicode в документации ICU преобразованы в snake case. -::: - -* `code_point` ([String](../../sql-reference/data-types/string.md)) — Представление кодовой точки в UTF-8. -* `code_point_value` ([Int32](../../sql-reference/data-types/int-uint.md)) — Числовое значение кодовой точки. -* `notation` ([String](../../sql-reference/data-types/string.md)) — Обозначение кодовой точки в Unicode. -* Binary Properties ([UInt8](../../sql-reference/data-types/int-uint.md)) — Бинарные свойства кодовой точки. - * `alphabetic`, `ascii_hex_digit`, `case_ignorable`... -* Enumerated Properties ([Int32](../../sql-reference/data-types/int-uint.md)) — Перечислимые свойства кодовой точки. - * `bidi_class`, `bidi_paired_bracket_type`, `block`... -* String Properties ([String](../../sql-reference/data-types/string.md)) — Строковые свойства (ASCII String, Unicode String или кодовая точка) кодовой точки. - * `case_folding`, `decomposition_mapping`, `name`... - -:::note -Отображение (mapping) несколько специфично, см. документацию ICU. Например, simple_uppercase_mapping и uppercase_mapping — не совсем одно и то же. Языкозависимые преобразования не реализованы (например, в турецком верхний регистр для i — это "İ" (U+0130)). -::: - -* `numeric_value` ([Float64](../../sql-reference/data-types/float.md)) — Числовое значение кодовой точки. -* `script_extensions` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — Расширения письма (script extensions) для кодовой точки. -* `identifier_type` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) — Тип идентификатора кодовой точки. -* `general_category_mask` ([Int32](../../sql-reference/data-types/int-uint.md)) — Маска общей категории кодовой точки. - -**Пример** - -```sql -SELECT * FROM system.unicode WHERE code_point = 'a' LIMIT 1; -``` - -```text -Row 1: -────── -code_point: a -code_point_value: 97 -notation: U+0061 -alphabetic: 1 -ascii_hex_digit: 1 -bidi_control: 0 -bidi_mirrored: 0 -dash: 0 -default_ignorable_code_point: 0 -deprecated: 0 -diacritic: 0 -extender: 0 -full_composition_exclusion: 0 -grapheme_base: 1 -grapheme_extend: 0 -grapheme_link: 0 -hex_digit: 1 -hyphen: 0 -id_continue: 1 -id_start: 1 -ideographic: 0 -ids_binary_operator: 0 -ids_trinary_operator: 0 -join_control: 0 -logical_order_exception: 0 -lowercase: 1 -math: 0 -noncharacter_code_point: 0 -quotation_mark: 0 -radical: 0 -soft_dotted: 0 -terminal_punctuation: 0 -unified_ideograph: 0 -uppercase: 0 -white_space: 0 -xid_continue: 1 -xid_start: 1 -case_sensitive: 1 -sentence_terminal: 0 -variation_selector: 0 -nfd_inert: 1 -nfkd_inert: 1 -nfc_inert: 0 -nfkc_inert: 0 -segment_starter: 1 -pattern_syntax: 0 -pattern_white_space: 0 -alnum: 1 -blank: 0 -graph: 1 -print: 1 -xdigit: 1 -cased: 1 -case_ignorable: 0 -changes_when_lowercased: 0 -changes_when_uppercased: 1 -changes_when_titlecased: 1 -changes_when_casefolded: 0 -changes_when_casemapped: 1 -changes_when_nfkc_casefolded: 0 -emoji: 0 -emoji_presentation: 0 -emoji_modifier: 0 -emoji_modifier_base: 0 -emoji_component: 0 -regional_indicator: 0 -prepended_concatenation_mark: 0 -extended_pictographic: 0 -basic_emoji: 0 -emoji_keycap_sequence: 0 -rgi_emoji_modifier_sequence: 0 -rgi_emoji_flag_sequence: 0 -rgi_emoji_tag_sequence: 0 -rgi_emoji_zwj_sequence: 0 -rgi_emoji: 0 -ids_unary_operator: 0 -id_compat_math_start: 0 -id_compat_math_continue: 0 -bidi_class: 0 -block: 1 -canonical_combining_class: 0 -decomposition_type: 0 -east_asian_width: 4 -general_category: 2 -joining_group: 0 -joining_type: 0 -line_break: 2 -numeric_type: 0 -script: 25 -hangul_syllable_type: 0 -nfd_quick_check: 1 -nfkd_quick_check: 1 -nfc_quick_check: 1 -nfkc_quick_check: 1 -lead_canonical_combining_class: 0 -trail_canonical_combining_class: 0 -grapheme_cluster_break: 0 -sentence_break: 4 -word_break: 1 -bidi_paired_bracket_type: 0 -indic_positional_category: 0 -indic_syllabic_category: 0 -vertical_orientation: 0 -identifier_status: 1 -general_category_mask: 4 -numeric_value: 0 -age: 1.1 -bidi_mirroring_glyph: a -case_folding: a -lowercase_mapping: a -name: LATIN SMALL LETTER A -simple_case_folding: a -simple_lowercase_mapping: a -simple_titlecase_mapping: A -simple_uppercase_mapping: A -titlecase_mapping: A -uppercase_mapping: A -bidi_paired_bracket: a -script_extensions: ['Latin'] -identifier_type: ['Recommended'] - -``` - -```sql -SELECT code_point, code_point_value, notation FROM system.unicode WHERE code_point = '😂'; -``` - -```text - ┌─code_point─┬─code_point_value─┬─notation─┐ -1. │ 😂 │ 128514 │ U+1F602 │ - └────────────┴──────────────────┴──────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md deleted file mode 100644 index af1a590e3ca..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию, полезную для общего представления об использовании памяти и ProfileEvents пользователей.' -keywords: ['системная таблица', 'user_processes'] -slug: /operations/system-tables/user_processes -title: 'system.user_processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.user_processes {#systemuser_processes} - - - -Эта системная таблица может использоваться для получения общей информации об использовании памяти и ProfileEvents пользователей. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.user_processes LIMIT 10 FORMAT Vertical; -``` - -```response -Строка 1: -────── -user: default -memory_usage: 9832 -peak_memory_usage: 9832 -ProfileEvents: {'Query':5,'SelectQuery':5,'QueriesWithSubqueries':38,'SelectQueriesWithSubqueries':38,'QueryTimeMicroseconds':842048,'SelectQueryTimeMicroseconds':842048,'ReadBufferFromFileDescriptorRead':6,'ReadBufferFromFileDescriptorReadBytes':234,'IOBufferAllocs':3,'IOBufferAllocBytes':98493,'ArenaAllocChunks':283,'ArenaAllocBytes':1482752,'FunctionExecute':670,'TableFunctionExecute':16,'DiskReadElapsedMicroseconds':19,'NetworkSendElapsedMicroseconds':684,'NetworkSendBytes':139498,'SelectedRows':6076,'SelectedBytes':685802,'ContextLock':1140,'RWLockAcquiredReadLocks':193,'RWLockReadersWaitMilliseconds':4,'RealTimeMicroseconds':1585163,'UserTimeMicroseconds':889767,'SystemTimeMicroseconds':13630,'SoftPageFaults':1947,'OSCPUWaitMicroseconds':6,'OSCPUVirtualTimeMicroseconds':903251,'OSReadChars':28631,'OSWriteChars':28888,'QueryProfilerRuns':3,'LogTrace':79,'LogDebug':24} - -Получена 1 строка. Прошло: 0.010 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/users.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/users.md deleted file mode 100644 index 6c9d18a8795..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/users.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'Системная таблица, содержащая список учетных записей пользователей, настроенных на сервере.' -keywords: ['системная таблица', 'пользователи'] -slug: /operations/system-tables/users -title: 'system.users' -doc_type: 'reference' ---- - -# system.users {#systemusers} - -Содержит список [учётных записей пользователей](../../guides/sre/user-management/index.md#user-account-management), сконфигурированных на сервере. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## См. также {#see-also} - -- [SHOW USERS](/sql-reference/statements/show#show-users) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md deleted file mode 100644 index 2388658257b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию об обновляемых материализованных представлениях.' -keywords: ['системная таблица', 'view_refreshes'] -slug: /operations/system-tables/view_refreshes -title: 'system.view_refreshes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.view_refreshes {#systemview_refreshes} - - - -Информация об [обновляемых материализованных представлениях](../../sql-reference/statements/create/view.md#refreshable-materialized-view). Содержит все обновляемые материализованные представления, независимо от того, выполняется ли их обновление в данный момент или нет. - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**Пример** - -```sql -SELECT - database, - view, - status, - last_refresh_result, - last_refresh_time, - next_refresh_time -FROM system.view_refreshes - -┌─database─┬─view───────────────────────┬─status────┬─last_refresh_result─┬───last_refresh_time─┬───next_refresh_time─┐ -│ default │ hello_documentation_reader │ Scheduled │ Finished │ 2023-12-01 01:24:00 │ 2023-12-01 01:25:00 │ -└──────────┴────────────────────────────┴───────────┴─────────────────────┴─────────────────────┴─────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md deleted file mode 100644 index 5cbe3c5f10e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: 'Системная таблица, содержащая сведения о нагрузках, выполняющихся на локальном сервере.' -keywords: ['системная таблица', 'нагрузки'] -slug: /operations/system-tables/workloads -title: 'system.workloads' -doc_type: 'reference' ---- - -# system.workloads {#systemworkloads} - -Содержит информацию о [рабочих нагрузках](/operations/workload-scheduling.md#workload_entity_storage), размещённых на локальном сервере. Для каждой рабочей нагрузки в таблице есть отдельная строка. - -Пример: - -```sql -SELECT * -FROM system.workloads -FORMAT Vertical -``` - -```text -Строка 1: -────── -name: production -parent: all -create_query: CREATE WORKLOAD production IN `all` SETTINGS weight = 9 - -Строка 2: -────── -name: development -parent: all -create_query: CREATE WORKLOAD development IN `all` - -Строка 3: -────── -name: all -parent: -create_query: CREATE WORKLOAD `all` -``` - -Столбцы: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md deleted file mode 100644 index 37566692cd0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: 'Системная таблица, которая существует только если настроен ClickHouse Keeper или ZooKeeper. Предоставляет данные из кластера Keeper, заданного в конфигурации.' -keywords: ['системная таблица', 'zookeeper'] -slug: /operations/system-tables/zookeeper -title: 'system.zookeeper' -doc_type: 'reference' ---- - -# system.zookeeper {#systemzookeeper} - -Таблица не существует, если не настроен ClickHouse Keeper или ZooKeeper. Таблица `system.zookeeper` предоставляет данные из кластеров Keeper, определённых в конфигурации. -Запрос должен содержать либо условие `path =`, либо условие `path IN` в секции `WHERE`, как показано ниже. Это соответствует пути дочерних узлов, для которых вы хотите получить данные. - -Запрос `SELECT * FROM system.zookeeper WHERE path = '/clickhouse'` выводит данные для всех дочерних узлов узла `/clickhouse`. -Чтобы вывести данные для всех корневых узлов, укажите path = '/'. -Если путь, указанный в 'path', не существует, будет сгенерировано исключение. - -Запрос `SELECT * FROM system.zookeeper WHERE path IN ('/', '/clickhouse')` выводит данные для всех дочерних узлов узлов `/` и `/clickhouse`. -Если в указанной коллекции 'path' отсутствует путь, будет сгенерировано исключение. -Эту таблицу можно использовать для пакетного выполнения запросов по путям в Keeper. - -Запрос `SELECT * FROM system.zookeeper WHERE path = '/clickhouse' AND zookeeperName = 'auxiliary_cluster'` выводит данные из кластера ZooKeeper `auxiliary_cluster`. -Если указанный 'auxiliary_cluster' не существует, будет сгенерировано исключение. - -Столбцы: - -* `name` (String) — Имя узла. -* `path` (String) — Путь к узлу. -* `value` (String) — Значение узла. -* `zookeeperName` (String) — Имя кластера ZooKeeper по умолчанию или одного из дополнительных кластеров. -* `dataLength` (Int32) — Размер значения. -* `numChildren` (Int32) — Количество потомков. -* `czxid` (Int64) — ID транзакции, создавшей узел. -* `mzxid` (Int64) — ID транзакции, последней изменившей узел. -* `pzxid` (Int64) — ID транзакции, последней удалившей или добавившей потомков. -* `ctime` (DateTime) — Время создания узла. -* `mtime` (DateTime) — Время последнего изменения узла. -* `version` (Int32) — Версия узла: количество изменений узла. -* `cversion` (Int32) — Количество добавленных или удалённых потомков. -* `aversion` (Int32) — Количество изменений ACL. -* `ephemeralOwner` (Int64) — Для эфемерных узлов — ID сессии, которой принадлежит этот узел. - -Пример: - -```sql -SELECT * -FROM system.zookeeper -WHERE path = '/clickhouse/tables/01-08/visits/replicas' -FORMAT Vertical -``` - -```text -Строка 1: -────── -name: example01-08-1 -value: -czxid: 932998691229 -mzxid: 932998691229 -ctime: 2015-03-27 16:49:51 -mtime: 2015-03-27 16:49:51 -version: 0 -cversion: 47 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021031383 -path: /clickhouse/tables/01-08/visits/replicas - -Строка 2: -────── -name: example01-08-2 -value: -czxid: 933002738135 -mzxid: 933002738135 -ctime: 2015-03-27 16:57:01 -mtime: 2015-03-27 16:57:01 -version: 0 -cversion: 37 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021252247 -path: /clickhouse/tables/01-08/visits/replicas -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md deleted file mode 100644 index d9eb2cf66b2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Системная таблица, которая существует только если ZooKeeper настроен. Показывает текущие подключения к ZooKeeper (включая вспомогательные экземпляры ZooKeeper).' -keywords: ['системная таблица', 'zookeeper_connection'] -slug: /operations/system-tables/zookeeper_connection -title: 'system.zookeeper_connection' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection {#systemzookeeper_connection} - - - -Эта таблица отсутствует, если ZooKeeper не настроен. Таблица `system.zookeeper_connection` показывает текущие подключения к ZooKeeper (включая вспомогательные экземпляры ZooKeeper). Каждая строка содержит информацию об одном подключении. - -Столбцы: - -* `name` ([String](../../sql-reference/data-types/string.md)) — Имя кластера ZooKeeper. -* `host` ([String](../../sql-reference/data-types/string.md)) — Имя хоста/IP-адрес узла ZooKeeper, к которому подключился ClickHouse. -* `port` ([UIn16](../../sql-reference/data-types/int-uint.md)) — Порт узла ZooKeeper, к которому подключился ClickHouse. -* `index` ([Nullable(UInt8)](../../sql-reference/data-types/int-uint.md)) — Индекс узла ZooKeeper, к которому подключился ClickHouse. Индекс берётся из конфигурации ZooKeeper. Если подключения нет, это значение равно NULL. -* `connected_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — Время установления подключения. -* `session_uptime_elapsed_seconds` ([UInt64](../../sql-reference/data-types/int-uint.md)) — Количество секунд, прошедших с момента установления подключения. -* `is_expired` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Признак того, что текущее подключение истекло. -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Версия Keeper API. -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — Идентификатор сеанса подключения. -* `xid` ([Int64](../../sql-reference/data-types/int-uint.md)) — XID текущего сеанса. -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — Включённые флаги функциональности. Применимо только к ClickHouse Keeper. Возможные значения: `FILTERED_LIST`, `MULTI_READ`, `CHECK_NOT_EXISTS`, `CREATE_IF_NOT_EXISTS`, `REMOVE_RECURSIVE`. -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — Зона доступности. - -Пример: - -```sql -SELECT * FROM system.zookeeper_connection; -``` - -```text -┌─name────┬─host──────┬─port─┬─index─┬──────connected_time─┬─session_uptime_elapsed_seconds─┬─is_expired─┬─keeper_api_version─┬─client_id─┬─xid─┬─enabled_feature_flags────────────────────────────────────────────────────┬─availability_zone─┐ -│ default │ 127.0.0.1 │ 2181 │ 0 │ 2025-04-10 14:30:00 │ 943 │ 0 │ 0 │ 420 │ 69 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS'] │ eu-west-1b │ -└─────────┴───────────┴──────┴───────┴─────────────────────┴────────────────────────────────┴────────────┴────────────────────┴───────────┴─────┴──────────────────────────────────────────────────────────────────────────┴───────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md deleted file mode 100644 index a58b96f4d73..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Показывает историю подключений к ZooKeeper (включая вспомогательные экземпляры ZooKeeper).' -keywords: ['system table', 'zookeeper_connection_log'] -slug: /operations/system-tables/zookeeper_connection_log -title: 'system.zookeeper_connection_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection_log {#systemzookeeper_connection_log} - - - -Таблица 'system.zookeeper_connection_log' показывает историю подключений к ZooKeeper (включая вспомогательные ZooKeeper). Каждая строка содержит информацию об одном событии, связанном с подключениями. - -:::note -Таблица не содержит событий для отключений, вызванных остановкой сервера. -::: - -Столбцы: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — имя хоста сервера, который подключается к ZooKeeper или отключается от него. -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) - тип события. Возможные значения: `Connected`, `Disconnected`. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) - дата записи. -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - время записи. -* `event_time_microseconds` ([Date](../../sql-reference/data-types/datetime64.md)) - время записи с точностью до микросекунд. -* `name` ([String](../../sql-reference/data-types/string.md)) — имя кластера ZooKeeper. -* `host` ([String](../../sql-reference/data-types/string.md)) — имя хоста/IP узла ZooKeeper, к которому подключился ClickHouse. -* `port` ([UIn16](../../sql-reference/data-types/int-uint.md)) — порт узла ZooKeeper, к которому подключился ClickHouse. -* `index` ([UInt8](../../sql-reference/data-types/int-uint.md)) — индекс узла ZooKeeper, к которому ClickHouse подключился или от которого отключился. Индекс берётся из конфигурации ZooKeeper. -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — идентификатор сессии подключения. -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — версия Keeper API. -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — включённые флаги функций. Применимо только к ClickHouse Keeper. Возможные значения: `FILTERED_LIST`, `MULTI_READ`, `CHECK_NOT_EXISTS`, `CREATE_IF_NOT_EXISTS`, `REMOVE_RECURSIVE`. -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — зона доступности. -* `reason` ([String](../../sql-reference/data-types/string.md)) — причина подключения или отключения. - -Пример: - -```sql -SELECT * FROM system.zookeeper_connection_log; -``` - -```text -┌─hostname─┬─type─────────┬─event_date─┬──────────event_time─┬────event_time_microseconds─┬─name───────────────┬─host─┬─port─┬─index─┬─client_id─┬─keeper_api_version─┬─enabled_feature_flags───────────────────────────────────────────────────────────────────────┬─availability_zone─┬─reason──────────────┐ - 1. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:35 │ 2025-05-12 19:49:35.713067 │ zk_conn_log_test_4 │ zoo2 │ 2181 │ 0 │ 10 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Инициализация │ - 2. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:23 │ 2025-05-12 19:49:23.981570 │ default │ zoo1 │ 2181 │ 0 │ 4 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Инициализация │ - 3. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:28 │ 2025-05-12 19:49:28.104021 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Инициализация │ - 4. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.459251 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Инициализация │ - 5. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.574312 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Инициализация │ - 6. │ node │ Отключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909890 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Изменена конфигурация │ - 7. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909895 │ default │ zoo2 │ 2181 │ 0 │ 8 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Изменена конфигурация │ - 8. │ node │ Отключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912010 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Изменена конфигурация │ - 9. │ node │ Подключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912014 │ zk_conn_log_test_2 │ zoo3 │ 2181 │ 0 │ 9 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Изменена конфигурация │ -10. │ node │ Отключено │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912061 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ Удалено из конфигурации │ - └──────────┴──────────────┴────────────┴─────────────────────┴────────────────────────────┴────────────────────┴──────┴──────┴───────┴───────────┴────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────┴─────────────────────┘ -``` \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md deleted file mode 100644 index dfa7c64e961..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'Системная таблица, содержащая информацию о параметрах запроса к серверу ZooKeeper и ответе сервера.' -keywords: ['system table', 'zookeeper_log'] -slug: /operations/system-tables/zookeeper_log -title: 'system.zookeeper_log' -doc_type: 'reference' ---- - -# system.zookeeper_log {#systemzookeeper_log} - -Эта таблица содержит информацию о параметрах запроса к серверу ZooKeeper и его ответе. - -Для запросов заполняются только столбцы с параметрами запроса, а остальные столбцы заполняются значениями по умолчанию (`0` или `NULL`). Когда приходит ответ, данные из ответа добавляются в другие столбцы. - -Столбцы с параметрами запроса: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — Имя хоста сервера, на котором выполняется запрос. -* `type` ([Enum](../../sql-reference/data-types/enum.md)) — Тип события в клиенте ZooKeeper. Может иметь одно из следующих значений: - * `Request` — Запрос был отправлен. - * `Response` — Ответ был получен. - * `Finalize` — Соединение потеряно, ответ не был получен. -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — Дата, когда произошло событие. -* `event_time` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — Дата и время, когда произошло событие. -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — IP-адрес сервера ZooKeeper, который использовался для выполнения запроса. -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — Порт сервера ZooKeeper, который использовался для выполнения запроса. -* `session_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — Идентификатор сессии, который сервер ZooKeeper устанавливает для каждого соединения. -* `xid` ([Int32](../../sql-reference/data-types/int-uint.md)) — Идентификатор запроса внутри сессии. Обычно это последовательный номер запроса. Он одинаков для строки с запросом и соответствующей строки `response`/`finalize`. -* `has_watch` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Устанавливается ли [watch](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkWatches) этим запросом. -* `op_num` ([Enum](../../sql-reference/data-types/enum.md)) — Тип запроса или ответа. -* `path` ([String](../../sql-reference/data-types/string.md)) — Путь к узлу ZooKeeper, указанному в запросе, или пустая строка, если запрос не требует указания пути. -* `data` ([String](../../sql-reference/data-types/string.md)) — Данные, записанные в узел ZooKeeper (для запросов `SET` и `CREATE` — что запрос хотел записать, для ответа на запрос `GET` — что было прочитано) или пустая строка. -* `is_ephemeral` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Создаётся ли узел ZooKeeper как [временный (ephemeral)](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Ephemeral+Nodes). -* `is_sequential` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Создаётся ли узел ZooKeeper как [последовательный (sequential)](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Sequence+Nodes+--+Unique+Naming). -* `version` ([Nullable(Int32)](../../sql-reference/data-types/nullable.md)) — Версия узла ZooKeeper, которую запрос ожидает при выполнении. Поддерживается для запросов `CHECK`, `SET`, `REMOVE` (имеет значение `-1`, если запрос не проверяет версию, или `NULL` для других запросов, которые не поддерживают проверку версии). -* `requests_size` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Количество запросов, входящих в multi-запрос (это специальный запрос, который состоит из нескольких последовательных обычных запросов и выполняет их атомарно). Все запросы, включённые в multi-запрос, будут иметь одинаковый `xid`. -* `request_idx` ([UInt32](../../sql-reference/data-types/int-uint.md)) — Номер запроса, входящего в multi-запрос (для самого multi-запроса — `0`, затем по порядку начиная с `1`). - -Столбцы с параметрами ответа на запрос: - -* `zxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — идентификатор транзакции ZooKeeper. Порядковый номер, выдаваемый сервером ZooKeeper в ответ на успешно выполненный запрос (`0`, если запрос не был выполнен/вернул ошибку/клиент не знает, был ли запрос выполнен). -* `error` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — код ошибки. Может иметь много значений, ниже приведены только некоторые из них: - * `ZOK` — запрос выполнен успешно. - * `ZCONNECTIONLOSS` — соединение потеряно. - * `ZOPERATIONTIMEOUT` — истёк таймаут выполнения запроса. - * `ZSESSIONEXPIRED` — срок действия сессии истёк. - * `NULL` — запрос завершён. -* `watch_type` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — тип события `watch` (для ответов с `op_num` = `Watch`), для остальных ответов: `NULL`. -* `watch_state` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — состояние события `watch` (для ответов с `op_num` = `Watch`), для остальных ответов: `NULL`. -* `path_created` ([String](../../sql-reference/data-types/string.md)) — путь к созданному узлу ZooKeeper (для ответов на запрос `CREATE`); может отличаться от `path`, если узел создан как `sequential`. -* `stat_czxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — `zxid` изменения, в результате которого был создан этот узел ZooKeeper. -* `stat_mzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — `zxid` изменения, которое последним модифицировало этот узел ZooKeeper. -* `stat_pzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — идентификатор транзакции изменения, которое последним модифицировало дочерние узлы этого узла ZooKeeper. -* `stat_version` ([Int32](../../sql-reference/data-types/int-uint.md)) — количество изменений данных этого узла ZooKeeper. -* `stat_cversion` ([Int32](../../sql-reference/data-types/int-uint.md)) — количество изменений дочерних узлов этого узла ZooKeeper. -* `stat_dataLength` ([Int32](../../sql-reference/data-types/int-uint.md)) — длина поля данных этого узла ZooKeeper. -* `stat_numChildren` ([Int32](../../sql-reference/data-types/int-uint.md)) — количество дочерних узлов этого узла ZooKeeper. -* `children` ([Array(String)](../../sql-reference/data-types/array.md)) — список дочерних узлов ZooKeeper (для ответов на запрос `LIST`). - -**Пример** - -Запрос: - -```sql -SELECT * FROM system.zookeeper_log WHERE (session_id = '106662742089334927') AND (xid = '10858') FORMAT Vertical; -``` - -Результат: - -```text -Строка 1: -────── -hostname: clickhouse.eu-central1.internal -type: Request -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.291792 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 0 -error: ᴺᵁᴸᴸ -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 0 -stat_mzxid: 0 -stat_pzxid: 0 -stat_version: 0 -stat_cversion: 0 -stat_dataLength: 0 -stat_numChildren: 0 -children: [] - -Строка 2: -────── -type: Response -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.292086 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 16926267 -error: ZOK -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 16925469 -stat_mzxid: 16925469 -stat_pzxid: 16926179 -stat_version: 0 -stat_cversion: 7 -stat_dataLength: 0 -stat_numChildren: 7 -children: ['query-0000000006','query-0000000005','query-0000000004','query-0000000003','query-0000000002','query-0000000001','query-0000000000'] -``` - -**См. также** - -* [ZooKeeper](../../operations/tips.md#zookeeper) -* [Руководство по ZooKeeper](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/tips.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/tips.md deleted file mode 100644 index 156876078dd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/tips.md +++ /dev/null @@ -1,334 +0,0 @@ ---- -description: 'Страница с рекомендациями по использованию ClickHouse с открытым исходным кодом' -sidebar_label: 'Рекомендации по использованию ClickHouse с открытым исходным кодом' -sidebar_position: 58 -slug: /operations/tips -title: 'Рекомендации по использованию ClickHouse с открытым исходным кодом' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - - -## Регулятор частоты CPU {#cpu-scaling-governor} - -Всегда используйте регулятор `performance`. Режим `on-demand` работает значительно хуже при постоянно высокой нагрузке. - -```bash -$ echo 'performance' | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor -``` - - -## Ограничения по CPU {#cpu-limitations} - -Процессоры могут перегреваться. Используйте `dmesg`, чтобы проверить, не была ли тактовая частота процессора ограничена из-за перегрева. -Ограничение также может быть установлено извне на уровне датацентра. Вы можете использовать `turbostat`, чтобы отслеживать это под нагрузкой. - -## ОЗУ {#ram} - -Для небольших объёмов данных (до ~200 ГБ в сжатом виде) оптимально, чтобы объём доступной памяти был сопоставим с объёмом данных. -Для больших объёмов данных и при обработке интерактивных (онлайн) запросов следует использовать достаточный объём ОЗУ (128 ГБ и более), чтобы «горячее» подмножество данных помещалось в кэш страниц. -Даже при объёмах данных ~50 ТБ на сервер использование 128 ГБ ОЗУ существенно улучшает производительность запросов по сравнению с 64 ГБ. - -Не отключайте overcommit. Значение в файле `/proc/sys/vm/overcommit_memory` должно быть 0 или 1. Выполните - -```bash -$ echo 0 | sudo tee /proc/sys/vm/overcommit_memory -``` - -Используйте `perf top`, чтобы отслеживать время, затрачиваемое в ядре на управление памятью. -Постоянные большие страницы (huge pages) также не требуется выделять. - - -### Использование менее 16 ГБ ОЗУ {#using-less-than-16gb-of-ram} - -Рекомендуемый объём ОЗУ — 32 ГБ или больше. - -Если в вашей системе меньше 16 ГБ ОЗУ, вы можете столкнуться с различными исключениями, связанными с памятью, поскольку настройки по умолчанию не рассчитаны на такой объём. Можно использовать ClickHouse в системе с небольшим объёмом ОЗУ (начиная с 2 ГБ), но такие конфигурации требуют дополнительной настройки и обеспечивают только низкую скорость приёма данных. - -При использовании ClickHouse с объёмом ОЗУ менее 16 ГБ мы рекомендуем следующее: - -- Уменьшите размер кэша меток в `config.xml`. Его можно установить вплоть до 500 МБ, но нельзя установить в ноль. -- Уменьшите количество потоков обработки запросов до `1`. -- Уменьшите `max_block_size` до `8192`. Значения вплоть до `1024` всё ещё могут быть практичными. -- Уменьшите `max_download_threads` до `1`. -- Установите `input_format_parallel_parsing` и `output_format_parallel_formatting` в `0`. -- Отключите запись в лог-таблицы, так как фоновая задача слияния резервирует ОЗУ для выполнения слияний лог-таблиц. Отключите `asynchronous_metric_log`, `metric_log`, `text_log`, `trace_log`. - -Дополнительные замечания: - -- Чтобы освободить память, закэшированную аллокатором, вы можете выполнить команду `SYSTEM JEMALLOC PURGE`. -- Мы не рекомендуем использовать интеграции с S3 или Kafka на машинах с небольшим объёмом памяти, так как им требуется значительный объём памяти для буферов. - -## Подсистема хранения {#storage-subsystem} - -Если бюджет позволяет, используйте SSD. -Если нет — используйте HDD. Подойдут SATA HDD 7200 RPM. - -Предпочитайте большее число серверов с локальными жесткими дисками меньшему количеству серверов с подключаемыми дисковыми полками. -Но для хранения архивов с редкими запросами дисковые полки подойдут. - -## RAID {#raid} - -При использовании HDD вы можете объединить их в RAID-10, RAID-5, RAID-6 или RAID-50. -Для Linux предпочтительнее программный RAID (с `mdadm`). -При создании RAID-10 выберите схему `far`. -Если бюджет позволяет, выбирайте RAID-10. - -Сам по себе LVM (без RAID или `mdadm`) приемлем, но создание RAID на его основе или комбинирование его с `mdadm` — менее распространённая и хуже проработанная конфигурация, в которой выше риск ошибок -(выбор неправильного размера блока; неверное выравнивание блоков; выбор неподходящего типа RAID; забыв очистить диски). Если вы уверенно -пользуетесь LVM, нет препятствий для его использования. - -Если у вас больше 4 дисков, используйте RAID-6 (предпочтительно) или RAID-50 вместо RAID-5. -При использовании RAID-5, RAID-6 или RAID-50 всегда увеличивайте значение stripe_cache_size, так как значение по умолчанию обычно далеко от оптимального. - -```bash -$ echo 4096 | sudo tee /sys/block/md2/md/stripe_cache_size -``` - -Рассчитайте точное значение, исходя из числа устройств и размера блока, используя формулу: `2 * num_devices * chunk_size_in_bytes / 4096`. - -Размер блока 64 KB достаточен для большинства конфигураций RAID. Средний размер записи clickhouse-server составляет примерно 1 MB (1024 KB), поэтому рекомендуемый размер страйпа (stripe size) также равен 1 MB. Размер блока при необходимости можно оптимизировать, установив его равным 1 MB, делённому на количество дисков без чётности (non-parity) в массиве RAID, чтобы каждая запись параллелизировалась по всем доступным дискам без чётности. -Никогда не устанавливайте размер блока слишком маленьким или слишком большим. - -Вы можете использовать RAID-0 на SSD. -Независимо от использования RAID, всегда используйте репликацию для обеспечения сохранности данных. - -Включите NCQ с длинной очередью. Для HDD выберите планировщик mq-deadline или CFQ, а для SSD — noop. Не уменьшайте настройку «readahead». -Для HDD включите кэш записи. - -Убедитесь, что [`fstrim`](https://en.wikipedia.org/wiki/Trim_\(computing\)) включён для дисков NVMe и SSD в вашей ОС (обычно это реализуется с помощью задания cron или сервиса systemd). - - -## Файловая система {#file-system} - -Ext4 — наиболее надёжный вариант. Установите опцию монтирования `noatime`. XFS тоже хорошо подходит. -Большинство других файловых систем, как правило, также будут работать корректно. - -FAT-32 и exFAT не поддерживаются из‑за отсутствия жёстких ссылок. - -Не используйте сжатые файловые системы, так как ClickHouse выполняет сжатие самостоятельно и делает это лучше. -Не рекомендуется использовать зашифрованные файловые системы, поскольку в ClickHouse есть встроенное шифрование, которое лучше. - -Хотя ClickHouse может работать по NFS, это не лучшая идея. - -## Ядро Linux {#linux-kernel} - -Не используйте устаревшие версии ядра Linux. - -## Сеть {#network} - -Если вы используете IPv6, увеличьте размер кэша маршрутов. -Ядро Linux до версии 3.2 имело множество проблем с реализацией IPv6. - -Используйте по возможности сеть не менее 10 Гбит/с. Сеть 1 Гбит/с тоже будет работать, но она значительно хуже подходит для синхронизации реплик с десятками терабайт данных или для обработки распределённых запросов с большим объёмом промежуточных данных. - -## Huge Pages {#huge-pages} - -Если вы используете устаревшую версию ядра Linux, отключите прозрачные huge pages. Они мешают работе аллокатора памяти, что приводит к значительному снижению производительности. -В более новых версиях ядра Linux использование прозрачных huge pages допустимо. - -```bash -$ echo 'madvise' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled -``` - -Если вы хотите изменить настройку transparent huge pages на постоянной основе, отредактируйте `/etc/default/grub`, добавив `transparent_hugepage=madvise` в опцию `GRUB_CMDLINE_LINUX_DEFAULT`: - -```bash -$ GRUB_CMDLINE_LINUX_DEFAULT="transparent_hugepage=madvise ..." -``` - -После этого выполните команду `sudo update-grub`, затем перезагрузите систему, чтобы изменения вступили в силу. - - -## Настройка гипервизора {#hypervisor-configuration} - -Если вы используете OpenStack, задайте - -```ini -cpu_mode=host-passthrough -``` - -в `nova.conf`. - -Если вы используете libvirt, установите - -```xml - -``` - -в конфигурации XML. - -Это важно для того, чтобы ClickHouse мог получать корректную информацию из инструкции `cpuid`. -В противном случае вы можете сталкиваться с аварийным завершением работы с ошибкой `Illegal instruction` при запуске гипервизора на старых моделях CPU. - - -## ClickHouse Keeper и ZooKeeper {#zookeeper} - -Рекомендуется использовать ClickHouse Keeper вместо ZooKeeper для кластеров ClickHouse. См. документацию по [ClickHouse Keeper](../guides/sre/keeper/index.md) - -Если вы хотите продолжить использовать ZooKeeper, лучше использовать свежую версию ZooKeeper – 3.4.9 или более новую. Версия в стабильных дистрибутивах Linux может быть устаревшей. - -Никогда не используйте самописные скрипты для переноса данных между разными кластерами ZooKeeper, так как результат будет некорректным для последовательных узлов. По этой же причине никогда не используйте утилиту «zkcopy»: [https://github.com/ksprojects/zkcopy/issues/15](https://github.com/ksprojects/zkcopy/issues/15) - -Если вы хотите разделить существующий кластер ZooKeeper на два, правильный способ – увеличить число его реплик, а затем переконфигурировать его как два независимых кластера. - -Вы можете запускать ClickHouse Keeper на том же сервере, что и ClickHouse, в тестовых средах или в средах с низкой скоростью ингестии. -Для продуктивных сред мы рекомендуем использовать отдельные серверы для ClickHouse и ZooKeeper/Keeper либо разместить файлы ClickHouse и файлы Keeper на отдельных дисках, поскольку ZooKeeper/Keeper очень чувствительны к задержкам дисковой подсистемы, а ClickHouse может использовать все доступные системные ресурсы. - -В ансамбле могут быть наблюдатели ZooKeeper, но серверы ClickHouse не должны взаимодействовать с наблюдателями. - -Не изменяйте настройку `minSessionTimeout`, большие значения могут повлиять на стабильность перезапуска ClickHouse. - -С настройками по умолчанию ZooKeeper – это мина замедленного действия: - -> При использовании конфигурации по умолчанию сервер ZooKeeper не удаляет файлы старых snapshots и журналов (см. `autopurge`), и это остаётся обязанностью оператора. - -Эту мину необходимо обезвредить. - -Ниже приведена конфигурация ZooKeeper (3.5.1), используемая в крупной продуктивной среде: - -zoo.cfg: - -```bash -# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html {#httphadoopapacheorgzookeeperdocscurrentzookeeperadminhtml} - -# Количество миллисекунд в каждом тике {#the-number-of-milliseconds-of-each-tick} -tickTime=2000 -# Количество тиков, которое может занять начальная {#the-number-of-ticks-that-the-initial} -# фаза синхронизации {#synchronization-phase-can-take} -# Это значение не имеет строгого обоснования {#this-value-is-not-quite-motivated} -initLimit=300 -# Количество тиков, которое может пройти между {#the-number-of-ticks-that-can-pass-between} -# отправкой запроса и получением подтверждения {#sending-a-request-and-getting-an-acknowledgement} -syncLimit=10 - -maxClientCnxns=2000 - -# Максимальное значение, которое может запросить клиент и которое примет сервер. {#it-is-the-maximum-value-that-client-may-request-and-the-server-will-accept} -# Допустимо устанавливать высокое значение maxSessionTimeout на сервере, чтобы клиенты могли работать с большим таймаутом сессии при необходимости. {#it-is-ok-to-have-high-maxsessiontimeout-on-server-to-allow-clients-to-work-with-high-session-timeout-if-they-want} -# По умолчанию запрашивается таймаут сессии 30 секунд (можно изменить с помощью параметра session_timeout_ms в конфигурации ClickHouse). {#but-we-request-session-timeout-of-30-seconds-by-default-you-can-change-it-with-session_timeout_ms-in-clickhouse-config} -maxSessionTimeout=60000000 -# Каталог для хранения снимков состояния. {#the-directory-where-the-snapshot-is-stored} -dataDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/data -# Разместите dataLogDir на отдельном физическом диске для повышения производительности {#place-the-datalogdir-to-a-separate-physical-disc-for-better-performance} -dataLogDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/logs - -autopurge.snapRetainCount=10 -autopurge.purgeInterval=1 - - -# Чтобы избежать операций поиска, ZooKeeper выделяет место в файле журнала транзакций {#to-avoid-seeks-zookeeper-allocates-space-in-the-transaction-log-file-in} -# блоками размером preAllocSize килобайт. Размер блока по умолчанию — 64M. Одна из причин {#blocks-of-preallocsize-kilobytes-the-default-block-size-is-64m-one-reason} -# изменения размера блоков — уменьшение размера блока при более частом создании снимков состояния. {#for-changing-the-size-of-the-blocks-is-to-reduce-the-block-size-if-snapshots} -# (См. также snapCount). {#are-taken-more-often-also-see-snapcount} -preAllocSize=131072 - -# Клиенты могут отправлять запросы быстрее, чем ZooKeeper способен их обработать, {#clients-can-submit-requests-faster-than-zookeeper-can-process-them} -# особенно при большом количестве клиентов. Чтобы предотвратить исчерпание памяти {#especially-if-there-are-a-lot-of-clients-to-prevent-zookeeper-from-running} -# из-за запросов в очереди, ZooKeeper ограничивает клиентов так, чтобы {#out-of-memory-due-to-queued-requests-zookeeper-will-throttle-clients-so-that} -# в системе находилось не более globalOutstandingLimit необработанных запросов. {#there-is-no-more-than-globaloutstandinglimit-outstanding-requests-in-the} -# Ограничение по умолчанию — 1000. {#system-the-default-limit-is-1000} -# globalOutstandingLimit=1000 {#globaloutstandinglimit1000} - -# ZooKeeper записывает транзакции в журнал транзакций. После записи snapCount транзакций {#zookeeper-logs-transactions-to-a-transaction-log-after-snapcount-transactions} -# в файл журнала создается снимок состояния и начинается новый файл журнала транзакций. {#are-written-to-a-log-file-a-snapshot-is-started-and-a-new-transaction-log-file} -# Значение snapCount по умолчанию — 100000. {#is-started-the-default-snapcount-is-100000} -snapCount=3000000 - -# Если эта опция определена, запросы будут записываться в файл трассировки с именем {#if-this-option-is-defined-requests-will-be-will-logged-to-a-trace-file-named} -# traceFile.year.month.day. {#tracefileyearmonthday} -#traceFile= - -# Лидер принимает клиентские соединения. Значение по умолчанию — "yes". Узел-лидер {#leader-accepts-client-connections-default-value-is-yes-the-leader-machine} -# координирует обновления. Для повышения пропускной способности обновлений за счет незначительного снижения {#coordinates-updates-for-higher-update-throughput-at-thes-slight-expense-of} -# пропускной способности чтения лидер может быть настроен так, чтобы не принимать клиентов и сосредоточиться {#read-throughput-the-leader-can-be-configured-to-not-accept-clients-and-focus} -# на координации. {#on-coordination} -leaderServes=yes - -standaloneEnabled=false -dynamicConfigFile=/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/zoo.cfg.dynamic -``` - -Версия для Java: - - -```text -openjdk 11.0.5-shenandoah 2019-10-15 -OpenJDK Runtime Environment (build 11.0.5-shenandoah+10-adhoc.heretic.src) -OpenJDK 64-Bit Server VM (build 11.0.5-shenandoah+10-adhoc.heretic.src, mixed mode) -``` - -Параметры JVM: - - -```bash -NAME=zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} -ZOOCFGDIR=/etc/$NAME/conf - -# TODO это выглядит неаккуратно {#todo-this-is-really-ugly} -# Как узнать, какие jar-файлы требуются? {#how-to-find-out-which-jars-are-needed} -# Судя по всему, log4j требует наличия файла log4j.properties в classpath {#seems-that-log4j-requires-the-log4jproperties-file-to-be-in-the-classpath} -CLASSPATH="$ZOOCFGDIR:/usr/build/classes:/usr/build/lib/*.jar:/usr/share/zookeeper-3.6.2/lib/audience-annotations-0.5.0.jar:/usr/share/zookeeper-3.6.2/lib/commons-cli-1.2.jar:/usr/share/zookeeper-3.6.2/lib/commons-lang-2.6.jar:/usr/share/zookeeper-3.6.2/lib/jackson-annotations-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-core-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-databind-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/javax.servlet-api-3.1.0.jar:/usr/share/zookeeper-3.6.2/lib/jetty-http-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-io-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-security-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-server-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-servlet-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-util-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jline-2.14.6.jar:/usr/share/zookeeper-3.6.2/lib/json-simple-1.1.1.jar:/usr/share/zookeeper-3.6.2/lib/log4j-1.2.17.jar:/usr/share/zookeeper-3.6.2/lib/metrics-core-3.2.5.jar:/usr/share/zookeeper-3.6.2/lib/netty-buffer-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-codec-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-handler-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-resolver-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-epoll-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-unix-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_common-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_hotspot-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_servlet-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-api-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-log4j12-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/snappy-java-1.1.7.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-jute-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-prometheus-metrics-3.6.2.jar:/usr/share/zookeeper-3.6.2/etc" - -ZOOCFG="$ZOOCFGDIR/zoo.cfg" -ZOO_LOG_DIR=/var/log/$NAME -USER=zookeeper -GROUP=zookeeper -PIDDIR=/var/run/$NAME -PIDFILE=$PIDDIR/$NAME.pid -SCRIPTNAME=/etc/init.d/$NAME -JAVA=/usr/local/jdk-11/bin/java -ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain" -ZOO_LOG4J_PROP="INFO,ROLLINGFILE" -JMXLOCALONLY=false -JAVA_OPTS="-Xms{{ '{{' }} cluster.get('xms','128M') {{ '}}' }} \ - -Xmx{{ '{{' }} cluster.get('xmx','1G') {{ '}}' }} \ - -Xlog:safepoint,gc*=info,age*=debug:file=/var/log/$NAME/zookeeper-gc.log:time,level,tags:filecount=16,filesize=16M - -verbose:gc \ - -XX:+UseG1GC \ - -Djute.maxbuffer=8388608 \ - -XX:MaxGCPauseMillis=50" -``` - -Инициализация соли: - -```text -description "zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} служба централизованной координации" - -start on runlevel [2345] -stop on runlevel [!2345] - -respawn - -limit nofile 8192 8192 - -pre-start script - [ -r "/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment" ] || exit 0 - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -d $ZOO_LOG_DIR ] || mkdir -p $ZOO_LOG_DIR - chown $USER:$GROUP $ZOO_LOG_DIR -end script - -script - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -r /etc/default/zookeeper ] && . /etc/default/zookeeper - if [ -z "$JMXDISABLE" ]; then - JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY" - fi - exec start-stop-daemon --start -c $USER --exec $JAVA --name zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} \ - -- -cp $CLASSPATH $JAVA_OPTS -Dzookeeper.log.dir=${ZOO_LOG_DIR} \ - -Dzookeeper.root.logger=${ZOO_LOG4J_PROP} $ZOOMAIN $ZOOCFG -end script -``` - - -## Антивирусное программное обеспечение {#antivirus-software} - -Если вы используете антивирусное программное обеспечение, настройте его так, чтобы оно не сканировало каталоги с данными ClickHouse (`/var/lib/clickhouse`), иначе производительность может снизиться, а во время ингестии данных и фоновых слияний могут возникать неожиданные ошибки. - -## Похожие материалы {#related-content} - -- [Только начинаете работать с ClickHouse? Вот 13 «смертных грехов» и способы их избежать](https://clickhouse.com/blog/common-getting-started-issues-with-clickhouse) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/update.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/update.md deleted file mode 100644 index 9f7ad0426ec..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/update.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -description: 'Документация по обновлению в самоуправляемой среде' -sidebar_title: 'Обновление в самоуправляемой среде' -slug: /operations/update -title: 'Обновление в самоуправляемой среде' -doc_type: 'guide' ---- - - - -## Обзор обновления ClickHouse {#clickhouse-upgrade-overview} - -В этом документе: -- общие рекомендации -- рекомендуемый план -- подробная информация об обновлении бинарных файлов на ваших системах - - - -## Общие рекомендации {#general-guidelines} - -Эти примечания помогут вам при планировании и объяснят, почему далее в документе мы даем такие рекомендации. - -### Обновляйте сервер ClickHouse отдельно от ClickHouse Keeper или ZooKeeper {#upgrade-clickhouse-server-separately-from-clickhouse-keeper-or-zookeeper} -Если для ClickHouse Keeper или Apache ZooKeeper не требуется установка патча безопасности, нет необходимости обновлять Keeper при обновлении сервера ClickHouse. Стабильность Keeper необходима в процессе обновления, поэтому завершите обновление серверов ClickHouse, прежде чем рассматривать обновление Keeper. - -### Минорные обновления следует выполнять часто {#minor-version-upgrades-should-be-adopted-often} -Настоятельно рекомендуется всегда обновляться до последней минорной версии сразу после ее выхода. Минорные релисы не содержат несовместимых изменений, но включают важные исправления ошибок (и могут содержать исправления уязвимостей). - -### Тестируйте экспериментальные функции на отдельном сервере ClickHouse с целевой версией {#test-experimental-features-on-a-separate-clickhouse-server-running-the-target-version} - -Совместимость экспериментальных функций может быть нарушена в любой момент и любым образом. Если вы используете экспериментальные функции, изучите журналы изменений и рассмотрите возможность развертывания отдельного сервера ClickHouse с установленной целевой версией и протестируйте использование экспериментальных функций там. - -### Откаты версий {#downgrades} -Если вы обновили систему и затем обнаружили, что новая версия несовместима с какой-то критически важной для вас функциональностью, вы можете выполнить откат на недавнюю версию (вышедшую менее года назад), при условии что вы еще не начали использовать какие-либо новые функции. После начала использования новых функций откат становится невозможным. - -### Несколько версий сервера ClickHouse в одном кластере {#multiple-clickhouse-server-versions-in-a-cluster} - -Мы прилагаем усилия, чтобы поддерживать окно совместимости длительностью один год (включая 2 LTS-версии). Это означает, что любые две версии должны уметь работать совместно в кластере, если разница между ними менее одного года (или между ними менее двух LTS-версий). Однако рекомендуется как можно быстрее обновить все узлы кластера до одной версии, так как возможны некоторые незначительные проблемы (такие как замедление распределенных запросов, повторяемые ошибки в некоторых фоновых операциях в ReplicatedMergeTree и т. п.). - -Мы никогда не рекомендуем запускать в одном кластере версии, даты релиза которых отличаются более чем на год. Хотя мы не ожидаем потери данных, кластер может стать непригодным для использования. Проблемы, которые вы можете ожидать при разнице версий более одного года, включают: - -- кластер может не работать -- некоторые (или даже все) запросы могут завершаться с произвольными ошибками -- в логах могут появляться произвольные ошибки/предупреждения -- откат может оказаться невозможным - -### Пошаговые (инкрементальные) обновления {#incremental-upgrades} - -Если разница между текущей версией и целевой версией превышает один год, рекомендуется либо: -- выполнить обновление с простоем (остановить все серверы, обновить все серверы, запустить все серверы); -- либо обновиться через промежуточную версию (версию, вышедшую менее чем на год позже текущей). - - - -## Рекомендуемый план {#recommended-plan} - -Ниже приведены рекомендуемые шаги для обновления ClickHouse без простоя: - -1. Убедитесь, что ваши изменения конфигурации не находятся в файле по умолчанию `/etc/clickhouse-server/config.xml`, а вместо этого размещены в `/etc/clickhouse-server/config.d/`, так как `/etc/clickhouse-server/config.xml` может быть перезаписан во время обновления. -2. Ознакомьтесь с [журналом изменений](/whats-new/changelog/index.md), чтобы выявить изменения, нарушающие обратную совместимость (двигаясь от целевого релиза к релизу, который вы используете в данный момент). -3. Внесите все изменения, выявленные как нарушающие обратную совместимость и которые можно сделать до обновления, а также составьте список изменений, которые потребуется внести после обновления. -4. Определите одну или несколько реплик для каждого шарда, которые будут оставаться в работе, пока остальные реплики каждого шарда обновляются. -5. На репликах, которые будут обновляться, по одной за раз: - -* остановите сервер ClickHouse -* обновите сервер до целевой версии -* запустите сервер ClickHouse -* дождитесь сообщений от Keeper, указывающих на стабильное состояние системы -* переходите к следующей реплике - -6. Проверьте наличие ошибок в журнале Keeper и журнале ClickHouse. -7. Обновите реплики, определенные на шаге 4, до новой версии. -8. Обратитесь к списку изменений, выполненных на шагах с 1 по 3, и внесите изменения, которые должны быть сделаны после обновления. - -:::note -Такое сообщение об ошибке ожидаемо, когда в реплицируемой среде одновременно работают несколько версий ClickHouse. Вы перестанете видеть эти сообщения, когда все реплики будут обновлены до одной и той же версии. - -```text -MergeFromLogEntryTask: Code: 40. DB::Exception: Контрольные суммы частей не совпадают: -хеш несжатых файлов не совпадает. (CHECKSUM_DOESNT_MATCH) Данные после слияния побайтово -не идентичны данным на других репликах. -``` - -::: - - -## Процесс обновления исполняемого файла сервера ClickHouse {#clickhouse-server-binary-upgrade-process} - -Если ClickHouse был установлен из пакетов `deb`, выполните на сервере следующие команды: - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-client clickhouse-server -$ sudo service clickhouse-server restart -``` - -Если вы устанавливали ClickHouse не с помощью рекомендуемых пакетов `deb`, используйте соответствующий метод обновления. - -:::note -Вы можете обновлять несколько серверов одновременно при условии, что ни в какой момент все реплики одного шарда не находятся одновременно офлайн. -::: - -Обновление более старой версии ClickHouse до заданной версии: - -Например: - -`xx.yy.a.b` — это текущая стабильная версия. Самую свежую стабильную версию можно найти [здесь](https://github.com/ClickHouse/ClickHouse/releases) - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-server=xx.yy.a.b clickhouse-client=xx.yy.a.b clickhouse-common-static=xx.yy.a.b -$ sudo service clickhouse-server restart -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md deleted file mode 100644 index f61fe18cdd8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -description: 'механизм кэширования, позволяющий размещать -данные в памяти пользовательского процесса, а не полагаться на кэш страниц ОС.' -sidebar_label: 'Кэш страниц в пространстве пользователя' -sidebar_position: 65 -slug: /operations/userspace-page-cache -title: 'Кэш страниц в пространстве пользователя' -doc_type: 'reference' ---- - - - -# Кэш страниц в пространстве пользователя {#userspace-page-cache} - - - -## Обзор {#overview} - -> Кэш страниц в пространстве пользователя — это новый механизм кэширования, который позволяет хранить данные в памяти процесса, а не полагаться на страничный кэш ОС. - -ClickHouse уже предлагает [Filesystem cache](/docs/operations/storing-data) -для кэширования поверх удалённого объектного хранилища, такого как Amazon S3, Google -Cloud Storage (GCS) или Azure Blob Storage. Кэш страниц в пространстве пользователя предназначен -для ускорения доступа к удалённым данным, когда обычное кэширование ОС работает -недостаточно эффективно. - -Он отличается от кэша файловой системы следующим: - -| Filesystem Cache | Кэш страниц в пространстве пользователя | -|---------------------------------------------------------|-----------------------------------------| -| Записывает данные на локальную файловую систему | Находится только в памяти | -| Занимает место на диске (может быть настроен на tmpfs) | Независим от файловой системы | -| Сохраняется при перезапусках сервера | Не сохраняется при перезапусках сервера | -| Не отражается в показателях использования памяти сервера | Отражается в показателях использования памяти сервера | -| Подходит как для хранения на диске, так и в памяти (страничный кэш ОС) | **Подходит для бездисковых серверов** | - - - -## Настройка и использование {#configuration-settings-and-usage} - -### Использование {#usage} - -Чтобы включить кэш страниц в пользовательском пространстве, сначала настройте его на сервере: - -```bash -cat config.d/page_cache.yaml -page_cache_max_size: 100G -``` - -:::note -Кэш страниц в пространстве пользователя будет использовать до указанного объёма памяти, но -этот объём памяти не резервируется. Память будет вытесняться, когда она потребуется -для других потребностей сервера. -::: - -Затем включите использование кэша на уровне запроса: - -```sql -SET use_page_cache_for_disks_without_file_cache=1; -``` - -### Настройки {#settings} - -| Setting | Description | Default | -| ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -| `use_page_cache_for_disks_without_file_cache` | Использовать userspace page cache для удалённых дисков, у которых не включён файловый кэш. | `0` | -| `use_page_cache_with_distributed_cache` | Использовать userspace page cache при использовании distributed cache. | `0` | -| `read_from_page_cache_if_exists_otherwise_bypass_cache` | Использовать userspace page cache в пассивном режиме, аналогично [`read_from_filesystem_cache_if_exists_otherwise_bypass_cache`](/docs/operations/settings/settings#read_from_filesystem_cache_if_exists_otherwise_bypass_cache). | `0` | -| `page_cache_inject_eviction` | Userspace page cache будет иногда случайным образом сбрасывать некоторые страницы. Предназначено для тестирования. | `0` | -| `page_cache_block_size` | Размер файловых блоков для хранения в userspace page cache, в байтах. Все чтения, проходящие через кэш, будут округляться до кратного этому размеру. | `1048576` | -| `page_cache_history_window_ms` | Задержка перед тем, как освобождённая память может быть использована userspace page cache. | `1000` | -| `page_cache_policy` | Имя политики userspace page cache. | `SLRU` | -| `page_cache_size_ratio` | Размер защищённой очереди в userspace page cache по отношению к общему размеру кэша. | `0.5` | -| `page_cache_min_size` | Минимальный размер userspace page cache. | `104857600` | -| `page_cache_max_size` | Максимальный размер userspace page cache. Установите 0, чтобы отключить кэш. Если значение больше `page_cache_min_size`, размер кэша будет постоянно подстраиваться в этих пределах, чтобы использовать большую часть доступной памяти, удерживая общее использование памяти ниже лимита (`max_server_memory_usage`[`_to_ram_ratio`]). | `0` | -| `page_cache_free_memory_ratio` | Доля лимита памяти, которую следует держать свободной от userspace page cache. Аналогично параметру Linux `min_free_kbytes`. | `0.15` | -| `page_cache_lookahead_blocks` | При промахе в userspace page cache читать за один раз до указанного количества последовательных блоков из нижележащего хранилища, если их также нет в кэше. Размер каждого блока — `page_cache_block_size` байт. | `16` | -| `page_cache_shards` | Разбивать userspace page cache на указанное количество шардов для снижения конкуренции за мьютексы. Экспериментальная настройка, маловероятно, что улучшит производительность. | `4` | - - -## Связанные материалы {#related-content} -- [Кэш файловой системы](/docs/operations/storing-data) -- [Вебинар по релизу ClickHouse v25.3](https://www.youtube.com/live/iCKEzp0_Z2Q?feature=shared&t=1320) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md deleted file mode 100644 index 0f21aebf8d9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'Документация по clickhouse_backupview {#clickhouse_backupview}' -slug: /operations/utilities/backupview -title: 'clickhouse_backupview' -doc_type: 'reference' ---- - - - -# clickhouse_backupview {#clickhouse_backupview} - -Модуль Python для анализа бэкапов, созданных командой [BACKUP](/operations/backup). -Основная цель — получить информацию из бэкапа без его фактического восстановления. - -Этот модуль предоставляет функции для: -- перечисления файлов, входящих в бэкап -- чтения файлов из бэкапа -- получения в удобном для чтения виде полезной информации о базах данных, таблицах и частях (parts), содержащихся в бэкапе -- проверки целостности бэкапа - - - -## Пример: {#example} - -```python -from clickhouse_backupview import open_backup, S3, FileInfo -``` - - -# Откройте резервную копию. Можно также использовать локальный путь: {#open-a-backup-we-could-also-use-a-local-path} -# backup = open_backup("/backups/my_backup_1/") {#backup-open_backupbackupsmy_backup_1} -backup = open_backup(S3("uri", "access_key_id", "secret_access_key")) - - - -# Получить список баз данных в резервной копии. {#get-a-list-of-databasess-inside-the-backup} -print(backup.get_databases())) - - - -# Получить список таблиц, включённых в резервную копию, {#get-a-list-of-tables-inside-the-backup} -# и для каждой таблицы — её запрос CREATE и списки партиций и частей. {#and-for-each-table-its-create-query-and-a-list-of-parts-and-partitions} -for db in backup.get_databases(): - for tbl in backup.get_tables(database=db): - print(backup.get_create_query(database=db, table=tbl)) - print(backup.get_partitions(database=db, table=tbl)) - print(backup.get_parts(database=db, table=tbl)) - - - -# Извлеките всё из резервной копии. {#extract-everything-from-the-backup} -backup.extract_all(table="mydb.mytable", out='/tmp/my_backup_1/all/') - - - -# Извлеките данные конкретной таблицы. {#extract-the-data-of-a-specific-table} -backup.extract_table_data(table="mydb.mytable", out='/tmp/my_backup_1/mytable/') - - - -# Извлечь одну партицию. {#extract-a-single-partition} -backup.extract_table_data(table="mydb.mytable", partition="202201", out='/tmp/my_backup_1/202201/') - - - -# Извлечь отдельную часть. {#extract-a-single-part} - -backup.extract_table_data(table="mydb.mytable", part="202201_100_200_3", out='/tmp/my_backup_1/202201_100_200_3/') - -``` - -Дополнительные примеры см. в [тестах](https://github.com/ClickHouse/ClickHouse/blob/master/utils/backupview/test/test.py). -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md deleted file mode 100644 index 743a70b9aac..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'Документация по утилите clickhouse-benchmark' -sidebar_label: 'clickhouse-benchmark' -sidebar_position: 61 -slug: /operations/utilities/clickhouse-benchmark -title: 'clickhouse-benchmark' -doc_type: 'reference' ---- - - - -# clickhouse-benchmark {#clickhouse-benchmark} - -Подключается к серверу ClickHouse и многократно отправляет заданные запросы. - -**Синтаксис** - -```bash -$ clickhouse-benchmark --query ["single query"] [keys] -``` - -или - -```bash -$ echo "единичный запрос" | clickhouse-benchmark [keys] -``` - -или - -```bash -$ clickhouse-benchmark [keys] <<< "single query" -``` - -Если вы хотите отправить набор запросов, создайте текстовый файл и поместите каждый запрос на отдельной строке в этом файле. Например: - -```sql -SELECT * FROM system.numbers LIMIT 10000000; -SELECT 1; -``` - -Затем передайте этот файл в стандартный ввод программы `clickhouse-benchmark`: - -```bash -clickhouse-benchmark [keys] < queries_file; -``` - - -## Параметры командной строки {#clickhouse-benchmark-command-line-options} - -- `--query=QUERY` — Запрос для выполнения. Если этот параметр не передан, `clickhouse-benchmark` будет читать запросы из стандартного ввода. -- `--query_id=ID` — ID запроса. -- `--query_id_prefix=ID_PREFIX` — Префикс ID запроса. -- `-c N`, `--concurrency=N` — Количество запросов, которые `clickhouse-benchmark` отправляет одновременно. Значение по умолчанию: 1. -- `-C N`, `--max_concurrency=N` — Постепенно увеличивает количество параллельных запросов до заданного значения, формируя один отчет для каждого уровня параллелизма. -- `--precise` — Включает точную поквартальную отчетность по интервалам с взвешенными метриками. -- `-d N`, `--delay=N` — Интервал в секундах между промежуточными отчетами (для отключения отчетов укажите 0). Значение по умолчанию: 1. -- `-h HOST`, `--host=HOST` — Хост сервера. Значение по умолчанию: `localhost`. Для [режима сравнения](#clickhouse-benchmark-comparison-mode) можно использовать несколько параметров `-h`. -- `-i N`, `--iterations=N` — Общее количество запросов. Значение по умолчанию: 0 (повторять бесконечно). -- `-r`, `--randomize` — Случайный порядок выполнения запросов, если на вход подано более одного запроса. -- `-s`, `--secure` — Использование соединения `TLS`. -- `-t N`, `--timelimit=N` — Ограничение по времени в секундах. `clickhouse-benchmark` прекращает отправку запросов при достижении указанного ограничения по времени. Значение по умолчанию: 0 (ограничение по времени отключено). -- `--port=N` — Порт сервера. Значение по умолчанию: 9000. Для [режима сравнения](#clickhouse-benchmark-comparison-mode) можно использовать несколько параметров `--port`. -- `--confidence=N` — Уровень доверия для t-теста. Возможные значения: 0 (80%), 1 (90%), 2 (95%), 3 (98%), 4 (99%), 5 (99.5%). Значение по умолчанию: 5. В [режиме сравнения](#clickhouse-benchmark-comparison-mode) `clickhouse-benchmark` выполняет [t-тест Стьюдента для двух независимых выборок](https://en.wikipedia.org/wiki/Student%27s_t-test#Independent_two-sample_t-test), чтобы определить, не различаются ли две выборки при выбранном уровне доверия. -- `--cumulative` — Вывод накопленных данных вместо данных по интервалам. -- `--database=DATABASE_NAME` — Имя базы данных ClickHouse. Значение по умолчанию: `default`. -- `--user=USERNAME` — Имя пользователя ClickHouse. Значение по умолчанию: `default`. -- `--password=PSWD` — Пароль пользователя ClickHouse. Значение по умолчанию: пустая строка. -- `--stacktrace` — Вывод трассировок стека. При указании этого параметра `clickhouse-bencmark` выводит трассировки стека для исключений. -- `--stage=WORD` — Стадия обработки запроса на сервере. ClickHouse останавливает обработку запроса и возвращает ответ `clickhouse-benchmark` на указанной стадии. Возможные значения: `complete`, `fetch_columns`, `with_mergeable_state`. Значение по умолчанию: `complete`. -- `--roundrobin` — Вместо сравнения запросов для разных `--host`/`--port` просто выбирается один случайный `--host`/`--port` для каждого запроса, и запрос отправляется на него. -- `--reconnect=N` — Управление поведением повторного подключения. Возможные значения: 0 (никогда не переподключаться), 1 (переподключаться для каждого запроса) или N (переподключаться после каждых N запросов). Значение по умолчанию: 0. -- `--max-consecutive-errors=N` — Количество допустимых последовательных ошибок. Значение по умолчанию: 0. -- `--ignore-error`, `--continue_on_errors` — Продолжать тестирование даже при неудачных запросах. -- `--client-side-time` — Показывать время с учетом сетевого взаимодействия вместо времени на стороне сервера; обратите внимание, что для версий сервера до 22.8 всегда показывается время на стороне клиента. -- `--proto-caps` — Включить/отключить чанкинг при передаче данных. Возможные варианты (могут быть разделены запятыми): `chunked_optional`, `notchunked`, `notchunked_optional`, `send_chunked`, `send_chunked_optional`, `send_notchunked`, `send_notchunked_optional`, `recv_chunked`, `recv_chunked_optional`, `recv_notchunked`, `recv_notchunked_optional`. Значение по умолчанию: `notchunked`. -- `--help` — Показывает справочное сообщение. -- `--verbose` — Увеличивает детализацию справочного сообщения. - -Если вы хотите применить некоторые [настройки](/operations/settings/overview) к запросам, передайте их в виде параметров `--= SETTING_VALUE`. Например, `--max_memory_usage=1048576`. - - - -## Параметры переменных окружения {#clickhouse-benchmark-environment-variable-options} - -Имя пользователя, пароль и хост могут быть заданы с помощью переменных окружения `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD` и `CLICKHOUSE_HOST`. -Аргументы командной строки `--user`, `--password` или `--host` имеют приоритет над переменными окружения. - - - -## Вывод {#clickhouse-benchmark-output} - -По умолчанию `clickhouse-benchmark` выводит отчет по каждому интервалу `--delay`. - -Пример отчета: - -```text -Queries executed: 10. - -localhost:9000, запросов 10, QPS: 6.772, RPS: 67904487.440, MiB/с: 518.070, результат RPS: 67721584.984, результат MiB/с: 516.675. - -0.000% 0.145 сек. -10.000% 0.146 сек. -20.000% 0.146 сек. -30.000% 0.146 сек. -40.000% 0.147 сек. -50.000% 0.148 сек. -60.000% 0.148 сек. -70.000% 0.148 сек. -80.000% 0.149 сек. -90.000% 0.150 сек. -95.000% 0.150 сек. -99.000% 0.150 сек. -99.900% 0.150 сек. -99.990% 0.150 сек. -``` - -В отчёте вы найдёте: - -* Количество запросов в поле `Queries executed:`. - -* Строку состояния, содержащую (в указанном порядке): - - * конечную точку сервера ClickHouse; - * количество обработанных запросов; - * QPS: сколько запросов сервер выполнил в секунду в течение периода, указанного в аргументе `--delay`; - * RPS: сколько строк сервер прочитал в секунду в течение периода, указанного в аргументе `--delay`; - * MiB/s: сколько мебибайт сервер прочитал в секунду в течение периода, указанного в аргументе `--delay`; - * result RPS: сколько строк сервер поместил в результат запроса в секунду в течение периода, указанного в аргументе `--delay`; - * result MiB/s: сколько мебибайт сервер поместил в результат запроса в секунду в течение периода, указанного в аргументе `--delay`. - -* Перцентили времени выполнения запросов. - - -## Режим сравнения {#clickhouse-benchmark-comparison-mode} - -`clickhouse-benchmark` может сравнивать производительность двух запущенных серверов ClickHouse. - -Чтобы использовать режим сравнения, укажите конечные точки обоих серверов двумя парами ключей `--host`, `--port`. Ключи сопоставляются по позициям в списке аргументов: первый `--host` — с первым `--port` и так далее. `clickhouse-benchmark` устанавливает подключения к обоим серверам, затем отправляет запросы. Каждый запрос отправляется на случайно выбранный сервер. Результаты отображаются в таблице. - - - -## Пример {#clickhouse-benchmark-example} - -```bash -$ echo "SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000" | clickhouse-benchmark --host=localhost --port=9001 --host=localhost --port=9000 -i 10 -``` - -```text -Загружен 1 запрос. - -Выполнено запросов: 5. - -localhost:9001, запросов 2, QPS: 3.764, RPS: 75446929.370, МиБ/с: 575.614, result RPS: 37639659.982, result МиБ/с: 287.168. -localhost:9000, запросов 3, QPS: 3.815, RPS: 76466659.385, МиБ/с: 583.394, result RPS: 38148392.297, result МиБ/с: 291.049. - -0.000% 0.258 sec. 0.250 sec. -10.000% 0.258 sec. 0.250 sec. -20.000% 0.258 sec. 0.250 sec. -30.000% 0.258 sec. 0.267 sec. -40.000% 0.258 sec. 0.267 sec. -50.000% 0.273 sec. 0.267 sec. -60.000% 0.273 sec. 0.267 sec. -70.000% 0.273 sec. 0.267 sec. -80.000% 0.273 sec. 0.269 sec. -90.000% 0.273 sec. 0.269 sec. -95.000% 0.273 sec. 0.269 sec. -99.000% 0.273 sec. 0.269 sec. -99.900% 0.273 sec. 0.269 sec. -99.990% 0.273 sec. 0.269 sec. - -Различия не выявлены при уровне достоверности 99,5% -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md deleted file mode 100644 index b59ec2071f1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'Документация по ClickHouse Compressor' -slug: /operations/utilities/clickhouse-compressor -title: 'clickhouse-compressor' -doc_type: 'reference' ---- - -Простая программа для сжатия и распаковки данных. - -### Примеры {#examples} - -Сжатие данных с помощью LZ4: - -```bash -$ ./clickhouse-compressor < input_file > output_file -``` - -Распаковка данных из формата LZ4: - -```bash -$ ./clickhouse-compressor --decompress < input_file > output_file -``` - -Сжимайте данные с помощью ZSTD с уровнем сжатия 5: - -```bash -$ ./clickhouse-compressor --codec 'ZSTD(5)' < input_file > output_file -``` - -Сжимайте данные с дельтой 4 байта и ZSTD уровня 10. - -```bash -$ ./clickhouse-compressor --codec 'Delta(4)' --codec 'ZSTD(10)' < input_file > output_file -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md deleted file mode 100644 index d1f2fc806f3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Документация по clickhouse-disks' -sidebar_label: 'clickhouse-disks' -sidebar_position: 59 -slug: /operations/utilities/clickhouse-disks -title: 'Clickhouse-disks' -doc_type: 'reference' ---- - - - -# Clickhouse-disks {#clickhouse-disks} - -Утилита, предоставляющая операции, аналогичные операциям файловой системы, для дисков ClickHouse. Может работать как в интерактивном, так и в неинтерактивном режиме. - - - -## Глобальные параметры программы {#program-wide-options} - -* `--config-file, -C` -- путь к конфигурационному файлу ClickHouse, по умолчанию `/etc/clickhouse-server/config.xml`. -* `--save-logs` -- записывать ход выполнения вызываемых команд в `/var/log/clickhouse-server/clickhouse-disks.log`. -* `--log-level` -- какой [тип](../server-configuration-parameters/settings#logger) событий логировать, по умолчанию `none`. -* `--disk` -- какой диск использовать для команд `mkdir, move, read, write, remove`. По умолчанию `default`. -* `--query, -q` -- одиночный запрос, который может быть выполнен без запуска интерактивного режима. -* `--help, -h` -- вывести все параметры и команды с описанием. - - - -## Отложенная инициализация {#lazy-initialization} -Все диски, указанные в конфигурации, инициализируются отложенно. Это означает, что объект диска создаётся только тогда, когда этот диск используется в какой-либо команде. Это сделано для повышения устойчивости утилиты и чтобы избежать обращения к дискам, которые описаны в конфигурации, но не используются пользователем и могут выйти из строя во время инициализации. Однако должен быть диск, который инициализируется при запуске clickhouse-disks. Этот диск задаётся параметром командной строки `--disk` (значение по умолчанию — `default`). - - - -## Диски по умолчанию {#default-disks} -После запуска доступны два диска, которые не указаны в конфигурации, но доступны для инициализации. - -1. **Диск `local`**: Этот диск предназначен для имитации локальной файловой системы, из которой была запущена утилита `clickhouse-disks`. Его начальный путь — это каталог, из которого был запущен `clickhouse-disks`, и он смонтирован в корневой каталог файловой системы. - -2. **Диск `default`**: Этот диск смонтирован в локальную файловую систему в каталоге, указанном параметром `clickhouse/path` в конфигурации (значение по умолчанию — `/var/lib/clickhouse`). Его начальный путь установлен в `/`. - - - -## Состояние Clickhouse-disks {#clickhouse-disks-state} -Для каждого добавленного диска утилита хранит текущий каталог (как в обычной файловой системе). Пользователь может изменять текущий каталог и переключаться между дисками. - -Состояние отображается в приглашении командной строки "`disk_name`:`path_name`" - - - -## Команды {#commands} - -В этой документации все обязательные позиционные аргументы обозначаются как ``, именованные аргументы — как `[--parameter value]`. Любой позиционный параметр может быть указан как именованный параметр с соответствующим именем. - -* `cd (change-dir, change_dir) [--disk disk] ` - Перейти в каталог `path` на диске `disk` (значение по умолчанию — текущий диск). Переключения диска не происходит. -* `copy (cp) [--disk-from disk_1] [--disk-to disk_2] `. - Рекурсивно скопировать данные из `path-from` на диске `disk_1` (значение по умолчанию — текущий диск (параметр `disk` в неинтерактивном режиме)) - в `path-to` на диске `disk_2` (значение по умолчанию — текущий диск (параметр `disk` в неинтерактивном режиме)). -* `current_disk_with_path (current, current_disk, current_path)` - Вывести текущее состояние в формате: - `Disk: "current_disk" Path: "текущий путь на текущем диске"` -* `help []` - Вывести справку по команде `command`. Если `command` не указана, вывести информацию обо всех командах. -* `move (mv) `. - Переместить файл или каталог из `path-from` в `path-to` в пределах текущего диска. -* `remove (rm, delete) `. - Рекурсивно удалить `path` на текущем диске. -* `link (ln) `. - Создать жёсткую ссылку от `path-from` к `path-to` на текущем диске. -* `list (ls) [--recursive] ` - Вывести список файлов по пути `path` на текущем диске. По умолчанию — нерекурсивно. -* `list-disks (list_disks, ls-disks, ls_disks)`. - Вывести имена дисков. -* `mkdir [--recursive] ` на текущем диске. - Создать каталог. По умолчанию — нерекурсивно. -* `read (r) [--path-to path]` - Прочитать файл из `path-from` в `path` (`stdout`, если не указан). -* `switch-disk [--path path] ` - Переключиться на диск `disk` по пути `path` (если `path` не указан, значением по умолчанию является предыдущий путь на диске `disk`). -* `write (w) [--path-from path] `. - Записать файл из `path` (`stdin`, если `path` не указан, ввод должен завершаться сочетанием Ctrl+D) в `path-to`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md deleted file mode 100644 index ff4dc73d126..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: 'Руководство по работе с утилитой clickhouse-format для форматов данных ClickHouse' -slug: /operations/utilities/clickhouse-format -title: 'clickhouse-format' -doc_type: 'reference' ---- - -# Утилита clickhouse-format {#clickhouse-format-utility} - -Позволяет форматировать входные запросы. - -Ключи: - -* `--help` или `-h` — вывести справочное сообщение. -* `--query` — форматировать запросы любой длины и сложности. -* `--hilite` или `--highlight` — добавить подсветку синтаксиса с использованием управляющих последовательностей терминала ANSI. -* `--oneline` — форматировать в одну строку. -* `--max_line_length` — форматировать в одну строку запросы с длиной меньше указанной. -* `--comments` — сохранять комментарии в выводе. -* `--quiet` или `-q` — только проверить синтаксис, без вывода при успешной проверке. -* `--multiquery` или `-n` — разрешить несколько запросов в одном файле. -* `--obfuscate` — обфусцировать вместо форматирования. -* `--seed ` — инициализирующая строка (seed), определяющая результат обфускации. -* `--backslash` — добавить обратный слэш в конец каждой строки форматированного запроса. Может быть полезно, когда вы копируете многострочный запрос из веба или ещё откуда-то и хотите выполнить его в командной строке. -* `--semicolons_inline` — в режиме multiquery записывать точки с запятой в последней строке запроса вместо новой строки. - -## Примеры {#examples} - -1. Форматирование запроса: - -```bash -$ clickhouse-format --query "select number from numbers(10) where number%2 order by number desc;" -``` - -Результат: - -```bash -SELECT number -FROM numbers(10) -WHERE number % 2 -ORDER BY number DESC -``` - -2. Подсветка и однострочные фрагменты: - -```bash -$ clickhouse-format --oneline --hilite <<< "SELECT sum(number) FROM numbers(5);" -``` - -Результат: - -```sql -SELECT sum(number) FROM numbers(5) -``` - -3. Множественные запросы: - -```bash -$ clickhouse-format -n <<< "SELECT min(number) FROM numbers(5); SELECT max(number) FROM numbers(5);" -``` - -Результат: - -```sql -SELECT min(number) -FROM numbers(5) -; - -SELECT max(number) -FROM numbers(5) -; - -``` - -4. Обфускация данных: - -```bash -$ clickhouse-format --seed Hello --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -Результат: - -```sql -SELECT treasury_mammoth_hazelnut BETWEEN nutmeg AND span, CASE WHEN chive >= 116 THEN switching ELSE ANYTHING END; -``` - -Тот же запрос, но другая строка-затравка: - -```bash -$ clickhouse-format --seed World --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -Результат: - -```sql -SELECT horse_tape_summer BETWEEN folklore AND moccasins, CASE WHEN intestine >= 116 THEN nonconformist ELSE FORESTRY END; -``` - -5. Добавление обратного слеша: - -```bash -$ clickhouse-format --backslash <<< "SELECT * FROM (SELECT 1 AS x UNION ALL SELECT 1 UNION DISTINCT SELECT 3);" -``` - -Результат: - -```sql -SELECT * \ -FROM \ -( \ - SELECT 1 AS x \ - UNION ALL \ - SELECT 1 \ - UNION DISTINCT \ - SELECT 3 \ -) -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md deleted file mode 100644 index aa77371efcd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Документация по клиентской утилите ClickHouse Keeper' -sidebar_label: 'clickhouse-keeper-client' -slug: /operations/utilities/clickhouse-keeper-client -title: 'Утилита clickhouse-keeper-client' -doc_type: 'reference' ---- - - - -# Утилита clickhouse-keeper-client {#clickhouse-keeper-client-utility} - -Клиентское приложение для взаимодействия с clickhouse-keeper по его родному протоколу. - - - -## Ключи {#clickhouse-keeper-client} - -- `-q QUERY`, `--query=QUERY` — Запрос для выполнения. Если этот параметр не передан, `clickhouse-keeper-client` запустится в интерактивном режиме. -- `-h HOST`, `--host=HOST` — Хост сервера. Значение по умолчанию: `localhost`. -- `-p N`, `--port=N` — Порт сервера. Значение по умолчанию: 9181. -- `-c FILE_PATH`, `--config-file=FILE_PATH` — Установить путь к конфигурационному файлу для получения строки подключения. Значение по умолчанию: `config.xml`. -- `--connection-timeout=TIMEOUT` — Установить время ожидания подключения в секундах. Значение по умолчанию: 10s. -- `--session-timeout=TIMEOUT` — Установить время ожидания сеанса в секундах. Значение по умолчанию: 10s. -- `--operation-timeout=TIMEOUT` — Установить время ожидания операции в секундах. Значение по умолчанию: 10s. -- `--history-file=FILE_PATH` — Установить путь к файлу истории. Значение по умолчанию: `~/.keeper-client-history`. -- `--log-level=LEVEL` — Установить уровень логирования. Значение по умолчанию: `information`. -- `--no-confirmation` — Если указан, не будет требоваться подтверждение для ряда команд. Значение по умолчанию: `false` для интерактивного режима и `true` для запроса. -- `--help` — Показать справочное сообщение. - - - -## Пример {#clickhouse-keeper-client-example} - -```bash -./clickhouse-keeper-client -h localhost -p 9181 --connection-timeout 30 --session-timeout 30 --operation-timeout 30 -Подключено к ZooKeeper по адресу [::1]:9181 с идентификатором сеанса 137 -/ :) ls -keeper foo bar -/ :) cd 'keeper' -/keeper :) ls -api_version -/keeper :) cd 'api_version' -/keeper/api_version :) ls - -/keeper/api_version :) cd 'xyz' -Путь /keeper/api_version/xyz не существует -/keeper/api_version :) cd ../../ -/ :) ls -keeper foo bar -/ :) get 'keeper/api_version' -2 -``` - - -## Команды {#clickhouse-keeper-client-commands} - -- `ls '[path]'` -- Выводит список узлов для указанного пути (по умолчанию: cwd) -- `cd '[path]'` -- Изменяет рабочий путь (по умолчанию `.`) -- `cp '' ''` -- Копирует узел `src` в путь `dest` -- `cpr '' ''` -- Копирует поддерево узла `src` в путь `dest` -- `mv '' ''` -- Перемещает узел `src` в путь `dest` -- `mvr '' ''` -- Перемещает поддерево узла `src` в путь `dest` -- `exists ''` -- Возвращает `1`, если узел существует, и `0` в противном случае -- `set '' [version]` -- Обновляет значение узла. Обновляет только, если версия совпадает (по умолчанию: -1) -- `create '' [mode]` -- Создаёт новый узел с указанным значением -- `touch ''` -- Создаёт новый узел с пустой строкой в качестве значения. Не вызывает исключение, если узел уже существует -- `get ''` -- Возвращает значение узла -- `rm '' [version]` -- Удаляет узел только, если версия совпадает (по умолчанию: -1) -- `rmr '' [limit]` -- Рекурсивно удаляет путь, если размер поддерева меньше лимита. Требуется подтверждение (лимит по умолчанию = 100) -- `flwc ` -- Выполняет четырёхбуквенную команду -- `help` -- Выводит это сообщение -- `get_direct_children_number '[path]'` -- Возвращает количество непосредственных дочерних узлов для указанного пути -- `get_all_children_number '[path]'` -- Возвращает общее количество дочерних узлов для указанного пути -- `get_stat '[path]'` -- Возвращает статистику узла (по умолчанию `.`) -- `find_super_nodes '[path]'` -- Находит узлы с числом дочерних узлов, превышающим заданный порог, для указанного пути (по умолчанию `.`) -- `delete_stale_backups` -- Удаляет узлы ClickHouse, используемые для резервных копий, которые сейчас неактивны -- `find_big_family [path] [n]` -- Возвращает первые `n` узлов с наибольшим числом дочерних узлов в поддереве (путь по умолчанию = `.` и n = 10) -- `sync ''` -- Синхронизирует узел между процессами и лидером -- `reconfig "" [version]` -- Изменяет конфигурацию кластера Keeper. См. /docs/en/guides/sre/keeper/clickhouse-keeper#reconfiguration diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md deleted file mode 100644 index a66925cddc2..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md +++ /dev/null @@ -1,323 +0,0 @@ ---- -description: 'Руководство по использованию clickhouse-local для обработки данных без сервера' -sidebar_label: 'clickhouse-local' -sidebar_position: 60 -slug: /operations/utilities/clickhouse-local -title: 'clickhouse-local' -doc_type: 'reference' ---- - - - -# clickhouse-local {#clickhouse-local} - - - -## Когда использовать clickhouse-local и когда ClickHouse {#when-to-use-clickhouse-local-vs-clickhouse} - -`clickhouse-local` — это простая в использовании версия ClickHouse, которая идеально подходит разработчикам, которым нужно быстро обрабатывать локальные и удалённые файлы с помощью SQL без установки полноценного сервера баз данных. С `clickhouse-local` разработчики могут выполнять SQL-команды (используя [SQL-диалект ClickHouse](../../sql-reference/index.md)) прямо из командной строки, что обеспечивает простой и эффективный способ доступа к возможностям ClickHouse без необходимости полной установки ClickHouse. Одно из основных преимуществ `clickhouse-local` заключается в том, что он уже включён при установке [clickhouse-client](/operations/utilities/clickhouse-local). Это означает, что разработчики могут быстро начать работу с `clickhouse-local` без сложного процесса установки. - -Хотя `clickhouse-local` — отличный инструмент для разработки и тестирования, а также для обработки файлов, он не подходит для обслуживания конечных пользователей или приложений. В таких сценариях рекомендуется использовать open‑source [ClickHouse](/install). ClickHouse — это мощная OLAP-база данных, предназначенная для работы с крупномасштабными аналитическими нагрузками. Она обеспечивает быстрое и эффективное выполнение сложных запросов по большим наборам данных, что делает её идеальной для производственных сред, где критична высокая производительность. Кроме того, ClickHouse предлагает широкий набор функций, таких как репликация, шардинг и высокая доступность, которые необходимы для масштабирования при работе с большими объёмами данных и обслуживания приложений. Если вам нужно обрабатывать большие наборы данных или обслуживать конечных пользователей либо приложения, мы рекомендуем использовать open‑source ClickHouse вместо `clickhouse-local`. - -Ознакомьтесь с документацией ниже, где приведены примеры сценариев использования `clickhouse-local`, таких как [запрос к локальному файлу](#query_data_in_file) или [чтение parquet-файла в S3](#query-data-in-a-parquet-file-in-aws-s3). - - - -## Скачайте clickhouse-local {#download-clickhouse-local} - -`clickhouse-local` использует тот же бинарный файл `clickhouse`, который запускает сервер ClickHouse и `clickhouse-client`. Проще всего скачать последнюю версию с помощью следующей команды: - -```bash -curl https://clickhouse.com/ | sh -``` - -:::note -Скачанный вами бинарный файл может запускать самые разные инструменты и утилиты ClickHouse. Если вы хотите запустить ClickHouse как сервер базы данных, ознакомьтесь с разделом [Quick Start](/get-started/quick-start). -::: - - -## Запрос данных из файла с помощью SQL {#query_data_in_file} - -Типичный способ использования `clickhouse-local` — выполнение разовых произвольных запросов к файлам, когда вам не нужно предварительно загружать данные в таблицу. `clickhouse-local` может потоково считывать данные из файла во временную таблицу и выполнять ваши SQL-запросы. - -Если файл находится на той же машине, что и `clickhouse-local`, вы можете просто указать его для загрузки. Следующий файл `reviews.tsv` содержит выборку отзывов о товарах с Amazon: - -```bash -./clickhouse local -q "SELECT * FROM 'reviews.tsv'" -``` - -Эта команда является сокращённой формой следующей команды: - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv')" -``` - -ClickHouse определяет, что файл имеет табличный формат с разделителем табуляцией, по его расширению. Если вам нужно явно задать формат, просто укажите один из [многочисленных форматов ввода ClickHouse](../../interfaces/formats.md): - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv', 'TabSeparated')" -``` - -Табличная функция `file` создаёт таблицу; вы можете использовать `DESCRIBE`, чтобы увидеть автоматически определённую схему: - -```bash -./clickhouse local -q "DESCRIBE file('reviews.tsv')" -``` - -:::tip -Можно использовать глоб-шаблоны в именах файлов (см. [glob substitutions](/sql-reference/table-functions/file.md/#globs-in-path)). - -Примеры: - -```bash -./clickhouse local -q "SELECT * FROM 'reviews*.jsonl'" -./clickhouse local -q "SELECT * FROM 'review_?.csv'" -./clickhouse local -q "SELECT * FROM 'review_{1..3}.csv'" -``` - -::: - -```response -marketplace Nullable(String) -customer_id Nullable(Int64) -review_id Nullable(String) -product_id Nullable(String) -product_parent Nullable(Int64) -product_title Nullable(String) -product_category Nullable(String) -star_rating Nullable(Int64) -helpful_votes Nullable(Int64) -total_votes Nullable(Int64) -vine Nullable(String) -verified_purchase Nullable(String) -review_headline Nullable(String) -review_body Nullable(String) -review_date Nullable(Date) -``` - -Давайте найдём товар с самым высоким рейтингом: - -```bash -./clickhouse local -q "SELECT - argMax(product_title,star_rating), - max(star_rating) -FROM file('reviews.tsv')" -``` - -```response -Monopoly Junior Board Game 5 -``` - - -## Выполнение запросов к данным в файле Parquet в AWS S3 {#query-data-in-a-parquet-file-in-aws-s3} - -Если у вас есть файл в S3, используйте `clickhouse-local` и табличную функцию `s3`, чтобы выполнять запросы к файлу непосредственно (без вставки данных в таблицу ClickHouse). У нас есть файл `house_0.parquet` в публичном бакете, который содержит цены на объекты недвижимости, проданные в Соединённом Королевстве. Давайте посмотрим, сколько строк он содержит: - -```bash -./clickhouse local -q " -SELECT count() -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -Файл содержит 2,7 млн строк: - -```response -2772030 -``` - -Всегда полезно посмотреть, какую схему данных ClickHouse определяет на основании файла: - -```bash -./clickhouse local -q "DESCRIBE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -```response -price Nullable(Int64) -date Nullable(UInt16) -postcode1 Nullable(String) -postcode2 Nullable(String) -type Nullable(String) -is_new Nullable(UInt8) -duration Nullable(String) -addr1 Nullable(String) -addr2 Nullable(String) -street Nullable(String) -locality Nullable(String) -town Nullable(String) -district Nullable(String) -county Nullable(String) -``` - -Давайте посмотрим, какие самые дорогие районы: - -```bash -./clickhouse local -q " -SELECT - town, - district, - count() AS c, - round(avg(price)) AS price, - bar(price, 0, 5000000, 100) -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet') -GROUP BY - town, - district -HAVING c >= 100 -ORDER BY price DESC -LIMIT 10" -``` - -```response -LONDON CITY OF LONDON 886 2271305 █████████████████████████████████████████████▍ -LEATHERHEAD ELMBRIDGE 206 1176680 ███████████████████████▌ -LONDON CITY OF WESTMINSTER 12577 1108221 ██████████████████████▏ -LONDON KENSINGTON AND CHELSEA 8728 1094496 █████████████████████▉ -HYTHE FOLKESTONE AND HYTHE 130 1023980 ████████████████████▍ -CHALFONT ST GILES CHILTERN 113 835754 ████████████████▋ -AMERSHAM BUCKINGHAMSHIRE 113 799596 ███████████████▉ -VIRGINIA WATER RUNNYMEDE 356 789301 ███████████████▊ -BARNET ENFIELD 282 740514 ██████████████▊ -NORTHWOOD THREE RIVERS 184 731609 ██████████████▋ -``` - -:::tip -Когда будете готовы загружать свои файлы в ClickHouse, запустите сервер ClickHouse и вставьте результаты выполнения табличных функций `file` и `s3` в таблицу на движке `MergeTree`. Для получения дополнительной информации см. раздел [Быстрый старт](/get-started/quick-start). -::: - - -## Преобразование форматов {#format-conversions} - -Вы можете использовать `clickhouse-local` для преобразования данных между различными форматами. Пример: - -```bash -$ clickhouse-local --input-format JSONLines --output-format CSV --query "SELECT * FROM table" < data.json > data.csv -``` - -Форматы автоматически определяются на основании расширений файлов: - -```bash -$ clickhouse-local --query "SELECT * FROM table" < data.json > data.csv -``` - -Для краткости можно записать это с использованием аргумента `--copy`: - -```bash -$ clickhouse-local --copy < data.json > data.csv -``` - - -## Использование {#usage} - -По умолчанию `clickhouse-local` имеет доступ к данным сервера ClickHouse на том же хосте и не зависит от конфигурации сервера. Также поддерживается загрузка конфигурации сервера с помощью аргумента `--config-file`. Для временных данных по умолчанию создаётся отдельный уникальный каталог. - -Основное использование (Linux): - -```bash -$ clickhouse-local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -Базовое использование на Mac: - -```bash -$ ./clickhouse local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -:::note -`clickhouse-local` также поддерживается на Windows через WSL2. -::: - -Аргументы: - -* `-S`, `--structure` — структура таблицы для входных данных. -* `--input-format` — входной формат, по умолчанию `TSV`. -* `-F`, `--file` — путь к данным, по умолчанию `stdin`. -* `-q`, `--query` — запросы для выполнения с `;` в качестве разделителя. `--query` может быть указан несколько раз, например, `--query "SELECT 1" --query "SELECT 2"`. Не может использоваться одновременно с `--queries-file`. -* `--queries-file` — путь к файлу с запросами для выполнения. `--queries-file` может быть указан несколько раз, например, `--queries-file queries1.sql --queries-file queries2.sql`. Не может использоваться одновременно с `--query`. -* `--multiquery, -n` – если указана, после опции `--query` можно перечислить несколько запросов, разделённых точкой с запятой. Для удобства также можно опустить `--query` и передать запросы непосредственно после `--multiquery`. -* `-N`, `--table` — имя таблицы, в которую помещаются выходные данные, по умолчанию `table`. -* `-f`, `--format`, `--output-format` — выходной формат, по умолчанию `TSV`. -* `-d`, `--database` — база данных по умолчанию `_local`. -* `--stacktrace` — выводить отладочную информацию в случае исключения. -* `--echo` — печатать запрос перед выполнением. -* `--verbose` — более подробная информация о выполнении запроса. -* `--logger.console` — вывод логов в консоль. -* `--logger.log` — имя файла лога. -* `--logger.level` — уровень логирования. -* `--ignore-error` — не останавливать обработку, если запрос завершился ошибкой. -* `-c`, `--config-file` — путь к конфигурационному файлу в том же формате, что и для сервера ClickHouse; по умолчанию конфигурация пустая. -* `--no-system-tables` — не подключать системные таблицы. -* `--help` — справка по аргументам для `clickhouse-local`. -* `-V`, `--version` — вывести информацию о версии и завершить работу. - -Также существуют аргументы для каждой конфигурационной переменной ClickHouse, которые чаще используются вместо `--config-file`. - - -## Примеры {#examples} - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local --structure "a Int64, b Int64" \ - --input-format "CSV" --query "SELECT * FROM table" -Прочитано 2 строки, 32,00 Б за 0,000 сек., 5182 строк/сек., 80,97 КиБ/сек. -1 2 -3 4 -``` - -Предыдущий пример эквивалентен следующему: - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -n --query " - CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); - SELECT a, b FROM table; - DROP TABLE table;" -Прочитано 2 строки, 32,00 Б за 0,000 сек., 4987 строк/сек., 77,93 КиБ/сек. -1 2 -3 4 -``` - -Необязательно использовать `stdin` или аргумент `--file`, вы можете открывать любое количество файлов с помощью [табличной функции `file`](../../sql-reference/table-functions/file.md): - -```bash -$ echo 1 | tee 1.tsv -1 - -$ echo 2 | tee 2.tsv -2 - -$ clickhouse-local --query " - select * from file('1.tsv', TSV, 'a int') t1 - cross join file('2.tsv', TSV, 'b int') t2" -1 2 -``` - -Теперь выведем объём памяти, потребляемый каждым пользователем Unix: - -Запрос: - -```bash -$ ps aux | tail -n +2 | awk '{ printf("%s\t%s\n", $1, $4) }' \ - | clickhouse-local --structure "user String, mem Float64" \ - --query "SELECT user, round(sum(mem), 2) as memTotal - FROM table GROUP BY user ORDER BY memTotal DESC FORMAT Pretty" -``` - -Результат: - -```text -Прочитано 186 строк, 4,15 КиБ за 0,035 сек., 5302 строк/сек., 118,34 КиБ/сек. -┏━━━━━━━━━━┳━━━━━━━━━━┓ -┃ user ┃ memTotal ┃ -┡━━━━━━━━━━╇━━━━━━━━━━┩ -│ bayonet │ 113.5 │ -├──────────┼──────────┤ -│ root │ 8.8 │ -├──────────┼──────────┤ -... -``` - - -## Дополнительные материалы {#related-content-1} - -- [Извлечение, преобразование и выполнение запросов к данным в локальных файлах с использованием clickhouse-local](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) -- [Загрузка данных в ClickHouse — часть 1](https://clickhouse.com/blog/getting-data-into-clickhouse-part-1) -- [Анализ крупных реальных наборов данных: более 100 лет метеонаблюдений в ClickHouse](https://clickhouse.com/blog/real-world-data-noaa-climate-data) -- Блог: [Извлечение, преобразование и выполнение запросов к данным в локальных файлах с использованием clickhouse-local](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md deleted file mode 100644 index 5cadbc03d04..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Документация по Clickhouse Obfuscator' -slug: /operations/utilities/clickhouse-obfuscator -title: 'clickhouse-obfuscator' -doc_type: 'reference' ---- - -Простой инструмент для обфускации табличных данных. - -Он читает входную таблицу и создает выходную таблицу, которая сохраняет некоторые свойства исходных данных, но содержит другие значения. -Это позволяет публиковать данные, почти идентичные боевым, для использования в бенчмарках. - -Он спроектирован так, чтобы сохранять следующие свойства данных: - -- кардинальности значений (количество различных значений) для каждого столбца и каждой комбинации столбцов (кортежей столбцов); -- условные кардинальности: количество различных значений одного столбца при условии на значение другого столбца; -- распределения вероятностей абсолютного значения целых чисел; знак знаковых целых; порядок (экспоненту) и знак для чисел с плавающей запятой; -- распределения вероятностей длины строк; -- вероятность нулевых значений чисел; пустых строк и массивов, `NULL`; - -- коэффициент сжатия данных при сжатии кодеками семейства LZ77 и энтропийными кодеками; -- непрерывность (величину изменения) значений времени по всей таблице; непрерывность значений с плавающей запятой; -- компоненту даты в значениях `DateTime`; - -- корректность строковых значений в кодировке UTF-8; -- строковые значения выглядят естественно. - -Большинство свойств, перечисленных выше, подходят для тестирования производительности: - -чтение данных, фильтрация, агрегация и сортировка будут работать практически с той же скоростью, -что и на исходных данных, благодаря сохранению кардинальностей, порядков величин, коэффициентов сжатия и т. п. - -Работает детерминированно: вы задаете начальное значение (seed), и преобразование определяется входными данными и этим seed. -Некоторые преобразования взаимно однозначны и могут быть обращены, поэтому вам нужно использовать большой seed и держать его в секрете. - -Он использует некоторые криптографические примитивы для преобразования данных, но с криптографической точки зрения делает это некорректно, поэтому не следует считать результат надежно защищенным, если только у вас нет других оснований. Результат может сохранять некоторые данные, которые вы не хотите публиковать. - -Он всегда оставляет числа 0, 1, -1, даты, длины массивов и флаги null в точности такими же, как в исходных данных. -Например, у вас есть столбец `IsMobile` в таблице со значениями 0 и 1. В преобразованных данных он будет иметь те же значения. - -Таким образом, пользователь сможет посчитать точное соотношение мобильного трафика. - -Еще один пример. Если в вашей таблице есть приватные данные, такие как email пользователя, и вы не хотите публиковать ни один реальный адрес электронной почты. -Если таблица достаточно большая и содержит много различных email-адресов, и ни один из них не встречается значительно чаще других, все данные будут анонимизированы. Но если в столбце содержится небольшое количество различных значений, некоторые из них могут быть воспроизведены. -Вам следует ознакомиться с алгоритмом работы этого инструмента и тонко настроить его параметры командной строки. - -Этот инструмент корректно работает только при хотя бы умеренном объеме данных (не менее тысяч строк). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/index.md deleted file mode 100644 index 09efee5ea9a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: 'Страница со списком различных полезных инструментов и утилит ClickHouse.' -keywords: ['tools', 'utilities'] -sidebar_label: 'Список инструментов и утилит' -sidebar_position: 56 -slug: /operations/utilities/ -title: 'Список инструментов и утилит' -doc_type: 'landing-page' ---- - -| Инструмент/утилита | Описание | -|------|-------------| -|[clickhouse-local](../../operations/utilities/clickhouse-local.md) | Позволяет выполнять SQL-запросы к данным без запуска сервера ClickHouse, аналогично тому, как это делает `awk`.| -|[clickhouse-benchmark](../../operations/utilities/clickhouse-benchmark.md) | Нагружает сервер с пользовательскими запросами и настройками.| -| [clickhouse-format](../../operations/utilities/clickhouse-format.md) | Форматирует входящие запросы.| -|[ClickHouse obfuscator](../../operations/utilities/clickhouse-obfuscator.md) | Маскирует данные.| -|[ClickHouse compressor](../../operations/utilities/clickhouse-compressor.md) | Сжимает и распаковывает данные.| -| [clickhouse-disks](../../operations/utilities/clickhouse-disks.md) | Предоставляет операции с файлами, аналогичные файловой системе, между различными дисками ClickHouse.| -| [clickhouse-odbc-bridge](../../operations/utilities/odbc-bridge.md) | Прокси-сервер для ODBC-драйвера.| -| [clickhouse_backupview](../../operations/utilities/backupview.md) | Модуль на Python для анализа резервных копий ClickHouse.| \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md deleted file mode 100644 index 500a03ac8fa..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Документация по Odbc Bridge' -slug: /operations/utilities/odbc-bridge -title: 'clickhouse-odbc-bridge' -doc_type: 'reference' ---- - -Простой HTTP-сервер, который работает как прокси для драйвера ODBC. Основная мотивация -заключалась в возможных аварийных завершениях (segfault) или других сбоях в реализациях ODBC, которые могут -привести к падению всего процесса clickhouse-server. - -Этот инструмент работает по HTTP, а не через каналы (pipes), разделяемую память или TCP, потому что: -- это проще реализовать; -- это проще отлаживать; -- jdbc-bridge можно реализовать таким же образом. - - - -## Использование {#usage} - -`clickhouse-server` использует этот инструмент внутри табличной функции `odbc` и StorageODBC. -Однако он может использоваться как самостоятельный инструмент из командной строки -со следующими параметрами в URL POST-запроса: -- `connection_string` -- строка подключения ODBC. -- `sample_block` -- описание столбцов в формате ClickHouse NamesAndTypesList, имя в обратных - кавычках, тип в виде строки. Имя и тип разделены пробелом, строки разделены - переводом строки. -- `max_block_size` -- необязательный параметр, задает максимальный размер одного блока. -Запрос передается в теле POST-запроса. Ответ возвращается в формате RowBinary. - - - -## Пример: {#example} - -```bash -$ clickhouse-odbc-bridge --http-port 9018 --daemon - -$ curl -d "query=SELECT PageID, ImpID, AdType FROM Keys ORDER BY PageID, ImpID" --data-urlencode "connection_string=DSN=ClickHouse;DATABASE=stat" --data-urlencode "sample_block=columns format version: 1 -3 columns: -\`PageID\` String -\`ImpID\` String -\`AdType\` String -" "http://localhost:9018/" > result.txt - -$ cat result.txt -12246623837185725195925621517 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md b/i18n/ru/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md deleted file mode 100644 index aa8ff152d3a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md +++ /dev/null @@ -1,420 +0,0 @@ ---- -description: 'Документация по планированию рабочих нагрузок' -sidebar_label: 'Планирование рабочих нагрузок' -sidebar_position: 69 -slug: /operations/workload-scheduling -title: 'Планирование рабочих нагрузок' -doc_type: 'reference' ---- - -Когда ClickHouse выполняет несколько запросов одновременно, они могут использовать общие ресурсы (например, диски и ядра CPU). Для регулирования того, как ресурсы используются и разделяются между различными рабочими нагрузками, могут быть применены ограничения и политики планирования. Для всех ресурсов может быть настроена общая иерархия планирования. Корень иерархии представляет общие ресурсы, а листья — конкретные рабочие нагрузки, в которых накапливаются запросы, превышающие доступную емкость ресурсов. - -:::note -В настоящее время [операции ввода-вывода на удалённых дисках](#disk_config) и [CPU](#cpu_scheduling) могут планироваться описанным методом. Для гибкой настройки ограничений по памяти см. [Memory overcommit](settings/memory-overcommit.md) -::: - - - -## Конфигурация диска {#disk_config} - -Чтобы включить планирование нагрузки ввода-вывода (I/O) для конкретного диска, необходимо создать ресурсы чтения (READ) и записи (WRITE): - -```sql -CREATE RESOURCE resource_name (WRITE DISK disk_name, READ DISK disk_name) --- or -CREATE RESOURCE read_resource_name (WRITE DISK write_disk_name) -CREATE RESOURCE write_resource_name (READ DISK read_disk_name) -``` - -Ресурс может быть использован для произвольного числа дисков — только для READ, только для WRITE или одновременно для READ и WRITE. Также предусмотрен синтаксис, позволяющий использовать ресурс для всех дисков: - -```sql -CREATE RESOURCE all_io (READ ANY DISK, WRITE ANY DISK); -``` - -Альтернативный способ указать, какие диски используются ресурсом, — это `storage_configuration` сервера: - -:::warning -Планирование нагрузок с использованием конфигурации ClickHouse устарело. Вместо этого следует использовать SQL-синтаксис. -::: - -Чтобы включить планирование операций ввода-вывода (I/O) для конкретного диска, необходимо указать `read_resource` и/или `write_resource` в конфигурации хранилища. Это сообщает ClickHouse, какой ресурс должен использоваться для каждого запроса на чтение и запись с данным диском. Ресурсы чтения и записи могут ссылаться на один и тот же ресурс, что полезно для локальных SSD или HDD. Несколько разных дисков также могут ссылаться на один и тот же ресурс, что полезно для удалённых дисков, если вы хотите обеспечить справедливое распределение сетевой пропускной способности между, например, «production»‑ и «development»‑нагрузками. - -Пример: - -```xml - - - ... - - - s3 - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - network_read - network_write - - - - - -
- s3 -
-
-
-
-
-
-``` - -Обратите внимание, что параметры конфигурации сервера имеют приоритет по сравнению с определением ресурсов с помощью SQL. - - -## Разметка рабочих нагрузок {#workload_markup} - -Запросы могут быть помечены с помощью настройки `workload`, чтобы различать различные рабочие нагрузки. Если `workload` не задан, используется значение «default». Обратите внимание, что вы можете указать другое значение с помощью профилей настроек. Ограничения настроек можно использовать, чтобы сделать значение `workload` постоянным, если вы хотите, чтобы все запросы пользователя помечались фиксированным значением настройки `workload`. - -Можно задать настройку `workload` и для фоновой активности. Слияния и мутации используют, соответственно, серверные настройки `merge_workload` и `mutation_workload`. Эти значения также могут быть переопределены для конкретных таблиц с помощью настроек MergeTree `merge_workload` и `mutation_workload`. - -Рассмотрим пример системы с двумя различными рабочими нагрузками: «production» и «development». - -```sql -SELECT count() FROM my_table WHERE value = 42 SETTINGS workload = 'production' -SELECT count() FROM my_table WHERE value = 13 SETTINGS workload = 'development' -``` - - -## Иерархия планирования ресурсов {#hierarchy} - -С точки зрения подсистемы планирования ресурс рассматривается как иерархия узлов планирования. - -```mermaid -graph TD - subgraph network_read - nr_root(("/")) - -->|100 одновременных запросов| nr_fair("fair") - -->|75% полосы пропускания| nr_prod["prod"] - nr_fair - -->|25% полосы пропускания| nr_dev["dev"] - end - - subgraph network_write - nw_root(("/")) - -->|100 одновременных запросов| nw_fair("fair") - -->|75% полосы пропускания| nw_prod["prod"] - nw_fair - -->|25% полосы пропускания| nw_dev["dev"] - end -``` - -:::warning -Планирование рабочих нагрузок с использованием конфигурации clickhouse устарело. Вместо этого следует использовать SQL-синтаксис. SQL-синтаксис автоматически создаёт все необходимые узлы планирования, а приведённое ниже описание узлов планирования следует рассматривать как детали низкоуровневой реализации, доступные через таблицу [system.scheduler](/operations/system-tables/scheduler.md). -::: - -**Возможные типы узлов:** - -* `inflight_limit` (constraint) — блокирует, если количество одновременно выполняемых запросов превышает `max_requests` или их суммарная стоимость превышает `max_cost`; должен иметь одного потомка. -* `bandwidth_limit` (constraint) — блокирует, если текущая пропускная способность превышает `max_speed` (0 означает отсутствие ограничений) или всплеск превышает `max_burst` (по умолчанию равно `max_speed`); должен иметь одного потомка. -* `fair` (policy) — выбирает следующий запрос на обработку из одного из дочерних узлов в соответствии с max-min fairness; дочерние узлы могут указывать `weight` (по умолчанию 1). -* `priority` (policy) — выбирает следующий запрос на обработку из одного из дочерних узлов в соответствии со статическими приоритетами (меньшее значение означает более высокий приоритет); дочерние узлы могут указывать `priority` (по умолчанию 0). -* `fifo` (queue) — лист иерархии, способный удерживать запросы, которые превышают ёмкость ресурса. - -Чтобы иметь возможность использовать полную ёмкость базового ресурса, следует использовать `inflight_limit`. Обратите внимание, что слишком маленькое значение `max_requests` или `max_cost` может привести к неполному использованию ресурса, тогда как слишком большие значения могут привести к пустым очередям внутри планировщика, что, в свою очередь, приведёт к игнорированию политик (несправедливости или игнорированию приоритетов) в поддереве. С другой стороны, если вы хотите защитить ресурсы от чрезмерного использования, следует использовать `bandwidth_limit`. Он ограничивает скорость, когда объём ресурса, потреблённый за `duration` секунд, превышает `max_burst + max_speed * duration` байт. Два узла `bandwidth_limit` для одного и того же ресурса могут использоваться для ограничения пикового потребления ресурса на коротких интервалах и среднего потребления на более длинных. - -Следующий пример показывает, как определить иерархии планирования ввода-вывода (IO), показанные на рисунке: - -```xml - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - -``` - - -## Классификаторы рабочих нагрузок {#workload_classifiers} - -:::warning -Планирование рабочих нагрузок с использованием конфигурации ClickHouse устарело. Вместо этого следует использовать SQL-синтаксис. При использовании SQL-синтаксиса классификаторы создаются автоматически. -::: - -Классификаторы рабочих нагрузок используются для определения сопоставления `workload`, указанной в запросе, с листовыми очередями (leaf-queues), которые должны использоваться для конкретных ресурсов. В данный момент классификация рабочих нагрузок проста: доступно только статическое сопоставление. - -Пример: - -```xml - - - - /fair/prod - /fair/prod - - - /fair/dev - /fair/dev - - - /fair/dev - /fair/dev - - - -``` - - -## Иерархия рабочих нагрузок {#workloads} - -ClickHouse предоставляет удобный синтаксис SQL для определения иерархии планирования. Все ресурсы, созданные с помощью `CREATE RESOURCE`, используют одну и ту же структуру иерархии, но могут отличаться в некоторых аспектах. Для каждой рабочей нагрузки, созданной с помощью `CREATE WORKLOAD`, автоматически создаётся несколько узлов планирования для каждого ресурса. Дочернюю рабочую нагрузку можно создать внутри другой, родительской рабочей нагрузки. Ниже приведён пример, который определяет точно такую же иерархию, как и XML‑конфигурация выше: - -```sql -CREATE RESOURCE network_write (WRITE DISK s3) -CREATE RESOURCE network_read (READ DISK s3) -CREATE WORKLOAD all SETTINGS max_io_requests = 100 -CREATE WORKLOAD development IN all -CREATE WORKLOAD production IN all SETTINGS weight = 3 -``` - -Имя листовой рабочей нагрузки без дочерних элементов может быть использовано в настройках запроса `SETTINGS workload = 'name'`. - -Для настройки рабочей нагрузки могут быть использованы следующие параметры: - -* `priority` — одноуровневые рабочие нагрузки обслуживаются в соответствии со статическими значениями приоритета (меньшее значение означает более высокий приоритет). -* `weight` — одноуровневые рабочие нагрузки с одинаковым статическим приоритетом делят ресурсы пропорционально весам. -* `max_io_requests` — ограничение на количество одновременных I/O-запросов в этой рабочей нагрузке. -* `max_bytes_inflight` — ограничение на общее количество байт «в полёте» для параллельных запросов в этой рабочей нагрузке. -* `max_bytes_per_second` — ограничение на скорость чтения или записи данных (в байтах в секунду) для этой рабочей нагрузки. -* `max_burst_bytes` — максимальное количество байт, которое может быть обработано рабочей нагрузкой без ограничения пропускной способности (независимо для каждого ресурса). -* `max_concurrent_threads` — ограничение на количество потоков для запросов в этой рабочей нагрузке. -* `max_concurrent_threads_ratio_to_cores` — то же, что и `max_concurrent_threads`, но нормализовано относительно количества доступных ядер CPU. -* `max_cpus` — ограничение на количество ядер CPU, используемых для обслуживания запросов в этой рабочей нагрузке. -* `max_cpu_share` — то же, что и `max_cpus`, но нормализовано относительно количества доступных ядер CPU. -* `max_burst_cpu_seconds` — максимальное количество CPU-секунд, которое может быть потреблено рабочей нагрузкой без ограничения из-за `max_cpus`. - -Все ограничения, задаваемые через настройки рабочей нагрузки, независимы для каждого ресурса. Например, рабочая нагрузка с `max_bytes_per_second = 10485760` будет иметь ограничение пропускной способности 10 МБ/с для каждого ресурса чтения и записи независимо. Если требуется общее ограничение для чтения и записи, рассмотрите возможность использования одного и того же ресурса для доступа READ и WRITE. - -Невозможно задать разные иерархии рабочих нагрузок для разных ресурсов. Но можно задать иное значение настройки рабочей нагрузки для конкретного ресурса: - -```sql -CREATE OR REPLACE WORKLOAD all SETTINGS max_io_requests = 100, max_bytes_per_second = 1000000 FOR network_read, max_bytes_per_second = 2000000 FOR network_write -``` - -Также обратите внимание, что рабочая нагрузка или ресурс не могут быть удалены, если на них ссылается другая рабочая нагрузка. Чтобы обновить определение рабочей нагрузки, используйте запрос `CREATE OR REPLACE WORKLOAD`. - -:::note -Параметры рабочей нагрузки преобразуются в соответствующий набор узлов планировщика. За более низкоуровневыми деталями обратитесь к описанию [типов и параметров](#hierarchy) узлов планировщика. -::: - - -## Планирование CPU {#cpu_scheduling} - -Чтобы включить планирование CPU для рабочих нагрузок, создайте ресурс CPU и установите ограничение на количество одновременно выполняемых потоков: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 -``` - -Когда сервер ClickHouse выполняет множество параллельных запросов с [несколькими потоками](/operations/settings/settings.md#max_threads) и все CPU-слоты заняты, наступает состояние перегрузки. В состоянии перегрузки каждый освободившийся CPU-слот переназначается на соответствующую нагрузку в соответствии с политиками планирования. Для запросов, относящихся к одной и той же нагрузке, слоты выделяются по принципу round-robin. Для запросов в отдельных нагрузках слоты выделяются в соответствии с весами, приоритетами и лимитами, заданными для нагрузок. - -Процессорное время потребляется потоками, когда они не заблокированы и выполняют CPU-интенсивные задачи. Для целей планирования различают два вида потоков: - -* master-поток — первый поток, который начинает обработку запроса или фоновой активности, такой как слияние или мутация. -* worker-поток — дополнительные потоки, которые master может порождать для выполнения CPU-интенсивных задач. - -Может быть целесообразно использовать отдельные ресурсы для master- и worker-потоков, чтобы повысить отзывчивость системы. Большое количество worker-потоков может легко монополизировать ресурсы CPU при использовании высоких значений настройки запроса `max_threads`. В этом случае входящие запросы будут блокироваться и ожидать CPU-слот для своего master-потока, чтобы начать выполнение. Чтобы избежать этого, можно использовать следующую конфигурацию: - -```sql -CREATE RESOURCE worker_cpu (WORKER THREAD) -CREATE RESOURCE master_cpu (MASTER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 FOR worker_cpu, max_concurrent_threads = 1000 FOR master_cpu -``` - -Будут созданы раздельные лимиты для мастер‑потоков и рабочих потоков. Даже если все 100 рабочих CPU‑слотов заняты, новые запросы не будут блокироваться, пока есть доступные мастер‑CPU‑слоты. Они начнут выполняться в одном потоке. Позже, когда освободятся рабочие CPU‑слоты, такие запросы смогут масштабироваться и порождать свои рабочие потоки. С другой стороны, такой подход не привязывает общее количество слотов к числу CPU‑процессоров, и запуск слишком большого числа параллельных потоков негативно скажется на производительности. - -Ограничение параллелизма мастер‑потоков не будет ограничивать число одновременных запросов. CPU‑слоты могут освобождаться в середине выполнения запроса и повторно запрашиваться другими потоками. Например, 4 одновременных запроса с лимитом в 2 параллельных мастер‑потока всё равно могут выполняться параллельно. В этом случае каждый запрос будет получать 50% CPU‑процессора. Для ограничения числа одновременных запросов должна использоваться отдельная логика, и в текущей версии она не поддерживается для рабочих нагрузок. - -Отдельные лимиты параллелизма потоков могут использоваться для рабочих нагрузок: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all -CREATE WORKLOAD admin IN all SETTINGS max_concurrent_threads = 10 -CREATE WORKLOAD production IN all SETTINGS max_concurrent_threads = 100 -CREATE WORKLOAD analytics IN production SETTINGS max_concurrent_threads = 60, weight = 9 -CREATE WORKLOAD ингестия IN production -``` - -В этом примере конфигурации задаются независимые пулы CPU-слотов для административной и продуктивной нагрузок. Продуктивный пул разделяется между аналитикой и ингестией. Кроме того, если продуктивный пул перегружен, 9 из 10 освобождённых слотов будут при необходимости переназначены аналитическим запросам. Запросы ингестии в периоды перегрузки получат только 1 из 10 слотов. Это может улучшить время отклика пользовательских запросов. Аналитика имеет собственный лимит в 60 параллельных потоков, что всегда оставляет как минимум 40 потоков для поддержки ингестии. При отсутствии перегрузки ингестия может использовать все 100 потоков. - -Чтобы исключить запрос из планирования по CPU, установите настройку запроса [use_concurrency_control](/operations/settings/settings.md/#use_concurrency_control) в значение 0. - -Планирование по CPU пока не поддерживается для слияний (merges) и мутаций. - -Для справедливого распределения ресурсов между типами нагрузки необходимо выполнять вытеснение (preemption) и масштабирование вниз (down-scaling) во время выполнения запроса. Вытеснение включается серверной настройкой `cpu_slot_preemption`. Если она включена, каждый поток периодически обновляет (продлевает) свой CPU-слот (в соответствии с серверной настройкой `cpu_slot_quantum_ns`). Такое обновление может блокировать выполнение, если CPU перегружен. Когда выполнение блокируется на продолжительное время (см. серверную настройку `cpu_slot_preemption_timeout_ms`), запрос масштабируется вниз, и количество одновременно работающих потоков динамически уменьшается. Обратите внимание, что справедливость по времени CPU гарантируется между типами нагрузок, но между запросами внутри одного типа нагрузки в некоторых крайних случаях она может нарушаться. - -:::warning -Планирование слотов предоставляет способ управлять [конкурентностью запросов](/operations/settings/settings.md#max_threads), но не гарантирует справедливое распределение CPU-времени, за исключением случая, когда серверная настройка `cpu_slot_preemption` установлена в `true`. В противном случае справедливость обеспечивается на основе количества выделенных CPU-слотов между конкурирующими типами нагрузок. Это не подразумевает равное количество секунд CPU, потому что без вытеснения CPU-слот может удерживаться неограниченно долго. Поток захватывает слот в начале и освобождает его, когда работа завершена. -::: - - -:::note -Объявление ресурса CPU отключает действие настроек [`concurrent_threads_soft_limit_num`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_num) и [`concurrent_threads_soft_limit_ratio_to_cores`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_ratio_to_cores). Вместо этого для ограничения количества CPU, выделенных для конкретной рабочей нагрузки, используется настройка рабочей нагрузки `max_concurrent_threads`. Чтобы добиться прежнего поведения, создайте только ресурс WORKER THREAD, установите `max_concurrent_threads` для рабочей нагрузки `all` равным тому же значению, что и `concurrent_threads_soft_limit_num`, и используйте настройку запроса `workload = "all"`. Эта конфигурация соответствует настройке [`concurrent_threads_scheduler`](server-configuration-parameters/settings.md#concurrent_threads_scheduler) со значением «fair_round_robin». -::: - - - -## Потоки и CPU {#threads_vs_cpus} - -Существует два способа контролировать потребление CPU рабочей нагрузкой: - -* Лимит количества потоков: `max_concurrent_threads` и `max_concurrent_threads_ratio_to_cores` -* Ограничение CPU (throttling): `max_cpus`, `max_cpu_share` и `max_burst_cpu_seconds` - -Первый способ позволяет динамически управлять количеством потоков, создаваемых для запроса, в зависимости от текущей загрузки сервера. Фактически он снижает значение, задаваемое настройкой запроса `max_threads`. Второй способ ограничивает потребление CPU рабочей нагрузкой с помощью алгоритма token bucket. Он не влияет напрямую на количество потоков, но ограничивает суммарное потребление CPU всеми потоками в рабочей нагрузке. - -Ограничение по token bucket с `max_cpus` и `max_burst_cpu_seconds` означает следующее. В течение любого интервала длительностью `delta` секунд суммарное потребление CPU всеми запросами в рабочей нагрузке не может превышать `max_cpus * delta + max_burst_cpu_seconds` процессорных секунд. Это ограничивает среднее потребление значением `max_cpus` в долгосрочной перспективе, но в краткосрочной перспективе этот лимит может быть превышен. Например, при `max_burst_cpu_seconds = 60` и `max_cpus=0.001` допускается запуск либо одного потока на 60 секунд, либо двух потоков на 30 секунд, либо 60 потоков на 1 секунду без применения ограничения. Значение по умолчанию для `max_burst_cpu_seconds` — 1 секунда. Более низкие значения могут привести к недоиспользованию разрешённых ядер `max_cpus` при большом количестве параллельных потоков. - -:::warning -Настройки ограничения CPU активны только если включена серверная настройка `cpu_slot_preemption`, в противном случае они игнорируются. -::: - -Удерживая слот CPU, поток может находиться в одном из трёх основных состояний: - -* **Running:** Фактически потребляет ресурс CPU. Время, проведённое в этом состоянии, учитывается механизмом ограничения CPU. -* **Ready:** Ожидает, пока станет доступен CPU. Не учитывается механизмом ограничения CPU. -* **Blocked:** Выполняет IO‑операции или другие блокирующие системные вызовы (например, ожидание мьютекса). Не учитывается механизмом ограничения CPU. - -Рассмотрим пример конфигурации, которая сочетает в себе и ограничение CPU, и лимиты на количество потоков: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads_ratio_to_cores = 2 -CREATE WORKLOAD admin IN all SETTINGS max_concurrent_threads = 2, priority = -1 -CREATE WORKLOAD production IN all SETTINGS weight = 4 -CREATE WORKLOAD analytics IN production SETTINGS max_cpu_share = 0.7, weight = 3 -CREATE WORKLOAD ingestion IN production -CREATE WORKLOAD development IN all SETTINGS max_cpu_share = 0.3 -``` - -Здесь мы ограничиваем общее количество потоков для всех запросов значением, вдвое превышающим число доступных CPU. Нагрузка администратора ограничена максимум двумя потоками, независимо от количества доступных CPU. Администратор имеет приоритет -1 (ниже стандартного 0) и при необходимости получает любой слот CPU в первую очередь. Когда администратор не выполняет запросы, ресурсы CPU делятся между рабочими нагрузками production и development. Гарантированные доли времени CPU основаны на весах (4 к 1): как минимум 80% выделяется под production (если требуется) и как минимум 20% — под development (если требуется). Веса задают гарантии, а ограничение CPU (throttling) формирует пределы: production не ограничен и может потреблять 100%, в то время как у development есть лимит 30%, который применяется даже при отсутствии запросов от других нагрузок. Нагрузка production не является конечным узлом (leaf), поэтому её ресурсы делятся между analytics и ингестией в соответствии с весами (3 к 1). Это означает, что analytics имеет гарантию как минимум 0.8 * 0.75 = 60% и, на основе `max_cpu_share`, имеет лимит в 70% от общих ресурсов CPU. В то время как у ингестии остаётся гарантия как минимум 0.8 * 0.25 = 20%, верхнего предела для неё нет. - -:::note -Если вы хотите максимизировать использование CPU на сервере ClickHouse, избегайте использования `max_cpus` и `max_cpu_share` для корневой рабочей нагрузки `all`. Вместо этого задайте более высокое значение для `max_concurrent_threads`. Например, на системе с 8 CPU установите `max_concurrent_threads = 16`. Это позволит 8 потокам выполнять задачи, нагружающие CPU, в то время как другие 8 потоков будут обрабатывать операции ввода-вывода (I/O). Дополнительные потоки создадут нагрузку на CPU, обеспечив применение правил планирования. Напротив, установка `max_cpus = 8` никогда не создаст нагрузку на CPU, потому что сервер не может превысить доступные 8 CPU. -::: - - -## Планирование слотов для запросов {#query_scheduling} - -Чтобы включить планирование слотов для запросов для рабочих нагрузок, создайте ресурс типа QUERY и задайте ограничение на количество одновременных запросов или запросов в секунду: - -```sql -CREATE RESOURCE query (QUERY) -CREATE WORKLOAD all SETTINGS max_concurrent_queries = 100, max_queries_per_second = 10, max_burst_queries = 20 -``` - -Настройка рабочей нагрузки `max_concurrent_queries` ограничивает количество одновременных запросов, которые могут выполняться для данной рабочей нагрузки. Это аналог настроек запроса [`max_concurrent_queries_for_all_users`](/operations/settings/settings#max_concurrent_queries_for_all_users) и сервера [max_concurrent_queries](/operations/server-configuration-parameters/settings#max_concurrent_queries). Асинхронные запросы на вставку и некоторые специфические запросы, такие как KILL, не учитываются в этом лимите. - -Настройки рабочей нагрузки `max_queries_per_second` и `max_burst_queries` ограничивают количество запросов для этой рабочей нагрузки с помощью алгоритма «token bucket» (ведро с токенами). Они гарантируют, что в любой временной интервал `T` не начнёт выполняться более чем `max_queries_per_second * T + max_burst_queries` новых запросов. - -Настройка рабочей нагрузки `max_waiting_queries` ограничивает количество ожидающих запросов для этой рабочей нагрузки. Когда лимит достигнут, сервер возвращает ошибку `SERVER_OVERLOADED`. - -:::note -Заблокированные запросы будут ожидать неограниченное время и не появятся в `SHOW PROCESSLIST`, пока все ограничения не будут соблюдены. -::: - - -## Хранение рабочих нагрузок и ресурсов {#workload_entity_storage} - -Определения всех рабочих нагрузок и ресурсов в виде запросов `CREATE WORKLOAD` и `CREATE RESOURCE` сохраняются на постоянной основе либо на диске по пути `workload_path`, либо в ZooKeeper по пути `workload_zookeeper_path`. Для обеспечения согласованности между узлами рекомендуется хранение в ZooKeeper. В качестве альтернативы можно использовать клаузу `ON CLUSTER` совместно с хранением на диске. - - - -## Рабочие нагрузки и ресурсы, задаваемые в конфигурации {#config_based_workloads} - -Помимо SQL-определений, рабочие нагрузки и ресурсы могут быть заранее заданы в конфигурационном файле сервера. Это полезно в облачных средах, где часть ограничений определяется инфраструктурой, тогда как другие могут изменяться клиентами. Сущности, определённые в конфигурации, имеют приоритет над сущностями, определёнными с помощью SQL, и не могут быть изменены или удалены с помощью SQL-команд. - -### Формат конфигурации {#config_based_workloads_format} - -```xml - - - RESOURCE s3disk_read (READ DISK s3); - RESOURCE s3disk_write (WRITE DISK s3); - WORKLOAD all SETTINGS max_io_requests = 500 FOR s3disk_read, max_io_requests = 1000 FOR s3disk_write, max_bytes_per_second = 1342177280 FOR s3disk_read, max_bytes_per_second = 3355443200 FOR s3disk_write; - WORKLOAD production IN all SETTINGS weight = 3; - - -``` - -Конфигурация использует тот же SQL-синтаксис, что и операторы `CREATE WORKLOAD` и `CREATE RESOURCE`. Все запросы должны быть корректными. - -### Рекомендации по использованию {#config_based_workloads_usage_recommendations} - -Для облачных сред типичная конфигурация может включать: - -1. Определите корневую рабочую нагрузку (workload) и ресурсы сетевого ввода-вывода в конфигурации, чтобы задать инфраструктурные лимиты -2. Установите `throw_on_unknown_workload`, чтобы обеспечить соблюдение этих лимитов -3. Создайте `CREATE WORKLOAD default IN all`, чтобы автоматически применять лимиты ко всем запросам (поскольку значение по умолчанию для настройки запроса `workload` — 'default') -4. Разрешите пользователям создавать дополнительные рабочие нагрузки в рамках настроенной иерархии - -Это гарантирует, что все фоновые операции и запросы учитывают ограничения инфраструктуры, при этом сохраняя гибкость для пользовательских политик планирования. - -Другой сценарий использования — различная конфигурация для разных узлов в гетерогенном кластере. - - -## Строгий доступ к ресурсам {#strict_resource_access} - -Для принудительного применения политик планирования ресурсов ко всем запросам существует серверная настройка `throw_on_unknown_workload`. Если она установлена в `true`, каждый запрос обязан использовать корректную настройку запроса `workload`, в противном случае будет сгенерировано исключение `RESOURCE_ACCESS_DENIED`. Если она установлена в `false`, такой запрос не использует планировщик ресурсов, т.е. получает неограниченный доступ к любому `RESOURCE`. Настройка запроса `use_concurrency_control = 0` позволяет запросу обойти планировщик CPU и получить неограниченный доступ к CPU. Чтобы обеспечить планирование по CPU, создайте ограничение настройки, которое зафиксирует для `use_concurrency_control` константное значение только для чтения. - -:::note -Не устанавливайте `throw_on_unknown_workload` в `true`, пока не будет выполнена команда `CREATE WORKLOAD default`. Это может привести к проблемам при запуске сервера, если во время старта будет выполнен запрос без явной настройки `workload`. -::: - - - -## См. также {#see-also} -- [system.scheduler](/operations/system-tables/scheduler.md) -- [system.workloads](/operations/system-tables/workloads.md) -- [system.resources](/operations/system-tables/resources.md) -- [merge_workload](/operations/settings/merge-tree-settings.md#merge_workload) настройка MergeTree -- [merge_workload](/operations/server-configuration-parameters/settings.md#merge_workload) глобальная настройка сервера -- [mutation_workload](/operations/settings/merge-tree-settings.md#mutation_workload) настройка MergeTree -- [mutation_workload](/operations/server-configuration-parameters/settings.md#mutation_workload) глобальная настройка сервера -- [workload_path](/operations/server-configuration-parameters/settings.md#workload_path) глобальная настройка сервера -- [workload_zookeeper_path](/operations/server-configuration-parameters/settings.md#workload_zookeeper_path) глобальная настройка сервера -- [cpu_slot_preemption](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption) глобальная настройка сервера -- [cpu_slot_quantum_ns](/operations/server-configuration-parameters/settings.md#cpu_slot_quantum_ns) глобальная настройка сервера -- [cpu_slot_preemption_timeout_ms](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption_timeout_ms) глобальная настройка сервера \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md deleted file mode 100644 index fb73fb3d870..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md +++ /dev/null @@ -1,362 +0,0 @@ ---- -description: 'Документация по комбинаторам агрегатных функций' -sidebar_label: 'Комбинаторы' -sidebar_position: 37 -slug: /sql-reference/aggregate-functions/combinators -title: 'Комбинаторы агрегатных функций' -doc_type: 'reference' ---- - - - -# Комбинаторы агрегатных функций {#aggregate-function-combinators} - -К имени агрегатной функции можно добавить суффикс. Это изменяет поведение агрегатной функции. - - - -## -If {#-if} - -Суффикс -If может быть добавлен к имени любой агрегатной функции. В этом случае агрегатная функция принимает дополнительный аргумент — условие (типа Uint8). Агрегатная функция обрабатывает только те строки, для которых условие выполняется. Если условие ни разу не выполнилось, возвращается значение по умолчанию (обычно нули или пустые строки). - -Примеры: `sumIf(column, cond)`, `countIf(cond)`, `avgIf(x, cond)`, `quantilesTimingIf(level1, level2)(x, cond)`, `argMinIf(arg, val, cond)` и так далее. - -С помощью условных агрегатных функций можно вычислять агрегатные значения сразу для нескольких условий без использования подзапросов и `JOIN`-ов. Например, условные агрегатные функции можно использовать для реализации функциональности сравнения сегментов. - - - -## -Array {#-array} - -Суффикс `-Array` может быть добавлен к любой агрегатной функции. В этом случае агрегатная функция принимает аргументы типа `Array(T)` (массивы) вместо аргументов типа `T`. Если агрегатная функция принимает несколько аргументов, это должны быть массивы одинаковой длины. При обработке массивов агрегатная функция выполняет те же действия, что и исходная агрегатная функция, но по всем элементам массивов. - -Пример 1: `sumArray(arr)` — суммирует все элементы всех массивов `arr`. В этом примере можно записать выражение проще: `sum(arraySum(arr))`. - -Пример 2: `uniqArray(arr)` — считает количество уникальных элементов во всех массивах `arr`. Это можно сделать и более простым способом: `uniq(arrayJoin(arr))`, но не всегда возможно добавить `arrayJoin` в запрос. - -`-If` и `-Array` могут комбинироваться. Однако сначала должен идти `Array`, затем `If`. Примеры: `uniqArrayIf(arr, cond)`, `quantilesTimingArrayIf(level1, level2)(arr, cond)`. Из-за такого порядка аргумент `cond` не будет массивом. - - - -## -Map {#-map} - -Суффикс -Map можно добавить к любой агрегатной функции. Это создаст агрегатную функцию, которая принимает аргумент типа Map и агрегирует значения для каждого ключа этой Map отдельно, используя указанную агрегатную функцию. Результат также имеет тип Map. - -**Пример** - -```sql -CREATE TABLE map_map( - date Date, - timeslot DateTime, - status Map(String, UInt64) -) ENGINE = Log; - -INSERT INTO map_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', (['a', 'b', 'c'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', (['c', 'd', 'e'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['d', 'e', 'f'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['f', 'g', 'g'], [10, 10, 10])); - -SELECT - timeslot, - sumMap(status), - avgMap(status), - minMap(status) -FROM map_map -GROUP BY timeslot; - -┌────────────timeslot─┬─sumMap(status)───────────────────────┬─avgMap(status)───────────────────────┬─minMap(status)───────────────────────┐ -│ 2000-01-01 00:00:00 │ {'a':10,'b':10,'c':20,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ -│ 2000-01-01 00:01:00 │ {'d':10,'e':10,'f':20,'g':20} │ {'d':10,'e':10,'f':10,'g':10} │ {'d':10,'e':10,'f':10,'g':10} │ -└─────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┘ -``` - - -## -SimpleState {#-simplestate} - -При применении этого комбинатора агрегатная функция возвращает то же значение, но с другим типом. Это тип данных [SimpleAggregateFunction(...)](../../sql-reference/data-types/simpleaggregatefunction.md), который можно хранить в таблице для работы с таблицами [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md). - -**Синтаксис** - -```sql -SimpleState(x) -``` - -**Аргументы** - -* `x` — параметры агрегатной функции. - -**Возвращаемые значения** - -Возвращаемое значение агрегатной функции с типом `SimpleAggregateFunction(...)`. - -**Пример** - -Запрос: - -```sql -WITH anySimpleState(number) AS c SELECT toTypeName(c), c FROM numbers(1); -``` - -Результат: - -```text -┌─toTypeName(c)────────────────────────┬─c─┐ -│ SimpleAggregateFunction(any, UInt64) │ 0 │ -└──────────────────────────────────────┴───┘ -``` - - -## -State {#-state} - -Если применить этот комбинатор, агрегатная функция не возвращает итоговое значение (например, количество уникальных значений для функции [uniq](/sql-reference/aggregate-functions/reference/uniq)), а возвращает промежуточное состояние агрегации (для `uniq` это хеш-таблица для вычисления количества уникальных значений). Это тип `AggregateFunction(...)`, который можно использовать для дальнейшей обработки или сохранить в таблице, чтобы завершить агрегацию позже. - -:::note -Обратите внимание, что -MapState не является инвариантом для одних и тех же данных, поскольку порядок данных в промежуточном состоянии может изменяться, хотя это не влияет на приём этих данных. -::: - -Для работы с этими состояниями используйте: - -- Табличный движок [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md). -- Функцию [finalizeAggregation](/sql-reference/functions/other-functions#finalizeAggregation). -- Функцию [runningAccumulate](../../sql-reference/functions/other-functions.md#runningAccumulate). -- Комбинатор [-Merge](#-merge). -- Комбинатор [-MergeState](#-mergestate). - - - -## -Merge {#-merge} - -При использовании этого комбинатора агрегатная функция принимает промежуточные состояния агрегации в качестве аргумента, объединяет их для завершения агрегации и возвращает итоговое значение. - - - -## -MergeState {#-mergestate} - -Объединяет промежуточные состояния агрегации так же, как комбинатор -Merge. Однако он не возвращает результирующее значение, а промежуточное состояние агрегации — аналогично комбинатору -State. - - - -## -ForEach {#-foreach} - -Преобразует агрегатную функцию для таблиц в агрегатную функцию для массивов, которая агрегирует соответствующие элементы массивов и возвращает массив результатов. Например, `sumForEach` для массивов `[1, 2]`, `[3, 4, 5]` и `[6, 7]` возвращает результат `[10, 13, 5]` после суммирования соответствующих элементов этих массивов. - - - -## -Distinct {#-distinct} - -Каждая уникальная комбинация аргументов учитывается при агрегации только один раз. Повторяющиеся значения игнорируются. -Примеры: `sum(DISTINCT x)` (или `sumDistinct(x)`), `groupArray(DISTINCT x)` (или `groupArrayDistinct(x)`), `corrStable(DISTINCT x, y)` (или `corrStableDistinct(x, y)`) и так далее. - - - -## -OrDefault {#-ordefault} - -Модифицирует поведение агрегатной функции. - -Если агрегатная функция не получает входных значений, с этим комбинатором она возвращает значение по умолчанию для своего возвращаемого типа данных. Применяется к агрегатным функциям, которые могут работать с пустыми входными данными. - -`-OrDefault` может использоваться с другими комбинаторами. - -**Синтаксис** - -```sql -OrDefault(x) -``` - -**Аргументы** - -* `x` — параметры агрегатной функции. - -**Возвращаемые значения** - -Возвращает значение по умолчанию для типа результата агрегатной функции, если отсутствуют данные для агрегации. - -Тип возвращаемого значения зависит от используемой агрегатной функции. - -**Пример** - -Запрос: - -```sql -SELECT avg(number), avgOrDefault(number) FROM numbers(0) -``` - -Результат: - -```text -┌─avg(number)─┬─avgOrDefault(number)─┐ -│ nan │ 0 │ -└─────────────┴──────────────────────┘ -``` - -Также `-OrDefault` можно использовать с другими комбинаторами. Это полезно, когда агрегатная функция не принимает пустой набор входных данных. - -Запрос: - -```sql -SELECT avgOrDefaultIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -Результат: - -```text -┌─avgOrDefaultIf(x, greater(x, 10))─┐ -│ 0.00 │ -└───────────────────────────────────┘ -``` - - -## -OrNull {#-ornull} - -Изменяет поведение агрегатной функции. - -Этот комбинатор преобразует результат агрегатной функции в тип данных [Nullable](../../sql-reference/data-types/nullable.md). Если у агрегатной функции нет значений для вычисления, она возвращает [NULL](/operations/settings/formats#input_format_null_as_default). - -`-OrNull` может использоваться с другими комбинаторами. - -**Синтаксис** - -```sql -OrNull(x) -``` - -**Аргументы** - -* `x` — параметры агрегатной функции. - -**Возвращаемые значения** - -* Результат агрегатной функции, преобразованный к типу данных `Nullable`. -* `NULL`, если нет данных для агрегации. - -Тип: `Nullable(тип результата агрегатной функции)`. - -**Пример** - -Добавьте `-orNull` в конец агрегатной функции. - -Запрос: - -```sql -SELECT sumOrNull(number), toTypeName(sumOrNull(number)) FROM numbers(10) WHERE number > 10 -``` - -Результат: - -```text -┌─sumOrNull(number)─┬─toTypeName(sumOrNull(number))─┐ -│ ᴺᵁᴸᴸ │ Nullable(UInt64) │ -└───────────────────┴───────────────────────────────┘ -``` - -Также `-OrNull` может использоваться и с другими комбинаторами. Это полезно, когда агрегатная функция не допускает пустой входной набор данных. - -Запрос: - -```sql -SELECT avgOrNullIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -Результат: - -```text -┌─avgOrNullIf(x, greater(x, 10))─┐ -│ ᴺᵁᴸᴸ │ -└────────────────────────────────┘ -``` - - -## -Resample {#-resample} - -Позволяет разбить данные на группы и затем по отдельности агрегировать данные в каждой группе. Группы формируются разбиением значений одного столбца на интервалы. - -```sql -Resample(начало, конец, шаг)(, ключ_ресемплирования) -``` - -**Аргументы** - -* `start` — Начальное значение всего требуемого интервала значений `resampling_key`. -* `stop` — Конечное значение всего требуемого интервала значений `resampling_key`. Весь интервал не включает значение `stop` — `[start, stop)`. -* `step` — Шаг для разбиения этого интервала на подынтервалы. `aggFunction` выполняется для каждого такого подынтервала независимо. -* `resampling_key` — Столбец, значения которого используются для разбиения данных на интервалы. -* `aggFunction_params` — Параметры `aggFunction`. - -**Возвращаемые значения** - -* Массив результатов `aggFunction` для каждого подынтервала. - -**Пример** - -Рассмотрим таблицу `people` со следующими данными: - -```text -┌─имя────┬─возраст─┬─зарплата─┐ -│ John │ 16 │ 10 │ -│ Alice │ 30 │ 15 │ -│ Mary │ 35 │ 8 │ -│ Evelyn │ 48 │ 11.5 │ -│ David │ 62 │ 9.9 │ -│ Brian │ 60 │ 16 │ -└────────┴─────┴──────┘ -``` - -Получим имена людей, возраст которых лежит в интервалах `[30,60)` и `[60,75)`. Поскольку мы используем целочисленное представление возраста, то фактически получаем значения возраста в диапазонах `[30, 59]` и `[60, 74]`. - -Чтобы агрегировать имена в массив, используем агрегатную функцию [groupArray](/sql-reference/aggregate-functions/reference/grouparray). Она принимает один аргумент — в нашем случае это столбец `name`. Функция `groupArrayResample` должна использовать столбец `age` для агрегации имен по возрасту. Чтобы задать требуемые интервалы, передаем аргументы `30, 75, 30` в функцию `groupArrayResample`. - -```sql -SELECT groupArrayResample(30, 75, 30)(name, age) FROM people -``` - -```text -┌─groupArrayResample(30, 75, 30)(name, age)─────┐ -│ [['Alice','Mary','Evelyn'],['David','Brian']] │ -└───────────────────────────────────────────────┘ -``` - -Рассмотрим результаты. - -`John` исключён из выборки, потому что он слишком молод. Остальные участники распределены в соответствии с указанными возрастными интервалами. - -Теперь рассчитаем общее количество людей и их среднюю заработную плату в указанных возрастных интервалах. - -```sql -SELECT - countResample(30, 75, 30)(name, age) AS amount, - avgResample(30, 75, 30)(wage, age) AS avg_wage -FROM people -``` - -```text -┌─amount─┬─avg_wage──────────────────┐ -│ [3,2] │ [11.5,12.949999809265137] │ -└────────┴───────────────────────────┘ -``` - - -## -ArgMin {#-argmin} - -Суффикс -ArgMin может быть добавлен к имени любой агрегатной функции. В этом случае агрегатная функция принимает дополнительный аргумент, которым может быть любое сравнимое выражение. Агрегатная функция обрабатывает только те строки, для которых указанное дополнительное выражение принимает минимальное значение. - -Примеры: `sumArgMin(column, expr)`, `countArgMin(expr)`, `avgArgMin(x, expr)` и так далее. - - - -## -ArgMax {#-argmax} - -Аналогичен суффиксу -ArgMin, но обрабатывает только строки с максимальным значением для указанного дополнительного выражения. - - - -## Связанные материалы {#related-content} - -- Блог: [Использование агрегатных комбинаторов в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md deleted file mode 100644 index 5f2c15ef4df..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md +++ /dev/null @@ -1,369 +0,0 @@ ---- -description: 'Документация по агрегатной функции GROUPING.' -slug: /sql-reference/aggregate-functions/grouping_function -title: 'GROUPING' -doc_type: 'reference' ---- - - - -# Группировка {#grouping} - - - -## GROUPING {#grouping} - -[ROLLUP](../statements/select/group-by.md/#rollup-modifier) и [CUBE](../statements/select/group-by.md/#cube-modifier) являются модификаторами GROUP BY. Оба модификатора вычисляют промежуточные итоги. ROLLUP принимает упорядоченный список столбцов, например `(day, month, year)`, и вычисляет промежуточные итоги на каждом уровне агрегации, а затем общий итог. CUBE вычисляет промежуточные итоги для всех возможных комбинаций указанных столбцов. GROUPING определяет, какие строки, возвращённые ROLLUP или CUBE, являются сверхагрегатами, а какие — строками, которые были бы возвращены немодифицированным GROUP BY. - -Функция GROUPING принимает несколько столбцов в качестве аргумента и возвращает битовую маску. -- `1` указывает, что строка, возвращённая модификатором `ROLLUP` или `CUBE` к `GROUP BY`, является промежуточным итогом -- `0` указывает, что строка, возвращённая `ROLLUP` или `CUBE`, не является промежуточным итогом - - - -## GROUPING SETS {#grouping-sets} - -По умолчанию модификатор CUBE вычисляет промежуточные итоги для всех возможных комбинаций переданных в CUBE столбцов. GROUPING SETS позволяет задать конкретные комбинации, для которых будут вычисляться итоги. - -Анализ иерархических данных — хороший сценарий применения модификаторов ROLLUP, CUBE и GROUPING SETS. В этом примере используется таблица с данными о том, какие дистрибутивы Linux и какие версии этих дистрибутивов установлены в двух дата-центрах. Может быть полезно просматривать эти данные по дистрибутиву, версии и дата-центру. - -### Загрузка примера данных {#load-sample-data} - -```sql -CREATE TABLE servers ( datacenter VARCHAR(255), - distro VARCHAR(255) NOT NULL, - version VARCHAR(50) NOT NULL, - quantity INT - ) - ORDER BY (datacenter, distro, version) -``` - -```sql -INSERT INTO servers(datacenter, distro, version, quantity) -VALUES ('Schenectady', 'Arch','2022.08.05',50), - ('Westport', 'Arch','2022.08.05',40), - ('Schenectady','Arch','2021.09.01',30), - ('Westport', 'Arch','2021.09.01',20), - ('Schenectady','Arch','2020.05.01',10), - ('Westport', 'Arch','2020.05.01',5), - ('Schenectady','RHEL','9',60), - ('Westport','RHEL','9',70), - ('Westport','RHEL','7',80), - ('Schenectady','RHEL','7',80) -``` - -```sql -SELECT - * -FROM - servers; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─quantity─┐ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ 9 │ 70 │ -└─────────────┴────────┴────────────┴──────────┘ - -10 строк в наборе. Затрачено: 0.409 сек. -``` - -### Простые запросы {#simple-queries} - -Получите число серверов в каждом дата-центре в разбивке по дистрибутивам: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ - -4 строки в наборе. Затрачено: 0.212 сек. -``` - -```sql -SELECT - datacenter, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter; -``` - -```response -┌─datacenter──┬─qty─┐ -│ Westport │ 215 │ -│ Schenectady │ 230 │ -└─────────────┴─────┘ - -Получено 2 строки. Прошло: 0.277 сек. -``` - -```sql -SELECT - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro; -``` - -```response - -┌─distro─┬─qty─┐ -│ Arch │ 155 │ -│ RHEL │ 290 │ -└────────┴─────┘ - -Получено 2 строки. Затрачено: 0,352 сек. -``` - - -```sql -SELECT - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─qty─┐ -│ 445 │ -└─────┘ - -Получена 1 строка. Время выполнения: 0.244 сек. -``` - -### Сравнение нескольких предложений GROUP BY и GROUPING SETS {#comparing-multiple-group-by-statements-with-grouping-sets} - -Разбиение данных без использования CUBE, ROLLUP или GROUPING SETS: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro -UNION ALL -SELECT - datacenter, - null, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter -UNION ALL -SELECT - null, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro -UNION ALL -SELECT - null, - null, - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ ᴺᵁᴸᴸ │ 215 │ -│ Schenectady │ ᴺᵁᴸᴸ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ Arch │ 155 │ -│ ᴺᵁᴸᴸ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -Получено 9 строк. Затрачено: 0.527 сек. -``` - -Получение тех же данных с использованием GROUPING SETS: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - GROUPING SETS( - (datacenter,distro), - (datacenter), - (distro), - () - ) -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ │ 215 │ -│ Schenectady │ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ Arch │ 155 │ -│ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -9 строк в наборе. Затрачено: 0.427 сек. -``` - -### Сравнение CUBE с GROUPING SETS {#comparing-cube-with-grouping-sets} - -CUBE в следующем запросе `CUBE(datacenter, distro, version)` создает иерархию, которая логически не имеет смысла. Некорректно рассматривать `version` одновременно для двух дистрибутивов (так как Arch и RHEL имеют разные циклы релизов и стандарты именования версий). Пример с GROUPING SETS, приведённый далее, более уместен, так как он группирует `distro` и `version` в одном наборе. - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM - servers -GROUP BY - CUBE(datacenter,distro,version) -ORDER BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ │ │ 7 │ 160 │ -│ │ │ 2020.05.01 │ 15 │ -│ │ │ 2021.09.01 │ 50 │ -│ │ │ 2022.08.05 │ 90 │ -│ │ │ 9 │ 130 │ -│ │ │ │ 445 │ -│ │ Arch │ 2021.09.01 │ 50 │ -│ │ Arch │ 2022.08.05 │ 90 │ -│ │ Arch │ 2020.05.01 │ 15 │ -│ │ Arch │ │ 155 │ -│ │ RHEL │ 9 │ 130 │ -│ │ RHEL │ 7 │ 160 │ -│ │ RHEL │ │ 290 │ -│ Schenectady │ │ 9 │ 60 │ -│ Schenectady │ │ 2021.09.01 │ 30 │ -│ Schenectady │ │ 7 │ 80 │ -│ Schenectady │ │ 2022.08.05 │ 50 │ -│ Schenectady │ │ 2020.05.01 │ 10 │ -│ Schenectady │ │ │ 230 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ │ 90 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ │ 9 │ 70 │ -│ Westport │ │ 2020.05.01 │ 5 │ -│ Westport │ │ 2022.08.05 │ 40 │ -│ Westport │ │ 7 │ 80 │ -│ Westport │ │ 2021.09.01 │ 20 │ -│ Westport │ │ │ 215 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ Arch │ │ 65 │ -│ Westport │ RHEL │ 9 │ 70 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴────────────┴───────────────┘ - -39 строк в наборе. Затрачено: 0.355 сек. -``` - -:::note -Версия в приведённом выше примере может быть неинформативной, если она не привязана к дистрибутиву; если бы мы отслеживали версию ядра, это могло бы быть логично, поскольку версию ядра можно связать с любым из дистрибутивов. Использование GROUPING SETS, как в следующем примере, может быть более удачным выбором. -::: - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM servers -GROUP BY - GROUPING SETS ( - (datacenter, distro, version), - (datacenter, distro)) -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ Westport │ RHEL │ 9 │ 70 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -└─────────────┴────────┴────────────┴───────────────┘ -┌─datacenter──┬─distro─┬─version─┬─sum(quantity)─┐ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ Arch │ │ 65 │ -│ Schenectady │ Arch │ │ 90 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴─────────┴───────────────┘ - -14 строк в наборе. Затрачено: 1.036 сек. -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md deleted file mode 100644 index 41c8a398eb3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -description: 'Справочник по агрегатным функциям' -sidebar_label: 'Агрегатные функции' -sidebar_position: 33 -slug: /sql-reference/aggregate-functions/ -title: 'Агрегатные функции' -doc_type: 'reference' ---- - -# Агрегатные функции {#aggregate-functions} - -Агрегатные функции работают [стандартным](http://www.sql-tutorial.com/sql-aggregate-functions-sql-tutorial) образом, привычным для специалистов по базам данных. - -ClickHouse также поддерживает: - -* [Параметрические агрегатные функции](/sql-reference/aggregate-functions/parametric-functions), которые, помимо столбцов, принимают дополнительные параметры. -* [Комбинаторы](/sql-reference/aggregate-functions/combinators), которые изменяют поведение агрегатных функций. - -## Обработка NULL {#null-processing} - -При агрегации все аргументы со значением `NULL` пропускаются. Если агрегатная функция имеет несколько аргументов, она игнорирует любую строку, в которой один или несколько из них равны NULL. - -Из этого правила есть исключение — функции [`first_value`](../../sql-reference/aggregate-functions/reference/first_value.md), [`last_value`](../../sql-reference/aggregate-functions/reference/last_value.md) и их псевдонимы (соответственно `any` и `anyLast`), когда используется модификатор `RESPECT NULLS`. Например, `FIRST_VALUE(b) RESPECT NULLS`. - -**Примеры:** - -Рассмотрим эту таблицу: - -```text -┌─x─┬────y─┐ -│ 1 │ 2 │ -│ 2 │ ᴺᵁᴸᴸ │ -│ 3 │ 2 │ -│ 3 │ 3 │ -│ 3 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -Предположим, вам нужно просуммировать значения в столбце `y`: - -```sql -SELECT sum(y) FROM t_null_big -``` - -```text -┌─sum(y)─┐ -│ 7 │ -└────────┘ -``` - -Теперь вы можете использовать функцию `groupArray`, чтобы создать массив из столбца `y`: - -```sql -SELECT groupArray(y) FROM t_null_big -``` - -```text -┌─groupArray(y)─┐ -│ [2,2,3] │ -└───────────────┘ -``` - -`groupArray` не включает `NULL` в результирующий массив. - -Вы можете использовать [COALESCE](../../sql-reference/functions/functions-for-nulls.md#coalesce), чтобы заменить NULL значением, которое имеет смысл в вашем варианте использования. Например: `avg(COALESCE(column, 0))` будет использовать значение столбца при агрегации или ноль, если значение равно NULL: - -```sql -SELECT - avg(y), - avg(coalesce(y, 0)) -FROM t_null_big -``` - -```text -┌─────────────avg(y)─┬─avg(coalesce(y, 0))─┐ -│ 2.3333333333333335 │ 1.4 │ -└────────────────────┴─────────────────────┘ -``` - -Также вы можете использовать [Tuple](sql-reference/data-types/tuple.md), чтобы обойти поведение пропуска значений `NULL`. `Tuple`, который содержит только значение `NULL`, сам по себе не является `NULL`, поэтому агрегатные функции не будут пропускать эту строку из‑за этого значения `NULL`. - -```sql -SELECT - groupArray(y), - groupArray(tuple(y)).1 -FROM t_null_big; - -┌─groupArray(y)─┬─tupleElement(groupArray(tuple(y)), 1)─┐ -│ [2,2,3] │ [2,NULL,2,3,NULL] │ -└───────────────┴───────────────────────────────────────┘ -``` - -Обратите внимание, что агрегации пропускаются, когда столбцы используются как аргументы агрегатной функции. Например, [`count`](../../sql-reference/aggregate-functions/reference/count.md) без параметров (`count()`) или с константными (`count(1)`) будет считать все строки в блоке (независимо от значения столбца в `GROUP BY`, так как он не является аргументом), тогда как `count(column)` вернет только количество строк, где `column` не равно `NULL`. - -```sql -SELECT - v, - count(1), - count(v) -FROM -( - SELECT if(number < 10, NULL, number % 3) AS v - FROM numbers(15) -) -GROUP BY v - -┌────v─┬─count()─┬─count(v)─┐ -│ ᴺᵁᴸᴸ │ 10 │ 0 │ -│ 0 │ 1 │ 1 │ -│ 1 │ 2 │ 2 │ -│ 2 │ 2 │ 2 │ -└──────┴─────────┴──────────┘ -``` - -И вот пример функции first_value с `RESPECT NULLS`, где мы видим, что входные значения NULL учитываются, и будет возвращено первое встретившееся значение, независимо от того, является оно NULL или нет: - -```sql -SELECT - col || '_' || ((col + 1) * 5 - 1) AS диапазон, - first_value(odd_or_null) AS первый, - first_value(odd_or_null) IGNORE NULLS AS первый_игнорируя_NULL, - first_value(odd_or_null) RESPECT NULLS AS первый_с_учётом_NULL -FROM -( - SELECT - intDiv(number, 5) AS col, - if(number % 2 == 0, NULL, number) AS odd_or_null - FROM numbers(15) -) -GROUP BY col -ORDER BY col - -┌─диапазон─┬─первый─┬─первый_игнорируя_NULL─┬─первый_с_учётом_NULL─┐ -│ 0_4 │ 1 │ 1 │ ᴺᵁᴸᴸ │ -│ 1_9 │ 5 │ 5 │ 5 │ -│ 2_14 │ 11 │ 11 │ ᴺᵁᴸᴸ │ -└──────────┴────────┴────────────────────────┴──────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md deleted file mode 100644 index 9ea4b54f953..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md +++ /dev/null @@ -1,941 +0,0 @@ ---- -description: 'Документация по параметрическим агрегатным функциям' -sidebar_label: 'Параметрические' -sidebar_position: 38 -slug: /sql-reference/aggregate-functions/parametric-functions -title: 'Параметрические агрегатные функции' -doc_type: 'reference' ---- - -# Параметрические агрегатные функции {#parametric-aggregate-functions} - -Некоторые агрегатные функции могут принимать не только столбцы-аргументы (используемые для сжатия), но и набор параметров — констант для инициализации. В синтаксисе для этого используются две пары круглых скобок вместо одной: первая — для параметров, вторая — для аргументов. - -## histogram {#histogram} - -Вычисляет адаптивную гистограмму. Не гарантирует точных результатов. - -```sql -histogram(число_интервалов)(значения) -``` - -Функция использует алгоритм [A Streaming Parallel Decision Tree Algorithm](http://jmlr.org/papers/volume11/ben-haim10a/ben-haim10a.pdf). Границы корзин гистограммы корректируются по мере поступления новых данных в функцию. В общем случае ширины корзин могут различаться. - -**Аргументы** - -`values` — [выражение](/sql-reference/syntax#expressions), возвращающее входные значения. - -**Параметры** - -`number_of_bins` — верхний предел количества корзин в гистограмме. Функция автоматически вычисляет количество корзин. Она пытается достичь указанного количества корзин, но если это не удаётся, использует меньшее количество. - -**Возвращаемые значения** - -* [Массив](../../sql-reference/data-types/array.md) [кортежей](../../sql-reference/data-types/tuple.md) следующего формата: - - ``` - [(lower_1, upper_1, height_1), ... (lower_N, upper_N, height_N)] - ``` - - * `lower` — нижняя граница корзины. - * `upper` — верхняя граница корзины. - * `height` — вычисленная высота корзины. - -**Пример** - -```sql -SELECT histogram(5)(number + 1) -FROM ( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─histogram(5)(plus(number, 1))───────────────────────────────────────────┐ -│ [(1,4.5,4),(4.5,8.5,4),(8.5,12.75,4.125),(12.75,17,4.625),(17,20,3.25)] │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -Вы можете построить гистограмму с помощью функции [bar](/sql-reference/functions/other-functions#bar), например: - -```sql -WITH histogram(5)(rand() % 100) AS hist -SELECT - arrayJoin(hist).3 AS height, - bar(height, 0, 6, 5) AS bar -FROM -( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─height─┬─bar───┐ -│ 2.125 │ █▋ │ -│ 3.25 │ ██▌ │ -│ 5.625 │ ████▏ │ -│ 5.625 │ ████▏ │ -│ 3.375 │ ██▌ │ -└────────┴───────┘ -``` - -В этом случае следует помнить, что вы не знаете границы интервалов гистограммы. - -## sequenceMatch {#sequencematch} - -Проверяет, содержит ли последовательность цепочку событий, соответствующую заданному шаблону. - -**Синтаксис** - -```sql -sequenceMatch(pattern)(timestamp, cond1, cond2, ...) -``` - -:::note -События, происходящие в одну и ту же секунду, могут располагаться в последовательности в неопределённом порядке, что влияет на результат. -::: - -**Аргументы** - -* `timestamp` — Столбец, содержащий данные времени. Типичные типы данных: `Date` и `DateTime`. Вы также можете использовать любой из поддерживаемых типов данных [UInt](../../sql-reference/data-types/int-uint.md). - -* `cond1`, `cond2` — Условия, описывающие цепочку событий. Тип данных: `UInt8`. Можно передать до 32 аргументов-условий. Функция учитывает только события, описанные этими условиями. Если последовательность содержит данные, не описанные ни одним условием, функция их пропускает. - -**Параметры** - -* `pattern` — Строка шаблона. См. [Синтаксис шаблона](#pattern-syntax). - -**Возвращаемые значения** - -* 1, если шаблон совпал. -* 0, если шаблон не совпал. - -Тип: `UInt8`. - -#### Синтаксис шаблона {#pattern-syntax} - -* `(?N)` — Соответствует аргументу-условию в позиции `N`. Условия нумеруются в диапазоне `[1, 32]`. Например, `(?1)` соответствует аргументу, переданному параметру `cond1`. - -* `.*` — Соответствует любому количеству событий. Для сопоставления этого элемента шаблона не требуются условные аргументы. - -* `(?t operator value)` — Задаёт время в секундах, которое должно разделять два события. Например, шаблон `(?1)(?t>1800)(?2)` соответствует событиям, между которыми проходит более 1800 секунд. Между этими событиями может быть произвольное количество любых событий. Можно использовать операторы `>=`, `>`, `<`, `<=`, `==`. - -**Примеры** - -Рассмотрим данные в таблице `t`: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -└──────┴────────┘ -``` - -Выполните запрос: - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 1 │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -Функция нашла цепочку событий, в которой число 2 следует за числом 1. Она пропустила число 3 между ними, потому что оно не задано как событие. Если мы хотим учитывать это число при поиске цепочки событий, приведённой в примере, нужно задать для него условие. - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 3) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 3))─┐ -│ 0 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -В этом случае функция не смогла найти цепочку событий, соответствующую шаблону, потому что событие с номером 3 произошло между 1 и 2. Если бы в этом же случае мы проверяли условие для числа 4, последовательность соответствовала бы шаблону. - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ 1 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [sequenceCount](#sequencecount) - -## sequenceCount {#sequencecount} - -Подсчитывает количество цепочек событий, соответствующих шаблону. Функция ищет цепочки событий, которые не перекрываются: после сопоставления текущей цепочки она начинает поиск следующей. - -:::note -События, происходящие в одну и ту же секунду, могут располагаться в последовательности в неопределённом порядке, что влияет на результат. -::: - -**Синтаксис** - -```sql -sequenceCount(pattern)(timestamp, cond1, cond2, ...) -``` - -**Аргументы** - -* `timestamp` — Столбец, содержащий временные данные. Типичные типы данных: `Date` и `DateTime`. Также можно использовать любой из поддерживаемых беззнаковых целочисленных типов [UInt](../../sql-reference/data-types/int-uint.md). - -* `cond1`, `cond2` — Условия, описывающие цепочку событий. Тип данных: `UInt8`. Можно передать до 32 аргументов-условий. Функция учитывает только события, описанные этими условиями. Если последовательность содержит данные, которые не описаны ни в одном условии, функция их пропускает. - -**Параметры** - -* `pattern` — Строка шаблона. См. [синтаксис шаблонов](#pattern-syntax). - -**Возвращаемые значения** - -* Количество непересекающихся цепочек событий, удовлетворяющих шаблону. - -Тип: `UInt64`. - -**Пример** - -Рассмотрим данные в таблице `t`: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -Посчитайте, сколько раз число 2 встречается после числа 1 с любым количеством других чисел между ними: - -```sql -SELECT sequenceCount('(?1).*(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceCount('(?1).*(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 2 │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -## sequenceMatchEvents {#sequencematchevents} - -Возвращает временные метки событий для наиболее длинных цепочек, соответствующих шаблону. - -:::note -События, происходящие в одну и ту же секунду, могут располагаться в последовательности в неопределённом порядке, что влияет на результат. -::: - -**Синтаксис** - -```sql -sequenceMatchEvents(pattern)(timestamp, cond1, cond2, ...) -``` - -**Аргументы** - -* `timestamp` — Столбец, содержащий данные о времени. Типичные типы данных: `Date` и `DateTime`. Также можно использовать любой из поддерживаемых типов данных [UInt](../../sql-reference/data-types/int-uint.md). - -* `cond1`, `cond2` — Условия, описывающие цепочку событий. Тип данных: `UInt8`. Можно передать до 32 аргументов-условий. Функция учитывает только события, описанные этими условиями. Если последовательность содержит данные, которые не описаны ни одним условием, функция их пропускает. - -**Параметры** - -* `pattern` — Строка шаблона. См. [Синтаксис шаблонов](#pattern-syntax). - -**Возвращаемые значения** - -* Массив меток времени для аргументов-условий (?N), удовлетворяющих шаблону, из цепочки событий. Позиция в массиве соответствует позиции аргумента-условия в шаблоне. - -Тип: Array. - -**Пример** - -Рассмотрим данные в таблице `t`: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -Возвращает временные метки событий для самой длинной цепочки - -```sql -SELECT sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ [1,3,4] │ -└────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [sequenceMatch](#sequencematch) - -## windowFunnel {#windowfunnel} - -Ищет цепочки событий в скользящем временном окне и вычисляет максимальное число событий из цепочки, произошедших в этом окне. - -Функция работает по следующему алгоритму: - -* Функция ищет данные, которые соответствуют первому условию в цепочке, и устанавливает счётчик событий равным 1. В этот момент начинается скользящее окно. - -* Если события из цепочки происходят последовательно в пределах окна, счётчик увеличивается. Если последовательность событий нарушается, счётчик не увеличивается. - -* Если в данных есть несколько цепочек событий с разной степенью завершённости, функция выводит только размер самой длинной цепочки. - -**Синтаксис** - -```sql -windowFunnel(window, [mode, [mode, ... ]])(timestamp, cond1, cond2, ..., condN) -``` - -**Аргументы** - -* `timestamp` — имя столбца, содержащего метку времени. Поддерживаемые типы данных: [Date](../../sql-reference/data-types/date.md), [DateTime](/sql-reference/data-types/datetime) и другие беззнаковые целочисленные типы (обратите внимание, что хотя `timestamp` поддерживает тип `UInt64`, его значение не может превышать максимальное значение для Int64, равное 2^63 - 1). -* `cond` — условия или данные, описывающие цепочку событий. [UInt8](../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `window` — длина скользящего окна, временной интервал между первым и последним условием. Единица измерения `window` зависит от самого `timestamp` и может различаться. Определяется выражением `timestamp of cond1 <= timestamp of cond2 <= ... <= timestamp of condN <= timestamp of cond1 + window`. -* `mode` — необязательный аргумент. Можно задать один или несколько режимов. - * `'strict_deduplication'` — если одно и то же условие выполняется для последовательности событий, то такое повторяющееся событие прерывает дальнейшую обработку. Примечание: может работать неожиданно, если для одного и того же события выполняется несколько условий. - * `'strict_order'` — не допускать вклинивания других событий. Например, в случае `A->B->D->C` поиск `A->B->C` останавливается на `D`, и максимальный уровень события равен 2. - * `'strict_increase'` — применять условия только к событиям со строго возрастающими метками времени. - * `'strict_once'` — учитывать каждое событие в цепочке только один раз, даже если оно удовлетворяет условию несколько раз. - -**Возвращаемое значение** - -Максимальное количество последовательных сработавших условий из цепочки в пределах скользящего временного окна. -Анализируются все цепочки в выборке. - -Тип: `Integer`. - -**Пример** - -Определить, достаточно ли заданного периода времени, чтобы пользователь выбрал телефон и купил его дважды в интернет‑магазине. - -Задайте следующую цепочку событий: - -1. Пользователь вошёл в свой аккаунт в магазине (`eventID = 1003`). -2. Пользователь ищет телефон (`eventID = 1007, product = 'phone'`). -3. Пользователь оформил заказ (`eventID = 1009`). -4. Пользователь оформил повторный заказ (`eventID = 1010`). - -Входная таблица: - -```text -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-28 │ 1 │ 2019-01-29 10:00:00 │ 1003 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-31 │ 1 │ 2019-01-31 09:00:00 │ 1007 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-30 │ 1 │ 2019-01-30 08:00:00 │ 1009 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-02-01 │ 1 │ 2019-02-01 08:00:00 │ 1010 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -``` - -Узнайте, как далеко пользователь `user_id` продвинулся по цепочке за период январь–февраль 2019 года. - -Запрос: - -```sql -SELECT - level, - count() AS c -FROM -( - SELECT - user_id, - windowFunnel(6048000000000000)(timestamp, eventID = 1003, eventID = 1009, eventID = 1007, eventID = 1010) AS level - FROM trend - WHERE (event_date >= '2019-01-01') AND (event_date <= '2019-02-02') - GROUP BY user_id -) -GROUP BY level -ORDER BY level ASC; -``` - -Результат: - -```text -┌─level─┬─c─┐ -│ 4 │ 1 │ -└───────┴───┘ -``` - -## retention {#retention} - -Функция принимает в качестве аргументов набор из 1–32 условий типа `UInt8`, которые указывают, было ли выполнено определённое условие для события. -Любое условие может быть указано в качестве аргумента (как в [WHERE](/sql-reference/statements/select/where)). - -Условия, кроме первого, применяются попарно: результат второго будет `true`, если первое и второе истинны; третьего — если первое и третье истинны и т. д. - -**Синтаксис** - -```sql -retention(cond1, cond2, ..., cond32); -``` - -**Аргументы** - -* `cond` — выражение, которое возвращает результат типа `UInt8` (1 или 0). - -**Возвращаемое значение** - -Массив значений 1 или 0. - -* 1 — условие было выполнено для события. -* 0 — условие не было выполнено для события. - -Тип: `UInt8`. - -**Пример** - -Рассмотрим пример вычисления функции `retention` для определения трафика сайта. - -**1.** Создайте таблицу для иллюстрации. - -```sql -CREATE TABLE retention_test(date Date, uid Int32) ENGINE = Memory; - -INSERT INTO retention_test SELECT '2020-01-01', number FROM numbers(5); -INSERT INTO retention_test SELECT '2020-01-02', number FROM numbers(10); -INSERT INTO retention_test SELECT '2020-01-03', number FROM numbers(15); -``` - -Входная таблица: - -Запрос: - -```sql -SELECT * FROM retention_test -``` - -Результат: - -```text -┌───────date─┬─uid─┐ -│ 2020-01-01 │ 0 │ -│ 2020-01-01 │ 1 │ -│ 2020-01-01 │ 2 │ -│ 2020-01-01 │ 3 │ -│ 2020-01-01 │ 4 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-02 │ 0 │ -│ 2020-01-02 │ 1 │ -│ 2020-01-02 │ 2 │ -│ 2020-01-02 │ 3 │ -│ 2020-01-02 │ 4 │ -│ 2020-01-02 │ 5 │ -│ 2020-01-02 │ 6 │ -│ 2020-01-02 │ 7 │ -│ 2020-01-02 │ 8 │ -│ 2020-01-02 │ 9 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-03 │ 0 │ -│ 2020-01-03 │ 1 │ -│ 2020-01-03 │ 2 │ -│ 2020-01-03 │ 3 │ -│ 2020-01-03 │ 4 │ -│ 2020-01-03 │ 5 │ -│ 2020-01-03 │ 6 │ -│ 2020-01-03 │ 7 │ -│ 2020-01-03 │ 8 │ -│ 2020-01-03 │ 9 │ -│ 2020-01-03 │ 10 │ -│ 2020-01-03 │ 11 │ -│ 2020-01-03 │ 12 │ -│ 2020-01-03 │ 13 │ -│ 2020-01-03 │ 14 │ -└────────────┴─────┘ -``` - -**2.** Сгруппируйте пользователей по уникальному идентификатору `uid`, используя функцию `retention`. - -Запрос: - -```sql -SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r -FROM retention_test -WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') -GROUP BY uid -ORDER BY uid ASC -``` - -Результат: - -```text -┌─uid─┬─r───────┐ -│ 0 │ [1,1,1] │ -│ 1 │ [1,1,1] │ -│ 2 │ [1,1,1] │ -│ 3 │ [1,1,1] │ -│ 4 │ [1,1,1] │ -│ 5 │ [0,0,0] │ -│ 6 │ [0,0,0] │ -│ 7 │ [0,0,0] │ -│ 8 │ [0,0,0] │ -│ 9 │ [0,0,0] │ -│ 10 │ [0,0,0] │ -│ 11 │ [0,0,0] │ -│ 12 │ [0,0,0] │ -│ 13 │ [0,0,0] │ -│ 14 │ [0,0,0] │ -└─────┴─────────┘ -``` - -**3.** Рассчитайте общее количество посещений сайта за каждый день. - -Запрос: - -```sql -SELECT - sum(r[1]) AS r1, - sum(r[2]) AS r2, - sum(r[3]) AS r3 -FROM -( - SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r - FROM retention_test - WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') - GROUP BY uid -) -``` - -Результат: - -```text -┌─r1─┬─r2─┬─r3─┐ -│ 5 │ 5 │ 5 │ -└────┴────┴────┘ -``` - -Где: - -* `r1` — количество уникальных посетителей, которые посетили сайт за 2020-01-01 (условие `cond1`). -* `r2` — количество уникальных посетителей, которые посетили сайт в течение определённого периода времени между 2020-01-01 и 2020-01-02 (условия `cond1` и `cond2`). -* `r3` — количество уникальных посетителей, которые посетили сайт в течение определённого периода времени в даты 2020-01-01 и 2020-01-03 (условия `cond1` и `cond3`). - -## uniqUpTo(N)(x) {#uniquptonx} - -Вычисляет количество различных значений аргумента до заданного предела `N`. Если количество различных значений аргумента больше `N`, функция возвращает `N` + 1, в противном случае вычисляет точное значение. - -Рекомендуется использовать с небольшими `N`, до 10. Максимальное значение `N` — 100. - -В состоянии агрегатной функции эта функция использует объём памяти, равный 1 + `N` * размер одного значения в байтах. -При работе со строками она сохраняет некриптографический хэш размером 8 байт; для строк вычисление является приблизительным. - -Например, предположим, что у вас есть таблица, в которой регистрируется каждый поисковый запрос, сделанный пользователями на вашем сайте. Каждая строка в таблице представляет один поисковый запрос, со столбцами для идентификатора пользователя, текста поискового запроса и временной метки запроса. Вы можете использовать `uniqUpTo`, чтобы сформировать отчёт, который показывает только те ключевые слова, по которым было как минимум 5 уникальных пользователей. - -```sql -SELECT SearchPhrase -FROM SearchLog -GROUP BY SearchPhrase -HAVING uniqUpTo(4)(UserID) >= 5 -``` - -`uniqUpTo(4)(UserID)` вычисляет количество уникальных значений `UserID` для каждого `SearchPhrase`, но считает только до 4 уникальных значений. Если для какого-либо `SearchPhrase` существует более 4 уникальных значений `UserID`, функция возвращает 5 (4 + 1). Затем условие `HAVING` отфильтровывает значения `SearchPhrase`, для которых количество уникальных значений `UserID` меньше 5. В результате вы получите список поисковых запросов, которые использовались как минимум 5 уникальными пользователями. - -## sumMapFiltered {#summapfiltered} - -Эта функция ведёт себя так же, как [sumMap](/sql-reference/aggregate-functions/reference/summap), за исключением того, что она также принимает в качестве параметра массив ключей для фильтрации. Это может быть особенно полезно при работе с высокой кардинальностью ключей. - -**Синтаксис** - -`sumMapFiltered(keys_to_keep)(keys, values)` - -**Параметры** - -* `keys_to_keep`: [Array](../data-types/array.md) — массив ключей, по которым выполняется фильтрация. -* `keys`: [Array](../data-types/array.md) — массив ключей. -* `values`: [Array](../data-types/array.md) — массив значений. - -**Возвращаемое значение** - -* Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, суммированные по соответствующим ключам. - -**Пример** - -Запрос: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt16, requests UInt64) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) FROM sum_map; -``` - -Результат: - -```response - ┌─sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests)─┐ -1. │ ([1,4,8],[10,20,10]) │ - └─────────────────────────────────────────────────────────────────┘ -``` - -## sumMapFilteredWithOverflow {#summapfilteredwithoverflow} - -Эта функция ведёт себя так же, как [sumMap](/sql-reference/aggregate-functions/reference/summap), за исключением того, что она дополнительно принимает в качестве параметра массив ключей для фильтрации. Это может быть особенно полезно при работе с высокой кардинальностью ключей. Она отличается от функции [sumMapFiltered](#summapfiltered) тем, что выполняет суммирование с переполнением, то есть возвращает для результата суммирования тот же тип данных, что и тип данных аргумента. - -**Синтаксис** - -`sumMapFilteredWithOverflow(keys_to_keep)(keys, values)` - -**Параметры** - -* `keys_to_keep`: [Array](../data-types/array.md) ключей для фильтрации. -* `keys`: [Array](../data-types/array.md) ключей. -* `values`: [Array](../data-types/array.md) значений. - -**Возвращаемое значение** - -* Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, суммированные для соответствующих ключей. - -**Пример** - -В этом примере мы создаём таблицу `sum_map`, вставляем в неё некоторые данные, а затем используем `sumMapFilteredWithOverflow` и `sumMapFiltered`, а также функцию `toTypeName` для сравнения результата. Поскольку `requests` имел тип `UInt8` в созданной таблице, `sumMapFiltered` привела тип суммируемых значений к `UInt64`, чтобы избежать переполнения, тогда как `sumMapFilteredWithOverflow` сохранила тип `UInt8`, который недостаточно велик для хранения результата, то есть произошло переполнение. - -Запрос: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt8, requests UInt8) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFilteredWithOverflow([1, 4, 8])(statusMap.status, statusMap.requests) as summap_overflow, toTypeName(summap_overflow) FROM sum_map; -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) as summap, toTypeName(summap) FROM sum_map; -``` - -Результат: - -```response - ┌─sum──────────────────┬─toTypeName(sum)───────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt8)) │ - └──────────────────────┴───────────────────────────────────┘ -``` - -```response - ┌─summap───────────────┬─toTypeName(summap)─────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt64)) │ - └──────────────────────┴────────────────────────────────────┘ -``` - -## sequenceNextNode {#sequencenextnode} - -Возвращает значение следующего события, которое соответствует цепочке событий. - -*Экспериментальная функция, включается с помощью `SET allow_experimental_funnel_functions = 1`.* - -**Синтаксис** - -```sql -sequenceNextNode(direction, base)(timestamp, event_column, base_condition, event1, event2, event3, ...) -``` - -**Параметры** - -* `direction` — Используется для указания направления поиска. - * forward — Движение вперёд. - * backward — Движение назад. - -* `base` — Используется для задания опорной точки. - * head — Устанавливает опорную точку на первое событие. - * tail — Устанавливает опорную точку на последнее событие. - * first_match — Устанавливает опорную точку на первое совпавшее `event1`. - * last_match — Устанавливает опорную точку на последнее совпавшее `event1`. - -**Аргументы** - -* `timestamp` — Имя столбца, содержащего метку времени. Поддерживаемые типы данных: [Date](../../sql-reference/data-types/date.md), [DateTime](/sql-reference/data-types/datetime) и другие беззнаковые целочисленные типы. -* `event_column` — Имя столбца, содержащего значение следующего события, которое должно быть возвращено. Поддерживаемые типы данных: [String](../../sql-reference/data-types/string.md) и [Nullable(String)](../../sql-reference/data-types/nullable.md). -* `base_condition` — Условие, которому должна удовлетворять опорная точка. -* `event1`, `event2`, ... — Условия, описывающие цепочку событий. Тип данных: [UInt8](../../sql-reference/data-types/int-uint.md). - -**Возвращаемые значения** - -* `event_column[next_index]` — Если шаблон совпал и следующее значение существует. -* `NULL` — Если шаблон не совпал или следующее значение не существует. - -Тип: [Nullable(String)](../../sql-reference/data-types/nullable.md). - -**Пример** - -Функцию можно использовать, когда события имеют вид A->B->C->D->E и нужно узнать событие, следующее за B->C, то есть D. - -Пример запроса, который ищет событие, следующее за A->B: - -```sql -CREATE TABLE test_flow ( - dt DateTime, - id int, - page String) -ENGINE = MergeTree() -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow VALUES (1, 1, 'A') (2, 1, 'B') (3, 1, 'C') (4, 1, 'D') (5, 1, 'E'); - -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'A', page = 'A', page = 'B') as next_flow FROM test_flow GROUP BY id; -``` - -Результат: - -```text -┌─id─┬─next_flow─┐ -│ 1 │ C │ -└────┴───────────┘ -``` - -**Поведение `forward` и `head`** - -```sql -ALTER TABLE test_flow DELETE WHERE 1 = 1 settings mutations_sync = 1; - -INSERT INTO test_flow VALUES (1, 1, 'Home') (2, 1, 'Gift') (3, 1, 'Exit'); -INSERT INTO test_flow VALUES (1, 2, 'Home') (2, 2, 'Home') (3, 2, 'Gift') (4, 2, 'Basket'); -INSERT INTO test_flow VALUES (1, 3, 'Gift') (2, 3, 'Home') (3, 3, 'Gift') (4, 3, 'Basket'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'Home', page = 'Home', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page - 1970-01-01 09:00:01 1 Home // Исходная точка, совпадает с Home - 1970-01-01 09:00:02 1 Gift // Совпадает с Gift - 1970-01-01 09:00:03 1 Exit // Результат - - 1970-01-01 09:00:01 2 Home // Исходная точка, совпадает с Home - 1970-01-01 09:00:02 2 Home // Не совпадает с Gift - 1970-01-01 09:00:03 2 Gift - 1970-01-01 09:00:04 2 Basket -``` - -1970-01-01 09:00:01 3 Gift // Опорная точка, не сопоставлена с Home -1970-01-01 09:00:02 3 Home -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket - -```` - -**Поведение для `backward` и `tail`** - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, page = 'Basket', page = 'Basket', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift -1970-01-01 09:00:03 1 Exit // Базовая точка, не соответствует Basket - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // Результат -1970-01-01 09:00:03 2 Gift // Соответствует Gift -1970-01-01 09:00:04 2 Basket // Базовая точка, соответствует Basket - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // Результат -1970-01-01 09:00:03 3 Gift // Базовая точка, соответствует Gift -1970-01-01 09:00:04 3 Basket // Базовая точка, соответствует Basket -```` - -**Поведение режимов `forward` и `first_match`** - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // Базовая точка -1970-01-01 09:00:03 1 Exit // Результат - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // Базовая точка -1970-01-01 09:00:04 2 Basket // Результат - -1970-01-01 09:00:01 3 Gift // Базовая точка -1970-01-01 09:00:02 3 Home // Результат -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // Базовая точка -1970-01-01 09:00:03 1 Exit // Не совпадает с Home - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // Базовая точка -1970-01-01 09:00:04 2 Basket // Не совпадает с Home - -1970-01-01 09:00:01 3 Gift // Базовая точка -1970-01-01 09:00:02 3 Home // Совпадает с Home -1970-01-01 09:00:03 3 Gift // Результат -1970-01-01 09:00:04 3 Basket -``` - -**Поведение `backward` и `last_match`** - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; -``` - -dt id page -1970-01-01 09:00:01 1 Home // Результат -1970-01-01 09:00:02 1 Gift // Базовая страница -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // Результат -1970-01-01 09:00:03 2 Gift // Базовая страница -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // Результат -1970-01-01 09:00:03 3 Gift // Базовая страница -1970-01-01 09:00:04 3 Basket - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home // Совпадает с Home, результат — null -1970-01-01 09:00:02 1 Gift // Базовая точка -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home // Результат -1970-01-01 09:00:02 2 Home // Совпадает с Home -1970-01-01 09:00:03 2 Gift // Базовая точка -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift // Результат -1970-01-01 09:00:02 3 Home // Совпадает с Home -1970-01-01 09:00:03 3 Gift // Базовая точка -1970-01-01 09:00:04 3 Basket -```` - -**Поведение параметра `base_condition`** - -```sql -CREATE TABLE test_flow_basecond -( - `dt` DateTime, - `id` int, - `page` String, - `ref` String -) -ENGINE = MergeTree -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow_basecond VALUES (1, 1, 'A', 'ref4') (2, 1, 'A', 'ref3') (3, 1, 'B', 'ref2') (4, 1, 'B', 'ref1'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, ref = 'ref1', page = 'A') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 // Начальная строка не может быть базовой точкой, поскольку значение столбца ref начальной строки не соответствует 'ref1'. - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 -``` - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, ref = 'ref4', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 // Хвост не может быть базовой точкой, поскольку значение столбца ref в хвосте не соответствует 'ref4'. -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, ref = 'ref3', page = 'A') FROM test_flow_basecond GROUP BY id; -``` - -dt id page ref -1970-01-01 09:00:01 1 A ref4 // Эта строка не может быть опорной точкой, потому что значение в столбце ref не совпадает с 'ref3'. -1970-01-01 09:00:02 1 A ref3 // Опорная точка -1970-01-01 09:00:03 1 B ref2 // Результат -1970-01-01 09:00:04 1 B ref1 - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, ref = 'ref2', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 // Результат - 1970-01-01 09:00:03 1 B ref2 // Базовая точка - 1970-01-01 09:00:04 1 B ref1 // Эта строка не может быть базовой точкой, поскольку значение столбца ref не совпадает с 'ref2'. -```` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md deleted file mode 100644 index 17b7e32a1d3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Эта функция может использоваться для тестирования устойчивости к исключениям. - Она будет выбрасывать исключение при создании с указанной вероятностью.' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/aggthrow -title: 'aggThrow' -doc_type: 'reference' ---- - -# aggThrow {#aggthrow} - -Эта функция может использоваться для тестирования устойчивости к исключениям. Она будет выбрасывать исключение при инициализации с указанной вероятностью. - -**Синтаксис** - -```sql -aggThrow(throw_prob) -``` - -**Аргументы** - -* `throw_prob` — Вероятность выброса исключения при создании. [Float64](../../data-types/float.md). - -**Возвращаемое значение** - -* Исключение: `Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully`. - -**Пример** - -Запрос: - -```sql -SELECT number % 2 AS even, aggThrow(number) FROM numbers(10) GROUP BY even; -``` - -Результат: - -```response -Получено исключение: -Code: 503. DB::Exception: Агрегатная функция aggThrow корректно вызвала исключение: при выполнении AggregatingTransform. (AGGREGATE_FUNCTION_THROW) -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md deleted file mode 100644 index 98344c2b827..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'Предоставляет статистический критерий для однофакторного дисперсионного анализа (ANOVA). Это критерий для нескольких групп нормально распределённых наблюдений, позволяющий проверить, совпадают ли средние значения во всех группах.' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/analysis_of_variance -title: 'analysisOfVariance' -doc_type: 'reference' ---- - -# analysisOfVariance {#analysisofvariance} - -Выполняет статистический тест однофакторного дисперсионного анализа (ANOVA). Это тест для нескольких групп нормально распределённых наблюдений, который позволяет определить, одинаковы ли средние значения во всех группах или нет. - -**Синтаксис** - -```sql -analysisOfVariance(val, group_no) -``` - -Псевдонимы: `anova` - -**Параметры** - -* `val`: значение. -* `group_no` : номер группы, к которой принадлежит `val`. - -:::note -Группы нумеруются начиная с 0, и для выполнения теста необходимо как минимум две группы. -Должна быть как минимум одна группа, в которой число наблюдений больше одного. -::: - -**Возвращаемое значение** - -* `(f_statistic, p_value)`. [Tuple](../../data-types/tuple.md)([Float64](../../data-types/float.md), [Float64](../../data-types/float.md)). - -**Пример** - -Запрос: - -```sql -SELECT analysisOfVariance(number, number % 2) FROM numbers(1048575); -``` - -Результат: - -```response -┌─analysisOfVariance(number, modulo(number, 2))─┐ -│ (0,1) │ -└───────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md deleted file mode 100644 index 87df82cdfb3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'Выбирает первое попавшееся значение столбца.' -sidebar_position: 102 -slug: /sql-reference/aggregate-functions/reference/any -title: 'any' -doc_type: 'reference' ---- - -# any {#any} - -Выбирает первое встретившееся значение в столбце. - -:::warning -Так как запрос может выполняться в произвольном порядке, результат этой функции является недетерминированным. -Если вам нужен произвольный, но детерминированный результат, используйте функции [`min`](../reference/min.md) или [`max`](../reference/max.md). -::: - -По умолчанию функция никогда не возвращает NULL, т.е. игнорирует значения NULL во входном столбце. -Однако, если функция используется с модификатором `RESPECT NULLS`, она возвращает первое прочитанное значение, независимо от того, является оно NULL или нет. - -**Синтаксис** - -```sql -any(столбец) [RESPECT NULLS] -``` - -Псевдонимы `any(column)` (без `RESPECT NULLS`) - -* `any_value` -* [`first_value`](../reference/first_value.md). - -Псевдонимы для `any(column) RESPECT NULLS` - -* `anyRespectNulls`, `any_respect_nulls` -* `firstValueRespectNulls`, `first_value_respect_nulls` -* `anyValueRespectNulls`, `any_value_respect_nulls` - -**Параметры** - -* `column`: имя столбца. - -**Возвращаемое значение** - -Первое встреченное значение. - -:::note -Тип возвращаемого значения функции совпадает с типом аргумента, за исключением LowCardinality, который отбрасывается. -Это означает, что при отсутствии строк во входных данных функция вернёт значение по умолчанию для этого типа (0 для целых чисел или Null для столбца типа Nullable()). -Вы можете использовать комбинатор `-OrNull` [combinator](../../../sql-reference/aggregate-functions/combinators.md), чтобы изменить это поведение. -::: - -**Особенности реализации** - -В некоторых случаях можно полагаться на порядок выполнения. -Это относится к ситуациям, когда `SELECT` берётся из подзапроса, использующего `ORDER BY`. - -Когда запрос `SELECT` содержит предложение `GROUP BY` или хотя бы одну агрегатную функцию, ClickHouse (в отличие от MySQL) требует, чтобы все выражения в предложениях `SELECT`, `HAVING` и `ORDER BY` вычислялись из ключей или из агрегатных функций. -Другими словами, каждый столбец, выбираемый из таблицы, должен использоваться либо в ключах, либо внутри агрегатных функций. -Чтобы получить поведение, похожее на MySQL, вы можете обернуть остальные столбцы в агрегатную функцию `any`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES (NULL), ('Amsterdam'), ('New York'), ('Tokyo'), ('Valencia'), (NULL); - -SELECT any(city), anyRespectNulls(city) FROM tab; -``` - -```response -┌─any(city)─┬─anyRespectNulls(city)─┐ -│ Amsterdam │ ᴺᵁᴸᴸ │ -└───────────┴───────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md deleted file mode 100644 index 08ba03d5541..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: 'Выбирает часто встречающееся значение с использованием алгоритма «heavy hitters». - Если существует значение, которое встречается более чем в половине случаев в каждом - потоке выполнения запроса, возвращается именно оно. Результат, как правило, недетерминирован.' -sidebar_position: 104 -slug: /sql-reference/aggregate-functions/reference/anyheavy -title: 'anyHeavy' -doc_type: 'reference' ---- - -# anyHeavy {#anyheavy} - -Выбирает часто встречающееся значение с использованием алгоритма [heavy hitters](https://doi.org/10.1145/762471.762473). Если существует значение, которое встречается более чем в половине случаев в каждом потоке выполнения запроса, возвращается именно оно. Обычно результат недетерминирован. - -```sql -anyHeavy(столбец) -``` - -**Аргументы** - -* `column` – имя столбца. - -**Пример** - -Возьмите набор данных [OnTime](../../../getting-started/example-datasets/ontime.md) и выберите любое часто встречающееся значение в столбце `AirlineID`. - -```sql -SELECT anyHeavy(AirlineID) AS res -FROM ontime -``` - -```text -┌───res─┐ -│ 19690 │ -└───────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md deleted file mode 100644 index d14e4b76c93..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: 'Выбирает последнее встреченное значение в столбце.' -sidebar_position: 105 -slug: /sql-reference/aggregate-functions/reference/anylast -title: 'anyLast' -doc_type: 'reference' ---- - -# anyLast {#anylast} - -Выбирает последнее встреченное значение столбца. - -:::warning -Поскольку запрос может выполняться в произвольном порядке, результат этой функции является недетерминированным. -Если вам нужен произвольный, но детерминированный результат, используйте функции [`min`](../reference/min.md) или [`max`](../reference/max.md). -::: - -По умолчанию функция никогда не возвращает NULL, то есть игнорирует значения NULL во входном столбце. -Однако, если функция используется с модификатором `RESPECT NULLS`, она возвращает первое прочитанное значение, независимо от того, является оно NULL или нет. - -**Синтаксис** - -```sql -anyLast(column) [RESPECT NULLS] -``` - -Псевдоним `anyLast(column)` (без `RESPECT NULLS`) - -* [`last_value`](../reference/last_value.md). - -Псевдонимы для `anyLast(column) RESPECT NULLS` - -* `anyLastRespectNulls`, `anyLast_respect_nulls` -* `lastValueRespectNulls`, `last_value_respect_nulls` - -**Параметры** - -* `column`: имя столбца. - -**Возвращаемое значение** - -* Последнее встреченное значение. - -**Пример** - -Запрос: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES ('Amsterdam'),(NULL),('New York'),('Tokyo'),('Valencia'),(NULL); - -SELECT anyLast(city), anyLastRespectNulls(city) FROM tab; -``` - -```response -┌─anyLast(city)─┬─anyLastRespectNulls(city)─┐ -│ Valencia │ ᴺᵁᴸᴸ │ -└───────────────┴───────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md deleted file mode 100644 index 79364e137eb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: 'Возвращает массив приблизительно наиболее частых значений и количества их вхождений в указанном столбце.' -sidebar_position: 107 -slug: /sql-reference/aggregate-functions/reference/approxtopk -title: 'approx_top_k' -doc_type: 'reference' ---- - -# approx_top_k {#approx_top_k} - -Возвращает массив приблизительно наиболее часто встречающихся значений и количества их вхождений в указанном столбце. Результирующий массив отсортирован по убыванию приблизительной частоты значений (а не по самим значениям). - -```sql -approx_top_k(N)(column) -approx_top_k(N, reserved)(column) -``` - -Эта функция не даёт гарантированного результата. В отдельных случаях возможны ошибки, и она может возвращать часто встречающиеся значения, которые на самом деле не являются самыми частотными. - -Максимальное значение `N = 65536`. - -**Параметры** - -* `N` — количество возвращаемых элементов. Необязательный параметр. Значение по умолчанию: 10. -* `reserved` — определяет, сколько ячеек зарезервировать под значения. Если uniq(column) > reserved, результат работы функции topK будет приблизительным. Необязательный параметр. Значение по умолчанию: N * 3. - -**Аргументы** - -* `column` — значение, для которого вычисляется частота. - -**Пример** - -Запрос: - -```sql -SELECT approx_top_k(2)(k) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)); -``` - -Результат: - -```text -┌─approx_top_k(2)(k)────┐ -│ [('y',3,0),('x',1,0)] │ -└───────────────────────┘ -``` - -# approx_top_count {#approx_top_count} - -Является синонимом функции `approx_top_k` - -**См. также** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md deleted file mode 100644 index bd182b217d5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'Возвращает массив приближённо самых частых значений и числа их вхождений в указанном столбце.' -sidebar_position: 108 -slug: /sql-reference/aggregate-functions/reference/approxtopsum -title: 'approx_top_sum' -doc_type: 'reference' ---- - -# approx_top_sum {#approx_top_sum} - -Возвращает массив приблизительно самых частых значений и количества их вхождений в указанном столбце. Полученный массив сортируется в порядке убывания приблизительной частоты значений (не по самим значениям). Дополнительно учитывается вес значения. - -```sql -approx_top_sum(N)(column, weight) -approx_top_sum(N, reserved)(column, weight) -``` - -Эта функция не гарантирует точный результат. В некоторых ситуациях возможны ошибки, и она может возвращать часто встречающиеся значения, которые не являются самыми частыми. - -Максимальное значение параметра `N` — 65536. - -**Параметры** - -* `N` — количество элементов для возврата. Необязательный параметр. Значение по умолчанию: 10. -* `reserved` — определяет, сколько ячеек зарезервировать для значений. Если uniq(column) > reserved, результат функции topK будет приближённым. Необязательный параметр. Значение по умолчанию: N * 3. - -**Аргументы** - -* `column` — значение, для которого вычисляется частота. -* `weight` — вес. Каждое значение учитывается `weight` раз при вычислении частоты. [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Пример** - -Запрос: - -```sql -SELECT approx_top_sum(2)(k, w) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -Результат: - -```text -┌─approx_top_sum(2)(k, w)─┐ -│ [('z',10,0),('x',5,0)] │ -└─────────────────────────┘ -``` - -**Смотрите также** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md deleted file mode 100644 index 9efe31cb20a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -description: 'Вычисляет значения `arg` и `val` для максимального значения `val`. Если есть несколько строк с одинаковым максимальным значением `val`, то то, какие связанные `arg` и `val` будут возвращены, недетерминировано.' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmax -title: 'argAndMax' -doc_type: 'reference' ---- - -# argAndMax {#argandmax} - -Вычисляет значения `arg` и `val` для максимального значения `val`. Если существует несколько строк с одинаковым максимальным значением `val`, то какая из соответствующих пар `arg` и `val` будет возвращена — недетерминировано. -Обе части — `arg` и `max` — ведут себя как [агрегатные функции](/sql-reference/aggregate-functions/index.md), обе [пропускают `Null`](/sql-reference/aggregate-functions/index.md#null-processing) при обработке и возвращают значения, отличные от `Null`, если такие значения доступны. - -:::note -Единственное отличие от `argMax` состоит в том, что `argAndMax` возвращает и аргумент, и значение. -::: - -**Синтаксис** - -```sql -argAndMax(arg, val) -``` - -**Аргументы** - -* `arg` — аргумент. -* `val` — Значение. - -**Возвращаемое значение** - -* значение `arg`, соответствующее максимальному значению `val`. -* максимальное значение `val` - -Тип: кортеж, соответствующий типам `arg` и `val` соответственно. - -**Пример** - -Входная таблица: - -```text -┌─пользователь─┬─зарплата─┐ -│ директор │ 5000 │ -│ менеджер │ 3000 │ -│ работник │ 1000 │ -└──────────────┴──────────┘ -``` - -Запрос: - -```sql -SELECT argAndMax(user, salary) FROM salary; -``` - -Результат: - -```text -┌─argAndMax(user, salary)─┐ -│ ('director',5000) │ -└─────────────────────────┘ -``` - -**Расширенный пример** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), argAndMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─argAndMax(a, b)─┬─max(b)─┐ -│ b │ ('b',2) │ 3 │ -- argMax = b, так как это первое не-NULL значение; max(b) взято из другой строки! -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMax(tuple(a), b) FROM test; -┌─argAndMax((a), b)─┐ -│ ((NULL),3) │-- `Tuple`, содержащий только значение `NULL`, сам не является `NULL`, поэтому агрегатные функции не пропустят эту строку из-за данного значения `NULL` -└───────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA──┬─argMaxB─┐ -│ (NULL,3) │ 3 │ -- можно использовать Tuple и получить оба столбца (все — tuple(*)) для соответствующего max(b) -└──────────┴─────────┘ - -SELECT argAndMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argAndMax(a, b)─┬─max(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │-- Все агрегируемые строки содержат хотя бы одно значение `NULL` из-за фильтра, поэтому все строки пропускаются, следовательно, результат будет `NULL` -└─────────────────┴────────┘ - -SELECT argAndMax(a, (b,a)) FROM test; -┌─argAndMax(a, (b, a))─┐ -│ ('c',(2,'c')) │ -- Имеется две строки с b=2; `Tuple` в `Max` позволяет получить не первый `arg` -└──────────────────────┘ - -SELECT argAndMax(a, tuple(b)) FROM test; -┌─argAndMax(a, (b))─┐ -│ ('b',(2)) │ -- `Tuple` можно использовать в `Max`, чтобы не пропускать NULL-значения в `Max` -└───────────────────┘ -``` - -**См. также** - -* [argMax](/sql-reference/aggregate-functions/reference/argmax.md) -* [Tuple](/sql-reference/data-types/tuple.md) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md deleted file mode 100644 index a948fbbd2c3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: 'Вычисляет значения `arg` и `val` для минимального значения `val`. Если существует несколько строк с одинаковым минимальным значением `val`, то то, какие связанные значения `arg` и `val` будут возвращены, не определено.' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmin -title: 'argAndMin' -doc_type: 'reference' ---- - -# argAndMin {#argandmin} - -Вычисляет значения `arg` и `val` для минимального значения `val`. Если существует несколько строк с одинаковым минимальным значением `val`, то выбор того, какие связанные `arg` и `val` будут возвращены, недетерминирован. -Обе части — `arg` и `val` — ведут себя как [агрегатные функции](/sql-reference/aggregate-functions/index.md), обе при обработке [пропускают значения `Null`](/sql-reference/aggregate-functions/index.md#null-processing) и возвращают значения, отличные от `Null`, если такие значения доступны. - -:::note -Единственное отличие от `argMin` заключается в том, что `argAndMin` возвращает и аргумент, и значение. -::: - -**Синтаксис** - -```sql -argAndMin(arg, val) -``` - -**Аргументы** - -* `arg` — аргумент. -* `val` — значение. - -**Возвращаемое значение** - -* Значение `arg`, которое соответствует минимальному значению `val`. -* `val` — минимальное значение `val`. - -Тип: кортеж, в котором типы элементов соответствуют типам `arg` и `val` соответственно. - -**Пример** - -Входная таблица: - -```text -┌─пользователь─┬─зарплата─┐ -│ директор │ 5000 │ -│ менеджер │ 3000 │ -│ работник │ 1000 │ -└──────────────┴──────────┘ -``` - -Запрос: - -```sql -SELECT argAndMin(user, salary) FROM salary -``` - -Результат: - -```text -┌─argAndMin(user, salary)─┐ -│ ('worker',1000) │ -└─────────────────────────┘ -``` - -**Расширенный пример** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a,b), argAndMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─argAndMin(a, b)─┬─min(b)─┐ -│ a │ ('a',1) │ 0 │ -- argMin = a, так как это первое значение, отличное от `NULL`; min(b) взято из другой строки! -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMin(tuple(a), b) FROM test; -┌─argAndMin((a), b)─┐ -│ ((NULL),0) │ -- `Tuple` 'a', содержащий только значение `NULL`, сам не является `NULL`, поэтому агрегатные функции не пропустят эту строку из-за данного значения `NULL` -└───────────────────┘ - -SELECT (argAndMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA──┬─argMinB─┐ -│ (NULL,0) │ 0 │ -- можно использовать `Tuple` и получить оба столбца (все столбцы — tuple(*)) для соответствующего min(b) -└──────────┴─────────┘ - -SELECT argAndMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argAndMin(a, b)─┬─min(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │ -- Все агрегируемые строки содержат хотя бы одно значение `NULL` из-за фильтра, поэтому все строки пропускаются, следовательно, результат будет `NULL` -└─────────────────┴────────┘ - -SELECT argAndMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin(a, (b, a))─┬─min((b, a))─┐ -│ ('a',(1,'a')) │ (0,NULL) │ -- 'a' — это первое значение, отличное от `NULL`, для min -└──────────────────────┴─────────────┘ - -SELECT argAndMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin((a, b), (b, a))─┬─min((b, a))─┐ -│ ((NULL,0),(0,NULL)) │ (0,NULL) │ -- argAndMin возвращает ((NULL,0),(0,NULL)), так как `Tuple` позволяет не пропускать `NULL`, и min(tuple(b, a)) в данном случае является минимальным значением для этого набора данных -└───────────────────────────┴─────────────┘ - -SELECT argAndMin(a, tuple(b)) FROM test; -┌─argAndMin(a, (b))─┐ -│ ('a',(1)) │ -- `Tuple` можно использовать в `min`, чтобы не пропускать строки со значениями `NULL` в b. -└───────────────────┘ -``` - -**См. также** - -* [argMin](/sql-reference/aggregate-functions/reference/argmin.md) -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md deleted file mode 100644 index 5c1112e06be..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -description: 'Вычисляет значение `arg` для максимального значения `val`.' -sidebar_position: 109 -slug: /sql-reference/aggregate-functions/reference/argmax -title: 'argMax' -doc_type: 'reference' ---- - -# argMax {#argmax} - -Вычисляет значение `arg` для максимального значения `val`. Если существует несколько строк с одинаковым максимальным `val`, то то, какое из соответствующих значений `arg` будет возвращено, не детерминировано. -Обе части — и `arg`, и `max` — ведут себя как [агрегатные функции](/sql-reference/aggregate-functions/index.md), при обработке обе [пропускают `Null`](/sql-reference/aggregate-functions/index.md#null-processing) и возвращают значения, отличные от `Null`, если такие значения есть. - -**Синтаксис** - -```sql -argMax(arg, val) -``` - -**Аргументы** - -* `arg` — аргумент. -* `val` — значение. - -**Возвращаемое значение** - -* Значение `arg`, которое соответствует максимальному значению `val`. - -Тип: тот же, что и у `arg`. - -**Пример** - -Входная таблица: - -```text -┌─пользователь─┬─зарплата─┐ -│ директор │ 5000 │ -│ менеджер │ 3000 │ -│ работник │ 1000 │ -└──────────────┴──────────┘ -``` - -Запрос: - -```sql -SELECT argMax(user, salary) FROM salary; -``` - -Результат: - -```text -┌─argMax(user, salary)─┐ -│ director │ -└──────────────────────┘ -``` - -**Расширенный пример** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─max(b)─┐ -│ b │ 3 │ -- argMax = 'b', так как это первое значение, отличное от Null; max(b) взято из другой строки! -└──────────────┴────────┘ - -SELECT argMax(tuple(a), b) FROM test; -┌─argMax(tuple(a), b)─┐ -│ (NULL) │ -- `Tuple`, содержащий только значение `NULL`, сам не является `NULL`, поэтому агрегатные функции не пропустят эту строку из-за этого значения `NULL` -└─────────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA─┬─argMaxB─┐ -│ ᴺᵁᴸᴸ │ 3 │ -- можно использовать Tuple и получить оба столбца (все — tuple(*)) для соответствующего max(b) -└─────────┴─────────┘ - -SELECT argMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argMax(a, b)─┬─max(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- Все агрегируемые строки содержат хотя бы одно значение `NULL` из-за фильтра, поэтому все строки пропускаются, следовательно, результат будет `NULL` -└──────────────┴────────┘ - -SELECT argMax(a, (b,a)) FROM test; -┌─argMax(a, tuple(b, a))─┐ -│ c │ -- Есть две строки с b=2; `Tuple` в `Max` позволяет получить не первый `arg` -└────────────────────────┘ - -SELECT argMax(a, tuple(b)) FROM test; -┌─argMax(a, tuple(b))─┐ -│ b │ -- `Tuple` можно использовать в `Max`, чтобы не пропускать значения Null в `Max` -└─────────────────────┘ -``` - -**Смотрите также** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md deleted file mode 100644 index 7c1df965a05..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'Вычисляет значение `arg` для минимального значения `val`. Если существует - несколько строк с одинаковым минимальным значением `val`, то то, какое из соответствующих - значений `arg` будет возвращено, не детерминировано.' -sidebar_position: 110 -slug: /sql-reference/aggregate-functions/reference/argmin -title: 'argMin' -doc_type: 'reference' ---- - -# argMin {#argmin} - -Вычисляет значение `arg` для минимального значения `val`. Если имеется несколько строк с одинаковым значением `val`, являющимся минимальным, то выбор возвращаемого связанного значения `arg` не детерминирован. - -Обе части — `arg` и `min` — ведут себя как [агрегатные функции](/sql-reference/aggregate-functions/index.md), обе [пропускают `Null`](/sql-reference/aggregate-functions/index.md#null-processing) при обработке и возвращают значения, отличные от `Null`, если такие значения доступны. - -**Синтаксис** - -```sql -argMin(arg, val) -``` - -**Аргументы** - -* `arg` — аргумент. -* `val` — значение. - -**Возвращаемое значение** - -* Значение `arg`, соответствующее минимальному значению `val`. - -Тип: тот же, что и у `arg`. - -**Пример** - -Входная таблица: - -```text -┌─пользователь─┬─зарплата─┐ -│ директор │ 5000 │ -│ менеджер │ 3000 │ -│ работник │ 1000 │ -└──────────────┴──────────┘ -``` - -Запрос: - -```sql -SELECT argMin(user, salary) FROM salary -``` - -Результат: - -```text -┌─argMin(user, salary)─┐ -│ worker │ -└──────────────────────┘ -``` - -**Расширенный пример** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─min(b)─┐ -│ a │ 0 │ -- argMin = a, так как это первое не-`NULL` значение; min(b) взято из другой строки! -└──────────────┴────────┘ - -SELECT argMin(tuple(a), b) FROM test; -┌─argMin(tuple(a), b)─┐ -│ (NULL) │ -- `Tuple`, содержащий только `NULL` значение, сам по себе не является `NULL`, поэтому агрегатные функции не пропустят эту строку из-за данного `NULL` значения -└─────────────────────┘ - -SELECT (argMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA─┬─argMinB─┐ -│ ᴺᵁᴸᴸ │ 0 │ -- можно использовать `Tuple` и получить оба (все — tuple(*)) столбца для соответствующего max(b) -└─────────┴─────────┘ - -SELECT argMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argMin(a, b)─┬─min(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- Все агрегируемые строки содержат хотя бы одно `NULL` значение из-за фильтра, поэтому все строки пропускаются, следовательно, результат будет `NULL` -└──────────────┴────────┘ - -SELECT argMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(a, tuple(b, a))─┬─min(tuple(b, a))─┐ -│ d │ (NULL,NULL) │ -- 'd' является первым не-`NULL` значением для min -└────────────────────────┴──────────────────┘ - -SELECT argMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(tuple(a, b), tuple(b, a))─┬─min(tuple(b, a))─┐ -│ (NULL,NULL) │ (NULL,NULL) │ -- argMin возвращает (NULL,NULL), так как `Tuple` позволяет не пропускать `NULL`, и min(tuple(b, a)) в данном случае является минимальным значением для этого набора данных -└──────────────────────────────────┴──────────────────┘ - -SELECT argMin(a, tuple(b)) FROM test; -┌─argMin(a, tuple(b))─┐ -│ d │ -- `Tuple` можно использовать в `min`, чтобы не пропускать строки с `NULL` значениями в b. -└─────────────────────┘ -``` - -**См. также** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md deleted file mode 100644 index c120834ccaa..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'Вычисляет среднее арифметическое.' -sidebar_position: 112 -slug: /sql-reference/aggregate-functions/reference/avg -title: 'avg' -doc_type: 'reference' ---- - -# avg {#avg} - -Вычисляет среднее арифметическое. - -**Синтаксис** - -```sql -avg(x) -``` - -**Аргументы** - -* `x` — входные значения, должны иметь тип [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемое значение** - -* Среднее арифметическое, всегда типа [Float64](../../../sql-reference/data-types/float.md). -* `NaN`, если входной параметр `x` пуст. - -**Пример** - -Запрос: - -```sql -SELECT avg(x) FROM VALUES('x Int8', 0, 1, 2, 3, 4, 5); -``` - -Результат: - -```text -┌─avg(x)─┐ -│ 2.5 │ -└────────┘ -``` - -**Пример** - -Создайте временную таблицу: - -Запрос: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -``` - -Найдите среднее арифметическое: - -Запрос: - -```sql -SELECT avg(t) FROM test; -``` - -Результат: - -```text -┌─avg(x)─┐ -│ nan │ -└────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md deleted file mode 100644 index dbfe9b5ac8e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -description: 'Вычисляет взвешенное среднее арифметическое.' -sidebar_position: 113 -slug: /sql-reference/aggregate-functions/reference/avgweighted -title: 'avgWeighted' -doc_type: 'reference' ---- - -# avgWeighted {#avgweighted} - -Вычисляет [взвешенное среднее арифметическое](https://en.wikipedia.org/wiki/Weighted_arithmetic_mean). - -**Синтаксис** - -```sql -avgWeighted(x, weight) -``` - -**Аргументы** - -* `x` — значения. -* `weight` — веса значений. - -`x` и `weight` должны быть либо -[целыми числами](../../../sql-reference/data-types/int-uint.md), либо [числами с плавающей запятой](../../../sql-reference/data-types/float.md), -но могут иметь разные типы. - -**Возвращаемое значение** - -* `NaN`, если все веса равны 0 или переданный параметр весов пуст. -* Взвешенное среднее в остальных случаях. - -**Тип возвращаемого значения** всегда [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Запрос: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (4, 1), (1, 0), (10, 2)) -``` - -Результат: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**Пример** - -Запрос: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Float64', (4, 1), (1, 0), (10, 2)) -``` - -Результат: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**Пример** - -Запрос: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (0, 0), (1, 0), (10, 0)) -``` - -Результат: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` - -**Пример** - -Запрос: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -SELECT avgWeighted(t) FROM test -``` - -Результат: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md deleted file mode 100644 index 9885f997b52..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет наклон между самой левой и самой правой точками в группе значений.' -sidebar_position: 114 -slug: /sql-reference/aggregate-functions/reference/boundingRatio -title: 'boundingRatio' -doc_type: 'reference' ---- - -Агрегатная функция, которая вычисляет наклон между самой левой и самой правой точками в группе значений. - -Пример: - -Пример данных: - -```sql -SELECT - number, - number * 1.5 -FROM numbers(10) -``` - -```response -┌─number─┬─multiply(number, 1.5)─┐ -│ 0 │ 0 │ -│ 1 │ 1.5 │ -│ 2 │ 3 │ -│ 3 │ 4.5 │ -│ 4 │ 6 │ -│ 5 │ 7.5 │ -│ 6 │ 9 │ -│ 7 │ 10.5 │ -│ 8 │ 12 │ -│ 9 │ 13.5 │ -└────────┴───────────────────────┘ -``` - -Функция boundingRatio() возвращает наклон прямой между крайней левой и крайней правой точками; в приведённых выше данных этими точками являются `(0,0)` и `(9,13.5)`. - -```sql -SELECT boundingRatio(number, number * 1.5) -FROM numbers(10) -``` - -```response -┌─boundingRatio(number, multiply(number, 1.5))─┐ -│ 1.5 │ -└──────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md deleted file mode 100644 index 819c27bfb73..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Вычисляет значение выражения `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - - log(P(tag = 0)))` для каждой категории.' -sidebar_position: 115 -slug: /sql-reference/aggregate-functions/reference/categoricalinformationvalue -title: 'categoricalInformationValue' -doc_type: 'reference' ---- - -Вычисляет значение выражения `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` для каждой категории. - -```sql -categoricalInformationValue(category1, category2, ..., tag) -``` - -Результат показывает, как дискретный (категориальный) признак со значениями `[category1, category2, ...]` вносит вклад в модель, предсказывающую значение `tag`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md deleted file mode 100644 index 5e7420ef0b8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'Функция `contingency` вычисляет коэффициент контингенции — величину, измеряющую степень связи между двумя столбцами таблицы. Вычисление аналогично функции `cramersV`, но с другим знаменателем под квадратным корнем.' -sidebar_position: 116 -slug: /sql-reference/aggregate-functions/reference/contingency -title: 'contingency' -doc_type: 'reference' ---- - -# contingency {#contingency} - -Функция `contingency` вычисляет [коэффициент контингенции](https://en.wikipedia.org/wiki/Contingency_table#Cram%C3%A9r's_V_and_the_contingency_coefficient_C) — величину, которая измеряет связь между двумя столбцами в таблице. Вычисление аналогично [функции `cramersV`](./cramersv.md), но отличается знаменателем под знаком квадратного корня. - -**Синтаксис** - -```sql -contingency(column1, column2) -``` - -**Аргументы** - -* `column1` и `column2` — столбцы, которые нужно сравнить - -**Возвращаемое значение** - -* значение между 0 и 1. Чем больше результат, тем сильнее связь между двумя столбцами. - -**Тип возвращаемого значения** всегда [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Два столбца, сравниваемые ниже, имеют слабую связь друг с другом. Мы также включили результат функции `cramersV` (для сравнения): - -```sql -SELECT - cramersV(a, b), - contingency(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -Результат: - -```response -┌─────cramersV(a, b)─┬──contingency(a, b)─┐ -│ 0.5798088336225178 │ 0.0817230766271248 │ -└────────────────────┴────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md deleted file mode 100644 index 215025e5b83..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: 'Вычисляет коэффициент корреляции Пирсона.' -sidebar_position: 117 -slug: /sql-reference/aggregate-functions/reference/corr -title: 'corr' -doc_type: 'reference' ---- - - - -# corr {#corr} - -Вычисляет [коэффициент корреляции Пирсона](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient): - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -
-:::note Эта функция использует численно неустойчивый алгоритм. Если вам нужна -[численная устойчивость](https://en.wikipedia.org/wiki/Numerical_stability) в -вычислениях, используйте функцию [`corrStable`](../reference/corrstable.md). Она -работает медленнее, но обеспечивает более точный результат. ::: - -**Синтаксис** - -```sql -corr(x, y) -``` - -**Аргументы** - -- `x` — первая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md). -- `y` — вторая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md). - -**Возвращаемое значение** - -- Коэффициент корреляции Пирсона. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corr(x_value, y_value) -FROM series; -``` - -Результат: - -```response -┌─corr(x_value, y_value)─┐ -│ 0.1730265755453256 │ -└────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md deleted file mode 100644 index 5a73253807f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Вычисляет матрицу корреляции для N переменных.' -sidebar_position: 118 -slug: /sql-reference/aggregate-functions/reference/corrmatrix -title: 'corrMatrix' -doc_type: 'reference' ---- - -# corrMatrix {#corrmatrix} - -Вычисляет матрицу корреляции для N переменных. - -**Синтаксис** - -```sql -corrMatrix(x[, ...]) -``` - -**Аргументы** - -* `x` — переменное число аргументов. [`(U)Int8/16/32/64`](../../data-types/int-uint.md), [`Float*`](../../data-types/float.md). - -**Возвращаемое значение** - -* Матрица корреляции. [`Array`](../../data-types/array.md)([`Array`](../../data-types/array.md)([`Float64`](../../data-types/float.md))). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(corrMatrix(a, b, c, d))) AS corrMatrix -FROM test; -``` - -Результат: - -```response - ┌─corrMatrix─────────────┐ -1. │ [1,-0.096,0.243,0.746] │ -2. │ [-0.096,1,0.173,0.106] │ -3. │ [0.243,0.173,1,0.258] │ -4. │ [0.746,0.106,0.258,1] │ - └────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md deleted file mode 100644 index 047ee9a827b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: 'Вычисляет коэффициент корреляции Пирсона, используя численно устойчивый алгоритм.' -sidebar_position: 119 -slug: /sql-reference/aggregate-functions/reference/corrstable -title: 'corrStable' -doc_type: 'reference' ---- - - - -# corrStable {#corrstable} - -Вычисляет [коэффициент корреляции Пирсона](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient): - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -Аналогична функции [`corr`](../reference/corr.md), но использует численно устойчивый алгоритм. В результате `corrStable` работает медленнее, чем `corr`, но обеспечивает более точный результат. - -**Синтаксис** - -```sql -corrStable(x, y) -``` - -**Аргументы** - -- `x` — первая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). -- `y` — вторая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Коэффициент корреляции Пирсона. [Float64](../../data-types/float.md). - -**\*Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corrStable(x_value, y_value) -FROM series; -``` - -Результат: - -```response -┌─corrStable(x_value, y_value)─┐ -│ 0.17302657554532558 │ -└──────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md deleted file mode 100644 index 3c81aa7b7a8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Подсчитывает число строк или значений, отличных от NULL.' -sidebar_position: 120 -slug: /sql-reference/aggregate-functions/reference/count -title: 'count' -doc_type: 'reference' ---- - -# count {#count} - -Подсчитывает количество строк или значений, отличных от NULL. - -ClickHouse поддерживает следующие варианты синтаксиса для `count`: - -* `count(expr)` или `COUNT(DISTINCT expr)`. -* `count()` или `COUNT(*)`. Синтаксис `count()` специфичен для ClickHouse. - -**Аргументы** - -Функция может принимать: - -* Ноль параметров. -* Одно [выражение](/sql-reference/syntax#expressions). - -**Возвращаемое значение** - -* Если функция вызывается без параметров, она подсчитывает количество строк. -* Если передано [выражение](/sql-reference/syntax#expressions), то функция считает, сколько раз это выражение вернуло не NULL. Если выражение возвращает значение типа [Nullable](../../../sql-reference/data-types/nullable.md), то результат `count` остаётся типом, не `Nullable`. Функция возвращает 0, если выражение вернуло `NULL` для всех строк. - -В обоих случаях тип возвращаемого значения — [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Подробности** - -ClickHouse поддерживает синтаксис `COUNT(DISTINCT ...)`. Поведение этой конструкции зависит от настройки [count_distinct_implementation](../../../operations/settings/settings.md#count_distinct_implementation). Она определяет, какая из функций семейства [uniq*](/sql-reference/aggregate-functions/reference/uniq) используется для выполнения операции. По умолчанию используется функция [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact). - -Запрос `SELECT count() FROM table` по умолчанию оптимизируется с использованием метаданных из MergeTree. Если вам нужно использовать построчную безопасность (row-level security), отключите эту оптимизацию с помощью настройки [optimize_trivial_count_query](/operations/settings/settings#optimize_trivial_count_query). - -Однако запрос `SELECT count(nullable_column) FROM table` может быть оптимизирован путём включения настройки [optimize_functions_to_subcolumns](/operations/settings/settings#optimize_functions_to_subcolumns). При `optimize_functions_to_subcolumns = 1` функция читает только подстолбец [null](../../../sql-reference/data-types/nullable.md#finding-null) вместо чтения и обработки всех данных столбца. Запрос `SELECT count(n) FROM table` преобразуется в `SELECT sum(NOT n.null) FROM table`. - -**Повышение производительности COUNT(DISTINCT expr)** - -Если ваш запрос `COUNT(DISTINCT expr)` выполняется медленно, рассмотрите возможность добавления предложения [`GROUP BY`](/sql-reference/statements/select/group-by), так как это улучшает распараллеливание. Вы также можете использовать [проекцию](../../../sql-reference/statements/alter/projection.md) для создания индекса по целевому столбцу, используемому с `COUNT(DISTINCT target_col)`. - -**Примеры** - -Пример 1: - -```sql -SELECT count() FROM t -``` - -```text -┌─count()─┐ -│ 5 │ -└─────────┘ -``` - -Пример 2: - -```sql -SELECT name, value FROM system.settings WHERE name = 'count_distinct_implementation' -``` - -```text -┌─name──────────────────────────┬─value─────┐ -│ count_distinct_implementation │ uniqExact │ -└───────────────────────────────┴───────────┘ -``` - -```sql -SELECT count(DISTINCT num) FROM t -``` - -```text -┌─uniqExact(num)─┐ -│ 3 │ -└────────────────┘ -``` - -Этот пример показывает, что `count(DISTINCT num)` вычисляется функцией `uniqExact` в соответствии со значением настройки `count_distinct_implementation`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md deleted file mode 100644 index f208a84d948..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: 'Вычисляет ковариацию генеральной совокупности' -sidebar_position: 121 -slug: /sql-reference/aggregate-functions/reference/covarpop -title: 'covarPop' -doc_type: 'reference' ---- - - - -# covarPop {#covarpop} - -Вычисляет ковариацию генеральной совокупности: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -:::note -Функция использует численно неустойчивый алгоритм. Если в вычислениях требуется [численная устойчивость](https://en.wikipedia.org/wiki/Numerical_stability), используйте функцию [`covarPopStable`](../reference/covarpopstable.md). Она работает медленнее, но обеспечивает меньшую вычислительную погрешность. -::: - -**Синтаксис** - -```sql -covarPop(x, y) -``` - -**Аргументы** - -- `x` — первая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). -- `y` — вторая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Ковариация генеральной совокупности между `x` и `y`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT covarPop(x_value, y_value) -FROM series; -``` - -Результат: - -```reference -┌─covarPop(x_value, y_value)─┐ -│ 6.485648 │ -└────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md deleted file mode 100644 index b3ca21100c5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Возвращает матрицу ковариации генеральной совокупности по N переменным.' -sidebar_position: 122 -slug: /sql-reference/aggregate-functions/reference/covarpopmatrix -title: 'covarPopMatrix' -doc_type: 'reference' ---- - -# covarPopMatrix {#covarpopmatrix} - -Возвращает матрицу ковариации генеральной совокупности для N переменных. - -**Синтаксис** - -```sql -covarPopMatrix(x[, ...]) -``` - -**Аргументы** - -* `x` — произвольное число аргументов. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Матрица ковариации генеральной совокупности. [Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md))). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarPopMatrix(a, b, c, d))) AS covarPopMatrix -FROM test; -``` - -Результат: - -```reference - ┌─covarPopMatrix────────────┐ -1. │ [8.25,-1.76,4.08,6.748] │ -2. │ [-1.76,41.07,6.486,2.132] │ -3. │ [4.08,6.486,34.21,4.755] │ -4. │ [6.748,2.132,4.755,9.93] │ - └───────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md deleted file mode 100644 index 9d2627cc8ed..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: 'Вычисляет значение ковариации генеральной совокупности' -sidebar_position: 123 -slug: /sql-reference/aggregate-functions/reference/covarpopstable -title: 'covarPopStable' -doc_type: 'reference' ---- - - - -# covarPopStable {#covarpopstable} - -Вычисляет значение ковариации для генеральной совокупности: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -Функция аналогична [covarPop](../reference/covarpop.md), но использует численно устойчивый алгоритм. В результате `covarPopStable` работает медленнее `covarPop`, но обеспечивает более точный результат. - -**Синтаксис** - -```sql -covarPop(x, y) -``` - -**Аргументы** - -- `x` — первая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). -- `y` — вторая переменная. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Ковариация генеральной совокупности между `x` и `y`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarPopStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -Результат: - -```reference -┌─covarPopStable(x_value, y_value)─┐ -│ 6.485648 │ -└──────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md deleted file mode 100644 index 2d9bc2ccdab..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Вычисляет значение `Σ((x - x̅)(y - y̅)) / (n - 1)`' -sidebar_position: 124 -slug: /sql-reference/aggregate-functions/reference/covarsamp -title: 'covarSamp' -doc_type: 'reference' ---- - -# covarSamp {#covarsamp} - -Вычисляет значение `Σ((x - x̅)(y - y̅)) / (n - 1)`. - -:::note -Эта функция использует численно неустойчивый алгоритм. Если вам важна [численная устойчивость](https://en.wikipedia.org/wiki/Numerical_stability) вычислений, используйте функцию [`covarSampStable`](../reference/covarsamp.md). Она работает медленнее, но даёт меньшую вычислительную погрешность. -::: - -**Синтаксис** - -```sql -covarSamp(x, y) -``` - -**Аргументы** - -* `x` — первая переменная. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). -* `y` — вторая переменная. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Выборочная ковариация между `x` и `y`. Если `n <= 1`, возвращается `nan`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -Результат: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ 7.206275555555556 │ -└─────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); - -``` - -Результат: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ nan │ -└─────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md deleted file mode 100644 index 80a98f14fd7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Возвращает выборочную ковариационную матрицу для N переменных.' -sidebar_position: 125 -slug: /sql-reference/aggregate-functions/reference/covarsampmatrix -title: 'covarSampMatrix' -doc_type: 'reference' ---- - -# covarSampMatrix {#covarsampmatrix} - -Возвращает выборочную ковариационную матрицу для N переменных. - -**Синтаксис** - -```sql -covarSampMatrix(x[, ...]) -``` - -**Аргументы** - -* `x` — произвольное количество аргументов. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Матрица выборочной ковариации. [Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md))). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarSampMatrix(a, b, c, d))) AS covarSampMatrix -FROM test; -``` - -Результат: - -```reference - ┌─covarSampMatrix─────────────┐ -1. │ [9.167,-1.956,4.534,7.498] │ -2. │ [-1.956,45.634,7.206,2.369] │ -3. │ [4.534,7.206,38.011,5.283] │ -4. │ [7.498,2.369,5.283,11.034] │ - └─────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md deleted file mode 100644 index 5094653751b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: 'Аналогична covarSamp, однако работает медленнее, обеспечивая при этом меньшую вычислительную погрешность.' -sidebar_position: 126 -slug: /sql-reference/aggregate-functions/reference/covarsampstable -title: 'covarSampStable' -doc_type: 'reference' ---- - -# covarSampStable {#covarsampstable} - -Вычисляет значение `Σ((x - x̅)(y - y̅)) / (n - 1)`. Аналогична функции [covarSamp](../reference/covarsamp.md), но работает медленнее, обеспечивая при этом меньшую вычислительную погрешность. - -**Синтаксис** - -```sql -covarSampStable(x, y) -``` - -**Аргументы** - -* `x` — первая переменная. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). -* `y` — вторая переменная. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Выборочная ковариация между `x` и `y`. Для `n <= 1` возвращается значение `inf`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -Результат: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ 7.206275555555556 │ -└───────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); -``` - -Результат: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ inf │ -└───────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md deleted file mode 100644 index f9374a7d89e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: 'Результат функции `cramersV` лежит в диапазоне от 0 (соответствует - отсутствию зависимости между переменными) до 1 и может достигать 1 только тогда, - когда каждое значение полностью определяется другим. Эту величину можно рассматривать - как меру связи между двумя переменными в процентах от их максимально возможной вариации.' -sidebar_position: 127 -slug: /sql-reference/aggregate-functions/reference/cramersv -title: 'cramersV' -doc_type: 'reference' ---- - -# cramersV {#cramersv} - -[Мера V Крамера](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V) (иногда также называемая фи Крамера) — это мера взаимосвязи между двумя столбцами в таблице. Результат функции `cramersV` лежит в диапазоне от 0 (что соответствует отсутствию связи между переменными) до 1 и может достигать 1 только тогда, когда каждое значение полностью определяется другим. Её можно рассматривать как степень взаимосвязи между двумя переменными, выраженную в процентах от их максимально возможной вариации. - -:::note -См. скорректированный с учётом смещения вариант меры V Крамера: [cramersVBiasCorrected](./cramersvbiascorrected.md) -::: - -**Синтаксис** - -```sql -cramersV(column1, column2) -``` - -**Параметры** - -* `column1`: первый столбец для сравнения. -* `column2`: второй столбец для сравнения. - -**Возвращаемое значение** - -* значение от 0 (соответствует отсутствию зависимости между значениями столбцов) до 1 (полная зависимость). - -Тип: всегда [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Следующие два столбца, сравниваемые ниже, не зависят друг от друга, поэтому результат `cramersV` равен 0: - -Запрос: - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 3 AS a, - number % 5 AS b - FROM - numbers(150) - ); -``` - -Результат: - -```response -┌─cramersV(a, b)─┐ -│ 0 │ -└────────────────┘ -``` - -Следующие два столбца тесно взаимосвязаны, поэтому значение `cramersV` высокое: - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 10 AS a, - if(number % 12 = 0, (number + 1) % 5, number % 5) AS b - FROM - numbers(150) - ); -``` - -Результат: - -```response -┌─────cramersV(a, b)─┐ -│ 0.9066801892162646 │ -└────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md deleted file mode 100644 index e65c36d495c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: 'Вычисляет V Крамера с поправкой на смещение.' -sidebar_position: 128 -slug: /sql-reference/aggregate-functions/reference/cramersvbiascorrected -title: 'cramersVBiasCorrected' -doc_type: 'reference' ---- - -# cramersVBiasCorrected {#cramersvbiascorrected} - -V Крамера — это мера ассоциации между двумя столбцами в таблице. Результат функции [`cramersV`](./cramersv.md) лежит в диапазоне от 0 (соответствует отсутствию связи между переменными) до 1 и достигает 1 только тогда, когда каждое значение полностью определяется другим. Оценка может иметь значительное смещение, поэтому в этой версии V Крамера используется [коррекция смещения](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V#Bias_correction). - -**Синтаксис** - -```sql -cramersVBiasCorrected(column1, column2) -``` - -**Параметры** - -* `column1`: первый сравниваемый столбец. -* `column2`: второй сравниваемый столбец. - -**Возвращаемое значение** - -* значение от 0 (соответствует отсутствию связи между значениями столбцов) до 1 (полная связь). - -Тип: всегда [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Следующие два столбца, сравниваемые ниже, имеют умеренную взаимосвязь друг с другом. Обратите внимание, что результат `cramersVBiasCorrected` меньше результата `cramersV`: - -Запрос: - -```sql -SELECT - cramersV(a, b), - cramersVBiasCorrected(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -Результат: - -```response -┌─────cramersV(a, b)─┬─cramersVBiasCorrected(a, b)─┐ -│ 0.5798088336225178 │ 0.5305112825189074 │ -└────────────────────┴─────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md deleted file mode 100644 index 822776aa449..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Суммирует арифметическую разность между последовательными строками.' -sidebar_position: 129 -slug: /sql-reference/aggregate-functions/reference/deltasum -title: 'deltaSum' -doc_type: 'reference' ---- - - - -# deltaSum {#deltasum} - -Суммирует арифметическую разность между последовательными строками. Если разность отрицательная, она не учитывается. - -:::note -Исходные данные должны быть отсортированы, чтобы эта функция работала корректно. Если вы хотите использовать эту функцию в [материализованном представлении](/sql-reference/statements/create/view#materialized-view), вам, скорее всего, следует использовать метод [deltaSumTimestamp](/sql-reference/aggregate-functions/reference/deltasumtimestamp) вместо неё. -::: - -**Синтаксис** - -```sql -deltaSum(value) -``` - -**Аргументы** - -* `value` — Входное значение, должно быть типа [Integer](../../data-types/int-uint.md) или [Float](../../data-types/float.md). - -**Возвращаемое значение** - -* Полученная арифметическая разность типа `Integer` или `Float`. - -**Примеры** - -Запрос: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3])); -``` - -Результат: - -```text -┌─deltaSum(arrayJoin([1, 2, 3]))─┐ -│ 2 │ -└────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3])); -``` - -Результат: - -```text -┌─deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3]))─┐ -│ 7 │ -└───────────────────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT deltaSum(arrayJoin([2.25, 3, 4.5])); -``` - -Результат: - -```text -┌─deltaSum(arrayJoin([2.25, 3, 4.5]))─┐ -│ 2.25 │ -└─────────────────────────────────────┘ -``` - - -## См. также {#see-also} - -- [runningDifference](/sql-reference/functions/other-functions#runningDifference) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md deleted file mode 100644 index f1a6b17a79b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'Добавляет разность между последовательными строками. Если разность отрицательная, - она игнорируется.' -sidebar_position: 130 -slug: /sql-reference/aggregate-functions/reference/deltasumtimestamp -title: 'deltaSumTimestamp' -doc_type: 'reference' ---- - -Добавляет разность между последовательными строками. Если разность отрицательная, она игнорируется. - -Эта функция в первую очередь предназначена для [материализованных представлений](/sql-reference/statements/create/view#materialized-view), которые хранят данные, упорядоченные по некоторому временному интервалу (time bucket), выровненному по метке времени, например, по интервалу `toStartOfMinute`. Поскольку строки в таком материализованном представлении будут иметь одинаковую метку времени, их невозможно объединить в правильном порядке без сохранения исходного, неокруглённого значения метки времени. Функция `deltaSumTimestamp` отслеживает исходный `timestamp` значений, которые она обработала, поэтому значения (состояния) функции корректно вычисляются во время слияния частей. - -Чтобы вычислить сумму дельт по упорядоченной коллекции, вы можете просто использовать функцию [deltaSum](/sql-reference/aggregate-functions/reference/deltasum). - -**Синтаксис** - -```sql -deltaSumTimestamp(value, timestamp) -``` - -**Аргументы** - -* `value` — Входные значения; должны иметь тип [Integer](../../data-types/int-uint.md) или [Float](../../data-types/float.md), либо тип [Date](../../data-types/date.md) или [DateTime](../../data-types/datetime.md). -* `timestamp` — Параметр упорядочивания значений; должен иметь тип [Integer](../../data-types/int-uint.md) или [Float](../../data-types/float.md), либо тип [Date](../../data-types/date.md) или [DateTime](../../data-types/datetime.md). - -**Возвращаемое значение** - -* Накопленные разности между последовательными значениями, упорядоченными по параметру `timestamp`. - -Тип: [Integer](../../data-types/int-uint.md) или [Float](../../data-types/float.md) или [Date](../../data-types/date.md) или [DateTime](../../data-types/datetime.md). - -**Пример** - -Запрос: - -```sql -SELECT deltaSumTimestamp(value, timestamp) -FROM (SELECT number AS timestamp, [0, 4, 8, 3, 0, 0, 0, 1, 3, 5][number] AS value FROM numbers(1, 10)); -``` - -Результат: - -```text -┌─deltaSumTimestamp(value, timestamp)─┐ -│ 13 │ -└─────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md deleted file mode 100644 index eda9b1ea20e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Вычисляет список уникальных типов данных, хранящихся в столбце Dynamic.' -sidebar_position: 215 -slug: /sql-reference/aggregate-functions/reference/distinctdynamictypes -title: 'distinctDynamicTypes' -doc_type: 'reference' ---- - -# distinctDynamicTypes {#distinctdynamictypes} - -Вычисляет список уникальных типов данных, содержащихся в столбце [Dynamic](../../data-types/dynamic.md). - -**Синтаксис** - -```sql -distinctDynamicTypes(dynamic) -``` - -**Аргументы** - -* `dynamic` — столбец типа [Dynamic](../../data-types/dynamic.md). - -**Возвращаемое значение** - -* Отсортированный список имён типов данных [Array(String)](../../data-types/array.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_dynamic; -CREATE TABLE test_dynamic(d Dynamic) ENGINE = Memory; -INSERT INTO test_dynamic VALUES (42), (NULL), ('Hello'), ([1, 2, 3]), ('2020-01-01'), (map(1, 2)), (43), ([4, 5]), (NULL), ('World'), (map(3, 4)) -``` - -```sql -SELECT distinctDynamicTypes(d) FROM test_dynamic; -``` - -Результат: - -```reference -┌─distinctDynamicTypes(d)──────────────────────────────────────┐ -│ ['Array(Int64)','Date','Int64','Map(UInt8, UInt8)','String'] │ -└──────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md deleted file mode 100644 index cb1fe4223dd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -description: 'Вычисляет список уникальных путей, хранящихся в JSON-столбце.' -sidebar_position: 216 -slug: /sql-reference/aggregate-functions/reference/distinctjsonpaths -title: 'distinctJSONPaths' -doc_type: 'reference' ---- - -# distinctJSONPaths {#distinctjsonpaths} - -Вычисляет список уникальных путей, хранящихся в столбце [JSON](../../data-types/newjson.md). - -**Синтаксис** - -```sql -distinctJSONPaths(json) -``` - -**Аргументы** - -* `json` — столбец [JSON](../../data-types/newjson.md). - -**Возвращаемое значение** - -* Отсортированный список путей типа [Array(String)](../../data-types/array.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "Hello"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -Результат: - -```reference -┌─distinctJSONPaths(json)───┐ -│ ['a','b','c.d.e','c.d.f'] │ -└───────────────────────────┘ -``` - -# distinctJSONPathsAndTypes {#distinctjsonpathsandtypes} - -Вычисляет список различных путей и их типов, хранящихся в столбце [JSON](../../data-types/newjson.md). - -**Синтаксис** - -```sql -distinctJSONPathsAndTypes(json) -``` - -**Аргументы** - -* `json` — столбец [JSON](../../data-types/newjson.md). - -**Возвращаемое значение** - -* Отсортированное отображение путей и типов в виде [Map(String, Array(String))](../../data-types/map.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "Hello"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -Результат: - -```reference -┌─distinctJSONPathsAndTypes(json)───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {'a':['Int64'],'b':['Array(Nullable(Int64))','String'],'c.d.e':['Date'],'c.d.f':['Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))']} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**Примечание** - -Если JSON-описание содержит пути с указанными типами, эти пути всегда будут включены в результат работы функций `distinctJSONPaths` и `distinctJSONPathsAndTypes`, даже если во входных данных не было значений для этих путей. - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON(a UInt32)) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"b" : "Hello"}'), ('{"b" : "World", "c" : [1, 2, 3]}'); -``` - -```sql -SELECT json FROM test_json; -``` - -```text -┌─json──────────────────────────────────┐ -│ {"a":0,"b":"Hello"} │ -│ {"a":0,"b":"World","c":["1","2","3"]} │ -└───────────────────────────────────────┘ -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -```text -┌─distinctJSONPaths(json)─┐ -│ ['a','b','c'] │ -└─────────────────────────┘ -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -```text -┌─distinctJSONPathsAndTypes(json)────────────────────────────────┐ -│ {'a':['UInt32'],'b':['String'],'c':['Array(Nullable(Int64))']} │ -└────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md deleted file mode 100644 index 1e075c9074e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Вычисляет энтропию Шеннона для столбца значений.' -sidebar_position: 131 -slug: /sql-reference/aggregate-functions/reference/entropy -title: 'entropy' -doc_type: 'reference' ---- - -# entropy {#entropy} - -Вычисляет [энтропию Шеннона](https://en.wikipedia.org/wiki/Entropy_\(information_theory\)) для столбца значений. - -**Синтаксис** - -```sql -entropy(val) -``` - -**Аргументы** - -* `val` — столбец значений любого типа. - -**Возвращаемое значение** - -* энтропия Шеннона. - -Тип: [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Запрос: - -```sql -CREATE TABLE entropy (`vals` UInt32,`strings` String) ENGINE = Memory; - -INSERT INTO entropy VALUES (1, 'A'), (1, 'A'), (1,'A'), (1,'A'), (2,'B'), (2,'B'), (2,'C'), (2,'D'); - -SELECT entropy(vals), entropy(strings) FROM entropy; -``` - -Результат: - -```text -┌─entropy(vals)─┬─entropy(strings)─┐ -│ 1 │ 1.75 │ -└───────────────┴──────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md deleted file mode 100644 index 0d85ad4d3df..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: 'Оценивает степень сжатия заданного столбца без выполнения его сжатия.' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/estimateCompressionRatio -title: 'estimateCompressionRatio' -doc_type: 'reference' ---- - -## estimateCompressionRatio {#estimatecompressionration} - -Оценивает коэффициент сжатия заданного столбца, не выполняя его сжатие. - -**Синтаксис** - -```sql -оценитьКоэффициентСжатия(codec, block_size_bytes)(column) -``` - -**Аргументы** - -* `column` - Столбец любого типа - -**Параметры** - -* `codec` - [String](../../../sql-reference/data-types/string.md), содержащая [кодек сжатия](/sql-reference/statements/create/table#column_compression_codec) или несколько кодеков, перечисленных через запятую в одной строке. -* `block_size_bytes` - Размер блока сжатых данных. Аналогично одновременному заданию параметров [`max_compress_block_size`](../../../operations/settings/merge-tree-settings.md#max_compress_block_size) и [`min_compress_block_size`](../../../operations/settings/merge-tree-settings.md#min_compress_block_size). Значение по умолчанию — 1 MiB (1048576 байт). - -Оба параметра являются необязательными. - -**Возвращаемые значения** - -* Возвращает примерный коэффициент сжатия для заданного столбца. - -Тип: [Float64](/sql-reference/data-types/float). - -**Примеры** - -```sql title="Input table" -CREATE TABLE compression_estimate_example -( - `number` UInt64 -) -ENGINE = MergeTree() -ORDER BY number -SETTINGS min_bytes_for_wide_part = 0; - -INSERT INTO compression_estimate_example -SELECT number FROM system.numbers LIMIT 100_000; -``` - -```sql title="Query" -SELECT estimateCompressionRatio(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌───────────estimate─┐ -│ 1.9988506608699999 │ -└────────────────────┘ -``` - -:::note -Результат, показанный выше, будет отличаться в зависимости от кодека сжатия по умолчанию на сервере. См. раздел [Кодеки сжатия столбцов](/sql-reference/statements/create/table#column_compression_codec). -::: - -```sql title="Query" -SELECT estimateCompressionRatio('T64')(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌──────────estimate─┐ -│ 3.762758101688538 │ -└───────────────────┘ -``` - -Функция также может задавать несколько кодеков: - -```sql title="Query" -SELECT estimateCompressionRatio('T64, ZSTD')(number) AS estimate FROM compression_estimate_example; -``` - -```response title="Response" -┌───────────estimate─┐ -│ 143.60078980434392 │ -└────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md deleted file mode 100644 index 850122780d8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md +++ /dev/null @@ -1,203 +0,0 @@ ---- -description: 'Вычисляет экспоненциальное скользящее среднее значений за заданный промежуток времени.' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/exponentialMovingAverage -title: 'exponentialMovingAverage' -doc_type: 'reference' ---- - -## exponentialMovingAverage {#exponentialmovingaverage} - -Вычисляет экспоненциальное скользящее среднее значений за заданный интервал времени. - -**Синтаксис** - -```sql -exponentialMovingAverage(x)(value, timeunit) -``` - -Каждому `value` соответствует определённый `timeunit`. Период полураспада `x` — это временной интервал, при котором экспоненциальные веса уменьшаются вдвое. Функция возвращает взвешенное среднее: чем старше точка по времени, тем меньший вес получает соответствующее значение. - -**Аргументы** - -* `value` — Значение. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `timeunit` — Временная единица. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). Временная единица — это не временная метка (секунды), а индекс временного интервала. Может быть вычислена с помощью [intDiv](/sql-reference/functions/arithmetic-functions#intDiv). - -**Параметры** - -* `x` — Период полураспада. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемые значения** - -* Возвращает [экспоненциально сглаженное скользящее среднее](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average) значений за последние `x` единиц времени в самой поздней точке времени. - -Тип: [Float64](/sql-reference/data-types/float). - -**Примеры** - -Входная таблица: - -```text -┌──температура─┬─временная_метка──┐ -│ 95 │ 1 │ -│ 95 │ 2 │ -│ 95 │ 3 │ -│ 96 │ 4 │ -│ 96 │ 5 │ -│ 96 │ 6 │ -│ 96 │ 7 │ -│ 97 │ 8 │ -│ 97 │ 9 │ -│ 97 │ 10 │ -│ 97 │ 11 │ -│ 98 │ 12 │ -│ 98 │ 13 │ -│ 98 │ 14 │ -│ 98 │ 15 │ -│ 99 │ 16 │ -│ 99 │ 17 │ -│ 99 │ 18 │ -│ 100 │ 19 │ -│ 100 │ 20 │ -└──────────────┴────────────┘ -``` - -Запрос: - -```sql -SELECT exponentialMovingAverage(5)(temperature, timestamp); -``` - -Результат: - -```text -┌──exponentialMovingAverage(5)(temperature, timestamp)──┐ -│ 92.25779635374204 │ -└───────────────────────────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 1, 50) AS bar -FROM -( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialMovingAverage(10)(value, time) OVER (Rows BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -) -``` - -Результат: - -```text -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────────────────────────────────────┐ -│ 1 │ 0 │ 0.067 │ ███▎ │ -│ 0 │ 1 │ 0.062 │ ███ │ -│ 0 │ 2 │ 0.058 │ ██▊ │ -│ 0 │ 3 │ 0.054 │ ██▋ │ -│ 0 │ 4 │ 0.051 │ ██▌ │ -│ 0 │ 5 │ 0.047 │ ██▎ │ -│ 0 │ 6 │ 0.044 │ ██▏ │ -│ 0 │ 7 │ 0.041 │ ██ │ -│ 0 │ 8 │ 0.038 │ █▊ │ -│ 0 │ 9 │ 0.036 │ █▋ │ -│ 0 │ 10 │ 0.033 │ █▋ │ -│ 0 │ 11 │ 0.031 │ █▌ │ -│ 0 │ 12 │ 0.029 │ █▍ │ -│ 0 │ 13 │ 0.027 │ █▎ │ -│ 0 │ 14 │ 0.025 │ █▎ │ -│ 0 │ 15 │ 0.024 │ █▏ │ -│ 0 │ 16 │ 0.022 │ █ │ -│ 0 │ 17 │ 0.021 │ █ │ -│ 0 │ 18 │ 0.019 │ ▊ │ -│ 0 │ 19 │ 0.018 │ ▊ │ -│ 0 │ 20 │ 0.017 │ ▋ │ -│ 0 │ 21 │ 0.016 │ ▋ │ -│ 0 │ 22 │ 0.015 │ ▋ │ -│ 0 │ 23 │ 0.014 │ ▋ │ -│ 0 │ 24 │ 0.013 │ ▋ │ -│ 1 │ 25 │ 0.079 │ ███▊ │ -│ 1 │ 26 │ 0.14 │ ███████ │ -│ 1 │ 27 │ 0.198 │ █████████▊ │ -│ 1 │ 28 │ 0.252 │ ████████████▌ │ -│ 1 │ 29 │ 0.302 │ ███████████████ │ -│ 1 │ 30 │ 0.349 │ █████████████████▍ │ -│ 1 │ 31 │ 0.392 │ ███████████████████▌ │ -│ 1 │ 32 │ 0.433 │ █████████████████████▋ │ -│ 1 │ 33 │ 0.471 │ ███████████████████████▌ │ -│ 1 │ 34 │ 0.506 │ █████████████████████████▎ │ -│ 1 │ 35 │ 0.539 │ ██████████████████████████▊ │ -│ 1 │ 36 │ 0.57 │ ████████████████████████████▌ │ -│ 1 │ 37 │ 0.599 │ █████████████████████████████▊ │ -│ 1 │ 38 │ 0.626 │ ███████████████████████████████▎ │ -│ 1 │ 39 │ 0.651 │ ████████████████████████████████▌ │ -│ 1 │ 40 │ 0.674 │ █████████████████████████████████▋ │ -│ 1 │ 41 │ 0.696 │ ██████████████████████████████████▋ │ -│ 1 │ 42 │ 0.716 │ ███████████████████████████████████▋ │ -│ 1 │ 43 │ 0.735 │ ████████████████████████████████████▋ │ -│ 1 │ 44 │ 0.753 │ █████████████████████████████████████▋ │ -│ 1 │ 45 │ 0.77 │ ██████████████████████████████████████▍ │ -│ 1 │ 46 │ 0.785 │ ███████████████████████████████████████▎ │ -│ 1 │ 47 │ 0.8 │ ███████████████████████████████████████▊ │ -│ 1 │ 48 │ 0.813 │ ████████████████████████████████████████▋ │ -│ 1 │ 49 │ 0.825 │ █████████████████████████████████████████▎ │ -└───────┴──────┴──────────────────────┴────────────────────────────────────────────┘ -``` - -```sql -CREATE TABLE data -ENGINE = Memory AS -SELECT - 10 AS value, - toDateTime('2020-01-01') + (3600 * number) AS time -FROM numbers_mt(10); --- Вычисление временной единицы с помощью intDiv -SELECT - value, - time, - exponentialMovingAverage(1)(value, intDiv(toUInt32(time), 3600)) OVER (ORDER BY time ASC) AS res, - intDiv(toUInt32(time), 3600) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ --- Вычисление временной единицы с помощью toRelativeHourNum -SELECT - value, - time, - exponentialMovingAverage(1)(value, toRelativeHourNum(time)) OVER (ORDER BY time ASC) AS res, - toRelativeHourNum(time) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md deleted file mode 100644 index 0bba4113516..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: 'Возвращает экспоненциально сглаженное взвешенное скользящее среднее значений временного ряда в момент времени `t`.' -sidebar_position: 133 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedAvg -title: 'exponentialTimeDecayedAvg' -doc_type: 'reference' ---- - -## exponentialTimeDecayedAvg {#exponentialtimedecayedavg} - -Возвращает экспоненциально сглаженное взвешенное скользящее среднее значений временного ряда в момент времени `t`. - -**Синтаксис** - -```sql -exponentialTimeDecayedAvg(x)(v, t) -``` - -**Аргументы** - -* `v` — значение. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `t` — время. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md), [DateTime](../../data-types/datetime.md), [DateTime64](../../data-types/datetime64.md). - -**Параметры** - -* `x` — период полураспада. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемые значения** - -* Возвращает экспоненциально сглаженное взвешенное скользящее среднее в момент времени с индексом `t`. [Float64](../../data-types/float.md). - -**Примеры** - -Запрос: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedAvg(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -Ответ: - -```sql -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ -1. │ 1 │ 0 │ 1 │ ██████████ │ -2. │ 0 │ 1 │ 0.475 │ ████▊ │ -3. │ 0 │ 2 │ 0.301 │ ███ │ -4. │ 0 │ 3 │ 0.214 │ ██▏ │ -5. │ 0 │ 4 │ 0.162 │ █▌ │ -6. │ 0 │ 5 │ 0.128 │ █▎ │ -7. │ 0 │ 6 │ 0.104 │ █ │ -8. │ 0 │ 7 │ 0.086 │ ▊ │ -9. │ 0 │ 8 │ 0.072 │ ▋ │ -0. │ 0 │ 9 │ 0.061 │ ▌ │ -1. │ 0 │ 10 │ 0.052 │ ▌ │ -2. │ 0 │ 11 │ 0.045 │ ▍ │ -3. │ 0 │ 12 │ 0.039 │ ▍ │ -4. │ 0 │ 13 │ 0.034 │ ▎ │ -5. │ 0 │ 14 │ 0.03 │ ▎ │ -6. │ 0 │ 15 │ 0.027 │ ▎ │ -7. │ 0 │ 16 │ 0.024 │ ▏ │ -8. │ 0 │ 17 │ 0.021 │ ▏ │ -9. │ 0 │ 18 │ 0.018 │ ▏ │ -0. │ 0 │ 19 │ 0.016 │ ▏ │ -1. │ 0 │ 20 │ 0.015 │ ▏ │ -2. │ 0 │ 21 │ 0.013 │ ▏ │ -3. │ 0 │ 22 │ 0.012 │ │ -4. │ 0 │ 23 │ 0.01 │ │ -5. │ 0 │ 24 │ 0.009 │ │ -6. │ 1 │ 25 │ 0.111 │ █ │ -7. │ 1 │ 26 │ 0.202 │ ██ │ -8. │ 1 │ 27 │ 0.283 │ ██▊ │ -9. │ 1 │ 28 │ 0.355 │ ███▌ │ -0. │ 1 │ 29 │ 0.42 │ ████▏ │ -1. │ 1 │ 30 │ 0.477 │ ████▊ │ -2. │ 1 │ 31 │ 0.529 │ █████▎ │ -3. │ 1 │ 32 │ 0.576 │ █████▊ │ -4. │ 1 │ 33 │ 0.618 │ ██████▏ │ -5. │ 1 │ 34 │ 0.655 │ ██████▌ │ -6. │ 1 │ 35 │ 0.689 │ ██████▉ │ -7. │ 1 │ 36 │ 0.719 │ ███████▏ │ -8. │ 1 │ 37 │ 0.747 │ ███████▍ │ -9. │ 1 │ 38 │ 0.771 │ ███████▋ │ -0. │ 1 │ 39 │ 0.793 │ ███████▉ │ -1. │ 1 │ 40 │ 0.813 │ ████████▏ │ -2. │ 1 │ 41 │ 0.831 │ ████████▎ │ -3. │ 1 │ 42 │ 0.848 │ ████████▍ │ -4. │ 1 │ 43 │ 0.862 │ ████████▌ │ -5. │ 1 │ 44 │ 0.876 │ ████████▊ │ -6. │ 1 │ 45 │ 0.888 │ ████████▉ │ -7. │ 1 │ 46 │ 0.898 │ ████████▉ │ -8. │ 1 │ 47 │ 0.908 │ █████████ │ -9. │ 1 │ 48 │ 0.917 │ █████████▏ │ -0. │ 1 │ 49 │ 0.925 │ █████████▏ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md deleted file mode 100644 index 893e773bcae..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: 'Возвращает значение накопленного экспоненциального затухания для временного ряда в момент времени с индексом `t`.' -sidebar_position: 134 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedCount -title: 'exponentialTimeDecayedCount' -doc_type: 'reference' ---- - -## exponentialTimeDecayedCount {#exponentialtimedecayedcount} - -Возвращает накопленный эффект экспоненциального затухания для временного ряда в момент времени с индексом `t`. - -**Синтаксис** - -```sql -exponentialTimeDecayedCount(x)(t) -``` - -**Аргументы** - -* `t` — Время. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md), [DateTime](../../data-types/datetime.md), [DateTime64](../../data-types/datetime64.md). - -**Параметры** - -* `x` — Период полураспада. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемые значения** - -* Возвращает суммарное экспоненциальное затухание в заданный момент времени. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 20, 50) AS bar -FROM -( - SELECT - (number % 5) = 0 AS value, - number AS time, - exponentialTimeDecayedCount(10)(time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -); -``` - -Результат: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ ██▌ │ - 2. │ 0 │ 1 │ 1.905 │ ████▊ │ - 3. │ 0 │ 2 │ 2.724 │ ██████▊ │ - 4. │ 0 │ 3 │ 3.464 │ ████████▋ │ - 5. │ 0 │ 4 │ 4.135 │ ██████████▎ │ - 6. │ 1 │ 5 │ 4.741 │ ███████████▊ │ - 7. │ 0 │ 6 │ 5.29 │ █████████████▏ │ - 8. │ 0 │ 7 │ 5.787 │ ██████████████▍ │ - 9. │ 0 │ 8 │ 6.236 │ ███████████████▌ │ -10. │ 0 │ 9 │ 6.643 │ ████████████████▌ │ -11. │ 1 │ 10 │ 7.01 │ █████████████████▌ │ -12. │ 0 │ 11 │ 7.343 │ ██████████████████▎ │ -13. │ 0 │ 12 │ 7.644 │ ███████████████████ │ -14. │ 0 │ 13 │ 7.917 │ ███████████████████▊ │ -15. │ 0 │ 14 │ 8.164 │ ████████████████████▍ │ -16. │ 1 │ 15 │ 8.387 │ ████████████████████▉ │ -17. │ 0 │ 16 │ 8.589 │ █████████████████████▍ │ -18. │ 0 │ 17 │ 8.771 │ █████████████████████▉ │ -19. │ 0 │ 18 │ 8.937 │ ██████████████████████▎ │ -20. │ 0 │ 19 │ 9.086 │ ██████████████████████▋ │ -21. │ 1 │ 20 │ 9.222 │ ███████████████████████ │ -22. │ 0 │ 21 │ 9.344 │ ███████████████████████▎ │ -23. │ 0 │ 22 │ 9.455 │ ███████████████████████▋ │ -24. │ 0 │ 23 │ 9.555 │ ███████████████████████▉ │ -25. │ 0 │ 24 │ 9.646 │ ████████████████████████ │ -26. │ 1 │ 25 │ 9.728 │ ████████████████████████▎ │ -27. │ 0 │ 26 │ 9.802 │ ████████████████████████▌ │ -28. │ 0 │ 27 │ 9.869 │ ████████████████████████▋ │ -29. │ 0 │ 28 │ 9.93 │ ████████████████████████▊ │ -30. │ 0 │ 29 │ 9.985 │ ████████████████████████▉ │ -31. │ 1 │ 30 │ 10.035 │ █████████████████████████ │ -32. │ 0 │ 31 │ 10.08 │ █████████████████████████▏ │ -33. │ 0 │ 32 │ 10.121 │ █████████████████████████▎ │ -34. │ 0 │ 33 │ 10.158 │ █████████████████████████▍ │ -35. │ 0 │ 34 │ 10.191 │ █████████████████████████▍ │ -36. │ 1 │ 35 │ 10.221 │ █████████████████████████▌ │ -37. │ 0 │ 36 │ 10.249 │ █████████████████████████▌ │ -38. │ 0 │ 37 │ 10.273 │ █████████████████████████▋ │ -39. │ 0 │ 38 │ 10.296 │ █████████████████████████▋ │ -40. │ 0 │ 39 │ 10.316 │ █████████████████████████▊ │ -41. │ 1 │ 40 │ 10.334 │ █████████████████████████▊ │ -42. │ 0 │ 41 │ 10.351 │ █████████████████████████▉ │ -43. │ 0 │ 42 │ 10.366 │ █████████████████████████▉ │ -44. │ 0 │ 43 │ 10.379 │ █████████████████████████▉ │ -45. │ 0 │ 44 │ 10.392 │ █████████████████████████▉ │ -46. │ 1 │ 45 │ 10.403 │ ██████████████████████████ │ -47. │ 0 │ 46 │ 10.413 │ ██████████████████████████ │ -48. │ 0 │ 47 │ 10.422 │ ██████████████████████████ │ -49. │ 0 │ 48 │ 10.43 │ ██████████████████████████ │ -50. │ 0 │ 49 │ 10.438 │ ██████████████████████████ │ - └───────┴──────┴──────────────────────┴────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md deleted file mode 100644 index ef3dc6cc4bb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -description: 'Возвращает максимум между вычисленным экспоненциально сглаженным скользящим средним - в момент времени с индексом `t` и значением в момент времени `t-1`.' -sidebar_position: 135 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedMax -title: 'exponentialTimeDecayedMax' -doc_type: 'reference' ---- - -## exponentialTimeDecayedMax {#exponentialtimedecayedmax} - -Возвращает максимальное значение между экспоненциально сглаженным скользящим средним, вычисленным в момент времени `t`, и его значением в момент `t-1`. - -**Синтаксис** - -```sql -exponentialTimeDecayedMax(x)(value, timeunit) -``` - -**Аргументы** - -* `value` — Значение. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `timeunit` — Единица измерения времени. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md), [DateTime](../../data-types/datetime.md), [DateTime64](../../data-types/datetime64.md). - -**Параметры** - -* `x` — Период полураспада. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемые значения** - -* Возвращает максимальное значение экспоненциально сглаженного взвешенного скользящего среднего в моменты времени `t` и `t-1`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedMax(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -Результат: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ - 1. │ 1 │ 0 │ 1 │ ██████████ │ - 2. │ 0 │ 1 │ 0.905 │ █████████ │ - 3. │ 0 │ 2 │ 0.819 │ ████████▏ │ - 4. │ 0 │ 3 │ 0.741 │ ███████▍ │ - 5. │ 0 │ 4 │ 0.67 │ ██████▋ │ - 6. │ 0 │ 5 │ 0.607 │ ██████ │ - 7. │ 0 │ 6 │ 0.549 │ █████▍ │ - 8. │ 0 │ 7 │ 0.497 │ ████▉ │ - 9. │ 0 │ 8 │ 0.449 │ ████▍ │ -10. │ 0 │ 9 │ 0.407 │ ████ │ -11. │ 0 │ 10 │ 0.368 │ ███▋ │ -12. │ 0 │ 11 │ 0.333 │ ███▎ │ -13. │ 0 │ 12 │ 0.301 │ ███ │ -14. │ 0 │ 13 │ 0.273 │ ██▋ │ -15. │ 0 │ 14 │ 0.247 │ ██▍ │ -16. │ 0 │ 15 │ 0.223 │ ██▏ │ -17. │ 0 │ 16 │ 0.202 │ ██ │ -18. │ 0 │ 17 │ 0.183 │ █▊ │ -19. │ 0 │ 18 │ 0.165 │ █▋ │ -20. │ 0 │ 19 │ 0.15 │ █▍ │ -21. │ 0 │ 20 │ 0.135 │ █▎ │ -22. │ 0 │ 21 │ 0.122 │ █▏ │ -23. │ 0 │ 22 │ 0.111 │ █ │ -24. │ 0 │ 23 │ 0.1 │ █ │ -25. │ 0 │ 24 │ 0.091 │ ▉ │ -26. │ 1 │ 25 │ 1 │ ██████████ │ -27. │ 1 │ 26 │ 1 │ ██████████ │ -28. │ 1 │ 27 │ 1 │ ██████████ │ -29. │ 1 │ 28 │ 1 │ ██████████ │ -30. │ 1 │ 29 │ 1 │ ██████████ │ -31. │ 1 │ 30 │ 1 │ ██████████ │ -32. │ 1 │ 31 │ 1 │ ██████████ │ -33. │ 1 │ 32 │ 1 │ ██████████ │ -34. │ 1 │ 33 │ 1 │ ██████████ │ -35. │ 1 │ 34 │ 1 │ ██████████ │ -36. │ 1 │ 35 │ 1 │ ██████████ │ -37. │ 1 │ 36 │ 1 │ ██████████ │ -38. │ 1 │ 37 │ 1 │ ██████████ │ -39. │ 1 │ 38 │ 1 │ ██████████ │ -40. │ 1 │ 39 │ 1 │ ██████████ │ -41. │ 1 │ 40 │ 1 │ ██████████ │ -42. │ 1 │ 41 │ 1 │ ██████████ │ -43. │ 1 │ 42 │ 1 │ ██████████ │ -44. │ 1 │ 43 │ 1 │ ██████████ │ -45. │ 1 │ 44 │ 1 │ ██████████ │ -46. │ 1 │ 45 │ 1 │ ██████████ │ -47. │ 1 │ 46 │ 1 │ ██████████ │ -48. │ 1 │ 47 │ 1 │ ██████████ │ -49. │ 1 │ 48 │ 1 │ ██████████ │ -50. │ 1 │ 49 │ 1 │ ██████████ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md deleted file mode 100644 index 483232b165a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: 'Возвращает сумму значений экспоненциально сглаженного скользящего среднего для временного ряда в момент времени с индексом `t`.' -sidebar_position: 136 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedSum -title: 'exponentialTimeDecayedSum' -doc_type: 'reference' ---- - -## exponentialTimeDecayedSum {#exponentialtimedecayedsum} - -Возвращает сумму значений экспоненциально сглаженного скользящего среднего временного ряда в момент времени с индексом `t`. - -**Синтаксис** - -```sql -exponentialTimeDecayedSum(x)(v, t) -``` - -**Аргументы** - -* `v` — значение. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `t` — время. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md), [DateTime](../../data-types/datetime.md), [DateTime64](../../data-types/datetime64.md). - -**Параметры** - -* `x` — разность во времени, при которой вес значения убывает до 1/e. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемые значения** - -* Возвращает сумму значений экспоненциально сглаженного скользящего среднего в заданной точке во времени. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -SELECT - значение, - время, - round(эксп_сглаживание, 3), - диаграмма(эксп_сглаживание, 0, 10, 50) AS диаграмма -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS значение, - number AS время, - exponentialTimeDecayedSum(10)(значение, время) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS эксп_сглаживание - FROM numbers(50) - ); -``` - -Результат: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar───────────────────────────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ █████ │ - 2. │ 0 │ 1 │ 0.905 │ ████▌ │ - 3. │ 0 │ 2 │ 0.819 │ ████ │ - 4. │ 0 │ 3 │ 0.741 │ ███▋ │ - 5. │ 0 │ 4 │ 0.67 │ ███▎ │ - 6. │ 0 │ 5 │ 0.607 │ ███ │ - 7. │ 0 │ 6 │ 0.549 │ ██▋ │ - 8. │ 0 │ 7 │ 0.497 │ ██▍ │ - 9. │ 0 │ 8 │ 0.449 │ ██▏ │ -10. │ 0 │ 9 │ 0.407 │ ██ │ -11. │ 0 │ 10 │ 0.368 │ █▊ │ -12. │ 0 │ 11 │ 0.333 │ █▋ │ -13. │ 0 │ 12 │ 0.301 │ █▌ │ -14. │ 0 │ 13 │ 0.273 │ █▎ │ -15. │ 0 │ 14 │ 0.247 │ █▏ │ -16. │ 0 │ 15 │ 0.223 │ █ │ -17. │ 0 │ 16 │ 0.202 │ █ │ -18. │ 0 │ 17 │ 0.183 │ ▉ │ -19. │ 0 │ 18 │ 0.165 │ ▊ │ -20. │ 0 │ 19 │ 0.15 │ ▋ │ -21. │ 0 │ 20 │ 0.135 │ ▋ │ -22. │ 0 │ 21 │ 0.122 │ ▌ │ -23. │ 0 │ 22 │ 0.111 │ ▌ │ -24. │ 0 │ 23 │ 0.1 │ ▌ │ -25. │ 0 │ 24 │ 0.091 │ ▍ │ -26. │ 1 │ 25 │ 1.082 │ █████▍ │ -27. │ 1 │ 26 │ 1.979 │ █████████▉ │ -28. │ 1 │ 27 │ 2.791 │ █████████████▉ │ -29. │ 1 │ 28 │ 3.525 │ █████████████████▋ │ -30. │ 1 │ 29 │ 4.19 │ ████████████████████▉ │ -31. │ 1 │ 30 │ 4.791 │ ███████████████████████▉ │ -32. │ 1 │ 31 │ 5.335 │ ██████████████████████████▋ │ -33. │ 1 │ 32 │ 5.827 │ █████████████████████████████▏ │ -34. │ 1 │ 33 │ 6.273 │ ███████████████████████████████▎ │ -35. │ 1 │ 34 │ 6.676 │ █████████████████████████████████▍ │ -36. │ 1 │ 35 │ 7.041 │ ███████████████████████████████████▏ │ -37. │ 1 │ 36 │ 7.371 │ ████████████████████████████████████▊ │ -38. │ 1 │ 37 │ 7.669 │ ██████████████████████████████████████▎ │ -39. │ 1 │ 38 │ 7.939 │ ███████████████████████████████████████▋ │ -40. │ 1 │ 39 │ 8.184 │ ████████████████████████████████████████▉ │ -41. │ 1 │ 40 │ 8.405 │ ██████████████████████████████████████████ │ -42. │ 1 │ 41 │ 8.605 │ ███████████████████████████████████████████ │ -43. │ 1 │ 42 │ 8.786 │ ███████████████████████████████████████████▉ │ -44. │ 1 │ 43 │ 8.95 │ ████████████████████████████████████████████▊ │ -45. │ 1 │ 44 │ 9.098 │ █████████████████████████████████████████████▍ │ -46. │ 1 │ 45 │ 9.233 │ ██████████████████████████████████████████████▏ │ -47. │ 1 │ 46 │ 9.354 │ ██████████████████████████████████████████████▊ │ -48. │ 1 │ 47 │ 9.464 │ ███████████████████████████████████████████████▎ │ -49. │ 1 │ 48 │ 9.563 │ ███████████████████████████████████████████████▊ │ -50. │ 1 │ 49 │ 9.653 │ ████████████████████████████████████████████████▎ │ - └───────┴──────┴──────────────────────┴───────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md deleted file mode 100644 index 95c8ad3b444..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'Это псевдоним для any, но он был введён для совместимости с оконными функциями (Window Functions), где иногда необходимо обрабатывать значения `NULL` (по умолчанию все агрегатные функции ClickHouse игнорируют значения `NULL`).' -sidebar_position: 137 -slug: /sql-reference/aggregate-functions/reference/first_value -title: 'first_value' -doc_type: 'reference' ---- - -# first_value {#first_value} - -Это псевдоним для [`any`](../../../sql-reference/aggregate-functions/reference/any.md), но он был добавлен для совместимости с [оконными функциями](../../window-functions/index.md), где иногда требуется обрабатывать значения `NULL` (по умолчанию все агрегатные функции ClickHouse игнорируют значения `NULL`). - -Поддерживается указание модификатора для учёта значений `NULL` (`RESPECT NULLS`) как в [оконных функциях](../../window-functions/index.md), так и в обычных агрегациях. - -Как и в случае с `any`, без использования оконных функций результат будет случайным, если входной поток не упорядочен, а тип возвращаемого значения совпадает с типом входного (значение `NULL` возвращается только в том случае, если входной тип является `Nullable` или добавлен комбинатор `-OrNull`). - -## Примеры {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null); -``` - -### Пример 1 {#example1} - -По умолчанию значение NULL игнорируется. - -```sql -SELECT first_value(b) FROM test_data; -``` - -```text -┌─any(b)─┐ -│ 3 │ -└────────┘ -``` - -### Пример 2 {#example2} - -Значение NULL пропускается. - -```sql -SELECT first_value(b) ignore nulls FROM test_data -``` - -```text -┌─any(b) IGNORE NULLS ─┐ -│ 3 │ -└──────────────────────┘ -``` - -### Пример 3 {#example3} - -Значение NULL допускается. - -```sql -SELECT first_value(b) respect nulls FROM test_data -``` - -```text -┌─any(b) RESPECT NULLS ─┐ -│ ᴺᵁᴸᴸ │ -└───────────────────────┘ -``` - -### Пример 4 {#example4} - -Стабильный результат, полученный с помощью подзапроса с `ORDER BY`. - -```sql -SELECT - first_value_respect_nulls(b), - first_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─any_respect_nulls(b)─┬─any(b)─┐ -│ ᴺᵁᴸᴸ │ 3 │ -└──────────────────────┴────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md deleted file mode 100644 index e39cc159fbb..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: 'Агрегатная функция, которая строит флеймграф по списку стек-трейсов.' -sidebar_position: 138 -slug: /sql-reference/aggregate-functions/reference/flame_graph -title: 'flameGraph' -doc_type: 'reference' ---- - - - -# flameGraph {#flamegraph} - -Агрегатная функция, которая строит [flamegraph](https://www.brendangregg.com/flamegraphs.html) на основе списка стек-трейсов. Возвращает массив строк, которые могут быть использованы утилитой [flamegraph.pl](https://github.com/brendangregg/FlameGraph) для построения SVG-графика flamegraph. - - - -## Синтаксис {#syntax} - -```sql -flameGraph(traces, [size], [ptr]) -``` - - -## Параметры {#parameters} - -- `traces` — стек-трейс. [Array](../../data-types/array.md)([UInt64](../../data-types/int-uint.md)). -- `size` — размер выделения для профилирования памяти (необязательный параметр, по умолчанию `1`). [UInt64](../../data-types/int-uint.md). -- `ptr` — адрес выделения (необязательный параметр, по умолчанию `0`). [UInt64](../../data-types/int-uint.md). - -:::note -При `ptr != 0` flame graph сопоставляет выделения (size > 0) и освобождения (size < 0) с одинаковыми значениями size и ptr. -Показываются только те выделения, которые не были освобождены. Несопоставленные операции освобождения игнорируются. -::: - - - -## Возвращаемое значение {#returned-value} - -- Массив строк, предназначенный для использования с [утилитой flamegraph.pl](https://github.com/brendangregg/FlameGraph). [Array](../../data-types/array.md)([String](../../data-types/string.md)). - - - -## Примеры {#examples} - -### Построение флеймграфа на основе профилировщика запросов по CPU {#building-a-flamegraph-based-on-a-cpu-query-profiler} - -```sql -SET query_profiler_cpu_time_period_ns=10000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(arrayReverse(trace))) from system.trace_log where trace_type = 'CPU' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl > flame_cpu.svg -``` - -### Построение флеймграфа на основе профилировщика памяти запросов, показывающего все выделения {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-all-allocations} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(trace, size)) from system.trace_log where trace_type = 'MemorySample' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem.svg -``` - -### Построение флеймграфа на основе профилировщика памяти запросов, показывающего выделения памяти, которые не были освобождены в контексте запроса {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-allocations-which-were-not-deallocated-in-query-context} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1, use_uncompressed_cache=1, merge_tree_max_rows_to_use_cache=100000000000, merge_tree_max_bytes_to_use_cache=1000000000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_untracked.svg -``` - -### Построение флеймграфа на основе профилировщика запросов по памяти, показывающего активные выделения памяти в фиксированный момент времени {#build-a-flamegraph-based-on-memory-query-profiler-showing-active-allocations-at-the-fixed-point-of-time} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -* 1 — использование памяти в секунду - -```sql -SELECT event_time, m, formatReadableSize(max(s) AS m) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample') GROUP BY event_time ORDER BY event_time; -``` - -* 2 - Найдите момент времени, когда использование памяти было максимальным - -```sql -SELECT argMax(event_time, s), max(s) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample'); -``` - -* 3 - Зафиксировать активные выделения памяти в определённый момент времени - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time <= 'yyy' ORDER BY event_time)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_pos.svg -``` - -* 4 - Найти освобождения памяти в фиксированный момент времени - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, -size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time > 'yyy' ORDER BY event_time desc)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_neg.svg -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md deleted file mode 100644 index f75e2700afc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'Создаёт массив значений аргумента. Значения могут быть добавлены в массив - в произвольном (неопределённом) порядке.' -sidebar_position: 139 -slug: /sql-reference/aggregate-functions/reference/grouparray -title: 'groupArray' -doc_type: 'reference' ---- - -# groupArray {#grouparray} - -Синтаксис: `groupArray(x)` или `groupArray(max_size)(x)` - -Создаёт массив значений аргумента. -Значения могут добавляться в массив в любом (произвольном) порядке. - -Вторая версия (с параметром `max_size`) ограничивает размер результирующего массива `max_size` элементами. Например, `groupArray(1)(x)` эквивалентно `[any (x)]`. - -В некоторых случаях вы тем не менее можете полагаться на порядок выполнения. Это относится к ситуациям, когда `SELECT` выполняется над подзапросом, использующим `ORDER BY`, и результат подзапроса достаточно мал. - -**Пример** - -```text -SELECT * FROM default.ck; - -┌─id─┬─name─────┐ -│ 1 │ zhangsan │ -│ 1 │ ᴺᵁᴸᴸ │ -│ 1 │ lisi │ -│ 2 │ wangwu │ -└────┴──────────┘ - -``` - -Запрос: - -```sql -SELECT id, groupArray(10)(name) FROM default.ck GROUP BY id; -``` - -Результат: - -```text -┌─id─┬─groupArray(10)(name)─┐ -│ 1 │ ['zhangsan','lisi'] │ -│ 2 │ ['wangwu'] │ -└────┴──────────────────────┘ -``` - -Функция groupArray удаляет значение NULL, как показано выше. - -* Псевдоним: `array_agg`. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md deleted file mode 100644 index 866d7e86547..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Агрегирует массивы в более крупный массив, состоящий из этих массивов.' -keywords: ['groupArrayArray', 'array_concat_agg'] -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/grouparrayarray -title: 'groupArrayArray' -doc_type: 'reference' ---- - -# groupArrayArray {#grouparrayarray} - -Агрегирует массивы в более крупный массив этих массивов. -Использует функцию [`groupArray`](/sql-reference/aggregate-functions/reference/grouparray) совместно с комбинатором [Array](/sql-reference/aggregate-functions/combinators#-array). - -Псевдоним: `array_concat_agg` - -**Пример** - -У нас есть данные, фиксирующие пользовательские сессии просмотра сайта. Каждая сессия записывает последовательность страниц, которые посетил конкретный пользователь. -Мы можем использовать функцию `groupArrayArray`, чтобы проанализировать паттерны посещений страниц для каждого пользователя. - -```sql title="Setup" -CREATE TABLE website_visits ( - user_id UInt32, - session_id UInt32, - page_visits Array(String) -) ENGINE = Memory; - -INSERT INTO website_visits VALUES -(101, 1, ['homepage', 'products', 'checkout']), -(101, 2, ['search', 'product_details', 'contact']), -(102, 1, ['homepage', 'about_us']), -(101, 3, ['blog', 'homepage']), -(102, 2, ['products', 'product_details', 'add_to_cart', 'checkout']); -``` - -```sql title="Query" -SELECT - user_id, - groupArrayArray(page_visits) AS user_session_page_sequences -FROM website_visits -GROUP BY user_id; -``` - -```sql title="Response" - ┌─user_id─┬─user_session_page_sequences───────────────────────────────────────────────────────────────┐ -1. │ 101 │ ['homepage','products','checkout','search','product_details','contact','blog','homepage'] │ -2. │ 102 │ ['homepage','about_us','products','product_details','add_to_cart','checkout'] │ - └─────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md deleted file mode 100644 index 5255b03e4e4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: 'Вставляет значение в массив на указанную позицию.' -sidebar_position: 140 -slug: /sql-reference/aggregate-functions/reference/grouparrayinsertat -title: 'groupArrayInsertAt' -doc_type: 'reference' ---- - -# groupArrayInsertAt {#grouparrayinsertat} - -Вставляет значение в массив в заданную позицию. - -**Синтаксис** - -```sql -groupArrayInsertAt(default_x, size)(x, pos) -``` - -Если в одном запросе в одну и ту же позицию вставляется несколько значений, функция ведёт себя следующим образом: - -* Если запрос выполняется в одном потоке, используется первое из вставленных значений. -* Если запрос выполняется в нескольких потоках, результирующим значением может оказаться любое из вставленных; заранее определить его нельзя. - -**Аргументы** - -* `x` — значение для вставки. [Выражение](/sql-reference/syntax#expressions), результатом которого является один из [поддерживаемых типов данных](../../../sql-reference/data-types/index.md). -* `pos` — позиция, в которую должно быть вставлено указанное значение `x`. Нумерация индексов в массиве начинается с нуля. [UInt32](/sql-reference/data-types/int-uint#integer-ranges). -* `default_x` — значение по умолчанию для подстановки в пустые позиции. Необязательный параметр. [Выражение](/sql-reference/syntax#expressions), результатом которого является тип данных, заданный для параметра `x`. Если `default_x` не определён, используются [значения по умолчанию](/sql-reference/statements/create/table). -* `size` — длина результирующего массива. Необязательный параметр. При использовании этого параметра значение по умолчанию `default_x` должно быть задано. [UInt32](/sql-reference/data-types/int-uint#integer-ranges). - -**Возвращаемое значение** - -* Массив с вставленными значениями. - -Тип: [Array](/sql-reference/data-types/array). - -**Пример** - -Запрос: - -```sql -SELECT groupArrayInsertAt(toString(number), number * 2) FROM numbers(5); -``` - -Результат: - -```text -┌─groupArrayInsertAt(toString(number), multiply(number, 2))─┐ -│ ['0','','1','','2','','3','','4'] │ -└───────────────────────────────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT groupArrayInsertAt('-')(toString(number), number * 2) FROM numbers(5); -``` - -Результат: - -```text -┌─groupArrayInsertAt('-')(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2','-','3','-','4'] │ -└────────────────────────────────────────────────────────────────┘ -``` - -Запрос: - -```sql -SELECT groupArrayInsertAt('-', 5)(toString(number), number * 2) FROM numbers(5); -``` - -Результат: - -```text -┌─groupArrayInsertAt('-', 5)(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2'] │ -└───────────────────────────────────────────────────────────────────┘ -``` - -Многопоточная вставка элементов в одну и ту же позицию. - -Запрос: - -```sql -SELECT groupArrayInsertAt(number, 0) FROM numbers_mt(10) SETTINGS max_block_size = 1; -``` - -В результате этого запроса вы получаете случайное целое число из диапазона `[0, 9]`. Например: - -```text -┌─groupArrayInsertAt(number, 0)─┐ -│ [7] │ -└───────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md deleted file mode 100644 index fd219dacfaf..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: 'Возвращает пересечение переданных массивов (все элементы массивов, - которые присутствуют во всех переданных массивах).' -sidebar_position: 141 -slug: /sql-reference/aggregate-functions/reference/grouparrayintersect -title: 'groupArrayIntersect' -doc_type: 'reference' ---- - -# groupArrayIntersect {#grouparrayintersect} - -Возвращает пересечение заданных массивов (все элементы, которые присутствуют во всех указанных массивах). - -**Синтаксис** - -```sql -groupArrayIntersect(x) -``` - -**Аргументы** - -* `x` — аргумент (имя столбца или выражение). - -**Возвращаемые значения** - -* Массив, содержащий элементы, присутствующие во всех массивах. - -Тип: [Array](../../data-types/array.md). - -**Примеры** - -Рассмотрим таблицу `numbers`: - -```text -┌─a──────────────┐ -│ [1,2,4] │ -│ [1,5,2,8,-1,0] │ -│ [1,5,7,5,8,2] │ -└────────────────┘ -``` - -Запрос, в котором имя столбца передаётся в качестве аргумента: - -```sql -SELECT groupArrayIntersect(a) AS intersection FROM numbers; -``` - -Результат: - -```text -┌─intersection──────┐ -│ [1, 2] │ -└───────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md deleted file mode 100644 index c3150b974a7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'Создаёт массив из последних значений аргумента.' -sidebar_position: 142 -slug: /sql-reference/aggregate-functions/reference/grouparraylast -title: 'groupArrayLast' -doc_type: 'reference' ---- - -# groupArrayLast {#grouparraylast} - -Синтаксис: `groupArrayLast(max_size)(x)` - -Создаёт массив из последних значений аргумента. -Например, `groupArrayLast(1)(x)` эквивалентен `[anyLast (x)]`. - -В некоторых случаях вы всё ещё можете полагаться на порядок выполнения. Это относится к случаям, когда оператор `SELECT` получает данные из подзапроса, использующего `ORDER BY`, если результат подзапроса достаточно мал. - -**Пример** - -Запрос: - -```sql -SELECT groupArrayLast(2)(number+1) numbers FROM numbers(10) -``` - -Результат: - -```text -┌─numbers─┐ -│ [9,10] │ -└─────────┘ -``` - -По сравнению с `groupArray`: - -```sql -SELECT groupArray(2)(number+1) numbers FROM numbers(10) -``` - -```text -┌─numbers─┐ -│ [1,2] │ -└─────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md deleted file mode 100644 index 768078ea6db..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -description: 'Вычисляет скользящее среднее для входных значений.' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingavg -title: 'groupArrayMovingAvg' -doc_type: 'reference' ---- - -# groupArrayMovingAvg {#grouparraymovingavg} - -Вычисляет скользящее среднее для входных значений. - -```sql -groupArrayMovingAvg(numbers_for_summing) -groupArrayMovingAvg(window_size)(numbers_for_summing) -``` - -Функция может принимать размер окна в качестве параметра. Если он не указан, функция использует размер окна, равный числу строк в столбце. - -**Аргументы** - -* `numbers_for_summing` — [выражение](/sql-reference/syntax#expressions), результатом которого является значение числового типа данных. -* `window_size` — размер окна вычислений. - -**Возвращаемые значения** - -* Массив того же размера и типа, что и входные данные. - -Функция использует [округление к нулю](https://en.wikipedia.org/wiki/Rounding#Rounding_towards_zero). Она усекает дробную часть, незначимую для результирующего типа данных. - -**Пример** - -Пример таблицы `b`: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -Запросы: - -```sql -SELECT - groupArrayMovingAvg(int) AS I, - groupArrayMovingAvg(float) AS F, - groupArrayMovingAvg(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F───────────────────────────────────┬─D─────────────────────┐ -│ [0,0,1,3] │ [0.275,0.82500005,1.9250001,3.8675] │ [0.27,0.82,1.92,3.86] │ -└───────────┴─────────────────────────────────────┴───────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingAvg(2)(int) AS I, - groupArrayMovingAvg(2)(float) AS F, - groupArrayMovingAvg(2)(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F────────────────────────────────┬─D─────────────────────┐ -│ [0,1,3,5] │ [0.55,1.6500001,3.3000002,6.085] │ [0.55,1.65,3.30,6.08] │ -└───────────┴──────────────────────────────────┴───────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md deleted file mode 100644 index 23d957fef9d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Вычисляет движущуюся сумму входных значений.' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingsum -title: 'groupArrayMovingSum' -doc_type: 'reference' ---- - -# groupArrayMovingSum {#grouparraymovingsum} - -Вычисляет скользящую сумму входных значений. - -```sql -groupArrayMovingSum(numbers_for_summing) -groupArrayMovingSum(window_size)(numbers_for_summing) -``` - -Функция может принимать размер окна в качестве параметра. Если он не указан, функция использует размер окна, равный числу строк в столбце. - -**Аргументы** - -* `numbers_for_summing` — [выражение](/sql-reference/syntax#expressions), результатом которого является значение числового типа данных. -* `window_size` — размер окна вычислений. - -**Возвращаемые значения** - -* Массив того же размера и типа, что и входные данные. - -**Пример** - -Пример таблицы: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -Запросы: - -```sql -SELECT - groupArrayMovingSum(int) AS I, - groupArrayMovingSum(float) AS F, - groupArrayMovingSum(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,7,14] │ [1.1,3.3000002,7.7000003,15.47] │ [1.10,3.30,7.70,15.47] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingSum(2)(int) AS I, - groupArrayMovingSum(2)(float) AS F, - groupArrayMovingSum(2)(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,6,11] │ [1.1,3.3000002,6.6000004,12.17] │ [1.10,3.30,6.60,12.17] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md deleted file mode 100644 index 32ffb0558d8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -description: 'Создает массив выборочных значений аргумента. Размер результирующего - массива ограничен `max_size` элементами. Значения аргумента выбираются и добавляются - в массив случайным образом.' -sidebar_position: 145 -slug: /sql-reference/aggregate-functions/reference/grouparraysample -title: 'groupArraySample' -doc_type: 'reference' ---- - -# groupArraySample {#grouparraysample} - -Создаёт массив случайных значений аргумента. Размер результирующего массива ограничен `max_size` элементами. Значения аргумента выбираются и добавляются в массив случайным образом. - -**Синтаксис** - -```sql -groupArraySample(max_size[, seed])(x) -``` - -**Аргументы** - -* `max_size` — Максимальный размер возвращаемого массива. [UInt64](../../data-types/int-uint.md). -* `seed` — Начальное значение для генератора случайных чисел (seed). Необязательный параметр. [UInt64](../../data-types/int-uint.md). Значение по умолчанию: `123456`. -* `x` — Аргумент (имя столбца или выражение). - -**Возвращаемые значения** - -* Массив случайным образом выбранных значений `x`. - -Тип: [Array](../../data-types/array.md). - -**Примеры** - -Рассмотрим таблицу `colors`: - -```text -┌─id─┬─color──┐ -│ 1 │ красный │ -│ 2 │ синий │ -│ 3 │ зеленый │ -│ 4 │ белый │ -│ 5 │ оранжевый │ -└────┴────────┘ -``` - -Запрос, в котором имя столбца передаётся как аргумент: - -```sql -SELECT groupArraySample(3)(color) as newcolors FROM colors; -``` - -Результат: - -```text -┌─newcolors──────────────────┐ -│ ['white','blue','green'] │ -└────────────────────────────┘ -``` - -Запрос с именем столбца и другим значением seed: - -```sql -SELECT groupArraySample(3, 987654321)(color) as newcolors FROM colors; -``` - -Результат: - -```text -┌─newcolors──────────────────┐ -│ ['red','orange','green'] │ -└────────────────────────────┘ -``` - -Запрос с выражением в качестве аргумента: - -```sql -SELECT groupArraySample(3)(concat('light-', color)) as newcolors FROM colors; -``` - -Результат: - -```text -┌─newcolors───────────────────────────────────┐ -│ ['light-blue','light-orange','light-green'] │ -└─────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md deleted file mode 100644 index d039750530b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'Возвращает массив из первых N элементов, отсортированных по возрастанию.' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/grouparraysorted -title: 'groupArraySorted' -doc_type: 'reference' ---- - -# groupArraySorted {#grouparraysorted} - -Возвращает массив из первых N элементов в порядке возрастания. - -```sql -groupArraySorted(N)(column) -``` - -**Аргументы** - -* `N` – Количество возвращаемых элементов. - -* `column` – Значение столбца (Integer, String, Float и другие обобщённые типы). - -**Пример** - -Возвращает первые 10 чисел: - -```sql -SELECT groupArraySorted(10)(number) FROM numbers(100) -``` - -```text -┌─groupArraySorted(10)(number)─┐ -│ [0,1,2,3,4,5,6,7,8,9] │ -└──────────────────────────────┘ -``` - -Возвращает все строковые представления всех чисел в столбце: - -```sql -SELECT groupArraySorted(5)(str) FROM (SELECT toString(number) AS str FROM numbers(5)); -``` - -```text -┌─groupArraySorted(5)(str)─┐ -│ ['0','1','2','3','4'] │ -└──────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md deleted file mode 100644 index 0bf7613065e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Применяет операцию побитового `AND` к последовательности чисел.' -sidebar_position: 147 -slug: /sql-reference/aggregate-functions/reference/groupbitand -title: 'groupBitAnd' -doc_type: 'reference' ---- - -# groupBitAnd {#groupbitand} - -Применяет побитовое `AND` к последовательности чисел. - -```sql -groupBitAnd(expr) -``` - -**Аргументы** - -`expr` – выражение, результат которого имеет тип `UInt*` или `Int*`. - -**Возвращаемое значение** - -Значение типа `UInt*` или `Int*`. - -**Пример** - -Тестовые данные: - -```text -двоичное десятичное -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -Запрос: - -```sql -SELECT groupBitAnd(num) FROM t -``` - -Где `num` — столбец с тестовыми данными. - -Результат: - -```text -двоичная десятичная -00000100 = 4 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md deleted file mode 100644 index b0285b51197..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'Битовые или агрегатные вычисления по беззнаковому целочисленному столбцу. Возвращают мощность множества типа UInt64, а при добавлении суффикса -State — объект bitmap.' -sidebar_position: 148 -slug: /sql-reference/aggregate-functions/reference/groupbitmap -title: 'groupBitmap' -doc_type: 'reference' ---- - -# groupBitmap {#groupbitmap} - -Выполняет Bitmap- или агрегатные вычисления по беззнаковому целочисленному столбцу и возвращает мощность множества (кардинальность) в виде значения типа UInt64. Если добавить суффикс `-State`, функция вернёт [bitmap-объект](../../../sql-reference/functions/bitmap-functions.md). - -```sql -groupBitmap(expr) -``` - -**Аргументы** - -`expr` – выражение, результатом которого является значение типа `UInt*`. - -**Возвращаемое значение** - -Значение типа `UInt64`. - -**Пример** - -Тестовые данные: - -```text -UserID -1 -1 -2 -3 -``` - -Запрос: - -```sql -SELECT groupBitmap(UserID) AS num FROM t -``` - -Результат: - -```text -num -3 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md deleted file mode 100644 index 731e9036a56..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'Выполняет операцию AND над столбцом bitmap, возвращает мощность множества типа - UInt64; если добавить суффикс -State, то возвращается объект bitmap.' -sidebar_position: 149 -slug: /sql-reference/aggregate-functions/reference/groupbitmapand -title: 'groupBitmapAnd' -doc_type: 'reference' ---- - -Выполняет операцию AND над столбцом bitmap, возвращает мощность множества типа UInt64; если добавить суффикс -State, то возвращается [объект bitmap](../../../sql-reference/functions/bitmap-functions.md). - -```sql -groupBitmapAnd(expr) -``` - -**Аргументы** - -`expr` – выражение, результатом вычисления которого является значение типа `AggregateFunction(groupBitmap, UInt*)`. - -**Возвращаемое значение** - -Значение типа `UInt64`. - -**Пример** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapAnd(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapAnd(z)─┐ -│ 3 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapAndState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapAndState(z)))─┐ -│ [6,8,10] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md deleted file mode 100644 index 0b857921a5e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Выполняет операцию OR над битмап-столбцом и возвращает кардинальность типа UInt64; при добавлении суффикса -State возвращает объект битмапа. Эквивалентна функции `groupBitmapMerge`.' -sidebar_position: 150 -slug: /sql-reference/aggregate-functions/reference/groupbitmapor -title: 'groupBitmapOr' -doc_type: 'reference' ---- - -# groupBitmapOr {#groupbitmapor} - -Вычисляет побитовое OR для битмап-столбца, возвращает мощность множества типа UInt64; если добавить суффикс `-State`, то возвращается [объект bitmap](../../../sql-reference/functions/bitmap-functions.md). Эквивалентно `groupBitmapMerge`. - -```sql -groupBitmapOr(expr) -``` - -**Аргументы** - -`expr` – выражение, результат вычисления которого имеет тип `AggregateFunction(groupBitmap, UInt*)`. - -**Возвращаемое значение** - -Значение типа `UInt64`. - -**Пример** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapOr(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapOr(z)─┐ -│ 15 │ -└──────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapOrState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapOrState(z)))─┐ -│ [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] │ -└─────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md deleted file mode 100644 index 332239e69c1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Вычисляет XOR для битмап-колонки и возвращает мощность множества типа UInt64; при использовании с суффиксом -State возвращает объект битовой карты' -sidebar_position: 151 -slug: /sql-reference/aggregate-functions/reference/groupbitmapxor -title: 'groupBitmapXor' -doc_type: 'reference' ---- - -# groupBitmapXor {#groupbitmapxor} - -`groupBitmapXor` вычисляет XOR столбца-битмапа и возвращает его кардинальность в виде значения типа UInt64; если используется с суффиксом -State, то возвращает [объект bitmap](../../../sql-reference/functions/bitmap-functions.md). - -```sql -groupBitmapXor(expr) -``` - -**Аргументы** - -`expr` – выражение, результатом вычисления которого является тип `AggregateFunction(groupBitmap, UInt*)`. - -**Возвращаемое значение** - -Значение типа `UInt64`. - -**Пример** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapXor(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapXor(z)─┐ -│ 10 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapXorState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapXorState(z)))─┐ -│ [1,3,5,6,8,10,11,13,14,15] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md deleted file mode 100644 index ebee6ae7335..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Применяет побитовую операцию `OR` к последовательности чисел.' -sidebar_position: 152 -slug: /sql-reference/aggregate-functions/reference/groupbitor -title: 'groupBitOr' -doc_type: 'reference' ---- - -# groupBitOr {#groupbitor} - -Применяет операцию побитового `OR` к последовательности чисел. - -```sql -groupBitOr(expr) -``` - -**Аргументы** - -`expr` – выражение, результатом которого является значение типа `UInt*` или `Int*`. - -**Возвращаемое значение** - -Значение типа `UInt*` или `Int*`. - -**Пример** - -Тестовые данные: - -```text -двоичное десятичное -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -Запрос: - -```sql -SELECT groupBitOr(num) FROM t -``` - -Где `num` — столбец с тестовыми данными. - -Результат: - -```text -двоичное десятичное -01111101 = 125 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md deleted file mode 100644 index 14affa074dd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Применяет побитовое `XOR` к последовательности чисел.' -sidebar_position: 153 -slug: /sql-reference/aggregate-functions/reference/groupbitxor -title: 'groupBitXor' -doc_type: 'reference' ---- - -# groupBitXor {#groupbitxor} - -Применяет операцию побитового `XOR` к серии чисел. - -```sql -groupBitXor(expr) -``` - -**Аргументы** - -`expr` – выражение, результатом вычисления которого является значение типа `UInt*` или `Int*`. - -**Возвращаемое значение** - -Значение типа `UInt*` или `Int*`. - -**Пример** - -Тестовые данные: - -```text -двоичное десятичное -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -Запрос: - -```sql -SELECT groupBitXor(num) FROM t -``` - -Где `num` — столбец с тестовыми данными. - -Результат: - -```text -двоичное десятичное -01101000 = 104 -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md deleted file mode 100644 index d0bdcce53dc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: 'Формирует строку, полученную конкатенацией группы строк, при необходимости разделённых разделителем и с возможностью ограничения максимального числа элементов.' -sidebar_label: 'groupConcat' -sidebar_position: 363 -slug: /sql-reference/aggregate-functions/reference/groupconcat -title: 'groupConcat' -doc_type: 'reference' ---- - -Формирует строку, полученную конкатенацией группы строк, при необходимости разделённых разделителем и с возможностью ограничения максимального числа элементов. - -**Синтаксис** - -```sql -groupConcat[(delimiter [, limit])](expression); -``` - -Псевдоним: `group_concat` - -**Аргументы** - -* `expression` — выражение или имя столбца, результатом которого являются строки для конкатенации. -* `delimiter` — [строка](../../../sql-reference/data-types/string.md), которая будет использоваться для разделения конкатенированных значений. Этот параметр является необязательным и по умолчанию имеет значение пустой строки или разделителя из параметров, если он не указан. - -**Параметры** - -* `delimiter` — [строка](../../../sql-reference/data-types/string.md), которая будет использоваться для разделения конкатенированных значений. Этот параметр является необязательным и по умолчанию имеет значение пустой строки, если он не указан. -* `limit` — положительное [целое число](../../../sql-reference/data-types/int-uint.md), задающее максимальное количество элементов для конкатенации. Если элементов больше, лишние элементы игнорируются. Этот параметр является необязательным. - -:::note -Если `delimiter` указан без `limit`, он должен быть первым параметром. Если указаны и `delimiter`, и `limit`, `delimiter` должен предшествовать `limit`. - -Также, если разные разделители указаны как параметры и как аргументы, будет использоваться только разделитель из аргументов. -::: - -**Возвращаемое значение** - -* Возвращает [строку](../../../sql-reference/data-types/string.md), состоящую из конкатенированных значений столбца или выражения. Если группа не содержит элементов или содержит только значения null, и функция не определяет отдельную обработку случая, когда есть только значения null, результатом является строка типа Nullable со значением null. - -**Примеры** - -Входная таблица: - -```text -┌─id─┬─name─┐ -│ 1 │ John │ -│ 2 │ Jane │ -│ 3 │ Bob │ -└────┴──────┘ -``` - -1. Базовое использование без разделителя: - -Запрос: - -```sql -SELECT groupConcat(Name) FROM Employees; -``` - -Результат: - -```text -JohnJaneBob -``` - -В результате все имена объединяются в одну строку без разделителя. - -2. Использование запятой в качестве разделителя: - -Запрос: - -```sql -SELECT groupConcat(', ')(Name) FROM Employees; -``` - -или - -```sql -SELECT groupConcat(Name, ', ') FROM Employees; -``` - -Результат: - -```text -John, Jane, Bob -``` - -Этот вывод показывает имена, разделённые запятой и пробелом. - -3. Ограничение количества объединяемых элементов - -Запрос: - -```sql -SELECT groupConcat(', ', 2)(Name) FROM Employees; -``` - -Результат: - -```text -Иван, Мария -``` - -Этот запрос ограничивает вывод первыми двумя именами, хотя в таблице содержится больше имён. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md deleted file mode 100644 index 0fe3628c43f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Создаёт массив из различных значений аргумента.' -sidebar_position: 154 -slug: /sql-reference/aggregate-functions/reference/groupuniqarray -title: 'groupUniqArray' -doc_type: 'reference' ---- - -# groupUniqArray {#groupuniqarray} - -Синтаксис: `groupUniqArray(x)` или `groupUniqArray(max_size)(x)` - -Создаёт массив из уникальных значений аргумента. Потребление памяти такое же, как у функции [uniqExact](../../../sql-reference/aggregate-functions/reference/uniqexact.md). - -Вторая версия (с параметром `max_size`) ограничивает размер результирующего массива до `max_size` элементов. -Например, `groupUniqArray(1)(x)` эквивалентно `[any(x)]`. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md deleted file mode 100644 index f8b7824bf59..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -description: 'Обзорная страница агрегатных функций с полным списком агрегатных функций' -sidebar_position: 36 -slug: /sql-reference/aggregate-functions/reference/ -title: 'Агрегатные функции' -toc_folder_title: 'Справочник' -toc_hidden: true -doc_type: 'landing-page' ---- - -# Агрегатные функции {#aggregate-functions} - -ClickHouse поддерживает все стандартные агрегатные функции SQL ([sum](../reference/sum.md), [avg](../reference/avg.md), [min](../reference/min.md), [max](../reference/max.md), [count](../reference/count.md)), а также широкий набор других агрегатных функций. - -{/* Оглавление для этой страницы автоматически генерируется скриптом - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - из полей YAML front matter: slug, description, title. - - Если вы обнаружили ошибку, отредактируйте поля YAML front matter на соответствующих страницах. - */ } - -{/*AUTOGENERATED_START*/ } - -| Страница | Описание | -| --------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [aggThrow](/sql-reference/aggregate-functions/reference/aggthrow) | Эта функция может использоваться для проверки гарантий безопасности при возникновении исключений. При создании она будет выбрасывать исключение с указанной вероятностью. | -| [analysisOfVariance](/sql-reference/aggregate-functions/reference/analysis_of_variance) | Предоставляет статистический тест для однофакторного дисперсионного анализа (тест ANOVA). Это тест для нескольких групп нормально распределённых наблюдений, позволяющий определить, имеют ли все группы одинаковое среднее значение или нет. | -| [any](/sql-reference/aggregate-functions/reference/any) | Возвращает первое встреченное значение в столбце. | -| [anyHeavy](/sql-reference/aggregate-functions/reference/anyheavy) | Выбирает часто встречающееся значение с использованием алгоритма «heavy hitters». Если существует значение, которое более чем в половине случаев встречается в каждом потоке выполнения запроса, возвращается именно оно. Обычно результат не детерминирован. | -| [anyLast](/sql-reference/aggregate-functions/reference/anylast) | Выбирает последнее встретившееся значение столбца. | -| [approx_top_k](/sql-reference/aggregate-functions/reference/approxtopk) | Возвращает массив приблизительно наиболее частых значений и количества их вхождений в указанном столбце. | -| [approx_top_sum](/sql-reference/aggregate-functions/reference/approxtopsum) | Возвращает массив приблизительно наиболее часто встречающихся значений и их количеств в указанном столбце. | -| [argMax](/sql-reference/aggregate-functions/reference/argmax) | Вычисляет значение `arg`, соответствующее максимальному значению `val`. | -| [argMin](/sql-reference/aggregate-functions/reference/argmin) | Вычисляет значение `arg` для минимального значения `val`. Если существует несколько строк с одинаковым минимальным значением `val`, то то, какое из соответствующих значений `arg` будет возвращено, не является детерминированным. | -| [argAndMax](/sql-reference/aggregate-functions/reference/argandmax) | Вычисляет значения `arg` и `val` для максимального значения `val`. Если существует несколько строк с одинаковым максимальным значением `val`, то то, какие из соответствующих `arg` и `val` будут возвращены, не определено. | -| [argAndMin](/sql-reference/aggregate-functions/reference/argandmin) | Вычисляет значения `arg` и `val` для минимального значения `val`. Если существует несколько строк с одинаковым минимальным значением `val`, то какие именно из соответствующих значений `arg` и `val` будут возвращены, не гарантируется. | -| [groupArrayArray](/sql-reference/aggregate-functions/reference/grouparrayarray) | Объединяет массивы в один массив массивов. | -| [avg](/sql-reference/aggregate-functions/reference/avg) | Вычисляет среднее арифметическое. | -| [avgWeighted](/sql-reference/aggregate-functions/reference/avgweighted) | Вычисляет взвешенное арифметическое среднее. | -| [boundingRatio](/sql-reference/aggregate-functions/reference/boundingRatio) | Агрегатная функция, вычисляющая наклон между самой левой и самой правой точками в группе значений. | -| [categoricalInformationValue](/sql-reference/aggregate-functions/reference/categoricalinformationvalue) | Вычисляет значение `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` для каждой категории. | -| [contingency](/sql-reference/aggregate-functions/reference/contingency) | Функция `contingency` вычисляет коэффициент сопряжённости — значение, которое измеряет степень связи между двумя столбцами в таблице. Вычисление аналогично функции `cramersV`, но использует другой знаменатель в выражении под знаком квадратного корня. | -| [corr](/sql-reference/aggregate-functions/reference/corr) | Вычисляет коэффициент корреляции Пирсона. | -| [corrMatrix](/sql-reference/aggregate-functions/reference/corrmatrix) | Вычисляет матрицу корреляции для N переменных. | -| [corrStable](/sql-reference/aggregate-functions/reference/corrstable) | Вычисляет коэффициент корреляции Пирсона, используя численно устойчивый алгоритм. | -| [count](/sql-reference/aggregate-functions/reference/count) | Подсчитывает количество строк или значений, отличных от NULL. | -| [covarPop](/sql-reference/aggregate-functions/reference/covarpop) | Вычисляет ковариацию по генеральной совокупности | -| [covarPopMatrix](/sql-reference/aggregate-functions/reference/covarpopmatrix) | Возвращает матрицу ковариаций генеральной совокупности для N переменных. | -| [covarPopStable](/sql-reference/aggregate-functions/reference/covarpopstable) | Вычисляет ковариацию генеральной совокупности | -| [covarSamp](/sql-reference/aggregate-functions/reference/covarsamp) | Вычисляет значение выражения `Σ((x - x̅)(y - y̅)) / (n - 1)` | -| [covarSampMatrix](/sql-reference/aggregate-functions/reference/covarsampmatrix) | Возвращает выборочную ковариационную матрицу для N переменных. | -| [covarSampStable](/sql-reference/aggregate-functions/reference/covarsampstable) | Аналогична `covarSamp`, но работает медленнее при меньшей вычислительной погрешности. | -| [cramersV](/sql-reference/aggregate-functions/reference/cramersv) | Результат функции `cramersV` лежит в диапазоне от 0 (что соответствует отсутствию связи между переменными) до 1 и может достигать 1 только в том случае, если каждое значение полностью определяется другим. Эту величину можно рассматривать как меру связи между двумя переменными, выраженную в процентах от их максимально возможной вариации. | -| [cramersVBiasCorrected](/sql-reference/aggregate-functions/reference/cramersvbiascorrected) | Вычисляет V Крамера, но с поправкой на смещение. | -| [deltaSum](/sql-reference/aggregate-functions/reference/deltasum) | Суммирует арифметическую разность между соседними строками. | -| [deltaSumTimestamp](/sql-reference/aggregate-functions/reference/deltasumtimestamp) | Вычисляет разности между соседними строками и суммирует их. Отрицательные разности игнорируются. | -| [entropy](/sql-reference/aggregate-functions/reference/entropy) | Вычисляет энтропию Шеннона для столбца значений. | -| [estimateCompressionRatio](/sql-reference/aggregate-functions/reference/estimateCompressionRatio) | Оценивает коэффициент сжатия для заданного столбца, не выполняя его сжатие. | -| [exponentialMovingAverage](/sql-reference/aggregate-functions/reference/exponentialMovingAverage) | Вычисляет экспоненциальное скользящее среднее значений за заданный интервал времени. | -| [exponentialTimeDecayedAvg](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedAvg) | Возвращает экспоненциально сглаженное взвешенное скользящее среднее значений временного ряда в момент времени `t`. | -| [exponentialTimeDecayedCount](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedCount) | Возвращает накопленное экспоненциальное затухание для временного ряда в момент времени с индексом `t`. | -| [exponentialTimeDecayedMax](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedMax) | Возвращает максимум из значений вычисленного экспоненциально сглаженного скользящего среднего во времени: в момент с индексом `t` и в момент `t-1`. | -| [exponentialTimeDecayedSum](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedSum) | Возвращает сумму значений экспоненциально сглаженного скользящего среднего временного ряда в момент времени `t`. | -| [first_value](/sql-reference/aggregate-functions/reference/first_value) | Это псевдоним для any, но он был добавлен для совместимости с Window Functions, где иногда требуется обрабатывать значения `NULL` (по умолчанию все агрегатные функции ClickHouse игнорируют значения `NULL`). | -| [flameGraph](/sql-reference/aggregate-functions/reference/flame_graph) | Агрегатная функция, строящая флеймграф по списку трассировок стека. | -| [groupArray](/sql-reference/aggregate-functions/reference/grouparray) | Создаёт массив значений аргументов. Значения могут добавляться в массив в произвольном (неопределённом) порядке. | -| [groupArrayInsertAt](/sql-reference/aggregate-functions/reference/grouparrayinsertat) | Вставляет значение в массив на указанную позицию. | -| [groupArrayIntersect](/sql-reference/aggregate-functions/reference/grouparrayintersect) | Возвращает пересечение заданных массивов (все элементы, которые присутствуют во всех этих массивах). | -| [groupArrayLast](/sql-reference/aggregate-functions/reference/grouparraylast) | Создает массив из значений последнего аргумента. | -| [groupArrayMovingAvg](/sql-reference/aggregate-functions/reference/grouparraymovingavg) | Вычисляет скользящее среднее для входных значений. | -| [groupArrayMovingSum](/sql-reference/aggregate-functions/reference/grouparraymovingsum) | Вычисляет скользящую сумму входных значений. | -| [groupArraySample](/sql-reference/aggregate-functions/reference/grouparraysample) | Создает массив примеров значений аргумента. Размер результирующего массива ограничен `max_size` элементами. Значения аргумента выбираются и добавляются в массив случайным образом. | -| [groupArraySorted](/sql-reference/aggregate-functions/reference/grouparraysorted) | Возвращает массив из первых N элементов, упорядоченных по возрастанию. | -| [timeSeriesGroupArray](/sql-reference/aggregate-functions/reference/timeSeriesGroupArray) | Сортирует временные ряды по метке времени в порядке возрастания. | -| [groupBitAnd](/sql-reference/aggregate-functions/reference/groupbitand) | Применяет операцию побитового `AND` к последовательности чисел. | -| [groupBitmap](/sql-reference/aggregate-functions/reference/groupbitmap) | Выполняет bitmap- или агрегатные вычисления по беззнаковому целочисленному столбцу, возвращает мощность множества в виде значения типа UInt64, а при добавлении суффикса -State возвращает объект bitmap | -| [groupBitmapAnd](/sql-reference/aggregate-functions/reference/groupbitmapand) | Выполняет побитовую операцию AND над bitmap-столбцом, возвращает кардинальность множества в виде значения типа UInt64; если добавить суффикс -State, то возвращает bitmap-объект. | -| [groupBitmapOr](/sql-reference/aggregate-functions/reference/groupbitmapor) | Вычисляет логическое ИЛИ по битмап-столбцу и возвращает кардинальность типа UInt64; при добавлении суффикса -State возвращает объект битмапа. Эквивалентна функции `groupBitmapMerge`. | -| [groupBitmapXor](/sql-reference/aggregate-functions/reference/groupbitmapxor) | Вычисляет XOR битового столбца и возвращает кардинальность (cardinality) типа UInt64; если используется с суффиксом -State, то возвращает объект битовой карты | -| [groupBitOr](/sql-reference/aggregate-functions/reference/groupbitor) | Применяет побитовую операцию `OR` к последовательности чисел. | -| [groupBitXor](/sql-reference/aggregate-functions/reference/groupbitxor) | Применяет побитовую операцию `XOR` к последовательности чисел. | -| [groupUniqArray](/sql-reference/aggregate-functions/reference/groupuniqarray) | Создаёт массив из значений переданных аргументов. | -| [intervalLengthSum](/sql-reference/aggregate-functions/reference/intervalLengthSum) | Вычисляет общую длину объединения всех диапазонов (отрезков на числовой прямой). | -| [kolmogorovSmirnovTest](/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest) | Применяет критерий Колмогорова–Смирнова к выборкам из двух распределений. | -| [kurtPop](/sql-reference/aggregate-functions/reference/kurtpop) | Вычисляет эксцесс для последовательности. | -| [kurtSamp](/sql-reference/aggregate-functions/reference/kurtsamp) | Вычисляет выборочный эксцесс для последовательности. | -| [largestTriangleThreeBuckets](/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets) | Применяет алгоритм «Largest-Triangle-Three-Buckets» к входным данным. | -| [last_value](/sql-reference/aggregate-functions/reference/last_value) | Выбирает последнее встретившееся значение, аналогично `anyLast`, но допускает значение NULL. | -| [mannWhitneyUTest](/sql-reference/aggregate-functions/reference/mannwhitneyutest) | Применяет ранговый критерий Манна — Уитни к выборкам из двух генеральных совокупностей. | -| [max](/sql-reference/aggregate-functions/reference/max) | Агрегатная функция, вычисляющая максимум по группе значений. | -| [maxIntersections](/sql-reference/aggregate-functions/reference/maxintersections) | Агрегатная функция, вычисляющая максимальное количество одновременных пересечений в группе интервалов (при условии, что все интервалы пересекаются хотя бы один раз). | -| [maxIntersectionsPosition](/sql-reference/aggregate-functions/reference/maxintersectionsposition) | Агрегатная функция, вычисляющая позиции вхождений функции maxIntersections. | -| [maxMap](/sql-reference/aggregate-functions/reference/maxmap) | Вычисляет максимальное значение в массиве `value` по ключам, указанным в массиве `key`. | -| [meanZTest](/sql-reference/aggregate-functions/reference/meanztest) | Применяет z‑критерий для сравнения средних по выборкам из двух генеральных совокупностей. | -| [median](/sql-reference/aggregate-functions/reference/median) | Функции `median*` являются псевдонимами соответствующих функций `quantile*`. Они вычисляют медиану выборки числовых данных. | -| [min](/sql-reference/aggregate-functions/reference/min) | Агрегатная функция, вычисляющая минимальное значение по группе значений. | -| [minMap](/sql-reference/aggregate-functions/reference/minmap) | Вычисляет минимальное значение из массива `value` по ключам, указанным в массиве `key`. | -| [quantile](/sql-reference/aggregate-functions/reference/quantile) | Вычисляет приближённый квантиль последовательности числовых данных. | -| [quantileBFloat16](/sql-reference/aggregate-functions/reference/quantilebfloat16) | Вычисляет приближённый квантиль выборки, состоящей из чисел типа bfloat16. | -| [quantileDD](/sql-reference/aggregate-functions/reference/quantileddsketch) | Вычисляет приближённый квантиль выборки с гарантированной относительной погрешностью. | -| [quantileDeterministic](/sql-reference/aggregate-functions/reference/quantiledeterministic) | Вычисляет приближённое значение квантили для числовой последовательности данных. | -| [Функции quantileExact](/sql-reference/aggregate-functions/reference/quantileexact) | функции quantileExact, quantileExactLow, quantileExactHigh, quantileExactExclusive, quantileExactInclusive | -| [quantileExactWeighted](/sql-reference/aggregate-functions/reference/quantileexactweighted) | Точно вычисляет квантиль последовательности числовых данных с учетом веса каждого значения. | -| [quantileGK](/sql-reference/aggregate-functions/reference/quantileGK) | Вычисляет квантиль числовой последовательности данных с использованием алгоритма Гринвальда — Ханны (Greenwald-Khanna). | -| [quantileExactWeightedInterpolated](/sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated) | Вычисляет квантиль последовательности числовых данных с использованием линейной интерполяции с учётом веса каждого элемента. | -| [quantileInterpolatedWeighted](/sql-reference/aggregate-functions/reference/quantileInterpolatedWeighted) | Вычисляет квантиль последовательности числовых данных с использованием линейной интерполяции с учётом веса каждого элемента. | -| [Функции quantiles](/sql-reference/aggregate-functions/reference/quantiles) | quantiles, quantilesExactExclusive, quantilesExactInclusive, quantilesGK | -| [quantileTDigest](/sql-reference/aggregate-functions/reference/quantiletdigest) | Вычисляет приближённый квантиль последовательности числовых данных с помощью алгоритма t-digest. | -| [quantileTDigestWeighted](/sql-reference/aggregate-functions/reference/quantiletdigestweighted) | Вычисляет приближённый квантиль по числовой последовательности данных на основе алгоритма t-digest. | -| [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) | Вычисляет квантиль числовой последовательности данных с заданной точностью. | -| [quantileTimingWeighted](/sql-reference/aggregate-functions/reference/quantiletimingweighted) | С заданной точностью вычисляет квантиль числовой последовательности с учётом веса каждого её элемента. | -| [rankCorr](/sql-reference/aggregate-functions/reference/rankCorr) | Вычисляет коэффициент ранговой корреляции. | -| [simpleLinearRegression](/sql-reference/aggregate-functions/reference/simplelinearregression) | Выполняет простую (одномерную) линейную регрессию. | -| [singleValueOrNull](/sql-reference/aggregate-functions/reference/singlevalueornull) | Агрегатная функция `singleValueOrNull` используется для реализации операторов с подзапросами, таких как `x = ALL (SELECT ...)`. Она проверяет, есть ли в данных ровно одно уникальное значение, отличное от NULL. | -| [skewPop](/sql-reference/aggregate-functions/reference/skewpop) | Вычисляет коэффициент асимметрии последовательности. | -| [skewSamp](/sql-reference/aggregate-functions/reference/skewsamp) | Вычисляет выборочный коэффициент асимметрии последовательности. | -| [sparkbar](/sql-reference/aggregate-functions/reference/sparkbar) | Функция строит гистограмму частот для значений `x` и их частоты повторения `y` на интервале `[min_x, max_x]`. | -| [stddevPop](/sql-reference/aggregate-functions/reference/stddevpop) | Результат равен квадратному корню из varPop. | -| [stddevPopStable](/sql-reference/aggregate-functions/reference/stddevpopstable) | Результат равен квадратному корню из varPop. В отличие от stddevPop, эта функция использует численно стабильный алгоритм. | -| [stddevSamp](/sql-reference/aggregate-functions/reference/stddevsamp) | Результат равен квадратному корню результата функции varSamp | -| [stddevSampStable](/sql-reference/aggregate-functions/reference/stddevsampstable) | Результат равен квадратному корню из результата varSamp. В отличие от функции varSamp, эта функция использует численно устойчивый алгоритм. | -| [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlinearregression) | Эта функция реализует стохастическую линейную регрессию. Она поддерживает пользовательские параметры для скорости обучения, коэффициента L2-регуляризации, размера мини-батча и несколько методов обновления весов (Adam, простой SGD, Momentum, Nesterov). | -| [stochasticLogisticRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) | Эта функция реализует стохастическую логистическую регрессию. Она может использоваться для решения задачи бинарной классификации, поддерживает те же настраиваемые параметры, что и stochasticLinearRegression, и работает аналогичным образом. | -| [studentTTest](/sql-reference/aggregate-functions/reference/studentttest) | Применяет t-критерий Стьюдента к выборкам из двух генеральных совокупностей. | -| [studentTTestOneSample](/sql-reference/aggregate-functions/reference/studentttestonesample) | Применяет одновыборочный t-критерий Стьюдента к выборке и известному среднему значению генеральной совокупности. | -| [sum](/sql-reference/aggregate-functions/reference/sum) | Вычисляет сумму. Применимо только к числам. | -| [sumCount](/sql-reference/aggregate-functions/reference/sumcount) | Вычисляет сумму чисел и одновременно считает количество строк. Функция используется оптимизатором запросов ClickHouse: если в запросе присутствует несколько функций `sum`, `count` или `avg`, их можно заменить одной функцией `sumCount`, чтобы повторно использовать результаты вычислений. Функцию редко требуется вызывать напрямую. | -| [sumKahan](/sql-reference/aggregate-functions/reference/sumkahan) | Вычисляет сумму чисел с использованием алгоритма компенсированного суммирования Кэхэна | -| [sumMap](/sql-reference/aggregate-functions/reference/summap) | Суммирует один или несколько массивов `value` в соответствии с ключами из массива `key`. Возвращает кортеж массивов: ключи в отсортированном порядке, далее значения, просуммированные для соответствующих ключей без переполнения. | -| [sumMapWithOverflow](/sql-reference/aggregate-functions/reference/summapwithoverflow) | Подсчитывает сумму элементов массива `value` с учётом ключей, указанных в массиве `key`. Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, просуммированные для соответствующих ключей. Отличается от функции sumMap тем, что выполняет суммирование с переполнением. | -| [sumWithOverflow](/sql-reference/aggregate-functions/reference/sumwithoverflow) | Вычисляет сумму чисел, используя для результата тот же тип данных, что и для входных параметров. Если сумма превышает максимальное значение для этого типа данных, она вычисляется с переполнением. | -| [theilsU](/sql-reference/aggregate-functions/reference/theilsu) | Функция `theilsU` вычисляет коэффициент неопределённости U Тейла — показатель, характеризующий взаимосвязь между двумя столбцами в таблице. | -| [topK](/sql-reference/aggregate-functions/reference/topk) | Возвращает массив приблизительно наиболее часто встречающихся значений в указанном столбце. Полученный массив упорядочен по убыванию их приблизительной частоты (а не по самим значениям). | -| [topKWeighted](/sql-reference/aggregate-functions/reference/topkweighted) | Возвращает массив примерно наиболее часто встречающихся значений в указанном столбце. Полученный массив отсортирован по убыванию примерной частоты значений (а не по самим значениям). При этом учитывается вес значения. | -| [uniq](/sql-reference/aggregate-functions/reference/uniq) | Вычисляет примерное количество уникальных значений аргумента. | -| [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) | Вычисляет приблизительное число различных значений аргумента. | -| [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) | Вычисляет приблизительное количество различных значений аргумента. Аналогична uniqCombined, но использует 64-битный хэш для всех типов данных, а не только для String. | -| [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) | Вычисляет точное количество различных значений аргумента. | -| [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) | Вычисляет приблизительное количество различных значений аргумента с использованием алгоритма HyperLogLog. | -| [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) | Вычисляет приблизительное количество различных значений аргумента с использованием фреймворка Theta Sketch. | -| [varPop](/sql-reference/aggregate-functions/reference/varPop) | Вычисляет генеральную дисперсию. | -| [varPopStable](/sql-reference/aggregate-functions/reference/varpopstable) | Возвращает генеральную дисперсию. В отличие от varPop, эта функция использует численно устойчивый алгоритм. Она работает медленнее, но даёт меньшую вычислительную погрешность. | -| [varSamp](/sql-reference/aggregate-functions/reference/varSamp) | Вычисляет выборочную дисперсию набора данных. | -| [varSampStable](/sql-reference/aggregate-functions/reference/varsampstable) | Вычисляет выборочную дисперсию набора данных. В отличие от `varSamp`, эта функция использует численно устойчивый алгоритм. Работает медленнее, но обеспечивает меньшую вычислительную погрешность. | -| [welchTTest](/sql-reference/aggregate-functions/reference/welchttest) | Применяет t‑критерий Уэлча к выборкам из двух совокупностей. | -| [distinctDynamicTypes](/sql-reference/aggregate-functions/reference/distinctdynamictypes) | Вычисляет список уникальных типов данных, хранящихся в столбце Dynamic. | -| [distinctJSONPaths](/sql-reference/aggregate-functions/reference/distinctjsonpaths) | Вычисляет список уникальных путей, хранящихся в JSON-столбце. | -| [timeSeriesDeltaToGrid](/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid) | Агрегатная функция, вычисляющая дельту в стиле PromQL для данных временных рядов на заданной сетке. | -| [timeSeriesInstantDeltaToGrid](/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid) | Агрегатная функция, вычисляющая PromQL-подобный idelta для данных временных рядов на заданной сетке. | -| [timeSeriesInstantRateToGrid](/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid) | Агрегатная функция, вычисляющая PromQL‑подобный irate для данных временных рядов на заданной временной сетке. | -| [timeSeriesLastTwoSamples](/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples) | Агрегатная функция для ресемплирования временных рядов при вычислении функций irate и idelta по аналогии с PromQL | -| [timeSeriesRateToGrid](/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid) | Агрегирующая функция, вычисляющая PromQL-подобный rate по данным временных рядов на заданной временной сетке. | -| [timeSeriesResampleToGridWithStaleness](/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness) | Агрегатная функция для ресемплирования данных временных рядов по заданной временной сетке. | -| [timeSeriesDerivToGrid](/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid) | Агрегатная функция, вычисляющая производную, аналогичную PromQL, по данным временных рядов на заданной сетке. | -| [timeSeriesPredictLinearToGrid](/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid) | Агрегатная функция, вычисляющая линейный прогноз по данным временных рядов в стиле PromQL на заданной сетке. | -| [timeSeriesChangesToGrid](/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid) | Агрегатная функция, вычисляющая изменения в данных временных рядов в стиле PromQL на заданной временной сетке. | -| [timeSeriesResetsToGrid](/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid) | Агрегатная функция, вычисляющая сбросы в стиле PromQL по данным временных рядов на заданной сетке. | -| [groupConcat](/sql-reference/aggregate-functions/reference/groupconcat) | Вычисляет строку, полученную конкатенацией группы строк, с необязательным разделителем и необязательным ограничением на максимальное число элементов. | -| [quantilePrometheusHistogram](/sql-reference/aggregate-functions/reference/quantilePrometheusHistogram) | Вычисляет квантиль по гистограмме с помощью линейной интерполяции. | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md deleted file mode 100644 index 581b3f88ae6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'Вычисляет общую длину объединения всех диапазонов (отрезков числовой оси).' -sidebar_label: 'intervalLengthSum' -sidebar_position: 155 -slug: /sql-reference/aggregate-functions/reference/intervalLengthSum -title: 'intervalLengthSum' -doc_type: 'reference' ---- - -Вычисляет общую длину объединения всех диапазонов (отрезков числовой оси). - -**Синтаксис** - -```sql -intervalLengthSum(start, end) -``` - -**Аргументы** - -* `start` — начальное значение интервала. [Int32](/sql-reference/data-types/int-uint#integer-ranges), [Int64](/sql-reference/data-types/int-uint#integer-ranges), [UInt32](/sql-reference/data-types/int-uint#integer-ranges), [UInt64](/sql-reference/data-types/int-uint#integer-ranges), [Float32](/sql-reference/data-types/float), [Float64](/sql-reference/data-types/float), [DateTime](/sql-reference/data-types/datetime) или [Date](/sql-reference/data-types/date). -* `end` — конечное значение интервала. [Int32](/sql-reference/data-types/int-uint#integer-ranges), [Int64](/sql-reference/data-types/int-uint#integer-ranges), [UInt32](/sql-reference/data-types/int-uint#integer-ranges), [UInt64](/sql-reference/data-types/int-uint#integer-ranges), [Float32](/sql-reference/data-types/float), [Float64](/sql-reference/data-types/float), [DateTime](/sql-reference/data-types/datetime) или [Date](/sql-reference/data-types/date). - -:::note -Аргументы должны иметь одинаковый тип данных. В противном случае будет сгенерировано исключение. -::: - -**Возвращаемое значение** - -* Общая длина объединения всех диапазонов (отрезков на числовой оси). В зависимости от типа аргументов возвращаемое значение может иметь тип [UInt64](/sql-reference/data-types/int-uint#integer-ranges) или [Float64](/sql-reference/data-types/float). - -**Примеры** - -1. Входная таблица: - -```text -┌─id─┬─start─┬─end─┐ -│ a │ 1.1 │ 2.9 │ -│ a │ 2.5 │ 3.2 │ -│ a │ 4 │ 5 │ -└────┴───────┴─────┘ -``` - -В этом примере используются аргументы типа Float32. Функция возвращает значение типа Float64. - -Результат — сумма длин интервалов `[1.1, 3.2]` (объединение `[1.1, 2.9]` и `[2.5, 3.2]`) и `[4, 5]`. - -Запрос: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM fl_interval GROUP BY id ORDER BY id; -``` - -Результат: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 3.1 │ Float64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -2. Таблица входных данных: - -```text -┌─id─┬───────────────start─┬─────────────────end─┐ -│ a │ 2020-01-01 01:12:30 │ 2020-01-01 02:10:10 │ -│ a │ 2020-01-01 02:05:30 │ 2020-01-01 02:50:31 │ -│ a │ 2020-01-01 03:11:22 │ 2020-01-01 03:23:31 │ -└────┴─────────────────────┴─────────────────────┘ -``` - -В этом примере используются аргументы типа DateTime. Функция возвращает значение в секундах. - -Запрос: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM dt_interval GROUP BY id ORDER BY id; -``` - -Результат: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 6610 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -3. Входная таблица: - - -```text -┌─id─┬──────start─┬────────end─┐ -│ a │ 2020-01-01 │ 2020-01-04 │ -│ a │ 2020-01-12 │ 2020-01-18 │ -└────┴────────────┴────────────┘ -``` - -В этом примере используются аргументы типа Date. Функция возвращает значение в днях. - -Запрос: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM date_interval GROUP BY id ORDER BY id; -``` - -Результат: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 9 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md deleted file mode 100644 index fedbc7b40ce..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: 'Применяет критерий Колмогорова–Смирнова к выборкам из двух распределений.' -sidebar_label: 'kolmogorovSmirnovTest' -sidebar_position: 156 -slug: /sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest -title: 'kolmogorovSmirnovTest' -doc_type: 'reference' ---- - -# kolmogorovSmirnovTest {#kolmogorovsmirnovtest} - -Применяет критерий Колмогорова–Смирнова к выборкам из двух генеральных совокупностей. - -**Синтаксис** - -```sql -kolmogorovSmirnovTest([alternative, computation_method])(sample_data, sample_index) -``` - -Значения обеих выборок находятся в столбце `sample_data`. Если `sample_index` равен 0, то значение в этой строке относится к выборке из первой генеральной совокупности. В противном случае оно относится к выборке из второй генеральной совокупности. -Выборки должны принадлежать непрерывным одномерным распределениям вероятности. - -**Аргументы** - -* `sample_data` — данные выборки. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `sample_index` — индекс выборки. [Integer](../../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `alternative` — альтернативная гипотеза. (Необязательный параметр, значение по умолчанию: `'two-sided'`.) [String](../../../sql-reference/data-types/string.md). - Пусть F(x) и G(x) — функции распределения (CDF) первого и второго распределений соответственно. - * `'two-sided'` - Нулевая гипотеза состоит в том, что выборки получены из одного и того же распределения, т.е. `F(x) = G(x)` для всех x, - а альтернативная гипотеза состоит в том, что распределения различаются. - * `'greater'` - Нулевая гипотеза состоит в том, что значения в первой выборке *стохастически меньше*, чем во второй, - то есть CDF первого распределения лежит выше и, следовательно, левее CDF второго. - Это означает, что `F(x) >= G(x)` для всех x. Альтернативная гипотеза в этом случае: `F(x) < G(x)` хотя бы для одного x. - * `'less'`. - Нулевая гипотеза состоит в том, что значения в первой выборке *стохастически больше*, чем во второй, - то есть CDF первого распределения лежит ниже и, следовательно, правее CDF второго. - Это означает, что `F(x) <= G(x)` для всех x. Альтернативная гипотеза в этом случае: `F(x) > G(x)` хотя бы для одного x. -* `computation_method` — метод вычисления p-value. (Необязательный параметр, значение по умолчанию: `'auto'`.) [String](../../../sql-reference/data-types/string.md). - * `'exact'` — вычисление выполняется с использованием точного распределения вероятностей статистики критерия. Вычислительно затратен и избыточен, за исключением случая малых выборок. - * `'asymp'` (`'asymptotic'`) — вычисление выполняется с использованием аппроксимации. Для больших выборок точные и асимптотические p-value очень близки. - * `'auto'` — метод `'exact'` используется, когда максимальный размер выборки меньше 10'000. - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) из двух элементов: - -* вычисленное значение статистики. [Float64](../../../sql-reference/data-types/float.md). -* вычисленное p-value. [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Запрос: - -```sql -SELECT kolmogorovSmirnovTest('less', 'exact')(value, num) -FROM -( - SELECT - randNormal(0, 10) AS value, - 0 AS num - FROM numbers(10000) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(10000) -) -``` - -Результат: - -```text -┌─kolmogorovSmirnovTest('less', 'exact')(value, num)─┐ -│ (0.009899999999999996,0.37528595205132287) │ -└────────────────────────────────────────────────────┘ -``` - -Примечание: -p-значение больше 0,05 (при доверительной вероятности 95 %), поэтому нулевая гипотеза не отвергается. - -Запрос: - -```sql -SELECT kolmogorovSmirnovTest('two-sided', 'exact')(value, num) -FROM -( - SELECT - randStudentT(10) AS value, - 0 AS num - FROM numbers(100) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(100) -) -``` - -Результат: - -```text -┌─kolmogorovSmirnovTest('two-sided', 'exact')(value, num)─┐ -│ (0.4100000000000002,6.61735760482795e-8) │ -└─────────────────────────────────────────────────────────┘ -``` - -Примечание: -p-value (p-значение) меньше 0,05 (при уровне доверия 95 %), поэтому нулевая гипотеза отвергается. - -**См. также** - -* [Критерий Колмогорова–Смирнова](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test) \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md deleted file mode 100644 index 60ec854f6cd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'Вычисляет эксцесс последовательности.' -sidebar_position: 157 -slug: /sql-reference/aggregate-functions/reference/kurtpop -title: 'kurtPop' -doc_type: 'reference' ---- - -# kurtPop {#kurtpop} - -Вычисляет [эксцесс](https://en.wikipedia.org/wiki/Kurtosis) для последовательности. - -```sql -kurtPop(expr) -``` - -**Аргументы** - -`expr` — [выражение](/sql-reference/syntax#expressions), возвращающее число. - -**Возвращаемое значение** - -Коэффициент эксцесса (kurtosis) заданного распределения. Тип — [Float64](../../../sql-reference/data-types/float.md) - -**Пример** - -```sql -SELECT kurtPop(value) FROM series_with_value_column; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md deleted file mode 100644 index ab9ceebff8c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'Вычисляет выборочный эксцесс последовательности.' -sidebar_position: 158 -slug: /sql-reference/aggregate-functions/reference/kurtsamp -title: 'kurtSamp' -doc_type: 'reference' ---- - -# kurtSamp {#kurtsamp} - -Вычисляет [выборочный эксцесс](https://ru.wikipedia.org/wiki/Эксцесс_распределения) последовательности. - -Представляет собой несмещённую оценку эксцесса случайной величины, если переданные значения образуют её выборку. Подробнее: [https://en.wikipedia.org/wiki/Kurtosis](https://en.wikipedia.org/wiki/Kurtosis) - -```sql -kurtSamp(expr) -``` - -**Аргументы** - -`expr` — [выражение](/sql-reference/syntax#expressions), возвращающее число. - -**Возвращаемое значение** - -Эксцесс заданного распределения. Тип — [Float64](../../../sql-reference/data-types/float.md). Если `n <= 1` (`n` — размер выборки), то функция возвращает `nan`. - -**Пример** - -```sql -SELECT kurtSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md deleted file mode 100644 index 4edd876905e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Применяет алгоритм Largest-Triangle-Three-Buckets к входным данным.' -sidebar_label: 'largestTriangleThreeBuckets' -sidebar_position: 159 -slug: /sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets -title: 'largestTriangleThreeBuckets' -doc_type: 'reference' ---- - -# largestTriangleThreeBuckets {#largesttrianglethreebuckets} - -Применяет алгоритм [Largest-Triangle-Three-Buckets](https://skemman.is/bitstream/1946/15343/3/SS_MSthesis.pdf) к входным данным. -Алгоритм используется для даунсэмплинга данных временного ряда при визуализации. Он предназначен для работы с рядами, отсортированными по координате x. -Алгоритм работает, разбивая отсортированный ряд на корзины (buckets), а затем находя наибольший треугольник в каждой корзине. Количество корзин равно количеству точек в результирующем ряду. -Функция сортирует данные по `x`, а затем применяет алгоритм даунсэмплинга к отсортированным данным. - -**Синтаксис** - -```sql -largestTriangleThreeBuckets(n)(x, y) -``` - -Псевдоним: `lttb`. - -**Аргументы** - -* `x` — координата x. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md), [Decimal](../../../sql-reference/data-types/decimal.md), [Date](../../../sql-reference/data-types/date.md), [Date32](../../../sql-reference/data-types/date32.md), [DateTime](../../../sql-reference/data-types/datetime.md), [DateTime64](../../../sql-reference/data-types/datetime64.md). -* `y` — координата y. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md), [Decimal](../../../sql-reference/data-types/decimal.md), [Date](../../../sql-reference/data-types/date.md), [Date32](../../../sql-reference/data-types/date32.md), [DateTime](../../../sql-reference/data-types/datetime.md), [DateTime64](../../../sql-reference/data-types/datetime64.md). - -Значения NaN во входной последовательности игнорируются, то есть любые значения NaN исключаются из анализа. Это гарантирует, что функция работает только с корректными числовыми данными. - -**Параметры** - -* `n` — количество точек в результирующей последовательности. [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Возвращаемые значения** - -[Array](../../../sql-reference/data-types/array.md) из [Tuple](../../../sql-reference/data-types/tuple.md) с двумя элементами: - -**Пример** - -Входная таблица: - -```text -┌─────x───────┬───────y──────┐ -│ 1.000000000 │ 10.000000000 │ -│ 2.000000000 │ 20.000000000 │ -│ 3.000000000 │ 15.000000000 │ -│ 8.000000000 │ 60.000000000 │ -│ 9.000000000 │ 55.000000000 │ -│ 10.00000000 │ 70.000000000 │ -│ 4.000000000 │ 30.000000000 │ -│ 5.000000000 │ 40.000000000 │ -│ 6.000000000 │ 35.000000000 │ -│ 7.000000000 │ 50.000000000 │ -└─────────────┴──────────────┘ -``` - -Запрос: - -```sql -SELECT largestTriangleThreeBuckets(4)(x, y) FROM largestTriangleThreeBuckets_test; -``` - -Результат: - -```text -┌────────largestTriangleThreeBuckets(4)(x, y)───────────┐ -│ [(1,10),(3,15),(9,55),(10,70)] │ -└───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md deleted file mode 100644 index 5d61832df88..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md +++ /dev/null @@ -1,91 +0,0 @@ ---- -description: 'Выбирает последнее встреченное значение, аналогично `anyLast`, но может - принимать значение NULL.' -sidebar_position: 160 -slug: /sql-reference/aggregate-functions/reference/last_value -title: 'last_value' -doc_type: 'reference' ---- - -# last_value {#last_value} - -Выбирает последнее встреченное значение, аналогично `anyLast`, но допускает значение NULL. -Чаще всего используется с [оконными функциями](../../window-functions/index.md). -Без оконных функций результат будет случайным, если исходный поток не упорядочен. - -## Примеры {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null) -``` - -### Пример 1 {#example1} - -По умолчанию значение NULL игнорируется. - -```sql -SELECT last_value(b) FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### Пример 2 {#example2} - -Значение NULL игнорируется. - -```sql -SELECT last_value(b) ignore nulls FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### Пример 3 {#example3} - -Допускается значение NULL. - -```sql -SELECT last_value(b) respect nulls FROM test_data -``` - -```text -┌─last_value_respect_nulls(b)─┐ -│ ᴺᵁᴸᴸ │ -└─────────────────────────────┘ -``` - -### Пример 4 {#example4} - -Стабильный результат при использовании подзапроса с `ORDER BY`. - -```sql -SELECT - last_value_respect_nulls(b), - last_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─last_value_respect_nulls(b)─┬─last_value(b)─┐ -│ ᴺᵁᴸᴸ │ 5 │ -└─────────────────────────────┴───────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md deleted file mode 100644 index b5f4f8d0d49..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'Применяет ранговый критерий Манна — Уитни к выборкам из двух генеральных совокупностей.' -sidebar_label: 'mannWhitneyUTest' -sidebar_position: 161 -slug: /sql-reference/aggregate-functions/reference/mannwhitneyutest -title: 'mannWhitneyUTest' -doc_type: 'reference' ---- - -# mannWhitneyUTest {#mannwhitneyutest} - -Применяет ранговый критерий Манна — Уитни к выборкам из двух генеральных совокупностей. - -**Синтаксис** - -```sql -mannWhitneyUTest[(alternative[, continuity_correction])](sample_data, sample_index) -``` - -Значения обеих выборок находятся в столбце `sample_data`. Если `sample_index` равен 0, то значение в этой строке принадлежит выборке из первой совокупности. В противном случае оно принадлежит выборке из второй совокупности. -Нулевая гипотеза состоит в том, что две совокупности стохастически равны. Также можно проверять односторонние гипотезы. Тест не предполагает, что данные имеют нормальное распределение. - -**Аргументы** - -* `sample_data` — данные выборки. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `sample_index` — индекс выборки. [Integer](../../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `alternative` — альтернативная гипотеза (необязательный параметр, по умолчанию — `'two-sided'`). [String](../../../sql-reference/data-types/string.md). - * `'two-sided'`; - * `'greater'`; - * `'less'`. -* `continuity_correction` — если не равен 0, применяется поправка на непрерывность в нормальном приближении p-значения (необязательный параметр, по умолчанию — 1). [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) из двух элементов: - -* вычисленная U-статистика. [Float64](../../../sql-reference/data-types/float.md). -* вычисленное p-значение. [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Входная таблица: - -```text -┌─sample_data─┬─sample_index─┐ -│ 10 │ 0 │ -│ 11 │ 0 │ -│ 12 │ 0 │ -│ 1 │ 1 │ -│ 2 │ 1 │ -│ 3 │ 1 │ -└─────────────┴──────────────┘ -``` - -Запрос: - -```sql -SELECT mannWhitneyUTest('greater')(sample_data, sample_index) FROM mww_ttest; -``` - -Результат: - -```text -┌─mannWhitneyUTest('greater')(sample_data, sample_index)─┐ -│ (9,0.04042779918503192) │ -└────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [Критерий Манна — Уитни](https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test) -* [Стохастический порядок](https://en.wikipedia.org/wiki/Stochastic_ordering) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md deleted file mode 100644 index 7e5c7e38062..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: 'Агрегатная функция, вычисляющая максимальное значение по группе значений.' -sidebar_position: 162 -slug: /sql-reference/aggregate-functions/reference/max -title: 'max' -doc_type: 'reference' ---- - -Агрегатная функция, вычисляющая максимальное значение по группе значений. - -Пример: - -```sql -SELECT max(salary) FROM employees; -``` - -```sql -SELECT department, max(salary) FROM employees GROUP BY department; -``` - -Если вам нужна неагрегатная функция для выбора максимального значения из двух, см. `greatest`: - -```sql -SELECT greatest(a, b) FROM table; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md deleted file mode 100644 index 26e20201054..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md +++ /dev/null @@ -1,69 +0,0 @@ ---- -description: 'Агрегатная функция, вычисляющая максимальное количество раз, которое - интервалы в группе пересекаются друг с другом (если все интервалы пересекаются - хотя бы один раз).' -sidebar_position: 163 -slug: /sql-reference/aggregate-functions/reference/maxintersections -title: 'maxIntersections' -doc_type: 'reference' ---- - -# maxIntersections {#maxintersections} - -Агрегатная функция, вычисляющая максимальное количество раз, когда интервалы в группе пересекаются друг с другом (при условии, что все интервалы пересекаются хотя бы один раз). - -Синтаксис: - -```sql -maxIntersections(start_column, end_column) -``` - -**Аргументы** - -* `start_column` – числовой столбец, представляющий начало каждого интервала. Если `start_column` имеет значение `NULL` или 0, интервал будет пропущен. - -* `end_column` – числовой столбец, представляющий конец каждого интервала. Если `end_column` имеет значение `NULL` или 0, интервал будет пропущен. - -**Возвращаемое значение** - -Возвращает максимальное количество пересекающихся интервалов. - -**Пример** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -Интервалы выглядят следующим образом: - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -Три из этих интервалов имеют общую точку (значение равно `4`, но то, какое именно значение общее, не важно — нас интересует количество пересечений). Интервалы `(1,3)` и `(3,7)` имеют общую границу, но не считаются пересекающимися функцией `maxIntersections`. - -```sql -SELECT maxIntersections(start, end) FROM my_events; -``` - -Ответ: - -```response -3 -``` - -Если у вас есть несколько вхождений максимального интервала, вы можете использовать функцию [`maxIntersectionsPosition`](./maxintersectionsposition.md), чтобы найти количество таких интервалов и их положение. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md deleted file mode 100644 index 376ec84219d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: 'Агрегатная функция, вычисляющая позиции вхождений функции maxIntersections.' -sidebar_position: 164 -slug: /sql-reference/aggregate-functions/reference/maxintersectionsposition -title: 'maxIntersectionsPosition' -doc_type: 'reference' ---- - -# maxIntersectionsPosition {#maxintersectionsposition} - -Агрегатная функция, вычисляющая позиции вхождений функции [`maxIntersections`](./maxintersections.md). - -Синтаксис следующий: - -```sql -maxIntersectionsPosition(start_column, end_column) -``` - -**Аргументы** - -* `start_column` – числовой столбец, задающий начало каждого интервала. Если `start_column` равен `NULL` или 0, этот интервал будет пропущен. - -* `end_column` – числовой столбец, задающий конец каждого интервала. Если `end_column` равен `NULL` или 0, этот интервал будет пропущен. - -**Возвращаемое значение** - -Возвращает позиции начала максимального количества пересекающихся интервалов. - -**Пример** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -Интервалы выглядят следующим образом: - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -Обратите внимание, что у трёх из этих интервалов общим является значение 4 — начиная со второго интервала: - -```sql -SELECT maxIntersectionsPosition(start, end) FROM my_events; -``` - -Ответ: - -```response -2 -``` - -Другими словами, строка `(1,6)` является началом трёх пересекающихся интервалов, и 3 — максимальное число пересекающихся интервалов. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md deleted file mode 100644 index 2355921be8a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Вычисляет максимальные значения элементов массива `value` в соответствии с ключами, заданными в массиве `key`.' -sidebar_position: 165 -slug: /sql-reference/aggregate-functions/reference/maxmap -title: 'maxMap' -doc_type: 'reference' ---- - -# maxMap {#maxmap} - -Вычисляет максимальное значение из массива `value` по ключам, указанным в массиве `key`. - -**Синтаксис** - -```sql -maxMap(key, value) -``` - -или - -```sql -maxMap(Tuple(key, value)) -``` - -Псевдоним: `maxMappedArrays` - -:::note - -* Передача кортежа из массивов ключей и значений эквивалентна передаче двух массивов — ключей и значений. -* Количество элементов в `key` и `value` должно совпадать для каждой строки, участвующей в агрегации. - ::: - -**Параметры** - -* `key` — массив ключей. [Array](../../data-types/array.md). -* `value` — массив значений. [Array](../../data-types/array.md). - -**Возвращаемое значение** - -* Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, вычисленные для соответствующих ключей. [Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md)). - -**Пример** - -Запрос: - -```sql -SELECT maxMap(a, b) -FROM VALUES('a Array(Char), b Array(Int64)', (['x', 'y'], [2, 2]), (['y', 'z'], [3, 1])) -``` - -Результат: - -```text -┌─maxMap(a, b)───────────┐ -│ [['x','y','z'],[2,3,1]]│ -└────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md deleted file mode 100644 index 8a77987e77e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -description: 'Применяет z-тест для сравнения средних по выборкам из двух совокупностей.' -sidebar_label: 'meanZTest' -sidebar_position: 166 -slug: /sql-reference/aggregate-functions/reference/meanztest -title: 'meanZTest' -doc_type: 'reference' ---- - -# meanZTest {#meanztest} - -Применяет z-критерий для сравнения средних по выборкам из двух генеральных совокупностей. - -**Синтаксис** - -```sql -meanZTest(дисперсия_популяции_x, дисперсия_популяции_y, уровень_доверия)(данные_выборки, индекс_выборки) -``` - -Значения обеих выборок находятся в столбце `sample_data`. Если `sample_index` равен 0, то значение в этой строке относится к выборке первой генеральной совокупности. В противном случае оно относится к выборке второй генеральной совокупности. -Нулевая гипотеза состоит в том, что средние значения генеральных совокупностей равны. Предполагается нормальное распределение. Дисперсии генеральных совокупностей могут быть неравны и считаются известными. - -**Аргументы** - -* `sample_data` — Данные выборки. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `sample_index` — Индекс выборки. [Integer](../../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `population_variance_x` — Дисперсия для генеральной совокупности x. [Float](../../../sql-reference/data-types/float.md). -* `population_variance_y` — Дисперсия для генеральной совокупности y. [Float](../../../sql-reference/data-types/float.md). -* `confidence_level` — Уровень доверия для вычисления доверительных интервалов. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) из четырех элементов: - -* вычисленная t-статистика. [Float64](../../../sql-reference/data-types/float.md). -* вычисленное p-значение. [Float64](../../../sql-reference/data-types/float.md). -* вычисленная нижняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md). -* вычисленная верхняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Входная таблица: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.9 │ 0 │ -│ 22.1 │ 0 │ -│ 18.9 │ 1 │ -│ 19 │ 1 │ -│ 20.3 │ 1 │ -└─────────────┴──────────────┘ -``` - -Запрос: - -```sql -SELECT meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index) FROM mean_ztest -``` - -Результат: - -```text -┌─meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index)────────────────────────────┐ -│ (3.2841296025548123,0.0010229786769086013,0.8198428246768334,3.2468238419898365) │ -└──────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md deleted file mode 100644 index b8cd78cee80..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: 'Функции `median*` являются псевдонимами соответствующих функций `quantile*`. Они вычисляют медиану по выборке числовых данных.' -sidebar_position: 167 -slug: /sql-reference/aggregate-functions/reference/median -title: 'median' -doc_type: 'reference' ---- - -# median {#median} - -Функции `median*` являются псевдонимами соответствующих функций `quantile*`. Они вычисляют медиану по числовой выборке данных. - -Функции: - -* `median` — псевдоним для [quantile](/sql-reference/aggregate-functions/reference/quantile). -* `medianDeterministic` — псевдоним для [quantileDeterministic](/sql-reference/aggregate-functions/reference/quantiledeterministic). -* `medianExact` — псевдоним для [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact). -* `medianExactWeighted` — псевдоним для [quantileExactWeighted](/sql-reference/aggregate-functions/reference/quantileexactweighted). -* `medianTiming` — псевдоним для [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming). -* `medianTimingWeighted` — псевдоним для [quantileTimingWeighted](/sql-reference/aggregate-functions/reference/quantiletimingweighted). -* `medianTDigest` — псевдоним для [quantileTDigest](/sql-reference/aggregate-functions/reference/quantiletdigest). -* `medianTDigestWeighted` — псевдоним для [quantileTDigestWeighted](/sql-reference/aggregate-functions/reference/quantiletdigestweighted). -* `medianBFloat16` — псевдоним для [quantileBFloat16](/sql-reference/aggregate-functions/reference/quantilebfloat16). -* `medianDD` — псевдоним для [quantileDD](/sql-reference/aggregate-functions/reference/quantileddsketch). - -**Пример** - -Входная таблица: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -Запрос: - -```sql -SELECT medianDeterministic(val, 1) FROM t; -``` - -Результат: - -```text -┌─medianDeterministic(val, 1)─┐ -│ 1.5 │ -└─────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md deleted file mode 100644 index 7cdd4371a08..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: 'Агрегатная функция, вычисляющая минимальное значение по группе значений.' -sidebar_position: 168 -slug: /sql-reference/aggregate-functions/reference/min -title: 'min' -doc_type: 'reference' ---- - -Агрегатная функция, вычисляющая минимальное значение по группе значений. - -Пример: - -```sql -SELECT min(salary) FROM employees; -``` - -```sql -SELECT department, min(salary) FROM employees GROUP BY department; -``` - -Если вам нужна неагрегирующая функция для выбора минимального значения из двух, см. `least`: - -```sql -SELECT least(a, b) FROM table; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md deleted file mode 100644 index 41a9db038db..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: 'Вычисляет минимальное значение из массива `value` в соответствии с ключами, указанными в массиве `key`.' -sidebar_position: 169 -slug: /sql-reference/aggregate-functions/reference/minmap -title: 'minMap' -doc_type: 'reference' ---- - -# minMap {#minmap} - -Вычисляет минимум по массиву `value` в соответствии с ключами из массива `key`. - -**Синтаксис** - -```sql -`minMap(key, value)` -``` - -или - -```sql -minMap(Tuple(key, value)) -``` - -Псевдоним: `minMappedArrays` - -:::note - -* Передача кортежа, содержащего массивы ключей и значений, эквивалентна передаче массива ключей и массива значений. -* Количество элементов в `key` и `value` должно совпадать для каждой строки, для которой вычисляется итог. - ::: - -**Параметры** - -* `key` — массив ключей. [Array](../../data-types/array.md). -* `value` — массив значений. [Array](../../data-types/array.md). - -**Возвращаемое значение** - -* Возвращает кортеж из двух массивов: массив ключей в отсортированном порядке и массив значений, вычисленных для соответствующих ключей. [Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md)). - -**Пример** - -Запрос: - -```sql -SELECT minMap(a, b) -FROM VALUES('a Array(Int32), b Array(Int64)', ([1, 2], [2, 2]), ([2, 3], [1, 1])) -``` - -Результат: - -```text -┌─minMap(a, b)──────┐ -│ ([1,2,3],[2,1,1]) │ -└───────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md deleted file mode 100644 index 8814e79abb5..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Вычисляет приблизительный квантиль последовательности числовых данных.' -sidebar_position: 170 -slug: /sql-reference/aggregate-functions/reference/quantile -title: 'quantile' -doc_type: 'reference' ---- - -# quantile {#quantile} - -Вычисляет приближённый [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных. - -Эта функция использует метод [reservoir sampling](https://en.wikipedia.org/wiki/Reservoir_sampling) с размером резервуара до 8192 и генератором случайных чисел для выборки. Результат недетерминирован. Чтобы получить точный квантиль, используйте функцию [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact). - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -Обратите внимание, что для пустой числовой последовательности `quantile` вернёт NaN, а его варианты `quantile*` вернут либо NaN, либо значение по умолчанию для типа последовательности, в зависимости от варианта. - -**Синтаксис** - -```sql -quantile(level)(expr) -``` - -Псевдоним: `median`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](/sql-reference/data-types/date) или [DateTime](/sql-reference/data-types/datetime). - -**Возвращаемое значение** - -* Приближённый квантиль указанного уровня. - -Тип: - -* [Float64](/sql-reference/data-types/float) для числового типа данных на входе. -* [Date](/sql-reference/data-types/date), если входные значения имеют тип `Date`. -* [DateTime](/sql-reference/data-types/datetime), если входные значения имеют тип `DateTime`. - -**Пример** - -Входная таблица: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -Запрос: - -```sql -SELECT quantile(val) FROM t -``` - -Результат: - -```text -┌─quantile(val)─┐ -│ 1.5 │ -└───────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md deleted file mode 100644 index c8d24b76022..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'Вычисляет квантиль последовательности числовых данных с использованием - алгоритма Гринвальда–Ханны.' -sidebar_position: 175 -slug: /sql-reference/aggregate-functions/reference/quantileGK -title: 'quantileGK' -doc_type: 'reference' ---- - -# quantileGK {#quantilegk} - -Вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных с использованием алгоритма [Гринвальда–Ханны](http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf) (Greenwald-Khanna). Алгоритм Гринвальда–Ханны используется для высокоэффективного вычисления квантилей на потоке данных. Он был предложен Майклом Гринвальдом (Michael Greenwald) и Сандживом Ханной (Sanjeev Khanna) в 2001 году. Широко используется в базах данных и системах обработки больших данных, где необходимо вычислять точные значения квантилей на большом потоке данных в режиме реального времени. Алгоритм очень эффективен, требуя лишь O(log n) памяти и O(log log n) времени на элемент (где n — размер входных данных). Он также обладает высокой точностью, обеспечивая приближённое значение квантиля с высокой вероятностью. - -`quantileGK` отличается от других функций вычисления квантилей в ClickHouse, поскольку позволяет пользователю управлять точностью приблизительного результата. - -**Синтаксис** - -```sql -quantileGK(accuracy, level)(expr) -``` - -Псевдоним: `medianGK`. - -**Аргументы** - -* `accuracy` — Точность квантиля. Константное положительное целое число. Чем больше значение, тем меньше погрешность. Например, если аргумент `accuracy` равен 100, вычисленный квантиль с высокой вероятностью будет иметь погрешность не более 1%. Существует компромисс между точностью вычисляемых квантилей и вычислительной сложностью алгоритма. Более высокая точность требует больше памяти и вычислительных ресурсов, тогда как меньшая точность позволяет выполнять вычисления быстрее и с меньшим потреблением памяти, но с несколько меньшей точностью. - -* `level` — Уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). - -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня и точности. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -```sql -SELECT quantileGK(1, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1, 0.25)(plus(number, 1))─┐ -│ 1 │ -└──────────────────────────────────────┘ - -SELECT quantileGK(10, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(10, 0.25)(plus(number, 1))─┐ -│ 156 │ -└───────────────────────────────────────┘ - -SELECT quantileGK(100, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(100, 0.25)(plus(number, 1))─┐ -│ 251 │ -└────────────────────────────────────────┘ - -SELECT quantileGK(1000, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1000, 0.25)(plus(number, 1))─┐ -│ 249 │ -└─────────────────────────────────────────┘ -``` - -**См. также** - -* [median]/sql-reference/aggregate-functions/reference/median -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md deleted file mode 100644 index 29c4828ff31..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Вычисляет приближённый квантиль выборки, состоящей из чисел в формате bfloat16.' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantilebfloat16 -title: 'quantileBFloat16' -doc_type: 'reference' ---- - -# quantileBFloat16Weighted {#quantilebfloat16weighted} - -Как `quantileBFloat16`, но с учётом веса каждого элемента последовательности. - -Вычисляет приближённый [квантиль](https://en.wikipedia.org/wiki/Quantile) выборки, состоящей из чисел в формате [bfloat16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format). `bfloat16` — это тип чисел с плавающей запятой с 1 битом знака, 8 битами порядка и 7 битами мантиссы. -Функция преобразует входные значения в 32-битные числа с плавающей запятой и берёт 16 наиболее значимых битов. Затем она вычисляет значение квантиля в формате `bfloat16` и преобразует результат в 64-битное число с плавающей запятой путём добавления нулевых битов. -Функция является быстрым оценивателем квантиля с относительной погрешностью не более 0,390625%. - -**Синтаксис** - -```sql -quantileBFloat16[(level)](expr) -``` - -Псевдоним: `medianBFloat16` - -**Аргументы** - -* `expr` — столбец с числовыми данными. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md). - -**Параметры** - -* `level` — уровень квантиля. Необязательный параметр. Возможные значения находятся в диапазоне от 0 до 1. Значение по умолчанию: 0.5. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* Приближённый квантиль указанного уровня. - -Тип: [Float64](/sql-reference/data-types/float). - -**Пример** - -Входная таблица содержит целочисленный столбец и столбец с плавающей запятой: - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -Запрос для вычисления 0,75-квантили (третьего квартиля): - -```sql -SELECT quantileBFloat16(0.75)(a), quantileBFloat16(0.75)(b) FROM example_table; -``` - -Результат: - -```text -┌─quantileBFloat16(0.75)(a)─┬─quantileBFloat16(0.75)(b)─┐ -│ 3 │ 1 │ -└───────────────────────────┴───────────────────────────┘ -``` - -Обратите внимание, что все числа с плавающей запятой в примере усекаются до 1.0 при преобразовании в `bfloat16`. - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md deleted file mode 100644 index 5d6dcf22f5a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: 'Вычисляет приблизительный квантиль выборки с гарантиями относительной ошибки.' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantileddsketch -title: 'quantileDD' -doc_type: 'reference' ---- - -Вычисляет приблизительный [квантиль](https://en.wikipedia.org/wiki/Quantile) выборки с гарантиями относительной ошибки. Для этого строится [DD](https://www.vldb.org/pvldb/vol12/p2195-masson.pdf). - -**Синтаксис** - -```sql -quantileDD(relative_accuracy, [level])(expr) -``` - -**Аргументы** - -* `expr` — столбец с числовыми данными. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md). - -**Параметры** - -* `relative_accuracy` — относительная точность квантиля. Возможные значения — в диапазоне от 0 до 1. [Float](../../../sql-reference/data-types/float.md). Размер скетча зависит от диапазона данных и относительной точности: чем больше диапазон и чем меньше относительная точность, тем больше скетч. Примерный объём памяти для скетча — `log(max_value/min_value)/relative_accuracy`. Рекомендуемое значение — 0.001 или выше. - -* `level` — уровень квантиля. Необязательный параметр. Возможные значения — в диапазоне от 0 до 1. Значение по умолчанию: 0.5. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* Приблизительный квантиль указанного уровня. - -Тип: [Float64](/sql-reference/data-types/float). - -**Пример** - -Входная таблица содержит столбец целого типа и столбец вещественного типа: - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -Запрос для вычисления 0,75-квантили (третьего квартиля): - -```sql -SELECT quantileDD(0.01, 0.75)(a), quantileDD(0.01, 0.75)(b) FROM example_table; -``` - -Результат: - -```text -┌─quantileDD(0.01, 0.75)(a)─┬─quantileDD(0.01, 0.75)(b)─┐ -│ 2.974233423476717 │ 1.01 │ -└─────────────────────────────────┴─────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md deleted file mode 100644 index 32a7ba312e7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Вычисляет аппроксимацию квантили числовой последовательности данных.' -sidebar_position: 172 -slug: /sql-reference/aggregate-functions/reference/quantiledeterministic -title: 'quantileDeterministic' -doc_type: 'reference' ---- - -# quantileDeterministic {#quantiledeterministic} - -Вычисляет приближённый [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных. - -Эта функция использует [reservoir sampling](https://en.wikipedia.org/wiki/Reservoir_sampling) с размером резервуара до 8192 и детерминированным алгоритмом выборки. Результат детерминирован (воспроизводим). Чтобы получить точный квантиль, используйте функцию [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact). - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileDeterministic(level)(expr, determinator) -``` - -Псевдоним: `medianDeterministic`. - -**Аргументы** - -* `level` — Уровень квантиля. Необязательный параметр. Постоянное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). -* `determinator` — Число, хеш которого используется вместо генератора случайных чисел в алгоритме выборки из резервуара (reservoir sampling), чтобы сделать результат выборки детерминированным. В качестве `determinator` можно использовать любое детерминированное положительное число, например идентификатор пользователя или идентификатор события. Если одно и то же значение `determinator` встречается слишком часто, функция работает некорректно. - -**Возвращаемое значение** - -* Приблизительный квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Входная таблица: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -Запрос: - -```sql -SELECT quantileDeterministic(val, 1) FROM t -``` - -Результат: - -```text -┌─quantileDeterministic(val, 1)─┐ -│ 1.5 │ -└───────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md deleted file mode 100644 index b8535d77d0c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md +++ /dev/null @@ -1,293 +0,0 @@ ---- -description: 'функции quantileExact, quantileExactLow, quantileExactHigh, quantileExactExclusive, - quantileExactInclusive' -sidebar_position: 173 -slug: /sql-reference/aggregate-functions/reference/quantileexact -title: 'Функции quantileExact' -doc_type: 'reference' ---- - -# Функции quantileExact {#quantileexact-functions} - -## quantileExact {#quantileexact} - -Точно вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) для числовой последовательности данных. - -Чтобы получить точное значение, все переданные значения объединяются в массив, который затем частично сортируется. Поэтому функция потребляет `O(n)` памяти, где `n` — количество переданных значений. Однако для небольшого количества значений функция очень эффективна. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileExact(level)(expr) -``` - -Псевдоним: `medianExact`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — выражение над значениями столбца, результатом которого может быть числовой [тип данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* Для числовых типов данных формат результата будет таким же, как формат входных данных. Например: - -```sql - -SELECT - toTypeName(quantileExact(number)) AS `quantile`, - toTypeName(quantileExact(number::Int32)) AS `quantile_int32`, - toTypeName(quantileExact(number::Float32)) AS `quantile_float32`, - toTypeName(quantileExact(number::Float64)) AS `quantile_float64`, - toTypeName(quantileExact(number::Int64)) AS `quantile_int64` -FROM numbers(1) - ┌─quantile─┬─quantile_int32─┬─quantile_float32─┬─quantile_float64─┬─quantile_int64─┐ -1. │ UInt64 │ Int32 │ Float32 │ Float64 │ Int64 │ - └──────────┴────────────────┴──────────────────┴──────────────────┴────────────────┘ - -1 строка в наборе. Затрачено: 0.002 сек. -``` - -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantileExact(number) FROM numbers(10) -``` - -Результат: - -```text -┌─quantileExact(number)─┐ -│ 5 │ -└───────────────────────┘ -``` - -## quantileExactLow {#quantileexactlow} - -Аналогично `quantileExact`, вычисляет точный [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных. - -Чтобы получить точное значение, все переданные значения объединяются в массив, который затем полностью сортируется. Сложность [алгоритма сортировки](https://en.cppreference.com/w/cpp/algorithm/sort) составляет `O(N·log(N))`, где `N = std::distance(first, last)` сравнений. - -Возвращаемое значение зависит от уровня квантиля и количества элементов в выборке, то есть если уровень равен 0.5, функция возвращает нижнее значение медианы для чётного количества элементов и среднее значение медианы для нечётного количества элементов. Медиана вычисляется аналогично реализации [median_low](https://docs.python.org/3/library/statistics.html#statistics.median_low), используемой в Python. - -Для всех остальных уровней возвращается элемент с индексом, соответствующим значению `level * size_of_array`. Например: - -```sql -SELECT quantileExactLow(0.1)(number) FROM numbers(10) - -┌─quantileExactLow(0.1)(number)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе внутренние состояния функций не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](/sql-reference/aggregate-functions/reference/quantiles). - -**Синтаксис** - -```sql -quantileExactLow(level)(expr) -``` - -Синоним: `medianExactLow`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числовых типов данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantileExactLow(number) FROM numbers(10) -``` - -Результат: - -```text -┌─quantileExactLow(number)─┐ -│ 4 │ -└──────────────────────────┘ -``` - -## quantileExactHigh {#quantileexacthigh} - -Аналогично `quantileExact`, эта функция вычисляет точный [квантиль](https://en.wikipedia.org/wiki/Quantile) последовательности числовых данных. - -Все переданные значения объединяются в массив, который затем полностью сортируется для получения точного значения. Сложность [алгоритма сортировки](https://en.cppreference.com/w/cpp/algorithm/sort) составляет `O(N·log(N))`, где `N = std::distance(first, last)` сравнений. - -Возвращаемое значение зависит от уровня квантили и количества элементов в выборке, например, если уровень равен 0.5, то функция возвращает большее из двух медианных значений для чётного числа элементов и среднее (единственное) медианное значение для нечётного числа элементов. Медиана вычисляется аналогично реализации функции [median_high](https://docs.python.org/3/library/statistics.html#statistics.median_high), которая используется в Python. Для всех остальных уровней возвращается элемент по индексу, соответствующему значению `level * size_of_array`. - -Эта реализация ведёт себя точно так же, как текущая реализация функции `quantileExact`. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос выполняется менее эффективно, чем мог бы). В таком случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileExactHigh(level)(expr) -``` - -Синоним: `medianExactHigh`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — выражение над значениями столбца, результатом которого будут числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantileExactHigh(number) FROM numbers(10) -``` - -Результат: - -```text -┌─quantileExactHigh(number)─┐ -│ 5 │ -└───────────────────────────┘ -``` - -## quantileExactExclusive {#quantileexactexclusive} - -Точно вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) последовательности числовых данных. - -Чтобы получить точное значение, все переданные значения объединяются в массив, который затем частично сортируется. Поэтому функция потребляет `O(n)` памяти, где `n` — количество переданных значений. Однако для небольшого количества значений функция очень эффективна. - -Эта функция эквивалентна функции Excel [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) ([тип R6](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample)). - -При использовании нескольких функций `quantileExactExclusive` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantilesExactExclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactexclusive). - -**Синтаксис** - -```sql -quantileExactExclusive(level)(expr) -``` - -**Аргументы** - -* `expr` — выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Параметры** - -* `level` — уровень квантиля. Необязательный параметр. Допустимые значения: (0, 1) — границы не включены. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). Тип параметра: [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactExclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -Результат: - -```text -┌─quantileExactExclusive(0.6)(x)─┐ -│ 599.6 │ -└────────────────────────────────┘ -``` - -## quantileExactInclusive {#quantileexactinclusive} - -Точно вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных. - -Чтобы получить точное значение, все переданные значения объединяются в массив, который затем частично сортируется. Поэтому функция потребляет `O(n)` памяти, где `n` — количество переданных значений. Однако при небольшом количестве значений функция работает очень эффективно. - -Эта функция эквивалентна функции Excel [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed) ([тип R7](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample)). - -При использовании нескольких функций `quantileExactInclusive` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantilesExactInclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactinclusive). - -**Синтаксис** - -```sql -quantileExactInclusive(level)(expr) -``` - -**Аргументы** - -* `expr` — выражение над значениями столбца, которое возвращает числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Параметры** - -* `level` — уровень квантиля. Необязательный параметр. Возможные значения: [0, 1] — границы включены. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactInclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -Результат: - -```text -┌─quantileExactInclusive(0.6)(x)─┐ -│ 599.4 │ -└────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md deleted file mode 100644 index bda8f402a14..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: 'Точно вычисляет квантиль последовательности числовых данных с учетом веса каждого элемента.' -sidebar_position: 174 -slug: /sql-reference/aggregate-functions/reference/quantileexactweighted -title: 'quantileExactWeighted' -doc_type: 'reference' ---- - -# quantileExactWeighted {#quantileexactweighted} - -Точно вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) последовательности числовых данных с учётом веса каждого элемента. - -Для получения точного значения все переданные значения объединяются в массив, который затем частично сортируется. Каждое значение учитывается с его весом так, как если бы оно присутствовало `weight` раз. В алгоритме используется хеш-таблица. Благодаря этому при частом повторении переданных значений функция потребляет меньше ОЗУ, чем [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact). Вы можете использовать эту функцию вместо `quantileExact`, задав вес, равный 1. - -При использовании в запросе нескольких функций `quantile*` с разными уровнями квантилей их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileExactWeighted(level)(expr, weight) -``` - -Псевдоним: `medianExactWeighted`. - -**Аргументы** - -* `level` — Уровень квантиля. Необязательный параметр. Константа с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). -* `weight` — Столбец с весами элементов последовательности. Вес — это количество вхождений значения, представленное [беззнаковым целочисленным типом](../../../sql-reference/data-types/int-uint.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип возвращаемого значения: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Входная таблица: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -Запрос: - -```sql -SELECT quantileExactWeighted(n, val) FROM t -``` - -Результат: - -```text -┌─quantileExactWeighted(n, val)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md deleted file mode 100644 index 5a59904c256..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -description: 'Вычисляет квантиль последовательности числовых данных с использованием линейной интерполяции с учётом веса каждого элемента.' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated -title: 'quantileExactWeightedInterpolated' -doc_type: 'reference' ---- - -# quantileExactWeightedInterpolated {#quantileexactweightedinterpolated} - -Вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных с использованием линейной интерполяции с учётом веса каждого элемента. - -Для получения интерполированного значения все переданные значения объединяются в массив, который затем сортируется по соответствующим им весам. После этого квантиль интерполируется с помощью [метода взвешенного процентиля](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method): по весам строится кумулятивное распределение, затем выполняется линейная интерполяция, использующая веса и значения для вычисления квантилей. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В таком случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -Мы настоятельно рекомендуем использовать `quantileExactWeightedInterpolated` вместо `quantileInterpolatedWeighted`, поскольку `quantileExactWeightedInterpolated` обеспечивает более высокую точность, чем `quantileInterpolatedWeighted`. Ниже приведён пример: - -```sql -SELECT - quantileExactWeightedInterpolated(0.99)(number, 1), - quantile(0.99)(number), - quantileInterpolatedWeighted(0.99)(number, 1) -FROM numbers(9) -┌─quantileExactWeightedInterpolated(0.99)(number, 1)─┬─quantile(0.99)(number)─┬─quantileInterpolatedWeighted(0.99)(number, 1)─┐ -│ 7.92 │ 7.92 │ 8 │ -└────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────────────────────────┘ -``` - -**Синтаксис** - -```sql -quantileExactWeightedInterpolated(level)(expr, weight) -``` - -Псевдоним: `medianExactWeightedInterpolated`. - -**Аргументы** - -* `level` — Уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — Выражение над значениями столбца, результатом которого является числовой [тип данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). -* `weight` — Столбец с весами элементов последовательности. Вес — это количество вхождений значения, представленное [беззнаковым целочисленным типом](../../../sql-reference/data-types/int-uint.md). - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Входная таблица: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -Результат: - -```text -┌─quantileExactWeightedInterpolated(n, val)─┐ -│ 1.5 │ -└───────────────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md deleted file mode 100644 index da7d5773172..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Вычисляет квантиль последовательности числовых данных с использованием линейной интерполяции - с учетом веса каждого элемента.' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileInterpolatedWeighted -title: 'quantileInterpolatedWeighted' -doc_type: 'reference' ---- - -# quantileInterpolatedWeighted {#quantileinterpolatedweighted} - -Вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных с использованием линейной интерполяции с учётом веса каждого элемента. - -Для получения интерполированного значения все переданные значения объединяются в массив, который затем сортируется по соответствующим весам. Интерполяция квантилей затем выполняется с использованием [метода взвешенного перцентиля](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method) путём построения накопленного распределения на основе весов, после чего выполняется линейная интерполяция по весам и значениям для вычисления квантилей. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе их внутренние состояния не объединяются (то есть запрос выполняется менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileInterpolatedWeighted(level)(expr, weight) -``` - -Псевдоним: `medianInterpolatedWeighted`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константа с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). -* `weight` — столбец с весами элементов последовательности. Вес — это количество вхождений значения. - -**Возвращаемое значение** - -* Квантиль заданного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных во входных данных. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Входная таблица: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -Запрос: - -```sql -SELECT quantileInterpolatedWeighted(n, val) FROM t -``` - -Результат: - -```text -┌─quantileInterpolatedWeighted(n, val)─┐ -│ 1 │ -└──────────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md deleted file mode 100644 index 6d873f81084..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Вычисляет квантиль гистограммы с использованием линейной интерполяции.' -sidebar_position: 364 -slug: /sql-reference/aggregate-functions/reference/quantilePrometheusHistogram -title: 'quantilePrometheusHistogram' -doc_type: 'reference' ---- - -# quantilePrometheusHistogram {#quantileprometheushistogram} - -Вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) гистограммы с использованием линейной интерполяции с учётом накопленного значения и верхних границ каждого интервала (бакета) гистограммы. - -Для получения интерполированного значения все переданные значения объединяются в массив, который затем сортируется по соответствующим верхним границам бакетов. После этого интерполяция квантили выполняется аналогично функции PromQL [histogram_quantile()](https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile) для классической гистограммы: выполняется линейная интерполяция, используя нижнюю и верхнюю границы бакета, в котором находится позиция квантили. - -**Синтаксис** - -```sql -quantilePrometheusHistogram(level)(bucket_upper_bound, cumulative_bucket_value) -``` - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: `0.5`. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). - -* `bucket_upper_bound` — верхние границы бакетов гистограммы. - - * Верхний (последний) бакет должен иметь верхнюю границу `+Inf`. - -* `cumulative_bucket_value` — накопительные значения типа [UInt](../../../sql-reference/data-types/int-uint) или [Float64](../../../sql-reference/data-types/float.md) для бакетов гистограммы. - - * Значения должны монотонно возрастать по мере увеличения верхней границы бакета. - -**Возвращаемое значение** - -* Квантиль заданного уровня. - -Тип: - -* `Float64`. - -**Пример** - -Входная таблица: - -```text - ┌─bucket_upper_bound─┬─cumulative_bucket_value─┐ -1. │ 0 │ 6 │ -2. │ 0.5 │ 11 │ -3. │ 1 │ 14 │ -4. │ inf │ 19 │ - └────────────────────┴─────────────────────────┘ -``` - -Результат: - -```text - ┌─quantilePrometheusHistogram(bucket_upper_bound, cumulative_bucket_value)─┐ -1. │ 0.35 │ - └──────────────────────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md deleted file mode 100644 index ed72fb4c009..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -description: 'quantiles, quantilesExactExclusive, quantilesExactInclusive, quantilesGK' -sidebar_position: 177 -slug: /sql-reference/aggregate-functions/reference/quantiles -title: 'Функции quantiles' -doc_type: 'reference' ---- - - - -# Функции квантилей {#quantiles-functions} - - - -## quantiles {#quantiles} - -Синтаксис: `quantiles(level1, level2, ...)(x)` - -Для всех функций квантилей существуют соответствующие функции `quantiles`: `quantiles`, `quantilesDeterministic`, `quantilesTiming`, `quantilesTimingWeighted`, `quantilesExact`, `quantilesExactWeighted`, `quantileExactWeightedInterpolated`, `quantileInterpolatedWeighted`, `quantilesTDigest`, `quantilesBFloat16`, `quantilesDD`. Эти функции вычисляют все квантили перечисленных уровней за один проход и возвращают массив полученных значений. - - - -## quantilesExactExclusive {#quantilesexactexclusive} - -Точно вычисляет [квантили](https://en.wikipedia.org/wiki/Quantile) последовательности числовых данных. - -Чтобы получить точное значение, все переданные значения объединяются в массив, который затем частично сортируется. Поэтому функция потребляет память объёмом `O(n)`, где `n` — количество переданных значений. Однако для небольшого количества значений функция работает очень эффективно. - -Эта функция эквивалентна функции Excel [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) (тип R6, см. [описание](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample)). - -Работает эффективнее с наборами уровней, чем [quantileExactExclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactexclusive). - -**Синтаксис** - -```sql -quantilesExactExclusive(level1, level2, ...)(expr) -``` - -**Аргументы** - -* `expr` — Выражение над значениями столбца, результатом которого могут быть числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Параметры** - -* `level` — Уровни квантилей. Возможные значения: (0, 1) — границы не включаются. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* [Array](../../../sql-reference/data-types/array.md) квантилей для указанных уровней. - -Тип значений массива: - -* [Float64](../../../sql-reference/data-types/float.md) для входных данных числового типа. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -Результат: - -```text -┌─quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.25,499.5,749.75,899.9,949.9499999999999,989.99,998.999] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesExactInclusive {#quantilesexactinclusive} - -Точно вычисляет [квантили](https://en.wikipedia.org/wiki/Quantile) для числовой последовательности данных. - -Для получения точного значения все переданные значения объединяются в массив, который затем частично сортируется. Поэтому функция использует `O(n)` памяти, где `n` — количество переданных значений. Однако при небольшом количестве значений функция работает очень эффективно. - -Эта функция эквивалентна функции Excel [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed) ([тип R7](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample)). - -Работает эффективнее с наборами уровней, чем [quantileExactInclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactinclusive). - -**Синтаксис** - -```sql -quantilesExactInclusive(level1, level2, ...)(expr) -``` - -**Аргументы** - -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Параметры** - -* `level` — Уровни квантилей. Возможные значения: [0, 1] — границы включены. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемое значение** - -* [Array](../../../sql-reference/data-types/array.md) квантилей заданных уровней. - -Тип значений массива: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных во входном выражении. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -Результат: - -```text -┌─quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.75,499.5,749.25,899.1,949.05,989.01,998.001] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesGK {#quantilesgk} - -`quantilesGK` работает аналогично функции `quantileGK`, но позволяет вычислять квантили для нескольких уровней одновременно и возвращает массив. - -**Синтаксис** - -```sql -quantilesGK(accuracy, level1, level2, ...)(expr) -``` - -**Возвращаемое значение** - -* [Array](../../../sql-reference/data-types/array.md) квантилей заданных уровней. - -Тип значений массива: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantilesGK(1, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [1,1,1] │ -└──────────────────────────────────────────────────┘ - -SELECT quantilesGK(10, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(10, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [156,413,659] │ -└───────────────────────────────────────────────────┘ -SELECT quantilesGK(100, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(100, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [251,498,741] │ -└────────────────────────────────────────────────────┘ - -SELECT quantilesGK(1000, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1000, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [249,499,749] │ -└─────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md deleted file mode 100644 index b95d5c430a0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: 'Вычисляет приблизительный квантиль последовательности числовых данных с использованием алгоритма t-digest.' -sidebar_position: 178 -slug: /sql-reference/aggregate-functions/reference/quantiletdigest -title: 'quantileTDigest' -doc_type: 'reference' ---- - -# quantileTDigest {#quantiletdigest} - -Вычисляет приближённый [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных с использованием алгоритма [t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf). - -Потребление памяти составляет `log(n)`, где `n` — количество значений. Результат зависит от порядка выполнения запроса и является недетерминированным. - -Производительность функции ниже, чем у функций [quantile](/sql-reference/aggregate-functions/reference/quantile) или [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming). Если говорить о соотношении размеров состояния и точности, эта функция существенно лучше, чем `quantile`. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе внутренние состояния не комбинируются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileTDigest(level)(expr) -``` - -Псевдоним: `medianTDigest`. - -**Аргументы** - -* `level` — Уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). - -**Возвращаемое значение** - -* Приближённый квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantileTDigest(number) FROM numbers(10) -``` - -Результат: - -```text -┌─quantileTDigest(number)─┐ -│ 4.5 │ -└─────────────────────────┘ -``` - -**Смотрите также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md deleted file mode 100644 index 85c5eca3a2d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 'Вычисляет приближённый квантиль последовательности числовых данных с использованием алгоритма t-digest.' -sidebar_position: 179 -slug: /sql-reference/aggregate-functions/reference/quantiletdigestweighted -title: 'quantileTDigestWeighted' -doc_type: 'reference' ---- - -# quantileTDigestWeighted {#quantiletdigestweighted} - -Вычисляет приближённый [квантиль](https://en.wikipedia.org/wiki/Quantile) последовательности числовых данных с использованием алгоритма [t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf). Функция учитывает вес каждого элемента последовательности. Максимальная погрешность — 1%. Потребление памяти — `log(n)`, где `n` — количество значений. - -Производительность функции ниже, чем у [quantile](/sql-reference/aggregate-functions/reference/quantile) или [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming). С точки зрения соотношения размера State к точности эта функция значительно лучше, чем `quantile`. - -Результат зависит от порядка выполнения запроса и является недетерминированным. - -При использовании нескольких функций `quantile*` с разными уровнями в одном запросе внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -:::note\ -Использование `quantileTDigestWeighted` [не рекомендуется для очень маленьких наборов данных](https://github.com/tdunning/t-digest/issues/167#issuecomment-828650275) и может приводить к значительной погрешности. В этом случае рассмотрите возможность использования [`quantileTDigest`](../../../sql-reference/aggregate-functions/reference/quantiletdigest.md) вместо неё. -::: - -**Синтаксис** - -```sql -quantileTDigestWeighted(level)(expr, weight) -``` - -Псевдоним: `medianTDigestWeighted`. - -**Аргументы** - -* `level` — Уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Рекомендуем использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). -* `expr` — Выражение над значениями столбца, результатом которого являются числовые [типы данных](/sql-reference/data-types), [Date](../../../sql-reference/data-types/date.md) или [DateTime](../../../sql-reference/data-types/datetime.md). -* `weight` — Столбец с весами элементов последовательности. Вес — это количество вхождений значения. - -**Возвращаемое значение** - -* Приближённый квантиль указанного уровня. - -Тип: - -* [Float64](../../../sql-reference/data-types/float.md) для числового типа данных на входе. -* [Date](../../../sql-reference/data-types/date.md), если входные значения имеют тип `Date`. -* [DateTime](../../../sql-reference/data-types/datetime.md), если входные значения имеют тип `DateTime`. - -**Пример** - -Запрос: - -```sql -SELECT quantileTDigestWeighted(number, 1) FROM numbers(10) -``` - -Результат: - -```text -┌─quantileTDigestWeighted(number, 1)─┐ -│ 4.5 │ -└────────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md deleted file mode 100644 index 485768de4b9..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'С заданной точностью вычисляет квантиль для числовой последовательности данных.' -sidebar_position: 180 -slug: /sql-reference/aggregate-functions/reference/quantiletiming -title: 'quantileTiming' -doc_type: 'reference' ---- - -# quantileTiming {#quantiletiming} - -С заданной точностью вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных. - -Результат детерминирован (не зависит от порядка выполнения запроса). Функция оптимизирована для работы с последовательностями, описывающими распределения, например временем загрузки веб‑страниц или временем ответа бэкенда. - -При использовании в одном запросе нескольких функций `quantile*` с разными уровнями квантилей их внутренние состояния не комбинируются (то есть запрос выполняется менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileTiming(level)(expr) -``` - -Псевдоним: `medianTiming`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Постоянное число с плавающей точкой от 0 до 1. Рекомендуется использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). - -* `expr` — [выражение](/sql-reference/syntax#expressions) над значениями столбца, возвращающее число типа [Float*](../../../sql-reference/data-types/float.md). - - * Если в функцию передаются отрицательные значения, поведение не определено. - * Если значение больше 30 000 (время загрузки страницы более 30 секунд), предполагается, что оно равно 30 000. - -**Точность** - -Вычисление является точным, если: - -* Общее количество значений не превышает 5670. -* Общее количество значений превышает 5670, но время загрузки страницы меньше 1024 мс. - -В противном случае результат вычисления округляется до ближайшего числа, кратного 16 мс. - -:::note\ -Для вычисления квантилей времени загрузки страниц эта функция более эффективна и точна, чем [quantile](/sql-reference/aggregate-functions/reference/quantile). -::: - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: `Float32`. - -:::note\ -Если в функцию не переданы значения (при использовании `quantileTimingIf`), возвращается [NaN](/sql-reference/data-types/float#nan-and-inf). Это сделано для того, чтобы отличать такие случаи от случаев, которые дают нулевой результат. См. раздел [ORDER BY clause](/sql-reference/statements/select/order-by) для примечаний по сортировке значений `NaN`. -::: - -**Пример** - -Исходная таблица: - -```text -┌─response_time─┐ -│ 72 │ -│ 112 │ -│ 126 │ -│ 145 │ -│ 104 │ -│ 242 │ -│ 313 │ -│ 168 │ -│ 108 │ -└───────────────┘ -``` - -Запрос: - -```sql -SELECT quantileTiming(response_time) FROM t -``` - -Результат: - -```text -┌─quantileTiming(response_time)─┐ -│ 126 │ -└───────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md deleted file mode 100644 index 90504928a3c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -description: 'С заданной точностью вычисляет квантиль числовой последовательности данных - с учётом веса каждого её элемента.' -sidebar_position: 181 -slug: /sql-reference/aggregate-functions/reference/quantiletimingweighted -title: 'quantileTimingWeighted' -doc_type: 'reference' ---- - -# quantileTimingWeighted {#quantiletimingweighted} - -С заданной точностью вычисляет [квантиль](https://en.wikipedia.org/wiki/Quantile) числовой последовательности данных с учетом веса каждого элемента последовательности. - -Результат детерминирован (не зависит от порядка обработки запроса). Функция оптимизирована для работы с последовательностями, описывающими распределения, такие как время загрузки веб-страниц или время ответа бэкенда. - -При использовании в одном запросе нескольких функций `quantile*` с различными уровнями их внутренние состояния не объединяются (то есть запрос работает менее эффективно, чем мог бы). В этом случае используйте функцию [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles). - -**Синтаксис** - -```sql -quantileTimingWeighted(level)(expr, weight) -``` - -Псевдоним: `medianTimingWeighted`. - -**Аргументы** - -* `level` — уровень квантиля. Необязательный параметр. Константное число с плавающей запятой от 0 до 1. Мы рекомендуем использовать значение `level` в диапазоне `[0.01, 0.99]`. Значение по умолчанию: 0.5. При `level=0.5` функция вычисляет [медиану](https://en.wikipedia.org/wiki/Median). - -* `expr` — [выражение](/sql-reference/syntax#expressions) над значениями столбца, возвращающее число типа [Float*](../../../sql-reference/data-types/float.md). - - * Если в функцию передаются отрицательные значения, поведение не определено. - * Если значение больше 30 000 (время загрузки страницы более 30 секунд), оно приравнивается к 30 000. - -* `weight` — столбец с весами элементов последовательности. Вес — это количество вхождений значения. - -**Точность** - -Результат вычислений точен, если: - -* общее количество значений не превышает 5670; -* общее количество значений превышает 5670, но время загрузки страницы меньше 1024 мс. - -В противном случае результат вычисления округляется до ближайшего кратного 16 мс. - -:::note\ -Для вычисления квантилей времени загрузки страницы эта функция более эффективна и точна, чем [quantile](/sql-reference/aggregate-functions/reference/quantile). -::: - -**Возвращаемое значение** - -* Квантиль указанного уровня. - -Тип: `Float32`. - -:::note\ -Если в функцию не переданы значения (при использовании `quantileTimingIf`), возвращается [NaN](/sql-reference/data-types/float#nan-and-inf). Это позволяет отличать такие случаи от случаев, когда результатом является ноль. См. примечания о сортировке значений `NaN` в разделе [ORDER BY](/sql-reference/statements/select/order-by). -::: - -**Пример** - -Исходная таблица: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -Запрос: - -```sql -SELECT quantileTimingWeighted(response_time, weight) FROM t -``` - -Результат: - -```text -┌─quantileTimingWeighted(response_time, weight)─┐ -│ 112 │ -└───────────────────────────────────────────────┘ -``` - -# quantilesTimingWeighted {#quantilestimingweighted} - -То же, что и `quantileTimingWeighted`, но принимает несколько аргументов уровней квантилей и возвращает массив, содержащий значения этих квантилей. - -**Пример** - -Входная таблица: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -Запрос: - -```sql -SELECT quantilesTimingWeighted(0,5, 0.99)(response_time, weight) FROM t -``` - -Результат: - -```text -┌─quantilesTimingWeighted(0.5, 0.99)(response_time, weight)─┐ -│ [112,162] │ -└───────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md deleted file mode 100644 index fdcacc1215e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'Вычисляет коэффициент ранговой корреляции.' -sidebar_position: 182 -slug: /sql-reference/aggregate-functions/reference/rankCorr -title: 'rankCorr' -doc_type: 'reference' ---- - -# rankCorr {#rankcorr} - -Вычисляет коэффициент ранговой корреляции. - -**Синтаксис** - -```sql -rankCorr(x, y) -``` - -**Аргументы** - -* `x` — Произвольное значение. [Float32](/sql-reference/data-types/float) или [Float64](/sql-reference/data-types/float). -* `y` — Произвольное значение. [Float32](/sql-reference/data-types/float) или [Float64](/sql-reference/data-types/float). - -**Возвращаемое значение** - -* Возвращает коэффициент ранговой корреляции для рангов `x` и `y`. Значение коэффициента корреляции лежит в диапазоне от -1 до +1. Если передано менее двух аргументов, функция возвращает исключение. Значение, близкое к +1, обозначает сильную линейную зависимость: с увеличением одной случайной величины вторая случайная величина также увеличивается. Значение, близкое к -1, обозначает сильную линейную зависимость: с увеличением одной случайной величины вторая случайная величина уменьшается. Значение, близкое или равное 0, обозначает отсутствие зависимости между двумя случайными величинами. - -Тип: [Float64](/sql-reference/data-types/float). - -**Пример** - -Запрос: - -```sql -SELECT rankCorr(number, number) FROM numbers(100); -``` - -Результат: - -```text -┌─rankCorr(number, number)─┐ -│ 1 │ -└──────────────────────────┘ -``` - -Запрос: - -```sql -SELECT roundBankers(rankCorr(exp(number), sin(number)), 3) FROM numbers(100); -``` - -Результат: - -```text -┌─roundBankers(rankCorr(exp(number), sin(number)), 3)─┐ -│ -0.037 │ -└─────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [Коэффициент ранговой корреляции Спирмена](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md deleted file mode 100644 index 08d2756485c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: 'Выполняет простую (одномерную) линейную регрессию.' -sidebar_position: 183 -slug: /sql-reference/aggregate-functions/reference/simplelinearregression -title: 'simpleLinearRegression' -doc_type: 'reference' ---- - -# simpleLinearRegression {#simplelinearregression} - -Выполняет простую (одномерную) линейную регрессию. - -```sql -simpleLinearRegression(x, y) -``` - -Параметры: - -* `x` — столбец, содержащий значения объясняющей переменной. -* `y` — столбец, содержащий значения зависимой переменной. - -Возвращаемые значения: - -Константы `(k, b)` результирующей прямой `y = k*x + b`. - -**Примеры** - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3])─┐ -│ (1,0) │ -└───────────────────────────────────────────────────────────────────┘ -``` - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6])─┐ -│ (1,3) │ -└───────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md deleted file mode 100644 index 2e1652158ca..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: 'Агрегатная функция `singleValueOrNull` используется для реализации операторов подзапросов, таких как `x = ALL (SELECT ...)`. Она проверяет, есть ли в данных ровно одно уникальное значение, отличное от NULL.' -sidebar_position: 184 -slug: /sql-reference/aggregate-functions/reference/singlevalueornull -title: 'singleValueOrNull' -doc_type: 'reference' ---- - -# singleValueOrNull {#singlevalueornull} - -Агрегатная функция `singleValueOrNull` используется для реализации операторов с подзапросами, таких как `x = ALL (SELECT ...)`. Она проверяет, есть ли в наборе данных только одно уникальное значение, отличное от NULL. -Если есть ровно одно уникальное значение, функция возвращает его. Если уникальных значений ноль или как минимум два, функция возвращает NULL. - -**Синтаксис** - -```sql -singleValueOrNull(x) -``` - -**Параметры** - -* `x` — столбец любого [типа данных](../../data-types/index.md) (кроме [Map](../../data-types/map.md), [Array](../../data-types/array.md) или [Tuple](../../data-types/tuple), которые не могут иметь тип [Nullable](../../data-types/nullable.md)). - -**Возвращаемые значения** - -* Уникальное значение, если в `x` есть ровно одно уникальное значение, отличное от `NULL`. -* `NULL`, если нет ни одного значения или имеется как минимум два различных значения. - -**Примеры** - -Запрос: - -```sql -CREATE TABLE test (x UInt8 NULL) ENGINE=Log; -INSERT INTO test (x) VALUES (NULL), (NULL), (5), (NULL), (NULL); -SELECT singleValueOrNull(x) FROM test; -``` - -Результат: - -```response -┌─singleValueOrNull(x)─┐ -│ 5 │ -└──────────────────────┘ -``` - -Запрос: - -```sql -INSERT INTO test (x) VALUES (10); -SELECT singleValueOrNull(x) FROM test; -``` - -Результат: - -```response -┌─singleValueOrNull(x)─┐ -│ ᴺᵁᴸᴸ │ -└──────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md deleted file mode 100644 index d9660506831..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: 'Вычисляет коэффициент асимметрии распределения последовательности.' -sidebar_position: 185 -slug: /sql-reference/aggregate-functions/reference/skewpop -title: 'skewPop' -doc_type: 'reference' ---- - -# skewPop {#skewpop} - -Вычисляет [коэффициент асимметрии](https://en.wikipedia.org/wiki/Skewness) последовательности значений. - -```sql -skewPop(expr) -``` - -**Аргументы** - -`expr` — [выражение](/sql-reference/syntax#expressions), возвращающее число. - -**Возвращаемое значение** - -Асимметрия заданного распределения. Тип — [Float64](../../../sql-reference/data-types/float.md) - -**Пример** - -```sql -SELECT skewPop(value) FROM series_with_value_column; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md deleted file mode 100644 index 4fec08c0f9d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: 'Вычисляет выборочную асимметрию последовательности.' -sidebar_position: 186 -slug: /sql-reference/aggregate-functions/reference/skewsamp -title: 'skewSamp' -doc_type: 'reference' ---- - -# skewSamp {#skewsamp} - -Вычисляет [выборочную асимметрию](https://en.wikipedia.org/wiki/Skewness) последовательности. - -Представляет собой несмещённую оценку асимметрии случайной величины, если переданные значения образуют её выборку. - -```sql -skewSamp(expr) -``` - -**Аргументы** - -`expr` — [выражение](/sql-reference/syntax#expressions), возвращающее число. - -**Возвращаемое значение** - -Асимметрия заданного распределения. Тип — [Float64](../../../sql-reference/data-types/float.md). Если `n <= 1` (`n` — размер выборки), функция возвращает `nan`. - -**Пример** - -```sql -SELECT skewSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md deleted file mode 100644 index 0bb0941a86b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: 'Функция строит частотную гистограмму значений `x` с частотой их повторения - `y` на интервале `[min_x, max_x]`.' -sidebar_label: 'sparkbar' -sidebar_position: 187 -slug: /sql-reference/aggregate-functions/reference/sparkbar -title: 'sparkbar' -doc_type: 'reference' ---- - -# sparkbar {#sparkbar} - -Функция строит гистограмму частот для значений `x` и частоты повторений `y` этих значений на интервале `[min_x, max_x]`. -Повторения для всех `x`, попадающих в один и тот же бакет, усредняются, поэтому данные должны быть предварительно агрегированы. -Отрицательные значения частоты повторений игнорируются. - -Если интервал не указан, минимальное `x` используется как начало интервала, а максимальное `x` — как конец интервала. -В противном случае значения вне интервала игнорируются. - -**Синтаксис** - -```sql -sparkbar(buckets[, min_x, max_x])(x, y) -``` - -**Параметры** - -* `buckets` — Количество сегментов. Тип: [Integer](../../../sql-reference/data-types/int-uint.md). -* `min_x` — Начало интервала. Необязательный параметр. -* `max_x` — Конец интервала. Необязательный параметр. - -**Аргументы** - -* `x` — Поле со значениями. -* `y` — Поле с частотами значений. - -**Возвращаемое значение** - -* Гистограмма частот. - -**Пример** - -Запрос: - -```sql -CREATE TABLE spark_bar_data (`value` Int64, `event_date` Date) ENGINE = MergeTree ORDER BY event_date; - -INSERT INTO spark_bar_data VALUES (1,'2020-01-01'), (3,'2020-01-02'), (4,'2020-01-02'), (-3,'2020-01-02'), (5,'2020-01-03'), (2,'2020-01-04'), (3,'2020-01-05'), (7,'2020-01-06'), (6,'2020-01-07'), (8,'2020-01-08'), (2,'2020-01-11'); - -SELECT sparkbar(9)(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); - -SELECT sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); -``` - -Результат: - -```text -┌─sparkbar(9)(event_date, cnt)─┐ -│ ▂▅▂▃▆█ ▂ │ -└──────────────────────────────┘ - -┌─sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date, cnt)─┐ -│ ▂▅▂▃▇▆█ │ -└──────────────────────────────────────────────────────────────────────────┘ -``` - -Псевдоним этой функции — sparkBar. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md deleted file mode 100644 index 51a477f57cc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Результат равен квадратному корню из значения varPop.' -sidebar_position: 188 -slug: /sql-reference/aggregate-functions/reference/stddevpop -title: 'stddevPop' -doc_type: 'reference' ---- - -# stddevPop {#stddevpop} - -Результат равен квадратному корню из [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md). - -Псевдонимы: `STD`, `STDDEV_POP`. - -:::note -Эта функция использует численно нестабильный алгоритм. Если вам нужна [численная устойчивость](https://en.wikipedia.org/wiki/Numerical_stability) при вычислениях, используйте функцию [`stddevPopStable`](../reference/stddevpopstable.md). Она работает медленнее, но обеспечивает меньшую вычислительную погрешность. -::: - -**Синтаксис** - -```sql -stddevPop(x) -``` - -**Параметры** - -* `x`: Набор значений, для которого вычисляется стандартное отклонение. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Квадратный корень из стандартного отклонения `x`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevPop(population) AS stddev -FROM test_data; -``` - -Результат: - -```response -┌────────────stddev─┐ -│ 3.794733192202055 │ -└───────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md deleted file mode 100644 index c953e18e33b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'Результат равен квадратному корню из varPop. В отличие от stddevPop, - эта функция использует численно устойчивый алгоритм.' -sidebar_position: 189 -slug: /sql-reference/aggregate-functions/reference/stddevpopstable -title: 'stddevPopStable' -doc_type: 'reference' ---- - -# stddevPopStable {#stddevpopstable} - -Результат равен квадратному корню из [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md). В отличие от [`stddevPop`](../reference/stddevpop.md), эта функция использует численно устойчивый алгоритм. Она работает медленнее, но обеспечивает меньшую вычислительную погрешность. - -**Синтаксис** - -```sql -stddevPopStable(x) -``` - -**Параметры** - -* `x`: Набор значений, для которого вычисляется стандартное отклонение. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -Квадратный корень из дисперсии значений `x`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population Float64, -) -ENGINE = Log; - -INSERT INTO test_data SELECT randUniform(5.5, 10) FROM numbers(1000000) - -SELECT - stddevPopStable(population) AS stddev -FROM test_data; -``` - -Результат: - -```response -┌─────────────stddev─┐ -│ 1.2999977786592576 │ -└────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md deleted file mode 100644 index eecdd490d5f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: 'Результат равен квадратному корню результата функции varSamp' -sidebar_position: 190 -slug: /sql-reference/aggregate-functions/reference/stddevsamp -title: 'stddevSamp' -doc_type: 'reference' ---- - -# stddevSamp {#stddevsamp} - -Результат равен квадратному корню из [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md). - -Псевдоним: `STDDEV_SAMP`. - -:::note -Эта функция использует численно неустойчивый алгоритм. Если вам важна [численная устойчивость](https://en.wikipedia.org/wiki/Numerical_stability) вычислений, используйте функцию [`stddevSampStable`](../reference/stddevsampstable.md). Она работает медленнее, но обеспечивает меньшую вычислительную погрешность. -::: - -**Синтаксис** - -```sql -stddevSamp(x) -``` - -**Параметры** - -* `x`: Значения, для которых вычисляется квадратный корень выборочной дисперсии. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -Квадратный корень выборочной дисперсии `x`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSamp(population) -FROM test_data; -``` - -Результат: - -```response -┌─stddevSamp(population)─┐ -│ 4 │ -└────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md deleted file mode 100644 index 4f0208c5429..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: 'Результат равен квадратному корню значения, вычисленного функцией varSamp. В отличие от varSamp, эта функция использует численно устойчивый алгоритм.' -sidebar_position: 191 -slug: /sql-reference/aggregate-functions/reference/stddevsampstable -title: 'stddevSampStable' -doc_type: 'reference' ---- - -# stddevSampStable {#stddevsampstable} - -Результат равен квадратному корню из [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md). В отличие от [`stddevSamp`](../reference/stddevsamp.md) эта функция использует численно устойчивый алгоритм. Она работает медленнее, но даёт меньшую вычислительную погрешность. - -**Синтаксис** - -```sql -stddevSampStable(x) -``` - -**Параметры** - -* `x`: Значения, для которых вычисляется квадратный корень из выборочной дисперсии. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -Квадратный корень из выборочной дисперсии значений `x`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSampStable(population) -FROM test_data; -``` - -Результат: - -```response -┌─stddevSampStable(population)─┐ -│ 4 │ -└──────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md deleted file mode 100644 index 22682750676..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: 'Эта функция реализует стохастическую линейную регрессию. Поддерживает пользовательские параметры для скорости обучения, коэффициента L2-регуляризации, размера мини-батча и несколько методов обновления весов (Adam, простой SGD, Momentum, Nesterov).' -sidebar_position: 192 -slug: /sql-reference/aggregate-functions/reference/stochasticlinearregression -title: 'stochasticLinearRegression' -doc_type: 'reference' ---- - -# stochasticLinearRegression {#agg_functions_stochasticlinearregression_parameters} - -Эта функция реализует стохастическую линейную регрессию. Она поддерживает настройку скорости обучения, коэффициента L2-регуляризации, размера мини-батча, а также несколько методов обновления весов ([Adam](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam) (используется по умолчанию), [simple SGD](https://en.wikipedia.org/wiki/Stochastic_gradient_descent), [Momentum](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum) и [Nesterov](https://mipt.ru/upload/medialibrary/d7e/41-91.pdf)). - -### Параметры {#parameters} - -Доступно 4 настраиваемых параметра. Они передаются функции последовательно, но нет необходимости передавать все четыре — будут использованы значения по умолчанию; однако для построения качественной модели обычно требуется дополнительная настройка параметров. - -```text -stochasticLinearRegression(0.00001, 0.1, 15, 'Adam') -``` - -1. `learning rate` — коэффициент, определяющий длину шага при выполнении шага градиентного спуска. Слишком большое значение может привести к стремлению весов модели к бесконечности. Значение по умолчанию — `0.00001`. -2. `l2 regularization coefficient` — коэффициент L2‑регуляризации, который может помочь предотвратить переобучение. Значение по умолчанию — `0.1`. -3. `mini-batch size` задаёт число элементов, по которым вычисляются и суммируются градиенты для выполнения одного шага градиентного спуска. Чистый стохастический спуск использует один элемент, однако использование небольших батчей (порядка 10 элементов) делает шаги градиентного спуска более стабильными. Значение по умолчанию — `15`. -4. `method for updating weights` — метод обновления весов; доступны: `Adam` (по умолчанию), `SGD`, `Momentum` и `Nesterov`. `Momentum` и `Nesterov` требуют немного больше вычислений и памяти, однако они оказываются полезны с точки зрения скорости сходимости и устойчивости стохастических градиентных методов. - -### Использование {#usage} - -`stochasticLinearRegression` используется в два этапа: обучение модели и предсказание на новых данных. Чтобы обучить модель и сохранить её состояние для последующего использования, применяется комбинатор `-State`, который сохраняет состояние (например, веса модели). -Для предсказаний используется функция [evalMLMethod](/sql-reference/functions/machine-learning-functions#evalmlmethod), которая принимает состояние как аргумент, а также признаки, по которым нужно сделать предсказание. - -
- -**1.** Обучение - -Можно использовать следующий запрос. - -```sql -CREATE TABLE IF NOT EXISTS train_data -( - param1 Float64, - param2 Float64, - target Float64 -) ENGINE = Memory; - -CREATE TABLE your_model ENGINE = Memory AS SELECT -stochasticLinearRegressionState(0.1, 0.0, 5, 'SGD')(target, param1, param2) -AS state FROM train_data; -``` - -Здесь нам также необходимо вставить данные в таблицу `train_data`. Количество параметров не фиксировано: оно зависит только от числа аргументов, переданных в `linearRegressionState`. Все они должны быть числовыми значениями. -Обратите внимание, что столбец с целевым значением (которое мы хотим научиться предсказывать) передаётся в качестве первого аргумента. - -**2.** Предсказание - -После сохранения состояния в таблицу мы можем использовать его многократно для предсказаний или даже объединять с другими состояниями и создавать новые, ещё более качественные модели. - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -Запрос вернёт столбец предсказанных значений. Обратите внимание, что первый аргумент `evalMLMethod` — это объект `AggregateFunctionState`, далее указаны столбцы с признаками. - -`test_data` — это таблица, аналогичная `train_data`, но она может не содержать целевого значения. - -### Примечания {#notes} - -1. Чтобы объединить две модели, пользователь может выполнить такой запрос: - `sql SELECT state1 + state2 FROM your_models` - где таблица `your_models` содержит обе модели. Этот запрос вернёт новый объект `AggregateFunctionState`. - -2. Пользователь может получить веса созданной модели для собственных целей, не сохраняя модель, если не используется комбинатор `-State`. - `sql SELECT stochasticLinearRegression(0.01)(target, param1, param2) FROM train_data` - Такой запрос обучит модель и вернёт её веса: сначала идут веса, соответствующие параметрам модели, последний — смещение (bias). Поэтому в приведённом выше примере запрос вернёт столбец из трёх значений. - -**См. также** - -* [stochasticLogisticRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [Различие между линейной и логистической регрессией](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md deleted file mode 100644 index 7744faf8d6e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: 'Эта функция реализует стохастическую логистическую регрессию. Она может использоваться для задач бинарной классификации, поддерживает те же настраиваемые параметры, что и stochasticLinearRegression, и работает аналогичным образом.' -sidebar_position: 193 -slug: /sql-reference/aggregate-functions/reference/stochasticlogisticregression -title: 'stochasticLogisticRegression' -doc_type: 'reference' ---- - -# stochasticLogisticRegression {#stochasticlogisticregression} - -Эта функция реализует стохастическую логистическую регрессию. Ее можно использовать для решения задачи бинарной классификации; она поддерживает те же настраиваемые параметры, что и stochasticLinearRegression, и работает аналогичным образом. - -### Параметры {#parameters} - -Параметры полностью совпадают с параметрами в stochasticLinearRegression: -`learning rate`, `l2 regularization coefficient`, `mini-batch size`, `method for updating weights`. -Для получения дополнительной информации см. раздел [параметры](../reference/stochasticlinearregression.md/#parameters). - -```text -stochasticLogisticRegression(1.0, 1.0, 10, 'SGD') -``` - -**1.** Подгонка - -{/* */ } - -См. раздел `Fitting` в описании функции [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlinearregression). - -Предсказанные метки должны находиться в диапазоне [-1, 1]. - -**2.** Прогнозирование - -{/* */ } - -Используя сохранённое состояние, мы можем предсказать вероятность того, что объект будет иметь метку `1`. - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -Запрос вернёт столбец вероятностей. Обратите внимание, что первый аргумент функции `evalMLMethod` — объект `AggregateFunctionState`, а затем идут столбцы признаков. - -Мы также можем задать порог вероятности, который распределяет элементы по разным меткам. - -```sql -SELECT ans < 1.1 AND ans > 0.5 FROM -(WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) AS ans FROM test_data) -``` - -В результате будут получены метки. - -`test_data` — это таблица, аналогичная `train_data`, но в ней может не быть целевого значения. - -**См. также** - -* [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [Разница между линейной и логистической регрессиями.](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md deleted file mode 100644 index c924eaa126a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'Применяет t-критерий Стьюдента к выборкам из двух генеральных совокупностей.' -sidebar_label: 'studentTTest' -sidebar_position: 194 -slug: /sql-reference/aggregate-functions/reference/studentttest -title: 'studentTTest' -doc_type: 'reference' ---- - -# studentTTest {#studentttest} - -Применяет t-критерий Стьюдента к выборкам из двух генеральных совокупностей. - -**Синтаксис** - -```sql -studentTTest([confidence_level])(sample_data, sample_index) -``` - -Значения обеих выборок находятся в столбце `sample_data`. Если `sample_index` равен 0, то значение в этой строке относится к выборке из первой генеральной совокупности. В противном случае оно относится к выборке из второй генеральной совокупности. -Нулевая гипотеза заключается в том, что средние значения генеральных совокупностей равны. Предполагается нормальное распределение с одинаковыми дисперсиями. - -**Аргументы** - -* `sample_data` — Данные выборки. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `sample_index` — Индекс выборки. [Integer](../../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `confidence_level` — Уровень доверия для вычисления доверительных интервалов. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) с двумя или четырьмя элементами (если указан необязательный параметр `confidence_level`): - -* вычисленная t-статистика. [Float64](../../../sql-reference/data-types/float.md). -* вычисленное p-значение. [Float64](../../../sql-reference/data-types/float.md). -* [вычисленная нижняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md).] -* [вычисленная верхняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md).] - -**Пример** - -Входная таблица: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.1 │ 0 │ -│ 21.9 │ 1 │ -│ 21.7 │ 0 │ -│ 19.9 │ 1 │ -│ 21.8 │ 1 │ -└─────────────┴──────────────┘ -``` - -Запрос: - -```sql -SELECT studentTTest(sample_data, sample_index) FROM student_ttest; -``` - -Результат: - -```text -┌─studentTTest(sample_data, sample_index)───┐ -│ (-0.21739130434783777,0.8385421208415731) │ -└───────────────────────────────────────────┘ -``` - -**См. также** - -* [t-критерий Стьюдента](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [функция welchTTest](/sql-reference/aggregate-functions/reference/welchttest) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md deleted file mode 100644 index 73df8d37bfa..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'Применяет одновыборочный t-тест Стьюдента к выборке и известному среднему значению генеральной совокупности.' -sidebar_label: 'studentTTestOneSample' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/studentttestonesample -title: 'studentTTestOneSample' -doc_type: 'reference' ---- - -# studentTTestOneSample {#studentttestonesample} - -Применяет одновыборочный t‑критерий Стьюдента для определения, отличается ли среднее выборки от известного среднего генеральной совокупности. - -Предполагается нормальность распределения. Нулевая гипотеза заключается в том, что среднее выборки равно среднему генеральной совокупности. - -**Синтаксис** - -```sql -studentTTestOneSample([уровень_доверия])(данные_выборки, среднее_генеральной_совокупности) -``` - -Необязательный параметр `confidence_level` включает вычисление доверительного интервала. - -**Аргументы** - -* `sample_data` — Выборочные данные. Integer, Float или Decimal. -* `population_mean` — Известное среднее по генеральной совокупности, с которым выполняется проверка. Integer, Float или Decimal (обычно константа). - -**Параметры** - -* `confidence_level` — Уровень доверия для доверительных интервалов. Float в диапазоне (0, 1). - -Примечания: - -* Требуются по крайней мере два наблюдения; в противном случае результат — `(nan, nan)` (а интервалы, если запрошены, равны `nan`). -* Постоянные или почти постоянные входные данные также вернут `nan` из‑за нулевой (или практически нулевой) стандартной ошибки. - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) с двумя или четырьмя элементами (если указан `confidence_level`): - -* вычисленная t-статистика. Float64. -* вычисленное p-значение (двустороннее). Float64. -* вычисленная нижняя граница доверительного интервала. Float64. (необязательно) -* вычисленная верхняя граница доверительного интервала. Float64. (необязательно) - -Доверительные интервалы относятся к среднему выборки при заданном уровне доверия. - -**Примеры** - -Входная таблица: - -```text -┌─value─┐ -│ 20.3 │ -│ 21.1 │ -│ 21.7 │ -│ 19.9 │ -│ 21.8 │ -└───────┘ -``` - -Без интервальной оценки: - -```sql -SELECT studentTTestOneSample()(value, 20.0) FROM t; --- or simply -SELECT studentTTestOneSample(value, 20.0) FROM t; -``` - -С доверительным интервалом 95 %: - -```sql -SELECT studentTTestOneSample(0.95)(value, 20.0) FROM t; -``` - -**См. также** - -* [t-критерий Стьюдента](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [функция studentTTest](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md deleted file mode 100644 index 12547489220..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Вычисляет сумму. Применима только к числам.' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/sum -title: 'sum' -doc_type: 'reference' ---- - -# sum {#sum} - -Вычисляет сумму. Может применяться только к числам. - -**Синтаксис** - -```sql -sum(num) -``` - -**Параметры** - -* `num`: Столбец числовых значений. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Сумма значений. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Пример** - -Сначала создадим таблицу `employees` и вставим в неё некоторые тестовые данные о сотрудниках. - -Запрос: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `salary` UInt32 -) -ENGINE = Log -``` - -```sql -INSERT INTO employees VALUES - (87432, 'John Smith', 45680), - (59018, 'Jane Smith', 72350), - (20376, 'Иван Иванович', 58900), - (71245, 'Анастасия Ивановна', 89210); -``` - -Запрашиваем общую сумму зарплат сотрудников с помощью функции `sum`. - -Запрос: - -```sql -SELECT sum(salary) FROM employees; -``` - -Результат: - -```response - ┌─sum(salary)─┐ -1. │ 266140 │ - └─────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md deleted file mode 100644 index 998390667bc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: 'Вычисляет сумму чисел и одновременно считает количество строк. Функция используется оптимизатором запросов ClickHouse: если в запросе присутствует несколько функций `sum`, `count` или `avg`, их можно заменить одной функцией `sumCount`, чтобы переиспользовать результаты вычислений. Необходимость явного использования этой функции возникает редко.' -sidebar_position: 196 -slug: /sql-reference/aggregate-functions/reference/sumcount -title: 'sumCount' -doc_type: 'reference' ---- - -Вычисляет сумму чисел и одновременно считает количество строк. Функция используется оптимизатором запросов ClickHouse: если в запросе присутствует несколько функций `sum`, `count` или `avg`, их можно заменить одной функцией `sumCount`, чтобы переиспользовать результаты вычислений. Необходимость явного использования этой функции возникает редко. - -**Синтаксис** - -```sql -sumCount(x) -``` - -**Аргументы** - -* `x` — входное значение, должно быть типа [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемое значение** - -* Кортеж `(sum, count)`, где `sum` — сумма чисел, а `count` — количество строк со значениями, отличными от NULL. - -Тип: [Tuple](../../../sql-reference/data-types/tuple.md). - -**Пример** - -Запрос: - -```sql -CREATE TABLE s_table (x Int8) ENGINE = Log; -INSERT INTO s_table SELECT number FROM numbers(0, 20); -INSERT INTO s_table VALUES (NULL); -SELECT sumCount(x) FROM s_table; -``` - -Результат: - -```text -┌─sumCount(x)─┐ -│ (190,20) │ -└─────────────┘ -``` - -**См. также** - -* Параметр [optimize_syntax_fuse_functions](../../../operations/settings/settings.md#optimize_syntax_fuse_functions). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md deleted file mode 100644 index e637ed00c3f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Вычисляет сумму чисел по алгоритму компенсированного суммирования Кэхэна' -sidebar_position: 197 -slug: /sql-reference/aggregate-functions/reference/sumkahan -title: 'sumKahan' -doc_type: 'reference' ---- - -Вычисляет сумму чисел по [алгоритму компенсированного суммирования Кэхэна](https://en.wikipedia.org/wiki/Kahan_summation_algorithm). -Работает медленнее, чем функция [sum](./sum.md). -Компенсация применяется только для типов [Float](../../../sql-reference/data-types/float.md). - -**Синтаксис** - -```sql -sumKahan(x) -``` - -**Аргументы** - -* `x` — входное значение, должно иметь тип [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). - -**Возвращаемое значение** - -* сумма чисел; тип [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md) зависит от типа входных аргументов. - -**Пример** - -Запрос: - -```sql -SELECT sum(0.1), sumKahan(0.1) FROM numbers(10); -``` - -Результат: - -```text -┌───────────sum(0.1)─┬─sumKahan(0.1)─┐ -│ 0.9999999999999999 │ 1 │ -└────────────────────┴───────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md deleted file mode 100644 index 6f1bf727fc0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: 'Суммирует один или несколько массивов `value` в соответствии с ключами, указанными в массиве `key`. Возвращает кортеж массивов: ключи в отсортированном порядке, затем значения, являющиеся суммами для соответствующих ключей без переполнения.' -sidebar_position: 198 -slug: /sql-reference/aggregate-functions/reference/summap -title: 'sumMap' -doc_type: 'reference' ---- - -# sumMap {#summap} - -Вычисляет сумму одного или нескольких массивов `value` в соответствии с ключами, указанными в массиве `key`. Возвращает кортеж массивов: ключи в отсортированном порядке, а затем значения, просуммированные для соответствующих ключей без переполнения. - -**Синтаксис** - -* `sumMap(key , value1 [, value2 , ...])` [тип Array](../../data-types/array.md). -* `sumMap(Tuple(key [, value1 , value2 , ...]))` [тип Tuple](../../data-types/tuple.md). - -Псевдоним: `sumMappedArrays`. - -**Аргументы** - -* `key`: [Array](../../data-types/array.md) ключей. -* `value1`, `value2`, ...: [Array](../../data-types/array.md) значений, которые суммируются для каждого ключа. - -Передача кортежа массивов ключей и значений равнозначна передаче отдельно массива ключей и массивов значений. - -:::note -Количество элементов в `key` и во всех массивах `value` должно быть одинаковым для каждой строки, по которой выполняется агрегация. -::: - -**Возвращаемое значение** - -* Возвращает кортеж массивов: первый массив содержит ключи в отсортированном порядке, за которым следуют массивы со значениями, просуммированными для соответствующих ключей. - -**Пример** - -Сначала создадим таблицу `sum_map` и вставим в неё некоторые данные. Массивы ключей и значений хранятся раздельно в столбце `statusMap` типа [Nested](../../data-types/nested-data-structures/index.md), а также вместе в столбце `statusMapTuple` типа [tuple](../../data-types/tuple.md), чтобы продемонстрировать использование двух различных синтаксисов этой функции, описанных выше. - -Запрос: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt16, - requests UInt64 - ), - statusMapTuple Tuple(Array(Int32), Array(Int32)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -Далее мы выполняем запрос к таблице с использованием функции `sumMap`, задействуя как синтаксис для массивов, так и для кортежей: - -Запрос: - -```sql -SELECT - timeslot, - sumMap(statusMap.status, statusMap.requests), - sumMap(statusMapTuple) -FROM sum_map -GROUP BY timeslot -``` - -Результат: - -```text -┌────────────timeslot─┬─sumMap(statusMap.status, statusMap.requests)─┬─sumMap(statusMapTuple)─────────┐ -│ 2000-01-01 00:00:00 │ ([1,2,3,4,5],[10,10,20,10,10]) │ ([1,2,3,4,5],[10,10,20,10,10]) │ -│ 2000-01-01 00:01:00 │ ([4,5,6,7,8],[10,10,20,10,10]) │ ([4,5,6,7,8],[10,10,20,10,10]) │ -└─────────────────────┴──────────────────────────────────────────────┴────────────────────────────────┘ -``` - -**Пример с несколькими массивами значений** - -`sumMap` также поддерживает агрегирование нескольких массивов значений одновременно. -Это полезно, когда у вас есть связанные метрики с общими ключами. - -```sql title="Query" -CREATE TABLE multi_metrics( - date Date, - browser_metrics Nested( - browser String, - impressions UInt32, - clicks UInt32 - ) -) -ENGINE = MergeTree() -ORDER BY tuple(); - -INSERT INTO multi_metrics VALUES - ('2000-01-01', ['Firefox', 'Chrome'], [100, 200], [10, 25]), - ('2000-01-01', ['Chrome', 'Safari'], [150, 50], [20, 5]), - ('2000-01-01', ['Firefox', 'Edge'], [80, 40], [8, 4]); - -SELECT - sumMap(browser_metrics.browser, browser_metrics.impressions, browser_metrics.clicks) AS result -FROM multi_metrics; -``` - -```text title="Response" -┌─result────────────────────────────────────────────────────────────────────────┐ -│ (['Chrome', 'Edge', 'Firefox', 'Safari'], [350, 40, 180, 50], [45, 4, 18, 5]) │ -└───────────────────────────────────────────────────────────────────────────────┘ -``` - -В этом примере: - -* Результирующий кортеж содержит три массива -* Первый массив: ключи (названия браузеров) в отсортированном порядке -* Второй массив: общее количество показов для каждого браузера -* Третий массив: общее количество кликов для каждого браузера - -**См. также** - -* [Комбинатор Map для типа данных Map](../combinators.md#-map) -* [sumMapWithOverflow](../reference/summapwithoverflow.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md deleted file mode 100644 index 157d75d01d0..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -description: 'Суммирует массив `value` по ключам, заданным в массиве `key`. Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, просуммированные для соответствующих ключей. Отличается от функции sumMap тем, что выполняет суммирование с переполнением.' -sidebar_position: 199 -slug: /sql-reference/aggregate-functions/reference/summapwithoverflow -title: 'sumMapWithOverflow' -doc_type: 'reference' ---- - -# sumMapWithOverflow {#summapwithoverflow} - -Подсчитывает сумму элементов массива `value` в соответствии с ключами, указанными в массиве `key`. Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, суммированные для соответствующих ключей. -Отличается от функции [sumMap](../reference/summap.md) тем, что выполняет суммирование с переполнением, то есть возвращает тот же тип данных для результата суммирования, что и тип данных аргумента. - -**Синтаксис** - -* `sumMapWithOverflow(key , value )` [тип Array](../../data-types/array.md). -* `sumMapWithOverflow(Tuple(key , value ))` [тип Tuple](../../data-types/tuple.md). - -**Аргументы** - -* `key`: [Array](../../data-types/array.md) ключей. -* `value`: [Array](../../data-types/array.md) значений. - -Передача кортежа массивов ключей и значений является синонимом передачи отдельно массива ключей и массива значений. - -:::note -Количество элементов в `key` и `value` должно быть одинаковым для каждой обрабатываемой строки. -::: - -**Возвращаемое значение** - -* Возвращает кортеж из двух массивов: ключи в отсортированном порядке и значения, суммированные для соответствующих ключей. - -**Пример** - -Сначала создадим таблицу `sum_map` и вставим в неё некоторые данные. Массивы ключей и значений хранятся отдельно в виде столбца `statusMap` типа [Nested](../../data-types/nested-data-structures/index.md) и вместе в виде столбца `statusMapTuple` типа [tuple](../../data-types/tuple.md), чтобы продемонстрировать использование двух разных синтаксисов этой функции, описанных выше. - -Запрос: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt8, - requests UInt8 - ), - statusMapTuple Tuple(Array(Int8), Array(Int8)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -Если мы выполним запрос к таблице, используя функции `sumMap`, `sumMapWithOverflow` с синтаксисом типа массив и `toTypeName`, то увидим, что -для функции `sumMapWithOverflow` тип данных массива просуммированных значений совпадает с типом аргумента, оба — `UInt8` (т. е. суммирование выполнялось с переполнением). Для `sumMap` тип данных массива просуммированных значений изменился с `UInt8` на `UInt64` таким образом, что переполнение не происходит. - -Запрос: - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMap.status, statusMap.requests)), - toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests)), -FROM sum_map -GROUP BY timeslot -``` - -Аналогично, мы могли бы использовать синтаксис кортежа, чтобы получить тот же результат. - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMapTuple)), - toTypeName(sumMapWithOverflow(statusMapTuple)), -FROM sum_map -GROUP BY timeslot -``` - -Результат: - -```text - ┌────────────timeslot─┬─toTypeName(sumMap(statusMap.status, statusMap.requests))─┬─toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests))─┐ -1. │ 2000-01-01 00:01:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ -2. │ 2000-01-01 00:00:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ - └─────────────────────┴──────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [sumMap](../reference/summap.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md deleted file mode 100644 index ee84f2280da..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'Вычисляет сумму чисел; для результата используется тот же тип данных, что и для входных параметров. Если сумма превышает максимальное значение для этого типа данных, результат вычисляется с переполнением.' -sidebar_position: 200 -slug: /sql-reference/aggregate-functions/reference/sumwithoverflow -title: 'sumWithOverflow' -doc_type: 'reference' ---- - -# sumWithOverflow {#sumwithoverflow} - -Вычисляет сумму чисел, используя для результата тот же тип данных, что и для входных параметров. Если сумма превышает максимально допустимое значение для этого типа данных, результат вычисляется с переполнением. - -Поддерживает только числовые типы. - -**Синтаксис** - -```sql -sumWithOverflow(num) -``` - -**Параметры** - -* `num`: Столбец числовых значений. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Сумма значений. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Пример** - -Сначала создаём таблицу `employees` и вставляем в неё некоторые вымышленные данные о сотрудниках. В этом примере мы зададим столбец `salary` типа `UInt16` так, чтобы сумма этих значений могла привести к переполнению. - -Запрос: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `monthly_salary` UInt16 -) -ENGINE = Log -``` - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow) -FROM employees -``` - -Мы выполняем запрос на сумму зарплат сотрудников, используя функции `sum` и `sumWithOverflow`, и отображаем их типы с помощью функции `toTypeName`. -Для функции `sum` результирующий тип — `UInt64`, достаточно велик, чтобы вместить сумму, тогда как для `sumWithOverflow` результирующий тип остаётся `UInt16`. - -Запрос: - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow), -FROM employees; -``` - -Результат: - -```response - ┌─no_overflow─┬─overflow─┬─toTypeName(no_overflow)─┬─toTypeName(overflow)─┐ -1. │ 118700 │ 53164 │ UInt64 │ UInt16 │ - └─────────────┴──────────┴─────────────────────────┴──────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md deleted file mode 100644 index 214cbd5ece3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'Функция `theilsU` вычисляет коэффициент неопределённости Тейла U (Theil’s U), - показатель, измеряющий степень зависимости между двумя столбцами в таблице.' -sidebar_position: 201 -slug: /sql-reference/aggregate-functions/reference/theilsu -title: 'theilsU' -doc_type: 'reference' ---- - -# theilsU {#theilsu} - -Функция `theilsU` вычисляет [коэффициент неопределённости U Тейла](https://en.wikipedia.org/wiki/Contingency_table#Uncertainty_coefficient) — величину, которая измеряет степень ассоциации между двумя столбцами в таблице. Её значения лежат в диапазоне от −1.0 (100% отрицательная ассоциация, или идеальная инверсия) до +1.0 (100% положительная ассоциация, или идеальное совпадение). Значение 0.0 указывает на отсутствие ассоциации. - -**Синтаксис** - -```sql -theilsU(column1, column2) -``` - -**Аргументы** - -* `column1` и `column2` — столбцы, которые сравниваются - -**Возвращаемое значение** - -* значение в диапазоне от -1 до 1 - -**Тип возвращаемого значения** всегда [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Два столбца, сравниваемые ниже, имеют слабую взаимосвязь друг с другом, поэтому значение `theilsU` отрицательное: - -```sql -SELECT - theilsU(a, b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -Результат: - -```response -┌────────theilsU(a, b)─┐ -│ -0.30195720557678846 │ -└──────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md deleted file mode 100644 index 77efde4fb25..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет изменения во временных рядах в стиле PromQL на указанной временной сетке.' -sidebar_position: 229 -slug: /sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid -title: 'timeSeriesChangesToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временного ряда в виде пар меток времени и значений и вычисляет [изменения в стиле PromQL](https://prometheus.io/docs/prometheus/latest/querying/functions/#changes) для этих данных на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки выборки для вычисления `changes` рассматриваются в пределах заданного временного окна. - -Параметры: - -* `start timestamp` — задаёт начало сетки -* `end timestamp` — задаёт конец сетки -* `grid step` — задаёт шаг сетки в секундах -* `staleness` — задаёт максимально допустимую «устарелость» в секундах для учитываемых выборок - -Аргументы: - -* `timestamp` — метка времени выборки -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -значения `changes` на указанной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне отсутствуют выборки, позволяющие вычислить значение изменений для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `changes` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210, 225]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 130 и 190 демонстрирует, как заполняются значения для ts = 180 согласно параметру окна - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- начало временной сетки - 90 + 135 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно устаревания -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Также можно передавать несколько наборов меток времени и значений в виде массивов одинаковой длины. Тот же запрос, но с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция экспериментальная. Для её включения установите `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md deleted file mode 100644 index 7c4ec23cbe4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет PromQL-подобную delta по данным временных рядов на заданной сетке.' -sidebar_position: 221 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid -title: 'timeSeriesDeltaToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и вычисляет [PromQL-подобную delta](https://prometheus.io/docs/prometheus/latest/querying/functions/#delta) из этих данных на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки на сетке образцы для вычисления `delta` рассматриваются в пределах указанного временного окна. - -Parameters: - -* `start timestamp` - Определяет начало сетки. -* `end timestamp` - Определяет конец сетки. -* `grid step` - Определяет шаг сетки в секундах. -* `staleness` - Определяет максимальную «устарелость» в секундах для рассматриваемых образцов. Окно устарелости — полуинтервал, открытый слева и закрытый справа. - -Arguments: - -* `timestamp` - метка времени образца -* `value` - значение временного ряда, соответствующее `timestamp` - -Return value: -Значения `delta` на заданной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне недостаточно образцов для вычисления `delta` для конкретной точки сетки. - -Example: -Следующий запрос вычисляет значения `delta` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 140 и 190 показывает, как заполняются значения для ts = 150, 165, 180 в соответствии с параметром окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- начало сетки временных меток - 90 + 120 AS end_ts, -- конец сетки временных меток - 15 AS step_seconds, -- шаг сетки временных меток - 45 AS window_seconds -- окно «устаревания» данных -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Этот подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesDeltaToGr⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,3,4.5,3.75,NULL,NULL,3.75] │ - └─────────────────────────────────────────┘ -``` - -Также можно передавать несколько выборок меток времени и значений в виде массивов одинаковой длины. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Это экспериментальная функция; включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md deleted file mode 100644 index 82241b06bd7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет производную, аналогичную PromQL, по временным рядам на заданной сетке.' -sidebar_position: 227 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid -title: 'timeSeriesDerivToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и вычисляет [производную, аналогичную PromQL](https://prometheus.io/docs/prometheus/latest/querying/functions/#deriv) по этим данным на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки образцы для вычисления `deriv` рассматриваются в пределах заданного временного окна. - -Параметры: - -* `start timestamp` — задаёт начало сетки. -* `end timestamp` — задаёт конец сетки. -* `grid step` — задаёт шаг сетки в секундах. -* `staleness` — задаёт максимальное «устаревание» в секундах для учитываемых образцов. Окно устаревания — это полуинтервал, открытый слева и закрытый справа. - -Аргументы: - -* `timestamp` — метка времени образца -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -Значения `deriv` на указанной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне недостаточно образцов для вычисления значения производной для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `deriv` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 140 и 190 демонстрирует заполнение значений для ts = 150, 165, 180 согласно параметру окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих временным меткам выше - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно устаревания -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,0.1,0.11,0.15,NULL,NULL,0.15] │ - └─────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Также можно передавать несколько временных меток и значений в виде массивов одинаковой длины. Тот же запрос, но с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция экспериментальная. Чтобы её включить, установите `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md deleted file mode 100644 index 644a70121c7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: 'Сортирует временные ряды по возрастанию метки времени.' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/timeSeriesGroupArray -title: 'timeSeriesGroupArray' -doc_type: 'reference' ---- - -# timeSeriesGroupArray {#timeseriesgrouparray} - -Сортирует временные ряды по метке времени по возрастанию. - -**Синтаксис** - -```sql -timeSeriesGroupArray(timestamp, value) -``` - -**Аргументы** - -* `timestamp` — временная метка выборки -* `value` — значение временного ряда, соответствующее `timestamp` - -**Возвращаемое значение** - -Функция возвращает массив кортежей (`timestamp`, `value`), отсортированный по возрастанию `timestamp`. -Если для одного и того же `timestamp` существует несколько значений, функция выбирает наибольшее из них. - -**Пример** - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- массив значений, соответствующих временным меткам выше -SELECT timeSeriesGroupArray(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp` и `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesGroupArray(timestamp, value)───────┐ -1. │ [(100,5),(110,1),(120,6),(130,8),(140,19)] │ - └──────────────────────────────────────────────┘ -``` - -Также можно передавать несколько меток времени и значений в виде массивов одинаковой длины. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- массив значений, соответствующих указанным выше временным меткам -SELECT timeSeriesGroupArray(timestamps, values); -``` - -:::note -Эта функция экспериментальная; включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md deleted file mode 100644 index 4fa4e4163c3..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет PromQL-подобный idelta по данным временных рядов на заданной сетке.' -sidebar_position: 222 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid -title: 'timeSeriesInstantDeltaToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и вычисляет [PromQL-подобный idelta](https://prometheus.io/docs/prometheus/latest/querying/functions/#idelta) по этим данным на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки учитываемые для вычисления `idelta` отсчёты берутся в пределах указанного временного окна. - -Параметры: - -* `start timestamp` — задаёт начало сетки. -* `end timestamp` — задаёт конец сетки. -* `grid step` — задаёт шаг сетки в секундах. -* `staleness` — задаёт максимальную «устарелость» в секундах для учитываемых отсчётов. Окно устарелости представляет собой интервал, открытый слева и закрытый справа. - -Аргументы: - -* `timestamp` — метка времени отсчёта. -* `value` — значение временного ряда, соответствующее `timestamp`. - -Возвращаемое значение: -Значения `idelta` на заданной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит по одному значению для каждой точки временной сетки. Значение равно NULL, если в окне недостаточно отсчётов для вычисления значения мгновенного приращения (instant delta) для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `idelta` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 140 и 190 демонстрирует заполнение значений для ts = 150, 165, 180 согласно параметру окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно устаревания -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesInsta⋯stamps, values)─┐ -1. │ [NULL,NULL,0,2,1,1,NULL,NULL,3] │ - └─────────────────────────────────┘ -``` - -Также можно передавать несколько меток времени и значений в виде массивов одинаковой длины. Тот же запрос, но с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Функция экспериментальная, включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md deleted file mode 100644 index 7728fc0df7e..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет PromQL-подобный irate для данных временных рядов на заданной сетке.' -sidebar_position: 223 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid -title: 'timeSeriesInstantRateToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временного ряда в виде пар меток времени и значений и вычисляет [PromQL-подобный irate](https://prometheus.io/docs/prometheus/latest/querying/functions/#irate) для этих данных на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки на сетке семплы для вычисления `irate` рассматриваются в пределах заданного временного окна. - -Параметры: - -* `start timestamp` — определяет начало сетки. -* `end timestamp` — определяет конец сетки. -* `grid step` — определяет шаг сетки в секундах. -* `staleness` — задает максимальное время «устаревания» рассматриваемых семплов в секундах. Окно устаревания представляет собой полуинтервал, открытый слева и закрытый справа. - -Аргументы: - -* `timestamp` — метка времени семпла -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -значения `irate` на заданной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне недостаточно семплов для вычисления значения мгновенной скорости изменения для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `irate` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: промежуток между 140 и 190 демонстрирует заполнение значений для ts = 150, 165, 180 согласно параметру окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих временным меткам выше - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно устаревания -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesInstantRa⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,0.2,0.1,0.1,NULL,NULL,0.3] │ - └─────────────────────────────────────────┘ -``` - -Также можно передать несколько выборок временных меток и значений в виде массивов одинакового размера. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция экспериментальная; чтобы её включить, установите `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md deleted file mode 100644 index 043af549bf8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md +++ /dev/null @@ -1,163 +0,0 @@ ---- -description: 'Агрегатная функция для ресемплинга временных рядов для вычислений irate и idelta в стиле PromQL' -sidebar_position: 224 -slug: /sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples -title: 'timeSeriesLastTwoSamples' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временного ряда в виде пар меток времени и значений и хранит не более чем 2 последних измерений. - -Аргументы: - -* `timestamp` — метка времени измерения -* `value` — значение временного ряда, соответствующее `timestamp`\ - Также можно передавать несколько измерений (метки времени и значения) в виде массивов одинакового размера. - -Возвращаемое значение:\ -`Tuple(Array(DateTime), Array(Float64))` — пара массивов одинаковой длины (от 0 до 2 элементов). Первый массив содержит метки времени выборок временного ряда, второй массив содержит соответствующие значения временного ряда. - -Пример:\ -Эта агрегатная функция предназначена для использования с материализованным представлением и агрегированной таблицей, которые хранят ресемплированные данные временных рядов для временных меток, выровненных по сетке (grid-aligned). Рассмотрим следующую таблицу с сырыми данными и таблицу для хранения ресемплированных данных: - -```sql --- Таблица для необработанных данных -CREATE TABLE t_raw_timeseries -( - metric_id UInt64, - timestamp DateTime64(3, 'UTC') CODEC(DoubleDelta, ZSTD), - value Float64 CODEC(DoubleDelta) -) -ENGINE = MergeTree() -ORDER BY (metric_id, timestamp); - --- Таблица с данными, пересэмплированными с более крупным временным шагом (15 с) -CREATE TABLE t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), -- Метка времени, выровненная с шагом 15 секунд - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -ENGINE = AggregatingMergeTree() -ORDER BY (metric_id, grid_timestamp); - --- Материализованное представление (MV) для заполнения пересэмплированной таблицы -CREATE MATERIALIZED VIEW mv_resampled_timeseries TO t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -AS SELECT - metric_id, - ceil(toUnixTimestamp(timestamp + interval 999 millisecond) / 15, 0) * 15 AS grid_timestamp, -- Округлить метку времени до следующей временной точки сетки - initializeAggregation('timeSeriesLastTwoSamplesState', timestamp, value) AS samples -FROM t_raw_timeseries -ORDER BY metric_id, grid_timestamp; -``` - -Вставьте немного тестовых данных и прочитайте данные за период между '2024-12-12 12:00:12' и '2024-12-12 12:00:30' - -```sql --- Вставим немного данных -INSERT INTO t_raw_timeseries(metric_id, timestamp, value) SELECT number%10 AS metric_id, '2024-12-12 12:00:00'::DateTime64(3, 'UTC') + interval ((number/10)%100)*900 millisecond as timestamp, number%3+number%29 AS value FROM numbers(1000); - --- Проверим сырые данные -SELECT * -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN '2024-12-12 12:00:12' AND '2024-12-12 12:00:31' -ORDER BY metric_id, timestamp; -``` - - -```response -3 2024-12-12 12:00:12.870 29 -3 2024-12-12 12:00:13.770 8 -3 2024-12-12 12:00:14.670 19 -3 2024-12-12 12:00:15.570 30 -3 2024-12-12 12:00:16.470 9 -3 2024-12-12 12:00:17.370 20 -3 2024-12-12 12:00:18.270 2 -3 2024-12-12 12:00:19.170 10 -3 2024-12-12 12:00:20.070 21 -3 2024-12-12 12:00:20.970 3 -3 2024-12-12 12:00:21.870 11 -3 2024-12-12 12:00:22.770 22 -3 2024-12-12 12:00:23.670 4 -3 2024-12-12 12:00:24.570 12 -3 2024-12-12 12:00:25.470 23 -3 2024-12-12 12:00:26.370 5 -3 2024-12-12 12:00:27.270 13 -3 2024-12-12 12:00:28.170 24 -3 2024-12-12 12:00:29.069 6 -3 2024-12-12 12:00:29.969 14 -3 2024-12-12 12:00:30.869 25 -``` - -Выполните запрос двух последних записей с метками времени '2024-12-12 12:00:15' и '2024-12-12 12:00:30': - -```sql --- Проверьте ресемплированные данные -SELECT metric_id, grid_timestamp, (finalizeAggregation(samples).1 as timestamp, finalizeAggregation(samples).2 as value) -FROM t_resampled_timeseries_15_sec -WHERE metric_id = 3 AND grid_timestamp BETWEEN '2024-12-12 12:00:15' AND '2024-12-12 12:00:30' -ORDER BY metric_id, grid_timestamp; -``` - -```response -3 2024-12-12 12:00:15 (['2024-12-12 12:00:14.670','2024-12-12 12:00:13.770'],[19,8]) -3 2024-12-12 12:00:30 (['2024-12-12 12:00:29.969','2024-12-12 12:00:29.069'],[14,6]) -``` - -Агрегированная таблица хранит только два последних значения для каждой 15‑секундной выровненной отметки времени. Это позволяет вычислять PromQL‑подобные `irate` и `idelta`, считывая значительно меньше данных, чем хранится в сырой таблице. - -```sql --- Вычислить idelta и irate по сырым данным -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- начало сетки временных меток - start_ts + INTERVAL 60 SECOND AS end_ts, -- конец сетки временных меток - 15 AS step_seconds, -- шаг сетки временных меток - 45 AS window_seconds -- окно устаревания данных -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - - -```sql --- Вычислить idelta и irate из пересемплированных данных -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- начало временной сетки - start_ts + INTERVAL 60 SECOND AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно «устаревания» данных -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values) -FROM ( - SELECT - metric_id, - finalizeAggregation(samples).1 AS timestamps, - finalizeAggregation(samples).2 AS values - FROM t_resampled_timeseries_15_sec - WHERE metric_id = 3 AND grid_timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -) -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - -:::note -Эта функция экспериментальная; включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md deleted file mode 100644 index 261a1cfba45..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: 'Агрегатная функция, рассчитывающая линейный прогноз в стиле PromQL по временным рядам на указанной сетке.' -sidebar_position: 228 -slug: /sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid -title: 'timeSeriesPredictLinearToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и рассчитывает [линейный прогноз в стиле PromQL](https://prometheus.io/docs/prometheus/latest/querying/functions/#predict_linear) с указанным смещением времени прогноза на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки на сетке выборки для вычисления `predict_linear` рассматриваются в пределах заданного временного окна. - -Параметры: - -* `start timestamp` - задаёт начало сетки. -* `end timestamp` - задаёт конец сетки. -* `grid step` - задаёт шаг сетки в секундах. -* `staleness` - задаёт максимальную «устарелость» рассматриваемых выборок в секундах. Окно устарелости представляет собой полуинтервал, открытый слева и закрытый справа. -* `predict_offset` - задаёт количество секунд смещения, добавляемого к времени прогноза. - -Аргументы: - -* `timestamp` - метка времени выборки -* `value` - значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -Значения `predict_linear` на указанной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если внутри окна недостаточно выборок для вычисления значения скорости изменения для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `predict_linear` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210] с 60-секундным смещением: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 140 и 190 демонстрирует, как заполняются значения для ts = 150, 165, 180 согласно параметру окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds, -- окно "устаревания" - 60 AS predict_offset -- временное смещение для прогнозирования -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value)─┐ -1. │ [NULL,NULL,1,9.166667,11.6,16.916666,NULL,NULL,16.5] │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Также можно передавать несколько значений меток времени и соответствующих значений в виде массивов одинакового размера. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds, - 60 AS predict_offset -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamps, values); -``` - -:::note -Эта функция является экспериментальной; для её включения установите значение `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md deleted file mode 100644 index 024dbda6294..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет похожую на PromQL скорость изменения (rate) по данным временных рядов на заданной сетке.' -sidebar_position: 225 -slug: /sql-reference/aggregate-functions/reference/timeSeriesRateToGrid -title: 'timeSeriesRateToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и вычисляет [похожий на PromQL rate](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) по этим данным на регулярной временной сетке, задаваемой начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки сэмплы для вычисления `rate` рассматриваются в указанном временном окне. - -Параметры: - -* `start timestamp` — задаёт начало сетки. -* `end timestamp` — задаёт конец сетки. -* `grid step` — задаёт шаг сетки в секундах. -* `staleness` — задаёт максимальную «устарелость» в секундах для учитываемых сэмплов. Окно устарелости — левооткрытый, право-замкнутый интервал. - -Аргументы: - -* `timestamp` — метка времени сэмпла -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -Значения `rate` на указанной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне недостаточно сэмплов для вычисления значения скорости для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `rate` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: промежуток между 140 и 190 демонстрирует заполнение значений для ts = 150, 165, 180 согласно параметру окна - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих временным меткам выше - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 45 AS window_seconds -- окно "устаревания" -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesRateToGrid(start_ts, ⋯w_seconds)(timestamps, values)─┐ -1. │ [NULL,NULL,0,0.06666667,0.1,0.083333336,NULL,NULL,0.083333336] │ - └────────────────────────────────────────────────────────────────┘ -``` - -Также можно передавать несколько наборов меток времени и значений в виде массивов одинакового размера. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция экспериментальная; включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md deleted file mode 100644 index fddb3a0142d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'Агрегатная функция, которая повторно дискретизирует данные временных рядов на заданную сетку.' -sidebar_position: 226 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness -title: 'timeSeriesResampleToGridWithStaleness' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временного ряда в виде пар меток времени и значений и повторно дискретизирует эти данные на регулярную временную сетку, задаваемую начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки выбирается самый свежий (в пределах заданного временного окна) отсчёт. - -Псевдоним: `timeSeriesLastToGrid`. - -Параметры: - -* `start timestamp` — определяет начало сетки -* `end timestamp` — определяет конец сетки -* `grid step` — определяет шаг сетки в секундах -* `staleness window` — определяет максимальную «устарелость» самого свежего отсчёта в секундах - -Аргументы: - -* `timestamp` — метка времени отсчёта -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -значения временного ряда, повторно дискретизированные на заданную сетку в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если для конкретной точки сетки нет отсчёта. - -Пример: -Следующий запрос повторно дискретизирует данные временного ряда на сетку [90, 105, 120, 135, 150, 165, 180, 195, 210], выбирая значение не старше 30 секунд для каждой точки сетки: - -```sql -WITH - -- ПРИМЕЧАНИЕ: разрыв между 140 и 190 демонстрирует заполнение значений для ts = 150, 165, 180 согласно параметру окна устаревания - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- начало временной сетки - 90 + 120 AS end_ts, -- конец временной сетки - 15 AS step_seconds, -- шаг временной сетки - 30 AS window_seconds -- окно устаревания -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Данный подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesResa⋯stamp, value)─┐ -1. │ [NULL,NULL,1,3,4,4,NULL,5,8] │ - └──────────────────────────────┘ -``` - -Также можно передать несколько пар меток времени и значений в виде массивов одинаковой длины. Тот же запрос, но с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 30 AS window_seconds -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция является экспериментальной. Чтобы её включить, установите `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md deleted file mode 100644 index 65b9a664268..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: 'Агрегатная функция, которая вычисляет PromQL-подобные resets для данных временных рядов на заданной сетке.' -sidebar_position: 230 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid -title: 'timeSeriesResetsToGrid' -doc_type: 'reference' ---- - -Агрегатная функция, которая принимает данные временных рядов в виде пар меток времени и значений и вычисляет [PromQL-подобные resets](https://prometheus.io/docs/prometheus/latest/querying/functions/#resets) для этих данных на регулярной временной сетке, описанной начальной меткой времени, конечной меткой времени и шагом. Для каждой точки сетки образцы для вычисления `resets` рассматриваются в пределах заданного временного окна. - -Параметры: - -* `start timestamp` — задаёт начало сетки -* `end timestamp` — задаёт конец сетки -* `grid step` — задаёт шаг сетки в секундах -* `staleness` — задаёт максимальный период «устаревания» в секундах для учитываемых образцов - -Аргументы: - -* `timestamp` — метка времени образца -* `value` — значение временного ряда, соответствующее `timestamp` - -Возвращаемое значение: -значения `resets` на заданной сетке в виде `Array(Nullable(Float64))`. Возвращаемый массив содержит одно значение для каждой точки временной сетки. Значение равно NULL, если в окне нет образцов для вычисления значения `resets` для конкретной точки сетки. - -Пример: -Следующий запрос вычисляет значения `resets` на сетке [90, 105, 120, 135, 150, 165, 180, 195, 210, 225]: - -```sql -WITH - -- ПРИМЕЧАНИЕ: интервал между 130 и 190 показывает, как заполняются значения для ts = 180 в соответствии с параметром окна - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, -- массив значений, соответствующих указанным выше временным меткам - 90 AS start_ts, -- start of timestamp grid - 90 + 135 AS end_ts, -- end of timestamp grid - 15 AS step_seconds, -- step of timestamp grid - 45 AS window_seconds -- "staleness" window -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- Этот подзапрос преобразует массивы временных меток и значений в строки с полями `timestamp`, `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -Ответ: - -```response - ┌─timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Также можно передать несколько пар меток времени и значений в виде массивов одинакового размера. Тот же запрос с аргументами-массивами: - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -Эта функция экспериментальная. Включите её, установив `allow_experimental_ts_to_grid_aggregate_function=true`. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md deleted file mode 100644 index d2874968ecd..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: 'Возвращает массив значений, которые приблизительно чаще всего встречаются в указанном столбце. Результирующий массив отсортирован по убыванию оценочной частоты значений (а не по самим значениям).' -sidebar_position: 202 -slug: /sql-reference/aggregate-functions/reference/topk -title: 'topK' -doc_type: 'reference' ---- - -# topK {#topk} - -Возвращает массив приблизительно наиболее часто встречающихся значений в указанном столбце. Полученный массив отсортирован по убыванию приблизительной частоты значений (а не по самим значениям). - -Реализует алгоритм [Filtered Space-Saving](https://doi.org/10.1016/j.ins.2010.08.024) для анализа topK, основанный на алгоритме reduce-and-combine из работы [Parallel Space Saving](https://doi.org/10.1016/j.ins.2015.09.003). - -```sql -topK(N)(column) -topK(N, load_factor)(column) -topK(N, load_factor, 'counts')(column) -``` - -Эта функция не гарантирует точный результат. В некоторых случаях могут возникать ошибки, и она может возвращать часто встречающиеся значения, которые при этом не являются самыми частыми. - -Максимальное значение `N = 65536`. - -**Параметры** - -* `N` — количество элементов для возврата. Необязательный параметр. Значение по умолчанию: 10. -* `load_factor` — определяет, сколько ячеек будет зарезервировано для значений. Если uniq(column) > N * load_factor, результат функции topK будет приблизительным. Необязательный параметр. Значение по умолчанию: 3. -* `counts` — определяет, должен ли результат содержать приблизительное количество и значение ошибки. - -**Аргументы** - -* `column` — значение, для которого рассчитывается частота. - -**Пример** - -Возьмем набор данных [OnTime](../../../getting-started/example-datasets/ontime.md) и выберем три наиболее часто встречающихся значения в столбце `AirlineID`. - -```sql -SELECT topK(3)(AirlineID) AS res -FROM ontime -``` - -```text -┌─res─────────────────┐ -│ [19393,19790,19805] │ -└─────────────────────┘ -``` - -**См. также** - -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md deleted file mode 100644 index b2d3497e34f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'Возвращает массив приблизительно наиболее часто встречающихся значений - в указанном столбце. Полученный массив отсортирован в порядке убывания приблизительной - частоты значений (не по самим значениям). Также учитывается вес значения.' -sidebar_position: 203 -slug: /sql-reference/aggregate-functions/reference/topkweighted -title: 'topKWeighted' -doc_type: 'reference' ---- - -# topKWeighted {#topkweighted} - -Возвращает массив наиболее часто встречающихся (примерно) значений в указанном столбце. Полученный массив отсортирован по убыванию приблизительной частоты появления значений (а не по самим значениям). Также учитывается вес значения. - -**Синтаксис** - -```sql -topKWeighted(N)(column, weight) -topKWeighted(N, load_factor)(column, weight) -topKWeighted(N, load_factor, 'counts')(column, weight) -``` - -**Параметры** - -* `N` — Количество элементов, которые нужно вернуть. Необязательный параметр. Значение по умолчанию: 10. -* `load_factor` — Определяет, сколько ячеек зарезервировано для значений. Если uniq(column) > N * load_factor, результат функции topK будет приближённым. Необязательный параметр. Значение по умолчанию: 3. -* `counts` — Определяет, должен ли результат содержать приближённое значение счётчика и оценку ошибки. - -**Аргументы** - -* `column` — Значение. -* `weight` — Вес. Каждое значение учитывается `weight` раз при вычислении частоты. [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Возвращаемое значение** - -Возвращает массив значений с максимальной приблизительной суммой весов. - -**Пример** - -Запрос: - -```sql -SELECT topKWeighted(2)(k, w) FROM -VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -Результат: - -```text -┌─topKWeighted(2)(k, w)──┐ -│ ['z','x'] │ -└────────────────────────┘ -``` - -Запрос: - -```sql -SELECT topKWeighted(2, 10, 'counts')(k, w) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -Результат: - -```text -┌─topKWeighted(2, 10, 'counts')(k, w)─┐ -│ [('z',10,0),('x',5,0)] │ -└─────────────────────────────────────┘ -``` - -**См. также** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md deleted file mode 100644 index afe13645a2f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: 'Вычисляет приблизительное количество различных значений аргумента.' -sidebar_position: 204 -slug: /sql-reference/aggregate-functions/reference/uniq -title: 'uniq' -doc_type: 'reference' ---- - -# uniq {#uniq} - -Вычисляет приблизительное количество уникальных значений аргумента. - -```sql -uniq(x[, ...]) -``` - -**Аргументы** - -Функция принимает переменное количество аргументов. Аргументы могут иметь типы `Tuple`, `Array`, `Date`, `DateTime`, `String` или числовые типы. - -**Возвращаемое значение** - -* Число типа [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Подробности реализации** - -Функция: - -* Вычисляет хэш для всех аргументов в агрегате, затем использует его в вычислениях. - -* Использует адаптивный алгоритм выборки. Для состояния вычисления функция использует выборку значений хэшей элементов размером до 65536. Этот алгоритм очень точен и очень эффективен с точки зрения использования CPU. Когда запрос содержит несколько таких функций, использование `uniq` почти так же быстро, как использование других агрегатных функций. - -* Обеспечивает детерминированный результат (он не зависит от порядка обработки запроса). - -Мы рекомендуем использовать эту функцию практически во всех сценариях. - -**См. также** - -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md deleted file mode 100644 index d544ac6ecce..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'Вычисляет приблизительное число различных значений аргумента.' -sidebar_position: 205 -slug: /sql-reference/aggregate-functions/reference/uniqcombined -title: 'uniqCombined' -doc_type: 'reference' ---- - -# uniqCombined {#uniqcombined} - -Вычисляет приблизительное количество различных значений аргументов. - -```sql -uniqCombined(HLL_precision)(x[, ...]) -``` - -Функция `uniqCombined` — хороший выбор для вычисления количества различных значений. - -**Аргументы** - -* `HLL_precision`: двоичный логарифм числа ячеек в [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog). Необязательный параметр, функцию можно вызывать как `uniqCombined(x[, ...])`. Значение по умолчанию для `HLL_precision` — 17, что фактически соответствует 96 КиБ памяти (2^17 ячеек, по 6 бит каждая). -* `X`: переменное количество параметров. Параметры могут иметь типы `Tuple`, `Array`, `Date`, `DateTime`, `String` или числовые типы. - -**Возвращаемое значение** - -* Число типа [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Особенности реализации** - -Функция `uniqCombined`: - -* Вычисляет хеш (64-битный хеш для `String` и 32-битный в остальных случаях) для всех параметров в агрегате и использует его в вычислениях. -* Использует комбинацию трёх алгоритмов: массив, хеш-таблицу и HyperLogLog с таблицей коррекции ошибок. - * Для небольшого количества различных элементов используется массив. - * При большем размере множества используется хеш-таблица. - * Для ещё большего количества элементов используется HyperLogLog, который занимает фиксированный объём памяти. -* Возвращает результат детерминированно (он не зависит от порядка обработки запроса). - -:::note\ -Поскольку для типов, отличных от `String`, используется 32-битный хеш, результат будет иметь очень большую погрешность для кардинальностей множеств, значительно превышающих `UINT_MAX` (ошибка начнёт быстро расти после нескольких десятков миллиардов различных значений). В этом случае следует использовать [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64). -::: - -По сравнению с функцией [uniq](/sql-reference/aggregate-functions/reference/uniq) функция `uniqCombined`: - -* Потребляет в несколько раз меньше памяти. -* Обеспечивает в несколько раз более высокую точность. -* Обычно имеет немного более низкую производительность. В некоторых сценариях `uniqCombined` может работать лучше, чем `uniq`, например, при распределённых запросах, которые передают по сети большое количество агрегатных состояний. - -**Пример** - -Запрос: - -```sql -SELECT uniqCombined(number) FROM numbers(1e6); -``` - -Результат: - -```response -┌─uniqCombined(number)─┐ -│ 1001148 │ -- 1,00 миллиона -└──────────────────────┘ -``` - -См. раздел с примерами в [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) для иллюстрации различий между `uniqCombined` и `uniqCombined64` на гораздо более крупных входных данных. - -**См. также** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md deleted file mode 100644 index 7288239edc8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -description: 'Вычисляет приблизительное количество различных значений аргумента. То же, - что uniqCombined, но использует 64-битный хеш для всех типов данных, а не только - для типа String.' -sidebar_position: 206 -slug: /sql-reference/aggregate-functions/reference/uniqcombined64 -title: 'uniqCombined64' -doc_type: 'reference' ---- - -# uniqCombined64 {#uniqcombined64} - -Вычисляет приблизительное количество различных значений аргумента. Аналогична функции [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined), но использует 64-битный хеш для всех типов данных, а не только для строкового типа данных String. - -```sql -uniqCombined64(HLL_precision)(x[, ...]) -``` - -**Параметры** - -* `HLL_precision`: двоичный логарифм числа ячеек в [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog). Также вы можете использовать функцию как `uniqCombined64(x[, ...])`. Значение по умолчанию для `HLL_precision` — 17, что фактически соответствует 96 КиБ памяти (2^17 ячеек по 6 бит каждая). -* `X`: Переменное количество параметров. Параметры могут иметь типы `Tuple`, `Array`, `Date`, `DateTime`, `String` или быть числовыми. - -**Возвращаемое значение** - -* Число типа [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Подробности реализации** - -Функция `uniqCombined64`: - -* Вычисляет хеш (64-битный хеш для всех типов данных) для всех параметров в агрегате и затем использует его в вычислениях. -* Использует комбинацию трёх алгоритмов: массив, хеш-таблица и HyperLogLog с таблицей коррекции погрешности. - * Для небольшого количества различных элементов используется массив. - * При увеличении размера множества используется хеш-таблица. - * Для большого количества элементов используется HyperLogLog, который занимает фиксированный объём памяти. -* Возвращает результат детерминированно (он не зависит от порядка обработки запроса). - -:::note -Поскольку для всех типов используется 64-битный хеш, результат не страдает от очень высокой погрешности для кардинальностей, значительно превышающих `UINT_MAX`, в отличие от [uniqCombined](../../../sql-reference/aggregate-functions/reference/uniqcombined.md), который использует 32-битный хеш для типов, отличных от `String`. -::: - -По сравнению с функцией [uniq](/sql-reference/aggregate-functions/reference/uniq) функция `uniqCombined64`: - -* Потребляет в несколько раз меньше памяти. -* Вычисляет результат с в несколько раз более высокой точностью. - -**Пример** - -В примере ниже `uniqCombined64` применяется к `1e10` различным числам и возвращает очень близкую оценку количества различных значений аргумента. - -Запрос: - -```sql -SELECT uniqCombined64(number) FROM numbers(1e10); -``` - -Результат: - -```response -┌─uniqCombined64(number)─┐ -│ 9998568925 │ -- 10,00 миллиардов -└────────────────────────┘ -``` - -Для сравнения функция `uniqCombined` возвращает довольно неточное приближение для такого объёма входных данных. - -Запрос: - -```sql -SELECT uniqCombined(number) FROM numbers(1e10); -``` - -Результат: - -```response -┌─uniqCombined(number)─┐ -│ 5545308725 │ -- 5,55 миллиарда -└──────────────────────┘ -``` - -**См. также** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md deleted file mode 100644 index b5105dc231c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'Вычисляет точное количество различных значений аргумента.' -sidebar_position: 207 -slug: /sql-reference/aggregate-functions/reference/uniqexact -title: 'uniqExact' -doc_type: 'reference' ---- - -# uniqExact {#uniqexact} - -Вычисляет точное количество различных значений аргументов. - -```sql -uniqExact(x[, ...]) -``` - -Используйте функцию `uniqExact` только в том случае, если вам абсолютно необходим точный результат. В остальных случаях используйте функцию [uniq](/sql-reference/aggregate-functions/reference/uniq). - -Функция `uniqExact` использует больше памяти, чем `uniq`, потому что состояние неограниченно растёт по мере увеличения числа различных значений. - -**Аргументы** - -Функция принимает переменное число параметров. Параметры могут иметь типы `Tuple`, `Array`, `Date`, `DateTime`, `String` или числовые типы. - -**Пример** - -В этом примере мы используем функцию `uniqExact`, чтобы посчитать количество уникальных кодов типов (краткий идентификатор типа самолёта) в [наборе данных OpenSky](https://sql.clickhouse.com?query=U0VMRUNUIHVuaXFFeGFjdCh0eXBlY29kZSkgRlJPTSBvcGVuc2t5Lm9wZW5za3k&). - -```sql title="Query" -SELECT uniqExact(typecode) FROM opensky.opensky -``` - -```response title="Response" -1106 -``` - -**См. также** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md deleted file mode 100644 index 38f7b7086d7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'Вычисляет приблизительное количество различных значений аргумента с использованием алгоритма HyperLogLog.' -sidebar_position: 208 -slug: /sql-reference/aggregate-functions/reference/uniqhll12 -title: 'uniqHLL12' -doc_type: 'reference' ---- - -# uniqHLL12 {#uniqhll12} - -Вычисляет приблизительное количество различных значений аргументов с использованием алгоритма [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog). - -```sql -uniqHLL12(x[, ...]) -``` - -**Аргументы** - -Функция принимает переменное число параметров. Параметры могут иметь типы `Tuple`, `Array`, `Date`, `DateTime`, `String` или числовые типы. - -**Возвращаемое значение** - -* Число типа [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Подробности реализации** - -Функция: - -* Вычисляет хэш для всех параметров в агрегате и затем использует его в вычислениях. - -* Использует алгоритм HyperLogLog для приближённого подсчёта количества различных значений аргументов. - - Используются 2^12 5-битных ячеек. Размер состояния немного больше 2,5 КБ. Результат не очень точен (погрешность до ~10%) для небольших наборов данных (<10K элементов). Однако результат достаточно точен для наборов данных с высокой кардинальностью (10K-100M), с максимальной погрешностью ~1,6%. Начиная со 100M элементов, ошибка оценки увеличивается, и функция будет возвращать крайне неточные результаты для наборов данных с чрезвычайно высокой кардинальностью (1B+ элементов). - -* Обеспечивает детерминированный результат (он не зависит от порядка обработки запроса). - -Мы не рекомендуем использовать эту функцию. В большинстве случаев используйте функцию [uniq](/sql-reference/aggregate-functions/reference/uniq) или [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined). - -**См. также** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md deleted file mode 100644 index 9b46e3f084b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Вычисляет приблизительное количество различных значений аргумента на основе фреймворка Theta Sketch.' -sidebar_position: 209 -slug: /sql-reference/aggregate-functions/reference/uniqthetasketch -title: 'uniqTheta' -doc_type: 'reference' ---- - -Вычисляет приблизительное количество различных значений аргумента на основе [фреймворка Theta Sketch](https://datasketches.apache.org/docs/Theta/ThetaSketches.html#theta-sketch-framework). - -```sql -uniqTheta(x[, ...]) -``` - -**Аргументы** - -Функция принимает переменное число параметров. Параметры могут иметь типы данных `Tuple`, `Array`, `Date`, `DateTime`, `String` или числовые типы. - -**Возвращаемое значение** - -* Число типа [UInt64](../../../sql-reference/data-types/int-uint.md). - -**Подробности реализации** - -Функция: - -* Вычисляет хеш для всех параметров в агрегате и затем использует его в вычислениях. - -* Использует алгоритм [KMV](https://datasketches.apache.org/docs/Theta/InverseEstimate.html) для приближённой оценки количества различных значений аргументов. - - Используются 4096 (2^12) 64-битных эскизов (sketches). Размер состояния составляет около 41 КБ. - -* Относительная погрешность — 3.125 % (95 % доверительный интервал), подробности см. в [таблице относительной погрешности](https://datasketches.apache.org/docs/Theta/ThetaErrorTable.html). - -**См. также** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md deleted file mode 100644 index 8871e21258d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: 'Вычисляет дисперсию генеральной совокупности.' -sidebar_position: 210 -slug: /sql-reference/aggregate-functions/reference/varPop -title: 'varPop' -doc_type: 'reference' ---- - - - -## varPop {#varpop} - -Вычисляет дисперсию генеральной совокупности: - -$$ -\frac{\Sigma{(x - \bar{x})^2}}{n} -$$ - -**Синтаксис** - -```sql -varPop(x) -``` - -Псевдоним: `VAR_POP`. - -**Параметры** - -- `x`: Совокупность значений, для которой вычисляется дисперсия. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal\*](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Дисперсия генеральной совокупности `x`. [`Float64`](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3), (3), (3), (4), (4), (5), (5), (7), (11), (15); - -SELECT - varPop(x) AS var_pop -FROM test_data; -``` - -Результат: - -```response -┌─var_pop─┐ -│ 14.4 │ -└─────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md deleted file mode 100644 index 8f2220c244f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: 'Возвращает генеральную дисперсию. В отличие от varPop, эта функция использует численно устойчивый алгоритм. Она работает медленнее, но обеспечивает меньшую вычислительную погрешность.' -sidebar_position: 211 -slug: /sql-reference/aggregate-functions/reference/varpopstable -title: 'varPopStable' -doc_type: 'reference' ---- - -## varPopStable {#varpopstable} - -Возвращает дисперсию генеральной совокупности. В отличие от [`varPop`](../reference/varpop.md), эта функция использует [численно устойчивый](https://en.wikipedia.org/wiki/Numerical_stability) алгоритм. Она работает медленнее, но обеспечивает меньшую вычислительную погрешность. - -**Синтаксис** - -```sql -varPopStable(x) -``` - -Псевдоним: `VAR_POP_STABLE`. - -**Параметры** - -* `x`: набор значений, для которого вычисляется генеральная дисперсия. [(U)Int*](../../data-types/int-uint.md), [Float*](../../data-types/float.md), [Decimal*](../../data-types/decimal.md). - -**Возвращаемое значение** - -* Возвращает генеральную дисперсию для `x`. [Float64](../../data-types/float.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - varPopStable(x) AS var_pop_stable -FROM test_data; -``` - -Результат: - -```response -┌─var_pop_stable─┐ -│ 14.4 │ -└────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md deleted file mode 100644 index c07d0575f3c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -description: 'Вычисляет выборочную дисперсию набора данных.' -sidebar_position: 212 -slug: /sql-reference/aggregate-functions/reference/varSamp -title: 'varSamp' -doc_type: 'reference' ---- - - - -## varSamp {#varsamp} - -Вычисляет выборочную дисперсию набора данных. - -**Синтаксис** - -```sql -varSamp(x) -``` - -Псевдоним: `VAR_SAMP`. - -**Параметры** - -- `x`: Набор данных, для которого требуется вычислить выборочную дисперсию. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal\*](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Возвращает выборочную дисперсию входного набора данных `x`. [Float64](../../data-types/float.md). - -**Детали реализации** - -Функция `varSamp` вычисляет выборочную дисперсию по следующей формуле: - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -Где: - -- `x` — каждая отдельная точка данных в наборе данных. -- `mean(x)` — среднее арифметическое набора данных. -- `n` — количество точек данных в наборе данных. - -Функция предполагает, что входной набор данных представляет собой выборку из более крупной генеральной совокупности. Если требуется вычислить дисперсию всей генеральной совокупности (при наличии полного набора данных), следует использовать [`varPop`](../reference/varpop.md). - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSamp(x),3) AS var_samp FROM test_data; -``` - -Ответ: - -```response -┌─var_samp─┐ -│ 0.865 │ -└──────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md deleted file mode 100644 index 992b150ec7a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'Вычисляет выборочную дисперсию набора данных. В отличие от `varSamp` эта функция использует численно устойчивый алгоритм. Работает медленнее, но даёт меньшую вычислительную погрешность.' -sidebar_position: 213 -slug: /sql-reference/aggregate-functions/reference/varsampstable -title: 'varSampStable' -doc_type: 'reference' ---- - - - -## varSampStable {#varsampstable} - -Вычисляет выборочную дисперсию набора данных. В отличие от [`varSamp`](../reference/varsamp.md), эта функция использует численно устойчивый алгоритм. Работает медленнее, но обеспечивает меньшую вычислительную погрешность. - -**Синтаксис** - -```sql -varSampStable(x) -``` - -Псевдоним: `VAR_SAMP_STABLE` - -**Параметры** - -- `x`: Набор данных, для которого требуется вычислить выборочную дисперсию. [(U)Int\*](../../data-types/int-uint.md), [Float\*](../../data-types/float.md), [Decimal\*](../../data-types/decimal.md). - -**Возвращаемое значение** - -- Возвращает выборочную дисперсию входного набора данных. [Float64](../../data-types/float.md). - -**Детали реализации** - -Функция `varSampStable` вычисляет выборочную дисперсию по той же формуле, что и [`varSamp`](../reference/varsamp.md): - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -Где: - -- `x` — каждое отдельное значение в наборе данных. -- `mean(x)` — среднее арифметическое набора данных. -- `n` — количество значений в наборе данных. - -**Пример** - -Запрос: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSampStable(x),3) AS var_samp_stable FROM test_data; -``` - -Ответ: - -```response -┌─var_samp_stable─┐ -│ 0.865 │ -└─────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md deleted file mode 100644 index e540c5cd14b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'Применяет t‑критерий Уэлча к выборкам из двух генеральных совокупностей.' -sidebar_label: 'welchTTest' -sidebar_position: 214 -slug: /sql-reference/aggregate-functions/reference/welchttest -title: 'welchTTest' -doc_type: 'reference' ---- - -# welchTTest {#welchttest} - -Применяет t-критерий Уэлча к выборкам из двух генеральных совокупностей. - -**Синтаксис** - -```sql -welchTTest([confidence_level])(sample_data, sample_index) -``` - -Значения обеих выборок находятся в столбце `sample_data`. Если `sample_index` равен 0, то значение в этой строке относится к выборке из первой совокупности. В противном случае оно относится к выборке из второй совокупности. -Нулевая гипотеза заключается в том, что средние значения совокупностей равны. Предполагается нормальное распределение. Дисперсии совокупностей могут быть неравными. - -**Аргументы** - -* `sample_data` — Данные выборки. [Integer](../../../sql-reference/data-types/int-uint.md), [Float](../../../sql-reference/data-types/float.md) или [Decimal](../../../sql-reference/data-types/decimal.md). -* `sample_index` — Индекс выборки. [Integer](../../../sql-reference/data-types/int-uint.md). - -**Параметры** - -* `confidence_level` — Уровень доверия, используемый для вычисления доверительных интервалов. [Float](../../../sql-reference/data-types/float.md). - -**Возвращаемые значения** - -[Tuple](../../../sql-reference/data-types/tuple.md) с двумя или четырьмя элементами (если указан необязательный параметр `confidence_level`) - -* вычисленная t-статистика. [Float64](../../../sql-reference/data-types/float.md). -* вычисленное p-значение. [Float64](../../../sql-reference/data-types/float.md). -* нижняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md). -* верхняя граница доверительного интервала. [Float64](../../../sql-reference/data-types/float.md). - -**Пример** - -Входная таблица: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 22.1 │ 0 │ -│ 21.9 │ 0 │ -│ 18.9 │ 1 │ -│ 20.3 │ 1 │ -│ 19 │ 1 │ -└─────────────┴──────────────┘ -``` - -Запрос: - -```sql -SELECT welchTTest(sample_data, sample_index) FROM welch_ttest; -``` - -Результат: - -```text -┌─welchTTest(sample_data, sample_index)─────┐ -│ (2.7988719532211235,0.051807360348581945) │ -└───────────────────────────────────────────┘ -``` - -**См. также** - -* [t-критерий Уэлча](https://en.wikipedia.org/wiki/Welch%27s_t-test) -* [функция studentTTest](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md deleted file mode 100644 index cf71dd8ea91..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -description: 'Документация по типу данных AggregateFunction в ClickHouse, который -хранит промежуточные состояния агрегатных функций' -keywords: ['AggregateFunction', 'Type'] -sidebar_label: 'AggregateFunction' -sidebar_position: 46 -slug: /sql-reference/data-types/aggregatefunction -title: 'Тип AggregateFunction' -doc_type: 'reference' ---- - -# Тип AggregateFunction {#aggregatefunction-type} - -## Описание {#description} - -Все [агрегатные функции](/sql-reference/aggregate-functions) в ClickHouse имеют -промежуточное состояние, зависящее от реализации, которое может быть -сериализовано в тип данных `AggregateFunction` и сохранено в таблице. Обычно это -делается с помощью [материализованного представления](../../sql-reference/statements/create/view.md). - -С типом `AggregateFunction` обычно используются два [комбинатора](/sql-reference/aggregate-functions/combinators) агрегатных функций: - -- Комбинатор агрегатной функции [`-State`](/sql-reference/aggregate-functions/combinators#-state), который при добавлении к имени агрегатной - функции формирует промежуточные состояния `AggregateFunction`. -- Комбинатор агрегатной функции [`-Merge`](/sql-reference/aggregate-functions/combinators#-merge), который используется для получения - конечного результата агрегации из промежуточных состояний. - -## Синтаксис {#syntax} - -```sql -AggregateFunction(имя_агрегатной_функции, типы_аргументов...) -``` - -**Параметры** - -* `aggregate_function_name` - Имя агрегатной функции. Если функция - параметрическая, необходимо также указать её параметры. -* `types_of_arguments` - Типы аргументов агрегатной функции. - -Например: - -```sql -CREATE TABLE t -( - column1 AggregateFunction(uniq, UInt64), - column2 AggregateFunction(anyIf, String, UInt8), - column3 AggregateFunction(quantiles(0.5, 0.9), UInt64) -) ENGINE = ... -``` - - -## Использование {#usage} - -### Вставка данных {#data-insertion} - -Чтобы вставить данные в таблицу со столбцами типа `AggregateFunction`, вы можете -использовать `INSERT SELECT` с агрегатными функциями и -комбинатором агрегатных функций -[`-State`](/sql-reference/aggregate-functions/combinators#-state). - -Например, чтобы вставить данные в столбцы типов `AggregateFunction(uniq, UInt64)` и -`AggregateFunction(quantiles(0.5, 0.9), UInt64)`, нужно использовать следующие -агрегатные функции с комбинаторами. - -```sql -uniqState(UserID) -quantilesState(0.5, 0.9)(SendTiming) -``` - -В отличие от функций `uniq` и `quantiles`, `uniqState` и `quantilesState` -(с добавленным комбинатором `-State`) возвращают состояние, а не итоговое значение. -Другими словами, они возвращают значение типа `AggregateFunction`. - -В результатах запроса `SELECT` значения типа `AggregateFunction` имеют -зависящее от реализации двоичное представление во всех форматах вывода ClickHouse. - -Существует специальный параметр уровня сессии `aggregate_function_input_format`, который позволяет формировать состояние из входных значений. -Он поддерживает следующие форматы: - -* `state` — двоичная строка с сериализованным состоянием (значение по умолчанию). - Если вы выгружаете данные, например, в формат `TabSeparated` с помощью запроса `SELECT`, - то этот дамп можно загрузить обратно с помощью запроса `INSERT`. -* `value` — формат будет ожидать одно значение аргумента агрегатной функции или, в случае нескольких аргументов, кортеж из них; это значение будет десериализовано для формирования соответствующего состояния. -* `array` — формат будет ожидать Array значений, как описано в варианте `value` выше; все элементы массива будут агрегированы для формирования состояния. - - -### Выборка данных {#data-selection} - -При выборке данных из таблицы `AggregatingMergeTree` используйте предложение `GROUP BY` -и те же агрегатные функции, что и при вставке данных, но с комбинатором -[`-Merge`](/sql-reference/aggregate-functions/combinators#-merge). - -Агрегатная функция с добавленным комбинатором `-Merge` принимает набор -состояний, объединяет их и возвращает результат полной агрегации данных. - -Например, следующие два запроса возвращают один и тот же результат: - -```sql -SELECT uniq(UserID) FROM table - -SELECT uniqMerge(state) FROM (SELECT uniqState(UserID) AS state FROM table GROUP BY RegionID) -``` - - -## Пример использования {#usage-example} - -См. описание табличного движка [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md). - -## Связанные материалы {#related-content} - -- Запись в блоге: [Использование агрегатных комбинаторов в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -- [MergeState](/sql-reference/aggregate-functions/combinators#-mergestate) - комбинатор MergeState. -- [State](/sql-reference/aggregate-functions/combinators#-state) комбинатор State. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md deleted file mode 100644 index d4e6f55e416..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'Документация по типу данных Array в ClickHouse' -sidebar_label: 'Array(T)' -sidebar_position: 32 -slug: /sql-reference/data-types/array -title: 'Array(T)' -doc_type: 'reference' ---- - -# Array(T) {#arrayt} - -Массив элементов типа `T` с индексацией, начинающейся с 1. `T` может быть любым типом данных, включая массив. - -## Создание массива {#creating-an-array} - -Для создания массива можно использовать функцию: - -```sql -array(T) -``` - -Можно также использовать квадратные скобки. - -```sql -[] -``` - -Пример создания массива: - -```sql -SELECT array(1, 2) AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName(array(1, 2))─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴─────────────────────────┘ -``` - -```sql -SELECT [1, 2] AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName([1, 2])─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴────────────────────┘ -``` - -## Работа с типами данных {#working-with-data-types} - -При создании массива «на лету» ClickHouse автоматически определяет тип аргумента как самый узкий тип данных, который может хранить все перечисленные аргументы. Если в массиве есть значения [Nullable](/sql-reference/data-types/nullable) или литералы [NULL](/operations/settings/formats#input_format_null_as_default), тип элемента массива также становится [Nullable](../../sql-reference/data-types/nullable.md). - -Если ClickHouse не может определить тип данных, генерируется исключение. Например, это происходит при попытке создать массив, содержащий одновременно строки и числа (`SELECT array(1, 'a')`). - -Примеры автоматического определения типа данных: - -```sql -SELECT array(1, 2, NULL) AS x, toTypeName(x) -``` - -```text -┌─x──────────┬─toTypeName(array(1, 2, NULL))─┐ -│ [1,2,NULL] │ Array(Nullable(UInt8)) │ -└────────────┴───────────────────────────────┘ -``` - -Если вы попытаетесь создать массив несовместимых типов данных, ClickHouse выбросит исключение: - -```sql -SELECT array(1, 'a') -``` - -```text -Получено исключение от сервера (версия 1.1.54388): -Код: 386. DB::Exception: Получено от localhost:9000, 127.0.0.1. DB::Exception: Отсутствует общий супертип для типов UInt8, String, так как часть из них относится к String/FixedString, а часть — нет. -``` - -## Размер массива {#array-size} - -Можно определить размер массива, используя подстолбец `size0`, не считывая весь столбец целиком. Для многомерных массивов вы можете использовать `sizeN-1`, где `N` — требуемая размерность. - -**Пример** - -Запрос: - -```sql -CREATE TABLE t_arr (`arr` Array(Array(Array(UInt32)))) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO t_arr VALUES ([[[12, 13, 0, 1],[12]]]); - -SELECT arr.size0, arr.size1, arr.size2 FROM t_arr; -``` - -Результат: - -```text -┌─arr.size0─┬─arr.size1─┬─arr.size2─┐ -│ 1 │ [2] │ [[4,1]] │ -└───────────┴───────────┴───────────┘ -``` - -## Чтение вложенных подколонок из Array {#reading-nested-subcolumns-from-array} - -Если вложенный тип `T` внутри `Array` имеет подколонки (например, если это [именованный кортеж](./tuple.md)), вы можете читать его подколонки из типа `Array(T)` с теми же именами подколонок. Тип подколонки будет `Array` от типа исходной подколонки. - -**Пример** - -```sql -CREATE TABLE t_arr (arr Array(Tuple(field1 UInt32, field2 String))) ENGINE = MergeTree ORDER BY tuple(); -INSERT INTO t_arr VALUES ([(1, 'Hello'), (2, 'World')]), ([(3, 'This'), (4, 'is'), (5, 'subcolumn')]); -SELECT arr.field1, toTypeName(arr.field1), arr.field2, toTypeName(arr.field2) from t_arr; -``` - -```test -┌─arr.field1─┬─toTypeName(arr.field1)─┬─arr.field2────────────────┬─toTypeName(arr.field2)─┐ -│ [1,2] │ Array(UInt32) │ ['Hello','World'] │ Array(String) │ -│ [3,4,5] │ Array(UInt32) │ ['This','is','subcolumn'] │ Array(String) │ -└────────────┴────────────────────────┴───────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md deleted file mode 100644 index 20f82e5e11f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'Документация по булевому типу данных в ClickHouse' -sidebar_label: 'Булевый' -sidebar_position: 33 -slug: /sql-reference/data-types/boolean -title: 'Bool' -doc_type: 'reference' ---- - -# Bool {#bool} - -Тип `bool` внутренне хранится как UInt8. Возможные значения: `true` (1), `false` (0). - -```sql -SELECT true AS col, toTypeName(col); -┌─col──┬─toTypeName(true)─┐ -│ true │ Bool │ -└──────┴──────────────────┘ - -select true == 1 as col, toTypeName(col); -┌─col─┬─toTypeName(equals(true, 1))─┐ -│ 1 │ UInt8 │ -└─────┴─────────────────────────────┘ -``` - -```sql -CREATE TABLE test_bool -( - `A` Int64, - `B` Bool -) -ENGINE = Memory; - -INSERT INTO test_bool VALUES (1, true),(2,0); - -SELECT * FROM test_bool; -┌─A─┬─B─────┐ -│ 1 │ true │ -│ 2 │ false │ -└───┴───────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md deleted file mode 100644 index 27fa1675143..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: 'Документация по спецификации двоичного кодирования типов данных' -sidebar_label: 'Спецификация двоичного кодирования типов данных' -sidebar_position: 56 -slug: /sql-reference/data-types/data-types-binary-encoding -title: 'Спецификация двоичного кодирования типов данных' -doc_type: 'reference' ---- - -# Спецификация двоичного кодирования типов данных {#data-types-binary-encoding-specification} - -В этой спецификации описывается двоичный формат, который может использоваться для двоичного кодирования и декодирования типов данных ClickHouse. Этот формат используется в [двоичной сериализации](dynamic.md#binary-output-format) столбца `Dynamic` и может использоваться во входных и выходных форматах [RowBinaryWithNamesAndTypes](/interfaces/formats/RowBinaryWithNamesAndTypes) и [Native](/interfaces/formats/Native) при соответствующих настройках. - -В таблице ниже описано, как каждый тип данных представляется в двоичном формате. Кодирование каждого типа данных состоит из 1 байта, обозначающего тип, и некоторой дополнительной необязательной информации. -`var_uint` в двоичном кодировании означает, что размер закодирован с использованием кодирования переменной длины (Variable-Length Quantity). - -| Тип данных ClickHouse | Двоичное кодирование | -| --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Ничего` | `0x00` | -| `UInt8` | `0x01` | -| `UInt16` | `0x02` | -| `UInt32` | `0x03` | -| `UInt64` | `0x04` | -| `UInt128` | `0x05` | -| `UInt256` | `0x06` | -| `Int8` | `0x07` | -| `Int16` | `0x08` | -| `Int32` | `0x09` | -| `Int64` | `0x0A` | -| `Int128` | `0x0B` | -| `Int256` | `0x0C` | -| `Float32` | `0x0D` | -| `Float64` | `0x0E` | -| `Дата` | `0x0F` | -| `Date32` | `0x10` | -| `DateTime` | `0x11` | -| `DateTime(time_zone)` | `0x12` | -| `DateTime64(P)` | `0x13` | -| `DateTime64(P, time_zone)` | `0x14` | -| `String` | `0x15` | -| `FixedString(N)` | `0x16` | -| `Enum8` | `0x17...` | -| `Enum16` | `0x18...>` | -| `Decimal32(P, S)` | `0x19` | -| `Decimal64(P, S)` | `0x1A` | -| `Decimal128(P, S)` | `0x1B` | -| `Decimal256(P, S)` | `0x1C` | -| `UUID` | `0x1D` | -| `Array(T)` | `0x1E` | -| `Tuple(T1, ..., TN)` | `0x1F...` | -| `Tuple(name1 T1, ..., nameN TN)` | `0x20...` | -| `Установить` | `0x21` | -| `Интервал` | `0x22` (см. [двоичное кодирование вида интервала](#interval-kind-binary-encoding)) | -| `Nullable(T)` | `0x23` | -| `Функция` | `0x24...` | -| `AggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x25......` (см. [двоичное кодирование параметров агрегатной функции](#aggregate-function-parameter-binary-encoding)) | -| `LowCardinality(T)` | `0x26` | -| `Map(K, V)` | `0x27` | -| `IPv4` | `0x28` | -| `IPv6` | `0x29` | -| `Variant(T1, ..., TN)` | `0x2A...` | -| `Dynamic(max_types=N)` | `0x2B` | -| `Пользовательский тип` (`Ring`, `Polygon` и т. д.) | `0x2C` | -| `Bool` | `0x2D` | -| `SimpleAggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x2E......` (см. [двоичное кодирование параметров агрегатной функции](#aggregate-function-parameter-binary-encoding)) | -| `Nested(name1 T1, ..., nameN TN)` | `0x2F...` | -| `JSON(max_dynamic_paths=N, max_dynamic_types=M, path Type, SKIP skip_path, SKIP REGEXP skip_path_regexp)` | `0x30.........` | -| `BFloat16` | `0x31` | -| `Time` | `0x32` | -| `Time64(P)` | `0x34` | -| `QBit(T, N)` | `0x36` | - -Для типа `JSON` байт `uint8_serialization_version` указывает версию сериализации. Сейчас версия всегда равна 0, но в будущем может измениться, если для типа `JSON` будут введены новые аргументы. - -### Двоичное кодирование видов интервалов {#interval-kind-binary-encoding} - -В таблице ниже показано, как в двоичном формате кодируются различные виды интервалов типа данных `Interval`. - -| Вид интервала | Двоичное кодирование | -|---------------|----------------------| -| `Nanosecond` | `0x00` | -| `Microsecond` | `0x01` | -| `Millisecond` | `0x02` | -| `Second` | `0x03` | -| `Minute` | `0x04` | -| `Hour` | `0x05` | -| `Day` | `0x06` | -| `Week` | `0x07` | -| `Month` | `0x08` | -| `Quarter` | `0x09` | -| `Year` | `0x1A` | - -### Бинарное кодирование параметров агрегатной функции {#aggregate-function-parameter-binary-encoding} - -В таблице ниже описано, как кодируются параметры `AggregateFunction` и `SimpleAggregateFunction`. -Кодирование параметра состоит из одного байта, обозначающего тип параметра, и самого значения. - -| Тип параметра | Бинарное кодирование | -|--------------------------|--------------------------------------------------------------------------------------------------------------------------------| -| `Null` | `0x00` | -| `UInt64` | `0x01` | -| `Int64` | `0x02` | -| `UInt128` | `0x03` | -| `Int128` | `0x04` | -| `UInt256` | `0x05` | -| `Int256` | `0x06` | -| `Float64` | `0x07` | -| `Decimal32` | `0x08` | -| `Decimal64` | `0x09` | -| `Decimal128` | `0x0A` | -| `Decimal256` | `0x0B` | -| `String` | `0x0C` | -| `Array` | `0x0D...` | -| `Tuple` | `0x0E...` | -| `Map` | `0x0F...` | -| `IPv4` | `0x10` | -| `IPv6` | `0x11` | -| `UUID` | `0x12` | -| `Bool` | `0x13` | -| `Object` | `0x14...` | -| `AggregateFunctionState` | `0x15` | -| `Negative infinity` | `0xFE` | -| `Positive infinity` | `0xFF` | \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md deleted file mode 100644 index de918d8dc4d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: 'Документация по типу данных Date в ClickHouse' -sidebar_label: 'Date' -sidebar_position: 12 -slug: /sql-reference/data-types/date -title: 'Date' -doc_type: 'reference' ---- - -# Date {#date} - -Дата. Хранится в двух байтах как количество дней, прошедших с 1970-01-01 (беззнаковое целое число). Позволяет хранить значения от момента сразу после начала эпохи Unix до верхней границы, определённой константой на этапе компиляции (в настоящее время — до 2149 года, но последний полностью поддерживаемый год — 2148). - -Поддерживаемый диапазон значений: [1970-01-01, 2149-06-06]. - -Значение даты хранится без часового пояса. - -**Пример** - -Создание таблицы со столбцом типа `Date` и вставка в него данных: - -```sql -CREATE TABLE dt -( - `timestamp` Date, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- Парсинг Date --- - из строки, --- - из «малого» целого числа, интерпретируемого как количество дней с 1970-01-01, и --- - из «большого» целого числа, интерпретируемого как количество секунд с 1970-01-01. -INSERT INTO dt VALUES ('2019-01-01', 1), (17897, 2), (1546300800, 3); - -SELECT * FROM dt; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2019-01-01 │ 1 │ -│ 2019-01-01 │ 2 │ -│ 2019-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**См. также** - -* [Функции для работы с датами и временем](../../sql-reference/functions/date-time-functions.md) -* [Операторы для работы с датами и временем](../../sql-reference/operators#operators-for-working-with-dates-and-times) -* [Тип данных `DateTime`](../../sql-reference/data-types/datetime.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md deleted file mode 100644 index ea850f45063..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'Документация по типу данных Date32 в ClickHouse, который хранит даты - с более широким диапазоном значений по сравнению с Date' -sidebar_label: 'Date32' -sidebar_position: 14 -slug: /sql-reference/data-types/date32 -title: 'Date32' -doc_type: 'reference' ---- - -# Date32 {#date32} - -Дата. Поддерживает тот же диапазон дат, что и [DateTime64](../../sql-reference/data-types/datetime64.md). Хранится как знаковое 32-битное целое число в нативном порядке байтов, значение которого равно количеству дней, прошедших с `1900-01-01`. **Важно!** 0 соответствует `1970-01-01`, а отрицательные значения — дням до `1970-01-01`. - -**Примеры** - -Создание таблицы со столбцом типа `Date32` и вставка данных в него: - -```sql -CREATE TABLE dt32 -( - `timestamp` Date32, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- Разбор даты --- - из строки, --- - из «малого» целого числа, интерпретируемого как количество дней с 1970-01-01, и --- - из «большого» целого числа, интерпретируемого как количество секунд с 1970-01-01. -INSERT INTO dt32 VALUES ('2100-01-01', 1), (47482, 2), (4102444800, 3); - -SELECT * FROM dt32; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2100-01-01 │ 1 │ -│ 2100-01-01 │ 2 │ -│ 2100-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**Смотрите также** - -* [toDate32](../../sql-reference/functions/type-conversion-functions.md#todate32) -* [toDate32OrZero](/sql-reference/functions/type-conversion-functions#todate32orzero) -* [toDate32OrNull](/sql-reference/functions/type-conversion-functions#todate32ornull) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md deleted file mode 100644 index a3d6211f21a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md +++ /dev/null @@ -1,209 +0,0 @@ ---- -description: 'Документация по типу данных DateTime в ClickHouse, предназначенному - для хранения временных меток с точностью до секунды' -sidebar_label: 'DateTime' -sidebar_position: 16 -slug: /sql-reference/data-types/datetime -title: 'DateTime' -doc_type: 'reference' ---- - - - -# DateTime {#datetime} - -Тип `DateTime` позволяет хранить момент времени, который может быть выражен как календарная дата и время суток. - -Синтаксис: - -```sql -DateTime([timezone]) -``` - -Поддерживаемый диапазон значений: [1970-01-01 00:00:00, 2106-02-07 06:28:15]. - -Точность: 1 секунда. - - -## Скорость {#speed} - -Тип данных `Date` работает быстрее, чем `DateTime` в _большинстве_ случаев. - -Тип `Date` занимает 2 байта, тогда как `DateTime` — 4. Однако при сжатии разница в размере между `Date` и `DateTime` становится более заметной. Это связано с тем, что минуты и секунды в `DateTime` хуже поддаются сжатию. Фильтрация и агрегация по `Date` вместо `DateTime` также выполняются быстрее. - - - -## Замечания по использованию {#usage-remarks} - -Момент времени сохраняется в виде [Unix timestamp](https://en.wikipedia.org/wiki/Unix_time), независимо от часового пояса или перехода на летнее время. Часовой пояс влияет на то, как значения типа `DateTime` отображаются в текстовом формате и как разбираются значения, заданные в виде строк ('2020-01-01 05:00:01'). - -Независимый от часового пояса Unix timestamp хранится в таблицах, а часовой пояс используется для преобразования его в текстовый формат или обратно при импорте/экспорте данных, а также для выполнения календарных вычислений над значениями (например, функциями `toDate`, `toHour` и т. д.). Часовой пояс не хранится в строках таблицы (или в результатах запроса), а хранится в метаданных столбца. - -Список поддерживаемых часовых поясов можно найти в [IANA Time Zone Database](https://www.iana.org/time-zones), а также получить запросом `SELECT * FROM system.time_zones`. [Список](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) также доступен в Википедии. - -Вы можете явно задать часовой пояс для столбцов типа `DateTime` при создании таблицы. Пример: `DateTime('UTC')`. Если часовой пояс не задан, ClickHouse использует значение параметра [timezone](../../operations/server-configuration-parameters/settings.md#timezone) в настройках сервера или настройки операционной системы на момент запуска сервера ClickHouse. - -По умолчанию [clickhouse-client](../../interfaces/cli.md) использует часовой пояс сервера, если часовой пояс явно не задан при инициализации типа данных. Чтобы использовать часовой пояс клиента, запустите `clickhouse-client` с параметром `--use_client_time_zone`. - -ClickHouse выводит значения в зависимости от значения настройки [date_time_output_format](../../operations/settings/settings-formats.md#date_time_output_format). По умолчанию используется текстовый формат `YYYY-MM-DD hh:mm:ss`. Кроме того, вы можете изменить формат вывода с помощью функции [formatDateTime](../../sql-reference/functions/date-time-functions.md#formatDateTime). - -При вставке данных в ClickHouse вы можете использовать различные форматы строк даты и времени в зависимости от значения настройки [date_time_input_format](../../operations/settings/settings-formats.md#date_time_input_format). - - - -## Примеры {#examples} - -**1.** Создание таблицы со столбцом типа `DateTime` и вставка в неё данных: - -```sql -CREATE TABLE dt -( - `timestamp` DateTime('Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- Парсинг DateTime --- - из строки, --- - из целого числа, интерпретируемого как количество секунд с 1970-01-01. -INSERT INTO dt VALUES ('2019-01-01 00:00:00', 1), (1546300800, 2); - -SELECT * FROM dt; -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -│ 2019-01-01 03:00:00 │ 2 │ -└─────────────────────┴──────────┘ -``` - -* При вставке значения типа datetime в виде целого числа оно интерпретируется как Unix timestamp (UTC). `1546300800` соответствует `'2019-01-01 00:00:00'` UTC. Однако, так как для столбца `timestamp` задан часовой пояс `Asia/Istanbul` (UTC+3), при выводе в виде строки значение будет показано как `'2019-01-01 03:00:00'`. -* При вставке строкового значения как datetime оно интерпретируется как значение в часовом поясе столбца. `'2019-01-01 00:00:00'` будет интерпретировано как значение в часовом поясе `Asia/Istanbul` и сохранено как `1546290000`. - -**2.** Фильтрация по значениям `DateTime` - -```sql -SELECT * FROM dt WHERE timestamp = toDateTime('2019-01-01 00:00:00', 'Asia/Istanbul') -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -Значения столбца `DateTime` можно фильтровать с помощью строкового значения в предикате `WHERE`. Оно будет автоматически приведено к типу `DateTime`: - -```sql -SELECT * FROM dt WHERE timestamp = '2019-01-01 00:00:00' -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -**3.** Получение часового пояса для столбца с типом данных `DateTime`: - -```sql -SELECT toDateTime(now(), 'Asia/Istanbul') AS column, toTypeName(column) AS x -``` - -```text -┌──────────────column─┬─x─────────────────────────┐ -│ 2019-10-16 04:12:04 │ DateTime('Asia/Istanbul') │ -└─────────────────────┴───────────────────────────┘ -``` - -**4.** Конвертация часовых поясов - -```sql -SELECT -toDateTime(timestamp, 'Europe/London') AS lon_time, -toDateTime(timestamp, 'Asia/Istanbul') AS ist_time -FROM dt -``` - -```text -┌───────────lon_time──┬────────────mos_time─┐ -│ 2019-01-01 00:00:00 │ 2019-01-01 03:00:00 │ -│ 2018-12-31 21:00:00 │ 2019-01-01 00:00:00 │ -└─────────────────────┴─────────────────────┘ -``` - -Поскольку преобразование часового пояса затрагивает только метаданные, эта операция не требует вычислительных ресурсов. - - -## Ограничения поддержки часовых поясов {#limitations-on-time-zones-support} - -Некоторые часовые пояса могут поддерживаться не полностью. Возможны следующие случаи: - -Если смещение от UTC не кратно 15 минутам, вычисление часов и минут может быть некорректным. Например, часовой пояс в Монровии (Либерия) имел смещение UTC −0:44:30 до 7 января 1972 года. Если вы выполняете вычисления с историческим временем в часовом поясе Монровии, функции обработки времени могут выдавать некорректные результаты. Тем не менее результаты после 7 января 1972 года будут корректными. - -Если переход времени (из‑за летнего времени или по другим причинам) был выполнен в момент времени, не кратный 15 минутам, вы также можете получить некорректные результаты в этот конкретный день. - -Немонотонные календарные даты. Например, в Happy Valley - Goose Bay время было переведено на один час назад в 00:01:00 7 ноября 2010 года (через одну минуту после полуночи). Таким образом, после окончания 6 ноября люди наблюдали целую одну минуту 7 ноября, затем время было изменено обратно на 23:01 6 ноября, и после следующих 59 минут 7 ноября началось снова. ClickHouse (пока) не поддерживает такие особенности. В эти дни результаты работы функций обработки времени могут быть слегка некорректными. - -Аналогичная проблема существует для антарктической станции Casey в 2010 году. Там перевели время на три часа назад 5 марта в 02:00. Если вы работаете на антарктической станции, пожалуйста, не бойтесь использовать ClickHouse. Просто убедитесь, что вы установили часовой пояс в UTC или учитываете возможные неточности. - -Смещения времени на несколько дней. Некоторые тихоокеанские острова изменили своё смещение часового пояса с UTC+14 на UTC-12. Это допустимо, но могут возникать некоторые неточности, если вы выполняете вычисления с их часовым поясом для исторических моментов времени в дни перехода. - - - -## Обработка перехода на летнее время (DST) {#handling-daylight-saving-time-dst} - -Тип `DateTime` с часовыми поясами в ClickHouse может вести себя неожиданным образом во время переходов на летнее время (Daylight Saving Time, DST), в частности, когда: - -* [`date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) установлен в значение `simple`. -* Часы переводятся назад ("Fall Back"), что приводит к перекрытию одного часа. -* Часы переводятся вперёд ("Spring Forward"), что приводит к пропуску одного часа. - -По умолчанию ClickHouse всегда выбирает более раннее из двух перекрывающихся значений времени и может интерпретировать несуществующие моменты времени при переводе часов вперёд. - -Например, рассмотрим следующий переход с летнего времени (DST) на стандартное время. - -* 29 октября 2023 года в 02:00:00 часы переводятся назад на 01:00:00 (BST → GMT). -* Промежуток 01:00:00 – 01:59:59 встречается дважды (один раз в BST и один раз в GMT). -* ClickHouse всегда выбирает первое появление (BST), что приводит к неожиданным результатам при добавлении временных интервалов. - -```sql -SELECT '2023-10-29 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-10-29 01:30:00 │ 2023-10-29 01:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -Аналогично, во время перехода со стандартного на летнее время может показаться, что один час пропущен. - -Например: - -* 26 марта 2023 года в `00:59:59` часы перескакивают вперёд на 02:00:00 (GMT → BST). -* Час `01:00:00` – `01:59:59` не существует. - -```sql -SELECT '2023-03-26 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-03-26 00:30:00 │ 2023-03-26 02:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -В этом случае ClickHouse переносит несуществующее время `2023-03-26 01:30:00` на `2023-03-26 00:30:00`. - - -## См. также {#see-also} - -- [Функции преобразования типов](../../sql-reference/functions/type-conversion-functions.md) -- [Функции для работы с датами и временем](../../sql-reference/functions/date-time-functions.md) -- [Функции для работы с массивами](../../sql-reference/functions/array-functions.md) -- [Настройка `date_time_input_format`](../../operations/settings/settings-formats.md#date_time_input_format) -- [Настройка `date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) -- [Параметр конфигурации сервера `timezone`](../../operations/server-configuration-parameters/settings.md#timezone) -- [Настройка `session_timezone`](../../operations/settings/settings.md#session_timezone) -- [Операторы для работы с датами и временем](../../sql-reference/operators#operators-for-working-with-dates-and-times) -- [Тип данных `Date`](../../sql-reference/data-types/date.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md deleted file mode 100644 index d842f919588..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: 'Документация по типу данных DateTime64 в ClickHouse, который хранит - временные метки с точностью до долей секунды' -sidebar_label: 'DateTime64' -sidebar_position: 18 -slug: /sql-reference/data-types/datetime64 -title: 'DateTime64' -doc_type: 'reference' ---- - -# DateTime64 {#datetime64} - -Позволяет хранить момент времени, который может быть выражен как календарная дата и время суток, с заданной точностью до долей секунды. - -Размер тика (точность): 10-precision секунды. Допустимый диапазон: от 0 до 9. -Обычно используются значения 3 (миллисекунды), 6 (микросекунды), 9 (наносекунды). - -**Синтаксис:** - -```sql -DateTime64(precision, [timezone]) -``` - -Внутренне данные хранятся как количество «тиков» с начала эпохи (1970-01-01 00:00:00 UTC) в виде Int64. Точность определяется параметром precision. Дополнительно тип `DateTime64` может хранить часовой пояс, общий для всего столбца, который влияет на то, как значения типа `DateTime64` отображаются в текстовом формате и как интерпретируются значения, заданные в виде строк ('2020-01-01 05:00:01.000'). Часовой пояс не хранится в строках таблицы (или в результате выборки), а сохраняется в метаданных столбца. Подробности см. в [DateTime](../../sql-reference/data-types/datetime.md). - -Поддерживаемый диапазон значений: [1900-01-01 00:00:00, 2299-12-31 23:59:59.999999999] - -Количество цифр после десятичной точки зависит от параметра precision. - -Примечание: точность максимального значения равна 8. Если используется максимальная точность в 9 цифр (наносекунды), максимальное поддерживаемое значение — `2262-04-11 23:47:16` в UTC. - -## Примеры {#examples} - -1. Создание таблицы со столбцом типа `DateTime64` и вставка данных в неё: - -```sql -CREATE TABLE dt64 -( - `timestamp` DateTime64(3, 'Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- Разбор DateTime --- - из целого числа, интерпретируемого как количество микросекунд (из-за точности 3) с 1970-01-01, --- - из десятичного числа, интерпретируемого как количество секунд до десятичной точки, с учётом точности после десятичной точки, --- - из строки. -INSERT INTO dt64 VALUES (1546300800123, 1), (1546300800.123, 2), ('2019-01-01 00:00:00', 3); - -SELECT * FROM dt64; -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -* При вставке значения даты и времени в виде целого числа оно интерпретируется как Unix‑временная метка (UTC) с соответствующей точностью. `1546300800000` (c точностью 3) представляет `'2019-01-01 00:00:00'` UTC. Однако, так как для столбца `timestamp` указана временная зона `Asia/Istanbul` (UTC+3), при выводе в виде строки значение будет показано как `'2019-01-01 03:00:00'`. Вставка даты и времени в виде десятичного числа обрабатывается аналогично целому числу, за исключением того, что значение до десятичной точки — это Unix‑временная метка с точностью до секунд включительно, а часть после десятичной точки интерпретируется как точность. -* При вставке строкового значения как даты и времени оно интерпретируется как значение во временной зоне столбца. `'2019-01-01 00:00:00'` будет интерпретировано как значение во временной зоне `Asia/Istanbul` и сохранено как `1546290000000`. - -2. Фильтрация по значениям `DateTime64` - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64('2019-01-01 00:00:00', 3, 'Asia/Istanbul'); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -В отличие от `DateTime`, значения `DateTime64` не преобразуются из `String` автоматически. - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64(1546300800.123, 3); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -└─────────────────────────┴──────────┘ -``` - -В отличие от операции вставки, функция `toDateTime64` будет рассматривать все значения как значения с дробной частью, поэтому точность должна -указываться после десятичного разделителя. - -3. Получение часового пояса для значения типа `DateTime64`: - -```sql -SELECT toDateTime64(now(), 3, 'Asia/Istanbul') AS column, toTypeName(column) AS x; -``` - -```text -┌──────────────────column─┬─x──────────────────────────────┐ -│ 2023-06-05 00:09:52.000 │ DateTime64(3, 'Asia/Istanbul') │ -└─────────────────────────┴────────────────────────────────┘ -``` - -4. Конвертация часовых поясов - -```sql -SELECT -toDateTime64(timestamp, 3, 'Europe/London') AS lon_time, -toDateTime64(timestamp, 3, 'Asia/Istanbul') AS istanbul_time -FROM dt64; -``` - -```text -┌────────────────lon_time─┬───────────istanbul_time─┐ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2018-12-31 21:00:00.000 │ 2019-01-01 00:00:00.000 │ -└─────────────────────────┴─────────────────────────┘ -``` - -**См. также** - -* [Функции преобразования типов](../../sql-reference/functions/type-conversion-functions.md) -* [Функции для работы с датами и временем](../../sql-reference/functions/date-time-functions.md) -* [Настройка `date_time_input_format`](../../operations/settings/settings-formats.md#date_time_input_format) -* [Настройка `date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) -* [Параметр конфигурации сервера `timezone`](../../operations/server-configuration-parameters/settings.md#timezone) -* [Настройка `session_timezone`](../../operations/settings/settings.md#session_timezone) -* [Операторы для работы с датами и временем](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [Тип данных `Date`](../../sql-reference/data-types/date.md) -* [Тип данных `DateTime`](../../sql-reference/data-types/datetime.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md deleted file mode 100644 index 41307a2fea4..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -description: 'Документация по типам данных Decimal в ClickHouse, которые обеспечивают - арифметику с фиксированной запятой и настраиваемой точностью' -sidebar_label: 'Decimal' -sidebar_position: 6 -slug: /sql-reference/data-types/decimal -title: 'Decimal, Decimal(P), Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), - Decimal256(S)' -doc_type: 'reference' ---- - - - -# Decimal, Decimal(P), Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), Decimal256(S) {#decimal-decimalp-decimalp-s-decimal32s-decimal64s-decimal128s-decimal256s} - -Знаковые числа с фиксированной запятой, которые сохраняют точность при операциях сложения, вычитания и умножения. При делении младшие разряды отбрасываются (без округления). - - - -## Параметры {#parameters} - -- P — точность. Допустимый диапазон: \[1 : 76\]. Определяет, сколько десятичных цифр может иметь число (включая дробную часть). По умолчанию точность равна 10. -- S — масштаб. Допустимый диапазон: \[0 : P\]. Определяет, сколько десятичных знаков может иметь дробная часть. - -Decimal(P) эквивалентен Decimal(P, 0). Аналогично, синтаксис Decimal эквивалентен Decimal(10, 0). - -В зависимости от значения параметра P тип Decimal(P, S) является синонимом: -- P в диапазоне \[1 : 9\] — Decimal32(S) -- P в диапазоне \[10 : 18\] — Decimal64(S) -- P в диапазоне \[19 : 38\] — Decimal128(S) -- P в диапазоне \[39 : 76\] — Decimal256(S) - - - -## Диапазоны значений Decimal {#decimal-value-ranges} - -- Decimal(P, S) - ( -1 \* 10^(P - S), 1 \* 10^(P - S) ) -- Decimal32(S) - ( -1 \* 10^(9 - S), 1 \* 10^(9 - S) ) -- Decimal64(S) - ( -1 \* 10^(18 - S), 1 \* 10^(18 - S) ) -- Decimal128(S) - ( -1 \* 10^(38 - S), 1 \* 10^(38 - S) ) -- Decimal256(S) - ( -1 \* 10^(76 - S), 1 \* 10^(76 - S) ) - -Например, Decimal32(4) может содержать числа от -99999.9999 до 99999.9999 с шагом 0.0001. - - - -## Внутреннее представление {#internal-representation} - -Внутри системы данные хранятся как обычные знаковые целые числа с соответствующей разрядностью (шириной в битах). Диапазоны фактических значений, которые могут быть сохранены в памяти, немного шире указанных выше и проверяются только при преобразовании из строки. - -Поскольку современные процессоры не поддерживают 128- и 256-битные целые числа на аппаратном уровне, операции над Decimal128 и Decimal256 эмулируются. Поэтому Decimal128 и Decimal256 работают значительно медленнее, чем Decimal32/Decimal64. - - - -## Операции и тип результата {#operations-and-result-type} - -Бинарные операции над Decimal приводят к типу результата с большей разрядностью (при любом порядке аргументов). - -- `Decimal64(S1) Decimal32(S2) -> Decimal64(S)` -- `Decimal128(S1) Decimal32(S2) -> Decimal128(S)` -- `Decimal128(S1) Decimal64(S2) -> Decimal128(S)` -- `Decimal256(S1) Decimal<32|64|128>(S2) -> Decimal256(S)` - -Правила для масштаба (scale): - -- сложение, вычитание: S = max(S1, S2). -- умножение: S = S1 + S2. -- деление: S = S1. - -Для аналогичных операций между Decimal и целыми числами результат имеет тип Decimal того же размера, что и аргумент. - -Операции между Decimal и Float32/Float64 не определены. Если они необходимы, вы можете явно привести один из аргументов, используя встроенные функции toDecimal32, toDecimal64, toDecimal128 или toFloat32, toFloat64. Имейте в виду, что результат потеряет точность, а преобразование типов является вычислительно затратной операцией. - -Некоторые функции для Decimal возвращают результат типа Float64 (например, var или stddev). Промежуточные вычисления по‑прежнему могут выполняться в Decimal, что может приводить к различным результатам для входных данных Float64 и Decimal с одинаковыми значениями. - - - -## Проверка переполнения {#overflow-checks} - -При вычислениях с типом Decimal могут происходить целочисленные переполнения. Лишние цифры в дробной части отбрасываются (без округления). Избыточные цифры в целой части приводят к исключению. - -:::warning -Проверка переполнения не реализована для Decimal128 и Decimal256. В случае переполнения возвращается некорректный результат, исключение не выбрасывается. -::: - -```sql -SELECT toDecimal32(2, 4) AS x, x / 3 -``` - -```text -┌──────x─┬─divide(toDecimal32(2, 4), 3)─┐ -│ 2.0000 │ 0.6666 │ -└────────┴──────────────────────────────┘ -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, x * x -``` - -```text -DB::Exception: Значение масштаба выходит за допустимые пределы. -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -DB::Exception: Переполнение при арифметических операциях с типом Decimal. -``` - -Проверки на переполнение приводят к замедлению операций. Если известно, что переполнения невозможны, имеет смысл отключить их с помощью настройки `decimal_check_overflow`. Когда проверки отключены и происходит переполнение, результат будет некорректным: - -```sql -SET decimal_check_overflow = 0; -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -┌──────────x─┬─multiply(6, toDecimal32(4.2, 8))─┐ -│ 4.20000000 │ -17.74967296 │ -└────────────┴──────────────────────────────────┘ -``` - -Проверки на переполнение выполняются не только при арифметических операциях, но и при сравнении значений: - -```sql -SELECT toDecimal32(1, 8) < 100 -``` - -```text -DB::Exception: Невозможно выполнить сравнение. -``` - -**См. также** - -* [isDecimalOverflow](/sql-reference/functions/other-functions#isDecimalOverflow) -* [countDigits](/sql-reference/functions/other-functions#countDigits) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md deleted file mode 100644 index 8820718e4e7..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'Обзор доменных типов в ClickHouse, расширяющих базовые типы дополнительными возможностями' -sidebar_label: 'Домены' -sidebar_position: 56 -slug: /sql-reference/data-types/domains/ -title: 'Домены' -doc_type: 'reference' ---- - - - -# Домены {#domains} - -Домены — это специализированные типы, которые добавляют дополнительные возможности поверх существующих базовых типов, при этом формат передачи по сети и хранения исходного типа данных на диске остается неизменным. В настоящее время ClickHouse не поддерживает пользовательские домены. - -Вы можете использовать домены везде, где может использоваться соответствующий базовый тип, например: - -- Создавать столбец доменного типа -- Читать/записывать значения из/в доменный столбец -- Использовать его как индекс, если базовый тип может использоваться как индекс -- Вызывать функции со значениями доменного столбца - -### Дополнительные возможности доменов {#extra-features-of-domains} - -- Явное имя типа столбца в `SHOW CREATE TABLE` или `DESCRIBE TABLE` -- Ввод в удобном для человека формате с помощью `INSERT INTO domain_table(domain_column) VALUES(...)` -- Вывод в удобном для человека формате для `SELECT domain_column FROM domain_table` -- Загрузка данных из внешнего источника в удобном для человека формате: `INSERT INTO domain_table FORMAT CSV ...` - -### Ограничения {#limitations} - -- Нельзя преобразовать индексный столбец базового типа в доменный тип через `ALTER TABLE`. -- Нельзя неявно преобразовать строковые значения в доменные при вставке данных из другого столбца или таблицы. -- Домен не накладывает ограничений на хранимые значения. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md deleted file mode 100644 index 75140653622..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md +++ /dev/null @@ -1,717 +0,0 @@ ---- -description: 'Документация о типе данных Dynamic в ClickHouse, который позволяет хранить значения различных типов в одном столбце' -sidebar_label: 'Dynamic' -sidebar_position: 62 -slug: /sql-reference/data-types/dynamic -title: 'Dynamic' -doc_type: 'guide' ---- - -# Dynamic {#dynamic} - -Этот тип позволяет хранить значения произвольных типов, не зная заранее всех возможных вариантов. - -Чтобы объявить столбец типа `Dynamic`, используйте следующий синтаксис: - -```sql -<имя_столбца> Dynamic(max_types=N) -``` - -Где `N` — необязательный параметр в диапазоне от `0` до `254`, задающий, сколько различных типов данных может быть сохранено в виде отдельных подстолбцов внутри столбца типа `Dynamic` в пределах одного отдельного блока данных (например, одной части данных таблицы MergeTree). Если этот предел превышен, все значения с новыми типами будут сохранены вместе в специальной общей структуре данных в двоичной форме. Значение `max_types` по умолчанию — `32`. - -## Создание типа Dynamic {#creating-dynamic} - -Использование типа `Dynamic` в определении столбца таблицы: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ Hello, World! │ String │ -│ [1,2,3] │ Array(Int64) │ -└───────────────┴────────────────┘ -``` - -Применение CAST к обычному столбцу: - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -Приведение типов с помощью CAST для столбца `Variant`: - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 3) = 0, number, (number % 3) = 1, range(number + 1), NULL)::Dynamic AS d, dynamicType(d) FROM numbers(3) -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 0 │ UInt64 │ -│ [0,1] │ Array(UInt64) │ -│ ᴺᵁᴸᴸ │ None │ -└───────┴────────────────┘ -``` - -## Чтение вложенных типов Dynamic как подстолбцов {#reading-dynamic-nested-types-as-subcolumns} - -Тип `Dynamic` поддерживает чтение отдельного вложенного типа из столбца `Dynamic`, используя имя типа как подстолбец. -Таким образом, если у вас есть столбец `d Dynamic`, вы можете считать подстолбец любого допустимого типа `T` с помощью синтаксиса `d.T`; -этот подстолбец будет иметь тип `Nullable(T)`, если `T` может находиться внутри `Nullable`, и тип `T` в противном случае. Этот подстолбец будет -той же длины, что и исходный столбец `Dynamic`, и будет содержать значения `NULL` (или пустые значения, если `T` не может находиться внутри `Nullable`) -во всех строках, в которых исходный столбец `Dynamic` не содержит значения типа `T`. - -Подстолбцы `Dynamic` также можно считывать с помощью функции `dynamicElement(dynamic_column, type_name)`. - -Примеры: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d), d.String, d.Int64, d.`Array(Int64)`, d.Date, d.`Array(String)` FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─d.String──────┬─d.Int64─┬─d.Array(Int64)─┬─d.Date─┬─d.Array(String)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴───────────────┴─────────┴────────────────┴────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(d.String), toTypeName(d.Int64), toTypeName(d.`Array(Int64)`), toTypeName(d.Date), toTypeName(d.`Array(String)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(d.String)─┬─toTypeName(d.Int64)─┬─toTypeName(d.Array(Int64))─┬─toTypeName(d.Date)─┬─toTypeName(d.Array(String))─┐ -│ Nullable(String) │ Nullable(Int64) │ Array(Int64) │ Nullable(Date) │ Array(String) │ -└──────────────────────┴─────────────────────┴────────────────────────────┴────────────────────┴─────────────────────────────┘ -``` - -````sql -SELECT d, dynamicType(d), dynamicElement(d, 'String'), dynamicElement(d, 'Int64'), dynamicElement(d, 'Array(Int64)'), dynamicElement(d, 'Date'), dynamicElement(d, 'Array(String)') FROM test;``` -```` - -```text -┌─d─────────────┬─dynamicType(d)─┬─dynamicElement(d, 'String')─┬─dynamicElement(d, 'Int64')─┬─dynamicElement(d, 'Array(Int64)')─┬─dynamicElement(d, 'Date')─┬─dynamicElement(d, 'Array(String)')─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴─────────────────────────────┴────────────────────────────┴───────────────────────────────────┴───────────────────────────┴────────────────────────────────────┘ -``` - -Чтобы узнать, какой вариант хранится в каждой строке, можно использовать функцию `dynamicType(dynamic_column)`. Она возвращает строку (`String`) с именем типа значения для каждой строки (или `'None'`, если в строке `NULL`). - -Пример: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT dynamicType(d) FROM test; -``` - -```text -┌─dynamicType(d)─┐ -│ None │ -│ Int64 │ -│ String │ -│ Array(Int64) │ -└────────────────┘ -``` - -## Преобразование между столбцом типа Dynamic и другими столбцами {#conversion-between-dynamic-column-and-other-columns} - -Существует 4 возможных преобразования, которые можно выполнить со столбцом типа `Dynamic`. - -### Преобразование обычного столбца в столбец типа Dynamic {#converting-an-ordinary-column-to-a-dynamic-column} - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -### Преобразование столбца String в столбец Dynamic путем разбора {#converting-a-string-column-to-a-dynamic-column-through-parsing} - -Чтобы разбирать значения типа `Dynamic` из столбца `String`, вы можете включить настройку `cast_string_to_dynamic_use_inference`: - -```sql -SET cast_string_to_dynamic_use_inference = 1; -SELECT CAST(materialize(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01')), 'Map(String, Dynamic)') as map_of_dynamic, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamic) as map_of_dynamic_types; -``` - -```text -┌─map_of_dynamic──────────────────────────────┬─map_of_dynamic_types─────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'Int64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴──────────────────────────────────────────────┘ -``` - -### Преобразование столбца типа Dynamic в обычный столбец {#converting-a-dynamic-column-to-an-ordinary-column} - -Можно преобразовать столбец типа `Dynamic` в обычный столбец. В этом случае все вложенные типы будут преобразованы в целевой тип: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'), (true), ('e10'); -SELECT d::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(d, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -│ 1 │ -│ 0 │ -└──────────────────────────────┘ -``` - -### Преобразование столбца типа Variant в столбец типа Dynamic {#converting-a-variant-column-to-dynamic-column} - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'), ([1, 2, 3]); -SELECT v::Dynamic AS d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ UInt64 │ -│ String │ String │ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴────────────────┘ -``` - -### Преобразование столбца Dynamic(max_types=N) в столбец Dynamic(max_types=K) {#converting-a-dynamicmax_typesn-column-to-another-dynamicmax_typesk} - -Если `K >= N`, то при преобразовании данные не изменяются: - -```sql -CREATE TABLE test (d Dynamic(max_types=3)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true); -SELECT d::Dynamic(max_types=5) as d2, dynamicType(d2) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ 42.42 │ String │ -│ true │ Bool │ -└───────┴────────────────┘ -``` - -Если `K < N`, то значения с наиболее редкими типами будут вставлены в один специальный подстолбец, но при этом по-прежнему будут доступны: - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=2) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ false │ -│ 43 │ Int64 │ 43 │ Int64 │ false │ -│ 42.42 │ String │ 42.42 │ String │ false │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -Функция `isDynamicElementInSharedData` возвращает `true` для строк, которые хранятся в специальной разделяемой структуре данных внутри `Dynamic`, и, как видно, результирующий столбец содержит только два типа, которые не хранятся в разделяемой структуре данных. - -Если `K=0`, все типы будут вставлены в один специальный подстолбец: - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=0) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ true │ -│ 43 │ Int64 │ 43 │ Int64 │ true │ -│ 42.42 │ String │ 42.42 │ String │ true │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -## Чтение типа Dynamic из данных {#reading-dynamic-type-from-the-data} - -Все текстовые форматы (TSV, CSV, CustomSeparated, Values, JSONEachRow и т. д.) поддерживают чтение типа `Dynamic`. При разборе данных ClickHouse пытается определить тип каждого значения и использует его при вставке в столбец `Dynamic`. - -Пример: - -```sql -SELECT - d, - dynamicType(d), - dynamicElement(d, 'String') AS str, - dynamicElement(d, 'Int64') AS num, - dynamicElement(d, 'Float64') AS float, - dynamicElement(d, 'Date') AS date, - dynamicElement(d, 'Array(Int64)') AS arr -FROM format(JSONEachRow, 'd Dynamic', $$ -{"d" : "Hello, World!"}, -{"d" : 42}, -{"d" : 42.42}, -{"d" : "2020-01-01"}, -{"d" : [1, 2, 3]} -$$) -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─str───────────┬──num─┬─float─┬───────date─┬─arr─────┐ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ Float64 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 │ Date │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴────────────────┴───────────────┴──────┴───────┴────────────┴─────────┘ -``` - -## Использование типа Dynamic в функциях {#using-dynamic-type-in-functions} - -Большинство функций поддерживают аргументы типа `Dynamic`. В этом случае функция выполняется отдельно для каждого внутреннего типа данных, представленного в столбце `Dynamic`. -Когда тип результата функции зависит от типов аргументов, результат такой функции, выполняемой с аргументами типа `Dynamic`, будет иметь тип `Dynamic`. Когда тип результата функции не зависит от типов аргументов, результатом будет `Nullable(T)`, где `T` — обычный тип результата этой функции. - -Примеры: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (NULL), (1::Int8), (2::Int16), (3::Int32), (4::Int64); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 1 │ Int8 │ -│ 2 │ Int16 │ -│ 3 │ Int32 │ -│ 4 │ Int64 │ -└──────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 3 │ Dynamic │ Int32 │ -│ 3 │ 4 │ Dynamic │ Int64 │ -│ 4 │ 5 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d + d AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 4 │ Dynamic │ Int32 │ -│ 3 │ 6 │ Dynamic │ Int64 │ -│ 4 │ 8 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d < 3 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(UInt8) │ -│ 1 │ 1 │ Nullable(UInt8) │ -│ 2 │ 1 │ Nullable(UInt8) │ -│ 3 │ 0 │ Nullable(UInt8) │ -│ 4 │ 0 │ Nullable(UInt8) │ -└──────┴──────┴─────────────────┘ -``` - -```sql -SELECT d, exp2(d) AS res, toTypeName(res) FROM test; -``` - -```sql -┌─d────┬──res─┬─toTypeName(res)───┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Float64) │ -│ 1 │ 2 │ Nullable(Float64) │ -│ 2 │ 4 │ Nullable(Float64) │ -│ 3 │ 8 │ Nullable(Float64) │ -│ 4 │ 16 │ Nullable(Float64) │ -└──────┴──────┴───────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ('str_1'), ('str_2'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ str_1 │ String │ -│ str_2 │ String │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, upper(d) AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res───┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ STR_1 │ Nullable(String) │ -│ str_2 │ STR_2 │ Nullable(String) │ -└───────┴───────┴──────────────────┘ -``` - -```sql -SELECT d, extract(d, '([0-3])') AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ 1 │ Nullable(String) │ -│ str_2 │ 2 │ Nullable(String) │ -└───────┴──────┴──────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ([1, 2]), ([3, 4]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d[1] AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ [1,2] │ 1 │ Dynamic │ Int64 │ -│ [3,4] │ 3 │ Dynamic │ Int64 │ -└───────┴──────┴─────────────────┴──────────────────┘ -``` - -Если функцию нельзя выполнить для какого-либо типа внутри столбца `Dynamic`, будет вызвано исключение: - -```sql -INSERT INTO test VALUES (42), (43), ('str_1'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ str_1 │ String │ -└───────┴────────────────┘ -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(d) FROM test; -``` - -```text -Получено исключение: -Code: 43. DB::Exception: Недопустимые типы Array(Int64) и UInt8 аргументов функции plus: во время выполнения 'FUNCTION plus(__table1.d : 3, 1_UInt8 :: 1) -> plus(__table1.d, 1_UInt8) Dynamic : 0'. (ILLEGAL_TYPE_OF_ARGUMENT) -``` - -Мы можем отфильтровать ненужные типы: - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test WHERE dynamicType(d) NOT IN ('String', 'Array(Int64)', 'None') -``` - -```text -┌─d──┬─res─┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ 42 │ 43 │ Dynamic │ Int64 │ -│ 43 │ 44 │ Dynamic │ Int64 │ -└────┴─────┴─────────────────┴──────────────────┘ -``` - -Или извлеките нужный тип в виде подстолбца: - -```sql -SELECT d, d.Int64 + 1 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ 42 │ 43 │ Nullable(Int64) │ -│ 43 │ 44 │ Nullable(Int64) │ -│ str_1 │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [1,2] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [3,4] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -``` - -## Использование типа Dynamic в ORDER BY и GROUP BY {#using-dynamic-type-in-order-by-and-group-by} - -При выполнении `ORDER BY` и `GROUP BY` значения типа `Dynamic` сравниваются аналогично значениям типа `Variant`: -результат оператора `<` для значений `d1` с базовым типом `T1` и `d2` с базовым типом `T2` типа `Dynamic` определяется следующим образом: - -* Если `T1 = T2 = T`, результатом будет `d1.T < d2.T` (сравниваются базовые значения). -* Если `T1 != T2`, результатом будет `T1 < T2` (сравниваются имена типов). - -По умолчанию тип `Dynamic` не может использоваться в ключах `GROUP BY`/`ORDER BY`. Если вы хотите его использовать, учитывайте его специальное правило сравнения и включите настройки `allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by`. - -Примеры: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (42), (43), ('abc'), ('abd'), ([1, 2, 3]), ([]), (NULL); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ [1,2,3] │ Array(Int64) │ -│ [] │ Array(Int64) │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```sql -┌─d───────┬─dynamicType(d)─┐ -│ [] │ Array(Int64) │ -│ [1,2,3] │ Array(Int64) │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -**Примечание:** значения динамических типов, основанных на разных числовых типах, считаются разными и не сравниваются между собой, вместо этого сравниваются их имена типов. - -Пример: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```text -┌─v───┬─dynamicType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test GROUP BY d SETTINGS allow_suspicious_types_in_group_by=1; -``` - -```text -┌─d───┬─dynamicType(d)─┐ -│ 1 │ Int64 │ -│ 100 │ UInt32 │ -│ 1 │ UInt32 │ -│ 100 │ Int64 │ -└─────┴────────────────┘ -``` - -**Примечание:** описанное правило сравнения не применяется при вычислении функций сравнения, таких как `<`/`>`/`=` и других, из-за [особенностей работы](#using-dynamic-type-in-functions) функций с типом `Dynamic`. - -## Достижение предела количества различных типов данных, хранящихся внутри Dynamic {#reaching-the-limit-in-number-of-different-data-types-stored-inside-dynamic} - -Тип данных `Dynamic` может хранить только ограниченное количество различных типов данных в виде отдельных подстолбцов. По умолчанию этот предел равен 32, но вы можете изменить его в объявлении типа, используя синтаксис `Dynamic(max_types=N)`, где N находится в диапазоне от 0 до 254 (из-за особенностей реализации невозможно использовать более 254 различных типов данных, которые могут быть сохранены как отдельные подстолбцы внутри Dynamic). -Когда предел достигнут, все новые значения с ранее не встречавшимися типами данных, вставляемые в столбец `Dynamic`, будут записываться в одну общую структуру данных, которая хранит значения с разными типами данных в двоичном виде. - -Рассмотрим, что происходит при достижении предела в разных сценариях. - -### Достижение предела во время разбора данных {#reaching-the-limit-during-data-parsing} - -Во время разбора значений `Dynamic` из данных, когда предел достигнут для текущего блока данных, все новые значения будут записаны в общую структуру данных: - -```sql -SELECT d, dynamicType(d), isDynamicElementInSharedData(d) FROM format(JSONEachRow, 'd Dynamic(max_types=3)', ' -{"d" : 42} -{"d" : [1, 2, 3]} -{"d" : "Hello, World!"} -{"d" : "2020-01-01"} -{"d" : ["str1", "str2", "str3"]} -{"d" : {"a" : 1, "b" : [1, 2, 3]}} -') -``` - -```text -┌─d──────────────────────┬─dynamicType(d)─────────────────┬─isDynamicElementInSharedData(d)─┐ -│ 42 │ Int64 │ false │ -│ [1,2,3] │ Array(Int64) │ false │ -│ Hello, World! │ String │ false │ -│ 2020-01-01 │ Date │ true │ -│ ['str1','str2','str3'] │ Array(String) │ true │ -│ (1,[1,2,3]) │ Tuple(a Int64, b Array(Int64)) │ true │ -└────────────────────────┴────────────────────────────────┴─────────────────────────────────┘ -``` - -Как мы видим, после вставки 3 разных типов данных `Int64`, `Array(Int64)` и `String` все новые типы были помещены в специальную общую структуру данных. - -### Во время слияний частей данных в движках таблиц MergeTree {#during-merges-of-data-parts-in-mergetree-table-engines} - -Во время слияния нескольких частей данных в таблице MergeTree столбец `Dynamic` в результирующей части может достигнуть предела по количеству различных типов данных, которые могут храниться во внутренних подстолбцах, и не сможет хранить все типы как подстолбцы из исходных частей. -В этом случае ClickHouse выбирает, какие типы останутся отдельными подстолбцами после слияния, а какие типы будут перенесены в общую структуру данных. В большинстве случаев ClickHouse старается сохранять наиболее часто встречающиеся типы и хранить самые редкие типы в общей структуре данных, но это зависит от реализации. - -Рассмотрим пример такого слияния. Сначала создадим таблицу со столбцом `Dynamic`, установим предел количества различных типов данных равным `3` и вставим значения пяти различных типов: - -```sql -CREATE TABLE test (id UInt64, d Dynamic(max_types=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, number FROM numbers(5); -INSERT INTO test SELECT number, range(number) FROM numbers(4); -INSERT INTO test SELECT number, toDate(number) FROM numbers(3); -INSERT INTO test SELECT number, map(number, number) FROM numbers(2); -INSERT INTO test SELECT number, 'str_' || toString(number) FROM numbers(1); -``` - -Каждая вставка будет создавать отдельную часть данных со столбцом `Dynamic`, содержащим данные только одного типа: - -```sql -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count(); -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_1_0 │ -│ 4 │ Array(UInt64) │ false │ all_2_2_0 │ -│ 3 │ Date │ false │ all_3_3_0 │ -│ 2 │ Map(UInt64, UInt64) │ false │ all_4_4_0 │ -│ 1 │ String │ false │ all_5_5_0 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -Теперь объединим все части и посмотрим, что получится: - -```sql -SYSTEM START MERGES test; -OPTIMIZE TABLE test FINAL; -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count() desc; -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_5_2 │ -│ 4 │ Array(UInt64) │ false │ all_1_5_2 │ -│ 3 │ Date │ false │ all_1_5_2 │ -│ 2 │ Map(UInt64, UInt64) │ true │ all_1_5_2 │ -│ 1 │ String │ true │ all_1_5_2 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -Как мы видим, ClickHouse сохранил наиболее часто встречающиеся типы `UInt64` и `Array(UInt64)` как подстолбцы, а все остальные типы поместил в общие данные. - -## Функции JSONExtract с Dynamic {#jsonextract-functions-with-dynamic} - -Все функции `JSONExtract*` поддерживают тип `Dynamic`: - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Dynamic') AS dynamic, dynamicType(dynamic) AS dynamic_type; -``` - -```text -┌─dynamic─┬─dynamic_type───────────┐ -│ [1,2,3] │ Array(Nullable(Int64)) │ -└─────────┴────────────────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Dynamic)') AS map_of_dynamics, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamics) AS map_of_dynamic_types -``` - -```text -┌─map_of_dynamics──────────────────┬─map_of_dynamic_types────────────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'Int64','b':'String','c':'Array(Nullable(Int64))'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────────────┘ -``` - -````sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Dynamic') AS dynamics, arrayMap(x -> (x.1, dynamicType(x.2)), dynamics) AS dynamic_types``` -```` - -```text -┌─dynamics───────────────────────────────┬─dynamic_types─────────────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','Int64'),('b','String'),('c','Array(Nullable(Int64))')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘ -``` - -### Двоичный формат вывода {#binary-output-format} - -В формате RowBinary значения типа `Dynamic` сериализуются в следующем формате: - -```text - -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md deleted file mode 100644 index 207b8314a52..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -description: 'Документация о типе данных Enum в ClickHouse, задающем набор именованных константных значений' -sidebar_label: 'Enum' -sidebar_position: 20 -slug: /sql-reference/data-types/enum -title: 'Enum' -doc_type: 'reference' ---- - -# Enum {#enum} - -Перечисляемый тип, состоящий из именованных значений. - -Именованные значения могут быть объявлены как пары `'string' = integer` или просто как имена `'string'`. ClickHouse хранит только числа, но поддерживает операции со значениями по их именам. - -ClickHouse поддерживает: - -* 8-битный `Enum`. Может содержать до 256 значений, перечисляемых в диапазоне `[-128, 127]`. -* 16-битный `Enum`. Может содержать до 65536 значений, перечисляемых в диапазоне `[-32768, 32767]`. - -ClickHouse автоматически выбирает тип `Enum` при вставке данных. Вы также можете использовать типы `Enum8` или `Enum16`, чтобы точно задать объем занимаемого места. - -## Примеры использования {#usage-examples} - -Здесь мы создаём таблицу со столбцом типа `Enum8('hello' = 1, 'world' = 2)`: - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world' = 2) -) -ENGINE = TinyLog -``` - -Аналогично вы можете не указывать числа. ClickHouse автоматически присвоит последовательные номера. По умолчанию нумерация начинается с 1. - -```sql -CREATE TABLE t_enum -( - x Enum('hello', 'world') -) -ENGINE = TinyLog -``` - -Вы также можете указать допустимый начальный номер для первого имени. - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world') -) -ENGINE = TinyLog -``` - -```sql -CREATE TABLE t_enum -( - x Enum8('hello' = -129, 'world') -) -ENGINE = TinyLog -``` - -```text -Исключение на сервере: -Код: 69. DB::Exception: Значение -129 для элемента 'hello' выходит за пределы диапазона Enum8. -``` - -Столбец `x` может хранить только значения, перечисленные в определении типа: `'hello'` или `'world'`. Если вы попытаетесь сохранить любое другое значение, ClickHouse выдаст исключение. Размер 8 бит для этого типа `Enum` выбирается автоматически. - -```sql -INSERT INTO t_enum VALUES ('hello'), ('world'), ('hello') -``` - -```text -Ok. -``` - -```sql -INSERT INTO t_enum VALUES('a') -``` - -```text -Исключение на клиенте: -Code: 49. DB::Exception: Unknown element 'a' for type Enum('hello' = 1, 'world' = 2) -``` - -При выборке данных из таблицы ClickHouse возвращает строковые значения из `Enum`. - -```sql -SELECT * FROM t_enum -``` - -```text -┌─x─────┐ -│ hello │ -│ world │ -│ hello │ -└───────┘ -``` - -Если вам нужно увидеть числовые значения для строк, необходимо привести значение `Enum` к целочисленному типу. - -```sql -SELECT CAST(x, 'Int8') FROM t_enum -``` - -```text -┌─CAST(x, 'Int8')─┐ -│ 1 │ -│ 2 │ -│ 1 │ -└─────────────────┘ -``` - -Чтобы создать значение Enum в запросе, также нужно использовать `CAST`. - -```sql -SELECT toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)')) -``` - -```text -┌─toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)'))─┐ -│ Enum8('a' = 1, 'b' = 2) │ -└─────────────────────────────────────────────────────┘ -``` - -## Общие правила и применение {#general-rules-and-usage} - -Каждому значению сопоставляется число в диапазоне `-128 ... 127` для `Enum8` или в диапазоне `-32768 ... 32767` для `Enum16`. Все строки и числа должны быть различными. Пустая строка допускается. Если этот тип задан в определении таблицы, числа могут идти в произвольном порядке. Однако порядок не имеет значения. - -Ни строковое, ни числовое значение в `Enum` не может быть [NULL](../../sql-reference/syntax.md). - -`Enum` может входить в состав типа [Nullable](../../sql-reference/data-types/nullable.md). Поэтому, если вы создаёте таблицу с помощью запроса - -```sql -CREATE TABLE t_enum_nullable -( - x Nullable( Enum8('hello' = 1, 'world' = 2) ) -) -ENGINE = TinyLog -``` - -он может хранить не только `'hello'` и `'world'`, но и значение `NULL`. - -```sql -INSERT INTO t_enum_nullable VALUES('hello'),('world'),(NULL) -``` - -В оперативной памяти столбец `Enum` хранится так же, как `Int8` или `Int16` с соответствующими числовыми значениями. - -При чтении в текстовом виде ClickHouse разбирает значение как строку и ищет соответствующую строку из набора значений Enum. Если строка не найдена, генерируется исключение. При чтении в текстовом формате строка считывается, и по ней ищется соответствующее числовое значение. Если оно не найдено, будет сгенерировано исключение. -При записи в текстовом виде значение выводится как соответствующая строка. Если данные в столбце содержат мусор (числа, не входящие в допустимый набор), генерируется исключение. При чтении и записи в бинарном виде поведение такое же, как для типов данных Int8 и Int16. -Неявным значением по умолчанию является значение с наименьшим числом. - -При выполнении `ORDER BY`, `GROUP BY`, `IN`, `DISTINCT` и т. п. значения Enum ведут себя так же, как соответствующие числа. Например, ORDER BY сортирует их по числовому значению. Операторы равенства и сравнения работают с Enum так же, как с его базовыми числовыми значениями. - -Значения Enum нельзя сравнивать с числами. Enum можно сравнивать с константной строкой. Если строка для сравнения не является допустимым значением для Enum, будет сгенерировано исключение. Оператор IN поддерживается с Enum слева и набором строк справа. Эти строки являются значениями соответствующего Enum. - -Большинство числовых и строковых операций не определены для значений Enum, например, сложение числа с Enum или конкатенация строки с Enum. -Однако для Enum естественным образом определена функция `toString`, которая возвращает его строковое значение. - -Значения Enum также могут быть преобразованы к числовым типам с помощью функции `toT`, где T — числовой тип. Когда T соответствует базовому числовому типу Enum, это преобразование выполняется без накладных расходов. -Тип Enum может быть изменён без накладных расходов с помощью ALTER, если изменяется только набор значений. С помощью ALTER можно как добавлять, так и удалять элементы Enum (удаление безопасно только в том случае, если удаляемое значение никогда не использовалось в таблице). В качестве защитного механизма изменение числового значения ранее определённого элемента Enum приведёт к генерации исключения. - -С помощью ALTER можно изменить Enum8 на Enum16 или наоборот, так же как изменить Int8 на Int16. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md deleted file mode 100644 index 24cf40d5fdc..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'Документация по типу данных FixedString(N) в ClickHouse' -sidebar_label: 'FixedString(N)' -sidebar_position: 10 -slug: /sql-reference/data-types/fixedstring -title: 'FixedString(N)' -doc_type: 'reference' ---- - -# FixedString(N) {#fixedstringn} - -Строка фиксированной длины из `N` байт (не в символах и не в кодовых точках). - -Чтобы объявить столбец типа `FixedString`, используйте следующий синтаксис: - -```sql - FixedString(N) -``` - -Где `N` — натуральное число. - -Тип `FixedString` эффективен, когда данные имеют длину ровно `N` байт. Во всех остальных случаях он, скорее всего, снизит эффективность. - -Примеры значений, которые могут эффективно храниться в столбцах типа `FixedString`: - -* Бинарное представление IP-адресов (`FixedString(16)` для IPv6). -* Коды языков (ru_RU, en_US ... ). -* Коды валют (USD, RUB ... ). -* Бинарное представление хешей (`FixedString(16)` для MD5, `FixedString(32)` для SHA256). - -Для хранения значений UUID используйте тип данных [UUID](../../sql-reference/data-types/uuid.md). - -При вставке данных ClickHouse: - -* Дополняет строку нулевыми байтами, если строка содержит меньше, чем `N` байт. -* Выбрасывает исключение `Too large value for FixedString(N)`, если строка содержит больше, чем `N` байт. - -Рассмотрим следующую таблицу с единственным столбцом типа `FixedString(2)`: - -```sql - - -INSERT INTO FixedStringTable VALUES ('a'), ('ab'), (''); -``` - -```sql -SELECT - name, - toTypeName(name), - length(name), - empty(name) -FROM FixedStringTable; -``` - -```text -┌─name─┬─toTypeName(name)─┬─length(name)─┬─empty(name)─┐ -│ a │ FixedString(2) │ 2 │ 0 │ -│ ab │ FixedString(2) │ 2 │ 0 │ -│ │ FixedString(2) │ 2 │ 1 │ -└──────┴──────────────────┴──────────────┴─────────────┘ -``` - -Обратите внимание, что длина значения `FixedString(N)` является фиксированной. Функция [length](/sql-reference/functions/array-functions#length) возвращает `N`, даже если значение `FixedString(N)` заполнено только нулевыми байтами, однако функция [empty](/sql-reference/functions/array-functions#empty) в этом случае возвращает `1`. - -Выборка данных с предложением `WHERE` возвращает разные результаты в зависимости от того, как сформулировано условие: - -* Если используется оператор равенства `=` или `==` или функция `equals`, ClickHouse *не* учитывает символ `\0`, т.е. запросы `SELECT * FROM FixedStringTable WHERE name = 'a';` и `SELECT * FROM FixedStringTable WHERE name = 'a\0';` возвращают один и тот же результат. -* Если используется предложение `LIKE`, ClickHouse *учитывает* символ `\0`, поэтому может потребоваться явно указать символ `\0` в условии фильтрации. - -```sql -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -Query id: c32cec28-bb9e-4650-86ce-d74a1694d79e - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a' -FORMAT JSONStringsEachRow - -0 строк в результате. - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md deleted file mode 100644 index 7bb13b4b824..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: 'Документация по типам данных с плавающей запятой в ClickHouse: Float32, - Float64 и BFloat16' -sidebar_label: 'Float32 | Float64 | BFloat16' -sidebar_position: 4 -slug: /sql-reference/data-types/float -title: 'Типы Float32 | Float64 | BFloat16' -doc_type: 'reference' ---- - -:::note -Если вам нужны точные вычисления, в частности, если вы работаете с финансовыми или бизнес-данными, требующими высокой точности, следует рассмотреть возможность использования [Decimal](../data-types/decimal.md). - -[Числа с плавающей запятой](https://en.wikipedia.org/wiki/IEEE_754) могут приводить к неточным результатам, как показано ниже: - -```sql -CREATE TABLE IF NOT EXISTS float_vs_decimal -( - my_float Float64, - my_decimal Decimal64(3) -) -ENGINE=MergeTree -ORDER BY tuple(); -``` - - -# Сгенерировать 1 000 000 случайных чисел с 2 знаками после запятой и сохранить их в форматах float и decimal {#generate-1-000-000-random-numbers-with-2-decimal-places-and-store-them-as-a-float-and-as-a-decimal} - -INSERT INTO float_vs_decimal SELECT round(randCanonical(), 3) AS res, res FROM system.numbers LIMIT 1000000; - -```` -```sql -SELECT sum(my_float), sum(my_decimal) FROM float_vs_decimal; - -┌──────sum(my_float)─┬─sum(my_decimal)─┐ -│ 499693.60500000004 │ 499693.605 │ -└────────────────────┴─────────────────┘ - -SELECT sumKahan(my_float), sumKahan(my_decimal) FROM float_vs_decimal; - -┌─sumKahan(my_float)─┬─sumKahan(my_decimal)─┐ -│ 499693.605 │ 499693.605 │ -└────────────────────┴──────────────────────┘ -```` - -::: - -Эквивалентные типы в ClickHouse и в C приведены ниже: - -* `Float32` — `float`. -* `Float64` — `double`. - -Типы с плавающей точкой в ClickHouse имеют следующие синонимы: - -* `Float32` — `FLOAT`, `REAL`, `SINGLE`. -* `Float64` — `DOUBLE`, `DOUBLE PRECISION`. - -При создании таблиц можно указывать числовые параметры для чисел с плавающей точкой (например, `FLOAT(12)`, `FLOAT(15, 22)`, `DOUBLE(12)`, `DOUBLE(4, 18)`), но ClickHouse их игнорирует. - - -## Использование чисел с плавающей запятой {#using-floating-point-numbers} - -* Вычисления с числами с плавающей запятой могут приводить к ошибке округления. - -{/* */ } - -```sql -SELECT 1 - 0.9 - -┌───────minus(1, 0.9)─┐ -│ 0.09999999999999998 │ -└─────────────────────┘ -``` - -* Результат вычислений зависит от способа их выполнения (типа процессора и архитектуры компьютерной системы). -* Вычисления с плавающей запятой могут приводить к значениям, таким как бесконечность (`Inf`) и «не число» (`NaN`). Это следует учитывать при обработке результатов вычислений. -* При разборе (парсинге) чисел с плавающей запятой из текста результат может отличаться от ближайшего машинно представимого числа. - - -## NaN и Inf {#nan-and-inf} - -В отличие от стандартного SQL, ClickHouse поддерживает следующие категории чисел с плавающей запятой: - -* `Inf` – бесконечность. - -{/* */ } - -```sql -SELECT 0.5 / 0 - -┌─divide(0.5, 0)─┐ -│ inf │ -└────────────────┘ -``` - -* `-Inf` — отрицательная бесконечность. - -{/* */ } - -```sql -SELECT -0.5 / 0 - -┌─divide(-0.5, 0)─┐ -│ -inf │ -└─────────────────┘ -``` - -* `NaN` — не число (Not a Number). - -{/* */ } - -```sql -SELECT 0 / 0 - -┌─divide(0, 0)─┐ -│ nan │ -└──────────────┘ -``` - -См. правила сортировки значения `NaN` в разделе [Предложение ORDER BY](../../sql-reference/statements/select/order-by.md). - - -## BFloat16 {#bfloat16} - -`BFloat16` — это 16-битный тип данных с плавающей запятой с 8-битной экспонентой, знаком и 7-битной мантиссой. -Он полезен для задач машинного обучения и приложений ИИ. - -ClickHouse поддерживает преобразования между `Float32` и `BFloat16`, которые -можно выполнять с помощью функций [`toFloat32()`](../functions/type-conversion-functions.md/#tofloat32) или [`toBFloat16`](../functions/type-conversion-functions.md/#tobfloat16). - -:::note -Большинство других операций не поддерживаются. -::: diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md deleted file mode 100644 index 29866a25d8a..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md +++ /dev/null @@ -1,206 +0,0 @@ ---- -description: 'Документация по геометрическим типам данных в ClickHouse, используемым для представления географических объектов и местоположений' -sidebar_label: 'Гео' -sidebar_position: 54 -slug: /sql-reference/data-types/geo -title: 'Геометрические' -doc_type: 'reference' ---- - -ClickHouse поддерживает геометрические типы данных для представления географических объектов — местоположений, территорий и т. д. - -**См. также** -- [Представление простых географических объектов](https://en.wikipedia.org/wiki/GeoJSON). - - - -## Point {#point} - -`Point` задаётся своими координатами X и Y, которые хранятся как [Tuple](tuple.md)([Float64](float.md), [Float64](float.md)). - -**Пример** - -Запрос: - -```sql -CREATE TABLE geo_point (p Point) ENGINE = Memory(); -INSERT INTO geo_point VALUES((10, 10)); -SELECT p, toTypeName(p) FROM geo_point; -``` - -Результат: - -```text -┌─p───────┬─toTypeName(p)─┐ -│ (10,10) │ Point │ -└─────────┴───────────────┘ -``` - - -## Кольцо {#ring} - -`Ring` — это простой многоугольник без отверстий, хранящийся в виде массива точек: [Array](array.md)([Point](#point)). - -**Пример** - -Запрос: - -```sql -CREATE TABLE geo_ring (r Ring) ENGINE = Memory(); -INSERT INTO geo_ring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT r, toTypeName(r) FROM geo_ring; -``` - -Результат: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ Ring │ -└───────────────────────────────┴───────────────┘ -``` - - -## LineString {#linestring} - -`LineString` — это линия, представленная в виде массива точек: [Array](array.md)([Point](#point)). - -**Пример** - -Запрос: - -```sql -CREATE TABLE geo_linestring (l LineString) ENGINE = Memory(); -INSERT INTO geo_linestring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT l, toTypeName(l) FROM geo_linestring; -``` - -Результат: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ LineString │ -└───────────────────────────────┴───────────────┘ -``` - - -## MultiLineString {#multilinestring} - -`MultiLineString` — это несколько линий, хранящихся в виде массива `LineString`: [Array](array.md)([LineString](#linestring)). - -**Пример** - -Запрос: - -```sql -CREATE TABLE geo_multilinestring (l MultiLineString) ENGINE = Memory(); -INSERT INTO geo_multilinestring VALUES([[(0, 0), (10, 0), (10, 10), (0, 10)], [(1, 1), (2, 2), (3, 3)]]); -SELECT l, toTypeName(l) FROM geo_multilinestring; -``` - -Результат: - -```text -┌─l───────────────────────────────────────────────────┬─toTypeName(l)───┐ -│ [[(0,0),(10,0),(10,10),(0,10)],[(1,1),(2,2),(3,3)]] │ MultiLineString │ -└─────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Многоугольник {#polygon} - -`Polygon` — многоугольник с отверстиями, представленный в виде массива колец: [Array](array.md)([Ring](#ring)). Первый элемент внешнего массива задаёт внешний контур многоугольника, а все последующие элементы — его отверстия. - -**Пример** - -Это многоугольник с одним отверстием: - -```sql -CREATE TABLE geo_polygon (pg Polygon) ENGINE = Memory(); -INSERT INTO geo_polygon VALUES([[(20, 20), (50, 20), (50, 50), (20, 50)], [(30, 30), (50, 50), (50, 30)]]); -SELECT pg, toTypeName(pg) FROM geo_polygon; -``` - -Результат: - -```text -┌─pg────────────────────────────────────────────────────────────┬─toTypeName(pg)─┐ -│ [[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]] │ Polygon │ -└───────────────────────────────────────────────────────────────┴────────────────┘ -``` - - -## MultiPolygon {#multipolygon} - -`MultiPolygon` состоит из нескольких полигонов и хранится как массив полигонов: [Array](array.md)([Polygon](#polygon)). - -**Пример** - -Этот мультиполигон состоит из двух отдельных полигонов — первый без отверстий, второй — с одним отверстием: - -```sql -CREATE TABLE geo_multipolygon (mpg MultiPolygon) ENGINE = Memory(); -INSERT INTO geo_multipolygon VALUES([[[(0, 0), (10, 0), (10, 10), (0, 10)]], [[(20, 20), (50, 20), (50, 50), (20, 50)],[(30, 30), (50, 50), (50, 30)]]]); -SELECT mpg, toTypeName(mpg) FROM geo_multipolygon; -``` - -Результат: - -```text -┌─mpg─────────────────────────────────────────────────────────────────────────────────────────────┬─toTypeName(mpg)─┐ -│ [[[(0,0),(10,0),(10,10),(0,10)]],[[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]]] │ MultiPolygon │ -└─────────────────────────────────────────────────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Геометрия {#geometry} - -`Geometry` — это общий тип для всех перечисленных выше типов. Он эквивалентен типу Variant, объединяющему эти типы. - -**Пример** - -```sql -CREATE TABLE IF NOT EXISTS geo (geom Geometry) ENGINE = Memory(); -INSERT INTO geo VALUES ((1, 2)); -SELECT * FROM geo; -``` - -Результат: - -```text - ┌─geom──┐ -1. │ (1,2) │ - └───────┘ -``` - -{/* */ } - -```sql -CREATE TABLE IF NOT EXISTS geo_dst (geom Geometry) ENGINE = Memory(); - -CREATE TABLE IF NOT EXISTS geo (geom String, id Int) ENGINE = Memory(); -INSERT INTO geo VALUES ('POLYGON((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 1); -INSERT INTO geo VALUES ('POINT(0 0)', 2); -INSERT INTO geo VALUES ('MULTIPOLYGON(((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4)),((-10 -10,-10 -9,-9 10,-10 -10)))', 3); -INSERT INTO geo VALUES ('LINESTRING(1 0,10 0,10 10,0 10,1 0)', 4); -INSERT INTO geo VALUES ('MULTILINESTRING((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 5); -INSERT INTO geo_dst SELECT readWkt(geom) FROM geo ORDER BY id; - -SELECT * FROM geo_dst; -``` - -Результат: - -```text - ┌─geom─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -1. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ -2. │ (0,0) │ -3. │ [[[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]],[[(-10,-10),(-10,-9),(-9,10),(-10,-10)]]] │ -4. │ [(1,0),(10,0),(10,10),(0,10),(1,0)] │ -5. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ - └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -## Связанные материалы {#related-content} - -- [Исследование масштабных реальных наборов данных: более чем 100 лет метеорологических наблюдений в ClickHouse](https://clickhouse.com/blog/real-world-data-noaa-climate-data) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md deleted file mode 100644 index 0b7be683e35..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'Документация по типам данных в ClickHouse' -sidebar_label: 'Список типов данных' -sidebar_position: 1 -slug: /sql-reference/data-types/ -title: 'Типы данных в ClickHouse' -doc_type: 'reference' ---- - -# Типы данных в ClickHouse {#data-types-in-clickhouse} - -В этом разделе описываются типы данных, поддерживаемые ClickHouse, например [целые числа](int-uint.md), [числа с плавающей запятой](float.md) и [строки](string.md). - -Системная таблица [system.data_type_families](/operations/system-tables/data_type_families) содержит -обзор всех доступных типов данных. -Она также показывает, является ли тип данных псевдонимом другого типа данных и чувствительно ли его имя к регистру (например, `bool` в отличие от `BOOL`). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md deleted file mode 100644 index 19f876e8f63..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'Документация по знаковым и беззнаковым целочисленным типам данных в ClickHouse - с разрядностью от 8 до 256 бит' -sidebar_label: 'Int | UInt' -sidebar_position: 2 -slug: /sql-reference/data-types/int-uint -title: 'Типы Int | UInt' -doc_type: 'reference' ---- - -ClickHouse предоставляет ряд целочисленных типов фиксированного размера, -со знаком (`Int`) или без знака (беззнаковый `UInt`) размером от одного до 32 байт. - -При создании таблиц можно задавать числовые параметры для целочисленных типов (например, `TINYINT(8)`, `SMALLINT(16)`, `INT(32)`, `BIGINT(64)`), но ClickHouse их игнорирует. - - - -## Диапазоны целых чисел {#integer-ranges} - -Целочисленные типы данных имеют следующие диапазоны: - -| Тип | Диапазон | -|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `Int8` | \[-128 : 127\] | -| `Int16` | \[-32768 : 32767\] | -| `Int32` | \[-2147483648 : 2147483647\] | -| `Int64` | \[-9223372036854775808 : 9223372036854775807\] | -| `Int128` | \[-170141183460469231731687303715884105728 : 170141183460469231731687303715884105727\] | -| `Int256` | \[-57896044618658097711785492504343953926634992332820282019728792003956564819968 : 57896044618658097711785492504343953926634992332820282019728792003956564819967\] | - -Беззнаковые целочисленные типы данных имеют следующие диапазоны: - -| Тип | Диапазон | -|-----------|----------------------------------------------------------------------------------------| -| `UInt8` | \[0 : 255\] | -| `UInt16` | \[0 : 65535\] | -| `UInt32` | \[0 : 4294967295\] | -| `UInt64` | \[0 : 18446744073709551615\] | -| `UInt128` | \[0 : 340282366920938463463374607431768211455\] | -| `UInt256` | \[0 : 115792089237316195423570985008687907853269984665640564039457584007913129639935\] | - - - -## Псевдонимы целочисленных типов {#integer-aliases} - -Целочисленные типы имеют следующие псевдонимы: - -| Тип | Псевдоним | -|---------|-----------------------------------------------------------------------------------| -| `Int8` | `TINYINT`, `INT1`, `BYTE`, `TINYINT SIGNED`, `INT1 SIGNED` | -| `Int16` | `SMALLINT`, `SMALLINT SIGNED` | -| `Int32` | `INT`, `INTEGER`, `MEDIUMINT`, `MEDIUMINT SIGNED`, `INT SIGNED`, `INTEGER SIGNED` | -| `Int64` | `BIGINT`, `SIGNED`, `BIGINT SIGNED`, `TIME` | - -Беззнаковые целочисленные типы имеют следующие псевдонимы: - -| Тип | Псевдоним | -|----------|-----------------------------------------------------------| -| `UInt8` | `TINYINT UNSIGNED`, `INT1 UNSIGNED` | -| `UInt16` | `SMALLINT UNSIGNED` | -| `UInt32` | `MEDIUMINT UNSIGNED`, `INT UNSIGNED`, `INTEGER UNSIGNED` | -| `UInt64` | `UNSIGNED`, `BIGINT UNSIGNED`, `BIT`, `SET` | diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md deleted file mode 100644 index dfccd848264..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'Документация по типу данных IPv4 в ClickHouse' -sidebar_label: 'IPv4' -sidebar_position: 28 -slug: /sql-reference/data-types/ipv4 -title: 'IPv4' -doc_type: 'reference' ---- - -## IPv4 {#ipv4} - -IPv4-адреса. Хранятся в 4 байтах в виде UInt32. - -### Базовое использование {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv4 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -Или можно использовать домен IPv4 в качестве ключа: - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY from; -``` - -Домен `IPv4` поддерживает особый формат ввода — строки IPv4: - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '116.253.40.133')('https://clickhouse.com', '183.247.232.58')('https://clickhouse.com/docs/en/', '116.106.34.242'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬───────────from─┐ -│ https://clickhouse.com/docs/en/ │ 116.106.34.242 │ -│ https://wikipedia.org │ 116.253.40.133 │ -│ https://clickhouse.com │ 183.247.232.58 │ -└────────────────────────────────────┴────────────────┘ -``` - -Значения хранятся в компактном двоичном формате: - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)─┐ -│ IPv4 │ B7F7E83A │ -└──────────────────┴───────────┘ -``` - -Адреса IPv4 можно сравнивать напрямую с адресами IPv6: - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [Функции для работы с адресами IPv4 и IPv6](../functions/ip-address-functions.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md deleted file mode 100644 index 671abd7e4f1..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'Документация по типу данных IPv6 в ClickHouse, который хранит IPv6-адреса в виде 16-байтовых значений' -sidebar_label: 'IPv6' -sidebar_position: 30 -slug: /sql-reference/data-types/ipv6 -title: 'IPv6' -doc_type: 'reference' ---- - -## IPv6 {#ipv6} - -IPv6-адреса. Хранятся в 16 байтах в виде UInt128 в формате big-endian. - -### Базовое использование {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv6 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -Или вы можете использовать домен `IPv6` в качестве ключа: - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY from; -``` - -Домен `IPv6` поддерживает произвольный ввод строк в формате IPv6: - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '2a02:aa08:e000:3100::2')('https://clickhouse.com', '2001:44c8:129:2632:33:0:252:2')('https://clickhouse.com/docs/en/', '2a02:e980:1e::1'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬─from──────────────────────────┐ -│ https://clickhouse.com │ 2001:44c8:129:2632:33:0:252:2 │ -│ https://clickhouse.com/docs/en/ │ 2a02:e980:1e::1 │ -│ https://wikipedia.org │ 2a02:aa08:e000:3100::2 │ -└────────────────────────────────────┴───────────────────────────────┘ -``` - -Значения хранятся в компактном двоичном формате: - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)────────────────────────┐ -│ IPv6 │ 200144C8012926320033000002520002 │ -└──────────────────┴──────────────────────────────────┘ -``` - -Адреса IPv6 можно напрямую сравнивать с адресами IPv4: - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**См. также** - -* [Функции для работы с IP-адресами IPv4 и IPv6](../functions/ip-address-functions.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md deleted file mode 100644 index d257cf643df..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'Документация по оптимизации LowCardinality для столбцов строкового типа' -sidebar_label: 'LowCardinality(T)' -sidebar_position: 42 -slug: /sql-reference/data-types/lowcardinality -title: 'LowCardinality(T)' -doc_type: 'reference' ---- - - - -# LowCardinality(T) {#lowcardinalityt} - -Изменяет внутреннее представление других типов данных на словарно-кодированное. - - - -## Синтаксис {#syntax} - -```sql -LowCardinality(data_type) -``` - -**Параметры** - -* `data_type` — [String](../../sql-reference/data-types/string.md), [FixedString](../../sql-reference/data-types/fixedstring.md), [Date](../../sql-reference/data-types/date.md), [DateTime](../../sql-reference/data-types/datetime.md) и числовые типы данных, за исключением [Decimal](../../sql-reference/data-types/decimal.md). `LowCardinality` неэффективен для некоторых типов данных, см. описание настройки [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types). - - -## Описание {#description} - -`LowCardinality` — это надстройка, которая изменяет способ хранения данных и правила их обработки. ClickHouse применяет [словарное кодирование](https://en.wikipedia.org/wiki/Dictionary_coder) к столбцам типа `LowCardinality`. Работа со словарно закодированными данными существенно повышает производительность выполнения запросов [SELECT](../../sql-reference/statements/select/index.md) для многих приложений. - -Эффективность использования типа данных `LowCardinality` зависит от разнообразия данных. Если словарь содержит менее 10 000 различных значений, ClickHouse в большинстве случаев показывает более высокую эффективность чтения и хранения данных. Если словарь содержит более 100 000 различных значений, ClickHouse может работать хуже по сравнению с использованием обычных типов данных. - -Рассмотрите возможность использования `LowCardinality` вместо [Enum](../../sql-reference/data-types/enum.md) при работе со строками. `LowCardinality` обеспечивает большую гибкость в использовании и часто демонстрирует такую же или более высокую эффективность. - - - -## Пример {#example} - -Создайте таблицу со столбцом типа `LowCardinality`: - -```sql -CREATE TABLE lc_t -( - `id` UInt16, - `strings` LowCardinality(String) -) -ENGINE = MergeTree() -ORDER BY id -``` - - -## Связанные настройки и функции {#related-settings-and-functions} - -Настройки: - -- [low_cardinality_max_dictionary_size](../../operations/settings/settings.md#low_cardinality_max_dictionary_size) -- [low_cardinality_use_single_dictionary_for_part](../../operations/settings/settings.md#low_cardinality_use_single_dictionary_for_part) -- [low_cardinality_allow_in_native_format](../../operations/settings/settings.md#low_cardinality_allow_in_native_format) -- [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types) -- [output_format_arrow_low_cardinality_as_dictionary](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary) - -Функции: - -- [toLowCardinality](../../sql-reference/functions/type-conversion-functions.md#tolowcardinality) - - - -## Связанные материалы {#related-content} - -- Блог: [Оптимизация ClickHouse с помощью схем и кодеков](https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema) -- Блог: [Работа с временными рядами в ClickHouse](https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse) -- [Оптимизация строк (видеодоклад на русском)](https://youtu.be/rqf-ILRgBdY?list=PL0Z2YDlm0b3iwXCpEFiOOYmwXzVmjJfEt). [Слайды на английском](https://github.com/ClickHouse/clickhouse-presentations/raw/master/meetup19/string_optimization.pdf) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md deleted file mode 100644 index ee20197e06d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -description: 'Документация по типу данных Map в ClickHouse' -sidebar_label: 'Map(K, V)' -sidebar_position: 36 -slug: /sql-reference/data-types/map -title: 'Map(K, V)' -doc_type: 'reference' ---- - - - -# Map(K, V) {#mapk-v} - -Тип данных `Map(K, V)` хранит пары «ключ–значение». - -В отличие от других баз данных, в ClickHouse элементы типа Map не обязаны быть уникальными, то есть Map может содержать два элемента с одинаковым ключом. -(Причина в том, что Map внутренне реализован как `Array(Tuple(K, V))`.) - -Вы можете использовать синтаксис `m[k]`, чтобы получить значение для ключа `k` в Map `m`. -Также операция `m[k]` последовательно сканирует Map, то есть время выполнения линейно зависит от размера Map. - -**Параметры** - -* `K` — тип ключей Map. Произвольный тип, за исключением [Nullable](../../sql-reference/data-types/nullable.md) и [LowCardinality](../../sql-reference/data-types/lowcardinality.md), совмещённых с типами [Nullable](../../sql-reference/data-types/nullable.md). -* `V` — тип значений Map. Произвольный тип. - -**Примеры** - -Создайте таблицу со столбцом типа Map: - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':1, 'key2':10}), ({'key1':2,'key2':20}), ({'key1':3,'key2':30}); -``` - -Чтобы выбрать значения `key2`: - -```sql -SELECT m['key2'] FROM tab; -``` - -Результат: - -```text -┌─arrayElement(m, 'key2')─┐ -│ 10 │ -│ 20 │ -│ 30 │ -└─────────────────────────┘ -``` - -Если запрошенный ключ `k` отсутствует в отображении (map), `m[k]` возвращает значение по умолчанию для типа значения, например `0` для целочисленных типов и `''` для строковых типов. -Чтобы проверить, существует ли ключ в отображении, можно использовать функцию [mapContains](../../sql-reference/functions/tuple-map-functions#mapcontains). - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':100}), ({}); -SELECT m['key1'] FROM tab; -``` - -Результат: - -```text -┌─arrayElement(m, 'key1')─┐ -│ 100 │ -│ 0 │ -└─────────────────────────┘ -``` - - -## Преобразование Tuple в Map {#converting-tuple-to-map} - -Значения типа `Tuple()` можно привести к значениям типа `Map()` с помощью функции [CAST](/sql-reference/functions/type-conversion-functions#cast): - -**Пример** - -Запрос: - -```sql -SELECT CAST(([1, 2, 3], ['Ready', 'Steady', 'Go']), 'Map(UInt8, String)') AS map; -``` - -Результат: - -```text -┌─map───────────────────────────┐ -│ {1:'Готово',2:'Внимание',3:'Марш'} │ -└───────────────────────────────┘ -``` - - -## Чтение подстолбцов Map {#reading-subcolumns-of-map} - -Чтобы избежать чтения всего столбца Map, в некоторых случаях можно использовать подстолбцы `keys` и `values`. - -**Пример** - -Запрос: - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE = Memory; -INSERT INTO tab VALUES (map('key1', 1, 'key2', 2, 'key3', 3)); - -SELECT m.keys FROM tab; -- то же, что mapKeys(m) -SELECT m.values FROM tab; -- то же, что mapValues(m) -``` - -Результат: - -```text -┌─m.keys─────────────────┐ -│ ['key1','key2','key3'] │ -└────────────────────────┘ - -┌─m.values─┐ -│ [1,2,3] │ -└──────────┘ -``` - -**См. также** - -* Функция [map()](/sql-reference/functions/tuple-map-functions#map) -* Функция [CAST()](/sql-reference/functions/type-conversion-functions#cast) -* [-Map-комбинатор для типа данных Map](../aggregate-functions/combinators.md#-map) - - -## Связанные материалы {#related-content} - -- Блог: [Решение для наблюдаемости на базе ClickHouse — часть 2: трейсы](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md deleted file mode 100644 index bc119df997c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -description: 'Обзор вложенных структур данных в ClickHouse' -sidebar_label: 'Nested(Name1 Type1, Name2 Type2, ...)' -sidebar_position: 57 -slug: /sql-reference/data-types/nested-data-structures/nested -title: 'Nested' -doc_type: 'guide' ---- - -# Вложенные структуры {#nested} - -## Nested(name1 Type1, Name2 Type2, ...) {#nestedname1-type1-name2-type2-} - -Вложенная структура данных похожа на таблицу внутри ячейки. Параметры вложенной структуры данных — имена столбцов и их типы — задаются так же, как в запросе [CREATE TABLE](../../../sql-reference/statements/create/table.md). Каждая строка таблицы может соответствовать произвольному количеству строк во вложенной структуре данных. - -Пример: - -```sql -CREATE TABLE test.visits -( - CounterID UInt32, - StartDate Date, - Sign Int8, - IsNew UInt8, - VisitID UInt64, - UserID UInt64, - ... - Goals Nested - ( - ID UInt32, - Serial UInt32, - EventTime DateTime, - Price Int64, - OrderID String, - CurrencyID UInt32 - ), - ... -) ENGINE = CollapsingMergeTree(StartDate, intHash32(UserID), (CounterID, StartDate, intHash32(UserID), VisitID), 8192, Sign) -``` - -В этом примере определяется вложенная структура данных `Goals`, которая содержит данные о конверсиях (достигнутых целях). Каждой строке в таблице 'visits' может соответствовать ноль или любое количество конверсий. - -Когда [flatten_nested](/operations/settings/settings#flatten_nested) установлен в `0` (что не является значением по умолчанию), поддерживаются произвольные уровни вложенности. - -В большинстве случаев при работе с вложенной структурой данных её столбцы задаются именами, разделёнными точкой. Эти столбцы образуют массив соответствующих типов. Все массивы столбцов одной вложенной структуры данных имеют одинаковую длину. - -Пример: - -```sql -SELECT - Goals.ID, - Goals.EventTime -FROM test.visits -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goals.ID───────────────────────┬─Goals.EventTime───────────────────────────────────────────────────────────────────────────┐ -│ [1073752,591325,591325] │ ['2014-03-17 16:38:10','2014-03-17 16:38:48','2014-03-17 16:42:27'] │ -│ [1073752] │ ['2014-03-17 00:28:25'] │ -│ [1073752] │ ['2014-03-17 10:46:20'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:59:20','2014-03-17 22:17:55','2014-03-17 22:18:07','2014-03-17 22:18:51'] │ -│ [] │ [] │ -│ [1073752,591325,591325] │ ['2014-03-17 11:37:06','2014-03-17 14:07:47','2014-03-17 14:36:21'] │ -│ [] │ [] │ -│ [] │ [] │ -│ [591325,1073752] │ ['2014-03-17 00:46:05','2014-03-17 00:46:05'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:28:33','2014-03-17 13:30:26','2014-03-17 18:51:21','2014-03-17 18:51:45'] │ -└────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -Проще всего воспринимать вложенную структуру данных как набор нескольких столбцов‑массивов одинаковой длины. - -Единственное место, где в запросе SELECT можно указать имя всей вложенной структуры данных вместо отдельных столбцов, — это предложение ARRAY JOIN. Для получения дополнительной информации см. раздел "ARRAY JOIN clause". Пример: - -```sql -SELECT - Goal.ID, - Goal.EventTime -FROM test.visits -ARRAY JOIN Goals AS Goal -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goal.ID─┬──────Goal.EventTime─┐ -│ 1073752 │ 2014-03-17 16:38:10 │ -│ 591325 │ 2014-03-17 16:38:48 │ -│ 591325 │ 2014-03-17 16:42:27 │ -│ 1073752 │ 2014-03-17 00:28:25 │ -│ 1073752 │ 2014-03-17 10:46:20 │ -│ 1073752 │ 2014-03-17 13:59:20 │ -│ 591325 │ 2014-03-17 22:17:55 │ -│ 591325 │ 2014-03-17 22:18:07 │ -│ 591325 │ 2014-03-17 22:18:51 │ -│ 1073752 │ 2014-03-17 11:37:06 │ -└─────────┴─────────────────────┘ -``` - -Нельзя выполнить SELECT для всей вложенной структуры данных целиком. Можно только явно перечислить отдельные столбцы, которые в неё входят. - -Для запроса INSERT нужно передавать все массивы столбцов, составляющие вложенную структуру данных, по отдельности (как будто это отдельные массивы столбцов). При вставке система проверяет, что их длины совпадают. - -Для запроса DESCRIBE столбцы во вложенной структуре данных перечисляются по отдельности аналогичным образом. - -Запрос ALTER для элементов вложенной структуры данных имеет ограничения. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md deleted file mode 100644 index 63c083f7c6f..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md +++ /dev/null @@ -1,1065 +0,0 @@ ---- -description: 'Документация по типу данных JSON в ClickHouse, который обеспечивает - встроенную поддержку работы с JSON-данными' -keywords: ['json', 'data type'] -sidebar_label: 'JSON' -sidebar_position: 63 -slug: /sql-reference/data-types/newjson -title: 'Тип данных JSON' -doc_type: 'reference' ---- - -import {CardSecondary} from '@clickhouse/click-ui/bundled'; -import Link from '@docusaurus/Link' - - - - - -
- -Тип `JSON` хранит документы в формате JavaScript Object Notation (JSON) в одном столбце. - -:::note -В ClickHouse Open-Source тип данных JSON признан готовым для промышленного использования начиная с версии 25.3. Не рекомендуется использовать этот тип в продакшене в более ранних версиях. -::: - -Чтобы объявить столбец типа `JSON`, используйте следующий синтаксис: - -```sql - JSON -( - max_dynamic_paths=N, - max_dynamic_types=M, - some.path TypeName, - SKIP path.to.skip, - SKIP REGEXP 'paths_regexp' -) -``` - -Где параметры в приведённом выше синтаксисе определяются следующим образом: - -| Параметр | Описание | Значение по умолчанию | -| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -| `max_dynamic_paths` | Необязательный параметр, задающий, сколько путей может храниться отдельно в виде подстолбцов в одном отдельно хранимом блоке данных (например, в одной части данных таблицы MergeTree).

Если этот лимит превышен, все остальные пути будут сохранены вместе в одной общей структуре. | `1024` | -| `max_dynamic_types` | Необязательный параметр в диапазоне от `1` до `255`, задающий, сколько различных типов данных может храниться внутри одного столбца пути типа `Dynamic` в одном отдельно хранимом блоке данных (например, в одной части данных таблицы MergeTree).

Если этот лимит превышен, все новые типы будут приведены к типу `String`. | `32` | -| `some.path TypeName` | Необязательная подсказка типа для конкретного пути в JSON. Такие пути всегда будут храниться как подстолбцы с указанным типом. | | -| `SKIP path.to.skip` | Необязательная подсказка для конкретного пути, который должен быть пропущен при разборе JSON. Такие пути никогда не будут сохранены в столбце JSON. Если указанный путь является вложенным объектом JSON, весь вложенный объект будет пропущен. | | -| `SKIP REGEXP 'path_regexp'` | Необязательная подсказка с регулярным выражением, которое используется для пропуска путей при разборе JSON. Все пути, соответствующие этому регулярному выражению, никогда не будут сохранены в столбце JSON. | | - - -## Создание JSON {#creating-json} - -В этом разделе мы рассмотрим различные способы создания `JSON`. - -### Использование `JSON` в определении столбца таблицы {#using-json-in-a-table-column-definition} - -```sql title="Query (Example 1)" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 1)" -┌─json────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ -│ {"f":"Привет, мир!"} │ -│ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │ -└─────────────────────────────────────────────┘ -``` - -```sql title="Query (Example 2)" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 2)" -┌─json──────────────────────────────┐ -│ {"a":{"b":42},"c":["1","2","3"]} │ -│ {"a":{"b":0},"f":"Привет, мир!"} │ -│ {"a":{"b":43},"c":["4","5","6"]} │ -└───────────────────────────────────┘ -``` - -### Использование CAST с `::JSON` {#using-cast-with-json} - -Можно приводить различные типы данных с помощью специального синтаксиса `::JSON`. - -#### CAST из типа `String` в тип `JSON` {#cast-from-string-to-json} - -```sql title="Query" -SELECT '{"a" : {"b" : 42},"c" : [1, 2, 3], "d" : "Hello, World!"}'::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"Привет, мир!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### Приведение типа из `Tuple` к `JSON` {#cast-from-tuple-to-json} - -```sql title="Query" -SET enable_named_columns_in_function_tuple = 1; -SELECT (tuple(42 AS b) AS a, [1, 2, 3] AS c, 'Hello, World!' AS d)::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"Привет, мир!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### Преобразование (CAST) из `Map` в `JSON` {#cast-from-map-to-json} - -```sql title="Query" -SET use_variant_as_common_type=1; -SELECT map('a', map('b', 42), 'c', [1,2,3], 'd', 'Hello, World!')::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"Привет, мир!"} │ -└────────────────────────────────────────────────────────┘ -``` - -:::note -JSON-пути хранятся в плоском виде. Это означает, что когда JSON-объект формируется на основе пути вида `a.b.c`, -невозможно однозначно определить, должен ли объект быть восстановлен как `{ "a.b.c" : ... }` или как `{ "a": { "b": { "c": ... } } }`. -Наша реализация всегда будет предполагать второй вариант. - -Например: - -```sql -SELECT CAST('{"a.b.c" : 42}', 'JSON') AS json -``` - -вернет: - -```response - ┌─json───────────────────┐ -1. │ {"a":{"b":{"c":"42"}}} │ - └────────────────────────┘ -``` - -а **не**: - - -```sql - ┌─json───────────┐ -1. │ {"a.b.c":"42"} │ - └────────────────┘ -``` - -::: - - -## Чтение JSON-путей как подстолбцов {#reading-json-paths-as-sub-columns} - -Тип `JSON` поддерживает чтение каждого пути как отдельного подстолбца. -Если тип запрашиваемого пути не указан в объявлении типа `JSON`, -то подстолбец для этого пути всегда будет иметь тип [Dynamic](/sql-reference/data-types/dynamic.md). - -Например: - -```sql title="Query" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42, "g" : 42.42}, "c" : [1, 2, 3], "d" : "2020-01-01"}'), ('{"f" : "Hello, World!", "d" : "2020-01-02"}'), ('{"a" : {"b" : 43, "e" : 10, "g" : 43.43}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────┐ -│ {"a":{"b":42,"g":42.42},"c":["1","2","3"],"d":"2020-01-01"} │ -│ {"a":{"b":0},"d":"2020-01-02","f":"Hello, World!"} │ -│ {"a":{"b":43,"g":43.43},"c":["4","5","6"]} │ -└─────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query (Reading JSON paths as sub-columns)" -SELECT json.a.b, json.a.g, json.c, json.d FROM test; -``` - -```text title="Response (Reading JSON paths as sub-columns)" -┌─json.a.b─┬─json.a.g─┬─json.c──┬─json.d─────┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└──────────┴──────────┴─────────┴────────────┘ -``` - -Вы также можете использовать функцию `getSubcolumn`, чтобы читать подколонки из типа JSON: - -```sql title="Query" -SELECT getSubcolumn(json, 'a.b'), getSubcolumn(json, 'a.g'), getSubcolumn(json, 'c'), getSubcolumn(json, 'd') FROM test; -``` - -```text title="Response" -┌─getSubcolumn(json, 'a.b')─┬─getSubcolumn(json, 'a.g')─┬─getSubcolumn(json, 'c')─┬─getSubcolumn(json, 'd')─┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└───────────────────────────┴───────────────────────────┴─────────────────────────┴─────────────────────────┘ -``` - -Если запрошенный путь отсутствует в данных, он будет заполнен значениями `NULL`: - -```sql title="Query" -SELECT json.non.existing.path FROM test; -``` - -```text title="Response" -┌─json.non.existing.path─┐ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -└────────────────────────┘ -``` - -Проверим типы данных возвращённых подстолбцов: - -```sql title="Query" -SELECT toTypeName(json.a.b), toTypeName(json.a.g), toTypeName(json.c), toTypeName(json.d) FROM test; -``` - - -```text title="Response" -┌─toTypeName(json.a.b)─┬─toTypeName(json.a.g)─┬─toTypeName(json.c)─┬─toTypeName(json.d)─┐ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -└──────────────────────┴──────────────────────┴────────────────────┴────────────────────┘ -``` - -Как мы видим, для `a.b` тип — `UInt32`, как и было задано в объявлении типа JSON, -а для всех остальных подстолбцов тип — `Dynamic`. - -Также можно читать подстолбцы столбца типа `Dynamic`, используя специальный синтаксис `json.some.path.:TypeName`: - -```sql title="Query" -SELECT - json.a.g.:Float64, - dynamicType(json.a.g), - json.d.:Date, - dynamicType(json.d) -FROM test -``` - -```text title="Response" -┌─json.a.g.:`Float64`─┬─dynamicType(json.a.g)─┬─json.d.:`Date`─┬─dynamicType(json.d)─┐ -│ 42.42 │ Float64 │ 2020-01-01 │ Date │ -│ ᴺᵁᴸᴸ │ None │ 2020-01-02 │ Date │ -│ 43.43 │ Float64 │ ᴺᵁᴸᴸ │ None │ -└─────────────────────┴───────────────────────┴────────────────┴─────────────────────┘ -``` - -Подстолбцы типа `Dynamic` можно привести к любому типу данных. При этом будет сгенерировано исключение, если внутренний тип в `Dynamic` не может быть приведён к запрошенному типу: - -```sql title="Query" -SELECT json.a.g::UInt64 AS uint -FROM test; -``` - -```text title="Response" -┌─uint─┐ -│ 42 │ -│ 0 │ -│ 43 │ -└──────┘ -``` - -```sql title="Query" -SELECT json.a.g::UUID AS float -FROM test; -``` - -```text title="Response" -Получено исключение от сервера: -Код: 48. DB::Exception: Получено от localhost:9000. DB::Exception: -Преобразование между числовыми типами и UUID не поддерживается. -Вероятно, переданный UUID не заключен в кавычки: -при выполнении 'FUNCTION CAST(__table1.json.a.g :: 2, 'UUID'_String :: 1) -> CAST(__table1.json.a.g, 'UUID'_String) UUID : 0'. -(NOT_IMPLEMENTED) -``` - -:::note -Чтобы эффективно считывать подстолбцы из частей Compact MergeTree, убедитесь, что настройка MergeTree [write_marks_for_substreams_in_compact_parts](../../operations/settings/merge-tree-settings.md#write_marks_for_substreams_in_compact_parts) включена. -::: - - -## Чтение вложенных объектов JSON как подстолбцов {#reading-json-sub-objects-as-sub-columns} - -Тип `JSON` поддерживает чтение вложенных объектов как подстолбцов типа `JSON` с использованием специального синтаксиса `json.^some.path`: - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : {"c" : 42, "g" : 42.42}}, "c" : [1, 2, 3], "d" : {"e" : {"f" : {"g" : "Hello, World", "h" : [1, 2, 3]}}}}'), ('{"f" : "Hello, World!", "d" : {"e" : {"f" : {"h" : [4, 5, 6]}}}}'), ('{"a" : {"b" : {"c" : 43, "e" : 10, "g" : 43.43}}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":"42","g":42.42}},"c":["1","2","3"],"d":{"e":{"f":{"g":"Hello, World","h":["1","2","3"]}}}} │ -│ {"d":{"e":{"f":{"h":["4","5","6"]}}},"f":"Hello, World!"} │ -│ {"a":{"b":{"c":"43","e":"10","g":43.43}},"c":["4","5","6"]} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.^a.b, json.^d.e.f FROM test; -``` - -```text title="Response" -┌─json.^`a`.b───────────────────┬─json.^`d`.e.f──────────────────────────┐ -│ {"c":"42","g":42.42} │ {"g":"Hello, World","h":["1","2","3"]} │ -│ {} │ {"h":["4","5","6"]} │ -│ {"c":"43","e":"10","g":43.43} │ {} │ -└───────────────────────────────┴────────────────────────────────────────┘ -``` - -:::note -Чтение вложенных объектов как подстолбцов может быть неэффективным, так как для этого может потребоваться практически полное сканирование данных JSON. -::: - - -## Определение типов для путей {#type-inference-for-paths} - -Во время разбора `JSON` ClickHouse пытается определить наиболее подходящий тип данных для каждого пути в JSON. -Это работает аналогично [автоматическому определению схемы по входным данным](/interfaces/schema-inference.md) -и управляется теми же настройками: - -* [input_format_try_infer_dates](/operations/settings/formats#input_format_try_infer_dates) -* [input_format_try_infer_datetimes](/operations/settings/formats#input_format_try_infer_datetimes) -* [schema_inference_make_columns_nullable](/operations/settings/formats#schema_inference_make_columns_nullable) -* [input_format_json_try_infer_numbers_from_strings](/operations/settings/formats#input_format_json_try_infer_numbers_from_strings) -* [input_format_json_infer_incomplete_types_as_strings](/operations/settings/formats#input_format_json_infer_incomplete_types_as_strings) -* [input_format_json_read_numbers_as_strings](/operations/settings/formats#input_format_json_read_numbers_as_strings) -* [input_format_json_read_bools_as_strings](/operations/settings/formats#input_format_json_read_bools_as_strings) -* [input_format_json_read_bools_as_numbers](/operations/settings/formats#input_format_json_read_bools_as_numbers) -* [input_format_json_read_arrays_as_strings](/operations/settings/formats#input_format_json_read_arrays_as_strings) -* [input_format_json_infer_array_of_dynamic_from_array_of_different_types](/operations/settings/formats#input_format_json_infer_array_of_dynamic_from_array_of_different_types) - -Рассмотрим несколько примеров: - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=1, input_format_try_infer_datetimes=1; -``` - -```text title="Response" -┌─paths_with_types─────────────────┐ -│ {'a':'Date','b':'DateTime64(9)'} │ -└──────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=0, input_format_try_infer_datetimes=0; -``` - -```text title="Response" -┌─paths_with_types────────────┐ -│ {'a':'String','b':'String'} │ -└─────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types settings schema_inference_make_columns_nullable=1; -``` - -```text title="Response" -┌─paths_with_types───────────────┐ -│ {'a':'Array(Nullable(Int64))'} │ -└────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types settings schema_inference_make_columns_nullable=0; -``` - -```text title="Response" -┌─paths_with_types─────┐ -│ {'a':'Array(Int64)'} │ -└──────────────────────┘ -``` - - -## Обработка массивов JSON-объектов {#handling-arrays-of-json-objects} - -JSON-пути, содержащие массив объектов, интерпретируются как тип `Array(JSON)` и записываются в столбец `Dynamic` для этого пути. -Чтобы прочитать массив объектов, вы можете извлечь его из столбца `Dynamic` в виде подстолбца: - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES -('{"a" : {"b" : [{"c" : 42, "d" : "Hello", "f" : [[{"g" : 42.42}]], "k" : {"j" : 1000}}, {"c" : 43}, {"e" : [1, 2, 3], "d" : "My", "f" : [[{"g" : 43.43, "h" : "2020-01-01"}]], "k" : {"j" : 2000}}]}}'), -('{"a" : {"b" : [1, 2, 3]}}'), -('{"a" : {"b" : [{"c" : 44, "f" : [[{"h" : "2020-01-02"}]]}, {"e" : [4, 5, 6], "d" : "World", "f" : [[{"g" : 44.44}]], "k" : {"j" : 3000}}]}}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":[{"c":"42","d":"Привет","f":[[{"g":42.42}]],"k":{"j":"1000"}},{"c":"43"},{"d":"Мой","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}]}} │ -│ {"a":{"b":["1","2","3"]}} │ -│ {"a":{"b":[{"c":"44","f":[[{"h":"2020-01-02"}]]},{"d":"Мир","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}]}} │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.a.b, dynamicType(json.a.b) FROM test; -``` - -```text title="Response" -┌─json.a.b──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─dynamicType(json.a.b)────────────────────────────────────┐ -│ ['{"c":"42","d":"Hello","f":[[{"g":42.42}]],"k":{"j":"1000"}}','{"c":"43"}','{"d":"My","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -│ [1,2,3] │ Array(Nullable(Int64)) │ -│ ['{"c":"44","f":[[{"h":"2020-01-02"}]]}','{"d":"World","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────┘ -``` - -Как вы, возможно, заметили, параметры `max_dynamic_types`/`max_dynamic_paths` вложенного типа `JSON` были уменьшены по сравнению со значениями по умолчанию. -Это необходимо, чтобы избежать неконтролируемого увеличения числа подстолбцов во вложенных массивах объектов JSON. - -Попробуем прочитать подстолбцы из вложенного столбца `JSON`: - -```sql title="Query" -SELECT json.a.b.:`Array(JSON)`.c, json.a.b.:`Array(JSON)`.f, json.a.b.:`Array(JSON)`.d FROM test; -``` - - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -Мы можем избежать необходимости указывать имена подстолбцов типа `Array(JSON)`, используя специальный синтаксис: - -```sql title="Query" -SELECT json.a.b[].c, json.a.b[].f, json.a.b[].d FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -Количество скобок `[]` после пути указывает уровень массива. Например, `json.path[][]` будет преобразовано в `json.path.:Array(Array(JSON))`. - -Давайте проверим пути и типы внутри нашего `Array(JSON)`: - -```sql title="Query" -SELECT DISTINCT arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b[]))) FROM test; -``` - -```text title="Response" -┌─arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b.:`Array(JSON)`)))──┐ -│ ('c','Int64') │ -│ ('d','String') │ -│ ('f','Array(Array(JSON(max_dynamic_types=8, max_dynamic_paths=64)))') │ -│ ('k.j','Int64') │ -│ ('e','Array(Nullable(Int64))') │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -Прочитаем подстолбцы из столбца `Array(JSON)`: - -```sql title="Query" -SELECT json.a.b[].c.:Int64, json.a.b[].f[][].g.:Float64, json.a.b[].f[][].h.:Date FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c.:`Int64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.g.:`Float64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.h.:`Date`─┐ -│ [42,43,NULL] │ [[[42.42]],[],[[43.43]]] │ [[[NULL]],[],[['2020-01-01']]] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[[NULL]],[[44.44]]] │ [[['2020-01-02']],[[NULL]]] │ -└────────────────────────────────────┴──────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────┘ -``` - -Также можно считывать подстолбцы вложенных объектов из вложенного столбца `JSON`: - -```sql title="Query" -SELECT json.a.b[].^k FROM test -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.^`k`─────────┐ -│ ['{"j":"1000"}','{}','{"j":"2000"}'] │ -│ [] │ -│ ['{}','{"j":"3000"}'] │ -└──────────────────────────────────────┘ -``` - - -## Обработка JSON-ключей со значением NULL {#handling-json-keys-with-nulls} - -В нашей реализации JSON значение `null` и отсутствие значения считаются эквивалентными: - -```sql title="Query" -SELECT '{}'::JSON AS json1, '{"a" : null}'::JSON AS json2, json1 = json2 -``` - -```text title="Response" -┌─json1─┬─json2─┬─equals(json1, json2)─┐ -│ {} │ {} │ 1 │ -└───────┴───────┴──────────────────────┘ -``` - -Это означает, что невозможно определить, содержали ли исходные данные JSON какой‑либо путь со значением NULL или не содержали его вовсе. - - -## Обработка ключей JSON с точками {#handling-json-keys-with-dots} - -Внутренне столбец JSON хранит все пути и значения в виде плоской структуры. Это означает, что по умолчанию следующие два объекта считаются одинаковыми: - -```json -{"a" : {"b" : 42}} -{"a.b" : 42} -``` - -Оба они во внутреннем представлении хранятся как пара: путь `a.b` и значение `42`. При форматировании JSON мы всегда формируем вложенные объекты на основе частей пути, разделённых точкой: - -```sql title="Query" -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a":{"b":"42"}} │ ['a.b'] │ ['a.b'] │ -└──────────────────┴──────────────────┴─────────────────────┴─────────────────────┘ -``` - -Как видите, исходный JSON `{"a.b" : 42}` теперь имеет вид `{"a" : {"b" : 42}}`. - -Это ограничение также приводит к ошибке при разборе корректных JSON-объектов, подобных этому: - -```sql title="Query" -SELECT '{"a.b" : 42, "a" : {"b" : "Привет, мир!"}}'::JSON AS json; -``` - -```text title="Response" -Код: 117. DB::Exception: Невозможно вставить данные в JSON-колонку: При разборе JSON-объекта обнаружен дублирующийся путь: a.b. Вы можете включить настройку type_json_skip_duplicated_paths, чтобы пропускать дублирующиеся пути при вставке: В области видимости SELECT CAST('{"a.b" : 42, "a" : {"b" : "Hello, World"}}', 'JSON') AS json. (INCORRECT_DATA) -``` - -Если вы хотите сохранить ключи с точками и избежать их интерпретации как вложенные объекты, вы можете включить настройку [json_type_escape_dots_in_keys](/operations/settings/formats#json_type_escape_dots_in_keys) (доступна, начиная с версии `25.8`). В этом случае при разборе JSON все точки в ключах будут экранированы в `%2E` и разэкранированы обратно при форматировании. - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a.b":"42"} │ ['a.b'] │ ['a%2Eb'] │ -└──────────────────┴──────────────┴─────────────────────┴─────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, JSONAllPaths(json); -``` - -```text title="Response" -┌─json──────────────────────────────────┬─JSONAllPaths(json)─┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ ['a%2Eb','a.b'] │ -└───────────────────────────────────────┴────────────────────┘ -``` - -Чтобы прочитать ключ с экранированной точкой как подстолбец, нужно использовать экранированную точку в имени подстолбца: - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┘ -``` - -Примечание: из-за ограничений парсера и анализатора идентификаторов подколонка `` json.`a.b` `` эквивалентна подколонке `json.a.b` и не распознаёт путь с экранированной точкой: - - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.`a.b`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┴──────────────┘ -``` - -Кроме того, если вы хотите указать подсказку для JSON-пути, содержащего ключи с точками (или использовать эту подсказку в секциях `SKIP`/`SKIP REGEX`), необходимо экранировать точки в подсказке: - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(`a%2Eb` UInt8) as json, json.`a%2Eb`, toTypeName(json.`a%2Eb`); -``` - -```text title="Response" -┌─json────────────────────────────────┬─json.a%2Eb─┬─toTypeName(json.a%2Eb)─┐ -│ {"a.b":42,"a":{"b":"Hello World!"}} │ 42 │ UInt8 │ -└─────────────────────────────────────┴────────────┴────────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(SKIP `a%2Eb`) as json, json.`a%2Eb`; -``` - -```text title="Response" -┌─json───────────────────────┬─json.a%2Eb─┐ -│ {"a":{"b":"Привет, мир!"}} │ ᴺᵁᴸᴸ │ -└────────────────────────────┴────────────┘ -``` - - -## Чтение типа JSON из данных {#reading-json-type-from-data} - -Все текстовые форматы -([`JSONEachRow`](/interfaces/formats/JSONEachRow), -[`TSV`](/interfaces/formats/TabSeparated), -[`CSV`](/interfaces/formats/CSV), -[`CustomSeparated`](/interfaces/formats/CustomSeparated), -[`Values`](/interfaces/formats/Values) и т. д.) поддерживают чтение значений типа `JSON`. - -Примеры: - -```sql title="Query" -SELECT json FROM format(JSONEachRow, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP d.e, SKIP REGEXP \'b.*\')', ' -{"json" : {"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}}} -{"json" : {"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}}} -{"json" : {"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"}} -{"json" : {"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43}} -{"json" : {"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}} -') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"Hello, World!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - -Для текстовых форматов, таких как `CSV`/`TSV`/и т. д., `JSON` разбирается из строки, содержащей объект JSON: - - -```sql title="Query" -SELECT json FROM format(TSV, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP REGEXP \'b.*\')', -'{"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}} -{"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}} -{"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"} -{"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43} -{"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"Hello, World!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - - -## Достижение предела динамических путей внутри JSON {#reaching-the-limit-of-dynamic-paths-inside-json} - -Тип данных `JSON` может хранить только ограниченное количество путей как отдельных внутренних подстолбцов. -По умолчанию этот предел равен `1024`, но вы можете изменить его в объявлении типа с помощью параметра `max_dynamic_paths`. - -Когда предел достигнут, все новые пути, вставляемые в столбец `JSON`, будут храниться в единой общей структуре данных. -По-прежнему можно считывать такие пути как подстолбцы, -но это может быть менее эффективно ([см. раздел об общей структуре данных](#shared-data-structure)). -Этот предел необходим для того, чтобы избежать чрезмерно большого числа различных подстолбцов, которое может сделать таблицу непригодной к использованию. - -Рассмотрим, что происходит при достижении предела в нескольких различных сценариях. - -### Достижение предела во время разбора данных {#reaching-the-limit-during-data-parsing} - -Во время разбора `JSON`-объектов из данных, когда предел достигнут для текущего блока данных, -все новые пути будут храниться в общей структуре данных. Мы можем использовать следующие две функции интроспекции: `JSONDynamicPaths`, `JSONSharedDataPaths`: - -```sql title="Query" -SELECT json, JSONDynamicPaths(json), JSONSharedDataPaths(json) FROM format(JSONEachRow, 'json JSON(max_dynamic_paths=3)', ' -{"json" : {"a" : {"b" : 42}, "c" : [1, 2, 3]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-01"}} -{"json" : {"a" : {"b" : 44}, "c" : [4, 5, 6]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-02", "e" : "Hello", "f" : {"g" : 42.42}}} -{"json" : {"a" : {"b" : 43}, "c" : [7, 8, 9], "f" : {"g" : 43.43}, "h" : "World"}} -') -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────────────┬─JSONDynamicPaths(json)─┬─JSONSharedDataPaths(json)─┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-01"} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"44"},"c":["4","5","6"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-02","e":"Hello","f":{"g":42.42}} │ ['a.b','c','d'] │ ['e','f.g'] │ -│ {"a":{"b":"43"},"c":["7","8","9"],"f":{"g":43.43},"h":"World"} │ ['a.b','c','d'] │ ['f.g','h'] │ -└────────────────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────┘ -``` - -Как мы видим, после вставки путей `e` и `f.g` лимит был достигнут, -и они были помещены в общую структуру данных. - -### Во время слияний кусков данных в табличных движках MergeTree {#during-merges-of-data-parts-in-mergetree-table-engines} - -Во время слияния нескольких кусков данных в таблице `MergeTree` столбец `JSON` в результирующем куске данных может достичь лимита динамических путей -и не сможет хранить все пути из исходных кусков в виде подстолбцов. -В этом случае ClickHouse выбирает, какие пути останутся подстолбцами после слияния, а какие будут сохранены в общей структуре данных. -В большинстве случаев ClickHouse старается сохранять пути, которые содержат -наибольшее число ненулевых значений, а самые редкие пути переносить в общую структуру данных. Однако это зависит от реализации. - -Рассмотрим пример такого слияния. -Сначала создадим таблицу со столбцом `JSON`, установим лимит динамических путей, равный `3`, а затем вставим значения с `5` разными путями: - - -```sql title="Query" -CREATE TABLE test (id UInt64, json JSON(max_dynamic_paths=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as a) FROM numbers(5); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as b) FROM numbers(4); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as c) FROM numbers(3); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as d) FROM numbers(2); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as e) FROM numbers(1); -``` - -Каждая операция вставки будет создавать отдельную часть данных со столбцом `JSON`, содержащим один путь: - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 5 │ ['a'] │ [] │ all_1_1_0 │ -│ 4 │ ['b'] │ [] │ all_2_2_0 │ -│ 3 │ ['c'] │ [] │ all_3_3_0 │ -│ 2 │ ['d'] │ [] │ all_4_4_0 │ -│ 1 │ ['e'] │ [] │ all_5_5_0 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -Теперь давайте объединим все части воедино и посмотрим, что из этого получится: - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 15 │ ['a','b','c'] │ ['d','e'] │ all_1_5_2 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -Как мы видим, ClickHouse сохранил наиболее частые пути `a`, `b` и `c` и перенёс пути `d` и `e` в общую структуру данных. - - -## Общая структура данных {#shared-data-structure} - -Как было описано в предыдущем разделе, когда достигается предел `max_dynamic_paths`, все новые пути сохраняются в одной общей структуре данных. -В этом разделе мы рассмотрим детали общей структуры данных и то, как мы читаем из неё подстолбцы путей. - -См. раздел ["функции интроспекции"](/sql-reference/data-types/newjson#introspection-functions) для подробностей о функциях, используемых для анализа содержимого столбца JSON. - -### Общая структура данных в памяти {#shared-data-structure-in-memory} - -В памяти общая структура данных — это просто подстолбец с типом `Map(String, String)`, который хранит отображение от развёрнутого JSON-пути к двоично закодированному значению. -Чтобы извлечь из него подстолбец пути, мы просто итерируемся по всем строкам в этом столбце `Map` и пытаемся найти требуемый путь и его значения. - -### Общая структура данных в частях MergeTree {#shared-data-structure-in-merge-tree-parts} - -В таблицах [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) мы храним данные в частях данных, которые содержат всё на диске (локальном или удалённом). При этом данные на диске могут храниться иначе, чем в памяти. -В настоящее время в частях данных MergeTree используются три разных варианта сериализации общей структуры данных: `map`, `map_with_buckets` -и `advanced`. - -Версия сериализации управляется настройками MergeTree -[object_shared_data_serialization_version](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version) -и [object_shared_data_serialization_version_for_zero_level_parts](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version_for_zero_level_parts) -(часть нулевого уровня — это часть, создаваемая при вставке данных в таблицу; во время слияний части получают более высокий уровень). - -Примечание: изменение формата сериализации общей структуры данных поддерживается только -для `v3` [версии сериализации объектов](../../operations/settings/merge-tree-settings.md#object_serialization_version) - -#### Map {#shared-data-map} - -В версии сериализации `map` общие данные сериализуются как один столбец с типом `Map(String, String)` так же, как и в -памяти. Чтобы прочитать подстолбец пути из такого типа сериализации, ClickHouse читает целиком столбец `Map` и -извлекает требуемый путь в памяти. - -Эта сериализация эффективна для записи данных и чтения всего столбца `JSON`, но неэффективна для чтения подстолбцов путей. - -#### Map with buckets {#shared-data-map-with-buckets} - -В версии сериализации `map_with_buckets` общие данные сериализуются как `N` столбцов («buckets») с типом `Map(String, String)`. -Каждый такой бакет содержит только подмножество путей. Чтобы прочитать подстолбец пути из такого типа сериализации, ClickHouse -читает целиком столбец `Map` из одного бакета и извлекает требуемый путь в памяти. - -Эта сериализация менее эффективна для записи данных и чтения всего столбца `JSON`, но более эффективна для чтения подстолбцов путей, -поскольку считываются данные только из нужных бакетов. - -Количество бакетов `N` управляется настройками MergeTree [object_shared_data_buckets_for_compact_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_compact_part) (по умолчанию 8) -и [object_shared_data_buckets_for_wide_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_wide_part) (по умолчанию 32). - -#### Advanced {#shared-data-advanced} - -В версии сериализации `advanced` общие данные сериализуются в специальной структуре данных, которая максимально повышает производительность -чтения подстолбцов путей за счёт хранения дополнительной информации, позволяющей читать только данные запрошенных путей. -Эта сериализация также поддерживает бакеты, поэтому каждый бакет содержит только подмножество путей. - -Эта сериализация довольно неэффективна для записи данных (поэтому не рекомендуется использовать её для частей нулевого уровня), чтение всего столбца `JSON` немного менее эффективно по сравнению с сериализацией `map`, но она очень эффективна для чтения подстолбцов путей. - -Примечание: из-за хранения дополнительной информации внутри структуры данных размер занимаемого дискового пространства при этой сериализации больше по сравнению с -сериализациями `map` и `map_with_buckets`. - -Более подробный обзор новых сериализаций общей структуры данных и деталей реализации см. в [публикации в блоге](https://clickhouse.com/blog/json-data-type-gets-even-better). - - - -## Функции интроспекции {#introspection-functions} - -Существует несколько функций, которые помогают исследовать содержимое столбца JSON: - -* [`JSONAllPaths`](../functions/json-functions.md#JSONAllPaths) -* [`JSONAllPathsWithTypes`](../functions/json-functions.md#JSONAllPathsWithTypes) -* [`JSONDynamicPaths`](../functions/json-functions.md#JSONDynamicPaths) -* [`JSONDynamicPathsWithTypes`](../functions/json-functions.md#JSONDynamicPathsWithTypes) -* [`JSONSharedDataPaths`](../functions/json-functions.md#JSONSharedDataPaths) -* [`JSONSharedDataPathsWithTypes`](../functions/json-functions.md#JSONSharedDataPathsWithTypes) -* [`distinctDynamicTypes`](../aggregate-functions/reference/distinctdynamictypes.md) -* [`distinctJSONPaths and distinctJSONPathsAndTypes`](../aggregate-functions/reference/distinctjsonpaths.md) - -**Примеры** - -Давайте исследуем содержимое набора данных [GH Archive](https://www.gharchive.org/) за `2020-01-01`: - -```sql title="Query" -SELECT arrayJoin(distinctJSONPaths(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -``` - -```text title="Response" -┌─arrayJoin(distinctJSONPaths(json))─────────────────────────┐ -│ actor.avatar_url │ -│ actor.display_login │ -│ actor.gravatar_id │ -│ actor.id │ -│ actor.login │ -│ actor.url │ -│ created_at │ -│ id │ -│ org.avatar_url │ -│ org.gravatar_id │ -│ org.id │ -│ org.login │ -│ org.url │ -│ payload.action │ -│ payload.before │ -│ payload.comment._links.html.href │ -│ payload.comment._links.pull_request.href │ -│ payload.comment._links.self.href │ -│ payload.comment.author_association │ -│ payload.comment.body │ -│ payload.comment.commit_id │ -│ payload.comment.created_at │ -│ payload.comment.diff_hunk │ -│ payload.comment.html_url │ -│ payload.comment.id │ -│ payload.comment.in_reply_to_id │ -│ payload.comment.issue_url │ -│ payload.comment.line │ -│ payload.comment.node_id │ -│ payload.comment.original_commit_id │ -│ payload.comment.original_position │ -│ payload.comment.path │ -│ payload.comment.position │ -│ payload.comment.pull_request_review_id │ -... -│ payload.release.node_id │ -│ payload.release.prerelease │ -│ payload.release.published_at │ -│ payload.release.tag_name │ -│ payload.release.tarball_url │ -│ payload.release.target_commitish │ -│ payload.release.upload_url │ -│ payload.release.url │ -│ payload.release.zipball_url │ -│ payload.size │ -│ public │ -│ repo.id │ -│ repo.name │ -│ repo.url │ -│ type │ -└─arrayJoin(distinctJSONPaths(json))─────────────────────────┘ -``` - -```sql -SELECT arrayJoin(distinctJSONPathsAndTypes(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -SETTINGS date_time_input_format = 'best_effort' -``` - - -```text -┌─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┐ -│ ('actor.avatar_url',['String']) │ -│ ('actor.display_login',['String']) │ -│ ('actor.gravatar_id',['String']) │ -│ ('actor.id',['Int64']) │ -│ ('actor.login',['String']) │ -│ ('actor.url',['String']) │ -│ ('created_at',['DateTime']) │ -│ ('id',['String']) │ -│ ('org.avatar_url',['String']) │ -│ ('org.gravatar_id',['String']) │ -│ ('org.id',['Int64']) │ -│ ('org.login',['String']) │ -│ ('org.url',['String']) │ -│ ('payload.action',['String']) │ -│ ('payload.before',['String']) │ -│ ('payload.comment._links.html.href',['String']) │ -│ ('payload.comment._links.pull_request.href',['String']) │ -│ ('payload.comment._links.self.href',['String']) │ -│ ('payload.comment.author_association',['String']) │ -│ ('payload.comment.body',['String']) │ -│ ('payload.comment.commit_id',['String']) │ -│ ('payload.comment.created_at',['DateTime']) │ -│ ('payload.comment.diff_hunk',['String']) │ -│ ('payload.comment.html_url',['String']) │ -│ ('payload.comment.id',['Int64']) │ -│ ('payload.comment.in_reply_to_id',['Int64']) │ -│ ('payload.comment.issue_url',['String']) │ -│ ('payload.comment.line',['Int64']) │ -│ ('payload.comment.node_id',['String']) │ -│ ('payload.comment.original_commit_id',['String']) │ -│ ('payload.comment.original_position',['Int64']) │ -│ ('payload.comment.path',['String']) │ -│ ('payload.comment.position',['Int64']) │ -│ ('payload.comment.pull_request_review_id',['Int64']) │ -... -│ ('payload.release.node_id',['String']) │ -│ ('payload.release.prerelease',['Bool']) │ -│ ('payload.release.published_at',['DateTime']) │ -│ ('payload.release.tag_name',['String']) │ -│ ('payload.release.tarball_url',['String']) │ -│ ('payload.release.target_commitish',['String']) │ -│ ('payload.release.upload_url',['String']) │ -│ ('payload.release.url',['String']) │ -│ ('payload.release.zipball_url',['String']) │ -│ ('payload.size',['Int64']) │ -│ ('public',['Bool']) │ -│ ('repo.id',['Int64']) │ -│ ('repo.name',['String']) │ -│ ('repo.url',['String']) │ -│ ('type',['String']) │ -└─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┘ -``` - - -## ALTER MODIFY COLUMN к типу JSON {#alter-modify-column-to-json-type} - -Можно изменить существующую таблицу и сменить тип столбца на новый тип `JSON`. На данный момент поддерживается только `ALTER` для столбцов типа `String`. - -**Пример** - -```sql title="Query" -CREATE TABLE test (json String) ENGINE=MergeTree ORDER BY tuple(); -INSERT INTO test VALUES ('{"a" : 42}'), ('{"a" : 43, "b" : "Hello"}'), ('{"a" : 44, "b" : [1, 2, 3]}'), ('{"c" : "2020-01-01"}'); -ALTER TABLE test MODIFY COLUMN json JSON; -SELECT json, json.a, json.b, json.c FROM test; -``` - -```text title="Response" -┌─json─────────────────────────┬─json.a─┬─json.b──┬─json.c─────┐ -│ {"a":"42"} │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ {"a":"43","b":"Hello"} │ 43 │ Hello │ ᴺᵁᴸᴸ │ -│ {"a":"44","b":["1","2","3"]} │ 44 │ [1,2,3] │ ᴺᵁᴸᴸ │ -│ {"c":"2020-01-01"} │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ -└──────────────────────────────┴────────┴─────────┴────────────┘ -``` - - -## Сравнение значений типа JSON {#comparison-between-values-of-the-json-type} - -Объекты JSON сравниваются аналогично значениям типа `Map`. - -Например: - -```sql title="Query" -CREATE TABLE test (json1 JSON, json2 JSON) ENGINE=Memory; -INSERT INTO test FORMAT JSONEachRow -{"json1" : {}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : [1, 2, 3]}} -{"json1" : {"a" : 42}, "json2" : {"a" : "Hello"}} -{"json1" : {"a" : 42}, "json2" : {"b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42, "b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41, "b" : 42}} - -SELECT json1, json2, json1 < json2, json1 = json2, json1 > json2 FROM test; -``` - -```text title="Response" -┌─json1──────┬─json2───────────────┬─less(json1, json2)─┬─equals(json1, json2)─┬─greater(json1, json2)─┐ -│ {} │ {} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"41"} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"42"} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {"a":["1","2","3"]} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"Hello"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"42","b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"41","b":"42"} │ 0 │ 0 │ 1 │ -└────────────┴─────────────────────┴────────────────────┴──────────────────────┴───────────────────────┘ -``` - -**Примечание:** когда два пути содержат значения разных типов данных, они сравниваются в соответствии с [правилом сравнения](/sql-reference/data-types/variant#comparing-values-of-variant-data) типа данных `Variant`. - - -## Рекомендации по более эффективному использованию типа JSON {#tips-for-better-usage-of-the-json-type} - -Прежде чем создавать столбец `JSON` и загружать в него данные, учитывайте следующие рекомендации: - -- Проанализируйте свои данные и укажите как можно больше подсказок по путям с указанием типов. Это сделает хранение и чтение гораздо более эффективными. -- Продумайте, какие пути вам понадобятся, а какие — никогда. Укажите пути, которые вам не нужны, в разделе `SKIP`, а при необходимости — в разделе `SKIP REGEXP`. Это улучшит эффективность хранения. -- Не устанавливайте параметр `max_dynamic_paths` на слишком большие значения, так как это может сделать хранение и чтение менее эффективными. - Хотя это сильно зависит от системных параметров, таких как память, CPU и т.д., в качестве общего ориентира не следует устанавливать `max_dynamic_paths` более 10 000 для хранилища на локальной файловой системе и 1024 для хранилища на удалённой файловой системе. - - - -## Дополнительные материалы {#further-reading} - -- [Как мы разработали новый мощный тип данных JSON для ClickHouse](https://clickhouse.com/blog/a-new-powerful-json-data-type-for-clickhouse) -- [Испытание «миллиард JSON‑документов»: ClickHouse против MongoDB, Elasticsearch и других](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md deleted file mode 100644 index bebd9efd59b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Документация по модификатору типа данных Nullable в ClickHouse' -sidebar_label: 'Nullable(T)' -sidebar_position: 44 -slug: /sql-reference/data-types/nullable -title: 'Nullable(T)' -doc_type: 'reference' ---- - - - -# Nullable(T) {#nullablet} - -Позволяет хранить специальный маркер ([NULL](../../sql-reference/syntax.md)), обозначающий «отсутствующее значение», наряду с обычными значениями, допустимыми для `T`. Например, столбец типа `Nullable(Int8)` может хранить значения типа `Int8`, а для строк, в которых нет значения, будет храниться `NULL`. - -`T` не может быть ни одним из составных типов данных [Array](../../sql-reference/data-types/array.md), [Map](../../sql-reference/data-types/map.md) и [Tuple](../../sql-reference/data-types/tuple.md), но составные типы данных могут содержать значения типа `Nullable`, например `Array(Nullable(Int8))`. - -Поле типа `Nullable` не может участвовать в индексах таблицы. - -`NULL` является значением по умолчанию для любого типа `Nullable`, если иное не указано в конфигурации сервера ClickHouse. - - - -## Особенности хранения {#storage-features} - -Для хранения значений типа `Nullable` в столбце таблицы ClickHouse использует отдельный файл с `NULL`-масками в дополнение к обычному файлу со значениями. Записи в файле масок позволяют ClickHouse различать `NULL` и значение по умолчанию соответствующего типа данных для каждой строки таблицы. Из-за дополнительного файла столбец `Nullable` потребляет больше дискового пространства по сравнению с аналогичным обычным столбцом. - -:::note -Использование `Nullable` почти всегда негативно влияет на производительность, учитывайте это при проектировании своих баз данных. -::: - - - -## Поиск значений NULL {#finding-null} - -Можно найти значения `NULL` в столбце, используя подстолбец `null`, не считывая весь столбец. Он возвращает `1`, если соответствующее значение равно `NULL`, и `0` в противном случае. - -**Пример** - -Запрос: - -```sql -CREATE TABLE nullable (`n` Nullable(UInt32)) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO nullable VALUES (1) (NULL) (2) (NULL); - -SELECT n.null FROM nullable; -``` - -Результат: - -```text -┌─n.null─┐ -│ 0 │ -│ 1 │ -│ 0 │ -│ 1 │ -└────────┘ -``` - - -## Пример использования {#usage-example} - -```sql -CREATE TABLE t_null(x Int8, y Nullable(Int8)) ENGINE TinyLog -``` - -```sql -INSERT INTO t_null VALUES (1, NULL), (2, 3) -``` - -```sql -SELECT x + y FROM t_null -``` - -```text -┌─plus(x, y)─┐ -│ ᴺᵁᴸᴸ │ -│ 5 │ -└────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md deleted file mode 100644 index 69f007e1eb6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'Документация по типу данных QBit в ClickHouse, который позволяет выполнять тонкозернистую квантизацию для приблизительного векторного поиска' -keywords: ['qbit', 'тип данных'] -sidebar_label: 'QBit' -sidebar_position: 64 -slug: /sql-reference/data-types/qbit -title: 'Тип данных QBit' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - - -Тип данных `QBit` реорганизует хранение векторов для более быстрого приближённого поиска. Вместо того чтобы хранить элементы каждого вектора вместе, он группирует одинаковые позиции двоичных разрядов по всем векторам. -Это позволяет хранить векторы с полной точностью и выбирать степень детальной квантизации во время поиска: считывать меньше бит для уменьшения объёма I/O и ускорения вычислений или больше бит для повышения точности. Вы получаете преимущества по скорости за счёт сокращения передачи данных и объёма вычислений благодаря квантизации, при этом все исходные данные остаются доступными при необходимости. - -:::note -Тип данных `QBit` и связанные с ним функции расстояния в данный момент являются экспериментальными. -Чтобы их включить, сначала выполните `SET allow_experimental_qbit_type = 1`. -Если вы столкнётесь с проблемами, пожалуйста, создайте обращение (issue) в [репозитории ClickHouse](https://github.com/clickhouse/clickhouse/issues). -::: - -Чтобы объявить столбец типа `QBit`, используйте следующий синтаксис: - -```sql -column_name QBit(element_type, dimension) -``` - -* `element_type` – тип каждого элемента вектора. Допустимые типы: `BFloat16`, `Float32` и `Float64` -* `dimension` – число элементов в каждом векторе - - -## Создание QBit {#creating-qbit} - -Использование типа `QBit` в определении столбца таблицы: - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [1, 2, 3, 4, 5, 6, 7, 8]), (2, [9, 10, 11, 12, 13, 14, 15, 16]); -SELECT vec FROM test ORDER BY id; -``` - -```text -┌─vec──────────────────────┐ -│ [1,2,3,4,5,6,7,8] │ -│ [9,10,11,12,13,14,15,16] │ -└──────────────────────────┘ -``` - - -## Подстолбцы QBit {#qbit-subcolumns} - -`QBit` реализует механизм доступа к подстолбцам, который позволяет обращаться к отдельным битовым плоскостям сохраняемых векторов. Каждая позиция бита может быть доступна с помощью синтаксиса `.N`, где `N` — позиция бита: - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [0, 0, 0, 0, 0, 0, 0, 0]); -INSERT INTO test VALUES (1, [-0, -0, -0, -0, -0, -0, -0, -0]); -SELECT bin(vec.1) FROM test; -``` - -```text -┌─bin(tupleElement(vec, 1))─┐ -│ 00000000 │ -│ 11111111 │ -└───────────────────────────┘ -``` - -Количество доступных подстолбцов зависит от типа элемента: - -* `BFloat16`: 16 подстолбцов (1–16) -* `Float32`: 32 подстолбца (1–32) -* `Float64`: 64 подстолбца (1–64) - - -## Функции векторного поиска {#vector-search-functions} - -Это функции вычисления расстояния для поиска похожих векторов, которые используют тип данных `QBit`: - -* [`L2DistanceTransposed`](../functions/distance-functions.md#L2DistanceTransposed) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md deleted file mode 100644 index 1f4027bd18c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'Документация по типу данных SimpleAggregateFunction' -sidebar_label: 'SimpleAggregateFunction' -sidebar_position: 48 -slug: /sql-reference/data-types/simpleaggregatefunction -title: 'Тип данных SimpleAggregateFunction' -doc_type: 'reference' ---- - - - -# Тип SimpleAggregateFunction {#simpleaggregatefunction-type} - - - -## Описание {#description} - -Тип данных `SimpleAggregateFunction` хранит промежуточное состояние -агрегатной функции, но не её полное состояние, как это делает тип -[`AggregateFunction`](../../sql-reference/data-types/aggregatefunction.md). - -Эта оптимизация может быть применена к функциям, для которых выполняется -следующее свойство: - -> результат применения функции `f` к набору строк `S1 UNION ALL S2` может быть -получен путём раздельного применения `f` к частям набора строк, а затем -повторного применения `f` к результатам: `f(S1 UNION ALL S2) = f(f(S1) UNION ALL f(S2))`. - -Это свойство гарантирует, что частичных результатов агрегации достаточно для -вычисления объединённого результата, поэтому нам не нужно хранить и обрабатывать -избыточные данные. Например, результат функций `min` или `max` не требует -дополнительных шагов для вычисления окончательного результата из -промежуточных шагов, тогда как функция `avg` требует хранения суммы и -количества, которые затем делятся для получения среднего значения -на заключительном шаге `Merge`, объединяющем промежуточные состояния. - -Значения агрегатных функций обычно получаются путём вызова агрегатной функции -с комбинатором [`-SimpleState`](/sql-reference/aggregate-functions/combinators#-simplestate), добавленным к имени функции. - - - -## Синтаксис {#syntax} - -```sql -SimpleAggregateFunction(aggregate_function_name, types_of_arguments...) -``` - -**Параметры** - -* `aggregate_function_name` — имя агрегатной функции. -* `Type` — типы аргументов агрегатной функции. - - -## Поддерживаемые функции {#supported-functions} - -Поддерживаются следующие агрегатные функции: - -- [`any`](/sql-reference/aggregate-functions/reference/any) -- [`any_respect_nulls`](/sql-reference/aggregate-functions/reference/any) -- [`anyLast`](/sql-reference/aggregate-functions/reference/anylast) -- [`anyLast_respect_nulls`](/sql-reference/aggregate-functions/reference/anylast) -- [`min`](/sql-reference/aggregate-functions/reference/min) -- [`max`](/sql-reference/aggregate-functions/reference/max) -- [`sum`](/sql-reference/aggregate-functions/reference/sum) -- [`sumWithOverflow`](/sql-reference/aggregate-functions/reference/sumwithoverflow) -- [`groupBitAnd`](/sql-reference/aggregate-functions/reference/groupbitand) -- [`groupBitOr`](/sql-reference/aggregate-functions/reference/groupbitor) -- [`groupBitXor`](/sql-reference/aggregate-functions/reference/groupbitxor) -- [`groupArrayArray`](/sql-reference/aggregate-functions/reference/grouparray) -- [`groupUniqArrayArray`](../../sql-reference/aggregate-functions/reference/groupuniqarray.md) -- [`groupUniqArrayArrayMap`](../../sql-reference/aggregate-functions/combinators#-map) -- [`sumMap`](/sql-reference/aggregate-functions/reference/summap) -- [`minMap`](/sql-reference/aggregate-functions/reference/minmap) -- [`maxMap`](/sql-reference/aggregate-functions/reference/maxmap) - -:::note -Значения типа `SimpleAggregateFunction(func, Type)` имеют тот же тип `Type`, -поэтому в отличие от типа `AggregateFunction` нет необходимости применять -комбинаторы `-Merge`/`-State`. - -Тип `SimpleAggregateFunction` обеспечивает более высокую производительность, чем `AggregateFunction` -для одних и тех же агрегатных функций. -::: - - - -## Пример {#example} - - - -```sql -CREATE TABLE simple (id UInt64, val SimpleAggregateFunction(sum, Double)) ENGINE=AggregatingMergeTree ORDER BY id; -``` - -## Связанные материалы {#related-content} - -* Блог: [Использование агрегатных комбинаторов в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) - Блог: [Использование агрегатных комбинаторов в ClickHouse](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -* Тип данных [AggregateFunction](/sql-reference/data-types/aggregatefunction). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md deleted file mode 100644 index 7ef1e27b439..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Документация по специальному типу данных Expression' -sidebar_label: 'Expression' -sidebar_position: 58 -slug: /sql-reference/data-types/special-data-types/expression -title: 'Expression' -doc_type: 'reference' ---- - -# Выражение {#expression} - -Выражения используются для представления лямбда-выражений в функциях высшего порядка. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md deleted file mode 100644 index 40a79cb6c30..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: 'Обзор специальных типов данных в ClickHouse, которые используются для промежуточных - результатов при выполнении запросов' -sidebar_label: 'Специальные типы данных' -sidebar_position: 55 -slug: /sql-reference/data-types/special-data-types/ -title: 'Специальные типы данных' -doc_type: 'reference' ---- - -# Особые типы данных {#special-data-types} - -Значения особых типов данных не могут быть сериализованы для сохранения в таблице или вывода в результатах запроса, но могут использоваться как промежуточные результаты во время выполнения запроса. \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md deleted file mode 100644 index 7e306fc870c..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: 'Документация по специальному типу данных Interval' -sidebar_label: 'Interval' -sidebar_position: 61 -slug: /sql-reference/data-types/special-data-types/interval -title: 'Interval' -doc_type: 'reference' ---- - - - -# Interval {#interval} - -Семейство типов данных для представления временных и календарных интервалов. Типы, возвращаемые оператором [INTERVAL](/sql-reference/operators#interval). - -Структура: - -* Интервал в виде беззнакового целого числа. -* Тип интервала. - -Поддерживаемые типы интервалов: - -* `NANOSECOND` -* `MICROSECOND` -* `MILLISECOND` -* `SECOND` -* `MINUTE` -* `HOUR` -* `DAY` -* `WEEK` -* `MONTH` -* `QUARTER` -* `YEAR` - -Для каждого типа интервала существует отдельный тип данных. Например, интервал `DAY` соответствует типу данных `IntervalDay`: - -```sql -SELECT toTypeName(INTERVAL 4 DAY) -``` - -```text -┌─toTypeName(toIntervalDay(4))─┐ -│ IntervalDay │ -└──────────────────────────────┘ -``` - - -## Примечания по использованию {#usage-remarks} - -Вы можете использовать значения типа `Interval` в арифметических операциях над значениями типов [Date](../../../sql-reference/data-types/date.md) и [DateTime](../../../sql-reference/data-types/datetime.md). Например, вы можете прибавить 4 дня к текущему времени: - -```sql -SELECT now() AS текущая_дата_время, текущая_дата_время + INTERVAL 4 DAY -``` - -```text -┌───current_date_time─┬─plus(now(), toIntervalDay(4))─┐ -│ 2019-10-23 10:58:45 │ 2019-10-27 10:58:45 │ -└─────────────────────┴───────────────────────────────┘ -``` - -Также можно одновременно использовать несколько интервалов: - -```sql -SELECT now() AS current_date_time, current_date_time + (INTERVAL 4 DAY + INTERVAL 3 HOUR) -``` - -```text -┌───current_date_time─┬─plus(current_date_time, plus(toIntervalDay(4), toIntervalHour(3)))─┐ -│ 2024-08-08 18:31:39 │ 2024-08-12 21:31:39 │ -└─────────────────────┴────────────────────────────────────────────────────────────────────┘ -``` - -А чтобы сравнить значения на разных интервалах: - -```sql -SELECT toIntervalMicrosecond(3600000000) = toIntervalHour(1); -``` - -```text -┌─less(toIntervalMicrosecond(179999999), toIntervalMinute(3))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────────┘ -``` - - -## См. также {#see-also} - -- [INTERVAL](/sql-reference/operators#interval) — оператор -- [toInterval](/sql-reference/functions/type-conversion-functions#tointervalyear) — функции преобразования типов diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md deleted file mode 100644 index 021638b3827..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -description: 'Документация по специальному типу данных «Nothing»' -sidebar_label: 'Nothing' -sidebar_position: 60 -slug: /sql-reference/data-types/special-data-types/nothing -title: 'Nothing' -doc_type: 'reference' ---- - -# Nothing {#nothing} - -Единственное назначение этого типа данных — служить для представления случаев, когда значение не предполагается. Поэтому вы не можете создать значение типа `Nothing`. - -Например, литерал [NULL](/sql-reference/syntax#null) имеет тип `Nullable(Nothing)`. Подробнее см. в разделе [Nullable](../../../sql-reference/data-types/nullable.md). - -Тип `Nothing` также может использоваться для обозначения пустых массивов: - -```sql -SELECT toTypeName(array()) -``` - -```text -┌─toTypeName(array())─┐ -│ Array(Nothing) │ -└─────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md deleted file mode 100644 index 21329e90872..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Документация по специальному типу данных Set, который используется в выражениях IN' -sidebar_label: 'Set' -sidebar_position: 59 -slug: /sql-reference/data-types/special-data-types/set -title: 'Set' -doc_type: 'reference' ---- - -# Set {#set} - -Используется для правой части выражения с оператором [IN](/sql-reference/operators/in). \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md deleted file mode 100644 index ee551a4265d..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'Документация о типе данных String в ClickHouse' -sidebar_label: 'String' -sidebar_position: 8 -slug: /sql-reference/data-types/string -title: 'String' -doc_type: 'reference' ---- - - - -# String {#string} - -Строки произвольной длины. Длина не ограничена. Значение может содержать произвольный набор байт, включая байты нулевого значения. -Тип String заменяет типы VARCHAR, BLOB, CLOB и другие типы из других СУБД. - -При создании таблиц для строковых полей можно задавать числовые параметры длины (например, `VARCHAR(255)`), но ClickHouse их игнорирует. - -Синонимы: - -- `String` — `LONGTEXT`, `MEDIUMTEXT`, `TINYTEXT`, `TEXT`, `LONGBLOB`, `MEDIUMBLOB`, `TINYBLOB`, `BLOB`, `VARCHAR`, `CHAR`, `CHAR LARGE OBJECT`, `CHAR VARYING`, `CHARACTER LARGE OBJECT`, `CHARACTER VARYING`, `NCHAR LARGE OBJECT`, `NCHAR VARYING`, `NATIONAL CHARACTER LARGE OBJECT`, `NATIONAL CHARACTER VARYING`, `NATIONAL CHAR VARYING`, `NATIONAL CHARACTER`, `NATIONAL CHAR`, `BINARY LARGE OBJECT`, `BINARY VARYING`, - - - -## Кодировки {#encodings} - -В ClickHouse нет понятия кодировок. Строки могут содержать произвольный набор байтов, которые сохраняются и выводятся как есть. -Если вам нужно хранить текст, мы рекомендуем использовать кодировку UTF-8. Как минимум, если ваш терминал использует UTF-8 (как и рекомендуется), вы сможете читать и записывать значения без выполнения преобразований. -Аналогично, некоторые функции для работы со строками имеют отдельные варианты, рассчитанные на то, что строка содержит набор байтов, представляющих текст в кодировке UTF-8. -Например, функция [length](/sql-reference/functions/array-functions#length) вычисляет длину строки в байтах, тогда как функция [lengthUTF8](../functions/string-functions.md#lengthUTF8) вычисляет длину строки в кодовых точках Unicode, предполагая, что значение закодировано в UTF-8. diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md deleted file mode 100644 index 54113b18e87..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: 'Документация по типу данных Time в ClickHouse, который представляет диапазон времени с точностью до секунды' -slug: /sql-reference/data-types/time -sidebar_position: 15 -sidebar_label: 'Time' -title: 'Time' -doc_type: 'reference' ---- - - - -# Time {#time} - -Тип данных `Time` представляет время с компонентами часов, минут и секунд. -Он не зависит от какой-либо календарной даты и подходит для значений, которым не нужны компоненты дня, месяца и года. - -Синтаксис: - -```sql -Время -``` - -Диапазон текстового представления: [-999:59:59, 999:59:59]. - -Точность: 1 секунда. - - -## Подробности реализации {#implementation-details} - -**Представление и производительность**. -Тип данных `Time` внутренне представляет собой знаковое 32-битное целое число, кодирующее количество секунд. -Значения типов `Time` и `DateTime` имеют одинаковый размер в байтах и, следовательно, сопоставимую производительность. - -**Нормализация**. -При разборе строк в значение типа `Time` компоненты времени нормализуются, но не проходят проверку корректности. -Например, `25:70:70` интерпретируется как `26:11:10`. - -**Отрицательные значения**. -Лидирующие знаки минус поддерживаются и сохраняются. -Отрицательные значения обычно возникают при выполнении арифметических операций над значениями `Time`. -Для типа `Time` отрицательные входные данные сохраняются как для текстовых (например, `'-01:02:03'`), так и для числовых значений (например, `-3723`). - -**Ограничение значений (saturation)**. -Компонента времени суток ограничивается диапазоном [-999:59:59, 999:59:59]. -Значения с количеством часов больше 999 (или меньше -999) в текстовом представлении и при обратном разборе передаются как `999:59:59` (или `-999:59:59`). - -**Часовые пояса**. -`Time` не поддерживает часовые пояса, то есть значения `Time` интерпретируются без регионального контекста. -Указание часового пояса для `Time` как параметра типа или при создании значения приводит к ошибке. -Аналогично, попытки применить или изменить часовой пояс для столбцов типа `Time` не поддерживаются и приводят к ошибке. -Значения `Time` не переинтерпретируются автоматически при смене часового пояса. - - - -## Примеры {#examples} - -**1.** Создание таблицы со столбцом типа `Time` и запись данных в неё: - -```sql -CREATE TABLE tab -( - `event_id` UInt8, - `time` Time -) -ENGINE = TinyLog; -``` - -```sql --- Разбор Time --- - из строки, --- - из целого числа, интерпретируемого как количество секунд с 00:00:00. -INSERT INTO tab VALUES (1, '14:30:25'), (2, 52225); - -SELECT * FROM tab ORDER BY event_id; -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**2.** Фильтрация по значениям поля `Time` - -```sql -SELECT * FROM tab WHERE time = toTime('14:30:25') -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -Значения столбца `Time` можно фильтровать по строковому значению в предикате `WHERE`. Оно будет автоматически приведено к типу `Time`: - -```sql -SELECT * FROM tab WHERE time = '14:30:25' -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**3.** Проверка полученного типа: - -```sql -SELECT CAST('14:30:25' AS Time) AS column, toTypeName(column) AS type -``` - -```text - ┌────column─┬─type─┐ -1. │ 14:30:25 │ Time │ - └───────────┴──────┘ -``` - - -## См. также {#see-also} - -- [Функции преобразования типов](../functions/type-conversion-functions.md) -- [Функции для работы с датами и временем](../functions/date-time-functions.md) -- [Функции для работы с массивами](../functions/array-functions.md) -- [Настройка `date_time_input_format`](../../operations/settings/settings-formats.md#date_time_input_format) -- [Настройка `date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) -- [Параметр конфигурации сервера `timezone`](../../operations/server-configuration-parameters/settings.md#timezone) -- [Настройка `session_timezone`](../../operations/settings/settings.md#session_timezone) -- [Тип данных `DateTime`](datetime.md) -- [Тип данных `Date`](date.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md deleted file mode 100644 index a0601a0b897..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'Документация по типу данных Time64 в ClickHouse, предназначенному для хранения диапазона времени с субсекундной точностью' -slug: /sql-reference/data-types/time64 -sidebar_position: 17 -sidebar_label: 'Time64' -title: 'Time64' -doc_type: 'reference' ---- - - - -# Time64 {#time64} - -Тип данных `Time64` представляет время суток с дробными секундами. -Он не содержит компонентов календарной даты (день, месяц, год). -Параметр `precision` определяет количество знаков после запятой и, соответственно, размер тика. - -Размер тика (precision): 10-precision секунды. Допустимый диапазон: 0..9. Наиболее часто используются значения: 3 (миллисекунды), 6 (микросекунды) и 9 (наносекунды). - -**Синтаксис:** - -```sql -Time64(precision) -``` - -Внутренне тип `Time64` хранит знаковое 64-битное десятичное число (Decimal64), представляющее дробные секунды. -Разрешение тиков определяется параметром `precision`. -Часовые пояса не поддерживаются: указание часового пояса для `Time64` приведёт к ошибке. - -В отличие от `DateTime64`, `Time64` не хранит дату. -См. также [`Time`](../../sql-reference/data-types/time.md). - -Диапазон текстового представления значений: [-999:59:59.000, 999:59:59.999] для `precision = 3`. В общем случае минимум — `-999:59:59`, максимум — `999:59:59` с количеством дробных цифр до значения `precision` (для `precision = 9` минимум — `-999:59:59.999999999`). - - -## Подробности реализации {#implementation-details} - -**Представление**. -Знаковое значение типа `Decimal64`, представляющее количество долей секунды с `precision` знаками после запятой. - -**Нормализация**. -При разборе строк в значение типа `Time64` компоненты времени нормализуются, но не проверяются на корректность. -Например, `25:70:70` интерпретируется как `26:11:10`. - -**Отрицательные значения**. -Ведущий знак минус поддерживается и сохраняется. -Отрицательные значения обычно возникают в результате арифметических операций над значениями `Time64`. -Для `Time64` отрицательные входные значения сохраняются как для текстовых (например, `'-01:02:03.123'`), так и для числовых входных данных (например, `-3723.123`). - -**Сатурация (ограничение диапазона)**. -Компонента времени суток ограничивается диапазоном [-999:59:59.xxx, 999:59:59.xxx] при преобразовании в компоненты или сериализации в текст. -Сохраняемое числовое значение может выходить за этот диапазон; однако любое извлечение компонентов (часы, минуты, секунды) и текстовое представление используют ограниченное (сатурированное) значение. - -**Часовые пояса**. -`Time64` не поддерживает часовые пояса. -Указание часового пояса при создании типа или значения `Time64` приводит к ошибке. -Аналогично, попытки применить или изменить часовой пояс для столбцов `Time64` не поддерживаются и приводят к ошибке. - - - -## Примеры {#examples} - -1. Создание таблицы со столбцом типа `Time64` и вставка в неё данных: - -```sql -CREATE TABLE tab64 -( - `event_id` UInt8, - `time` Time64(3) -) -ENGINE = TinyLog; -``` - -```sql --- Разбор Time64 --- - из строки, --- - из количества секунд с 00:00:00 (дробная часть согласно заданной точности). -INSERT INTO tab64 VALUES (1, '14:30:25'), (2, 52225.123), (3, '14:30:25'); - -SELECT * FROM tab64 ORDER BY event_id; -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 2 │ 14:30:25.123 │ -3. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -2. Фильтрация по значениям типа `Time64` - -```sql -SELECT * FROM tab64 WHERE time = toTime64('14:30:25', 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -```sql -SELECT * FROM tab64 WHERE time = toTime64(52225.123, 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 2 │ 14:30:25.123 │ - └──────────┴──────────────┘ -``` - -Примечание: `toTime64` разбирает числовые литералы как секунды с дробной частью в соответствии с указанной точностью, поэтому явно указывайте требуемое количество знаков после запятой. - -3. Проверка получившегося типа: - -```sql -SELECT CAST('14:30:25.250' AS Time64(3)) AS column, toTypeName(column) AS type; -``` - -```text - ┌────────column─┬─type──────┐ -1. │ 14:30:25.250 │ Time64(3) │ - └───────────────┴───────────┘ -``` - -**См. также** - -* [Функции преобразования типов](../../sql-reference/functions/type-conversion-functions.md) -* [Функции для работы с датами и временем](../../sql-reference/functions/date-time-functions.md) -* [Настройка `date_time_input_format`](../../operations/settings/settings-formats.md#date_time_input_format) -* [Настройка `date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) -* [Параметр конфигурации сервера `timezone`](../../operations/server-configuration-parameters/settings.md#timezone) -* [Настройка `session_timezone`](../../operations/settings/settings.md#session_timezone) -* [Операторы для работы с датами и временем](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [Тип данных `Date`](../../sql-reference/data-types/date.md) -* [Тип данных `Time`](../../sql-reference/data-types/time.md) -* [Тип данных `DateTime`](../../sql-reference/data-types/datetime.md) diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md deleted file mode 100644 index deceb60c009..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -description: 'Документация по типу данных Tuple в ClickHouse' -sidebar_label: 'Tuple(T1, T2, ...)' -sidebar_position: 34 -slug: /sql-reference/data-types/tuple -title: 'Tuple(T1, T2, ...)' -doc_type: 'reference' ---- - -# Tuple(T1, T2, ...) {#tuplet1-t2} - -Кортеж элементов, каждый из которых имеет собственный [тип](/sql-reference/data-types). Кортеж должен содержать как минимум один элемент. - -Кортежи используются для временной группировки столбцов. Столбцы могут быть сгруппированы при использовании выражения IN в запросе, а также при указании некоторых формальных параметров лямбда-функций. Дополнительную информацию см. в разделах [Операторы IN](../../sql-reference/operators/in.md) и [Функции высшего порядка](/sql-reference/functions/overview#higher-order-functions). - -Кортежи могут быть результатом запроса. В этом случае в текстовых форматах, отличных от JSON, значения перечисляются через запятую в скобках. В форматах JSON кортежи выводятся как массивы (в квадратных скобках). - -## Создание кортежей {#creating-tuples} - -Вы можете использовать функцию для создания кортежа: - -```sql -tuple(T1, T2, ...) -``` - -Пример создания tuple: - -```sql -SELECT tuple(1, 'a') AS x, toTypeName(x) -``` - -```text -┌─x───────┬─toTypeName(tuple(1, 'a'))─┐ -│ (1,'a') │ Tuple(UInt8, String) │ -└─────────┴───────────────────────────┘ -``` - -Кортеж может содержать один элемент - -Пример: - -```sql -SELECT tuple('a') AS x; -``` - -```text -┌─x─────┐ -│ ('a') │ -└───────┘ -``` - -Синтаксис `(tuple_element1, tuple_element2)` можно использовать для создания кортежа из нескольких элементов без вызова функции `tuple()`. - -Пример: - -```sql -SELECT (1, 'a') AS x, (today(), rand(), 'someString') AS y, ('a') AS not_a_tuple; -``` - -```text -┌─x───────┬─y──────────────────────────────────────┬─not_a_tuple─┐ -│ (1,'a') │ ('2022-09-21',2006973416,'someString') │ a │ -└─────────┴────────────────────────────────────────┴─────────────┘ -``` - -## Определение типов данных {#data-type-detection} - -При создании кортежей на лету ClickHouse определяет тип аргументов кортежа как наименьший возможный тип, который может вместить переданное значение аргумента. Если значение — [NULL](/operations/settings/formats#input_format_null_as_default), определённый тип — [Nullable](../../sql-reference/data-types/nullable.md). - -Пример автоматического определения типа данных: - -```sql -SELECT tuple(1, NULL) AS x, toTypeName(x) -``` - -```text -┌─x─────────┬─toTypeName(tuple(1, NULL))──────┐ -│ (1, NULL) │ Tuple(UInt8, Nullable(Nothing)) │ -└───────────┴─────────────────────────────────┘ -``` - -## Обращение к элементам кортежа (Tuple) {#referring-to-tuple-elements} - -К элементам кортежа (Tuple) можно обращаться по имени или по индексу: - -```sql -CREATE TABLE named_tuples (`a` Tuple(s String, i Int64)) ENGINE = Memory; -INSERT INTO named_tuples VALUES (('y', 10)), (('x',-10)); - -SELECT a.s FROM named_tuples; -- по имени -SELECT a.2 FROM named_tuples; -- по индексу -``` - -Результат: - -```text -┌─a.s─┐ -│ y │ -│ x │ -└─────┘ - -┌─tupleElement(a, 2)─┐ -│ 10 │ -│ -10 │ -└────────────────────┘ -``` - -## Операции сравнения для Tuple {#comparison-operations-with-tuple} - -Два кортежа сравниваются последовательным сравнением их элементов слева направо. Если первый элемент кортежа больше (меньше) соответствующего элемента второго кортежа, то первый кортеж больше (меньше) второго; иначе (если оба элемента равны) сравнивается следующий элемент. - -Пример: - -```sql -SELECT (1, 'z') > (1, 'a') c1, (2022, 01, 02) > (2023, 04, 02) c2, (1,2,3) = (3,2,1) c3; -``` - -```text -┌─c1─┬─c2─┬─c3─┐ -│ 1 │ 0 │ 0 │ -└────┴────┴────┘ -``` - -Практические примеры: - -```sql -CREATE TABLE test -( - `year` Int16, - `month` Int8, - `day` Int8 -) -ENGINE = Memory AS -SELECT * -FROM values((2022, 12, 31), (2000, 1, 1)); - -SELECT * FROM test; - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -│ 2000 │ 1 │ 1 │ -└──────┴───────┴─────┘ - -SELECT * -FROM test -WHERE (year, month, day) > (2010, 1, 1); - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -└──────┴───────┴─────┘ -CREATE TABLE test -( - `key` Int64, - `duration` UInt32, - `value` Float64 -) -ENGINE = Memory AS -SELECT * -FROM values((1, 42, 66.5), (1, 42, 70), (2, 1, 10), (2, 2, 0)); - -SELECT * FROM test; - -┌─key─┬─duration─┬─value─┐ -│ 1 │ 42 │ 66.5 │ -│ 1 │ 42 │ 70 │ -│ 2 │ 1 │ 10 │ -│ 2 │ 2 │ 0 │ -└─────┴──────────┴───────┘ - --- Найдём значение для каждого ключа с максимальной длительностью; если длительности равны, выберем максимальное значение - -SELECT - key, - max(duration), - argMax(value, (duration, value)) -FROM test -GROUP BY key -ORDER BY key ASC; - -┌─key─┬─max(duration)─┬─argMax(value, tuple(duration, value))─┐ -│ 1 │ 42 │ 70 │ -│ 2 │ 2 │ 0 │ -└─────┴───────────────┴───────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md deleted file mode 100644 index e589dca510b..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'Документация о типе данных UUID в ClickHouse' -sidebar_label: 'UUID' -sidebar_position: 24 -slug: /sql-reference/data-types/uuid -title: 'UUID' -doc_type: 'reference' ---- - - - -# UUID {#uuid} - -Универсальный уникальный идентификатор (UUID) — это 16-байтовое значение, используемое для идентификации записей. Подробную информацию о UUID см. в статье на [Википедии](https://en.wikipedia.org/wiki/Universally_unique_identifier). - -Хотя существуют разные варианты UUID (см. [здесь](https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis)), ClickHouse не проверяет, соответствуют ли вставленные значения UUID какому-либо конкретному варианту. -Внутри ClickHouse UUID рассматриваются как последовательность из 16 случайных байт с [представлением 8-4-4-4-12](https://en.wikipedia.org/wiki/Universally_unique_identifier#Textual_representation) на уровне SQL. - -Пример значения UUID: - -```text -61f0c404-5cb3-11e7-907b-a6006ad3dba0 -``` - -UUID по умолчанию состоит из одних нулей. Он используется, например, когда вставляется новая запись, но для столбца с типом UUID не задано значение: - -```text -00000000-0000-0000-0000-000000000000 -``` - -По историческим причинам UUID сортируются по своей второй половине. -Поэтому UUID не следует использовать напрямую в качестве первичного, сортировочного или партиционного ключа таблицы. - -Пример: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY uuid; -``` - -Результат: - -```text -┌─uuid─────────────────────────────────┐ -│ 36a0b67c-b74a-4640-803b-e44bb4547e3c │ -│ 3a00aeb8-2605-4eec-8215-08c0ecb51112 │ -│ 3fda7c49-282e-421a-85ab-c5684ef1d350 │ -│ 16ab55a7-45f6-44a8-873c-7a0b44346b3e │ -│ e3776711-6359-4f22-878d-bf290d052c85 │ -│ [...] │ -│ 9eceda2f-6946-40e3-b725-16f2709ca41a │ -│ 03644f74-47ba-4020-b865-be5fd4c8c7ff │ -│ ce3bc93d-ab19-4c74-b8cc-737cb9212099 │ -│ b7ad6c91-23d6-4b5e-b8e4-a52297490b56 │ -│ 06892f64-cc2d-45f3-bf86-f5c5af5768a9 │ -└──────────────────────────────────────┘ -``` - -В качестве обходного решения UUID можно преобразовать в тип с более интуитивным порядком сортировки. - -Пример с использованием преобразования в UInt128: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY toUInt128(uuid); -``` - -Результат: - -```sql -┌─uuid─────────────────────────────────┐ -│ 018b81cd-aca1-4e9c-9e56-a84a074dc1a8 │ -│ 02380033-c96a-438e-913f-a2c67e341def │ -│ 057cf435-7044-456a-893b-9183a4475cea │ -│ 0a3c1d4c-f57d-44cc-8567-60cb0c46f76e │ -│ 0c15bf1c-8633-4414-a084-7017eead9e41 │ -│ [...] │ -│ f808cf05-ea57-4e81-8add-29a195bde63d │ -│ f859fb5d-764b-4a33-81e6-9e4239dae083 │ -│ fb1b7e37-ab7b-421a-910b-80e60e2bf9eb │ -│ fc3174ff-517b-49b5-bfe2-9b369a5c506d │ -│ fece9bf6-3832-449a-b058-cd1d70a02c8b │ -└──────────────────────────────────────┘ -``` - - -## Генерация UUID {#generating-uuids} - -ClickHouse предоставляет функцию [generateUUIDv4](../../sql-reference/functions/uuid-functions.md) для генерации случайных UUID версии 4. - - - -## Пример использования {#usage-example} - -**Пример 1** - -Этот пример демонстрирует создание таблицы со столбцом UUID и вставку значения в таблицу. - -```sql -CREATE TABLE t_uuid (x UUID, y String) ENGINE=TinyLog - -INSERT INTO t_uuid SELECT generateUUIDv4(), 'Пример 1' - -SELECT * FROM t_uuid -``` - -Результат: - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ Пример 1 │ -└──────────────────────────────────────┴───────────┘ -``` - -**Пример 2** - -В этом примере при вставке записи значение для столбца UUID не указывается, то есть вставляется значение UUID по умолчанию: - -```sql -INSERT INTO t_uuid (y) VALUES ('Пример 2') - -SELECT * FROM t_uuid -``` - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ Пример 1 │ -│ 00000000-0000-0000-0000-000000000000 │ Пример 2 │ -└──────────────────────────────────────┴───────────┘ -``` - - -## Ограничения {#restrictions} - -Тип данных UUID поддерживает только те функции, которые также поддерживает тип данных [String](../../sql-reference/data-types/string.md) (например, [min](/sql-reference/aggregate-functions/reference/min), [max](/sql-reference/aggregate-functions/reference/max) и [count](/sql-reference/aggregate-functions/reference/count)). - -Тип данных UUID не поддерживает арифметические операции (например, [abs](/sql-reference/functions/arithmetic-functions#abs)) и агрегатные функции, такие как [sum](/sql-reference/aggregate-functions/reference/sum) и [avg](/sql-reference/aggregate-functions/reference/avg). diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md deleted file mode 100644 index 1fc1de0c030..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md +++ /dev/null @@ -1,491 +0,0 @@ ---- -description: 'Документация о типе данных Variant в ClickHouse' -sidebar_label: 'Variant(T1, T2, ...)' -sidebar_position: 40 -slug: /sql-reference/data-types/variant -title: 'Variant(T1, T2, ...)' -doc_type: 'reference' ---- - -# Variant(T1, T2, ...) {#variantt1-t2} - -Этот тип представляет собой объединение других типов данных. Тип `Variant(T1, T2, ..., TN)` означает, что каждая строка этого типа -имеет значение либо типа `T1`, либо `T2`, ... либо `TN`, либо не имеет значения (`NULL`). - -Порядок вложенных типов не имеет значения: Variant(T1, T2) = Variant(T2, T1). -Вложенными типами могут быть произвольные типы, за исключением типов Nullable(...), LowCardinality(Nullable(...)) и Variant(...). - -:::note -Не рекомендуется использовать похожие типы в качестве вариантов (например, разные числовые типы, такие как `Variant(UInt32, Int64)`, или разные типы дат, такие как `Variant(Date, DateTime)`), -поскольку работа со значениями таких типов может приводить к неоднозначности. По умолчанию создание такого типа `Variant` приведёт к исключению, но это поведение можно изменить с помощью настройки `allow_suspicious_variant_types`. -::: - -## Создание типа Variant {#creating-variant} - -Использование типа `Variant` в определении столбца таблицы: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v FROM test; -``` - -```text -┌─v─────────────┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ Hello, World! │ -│ [1,2,3] │ -└───────────────┘ -``` - -Использование CAST для обычных столбцов: - -```sql -SELECT toTypeName(variant) AS type_name, 'Привет, мир!'::Variant(UInt64, String, Array(UInt64)) as variant; -``` - -```text -┌─type_name──────────────────────────────┬─variant───────┐ -│ Variant(Array(UInt64), String, UInt64) │ Hello, World! │ -└────────────────────────────────────────┴───────────────┘ -``` - -Использование функций `if/multiIf`, когда аргументы не имеют общего типа (для этого должна быть включена настройка `use_variant_as_common_type`): - -```sql -SET use_variant_as_common_type = 1; -SELECT if(number % 2, number, range(number)) as variant FROM numbers(5); -``` - -```text -┌─variant───┐ -│ [] │ -│ 1 │ -│ [0,1] │ -│ 3 │ -│ [0,1,2,3] │ -└───────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 4) = 0, 42, (number % 4) = 1, [1, 2, 3], (number % 4) = 2, 'Hello, World!', NULL) AS variant FROM numbers(4); -``` - -```text -┌─variant───────┐ -│ 42 │ -│ [1,2,3] │ -│ Hello, World! │ -│ ᴺᵁᴸᴸ │ -└───────────────┘ -``` - -Использование функций `array`/`map`, если элементы массива или значения Map не имеют общего типа (для этого должен быть включён настройка `use_variant_as_common_type`): - -```sql -SET use_variant_as_common_type = 1; -SELECT array(range(number), number, 'str_' || toString(number)) as array_of_variants FROM numbers(3); -``` - -```text -┌─array_of_variants─┐ -│ [[],0,'str_0'] │ -│ [[0],1,'str_1'] │ -│ [[0,1],2,'str_2'] │ -└───────────────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT map('a', range(number), 'b', number, 'c', 'str_' || toString(number)) as map_of_variants FROM numbers(3); -``` - -```text -┌─map_of_variants───────────────┐ -│ {'a':[],'b':0,'c':'str_0'} │ -│ {'a':[0],'b':1,'c':'str_1'} │ -│ {'a':[0,1],'b':2,'c':'str_2'} │ -└───────────────────────────────┘ -``` - -## Чтение вложенных типов Variant как подколонок {#reading-variant-nested-types-as-subcolumns} - -Тип Variant поддерживает чтение отдельного вложенного типа из столбца Variant, используя имя типа как подколонку. -Таким образом, если у вас есть столбец `variant Variant(T1, T2, T3)`, вы можете прочитать подколонку типа `T2`, используя синтаксис `variant.T2`, -эта подколонка будет иметь тип `Nullable(T2)`, если `T2` может быть обёрнут в `Nullable`, и `T2` в противном случае. Эта подколонка будет -того же размера, что и исходный столбец `Variant`, и будет содержать значения `NULL` (или пустые значения, если `T2` не может быть обёрнут в `Nullable`) -во всех строках, в которых значение в исходном столбце `Variant` не имеет типа `T2`. - -Подколонки Variant также могут читаться с помощью функции `variantElement(variant_column, type_name)`. - -Примеры: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v, v.String, v.UInt64, v.`Array(UInt64)` FROM test; -``` - -```text -┌─v─────────────┬─v.String──────┬─v.UInt64─┬─v.Array(UInt64)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴───────────────┴──────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(v.String), toTypeName(v.UInt64), toTypeName(v.`Array(UInt64)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(v.String)─┬─toTypeName(v.UInt64)─┬─toTypeName(v.Array(UInt64))─┐ -│ Nullable(String) │ Nullable(UInt64) │ Array(UInt64) │ -└──────────────────────┴──────────────────────┴─────────────────────────────┘ -``` - -```sql -SELECT v, variantElement(v, 'String'), variantElement(v, 'UInt64'), variantElement(v, 'Array(UInt64)') FROM test; -``` - -```text -┌─v─────────────┬─variantElement(v, 'String')─┬─variantElement(v, 'UInt64')─┬─variantElement(v, 'Array(UInt64)')─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴─────────────────────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - -Чтобы узнать, какой вариант хранится в каждой строке, можно использовать функцию `variantType(variant_column)`. Она возвращает значение типа `Enum` с именем типа варианта для каждой строки (или `'None'`, если строка имеет значение `NULL`). - -Пример: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT variantType(v) FROM test; -``` - -```text -┌─variantType(v)─┐ -│ None │ -│ UInt64 │ -│ String │ -│ Array(UInt64) │ -└────────────────┘ -``` - -```sql -SELECT toTypeName(variantType(v)) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(variantType(v))──────────────────────────────────────────┐ -│ Enum8('None' = -1, 'Array(UInt64)' = 0, 'String' = 1, 'UInt64' = 2) │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -## Преобразование между столбцом Variant и другими столбцами {#conversion-between-a-variant-column-and-other-columns} - -Существует четыре возможных преобразования, которые можно выполнить для столбца типа `Variant`. - -### Преобразование столбца String в столбец Variant {#converting-a-string-column-to-a-variant-column} - -Преобразование из `String` в `Variant` выполняется путём парсинга значения типа `Variant` из строкового значения: - -```sql -SELECT '42'::Variant(String, UInt64) AS variant, variantType(variant) AS variant_type -``` - -```text -┌─variant─┬─variant_type─┐ -│ 42 │ UInt64 │ -└─────────┴──────────────┘ -``` - -```sql -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴───────────────┘ -``` - -````sql -SELECT CAST(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01'), 'Map(String, Variant(UInt64, Bool, Date))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types``` -```` - -```text -┌─map_of_variants─────────────────────────────┬─map_of_variant_types──────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'UInt64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴───────────────────────────────────────────────┘ -``` - -Чтобы отключить парсинг при преобразовании из `String` в `Variant`, вы можете отключить параметр `cast_string_to_dynamic_use_inference`: - -```sql -SET cast_string_to_variant_use_inference = 0; -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### Преобразование обычного столбца в столбец Variant {#converting-an-ordinary-column-to-a-variant-column} - -Можно преобразовать обычный столбец типа `T` в столбец `Variant`, содержащий этот тип: - -```sql -SELECT toTypeName(variant) AS type_name, [1,2,3]::Array(UInt64)::Variant(UInt64, String, Array(UInt64)) as variant, variantType(variant) as variant_name -``` - -```text -┌─type_name──────────────────────────────┬─variant─┬─variant_name──┐ -│ Variant(Array(UInt64), String, UInt64) │ [1,2,3] │ Array(UInt64) │ -└────────────────────────────────────────┴─────────┴───────────────┘ -``` - -Примечание: преобразование из типа `String` всегда выполняется через парсинг. Если вам нужно преобразовать столбец `String` в строковый (`String`) вариант типа `Variant` без парсинга, вы можете сделать следующее: - -```sql -SELECT '[1, 2, 3]'::Variant(String)::Variant(String, Array(UInt64), UInt64) as variant, variantType(variant) as variant_type -``` - -```sql -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### Преобразование столбца Variant в обычный столбец {#converting-a-variant-column-to-an-ordinary-column} - -Можно преобразовать столбец `Variant` в обычный столбец. В этом случае все вложенные значения будут преобразованы в целевой тип: - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'); -SELECT v::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(v, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -└──────────────────────────────┘ -``` - -### Преобразование одного значения Variant в другое {#converting-a-variant-to-another-variant} - -Можно преобразовать столбец `Variant` в другой столбец `Variant`, но только если целевой столбец `Variant` содержит все вложенные типы исходного `Variant`: - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'); -SELECT v::Variant(UInt64, String, Array(UInt64)) FROM test; -``` - -```text -┌─CAST(v, 'Variant(UInt64, String, Array(UInt64))')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ String │ -└───────────────────────────────────────────────────┘ -``` - -## Чтение типа Variant из данных {#reading-variant-type-from-the-data} - -Все текстовые форматы (TSV, CSV, CustomSeparated, Values, JSONEachRow и т. д.) поддерживают чтение значений типа `Variant`. Во время разбора данных ClickHouse пытается поместить значение в наиболее подходящий вариант. - -Пример: - -```sql -SELECT - v, - variantElement(v, 'String') AS str, - variantElement(v, 'UInt64') AS num, - variantElement(v, 'Float64') AS float, - variantElement(v, 'DateTime') AS date, - variantElement(v, 'Array(UInt64)') AS arr -FROM format(JSONEachRow, 'v Variant(String, UInt64, Float64, DateTime, Array(UInt64))', $$ -{"v" : "Hello, World!"}, -{"v" : 42}, -{"v" : 42.42}, -{"v" : "2020-01-01 00:00:00"}, -{"v" : [1, 2, 3]} -$$) -``` - -```text -┌─v───────────────────┬─str───────────┬──num─┬─float─┬────────────────date─┬─arr─────┐ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 00:00:00 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 00:00:00 │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└─────────────────────┴───────────────┴──────┴───────┴─────────────────────┴─────────┘ -``` - -## Сравнение значений типа Variant {#comparing-values-of-variant-data} - -Значения типа `Variant` можно сравнивать только со значениями того же типа `Variant`. - -Результат оператора `<` для значений `v1` с базовым типом `T1` и `v2` с базовым типом `T2` типа `Variant(..., T1, ... T2, ...)` определяется следующим образом: - -* Если `T1 = T2 = T`, результатом будет `v1.T < v2.T` (сравниваются базовые значения). -* Если `T1 != T2`, результатом будет `T1 < T2` (сравниваются имена типов). - -Примеры: - -```sql -CREATE TABLE test (v1 Variant(String, UInt64, Array(UInt32)), v2 Variant(String, UInt64, Array(UInt32))) ENGINE=Memory; -INSERT INTO test VALUES (42, 42), (42, 43), (42, 'abc'), (42, [1, 2, 3]), (42, []), (42, NULL); -``` - -```sql -SELECT v2, variantType(v2) AS v2_type FROM test ORDER BY v2; -``` - -```text -┌─v2──────┬─v2_type───────┐ -│ [] │ Array(UInt32) │ -│ [1,2,3] │ Array(UInt32) │ -│ abc │ String │ -│ 42 │ UInt64 │ -│ 43 │ UInt64 │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴───────────────┘ -``` - -```sql -SELECT v1, variantType(v1) AS v1_type, v2, variantType(v2) AS v2_type, v1 = v2, v1 < v2, v1 > v2 FROM test; -``` - -```text -┌─v1─┬─v1_type─┬─v2──────┬─v2_type───────┬─equals(v1, v2)─┬─less(v1, v2)─┬─greater(v1, v2)─┐ -│ 42 │ UInt64 │ 42 │ UInt64 │ 1 │ 0 │ 0 │ -│ 42 │ UInt64 │ 43 │ UInt64 │ 0 │ 1 │ 0 │ -│ 42 │ UInt64 │ abc │ String │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [1,2,3] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ ᴺᵁᴸᴸ │ None │ 0 │ 1 │ 0 │ -└────┴─────────┴─────────┴───────────────┴────────────────┴──────────────┴─────────────────┘ - -``` - -Если вам нужно найти строку с определённым значением `Variant`, вы можете поступить одним из следующих способов: - -* Привести значение к соответствующему типу `Variant`: - -```sql -SELECT * FROM test WHERE v2 == [1,2,3]::Array(UInt32)::Variant(String, UInt64, Array(UInt32)); -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -* Сравните подстолбец `Variant` с требуемым типом: - -```sql -SELECT * FROM test WHERE v2.`Array(UInt32)` == [1,2,3] -- или с использованием variantElement(v2, 'Array(UInt32)') -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -Иногда полезно дополнительно проверить тип Variant, так как подколонки со сложными типами, такими как `Array/Map/Tuple`, не могут быть внутри `Nullable` и в строках с другими типами вместо `NULL` будут иметь значения по умолчанию: - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE v2.`Array(UInt32)` == []; -``` - -```text -┌─v2───┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ 42 │ [] │ UInt64 │ -│ 43 │ [] │ UInt64 │ -│ abc │ [] │ String │ -│ [] │ [] │ Array(UInt32) │ -│ ᴺᵁᴸᴸ │ [] │ None │ -└──────┴──────────────────┴─────────────────┘ -``` - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE variantType(v2) == 'Array(UInt32)' AND v2.`Array(UInt32)` == []; -``` - -```text -┌─v2─┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ [] │ [] │ Array(UInt32) │ -└────┴──────────────────┴─────────────────┘ -``` - -**Примечание:** значения вариантов с различными числовыми типами считаются различными и не сравниваются между собой; вместо этого сравниваются их имена типов. - -Пример: - -```sql -SET allow_suspicious_variant_types = 1; -CREATE TABLE test (v Variant(UInt32, Int64)) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT v, variantType(v) FROM test ORDER by v; -``` - -```text -┌─v───┬─variantType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -**Примечание.** По умолчанию тип `Variant` нельзя использовать в ключах `GROUP BY`/`ORDER BY`. Если вы хотите его применять, учтите его особое правило сравнения и включите настройки `allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by`. - -## Функции JSONExtract с Variant {#jsonextract-functions-with-variant} - -Все функции `JSONExtract*` поддерживают тип данных `Variant`: - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Variant(UInt32, String, Array(UInt32))') AS variant, variantType(variant) AS variant_type; -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt32) │ -└─────────┴───────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Variant(UInt32, String, Array(UInt32)))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types -``` - -```text -┌─map_of_variants──────────────────┬─map_of_variant_types────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'UInt32','b':'String','c':'Array(UInt32)'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────┘ -``` - -```sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Variant(UInt32, String, Array(UInt32))') AS variants, arrayMap(x -> (x.1, variantType(x.2)), variants) AS variant_types -``` - -```text -┌─variants───────────────────────────────┬─variant_types─────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','UInt32'),('b','String'),('c','Array(UInt32)')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md deleted file mode 100644 index 10a81801ce6..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md +++ /dev/null @@ -1,4 +0,0 @@ -:::tip -Если вы используете словари с ClickHouse Cloud, используйте DDL-запрос для их создания и создавайте словари от имени пользователя `default`. -Также проверьте список поддерживаемых источников словарей в [руководстве по совместимости с ClickHouse Cloud](/whats-new/cloud-compatibility). -::: \ No newline at end of file diff --git a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md b/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md deleted file mode 100644 index 7ff89117ba8..00000000000 --- a/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md +++ /dev/null @@ -1,2531 +0,0 @@ ---- -description: 'Обзор функциональных возможностей внешних словарей в ClickHouse' -sidebar_label: 'Определение словарей' -sidebar_position: 35 -slug: /sql-reference/dictionaries -title: 'Словари' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/ru/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; -import CloudDetails from '@site/i18n/ru/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# Словари {#dictionaries} - -Словарь — это отображение (`key -> attributes`), удобное для различных типов справочных списков. - -ClickHouse поддерживает специальные функции для работы со словарями, которые можно использовать в запросах. Использовать словари с функциями проще и эффективнее, чем `JOIN` со справочными таблицами. - -ClickHouse поддерживает: - -- Словари с [набором функций](../../sql-reference/functions/ext-dict-functions.md). -- [Встроенные словари](#embedded-dictionaries) с определённым [набором функций](../../sql-reference/functions/embedded-dict-functions.md). - -:::tip Tutorial -Если вы только начинаете работать со словарями в ClickHouse, у нас есть руководство, посвящённое этой теме. Ознакомьтесь с ним [здесь](tutorial.md). -::: - -Вы можете добавлять собственные словари из различных источников данных. Источником для словаря может быть таблица ClickHouse, локальный текстовый или исполняемый файл, ресурс HTTP(s) или другая СУБД. Для получения дополнительной информации см. раздел «[Источники словарей](#dictionary-sources)». - -ClickHouse: - -- Полностью или частично хранит словари в оперативной памяти (RAM). -- Периодически обновляет словари и динамически загружает отсутствующие значения. Другими словами, словари могут загружаться динамически. -- Позволяет создавать словари с помощью XML-файлов или [DDL-запросов](../../sql-reference/statements/create/dictionary.md). - -Конфигурация словарей может быть расположена в одном или нескольких XML-файлах. Путь к конфигурации задаётся параметром [dictionaries_config](../../operations/server-configuration-parameters/settings.md#dictionaries_config). - -Словари могут загружаться при запуске сервера или при первом использовании, в зависимости от настройки [dictionaries_lazy_load](../../operations/server-configuration-parameters/settings.md#dictionaries_lazy_load). - -Системная таблица [dictionaries](/operations/system-tables/dictionaries) содержит информацию о словарях, настроенных на сервере. Для каждого словаря вы можете найти там: - -- Статус словаря. -- Параметры конфигурации. -- Метрики, такие как объём RAM, выделенный для словаря, или количество запросов с момента успешной загрузки словаря. - - - -## Создание словаря с помощью DDL-запроса {#creating-a-dictionary-with-a-ddl-query} - -Словари можно создавать с помощью [DDL-запросов](../../sql-reference/statements/create/dictionary.md), и это рекомендуемый способ, поскольку у словарей, созданных с помощью DDL: -- В конфигурационные файлы сервера не добавляются дополнительные записи. -- Со словарями можно работать как с полноправными сущностями, подобно таблицам или представлениям. -- Данные можно читать напрямую, используя привычный SELECT, а не табличные функции для словарей. Обратите внимание, что при непосредственном доступе к словарю через оператор SELECT кэшируемый словарь вернёт только данные, уже находящиеся в кэше, тогда как некэшируемый словарь вернёт все данные, которые он хранит. -- Словари можно легко переименовывать. - -## Создание словаря с помощью файла конфигурации {#creating-a-dictionary-with-a-configuration-file} - - - -:::note -Создание словаря с помощью файла конфигурации в ClickHouse Cloud не поддерживается. Пожалуйста, используйте DDL (см. выше) и создайте словарь от имени пользователя `default`. -::: - -Файл конфигурации словаря имеет следующий формат: - -```xml - - Необязательный элемент с любым содержимым. Игнорируется сервером ClickHouse. - - - /etc/metrika.xml - - - - - - - - -``` - -Вы можете [настроить](#configuring-a-dictionary) любое количество словарей в одном файле. - -:::note -Вы можете преобразовать значения для небольшого словаря, описав его в запросе `SELECT` (см. функцию [transform](../../sql-reference/functions/other-functions.md)). Данная функциональность не относится к словарям. -::: - -## Настройка словаря {#configuring-a-dictionary} - - - -Если словарь настраивается с помощью XML-файла, конфигурация словаря имеет следующую структуру: - -```xml - - dict_name - - - - - - - - - - - - - - - - - -``` - -Соответствующий [DDL-запрос](../../sql-reference/statements/create/dictionary.md) имеет следующую структуру: - -```sql -CREATE DICTIONARY dict_name -( - ... -- атрибуты -) -PRIMARY KEY ... -- настройка составного или одиночного ключа -SOURCE(...) -- Настройка источника -LAYOUT(...) -- Настройка размещения в памяти -LIFETIME(...) -- Время жизни словаря в памяти -``` - -## Хранение словарей в памяти {#storing-dictionaries-in-memory} - -Существует несколько способов хранения словарей в памяти. - -Мы рекомендуем [flat](#flat), [hashed](#hashed) и [complex_key_hashed](#complex_key_hashed), которые обеспечивают оптимальную скорость обработки. - -Кэширование не рекомендуется из-за потенциально низкой производительности и сложности подбора оптимальных параметров. Подробнее см. в разделе [cache](#cache). - -Существует несколько способов повысить производительность словарей: - -* Вызывайте функцию для работы со словарём после `GROUP BY`. -* Помечайте извлекаемые атрибуты как инъективные. Атрибут называется инъективным, если разным ключам соответствуют разные значения атрибута. Поэтому, когда `GROUP BY` использует функцию, извлекающую значение атрибута по ключу, эта функция автоматически выносится из `GROUP BY`. - -ClickHouse генерирует исключение при ошибках, связанных со словарями. Примеры ошибок: - -* Не удалось загрузить словарь, к которому выполняется обращение. -* Ошибка при запросе к словарю типа `cached`. - -Вы можете посмотреть список словарей и их статусы в таблице [system.dictionaries](../../operations/system-tables/dictionaries.md). - - - -Конфигурация выглядит следующим образом: - -```xml - - - ... - - - - - - ... - - -``` - -Соответствующий [DDL-запрос](../../sql-reference/statements/create/dictionary.md): - -```sql -CREATE DICTIONARY (...) -... -LAYOUT(LAYOUT_TYPE(param value)) -- настройки структуры хранения -... -``` - -Словари, в названии макета которых отсутствует слово `complex-key*`, имеют ключ типа [UInt64](../../sql-reference/data-types/int-uint.md), словари с макетом `complex-key*` используют составной ключ (complex, с произвольными типами). - -Ключи [UInt64](../../sql-reference/data-types/int-uint.md) в XML-словарях задаются с помощью тега ``. - -Пример конфигурации (столбец key_column имеет тип UInt64): - -```xml -... - - - key_column - -... -``` - -Составные ключи типа `complex` в XML-словарях определяются с помощью тега ``. - -Пример конфигурации составного ключа (ключ имеет один элемент типа [String](../../sql-reference/data-types/string.md)): - -```xml -... - - - - country_code - String - - -... -``` - -## Способы хранения словарей в памяти {#ways-to-store-dictionaries-in-memory} - -Различные способы хранения данных словаря в памяти связаны с компромиссами по потреблению CPU и RAM. Дерево решений, опубликованное в разделе [Choosing a Layout](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout) [статьи в блоге](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse), посвящённой словарям, является хорошей отправной точкой для выбора подходящего типа размещения. - -* [flat](#flat) -* [hashed](#hashed) -* [sparse_hashed](#sparse_hashed) -* [complex_key_hashed](#complex_key_hashed) -* [complex_key_sparse_hashed](#complex_key_sparse_hashed) -* [hashed_array](#hashed_array) -* [complex_key_hashed_array](#complex_key_hashed_array) -* [range_hashed](#range_hashed) -* [complex_key_range_hashed](#complex_key_range_hashed) -* [cache](#cache) -* [complex_key_cache](#complex_key_cache) -* [ssd_cache](#ssd_cache) -* [complex_key_ssd_cache](#complex_key_ssd_cache) -* [direct](#direct) -* [complex_key_direct](#complex_key_direct) -* [ip_trie](#ip_trie) - -### flat {#flat} - -Словарь полностью хранится в памяти в виде плоских массивов. Сколько памяти использует словарь? Объём пропорционален значению наибольшего ключа (в занимаемом им пространстве). - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md), а значение ограничено `max_array_size` (по умолчанию — 500,000). Если при создании словаря обнаруживается ключ с большим значением, ClickHouse генерирует исключение и не создаёт словарь. Начальный размер плоских массивов словаря задаётся настройкой `initial_array_size` (по умолчанию — 1024). - -Поддерживаются все типы источников. При обновлении данные (из файла или таблицы) читаются целиком. - -Этот метод обеспечивает наилучшую производительность среди всех доступных методов хранения словаря. - -Пример конфигурации: - -```xml - - - 50000 - 5000000 - - -``` - -или - -```sql -LAYOUT(FLAT(INITIAL_ARRAY_SIZE 50000 MAX_ARRAY_SIZE 5000000)) -``` - -### hashed {#hashed} - -Словарь полностью хранится в памяти в виде хеш-таблицы. Словарь может содержать любое количество элементов с произвольными идентификаторами. На практике количество ключей может достигать десятков миллионов. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -Поддерживаются все типы источников. При обновлении данные (из файла или из таблицы) считываются целиком. - -Пример конфигурации: - -```xml - - - -``` - -или - -```sql -LAYOUT(HASHED()) -``` - -Пример конфигурации: - -```xml - - - - 10 - - - 10000 - - - 0.5 - - -``` - -или - -```sql -LAYOUT(HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### sparse_hashed {#sparse_hashed} - -Похожа на `hashed`, но использует меньше памяти за счёт большего потребления ресурсов CPU. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -Пример конфигурации: - -```xml - - - - - - - -``` - -или - -```sql -LAYOUT(SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -Для этого типа словаря также можно использовать `shards`, и опять же это более важно для `sparse_hashed`, чем для `hashed`, так как `sparse_hashed` работает медленнее. - -### complex_key_hashed {#complex_key_hashed} - -Этот тип хранения словаря предназначен для использования с составными [ключами](#dictionary-key-and-fields). Аналогичен типу `hashed`. - -Пример конфигурации: - -```xml - - - - - - - -``` - -или - -```sql -LAYOUT(COMPLEX_KEY_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### complex_key_sparse_hashed {#complex_key_sparse_hashed} - -Этот тип хранилища предназначен для использования с составными [ключами](#dictionary-key-and-fields). Аналогичен [sparse_hashed](#sparse_hashed). - -Пример конфигурации: - -```xml - - - - - - - -``` - -или - -```sql -LAYOUT(COMPLEX_KEY_SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### hashed_array {#hashed_array} - -Словарь полностью хранится в памяти. Каждый атрибут хранится в массиве. Атрибут-ключ хранится в виде хеш-таблицы, где значение — это индекс в массиве атрибутов. Словарь может содержать любое количество элементов с любыми идентификаторами. На практике число ключей может достигать десятков миллионов. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -Поддерживаются все типы источников. При обновлении данные (из файла или из таблицы) читаются целиком. - -Пример конфигурации: - -```xml - - - - -``` - -или - -```sql -LAYOUT(HASHED_ARRAY([SHARDS 1])) -``` - -### complex_key_hashed_array {#complex_key_hashed_array} - -Этот тип хранилища предназначен для использования с составными [ключами](#dictionary-key-and-fields). Аналогичен [hashed_array](#hashed_array). - -Пример конфигурации: - -```xml - - - -``` - -или - -```sql -LAYOUT(COMPLEX_KEY_HASHED_ARRAY([SHARDS 1])) -``` - -### range_hashed {#range_hashed} - -Словарь хранится в памяти в виде хеш-таблицы с упорядоченным массивом диапазонов и соответствующими им значениями. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). -Этот способ хранения работает так же, как словарь типа `hashed`, и позволяет использовать диапазоны значений даты/времени (любого числового типа) в дополнение к ключу. - -Пример: таблица содержит скидки для каждого рекламодателя в формате: - -```text -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 123 │ 2015-01-16 │ 2015-01-31 │ 0.25 │ -│ 123 │ 2015-01-01 │ 2015-01-15 │ 0.15 │ -│ 456 │ 2015-01-01 │ 2015-01-15 │ 0.05 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ -``` - -Чтобы использовать выборку по диапазонам дат, определите элементы `range_min` и `range_max` в [структуре](#dictionary-key-and-fields). Эти элементы должны содержать элементы `name` и `type` (если `type` не указан, по умолчанию используется тип Date). `type` может быть любым числовым типом (Date / DateTime / UInt64 / Int32 / другие). - -:::note -Значения `range_min` и `range_max` должны умещаться в диапазон типа `Int64`. -::: - -Пример: - -```xml - - - - min - - - - - advertiser_id - - - discount_start_date - Date - - - discount_end_date - Date - - ... -``` - -или - -```sql -CREATE DICTIONARY discounts_dict ( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Date, - amount Float64 -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(TABLE 'discounts')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(RANGE_HASHED(range_lookup_strategy 'max')) -RANGE(MIN discount_start_date MAX discount_end_date) -``` - -Чтобы работать с этими словарями, необходимо передать функции `dictGet` дополнительный аргумент, для которого задаётся диапазон: - -```sql -dictGet('dict_name', 'attr_name', id, date) -``` - -Пример запроса: - -```sql -SELECT dictGet('discounts_dict', 'amount', 1, '2022-10-20'::Date); -``` - -Эта функция возвращает значение для указанных `id` и диапазона дат, охватывающего переданную дату. - -Подробности алгоритма: - -* Если `id` не найден или для `id` не найден диапазон, возвращается значение по умолчанию для типа атрибута. -* Если есть пересекающиеся диапазоны и `range_lookup_strategy=min`, возвращается подходящий диапазон с минимальным `range_min`; если найдено несколько таких диапазонов, возвращается диапазон с минимальным `range_max`; если снова найдено несколько диапазонов (несколько диапазонов имеют одинаковые `range_min` и `range_max`), возвращается случайный диапазон из них. -* Если есть пересекающиеся диапазоны и `range_lookup_strategy=max`, возвращается подходящий диапазон с максимальным `range_min`; если найдено несколько таких диапазонов, возвращается диапазон с максимальным `range_max`; если снова найдено несколько диапазонов (несколько диапазонов имеют одинаковые `range_min` и `range_max`), возвращается случайный диапазон из них. -* Если `range_max` равен `NULL`, диапазон считается открытым. `NULL` трактуется как максимально возможное значение. Для `range_min` в качестве открытого значения могут использоваться `1970-01-01` или `0` (-MAX_INT). - -Пример конфигурации: - -```xml - - - ... - - - - - - - - Abcdef - - - StartTimeStamp - UInt64 - - - EndTimeStamp - UInt64 - - - XXXType - String - - - - - - -``` - -или - -```sql -CREATE DICTIONARY somedict( - Abcdef UInt64, - StartTimeStamp UInt64, - EndTimeStamp UInt64, - XXXType String DEFAULT '' -) -PRIMARY KEY Abcdef -RANGE(MIN StartTimeStamp MAX EndTimeStamp) -``` - -Пример конфигурации с перекрывающимися и открытыми диапазонами: - -```sql -CREATE TABLE discounts -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -ENGINE = Memory; -``` - -INSERT INTO discounts VALUES (1, '2015-01-01', Null, 0.1); -INSERT INTO discounts VALUES (1, '2015-01-15', Null, 0.2); -INSERT INTO discounts VALUES (2, '2015-01-01', '2015-01-15', 0.3); -INSERT INTO discounts VALUES (2, '2015-01-04', '2015-01-10', 0.4); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-15', 0.5); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-10', 0.6); - -SELECT * FROM discounts ORDER BY advertiser_id, discount_start_date; -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 1 │ 2015-01-01 │ ᴺᵁᴸᴸ │ 0.1 │ -│ 1 │ 2015-01-15 │ ᴺᵁᴸᴸ │ 0.2 │ -│ 2 │ 2015-01-01 │ 2015-01-15 │ 0.3 │ -│ 2 │ 2015-01-04 │ 2015-01-10 │ 0.4 │ -│ 3 │ 1970-01-01 │ 2015-01-15 │ 0.5 │ -│ 3 │ 1970-01-01 │ 2015-01-10 │ 0.6 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ - --- RANGE_LOOKUP_STRATEGY 'max' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'max')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- совпадает только один диапазон: 2015-01-01 – Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.2 │ -- совпадают два диапазона, range_min 2015-01-15 (0.2) больше, чем 2015-01-01 (0.1) -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.4 │ -- совпадают два диапазона, range_min 2015-01-04 (0.4) больше, чем 2015-01-01 (0.3) -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.5 │ -- совпадают два диапазона, значения range_min равны; 2015-01-15 (0.5) больше, чем 2015-01-10 (0.6) -└─────┘ - -DROP DICTIONARY discounts_dict; - --- RANGE_LOOKUP_STRATEGY 'min' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'min')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- совпадает только один диапазон: 2015-01-01 – Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.1 │ -- совпадают два диапазона, range_min 2015-01-01 (0.1) меньше чем 2015-01-15 (0.2) -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.3 │ -- совпадают два диапазона, range_min 2015-01-01 (0.3) меньше чем 2015-01-04 (0.4) -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.6 │ -- совпадают два диапазона, значения range_min равны, 2015-01-10 (0.6) меньше чем 2015-01-15 (0.5) -└─────┘ - -```` - -### complex_key_range_hashed {#complex_key_range_hashed} - -Словарь хранится в памяти в виде хеш-таблицы с упорядоченным массивом диапазонов и их соответствующими значениями (см. [range_hashed](#range_hashed)). Данный тип хранения используется с составными [ключами](#dictionary-key-and-fields). - -Пример конфигурации: - -```sql -CREATE DICTIONARY range_dictionary -( - CountryID UInt64, - CountryKey String, - StartDate Date, - EndDate Date, - Tax Float64 DEFAULT 0.2 -) -PRIMARY KEY CountryID, CountryKey -SOURCE(CLICKHOUSE(TABLE 'date_table')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(COMPLEX_KEY_RANGE_HASHED()) -RANGE(MIN StartDate MAX EndDate); -```` - -### cache {#cache} - -Словарь хранится в кэше с фиксированным количеством ячеек. Эти ячейки содержат часто используемые элементы. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -При обращении к словарю сначала производится поиск в кэше. Для каждого блока данных все ключи, которые не найдены в кэше или устарели, запрашиваются из источника с помощью `SELECT attrs... FROM db.table WHERE id IN (k1, k2, ...)`. Полученные данные затем записываются в кэш. - -Если часть ключей не найдена в словаре, создаётся задача обновления кэша и добавляется в очередь обновлений. Свойствами очереди обновлений можно управлять с помощью настроек `max_update_queue_size`, `update_queue_push_timeout_milliseconds`, `query_wait_timeout_milliseconds`, `max_threads_for_updates`. - -Для словарей типа cache можно задать время жизни ([lifetime](#refreshing-dictionary-data-using-lifetime)) данных в кэше. Если с момента загрузки данных в ячейку прошло больше времени, чем `lifetime`, значение ячейки не используется, и ключ считается просроченным. Ключ будет повторно запрошен при следующем обращении. Такое поведение можно настроить с помощью параметра `allow_read_expired_keys`. - -Это наименее эффективный из всех способов хранения словарей. Производительность кэша сильно зависит от корректных настроек и сценариев использования. Словарь типа cache работает хорошо только при достаточно высоком уровне попаданий (рекомендуется 99% и выше). Средний уровень попаданий можно посмотреть в таблице [system.dictionaries](../../operations/system-tables/dictionaries.md). - -Если настройка `allow_read_expired_keys` установлена в 1 (по умолчанию 0), словарь может поддерживать асинхронные обновления. Если клиент запрашивает ключи и все они находятся в кэше, но некоторые из них просрочены, словарь вернёт клиенту просроченные значения и асинхронно запросит их из источника. - -Для повышения производительности кэша используйте подзапрос с `LIMIT` и вызывайте функцию, использующую словарь, снаружи. - -Поддерживаются все типы источников. - -Пример настроек: - -```xml - - - - 1000000000 - - 0 - - 100000 - - 10 - - 60000 - - 4 - - -``` - -или - -```sql -LAYOUT(CACHE(SIZE_IN_CELLS 1000000000)) -``` - -Задайте достаточно большой размер кэша. Необходимо поэкспериментировать, чтобы подобрать количество ячеек: - -1. Задайте некоторое значение. -2. Выполняйте запросы, пока кэш полностью не заполнится. -3. Оцените потребление памяти с помощью таблицы `system.dictionaries`. -4. Увеличивайте или уменьшайте количество ячеек, пока не будет достигнут требуемый уровень потребления памяти. - -:::note -Не используйте ClickHouse в качестве источника, так как он медленно обрабатывает запросы со случайным чтением. -::: - -### complex_key_cache {#complex_key_cache} - -Этот тип хранилища предназначен для работы с составными [ключами](#dictionary-key-and-fields). Аналогичен `cache`. - -### ssd_cache {#ssd_cache} - -Аналогичен `cache`, но хранит данные на SSD, а индекс — в RAM. Все настройки словарей типа cache, связанные с очередью обновления, также могут применяться к словарям SSD cache. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -```xml - - - - 4096 - - 16777216 - - 131072 - - 1048576 - - /var/lib/clickhouse/user_files/test_dict - - -``` - -или - -```sql -LAYOUT(SSD_CACHE(BLOCK_SIZE 4096 FILE_SIZE 16777216 READ_BUFFER_SIZE 1048576 - PATH '/var/lib/clickhouse/user_files/test_dict')) -``` - -### complex_key_ssd_cache {#complex_key_ssd_cache} - -Этот тип хранилища предназначен для составных [ключей](#dictionary-key-and-fields). Аналогичен `ssd_cache`. - -### direct {#direct} - -Словарь не хранится в памяти, и при обработке запроса данные запрашиваются непосредственно из источника. - -Ключ словаря имеет тип [UInt64](../../sql-reference/data-types/int-uint.md). - -Поддерживаются все типы [источников](#dictionary-sources), кроме локальных файлов. - -Пример конфигурации: - -```xml - - - -``` - -или - -```sql -LAYOUT(DIRECT()) -``` - -### complex_key_direct {#complex_key_direct} - -Этот тип хранилища предназначен для использования с составными [ключами](#dictionary-key-and-fields). Аналогичен `direct`. - -### ip_trie {#ip_trie} - -Этот словарь предназначен для поиска IP-адресов по сетевому префиксу. Он хранит IP-диапазоны в нотации CIDR и позволяет быстро определить, к какому префиксу (например, подсети или диапазону ASN) относится заданный IP, что делает его идеальным для поисковых операций по IP, таких как геолокация или классификация сетей. - - - - -该引擎会处理所有具有以下类型的列: - -- [`AggregateFunction`](../../../sql-reference/data-types/aggregatefunction.md) -- [`SimpleAggregateFunction`](../../../sql-reference/data-types/simpleaggregatefunction.md) - -当使用 `AggregatingMergeTree` 能够将行数减少若干个数量级时,就适合采用该引擎。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = AggregatingMergeTree() -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[TTL expr] -[SETTINGS name=value, ...] -``` - -有关请求参数的说明,请参阅[请求描述](../../../sql-reference/statements/create/table.md)。 - -**查询子句** - -在创建 `AggregatingMergeTree` 表时,所需的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)与创建 `MergeTree` 表时相同。 - -
- 已弃用的建表方法 - - :::note - 不要在新项目中使用此方法,并尽可能将旧项目迁移到上文所述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] AggregatingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - 所有参数的含义都与 `MergeTree` 引擎中的相同。 -
- - -## SELECT 和 INSERT {#select-and-insert} - -要插入数据,请使用包含带有 `-State` 后缀聚合函数的 [INSERT SELECT](../../../sql-reference/statements/insert-into.md) 查询。 -从 `AggregatingMergeTree` 表中选择数据时,使用 `GROUP BY` 子句,并使用与插入数据时相同的聚合函数,但需改用带有 `-Merge` 后缀的版本。 - -在 `SELECT` 查询的结果中,`AggregateFunction` 类型的值在所有 ClickHouse 输出格式中都采用与实现相关的二进制表示形式。例如,如果使用 `SELECT` 查询将数据转储为 `TabSeparated` 格式,则可以通过 `INSERT` 查询将该转储重新加载回去。 - - - -## 聚合物化视图示例 {#example-of-an-aggregated-materialized-view} - -以下示例假设已存在名为 `test` 的数据库。如尚不存在,请使用以下命令创建: - -```sql -CREATE DATABASE test; -``` - -现在创建表 `test.visits`,用于存放原始数据: - -```sql -CREATE TABLE test.visits - ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Sign Nullable(Int32), - UserID Nullable(Int32) -) ENGINE = MergeTree ORDER BY (StartDate, CounterID); -``` - -接下来,您需要一个 `AggregatingMergeTree` 表,用于存储 `AggregationFunction`,以记录访问总次数和唯一用户数量。 - -创建一个使用 `AggregatingMergeTree` 的物化视图,用于监听 `test.visits` 表,并使用 [`AggregateFunction`](/sql-reference/data-types/aggregatefunction) 类型: - -```sql -CREATE TABLE test.agg_visits ( - StartDate DateTime64 NOT NULL, - CounterID UInt64, - Visits AggregateFunction(sum, Nullable(Int32)), - Users AggregateFunction(uniq, Nullable(Int32)) -) -ENGINE = AggregatingMergeTree() ORDER BY (StartDate, CounterID); -``` - -创建一个物化视图,将 `test.visits` 的数据写入 `test.agg_visits`: - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - sumState(Sign) AS Visits, - uniqState(UserID) AS Users -FROM test.visits -GROUP BY StartDate, CounterID; -``` - -向 `test.visits` 表中插入数据: - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1667446031000, 1, 3, 4), (1667446031000, 1, 6, 3); -``` - -数据会被插入到 `test.visits` 和 `test.agg_visits` 表中。 - -要获取聚合后的数据,可以对物化视图 `test.visits_mv` 执行类似 `SELECT ... GROUP BY ...` 的查询: - -```sql -SELECT - StartDate, - sumMerge(Visits) AS Visits, - uniqMerge(Users) AS Users -FROM test.visits_mv -GROUP BY StartDate -ORDER BY StartDate; -``` - -```text -┌───────────────StartDate─┬─Visits─┬─Users─┐ -│ 2022-11-03 03:27:11.000 │ 9 │ 2 │ -└─────────────────────────┴────────┴───────┘ -``` - -向 `test.visits` 再添加两条记录,但这次请为其中一条记录使用不同的时间戳: - -```sql -INSERT INTO test.visits (StartDate, CounterID, Sign, UserID) - VALUES (1669446031000, 2, 5, 10), (1667446031000, 3, 7, 5); -``` - -再次运行 `SELECT` 查询,此时会返回如下输出: - -```text -┌───────────────开始日期─┬─访问次数─┬─用户数─┐ -│ 2022-11-03 03:27:11.000 │ 16 │ 3 │ -│ 2022-11-26 07:00:31.000 │ 5 │ 1 │ -└─────────────────────────┴────────┴───────┘ -``` - -在某些情况下,您可能希望在插入时避免对行进行预聚合,而是将聚合的开销从插入阶段转移到合并阶段。通常,为了避免报错,必须在物化视图定义的 `GROUP BY` 子句中包含那些不参与聚合的列。不过,您可以结合使用 [`initializeAggregation`](/sql-reference/functions/other-functions#initializeAggregation) 函数,并将 `optimize_on_insert = 0`(默认值为开启)来实现这一点。在这种情况下,就不再需要使用 `GROUP BY` 了: - -```sql -CREATE MATERIALIZED VIEW test.visits_mv TO test.agg_visits -AS SELECT - StartDate, - CounterID, - initializeAggregation('sumState', Sign) AS Visits, - initializeAggregation('uniqState', UserID) AS Users -FROM test.visits; -``` - - -:::note -在使用 `initializeAggregation` 时,会为每一行单独创建一个聚合状态,而不进行分组。 -每一行源数据会在物化视图中生成一行,实际的聚合操作会在稍后 `AggregatingMergeTree` 合并数据分片(parts)时进行。仅当 `optimize_on_insert = 0` 时才是如此。 -::: - - - -## 相关内容 {#related-content} - -- 博客文章:[在 ClickHouse 中使用聚合组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md deleted file mode 100644 index 188027b0fa0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/annindexes.md +++ /dev/null @@ -1,736 +0,0 @@ ---- -description: '精确和近似向量搜索文档' -keywords: ['向量相似搜索', 'ann', 'knn', 'hnsw', '索引(indices)', '索引', '最近邻', '向量搜索'] -sidebar_label: '精确和近似向量搜索' -slug: /engines/table-engines/mergetree-family/annindexes -title: '精确和近似向量搜索' -doc_type: 'guide' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# 精确与近似向量搜索 {#exact-and-approximate-vector-search} - -在给定多维(向量)空间中的一个点时,寻找与其距离最近的 N 个点的问题,被称为[最近邻搜索](https://en.wikipedia.org/wiki/Nearest_neighbor_search),简称向量搜索。 -解决向量搜索通常有两种通用方法: - -* 精确向量搜索会计算查询点与向量空间中所有点之间的距离。这可以保证尽可能高的准确性,即返回的点一定是实际的最近邻。由于需要对整个向量空间进行穷举搜索,精确向量搜索在真实场景中往往会过于缓慢。 -* 近似向量搜索指一类技术(例如使用图或随机森林等特殊数据结构),它们比精确向量搜索能更快地计算结果。结果的准确性通常对实际使用来说“足够好”。许多近似技术都提供参数,用于在结果准确性和搜索时间之间进行权衡调优。 - -一次向量搜索(无论精确还是近似)可以用 SQL 表达如下: - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- WHERE 子句为可选项 -ORDER BY (vectors, reference_vector) -LIMIT -``` - -向量空间中的点存储在名为 `vectors` 的数组类型列中,例如 [Array(Float64)](../../../sql-reference/data-types/array.md)、[Array(Float32)](../../../sql-reference/data-types/array.md) 或 [Array(BFloat16)](../../../sql-reference/data-types/array.md)。 -参考向量是一个常量数组,并通过公用表表达式(CTE)提供。 -`<DistanceFunction>` 计算参考点与所有已存储点之间的距离。 -可以使用任意可用的[距离函数](/sql-reference/functions/distance-functions)来实现。 -`<N>` 指定应返回多少个近邻。 - - -## 精确向量搜索 {#exact-nearest-neighbor-search} - -可以直接使用上面的 SELECT 查询执行精确向量搜索。 -此类查询的运行时间通常与已存储向量的数量及其维度成正比,即数组元素的数量。 -此外,由于 ClickHouse 会对所有向量进行暴力扫描(brute-force scan),运行时间还取决于查询使用的线程数(参见设置 [max_threads](../../../operations/settings/settings.md#max_threads))。 - -### 示例 {#exact-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -返回 - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - - -## 近似向量搜索 {#approximate-nearest-neighbor-search} - -### 向量相似度索引 {#vector-similarity-index} - -ClickHouse 提供了一种专用的“向量相似度”索引,用于执行近似向量搜索。 - -:::note -向量相似度索引在 ClickHouse 25.8 及更高版本中可用。 -如果遇到问题,请在 [ClickHouse 仓库](https://github.com/clickhouse/clickhouse/issues) 中提交 issue。 -::: - -#### 创建向量相似度索引 {#creating-a-vector-similarity-index} - -可以在新表上按如下方式创建向量相似度索引: - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -或者,要在现有表上添加向量相似索引: - -```sql -ALTER TABLE table ADD INDEX vectors TYPE vector_similarity(, , ) [GRANULARITY ]; -``` - -向量相似度索引是一种特殊的跳过索引(参见[这里](mergetree.md#table_engine-mergetree-data_skipping-indexes)和[这里](../../../optimize/skipping-indexes))。 -因此,上面的 `ALTER TABLE` 语句只会为之后插入到该表中的新数据构建索引。 -要同时为已有数据构建索引,你需要对索引进行物化: - -```sql -ALTER TABLE table MATERIALIZE INDEX SETTINGS mutations_sync = 2; -``` - -函数 `` 必须是 - -* `L2Distance`,[Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance)(欧几里得距离),表示欧几里得空间中两点之间线段的长度,或 -* `cosineDistance`,[cosine distance](https://en.wikipedia.org/wiki/Cosine_similarity#Cosine_distance)(余弦距离),表示两个非零向量之间的夹角。 - -对于已归一化的数据,通常 `L2Distance` 是最佳选择;否则,推荐使用 `cosineDistance` 来补偿尺度差异。 - -`` 指定底层列中数组的基数(元素数量)。 -如果 ClickHouse 在创建索引时发现数组的基数不一致,则会丢弃该索引并返回错误。 - -可选的 GRANULARITY 参数 `` 指的是索引粒度的大小(参见[此处](../../../optimize/skipping-indexes))。 -默认值 1 亿在大多数用例中应该都能有不错的表现,但也可以进行调优。 -我们建议仅由了解其影响的高级用户进行调优(参见[下文](#differences-to-regular-skipping-indexes))。 - -向量相似性索引在通用意义上来说是通用的,即可以适配不同的近似搜索方法。 -实际使用的方法由参数 `` 指定。 -当前唯一可用的近似搜索方法是 HNSW([academic paper](https://arxiv.org/abs/1603.09320)),这是一种基于分层近邻图的流行且最先进的近似向量搜索技术。 -如果将 HNSW 指定为 type,用户还可以选择性地指定更多 HNSW 专用参数: - -```sql -CREATE TABLE table -( - [...], - vectors Array(Float*), - INDEX index_name vectors TYPE vector_similarity('hnsw', , [, , , ]) [GRANULARITY N] -) -ENGINE = MergeTree -ORDER BY [...] -``` - -以下是可用的 HNSW 专用参数: - -* `` 控制邻近图中向量的量化方式。可选值为 `f64`、`f32`、`f16`、`bf16`、`i8` 或 `b1`。默认值为 `bf16`。请注意,此参数不会影响底层列中向量的表示形式。 -* `` 控制每个图节点的邻居数量,也称为 HNSW 超参数 `M`。默认值为 `32`。值为 `0` 表示使用默认值。 -* `` 控制构建 HNSW 图时动态候选列表的大小,也称为 HNSW 超参数 `ef_construction`。默认值为 `128`。值为 `0` 表示使用默认值。 - -所有 HNSW 专用参数的默认值在大多数用例中都有良好表现。 -因此,我们不建议自定义这些 HNSW 专用参数。 - - -适用以下进一步限制: - -* 向量相似度索引只能建立在类型为 [Array(Float32)](../../../sql-reference/data-types/array.md)、[Array(Float64)](../../../sql-reference/data-types/array.md) 或 [Array(BFloat16)](../../../sql-reference/data-types/array.md) 的列上。诸如 `Array(Nullable(Float32))` 和 `Array(LowCardinality(Float32))` 这类可为空或低基数浮点数组不被允许。 -* 向量相似度索引必须建立在单个列上。 -* 向量相似度索引可以建立在计算表达式上(例如 `INDEX index_name arraySort(vectors) TYPE vector_similarity([...])`),但此类索引之后不能用于近似近邻搜索。 -* 向量相似度索引要求底层列中的所有数组都具有 `` 个元素——这一点会在创建索引时进行检查。为了尽早发现对此要求的违规情况,用户可以为向量列添加一个[约束](/sql-reference/statements/create/table.md#constraints),例如:`CONSTRAINT same_length CHECK length(vectors) = 256`。 -* 同样,底层列中的数组值不能为空(`[]`),也不能为默认值(同样是 `[]`)。 - -**估算存储和内存占用** - -为典型 AI 模型(例如大语言模型,[LLMs](https://en.wikipedia.org/wiki/Large_language_model))生成的向量由数百或数千个浮点值组成。 -因此,单个向量值就可能消耗数千字节的内存。 -希望估算表中底层向量列所需存储空间,以及向量相似度索引所需内存的用户,可以使用下面两个公式: - -表中向量列的存储占用(未压缩): - -```text -存储消耗 = 向量数量 × 维度 × 列数据类型大小 -``` - -以 [DBpedia 数据集](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) 为例: - -```text -存储消耗 = 100 万 × 1536 × 4(Float32 类型)= 6.1 GB -``` - -在执行搜索时,必须将向量相似度索引从磁盘完整加载到内存中。 -同样,向量索引也是先在内存中完全构建,然后再保存到磁盘。 - -加载一个向量索引所需的内存占用: - -```text -索引中向量的内存占用 (mv) = 向量数量 × 维度 × 量化数据类型大小 -内存图的内存占用 (mg) = 向量数量 × hnsw_max_connections_per_layer × 每节点 ID 字节数 (= 4) × 层节点重复因子 (= 2) - -内存消耗总量:mv + mg -``` - -[dbpedia 数据集](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M)的示例: - -```text -索引中向量的内存占用 (mv) = 100 万 × 1536 × 2 (BFloat16 格式) = 3072 MB -内存图的内存占用 (mg) = 100 万 × 64 × 2 × 4 = 512 MB - -总内存消耗 = 3072 + 512 = 3584 MB -``` - -上述公式未将向量相似度索引在分配运行时数据结构(例如预分配缓冲区和缓存)时所需的额外内存考虑在内。 - -#### 使用向量相似度索引 {#using-a-vector-similarity-index} - -:::note -要使用向量相似度索引,[compatibility](../../../operations/settings/settings.md) 设置必须为 `''`(默认值)或不低于 `'25.1'` 的版本。 -::: - -向量相似度索引支持如下形式的 SELECT 查询: - -```sql -WITH [...] AS reference_vector -SELECT [...] -FROM table -WHERE [...] -- WHERE 子句为可选项 -ORDER BY (vectors, reference_vector) -LIMIT -``` - -ClickHouse 的查询优化器会尝试匹配上述查询模板,并利用可用的向量相似索引。 -只有当 SELECT 查询中的距离函数与索引定义中的距离函数相同时,查询才能使用向量相似索引。 - -高级用户可以为设置 [hnsw_candidate_list_size_for_search](../../../operations/settings/settings.md#hnsw_candidate_list_size_for_search) 提供自定义值(也称为 HNSW 超参数 "ef_search"),以在搜索过程中调节候选列表的大小(例如 `SELECT [...] SETTINGS hnsw_candidate_list_size_for_search = `)。 -该设置的默认值为 256,在大多数用例中表现良好。 -更高的取值意味着更高的准确性,但会以性能变慢为代价。 - - -如果查询可以使用向量相似性索引,ClickHouse 会检查在 SELECT 查询中提供的 LIMIT `` 是否处于合理范围内。 -更具体地说,如果 `` 大于设置项 [max_limit_for_vector_search_queries](../../../operations/settings/settings.md#max_limit_for_vector_search_queries) 的值(默认值为 100),则会返回错误。 -过大的 LIMIT 值会减慢搜索速度,并且通常表示用法错误。 - -要检查某个 SELECT 查询是否使用了向量相似性索引,可以在查询前加上前缀 `EXPLAIN indexes = 1`。 - -例如,查询 - -```sql -EXPLAIN indexes = 1 -WITH [0.462, 0.084, ..., -0.110] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 10; -``` - -可能会返回 - -```result - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (投影名称) │ - 2. │ Limit (初步 LIMIT(不含 OFFSET)) │ - 3. │ Sorting (ORDER BY 排序) │ - 4. │ Expression ((ORDER BY 之前 + (投影 + 将列名更改为列标识符))) │ - 5. │ ReadFromMergeTree (default.tab) │ - 6. │ Indexes: │ - 7. │ PrimaryKey │ - 8. │ Condition: true │ - 9. │ Parts: 1/1 │ -10. │ Granules: 575/575 │ -11. │ Skip │ -12. │ Name: idx │ -13. │ Description: vector_similarity GRANULARITY 100000000 │ -14. │ Parts: 1/1 │ -15. │ Granules: 10/575 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -在这个示例中,[dbpedia dataset](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) 中的 100 万个向量(每个向量维度为 1536)被存储在 575 个 granule 中,即每个 granule 约 1.7k 行。 -查询请求 10 个近邻,向量相似度索引在 10 个不同的 granule 中找到了这 10 个近邻。 -在查询执行过程中会读取这 10 个 granule。 - -如果输出中包含 `Skip` 以及向量索引的名称和类型(在示例中为 `idx` 和 `vector_similarity`),则表示使用了向量相似度索引。 -在这种情况下,向量相似度索引丢弃了 4 个 granule 中的 2 个,即丢弃了 50% 的数据。 -能够丢弃的 granule 越多,索引的使用就越高效。 - -:::tip -若要强制使用索引,可以在运行 SELECT 查询时设置 [force_data_skipping_indexes](../../../operations/settings/settings#force_data_skipping_indices)(将索引名称作为该设置的值提供)。 -::: - -**后过滤与预过滤** - -用户可以选择在 SELECT 查询中通过 `WHERE` 子句指定额外的过滤条件。 -ClickHouse 将使用后过滤或预过滤策略来评估这些过滤条件。 -简而言之,这两种策略决定了过滤条件的执行顺序: - -* 后过滤表示首先评估向量相似度索引,然后 ClickHouse 再评估 `WHERE` 子句中指定的额外过滤条件。 -* 预过滤表示过滤条件的评估顺序相反。 - -这两种策略有不同的权衡: - -* 后过滤的一个普遍问题在于,它可能返回少于 `LIMIT ` 子句所请求的行数。当由向量相似度索引返回的一个或多个结果行无法满足附加过滤条件时,就会出现这种情况。 -* 预过滤在总体上仍是一个未解决的问题。某些专用向量数据库提供了预过滤算法,但大多数关系型数据库(包括 ClickHouse)会退回到精确近邻搜索,即不使用索引的暴力扫描。 - -采用何种策略取决于过滤条件。 - -*附加过滤条件是分区键的一部分* - -如果附加过滤条件是分区键的一部分,那么 ClickHouse 将应用分区裁剪。 -例如,某个表按列 `year` 进行范围分区,并运行如下查询: - -```sql -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -WHERE year = 2025 -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -ClickHouse 将裁剪除 2025 分区外的所有分区。 - -*无法使用索引计算的额外过滤条件* - -如果额外过滤条件无法通过索引(主键索引、跳跃索引)进行计算,ClickHouse 将在扫描结果上执行后置过滤。 - - -*可以使用主键索引评估的附加过滤条件* - -如果附加过滤条件可以使用[主键](mergetree.md#primary-key)进行评估(即它们构成主键的前缀),并且 - -* 如果过滤条件在一个 part 内剔除了至少一行,则 ClickHouse 将改为对该 part 内“存活”的范围执行预过滤, -* 如果过滤条件在一个 part 内没有剔除任何行,则 ClickHouse 将对该 part 执行后过滤。 - -在实际使用场景中,后一种情况相对不太常见。 - -*可以使用 skipping index 评估的附加过滤条件* - -如果附加过滤条件可以使用[数据跳过索引(skipping indexes)](mergetree.md#table_engine-mergetree-data_skipping-indexes)(minmax 索引、set 索引等)进行评估,ClickHouse 会执行后过滤。 -在这种情况下,会首先评估向量相似度索引,因为预期它相对于其他 skipping indexes 能剔除最多的行。 - -为了对后过滤与预过滤进行更精细的控制,可以使用两个设置项: - -[vector_search_filter_strategy](../../../operations/settings/settings#vector_search_filter_strategy) 设置(默认值:`auto`,实现了上述启发式策略)可以设为 `prefilter`。 -在附加过滤条件极具选择性时,这对于强制启用预过滤非常有用。 -例如,下面的查询可能会从预过滤中获益: - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('古代亚洲帝国相关书籍')) -LIMIT 10 -``` - -假设只有极少数书籍的价格低于 2 美元,后过滤(post-filtering)可能会返回零行,因为向量索引返回的前 10 个匹配结果的价格可能全部高于 2 美元。 -通过强制使用预过滤(在查询中添加 `SETTINGS vector_search_filter_strategy = 'prefilter'`),ClickHouse 会先找到所有价格低于 2 美元的书籍,然后对这些书籍执行穷举(brute-force)向量搜索。 - -作为解决上述问题的另一种方法,可以将 [vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier)(默认值:`1.0`,最大值:`1000.0`)配置为大于 `1.0` 的值(例如 `2.0`)。 -从向量索引中获取的最近邻数量会乘以该设置的值,然后再对这些行应用额外过滤条件,以返回满足 LIMIT 的行数。 -例如,我们可以再次进行查询,但将 multiplier 设置为 `3.0`: - -```sql -SELECT bookid, author, title -FROM books -WHERE price < 2.00 -ORDER BY cosineDistance(book_vector, getEmbedding('古代亚洲帝国相关书籍')) -LIMIT 10 -SETTING vector_search_index_fetch_multiplier = 3.0; -``` - -ClickHouse 将在每个 part 中的向量索引中获取 3.0 x 10 = 30 个最近邻,然后再评估额外的过滤条件。 -最终只会返回距离最近的 10 个邻居。 -需要注意的是,通过设置 `vector_search_index_fetch_multiplier` 可以缓解这个问题,但在极端情况下(例如 WHERE 条件选择性很强时),仍然可能出现返回的行数少于请求的 N 行的情况。 - -**重新打分(Rescoring)** - -ClickHouse 中的 skip index 通常在 granule 级别进行过滤,即对 skip index 的一次查找(在内部)会返回一个潜在匹配 granule 的列表,从而减少后续扫描中需要读取的数据量。 -这对一般的 skip index 效果很好,但在向量相似度索引的场景中,会造成一个“粒度不匹配(granularity mismatch)”的问题。 -更具体地说,向量相似度索引会为给定的参考向量确定 N 个最相似向量的行号,但接下来需要将这些行号外推为 granule 编号。 -ClickHouse 随后会从磁盘加载这些 granule,并对这些 granule 中的所有向量重新计算距离。 -这一步称为重新打分(rescoring),虽然从理论上讲它可以提升准确性——请记住,向量相似度索引只返回*近似*结果——但从性能角度看显然并不理想。 - -因此,ClickHouse 提供了一项优化:禁用重新打分,直接从索引中返回最相似的向量及其距离。 -该优化默认启用,参见设置 [vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring)。 -在高层级上的工作方式是:ClickHouse 将最相似的向量及其距离作为一个虚拟列 `_distances` 暴露出来。 -要查看这一点,可运行带有 `EXPLAIN header = 1` 的向量搜索查询: - - -```sql -EXPLAIN header = 1 -WITH [0., 2.] AS reference_vec -SELECT id -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3 -SETTINGS vector_search_with_rescoring = 0 -``` - -```result -Query id: a2a9d0c8-a525-45c1-96ca-c5a11fa66f47 - - ┌─explain─────────────────────────────────────────────────────────────────────────────────────────────────┐ - 1. │ Expression (投影列名) │ - 2. │ Header: id Int32 │ - 3. │ Limit (初步 LIMIT(无 OFFSET)) │ - 4. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 5. │ __table1.id Int32 │ - 6. │ Sorting (ORDER BY 排序) │ - 7. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ - 8. │ __table1.id Int32 │ - 9. │ Expression ((ORDER BY 之前 + (投影 + 列名转换为列标识符))) │ -10. │ Header: L2Distance(__table1.vec, _CAST([0., 2.]_Array(Float64), 'Array(Float64)'_String)) Float64 │ -11. │ __table1.id Int32 │ -12. │ ReadFromMergeTree (default.tab) │ -13. │ Header: id Int32 │ -14. │ _distance Float32 │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -:::note -在未启用重打分(`vector_search_with_rescoring = 0`)且启用了并行副本的情况下运行的查询,可能会回退为执行重打分。 -::: - -#### 性能调优 {#performance-tuning} - -**压缩调优** - -在几乎所有使用场景中,底层列中的向量都是稠密的,且压缩效果不佳。 -因此,[压缩](/sql-reference/statements/create/table.md#column_compression_codec) 会降低向量列的写入和读取性能。 -因此我们建议禁用压缩。 -为此,请像下面这样为向量列指定 `CODEC(NONE)`: - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32) CODEC(NONE), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; -``` - -**调优索引创建** - -向量相似度索引的生命周期与数据分片(part)的生命周期绑定。 -换句话说,每当创建一个定义了向量相似度索引的新分片时,索引也会随之创建。 -这通常发生在数据被[插入](https://clickhouse.com/docs/guides/inserting-data)时或在[合并](https://clickhouse.com/docs/merges)过程中。 -众所周知,HNSW 的索引创建耗时较长,会显著拖慢插入和合并操作。 -向量相似度索引在理想情况下只应用于不可变或很少变更的数据。 - -为了加速索引创建,可以采用以下技术: - -首先,可以并行化索引创建过程。 -索引创建线程的最大数量可以通过服务器设置 [max_build_vector_similarity_index_thread_pool_size](/operations/server-configuration-parameters/settings#max_build_vector_similarity_index_thread_pool_size) 进行配置。 -为获得最佳性能,该设置值应配置为 CPU 核心数。 - -其次,为了加速 INSERT 语句,用户可以通过会话设置 [materialize_skip_indexes_on_insert](../../../operations/settings/settings.md#materialize_skip_indexes_on_insert) 禁用在新插入分片上创建跳过索引(skipping index)。 -对此类分片执行的 SELECT 查询将回退为精确搜索。 -由于插入分片相对于整个表的大小通常较小,因此这种回退带来的性能影响预计可以忽略不计。 - -第三,为了加速合并,用户可以通过会话设置 [materialize_skip_indexes_on_merge](../../../operations/settings/merge-tree-settings.md#materialize_skip_indexes_on_merge) 禁用在合并后的分片上创建跳过索引。 -这与语句 [ALTER TABLE [...] MATERIALIZE INDEX [...]](../../../sql-reference/statements/alter/skipping-index.md#materialize-index) 结合使用,可以对向量相似度索引的生命周期进行显式控制。 -例如,可以将索引创建延后到所有数据都已摄取完成之后,或延后到系统负载较低的时间段(例如周末)。 - -**调优索引用法** - - -为了执行 `SELECT` 查询并使用向量相似度索引,需要先将这些索引加载到主内存中。 -为避免同一个向量相似度索引被反复加载到主内存,ClickHouse 提供了专用的内存缓存来存储此类索引。 -该缓存越大,不必要的加载就越少。 -最大缓存大小可以通过服务器设置 [vector_similarity_index_cache_size](../../../operations/server-configuration-parameters/settings.md#vector_similarity_index_cache_size) 进行配置。 -默认情况下,缓存最大可增长到 5 GB。 - -:::note -向量相似度索引缓存存储的是向量索引的 granule(粒度单元)。 -如果单个向量索引 granule 的大小超过缓存大小,则不会被缓存。 -因此,请务必根据 “Estimating storage and memory consumption” 中的公式或 [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) 计算向量索引大小,并据此合理设置缓存大小。 -::: - -当前向量相似度索引缓存的大小可以在 [system.metrics](../../../operations/system-tables/metrics.md) 中查看: - -```sql -SELECT metric, value -FROM system.metrics -WHERE metric = 'VectorSimilarityIndexCacheBytes' -``` - -可以从 [system.query_log](../../../operations/system-tables/query_log.md) 中获取某个查询 ID 对应查询的缓存命中和未命中情况: - -```sql -SYSTEM FLUSH LOGS query_log; - -SELECT ProfileEvents['VectorSimilarityIndexCacheHits'], ProfileEvents['VectorSimilarityIndexCacheMisses'] -FROM system.query_log -WHERE type = 'QueryFinish' AND query_id = '<...>' -ORDER BY event_time_microseconds; -``` - -对于生产环境的使用场景,我们建议将缓存配置得足够大,使所有向量索引始终都能常驻内存。 - -**量化调优** - -[量化](https://huggingface.co/blog/embedding-quantization) 是一种用于减少向量内存占用,以及降低构建和遍历向量索引计算成本的技术。 -ClickHouse 向量索引支持以下量化选项: - -| Quantization | Name | Storage per dimension | -| -------------- | ---------------------------- | --------------------- | -| f32 | Single precision | 4 bytes | -| f16 | Half precision | 2 bytes | -| bf16 (default) | Half precision (brain float) | 2 bytes | -| i8 | Quarter precision | 1 byte | -| b1 | Binary | 1 bit | - -与直接搜索原始全精度浮点值(`f32`)相比,引入量化会降低向量搜索的精度。 -不过,在大多数数据集上,半精度 brain float 量化(`bf16`)带来的精度损失可以忽略不计,因此向量相似度索引默认采用这种量化技术。 -四分之一精度(`i8`)和二进制(`b1`)量化会在向量搜索中引入较为明显的精度损失。 -我们仅在向量相似度索引的大小显著大于可用 DRAM 容量时,才推荐使用这两种量化方式。 -在这种情况下,我们也建议启用重评分([vector_search_index_fetch_multiplier](../../../operations/settings/settings#vector_search_index_fetch_multiplier)、[vector_search_with_rescoring](../../../operations/settings/settings#vector_search_with_rescoring))以提升准确性。 -二进制量化仅在以下两种情况下推荐使用:1)对归一化后的嵌入向量(即向量长度 = 1,OpenAI 模型通常是归一化的),以及 2)使用余弦距离作为距离函数时。 -二进制量化在内部使用 Hamming 距离来构建和搜索近邻图。 -重评分步骤会使用表中存储的原始全精度向量,通过余弦距离来识别最近邻。 - -**数据传输调优** - -向量搜索查询中的参考向量由用户提供,通常是通过调用大型语言模型(LLM)获取。 -在 ClickHouse 中运行向量搜索的典型 Python 代码如下所示: - -```python -search_v = openai_client.embeddings.create(input = "[好书]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'search_v': search_v} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, %(search_v)s) - LIMIT 10", - parameters = params) -``` - - -嵌入向量(上面代码片段中的 `search_v`)的维度可能非常大。 -例如,OpenAI 提供的模型会生成维度为 1536 甚至 3072 的嵌入向量。 -在上面的代码中,ClickHouse Python 驱动会将嵌入向量替换为一个可读的字符串,然后将整个 SELECT 查询作为字符串发送。 -假设嵌入向量由 1536 个单精度浮点值组成,发送的字符串长度将达到 20 kB。 -这会在分词、解析以及执行数千次字符串到浮点数转换时造成较高的 CPU 使用率。 -此外,ClickHouse 服务器日志文件也需要大量空间,进而导致 `system.query_log` 膨胀。 - -请注意,大多数 LLM 模型返回的嵌入向量是由原生浮点数组成的列表或 NumPy 数组。 -因此,我们建议 Python 应用以二进制形式绑定参考向量参数,使用如下方式: - -```python -search_v = openai_client.embeddings.create(input = "[好书]", model='text-embedding-3-large', dimensions=1536).data[0].embedding - -params = {'$search_v_binary$': np.array(search_v, dtype=np.float32).tobytes()} -result = chclient.query( - "SELECT id FROM items - ORDER BY cosineDistance(vector, (SELECT reinterpret($search_v_binary$, 'Array(Float32)'))) - LIMIT 10" - parameters = params) -``` - -在本示例中,参考向量以原始二进制形式发送到服务器,并在服务器端被重新解释为浮点数数组。 -这可以节省服务器端的 CPU 时间,并避免服务器日志和 `system.query_log` 的膨胀。 - -#### 管理和监控 {#administration} - -向量相似性索引在磁盘上的大小可以通过 [system.data_skipping_indices](../../../operations/system-tables/data_skipping_indices) 获取: - -```sql -SELECT database, table, name, formatReadableSize(data_compressed_bytes) -FROM system.data_skipping_indices -WHERE type = 'vector_similarity'; -``` - -示例输出: - -```result -┌─database─┬─table─┬─name─┬─formatReadab⋯ssed_bytes)─┐ -│ default │ tab │ idx │ 348.00 MB │ -└──────────┴───────┴──────┴──────────────────────────┘ -``` - -#### 与常规跳过索引的区别 {#differences-to-regular-skipping-indexes} - -与所有常规[跳过索引](/optimize/skipping-indexes)类似,向量相似度索引也是在 granule 之上构建的,每个已建立索引的块由 `GRANULARITY = [N]` 个 granule 组成(对普通跳过索引而言,`[N]` 默认为 1)。 -例如,如果表的主索引粒度为 8192(设置 `index_granularity = 8192`)且 `GRANULARITY = 2`,则每个已建立索引的块将包含 16384 行。 -然而,用于近似最近邻搜索的数据结构和算法在本质上是面向行的。 -它们存储一组行的紧凑表示,并且在向量搜索查询中也会返回行。 -这导致向量相似度索引在行为方式上与普通跳过索引相比存在一些相当不直观的差异。 - -当用户在某列上定义向量相似度索引时,ClickHouse 会在内部为每个索引块创建一个向量相似度“子索引”。 -子索引是“本地”的,这意味着它只涉及其所属索引块中的行。 -延续前面的例子,并假设某列有 65536 行,我们会得到四个索引块(跨越八个 granule),以及每个索引块对应的一个向量相似度子索引。 -理论上,子索引能够直接返回其索引块内距离给定点最近的 N 个点所对应的行。 -然而,由于 ClickHouse 从磁盘加载数据到内存时的粒度是 granule,子索引会将匹配行扩展到 granule 粒度。 -这与常规跳过索引不同,后者是以索引块粒度来跳过数据的。 - - -`GRANULARITY` 参数决定会创建多少个向量相似度子索引。 -更大的 `GRANULARITY` 值意味着子索引更少但更大,极端情况下,一个列(或列的数据分片)只会有单个子索引。 -在这种情况下,该子索引对该列的所有行具有“全局”视图,并且可以直接返回该列(分片)中所有包含相关行的 granule(最多只有 `LIMIT [N]` 个这样的 granule)。 -在第二步中,ClickHouse 会加载这些 granule,并通过对这些 granule 中的所有行执行穷举式距离计算来识别真正最优的行。 -当 `GRANULARITY` 值较小时,每个子索引会返回最多 `LIMIT N` 个 granule。 -结果是需要加载更多 granule 并进行后置过滤。 -请注意,两种情况下的搜索精度同样高,只是处理性能不同。 -通常建议为向量相似度索引使用较大的 `GRANULARITY`,只有在出现例如向量相似度结构内存占用过高等问题时,才改用较小的 `GRANULARITY` 值。 -如果没有为向量相似度索引指定 `GRANULARITY`,则默认值为 1 亿。 - -#### 示例 {#approximate-nearest-neighbor-search-example} - -```sql -CREATE TABLE tab(id Int32, vec Array(Float32), INDEX idx vec TYPE vector_similarity('hnsw', 'L2Distance', 2)) ENGINE = MergeTree ORDER BY id; - -INSERT INTO tab VALUES (0, [1.0, 0.0]), (1, [1.1, 0.0]), (2, [1.2, 0.0]), (3, [1.3, 0.0]), (4, [1.4, 0.0]), (5, [1.5, 0.0]), (6, [0.0, 2.0]), (7, [0.0, 2.1]), (8, [0.0, 2.2]), (9, [0.0, 2.3]), (10, [0.0, 2.4]), (11, [0.0, 2.5]); - -WITH [0., 2.] AS reference_vec -SELECT id, vec -FROM tab -ORDER BY L2Distance(vec, reference_vec) ASC -LIMIT 3; -``` - -返回值 - -```result - ┌─id─┬─vec─────┐ -1. │ 6 │ [0,2] │ -2. │ 7 │ [0,2.1] │ -3. │ 8 │ [0,2.2] │ - └────┴─────────┘ -``` - -使用近似向量搜索的更多示例数据集: - -* [LAION-400M](../../../getting-started/example-datasets/laion-400m-dataset) -* [LAION-5B](../../../getting-started/example-datasets/laion-5b-dataset) -* [dbpedia](../../../getting-started/example-datasets/dbpedia-dataset) -* [hackernews](../../../getting-started/example-datasets/hackernews-vector-search-dataset) - -### 量化比特(QBit) {#approximate-nearest-neighbor-search-qbit} - - - -加速精确向量搜索的一种常见方法是使用更低精度的 [浮点数数据类型](../../../sql-reference/data-types/float.md)。 -例如,如果向量存储为 `Array(BFloat16)` 而不是 `Array(Float32)`,数据大小会减半,并且查询运行时间预计会按比例缩短。 -这种方法称为量化。尽管它加快了计算速度,但即便对所有向量进行穷举扫描,结果的准确性也可能会降低。 - -在传统量化中,我们在搜索和存储数据时都会丢失精度。在上述示例中,我们会存储 `BFloat16` 而不是 `Float32`,这意味着即使之后有需求,我们也永远无法执行更高精度的搜索。另一种方法是存储两份数据:量化版本和全精度版本。尽管这种方式可行,但需要冗余存储。设想一种场景,我们的原始数据是 `Float64`,并希望以不同精度(16 位、32 位或完整的 64 位)运行搜索,那么就需要存储三份独立的数据副本。 - -ClickHouse 提供了 Quantized Bit(`QBit`)数据类型,通过以下方式克服这些限制: - -1. 存储原始的全精度数据。 -2. 允许在查询时指定量化精度。 - - -这是通过以按位分组(bit-grouped)格式存储数据实现的(即将所有向量的第 i 位比特存放在一起),从而只在请求的精度级别进行读取。这样,你可以在保留全部原始数据、按需访问的前提下,从量化带来的 I/O 和计算量减少中获得速度优势。当选择最大精度时,搜索结果即为精确匹配。 - -:::note -`QBit` 数据类型及其相关距离函数目前是实验特性。要启用它们,请运行 `SET allow_experimental_qbit_type = 1`。 -如果遇到问题,请在 [ClickHouse 代码仓库](https://github.com/clickhouse/clickhouse/issues) 中提交 issue。 -::: - -要声明一个 `QBit` 类型的列,请使用以下语法: - -```sql -列名 QBit(元素类型, 维度) -``` - -其中: - -* `element_type` – 每个向量元素的类型。支持的类型包括 `BFloat16`、`Float32` 和 `Float64` -* `dimension` – 每个向量中的元素个数 - -#### 创建 `QBit` 表并添加数据 {#qbit-create} - -```sql -CREATE TABLE fruit_animal ( - word String, - vec QBit(Float64, 5) -) ENGINE = MergeTree -ORDER BY word; - -INSERT INTO fruit_animal VALUES - ('apple', [-0.99105519, 1.28887844, -0.43526649, -0.98520696, 0.66154391]), - ('banana', [-0.69372815, 0.25587061, -0.88226235, -2.54593015, 0.05300475]), - ('orange', [0.93338752, 2.06571317, -0.54612565, -1.51625717, 0.69775337]), - ('dog', [0.72138876, 1.55757105, 2.10953259, -0.33961248, -0.62217325]), - ('cat', [-0.56611276, 0.52267331, 1.27839863, -0.59809804, -1.26721048]), - ('horse', [-0.61435682, 0.48542571, 1.21091247, -0.62530446, -1.33082533]); -``` - -#### 使用 `QBit` 进行向量搜索 {#qbit-search} - -我们使用 L2 距离查找与表示单词 “lemon” 的向量最接近的邻居向量。距离函数的第三个参数指定精度的位数——值越高,精度越高,但计算量也越大。 - -你可以在[这里](../../../sql-reference/data-types/qbit.md#vector-search-functions)找到 `QBit` 支持的所有距离函数。 - -**全精度搜索(64 位):** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 64) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬────────────distance─┐ -1. │ apple │ 0.14639757188169716 │ -2. │ banana │ 1.998961369007679 │ -3. │ orange │ 2.039041552613732 │ -4. │ cat │ 2.752802631487914 │ -5. │ horse │ 2.7555776805484813 │ -6. │ dog │ 3.382295083120104 │ - └────────┴─────────────────────┘ -``` - -**低精度搜索:** - -```sql -SELECT - word, - L2DistanceTransposed(vec, [-0.88693672, 1.31532824, -0.51182908, -0.99652702, 0.59907770], 12) AS distance -FROM fruit_animal -ORDER BY distance; -``` - -```text - ┌─word───┬───────────distance─┐ -1. │ apple │ 0.757668703053566 │ -2. │ orange │ 1.5499475034938677 │ -3. │ banana │ 1.6168396735102937 │ -4. │ cat │ 2.429752230904804 │ -5. │ horse │ 2.524650475528617 │ -6. │ dog │ 3.17766975527459 │ - └────────┴────────────────────┘ -``` - - -请注意,使用 12 位量化时,我们在加快查询执行的同时,依然能够很好地逼近实际距离。相对排序基本保持一致,`apple` 仍然是距离最近的匹配项。 - -:::note -在目前的状态下,加速主要来自减少 I/O,因为我们读取的数据更少。如果原始数据比较“宽”,例如 `Float64`,选择更低的精度时,距离计算依然会在相同宽度的数据上进行——只是精度更低。 -::: - -#### 性能考量 {#qbit-performance} - -`QBit` 的性能收益主要来源于 I/O 操作的减少:在使用较低精度时,需要从存储中读取的数据量更少。此外,当 `QBit` 中包含 `Float32` 数据且精度参数为 16 或更低时,还可以通过减少计算获得额外收益。精度参数直接控制准确性与速度之间的权衡: - -- **更高的精度**(更接近原始数据宽度):结果更准确,查询更慢 -- **更低的精度**:查询更快但结果为近似值,内存占用更低 - -### 参考资料 {#references} - -博客: -- [Vector Search with ClickHouse - Part 1](https://clickhouse.com/blog/vector-search-clickhouse-p1) -- [Vector Search with ClickHouse - Part 2](https://clickhouse.com/blog/vector-search-clickhouse-p2) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md deleted file mode 100644 index 5217eb20dc4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/coalescingmergetree.md +++ /dev/null @@ -1,137 +0,0 @@ ---- -description: 'CoalescingMergeTree 继承自 MergeTree 引擎。其关键特性是在数据部分(part)合并期间,能够自动保留每列最后一个非空(non-null)值。' -sidebar_label: 'CoalescingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/coalescingmergetree -title: 'CoalescingMergeTree 表引擎' -keywords: ['CoalescingMergeTree'] -show_related_blogs: true -doc_type: 'reference' ---- - -# CoalescingMergeTree 表引擎 {#coalescingmergetree-table-engine} - -:::note Available from version 25.6 -此表引擎从 25.6 及更高版本开始在 OSS 和 Cloud 中可用。 -::: - -该引擎继承自 [MergeTree](/engines/table-engines/mergetree-family/mergetree)。关键区别在于数据部分的合并方式:对于 `CoalescingMergeTree` 表,ClickHouse 会将所有具有相同主键(更准确地说,相同的[排序键](../../../engines/table-engines/mergetree-family/mergetree.md))的行合并为一行,该行在每一列上都包含最新的非 NULL 值。 - -这实现了列级别的 upsert(插入或更新),也就是说,您可以只更新特定列,而不是整行。 - -`CoalescingMergeTree` 旨在与非键列中的 Nullable 类型配合使用。如果这些列不是 Nullable,其行为与 [ReplacingMergeTree](/engines/table-engines/mergetree-family/replacingmergetree) 相同。 - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = CoalescingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -有关请求参数的说明,请参阅[请求描述](../../../sql-reference/statements/create/table.md)。 - -### CoalescingMergeTree 的参数 {#parameters-of-coalescingmergetree} - -#### 列 {#columns} - -`columns` - 一个包含需要合并其值的列名的元组(tuple)。可选参数。 -这些列必须是数值类型,并且不能出现在分区键或排序键中。 - -如果未指定 `columns`,ClickHouse 会合并所有不在排序键中的列的值。 - -### 查询子句 {#query-clauses} - -在创建 `CoalescingMergeTree` 表时,所需的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)与创建 `MergeTree` 表时相同。 - -
- 已弃用的建表方法 - - :::note - 不要在新项目中使用此方法,并尽可能将旧项目切换到上面描述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] CoalescingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - 除 `columns` 之外的所有参数与 `MergeTree` 中的含义相同。 - - * `columns` — 一个包含列名的元组(tuple),这些列的值将被求和。可选参数。相关说明见上文。 -
- -## 使用示例 {#usage-example} - -请看下表: - -```sql -CREATE TABLE test_table -( - key UInt64, - value_int Nullable(UInt32), - value_string Nullable(String), - value_date Nullable(Date) -) -ENGINE = CoalescingMergeTree() -ORDER BY key -``` - -向其中插入数据: - -```sql -INSERT INTO test_table VALUES(1, NULL, NULL, '2025-01-01'), (2, 10, 'test', NULL); -INSERT INTO test_table VALUES(1, 42, 'win', '2025-02-01'); -INSERT INTO test_table(key, value_date) VALUES(2, '2025-02-01'); -``` - -结果将如下: - -```sql -SELECT * FROM test_table ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 1 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-01-01 │ -│ 2 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2025-02-01 │ -│ 2 │ 10 │ test │ ᴺᵁᴸᴸ │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -获取最终正确结果的推荐查询: - -```sql -SELECT * FROM test_table FINAL ORDER BY key; -``` - -```text -┌─key─┬─value_int─┬─value_string─┬─value_date─┐ -│ 1 │ 42 │ win │ 2025-02-01 │ -│ 2 │ 10 │ test │ 2025-02-01 │ -└─────┴───────────┴──────────────┴────────────┘ -``` - -在查询中使用 `FINAL` 修饰符会强制 ClickHouse 在查询阶段应用合并逻辑,确保能够为每一列得到正确、已合并的“最新”值。在从 CoalescingMergeTree 表查询时,这是最安全且最精确的方法。 - -:::note - -如果底层数据分片(parts)尚未完全合并,使用 `GROUP BY` 的方式可能会返回不正确的结果。 - -```sql -SELECT key, last_value(value_int), last_value(value_string), last_value(value_date) FROM test_table GROUP BY key; -- 不建议使用。 -``` - -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md deleted file mode 100644 index bab22872ef8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/collapsingmergetree.md +++ /dev/null @@ -1,376 +0,0 @@ ---- -description: '继承自 MergeTree,但添加了在合并过程中折叠行的逻辑。' -keywords: ['更新', '折叠'] -sidebar_label: 'CollapsingMergeTree' -sidebar_position: 70 -slug: /engines/table-engines/mergetree-family/collapsingmergetree -title: 'CollapsingMergeTree 表引擎' -doc_type: 'guide' ---- - - - -# CollapsingMergeTree 表引擎 {#collapsingmergetree-table-engine} - - - -## 描述 {#description} - -`CollapsingMergeTree` 引擎继承自 [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md), -并在合并过程中增加了对行进行折叠的逻辑。 -`CollapsingMergeTree` 表引擎会异步删除(折叠) -成对的行,如果排序键(`ORDER BY`)中的所有字段都相同,且仅特殊字段 `Sign` 不同, -并且 `Sign` 字段只能取值 `1` 或 `-1`。 -没有与之构成 `Sign` 取值相反配对的行会被保留。 - -更多详细信息,参见本文档的 [Collapsing](#table_engine-collapsingmergetree-collapsing) 部分。 - -:::note -此引擎可以显著减少存储空间占用, -从而提高 `SELECT` 查询的效率。 -::: - - - -## 参数 {#parameters} - -此表引擎的所有参数(`Sign` 参数除外)与 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 中的含义相同。 - -- `Sign` — 行类型标记列的名称,其中 `1` 表示“状态”行,`-1` 表示“撤销”行。类型:[Int8](/sql-reference/data-types/int-uint)。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) -ENGINE = CollapsingMergeTree(Sign) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -
- 已弃用的建表方法 - - :::note - 以下方法不建议在新项目中使用。 - 如果可能,建议将旧项目更新为使用新方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) - ENGINE [=] CollapsingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, Sign) - ``` - - `Sign` — 分配给某列的名称,该列用于表示行的类型,其中 `1` 表示“state”行,`-1` 表示“cancel”行。[Int8](/sql-reference/data-types/int-uint)。 -
- -* 有关查询参数的说明,请参阅[查询说明](../../../sql-reference/statements/create/table.md)。 -* 创建 `CollapsingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[查询子句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。 - - -## 折叠 {#table_engine-collapsingmergetree-collapsing} - -### 数据 {#data} - -考虑这样一种情况:你需要为某个给定对象保存持续变化的数据。 -看起来为每个对象只保留一行并在有变化时更新它似乎是合乎逻辑的, -然而,更新操作对数据库管理系统(DBMS)来说代价高且缓慢,因为它们需要在存储中重写数据。 -如果我们需要快速写入数据,执行大量更新操作并不是可接受的方法, -但我们始终可以按顺序写入某个对象的变更。 -为此,我们使用特殊列 `Sign`。 - -* 如果 `Sign` = `1`,表示该行是一个“状态(state)”行:*一行包含表示当前有效状态的字段*。 -* 如果 `Sign` = `-1`,表示该行是一个“撤销(cancel)”行:*一行用于撤销具有相同属性的对象状态*。 - -例如,我们希望统计用户在某个网站上查看了多少页面以及访问这些页面的时长。 -在某个给定时间点,我们写入如下记录用户活动状态的一行数据: - -```text -┌──────────────UserID─┬─页面浏览量─┬─持续时长─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -随后,当我们检测到用户活动发生变化时,会使用以下两行将其写入表中: - -```text -┌──────────────用户ID─┬─页面浏览数─┬─时长─┬─符号─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -第一行会取消该对象之前的状态(在本例中表示一个用户)。 -对于该“已取消”行,应复制所有排序键字段,`Sign` 字段除外。 -上面的第二行表示当前状态。 - -由于我们只需要用户活动的最终状态,原始的 “state” 行和我们插入的 “cancel” -行可以像下方所示那样删除,从而折叠对象的无效(旧)状态: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -- 旧的 "state" 行可以删除 -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -- "cancel" 行可以删除 -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -- 新的 "state" 行会保留 -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -`CollapsingMergeTree` 在合并数据分片时,会执行这种*折叠*行为。 - -:::note -关于为什么每次更改需要两行的原因, -将在[算法](#table_engine-collapsingmergetree-collapsing-algorithm)一节中进一步讨论。 -::: - -**这种方法的特点** - -1. 写入数据的程序必须记住对象的状态,才能在需要时将其取消。“cancel” 行应包含与 “state” 行相同的排序键字段副本,以及相反的 `Sign`。这会增加初始存储占用,但可以让我们快速写入数据。 -2. 列中不断增长的长数组会因为写入负载增加而降低引擎效率。数据越简单,效率越高。 -3. `SELECT` 的结果高度依赖于对象变更历史的一致性。在准备要插入的数据时要谨慎。对于不一致的数据,可能会得到不可预测的结果。例如,本应非负的指标(如会话深度)出现负值。 - -### 算法 {#table_engine-collapsingmergetree-collapsing-algorithm} - -当 ClickHouse 合并数据[分片](/concepts/glossary#parts)时, -每组具有相同排序键(`ORDER BY`)的连续行会被折叠为最多两行, -一行 `Sign` = `1` 的 “state” 行和一行 `Sign` = `-1` 的 “cancel” 行。 -换言之,ClickHouse 会对这些记录进行折叠处理。 - - -对于每个生成的数据部分,ClickHouse 会保存: - -| | | -|--|-------------------------------------------------------------------------------------------------------------------------------------| -|1.| 如果 `"state"` 行和 `"cancel"` 行的数量相同,且最后一行为 `"state"` 行,则保存第一条 `"cancel"` 行和最后一条 `"state"` 行。 | -|2.| 如果 `"state"` 行多于 `"cancel"` 行,则保存最后一条 `"state"` 行。 | -|3.| 如果 `"cancel"` 行多于 `"state"` 行,则保存第一条 `"cancel"` 行。 | -|4.| 在所有其他情况下,不保存任何行。 | - -另外,当 `"state"` 行比 `"cancel"` 行至少多 2 行,或者 `"cancel"` 行比 `"state"` 行至少多 2 行时,合并会继续进行。 -不过,ClickHouse 会将这种情况视为逻辑错误,并将其记录到服务器日志中。 -如果相同的数据被多次插入,就有可能出现此错误。 -因此,折叠不应改变统计结果。 -变更会被逐步折叠,最终几乎只保留每个对象的最后状态。 - -需要 `Sign` 列,是因为合并算法不能保证具有相同排序键的所有行都处于同一个结果数据部分中,甚至不在同一台物理服务器上。 -ClickHouse 使用多个线程处理 `SELECT` 查询,因此无法预测结果中行的顺序。 - -如果需要从 `CollapsingMergeTree` 表中获取完全“折叠”的数据,则需要进行聚合。 -要完成折叠,请编写带有 `GROUP BY` 子句的查询,并使用会考虑 `Sign` 值的聚合函数。 -例如,要计算数量,应使用 `sum(Sign)` 而不是 `count()`。 -要计算某个量的总和,应使用 `sum(Sign * x)` 并配合 `HAVING sum(Sign) > 0`,而不是像下面[示例](#example-of-use)中那样使用 `sum(x)`。 - -聚合函数 `count`、`sum` 和 `avg` 可以用这种方式计算。 -如果对象至少有一个未折叠的状态,则可以计算聚合函数 `uniq`。 -聚合函数 `min` 和 `max` 无法计算, -因为 `CollapsingMergeTree` 不保存已折叠状态的历史。 - -:::note -如果需要在不进行聚合的情况下提取数据 -(例如,检查其最新值满足某些条件的行是否存在), -可以在 `FROM` 子句中使用 [`FINAL`](../../../sql-reference/statements/select/from.md#final-modifier) 修饰符。它会在返回结果之前合并数据。 -对于 CollapsingMergeTree,每个键只返回最新的状态行。 -::: - - - -## 示例 {#examples} - -### 使用示例 {#example-of-use} - -给出以下示例数据: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -接下来使用 `CollapsingMergeTree` 创建一张名为 `UAct` 的表: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -接下来我们将插入一些数据: - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1),(4324182021466249494, 6, 185, 1) -``` - -我们使用两个 `INSERT` 查询来创建两个不同的数据片段。 - -:::note -如果我们使用单个查询插入数据,ClickHouse 只会创建一个数据片段,并且之后不会执行任何合并操作。 -::: - -我们可以使用以下方式来查询数据: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -让我们来看一下上面返回的数据,检查是否发生了折叠(collapsing)…… -通过两条 `INSERT` 语句,我们创建了两个数据 part。 -`SELECT` 语句在两个线程中执行,因此得到的行顺序是随机的。 -然而,折叠**并没有发生**,因为这些数据 part 尚未被合并, -而 ClickHouse 会在后台的某个未知时间点合并数据 part,这一点是无法预测的。 - -因此,我们需要进行一次聚合, -可以使用 [`sum`](/sql-reference/aggregate-functions/reference/sum) -聚合函数,并配合 [`HAVING`](/sql-reference/statements/select/having) 子句: - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration -FROM UAct -GROUP BY UserID -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─页面浏览量─┬─停留时长─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -如果我们不需要聚合并且想要强制折叠,还可以在 `FROM` 子句中使用 `FINAL` 修饰符。 - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -:::note -这种数据选取方式效率较低,不建议在扫描数据量很大(数百万行)时使用。 -::: - -### 另一种方法示例 {#example-of-another-approach} - -这种方法的思路是,合并操作只考虑键字段。 -因此,在 "cancel" 行中,我们可以指定负值, -使其在不使用 `Sign` 列进行求和时抵消该行的先前版本。 - -在本示例中,我们将使用下面的示例数据: - - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ -│ 4324182021466249494 │ -5 │ -146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -对于这种方法,需要将 `PageViews` 和 `Duration` 的数据类型更改为可以存储负值的类型。 -因此,在使用 `collapsingMergeTree` 创建表 `UAct` 时,我们将这些列的数据类型从 `UInt8` 更改为 `Int16`: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews Int16, - Duration Int16, - Sign Int8 -) -ENGINE = CollapsingMergeTree(Sign) -ORDER BY UserID -``` - -让我们通过向表中插入数据来测试此方法。 - -不过,对于示例或小型表,这样做是可以接受的: - -```sql -INSERT INTO UAct VALUES(4324182021466249494, 5, 146, 1); -INSERT INTO UAct VALUES(4324182021466249494, -5, -146, -1); -INSERT INTO UAct VALUES(4324182021466249494, 6, 185, 1); - -SELECT * FROM UAct FINAL; -``` - -```text -┌──────────────UserID─┬─页面浏览次数─┬─持续时间─┬─符号─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -```sql -SELECT - UserID, - sum(PageViews) AS PageViews, - sum(Duration) AS Duration -FROM UAct -GROUP BY UserID -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┐ -│ 4324182021466249494 │ 6 │ 185 │ -└─────────────────────┴───────────┴──────────┘ -``` - -```sql -SELECT COUNT() FROM UAct -``` - -```text -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -```sql -OPTIMIZE TABLE UAct FINAL; - -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md deleted file mode 100644 index 8f6caa9e637..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/custom-partitioning-key.md +++ /dev/null @@ -1,183 +0,0 @@ ---- -description: '了解如何为 MergeTree 表添加自定义分区键。' -sidebar_label: '自定义分区键' -sidebar_position: 30 -slug: /engines/table-engines/mergetree-family/custom-partitioning-key -title: '自定义分区键' -doc_type: 'guide' ---- - -# 自定义分区键 {#custom-partitioning-key} - -:::note -在大多数情况下,无需使用分区键;在其他大多数情况下,除非是针对按天分区较为常见的可观测性场景,否则也不需要比“按月”更细粒度的分区键。 - -切勿使用过于细粒度的分区。不要按客户端标识符或名称对数据进行分区,而应将客户端标识符或名称设置为 ORDER BY 表达式中的第一列。 -::: - -[MergeTree 系列表](../../../engines/table-engines/mergetree-family/mergetree.md)支持分区,包括[复制表](../../../engines/table-engines/mergetree-family/replication.md)和[物化视图](/sql-reference/statements/create/view#materialized-view)。 - -分区是按指定条件在表中对记录进行的逻辑归组。可以按任意条件设置分区,例如按月、按日或按事件类型。每个分区单独存储,以简化对这部分数据的操作。访问数据时,ClickHouse 会尽可能只访问最小数量的分区。对于包含分区键的查询,分区可以提升性能,因为 ClickHouse 会在选择分区内的 part 和 granule 之前,先根据分区进行过滤。 - -在[创建表](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)时,通过 `PARTITION BY expr` 子句指定分区。分区键可以是基于表列的任意表达式。例如,要按月进行分区,可以使用表达式 `toYYYYMM(date_column)`: - -```sql -CREATE TABLE visits -( - VisitDate Date, - Hour UInt8, - ClientID UUID -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(VisitDate) -ORDER BY Hour; -``` - -分区键也可以是表达式元组(类似于[主键](../../../engines/table-engines/mergetree-family/mergetree.md#primary-keys-and-indexes-in-queries))。例如: - -```sql -ENGINE = ReplicatedCollapsingMergeTree('/clickhouse/tables/name', 'replica1', Sign) -PARTITION BY (toMonday(StartDate), EventType) -ORDER BY (CounterID, StartDate, intHash32(UserID)); -``` - -在本示例中,我们按当前周内发生的事件类型进行分区。 - -默认情况下,不支持使用浮点类型作为分区键。要使用它,请启用设置 [allow_floating_point_partition_key](../../../operations/settings/merge-tree-settings.md#allow_floating_point_partition_key)。 - -当向表中插入新数据时,这些数据会作为一个单独的数据部分(part),按主键排序后存储。在插入后的 10–15 分钟内,同一分区中的各个部分会被合并为一个完整的数据部分。 - -:::info -合并仅适用于分区表达式取值相同的数据部分。这意味着**不应创建过于细粒度的分区**(超过大约一千个分区)。否则,由于文件系统中文件数量和打开的文件描述符数量过多,`SELECT` 查询的性能会很差。 -::: - -使用 [system.parts](../../../operations/system-tables/parts.md) 表可以查看表的数据部分和分区。例如,假设我们有一个按月分区的 `visits` 表。让我们对 `system.parts` 表执行 `SELECT` 查询: - -```sql -SELECT - partition, - name, - active -FROM system.parts -WHERE table = 'visits' -``` - -```text -┌─partition─┬─name──────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1_11 │ 1 │ -│ 201902 │ 201902_10_10_0_11 │ 1 │ -│ 201902 │ 201902_11_11_0_11 │ 1 │ -└───────────┴───────────────────┴────────┘ -``` - -`partition` 列包含分区的名称。在此示例中有两个分区:`201901` 和 `201902`。可以使用该列的值在 [ALTER ... PARTITION](../../../sql-reference/statements/alter/partition.md) 查询中指定分区名称。 - -`name` 列包含分区数据 part 的名称。你可以在 [ALTER ATTACH PART](/sql-reference/statements/alter/partition#attach-partitionpart) 查询中使用该列来指定 part 的名称。 - -下面我们来拆解这个 part 名称:`201901_1_9_2_11`: - -* `201901` 是分区名。 -* `1` 是数据块的最小编号。 -* `9` 是数据块的最大编号。 -* `2` 是 chunk 的层级(其在 MergeTree 中形成时所处的深度)。 -* `11` 是 mutation 版本(如果该 part 发生过 mutation)。 - -:::info -旧类型的表的 part 名称格式为:`20190117_20190123_2_2_0`(最小日期 - 最大日期 - 最小块编号 - 最大块编号 - 层级)。 -::: - -`active` 列表示 part 的状态。`1` 表示活跃(active);`0` 表示非活跃(inactive)。例如,合并为更大 part 后保留下来的源 part 是非活跃的 part。损坏的数据 part 也会被标记为非活跃。 - -如示例所示,同一分区可以包含多个相互独立的 part(例如 `201901_1_3_1` 和 `201901_1_9_2`)。这表示这些 part 尚未被合并。ClickHouse 会周期性地合并已插入的数据 part,大约会在插入后 15 分钟触发一次合并。此外,你可以使用 [OPTIMIZE](../../../sql-reference/statements/optimize.md) 查询执行一次非计划的合并。示例: - -```sql -OPTIMIZE TABLE visits PARTITION 201902; -``` - -```text -┌─partition─┬─name─────────────┬─active─┐ -│ 201901 │ 201901_1_3_1 │ 0 │ -│ 201901 │ 201901_1_9_2_11 │ 1 │ -│ 201901 │ 201901_8_8_0 │ 0 │ -│ 201901 │ 201901_9_9_0 │ 0 │ -│ 201902 │ 201902_4_6_1 │ 0 │ -│ 201902 │ 201902_4_11_2_11 │ 1 │ -│ 201902 │ 201902_10_10_0 │ 0 │ -│ 201902 │ 201902_11_11_0 │ 0 │ -└───────────┴──────────────────┴────────┘ -``` - -非活跃的 parts 将在合并后大约 10 分钟内被删除。 - -查看一组 parts 和 partitions 的另一种方法,是进入该表所在的目录:`/var/lib/clickhouse/data///`。例如: - -```bash -/var/lib/clickhouse/data/default/visits$ ls -l -total 40 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 201901_1_3_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201901_1_9_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_8_8_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 15:52 201901_9_9_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_10_10_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:17 201902_11_11_0 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 16:19 201902_4_11_2_11 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 5 12:09 201902_4_6_1 -drwxr-xr-x 2 clickhouse clickhouse 4096 Feb 1 16:48 detached -``` - -文件夹 '201901_1_1_0'、'201901_1_7_1' 等,是各个 part 的目录。每个 part 属于一个分区,并且只包含某一个月的数据(本示例中的表是按月分区的)。 - -`detached` 目录包含通过 [DETACH](/sql-reference/statements/detach) 查询从表中分离出去的 part。损坏的 part 也会被移动到该目录,而不是直接删除。服务器不会使用 `detached` 目录中的这些 part。你可以在任何时候在该目录中添加、删除或修改数据——在你执行 [ATTACH](/sql-reference/statements/alter/partition#attach-partitionpart) 查询之前,服务器都不会察觉到这些更改。 - -请注意,在正在运行的服务器上,你不能在文件系统中手动更改 part 的集合或其数据,因为服务器不会感知到这些变更。对于非复制表,你可以在服务器停止时进行此操作,但不推荐这样做。对于复制表,在任何情况下都不能更改 part 的集合。 - -ClickHouse 允许你对分区执行操作:删除分区、在表之间复制分区,或者创建备份。所有操作的完整列表请参见 [Manipulations With Partitions and Parts](/sql-reference/statements/alter/partition) 一节。 - -## 使用分区键进行 GROUP BY 优化 {#group-by-optimisation-using-partition-key} - -对于某些表的分区键与查询的 GROUP BY 键的特定组合,可以针对每个分区独立执行聚合。 -这样在最后我们就不必合并所有执行线程产生的部分聚合结果, -因为可以保证相同的 GROUP BY 键值不会同时出现在两个不同线程的工作集里。 - -一个典型示例如下: - -```sql -CREATE TABLE session_log -( - UserID UInt64, - SessionID UUID -) -ENGINE = MergeTree -PARTITION BY sipHash64(UserID) % 16 -ORDER BY tuple(); - -SELECT - UserID, - COUNT() -FROM session_log -GROUP BY UserID; -``` - -:::note -此类查询的性能在很大程度上取决于表结构。因此,该优化默认未启用。 -::: - -获得良好性能的关键因素: - -* 查询涉及的分区数量应足够大(大于 `max_threads / 2`),否则查询将无法充分利用机器资源 -* 分区不应过小,否则批处理会退化为逐行处理 -* 分区大小应大致相当,这样所有线程执行的工作量大致相同 - -:::info -建议对 `partition by` 子句中的列应用某种哈希函数,以便在分区之间均匀分布数据。 -::: - -相关设置如下: - -* `allow_aggregate_partitions_independently` - 控制是否启用该优化 -* `force_aggregate_partitions_independently` - 当从正确性角度看可用,但因内部评估其性价比的逻辑而被禁用时,强制使用该优化 -* `max_number_of_partitions_for_independent_aggregation` - 对表可包含的最大分区数量的硬性限制 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md deleted file mode 100644 index 69e4427819e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/graphitemergetree.md +++ /dev/null @@ -1,272 +0,0 @@ ---- -description: '专为对 Graphite 数据进行降采样和聚合/平均(rollup)而设计。' -sidebar_label: 'GraphiteMergeTree' -sidebar_position: 90 -slug: /engines/table-engines/mergetree-family/graphitemergetree -title: 'GraphiteMergeTree 表引擎' -doc_type: 'guide' ---- - -# GraphiteMergeTree 表引擎 {#graphitemergetree-table-engine} - -该引擎用于对 [Graphite](http://graphite.readthedocs.io/en/latest/index.html) 数据进行稀疏化和聚合/平均(rollup)处理。它对希望使用 ClickHouse 作为 Graphite 数据存储的开发者非常有用。 - -如果不需要 rollup,可以使用任意 ClickHouse 表引擎来存储 Graphite 数据;但如果需要 rollup,请使用 `GraphiteMergeTree`。该引擎可以减少存储占用并提高针对 Graphite 的查询效率。 - -该引擎继承自 [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) 的属性。 - -## 创建表 {#creating-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - Path String, - Time DateTime, - Value Float64, - Version - ... -) ENGINE = GraphiteMergeTree(config_section) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -参阅 [CREATE TABLE](/sql-reference/statements/create/table) 查询的详细说明。 - -用于 Graphite 数据的表应包含以下列,对应如下数据: - -* 指标名称(Graphite 指标名)。数据类型:`String`。 - -* 指标的测量时间。数据类型:`DateTime`。 - -* 指标的值。数据类型:`Float64`。 - -* 指标的版本。数据类型:任意数值类型(ClickHouse 会保留具有最高版本号的行,或者在版本号相同时保留最后写入的行。其他行会在数据分片合并时被删除)。 - -这些列的名称应在 rollup 配置中指定。 - -**GraphiteMergeTree 参数** - -* `config_section` — 配置文件中定义 rollup 规则的节名称。 - -**查询子句** - -在创建 `GraphiteMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。 - -
- 已弃用的建表方法 - - :::note - 不要在新项目中使用此方法,并在可能的情况下,将旧项目切换到上面描述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - EventDate Date, - Path String, - Time DateTime, - Value Float64, - Version - ... - ) ENGINE [=] GraphiteMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, config_section) - ``` - - 除 `config_section` 以外,所有参数的含义都与 `MergeTree` 中相同。 - - * `config_section` — 配置文件中定义 rollup 规则的节名称。 -
- -## Rollup 配置 {#rollup-configuration} - -Rollup 的配置由服务器配置中的 [graphite_rollup](../../../operations/server-configuration-parameters/settings.md#graphite) 参数定义。该参数的名称可以任意。你可以创建多个配置,并将它们用于不同的表。 - -Rollup 配置结构: - -required-columns -patterns - -### 必需列 {#required-columns} - -#### `path_column_name` {#path_column_name} - -`path_column_name` — 存储指标名称(Graphite 指标)的列名。默认值:`Path`。 - -#### `time_column_name` {#time_column_name} - -`time_column_name` — 存储该指标采集时间的列名。默认值:`Time`。 - -#### `value_column_name` {#value_column_name} - -`value_column_name` — 存储在 `time_column_name` 中指定时间点的指标值的列名。默认值:`Value`。 - -#### `version_column_name` {#version_column_name} - -`version_column_name` — 存储指标版本的列名。默认值:`Timestamp`。 - -### 模式(Patterns) {#patterns} - -`patterns` 部分的结构: - -```text -pattern - rule_type - regexp - function -pattern - rule_type - regexp - age + precision - ... -pattern - rule_type - regexp - function - age + precision - ... -pattern - ... -default - function - age + precision - ... -``` - -:::important -模式必须严格按以下顺序排列: - -1. 不带 `function` 或 `retention` 的模式。 -2. 同时带有 `function` 和 `retention` 的模式。 -3. `default` 模式。 - ::: - -在处理一行数据时,ClickHouse 会检查 `pattern` 部分中的规则。每个 `pattern`(包括 `default`)部分都可以包含用于聚合的 `function` 参数、`retention` 参数,或两者兼有。如果指标名称匹配 `regexp`,则应用该 `pattern` 部分(或多个部分)中的规则;否则,使用 `default` 部分中的规则。 - -`pattern` 和 `default` 部分的字段: - -* `rule_type` - 规则类型。只应用于特定类型的指标。引擎使用它来区分普通指标和带标签的指标。可选参数。默认值:`all`。 - 当对性能没有严苛要求,或者只使用一种指标类型(例如普通指标)时,该字段不是必需的。默认情况下,只创建一套规则。否则,一旦定义了任一特殊类型,就会创建两套不同的规则:一套用于普通指标(root.branch.leaf),一套用于带标签指标(root.branch.leaf;tag1=value1)。\ - 默认规则最终会进入这两套规则中。\ - 合法取值: - * `all`(默认)- 通用规则,在省略 `rule_type` 时使用。 - * `plain` - 用于普通指标的规则。字段 `regexp` 按正则表达式处理。 - * `tagged` - 用于带标签指标的规则(在数据库中以 `someName?tag1=value1&tag2=value2&tag3=value3` 形式存储)。正则表达式中的标签必须按标签名排序,如果存在,第一个标签必须是 `__name__`。字段 `regexp` 按正则表达式处理。 - * `tag_list` - 用于带标签指标的规则,一种用于简化以 Graphite 格式描述指标的简单 DSL,例如 `someName;tag1=value1;tag2=value2`、`someName` 或 `tag1=value1;tag2=value2`。字段 `regexp` 会被转换成一个 `tagged` 规则。标签名排序不是必须的,会自动完成。标签的值(而不是名称)可以是正则表达式,例如 `env=(dev|staging)`。 -* `regexp` – 指标名称的匹配模式(正则或 DSL)。 -* `age` – 数据的最小年龄(秒)。 -* `precision` – 以秒为单位定义数据年龄的精度。应当是 86400(一天的秒数)的约数。 -* `function` – 要应用于其年龄落在 `[age, age + precision]` 区间内数据的聚合函数名称。可用函数:min / max / any / avg。平均值的计算是近似的,即“平均的平均值”。 - -### 没有规则类型的配置示例 {#configuration-example} - -```xml - - Version - - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -### 不同规则类型的配置示例 {#configuration-typed-example} - -```xml - - Version - - plain - click_cost - any - - 0 - 5 - - - 86400 - 60 - - - - tagged - ^((.*)|.)min\? - min - - 0 - 5 - - - 86400 - 60 - - - - tagged - - min - - 0 - 5 - - - 86400 - 60 - - - - tag_list - someName;tag2=value2 - - 0 - 5 - - - 86400 - 60 - - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - -:::note -数据汇总是在合并过程中完成的。通常不会对旧分区启动合并操作,因此要进行汇总,需要使用 [optimize](../../../sql-reference/statements/optimize.md) 触发一次未计划的合并。也可以使用其他工具,例如 [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer)。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md deleted file mode 100644 index e5cc8cbc4c4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/index.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: 'MergeTree 引擎系列文档' -sidebar_label: 'MergeTree 引擎系列' -sidebar_position: 10 -slug: /engines/table-engines/mergetree-family/ -title: 'MergeTree 引擎系列' -doc_type: 'reference' ---- - -# MergeTree 引擎家族 {#mergetree-engine-family} - -MergeTree 家族中的表引擎是 ClickHouse 数据存储能力的核心。它们提供了实现高可靠性和高性能数据检索的大部分特性:列式存储、自定义分区、稀疏主索引、二级数据跳过索引等。 - -基础的 [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md) 表引擎可以视为单节点 ClickHouse 实例的默认表引擎,因为它用途广泛且适用于多种场景。 - -在生产环境中,更推荐使用 [ReplicatedMergeTree](../../../engines/table-engines/mergetree-family/replication.md),因为它在常规 MergeTree 引擎的所有特性之上增加了高可用性。一个额外的优势是在数据摄取时自动去重,因此在插入期间如果发生网络问题,软件可以安全地重试。 - -MergeTree 家族中的其他所有引擎都为某些特定用例增加了额外功能。通常,这是通过在后台执行附加的数据处理操作来实现的。 - -MergeTree 引擎的主要缺点是相对较为“重量级”。因此,典型的模式是不要创建太多这类表。如果需要大量小表,例如用于临时数据,可以考虑使用 [Log 引擎家族](../../../engines/table-engines/log-family/index.md)。 - -{/* 本页面的目录由以下脚本自动生成: - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 数据来源为 YAML front matter 字段:slug、description、title。 - - 如发现错误,请直接编辑页面本身的 YML front matter。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ----------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -| [MergeTree table engine](/engines/table-engines/mergetree-family/mergetree) | `MergeTree` 系列表引擎专为高数据摄取速率和海量数据量而设计。 | -| [Replicated* table engines](/engines/table-engines/mergetree-family/replication) | 介绍 ClickHouse 中 Replicated* 系列表引擎的数据复制机制概览。 | -| [Custom Partitioning Key](/engines/table-engines/mergetree-family/custom-partitioning-key) | 了解如何为 MergeTree 表添加自定义分区键。 | -| [ReplacingMergeTree table engine](/engines/table-engines/mergetree-family/replacingmergetree) | 与 MergeTree 的区别在于,它会删除具有相同排序键值(`ORDER BY` 表子句,而非 `PRIMARY KEY`)的重复记录。 | -| [CoalescingMergeTree table engine](/engines/table-engines/mergetree-family/coalescingmergetree) | CoalescingMergeTree 继承自 MergeTree 引擎。其核心特性是在数据分片合并期间,能够自动存储每一列的最新非空值。 | -| [SummingMergeTree table engine](/engines/table-engines/mergetree-family/summingmergetree) | SummingMergeTree 继承自 MergeTree 引擎。其核心特性是在数据分片合并期间,能够自动对数值数据求和。 | -| [AggregatingMergeTree table engine](/engines/table-engines/mergetree-family/aggregatingmergetree) | 将所有具有相同主键(更准确地说,是相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md))的行,替换为单行(在同一数据分片内),该行存储聚合函数状态的组合结果。 | -| [CollapsingMergeTree table engine](/engines/table-engines/mergetree-family/collapsingmergetree) | 继承自 MergeTree,但在合并过程中增加了折叠行的逻辑。 | -| [VersionedCollapsingMergeTree table engine](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) | 适用于对持续变化的对象状态进行快速写入,并在后台删除旧的对象状态。 | -| [GraphiteMergeTree table engine](/engines/table-engines/mergetree-family/graphitemergetree) | 专为对 Graphite 数据进行抽稀和聚合/平均(rollup)而设计。 | -| [Exact and Approximate Vector Search](/engines/table-engines/mergetree-family/annindexes) | 精确和近似向量搜索的文档。 | -| [Full-text Search using Text Indexes](/engines/table-engines/mergetree-family/invertedindexes) | 在文本中快速查找搜索词。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md deleted file mode 100644 index 89c513dbddc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/invertedindexes.md +++ /dev/null @@ -1,816 +0,0 @@ ---- -description: '快速在文本中查找搜索词。' -keywords: ['全文搜索', '文本索引', '索引', '索引'] -sidebar_label: '使用文本索引实现全文搜索' -slug: /engines/table-engines/mergetree-family/invertedindexes -title: '使用文本索引实现全文搜索' -doc_type: 'reference' ---- - -import PrivatePreviewBadge from '@theme/badges/PrivatePreviewBadge'; - - -# 使用文本索引进行全文搜索 {#full-text-search-using-text-indexes} - - - -ClickHouse 中的文本索引(也称为["倒排索引"](https://en.wikipedia.org/wiki/Inverted_index))为字符串数据提供快速的全文检索能力。 -索引会将列中的每个标记(token)映射到包含该标记的行。 -这些标记由称为分词(tokenization)的过程生成。 -例如,ClickHouse 默认会将英文句子 "All cat like mice." 分词为 ["All", "cat", "like", "mice"](注意末尾的句点会被忽略)。 -还可以使用更高级的分词器,例如针对日志数据的分词器。 - - - -## 创建文本索引 {#creating-a-text-index} - -要创建文本索引,首先启用相应的实验性设置: - -```sql -SET allow_experimental_full_text_index = true; -``` - -可以使用以下语法在 [String](/sql-reference/data-types/string.md)、[FixedString](/sql-reference/data-types/fixedstring.md)、[Array(String)](/sql-reference/data-types/array.md)、[Array(FixedString)](/sql-reference/data-types/array.md) 以及 [Map](/sql-reference/data-types/map.md) 列(通过 [mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) 和 [mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 映射函数)上定义文本索引: - -```sql -CREATE TABLE tab -( - `key` UInt64, - `str` String, - INDEX text_idx(str) TYPE text( - -- 必需参数: - tokenizer = splitByNonAlpha - | splitByString[(S)] - | ngrams[(N)] - | sparseGrams[(min_length[, max_length[, min_cutoff_length]])] - | array - -- 可选参数: - [, preprocessor = expression(str)] - -- 可选高级参数: - [, dictionary_block_size = D] - [, dictionary_block_frontcoding_compression = B] - [, max_cardinality_for_embedded_postings = M] - [, bloom_filter_false_positive_rate = R] - ) [GRANULARITY 64] -) -ENGINE = MergeTree -ORDER BY key -``` - -**分词器参数(必需)**。`tokenizer` 参数用于指定分词器: - -* `splitByNonAlpha` 按非字母数字的 ASCII 字符拆分字符串(另见函数 [splitByNonAlpha](/sql-reference/functions/splitting-merging-functions.md/#splitByNonAlpha))。 -* `splitByString(S)` 按用户自定义的分隔字符串 `S` 拆分字符串(另见函数 [splitByString](/sql-reference/functions/splitting-merging-functions.md/#splitByString))。 - 可以通过可选参数指定分隔符,例如:`tokenizer = splitByString([', ', '; ', '\n', '\\'])`。 - 注意,每个分隔字符串可以由多个字符组成(如示例中的 `', '`)。 - 如果未显式指定(例如 `tokenizer = splitByString`),默认的分隔符列表是单个空格 `[' ']`。 -* `ngrams(N)` 将字符串拆分为等长的 `N`-gram(另见函数 [ngrams](/sql-reference/functions/splitting-merging-functions.md/#ngrams))。 - N-gram 的长度可以通过 2 到 8 之间的可选整数参数指定,例如:`tokenizer = ngrams(3)`。 - 如果未显式指定(例如 `tokenizer = ngrams`),默认的 N-gram 大小为 3。 -* `sparseGrams(min_length, max_length, min_cutoff_length)` 将字符串拆分为长度在 `min_length` 到 `max_length`(含)之间的可变长度 N-gram(另见函数 [sparseGrams](/sql-reference/functions/string-functions#sparseGrams))。 - 如果未显式指定,`min_length` 和 `max_length` 的默认值分别为 3 和 100。 - 如果提供参数 `min_cutoff_length`,则索引中仅存储长度大于等于 `min_cutoff_length` 的 N-gram。 - 与 `ngrams(N)` 相比,`sparseGrams` 分词器会生成可变长度的 N-gram,从而对原始文本提供更灵活的表示。 - 例如,`tokenizer = sparseGrams(3, 5, 4)` 在内部会从输入字符串生成长度为 3、4、5 的 N-gram,但索引中仅存储长度为 4 和 5 的 N-gram。 -* `array` 不执行任何分词,即每一行的取值本身就是一个词元(另见函数 [array](/sql-reference/functions/array-functions.md/#array))。 - -:::note -`splitByString` 分词器会按从左到右的顺序应用分隔符。 -这可能会产生歧义。 -例如,分隔字符串 `['%21', '%']` 会将 `%21abc` 分词为 `['abc']`,而将分隔字符串顺序交换为 `['%', '%21']` 时,会输出 `['21abc']`。 -在大多数情况下,应当优先匹配更长的分隔符。 -通常可以通过按分隔字符串长度递减的顺序传入来实现这一点。 -如果分隔字符串碰巧构成一个[前缀码](https://en.wikipedia.org/wiki/Prefix_code),则可以以任意顺序传入。 -::: - - -:::warning -目前不建议在非西方语言文本(例如中文)上构建文本索引。 -当前支持的分词器可能导致索引体积过大和查询时间过长。 -我们计划在未来添加专门的语言特定分词器,以更好地处理这些情况。 -::: - -要测试分词器如何拆分输入字符串,可以使用 ClickHouse 的 [tokens](/sql-reference/functions/splitting-merging-functions.md/#tokens) 函数: - -示例: - -```sql -SELECT tokens('abc def', 'ngrams', 3); -``` - -结果: - -```result -['abc','bc ','c d',' de','def'] -``` - -**预处理器参数(可选)**。`preprocessor` 参数是在分词之前应用于输入字符串的表达式。 - -预处理器参数的典型用例包括 - -1. 转换为小写或大写以启用不区分大小写的匹配,例如 [lower](/sql-reference/functions/string-functions.md/#lower)、[lowerUTF8](/sql-reference/functions/string-functions.md/#lowerUTF8),请参见下面的第一个示例。 -2. UTF-8 规范化,例如 [normalizeUTF8NFC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFC)、[normalizeUTF8NFD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFD)、[normalizeUTF8NFKC](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKC)、[normalizeUTF8NFKD](/sql-reference/functions/string-functions.md/#normalizeUTF8NFKD)、[toValidUTF8](/sql-reference/functions/string-functions.md/#toValidUTF8)。 -3. 删除或转换不需要的字符或子字符串,例如 [extractTextFromHTML](/sql-reference/functions/string-functions.md/#extractTextFromHTML)、[substring](/sql-reference/functions/string-functions.md/#substring)、[idnaEncode](/sql-reference/functions/string-functions.md/#idnaEncode)。 - -预处理器表达式必须将 [String](/sql-reference/data-types/string.md) 或 [FixedString](/sql-reference/data-types/fixedstring.md) 类型的输入值转换为相同类型的值。 - -示例: - -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(col))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = substringIndex(col, '\n', 1))` -- `INDEX idx(col) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(extractTextFromHTML(col))` - -此外,预处理器表达式只能引用定义文本索引的列。 -不允许使用非确定性函数。 - -函数 [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken)、[hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) 和 [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) 使用预处理器在分词之前首先转换搜索词。 - -例如, - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(str) TYPE text(tokenizer = 'splitByNonAlpha', preprocessor = lower(str)) -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, 'Foo'); -``` - -等价于: - -```sql -CREATE TABLE tab -( - key UInt64, - str String, - INDEX idx(lower(str)) TYPE text(tokenizer = 'splitByNonAlpha') -) -ENGINE = MergeTree -ORDER BY tuple(); - -SELECT count() FROM tab WHERE hasToken(str, lower('Foo')); -``` - -**其他参数(可选)**。ClickHouse 中的文本索引实现为[二级索引](/engines/table-engines/mergetree-family/mergetree.md/#skip-index-types)。 -但是,与其他跳数索引不同,文本索引的默认索引粒度(GRANULARITY)为 64。 -该值是根据经验选择的,对于大多数用例,它在速度和索引大小之间提供了良好的平衡。 -高级用户可以指定不同的索引粒度(我们不建议这样做)。 - -
- -可选高级参数 - -以下高级参数的默认值几乎在所有情况下都能很好地工作。 -我们不建议更改它们。 - -可选参数 `dictionary_block_size`(默认值:128)指定字典块的行数大小。 - -可选参数 `dictionary_block_frontcoding_compression`(默认值:1)指定字典块是否使用前缀编码作为压缩方式。 - -可选参数 `max_cardinality_for_embedded_postings`(默认值:16)指定基数阈值,低于该阈值时倒排列表应嵌入到字典块中。 - - -可选参数 `bloom_filter_false_positive_rate`(默认值:0.1)用于指定字典布隆过滤器的误报率。 - -
- -表创建后,可以对列添加或删除文本索引: - -```sql -ALTER TABLE tab DROP INDEX text_idx; -ALTER TABLE tab ADD INDEX text_idx(s) TYPE text(tokenizer = splitByNonAlpha); -``` - - -## 使用文本索引 {#using-a-text-index} - -在 SELECT 查询中使用文本索引非常简单,常见的字符串搜索函数会自动使用该索引。 -如果不存在索引,下面这些字符串搜索函数将会回退到较慢的暴力扫描。 - -### 支持的函数 {#functions-support} - -当在 SELECT 查询的 `WHERE` 子句中使用文本函数时,即可使用文本索引: - -```sql -SELECT [...] -FROM [...] -WHERE string_search_function(column_with_text_index) -``` - -#### `=` 和 `!=` {#functions-example-equals-notequals} - -`=`([`equals`](/sql-reference/functions/comparison-functions.md/#equals))和 `!=`([`notEquals`](/sql-reference/functions/comparison-functions.md/#notEquals))会匹配给定搜索词的整体。 - -示例: - -```sql -SELECT * from tab WHERE str = 'Hello'; -``` - -文本索引支持 `=` 和 `!=` 运算,但只有在使用 `array` 分词器时,等值和不等值搜索才有意义(它会使索引存储整行的值)。 - -#### `IN` 和 `NOT IN` {#functions-example-in-notin} - -`IN`([in](/sql-reference/functions/in-functions))和 `NOT IN`([notIn](/sql-reference/functions/in-functions))类似于函数 `equals` 和 `notEquals`,但它们分别要求匹配所有搜索词(`IN`)或完全与所有搜索词都不匹配(`NOT IN`)。 - -示例: - -```sql -SELECT * from tab WHERE str IN ('Hello', 'World'); -``` - -适用与 `=` 和 `!=` 相同的限制,也就是说,只有在配合 `array` tokenizer 使用时,`IN` 和 `NOT IN` 才有意义。 - -#### `LIKE`、`NOT LIKE` 和 `match` {#functions-example-like-notlike-match} - -:::note -这些函数目前仅在索引 tokenizer 为 `splitByNonAlpha` 或 `ngrams` 时才会使用文本索引进行过滤。 -::: - -要在文本索引中使用 `LIKE`([like](/sql-reference/functions/string-search-functions.md/#like))、`NOT LIKE`([notLike](/sql-reference/functions/string-search-functions.md/#notLike))以及 [match](/sql-reference/functions/string-search-functions.md/#match) 函数,ClickHouse 必须能够从搜索词中提取完整的 token。 - -示例: - -```sql -SELECT count() FROM tab WHERE comment LIKE 'support%'; -``` - -示例中的 `support` 可以匹配 `support`、`supports`、`supporting` 等。 -这类查询属于子串查询,无法利用文本索引加速。 - -要在 LIKE 查询中使用文本索引,必须按如下方式改写 LIKE 模式: - -```sql -SELECT count() FROM tab WHERE comment LIKE ' support %'; -- 或者 `% support %` -``` - -`support` 左右的空格确保该术语可以被提取为一个 token。 - -#### `startsWith` 和 `endsWith` {#functions-example-startswith-endswith} - -与 `LIKE` 类似,[startsWith](/sql-reference/functions/string-functions.md/#startsWith) 和 [endsWith](/sql-reference/functions/string-functions.md/#endsWith) 函数只有在能从搜索词中提取出完整的 token 时,才能使用文本索引。 - -示例: - -```sql -SELECT count() FROM tab WHERE startsWith(comment, 'ClickHouse 支持'); -``` - -在这个示例中,只有 `clickhouse` 被视为一个词元(token)。 -`support` 不是词元,因为它可以匹配 `support`、`supports`、`supporting` 等。 - -要查找所有以 `clickhouse supports` 开头的行,请在搜索模式的末尾保留一个空格: - -```sql -startsWith(comment, 'clickhouse supports ')` -``` - -同样地,`endsWith` 也应当在前面加上一个空格再使用: - -```sql -SELECT count() FROM tab WHERE endsWith(comment, ' olap 引擎'); -``` - -#### `hasToken` 和 `hasTokenOrNull` {#functions-example-hastoken-hastokenornull} - -函数 [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) 和 [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) 用于匹配单个指定的 token。 - -与前面提到的函数不同,它们不会对搜索项进行分词(假定输入本身就是单个 token)。 - -示例: - -```sql -SELECT count() FROM tab WHERE hasToken(comment, 'clickhouse'); -``` - -函数 `hasToken` 和 `hasTokenOrNull` 是在与 `text` 索引配合使用时性能最优的函数。 - -#### `hasAnyTokens` 和 `hasAllTokens` {#functions-example-hasanytokens-hasalltokens} - - -函数 [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) 和 [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) 分别用于匹配给定 token 中的任意一个或全部。 - -这两个函数接受的搜索 token 可以是一个字符串(将使用与索引列相同的 tokenizer 进行分词),也可以是一个已处理好的 token 数组(在搜索前不会再进行分词)。\ -有关更多信息,请参阅函数文档。 - -示例: - -```sql --- 以字符串参数形式传递搜索词元 -SELECT count() FROM tab WHERE hasAnyTokens(comment, 'clickhouse olap'); -SELECT count() FROM tab WHERE hasAllTokens(comment, 'clickhouse olap'); - --- 以 Array(String) 形式传递搜索词元 -SELECT count() FROM tab WHERE hasAnyTokens(comment, ['clickhouse', 'olap']); -SELECT count() FROM tab WHERE hasAllTokens(comment, ['clickhouse', 'olap']); -``` - -#### `has` {#functions-example-has} - -数组函数 [has](/sql-reference/functions/array-functions#has) 用于判断字符串数组中是否包含某个单独的元素。 - -示例: - -```sql -SELECT count() FROM tab WHERE has(array, 'clickhouse'); -``` - -#### `mapContains` {#functions-example-mapcontains} - -函数 [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains)(`mapContainsKey` 的别名)用于在 map 的键中匹配单个 token。 - -示例: - -```sql -SELECT count() FROM tab WHERE mapContainsKey(map, 'clickhouse'); --- OR -SELECT count() FROM tab WHERE mapContains(map, 'clickhouse'); -``` - -#### `operator[]` {#functions-example-access-operator} - -可以将访问运算符 [operator[]](/sql-reference/operators#access-operators) 与文本索引配合使用,以过滤键和值。 - -示例: - -```sql -SELECT count() FROM tab WHERE map['engine'] = 'clickhouse'; -``` - -请参阅以下示例,了解如何配合文本索引使用 `Array(T)` 和 `Map(K, V)` 类型的列。 - -### 带有文本索引的 `Array` 和 `Map` 列示例 {#text-index-array-and-map-examples} - -#### 为 Array(String) 列建立索引 {#text-index-example-array} - -想象一个博客平台,作者会使用关键词为他们的博文打标签并进行分类。 -我们希望用户能够通过搜索或点击主题来发现相关内容。 - -考虑如下数据表定义: - -```sql -CREATE TABLE posts ( - post_id UInt64, - title String, - content String, - keywords Array(String) -) -ENGINE = MergeTree -ORDER BY (post_id); -``` - -在没有文本索引的情况下,要查找包含特定关键字(例如 `clickhouse`)的帖子,就需要扫描所有条目: - -```sql -SELECT count() FROM posts WHERE has(keywords, 'clickhouse'); -- 全表扫描速度慢 - 检查每条记录中的所有关键词 -``` - -随着平台规模的扩大,这种查询会变得越来越慢,因为必须检查每一行中的 `keywords` 数组。 -为解决这一性能问题,我们在 `keywords` 列上定义一个文本索引: - -```sql -ALTER TABLE posts ADD INDEX keywords_idx(keywords) TYPE text(tokenizer = splitByNonAlpha); -ALTER TABLE posts MATERIALIZE INDEX keywords_idx; -- 务必为现有数据重建索引 -``` - -#### 为 Map 列建立索引 {#text-index-example-map} - -在许多可观测性场景中,日志消息会被拆分为「组件」,并按合适的数据类型存储,例如时间戳使用 DateTime 类型、日志级别使用 Enum 类型等。 -指标字段最好以键值对的形式存储。 -运维团队需要高效地搜索日志,以用于调试、安全事件排查和监控。 - -请看如下日志表: - -```sql -CREATE TABLE logs ( - id UInt64, - timestamp DateTime, - message String, - attributes Map(String, String) -) -ENGINE = MergeTree -ORDER BY (timestamp); -``` - -若没有文本索引,搜索 [Map](/sql-reference/data-types/map.md) 数据时需要进行全表扫描: - -```sql --- 查找所有包含速率限制数据的日志: -SELECT count() FROM logs WHERE has(mapKeys(attributes), 'rate_limit'); -- 全表扫描,性能较慢 - --- 查找来自特定 IP 的所有日志: -SELECT count() FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- 全表扫描,性能较慢 -``` - -随着日志量增长,这些查询会变慢。 - -解决方案是为 [Map](/sql-reference/data-types/map.md) 的键和值创建文本索引。 -当需要按字段名或属性类型检索日志时,可以使用 [mapKeys](/sql-reference/functions/tuple-map-functions.md/#mapkeys) 创建文本索引: - - -```sql -ALTER TABLE logs ADD INDEX attributes_keys_idx mapKeys(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_keys_idx; -``` - -当你需要在属性值的内容中进行搜索时,使用 [mapValues](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 来创建文本索引: - -```sql -ALTER TABLE logs ADD INDEX attributes_vals_idx mapValues(attributes) TYPE text(tokenizer = array); -ALTER TABLE posts MATERIALIZE INDEX attributes_vals_idx; -``` - -查询示例: - -```sql --- 查找所有受速率限制的请求: -SELECT * FROM logs WHERE mapContainsKey(attributes, 'rate_limit'); -- 快速 - --- 查找来自特定 IP 的所有日志: -SELECT * FROM logs WHERE has(mapValues(attributes), '192.168.1.1'); -- 快速 -``` - - -## 性能调优 {#performance-tuning} - -### 直接读取 {#direct-read} - -某些类型的文本查询可以通过一种称为“直接读取”(direct read)的优化显著提速。 -更具体地说,当 SELECT 查询中 *没有* 从文本列进行投影时,可以应用此优化。 - -示例: - -```sql -SELECT column_a, column_b, ... -- 注意:不要选择 column_with_text_index -FROM [...] -WHERE string_search_function(column_with_text_index) -``` - -ClickHouse 中的直接读取(direct read)优化仅使用文本索引(即通过文本索引查找)来回答查询,而无需访问底层文本列。 -文本索引查找读取的数据量相对较少,因此比 ClickHouse 中通常的 skip 索引(skip index)要快得多(后者会先进行 skip 索引查找,然后再加载并过滤存活的数据粒度(granule))。 - -直接读取由两个设置控制: - -* 设置 [query_plan_direct_read_from_text_index](../../../operations/settings/settings#query_plan_direct_read_from_text_index),用于指定是否启用直接读取。 -* 设置 [use_skip_indexes_on_data_read](../../../operations/settings/settings#use_skip_indexes_on_data_read),这是启用直接读取的另一个前提条件。注意,在 ClickHouse 数据库上如果 [compatibility](../../../operations/settings/settings#compatibility) < 25.10,则 `use_skip_indexes_on_data_read` 被禁用,因此你要么需要提升 `compatibility` 设置的值,要么显式执行 `SET use_skip_indexes_on_data_read = 1`。 - -此外,要使用直接读取,文本索引必须已完全物化(为此可使用 `ALTER TABLE ... MATERIALIZE INDEX`)。 - -**支持的函数** -直接读取优化支持函数 `hasToken`、`hasAllTokens` 和 `hasAnyTokens`。 -这些函数也可以通过 AND、OR 和 NOT 运算符组合使用。 -WHERE 子句还可以包含额外的非文本搜索函数过滤条件(针对文本列或其他列)——在这种情况下,仍会使用直接读取优化,但效果会较差一些(它只应用于受支持的文本搜索函数)。 - -要判断查询是否使用了直接读取,请使用 `EXPLAIN PLAN actions = 1` 来运行该查询。 -例如,一个禁用直接读取的查询如下所示: - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 0, -- disable direct read - use_skip_indexes_on_data_read = 1; -``` - -返回值 - -```text -[...] -Filter ((WHERE + 将列名替换为列标识符)) -过滤条件列: hasToken(__table1.col, 'some_token'_String) (已移除) -操作: INPUT : 0 -> col String : 0 - COLUMN Const(String) -> 'some_token'_String String : 1 - FUNCTION hasToken(col :: 0, 'some_token'_String :: 1) -> hasToken(__table1.col, 'some_token'_String) UInt8 : 2 -[...] -``` - -而当在将 `query_plan_direct_read_from_text_index` 设置为 `1` 的情况下运行相同的查询时 - -```sql -EXPLAIN PLAN actions = 1 -SELECT count() -FROM tab -WHERE hasToken(col, 'some_token') -SETTINGS query_plan_direct_read_from_text_index = 1, -- 启用直接读取 - use_skip_indexes_on_data_read = 1; -``` - -返回值 - -```text -[...] -Expression (Before GROUP BY) -Positions: - Filter - Filter column: __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 (removed) - Actions: INPUT :: 0 -> __text_index_idx_hasToken_94cc2a813036b453d84b6fb344a63ad3 UInt8 : 0 -[...] -``` - -EXPLAIN PLAN 的第二个输出结果中包含一个虚拟列 `__text_index___`。 -如果存在该列,则会使用直接读取。 - -### 缓存 {#caching} - -可以使用不同的缓存将文本索引的部分内容缓存在内存中(参见[实现细节](#implementation)部分): -目前,对文本索引的反序列化字典块、头信息和倒排列表都提供了缓存,以减少 I/O。 -可以通过以下设置启用这些缓存:[use_text_index_dictionary_cache](/operations/settings/settings#use_text_index_dictionary_cache)、[use_text_index_header_cache](/operations/settings/settings#use_text_index_header_cache) 和 [use_text_index_postings_cache](/operations/settings/settings#use_text_index_postings_cache)。 -默认情况下,所有缓存均为禁用状态。 - -有关配置这些缓存,请参阅以下服务器设置。 - -#### 字典块缓存设置 {#caching-dictionary} - - -| Setting | Description | -|----------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------| -| [text_index_dictionary_block_cache_policy](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_policy) | 文本索引字典块缓存策略名称。 | -| [text_index_dictionary_block_cache_size](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size) | 最大缓存大小(字节)。 | -| [text_index_dictionary_block_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_max_entries) | 缓存中反序列化字典块的最大数量。 | -| [text_index_dictionary_block_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_dictionary_block_cache_size_ratio) | 文本索引字典块缓存中受保护队列大小占缓存总大小的比例。 | - -#### Header cache settings {#caching-header} - -| Setting | Description | -|--------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------| -| [text_index_header_cache_policy](/operations/server-configuration-parameters/settings#text_index_header_cache_policy) | 文本索引头部缓存策略名称。 | -| [text_index_header_cache_size](/operations/server-configuration-parameters/settings#text_index_header_cache_size) | 最大缓存大小(字节)。 | -| [text_index_header_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_header_cache_max_entries) | 缓存中反序列化头部的最大数量。 | -| [text_index_header_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_header_cache_size_ratio) | 文本索引头部缓存中受保护队列大小占缓存总大小的比例。 | - -#### Posting lists cache settings {#caching-posting-lists} - -| Setting | Description | -|---------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------| -| [text_index_postings_cache_policy](/operations/server-configuration-parameters/settings#text_index_postings_cache_policy) | 文本索引倒排列表缓存策略名称。 | -| [text_index_postings_cache_size](/operations/server-configuration-parameters/settings#text_index_postings_cache_size) | 最大缓存大小(字节)。 | -| [text_index_postings_cache_max_entries](/operations/server-configuration-parameters/settings#text_index_postings_cache_max_entries) | 缓存中反序列化倒排列表项的最大数量。 | -| [text_index_postings_cache_size_ratio](/operations/server-configuration-parameters/settings#text_index_postings_cache_size_ratio) | 文本索引倒排列表缓存中受保护队列大小占缓存总大小的比例。 | - - - -## 实现细节 {#implementation} - -每个文本索引由两个(抽象)数据结构组成: -- 一个将每个 token 映射到倒排列表(postings list)的字典,和 -- 一组倒排列表,每个倒排列表表示一组行号。 - -由于文本索引是一种跳跃索引(skip index),这些数据结构在逻辑上是按索引粒度(index granule)划分的。 - -在创建索引时,会创建三个文件(每个 part 一个): - -**字典块文件 (.dct)** - -索引粒度中的 token 会被排序,并以每 128 个 token 为一组存储在字典块中(块大小可通过参数 `dictionary_block_size` 配置)。 -字典块文件 (.dct) 由某个 part 中所有索引粒度的全部字典块组成。 - -**索引粒度文件 (.idx)** - -索引粒度文件中,对于每个字典块,存储该块的第一个 token、它在字典块文件中的相对偏移量,以及该块中所有 token 的 Bloom 过滤器。 -这种稀疏索引结构类似于 ClickHouse 的 [稀疏主键索引](https://clickhouse.com/docs/guides/best-practices/sparse-primary-indexes)。 -Bloom 过滤器可以在待查询的 token 不包含在某个字典块中时提前跳过该字典块。 - -**倒排列表文件 (.pst)** - -所有 token 的倒排列表顺序存储在倒排列表文件中。 -为了节省空间,同时仍然支持快速的交集与并集操作,倒排列表被存储为 [roaring bitmaps](https://roaringbitmap.org/)。 -如果某个倒排列表的基数小于 16(可通过参数 `max_cardinality_for_embedded_postings` 配置),则会将其内嵌到字典中。 - - - -## 示例:Hacker News 数据集 {#hacker-news-dataset} - -来看一下在包含大量文本的大型数据集上,使用文本索引带来的性能提升。 -我们将使用来自热门网站 Hacker News 的 2870 万行评论数据。 -下面是未使用文本索引的表: - -```sql -CREATE TABLE hackernews ( - id UInt64, - deleted UInt8, - type String, - author String, - timestamp DateTime, - comment String, - dead UInt8, - parent UInt64, - poll UInt64, - children Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32 -) -ENGINE = MergeTree -ORDER BY (type, author); -``` - -这 2870 万行数据存储在 S3 上的一个 Parquet 文件中——让我们将它们插入到 `hackernews` 表中: - -```sql -INSERT INTO hackernews - SELECT * FROM s3Cluster( - 'default', - 'https://datasets-documentation.s3.eu-west-3.amazonaws.com/hackernews/hacknernews.parquet', - 'Parquet', - ' - id UInt64, - deleted UInt8, - type String, - by String, - time DateTime, - text String, - dead UInt8, - parent UInt64, - poll UInt64, - kids Array(UInt32), - url String, - score UInt32, - title String, - parts Array(UInt32), - descendants UInt32'); -``` - -我们将使用 `ALTER TABLE` 在 comment 列上添加文本索引,然后对其进行物化: - -```sql --- 添加索引 -ALTER TABLE hackernews ADD INDEX comment_idx(comment) TYPE text(tokenizer = splitByNonAlpha); - --- 物化现有数据的索引 -ALTER TABLE hackernews MATERIALIZE INDEX comment_idx SETTINGS mutations_sync = 2; -``` - -现在,让我们使用 `hasToken`、`hasAnyTokens` 和 `hasAllTokens` 函数来运行查询。 -下面的示例将展示标准索引扫描与直接读取优化之间显著的性能差异。 - -### 1. 使用 `hasToken` {#using-hasToken} - -`hasToken` 会检查文本是否包含某个特定的单一 token。 -我们将搜索区分大小写的 token 'ClickHouse'。 - -**禁用直接读取(标准扫描)** -默认情况下,ClickHouse 使用跳过索引(skip index)来过滤 granule,然后读取这些 granule 的列数据。 -我们可以通过禁用直接读取来模拟这种行为。 - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -1 行结果。用时:0.362 秒。已处理 2490 万行,9.51 GB -``` - -**已启用直接读取(快速索引读取)** -现在我们在启用直接读取(默认行为)的情况下运行同一个查询。 - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 516 │ -└─────────┘ - -1 行结果。耗时:0.008 秒。已处理 315 万行,3.15 MB -``` - -直接读取的查询速度快了超过 45 倍(0.362s 对比 0.008s),并且仅通过从索引中读取就处理了显著更少的数据量(9.51 GB 对比 3.15 MB)。 - -### 2. 使用 `hasAnyTokens` {#using-hasAnyTokens} - -`hasAnyTokens` 会检查文本是否包含至少一个给定的 token。 -我们将搜索包含 “love” 或 “ClickHouse” 的评论。 - -**禁用直接读取(标准扫描)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 408426 │ -└─────────┘ - -返回 1 行。用时:1.329 秒。已处理 2874 万行,9.72 GB -``` - -**已启用直接读取(快速索引读取)** - -```sql -SELECT count() -FROM hackernews -WHERE hasAnyTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 408426 │ -└─────────┘ -``` - - -1 行结果。耗时 0.015 秒。已处理 27.99 百万行数据,27.99 MB - -```` -对于这种常见的"OR"搜索,性能提升更为显著。 -通过避免全列扫描,查询速度提升了近89倍(从1.329秒降至0.015秒)。 - -### 3. 使用 `hasAllTokens` {#using-hasAllTokens} - -`hasAllTokens` 用于检查文本是否包含所有给定的词元。 -我们将搜索同时包含 'love' 和 'ClickHouse' 的评论。 - -**禁用直接读取(标准扫描)** -即使禁用直接读取,标准跳数索引仍然有效。 -它将2870万行过滤至仅14.746万行,但仍需从列中读取57.03 MB数据。 - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -返回1行。耗时:0.184秒。已处理14.746万行,57.03 MB -```` - -**已启用直接读取(快速索引读取)** -直接读取仅在索引数据上执行以响应查询,仅读取 147.46 KB。 - -```sql -SELECT count() -FROM hackernews -WHERE hasAllTokens(comment, 'love ClickHouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 11 │ -└─────────┘ - -返回 1 行。用时:0.007 秒。已处理 147.46 千行,147.46 KB -``` - -对于这个“AND”搜索,直接读取优化相比标准的跳过索引(skip index)扫描快超过 26 倍(0.184 秒 vs 0.007 秒)。 - -### 4. 复合搜索:OR、AND、NOT,… {#compound-search} - -直接读取优化同样适用于复合布尔表达式。 -在这里,我们将执行一次不区分大小写的搜索:'ClickHouse' OR 'clickhouse'。 - -**已禁用直接读取(标准扫描)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 0, use_skip_indexes_on_data_read = 0; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -返回 1 行。用时:0.450 秒。已处理 2587 万行,9.58 GB -``` - -**直接读取功能已启用(快速索引读取)** - -```sql -SELECT count() -FROM hackernews -WHERE hasToken(comment, 'ClickHouse') OR hasToken(comment, 'clickhouse') -SETTINGS query_plan_direct_read_from_text_index = 1, use_skip_indexes_on_data_read = 1; - -┌─count()─┐ -│ 769 │ -└─────────┘ - -1 row in set. Elapsed: 0.013 sec. Processed 25.87 million rows, 51.73 MB -``` - -通过结合索引查询结果,直接读取的查询速度快了 34 倍(0.450s 对比 0.013s),并且避免读取 9.58 GB 的列数据。 -在这个特定场景下,`hasAnyTokens(comment, ['ClickHouse', 'clickhouse'])` 是首选且更高效的写法。 - - -## 相关内容 {#related-content} - -- 博客:[在 ClickHouse 中引入倒排索引](https://clickhouse.com/blog/clickhouse-search-with-inverted-indices) -- 博客:[ClickHouse 全文搜索内部解析:快速、原生、列式](https://clickhouse.com/blog/clickhouse-full-text-search) -- 视频:[全文索引:设计与实验](https://www.youtube.com/watch?v=O_MnyUkrIq8) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md deleted file mode 100644 index 4cd4adf21fd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/mergetree.md +++ /dev/null @@ -1,1162 +0,0 @@ ---- -description: '`MergeTree` 系列表引擎专为高数据摄取速率和海量数据规模而设计。' -sidebar_label: 'MergeTree' -sidebar_position: 11 -slug: /engines/table-engines/mergetree-family/mergetree -title: 'MergeTree 表引擎' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# MergeTree 表引擎 {#mergetree-table-engine} - -`MergeTree` 引擎以及 `MergeTree` 家族中的其他引擎(例如 `ReplacingMergeTree`、`AggregatingMergeTree`)是 ClickHouse 中最常用、也最健壮的表引擎。 - -`MergeTree` 家族表引擎专为高数据摄取速率和海量数据规模而设计。 -插入操作会创建表部件(part),这些部件会由后台进程与其他表部件进行合并。 - -`MergeTree` 家族表引擎的主要特性: - -* 表的主键决定了每个表部件内部的排序顺序(聚簇索引)。主键并不引用单独的行,而是引用称为粒度(granule)的 8192 行数据块。这样可以使超大数据集的主键足够小,从而始终保留在主内存中,同时仍然能够快速访问磁盘上的数据。 - -* 表可以使用任意分区表达式进行分区。分区裁剪可以在查询条件允许的情况下跳过读取某些分区。 - -* 数据可以在多个集群节点之间进行复制,以实现高可用、故障切换以及零停机升级。参见 [Data replication](/engines/table-engines/mergetree-family/replication.md)。 - -* `MergeTree` 表引擎支持多种统计信息种类和采样方法,以帮助进行查询优化。 - -:::note -尽管名称相似,[Merge](/engines/table-engines/special/merge) 引擎与 `*MergeTree` 引擎是不同的。 -::: - -## 创建表 {#table_engine-mergetree-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr1] [COMMENT ...] [CODEC(codec1)] [STATISTICS(stat1)] [TTL expr1] [PRIMARY KEY] [SETTINGS (name = value, ...)], - name2 [type2] [[NOT] NULL] [DEFAULT|MATERIALIZED|ALIAS|EPHEMERAL expr2] [COMMENT ...] [CODEC(codec2)] [STATISTICS(stat2)] [TTL expr2] [PRIMARY KEY] [SETTINGS (name = value, ...)], - ... - INDEX index_name1 expr1 TYPE type1(...) [GRANULARITY value1], - INDEX index_name2 expr2 TYPE type2(...) [GRANULARITY value2], - ... - PROJECTION projection_name_1 (SELECT [GROUP BY] [ORDER BY]), - PROJECTION projection_name_2 (SELECT [GROUP BY] [ORDER BY]) -) ENGINE = MergeTree() -ORDER BY expr -[PARTITION BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[TTL expr - [DELETE|TO DISK 'xxx'|TO VOLUME 'xxx' [, ...] ] - [WHERE conditions] - [GROUP BY key_expr [SET v1 = aggr_func(v1) [, v2 = aggr_func(v2) ...]] ] ] -[SETTINGS name = value, ...] -``` - -有关这些参数的详细说明,请参阅 [CREATE TABLE](/sql-reference/statements/create/table.md) 语句。 - -### 查询子句 {#mergetree-query-clauses} - -#### ENGINE {#engine} - -`ENGINE` — 引擎名称和参数。`ENGINE = MergeTree()`。`MergeTree` 引擎没有参数。 - -#### ORDER BY {#order_by} - -`ORDER BY` — 排序键。 - -由列名或任意表达式组成的元组(tuple)。示例:`ORDER BY (CounterID + 1, EventDate)`。 - -如果未定义主键(即未指定 `PRIMARY KEY`),ClickHouse 会将排序键用作主键。 - -如果不需要排序,可以使用语法 `ORDER BY tuple()`。 -或者,如果启用了 `create_table_empty_primary_key_by_default` 设置,则会在 `CREATE TABLE` 语句中隐式添加 `ORDER BY ()`。参见 [选择主键](#selecting-a-primary-key)。 - -#### PARTITION BY {#partition-by} - -`PARTITION BY` — 即[分区键](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。可选。在大多数情况下不需要分区键;即使需要分区,通常按月分区已经足够,无需使用比“按月”更细粒度的分区键。分区并不会加速查询(与 ORDER BY 表达式不同)。不要使用过于细粒度的分区。不要按客户端标识符或名称对数据进行分区(应将客户端标识符或名称作为 ORDER BY 表达式中的第一列)。 - -要按月进行分区,使用 `toYYYYMM(date_column)` 表达式,其中 `date_column` 是一个类型为 [Date](/sql-reference/data-types/date.md) 的日期列。此处的分区名称采用 `"YYYYMM"` 格式。 - -#### PRIMARY KEY {#primary-key} - -`PRIMARY KEY` — 主键,如果它[与排序键不同](#choosing-a-primary-key-that-differs-from-the-sorting-key)。可选。 - -指定排序键(使用 `ORDER BY` 子句)会隐式地指定主键。 -通常无需在排序键之外再单独指定主键。 - -#### SAMPLE BY {#sample-by} - -`SAMPLE BY` — 采样表达式。可选。 - -如果指定该表达式,则它必须包含在主键中。 -采样表达式的结果必须为无符号整数。 - -示例:`SAMPLE BY intHash32(UserID) ORDER BY (CounterID, EventDate, intHash32(UserID))`。 - -#### TTL {#ttl} - -`TTL` — 一组规则,用于指定行的保留期限以及数据片在[磁盘与卷之间](#table_engine-mergetree-multiple-volumes)自动迁移的逻辑。可选。 - -表达式的结果必须是 `Date` 或 `DateTime`,例如 `TTL date + INTERVAL 1 DAY`。 - -规则类型 `DELETE|TO DISK 'xxx'|TO VOLUME 'xxx'|GROUP BY` 指定在表达式满足条件(达到当前时间)时,对该数据片执行的操作:删除过期行,将数据片(当该片中所有行的表达式都满足条件时)移动到指定磁盘(`TO DISK 'xxx'`)或卷(`TO VOLUME 'xxx'`),或对过期行中的值进行聚合。规则的默认类型为删除(`DELETE`)。可以指定多条规则,但 `DELETE` 规则不得超过一条。 - -更多细节,参见 [列和表的 TTL](#table_engine-mergetree-ttl)。 - -#### 设置 {#settings} - -参见 [MergeTree 设置](../../../operations/settings/merge-tree-settings.md)。 - -**Sections 设置示例** - -```sql -ENGINE MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity=8192 -``` - -在示例中,我们按月份设置了分区。 - -我们还将采样表达式设置为基于用户 ID 的哈希。这允许你针对每个 `CounterID` 和 `EventDate` 对表中的数据进行伪随机化。如果在查询数据时指定了 [SAMPLE](/sql-reference/statements/select/sample) 子句,ClickHouse 会为一部分用户返回均匀的伪随机数据样本。 - -`index_granularity` 设置可以省略,因为 8192 是默认值。 - -
- 已弃用的建表方法 - - :::note - 不要在新项目中使用此方法。如有可能,请将旧项目切换到上面描述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] MergeTree(date-column [, sampling_expression], (primary, key), index_granularity) - ``` - - **MergeTree() 参数** - - * `date-column` — [Date](/sql-reference/data-types/date.md) 类型的列名。ClickHouse 会基于该列按月自动创建分区。分区名称采用 `"YYYYMM"` 格式。 - * `sampling_expression` — 用于采样的表达式。 - * `(primary, key)` — 主键。类型:[Tuple()](/sql-reference/data-types/tuple.md) - * `index_granularity` — 索引粒度,即索引“marks”之间的数据行数。8192 这一数值适用于大多数任务。 - - **示例** - - ```sql - MergeTree(EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID)), 8192) - ``` - - `MergeTree` 引擎的配置方式与上面主要引擎配置方法中的示例相同。 -
- -## 数据存储 {#mergetree-data-storage} - -一张表由按主键排序的数据部分(data parts)组成。 - -当向表中插入数据时,会创建独立的数据部分,每个数据部分都会按主键进行字典序排序。比如,如果主键是 `(CounterID, Date)`,那么该数据部分中的数据首先按 `CounterID` 排序,并且在每个 `CounterID` 内部再按 `Date` 排序。 - -属于不同分区的数据会被存放到不同的数据部分中。在后台,ClickHouse 会合并数据部分以实现更高效的存储。属于不同分区的数据部分不会被合并。合并机制并不保证具有相同主键的所有行都会落在同一个数据部分中。 - -数据部分可以以 `Wide` 或 `Compact` 格式存储。在 `Wide` 格式下,每一列都作为单独的文件存储在文件系统中;在 `Compact` 格式下,所有列都存储在同一个文件中。`Compact` 格式可用于提升小批量且频繁插入场景下的性能。 - -数据存储格式由表引擎的 `min_bytes_for_wide_part` 和 `min_rows_for_wide_part` 设置控制。如果某个数据部分中的字节数或行数小于对应设置的值,则该数据部分会以 `Compact` 格式存储;否则将以 `Wide` 格式存储。如果这两个设置都未配置,数据部分将以 `Wide` 格式存储。 - -每个数据部分在逻辑上被划分为多个粒度(granule)。粒度是 ClickHouse 在查询数据时读取的最小不可再分的数据集。ClickHouse 不会拆分行或单个值,因此每个粒度始终包含整数数量的行。粒度的第一行会用该行的主键值进行标记。对于每个数据部分,ClickHouse 会创建一个索引文件来存储这些标记(marks)。对于每一列(无论是否包含在主键中),ClickHouse 也会存储相同的标记。这些标记可以让系统直接在列文件中定位数据。 - -粒度大小受表引擎的 `index_granularity` 和 `index_granularity_bytes` 设置限制。每个粒度中的行数位于 `[1, index_granularity]` 范围内,具体取决于每行数据的大小。如果单行数据的大小超过 `index_granularity_bytes` 的值,则粒度的大小可以超过 `index_granularity_bytes`。在这种情况下,粒度大小等于该行数据的大小。 - -## 查询中的主键和索引 {#primary-keys-and-indexes-in-queries} - -以 `(CounterID, Date)` 主键为例。在这种情况下,排序和索引可以表示如下: - -```text -全部数据: [---------------------------------------------] -CounterID: [aaaaaaaaaaaaaaaaaabbbbcdeeeeeeeeeeeeefgggggggghhhhhhhhhiiiiiiiiikllllllll] -Date: [1111111222222233331233211111222222333211111112122222223111112223311122333] -标记点: | | | | | | | | | | | - a,1 a,2 a,3 b,3 e,2 e,3 g,1 h,2 i,1 i,3 l,3 -标记点编号: 0 1 2 3 4 5 6 7 8 9 10 -``` - -如果数据查询包含以下条件: - -* `CounterID in ('a', 'h')`,服务器会读取标记区间 `[0, 3)` 和 `[6, 8)` 内的数据。 -* `CounterID IN ('a', 'h') AND Date = 3`,服务器会读取标记区间 `[1, 3)` 和 `[7, 8)` 内的数据。 -* `Date = 3`,服务器会读取标记区间 `[1, 10]` 内的数据。 - -上面的示例表明,使用索引总是比全表扫描更高效。 - -稀疏索引会多读一些额外数据。在读取一个主键范围时,每个数据块中最多会额外读取 `index_granularity * 2` 行。 - -稀疏索引允许你处理行数非常巨大的表,因为在大多数情况下,这类索引可以完全放入计算机内存中。 - -ClickHouse 不要求主键唯一。你可以插入多行具有相同主键的记录。 - -你可以在 `PRIMARY KEY` 和 `ORDER BY` 子句中使用 `Nullable` 类型的表达式,但强烈不建议这样做。要启用此功能,请开启 [allow_nullable_key](/operations/settings/merge-tree-settings/#allow_nullable_key) 设置。对于 `ORDER BY` 子句中的 `NULL` 值,适用 [NULLS_LAST](/sql-reference/statements/select/order-by.md/#sorting-of-special-values) 原则。 - -### 选择主键 {#selecting-a-primary-key} - -主键中的列数没有显式限制。可以根据数据结构,在主键中包含更多或更少的列。这可能会: - -* 提高索引性能。 - - 如果主键是 `(a, b)`,那么在满足以下条件时,添加另一列 `c` 会提高性能: - - * 存在带有列 `c` 条件的查询。 - * 通常会出现较长的数据范围(长度是 `index_granularity` 的数倍)在 `(a, b)` 上具有相同的值。换句话说,添加另一列可以使系统跳过相当长的数据范围。 - -* 改善数据压缩。 - - ClickHouse 会按主键对数据进行排序,因此数据按主键越集中、有序,压缩效果越好。 - -* 在 [CollapsingMergeTree](/engines/table-engines/mergetree-family/collapsingmergetree) 和 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 引擎中,为合并数据部分提供额外的逻辑。 - - 在这种情况下,指定与主键不同的*排序键(sorting key)*是有意义的。 - -较长的主键会对插入性能和内存消耗产生负面影响,但在执行 `SELECT` 查询时,主键中的额外列不会影响 ClickHouse 的性能。 - -可以使用 `ORDER BY tuple()` 语法创建没有主键的表。在这种情况下,ClickHouse 按插入顺序存储数据。如果希望在使用 `INSERT ... SELECT` 查询插入数据时保持数据顺序,请将 [max_insert_threads = 1](/operations/settings/settings#max_insert_threads) 设置为 1。 - -要按初始顺序选择数据,请使用[单线程](/operations/settings/settings.md/#max_threads)的 `SELECT` 查询。 - -### 选择与排序键不同的主键 {#choosing-a-primary-key-that-differs-from-the-sorting-key} - -可以指定一个与排序键不同的主键(一个表达式,其值会在每个标记的索引文件中写入)。在这种情况下,主键表达式元组必须是排序键表达式元组的前缀。 - -在使用 [SummingMergeTree](/engines/table-engines/mergetree-family/summingmergetree.md) 和 -[AggregatingMergeTree](/engines/table-engines/mergetree-family/aggregatingmergetree.md) 表引擎时,这一特性非常有用。在这些引擎的常见使用场景中,表通常有两类列:*维度(dimensions)* 和 *度量(measures)*。典型查询会对度量列的值在任意 `GROUP BY` 条件下进行聚合,并按维度进行过滤。由于 SummingMergeTree 和 AggregatingMergeTree 会对具有相同排序键值的行进行聚合,因此将所有维度都加入排序键是很自然的做法。结果是,键表达式会由一个很长的列列表组成,并且在新增维度时必须频繁更新该列表。 - -在这种情况下,更合理的做法是只在主键中保留少数几列,以保证高效的范围扫描,并将其余维度列加入排序键元组中。 - -对排序键执行 [ALTER](/sql-reference/statements/alter/index.md) 是一项轻量级操作,因为当新列同时被添加到表和排序键中时,现有数据部分不需要被修改。由于旧排序键是新排序键的前缀,并且在新添加的列中还没有数据,因此在进行表修改时,数据在逻辑上同时满足按旧排序键和新排序键排序。 - -### 在查询中使用索引和分区 {#use-of-indexes-and-partitions-in-queries} - -对于 `SELECT` 查询,ClickHouse 会分析是否可以使用索引。若 `WHERE/PREWHERE` 子句中包含(作为某个合取项或整体)表示等值或不等比较运算的表达式,或者在主键或分区键中的列或表达式,或这些列上的某些特定函数,或这些表达式的逻辑组合上使用了带固定前缀的 `IN` 或 `LIKE`,则可以使用索引。 - -因此,可以对主键的一个或多个范围快速执行查询。在此示例中,当针对特定的跟踪标签、特定标签与日期范围、特定标签与日期、带日期范围的多个标签等进行查询时,查询都会很快。 - -来看一个如下配置的引擎: - -```sql -ENGINE MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate) -SETTINGS index_granularity=8192 -``` - -在这种情况下,在查询时: - -```sql -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND CounterID = 34 - -SELECT count() FROM table -WHERE EventDate = toDate(now()) -AND (CounterID = 34 OR CounterID = 42) - -SELECT count() FROM table -WHERE ((EventDate >= toDate('2014-01-01') -AND EventDate <= toDate('2014-01-31')) OR EventDate = toDate('2014-05-01')) -AND CounterID IN (101500, 731962, 160656) -AND (CounterID = 101500 OR EventDate != toDate('2014-05-01')) -``` - -ClickHouse 将使用主键索引来跳过不符合条件的数据,并使用按月分区键来跳过处于不符合日期范围内的分区。 - -上面的查询展示了,即使是复杂表达式也会使用索引。表的数据读取经过组织,保证使用索引不会比全表扫描更慢。 - -在下面的示例中,将无法利用索引。 - -```sql -SELECT count() FROM table WHERE CounterID = 34 OR URL LIKE '%upyachka%' -``` - -要检查 ClickHouse 在运行查询时是否可以使用索引,请使用设置项 [force_index_by_date](/operations/settings/settings.md/#force_index_by_date) 和 [force_primary_key](/operations/settings/settings#force_primary_key)。 - -按月分区的分区键可以使查询仅读取包含目标日期范围的数据块。在这种情况下,一个数据块可能包含多个日期的数据(最多可覆盖整个月)。在一个数据块内,数据按主键排序,而主键的首列不一定是日期。正因为如此,如果查询中只包含日期条件而未指定主键前缀,就会为获取某个单一日期而读取比实际需要更多的数据。 - -### 对部分单调主键使用索引 {#use-of-index-for-partially-monotonic-primary-keys} - -以月份中的日期为例。在一个月内,它们构成一个[单调序列](https://en.wikipedia.org/wiki/Monotonic_function),但在更长的时间范围内则不是单调的。这就是一个部分单调序列。如果用户使用部分单调的主键创建表,ClickHouse 会像往常一样创建稀疏索引。当用户从这种类型的表中查询数据时,ClickHouse 会分析查询条件。如果用户希望获取索引中两个标记点之间的数据,并且这两个标记点都落在同一个月内,ClickHouse 就可以在这种特定情况下使用索引,因为它可以计算查询参数与索引标记之间的距离。 - -如果查询参数范围内的主键值不构成单调序列,ClickHouse 无法使用索引。在这种情况下,ClickHouse 会使用全表扫描方法。 - -ClickHouse 不仅对月份日期序列使用这一逻辑,也会对任何表示部分单调序列的主键使用这一逻辑。 - -### 数据跳过索引 {#table_engine-mergetree-data_skipping-indexes} - -索引声明在 `CREATE` 查询的 `columns` 部分中。 - -```sql -INDEX index_name expr TYPE type(...) [GRANULARITY granularity_value] -``` - -对于 `*MergeTree` 家族的表,可以指定数据跳过索引。 - -这些索引会在由 `granularity_value` 个粒度组成的数据块上聚合指定表达式的一些信息(粒度的大小通过表引擎中的 `index_granularity` 设置指定)。随后,这些聚合结果会在 `SELECT` 查询中用于减少从磁盘读取的数据量,通过跳过那些不可能满足 `where` 查询条件的大数据块来实现。 - -可以省略 `GRANULARITY` 子句,此时 `granularity_value` 的默认值为 1。 - -**示例** - -```sql -CREATE TABLE table_name -( - u64 UInt64, - i32 Int32, - s String, - ... - INDEX idx1 u64 TYPE bloom_filter GRANULARITY 3, - INDEX idx2 u64 * i32 TYPE minmax GRANULARITY 3, - INDEX idx3 u64 * length(s) TYPE set(1000) GRANULARITY 4 -) ENGINE = MergeTree() -... -``` - -示例中的索引可供 ClickHouse 在以下查询中使用,以减少从磁盘读取的数据量: - -```sql -SELECT count() FROM table WHERE u64 == 10; -SELECT count() FROM table WHERE u64 * i32 >= 1234 -SELECT count() FROM table WHERE u64 * length(s) == 1234 -``` - -数据跳过索引也可以建立在复合列上: - -```sql --- 在 Map 类型的列上: -INDEX map_key_index mapKeys(map_column) TYPE bloom_filter -INDEX map_value_index mapValues(map_column) TYPE bloom_filter - --- 在 Tuple 类型的列上: -INDEX tuple_1_index tuple_column.1 TYPE bloom_filter -INDEX tuple_2_index tuple_column.2 TYPE bloom_filter - --- 在 Nested 类型的列上: -INDEX nested_1_index col.nested_col1 TYPE bloom_filter -INDEX nested_2_index col.nested_col2 TYPE bloom_filter -``` - -### 跳过索引类型 {#skip-index-types} - -`MergeTree` 表引擎支持以下几种跳过索引类型。\ -有关如何使用跳过索引进行性能优化的更多信息,\ -请参阅[《理解 ClickHouse 数据跳过索引》](/optimize/skipping-indexes)。 - -* [`MinMax`](#minmax) 索引 -* [`Set`](#set) 索引 -* [`bloom_filter`](#bloom-filter) 索引 -* [`ngrambf_v1`](#n-gram-bloom-filter) 索引 -* [`tokenbf_v1`](#token-bloom-filter) 索引 - -#### MinMax 跳过索引 {#minmax} - -对于每个索引粒度,会存储某个表达式的最小值和最大值。 -(如果表达式的类型是 `tuple`,则会为元组中的每个元素分别存储最小值和最大值。) - -```text title="Syntax" -minmax -``` - -#### Set {#set} - -对于每个索引粒度,最多会存储 `max_rows` 个指定表达式的唯一值。 -`max_rows = 0` 表示“存储所有唯一值”。 - -```text title="Syntax" -set(max_rows) -``` - -#### 布隆过滤器 {#bloom-filter} - -对于每个索引粒度,都会为指定列存储一个[布隆过滤器](https://en.wikipedia.org/wiki/Bloom_filter)。 - -```text title="Syntax" -bloom_filter([false_positive_rate]) -``` - -`false_positive_rate` 参数可以取 0 到 1 之间的值(默认值:`0.025`),用于指定产生假阳性(false positive)结果的概率(该值越大,需要读取的数据量越多)。 - -支持以下数据类型: - -* `(U)Int*` -* `Float*` -* `Enum` -* `Date` -* `DateTime` -* `String` -* `FixedString` -* `Array` -* `LowCardinality` -* `Nullable` -* `UUID` -* `Map` - -:::note Map 数据类型:使用键或值创建索引 -对于 `Map` 数据类型,客户端可以通过 [`mapKeys`](/sql-reference/functions/tuple-map-functions.md/#mapkeys) 或 [`mapValues`](/sql-reference/functions/tuple-map-functions.md/#mapvalues) 函数指定索引是针对键还是针对值创建。 -::: - -#### N-gram 布隆过滤器 {#n-gram-bloom-filter} - -对于每个索引粒度,会为指定列的 [n-gram](https://en.wikipedia.org/wiki/N-gram) 存储一个 [布隆过滤器](https://en.wikipedia.org/wiki/Bloom_filter)。 - -```text title="Syntax" -ngrambf_v1(n, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -| 参数 | 描述 | -| ------------------------------- | ---------------------------------------------------------------- | -| `n` | n-gram 大小 | -| `size_of_bloom_filter_in_bytes` | 布隆过滤器(Bloom filter)的字节大小。此处可以使用较大的值,例如 `256` 或 `512`,因为它可以很好地压缩。 | -| `number_of_hash_functions` | 布隆过滤器中使用的哈希函数数量。 | -| `random_seed` | 布隆过滤器哈希函数使用的随机种子。 | - -此索引仅适用于以下数据类型: - -* [`String`](/sql-reference/data-types/string.md) -* [`FixedString`](/sql-reference/data-types/fixedstring.md) -* [`Map`](/sql-reference/data-types/map.md) - -要估算 `ngrambf_v1` 的参数,可以使用以下[用户自定义函数(UDF)](/sql-reference/statements/create/function.md)。 - -```sql title="UDFs for ngrambf_v1" -CREATE FUNCTION bfEstimateFunctions [ON CLUSTER cluster] -AS -(total_number_of_all_grams, size_of_bloom_filter_in_bits) -> round((size_of_bloom_filter_in_bits / total_number_of_all_grams) * log(2)); - -CREATE FUNCTION bfEstimateBmSize [ON CLUSTER cluster] -AS -(total_number_of_all_grams, probability_of_false_positives) -> ceil((total_number_of_all_grams * log(probability_of_false_positives)) / log(1 / pow(2, log(2)))); - -CREATE FUNCTION bfEstimateFalsePositive [ON CLUSTER cluster] -AS -(total_number_of_all_grams, number_of_hash_functions, size_of_bloom_filter_in_bytes) -> pow(1 - exp(-number_of_hash_functions/ (size_of_bloom_filter_in_bytes / total_number_of_all_grams)), number_of_hash_functions); - -CREATE FUNCTION bfEstimateGramNumber [ON CLUSTER cluster] -AS -(number_of_hash_functions, probability_of_false_positives, size_of_bloom_filter_in_bytes) -> ceil(size_of_bloom_filter_in_bytes / (-number_of_hash_functions / log(1 - exp(log(probability_of_false_positives) / number_of_hash_functions)))) -``` - -要使用这些函数,您至少需要指定两个参数: - -* `total_number_of_all_grams` -* `probability_of_false_positives` - -例如,在一个 granule 中有 `4300` 个 ngram,并且您预期误报率小于 `0.0001`。 -然后可以通过执行以下查询来估算其余参数: - -```sql ---- 估算过滤器中的位数 -SELECT bfEstimateBmSize(4300, 0.0001) / 8 AS size_of_bloom_filter_in_bytes; - -┌─size_of_bloom_filter_in_bytes─┐ -│ 10304 │ -└───────────────────────────────┘ - ---- 估算哈希函数的数量 -SELECT bfEstimateFunctions(4300, bfEstimateBmSize(4300, 0.0001)) as number_of_hash_functions - -┌─number_of_hash_functions─┐ -│ 13 │ -└──────────────────────────┘ -``` - -当然,你也可以使用这些函数来估算其他条件下的参数。 -上述函数参考了[此处](https://hur.st/bloomfilter)的布隆过滤器计算器。 - -#### Token bloom filter {#token-bloom-filter} - -Token bloom filter 与 `ngrambf_v1` 相同,但存储的是 token(由非字母数字字符分隔的序列),而不是 ngram。 - -```text title="Syntax" -tokenbf_v1(布隆过滤器字节大小, 哈希函数数量, 随机种子) -``` - -#### 稀疏 grams 布隆过滤器 {#sparse-grams-bloom-filter} - -稀疏 grams 布隆过滤器与 `ngrambf_v1` 类似,但使用的是[稀疏 grams 标记](/sql-reference/functions/string-functions.md/#sparseGrams)而不是 ngrams。 - -```text title="Syntax" -sparse_grams(min_ngram_length, max_ngram_length, min_cutoff_length, size_of_bloom_filter_in_bytes, number_of_hash_functions, random_seed) -``` - -### 文本索引 {#text} - -支持全文搜索,详情见[这里](invertedindexes.md)。 - -#### 向量相似度 {#vector-similarity} - -支持近似最近邻检索,详见[此处](annindexes.md)。 - -### 函数支持 {#functions-support} - -`WHERE` 子句中的条件可能包含对作用于列的函数的调用。如果该列是索引的一部分,ClickHouse 会在执行这些函数时尝试使用该索引。ClickHouse 对可用于索引的函数提供了不同的支持子集。 - -类型为 `set` 的索引可被所有函数使用。其他类型的索引支持情况如下: - -| 函数(运算符)/ 索引 | 主键 | minmax | ngrambf_v1 | tokenbf_v1 | bloom_filter | sparse_grams | text | -| ------------------------------------------------------------------------------------------------------------------------- | -- | ------ | -------------- | -------------- | ---------------- | ---------------- | ---- | -| [equals(=,==)](/sql-reference/functions/comparison-functions.md/#equals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notEquals(!=, <>)](/sql-reference/functions/comparison-functions.md/#notEquals) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [like](/sql-reference/functions/string-search-functions.md/#like) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [notLike](/sql-reference/functions/string-search-functions.md/#notLike) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [match](/sql-reference/functions/string-search-functions.md/#match) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [startsWith](/sql-reference/functions/string-functions.md/#startsWith) | ✔ | ✔ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [endsWith](/sql-reference/functions/string-functions.md/#endsWith) | ✗ | ✗ | ✔ | ✔ | ✗ | ✔ | ✔ | -| [multiSearchAny](/sql-reference/functions/string-search-functions.md/#multiSearchAny) | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | ✗ | -| [in](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [notIn](/sql-reference/functions/in-functions) | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [小于(`<`)](/sql-reference/functions/comparison-functions.md/#less) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [大于 (`>`)](/sql-reference/functions/comparison-functions.md/#greater) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [lessOrEquals (`<=`)](/sql-reference/functions/comparison-functions.md/#lessOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [大于等于 (`>=`)](/sql-reference/functions/comparison-functions.md/#greaterOrEquals) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [empty](/sql-reference/functions/array-functions/#empty) | ✔ | ✔ | ✗ | ✗ | ✗ | ✗ | ✗ | -| [notEmpty](/sql-reference/functions/array-functions/#notEmpty) | ✗ | ✔ | ✗ | ✗ | ✗ | ✔ | ✗ | -| [has](/sql-reference/functions/array-functions#has) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✔ | -| [hasAny](/sql-reference/functions/array-functions#hasAny) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasAll](/sql-reference/functions/array-functions#hasAll) | ✗ | ✗ | ✔ | ✔ | ✔ | ✔ | ✗ | -| [hasToken](/sql-reference/functions/string-search-functions.md/#hasToken) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenOrNull](/sql-reference/functions/string-search-functions.md/#hasTokenOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✔ | -| [hasTokenCaseInsensitive(`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitive) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasTokenCaseInsensitiveOrNull (`*`)](/sql-reference/functions/string-search-functions.md/#hasTokenCaseInsensitiveOrNull) | ✗ | ✗ | ✗ | ✔ | ✗ | ✗ | ✗ | -| [hasAnyTokens](/sql-reference/functions/string-search-functions.md/#hasAnyTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [hasAllTokens](/sql-reference/functions/string-search-functions.md/#hasAllTokens) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | -| [mapContains](/sql-reference/functions/tuple-map-functions#mapcontains) | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✔ | - -对于常量参数小于 ngram 大小的函数,`ngrambf_v1` 不能用于查询优化。 - -(*) 要让 `hasTokenCaseInsensitive` 和 `hasTokenCaseInsensitiveOrNull` 生效,必须在转为小写的数据上创建 `tokenbf_v1` 索引,例如:`INDEX idx (lower(str_col)) TYPE tokenbf_v1(512, 3, 0)`。 - -:::note -由于布隆过滤器可能产生假阳性匹配,因此在期望函数结果为 false 的查询中,`ngrambf_v1`、`tokenbf_v1`、`sparse_grams` 和 `bloom_filter` 索引不能用于查询优化。 - -例如: - -* 可以被优化: - * `s LIKE '%test%'` - * `NOT s NOT LIKE '%test%'` - * `s = 1` - * `NOT s != 1` - * `startsWith(s, 'test')` -* 不能被优化: - * `NOT s LIKE '%test%'` - * `s NOT LIKE '%test%'` - * `NOT s = 1` - * `s != 1` - * `NOT startsWith(s, 'test')` - ::: - -## 投影 {#projections} - -投影类似于[物化视图](/sql-reference/statements/create/view),但定义在数据片段(part)级别。它在提供一致性保证的同时,还能在查询中被自动使用。 - -:::note -在使用投影时,你还应考虑 [force_optimize_projection](/operations/settings/settings#force_optimize_projection) 设置。 -::: - -在带有 [FINAL](/sql-reference/statements/select/from#final-modifier) 修饰符的 `SELECT` 语句中不支持投影。 - -### 投影查询 {#projection-query} - -投影查询用于定义一个投影。它会隐式地从父表中选取数据。 -**语法** - -```sql -SELECT [GROUP BY] [ORDER BY] -``` - -可以使用 [ALTER](/sql-reference/statements/alter/projection.md) 语句修改或删除投影。 - -### 投影存储 {#projection-storage} - -投影存储在数据分片(part)目录中。它类似于索引,但包含一个子目录,用于存放一个匿名 `MergeTree` 表的分片。该表由投影的定义查询所派生。如果存在 `GROUP BY` 子句,则其底层存储引擎变为 [AggregatingMergeTree](aggregatingmergetree.md),并且所有聚合函数都会被转换为 `AggregateFunction`。如果存在 `ORDER BY` 子句,则该 `MergeTree` 表会将其作为主键表达式使用。在合并过程中,投影分片通过其存储引擎的合并流程进行合并。父表分片的校验和会与投影分片的校验和组合在一起。其他维护任务与跳过索引(skip index)类似。 - -### 查询分析 {#projection-query-analysis} - -1. 检查投影是否可以用于回答给定查询,即它是否能生成与查询基表相同的结果。 -2. 选择最优的可行匹配方案,即需要读取的数据颗粒(granule)最少的那个。 -3. 使用投影的查询管道将不同于使用原始数据分片的管道。如果某些数据分片中缺少该投影,可以在查询管道中动态增加步骤以“实时投影”出来。 - -## 并发数据访问 {#concurrent-data-access} - -对于对表的并发访问,我们使用多版本机制。换句话说,当一个表被同时读取和更新时,查询会从在查询时刻“当前”的那一组数据分片中读取数据。不会出现长时间持有的锁。插入操作不会阻塞读取操作。 - -从表中读取会自动并行执行。 - -## 列和表的 TTL {#table_engine-mergetree-ttl} - -用于指定数据值的生命周期。 - -可以为整张表以及每个单独的列设置 `TTL` 子句。表级 `TTL` 还可以指定在不同磁盘和卷之间自动迁移数据的逻辑,或者对数据已全部过期的部件进行重新压缩。 - -表达式的计算结果必须是 [Date](/sql-reference/data-types/date.md)、[Date32](/sql-reference/data-types/date32.md)、[DateTime](/sql-reference/data-types/datetime.md) 或 [DateTime64](/sql-reference/data-types/datetime64.md) 数据类型。 - -**语法** - -为列设置 TTL(生存时间): - -```sql -TTL time_column -TTL time_column + interval -``` - -要定义 `interval`,请使用 [时间间隔](/sql-reference/operators#operators-for-working-with-dates-and-times) 运算符,例如: - -```sql -TTL date_time + INTERVAL 1 MONTH -TTL date_time + INTERVAL 15 HOUR -``` - -### 列 TTL {#mergetree-column-ttl} - -当列中的值过期时,ClickHouse 会将其替换为该列数据类型的默认值。如果某个数据部分中该列的所有值都已过期,ClickHouse 会从文件系统中的该数据部分删除此列。 - -`TTL` 子句不能用于键列。 - -**示例** - -#### 创建带 `TTL` 的表: {#creating-a-table-with-ttl} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int TTL d + INTERVAL 1 MONTH, - b Int TTL d + INTERVAL 1 MONTH, - c String -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d; -``` - -#### 向现有表的列添加 TTL {#adding-ttl-to-a-column-of-an-existing-table} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 DAY; -``` - -#### 更改列的 TTL {#altering-ttl-of-the-column} - -```sql -ALTER TABLE tab - MODIFY COLUMN - c String TTL d + INTERVAL 1 MONTH; -``` - -### 表 TTL {#mergetree-table-ttl} - -表可以定义一个用于删除过期行的表达式,以及多个用于在[磁盘或卷](#table_engine-mergetree-multiple-volumes)之间自动迁移分片的表达式。当表中的行过期时,ClickHouse 会删除所有对应的行。对于分片移动或重新压缩操作,某个分片中的所有行都必须满足 `TTL` 表达式所定义的条件。 - -```sql -TTL 表达式 - [DELETE|RECOMPRESS 编解码器名称1|TO DISK 'xxx'|TO VOLUME 'xxx'][, DELETE|RECOMPRESS 编解码器名称2|TO DISK 'aaa'|TO VOLUME 'bbb'] ... - [WHERE 条件] - [GROUP BY 键表达式 [SET v1 = 聚合函数(v1) [, v2 = 聚合函数(v2) ...]] ] -``` - -TTL 规则的类型可以紧跟在每个 TTL 表达式之后。它会影响在表达式满足条件(到达当前时间)时要执行的操作: - -* `DELETE` - 删除已过期的行(默认操作); -* `RECOMPRESS codec_name` - 使用 `codec_name` 重新压缩数据分片; -* `TO DISK 'aaa'` - 将分片移动到名为 `aaa` 的磁盘; -* `TO VOLUME 'bbb'` - 将分片移动到名为 `bbb` 的卷; -* `GROUP BY` - 聚合已过期的行。 - -`DELETE` 操作可以与 `WHERE` 子句一起使用,根据筛选条件只删除部分已过期的行: - -```sql -TTL time_column + INTERVAL 1 MONTH DELETE WHERE column = 'value' -``` - -`GROUP BY` 表达式必须是表主键的前缀。 - -如果某列既不是 `GROUP BY` 表达式的一部分,又没有在 `SET` 子句中显式设置,那么在结果行中,该列会包含分组行中的任意一个值(就像对该列应用了聚合函数 `any` 一样)。 - -**示例** - -#### 创建带有 `TTL` 的表: {#creating-a-table-with-ttl-1} - -```sql -CREATE TABLE tab -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE, - d + INTERVAL 1 WEEK TO VOLUME 'aaa', - d + INTERVAL 2 WEEK TO DISK 'bbb'; -``` - -#### 修改表的 `TTL`: {#altering-ttl-of-the-table} - -```sql -ALTER TABLE tab - MODIFY TTL d + INTERVAL 1 DAY; -``` - -创建一个表,行在一个月后过期。对已过期的行,仅删除日期为星期一的记录: - -```sql -CREATE TABLE table_with_where -( - d DateTime, - a Int -) -ENGINE = MergeTree -PARTITION BY toYYYYMM(d) -ORDER BY d -TTL d + INTERVAL 1 MONTH DELETE WHERE toDayOfWeek(d) = 1; -``` - -#### 创建对过期行重新压缩的表: {#creating-a-table-where-expired-rows-are-recompressed} - -```sql -CREATE TABLE table_for_recompression -( - d DateTime, - key UInt64, - value String -) ENGINE MergeTree() -ORDER BY tuple() -PARTITION BY key -TTL d + INTERVAL 1 MONTH RECOMPRESS CODEC(ZSTD(17)), d + INTERVAL 1 YEAR RECOMPRESS CODEC(LZ4HC(10)) -SETTINGS min_rows_for_wide_part = 0, min_bytes_for_wide_part = 0; -``` - -创建一个用于聚合已过期行的表。最终结果行中,`x` 包含该分组内的最大值,`y` 为最小值,`d` 为该分组中的任意一个值。 - -```sql -CREATE TABLE table_for_aggregation -( - d DateTime, - k1 Int, - k2 Int, - x Int, - y Int -) -ENGINE = MergeTree -ORDER BY (k1, k2) -TTL d + INTERVAL 1 MONTH GROUP BY k1, k2 SET x = max(x), y = min(y); -``` - -### 删除过期数据 {#mergetree-removing-expired-data} - -TTL 已过期的数据会在 ClickHouse 合并数据部分时被删除。 - -当 ClickHouse 检测到数据已过期时,会执行一次非计划合并。要控制此类合并的频率,可以设置 `merge_with_ttl_timeout`。如果该值过低,可能会触发大量非计划合并,消耗大量资源。 - -在两次合并之间执行 `SELECT` 查询时,可能会读到已过期的数据。为避免这种情况,请在执行 `SELECT` 之前先执行 [OPTIMIZE](/sql-reference/statements/optimize.md) 查询。 - -**另请参阅** - -* [ttl_only_drop_parts](/operations/settings/merge-tree-settings#ttl_only_drop_parts) 设置 - -## 磁盘类型 {#disk-types} - -除了本地块设备之外,ClickHouse 还支持以下存储类型: - -* [`s3` 用于 S3 和 MinIO](#table_engine-mergetree-s3) -* [`gcs` 用于 GCS](/integrations/data-ingestion/gcs/index.md/#creating-a-disk) -* [`blob_storage_disk` 用于 Azure Blob Storage](/operations/storing-data#azure-blob-storage) -* [`hdfs` 用于 HDFS](/engines/table-engines/integrations/hdfs) -* [`web` 用于从 Web 进行只读访问](/operations/storing-data#web-storage) -* [`cache` 用于本地缓存](/operations/storing-data#using-local-cache) -* [`s3_plain` 用于备份到 S3](/operations/backup#backuprestore-using-an-s3-disk) -* [`s3_plain_rewritable` 用于 S3 中的不可变且非复制的表](/operations/storing-data.md#s3-plain-rewritable-storage) - -## 使用多个块设备用于数据存储 {#table_engine-mergetree-multiple-volumes} - -### 简介 {#introduction} - -`MergeTree` 系列表引擎可以将数据存储在多个块设备上。举例来说,当某个表中的数据被隐式划分为「热数据」和「冷数据」时,这会非常有用。最新的数据会被频繁访问,但只占用很小的空间。相反,具有长尾特征的历史数据则很少被访问。如果有多块磁盘可用,可以将「热数据」放在高速磁盘上(例如 NVMe SSD 或内存中),而将「冷数据」放在相对较慢的磁盘上(例如 HDD)。 - -数据片段是 `MergeTree` 引擎表中可移动的最小单元。属于同一数据片段的数据存储在同一块磁盘上。数据片段既可以在后台根据用户设置在磁盘之间移动,也可以通过 [ALTER](/sql-reference/statements/alter/partition) 查询进行移动。 - -### 术语 {#terms} - -* Disk — 挂载到文件系统的块设备。 -* Default disk — 存储 [path](/operations/server-configuration-parameters/settings.md/#path) 服务器设置中所指定路径数据的磁盘。 -* Volume — 由一组相同磁盘按顺序组织而成的集合(类似于 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures))。 -* Storage policy — 卷的集合以及在这些卷之间迁移数据的规则。 - -这些实体的名称可以在系统表 [system.storage_policies](/operations/system-tables/storage_policies) 和 [system.disks](/operations/system-tables/disks) 中找到。要为某个表应用已配置的存储策略之一,请在 `MergeTree` 引擎族表中使用 `storage_policy` 设置。 - -### 配置 {#table_engine-mergetree-multiple-volumes_configure} - -应在 `config.d` 目录下的配置文件中,通过 `` 标签声明磁盘、卷和存储策略。 - -:::tip -也可以在查询的 `SETTINGS` 部分声明磁盘。这对于临时分析时挂载磁盘(例如托管在某个 URL 上的磁盘)非常有用。 -更多详情参见[动态存储](/operations/storing-data#dynamic-configuration)。 -::: - -配置结构: - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - - ... - - - ... - -``` - -标签: - -* `` — 磁盘名称。所有磁盘的名称必须互不相同。 -* `path` — 服务器用于存储数据(`data` 和 `shadow` 目录)的路径,必须以 '/' 结尾。 -* `keep_free_space_bytes` — 需要预留的空闲磁盘空间大小。 - -磁盘定义的顺序无关紧要。 - -存储策略配置标记: - -```xml - - ... - - - - - disk_name_from_disks_configuration - 1073741824 - round_robin - - - - - - - 0.2 - - - - - - - - ... - -``` - -标签: - -* `policy_name_N` — 策略名称。策略名称必须唯一。 -* `volume_name_N` — 卷名。卷名必须唯一。 -* `disk` — 卷中的一个磁盘。 -* `max_data_part_size_bytes` — 可以存储在任意卷磁盘上的数据分片的最大大小。如果预计合并后的分片大小会大于 `max_data_part_size_bytes`,那么该分片会被写入下一个卷。基本上,此功能允许将新的/较小的分片保存在“热”(SSD)卷上,并在它们变大时移动到“冷”(HDD)卷。如果你的策略只有一个卷,请不要使用此设置。 -* `move_factor` — 当可用空间比例低于该系数时,如果存在下一个卷,数据会自动开始移动到下一个卷(默认值为 0.1)。ClickHouse 会按从大到小(降序)对现有分片按大小排序,并选择其总大小足以满足 `move_factor` 条件的分片。如果所有分片的总大小仍不足,则会移动所有分片。 -* `perform_ttl_move_on_insert` — 禁用在插入数据分片(data part)时执行 TTL 移动。默认情况下(启用时),如果插入的分片根据 TTL 移动规则已经过期,它会立即被写入移动规则中指定的卷/磁盘。如果目标卷/磁盘较慢(例如 S3),这可能会显著减慢插入速度。如果禁用,则已过期的数据分片会先写入默认卷,然后紧接着再移动到 TTL 卷。 -* `load_balancing` - 磁盘负载均衡策略,`round_robin` 或 `least_used`。 -* `least_used_ttl_ms` - 配置在所有磁盘上更新可用空间的超时时间(毫秒)(`0` - 始终更新,`-1` - 从不更新,默认值为 `60000`)。注意,如果磁盘只能被 ClickHouse 使用,并且不会进行在线文件系统扩容/缩容,则可以使用 `-1`;在其他所有情况下都不推荐使用,因为最终会导致空间分布不正确。 -* `prefer_not_to_merge` — 你不应该使用此设置。禁用在该卷上合并数据分片(这有害并会导致性能下降)。当启用此设置时(不要这样做),不允许在该卷上进行数据合并(这很糟糕)。这允许你(但你并不需要)控制(如果你想控制什么,你就是在犯错)ClickHouse 如何与慢磁盘交互(但 ClickHouse 更了解情况,所以请不要使用此设置)。 -* `volume_priority` — 定义填充卷的优先级(顺序)。数值越小优先级越高。该参数的取值应为自然数,并且整体上从 1 到 N(给出的最低优先级)连续覆盖,中间不能缺少任何数字。 - * 如果 *所有* 卷都打了标签,则按给定顺序确定它们的优先级。 - * 如果只有 *部分* 卷打了标签,则未打标签的卷具有最低优先级,并按它们在配置中的定义顺序确定优先级。 - * 如果 *没有* 卷打标签,则它们的优先级对应于它们在配置中声明的顺序。 - * 两个卷不能具有相同的优先级值。 - -配置示例: - -```xml - - ... - - - - - disk1 - disk2 - - - - - - - - fast_ssd - 1073741824 - - - disk1 - - - 0.2 - - - - -
- jbod1 -
- - external - -
-
-
- ... -
-``` - -在给定的示例中,`hdd_in_order` 策略实现了 [round-robin](https://en.wikipedia.org/wiki/Round-robin_scheduling)(轮询)方式。因此,该策略仅定义了一个卷(`single`),数据 part 会以循环顺序存储在该卷的所有磁盘上。如果系统中挂载了多个相似的磁盘但没有配置 RAID,此类策略会非常有用。请记住,每个单独的磁盘驱动器都不够可靠,您可能需要通过将复制因子设置为 3 或更多来进行补偿。 - -如果系统中存在不同类型的磁盘,可以使用 `moving_from_ssd_to_hdd` 策略。卷 `hot` 由一个 SSD 磁盘(`fast_ssd`)组成,可以存储在该卷上的单个 part 的最大大小为 1GB。所有大小超过 1GB 的 part 将直接存储在 `cold` 卷上,该卷包含一个 HDD 磁盘 `disk1`。 -另外,一旦磁盘 `fast_ssd` 的使用率超过 80%,后台进程会将数据迁移到 `disk1` 上。 - -在存储策略中,卷的列举顺序非常重要,尤其是在列出的卷中至少有一个未显式设置 `volume_priority` 参数时。 -一旦某个卷被写满,数据会被移动到下一个卷。磁盘的列举顺序同样重要,因为数据会依次存储到这些磁盘上。 - -在创建表时,可以为其应用一个已配置好的存储策略: - -```sql -CREATE TABLE table_with_non_default_policy ( - EventDate Date, - OrderID UInt64, - BannerID UInt64, - SearchPhrase String -) ENGINE = MergeTree -ORDER BY (OrderID, BannerID) -PARTITION BY toYYYYMM(EventDate) -SETTINGS storage_policy = 'moving_from_ssd_to_hdd' -``` - -`default` 存储策略意味着只使用一个卷,该卷仅由 `` 中指定的一块磁盘组成。 -你可以在建表之后通过 [ALTER TABLE ... MODIFY SETTING] 查询更改存储策略,新策略必须包含所有原有的磁盘和卷,并使用相同的名称。 - -用于在后台移动数据部分的线程数量可以通过 [background_move_pool_size](/operations/server-configuration-parameters/settings.md/#background_move_pool_size) 设置进行修改。 - -### 细节 {#details} - -对于 `MergeTree` 表,数据以不同的方式写入磁盘: - -* 作为插入操作(`INSERT` 查询)的结果。 -* 在执行后台合并和[变更](/sql-reference/statements/alter#mutations)时。 -* 从其他副本下载时。 -* 作为分区冻结 [ALTER TABLE ... FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) 的结果。 - -在除变更和分区冻结之外的所有这些情况下,数据部件(part)会根据给定的存储策略被存储在某个卷和磁盘上: - -1. 选择第一个(按定义顺序)同时满足以下条件的卷:该卷拥有足够的磁盘空间来存储该部件(`unreserved_space > current_part_size`),并且允许存储该大小的部件(`max_data_part_size_bytes > current_part_size`)。 -2. 在该卷中,选择这样的磁盘:在上一次用于存储数据块的磁盘之后的那个磁盘,且其可用空间大于该部件的大小(`unreserved_space - keep_free_space_bytes > current_part_size`)。 - -在底层实现中,变更和分区冻结会使用[硬链接](https://en.wikipedia.org/wiki/Hard_link)。不同磁盘之间不支持硬链接,因此在这些情况下,生成的部件会存储在与初始部件相同的磁盘上。 - -在后台,部件会基于空闲空间的大小(`move_factor` 参数),按照配置文件中声明卷的顺序在卷之间移动。 -数据永远不会从最后一个卷迁出,也不会被迁移到第一个卷中。可以使用系统表 [system.part_log](/operations/system-tables/part_log)(字段 `type = MOVE_PART`)和 [system.parts](/operations/system-tables/parts.md)(字段 `path` 和 `disk`)来监控后台移动操作。更详细的信息可以在服务器日志中找到。 - -用户可以通过查询 [ALTER TABLE ... MOVE PART|PARTITION ... TO VOLUME|DISK ...](/sql-reference/statements/alter/partition) 强制将某个部件或分区从一个卷移动到另一个卷,所有对后台操作的限制同样会被考虑在内。该查询会自行发起移动操作,而不会等待后台操作完成。如果没有足够的可用空间,或任一必需条件未满足,用户将收到一条错误信息。 - -数据移动不会影响数据复制。因此,可以在不同副本上的同一张表上指定不同的存储策略。 - -在后台合并和变更完成后,旧部件只会在经过一定时间(`old_parts_lifetime`)后才会被删除。 -在此期间,它们不会被移动到其他卷或磁盘。因此,在这些部件被最终删除之前,它们仍然会被计入磁盘空间占用情况的计算中。 - -用户可以使用 [min_bytes_to_rebalance_partition_over_jbod](/operations/settings/merge-tree-settings.md/#min_bytes_to_rebalance_partition_over_jbod) 设置,将新的大型部件在 [JBOD](https://en.wikipedia.org/wiki/Non-RAID_drive_architectures) 卷的不同磁盘上以均衡的方式进行分配。 - -## 使用外部存储进行数据存储 {#table_engine-mergetree-s3} - -[MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 系列表引擎可以通过类型分别为 `s3`、`azure_blob_storage`、`hdfs` 的磁盘,将数据存储到 `S3`、`AzureBlobStorage`、`HDFS` 中。有关更多详细信息,请参见[配置外部存储选项](/operations/storing-data.md/#configuring-external-storage)。 - -下面是使用类型为 `s3` 的磁盘,将 [S3](https://aws.amazon.com/s3/) 用作外部存储的示例。 - -配置示例: - -```xml - - ... - - - s3 - true - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - -
Authorization: Bearer SOME-TOKEN
- your_base64_encoded_customer_key - your_kms_key_id - your_kms_encryption_context - true - - http://proxy1 - http://proxy2 - - 10000 - 5000 - 10 - 4 - 1000 - /var/lib/clickhouse/disks/s3/ - false -
- - cache - s3 - /var/lib/clickhouse/disks/s3_cache/ - 10Gi - -
- ... -
-``` - -另请参阅[配置外部存储选项](/operations/storing-data.md/#configuring-external-storage)。 - -:::note 缓存配置 -ClickHouse 版本 22.3 至 22.7 使用了不同的缓存配置,如果你正在使用这些版本之一,请参阅[使用本地缓存](/operations/storing-data.md/#using-local-cache)。 -::: - -## 虚拟列 {#virtual-columns} - -* `_part` — 数据部分(part)的名称。 -* `_part_index` — 该数据部分在查询结果中的顺序索引。 -* `_part_starting_offset` — 该数据部分在查询结果中的累计起始行号。 -* `_part_offset` — 该数据部分中的行号。 -* `_part_granule_offset` — 该数据部分中的 granule 编号。 -* `_partition_id` — 分区的名称。 -* `_part_uuid` — 数据部分的唯一标识符(如果启用了 MergeTree 设置 `assign_part_uuids`)。 -* `_part_data_version` — 数据部分的数据版本(最小块号或变更版本)。 -* `_partition_value` — `partition by` 表达式的值(一个元组)。 -* `_sample_factor` — 采样因子(来自查询)。 -* `_block_number` — 插入时为该行分配的原始块号;在启用设置 `enable_block_number_column` 时,在合并过程中会被保留。 -* `_block_offset` — 插入时为该行在块中的原始行号;在启用设置 `enable_block_offset_column` 时,在合并过程中会被保留。 -* `_disk_name` — 用于存储的磁盘名称。 - -## 列统计信息 {#column-statistics} - - - - - -在启用 `set allow_experimental_statistics = 1` 时,对于 `*MergeTree*` 系列表,可以在 `CREATE` 查询的列(columns)部分中声明统计信息。 - -```sql -CREATE TABLE tab -( - a Int64 STATISTICS(TDigest, Uniq), - b Float64 -) -ENGINE = MergeTree -ORDER BY a -``` - -我们也可以使用 `ALTER` 语句来调整统计信息。 - -```sql -ALTER TABLE tab ADD STATISTICS b TYPE TDigest, Uniq; -ALTER TABLE tab DROP STATISTICS a; -``` - -这些轻量级统计信息汇总了列中值的分布情况。统计信息存储在每个数据片段中,并在每次插入时都会更新。 -只有在启用 `set allow_statistics_optimize = 1` 时,它们才会用于 `PREWHERE` 优化。 - -### 可用的列统计类型 {#available-types-of-column-statistics} - -* `MinMax` - - 列的最小值和最大值,用于估计数值列上范围过滤条件的选择性。 - - 语法:`minmax` - -* `TDigest` - - [TDigest](https://github.com/tdunning/t-digest) Sketch 数据结构,用于计算数值列的近似分位数(例如第 90 个百分位数)。 - - 语法:`tdigest` - -* `Uniq` - - [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) Sketch 数据结构,用于估算某列中包含的不同值的数量。 - - 语法:`uniq` - -* `CountMin` - - [CountMin](https://en.wikipedia.org/wiki/Count%E2%80%93min_sketch) Sketch 数据结构,用于对某列中每个值的出现频率进行近似计数。 - - 语法:`countmin` - -### 支持的数据类型 {#supported-data-types} - -| | (U)Int*, Float*, Decimal(*), Date*, Boolean, Enum* | String 或 FixedString | -|-----------|----------------------------------------------------|-----------------------| -| CountMin | ✔ | ✔ | -| MinMax | ✔ | ✗ | -| TDigest | ✔ | ✗ | -| Uniq | ✔ | ✔ | - -### 支持的操作 {#supported-operations} - -| | 等值过滤 (==) | 范围过滤 (`>, >=, <, <=`) | -|-----------|----------------|---------------------------| -| CountMin | ✔ | ✗ | -| MinMax | ✗ | ✔ | -| TDigest | ✗ | ✔ | -| Uniq | ✔ | ✗ | - -## 列级别设置 {#column-level-settings} - -某些 MergeTree 设置可以在列级别进行覆盖: - -* `max_compress_block_size` — 在写入表之前,对未压缩数据块进行压缩时所允许的最大未压缩数据块大小。 -* `min_compress_block_size` — 在写入下一个标记时,为执行压缩所需的最小未压缩数据块大小。 - -示例: - -```sql -CREATE TABLE tab -( - id Int64, - document String SETTINGS (min_compress_block_size = 16777216, max_compress_block_size = 16777216) -) -ENGINE = MergeTree -ORDER BY id -``` - -可以使用 [ALTER MODIFY COLUMN](/sql-reference/statements/alter/column.md) 修改或删除列级设置,例如: - -* 从列定义中删除 `SETTINGS`: - -```sql -ALTER TABLE tab MODIFY COLUMN document REMOVE SETTINGS; -``` - -* 修改某项设置: - -```sql -ALTER TABLE tab MODIFY COLUMN document MODIFY SETTING min_compress_block_size = 8192; -``` - -* 重置一个或多个设置,同时会从该表的 `CREATE` 查询语句中的列表达式里删除相应的设置声明。 - -```sql -ALTER TABLE tab MODIFY COLUMN document RESET SETTING min_compress_block_size; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md deleted file mode 100644 index 976823f2af9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replacingmergetree.md +++ /dev/null @@ -1,237 +0,0 @@ ---- -description: '不同于 MergeTree,它会移除具有相同排序键值(表定义中的 `ORDER BY` 部分,而不是 `PRIMARY KEY`)的重复记录。' -sidebar_label: 'ReplacingMergeTree' -sidebar_position: 40 -slug: /engines/table-engines/mergetree-family/replacingmergetree -title: 'ReplacingMergeTree 表引擎' -doc_type: 'reference' ---- - - - -# ReplacingMergeTree 表引擎 {#replacingmergetree-table-engine} - -该引擎与 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree) 的不同之处在于,它会删除具有相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md)值的重复记录(指表的 `ORDER BY` 子句,而非 `PRIMARY KEY`)。 - -数据去重仅在合并期间发生。合并在后台于未知时间执行,因此无法提前规划,且部分数据可能长时间保持未处理状态。尽管可以通过 `OPTIMIZE` 查询触发一次临时合并,但不要依赖这种方式,因为 `OPTIMIZE` 查询会读写大量数据。 - -因此,`ReplacingMergeTree` 适用于在后台清理重复数据以节省存储空间,但并不能保证数据中完全不存在重复项。 - -:::note -关于 ReplacingMergeTree 的详细指南(包括最佳实践以及性能优化方法)可在[此处](/guides/replacing-merge-tree)查阅。 -::: - - - -## 创建数据表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = ReplacingMergeTree([ver [, is_deleted]]) -[PARTITION BY expr] -[ORDER BY expr] -[PRIMARY KEY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -有关请求参数的说明,请参阅[语句说明](../../../sql-reference/statements/create/table.md)。 - -:::note -行的唯一性是由表的 `ORDER BY` 子句决定的,而不是由 `PRIMARY KEY` 决定。 -::: - - -## ReplacingMergeTree 参数 {#replacingmergetree-parameters} - -### `ver` {#ver} - -`ver` — 带有版本号的列。类型为 `UInt*`、`Date`、`DateTime` 或 `DateTime64`。可选参数。 - -在合并时,`ReplacingMergeTree` 会在所有具有相同排序键的行中只保留一行: - -* 如果未设置 `ver`,则保留选集中最后一行。一次选集是指参与该次合并的多个数据 part 中的一组行。最新创建的 part(最后一次插入)会排在选集的最后。因此,去重后,对于每个唯一排序键,将保留最近一次插入中的最后一行。 -* 如果指定了 `ver`,则保留具有最大版本号的行。如果多行的 `ver` 相同,则对这些行应用“未指定 `ver` 时”的规则,即保留最新插入的那一行。 - -示例: - -```sql --- 没有 ver - 最后插入的'获胜' -CREATE TABLE myFirstReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree -ORDER BY key; - -INSERT INTO myFirstReplacingMT Values (1, 'first', '2020-01-01 01:01:01'); -INSERT INTO myFirstReplacingMT Values (1, 'second', '2020-01-01 00:00:00'); - -SELECT * FROM myFirstReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ second │ 2020-01-01 00:00:00 │ -└─────┴─────────┴─────────────────────┘ - - --- 有 ver - 具有最大 ver 的行'获胜' -CREATE TABLE mySecondReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime -) -ENGINE = ReplacingMergeTree(eventTime) -ORDER BY key; - -INSERT INTO mySecondReplacingMT Values (1, 'first', '2020-01-01 01:01:01'); -INSERT INTO mySecondReplacingMT Values (1, 'second', '2020-01-01 00:00:00'); - -SELECT * FROM mySecondReplacingMT FINAL; - -┌─key─┬─someCol─┬───────────eventTime─┐ -│ 1 │ first │ 2020-01-01 01:01:01 │ -└─────┴─────────┴─────────────────────┘ -``` - -### `is_deleted` {#is_deleted} - -`is_deleted` — 在合并过程中用于确定该行数据表示的是当前状态还是应被删除的列名;`1` 表示“删除”行,`0` 表示“状态”行。 - -列数据类型 — `UInt8`。 - -:::note -只有在使用 `ver` 时才可以启用 `is_deleted`。 - -无论对数据执行何种操作,都应增加版本号。如果插入的两行数据具有相同的版本号,则会保留最后插入的那一行。 - -默认情况下,即使最后一行是删除行,ClickHouse 也会为某个键保留最后一行。这样可以确保将来插入版本号更低的行时,仍然可以安全插入,并且删除行依然会被应用。 - -要永久删除此类删除行,请启用表设置 `allow_experimental_replacing_merge_with_cleanup`,并执行以下任一操作: - -1. 设置表设置项 `enable_replacing_merge_with_cleanup_for_min_age_to_force_merge`、`min_age_to_force_merge_on_partition_only` 和 `min_age_to_force_merge_seconds`。如果某个分区中的所有 part 的存在时间都超过 `min_age_to_force_merge_seconds`,ClickHouse 会将它们全部合并为单个 part 并移除所有删除行。 - -2. 手动运行 `OPTIMIZE TABLE table [PARTITION partition | PARTITION ID 'partition_id'] FINAL CLEANUP`。 - ::: - -示例: - -```sql --- 使用 ver 和 is_deleted -CREATE OR REPLACE TABLE myThirdReplacingMT -( - `key` Int64, - `someCol` String, - `eventTime` DateTime, - `is_deleted` UInt8 -) -ENGINE = ReplacingMergeTree(eventTime, is_deleted) -ORDER BY key -SETTINGS allow_experimental_replacing_merge_with_cleanup = 1; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 0); -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 01:01:01', 1); -``` - - -select * from myThirdReplacingMT final; - -0 行记录。耗时:0.003 秒。 - --- 删除带有 is_deleted 的行 -OPTIMIZE TABLE myThirdReplacingMT FINAL CLEANUP; - -INSERT INTO myThirdReplacingMT Values (1, 'first', '2020-01-01 00:00:00', 0); - -select * from myThirdReplacingMT final; - -┌─key─┬─someCol─┬───────────eventTime─┬─is_deleted─┐ -│ 1 │ first │ 2020-01-01 00:00:00 │ 0 │ -└─────┴─────────┴─────────────────────┴────────────┘ - -``` -``` - - -## 查询子句 {#query-clauses} - -在创建 `ReplacingMergeTree` 表时,需要使用与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)。 - -
- -已弃用的建表方法 - -:::note -不要在新项目中使用此方法,如有可能,请将旧项目迁移到上面所述的方法。 -::: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE [=] ReplacingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [ver]) -``` - -除 `ver` 之外的所有参数与 `MergeTree` 中的含义相同。 - -- `ver` - 版本列。可选参数。相关说明参见上文。 -
- - - -## 查询时去重 & FINAL {#query-time-de-duplication--final} - -在合并阶段,ReplacingMergeTree 使用 `ORDER BY` 列(用于创建表)中的值作为唯一标识来识别重复行,并仅保留版本最高的那一行。不过,这种方式只能在最终状态上接近正确——它并不保证所有重复行都会被去重,因此不应将其作为严格依赖。由于更新和删除记录在查询时仍可能被计算在内,查询结果因此可能不正确。 - -为了获得准确的结果,用户需要在后台合并的基础上,再配合查询时去重以及删除记录的剔除。这可以通过使用 `FINAL` 运算符来实现。例如,考虑以下示例: - -```sql -CREATE TABLE rmt_example -( - `number` UInt16 -) -ENGINE = ReplacingMergeTree -ORDER BY number - -INSERT INTO rmt_example SELECT floor(randUniform(0, 100)) AS number -FROM numbers(1000000000) - -返回 0 行。耗时:19.958 秒。处理了 10 亿行,8.00 GB(每秒 5011 万行,400.84 MB/s)。 -``` - -在不使用 `FINAL` 的情况下进行查询会返回不正确的计数结果(具体数值会因合并情况而异): - -```sql -SELECT count() -FROM rmt_example - -┌─count()─┐ -│ 200 │ -└─────────┘ - -返回 1 行。耗时: 0.002 秒。 -``` - -添加 FINAL 后即可得到正确的结果: - -```sql -SELECT count() -FROM rmt_example -FINAL - -┌─count()─┐ -│ 100 │ -└─────────┘ - -1 行在集合中。耗时: 0.002 秒。 -``` - -若需了解 `FINAL` 的更多细节,包括如何优化 `FINAL` 的性能,建议阅读我们的 [ReplacingMergeTree 详细指南](/guides/replacing-merge-tree)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md deleted file mode 100644 index 35717f42c4d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/replication.md +++ /dev/null @@ -1,348 +0,0 @@ ---- -description: '基于 ClickHouse 中 Replicated* 系列表引擎的数据复制概述' -sidebar_label: 'Replicated*' -sidebar_position: 20 -slug: /engines/table-engines/mergetree-family/replication -title: 'Replicated* 系列表引擎' -doc_type: 'reference' ---- - -# Replicated* 表引擎 {#replicated-table-engines} - -:::note -在 ClickHouse Cloud 中,复制由系统自动管理。请在创建表时不要添加这些参数。例如,在下面的文本中,你应将其替换为: - -```sql -ENGINE = ReplicatedMergeTree( - '/clickhouse/tables/{shard}/table_name', - '{replica}' -) -``` - -包括: - -```sql -ENGINE = ReplicatedMergeTree -``` - -::: - -复制仅支持属于 MergeTree 系列的表: - -* ReplicatedMergeTree -* ReplicatedSummingMergeTree -* ReplicatedReplacingMergeTree -* ReplicatedAggregatingMergeTree -* ReplicatedCollapsingMergeTree -* ReplicatedVersionedCollapsingMergeTree -* ReplicatedGraphiteMergeTree - -复制在单表级别进行,而不是在整个服务器级别进行。单个服务器可以同时存储已复制表和未复制表。 - -复制不依赖于分片。每个分片都有自己独立的复制。 - -`INSERT` 和 `ALTER` 查询的压缩数据会被复制(更多信息,参见 [ALTER](/sql-reference/statements/alter) 文档)。 - -`CREATE`、`DROP`、`ATTACH`、`DETACH` 和 `RENAME` 查询在单个服务器上执行,不会被复制: - -* `CREATE TABLE` 查询会在执行该查询的服务器上创建一个新的可复制表。如果此表已经存在于其他服务器上,它会增加一个新的副本。 -* `DROP TABLE` 查询会删除位于执行该查询的服务器上的该副本。 -* `RENAME` 查询会在其中一个副本上重命名表。换句话说,复制表在不同副本上可以有不同的名称。 - -ClickHouse 使用 [ClickHouse Keeper](/guides/sre/keeper/index.md) 来存储副本的元信息。也可以使用 3.4.5 或更新版本的 ZooKeeper,但推荐使用 ClickHouse Keeper。 - -要使用复制功能,请在服务器配置的 [zookeeper](/operations/server-configuration-parameters/settings#zookeeper) 部分设置相关参数。 - -:::note -不要忽视安全设置。ClickHouse 支持 ZooKeeper 安全子系统的 `digest` [ACL 方案](https://zookeeper.apache.org/doc/current/zookeeperProgrammers.html#sc_ZooKeeperAccessControl)。 -::: - -设置 ClickHouse Keeper 集群地址的示例: - -```xml - - - example1 - 2181 - - - example2 - 2181 - - - example3 - 2181 - - -``` - -ClickHouse 也支持将副本元数据存储在辅助 ZooKeeper 集群中。可以通过在引擎参数中指定 ZooKeeper 集群名称和路径来实现。 -换句话说,它支持将不同表的元数据存储在不同的 ZooKeeper 集群中。 - -辅助 ZooKeeper 集群地址配置示例: - -```xml - - - - example_2_1 - 2181 - - - example_2_2 - 2181 - - - example_2_3 - 2181 - - - - - example_3_1 - 2181 - - - -``` - -要将表元数据存储在辅助 ZooKeeper 集群而不是默认的 ZooKeeper 集群中,可以使用 SQL 创建基于 ReplicatedMergeTree 引擎的表,如下所示: - -```sql -CREATE TABLE table_name ( ... ) ENGINE = ReplicatedMergeTree('zookeeper_name_configured_in_auxiliary_zookeepers:path', 'replica_name') ... -``` - -你可以指定任意现有的 ZooKeeper 集群,系统会在该集群上使用一个目录来存放自身数据(该目录在创建副本表时指定)。 - -如果在配置文件中未配置 ZooKeeper,你将无法创建副本表,且任何已有的副本表都将变为只读。 - - -ZooKeeper 不参与 `SELECT` 查询,因为复制不会影响 `SELECT` 的性能,查询速度与非复制表一样快。对于分布式复制表的查询,ClickHouse 的行为由 [max_replica_delay_for_distributed_queries](/operations/settings/settings.md/#max_replica_delay_for_distributed_queries) 和 [fallback_to_stale_replicas_for_distributed_queries](/operations/settings/settings.md/#fallback_to_stale_replicas_for_distributed_queries) 这两个设置控制。 - -对于每个 `INSERT` 查询,会通过若干事务向 ZooKeeper 添加大约十条记录。(更精确地说,这是针对每个被插入的数据块;一个 INSERT 查询包含一个数据块,或者每 `max_insert_block_size = 1048576` 行一个数据块。)这会导致 `INSERT` 相比非复制表有略高的延迟。但如果遵循建议,以每秒不超过一个 `INSERT` 的批量方式插入数据,就不会产生任何问题。一个用于协调单个 ZooKeeper 集群的整个 ClickHouse 集群,总共每秒可处理数百个 `INSERT`。数据插入的吞吐量(每秒行数)与非复制数据一样高。 - -对于非常大的集群,可以为不同的分片使用不同的 ZooKeeper 集群。不过根据我们的经验,在大约 300 台服务器的生产集群中,还没有发现这种做法是必要的。 - -复制是异步的,并且是多主(multi-master)。`INSERT` 查询(以及 `ALTER`)可以发送到任何可用服务器。数据会先插入到执行查询的服务器上,然后再复制到其他服务器。由于复制是异步的,新插入的数据会在一定延迟后才出现在其他副本上。如果部分副本不可用,则当这些副本重新可用时数据才会被写入。如果某个副本是可用的,延迟就是在网络上传输压缩数据块所需的时间。用于处理复制表后台任务的线程数量可以通过 [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) 这个设置来配置。 - -`ReplicatedMergeTree` 引擎使用单独的线程池来处理复制数据拉取(replicated fetches)。该线程池的大小由 [background_fetches_pool_size](/operations/server-configuration-parameters/settings#background_fetches_pool_size) 设置限制,并且可以在重启服务器时进行调整。 - -默认情况下,INSERT 查询只等待来自一个副本的数据写入确认。如果数据仅成功写入一个副本,而该副本所在的服务器随之完全宕机,则已存储的数据会丢失。要启用从多个副本获取写入确认的能力,请使用 `insert_quorum` 选项。 - -每个数据块的写入是原子的。INSERT 查询会被拆分为最多 `max_insert_block_size = 1048576` 行的数据块。换句话说,如果 `INSERT` 查询中少于 1048576 行,则整个查询是以原子方式完成的。 - -数据块会被去重。对于多次写入同一个数据块(具有相同大小、包含相同行且行顺序相同的数据块),该数据块只会被写入一次。这样设计的原因在于,网络故障时客户端应用可能不知道数据是否已写入数据库,因此可以直接重试 `INSERT` 查询。对于包含相同数据的 INSERT 来说,它被发送到了哪个副本并不重要。`INSERT` 操作是幂等的。去重参数由 [merge_tree](/operations/server-configuration-parameters/settings.md/#merge_tree) 服务器设置控制。 - -在复制过程中,只有要插入的源数据会通过网络传输。后续的数据转换(合并)会在所有副本上以相同方式进行协调并执行。这样可以最大限度地减少网络使用量,这意味着当副本位于不同数据中心时,复制依然工作良好。(注意,将数据复制到不同数据中心是复制的主要目标。) - -同一份数据可以拥有任意数量的副本。根据我们的经验,在生产环境中,一个相对可靠且易于运维的方案是使用双副本(double replication),并在每台服务器上使用 RAID-5 或 RAID-6(在某些情况下使用 RAID-10)。 - -系统会监控副本上的数据同步情况,并且能够在故障后恢复。对于数据差异较小的情况,故障转移是自动完成的;对于数据差异过大的情况(可能表明存在配置错误),故障转移是半自动完成的。 - -## 创建复制表 {#creating-replicated-tables} - -:::note -在 ClickHouse Cloud 中,复制由系统自动处理。 - -使用不带复制参数的 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 创建表。系统会在内部将 [`MergeTree`](/engines/table-engines/mergetree-family/mergetree) 重写为用于复制和数据分布的 [`SharedMergeTree`](/cloud/reference/shared-merge-tree)。 - -请避免使用 `ReplicatedMergeTree` 或指定复制参数,因为复制由平台统一管理。 - -::: - -### Replicated*MergeTree 参数 {#replicatedmergetree-parameters} - -| 参数 | 描述 | -| ------------------ | -------------------------------------------------- | -| `zoo_path` | ClickHouse Keeper 中该表的路径。 | -| `replica_name` | ClickHouse Keeper 中的副本名称。 | -| `other_parameters` | 用于创建该副本引擎版本的参数,例如 `ReplacingMergeTree` 中的 version。 | - -示例: - -```sql -CREATE TABLE table_name -( - EventDate DateTime, - CounterID UInt32, - UserID UInt32, - ver UInt16 -) -ENGINE = ReplicatedReplacingMergeTree('/clickhouse/tables/{layer}-{shard}/table_name', '{replica}', ver) -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID); -``` - -
- 已弃用语法示例 - - ```sql - CREATE TABLE table_name - ( - EventDate DateTime, - CounterID UInt32, - UserID UInt32 - ) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/table_name', '{replica}', EventDate, intHash32(UserID), (CounterID, EventDate, intHash32(UserID), EventTime), 8192); - ``` -
- -如该示例所示,这些参数可以在花括号中包含可替换的占位符。替换后的值取自配置文件的 [macros](/operations/server-configuration-parameters/settings.md/#macros) 部分。 - -示例: - -```xml - - 02 - example05-02-1 - -``` - -在 ClickHouse Keeper 中,指向表的路径对于每个 Replicated 表都必须是唯一的。不同分片上的表应使用不同的路径。 -在本例中,路径由以下部分组成: - -`/clickhouse/tables/` 是公共前缀。建议原样使用该前缀。 - -`{shard}` 将被展开为分片标识符。 - -`table_name` 是该表在 ClickHouse Keeper 中对应节点的名称。通常将其设置为与表名相同是个好主意。之所以要显式定义,是因为与表名不同的是,它在执行 RENAME 查询后不会改变。 -*提示*:也可以在 `table_name` 前加上数据库名,例如 `db_name.table_name`。 - -可以使用两个内置替换 `{database}` 和 `{table}`,它们分别展开为表名和数据库名(除非在 `macros` 部分中定义了这些宏)。因此,ZooKeeper 路径可以指定为 `'/clickhouse/tables/{shard}/{database}/{table}'`。 -在使用这些内置替换时,请谨慎处理表重命名。ClickHouse Keeper 中的路径无法更改,当表被重命名后,宏会展开为不同的路径,表将指向 ClickHouse Keeper 中不存在的路径,并进入只读模式。 - -副本名称用于标识同一张表的不同副本。可以像示例中那样使用服务器名。该名称只需要在每个分片内保持唯一即可。 - -可以显式定义这些参数,而不是使用替换。这在测试以及配置小型集群时可能更为方便。但是,在这种情况下将无法使用分布式 DDL 查询(`ON CLUSTER`)。 - -在处理大型集群时,建议使用替换,因为它们可以降低出错概率。 - -可以在服务器配置文件中为 `Replicated` 表引擎指定默认参数。例如: - -```xml -/clickhouse/tables/{shard}/{database}/{table} -{replica} -``` - -在这种情况下,创建表时可以省略参数: - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree -ORDER BY x; -``` - -它等同于: - -```sql -CREATE TABLE table_name ( - x UInt32 -) ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/{database}/table_name', '{replica}') -ORDER BY x; -``` - -在每个副本上运行 `CREATE TABLE` 查询。此查询会创建一个新的复制表,或者将一个新的副本添加到现有表中。 - -如果是在其他副本上已经存在部分数据之后才添加新的副本,那么在运行该查询后,数据会从其他副本复制到新的副本。换句话说,新副本会与其他副本进行同步。 - -要删除一个副本,请运行 `DROP TABLE`。但这只会删除一个副本——也就是你运行该查询所在服务器上的那个副本。 - - -## 故障后的恢复 {#recovery-after-failures} - -如果在服务器启动时 ClickHouse Keeper 不可用,复制表会切换为只读模式。系统会定期尝试连接 ClickHouse Keeper。 - -如果在执行 `INSERT` 期间 ClickHouse Keeper 不可用,或者在与 ClickHouse Keeper 交互时发生错误,系统会抛出异常。 - -连接到 ClickHouse Keeper 之后,系统会检查本地文件系统中的数据集是否与预期的数据集匹配(ClickHouse Keeper 中保存了这些信息)。如果存在轻微不一致,系统会通过与副本同步数据来解决。 - -如果系统检测到损坏的数据分区片段(文件大小错误)或无法识别的分区片段(已写入文件系统但未在 ClickHouse Keeper 中记录的分区片段),则会将它们移动到 `detached` 子目录中(不会被删除)。任何缺失的分区片段都会从副本中复制。 - -请注意,ClickHouse 不会执行任何破坏性操作,例如自动删除大量数据。 - -在服务器启动时(或与 ClickHouse Keeper 建立新会话时),系统只会检查所有文件的数量和大小。如果文件大小匹配,但中间某处的字节发生了更改,则不会立刻被检测到,只有在尝试为 `SELECT` 查询读取数据时才会发现。此时查询会抛出关于校验和不匹配或压缩块大小不匹配的异常。在这种情况下,数据分区片段会被加入验证队列,并在必要时从副本中复制。 - -如果本地数据集与预期的数据集差异过大,会触发安全机制。服务器会将此写入日志,并拒绝启动。原因是这种情况可能表明存在配置错误,例如某个分片上的副本被误配置为另一个分片上的副本。然而,该机制的阈值设置得相当低,这种情况也可能在正常的故障恢复过程中出现。在这种情况下,数据会通过“按下一个按钮”的方式进行半自动恢复。 - -要开始恢复,请在 ClickHouse Keeper 中创建节点 `/path_to_table/replica_name/flags/force_restore_data`(内容任意),或者运行命令以恢复所有复制表: - -```bash -sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data -``` - -随后重启服务器。服务器在启动时会删除这些标志文件并开始恢复。 - - -## 完全数据丢失后的恢复 {#recovery-after-complete-data-loss} - -如果某台服务器上的所有数据和元数据都丢失了,请按以下步骤进行恢复: - -1. 在该服务器上安装 ClickHouse。在包含分片标识符和副本信息(如使用副本)的配置文件中正确配置 substitutions。 -2. 如果有需要在服务器上手动复制的非复制表(unreplicated tables),请从副本中拷贝其数据(目录 `/var/lib/clickhouse/data/db_name/table_name/`)。 -3. 从副本中拷贝位于 `/var/lib/clickhouse/metadata/` 的表定义。如果在表定义中显式定义了分片或副本标识符,请更正为与此副本相对应。(或者,启动服务器并执行所有本应位于 `/var/lib/clickhouse/metadata/` 目录下 .sql 文件中的 `ATTACH TABLE` 查询。) -4. 要开始恢复,请在 ClickHouse Keeper 中创建节点 `/path_to_table/replica_name/flags/force_restore_data`(内容任意),或者运行命令以恢复所有复制表:`sudo -u clickhouse touch /var/lib/clickhouse/flags/force_restore_data` - -然后启动服务器(如果已经在运行,则重启)。数据会从其他副本中下载。 - -另一种恢复方式是从 ClickHouse Keeper 中删除与丢失副本相关的信息(`/path_to_table/replica_name`),然后按照“[创建复制表](#creating-replicated-tables)”中的说明重新创建该副本。 - -恢复过程中对网络带宽没有限制。如果你同时恢复大量副本,请注意这一点。 - -## 从 MergeTree 转换为 ReplicatedMergeTree {#converting-from-mergetree-to-replicatedmergetree} - -我们使用术语 `MergeTree` 指代 `MergeTree family` 中的所有表引擎,对 `ReplicatedMergeTree` 也采用相同的约定。 - -如果有一个通过手动方式进行复制的 `MergeTree` 表,可以将其转换为复制表。如果已经在某个 `MergeTree` 表中累积了大量数据,并且现在希望启用复制功能,则可能需要这样做。 - -[ATTACH TABLE ... AS REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句允许将已分离的 `MergeTree` 表附加为 `ReplicatedMergeTree`。 - -如果在表的数据目录(对于 `Atomic` 数据库为 `/store/xxx/xxxyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/`)下设置了 `convert_to_replicated` 标记,则在服务器重启时可以自动将 `MergeTree` 表转换为复制表。 -创建一个空的 `convert_to_replicated` 文件,下一次服务器重启时,该表将作为复制表加载。 - -可以使用以下查询获取表的数据路径。如果表有多个数据路径,则必须使用第一个。 - -```sql -SELECT data_paths FROM system.tables WHERE table = 'table_name' AND database = 'database_name'; -``` - -请注意,ReplicatedMergeTree 表将会使用 `default_replica_path` 和 `default_replica_name` 设置的值来创建。 -要在其他副本上创建转换后的表,需要在 `ReplicatedMergeTree` 引擎的第一个参数中显式指定其路径。可以使用以下查询来获取该路径。 - -```sql -SELECT zookeeper_path FROM system.replicas WHERE table = 'table_name'; -``` - -还有一种手动方式可以实现此操作。 - -如果各个副本上的数据不一致,请先对其进行同步,或者在除一个副本外的所有副本上删除这些数据。 - -先重命名现有的 MergeTree 表,然后使用旧表名创建一个 `ReplicatedMergeTree` 表。 -将旧表中的数据移动到新表数据目录下的 `detached` 子目录中(`/var/lib/clickhouse/data/db_name/table_name/`)。 -然后在其中一个副本上运行 `ALTER TABLE ATTACH PARTITION`,将这些分区片段添加到正在使用的工作集中。 - - -## 从 ReplicatedMergeTree 转换为 MergeTree {#converting-from-replicatedmergetree-to-mergetree} - -使用 [ATTACH TABLE ... AS NOT REPLICATED](/sql-reference/statements/attach.md#attach-mergetree-table-as-replicatedmergetree) 语句,在单个服务器上将已分离的 `ReplicatedMergeTree` 表作为 `MergeTree` 表附加。 - -另一种做法需要重启服务器。先创建一个名称不同的 MergeTree 表。将 `ReplicatedMergeTree` 表数据目录中的所有数据移动到新表的数据目录中。然后删除 `ReplicatedMergeTree` 表并重启服务器。 - -如果希望在不启动服务器的情况下移除 `ReplicatedMergeTree` 表: - -- 删除元数据目录(`/var/lib/clickhouse/metadata/`)中对应的 `.sql` 文件。 -- 删除 ClickHouse Keeper 中相应的路径(`/path_to_table/replica_name`)。 - -完成上述操作后,可以启动服务器,创建一个 `MergeTree` 表,将数据移动到该表的数据目录中,然后重启服务器。 - -## 当 ClickHouse Keeper 集群中的元数据丢失或损坏时的恢复 {#recovery-when-metadata-in-the-zookeeper-cluster-is-lost-or-damaged} - -如果 ClickHouse Keeper 中的数据丢失或损坏,可以按照上文所述,将数据迁移到未复制表中以进行保存。 - -**另请参阅** - -- [background_schedule_pool_size](/operations/server-configuration-parameters/settings.md/#background_schedule_pool_size) -- [background_fetches_pool_size](/operations/server-configuration-parameters/settings.md/#background_fetches_pool_size) -- [execute_merges_on_single_replica_time_threshold](/operations/settings/merge-tree-settings#execute_merges_on_single_replica_time_threshold) -- [max_replicated_fetches_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_fetches_network_bandwidth) -- [max_replicated_sends_network_bandwidth](/operations/settings/merge-tree-settings.md/#max_replicated_sends_network_bandwidth) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md deleted file mode 100644 index 8d57f59172d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/summingmergetree.md +++ /dev/null @@ -1,202 +0,0 @@ ---- -description: 'SummingMergeTree 继承自 MergeTree 引擎。其关键特性是在数据部分合并时可以自动对数值数据进行求和。' -sidebar_label: 'SummingMergeTree' -sidebar_position: 50 -slug: /engines/table-engines/mergetree-family/summingmergetree -title: 'SummingMergeTree 表引擎' -doc_type: 'reference' ---- - - - -# SummingMergeTree 表引擎 {#summingmergetree-table-engine} - -该引擎继承自 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree)。不同之处在于,当对 `SummingMergeTree` 表的数据部分进行合并时,ClickHouse 会将所有具有相同主键(更准确地说,是具有相同[排序键](../../../engines/table-engines/mergetree-family/mergetree.md))的多行,替换为一行,其中数值数据类型列的值为这些行的求和结果。如果排序键的设计使得单个键值对应大量行,这种方式可以显著减少存储体积并加速数据查询。 - -我们建议将此引擎与 `MergeTree` 结合使用。在 `MergeTree` 表中存储完整数据,并使用 `SummingMergeTree` 存储聚合后的数据,例如在生成报表时使用。这样的做法可以避免由于主键设计不当而导致有价值数据的丢失。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = SummingMergeTree([columns]) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -有关请求参数的描述,请参阅[请求描述](../../../sql-reference/statements/create/table.md)。 - -### SummingMergeTree 的参数 {#parameters-of-summingmergetree} - -#### 列 {#columns} - -`columns` — 一个包含需要求和的列名的元组(tuple)。可选参数。 -这些列必须是数值类型,且不能出现在分区键或排序键中。 - -如果未指定 `columns`,ClickHouse 会对所有数值类型且不在排序键中的列进行求和。 - -### 查询子句 {#query-clauses} - -在创建 `SummingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)。 - -
- 创建表的已弃用方法 - - :::note - 不要在新项目中使用此方法,并且在可能的情况下,将旧项目切换到上文描述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] SummingMergeTree(date-column [, sampling_expression], (primary, key), index_granularity, [columns]) - ``` - - 除 `columns` 之外的所有参数与 `MergeTree` 中的含义相同。 - - * `columns` — 一个由其值将被求和的列名组成的元组(tuple)。可选参数。说明见上文。 -
- - -## 使用示例 {#usage-example} - -请看下表: - -```sql -CREATE TABLE summtt -( - key UInt32, - value UInt32 -) -ENGINE = SummingMergeTree() -ORDER BY key -``` - -向其中写入数据: - -```sql -INSERT INTO summtt VALUES(1,1),(1,2),(2,1) -``` - -ClickHouse 可能不会对所有行进行完整的求和处理([见下文](#data-processing)),因此我们在查询中使用聚合函数 `sum` 和 `GROUP BY` 子句。 - -```sql -SELECT key, sum(value) FROM summtt GROUP BY key -``` - -```text -┌─key─┬─sum(value)─┐ -│ 2 │ 1 │ -│ 1 │ 3 │ -└─────┴────────────┘ -``` - - -## 数据处理 {#data-processing} - -当数据被插入到表中时,会按原样保存。ClickHouse 会定期合并已插入的数据部分,在此过程中,具有相同主键的行会被求和,并在每个合并结果的数据部分中替换为一行。 - -ClickHouse 合并数据部分的方式可能导致:不同的合并结果数据部分中仍然可能包含具有相同主键的行,即求和可能是不完整的。因此,在执行查询时(`SELECT`),应按照上面的示例使用聚合函数 [sum()](/sql-reference/aggregate-functions/reference/sum) 和 `GROUP BY` 子句。 - -### 求和的通用规则 {#common-rules-for-summation} - -数值数据类型列中的值会被求和。参与求和的列集合由参数 `columns` 定义。 - -如果用于求和的所有列的值都为 0,则该行会被删除。 - -如果某列不在主键中且不参与求和,则会从已有值中任意选取一个值。 - -主键列中的值不会被求和。 - -### AggregateFunction 列中的求和 {#the-summation-in-the-aggregatefunction-columns} - -对于 [AggregateFunction 类型](../../../sql-reference/data-types/aggregatefunction.md) 的列,ClickHouse 的行为类似于 [AggregatingMergeTree](../../../engines/table-engines/mergetree-family/aggregatingmergetree.md) 引擎,会根据该函数进行聚合。 - -### 嵌套结构 {#nested-structures} - -表可以包含以特殊方式处理的嵌套数据结构。 - -如果嵌套表的名称以 `Map` 结尾,并且包含至少两个满足以下条件的列: - -* 第一列是数值类型 `(*Int*, Date, DateTime)` 或字符串类型 `(String, FixedString)`,我们称其为 `key`, -* 其他列是算术类型 `(*Int*, Float32/64)`,我们称其为 `(values...)`, - -则此嵌套表会被解释为 `key => (values...)` 的映射,并且在合并其行时,会以 `key` 进行匹配,将两个数据集中的元素按对应的 `(values...)` 进行求和合并。 - -示例: - -```text -DROP TABLE IF EXISTS nested_sum; -CREATE TABLE nested_sum -( - date Date, - site UInt32, - hitsMap Nested( - browser String, - imps UInt32, - clicks UInt32 - ) -) ENGINE = SummingMergeTree -PRIMARY KEY (date, site); - -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Firefox', 'Opera'], [10, 5], [2, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['Chrome', 'Firefox'], [20, 1], [1, 1]); -INSERT INTO nested_sum VALUES ('2020-01-01', 12, ['IE'], [22], [0]); -INSERT INTO nested_sum VALUES ('2020-01-01', 10, ['Chrome'], [4], [3]); - -OPTIMIZE TABLE nested_sum FINAL; -- 模拟合并 - -SELECT * FROM nested_sum; -┌───────date─┬─site─┬─hitsMap.browser───────────────────┬─hitsMap.imps─┬─hitsMap.clicks─┐ -│ 2020-01-01 │ 10 │ ['Chrome'] │ [4] │ [3] │ -│ 2020-01-01 │ 12 │ ['Chrome','Firefox','IE','Opera'] │ [20,11,22,5] │ [1,3,0,1] │ -└────────────┴──────┴───────────────────────────────────┴──────────────┴────────────────┘ - -SELECT - site, - browser, - impressions, - clicks -FROM -( - SELECT - site, - sumMap(hitsMap.browser, hitsMap.imps, hitsMap.clicks) AS imps_map - FROM nested_sum - GROUP BY site -) -ARRAY JOIN - imps_map.1 AS browser, - imps_map.2 AS impressions, - imps_map.3 AS clicks; - -┌─site─┬─browser─┬─impressions─┬─clicks─┐ -│ 12 │ Chrome │ 20 │ 1 │ -│ 12 │ Firefox │ 11 │ 3 │ -│ 12 │ IE │ 22 │ 0 │ -│ 12 │ Opera │ 5 │ 1 │ -│ 10 │ Chrome │ 4 │ 3 │ -└──────┴─────────┴─────────────┴────────┘ -``` - - -在查询数据时,对 `Map` 进行聚合请使用 [sumMap(key, value)](../../../sql-reference/aggregate-functions/reference/summap.md) 函数。 - -对于嵌套数据结构,在用于求和的列元组中,无需单独指定其各个字段。 - - - -## 相关内容 {#related-content} - -- 博客文章:[在 ClickHouse 中使用聚合函数组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md deleted file mode 100644 index 31d7d8e3314..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/mergetree-family/versionedcollapsingmergetree.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -description: '允许对持续变化的对象状态进行快速写入,并在后台删除旧的对象状态。' -sidebar_label: 'VersionedCollapsingMergeTree' -sidebar_position: 80 -slug: /engines/table-engines/mergetree-family/versionedcollapsingmergetree -title: 'VersionedCollapsingMergeTree 表引擎' -doc_type: 'reference' ---- - - - -# VersionedCollapsingMergeTree 表引擎 {#versionedcollapsingmergetree-table-engine} - -该引擎: - -- 允许快速写入持续变化的对象状态。 -- 在后台删除旧的对象状态,从而显著减少存储占用。 - -详细信息参见 [Collapsing](#table_engines_versionedcollapsingmergetree) 部分。 - -该引擎继承自 [MergeTree](/engines/table-engines/mergetree-family/versionedcollapsingmergetree),并在数据部分合并算法中增加了对行进行折叠的逻辑。`VersionedCollapsingMergeTree` 与 [CollapsingMergeTree](../../../engines/table-engines/mergetree-family/collapsingmergetree.md) 具有相同用途,但使用了不同的折叠算法,允许在多线程环境下以任意顺序插入数据。特别是,`Version` 列有助于在插入顺序不正确时仍能正确折叠行。相比之下,`CollapsingMergeTree` 只允许严格按顺序插入。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = VersionedCollapsingMergeTree(sign, version) -[PARTITION BY expr] -[ORDER BY expr] -[SAMPLE BY expr] -[SETTINGS name=value, ...] -``` - -有关查询参数的详细说明,请参阅[查询说明](../../../sql-reference/statements/create/table.md)。 - -### 引擎参数 {#engine-parameters} - -```sql -VersionedCollapsingMergeTree(sign, version) -``` - -| Parameter | Description | Type | -| --------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `sign` | 行类型列的列名:`1` 表示“state”行,`-1` 表示“cancel”行。 | [`Int8`](/sql-reference/data-types/int-uint) | -| `version` | 对象状态版本列的列名。 | [`Int*`](/sql-reference/data-types/int-uint), [`UInt*`](/sql-reference/data-types/int-uint), [`Date`](/sql-reference/data-types/date), [`Date32`](/sql-reference/data-types/date32), [`DateTime`](/sql-reference/data-types/datetime) 或 [`DateTime64`](/sql-reference/data-types/datetime64) | - -### 查询子句 {#query-clauses} - -在创建 `VersionedCollapsingMergeTree` 表时,需要与创建 `MergeTree` 表时相同的[子句](../../../engines/table-engines/mergetree-family/mergetree.md)。 - -
- 已弃用的建表方法 - - :::note - 请不要在新项目中使用此方法。如有可能,请将旧项目切换为上文所述的方法。 - ::: - - ```sql - CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] - ( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... - ) ENGINE [=] VersionedCollapsingMergeTree(date-column [, samp#table_engines_versionedcollapsingmergetreeling_expression], (primary, key), index_granularity, sign, version) - ``` - - 除 `sign` 和 `version` 之外的所有参数,其含义与 `MergeTree` 中相同。 - - * `sign` — 行类型列的列名:`1` 表示“state”行,`-1` 表示“cancel”行。 - - 列数据类型 — `Int8`。 - - * `version` — 对象状态版本列的列名。 - - 列数据类型应为 `UInt*`。 -
- - -## 折叠 {#table_engines_versionedcollapsingmergetree} - -### 数据 {#data} - -考虑这样一种情况:你需要为某个对象保存不断变化的数据。为某个对象仅保留一行记录,并在有变化时更新这一行是合理的。然而,对于 DBMS 来说,执行 `UPDATE` 操作代价高且速度慢,因为这需要在存储中重写数据。如果你需要快速写入数据,则不适合使用 `UPDATE`,但可以按如下方式顺序写入对象的变更。 - -在写入行时使用 `Sign` 列。如果 `Sign = 1`,表示该行为对象的某个状态(我们称其为“state”行)。如果 `Sign = -1`,表示对具有相同属性的对象状态进行取消(我们称其为“cancel”行)。还需要使用 `Version` 列,它应通过不同的数字标识对象的每一个状态。 - -例如,我们希望统计用户在某个网站上访问了多少页面以及停留了多长时间。在某个时间点,我们写入如下记录来表示用户活动的状态: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -在随后的某个时间点,我们检测到用户活动发生变化,并通过下面这两行将其写入表中。 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -第一行会抵销对象(用户)之前的状态。它应当复制被抵销状态中除 `Sign` 字段以外的所有字段。 - -第二行表示当前状态。 - -因为我们只需要用户活动的最终状态,这些行 - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -这些行可以被删除,从而折叠该对象无效(旧)的状态。`VersionedCollapsingMergeTree` 在合并数据分片时执行这一操作。 - -要了解为什么每次更改需要两行,请参阅[算法](#table_engines-versionedcollapsingmergetree-algorithm)。 - -**使用注意事项** - -1. 写入数据的程序应当记住对象的状态,以便能够对其进行撤销。“Cancel” 行应包含主键字段的副本、“state” 行的版本以及相反的 `Sign`。这会增加初始存储空间占用,但可以实现快速写入。 -2. 列中持续增长的长数组会因为写入负载而降低引擎效率。数据越简单直接,引擎效率越高。 -3. `SELECT` 结果高度依赖于对象变更历史的一致性。在准备要插入的数据时要非常谨慎。对于不一致的数据,你可能会得到不可预测的结果,例如本应为非负指标(如会话深度)的负值。 - -### 算法 {#table_engines-versionedcollapsingmergetree-algorithm} - -当 ClickHouse 合并数据分片时,会删除每一对具有相同主键和版本、但 `Sign` 不同的行。行的顺序无关紧要。 - -当 ClickHouse 插入数据时,会按主键对行进行排序。如果 `Version` 列不在主键中,ClickHouse 会隐式地将其作为最后一个字段加入主键,并使用它进行排序。 - - -## 选择数据 {#selecting-data} - -ClickHouse 不保证具有相同主键的所有行会位于同一个结果数据部件中,甚至不保证在同一台物理服务器上。这对于数据写入以及之后的数据部件合并都成立。此外,ClickHouse 会使用多个线程处理 `SELECT` 查询,因此无法预测结果集中各行的顺序。这意味着,如果需要从 `VersionedCollapsingMergeTree` 表中获取完全“折叠”的数据,就必须进行聚合。 - -要完成折叠,编写带有 `GROUP BY` 子句的查询,并使用能够考虑符号(Sign)的聚合函数。例如,要计算数量,用 `sum(Sign)` 代替 `count()`。要计算某个字段的和,用 `sum(Sign * x)` 代替 `sum(x)`,并添加 `HAVING sum(Sign) > 0`。 - -可以通过这种方式计算的聚合函数包括 `count`、`sum` 和 `avg`。如果对象至少有一个未折叠状态,则可以计算聚合函数 `uniq`。无法计算聚合函数 `min` 和 `max`,因为 `VersionedCollapsingMergeTree` 不保存折叠状态的值历史。 - -如果需要在不进行聚合的情况下以“折叠”的方式提取数据(例如,检查是否存在其最新值满足某些条件的行),可以在 `FROM` 子句中使用 `FINAL` 修饰符。这种方法效率较低,不应在大表上使用。 - - - -## 使用示例 {#example-of-use} - -示例数据: - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 | -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 | -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 | -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -创建表: - -```sql -CREATE TABLE UAct -( - UserID UInt64, - PageViews UInt8, - Duration UInt8, - Sign Int8, - Version UInt8 -) -ENGINE = VersionedCollapsingMergeTree(Sign, Version) -ORDER BY UserID -``` - -写入数据: - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, 1, 1) -``` - -```sql -INSERT INTO UAct VALUES (4324182021466249494, 5, 146, -1, 1),(4324182021466249494, 6, 185, 1, 2) -``` - -我们使用两个 `INSERT` 查询来创建两个不同的数据块。如果我们使用单个查询插入数据,ClickHouse 只会创建一个数据块,并且永远不会执行任何合并。 - -获取数据: - -```sql -SELECT * FROM UAct -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ 1 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ 1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -我们在这里看到了什么?折叠后的数据去了哪里? -我们通过两条 `INSERT` 查询创建了两个数据 part。`SELECT` 查询在两个线程中执行,因此结果中行的顺序是随机的。 -没有发生折叠,是因为这些数据 part 尚未被合并。ClickHouse 会在一个我们无法预知的时间点合并数据 part。 - -这就是我们需要进行聚合的原因: - -```sql -SELECT - UserID, - sum(PageViews * Sign) AS PageViews, - sum(Duration * Sign) AS Duration, - Version -FROM UAct -GROUP BY UserID, Version -HAVING sum(Sign) > 0 -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Version─┐ -│ 4324182021466249494 │ 6 │ 185 │ 2 │ -└─────────────────────┴───────────┴──────────┴─────────┘ -``` - -如果我们不需要聚合,并且希望强制进行折叠,可以在 `FROM` 子句中使用 `FINAL` 修饰符。 - -```sql -SELECT * FROM UAct FINAL -``` - -```text -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┬─Version─┐ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ 2 │ -└─────────────────────┴───────────┴──────────┴──────┴─────────┘ -``` - -这是一种非常低效的数据检索方式。不要在大表上使用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md deleted file mode 100644 index ad12189d81c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/alias.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -description: 'Alias 表引擎创建一个指向另一张表的透明代理。所有操作都会转发至目标表,而别名本身不存储任何数据。' -sidebar_label: 'Alias' -sidebar_position: 5 -slug: /engines/table-engines/special/alias -title: 'Alias 表引擎' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - -# Alias 表引擎 {#alias-table-engine} - - - -`Alias` 引擎会创建指向另一张表的代理。所有读写操作都会被转发到目标表,而别名表本身不存储任何数据,只维护对目标表的引用。 - -:::info -这是一个实验性特性,在未来版本中可能会以不向后兼容的方式发生变更。 -要启用 Alias 表引擎,请通过设置 [allow_experimental_alias_table_engine](/operations/settings/settings#allow_experimental_alias_table_engine)。 -输入命令 `set allow_experimental_alias_table_engine = 1`。 -::: - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_table) -``` - -或者显式地指定数据库名称: - -```sql -CREATE TABLE [db_name.]alias_name -ENGINE = Alias(target_db, target_table) -``` - -:::note -`Alias` 表不支持显式定义列。列会自动从目标表继承,从而确保该别名表始终与目标表的 schema 保持一致。 -::: - - -## 引擎参数 {#engine-parameters} - -- **`target_db(可选)`** — 包含目标表的数据库名称。 -- **`target_table`** — 目标表的名称。 - -## 支持的操作 {#supported-operations} - -`Alias` 表引擎支持所有主要操作。 - -### 目标表上的操作 {#operations-on-target} - -这些操作会被转发到目标表: - -| Operation | Support | Description | -|-----------|---------|-------------| -| `SELECT` | ✅ | 从目标表读取数据 | -| `INSERT` | ✅ | 向目标表写入数据 | -| `INSERT SELECT` | ✅ | 批量向目标表插入数据 | -| `ALTER TABLE ADD COLUMN` | ✅ | 向目标表添加列 | -| `ALTER TABLE MODIFY SETTING` | ✅ | 修改目标表的设置 | -| `ALTER TABLE PARTITION` | ✅ | 在目标表上执行分区操作(DETACH/ATTACH/DROP) | -| `ALTER TABLE UPDATE` | ✅ | 更新目标表中的行(mutation 变更) | -| `ALTER TABLE DELETE` | ✅ | 从目标表删除行(mutation 变更) | -| `OPTIMIZE TABLE` | ✅ | 优化目标表(合并数据片段) | -| `TRUNCATE TABLE` | ✅ | 截断目标表 | - -### 对别名本身的操作 {#operations-on-alias} - -这些操作只会影响别名,**不会**影响目标表: - -| 操作 | 支持情况 | 描述 | -|-----------|---------|-------------| -| `DROP TABLE` | ✅ | 仅删除别名,目标表保持不变 | -| `RENAME TABLE` | ✅ | 仅重命名别名,目标表保持不变 | - -## 使用示例 {#usage-examples} - -### 创建基本别名 {#basic-alias-creation} - -在同一数据库中创建一个简单的别名: - -```sql --- 创建源表 -CREATE TABLE source_data ( - id UInt32, - name String, - value Float64 -) ENGINE = MergeTree -ORDER BY id; - --- 插入数据 -INSERT INTO source_data VALUES (1, 'one', 10.1), (2, 'two', 20.2); - --- 创建别名 -CREATE TABLE data_alias ENGINE = Alias('source_data'); - --- 通过别名查询 -SELECT * FROM data_alias; -``` - -```text -┌─id─┬─name─┬─value─┐ -│ 1 │ one │ 10.1 │ -│ 2 │ two │ 20.2 │ -└────┴──────┴───────┘ -``` - - -### 跨数据库别名 {#cross-database-alias} - -创建一个指向不同数据库中某个表的别名: - -```sql --- 创建数据库 -CREATE DATABASE db1; -CREATE DATABASE db2; - --- 在 db1 中创建源表 -CREATE TABLE db1.events ( - timestamp DateTime, - event_type String, - user_id UInt32 -) ENGINE = MergeTree -ORDER BY timestamp; - --- 在 db2 中创建指向 db1.events 的别名表 -CREATE TABLE db2.events_alias ENGINE = Alias('db1', 'events'); - --- 或使用 database.table 格式 -CREATE TABLE db2.events_alias2 ENGINE = Alias('db1.events'); - --- 两个别名表的功能完全相同 -INSERT INTO db2.events_alias VALUES (now(), 'click', 100); -SELECT * FROM db2.events_alias2; -``` - - -### 通过别名执行写入操作 {#write-operations} - -经由别名的所有写入操作都会被转发到其目标表: - -```sql -CREATE TABLE metrics ( - ts DateTime, - metric_name String, - value Float64 -) ENGINE = MergeTree -ORDER BY ts; - -CREATE TABLE metrics_alias ENGINE = Alias('metrics'); - --- 通过别名插入 -INSERT INTO metrics_alias VALUES - (now(), 'cpu_usage', 45.2), - (now(), 'memory_usage', 78.5); - --- 使用 SELECT 插入 -INSERT INTO metrics_alias -SELECT now(), 'disk_usage', number * 10 -FROM system.numbers -LIMIT 5; - --- 验证目标表中的数据 -SELECT count() FROM metrics; -- 返回 7 -SELECT count() FROM metrics_alias; -- 返回 7 -``` - - -### 表结构修改 {#schema-modification} - -`ALTER` 操作用于修改目标表的表结构: - -```sql -CREATE TABLE users ( - id UInt32, - name String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE users_alias ENGINE = Alias('users'); - --- 通过别名添加列 -ALTER TABLE users_alias ADD COLUMN email String DEFAULT ''; - --- 列已添加至目标表 -DESCRIBE users; -``` - -```text -┌─name──┬─type───┬─default_type─┬─default_expression─┐ -│ id │ UInt32 │ │ │ -│ name │ String │ │ │ -│ email │ String │ DEFAULT │ '' │ -└───────┴────────┴──────────────┴────────────────────┘ -``` - - -### 数据变更 {#data-mutations} - -支持 UPDATE 和 DELETE 操作: - -```sql -CREATE TABLE products ( - id UInt32, - name String, - price Float64, - status String DEFAULT 'active' -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE products_alias ENGINE = Alias('products'); - -INSERT INTO products_alias VALUES - (1, 'item_one', 100.0, 'active'), - (2, 'item_two', 200.0, 'active'), - (3, 'item_three', 300.0, 'inactive'); - --- 通过别名进行更新 -ALTER TABLE products_alias UPDATE price = price * 1.1 WHERE status = 'active'; - --- 通过别名进行删除 -ALTER TABLE products_alias DELETE WHERE status = 'inactive'; - --- 更改将应用到目标表 -SELECT * FROM products ORDER BY id; -``` - -```text -┌─id─┬─name─────┬─price─┬─status─┐ -│ 1 │ item_one │ 110.0 │ active │ -│ 2 │ item_two │ 220.0 │ active │ -└────┴──────────┴───────┴────────┘ -``` - - -### 分区操作 {#partition-operations} - -对于分区表,分区操作将被转发: - -```sql -CREATE TABLE logs ( - date Date, - level String, - message String -) ENGINE = MergeTree -PARTITION BY toYYYYMM(date) -ORDER BY date; - -CREATE TABLE logs_alias ENGINE = Alias('logs'); - -INSERT INTO logs_alias VALUES - ('2024-01-15', 'INFO', 'message1'), - ('2024-02-15', 'ERROR', 'message2'), - ('2024-03-15', 'INFO', 'message3'); - --- 通过别名分离分区 -ALTER TABLE logs_alias DETACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- 返回 2(分区 202402 已分离) - --- 重新附加分区 -ALTER TABLE logs_alias ATTACH PARTITION '202402'; - -SELECT count() FROM logs_alias; -- 返回 3 -``` - - -### 表优化 {#table-optimization} - -对目标表中的分片执行合并优化操作: - -```sql -CREATE TABLE events ( - id UInt32, - data String -) ENGINE = MergeTree -ORDER BY id; - -CREATE TABLE events_alias ENGINE = Alias('events'); - --- 多次插入创建多个部分 -INSERT INTO events_alias VALUES (1, 'data1'); -INSERT INTO events_alias VALUES (2, 'data2'); -INSERT INTO events_alias VALUES (3, 'data3'); - --- 检查部分数量 -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; - --- 通过别名优化 -OPTIMIZE TABLE events_alias FINAL; - --- 部分在目标表中合并 -SELECT count() FROM system.parts -WHERE database = currentDatabase() - AND table = 'events' - AND active; -- 返回 1 -``` - - -### 别名管理 {#alias-management} - -可以分别对别名进行重命名或删除: - -```sql -CREATE TABLE important_data ( - id UInt32, - value String -) ENGINE = MergeTree -ORDER BY id; - -INSERT INTO important_data VALUES (1, 'critical'), (2, 'important'); - -CREATE TABLE old_alias ENGINE = Alias('important_data'); - --- 重命名别名(目标表保持不变) -RENAME TABLE old_alias TO new_alias; - --- 为同一张表创建另一个别名 -CREATE TABLE another_alias ENGINE = Alias('important_data'); - --- 删除一个别名(目标表和其他别名保持不变) -DROP TABLE new_alias; - -SELECT * FROM another_alias; -- 仍然有效 -SELECT count() FROM important_data; -- 数据完整,返回 2 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md deleted file mode 100644 index 92663abc42d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/buffer.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: '在写入时将数据缓存在 RAM(内存)中,并定期刷新到另一张表。在读操作时,会同时从缓冲表和另一张表中读取数据。' -sidebar_label: 'Buffer' -sidebar_position: 120 -slug: /engines/table-engines/special/buffer -title: 'Buffer 表引擎' -doc_type: 'reference' ---- - -# Buffer 表引擎 {#buffer-table-engine} - -在写入时先将要写入的数据缓存在 RAM 中,并定期刷新到另一张表。读取时会同时从缓冲区和另一张表中读取数据。 - -:::note -相较于使用 Buffer 表引擎,更推荐的替代方案是启用[异步插入](/guides/best-practices/asyncinserts.md)。 -::: - -```sql -Buffer(database, table, num_layers, min_time, max_time, min_rows, max_rows, min_bytes, max_bytes [,flush_time [,flush_rows [,flush_bytes]]]) -``` - -### 引擎参数 {#engine-parameters} - -#### `database` {#database} - -`database` – 数据库名称。可以使用 `currentDatabase()` 或其他返回字符串的常量表达式。 - -#### `table` {#table} - -`table` – 要将数据刷新到的目标表。 - -#### `num_layers` {#num_layers} - -`num_layers` – 并行层数。从物理上看,该表将表示为 `num_layers` 个彼此独立的缓冲区。 - -#### `min_time`, `max_time`, `min_rows`, `max_rows`, `min_bytes`, 和 `max_bytes` {#min_time-max_time-min_rows-max_rows-min_bytes-and-max_bytes} - -用于从缓冲区刷新数据的触发条件。 - -### 可选引擎参数 {#optional-engine-parameters} - -#### `flush_time`, `flush_rows`, 和 `flush_bytes` {#flush_time-flush_rows-and-flush_bytes} - -在后台从缓冲区刷新数据的触发条件(省略或设为零表示没有 `flush*` 参数)。 - -当全部 `min*` 条件满足或至少一个 `max*` 条件满足时,数据会从缓冲区刷新并写入目标表。 - -另外,如果至少一个 `flush*` 条件满足,就会在后台发起一次刷新操作。这与 `max*` 不同,因为 `flush*` 允许单独配置后台刷新,以避免对写入 Buffer 表的 `INSERT` 查询增加延迟。 - -#### `min_time`, `max_time`, 和 `flush_time` {#min_time-max_time-and-flush_time} - -从第一次写入缓冲区开始计算的时间(秒)条件。 - -#### `min_rows`, `max_rows`, 和 `flush_rows` {#min_rows-max_rows-and-flush_rows} - -缓冲区中行数的条件。 - -#### `min_bytes`, `max_bytes`, 和 `flush_bytes` {#min_bytes-max_bytes-and-flush_bytes} - -缓冲区中字节数的条件。 - -在写入操作期间,数据会被插入到一个或多个随机选择的缓冲区(由 `num_layers` 配置)。或者,如果要插入的数据块足够大(大于 `max_rows` 或 `max_bytes`),则会直接写入目标表,跳过缓冲区。 - -刷新数据的条件会对每个 `num_layers` 缓冲区分别计算。比如,如果 `num_layers = 16` 且 `max_bytes = 100000000`,则最大内存占用为 1.6 GB。 - -示例: - -```sql -CREATE TABLE merge.hits_buffer AS merge.hits ENGINE = Buffer(merge, hits, 1, 10, 100, 10000, 1000000, 10000000, 100000000) -``` - -创建一个与 `merge.hits` 结构相同并使用 Buffer 引擎的 `merge.hits_buffer` 表。向该表写入数据时,数据会先缓存在 RAM 中,随后再写入 `merge.hits` 表。系统会创建一个单个缓冲区,并在满足以下任一条件时刷新数据: - -* 自上次刷新以来已过去 100 秒(`max_time`),或 -* 已写入 100 万行(`max_rows`),或 -* 已写入 100 MB 的数据(`max_bytes`),或 -* 已经过 10 秒(`min_time`),且已写入 10,000 行(`min_rows`)和 10 MB(`min_bytes`)数据 - -例如,如果只写入了一行数据,那么在 100 秒后,无论如何它都会被刷新。但如果写入了大量行,数据会更早被刷新。 - -当服务器停止,或执行 `DROP TABLE` / `DETACH TABLE` 时,缓冲的数据也会被刷新到目标表。 - -可以为数据库名和表名设置用单引号括起来的空字符串。这表示不存在目标表。在这种情况下,当达到数据刷新条件时,缓冲区会直接被清空。这对于在内存中保留一个数据时间窗口可能很有用。 - -从 Buffer 表中读取时,数据会同时从缓冲区和目标表(如果存在)中进行处理。 -注意,Buffer 表不支持索引。换句话说,缓冲区中的数据会被全量扫描,这在缓冲区很大时可能会比较慢。(对于从属表中的数据,将使用该表所支持的索引。) - -如果 Buffer 表中的列集合与从属表中的列集合不匹配,则只会插入两个表中共同存在的那部分列。 - -如果 Buffer 表与从属表中某一列的数据类型不匹配,则会在服务器日志中记录错误信息,并清空缓冲区。 -在刷新缓冲区时,如果从属表不存在,也会发生同样的情况。 - -:::note -在 2021 年 10 月 26 日之前的版本中,对 Buffer 表执行 ALTER 会导致 `Block structure mismatch` 错误(参见 [#15117](https://github.com/ClickHouse/ClickHouse/issues/15117) 和 [#30565](https://github.com/ClickHouse/ClickHouse/pull/30565)),因此只能先删除 Buffer 表然后重新创建。在尝试对 Buffer 表执行 ALTER 之前,请确认你使用的版本中该错误已被修复。 -::: - -如果服务器异常重启,缓冲区中的数据会丢失。 - -`FINAL` 和 `SAMPLE` 在 Buffer 表上不能正确工作。这些条件会被传递到目标表,但不会用于处理缓冲区中的数据。如果必须使用这些功能,建议只对 Buffer 表执行写入操作,而从目标表进行读取。 - -向 Buffer 表添加数据时,其中一个缓冲区会被加锁。如果此时同时对该表执行读操作,就会产生延迟。 - -插入到 Buffer 表中的数据,可能以不同的顺序、不同的数据块写入从属表。因此,Buffer 表很难被正确用于向 CollapsingMergeTree 写入。为避免问题,你可以将 `num_layers` 设置为 1。 - -如果目标表是复制表(replicated table),在向 Buffer 表写入时,复制表的一些预期特性会丢失。行顺序和数据分片大小的随机变化会导致数据去重失效,这意味着无法对复制表实现可靠的 “exactly once” 写入。 - -由于这些缺点,我们只建议在少数场景中使用 Buffer 表。 - -Buffer 表适用于这样的场景:在单位时间内,从大量服务器接收到的 INSERT 请求过多,并且在插入前无法对数据进行缓冲,从而导致 INSERT 无法以足够快的速度执行。 - -注意,即使对于 Buffer 表,一次只插入一行数据也是没有意义的。这样每秒只能插入几千行,而插入更大的数据块则可以达到每秒超过一百万行。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md deleted file mode 100644 index b01ccb05e74..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/dictionary.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: '`Dictionary` 引擎将字典数据展示为 ClickHouse - 表。' -sidebar_label: 'Dictionary' -sidebar_position: 20 -slug: /engines/table-engines/special/dictionary -title: 'Dictionary 表引擎' -doc_type: 'reference' ---- - -# Dictionary 表引擎 {#dictionary-table-engine} - -`Dictionary` 引擎将 [dictionary](../../../sql-reference/dictionaries/index.md) 数据显示为 ClickHouse 表。 - -## 示例 {#example} - -例如,假设有一个名为 `products` 的字典,其配置如下: - -```xml - - - products - - -
products
- DSN=some-db-server - - - - 300 - 360 - - - - - - - product_id - - - title - String - - - - - -``` - -查询字典中的数据: - -```sql -SELECT - name, - type, - key, - attribute.names, - attribute.types, - bytes_allocated, - element_count, - source -FROM system.dictionaries -WHERE name = 'products' -``` - -```text -┌─name─────┬─type─┬─key────┬─attribute.names─┬─attribute.types─┬─bytes_allocated─┬─element_count─┬─source──────────┐ -│ products │ Flat │ UInt64 │ ['title'] │ ['String'] │ 23065376 │ 175032 │ ODBC: .products │ -└──────────┴──────┴────────┴─────────────────┴─────────────────┴─────────────────┴───────────────┴─────────────────┘ -``` - -你可以使用 [dictGet*](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull) 函数以这种格式获取字典数据。 - -当你需要获取原始数据或执行 `JOIN` 操作时,此视图并不太有用。对于这些场景,你可以使用 `Dictionary` 引擎,它会以表格形式展示字典数据。 - -语法: - -```sql -CREATE TABLE %table_name% (%fields%) engine = Dictionary(%dictionary_name%)` -``` - -用法示例: - -```sql -CREATE TABLE products (product_id UInt64, title String) ENGINE = Dictionary(products); -``` - -好的 - -来看一下表里的内容。 - -```sql -SELECT * FROM products LIMIT 1; -``` - -```text -┌────product_id─┬─title───────────┐ -│ 152689 │ 某个商品 │ -└───────────────┴─────────────────┘ -``` - -**另请参阅** - -* [Dictionary 函数](/sql-reference/table-functions/dictionary) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md deleted file mode 100644 index 4378297fbb4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/distributed.md +++ /dev/null @@ -1,251 +0,0 @@ ---- -description: '使用 Distributed 引擎的表自身不存储任何数据,但允许在多台服务器上执行分布式查询处理。查询读取会自动并行化。在读取过程中,如果远程服务器上存在表索引,则会使用这些索引。' -sidebar_label: 'Distributed' -sidebar_position: 10 -slug: /engines/table-engines/special/distributed -title: 'Distributed 表引擎' -doc_type: 'reference' ---- - - - -# 分布式表引擎 {#distributed-table-engine} - -:::warning 在 Cloud 中使用 Distributed 引擎 -要在 ClickHouse Cloud 中创建分布式表引擎,可以使用 [`remote` 和 `remoteSecure`](../../../sql-reference/table-functions/remote) 表函数。 -在 ClickHouse Cloud 中不能使用 `Distributed(...)` 语法。 -::: - -使用 Distributed 引擎的表本身不存储任何数据,但允许在多个服务器上进行分布式查询处理。 -读操作会自动并行执行。在读取时,如果远程服务器上存在表索引,则会使用这些索引。 - - - -## 创建表 {#distributed-creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) -[SETTINGS name=value, ...] -``` - -### 基于现有表 {#distributed-from-a-table} - -当 `Distributed` 表指向当前服务器上的某个表时,你可以沿用该表的表结构: - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] AS [db2.]name2 ENGINE = Distributed(cluster, database, table[, sharding_key[, policy_name]]) [SETTINGS name=value, ...] -``` - -### 分布式参数 {#distributed-parameters} - -| Parameter | Description | -| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `cluster` | 服务器配置文件中的集群名称 | -| `database` | 远程数据库的名称 | -| `table` | 远程表的名称 | -| `sharding_key` (Optional) | 分片键。
在以下场景中必须指定 `sharding_key`:
  • 对分布式表执行 `INSERT` 时(因为表引擎需要 `sharding_key` 来确定如何拆分数据)。但是,如果启用了 `insert_distributed_one_random_shard` 设置,则 `INSERT` 不需要分片键。
  • 与 `optimize_skip_unused_shards` 配合使用时,因为需要 `sharding_key` 来确定应查询哪些分片
| -| `policy_name` (Optional) | 策略名称,将用于存储后台发送的临时文件 | - -**另请参阅** - -* [distributed_foreground_insert](../../../operations/settings/settings.md#distributed_foreground_insert) 设置 -* [MergeTree](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) 使用示例 - -### 分布式设置 {#distributed-settings} - - -| Setting | Description | Default value | -| ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| `fsync_after_insert` | 在对 Distributed 执行后台插入后,对文件数据执行 `fsync`。保证操作系统已将本次插入的全部数据刷新到**发起节点**磁盘上的文件中。 | `false` | -| `fsync_directories` | 对目录执行 `fsync`。保证操作系统在执行与 Distributed 表后台插入相关的操作之后(插入之后、将数据发送到分片之后等)刷新了目录元数据。 | `false` | -| `skip_unavailable_shards` | 如果为 true,ClickHouse 会静默跳过不可用的分片。分片在以下情况下会被标记为不可用:1)由于连接失败无法访问该分片。2)无法通过 DNS 解析该分片。3)该分片上不存在目标表。 | `false` | -| `bytes_to_throw_insert` | 如果待后台 `INSERT` 处理的压缩字节数超过该值,将抛出异常。`0` 表示不抛出。 | `0` | -| `bytes_to_delay_insert` | 如果待后台 `INSERT` 处理的压缩字节数超过该值,查询将被延迟。`0` 表示不延迟。 | `0` | -| `max_delay_to_insert` | 当存在大量待后台发送的字节时,将数据插入 Distributed 表的最大延迟时间(秒)。 | `60` | -| `background_insert_batch` | 等同于 [`distributed_background_insert_batch`](../../../operations/settings/settings.md#distributed_background_insert_batch) | `0` | -| `background_insert_split_batch_on_failure` | 等同于 [`distributed_background_insert_split_batch_on_failure`](../../../operations/settings/settings.md#distributed_background_insert_split_batch_on_failure) | `0` | -| `background_insert_sleep_time_ms` | 等同于 [`distributed_background_insert_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms) | `0` | -| `background_insert_max_sleep_time_ms` | 等同于 [`distributed_background_insert_max_sleep_time_ms`](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms) | `0` | -| `flush_on_detach` | 在执行 `DETACH` / `DROP` / 服务器关闭时,将数据刷新到远程节点。 | `true` | - -:::note -**耐久性设置**(`fsync_...`): - -* 仅影响后台 `INSERT`(即 `distributed_foreground_insert=false`),当数据首先存储在发起节点磁盘上,随后在后台被发送到各分片时生效。 -* 可能显著降低 `INSERT` 性能 -* 影响将存储在 Distributed 表目录中的数据写入到**接收你插入请求的节点**。如果你需要对底层 MergeTree 表的写入提供保证,请参阅 `system.merge_tree_settings` 中的耐久性设置(`...fsync...`) - -关于**插入限制设置**(`..._insert`)另见: - -* [`distributed_foreground_insert`](../../../operations/settings/settings.md#distributed_foreground_insert) 设置 -* [`prefer_localhost_replica`](/operations/settings/settings#prefer_localhost_replica) 设置 -* `bytes_to_throw_insert` 会在 `bytes_to_delay_insert` 之前处理,因此不应将其设置为小于 `bytes_to_delay_insert` 的值 - ::: - -**示例** - -```sql -CREATE TABLE hits_all AS hits -ENGINE = Distributed(logs, default, hits[, sharding_key[, policy_name]]) -SETTINGS - fsync_after_insert=0, - fsync_directories=0; -``` - -数据将从 `logs` 集群中的所有服务器读取,来源是集群中每台服务器上的 `default.hits` 表。数据不仅会被读取,还会在远程服务器上进行部分处理(在可能的范围内)。例如,对于带有 `GROUP BY` 的查询,数据会先在远程服务器上聚合,然后将聚合函数的中间状态发送到发起查询的服务器,在该服务器上再进一步聚合。 - -在数据库名的位置上,你可以使用返回字符串的常量表达式。例如:`currentDatabase()`。 - - -## 集群 {#distributed-clusters} - -集群是在[服务器配置文件](../../../operations/configuration-files.md)中配置的: - -```xml - - - - - - - - - - - 1 - - shard_01 - - false - - - 1 - example01-01-1 - 9000 - - - example01-01-2 - 9000 - - - - 2 - shard_02 - false - - example01-02-1 - 9000 - - - example01-02-2 - 1 - 9440 - - - - -``` - -这里定义了一个名为 `logs` 的集群,它由两个分片组成,其中每个分片包含两个副本。分片指的是包含不同数据部分的服务器(要读取全部数据,必须访问所有分片)。副本是用于数据冗余的复制服务器(要读取全部数据,可以访问任意一个副本上的数据)。 - -集群名称中不能包含句点(.)字符。 - -参数 `host`、`port`,以及可选的 `user`、`password`、`secure`、`compression`、`bind_host` 需要为每台服务器单独指定: - - -| Parameter | Description | Default Value | -|---------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| -| `host` | 远程服务器的地址。可以使用域名或 IPv4/IPv6 地址。如果指定域名,服务器在启动时会发起一次 DNS 请求,并在服务器运行期间缓存该结果。如果 DNS 请求失败,服务器将无法启动。如果更改了 DNS 记录,需要重启服务器。 | - | -| `port` | 用于客户端通信的 TCP 端口(配置中的 `tcp_port`,通常设置为 9000)。不要与 `http_port` 混淆。 | - | -| `user` | 用于连接远程服务器的用户名。该用户必须拥有连接到指定服务器的权限。访问控制在 `users.xml` 文件中配置。更多信息,参见 [Access rights](../../../guides/sre/user-management/index.md) 部分。 | `default` | -| `password` | 用于连接远程服务器的密码(不会被掩码)。 | '' | -| `secure` | 是否使用安全的 SSL/TLS 连接。通常还需要显式指定端口(默认安全端口为 `9440`)。服务器应监听 `9440`,并正确配置证书。 | `false` | -| `compression` | 是否使用数据压缩。 | `true` | -| `bind_host` | 从当前节点连接到远程服务器时使用的源地址。仅支持 IPv4 地址。适用于需要为 ClickHouse 分布式查询设置源 IP 地址的高级部署场景。 | - | - -在指定副本时,读取时会为每个分片选择一个可用的副本。可以配置负载均衡算法(优先访问哪一个副本)——参见 [load_balancing](../../../operations/settings/settings.md#load_balancing) 设置。如果无法与服务器建立连接,将以较短的超时时间尝试建立连接。如果连接失败,将选择下一个副本,对所有副本依次尝试。如果对所有副本的连接尝试都失败,则会以相同方式重复多次尝试。这有助于提升系统的可靠性,但不能提供完全的容错能力:远程服务器可能会接受连接,但可能无法正常工作,或工作不佳。 - -可以只指定一个分片(在这种情况下,该查询处理应称为 remote,而不是 distributed),也可以指定任意数量的分片。在每个分片内,可以指定从一个到任意数量的副本。对于每个分片,都可以指定不同数量的副本。 - -可以在配置中指定任意数量的集群。 - -要查看集群,请使用 `system.clusters` 表。 - -`Distributed` 引擎允许像操作本地服务器一样操作集群。但是,集群的配置不能动态指定,必须在服务器配置文件中进行配置。通常,集群中的所有服务器会使用相同的集群配置(但这不是强制要求)。来自配置文件的集群会在运行时被更新,无需重启服务器。 - -如果每次都需要向一组未知的分片和副本发送查询,则不需要创建 `Distributed` 表——改用 `remote` 表函数。参见 [Table functions](../../../sql-reference/table-functions/index.md) 部分。 - - - -## 写入数据 {#distributed-writing-data} - -向集群写入数据有两种方法: - -第一种方法是:可以自行定义每台服务器上写入哪些数据,并在每个分片上直接执行写入。换句话说,在 `Distributed` 表所指向的集群远程表上直接执行 `INSERT` 语句。这是最灵活的方案,因为可以使用任意分片方案,即便该方案由于业务领域需求而非常复杂。同时,这也是最优的方案,因为数据可以完全独立地写入不同的分片。 - -第二种方法是:在 `Distributed` 表上执行 `INSERT` 语句。在这种情况下,表会自行将插入的数据分发到各个服务器。要向 `Distributed` 表写入数据,必须为其配置 `sharding_key` 参数(除非只有一个分片)。 - -每个分片都可以在配置文件中定义一个 ``。默认权重为 `1`。数据会按照与分片权重成比例的方式分布到各个分片。会先对所有分片的权重求和,然后用每个分片的权重除以该总和,以确定该分片的占比。例如,如果有两个分片,第一个的权重为 1,第二个的权重为 2,则第一分片会接收三分之一 (1 / 3) 的插入行,第二分片会接收三分之二 (2 / 3)。 - -每个分片还可以在配置文件中定义 `internal_replication` 参数。如果该参数设置为 `true`,写入操作会选择第一个健康副本并将数据写入其中。如果 `Distributed` 表所依赖的底层表是复制表(例如任意 `Replicated*MergeTree` 表引擎),请使用此方式。表的某一个副本会接收写入,然后数据会自动复制到其他副本。 - -如果 `internal_replication` 设置为 `false`(默认值),数据会写入所有副本。在这种情况下,由 `Distributed` 表自身来复制数据。这比使用复制表要差,因为不会检查副本之间的一致性,随着时间推移,各副本中会包含略有不同的数据。 - -为了选择某一行数据要发送到哪个分片,系统会先计算分片表达式,然后将其对所有分片总权重取余。该行会被发送到与余数对应的从 `prev_weights` 到 `prev_weights + weight` 的半开区间内的分片,其中 `prev_weights` 是编号更小的分片的权重总和,`weight` 是当前分片的权重。例如,如果有两个分片,第一个分片的权重为 9,第二个分片的权重为 10,则余数在区间 \[0, 9) 的行会被发送到第一个分片,而余数在区间 \[9, 19) 的行会被发送到第二个分片。 - -分片表达式可以是任何由常量和表列构成且返回整数的表达式。例如,可以使用表达式 `rand()` 来随机分布数据,或者使用 `UserID` 按用户 ID 取余来分布数据(这样单个用户的数据会位于同一个分片上,便于基于用户执行 `IN` 和 `JOIN`)。如果某个列的分布不够均匀,可以将其包裹在哈希函数中,例如 `intHash64(UserID)`。 - -简单的取余分片方案是一种受限的解决方案,并不总是合适。它适用于中等和大规模数据量(数十台服务器),但不适用于超大规模数据量(数百台服务器或更多)。在后一种情况下,应根据业务领域需求设计分片方案,而不是依赖 `Distributed` 表中的记录。 - -在以下情况中,需要特别关注分片方案: - - - -- 在执行需要按特定键进行数据关联(`IN` 或 `JOIN`)的查询时,如果数据按该键进行了分片,就可以使用本地的 `IN` 或 `JOIN` 来代替 `GLOBAL IN` 或 `GLOBAL JOIN`,这样效率要高得多。 -- 在使用大量服务器(数百台或更多)并伴随大量小查询的场景下,例如针对单个客户(如网站、广告主或合作伙伴)数据的查询,为了避免这些小查询影响整个集群,将单个客户的数据放置在单个分片上是合理的选择。或者,可以设置两级分片:将整个集群划分为多个“层”,每一层可以由多个分片组成。单个客户的数据位于单个层中,但可以按需向该层添加分片,数据在该层内的分片之间随机分布。为每个层创建各自的 `Distributed` 表,同时为全局查询创建一个共享的分布式表。 - -数据在后台写入。当向表中执行插入操作时,数据块只是被写入本地文件系统。数据会在后台尽快发送到远程服务器。发送数据的周期由 [distributed_background_insert_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_sleep_time_ms) 和 [distributed_background_insert_max_sleep_time_ms](../../../operations/settings/settings.md#distributed_background_insert_max_sleep_time_ms) 设置进行管理。`Distributed` 引擎会分别发送每个插入数据的文件,但可以通过 [distributed_background_insert_batch](../../../operations/settings/settings.md#distributed_background_insert_batch) 设置启用批量发送文件。该设置能够通过更好地利用本地服务器和网络资源来提升集群性能。应当通过检查表目录中待发送数据文件列表来确认数据是否已成功发送:`/var/lib/clickhouse/data/database/table/`。执行后台任务的线程数量可以通过 [background_distributed_schedule_pool_size](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size) 设置来指定。 - -如果服务器在对 `Distributed` 表执行 `INSERT` 之后宕机或发生了异常重启(例如由于硬件故障),插入的数据可能会丢失。如果在表目录中检测到损坏的数据部分,它会被移动到 `broken` 子目录中并不再使用。 - - - -## 读取数据 {#distributed-reading-data} - -在查询 `Distributed` 表时,`SELECT` 查询会被发送到所有分片,并且无论数据如何分布在这些分片上(可以是完全随机分布),都可以正常工作。添加新分片时,无需将旧数据迁移到其中。相反,你可以通过为该分片指定更大的权重,将新数据写入其中——这样数据分布会略有不均,但查询仍能正确且高效地执行。 - -当启用 `max_parallel_replicas` 选项时,查询处理会在单个分片内的所有副本之间并行化。有关更多信息,请参阅 [max_parallel_replicas](../../../operations/settings/settings.md#max_parallel_replicas) 一节。 - -要了解分布式 `in` 和 `global in` 查询的处理方式,请参阅[此处](/sql-reference/operators/in#distributed-subqueries)的文档。 - - - -## 虚拟列 {#virtual-columns} - -#### _Shard_num {#_shard_num} - -`_shard_num` — 包含表 `system.clusters` 中的 `shard_num` 值。类型:[UInt32](../../../sql-reference/data-types/int-uint.md)。 - -:::note -由于 [`remote`](../../../sql-reference/table-functions/remote.md) 和 [`cluster](../../../sql-reference/table-functions/cluster.md) 表函数在内部会创建临时的 Distributed 表,`_shard_num` 在这些临时表中同样可用。 -::: - -**另请参阅** - -- [虚拟列](../../../engines/table-engines/index.md#table_engines-virtual_columns) 说明 -- [`background_distributed_schedule_pool_size`](/operations/server-configuration-parameters/settings#background_distributed_schedule_pool_size) 设置 -- [`shardNum()`](../../../sql-reference/functions/other-functions.md#shardNum) 和 [`shardCount()`](../../../sql-reference/functions/other-functions.md#shardCount) 函数 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md deleted file mode 100644 index d467f21d67f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/executable.md +++ /dev/null @@ -1,227 +0,0 @@ ---- -description: '`Executable` 和 `ExecutablePool` 表引擎允许你定义一张表,其行由你编写的脚本生成(脚本通过向 **stdout** 写入行)。' -sidebar_label: 'Executable/ExecutablePool' -sidebar_position: 40 -slug: /engines/table-engines/special/executable -title: 'Executable 和 ExecutablePool 表引擎' -doc_type: 'reference' ---- - -# Executable 和 ExecutablePool 表引擎 {#executable-and-executablepool-table-engines} - -`Executable` 和 `ExecutablePool` 表引擎允许你定义一张表,其行由你编写的脚本生成(通过向 **stdout** 写入行)。可执行脚本存储在 `users_scripts` 目录中,并且可以从任意数据源读取数据。 - -* `Executable` 表:每次查询都会运行脚本 -* `ExecutablePool` 表:维护一个持久进程池,并从池中获取进程来执行读取操作 - -你还可以选择性地添加一个或多个输入查询,将其结果以流式方式写入 **stdin**,供脚本读取。 - -## 创建 `Executable` 表 {#creating-an-executable-table} - -`Executable` 表引擎需要两个参数:脚本名称和输入数据的格式。你还可以可选地传入一个或多个输入查询: - -```sql -可执行文件(script_name, format, [input_query...]) -``` - -以下是 `Executable` 表的相关设置: - -* `send_chunk_header` - * Description: 在发送要处理的数据块之前,先发送每个数据块中的行数。此设置可以帮助你更高效地编写脚本,从而提前预分配一些资源 - * Default value: false -* `command_termination_timeout` - * Description: 命令终止超时时间(秒) - * Default value: 10 -* `command_read_timeout` - * Description: 从命令 stdout 读取数据的超时时间(毫秒) - * Default value: 10000 -* `command_write_timeout` - * Description: 向命令 stdin 写入数据的超时时间(毫秒) - * Default value: 10000 - -来看一个示例。下面的 Python 脚本名为 `my_script.py`,保存在 `user_scripts` 文件夹中。它读取一个数字 `i`,并输出 `i` 个随机字符串,每个字符串前面带有一个数字,两者之间用制表符分隔: - -```python -#!/usr/bin/python3 - -import sys -import string -import random - -def main(): - - # 读取输入值 - for number in sys.stdin: - i = int(number) - - # 生成一些随机行 - for id in range(0, i): - letters = string.ascii_letters - random_string = ''.join(random.choices(letters ,k=10)) - print(str(id) + '\t' + random_string + '\n', end='') - - # 将结果刷新到标准输出 - sys.stdout.flush() - -if __name__ == "__main__": - main() -``` - -下面的 `my_executable_table` 是由 `my_script.py` 的输出构建而成的,每次你对 `my_executable_table` 执行一次 `SELECT` 查询时,`my_script.py` 都会生成 10 个随机字符串: - -```sql -CREATE TABLE my_executable_table ( - x UInt32, - y String -) -ENGINE = Executable('my_script.py', TabSeparated, (SELECT 10)) -``` - -创建该表会立即返回,不会触发脚本执行。对 `my_executable_table` 发起查询时会触发脚本执行: - -```sql -SELECT * FROM my_executable_table -``` - -```response -┌─x─┬─y──────────┐ -│ 0 │ BsnKBsNGNH │ -│ 1 │ mgHfBCUrWM │ -│ 2 │ iDQAVhlygr │ -│ 3 │ uNGwDuXyCk │ -│ 4 │ GcFdQWvoLB │ -│ 5 │ UkciuuOTVO │ -│ 6 │ HoKeCdHkbs │ -│ 7 │ xRvySxqAcR │ -│ 8 │ LKbXPHpyDI │ -│ 9 │ zxogHTzEVV │ -└───┴────────────┘ -``` - -## 将查询结果传递给脚本 {#passing-query-results-to-a-script} - -Hacker News 网站的用户会留下评论。Python 提供了一个自然语言处理工具包(`nltk`),其中的 `SentimentIntensityAnalyzer` 可用于判断评论是正面、负面还是中性,并为其打分,分值范围在 -1(极度负面评论)到 1(极度正面评论)之间。我们来创建一个 `Executable` 表,使用 `nltk` 计算 Hacker News 评论的情感。 - -本示例使用了 [此处](/engines/table-engines/mergetree-family/invertedindexes/#hacker-news-dataset) 描述的 `hackernews` 表。`hackernews` 表包含一个类型为 `UInt64` 的 `id` 列,以及一个名为 `comment` 的 `String` 列。我们先从定义 `Executable` 表开始: - -```sql -CREATE TABLE sentiment ( - id UInt64, - sentiment Float32 -) -ENGINE = Executable( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20) -); -``` - -关于 `sentiment` 表的一些说明: - -* 文件 `sentiment.py` 保存在 `user_scripts` 文件夹中(`user_scripts_path` 配置项的默认目录) -* `TabSeparated` 格式意味着我们的 Python 脚本需要生成每行包含以制表符分隔的字段值的原始数据 -* 查询从 `hackernews` 中选取两列。Python 脚本需要从输入的每一行中解析出这两列的值 - -下面是 `sentiment.py` 的定义: - -```python -#!/usr/local/bin/python3.9 - -import sys -import nltk -from nltk.sentiment import SentimentIntensityAnalyzer - -def main(): - sentiment_analyzer = SentimentIntensityAnalyzer() - - while True: - try: - row = sys.stdin.readline() - if row == '': - break - - split_line = row.split("\t") - - id = str(split_line[0]) - comment = split_line[1] - - score = sentiment_analyzer.polarity_scores(comment)['compound'] - print(id + '\t' + str(score) + '\n', end='') - sys.stdout.flush() - except BaseException as x: - break - -if __name__ == "__main__": - main() -``` - -关于这个 Python 脚本的一些说明: - -* 要让它生效,你需要运行 `nltk.downloader.download('vader_lexicon')`。本可以把这行命令写进脚本里,但那样每次对 `sentiment` 表执行查询时都会重新下载——这并不高效 -* 变量 `row` 在每次迭代时都表示以下查询结果集中的一行:`SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20` -* 传入的行是以制表符分隔的,因此我们使用 Python 的 `split` 函数来解析出 `id` 和 `comment` -* `polarity_scores` 的结果是一个包含若干值的 JSON 对象。我们决定只获取这个 JSON 对象中的 `compound` 值 -* 回忆一下,ClickHouse 中的 `sentiment` 表使用的是 `TabSeparated` 格式并包含两列,因此我们的 `print` 函数通过制表符来分隔这两列 - -每次你编写一个从 `sentiment` 表中选取行的查询时,都会执行 `SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20` 查询,并将结果传递给 `sentiment.py`。我们来测试一下: - -```sql -SELECT * -FROM sentiment -``` - -响应如下: - -```response -┌───────id─┬─情感值────┐ -│ 7398199 │ 0.4404 │ -│ 21640317 │ 0.1779 │ -│ 21462000 │ 0 │ -│ 25168863 │ 0 │ -│ 25168978 │ -0.1531 │ -│ 25169359 │ 0 │ -│ 25169394 │ -0.9231 │ -│ 25169766 │ 0.4137 │ -│ 25172570 │ 0.7469 │ -│ 25173687 │ 0.6249 │ -│ 28291534 │ 0 │ -│ 28291669 │ -0.4767 │ -│ 28291731 │ 0 │ -│ 28291949 │ -0.4767 │ -│ 28292004 │ 0.3612 │ -│ 28292050 │ -0.296 │ -│ 28292322 │ 0 │ -│ 28295172 │ 0.7717 │ -│ 28295288 │ 0.4404 │ -│ 21465723 │ -0.6956 │ -└──────────┴───────────┘ -``` - -## 创建 `ExecutablePool` 表 {#creating-an-executablepool-table} - -`ExecutablePool` 的语法与 `Executable` 类似,但 `ExecutablePool` 表有几个特有的相关设置: - -* `pool_size` - * 描述:进程池大小。如果设置为 0,则表示不限制大小 - * 默认值:16 -* `max_command_execution_time` - * 描述:命令的最大执行时间(秒) - * 默认值:10 - -我们可以很容易地将上面的 `sentiment` 表从使用 `Executable` 改为使用 `ExecutablePool`: - -```sql -CREATE TABLE sentiment_pooled ( - id UInt64, - sentiment Float32 -) -ENGINE = ExecutablePool( - 'sentiment.py', - TabSeparated, - (SELECT id, comment FROM hackernews WHERE id > 0 AND comment != '' LIMIT 20000) -) -SETTINGS - pool_size = 4; -``` - -当客户端查询 `sentiment_pooled` 表时,ClickHouse 将按需保留 4 个进程用于处理请求。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md deleted file mode 100644 index 5685af197ad..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/external-data.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: 'ClickHouse 允许在发送 `SELECT` 查询的同时,将处理该查询所需的数据一并发送到服务器。该数据会被放入一张临时表中,并且可以在查询中使用(例如在 `IN` 运算符中)。' -sidebar_label: '用于查询处理的外部数据' -sidebar_position: 130 -slug: /engines/table-engines/special/external-data -title: '用于查询处理的外部数据' -doc_type: 'reference' ---- - -# 用于查询处理的外部数据 {#external-data-for-query-processing} - -ClickHouse 允许在发送 `SELECT` 查询时,将查询处理所需的数据一并发送到服务器。此数据会被放入一张临时表(参见“临时表”一节),并且可以在查询中使用(例如用于 `IN` 运算符)。 - -例如,如果你有一个包含重要用户标识符的文本文件,可以在执行按该列表进行过滤的查询时,将该文件一并上传到服务器。 - -如果你需要在包含大量外部数据的场景下运行多条查询,请不要使用此功能。更好的做法是预先将数据导入数据库。 - -外部数据可以通过命令行客户端(非交互模式)上传,或者通过 HTTP 接口上传。 - -在命令行客户端中,你可以按如下格式指定一个参数部分 - -```bash ---external --file=... [--name=...] [--format=...] [--types=...|--structure=...] -``` - -对于要传输的每个表,你可以有像这样的一段配置。 - -**–external** – 标志一个子句的开始。 -**–file** – 包含表转储的文件路径,或者 -,表示使用标准输入(stdin)。 -只能从标准输入中获取单个表。 - -以下参数是可选的:**–name** – 表名。如果省略,则使用 _data。 -**–format** – 文件中的数据格式。如果省略,则使用 TabSeparated。 - -以下参数中至少需要提供一个:**–types** – 以逗号分隔的列类型列表。例如:`UInt64,String`。列名将为 _1、_2、... -**–structure** – 表结构,格式为 `UserID UInt64`、`URL String`。用于定义列名和列类型。 - -在 'file' 中指定的文件将按 'format' 中指定的格式解析,并使用在 'types' 或 'structure' 中指定的数据类型。该表将被上传到服务器,并可在服务器端作为名为 'name' 的临时表进行访问。 - -示例: - -```bash -$ echo -ne "1\n2\n3\n" | clickhouse-client --query="SELECT count() FROM test.visits WHERE TraficSourceID IN _data" --external --file=- --types=Int8 -849897 -$ cat /etc/passwd | sed 's/:/\t/g' | clickhouse-client --query="SELECT shell, count() AS c FROM passwd GROUP BY shell ORDER BY c DESC" --external --file=- --name=passwd --structure='login String, unused String, uid UInt16, gid UInt16, comment String, home String, shell String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -通过 HTTP 接口时,外部数据以 `multipart/form-data` 格式传输。每个表作为单独的文件发送,表名取自文件名。通过 `query_string` 传递参数 `name_format`、`name_types` 和 `name_structure`,其中 `name` 是这些参数对应的表名。这些参数的含义与使用命令行客户端时相同。 - -示例: - -```bash -$ cat /etc/passwd | sed 's/:/\t/g' > passwd.tsv - -$ curl -F 'passwd=@passwd.tsv;' 'http://localhost:8123/?query=SELECT+shell,+count()+AS+c+FROM+passwd+GROUP+BY+shell+ORDER+BY+c+DESC&passwd_structure=login+String,+unused+String,+uid+UInt16,+gid+UInt16,+comment+String,+home+String,+shell+String' -/bin/sh 20 -/bin/false 5 -/bin/bash 4 -/usr/sbin/nologin 1 -/bin/sync 1 -``` - -在进行分布式查询处理时,会将临时表发送到所有远程服务器。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md deleted file mode 100644 index 1b775e7e30d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/file.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: 'File 表引擎将数据保存在采用受支持的文件格式(`TabSeparated`、`Native` 等)之一的文件中。' -sidebar_label: 'File' -sidebar_position: 40 -slug: /engines/table-engines/special/file -title: 'File 表引擎' -doc_type: 'reference' ---- - - - -# File 表引擎 {#file-table-engine} - -`File` 表引擎将数据保存在一个文件中,文件使用受支持的[文件格式](/interfaces/formats#formats-overview)之一(如 `TabSeparated`、`Native` 等)。 - -使用场景: - -- 将数据从 ClickHouse 导出到文件。 -- 将数据从一种格式转换为另一种格式。 -- 通过编辑磁盘上的文件来更新 ClickHouse 中的数据。 - -:::note -该引擎目前在 ClickHouse Cloud 中不可用,请[改用 S3 表函数](/sql-reference/table-functions/s3.md)。 -::: - - - -## 在 ClickHouse 服务器中的使用 {#usage-in-clickhouse-server} - -```sql -File(Format) -``` - -`Format` 参数指定可用文件格式之一。要执行 `SELECT` 查询,该格式必须支持输入;要执行 `INSERT` 查询,该格式必须支持输出。可用格式列在 [Formats](/interfaces/formats#formats-overview) 部分。 - -ClickHouse 不允许为 `File` 指定文件系统路径。它将使用服务器配置中 [path](../../../operations/server-configuration-parameters/settings.md) 设置所定义的目录。 - -使用 `File(Format)` 创建表时,会在该目录中创建一个空的子目录。将数据写入该表时,数据会被写入该子目录中的 `data.Format` 文件。 - -你可以在服务器文件系统中手动创建这个子目录和文件,然后使用与其名称匹配的表信息对其执行 [ATTACH](../../../sql-reference/statements/attach.md) 操作,这样就可以从该文件中查询数据。 - -:::note -请谨慎使用此功能,因为 ClickHouse 不会跟踪对此类文件的外部修改。通过 ClickHouse 和 ClickHouse 之外同时对其进行写入的结果是未定义的。 -::: - - -## 示例 {#example} - -**1.** 创建 `file_engine_table` 表: - -```sql -CREATE TABLE file_engine_table (name String, value UInt32) ENGINE=File(TabSeparated) -``` - -默认情况下,ClickHouse 会在 `/var/lib/clickhouse/data/default/file_engine_table` 创建目录。 - -**2.** 手动创建文件 `/var/lib/clickhouse/data/default/file_engine_table/data.TabSeparated`,并写入以下内容: - -```bash -$ cat data.TabSeparated -one 1 -two 2 -``` - -**3.** 查询数据: - -```sql -SELECT * FROM file_engine_table -``` - -```text -┌─name─┬─value─┐ -│ one │ 1 │ -│ two │ 2 │ -└──────┴───────┘ -``` - - -## 在 ClickHouse-local 中的用法 {#usage-in-clickhouse-local} - -在 [clickhouse-local](../../../operations/utilities/clickhouse-local.md) 中,File 引擎除了 `Format` 外还可以接收文件路径。可以使用数字或人类可读的名称(例如 `0` 或 `stdin`、`1` 或 `stdout`)来指定默认输入/输出流。可以根据额外的引擎参数或文件扩展名(`gz`、`br` 或 `xz`)来读写压缩文件。 - -**示例:** - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -q "CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); SELECT a, b FROM table; DROP TABLE table" -``` - - -## 实现细节 {#details-of-implementation} - -- 可以并发执行多个 `SELECT` 查询,但各个 `INSERT` 查询之间会互相等待。 -- 支持通过 `INSERT` 查询创建新文件。 -- 如果文件已存在,`INSERT` 会向其中追加新数据。 -- 不支持: - - `ALTER` - - `SELECT ... SAMPLE` - - 索引 - - 复制 - - - -## PARTITION BY {#partition-by} - -`PARTITION BY` — 可选。可以按分区键对数据进行分区,从而生成各自独立的文件。在大多数情况下,不需要分区键;即便需要,一般也不需要比按月更细的分区粒度。分区并不会加速查询(与 ORDER BY 表达式不同)。绝不应该使用过于细粒度的分区。不要按客户端标识符或名称对数据进行分区(相反,应将客户端标识符或名称作为 ORDER BY 表达式中的第一列)。 - -要按月进行分区,使用 `toYYYYMM(date_column)` 表达式,其中 `date_column` 是一个类型为 [Date](/sql-reference/data-types/date.md) 的日期列。此时分区名称采用 `"YYYYMM"` 格式。 - - - -## 虚拟列 {#virtual-columns} - -- `_path` — 文件路径。类型:`LowCardinality(String)`。 -- `_file` — 文件名。类型:`LowCardinality(String)`。 -- `_size` — 文件大小(以字节为单位)。类型:`Nullable(UInt64)`。如果大小未知,则值为 `NULL`。 -- `_time` — 文件的最后修改时间。类型:`Nullable(DateTime)`。如果时间未知,则值为 `NULL`。 - - - -## 设置 {#settings} - -- [engine_file_empty_if_not_exists](/operations/settings/settings#engine_file_empty_if_not_exists) - 允许在文件不存在时返回空结果。默认禁用。 -- [engine_file_truncate_on_insert](/operations/settings/settings#engine_file_truncate_on_insert) - 允许在插入数据前截断文件。默认禁用。 -- [engine_file_allow_create_multiple_files](/operations/settings/settings.md#engine_file_allow_create_multiple_files) - 如果格式带有后缀,允许在每次插入时创建一个新文件。默认禁用。 -- [engine_file_skip_empty_files](/operations/settings/settings.md#engine_file_skip_empty_files) - 允许在读取时跳过空文件。默认禁用。 -- [storage_file_read_method](/operations/settings/settings#engine_file_empty_if_not_exists) - 从存储文件读取数据的方法,可选值:`read`、`pread`、`mmap`。`mmap` 方法不适用于 clickhouse-server(用于 clickhouse-local)。默认值:clickhouse-server 为 `pread`,clickhouse-local 为 `mmap`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md deleted file mode 100644 index d2b5a7fdf9e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/filelog.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -description: '此引擎允许将应用日志文件以记录流的形式进行处理。' -sidebar_label: 'FileLog' -sidebar_position: 160 -slug: /engines/table-engines/special/filelog -title: 'FileLog 表引擎' -doc_type: 'reference' ---- - - - -# FileLog 表引擎 {#filelog-engine} - -该引擎允许将应用程序日志文件作为一条记录流进行处理。 - -`FileLog` 可以: - -- 订阅日志文件。 -- 在新记录追加到已订阅的日志文件时对其进行处理。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = FileLog('path_to_logs', 'format_name') SETTINGS - [poll_timeout_ms = 0,] - [poll_max_batch_size = 0,] - [max_block_size = 0,] - [max_threads = 0,] - [poll_directory_watch_events_backoff_init = 500,] - [poll_directory_watch_events_backoff_max = 32000,] - [poll_directory_watch_events_backoff_factor = 2,] - [handle_error_mode = 'default'] -``` - -引擎参数: - -* `path_to_logs` – 要订阅的日志文件路径。可以是包含日志文件的目录路径,也可以是单个日志文件的路径。注意 ClickHouse 只允许使用 `user_files` 目录内的路径。 -* `format_name` - 记录格式。注意 FileLog 会将文件中的每一行作为一条独立记录进行处理,并非所有数据格式都适用于这种方式。 - -可选参数: - -* `poll_timeout_ms` - 从日志文件进行单次轮询的超时时间。默认值:[stream_poll_timeout_ms](../../../operations/settings/settings.md#stream_poll_timeout_ms)。 -* `poll_max_batch_size` — 单次轮询中可拉取的最大记录数。默认值:[max_block_size](/operations/settings/settings#max_block_size)。 -* `max_block_size` — 单次轮询的最大批大小(以记录数计)。默认值:[max_insert_block_size](../../../operations/settings/settings.md#max_insert_block_size)。 -* `max_threads` - 用于解析文件的最大线程数,默认值为 0,表示线程数为 max(1, physical_cpu_cores / 4)。 -* `poll_directory_watch_events_backoff_init` - 目录监听线程的初始休眠时间。默认值:`500`。 -* `poll_directory_watch_events_backoff_max` - 目录监听线程的最大休眠时间。默认值:`32000`。 -* `poll_directory_watch_events_backoff_factor` - 回退速度,默认为指数回退。默认值:`2`。 -* `handle_error_mode` — FileLog 引擎的错误处理方式。可选值:`default`(如果解析消息失败则抛出异常)、`stream`(异常信息和原始消息将保存在虚拟列 `_error` 和 `_raw_message` 中)。 - - -## 描述 {#description} - -已送达的记录会被自动跟踪,因此日志文件中的每条记录只会被计数一次。 - -`SELECT` 对于读取记录并不是特别有用(除调试外),因为每条记录只能被读取一次。更实用的方式是使用[物化视图](../../../sql-reference/statements/create/view.md)来创建实时处理流水线。要做到这一点: - -1. 使用该引擎创建一个 FileLog 表,并将其视为数据流。 -2. 创建一个具有所需结构的表。 -3. 创建一个物化视图,将来自引擎的数据转换后写入先前创建的表中。 - -当 `MATERIALIZED VIEW` 附加到该引擎时,它会开始在后台收集数据。这使您能够持续地从日志文件中接收记录,并使用 `SELECT` 将其转换为所需的格式。 -一个 FileLog 表可以拥有任意数量的物化视图,它们并不直接从该表读取数据,而是接收新的记录(以数据块的形式接收),这样可以向多个具有不同明细级别(带分组聚合和不带分组聚合)的表中写入数据。 - -示例: - -```sql - CREATE TABLE logs ( - timestamp UInt64, - level String, - message String - ) ENGINE = FileLog('user_files/my_app/app.log', 'JSONEachRow'); - - CREATE TABLE daily ( - day Date, - level String, - total UInt64 - ) ENGINE = SummingMergeTree(day, (day, level), 8192); - - CREATE MATERIALIZED VIEW consumer TO daily - AS SELECT toDate(toDateTime(timestamp)) AS day, level, count() AS total - FROM queue GROUP BY day, level; - - SELECT level, sum(total) FROM daily GROUP BY level; -``` - -要停止接收流式数据或修改转换逻辑,请分离该物化视图: - -```sql - DETACH TABLE consumer; - ATTACH TABLE consumer; -``` - -如果你想使用 `ALTER` 更改目标表,建议先禁用该物化视图,以避免目标表与视图数据之间出现不一致。 - - -## 虚拟列 {#virtual-columns} - -- `_filename` - 日志文件名。数据类型:`LowCardinality(String)`。 -- `_offset` - 在日志文件中的偏移量。数据类型:`UInt64`。 - -当 `handle_error_mode='stream'` 时的额外虚拟列: - -- `_raw_record` - 无法成功解析的原始记录。数据类型:`Nullable(String)`。 -- `_error` - 解析失败时产生的异常消息。数据类型:`Nullable(String)`。 - -注意:只有在解析过程中发生异常时,虚拟列 `_raw_record` 和 `_error` 才会被填充;当消息成功解析时,它们的值始终为 `NULL`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md deleted file mode 100644 index 392d398434f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/generate.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: 'GenerateRandom 表引擎根据给定的表结构生成随机数据。' -sidebar_label: 'GenerateRandom' -sidebar_position: 140 -slug: /engines/table-engines/special/generate -title: 'GenerateRandom 表引擎' -doc_type: 'reference' ---- - - - -# GenerateRandom 表引擎 {#generaterandom-table-engine} - -`GenerateRandom` 表引擎根据给定的表结构生成随机数据。 - -使用示例: - -- 在测试中用于填充可复现的大规模表数据。 -- 为模糊测试(fuzzing)生成随机输入。 - - - -## 在 ClickHouse Server 中的使用 {#usage-in-clickhouse-server} - -```sql -ENGINE = GenerateRandom([random_seed [,max_string_length [,max_array_length]]]) -``` - -`max_array_length` 和 `max_string_length` 参数分别指定在生成的数据中,所有数组或 map 列以及字符串的最大长度。 - -`Generate` 表引擎仅支持 `SELECT` 查询。 - -它支持所有可以存储在表中的 [DataTypes](../../../sql-reference/data-types/index.md),`AggregateFunction` 类型除外。 - - -## 示例 {#example} - -**1.** 创建 `generate_engine_table` 表: - -```sql -CREATE TABLE generate_engine_table (name String, value UInt32) ENGINE = GenerateRandom(1, 5, 3) -``` - -**2.** 查询数据: - -```sql -SELECT * FROM generate_engine_table LIMIT 3 -``` - -```text -┌─name─┬──────value─┐ -│ c4xJ │ 1412771199 │ -│ r │ 1791099446 │ -│ 7#$ │ 124312908 │ -└──────┴────────────┘ -``` - - -## 实现细节 {#details-of-implementation} - -- 不支持: - - `ALTER` - - `SELECT ... SAMPLE` - - `INSERT` - - 索引 - - 复制 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md deleted file mode 100644 index 6323ae0b209..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/index.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '特殊表引擎相关文档' -sidebar_label: '特殊' -sidebar_position: 50 -slug: /engines/table-engines/special/ -title: '特殊表引擎' -doc_type: 'reference' ---- - -# 特殊表引擎 {#special-table-engines} - -表引擎主要分为三大类: - -* [MergeTree 引擎家族](../../../engines/table-engines/mergetree-family/index.md),用于主要的生产场景。 -* [Log 引擎家族](../../../engines/table-engines/log-family/index.md),用于小规模的临时数据。 -* [用于集成的表引擎](../../../engines/table-engines/integrations/index.md)。 - -其余引擎在用途上各不相同,目前尚未归入某个家族,因此被归类到这个「特殊」类别中。 - -{/* 本页的目录表由以下脚本自动生成: - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 该脚本会根据 YAML front matter 字段:slug、description、title 生成目录。 - - 如果发现错误,请直接编辑对应页面的 YAML front matter。 - */ } - -{/*AUTOGENERATED_START*/ } - -| Page | Description | -| ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | -| [Alias table engine](/engines/table-engines/special/alias) | Alias 表引擎创建到另一张表的透明代理。所有操作都会被转发到目标表,而别名本身不存储任何数据。 | -| [Distributed table engine](/engines/table-engines/special/distributed) | 使用 Distributed 引擎的表本身不存储任何数据,但允许在多台服务器上进行分布式查询处理。读取会自动并行化。在读取期间,如果远程服务器上有表索引,则会使用这些索引。 | -| [Dictionary table engine](/engines/table-engines/special/dictionary) | `Dictionary` 引擎将字典数据展示为一张 ClickHouse 表。 | -| [Merge table engine](/engines/table-engines/special/merge) | `Merge` 引擎(不要与 `MergeTree` 混淆)本身不存储数据,但允许同时从任意数量的其他表中读取数据。 | -| [Executable and ExecutablePool table engines](/engines/table-engines/special/executable) | `Executable` 和 `ExecutablePool` 表引擎允许定义一张表,其行由你定义的脚本生成(通过向 **stdout** 写入行)。 | -| [File table engine](/engines/table-engines/special/file) | File 表引擎将数据保存在文件中,文件格式为支持的格式之一(`TabSeparated`、`Native` 等)。 | -| [Null table engine](/engines/table-engines/special/null) | 向 `Null` 表写入时,数据会被忽略。从 `Null` 表读取时,响应为空。 | -| [Set table engine](/engines/table-engines/special/set) | 始终存储在 RAM 中的数据集。它旨在用于 `IN` 运算符的右侧。 | -| [Join table engine](/engines/table-engines/special/join) | 用于 JOIN 操作的可选预构建数据结构。 | -| [URL table engine](/engines/table-engines/special/url) | 从远程 HTTP/HTTPS 服务器读取或写入数据。此引擎类似于 File 引擎。 | -| [View table engine](/engines/table-engines/special/view) | 用于实现视图(更多信息请参见 `CREATE VIEW query`)。它不存储数据,而只存储指定的 `SELECT` 查询。从该表读取时,会运行此查询(并从查询中删除所有不需要的列)。 | -| [Memory table engine](/engines/table-engines/special/memory) | Memory 引擎以未压缩形式将数据存储在 RAM 中。数据以读取时接收到的完全相同形式存储。换句话说,从此表中读取几乎不消耗资源。 | -| [Buffer table engine](/engines/table-engines/special/buffer) | 在 RAM 中对要写入的数据进行缓冲,并定期将其刷新到另一张表中。在读取操作期间,会同时从缓冲区和另一张表中读取数据。 | -| [External data for query processing](/engines/table-engines/special/external-data) | ClickHouse 允许在发送 `SELECT` 查询时,连同查询一起向服务器发送处理该查询所需的数据。此数据会被放入一张临时表中,并可在查询中使用(例如在 `IN` 运算符中)。 | -| [GenerateRandom table engine](/engines/table-engines/special/generate) | GenerateRandom 表引擎根据给定的表结构生成随机数据。 | -| [KeeperMap table engine](/engines/table-engines/special/keeper-map) | 此引擎允许将 Keeper/ZooKeeper 集群用作一致性的键值存储,提供线性化写入和顺序一致的读取。 | -| [FileLog table engine](/engines/table-engines/special/filelog) | 此引擎允许将应用程序日志文件作为记录流进行处理。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md deleted file mode 100644 index 2a243707b83..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/join.md +++ /dev/null @@ -1,176 +0,0 @@ ---- -description: '用于 JOIN 操作的可选预构建数据结构。' -sidebar_label: 'Join' -sidebar_position: 70 -slug: /engines/table-engines/special/join -title: 'Join 表引擎' -doc_type: 'reference' ---- - - - -# Join 表引擎 {#join-table-engine} - -用于 [JOIN](/sql-reference/statements/select/join) 操作的可选预构建数据结构。 - -:::note -在 ClickHouse Cloud 中,如果服务创建时使用的版本早于 25.4,则需要通过 `SET compatibility=25.4` 将兼容性级别设置为不低于 25.4。 -::: - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1] [TTL expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2] [TTL expr2], -) ENGINE = Join(join_strictness, join_type, k1[, k2, ...]) -``` - -请参阅 [CREATE TABLE](/sql-reference/statements/create/table) 查询的详细描述。 - - -## 引擎参数 {#engine-parameters} - -### `join_strictness` {#join_strictness} - -`join_strictness` – [JOIN 严格性](/sql-reference/statements/select/join#supported-types-of-join)。 - -### `join_type` {#join_type} - -`join_type` – [JOIN 类型](/sql-reference/statements/select/join#supported-types-of-join)。 - -### 键列 {#key-columns} - -`k1[, k2, ...]` – 来自 `USING` 子句的键列,`JOIN` 操作基于这些列进行。 - -输入 `join_strictness` 和 `join_type` 参数时不要使用引号,例如 `Join(ANY, LEFT, col1)`。它们必须与该表将要使用的 `JOIN` 操作相匹配。如果参数不匹配,ClickHouse 不会抛出异常,并且可能返回错误的数据。 - - - -## 细节与建议 {#specifics-and-recommendations} - -### 数据存储 {#data-storage} - -`Join` 表的数据始终位于内存(RAM)中。向表中插入行时,ClickHouse 会将数据块写入磁盘上的目录,以便在服务器重启时可以进行恢复。 - -如果服务器未正常重启,磁盘上的数据块可能会丢失或损坏。在这种情况下,可能需要手动删除包含损坏数据的文件。 - -### 查询与插入数据 {#selecting-and-inserting-data} - -可以使用 `INSERT` 查询向 `Join` 引擎表中添加数据。如果表在创建时使用了 `ANY` 严格性,则会忽略重复键的数据。在 `ALL` 严格性下,所有行都会被添加。 - -`Join` 引擎表的主要使用场景如下: - -- 将表放在 `JOIN` 子句的右侧。 -- 调用 [joinGet](/sql-reference/functions/other-functions.md/#joinGet) 函数,以与从字典中提取数据相同的方式从表中提取数据。 - -### 删除数据 {#deleting-data} - -针对 `Join` 引擎表的 `ALTER DELETE` 查询是作为[变更(mutation)](/sql-reference/statements/alter/index.md#mutations)实现的。`DELETE` 变更会读取过滤后的数据,并覆盖内存和磁盘中的数据。 - -### 限制与设置 {#join-limitations-and-settings} - -创建表时,会应用以下设置: - -#### `join_use_nulls` {#join_use_nulls} - -[join_use_nulls](/operations/settings/settings.md/#join_use_nulls) - -#### `max_rows_in_join` {#max_rows_in_join} - -[max_rows_in_join](/operations/settings/settings#max_rows_in_join) - -#### `max_bytes_in_join` {#max_bytes_in_join} - -[max_bytes_in_join](/operations/settings/settings#max_bytes_in_join) - -#### `join_overflow_mode` {#join_overflow_mode} - -[join_overflow_mode](/operations/settings/settings#join_overflow_mode) - -#### `join_any_take_last_row` {#join_any_take_last_row} - -[join_any_take_last_row](/operations/settings/settings.md/#join_any_take_last_row) -#### `join_use_nulls` {#join_use_nulls-1} - -#### Persistent {#persistent} - -为 Join 和 [Set](/engines/table-engines/special/set.md) 表引擎禁用持久化。 - -降低 I/O 开销。适用于追求性能但不需要持久化的场景。 - -可能的取值: - -- 1 — 启用。 -- 0 — 禁用。 - -默认值:`1`。 - -`Join` 引擎表不能用于 `GLOBAL JOIN` 操作。 - -`Join` 引擎允许在 `CREATE TABLE` 语句中指定 [join_use_nulls](/operations/settings/settings.md/#join_use_nulls) 设置。[SELECT](/sql-reference/statements/select/index.md) 查询必须使用相同的 `join_use_nulls` 值。 - - - -## 用法示例 {#example} - -创建左侧的表: - -```sql -CREATE TABLE id_val(`id` UInt32, `val` UInt32) ENGINE = TinyLog; -``` - -```sql -INSERT INTO id_val VALUES (1,11)(2,12)(3,13); -``` - -创建右表(`Join` 右侧的表): - -```sql -CREATE TABLE id_val_join(`id` UInt32, `val` UInt8) ENGINE = Join(ANY, LEFT, id); -``` - -```sql -INSERT INTO id_val_join VALUES (1,21)(1,22)(3,23); -``` - -联接这些表: - -```sql -SELECT * FROM id_val ANY LEFT JOIN id_val_join USING (id); -``` - -```text -┌─id─┬─val─┬─id_val_join.val─┐ -│ 1 │ 11 │ 21 │ -│ 2 │ 12 │ 0 │ -│ 3 │ 13 │ 23 │ -└────┴─────┴─────────────────┘ -``` - -作为替代方案,你可以通过指定连接键的值,从 `Join` 表中检索数据: - -```sql -SELECT joinGet('id_val_join', 'val', toUInt32(1)); -``` - -```text -┌─joinGet('id_val_join', 'val', toUInt32(1))─┐ -│ 21 │ -└────────────────────────────────────────────┘ -``` - -从 `Join` 表中删除一行记录: - -```sql -ALTER TABLE id_val_join DELETE WHERE id = 3; -``` - -```text -┌─id─┬─val─┐ -│ 1 │ 21 │ -└────┴─────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md deleted file mode 100644 index 1a720510d55..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/keepermap.md +++ /dev/null @@ -1,125 +0,0 @@ ---- -description: '此引擎允许你将 Keeper/ZooKeeper 集群用作具有线性化写入和顺序一致性读取的一致性键值存储。' -sidebar_label: 'KeeperMap' -sidebar_position: 150 -slug: /engines/table-engines/special/keeper-map -title: 'KeeperMap 表引擎' -doc_type: 'reference' ---- - - - -# KeeperMap 表引擎 {#keepermap-table-engine} - -此引擎允许你将 Keeper/ZooKeeper 集群用作一致性的键值存储,支持线性化写入和顺序一致的读取。 - -要启用 KeeperMap 存储引擎,你需要通过 `` 配置项定义用于存放表的 ZooKeeper 路径。 - -例如: - -```xml - - /keeper_map_tables - -``` - -其中 `path` 可以是任意其他有效的 ZooKeeper 路径。 - - -## 创建数据表 {#creating-a-table} - -```sql -CREATE TABLE [IF NOT EXISTS] [db.]table_name [ON CLUSTER cluster] -( - name1 [type1] [DEFAULT|MATERIALIZED|ALIAS expr1], - name2 [type2] [DEFAULT|MATERIALIZED|ALIAS expr2], - ... -) ENGINE = KeeperMap(root_path, [keys_limit]) PRIMARY KEY(primary_key_name) -``` - -引擎参数: - -* `root_path` - 存储 `table_name` 的 ZooKeeper 路径。\ - 此路径不应包含在 `` 配置中定义的前缀,因为该前缀会自动追加到 `root_path`。\ - 另外,还支持 `auxiliary_zookeeper_cluster_name:/some/path` 的格式,其中 `auxiliary_zookeeper_cluster_name` 是在 `` 配置中定义的 ZooKeeper 集群名称。\ - 默认情况下,使用在 `` 配置中定义的 ZooKeeper 集群。 -* `keys_limit` - 表中允许存在的键数量。\ - 该限制是软限制,在某些极端情况下,表中可能会出现更多的键。 -* `primary_key_name` – 列表中的任意列名。 -* 必须指定 `primary key`,并且主键只支持单列。主键将以二进制形式序列化为 ZooKeeper 中的 `node name`(节点名)。 -* 主键以外的列将按对应顺序序列化为二进制,并作为由序列化键定义的结果节点的值进行存储。 -* 带有 `equals` 或 `in` 键过滤条件的查询将被优化为从 `Keeper` 进行多键查找,否则将获取所有值。 - -示例: - -```sql -CREATE TABLE keeper_map_table -( - `key` String, - `v1` UInt32, - `v2` String, - `v3` Float32 -) -ENGINE = KeeperMap('/keeper_map_table', 4) -PRIMARY KEY key -``` - -使用 - -```xml - - /keeper_map_tables - -``` - -每个值(即 `(v1, v2, v3)` 的二进制序列化结果)都会存储在 `Keeper` 中的 `/keeper_map_tables/keeper_map_table/data/serialized_key` 路径下。 -另外,键的数量有一个软上限,目前为 4 个。 - -如果在同一个 ZooKeeper 路径上创建了多个表,那么只要仍然至少有 1 个表在使用该路径,其对应的值就会被持久化。\ -因此,在创建表时可以使用 `ON CLUSTER` 子句,在多个 ClickHouse 实例之间共享这些数据。\ -当然,也可以在彼此无关联的 ClickHouse 实例上,手动使用相同路径运行 `CREATE TABLE`,以达到相同的数据共享效果。 - - -## 支持的操作 {#supported-operations} - -### 插入 {#inserts} - -当向 `KeeperMap` 插入新行时,如果键不存在,则会为该键创建一个新条目。 -如果键已存在且 `keeper_map_strict_mode` 被设为 `true`,则会抛出异常;否则,该键对应的值将被覆盖。 - -示例: - -```sql -INSERT INTO keeper_map_table VALUES ('some key', 1, 'value', 3.2); -``` - -### 删除 {#deletes} - -可以使用 `DELETE` 查询或 `TRUNCATE` 删除行。 -如果键存在,并且将 `keeper_map_strict_mode` 设置为 `true`,则只有在能够以原子方式执行时,获取和删除数据才会成功。 - -```sql -DELETE FROM keeper_map_table WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -ALTER TABLE keeper_map_table DELETE WHERE key LIKE 'some%' AND v1 > 1; -``` - -```sql -TRUNCATE TABLE keeper_map_table; -``` - -### 更新 {#updates} - -可以使用 `ALTER TABLE` 查询来更新值。主键不可更新。 -如果将 `keeper_map_strict_mode` 设置为 `true`,只有在以原子方式执行时,读取和更新数据才会成功。 - -```sql -ALTER TABLE keeper_map_table UPDATE v1 = v1 * 10 + 2 WHERE key LIKE 'some%' AND v3 > 3.1; -``` - - -## 相关内容 {#related-content} - -- 博客文章:[使用 ClickHouse 和 Hex 构建实时分析应用](https://clickhouse.com/blog/building-real-time-applications-with-clickhouse-and-hex-notebook-keeper-engine) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md deleted file mode 100644 index 91317761af5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/memory.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -description: 'Memory 引擎以未压缩形式将数据存储在内存(RAM)中。数据的存储形式与写入时接收到的形式完全相同。换句话说,从该表中读取数据没有任何额外开销。' -sidebar_label: 'Memory' -sidebar_position: 110 -slug: /engines/table-engines/special/memory -title: 'Memory 表引擎' -doc_type: 'reference' ---- - - - -# Memory 表引擎 {#memory-table-engine} - -:::note -在 ClickHouse Cloud 上使用 Memory 表引擎时,数据出于设计原因不会在所有节点之间复制。若要保证所有查询都被路由到同一节点,并使 Memory 表引擎按预期工作,可以采用以下任一方式: -- 在同一个会话中执行所有操作 -- 使用通过 TCP 或原生接口(支持粘性连接)的客户端,例如 [clickhouse-client](/interfaces/cli) -::: - -Memory 引擎以未压缩形式将数据存储在 RAM 中。数据以与读取时接收到的完全相同的形式存储。换句话说,从该表中读取几乎没有任何开销。 -并发数据访问是同步的。锁的持有时间非常短:读写操作不会互相阻塞。 -不支持索引。读取会被并行化。 - -在简单查询上可以达到最高性能(超过 10 GB/sec),因为没有磁盘读取、解压缩或数据反序列化的开销。(需要指出的是,在很多情况下,MergeTree 引擎的性能几乎同样高。) -当服务器重启时,表中的数据会消失,表将变为空表。 -通常情况下,没有使用该表引擎的充分理由。不过,它可以用于测试,以及在行数相对较少(大约不超过 100,000,000 行)且对极致性能有要求的任务中。 - -Memory 引擎被系统用于带有外部查询数据的临时表(参见“External data for processing a query”一节),以及实现 `GLOBAL IN`(参见“IN operators”一节)。 - -可以指定上限和下限来限制 Memory 引擎表的大小,从而有效地使其充当一个环形缓冲区(参见 [Engine Parameters](#engine-parameters))。 - - - -## 引擎参数 {#engine-parameters} - -- `min_bytes_to_keep` — 当内存表设置了大小上限时需要保留的最小字节数。 - - 默认值:`0` - - 需同时设置 `max_bytes_to_keep` -- `max_bytes_to_keep` — 内存表中允许保留的最大字节数,最旧的行会在每次插入时被删除(即环形缓冲区)。当添加一个较大的数据块时,如果要删除的最旧一批行的总大小低于 `min_bytes_to_keep` 限制,则最大字节数可以暂时超过该限制。 - - 默认值:`0` -- `min_rows_to_keep` — 当内存表设置了大小上限时需要保留的最小行数。 - - 默认值:`0` - - 需同时设置 `max_rows_to_keep` -- `max_rows_to_keep` — 内存表中允许保留的最大行数,最旧的行会在每次插入时被删除(即环形缓冲区)。当添加一个较大的数据块时,如果要删除的最旧一批行的行数低于 `min_rows_to_keep` 限制,则最大行数可以暂时超过该限制。 - - 默认值:`0` -- `compress` - 是否在内存中对数据进行压缩。 - - 默认值:`false` - - - -## 使用说明 {#usage} - -**初始化配置** - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**修改设置** - -```sql -ALTER TABLE memory MODIFY SETTING min_rows_to_keep = 100, max_rows_to_keep = 1000; -``` - -**注意:** `bytes` 和 `rows` 的封顶参数可以同时设置,但会始终遵守由 `max` 和 `min` 所定义的最低限制。 - - -## 示例 {#examples} - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_bytes_to_keep = 4096, max_bytes_to_keep = 16384; - -/* 1. 测试最旧的数据块因最小阈值限制而不被删除 - 3000 行 */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 8'192 bytes - -/* 2. 添加不会被删除的数据块 */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 1'024 bytes - -/* 3. 测试最旧的数据块被删除 - 9216 字节 - 1100 */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 8'192 bytes - -/* 4. 验证超大数据块覆盖所有现有数据 */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 65'536 bytes - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` - -另外,对于行: - -```sql -CREATE TABLE memory (i UInt32) ENGINE = Memory SETTINGS min_rows_to_keep = 4000, max_rows_to_keep = 10000; - -/* 1. 测试最旧的数据块因最小阈值限制而不会被删除 - 3000 行 */ -INSERT INTO memory SELECT * FROM numbers(0, 1600); -- 1'600 行 - -/* 2. 添加不会被删除的数据块 */ -INSERT INTO memory SELECT * FROM numbers(1000, 100); -- 100 行 - -/* 3. 测试最旧的数据块被删除 - 9216 字节 - 1100 */ -INSERT INTO memory SELECT * FROM numbers(9000, 1000); -- 1'000 行 - -/* 4. 验证超大数据块会覆盖所有现有数据 */ -INSERT INTO memory SELECT * FROM numbers(9000, 10000); -- 10'000 行 - -SELECT total_bytes, total_rows FROM system.tables WHERE name = 'memory' AND database = currentDatabase(); -``` - -```text -┌─total_bytes─┬─total_rows─┐ -│ 65536 │ 10000 │ -└─────────────┴────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md deleted file mode 100644 index facd208ad44..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/merge.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: '`Merge` 引擎(不要与 `MergeTree` 混淆)本身不存储数据,而是允许同时从任意数量的其他表中读取数据。' -sidebar_label: 'Merge' -sidebar_position: 30 -slug: /engines/table-engines/special/merge -title: 'Merge 表引擎' -doc_type: 'reference' ---- - - - -# Merge 表引擎 {#merge-table-engine} - -`Merge` 引擎(不要与 `MergeTree` 混淆)自身不存储数据,而是允许同时从任意数量的其他表中读取数据。 - -读取会自动并行执行。不支持向该表写入数据。读取时,如果被实际读取的表存在索引,则会使用这些索引。 - - - -## 创建表 {#creating-a-table} - -```sql -CREATE TABLE ... 引擎=Merge(db_name, tables_regexp) -``` - - -## 引擎参数 {#engine-parameters} - -### `db_name` {#db_name} - -`db_name` — 可能的取值包括: - - 数据库名称, - - 返回数据库名称字符串的常量表达式,例如 `currentDatabase()`, - - `REGEXP(expression)`,其中 `expression` 是用于匹配数据库名称的正则表达式。 - -### `tables_regexp` {#tables_regexp} - -`tables_regexp` — 用于在指定数据库(单个或多个)中匹配表名的正则表达式。 - -正则表达式引擎 — [re2](https://github.com/google/re2)(支持 PCRE 的子集),区分大小写。 -关于在正则表达式中转义字符的说明,请参见 “match” 部分。 - - - -## 使用方法 {#usage} - -在选择要读取的表时,即使 `Merge` 表名本身匹配正则表达式,也不会被选中。这是为了避免形成循环。 -确实可以创建两个 `Merge` 表,让它们无休止地尝试互相读取数据,但这并不是一个好主意。 - -`Merge` 引擎的典型用法是将大量 `TinyLog` 表当作一张表来处理。 - - - -## 示例 {#examples} - -**示例 1** - -假设有两个数据库 `ABC_corporate_site` 和 `ABC_store`。`all_visitors` 表将包含这两个数据库中各自 `visitors` 表里的 ID。 - -```sql -CREATE TABLE all_visitors (id UInt32) ENGINE=Merge(REGEXP('ABC_*'), 'visitors'); -``` - -**示例 2** - -假设你有一个旧表 `WatchLog_old`,并决定更改分区方式,但不将数据迁移到新表 `WatchLog_new`,同时还需要查看这两个表中的数据。 - -```sql -CREATE TABLE WatchLog_old( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -ORDER BY (date, UserId, EventType); - -INSERT INTO WatchLog_old VALUES ('2018-01-01', 1, 'hit', 3); - -CREATE TABLE WatchLog_new( - date Date, - UserId Int64, - EventType String, - Cnt UInt64 -) -ENGINE=MergeTree -PARTITION BY date -ORDER BY (UserId, EventType) -SETTINGS index_granularity=8192; - -INSERT INTO WatchLog_new VALUES ('2018-01-02', 2, 'hit', 3); - -CREATE TABLE WatchLog AS WatchLog_old ENGINE=Merge(currentDatabase(), '^WatchLog'); - -SELECT * FROM WatchLog; -``` - -```text -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-01 │ 1 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -┌───────date─┬─UserId─┬─EventType─┬─Cnt─┐ -│ 2018-01-02 │ 2 │ hit │ 3 │ -└────────────┴────────┴───────────┴─────┘ -``` - - -## 虚拟列 {#virtual-columns} - -- `_table` — 读取数据的表名。类型:[String](../../../sql-reference/data-types/string.md)。 - - 如果在 `_table` 上进行过滤(例如 `WHERE _table='xyz'`),则只会从满足过滤条件的表中读取数据。 - -- `_database` — 读取数据的数据库名。类型:[String](../../../sql-reference/data-types/string.md)。 - -**另请参阅** - -- [虚拟列](../../../engines/table-engines/index.md#table_engines-virtual_columns) -- [merge](../../../sql-reference/table-functions/merge.md) 表函数 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md deleted file mode 100644 index c2cb4630004..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/null.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '向 `Null` 表写入时,数据会被忽略。从 `Null` 表读取时,结果为空。' -sidebar_label: 'Null' -sidebar_position: 50 -slug: /engines/table-engines/special/null -title: 'Null 表引擎' -doc_type: 'reference' ---- - -# Null 表引擎 {#null-table-engine} - -当向 `Null` 表写入数据时,这些数据会被忽略。 -当从 `Null` 表读取数据时,返回结果是空的。 - -`Null` 表引擎适用于在数据转换后不再需要原始数据的场景。 -为此,可以在 `Null` 表上创建一个物化视图。 -写入该表的数据将会被视图消费,但原始数据会被丢弃。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md deleted file mode 100644 index 8466531679c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/set.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '始终保存在 RAM 中的数据集。用于作为 `IN` 运算符右侧的操作数。' -sidebar_label: 'Set' -sidebar_position: 60 -slug: /engines/table-engines/special/set -title: 'Set 表引擎' -doc_type: 'reference' ---- - - - -# Set 表引擎 {#set-table-engine} - -:::note -在 ClickHouse Cloud 中,如果服务创建时使用的版本早于 25.4,则需要使用 `SET compatibility=25.4` 将兼容性设置为至少 25.4。 -::: - -一个始终驻留在 RAM 中的数据集。它用于 `IN` 运算符的右侧(参见 “IN operators” 章节)。 - -你可以使用 `INSERT` 向表中插入数据。新元素将被添加到数据集中,而重复元素会被忽略。 -但是你不能对该表执行 `SELECT`。检索数据的唯一方式是将其用于 `IN` 运算符的右半部分。 - -数据始终位于 RAM 中。对于 `INSERT` 操作,插入的数据块也会被写入磁盘上的表目录。在服务器启动时,这些数据会被加载到 RAM 中。换句话说,重启之后,数据仍会保留。 - -在发生非正常服务器重启时,磁盘上的数据块可能会丢失或损坏。后一种情况下,你可能需要手动删除包含损坏数据的文件。 - -### 限制和设置 {#join-limitations-and-settings} - -创建表时,会应用以下设置: - -#### Persistent {#persistent} - -为 Set 和 [Join](/engines/table-engines/special/join) 表引擎禁用持久化功能。 - -降低 I/O 开销。适用于追求性能且不需要持久化的场景。 - -可能的取值: - -- 1 — 启用。 -- 0 — 禁用。 - -默认值:`1`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md deleted file mode 100644 index 6e23b82cf74..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/url.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: '用于从/向远程 HTTP/HTTPS 服务器读写数据。该引擎类似于 File 引擎。' -sidebar_label: 'URL' -sidebar_position: 80 -slug: /engines/table-engines/special/url -title: 'URL 表引擎' -doc_type: 'reference' ---- - - - -# URL 表引擎 {#url-table-engine} - -对远程 HTTP/HTTPS 服务器进行数据查询和写入。该引擎类似于 [File](../../../engines/table-engines/special/file.md) 引擎。 - -语法:`URL(URL [,Format] [,CompressionMethod])` - -- `URL` 参数必须符合统一资源定位符(URL)的结构。指定的 URL 必须指向一个使用 HTTP 或 HTTPS 的服务器。无需附加请求头即可从服务器获取响应。 - -- `Format` 必须是 ClickHouse 可以在 `SELECT` 查询中使用的格式,并在需要时可用于 `INSERT`。有关支持的完整格式列表,请参见 [Formats](/interfaces/formats#formats-overview)。 - - 如果未指定该参数,ClickHouse 会根据 `URL` 参数的后缀自动检测格式。如果 `URL` 参数的后缀不匹配任何受支持的格式,则表创建失败。例如,对于引擎表达式 `URL('http://localhost/test.json')`,将应用 `JSON` 格式。 - -- `CompressionMethod` 指示是否需要对 HTTP body 进行压缩。如果启用了压缩,由 URL 引擎发送的 HTTP 数据包会包含 `Content-Encoding` 头,以指示所使用的压缩方法。 - -要启用压缩,请首先确保由 `URL` 参数指明的远程 HTTP 端点支持相应的压缩算法。 - -支持的 `CompressionMethod` 必须是以下之一: -- gzip 或 gz -- deflate -- brotli 或 br -- lzma 或 xz -- zstd 或 zst -- lz4 -- bz2 -- snappy -- none -- auto - -如果未指定 `CompressionMethod`,其默认值为 `auto`。这意味着 ClickHouse 会根据 `URL` 参数的后缀自动检测压缩方法。如果后缀与上述任一压缩方法匹配,则会应用相应的压缩;否则不会启用任何压缩。 - -例如,对于引擎表达式 `URL('http://localhost/test.gzip')`,会应用 `gzip` 压缩方法;而对于 `URL('http://localhost/test.fr')`,不会启用压缩,因为后缀 `fr` 不匹配上述任何压缩方法。 - - - -## 使用方法 {#using-the-engine-in-the-clickhouse-server} - -`INSERT` 和 `SELECT` 查询分别会被转换为 `POST` 和 `GET` 请求。处理 `POST` 请求时,远程服务器必须支持 -[分块传输编码(Chunked transfer encoding)](https://en.wikipedia.org/wiki/Chunked_transfer_encoding)。 - -你可以使用 [max_http_get_redirects](/operations/settings/settings#max_http_get_redirects) 设置来限制 HTTP GET 重定向的最大次数。 - - - -## 示例 {#example} - -**1.** 在服务器上创建一个 `url_engine_table` 表: - -```sql -CREATE TABLE url_engine_table (word String, value UInt64) -ENGINE=URL('http://127.0.0.1:12345/', CSV) -``` - -**2.** 使用 Python 3 标准库创建一个基本的 HTTP 服务器,并启动它: - -```python3 -from http.server import BaseHTTPRequestHandler, HTTPServer - -class CSVHTTPServer(BaseHTTPRequestHandler): - def do_GET(self): - self.send_response(200) - self.send_header('Content-type', 'text/csv') - self.end_headers() - - self.wfile.write(bytes('Hello,1\nWorld,2\n', "utf-8")) - -if __name__ == "__main__": - server_address = ('127.0.0.1', 12345) - HTTPServer(server_address, CSVHTTPServer).serve_forever() -``` - -```bash -$ python3 server.py -``` - -**3.** 请求数据: - -```sql -SELECT * FROM url_engine_table -``` - -```text -┌─word──┬─value─┐ -│ 你好 │ 1 │ -│ 世界 │ 2 │ -└───────┴───────┘ -``` - - -## 实现细节 {#details-of-implementation} - -- 读写可以并行进行 -- 不支持: - - `ALTER` 和 `SELECT...SAMPLE` 操作。 - - 索引。 - - 复制。 - - - -## 虚拟列 {#virtual-columns} - -- `_path` — `URL` 的路径。类型:`LowCardinality(String)`。 -- `_file` — `URL` 的资源名。类型:`LowCardinality(String)`。 -- `_size` — 资源的大小,单位为字节。类型:`Nullable(UInt64)`。如果大小未知,则值为 `NULL`。 -- `_time` — 文件的最后修改时间。类型:`Nullable(DateTime)`。如果时间未知,则值为 `NULL`。 -- `_headers` - HTTP 响应头部。类型:`Map(LowCardinality(String), LowCardinality(String))`。 - - - -## 存储设置 {#storage-settings} - -- [engine_url_skip_empty_files](/operations/settings/settings.md#engine_url_skip_empty_files) - 在读取时跳过空文件。默认禁用。 -- [enable_url_encoding](/operations/settings/settings.md#enable_url_encoding) - 控制是否对 URI 中的路径进行编码/解码。默认启用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md b/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md deleted file mode 100644 index ae9b9d56a94..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/engines/table-engines/special/view.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: '用于实现视图(更多信息参见 `CREATE VIEW` 查询)。它本身不存储数据,只保存指定的 `SELECT` 查询。从该表读取数据时,会执行此查询(并从查询中移除所有不需要的列)。' -sidebar_label: 'View' -sidebar_position: 90 -slug: /engines/table-engines/special/view -title: 'View 表引擎' -doc_type: 'reference' ---- - -# View 表引擎 {#view-table-engine} - -用于实现视图(更多信息请参见 `CREATE VIEW` 查询)。它本身不存储数据,而只存储指定的 `SELECT` 查询。在从该表读取数据时,会运行此查询(并从查询中删除所有不需要的列)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md deleted file mode 100644 index b53a89f3f23..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/arrowflight.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: 'ClickHouse 中 Apache Arrow Flight 接口的相关文档,用于让 Flight SQL 客户端连接到 ClickHouse' -sidebar_label: 'Arrow Flight 接口' -sidebar_position: 26 -slug: /interfaces/arrowflight -title: 'Arrow Flight 接口' -doc_type: 'reference' ---- - - - -# Apache Arrow Flight 接口 {#apache-arrow-flight-interface} - -ClickHouse 支持集成 [Apache Arrow Flight](https://arrow.apache.org/docs/format/Flight.html) 协议——一个为在 gRPC 之上使用 Arrow IPC 格式进行高效列式数据传输而设计的高性能 RPC 框架。 - -通过此接口,Flight SQL 客户端可以查询 ClickHouse,并以 Arrow 格式获取结果,为分析型工作负载提供高吞吐量和低延迟。 - - - -## 功能 {#features} - -* 通过 Arrow Flight SQL 协议执行 SQL 查询 -* 以 Apache Arrow 格式流式返回查询结果 -* 与支持 Arrow Flight 的 BI 工具和自定义数据应用程序集成 -* 通过 gRPC 实现轻量且高效的通信 - - - -## 限制 {#limitations} - -Arrow Flight 接口目前为实验性功能,仍在积极开发中。已知限制包括: - -* 对 ClickHouse 特有的复杂 SQL 功能支持有限 -* 部分 Arrow Flight SQL 元数据操作尚未实现 -* 参考实现中未内置身份验证或 TLS 配置 - -如果您遇到兼容性问题或希望参与贡献,请在 ClickHouse 仓库中[创建一个 issue](https://github.com/ClickHouse/ClickHouse/issues)。 - - - -## 运行 Arrow Flight 服务器 {#running-server} - -要在自托管的 ClickHouse 实例中启用 Arrow Flight 服务器,请在服务器的配置文件中加入如下配置: - -```xml - - 9005 - -``` - -重启 ClickHouse 服务器。启动成功后,您应当能看到类似下面的日志输出: - -```bash -{} Application: Arrow Flight 兼容协议:0.0.0.0:9005 -``` - - -## 通过 Arrow Flight SQL 连接 ClickHouse {#connecting-to-clickhouse} - -可以使用任何支持 Arrow Flight SQL 的客户端。例如,使用 `pyarrow`: - -```python -import pyarrow.flight - -client = pyarrow.flight.FlightClient("grpc://localhost:9005") -ticket = pyarrow.flight.Ticket(b"SELECT number FROM system.numbers LIMIT 10") -reader = client.do_get(ticket) - -for batch in reader: - print(batch.to_pandas()) -``` - - -## 兼容性 {#compatibility} - -Arrow Flight 接口与支持 Arrow Flight SQL 的工具兼容,包括使用以下技术构建的自定义应用程序: - -* Python (`pyarrow`) -* Java (`arrow-flight`) -* C++ 以及其他兼容 gRPC 的语言 - -如果您的工具提供原生 ClickHouse 连接器(例如 JDBC、ODBC),除非在性能或格式兼容性方面有明确需要使用 Arrow Flight,否则应优先选择原生连接器。 - - - -## 查询取消 {#query-cancellation} - -可以通过在客户端关闭 gRPC 连接来取消长时间运行的查询。计划在未来提供对更高级取消功能的支持。 - ---- - -更多详情,参见: - -* [Apache Arrow Flight SQL 规范](https://arrow.apache.org/docs/format/FlightSql.html) -* [ClickHouse GitHub Issue #7554](https://github.com/ClickHouse/ClickHouse/issues/7554) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cli.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cli.md deleted file mode 100644 index a61b8111f6c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cli.md +++ /dev/null @@ -1,918 +0,0 @@ ---- -description: 'ClickHouse 命令行客户端文档' -sidebar_label: 'ClickHouse 客户端' -sidebar_position: 17 -slug: /interfaces/cli -title: 'ClickHouse 客户端' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; -import cloud_connect_button from '@site/static/images/_snippets/cloud-connect-button.png'; -import connection_details_native from '@site/static/images/_snippets/connection-details-native.png'; -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -ClickHouse 提供了一个原生命令行客户端,用于直接对 ClickHouse 服务器执行 SQL 查询。 -它支持交互模式(用于实时执行查询)和批处理模式(用于脚本和自动化)。 -查询结果可以在终端中显示或导出到文件,并支持所有 ClickHouse 输出[格式](formats.md),例如 Pretty、CSV、JSON 等。 - -该客户端通过进度条以及已读取行数、已处理字节数和查询执行时间,为查询执行提供实时反馈。 -它既支持[命令行选项](#command-line-options),也支持[配置文件](#configuration_files)。 - - -## 安装 {#install} - -若要下载 ClickHouse,请运行: - -```bash -curl https://clickhouse.com/ | sh -``` - -若要一并安装它,请运行: - -```bash -sudo ./clickhouse install -``` - -有关更多安装选项,请参阅 [Install ClickHouse](../getting-started/install/install.mdx)。 - -不同版本的客户端和服务器之间是兼容的,但某些功能在较旧的客户端中可能不可用。我们建议客户端和服务器使用相同的版本。 - - -## 运行 {#run} - -:::note -如果你只是下载了但尚未安装 ClickHouse,请使用 `./clickhouse client` 而不是 `clickhouse-client`。 -::: - -要连接到 ClickHouse 服务器,请运行: - -```bash -$ clickhouse-client --host server - -ClickHouse 客户端版本 24.12.2.29(官方构建)。 -正在以用户 default 身份连接到 server:9000。 -已连接到 ClickHouse 服务器版本 24.12.2。 - -:) -``` - -根据需要指定其他连接详细信息: - -| Option | Description | -| -------------------------------- | -------------------------------------------------------------------------------------------- | -| `--port ` | ClickHouse 服务器接受连接的端口。默认端口为 9440(TLS)和 9000(无 TLS)。请注意,ClickHouse Client 使用的是原生协议而非 HTTP(S)。 | -| `-s [ --secure ]` | 是否使用 TLS(通常会自动检测)。 | -| `-u [ --user ] ` | 要用于连接的数据库用户。默认情况下以 `default` 用户连接。 | -| `--password ` | 数据库用户的密码。也可以在配置文件中为连接指定密码。如果未指定密码,客户端会提示输入。 | -| `-c [ --config ] ` | ClickHouse Client 配置文件的位置(如果不在任何默认位置)。参见 [配置文件](#configuration_files)。 | -| `--connection ` | 来自[配置文件](#connection-credentials)的预配置连接详细信息名称。 | - -有关命令行选项的完整列表,请参阅[命令行选项](#command-line-options)。 - - -### 连接到 ClickHouse Cloud {#connecting-cloud} - -ClickHouse Cloud 服务的详细信息可在 ClickHouse Cloud 控制台中查看。选择要连接的服务并点击 **Connect**(连接): - -ClickHouse Cloud 服务 Connect 按钮 - -
- -
- -选择 **Native**,即可看到包含示例 `clickhouse-client` 命令的连接详情: - -ClickHouse Cloud 原生 TCP 连接详情 - -### 在配置文件中存储连接 {#connection-credentials} - -你可以在[配置文件](#configuration_files)中保存一个或多个 ClickHouse 服务器的连接信息。 - -格式如下: - -```xml - - - - default - hostname - 9440 - 1 - default - password - - - - - - - -``` - -有关更多信息,请参阅[配置文件部分](#configuration_files)。 - -:::note -为了专注于查询语法,其余示例省略了连接详细信息(`--host`、`--port` 等)。在实际使用这些命令时,请记得补充这些参数。 -::: - - -## 交互模式 {#interactive-mode} - -### 使用交互式模式 {#using-interactive-mode} - -要以交互式模式运行 ClickHouse,只需执行: - -```bash -clickhouse-client -``` - -这将打开 Read-Eval-Print Loop(REPL)交互环境,你可以在其中直接输入 SQL 查询。 -连接成功后,你会看到一个提示符,在那里即可输入查询: - -```bash -ClickHouse 客户端版本 25.x.x.x -正在以 default 用户身份连接到 localhost:9000。 -已连接到 ClickHouse 服务器版本 25.x.x.x - -hostname :) -``` - -在交互模式下,默认输出格式为 `PrettyCompact`。 -可以在查询的 `FORMAT` 子句中更改格式,或者通过指定 `--format` 命令行选项进行更改。 -要使用 Vertical 格式,可以使用 `--vertical`,或者在查询末尾添加 `\G`。 -在这种格式下,每个值都会单独打印在一行上,这对于宽表非常方便。 - -在交互模式下,默认情况下,按下 `Enter` 后会立即执行当前输入的内容。 -查询末尾不需要添加分号。 - -可以使用 `-m, --multiline` 参数启动客户端。 -要输入多行查询,请在换行前输入反斜杠 `\`。 -按下 `Enter` 之后,系统会提示输入查询的下一行。 -要运行查询,请在末尾添加分号并按 `Enter`。 - -ClickHouse Client 基于 `replxx`(类似于 `readline`),因此支持常见的键盘快捷键并保留历史记录。 -历史记录默认写入 `~/.clickhouse-client-history`。 - -要退出客户端,请按 `Ctrl+D`,或者在查询位置输入以下之一: - -* `exit` 或 `exit;` -* `quit` 或 `quit;` -* `q`、`Q` 或 `:q` -* `logout` 或 `logout;` - - -### 查询处理信息 {#processing-info} - -在处理查询时,客户端会显示: - -1. 进度,默认情况下每秒更新不超过 10 次。 - 对于执行很快的查询,进度可能来不及显示。 -2. 解析后的格式化查询文本,用于调试。 -3. 指定格式的查询结果。 -4. 结果中的行数、已用时间以及查询处理的平均速度。 - 所有数据量均指未压缩数据。 - -您可以通过按 `Ctrl+C` 来取消一个耗时较长的查询。 -但是,仍然需要稍等片刻,以便服务器中止该请求。 -在查询执行的某些阶段无法取消查询。 -如果您不等待并第二次按下 `Ctrl+C`,客户端将退出。 - -ClickHouse Client 允许在执行查询时传入外部数据(外部临时表)。 -有关更多信息,请参阅[用于查询处理的外部数据](../engines/table-engines/special/external-data.md)一节。 - -### 别名 {#cli_aliases} - -你可以在 REPL 中使用以下别名: - -- `\l` - SHOW DATABASES -- `\d` - SHOW TABLES -- `\c ` - USE DATABASE -- `.` - 重复上一条查询 - -### 键盘快捷键 {#keyboard_shortcuts} - -- `Alt (Option) + Shift + e` - 使用当前查询打开编辑器。可以通过环境变量 `EDITOR` 指定要使用的编辑器,默认使用 `vim`。 -- `Alt (Option) + #` - 注释当前行。 -- `Ctrl + r` - 模糊搜索历史记录。 - -包含所有可用键盘快捷键的完整列表请参见 [replxx](https://github.com/AmokHuginnsson/replxx/blob/1f149bf/src/replxx_impl.cxx#L262)。 - -:::tip -要在 macOS 上正确配置 meta 键(Option)的行为: - -iTerm2:依次进入 Preferences -> Profiles -> Keys -> Left Option key,并将其设置为 Esc+。 -::: - -## 批处理模式 {#batch-mode} - -### 使用批处理模式 {#using-batch-mode} - -与交互式使用 ClickHouse Client 不同,您可以以批处理模式运行它。 -在批处理模式下,ClickHouse 只执行一个查询并立即退出——不会进入交互式提示符或循环。 - -您可以像这样指定一个单独的查询: - -```bash -$ clickhouse-client "SELECT sum(number) FROM numbers(10)" -45 -``` - -你还可以使用 `--query` 命令行选项: - -```bash -$ clickhouse-client --query "SELECT uniq(number) FROM numbers(10)" -10 -``` - -您可以通过 `stdin` 输入查询: - -```bash -$ echo "SELECT avg(number) FROM numbers(10)" | clickhouse-client -4.5 -``` - -假设已存在一张名为 `messages` 的表,还可以从命令行插入数据: - -```bash -$ echo "Hello\nGoodbye" | clickhouse-client --query "INSERT INTO messages FORMAT CSV" -``` - -当指定 `--query` 时,所有输入内容都会在一个换行符之后被追加到请求中。 - - -### 向远程 ClickHouse 服务插入 CSV 文件 {#cloud-example} - -本示例将示例数据集 CSV 文件 `cell_towers.csv` 插入到 `default` 数据库中已存在的 `cell_towers` 表中: - -```bash -clickhouse-client --host HOSTNAME.clickhouse.cloud \ - --port 9440 \ - --user default \ - --password PASSWORD \ - --query "INSERT INTO cell_towers FORMAT CSVWithNames" \ - < cell_towers.csv -``` - - -### 从命令行插入数据的示例 {#more-examples} - -可以通过多种方式在命令行中插入数据。 -下面的示例使用批量模式将两行 CSV 数据插入到一个 ClickHouse 表中: - -```bash -echo -ne "1, 'some text', '2016-08-14 00:00:00'\n2, 'some more text', '2016-08-14 00:00:01'" | \ - clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -在下面的示例中,`cat <<_EOF` 会开始一个 heredoc,它会读取所有内容,直到再次遇到 `_EOF`,然后将其输出: - -```bash -cat <<_EOF | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -3, 'some text', '2016-08-14 00:00:00' -4, 'some more text', '2016-08-14 00:00:01' -_EOF -``` - -在下面的示例中,使用 `cat` 将 file.csv 文件的内容输出到标准输出(stdout),并通过管道传递给 `clickhouse-client` 作为输入: - -```bash -cat file.csv | clickhouse-client --database=test --query="INSERT INTO test FORMAT CSV"; -``` - -在批量模式下,默认的数据[格式](formats.md)为 `TabSeparated`。 -您可以在查询的 `FORMAT` 子句中设置格式,如上例所示。 - - -## 带参数的查询 {#cli-queries-with-parameters} - -你可以在查询中指定参数,并通过命令行选项向其传递参数值。 -这样可以避免在客户端使用特定的动态值来格式化查询。 -例如: - -```bash -$ clickhouse-client --param_parName="[1, 2]" --query "SELECT {parName: Array(UInt16)}" -[1,2] -``` - -也可以在[交互式会话](#interactive-mode)中设置参数: - -```text -$ clickhouse-client -ClickHouse 客户端版本 25.X.X.XXX(官方构建)。 - -#highlight-next-line -:) SET param_parName='[1, 2]'; - -SET param_parName = '[1, 2]' - -Query id: 7ac1f84e-e89a-4eeb-a4bb-d24b8f9fd977 - -完成。 - -结果集包含 0 行。耗时:0.000 秒。 - -#highlight-next-line -:) SELECT {parName:Array(UInt16)} - -SELECT {parName:Array(UInt16)} - -Query id: 0358a729-7bbe-4191-bb48-29b063c548a7 - - ┌─_CAST([1, 2]⋯y(UInt16)')─┐ -1. │ [1,2] │ - └──────────────────────────┘ - -结果集包含 1 行。耗时:0.006 秒。 -``` - - -### 查询语法 {#cli-queries-with-parameters-syntax} - -在查询中,将你希望通过命令行参数传入的值用大括号括起来,格式如下: - -```sql -{:} -``` - -| Parameter | Description | -| ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `name` | 占位符标识符。对应的命令行选项为 `--param_ = value`。 | -| `data type` | 参数的[数据类型](../sql-reference/data-types/index.md)。

例如,类似 `(integer, ('string', integer))` 的数据结构可以使用 `Tuple(UInt8, Tuple(String, UInt8))` 数据类型(也可以采用其他[整数](../sql-reference/data-types/int-uint.md)类型)。

也可以将表名、数据库名和列名作为参数传递,在这种情况下,则需要将其数据类型指定为 `Identifier`。 | - - -### 示例 {#cli-queries-with-parameters-examples} - -```bash -$ clickhouse-client --param_tuple_in_tuple="(10, ('dt', 10))" \ - --query "SELECT * FROM table WHERE val = {tuple_in_tuple:Tuple(UInt8, Tuple(String, UInt8))}" - -$ clickhouse-client --param_tbl="numbers" --param_db="system" --param_col="number" --param_alias="top_ten" \ - --query "SELECT {col:Identifier} as {alias:Identifier} FROM {db:Identifier}.{tbl:Identifier} LIMIT 10" -``` - - -## 基于 AI 的 SQL 生成 {#ai-sql-generation} - -ClickHouse 客户端内置了 AI 助手,可以根据自然语言描述生成 SQL 查询。此功能可帮助用户在不具备深厚 SQL 知识的情况下编写复杂查询。 - -如果已设置 `OPENAI_API_KEY` 或 `ANTHROPIC_API_KEY` 环境变量,AI 助手即可开箱即用。要进行更高级的配置,请参阅[配置](#ai-sql-generation-configuration)章节。 - -### 使用方法 {#ai-sql-generation-usage} - -要使用 AI SQL 生成功能,请在自然语言查询前添加前缀 `??`: - -```bash -:) ?? 显示过去 30 天内有购买记录的所有用户 -``` - -AI 将会: - -1. 自动探索您的数据库结构(schema) -2. 基于发现的表和列生成合适的 SQL 查询 -3. 立即执行生成的查询 - - -### 示例 {#ai-sql-generation-example} - -```bash -:) ?? count orders by product category - -正在启动 AI SQL 生成并进行架构发现... -────────────────────────────────────────────────── - -🔍 list_databases - ➜ system, default, sales_db - -🔍 list_tables_in_database - database: sales_db - ➜ orders, products, categories - -🔍 get_schema_for_table - database: sales_db - table: orders - ➜ CREATE TABLE orders (order_id UInt64, product_id UInt64, quantity UInt32, ...) - -✨ SQL 查询生成成功! -────────────────────────────────────────────────── - -SELECT - c.name AS category, - COUNT(DISTINCT o.order_id) AS order_count -FROM sales_db.orders o -JOIN sales_db.products p ON o.product_id = p.product_id -JOIN sales_db.categories c ON p.category_id = c.category_id -GROUP BY c.name -ORDER BY order_count DESC -``` - - -### 配置 {#ai-sql-generation-configuration} - -要使用 AI 生成 SQL,需要在 ClickHouse Client 配置文件中配置一个 AI 提供方。你可以使用 OpenAI、Anthropic,或任意与 OpenAI 兼容的 API 服务。 - -#### 基于环境变量的回退机制 {#ai-sql-generation-fallback} - -如果在配置文件中未指定 AI 配置,ClickHouse Client 将自动尝试使用环境变量: - -1. 首先检查 `OPENAI_API_KEY` 环境变量 -2. 如果未找到,则检查 `ANTHROPIC_API_KEY` 环境变量 -3. 如果两者都未找到,则会禁用 AI 功能 - -这样无需配置文件即可快速完成设置: - -```bash -# 使用 OpenAI {#using-openai} -export OPENAI_API_KEY=your-openai-key -clickhouse-client - -# 使用 Anthropic {#using-anthropic} -export ANTHROPIC_API_KEY=your-anthropic-key -clickhouse-client -``` - - -#### 配置文件 {#ai-sql-generation-configuration-file} - -若要对 AI 设置进行更精细的控制,请在以下位置的 ClickHouse 客户端配置文件中进行配置: - -* `$XDG_CONFIG_HOME/clickhouse/config.xml`(如果未设置 `XDG_CONFIG_HOME`,则为 `~/.config/clickhouse/config.xml`)(XML 格式) -* `$XDG_CONFIG_HOME/clickhouse/config.yaml`(如果未设置 `XDG_CONFIG_HOME`,则为 `~/.config/clickhouse/config.yaml`)(YAML 格式) -* `~/.clickhouse-client/config.xml`(XML 格式,旧版位置) -* `~/.clickhouse-client/config.yaml`(YAML 格式,旧版位置) -* 或使用 `--config-file` 指定自定义位置 - - - - ```xml - - - - your-api-key-here - - - openai - - - gpt-4o - - - - - - true - - - 0.0 - 1000 - 30 - 10 - - - - - - ``` - - - - ```yaml - ai: - # 必填:您的 API 密钥(或通过环境变量设置) - api_key: your-api-key-here - - # 必填:提供方类型(openai, anthropic) - provider: openai - - # 要使用的模型 - model: gpt-4o - - # 可选:用于兼容 OpenAI 的服务的自定义 API 端点 - # base_url: https://openrouter.ai/api - - # 启用 schema 访问 —— 允许 AI 查询数据库/数据表信息 - enable_schema_access: true - - # 生成参数 - temperature: 0.0 # 控制随机性(0.0 = 确定性) - max_tokens: 1000 # 最大响应长度 - timeout_seconds: 30 # 请求超时时间 - max_steps: 10 # 最大 schema 探索步数 - - # 可选:自定义 system prompt - # system_prompt: | - # You are an expert ClickHouse SQL assistant. Convert natural language to SQL. - # Focus on performance and use ClickHouse-specific optimizations. - # Always return executable SQL without explanations. - ``` - - - -
- -**使用兼容 OpenAI 的 API(例如 OpenRouter):** - -```yaml -ai: - provider: openai # 使用 'openai' 以确保兼容性 - api_key: your-openrouter-api-key - base_url: https://openrouter.ai/api/v1 - model: anthropic/claude-3.5-sonnet # 使用 OpenRouter 模型命名规范 -``` - -**最小化配置示例:** - -```yaml -# 最小配置 - 使用环境变量提供 API 密钥 {#minimal-config-uses-environment-variable-for-api-key} -ai: - provider: openai # 将使用 OPENAI_API_KEY 环境变量 - -# 无配置 - 自动回退 {#no-config-at-all-automatic-fallback} -# (空配置或无 ai 部分 - 将依次尝试 OPENAI_API_KEY 然后 ANTHROPIC_API_KEY) {#empty-or-no-ai-section-will-try-openai_api_key-then-anthropic_api_key} - -# 仅覆盖模型 - 使用环境变量提供 API 密钥 {#only-override-model-uses-env-var-for-api-key} -ai: - provider: openai - model: gpt-3.5-turbo -``` - - -### 参数 {#ai-sql-generation-parameters} - -
-必需参数 - -- `api_key` - AI 服务的 API 密钥。如果通过环境变量设置,则可以省略: - - OpenAI: `OPENAI_API_KEY` - - Anthropic: `ANTHROPIC_API_KEY` - - 注意:配置文件中的 API 密钥优先于环境变量 -- `provider` - AI 提供商:`openai` 或 `anthropic` - - 如果省略,则会根据可用的环境变量自动选择 - -
- -
-模型配置 - -- `model` - 要使用的模型(默认:由提供商决定) - - OpenAI: `gpt-4o`, `gpt-4`, `gpt-3.5-turbo` 等 - - Anthropic: `claude-3-5-sonnet-20241022`, `claude-3-opus-20240229` 等 - - OpenRouter: 使用其模型命名方式,如 `anthropic/claude-3.5-sonnet` - -
- -
-连接设置 - -- `base_url` - 兼容 OpenAI 的服务所使用的自定义 API 端点(可选) -- `timeout_seconds` - 请求超时时间(秒)(默认:`30`) - -
- -
-Schema 探索 - -- `enable_schema_access` - 允许 AI 探索数据库模式(schema)(默认:`true`) -- `max_steps` - Schema 探索时工具调用的最大步数(默认:`10`) - -
- -
-生成参数 - -- `temperature` - 控制随机性,0.0 = 输出更确定,1.0 = 更具创造性(默认:`0.0`) -- `max_tokens` - 最大响应长度(以 token 计)(默认:`1000`) -- `system_prompt` - 给 AI 的自定义指令(可选) - -
- -### 工作原理 {#ai-sql-generation-how-it-works} - -AI SQL 生成器使用多步骤流程: - - - -1. **模式发现** - -AI 使用内置工具来探索你的数据库: -- 列出可用的数据库 -- 发现相关数据库中的表 -- 通过 `CREATE TABLE` 语句检查表结构 - -2. **查询生成** - -基于已发现的模式,AI 生成的 SQL 会: -- 符合你的自然语言意图 -- 使用正确的表名和列名 -- 应用合适的表连接和聚合 - -3. **执行** - -生成的 SQL 会被自动执行,并显示结果 - - - -### 限制 {#ai-sql-generation-limitations} - -- 需要保持有效的网络连接 -- API 使用受限于 AI 提供商的速率限制和费用 -- 复杂查询可能需要多次优化调整 -- AI 只能以只读方式访问架构(schema)信息,无法访问实际数据 - -### 安全性 {#ai-sql-generation-security} - -- API 密钥绝不会发送至 ClickHouse 服务器 -- AI 只能访问 schema 信息(表/列名称和类型),而不会接触实际数据 -- 所有生成的查询都会遵循您现有的数据库权限 - -## 连接字符串 {#connection_string} - -### 用法 {#connection-string-usage} - -ClickHouse Client 还支持使用类似 [MongoDB](https://www.mongodb.com/docs/manual/reference/connection-string/)、[PostgreSQL](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING)、[MySQL](https://dev.mysql.com/doc/refman/8.0/en/connecting-using-uri-or-key-value-pairs.html#connecting-using-uri) 的连接字符串来连接 ClickHouse 服务器。其语法格式如下: - -```text -clickhouse:[//[user[:password]@][hosts_and_ports]][/database][?query_parameters] -``` - -| 组件(均为可选) | 说明 | 默认值 | -| ------------------ | ---------------------------------------------------------------------- | ---------------- | -| `user` | 数据库用户名。 | `default` | -| `password` | 数据库用户密码。如果指定了 `:` 且密码为空,客户端会提示输入该用户的密码。 | - | -| `hosts_and_ports` | 主机及可选端口列表:`host[:port] [, host:[port]], ...`。 | `localhost:9000` | -| `database` | 数据库名称。 | `default` | -| `query_parameters` | 键值对列表:`param1=value1[,¶m2=value2], ...`。对于某些参数,可以不指定值。参数名称和值区分大小写。 | - | - - -### 注意事项 {#connection-string-notes} - -如果在连接字符串中已经指定了用户名、密码或数据库,则不能再通过 `--user`、`--password` 或 `--database` 指定(反之亦然)。 - -主机部分可以是主机名,或 IPv4 或 IPv6 地址。 -IPv6 地址应放在方括号中: - -```text -clickhouse://[2001:db8::1234] -``` - -连接字符串可以包含多个主机。 -ClickHouse 客户端会按顺序(从左到右)尝试连接这些主机。 -一旦连接建立,就不会再尝试连接剩余的主机。 - -连接字符串必须作为 `clickhouse-client` 的第一个参数指定。 -连接字符串可以与任意数量的其他[命令行选项](#command-line-options)组合使用,但不能与 `--host` 和 `--port` 一起使用。 - -`query_parameters` 支持以下键: - -| Key | Description | -| ----------------- | ---------------------------------------------------------------------------- | -| `secure` (or `s`) | 如果指定此键,客户端将通过安全连接(TLS)连接到服务器。请参阅[命令行选项](#command-line-options)中的 `--secure`。 | - -**百分号编码** - -以下参数中的非 US-ASCII 字符、空格和特殊字符必须进行[百分号编码](https://en.wikipedia.org/wiki/URL_encoding): - -* `user` -* `password` -* `hosts` -* `database` -* `query parameters` - - -### 示例 {#connection_string_examples} - -连接到 `localhost` 的 9000 端口并执行查询 `SELECT 1`。 - -```bash -clickhouse-client clickhouse://localhost:9000 --query "SELECT 1" -``` - -以用户 `john`、密码 `secret` 连接到 `localhost`,主机地址为 `127.0.0.1`,端口为 `9000` - -```bash -clickhouse-client clickhouse://john:secret@127.0.0.1:9000 -``` - -以 `default` 用户身份连接到 `localhost`,主机 IPv6 地址为 `[::1]`,端口为 `9000`。 - -```bash -clickhouse-client clickhouse://[::1]:9000 -``` - -使用多行模式连接到 `localhost` 的 9000 端口。 - -```bash -clickhouse-client clickhouse://localhost:9000 '-m' -``` - -使用端口 9000 以用户 `default` 连接到 `localhost`。 - -```bash -clickhouse-client clickhouse://default@localhost:9000 - -# 等同于: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --user default -``` - -连接到 `localhost` 的 9000 端口,并将默认数据库设置为 `my_database`。 - -```bash -clickhouse-client clickhouse://localhost:9000/my_database - -# 等效于: {#equivalent-to} -clickhouse-client clickhouse://localhost:9000 --database my_database -``` - -连接到 `localhost` 的 9000 端口,默认使用连接字符串中指定的 `my_database` 数据库,并通过简写参数 `s` 启用安全连接。 - -```bash -clickhouse-client clickhouse://localhost/my_database?s - -# 等同于: {#equivalent-to} -clickhouse-client clickhouse://localhost/my_database -s -``` - -使用默认端口、默认用户和默认数据库连接到默认主机。 - -```bash -clickhouse-client clickhouse: -``` - -以用户 `my_user` 且不使用密码的身份,连接到默认主机的默认端口。 - -```bash -clickhouse-client clickhouse://my_user@ - -# 在 : 和 @ 之间使用空密码表示在建立连接前提示用户输入密码。 {#using-a-blank-password-between-and-means-to-asking-the-user-to-enter-the-password-before-starting-the-connection} -clickhouse-client clickhouse://my_user:@ -``` - -使用电子邮件地址作为用户名连接到 `localhost`。`@` 符号会被百分号编码为 `%40`。 - -```bash -clickhouse-client clickhouse://some_user%40some_mail.com@localhost:9000 -``` - -连接到以下任一主机:`192.168.1.15`、`192.168.1.25`。 - -```bash -clickhouse-client clickhouse://192.168.1.15,192.168.1.25 -``` - - -## 查询 ID 格式 {#query-id-format} - -在交互模式下,ClickHouse 客户端会为每个查询显示其查询 ID。默认情况下,ID 的格式如下: - -```sql -查询 ID:927f137d-00f1-4175-8914-0dd066365e96 -``` - -可以在配置文件的 `query_id_formats` 标签中指定自定义格式。格式字符串中的 `{query_id}` 占位符会被替换为查询 ID。该标签中可以包含多个格式字符串。 -此功能可用于生成 URL,以便对查询进行性能分析。 - -**示例** - -```xml - - - http://speedscope-host/#profileURL=qp%3Fid%3D{query_id} - - -``` - -根据上述配置,查询 ID 将以以下格式显示: - -```response -speedscope:http://speedscope-host/#profileURL=qp%3Fid%3Dc8ecc783-e753-4b38-97f1-42cddfb98b7d -``` - - -## 配置文件 {#configuration_files} - -ClickHouse 客户端会按以下顺序查找,并使用第一个存在的配置文件: - -- 通过 `-c [ -C, --config, --config-file ]` 参数指定的文件。 -- `./clickhouse-client.[xml|yaml|yml]` -- `$XDG_CONFIG_HOME/clickhouse/config.[xml|yaml|yml]`(如果未设置 `XDG_CONFIG_HOME`,则为 `~/.config/clickhouse/config.[xml|yaml|yml]`) -- `~/.clickhouse-client/config.[xml|yaml|yml]` -- `/etc/clickhouse-client/config.[xml|yaml|yml]` - -示例配置文件请参阅 ClickHouse 仓库中的 [`clickhouse-client.xml`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/client/clickhouse-client.xml) - - - - ```xml - - username - password - true - - - /etc/ssl/cert.pem - - - - ``` - - - ```yaml - user: username - password: 'password' - secure: true - openSSL: - client: - caConfig: '/etc/ssl/cert.pem' - ``` - - - -## 环境变量选项 {#environment-variable-options} - -可以通过环境变量 `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD` 和 `CLICKHOUSE_HOST` 设置用户名、密码和主机。 -命令行参数 `--user`、`--password` 或 `--host`,或者(如果已指定)[连接字符串](#connection_string) 的优先级高于环境变量。 - -## 命令行选项 {#command-line-options} - -所有命令行选项都可以直接在命令行中指定,也可以在[配置文件](#configuration_files)中设置为默认值。 - -### 常规选项 {#command-line-options-general} - -| Option | Description | Default | -|-----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|------------------------------| -| `-c [ -C, --config, --config-file ] ` | 客户端配置文件的路径(如果它不在任何默认路径下)。参见 [Configuration Files](#configuration_files)。 | - | -| `--help` | 打印使用帮助摘要并退出。与 `--verbose` 一起使用可显示所有可用选项,包括查询设置。 | - | -| `--history_file ` | 命令历史记录文件的路径。 | - | -| `--history_max_entries` | 历史记录文件中允许的最大条目数。 | `1000000`(100 万) | -| `--prompt ` | 指定自定义提示符。 | 服务器的 `display_name` | -| `--verbose` | 增加输出的详细程度。 | - | -| `-V [ --version ]` | 打印版本并退出。 | - | - -### 连接选项 {#command-line-options-connection} - -| Option | Description | Default | -|----------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------| -| `--connection ` | 配置文件中预先配置的连接名称。参见 [连接凭证](#connection-credentials)。 | - | -| `-d [ --database ] ` | 选择此连接默认使用的数据库。 | 来自服务器设置的当前数据库(默认是 `default`) | -| `-h [ --host ] ` | 要连接的 ClickHouse 服务器的主机名。可以是主机名,也可以是 IPv4 或 IPv6 地址。可以通过多次传入该参数来指定多个主机。 | `localhost` | -| `--jwt ` | 使用 JSON Web Token (JWT) 进行身份验证。

服务器端 JWT 授权仅在 ClickHouse Cloud 中可用。 | - | -| `--no-warnings` | 在客户端连接到服务器时,不显示来自 `system.warnings` 的警告。 | - | -| `--no-server-client-version-message` | 在客户端连接到服务器时,抑制服务器与客户端版本不匹配的提示信息。 | - | -| `--password ` | 数据库用户的密码。也可以在配置文件中为连接指定密码。如果未指定密码,客户端会提示输入。 | - | -| `--port ` | 服务器接受连接的端口。默认端口为 9440(TLS)和 9000(非 TLS)。

注意:客户端使用的是原生协议而不是 HTTP(S)。 | 如果指定了 `--secure`,则为 `9440`,否则为 `9000`。当主机名以 `.clickhouse.cloud` 结尾时,始终默认为 `9440`。 | -| `-s [ --secure ]` | 是否使用 TLS。

在连接到端口 9440(默认安全端口)或 ClickHouse Cloud 时会自动启用。

可能需要在[配置文件](#configuration_files)中配置 CA 证书。可用的配置项与[服务器端 TLS 配置](../operations/server-configuration-parameters/settings.md#openssl) 相同。 | 在连接到端口 9440 或 ClickHouse Cloud 时自动启用 | -| `--ssh-key-file ` | 包含用于与服务器进行身份验证的 SSH 私钥的文件。 | - | -| `--ssh-key-passphrase ` | `--ssh-key-file` 中指定的 SSH 私钥的密码短语。 | - | -| `-u [ --user ] ` | 要用于连接的数据库用户。 | `default` | - -:::note -除了 `--host`、`--port`、`--user` 和 `--password` 选项外,客户端还支持[连接字符串](#connection_string)。 -::: - -### 查询选项 {#command-line-options-query} - -| 选项 | 说明 | -|---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `--param_=` | [带参数查询](#cli-queries-with-parameters) 中某个参数的替换值。 | -| `-q [ --query ] ` | 在批处理模式下执行的查询。可以多次指定(`--query "SELECT 1" --query "SELECT 2"`),也可以只指定一次并在其中包含多个用分号分隔的查询(`--query "SELECT 1; SELECT 2;"`)。在后一种情况下,所有使用 `VALUES` 以外格式的 `INSERT` 查询之间必须以空行分隔。

也可以在不带选项名的情况下指定单个查询:`clickhouse-client "SELECT 1"`

不能与 `--queries-file` 同时使用。 | -| `--queries-file ` | 包含查询语句的文件路径。`--queries-file` 可以多次指定,例如:`--queries-file queries1.sql --queries-file queries2.sql`。

不能与 `--query` 同时使用。 | -| `-m [ --multiline ]` | 如果指定该选项,则允许多行查询(按 Enter 键不会立即发送查询)。只有当查询以分号结尾时才会发送。 | - -### 查询设置 {#command-line-options-query-settings} - -可以在客户端中通过命令行选项指定查询设置,例如: - -```bash -$ clickhouse-client --max_threads 1 -``` - -有关所有设置的列表,请参阅[设置](../operations/settings/settings.md)。 - - -### 格式选项 {#command-line-options-formatting} - -| 选项 | 说明 | 默认值 | -|---------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------| -| `-f [ --format ] ` | 使用指定的格式输出结果。

有关支持的格式列表,请参阅[输入和输出数据的格式](formats.md)。 | `TabSeparated` | -| `--pager ` | 将所有输出通过管道传给此命令。通常为 `less`(例如使用 `less -S` 来显示较宽的结果集)或类似命令。 | - | -| `-E [ --vertical ]` | 使用 [Vertical 格式](/interfaces/formats/Vertical) 输出结果。等同于 `--format Vertical`。在此格式中,每个值都会单独显示在一行上,便于查看列数较多的表。 | - | - -### 执行详情 {#command-line-options-execution-details} - -| 选项 | 描述 | 默认值 | -|-----------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------| -| `--enable-progress-table-toggle` | 启用通过按下 Ctrl+Space 组合键切换进度表。仅在启用了进度表打印的交互模式下生效。 | `enabled` | -| `--hardware-utilization` | 在进度条中打印硬件利用率信息。 | - | -| `--memory-usage` | 如指定,在非交互模式下将内存使用情况打印到 `stderr`。

可能的取值:
• `none` - 不打印内存使用情况
• `default` - 打印字节数
• `readable` - 以人类可读的格式打印内存使用情况 | - | -| `--print-profile-events` | 打印 `ProfileEvents` 数据包。 | - | -| `--progress` | 打印查询执行进度。

可能的取值:
• `tty\|on\|1\|true\|yes` - 在交互模式下输出到终端
• `err` - 在非交互模式下输出到 `stderr`
• `off\|0\|false\|no` - 禁用进度输出 | 交互模式下为 `tty`,非交互(批处理)模式下为 `off` | -| `--progress-table` | 在查询执行期间打印包含动态变化指标的进度表。

可能的取值:
• `tty\|on\|1\|true\|yes` - 在交互模式下输出到终端
• `err` - 在非交互模式下输出到 `stderr`
• `off\|0\|false\|no` - 禁用进度表 | 交互模式下为 `tty`,非交互(批处理)模式下为 `off` | -| `--stacktrace` | 打印异常的堆栈跟踪信息。 | - | -| `-t [ --time ]` | 在非交互模式下将查询执行时间打印到 `stderr`(用于基准测试)。 | - | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cpp.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cpp.md deleted file mode 100644 index 25190328a2f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/cpp.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: 'ClickHouse C++ 客户端库及其与 u-server 框架集成的文档' -sidebar_label: 'C++ 客户端库' -sidebar_position: 24 -slug: /interfaces/cpp -title: 'C++ 客户端库' -doc_type: 'reference' ---- - -# C++ 客户端库 {#c-client-library} - -请参阅 [clickhouse-cpp](https://github.com/ClickHouse/clickhouse-cpp) 仓库中的 README 文件。 - -# Userver 异步框架 {#userver-asynchronous-framework} - -[userver(测试版)](https://github.com/userver-framework/userver) 内置对 ClickHouse 的支持。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats.md deleted file mode 100644 index f03425fe5b2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'ClickHouse 中输入和输出数据格式概览' -sidebar_label: '查看所有格式...' -sidebar_position: 21 -slug: /interfaces/formats -title: '输入和输出数据格式' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - -# 输入和输出数据的格式 {#formats-for-input-and-output-data} - -ClickHouse 支持大多数已知的文本和二进制数据格式,从而可以轻松集成到几乎任何现有的数据管道中,充分发挥 ClickHouse 的优势。 - - - -## 输入格式 {#input-formats} - -输入格式用于: -- 解析提供给 `INSERT` 语句的数据 -- 对诸如 `File`、`URL` 或 `HDFS` 等文件后端表执行 `SELECT` 查询 -- 读取字典 - -为 ClickHouse 选择合适的输入格式对高效数据摄取至关重要。ClickHouse 支持 70 多种格式,选择性能最佳的选项会显著影响插入速度、CPU 与内存使用率以及整体系统效率。为帮助你在这些选项中进行选择,我们对不同格式的摄取性能进行了基准测试,并得出了以下关键结论: - -- **[Native](formats/Native.md) 格式是最高效的输入格式**,提供最佳压缩率、最低资源占用以及最小的服务器端处理开销。 -- **压缩至关重要** —— LZ4 能在较低 CPU 开销下减小数据体积,而 ZSTD 则以更高压缩率为代价增加了 CPU 开销。 -- **预排序的影响中等**,因为 ClickHouse 本身就能高效完成排序。 -- **批量写入显著提高效率** —— 更大的批次可以减少插入开销并提升吞吐量。 - -如需深入了解测试结果和最佳实践, -请阅读完整的[基准分析](https://www.clickhouse.com/blog/clickhouse-input-format-matchup-which-is-fastest-most-efficient)。 -要查看完整测试结果,请访问 [FastFormats](https://fastformats.clickhouse.com/) 在线仪表盘。 - - - -## 输出格式 {#output-formats} - -支持的输出格式用于: -- 组织 `SELECT` 查询的结果 -- 向以文件为后端的表执行 `INSERT` 操作 - - - -## 格式概览 {#formats-overview} - -受支持的格式有: - - - -| 格式 | 输入 | 输出 | -| ---------------------------------------------------------------------------------------------------------- | -- | -- | -| [TabSeparated](./formats/TabSeparated/TabSeparated.md) | ✔ | ✔ | -| [TabSeparatedRaw](./formats/TabSeparated/TabSeparatedRaw.md) | ✔ | ✔ | -| [TabSeparatedWithNames](./formats/TabSeparated/TabSeparatedWithNames.md) | ✔ | ✔ | -| [TabSeparatedWithNamesAndTypes](./formats/TabSeparated/TabSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [TabSeparatedRawWithNames](./formats/TabSeparated/TabSeparatedRawWithNames.md) | ✔ | ✔ | -| [TabSeparatedRawWithNamesAndTypes](./formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md) | ✔ | ✔ | -| [模板](./formats/Template/Template.md) | ✔ | ✔ | -| [TemplateIgnoreSpaces](./formats/Template/TemplateIgnoreSpaces.md) | ✔ | ✗ | -| [CSV](./formats/CSV/CSV.md) | ✔ | ✔ | -| [CSVWithNames](./formats/CSV/CSVWithNames.md) | ✔ | ✔ | -| [CSVWithNamesAndTypes](./formats/CSV/CSVWithNamesAndTypes.md) | ✔ | ✔ | -| [CustomSeparated](./formats/CustomSeparated/CustomSeparated.md) | ✔ | ✔ | -| [CustomSeparatedWithNames](./formats/CustomSeparated/CustomSeparatedWithNames.md) | ✔ | ✔ | -| [CustomSeparatedWithNamesAndTypes](./formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md) | ✔ | ✔ | -| [SQLInsert](./formats/SQLInsert.md) | ✗ | ✔ | -| [值](./formats/Values.md) | ✔ | ✔ | -| [垂直](./formats/Vertical.md) | ✗ | ✔ | -| [JSON](./formats/JSON/JSON.md) | ✔ | ✔ | -| [JSONAsString](./formats/JSON/JSONAsString.md) | ✔ | ✗ | -| [JSONAsObject](./formats/JSON/JSONAsObject.md) | ✔ | ✗ | -| [JSONStrings](./formats/JSON/JSONStrings.md) | ✔ | ✔ | -| [JSONColumns](./formats/JSON/JSONColumns.md) | ✔ | ✔ | -| [JSONColumnsWithMetadata](./formats/JSON/JSONColumnsWithMetadata.md) | ✔ | ✔ | -| [JSONCompact](./formats/JSON/JSONCompact.md) | ✔ | ✔ | -| [JSONCompactStrings](./formats/JSON/JSONCompactStrings.md) | ✗ | ✔ | -| [JSONCompactColumns](./formats/JSON/JSONCompactColumns.md) | ✔ | ✔ | -| [JSONEachRow](./formats/JSON/JSONEachRow.md) | ✔ | ✔ | -| [PrettyJSONEachRow](./formats/JSON/PrettyJSONEachRow.md) | ✗ | ✔ | -| [JSONEachRowWithProgress](./formats/JSON/JSONEachRowWithProgress.md) | ✗ | ✔ | -| [JSONStringsEachRow](./formats/JSON/JSONStringsEachRow.md) | ✔ | ✔ | -| [JSONStringsEachRowWithProgress](./formats/JSON/JSONStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactEachRow](./formats/JSON/JSONCompactEachRow.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNames](./formats/JSON/JSONCompactEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactEachRowWithNamesAndTypes](./formats/JSON/JSONCompactEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactEachRowWithProgress](./formats/JSON/JSONCompactEachRowWithProgress.md) | ✗ | ✔ | -| [JSONCompactStringsEachRow](./formats/JSON/JSONCompactStringsEachRow.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNames](./formats/JSON/JSONCompactStringsEachRowWithNames.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithNamesAndTypes](./formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md) | ✔ | ✔ | -| [JSONCompactStringsEachRowWithProgress](./formats/JSON/JSONCompactStringsEachRowWithProgress.md) | ✗ | ✔ | -| [JSONObjectEachRow](./formats/JSON/JSONObjectEachRow.md) | ✔ | ✔ | -| [BSONEachRow](./formats/BSONEachRow.md) | ✔ | ✔ | -| [TSKV](./formats/TabSeparated/TSKV.md) | ✔ | ✔ | -| [Pretty](./formats/Pretty/Pretty.md) | ✗ | ✔ | -| [PrettyNoEscapes](./formats/Pretty/PrettyNoEscapes.md) | ✗ | ✔ | -| [PrettyMonoBlock](./formats/Pretty/PrettyMonoBlock.md) | ✗ | ✔ | -| [PrettyNoEscapesMonoBlock](./formats/Pretty/PrettyNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettyCompact](./formats/Pretty/PrettyCompact.md) | ✗ | ✔ | -| [PrettyCompactNoEscapes](./formats/Pretty/PrettyCompactNoEscapes.md) | ✗ | ✔ | -| [PrettyCompactMonoBlock](./formats/Pretty/PrettyCompactMonoBlock.md) | ✗ | ✔ | -| [PrettyCompactNoEscapesMonoBlock](./formats/Pretty/PrettyCompactNoEscapesMonoBlock.md) | ✗ | ✔ | -| [PrettySpace](./formats/Pretty/PrettySpace.md) | ✗ | ✔ | -| [PrettySpaceNoEscapes](./formats/Pretty/PrettySpaceNoEscapes.md) | ✗ | ✔ | -| [PrettySpaceMonoBlock](./formats/Pretty/PrettySpaceMonoBlock.md) | ✗ | ✔ | -| [PrettySpaceNoEscapesMonoBlock](./formats/Pretty/PrettySpaceNoEscapesMonoBlock.md) | ✗ | ✔ | -| [Prometheus](./formats/Prometheus.md) | ✗ | ✔ | -| [Protobuf](./formats/Protobuf/Protobuf.md) | ✔ | ✔ | -| [ProtobufSingle](./formats/Protobuf/ProtobufSingle.md) | ✔ | ✔ | -| [ProtobufList](./formats/Protobuf/ProtobufList.md) | ✔ | ✔ | -| [Avro](./formats/Avro/Avro.md) | ✔ | ✔ | -| [AvroConfluent](./formats/Avro/AvroConfluent.md) | ✔ | ✗ | -| [Parquet](./formats/Parquet/Parquet.md) | ✔ | ✔ | -| [ParquetMetadata](./formats/Parquet/ParquetMetadata.md) | ✔ | ✗ | -| [Arrow](./formats/Arrow/Arrow.md) | ✔ | ✔ | -| [ArrowStream](./formats/Arrow/ArrowStream.md) | ✔ | ✔ | -| [ORC](./formats/ORC.md) | ✔ | ✔ | -| [One](./formats/One.md) | ✔ | ✗ | -| [Npy](./formats/Npy.md) | ✔ | ✔ | -| [RowBinary](./formats/RowBinary/RowBinary.md) | ✔ | ✔ | -| [RowBinaryWithNames](./formats/RowBinary/RowBinaryWithNames.md) | ✔ | ✔ | -| [RowBinaryWithNamesAndTypes](./formats/RowBinary/RowBinaryWithNamesAndTypes.md) | ✔ | ✔ | -| [RowBinaryWithDefaults](./formats/RowBinary/RowBinaryWithDefaults.md) | ✔ | ✗ | -| [Native](./formats/Native.md) | ✔ | ✔ | -| [Null](./formats/Null.md) | ✗ | ✔ | -| [Hash](./formats/Hash.md) | ✗ | ✔ | -| [XML](./formats/XML.md) | ✗ | ✔ | -| [CapnProto](./formats/CapnProto.md) | ✔ | ✔ | -| [LineAsString](./formats/LineAsString/LineAsString.md) | ✔ | ✔ | -| [LineAsStringWithNames](./formats/LineAsString/LineAsStringWithNames.md) | ✔ | ✔ | -| [LineAsStringWithNamesAndTypes](./formats/LineAsString/LineAsStringWithNamesAndTypes.md) | ✔ | ✔ | -| [正则表达式(Regexp)](./formats/Regexp.md) | ✔ | ✗ | -| [RawBLOB](./formats/RawBLOB.md) | ✔ | ✔ | -| [MsgPack](./formats/MsgPack.md) | ✔ | ✔ | -| [MySQLDump](./formats/MySQLDump.md) | ✔ | ✗ | -| [DWARF](./formats/DWARF.md) | ✔ | ✗ | -| [Markdown](./formats/Markdown.md) | ✗ | ✔ | -| [表单](./formats/Form.md) | ✔ | ✗ | - - - -你可以通过 ClickHouse 的设置来控制某些格式处理参数。欲了解更多信息,请参阅[设置](/operations/settings/settings-formats.md)部分。 - - - -## 格式模式 {#formatschema} - -包含格式模式的文件名由设置 `format_schema` 指定。 -在使用 `Cap'n Proto` 或 `Protobuf` 任一格式时,必须设置该配置。 -格式模式由文件名和该文件中消息类型的名称组成,两者以冒号分隔, -例如:`schemafile.proto:MessageType`。 -如果文件具有该格式的标准扩展名(例如 `Protobuf` 的 `.proto`), -则扩展名可以省略,此时格式模式为 `schemafile:MessageType`。 - -如果通过交互模式下的[客户端](/interfaces/cli.md)进行数据输入或输出,格式模式中指定的文件名 -可以是绝对路径,也可以是相对于客户端当前目录的相对路径。 -如果在[批处理模式](/interfaces/cli.md/#batch-mode)下使用客户端,出于安全考虑,模式文件的路径必须是相对路径。 - -如果通过 [HTTP 接口](/interfaces/http.md) 进行数据输入或输出,则格式模式中指定的文件名 -必须位于服务器配置中由 [format_schema_path](/operations/server-configuration-parameters/settings.md/#format_schema_path) 指定的目录中。 - - - -## 跳过错误 {#skippingerrors} - -某些格式,例如 `CSV`、`TabSeparated`、`TSKV`、`JSONEachRow`、`Template`、`CustomSeparated` 和 `Protobuf`,在发生解析错误时可以跳过有问题的行,并从下一行的开头继续解析。参见 [input_format_allow_errors_num](/operations/settings/settings-formats.md/#input_format_allow_errors_num) 和 -[input_format_allow_errors_ratio](/operations/settings/settings-formats.md/#input_format_allow_errors_ratio) 设置。 -限制条件: -- 在发生解析错误的情况下,`JSONEachRow` 会跳过直到换行符(或 EOF)之前的所有数据,因此行必须由 `\n` 分隔,才能正确统计错误数量。 -- `Template` 和 `CustomSeparated` 使用最后一列之后的分隔符以及行与行之间的分隔符来查找下一行的开头,因此仅当这两类分隔符中至少有一个非空时,错误跳过机制才会生效。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md deleted file mode 100644 index eb2c6f678ab..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/Arrow.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Arrow 格式文档' -input_format: true -keywords: ['Arrow'] -output_format: true -slug: /interfaces/formats/Arrow -title: 'Arrow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -[Apache Arrow](https://arrow.apache.org/) 提供两种内置的列式存储格式。ClickHouse 支持对这两种格式进行读写操作。 -`Arrow` 是 Apache Arrow 的“文件模式(file mode)”格式,适用于内存中的随机访问。 - -## 数据类型匹配 {#data-types-matching} - -下表列出了支持的数据类型,以及它们在 `INSERT` 和 `SELECT` 查询中与 ClickHouse [数据类型](/sql-reference/data-types/index.md) 的对应关系。 - -| Arrow data type (`INSERT`) | ClickHouse data type | Arrow data type (`SELECT`) | -|-----------------------------------------|------------------------------------------------------------------------------------------------------------|----------------------------| -| `BOOL` | [Bool](/sql-reference/data-types/boolean.md) | `BOOL` | -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md) | `INT64` | -| `FLOAT`, `HALF_FLOAT` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `DATE32` | [Date32](/sql-reference/data-types/date32.md) | `UINT16` | -| `DATE64` | [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](/sql-reference/data-types/datetime64.md) | `TIMESTAMP` | -| `STRING`, `BINARY` | [String](/sql-reference/data-types/string.md) | `BINARY` | -| `STRING`, `BINARY`, `FIXED_SIZE_BINARY` | [FixedString](/sql-reference/data-types/fixedstring.md) | `FIXED_SIZE_BINARY` | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | `DECIMAL` | -| `DECIMAL256` | [Decimal256](/sql-reference/data-types/decimal.md) | `DECIMAL256` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `MAP` | [Map](/sql-reference/data-types/map.md) | `MAP` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `FIXED_SIZE_BINARY`, `BINARY` | [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_SIZE_BINARY` | -| `FIXED_SIZE_BINARY`, `BINARY` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_SIZE_BINARY` | - -Array 类型可以嵌套,并且可以将 `Nullable` 类型的值作为参数。`Tuple` 和 `Map` 类型同样可以嵌套。 - -在 `INSERT` 查询中支持使用 `DICTIONARY` 类型;对于 `SELECT` 查询,可以通过 [`output_format_arrow_low_cardinality_as_dictionary`](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary) 设置,将 [LowCardinality](/sql-reference/data-types/lowcardinality.md) 类型作为 `DICTIONARY` 类型输出。 - -不支持的 Arrow 数据类型: - -- `FIXED_SIZE_BINARY` -- `JSON` -- `UUID` -- `ENUM` - -ClickHouse 表列的数据类型不必与对应的 Arrow 数据字段完全一致。插入数据时,ClickHouse 会根据上表解析数据类型,然后将数据[转换](/sql-reference/functions/type-conversion-functions#cast)为为 ClickHouse 表列所设置的数据类型。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -可以使用以下命令,将文件中的 Arrow 数据插入到 ClickHouse 表中: - -```bash -$ cat filename.arrow | clickhouse-client --query="INSERT INTO some_table FORMAT Arrow" -``` - - -### 选择数据 {#selecting-data} - -可以使用以下命令,从 ClickHouse 表中选择数据,并将其保存为 Arrow 格式的文件: - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Arrow" > {filename.arrow} -``` - - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | -|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------|--------------| -| `input_format_arrow_allow_missing_columns` | 读取 Arrow 输入格式时允许列缺失 | `1` | -| `input_format_arrow_case_insensitive_column_matching` | 在将 Arrow 列与 CH 列匹配时忽略大小写 | `0` | -| `input_format_arrow_import_nested` | 已废弃的设置,不再产生任何效果。 | `0` | -| `input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference` | 在推断 Arrow 格式的模式时,跳过类型不受支持的列 | `0` | -| `output_format_arrow_compression_method` | Arrow 输出格式的压缩方法。支持的编解码器:`lz4_frame`、`zstd`、`none`(不压缩) | `lz4_frame` | -| `output_format_arrow_fixed_string_as_fixed_byte_array` | 对 FixedString 列使用 Arrow 的 FIXED_SIZE_BINARY 类型而不是 Binary | `1` | -| `output_format_arrow_low_cardinality_as_dictionary` | 启用将 LowCardinality 类型输出为 Arrow 的 Dictionary 类型 | `0` | -| `output_format_arrow_string_as_string` | 对 String 列使用 Arrow 的 String 类型而不是 Binary | `1` | -| `output_format_arrow_use_64_bit_indexes_for_dictionary` | 在 Arrow 格式中始终对字典索引使用 64 位整数 | `0` | -| `output_format_arrow_use_signed_indexes_for_dictionary` | 在 Arrow 格式中对字典索引使用有符号整数 | `1` | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md deleted file mode 100644 index c7e5f1f91aa..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Arrow/ArrowStream.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -alias: [] -description: 'ArrowStream 格式的文档' -input_format: true -keywords: ['ArrowStream'] -output_format: true -slug: /interfaces/formats/ArrowStream -title: 'ArrowStream' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -`ArrowStream` 是 Apache Arrow 的“流式模式”格式,专为内存流处理而设计。 - -## 示例用法 {#example-usage} - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md deleted file mode 100644 index e8273480aff..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/Avro.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'Avro 格式文档' -input_format: true -keywords: ['Avro'] -output_format: true -slug: /interfaces/formats/Avro -title: 'Avro' -doc_type: 'reference' ---- - -import DataTypeMapping from './_snippets/data-types-matching.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -[Apache Avro](https://avro.apache.org/) 是一种面向行的序列化格式,使用二进制编码,实现高效的数据处理。`Avro` 格式支持读写 [Avro 数据文件](https://avro.apache.org/docs/++version++/specification/#object-container-files)。此格式要求消息是自描述的,并在其中内嵌 schema。如果您将 Avro 与 schema registry 结合使用,请参阅 [`AvroConfluent`](./AvroConfluent.md) 格式。 - -## 数据类型映射 {#data-type-mapping} - - - -## 格式设置 {#format-settings} - -| 设置 | 说明 | 默认值 | -|---------------------------------------------|----------------------------------------------------------------------------------------------------|---------| -| `input_format_avro_allow_missing_fields` | 当在模式(schema)中找不到某个字段时,是否使用默认值而不是抛出错误。 | `0` | -| `input_format_avro_null_as_default` | 当向非空列插入 `null` 值时,是否使用默认值而不是抛出错误。 | `0` | -| `output_format_avro_codec` | Avro 输出文件的压缩算法。可选值:`null`、`deflate`、`snappy`、`zstd`。 | | -| `output_format_avro_sync_interval` | Avro 文件中的同步标记频率(以字节为单位)。 | `16384` | -| `output_format_avro_string_column_pattern` | 用于识别需要映射为 Avro 字符串类型的 `String` 列的正则表达式。默认情况下,ClickHouse 的 `String` 列会被写入为 Avro 的 `bytes` 类型。 | | -| `output_format_avro_rows_in_file` | 每个 Avro 输出文件允许的最大行数。达到此限制时会创建一个新文件(如果存储系统支持文件拆分)。 | `1` | - -## 示例 {#examples} - -### 读取 Avro 数据 {#reading-avro-data} - -要从 Avro 文件中读取数据到 ClickHouse 表中: - -```bash -$ cat file.avro | clickhouse-client --query="INSERT INTO {some_table} FORMAT Avro" -``` - -摄取的 Avro 文件的根模式必须是 `record` 类型。 - -为了在表列与 Avro 模式中的字段之间建立对应关系,ClickHouse 会比较它们的名称。 -此比较区分大小写,且未使用的字段会被跳过。 - -ClickHouse 表列的数据类型可以与插入的 Avro 数据中对应字段的数据类型不同。在插入数据时,ClickHouse 会根据上表解释数据类型,然后将数据按照对应的列类型进行[类型转换(cast)](/sql-reference/functions/type-conversion-functions#cast)。 - -在导入数据时,如果在模式中找不到某个字段,并且已启用设置 [`input_format_avro_allow_missing_fields`](/operations/settings/settings-formats.md/#input_format_avro_allow_missing_fields),则会使用默认值,而不是抛出错误。 - - -### 写入 Avro 数据 {#writing-avro-data} - -要将 ClickHouse 表中的数据写入 Avro 文件: - -```bash -$ clickhouse-client --query="SELECT * FROM {some_table} FORMAT Avro" > file.avro -``` - -列名必须: - -* 以 `[A-Za-z_]` 开头 -* 后续字符只能包含 `[A-Za-z0-9_]` - -Avro 文件的输出压缩方式和同步间隔可以分别通过 [`output_format_avro_codec`](/operations/settings/settings-formats.md/#output_format_avro_codec) 和 [`output_format_avro_sync_interval`](/operations/settings/settings-formats.md/#output_format_avro_sync_interval) 设置进行配置。 - - -### 推断 Avro 模式 {#inferring-the-avro-schema} - -使用 ClickHouse 的 [`DESCRIBE`](/sql-reference/statements/describe-table) 函数,可以快速查看 Avro 文件的推断格式,如下例所示。 -此示例包含 ClickHouse S3 公共 bucket 中一个可公开访问的 Avro 文件的 URL: - -```sql -DESCRIBE url('https://clickhouse-public-datasets.s3.eu-central-1.amazonaws.com/hits.avro','Avro); - -┌─name───────────────────────┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ WatchID │ Int64 │ │ │ │ │ │ -│ JavaEnable │ Int32 │ │ │ │ │ │ -│ Title │ String │ │ │ │ │ │ -│ GoodEvent │ Int32 │ │ │ │ │ │ -│ EventTime │ Int32 │ │ │ │ │ │ -│ EventDate │ Date32 │ │ │ │ │ │ -│ CounterID │ Int32 │ │ │ │ │ │ -│ ClientIP │ Int32 │ │ │ │ │ │ -│ ClientIP6 │ FixedString(16) │ │ │ │ │ │ -│ RegionID │ Int32 │ │ │ │ │ │ -... -│ IslandID │ FixedString(16) │ │ │ │ │ │ -│ RequestNum │ Int32 │ │ │ │ │ │ -│ RequestTry │ Int32 │ │ │ │ │ │ -└────────────────────────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md deleted file mode 100644 index a08ee563204..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/AvroConfluent.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -alias: [] -description: 'AvroConfluent 格式文档' -input_format: true -keywords: ['AvroConfluent'] -output_format: false -slug: /interfaces/formats/AvroConfluent -title: 'AvroConfluent' -doc_type: 'reference' ---- - -import DataTypesMatching from './_snippets/data-types-matching.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✗ | | - - -## 描述 {#description} - -[Apache Avro](https://avro.apache.org/) 是一种面向行的序列化格式,使用二进制编码以实现高效的数据处理。`AvroConfluent` 格式支持解码使用 [Confluent Schema Registry](https://docs.confluent.io/current/schema-registry/index.html)(或 API 兼容服务)序列化的、单对象且使用 Avro 编码的 Kafka 消息。 - -每条 Avro 消息都会嵌入一个模式 ID(schema ID),ClickHouse 会通过查询已配置的 Schema Registry 自动解析该 ID。模式解析完成后会被缓存,以获得最佳性能。 - -
- -## 数据类型映射 {#data-type-mapping} - - - -## 格式设置 {#format-settings} - -[//]: # "NOTE These settings can be set at a session-level, but this isn't common and documenting it too prominently can be confusing to users." - -| Setting | Description | Default | -|---------------------------------------------|-----------------------------------------------------------------------------------------------------|---------| -| `input_format_avro_allow_missing_fields` | 指定在模式中找不到字段时,是否使用默认值而不是抛出错误。 | `0` | -| `input_format_avro_null_as_default` | 指定在向非空列插入 `null` 值时,是否使用默认值而不是抛出错误。 | `0` | -| `format_avro_schema_registry_url` | Confluent Schema Registry 的 URL。对于基本身份验证,可以在 URL 中直接包含经过 URL 编码的凭据。 | | - -## 示例 {#examples} - -### 使用 schema registry {#using-a-schema-registry} - -要使用 [Kafka 表引擎](/engines/table-engines/integrations/kafka.md) 读取使用 Avro 编码的 Kafka 主题,请通过 `format_avro_schema_registry_url` 设置指定 schema registry 的 URL。 - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'http://schema-registry-url'; - -SELECT * FROM topic1_stream; -``` - - -#### 使用基本身份验证 {#using-basic-authentication} - -如果 schema registry 需要基本身份验证(例如使用 Confluent Cloud 时),可以在 `format_avro_schema_registry_url` 设置中提供经过 URL 编码的凭证。 - -```sql -CREATE TABLE topic1_stream -( - field1 String, - field2 String -) -ENGINE = Kafka() -SETTINGS -kafka_broker_list = 'kafka-broker', -kafka_topic_list = 'topic1', -kafka_group_name = 'group1', -kafka_format = 'AvroConfluent', -format_avro_schema_registry_url = 'https://:@schema-registry-url'; -``` - - -## 故障排查 {#troubleshooting} - -要监控摄取进度并调试 Kafka 消费者的错误,可以查询 [`system.kafka_consumers` 系统表](../../../operations/system-tables/kafka_consumers.md)。如果您的部署有多个副本(例如 ClickHouse Cloud),则必须使用 [`clusterAllReplicas`](../../../sql-reference/table-functions/cluster.md) 表函数。 - -```sql -SELECT * FROM clusterAllReplicas('default',system.kafka_consumers) -ORDER BY assignments.partition_id ASC; -``` - -如果遇到 schema 解析相关问题,可以使用 [kafkacat](https://github.com/edenhill/kafkacat) 搭配 [clickhouse-local](/operations/utilities/clickhouse-local.md) 进行排查: - -```bash -$ kafkacat -b kafka-broker -C -t topic1 -o beginning -f '%s' -c 3 | clickhouse-local --input-format AvroConfluent --format_avro_schema_registry_url 'http://schema-registry' -S "field1 Int64, field2 String" -q 'select * from table' -1 a -2 b -3 c -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md deleted file mode 100644 index aac3610bf27..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Avro/_snippets/data-types-matching.md +++ /dev/null @@ -1,40 +0,0 @@ -下表展示了 Apache Avro 格式支持的所有数据类型,以及它们在 `INSERT` 和 `SELECT` 查询中对应的 ClickHouse [数据类型](/sql-reference/data-types/index.md)。 - -| Avro 数据类型 `INSERT` | ClickHouse 数据类型 | Avro 数据类型 `SELECT` | -|---------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------|---------------------------------| -| `boolean`, `int`, `long`, `float`, `double` | [Int(8\16\32)](/sql-reference/data-types/int-uint.md), [UInt(8\16\32)](/sql-reference/data-types/int-uint.md) | `int` | -| `boolean`, `int`, `long`, `float`, `double` | [Int64](/sql-reference/data-types/int-uint.md), [UInt64](/sql-reference/data-types/int-uint.md) | `long` | -| `boolean`, `int`, `long`, `float`, `double` | [Float32](/sql-reference/data-types/float.md) | `float` | -| `boolean`, `int`, `long`, `float`, `double` | [Float64](/sql-reference/data-types/float.md) | `double` | -| `bytes`, `string`, `fixed`, `enum` | [String](/sql-reference/data-types/string.md) | `bytes` 或 `string` \* | -| `bytes`, `string`, `fixed` | [FixedString(N)](/sql-reference/data-types/fixedstring.md) | `fixed(N)` | -| `enum` | [Enum(8\16)](/sql-reference/data-types/enum.md) | `enum` | -| `array(T)` | [Array(T)](/sql-reference/data-types/array.md) | `array(T)` | -| `map(V, K)` | [Map(V, K)](/sql-reference/data-types/map.md) | `map(string, K)` | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(null, T)` | -| `union(T1, T2, …)` \** | [Variant(T1, T2, …)](/sql-reference/data-types/variant.md) | `union(T1, T2, …)` \** | -| `null` | [Nullable(Nothing)](/sql-reference/data-types/special-data-types/nothing.md) | `null` | -| `int (date)` \**\* | [Date](/sql-reference/data-types/date.md), [Date32](/sql-reference/data-types/date32.md) | `int (date)` \**\* | -| `long (timestamp-millis)` \**\* | [DateTime64(3)](/sql-reference/data-types/datetime.md) | `long (timestamp-millis)` \**\* | -| `long (timestamp-micros)` \**\* | [DateTime64(6)](/sql-reference/data-types/datetime.md) | `long (timestamp-micros)` \**\* | -| `bytes (decimal)` \**\* | [DateTime64(N)](/sql-reference/data-types/datetime.md) | `bytes (decimal)` \**\* | -| `int` | [IPv4](/sql-reference/data-types/ipv4.md) | `int` | -| `fixed(16)` | [IPv6](/sql-reference/data-types/ipv6.md) | `fixed(16)` | -| `bytes (decimal)` \**\* | [Decimal(P, S)](/sql-reference/data-types/decimal.md) | `bytes (decimal)` \**\* | -| `string (uuid)` \**\* | [UUID](/sql-reference/data-types/uuid.md) | `string (uuid)` \**\* | -| `fixed(16)` | [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `fixed(16)` | -| `fixed(32)` | [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `fixed(32)` | -| `record` | [Tuple](/sql-reference/data-types/tuple.md) | `record` | - -\* 默认值为 `bytes`,此行为由设置 [`output_format_avro_string_column_pattern`](/operations/settings/settings-formats.md/#output_format_avro_string_column_pattern) 控制 - -\** [Variant 类型](/sql-reference/data-types/variant) 会隐式接受 `null` 作为字段值,因此,例如 Avro 的 `union(T1, T2, null)` 会被转换为 `Variant(T1, T2)`。 -因此,当从 ClickHouse 生成 Avro 时,我们必须始终在 Avro 的 `union` 类型集合中包含 `null` 类型,因为在模式推断期间我们无法得知是否有任何值实际为 `null`。 - -\**\* [Avro 逻辑类型](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -不支持的 Avro 逻辑数据类型: - -- `time-millis` -- `time-micros` -- `duration` \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md deleted file mode 100644 index 4f04e2ef274..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/BSONEachRow.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -alias: [] -description: 'BSONEachRow 格式文档' -input_format: true -keywords: ['BSONEachRow'] -output_format: true -slug: /interfaces/formats/BSONEachRow -title: 'BSONEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -`BSONEachRow` 格式将数据解析为一系列二进制 JSON(BSON)文档,文档之间没有任何分隔符。 -每一行会被格式化为一个单独的文档,每一列会被格式化为该文档中的一个 BSON 字段,列名作为字段名。 - -## 数据类型匹配 {#data-types-matching} - -在输出时,使用以下 ClickHouse 类型与 BSON 类型之间的对应关系: - -| ClickHouse 类型 | BSON 类型 | -|-----------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------| -| [Bool](/sql-reference/data-types/boolean.md) | `\x08` boolean | -| [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `\x10` int32 | -| [Int32](/sql-reference/data-types/int-uint.md) | `\x10` int32 | -| [UInt32](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Int64/UInt64](/sql-reference/data-types/int-uint.md) | `\x12` int64 | -| [Float32/Float64](/sql-reference/data-types/float.md) | `\x01` double | -| [Date](/sql-reference/data-types/date.md)/[Date32](/sql-reference/data-types/date32.md) | `\x10` int32 | -| [DateTime](/sql-reference/data-types/datetime.md) | `\x12` int64 | -| [DateTime64](/sql-reference/data-types/datetime64.md) | `\x09` datetime | -| [Decimal32](/sql-reference/data-types/decimal.md) | `\x10` int32 | -| [Decimal64](/sql-reference/data-types/decimal.md) | `\x12` int64 | -| [Decimal128](/sql-reference/data-types/decimal.md) | `\x05` binary, `\x00` binary subtype, size = 16 | -| [Decimal256](/sql-reference/data-types/decimal.md) | `\x05` binary, `\x00` binary subtype, size = 32 | -| [Int128/UInt128](/sql-reference/data-types/int-uint.md) | `\x05` binary, `\x00` binary subtype, size = 16 | -| [Int256/UInt256](/sql-reference/data-types/int-uint.md) | `\x05` binary, `\x00` binary subtype, size = 32 | -| [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | `\x05` binary, `\x00` binary subtype 或 \x02 string(如果启用了 output_format_bson_string_as_string 设置) | -| [UUID](/sql-reference/data-types/uuid.md) | `\x05` binary, `\x04` uuid subtype, size = 16 | -| [Array](/sql-reference/data-types/array.md) | `\x04` array | -| [Tuple](/sql-reference/data-types/tuple.md) | `\x04` array | -| [Named Tuple](/sql-reference/data-types/tuple.md) | `\x03` document | -| [Map](/sql-reference/data-types/map.md) | `\x03` document | -| [IPv4](/sql-reference/data-types/ipv4.md) | `\x10` int32 | -| [IPv6](/sql-reference/data-types/ipv6.md) | `\x05` binary, `\x00` binary subtype | - -在输入时,使用以下 BSON 类型与 ClickHouse 类型之间的对应关系: - -| BSON Type | ClickHouse 类型 | -|------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `\x01` double | [Float32/Float64](/sql-reference/data-types/float.md) | -| `\x02` string | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x03` document | [Map](/sql-reference/data-types/map.md)/[Named Tuple](/sql-reference/data-types/tuple.md) | -| `\x04` array | [Array](/sql-reference/data-types/array.md)/[Tuple](/sql-reference/data-types/tuple.md) | -| `\x05` binary, `\x00` binary subtype | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md)/[IPv6](/sql-reference/data-types/ipv6.md) | -| `\x05` binary, `\x02` old binary subtype | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x05` binary, `\x03` old uuid subtype | [UUID](/sql-reference/data-types/uuid.md) | -| `\x05` binary, `\x04` uuid subtype | [UUID](/sql-reference/data-types/uuid.md) | -| `\x07` ObjectId | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x08` boolean | [Bool](/sql-reference/data-types/boolean.md) | -| `\x09` datetime | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `\x0A` null value | [NULL](/sql-reference/data-types/nullable.md) | -| `\x0D` JavaScript code | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x0E` symbol | [String](/sql-reference/data-types/string.md)/[FixedString](/sql-reference/data-types/fixedstring.md) | -| `\x10` int32 | [Int32/UInt32](/sql-reference/data-types/int-uint.md)/[Decimal32](/sql-reference/data-types/decimal.md)/[IPv4](/sql-reference/data-types/ipv4.md)/[Enum8/Enum16](/sql-reference/data-types/enum.md) | -| `\x12` int64 | [Int64/UInt64](/sql-reference/data-types/int-uint.md)/[Decimal64](/sql-reference/data-types/decimal.md)/[DateTime64](/sql-reference/data-types/datetime64.md) | - -其他 BSON 类型不受支持。此外,该格式也会在不同整数类型之间进行转换。 -例如,可以将 BSON `int32` 值作为 [`UInt8`](../../sql-reference/data-types/int-uint.md) 插入到 ClickHouse 中。 - -`Int128`/`UInt128`/`Int256`/`UInt256`/`Decimal128`/`Decimal256` 等大整数和小数可以从具有 `\x00` 二进制子类型的 BSON Binary 值中解析。 -在这种情况下,该格式会验证二进制数据的大小是否与预期值的大小一致。 - -:::note -此格式在大端(Big-Endian)平台上不能正常工作。 -::: - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用名为 `football.bson`、包含以下数据的 BSON 文件: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.bson' FORMAT BSONEachRow; -``` - - -### 读取数据 {#reading-data} - -以 `BSONEachRow` 格式读取数据: - -```sql -SELECT * -FROM football INTO OUTFILE 'docs_data/bson/football.bson' -FORMAT BSONEachRow -``` - -:::tip -BSON 是一种二进制格式,无法在终端以人类可读形式显示。使用 `INTO OUTFILE` 将数据输出为 BSON 文件。 -::: - - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|----------| -| [`output_format_bson_string_as_string`](../../operations/settings/settings-formats.md/#output_format_bson_string_as_string) | 对 String 列使用 BSON String 类型而不是 Binary 类型。 | `false` | -| [`input_format_bson_skip_fields_with_unsupported_types_in_schema_inference`](../../operations/settings/settings-formats.md/#input_format_bson_skip_fields_with_unsupported_types_in_schema_inference) | 在对 BSONEachRow 格式进行 schema 推断时,允许跳过类型不受支持的列。 | `false` | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md deleted file mode 100644 index 408f2268c09..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSV.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -alias: [] -description: 'CSV 格式的文档' -input_format: true -keywords: ['CSV'] -output_format: true -slug: /interfaces/formats/CSV -title: 'CSV' -doc_type: 'reference' ---- - -## 描述 {#description} - -逗号分隔值格式([RFC](https://tools.ietf.org/html/rfc4180))。 -在格式化时,每行都用双引号括起来。字符串内部的双引号会输出为连续的两个双引号。 -没有其他用于转义字符的规则。 - -* 日期和日期时间用双引号括起来。 -* 数字在输出时不带引号。 -* 值由分隔符分隔,默认是 `,`。分隔符通过设置 [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) 定义。 -* 行之间使用 Unix 换行符(LF)分隔。 -* 数组在 CSV 中按如下方式序列化: - * 首先,数组按 TabSeparated 格式序列化为字符串; - * 然后将得到的字符串在 CSV 中以双引号输出。 -* CSV 格式中的元组会被序列化为单独的列(即元组中的嵌套结构会丢失)。 - -```bash -$ clickhouse-client --format_csv_delimiter="|" --query="INSERT INTO test.csv FORMAT CSV" < data.csv -``` - -:::note -默认情况下,分隔符为 `,`。 -有关更多信息,请参阅设置 [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter)。 -::: - -在解析时,所有值可以带引号或不带引号进行解析。支持双引号和单引号。 - -行也可以在没有引号的情况下书写。在这种情况下,将一直解析到分隔符字符或换行符(CR 或 LF)为止。 -但是,与 RFC 规范不一致的是,在解析无引号的行时,开头和结尾的空格及制表符会被忽略。 -换行符支持:Unix (LF)、Windows (CR LF) 和 Mac OS Classic (CR LF) 类型。 - -`NULL` 的格式由设置 [format_csv_null_representation](/operations/settings/settings-formats.md/#format_csv_null_representation) 决定(默认值为 `\N`)。 - -在输入数据中,`ENUM` 值可以表示为名称或 ID。 -首先,我们会尝试将输入值与 ENUM 名称匹配。 -如果失败并且输入值是数字,则会尝试将该数字与 ENUM ID 匹配。 -如果输入数据只包含 ENUM ID,建议启用设置 [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) 以优化 `ENUM` 解析。 - - -## 示例用法 {#example-usage} - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | 说明 | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [format_csv_delimiter](/operations/settings/settings-formats.md/#format_csv_delimiter) | 在 CSV 数据中视作分隔符的字符。 | `,` | | -| [format_csv_allow_single_quotes](/operations/settings/settings-formats.md/#format_csv_allow_single_quotes) | 允许使用单引号包裹字符串。 | `true` | | -| [format_csv_allow_double_quotes](/operations/settings/settings-formats.md/#format_csv_allow_double_quotes) | 允许使用双引号包裹字符串。 | `true` | | -| [format_csv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation) | 在 CSV 格式中自定义 NULL 的表示形式。 | `\N` | | -| [input_format_csv_empty_as_default](/operations/settings/settings-formats.md/#input_format_csv_empty_as_default) | 将 CSV 输入中的空字段视为默认值。 | `true` | 对于复杂的默认表达式,还必须启用 [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)。 | -| [input_format_csv_enum_as_number](/operations/settings/settings-formats.md/#input_format_csv_enum_as_number) | 在 CSV 格式中将插入的枚举值视为枚举索引。 | `false` | | -| [input_format_csv_use_best_effort_in_schema_inference](/operations/settings/settings-formats.md/#input_format_csv_use_best_effort_in_schema_inference) | 使用一些优化和启发式规则来推断 CSV 格式中的 schema(模式)。若禁用,所有字段都会被推断为 String 类型。 | `true` | | -| [input_format_csv_arrays_as_nested_csv](/operations/settings/settings-formats.md/#input_format_csv_arrays_as_nested_csv) | 从 CSV 读取 Array 时,假定其元素已按嵌套 CSV 方式序列化后再写入为字符串。 | `false` | | -| [output_format_csv_crlf_end_of_line](/operations/settings/settings-formats.md/#output_format_csv_crlf_end_of_line) | 如果设置为 true,则 CSV 输出格式中的行结束符为 `\r\n` 而不是 `\n`。 | `false` | | -| [input_format_csv_skip_first_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_first_lines) | 跳过数据开头指定数量的行。 | `0` | | -| [input_format_csv_detect_header](/operations/settings/settings-formats.md/#input_format_csv_detect_header) | 在 CSV 格式中自动检测包含名称和类型的表头。 | `true` | | -| [input_format_csv_skip_trailing_empty_lines](/operations/settings/settings-formats.md/#input_format_csv_skip_trailing_empty_lines) | 跳过数据末尾的空行。 | `false` | | -| [input_format_csv_trim_whitespaces](/operations/settings/settings-formats.md/#input_format_csv_trim_whitespaces) | 去除未加引号的 CSV 字符串中的空格和制表符。 | `true` | | -| [input_format_csv_allow_whitespace_or_tab_as_delimiter](/operations/settings/settings-formats.md/#input_format_csv_allow_whitespace_or_tab_as_delimiter) | 允许在 CSV 字符串中使用空格或制表符作为字段分隔符。 | `false` | | -| [input_format_csv_allow_variable_number_of_columns](/operations/settings/settings-formats.md/#input_format_csv_allow_variable_number_of_columns) | 允许 CSV 格式中的列数可变,忽略多余列,并对缺失列使用默认值。 | `false` | | -| [input_format_csv_use_default_on_bad_values](/operations/settings/settings-formats.md/#input_format_csv_use_default_on_bad_values) | 当 CSV 字段因非法值反序列化失败时,允许为该列设置默认值。 | `false` | | -| [input_format_csv_try_infer_numbers_from_strings](/operations/settings/settings-formats.md/#input_format_csv_try_infer_numbers_from_strings) | 在 schema 推断期间,尝试从字符串字段中推断数字类型。 | `false` | | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md deleted file mode 100644 index bdb8ff80347..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNames.md +++ /dev/null @@ -1,114 +0,0 @@ ---- -alias: [] -description: 'CSV 格式文档' -input_format: true -keywords: ['CSVWithNames'] -output_format: true -slug: /interfaces/formats/CSVWithNames -title: 'CSVWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -同样会输出包含列名的表头行,类似于 [TabSeparatedWithNames](/interfaces/formats/TabSeparatedWithNames)。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -:::tip -自 [23.1 版本](https://github.com/ClickHouse/ClickHouse/releases)起,在使用 `CSV` 格式时,ClickHouse 会自动检测 CSV 文件中的表头,因此无需再使用 `CSVWithNames` 或 `CSVWithNamesAndTypes`。 -::: - -使用以下名为 `football.csv` 的 CSV 文件: - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -创建表: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -使用 `CSVWithNames` 格式插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.csv' FORMAT CSVWithNames; -``` - - -### 读取数据 {#reading-data} - -使用 `CSVWithNames` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT CSVWithNames -``` - -输出将是一个仅包含单行表头的 CSV 文件: - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则输入数据中的列会按名称映射到表中的列;如果同时将 [input_format_skip_unknown_fields](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则名称未知的列会被跳过。 -否则,第一行会被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md deleted file mode 100644 index 0a89c95f2df..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CSV/CSVWithNamesAndTypes.md +++ /dev/null @@ -1,121 +0,0 @@ ---- -alias: [] -description: 'CSVWithNamesAndTypes 格式文档' -input_format: true -keywords: ['CSVWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CSVWithNamesAndTypes -title: 'CSVWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -还会输出两行表头,其中包含列名和类型,类似于 [TabSeparatedWithNamesAndTypes](../formats/TabSeparatedWithNamesAndTypes)。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -:::tip -从 [版本](https://github.com/ClickHouse/ClickHouse/releases) 23.1 开始,ClickHouse 在使用 `CSV` 格式时会自动检测 CSV 文件中的标题行,因此不再需要使用 `CSVWithNames` 或 `CSVWithNamesAndTypes`。 -::: - -使用以下名为 `football_types.csv` 的 CSV 文件: - -```csv -date,season,home_team,away_team,home_team_goals,away_team_goals -Date,Int16,LowCardinality(String),LowCardinality(String),Int8,Int8 -2022-04-30,2021,Sutton United,Bradford City,1,4 -2022-04-30,2021,Swindon Town,Barrow,2,1 -2022-04-30,2021,Tranmere Rovers,Oldham Athletic,2,0 -2022-05-02,2021,Salford City,Mansfield Town,2,2 -2022-05-02,2021,Port Vale,Newport County,1,2 -2022-05-07,2021,Barrow,Northampton Town,1,3 -2022-05-07,2021,Bradford City,Carlisle United,2,0 -2022-05-07,2021,Bristol Rovers,Scunthorpe United,7,0 -2022-05-07,2021,Exeter City,Port Vale,0,1 -2022-05-07,2021,Harrogate Town A.F.C.,Sutton United,0,2 -2022-05-07,2021,Hartlepool United,Colchester United,0,2 -2022-05-07,2021,Leyton Orient,Tranmere Rovers,0,1 -2022-05-07,2021,Mansfield Town,Forest Green Rovers,2,2 -2022-05-07,2021,Newport County,Rochdale,0,2 -2022-05-07,2021,Oldham Athletic,Crawley Town,3,3 -2022-05-07,2021,Stevenage Borough,Salford City,4,2 -2022-05-07,2021,Walsall,Swindon Town,0,3 -``` - -创建表: - -```sql -CREATE TABLE football -( - `date` Date, - `season` Int16, - `home_team` LowCardinality(String), - `away_team` LowCardinality(String), - `home_team_goals` Int8, - `away_team_goals` Int8 -) -ENGINE = MergeTree -ORDER BY (date, home_team); -``` - -使用 `CSVWithNamesAndTypes` 格式插入数据: - -```sql -INSERT INTO football FROM INFILE 'football_types.csv' FORMAT CSVWithNamesAndTypes; -``` - - -### 读取数据 {#reading-data} - -使用 `CSVWithNamesAndTypes` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT CSVWithNamesAndTypes -``` - -输出将是一个包含两行表头的 CSV 文件:第一行为列名,第二行为列类型: - -```csv -"date","season","home_team","away_team","home_team_goals","away_team_goals" -"Date","Int16","LowCardinality(String)","LowCardinality(String)","Int8","Int8" -"2022-04-30",2021,"Sutton United","Bradford City",1,4 -"2022-04-30",2021,"Swindon Town","Barrow",2,1 -"2022-04-30",2021,"Tranmere Rovers","Oldham Athletic",2,0 -"2022-05-02",2021,"Port Vale","Newport County",1,2 -"2022-05-02",2021,"Salford City","Mansfield Town",2,2 -"2022-05-07",2021,"Barrow","Northampton Town",1,3 -"2022-05-07",2021,"Bradford City","Carlisle United",2,0 -"2022-05-07",2021,"Bristol Rovers","Scunthorpe United",7,0 -"2022-05-07",2021,"Exeter City","Port Vale",0,1 -"2022-05-07",2021,"Harrogate Town A.F.C.","Sutton United",0,2 -"2022-05-07",2021,"Hartlepool United","Colchester United",0,2 -"2022-05-07",2021,"Leyton Orient","Tranmere Rovers",0,1 -"2022-05-07",2021,"Mansfield Town","Forest Green Rovers",2,2 -"2022-05-07",2021,"Newport County","Rochdale",0,2 -"2022-05-07",2021,"Oldham Athletic","Crawley Town",3,3 -"2022-05-07",2021,"Stevenage Borough","Salford City",4,2 -"2022-05-07",2021,"Walsall","Swindon Town",0,3 -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则会根据列名将输入数据中的列映射到表中的列;如果将 [input_format_skip_unknown_fields](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则会跳过名称未知的列。 -否则,将跳过第一行。 -::: - -:::note -如果将 [input_format_with_types_use_header](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) 设置为 `1`, -则会将输入数据中的类型与表中对应列的类型进行比较。否则,将跳过第二行。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md deleted file mode 100644 index 29e85b44a93..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CapnProto.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -alias: [] -description: 'CapnProto 文档' -input_format: true -keywords: ['CapnProto'] -output_format: true -slug: /interfaces/formats/CapnProto -title: 'CapnProto' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -`CapnProto` 格式是一种二进制消息格式,类似 [`Protocol Buffers`](https://developers.google.com/protocol-buffers/) 格式和 [Thrift](https://en.wikipedia.org/wiki/Apache_Thrift),但不同于 [JSON](./JSON/JSON.md) 或 [MessagePack](https://msgpack.org/)。 -CapnProto 消息是严格类型且非自描述的,这意味着它们需要外部的 schema 定义。Schema 会在运行时应用,并针对每个查询进行缓存。 - -另请参阅 [Format Schema](/interfaces/formats/#formatschema)。 - -## 数据类型匹配 {#data_types-matching-capnproto} - -下表显示了支持的数据类型,以及它们在 `INSERT` 和 `SELECT` 查询中对应的 ClickHouse [数据类型](/sql-reference/data-types/index.md)。 - -| CapnProto 数据类型(`INSERT`) | ClickHouse 数据类型 | CapnProto 数据类型(`SELECT`) | -|------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------| -| `UINT8`, `BOOL` | [UInt8](/sql-reference/data-types/int-uint.md) | `UINT8` | -| `INT8` | [Int8](/sql-reference/data-types/int-uint.md) | `INT8` | -| `UINT16` | [UInt16](/sql-reference/data-types/int-uint.md), [Date](/sql-reference/data-types/date.md) | `UINT16` | -| `INT16` | [Int16](/sql-reference/data-types/int-uint.md) | `INT16` | -| `UINT32` | [UInt32](/sql-reference/data-types/int-uint.md), [DateTime](/sql-reference/data-types/datetime.md) | `UINT32` | -| `INT32` | [Int32](/sql-reference/data-types/int-uint.md), [Decimal32](/sql-reference/data-types/decimal.md) | `INT32` | -| `UINT64` | [UInt64](/sql-reference/data-types/int-uint.md) | `UINT64` | -| `INT64` | [Int64](/sql-reference/data-types/int-uint.md), [DateTime64](/sql-reference/data-types/datetime.md), [Decimal64](/sql-reference/data-types/decimal.md) | `INT64` | -| `FLOAT32` | [Float32](/sql-reference/data-types/float.md) | `FLOAT32` | -| `FLOAT64` | [Float64](/sql-reference/data-types/float.md) | `FLOAT64` | -| `TEXT, DATA` | [String](/sql-reference/data-types/string.md), [FixedString](/sql-reference/data-types/fixedstring.md) | `TEXT, DATA` | -| `union(T, Void), union(Void, T)` | [Nullable(T)](/sql-reference/data-types/date.md) | `union(T, Void), union(Void, T)` | -| `ENUM` | [Enum(8/16)](/sql-reference/data-types/enum.md) | `ENUM` | -| `LIST` | [Array](/sql-reference/data-types/array.md) | `LIST` | -| `STRUCT` | [Tuple](/sql-reference/data-types/tuple.md) | `STRUCT` | -| `UINT32` | [IPv4](/sql-reference/data-types/ipv4.md) | `UINT32` | -| `DATA` | [IPv6](/sql-reference/data-types/ipv6.md) | `DATA` | -| `DATA` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `DATA` | -| `DATA` | [Decimal128/Decimal256](/sql-reference/data-types/decimal.md) | `DATA` | -| `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | [Map](/sql-reference/data-types/map.md) | `STRUCT(entries LIST(STRUCT(key Key, value Value)))` | - -- 整数类型在输入和输出时可以相互转换。 -- 要在 CapnProto 格式中使用 `Enum`,请使用 [format_capn_proto_enum_comparising_mode](/operations/settings/settings-formats.md/#format_capn_proto_enum_comparising_mode) 设置。 -- 数组可以嵌套,并且其元素可以是 `Nullable` 类型。`Tuple` 和 `Map` 类型也可以嵌套。 - -## 示例用法 {#example-usage} - -### 插入和查询数据 {#inserting-and-selecting-data-capnproto} - -可以通过以下命令,将文件中的 CapnProto 数据插入到 ClickHouse 表中: - -```bash -$ cat capnproto_messages.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_schema = 'schema:Message' FORMAT CapnProto" -``` - -其中 `schema.capnp` 文件内容如下: - -```capnp -struct Message { - SearchPhrase @0 :Text; - c @1 :Uint64; -} -``` - -您可以通过以下命令,从 ClickHouse 表中查询数据,并将其以 `CapnProto` 格式保存到某个文件中: - -```bash -$ clickhouse-client --query = "SELECT * FROM test.hits FORMAT CapnProto SETTINGS format_schema = 'schema:Message'" -``` - - -### 使用自动生成的 schema {#using-autogenerated-capn-proto-schema} - -如果你的数据没有外部定义的 `CapnProto` schema,你仍然可以使用自动生成的 schema 以 `CapnProto` 格式输入/输出数据。 - -例如: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS format_capn_proto_use_autogenerated_schema=1 -``` - -在这种情况下,ClickHouse 会根据表结构使用函数 [structureToCapnProtoSchema](/sql-reference/functions/other-functions.md#structureToCapnProtoSchema) 自动生成 CapnProto schema,并使用该 schema 以 CapnProto 格式序列化数据。 - -你也可以读取使用自动生成 schema 的 CapnProto 文件(在这种情况下,文件必须使用相同的 schema 创建): - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_capn_proto_use_autogenerated_schema=1 FORMAT CapnProto" -``` - - -## 格式设置 {#format-settings} - -设置 [`format_capn_proto_use_autogenerated_schema`](../../operations/settings/settings-formats.md/#format_capn_proto_use_autogenerated_schema) 默认启用,仅在未设置 [`format_schema`](/interfaces/formats#formatschema) 时生效。 - -你也可以在输入/输出时通过设置 [`output_format_schema`](/operations/settings/formats#output_format_schema) 将自动生成的 schema 保存到文件中。 - -例如: - -```sql -SELECT * FROM test.hits -FORMAT CapnProto -SETTINGS - format_capn_proto_use_autogenerated_schema=1, - output_format_schema='path/to/schema/schema.capnp' -``` - -在这种情况下,自动生成的 `CapnProto` 模式将会保存在文件 `path/to/schema/schema.capnp` 中。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md deleted file mode 100644 index 7918570174a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparated.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: [] -description: 'CustomSeparated 格式的文档' -input_format: true -keywords: ['CustomSeparated'] -output_format: true -slug: /interfaces/formats/CustomSeparated -title: 'CustomSeparated' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -类似于 [Template](../Template/Template.md),但它会输出或读取所有列的名称和类型,并使用 [format_custom_escaping_rule](../../../operations/settings/settings-formats.md/#format_custom_escaping_rule) 设置中的转义规则,以及以下设置中的分隔符: - -- [format_custom_field_delimiter](/operations/settings/settings-formats.md/#format_custom_field_delimiter) -- [format_custom_row_before_delimiter](/operations/settings/settings-formats.md/#format_custom_row_before_delimiter) -- [format_custom_row_after_delimiter](/operations/settings/settings-formats.md/#format_custom_row_after_delimiter) -- [format_custom_row_between_delimiter](/operations/settings/settings-formats.md/#format_custom_row_between_delimiter) -- [format_custom_result_before_delimiter](/operations/settings/settings-formats.md/#format_custom_result_before_delimiter) -- [format_custom_result_after_delimiter](/operations/settings/settings-formats.md/#format_custom_result_after_delimiter) - -:::note -它不会使用格式字符串中指定的转义规则和分隔符。 -::: - -还有一种 [`CustomSeparatedIgnoreSpaces`](../CustomSeparated/CustomSeparatedIgnoreSpaces.md) 格式,与 [TemplateIgnoreSpaces](../Template//TemplateIgnoreSpaces.md) 类似。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用如下名为 `football.txt` 的 txt 文件: - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparated; -``` - - -### 读取数据 {#reading-data} - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -以 `CustomSeparated` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT CustomSeparated -``` - -输出将采用配置的自定义格式: - -```text -row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## 格式设置 {#format-settings} - -其他设置: - -| 设置 | 说明 | 默认值 | -|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|---------| -| [input_format_custom_detect_header](../../../operations/settings/settings-formats.md/#input_format_custom_detect_header) | 如果存在,则自动检测包含列名和类型的表头。 | `true` | -| [input_format_custom_skip_trailing_empty_lines](../../../operations/settings/settings-formats.md/#input_format_custom_skip_trailing_empty_lines) | 跳过文件末尾的尾随空行。 | `false` | -| [input_format_custom_allow_variable_number_of_columns](../../../operations/settings/settings-formats.md/#input_format_custom_allow_variable_number_of_columns) | 允许在 CustomSeparated 格式中使用可变数量的列,忽略多余列,并对缺失列使用默认值。 | `false` | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md deleted file mode 100644 index 5bd32a4d72c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpaces.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpaces 格式文档' -keywords: ['CustomSeparatedIgnoreSpaces'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpaces -title: 'CustomSeparatedIgnoreSpaces' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | | | - -## 说明 {#description} - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用下面这个名为 `football.txt` 的文本文件: - -```text -row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpaces; -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md deleted file mode 100644 index 9c93e28b8b4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNames.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpacesWithNames 格式文档' -keywords: ['CustomSeparatedIgnoreSpacesWithNames'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNames -title: 'CustomSeparatedIgnoreSpacesWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | | | - -## 描述 {#description} - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.txt` 的 txt 文件: - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -配置自定义分隔符相关设置: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNames; -``` - - -## 格式配置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md deleted file mode 100644 index 508bc4d51db..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedIgnoreSpacesWithNamesAndTypes.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: 'CustomSeparatedIgnoreSpacesWithNamesAndTypes 格式文档' -keywords: ['CustomSeparatedIgnoreSpacesWithNamesAndTypes'] -slug: /interfaces/formats/CustomSeparatedIgnoreSpacesWithNamesAndTypes -title: 'CustomSeparatedIgnoreSpacesWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | | | - -## 说明 {#description} - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.txt` 的 txt 文件: - -```text -row('date'; 'season'; 'home_team'; 'away_team'; 'home_team_goals'; 'away_team_goals'), row('Date'; 'Int16'; 'LowCardinality(String)'; 'LowCardinality(String)'; 'Int8'; 'Int8'), row('2022-04-30'; 2021; 'Sutton United'; 'Bradford City'; 1; 4), row( '2022-04-30'; 2021; 'Swindon Town'; 'Barrow'; 2; 1), row( '2022-04-30'; 2021; 'Tranmere Rovers'; 'Oldham Athletic'; 2; 0), row('2022-05-02'; 2021; 'Salford City'; 'Mansfield Town'; 2; 2), row('2022-05-02'; 2021; 'Port Vale'; 'Newport County'; 1; 2), row('2022-05-07'; 2021; 'Barrow'; 'Northampton Town'; 1; 3), row('2022-05-07'; 2021; 'Bradford City'; 'Carlisle United'; 2; 0), row('2022-05-07'; 2021; 'Bristol Rovers'; 'Scunthorpe United'; 7; 0), row('2022-05-07'; 2021; 'Exeter City'; 'Port Vale'; 0; 1), row('2022-05-07'; 2021; 'Harrogate Town A.F.C.'; 'Sutton United'; 0; 2), row('2022-05-07'; 2021; 'Hartlepool United'; 'Colchester United'; 0; 2), row('2022-05-07'; 2021; 'Leyton Orient'; 'Tranmere Rovers'; 0; 1), row('2022-05-07'; 2021; 'Mansfield Town'; 'Forest Green Rovers'; 2; 2), row('2022-05-07'; 2021; 'Newport County'; 'Rochdale'; 0; 2), row('2022-05-07'; 2021; 'Oldham Athletic'; 'Crawley Town'; 3; 3), row('2022-05-07'; 2021; 'Stevenage Borough'; 'Salford City'; 4; 2), row('2022-05-07'; 2021; 'Walsall'; 'Swindon Town'; 0; 3) -``` - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedIgnoreSpacesWithNamesAndTypes; -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md deleted file mode 100644 index 1075e6aa9d3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNames.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -alias: [] -description: 'CustomSeparatedWithNames 格式文档' -input_format: true -keywords: ['CustomSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNames -title: 'CustomSeparatedWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -同时会输出包含列名的表头行,其行为类似于 [TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md)。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.txt` 的文本文件: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNames; -``` - - -### 读取数据 {#reading-data} - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -使用 `CustomSeparatedWithNames` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNames -``` - -输出将使用已配置的自定义格式: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则会根据列名将输入数据中的列映射到表中的列; -如果将 [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则具有未知列名的列会被跳过。 -否则,第一行将被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md deleted file mode 100644 index 0ad6c5e5bc5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/CustomSeparated/CustomSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -alias: [] -description: 'CustomSeparatedWithNamesAndTypes 格式文档' -input_format: true -keywords: ['CustomSeparatedWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/CustomSeparatedWithNamesAndTypes -title: 'CustomSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -另外还会输出两行表头,包含列名和类型,与 [TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md) 类似。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用下面这个名为 `football.txt` 的文本文件: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.txt' FORMAT CustomSeparatedWithNamesAndTypes; -``` - - -### 读取数据 {#reading-data} - -配置自定义分隔符: - -```sql -SET format_custom_row_before_delimiter = 'row('; -SET format_custom_row_after_delimiter = ')'; -SET format_custom_field_delimiter = ';'; -SET format_custom_row_between_delimiter = ','; -SET format_custom_escaping_rule = 'Quoted'; -``` - -以 `CustomSeparatedWithNamesAndTypes` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT CustomSeparatedWithNamesAndTypes -``` - -输出将使用已配置的自定义格式: - -```text -row('date';'season';'home_team';'away_team';'home_team_goals';'away_team_goals'),row('Date';'Int16';'LowCardinality(String)';'LowCardinality(String)';'Int8';'Int8'),row('2022-04-30';2021;'Sutton United';'Bradford City';1;4),row('2022-04-30';2021;'Swindon Town';'Barrow';2;1),row('2022-04-30';2021;'Tranmere Rovers';'Oldham Athletic';2;0),row('2022-05-02';2021;'Port Vale';'Newport County';1;2),row('2022-05-02';2021;'Salford City';'Mansfield Town';2;2),row('2022-05-07';2021;'Barrow';'Northampton Town';1;3),row('2022-05-07';2021;'Bradford City';'Carlisle United';2;0),row('2022-05-07';2021;'Bristol Rovers';'Scunthorpe United';7;0),row('2022-05-07';2021;'Exeter City';'Port Vale';0;1),row('2022-05-07';2021;'Harrogate Town A.F.C.';'Sutton United';0;2),row('2022-05-07';2021;'Hartlepool United';'Colchester United';0;2),row('2022-05-07';2021;'Leyton Orient';'Tranmere Rovers';0;1),row('2022-05-07';2021;'Mansfield Town';'Forest Green Rovers';2;2),row('2022-05-07';2021;'Newport County';'Rochdale';0;2),row('2022-05-07';2021;'Oldham Athletic';'Crawley Town';3;3),row('2022-05-07';2021;'Stevenage Borough';'Salford City';4;2),row('2022-05-07';2021;'Walsall';'Swindon Town';0;3) -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则会按照列名将输入数据中的列映射到表中的列;如果将 [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则会跳过列名无法识别的列。 -否则,将跳过第一行。 -::: - -:::note -如果将 [`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) 设置为 `1`, -则会将输入数据中的类型与表中相应列的类型进行比较。否则,将跳过第二行。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md deleted file mode 100644 index 2fa4daa3aef..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/DWARF.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -alias: [] -description: 'DWARF 格式文档' -input_format: true -keywords: ['DWARF'] -output_format: false -slug: /interfaces/formats/DWARF -title: 'DWARF' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|---------|-------| -| ✔ | ✗ | | - -## 描述 {#description} - -`DWARF` 格式用于从 ELF 文件(可执行文件、库或目标文件)中解析 DWARF 调试符号。 -它与 `dwarfdump` 类似,但速度要快得多(数百 MB/s),并且支持 SQL。 -它会为 `.debug_info` 区段中的每个调试信息条目(DIE,Debug Information Entry)生成一行记录, -并包括 DWARF 编码中用于在树形结构中终止子节点列表的“空”条目。 - -:::info -`.debug_info` 由若干 *unit* 组成,对应于编译单元(compilation unit): - -- 每个 unit 是一个由若干 *DIE* 构成的树,根节点是一个 `compile_unit` DIE。 -- 每个 DIE 具有一个 *tag* 和一个属性(*attributes*)列表。 -- 每个 attribute 具有 *name* 和 *value*(以及一个 *form*,用于指定该值的编码方式)。 - -这些 DIE 表示源代码中的实体,其 *tag* 指明了它代表的实体类型。例如,有: - -- 函数(tag = `subprogram`) -- 类/结构体/枚举(`class_type`/`structure_type`/`enumeration_type`) -- 变量(`variable`) -- 函数参数(`formal_parameter`)。 - -树形结构反映了对应的源代码结构。例如,一个 `class_type` DIE 可以包含若干 `subprogram` DIE,用于表示该类的方法。 -::: - -`DWARF` 格式输出以下列: - -- `offset` - DIE 在 `.debug_info` 区段中的位置 -- `size` - 编码后的 DIE 所占的字节数(包括属性) -- `tag` - DIE 的类型;省略常规前缀 "DW_TAG_" -- `unit_name` - 包含此 DIE 的编译单元名称 -- `unit_offset` - 包含此 DIE 的编译单元在 `.debug_info` 区段中的位置 -- `ancestor_tags` - 当前 DIE 在树中的各级祖先节点的 tag 数组,从最内层到最外层依次排列 -- `ancestor_offsets` - 各级祖先节点的 offset,与 `ancestor_tags` 一一对应 -- 为了方便使用,从属性数组中复制出的若干常见属性: - - `name` - - `linkage_name` - 经重整(mangled)的完全限定名称;通常只有函数才有(但并非所有函数都有) - - `decl_file` - 声明该实体的源代码文件名 - - `decl_line` - 声明该实体所在的源代码行号 -- 使用并行数组描述属性: - - `attr_name` - 属性名称;省略常规前缀 "DW_AT_" - - `attr_form` - 属性的编码和解释方式;省略常规前缀 `DW_FORM_` - - `attr_int` - 属性的整数值;如果该属性没有数值型值则为 0 - - `attr_str` - 属性的字符串值;如果该属性没有字符串值则为空 - -## 使用示例 {#example-usage} - -`DWARF` 格式可用于查找函数定义数量最多的编译单元(包括模板实例化以及来自所包含头文件的函数): - -```sql title="Query" -SELECT - unit_name, - count() AS c -FROM file('programs/clickhouse', DWARF) -WHERE tag = 'subprogram' AND NOT has(attr_name, 'declaration') -GROUP BY unit_name -ORDER BY c DESC -LIMIT 3 -``` - -```text title="Response" -┌─unit_name──────────────────────────────────────────────────┬─────c─┐ -│ ./src/Core/Settings.cpp │ 28939 │ -│ ./src/AggregateFunctions/AggregateFunctionSumMap.cpp │ 23327 │ -│ ./src/AggregateFunctions/AggregateFunctionUniqCombined.cpp │ 22649 │ -└────────────────────────────────────────────────────────────┴───────┘ - -返回了 3 行。耗时:1.487 秒。处理了 1.3976 亿行,1.12 GB(9397 万行/秒,752.77 MB/秒)。 -峰值内存使用:271.92 MiB。 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md deleted file mode 100644 index 271946999f6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Form.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -alias: [] -description: 'Form 格式文档' -input_format: true -keywords: ['Form'] -output_format: false -slug: /interfaces/formats/Form -title: 'Form' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✗ | | - -## 描述 {#description} - -`Form` 格式可用于读取一条采用 `application/x-www-form-urlencoded` 格式编码的记录, -其中数据的格式为 `key1=value1&key2=value2`。 - -## 示例用法 {#example-usage} - -假设在 `user_files` 路径下放置了一个名为 `data.tmp`、包含一些 URL 编码数据的文件: - -```text title="data.tmp" -t_page=116&c.e=ls7xfkpm&c.tti.m=raf&rt.start=navigation&rt.bmr=390%2C11%2C10 -``` - -```sql title="Query" -SELECT * FROM file(data.tmp, Form) FORMAT vertical; -``` - -```response title="Response" -第 1 行: -────── -t_page: 116 -c.e: ls7xfkpm -c.tti.m: raf -rt.start: navigation -rt.bmr: 390,11,10 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md deleted file mode 100644 index 2f4eae3a785..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Hash.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'Hash 格式的文档' -input_format: false -keywords: ['hash', 'format'] -output_format: true -slug: /interfaces/formats/Hash -title: 'Hash' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - -## 描述 {#description} - -`Hash` 输出格式会为结果中的所有列和行计算一个单个哈希值。 -这在数据传输成为瓶颈的场景中,对于计算结果的“指纹”非常有用。 - -## 示例用法 {#example-usage} - -### 读取数据 {#reading-data} - -假设有一张名为 `football` 的表,包含如下数据: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -以 `Hash` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT Hash -``` - -此查询将处理数据,但不会产生任何输出。 - -```response -df2ec2f0669b000edff6adee264e7d68 - -返回 1 行。用时:0.154 秒。 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md deleted file mode 100644 index 1a3f37b4e9a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/HiveText.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: 'HiveText 格式文档' -keywords: ['HiveText'] -slug: /interfaces/formats/HiveText -title: 'HiveText' -doc_type: 'reference' ---- - -## 说明 {#description} - -## 使用示例 {#example-usage} - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md deleted file mode 100644 index c4b4f553c76..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSON.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -alias: [] -description: 'JSON 格式文档' -input_format: true -keywords: ['JSON'] -output_format: true -slug: /interfaces/formats/JSON -title: 'JSON' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## Description {#description} - -`JSON` 格式以 JSON 格式读取和输出数据。 - -`JSON` 格式返回如下内容: - -| Parameter | Description | -|------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `meta` | 列名和类型。 | -| `data` | 数据表。 | -| `rows` | 输出的总行数。 | -| `rows_before_limit_at_least` | 在没有使用 LIMIT 时可能存在的行数的较低估计值。仅在查询包含 LIMIT 时输出。该估计值是基于在到达 LIMIT 变换之前在查询管道中处理的数据块计算得出的,但之后可能会被 LIMIT 变换丢弃。如果数据块在查询管道中甚至没有到达 LIMIT 变换阶段,则不会参与估计。 | -| `statistics` | 诸如 `elapsed`、`rows_read`、`bytes_read` 等统计信息。 | -| `totals` | 总计值(在使用 WITH TOTALS 时)。 | -| `extremes` | 极值(当 extremes 被设置为 1 时)。 | - -`JSON` 类型与 JavaScript 兼容。为确保这一点,会对某些字符进行额外转义: - -- 斜杠 `/` 被转义为 `\/`。 -- 会导致部分浏览器出错的替代换行符 `U+2028` 和 `U+2029` 被转义为 `\uXXXX`。 -- ASCII 控制字符会被转义:退格、换页、换行、回车和水平制表符分别被替换为 `\b`、`\f`、`\n`、`\r`、`\t`,其余 0x00–0x1F 范围内的字节则使用 `\uXXXX` 序列替换。 -- 无效的 UTF-8 序列会被替换为替代字符 `�`,从而保证输出文本仅由有效的 UTF-8 序列组成。 - -为了与 JavaScript 兼容,Int64 和 UInt64 整数默认会用双引号括起来。 -要去掉引号,可以将配置参数 [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) 设置为 `0`。 - -ClickHouse 支持 [NULL](/sql-reference/syntax.md),在 JSON 输出中显示为 `null`。要在输出中启用 `+nan`、`-nan`、`+inf`、`-inf` 值,请将 [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) 设置为 `1`。 - -## 示例用法 {#example-usage} - -示例: - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase WITH TOTALS ORDER BY c DESC LIMIT 5 FORMAT JSON -``` - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "hello", - "arr": [0,1] - }, - { - "num": 43, - "str": "hello", - "arr": [0,1,2] - }, - { - "num": 44, - "str": "hello", - "arr": [0,1,2,3] - } - ], - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.001137687, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - - -## 格式设置 {#format-settings} - -对于 JSON 输入格式,如果将设置项 [`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) 设为 `1`, -则会将输入数据中元数据里的类型与表中对应列的类型进行比较。 - -## 另请参阅 {#see-also} - -- [JSONEachRow](/interfaces/formats/JSONEachRow) 格式 -- [output_format_json_array_of_rows](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) 设置 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md deleted file mode 100644 index 8f20428bc04..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsObject.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -alias: [] -description: 'JSONAsObject 格式文档' -input_format: true -keywords: ['JSONAsObject'] -output_format: false -slug: /interfaces/formats/JSONAsObject -title: 'JSONAsObject' -doc_type: 'reference' ---- - -## 描述 {#description} - -在此格式中,单个 JSON 对象会被解释为一个 [JSON](/sql-reference/data-types/newjson.md) 值。如果输入包含多个 JSON 对象(以逗号分隔),则它们会被解释为多行数据。如果输入数据被方括号包裹,则会被解释为一个 JSON 数组。 - -此格式只能用于解析到仅包含一个 [JSON](/sql-reference/data-types/newjson.md) 类型字段的表中。其余列必须设置为 [`DEFAULT`](/sql-reference/statements/create/table.md/#default) 或 [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view)。 - -## 使用示例 {#example-usage} - -### 基本示例 {#basic-example} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_object FORMAT JSONEachRow; -``` - -```response title="Response" -{"json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -{"json":{}} -{"json":{"any json stucture":"1"}} -``` - - -### JSON 对象数组 {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field JSON) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsObject [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; -SELECT * FROM json_square_brackets FORMAT JSONEachRow; -``` - -```response title="Response" -{"field":{"id":"1","name":"name1"}} -{"field":{"id":"2","name":"name2"}} -``` - - -### 具有默认值的列 {#columns-with-default-values} - -```sql title="Query" -CREATE TABLE json_as_object (json JSON, time DateTime MATERIALIZED now()) ENGINE = Memory; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"foo":{"bar":{"x":"y"},"baz":1}}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {}; -INSERT INTO json_as_object (json) FORMAT JSONAsObject {"any json stucture":1} -SELECT time, json FROM json_as_object FORMAT JSONEachRow -``` - -```response title="Response" -{"time":"2024-09-16 12:18:10","json":{}} -{"time":"2024-09-16 12:18:13","json":{"any json stucture":"1"}} -{"time":"2024-09-16 12:18:08","json":{"foo":{"bar":{"x":"y"},"baz":"1"}}} -``` - - -## 格式设定 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md deleted file mode 100644 index 81fd2640fdb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONAsString.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'JSONAsString 格式文档' -input_format: true -keywords: ['JSONAsString'] -output_format: false -slug: /interfaces/formats/JSONAsString -title: 'JSONAsString' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|------|-------|------| -| ✔ | ✗ | | - -## 描述 {#description} - -在这种格式中,单个 JSON 对象会被解释为一个单独的值。 -如果输入包含多个 JSON 对象(用逗号分隔),它们会被解释为多行记录。 -如果输入数据被方括号包裹,则会被解释为一个 JSON 对象数组。 - -:::note -此格式只能用于解析仅包含一个 [String](/sql-reference/data-types/string.md) 类型字段的表。 -其余列必须设置为 [`DEFAULT`](/sql-reference/statements/create/table.md/#default) 或 [`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view), -或者直接省略。 -::: - -将整个 JSON 对象序列化为一个 String 值之后,就可以使用 [JSON 函数](/sql-reference/functions/json-functions.md) 来处理它。 - -## 使用示例 {#example-usage} - -### 基本示例 {#basic-example} - -```sql title="Query" -DROP TABLE IF EXISTS json_as_string; -CREATE TABLE json_as_string (json String) ENGINE = Memory; -INSERT INTO json_as_string (json) FORMAT JSONAsString {"foo":{"bar":{"x":"y"},"baz":1}},{},{"any json stucture":1} -SELECT * FROM json_as_string; -``` - -```response title="Response" -┌─json──────────────────────────────┐ -│ {"foo":{"bar":{"x":"y"},"baz":1}} │ -│ {} │ -│ {"any json stucture":1} │ -└───────────────────────────────────┘ -``` - - -### JSON 对象数组 {#an-array-of-json-objects} - -```sql title="Query" -CREATE TABLE json_square_brackets (field String) ENGINE = Memory; -INSERT INTO json_square_brackets FORMAT JSONAsString [{"id": 1, "name": "name1"}, {"id": 2, "name": "name2"}]; - -SELECT * FROM json_square_brackets; -``` - -```response title="Response" -┌─field──────────────────────┐ -│ {"id": 1, "name": "name1"} │ -│ {"id": 2, "name": "name2"} │ -└────────────────────────────┘ -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md deleted file mode 100644 index 04e7424f95d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumns.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -alias: [] -description: 'JSONColumns 格式说明' -input_format: true -keywords: ['JSONColumns'] -output_format: true -slug: /interfaces/formats/JSONColumns -title: 'JSONColumns' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -:::tip -JSONColumns* 格式的输出首先输出 ClickHouse 字段名,随后给出该字段在表中每一行的内容;直观上看,相当于将数据向左旋转了 90 度。 -::: - -在这种格式下,所有数据都表示为一个 JSON 对象。 - -:::note -`JSONColumns` 格式会将所有数据缓冲在内存中,并一次性作为单个数据块输出,因此可能会导致较高的内存消耗。 -::: - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用包含以下数据的 JSON 文件,并将其命名为 `football.json`: - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONColumns; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONColumns` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONColumns -``` - -输出将为 JSON 格式: - -```json -{ - "date": ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - "season": [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - "home_team": ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - "away_team": ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - "home_team_goals": [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - "away_team_goals": [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -} -``` - - -## 格式设置 {#format-settings} - -在导入过程中,如果将 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则名称未知的列会被跳过。 -在数据块中不存在的列将用默认值填充(此处可以使用 [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) 设置)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md deleted file mode 100644 index 6473a277058..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONColumnsWithMetadata.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -alias: [] -description: 'JSONColumnsWithMetadata 格式说明文档' -input_format: true -keywords: ['JSONColumnsWithMetadata'] -output_format: true -slug: /interfaces/formats/JSONColumnsWithMetadata -title: 'JSONColumnsWithMetadata' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [`JSONColumns`](./JSONColumns.md) 格式的区别在于,它还包含一些元数据和统计信息(类似于 [`JSON`](./JSON.md) 格式)。 - -:::note -`JSONColumnsWithMetadata` 格式会将所有数据缓存在内存中,然后以单个数据块输出,因此可能会导致较高的内存占用。 -::: - -## 使用示例 {#example-usage} - -示例: - -```json -{ - "meta": - [ - { - "name": "num", - "type": "Int32" - }, - { - "name": "str", - "type": "String" - }, - - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - { - "num": [42, 43, 44], - "str": ["hello", "hello", "hello"], - "arr": [[0,1], [0,1,2], [0,1,2,3]] - }, - - "rows": 3, - - "rows_before_limit_at_least": 3, - - "statistics": - { - "elapsed": 0.000272376, - "rows_read": 3, - "bytes_read": 24 - } -} -``` - -对于 `JSONColumnsWithMetadata` 输入格式,如果将 [`input_format_json_validate_types_from_metadata`](/operations/settings/settings-formats.md/#input_format_json_validate_types_from_metadata) 设置为 `1`,则会将输入数据中元数据中的类型与表中对应列的类型进行比较。 - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md deleted file mode 100644 index 3e5c48ded13..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompact.md +++ /dev/null @@ -1,160 +0,0 @@ ---- -alias: [] -description: 'JSONCompact 格式文档' -input_format: true -keywords: ['JSONCompact'] -output_format: true -slug: /interfaces/formats/JSONCompact -title: 'JSONCompact' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [JSON](./JSON.md) 的唯一区别在于,数据行以数组形式输出,而不是以对象形式输出。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用包含以下数据的 JSON 文件,将其命名为 `football.json`: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ] -} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompact; -``` - - -### 读取数据 {#reading-data} - -以 `JSONCompact` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompact -``` - -输出将为 JSON 格式: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4], - ["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1], - ["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0], - ["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2], - ["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2], - ["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3], - ["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0], - ["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0], - ["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1], - ["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2], - ["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2], - ["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1], - ["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2], - ["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2], - ["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3], - ["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2], - ["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.223690876, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md deleted file mode 100644 index 474439024ec..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactColumns.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -alias: [] -description: 'JSONCompactColumns 格式文档' -input_format: true -keywords: ['JSONCompactColumns'] -output_format: true -slug: /interfaces/formats/JSONCompactColumns -title: 'JSONCompactColumns' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -在这种格式下,所有数据都表示为单个 JSON 数组。 - -:::note -`JSONCompactColumns` 输出格式会将所有数据缓冲至内存中,然后以单个数据块的形式输出,这可能导致较高的内存占用。 -::: - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个名为 `football.json` 的 JSON 文件,包含如下数据: - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactColumns; -``` - - -### 读取数据 {#reading-data} - -以 `JSONCompactColumns` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactColumns -``` - -输出将为 JSON 格式: - -```json -[ - ["2022-04-30", "2022-04-30", "2022-04-30", "2022-05-02", "2022-05-02", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07", "2022-05-07"], - [2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021, 2021], - ["Sutton United", "Swindon Town", "Tranmere Rovers", "Port Vale", "Salford City", "Barrow", "Bradford City", "Bristol Rovers", "Exeter City", "Harrogate Town A.F.C.", "Hartlepool United", "Leyton Orient", "Mansfield Town", "Newport County", "Oldham Athletic", "Stevenage Borough", "Walsall"], - ["Bradford City", "Barrow", "Oldham Athletic", "Newport County", "Mansfield Town", "Northampton Town", "Carlisle United", "Scunthorpe United", "Port Vale", "Sutton United", "Colchester United", "Tranmere Rovers", "Forest Green Rovers", "Rochdale", "Crawley Town", "Salford City", "Swindon Town"], - [1, 2, 2, 1, 2, 1, 2, 7, 0, 0, 0, 0, 2, 0, 3, 4, 0], - [4, 1, 0, 2, 2, 3, 0, 0, 1, 2, 2, 1, 2, 2, 3, 2, 3] -] -``` - -在该数据块中不存在的列会被填充为默认值(此处可以使用 [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) 设置) - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md deleted file mode 100644 index bf7aa239952..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRow 格式文档' -input_format: true -keywords: ['JSONCompactEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRow -title: 'JSONCompactEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [`JSONEachRow`](./JSONEachRow.md) 的唯一区别在于,数据行以数组而非对象的形式输出。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用包含以下数据的 JSON 文件,并将其命名为 `football.json`: - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRow; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONCompactEachRow` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRow -``` - -输出将为 JSON 格式: - -```json -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md deleted file mode 100644 index 752c07792cf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNames.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithNames 格式文档' -input_format: true -keywords: ['JSONCompactEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNames -title: 'JSONCompactEachRowWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [`JSONCompactEachRow`](./JSONCompactEachRow.md) 格式不同,它还会输出包含列名的表头行,类似于 [`TabSeparatedWithNames`](../TabSeparated/TabSeparatedWithNames.md) 格式。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含以下数据的 JSON 文件,命名为 `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNames; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONCompactEachRowWithNames` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNames -``` - -输出将为 JSON 格式: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 1, -则会按名称将输入数据中的列映射到表中的列;如果将 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 1,则会跳过名称未知的列。 -否则,将跳过第一行。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md deleted file mode 100644 index 9358bbaeff2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithNamesAndTypes 格式文档' -input_format: true -keywords: ['JSONCompactEachRowWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithNamesAndTypes -title: 'JSONCompactEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与[`JSONCompactEachRow`](./JSONCompactEachRow.md)格式的区别在于,它还会输出两行表头,包含列名和列类型,类似于[TabSeparatedWithNamesAndTypes](../TabSeparated/TabSeparatedWithNamesAndTypes.md)格式。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含以下数据的 JSON 文件,文件名为 `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactEachRowWithNamesAndTypes; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONCompactEachRowWithNamesAndTypes` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactEachRowWithNamesAndTypes -``` - -输出将为 JSON 格式: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", 2021, "Sutton United", "Bradford City", 1, 4] -["2022-04-30", 2021, "Swindon Town", "Barrow", 2, 1] -["2022-04-30", 2021, "Tranmere Rovers", "Oldham Athletic", 2, 0] -["2022-05-02", 2021, "Port Vale", "Newport County", 1, 2] -["2022-05-02", 2021, "Salford City", "Mansfield Town", 2, 2] -["2022-05-07", 2021, "Barrow", "Northampton Town", 1, 3] -["2022-05-07", 2021, "Bradford City", "Carlisle United", 2, 0] -["2022-05-07", 2021, "Bristol Rovers", "Scunthorpe United", 7, 0] -["2022-05-07", 2021, "Exeter City", "Port Vale", 0, 1] -["2022-05-07", 2021, "Harrogate Town A.F.C.", "Sutton United", 0, 2] -["2022-05-07", 2021, "Hartlepool United", "Colchester United", 0, 2] -["2022-05-07", 2021, "Leyton Orient", "Tranmere Rovers", 0, 1] -["2022-05-07", 2021, "Mansfield Town", "Forest Green Rovers", 2, 2] -["2022-05-07", 2021, "Newport County", "Rochdale", 0, 2] -["2022-05-07", 2021, "Oldham Athletic", "Crawley Town", 3, 3] -["2022-05-07", 2021, "Stevenage Borough", "Salford City", 4, 2] -["2022-05-07", 2021, "Walsall", "Swindon Town", 0, 3] -``` - - -## 格式设置 {#format-settings} - -:::note -如果将[`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header)设置为 `1`, -则会按名称将输入数据中的列映射到表中的列。如果将[input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields)设置为 `1`,则具有未知名称的列会被跳过。 -否则,第一行将被跳过。 -如果将[`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header)设置为 `1`, -则会将输入数据中的类型与表中对应列的类型进行比较。否则,第二行将被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md deleted file mode 100644 index c0223a66b92..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactEachRowWithProgress.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -alias: [] -description: 'JSONCompactEachRowWithProgress 格式文档' -input_format: false -keywords: ['JSONCompactEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactEachRowWithProgress -title: 'JSONCompactEachRowWithProgress' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - -## 描述 {#description} - -此格式将 `JSONCompactEachRow` 的紧凑按行输出与流式进度信息结合在一起。 -它将元数据、单独的行、进度更新、汇总以及异常分别作为独立的 JSON 对象输出。各个值以其原生类型进行表示。 - -主要特性: - -- 首先输出包含列名和列类型的元数据 -- 每一行都是一个独立的 JSON 对象,使用 `"row"` 键包含一个值数组 -- 在查询执行期间包含进度更新(作为 `{"progress":...}` 对象) -- 支持 totals(汇总)和 extremes(极值) -- 保留值的原生类型(数字为数字,字符串为字符串) - -## 使用示例 {#example-usage} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":[[-8], 46848.5225, ["2064-06-11 14:00:36.578","b06f4fa1-22ff-f84f-a1b7-a5807d983ae6"]]} -{"row":[[-76], -85331.598, ["2038-06-16 04:10:27.271","2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c"]]} -{"row":[[-32], -31470.8994, ["2027-07-18 16:58:34.654","1cdbae4c-ceb2-1337-b954-b175f5efbef8"]]} -{"row":[[-116], 32104.097, ["1979-04-27 21:51:53.321","66903704-3c83-8f8a-648a-da4ac1ffa9fc"]]} -{"row":[[], 2427.6614, ["1980-04-24 11:30:35.487","fee19be8-0f46-149b-ed98-43e7455ce2b2"]]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"335771"}} -{"rows_before_limit_at_least":5} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md deleted file mode 100644 index 9ae9808f14d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStrings.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStrings 格式文档' -input_format: false -keywords: ['JSONCompactStrings'] -output_format: true -slug: /interfaces/formats/JSONCompactStrings -title: 'JSONCompactStrings' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - -## 描述 {#description} - -`JSONCompactStrings` 格式与 [JSONStrings](./JSONStrings.md) 的唯一区别是数据行以数组形式输出,而不是对象。 - -## 使用示例 {#example-usage} - -### 读取数据 {#reading-data} - -以 `JSONCompactStrings` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStrings -``` - -输出将为 JSON 格式: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - ["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"], - ["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"], - ["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"], - ["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"], - ["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"], - ["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"], - ["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"], - ["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"], - ["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"], - ["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"], - ["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"], - ["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"], - ["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"], - ["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"], - ["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"], - ["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"], - ["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.112012501, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md deleted file mode 100644 index fc0295544bb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRow 格式文档' -input_format: true -keywords: ['JSONCompactStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRow -title: 'JSONCompactStringsEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [`JSONCompactEachRow`](./JSONCompactEachRow.md) 的唯一区别在于,数据字段以字符串形式输出,而不是以带类型的 JSON 值输出。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个名为 `football.json`、包含以下数据的 JSON 文件: - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRow; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONCompactStringsEachRow` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRow -``` - -输出将为 JSON 格式: - -```json -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md deleted file mode 100644 index aa1f53f303f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNames.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRowWithNames 格式文档' -input_format: true -keywords: ['JSONCompactStringsEachRowWithNames'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithNames -title: 'JSONCompactStringsEachRowWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [`JSONCompactEachRow`](./JSONCompactEachRow.md) 格式的区别在于,它还会输出带列名的表头行,类似于 [TabSeparatedWithNames](../TabSeparated/TabSeparatedWithNames.md) 格式。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含以下数据的 JSON 文件,并将其命名为 `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNames; -``` - - -### 读取数据 {#reading-data} - -以 `JSONCompactStringsEachRowWithNames` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNames -``` - -输出将为 JSON 格式: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## 格式设置 {#format-settings} - -:::note -如果将设置项 [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设为 `1`, -则输入数据中的列会按名称映射到表中的列;如果将设置项 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设为 `1`,则名称未知的列会被跳过。 -否则,第一行将被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md deleted file mode 100644 index 81a23f850c9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithNamesAndTypes.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: 'JSONCompactStringsEachRowWithNamesAndTypes 格式文档' -keywords: ['JSONCompactStringsEachRowWithNamesAndTypes'] -slug: /interfaces/formats/JSONCompactStringsEachRowWithNamesAndTypes -title: 'JSONCompactStringsEachRowWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 `JSONCompactEachRow` 格式的区别在于,它还会输出两行表头,包含列名和列类型,类似于 [TabSeparatedWithNamesAndTypes](/interfaces/formats/TabSeparatedRawWithNamesAndTypes)。 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含如下数据的 JSON 文件,并命名为 `football.json`: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONCompactStringsEachRowWithNamesAndTypes; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONCompactStringsEachRowWithNamesAndTypes` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONCompactStringsEachRowWithNamesAndTypes -``` - -输出结果为 JSON 格式: - -```json -["date", "season", "home_team", "away_team", "home_team_goals", "away_team_goals"] -["Date", "Int16", "LowCardinality(String)", "LowCardinality(String)", "Int8", "Int8"] -["2022-04-30", "2021", "Sutton United", "Bradford City", "1", "4"] -["2022-04-30", "2021", "Swindon Town", "Barrow", "2", "1"] -["2022-04-30", "2021", "Tranmere Rovers", "Oldham Athletic", "2", "0"] -["2022-05-02", "2021", "Port Vale", "Newport County", "1", "2"] -["2022-05-02", "2021", "Salford City", "Mansfield Town", "2", "2"] -["2022-05-07", "2021", "Barrow", "Northampton Town", "1", "3"] -["2022-05-07", "2021", "Bradford City", "Carlisle United", "2", "0"] -["2022-05-07", "2021", "Bristol Rovers", "Scunthorpe United", "7", "0"] -["2022-05-07", "2021", "Exeter City", "Port Vale", "0", "1"] -["2022-05-07", "2021", "Harrogate Town A.F.C.", "Sutton United", "0", "2"] -["2022-05-07", "2021", "Hartlepool United", "Colchester United", "0", "2"] -["2022-05-07", "2021", "Leyton Orient", "Tranmere Rovers", "0", "1"] -["2022-05-07", "2021", "Mansfield Town", "Forest Green Rovers", "2", "2"] -["2022-05-07", "2021", "Newport County", "Rochdale", "0", "2"] -["2022-05-07", "2021", "Oldham Athletic", "Crawley Town", "3", "3"] -["2022-05-07", "2021", "Stevenage Borough", "Salford City", "4", "2"] -["2022-05-07", "2021", "Walsall", "Swindon Town", "0", "3"] -``` - - -## 格式设置 {#format-settings} - -:::note -如果将 [input_format_with_names_use_header](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 1, -则输入数据中的列会按名称映射到表中的列;如果将 [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 1,则名称未知的列会被跳过。 -否则,第一行将被跳过。 -::: - -:::note -如果将 [input_format_with_types_use_header](/operations/settings/settings-formats.md/#input_format_with_types_use_header) 设置为 1, -则输入数据中的类型会与表中对应列的类型进行比较。否则,第二行将被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md deleted file mode 100644 index 1a2024c994d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONCompactStringsEachRowWithProgress.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'JSONCompactStringsEachRowWithProgress 格式说明文档' -input_format: true -keywords: ['JSONCompactStringsEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONCompactStringsEachRowWithProgress -title: 'JSONCompactStringsEachRowWithProgress' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|---------|--------| -| ✗ | ✔ | | - -## Description {#description} - -类似于 [`JSONCompactEachRowWithProgress`](/interfaces/formats/JSONCompactEachRowWithProgress),但所有值都会被转换为字符串。 -当需要对所有数据类型进行统一的字符串表示时,此格式非常有用。 - -关键特性: - -- 结构与 `JSONCompactEachRowWithProgress` 相同 -- 所有值都表示为字符串(数字、数组等均以加引号的字符串形式输出) -- 包含进度更新、汇总以及异常处理 -- 适用于偏好或要求基于字符串数据的客户端 - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -```sql title="Query" -SELECT * -FROM generateRandom('a Array(Int8), d Decimal32(4), c Tuple(DateTime64(3), UUID)', 1, 10, 2) -LIMIT 5 -FORMAT JSONCompactStringsEachRowWithProgress -``` - -```response title="Response" -{"meta":[{"name":"a","type":"Array(Int8)"},{"name":"d","type":"Decimal(9, 4)"},{"name":"c","type":"Tuple(DateTime64(3), UUID)"}]} -{"row":["[-8]", "46848.5225", "('2064-06-11 14:00:36.578','b06f4fa1-22ff-f84f-a1b7-a5807d983ae6')"]} -{"row":["[-76]", "-85331.598", "('2038-06-16 04:10:27.271','2bb0de60-3a2c-ffc0-d7a7-a5c88ed8177c')"]} -{"row":["[-32]", "-31470.8994", "('2027-07-18 16:58:34.654','1cdbae4c-ceb2-1337-b954-b175f5efbef8')"]} -{"row":["[-116]", "32104.097", "('1979-04-27 21:51:53.321','66903704-3c83-8f8a-648a-da4ac1ffa9fc')"]} -{"row":["[]", "2427.6614", "('1980-04-24 11:30:35.487','fee19be8-0f46-149b-ed98-43e7455ce2b2')"]} -{"progress":{"read_rows":"5","read_bytes":"184","total_rows_to_read":"5","elapsed_ns":"191151"}} -{"rows_before_limit_at_least":5} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md deleted file mode 100644 index 0295e1511f6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRow.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -alias: ['JSONLines', 'NDJSON'] -description: 'JSONEachRow 格式文档' -keywords: ['JSONEachRow'] -slug: /interfaces/formats/JSONEachRow -title: 'JSONEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-----------------------| -| ✔ | ✔ | `JSONLines`, `NDJSON` | - -## 描述 {#description} - -在这种格式下,ClickHouse 会将每一行输出为一个独立的 JSON 对象,并以换行符分隔各对象。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用名为 `football.json` 的 JSON 文件,其内容如下: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONEachRow; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONEachRow` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONEachRow -``` - -输出将是 JSON 格式: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -若将设置 [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设为 1,则会跳过导入名称未知的数据列。 - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md deleted file mode 100644 index 3ffbc0cf0ba..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONEachRowWithProgress.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -alias: [] -description: 'JSONEachRowWithProgress 格式文档' -input_format: false -keywords: ['JSONEachRowWithProgress'] -output_format: true -slug: /interfaces/formats/JSONEachRowWithProgress -title: 'JSONEachRowWithProgress' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - -## 描述 {#description} - -与 [`JSONEachRow`](./JSONEachRow.md)/[`JSONStringsEachRow`](./JSONStringsEachRow.md) 的区别在于,ClickHouse 还会以 JSON 值的形式输出进度信息。 - -## 使用示例 {#example-usage} - -```json -{"row":{"num":42,"str":"hello","arr":[0,1]}} -{"row":{"num":43,"str":"hello","arr":[0,1,2]}} -{"row":{"num":44,"str":"hello","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md deleted file mode 100644 index 5cbe513eb4c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONLines.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -alias: ['JSONEachRow', 'JSONLines', 'NDJSON', 'JSONL'] -description: 'JSONLines 格式文档' -keywords: ['JSONLines'] -slug: /interfaces/formats/JSONLines -title: 'JSONLines' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|----------------------------------------------| -| ✔ | ✔ | `JSONEachRow`, `JSONLines`, `NDJSON`, `JSONL` | - -## 描述 {#description} - -在这种格式中,ClickHouse 会将每一行输出为一个独立的 JSON 对象,各对象之间以换行符分隔。 - -这种格式也被称为 `JSONEachRow`、`NDJSON`(Newline Delimited JSON)或 `JSONL`(`JSONLines`)。这些名称都是同一格式的别名,可以互换使用。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个名为 `football.json`、包含以下数据的 JSON 文件: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONLines; -``` - - -### 读取数据 {#reading-data} - -以 `JSONLines` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONLines -``` - -输出结果将为 JSON 格式: - -```json -{"date":"2022-04-30","season":2021,"home_team":"Sutton United","away_team":"Bradford City","home_team_goals":1,"away_team_goals":4} -{"date":"2022-04-30","season":2021,"home_team":"Swindon Town","away_team":"Barrow","home_team_goals":2,"away_team_goals":1} -{"date":"2022-04-30","season":2021,"home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-02","season":2021,"home_team":"Port Vale","away_team":"Newport County","home_team_goals":1,"away_team_goals":2} -{"date":"2022-05-02","season":2021,"home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Barrow","away_team":"Northampton Town","home_team_goals":1,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":2,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":7,"away_team_goals":0} -{"date":"2022-05-07","season":2021,"home_team":"Exeter City","away_team":"Port Vale","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":0,"away_team_goals":1} -{"date":"2022-05-07","season":2021,"home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":2,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Newport County","away_team":"Rochdale","home_team_goals":0,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":3,"away_team_goals":3} -{"date":"2022-05-07","season":2021,"home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":4,"away_team_goals":2} -{"date":"2022-05-07","season":2021,"home_team":"Walsall","away_team":"Swindon Town","home_team_goals":0,"away_team_goals":3} -``` - -如果将 [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 1,则导入数据时会跳过名称未知的列。 - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md deleted file mode 100644 index 30049a00104..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONObjectEachRow.md +++ /dev/null @@ -1,245 +0,0 @@ ---- -alias: [] -description: 'JSONObjectEachRow 格式文档' -input_format: true -keywords: ['JSONObjectEachRow'] -output_format: true -slug: /interfaces/formats/JSONObjectEachRow -title: 'JSONObjectEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -在这种格式中,所有数据都表示为单个 JSON 对象,其中每一行对应该对象的一个独立字段,类似于 [`JSONEachRow`](./JSONEachRow.md) 格式。 - - - -## 示例用法 {#example-usage} - -### 基本示例 {#basic-example} - -假设有如下 JSON: - -```json -{ - "row_1": {"num": 42, "str": "hello", "arr": [0,1]}, - "row_2": {"num": 43, "str": "hello", "arr": [0,1,2]}, - "row_3": {"num": 44, "str": "hello", "arr": [0,1,2,3]} -} -``` - -要将对象名用作列值,可以使用特殊设置 [`format_json_object_each_row_column_for_object_name`](/operations/settings/settings-formats.md/#format_json_object_each_row_column_for_object_name)。 -该设置的值应设为某一列的名称,该列将在生成的对象中作为每一行的 JSON 键名。 - -#### 输出 {#output} - -假设我们有一张名为 `test` 的表,其中包含两列: - -```text -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -我们将它输出为 `JSONObjectEachRow` 格式,并使用 `format_json_object_each_row_column_for_object_name` 设置: - -```sql title="Query" -SELECT * FROM test SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```json title="Response" -{ - "first_obj": {"number": 1}, - "second_obj": {"number": 2}, - "third_obj": {"number": 3} -} -``` - -#### 输入 {#input} - -假设我们将上一个示例的输出保存到名为 `data.json` 的文件中: - -```sql title="Query" -SELECT * FROM file('data.json', JSONObjectEachRow, 'object_name String, number UInt64') SETTINGS format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─object_name─┬─number─┐ -│ first_obj │ 1 │ -│ second_obj │ 2 │ -│ third_obj │ 3 │ -└─────────────┴────────┘ -``` - -同样适用于模式推断: - -```sql title="Query" -DESCRIBE file('data.json', JSONObjectEachRow) SETTING format_json_object_each_row_column_for_object_name='object_name' -``` - -```response title="Response" -┌─name────────┬─type────────────┐ -│ object_name │ String │ -│ number │ Nullable(Int64) │ -└─────────────┴─────────────────┘ -``` - -### 写入数据 {#json-inserting-data} - -```sql title="Query" -INSERT INTO UserActivity FORMAT JSONEachRow {"PageViews":5, "UserID":"4324182021466249494", "Duration":146,"Sign":-1} {"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -ClickHouse 允许: - -* 对象中的键值对可以以任意顺序出现。 -* 省略某些值。 - -ClickHouse 会忽略元素之间的空格以及对象后面的逗号。可以在同一行中传递所有对象,无需使用换行符将它们分隔开。 - -#### 省略值的处理 {#omitted-values-processing} - -ClickHouse 会使用相应[数据类型](/sql-reference/data-types/index.md)的默认值来替换被省略的值。 - -如果指定了 `DEFAULT expr`,ClickHouse 会根据 [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields) 设置采用不同的替换规则。 - -考虑下列表: - -```sql title="Query" -CREATE TABLE IF NOT EXISTS example_table -( - x UInt32, - a DEFAULT x * 2 -) ENGINE = Memory; -``` - -* 如果 `input_format_defaults_for_omitted_fields = 0`,则 `x` 和 `a` 的默认值为 `0`(即 `UInt32` 数据类型的默认值)。 -* 如果 `input_format_defaults_for_omitted_fields = 1`,则 `x` 的默认值为 `0`,但 `a` 的默认值为 `x * 2`。 - -:::note -在使用 `input_format_defaults_for_omitted_fields = 1` 插入数据时,相比使用 `input_format_defaults_for_omitted_fields = 0` 插入,ClickHouse 会消耗更多计算资源。 -::: - -### 查询数据 {#json-selecting-data} - -以 `UserActivity` 表为例: - - -```response -┌──────────────UserID─┬─PageViews─┬─Duration─┬─Sign─┐ -│ 4324182021466249494 │ 5 │ 146 │ -1 │ -│ 4324182021466249494 │ 6 │ 185 │ 1 │ -└─────────────────────┴───────────┴──────────┴──────┘ -``` - -查询 `SELECT * FROM UserActivity FORMAT JSONEachRow` 将返回: - -```response -{"UserID":"4324182021466249494","PageViews":5,"Duration":146,"Sign":-1} -{"UserID":"4324182021466249494","PageViews":6,"Duration":185,"Sign":1} -``` - -与 [JSON](/interfaces/formats/JSON) 格式不同,这里不会对无效的 UTF-8 序列进行替换。值的转义方式与 `JSON` 相同。 - -:::info -字符串中可以输出任意字节序列。如果确定表中的数据可以在不丢失任何信息的情况下被格式化为 JSON,请使用 [`JSONEachRow`](./JSONEachRow.md) 格式。 -::: - -### 嵌套结构的使用 {#jsoneachrow-nested} - -如果有一张带有 [`Nested`](/sql-reference/data-types/nested-data-structures/index.md) 数据类型列的表,就可以插入具有相同结构的 JSON 数据。通过 [input_format_import_nested_json](/operations/settings/settings-formats.md/#input_format_import_nested_json) 设置启用此功能。 - -例如,考虑以下数据表: - -```sql -CREATE TABLE json_each_row_nested (n Nested (s String, i Int32) ) ENGINE = Memory -``` - -正如您在 `Nested` 数据类型的说明中所看到的,ClickHouse 将嵌套结构的每个组件视为单独的一列(在我们的表中为 `n.s` 和 `n.i`)。您可以通过以下方式插入数据: - -```sql -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n.s": ["abc", "def"], "n.i": [1, 23]} -``` - -要将数据作为层级 JSON 对象插入,请设置 [`input_format_import_nested_json=1`](/operations/settings/settings-formats.md/#input_format_import_nested_json)。 - -```json -{ - "n": { - "s": ["abc", "def"], - "i": [1, 23] - } -} -``` - -如果未进行此设置,ClickHouse 会抛出异常。 - -```sql title="Query" -SELECT name, value FROM system.settings WHERE name = 'input_format_import_nested_json' -``` - -```response title="Response" -┌─name────────────────────────────┬─value─┐ -│ input_format_import_nested_json │ 0 │ -└─────────────────────────────────┴───────┘ -``` - -```sql title="Query" -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -``` - -```response title="Response" -代码:117. DB::Exception:解析 JSONEachRow 格式时发现未知字段:n:(第 1 行) -``` - -```sql title="Query" -SET input_format_import_nested_json=1 -INSERT INTO json_each_row_nested FORMAT JSONEachRow {"n": {"s": ["abc", "def"], "i": [1, 23]}} -SELECT * FROM json_each_row_nested -``` - -```response title="Response" -┌─n.s───────────┬─n.i────┐ -│ ['abc','def'] │ [1,23] │ -└───────────────┴────────┘ -``` - - -## 格式设置 {#format-settings} - - - -| 配置 | 说明 | 默认 | 注意事项 | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | 将嵌套 JSON 数据映射为嵌套表(适用于 JSONEachRow 格式)。 | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | 允许在 JSON 输入格式中将布尔值解析为数值。 | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | 允许在 JSON 输入格式中将布尔值作为字符串进行解析。 | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | 允许在 JSON 输入格式中按字符串方式解析数值。 | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | 允许在 JSON 输入格式中将 JSON 数组解析为字符串值。 | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | 允许在 JSON 输入格式中将 JSON 对象作为字符串进行解析。 | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | 将 Named Tuple 类型的列解析为 JSON 对象。 | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | 在进行模式推断时,尝试从字符串字段中推断数值。 | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | 在模式推断时尝试将 JSON 对象推断为命名元组。 | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | 在对 JSON 输入格式进行模式推断时,对于仅包含 Null 或空对象/数组的键,将其类型设为 String。 | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | 在解析 named tuple 时,为 JSON 对象中缺失的字段填充默认值。 | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | 对命名元组的 JSON 对象忽略未知键。 | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | 允许 JSONCompact/JSONCompactEachRow 格式使用可变数量的列,忽略多余的列,并对缺失的列使用默认值。 | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | 如果 JSON 字符串包含错误的转义序列,则抛出异常。若禁用,则错误的转义序列将原样保留在数据中。 | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | 将 JSON 输入中的空字段按默认值处理。 | `false`。 | 对于复杂的默认值表达式,也必须启用 [`input_format_defaults_for_omitted_fields`](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)。 | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | 控制在 JSON 输出格式中是否为 64 位整数加引号。 | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | 控制 JSON 输出格式中 64 位浮点数是否以带引号的形式输出。 | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | 在 JSON 输出格式中允许输出 '+nan'、'-nan'、'+inf'、'-inf'。 | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | 控制在 JSON 输出格式中是否为小数加引号。 | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | 控制是否在 JSON 输出格式的字符串中对正斜杠进行转义。 | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | 将 NamedTuple 列序列化为 JSON 对象。 | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | 输出一个包含所有行的 JSON 数组,采用 JSONEachRow(Compact) 格式。 | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | 在 JSON 输出格式中启用对 UTF-8 序列的校验(注意,这不会影响 JSON/JSONCompact/JSONColumnsWithMetadata 这些格式,它们始终会进行 UTF-8 校验)。 | `false` | | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md deleted file mode 100644 index 9126d28c42e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStrings.md +++ /dev/null @@ -1,398 +0,0 @@ ---- -alias: [] -description: 'JSONStrings 格式文档' -input_format: true -keywords: ['JSONStrings'] -output_format: true -slug: /interfaces/formats/JSONStrings -title: 'JSONStrings' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -与 [JSON](./JSON.md) 格式的唯一不同之处在于,数据字段以字符串形式输出,而不是输出为带类型的 JSON 值。 - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用包含以下数据的 JSON 文件,并将其命名为 `football.json`: - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ] -} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStrings; -``` - - -### 读取数据 {#reading-data} - -使用 `JSONStrings` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONStrings -``` - -输出为 JSON 格式: - - -```json -{ - "meta": - [ - { - "name": "date", - "type": "Date" - }, - { - "name": "season", - "type": "Int16" - }, - { - "name": "home_team", - "type": "LowCardinality(String)" - }, - { - "name": "away_team", - "type": "LowCardinality(String)" - }, - { - "name": "home_team_goals", - "type": "Int8" - }, - { - "name": "away_team_goals", - "type": "Int8" - } - ], - - "data": - [ - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": "1", - "away_team_goals": "4" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": "2", - "away_team_goals": "1" - }, - { - "date": "2022-04-30", - "season": "2021", - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": "1", - "away_team_goals": "2" - }, - { - "date": "2022-05-02", - "season": "2021", - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": "1", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": "2", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": "7", - "away_team_goals": "0" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": "0", - "away_team_goals": "1" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": "2", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": "0", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": "3", - "away_team_goals": "3" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": "4", - "away_team_goals": "2" - }, - { - "date": "2022-05-07", - "season": "2021", - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": "0", - "away_team_goals": "3" - } - ], - - "rows": 17, - - "statistics": - { - "elapsed": 0.173464376, - "rows_read": 0, - "bytes_read": 0 - } -} -``` - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md deleted file mode 100644 index a1d26c09dbb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRow.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -alias: [] -description: 'JSONStringsEachRow 格式文档' -input_format: false -keywords: ['JSONStringsEachRow'] -output_format: true -slug: /interfaces/formats/JSONStringsEachRow -title: 'JSONStringsEachRow' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -与 [`JSONEachRow`](./JSONEachRow.md) 的唯一区别在于,数据字段会以字符串形式输出,而不是输出为带类型的 JSON 值。 - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含以下数据的 JSON 文件,并将其命名为 `football.json`: - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT JSONStringsEachRow; -``` - -### 读取数据 {#reading-data} - -以 `JSONStringsEachRow` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT JSONStringsEachRow -``` - -输出将为 JSON 格式: - - -```json -{"date":"2022-04-30","season":"2021","home_team":"Sutton United","away_team":"Bradford City","home_team_goals":"1","away_team_goals":"4"} -{"date":"2022-04-30","season":"2021","home_team":"Swindon Town","away_team":"Barrow","home_team_goals":"2","away_team_goals":"1"} -{"date":"2022-04-30","season":"2021","home_team":"Tranmere Rovers","away_team":"Oldham Athletic","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-02","season":"2021","home_team":"Port Vale","away_team":"Newport County","home_team_goals":"1","away_team_goals":"2"} -{"date":"2022-05-02","season":"2021","home_team":"Salford City","away_team":"Mansfield Town","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Barrow","away_team":"Northampton Town","home_team_goals":"1","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Bradford City","away_team":"Carlisle United","home_team_goals":"2","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Bristol Rovers","away_team":"Scunthorpe United","home_team_goals":"7","away_team_goals":"0"} -{"date":"2022-05-07","season":"2021","home_team":"Exeter City","away_team":"Port Vale","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Harrogate Town A.F.C.","away_team":"Sutton United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Hartlepool United","away_team":"Colchester United","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Leyton Orient","away_team":"Tranmere Rovers","home_team_goals":"0","away_team_goals":"1"} -{"date":"2022-05-07","season":"2021","home_team":"Mansfield Town","away_team":"Forest Green Rovers","home_team_goals":"2","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Newport County","away_team":"Rochdale","home_team_goals":"0","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Oldham Athletic","away_team":"Crawley Town","home_team_goals":"3","away_team_goals":"3"} -{"date":"2022-05-07","season":"2021","home_team":"Stevenage Borough","away_team":"Salford City","home_team_goals":"4","away_team_goals":"2"} -{"date":"2022-05-07","season":"2021","home_team":"Walsall","away_team":"Swindon Town","home_team_goals":"0","away_team_goals":"3"} -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md deleted file mode 100644 index c895d31e5f1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/JSONStringsEachRowWithProgress.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: 'JSONStringsEachRowWithProgress 格式文档' -keywords: ['JSONStringsEachRowWithProgress'] -slug: /interfaces/formats/JSONStringsEachRowWithProgress -title: 'JSONStringsEachRowWithProgress' -doc_type: 'reference' ---- - - - -## 描述 {#description} - -与 `JSONEachRow`/`JSONStringsEachRow` 不同,ClickHouse 还会以 JSON 值的形式返回进度信息。 - - - -## 使用示例 {#example-usage} - -```json -{"row":{"num":42,"str":"hello","arr":[0,1]}} -{"row":{"num":43,"str":"hello","arr":[0,1,2]}} -{"row":{"num":44,"str":"hello","arr":[0,1,2,3]}} -{"progress":{"read_rows":"3","read_bytes":"24","written_rows":"0","written_bytes":"0","total_rows_to_read":"3"}} -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md deleted file mode 100644 index 73a766115e5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/PrettyJSONEachRow.md +++ /dev/null @@ -1,332 +0,0 @@ ---- -alias: ['PrettyJSONLines', 'PrettyNDJSON'] -description: 'PrettyJSONLines 格式文档' -input_format: false -keywords: ['PrettyJSONEachRow', 'PrettyJSONLines', 'PrettyNDJSON'] -output_format: true -slug: /interfaces/formats/PrettyJSONEachRow -title: 'PrettyJSONEachRow' -doc_type: 'guide' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-----------------------------------| -| ✗ | ✔ | `PrettyJSONLines`, `PrettyNDJSON` | - - - -## 描述 {#description} - -与 [JSONEachRow](./JSONEachRow.md) 的唯一区别在于,JSON 采用带换行符和四个空格缩进的美化格式。 - - - -## 使用示例 {#example-usage} -### 插入数据 {#inserting-data} - -使用包含以下数据的 JSON 文件,并将其命名为 `football.json`: - - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.json' FORMAT PrettyJSONEachRow; -``` - -### 读取数据 {#reading-data} - -以 `PrettyJSONEachRow` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT PrettyJSONEachRow -``` - -输出结果将为 JSON 格式: - - -```json -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Sutton United", - "away_team": "Bradford City", - "home_team_goals": 1, - "away_team_goals": 4 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Swindon Town", - "away_team": "Barrow", - "home_team_goals": 2, - "away_team_goals": 1 -} -{ - "date": "2022-04-30", - "season": 2021, - "home_team": "Tranmere Rovers", - "away_team": "Oldham Athletic", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Port Vale", - "away_team": "Newport County", - "home_team_goals": 1, - "away_team_goals": 2 -} -{ - "date": "2022-05-02", - "season": 2021, - "home_team": "Salford City", - "away_team": "Mansfield Town", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Barrow", - "away_team": "Northampton Town", - "home_team_goals": 1, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bradford City", - "away_team": "Carlisle United", - "home_team_goals": 2, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Bristol Rovers", - "away_team": "Scunthorpe United", - "home_team_goals": 7, - "away_team_goals": 0 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Exeter City", - "away_team": "Port Vale", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Harrogate Town A.F.C.", - "away_team": "Sutton United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Hartlepool United", - "away_team": "Colchester United", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Leyton Orient", - "away_team": "Tranmere Rovers", - "home_team_goals": 0, - "away_team_goals": 1 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Mansfield Town", - "away_team": "Forest Green Rovers", - "home_team_goals": 2, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Newport County", - "away_team": "Rochdale", - "home_team_goals": 0, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Oldham Athletic", - "away_team": "Crawley Town", - "home_team_goals": 3, - "away_team_goals": 3 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Stevenage Borough", - "away_team": "Salford City", - "home_team_goals": 4, - "away_team_goals": 2 -} -{ - "date": "2022-05-07", - "season": 2021, - "home_team": "Walsall", - "away_team": "Swindon Town", - "home_team_goals": 0, - "away_team_goals": 3 -} -``` - - - - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md deleted file mode 100644 index a667870985e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/JSON/format-settings.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'JSON 格式的格式设置列表' -keywords: ['Format Settings', 'JSON'] -slug: /interfaces/formats/JSON/format-settings -title: 'JSON 的格式设置' -doc_type: 'reference' ---- - -本页列出了所有 JSON 格式通用的格式设置。 - -{/* TODO - 在下方自动生成表格 */ } - - -| 设置 | 说明 | 默认 | 注意 | -| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| [`input_format_import_nested_json`](/operations/settings/settings-formats.md/#input_format_import_nested_json) | 将嵌套 JSON 数据映射为嵌套表(适用于 JSONEachRow 格式)。 | `false` | | -| [`input_format_json_read_bools_as_numbers`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_numbers) | 允许在 JSON 输入格式中将布尔类型解析为数值。 | `true` | | -| [`input_format_json_read_bools_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_bools_as_strings) | 允许在 JSON 输入格式中将布尔值按字符串进行解析。 | `true` | | -| [`input_format_json_read_numbers_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_numbers_as_strings) | 允许在 JSON 输入格式中将数字解析为字符串类型。 | `true` | | -| [`input_format_json_read_arrays_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_arrays_as_strings) | 允许在 JSON 输入格式中将 JSON 数组解析为字符串。 | `true` | | -| [`input_format_json_read_objects_as_strings`](/operations/settings/settings-formats.md/#input_format_json_read_objects_as_strings) | 允许在 JSON 输入格式中将 JSON 对象解析为字符串。 | `true` | | -| [`input_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#input_format_json_named_tuples_as_objects) | 将命名元组类型的列解析为 JSON 对象。 | `true` | | -| [`input_format_json_try_infer_numbers_from_strings`](/operations/settings/settings-formats.md/#input_format_json_try_infer_numbers_from_strings) | 在推断模式时,尝试将字符串字段解析为数值。 | `false` | | -| [`input_format_json_try_infer_named_tuples_from_objects`](/operations/settings/settings-formats.md/#input_format_json_try_infer_named_tuples_from_objects) | 在模式推断时,尝试从 JSON 对象推断命名元组类型。 | `true` | | -| [`input_format_json_infer_incomplete_types_as_strings`](/operations/settings/settings-formats.md/#input_format_json_infer_incomplete_types_as_strings) | 在对 JSON 输入格式进行模式推断时,应对值仅为 Null 或空对象/数组的键使用 String 类型。 | `true` | | -| [`input_format_json_defaults_for_missing_elements_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_defaults_for_missing_elements_in_named_tuple) | 在解析命名元组(named tuple)的 JSON 对象时,为缺失的字段插入默认值。 | `true` | | -| [`input_format_json_ignore_unknown_keys_in_named_tuple`](/operations/settings/settings-formats.md/#input_format_json_ignore_unknown_keys_in_named_tuple) | 对命名元组的 JSON 对象忽略未知键。 | `false` | | -| [`input_format_json_compact_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_json_compact_allow_variable_number_of_columns) | 允许在 JSONCompact/JSONCompactEachRow 格式中使用可变数量的列,忽略多余的列,并对缺失的列使用默认值。 | `false` | | -| [`input_format_json_throw_on_bad_escape_sequence`](/operations/settings/settings-formats.md/#input_format_json_throw_on_bad_escape_sequence) | 启用时,如果 JSON 字符串包含错误的转义序列,将抛出异常。禁用时,错误的转义序列将在数据中原样保留。 | `true` | | -| [`input_format_json_empty_as_default`](/operations/settings/settings-formats.md/#input_format_json_empty_as_default) | 将 JSON 输入中的空字段按默认值处理。 | `false` | 对于复杂的默认表达式,还必须启用 [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)。 | -| [`output_format_json_quote_64bit_integers`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_integers) | 控制在 JSON 输出格式中对 64 位整数是否加引号。 | `true` | | -| [`output_format_json_quote_64bit_floats`](/operations/settings/settings-formats.md/#output_format_json_quote_64bit_floats) | 控制在 JSON 输出格式中对 64 位浮点数加引号的方式。 | `false` | | -| [`output_format_json_quote_denormals`](/operations/settings/settings-formats.md/#output_format_json_quote_denormals) | 在 JSON 输出格式中启用 '+nan'、'-nan'、'+inf'、'-inf' 的输出。 | `false` | | -| [`output_format_json_quote_decimals`](/operations/settings/settings-formats.md/#output_format_json_quote_decimals) | 控制在 JSON 输出格式中是否为小数加引号。 | `false` | | -| [`output_format_json_escape_forward_slashes`](/operations/settings/settings-formats.md/#output_format_json_escape_forward_slashes) | 控制在 JSON 输出格式中是否对字符串中的正斜杠进行转义。 | `true` | | -| [`output_format_json_named_tuples_as_objects`](/operations/settings/settings-formats.md/#output_format_json_named_tuples_as_objects) | 将命名元组列序列化为 JSON 对象。 | `true` | | -| [`output_format_json_array_of_rows`](/operations/settings/settings-formats.md/#output_format_json_array_of_rows) | 以 JSONEachRow(Compact) 格式输出由所有行组成的 JSON 数组。 | `false` | | -| [`output_format_json_validate_utf8`](/operations/settings/settings-formats.md/#output_format_json_validate_utf8) | 启用对 JSON 输出格式中 UTF-8 序列的校验 | `false` | 请注意,这不会影响 JSON/JSONCompact/JSONColumnsWithMetadata 格式,它们始终会验证 UTF-8 编码。 | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md deleted file mode 100644 index a3265b31253..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsString.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -alias: [] -description: 'LineAsString 格式文档' -input_format: true -keywords: ['LineAsString'] -output_format: true -slug: /interfaces/formats/LineAsString -title: 'LineAsString' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -`LineAsString` 格式将输入数据的每一行都视为一个单独的字符串值。 -此格式只能用于解析仅包含一个 [String](/sql-reference/data-types/string.md) 类型字段的表。 -其余列必须设置为 [`DEFAULT`](/sql-reference/statements/create/table.md/#default)、[`MATERIALIZED`](/sql-reference/statements/create/view#materialized-view) 或省略。 - - - -## 使用示例 {#example-usage} - -```sql title="Query" -DROP TABLE IF EXISTS line_as_string; -CREATE TABLE line_as_string (field String) ENGINE = Memory; -INSERT INTO line_as_string FORMAT LineAsString "我喜欢苹果", "我喜欢香蕉", "我喜欢橙子"; -SELECT * FROM line_as_string; -``` - -```text title="Response" -┌─field─────────────────────────────────────────────┐ -│ "我喜欢苹果", "我喜欢香蕉", "我喜欢橙子"; │ -└───────────────────────────────────────────────────┘ -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md deleted file mode 100644 index 5ccf0840814..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNames.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -alias: [] -description: 'LineAsStringWithNames 格式文档' -input_format: true -keywords: ['LineAsStringWithNames'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNames -title: 'LineAsStringWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -`LineAsStringWithNames` 格式与 [`LineAsString`](./LineAsString.md) 格式类似,但会额外输出包含列名的表头行。 - - - -## 使用示例 {#example-usage} - -```sql title="Query" -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNames; -``` - -```response title="Response" -name value -John 30 -Jane 25 -Peter 35 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md deleted file mode 100644 index 5722541644b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/LineAsString/LineAsStringWithNamesAndTypes.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -alias: [] -description: 'LineAsStringWithNamesAndTypes 格式文档' -input_format: false -keywords: ['LineAsStringWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/LineAsStringWithNamesAndTypes -title: 'LineAsStringWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -`LineAsStringWithNames` 格式类似于 [`LineAsString`](./LineAsString.md) 格式, -但会打印两行表头:第一行是列名,第二行是类型。 - - - -## 使用示例 {#example-usage} - -```sql -CREATE TABLE example ( - name String, - value Int32 -) -ENGINE = Memory; - -INSERT INTO example VALUES ('John', 30), ('Jane', 25), ('Peter', 35); - -SELECT * FROM example FORMAT LineAsStringWithNamesAndTypes; -``` - -```response title="Response" -name value -String Int32 -John 30 -Jane 25 -Peter 35 -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md deleted file mode 100644 index d3046fbcf24..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Markdown.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: ['MD'] -description: 'Markdown 格式文档的说明' -keywords: ['Markdown'] -slug: /interfaces/formats/Markdown -title: 'Markdown' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | `MD` | - - - -## 描述 {#description} - -你可以使用 [Markdown](https://en.wikipedia.org/wiki/Markdown) 格式导出结果,生成可以直接粘贴到 `.md` 文件中的输出: - -Markdown 表格会自动生成,并且可以在支持 Markdown 的平台(例如 GitHub)上使用。此格式仅用于输出。 - - - -## 示例用法 {#example-usage} - -```sql -SELECT - number, - number * 2 -FROM numbers(5) -FORMAT Markdown -``` - -```results -| number | multiply(number, 2) | -|-:|-:| -| 0 | 0 | -| 1 | 2 | -| 2 | 4 | -| 3 | 6 | -| 4 | 8 | -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md deleted file mode 100644 index 434c4901b6f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MsgPack.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -alias: [] -description: 'MsgPack 格式文档' -input_format: true -keywords: ['MsgPack'] -output_format: true -slug: /interfaces/formats/MsgPack -title: 'MsgPack' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -ClickHouse 支持读写 [MessagePack](https://msgpack.org/) 数据文件。 - - - -## 数据类型对应关系 {#data-types-matching} - -| MessagePack 数据类型(`INSERT`) | ClickHouse 数据类型 | MessagePack 数据类型(`SELECT`) | -|--------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|----------------------------------| -| `uint N`, `positive fixint` | [`UIntN`](/sql-reference/data-types/int-uint.md) | `uint N` | -| `int N`, `negative fixint` | [`IntN`](/sql-reference/data-types/int-uint.md) | `int N` | -| `bool` | [`UInt8`](/sql-reference/data-types/int-uint.md) | `uint 8` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`String`](/sql-reference/data-types/string.md) | `bin 8`, `bin 16`, `bin 32` | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [`FixedString`](/sql-reference/data-types/fixedstring.md) | `bin 8`, `bin 16`, `bin 32` | -| `float 32` | [`Float32`](/sql-reference/data-types/float.md) | `float 32` | -| `float 64` | [`Float64`](/sql-reference/data-types/float.md) | `float 64` | -| `uint 16` | [`Date`](/sql-reference/data-types/date.md) | `uint 16` | -| `int 32` | [`Date32`](/sql-reference/data-types/date32.md) | `int 32` | -| `uint 32` | [`DateTime`](/sql-reference/data-types/datetime.md) | `uint 32` | -| `uint 64` | [`DateTime64`](/sql-reference/data-types/datetime.md) | `uint 64` | -| `fixarray`, `array 16`, `array 32` | [`Array`](/sql-reference/data-types/array.md)/[`Tuple`](/sql-reference/data-types/tuple.md) | `fixarray`, `array 16`, `array 32` | -| `fixmap`, `map 16`, `map 32` | [`Map`](/sql-reference/data-types/map.md) | `fixmap`, `map 16`, `map 32` | -| `uint 32` | [`IPv4`](/sql-reference/data-types/ipv4.md) | `uint 32` | -| `bin 8` | [`String`](/sql-reference/data-types/string.md) | `bin 8` | -| `int 8` | [`Enum8`](/sql-reference/data-types/enum.md) | `int 8` | -| `bin 8` | [`(U)Int128`/`(U)Int256`](/sql-reference/data-types/int-uint.md) | `bin 8` | -| `int 32` | [`Decimal32`](/sql-reference/data-types/decimal.md) | `int 32` | -| `int 64` | [`Decimal64`](/sql-reference/data-types/decimal.md) | `int 64` | -| `bin 8` | [`Decimal128`/`Decimal256`](/sql-reference/data-types/decimal.md) | `bin 8 ` | - - - -## 示例用法 {#example-usage} - -写入“.msgpk”文件: - -```sql -$ clickhouse-client --query="CREATE TABLE msgpack (array Array(UInt8)) ENGINE = Memory;" -$ clickhouse-client --query="INSERT INTO msgpack VALUES ([0, 1, 2, 3, 42, 253, 254, 255]), ([255, 254, 253, 42, 3, 2, 1, 0])"; -$ clickhouse-client --query="SELECT * FROM msgpack FORMAT MsgPack" > tmp_msgpack.msgpk; -``` - - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | -|--------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|---------| -| [`input_format_msgpack_number_of_columns`](/operations/settings/settings-formats.md/#input_format_msgpack_number_of_columns) | 插入的 MsgPack 数据中的列数。用于根据数据自动推断表结构。 | `0` | -| [`output_format_msgpack_uuid_representation`](/operations/settings/settings-formats.md/#output_format_msgpack_uuid_representation) | UUID 在 MsgPack 格式中的输出表示方式。 | `EXT` | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md deleted file mode 100644 index 82c99c80270..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLDump.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -alias: [] -description: 'MySQLDump 格式文档' -input_format: true -keywords: ['MySQLDump'] -output_format: false -slug: /interfaces/formats/MySQLDump -title: 'MySQLDump' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|---------|-------| -| ✔ | ✗ | | - - - -## 描述 {#description} - -ClickHouse 支持读取 MySQL 的 [转储文件(dump)](https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html)。 - -它会从转储文件中属于某个单表的 `INSERT` 语句中读取所有数据。 -如果存在多个表,默认从第一个表中读取数据。 - -:::note -此格式支持表结构推断:如果转储文件中包含该指定表的 `CREATE` 语句,则从中推断表结构;否则从 `INSERT` 语句中的数据推断表结构。 -::: - - - -## 示例用法 {#example-usage} - -假设有如下 SQL 转储文件: - -```sql title="dump.sql" -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test` ( - `x` int DEFAULT NULL, - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test` VALUES (1,NULL),(2,NULL),(3,NULL),(3,NULL),(4,NULL),(5,NULL),(6,7); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test 3` ( - `y` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test 3` VALUES (1); -/*!40101 SET @saved_cs_client = @@character_set_client */; -/*!50503 SET character_set_client = utf8mb4 */; -CREATE TABLE `test2` ( - `x` int DEFAULT NULL -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci; -/*!40101 SET character_set_client = @saved_cs_client */; -INSERT INTO `test2` VALUES (1),(2),(3); -``` - -我们可以执行以下查询: - -```sql title="Query" -DESCRIBE TABLE file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─name─┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ x │ Nullable(Int32) │ │ │ │ │ │ -└──────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql title="Query" -SELECT * -FROM file(dump.sql, MySQLDump) -SETTINGS input_format_mysql_dump_table_name = 'test2' -``` - -```response title="Response" -┌─x─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - - -## 格式设置 {#format-settings} - -可以使用 [`input_format_mysql_dump_table_name`](/operations/settings/settings-formats.md/#input_format_mysql_dump_table_name) 设置来指定要读取数据的表名。 -如果将 `input_format_mysql_dump_map_columns` 设置为 `1`,并且转储中包含该表的 `CREATE` 查询,或者在 `INSERT` 查询中包含指定的列名,则输入数据中的列会按名称映射到该表中的列。 -如果将 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,具有未知名称的列将会被忽略。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md deleted file mode 100644 index 9928fb238aa..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/MySQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'MySQLWire 格式文档' -keywords: ['MySQLWire'] -slug: /interfaces/formats/MySQLWire -title: 'MySQLWire' -doc_type: 'reference' ---- - - - -## 说明 {#description} - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md deleted file mode 100644 index 8106fc89979..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Native.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -alias: [] -description: 'Native 格式文档' -input_format: true -keywords: ['Native'] -output_format: true -slug: /interfaces/formats/Native -title: 'Native' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -`Native` 格式是 ClickHouse 中最高效的格式,因为它是真正意义上的“列式”格式, -不会将列转换为行。 - -在这种格式下,数据以二进制格式按[块](/development/architecture#block)进行读写。 -对于每个块,会依次记录行数、列数、列名和类型,以及该块中列的数据部分。 - -这是在原生接口中用于服务器之间交互、使用命令行客户端以及 C++ 客户端时所使用的格式。 - -:::tip -你可以使用这种格式快速生成只能由 ClickHouse 数据库管理系统(DBMS)读取的转储文件。 -自己直接使用这种格式进行操作可能并不实用。 -::: - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md deleted file mode 100644 index 5f6a48db91d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Npy.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -alias: [] -description: 'Npy 格式文档' -input_format: true -keywords: ['Npy'] -output_format: true -slug: /interfaces/formats/Npy -title: 'Npy' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -`Npy` 格式用于将 `.npy` 文件中的 NumPy 数组加载到 ClickHouse 中。 -NumPy 文件格式是一种用于高效存储数值数据数组的二进制格式。 -在导入过程中,ClickHouse 将最外层维度视为由单列表组成的行数组。 - -下表列出了受支持的 Npy 数据类型及其在 ClickHouse 中对应的类型: - - - -## 数据类型对应关系 {#data_types-matching} - -| Npy 数据类型(`INSERT`) | ClickHouse 数据类型 | Npy 数据类型(`SELECT`) | -|--------------------------|-----------------------------------------------------------------|--------------------------| -| `i1` | [Int8](/sql-reference/data-types/int-uint.md) | `i1` | -| `i2` | [Int16](/sql-reference/data-types/int-uint.md) | `i2` | -| `i4` | [Int32](/sql-reference/data-types/int-uint.md) | `i4` | -| `i8` | [Int64](/sql-reference/data-types/int-uint.md) | `i8` | -| `u1`, `b1` | [UInt8](/sql-reference/data-types/int-uint.md) | `u1` | -| `u2` | [UInt16](/sql-reference/data-types/int-uint.md) | `u2` | -| `u4` | [UInt32](/sql-reference/data-types/int-uint.md) | `u4` | -| `u8` | [UInt64](/sql-reference/data-types/int-uint.md) | `u8` | -| `f2`, `f4` | [Float32](/sql-reference/data-types/float.md) | `f4` | -| `f8` | [Float64](/sql-reference/data-types/float.md) | `f8` | -| `S`, `U` | [String](/sql-reference/data-types/string.md) | `S` | -| | [FixedString](/sql-reference/data-types/fixedstring.md) | `S` | - - - -## 示例用法 {#example-usage} - -### 使用 Python 将数组保存为 .npy 格式 {#saving-an-array-in-npy-format-using-python} - -```Python -import numpy as np -arr = np.array([[[1],[2],[3]],[[4],[5],[6]]]) -np.save('example_array.npy', arr) -``` - -### 在 ClickHouse 中读取 NumPy 文件 {#reading-a-numpy-file-in-clickhouse} - -```sql title="Query" -SELECT * -FROM file('example_array.npy', Npy) -``` - -```response title="Response" -┌─array─────────┐ -│ [[1],[2],[3]] │ -│ [[4],[5],[6]] │ -└───────────────┘ -``` - -### 选择数据 {#selecting-data} - -可以使用 `clickhouse-client` 运行以下命令,将 ClickHouse 表中的数据查询出来并保存为 Npy 格式的文件: - -```bash -$ clickhouse-client --query="SELECT {column} FROM {some_table} FORMAT Npy" > {filename.npy} -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md deleted file mode 100644 index ff1f9c6cf9f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Null.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -alias: [] -description: 'Null 格式文档' -input_format: false -keywords: ['Null', 'format'] -output_format: true -slug: /interfaces/formats/Null -title: 'Null' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -在 `Null` 格式下,不会输出任何内容。 -乍一看可能有些奇怪,但需要注意的是,尽管没有任何输出,查询仍然会被处理, -并且在使用命令行客户端时,数据依然会被传输到客户端。 - -:::tip -`Null` 格式可用于性能测试。 -::: - - - -## 示例用法 {#example-usage} - -### 读取数据 {#reading-data} - -假设有一个名为 `football` 的表,其中包含以下数据: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -以 `Null` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT Null -``` - -此查询会处理数据,但不会产生任何输出。 - -```response -结果集包含 0 行。耗时:0.154 秒。 -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md deleted file mode 100644 index e34f6156b49..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ODBCDriver2.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'ODBCDriver2 格式文档' -keywords: ['ODBCDriver2'] -slug: /interfaces/formats/ODBCDriver2 -title: 'ODBCDriver2' -doc_type: 'reference' ---- - - - -## 说明 {#description} - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md deleted file mode 100644 index 417e3bf2895..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/ORC.md +++ /dev/null @@ -1,115 +0,0 @@ ---- -alias: [] -description: 'ORC 格式说明' -input_format: true -keywords: ['ORC'] -output_format: true -slug: /interfaces/formats/ORC -title: 'ORC' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -[Apache ORC](https://orc.apache.org/) 是一种列式存储格式,在 [Hadoop](https://hadoop.apache.org/) 生态系统中被广泛使用。 - - - -## 数据类型匹配 {#data-types-matching-orc} - -下表比较了在 `INSERT` 和 `SELECT` 查询中支持的 ORC 数据类型及其对应的 ClickHouse [数据类型](/sql-reference/data-types/index.md)。 - -| ORC data type (`INSERT`) | ClickHouse data type | ORC data type (`SELECT`) | -|---------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------------------| -| `Boolean` | [UInt8](/sql-reference/data-types/int-uint.md) | `Boolean` | -| `Tinyint` | [Int8/UInt8](/sql-reference/data-types/int-uint.md)/[Enum8](/sql-reference/data-types/enum.md) | `Tinyint` | -| `Smallint` | [Int16/UInt16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | `Smallint` | -| `Int` | [Int32/UInt32](/sql-reference/data-types/int-uint.md) | `Int` | -| `Bigint` | [Int64/UInt32](/sql-reference/data-types/int-uint.md) | `Bigint` | -| `Float` | [Float32](/sql-reference/data-types/float.md) | `Float` | -| `Double` | [Float64](/sql-reference/data-types/float.md) | `Double` | -| `Decimal` | [Decimal](/sql-reference/data-types/decimal.md) | `Decimal` | -| `Date` | [Date32](/sql-reference/data-types/date32.md) | `Date` | -| `Timestamp` | [DateTime64](/sql-reference/data-types/datetime64.md) | `Timestamp` | -| `String`, `Char`, `Varchar`, `Binary` | [String](/sql-reference/data-types/string.md) | `Binary` | -| `List` | [Array](/sql-reference/data-types/array.md) | `List` | -| `Struct` | [Tuple](/sql-reference/data-types/tuple.md) | `Struct` | -| `Map` | [Map](/sql-reference/data-types/map.md) | `Map` | -| `Int` | [IPv4](/sql-reference/data-types/int-uint.md) | `Int` | -| `Binary` | [IPv6](/sql-reference/data-types/ipv6.md) | `Binary` | -| `Binary` | [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `Binary` | -| `Binary` | [Decimal256](/sql-reference/data-types/decimal.md) | `Binary` | - -- 其他类型不受支持。 -- 数组可以嵌套,并且其参数可以是 `Nullable` 类型。`Tuple` 和 `Map` 类型也可以嵌套。 -- ClickHouse 表列的数据类型不必与对应的 ORC 数据字段完全一致。在插入数据时,ClickHouse 会根据上表解析数据类型,然后将数据[转换](/sql-reference/functions/type-conversion-functions#cast)为为 ClickHouse 表列设置的数据类型。 - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用名为 `football.orc` 的 ORC 文件,其内容如下: - -```text - ┌───────date─┬─season─┬─home_team─────────────┬─away_team───────────┬─home_team_goals─┬─away_team_goals─┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.orc' FORMAT ORC; -``` - -### 读取数据 {#reading-data} - -使用 `ORC` 格式来读取数据: - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.orc' -FORMAT ORC -``` - -:::tip -ORC 是一种二进制格式,无法在终端中以人类可读的形式查看。请使用 `INTO OUTFILE` 来输出 ORC 文件。 -::: - - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|---------| -| [`output_format_arrow_string_as_string`](/operations/settings/settings-formats.md/#output_format_arrow_string_as_string) | 对 String 列使用 Arrow String 类型而不是 Binary 类型。 | `false` | -| [`output_format_orc_compression_method`](/operations/settings/settings-formats.md/#output_format_orc_compression_method) | 输出 ORC 格式时使用的压缩方法。 | `none` | -| [`input_format_arrow_case_insensitive_column_matching`](/operations/settings/settings-formats.md/#input_format_arrow_case_insensitive_column_matching) | 在将 Arrow 列与 ClickHouse 列进行匹配时忽略大小写。 | `false` | -| [`input_format_arrow_allow_missing_columns`](/operations/settings/settings-formats.md/#input_format_arrow_allow_missing_columns) | 读取 Arrow 数据时允许缺失列。 | `false` | -| [`input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_arrow_skip_columns_with_unsupported_types_in_schema_inference) | 在对 Arrow 格式进行 schema 推断时,允许跳过包含不受支持类型的列。 | `false` | - -要与 Hadoop 进行数据交换,可以使用 [HDFS 表引擎](/engines/table-engines/integrations/hdfs.md)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/One.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/One.md deleted file mode 100644 index ce335b6ff81..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/One.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -alias: [] -description: 'One 格式文档' -input_format: true -keywords: ['One'] -output_format: false -slug: /interfaces/formats/One -title: 'One' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 描述 {#description} - -`One` 格式是一种特殊的输入格式,它不会从文件中读取任何数据,而是只返回一行数据,该行包含一列,类型为 [`UInt8`](../../sql-reference/data-types/int-uint.md)、名称为 `dummy`、值为 `0`(类似于 `system.one` 表)。 -可以配合虚拟列 `_file/_path` 使用,在不读取实际数据的情况下列出所有文件。 - - - -## 使用示例 {#example-usage} - -示例: - -```sql title="Query" -SELECT _file FROM file('path/to/files/data*', One); -``` - -```text title="Response" -┌─_file────┐ -│ data.csv │ -└──────────┘ -┌─_file──────┐ -│ data.jsonl │ -└────────────┘ -┌─_file────┐ -│ data.tsv │ -└──────────┘ -┌─_file────────┐ -│ data.parquet │ -└──────────────┘ -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md deleted file mode 100644 index 1632ca2cd0d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/Parquet.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -alias: [] -description: 'Parquet 格式文档' -input_format: true -keywords: ['Parquet'] -output_format: true -slug: /interfaces/formats/Parquet -title: 'Parquet' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -[Apache Parquet](https://parquet.apache.org/) 是 Hadoop 生态系统中广泛使用的列式存储格式。ClickHouse 支持对该格式进行读写操作。 - - - -## 数据类型匹配 {#data-types-matching-parquet} - -下表展示了 Parquet 数据类型与 ClickHouse [数据类型](/sql-reference/data-types/index.md)之间的对应关系。 - -| Parquet 类型(逻辑、转换或物理) | ClickHouse 数据类型 | -|------------------------------------------------|----------------------| -| `BOOLEAN` | [Bool](/sql-reference/data-types/boolean.md) | -| `UINT_8` | [UInt8](/sql-reference/data-types/int-uint.md) | -| `INT_8` | [Int8](/sql-reference/data-types/int-uint.md) | -| `UINT_16` | [UInt16](/sql-reference/data-types/int-uint.md) | -| `INT_16` | [Int16](/sql-reference/data-types/int-uint.md)/[Enum16](/sql-reference/data-types/enum.md) | -| `UINT_32` | [UInt32](/sql-reference/data-types/int-uint.md) | -| `INT_32` | [Int32](/sql-reference/data-types/int-uint.md) | -| `UINT_64` | [UInt64](/sql-reference/data-types/int-uint.md) | -| `INT_64` | [Int64](/sql-reference/data-types/int-uint.md) | -| `DATE` | [Date32](/sql-reference/data-types/date.md) | -| `TIMESTAMP`, `TIME` | [DateTime64](/sql-reference/data-types/datetime64.md) | -| `FLOAT` | [Float32](/sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](/sql-reference/data-types/float.md) | -| `INT96` | [DateTime64(9, 'UTC')](/sql-reference/data-types/datetime64.md) | -| `BYTE_ARRAY`, `UTF8`, `ENUM`, `BSON` | [String](/sql-reference/data-types/string.md) | -| `JSON` | [JSON](/sql-reference/data-types/newjson.md) | -| `FIXED_LEN_BYTE_ARRAY` | [FixedString](/sql-reference/data-types/fixedstring.md) | -| `DECIMAL` | [Decimal](/sql-reference/data-types/decimal.md) | -| `LIST` | [Array](/sql-reference/data-types/array.md) | -| `MAP` | [Map](/sql-reference/data-types/map.md) | -| struct | [Tuple](/sql-reference/data-types/tuple.md) | -| `FLOAT16` | [Float32](/sql-reference/data-types/float.md) | -| `UUID` | [FixedString(16)](/sql-reference/data-types/fixedstring.md) | -| `INTERVAL` | [FixedString(12)](/sql-reference/data-types/fixedstring.md) | - -在写入 Parquet 文件时,没有对应 Parquet 类型的 ClickHouse 数据类型会被转换为最接近的可用类型: - -| ClickHouse 数据类型 | Parquet 类型 | -|----------------------|--------------| -| [IPv4](/sql-reference/data-types/ipv4.md) | `UINT_32` | -| [IPv6](/sql-reference/data-types/ipv6.md) | `FIXED_LEN_BYTE_ARRAY`(16 字节) | -| [Date](/sql-reference/data-types/date.md)(16 位) | `DATE`(32 位) | -| [DateTime](/sql-reference/data-types/datetime.md)(32 位,秒) | `TIMESTAMP`(64 位,毫秒) | -| [Int128/UInt128/Int256/UInt256](/sql-reference/data-types/int-uint.md) | `FIXED_LEN_BYTE_ARRAY`(16/32 字节,小端序) | - -`Array` 可以嵌套,其元素可以为 `Nullable` 类型。`Tuple` 和 `Map` 类型也可以嵌套。 - -ClickHouse 表列的数据类型可以与插入的 Parquet 数据中对应字段的数据类型不同。插入数据时,ClickHouse 会根据上表解释数据类型,然后将数据[转换](/sql-reference/functions/type-conversion-functions#cast)为 ClickHouse 表列所设置的数据类型。例如,可以将一个 `UINT_32` Parquet 列读取到一个 [IPv4](/sql-reference/data-types/ipv4.md) ClickHouse 列中。 - - - -对于某些 Parquet 类型,没有与之紧密匹配的 ClickHouse 类型。我们按如下方式读取它们: -* `TIME`(一天中的时间)会被读取为时间戳。例如,`10:23:13.000` 会变成 `1970-01-01 10:23:13.000`。 -* 具有 `isAdjustedToUTC=false` 的 `TIMESTAMP`/`TIME` 表示本地挂钟时间(本地时区下的年、月、日、时、分、秒和子秒字段,而不考虑具体本地时区是哪一个),等同于 SQL 中的 `TIMESTAMP WITHOUT TIME ZONE`。ClickHouse 会将其当作 UTC 时间戳来读取。例如,`2025-09-29 18:42:13.000`(表示本地挂钟的读数)会变成 `2025-09-29 18:42:13.000`(`DateTime64(3, 'UTC')`,表示某个时间点)。如果将其转换为 String,它会显示正确的年、月、日、时、分、秒和子秒,然后可以将其解释为某个本地时区中的时间,而不是 UTC。违背直觉的是,将类型从 `DateTime64(3, 'UTC')` 改为 `DateTime64(3)` 并不会有帮助,因为这两种类型都表示时间点而不是挂钟读数,但 `DateTime64(3)` 会错误地使用本地时区来格式化。 -* `INTERVAL` 当前会被读取为 `FixedString(12)`,其内容是 Parquet 文件中编码的时间间隔的原始二进制表示。 - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用一个包含以下数据的 Parquet 文件,文件名为 `football.parquet`: - -```text - ┌───────日期─┬─赛季───┬─主队─────────────────┬─客队───────────────┬─主队进球数───────┬─客队进球数───────┐ - 1. │ 2022-04-30 │ 2021 │ Sutton United │ Bradford City │ 1 │ 4 │ - 2. │ 2022-04-30 │ 2021 │ Swindon Town │ Barrow │ 2 │ 1 │ - 3. │ 2022-04-30 │ 2021 │ Tranmere Rovers │ Oldham Athletic │ 2 │ 0 │ - 4. │ 2022-05-02 │ 2021 │ Port Vale │ Newport County │ 1 │ 2 │ - 5. │ 2022-05-02 │ 2021 │ Salford City │ Mansfield Town │ 2 │ 2 │ - 6. │ 2022-05-07 │ 2021 │ Barrow │ Northampton Town │ 1 │ 3 │ - 7. │ 2022-05-07 │ 2021 │ Bradford City │ Carlisle United │ 2 │ 0 │ - 8. │ 2022-05-07 │ 2021 │ Bristol Rovers │ Scunthorpe United │ 7 │ 0 │ - 9. │ 2022-05-07 │ 2021 │ Exeter City │ Port Vale │ 0 │ 1 │ -10. │ 2022-05-07 │ 2021 │ Harrogate Town A.F.C. │ Sutton United │ 0 │ 2 │ -11. │ 2022-05-07 │ 2021 │ Hartlepool United │ Colchester United │ 0 │ 2 │ -12. │ 2022-05-07 │ 2021 │ Leyton Orient │ Tranmere Rovers │ 0 │ 1 │ -13. │ 2022-05-07 │ 2021 │ Mansfield Town │ Forest Green Rovers │ 2 │ 2 │ -14. │ 2022-05-07 │ 2021 │ Newport County │ Rochdale │ 0 │ 2 │ -15. │ 2022-05-07 │ 2021 │ Oldham Athletic │ Crawley Town │ 3 │ 3 │ -16. │ 2022-05-07 │ 2021 │ Stevenage Borough │ Salford City │ 4 │ 2 │ -17. │ 2022-05-07 │ 2021 │ Walsall │ Swindon Town │ 0 │ 3 │ - └────────────┴────────┴───────────────────────┴─────────────────────┴─────────────────┴─────────────────┘ -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.parquet' FORMAT Parquet; -``` - -### 读取数据 {#reading-data} - -以 `Parquet` 格式读取数据: - -```sql -SELECT * -FROM football -INTO OUTFILE 'football.parquet' -FORMAT Parquet -``` - -:::tip -Parquet 是一种二进制格式,无法在终端中以人类可读的形式显示。请使用 `INTO OUTFILE` 语句导出 Parquet 文件。 -::: - -要与 Hadoop 进行数据交换,可以使用 [`HDFS table engine`](/engines/table-engines/integrations/hdfs.md)。 - - -## 格式设置 {#format-settings} - - - -| 设置 | 描述 | 默认 | -| ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `input_format_parquet_case_insensitive_column_matching` | 在匹配 Parquet 列与 ClickHouse 列时不区分大小写。 | `0` | -| `input_format_parquet_preserve_order` | 在读取 Parquet 文件时避免对行重新排序,这通常会大大降低读取性能。 | `0` | -| `input_format_parquet_filter_push_down` | 在读取 Parquet 文件时,可以根据 WHERE/PREWHERE 表达式以及 Parquet 元数据中的最小值/最大值统计信息跳过整个行组。 | `1` | -| `input_format_parquet_bloom_filter_push_down` | 在读取 Parquet 文件时,可根据 WHERE 条件和 Parquet 元数据中的布隆过滤器跳过整个行组。 | `0` | -| `input_format_parquet_use_native_reader` | 在读取 Parquet 文件时,使用原生读取器而不是 Arrow 读取器。 | `0` | -| `input_format_parquet_allow_missing_columns` | 在读取 Parquet 输入格式时允许列缺失 | `1` | -| `input_format_parquet_local_file_min_bytes_for_seek` | 在 Parquet 输入格式中,本地读取文件时,为执行 seek 而不是执行带 ignore 选项的读取所需的最小字节数 | `8192` | -| `input_format_parquet_enable_row_group_prefetch` | 在解析 Parquet 时启用行组预取。当前仅支持在单线程解析时进行预取。 | `1` | -| `input_format_parquet_skip_columns_with_unsupported_types_in_schema_inference` | 在为 Parquet 格式进行模式推断时跳过类型不受支持的列 | `0` | -| `input_format_parquet_max_block_size` | Parquet 读取器的最大块大小。 | `65409` | -| `input_format_parquet_prefer_block_bytes` | Parquet 读取器输出的平均数据块大小(字节) | `16744704` | -| `input_format_parquet_enable_json_parsing` | 在读取 Parquet 文件时,将 JSON 列解析为 ClickHouse 的 JSON Column 类型。 | `1` | -| `output_format_parquet_row_group_size` | 目标行组大小(以行数计)。 | `1000000` | -| `output_format_parquet_row_group_size_bytes` | 目标行组大小(压缩前),单位为字节。 | `536870912` | -| `output_format_parquet_string_as_string` | 对 String 列使用 Parquet 的 String 类型,而不是 Binary 类型。 | `1` | -| `output_format_parquet_fixed_string_as_fixed_byte_array` | 对于 FixedString 列,请使用 Parquet 的 FIXED_LEN_BYTE_ARRAY 类型而不是 Binary。 | `1` | -| `output_format_parquet_version` | 输出时使用的 Parquet 格式版本。支持的版本:1.0、2.4、2.6 和 2.latest(默认) | `2.latest` | -| `output_format_parquet_compression_method` | Parquet 输出格式的压缩方式。支持的编解码器:snappy、lz4、brotli、zstd、gzip、none(不压缩) | `zstd` | -| `output_format_parquet_compliant_nested_types` | 在 Parquet 文件模式中,列表元素的名称应使用 'element' 而不是 'item'。这是 Arrow 库实现中的历史遗留问题。一般情况下可以提高兼容性,但某些旧版本的 Arrow 可能不兼容。 | `1` | -| `output_format_parquet_use_custom_encoder` | 使用更快速的 Parquet 编码器实现。 | `1` | -| `output_format_parquet_parallel_encoding` | 使用多线程进行 Parquet 编码。需要启用 output_format_parquet_use_custom_encoder。 | `1` | -| `output_format_parquet_data_page_size` | 压缩前的目标页大小(字节)。 | `1048576` | -| `output_format_parquet_batch_size` | 每隔这么多行检查一次页大小。如果某些列中单个值的平均大小超过数 KB,建议适当减小该值。 | `1024` | -| `output_format_parquet_write_page_index` | 新增在 Parquet 文件中写入页索引的能力。 | `1` | -| `input_format_parquet_import_nested` | 已废弃的设置,不会产生任何作用。 | `0` | -| `input_format_parquet_local_time_as_utc` | true | 确定在 `isAdjustedToUTC=false` 的情况下,模式推断对 Parquet 时间戳使用的数据类型。若为 true:DateTime64(..., 'UTC'),若为 false:DateTime64(...)。这两种行为都不完全正确,因为 ClickHouse 没有用于本地墙钟时间的数据类型。看似有些反直觉,但 true 可能是错误更小的选项,因为将带有 'UTC' 的时间戳格式化为 String 时,会得到正确本地时间的表示。 | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md deleted file mode 100644 index 73b7099a2a5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Parquet/ParquetMetadata.md +++ /dev/null @@ -1,147 +0,0 @@ ---- -description: 'ParquetMetadata 格式说明文档' -keywords: ['ParquetMetadata'] -slug: /interfaces/formats/ParquetMetadata -title: 'ParquetMetadata' -doc_type: 'reference' ---- - - - -## 描述 {#description} - -用于读取 Parquet 文件元数据(https://parquet.apache.org/docs/file-format/metadata/)的特殊格式。它始终只输出一行,结构/内容如下: -- `num_columns` - 列的数量 -- `num_rows` - 行的总数 -- `num_row_groups` - 行组的总数 -- `format_version` - Parquet 格式版本,始终为 1.0 或 2.6 -- `total_uncompressed_size` - 数据的未压缩字节总大小,按所有行组的 total_byte_size 之和计算 -- `total_compressed_size` - 数据的压缩字节总大小,按所有行组的 total_compressed_size 之和计算 -- `columns` - 列元数据列表,其结构如下: - - `name` - 列名 - - `path` - 列路径(对嵌套列与列名不同) - - `max_definition_level` - 最大定义级别(definition level) - - `max_repetition_level` - 最大重复级别(repetition level) - - `physical_type` - 列的物理类型 - - `logical_type` - 列的逻辑类型 - - `compression` - 此列使用的压缩方式 - - `total_uncompressed_size` - 列的未压缩字节总大小,按该列在所有行组中的 total_uncompressed_size 之和计算 - - `total_compressed_size` - 列的压缩字节总大小,按该列在所有行组中的 total_compressed_size 之和计算 - - `space_saved` - 由于压缩节省的空间百分比,计算公式为 (1 - total_compressed_size/total_uncompressed_size)。 - - `encodings` - 此列使用的编码列表 -- `row_groups` - 行组元数据列表,其结构如下: - - `num_columns` - 行组中的列数 - - `num_rows` - 行组中的行数 - - `total_uncompressed_size` - 行组的未压缩字节总大小 - - `total_compressed_size` - 行组的压缩字节总大小 - - `columns` - 列块元数据列表,其结构如下: - - `name` - 列名 - - `path` - 列路径 - - `total_compressed_size` - 列的压缩字节总大小 - - `total_uncompressed_size` - 行组的未压缩字节总大小 - - `have_statistics` - 布尔标志,指示列块元数据是否包含列统计信息 - - `statistics` - 列块统计信息(如果 have_statistics = false,则所有字段为 NULL),其结构如下: - - `num_values` - 列块中非 NULL 值的数量 - - `null_count` - 列块中 NULL 值的数量 - - `distinct_count` - 列块中不同值的数量 - - `min` - 列块的最小值 - - `max` - 列块的最大值 - - - -## 使用示例 {#example-usage} - -示例: - -```sql -SELECT * -FROM file(data.parquet, ParquetMetadata) -FORMAT PrettyJSONEachRow -``` - -```json -{ - "num_columns": "2", - "num_rows": "100000", - "num_row_groups": "2", - "format_version": "2.6", - "metadata_size": "577", - "total_uncompressed_size": "282436", - "total_compressed_size": "26633", - "columns": [ - { - "name": "number", - "path": "number", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "INT32", - "logical_type": "Int(bitWidth=16, isSigned=false)", - "compression": "LZ4", - "total_uncompressed_size": "133321", - "total_compressed_size": "13293", - "space_saved": "90.03%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "max_definition_level": "0", - "max_repetition_level": "0", - "physical_type": "BYTE_ARRAY", - "logical_type": "None", - "compression": "LZ4", - "total_uncompressed_size": "149115", - "total_compressed_size": "13340", - "space_saved": "91.05%", - "encodings": [ - "RLE_DICTIONARY", - "PLAIN", - "RLE" - ] - } - ], - "row_groups": [ - { - "num_columns": "2", - "num_rows": "65409", - "total_uncompressed_size": "179809", - "total_compressed_size": "14163", - "columns": [ - { - "name": "number", - "path": "number", - "total_compressed_size": "7070", - "total_uncompressed_size": "85956", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "0", - "max": "999" - } - }, - { - "name": "concat('Hello', toString(modulo(number, 1000)))", - "path": "concat('Hello', toString(modulo(number, 1000)))", - "total_compressed_size": "7093", - "total_uncompressed_size": "93853", - "have_statistics": true, - "statistics": { - "num_values": "65409", - "null_count": "0", - "distinct_count": null, - "min": "Hello0", - "max": "Hello999" - } - } - ] - }, - ... - ] -} -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md deleted file mode 100644 index 0e8fcf472cb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/PostgreSQLWire.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'PostgreSQLWire 格式文档' -keywords: ['PostgreSQLWire'] -slug: /interfaces/formats/PostgreSQLWire -title: 'PostgreSQLWire' -doc_type: 'reference' ---- - - - -## 说明 {#description} - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md deleted file mode 100644 index ef7c895a9a1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/Pretty.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -alias: [] -description: 'Pretty 格式文档' -input_format: false -keywords: ['Pretty'] -output_format: true -slug: /interfaces/formats/Pretty -title: 'Pretty' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -`Pretty` 格式以 Unicode 绘制的字符表格形式输出数据, -在终端中使用 ANSI 转义序列来显示颜色。 -表格的完整网格会被绘制出来,并且每一行在终端中占用两行。 -每个结果块都会作为一个单独的表格输出。 -这样就可以在不对结果进行缓冲的情况下输出这些块(否则就必须先缓冲结果,以便预先计算所有值的可见宽度)。 - -[NULL](/sql-reference/syntax.md) 会被输出为 `ᴺᵁᴸᴸ`。 - - - -## 使用示例 {#example-usage} - -示例(以 [`PrettyCompact`](./PrettyCompact.md) 格式为例): - -```sql title="Query" -SELECT * FROM t_null -``` - -```response title="Response" -┌─x─┬────y─┐ -│ 1 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -在所有 `Pretty` 系列格式中,行都不会被转义。以下示例展示的是 [`PrettyCompact`](./PrettyCompact.md) 格式: - -```sql title="Query" -SELECT 'String with \'quotes\' and \t character' AS Escaping_test -``` - -```response title="Response" -┌─Escaping_test────────────────────────┐ -│ 包含 'quotes' 和 字符的字符串 │ -└──────────────────────────────────────┘ -``` - -为了避免向终端输出过多数据,仅打印前 `10,000` 行。 -如果行数大于或等于 `10,000`,则会打印消息 "Showed first 10 000"。 - -:::note -此格式只适用于输出查询结果,不适合用于解析数据。 -::: - -Pretty 格式支持输出总计(使用 `WITH TOTALS` 时)和极值(当 `extremes` 被设置为 1 时)。 -在这些情况下,总计和极值会在主数据之后以单独的表格输出。 -下面的示例展示了这一点,示例中使用了 [`PrettyCompact`](./PrettyCompact.md) 格式: - -```sql title="Query" -SELECT EventDate, count() AS c -FROM test.hits -GROUP BY EventDate -WITH TOTALS -ORDER BY EventDate -FORMAT PrettyCompact -``` - -```response title="Response" -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1406958 │ -│ 2014-03-18 │ 1383658 │ -│ 2014-03-19 │ 1405797 │ -│ 2014-03-20 │ 1353623 │ -│ 2014-03-21 │ 1245779 │ -│ 2014-03-22 │ 1031592 │ -│ 2014-03-23 │ 1046491 │ -└────────────┴─────────┘ - -总计: -┌──EventDate─┬───────c─┐ -│ 1970-01-01 │ 8873898 │ -└────────────┴─────────┘ - -极值: -┌──EventDate─┬───────c─┐ -│ 2014-03-17 │ 1031592 │ -│ 2014-03-23 │ 1406958 │ -└────────────┴─────────┘ -``` - - -## 格式设置 {#format-settings} - - diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md deleted file mode 100644 index 08033e0b60f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompact.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -alias: [] -description: 'PrettyCompact 格式文档' -input_format: false -keywords: ['PrettyCompact'] -output_format: true -slug: /interfaces/formats/PrettyCompact -title: 'PrettyCompact' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`Pretty`](./Pretty.md) 格式的区别在于,该格式会在表格的行之间绘制网格线。 -因此,输出更加紧凑。 - -:::note -在交互模式下,该格式是命令行客户端的默认输出格式。 -::: - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md deleted file mode 100644 index 5eeadd7e530..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactMonoBlock 格式文档' -input_format: false -keywords: ['PrettyCompactMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactMonoBlock -title: 'PrettyCompactMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettyCompact`](./PrettyCompact.md) 格式不同之处在于,它会先最多缓冲 `10,000` 行数据, -然后作为单个表输出,而不是按[块](/development/architecture#block)输出。 - - - -## 使用示例 {#example-usage} - - - -## 格式化设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md deleted file mode 100644 index 489e63aa508..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactNoEscapes 格式文档' -input_format: false -keywords: ['PrettyCompactNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapes -title: 'PrettyCompactNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettyCompact`](./PrettyCompact.md) 格式的区别在于,它不使用 [ANSI 转义序列](http://en.wikipedia.org/wiki/ANSI_escape_code)。 -这样才能在浏览器中正确显示该格式,并配合 `watch` 命令行工具使用。 - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} - - diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md deleted file mode 100644 index ae904320ef1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyCompactNoEscapesMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettyCompactNoEscapesMonoBlock 格式文档' -input_format: false -keywords: ['PrettyCompactNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyCompactNoEscapesMonoBlock -title: 'PrettyCompactNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettyCompactNoEscapes`](./PrettyCompactNoEscapes.md) 格式不同之处在于,它会先缓冲至多 `10,000` 行, -然后将其作为一张完整的表输出,而不是按[数据块](/development/architecture#block)输出。 - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md deleted file mode 100644 index 891534f151f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettyMonoBlock 格式的文档' -input_format: false -keywords: ['PrettyMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyMonoBlock -title: 'PrettyMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`Pretty`](/interfaces/formats/Pretty) 格式不同之处在于,它会最多缓冲 `10,000` 行, -然后一次性输出为一个完整的表,而不是按[数据块](/development/architecture#block)输出。 - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md deleted file mode 100644 index c1dadbf2f5d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapes.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -alias: [] -description: 'PrettyNoEscapes 格式文档' -input_format: false -keywords: ['PrettyNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapes -title: 'PrettyNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [Pretty](/interfaces/formats/Pretty) 的区别在于它不使用 [ANSI 转义序列](http://en.wikipedia.org/wiki/ANSI_escape_code)。 -这对于在浏览器中显示该格式以及使用 `watch` 命令行工具是必需的。 - - - -## 使用示例 {#example-usage} - -示例: - -```bash -$ watch -n1 "clickhouse-client --query='SELECT event, value FROM system.events FORMAT PrettyCompactNoEscapes'" -``` - -:::note -可以使用 [HTTP 接口](../../../interfaces/http.md) 在浏览器中显示该格式。 -::: - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md deleted file mode 100644 index cc4bc3612d0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettyNoEscapesMonoBlock.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettyNoEscapesMonoBlock 格式文档' -input_format: false -keywords: ['PrettyNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettyNoEscapesMonoBlock -title: 'PrettyNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettyNoEscapes`](./PrettyNoEscapes.md) 格式的区别在于:最多会缓存 `10,000` 行数据,然后一次性以单个表格输出,而不是分块输出。 - - - -## 示例用法 {#example-usage} - - - -## 格式化设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md deleted file mode 100644 index 81501e29aed..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpace.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'PrettySpace 格式文档' -input_format: false -keywords: ['PrettySpace'] -output_format: true -slug: /interfaces/formats/PrettySpace -title: 'PrettySpace' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettyCompact`](./PrettyCompact.md) 格式不同之处在于,该格式使用空白(空格字符)来显示表格,而不是使用网格。 - - - -## 用法示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md deleted file mode 100644 index 94ef78210a6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceMonoBlock 格式的文档' -input_format: false -keywords: ['PrettySpaceMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceMonoBlock -title: 'PrettySpaceMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettySpace`](./PrettySpace.md) 格式不同,此格式最多缓冲 `10,000` 行, -然后将其作为单个表输出,而不是按[数据块](/development/architecture#block)输出。 - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md deleted file mode 100644 index 9c3482a3ea4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapes.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceNoEscapes 格式文档' -input_format: false -keywords: ['PrettySpaceNoEscapes'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapes -title: 'PrettySpaceNoEscapes' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettySpace`](./PrettySpace.md) 格式的不同之处在于,它不使用 [ANSI 转义序列](http://en.wikipedia.org/wiki/ANSI_escape_code)。 -这对于在浏览器中显示此格式,以及配合 `watch` 命令行工具使用是必需的。 - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md deleted file mode 100644 index f341e270f50..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/PrettySpaceNoEscapesMonoBlock.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -alias: [] -description: 'PrettySpaceNoEscapesMonoBlock 格式文档' -input_format: false -keywords: ['PrettySpaceNoEscapesMonoBlock'] -output_format: true -slug: /interfaces/formats/PrettySpaceNoEscapesMonoBlock -title: 'PrettySpaceNoEscapesMonoBlock' -doc_type: 'reference' ---- - -import PrettyFormatSettings from './_snippets/common-pretty-format-settings.md'; - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✗ | ✔ | | - - -## 描述 {#description} - -与 [`PrettySpaceNoEscapes`](./PrettySpaceNoEscapes.md) 格式不同之处在于,会先最多缓冲 `10,000` 行数据, -然后将其作为一张完整的表输出,而不是按[数据块](/development/architecture#block)输出。 - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md deleted file mode 100644 index 648c7db512a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Pretty/_snippets/common-pretty-format-settings.md +++ /dev/null @@ -1,14 +0,0 @@ -{/* 注意:此文件会在所有导入它的文件中作为代码片段使用 */ } - -适用于所有 `Pretty` 格式的通用设置如下: - -| Setting | Description | Default | -| ------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| [`output_format_pretty_max_rows`](/operations/settings/settings-formats.md/#output_format_pretty_max_rows) | Pretty 格式的行数上限。 | `10000` | -| [`output_format_pretty_max_column_pad_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_column_pad_width) | 在 Pretty 格式中,用于对列内所有值进行填充的最大宽度。 | `250` | -| [`output_format_pretty_max_value_width`](/operations/settings/settings-formats.md/#output_format_pretty_max_value_width) | 在 Pretty 格式中要显示的单个值的最大宽度。如果超过该值,则会被截断。 | `10000` | -| [`output_format_pretty_color`](/operations/settings/settings-formats.md/#output_format_pretty_color) | 在 Pretty 格式中使用 ANSI 转义序列输出彩色内容。 | `true` | -| [`output_format_pretty_grid_charset`](/operations/settings/settings-formats.md/#output_format_pretty_grid_charset) | 打印表格边框所用的字符集。可用字符集:ASCII、UTF-8。 | `UTF-8` | -| [`output_format_pretty_row_numbers`](/operations/settings/settings-formats.md/#output_format_pretty_row_numbers) | 对 Pretty 输出格式,在每一行前添加行号。 | `true` | -| [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names) | 如果表包含大量行,则在页脚显示列名。 | `true` | -| [`output_format_pretty_display_footer_column_names_min_rows`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names_min_rows) | 当启用 [`output_format_pretty_display_footer_column_names`](/operations/settings/settings-formats.md/#output_format_pretty_display_footer_column_names) 时,设置显示页脚所需的最小行数。 | `50` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md deleted file mode 100644 index 64221609726..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Prometheus.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -alias: [] -description: 'Prometheus 格式文档' -input_format: false -keywords: ['Prometheus'] -output_format: true -slug: /interfaces/formats/Prometheus -title: 'Prometheus' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - -## 描述 {#description} - -以 [Prometheus 文本暴露格式](https://prometheus.io/docs/instrumenting/exposition_formats/#text-based-format)导出指标数据。 - -对于该格式,输出表必须满足以下结构规则: - -- 必须包含列 `name`([String](/sql-reference/data-types/string.md))和 `value`(数值)。 -- 行中还可以包含可选列 `help`([String](/sql-reference/data-types/string.md))和 `timestamp`(数值)。 -- 列 `type`([String](/sql-reference/data-types/string.md))的取值应为 `counter`、`gauge`、`histogram`、`summary`、`untyped` 之一,或为空。 -- 每个指标值还可以带有若干 `labels`([Map(String, String)](/sql-reference/data-types/map.md))。 -- 若干连续的行可以对应同一个指标,但具有不同的 labels。表应按指标名称排序(例如使用 `ORDER BY name`)。 - -对于 `histogram` 和 `summary` 的 labels 有特殊要求,详情参见 [Prometheus 文档](https://prometheus.io/docs/instrumenting/exposition_formats/#histograms-and-summaries)。 -对标签为 `{'count':''}` 和 `{'sum':''}` 的行会应用特殊规则,它们分别会被转换为 `_count` 和 `_sum`。 - -## 使用示例 {#example-usage} - -```yaml -┌─name────────────────────────────────┬─type──────┬─help──────────────────────────────────────┬─labels─────────────────────────┬────value─┬─────timestamp─┐ -│ http_request_duration_seconds │ histogram │ 请求持续时间直方图。 │ {'le':'0.05'} │ 24054 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.1'} │ 33444 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.2'} │ 100392 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'0.5'} │ 129389 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'1'} │ 133988 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'le':'+Inf'} │ 144320 │ 0 │ -│ http_request_duration_seconds │ histogram │ │ {'sum':''} │ 53423 │ 0 │ -│ http_requests_total │ counter │ HTTP 请求总数 │ {'method':'post','code':'200'} │ 1027 │ 1395066363000 │ -│ http_requests_total │ counter │ │ {'method':'post','code':'400'} │ 3 │ 1395066363000 │ -│ metric_without_timestamp_and_labels │ │ │ {} │ 12.47 │ 0 │ -│ rpc_duration_seconds │ summary │ RPC 持续时间(秒)摘要。 │ {'quantile':'0.01'} │ 3102 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.05'} │ 3272 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.5'} │ 4773 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.9'} │ 9001 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'quantile':'0.99'} │ 76656 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'count':''} │ 2693 │ 0 │ -│ rpc_duration_seconds │ summary │ │ {'sum':''} │ 17560473 │ 0 │ -│ something_weird │ │ │ {'problem':'division by zero'} │ inf │ -3982045 │ -└─────────────────────────────────────┴───────────┴───────────────────────────────────────────┴────────────────────────────────┴──────────┴───────────────┘ -``` - -格式如下: - -```text -# HELP http_request_duration_seconds 请求持续时间的直方图。 {#help-http_request_duration_seconds-a-histogram-of-the-request-duration} -# TYPE http_request_duration_seconds histogram {#type-http_request_duration_seconds-histogram} -http_request_duration_seconds_bucket{le="0.05"} 24054 -http_request_duration_seconds_bucket{le="0.1"} 33444 -http_request_duration_seconds_bucket{le="0.5"} 129389 -http_request_duration_seconds_bucket{le="1"} 133988 -http_request_duration_seconds_bucket{le="+Inf"} 144320 -http_request_duration_seconds_sum 53423 -http_request_duration_seconds_count 144320 - -# HELP http_requests_total HTTP 请求总数 {#help-http_requests_total-total-number-of-http-requests} -# TYPE http_requests_total counter {#type-http_requests_total-counter} -http_requests_total{code="200",method="post"} 1027 1395066363000 -http_requests_total{code="400",method="post"} 3 1395066363000 - -metric_without_timestamp_and_labels 12.47 - -# HELP rpc_duration_seconds RPC 调用时长(秒)的概要。 {#help-rpc_duration_seconds-a-summary-of-the-rpc-duration-in-seconds} -# TYPE rpc_duration_seconds summary {#type-rpc_duration_seconds-summary} -rpc_duration_seconds{quantile="0.01"} 3102 -rpc_duration_seconds{quantile="0.05"} 3272 -rpc_duration_seconds{quantile="0.5"} 4773 -rpc_duration_seconds{quantile="0.9"} 9001 -rpc_duration_seconds{quantile="0.99"} 76656 -rpc_duration_seconds_sum 17560473 -rpc_duration_seconds_count 2693 - -something_weird{problem="division by zero"} +Inf -3982045 -``` - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md deleted file mode 100644 index 1c0d0c7a18a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/Protobuf.md +++ /dev/null @@ -1,387 +0,0 @@ ---- -alias: [] -description: 'Protobuf 格式文档' -input_format: true -keywords: ['Protobuf'] -output_format: true -slug: /interfaces/formats/Protobuf -title: 'Protobuf' -doc_type: 'guide' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - -## 描述 {#description} - -`Protobuf` 格式是 [Protocol Buffers](https://protobuf.dev/) 格式。 - -此格式需要一个外部格式 schema,并且会在不同查询之间缓存。 - -ClickHouse 支持: - -* `proto2` 和 `proto3` 两种语法。 -* `Repeated`/`optional`/`required` 字段。 - -为了确定表列与 Protocol Buffers 消息类型字段之间的对应关系,ClickHouse 会比较它们的名称。 -该比较不区分大小写,并且将字符 `_`(下划线)和 `.`(点)视为相同。 -如果表列类型与 Protocol Buffers 消息字段类型不同,则会应用必要的类型转换。 - -支持嵌套消息。例如,对于下列消息类型中的字段 `z`: - -```capnp -message MessageType { - message XType { - message YType { - int32 z; - }; - repeated YType y; - }; - XType x; -}; -``` - -ClickHouse 会尝试查找名为 `x.y.z`(或 `x_y_z`、`X.y_Z` 等)的列。 - -嵌套消息适合作为[嵌套数据结构](/sql-reference/data-types/nested-data-structures/index.md)的输入或输出。 - -如下所示,在 protobuf 架构(schema)中定义的默认值不会被应用,而是会使用[表默认值](/sql-reference/statements/create/table#default_values)来代替它们: - -```capnp -syntax = "proto2"; - -message MessageType { - optional int32 result_per_page = 3 [default = 10]; -} -``` - -如果消息包含 [oneof](https://protobuf.dev/programming-guides/proto3/#oneof),并且设置了 `input_format_protobuf_oneof_presence`,ClickHouse 会填充一个列,用于指示该 oneof 中实际出现的是哪个字段。 - -```capnp -syntax = "proto3"; - -message StringOrString { - oneof string_oneof { - string string1 = 1; - string string2 = 42; - } -} -``` - -```sql -CREATE TABLE string_or_string ( string1 String, string2 String, string_oneof Enum('no'=0, 'hello' = 1, 'world' = 42)) Engine=MergeTree ORDER BY tuple(); -INSERT INTO string_or_string from INFILE '$CURDIR/data_protobuf/String1' SETTINGS format_schema='$SCHEMADIR/string_or_string.proto:StringOrString' FORMAT ProtobufSingle; -SELECT * FROM string_or_string -``` - -```text - ┌─────────┬─────────┬──────────────┐ - │ string1 │ string2 │ string_oneof │ - ├─────────┼─────────┼──────────────┤ -1. │ │ string2 │ world │ - ├─────────┼─────────┼──────────────┤ -2. │ string1 │ │ hello │ - └─────────┴─────────┴──────────────┘ -``` - -表示存在性的列名必须与 oneof 的名称相同。支持嵌套消息(参见 [basic-examples](#basic-examples))。 -允许的类型为 Int8、UInt8、Int16、UInt16、Int32、UInt32、Int64、UInt64、Enum、Enum8 或 Enum16。 -Enum(以及 Enum8 或 Enum16)必须包含 oneof 的所有可能标签外加 0(用于表示不存在),其字符串表示形式无关紧要。 - -设置 [`input_format_protobuf_oneof_presence`](/operations/settings/settings-formats.md#input_format_protobuf_oneof_presence) 默认是禁用的。 - -ClickHouse 以 `length-delimited` 格式输入和输出 protobuf 消息。 -这意味着在每条消息之前,都应将其长度写为[可变宽度整数(varint)](https://developers.google.com/protocol-buffers/docs/encoding#varints)。 - - -## 示例用法 {#example-usage} - -### 读取和写入数据 {#basic-examples} - -:::note 示例文件 -本示例中使用的文件可在 [examples 仓库](https://github.com/ClickHouse/formats/ProtoBuf) 中获取。 -::: - -在本示例中,我们将把文件 `protobuf_message.bin` 中的一些数据读取到一个 ClickHouse 表中,然后使用 `Protobuf` 格式将其写回到名为 `protobuf_message_from_clickhouse.bin` 的文件中。 - -给定文件 `schemafile.proto`: - -```capnp -syntax = "proto3"; - -message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; -}; -``` - - -
- 生成二进制文件 - - 如果您已经了解如何以 `Protobuf` 格式序列化和反序列化数据,可以跳过此步骤。 - - 我们将使用 Python 将一些数据序列化到 `protobuf_message.bin` 中,然后将其读入 ClickHouse。 - 如果您希望使用其他语言,请参阅:[《如何在常用语言中读写按长度分隔的 Protobuf 消息》](https://cwiki.apache.org/confluence/display/GEODE/Delimiting+Protobuf+Messages)。 - - 运行以下命令,在与 `schemafile.proto` 相同的目录中生成一个名为 `schemafile_pb2.py` 的 Python 文件。 - 此文件包含表示 `UserData` Protobuf 消息的 Python 类: - - ```bash - protoc --python_out=. schemafile.proto - ``` - - 现在,在与 `schemafile_pb2.py` 相同的目录中创建一个名为 `generate_protobuf_data.py` 的新 Python 文件, - 并将以下代码粘贴到该文件中: - - ```python - import schemafile_pb2 # Module generated by 'protoc' - from google.protobuf import text_format - from google.protobuf.internal.encoder import _VarintBytes # Import the internal varint encoder - - def create_user_data_message(name, surname, birthDate, phoneNumbers): - """ - Creates and populates a UserData Protobuf message. - """ - message = schemafile_pb2.MessageType() - message.name = name - message.surname = surname - message.birthDate = birthDate - message.phoneNumbers.extend(phoneNumbers) - return message - - # The data for our example users - data_to_serialize = [ - {"name": "Aisha", "surname": "Khan", "birthDate": 19920815, "phoneNumbers": ["(555) 247-8903", "(555) 612-3457"]}, - {"name": "Javier", "surname": "Rodriguez", "birthDate": 20001015, "phoneNumbers": ["(555) 891-2046", "(555) 738-5129"]}, - {"name": "Mei", "surname": "Ling", "birthDate": 19980616, "phoneNumbers": ["(555) 956-1834", "(555) 403-7682"]}, - ] - - output_filename = "protobuf_messages.bin" - - # Open the binary file in write-binary mode ('wb') - with open(output_filename, "wb") as f: - for item in data_to_serialize: - # Create a Protobuf message instance for the current user - message = create_user_data_message( - item["name"], - item["surname"], - item["birthDate"], - item["phoneNumbers"] - ) - - # Serialize the message - serialized_data = message.SerializeToString() - - # Get the length of the serialized data - message_length = len(serialized_data) - - # Use the Protobuf library's internal _VarintBytes to encode the length - length_prefix = _VarintBytes(message_length) - - # Write the length prefix - f.write(length_prefix) - # Write the serialized message data - f.write(serialized_data) - - print(f"Protobuf messages (length-delimited) written to {output_filename}") - - # --- Optional: Verification (reading back and printing) --- - # For reading back, we'll also use the internal Protobuf decoder for varints. - from google.protobuf.internal.decoder import _DecodeVarint32 - - print("\n--- Verifying by reading back ---") - with open(output_filename, "rb") as f: - buf = f.read() # Read the whole file into a buffer for easier varint decoding - n = 0 - while n < len(buf): - # Decode the varint length prefix - msg_len, new_pos = _DecodeVarint32(buf, n) - n = new_pos - - # Extract the message data - message_data = buf[n:n+msg_len] - n += msg_len - - # Parse the message - decoded_message = schemafile_pb2.MessageType() - decoded_message.ParseFromString(message_data) - print(text_format.MessageToString(decoded_message, as_utf8=True)) - ``` - - 现在从命令行运行该脚本。建议在 Python 虚拟环境中运行,例如使用 `uv`: - - ```bash - uv venv proto-venv - source proto-venv/bin/activate - ``` - - 您需要安装以下 Python 库: - - ```bash - uv pip install --upgrade protobuf - ``` - - 运行脚本以生成二进制文件: - - ```bash - python generate_protobuf_data.py - ``` -
- -创建一个与该 schema 匹配的 ClickHouse 表: - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - - -在命令行中将数据插入表中: - -```bash -cat protobuf_messages.bin | clickhouse-client --query "INSERT INTO test.protobuf_messages SETTINGS format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -你还可以使用 `Protobuf` 格式将数据写回二进制文件中: - -```sql -SELECT * FROM test.protobuf_messages INTO OUTFILE 'protobuf_message_from_clickhouse.bin' FORMAT Protobuf SETTINGS format_schema = 'schemafile:MessageType' -``` - -有了你的 Protobuf 模式定义,你现在可以对 ClickHouse 写入到文件 `protobuf_message_from_clickhouse.bin` 的数据进行反序列化了。 - - -### 使用 ClickHouse Cloud 读取和写入数据 - -在 ClickHouse Cloud 中,您无法上传 Protobuf schema 文件。不过,您可以使用 `format_protobuf_schema` -设置项在查询中指定该 schema。下面的示例演示如何从本地机器读取序列化数据,并将其插入 ClickHouse Cloud 中的一张表。 - -与前一个示例类似,请在 ClickHouse Cloud 中根据 Protobuf schema 定义的结构创建表: - -```sql -CREATE DATABASE IF NOT EXISTS test; -CREATE TABLE IF NOT EXISTS test.protobuf_messages ( - name String, - surname String, - birthDate UInt32, - phoneNumbers Array(String) -) -ENGINE = MergeTree() -ORDER BY tuple() -``` - -`format_schema_source` 设置用于定义 `format_schema` 的来源。 - -可能的取值: - -* 'file'(默认):在 Cloud 中不支持 -* 'string':`format_schema` 为 schema 的字面内容。 -* 'query':`format_schema` 为用于获取 schema 的查询语句。 - - -### `format_schema_source='string'` - -要将数据插入 ClickHouse Cloud,并将 schema 以字符串形式指定时,请运行: - -```bash -cat protobuf_messages.bin | clickhouse client --host <主机名> --secure --password <密码> --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -选择已插入表中的数据: - -```sql -clickhouse client --host <主机名> --secure --password <密码> --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### `format_schema_source='query'` - -你还可以将 Protobuf schema 存储在一张表中。 - -在 ClickHouse Cloud 上创建一张用于插入数据的表: - -```sql -CREATE TABLE testing.protobuf_schema ( - schema String -) -ENGINE = MergeTree() -ORDER BY tuple(); -``` - -```sql -INSERT INTO testing.protobuf_schema VALUES ('syntax = "proto3";message MessageType { string name = 1; string surname = 2; uint32 birthDate = 3; repeated string phoneNumbers = 4;};'); -``` - -将数据插入 ClickHouse Cloud,并通过查询语句指定 schema: - -```bash -cat protobuf_messages.bin | clickhouse client --host <主机名> --secure --password <密码> --query "INSERT INTO testing.protobuf_messages SETTINGS format_schema_source='SELECT schema FROM testing.protobuf_schema', format_schema='schemafile:MessageType' FORMAT Protobuf" -``` - -查询已插入到该表中的数据: - -```sql -clickhouse client --host <主机名> --secure --password <密码> --query "SELECT * FROM testing.protobuf_messages" -``` - -```response -Aisha Khan 19920815 ['(555) 247-8903','(555) 612-3457'] -Javier Rodriguez 20001015 ['(555) 891-2046','(555) 738-5129'] -Mei Ling 19980616 ['(555) 956-1834','(555) 403-7682'] -``` - - -### 使用自动生成的 schema - -如果你的数据没有外部的 Protobuf schema,仍然可以使用自动生成的 schema 以 Protobuf 格式输出/输入数据。为此,请使用 `format_protobuf_use_autogenerated_schema` 设置。 - -例如: - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1 -``` - -在这种情况下,ClickHouse 会根据表结构,使用函数 -[`structureToProtobufSchema`](/sql-reference/functions/other-functions#structureToProtobufSchema) 自动生成 Protobuf 模式。然后它会使用该模式以 Protobuf 格式序列化数据。 - -你也可以使用自动生成的模式来读取 Protobuf 文件。在这种情况下,必须使用相同的模式来创建该文件: - -```bash -$ cat hits.bin | clickhouse-client --query "INSERT INTO test.hits SETTINGS format_protobuf_use_autogenerated_schema=1 FORMAT Protobuf" -``` - -设置 [`format_protobuf_use_autogenerated_schema`](/operations/settings/settings-formats.md#format_protobuf_use_autogenerated_schema) 默认启用,并且在未设置 [`format_schema`](/operations/settings/formats#format_schema) 时生效。 - -你还可以在输入/输出期间,通过设置 [`output_format_schema`](/operations/settings/formats#output_format_schema) 将自动生成的 schema 保存到文件中。例如: - -```sql -SELECT * FROM test.hits format Protobuf SETTINGS format_protobuf_use_autogenerated_schema=1, output_format_schema='path/to/schema/schema.proto' -``` - -在这种情况下,会将自动生成的 Protobuf schema 保存在文件 `path/to/schema/schema.capnp` 中。 - - -### 删除 Protobuf 缓存 {#basic-examples-cloud} - -要重新加载从 [`format_schema_path`](/operations/server-configuration-parameters/settings.md/#format_schema_path) 加载的 Protobuf 架构,请使用 [`SYSTEM DROP ... FORMAT CACHE`](/sql-reference/statements/system.md/#system-drop-schema-format) 语句。 - -```sql -SYSTEM DROP FORMAT SCHEMA CACHE FOR Protobuf -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md deleted file mode 100644 index 38d600f4ea8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufList.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -alias: [] -description: 'ProtobufList 格式文档' -input_format: true -keywords: ['ProtobufList'] -output_format: true -slug: /interfaces/formats/ProtobufList -title: 'ProtobufList' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -`ProtobufList` 格式与 [`Protobuf`](./Protobuf.md) 格式类似,但每一行表示为一系列子消息,这些子消息包含在一个名称固定为 "Envelope" 的消息中。 - - - -## 示例用法 {#example-usage} - -例如: - -```sql -SELECT * FROM test.table FORMAT ProtobufList SETTINGS format_schema = 'schemafile:MessageType' -``` - -```bash -cat protobuflist_messages.bin | clickhouse-client --query "INSERT INTO test.table FORMAT ProtobufList SETTINGS format_schema='schemafile:MessageType'" -``` - -其中 `schemafile.proto` 文件内容如下: - -```capnp title="schemafile.proto" -syntax = "proto3"; -message Envelope { - message MessageType { - string name = 1; - string surname = 2; - uint32 birthDate = 3; - repeated string phoneNumbers = 4; - }; - MessageType row = 1; -}; -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md deleted file mode 100644 index 4ae8754a316..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Protobuf/ProtobufSingle.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -alias: [] -description: 'ProtobufSingle 格式文档' -input_format: true -keywords: ['ProtobufSingle'] -output_format: true -slug: /interfaces/formats/ProtobufSingle -title: 'ProtobufSingle' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -`ProtobufSingle` 格式与 [`Protobuf`](./Protobuf.md) 格式相同,但用于在没有长度分隔符的情况下存储和解析单个 Protobuf 消息。 - - - -## 示例用法 {#example-usage} - - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md deleted file mode 100644 index 92f30f2f18a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RawBLOB.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: 'RawBLOB 格式文档' -keywords: ['RawBLOB'] -slug: /interfaces/formats/RawBLOB -title: 'RawBLOB' -doc_type: 'reference' ---- - - - -## 描述 {#description} - -`RawBLOB` 格式会将所有输入数据读取为单个值。它只能用于解析仅包含一个 [`String`](/sql-reference/data-types/string.md) 类型或类似类型字段的表。 -结果以二进制格式输出,没有分隔符也没有转义。如果输出了多个值,该格式将变得不明确,并且无法将数据读回。 - -### 原始格式对比 {#raw-formats-comparison} - -下面是 `RawBLOB` 与 [`TabSeparatedRaw`](./TabSeparated/TabSeparatedRaw.md) 格式的比较。 - -`RawBLOB`: - -* 数据以二进制格式输出,不进行转义; -* 值之间没有分隔符; -* 每个值的末尾没有换行符。 - -`TabSeparatedRaw`: - -* 数据输出时不进行转义; -* 每一行中包含以制表符分隔的值; -* 每一行最后一个值之后有换行符。 - -下面是 `RawBLOB` 和 [RowBinary](./RowBinary/RowBinary.md) 格式的比较。 - -`RawBLOB`: - -* String 字段输出时不会带有长度前缀。 - -`RowBinary`: - -* String 字段表示为 varint 格式的长度(无符号 [LEB128](https://en.wikipedia.org/wiki/LEB128)),后跟字符串的字节。 - -当向 `RawBLOB` 输入传递空数据时,ClickHouse 会抛出异常: - -```text -代码:108. DB::Exception:无可插入的数据 -``` - - -## 使用示例 {#example-usage} - -```bash title="Query" -$ clickhouse-client --query "CREATE TABLE {some_table} (a String) ENGINE = Memory;" -$ cat {filename} | clickhouse-client --query="INSERT INTO {some_table} FORMAT RawBLOB" -$ clickhouse-client --query "SELECT * FROM {some_table} FORMAT RawBLOB" | md5sum -``` - -```text title="Response" -f9725a22f9191e064120d718e26862a9 - -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md deleted file mode 100644 index 969658bd8d7..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Regexp.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -alias: [] -description: 'Regexp 格式文档' -input_format: true -keywords: ['Regexp'] -output_format: false -slug: /interfaces/formats/Regexp -title: 'Regexp' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 描述 {#description} - -`Regex` 格式会根据提供的正则表达式,对导入数据的每一行进行解析。 - -**用法** - -来自 [format_regexp](/operations/settings/settings-formats.md/#format_regexp) 设置的正则表达式会应用到导入数据的每一行。正则表达式中的子模式数量必须等于导入数据集中列的数量。 - -导入数据的各行必须使用换行符 `'\n'` 或 DOS 风格的换行符 `"\r\n"` 分隔。 - -每个匹配到的子模式内容会根据 [format_regexp_escaping_rule](/operations/settings/settings-formats.md/#format_regexp_escaping_rule) 设置,使用对应数据类型的解析方法进行解析。 - -如果正则表达式未能匹配某一行,并且 [format_regexp_skip_unmatched](/operations/settings/settings-formats.md/#format_regexp_escaping_rule) 被设置为 1,则该行会被静默跳过。否则会抛出异常。 - - - -## 示例用法 {#example-usage} - -假设有文件 `data.tsv`: - -```text title="data.tsv" -id: 1 array: [1,2,3] string: str1 date: 2020-01-01 -id: 2 array: [1,2,3] string: str2 date: 2020-01-02 -id: 3 array: [1,2,3] string: str3 date: 2020-01-03 -``` - -以及 `imp_regex_table` 表: - -```sql -CREATE TABLE imp_regex_table (id UInt32, array Array(UInt32), string String, date Date) ENGINE = Memory; -``` - -我们将使用以下查询,把前面提到的文件中的数据插入到上述表中: - -```bash -$ cat data.tsv | clickhouse-client --query "INSERT INTO imp_regex_table SETTINGS format_regexp='id: (.+?) array: (.+?) string: (.+?) date: (.+?)', format_regexp_escaping_rule='Escaped', format_regexp_skip_unmatched=0 FORMAT Regexp;" -``` - -现在我们可以对表执行 `SELECT` 查询,查看 `Regex` 格式是如何解析文件中的数据的: - -```sql title="Query" -SELECT * FROM imp_regex_table; -``` - -```text title="Response" -┌─id─┬─array───┬─string─┬───────date─┐ -│ 1 │ [1,2,3] │ str1 │ 2020-01-01 │ -│ 2 │ [1,2,3] │ str2 │ 2020-01-02 │ -│ 3 │ [1,2,3] │ str3 │ 2020-01-03 │ -└────┴─────────┴────────┴────────────┘ -``` - - -## 格式设置 {#format-settings} - -在使用 `Regexp` 格式时,可以使用以下设置: - -- `format_regexp` — [String](/sql-reference/data-types/string.md)。包含以 [re2](https://github.com/google/re2/wiki/Syntax) 语法编写的正则表达式。 -- `format_regexp_escaping_rule` — [String](/sql-reference/data-types/string.md)。支持以下转义规则: - - - CSV(类似于 [CSV](/interfaces/formats/CSV)) - - JSON(类似于 [JSONEachRow](/interfaces/formats/JSONEachRow)) - - Escaped(类似于 [TSV](/interfaces/formats/TabSeparated)) - - Quoted(类似于 [Values](/interfaces/formats/Values)) - - Raw(按整体提取子模式,不应用任何转义规则,类似于 [TSVRaw](/interfaces/formats/TabSeparated)) - -- `format_regexp_skip_unmatched` — [UInt8](/sql-reference/data-types/int-uint.md)。用于指定当 `format_regexp` 表达式与导入数据不匹配时是否抛出异常。可设置为 `0` 或 `1`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md deleted file mode 100644 index 1dd84417cf2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinary.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -alias: [] -description: 'RowBinary 格式文档' -input_format: true -keywords: ['RowBinary'] -output_format: true -slug: /interfaces/formats/RowBinary -title: 'RowBinary' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -`RowBinary` 格式按行以二进制形式解析数据。 -行和数值按顺序连续列出,没有分隔符。 -由于数据以二进制形式表示,`FORMAT RowBinary` 之后的分隔符必须严格按如下方式指定: - -- 任意数量的空白字符: - - `' '`(空格 - 代码 `0x20`) - - `'\t'`(制表符 - 代码 `0x09`) - - `'\f'`(换页符 - 代码 `0x0C`) -- 后面紧跟且只能跟一个换行序列: - - Windows 风格 `"\r\n"` - - 或 Unix 风格 `'\n'` -- 紧接着就是二进制数据。 - -:::note -由于是按行存储,该格式比 [Native](../Native.md) 格式效率更低。 -::: - -对于以下数据类型,需要注意: - -- [整数](../../../sql-reference/data-types/int-uint.md) 使用定长小端表示。例如,`UInt64` 使用 8 字节。 -- [DateTime](../../../sql-reference/data-types/datetime.md) 表示为 `UInt32`,其值为 Unix 时间戳。 -- [Date](../../../sql-reference/data-types/date.md) 表示为 `UInt16`,其值为自 `1970-01-01` 起的天数。 -- [String](../../../sql-reference/data-types/string.md) 表示为可变宽度整数(varint,使用无符号 [`LEB128`](https://en.wikipedia.org/wiki/LEB128) 编码),后面跟字符串的字节序列。 -- [FixedString](../../../sql-reference/data-types/fixedstring.md) 表示为简单的字节序列。 -- [Array](../../../sql-reference/data-types/array.md) 表示为可变宽度整数(varint,使用无符号 [LEB128](https://en.wikipedia.org/wiki/LEB128) 编码),后面跟数组的各个元素。 - -对于 [NULL](/sql-reference/syntax#null) 支持,每个 [Nullable](/sql-reference/data-types/nullable.md) 值前会额外增加一个包含 `1` 或 `0` 的字节。 -- 如果为 `1`,则该值为 `NULL`,并且这个字节本身被解释为一个独立的值。 -- 如果为 `0`,则该字节后面的值不是 `NULL`。 - -关于 `RowBinary` 格式与 `RawBlob` 格式的对比,请参阅:[原始格式比较](../RawBLOB.md/#raw-formats-comparison) - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md deleted file mode 100644 index de737c37a22..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithDefaults.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -alias: [] -description: 'RowBinaryWithDefaults 格式文档' -input_format: true -keywords: ['RowBinaryWithDefaults'] -output_format: false -slug: /interfaces/formats/RowBinaryWithDefaults -title: 'RowBinaryWithDefaults' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✗ | | - - -## 描述 {#description} - -与 [`RowBinary`](./RowBinary.md) 格式类似,但在每一列前多了一个字节,用于表示是否使用默认值。 - - - -## 使用示例 {#example-usage} - -示例: - -```sql title="Query" -SELECT * FROM FORMAT('RowBinaryWithDefaults', 'x UInt32 default 42, y UInt32', x'010001000000') -``` - -```response title="Response" -┌──x─┬─y─┐ -│ 42 │ 1 │ -└────┴───┘ -``` - -* 对于列 `x`,只有一个字节 `01`,它表示应使用默认值,在此字节之后不再提供其他数据。 -* 对于列 `y`,数据以字节 `00` 开头,它表示该列包含实际值,应从后续数据 `01000000` 中读取。 - - -## 格式设置 {#format-settings} - - diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md deleted file mode 100644 index 423a1030aa8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNames.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: 'RowBinaryWithNames 格式文档' -input_format: true -keywords: ['RowBinaryWithNames'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNames -title: 'RowBinaryWithNames' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -与 [`RowBinary`](./RowBinary.md) 格式类似,但在前面增加了头部: - -- 使用 [`LEB128`](https://en.wikipedia.org/wiki/LEB128) 编码的列数(N)。 -- N 个 `String`,用于指定列名。 - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - - -:::note -- 当将 [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1` 时,将按列名把输入数据中的列映射到表中的列。 -- 当将 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1` 时,具有未知名称的列会被跳过;否则,第一行会被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md deleted file mode 100644 index 6e6260c7107..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/RowBinaryWithNamesAndTypes.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -alias: [] -description: 'RowBinaryWithNamesAndTypes 格式文档' -input_format: true -keywords: ['RowBinaryWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/RowBinaryWithNamesAndTypes -title: 'RowBinaryWithNamesAndTypes' -doc_type: 'reference' ---- - -import RowBinaryFormatSettings from './_snippets/common-row-binary-format-settings.md' - -| 输入 | 输出 | 别名 | -| -- | -- | -- | -| ✔ | ✔ | | - - -## 描述 {#description} - -与 [RowBinary](./RowBinary.md) 格式类似,但增加了头部: - -- 使用 [`LEB128`](https://en.wikipedia.org/wiki/LEB128) 编码的列数(N)。 -- N 个 `String`,指定列名。 -- N 个 `String`,指定列类型。 - - - -## 用法示例 {#example-usage} - - - -## 格式设置 {#format-settings} - - - -:::note -如果将 [`input_format_with_names_use_header`](/operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 1, -则输入数据中的列会按照名称映射到表中的列;如果将 [input_format_skip_unknown_fields](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 1,则名称未知的列将被忽略。 -否则,第一行将被跳过。 -如果将 [`input_format_with_types_use_header`](/operations/settings/settings-formats.md/#input_format_with_types_use_header) 设置为 `1`, -则输入数据中的类型会与表中相应列的类型进行比较。否则,第二行将被跳过。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md deleted file mode 100644 index 61c3ccdee1e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/RowBinary/_snippets/common-row-binary-format-settings.md +++ /dev/null @@ -1,11 +0,0 @@ -{/* 注意:此代码片段将在所有导入它的文件中被重复使用 */ } - -以下设置适用于所有 `RowBinary` 类型的格式。 - -| Setting | Description | Default | -| -------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| [`format_binary_max_string_size`](/operations/settings/settings-formats.md/#format_binary_max_string_size) | RowBinary 格式中 `String` 类型允许的最大大小。 | `1GiB` | -| [`output_format_binary_encode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | 允许在 [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md) 输出格式的头部使用 [`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) 编码并写入类型信息,而不是使用带有类型名称的字符串。 | `false` | -| [`input_format_binary_decode_types_in_binary_format`](/operations/settings/formats#input_format_binary_decode_types_in_binary_format) | 允许在 [`RowBinaryWithNamesAndTypes`](../RowBinaryWithNamesAndTypes.md) 输入格式的头部使用 [`binary encoding`](/sql-reference/data-types/data-types-binary-encoding.md) 解码并读取类型信息,而不是使用带有类型名称的字符串。 | `false` | -| [`output_format_binary_write_json_as_string`](/operations/settings/settings-formats.md/#output_format_binary_write_json_as_string) | 允许在 [`RowBinary`](../RowBinary.md) 输出格式中,将 [`JSON`](/sql-reference/data-types/newjson.md) 数据类型的值写为 `JSON` [String](/sql-reference/data-types/string.md) 值。 | `false` | -| [`input_format_binary_read_json_as_string`](/operations/settings/settings-formats.md/#input_format_binary_read_json_as_string) | 允许在 [`RowBinary`](../RowBinary.md) 输入格式中,将 [`JSON`](/sql-reference/data-types/newjson.md) 数据类型的值读取为 `JSON` [String](/sql-reference/data-types/string.md) 值。 | `false` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md deleted file mode 100644 index 74330dce043..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/SQLInsert.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -alias: [] -description: 'SQLInsert 格式文档' -input_format: false -keywords: ['SQLInsert'] -output_format: true -slug: /interfaces/formats/SQLInsert -title: 'SQLInsert' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -以一系列 `INSERT INTO table (columns...) VALUES (...), (...) ...;` 语句的形式输出数据。 - - - -## 使用示例 {#example-usage} - -示例: - -```sql -SELECT number AS x, number + 1 AS y, 'Hello' AS z FROM numbers(10) FORMAT SQLInsert SETTINGS output_format_sql_insert_max_batch_size = 2 -``` - -```sql -INSERT INTO table (x, y, z) VALUES (0, 1, '你好'), (1, 2, '你好'); -INSERT INTO table (x, y, z) VALUES (2, 3, '你好'), (3, 4, '你好'); -INSERT INTO table (x, y, z) VALUES (4, 5, '你好'), (5, 6, '你好'); -INSERT INTO table (x, y, z) VALUES (6, 7, '你好'), (7, 8, '你好'); -INSERT INTO table (x, y, z) VALUES (8, 9, '你好'), (9, 10, '你好'); -``` - -可以使用 [MySQLDump](../formats/MySQLDump.md) 输入格式来读取此格式输出的数据。 - - -## 格式设置 {#format-settings} - -| 设置 | 描述 | 默认值 | -|----------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------|-----------| -| [`output_format_sql_insert_max_batch_size`](../../operations/settings/settings-formats.md/#output_format_sql_insert_max_batch_size) | 单个 INSERT 语句中的最大行数。 | `65505` | -| [`output_format_sql_insert_table_name`](../../operations/settings/settings-formats.md/#output_format_sql_insert_table_name) | 输出 INSERT 查询中使用的表名。 | `'table'` | -| [`output_format_sql_insert_include_column_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_include_column_names) | 在 INSERT 查询中包含列名。 | `true` | -| [`output_format_sql_insert_use_replace`](../../operations/settings/settings-formats.md/#output_format_sql_insert_use_replace) | 使用 REPLACE 语句而不是 INSERT。 | `false` | -| [`output_format_sql_insert_quote_names`](../../operations/settings/settings-formats.md/#output_format_sql_insert_quote_names) | 使用 "\`" 字符为列名加引号。 | `true` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md deleted file mode 100644 index 3e902084de2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TSKV.md +++ /dev/null @@ -1,127 +0,0 @@ ---- -alias: [] -description: 'TSKV 格式文档' -input_format: true -keywords: ['TSKV'] -output_format: true -slug: /interfaces/formats/TSKV -title: 'TSKV' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -类似于 [`TabSeparated`](./TabSeparated.md) 格式,但以 `name=value` 格式输出值。 -名称的转义方式与 [`TabSeparated`](./TabSeparated.md) 格式相同,且 `=` 符号也会被转义。 - -```text -SearchPhrase= count()=8267016 -SearchPhrase=卫生间室内设计 count()=2166 -SearchPhrase=clickhouse count()=1655 -SearchPhrase=2014年春季时尚 count()=1549 -SearchPhrase=自由形式照片 count()=1480 -SearchPhrase=angelina jolie count()=1245 -SearchPhrase=omsk count()=1112 -SearchPhrase=各种犬种的照片 count()=1091 -SearchPhrase=窗帘设计 count()=1064 -SearchPhrase=baku count()=1000 -``` - -```sql title="Query" -SELECT * FROM t_null FORMAT TSKV -``` - -```text title="Response" -x=1 y=\N -``` - -:::note -当存在大量列且每列数据量较小时,此格式效率低下,一般没有使用它的理由。 -不过,就效率而言,它并不比 [`JSONEachRow`](../JSON/JSONEachRow.md) 格式更差。 -::: - -在解析时,不同列取值的顺序可以是任意的。 -允许省略某些值,此时这些值将被视为其默认值。 -在这种情况下,默认值为零和空字符串。 -不能将本可以在表中指定的复杂值用作默认值。 - -在解析过程中,允许添加一个额外字段 `tskv`,该字段不带等号或值。该字段会被忽略。 - -在导入时,如果将设置 [`input_format_skip_unknown_fields`](/operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设为 `1`,则具有未知名称的列将被跳过。 - -[NULL](/sql-reference/syntax.md) 会被格式化为 `\N`。 - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tskv` 的 tskv 文件: - -```tsv -date=2022-04-30 season=2021 home_team=萨顿联队 away_team=布拉德福德城 home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=斯温登城 away_team=巴罗 home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=特兰米尔流浪者 away_team=奥尔德姆竞技 home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=波特拜尔 away_team=纽波特郡 home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=索尔福德城 away_team=曼斯菲尔德城 home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=巴罗 away_team=诺桑普顿镇 home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=布拉德福德城 away_team=卡莱尔联 home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=布里斯托尔流浪者 away_team=斯坎索普联 home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=埃克塞特城 away_team=波特拜尔 home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=哈罗盖特镇 A.F.C. away_team=萨顿联队 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=哈特尔浦联 away_team=科尔切斯特联 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=莱顿东方 away_team=特兰米尔流浪者 home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=曼斯菲尔德城 away_team=森林绿流浪者 home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=纽波特郡 away_team=罗奇代尔 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=奥尔德姆竞技 away_team=克劳利城 home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=史蒂文尼奇自治市 away_team=索尔福德城 home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=沃尔索尔 away_team=斯温登城 home_team_goals=0 away_team_goals=3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tskv' FORMAT TSKV; -``` - -### 读取数据 {#reading-data} - -以 `TSKV` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT TSKV -``` - -输出将采用制表符分隔格式,并包含两行表头:第一行为列名,第二行为列类型: - - -```tsv -date=2022-04-30 season=2021 home_team=萨顿联队 away_team=布拉德福德城 home_team_goals=1 away_team_goals=4 -date=2022-04-30 season=2021 home_team=斯温登镇 away_team=巴罗 home_team_goals=2 away_team_goals=1 -date=2022-04-30 season=2021 home_team=特兰米尔流浪者 away_team=奥尔德姆竞技 home_team_goals=2 away_team_goals=0 -date=2022-05-02 season=2021 home_team=波特维尔 away_team=纽波特郡 home_team_goals=1 away_team_goals=2 -date=2022-05-02 season=2021 home_team=索尔福德城 away_team=曼斯菲尔德镇 home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=巴罗 away_team=北安普敦镇 home_team_goals=1 away_team_goals=3 -date=2022-05-07 season=2021 home_team=布拉德福德城 away_team=卡莱尔联 home_team_goals=2 away_team_goals=0 -date=2022-05-07 season=2021 home_team=布里斯托尔流浪者 away_team=斯肯索普联 home_team_goals=7 away_team_goals=0 -date=2022-05-07 season=2021 home_team=埃克塞特城 away_team=波特维尔 home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=哈罗盖特镇 A.F.C. away_team=萨顿联队 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=哈特尔浦联 away_team=科尔切斯特联 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=莱顿东方 away_team=特兰米尔流浪者 home_team_goals=0 away_team_goals=1 -date=2022-05-07 season=2021 home_team=曼斯菲尔德镇 away_team=森林绿流浪者 home_team_goals=2 away_team_goals=2 -date=2022-05-07 season=2021 home_team=纽波特郡 away_team=罗奇代尔 home_team_goals=0 away_team_goals=2 -date=2022-05-07 season=2021 home_team=奥尔德姆竞技 away_team=克劳利镇 home_team_goals=3 away_team_goals=3 -date=2022-05-07 season=2021 home_team=斯蒂夫尼奇自治市 away_team=索尔福德城 home_team_goals=4 away_team_goals=2 -date=2022-05-07 season=2021 home_team=沃尔索尔 away_team=斯温登镇 home_team_goals=0 away_team_goals=3 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md deleted file mode 100644 index bbe2bbc576f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparated.md +++ /dev/null @@ -1,190 +0,0 @@ ---- -alias: ['TSV'] -description: 'TSV 格式文档' -input_format: true -keywords: ['TabSeparated', 'TSV'] -output_format: true -slug: /interfaces/formats/TabSeparated -title: 'TabSeparated' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|--------| -| ✔ | ✔ | `TSV` | - - - -## 描述 {#description} - -在 TabSeparated 格式中,数据按行写入。每一行包含由制表符分隔的值。每个值后面都跟着一个制表符,除了该行的最后一个值,它后面跟的是换行符。在所有场景下都假定使用 Unix 风格换行符。最后一行的末尾也必须包含一个换行符。各个值以文本格式写入,不带引号,且特殊字符会被转义。 - -这种格式也称为 `TSV`。 - -`TabSeparated` 格式适合通过自定义程序和脚本处理数据。它在 HTTP 接口以及命令行客户端的批处理模式中是默认使用的格式。此格式也便于在不同数据库管理系统(DBMS)之间传输数据。例如,你可以从 MySQL 获取转储并上传到 ClickHouse,反之亦然。 - -`TabSeparated` 格式支持输出汇总值(使用 WITH TOTALS 时)和极值(当将 `'extremes'` 设置为 1 时)。在这些情况下,汇总值和极值会输出在主数据之后。主结果集、汇总值和极值之间通过一个空行分隔。示例: - -```sql -SELECT EventDate, count() AS c FROM test.hits GROUP BY EventDate WITH TOTALS ORDER BY EventDate FORMAT TabSeparated - -2014-03-17 1406958 -2014-03-18 1383658 -2014-03-19 1405797 -2014-03-20 1353623 -2014-03-21 1245779 -2014-03-22 1031592 -2014-03-23 1046491 - -1970-01-01 8873898 - -2014-03-17 1031592 -2014-03-23 1406958 -``` - - -## 数据格式化 {#tabseparated-data-formatting} - -整数以十进制形式书写。数字开头可以包含额外的“+”字符(解析时会被忽略,格式化输出时也不会被输出)。非负数不能包含负号。在读取时,允许将空字符串解析为零,或者(对于有符号类型)将仅包含一个减号的字符串解析为零。超出对应数据类型范围的数字可能会被解析为其他数值,而不会产生错误信息。 - -浮点数以十进制形式书写,小数点使用点号作为分隔符。支持指数表示,以及 `inf`、`+inf`、`-inf` 和 `nan`。浮点数的表示形式可以以小数点开头或结尾。 -在格式化过程中,浮点数可能会丢失精度。 -在解析过程中,不严格要求读取为最接近的机器可表示数值。 - -日期以 `YYYY-MM-DD` 格式书写,并以同一格式解析,但分隔符可以是任意字符。 -带时间的日期以 `YYYY-MM-DD hh:mm:ss` 格式书写,并以同一格式解析,但分隔符可以是任意字符。 -上述操作均在客户端或服务端启动时所使用的系统时区中进行(取决于哪一方在进行数据格式化)。对于带时间的日期,不指定夏令时。因此,如果转储中包含处于夏令时段的时间,则该转储与数据并非一一对应,解析时会在两个时间中选择其一。 -在读取操作中,不正确的日期和带时间的日期可以按自然溢出的方式解析,或者解析为空日期和时间,而不会产生错误信息。 - -作为例外,如果带时间的日期以 Unix 时间戳格式表示,并且恰好由 10 位十进制数字组成,也支持这种解析方式。解析结果与时区无关。格式 `YYYY-MM-DD hh:mm:ss` 和 `NNNNNNNNNN` 会自动区分。 - -字符串输出时会对特殊字符进行反斜杠转义。输出时使用以下转义序列:`\b`、`\f`、`\r`、`\n`、`\t`、`\0`、`\'`、`\\`。解析时还支持序列 `\a`、`\v` 和 `\xHH`(十六进制转义序列),以及任意 `\c` 序列,其中 `c` 为任意字符(这些序列会被转换为 `c`)。因此,读取数据时支持多种格式,其中换行符可以写作 `\n`、写作反斜杠加换行,或直接写为换行符。例如,字符串 `Hello world` 如果在单词之间使用换行符而不是空格,则可以用以下任意变体进行解析: - -```text -你好\n世界 - -你好\ -世界 -``` - -之所以支持第二种变体,是因为 MySQL 在写入制表符分隔的转储文件时会使用它。 - -在以 TabSeparated 格式传递数据时,必须转义的最小字符集为:制表符、换行符(LF)和反斜杠。 - -只有一小部分字符会被转义。你很容易遇到在终端输出时显示会被破坏的字符串值。 - -数组以方括号包裹的逗号分隔值列表的形式写出。数组中的数值元素按常规方式格式化。`Date` 和 `DateTime` 类型用单引号包裹。字符串也用单引号包裹,并使用与上文相同的转义规则。 - -[NULL](/sql-reference/syntax.md) 的格式取决于设置 [format_tsv_null_representation](/operations/settings/settings-formats.md/#format_tsv_null_representation)(默认值为 `\N`)。 - -在输入数据中,ENUM 值可以用名称或 id 表示。我们会首先尝试将输入值与 ENUM 名称匹配。如果失败且输入值是数字,则尝试将该数字与 ENUM id 匹配。 -如果输入数据只包含 ENUM id,建议启用设置 [input_format_tsv_enum_as_number](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) 以优化 ENUM 解析。 - -[Nested](/sql-reference/data-types/nested-data-structures/index.md) 结构的每个元素都表示为一个数组。 - -例如: - -```sql -CREATE TABLE nestedt -( - `id` UInt8, - `aux` Nested( - a UInt8, - b String - ) -) -ENGINE = TinyLog -``` - -```sql -INSERT INTO nestedt VALUES ( 1, [1], ['a']) -``` - -```sql -SELECT * FROM nestedt FORMAT TSV -``` - -```response -1 [1] ['a'] -``` - - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparated; -``` - -### 读取数据 {#reading-data} - -以 `TabSeparated` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparated -``` - -输出将为制表符分隔的格式: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式设置 {#format-settings} - -| Setting | Description | Default | -|------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| -| [`format_tsv_null_representation`](/operations/settings/settings-formats.md/#format_tsv_null_representation) | 自定义 TSV 格式中 NULL 的表示形式。 | `\N` | -| [`input_format_tsv_empty_as_default`](/operations/settings/settings-formats.md/#input_format_tsv_empty_as_default) | 将 TSV 输入中的空字段视为默认值。对于复杂的默认表达式,还必须启用 [input_format_defaults_for_omitted_fields](/operations/settings/settings-formats.md/#input_format_defaults_for_omitted_fields)。 | `false` | -| [`input_format_tsv_enum_as_number`](/operations/settings/settings-formats.md/#input_format_tsv_enum_as_number) | 将 TSV 格式中插入的枚举值按枚举索引处理。 | `false` | -| [`input_format_tsv_use_best_effort_in_schema_inference`](/operations/settings/settings-formats.md/#input_format_tsv_use_best_effort_in_schema_inference) | 使用若干调整和启发式规则来推断 TSV 格式中的 schema。若禁用,所有字段都将被推断为 String。 | `true` | -| [`output_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#output_format_tsv_crlf_end_of_line) | 若设置为 true,TSV 输出格式中的行尾将使用 `\r\n` 而不是 `\n`。 | `false` | -| [`input_format_tsv_crlf_end_of_line`](/operations/settings/settings-formats.md/#input_format_tsv_crlf_end_of_line) | 若设置为 true,TSV 输入格式中的行尾将使用 `\r\n` 而不是 `\n`。 | `false` | -| [`input_format_tsv_skip_first_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_first_lines) | 跳过数据开头指定数量的行。 | `0` | -| [`input_format_tsv_detect_header`](/operations/settings/settings-formats.md/#input_format_tsv_detect_header) | 在 TSV 格式中自动检测包含名称和类型的表头行。 | `true` | -| [`input_format_tsv_skip_trailing_empty_lines`](/operations/settings/settings-formats.md/#input_format_tsv_skip_trailing_empty_lines) | 跳过数据末尾的空行。 | `false` | -| [`input_format_tsv_allow_variable_number_of_columns`](/operations/settings/settings-formats.md/#input_format_tsv_allow_variable_number_of_columns) | 允许 TSV 格式中列数可变,忽略多余列,并对缺失列使用默认值。 | `false` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md deleted file mode 100644 index 0b2246d103b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRaw.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -alias: ['TSVRaw', 'Raw'] -description: 'TabSeparatedRaw 格式文档' -input_format: true -keywords: ['TabSeparatedRaw'] -output_format: true -slug: /interfaces/formats/TabSeparatedRaw -title: 'TabSeparatedRaw' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|------|------|-------------------| -| ✔ | ✔ | `TSVRaw`, `Raw` | - - - -## 描述 {#description} - -本格式与 [`TabSeparated`](/interfaces/formats/TabSeparated) 格式的不同之处在于,写入行时不会进行转义。 - -:::note -使用此格式进行解析时,每个字段中不允许出现制表符或换行符。 -::: - -关于 `TabSeparatedRaw` 格式与 `RawBlob` 格式的比较,请参见:[原始格式比较](../RawBLOB.md/#raw-formats-comparison) - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRaw; -``` - -### 读取数据 {#reading-data} - -使用 `TabSeparatedRaw` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRaw -``` - -输出将采用制表符分隔的格式: - -```tsv -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式配置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md deleted file mode 100644 index cc0b7f54fc8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNames.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -alias: ['TSVRawWithNames', 'RawWithNames'] -description: 'TabSeparatedRawWithNames 格式文档' -input_format: true -keywords: ['TabSeparatedRawWithNames', 'TSVRawWithNames', 'RawWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNames -title: 'TabSeparatedRawWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-----------------------------------| -| ✔ | ✔ | `TSVRawWithNames`, `RawWithNames` | - - - -## 描述 {#description} - -与 [`TabSeparatedWithNames`](./TabSeparatedWithNames.md) 格式不同, -该格式在写入行数据时不会对内容进行转义。 - -:::note -使用此格式进行解析时,每个字段中不允许包含制表符或换行符。 -::: - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNames; -``` - -### 读取数据 {#reading-data} - -使用 `TabSeparatedRawWithNames` 格式来读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNames -``` - -输出将采用制表符分隔的格式,且只有一行表头: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md deleted file mode 100644 index 08555fd09ab..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedRawWithNamesAndTypes.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -alias: ['TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -description: 'TabSeparatedRawWithNamesAndTypes 格式说明文档' -input_format: true -keywords: ['TabSeparatedRawWithNamesAndTypes', 'TSVRawWithNamesAndTypes', 'RawWithNamesAndTypes'] -output_format: true -slug: /interfaces/formats/TabSeparatedRawWithNamesAndTypes -title: 'TabSeparatedRawWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|---------------------------------------------------| -| ✔ | ✔ | `TSVRawWithNamesAndNames`, `RawWithNamesAndNames` | - - - -## 描述 {#description} - -与 [`TabSeparatedWithNamesAndTypes`](./TabSeparatedWithNamesAndTypes.md) 格式不同, -该格式在写入行数据时不会进行转义。 - -:::note -使用此格式进行解析时,每个字段中不允许包含制表符或换行符。 -::: - - - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedRawWithNamesAndTypes; -``` - -### 读取数据 {#reading-data} - -以 `TabSeparatedRawWithNamesAndTypes` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedRawWithNamesAndTypes -``` - -输出将采用制表符分隔的格式,并包含两行表头,分别用于列名和列类型: - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md deleted file mode 100644 index 170c72f6537..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNames.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -alias: ['TSVWithNames'] -description: 'TabSeparatedWithNames 格式文档' -input_format: true -keywords: ['TabSeparatedWithNames'] -output_format: true -slug: /interfaces/formats/TabSeparatedWithNames -title: 'TabSeparatedWithNames' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|--------------------------------| -| ✔ | ✔ | `TSVWithNames`, `RawWithNames` | - - - -## 描述 {#description} - -与 [`TabSeparated`](./TabSeparated.md) 格式的区别在于,第一行写入了列名。 - -在解析时,预期第一行包含列名。可以使用列名来确定它们的位置,并检查其是否正确。 - -:::note -如果将设置项 [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则会根据名称将输入数据中的列映射到表中的列;如果将设置项 [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则会跳过名称未知的列。 -否则,将跳过第一行。 -::: - - - -## 示例用法 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNames; -``` - -### 读取数据 {#reading-data} - -以 `TabSeparatedWithNames` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNames -``` - -输出将为制表符分隔格式: - -```tsv -date season home_team away_team home_team_goals away_team_goals -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md deleted file mode 100644 index 48d54050d91..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/TabSeparated/TabSeparatedWithNamesAndTypes.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: 'TabSeparatedWithNamesAndTypes 格式文档' -keywords: ['TabSeparatedWithNamesAndTypes'] -slug: /interfaces/formats/TabSeparatedWithNamesAndTypes -title: 'TabSeparatedWithNamesAndTypes' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|------------------------------------------------| -| ✔ | ✔ | `TSVWithNamesAndTypes`, `RawWithNamesAndTypes` | - - - -## 描述 {#description} - -与 [`TabSeparated`](./TabSeparated.md) 格式的区别在于:第一行写入列名,第二行写入列类型。 - -:::note -- 如果将 [`input_format_with_names_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_names_use_header) 设置为 `1`, -则会根据列名将输入数据中的列映射到表中的列;如果将 [`input_format_skip_unknown_fields`](../../../operations/settings/settings-formats.md/#input_format_skip_unknown_fields) 设置为 `1`,则名称未知的列将被跳过。 -否则,第一行将被跳过。 -- 如果将 [`input_format_with_types_use_header`](../../../operations/settings/settings-formats.md/#input_format_with_types_use_header) 设置为 `1`, -则会将输入数据中的类型与表中对应列的类型进行比较。否则,第二行将被跳过。 -::: - - - -## 使用示例 {#example-usage} - -### 插入数据 {#inserting-data} - -使用以下名为 `football.tsv` 的 TSV 文件: - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - -插入数据: - -```sql -INSERT INTO football FROM INFILE 'football.tsv' FORMAT TabSeparatedWithNamesAndTypes; -``` - -### 读取数据 {#reading-data} - -以 `TabSeparatedWithNamesAndTypes` 格式读取数据: - -```sql -SELECT * -FROM football -FORMAT TabSeparatedWithNamesAndTypes -``` - -输出将采用制表符分隔的格式,并包含两行表头,分别表示列名和列类型: - - -```tsv -date season home_team away_team home_team_goals away_team_goals -Date Int16 LowCardinality(String) LowCardinality(String) Int8 Int8 -2022-04-30 2021 Sutton United Bradford City 1 4 -2022-04-30 2021 Swindon Town Barrow 2 1 -2022-04-30 2021 Tranmere Rovers Oldham Athletic 2 0 -2022-05-02 2021 Port Vale Newport County 1 2 -2022-05-02 2021 Salford City Mansfield Town 2 2 -2022-05-07 2021 Barrow Northampton Town 1 3 -2022-05-07 2021 Bradford City Carlisle United 2 0 -2022-05-07 2021 Bristol Rovers Scunthorpe United 7 0 -2022-05-07 2021 Exeter City Port Vale 0 1 -2022-05-07 2021 Harrogate Town A.F.C. Sutton United 0 2 -2022-05-07 2021 Hartlepool United Colchester United 0 2 -2022-05-07 2021 Leyton Orient Tranmere Rovers 0 1 -2022-05-07 2021 Mansfield Town Forest Green Rovers 2 2 -2022-05-07 2021 Newport County Rochdale 0 2 -2022-05-07 2021 Oldham Athletic Crawley Town 3 3 -2022-05-07 2021 Stevenage Borough Salford City 4 2 -2022-05-07 2021 Walsall Swindon Town 0 3 -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md deleted file mode 100644 index cf553b2810f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/Template.md +++ /dev/null @@ -1,247 +0,0 @@ ---- -alias: [] -description: 'Template 格式文档' -input_format: true -keywords: ['Template'] -output_format: true -slug: /interfaces/formats/Template -title: 'Template' -doc_type: 'guide' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## 描述 {#description} - -在需要比其他标准格式更高的自定义能力时, -可以使用 `Template` 格式,让用户指定带有值占位符的自定义格式字符串, -并为数据指定转义规则。 - -它使用以下设置: - -| Setting | Description | -|----------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------| -| [`format_template_row`](#format_template_row) | 指定包含行格式字符串的文件路径。 | -| [`format_template_resultset`](#format_template_resultset) | 指定包含结果集行格式字符串的文件路径。 | -| [`format_template_rows_between_delimiter`](#format_template_rows_between_delimiter) | 指定行与行之间的分隔符,它会在每一行(除最后一行)之后被打印(或被期望读取)(默认值为 `\n`)。 | -| `format_template_row_format` | 指定[内联](#inline_specification)的行格式字符串。 | -| `format_template_resultset_format` | 指定[内联](#inline_specification)的结果集格式字符串。 | -| 某些其他格式的设置(例如使用 `JSON` 转义时的 `output_format_json_quote_64bit_integers`) | | - - - -## 设置和转义规则 {#settings-and-escaping-rules} - -### format_template_row {#format_template_row} - -`format_template_row` 设置指定包含行格式字符串的文件路径,该文件中行格式字符串的语法如下: - -```text -分隔符_1${列_1:序列化为_1}分隔符_2${列_2:序列化为_2} ... 分隔符_N -``` - -其中: - -| Part of syntax | Description | -| --------------- | ---------------------------- | -| `delimiter_i` | 值之间的分隔符(`$` 符号可以通过 `$$` 转义) | -| `column_i` | 要选择或插入其值的列名或列索引(如果为空,则会跳过该列) | -| `serializeAs_i` | 列值所使用的转义规则。 | - -支持以下转义规则: - -| Escaping Rule | Description | -| -------------------- | ------------------ | -| `CSV`, `JSON`, `XML` | 与同名格式的行为类似 | -| `Escaped` | 类似于 `TSV` | -| `Quoted` | 类似于 `Values` | -| `Raw` | 不进行转义,类似于 `TSVRaw` | -| `None` | 不使用转义规则 —— 详见下文说明 | - -:::note -如果省略转义规则,则会使用 `None`。`XML` 仅适用于输出。 -::: - -下面通过一个示例来说明。给定如下格式字符串: - -```text -搜索词组:${s:Quoted},数量:${c:Escaped},广告价格:$$${p:JSON}; -``` - -以下值将在列标记 `Search phrase:`、`, count:`、`, ad price: $` 和分隔符 `;` 之间依次被输出(如果使用 `SELECT`)或被期望作为输入提供(如果使用 `INPUT`): - -* `s`(转义规则为 `Quoted`) -* `c`(转义规则为 `Escaped`) -* `p`(转义规则为 `JSON`) - -例如: - -* 如果执行 `INSERT`,下方这一行与预期模板匹配,并会将值 `bathroom interior design`、`2166`、`$3` 写入到列 `Search phrase`、`count`、`ad price` 中。 -* 如果执行 `SELECT`,在值 `bathroom interior design`、`2166`、`$3` 已经存储在表的 `Search phrase`、`count`、`ad price` 列中的前提下,下方这一行就是输出结果。 - -```yaml -搜索词组:'bathroom interior design',数量:2166,广告价格:$3; -``` - -### format_template_rows_between_delimiter {#format_template_rows_between_delimiter} - -`format_template_rows_between_delimiter` 设置用于指定行与行之间的分隔符,该分隔符会在每一行(除了最后一行)之后输出(默认是 `\n`)。 - -### format_template_resultset {#format_template_resultset} - -`format_template_resultset` 设置用于指定包含结果集格式字符串的文件路径。 - -结果集的格式字符串与行的格式字符串具有相同的语法。 -它允许指定前缀、后缀以及打印一些附加信息的方式,并包含以下用来替代列名的占位符: - -* `data` 是以 `format_template_row` 格式表示的数据行,并由 `format_template_rows_between_delimiter` 分隔。此占位符必须是格式字符串中的第一个占位符。 -* `totals` 是以 `format_template_row` 格式表示的总计值行(使用 WITH TOTALS 时)。 -* `min` 是以 `format_template_row` 格式表示的最小值行(当 extremes 被设置为 1 时)。 -* `max` 是以 `format_template_row` 格式表示的最大值行(当 extremes 被设置为 1 时)。 -* `rows` 是输出行的总数。 -* `rows_before_limit` 是在没有 LIMIT 的情况下本应返回的最小可能行数。仅在查询包含 LIMIT 时输出。如果查询包含 GROUP BY,则 rows_before_limit_at_least 是在没有 LIMIT 时本应返回行数的精确值。 -* `time` 是请求的执行时间(秒)。 -* `rows_read` 是已读取的行数。 -* `bytes_read` 是已读取的字节数(未压缩)。 - -占位符 `data`、`totals`、`min` 和 `max` 不得指定转义规则(或者必须显式指定为 `None`)。其余占位符可以指定任意转义规则。 - -:::note -如果 `format_template_resultset` 设置为空字符串,则默认使用 `${data}`。 -::: - - -对于 INSERT 查询,如果存在前缀或后缀(见示例),该格式允许省略某些列或字段。 - -### 内联指定 {#inline_specification} - -在很多情况下,要将模板格式所需的格式配置 -(由 `format_template_row`、`format_template_resultset` 设定)部署到集群中所有节点的某个目录是非常困难的,甚至是不可能的。 -此外,某些格式可能非常简单,以至于不需要单独存放在文件中。 - -在这些情况下,可以使用 `format_template_row_format`(对应 `format_template_row`)和 `format_template_resultset_format`(对应 `format_template_resultset`)在查询中直接设置模板字符串, -而不是指定包含该模板的文件路径。 - -:::note -格式字符串和转义序列的规则与以下情况相同: -- 使用 `format_template_row_format` 时,对应 [`format_template_row`](#format_template_row)。 -- 使用 `format_template_resultset_format` 时,对应 [`format_template_resultset`](#format_template_resultset)。 -::: - - - -## 示例用法 {#example-usage} - -让我们来看两个关于如何使用 `Template` 格式的示例,首先是用于查询数据,其次是用于插入数据。 - -### 查询数据 {#selecting-data} - -```sql -SELECT SearchPhrase, count() AS c FROM test.hits GROUP BY SearchPhrase ORDER BY c DESC LIMIT 5 FORMAT Template SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format', format_template_rows_between_delimiter = '\n ' -``` - -```text title="/some/path/resultset.format" - - 搜索词组 - - - - ${data} -
搜索词组
搜索词组 次数
- - ${max} -
最大值
- 已处理 ${rows_read:XML} 行,耗时 ${time:XML} 秒 - - -``` - -```text title="/some/path/row.format" - ${0:XML} ${1:XML} -``` - -结果: - -```html - - 搜索词组 - - - - - - - - -
搜索词组
搜索词组 次数
8267016
bathroom interior design 2166
clickhouse 1655
spring 2014 fashion 1549
freeform photos 1480
- - -
最大值
8873898
- 已处理 3095973 行,耗时 0.1569913 秒 - - -``` - -### 写入数据 {#inserting-data} - -```text -某个标题 -页面浏览量:5,用户 ID:4324182021466249494,无用字段:hello,持续时间:146,符号:-1 -页面浏览量:6,用户 ID:4324182021466249494,无用字段:world,持续时间:185,符号:1 -总行数:2 -``` - -```sql -INSERT INTO UserActivity SETTINGS -format_template_resultset = '/some/path/resultset.format', format_template_row = '/some/path/row.format' -FORMAT Template -``` - -```text title="/some/path/resultset.format" -标题\n${data}\n总行数:${:CSV}\n -``` - -```text title="/some/path/row.format" -页面浏览量: ${PageViews:CSV}, 用户 ID: ${UserID:CSV}, 无用字段: ${:CSV}, 持续时间: ${Duration:CSV}, 签名: ${Sign:CSV} -``` - -占位符中的 `PageViews`、`UserID`、`Duration` 和 `Sign` 是表中的列名。行中 `Useless field` 之后的值,以及后缀中 `\nTotal rows:` 之后的值将被忽略。 -输入数据中的所有分隔符必须与指定格式字符串中的分隔符完全一致。 - -### 内联规格 {#in-line-specification} - -厌倦了手动编写和排版 Markdown 表格?在本示例中,我们将介绍如何使用 `Template` 格式和内联规格设置来完成一个简单任务——从 `system.formats` 表中 `SELECT` 出若干 ClickHouse 格式的名称,并将它们格式化为 Markdown 表格。通过使用 `Template` 格式以及 `format_template_row_format` 和 `format_template_resultset_format` 设置,即可轻松实现这一点。 - - -在之前的示例中,我们将结果集和行格式字符串放在单独的文件中,并分别通过设置 `format_template_resultset` 和 `format_template_row` 来指定这些文件的路径。这里我们会直接内联定义这些内容,因为我们的模板非常简单,只包含少量的 `|` 和 `-` 用于构造 Markdown 表格。我们将使用设置 `format_template_resultset_format` 来指定结果集模板字符串。为了生成表头,我们在 `${data}` 之前添加了 `|ClickHouse Formats|\n|---|\n`。我们使用设置 `format_template_row_format` 为每一行指定模板字符串 ``|`{0:XML}`|``。`Template` 格式会将按给定格式生成的行插入到占位符 `${data}` 中。在这个示例中我们只有一列,但如果你想添加更多列,可以在行模板字符串中添加 `{1:XML}`、`{2:XML}` 等,并根据需要选择合适的转义规则。在本示例中我们使用的是转义规则 `XML`。 - -```sql title="Query" -WITH formats AS -( - SELECT * FROM system.formats - ORDER BY rand() - LIMIT 5 -) -SELECT * FROM formats -FORMAT Template -SETTINGS - format_template_row_format='|`${0:XML}`|', - format_template_resultset_format='|ClickHouse 格式|\n|---|\n${data}\n' -``` - -看看这个!我们不必再为构建那个 Markdown 表格而手动添加那些 `|` 和 `-` 了: - -```response title="Response" -|ClickHouse 格式| -|---| -|`BSONEachRow`| -|`CustomSeparatedWithNames`| -|`Prometheus`| -|`DWARF`| -|`Avro`| -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md deleted file mode 100644 index 8791fa9b103..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Template/TemplateIgnoreSpaces.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -alias: [] -description: 'TemplateIgnoreSpaces 格式文档' -input_format: true -keywords: ['TemplateIgnoreSpaces'] -output_format: false -slug: /interfaces/formats/TemplateIgnoreSpaces -title: 'TemplateIgnoreSpaces' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✗ | | - - - -## 描述 {#description} - -与 [`Template`] 类似,但会跳过输入流中分隔符与值之间的空白字符。 -但是,如果格式字符串本身包含空白字符,则会在输入流中严格匹配这些空白字符。 -还允许指定空占位符(`${}` 或 `${:None}`),用于将某个分隔符拆分为多个部分,从而忽略这些部分之间的空格。 -这些占位符仅用于跳过空白字符。 -如果所有行中列值的顺序相同,则可以使用此格式读取 `JSON`。 - -:::note -此格式仅支持输入。 -::: - - - -## 示例用法 {#example-usage} - -以下请求可用于根据其 [JSON](/interfaces/formats/JSON) 格式的输出示例插入数据: - -```sql -INSERT INTO table_name -SETTINGS - format_template_resultset = '/some/path/resultset.format', - format_template_row = '/some/path/row.format', - format_template_rows_between_delimiter = ',' -FORMAT TemplateIgnoreSpaces -``` - -```text title="/some/path/resultset.format" -{${}"meta"${}:${:JSON},${}"data"${}:${}[${data}]${},${}"totals"${}:${:JSON},${}"extremes"${}:${:JSON},${}"rows"${}:${:JSON},${}"rows_before_limit_at_least"${}:${:JSON}${}} -``` - -```text title="/some/path/row.format" -{${}"SearchPhrase"${}:${}${phrase:JSON}${},${}"c"${}:${}${cnt:JSON}${}} -``` - - -## 格式设置 {#format-settings} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md deleted file mode 100644 index ee996ba88e7..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Values.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -alias: [] -description: 'Values 格式文档' -input_format: true -keywords: ['Values'] -output_format: true -slug: /interfaces/formats/Values -title: 'Values' -doc_type: 'guide' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✔ | ✔ | | - - - -## Description {#description} - -`Values` 格式会将每一行数据打印在一对括号中。 - -- 各行之间使用逗号分隔,最后一行后面没有逗号。 -- 括号中的各个值也使用逗号分隔。 -- 数字以十进制格式输出且不带引号。 -- 数组以方括号形式输出。 -- 字符串、日期以及带时间的日期以带引号的形式输出。 -- 转义规则和解析方式与 [TabSeparated](TabSeparated/TabSeparated.md) 格式类似。 - -在格式化输出时,不会插入多余的空格;在解析输入时,允许出现空格并会被跳过(数组值内部的空格除外,不被允许)。 -[`NULL`](/sql-reference/syntax.md) 表示为 `NULL`。 - -在以 `Values` 格式传递数据时,需要转义的最小字符集合为: -- 单引号 -- 反斜杠 - -这是在 `INSERT INTO t VALUES ...` 语句中使用的格式,但你也可以用它来格式化查询结果。 - - - -## 使用示例 {#example-usage} - - - -## 格式设置 {#format-settings} - -| 设置 | 说明 | 默认值 | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| -| [`input_format_values_interpret_expressions`](../../operations/settings/settings-formats.md/#input_format_values_interpret_expressions) | 如果字段无法被流式解析器解析,则运行 SQL 解析器并尝试将其解释为 SQL 表达式。 | `true` | -| [`input_format_values_deduce_templates_of_expressions`](../../operations/settings/settings-formats.md/#input_format_values_deduce_templates_of_expressions) | 如果字段无法被流式解析器解析,则运行 SQL 解析器,推断 SQL 表达式模板,尝试使用该模板解析所有行,然后对所有行的表达式进行解释。 | `true` | -| [`input_format_values_accurate_types_of_literals`](../../operations/settings/settings-formats.md/#input_format_values_accurate_types_of_literals) | 使用模板解析并解释表达式时,检查字面量的实际类型,以避免可能的溢出和精度问题。 | `true` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md deleted file mode 100644 index bfc9b14ec54..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/Vertical.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -alias: [] -description: 'Vertical 格式文档' -input_format: false -keywords: ['Vertical'] -output_format: true -slug: /interfaces/formats/Vertical -title: 'Vertical' -doc_type: 'reference' ---- - -| Input | Output | Alias | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -将每个值与其列名一起输出在单独的一行上。如果每行包含大量列,这种格式便于打印单行或少量行的数据。 - -请注意,[`NULL`](/sql-reference/syntax.md) 会输出为 `ᴺᵁᴸᴸ`,以便更容易区分字符串值 `NULL` 和空值。JSON 列会以美化后的格式输出,并且 `NULL` 会输出为 `null`,因为它是一个有效的 JSON 值,并且与 `"null"` 容易区分。 - - - -## 使用示例 {#example-usage} - -示例: - -```sql -SELECT * FROM t_null FORMAT Vertical -``` - -```response -第 1 行: -────── -x: 1 -y: ᴺᵁᴸᴸ -``` - -在 Vertical 输出格式中,行不会被转义: - -```sql -SELECT 'string with \'quotes\' and \t with some special \n characters' AS test FORMAT Vertical -``` - -```response -第 1 行: -────── -test: string with 'quotes' and with some special - characters -``` - -此格式仅适合用于输出查询结果,不适合用于解析(检索要插入到表中的数据)。 - - -## 格式设置 {#format-settings} diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md deleted file mode 100644 index 30d16752843..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/formats/XML.md +++ /dev/null @@ -1,102 +0,0 @@ ---- -alias: [] -description: 'XML 格式参考文档' -input_format: false -keywords: ['XML'] -output_format: true -slug: /interfaces/formats/XML -title: 'XML' -doc_type: 'reference' ---- - -| 输入 | 输出 | 别名 | -|-------|--------|-------| -| ✗ | ✔ | | - - - -## 描述 {#description} - -`XML` 格式仅适用于输出,不适用于解析。 - -如果列名不符合可接受的格式,则使用 `field` 作为元素名称。总体而言,XML 结构遵循 JSON 结构。 -与 JSON 相同,无效的 UTF-8 序列会被替换为替换字符 `�`,使输出文本只包含有效的 UTF-8 序列。 - -在字符串值中,字符 `<` 和 `&` 会分别转义为 `<` 和 `&`。 - -数组会输出为 `HelloWorld...`,元组会输出为 `HelloWorld...`。 - - - -## 使用示例 {#example-usage} - -示例: - -```xml - - - - - - SearchPhrase - String - - - count() - UInt64 - - - - - - - 8267016 - - - bathroom interior design - 2166 - - - clickhouse - 1655 - - - 2014 spring fashion - 1549 - - - freeform photos - 1480 - - - angelina jolie - 1245 - - - omsk - 1112 - - - photos of dog breeds - 1091 - - - curtain designs - 1064 - - - baku - 1000 - - - 10 - 141137 - -``` - - -## 格式设置 {#format-settings} - - - -## XML {#xml} \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/grpc.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/grpc.md deleted file mode 100644 index a0c16ddf83f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/grpc.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -description: 'ClickHouse gRPC 接口文档' -sidebar_label: 'gRPC 接口' -sidebar_position: 25 -slug: /interfaces/grpc -title: 'gRPC 接口' -doc_type: 'reference' ---- - - - -# gRPC 接口 {#grpc-interface} - - - -## 简介 {#grpc-interface-introduction} - -ClickHouse 支持 [gRPC](https://grpc.io/) 接口。gRPC 是一个开源的远程过程调用系统,使用 HTTP/2 和 [Protocol Buffers](https://en.wikipedia.org/wiki/Protocol_Buffers)。ClickHouse 中 gRPC 的实现支持: - -- SSL; -- 身份验证; -- 会话; -- 压缩; -- 通过同一通道执行并行查询; -- 取消查询; -- 获取进度和日志; -- 外部表。 - -接口规范定义在 [clickhouse_grpc.proto](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto) 中。 - - - -## gRPC 配置 {#grpc-interface-configuration} - -要使用 gRPC 接口,请在[主服务器配置文件](../operations/configuration-files.md)中设置 `grpc_port`。其他配置选项请参考下例: - -```xml -9100 - - false - - - /path/to/ssl_cert_file - /path/to/ssl_key_file - - - false - - - /path/to/ssl_ca_cert_file - - - deflate - - - medium - - - -1 - -1 - - - false - -``` - - -## 内置客户端 {#grpc-client} - -你可以使用 gRPC 支持的任意编程语言,基于提供的[规范](https://github.com/ClickHouse/ClickHouse/blob/master/src/Server/grpc_protos/clickhouse_grpc.proto)编写客户端。 -也可以使用内置的 Python 客户端。它位于仓库中的 [utils/grpc-client/clickhouse-grpc-client.py](https://github.com/ClickHouse/ClickHouse/blob/master/utils/grpc-client/clickhouse-grpc-client.py)。内置客户端需要安装 [grpcio 和 grpcio-tools](https://grpc.io/docs/languages/python/quickstart) 这两个 Python 模块。 - -客户端支持以下参数: - -* `--help` – 显示帮助信息并退出。 -* `--host HOST, -h HOST` – 服务器名。默认值:`localhost`。也可以使用 IPv4 或 IPv6 地址。 -* `--port PORT` – 要连接的端口。此端口需要在 ClickHouse 服务器配置中启用(见 `grpc_port`)。默认值:`9100`。 -* `--user USER_NAME, -u USER_NAME` – 用户名。默认值:`default`。 -* `--password PASSWORD` – 密码。默认值:空字符串。 -* `--query QUERY, -q QUERY` – 在非交互模式下要执行的查询。 -* `--database DATABASE, -d DATABASE` – 默认数据库。如果未指定,则使用服务器设置中当前的数据库(默认是 `default`)。 -* `--format OUTPUT_FORMAT, -f OUTPUT_FORMAT` – 结果输出[格式](formats.md)。交互模式下的默认值:`PrettyCompact`。 -* `--debug` – 启用显示调试信息。 - -要以交互模式运行客户端,请在调用时不要指定 `--query` 参数。 - -在批处理模式下,可以通过 `stdin` 传递查询数据。 - -**客户端使用示例** - -在以下示例中,将创建一个表并从 CSV 文件中加载数据,然后查询该表的内容。 - -```bash -./clickhouse-grpc-client.py -q "CREATE TABLE grpc_example_table (id UInt32, text String) ENGINE = MergeTree() ORDER BY id;" -echo -e "0,Input data for\n1,gRPC protocol example" > a.csv -cat a.csv | ./clickhouse-grpc-client.py -q "INSERT INTO grpc_example_table FORMAT CSV" - -./clickhouse-grpc-client.py --format PrettyCompact -q "SELECT * FROM grpc_example_table;" -``` - -结果: - -```text -┌─id─┬─text──────────────────┐ -│ 0 │ gRPC 协议示例的 │ -│ 1 │ 输入数据 │ -└────┴───────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/http.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/http.md deleted file mode 100644 index 834673aa07e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/http.md +++ /dev/null @@ -1,1221 +0,0 @@ ---- -description: 'ClickHouse HTTP 接口的文档。该接口提供 REST API,可从任意平台和编程语言访问 ClickHouse' -sidebar_label: 'HTTP 接口' -sidebar_position: 15 -slug: /interfaces/http -title: 'HTTP 接口' -doc_type: 'reference' ---- - -import PlayUI from '@site/static/images/play.png'; -import Image from '@theme/IdealImage'; - - -# HTTP 接口 {#http-interface} - - - -## 前置条件 {#prerequisites} - -要完成本文中的示例,你需要: -- 一个处于运行状态的 ClickHouse 服务器实例 -- 已安装 `curl`。在 Ubuntu 或 Debian 上,运行 `sudo apt install curl`,或参阅此[文档](https://curl.se/download.html)获取安装说明。 - - - -## 概览 {#overview} - -HTTP 接口以 REST API 的形式提供服务,让你可以在任何平台、使用任何编程语言来使用 ClickHouse。HTTP 接口相比原生接口功能更有限,但对各类编程语言有更好的支持。 - -默认情况下,`clickhouse-server` 监听以下端口: - -* 端口 8123:HTTP -* 端口 8443:可启用 HTTPS - -如果在没有任何参数的情况下发出 `GET /` 请求,会返回 200 状态码以及字符串 "Ok.": - -```bash -$ curl 'http://localhost:8123/' -Ok. -``` - -"Ok." 是在 [`http_server_default_response`](../operations/server-configuration-parameters/settings.md#http_server_default_response) 中定义的默认值,可根据需要进行修改。 - -另请参阅:[HTTP 响应码注意事项](#http_response_codes_caveats)。 - - -## Web 用户界面 {#web-ui} - -ClickHouse 提供了一个 Web 用户界面,可通过以下地址访问: - -```text -http://localhost:8123/play -``` - -Web UI 支持在查询运行期间显示进度、取消查询以及对结果进行流式输出。 -它还有一个隐藏功能,用于展示查询管线的图表和可视化。 - -成功执行查询后,会出现一个下载按钮,允许你以多种格式下载查询结果,包括 CSV、TSV、JSON、JSONLines、Parquet、Markdown,或 ClickHouse 支持的任意自定义格式。下载功能使用查询缓存高效地获取结果,而无需重新执行查询。即使 UI 只显示了多页结果中的一页,它也会下载完整的结果集。 - -Web UI 专为像你这样的专业用户设计。 - -ClickHouse Web UI 截图 - -在健康检查脚本中使用 `GET /ping` 请求。此处理程序始终返回 “Ok.”(末尾带有换行符)。从 18.12.13 版本起可用。另请参阅 `/replicas_status` 用于检查副本延迟。 - -```bash -$ curl 'http://localhost:8123/ping' -Ok. -$ curl 'http://localhost:8123/replicas_status' -Ok. -``` - - -## 通过 HTTP/HTTPS 查询 {#querying} - -要通过 HTTP/HTTPS 执行查询,有三种方式: - -* 将请求作为 URL 的 `query` 参数发送 -* 使用 POST 方法 -* 在 `query` 参数中发送查询的开头部分,其余部分通过 POST 发送 - -:::note -URL 的长度默认限制为 1 MiB,可通过 `http_max_uri_size` 设置进行修改。 -::: - -如果请求成功,会返回状态码 200,并在响应体中包含结果。 -如果发生错误,会返回状态码 500,并在响应体中包含错误描述文本。 - -使用 GET 的请求是“只读”的。这意味着对于修改数据的查询,只能使用 POST 方法。 -查询语句本身既可以放在 POST 请求体中,也可以放在 URL 参数中。下面来看一些示例。 - -在下面的示例中,使用 curl 发送查询 `SELECT 1`。请注意空格的 URL 编码:`%20`。 - -```bash title="command" -curl 'http://localhost:8123/?query=SELECT%201' -``` - -```response title="Response" -1 -``` - -在本示例中,wget 使用 `-nv`(非详细输出)和 `-O-` 参数将结果输出到终端。 -在这种情况下,无需对空格进行 URL 编码: - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1' -``` - -```response -1 -``` - -在本示例中,我们将原始 HTTP 请求通过管道送入 netcat: - -```bash title="command" -echo -ne 'GET /?query=SELECT%201 HTTP/1.0\r\n\r\n' | nc localhost 8123 -``` - -```response title="response" -HTTP/1.0 200 OK -X-ClickHouse-Summary: {"read_rows":"1","read_bytes":"1","written_rows":"0","written_bytes":"0","total_rows_to_read":"1","result_rows":"0","result_bytes":"0","elapsed_ns":"4505959","memory_usage":"1111711"} -Date: Tue, 11 Nov 2025 18:16:01 GMT -Connection: Close -Content-Type: text/tab-separated-values; charset=UTF-8 -Access-Control-Expose-Headers: X-ClickHouse-Query-Id,X-ClickHouse-Summary,X-ClickHouse-Server-Display-Name,X-ClickHouse-Format,X-ClickHouse-Timezone,X-ClickHouse-Exception-Code,X-ClickHouse-Exception-Tag -X-ClickHouse-Server-Display-Name: MacBook-Pro.local -X-ClickHouse-Query-Id: ec0d8ec6-efc4-4e1d-a14f-b748e01f5294 -X-ClickHouse-Format: TabSeparated -X-ClickHouse-Timezone: Europe/London -X-ClickHouse-Exception-Tag: dngjzjnxkvlwkeua - -1 -``` - -可以看到,`curl` 命令有些不便,因为空格必须进行 URL 编码。 -虽然 `wget` 会自行对所有内容进行转义,但我们不推荐使用它,因为在使用 keep-alive 和 `Transfer-Encoding: chunked` 的 HTTP/1.1 连接中,它的表现并不好。 - -```bash -$ echo 'SELECT 1' | curl 'http://localhost:8123/' --data-binary @- -1 - -$ echo 'SELECT 1' | curl 'http://localhost:8123/?query=' --data-binary @- -1 - -$ echo '1' | curl 'http://localhost:8123/?query=SELECT' --data-binary @- -1 -``` - -如果查询的一部分通过参数传递,另一部分通过 POST 发送,这两部分数据之间会被插入一个换行符。 -例如,下面这样将不起作用: - -```bash -$ echo 'ECT 1' | curl 'http://localhost:8123/?query=SEL' --data-binary @- -Code: 59, e.displayText() = DB::Exception: 语法错误:位置 0 失败:SEL -ECT 1 -,期望的关键字:SHOW TABLES、SHOW DATABASES、SELECT、INSERT、CREATE、ATTACH、RENAME、DROP、DETACH、USE、SET、OPTIMIZE.,e.what() = DB::Exception -``` - -默认情况下,数据以 [`TabSeparated`](/interfaces/formats/TabSeparated) 格式返回。 - -在查询中使用 `FORMAT` 子句可以请求其他格式。例如: - -```bash title="command" -wget -nv -O- 'http://localhost:8123/?query=SELECT 1, 2, 3 FORMAT JSON' -``` - - -```response title="Response" -{ - "meta": - [ - { - "name": "1", - "type": "UInt8" - }, - { - "name": "2", - "type": "UInt8" - }, - { - "name": "3", - "type": "UInt8" - } - ], - - "data": - [ - { - "1": 1, - "2": 2, - "3": 3 - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.000515, - "rows_read": 1, - "bytes_read": 1 - } -} -``` - -可以通过 `default_format` URL 参数或 `X-ClickHouse-Format` 请求头来指定一个不同于 `TabSeparated` 的默认格式。 - -```bash -$ echo 'SELECT 1 FORMAT Pretty' | curl 'http://localhost:8123/?' --data-binary @- -┏━━━┓ -┃ 1 ┃ -┡━━━┩ -│ 1 │ -└───┘ -``` - -你可以使用 POST 方法发送参数化查询。参数通过花括号加参数名和类型来指定,例如 `{name:Type}`。参数值通过对应的 `param_name` 传递: - -```bash -$ curl -X POST -F 'query=select {p1:UInt8} + {p2:UInt8}' -F "param_p1=3" -F "param_p2=4" 'http://localhost:8123/' - -7 -``` - - -## 通过 HTTP/HTTPS 执行 INSERT 查询 {#insert-queries} - -在执行 `INSERT` 查询时,需要使用用于传输数据的 `POST` 方法。在这种情况下,可以在 URL 参数中写入查询的开头部分,并使用 POST 传递要插入的数据。要插入的数据可以是例如来自 MySQL 的制表符分隔导出数据。通过这种方式,`INSERT` 查询可以替代 MySQL 中的 `LOAD DATA LOCAL INFILE`。 - -### 示例 {#examples} - -创建一张表: - -```bash -$ echo 'CREATE TABLE t (a UInt8) ENGINE = Memory' | curl 'http://localhost:8123/' --data-binary @- -``` - -要使用熟悉的 `INSERT` 查询插入数据: - -```bash -$ echo 'INSERT INTO t VALUES (1),(2),(3)' | curl 'http://localhost:8123/' --data-binary @- -``` - -如果要将数据与查询分开发送: - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -可以指定任意数据格式。例如,可以指定 `Values` 格式,它与执行 `INSERT INTO t VALUES` 时使用的格式相同: - -```bash -$ echo '(7),(8),(9)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20Values' --data-binary @- -``` - -要从制表符分隔的转储文件中插入数据,请指定对应的格式: - -```bash -$ echo -ne '10\n11\n12\n' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20FORMAT%20TabSeparated' --data-binary @- -``` - -要查看该表的内容: - -```bash -$ curl 'http://localhost:8123/?query=SELECT%20a%20FROM%20t' -7 -8 -9 -10 -11 -12 -1 -2 -3 -4 -5 -6 -``` - -:::note -由于并行查询处理,数据的输出顺序是随机的 -::: - -要删除该表: - -```bash -$ echo 'DROP TABLE t' | curl 'http://localhost:8123/' --data-binary @- -``` - -对于成功但不返回数据表的请求,将返回空响应体。 - - -## 压缩 {#compression} - -压缩可用于在传输大量数据时减少网络流量,也可用于创建直接以压缩形式保存的转储文件。 - -在传输数据时,可以使用 ClickHouse 的内部压缩格式。压缩后的数据采用非标准格式,必须使用 `clickhouse-compressor` 程序进行处理。该程序默认随 `clickhouse-client` 包一起安装。 - -为提高数据写入效率,可以通过 [`http_native_compression_disable_checksumming_on_decompress`](../operations/settings/settings.md#http_native_compression_disable_checksumming_on_decompress) 设置禁用服务端在解压时的校验和验证。 - -如果在 URL 中指定 `compress=1`,服务端会压缩发送给客户端的数据。如果在 URL 中指定 `decompress=1`,服务端会解压通过 `POST` 方法传入的数据。 - -也可以选择使用 [HTTP 压缩](https://en.wikipedia.org/wiki/HTTP_compression)。ClickHouse 支持以下[压缩方法](https://en.wikipedia.org/wiki/HTTP_compression#Content-Encoding_tokens): - -- `gzip` -- `br` -- `deflate` -- `xz` -- `zstd` -- `lz4` -- `bz2` -- `snappy` - -要发送压缩的 `POST` 请求,请在请求头中添加 `Content-Encoding: compression_method`。 - -要让 ClickHouse 压缩响应,请在请求中添加 `Accept-Encoding: compression_method` 请求头。 - -可以使用 [`http_zlib_compression_level`](../operations/settings/settings.md#http_zlib_compression_level) 设置为所有压缩方法配置数据压缩级别。 - -:::info -某些 HTTP 客户端可能会默认解压来自服务器的数据(例如使用 `gzip` 和 `deflate` 时),因此即使正确配置了压缩设置,仍有可能收到已解压的数据。 -::: - - - -## 示例 {#examples-compression} - -要向服务器发送压缩后的数据: - -```bash -echo "SELECT 1" | gzip -c | \ -curl -sS --data-binary @- -H 'Content-Encoding: gzip' 'http://localhost:8123/' -``` - -要从服务器接收压缩的数据归档: - -```bash -curl -vsS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' --output result.gz -d 'SELECT number FROM system.numbers LIMIT 3' - -zcat result.gz -0 -1 -2 -``` - -从服务器接收压缩数据,并使用 gunzip 解压得到原始数据: - -```bash -curl -sS "http://localhost:8123/?enable_http_compression=1" \ --H 'Accept-Encoding: gzip' -d 'SELECT number FROM system.numbers LIMIT 3' | gunzip - -0 -1 -2 -``` - - -## 默认数据库 {#default-database} - -你可以使用 `database` URL 参数或 `X-ClickHouse-Database` 请求头来指定默认数据库。 - -```bash -echo 'SELECT number FROM numbers LIMIT 10' | curl 'http://localhost:8123/?database=system' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -默认情况下,服务器设置中登记的数据库会被用作默认数据库。开箱即用时,该数据库名为 `default`。另外,你也可以通过在表名前加上“数据库名.” 的方式来显式指定要使用的数据库。 - - -## 认证 {#authentication} - -可以通过以下三种方式之一指定用户名和密码: - -1. 使用 HTTP Basic 认证。 - -例如: - -```bash -echo 'SELECT 1' | curl 'http://user:password@localhost:8123/' -d @- -``` - -2. 在 URL 的 `user` 和 `password` 参数中 - -:::warning -我们不建议使用此方法,因为这些参数可能会被 web 代理记录,并缓存到浏览器中 -::: - -例如: - -```bash -echo 'SELECT 1' | curl 'http://localhost:8123/?user=user&password=password' -d @- -``` - -3. 使用 'X-ClickHouse-User' 和 'X-ClickHouse-Key' 请求头 - -例如: - -```bash -echo 'SELECT 1' | curl -H 'X-ClickHouse-User: user' -H 'X-ClickHouse-Key: password' 'http://localhost:8123/' -d @- -``` - -如果未指定用户名,则使用 `default` 用户名。如果未指定密码,则使用空密码。 -你还可以使用 URL 参数,为单个查询的处理或整个设置配置文件指定任意设置。 - -例如: - -```text -http://localhost:8123/?profile=web&max_rows_to_read=1000000000&query=SELECT+1 -``` - -```bash -$ echo 'SELECT number FROM system.numbers LIMIT 10' | curl 'http://localhost:8123/?' --data-binary @- -0 -1 -2 -3 -4 -5 -6 -7 -8 -9 -``` - -如需了解更多信息,请参见: - -* [设置](/operations/settings/settings) -* [SET](/sql-reference/statements/set) - - -## 在 HTTP 协议中使用 ClickHouse 会话 {#using-clickhouse-sessions-in-the-http-protocol} - -你也可以在 HTTP 协议中使用 ClickHouse 会话。为此,需要在请求中添加 `session_id` `GET` 参数。你可以使用任意字符串作为会话 ID。 - -默认情况下,会话在 60 秒无活动后终止。要更改此超时时间(以秒为单位),请在服务器配置中修改 `default_session_timeout` 设置,或在请求中添加 `session_timeout` `GET` 参数。 - -要检查会话状态,请使用 `session_check=1` 参数。同一会话在任意时刻只能执行一个查询。 - -你可以在 `X-ClickHouse-Progress` 响应头中获取查询进度信息。为此,请启用 [`send_progress_in_http_headers`](../operations/settings/settings.md#send_progress_in_http_headers)。 - -下面是一个报文头顺序示例: - -```text -X-ClickHouse-Progress: {"read_rows":"261636","read_bytes":"2093088","total_rows_to_read":"1000000","elapsed_ns":"14050417","memory_usage":"22205975"} -X-ClickHouse-Progress: {"read_rows":"654090","read_bytes":"5232720","total_rows_to_read":"1000000","elapsed_ns":"27948667","memory_usage":"83400279"} -X-ClickHouse-Progress: {"read_rows":"1000000","read_bytes":"8000000","total_rows_to_read":"1000000","elapsed_ns":"38002417","memory_usage":"80715679"} -``` - -可能的头部字段为: - -| Header field | Description | -| -------------------- | ------------ | -| `read_rows` | 已读取的行数。 | -| `read_bytes` | 已读取的数据量(字节)。 | -| `total_rows_to_read` | 将要读取的总行数。 | -| `written_rows` | 已写入的行数。 | -| `written_bytes` | 已写入的数据量(字节)。 | -| `elapsed_ns` | 查询运行时间(纳秒)。 | -| `memory_usage` | 查询使用的内存(字节)。 | - -当 HTTP 连接丢失时,正在运行的请求不会自动停止。解析和数据格式化在服务器端执行,此时通过网络传输数据可能并不高效。 - -支持以下可选参数: - -| Parameters | Description | -| ---------------------- | ------------------------------------------------------------------------------------------------- | -| `query_id` (optional) | 可作为查询 ID 传递(任意字符串)。[`replace_running_query`](/operations/settings/settings#replace_running_query) | -| `quota_key` (optional) | 可作为配额键传递(任意字符串)。[“配额 (Quotas)”](/operations/quotas) | - -HTTP 接口允许传递外部数据(外部临时表)用于查询。更多信息请参见[“用于查询处理的外部数据”](/engines/table-engines/special/external-data)。 - - -## 响应缓冲 {#response-buffering} - -可以在服务端启用响应缓冲。为此可使用以下 URL 参数: - -* `buffer_size` -* `wait_end_of_query` - -可以使用以下设置: - -* [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size) -* [`http_wait_end_of_query`](/operations/settings/settings#http_wait_end_of_query) - -`buffer_size` 用于指定在服务器内存中对结果进行缓冲的字节数。如果结果体大于此阈值,当前缓冲内容会被写入 HTTP 通道,其余数据将直接发送到 HTTP 通道。 - -要确保整个响应都被缓冲,请设置 `wait_end_of_query=1`。在这种情况下,未存储在内存中的数据会缓存在服务器的临时文件中。 - -例如: - -```bash -curl -sS 'http://localhost:8123/?max_result_bytes=4000000&buffer_size=3000000&wait_end_of_query=1' -d 'SELECT toUInt8(number) FROM system.numbers LIMIT 9000000 FORMAT RowBinary' -``` - -:::tip -使用缓冲可以避免出现这样一种情况:在已经向客户端发送响应状态码和 HTTP 头之后,查询处理才发生错误。在这种情况下,错误消息会被写入响应正文的末尾,而在客户端只能在解析阶段才能检测到该错误。 -::: - - -## 使用查询参数设置角色 {#setting-role-with-query-parameters} - -该功能在 ClickHouse 24.4 中引入。 - -在某些特定场景下,可能需要在执行语句本身之前先设置已授予的角色。 -但是,无法同时发送 `SET ROLE` 和该语句,因为不支持多语句: - -```bash -curl -sS "http://localhost:8123" --data-binary "SET ROLE my_role;SELECT * FROM my_table;" -``` - -上面的命令会报错: - -```sql -代码:62. DB::Exception:语法错误(不允许使用多条语句) -``` - -要规避这一限制,请改用 `role` 查询参数: - -```bash -curl -sS "http://localhost:8123?role=my_role" --data-binary "SELECT * FROM my_table;" -``` - -这相当于在该语句前执行 `SET ROLE my_role`。 - -此外,也可以指定多个 `role` 查询参数: - -```bash -curl -sS "http://localhost:8123?role=my_role&role=my_other_role" --data-binary "SELECT * FROM my_table;" -``` - -在这种情况下,`?role=my_role&role=my_other_role` 与在执行该语句之前运行 `SET ROLE my_role, my_other_role` 的效果类似。 - - -## HTTP 响应状态码注意事项 {#http_response_codes_caveats} - -由于 HTTP 协议的限制,HTTP 200 响应状态码并不能保证查询一定成功。 - -下面是一个示例: - -```bash -curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)" -* 正在尝试连接 127.0.0.1:8123... -... -< HTTP/1.1 200 OK -... -代码: 395. DB::Exception: 传递给 'throwIf' 函数的值为非零值: 执行 'FUNCTION throwIf(equals(number, 2) :: 1) -> throwIf(equals(number, 2))' 时发生错误 -``` - -出现这种行为的原因在于 HTTP 协议的特性。HTTP 首先会发送状态码为 200 的 HTTP 头部,然后是 HTTP body,接着错误会作为纯文本被注入到 body 中。 - -这种行为与所使用的格式无关,无论是 `Native`、`TSV` 还是 `JSON`,错误信息始终会出现在响应流的中间部分。 - -你可以通过开启 `wait_end_of_query=1`([响应缓冲](#response-buffering))来缓解这个问题。在这种情况下,HTTP 头部的发送会被推迟到整个查询完成之后。然而,这并不能完全解决问题,因为结果仍然必须放入 [`http_response_buffer_size`](/operations/settings/settings#http_response_buffer_size) 限制之内,并且像 [`send_progress_in_http_headers`](/operations/settings/settings#send_progress_in_http_headers) 这样的其他设置可能会干扰头部发送的延迟。 - -:::tip -捕获所有错误的唯一方法,是在使用所需格式进行解析之前,先分析 HTTP body。 -::: - -在 ClickHouse 中,此类异常在 `http_write_exception_in_output_format=0`(默认)时,无论使用哪种格式(例如 `Native`、`TSV`、`JSON` 等),其异常格式都是一致的。这使得在客户端解析并提取错误消息变得更加容易。 - -```text -\r\n -__exception__\r\n -\r\n -<错误信息>\r\n - \r\n -__exception__\r\n - -``` - -其中 `` 是一个 16 字节的随机标记,与在响应头 `X-ClickHouse-Exception-Tag` 中发送的标记相同。 -`` 是实际的异常信息(其精确长度可在 `` 中找到)。上述整个异常块的大小最多为 16 KiB。 - -下面是一个 `JSON` 格式的示例。 - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+JSON" -... -{ - "meta": - [ - { - "name": "sleepEachRow(0.001)", - "type": "UInt8" - }, - { - "name": "throwIf(equals(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - }, - { - "sleepEachRow(0.001)": 0, - "throwIf(equals(number, 2))": 0 - } -__exception__ -dmrdfnujjqvszhav -Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 25.11.1.1) -262 dmrdfnujjqvszhav -__exception__ -``` - -下面是一个类似的示例,不过使用 `CSV` 格式 - -```bash -$ curl -v -Ss "http://localhost:8123/?max_block_size=1&query=select+sleepEachRow(0.001),throwIf(number=2)from+numbers(5)+FORMAT+CSV" -... -< -0,0 -0,0 -``` - - -**异常** -rumfyutuqkncbgau -Code: 395. DB::Exception: 传递给 'throwIf' 函数的值为非零:在执行 'FUNCTION throwIf(equals(__table1.number, 2_UInt8) :: 1) -> throwIf(equals(__table1.number, 2_UInt8)) UInt8 : 0' 时。(FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 25.11.1.1) -262 rumfyutuqkncbgau -**异常** - -``` -``` - - -## 参数化查询 {#cli-queries-with-parameters} - -可以创建参数化查询,并通过相应 HTTP 请求中的参数为其传递值。欲了解更多信息,请参阅[CLI 参数化查询](../interfaces/cli.md#cli-queries-with-parameters)。 - -### 示例 {#example-3} - -```bash -$ curl -sS "
?param_id=2¶m_phrase=test" -d "SELECT * FROM table WHERE int_column = {id:UInt8} and string_column = {phrase:String}" -``` - -### URL 参数中的制表符 {#tabs-in-url-parameters} - -查询参数是从“转义”格式中解析的。这样做有一些好处,比如可以将空值明确地解析为 `\N`。这意味着制表符应编码为 `\t`(或 `\` 加一个制表符)。例如,下面的字符串在 `abc` 和 `123` 之间包含一个实际的制表符,输入字符串会被拆分成两个值: - -```bash -curl -sS "http://localhost:8123" -d "SELECT splitByChar('\t', 'abc 123')" -``` - -```response -['abc','123'] -``` - -但是,如果你尝试在 URL 参数中使用 `%09` 来编码一个实际的 Tab 字符,它将无法被正确解析: - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%09123" -d "SELECT splitByChar('\t', {arg1:String})" -代码:457. DB::Exception:查询参数 'arg1' 的值 abc 123 无法解析为 String 类型,原因是解析不完整:7 字节中仅解析了 3 字节:abc。(BAD_QUERY_PARAMETER)(版本 23.4.1.869(官方构建)) -``` - -如果您使用 URL 参数,则需要将 `\t` 编码为 `%5C%09`。例如: - -```bash -curl -sS "http://localhost:8123?param_arg1=abc%5C%09123" -d "SELECT splitByChar('\t', {arg1:String})" -``` - -```response -['abc','123'] -``` - - -## 预定义的 HTTP 接口 {#predefined_http_interface} - -ClickHouse 通过 HTTP 接口支持特定查询。例如,可以通过以下方式向表中写入数据: - -```bash -$ echo '(4),(5),(6)' | curl 'http://localhost:8123/?query=INSERT%20INTO%20t%20VALUES' --data-binary @- -``` - -ClickHouse 还支持预定义 HTTP 接口(Predefined HTTP Interface),可以帮助你更轻松地与第三方工具集成,比如 [Prometheus exporter](https://github.com/ClickHouse/clickhouse_exporter)。下面来看一个示例。 - -首先,将以下配置段添加到你的服务器配置文件中。 - -`http_handlers` 被配置为包含多个 `rule`。ClickHouse 会将收到的 HTTP 请求与 `rule` 中预定义的类型进行匹配,并运行第一个匹配到的处理器。如果匹配成功,ClickHouse 随后会执行对应的预定义查询。 - -```yaml title="config.xml" - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.metrics LIMIT 5 FORMAT Template SETTINGS format_template_resultset = 'prometheus_template_output_format_resultset', format_template_row = 'prometheus_template_output_format_row', format_template_rows_between_delimiter = '\n' - - - ... - ... - -``` - -现在可以直接通过该 URL 请求 Prometheus 格式的数据: - - -```bash -$ curl -v 'http://localhost:8123/predefined_query' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /predefined_query HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> -< HTTP/1.1 200 OK -< Date: Tue, 28 Apr 2020 08:52:56 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< X-ClickHouse-Server-Display-Name: i-mloy5trc -< Transfer-Encoding: chunked -< X-ClickHouse-Query-Id: 96fe0052-01e6-43ce-b12a-6b7370de6e8a -< X-ClickHouse-Format: Template -< X-ClickHouse-Timezone: Asia/Shanghai -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -# HELP "Query" "Number of executing queries" {#help-query-number-of-executing-queries} -# TYPE "Query" counter {#type-query-counter} -"Query" 1 -``` - - -# HELP "Merge" "后台正在执行的合并数量" {#help-merge-number-of-executing-background-merges} -# TYPE "Merge" counter {#type-merge-counter} -"Merge" 0 - - - -# HELP "PartMutation" "Mutation 操作次数(ALTER DELETE/UPDATE)" {#help-partmutation-number-of-mutations-alter-deleteupdate} -# TYPE "PartMutation" counter {#type-partmutation-counter} -"PartMutation" 0 - - - -# HELP "ReplicatedFetch" "正在从副本拉取的数据分片数量" {#help-replicatedfetch-number-of-data-parts-being-fetched-from-replica} -# TYPE "ReplicatedFetch" counter {#type-replicatedfetch-counter} -"ReplicatedFetch" 0 - - - -# HELP "ReplicatedSend" "正在发送到副本的数据分片数量" {#help-replicatedsend-number-of-data-parts-being-sent-to-replicas} - -# TYPE "ReplicatedSend" counter {#type-replicatedsend-counter} - -"ReplicatedSend" 0 - -* 与主机 localhost 的连接 #0 保持打开 - -* 与主机 localhost 的连接 #0 保持打开 - -``` - -`http_handlers` 的配置选项工作方式如下。 - -`rule` 可以配置以下参数: -- `method` -- `headers` -- `url` -- `full_url` -- `handler` - -下面将逐一讨论这些参数: - -- `method` 负责匹配 HTTP 请求的方法部分。`method` 完全符合 HTTP 协议中 [`method`] - (https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods) 的定义。此为可选配置。如果在配置文件中未定义,则不会匹配 HTTP 请求的方法部分。 - -- `url` 负责匹配 HTTP 请求的 URL 部分(路径和查询字符串)。 - 如果 `url` 以 `regex:` 为前缀,则需使用 [RE2](https://github.com/google/re2) 正则表达式。 - 此为可选配置。如果在配置文件中未定义,则不会匹配 HTTP 请求的 URL 部分。 - -- `full_url` 与 `url` 相同,但包含完整的 URL,即 `schema://host:port/path?query_string`。 - 注意,ClickHouse 不支持"虚拟主机",因此 `host` 为 IP 地址(而非 `Host` 头的值)。 - -- `empty_query_string` - 确保请求中不包含查询字符串(`?query_string`) - -- `headers` 负责匹配 HTTP 请求的头部分。它兼容 RE2 正则表达式。此为可选配置。如果在配置文件中未定义,则不会匹配 HTTP 请求的头部分。 - -- `handler` 包含主要的处理逻辑。 - - 它可以具有以下 `type`: - - [`predefined_query_handler`](#predefined_query_handler) - - [`dynamic_query_handler`](#dynamic_query_handler) - - [`static`](#static) - - [`redirect`](#redirect) - - 以及以下参数: - - `query` — 与 `predefined_query_handler` 类型配合使用,在调用处理程序时执行查询。 - - `query_param_name` — 与 `dynamic_query_handler` 类型配合使用,提取并执行 HTTP 请求参数中与 `query_param_name` 值对应的值。 - - `status` — 与 `static` 类型配合使用,指定响应状态码。 - - `content_type` — 与任何类型配合使用,指定响应的 [content-type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type)。 - - `http_response_headers` — 与任何类型配合使用,指定响应头映射。也可用于设置内容类型。 - - `response_content` — 与 `static` 类型配合使用,指定发送给客户端的响应内容。当使用前缀 'file://' 或 'config://' 时,从文件或配置中查找内容并发送给客户端。 - - `user` - 执行查询的用户(默认用户为 `default`)。 - **注意**,无需为此用户指定密码。 - -接下来将讨论不同 `type` 的配置方法。 - -### predefined_query_handler {#predefined_query_handler} - -`predefined_query_handler` 支持设置 `Settings` 和 `query_params` 值。您可以在 `predefined_query_handler` 类型中配置 `query`。 - -`query` 值是 `predefined_query_handler` 的预定义查询,当 HTTP 请求匹配时由 ClickHouse 执行并返回查询结果。此为必需配置。 - -以下示例定义了 [`max_threads`](../operations/settings/settings.md#max_threads) 和 [`max_final_threads`](/operations/settings/settings#max_final_threads) 设置的值,然后查询系统表以检查这些设置是否成功设置。 - -:::note -要保留默认的 `handlers`,例如 `query`、`play`、`ping`,请添加 `` 规则。 -::: - -例如: -``` - - -```yaml - - - [^/]+)]]> - GET - - TEST_HEADER_VALUE - [^/]+)]]> - - - predefined_query_handler - - SELECT name, value FROM system.settings - WHERE name IN ({name_1:String}, {name_2:String}) - - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE' -H 'PARAMS_XXX:max_final_threads' 'http://localhost:8123/query_param_with_url/max_threads?max_threads=1&max_final_threads=2' -max_final_threads 2 -max_threads 1 -``` - -:::note -在一个 `predefined_query_handler` 中仅支持一个 `query`。 -::: - -### dynamic_query_handler {#dynamic_query_handler} - -在 `dynamic_query_handler` 中,查询通过 HTTP 请求参数传递。不同之处在于,在 `predefined_query_handler` 中,查询是写在配置文件中的。`query_param_name` 可以在 `dynamic_query_handler` 中进行配置。 - -ClickHouse 会从 HTTP 请求的 URL 中提取并执行对应于 `query_param_name` 的值。`query_param_name` 的默认值是 `/query`。这是一个可选配置。如果在配置文件中没有定义,则不会传入该参数。 - -要测试此功能,下面的示例会设置 [`max_threads`](../operations/settings/settings.md#max_threads) 和 `max_final_threads` 的值,并通过查询验证这些设置是否成功生效。 - -示例: - -```yaml - - - - TEST_HEADER_VALUE_DYNAMIC - - dynamic_query_handler - query_param - - - - -``` - -```bash -curl -H 'XXX:TEST_HEADER_VALUE_DYNAMIC' 'http://localhost:8123/own?max_threads=1&max_final_threads=2¶m_name_1=max_threads¶m_name_2=max_final_threads&query_param=SELECT%20name,value%20FROM%20system.settings%20where%20name%20=%20%7Bname_1:String%7D%20OR%20name%20=%20%7Bname_2:String%7D' -max_threads 1 -max_final_threads 2 -``` - -### static {#static} - -`static` 可以返回 [`content_type`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type)、[status](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) 和 `response_content`。其中,`response_content` 用于返回指定的内容。 - -例如,要返回消息 "Say Hi!": - -```yaml - - - GET - xxx - /hi - - static - 402 - text/html; charset=UTF-8 - - en - 43 - - #highlight-next-line - Say Hi! - - - - -``` - -可以使用 `http_response_headers` 来设置内容类型,而无需使用 `content_type`。 - - -```yaml - - - GET - xxx - /hi - - static - 402 - #begin-highlight - - text/html; charset=UTF-8 - en - 43 - - #end-highlight - 你好! - - - - -``` - -```bash -curl -vv -H 'XXX:xxx' 'http://localhost:8123/hi' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /hi HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 402 Payment Required -< Date: Wed, 29 Apr 2020 03:51:26 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -Say Hi!% -``` - -在发送给客户端的配置中查找内容。 - -```yaml -
]]>
- - - - GET - xxx - /get_config_static_handler - - static - config://get_config_static_handler - - - -``` - -```bash -$ curl -v -H 'XXX:xxx' 'http://localhost:8123/get_config_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_config_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:01:24 GMT -< Connection: Keep-Alive -< Content-Type: text/plain; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -* Connection #0 to host localhost left intact -
% -``` - -要在发送给客户端的文件中查找内容: - - -```yaml - - - GET - xxx - /get_absolute_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file:///absolute_path_file.html - - - - GET - xxx - /get_relative_path_static_handler - - static - text/html; charset=UTF-8 - - 737060cd8c284d8af7ad3082f209582d - - file://./relative_path_file.html - - - -``` - -```bash -$ user_files_path='/var/lib/clickhouse/user_files' -$ sudo echo "相对路径文件" > $user_files_path/relative_path_file.html -$ sudo echo "绝对路径文件" > $user_files_path/absolute_path_file.html -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_absolute_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_absolute_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:16 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -绝对路径文件 -* Connection #0 to host localhost left intact -$ curl -vv -H 'XXX:xxx' 'http://localhost:8123/get_relative_path_static_handler' -* Trying ::1... -* Connected to localhost (::1) port 8123 (#0) -> GET /get_relative_path_static_handler HTTP/1.1 -> Host: localhost:8123 -> User-Agent: curl/7.47.0 -> Accept: */* -> XXX:xxx -> -< HTTP/1.1 200 OK -< Date: Wed, 29 Apr 2020 04:18:31 GMT -< Connection: Keep-Alive -< Content-Type: text/html; charset=UTF-8 -< Transfer-Encoding: chunked -< Keep-Alive: timeout=10 -< X-ClickHouse-Summary: {"read_rows":"0","read_bytes":"0","written_rows":"0","written_bytes":"0","total_rows_to_read":"0","elapsed_ns":"662334","memory_usage":"8451671"} -< -相对路径文件 -* Connection #0 to host localhost left intact -``` - -### redirect {#redirect} - -`redirect` 会将请求以 `302` 状态码重定向到 `location` - -例如,下面展示了如何在 ClickHouse Play 中自动将用户设置为 `play`: - -```xml - - - - GET - /play - - redirect - /play?user=play - - - - -``` - - -## HTTP 响应头 {#http-response-headers} - -ClickHouse 允许配置自定义的 HTTP 响应头,这些响应头可以应用于任何可配置的处理程序。可以通过 `http_response_headers` 设置这些响应头,该设置接受表示响应头名称及其值的键值对。此功能对于实现自定义安全响应头、CORS 策略,或在 ClickHouse HTTP 接口中统一满足其他 HTTP 响应头需求特别有用。 - -例如,可以为以下内容配置响应头: - -* 常规查询端点 -* Web UI -* 健康检查。 - -也可以指定 `common_http_response_headers`。这些设置将应用于配置中定义的所有 HTTP 处理程序。 - -这些响应头会包含在每个已配置处理程序的 HTTP 响应中。 - -在下面的示例中,每个服务器响应都将包含两个自定义响应头:`X-My-Common-Header` 和 `X-My-Custom-Header`。 - -```xml - - - - 通用标头 - - - GET - /ping - - ping - - 确实是自定义的 - - - - - -``` - - -## 在 HTTP 流式传输期间出现异常时返回合法的 JSON/XML 响应 {#valid-output-on-exception-http-streaming} - -当通过 HTTP 执行查询时,即便部分数据已经发送,仍然可能会抛出异常。通常情况下,异常会以纯文本的形式发送给客户端。 -即便使用了特定的数据格式来输出数据,发生这种情况时,输出结果在该数据格式规范上可能变为不合法。 -为避免这种情况,可以通过设置 [`http_write_exception_in_output_format`](/operations/settings/settings#http_write_exception_in_output_format)(默认禁用),让 ClickHouse 按指定格式写出异常(当前支持 XML 和 JSON* 格式)。 - -示例: - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>3)+from+system.numbers+format+JSON+settings+max_block_size=1&http_write_exception_in_output_format=1' -{ - "meta": - [ - { - "name": "number", - "type": "UInt64" - }, - { - "name": "throwIf(greater(number, 2))", - "type": "UInt8" - } - ], - - "data": - [ - { - "number": "0", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "1", - "throwIf(greater(number, 2))": 0 - }, - { - "number": "2", - "throwIf(greater(number, 2))": 0 - } - ], - - "rows": 3, - - "exception": "代码: 395. DB::Exception: 传递给 'throwIf' 函数的值为非零值: 执行 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1' 时。(FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (版本 23.8.1.1)" -} -``` - -```bash -$ curl 'http://localhost:8123/?query=SELECT+number,+throwIf(number>2)+from+system.numbers+format+XML+settings+max_block_size=1&http_write_exception_in_output_format=1' - - - - - - number - UInt64 - - - throwIf(greater(number, 2)) - UInt8 - - - - - - 0 - 0 - - - 1 - 0 - - - 2 - 0 - - - 3 - Code: 395. DB::Exception: Value passed to 'throwIf' function is non-zero: while executing 'FUNCTION throwIf(greater(number, 2) :: 2) -> throwIf(greater(number, 2)) UInt8 : 1'. (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 23.8.1.1) - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/jdbc.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/jdbc.md deleted file mode 100644 index 5f899572d79..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/jdbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: '使用 JDBC 驱动程序将 Java 应用程序连接到 ClickHouse 的指南' -sidebar_label: 'JDBC 驱动程序' -sidebar_position: 20 -slug: /interfaces/jdbc -title: 'JDBC 驱动程序' -doc_type: 'guide' ---- - -# JDBC 驱动程序 {#jdbc-driver} - -使用[官方 JDBC 驱动程序](/docs/integrations/language-clients/java/jdbc)(及其 Java 客户端)从 Java 应用程序访问 ClickHouse。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/mysql.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/mysql.md deleted file mode 100644 index e5c9234b283..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/mysql.md +++ /dev/null @@ -1,181 +0,0 @@ ---- -description: 'ClickHouse 中 MySQL 协议接口的文档,允许 MySQL 客户端连接 ClickHouse' -sidebar_label: 'MySQL 接口' -sidebar_position: 25 -slug: /interfaces/mysql -title: 'MySQL 接口' -doc_type: 'guide' ---- - -import Image from '@theme/IdealImage'; -import mysql0 from '@site/static/images/interfaces/mysql0.png'; -import mysql1 from '@site/static/images/interfaces/mysql1.png'; -import mysql2 from '@site/static/images/interfaces/mysql2.png'; -import mysql3 from '@site/static/images/interfaces/mysql3.png'; - - -# MySQL 接口 {#mysql-interface} - -ClickHouse 支持 MySQL 线协议(wire protocol)。这使得某些没有原生 ClickHouse 连接器的客户端可以改用 MySQL 协议进行连接,并且已经与以下 BI 工具完成验证: - -- [Looker Studio](../integrations/data-visualization/looker-studio-and-clickhouse.md) -- [Tableau Online](../integrations/tableau-online) -- [QuickSight](../integrations/quicksight) - -如果你在尝试其它尚未测试的客户端或集成,需要注意可能存在以下限制: - -- SSL 实现可能不完全兼容;可能会出现与 [TLS SNI](https://www.cloudflare.com/learning/ssl/what-is-sni/) 相关的问题。 -- 某些工具可能需要尚未实现的方言特性(例如特定于 MySQL 的函数或设置)。 - -如果存在原生驱动(例如 [DBeaver](../integrations/dbeaver)),应优先使用原生驱动而不是 MySQL 接口。此外,尽管大多数通过 MySQL 协议工作的客户端通常可以正常使用,但 MySQL 接口不能保证可作为已有 MySQL 查询代码库的即插即用替代方案。 - -如果你的使用场景涉及某个没有原生 ClickHouse 驱动的特定工具,并希望通过 MySQL 接口来使用它,但发现存在某些不兼容之处,请在 ClickHouse 仓库中[创建一个 issue](https://github.com/ClickHouse/ClickHouse/issues)。 - -::::note -为了更好地支持上述 BI 工具的 SQL 方言,ClickHouse 的 MySQL 接口会在设置 [prefer_column_name_to_alias = 1](/operations/settings/settings#prefer_column_name_to_alias) 的情况下隐式运行 SELECT 查询。 -这一行为无法关闭,并且在极少数边缘场景下,可能会导致发送到 ClickHouse 常规查询接口与 MySQL 查询接口的查询产生不同行为。 -:::: - - - -## 在 ClickHouse Cloud 上启用 MySQL 接口 {#enabling-the-mysql-interface-on-clickhouse-cloud} - -1. 创建好 ClickHouse Cloud 服务后,点击 `Connect` 按钮。 - -
- -凭据界面 - 提示 - -2. 将 `Connect with` 下拉框更改为 `MySQL`。 - -
- -凭据界面 - 已选择 MySQL - -3. 打开开关,为该特定服务启用 MySQL 接口。这样会为该服务开放端口 `3306`,并显示包含您唯一 MySQL 用户名的 MySQL 连接界面。密码与该服务默认用户的密码相同。 - -
- -凭据界面 - 已启用 MySQL - -复制显示的 MySQL 连接字符串。 - -凭据界面 - 连接字符串 - - - -## 在 ClickHouse Cloud 中创建多个 MySQL 用户 {#creating-multiple-mysql-users-in-clickhouse-cloud} - -默认情况下,系统内置了一个 `mysql4` 用户,它使用与 `default` 用户相同的密码。`` 部分是你的 ClickHouse Cloud 主机名的第一个片段。要与那些实现了安全连接、但在 TLS 握手中**不**提供 [SNI 信息](https://www.cloudflare.com/learning/ssl/what-is-sni) 的工具配合使用,就必须采用这种格式;否则在用户名中没有这个额外提示的情况下,无法完成内部路由(MySQL 控制台客户端就是此类工具之一)。 - -因此,当创建要通过 MySQL 接口使用的新用户时,我们*强烈建议*遵循 `mysql4_` 这种格式,其中 `` 是用来标识你的 Cloud 服务的提示,而 `` 则是你自定义的任意后缀。 - -:::tip -对于像 `foobar.us-east1.aws.clickhouse.cloud` 这样的 ClickHouse Cloud 主机名,`` 部分等于 `foobar`,一个自定义的 MySQL 用户名可以类似于 `mysql4foobar_team1`。 -::: - -如果你需要应用额外设置等,可以创建额外用户用于 MySQL 接口。 - -1. (可选)创建一个要应用于自定义用户的[设置配置文件](/sql-reference/statements/create/settings-profile)。例如,创建一个带有额外设置的 `my_custom_profile`,它会在稍后使用我们创建的用户连接时默认生效: - - ```sql - CREATE SETTINGS PROFILE my_custom_profile SETTINGS prefer_column_name_to_alias=1; - ``` - - `prefer_column_name_to_alias` 只是一个示例,你也可以在这里使用其他设置。 - -2. 使用以下格式[创建用户](/sql-reference/statements/create/user):`mysql4_`([见上文](#creating-multiple-mysql-users-in-clickhouse-cloud))。密码必须为 double SHA1 格式。例如: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$'; - ``` - - 或者,如果你希望为该用户使用自定义配置文件: - - ```sql - CREATE USER mysql4foobar_team1 IDENTIFIED WITH double_sha1_password BY 'YourPassword42$' SETTINGS PROFILE 'my_custom_profile'; - ``` - - 其中 `my_custom_profile` 是你之前创建的配置文件名称。 - -3. 为新用户[授予](/sql-reference/statements/grant)与目标表或数据库交互所需的权限。例如,如果你只想授予对 `system.query_log` 的访问权限: - - ```sql - GRANT SELECT ON system.query_log TO mysql4foobar_team1; - ``` - -4. 使用你创建的用户,通过 MySQL 接口连接到你的 ClickHouse Cloud 服务。 - -### 在 ClickHouse Cloud 中排查多个 MySQL 用户问题 {#troubleshooting-multiple-mysql-users-in-clickhouse-cloud} - -如果你创建了一个新的 MySQL 用户,并且在通过 MySQL CLI 客户端连接时看到如下错误: - -```sql -ERROR 2013 (HY000): 在'读取授权数据包'时与 MySQL 服务器失去连接,系统错误: 54 -``` - -在这种情况下,请确保用户名符合 `mysql4_` 格式,如[上文](#creating-multiple-mysql-users-in-clickhouse-cloud)所述。 - - -## 在自管 ClickHouse 上启用 MySQL 接口 {#enabling-the-mysql-interface-on-self-managed-clickhouse} - -将 [mysql_port](../operations/server-configuration-parameters/settings.md#mysql_port) 设置添加到服务器的配置文件中。例如,你可以在 `config.d/` [文件夹](../operations/configuration-files) 中新建一个 XML 文件来定义该端口: - -```xml - - 9004 - -``` - -启动 ClickHouse 服务器,并在日志中查找类似如下的信息,其中包含 “Listening for MySQL compatibility protocol”: - -```bash -{} Application: 正在监听 MySQL 兼容协议:127.0.0.1:9004 -``` - - -## 将 MySQL 连接到 ClickHouse {#connect-mysql-to-clickhouse} - -以下命令演示了如何使用 MySQL 客户端 `mysql` 连接到 ClickHouse: - -```bash -mysql --protocol tcp -h [hostname] -u [username] -P [port_number] [database_name] -``` - -例如: - -```bash -$ mysql --protocol tcp -h 127.0.0.1 -u default -P 9004 default -``` - -连接成功时的输出: - -```text -欢迎使用 MySQL 监视器。命令以 ; 或 \g 结束。 -您的 MySQL 连接 ID 为 4 -服务器版本:20.2.1.1-ClickHouse - -版权所有 (c) 2000, 2019, Oracle 和/或其关联公司。保留所有权利。 - -Oracle 是 Oracle Corporation 和/或其关联公司的注册商标。 -其他名称可能是其各自所有者的商标。 - -输入 'help;' 或 '\h' 获取帮助。输入 '\c' 清除当前输入语句。 - -mysql> -``` - -为了兼容所有 MySQL 客户端,建议在配置文件中使用 [双重 SHA1](/operations/settings/settings-users#user-namepassword) 指定用户密码。 -如果使用 [SHA256](/sql-reference/functions/hash-functions#SHA256) 指定用户密码,一些客户端将无法完成身份验证(如 mysqljs 和旧版本的 MySQL、MariaDB 命令行工具)。 - -限制: - -* 不支持预处理查询(prepared queries) - -* 某些数据类型会以字符串形式发送 - -要取消一个长时间运行的查询,请使用 `KILL QUERY connection_id` 语句(在执行时会被替换为 `KILL QUERY WHERE query_id = connection_id`)。例如: - -```bash -$ mysql --protocol tcp -h mysql_server -P 9004 default -u default --password=123 -e "KILL QUERY 123456;" -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md deleted file mode 100644 index 42972279b9e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/native-clients-interfaces-index.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -description: 'ClickHouse 的原生客户端和接口' -keywords: ['客户端', '接口', 'CLI', 'SQL 控制台', '驱动'] -slug: /interfaces/natives-clients-and-interfaces -title: '原生客户端和接口' -doc_type: 'landing-page' ---- - -# 原生客户端和接口 {#native-clients-interfaces} - -ClickHouse 提供多种原生客户端和接口,可用于连接 ClickHouse。 - -更多信息请参阅以下页面: - -| Section | Summary | -|--------------------------------------------------------------|-------------------------------------------------------------------------------------| -| [Command-Line Client](/interfaces/cli) | 原生命令行客户端,支持命令行选项和配置文件。 | -| [Drivers & Interfaces](/interfaces/overview) | 多种网络接口、库以及图形化界面。 | -| [SQL Console](/integrations/sql-clients/sql-console) | 在 ClickHouse Cloud 中与数据交互的快捷简便方式。 | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/odbc.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/odbc.md deleted file mode 100644 index 7666a958068..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/odbc.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ClickHouse ODBC 驱动程序文档' -sidebar_label: 'ODBC 驱动程序' -sidebar_position: 35 -slug: /interfaces/odbc -title: 'ODBC 驱动程序' -doc_type: 'reference' ---- - -# ODBC 驱动程序 {#odbc-driver} - -使用 [官方 ODBC 驱动程序](https://github.com/ClickHouse/clickhouse-odbc) 访问 ClickHouse 数据源。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/overview.md deleted file mode 100644 index f60af30d6c3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/overview.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '用于连接 ClickHouse 的网络接口、驱动和工具概览' -keywords: ['clickhouse', '网络', '接口', 'http', 'tcp', 'grpc', '命令行', - '客户端', 'jdbc', 'odbc', '驱动'] -sidebar_label: '概览' -slug: /interfaces/overview -title: '驱动与接口' -doc_type: 'reference' ---- - -# 驱动与接口 {#drivers-and-interfaces} - -ClickHouse 提供了两个网络接口(可选择通过 TLS 封装以增强安全性): - -* [HTTP](http.md),文档完善且便于直接使用。 -* [原生 TCP](../interfaces/tcp.md),开销更低。 - -在大多数情况下,推荐使用合适的工具或库,而不是直接与这些接口交互。以下是 ClickHouse 官方支持的方式: - -* [命令行客户端](../interfaces/cli.md) -* [JDBC 驱动](../interfaces/jdbc.md) -* [ODBC 驱动](../interfaces/odbc.md) -* [C++ 客户端库](../interfaces/cpp.md) - -ClickHouse 还支持两种 RPC 协议: - -* 专为 ClickHouse 设计的 [gRPC 协议](grpc.md)。 -* [Apache Arrow Flight](arrowflight.md)。 - -ClickHouse 服务器为高级用户提供了内置的可视化界面: - -* Play UI:在浏览器中打开 `/play`; -* 高级仪表盘:在浏览器中打开 `/dashboard`; -* 面向 ClickHouse 工程师的二进制符号查看器:在浏览器中打开 `/binary`。 - -同时,还有大量用于配合 ClickHouse 使用的第三方库: - -* [客户端库](../interfaces/third-party/client-libraries.md) -* [集成](../interfaces/third-party/integrations.md) -* [可视化界面](../interfaces/third-party/gui.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/postgresql.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/postgresql.md deleted file mode 100644 index be8f7556b26..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/postgresql.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: 'ClickHouse 中 PostgreSQL wire 协议接口的文档' -sidebar_label: 'PostgreSQL 接口' -sidebar_position: 20 -slug: /interfaces/postgresql -title: 'PostgreSQL 接口' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# PostgreSQL 接口 {#postgresql-interface} - - - -ClickHouse 支持 PostgreSQL 线协议,这使您可以使用 PostgreSQL 客户端连接到 ClickHouse。从某种意义上说,ClickHouse 可以充当一个 PostgreSQL 实例——从而允许您将 PostgreSQL 客户端应用程序连接到 ClickHouse,即便该应用程序尚未被 ClickHouse 直接支持(例如 Amazon Redshift)。 - -要启用 PostgreSQL 线协议,请在服务器的配置文件中添加 [postgresql_port](../operations/server-configuration-parameters/settings.md#postgresql_port) 设置。例如,您可以在 `config.d` 文件夹中新建一个 XML 文件来定义该端口: - -```xml - - 9005 - -``` - -启动 ClickHouse 服务器,并在日志中查找类似如下的信息,其中包含 **Listening for PostgreSQL compatibility protocol**: - -```response -{} 应用程序:监听 PostgreSQL 兼容协议:127.0.0.1:9005 -``` - -## 使用 psql 连接到 ClickHouse {#connect-psql-to-clickhouse} - -下面的命令演示如何使用 PostgreSQL 客户端 `psql` 连接到 ClickHouse: - -```bash -psql -p [port] -h [hostname] -U [username] [database_name] -``` - -例如: - -```bash -psql -p 9005 -h 127.0.0.1 -U alice default -``` - -:::note -`psql` 客户端需要使用密码登录,因此无法使用未设置密码的 `default` 用户进行连接。可以为 `default` 用户设置密码,或者使用其他用户登录。 -::: - -`psql` 客户端会提示输入密码: - -```response -用户 alice 的密码: -psql(14.2,服务器 22.3.1.1) -警告:psql 主版本号为 14,服务器主版本号为 22。 - 某些 psql 功能可能无法使用。 -输入 "help" 获取帮助。 - -default=> -``` - -就完成了!你现在已经有一个已连接到 ClickHouse 的 PostgreSQL 客户端,所有命令和查询都会在 ClickHouse 上执行。 - -:::note -PostgreSQL 协议目前只支持明文密码。 -::: - -## 使用 SSL {#using-ssl} - -如果在 ClickHouse 实例上配置了 SSL/TLS,那么 `postgresql_port` 将使用相同的设置(该端口同时供启用和未启用加密的客户端使用)。 - -每种客户端都有各自使用 SSL 建立连接的方法。下面的命令演示了如何传入证书和密钥,以安全地将 `psql` 连接到 ClickHouse: - -```bash -psql "port=9005 host=127.0.0.1 user=alice dbname=default sslcert=/path/to/certificate.pem sslkey=/path/to/key.pem sslrootcert=/path/to/rootcert.pem sslmode=verify-ca" -``` - -## 使用 SCRAM-SHA-256 配置 ClickHouse 用户认证 {#using-scram-sha256} - -为确保 ClickHouse 用户认证的安全性,推荐使用 SCRAM-SHA-256 协议。可通过在 users.xml 文件中指定 `password_scram_sha256_hex` 元素来配置用户。密码哈希必须在生成时使用 num_iterations=4096 参数。 - -请确保 psql 客户端在建立连接时支持并协商使用 SCRAM-SHA-256。 - -下面是用户 `user_with_sha256`,密码为 `abacaba` 的示例配置: - -```xml - - 04e7a70338d7af7bb6142fe7e19fef46d9b605f3e78b932a60e8200ef9154976 - -``` - -有关其 SSL 设置的更多详细信息,请参阅 [PostgreSQL 文档](https://jdbc.postgresql.org/documentation/head/ssl-client.html)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/prometheus.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/prometheus.md deleted file mode 100644 index eda5ad23387..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/prometheus.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: '介绍 ClickHouse 对 Prometheus 协议支持的文档' -sidebar_label: 'Prometheus 协议' -sidebar_position: 19 -slug: /interfaces/prometheus -title: 'Prometheus 协议' -doc_type: 'reference' ---- - -# Prometheus 协议 {#prometheus-protocols} - -## 暴露指标 {#expose} - -:::note -如果使用 ClickHouse Cloud,可以通过 [Prometheus Integration](/integrations/prometheus) 将指标暴露给 Prometheus。 -::: - -ClickHouse 可以将自身的指标暴露出来,以供 Prometheus 抓取: - -````xml - - 9363 - /metrics - true - true - true - true - true - true - - -可以使用 `` 部分来创建更多扩展处理器。 -此部分类似于 [](/interfaces/http),但用于 Prometheus 协议: - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - -```` - -设置: - -| Name | Default | Description | -| ---------------------------- | ---------- | ---------------------------------------------------------------------------------------------- | -| `port` | none | 用于对外提供指标协议服务的端口。 | -| `endpoint` | `/metrics` | 供 Prometheus 服务器抓取指标的 HTTP endpoint。必须以 `/` 开头。不应与 `` 小节同时使用。 | -| `url` / `headers` / `method` | none | 用于根据请求查找匹配 handler 的过滤条件。与 [``](/interfaces/http) 小节中同名字段的含义相同。 | -| `metrics` | true | 暴露 [system.metrics](/operations/system-tables/metrics) 表中的指标。 | -| `asynchronous_metrics` | true | 暴露 [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) 表中的当前指标值。 | -| `events` | true | 暴露 [system.events](/operations/system-tables/events) 表中的指标。 | -| `errors` | true | 暴露自上次服务器重启以来按错误码统计的错误次数。该信息也可以从 [system.errors](/operations/system-tables/errors) 表中获取。 | -| `histograms` | true | 暴露来自 [system.histogram_metrics](/operations/system-tables/histogram_metrics) 的直方图指标。 | -| `dimensional_metrics` | true | 暴露来自 [system.dimensional_metrics](/operations/system-tables/dimensional_metrics) 的维度指标。 | - -检查(将 `127.0.0.1` 替换为 ClickHouse 服务器的 IP 地址或主机名): - -```bash -curl 127.0.0.1:9363/metrics -``` - -## Remote-write 协议 {#remote-write} - -ClickHouse 支持 [remote-write](https://prometheus.io/docs/specs/remote_write_spec/) 协议。 -通过该协议接收的数据会被写入一个 [TimeSeries](/engines/table-engines/special/time_series) 表 -(该表需要事先创建)。 - -```xml - - 9363 - - - /write - - remote_write - db_name - time_series_table
-
-
-
-
-``` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | -| `port` | none | 用于提供 `remote-write` 协议服务的端口。 | -| `url` / `headers` / `method` | none | 用于为请求查找匹配处理器的筛选条件。类似于 [``](/interfaces/http) 部分中具有相同名称的字段。 | -| `table` | none | 用于写入通过 `remote-write` 协议接收数据的 [TimeSeries](/engines/table-engines/special/time_series) 表名。该名称还可以选择性地包含数据库名称。 | -| `database` | none | 在 `table` 设置中未指定数据库名称时,此处指定 `table` 设置中所述表所在的数据库名称。 | - -## 远程读取协议 {#remote-read} - -ClickHouse 支持 [remote-read](https://prometheus.io/docs/prometheus/latest/querying/remote_read_api/) 协议。 -数据从 [TimeSeries](/engines/table-engines/special/time_series) 表中读取,并通过该协议进行传输。 - -```xml - - 9363 - - - /read - - remote_read - db_name - time_series_table
-
-
-
-
-``` - -Settings: - -| Name | Default | Description | -| ---------------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | -| `port` | none | 用于提供 `remote-read` 协议服务的端口。 | -| `url` / `headers` / `method` | none | 用于为请求查找匹配的处理程序的过滤器。类似于 [``](/interfaces/http) 部分中同名字段。 | -| `table` | none | 通过 `remote-read` 协议发送数据时要读取数据的 [TimeSeries](/engines/table-engines/special/time_series) 表名。该名称还可以可选地包含数据库名称。 | -| `database` | none | 如果在 `table` 设置中未指定数据库名称,则此处为包含该表的数据库名称。 | - -## 多协议配置 {#multiple-protocols} - -可以在同一个位置同时指定多个协议: - -```xml - - 9363 - - - /metrics - - expose_metrics - true - true - true - true - true - true - - - - /write - - remote_write - db_name.time_series_table
-
-
- - /read - - remote_read - db_name.time_series_table
-
-
-
-
-``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md deleted file mode 100644 index 4a53a95c008..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/schema-inference.md +++ /dev/null @@ -1,2376 +0,0 @@ ---- -description: '介绍 ClickHouse 中根据输入数据自动推断 schema 的页面' -sidebar_label: 'Schema 推断' -slug: /interfaces/schema-inference -title: '从输入数据自动推断 schema' -doc_type: 'reference' ---- - -ClickHouse 可以在几乎所有受支持的 [输入格式](formats.md) 中自动确定输入数据的结构。 -本文档将介绍在何种情况下会使用 schema 推断、它在不同输入格式中的工作方式,以及可以用来控制它的设置。 - - - -## 用法 {#usage} - -当 ClickHouse 需要读取特定格式的数据但其结构未知时,会使用模式推断。 - - - -## 表函数 [file](../sql-reference/table-functions/file.md)、[s3](../sql-reference/table-functions/s3.md)、[url](../sql-reference/table-functions/url.md)、[hdfs](../sql-reference/table-functions/hdfs.md)、[azureBlobStorage](../sql-reference/table-functions/azureBlobStorage.md)。 {#table-functions-file-s3-url-hdfs-azureblobstorage} - -这些表函数支持一个可选参数 `structure`,用于指定输入数据的结构。如果未指定该参数或将其设置为 `auto`,则会自动从数据中推断结构。 - -**示例:** - -假设我们在 `user_files` 目录下有一个采用 JSONEachRow 格式的文件 `hobbies.jsonl`,其内容如下: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["足球", "烹饪", "音乐"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["网球", "艺术"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["健身", "阅读", "购物"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["电影", "跳伞"]} -``` - -ClickHouse 可以在无需指定其结构的情况下读取这些数据: - -```sql -SELECT * FROM file('hobbies.jsonl') -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -注意:格式 `JSONEachRow` 是根据文件扩展名 `.jsonl` 自动推断的。 - -你可以通过 `DESCRIBE` 查询查看自动推断出的结构: - -```sql -DESCRIBE file('hobbies.jsonl') -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## 表引擎 [File](../engines/table-engines/special/file.md)、[S3](../engines/table-engines/integrations/s3.md)、[URL](../engines/table-engines/special/url.md)、[HDFS](../engines/table-engines/integrations/hdfs.md)、[azureBlobStorage](../engines/table-engines/integrations/azureBlobStorage.md) {#table-engines-file-s3-url-hdfs-azureblobstorage} - -如果在 `CREATE TABLE` 查询中未指定列列表,表结构将会根据数据自动推断。 - -**示例:** - -我们来使用文件 `hobbies.jsonl`。我们可以使用 `File` 表引擎,根据该文件中的数据创建一张表: - -```sql -CREATE TABLE hobbies ENGINE=File(JSONEachRow, 'hobbies.jsonl') -``` - -```response -完成。 -``` - -```sql -SELECT * FROM hobbies -``` - -```response -┌─id─┬─age─┬─name───┬─hobbies──────────────────────────┐ -│ 1 │ 25 │ Josh │ ['football','cooking','music'] │ -│ 2 │ 19 │ Alan │ ['tennis','art'] │ -│ 3 │ 32 │ Lana │ ['fitness','reading','shopping'] │ -│ 4 │ 47 │ Brayan │ ['movies','skydiving'] │ -└────┴─────┴────────┴──────────────────────────────────┘ -``` - -```sql -DESCRIBE TABLE hobbies -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## clickhouse-local {#clickhouse-local} - -`clickhouse-local` 提供一个可选参数 `-S/--structure`,用于指定输入数据的结构。如果未指定该参数或将其设置为 `auto`,则会从数据中自动推断结构。 - -**示例:** - -我们使用文件 `hobbies.jsonl`,并通过 `clickhouse-local` 从该文件中查询数据: - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='DESCRIBE TABLE hobbies' -``` - -```response -id Nullable(Int64) -age Nullable(Int64) -name Nullable(String) -hobbies Array(Nullable(String)) -``` - -```shell -clickhouse-local --file='hobbies.jsonl' --table='hobbies' --query='SELECT * FROM hobbies' -``` - -```response -1 25 Josh ['football','cooking','music'] -2 19 Alan ['tennis','art'] -3 32 Lana ['fitness','reading','shopping'] -4 47 Brayan ['movies','skydiving'] -``` - - -## 使用插入表的结构 {#using-structure-from-insertion-table} - -当使用 `file/s3/url/hdfs` 表函数向表中插入数据时, -可以选择使用插入表的结构,而不是从数据中推断结构。 -这可以提升插入性能,因为模式推断可能会耗费一些时间。此外,当表已经过模式优化时,这也非常有用, -这样就不会在类型之间执行转换。 - -有一个专门用于控制此行为的设置项 [use_structure_from_insertion_table_in_table_functions](/operations/settings/settings.md/#use_structure_from_insertion_table_in_table_functions)。 -它有 3 个可能的取值: - -* 0 - 表函数将从数据中提取结构。 -* 1 - 表函数将使用插入表的结构。 -* 2 - ClickHouse 将自动确定是可以使用插入表的结构,还是需要进行模式推断。默认值。 - -**示例 1:** - -创建名为 `hobbies1` 的表,其结构如下: - -```sql -CREATE TABLE hobbies1 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `name` String, - `hobbies` Array(String) -) -ENGINE = MergeTree -ORDER BY id; -``` - -然后从文件 `hobbies.jsonl` 中插入数据: - -```sql -INSERT INTO hobbies1 SELECT * FROM file(hobbies.jsonl) -``` - -在这种情况下,文件中的所有列都会原样插入到表中,因此 ClickHouse 将使用目标表的结构,而不是进行模式推断。 - -**示例 2:** - -让我们创建表 `hobbies2`,其结构如下: - -```sql -CREATE TABLE hobbies2 -( - `id` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -然后从文件 `hobbies.jsonl` 中插入数据: - -```sql -INSERT INTO hobbies2 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -在这种情况下,`SELECT` 查询中的所有列都存在于该表中,因此 ClickHouse 将使用要插入的表的结构。 -请注意,这只适用于支持仅读取部分列的输入格式,比如 JSONEachRow、TSKV、Parquet 等(因此例如对 TSV 格式则不起作用)。 - -**示例 3:** - -让我们创建结构如下的表 `hobbies3`: - -```sql -CREATE TABLE hobbies3 -( - `identifier` UInt64, - `age` LowCardinality(UInt8), - `hobbies` Array(String) -) - ENGINE = MergeTree -ORDER BY identifier; -``` - -并从文件 `hobbies.jsonl` 中插入数据: - -```sql -INSERT INTO hobbies3 SELECT id, age, hobbies FROM file(hobbies.jsonl) -``` - -在这种情况下,在 `SELECT` 查询中使用了列 `id`,但该表并不包含该列(它只有一个名为 `identifier` 的列), -因此 ClickHouse 无法使用插入表的结构,而会使用模式推断。 - -**示例 4:** - -让我们创建一个结构如下的表 `hobbies4`: - -```sql -CREATE TABLE hobbies4 -( - `id` UInt64, - `any_hobby` Nullable(String) -) - ENGINE = MergeTree -ORDER BY id; -``` - -然后从文件 `hobbies.jsonl` 中插入数据: - -```sql -INSERT INTO hobbies4 SELECT id, empty(hobbies) ? NULL : hobbies[1] FROM file(hobbies.jsonl) -``` - -在这种情况下,由于在 `SELECT` 查询中对列 `hobbies` 进行了某些操作后再将其插入表中,ClickHouse 无法复用插入目标表的结构,而是会使用 schema 推断。 - - -## Schema inference cache {#schema-inference-cache} - -对于大多数输入格式,schema 推断会读取一部分数据来确定其结构,这个过程可能需要一定时间。 -为了避免 ClickHouse 每次从同一个文件读取数据时都重新推断相同的 schema,推断出的 schema 会被缓存,当再次访问同一个文件时,ClickHouse 将直接使用缓存中的 schema。 - -有一些专门的设置用于控制该缓存: -- `schema_inference_cache_max_elements_for_{file/s3/hdfs/url/azure}` —— 对应表函数可缓存的 schema 的最大数量。默认值为 `4096`。这些设置应在服务器配置中进行设置。 -- `schema_inference_use_cache_for_{file,s3,hdfs,url,azure}` —— 用于开启或关闭在 schema 推断中使用缓存。这些设置可以在查询中使用。 - -文件的 schema 既可以通过修改数据而改变,也可以通过更改格式设置而改变。 -因此,schema 推断缓存会根据文件来源、格式名称、所使用的格式设置以及文件的最后修改时间来区分 schema。 - -注意:在 `url` 表函数中通过 URL 访问的一些文件可能不包含最后修改时间的信息;对于这种情况,可以使用特殊设置 -`schema_inference_cache_require_modification_time_for_url`。禁用该设置后,即使这些文件没有最后修改时间,也允许使用缓存中的 schema。 - -另外还有一个系统表 [schema_inference_cache](../operations/system-tables/schema_inference_cache.md),其中包含缓存中的所有当前 schema,以及系统查询 `SYSTEM DROP SCHEMA CACHE [FOR File/S3/URL/HDFS]`, -用于清理所有来源或特定来源的 schema 缓存。 - -**示例:** - -我们来尝试对来自 S3 的示例数据集 `github-2022.ndjson.gz` 进行结构推断,并观察 schema 推断缓存是如何工作的: - - - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -返回 5 行。用时:0.601 秒。 -``` - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -``` - -```response -┌─name───────┬─type─────────────────────────────────────────┐ -│ type │ Nullable(String) │ -│ actor │ Tuple( ↴│ -│ │↳ avatar_url Nullable(String), ↴│ -│ │↳ display_login Nullable(String), ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ login Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ repo │ Tuple( ↴│ -│ │↳ id Nullable(Int64), ↴│ -│ │↳ name Nullable(String), ↴│ -│ │↳ url Nullable(String)) │ -│ created_at │ Nullable(String) │ -│ payload │ Tuple( ↴│ -│ │↳ action Nullable(String), ↴│ -│ │↳ distinct_size Nullable(Int64), ↴│ -│ │↳ pull_request Tuple( ↴│ -│ │↳ author_association Nullable(String),↴│ -│ │↳ base Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ head Tuple( ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ sha Nullable(String)), ↴│ -│ │↳ number Nullable(Int64), ↴│ -│ │↳ state Nullable(String), ↴│ -│ │↳ title Nullable(String), ↴│ -│ │↳ updated_at Nullable(String), ↴│ -│ │↳ user Tuple( ↴│ -│ │↳ login Nullable(String))), ↴│ -│ │↳ ref Nullable(String), ↴│ -│ │↳ ref_type Nullable(String), ↴│ -│ │↳ size Nullable(Int64)) │ -└────────────┴──────────────────────────────────────────────┘ -``` - -5 行数据。耗时 0.059 秒。 - -```` - -如您所见,第二个查询几乎立即成功。 - -让我们尝试更改一些可能影响推断架构的设置: - -```sql -DESCRIBE TABLE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/github/github-2022.ndjson.gz') -SETTINGS input_format_json_try_infer_named_tuples_from_objects=0, input_format_json_read_objects_as_strings = 1 - -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ type │ Nullable(String) │ │ │ │ │ │ -│ actor │ Nullable(String) │ │ │ │ │ │ -│ repo │ Nullable(String) │ │ │ │ │ │ -│ created_at │ Nullable(String) │ │ │ │ │ │ -│ payload │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -返回 5 行。耗时:0.611 秒 -```` - -正如你所见,对于同一个文件,缓存中的 schema 并没有被使用,因为用于推断 schema 的设置已被更改。 - -让我们检查一下 `system.schema_inference_cache` 表的内容: - -```sql -SELECT schema, format, source FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─schema──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─format─┬─source───────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ type Nullable(String), actor Tuple(avatar_url Nullable(String), display_login Nullable(String), id Nullable(Int64), login Nullable(String), url Nullable(String)), repo Tuple(id Nullable(Int64), name Nullable(String), url Nullable(String)), created_at Nullable(String), payload Tuple(action Nullable(String), distinct_size Nullable(Int64), pull_request Tuple(author_association Nullable(String), base Tuple(ref Nullable(String), sha Nullable(String)), head Tuple(ref Nullable(String), sha Nullable(String)), number Nullable(Int64), state Nullable(String), title Nullable(String), updated_at Nullable(String), user Tuple(login Nullable(String))), ref Nullable(String), ref_type Nullable(String), size Nullable(Int64)) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -│ type Nullable(String), actor Nullable(String), repo Nullable(String), created_at Nullable(String), payload Nullable(String) │ NDJSON │ datasets-documentation.s3.eu-west-3.amazonaws.com443/datasets-documentation/github/github-2022.ndjson.gz │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -可以看到,同一个文件有两个不同的 schema。 - -我们可以使用一条系统查询来清除 schema 缓存: - -```sql -SYSTEM DROP SCHEMA CACHE FOR S3 -``` - -```response -完成。 -``` - -```sql -SELECT count() FROM system.schema_inference_cache WHERE storage='S3' -``` - -```response -┌─count()─┐ -│ 0 │ -└─────────┘ -``` - - -## 文本格式 {#text-formats} - -对于文本格式,ClickHouse 逐行读取数据,根据格式提取列值, -然后使用一些递归解析器和启发式方法来确定每个值的类型。在模式推断中,从数据中读取的最大行数和字节数 -由设置 `input_format_max_rows_to_read_for_schema_inference`(默认 25000)和 `input_format_max_bytes_to_read_for_schema_inference`(默认 32Mb)控制。 -默认情况下,所有推断出的类型都是 [Nullable](../sql-reference/data-types/nullable.md),但可以通过设置 `schema_inference_make_columns_nullable` 来更改这一点(参见[文本格式设置](#settings-for-text-formats)部分中的示例)。 - -### JSON 格式 {#json-formats} - -在 JSON 格式中,ClickHouse 根据 JSON 规范解析值,然后尝试为它们找到最合适的数据类型。 - -下面来看一下其工作原理、可以推断出哪些类型,以及在 JSON 格式中可以使用哪些特定设置。 - -**示例** - -在这里及后续示例中,将使用 [format](../sql-reference/table-functions/format.md) 表函数。 - -整数、浮点数、布尔值、字符串: - -```sql -DESC format(JSONEachRow, '{"int" : 42, "float" : 42.42, "string" : "Hello, World!"}'); -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -日期与日期时间: - -```sql -DESC format(JSONEachRow, '{"date" : "2022-01-01", "datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"}') -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -数组: - -```sql -DESC format(JSONEachRow, '{"arr" : [1, 2, 3], "nested_arrays" : [[1, 2, 3], [4, 5, 6], []]}') -``` - -```response -┌─name──────────┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ nested_arrays │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└───────────────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果数组包含 `null`,ClickHouse 将根据该数组中其他元素的类型来确定类型: - -```sql -DESC format(JSONEachRow, '{"arr" : [null, 42, null]}') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -如果一个数组包含不同类型的值,并且启用了设置 `input_format_json_infer_array_of_dynamic_from_array_of_different_types`(该设置默认启用),则该数组的类型为 `Array(Dynamic)`: - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=1; -DESC format(JSONEachRow, '{"arr" : [42, "hello", [1, 2, 3]]}'); -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Dynamic) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -命名元组(Named Tuple): - -当启用 `input_format_json_try_infer_named_tuples_from_objects` 设置时,ClickHouse 会在进行模式推断时尝试从 JSON 对象推断命名元组。 -生成的命名元组将包含示例数据中所有相应 JSON 对象中的全部元素。 - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -未命名元组(Unnamed Tuple): - -当 `input_format_json_infer_array_of_dynamic_from_array_of_different_types` 设置被禁用时,我们会在 JSON 格式中将包含不同类型元素的数组视为未命名元组。 - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types = 0; -DESC format(JSONEachRow, '{"tuple" : [1, "Hello, World!", [1, 2, 3]]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果某些值为 `null` 或为空,则使用其他行中对应值的类型: - -```sql -SET input_format_json_infer_array_of_dynamic_from_array_of_different_types=0; -DESC format(JSONEachRow, $$ - {"tuple" : [1, null, null]} - {"tuple" : [null, "Hello, World!", []]} - {"tuple" : [null, null, [1, 2, 3]]} - $$) -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ tuple │ Tuple(Nullable(Int64), Nullable(String), Array(Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Maps: - -在 JSON 中,我们可以将值类型相同的对象读取为 Map 类型。 -注意:仅当设置 `input_format_json_read_objects_as_strings` 和 `input_format_json_try_infer_named_tuples_from_objects` 被禁用时,此功能才会生效。 - - -```sql -SET input_format_json_read_objects_as_strings = 0, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, '{"map" : {"key1" : 42, "key2" : 24, "key3" : 4}}') -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ map │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -嵌套复杂类型: - -```sql -DESC format(JSONEachRow, '{"value" : [[[42, 24], []], {"key1" : 42, "key2" : 24}]}') -``` - -```response -┌─name──┬─type─────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Tuple(Array(Array(Nullable(String))), Tuple(key1 Nullable(Int64), key2 Nullable(Int64))) │ │ │ │ │ │ -└───────┴──────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果 ClickHouse 无法为某个键确定类型,而数据又只包含 null/空对象/空数组,则在启用设置 `input_format_json_infer_incomplete_types_as_strings` 时会使用 `String` 类型,否则会抛出异常: - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 1; -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ arr │ Array(Nullable(String)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, '{"arr" : [null, null]}') SETTINGS input_format_json_infer_incomplete_types_as_strings = 0; -``` - -```response -代码: 652. DB::Exception: 从 localhost:9000 接收。DB::Exception: -无法根据前 1 行数据确定列 'arr' 的类型, -该列很可能仅包含 Null 值或空数组/映射。 -... -``` - -#### JSON 设置 {#json-settings} - -##### input_format_json_try_infer_numbers_from_strings {#input_format_json_try_infer_numbers_from_strings} - -启用此设置后,将会尝试从字符串值中推断数值。 - -该设置默认禁用。 - -**示例:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : "42"} - {"value" : "424242424242"} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_try_infer_named_tuples_from_objects {#input_format_json_try_infer_named_tuples_from_objects} - -启用此设置后,可以从 JSON 对象中推断出命名 Tuple。生成的命名 Tuple 将包含样本数据中所有相关 JSON 对象的全部元素。 -当 JSON 数据并不稀疏,即数据样本包含所有可能的对象键时,此功能会非常有用。 - -此设置默认启用。 - -**示例** - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : {"a" : 42, "b" : "Hello"}}, {"obj" : {"a" : 43, "c" : [1, 2, 3]}}, {"obj" : {"d" : {"e" : 42}}}') -``` - -结果: - - -```response -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Tuple(e Nullable(Int64))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -DESC format(JSONEachRow, '{"array" : [{"a" : 42, "b" : "Hello"}, {}, {"c" : [1,2,3]}, {"d" : "2020-01-01"}]}') -``` - -结果: - -```markdown -┌─name──┬─type────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ array │ Array(Tuple(a Nullable(Int64), b Nullable(String), c Array(Nullable(Int64)), d Nullable(Date))) │ │ │ │ │ │ -└───────┴─────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects {#input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects} - -启用此设置后,在从 JSON 对象推断命名 Tuple(且 `input_format_json_try_infer_named_tuples_from_objects` 已启用)时,对于含有歧义的路径,将使用 String 类型,而不是抛出异常。 -这样,即便存在歧义路径,也可以将 JSON 对象读取为命名 Tuple。 - -默认禁用。 - -**示例** - -在禁用该设置的情况下: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 0; -DESC format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -结果: - -```response -代码:636. DB::Exception:无法从 JSONEachRow 格式文件中提取表结构。错误: -代码:117. DB::Exception:JSON 对象数据存在歧义:某些对象中路径 'a' 的类型为 'Int64',而另一些对象中为 'Tuple(b String)'。您可以启用设置 input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects 来对路径 'a' 使用 String 类型。(INCORRECT_DATA)(版本 24.3.1.1)。 -您可以手动指定结构。(CANNOT_EXTRACT_TABLE_STRUCTURE) -``` - -启用该设置后: - -```sql -SET input_format_json_try_infer_named_tuples_from_objects = 1; -SET input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects = 1; -DESC format(JSONEachRow, '{"obj" : "a" : 42}, {"obj" : {"a" : {"b" : "Hello"}}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : 42}}, {"obj" : {"a" : {"b" : "Hello"}}}'); -``` - -结果: - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Nullable(String)) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -┌─obj─────────────────┐ -│ ('42') │ -│ ('{"b" : "Hello"}') │ -└─────────────────────┘ -``` - -##### input_format_json_read_objects_as_strings {#input_format_json_read_objects_as_strings} - -启用此设置后,可以将嵌套的 JSON 对象作为字符串读取。 -此设置可用于在不使用 JSON 对象类型的情况下读取嵌套 JSON 对象。 - -默认情况下,此设置处于启用状态。 - -注意:仅当设置 `input_format_json_try_infer_named_tuples_from_objects` 被禁用时,启用此设置才会生效。 - - -```sql -SET input_format_json_read_objects_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 0; -DESC format(JSONEachRow, $$ - {"obj" : {"key1" : 42, "key2" : [1,2,3,4]}} - {"obj" : {"key3" : {"nested_key" : 1}}} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_numbers_as_strings {#input_format_json_read_numbers_as_strings} - -启用此设置后,会将数值按字符串读取。 - -此设置默认启用。 - -**示例** - -```sql -SET input_format_json_read_numbers_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : 1055} - {"value" : "unknown"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_numbers {#input_format_json_read_bools_as_numbers} - -启用此设置后,可以将布尔值按数字读取。 - -该设置默认处于启用状态。 - -**示例:** - -```sql -SET input_format_json_read_bools_as_numbers = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : 42} - $$) -``` - -```response -┌─name──┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(Int64) │ │ │ │ │ │ -└───────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_bools_as_strings {#input_format_json_read_bools_as_strings} - -启用此设置后,可以将布尔(Bool)值作为字符串读取。 - -此设置默认开启。 - -**示例:** - -```sql -SET input_format_json_read_bools_as_strings = 1; -DESC format(JSONEachRow, $$ - {"value" : true} - {"value" : "Hello, World"} - $$) -``` - -```response -┌─name──┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ value │ Nullable(String) │ │ │ │ │ │ -└───────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -##### input_format_json_read_arrays_as_strings {#input_format_json_read_arrays_as_strings} - -启用此设置后,可以将 JSON 数组中的值作为字符串读取。 - -此设置默认启用。 - -**示例** - -```sql -SET input_format_json_read_arrays_as_strings = 1; -SELECT arr, toTypeName(arr), JSONExtractArrayRaw(arr)[3] from format(JSONEachRow, 'arr String', '{"arr" : [1, "Hello", [1,2,3]]}'); -``` - -```response -┌─arr───────────────────┬─toTypeName(arr)─┬─arrayElement(JSONExtractArrayRaw(arr), 3)─┐ -│ [1, "Hello", [1,2,3]] │ String │ [1,2,3] │ -└───────────────────────┴─────────────────┴───────────────────────────────────────────┘ -``` - -##### input_format_json_infer_incomplete_types_as_strings {#input_format_json_infer_incomplete_types_as_strings} - - -启用此设置后,在模式推断期间,对于数据样本中仅包含 `Null`/`{}`/`[]` 的 JSON 键,可以使用 String 类型。 - -在 JSON 格式中,如果启用了所有相关设置(默认均启用),任何值都可以按 String 类型读取。这样,在模式推断时,对于类型未知的键使用 String 类型,就可以避免出现类似 `Cannot determine type for column 'column_name' by first 25000 rows of data, most likely this column contains only Nulls or empty Arrays/Maps` 的错误。 - -示例: - -```sql -SET input_format_json_infer_incomplete_types_as_strings = 1, input_format_json_try_infer_named_tuples_from_objects = 1; -DESCRIBE format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -SELECT * FROM format(JSONEachRow, '{"obj" : {"a" : [1,2,3], "b" : "hello", "c" : null, "d" : {}, "e" : []}}'); -``` - -结果: - -```markdown -┌─name─┬─type───────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ obj │ Tuple(a Array(Nullable(Int64)), b Nullable(String), c Nullable(String), d Nullable(String), e Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ - -┌─obj────────────────────────────┐ -│ ([1,2,3],'hello',NULL,'{}',[]) │ -└────────────────────────────────┘ -``` - -### CSV {#csv} - -在 CSV 格式中,ClickHouse 会根据分隔符从每一行中提取列值。对于除数字和字符串之外的所有类型,ClickHouse 要求它们用双引号括起来。如果值被双引号括起来,ClickHouse 会尝试使用递归解析器解析引号内的数据,然后为其找到最合适的数据类型。如果值没有被双引号括起来,ClickHouse 会先尝试将其解析为数字;如果该值不是数字,ClickHouse 会将其视为字符串。 - -如果不希望 ClickHouse 使用解析器和启发式方法来尝试推断复杂类型,可以禁用参数 `input_format_csv_use_best_effort_in_schema_inference`,此时 ClickHouse 会将所有列都视为 String。 - -如果启用了参数 `input_format_csv_detect_header`,ClickHouse 会在推断表结构(schema)时尝试检测包含列名(以及可能的类型)的表头。该参数默认启用。 - -**示例:** - -Integers、Floats、Bools、Strings: - -```sql -DESC format(CSV, '42,42.42,true,"Hello,World!"') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -未加引号的字符串: - -```sql -DESC format(CSV, 'Hello world!,World hello!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date 和 DateTime: - - -```sql -DESC format(CSV, '"2020-01-01","2020-01-01 00:00:00","2022-01-01 00:00:00.000"') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -数组: - -```sql -DESC format(CSV, '"[1,2,3]","[[1, 2], [], [3, 4]]"') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(CSV, $$"['Hello', 'world']","[['Abc', 'Def'], []]"$$) -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果数组中包含 null,ClickHouse 将使用其他数组元素的类型: - -```sql -DESC format(CSV, '"[NULL, 42, NULL]"') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -地图: - -```sql -DESC format(CSV, $$"{'key1' : 42, 'key2' : 24}"$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -嵌套数组和映射(Map): - -```sql -DESC format(CSV, $$"[{'key1' : [[42, 42], []], 'key2' : [[null], [42]]}]"$$) -``` - -```response -┌─name─┬─type──────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Array(Nullable(Int64))))) │ │ │ │ │ │ -└──────┴───────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -如果 ClickHouse 无法根据引号中的内容确定类型,而数据又只包含 null 值时,ClickHouse 会将其视为 String: - -```sql -DESC format(CSV, '"[NULL, NULL]"') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -禁用设置 `input_format_csv_use_best_effort_in_schema_inference` 时的示例: - -```sql -SET input_format_csv_use_best_effort_in_schema_inference = 0 -DESC format(CSV, '"[1,2,3]",42.42,Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -表头自动检测示例(启用 `input_format_csv_detect_header` 时): - -仅包含列名: - -```sql -SELECT * FROM format(CSV, -$$"number","string","array" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -名称与类型: - -```sql -DESC format(CSV, -$$"number","string","array" -"UInt32","String","Array(UInt16)" -42,"Hello","[1, 2, 3]" -43,"World","[4, 5, 6]" -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -请注意,只有当至少有一列的数据类型不是 String 时,才会检测到表头。如果所有列的数据类型都是 String,则不会检测到表头: - -```sql -SELECT * FROM format(CSV, -$$"first_column","second_column" -"Hello","World" -"World","Hello" -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ 第一列 │ 第二列 │ -│ 你好 │ 世界 │ -│ 世界 │ 你好 │ -└──────────────┴───────────────┘ -``` - -#### CSV 设置 {#csv-settings} - -##### input_format_csv_try_infer_numbers_from_strings {#input_format_csv_try_infer_numbers_from_strings} - -启用此设置后,将尝试自动从字符串值中推断数值。 - -此设置默认关闭。 - -**示例:** - -```sql -SET input_format_json_try_infer_numbers_from_strings = 1; -DESC format(CSV, '42,42.42'); -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -### TSV/TSKV {#tsv-tskv} - -在 TSV/TSKV 格式中,ClickHouse 会根据制表符分隔符从行中提取列值,然后使用递归解析器解析该值以确定最合适的类型。若无法确定类型,ClickHouse 会将该值视为 String。 - -如果你不希望 ClickHouse 使用某些解析器和启发式方法来尝试推断复杂类型,你可以关闭配置项 `input_format_tsv_use_best_effort_in_schema_inference`, -此时 ClickHouse 会将所有列都视为 String。 - -如果启用了配置项 `input_format_tsv_detect_header`,ClickHouse 会在推断 schema 时尝试检测包含列名(以及可能的类型)的表头。该配置默认启用。 - -**示例:** - -Integers、Floats、Bools、Strings: - -```sql -DESC format(TSV, '42 42.42 true Hello,World!') -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSKV, 'int=42 float=42.42 bool=true string=Hello,World!\n') -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ float │ Nullable(Float64) │ │ │ │ │ │ -│ bool │ Nullable(Bool) │ │ │ │ │ │ -│ string │ Nullable(String) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -Date、DateTime: - -```sql -DESC format(TSV, '2020-01-01 2020-01-01 00:00:00 2022-01-01 00:00:00.000') -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -数组: - -```sql -DESC format(TSV, '[1,2,3] [[1, 2], [], [3, 4]]') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(TSV, '[''Hello'', ''world''] [[''Abc'', ''Def''], []]') -``` - -```response -┌─name─┬─type───────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(String)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(String))) │ │ │ │ │ │ -└──────┴────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果数组包含 null,ClickHouse 将根据其他数组元素推断类型: - -```sql -DESC format(TSV, '[NULL, 42, NULL]') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -元组: - -```sql -DESC format(TSV, $$(42, 'Hello, world!')$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -地图: - -```sql -DESC format(TSV, $${'key1' : 42, 'key2' : 24}$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -嵌套数组、元组和映射: - -```sql -DESC format(TSV, $$[{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}]$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -如果 ClickHouse 无法确定类型,因为数据仅包含 null 值,则会将其视为 String: - -```sql -DESC format(TSV, '[NULL, NULL]') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -关闭 `input_format_tsv_use_best_effort_in_schema_inference` 设置时的示例: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -表头自动检测示例(启用 `input_format_tsv_detect_header` 时): - -仅包含列名: - -```sql -SELECT * FROM format(TSV, -$$number string array -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$); -``` - -```response -┌─number─┬─string─┬─array───┐ -│ 42 │ Hello │ [1,2,3] │ -│ 43 │ World │ [4,5,6] │ -└────────┴────────┴─────────┘ -``` - -名称与类型: - -```sql -DESC format(TSV, -$$number string array -UInt32 String Array(UInt16) -42 Hello [1, 2, 3] -43 World [4, 5, 6] -$$) -``` - -```response -┌─name───┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ UInt32 │ │ │ │ │ │ -│ string │ String │ │ │ │ │ │ -│ array │ Array(UInt16) │ │ │ │ │ │ -└────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -请注意,只有在至少存在一列为非 String 类型时,才能检测到表头。如果所有列都是 String 类型,则不会检测到表头: - -```sql -SELECT * FROM format(TSV, -$$first_column second_column -Hello World -World Hello -$$) -``` - -```response -┌─c1───────────┬─c2────────────┐ -│ 第一列 │ 第二列 │ -│ 你好 │ 世界 │ -│ 世界 │ 你好 │ -└──────────────┴───────────────┘ -``` - -### Values {#values} - -在 Values 格式中,ClickHouse 从行中提取列值,然后使用递归解析器对其进行解析,其方式与解析字面量类似。 - -**示例:** - - -整数、浮点数、布尔值、字符串: - -```sql -DESC format(Values, $$(42, 42.42, true, 'Hello,World!')$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(Float64) │ │ │ │ │ │ -│ c3 │ Nullable(Bool) │ │ │ │ │ │ -│ c4 │ Nullable(String) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -日期(Date)、日期时间(DateTime): - -```sql - DESC format(Values, $$('2020-01-01', '2020-01-01 00:00:00', '2022-01-01 00:00:00.000')$$) -``` - -```response -┌─name─┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Date) │ │ │ │ │ │ -│ c2 │ Nullable(DateTime) │ │ │ │ │ │ -│ c3 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└──────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -数组: - -```sql -DESC format(Values, '([1,2,3], [[1, 2], [], [3, 4]])') -``` - -```response -┌─name─┬─type──────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -│ c2 │ Array(Array(Nullable(Int64))) │ │ │ │ │ │ -└──────┴───────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果数组中包含 null,ClickHouse 将根据该数组中其他元素的类型来确定类型: - -```sql -DESC format(Values, '([NULL, 42, NULL])') -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -元组: - -```sql -DESC format(Values, $$((42, 'Hello, world!'))$$) -``` - -```response -┌─name─┬─type─────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Tuple(Nullable(Int64), Nullable(String)) │ │ │ │ │ │ -└──────┴──────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -地图: - -```sql -DESC format(Values, $$({'key1' : 42, 'key2' : 24})$$) -``` - -```response -┌─name─┬─type─────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Map(String, Nullable(Int64)) │ │ │ │ │ │ -└──────┴──────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -嵌套数组、元组和映射: - -```sql -DESC format(Values, $$([{'key1' : [(42, 'Hello'), (24, NULL)], 'key2' : [(NULL, ','), (42, 'world!')]}])$$) -``` - -```response -┌─name─┬─type────────────────────────────────────────────────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Array(Map(String, Array(Tuple(Nullable(Int64), Nullable(String))))) │ │ │ │ │ │ -└──────┴─────────────────────────────────────────────────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -如果 ClickHouse 无法确定类型(因为数据只包含 null),则会抛出异常: - -```sql -DESC format(Values, '([NULL, NULL])') -``` - -```response -代码: 652. DB::Exception: 从 localhost:9000 接收。DB::Exception: -无法根据前 1 行数据确定列 'c1' 的类型, -该列很可能仅包含 Null 值或空数组/映射。 -... -``` - -将设置 `input_format_tsv_use_best_effort_in_schema_inference` 禁用时的示例: - -```sql -SET input_format_tsv_use_best_effort_in_schema_inference = 0 -DESC format(TSV, '[1,2,3] 42.42 Hello World!') -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(String) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### CustomSeparated {#custom-separated} - -在 CustomSeparated 格式中,ClickHouse 会先按指定的分隔符从每一行中提取所有列值,然后再根据转义规则尝试推断每个值的数据类型。 - -如果启用了 `input_format_custom_detect_header` 设置,ClickHouse 在推断表结构时会尝试检测包含列名(以及可能的列类型)的表头。此设置默认启用。 - -**示例** - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -表头自动检测示例(启用 `input_format_custom_detect_header` 后): - - -```sql -SET format_custom_row_before_delimiter = '', - format_custom_row_after_delimiter = '\n', - format_custom_row_between_delimiter = '\n', - format_custom_result_before_delimiter = '\n', - format_custom_result_after_delimiter = '\n', - format_custom_field_delimiter = '', - format_custom_escaping_rule = 'Quoted' - -DESC format(CustomSeparated, $$ -'number''string''array' - -42.42'Some string 1'[1, NULL, 3] - -NULL'Some string 3'[1, 2, NULL] - -$$) -``` - -```response -┌─number─┬─string────────┬─array──────┐ -│ 42.42 │ Some string 1 │ [1,NULL,3] │ -│ ᴺᵁᴸᴸ │ Some string 3 │ [1,2,NULL] │ -└────────┴───────────────┴────────────┘ -``` - -### 模板 {#template} - -在 Template 格式中,ClickHouse 会先根据指定的模板从每一行中提取所有列值,然后再根据每个值的转义规则尝试推断其数据类型。 - -**示例** - -假设我们有一个名为 `resultset` 的文件,内容如下: - -```bash - -${data} -``` - -以及一个名为 `row_format` 的文件,内容如下: - -```text -${column_1:CSV}${column_2:Quoted}${column_3:JSON} -``` - -接下来我们可以运行如下查询: - -```sql -SET format_template_rows_between_delimiter = '\n', - format_template_row = 'row_format', - format_template_resultset = 'resultset_format' - -DESC format(Template, $$ -42.42'Some string 1'[1, null, 2] - -\N'Some string 3'[1, 2, null] - -$$) -``` - -```response -┌─name─────┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ column_1 │ Nullable(Float64) │ │ │ │ │ │ -│ column_2 │ Nullable(String) │ │ │ │ │ │ -│ column_3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Regexp {#regexp} - -与 Template 类似,在 Regexp 格式中,ClickHouse 首先根据指定的正则表达式从行中提取所有列值,然后根据指定的转义规则尝试为每个值推断数据类型。 - -**示例** - -```sql -SET format_regexp = '^Line: value_1=(.+?), value_2=(.+?), value_3=(.+?)', - format_regexp_escaping_rule = 'CSV' -``` - - -DESC format(Regexp, $$Line: value_1=42, value_2="Some string 1", value_3="[1, NULL, 3]" -Line: value_1=2, value_2="Some string 2", value_3="[4, 5, NULL]"$$) - -```` -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Int64) │ │ │ │ │ │ -│ c2 │ Nullable(String) │ │ │ │ │ │ -│ c3 │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -```` - -### 文本格式的设置 {#settings-for-text-formats} - -#### input_format_max_rows_to_read_for_schema_inference/input_format_max_bytes_to_read_for_schema_inference {#input-format-max-rows-to-read-for-schema-inference} - -这些设置控制在执行模式推断时需读取的数据量。读取的行数/字节数越多,在模式推断上花费的时间就越长,但越有可能正确确定类型(尤其是当数据包含大量 null 时)。 - -默认值: - -* `input_format_max_rows_to_read_for_schema_inference` 的默认值为 `25000`。 -* `input_format_max_bytes_to_read_for_schema_inference` 的默认值为 `33554432`(32 MB)。 - -#### column_names_for_schema_inference {#column-names-for-schema-inference} - -在对不含显式列名的格式进行模式推断时使用的列名列表。指定的名称将替代默认的 `c1,c2,c3,...`。格式:`column1,column2,column3,...`。 - -**示例** - -```sql -DESC format(TSV, 'Hello, World! 42 [1, 2, 3]') settings column_names_for_schema_inference = 'str,int,arr' -``` - -```response -┌─name─┬─type───────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ str │ Nullable(String) │ │ │ │ │ │ -│ int │ Nullable(Int64) │ │ │ │ │ │ -│ arr │ Array(Nullable(Int64)) │ │ │ │ │ │ -└──────┴────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_hints {#schema-inference-hints} - -在进行 schema 推断时,用于替代自动推断类型的列名和列类型列表。格式为:'column_name1 column_type1, column_name2 column_type2, ...'。 -该设置可用于指定无法自动确定类型的列的类型,或用于优化 schema。 - -**示例** - -```sql -DESC format(JSONEachRow, '{"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]}') SETTINGS schema_inference_hints = 'age LowCardinality(UInt8), status Nullable(String)', allow_suspicious_low_cardinality_types=1 -``` - -```response -┌─名称────┬─类型────────────────────┬─默认类型─────┬─默认表达式─────────┬─注释────┬─编解码表达式─────┬─TTL 表达式─────┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ LowCardinality(UInt8) │ │ │ │ │ │ -│ 名称 │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### schema_inference_make_columns_nullable $ {#schema-inference-make-columns-nullable} - - -控制在对缺少空值信息的格式进行 schema 推断时,是否将推断出的类型设为 `Nullable`。可能的取值: - -* 0 - 推断类型永远不会是 `Nullable`, -* 1 - 所有推断类型都将是 `Nullable`, -* 2 或 'auto' - 对于文本格式,仅当在 schema 推断期间解析的样本中该列包含 `NULL` 时,推断类型才会是 `Nullable`;对于强类型格式(Parquet、ORC、Arrow),空值信息来自文件元数据, -* 3 - 对于文本格式,一律使用 `Nullable`;对于强类型格式,使用文件元数据。 - -默认值:3。 - -**示例** - -```sql -SET schema_inference_make_columns_nullable = 1; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Int64) │ │ │ │ │ │ -│ age │ Nullable(Int64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 'auto'; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response -┌─name────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET schema_inference_make_columns_nullable = 0; -DESC format(JSONEachRow, $$ - {"id" : 1, "age" : 25, "name" : "Josh", "status" : null, "hobbies" : ["football", "cooking"]} - {"id" : 2, "age" : 19, "name" : "Alan", "status" : "married", "hobbies" : ["tennis", "art"]} - $$) -``` - -```response - -┌─name────┬─type──────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Int64 │ │ │ │ │ │ -│ age │ Int64 │ │ │ │ │ │ -│ name │ String │ │ │ │ │ │ -│ status │ String │ │ │ │ │ │ -│ hobbies │ Array(String) │ │ │ │ │ │ -└─────────┴───────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -#### input_format_try_infer_integers {#input-format-try-infer-integers} - -:::note -此设置不适用于 `JSON` 数据类型。 -::: - -启用后,ClickHouse 会在对文本格式进行模式推断时,优先尝试推断为整数而不是浮点数。 -如果示例数据中该列的所有数字都是整数,结果类型将为 `Int64`;如果至少有一个数字是浮点数,结果类型将为 `Float64`。 -如果示例数据仅包含整数,且至少有一个正整数超过了 `Int64` 的范围,ClickHouse 将推断为 `UInt64`。 - -默认启用。 - -**示例** - -```sql -SET input_format_try_infer_integers = 0 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_integers = 1 -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2} - $$) -``` - -```response -┌─name───┬─type────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Int64) │ │ │ │ │ │ -└────────┴─────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 18446744073709551615} - $$) -``` - -```response -┌─name───┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(UInt64) │ │ │ │ │ │ -└────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"number" : 1} - {"number" : 2.2} - $$) -``` - -```response -┌─name───┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ number │ Nullable(Float64) │ │ │ │ │ │ -└────────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes {#input-format-try-infer-datetimes} - -如果启用,ClickHouse 会在文本格式的模式推断时,尝试从字符串字段推断出 `DateTime` 或 `DateTime64` 类型。 -如果示例数据中某一列的所有字段都能成功解析为日期时间,则结果类型为 `DateTime`,或在任意日期时间值包含小数部分时为 `DateTime64(9)`; -如果至少有一个字段无法解析为日期时间,则结果类型为 `String`。 - -默认启用。 - -**示例** - - -```sql -SET input_format_try_infer_datetimes = 0; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_datetimes = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "unknown", "datetime64" : "unknown"} - $$) -``` - -```response -┌─name───────┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(String) │ │ │ │ │ │ -│ datetime64 │ Nullable(String) │ │ │ │ │ │ -└────────────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_datetimes_only_datetime64 {#input-format-try-infer-datetimes-only-datetime64} - -如果启用,在 `input_format_try_infer_datetimes` 启用的情况下,即使 datetime 值不包含小数部分,ClickHouse 也始终会推断为 `DateTime64(9)`。 - -默认禁用。 - -**示例** - -```sql -SET input_format_try_infer_datetimes = 1; -SET input_format_try_infer_datetimes_only_datetime64 = 1; -DESC format(JSONEachRow, $$ - {"datetime" : "2021-01-01 00:00:00", "datetime64" : "2021-01-01 00:00:00.000"} - {"datetime" : "2022-01-01 00:00:00", "datetime64" : "2022-01-01 00:00:00.000"} - $$) -``` - -```response -┌─name───────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ datetime │ Nullable(DateTime64(9)) │ │ │ │ │ │ -│ datetime64 │ Nullable(DateTime64(9)) │ │ │ │ │ │ -└────────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -注意:在进行模式推断时解析日期时间值,会遵守设置 [date_time_input_format](/operations/settings/settings-formats.md#date_time_input_format)。 - - -#### input_format_try_infer_dates {#input-format-try-infer-dates} - -启用后,ClickHouse 会在对文本格式进行 schema 推断时,尝试从字符串字段中推断出 `Date` 类型。 -如果示例数据中某一列的所有字段都能成功解析为日期,则结果类型为 `Date`, -如果至少有一个字段无法解析为日期,则结果类型为 `String`。 - -默认启用。 - -**示例** - -```sql -SET input_format_try_infer_datetimes = 0, input_format_try_infer_dates = 0 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -SET input_format_try_infer_dates = 1 -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "2022-01-01"} - $$) -``` - -```response -┌─name─┬─type───────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(Date) │ │ │ │ │ │ -└──────┴────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -```sql -DESC format(JSONEachRow, $$ - {"date" : "2021-01-01"} - {"date" : "unknown"} - $$) -``` - -```response -┌─name─┬─type─────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ date │ Nullable(String) │ │ │ │ │ │ -└──────┴──────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -#### input_format_try_infer_exponent_floats {#input-format-try-infer-exponent-floats} - -如果启用,ClickHouse 会在文本格式中尝试将指数形式的数字推断为浮点数(`JSON` 除外,在 `JSON` 中指数形式的数字始终会被推断为浮点数)。 - -默认禁用。 - -**示例** - -```sql -SET input_format_try_infer_exponent_floats = 1; -DESC format(CSV, -$$1.1E10 -2.3e-12 -42E00 -$$) -``` - -```response -┌─name─┬─type──────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ c1 │ Nullable(Float64) │ │ │ │ │ │ -└──────┴───────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## 自描述格式 {#self-describing-formats} - -自描述格式在数据本身中就包含关于数据结构的信息, -可以是带有描述的某种头部、二进制类型树,或者某种表结构。 -为了从此类格式的文件中自动推断表结构,ClickHouse 会读取包含类型信息的数据部分, -并将其转换为 ClickHouse 表的模式(schema)。 - -### 带有 -WithNamesAndTypes 后缀的格式 {#formats-with-names-and-types} - -ClickHouse 支持一些带有 -WithNamesAndTypes 后缀的文本格式。该后缀表示,在实际数据之前,数据中会额外包含两行内容,分别为列名和列类型。 -在对这类格式进行模式推断时,ClickHouse 会读取前两行并提取列名和列类型。 - -**示例** - -```sql -DESC format(TSVWithNamesAndTypes, -$$num str arr -UInt8 String Array(UInt8) -42 Hello, World! [1,2,3] -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### 带元数据的 JSON 格式 {#json-with-metadata} - -某些 JSON 输入格式([JSON](/interfaces/formats/JSON)、[JSONCompact](/interfaces/formats/JSONCompact)、[JSONColumnsWithMetadata](/interfaces/formats/JSONColumnsWithMetadata))包含列名和类型等元数据。 -在对这类格式进行模式推断时,ClickHouse 会读取这些元数据。 - -**示例** - -```sql -DESC format(JSON, $$ -{ - "meta": - [ - { - "name": "num", - "type": "UInt8" - }, - { - "name": "str", - "type": "String" - }, - { - "name": "arr", - "type": "Array(UInt8)" - } - ], - - "data": - [ - { - "num": 42, - "str": "Hello, World", - "arr": [1,2,3] - } - ], - - "rows": 1, - - "statistics": - { - "elapsed": 0.005723915, - "rows_read": 1, - "bytes_read": 1 - } -} -$$) -``` - -```response -┌─name─┬─type─────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ num │ UInt8 │ │ │ │ │ │ -│ str │ String │ │ │ │ │ │ -│ arr │ Array(UInt8) │ │ │ │ │ │ -└──────┴──────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### Avro {#avro} - -在 Avro 格式中,ClickHouse 从数据中读取模式(schema),并使用以下类型映射将其转换为 ClickHouse 的模式: - - -| Avro 数据类型 | ClickHouse 数据类型 | -|------------------------------------|--------------------------------------------------------------------------------| -| `boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int (date)` \* | [Date32](../sql-reference/data-types/date32.md) | -| `long` | [Int64](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `bytes`, `string` | [String](../sql-reference/data-types/string.md) | -| `fixed` | [FixedString(N)](../sql-reference/data-types/fixedstring.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `array(T)` | [Array(T)](../sql-reference/data-types/array.md) | -| `union(null, T)`, `union(T, null)` | [Nullable(T)](../sql-reference/data-types/date.md) | -| `null` | [Nullable(Nothing)](../sql-reference/data-types/special-data-types/nothing.md) | -| `string (uuid)` \* | [UUID](../sql-reference/data-types/uuid.md) | -| `binary (decimal)` \* | [Decimal(P, S)](../sql-reference/data-types/decimal.md) | - -\* [Avro 逻辑类型](https://avro.apache.org/docs/current/spec.html#Logical+Types) - -不支持其他 Avro 类型。 - -### Parquet {#parquet} - -在 Parquet 格式中,ClickHouse 从数据中读取 schema,并使用以下类型映射将其转换为 ClickHouse 的 schema: - -| Parquet 数据类型 | ClickHouse 数据类型 | -|------------------------------|---------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE` | [Date32](../sql-reference/data-types/date32.md) | -| `TIME (ms)` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME (us, ns)` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -不支持其他 Parquet 类型。 - -### Arrow {#arrow} - -在 Arrow 格式中,ClickHouse 从数据中读取 schema,并使用以下类型映射将其转换为 ClickHouse 的 schema: - - - -| Arrow 数据类型 | ClickHouse 数据类型 | -|---------------------------------|---------------------------------------------------------| -| `BOOL` | [Bool](../sql-reference/data-types/boolean.md) | -| `UINT8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `INT8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UINT16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `INT16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UINT32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `INT32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UINT64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `INT64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `FLOAT`, `HALF_FLOAT` | [Float32](../sql-reference/data-types/float.md) | -| `DOUBLE` | [Float64](../sql-reference/data-types/float.md) | -| `DATE32` | [Date32](../sql-reference/data-types/date32.md) | -| `DATE64` | [DateTime](../sql-reference/data-types/datetime.md) | -| `TIMESTAMP`, `TIME32`, `TIME64` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `STRING`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `DECIMAL128`, `DECIMAL256` | [Decimal](../sql-reference/data-types/decimal.md) | -| `LIST` | [Array](../sql-reference/data-types/array.md) | -| `STRUCT` | [Tuple](../sql-reference/data-types/tuple.md) | -| `MAP` | [Map](../sql-reference/data-types/map.md) | - -不支持其他 Arrow 类型。 - -### ORC {#orc} - -在 ORC 格式中,ClickHouse 从数据中读取模式(schema),并使用下表中的类型对应关系将其转换为 ClickHouse 模式: - -| ORC 数据类型 | ClickHouse 数据类型 | -|--------------------------------------|---------------------------------------------------------| -| `Boolean` | [Bool](../sql-reference/data-types/boolean.md) | -| `Tinyint` | [Int8](../sql-reference/data-types/int-uint.md) | -| `Smallint` | [Int16](../sql-reference/data-types/int-uint.md) | -| `Int` | [Int32](../sql-reference/data-types/int-uint.md) | -| `Bigint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `Float` | [Float32](../sql-reference/data-types/float.md) | -| `Double` | [Float64](../sql-reference/data-types/float.md) | -| `Date` | [Date32](../sql-reference/data-types/date32.md) | -| `Timestamp` | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `String`, `Char`, `Varchar`, `BINARY` | [String](../sql-reference/data-types/string.md) | -| `Decimal` | [Decimal](../sql-reference/data-types/decimal.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `Struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `Map` | [Map](../sql-reference/data-types/map.md) | - -不支持其他 ORC 类型。 - -### Native {#native} - -Native 格式在 ClickHouse 内部使用,并在数据中包含模式(schema)。 -在模式推断时,ClickHouse 直接从数据中读取模式,而不进行任何转换。 - - - -## 具有外部模式的格式 {#formats-with-external-schema} - -此类格式需要在单独的文件中,使用特定的模式语言来描述数据的模式。 -要从此类格式的文件中自动推断模式,ClickHouse 会从单独的文件中读取外部模式,并将其转换为 ClickHouse 表的模式定义。 - -### Protobuf {#protobuf} - -在对 Protobuf 格式进行模式推断时,ClickHouse 使用以下类型对应关系: - -| Protobuf data type | ClickHouse data type | -|-------------------------------|---------------------------------------------------| -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `float` | [Float32](../sql-reference/data-types/float.md) | -| `double` | [Float64](../sql-reference/data-types/float.md) | -| `int32`, `sint32`, `sfixed32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `int64`, `sint64`, `sfixed64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `uint32`, `fixed32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `uint64`, `fixed64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `string`, `bytes` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `repeated T` | [Array(T)](../sql-reference/data-types/array.md) | -| `message`, `group` | [Tuple](../sql-reference/data-types/tuple.md) | - -### CapnProto {#capnproto} - -在对 CapnProto 格式进行模式推断时,ClickHouse 使用以下类型对应关系: - -| CapnProto data type | ClickHouse data type | -|------------------------------------|--------------------------------------------------------| -| `Bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int8` | [Int8](../sql-reference/data-types/int-uint.md) | -| `UInt8` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `Int16` | [Int16](../sql-reference/data-types/int-uint.md) | -| `UInt16` | [UInt16](../sql-reference/data-types/int-uint.md) | -| `Int32` | [Int32](../sql-reference/data-types/int-uint.md) | -| `UInt32` | [UInt32](../sql-reference/data-types/int-uint.md) | -| `Int64` | [Int64](../sql-reference/data-types/int-uint.md) | -| `UInt64` | [UInt64](../sql-reference/data-types/int-uint.md) | -| `Float32` | [Float32](../sql-reference/data-types/float.md) | -| `Float64` | [Float64](../sql-reference/data-types/float.md) | -| `Text`, `Data` | [String](../sql-reference/data-types/string.md) | -| `enum` | [Enum](../sql-reference/data-types/enum.md) | -| `List` | [Array](../sql-reference/data-types/array.md) | -| `struct` | [Tuple](../sql-reference/data-types/tuple.md) | -| `union(T, Void)`, `union(Void, T)` | [Nullable(T)](../sql-reference/data-types/nullable.md) | - - - -## 强类型二进制格式 {#strong-typed-binary-formats} - -在此类格式中,每个序列化值都包含其类型的信息(以及可能包含其名称的信息),但不会包含关于整个表的信息。 -在对这类格式进行模式推断时,ClickHouse 会逐行读取数据(最多读取 `input_format_max_rows_to_read_for_schema_inference` 行或 `input_format_max_bytes_to_read_for_schema_inference` 字节),并从数据中提取 -每个值的类型(以及可能的名称),然后将这些类型转换为 ClickHouse 类型。 - -### MsgPack {#msgpack} - -在 MsgPack 格式中,行与行之间没有分隔符。要对该格式使用模式推断,你需要通过设置 `input_format_msgpack_number_of_columns` 指定表中的列数。 -ClickHouse 使用以下类型对应关系: - -| MessagePack 数据类型(`INSERT`) | ClickHouse 数据类型 | -|--------------------------------------------------------------------|-----------------------------------------------------------| -| `int N`, `uint N`, `negative fixint`, `positive fixint` | [Int64](../sql-reference/data-types/int-uint.md) | -| `bool` | [UInt8](../sql-reference/data-types/int-uint.md) | -| `fixstr`, `str 8`, `str 16`, `str 32`, `bin 8`, `bin 16`, `bin 32` | [String](../sql-reference/data-types/string.md) | -| `float 32` | [Float32](../sql-reference/data-types/float.md) | -| `float 64` | [Float64](../sql-reference/data-types/float.md) | -| `uint 16` | [Date](../sql-reference/data-types/date.md) | -| `uint 32` | [DateTime](../sql-reference/data-types/datetime.md) | -| `uint 64` | [DateTime64](../sql-reference/data-types/datetime.md) | -| `fixarray`, `array 16`, `array 32` | [Array](../sql-reference/data-types/array.md) | -| `fixmap`, `map 16`, `map 32` | [Map](../sql-reference/data-types/map.md) | - -默认情况下,所有推断出的类型都会被包装在 `Nullable` 中,但可以通过设置 `schema_inference_make_columns_nullable` 来更改这一行为。 - -### BSONEachRow {#bsoneachrow} - -在 BSONEachRow 格式中,每一行数据都表示为一个 BSON 文档。在进行模式推断时,ClickHouse 会逐个读取 BSON 文档,并从数据中提取 -值、名称和类型,然后使用下表所示的类型对应关系将这些类型转换为 ClickHouse 类型: - -| BSON 类型 | ClickHouse 类型 | -|-----------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------| -| `\x08` boolean | [Bool](../sql-reference/data-types/boolean.md) | -| `\x10` int32 | [Int32](../sql-reference/data-types/int-uint.md) | -| `\x12` int64 | [Int64](../sql-reference/data-types/int-uint.md) | -| `\x01` double | [Float64](../sql-reference/data-types/float.md) | -| `\x09` datetime | [DateTime64](../sql-reference/data-types/datetime64.md) | -| `\x05` binary with`\x00` binary subtype, `\x02` string, `\x0E` symbol, `\x0D` JavaScript code | [String](../sql-reference/data-types/string.md) | -| `\x07` ObjectId, | [FixedString(12)](../sql-reference/data-types/fixedstring.md) | -| `\x05` binary with `\x04` uuid subtype, size = 16 | [UUID](../sql-reference/data-types/uuid.md) | -| `\x04` array | [Array](../sql-reference/data-types/array.md)/[Tuple](../sql-reference/data-types/tuple.md)(如果内部元素类型不相同) | -| `\x03` document | [Named Tuple](../sql-reference/data-types/tuple.md)/[Map](../sql-reference/data-types/map.md)(键为 String) | - -默认情况下,所有推断出的类型都会被包装在 `Nullable` 中,但可以通过设置 `schema_inference_make_columns_nullable` 来更改这一行为。 - - - -## 具有固定 schema 的格式 {#formats-with-constant-schema} - -此类格式中的数据始终使用相同的 schema。 - -### LineAsString {#line-as-string} - -在此格式中,ClickHouse 会将数据中的整行内容读取到一个 `String` 类型的单列中。此格式推断的数据类型始终为 `String`,列名为 `line`。 - -**示例** - -```sql -DESC format(LineAsString, 'Hello\nworld!') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ line │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsString {#json-as-string} - -在这种格式下,ClickHouse 会将数据中的整个 JSON 对象读取到一个 `String` 类型的单列中。此格式推断出的类型始终为 `String`,且列名为 `json`。 - -**示例** - -```sql -DESC format(JSONAsString, '{"x" : 42, "y" : "Hello, World!"}') -``` - -```response -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ String │ │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -### JSONAsObject {#json-as-object} - -在这种格式下,ClickHouse 会将数据中的整个 JSON 对象读取到一个单独的列中,该列的数据类型为 `JSON`。此格式推断的数据类型始终为 `JSON`,且列名为 `json`。 - -**示例** - -```sql -DESC format(JSONAsObject, '{"x" : 42, "y" : "Hello, World!"}'); -``` - -```response -┌─name─┬─type─┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ json │ JSON │ │ │ │ │ │ -└──────┴──────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - - -## Schema 推断模式 {#schema-inference-modes} - -从一组数据文件中进行 Schema 推断时,可以使用两种不同的工作模式:`default` 和 `union`。 -工作模式由设置项 `schema_inference_mode` 控制。 - -### 默认模式 {#default-schema-inference-mode} - -在默认模式下,ClickHouse 假定所有文件具有相同的 Schema,并尝试通过逐个读取文件来推断 Schema,直到成功为止。 - -示例: - -假设我们有 3 个文件 `data1.jsonl`、`data2.jsonl` 和 `data3.jsonl`,其内容如下: - -`data1.jsonl`: - -```json -{"field1" : 1, "field2" : null} -{"field1" : 2, "field2" : null} -{"field1" : 3, "field2" : null} -``` - -`data2.jsonl`: - -```json -{"field1" : 4, "field2" : "Data4"} -{"field1" : 5, "field2" : "Data5"} -{"field1" : 6, "field2" : "Data5"} -``` - -`data3.jsonl`: - -```json -{"field1" : 7, "field2" : "Data7", "field3" : [1, 2, 3]} -{"field1" : 8, "field2" : "Data8", "field3" : [4, 5, 6]} -{"field1" : 9, "field2" : "Data9", "field3" : [7, 8, 9]} -``` - -让我们试着对这 3 个文件进行模式推断: - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='default' -``` - -结果: - -```response -┌─name───┬─type─────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -└────────┴──────────────────┘ -``` - -正如我们所见,数据中没有来自文件 `data3.jsonl` 的字段 `field3`。 -这是因为 ClickHouse 首先尝试从文件 `data1.jsonl` 推断 schema,但由于字段 `field2` 仅包含 null 值而失败, -然后又尝试从 `data2.jsonl` 推断 schema 并成功,因此文件 `data3.jsonl` 中的数据未被读取。 - -### Union 模式 {#default-schema-inference-mode-1} - -在 union 模式下,ClickHouse 假定文件可以具有不同的 schema,因此会推断所有文件的 schema,然后将它们合并为一个公共 schema。 - -假设我们有 3 个文件 `data1.jsonl`、`data2.jsonl` 和 `data3.jsonl`,其内容如下: - -`data1.jsonl`: - -```json -{"field1" : 1} -{"field1" : 2} -{"field1" : 3} -``` - -`data2.jsonl`: - -```json -{"field2" : "Data4"} -{"field2" : "Data5"} -{"field2" : "Data5"} -``` - -`data3.jsonl`: - -```json -{"field3" : [1, 2, 3]} -{"field3" : [4, 5, 6]} -{"field3" : [7, 8, 9]} -``` - -我们来对这 3 个文件使用 schema 推断: - -```sql -:) DESCRIBE file('data{1,2,3}.jsonl') SETTINGS schema_inference_mode='union' -``` - -结果: - -```response -┌─name───┬─type───────────────────┐ -│ field1 │ Nullable(Int64) │ -│ field2 │ Nullable(String) │ -│ field3 │ Array(Nullable(Int64)) │ -└────────┴────────────────────────┘ -``` - -可以看到,我们已经从所有文件中获取到了全部字段。 - -注意: - -* 由于某些文件可能不包含最终 schema 中的某些列,`union` 模式仅适用于支持按列子集读取的格式(例如 `JSONEachRow`、`Parquet`、`TSVWithNames` 等),对其他格式(例如 `CSV`、`TSV`、`JSONCompactEachRow` 等)将无法使用。 -* 如果 ClickHouse 无法从某个文件推断出 schema,将会抛出异常。 -* 如果你有大量文件,从所有文件中读取 schema 可能会花费很多时间。 - - -## 自动格式检测 {#automatic-format-detection} - -如果未指定数据格式且无法通过文件扩展名确定,ClickHouse 将尝试根据文件内容检测文件格式。 - -**示例:** - -假设我们有一个名为 `data` 的文件,内容如下: - -```csv -"a","b" -1,"Data1" -2,"Data2" -3,"Data3" -``` - -我们可以在无需指定格式或结构的情况下查看和查询此文件: - -```sql -:) desc file(data); -``` - -```repsonse -┌─name─┬─type─────────────┐ -│ a │ Nullable(Int64) │ -│ b │ Nullable(String) │ -└──────┴──────────────────┘ -``` - -```sql -:) select * from file(data); -``` - -```response -┌─a─┬─b─────┐ -│ 1 │ Data1 │ -│ 2 │ Data2 │ -│ 3 │ Data3 │ -└───┴───────┘ -``` - -:::note -ClickHouse 只能检测部分格式,而且这种检测会花费一定时间,因此最好始终显式指定格式。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/ssh.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/ssh.md deleted file mode 100644 index 8d83b09a728..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/ssh.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: 'ClickHouse 中 SSH 接口文档' -keywords: ['client', 'ssh', 'putty'] -sidebar_label: 'SSH 接口' -sidebar_position: 60 -slug: /interfaces/ssh -title: 'SSH 接口' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# 支持 PTY 的 SSH 接口 {#ssh-interface-with-pty} - - - - - -## 前言 {#preface} - -ClickHouse 服务器允许客户端通过 SSH 协议直接连接到服务器本身。任何客户端都可以连接。 - -在创建了一个[以 SSH 密钥标识的数据库用户](/knowledgebase/how-to-connect-to-ch-cloud-using-ssh-keys)之后: - -```sql -CREATE USER abcuser IDENTIFIED WITH ssh_key BY KEY '<已隐藏>' TYPE 'ssh-ed25519'; -``` - -你可以使用此密钥连接到 ClickHouse 服务器。这会打开一个伪终端(PTY),并启动一个交互式的 clickhouse-client 会话。 - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 -ClickHouse embedded version 25.1.1.1. - -ip-10-1-13-116.us-west-2.compute.internal :) SELECT 1; - -SELECT 1 - -Query id: cdd91b7f-215b-4537-b7df-86d19bf63f64 - - ┌─1─┐ -1. │ 1 │ - └───┘ - -1 row in set. Elapsed: 0.002 sec. -``` - -还支持通过 SSH 以非交互模式执行命令: - -```bash -> ssh -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "select 1" -1 -``` - -## 服务器配置 {#server-configuration} - -要启用 SSH 服务器功能,需要在 `config.xml` 文件中取消对以下部分的注释,或将其添加到文件中: - -```xml -9022 - - 密钥文件路径 - - - -``` - -主机密钥是 SSH 协议中的重要组成部分。该密钥的公钥部分保存在客户端的 `~/.ssh/known_hosts` 文件中,通常用于防止中间人攻击。首次连接到服务器时,您会看到类似下面的消息: - -```shell -无法确认主机 '[localhost]:9022 ([127.0.0.1]:9022)' 的真实性。 -RSA 密钥指纹为 SHA256:3qxVlJKMr/PEKw/hfeg06HAK451Tt0eenhwqQvh58Do。 -此密钥未被任何其他名称识别。 -确定要继续连接吗(yes/no/[fingerprint])? -``` - -事实上,这句话的意思是:“是否要记住该主机的公钥并继续连接?” - -你可以通过添加一个参数,让 SSH 客户端跳过主机验证: - -```bash -ssh -o "StrictHostKeyChecking no" user@host -``` - -## 配置嵌入式客户端 {#configuring-embedded-client} - -您可以像使用普通的 `clickhouse-client` 一样向嵌入式客户端传递选项,但会有一些限制。 -由于使用的是 SSH 协议,向目标主机传递参数的唯一方式是通过环境变量。 - -例如,可以通过以下方式设置 `format`: - -```bash -> ssh -o SetEnv="format=Pretty" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` - -你可以通过这种方式更改任意用户级设置,并且还可以额外传递大多数常规的 `clickhouse-client` 选项(不包括在此场景下没有意义的那些选项)。 - -重要说明: - -如果同时传递了 `query` 选项和 SSH 命令,则后者会被添加到要执行的查询列表中: - -```bash -ubuntu ip-10-1-13-116@~$ ssh -o SetEnv="format=Pretty query=\"SELECT 2;\"" -i ~/test_ssh/id_ed25519 abcuser@localhost -p 9022 "SELECT 1" - ┏━━━┓ - ┃ 2 ┃ - ┡━━━┩ -1. │ 2 │ - └───┘ - ┏━━━┓ - ┃ 1 ┃ - ┡━━━┩ -1. │ 1 │ - └───┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/tcp.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/tcp.md deleted file mode 100644 index b31a696589e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/tcp.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ClickHouse 原生 TCP 接口文档' -sidebar_label: '原生接口(TCP)' -sidebar_position: 18 -slug: /interfaces/tcp -title: '原生接口(TCP)' -doc_type: 'reference' ---- - -# 原生接口(TCP) {#native-interface-tcp} - -原生协议用于[命令行客户端](../interfaces/cli.md)、分布式查询处理期间的服务器间通信,以及其他 C++ 程序。不幸的是,ClickHouse 原生协议目前还没有正式规范,但可以通过对 ClickHouse 源码进行逆向分析(从[此处附近](https://github.com/ClickHouse/ClickHouse/tree/master/src/Client)开始)和/或拦截并分析 TCP 流量来加以还原。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md deleted file mode 100644 index ae64b43eee8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/client-libraries.md +++ /dev/null @@ -1,119 +0,0 @@ ---- -description: '针对不同编程语言的可用第三方客户端库概览' -sidebar_label: '客户端库' -sidebar_position: 26 -slug: /interfaces/third-party/client-libraries -title: '第三方开发者提供的客户端库' -doc_type: 'reference' ---- - -# 第三方开发者提供的客户端库 {#client-libraries-from-third-party-developers} - -:::note -ClickHouse Inc **不**负责维护下列库,也未对其质量进行任何广泛测试。 -::: - -### Python {#python} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm) -* [clickhouse-driver](https://github.com/mymarilyn/clickhouse-driver) -* [clickhouse-client](https://github.com/yurial/clickhouse-client) -* [aiochclient](https://github.com/maximdanilchenko/aiochclient) -* [asynch](https://github.com/long2ice/asynch) - -### PHP {#php} - -* [smi2/phpclickhouse](https://packagist.org/packages/smi2/phpClickHouse) -* [8bitov/clickhouse-php-client](https://packagist.org/packages/8bitov/clickhouse-php-client) -* [bozerkins/clickhouse-client](https://packagist.org/packages/bozerkins/clickhouse-client) -* [simpod/clickhouse-client](https://packagist.org/packages/simpod/clickhouse-client) -* [seva-code/php-click-house-client](https://packagist.org/packages/seva-code/php-click-house-client) -* [SeasClick C++ 客户端](https://github.com/SeasX/SeasClick) -* [one-ck](https://github.com/lizhichao/one-ck) -* [glushkovds/phpclickhouse-laravel](https://packagist.org/packages/glushkovds/phpclickhouse-laravel) -* [glushkovds/php-clickhouse-schema-builder](https://packagist.org/packages/glushkovds/php-clickhouse-schema-builder) -* [kolya7k ClickHouse PHP 扩展](https://github.com//kolya7k/clickhouse-php) -* [hyvor/clickhouse-php](https://github.com/hyvor/clickhouse-php) - -### Go {#go} - -* [ClickHouse](https://github.com/kshvakov/clickhouse/) -* [go-clickhouse](https://github.com/roistat/go-clickhouse) -* [chconn](https://github.com/vahid-sohrabloo/chconn) -* [mailrugo-clickhouse](https://github.com/mailru/go-clickhouse) -* [golang-clickhouse](https://github.com/leprosus/golang-clickhouse) -* [uptrace/go-clickhouse](https://clickhouse.uptrace.dev/) - -### Swift {#swift} - -* [ClickHouseNIO](https://github.com/patrick-zippenfenig/ClickHouseNIO) -* [ClickHouseVapor ORM](https://github.com/patrick-zippenfenig/ClickHouseVapor) - -### NodeJs {#nodejs} - -* [Moose OLAP](https://docs.fiveonefour.com/moose/olap) -* [ClickHouse(Node.js)](https://github.com/TimonKK/clickhouse) -* [node-clickhouse](https://github.com/apla/node-clickhouse) -* [nestjs-clickhouse](https://github.com/depyronick/nestjs-clickhouse) -* [clickhouse-client](https://github.com/depyronick/clickhouse-client) -* [node-clickhouse-orm](https://github.com/zimv/node-clickhouse-orm) -* [clickhouse-ts](https://github.com/bytadaniel/clickhouse-ts) -* [clickcache](https://github.com/bytadaniel/clickcache) - -### Perl {#perl} - -* [perl-DBD-ClickHouse](https://github.com/elcamlost/perl-DBD-ClickHouse) -* [HTTP-ClickHouse](https://metacpan.org/release/HTTP-ClickHouse) -* [AnyEvent-ClickHouse](https://metacpan.org/release/AnyEvent-ClickHouse) - -### Ruby {#ruby} - -* [ClickHouse(Ruby)](https://github.com/shlima/click_house) -* [clickhouse-activerecord](https://github.com/PNixx/clickhouse-activerecord) - -### Rust {#rust} - -* [clickhouse.rs](https://github.com/loyd/clickhouse.rs) -* [clickhouse-rs](https://github.com/suharev7/clickhouse-rs) -* [Klickhouse](https://github.com/Protryon/klickhouse) - -### R {#r} - -* [RClickHouse](https://github.com/IMSMWU/RClickHouse) - -### Java {#java} - -* [clickhouse-client-java](https://github.com/VirtusAI/clickhouse-client-java) -* [clickhouse-client](https://github.com/Ecwid/clickhouse-client) - -### Scala {#scala} - -* [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) - -### Kotlin {#kotlin} - -* [AORM](https://github.com/TanVD/AORM) - -### C# {#c} - -* [Octonica.ClickHouseClient](https://github.com/Octonica/ClickHouseClient) -* [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) -* [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) -* [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - -### Elixir {#elixir} - -* [clickhousex](https://github.com/appodeal/clickhousex/) -* [pillar](https://github.com/sofakingworld/pillar) -* [ecto_ch](https://github.com/plausible/ecto_ch) -* [req_ch](https://github.com/livebook-dev/req_ch) - -### Nim {#nim} - -* [nim-clickhouse](https://github.com/leonardoce/nim-clickhouse) - -### Haskell {#haskell} - -* [hdbc-clickhouse](https://github.com/zaneli/hdbc-clickhouse) -* [ClickHaskell](https://clickhaskell.dev/) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md deleted file mode 100644 index 3de0f498e2b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/gui.md +++ /dev/null @@ -1,444 +0,0 @@ ---- -description: '用于操作 ClickHouse 的第三方图形界面(GUI)工具和应用程序列表' -sidebar_label: '图形界面' -sidebar_position: 28 -slug: /interfaces/third-party/gui -title: '第三方开发者提供的图形界面' -doc_type: 'reference' ---- - - - -# 第三方开发者的可视化界面 {#visual-interfaces-from-third-party-developers} - - - -## 开源 {#open-source} - -### agx {#agx} - -[agx](https://github.com/agnosticeng/agx) 是一款使用 Tauri 和 SvelteKit 构建的桌面应用程序,为使用 ClickHouse 内置数据库引擎(chdb)进行数据探索和查询提供了现代化界面。 - -- 在运行原生应用时利用 chdb。 -- 在运行 Web 实例时可以连接到 ClickHouse 实例。 -- 集成 Monaco 编辑器,带来熟悉的编辑体验。 -- 提供多种且持续演进的数据可视化方式。 - -### ch-ui {#ch-ui} - -[ch-ui](https://github.com/caioricciuti/ch-ui) 是一个为 ClickHouse 数据库设计的简洁 React.js 应用界面,用于执行查询和可视化数据。基于 React 和 ClickHouse Web 客户端构建,提供简洁且易用的 UI,便于进行数据库交互。 - -功能特性: - -- ClickHouse 集成:轻松管理连接并执行查询。 -- 响应式标签页管理:可动态处理多个标签页,例如查询标签页和数据表标签页。 -- 性能优化:使用 IndexedDB 进行高效缓存和状态管理。 -- 本地数据存储:所有数据都存储在浏览器本地,不会发送到任何其他地方。 - -### ChartDB {#chartdb} - -[ChartDB](https://chartdb.io) 是一个免费开源工具,可通过单条查询对包括 ClickHouse 在内的数据库 schema 进行可视化和设计。基于 React 构建,提供无缝且易用的体验,无需数据库凭证或注册即可开始使用。 - -功能特性: - -- schema 可视化:即时导入并可视化你的 ClickHouse schema,包括带有物化视图和标准视图的 ER 图,并展示对数据表的引用关系。 -- AI 驱动的 DDL 导出:轻松生成 DDL 脚本,以改进 schema 管理和文档化。 -- 多 SQL 方言支持:兼容多种 SQL 方言,可用于不同的数据库环境。 -- 无需注册或凭证:所有功能都可直接在浏览器中使用,体验顺畅且安全。 - -[ChartDB 源代码](https://github.com/chartdb/chartdb)。 - -### DataPup {#datapup} - -[DataPup](https://github.com/DataPupOrg/DataPup) 是一款现代的、AI 辅助的跨平台数据库客户端,对 ClickHouse 提供原生支持。 - -功能特性: - -- AI 驱动的 SQL 查询辅助,提供智能建议 -- 原生 ClickHouse 连接支持,并安全处理凭证 -- 美观、易用且无障碍的界面,支持多种主题(浅色、深色及多彩变体) -- 高级查询结果过滤与探索能力 -- 跨平台支持(macOS、Windows、Linux) -- 快速且响应灵敏的性能 -- 开源并采用 MIT 许可证 - -### ClickHouse Schema Flow Visualizer {#clickhouse-schemaflow-visualizer} - -[ClickHouse Schema Flow Visualizer](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) 是一个用于通过 Mermaid.js 图表可视化 ClickHouse 表关系的强大开源 Web 应用。你可以通过直观界面浏览数据库和数据表,利用可选的行数与大小信息探索表元数据,并导出交互式 schema 图。 - -功能特性: - -- 通过直观界面浏览 ClickHouse 数据库和数据表 -- 使用 Mermaid.js 图表可视化表之间的关系 -- 使用与表类型匹配的颜色编码图标,以获得更佳可视化效果 -- 查看数据在数据表之间流动的方向 -- 将图表导出为独立的 HTML 文件 -- 切换元数据可见性(表行数和大小信息) -- 通过 TLS 与 ClickHouse 建立安全连接 -- 适配所有设备的响应式 Web 界面 - -[ClickHouse Schema Flow Visualizer - 源代码](https://github.com/FulgerX2007/clickhouse-schemaflow-visualizer) - -### Tabix {#tabix} - -[Tabix](https://github.com/tabixio/tabix) 项目中的 ClickHouse Web 界面。 - -功能特性: - -- 直接通过浏览器与 ClickHouse 交互,无需安装额外软件。 -- 带语法高亮的查询编辑器。 -- 命令自动补全。 -- 用于图形化分析查询执行情况的工具。 -- 多种配色方案选项。 - -[Tabix 文档](https://tabix.io/doc/)。 - -### HouseOps {#houseops} - -[HouseOps](https://github.com/HouseOps/HouseOps) 是适用于 macOS、Linux 和 Windows 的 UI/IDE。 - -功能特性: - -- 带语法高亮的查询构建器,可通过表格或 JSON 视图查看响应。 -- 将查询结果导出为 CSV 或 JSON。 -- 带描述的进程列表,支持写入模式,并可停止(`KILL`)进程。 -- 数据库拓扑图,展示所有数据表及其列,并附带额外信息。 -- 快速查看列大小。 -- 服务器配置管理。 - -以下功能计划后续开发: - -- 数据库管理。 -- 用户管理。 -- 实时数据分析。 -- 集群监控。 -- 集群管理。 -- 监控复制表和 Kafka 表。 - -### LightHouse {#lighthouse} - - - -[LightHouse](https://github.com/VKCOM/lighthouse) 是一个适用于 ClickHouse 的轻量级 Web 界面。 - -功能: - -- 带有过滤和元数据的表列表。 -- 带有过滤和排序功能的表预览。 -- 只读查询执行。 - -### Redash {#redash} - -[Redash](https://github.com/getredash/redash) 是一个数据可视化平台。 - -Redash 支持包括 ClickHouse 在内的多种数据源,可以将来自不同数据源的查询结果关联为一个最终数据集。 - -功能: - -- 功能强大的查询编辑器。 -- 数据库浏览器。 -- 可视化工具,允许你以不同形式呈现数据。 - -### Grafana {#grafana} - -[Grafana](https://grafana.com/grafana/plugins/grafana-clickhouse-datasource/) 是一个监控和可视化平台。 - -“Grafana allows you to query, visualize, alert on and understand your metrics no matter where they are stored. Create, explore, and share dashboards with your team and foster a data-driven culture. Trusted and loved by the community” — grafana.com。 - -ClickHouse 数据源插件为 ClickHouse 作为后端数据库提供支持。 - -### qryn {#qryn} - -[qryn](https://metrico.in) 是一个基于 ClickHouse 的多协议高性能可观测性栈 _(前身为 cLoki)_,与 Grafana 原生集成,允许用户从任何支持 Loki/LogQL、Prometheus/PromQL、OTLP/Tempo、Elastic、InfluxDB 等众多代理中摄取并分析日志、指标和遥测追踪。 - -功能: - -- 内置 Explore UI 和 LogQL CLI,用于查询、提取和可视化数据 -- 原生 Grafana API 支持,无需插件即可完成查询、处理、摄取、追踪和告警 -- 功能强大的流水线,可动态搜索、过滤并从日志、事件、追踪等中提取数据 -- 摄取和 PUSH API 与 LogQL、PromQL、InfluxDB、Elastic 等透明兼容 -- 开箱即用,可与 Promtail、Grafana-Agent、Vector、Logstash、Telegraf 等多种 Agent 搭配使用 - -### DBeaver {#dbeaver} - -[DBeaver](https://dbeaver.io/) —— 通用桌面数据库客户端,支持 ClickHouse。 - -功能: - -- 支持语法高亮和自动补全的查询开发。 -- 带过滤器和元数据搜索的表列表。 -- 表数据预览。 -- 全文搜索。 - -默认情况下,DBeaver 连接时不使用会话(例如 CLI 会使用会话)。如果你需要会话支持(例如为你的会话设置参数),请编辑驱动连接属性并将 `session_id` 设置为一个随机字符串(其底层使用 HTTP 连接)。然后你就可以在查询窗口中使用任意设置。 - -### clickhouse-cli {#clickhouse-cli} - -[clickhouse-cli](https://github.com/hatarist/clickhouse-cli) 是一个用 Python 3 编写的 ClickHouse 命令行客户端替代工具。 - -功能: - -- 自动补全。 -- 查询和数据输出的语法高亮。 -- 数据输出的分页器支持。 -- 自定义类似 PostgreSQL 的命令。 - -### clickhouse-flamegraph {#clickhouse-flamegraph} - -[clickhouse-flamegraph](https://github.com/Slach/clickhouse-flamegraph) 是一个专门用于将 `system.trace_log` 可视化为 [flamegraph](http://www.brendangregg.com/flamegraphs.html) 的工具。 - -### clickhouse-plantuml {#clickhouse-plantuml} - -[cickhouse-plantuml](https://pypi.org/project/clickhouse-plantuml/) 是一个用于生成表结构 [PlantUML](https://plantuml.com/) 图的脚本。 - -### ClickHouse table graph {#clickhouse-table-graph} - -[ClickHouse table graph](https://github.com/mbaksheev/clickhouse-table-graph) 是一个用于可视化 ClickHouse 表之间依赖关系的简单 CLI 工具。该工具从 `system.tables` 表中检索表之间的连接关系,并以 [mermaid](https://mermaid.js.org/syntax/flowchart.html) 格式构建依赖流程图。借助该工具,你可以轻松可视化表依赖关系,并理解 ClickHouse 数据库中的数据流。得益于 mermaid,生成的流程图十分美观,并且可以轻松添加到你的 Markdown 文档中。 - -### xeus-clickhouse {#xeus-clickhouse} - -[xeus-clickhouse](https://github.com/wangfenjin/xeus-clickhouse) 是一个适用于 ClickHouse 的 Jupyter 内核,支持在 Jupyter 中使用 SQL 查询 ClickHouse 数据。 - -### MindsDB Studio {#mindsdb} - - - -[MindsDB](https://mindsdb.com/) 是一个面向包括 ClickHouse 在内的数据库的开源 AI 层,可以让你轻松开发、训练和部署最先进的机器学习模型。MindsDB Studio(GUI)可以从数据库中训练新模型、解释模型生成的预测结果、识别潜在的数据偏差,并使用可解释 AI 功能评估和可视化模型精度,从而更快速地调整和优化你的机器学习模型。 - -### DBM {#dbm} - -[DBM](https://github.com/devlive-community/dbm) DBM 是一个用于 ClickHouse 的可视化管理工具! - -特性: - -- 支持查询历史(分页、清空等) -- 支持选中 SQL 片段执行查询 -- 支持终止查询 -- 支持表管理(元数据、删除、预览) -- 支持数据库管理(删除、创建) -- 支持自定义查询 -- 支持多数据源管理(连接测试、监控) -- 支持监控(处理器、连接、查询) -- 支持数据迁移 - -### Bytebase {#bytebase} - -[Bytebase](https://bytebase.com) 是一个基于 Web 的开源架构变更和版本控制工具,面向团队使用。它支持包括 ClickHouse 在内的多种数据库。 - -特性: - -- 支持开发人员与 DBA 之间的架构评审。 -- Database-as-Code,将架构在 GitLab 等 VCS 中进行版本控制,并在代码提交时触发部署。 -- 基于环境策略的简化部署流程。 -- 完整的迁移历史。 -- 架构漂移检测。 -- 备份与恢复。 -- RBAC(基于角色的访问控制)。 - -### Zeppelin-Interpreter-for-ClickHouse {#zeppelin-interpreter-for-clickhouse} - -[Zeppelin-Interpreter-for-ClickHouse](https://github.com/SiderZhang/Zeppelin-Interpreter-for-ClickHouse) 是一个适用于 ClickHouse 的 [Zeppelin](https://zeppelin.apache.org) 解释器。与 JDBC 解释器相比,它可以为长时间运行的查询提供更好的超时控制。 - -### ClickCat {#clickcat} - -[ClickCat](https://github.com/clickcat-project/ClickCat) 是一个直观友好的用户界面,可用于搜索、探索和可视化你的 ClickHouse 数据。 - -特性: - -- 一个在线 SQL 编辑器,无需任何安装即可运行你的 SQL 代码。 -- 你可以查看所有进程和变更(mutations)。对于尚未完成的进程,可以在 UI 中将其终止。 -- 指标包括集群分析、数据分析和查询分析。 - -### ClickVisual {#clickvisual} - -[ClickVisual](https://clickvisual.net/) ClickVisual 是一个轻量级的开源日志查询、分析与告警可视化平台。 - -特性: - -- 支持一键创建分析日志库 -- 支持日志采集配置管理 -- 支持用户自定义索引配置 -- 支持告警配置 -- 支持将权限粒度细化到库级和表级的权限配置 - -### ClickHouse-Mate {#clickmate} - -[ClickHouse-Mate](https://github.com/metrico/clickhouse-mate) 是一个 Angular Web 客户端和用户界面,用于在 ClickHouse 中搜索和探索数据。 - -特性: - -- ClickHouse SQL 查询自动补全 -- 快速的数据库与数据表树状导航 -- 高级结果过滤与排序 -- 内联 ClickHouse SQL 文档 -- 查询预设和历史记录 -- 100% 基于浏览器,无需服务器/后端 - -该客户端可通过 GitHub Pages 即时使用:https://metrico.github.io/clickhouse-mate/ - -### Uptrace {#uptrace} - -[Uptrace](https://github.com/uptrace/uptrace) 是一个 APM(应用性能监控)工具,基于 OpenTelemetry 和 ClickHouse 提供分布式追踪和指标能力。 - -特性: - -- [OpenTelemetry 追踪](https://uptrace.dev/opentelemetry/distributed-tracing.html)、指标与日志。 -- 通过 AlertManager 实现 Email/Slack/PagerDuty 通知。 -- 类 SQL 查询语言用于聚合 span。 -- 类 PromQL 的语言用于查询指标。 -- 预构建的指标仪表盘。 -- 通过 YAML 配置支持多用户/多项目。 - -### clickhouse-monitoring {#clickhouse-monitoring} - -[clickhouse-monitoring](https://github.com/duyet/clickhouse-monitoring) 是一个基于 Next.js 的简单仪表盘,依赖 `system.*` 表来帮助监控并提供 ClickHouse 集群的概览。 - -特性: - -- 查询监控:当前查询、查询历史、查询资源(内存、读取的 part、file_open 等)、最耗资源的查询、最常使用的表或列等。 -- 集群监控:总内存/CPU 使用、分布式队列、全局设置、MergeTree 设置、各种指标等。 -- 表和 part 信息:大小、行数、压缩情况、part 大小等,细化到列级别的详细信息。 -- 实用工具:Zookeeper 数据探索、查询 EXPLAIN、终止查询等。 -- 指标可视化图表:查询和资源使用、合并/变更数量、合并性能、查询性能等。 - -### CKibana {#ckibana} - - - -[CKibana](https://github.com/TongchengOpenSource/ckibana) 是一款轻量级服务,可让你使用原生 Kibana UI 轻松搜索、探索和可视化 ClickHouse 数据。 - -功能特性: - -- 将来自原生 Kibana UI 的图表请求转换为 ClickHouse 查询语法。 -- 支持采样与缓存等高级特性,以提升查询性能。 -- 从 Elasticsearch 迁移到 ClickHouse 后,可将用户的学习成本降到最低。 - -### Telescope {#telescope} - -[Telescope](https://iamtelescope.net/) 是一个用于探索存储在 ClickHouse 中日志的现代 Web 界面。它提供用户友好的 UI,可对日志数据进行查询、可视化和管理,并支持细粒度的访问控制。 - -功能特性: - -- 简洁、响应式的 UI,提供强大的过滤器和可自定义字段选择。 -- 使用 FlyQL 语法,实现直观且表达力强的日志过滤。 -- 基于时间的图表,支持 group by,包括嵌套 JSON、Map 和 Array 字段。 -- 可选的原生 SQL `WHERE` 查询支持,用于高级过滤(带权限检查)。 -- 已保存视图:持久化并分享查询和布局的自定义 UI 配置。 -- 基于角色的访问控制(RBAC)以及 GitHub 身份验证集成。 -- 在 ClickHouse 端无需额外的 Agent 或组件。 - -[Telescope 源码](https://github.com/iamtelescope/telescope) · [在线演示](https://demo.iamtelescope.net) - - - -## 商业版 {#commercial} - -### DataGrip {#datagrip} - -[DataGrip](https://www.jetbrains.com/datagrip/) 是 JetBrains 出品的数据库 IDE,对 ClickHouse 提供专门支持。它也集成在其他基于 IntelliJ 的工具中:PyCharm、IntelliJ IDEA、GoLand、PhpStorm 等。 - -功能特性: - -- 极快的代码补全。 -- ClickHouse 语法高亮。 -- 支持 ClickHouse 特有功能,例如嵌套列、表引擎。 -- 数据编辑器。 -- 重构功能。 -- 搜索与导航。 - -### Yandex DataLens {#yandex-datalens} - -[Yandex DataLens](https://cloud.yandex.ru/services/datalens) 是一个数据可视化与分析服务。 - -功能特性: - -- 提供广泛的可视化形式,从简单的条形图到复杂的仪表板。 -- 仪表板可以公开访问。 -- 支持多种数据源,包括 ClickHouse。 -- 基于 ClickHouse 的物化数据存储。 - -对于低负载项目,即使是商业用途,DataLens 也[可免费使用](https://cloud.yandex.com/docs/datalens/pricing)。 - -- [DataLens 文档](https://cloud.yandex.com/docs/datalens/)。 -- [教程](https://cloud.yandex.com/docs/solutions/datalens/data-from-ch-visualization):可视化来自 ClickHouse 数据库的数据。 - -### Holistics Software {#holistics-software} - -[Holistics](https://www.holistics.io/) 是一套全栈数据平台与商业智能工具。 - -功能特性: - -- 支持通过邮件、Slack 和 Google Sheet 自动定时发送报表。 -- 带有可视化、版本控制、自动补全、可复用查询组件和动态过滤器的 SQL 编辑器。 -- 通过 iframe 嵌入的报表和仪表板分析。 -- 数据准备和 ETL 能力。 -- 支持 SQL 数据建模,用于数据的关系映射。 - -### Looker {#looker} - -[Looker](https://looker.com) 是一个数据平台和商业智能工具,支持 50 多种数据库方言,包括 ClickHouse。Looker 既可作为 SaaS 平台使用,也可自托管。用户可以通过浏览器使用 Looker 来探索数据、构建可视化和仪表板、调度报表,并与同事分享分析洞见。Looker 提供了一套丰富的工具,将这些功能嵌入到其他应用程序中,并提供 API 以便与其他应用集成数据。 - -功能特性: - -- 使用 LookML 进行轻量敏捷的开发,这是一门支持精心设计[数据建模](https://looker.com/platform/data-modeling)的语言,可帮助报表作者和终端用户。 -- 通过 Looker 的 [Data Actions](https://looker.com/platform/actions) 实现强大的工作流集成。 - -[如何在 Looker 中配置 ClickHouse。](https://docs.looker.com/setup-and-management/database-config/clickhouse) - -### SeekTable {#seektable} - -[SeekTable](https://www.seektable.com) 是一款用于数据探索和运营报表的自助式 BI 工具,既提供云服务,也提供自托管版本。SeekTable 生成的报表可以嵌入到任意 Web 应用中。 - -功能特性: - -- 面向业务用户的友好报表构建器。 -- 强大的报表参数,用于 SQL 过滤和报表级查询定制。 -- 既可以通过原生 TCP/IP 端点,也可以通过 HTTP(S) 接口连接 ClickHouse(2 种不同驱动)。 -- 在维度/度量定义中可以充分发挥 ClickHouse SQL 方言的全部能力。 -- 提供用于自动生成报表的 [Web API](https://www.seektable.com/help/web-api-integration)。 -- 支持带有账号数据[备份/恢复](https://www.seektable.com/help/self-hosted-backup-restore)的报表开发流程;数据模型(多维立方体)/报表配置采用人类可读的 XML 格式,可以存放在版本控制系统中。 - -SeekTable 对于个人/个体使用是[免费的](https://www.seektable.com/help/cloud-pricing)。 - -[如何在 SeekTable 中配置 ClickHouse 连接。](https://www.seektable.com/help/clickhouse-pivot-table) - -### Chadmin {#chadmin} - -[Chadmin](https://github.com/bun4uk/chadmin) 是一个简单的 UI,用于可视化你在 ClickHouse 集群上当前正在运行的查询及其相关信息,并在需要时终止这些查询。 - -### TABLUM.IO {#tablum_io} - -[TABLUM.IO](https://tablum.io/) 是一款用于 ETL 和可视化的在线查询与分析工具。它支持连接 ClickHouse,可通过通用的 SQL 控制台查询数据,也可以从静态文件和第三方服务加载数据。TABLUM.IO 可以将查询结果可视化为图表和表格。 - - - -功能: -- ETL:从常见数据库、本地和远程文件以及 API 调用中加载数据。 -- 功能强大的 SQL 控制台,支持语法高亮和可视化查询构建器。 -- 以图表和表格形式进行数据可视化。 -- 数据物化和子查询。 -- 向 Slack、Telegram 或电子邮件发送数据报表。 -- 通过专有 API 构建数据流水线。 -- 以 JSON、CSV、SQL、HTML 格式导出数据。 -- 基于 Web 的界面。 - -TABLUM.IO 既可以以 Docker 镜像形式自托管部署,也可以在云端运行。 -许可证:[商业版](https://tablum.io/pricing) 产品,提供 3 个月免费试用期。 - -在[云端免费试用](https://tablum.io/try)。 -在 [TABLUM.IO](https://tablum.io/) 了解更多产品信息。 - -### CKMAN {#ckman} - -[CKMAN](https://www.github.com/housepower/ckman) 是一个用于管理和监控 ClickHouse 集群的工具! - -功能: - -- 通过浏览器界面实现快速便捷的集群自动化部署 -- 集群可灵活扩缩容 -- 对集群数据进行负载均衡 -- 在线升级集群 -- 在页面上修改集群配置 -- 提供集群节点监控和 ZooKeeper 监控 -- 监控表和分区状态,并监控慢 SQL 语句 -- 提供易于使用的 SQL 执行页面 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md deleted file mode 100644 index 4ce5dd27ea5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/index.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: '可用于 ClickHouse 的第三方工具、库和集成概览' -sidebar_position: 24 -slug: /interfaces/third-party/ -toc_folder_title: '第三方' -title: '第三方接口' -doc_type: 'landing-page' ---- - -# 第三方接口 {#third-party-interfaces} - -这里汇总了一些第三方工具的链接,这些工具为 ClickHouse 提供各种形式的接口,包括可视化界面、命令行界面或 API: - -* [客户端库](../../interfaces/third-party/client-libraries.md) -* [集成](../../interfaces/third-party/integrations.md) -* [GUI](../../interfaces/third-party/gui.md) -* [代理](../../interfaces/third-party/proxy.md) - -:::note -支持诸如 [ODBC](../../interfaces/odbc.md) 或 [JDBC](../../interfaces/jdbc.md) 等通用 API 的通用工具通常也可以与 ClickHouse 一起配合使用,但由于这类工具数量过多,因此未在此列出。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md deleted file mode 100644 index 2952116bc76..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/integrations.md +++ /dev/null @@ -1,192 +0,0 @@ ---- -description: '关于将 ClickHouse 与各种第三方系统和工具集成的文档' -sidebar_label: '集成' -sidebar_position: 27 -slug: /interfaces/third-party/integrations -title: '由第三方开发者提供的集成库' -doc_type: 'reference' ---- - -# 来自第三方开发者的集成库 {#integration-libraries-from-third-party-developers} - -:::warning 免责声明 -ClickHouse, Inc. **不**维护以下列出的这些工具和库,也未对其质量进行充分测试。 -如需官方集成,请参阅[集成页面](/integrations)。 -::: - -## 基础设施产品 {#infrastructure-products} - -
-关系型数据库管理系统 - -- [MySQL](https://www.mysql.com) - - [mysql2ch](https://github.com/long2ice/mysql2ch) - - [ProxySQL](https://github.com/sysown/proxysql/wiki/ClickHouse-Support) - - [clickhouse-mysql-data-reader](https://github.com/Altinity/clickhouse-mysql-data-reader) - - [horgh-replicator](https://github.com/larsnovikov/horgh-replicator) -- [PostgreSQL](https://www.postgresql.org) - - [clickhousedb_fdw](https://github.com/Percona-Lab/clickhousedb_fdw) - - [infi.clickhouse_fdw](https://github.com/Infinidat/infi.clickhouse_fdw)(使用 [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)) - - [pg2ch](https://github.com/mkabilov/pg2ch) - - [clickhouse_fdw](https://github.com/adjust/clickhouse_fdw) -- [MSSQL](https://en.wikipedia.org/wiki/Microsoft_SQL_Server) - - [ClickHouseMigrator](https://github.com/zlzforever/ClickHouseMigrator) -
- -
-消息队列 - -- [Kafka](https://kafka.apache.org) - - [clickhouse_sinker](https://github.com/housepower/clickhouse_sinker)(使用 [Go 客户端](https://github.com/ClickHouse/clickhouse-go/)) - - [stream-loader-clickhouse](https://github.com/adform/stream-loader) -
- -
-批处理 - -- [Spark](https://spark.apache.org) - - [spark-clickhouse-connector](https://github.com/housepower/spark-clickhouse-connector) -
- -
-流处理 - -- [Flink](https://flink.apache.org) - - [flink-clickhouse-sink](https://github.com/ivi-ru/flink-clickhouse-sink) -
- -
-对象存储 - -- [S3](https://en.wikipedia.org/wiki/Amazon_S3) - - [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) -
- -
-容器编排 - -- [Kubernetes](https://kubernetes.io) - - [clickhouse-operator](https://github.com/Altinity/clickhouse-operator) -
- -
-配置管理 -- [puppet](https://puppet.com) - - [innogames/clickhouse](https://forge.puppet.com/innogames/clickhouse) - - [mfedotov/clickhouse](https://forge.puppet.com/mfedotov/clickhouse) -
- -
-监控 - -- [Graphite](https://graphiteapp.org) - - [graphouse](https://github.com/ClickHouse/graphouse) - - [carbon-clickhouse](https://github.com/lomik/carbon-clickhouse) - - [graphite-clickhouse](https://github.com/lomik/graphite-clickhouse) - - [graphite-ch-optimizer](https://github.com/innogames/graphite-ch-optimizer) - 如果可以应用来自 [rollup configuration](../../engines/table-engines/mergetree-family/graphitemergetree.md#rollup-configuration) 的规则,则用于优化 [\*GraphiteMergeTree](/engines/table-engines/mergetree-family/graphitemergetree) 中的过期分区 -- [Grafana](https://grafana.com/) - - [clickhouse-grafana](https://github.com/Altinity/clickhouse-grafana) -- [Prometheus](https://prometheus.io/) - - [clickhouse_exporter](https://github.com/f1yegor/clickhouse_exporter) - - [PromHouse](https://github.com/Percona-Lab/PromHouse) - - [clickhouse_exporter](https://github.com/hot-wifi/clickhouse_exporter)(使用 [Go 客户端](https://github.com/kshvakov/clickhouse/)) -- [Nagios](https://www.nagios.org/) - - [check_clickhouse](https://github.com/exogroup/check_clickhouse/) - - [check_clickhouse.py](https://github.com/innogames/igmonplugins/blob/master/src/check_clickhouse.py) -- [Zabbix](https://www.zabbix.com) - - [clickhouse-zabbix-template](https://github.com/Altinity/clickhouse-zabbix-template) -- [Sematext](https://sematext.com/) - - [clickhouse integration](https://github.com/sematext/sematext-agent-integrations/tree/master/clickhouse) -
- -
-日志 - -- [rsyslog](https://www.rsyslog.com/) - - [omclickhouse](https://www.rsyslog.com/doc/master/configuration/modules/omclickhouse.html) -- [fluentd](https://www.fluentd.org) - - [loghouse](https://github.com/flant/loghouse)(适用于 [Kubernetes](https://kubernetes.io)) -- [logagent](https://www.sematext.com/logagent) - - [logagent output-plugin-clickhouse](https://sematext.com/docs/logagent/output-plugin-clickhouse/) -
- -
-地理位置 - -- [MaxMind](https://dev.maxmind.com/geoip/) - - [clickhouse-maxmind-geoip](https://github.com/AlexeyKupershtokh/clickhouse-maxmind-geoip) -
- -
-AutoML - -- [MindsDB](https://mindsdb.com/) - - [MindsDB](https://github.com/mindsdb/mindsdb) - 与 ClickHouse 集成,使 ClickHouse 中的数据可供各类 AI/ML 模型使用。 -
- -## 编程语言生态系统 {#programming-language-ecosystems} - -
-Python - -- [SQLAlchemy](https://www.sqlalchemy.org) - - [sqlalchemy-clickhouse](https://github.com/cloudflare/sqlalchemy-clickhouse)(使用 [infi.clickhouse_orm](https://github.com/Infinidat/infi.clickhouse_orm)) -- [PyArrow/Pandas](https://pandas.pydata.org) - - [Ibis](https://github.com/ibis-project/ibis) -
- -
-PHP - -- [Doctrine](https://www.doctrine-project.org/) - - [dbal-clickhouse](https://packagist.org/packages/friendsofdoctrine/dbal-clickhouse) -
- -
-R - -- [dplyr](https://db.rstudio.com/dplyr/) - - [RClickHouse](https://github.com/IMSMWU/RClickHouse)(使用 [clickhouse-cpp](https://github.com/artpaul/clickhouse-cpp)) -
- -
-Java - -- [Hadoop](http://hadoop.apache.org) - - [clickhouse-hdfs-loader](https://github.com/jaykelin/clickhouse-hdfs-loader)(使用 [JDBC](../../sql-reference/table-functions/jdbc.md)) -
- -
-Scala - -- [Akka](https://akka.io) - - [clickhouse-scala-client](https://github.com/crobox/clickhouse-scala-client) -
- -
-C# - -- [ADO.NET](https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/ado-net-overview) - - [ClickHouse.Ado](https://github.com/killwort/ClickHouse-Net) - - [ClickHouse.Client](https://github.com/DarkWanderer/ClickHouse.Client) - - [ClickHouse.Net](https://github.com/ilyabreev/ClickHouse.Net) - - [ClickHouse.Net.Migrations](https://github.com/ilyabreev/ClickHouse.Net.Migrations) - - [Linq To DB](https://github.com/linq2db/linq2db) -
- -
-Elixir - -- [Ecto](https://github.com/elixir-ecto/ecto) - - [clickhouse_ecto](https://github.com/appodeal/clickhouse_ecto) -
- -
-Ruby - -- [Ruby on Rails](https://rubyonrails.org/) - - [activecube](https://github.com/bitquery/activecube) - - [ActiveRecord](https://github.com/PNixx/clickhouse-activerecord) -- [GraphQL](https://github.com/graphql) - - [activecube-graphql](https://github.com/bitquery/activecube-graphql) -
\ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md b/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md deleted file mode 100644 index c1f85a23bb5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/interfaces/third-party/proxy.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: '描述可用于 ClickHouse 的第三方代理解决方案' -sidebar_label: '代理' -sidebar_position: 29 -slug: /interfaces/third-party/proxy -title: '第三方开发的代理服务器' -doc_type: 'reference' ---- - - - -# 第三方开发的代理服务器 {#proxy-servers-from-third-party-developers} - - - -## chproxy {#chproxy} - -[chproxy](https://github.com/Vertamedia/chproxy) 是一个用于 ClickHouse 数据库的 HTTP 代理和负载均衡器。 - -功能特性: - -- 按用户划分的路由和响应缓存。 -- 灵活的限制机制。 -- 自动续期 SSL 证书。 - -由 Go 语言实现。 - - - -## KittenHouse {#kittenhouse} - -[KittenHouse](https://github.com/VKCOM/kittenhouse) 旨在在无法或不方便在应用程序端对 `INSERT` 数据进行缓冲时,作为 ClickHouse 与应用服务器之间的本地代理。 - -特性: - -- 内存与磁盘级数据缓冲。 -- 基于表的路由。 -- 负载均衡与健康检查。 - -使用 Go 语言实现。 - - - -## ClickHouse-Bulk {#clickhouse-bulk} - -[ClickHouse-Bulk](https://github.com/nikepan/clickhouse-bulk) 是一个简单的 ClickHouse 数据写入收集器。 - -功能: - -- 将请求分组,并按阈值或时间间隔批量发送。 -- 支持多个远程服务器。 -- 支持基本身份验证。 - -用 Go 编写。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md deleted file mode 100644 index a2c7ec5c88d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/_troubleshooting.md +++ /dev/null @@ -1,228 +0,0 @@ - -[//]: # (This file is included in FAQ > Troubleshooting) - -- [安装](#troubleshooting-installation-errors) -- [连接服务器](#troubleshooting-accepts-no-connections) -- [查询处理](#troubleshooting-does-not-process-queries) -- [查询效率](#troubleshooting-too-slow) - - - -## 安装 {#troubleshooting-installation-errors} - -### 无法通过 apt-get 从 ClickHouse 仓库获取 deb 包 {#you-cannot-get-deb-packages-from-clickhouse-repository-with-apt-get} - -* 检查防火墙设置。 -* 如果由于任何原因无法访问该仓库,请按照[安装指南](../getting-started/install.md)中的说明下载软件包,并使用 `sudo dpkg -i ` 命令手动安装。您还需要安装 `tzdata` 软件包。 - -### 无法通过 apt-get 从 ClickHouse 仓库更新 deb 包 {#you-cannot-update-deb-packages-from-clickhouse-repository-with-apt-get} - -* 当 GPG 密钥更改时,可能会出现此问题。 - -请按照 [setup](../getting-started/install.md#setup-the-debian-repository) 页面中的说明更新仓库配置。 - -### 运行 `apt-get update` 时收到不同的警告 {#you-get-different-warnings-with-apt-get-update} - -* 完整的警告信息类似于以下几种情况之一: - -```bash -N: 跳过获取已配置的文件 'main/binary-i386/Packages',因为软件源 'https://packages.clickhouse.com/deb stable InRelease' 不支持 'i386' 架构 -``` - -```bash -E: 无法获取 https://packages.clickhouse.com/deb/dists/stable/main/binary-amd64/Packages.gz 文件大小不符合预期 (30451 != 28154)。镜像同步正在进行中? -``` - -```text -E: 仓库 'https://packages.clickhouse.com/deb stable InRelease' 的 'Origin' 值已从 'Artifactory' 变更为 'ClickHouse' -E: 仓库 'https://packages.clickhouse.com/deb stable InRelease' 的 'Label' 值已从 'Artifactory' 变更为 'ClickHouse' -N: 仓库 'https://packages.clickhouse.com/deb stable InRelease' 的 'Suite' 值已从 'stable' 变更为 '' -N: 在应用此仓库的更新之前,必须明确接受此变更。详情请参阅 apt-secure(8) 手册页。 -``` - -```bash -错误:11 https://packages.clickhouse.com/deb stable InRelease - 400 错误请求 [IP: 172.66.40.249 443] -``` - -要解决上述问题,请使用以下脚本: - -```bash -sudo rm /var/lib/apt/lists/packages.clickhouse.com_* /var/lib/dpkg/arch /var/lib/apt/lists/partial/packages.clickhouse.com_* -sudo apt-get clean -sudo apt-get autoclean -``` - -### 由于签名不正确,你无法通过 yum 获取软件包 {#you-cant-get-packages-with-yum-because-of-wrong-signature} - -可能的问题:缓存不正确;在 2022 年 9 月更新 GPG 密钥后,缓存可能已损坏。 - -解决办法是清理 yum 的缓存和 lib 目录: - -```bash -sudo find /var/lib/yum/repos/ /var/cache/yum/ -name 'clickhouse-*' -type d -exec rm -rf {} + -sudo rm -f /etc/yum.repos.d/clickhouse.repo -``` - -之后请按照[安装指南](../getting-started/install.md#from-rpm-packages)进行操作。 - -### 无法运行 Docker 容器 {#you-cant-run-docker-container} - -当你运行一个简单的命令 `docker run clickhouse/clickhouse-server` 时,它会崩溃并输出类似如下的堆栈跟踪: - -```bash -$ docker run -it clickhouse/clickhouse-server -........ -Poco::Exception. Code: 1000, e.code() = 0, System exception: cannot start thread, Stack trace (复制此消息时,务必包含以下内容): -``` - - -0. Poco::ThreadImpl::startImpl(Poco::SharedPtr>) @ 0x00000000157c7b34 -1. Poco::Thread::start(Poco::Runnable&) @ 0x00000000157c8a0e -2. BaseDaemon::initializeTerminationAndSignalProcessing() @ 0x000000000d267a14 -3. BaseDaemon::initialize(Poco::Util::Application&) @ 0x000000000d2652cb -4. DB::Server::initialize(Poco::Util::Application&) @ 0x000000000d128b38 -5. Poco::Util::Application::run() @ 0x000000001581cfda -6. DB::Server::run() @ 0x000000000d1288f0 -7. Poco::Util::ServerApplication::run(int, char\*\*) @ 0x0000000015825e27 -8. mainEntryClickHouseServer(int, char\*\*) @ 0x000000000d125b38 -9. main @ 0x0000000007ea4eee -10. ? @ 0x00007f67ff946d90 -11. ? @ 0x00007f67ff946e40 -12. \_start @ 0x00000000062e802e - (version 24.10.1.2812 (official build)) - -``` - -原因是 Docker 守护进程版本低于 `20.10.10`。解决方法是升级 Docker 守护进程,或运行 `docker run [--privileged | --security-opt seccomp=unconfined]`。后者具有安全风险。 - -``` - - -## 连接到服务器 {#troubleshooting-accepts-no-connections} - -可能出现的问题: - -* 服务器未运行。 -* 配置参数异常或错误。 - -### 服务器未运行 {#server-is-not-running} - -**检查服务器是否正在运行** - -命令: - -```bash -$ sudo service clickhouse-server status -``` - -如果服务器未运行,请使用以下命令启动: - -```bash -$ sudo service clickhouse-server start -``` - -**检查日志** - -`clickhouse-server` 的主日志默认位于 `/var/log/clickhouse-server/clickhouse-server.log`。 - -如果服务器成功启动,你应该看到如下日志行: - -* ` Application: starting up.` — 服务器已启动。 -* ` Application: Ready for connections.` — 服务器正在运行并已准备好接受连接。 - -如果 `clickhouse-server` 因配置错误而启动失败,你应该会看到带有错误描述的 `` 日志行。例如: - -```text -2019.01.11 15:23:25.549505 [ 45 ] {} ExternalDictionaries: 重新加载外部字典 'event2id' 失败:Poco::Exception. Code: 1000, e.code() = 111, e.displayText() = Connection refused, e.what() = Connection refused -``` - -如果在文件末尾没有看到错误,请从以下字符串开始检查整个文件: - -```text - 应用程序:正在启动。 -``` - -如果你尝试在同一台服务器上启动第二个 `clickhouse-server` 实例,将会看到如下日志: - -```text -2019.01.11 15:25:11.151730 [ 1 ] {} : 正在启动 ClickHouse 19.1.0,修订版 54413 -2019.01.11 15:25:11.154578 [ 1 ] {} Application: 正在启动 -2019.01.11 15:25:11.156361 [ 1 ] {} StatusFile: 状态文件 ./status 已存在 - 非正常重启。内容: -PID: 8510 -启动于: 2019-01-11 15:24:23 -修订版: 54413 - -2019.01.11 15:25:11.156673 [ 1 ] {} Application: DB::Exception: 无法锁定文件 ./status。同一目录中已有另一个服务器实例正在运行。 -2019.01.11 15:25:11.156682 [ 1 ] {} Application: 正在关闭 -2019.01.11 15:25:11.156686 [ 1 ] {} Application: 正在取消初始化子系统: 日志子系统 -2019.01.11 15:25:11.156716 [ 2 ] {} BaseDaemon: 停止 SignalListener 线程 -``` - -**查看 system.d 日志** - -如果在 `clickhouse-server` 日志中找不到任何有用的信息,或者根本没有日志,可以使用以下命令查看 `system.d` 日志: - -```bash -$ sudo journalctl -u clickhouse-server -``` - -**以交互式模式启动 clickhouse-server** - -```bash -$ sudo -u clickhouse /usr/bin/clickhouse-server --config-file /etc/clickhouse-server/config.xml -``` - -此命令会使用自动启动脚本的标准参数,以交互式应用程序的方式启动服务器。在此模式下,`clickhouse-server` 会在控制台输出所有事件消息。 - -### 配置参数 {#configuration-parameters} - -请检查: - -* Docker 设置 - - 如果你在 IPv6 网络中通过 Docker 运行 ClickHouse,确保设置了 `network=host`。 - -* 端点(Endpoint)设置 - - 检查 [listen_host](../operations/server-configuration-parameters/settings.md#listen_host) 和 [tcp_port](../operations/server-configuration-parameters/settings.md#tcp_port) 设置。 - - 默认情况下,ClickHouse 服务器只接受来自 localhost 的连接。 - -* HTTP 协议设置 - - 检查 HTTP API 的协议相关设置。 - -* 安全连接设置 - - 请检查: - - * [tcp_port_secure](../operations/server-configuration-parameters/settings.md#tcp_port_secure) 设置。 - * [SSL 证书](../operations/server-configuration-parameters/settings.md#openssl) 相关设置。 - - 连接时请使用正确的参数。例如,在使用 `clickhouse_client` 时应使用 `port_secure` 参数。 - -* 用户设置 - - 你可能使用了错误的用户名或密码。 - - -## 查询处理 {#troubleshooting-does-not-process-queries} - -如果 ClickHouse 无法处理查询,它会将错误描述发送给客户端。在 `clickhouse-client` 中,错误描述会显示在控制台中。如果使用 HTTP 接口,ClickHouse 会在响应体中返回错误描述。例如: - -```bash -$ curl 'http://localhost:8123/' --data-binary "SELECT a" -Code: 47, e.displayText() = DB::Exception: 未知标识符:a。注意您的查询中没有表(缺少 FROM 子句),上下文:required_names: 'a' source_tables: table_aliases: private_aliases: column_aliases: public_columns: 'a' masked_columns: array_join_columns: source_columns: , e.what() = DB::Exception -``` - -如果使用 `stack-trace` 参数启动 `clickhouse-client`,ClickHouse 会连同错误描述一起返回服务器端的堆栈跟踪信息。 - -你可能会看到一条有关连接断开的消息。在这种情况下,可以重试该查询。如果每次执行该查询时连接都会断开,请检查服务器端日志以查找错误。 - - -## 查询处理效率 {#troubleshooting-too-slow} - -如果发现 ClickHouse 工作得过慢,应对相关查询在服务器资源和网络上的负载进行分析。 - -你可以使用 `clickhouse-benchmark` 工具对查询进行分析。它会显示每秒处理的查询数量、每秒处理的行数,以及查询处理时间的百分位数。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md deleted file mode 100644 index 9a2b33228b6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling-old.md +++ /dev/null @@ -1,224 +0,0 @@ ---- -description: '介绍 ClickHouse 中内存分配分析的页面' -sidebar_label: '25.9 之前版本的内存分配分析' -slug: /operations/allocation-profiling-old -title: '25.9 之前版本的内存分配分析' -doc_type: 'reference' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# 25.9 之前版本的分配分析 {#allocation-profiling-for-versions-before-259} - -ClickHouse 使用 [jemalloc](https://github.com/jemalloc/jemalloc) 作为其全局分配器。jemalloc 自带了一些用于分配采样和分析的工具。 -为了让分配分析更加方便,除了提供 `SYSTEM` 命令外,还在 Keeper 中提供了四字命令(4LW)。 - - - -## 分配采样与堆分析数据刷新 {#sampling-allocations-and-flushing-heap-profiles} - -如果你想在 `jemalloc` 中对内存分配进行采样和分析,需要通过设置环境变量 `MALLOC_CONF`,在启用分析功能的情况下启动 ClickHouse/Keeper: - -```sh -MALLOC_CONF=background_thread:true,prof:true -``` - -`jemalloc` 会对内存分配进行采样,并在内部存储相关信息。 - -你可以通过运行以下命令让 `jemalloc` 刷写当前的 profile: - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -默认情况下,堆 profile 文件会生成在 `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap`,其中 `_pid_` 是 ClickHouse 的 PID,`_seqnum_` 是当前堆 profile 的全局序列号。\ -对于 Keeper,默认文件为 `/tmp/jemalloc_keeper._pid_._seqnum_.heap`,规则相同。 - -可以通过在 `MALLOC_CONF` 环境变量中追加 `prof_prefix` 选项来指定不同的位置。\ -例如,如果你希望在 `/data` 目录中生成 profile,并将文件名前缀设置为 `my_current_profile`,可以在运行 ClickHouse/Keeper 时设置如下环境变量: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_prefix:/data/my_current_profile -``` - -生成的文件名会在前缀后追加 PID 和序列号。 - - -## 分析堆内存剖析文件 {#analyzing-heap-profiles} - -生成堆内存剖析文件后,需要对其进行分析。\ -为此,可以使用 `jemalloc` 的工具 [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in)。它可以通过多种方式安装: - -* 使用系统的包管理器 -* 克隆 [jemalloc 代码仓库](https://github.com/jemalloc/jemalloc),并在根目录下运行 `autogen.sh`。这会在 `bin` 目录中生成 `jeprof` 脚本 - -:::note -`jeprof` 使用 `addr2line` 来生成调用栈,这个过程可能会非常慢。\ -如果出现这种情况,建议安装该工具的[替代实现](https://github.com/gimli-rs/addr2line)。 - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -::: - -使用 `jeprof` 可以从堆分析文件生成多种不同的输出格式。 -建议运行 `jeprof --help` 来了解该工具的用法以及它提供的各种选项。 - -一般来说,`jeprof` 命令通常这样使用: - -```sh -jeprof 二进制文件路径 堆配置文件路径 --output_format [ > 输出文件] -``` - -如果你想比较在两个性能分析结果之间新增了哪些内存分配,可以设置 `base` 参数: - -```sh -jeprof 二进制文件路径 --base 第一个堆配置文件路径 第二个堆配置文件路径 --output_format [ > 输出文件] -``` - -### 示例 {#examples} - -* 如果你想生成一个文本文件,每行写一个存储过程: - -```sh -jeprof 二进制文件路径 堆配置文件路径 --text > result.txt -``` - -* 如果需要生成包含调用图的 PDF 文件: - -```sh -jeprof path/to/binary path/to/heap/profile --pdf > result.pdf -``` - -### 生成火焰图 {#generating-flame-graph} - -`jeprof` 可以用来生成用于构建火焰图的折叠堆栈数据。 - -你需要使用 `--collapsed` 参数: - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -之后,你可以使用许多不同的工具来可视化折叠后的调用栈。 - -最常用的是 [FlameGraph](https://github.com/brendangregg/FlameGraph),其中包含一个名为 `flamegraph.pl` 的脚本: - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg -``` - -另一个值得一提的工具是 [speedscope](https://www.speedscope.app/),它可以让你以更交互的方式分析采集到的堆栈数据。 - - -## 在运行时控制分配分析器 {#controlling-allocation-profiler-during-runtime} - -如果在启用分配分析器的情况下启动 ClickHouse/Keeper,则可以在运行时使用额外命令来启用或禁用内存分配分析。 -使用这些命令,可以更方便地只在特定时间区间进行分析。 - -要禁用分析器: - - - - ```sql - SYSTEM JEMALLOC DISABLE PROFILE - ``` - - - - ```sh - echo jmdp | nc localhost 9181 - ``` - - - -要启用分析器: - - - - ```sql - SYSTEM JEMALLOC ENABLE PROFILE - ``` - - - - ```sh - echo jmep | nc localhost 9181 - ``` - - - -还可以通过设置 `prof_active` 选项来控制分析器的初始状态,该选项默认启用。\ -例如,如果不希望在启动期间采样分配,而只在启动完成后开始采样,可以在之后再启用分析器。可以使用以下环境变量来启动 ClickHouse/Keeper: - -```sh -MALLOC_CONF=background_thread:true,prof:true,prof_active:false -``` - -稍后可以启用分析器。 - - -## 分析器的其他选项 {#additional-options-for-profiler} - -`jemalloc` 提供了许多与分析器相关的选项,可以通过修改 `MALLOC_CONF` 环境变量进行控制。 -例如,分配采样之间的间隔可以通过 `lg_prof_sample` 控制。 -如果希望每分配 N 个字节就转储一次堆分析数据,可以通过 `lg_prof_interval` 启用该功能。 - -建议查阅 `jemalloc` 的[参考页面](https://jemalloc.net/jemalloc.3.html)以获取完整的选项列表。 - - - -## 其他资源 {#other-resources} - -ClickHouse/Keeper 通过多种不同方式公开与 `jemalloc` 相关的指标。 - -:::warning 警告 -需要注意的是,这些指标彼此之间并不同步,数值可能会产生漂移。 -::: - -### 系统表 `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[参考](/operations/system-tables/asynchronous_metrics) - -### 系统表 `jemalloc_bins` {#system-table-jemalloc_bins} - -包含通过 jemalloc 内存分配器在不同大小类别(bins)中进行的内存分配情况,这些信息是从所有 arena 聚合而来的。 - -[参考](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -来自 `asynchronous_metrics` 的所有与 `jemalloc` 相关的指标也会在 ClickHouse 和 Keeper 中通过 Prometheus 端点对外暴露。 - -[参考](/operations/server-configuration-parameters/settings#prometheus) - -### Keeper 中的 `jmst` 4LW 命令 {#jmst-4lw-command-in-keeper} - -Keeper 支持 `jmst` 4LW 命令,该命令返回[基础的分配器统计数据](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics): - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md deleted file mode 100644 index cf34f600a9c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/allocation-profiling.md +++ /dev/null @@ -1,337 +0,0 @@ ---- -description: '介绍 ClickHouse 中内存分配分析的页面' -sidebar_label: '内存分配分析' -slug: /operations/allocation-profiling -title: '内存分配分析' -doc_type: 'guide' ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - - -# 内存分配分析 {#allocation-profiling} - -ClickHouse 使用 [jemalloc](https://github.com/jemalloc/jemalloc) 作为其全局分配器。Jemalloc 自带了一些用于内存分配采样和分析的工具。 -为了让内存分配分析更方便,ClickHouse 和 Keeper 允许通过配置、查询设置、`SYSTEM` 命令以及 Keeper 中的四字命令(4LW)来控制采样。 -此外,采样数据可以以 `JemallocSample` 类型收集到 `system.trace_log` 表中。 - -:::note - -本指南适用于 25.9 及以上版本。 -对于更早的版本,请参阅[25.9 之前版本的内存分配分析](/operations/allocation-profiling-old.md)。 - -::: - - - -## 采样内存分配 {#sampling-allocations} - -如需在 `jemalloc` 中对内存分配进行采样和分析,你需要在启动 ClickHouse/Keeper 时启用配置项 `jemalloc_enable_global_profiler`。 - -```xml - - 1 - -``` - -`jemalloc` 将对内存分配进行采样,并在内部存储相关信息。 - -你也可以通过 `jemalloc_enable_profiler` 设置来为每个查询启用内存分配采样。 - -:::warning 警告 -由于 ClickHouse 是一个内存分配密集型应用程序,jemalloc 采样可能会带来性能开销。 -::: - - -## 在 `system.trace_log` 中存储 jemalloc 采样数据 {#storing-jemalloc-samples-in-system-trace-log} - -你可以将所有 jemalloc 采样数据以 `JemallocSample` 类型存储到 `system.trace_log` 中。 -要在全局范围内启用此功能,可以使用配置项 `jemalloc_collect_global_profile_samples_in_trace_log`。 - -```xml - - 1 - -``` - -:::warning 警告 -由于 ClickHouse 是一个内存分配非常密集的应用程序,在 `system.trace_log` 中收集所有样本可能会带来较高负载。 -::: - -你也可以通过使用 `jemalloc_collect_profile_samples_in_trace_log` 设置,为单个查询启用该功能。 - -### 使用 `system.trace_log` 分析查询内存使用情况的示例 {#example-analyzing-memory-usage-trace-log} - -首先,我们需要在启用 jemalloc profiler 的情况下运行查询,并将该查询的样本收集到 `system.trace_log` 中: - -```sql -SELECT * -FROM numbers(1000000) -ORDER BY number DESC -SETTINGS max_bytes_ratio_before_external_sort = 0 -FORMAT `Null` -SETTINGS jemalloc_enable_profiler = 1, jemalloc_collect_profile_samples_in_trace_log = 1 - -Query id: 8678d8fe-62c5-48b8-b0cd-26851c62dd75 - -Ok. - -0 rows in set. Elapsed: 0.009 sec. Processed 1.00 million rows, 8.00 MB (108.58 million rows/s., 868.61 MB/s.) -Peak memory usage: 12.65 MiB. -``` - -:::note -如果 ClickHouse 启动时已启用 `jemalloc_enable_global_profiler`,则无需再启用 `jemalloc_enable_profiler`。\ -对于 `jemalloc_collect_global_profile_samples_in_trace_log` 和 `jemalloc_collect_profile_samples_in_trace_log` 也是同样的。 -::: - -我们将刷新 `system.trace_log`: - -```sql -SYSTEM FLUSH LOGS trace_log -``` - -然后对其进行查询,以获取我们在每个时间点运行的查询的内存使用情况: - -```sql -WITH per_bucket AS -( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time -) -SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable -FROM per_bucket -ORDER BY bucket_time -``` - -我们还可以找出内存使用量最高的时间点: - -```sql -SELECT - argMax(bucket_time, cumulative_size), - max(cumulative_size) -FROM -( - WITH per_bucket AS - ( - SELECT - event_time_microseconds AS bucket_time, - sum(size) AS bucket_sum - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - GROUP BY bucket_time - ) - SELECT - bucket_time, - sum(bucket_sum) OVER ( - ORDER BY bucket_time ASC - ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW - ) AS cumulative_size, - formatReadableSize(cumulative_size) AS cumulative_size_readable - FROM per_bucket - ORDER BY bucket_time -) -``` - -我们可以利用该结果查看在该时间点上哪些位置的活跃内存分配最多: - - -```sql -SELECT - concat( - '\n', - arrayStringConcat( - arrayMap( - (x, y) -> concat(x, ': ', y), - arrayMap(x -> addressToLine(x), allocation_trace), - arrayMap(x -> demangle(addressToSymbol(x)), allocation_trace) - ), - '\n' - ) - ) AS symbolized_trace, - sum(s) AS per_trace_sum -FROM -( - SELECT - ptr, - sum(size) AS s, - argMax(trace, event_time_microseconds) AS allocation_trace - FROM system.trace_log - WHERE trace_type = 'JemallocSample' - AND query_id = '8678d8fe-62c5-48b8-b0cd-26851c62dd75' - AND event_time_microseconds <= '2025-09-04 11:56:21.737139' - GROUP BY ptr - HAVING s > 0 -) -GROUP BY ALL -ORDER BY per_trace_sum ASC -``` - - -## 刷新堆内存剖析文件 {#flushing-heap-profiles} - -默认情况下,堆剖析文件会生成在 `/tmp/jemalloc_clickhouse._pid_._seqnum_.heap` 中,其中 `_pid_` 是 ClickHouse 的 PID,`_seqnum_` 是当前堆剖析文件的全局序号。\ -对于 Keeper,默认文件为 `/tmp/jemalloc_keeper._pid_._seqnum_.heap`,并遵循相同规则。 - -你可以通过运行以下命令,让 `jemalloc` 将当前剖析文件刷新到磁盘: - - - - ```sql - SYSTEM JEMALLOC FLUSH PROFILE - ``` - - 它会返回已刷新的剖析文件的位置。 - - - - ```sh - echo jmfp | nc localhost 9181 - ``` - - - -你可以通过在 `MALLOC_CONF` 环境变量中追加 `prof_prefix` 选项来定义不同的存储位置。\ -例如,如果你希望在 `/data` 目录中生成剖析文件,并将文件名前缀设置为 `my_current_profile`,可以使用如下环境变量来运行 ClickHouse/Keeper: - -```sh -MALLOC_CONF=prof_prefix:/data/my_current_profile -``` - -生成的文件名将由前缀、PID 和序列号组成。 - - -## 分析堆内存剖析数据 {#analyzing-heap-profiles} - -在生成堆内存剖析数据之后,需要对其进行分析。\ -为此,可以使用 `jemalloc` 提供的工具 [jeprof](https://github.com/jemalloc/jemalloc/blob/dev/bin/jeprof.in)。它可以通过多种方式安装: - -* 使用系统的包管理器 -* 克隆 [jemalloc 仓库](https://github.com/jemalloc/jemalloc) 并在根目录运行 `autogen.sh`。这样会在 `bin` 目录中生成 `jeprof` 脚本 - -:::note -`jeprof` 使用 `addr2line` 来生成堆栈跟踪,这个过程可能非常缓慢。\ -如果遇到这种情况,建议安装该工具的[替代实现](https://github.com/gimli-rs/addr2line)。 - -```bash -git clone https://github.com/gimli-rs/addr2line.git --depth=1 --branch=0.23.0 -cd addr2line -cargo build --features bin --release -cp ./target/release/addr2line path/to/current/addr2line -``` - -或者,`llvm-addr2line` 同样适用。 - -::: - -可以使用 `jeprof` 从堆内存分析结果生成多种不同的输出格式。 -建议运行 `jeprof --help` 来查看该工具的用法以及提供的各类选项。 - -通常情况下,`jeprof` 命令的用法如下: - -```sh -jeprof 二进制文件路径 堆配置文件路径 --output_format [ > 输出文件] -``` - -如果你想比较在两个分析概要之间发生了哪些分配,可以设置 `base` 参数: - -```sh -jeprof 二进制文件路径 --base 第一个堆配置文件路径 第二个堆配置文件路径 --output_format [ > 输出文件] -``` - -### 示例 {#examples} - -* 如果你想生成一个文本文件,使每个过程各写在单独一行: - -```sh -jeprof 二进制文件路径 堆配置文件路径 --text > result.txt -``` - -* 如果你想生成带有调用关系图的 PDF 文件: - -```sh -jeprof path/to/binary path/to/heap/profile --pdf > result.pdf -``` - -### 生成火焰图 {#generating-flame-graph} - -`jeprof` 可以生成用于构建火焰图的折叠后调用栈。 - -需要使用 `--collapsed` 参数: - -```sh -jeprof path/to/binary path/to/heap/profile --collapsed > result.collapsed -``` - -接下来,你可以使用许多不同的工具来可视化折叠后的调用栈。 - -最常用的是 [FlameGraph](https://github.com/brendangregg/FlameGraph),其中包含一个名为 `flamegraph.pl` 的脚本: - -```sh -cat result.collapsed | /path/to/FlameGraph/flamegraph.pl --color=mem --title="Allocation Flame Graph" --width 2400 > result.svg -``` - -另一个有用的工具是 [speedscope](https://www.speedscope.app/),它使你能够以更直观、交互的方式分析收集到的调用栈。 - - -## 分析器的其他选项 {#additional-options-for-profiler} - -`jemalloc` 提供了许多与分析器相关的选项。可以通过设置 `MALLOC_CONF` 环境变量来进行配置。 -例如,可以使用 `lg_prof_sample` 控制分配采样之间的间隔。 -如果你希望每分配 N 字节就导出一次堆分析概要,可以通过 `lg_prof_interval` 启用该功能。 - -建议查看 `jemalloc` 的[参考页面](https://jemalloc.net/jemalloc.3.html)以获取完整的选项列表。 - - - -## 其他资源 {#other-resources} - -ClickHouse/Keeper 通过多种方式暴露与 `jemalloc` 相关的指标。 - -:::warning 警告 -需要注意的是,这些指标之间并不同步,数值可能会出现偏移。 -::: - -### 系统表 `asynchronous_metrics` {#system-table-asynchronous_metrics} - -```sql -SELECT * -FROM system.asynchronous_metrics -WHERE metric LIKE '%jemalloc%' -FORMAT Vertical -``` - -[参考](/operations/system-tables/asynchronous_metrics) - -### 系统表 `jemalloc_bins` {#system-table-jemalloc_bins} - -包含通过 jemalloc 分配器在不同大小类(bins)中的内存分配情况,这些信息从所有 arena 聚合而来。 - -[参考](/operations/system-tables/jemalloc_bins) - -### Prometheus {#prometheus} - -来自 `asynchronous_metrics` 的所有与 `jemalloc` 相关的指标,也会通过 ClickHouse 和 Keeper 中的 Prometheus 端点对外暴露。 - -[参考](/operations/server-configuration-parameters/settings#prometheus) - -### Keeper 中的 `jmst` 4LW 命令 {#jmst-4lw-command-in-keeper} - -Keeper 支持 `jmst` 4LW 命令,其返回[基础分配器统计信息](https://github.com/jemalloc/jemalloc/wiki/Use-Case%3A-Basic-Allocator-Statistics): - -```sh -echo jmst | nc localhost 9181 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/analyzer.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/analyzer.md deleted file mode 100644 index 4485960ecdf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/analyzer.md +++ /dev/null @@ -1,201 +0,0 @@ ---- -description: '详细介绍 ClickHouse 查询分析器的页面' -keywords: ['分析器'] -sidebar_label: '分析器' -slug: /operations/analyzer -title: '分析器' -doc_type: 'reference' ---- - -# 分析器 {#analyzer} - -在 ClickHouse `24.3` 版本中,新的查询分析器默认启用。 -您可以在[此处](/guides/developer/understanding-query-execution-with-the-analyzer#analyzer)阅读其工作原理的更多细节。 - -## 已知不兼容项 {#known-incompatibilities} - -尽管修复了大量 bug 并引入了新的优化,但这也对 ClickHouse 的行为带来了一些不兼容的变更。请阅读以下变更说明,以确定如何为新的 analyzer 重写你的查询。 - -### 无效查询不再被优化 {#invalid-queries-are-no-longer-optimized} - -之前的查询规划架构会在查询验证步骤之前应用 AST 级别的优化。 -这些优化可能将初始查询改写为有效且可执行的形式。 - -在新的 analyzer 中,查询验证发生在优化步骤之前。 -这意味着此前仍然可以执行的无效查询,现在将不再被支持。 -在这种情况下,必须手动修正查询。 - -#### 示例 1 {#example-1} - -下面的查询在投影列表中使用了列 `number`,但在聚合之后只有 `toString(number)` 可用。 -在旧的 analyzer 中,`GROUP BY toString(number)` 会被优化为 `GROUP BY number,`,从而使查询变为有效。 - -```sql -SELECT number -FROM numbers(1) -GROUP BY toString(number) -``` - -#### 示例 2 {#example-2} - -在这个查询中也会出现相同的问题。列 `number` 在与另一个键一起聚合之后被使用。 -之前的查询分析器通过将 `number > 5` 过滤条件从 `HAVING` 子句移动到 `WHERE` 子句来修复这个查询。 - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -GROUP BY n -HAVING number > 5 -``` - -要更正该查询,你应将所有适用于非聚合列的条件移到 `WHERE` 子句中,以符合标准 SQL 语法: - -```sql -SELECT - number % 2 AS n, - sum(number) -FROM numbers(10) -WHERE number > 5 -GROUP BY n -``` - -### 使用无效查询的 `CREATE VIEW` {#create-view-with-invalid-query} - -新的分析器始终会执行类型检查。 -此前,可以使用无效的 `SELECT` 查询创建一个 `VIEW`。 -然后它会在第一次执行 `SELECT` 或 `INSERT` 时失败(对于 `MATERIALIZED VIEW` 也是如此)。 - -现在不再允许以这种方式创建 `VIEW`。 - -#### 示例 {#example-view} - -```sql -CREATE TABLE source (data String) -ENGINE=MergeTree -ORDER BY tuple(); - -CREATE VIEW some_view -AS SELECT JSONExtract(data, 'test', 'DateTime64(3)') -FROM source; -``` - -### `JOIN` 子句的已知不兼容性 {#known-incompatibilities-of-the-join-clause} - -#### 使用投影中的列进行 `JOIN` {#join-using-column-from-projection} - -默认情况下,来自 `SELECT` 列表的别名不能用作 `JOIN USING` 的键。 - -有一个新的设置 `analyzer_compatibility_join_using_top_level_identifier`,启用后会改变 `JOIN USING` 的行为,在解析标识符时,将优先基于 `SELECT` 查询投影列表中的表达式,而不是直接使用左表中的列。 - -例如: - -```sql -SELECT a + 1 AS b, t2.s -FROM VALUES('a UInt64, b UInt64', (1, 1)) AS t1 -JOIN VALUES('b UInt64, s String', (1, 'one'), (2, 'two')) t2 -USING (b); -``` - -将 `analyzer_compatibility_join_using_top_level_identifier` 设置为 `true` 时,连接条件会被解释为 `t1.a + 1 = t2.b`,这与早期版本的行为一致。 -结果将会是 `2, 'two'`。 -当该设置为 `false` 时,连接条件默认为 `t1.b = t2.b`,查询将返回 `2, 'one'`。 -如果 `t1` 中不存在 `b`,查询将失败并报错。 - -#### 使用 `JOIN USING` 与 `ALIAS`/`MATERIALIZED` 列时的行为变化 {#changes-in-behavior-with-join-using-and-aliasmaterialized-columns} - -在新的 analyzer 中,在包含 `ALIAS` 或 `MATERIALIZED` 列的 `JOIN USING` 查询中使用 `*` 时,这些列默认会包含在结果集中。 - -例如: - -```sql -CREATE TABLE t1 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t1 VALUES (1), (2); - -CREATE TABLE t2 (id UInt64, payload ALIAS sipHash64(id)) ENGINE = MergeTree ORDER BY id; -INSERT INTO t2 VALUES (2), (3); - -SELECT * FROM t1 -FULL JOIN t2 USING (payload); -``` - -在新的分析器中,此查询的结果将包含两个表中的 `id` 列以及 `payload` 列。 -相比之下,旧的分析器只有在启用了特定设置(`asterisk_include_alias_columns` 或 `asterisk_include_materialized_columns`)时才会包含这些 `ALIAS` 列, -并且这些列的顺序可能会不同。 - -为确保结果一致且符合预期,尤其是在将旧查询迁移到新的分析器时,建议在 `SELECT` 子句中显式指定列,而不是使用 `*`。 - -#### 在 `USING` 子句中对列表达式类型修饰符的处理 {#handling-of-type-modifiers-for-columns-in-using-clause} - -在新版本的分析器中,用于确定 `USING` 子句中指定列的公共超类型的规则已被统一,以产生更加可预测的结果, -尤其是在处理 `LowCardinality` 和 `Nullable` 等类型修饰符时。 - -* `LowCardinality(T)` 和 `T`:当类型为 `LowCardinality(T)` 的列与类型为 `T` 的列进行联接时,得到的公共超类型将是 `T`,`LowCardinality` 修饰符会被舍弃。 -* `Nullable(T)` 和 `T`:当类型为 `Nullable(T)` 的列与类型为 `T` 的列进行联接时,得到的公共超类型将是 `Nullable(T)`,从而确保可为空属性被保留。 - -例如: - -```sql -SELECT id, toTypeName(id) -FROM VALUES('id LowCardinality(String)', ('a')) AS t1 -FULL OUTER JOIN VALUES('id String', ('b')) AS t2 -USING (id); -``` - -在此查询中,`id` 的共同超类型被确定为 `String`,并丢弃来自 `t1` 的 `LowCardinality` 修饰符。 - -### 投影列名的变化 {#projection-column-names-changes} - -在计算投影的列名时,不会替换别名。 - -```sql -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 0 -FORMAT PrettyCompact - - ┌─x─┬─plus(plus(1, 1), 1)─┐ -1. │ 2 │ 3 │ - └───┴─────────────────────┘ - -SELECT - 1 + 1 AS x, - x + 1 -SETTINGS enable_analyzer = 1 -FORMAT PrettyCompact - - ┌─x─┬─plus(x, 1)─┐ -1. │ 2 │ 3 │ - └───┴────────────┘ -``` - -### 不兼容的函数参数类型 {#incompatible-function-arguments-types} - -在新的分析器中,类型推断发生在初始查询分析阶段。 -这一变更意味着类型检查会在短路求值之前进行;因此,`if` 函数的参数必须始终具有一个公共超类型。 - -例如,下面的查询会失败,并报错 `There is no supertype for types Array(UInt8), String because some of them are Array and some of them are not`: - -```sql -SELECT toTypeName(if(0, [2, 3, 4], 'String')) -``` - -### 异构集群 {#heterogeneous-clusters} - -新的 analyzer 显著改变了集群中服务器之间的通信协议。因此,无法在 `enable_analyzer` 设置值不同的服务器上运行分布式查询。 - -### 变更语句仍由旧版 analyzer 解析 {#mutations-are-interpreted-by-previous-analyzer} - -变更语句(mutations)仍然使用旧版 analyzer 进行解析。 -这意味着某些新的 ClickHouse SQL 功能目前无法用于变更语句中。例如,`QUALIFY` 子句。 -可以在[这里](https://github.com/ClickHouse/ClickHouse/issues/61563)查看当前状态。 - -### 不支持的功能 {#unsupported-features} - -当前新 analyzer 尚不支持的功能列表如下: - -* Annoy 索引。 -* Hypothesis 索引。支持仍在开发中,[见此处](https://github.com/ClickHouse/ClickHouse/pull/48381)。 -* 不支持 Window View。未来也没有支持的计划。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/backup.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/backup.md deleted file mode 100644 index 76102e0e533..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/backup.md +++ /dev/null @@ -1,574 +0,0 @@ ---- -description: 'ClickHouse 数据库和表的备份与恢复指南' -sidebar_label: '备份与恢复' -sidebar_position: 10 -slug: /operations/backup -title: '备份与恢复' -doc_type: 'guide' ---- - - - -# 备份与恢复 {#backup-and-restore} - -- [备份到本地磁盘](#backup-to-a-local-disk) -- [配置使用 S3 端点进行备份/恢复](#configuring-backuprestore-to-use-an-s3-endpoint) -- [使用 S3 磁盘进行备份/恢复](#backuprestore-using-an-s3-disk) -- [其他方案](#alternatives) - - - -## 命令概览 {#command-summary} - -```bash - BACKUP|RESTORE - TABLE [db.]table_name [AS [db.]table_name_in_backup] - [PARTITION[S] partition_expr [, ...]] | - DICTIONARY [db.]dictionary_name [AS [db.]name_in_backup] | - DATABASE database_name [AS database_name_in_backup] - [EXCEPT TABLES ...] | - TEMPORARY TABLE table_name [AS table_name_in_backup] | - VIEW view_name [AS view_name_in_backup] | - ALL [EXCEPT {TABLES|DATABASES}...] } [, ...] - [ON CLUSTER 'cluster_name'] - TO|FROM File('/') | Disk('', '/') | S3('/', '', '') - [SETTINGS base_backup = File('/') | Disk(...) | S3('/', '', '')] - [SYNC|ASYNC] - -``` - -:::note ALL -在 ClickHouse 23.4 版本之前,`ALL` 仅可用于 `RESTORE` 命令。 -::: - - -## 背景 {#background} - -虽然[复制](../engines/table-engines/mergetree-family/replication.md)可以防止硬件故障带来的影响,但它无法防止人为错误:例如误删数据、删除了错误的表或错误集群上的表,以及由于软件缺陷导致的数据处理错误或数据损坏。在许多情况下,这类错误会影响所有副本。ClickHouse 内置了一些保护机制来防止某些类型的错误——例如,默认情况下,[你不能直接删除使用类 MergeTree 引擎且包含超过 50 GB 数据的表](/operations/settings/settings#max_table_size_to_drop)。但是,这些保护机制并不能覆盖所有可能的情况,并且可能被绕过。 - -为了有效降低人为错误带来的风险,你应当**提前**认真制定数据备份与恢复策略。 - -每家公司的可用资源和业务需求都不同,因此不存在一种适用于所有场景的 ClickHouse 备份与恢复通用方案。对 1 GB 数据有效的方法,很可能并不适用于数十 PB 的数据。有多种可选方案,各自都有优缺点,后文会进行讨论。建议不要只依赖单一方案,而是结合多种方法,以相互弥补各自的不足。 - -:::note -请记住,如果你只做了备份却从未尝试过恢复,那么在你真正需要恢复时,它很有可能无法按预期工作(或者至少,其耗时会超过业务可接受的范围)。因此,无论你选择哪种备份方案,都务必同时实现恢复过程的自动化,并在备用的 ClickHouse 集群上定期演练恢复。 -::: - - - -## 备份到本地磁盘 {#backup-to-a-local-disk} - -### 配置备份目标 {#configure-a-backup-destination} - -在下面的示例中,备份目标被指定为 `Disk('backups', '1.zip')`。要准备该目标,请在 `/etc/clickhouse-server/config.d/backup_disk.xml` 中创建一个文件,用于指定备份目标。例如,下面这个文件定义了名为 `backups` 的磁盘,然后将该磁盘添加到 **backups > allowed_disk** 列表中: - -```xml - - - - - - local - /backups/ - - - - - - backups - /backups/ - - - -``` - -### 参数 {#parameters} - -备份可以是全量或增量的,并且可以包含表(包括物化视图、投影(projection)和字典)以及数据库。备份可以是同步的(默认)或异步的,可以进行压缩,也可以通过密码进行保护。 - -`BACKUP` 和 `RESTORE` 语句接收一个由 `DATABASE` 和 `TABLE` 名称、目标(或源)、选项和设置组成的列表: - -* 备份的目标,或恢复时的源。它基于前面定义的磁盘。例如 `Disk('backups', 'filename.zip')` -* `ASYNC`:异步备份或恢复 -* `PARTITIONS`:要恢复的分区列表 -* `SETTINGS`: - * `id`:备份或恢复操作的标识符。如果未设置或为空,将使用随机生成的 UUID。 - 如果显式设置为非空字符串,则每次都应不同。该 `id` 用于在 `system.backups` 表中查找与特定备份或恢复操作相关的行。 - * [`compression_method`](/sql-reference/statements/create/table#column_compression_codec) 和 `compression_level` - * 磁盘上文件的 `password` - * `base_backup`:此源上一次备份的目标位置。例如 `Disk('backups', '1.zip')` - * `use_same_s3_credentials_for_base_backup`:基础备份到 S3 时,是否应从查询继承凭证。仅在使用 `S3` 时有效。 - * `use_same_password_for_base_backup`:基础备份归档是否应从查询继承密码。 - * `structure_only`:如果启用,则仅备份或恢复 `CREATE` 语句,而不包含表数据 - * `storage_policy`:要恢复的表的存储策略。参见 [使用多个块设备进行数据存储](../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes)。此设置仅适用于 `RESTORE` 命令。指定的存储策略只应用于使用 `MergeTree` 系列引擎的表。 - * `s3_storage_class`:用于 S3 备份的存储类别。例如 `STANDARD` - * `azure_attempt_to_create_container`:在使用 Azure Blob Storage 时,如果指定的容器不存在,是否尝试创建该容器。默认值:`true`。 - * 这里也可以使用[核心设置](/operations/settings/settings) - -### 使用示例 {#usage-examples} - -先备份,再恢复一个表: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.zip') -``` - -对应的恢复操作: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -``` - -:::note -如果表 `test.table` 中已经包含数据,上面的 RESTORE 将失败。若要测试 RESTORE 操作,你需要先删除该表,或者使用设置 `allow_non_empty_tables=true`: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.zip') -SETTINGS allow_non_empty_tables=true -``` - -::: - -恢复或备份表时,可以使用新名称: - -```sql -RESTORE TABLE test.table AS test.table2 FROM Disk('backups', '1.zip') -``` - -```sql -BACKUP TABLE test.table3 AS test.table4 TO Disk('backups', '2.zip') -``` - -### 增量备份 {#incremental-backups} - -可以通过指定 `base_backup` 来创建增量备份。 -:::note -增量备份依赖于基础备份。必须确保基础备份始终可用,才能从增量备份中完成恢复。 -::: - - -以增量方式存储新数据。将 `base_backup` 进行相应设置后,自上一次备份到 `Disk('backups', 'd.zip')` 以来产生的数据会被存储到 `Disk('backups', 'incremental-a.zip')` 中: - -```sql -BACKUP TABLE test.table TO Disk('backups', 'incremental-a.zip') - SETTINGS base_backup = Disk('backups', 'd.zip') -``` - -从增量备份和 base_backup 中将所有数据恢复到新表 `test.table2` 中: - -```sql -RESTORE TABLE test.table AS test.table2 - FROM Disk('backups', 'incremental-a.zip'); -``` - -### 为备份设置密码 {#assign-a-password-to-the-backup} - -可以为写入磁盘的备份文件设置密码: - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -恢复: - -```sql -RESTORE TABLE test.table - FROM Disk('backups', 'password-protected.zip') - SETTINGS password='qwerty' -``` - -### 压缩设置 {#compression-settings} - -如果需要指定压缩方式或压缩级别: - -```sql -BACKUP TABLE test.table - TO Disk('backups', 'filename.zip') - SETTINGS compression_method='lzma', compression_level=3 -``` - -### 恢复特定分区 {#restore-specific-partitions} - -如果需要恢复与某个表关联的特定分区,可以单独指定这些分区。要从备份中恢复分区 1 和 4: - -```sql -RESTORE TABLE test.table PARTITIONS '2', '3' - FROM Disk('backups', 'filename.zip') -``` - -### 以 tar 归档形式存储备份 {#backups-as-tar-archives} - -备份也可以以 tar 归档形式存储。其功能与 zip 相同,但不支持密码。 - -将备份写成 tar 归档: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar') -``` - -对应的恢复操作: - -```sql -RESTORE TABLE test.table FROM Disk('backups', '1.tar') -``` - -要更改压缩方式,需要在备份名称后添加正确的文件后缀。例如,要使用 gzip 压缩 tar 归档: - -```sql -BACKUP TABLE test.table TO Disk('backups', '1.tar.gz') -``` - -支持的压缩文件后缀包括 `tar.gz`、`.tgz`、`tar.bz2`、`tar.lzma`、`.tar.zst`、`.tzst` 和 `.tar.xz`。 - -### 检查备份状态 {#check-the-status-of-backups} - -备份命令会返回一个 `id` 和 `status`,该 `id` 可用于获取备份的状态。这对于检查长时间运行的 ASYNC(异步)备份进度非常有用。下面的示例展示了在尝试覆盖已有备份文件时发生的失败情况: - -```sql -BACKUP TABLE helloworld.my_first_table TO Disk('backups', '1.zip') ASYNC -``` - -```response -┌─id───────────────────────────────────┬─status──────────┐ -│ 7678b0b3-f519-4e6e-811f-5a0781a4eb52 │ CREATING_BACKUP │ -└──────────────────────────────────────┴─────────────────┘ - -返回 1 行。耗时: 0.001 秒。 -``` - -```sql -SELECT - * -FROM system.backups -WHERE id='7678b0b3-f519-4e6e-811f-5a0781a4eb52' -FORMAT Vertical -``` - -```response -第 1 行: -────── -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -#highlight-next-line -status: BACKUP_FAILED -num_files: 0 -uncompressed_size: 0 -compressed_size: 0 -#highlight-next-line -error: Code: 598. DB::Exception: Backup Disk('backups', '1.zip') already exists. (BACKUP_ALREADY_EXISTS) (version 22.8.2.11 (official build)) -start_time: 2022-08-30 09:21:46 -end_time: 2022-08-30 09:21:46 - -返回 1 行。用时:0.002 秒。 -``` - - -除了 `system.backups` 表之外,所有备份和恢复操作还会记录在系统日志表 [backup_log](../operations/system-tables/backup_log.md) 中: - -```sql -SELECT * -FROM system.backup_log -WHERE id = '7678b0b3-f519-4e6e-811f-5a0781a4eb52' -ORDER BY event_time_microseconds ASC -FORMAT Vertical -``` - -```response -第 1 行: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.097414 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-18 11:13:43 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -第 2 行: -────── -event_date: 2023-08-18 -event_time_microseconds: 2023-08-18 11:13:43.174782 -id: 7678b0b3-f519-4e6e-811f-5a0781a4eb52 -name: Disk('backups', '1.zip') -status: BACKUP_FAILED -#highlight-next-line -error: Code: 598. DB::Exception: Backup Disk('backups', '1.zip') already exists. (BACKUP_ALREADY_EXISTS) (version 23.8.1.1) -start_time: 2023-08-18 11:13:43 -end_time: 2023-08-18 11:13:43 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -2 行结果集。用时:0.075 秒。 -``` - - -## 配置 BACKUP/RESTORE 以使用 S3 Endpoint {#configuring-backuprestore-to-use-an-s3-endpoint} - -要将备份写入 S3 bucket,您需要以下三项信息: - -* S3 endpoint, - 例如 `https://mars-doc-test.s3.amazonaws.com/backup-S3/` -* Access key ID, - 例如 `ABC123` -* Secret access key, - 例如 `Abc+123` - -:::note -创建 S3 bucket 的步骤已在 [将 S3 对象存储用作 ClickHouse 磁盘](/integrations/data-ingestion/s3/index.md#configuring-s3-for-clickhouse-use) 中说明。保存策略后再回到本文档即可,无需将 ClickHouse 配置为使用该 S3 bucket。 -::: - -备份的目标位置将按如下方式指定: - -```sql -S3('/<目录>', '<访问密钥 ID>', '<访问密钥密文>') -``` - -```sql -CREATE TABLE data -( - `key` Int, - `value` String, - `array` Array(String) -) -ENGINE = MergeTree -ORDER BY tuple() -``` - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 1000 -``` - -### 创建基础(初始)备份 {#create-a-base-initial-backup} - -增量备份需要先有一个*基础*备份作为起点,本示例稍后将作为基础备份使用。S3 目标的第一个参数是 S3 endpoint,后面是本次备份在 bucket 中使用的目录。在本示例中,该目录名为 `my_backup`。 - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ de442b75-a66c-4a3c-a193-f76f278c70f3 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### 添加更多数据 {#add-more-data} - -增量备份会保存基础备份与当前表内容之间的差异。在执行增量备份之前,先向表中添加更多数据: - -```sql -INSERT INTO data SELECT * -FROM generateRandom('key Int, value String, array Array(String)') -LIMIT 100 -``` - -### 执行增量备份 {#take-an-incremental-backup} - -此备份命令与基础备份类似,但额外指定了 `SETTINGS base_backup` 以及基础备份的位置。请注意,增量备份的目标路径与基础备份并非同一目录,而是在同一端点下、存储桶内的另一个目标目录。基础备份位于 `my_backup`,增量备份将写入 `my_incremental`: - -```sql -BACKUP TABLE data TO S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') SETTINGS base_backup = S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_backup', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ f6cd3900-850f-41c9-94f1-0c4df33ea528 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -### 从增量备份恢复 {#restore-from-the-incremental-backup} - -此命令会将增量备份还原到一个名为 `data3` 的新表中。请注意,当恢复增量备份时,基础备份也会一并包含在内。恢复时只需指定增量备份即可: - -```sql -RESTORE TABLE data AS data3 FROM S3('https://mars-doc-test.s3.amazonaws.com/backup-S3/my_incremental', 'ABC123', 'Abc+123') -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ ff0c8c39-7dff-4324-a241-000796de11ca │ 已恢复 │ -└──────────────────────────────────────┴──────────┘ -``` - -### 验证计数 {#verify-the-count} - - -在原始表 `data` 中进行了两次插入操作,一次插入 1,000 行,一次插入 100 行,共计 1,100 行。请验证还原后的表是否有 1,100 行: - -```sql -SELECT count() -FROM data3 -``` - -```response -┌─count()─┐ -│ 1100 │ -└─────────┘ -``` - - -### 验证内容 {#verify-the-content} - -这会将原始表 `data` 的内容与恢复后的表 `data3` 进行比较: - -```sql -SELECT throwIf(( - SELECT groupArray(tuple(*)) - FROM data - ) != ( - SELECT groupArray(tuple(*)) - FROM data3 - ), 'BACKUP/RESTORE 后数据不匹配') -``` - -## 使用 S3 磁盘执行 BACKUP/RESTORE {#backuprestore-using-an-s3-disk} - -也可以通过在 ClickHouse 存储配置中配置一个 S3 磁盘,将数据 `BACKUP`/`RESTORE` 到 S3。在 `/etc/clickhouse-server/config.d` 中添加一个文件,按如下方式配置该磁盘: - -```xml - - - - - s3_plain - - - - - - - - -
- s3_plain -
-
-
-
-
- - - s3_plain - -
-``` - -然后照常执行 `BACKUP`/`RESTORE`: - -```sql -BACKUP TABLE data TO Disk('s3_plain', 'cloud_backup'); -RESTORE TABLE data AS data_restored FROM Disk('s3_plain', 'cloud_backup'); -``` - -:::note -但请注意: - -* 此磁盘不应被用于 `MergeTree` 本身,仅应用于 `BACKUP`/`RESTORE` -* 如果你的表使用 S3 存储作为后端,系统会尝试通过 `CopyObject` 调用在 S3 侧进行服务器端拷贝,使用相应凭证将数据分片复制到目标 bucket。若发生身份验证错误,则会退回为使用缓冲区拷贝的方法(先下载分片再上传),这种方式效率非常低。在这种情况下,你可能需要确保使用目标 bucket 的凭证对源 bucket 拥有 `read` 权限。 - ::: - - -## 使用命名集合 {#using-named-collections} - -命名集合可以用于 `BACKUP`/`RESTORE` 参数。示例请参见 [此处](./named-collections.md#named-collections-for-backups)。 - - - -## 替代方案 {#alternatives} - -ClickHouse 将数据存储在磁盘上,而对磁盘进行备份的方法有很多。下面是一些过去曾经使用过的方案,也有可能非常适合你的环境。 - -### 在其他位置复制源数据 {#duplicating-source-data-somewhere-else} - -通常,摄取到 ClickHouse 的数据是通过某种持久化队列传送过来的,例如 [Apache Kafka](https://kafka.apache.org)。在这种情况下,可以配置额外的一组订阅者,在数据被写入 ClickHouse 的同时读取同一条数据流,并将其存储到某个冷存储中。大多数公司通常已经有默认推荐的冷存储方案,例如对象存储或类似 [HDFS](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) 这样的分布式文件系统。 - -### 文件系统快照 {#filesystem-snapshots} - -某些本地文件系统提供快照功能(例如 [ZFS](https://en.wikipedia.org/wiki/ZFS)),但它们可能并不是为在线查询提供服务的最佳选择。一种可行的解决方案是使用这种文件系统创建额外副本,并将这些副本从用于 `SELECT` 查询的 [Distributed](../engines/table-engines/special/distributed.md) 表中排除。对这些副本进行快照时,将不会受到任何修改数据的查询影响。额外的好处是,这些副本可以使用具有更多磁盘挂载到单台服务器上的特殊硬件配置,从而更具成本效益。 - -对于数据量较小的场景,对远程表执行一个简单的 `INSERT INTO ... SELECT ...` 也同样可行。 - -### 对数据分片的操作 {#manipulations-with-parts} - -ClickHouse 允许使用 `ALTER TABLE ... FREEZE PARTITION ...` 查询来创建表分区的本地副本。其实现方式是对 `/var/lib/clickhouse/shadow/` 目录使用硬链接,因此对于旧数据通常不会额外占用磁盘空间。生成的文件副本不会被 ClickHouse 服务器管理,因此可以直接将它们保留在那里:你将获得一个无需任何额外外部系统的简单备份,但这种方式仍然容易受到硬件故障的影响。基于这一原因,更好的做法是将它们远程复制到其他位置,然后再删除本地副本。分布式文件系统和对象存储依然是不错的选择,但容量足够大的普通挂载式文件服务器也同样可行(在这种情况下,传输会通过网络文件系统,或者可能使用 [rsync](https://en.wikipedia.org/wiki/Rsync))。 -可以使用 `ALTER TABLE ... ATTACH PARTITION ...` 从备份中恢复数据。 - -关于分区操作相关查询的更多信息,请参阅 [ALTER 文档](/sql-reference/statements/alter/partition)。 - -有一个第三方工具可以用来自动化此方案:[clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup)。 - - - -## 禁止并发备份/恢复的设置 {#settings-to-disallow-concurrent-backuprestore} - -要禁止备份和恢复操作并发执行,可以分别使用以下设置。 - -```xml - - - false - false - - -``` - -这两个设置的默认值都是 true,因此默认情况下允许并发执行备份和还原。 -当在集群上将这两个设置设为 false 时,集群中同一时间只能运行 1 个备份或还原任务。 - - -## 配置 BACKUP/RESTORE 以使用 AzureBlobStorage 端点 {#configuring-backuprestore-to-use-an-azureblobstorage-endpoint} - -要将备份写入 AzureBlobStorage 容器,您需要以下信息: - -* AzureBlobStorage 端点连接字符串 / URL, -* 容器, -* 路径, -* 帐户名称(如果指定了 URL), -* 帐户密钥(如果指定了 URL) - -备份目标应按如下方式指定: - -```sql -AzureBlobStorage('/', '', '', '', '') -``` - -```sql -BACKUP TABLE data TO AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -RESTORE TABLE data AS data_restored FROM AzureBlobStorage('DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:10000/devstoreaccount1/;', - 'testcontainer', 'data_backup'); -``` - - -## 备份系统表 {#backup-up-system-tables} - -系统表也可以纳入备份和恢复流程,但是否包含它们取决于具体使用场景。 - -### 备份日志表 {#backing-up-log-tables} - -存储历史数据的系统表(例如带有 `_log` 后缀的表,如 `query_log`、`part_log`)可以像其他任何表一样进行备份和恢复。如果你的使用场景依赖分析历史数据——例如使用 `query_log` 跟踪查询性能或排查问题——建议在备份策略中包含这些表。相反,如果不需要这些表中的历史数据,则可以将其排除,以节省备份存储空间。 - -### 备份访问管理表 {#backing-up-access-management-tables} - -与访问管理相关的系统表,如 `users`、`roles`、`row_policies`、`settings_profiles` 和 `quotas`,在备份和恢复操作中会被特殊处理。当这些表被包含在备份中时,它们的内容会被导出到一个特殊的 `accessXX.txt` 文件中,该文件封装了用于创建和配置访问实体的等效 SQL 语句。在恢复时,恢复过程会解析这些文件并重新执行其中的 SQL 命令,以重新创建用户、角色和其他配置。 - -此功能确保 ClickHouse 集群的访问控制配置可以作为集群整体配置的一部分进行备份和恢复。 - -注意:该功能仅适用于通过 SQL 命令管理的配置(称为 ["SQL-driven Access Control and Account Management"](/operations/access-rights#enabling-access-control))。在 ClickHouse 服务器配置文件(例如 `users.xml`)中定义的访问配置不会包含在备份中,且无法通过此方法恢复。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/caches.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/caches.md deleted file mode 100644 index 119ec7836c1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/caches.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: '在执行查询时,ClickHouse 会使用不同的缓存。' -sidebar_label: '缓存' -sidebar_position: 65 -slug: /operations/caches -title: '缓存类型' -keywords: ['cache'] -doc_type: 'reference' ---- - -# 缓存类型 {#cache-types} - -在执行查询时,ClickHouse 使用不同类型的缓存来加速查询, -并减少对磁盘的读写需求。 - -主要的缓存类型包括: - -* `mark_cache` — [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md) 系列表引擎使用的[标记](/development/architecture#merge-tree)缓存。 -* `uncompressed_cache` — [`MergeTree`](../engines/table-engines/mergetree-family/mergetree.md) 系列表引擎使用的未压缩数据缓存。 -* 操作系统提供的页缓存(间接使用,用于实际数据文件)。 - -此外,还有多种其他缓存类型: - -* DNS 缓存。 -* [Regexp](/interfaces/formats/Regexp) 缓存。 -* 已编译表达式缓存。 -* [向量相似度索引](../engines/table-engines/mergetree-family/annindexes.md)缓存。 -* [文本索引](../engines/table-engines/mergetree-family/invertedindexes.md#caching)缓存。 -* [Avro 格式](/interfaces/formats/Avro) Schema 缓存。 -* [字典](../sql-reference/dictionaries/index.md)数据缓存。 -* Schema 推断缓存。 -* 基于 S3、Azure、本地以及其他磁盘的[文件系统缓存](storing-data.md)。 -* [用户态页缓存](/operations/userspace-page-cache)。 -* [查询缓存](query-cache.md)。 -* [查询条件缓存](query-condition-cache.md)。 -* 格式 Schema 缓存。 - -如果希望出于性能调优、故障排查或数据一致性等原因清除某一种缓存, -可以使用 [`SYSTEM DROP ... CACHE`](../sql-reference/statements/system.md) 语句。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md deleted file mode 100644 index 33922916c15..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/cluster-discovery.md +++ /dev/null @@ -1,229 +0,0 @@ ---- -description: 'ClickHouse 集群发现文档' -sidebar_label: '集群发现' -slug: /operations/cluster-discovery -title: '集群发现' -doc_type: 'guide' ---- - -# 发现集群 {#cluster-discovery} - -## 概述 {#overview} - -ClickHouse 的集群发现(Cluster Discovery)功能通过允许节点在无需在配置文件中显式定义的情况下自动发现并注册自身,从而简化了集群配置。当需要手动定义每个节点变得繁琐时,这一功能尤其有用。 - -:::note - -集群发现是实验性功能,在未来版本中可能会被更改或移除。 -要启用它,请在配置文件中加入 `allow_experimental_cluster_discovery` 设置: - -```xml - - - 1 - - -``` - -::: - -## 远程服务器配置 {#remote-servers-configuration} - -### 传统的手动配置方式 {#traditional-manual-configuration} - -在过去,ClickHouse 集群中的每个分片和副本都需要在配置中手动指定: - -```xml - - - - - node1 - 9000 - - - node2 - 9000 - - - - - node3 - 9000 - - - node4 - 9000 - - - - - -``` - -### 使用集群发现 {#using-cluster-discovery} - -通过集群发现(Cluster Discovery),你无需显式定义每个节点,只需在 ZooKeeper 中指定一个路径。注册到该路径下的所有节点都会被自动发现并加入集群。 - -```xml - - - - /clickhouse/discovery/cluster_name - - - - - - - - - - - - - - - - - -``` - -若要为某个特定节点指定分片编号,可以在 `` 部分中添加 `` 标签: - -对于 `node1` 和 `node2`: - -```xml - - /clickhouse/discovery/cluster_name - 1 - -``` - -针对 `node3` 和 `node4`: - -```xml - - /clickhouse/discovery/cluster_name - 2 - -``` - -### 观察者模式 {#observer-mode} - -以观察者模式配置的节点不会将自身注册为副本。 -它们只会在集群中观察并发现其他活动副本,而不会主动参与。 -要启用观察者模式,请在 `` 配置段中添加 `` 标签: - -```xml - - /clickhouse/discovery/cluster_name - - -``` - -### 集群发现 {#discovery-of-clusters} - -有时你可能不仅需要在集群中添加或删除主机,还需要添加或删除整个集群本身。你可以使用 `` 节点,将其作为多个集群的根路径: - -```xml - - - - /clickhouse/discovery - - - - -``` - -在这种情况下,当其他主机使用路径 `/clickhouse/discovery/some_new_cluster` 注册自身时,将会添加一个名为 `some_new_cluster` 的集群。 - -你可以同时使用这两种功能:该主机既可以在集群 `my_cluster` 中注册自身,又可以发现任何其他集群: - -```xml - - - - /clickhouse/discovery/my_cluster - - - - - /clickhouse/discovery - - - - -``` - -限制: - -* 在同一个 `remote_servers` 子树中不能同时使用 `` 和 ``。 -* `` 只能与 `` 搭配使用。 -* 来自 Keeper 的路径最后一段会被用作集群名称,而在注册时,名称是从 XML 标签中获取的。 - -## 使用场景和限制 {#use-cases-and-limitations} - -当在指定的 ZooKeeper 路径下添加或移除节点时,这些节点会在无需修改配置或重启服务器的情况下被自动发现或从集群中移除。 - -但需要注意,更改只会影响集群配置,不会影响数据或现有的数据库与表。 - -考虑以下示例:该集群包含 3 个节点: - -```xml - - - - /clickhouse/discovery/default_cluster - - - -``` - -```sql -SELECT * EXCEPT (default_database, errors_count, slowdowns_count, estimated_recovery_time, database_shard_name, database_replica_name) -FROM system.clusters WHERE cluster = 'default'; - -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -```sql -CREATE TABLE event_table ON CLUSTER default (event_time DateTime, value String) -ENGINE = ReplicatedMergeTree('/clickhouse/tables/event_table', '{replica}') -ORDER BY event_time PARTITION BY toYYYYMM(event_time); - -INSERT INTO event_table ... -``` - -然后,我们向集群中添加一个新节点,即启动一个其配置文件中 `remote_servers` 部分包含相同条目的节点: - -```response -┌─cluster─┬─shard_num─┬─shard_weight─┬─replica_num─┬─host_name────┬─host_address─┬─port─┬─is_local─┬─user─┬─is_active─┐ -│ default │ 1 │ 1 │ 1 │ 92d3c04025e8 │ 172.26.0.5 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 2 │ a6a68731c21b │ 172.26.0.4 │ 9000 │ 1 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 3 │ 8e62b9cb17a1 │ 172.26.0.2 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -│ default │ 1 │ 1 │ 4 │ b0df3669b81f │ 172.26.0.6 │ 9000 │ 0 │ │ ᴺᵁᴸᴸ │ -└─────────┴───────────┴──────────────┴─────────────┴──────────────┴──────────────┴──────┴──────────┴──────┴───────────┘ -``` - -第四个节点已参与集群,但 `event_table` 表仍然只存在于前三个节点上: - -```sql -SELECT hostname(), database, table FROM clusterAllReplicas(default, system.tables) WHERE table = 'event_table' FORMAT PrettyCompactMonoBlock -``` - -┌─hostname()───┬─database─┬─table───────┐ -│ a6a68731c21b │ default │ event_table │ -│ 92d3c04025e8 │ default │ event_table │ -│ 8e62b9cb17a1 │ default │ event_table │ -└──────────────┴──────────┴─────────────┘ - -``` - -如果需要在所有节点上复制表,可以使用 [Replicated](../engines/database-engines/replicated.md) 数据库引擎来替代集群发现功能。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/configuration-files.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/configuration-files.md deleted file mode 100644 index ebd34113304..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/configuration-files.md +++ /dev/null @@ -1,471 +0,0 @@ ---- -description: '本页说明如何使用 XML 或 YAML 语法的配置文件来配置 ClickHouse 服务器。' -sidebar_label: '配置文件' -sidebar_position: 50 -slug: /operations/configuration-files -title: '配置文件' -doc_type: 'guide' ---- - -:::note -基于 XML 的设置配置集和配置文件在 ClickHouse Cloud 中不受支持。因此,在 ClickHouse Cloud 中不会存在 `config.xml` 文件。相应地,应使用 SQL 命令,通过设置配置集来管理相关设置。 - -更多详情,参见 ["配置设置"](/manage/settings) -::: - -ClickHouse 服务器可以通过 XML 或 YAML 语法的配置文件进行配置。 -在大多数安装方式中,ClickHouse 服务器默认使用 `/etc/clickhouse-server/config.xml` 作为配置文件,但也可以在服务器启动时通过命令行选项 `--config-file` 或 `-C` 手动指定配置文件的位置。 -可以将其他配置文件放在相对于主配置文件的 `config.d/` 目录中,例如 `/etc/clickhouse-server/config.d/` 目录。 -在将配置应用到 ClickHouse 服务器之前,会有一个预处理步骤,将该目录中的文件与主配置文件合并。 -配置文件按字母顺序进行合并。 -为了简化更新并提高模块化程度,最佳实践是保持默认的 `config.xml` 文件不做修改,并将额外的自定义配置放入 `config.d/`。 -ClickHouse Keeper 的配置位于 `/etc/clickhouse-keeper/keeper_config.xml`。 -类似地,Keeper 的额外配置文件需要放在 `/etc/clickhouse-keeper/keeper_config.d/` 中。 - -可以混合使用 XML 和 YAML 配置文件,例如,你可以有一个主配置文件 `config.xml`,以及额外的配置文件 `config.d/network.xml`、`config.d/timezone.yaml` 和 `config.d/keeper.yaml`。 -不支持在单个配置文件中混用 XML 和 YAML。 -XML 配置文件应使用 `...` 作为顶层标签。 -在 YAML 配置文件中,`clickhouse:` 是可选的,如果缺失,解析器会自动插入该项。 - - - -## 合并配置 {#merging} - -两个配置文件(通常是主配置文件和来自 `config.d/` 的另一个配置文件)按如下规则进行合并: - -* 如果某个节点(即指向某个元素的路径)在两个文件中都出现,且没有 `replace` 或 `remove` 属性,则该节点会包含在合并后的配置文件中,并且会将两个节点的所有子节点一并包含并递归合并。 -* 如果两个节点中有一个包含 `replace` 属性,则该节点会包含在合并后的配置文件中,但只包含带有 `replace` 属性的那个节点的子节点。 -* 如果两个节点中有一个包含 `remove` 属性,则该节点不会包含在合并后的配置文件中(如果该节点已经存在,则会被删除)。 - -例如,给定两个配置文件: - -```xml title="config.xml" - - - 1 - - - 2 - - - 3 - - -``` - -和 - -```xml title="config.d/other_config.xml" - - - 4 - - - 5 - - - 6 - - -``` - -合并后的配置文件如下: - -```xml - - - 1 - 4 - - - 5 - - -``` - -### 使用环境变量和 ZooKeeper 节点进行替换 {#from_env_zk} - -要指定某个元素的值由环境变量的值替换,可以使用属性 `from_env`。 - -例如,若设置环境变量 `$MAX_QUERY_SIZE = 150000`: - -```xml - - - - - - - -``` - -生成的配置如下: - -```xml - - - - 150000 - - - -``` - -同样可以使用 `from_zk`(ZooKeeper 节点)来实现: - -```xml - - - -``` - - -```shell -# clickhouse-keeper-client {#clickhouse-keeper-client} -/ :) touch /zk_configs -/ :) create /zk_configs/postgresql_port "9005" -/ :) get /zk_configs/postgresql_port -9005 -``` - -最终得到如下配置: - -```xml - - 9005 - -``` - -#### 默认值 {#default-values} - -带有 `from_env` 或 `from_zk` 属性的元素还可以带有属性 `replace="1"`(该属性必须出现在 `from_env`/`from_zk` 之前)。 -在这种情况下,元素可以定义一个默认值。 -如果环境变量或 ZooKeeper 节点已设置,元素将取其值,否则将采用默认值。 - -下面重复前面的示例,不过假设未设置 `MAX_QUERY_SIZE`: - -```xml - - - - 150000 - - - -``` - -结果配置如下: - -```xml - - - - 150000 - - - -``` - - -## 使用文件内容进行替换 {#substitution-with-file-content} - -也可以使用文件内容来替换配置中的部分内容。这可以通过两种方式完成: - -* *替换值*:如果某个元素具有 `incl` 属性,其值会被引用文件的内容所替换。默认情况下,包含替换内容的文件路径为 `/etc/metrika.xml`。可以在服务器配置中的 [`include_from`](../operations/server-configuration-parameters/settings.md#include_from) 元素中更改该路径。替换值在此文件中的 `/clickhouse/substitution_name` 元素中指定。如果 `incl` 中指定的替换不存在,会被记录到日志中。要阻止 ClickHouse 记录缺失的替换项,请指定属性 `optional="true"`(例如,用于 [macros](../operations/server-configuration-parameters/settings.md#macros) 的设置)。 -* *替换元素*:如果要通过替换来替换整个元素,请使用 `include` 作为元素名。元素名 `include` 可以与属性 `from_zk = "/path/to/node"` 结合使用。在这种情况下,元素值会被 ZooKeeper 中 `/path/to/node` 节点的内容所替代。这同样适用于将整个 XML 子树存储为一个 ZooKeeper 节点的情况,此时该子树会完整插入到源元素中。 - -下面给出一个示例: - -```xml - - - - - - - - - - -``` - -如果你希望将替换内容与现有配置合并而不是追加,可以使用属性 `merge="true"`。例如:``。在这种情况下,现有配置会与替换内容合并,且现有配置中的设置会被替换内容中的值覆盖。 - - -## 加密和隐藏配置 {#encryption} - -可以使用对称加密来加密某个配置元素,例如明文密码或私钥。 -为此,先配置[加密编解码器(encryption codec)](../sql-reference/statements/create/table.md#encryption-codecs),然后在要加密的元素上添加属性 `encrypted_by`,其值设为该加密编解码器的名称。 - -与属性 `from_zk`、`from_env` 和 `incl`,或元素 `include` 不同,在预处理后的文件中不会执行替换(即不会对加密值进行解密)。 -解密仅在服务器进程的运行时进行。 - -例如: - -```xml - - - - - 00112233445566778899aabbccddeeff - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -属性 [`from_env`](#from_env_zk) 和 [`from_zk`](#from_env_zk) 同样可以用于 `encryption_codecs`: - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -```xml - - - - - - - - - - admin - 961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 - - - -``` - -可以在任意一个配置文件中定义加密密钥和加密值。 - -示例 `config.xml` 如下所示: - -```xml - - - - - - - - - -``` - -一个示例 `users.xml` 文件如下: - -```xml - - - - - 96280000000D000000000030D4632962295D46C6FA4ABF007CCEC9C1D0E19DA5AF719C1D9A46C446 - default - - - - -``` - -要对某个值进行加密,可以使用示例程序 `encrypt_decrypt`: - -```bash -./encrypt_decrypt /etc/clickhouse-server/config.xml -e AES_128_GCM_SIV abcd -``` - -```text -961F000000040000000000EEDDEF4F453CFE6457C4234BD7C09258BD651D85 -``` - -即使使用了加密的配置项,这些加密内容在预处理生成的配置文件中仍然会出现。 -如果这会对你的 ClickHouse 部署造成影响,有两种可选方案:要么将预处理文件的权限设置为 600,要么使用属性 `hide_in_preprocessed`。 - -例如: - -```xml - - - - admin - secret - - - -``` - - -## 用户设置 {#user-settings} - -`config.xml` 文件可以指定一个单独的配置文件,用于定义用户设置、配置概要(profile)和配额(quota)。该配置文件的相对路径通过 `users_config` 元素进行设置,默认值为 `users.xml`。如果省略 `users_config`,则用户设置、配置概要和配额会直接在 `config.xml` 中指定。 - -用户配置可以像 `config.xml` 和 `config.d/` 一样拆分到单独的文件中。 -目录名称为 `users_config` 设置值去掉 `.xml` 后缀后再拼接 `.d`。 -默认使用目录 `users.d`,因为 `users_config` 默认是 `users.xml`。 - -请注意,配置文件会首先根据设置进行[合并](#merging),然后才会处理 include。 - - - -## XML 示例 {#example} - -例如,你可以为每个用户使用单独的配置文件,如下所示: - -```bash -$ cat /etc/clickhouse-server/users.d/alice.xml -``` - -```xml - - - - analytics - - ::/0 - - ... - analytics - - - -``` - - -## YAML 示例 {#example-1} - -此处展示了用 YAML 编写的默认配置:[`config.yaml.example`](https://github.com/ClickHouse/ClickHouse/blob/master/programs/server/config.yaml.example)。 - -在 ClickHouse 配置方面,YAML 和 XML 格式之间存在一些差异。 -下面是使用 YAML 格式编写配置的一些提示。 - -具有文本值的 XML 标签在 YAML 中表示为一个键值对 - -```yaml -key: value -``` - -相应的 XML: - -```xml -value -``` - -嵌套的 XML 节点表示为 YAML 映射: - -```yaml -map_key: - key1: val1 - key2: val2 - key3: val3 -``` - -对应的 XML: - -```xml - - val1 - val2 - val3 - -``` - -要多次定义同一个 XML 标签,请使用 YAML 序列: - -```yaml -seq_key: - - val1 - - val2 - - key1: val3 - - map: - key2: val4 - key3: val5 -``` - -对应的 XML: - -```xml -val1 -val2 - - val3 - - - - val4 - val5 - - -``` - -要指定一个 XML 属性,可以使用带有 `@` 前缀的属性键。请注意,`@` 是 YAML 标准中的保留符号,因此必须用双引号括起来: - -```yaml -map: - "@attr1": value1 - "@attr2": value2 - key: 123 -``` - -相应的 XML: - -```xml - - 123 - -``` - -在 YAML 序列中也可以使用属性: - -```yaml -seq: - - "@attr1": value1 - - "@attr2": value2 - - 123 - - abc -``` - -相应的 XML: - -```xml -123 -abc -``` - -前述语法无法在 YAML 中表示带有 XML 属性的 XML 文本节点。可以通过使用 `#text` 属性键来处理这种特殊情况: - -```yaml -map_key: - "@attr1": value1 - "#text": value2 -``` - -相应的 XML: - -```xml -value2 -``` - - -## 实现细节 {#implementation-details} - -对于每个配置文件,服务器在启动时还会生成 `file-preprocessed.xml` 文件。这些文件包含所有已完成的替换和覆盖,仅供参考使用。如果在配置文件中使用了 ZooKeeper 替换,但在服务器启动时 ZooKeeper 不可用,服务器将从预处理文件中加载配置。 - -服务器会跟踪配置文件的更改,以及在执行替换和覆盖时所使用的文件和 ZooKeeper 节点,并动态重新加载用户和集群的设置。这意味着你可以在不重启服务器的情况下修改集群、用户及其设置。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md deleted file mode 100644 index 6567bb46dda..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/http.md +++ /dev/null @@ -1,109 +0,0 @@ ---- -description: 'HTTP 文档' -slug: /operations/external-authenticators/http -title: 'HTTP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -HTTP 服务器可用于对 ClickHouse 用户进行身份验证。HTTP 身份验证只能作为现有用户的外部验证方式,这些用户在 `users.xml` 或本地访问控制路径中定义。目前支持使用 GET 方法的 [Basic](https://datatracker.ietf.org/doc/html/rfc7617) 身份验证方案。 - -## HTTP 身份验证服务器定义 {#http-auth-server-definition} - -要定义 HTTP 身份验证服务器,必须在 `config.xml` 中添加 `http_authentication_servers` 节。 - -**示例** - -```xml - - - - - http://localhost:8000/auth - 1000 - 1000 - 1000 - 3 - 50 - 1000 - - Custom-Auth-Header-1 - Custom-Auth-Header-2 - - - - - - -``` - -请注意,你可以在 `http_authentication_servers` 部分中使用不同的名称定义多个 HTTP 服务器。 - -**参数** - -* `uri` - 用于发送认证请求的 URI - -用于与服务器通信的套接字上的超时时间(单位:毫秒): - -* `connection_timeout_ms` - 默认值:1000 ms。 -* `receive_timeout_ms` - 默认值:1000 ms。 -* `send_timeout_ms` - 默认值:1000 ms。 - -重试参数: - -* `max_tries` - 发起认证请求的最大尝试次数。默认值:3 -* `retry_initial_backoff_ms` - 重试时的退避初始间隔。默认值:50 ms -* `retry_max_backoff_ms` - 最大退避间隔。默认值:1000 ms - -转发的请求头(headers): - -本部分定义从客户端请求头中转发到外部 HTTP 认证服务的请求头列表。注意,请求头在匹配配置中的名称时不区分大小写,但转发时会保持原样,即不作修改。 - -### 在 `users.xml` 中启用 HTTP 认证 {#enabling-http-auth-in-users-xml} - -要为用户启用 HTTP 认证,请在用户定义中指定 `http_authentication` 部分,而不是使用 `password` 或类似部分。 - -参数: - -* `server` - 在主 `config.xml` 文件中配置的 HTTP 认证服务器名称,如前文所述。 -* `scheme` - HTTP 认证方案。目前仅支持 `Basic`。默认值:Basic - -示例(放入 `users.xml` 中): - -```xml - - - - - - basic_server - basic - - - -``` - -:::note -请注意,HTTP 认证不能与任何其他认证机制同时使用。若在配置中同时存在 `http_authentication` 和 `password` 等其他字段,将会导致 ClickHouse 被强制退出。 -::: - -### 使用 SQL 启用 HTTP 认证 {#enabling-http-auth-using-sql} - -当在 ClickHouse 中启用 [基于 SQL 的访问控制和账户管理](/operations/access-rights#access-control-usage) 时,也可以使用 SQL 语句创建通过 HTTP 认证标识的用户。 - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' SCHEME 'Basic' -``` - -...或者,如果未显式指定认证方案,则默认使用 `Basic` - -```sql -CREATE USER my_user IDENTIFIED WITH HTTP SERVER 'basic_server' -``` - -### 传递会话设置 {#passing-session-settings} - -如果来自 HTTP 身份验证服务器的响应体为 JSON 格式,并且包含 `settings` 子对象,ClickHouse 会尝试将其中的键值对解析为字符串值,并将它们设置为已通过验证用户当前会话的会话设置。如果解析失败,则会忽略来自服务器的响应体。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md deleted file mode 100644 index 03e467fc721..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/index.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: 'ClickHouse 支持的外部身份验证方法总览' -pagination_next: operations/external-authenticators/kerberos -sidebar_label: '外部用户认证器和目录' -sidebar_position: 48 -slug: /operations/external-authenticators/ -title: '外部用户认证器和目录' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -ClickHouse 支持通过外部服务进行用户身份验证和管理。 - -支持以下外部身份验证器和用户目录: - -* [LDAP](/operations/external-authenticators/ldap#ldap-external-authenticator) [身份验证器](./ldap.md#ldap-external-authenticator) 和 [用户目录](./ldap.md#ldap-external-user-directory) -* Kerberos [身份验证器](/operations/external-authenticators/kerberos#kerberos-as-an-external-authenticator-for-existing-users) -* [SSL X.509 身份验证](/operations/external-authenticators/ssl-x509) -* HTTP [身份验证器](./http.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md deleted file mode 100644 index fa1eb862f12..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/kerberos.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: '已存在且已正确配置的 ClickHouse 用户可以通过 Kerberos 协议进行身份验证。' -slug: /operations/external-authenticators/kerberos -title: 'Kerberos' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# Kerberos {#kerberos} - - - -已存在且正确配置的 ClickHouse 用户可以通过 Kerberos 身份验证协议进行认证。 - -当前,Kerberos 只能作为外部身份验证器用于现有用户,这些用户定义在 `users.xml` 或本地访问控制路径中。这些用户只能使用 HTTP 请求,并且必须能够使用 GSS-SPNEGO 机制进行认证。 - -采用此方式时,必须在系统中完成 Kerberos 配置,并在 ClickHouse 配置中将其启用。 - -## 在 ClickHouse 中启用 Kerberos {#enabling-kerberos-in-clickhouse} - -要启用 Kerberos,应在 `config.xml` 中添加 `kerberos` 部分。该部分可以包含其他附加参数。 - -#### 参数 {#parameters} - -* `principal` - 在接受安全上下文时将被获取和使用的规范服务主体名称(service principal name)。 - * 此参数是可选的,如果省略,将使用默认 principal。 - -* `realm` - 要使用的 realm,用于将认证限制为仅允许发起方 realm 与其匹配的请求。 - * 此参数是可选的,如果省略,则不会应用任何额外的 realm 过滤。 - -* `keytab` - 服务 keytab 文件的路径。 - * 此参数是可选的,如果省略,则必须在 `KRB5_KTNAME` 环境变量中设置服务 keytab 文件的路径。 - -示例(添加到 `config.xml` 中): - -```xml - - - - -``` - -包含主体(principal)规范: - -```xml - - - - HTTP/clickhouse.example.com@EXAMPLE.COM - - -``` - -按 realm 筛选: - -```xml - - - - EXAMPLE.COM - - -``` - -:::note -只能定义一个 `kerberos` 节。存在多个 `kerberos` 节时,ClickHouse 会禁用 Kerberos 认证。 -::: - -:::note -`principal` 和 `realm` 节不能同时指定。如果同时存在 `principal` 和 `realm` 节,ClickHouse 会禁用 Kerberos 认证。 -::: - -## 将 Kerberos 用作现有用户的外部认证器 {#kerberos-as-an-external-authenticator-for-existing-users} - -Kerberos 可以作为一种方式,用于验证本地定义用户(在 `users.xml` 或本地访问控制路径中定义的用户)的身份。目前,**只**支持通过 HTTP 接口的请求进行 Kerberos 认证(通过 GSS-SPNEGO 机制)。 - -Kerberos 主体(principal)名称格式通常遵循以下模式: - -* *primary/instance@REALM* - -其中 */instance* 部分可以出现零次或多次。**发起方规范主体名称(canonical principal name)中的 *primary* 部分需要与启用 Kerberos 的用户名匹配,认证才能成功**。 - -### 在 `users.xml` 中启用 Kerberos {#enabling-kerberos-in-users-xml} - -要为用户启用 Kerberos 认证,请在用户定义中指定 `kerberos` 段,而不是使用 `password` 或类似配置段。 - -参数: - -* `realm` - 用于限制认证,仅允许发起方 realm 与该值匹配的请求通过认证。 - * 该参数是可选的,如果省略,则不会应用任何额外的 realm 过滤。 - -示例(写入 `users.xml` 中): - -```xml - - - - - - - - EXAMPLE.COM - - - - -``` - -:::note -请注意,Kerberos 认证不能与任何其他认证机制同时使用。如果在配置中 `kerberos` 段落旁边还存在 `password` 等其他认证段落,将会导致 ClickHouse 强制关闭。 -::: - -:::info Reminder -请注意,现在一旦用户 `my_user` 使用 `kerberos`,就必须按照前文所述在主配置文件 `config.xml` 中启用 Kerberos。 -::: - -### 使用 SQL 启用 Kerberos {#enabling-kerberos-using-sql} - -当在 ClickHouse 中启用 [基于 SQL 的访问控制和账号管理](/operations/access-rights#access-control-usage) 时,也可以通过 SQL 语句创建由 Kerberos 标识的用户。 - -```sql -CREATE USER my_user IDENTIFIED WITH kerberos REALM 'EXAMPLE.COM' -``` - -……或者,不按 realm 过滤: - -```sql -CREATE USER my_user IDENTIFIED WITH kerberos -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md deleted file mode 100644 index 2fd07ed0e4f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ldap.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: '配置 ClickHouse 的 LDAP 身份验证指南' -slug: /operations/external-authenticators/ldap -title: 'LDAP' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -LDAP 服务器可用于对 ClickHouse 用户进行身份验证。实现这一点有两种不同的方法: - -* 将 LDAP 用作现有用户的外部认证器,这些用户定义在 `users.xml` 或本地访问控制配置中。 -* 将 LDAP 用作外部用户目录,允许那些本地未定义但在 LDAP 服务器上存在的用户通过身份验证。 - -对于这两种方法,必须在 ClickHouse 配置中定义一个具有内部名称的 LDAP 服务器,以便配置的其他部分可以引用它。 - -## LDAP 服务器定义 {#ldap-server-definition} - -要定义 LDAP 服务器,必须在 `config.xml` 中添加 `ldap_servers` 配置节。 - -**示例** - -```xml - - - - - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - - - - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - - - -``` - -请注意,您可以在 `ldap_servers` 部分中定义多个 LDAP 服务器,只需使用不同的名称进行区分即可。 - -**参数** - -* `host` — LDAP 服务器的主机名或 IP,此参数为必需参数,且不能为空。 -* `port` — LDAP 服务器端口,如果 `enable_tls` 设置为 `true`,默认值为 `636`,否则为 `389`。 -* `bind_dn` — 用于构造绑定 DN 的模板。 - * 最终的 DN 将在每次认证尝试期间,通过将模板中所有 `{user_name}` 子串替换为实际用户名来构造。 -* `user_dn_detection` — 包含用于检测绑定用户实际用户 DN 的 LDAP 搜索参数的部分。 - * 这主要用于在服务器为 Active Directory 时,在搜索过滤器中进行后续角色映射。生成的用户 DN 将用于在允许的位置替换 `{user_dn}` 子串。默认情况下,用户 DN 与 bind DN 相同,但一旦执行搜索,它将被更新为实际检测到的用户 DN 值。 - * `base_dn` — 用于构造 LDAP 搜索基础 DN 的模板。 - * 最终的 DN 将在 LDAP 搜索期间,通过将模板中所有 `{user_name}` 和 `{bind_dn}` 子串替换为实际用户名和 bind DN 来构造。 - * `scope` — LDAP 搜索的作用域。 - * 可接受的值为:`base`、`one_level`、`children`、`subtree`(默认)。 - * `search_filter` — 用于构造 LDAP 搜索过滤器的模板。 - * 最终的过滤器将在 LDAP 搜索期间,通过将模板中所有 `{user_name}`、`{bind_dn}` 和 `{base_dn}` 子串替换为实际用户名、bind DN 和 base DN 来构造。 - * 注意,必须在 XML 中正确转义特殊字符。 -* `verification_cooldown` — 成功绑定尝试之后的一段时间(以秒为单位),在此期间内,所有连续请求在不联系 LDAP 服务器的情况下,都会被视为用户已成功认证。 - * 指定 `0`(默认)以禁用缓存,并强制对每个认证请求都联系 LDAP 服务器。 -* `enable_tls` — 用于启用与 LDAP 服务器安全连接的标志。 - * 指定 `no` 使用明文 `ldap://` 协议(不推荐)。 - * 指定 `yes` 使用基于 SSL/TLS 的 LDAP `ldaps://` 协议(推荐,默认)。 - * 指定 `starttls` 使用传统的 StartTLS 协议(明文 `ldap://` 协议,升级为 TLS)。 -* `tls_minimum_protocol_version` — SSL/TLS 的最小协议版本。 - * 可接受的值为:`ssl2`、`ssl3`、`tls1.0`、`tls1.1`、`tls1.2`(默认)。 -* `tls_require_cert` — SSL/TLS 对端证书验证行为。 - * 可接受的值为:`never`、`allow`、`try`、`demand`(默认)。 -* `tls_cert_file` — 证书文件路径。 -* `tls_key_file` — 证书密钥文件路径。 -* `tls_ca_cert_file` — CA 证书文件路径。 -* `tls_ca_cert_dir` — 包含 CA 证书的目录路径。 -* `tls_cipher_suite` — 允许的密码套件(使用 OpenSSL 表示法)。 - -## LDAP 外部认证器 {#ldap-external-authenticator} - -可以使用远程 LDAP 服务器作为验证本地定义用户(在 `users.xml` 或本地访问控制路径中定义的用户)密码的一种方式。为此,在用户定义中,将原本的 `password` 或类似字段替换为之前定义的 LDAP 服务器名称。 - -在每次登录尝试时,ClickHouse 会尝试使用提供的凭证,对 [LDAP 服务器定义](#ldap-server-definition) 中由 `bind_dn` 参数指定的 DN 执行“绑定”操作,如果成功,则认为该用户已通过认证。这通常被称为“简单绑定(simple bind)”方法。 - -**示例** - -```xml - - - - - - - - my_ldap_server - - - - -``` - -请注意,用户 `my_user` 关联到 `my_ldap_server`。必须在主配置文件 `config.xml` 中按前文所述配置此 LDAP 服务器。 - -当启用基于 SQL 的[访问控制和账户管理](/operations/access-rights#access-control-usage)时,通过 LDAP 服务器进行身份验证的用户也可以使用 [CREATE USER](/sql-reference/statements/create/user) 语句创建。 - -查询: - -```sql -CREATE USER my_user IDENTIFIED WITH ldap SERVER 'my_ldap_server'; -``` - -## LDAP 外部用户目录 {#ldap-external-user-directory} - -除了本地定义的用户之外,还可以使用远程 LDAP 服务器作为用户定义的来源。为此,需要在 `config.xml` 文件的 `users_directories` 部分中的 `ldap` 部分里,指定之前定义好的 LDAP 服务器名称(参见 [LDAP Server Definition](#ldap-server-definition))。 - -在每次登录尝试时,ClickHouse 会首先尝试在本地查找用户定义并按常规方式进行身份验证。如果未找到该用户,ClickHouse 将假定该用户已在外部 LDAP 目录中定义,并尝试使用提供的凭证在 LDAP 服务器上对指定的 DN 执行 bind 操作。如果成功,该用户将被视为存在并通过认证。用户会被分配在 `roles` 部分中指定列表里的角色。此外,还可以执行 LDAP search 操作,并将结果转换后视为角色名称;如果同时配置了 `role_mapping` 部分,则这些角色将被分配给该用户。所有这些都意味着必须启用由 SQL 驱动的[访问控制与账户管理](/operations/access-rights#access-control-usage),并且需要通过 [CREATE ROLE](/sql-reference/statements/create/role) 语句来创建角色。 - -**示例** - -写入 `config.xml`。 - -```xml - - - - - - my_ldap_server - - - - - - ou=groups,dc=example,dc=com - subtree - (&(objectClass=groupOfNames)(member={bind_dn})) - cn - clickhouse_ - - - - - - my_ad_server - - CN=Users,DC=example,DC=com - CN - subtree - (&(objectClass=group)(member={user_dn})) - clickhouse_ - - - - -``` - -请注意,`user_directories` 部分中 `ldap` 小节引用的 `my_ldap_server`,必须是在 `config.xml` 中事先定义并完成配置的 LDAP 服务器(参见 [LDAP Server Definition](#ldap-server-definition))。 - -**参数** - -* `server` — 在上面 `ldap_servers` 配置部分中定义的 LDAP 服务器名称之一。此参数为必填项,且不能为空。 -* `roles` — 包含本地定义角色列表的部分,这些角色将被分配给每个从 LDAP 服务器检索到的用户。 - * 如果此处未指定任何角色,或者在下面的角色映射过程中未分配角色,用户在完成认证后将无法执行任何操作。 -* `role_mapping` — 包含 LDAP 搜索参数和映射规则的部分。 - * 当用户进行认证时,在仍然绑定到 LDAP 的情况下,将使用 `search_filter` 和已登录用户的名称执行 LDAP 搜索。对在该搜索中找到的每个条目,将提取指定属性的值。对于每个具有指定前缀的属性值,将移除该前缀,其余部分的值将作为在 ClickHouse 中本地定义的角色名称,且这些角色应事先通过 [CREATE ROLE](/sql-reference/statements/create/role) 语句创建。 - * 在同一个 `ldap` 部分中可以定义多个 `role_mapping` 小节,且会全部生效。 - * `base_dn` — 用于构造 LDAP 搜索基础 DN 的模板。 - * 最终 DN 将通过在每次 LDAP 搜索期间,用实际用户名、绑定 DN 和用户 DN 分别替换模板中的 `{user_name}`、`{bind_dn}` 和 `{user_dn}` 子串来构造。 - * `scope` — LDAP 搜索的范围。 - * 可接受的取值为:`base`、`one_level`、`children`、`subtree`(默认)。 - * `search_filter` — 用于构造 LDAP 搜索过滤器的模板。 - * 最终过滤器将通过在每次 LDAP 搜索期间,用实际用户名、绑定 DN、用户 DN 和基础 DN 分别替换模板中的 `{user_name}`、`{bind_dn}`、`{user_dn}` 和 `{base_dn}` 子串来构造。 - * 注意,必须在 XML 中对特殊字符进行正确转义。 - * `attribute` — 其值将由 LDAP 搜索返回的属性名。默认为 `cn`。 - * `prefix` — 预期出现在 LDAP 搜索返回的原始字符串列表中每个字符串前面的前缀。该前缀会从原始字符串中移除,得到的字符串将被视为本地角色名称。默认为空。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md deleted file mode 100644 index 06bad7b8514..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/external-authenticators/ssl-x509.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'SSL X.509 文档' -slug: /operations/external-authenticators/ssl-x509 -title: 'SSL X.509 证书身份验证' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -[SSL 'strict' option](../server-configuration-parameters/settings.md#openssl) 会对传入连接启用强制证书验证。在这种情况下,只能建立使用受信任证书的连接,使用不受信任证书的连接将被拒绝。因此,通过证书验证可以对传入连接进行唯一标识和认证。证书中的 `Common Name` 或 `subjectAltName extension` 字段用于标识已连接的用户。`subjectAltName extension` 在服务器配置中支持使用一个通配符 '*',这允许将多个证书关联到同一用户。此外,重新签发和吊销证书不会影响 ClickHouse 的配置。 - -要启用 SSL 证书认证,必须在配置文件 `users.xml` 中为每个 ClickHouse 用户指定 `Common Name` 或 `Subject Alt Name` 的列表: - -**示例** - -```xml - - - - - - host.domain.com:example_user - host.domain.com:example_user_dev - - - - - - - DNS:host.domain.com - - - - - - - - URI:spiffe://foo.com/*/bar - - - - -``` - -为了确保 SSL 的 [`chain of trust`](https://en.wikipedia.org/wiki/Chain_of_trust) 能正常工作,还必须确保正确配置 [`caConfig`](../server-configuration-parameters/settings.md#openssl) 参数。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/monitoring.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/monitoring.md deleted file mode 100644 index de14451969f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/monitoring.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: '您可以监控硬件资源利用率以及 ClickHouse 服务器指标。' -keywords: ['monitoring', 'observability', 'advanced dashboard', 'dashboard', 'observability - dashboard'] -sidebar_label: '监控' -sidebar_position: 45 -slug: /operations/monitoring -title: '监控' -doc_type: 'reference' ---- - -import Image from '@theme/IdealImage'; - - -# 监控 {#monitoring} - -:::note -本指南中介绍的监控数据可在 ClickHouse Cloud 中获取。除了可以通过下文所述的内置仪表板查看外,基础和高级性能指标也都可以直接在主服务控制台中查看。 -::: - -您可以监控: - -- 硬件资源使用率。 -- ClickHouse 服务器指标。 - - - -## 内置高级可观测性仪表板 {#built-in-advanced-observability-dashboard} - -屏幕截图 2023-11-12 6 08 58 PM - -ClickHouse 提供内置的高级可观测性仪表板功能,可通过 `$HOST:$PORT/dashboard` 访问(需要用户名和密码),该仪表板展示以下指标: -- 每秒查询数 -- CPU 使用情况(核心数) -- 正在运行的查询数 -- 正在进行的合并数 -- 每秒选取字节数 -- I/O 等待 -- CPU 等待 -- 操作系统 CPU 使用率(用户态) -- 操作系统 CPU 使用率(内核态) -- 磁盘读取量 -- 文件系统读取量 -- 内存(已跟踪) -- 每秒插入行数 -- MergeTree 数据片段总数 -- 每个分区的最大数据片段数 - - - -## 资源使用情况 {#resource-utilization} - -ClickHouse 还会自行监控硬件资源的状态,例如: - -- 处理器的负载和温度。 -- 存储系统、内存(RAM)和网络的使用率。 - -这些数据会被收集到 `system.asynchronous_metric_log` 表中。 - - - -## ClickHouse 服务器指标 {#clickhouse-server-metrics} - -ClickHouse 服务器内置了用于自我状态监控的工具。 - -要跟踪服务器事件,请使用服务器日志。请参阅配置文件中的 [logger](../operations/server-configuration-parameters/settings.md#logger) 部分。 - -ClickHouse 会收集: - -- 服务器对计算资源使用情况的各类指标。 -- 查询处理的常规统计信息。 - -可以在 [system.metrics](/operations/system-tables/metrics)、[system.events](/operations/system-tables/events) 和 [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) 表中找到这些指标。 - -可以将 ClickHouse 配置为将指标导出到 [Graphite](https://github.com/graphite-project)。请参阅 ClickHouse 服务器配置文件中的 [Graphite 部分](../operations/server-configuration-parameters/settings.md#graphite)。在配置指标导出之前,应先按照其官方[指南](https://graphite.readthedocs.io/en/latest/install.html)完成 Graphite 的部署。 - -可以将 ClickHouse 配置为将指标导出到 [Prometheus](https://prometheus.io)。请参阅 ClickHouse 服务器配置文件中的 [Prometheus 部分](../operations/server-configuration-parameters/settings.md#prometheus)。在配置指标导出之前,应先按照其官方[指南](https://prometheus.io/docs/prometheus/latest/installation/)完成 Prometheus 的部署。 - -此外,可以通过 HTTP API 监控服务器可用性。向 `/ping` 发送 `HTTP GET` 请求。如果服务器可用,它会返回 `200 OK`。 - -要监控集群配置中的服务器,需要设置 [max_replica_delay_for_distributed_queries](../operations/settings/settings.md#max_replica_delay_for_distributed_queries) 参数并使用 HTTP 资源 `/replicas_status`。对 `/replicas_status` 的请求在副本可用且未落后于其他副本时返回 `200 OK`。如果某个副本存在延迟,它会返回 `503 HTTP_SERVICE_UNAVAILABLE`,并包含有关延迟情况的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/named-collections.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/named-collections.md deleted file mode 100644 index 9c2ec526e99..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/named-collections.md +++ /dev/null @@ -1,647 +0,0 @@ ---- -description: '命名集合文档' -sidebar_label: '命名集合' -sidebar_position: 69 -slug: /operations/named-collections -title: '命名集合' -doc_type: 'reference' ---- - -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - - - -命名集合提供了一种机制,用于存储键值对集合,以配置与外部数据源的集成。可以在字典、表、表函数以及对象存储中使用命名集合。 - -命名集合可以通过 DDL 或配置文件进行配置,并在 ClickHouse 启动时生效。它们简化了对象的创建,并将凭证对无管理权限的用户进行隐藏。 - -命名集合中的键必须与相应函数、表引擎、数据库等的参数名相匹配。下面的示例中,对每种类型都给出了参数列表的链接。 - -在命名集合中设置的参数可以在 SQL 中被覆盖,下面的示例展示了这一点。可以通过使用 `[NOT] OVERRIDABLE` 关键字和 XML 属性和/或配置项 `allow_named_collection_override_by_default` 来限制这种行为。 - -:::warning -如果允许覆盖,无管理权限的用户可能能够推断出试图隐藏的凭证。 -如果是为了这一目的使用命名集合,应当禁用 -`allow_named_collection_override_by_default`(该选项默认启用)。 -::: - -## 在 system 数据库中存储命名集合 {#storing-named-collections-in-the-system-database} - -### DDL 示例 {#ddl-example} - -```sql -CREATE NAMED COLLECTION name AS -key_1 = 'value' OVERRIDABLE, -key_2 = 'value2' NOT OVERRIDABLE, -url = 'https://connection.url/' -``` - -在上面的示例中: - -* `key_1` 始终可以被覆盖。 -* `key_2` 永远不能被覆盖。 -* `url` 是否可以被覆盖取决于 `allow_named_collection_override_by_default` 的取值。 - -### 使用 DDL 创建命名集合的权限 {#permissions-to-create-named-collections-with-ddl} - -要使用 DDL 管理命名集合,用户必须拥有 `named_collection_control` 权限。可以通过在 `/etc/clickhouse-server/users.d/` 中添加一个文件来授予该权限。下面的示例为用户 `default` 同时授予了 `access_management` 和 `named_collection_control` 权限: - -```xml title='/etc/clickhouse-server/users.d/user_default.xml' - - - - 65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5 - 1 - - 1 - - - - -``` - -:::tip -在上面的示例中,`password_sha256_hex` 的值是该密码的 SHA256 哈希的十六进制表示。针对用户 `default` 的这段配置包含属性 `replace=true`,因为在默认配置中为该用户设置的是明文 `password`,而同一个用户不能同时设置明文密码和 SHA256 十六进制密码。 -::: - -### 命名集合的存储 {#storage-for-named-collections} - -命名集合可以存储在本地磁盘或 ZooKeeper/Keeper 中,默认使用本地存储。 -它们也可以使用与 [磁盘加密](storing-data#encrypted-virtual-file-system) 相同的算法进行加密存储, -其中默认使用 `aes_128_ctr`。 - -要配置命名集合存储,需要指定一个 `type`。它可以是 `local` 或 `keeper`/`zookeeper`。对于加密存储, -可以使用 `local_encrypted` 或 `keeper_encrypted`/`zookeeper_encrypted`。 - -要使用 ZooKeeper/Keeper,我们还需要在配置文件的 `named_collections_storage` 部分设置一个 `path`(在 ZooKeeper/Keeper 中存储命名集合的路径)。 -下面的示例使用了加密和 ZooKeeper/Keeper: - -```xml - - - zookeeper_encrypted - bebec0cabebec0cabebec0cabebec0ca - aes_128_ctr - /named_collections_path/ - 1000 - - -``` - -可选配置参数 `update_timeout_ms` 的默认值为 `5000` 毫秒。 - -## 在配置文件中存储命名集合 {#storing-named-collections-in-configuration-files} - -### XML 示例 {#xml-example} - -```xml title='/etc/clickhouse-server/config.d/named_collections.xml' - - - - value - value_2 - https://connection.url/ - - - -``` - -在上述示例中: - -* `key_1` 始终可以被覆盖。 -* `key_2` 不可被覆盖。 -* `url` 是否可以被覆盖取决于 `allow_named_collection_override_by_default` 的值。 - -## 修改命名集合 {#modifying-named-collections} - -使用 DDL 查询创建的命名集合可以通过 DDL 进行修改或删除。使用 XML 文件创建的命名集合可以通过编辑或删除相应的 XML 文件进行管理。 - -### 修改 DDL 创建的命名集合 {#alter-a-ddl-named-collection} - -更改或添加集合 `collection2` 的键 `key1` 和 `key3` -(这不会更改这些键的 `overridable` 标志位的值): - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, key3='value3' -``` - -更改或添加键 `key1`,并允许其始终可被覆盖: - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4 OVERRIDABLE -``` - -从 `collection2` 中删除键 `key2`: - -```sql -ALTER NAMED COLLECTION collection2 DELETE key2 -``` - -修改或添加集合 `collection2` 中的键 `key1`,并删除键 `key3`: - -```sql -ALTER NAMED COLLECTION collection2 SET key1=4, DELETE key3 -``` - -若要强制某个键使用 `overridable` 标志的默认设置,必须先删除该键,然后再重新添加。 - -```sql -ALTER NAMED COLLECTION collection2 DELETE key1; -ALTER NAMED COLLECTION collection2 SET key1=4; -``` - -### 删除 DDL 命名集合 `collection2`: {#drop-the-ddl-named-collection-collection2} - -```sql -DROP NAMED COLLECTION collection2 -``` - -## 用于访问 S3 的命名集合 {#named-collections-for-accessing-s3} - -有关参数说明,请参阅 [S3 表函数](../sql-reference/table-functions/s3.md)。 - -### DDL 示例 {#ddl-example-1} - -```sql -CREATE NAMED COLLECTION s3_mydata AS -access_key_id = 'AKIAIOSFODNN7EXAMPLE', -secret_access_key = 'wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY', -format = 'CSV', -url = 'https://s3.us-east-1.amazonaws.com/yourbucket/mydata/' -``` - -### XML 示例 {#xml-example-1} - -```xml - - - - AKIAIOSFODNN7EXAMPLE - wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY - CSV - https://s3.us-east-1.amazonaws.com/yourbucket/mydata/ - - - -``` - -### s3() 函数和 S3 表命名集合示例 {#s3-function-and-s3-table-named-collection-examples} - -以下两个示例都使用同一个命名集合 `s3_mydata`: - -#### s3() 函数 {#s3-function} - -```sql -INSERT INTO FUNCTION s3(s3_mydata, filename = 'test_file.tsv.gz', - format = 'TSV', structure = 'number UInt64', compression_method = 'gzip') -SELECT * FROM numbers(10000); -``` - -:::tip -上述 `s3()` 函数的第一个参数是集合名称 `s3_mydata`。如果不使用具名集合,那么在每次调用 `s3()` 函数时,都必须传入访问密钥 ID、秘密访问密钥、格式和 URL。 -::: - -#### S3 表 {#s3-table} - -```sql -CREATE TABLE s3_engine_table (number Int64) -ENGINE=S3(s3_mydata, url='https://s3.us-east-1.amazonaws.com/yourbucket/mydata/test_file.tsv.gz', format = 'TSV') -SETTINGS input_format_with_names_use_header = 0; - -SELECT * FROM s3_engine_table LIMIT 3; -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -└────────┘ -``` - -## 用于访问 MySQL 数据库的命名集合 {#named-collections-for-accessing-mysql-database} - -有关参数的说明,请参见 [mysql](../sql-reference/table-functions/mysql.md)。 - -### DDL 示例 {#ddl-example-2} - -```sql -CREATE NAMED COLLECTION mymysql AS -user = 'myuser', -password = 'mypass', -host = '127.0.0.1', -port = 3306, -database = 'test', -connection_pool_size = 8, -replace_query = 1 -``` - -### XML 示例 {#xml-example-2} - -```xml - - - - myuser - mypass - 127.0.0.1 - 3306 - test - 8 - 1 - - - -``` - -### mysql() 函数、MySQL 表、MySQL 数据库和 Dictionary 命名集合示例 {#mysql-function-mysql-table-mysql-database-and-dictionary-named-collection-examples} - -下面四个示例都使用同一个名为 `mymysql` 的命名集合: - -#### mysql() 函数 {#mysql-function} - -```sql -SELECT count() FROM mysql(mymysql, table = 'test'); - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -该命名集合未指定 `table` 参数,因此在函数调用时通过 `table = 'test'` 来指定该参数。 -::: - -#### MySQL 表 {#mysql-table} - -```sql -CREATE TABLE mytable(A Int64) ENGINE = MySQL(mymysql, table = 'test', connection_pool_size=3, replace_query=0); -SELECT count() FROM mytable; - -┌─count()─┐ -│ 3 │ -└─────────┘ -``` - -:::note -该 DDL 会覆盖命名集合中对 connection_pool_size 的配置。 -::: - -#### MySQL 数据库 {#mysql-database} - -```sql -CREATE DATABASE mydatabase ENGINE = MySQL(mymysql); - -SHOW TABLES FROM mydatabase; - -┌─name───┐ -│ source │ -│ test │ -└────────┘ -``` - -#### MySQL 字典 {#mysql-dictionary} - -```sql -CREATE DICTIONARY dict (A Int64, B String) -PRIMARY KEY A -SOURCE(MYSQL(NAME mymysql TABLE 'source')) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'B', 2); - -┌─dictGet('dict', 'B', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## 用于访问 PostgreSQL 数据库的命名集合 {#named-collections-for-accessing-postgresql-database} - -参数说明请参见 [postgresql](../sql-reference/table-functions/postgresql.md)。此外,还有以下别名: - -* `username` 对应 `user` -* `db` 对应 `database`。 - -在命名集合中,使用参数 `addresses_expr` 来替代 `host:port`。该参数是可选的,因为还存在其他可选参数:`host`、`hostname`、`port`。下面的伪代码说明了优先级: - -```sql -CASE - WHEN collection['addresses_expr'] != '' THEN collection['addresses_expr'] - WHEN collection['host'] != '' THEN collection['host'] || ':' || if(collection['port'] != '', collection['port'], '5432') - WHEN collection['hostname'] != '' THEN collection['hostname'] || ':' || if(collection['port'] != '', collection['port'], '5432') -END -``` - -创建示例: - -```sql -CREATE NAMED COLLECTION mypg AS -user = 'pguser', -password = 'jw8s0F4', -host = '127.0.0.1', -port = 5432, -database = 'test', -schema = 'test_schema' -``` - -示例配置: - -```xml - - - - pguser - jw8s0F4 - 127.0.0.1 - 5432 - test - test_schema - - - -``` - -### 在 PostgreSQL 函数中使用命名集合的示例 {#example-of-using-named-collections-with-the-postgresql-function} - -```sql -SELECT * FROM postgresql(mypg, table = 'test'); - -┌─a─┬─b───┐ -│ 2 │ two │ -│ 1 │ one │ -└───┴─────┘ -SELECT * FROM postgresql(mypg, table = 'test', schema = 'public'); - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -### 在 PostgreSQL 引擎数据库中使用命名集合的示例 {#example-of-using-named-collections-with-database-with-engine-postgresql} - -```sql -CREATE TABLE mypgtable (a Int64) ENGINE = PostgreSQL(mypg, table = 'test', schema = 'public'); - -SELECT * FROM mypgtable; - -┌─a─┐ -│ 1 │ -│ 2 │ -│ 3 │ -└───┘ -``` - -:::note -在创建表时,PostgreSQL 会从命名集合中复制数据。之后对该集合的更改不会影响现有的表。 -::: - -### 在使用 PostgreSQL 引擎的数据库中使用命名集合的示例 {#example-of-using-named-collections-with-database-with-engine-postgresql-1} - -```sql -CREATE DATABASE mydatabase ENGINE = PostgreSQL(mypg); - -SHOW TABLES FROM mydatabase - -┌─name─┐ -│ test │ -└──────┘ -``` - -### 在以 POSTGRESQL 为源的字典中使用具名集合的示例 {#example-of-using-named-collections-with-a-dictionary-with-source-postgresql} - -```sql -CREATE DICTIONARY dict (a Int64, b String) -PRIMARY KEY a -SOURCE(POSTGRESQL(NAME mypg TABLE test)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -## 用于访问远程 ClickHouse 数据库的命名集合 {#named-collections-for-accessing-a-remote-clickhouse-database} - -有关参数的说明,参见 [remote](../sql-reference/table-functions/remote.md/#parameters)。 - -配置示例: - -```sql -CREATE NAMED COLLECTION remote1 AS -host = 'remote_host', -port = 9000, -database = 'system', -user = 'foo', -password = 'secret', -secure = 1 -``` - -```xml - - - - remote_host - 9000 - system - foo - secret - 1 - - - -``` - -由于已使用 `remoteSecure`,建立连接时不需要设置 `secure`,但它仍可用于字典。 - -### 使用命名集合与 `remote`/`remoteSecure` 函数的示例 {#example-of-using-named-collections-with-the-remoteremotesecure-functions} - -```sql -SELECT * FROM remote(remote1, table = one); -┌─dummy─┐ -│ 0 │ -└───────┘ - -SELECT * FROM remote(remote1, database = merge(system, '^one')); -┌─dummy─┐ -│ 0 │ -└───────┘ - -INSERT INTO FUNCTION remote(remote1, database = default, table = test) VALUES (1,'a'); - -SELECT * FROM remote(remote1, database = default, table = test); -┌─a─┬─b─┐ -│ 1 │ a │ -└───┴───┘ -``` - -### 在以 ClickHouse 为源的字典中使用命名集合的示例 {#example-of-using-named-collections-with-a-dictionary-with-source-clickhouse} - -```sql -CREATE DICTIONARY dict(a Int64, b String) -PRIMARY KEY a -SOURCE(CLICKHOUSE(NAME remote1 TABLE test DB default)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()); - -SELECT dictGet('dict', 'b', 1); -┌─dictGet('dict', 'b', 1)─┐ -│ a │ -└─────────────────────────┘ -``` - -## 用于访问 Kafka 的命名集合 {#named-collections-for-accessing-kafka} - -参数说明参见 [Kafka](../engines/table-engines/integrations/kafka.md)。 - -### DDL 示例 {#ddl-example-3} - -```sql -CREATE NAMED COLLECTION my_kafka_cluster AS -kafka_broker_list = 'localhost:9092', -kafka_topic_list = 'kafka_topic', -kafka_group_name = 'consumer_group', -kafka_format = 'JSONEachRow', -kafka_max_block_size = '1048576'; - -``` - -### XML 示例 {#xml-example-3} - -```xml - - - - localhost:9092 - kafka_topic - consumer_group - JSONEachRow - 1048576 - - - -``` - -### 在 Kafka 表中使用命名集合的示例 {#example-of-using-named-collections-with-a-kafka-table} - -以下两个示例都使用同一个命名集合 `my_kafka_cluster`: - -```sql -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) - -CREATE TABLE queue -( - timestamp UInt64, - level String, - message String -) -ENGINE = Kafka(my_kafka_cluster) -SETTINGS kafka_num_consumers = 4, - kafka_thread_per_consumer = 1; -``` - -## 用于备份的命名集合 {#named-collections-for-backups} - -有关参数说明,请参阅 [备份和恢复](./backup.md)。 - -### DDL 示例 {#ddl-example-4} - -```sql -BACKUP TABLE default.test to S3(named_collection_s3_backups, 'directory') -``` - -### XML 示例 {#xml-example-4} - -```xml - - - - https://my-s3-bucket.s3.amazonaws.com/backup-S3/ - ABC123 - Abc+123 - - - -``` - -## 用于访问 MongoDB 表和字典的命名集合 {#named-collections-for-accessing-mongodb-table-and-dictionary} - -有关参数的说明,请参阅 [mongodb](../sql-reference/table-functions/mongodb.md)。 - -### DDL 示例 {#ddl-example-5} - -```sql -CREATE NAMED COLLECTION mymongo AS -user = '', -password = '', -host = '127.0.0.1', -port = 27017, -database = 'test', -collection = 'my_collection', -options = 'connectTimeoutMS=10000' -``` - -### XML 示例 {#xml-example-5} - -```xml - - - - - - 127.0.0.1 - 27017 - test - my_collection - connectTimeoutMS=10000 - - - -``` - -#### MongoDB 集合 {#mongodb-table} - -```sql -CREATE TABLE mytable(log_type VARCHAR, host VARCHAR, command VARCHAR) ENGINE = MongoDB(mymongo, options='connectTimeoutMS=10000&compressors=zstd') -SELECT count() FROM mytable; - -┌─count()─┐ -│ 2 │ -└─────────┘ -``` - -:::note -DDL 会覆盖选项中指定集合的设置。 -::: - -#### MongoDB 字典 {#mongodb-dictionary} - -```sql -CREATE DICTIONARY dict -( - `a` Int64, - `b` String -) -PRIMARY KEY a -SOURCE(MONGODB(NAME mymongo COLLECTION my_dict)) -LIFETIME(MIN 1 MAX 2) -LAYOUT(HASHED()) - -SELECT dictGet('dict', 'b', 2); - -┌─dictGet('dict', 'b', 2)─┐ -│ two │ -└─────────────────────────┘ -``` - -:::note -命名的集合将集合名称指定为 `my_collection`。在函数调用中,通过 `collection = 'my_dict'` 覆盖该名称,以选择另一个集合。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/opentelemetry.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/opentelemetry.md deleted file mode 100644 index cc69779fd39..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/opentelemetry.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '在 ClickHouse 中使用 OpenTelemetry 进行分布式链路追踪与指标采集的指南' -sidebar_label: '使用 OpenTelemetry 追踪 ClickHouse' -sidebar_position: 62 -slug: /operations/opentelemetry -title: '使用 OpenTelemetry 追踪 ClickHouse' -doc_type: 'guide' ---- - -[OpenTelemetry](https://opentelemetry.io/) 是一个用于从分布式应用程序中收集链路追踪和指标的开放标准。ClickHouse 内置了对 OpenTelemetry 的部分支持。 - - - -## 向 ClickHouse 提供 trace 上下文 {#supplying-trace-context-to-clickhouse} - -ClickHouse 接收符合 [W3C 规范](https://www.w3.org/TR/trace-context/)的 trace 上下文 HTTP 头部。它还可以通过一种原生协议接收 trace 上下文,该协议用于 ClickHouse 服务器之间或客户端与服务器之间的通信。对于手动测试,可以使用 `--opentelemetry-traceparent` 和 `--opentelemetry-tracestate` 参数,将符合 Trace Context 规范的 trace 上下文头部传递给 `clickhouse-client`。 - -如果未提供父级 trace 上下文,或者提供的 trace 上下文不符合上述 W3C 标准,ClickHouse 可以启动一个新的 trace,其触发概率由 [opentelemetry_start_trace_probability](/operations/settings/settings#opentelemetry_start_trace_probability) 设置控制。 - - - -## 传播 trace 上下文 {#propagating-the-trace-context} - -在以下情况下,trace 上下文会被传播到下游服务: - -* 对远程 ClickHouse 服务器的查询,例如使用 [Distributed](../engines/table-engines/special/distributed.md) 表引擎时。 - -* [url](../sql-reference/table-functions/url.md) 表函数。trace 上下文信息会通过 HTTP 请求头发送。 - - - -## 对 ClickHouse 本身进行追踪 {#tracing-the-clickhouse-itself} - -ClickHouse 会为每个查询以及部分查询执行阶段(例如查询规划或分布式查询)创建 `trace spans`。 - -为了发挥作用,追踪信息必须导出到支持 OpenTelemetry 的监控系统中,例如 [Jaeger](https://jaegertracing.io/) 或 [Prometheus](https://prometheus.io/)。ClickHouse 避免依赖特定监控系统,而是仅通过系统表提供追踪数据。符合 OpenTelemetry 标准要求的 trace span 信息(参见[标准定义](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/overview.md#span))存储在 [system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md) 表中。 - -必须在服务器配置中启用该表,参见默认配置文件 `config.xml` 中的 `opentelemetry_span_log` 元素。该功能默认已启用。 - -标签或属性以两个并行数组的形式保存,分别包含键和值。使用 [ARRAY JOIN](../sql-reference/statements/select/array-join.md) 来处理它们。 - - - -## Log-query-settings {#log-query-settings} - -[log_query_settings](settings/settings.md) 设置允许在查询执行期间将对查询设置所做的更改写入日志。启用后,对查询设置的任何修改都会记录到 OpenTelemetry span 日志中。此功能在生产环境中尤为有用,可用于跟踪可能影响查询性能的配置更改。 - - - -## 与监控系统的集成 {#integration-with-monitoring-systems} - -目前,还没有可将 ClickHouse 的跟踪数据导出到监控系统的现成工具。 - -在测试时,可以通过在 [system.opentelemetry_span_log](../operations/system-tables/opentelemetry_span_log.md) 表之上创建一个使用 [URL](../engines/table-engines/special/url.md) 引擎的物化视图来配置导出,该视图会将接收到的日志数据推送到某个 trace 收集器的 HTTP 端点。例如,要将最精简的 span 数据以 Zipkin v2 JSON 格式推送到运行在 `http://localhost:9411` 的 Zipkin 实例,可以这样做: - -```sql -CREATE MATERIALIZED VIEW default.zipkin_spans -ENGINE = URL('http://127.0.0.1:9411/api/v2/spans', 'JSONEachRow') -SETTINGS output_format_json_named_tuples_as_objects = 1, - output_format_json_array_of_rows = 1 AS -SELECT - lower(hex(trace_id)) AS traceId, - CASE WHEN parent_span_id = 0 THEN '' ELSE lower(hex(parent_span_id)) END AS parentId, - lower(hex(span_id)) AS id, - operation_name AS name, - start_time_us AS timestamp, - finish_time_us - start_time_us AS duration, - cast(tuple('clickhouse'), 'Tuple(serviceName text)') AS localEndpoint, - cast(tuple( - attribute.values[indexOf(attribute.names, 'db.statement')]), - 'Tuple("db.statement" text)') AS tags -FROM system.opentelemetry_span_log -``` - -如果发生任何错误,发生错误的那部分日志数据将被静默丢弃。若数据未到达,请检查服务器日志中的错误消息。 - - -## 相关内容 {#related-content} - -- 博客:[使用 ClickHouse 构建可观测性解决方案 - 第 2 部分:跟踪(Traces)](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md deleted file mode 100644 index 3e817a715dc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/profile-guided-optimization.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: '性能分析引导优化(PGO)文档' -sidebar_label: '性能分析引导优化(PGO)' -sidebar_position: 54 -slug: /operations/optimizing-performance/profile-guided-optimization -title: '性能分析引导优化' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# 基于性能分析的优化 {#profile-guided-optimization} - -Profile-Guided Optimization(PGO,基于性能分析的优化)是一种编译器优化技术,它根据程序的运行时性能分析数据对程序进行优化。 - -根据测试结果,PGO 有助于提升 ClickHouse 的性能。在 ClickBench 测试套件中,我们观测到 QPS 最高可提升约 15%。更详细的结果可在[这里](https://pastebin.com/xbue3HMU)查看。性能收益取决于典型工作负载,实际结果可能更好或更差。 - -关于 ClickHouse 中 PGO 的更多信息,可在对应的 GitHub [issue](https://github.com/ClickHouse/ClickHouse/issues/44567) 中查阅。 - -## 如何使用 PGO 构建 ClickHouse? {#how-to-build-clickhouse-with-pgo} - -PGO 主要有两种类型:[Instrumentation](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers) 和 [Sampling](https://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers)(也称为 AutoFDO)。本指南介绍的是在 ClickHouse 中使用 Instrumentation PGO 的方法。 - -1. 以 Instrumentation 模式构建 ClickHouse。在 Clang 中,可以通过在 `CXXFLAGS` 中加入 `-fprofile-generate` 选项来实现。 -2. 在样本工作负载上运行插桩版 ClickHouse。这里应使用您常规的工作负载,其中一种做法是使用 [ClickBench](https://github.com/ClickHouse/ClickBench) 作为样本工作负载。处于 Instrumentation 模式下的 ClickHouse 可能运行较慢,请提前做好预期,并避免在对性能敏感的环境中运行插桩版 ClickHouse。 -3. 使用 `-fprofile-use` 编译器选项以及在上一步中收集到的 profile,再次编译 ClickHouse。 - -关于如何应用 PGO 的更详细说明,请参阅 Clang 的[文档](https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization)。 - -如果您计划直接在生产环境中收集样本工作负载,我们建议尝试使用 Sampling PGO。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md deleted file mode 100644 index 468ec84dd40..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/optimizing-performance/sampling-query-profiler.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse 采样查询分析器工具文档' -sidebar_label: '查询性能分析' -sidebar_position: 54 -slug: /operations/optimizing-performance/sampling-query-profiler -title: '采样查询分析器' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - -# 采样查询分析器 {#sampling-query-profiler} - -ClickHouse 运行一个采样分析器,用于分析查询的执行情况。通过该分析器,您可以找到在查询执行期间被最频繁调用的源代码函数/例程。您可以跟踪 CPU 时间以及包括空闲时间在内的实际耗时(wall-clock time)。 - -在 ClickHouse Cloud 中,查询分析器会自动启用,您可以按如下方式运行一个示例查询: - -:::note 如果您在 ClickHouse Cloud 中运行以下查询,请确保将 `FROM system.trace_log` 更改为 `FROM clusterAllReplicas(default, system.trace_log)`,以便从集群中所有节点读取数据 -::: - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c' AND trace_type = 'CPU' AND event_date = today() -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -SETTINGS allow_introspection_functions = 1 -``` - -在自托管部署中,要使用查询分析器(query profiler): - -* 配置服务器配置中的 [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log) 部分。 - - 该部分用于配置 [trace_log](/operations/system-tables/trace_log) 系统表,其中包含分析器运行的结果。默认情况下已启用该配置。请注意,此表中的数据仅对正在运行的服务器有效。服务器重启后,ClickHouse 不会清理该表,其中存储的所有虚拟内存地址都可能失效。 - -* 配置 [query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns) 或 [query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns) 设置。这两个设置可以同时使用。 - - 这些设置用于配置分析器的计时器。由于它们是会话级设置,因此可以为整个服务器、单个用户或用户配置文件、交互式会话以及每条单独查询设置不同的采样频率。 - -默认采样频率为每秒采集一个样本,且 CPU 和 real 两种计时器均处于启用状态。该频率可以收集到足够的 ClickHouse 集群信息。同时,在该频率下工作时,分析器不会影响 ClickHouse 服务器的性能。如果需要对每条单独的查询进行分析,尝试使用更高的采样频率。 - -要分析 `trace_log` 系统表: - -* 安装 `clickhouse-common-static-dbg` 软件包。参见 [从 DEB 软件包安装](../../getting-started/install/install.mdx)。 - -* 通过 [allow_introspection_functions](../../operations/settings/settings.md#allow_introspection_functions) 设置允许使用内省函数。 - - 出于安全原因,内省函数默认被禁用。 - -* 使用 `addressToLine`、`addressToLineWithInlines`、`addressToSymbol` 和 `demangle` 等[内省函数](../../sql-reference/functions/introspection.md),以获取 ClickHouse 代码中的函数名及其位置。要获取某条查询的分析信息,需要对 `trace_log` 表中的数据进行聚合。可以按单个函数或整条堆栈跟踪进行聚合。 - -如果需要可视化 `trace_log` 信息,可尝试使用 [flamegraph](/interfaces/third-party/gui#clickhouse-flamegraph) 和 [speedscope](https://github.com/laplab/clickhouse-speedscope)。 - -## 示例 {#example} - -在本示例中,我们将: - -* 使用查询标识符和当前日期过滤 `trace_log` 数据。 - -* 按堆栈跟踪进行聚合。 - -* 使用自省函数生成一份报告,其中包括: - - * 符号名称及其对应的源代码函数。 - * 这些函数在源代码中的位置。 - -{/* */ } - -```sql -SELECT - count(), - arrayStringConcat(arrayMap(x -> concat(demangle(addressToSymbol(x)), '\n ', addressToLine(x)), trace), '\n') AS sym -FROM system.trace_log -WHERE (query_id = 'ebca3574-ad0a-400a-9cbc-dca382f5998c') AND (event_date = today()) -GROUP BY trace -ORDER BY count() DESC -LIMIT 10 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/performance-test.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/performance-test.md deleted file mode 100644 index 3728d9d47a5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/performance-test.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: '使用 ClickHouse 测试和基准测试硬件性能的指南' -sidebar_label: '测试硬件' -sidebar_position: 54 -slug: /operations/performance-test -title: '如何使用 ClickHouse 测试硬件性能' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; - - - -你可以在任何服务器上运行基本的 ClickHouse 性能测试,而无需安装 ClickHouse 软件包。 - -## 自动化运行 {#automated-run} - -你只需使用一个脚本即可运行基准测试。 - -1. 下载脚本。 - -```bash -wget https://raw.githubusercontent.com/ClickHouse/ClickBench/main/hardware/hardware.sh -``` - -2. 运行脚本。 - -```bash -chmod a+x ./hardware.sh -./hardware.sh -``` - -3. 复制输出内容并发送到 [feedback@clickhouse.com](mailto:feedback@clickhouse.com) - -所有结果均发布于此页面: [https://clickhouse.com/benchmark/hardware/](https://clickhouse.com/benchmark/hardware/) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-cache.md deleted file mode 100644 index 1a02919b38b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-cache.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -description: '使用和配置 ClickHouse 查询缓存功能的指南' -sidebar_label: '查询缓存' -sidebar_position: 65 -slug: /operations/query-cache -title: '查询缓存' -doc_type: 'guide' ---- - -# 查询缓存 {#query-cache} - -查询缓存允许某个 `SELECT` 查询只需计算一次,之后再次执行相同查询时可直接从缓存中返回结果。 -根据查询类型的不同,这可以显著降低 ClickHouse 服务器的延迟和资源消耗。 - -## 背景、设计和限制 {#background-design-and-limitations} - -查询缓存通常可以被视为事务一致或事务不一致两类。 - -* 在事务一致的缓存中,如果 `SELECT` 查询的结果发生变化或可能发生变化,数据库会使缓存的查询结果失效(丢弃)。 - 在 ClickHouse 中,更改数据的操作包括对表的插入 / 更新 / 删除操作,或者折叠合并(collapsing merges)。事务一致的缓存特别适用于 OLTP 数据库,例如 - [MySQL](https://dev.mysql.com/doc/refman/5.6/en/query-cache.html)(在 v8.0 之后移除了查询缓存)和 - [Oracle](https://docs.oracle.com/database/121/TGDBA/tune_result_cache.htm)。 -* 在事务不一致的缓存中,在假设所有缓存条目都被分配了一个有效期(例如 1 分钟),并且在此期间底层数据变化很少的前提下,可以接受查询结果存在轻微不准确。 - 这种方式总体上更适合 OLAP 数据库。作为一个事务不一致缓存已足够的示例,可以考虑报表工具中的每小时销售报表,该报表会被多个用户同时访问。销售数据通常变化得足够缓慢,因此数据库只需计算一次报表(由第一次 `SELECT` 查询表示),后续查询可以直接从查询缓存中返回。 - 在这个例子中,一个合理的有效期可以是 30 分钟。 - -事务不一致缓存传统上由与数据库交互的客户端工具或代理程序(例如 -[chproxy](https://www.chproxy.org/configuration/caching/))提供。因此,相同的缓存逻辑和配置往往会被重复实现。通过 ClickHouse 的查询缓存,缓存逻辑被移动到了服务端,从而减少维护工作量并避免冗余。 - -## 配置与使用 {#configuration-settings-and-usage} - -:::note -在 ClickHouse Cloud 中,必须使用[查询级别设置](/operations/settings/query-level)来编辑查询缓存设置。目前不支持编辑[配置级别设置](/operations/configuration-files)。 -::: - -:::note -[clickhouse-local](utilities/clickhouse-local.md) 一次只运行一个查询。由于查询结果缓存在这种场景下并不适用,因此在 clickhouse-local 中禁用了查询结果缓存。 -::: - -可以使用 [use_query_cache](/operations/settings/settings#use_query_cache) 设置来控制特定查询或当前会话中的所有查询是否使用查询缓存。例如,第一次执行查询时 - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true; -``` - -会将查询结果存储到查询缓存中。随后再次执行相同的查询(同样将参数 `use_query_cache = true` 设置为启用)时, -会从缓存中读取已计算的结果并立即返回。 - -:::note -设置 `use_query_cache` 以及所有其他与查询缓存相关的设置仅对独立的 `SELECT` 语句生效。特别地, -通过 `CREATE VIEW AS SELECT [...] SETTINGS use_query_cache = true` 创建的视图,其中的 `SELECT` 结果并不会被缓存,除非该 `SELECT` -语句在运行时也使用 `SETTINGS use_query_cache = true`。 -::: - -可以使用设置项 [enable_writes_to_query_cache](/operations/settings/settings#enable_writes_to_query_cache) -和 [enable_reads_from_query_cache](/operations/settings/settings#enable_reads_from_query_cache)(两者默认均为 `true`)来更细粒度地配置缓存的使用方式。前者控制查询结果是否会写入缓存,而后者决定数据库是否应尝试从缓存中读取查询结果。例如,下列查询仅被动地使用缓存,即尝试从缓存中读取结果,但不会将其结果写入缓存: - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, enable_writes_to_query_cache = false; -``` - -为了实现最大程度的控制,一般建议仅在特定查询上设置 `use_query_cache`、`enable_writes_to_query_cache` 和 -`enable_reads_from_query_cache`。也可以在用户或配置文件级别启用缓存(例如通过 `SET -use_query_cache = true`),但需要注意,此时所有 `SELECT` 查询都可能返回缓存结果。 - -可以使用语句 `SYSTEM DROP QUERY CACHE` 来清空查询缓存。查询缓存的内容显示在系统表 -[system.query_cache](system-tables/query_cache.md) 中。自数据库启动以来的查询缓存命中和未命中次数,会分别作为事件 -“QueryCacheHits”和“QueryCacheMisses”显示在系统表 [system.events](system-tables/events.md) 中。两个计数器仅会在 -`SELECT` 查询在设置 `use_query_cache = true` 的情况下运行时更新,其他查询不会影响 “QueryCacheMisses”。系统表 -[system.query_log](system-tables/query_log.md) 中字段 `query_cache_usage` 会对每个已执行的查询指示其结果是写入了查询缓存还是从查询缓存中读取。系统表 -[system.metrics](system-tables/metrics.md) 中的指标 `QueryCacheEntries` 和 `QueryCacheBytes` 显示查询缓存当前包含的条目数量和字节数。 - -查询缓存在每个 ClickHouse 服务器进程中各自独立存在一份。然而,默认情况下缓存结果不会在用户之间共享。可以更改这种行为(参见下文),但出于安全原因不推荐这样做。 - -查询结果在查询缓存中是通过其查询的[抽象语法树(AST)](https://en.wikipedia.org/wiki/Abstract_syntax_tree) 来引用的。这意味着缓存对大小写不敏感,例如 `SELECT 1` 和 `select 1` 被视为同一个查询。为了使匹配更加自然,所有与查询缓存和[输出格式](settings/settings-formats.md) 相关的查询级别设置都会从 AST 中移除。 - -如果查询因异常或用户取消而中止,则不会向查询缓存写入任何条目。 - -查询缓存的大小(字节数)、缓存条目的最大数量以及单个缓存条目的最大大小(按字节数和记录数)可以通过不同的 -[服务器配置选项](/operations/server-configuration-parameters/settings#query_cache) 进行配置。 - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - -还可以使用 [settings profiles](settings/settings-profiles.md) 和 [settings -constraints](settings/constraints-on-settings.md) 来限制单个用户的缓存使用量。更具体地说,可以限制用户在查询缓存中可分配的最大内存(以字节为单位)以及可存储的查询结果的最大数量。为此,首先在 `users.xml` 中的用户配置文件中设置 -[query_cache_max_size_in_bytes](/operations/settings/settings#query_cache_max_size_in_bytes) 和 -[query_cache_max_entries](/operations/settings/settings#query_cache_max_entries) 这两个配置项,然后将这两个设置设为只读: - -```xml - - - - 10000 - - 100 - - - - - - - - - - - -``` - -要设置查询结果可被缓存所需的最小运行时长,可以使用参数 -[query_cache_min_query_duration](/operations/settings/settings#query_cache_min_query_duration)。例如,下面这个查询的结果 - -```sql -SELECT some_expensive_calculation(column_1, column_2) -FROM table -SETTINGS use_query_cache = true, query_cache_min_query_duration = 5000; -``` - -仅当查询运行时间超过 5 秒时才会被缓存。也可以指定查询需要运行多少次其结果才会被缓存——为此请使用设置 [query_cache_min_query_runs](/operations/settings/settings#query_cache_min_query_runs)。 - -查询缓存中的条目在经过一段时间后会变为陈旧(time-to-live,TTL)。默认情况下,此时间为 60 秒,但可以在会话、配置文件或查询级别使用设置 [query_cache_ttl](/operations/settings/settings#query_cache_ttl) 指定不同的值。查询缓存会“惰性”地淘汰条目,即当一个条目变为陈旧时,并不会立刻从缓存中移除。相反,当要向查询缓存中插入一个新条目时,数据库会检查缓存中是否有足够的可用空间来存放新条目。如果没有,数据库会尝试移除所有陈旧条目。如果缓存仍然没有足够的可用空间,则不会插入新的条目。 - -如果通过 HTTP 运行查询,则 ClickHouse 会设置 `Age` 和 `Expires` 头部,其中包含缓存条目的存活时间(秒)和过期时间戳。 - -查询缓存中的条目默认会被压缩。这在一定程度上降低了总体内存占用,但会带来写入/读取查询缓存变慢的代价。要禁用压缩,请使用设置 [query_cache_compress_entries](/operations/settings/settings#query_cache_compress_entries)。 - -有时,为同一个查询保留多个缓存结果是有用的。可以使用设置 [query_cache_tag](/operations/settings/settings#query_cache_tag) 来实现,它充当查询缓存条目的标签(或命名空间)。查询缓存会将同一查询在不同标签下的结果视为不同的结果。 - -为同一个查询创建三个不同查询缓存条目的示例: - -```sql -SELECT 1 SETTINGS use_query_cache = true; -- query_cache_tag 隐式为 ''(空字符串) -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 1'; -SELECT 1 SETTINGS use_query_cache = true, query_cache_tag = 'tag 2'; -``` - -若只想从查询缓存中移除带有标签 `tag` 的条目,可以使用语句 `SYSTEM DROP QUERY CACHE TAG 'tag'`。 - -ClickHouse 按 [max_block_size](/operations/settings/settings#max_block_size) 行数以块(block)的形式读取表数据。由于过滤、聚合等操作,结果块通常远小于 `max_block_size`,但也存在远大于该值的情况。设置 -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results)(默认启用)用于控制在将结果块插入查询结果缓存之前,如果块太小则进行压缩合并(squash),如果块太大则拆分为大小为 `max_block_size` 的块。这样会降低向查询缓存写入的性能,但可以提高缓存条目的压缩率,并在之后从查询缓存返回查询结果时提供更自然的块粒度。 - -因此,查询缓存会为每个查询存储多个(部分)结果块。虽然这种行为是一个合理的默认设置,但可以通过设置 -[query_cache_squash_partial_results](/operations/settings/settings#query_cache_squash_partial_results) 来禁用。 - -另外,包含非确定性函数的查询结果默认不会被缓存。这类函数包括: - -* 访问字典的函数:[`dictGet()`](/sql-reference/functions/ext-dict-functions#dictget-dictgetordefault-dictgetornull) 等; -* [用户自定义函数](../sql-reference/statements/create/function.md),其 XML 定义中不包含 `true` 标签; -* 返回当前日期或时间的函数:[`now()`](../sql-reference/functions/date-time-functions.md#now), - [`today()`](../sql-reference/functions/date-time-functions.md#today), - [`yesterday()`](../sql-reference/functions/date-time-functions.md#yesterday) 等; -* 返回随机值的函数:[`randomString()`](../sql-reference/functions/random-functions.md#randomString), - [`fuzzBits()`](../sql-reference/functions/random-functions.md#fuzzBits) 等; -* 其结果依赖于查询处理中使用的内部块大小和顺序的函数: - [`nowInBlock()`](../sql-reference/functions/date-time-functions.md#nowInBlock) 等, - [`rowNumberInBlock()`](../sql-reference/functions/other-functions.md#rowNumberInBlock), - [`runningDifference()`](../sql-reference/functions/other-functions.md#runningDifference), - [`blockSize()`](../sql-reference/functions/other-functions.md#blockSize) 等; -* 依赖环境的函数:[`currentUser()`](../sql-reference/functions/other-functions.md#currentUser), - [`queryID()`](/sql-reference/functions/other-functions#queryID), - [`getMacro()`](../sql-reference/functions/other-functions.md#getMacro) 等。 - -若希望无论如何都缓存包含非确定性函数的查询结果,请使用设置 -[query_cache_nondeterministic_function_handling](/operations/settings/settings#query_cache_nondeterministic_function_handling)。 - -涉及 system 表的查询结果(例如 [system.processes](system-tables/processes.md) 或 -[information_schema.tables](system-tables/information_schema.md))默认不会被缓存。若希望无论如何都缓存包含 system 表的查询结果,请使用设置 [query_cache_system_table_handling](/operations/settings/settings#query_cache_system_table_handling)。 - -最后,出于安全原因,查询缓存中的条目不会在不同用户之间共享。例如,用户 A 不应当能够通过运行与另一位没有该行策略限制的用户 B 相同的查询,来绕过某个表上的行策略。不过,如有需要,可以通过设置 -[query_cache_share_between_users](/operations/settings/settings#query_cache_share_between_users) 将缓存条目标记为可被其他用户访问(即共享)。 - -## 相关内容 {#related-content} - -* 博客文章:[ClickHouse 查询缓存简介](https://clickhouse.com/blog/introduction-to-the-clickhouse-query-cache-and-design) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md deleted file mode 100644 index 51f4d8b3d0b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/query-condition-cache.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: '使用和配置 ClickHouse 查询条件缓存功能的指南' -sidebar_label: '查询条件缓存' -sidebar_position: 64 -slug: /operations/query-condition-cache -title: '查询条件缓存' -doc_type: 'guide' ---- - - - -# 查询条件缓存 {#query-condition-cache} - -:::note -查询条件缓存仅在 [enable_analyzer](https://clickhouse.com/docs/operations/settings/settings#enable_analyzer) 设置为 true 时生效,这是默认值。 -::: - -许多实际环境中的负载都会针对相同或几乎相同的数据反复执行查询(例如,既有数据加上新写入的数据)。 -ClickHouse 提供了多种优化技术来针对这类查询模式进行优化。 -一种做法是通过索引结构(例如主键索引、跳过索引、投影(projections))或预计算(物化视图)来调整物理数据布局。 -另一种做法是使用 ClickHouse 的 [query cache](query-cache.md)(查询缓存)来避免重复执行查询。 -第一种做法的缺点是需要数据库管理员进行人工调优和监控。 -第二种做法可能会返回过期结果(因为 query cache 在事务层面并非一致),这在某些用例中可以接受,而在另一些用例中则不行。 - -查询条件缓存为这两个问题提供了一种优雅的解决方案。 -它基于这样一个思想:在相同数据上评估某个过滤条件(例如 `WHERE col = 'xyz'`)将始终返回相同的结果。 -更具体地说,查询条件缓存会记住对于每个已评估的过滤条件以及每个 granule(默认情况下为包含 8192 行的一个数据块),该 granule 中是否没有任何行满足该过滤条件。 -该信息以单个位来记录:位 0 表示没有行匹配过滤条件,而位 1 表示至少存在一行匹配。 -在前一种情况下,ClickHouse 在执行过滤时可以跳过相应的 granule;在后一种情况下,则必须加载并评估该 granule。 - -在满足以下三个前提条件时,查询条件缓存效果显著: -- 第一,负载必须反复评估相同的过滤条件。如果一个查询被多次重复执行,这会自然发生;或者当两个查询共享相同的过滤条件时也会发生,例如 `SELECT product FROM products WHERE quality > 3` 和 `SELECT vendor, count() FROM products WHERE quality > 3`。 -- 第二,大部分数据是不可变的,即在查询之间不会变化。这在 ClickHouse 中通常成立,因为数据分片(part)是不可变的,只会通过 INSERT 创建。 -- 第三,过滤条件需要具有较高选择性,即只有相对较少的行满足过滤条件。匹配过滤条件的行越少,被记录为位 0(无匹配行)的 granule 就越多,从而可以在后续的过滤评估中“裁剪”掉更多数据。 - - - -## 内存消耗 {#memory-consumption} - -由于查询条件缓存针对每个过滤条件和粒度仅存储 1 位信息,因此只占用很少的内存。 -查询条件缓存的最大容量可以通过服务器设置 [`query_condition_cache_size`](server-configuration-parameters/settings.md#query_condition_cache_size) 进行配置(默认值:100 MB)。 -100 MB 的缓存容量对应 100 * 1024 * 1024 * 8 = 838,860,800 个条目。 -由于每个条目表示一个标记(默认对应 8192 行),因此缓存最多可以覆盖单个列的 6,871,947,673,600(6.8 万亿)行。 -在实际使用中,过滤通常会在多个列上进行评估,因此该数值需要除以参与过滤的列数。 - - - -## 配置项与用法 {#configuration-settings-and-usage} - -设置项 [use_query_condition_cache](settings/settings#use_query_condition_cache) 用于控制特定查询或当前会话中的所有查询是否使用查询条件缓存。 - -例如,第一次执行查询时 - -```sql -SELECT col1, col2 -FROM table -WHERE col1 = 'x' -SETTINGS use_query_condition_cache = true; -``` - -会缓存不满足该谓词的表数据范围。 -之后再次执行相同查询,并将参数 `use_query_condition_cache` 设为 `true` 时,会利用查询条件缓存,从而扫描更少的数据。 - - -## 管理 {#administration} - -查询条件缓存在 ClickHouse 重启后不会被保留。 - -要清除查询条件缓存,运行 [`SYSTEM DROP QUERY CONDITION CACHE`](../sql-reference/statements/system.md#drop-query-condition-cache)。 - -缓存的内容显示在系统表 [system.query_condition_cache](system-tables/query_condition_cache.md) 中。 -要计算当前查询条件缓存的大小(以 MB 为单位),运行 `SELECT formatReadableSize(sum(entry_size)) FROM system.query_condition_cache`。 -如需分析单个过滤条件,可以检查 `system.query_condition_cache` 中的 `condition` 字段。 -请注意,只有在启用了设置 [query_condition_cache_store_conditions_as_plaintext](settings/settings#query_condition_cache_store_conditions_as_plaintext) 的情况下运行查询时,该字段才会被填充。 - -自数据库启动以来查询条件缓存的命中与未命中次数,分别作为事件 "QueryConditionCacheHits" 和 "QueryConditionCacheMisses" 显示在系统表 [system.events](system-tables/events.md) 中。 -这两个计数器仅在设置 `use_query_condition_cache = true` 的 `SELECT` 查询时才会更新,其他查询不会影响 "QueryCacheMisses"。 - - - -## 相关内容 {#related-content} - -- 博客:[Introducing the Query Condition Cache](https://clickhouse.com/blog/introducing-the-clickhouse-query-condition-cache) -- [Predicate Caching: Query-Driven Secondary Indexing for Cloud Data Warehouses (Schmidt et. al., 2024)](https://doi.org/10.1145/3626246.3653395) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/quotas.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/quotas.md deleted file mode 100644 index 5ae8cb025f6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/quotas.md +++ /dev/null @@ -1,142 +0,0 @@ ---- -description: '在 ClickHouse 中配置和管理资源使用配额的指南' -sidebar_label: 'Quotas' -sidebar_position: 51 -slug: /operations/quotas -title: 'Quotas' -doc_type: 'guide' ---- - -:::note ClickHouse Cloud 中的配额 -ClickHouse Cloud 支持配额,但必须使用 [DDL 语法](/sql-reference/statements/create/quota) 来创建。下面文档中所述的 XML 配置方式**不受支持**。 -::: - -配额允许你在一段时间内限制资源使用,或跟踪资源的使用情况。 -配额是在用户配置中设置的,通常位于 'users.xml' 文件中。 - -系统还提供了限制单个查询复杂度的功能。参见[查询复杂度限制](../operations/settings/query-complexity.md)一节。 - -与查询复杂度限制不同,配额具有以下特性: - -* 对在一段时间内可运行的一组查询施加限制,而不是限制单个查询。 -* 统计分布式查询处理过程中在所有远程服务器上消耗的资源。 - -下面来看一下 'users.xml' 文件中定义配额的片段。 - -```xml - - - - - - - - 3600 - - - 0 - 0 - 0 - 0 - 0 - 0 - 0 - - -``` - -默认情况下,配额会按小时跟踪资源使用情况,但不会对使用量进行限制。 -为每个时间间隔计算出的资源消耗会在每次请求后输出到服务器日志中。 - -```xml - - - - - 3600 - - 1000 - 100 - 100 - 5000000 - 100 - 1000000000 - 100000000000 - 900 - 5 - - - - 86400 - - 10000 - 10000 - 10000 - 1000 - 5000000000 - 160000000000 - 500000000000 - 16000000000000 - 7200 - - -``` - -对于 'statbox' 配额,会分别设置每小时和每 24 小时(86,400 秒)的限制。时间区间从一个由实现定义的固定时刻开始计算。换句话说,这个 24 小时间隔不一定从午夜开始。 - -当时间区间结束时,所有已收集的数值都会被清空。在接下来的一个小时内,配额计算会重新开始。 - -以下是可以被限制的指标: - -`queries` – 请求总数。 - -`query_selects` – SELECT 请求总数。 - -`query_inserts` – INSERT 请求总数。 - -`errors` – 抛出异常的查询数量。 - -`result_rows` – 作为结果返回的行总数。 - -`result_bytes` - 作为结果返回的行的总大小。 - -`read_rows` – 为在所有远程服务器上运行查询而从表中读取的源行总数。 - -`read_bytes` - 为在所有远程服务器上运行查询而从表中读取的数据总大小。 - -`written_bytes` - 写入操作的数据总大小。 - -`execution_time` – 查询执行总时间(以秒为单位的墙钟时间)。 - -`failed_sequential_authentications` - 连续身份验证错误的总次数。 - - -如果在至少一个时间间隔内超出限制,将抛出一个异常,异常文本会说明超出了哪个限制、对应的是哪个时间间隔,以及新的时间间隔何时开始(即何时可以再次发送查询)。 - -配额可以使用“quota key”功能,对多个键的资源进行相互独立的统计和报告。下面是一个示例: - -```xml - - - - -``` - -在配置的 'users' 部分为用户分配配额。参见“Access rights(访问权限)”章节。 - -在分布式查询处理中,累积用量存储在发起请求的服务器上。因此,如果用户切换到另一台服务器,该服务器上的配额将会从头开始重新计算。 - -当服务器重启时,配额会被重置。 - - -## 相关内容 {#related-content} - -- 博文:[使用 ClickHouse 构建单页应用](https://clickhouse.com/blog/building-single-page-applications-with-clickhouse-and-http) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md deleted file mode 100644 index e35a6bd2d33..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_server_settings_outside_source.md +++ /dev/null @@ -1,2679 +0,0 @@ -## asynchronous_metric_log {#asynchronous_metric_log} - -在 ClickHouse Cloud 部署中默认启用。 - -如果在你的环境中该设置不是默认启用的,则根据 ClickHouse 的安装方式不同,你可以按照以下说明来启用或禁用它。 - -**启用** - -要手动开启异步指标日志历史记录收集 [`system.asynchronous_metric_log`](../../operations/system-tables/asynchronous_metric_log.md),请创建 `/etc/clickhouse-server/config.d/asynchronous_metric_log.xml` 文件,并写入以下内容: - -```xml - - - system - asynchronous_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**禁用** - -要禁用 `asynchronous_metric_log` 设置,请创建以下文件 `/etc/clickhouse-server/config.d/disable_asynchronous_metric_log.xml`,内容如下: - -```xml - -``` - - - - -## auth_use_forwarded_address {#auth_use_forwarded_address} - -对通过代理连接的客户端,在认证时使用其源地址。 - -:::note -此设置应格外谨慎使用,因为转发地址很容易被伪造——接受此类认证的服务器不应被直接访问,而应仅通过受信任的代理访问。 -::: - - - -## 备份 {#backups} - -用于在执行 [`BACKUP` 和 `RESTORE`](../backup.md) 语句时的备份相关设置。 - -以下设置可通过子标签进行配置: - - - -{/* SQL - WITH settings AS ( - SELECT arrayJoin([ - ('allow_concurrent_backups', 'Bool','确定是否允许在同一主机上并发运行多个备份操作。', 'true'), - ('allow_concurrent_restores', 'Bool', '确定是否允许在同一主机上并发运行多个恢复操作。', 'true'), - ('allowed_disk', 'String', '使用 `File()` 时用于备份的磁盘。必须先设置此参数才能使用 `File`。', ''), - ('allowed_path', 'String', '使用 `File()` 时用于备份的路径。必须先设置此参数才能使用 `File`。', ''), - ('attempts_to_collect_metadata_before_sleep', 'UInt', '在比较收集到的元数据后发现不一致的情况下,在进入休眠前尝试收集元数据的次数。', '2'), - ('collect_metadata_timeout', 'UInt64', '在备份期间收集元数据的超时时间(毫秒)。', '600000'), - ('compare_collected_metadata', 'Bool', '如果为 true,则会将收集到的元数据与现有元数据进行比较,以确保它们在备份过程中未被更改。', 'true'), - ('create_table_timeout', 'UInt64', '在恢复期间创建表的超时时间(毫秒)。', '300000'), - ('max_attempts_after_bad_version', 'UInt64', '在协调备份/恢复时遇到版本错误后重试的最大次数。', '3'), - ('max_sleep_before_next_attempt_to_collect_metadata', 'UInt64', '在下一次尝试收集元数据之前的最大休眠时间(毫秒)。', '100'), - ('min_sleep_before_next_attempt_to_collect_metadata', 'UInt64', '在下一次尝试收集元数据之前的最小休眠时间(毫秒)。', '5000'), - ('remove_backup_files_after_failure', 'Bool', '如果 `BACKUP` 命令失败,ClickHouse 将尝试删除在失败前已复制到备份中的文件;否则会保留这些已复制的文件。', 'true'), - ('sync_period_ms', 'UInt64', '协调备份/恢复的同步周期(毫秒)。', '5000'), - ('test_inject_sleep', 'Bool', '用于测试的休眠。', 'false'), - ('test_randomize_order', 'Bool', '如果为 true,为测试目的随机化某些操作的执行顺序。', 'false'), - ('zookeeper_path', 'String', '在使用 `ON CLUSTER` 子句时,存储备份和恢复元数据的 ZooKeeper 路径。', '/clickhouse/backups') - ]) AS t ) - SELECT concat('`', t.1, '`') AS Setting, t.2 AS Type, t.3 AS Description, concat('`', t.4, '`') AS Default FROM settings FORMAT Markdown - */ } - -| 设置 | 类型 | 描述 | 默认 | -| :-------------------------------------------------- | :------- | :----------------------------------------------------------------- | :-------------------- | -| `allow_concurrent_backups` | 布尔型 | 确定是否允许在同一主机上同时运行多个备份操作。 | `true` | -| `allow_concurrent_restores` | Bool | 确定是否允许在同一主机上并行执行多个恢复操作。 | `true` | -| `allowed_disk` | 字符串 | 使用 `File()` 时用于备份的磁盘。要使用 `File`,必须先配置此设置。 | `` | -| `allowed_path` | 字符串 | 使用 `File()` 时的备份路径。必须配置此项才能使用 `File`。 | `` | -| `attempts_to_collect_metadata_before_sleep` | UInt | 在比较已收集的元数据后如果发现不一致,在进入休眠前重试收集元数据的次数。 | `2` | -| `collect_metadata_timeout` | UInt64 | 用于在备份期间收集元数据的超时时间(毫秒)。 | `600000` | -| `compare_collected_metadata` | Bool | 若设置为 `true`,则会将收集的元数据与现有元数据进行比较,以确保它们在备份过程中未发生更改。 | `true` | -| `create_table_timeout` | UInt64 | 在恢复过程中创建表的超时时间(毫秒)。 | `300000` | -| `max_attempts_after_bad_version` | UInt64 | 在协调备份或恢复过程中遇到 bad version 错误时的最大重试次数。 | `3` | -| `max_sleep_before_next_attempt_to_collect_metadata` | UInt64 | 在下一次尝试收集元数据之前的最大休眠时间(毫秒)。 | `100` | -| `min_sleep_before_next_attempt_to_collect_metadata` | UInt64 | 在下一次尝试收集元数据前的最小休眠时间(以毫秒为单位)。 | `5000` | -| `remove_backup_files_after_failure` | Bool(布尔) | 如果 `BACKUP` 命令失败,ClickHouse 会尝试删除在失败前已复制到备份中的文件,否则会保留这些已复制的文件原样不动。 | `true` | -| `sync_period_ms` | UInt64 | 用于协调备份和恢复的同步周期,单位为毫秒。 | `5000` | -| `test_inject_sleep` | Bool | 测试中的休眠 | `false` | -| `test_randomize_order` | 布尔型 | 如果为 true,则会将某些操作的执行顺序随机化,用于测试。 | `false` | -| `zookeeper_path` | 字符串 | 在使用 `ON CLUSTER` 子句时,ZooKeeper 中存储备份和恢复元数据的路径。 | `/clickhouse/backups` | - -此设置的默认配置如下: - -```xml - - .... - -``` - - -## bcrypt_workfactor {#bcrypt_workfactor} - -用于 `bcrypt_password` 认证类型的工作因子,该类型使用 [Bcrypt 算法](https://wildlyinaccurate.com/bcrypt-choosing-a-work-factor/)。 -工作因子决定了计算哈希值和验证密码所需的计算量和时间。 - -```xml -12 -``` - -:::warning -对于需要高频鉴权的应用程序, -建议考虑使用其他鉴权方式, -因为在较高成本因子(cost factor)下,bcrypt 的计算开销较大。 -::: - - -## table_engines_require_grant {#table_engines_require_grant} - -如果设置为 `true`,用户需要被授予相应权限才能创建具有特定引擎的表,例如:`GRANT TABLE ENGINE ON TinyLog to user`。 - -:::note -默认情况下,为了向后兼容,使用特定表引擎创建表时会忽略权限检查,不过你可以通过将此设置为 `true` 来更改该行为。 -::: - - - -## builtin_dictionaries_reload_interval {#builtin_dictionaries_reload_interval} - -以秒为单位设置重新加载内置字典的时间间隔。 - -ClickHouse 每隔 x 秒重新加载一次内置字典。这样就可以在不重启服务器的情况下“即时”编辑字典。 - -**示例** - -```xml -3600 -``` - - -## 压缩 {#compression} - -[MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 引擎表的数据压缩设置。 - -:::note -如果您刚开始使用 ClickHouse,建议不要修改此设置。 -::: - -**配置模板**: - -```xml - - - ... - ... - ... - ... - - ... - -``` - -**`` 字段**: - -* `min_part_size` – 数据分片的最小大小。 -* `min_part_size_ratio` – 数据分片大小与表大小的比例。 -* `method` – 压缩方法。可接受的取值:`lz4`、`lz4hc`、`zstd`、`deflate_qpl`。 -* `level` – 压缩级别。参见 [Codecs](/sql-reference/statements/create/table#general-purpose-codecs)。 - -:::note -可以配置多个 `` 区块。 -::: - -**条件满足时的操作**: - -* 如果数据分片满足某个条件集,ClickHouse 使用指定的压缩方法。 -* 如果数据分片同时满足多个条件集,ClickHouse 使用第一个匹配的条件集。 - -:::note -如果某个数据分片不满足任何条件,ClickHouse 使用 `lz4` 压缩。 -::: - -**示例** - -```xml - - - 10000000000 - 0.01 - zstd - 1 - - -``` - - -## encryption {#encryption} - -配置用于获取密钥的命令,该密钥将用于[加密编解码器](/sql-reference/statements/create/table#encryption-codecs)。密钥(或多个密钥)应通过环境变量提供,或在配置文件中进行设置。 - -密钥可以是十六进制字符串,或长度等于 16 字节的普通字符串。 - -**示例** - -从配置中加载: - -```xml - - - 1234567812345678 - - -``` - -:::note -不建议在配置文件中存储密钥,这样做并不安全。你可以将密钥移到安全磁盘上的单独配置文件中,然后在 `config.d/` 文件夹中创建指向该配置文件的符号链接。 -::: - -当密钥为十六进制格式时,从配置中加载: - -```xml - - - 00112233445566778899aabbccddeeff - - -``` - -从环境变量中加载密钥: - -```xml - - - - - -``` - -这里,`current_key_id` 指定当前用于加密的密钥,而所有已指定的密钥都可用于解密。 - -以下每种方法都可以应用于多个密钥: - -```xml - - - 00112233445566778899aabbccddeeff - - 1 - - -``` - -此处的 `current_key_id` 表示当前用于加密的密钥。 - -此外,用户可以添加长度必须为 12 字节的 nonce(默认情况下,加密和解密过程使用由全零字节组成的 nonce): - -```xml - - - 012345678910 - - -``` - -或者可以设置为十六进制: - -```xml - - - abcdefabcdef - - -``` - -:::note -上述所有内容同样适用于 `aes_256_gcm_siv`(但密钥长度必须为 32 字节)。 -::: - - -## error_log {#error_log} - -默认情况下处于禁用状态。 - -**启用** - -要手动开启错误历史记录收集功能 [`system.error_log`](../../operations/system-tables/error_log.md),请创建 `/etc/clickhouse-server/config.d/error_log.xml` 文件,并填写如下内容: - -```xml - - - system - error_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**禁用** - -要禁用 `error_log` 设置,应创建以下文件 `/etc/clickhouse-server/config.d/disable_error_log.xml`,其内容如下: - -```xml - - - -``` - - - - -## custom_settings_prefixes {#custom_settings_prefixes} - -[自定义设置](/operations/settings/query-level#custom_settings) 的前缀列表。多个前缀之间必须以逗号分隔。 - -**示例** - -```xml -custom_ -``` - -**另请参阅** - -* [自定义设置](/operations/settings/query-level#custom_settings) - - -## core_dump {#core_dump} - -配置核心转储(core dump)文件大小的软限制。 - -:::note -硬限制需通过系统工具进行配置。 -::: - -**示例** - -```xml - - 1073741824 - -``` - - -## default_profile {#default_profile} - -默认设置概要。设置概要位于由 `user_config` 设置指定的文件中。 - -**示例** - -```xml -default -``` - - -## dictionaries_config {#dictionaries_config} - -字典配置文件的路径。 - -路径: - -* 指定绝对路径或相对于服务器配置文件的相对路径。 -* 路径中可以包含通配符 * 和 ?。 - -另请参阅: - -* "[Dictionaries](../../sql-reference/dictionaries/index.md)"。 - -**示例** - -```xml -*_dictionary.xml -``` - - -## user_defined_executable_functions_config {#user_defined_executable_functions_config} - -可执行用户自定义函数配置文件的路径。 - -路径: - -* 指定绝对路径或相对于服务器配置文件的相对路径。 -* 路径可以包含通配符 * 和 ?。 - -另请参阅: - -* “[可执行用户自定义函数](/sql-reference/functions/udf#executable-user-defined-functions)”。 - -**示例** - -```xml -*_function.xml -``` - - -## format_schema_path {#format_schema_path} - -包含输入数据 schema 的目录路径,例如用于 [CapnProto](/interfaces/formats/CapnProto) 格式的 schema。 - -**示例** - -```xml - -format_schemas/ -``` - - -## graphite {#graphite} - -将数据发送至 [Graphite](https://github.com/graphite-project)。 - -配置: - -* `host` – Graphite 服务器地址。 -* `port` – Graphite 服务器的端口。 -* `interval` – 发送间隔(秒)。 -* `timeout` – 发送数据的超时时间(秒)。 -* `root_path` – 键的前缀。 -* `metrics` – 从 [system.metrics](/operations/system-tables/metrics) 表发送数据。 -* `events` – 从 [system.events](/operations/system-tables/events) 表发送在该时间段内累积的增量数据。 -* `events_cumulative` – 从 [system.events](/operations/system-tables/events) 表发送累计数据。 -* `asynchronous_metrics` – 从 [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) 表发送数据。 - -你可以配置多个 `` 配置段。例如,可以用它们以不同的时间间隔发送不同的数据。 - -**示例** - -```xml - - localhost - 42000 - 0.1 - 60 - one_min - true - true - false - true - -``` - - -## graphite_rollup {#graphite_rollup} - -用于对 Graphite 数据进行降采样的设置。 - -更多详情参见 [GraphiteMergeTree](../../engines/table-engines/mergetree-family/graphitemergetree.md)。 - -**示例** - -```xml - - - max - - 0 - 60 - - - 3600 - 300 - - - 86400 - 3600 - - - -``` - - -## google_protos_path {#google_protos_path} - -定义一个包含 Protobuf 类型所需 proto 文件的目录。 - -示例: - -```xml -/usr/share/clickhouse/protos/ -``` - - -## http_handlers {#http_handlers} - -允许使用自定义 HTTP 处理器。 -要添加一个新的 http 处理器,只需添加一个新的 ``。 -规则会按照定义的顺序从上到下进行检查, -第一个匹配到的规则会运行对应的处理器。 - -以下设置可以通过子标签进行配置: - -| Sub-tags | Definition | -| -------------------- | ---------------------------------------------------------------- | -| `url` | 用于匹配请求 URL,可以使用前缀 'regex:' 来启用正则匹配(可选) | -| `methods` | 用于匹配请求方法,可以使用逗号分隔多个待匹配方法(可选) | -| `headers` | 用于匹配请求头,匹配每个子元素(子元素名称为请求头名称),可以使用前缀 'regex:' 来启用正则匹配(可选) | -| `handler` | 请求处理器 | -| `empty_query_string` | 检查 URL 中是否不存在查询字符串 | - -`handler` 包含以下设置,这些设置可以通过子标签进行配置: - -| Sub-tags | Definition | -| ------------------ | ----------------------------------------------------------------------------------------------- | -| `url` | 重定向的目标地址 | -| `type` | 支持的类型:static、dynamic_query_handler、predefined_query_handler、redirect | -| `status` | 与 static 类型配合使用,响应状态码 | -| `query_param_name` | 与 dynamic_query_handler 类型配合使用,从 HTTP 请求参数中提取并执行名称为 `` 的参数值 | -| `query` | 与 predefined_query_handler 类型配合使用,在处理器被调用时执行该查询 | -| `content_type` | 与 static 类型配合使用,响应的 content-type | -| `response_content` | 与 static 类型配合使用,发送给客户端的响应内容。当使用前缀 'file://' 或 'config://' 时,将从文件或配置中读取内容并发送给客户端 | - -除了规则列表之外,你还可以指定 ``,用于启用所有默认处理器。 - -示例: - -```xml - - - / - POST,GET - no-cache - - dynamic_query_handler - query - - - - - /predefined_query - POST,GET - - predefined_query_handler - SELECT * FROM system.settings - - - - - - static - 200 - text/plain; charset=UTF-8 - config://http_server_default_response - - - -``` - - -## http_server_default_response {#http_server_default_response} - -在访问 ClickHouse HTTP(s) 服务器时默认显示的页面。 -默认值为 "Ok."(结尾带有换行符)。 - -**示例** - -在访问 `http://localhost: http_port` 时会打开 `https://tabix.io/`。 - -```xml - -
]]> -
-``` - - -## http_options_response {#http_options_response} - -用于在 `OPTIONS` HTTP 请求的响应中添加响应头。 -`OPTIONS` 方法用于发起 CORS 预检请求。 - -更多信息参见 [OPTIONS](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/OPTIONS)。 - -示例: - -```xml - -
- Access-Control-Allow-Origin - * -
-
- Access-Control-Allow-Headers - origin, x-requested-with, x-clickhouse-format, x-clickhouse-user, x-clickhouse-key, Authorization -
-
- Access-Control-Allow-Methods - POST, GET, OPTIONS -
-
- Access-Control-Max-Age - 86400 -
-
-``` - - -## hsts_max_age {#hsts_max_age} - -HSTS 的有效期(单位:秒)。 - -:::note -值为 `0` 表示 ClickHouse 会禁用 HSTS。若设置为正数,则会启用 HSTS,且 max-age 即为你设置的数值。 -::: - -**示例** - -```xml -600000 -``` - - -## mlock_executable {#mlock_executable} - -在启动后执行 `mlockall`,以降低首次查询的延迟,并防止在高 IO 负载下 ClickHouse 可执行文件被换出到磁盘。 - -:::note -建议启用此选项,但会导致启动时间最多增加几秒钟。 -请注意,如果没有 `CAP_IPC_LOCK` 能力,此设置将不起作用。 -::: - -**示例** - -```xml -false -``` - - -## include_from {#include_from} - -替换定义所在的文件路径。支持 XML 和 YAML 格式。 - -有关更多信息,请参阅“[配置文件](/operations/configuration-files)”一节。 - -**示例** - -```xml -/etc/metrica.xml -``` - - -## interserver_listen_host {#interserver_listen_host} - -对可在 ClickHouse 服务器之间交换数据的主机进行限制。 -如果使用 Keeper,则同样的限制也会应用于不同 Keeper 实例之间的通信。 - -:::note -默认情况下,该值等于 [`listen_host`](#listen_host) 设置。 -::: - -**示例** - -```xml -::ffff:a00:1 -10.0.0.1 -``` - -类型: - -默认值: - - -## interserver_http_port {#interserver_http_port} - -用于 ClickHouse 服务器之间数据交换的端口。 - -**示例** - -```xml -9009 -``` - - -## interserver_http_host {#interserver_http_host} - -可供其他服务器访问本服务器时使用的主机名。 - -如果省略,则会以与 `hostname -f` 命令相同的方式确定。 - -在需要不再绑定到某个特定网络接口时非常有用。 - -**示例** - -```xml -example.clickhouse.com -``` - - -## interserver_https_port {#interserver_https_port} - -用于通过 `HTTPS` 在 ClickHouse 服务器之间进行数据交换的端口。 - -**示例** - -```xml -9010 -``` - - -## interserver_https_host {#interserver_https_host} - -与 [`interserver_http_host`](#interserver_http_host) 类似,不同之处在于,该主机名供其他服务器通过 `HTTPS` 访问本服务器时使用。 - -**示例** - -```xml -example.clickhouse.com -``` - - -## interserver_http_credentials {#interserver_http_credentials} - -在[复制](../../engines/table-engines/mergetree-family/replication.md)期间用于连接其他服务器的用户名和密码。此外,服务器也使用这些凭据对其他副本进行身份验证。 -因此,集群中所有副本的 `interserver_http_credentials` 必须相同。 - -:::note - -* 默认情况下,如果省略 `interserver_http_credentials` 部分,在复制过程中将不使用身份验证。 -* `interserver_http_credentials` 设置与 ClickHouse 客户端凭据[配置](../../interfaces/cli.md#configuration_files)无关。 -* 这些凭据同时适用于通过 `HTTP` 和 `HTTPS` 进行的复制。 - ::: - -可以通过子标签配置以下设置: - -* `user` — 用户名。 -* `password` — 密码。 -* `allow_empty` — 如果为 `true`,即使已设置凭据,也允许其他副本在不进行身份验证的情况下连接。如果为 `false`,则拒绝未进行身份验证的连接。默认值:`false`。 -* `old` — 包含在凭据轮换期间使用的旧 `user` 和 `password`。可以指定多个 `old` 部分。 - -**凭据轮换** - -ClickHouse 支持在无需同时停止所有副本以更新其配置的情况下,动态轮换 interserver 凭据。可以分几步更改凭据。 - -要启用身份验证,请将 `interserver_http_credentials.allow_empty` 设置为 `true` 并添加凭据。这样既允许使用身份验证的连接,也允许不使用身份验证的连接。 - -```xml - - admin - 111 - true - -``` - -在所有副本都配置完成后,将 `allow_empty` 设置为 `false`,或者移除该设置。这样会强制要求必须使用新凭据进行身份验证。 - -要更改现有凭据,将用户名和密码移动到 `interserver_http_credentials.old` 部分,并使用新值更新 `user` 和 `password`。此时,服务器会使用新凭据连接其他副本,并接受使用新旧两套凭据发起的连接。 - -```xml - - admin - 222 - - admin - 111 - - - temp - 000 - - -``` - -当新凭证已应用于所有副本后,即可删除旧凭证。 - - -## ldap_servers {#ldap_servers} - -在此列出 LDAP 服务器及其连接参数,以便: -- 将其用作指定本地用户的认证源,这类用户在认证机制中指定 `ldap` 而不是 `password` -- 将其用作远程用户目录。 - -可以通过子标签配置以下设置: - -| Setting | Description | -|--------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `host` | LDAP 服务器主机名或 IP,此参数为必填项且不能为空。 | -| `port` | LDAP 服务器端口,如果 `enable_tls` 设置为 true,则默认值为 636,否则为 `389`。 | -| `bind_dn` | 用于构造绑定 DN 的模板。最终 DN 将通过在每次认证尝试期间,将模板中的所有 `\{user_name\}` 子串替换为实际的用户名来构造。 | -| `user_dn_detection` | 用于检测已绑定用户实际用户 DN 的 LDAP 搜索参数配置段。当服务器为 Active Directory 时,这主要用于在后续角色映射时的搜索过滤器中。最终用户 DN 将在需要替换 `\{user_dn\}` 子串的地方使用。默认情况下,用户 DN 被设置为绑定 DN,但一旦搜索执行完成,它将被更新为实际检测到的用户 DN 值。 | -| `verification_cooldown` | 在一次成功的绑定尝试之后的一段时间(以秒为单位),在此期间会假定该用户在所有连续请求中均已成功通过认证,而无需联系 LDAP 服务器。指定 `0`(默认值)以禁用缓存,并强制对每个认证请求都联系 LDAP 服务器。 | -| `enable_tls` | 用于触发与 LDAP 服务器建立安全连接的标志。指定 `no` 以使用明文协议 (`ldap://`)(不推荐)。指定 `yes` 以使用基于 SSL/TLS 的 LDAP (`ldaps://`) 协议(推荐,默认值)。指定 `starttls` 以使用传统 StartTLS 协议(先使用明文协议 `ldap://`,然后升级为 TLS)。 | -| `tls_minimum_protocol_version` | SSL/TLS 的最小协议版本。可接受的值为:`ssl2`、`ssl3`、`tls1.0`、`tls1.1`、`tls1.2`(默认值)。 | -| `tls_require_cert` | SSL/TLS 对端证书验证行为。可接受的值为:`never`、`allow`、`try`、`demand`(默认值)。 | -| `tls_cert_file` | 证书文件路径。 | -| `tls_key_file` | 证书密钥文件路径。 | -| `tls_ca_cert_file` | CA 证书文件路径。 | -| `tls_ca_cert_dir` | 包含 CA 证书的目录路径。 | -| `tls_cipher_suite` | 允许的密码套件(使用 OpenSSL 表示法)。 | - -`user_dn_detection` 设置可以使用子标签进行配置: - -| Setting | Description | -|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `base_dn` | 用于构造 LDAP 搜索基础 DN 的模板。最终 DN 将通过在 LDAP 搜索期间,将模板中的所有 `\{user_name\}` 和 `\{bind_dn\}` 子串替换为实际用户名和绑定 DN 来构造。 | -| `scope` | LDAP 搜索的范围。可接受的值为:`base`、`one_level`、`children`、`subtree`(默认值)。 | -| `search_filter` | 用于构造 LDAP 搜索过滤器的模板。最终过滤器将通过在 LDAP 搜索期间,将模板中的所有 `\{user_name\}`、`\{bind_dn\}` 和 `\{base_dn\}` 子串替换为实际用户名、绑定 DN 和基础 DN 来构造。注意,特殊字符必须在 XML 中正确转义。 | - -示例: - - - -```xml - - localhost - 636 - uid={user_name},ou=users,dc=example,dc=com - 300 - yes - tls1.2 - demand - /path/to/tls_cert_file - /path/to/tls_key_file - /path/to/tls_ca_cert_file - /path/to/tls_ca_cert_dir - ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:AES256-GCM-SHA384 - -``` - -示例(典型 Active Directory,已配置用户 DN 检测,以便进行后续角色映射): - -```xml - - localhost - 389 - EXAMPLE\{user_name} - - CN=Users,DC=example,DC=com - (&(objectClass=user)(sAMAccountName={user_name})) - - no - -``` - - -## listen_host {#listen_host} - -用于限制允许发起请求的主机范围。如果希望服务器响应所有主机,请指定 `::`。 - -示例: - -```xml -::1 -127.0.0.1 -``` - - -## listen_try {#listen_try} - -在尝试开始监听时,即使 IPv6 或 IPv4 网络不可用,服务器也不会退出。 - -**示例** - -```xml -0 -``` - - -## listen_reuse_port {#listen_reuse_port} - -允许多个服务器监听同一地址和端口(address:port)。操作系统会将请求随机路由到某个服务器。不建议启用此设置。 - -**示例** - -```xml -0 -``` - -类型: - -默认值: - - -## listen_backlog {#listen_backlog} - -监听套接字的 backlog(待处理连接的队列大小)。默认值 `4096` 与 Linux [5.4+](https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=19f92a030ca6d772ab44b22ee6a01378a8cb32d4)) 的默认值相同。 - -通常不需要修改该值,因为: - -* 默认值已经足够大, -* 为了接受客户端连接,服务器有单独的线程。 - -因此,即使你看到 ClickHouse 服务器的 `TcpExtListenOverflows`(来自 `nstat`)为非零且该计数器在增长,也并不意味着需要增大这个值,因为: - -* 通常如果 `4096` 都不够,这说明存在某些 ClickHouse 内部的扩展性问题,因此最好报告一个问题(提交 issue)。 -* 这并不意味着服务器之后就能处理更多连接(即便可以,到那时客户端可能已经离开或断开连接)。 - -**示例** - -```xml -4096 -``` - - -## logger {#logger} - -日志消息的位置和格式。 - -**键**: - -| Key | Description | -|------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `level` | 日志级别。可接受的值:`none`(关闭日志),`fatal`,`critical`,`error`,`warning`,`notice`,`information`,`debug`,`trace`,`test` | -| `log` | 日志文件的路径。 | -| `errorlog` | 错误日志文件的路径。 | -| `size` | 轮转策略:日志文件的最大大小(字节数)。一旦日志文件大小超过该阈值,它会被重命名并归档,然后创建新的日志文件。 | -| `count` | 轮转策略:最多保留的历史日志文件数量。 | -| `stream_compress` | 使用 LZ4 压缩日志消息。设置为 `1` 或 `true` 以启用。 | -| `console` | 启用日志输出到控制台。设置为 `1` 或 `true` 以启用。如果 ClickHouse 不以守护进程模式运行,默认值为 `1`,否则为 `0`。 | -| `console_log_level` | 控制台输出的日志级别。默认为 `level`。 | -| `formatting.type` | 控制台输出的日志格式。目前仅支持 `json`。 | -| `use_syslog` | 同时将日志输出转发到 syslog。 | -| `syslog_level` | 输出到 syslog 的日志级别。 | -| `async` | 当为 `true`(默认)时,日志将异步记录(每个输出通道使用一个后台线程)。否则将在调用 LOG 的线程中同步记录。 | -| `async_queue_max_size` | 使用异步日志记录时,队列中等待刷新的最大消息数量。超过部分的消息将被丢弃。 | -| `startup_level` | 启动级别,用于在服务器启动时设置根 logger 的级别。启动完成后,日志级别会恢复为 `level` 设置。 | -| `shutdown_level` | 关闭级别,用于在服务器关闭时设置根 logger 的级别。 | - -**日志格式说明符** - -`log` 和 `errorLog` 路径中的文件名支持以下格式说明符,用于生成最终的文件名(目录部分不支持这些说明符)。 - -“Example” 列展示了在 `2023-07-06 18:32:07` 时的输出结果。 - - - -| 说明符 | 说明 | 示例 | -| ---- | ------------------------------------------------------------------------------------------------------ | -------------------------- | -| `%%` | 字面百分号 (%) | `%` | -| `%n` | 换行符 | | -| `%t` | 水平制表符字符 | | -| `%Y` | 年份(十进制表示),例如 2017 | `2023` | -| `%y` | 年份最后两位,以十进制表示(范围 [00,99]) | `23` | -| `%C` | 年份的前 2 位数字,以十进制表示(范围 [00,99]) | `20` | -| `%G` | 四位数的 [ISO 8601 周历年份](https://en.wikipedia.org/wiki/ISO_8601#Week_dates),即包含指定周的年份。通常仅在与 `%V` 搭配使用时才有意义 | `2023` | -| `%g` | [基于周的 ISO 8601 年份](https://en.wikipedia.org/wiki/ISO_8601#Week_dates) 的后 2 位数字,即包含指定周的年份。 | `23` | -| `%b` | 缩写的月份名称,例如 Oct(取决于区域设置) | `7月` | -| `%h` | %b 的同义词 | `7月` | -| `%B` | 完整的月份名称,例如 “October”(依赖于区域设置) | `7月` | -| `%m` | 以十进制数字表示的月份(范围为 [01,12]) | `07` | -| `%U` | 一年中的第几周,以十进制数表示(星期日为每周的第一天)(范围 [00,53]) | `27` | -| `%W` | 以十进制数字表示的一年中的周序号(星期一为每周的第一天)(范围 [00,53]) | `27` | -| `%V` | ISO 8601 周编号(范围 [01,53]) | `27` | -| `%j` | 一年中的第几天,按十进制数表示(范围为 [001,366]) | `187` | -| `%d` | 月份中的日期,以零填充的十进制数字表示(范围 [01,31])。个位数前加前导零。 | `06` | -| `%e` | 月份中的日期,使用空格填充的十进制数字(范围 [1,31])。若为一位数,则在前面补一个空格。 | `  6` | -| `%a` | 缩写的星期名称,例如 Fri(取决于区域设置) | `周四` | -| `%A` | 完整的星期名称,例如 Friday(依区域设置而定) | `星期四` | -| `%w` | 以整数表示星期几,其中星期日为 0(范围 [0-6]) | `4` | -| `%u` | 星期几的十进制表示,其中星期一为 1(ISO 8601 格式)(范围 [1-7]) | `4` | -| `%H` | 以十进制表示的小时数,24 小时制(范围 [00-23]) | `18` | -| `%I` | 以十进制表示的小时数(12 小时制,范围 [01,12]) | `06` | -| `%M` | 以十进制数表示的分钟(范围 [00,59]) | `32` | -| `%S` | 秒(十进制数,范围 [00,60]) | `07` | -| `%c` | 标准日期和时间字符串,例如 Sun Oct 17 04:41:13 2010(取决于区域设置) | `Thu Jul 6 18:32:07 2023` | -| `%x` | 本地化日期表示(取决于区域设置) | `07/06/23` | -| `%X` | 本地化的时间表示形式,例如 18:40:20 或 6:40:20 PM(因区域设置而异) | `18:32:07` | -| `%D` | 短格式 MM/DD/YY 日期,与 %m/%d/%y 等价 | `07/06/23` | -| `%F` | 短日期格式 YYYY-MM-DD,与 %Y-%m-%d 等价 | `2023-07-06` | -| `%r` | 本地化的 12 小时制时间(取决于区域设置) | `06:32:07 PM` | -| `%R` | 等同于 "%H:%M" | `18:32` | -| `%T` | 等同于 "%H:%M:%S"(ISO 8601 时间格式) | `18:32:07` | -| `%p` | 本地化的 a.m./p.m. 标记(依区域设置而定) | `下午` | -| `%z` | 以 ISO 8601 格式表示的相对于 UTC 的偏移量(例如 -0430),如果时区信息不可用则留空 | `+0800` | -| `%Z` | 与当前语言环境相关的时区名称或缩写;如果时区信息不可用,则为空字符串 | `Z AWST ` | - -**示例** - -```xml - - trace - /var/log/clickhouse-server/clickhouse-server-%F-%T.log - /var/log/clickhouse-server/clickhouse-server-%F-%T.err.log - 1000M - 10 - true - -``` - -若只想在控制台输出日志消息: - -```xml - - information - true - -``` - -**按级别覆盖** - -可以单独覆盖某些日志名称的日志级别。例如,要静默日志记录器 “Backup” 和 “RBAC” 的所有消息。 - -```xml - - - - Backup - none - - - RBAC - none - - - -``` - -**syslog** - -要同时将日志消息写入 syslog: - -```xml - - 1 - -
syslog.remote:10514
- myhost.local - LOG_LOCAL6 - syslog -
-
-``` - -`` 的键: - -| Key | Description | -| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `address` | syslog 的地址,格式为 `host\[:port\]`。如果省略,则使用本地 syslog 守护进程。 | -| `hostname` | 产生日志的主机名(可选)。 | -| `facility` | syslog 的 [facility 关键字](https://en.wikipedia.org/wiki/Syslog#Facility)。必须使用大写并以 “LOG_” 为前缀,例如 `LOG_USER`、`LOG_DAEMON`、`LOG_LOCAL3` 等。默认值:如果指定了 `address`,则为 `LOG_USER`,否则为 `LOG_DAEMON`。 | -| `format` | 日志消息格式。可选值:`bsd` 和 `syslog`。 | - -**日志格式** - -可以指定输出到控制台的日志格式。目前仅支持 JSON。 - -**示例** - -下面是一个 JSON 日志输出示例: - -```json -{ - "date_time_utc": "2024-11-06T09:06:09Z", - "date_time": "1650918987.180175", - "thread_name": "#1", - "thread_id": "254545", - "level": "Trace", - "query_id": "", - "logger_name": "BaseDaemon", - "message": "已接收信号 2", - "source_file": "../base/daemon/BaseDaemon.cpp; virtual void SignalListener::run()", - "source_line": "192" -} -``` - -要启用对 JSON 日志的支持,请使用以下代码片段: - -```xml - - - json - - - - date_time - thread_name - thread_id - level - query_id - logger_name - message - source_file - source_line - - - -``` - -**为 JSON 日志重命名键** - -可以通过修改 `` 标签内各标签的值来更改键名。例如,要将 `DATE_TIME` 更改为 `MY_DATE_TIME`,可以使用 `MY_DATE_TIME`。 - -**在 JSON 日志中省略键** - -可以通过注释掉属性来省略日志属性。例如,如果不希望日志打印 `query_id`,可以注释掉 `` 标签。 - - -## send_crash_reports {#send_crash_reports} - -用于向 ClickHouse 核心开发团队发送崩溃报告的设置。 - -特别是在预生产环境中,强烈建议启用此功能。 - -键: - -| Key | Description | -| --------------------- | -------------------------------------------------------------------------------- | -| `enabled` | 控制是否启用该功能的布尔开关,默认为 `true`。设置为 `false` 可避免发送崩溃报告。 | -| `send_logical_errors` | `LOGICAL_ERROR` 类似于 `assert`,表示 ClickHouse 中的一个缺陷。此布尔开关用于控制是否发送这些异常(默认值:`true`)。 | -| `endpoint` | 可自定义用于发送崩溃报告的 endpoint URL。 | - -**推荐用法** - -```xml - - true - -``` - - -## ssh_server {#ssh_server} - -主机密钥的公钥部分会在首次连接时写入 SSH 客户端的 known_hosts 文件中。 - -主机密钥配置默认处于未启用状态。 -取消注释主机密钥配置,并提供相应 SSH 密钥的路径以将其启用: - -示例: - -```xml - - path_to_the_ssh_key - path_to_the_ssh_key - path_to_the_ssh_key - -``` - - -## tcp_ssh_port {#tcp_ssh_port} - -用于 SSH 服务器的端口,允许用户通过 PTY 使用内置客户端进行交互式连接并执行查询。 - -示例: - -```xml -9022 -``` - - -## storage_configuration {#storage_configuration} - -允许进行多磁盘存储配置。 - -存储配置的结构如下: - -```xml - - - - - - - - -``` - -### 磁盘配置 {#configuration-of-disks} - -`disks` 的配置结构如下: - -```xml - - - - /mnt/fast_ssd/clickhouse/ - - - /mnt/hdd1/clickhouse/ - 10485760 - - - /mnt/hdd2/clickhouse/ - 10485760 - - ... - - -``` - -以上子标签定义了 `disks` 的以下设置: - -| Setting | Description | -| ----------------------- | ---------------------------------------------- | -| `` | 磁盘名称,必须唯一。 | -| `path` | 用于存储服务器数据(`data` 和 `shadow` 目录)的路径。必须以 `/` 结尾。 | -| `keep_free_space_bytes` | 磁盘上预留空闲空间的大小。 | - -:::note -磁盘的顺序没有影响。 -::: - -### 策略配置 {#configuration-of-policies} - -以上子标签定义了 `policies` 的以下设置: - - -| Setting | Description | -|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `policy_name_N` | 策略名称。策略名称必须唯一。 | -| `volume_name_N` | 卷名称。卷名称必须唯一。 | -| `disk` | 位于该卷内部的磁盘。 | -| `max_data_part_size_bytes` | 可以驻留在此卷中任一磁盘上的数据分片的最大大小。如果合并结果预计会产生大于 `max_data_part_size_bytes` 的分片,则该分片会被写入下一个卷。基本上,该特性允许你将新的/较小的分片存储在热(SSD)卷上,并在其达到较大尺寸时将其移动到冷(HDD)卷。如果策略仅包含一个卷,请不要使用此选项。 | -| `move_factor` | 卷上可用空闲空间的占比。如果可用空间低于该占比,数据将开始转移到下一个卷(如果存在)。在转移过程中,分片按大小从大到小(降序)排序,选择其总大小足以满足 `move_factor` 条件的分片;如果所有分片的总大小仍不足以满足该条件,则会移动所有分片。 | -| `perform_ttl_move_on_insert` | 禁用在插入时移动已过期 TTL 的数据。默认情况下(启用时),如果插入的数据根据 TTL 移动规则已过期,则会立刻被移动到规则中指定的卷/磁盘。如果目标卷/磁盘较慢(例如 S3),这会显著减慢写入。如果禁用该选项,已过期的数据部分会先写入默认卷,然后根据过期 TTL 规则立即移动到指定的卷。 | -| `load_balancing` | 磁盘负载均衡策略,`round_robin` 或 `least_used`。 | -| `least_used_ttl_ms` | 设置更新所有磁盘可用空间的超时时间(以毫秒为单位)(`0` — 始终更新,`-1` — 从不更新,默认值为 `60000`)。注意,如果磁盘仅由 ClickHouse 使用,并且不会在运行中进行文件系统容量调整,可以使用 `-1`。在其他所有情况下都不推荐这样做,因为最终会导致错误的空间分配。 | -| `prefer_not_to_merge` | 禁用在该卷上对数据分片进行合并。注意:这可能有害并导致性能下降。当启用此设置时(不要这样做),禁止在该卷上合并数据(这很糟糕)。这允许控制 ClickHouse 与慢磁盘的交互方式。我们建议完全不要使用此设置。 | -| `volume_priority` | 定义填充卷的优先级(顺序)。值越小,优先级越高。参数值必须是自然数,并且无间断地覆盖从 1 到 N 的范围(N 为指定的最大参数值)。 | - -对于 `volume_priority`: -- 如果所有卷都设置了该参数,则按指定顺序确定优先级。 -- 如果只有_部分_卷设置了该参数,未设置该参数的卷优先级最低。已设置该参数的卷根据该参数值确定优先级,其余卷的优先级由它们在配置文件中的描述顺序相互之间决定。 -- 如果_没有任何_卷设置该参数,则按它们在配置文件中的描述顺序确定优先级。 -- 各卷的优先级可以不同。 - - - -## macros {#macros} - -用于复制表的参数替换。 - -如果不使用复制表,则可以省略此配置。 - -有关更多信息,请参阅[创建复制表](../../engines/table-engines/mergetree-family/replication.md#creating-replicated-tables)部分。 - -**示例** - -```xml - -``` - - -## replica_group_name {#replica_group_name} - -用于 Replicated 数据库的副本组名称。 - -由 Replicated 数据库创建的集群将由同一副本组中的副本组成。 -DDL 查询只会等待同一副本组中的副本完成。 - -默认为空。 - -**示例** - -```xml -备份 -``` - - -## remap_executable {#remap_executable} - -用于将机器码(“text”)的内存重新映射到大页上的设置。 - -:::note -此功能仍处于高度实验阶段。 -::: - -示例: - -```xml -false -``` - - -## max_open_files {#max_open_files} - -最大可打开文件数。 - -:::note -我们建议在 macOS 上使用此选项,因为 `getrlimit()` 函数返回的值不正确。 -::: - -**示例** - -```xml -262144 -``` - - -## max_session_timeout {#max_session_timeout} - -会话的最大超时时长(单位:秒)。 - -示例: - -```xml -3600 -``` - - -## merge_tree {#merge_tree} - -针对 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表的调优设置。 - -有关更多信息,请参阅 MergeTreeSettings.h 头文件。 - -**示例** - -```xml - - 5 - -``` - - -## metric_log {#metric_log} - -默认禁用。 - -**启用** - -要手动开启指标历史数据收集功能 [`system.metric_log`](../../operations/system-tables/metric_log.md),请创建 `/etc/clickhouse-server/config.d/metric_log.xml` 文件,并填入以下内容: - -```xml - - - system - metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**禁用** - -要禁用 `metric_log` 设置,请创建以下文件 `/etc/clickhouse-server/config.d/disable_metric_log.xml`,内容如下: - -```xml - - - -``` - - - - -## replicated_merge_tree {#replicated_merge_tree} - -针对 [ReplicatedMergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 中各表的微调配置。此设置具有更高优先级。 - -有关更多信息,请参阅 MergeTreeSettings.h 头文件。 - -**示例** - -```xml - - 5 - -``` - - -## opentelemetry_span_log {#opentelemetry_span_log} - -[`opentelemetry_span_log`](../system-tables/opentelemetry_span_log.md) 系统表的设置。 - - - -示例: - -```xml - - - engine MergeTree - partition by toYYYYMM(finish_date) - order by (finish_date, finish_time_us, trace_id) - - system - opentelemetry_span_log
- 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## openSSL {#openSSL} - -SSL 客户端/服务器配置。 - -SSL 支持由 `libpoco` 库提供。可用的配置选项详见 [SSLManager.h](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/SSLManager.h)。默认值可在 [SSLManager.cpp](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/src/SSLManager.cpp) 中找到。 - -服务器/客户端设置的键名: - - - -| 选项 | 说明 | 默认值 | -| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | -| `privateKeyFile` | PEM 证书私钥文件的路径。该文件也可以同时包含私钥和证书。 | | -| `certificateFile` | PEM 格式的客户端/服务器证书文件路径。如果 `privateKeyFile` 已包含证书,则可以省略本项。 | | -| `caConfig` | 包含受信任 CA 证书的文件或目录路径。若指向文件,则该文件必须为 PEM 格式,并且可以包含多个 CA 证书。若指向目录,则该目录中每个 CA 证书必须对应一个 .pem 文件。文件名是根据 CA 主体名称的哈希值进行查找的。更多细节请参阅 [SSL_CTX_load_verify_locations](https://www.openssl.org/docs/man3.0/man3/SSL_CTX_load_verify_locations.html) 的 man 手册。 | | -| `verificationMode` | 用于校验节点证书的方法。详细信息请参见 [Context](https://github.com/ClickHouse-Extras/poco/blob/master/NetSSL_OpenSSL/include/Poco/Net/Context.h) 类的描述。可选值:`none`、`relaxed`、`strict`、`once`。 | `relaxed` | -| `verificationDepth` | 验证链的最大长度。如果证书链长度超过该值,则验证将失败。 | `9` | -| `loadDefaultCAFile` | 是否使用 OpenSSL 的内置 CA 证书。ClickHouse 假定 OpenSSL 的内置 CA 证书位于文件 `/etc/ssl/cert.pem`(或对应的目录 `/etc/ssl/certs`)中,或者位于由环境变量 `SSL_CERT_FILE`(或对应的 `SSL_CERT_DIR`)指定的文件(或目录)中。 | `true` | -| `cipherList` | 支持的 OpenSSL 加密算法。 | `ALL:!ADH:!LOW:!EXP:!MD5:!3DES:@STRENGTH` | -| `cacheSessions` | 启用或禁用会话缓存。必须与 `sessionIdContext` 一起使用。可选值:`true`、`false`。 | `false` | -| `sessionIdContext` | 服务器为每个生成的标识符追加的一组唯一的随机字符。该字符串的长度不得超过 `SSL_MAX_SSL_SESSION_ID_LENGTH`。始终建议设置此参数,因为无论是服务器缓存会话还是客户端请求缓存,它都有助于避免出现问题。 | `$\{application.name\}` | -| `sessionCacheSize` | 服务器可缓存的最大会话数量。值为 `0` 表示不限制会话数量。 | [1024*20](https://github.com/ClickHouse/boringssl/blob/master/include/openssl/ssl.h#L1978) | -| `sessionTimeout` | 会话在服务器上的缓存时间(单位:小时)。 | `2` | -| `extendedVerification` | 如果启用此选项,请验证证书的 CN 或 SAN 是否与对端主机名匹配。 | `false` | -| `requireTLSv1` | 是否要求使用 TLSv1 连接。可选值:`true`、`false`。 | `false` | -| `requireTLSv1_1` | 是否要求使用 TLSv1.1 连接。可选值:`true`、`false`。 | `false` | -| `requireTLSv1_2` | 是否要求 TLSv1.2 连接。可选值:`true`、`false`。 | `false` | -| `fips` | 启用 OpenSSL 的 FIPS 模式。仅当该库所使用的 OpenSSL 版本支持 FIPS 时才受支持。 | `false` | -| `privateKeyPassphraseHandler` | 用于请求私钥访问口令的类(PrivateKeyPassphraseHandler 的子类)。例如:``、`KeyFileHandler`、`test`、``。 | `KeyConsoleHandler` | -| `invalidCertificateHandler` | 用于验证无效证书的类(CertificateHandler 的子类)。例如:` RejectCertificateHandler `。 | `RejectCertificateHandler` | -| `disableProtocols` | 禁止使用的协议。 | | -| `preferServerCiphers` | 客户端优先的服务器端密码套件。 | `false` | - -**配置示例:** - -```xml - - - - /etc/clickhouse-server/server.crt - /etc/clickhouse-server/server.key - - /etc/clickhouse-server/dhparam.pem - none - true - true - sslv2,sslv3 - true - - - true - true - sslv2,sslv3 - true - - - - RejectCertificateHandler - - - -``` - - -## part_log {#part_log} - -记录与 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 相关的日志事件,例如添加或合并数据。可以使用该日志来模拟合并算法并比较它们的特性,也可以将合并过程可视化。 - -查询会记录在 [system.part_log](/operations/system-tables/part_log) 表中,而不是单独的文件中。可以通过 `table` 参数(见下文)配置该表的名称。 - - - -**示例** - -```xml - - system - part_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## path {#path} - -包含数据的目录路径。 - -:::note -路径末尾必须带斜杠。 -::: - -**示例** - -```xml -/var/lib/clickhouse/ -``` - - -## processors_profile_log {#processors_profile_log} - -[`processors_profile_log`](../system-tables/processors_profile_log.md) 系统表的相关设置。 - - - -默认设置如下: - -```xml - - system - processors_profile_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## prometheus {#prometheus} - -将指标数据暴露出来,供 [Prometheus](https://prometheus.io) 抓取。 - -设置: - -* `endpoint` – Prometheus 服务器抓取指标的 HTTP 端点。以 '/' 开头。 -* `port` – `endpoint` 使用的端口。 -* `metrics` – 暴露 [system.metrics](/operations/system-tables/metrics) 表中的指标。 -* `events` – 暴露 [system.events](/operations/system-tables/events) 表中的指标。 -* `asynchronous_metrics` – 暴露 [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) 表中的当前指标值。 -* `errors` - 暴露自上次服务器重启以来按错误码统计的错误数量。也可以从 [system.errors](/operations/system-tables/errors) 中获取这类信息。 - -**Example** - -```xml - - 0.0.0.0 - 8123 - 9000 - - - /metrics - 9363 - true - true - true - true - - - -``` - -检查(将 `127.0.0.1` 替换为你的 ClickHouse 服务器 IP 地址或主机名): - -```bash -curl 127.0.0.1:9363/metrics -``` - - -## query_log {#query_log} - -用于在启用 [log_queries=1](../../operations/settings/settings.md) 设置时记录接收到的查询。 - -查询会被记录到 [system.query_log](/operations/system-tables/query_log) 表中,而不是单独的文件。可以在 `table` 参数中更改该表的名称(见下文)。 - - - -如果该表不存在,ClickHouse 会自动创建它。如果在更新 ClickHouse 服务器时查询日志的结构发生了变化,具有旧结构的表会被重命名,并自动创建一个新表。 - -**示例** - -```xml - - system - query_log
- Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_metric_log {#query_metric_log} - -默认为禁用。 - -**启用** - -要手动启用指标历史记录收集功能 [`system.query_metric_log`](../../operations/system-tables/query_metric_log.md),请创建 `/etc/clickhouse-server/config.d/query_metric_log.xml` 文件,内容如下: - -```xml - - - system - query_metric_log
- 7500 - 1000 - 1048576 - 8192 - 524288 - false -
-
-``` - -**禁用** - -要禁用 `query_metric_log` 设置,请创建以下文件 `/etc/clickhouse-server/config.d/disable_query_metric_log.xml`,内容如下: - -```xml - - - -``` - - - - -## query_cache {#query_cache} - -[查询缓存](../query-cache.md) 配置。 - -可用的设置如下: - -| Setting | Description | Default Value | -| ------------------------- | -------------------------------- | ------------- | -| `max_size_in_bytes` | 以字节为单位的最大缓存大小。`0` 表示禁用查询缓存。 | `1073741824` | -| `max_entries` | 在缓存中可存储的 `SELECT` 查询结果的最大数量。 | `1024` | -| `max_entry_size_in_bytes` | 能够被缓存的单个 `SELECT` 查询结果的最大大小(字节)。 | `1048576` | -| `max_entry_size_in_rows` | 能够被缓存的单个 `SELECT` 查询结果的最大行数。 | `30000000` | - -:::note - -* 修改后的设置会立即生效。 -* 查询缓存的数据分配在 DRAM 中。如果内存紧张,请确保为 `max_size_in_bytes` 设置较小的值,或者完全禁用查询缓存。 - ::: - -**示例** - -```xml - - 1073741824 - 1024 - 1048576 - 30000000 - -``` - - -## query_thread_log {#query_thread_log} - -用于在启用 [log_query_threads=1](/operations/settings/settings#log_query_threads) 设置时记录接收查询的线程。 - -查询会记录到 [system.query_thread_log](/operations/system-tables/query_thread_log) 表中,而不是单独的文件中。您可以通过 `table` 参数更改该表的名称(见下文)。 - - - -如果该表不存在,ClickHouse 会创建它。如果在更新 ClickHouse 服务器后查询线程日志的结构发生了变化,则具有旧结构的表会被重命名,并自动创建一个新表。 - -**示例** - -```xml - - system - query_thread_log
- toMonday(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## query_views_log {#query_views_log} - -用于记录视图(live、materialized 等)访问情况的日志设置,受接收到的查询以及 [log_query_views=1](/operations/settings/settings#log_query_views) 设置的影响。 - -查询会被记录到 [system.query_views_log](/operations/system-tables/query_views_log) 表中,而不是写入单独的文件。可以通过 `table` 参数更改该表的名称(见下文)。 - - - -如果该表不存在,ClickHouse 会创建它。如果在更新 ClickHouse 服务器时查询视图日志的结构发生变化,旧结构的表会被重命名,并自动创建一个新表。 - -**示例** - -```xml - - system - query_views_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false -
-``` - - -## text_log {#text_log} - -用于记录文本消息的 [text_log](/operations/system-tables/text_log) 系统表的设置。 - - - -此外: - -| 设置 | 描述 | 默认值 | -| ------- | -------------------------- | ------- | -| `level` | 要写入该表的最大消息级别(默认为 `Trace`)。 | `Trace` | - -**示例** - -```xml - - - notice - system - text_log
- 7500 - 1048576 - 8192 - 524288 - false - - Engine = MergeTree PARTITION BY event_date ORDER BY event_time TTL event_date + INTERVAL 30 day -
-
-``` - - -## trace_log {#trace_log} - -用于 [trace_log](/operations/system-tables/trace_log) 系统表的相关设置。 - - - -默认的服务器配置文件 `config.xml` 包含以下配置段: - -```xml - - system - trace_log
- toYYYYMM(event_date) - 7500 - 1048576 - 8192 - 524288 - false - false -
-``` - - -## asynchronous_insert_log {#asynchronous_insert_log} - -用于配置 [asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) 系统表,以记录异步插入操作的日志。 - - - -**示例** - -```xml - - - system - asynchronous_insert_log
- 7500 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## crash_log {#crash_log} - -[crash_log](../../operations/system-tables/crash_log.md) 系统表相关操作的设置。 - -可以通过以下子标签来配置这些设置: - -| Setting | Description | Default | Note | -| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------ | -| `database` | 数据库名称。 | | | -| `table` | 系统表名称。 | | | -| `engine` | 系统表的 [MergeTree 引擎定义](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-creating-a-table)。 | | 如果已定义 `partition_by` 或 `order_by`,则不能使用该参数。如果未指定,则默认使用 `MergeTree` | -| `partition_by` | 系统表的[自定义分区键](/engines/table-engines/mergetree-family/custom-partitioning-key.md)。 | | 如果为系统表指定了 `engine`,则应在 'engine' 中直接指定 `partition_by` 参数 | -| `ttl` | 指定表的 [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl)。 | | 如果为系统表指定了 `engine`,则应在 'engine' 中直接指定 `ttl` 参数 | -| `order_by` | 系统表的[自定义排序键](/engines/table-engines/mergetree-family/mergetree#order_by)。如果已定义 `engine`,则不能使用该参数。 | | 如果为系统表指定了 `engine`,则应在 'engine' 中直接指定 `order_by` 参数 | -| `storage_policy` | 表所使用的存储策略名称(可选)。 | | 如果为系统表指定了 `engine`,则应在 'engine' 中直接指定 `storage_policy` 参数 | -| `settings` | 控制 MergeTree 行为的[附加参数](/engines/table-engines/mergetree-family/mergetree/#settings)(可选)。 | | 如果为系统表指定了 `engine`,则应在 'engine' 中直接指定 `settings` 参数 | -| `flush_interval_milliseconds` | 将内存中缓冲区的数据刷新到表中的时间间隔。 | `7500` | | -| `max_size_rows` | 日志的最大行数。当未刷新的日志数量达到 `max_size_rows` 时,会将日志写入磁盘。 | `1024` | | -| `reserved_size_rows` | 为日志预分配的内存行数。 | `1024` | | -| `buffer_size_rows_flush_threshold` | 行数阈值。如果达到该阈值,则会在后台触发日志刷盘操作。 | `max_size_rows / 2` | | -| `flush_on_crash` | 设置在发生崩溃时是否应将日志写入磁盘。 | `false` | | - -默认的服务器配置文件 `config.xml` 包含以下设置部分: - -```xml - - system - crash_log
- toYYYYMM(event_date) - 7500 - 1024 - 1024 - 512 - false -
-``` - - -## custom_cached_disks_base_directory {#custom_cached_disks_base_directory} - -此设置用于指定自定义(通过 SQL 创建的)缓存磁盘的缓存路径。 -对于自定义磁盘,`custom_cached_disks_base_directory` 的优先级高于 `filesystem_caches_path`(定义在 `filesystem_caches_path.xml` 中), -如果前者不存在,则使用后者。 -文件系统缓存配置的路径必须位于该目录之内, -否则将抛出异常,阻止磁盘被创建。 - -:::note -这不会影响那些在旧版本中创建、随后升级服务器得到的磁盘。 -在这种情况下,为了允许服务器成功启动,将不会抛出异常。 -::: - -示例: - -```xml -/var/lib/clickhouse/caches/ -``` - - -## backup_log {#backup_log} - -用于 [backup_log](../../operations/system-tables/backup_log.md) 系统表的设置,该系统表用于记录 `BACKUP` 和 `RESTORE` 操作。 - - - -**示例** - -```xml - - - system - backup_log
- 1000 - toYYYYMM(event_date) - 1048576 - 8192 - 524288 - false - -
-
-``` - - -## blob_storage_log {#blob_storage_log} - -[`blob_storage_log`](../system-tables/blob_storage_log.md) 系统表的相关设置。 - - - -示例: - -```xml - - systemblob_storage_logtoYYYYMM(event_date) - 7500event_date + INTERVAL 30 DAY - -``` - - -## query_masking_rules {#query_masking_rules} - -基于正则表达式的规则,在将查询以及所有日志消息写入服务器日志([`system.query_log`](/operations/system-tables/query_log)、[`system.text_log`](/operations/system-tables/text_log)、[`system.processes`](/operations/system-tables/processes) 表)以及发送给客户端之前应用。这样可以防止 SQL 查询中的敏感数据(例如姓名、电子邮件、个人身份标识信息或信用卡号)泄露到日志中。 - -**示例** - -```xml - - - 隐藏 SSN - (^|\D)\d{3}-\d{2}-\d{4}($|\D) - 000-00-0000 - - -``` - -**配置字段**: - -| Setting | Description | -| --------- | ------------------------ | -| `name` | 规则名称(可选) | -| `regexp` | RE2 兼容的正则表达式(必需) | -| `replace` | 用于替换敏感数据的字符串(可选,默认是六个星号) | - -掩码规则会应用到整个查询上(以防止因格式错误 / 无法解析的查询泄露敏感数据)。 - -[`system.events`](/operations/system-tables/events) 表中有计数器 `QueryMaskingRulesMatch`,记录查询掩码规则匹配的总次数。 - -对于分布式查询,每台服务器都必须单独配置,否则传递到其他节点的子查询将会在未进行掩码的情况下被存储。 - - -## remote_servers {#remote_servers} - -用于 [Distributed](../../engines/table-engines/special/distributed.md) 表引擎和 `cluster` 表函数的集群配置。 - -**示例** - -```xml - -``` - -有关 `incl` 属性值,请参阅“[配置文件](/operations/configuration-files)”一节。 - -**另请参阅** - -* [skip_unavailable_shards](../../operations/settings/settings.md#skip_unavailable_shards) -* [集群发现](../../operations/cluster-discovery.md) -* [复制数据库引擎](../../engines/database-engines/replicated.md) - - -## remote_url_allow_hosts {#remote_url_allow_hosts} - -允许在与 URL 相关的存储引擎和表函数中使用的主机列表。 - -在使用 `\` XML 标签添加主机时: - -* 必须与 URL 中的写法完全一致地指定,因为在 DNS 解析之前会先对名称进行检查。例如:`clickhouse.com` -* 如果在 URL 中显式指定了端口,则会将 host:port 作为整体进行检查。例如:`clickhouse.com:80` -* 如果主机未带端口进行指定,则该主机上的任意端口都被允许。例如:如果指定了 `clickhouse.com`,则 `clickhouse.com:20`(FTP)、`clickhouse.com:80`(HTTP)、`clickhouse.com:443`(HTTPS)等都是允许的。 -* 如果主机被指定为 IP 地址,则会按 URL 中的写法进行检查。例如:`[2a02:6b8:a::a]`。 -* 如果存在重定向且已启用重定向支持,则每一次重定向(location 字段)都会被检查。 - -例如: - -```sql - - clickhouse.com - -``` - - -## timezone {#timezone} - -服务器的时区。 - -指定为表示 UTC 时区或地理位置的 IANA 标识符(例如,Africa/Abidjan)。 - -在 String 与 DateTime 格式之间进行转换时需要使用时区:当以文本格式输出 DateTime 字段(打印到屏幕或写入文件),以及从字符串解析 DateTime 时,都需要依赖时区。除此之外,对于处理时间和日期的函数,如果其输入参数中未显式传入时区,也会使用该时区。 - -**示例** - -```xml -Asia/Istanbul -``` - -**另请参见** - -* [session_timezone](../settings/settings.md#session_timezone) - - -## tcp_port {#tcp_port} - -用于与客户端进行 TCP 协议通信的端口。 - -**示例** - -```xml -9000 -``` - - -## tcp_port_secure {#tcp_port_secure} - -用于与客户端进行安全通信的 TCP 端口。应配合 [OpenSSL](#openssl) 设置一起使用。 - -**默认值** - -```xml -9440 -``` - - -## mysql_port {#mysql_port} - -用于通过 MySQL 协议与客户端进行通信的端口。 - -:::note - -* 正整数表示要监听的端口号 -* 留空则表示禁用通过 MySQL 协议与客户端的通信。 - ::: - -**示例** - -```xml -9004 -``` - - -## postgresql_port {#postgresql_port} - -用于通过 PostgreSQL 协议与客户端进行通信的端口。 - -:::note - -* 正整数表示要监听的端口号 -* 空值表示禁用通过 PostgreSQL 协议与客户端的通信 - ::: - -**示例** - -```xml -9005 -``` - - -## mysql_require_secure_transport {#mysql_require_secure_transport} - -如果设置为 true,则要求通过 [mysql_port](#mysql_port) 与客户端进行安全通信。带有 `--ssl-mode=none` 选项的连接将被拒绝。应与 [OpenSSL](#openssl) 相关设置配合使用。 - - - -## postgresql_require_secure_transport {#postgresql_require_secure_transport} - -当设置为 true 时,要求通过 [postgresql_port](#postgresql_port) 与客户端进行安全通信。带有 `sslmode=disable` 选项的连接将被拒绝。请与 [OpenSSL](#openssl) 相关设置配合使用。 - - - -## tmp_path {#tmp_path} - -本地文件系统上用于存储大查询处理中临时数据的路径。 - -:::note - -* 临时数据存储只能从以下选项中选择一个进行配置:`tmp_path`、`tmp_policy`、`temporary_data_in_cache`。 -* 路径末尾必须包含斜杠。 - ::: - -**示例** - -```xml -/var/lib/clickhouse/tmp/ -``` - - -## url_scheme_mappers {#url_scheme_mappers} - -用于将简写或符号化的 URL 前缀映射为完整 URL 的配置。 - -示例: - -```xml - - - https://{bucket}.s3.amazonaws.com - - - https://storage.googleapis.com/{bucket} - - - https://{bucket}.oss.aliyuncs.com - - -``` - - -## user_files_path {#user_files_path} - -用户文件所在的目录。用于表函数 [file()](../../sql-reference/table-functions/file.md)、[fileCluster()](../../sql-reference/table-functions/fileCluster.md)。 - -**示例** - -```xml -/var/lib/clickhouse/user_files/ -``` - - -## user_scripts_path {#user_scripts_path} - -用户脚本所在的目录。供可执行用户定义函数(Executable User Defined Functions)使用,参见 [Executable User Defined Functions](/sql-reference/functions/udf#executable-user-defined-functions)。 - -**示例** - -```xml -/var/lib/clickhouse/user_scripts/ -``` - -类型: - -默认值: - - -## user_defined_path {#user_defined_path} - -用于存放用户定义文件的目录。供 SQL 用户定义函数使用,详见 [SQL 用户定义函数](/sql-reference/functions/udf)。 - -**示例** - -```xml -/var/lib/clickhouse/user_defined/ -``` - - -## users_config {#users_config} - -包含以下内容的文件的路径: - -* 用户配置。 -* 访问权限。 -* 设置配置文件。 -* 配额设置。 - -**示例** - -```xml -users.xml -``` - - -## access_control_improvements {#access_control_improvements} - -访问控制系统可选增强功能的相关设置。 - -| Setting | Description | Default | -| ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | -| `users_without_row_policies_can_read_rows` | 设置没有宽松行策略的用户是否仍然可以通过 `SELECT` 查询读取行。例如,如果有两个用户 A 和 B,并且只为 A 定义了行策略,那么当此设置为 true 时,用户 B 将看到所有行;当此设置为 false 时,用户 B 将看不到任何行。 | `true` | -| `on_cluster_queries_require_cluster_grant` | 设置 `ON CLUSTER` 查询是否需要 `CLUSTER` 授权。 | `true` | -| `select_from_system_db_requires_grant` | 设置 `SELECT * FROM system.` 是否需要任何权限,以及是否可由任意用户执行。如果设置为 true,则该查询需要 `GRANT SELECT ON system.
`,与非 system 表相同。例外情况:少数几个 system 表(`tables`、`columns`、`databases`,以及一些常量表,如 `one`、`contributors`)仍然对所有人可访问;并且如果授予了某个 `SHOW` 权限(例如 `SHOW USERS`),则相应的 system 表(即 `system.users`)将可访问。 | `true` | -| `select_from_information_schema_requires_grant` | 设置 `SELECT * FROM information_schema.
` 是否需要任何权限,以及是否可由任意用户执行。如果设置为 true,则此查询需要 `GRANT SELECT ON information_schema.
`,与普通表相同。 | `true` | -| `settings_constraints_replace_previous` | 设置配置文件中针对某个设置的约束,是否会覆盖该设置上先前的约束(在其他配置文件中定义),包括那些未被新约束显式设置的字段。它还会启用 `changeable_in_readonly` 约束类型。 | `true` | -| `table_engines_require_grant` | 设置在使用特定表引擎创建表时是否需要授权。 | `false` | -| `role_cache_expiration_time_seconds` | 设置自上次访问以来角色在 Role Cache 中保留的时间(以秒为单位)。 | `600` | - -Example: - -```xml - - true - true - true - true - true - false - 600 - -``` - - -## s3queue_log {#s3queue_log} - -用于 `s3queue_log` 系统表的设置。 - - - -默认设置如下: - -```xml - - system -
s3queue_log
- toYYYYMM(event_date) - 7500 - -``` - - -## dead_letter_queue {#dead_letter_queue} - -`dead_letter_queue` 系统表的设置。 - - - -默认设置为: - -```xml - - system - dead_letter
- toYYYYMM(event_date) - 7500 -
-``` - - -## zookeeper {#zookeeper} - -包含允许 ClickHouse 与 [ZooKeeper](http://zookeeper.apache.org/) 集群交互的设置。ClickHouse 在使用复制表(replicated tables)时,会使用 ZooKeeper 存储副本的元数据。如果不使用复制表,可以省略本节参数。 - -以下设置可以通过子标签进行配置: - -| Setting | Description | -| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | -| `node` | ZooKeeper 端点。可以设置多个端点。例如:`example_host2181`。`index` 属性指定在尝试连接 ZooKeeper 集群时节点的顺序。 | -| `session_timeout_ms` | 客户端会话的最大超时时间,单位为毫秒。 | -| `operation_timeout_ms` | 单个操作的最大超时时间,单位为毫秒。 | -| `root` (optional) | ClickHouse 服务器用于其 znodes 的根 znode。 | -| `fallback_session_lifetime.min` (optional) | 当主节点不可用时(负载均衡场景),到回退节点的 ZooKeeper 会话生命周期的最小限制。单位为秒。默认值:3 小时。 | -| `fallback_session_lifetime.max` (optional) | 当主节点不可用时(负载均衡场景),到回退节点的 ZooKeeper 会话生命周期的最大限制。单位为秒。默认值:6 小时。 | -| `identity` (optional) | 访问目标 znodes 时 ZooKeeper 所需的用户名和密码。 | -| `use_compression` (optional) | 若设为 true,则在 Keeper 协议中启用压缩。 | - -还有一个可选设置 `zookeeper_load_balancing`,用于选择 ZooKeeper 节点的负载均衡算法: - -| Algorithm Name | Description | -| ------------------------------- | ------------------------------------------------- | -| `random` | 随机选择一个 ZooKeeper 节点。 | -| `in_order` | 选择第一个 ZooKeeper 节点,如果不可用则选择第二个,依此类推。 | -| `nearest_hostname` | 选择主机名与服务器主机名最相似的 ZooKeeper 节点,主机名根据名称前缀进行比较。 | -| `hostname_levenshtein_distance` | 与 `nearest_hostname` 类似,但以 Levenshtein 距离方式比较主机名。 | -| `first_or_random` | 选择第一个 ZooKeeper 节点,如果不可用则从剩余 ZooKeeper 节点中随机选择一个。 | -| `round_robin` | 选择第一个 ZooKeeper 节点,如果发生重连,则选择下一个。 | - -**配置示例** - -```xml - - - example1 - 2181 - - - example2 - 2181 - - 30000 - 10000 - - /path/to/zookeeper/node - - user:password - - random - -``` - -**另请参阅** - -* [复制](../../engines/table-engines/mergetree-family/replication.md) -* [ZooKeeper 程序员指南](http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html) -* [ClickHouse 与 ZooKeeper 之间的可选安全通信](/operations/ssl-zookeeper) - - -## use_minimalistic_part_header_in_zookeeper {#use_minimalistic_part_header_in_zookeeper} - -在 ZooKeeper 中存储数据分片(data part)头部的方式。此设置仅适用于 [`MergeTree`](/engines/table-engines/mergetree-family) 系列表引擎。可以通过以下方式指定: - -**在 `config.xml` 文件的 [merge_tree](#merge_tree) 部分中进行全局设置** - -ClickHouse 会对服务器上的所有表使用该设置。可以随时更改这一设置。现有表在设置变更后会改变其行为。 - -**对每个表进行单独设置** - -在创建表时,指定相应的 [engine setting](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。已有表在创建时配置了该设置后,其行为不会因全局设置的改变而发生变化。 - -**可能的取值** - -- `0` — 功能关闭。 -- `1` — 功能开启。 - -如果 [`use_minimalistic_part_header_in_zookeeper = 1`](#use_minimalistic_part_header_in_zookeeper),则 [replicated](../../engines/table-engines/mergetree-family/replication.md) 表会使用单个 `znode` 以紧凑方式存储数据分片头部。如果表包含大量列,这种存储方式可以显著减少在 ZooKeeper 中存储的数据量。 - -:::note -在应用 `use_minimalistic_part_header_in_zookeeper = 1` 之后,无法将 ClickHouse 服务器降级到不支持该设置的版本。在集群中的服务器上升级 ClickHouse 时请谨慎操作。不要一次性升级所有服务器。更安全的做法是在测试环境或集群中的少量服务器上先测试新版本的 ClickHouse。 - -已经使用此设置存储的数据分片头部无法恢复为之前的(非紧凑)表示形式。 -::: - - - -## distributed_ddl {#distributed_ddl} - -用于管理在集群上执行[分布式 DDL 查询](../../sql-reference/distributed-ddl.md)(`CREATE`、`DROP`、`ALTER`、`RENAME`)。 -仅在启用 [ZooKeeper](/operations/server-configuration-parameters/settings#zookeeper) 时生效。 - -`` 中可配置的参数包括: - -| Setting | Description | Default Value | -| ---------------------- | ------------------------------------------------------- | ------------------------- | -| `path` | Keeper 中用于 DDL 查询 `task_queue` 的路径 | | -| `profile` | 用于执行 DDL 查询的 profile | | -| `pool_size` | 可同时运行的 `ON CLUSTER` 查询数量 | | -| `max_tasks_in_queue` | 队列中可存在的最大任务数量。 | `1,000` | -| `task_max_lifetime` | 如果节点存在时间超过该值则会被删除。 | `7 * 24 * 60 * 60`(一周的秒数) | -| `cleanup_delay_period` | 如果距离上次清理已过去至少 `cleanup_delay_period` 秒,则在接收到新节点事件后开始清理。 | `60` 秒 | - -**示例** - -```xml - - - /clickhouse/task_queue/ddl - - - default - - - 1 - - - - - 604800 - - - 60 - - - 1000 - -``` - - -## access_control_path {#access_control_path} - -ClickHouse 服务器用于存储通过 SQL 命令创建的用户和角色配置的文件夹路径。 - -**另请参阅** - -- [访问控制和账户管理](/operations/access-rights#access-control-usage) - - - -## allow_plaintext_password {#allow_plaintext_password} - -设置是否允许使用明文密码类型(不安全)。 - -```xml -1 -``` - - -## allow_no_password {#allow_no_password} - -设置是否允许使用不安全的 `no_password` 密码类型。 - -```xml -1 -``` - - -## allow_implicit_no_password {#allow_implicit_no_password} - -禁止在未显式指定 'IDENTIFIED WITH no_password' 的情况下创建无密码用户。 - -```xml -1 -``` - - -## default_session_timeout {#default_session_timeout} - -默认会话超时时间(秒)。 - -```xml -60 -``` - - -## default_password_type {#default_password_type} - -设置在类似 `CREATE USER u IDENTIFIED BY 'p'` 这样的查询中自动设置的密码类型。 - -可接受的值为: - -* `plaintext_password` -* `sha256_password` -* `double_sha1_password` -* `bcrypt_password` - -```xml -sha256_password -``` - - -## user_directories {#user_directories} - -配置文件中包含以下设置的部分: - -* 预定义用户配置文件的路径。 -* 通过 SQL 命令创建的用户所存储的文件夹路径。 -* 通过 SQL 命令创建并进行复制的用户在 ZooKeeper 中存储的节点路径。 - -如果指定了此部分,则不会使用 [users_config](/operations/server-configuration-parameters/settings#users_config) 和 [access_control_path](../../operations/server-configuration-parameters/settings.md#access_control_path) 中的路径。 - -`user_directories` 部分可以包含任意数量的条目,条目的顺序表示其优先级(位置越靠前优先级越高)。 - -**示例** - -```xml - - - /etc/clickhouse-server/users.xml - - - /var/lib/clickhouse/access/ - - -``` - -用户、角色、行级策略、配额和设置配置文件也可以存储在 ZooKeeper 中: - -```xml - - - /etc/clickhouse-server/users.xml - - - /clickhouse/access/ - - -``` - -你还可以定义 `memory` 节 —— 表示仅在内存中存储信息,不写入磁盘,以及 `ldap` 节 —— 表示在 LDAP 服务器上存储信息。 - -要将 LDAP 服务器添加为远程用户目录,用于那些未在本地定义的用户,请定义一个单独的 `ldap` 节,并使用以下设置: - -| Setting | Description | -| -------- | --------------------------------------------------------------------------------------------------------------------- | -| `server` | 在 `ldap_servers` 配置节中定义的某个 LDAP 服务器名称。此参数为必填项,不能为空。 | -| `roles` | 包含本地定义角色列表的节,这些角色将被分配给从 LDAP 服务器获取的每个用户。如果未指定任何角色,用户在通过身份验证后将无法执行任何操作。如果在身份验证时,列出的任意角色在本地尚未定义,则身份验证尝试将失败,就像提供了错误密码一样。 | - -**示例** - -```xml - - my_ldap_server - - - - - -``` - - -## top_level_domains_list {#top_level_domains_list} - -定义要添加的自定义顶级域名列表,其中每个条目的格式为 `/path/to/file`。 - -例如: - -```xml - - /path/to/public_suffix_list.dat - -``` - -另请参阅: - -* 函数 [`cutToFirstSignificantSubdomainCustom`](../../sql-reference/functions/url-functions.md/#cutToFirstSignificantSubdomainCustom) 及其变体, - 它接受一个自定义 TLD 列表的名称,并返回域名中包含顶级子域在内、直到第一个重要子域名的那一部分。 - - -## proxy {#proxy} - -为 HTTP 和 HTTPS 请求定义代理服务器,目前 S3 存储、S3 表函数以及 URL 函数支持该功能。 - -定义代理服务器有三种方式: - -* 环境变量 -* 代理列表 -* 远程代理解析器。 - -还可以通过使用 `no_proxy` 为特定主机绕过代理服务器。 - -**环境变量** - -通过环境变量 `http_proxy` 和 `https_proxy` 可以为相应协议指定 -代理服务器。如果已在系统中进行了配置,将会自动生效并无缝工作。 - -如果某个协议只有一个代理服务器且该代理服务器不会变化, -这是最简单的方式。 - -**代理列表** - -这种方式允许为某个协议指定一个或多个 -代理服务器。如果定义了多个代理服务器, -ClickHouse 会以轮询(round-robin)的方式使用不同的代理,以在服务器之间平衡 -负载。如果某个协议有多个代理服务器,并且代理服务器列表不会变化,这是最简单的方式。 - -**配置模板** - -```xml - - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - -在下方的选项卡中选择一个父字段以查看其子字段: - - - - | 字段 | 描述 | - | --------- | ----------------- | - | `` | 一个或多个 HTTP 代理的列表 | - | `` | 一个或多个 HTTPS 代理的列表 | - - - - | 字段 | 描述 | - | ------- | ------- | - | `` | 代理的 URI | - - - -**远程代理解析器** - -代理服务器可能会动态变化。在这种情况下,可以定义解析器的端点(endpoint)。ClickHouse 会向该端点发送一个空的 GET 请求,远程解析器应返回代理主机。 -ClickHouse 将使用以下模板构造代理 URI:`\{proxy_scheme\}://\{proxy_host\}:{proxy_port}` - -**配置模板** - -```xml - - - - http://resolver:8080/hostname - http - 80 - 10 - - - - - - http://resolver:8080/hostname - http - 3128 - 10 - - - - -``` - -在下方选项卡中选择一个父级字段以查看其子级字段: - - - - | 字段 | 描述 | - | --------- | ------------------- | - | `` | 一个或多个 resolver 的列表* | - | `` | 一个或多个 resolver 的列表* | - - - - | 字段 | 描述 | - | ------------ | -------------------------------- | - | `` | resolver 的端点以及该 resolver 的其他详细信息 | - - :::note - 您可以定义多个 `` 元素,但对于给定协议,只会使用第一个 - ``。对于该协议的其他任意 `` - 元素都会被忽略。这意味着(如有需要)负载均衡应由远程 resolver 实现。 - ::: - - - - | 字段 | 描述 | - | -------------------- | ------------------------------------------------------------------------------------------------- | - | `` | 代理 resolver 的 URI | - | `` | 最终代理 URI 的协议。可以是 `http` 或 `https`。 | - | `` | 代理 resolver 的端口号 | - | `` | ClickHouse 应该将来自 resolver 的值缓存的时间(秒)。将该值设置为 `0` 会导致 ClickHouse 针对每个 HTTP 或 HTTPS 请求都联系该 resolver。 | - - - -**优先级** - -代理设置按以下顺序确定: - - -| 顺序 | 设置 | -|------|--------------------------| -| 1. | 远程代理解析器 | -| 2. | 代理列表 | -| 3. | 环境变量 | - -ClickHouse 会根据请求协议,先检查最高优先级的解析器类型。若未定义, -则会检查优先级次高的解析器类型,直到检查到环境变量解析器为止。 -因此也可以混合使用多种解析器类型。 - - - -## disable_tunneling_for_https_requests_over_http_proxy {#disable_tunneling_for_https_requests_over_http_proxy} - -默认情况下,会使用隧道(即 `HTTP CONNECT`)通过 `HTTP` 代理发起 `HTTPS` 请求。可以通过此设置禁用该行为。 - -**no_proxy** - -默认情况下,所有请求都会经过代理。若要对特定主机禁用代理,必须设置 `no_proxy` 变量。 -对于 list 和 remote 解析器,可以在 `` 子句中进行设置;对于 environment 解析器,可以通过环境变量进行设置。 -它支持 IP 地址、域名、子域名以及用于完全绕过的通配符 `'*'`。开头的点号会被去除,其行为与 curl 一致。 - -**Example** - -下面的配置会绕过对 `clickhouse.cloud` 以及其所有子域名(例如 `auth.clickhouse.cloud`)的代理请求。 -GitLab 也是如此,即使它前面带有一个点号。`gitlab.com` 和 `about.gitlab.com` 都会绕过代理。 - -```xml - - clickhouse.cloud,.gitlab.com - - http://proxy1 - http://proxy2:3128 - - - http://proxy1:3128 - - -``` - - -## workload_path {#workload_path} - -作为所有 `CREATE WORKLOAD` 和 `CREATE RESOURCE` 查询存储位置的目录。默认情况下,使用服务器工作目录下的 `/workload/` 文件夹。 - -**示例** - -```xml -/var/lib/clickhouse/workload/ -``` - -**另请参阅** - -* [工作负载层次结构](/operations/workload-scheduling.md#workloads) -* [workload_zookeeper_path](#workload_zookeeper_path) - - -## workload_zookeeper_path {#workload_zookeeper_path} - -指向 ZooKeeper 节点的路径,用作所有 `CREATE WORKLOAD` 和 `CREATE RESOURCE` 查询的存储位置。为保持一致性,所有 SQL 定义都会作为同一个 znode 的值进行存储。默认情况下不使用 ZooKeeper,而是将定义存储在[磁盘](#workload_path)上。 - -**示例** - -```xml -/clickhouse/workload/definitions.sql -``` - -**另请参阅** - -* [工作负载层次结构](/operations/workload-scheduling.md#workloads) -* [workload_path](#workload_path) - - -## zookeeper_log {#zookeeper_log} - -[`zookeeper_log`](/operations/system-tables/zookeeper_log) 系统表的设置。 - -以下设置可以通过子标签进行配置: - - - -**示例** - -```xml - - - system - zookeeper_log
- 7500 - event_date + INTERVAL 1 WEEK DELETE -
-
-``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md deleted file mode 100644 index f1b21e48225..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/_snippets/_system-log-parameters.md +++ /dev/null @@ -1,17 +0,0 @@ -以下设置可以通过子标签进行配置: - -| Setting | Description | Default | Note | -|------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|--------------------------------------------------------------------------------------------------------------------| -| `database` | 数据库名称。 | | | -| `table` | 系统表名称。 | | | -| `engine` | 系统表的 [MergeTree 引擎定义](../../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-creating-a-table)。 | | 如果已定义 `partition_by` 或 `order_by`,则不能使用该设置。如果未指定,则默认选择 `MergeTree` | -| `partition_by` | 系统表的[自定义分区键](../../../engines/table-engines/mergetree-family/custom-partitioning-key.md)。 | | 如果为系统表指定了 `engine`,则应在 `engine` 内部直接指定 `partition_by` 参数 | -| `ttl` | 指定表的 [TTL](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-ttl)。 | | 如果为系统表指定了 `engine`,则应在 `engine` 内部直接指定 `ttl` 参数 | -| `order_by` | 系统表的[自定义排序键](../../../engines/table-engines/mergetree-family/mergetree.md#order_by)。如果已定义 `engine`,则不能使用该设置。 | | 如果为系统表指定了 `engine`,则应在 `engine` 内部直接指定 `order_by` 参数 | -| `storage_policy` | 要用于该表的存储策略名称(可选)。 | | 如果为系统表指定了 `engine`,则应在 `engine` 内部直接指定 `storage_policy` 参数 | -| `settings` | 控制 MergeTree 行为的[额外参数](../../../engines/table-engines/mergetree-family/mergetree.md/#settings)(可选)。 | | 如果为系统表指定了 `engine`,则应在 `engine` 内部直接指定 `settings` 参数 | -| `flush_interval_milliseconds` | 将内存缓冲区中的数据刷新到表的时间间隔。 | `7500` | | -| `max_size_rows` | 日志的最大行数。当未刷新的日志数量达到最大值时,日志会被写入磁盘。 | `1048576` | | -| `reserved_size_rows` | 为日志预分配的内存行数。 | `8192` | | -| `buffer_size_rows_flush_threshold` | 行数阈值。如果达到该阈值,则会在后台触发将日志刷新到磁盘的操作。 | `max_size_rows / 2` | | -| `flush_on_crash` | 设置在发生崩溃时是否应将日志写入磁盘。 | `false` | | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md deleted file mode 100644 index 7598e7124c5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/server-configuration-parameters/settings.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -description: '本节介绍服务器级设置,即那些不能在会话或查询级别修改的设置。' -keywords: ['全局服务器设置'] -sidebar_label: '服务器设置' -sidebar_position: 57 -slug: /operations/server-configuration-parameters/settings -title: '服务器设置' -doc_type: 'reference' ---- - -{/* 注意:本文件中的设置为自动生成 - 更多信息请参见:["从源代码生成文档"](https://github.com/ClickHouse/clickhouse-docs/blob/main/contribute/autogenerated-documentation-from-source.md) - */ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md deleted file mode 100644 index 13abbb7a369..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/composable-protocols.md +++ /dev/null @@ -1,188 +0,0 @@ ---- -description: '组合式协议允许以更灵活的方式配置对 ClickHouse 服务器的 TCP 访问。' -sidebar_label: '组合式协议' -sidebar_position: 64 -slug: /operations/settings/composable-protocols -title: '组合式协议' -doc_type: 'reference' ---- - -# 组合式协议 {#composable-protocols} - -## 概览 {#overview} - -组合式协议允许更灵活地配置对 ClickHouse 服务器的 TCP 访问。此类配置可以与传统配置并存,也可以替代传统配置。 - -## 配置可组合协议 {#composable-protocols-section-is-denoted-as-protocols-in-configuration-xml} - -可组合协议可以通过 XML 配置文件进行配置。协议部分在 XML 配置文件中由 `protocols` 标签标识: - -```xml - - - -``` - - -### 配置协议层 {#basic-modules-define-protocol-layers} - -可以使用基本模块(basic modules)来定义协议层。例如,要定义一个 -HTTP 层,可以在 `protocols` 配置节中添加一个新的基本模块(basic module): - -```xml - - - - - http - - - -``` - -模块可以按如下方式配置: - -* `plain_http` - 名称,可被其他层引用 -* `type` - 表示用于处理数据、将被实例化的协议处理器。 - 它具有以下预定义的协议处理器集合: - * `tcp` - ClickHouse 原生协议处理器 - * `http` - HTTP ClickHouse 协议处理器 - * `tls` - TLS 加密层 - * `proxy1` - PROXYv1 层 - * `mysql` - MySQL 兼容协议处理器 - * `postgres` - PostgreSQL 兼容协议处理器 - * `prometheus` - Prometheus 协议处理器 - * `interserver` - ClickHouse 服务器间通信处理器 - -:::note -`gRPC` 协议处理器尚未在 `Composable protocols` 中实现。 -::: - - -### 配置端点 {#endpoint-ie-listening-port-is-denoted-by-port-and-optional-host-tags} - -端点(监听端口)由 `` 标签和可选的 `` 标签表示。 -例如,要在先前添加的 HTTP 层上配置一个端点, -可以按如下方式修改配置: - -```xml - - - - - http - - 127.0.0.1 - 8123 - - - - -``` - -如果省略 `` 标签,则会使用根级配置中的 ``。 - - -### 配置层级序列 {#layers-sequence-is-defined-by-impl-tag-referencing-another-module} - -层级序列是通过使用 `` 标签并引用另一个模块来定义的。例如,要在我们的 plain_http 模块之上配置一个 TLS 层,可以进一步按如下方式修改配置: - -```xml - - - - - http - - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### 将端点关联到层 {#endpoint-can-be-attached-to-any-layer} - -端点可以关联到任意层。例如,我们可以为 HTTP(端口 8123)和 HTTPS(端口 8443)定义端点: - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - -``` - - -### 定义附加端点 {#additional-endpoints-can-be-defined-by-referencing-any-module-and-omitting-type-tag} - -可以通过引用任意模块并省略 `` 标签来定义附加端点。比如,我们可以为 `plain_http` 模块定义 `another_http` 端点,如下所示: - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - - - - plain_http - 127.0.0.1 - 8223 - - - -``` - - -### 指定额外层参数 {#some-modules-can-contain-specific-for-its-layer-parameters} - -某些模块可以包含额外的层参数。例如,TLS 层 -允许按如下方式指定私钥(`privateKeyFile`)和证书文件(`certificateFile`): - -```xml - - - - http - 127.0.0.1 - 8123 - - - - tls - plain_http - 127.0.0.1 - 8443 - another_server.key - another_server.crt - - - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md deleted file mode 100644 index c4be4eb500c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/constraints-on-settings.md +++ /dev/null @@ -1,183 +0,0 @@ ---- -description: '可以在 `user.xml` 配置文件的 `profiles` 部分为某些设置定义约束条件,从而禁止用户通过 `SET` 查询更改这些设置。' -sidebar_label: '设置约束条件' -sidebar_position: 62 -slug: /operations/settings/constraints-on-settings -title: '设置约束条件' -doc_type: 'reference' ---- - - - -# 配置约束 {#constraints-on-settings} - - - -## 概览 {#overview} - -在 ClickHouse 中,设置上的“约束”是指可以应用于这些设置的限制和规则。这些约束可以帮助维护数据库的稳定性、安全性以及行为的可预测性。 - - - -## 定义约束 {#defining-constraints} - -可以在 `user.xml` 配置文件的 `profiles` 部分中定义设置约束。它们会禁止用户通过 -[`SET`](/sql-reference/statements/set) 语句更改某些设置。 - -约束的定义方式如下: - -```xml - - - - - lower_boundary - - - upper_boundary - - - lower_boundary - upper_boundary - - - - - - lower_boundary - upper_boundary - - - - lower_boundary - upper_boundary - value1 - value2 - value3 - - - - - -``` - -如果用户尝试违反这些约束,则会抛出异常,且该设置保持不变。 - - -## 约束类型 {#types-of-constraints} - -ClickHouse 中支持以下几种类型的约束: - -* `min` -* `max` -* `disallowed` -* `readonly`(别名为 `const`) -* `changeable_in_readonly` - -`min` 和 `max` 约束为数值型设置指定上下界,并且可以组合使用。 - -`disallowed` 约束可用于指定某个设置不允许使用的特定取值(或取值集合)。 - -`readonly` 或 `const` 约束表示用户完全不能修改对应的设置。 - -`changeable_in_readonly` 约束类型允许用户在 `min`/`max` 范围内修改设置,即使 `readonly` 设置为 `1`;否则在 `readonly=1` 模式下将不允许修改该设置。 - -:::note -仅当启用了 `settings_constraints_replace_previous` 时,才支持 `changeable_in_readonly`: - -```xml - - true - -``` - -::: - - -## 多个约束配置文件 {#multiple-constraint-profiles} - -如果某个用户同时有多个配置文件生效,这些约束会被合并。 -合并行为取决于 `settings_constraints_replace_previous`: -- **true**(推荐):在合并时,相同设置的约束会被替换,后面的约束生效,之前的全部被忽略。 - 这也包括那些在新约束中未设置的字段(不会从之前的约束中继承)。 -- **false**(默认):在合并时,相同设置的约束会按以下方式处理: - 所有未设置的约束类型从之前的配置文件中继承,所有已设置的约束类型则被新配置文件中的值所替换。 - - - -## 只读模式 {#read-only} - -只读模式通过 `readonly` 设置启用,不要将其与 `readonly` 约束类型混淆: - -* `readonly=0`:无只读限制。 -* `readonly=1`:只允许执行只读查询,且无法更改设置, - 除非设置了 `changeable_in_readonly`。 -* `readonly=2`:只允许执行只读查询,但可以更改设置, - `readonly` 设置本身除外。 - -### 示例 {#example-read-only} - -假设 `users.xml` 包含以下几行: - -```xml - - - 10000000000 - 0 - ... - - - 5000000000 - 20000000000 - - - - - - - -``` - -以下查询都会抛出异常: - -```sql -SET max_memory_usage=20000000001; -SET max_memory_usage=4999999999; -SET force_index_by_date=1; -``` - -```text -Code: 452, e.displayText() = DB::Exception: 设置 max_memory_usage 不得大于 20000000000。 -Code: 452, e.displayText() = DB::Exception: 设置 max_memory_usage 不得小于 5000000000。 -Code: 452, e.displayText() = DB::Exception: 设置 force_index_by_date 不得更改。 -``` - -:::note -`default` 配置文件有特殊处理方式:为 `default` 配置文件定义的所有约束都会成为默认约束,因此它们会对所有用户生效,直到为这些用户显式设置了新的约束将其覆盖为止。 -::: - - -## 对 MergeTree 设置的约束 {#constraints-on-merge-tree-settings} - -可以为 [MergeTree 设置](merge-tree-settings.md) 定义约束。 -在创建使用 MergeTree 引擎的表, -或修改其存储设置时,这些约束会被应用。 - -在 `` 部分中引用 MergeTree 设置名称时, -必须在其前面加上 `merge_tree_` 前缀。 - -### 示例 {#example-mergetree} - -可以禁止创建显式指定 `storage_policy` 的新表。 - -```xml - - - - - - - - - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/index.md deleted file mode 100644 index f6f01187e89..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: '“设置”目录页' -sidebar_position: 1 -slug: /operations/settings/ -title: '设置' -doc_type: 'landing-page' ---- - -{/* 本页的目录由 - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 根据 YAML front matter 中的 slug、description、title 字段自动生成。 - - 如果你发现错误,请直接编辑各页面本身的 YML frontmatter。 - */ } - -{/*AUTOGENERATED_START*/ } - -| 页面 | 描述 | -| ------------------------------------------------------------------------- | ------------------------------------------------------------------ | -| [Settings Overview](/operations/settings/overview) | 设置总览页面。 | -| [Permissions for queries](/operations/settings/permissions-for-queries) | 查询权限相关的设置。 | -| [Restrictions on query complexity](/operations/settings/query-complexity) | 限制查询复杂度的设置。 | -| [Settings profiles](/operations/settings/settings-profiles) | 在同一名称下分组的一组设置。 | -| [Constraints on settings](/operations/settings/constraints-on-settings) | 可以在 `user.xml` 配置文件的 `profiles` 部分中定义设置约束,从而禁止用户通过 `SET` 查询更改某些设置。 | -| [Users and roles settings](/operations/settings/settings-users) | 用于配置用户和角色的设置。 | -| [Composable protocols](/operations/settings/composable-protocols) | 可组合协议允许对 ClickHouse 服务器的 TCP 访问进行更灵活的配置。 | -| [Format Settings](/operations/settings/formats) | 与格式相关的设置。 | -| [Memory overcommit](/operations/settings/memory-overcommit) | 一种实验性技术,旨在允许为查询设置更灵活的内存限制。 | -| [MergeTree tables settings](/operations/settings/merge-tree-settings) | 针对 MergeTree 的设置,这些设置位于 `system.merge_tree_settings` 中。 | -| [Query-level Session Settings](/operations/settings/query-level) | 查询级别的会话设置。 | -| [Server overload](/operations/settings/server-overload) | 控制服务器 CPU 过载时的行为。 | -| [Session Settings](/operations/settings/settings) | 在 system.settings 中定义的会话设置。 | -| [TCP connection limits](/operations/settings/tcp-connection-limits) | TCP 连接限制。 | - -{/*AUTOGENERATED_START*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md deleted file mode 100644 index 9e58f5e3a75..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/memory-overcommit.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '一种实验性技术,旨在为查询设置更灵活的内存限制。' -slug: /operations/settings/memory-overcommit -title: '内存超量分配' -doc_type: 'reference' ---- - - - -# 内存超额分配(memory overcommit) {#memory-overcommit} - -内存超额分配是一种实验性机制,旨在为查询设置更灵活的内存限制。 - -该机制的核心思想是通过一组设置来表示查询可以使用的“保证内存”数量。 -当启用了内存超额分配且达到内存限制时,ClickHouse 会选择超额分配程度最高的查询,并尝试通过终止该查询来释放内存。 - -当达到内存限制时,任何查询在尝试分配新内存时都会先等待一段时间。 -如果在超时时间内有内存被释放,查询将继续执行。 -否则将抛出异常并终止该查询。 - -要停止或终止的查询由全局或用户级 overcommit 跟踪器选择,具体取决于触发的是哪种内存限制。 -如果 overcommit 跟踪器无法选择要停止的查询,将抛出 MEMORY_LIMIT_EXCEEDED 异常。 - - - -## 用户超量使用跟踪器 {#user-overcommit-tracker} - -用户超量使用跟踪器会在用户的查询列表中找到超量使用比例最大的查询。 -查询的超量使用比例是通过已分配字节数除以 `memory_overcommit_ratio_denominator_for_user` 设置的值来计算的。 - -如果该查询的 `memory_overcommit_ratio_denominator_for_user` 等于零,超量使用跟踪器不会选择此查询。 - -等待超时时间由 `memory_usage_overcommit_max_wait_microseconds` 设置进行配置。 - -**示例** - -```sql -SELECT number FROM numbers(1000) GROUP BY number SETTINGS memory_overcommit_ratio_denominator_for_user=4000, memory_usage_overcommit_max_wait_microseconds=500 -``` - - -## 全局 overcommit 跟踪器 {#global-overcommit-tracker} - -全局 overcommit 跟踪器会在所有查询中找到 overcommit 比率最高的查询。 -这里的 overcommit 比率计算方式为:已分配的字节数除以 `memory_overcommit_ratio_denominator` 设置的值。 - -如果某个查询的 `memory_overcommit_ratio_denominator` 等于零,overcommit 跟踪器不会选择该查询。 - -等待超时时间由配置文件中的 `memory_usage_overcommit_max_wait_microseconds` 参数指定。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md deleted file mode 100644 index ce6b1c39be6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/merge-tree-settings.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '用于 MergeTree 的设置,位于 `system.merge_tree_settings` 中' -slug: /operations/settings/merge-tree-settings -title: 'MergeTree 表的设置' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; -import BetaBadge from '@theme/badges/BetaBadge'; -import SettingsInfoBlock from '@theme/SettingsInfoBlock/SettingsInfoBlock'; -import VersionHistory from '@theme/VersionHistory/VersionHistory'; - -系统表 `system.merge_tree_settings` 显示全局生效的 MergeTree 设置。 - -MergeTree 设置可以在服务器配置文件的 `merge_tree` 部分中进行全局配置,或者在 `CREATE TABLE` 语句的 `SETTINGS` 子句中为每个 `MergeTree` 表单独指定。 - -自定义设置 `max_suspicious_broken_parts` 的示例: - -在服务器配置文件中为所有 `MergeTree` 表配置默认值: - -```text - - 5 - -``` - -为特定表设置: - -```sql -CREATE TABLE tab -( - `A` Int64 -) -ENGINE = MergeTree -ORDER BY tuple() -SETTINGS max_suspicious_broken_parts = 500; -``` - -使用 `ALTER TABLE ... MODIFY SETTING` 命令更改某个表的设置: - -```sql -ALTER TABLE tab MODIFY SETTING max_suspicious_broken_parts = 100; - --- 重置为全局默认值(取自 system.merge_tree_settings) -ALTER TABLE tab RESET SETTING max_suspicious_broken_parts; -``` - -## MergeTree 设置 {#mergetree-settings} - -{/* 以下设置由以下脚本自动生成: - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/settings/autogenerate-settings.sh - */ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/overview.md deleted file mode 100644 index 603d6b1e608..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/overview.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: '设置概览页面。' -sidebar_position: 1 -slug: /operations/settings/overview -title: '设置概览' -doc_type: 'reference' ---- - - - -# 设置概览 {#settings-overview} - - - -## 概览 {#overview} - -:::note -基于 XML 的 Settings Profiles 和[配置文件](/operations/configuration-files) 目前在 ClickHouse Cloud 中不受支持。若要为 ClickHouse Cloud 服务指定设置,必须使用[基于 SQL 的 Settings Profiles](/operations/access-rights#settings-profiles-management)。 -::: - -ClickHouse 的设置主要分为两大类: - -- 全局服务器设置 -- 会话设置 - -二者的主要区别在于,全局服务器设置作用于整个 ClickHouse 服务器,而会话设置则作用于用户会话,甚至可以细化到单个查询。 - - - -## 查看非默认设置 {#see-non-default-settings} - -要查看哪些设置已被修改为非默认值,可以查询 -`system.settings` 表: - -```sql -SELECT name, value FROM system.settings WHERE changed -``` - -如果所有设置都保持默认值且未作修改,ClickHouse 将不会返回任何结果。 - -要查看某个特定设置的值,可以在查询中指定该设置的 `name`: - -```sql -SELECT name, value FROM system.settings WHERE name = 'max_threads' -``` - -会返回类似下面的结果: - -```response -┌─name────────┬─value─────┐ -│ max_threads │ 'auto(8)' │ -└─────────────┴───────────┘ - -返回 1 行。耗时:0.002 秒。 -``` - - -## 延伸阅读 {#further-reading} - -- 参阅[全局服务器设置](/operations/server-configuration-parameters/settings.md),以进一步了解如何在全局服务器层面配置 ClickHouse 服务器。 -- 参阅[会话设置](/operations/settings/settings-query-level.md),以进一步了解如何在会话层面配置 ClickHouse 服务器。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md deleted file mode 100644 index 61cc676de9c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/permissions-for-queries.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: '查询权限设置。' -sidebar_label: '查询权限' -sidebar_position: 58 -slug: /operations/settings/permissions-for-queries -title: '查询权限' -doc_type: 'reference' ---- - - - -# 查询权限 {#permissions-for-queries} - -ClickHouse 中的查询可以分为几种类型: - -1. 读数据查询:`SELECT`、`SHOW`、`DESCRIBE`、`EXISTS`。 -2. 写数据查询:`INSERT`、`OPTIMIZE`。 -3. 更改设置查询:`SET`、`USE`。 -4. [DDL](https://en.wikipedia.org/wiki/Data_definition_language) 查询:`CREATE`、`ALTER`、`RENAME`、`ATTACH`、`DETACH`、`DROP`、`TRUNCATE`。 -5. `KILL QUERY`。 - -以下设置用于按查询类型控制用户权限: - - - -## readonly {#readonly} -限制对读取数据、写入数据以及修改设置类查询的权限。 - -当设置为 1 时,允许: - -- 所有类型的读查询(如 SELECT 以及等价查询)。 -- 仅修改会话上下文的查询(如 USE)。 - -当设置为 2 时,在上述基础上额外允许: -- SET 和 CREATE TEMPORARY TABLE - - :::tip - 像 EXISTS、DESCRIBE、EXPLAIN、SHOW PROCESSLIST 等查询等价于 SELECT,因为它们只是从 system 表中进行查询。 - ::: - -可能的取值: - -- 0 — 允许读取、写入以及修改设置类查询。 -- 1 — 只允许读取数据查询。 -- 2 — 允许读取数据以及修改设置类查询。 - -默认值:0 - -:::note -在设置 `readonly = 1` 之后,用户无法在当前会话中更改 `readonly` 和 `allow_ddl` 设置。 - -在 [HTTP 接口](../../interfaces/http.md) 中使用 `GET` 方法时,会自动设置 `readonly = 1`。要修改数据,请使用 `POST` 方法。 - -设置 `readonly = 1` 会禁止用户更改设置。也可以只禁止用户更改特定设置;同样,也可以在 `readonly = 1` 的限制下,仅允许更改特定设置。详情参见[设置约束](../../operations/settings/constraints-on-settings.md)。 -::: - - - -## allow_ddl {#allow_ddl} - -允许或禁止执行 [DDL](https://en.wikipedia.org/wiki/Data_definition_language) 查询。 - -可选值: - -- 0 — 不允许执行 DDL 查询。 -- 1 — 允许执行 DDL 查询。 - -默认值:1 - -:::note -如果当前会话的 `allow_ddl = 0`,则无法执行 `SET allow_ddl = 1`。 -::: - -:::note KILL QUERY -无论 readonly 和 allow_ddl 如何组合设置,都可以执行 `KILL QUERY`。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md deleted file mode 100644 index 669d080af8d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/query-complexity.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: '用于限制查询复杂度的设置。' -sidebar_label: '查询复杂度限制' -sidebar_position: 59 -slug: /operations/settings/query-complexity -title: '查询复杂度限制' -doc_type: 'reference' ---- - - - -# 对查询复杂度的限制 {#restrictions-on-query-complexity} - - - -## 概览 {#overview} - -作为[设置](/operations/settings/overview)的一部分,ClickHouse 支持 -对查询复杂度进行限制。这样有助于防止潜在的高资源消耗查询, -从而确保执行过程更安全、更可预测,尤其是在使用用户界面时。 - -几乎所有限制仅适用于 `SELECT` 查询;对于分布式 -查询处理,这些限制会在每台服务器上分别生效。 - -通常,ClickHouse 只会在数据片段(data parts) -全部处理完成后才检查这些限制,而不是对每一行进行检查。这可能 -导致在处理某个片段的过程中就已经违反了限制条件。 - - - -## `overflow_mode` 设置 {#overflow_mode_setting} - -大多数限制项也有一个 `overflow_mode` 设置,用于定义在超过限制时的处理方式,其取值可以是以下两种之一: -- `throw`:抛出异常(默认)。 -- `break`:停止执行查询并返回部分结果,就像源数据已经耗尽一样。 - - - -## `group_by_overflow_mode` 设置 {#group_by_overflow_mode_settings} - -`group_by_overflow_mode` 设置的一个取值为 `any`: -- `any`: 对已进入集合的键继续进行聚合,但不再向集合中添加新的键。 - - - -## 设置列表 {#relevant-settings} - -以下设置用于限制查询的复杂度。 - -:::note -对“某项上限值”的限制可以设置为 `0`, -表示“不受限制”。 -::: - - - -| 设置 | 简要说明 | -| ---------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -| [`max_memory_usage`](/operations/settings/settings#max_memory_usage) | 在单台服务器上执行查询时可使用的最大内存。 | -| [`max_memory_usage_for_user`](/operations/settings/settings#max_memory_usage_for_user) | 在单个服务器上执行某个用户查询时可使用的最大内存量。 | -| [`max_rows_to_read`](/operations/settings/settings#max_rows_to_read) | 在执行查询时可从表中读取的最大行数。 | -| [`max_bytes_to_read`](/operations/settings/settings#max_bytes_to_read) | 在执行查询时允许从表中读取的未压缩数据的最大字节数。 | -| [`read_overflow_mode_leaf`](/operations/settings/settings#read_overflow_mode_leaf) | 设置当读取的数据量超过任一叶子限制时的处理行为 | -| [`max_rows_to_read_leaf`](/operations/settings/settings#max_rows_to_read_leaf) | 在执行分布式查询时,可从叶节点上的本地表读取的最大行数 | -| [`max_bytes_to_read_leaf`](/operations/settings/settings#max_bytes_to_read_leaf) | 在执行分布式查询时,从叶节点上的本地表可读取的未压缩数据的最大字节数。 | -| [`read_overflow_mode_leaf`](/docs/operations/settings/settings#read_overflow_mode_leaf) | 设置当读取的数据量超过某个叶级限制时应采取的行为。 | -| [`max_rows_to_group_by`](/operations/settings/settings#max_rows_to_group_by) | 聚合接收到的唯一键数量上限。 | -| [`group_by_overflow_mode`](/operations/settings/settings#group_by_overflow_mode) | 设置在用于聚合的唯一键数量超过限制时的处理策略 | -| [`max_bytes_before_external_group_by`](/operations/settings/settings#max_bytes_before_external_group_by) | 启用或禁用在外部内存中执行 `GROUP BY` 子句的功能。 | -| [`max_bytes_ratio_before_external_group_by`](/operations/settings/settings#max_bytes_ratio_before_external_group_by) | 允许 `GROUP BY` 使用的可用内存比例。达到该比例后,将使用外部内存执行聚合。 | -| [`max_bytes_before_external_sort`](/operations/settings/settings#max_bytes_before_external_sort) | 启用或禁用在外部内存中执行 `ORDER BY` 子句。 | -| [`max_bytes_ratio_before_external_sort`](/operations/settings/settings#max_bytes_ratio_before_external_sort) | 允许 `ORDER BY` 使用的可用内存占比。一旦达到该占比,就会使用外部排序。 | -| [`max_rows_to_sort`](/operations/settings/settings#max_rows_to_sort) | 排序前的最大行数。用于在排序时限制内存占用。 | -| [`max_bytes_to_sort`](/operations/settings/settings#max_rows_to_sort) | 排序前的最大字节数。 | -| [`sort_overflow_mode`](/operations/settings/settings#sort_overflow_mode) | 设置当排序前接收的行数超过任一限制时要执行的操作。 | -| [`max_result_rows`](/operations/settings/settings#max_result_rows) | 限制结果返回的行数。 | -| [`max_result_bytes`](/operations/settings/settings#max_result_bytes) | 限制结果大小(未压缩,以字节为单位) | -| [`result_overflow_mode`](/operations/settings/settings#result_overflow_mode) | 设置当结果大小超出任一限制时要执行的操作。 | -| [`max_execution_time`](/operations/settings/settings#max_execution_time) | 查询的最⼤执⾏时间(秒)。 | -| [`timeout_overflow_mode`](/operations/settings/settings#timeout_overflow_mode) | 设置在查询运行时间超过 `max_execution_time` 或其预计运行时间超过 `max_estimated_execution_time` 时应执行的操作。 | -| [`max_execution_time_leaf`](/operations/settings/settings#max_execution_time_leaf) | 在语义上与 `max_execution_time` 类似,但仅在分布式或远程查询中应用于叶节点。 | -| [`timeout_overflow_mode_leaf`](/operations/settings/settings#timeout_overflow_mode_leaf) | 指定当叶节点上的查询运行时间超过 `max_execution_time_leaf` 时的处理方式。 | -| [`min_execution_speed`](/operations/settings/settings#min_execution_speed) | 以每秒处理行数计的最低执行速度。 | -| [`min_execution_speed_bytes`](/operations/settings/settings#min_execution_speed_bytes) | 每秒执行的最小字节数。 | -| [`max_execution_speed`](/operations/settings/settings#max_execution_speed) | 每秒最大执行行数。 | -| [`max_execution_speed_bytes`](/operations/settings/settings#max_execution_speed_bytes) | 每秒可执行的最大字节数。 | -| [`timeout_before_checking_execution_speed`](/operations/settings/settings#timeout_before_checking_execution_speed) | 在经过指定的秒数后,检查执行速度是否不低于 `min_execution_speed`。 | -| [`max_estimated_execution_time`](/operations/settings/settings#max_estimated_execution_time) | 查询的最大预估执行时间(秒)。 | -| [`max_columns_to_read`](/operations/settings/settings#max_columns_to_read) | 在单个查询中可以从表中读取的最大列数。 | -| [`max_temporary_columns`](/operations/settings/settings#max_temporary_columns) | 在执行查询时必须同时保存在内存中的临时列(包括常量列)的最大数量。 | -| [`max_temporary_non_const_columns`](/operations/settings/settings#max_temporary_non_const_columns) | 在执行查询时需要同时保存在内存(RAM)中的临时列的最大数量,不包括常量列。 | -| [`max_subquery_depth`](/operations/settings/settings#max_subquery_depth) | 设置当查询中的嵌套子查询数量超过指定值时的处理行为。 | -| [`max_ast_depth`](/operations/settings/settings#max_ast_depth) | 查询语法树的最大嵌套深度。 | -| [`max_ast_elements`](/operations/settings/settings#max_ast_elements) | 查询语法树中元素的最大数量。 | -| [`max_rows_in_set`](/operations/settings/settings#max_rows_in_set) | 由子查询生成并用于 IN 子句的数据集所允许的最大行数。 | -| [`max_bytes_in_set`](/operations/settings/settings#max_bytes_in_set) | 在由子查询在 IN 子句中创建的集合中可使用的未压缩数据的最大字节数。 | -| [`set_overflow_mode`](/operations/settings/settings#max_bytes_in_set) | 设置当数据量超过任一限制时的处理方式。 | -| [`max_rows_in_distinct`](/operations/settings/settings#max_rows_in_distinct) | 使用 DISTINCT 时的最大不同行数。 | -| [`max_bytes_in_distinct`](/operations/settings/settings#max_bytes_in_distinct) | 在使用 DISTINCT 时,哈希表在内存中可占用的状态最大字节数(按未压缩字节计算)。 | -| [`distinct_overflow_mode`](/operations/settings/settings#distinct_overflow_mode) | 设置当数据量超过任一限制时的处理方式。 | -| [`max_rows_to_transfer`](/operations/settings/settings#max_rows_to_transfer) | 在执行 GLOBAL IN/JOIN 子句时,可传递到远程服务器或保存到临时表中的数据的最大行数。 | -| [`max_bytes_to_transfer`](/operations/settings/settings#max_bytes_to_transfer) | 在执行 GLOBAL IN/JOIN 子句时,可传递到远程服务器或保存到临时表中的未压缩数据的最大字节数。 | -| [`transfer_overflow_mode`](/operations/settings/settings#transfer_overflow_mode) | 设置当数据量超过任一限制时的处理策略。 | -| [`max_rows_in_join`](/operations/settings/settings#max_rows_in_join) | 限制用于表连接的哈希表中的最大行数。 | -| [`max_bytes_in_join`](/operations/settings/settings#max_bytes_in_join) | 用于表连接的哈希表的最大字节数。 | -| [`join_overflow_mode`](/operations/settings/settings#join_overflow_mode) | 定义当达到以下任一 JOIN 限制时,ClickHouse 将执行的操作。 | -| [`max_partitions_per_insert_block`](/operations/settings/settings#max_partitions_per_insert_block) | 限制单个插入块中可包含的最大分区数;如果该块中的分区数量超出此限制,则抛出异常。 | -| [`throw_on_max_partitions_per_insert_block`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | 用于控制在达到 `max_partitions_per_insert_block` 限制时的行为。 | -| [`max_temporary_data_on_disk_size_for_user`](/operations/settings/settings#throw_on_max_partitions_per_insert_block) | 所有并发运行的用户查询在磁盘上用于临时文件的数据最大总量(以字节为单位)。 | -| [`max_temporary_data_on_disk_size_for_query`](/operations/settings/settings#max_temporary_data_on_disk_size_for_query) | 所有并发运行查询在磁盘上的临时文件可消耗的最大数据量(以字节为单位)。 | -| [`max_sessions_for_user`](/operations/settings/settings#max_sessions_for_user) | 每个已认证用户连接到 ClickHouse 服务器时允许的最大并发会话数。 | -| [`max_partitions_to_read`](/operations/settings/settings#max_partitions_to_read) | 限制在单个查询中可访问的最大分区数量。 | - - - - - -## 已过时的设置 {#obsolete-settings} - -:::note -以下设置已过时 -::: - -### max_pipeline_depth {#max-pipeline-depth} - -最大管道深度。表示每个数据块在查询处理过程中经历的转换次数。仅在单个服务器内计算。如果管道深度过大,将抛出异常。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md deleted file mode 100644 index 4ffabfcfee0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/server-overload.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '控制服务器在 CPU 过载时的行为。' -sidebar_label: '服务器过载' -slug: /operations/settings/server-overload -title: '服务器过载' -doc_type: 'reference' ---- - - - -# 服务器过载 {#server-overload} - - - -## 概览 {#overview} - -有时服务器可能会由于各种原因而过载。为了判断当前的 CPU 过载情况, -ClickHouse 服务器会计算 CPU 等待时间(`OSCPUWaitMicroseconds` 指标)与忙碌时间 -(`OSCPUVirtualTimeMicroseconds` 指标)的比值。当服务器的过载程度超过某个比值阈值时, -丢弃部分查询,甚至拒绝新的连接请求,以避免进一步增加负载,是合理的。 - -服务器有一个配置项 `os_cpu_busy_time_threshold`,用于控制将 CPU 视为在执行有用工作时所需的最小忙碌时间。 -如果 `OSCPUVirtualTimeMicroseconds` 指标的当前值低于该阈值, -则认为当前不存在 CPU 过载。 - - - -## 拒绝查询 {#rejecting-queries} - -拒绝查询的行为由查询级别设置 `min_os_cpu_wait_time_ratio_to_throw` 和 -`max_os_cpu_wait_time_ratio_to_throw` 控制。如果设置了这些参数,并且 `min_os_cpu_wait_time_ratio_to_throw` 小于 -`max_os_cpu_wait_time_ratio_to_throw`,则当过载比例至少达到 `min_os_cpu_wait_time_ratio_to_throw` 时,查询会按一定概率被拒绝,并抛出 `SERVER_OVERLOADED` 错误。 -该概率通过在最小和最大比例之间进行线性插值计算得到。例如,如果 `min_os_cpu_wait_time_ratio_to_throw = 2`、 -`max_os_cpu_wait_time_ratio_to_throw = 6`,并且 `cpu_overload = 4`,则查询将以 `0.5` 的概率被拒绝。 - - - -## 丢弃连接 {#dropping-connections} - -丢弃连接由服务器级别设置 `min_os_cpu_wait_time_ratio_to_drop_connection` 和 -`max_os_cpu_wait_time_ratio_to_drop_connection` 控制。这些设置可以在不重启服务器的情况下进行更改。这些设置背后的思路与拒绝查询类似。唯一的区别在于,在这种情况下,如果服务器过载,新的连接尝试会在服务器端被拒绝。 - - - -## 资源过载警告 {#resource-overload-warnings} - -当服务器过载时,ClickHouse 还会将 CPU 和内存过载警告记录到 `system.warnings` 表中。您可以 -通过服务器配置来自定义这些阈值。 - -**示例** - -```xml - - - 0.9 - 0.8 - 600 - 0.9 - 0.8 - 600 - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md deleted file mode 100644 index 65097d36f1a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-formats.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -description: '与格式相关的设置' -sidebar_label: '格式设置' -slug: /operations/settings/formats -title: '格式设置' -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/*请勿编辑——此文件为自动生成的*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md deleted file mode 100644 index 51b1a6806a7..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-profiles.md +++ /dev/null @@ -1,84 +0,0 @@ ---- -description: '在同一名称下分组的设置集合。' -sidebar_label: '设置配置集' -sidebar_position: 61 -slug: /operations/settings/settings-profiles -title: '设置配置集' -doc_type: 'reference' ---- - -# 设置配置文件 {#settings-profiles} - -设置配置文件是按同一名称分组的一组设置。 - -:::note -ClickHouse 也支持用于管理设置配置文件的[基于 SQL 的工作流程](/operations/access-rights#access-control-usage)。建议优先使用该方式。 -::: - -配置文件可以有任意名称。可以为不同用户指定相同的配置文件。在设置配置文件中,最重要的一项是 `readonly=1`,它可以确保只读访问。 - -设置配置文件之间可以相互继承。要使用继承,请在配置文件中列出的其他设置之前,先指定一个或多个 `profile` 设置。如果某个设置在不同的配置文件中都有定义,则以最后定义的为准。 - -要应用配置文件中的所有设置,请设置 `profile` 参数。 - -示例: - -安装 `web` 配置文件。 - -```sql -SET profile = 'web' -``` - -设置配置在用户配置文件中声明,通常是 `users.xml`。 - -示例: - -```xml - - - - - - 8 - - - - - 1000000000 - 100000000000 - - 1000000 - any - - 1000000 - 1000000000 - - 100000 - 100000000 - break - - 600 - 1000000 - 15 - - 25 - 100 - 50 - - 2 - 25 - 50 - 100 - - 4 - - 1 - - -``` - -该示例定义了两个配置文件:`default` 和 `web`。 - -`default` 配置文件有一个特殊用途:它必须始终存在,并在启动服务器时应用。换句话说,`default` 配置文件包含默认设置。 - -`web` 配置文件是一个常规配置文件,可以通过 `SET` 查询或在 HTTP 查询中使用 URL 参数来设置。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md deleted file mode 100644 index 1879b32d957..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-query-level.md +++ /dev/null @@ -1,228 +0,0 @@ ---- -description: '查询级别会话设置' -sidebar_label: '查询级别会话设置' -slug: /operations/settings/query-level -title: '查询级别会话设置' -doc_type: 'reference' ---- - -## 概述 {#overview} - -有多种方式可以在运行语句时指定特定的设置。 -设置是分层配置的,每一后续层都会覆盖该设置在前一层中的值。 - -## 优先级顺序 {#order-of-priority} - -定义设置时的优先级顺序为: - -1. 直接将设置应用于某个用户,或在某个设置配置文件中为用户应用设置 - - - 使用 SQL(推荐) - - 将一个或多个 XML 或 YAML 文件添加到 `/etc/clickhouse-server/users.d` 目录 - -2. 会话级设置 - - - 从 ClickHouse Cloud SQL 控制台或 `clickhouse client` 的交互模式发送 - `SET setting=value`。类似地,也可以在 HTTP 协议中使用 ClickHouse 会话。 - 为此,需要指定 `session_id` HTTP 参数。 - -3. 查询级设置 - - - 在非交互模式启动 `clickhouse client` 时,通过启动参数 `--setting=value` 设置。 - - 使用 HTTP API 时,以 CGI 参数形式传递(`URL?setting_1=value&setting_2=value...`)。 - - 在 SELECT 查询的 - [SETTINGS](../../sql-reference/statements/select/index.md#settings-in-select-query) - 子句中定义设置。该设置值仅应用于该次查询,在查询执行完成后将被重置为默认值或先前的值。 - -## 将设置恢复为默认值 {#converting-a-setting-to-its-default-value} - -如果您修改了某个设置并希望将其恢复为默认值,请将该值设为 `DEFAULT`。语法如下: - -```sql -SET setting_name = DEFAULT -``` - -例如,`async_insert` 的默认值为 `0`。假设你将该参数的值修改为 `1`: - -```sql -SET async_insert = 1; - -SELECT value FROM system.settings where name='async_insert'; -``` - -响应如下: - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - -以下命令将其值重置为 0: - -```sql -SET async_insert = DEFAULT; - -SELECT value FROM system.settings where name='async_insert'; -``` - -此设置现已恢复为默认值: - -```response -┌─value───┐ -│ 0 │ -└─────────┘ -``` - - -## 自定义设置 {#custom_settings} - -除了通用的[设置](/operations/settings/settings.md)之外,用户还可以定义自定义设置。 - -自定义设置名称必须以前缀列表中的某个预定义前缀开头。该前缀列表需要在服务器配置文件的 [custom_settings_prefixes](../../operations/server-configuration-parameters/settings.md#custom_settings_prefixes) 参数中声明。 - -```xml -custom_ -``` - -若要定义自定义设置,请使用 `SET` 命令: - -```sql -SET custom_a = 123; -``` - -要获取某个自定义设置的当前值,请使用 `getSetting()` 函数: - -```sql -SELECT getSetting('custom_a'); -``` - - -## 示例 {#examples} - -这些示例都将 `async_insert` 设置为 `1`,并展示如何在正在运行的系统中查看这些设置。 - -### 使用 SQL 将设置直接应用到用户 {#using-sql-to-apply-a-setting-to-a-user-directly} - -以下示例创建用户 `ingester`,并为其设置 `async_insert = 1`: - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS async_insert = 1 -``` - - -#### 检查设置配置文件和分配 {#examine-the-settings-profile-and-assignment} - -```sql -SHOW ACCESS -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ ... │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS async_insert = true │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### 使用 SQL 创建设置配置文件并分配给用户 {#using-sql-to-create-a-settings-profile-and-assign-to-a-user} - -以下语句会创建名为 `log_ingest` 的设置配置文件,并设置 `async_inset = 1`: - -```sql -CREATE -SETTINGS PROFILE log_ingest SETTINGS async_insert = 1 -``` - -这将创建用户 `ingester`,并为其分配设置配置文件 `log_ingest`: - -```sql -CREATE USER ingester -IDENTIFIED WITH sha256_hash BY '7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3' --- highlight-next-line -SETTINGS PROFILE log_ingest -``` - - -### 使用 XML 创建设置配置文件及用户 {#using-xml-to-create-a-settings-profile-and-user} - -```xml title=/etc/clickhouse-server/users.d/users.xml - -# highlight-start {#highlight-start} - - - 1 - - -# highlight-end {#highlight-end} - - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 -# highlight-start {#highlight-start} - log_ingest -# highlight-end {#highlight-end} - - - 7e099f39b84ea79559b3e85ea046804e63725fd1f46b37f281276aae20f86dc3 - 1 - 1 - - - -``` - - -#### 查看设置配置文件及其分配 {#examine-the-settings-profile-and-assignment-1} - -```sql -显示访问权限 -``` - -```response -┌─ACCESS─────────────────────────────────────────────────────────────────────────────┐ -│ CREATE USER default IDENTIFIED WITH sha256_password │ -# highlight-next-line {#highlight-next-line} -│ CREATE USER ingester IDENTIFIED WITH sha256_password SETTINGS PROFILE log_ingest │ -│ CREATE SETTINGS PROFILE default │ -# highlight-next-line {#highlight-next-line} -│ CREATE SETTINGS PROFILE log_ingest SETTINGS async_insert = true │ -│ CREATE SETTINGS PROFILE readonly SETTINGS readonly = 1 │ -│ ... │ -└────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -### 为会话指定设置 {#assign-a-setting-to-a-session} - -```sql -SET async_insert =1; -SELECT value FROM system.settings where name='async_insert'; -``` - -```response -┌─value──┐ -│ 1 │ -└────────┘ -``` - - -### 在查询时指定设置 {#assign-a-setting-during-a-query} - -```sql -INSERT INTO YourTable --- highlight-next-line -SETTINGS async_insert=1 -VALUES (...) -``` - - -## 另请参阅 {#see-also} - -- 查看 [Settings](/operations/settings/settings.md) 页面,了解 ClickHouse 各项设置的说明。 -- [全局服务器设置](/operations/server-configuration-parameters/settings.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md deleted file mode 100644 index f9b354ce523..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings-users.md +++ /dev/null @@ -1,259 +0,0 @@ ---- -description: '用于配置用户和角色的相关设置。' -sidebar_label: '用户设置' -sidebar_position: 63 -slug: /operations/settings/settings-users -title: '用户和角色设置' -doc_type: 'reference' ---- - -# 用户和角色设置 {#users-and-roles-settings} - -`users.xml` 配置文件中的 `users` 部分包含用户配置。 - -:::note -ClickHouse 也支持用于管理用户的 [基于 SQL 的工作流](/operations/access-rights#access-control-usage),我们建议优先采用这种方式。 -::: - -`users` 部分的结构: - -```xml - - - - - - - - - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - - - ecdsa-sha2-nistp256 - AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxeV2uN5UY6CUbCzTA1rXfYimKQA5ivNIqxdax4bcMXz4D0nSk2l5E1TkR5mG8EBWtmExSPbcEPJ8V7lyWWbA8= - - - ssh-rsa - AAAAB3NzaC1yc2EAAAADAQABAAABgQCpgqL1SHhPVBOTFlOm0pu+cYBbADzC2jL41sPMawYCJHDyHuq7t+htaVVh2fRgpAPmSEnLEC2d4BEIKMtPK3bfR8plJqVXlLt6Q8t4b1oUlnjb3VPA9P6iGcW7CV1FBkZQEVx8ckOfJ3F+kI5VsrRlEDgiecm/C1VPl0/9M2llW/mPUMaD65cM9nlZgM/hUeBrfxOEqM11gDYxEZm1aRSbZoY4dfdm3vzvpSQ6lrCrkjn3X2aSmaCLcOWJhfBWMovNDB8uiPuw54g3ioZ++qEQMlfxVsqXDGYhXCrsArOVuW/5RbReO79BvXqdssiYShfwo+GhQ0+aLWMIW/jgBkkqx/n7uKLzCMX7b2F+aebRYFh+/QXEj7SnihdVfr9ud6NN3MWzZ1ltfIczlEcFLrLJ1Yq57wW6wXtviWh59WvTWFiPejGjeSjjJyqqB49tKdFVFuBnIU5u/bch2DXVgiAEdQwUrIp1ACoYPq22HFFAYUJrL32y7RxX3PGzuAv3LOc= - - - - 0|1 - - - - - profile_name - - default - default - - - - 过滤表达式 - - - - - - GRANT SELECT ON system.* - - - - -``` - -### user_name/password {#user-namepassword} - -密码可以以明文形式或 SHA256 哈希值(十六进制格式)指定。 - -* 要以明文形式设置密码(**不推荐**),请将其放在 `password` 元素中。 - - 例如,`qwerty`。密码可以留空。 - - - -* 要使用密码的 SHA256 哈希值来设置密码,请将其放在 `password_sha256_hex` 元素中。 - -例如,`65e84be33532fb784c48129675f9eff3a682b27168c0ea744b2cf58ee02337c5`。 - -在 shell 中生成密码的示例: - -```bash -PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha256sum | tr -d '-' -``` - -结果的第一行是密码。第二行是对应的 SHA256 哈希值。 - - - -* 为了与 MySQL 客户端兼容,可以以双重 SHA1 哈希的形式指定密码。将其放在 `password_double_sha1_hex` 元素中。 - - 例如:`08b4a0f1de6ad37da17359e592c8d74788a83eb0`。 - - 从 shell 生成密码的示例: - - ```bash - PASSWORD=$(base64 < /dev/urandom | head -c8); echo "$PASSWORD"; echo -n "$PASSWORD" | sha1sum | tr -d '-' | xxd -r -p | sha1sum | tr -d '-' - ``` - - 结果的第一行是密码。第二行是对应的双重 SHA1 哈希值。 - -### username/ssh-key {#user-sshkey} - -此设置允许使用 SSH 密钥进行身份验证。 - -给定一个 SSH 密钥(由 `ssh-keygen` 生成),例如 - -```text -ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj john@example.com -``` - -`ssh_key` 元素应为 - -```xml - - ssh-ed25519 - AAAAC3NzaC1lZDI1NTE5AAAAIDNf0r6vRl24Ix3tv2IgPmNPO2ATa2krvt80DdcTatLj - -``` - -将 `ssh-ed25519` 替换为 `ssh-rsa` 或 `ecdsa-sha2-nistp256`,以使用其他受支持的算法。 - -### access_management {#access_management-user-setting} - -此设置用于启用或禁用对该用户使用 SQL 驱动的[访问控制和账户管理](/operations/access-rights#access-control-usage)。 - -可能的取值: - -* 0 — 禁用。 -* 1 — 启用。 - -默认值:0。 - -### grants {#grants-user-setting} - -此设置允许为选定的用户授予任意权限。 -列表中的每个元素都应为未指定任何被授权者的 `GRANT` 语句。 - -示例: - -```xml - - - GRANT SHOW ON *.* - GRANT CREATE ON *.* WITH GRANT OPTION - GRANT SELECT ON system.* - - -``` - -此设置不能与 `dictionaries`、`access_management`、`named_collection_control`、`show_named_collections_secrets` 和 `allow_databases` 设置同时指定。 - -### user_name/networks {#user-namenetworks} - -用于指定用户可以从哪些网络连接到 ClickHouse 服务器的列表。 - -列表中的每个元素可以具有以下形式之一: - -* `` — IP 地址或网络掩码。 - - 示例:`213.180.204.3`、`10.0.0.1/8`、`10.0.0.1/255.255.255.0`、`2a02:6b8::3`、`2a02:6b8::3/64`、`2a02:6b8::3/ffff:ffff:ffff:ffff::`。 - -* `` — 主机名。 - - 示例:`example01.host.ru`。 - - 为检查访问权限,会执行 DNS 查询,并将所有返回的 IP 地址与对端地址进行比较。 - -* `` — 主机名的正则表达式。 - - 示例:`^example\d\d-\d\d-\d\.host\.ru$` - -为了检查访问权限,系统会针对对端地址执行一次 [DNS PTR 查询](https://en.wikipedia.org/wiki/Reverse_DNS_lookup),然后应用指定的正则表达式。接着,会对 PTR 查询结果再次执行 DNS 查询,并将收到的所有地址与对端地址进行比较。我们强烈建议正则表达式以 $ 结尾。 - -所有 DNS 请求的结果都会被缓存,直到服务器重启。 - -**示例** - -若要为来自任意网络的用户开放访问权限,请指定: - -```xml -::/0 -``` - -:::note -除非已正确配置防火墙,或服务器未直接连接到互联网,否则对任意网络开放访问是不安全的。 -::: - -若只允许从 localhost 访问,请指定: - -```xml -::1 -127.0.0.1 -``` - -### user_name/profile {#user-nameprofile} - -您可以为用户分配一个设置配置文件。设置配置文件在 `users.xml` 文件的单独部分中进行配置。有关更多信息,请参阅[设置配置文件](../../operations/settings/settings-profiles.md)。 - -### user_name/quota {#user-namequota} - -配额允许您在一段时间内跟踪或限制资源使用。配额在 `users.xml` 配置文件的 `quotas` 部分中进行配置。 - -您可以为用户分配一组配额。有关配额配置的详细说明,请参阅[配额](/operations/quotas)。 - -### user_name/databases {#user-namedatabases} - -在此部分中,您可以限制当前用户执行 `SELECT` 查询时 ClickHouse 返回的行,从而实现基本的行级安全。 - -**示例** - -以下配置会使用户 `user1` 在执行 `SELECT` 查询时,仅能看到 `table1` 中 `id` 字段值为 1000 的那些行。 - -```xml - - - - - id = 1000 - - - - -``` - -`filter` 可以是任意返回 [UInt8](../../sql-reference/data-types/int-uint.md) 类型值的表达式。它通常包含比较和逻辑运算符。对于该用户,`database_name.table1` 中 `filter` 结果为 0 的行将不会返回。该过滤机制与 `PREWHERE` 操作不兼容,并会禁用 `WHERE→PREWHERE` 优化。 - -## 角色 {#roles} - -可以在 `user.xml` 配置文件的 `roles` 部分中创建任意预定义角色。 - -`roles` 部分的结构: - -```xml - - - - GRANT SHOW ON *.* - REVOKE SHOW ON system.* - GRANT CREATE ON *.* WITH GRANT OPTION - - - -``` - -也可以在 `users` 部分为用户授予这些角色: - -```xml - - - ... - - GRANT test_role - - - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings.md deleted file mode 100644 index 4ac5c1aa0b9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/settings.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: '会话设置' -description: '通过 system.settings 设置的会话参数' -sidebar_label: '会话设置' -slug: /operations/settings/settings -toc_max_heading_level: 2 -doc_type: 'reference' ---- - -{/*请勿编辑 – 此文件为自动生成文件*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md deleted file mode 100644 index 128468d5c90..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/settings/tcp-connection-limits.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: 'TCP 连接数限制。' -sidebar_label: 'TCP 连接数限制' -slug: /operations/settings/tcp-connection-limits -title: 'TCP 连接数限制' -doc_type: 'reference' ---- - - - -# TCP 连接数限制 {#tcp-connection-limits} - - - -## 概览 {#overview} - -您可能会遇到 ClickHouse 的 TCP 连接(例如,通过[命令行客户端](https://clickhouse.com/docs/interfaces/cli)建立的连接) -在执行一定数量的查询或经过一段时间后自动断开。 -在断开连接后,不会发生自动重连(除非通过其他操作触发, -比如在命令行客户端中再次发送查询)。 - -可以通过设置服务端配置项来启用连接限制: -`tcp_close_connection_after_queries_num`(按查询数量限制) -或 `tcp_close_connection_after_queries_seconds`(按持续时间限制),并将其值设置为大于 0。 -如果两个限制都被启用,则连接会在任一限制先被触发时关闭。 - -在触发限制并断开连接时,客户端会收到 -`TCP_CONNECTION_LIMIT_REACHED` 异常,且**导致此次断开的那个查询不会被执行**。 - - - -## 查询限制 {#query-limits} - -假设 `tcp_close_connection_after_queries_num` 设置为 N,则该连接允许进行 -N 次成功查询。在第 N + 1 次查询时,客户端会断开连接。 - -处理的每个查询都会计入查询限制。因此,当使用命令行客户端连接时, -可能会自动发出一次初始系统警告查询,这个查询同样计入限制内。 - -当一个 TCP 连接处于空闲状态(即在一段时间内未处理查询, -该时间由会话设置 `poll_interval` 指定)时,至此为止统计的查询次数会重置为 0。 -这意味着,如果连接在中途出现空闲,同一个连接中的总查询次数可以超过 -`tcp_close_connection_after_queries_num`。 - - - -## 持续时间限制 {#duration-limits} - -连接持续时间从客户端建立连接的瞬间开始计时。 -在经过 `tcp_close_connection_after_queries_seconds` 秒后,客户端在其之后发出的第一条查询时会被断开连接。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md deleted file mode 100644 index 95834d79168..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/ssl-zookeeper.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: '在 ClickHouse 与 ZooKeeper 之间配置安全 SSL/TLS 通信的指南' -sidebar_label: '与 ZooKeeper 的安全通信' -sidebar_position: 45 -slug: /operations/ssl-zookeeper -title: 'ClickHouse 与 ZooKeeper 之间可选的安全通信' -doc_type: 'guide' ---- - -# 可选的 ClickHouse 与 Zookeeper 之间的安全通信 {#optional-secured-communication-between-clickhouse-and-zookeeper} - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - -你需要在通过 SSL 与 ClickHouse 客户端通信时指定 `ssl.keyStore.location`、`ssl.keyStore.password` 以及 `ssl.trustStore.location`、`ssl.trustStore.password`。这些选项从 Zookeeper 3.5.2 版本开始可用。 - -你可以将 `zookeeper.crt` 添加到受信任证书列表中。 - -```bash -sudo cp zookeeper.crt /usr/local/share/ca-certificates/zookeeper.crt -sudo update-ca-certificates -``` - -`config.xml` 中的 client 配置段如下所示:` - -```xml - - /etc/clickhouse-server/client.crt - /etc/clickhouse-server/client.key - true - true - sslv2,sslv3 - true - - RejectCertificateHandler - - -``` - -在 ClickHouse 配置中添加 Zookeeper,并配置相应的集群和宏: - -```xml - - - - localhost - 2281 - 1 - - - -``` - -启动 `clickhouse-server`。在日志中应看到: - -```text - ZooKeeper: 已初始化,主机:secure://localhost:2281 -``` - -前缀 `secure://` 表示连接已通过 SSL 加密保护。 - -要验证流量已加密,可在该安全端口上运行 `tcpdump`: - -```bash -tcpdump -i any dst port 2281 -nnXS -``` - -然后在 `clickhouse-client` 中执行查询: - -```sql -SELECT * FROM system.zookeeper WHERE path = '/'; -``` - -在未加密的连接中,可以在 `tcpdump` 的输出中看到类似如下的内容: - -```text -..../zookeeper/quota. -``` - -在加密连接下,你不应该看到此内容。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/startup-scripts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/startup-scripts.md deleted file mode 100644 index 57d73f2df58..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/startup-scripts.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: '在 ClickHouse 中配置和使用 SQL 启动脚本以自动创建 schema 并执行迁移操作的指南' -sidebar_label: '启动脚本' -slug: /operations/startup-scripts -title: '启动脚本' -doc_type: 'guide' ---- - -# 启动脚本 {#startup-scripts} - -ClickHouse 可以在启动时根据服务器配置执行任意 SQL 查询。这对于迁移或自动创建模式很有用。 - -```xml - - - false - - CREATE ROLE OR REPLACE test_role - - - CREATE TABLE TestTable (id UInt64) ENGINE=TinyLog - SELECT 1; - - - CREATE DICTIONARY test_dict (...) SOURCE(CLICKHOUSE(...)) - default - - - -``` - -ClickHouse 会按照指定顺序依次执行 `startup_scripts` 中的所有查询。如果某个查询失败,后续查询的执行不会被中断。但是,如果将 `throw_on_error` 设为 true,一旦在脚本执行过程中发生错误,服务器将不会启动。 - -你可以在配置中指定条件查询。在这种情况下,仅当条件查询返回值为 `1` 或 `true` 时,才会执行相应的查询。 - -:::note -如果条件查询返回的值不是 `1` 或 `true`,结果会被视为 `false`,相应的查询将不会被执行。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/storing-data.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/storing-data.md deleted file mode 100644 index 2945d527bf8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/storing-data.md +++ /dev/null @@ -1,1051 +0,0 @@ ---- -description: 'highlight-next-line 的文档' -sidebar_label: '用于存储数据的外部磁盘' -sidebar_position: 68 -slug: /operations/storing-data -title: '用于存储数据的外部磁盘' -doc_type: 'guide' ---- - -在 ClickHouse 中处理的数据通常存储在运行 ClickHouse 服务器的 -机器的本地文件系统中。这需要大容量磁盘,而这可能比较昂贵。为了避免在本地存储数据,ClickHouse 支持多种存储选项: -1. [Amazon S3](https://aws.amazon.com/s3/) 对象存储。 -2. [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs)。 -3. 不受支持:Hadoop 分布式文件系统([HDFS](https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html)) - -
- -:::note -ClickHouse 还支持外部表引擎,它们与本页所描述的外部存储选项不同,因为这些引擎允许读取以通用文件格式(例如 Parquet)存储的数据。本页描述的是 ClickHouse `MergeTree` 系列表或 `Log` 系列表的存储配置。 - -1. 要处理存储在 `Amazon S3` 磁盘上的数据,请使用 [S3](/engines/table-engines/integrations/s3.md) 表引擎。 -2. 要处理存储在 Azure Blob Storage 中的数据,请使用 [AzureBlobStorage](/engines/table-engines/integrations/azureBlobStorage.md) 表引擎。 -3. 要处理 Hadoop 分布式文件系统(不受支持)中的数据,请使用 [HDFS](/engines/table-engines/integrations/hdfs.md) 表引擎。 -::: - - - -## 配置外部存储 {#configuring-external-storage} - -[`MergeTree`](/engines/table-engines/mergetree-family/mergetree.md) 和 [`Log`](/engines/table-engines/log-family/log.md) -系列表引擎可以通过使用类型分别为 `s3`、`azure_blob_storage`、`hdfs`(不支持)的磁盘,将数据存储到 `S3`、`AzureBlobStorage`、`HDFS`(不支持)中。 - -磁盘配置需要: - -1. 一个 `type` 段,其值为 `s3`、`azure_blob_storage`、`hdfs`(不支持)、`local_blob_storage`、`web` 之一。 -2. 指定相应外部存储类型的配置。 - -从 ClickHouse 24.1 版本起,可以使用一个新的配置选项。 -它需要指定: - -1. 一个 `type`,其值为 `object_storage` -2. 一个 `object_storage_type`,其值为 `s3`、`azure_blob_storage`(或从 `24.3` 起简写为 `azure`)、`hdfs`(不支持)、`local_blob_storage`(或从 `24.3` 起简写为 `local`)、`web` 之一。 - -
- -可以选配 `metadata_type`(默认值为 `local`),也可以将其设置为 `plain`、`web`,并且从 `24.4` 起可以设置为 `plain_rewritable`。 -`plain` 元数据类型的用法在 [plain 存储部分](/operations/storing-data#plain-storage)中进行了说明;`web` 元数据类型只能与 `web` 对象存储类型一起使用;`local` 元数据类型会在本地存储元数据文件(每个元数据文件都包含对象存储中文件的映射关系,以及关于这些文件的一些附加元信息)。 - -例如: - -```xml - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -相当于以下配置(自 `24.1` 版本起): - -```xml - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -如下配置: - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -等于: - -```xml - - object_storage - s3 - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -一个完整的存储配置示例如下: - -```xml - - - - - s3 - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -从 24.1 版本开始,它还可以写成: - - -```xml - - - - - object_storage - s3 - local - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - - - - - -
- s3 -
-
-
-
-
-
-``` - -要将特定类型的存储设为所有 `MergeTree` 表的默认选项, -请在配置文件中添加以下配置段: - -```xml - - - s3 - - -``` - -要为某个特定表配置专用的存储策略,可以在创建该表时通过 `SETTINGS` 子句进行定义: - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS storage_policy = 's3'; -``` - -你也可以使用 `disk` 替代 `storage_policy`。在这种情况下,无需在配置文件中包含 `storage_policy` 部分,只保留一个 `disk` 部分即可。 - -```sql -CREATE TABLE test (a Int32, b String) -ENGINE = MergeTree() ORDER BY a -SETTINGS disk = 's3'; -``` - - -## 动态配置 {#dynamic-configuration} - -还可以在无需在配置文件中预先定义磁盘的情况下指定存储配置,而是通过 -`CREATE`/`ATTACH` 查询的设置来进行配置。 - -下面的示例查询基于上述动态磁盘配置,并展示如何使用本地磁盘 -来缓存存储在某个 URL 上的表的数据。 - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -以下示例演示如何为外部存储添加缓存。 - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) --- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); --- highlight-end -``` - -在下方高亮显示的配置中可以看到,`type=web` 的磁盘被嵌套在 -`type=cache` 的磁盘之内。 - -:::note -本示例使用了 `type=web`,但任何磁盘类型都可以配置为动态的, -包括本地磁盘。本地磁盘要求其路径参数位于服务器配置参数 -`custom_local_disks_base_directory` 指定的目录下。该参数没有默认值,因此在使用本地磁盘时也需要进行设置。 -::: - -还可以组合使用基于配置文件的配置和基于 SQL 定义的配置: - -```sql -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=cache, - max_size='1Gi', - path='/var/lib/clickhouse/custom_disk_cache/', - disk=disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ) - ); - -- highlight-end -``` - -其中的 `web` 来自服务器配置文件: - - -```xml - - - - web - 'https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - - - -``` - -### 使用 S3 存储 {#s3-storage} - -#### 必需参数 {#required-parameters-s3} - -| Parameter | Description | -| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | -| `endpoint` | 使用 `path` 或 `virtual hosted` [风格](https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html) 的 S3 endpoint URL。应包含用于数据存储的 bucket 和根路径。 | -| `access_key_id` | 用于身份验证的 S3 access key ID。 | -| `secret_access_key` | 用于身份验证的 S3 secret access key。 | - -#### 可选参数 {#optional-parameters-s3} - - -| Parameter | Description | Default Value | -|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------| -| `region` | S3 区域名称。 | - | -| `support_batch_delete` | 控制是否检查是否支持批量删除。在使用 Google Cloud Storage (GCS) 时将其设置为 `false`,因为 GCS 不支持批量删除。 | `true` | -| `use_environment_credentials` | 如果存在,则从环境变量 `AWS_ACCESS_KEY_ID`、`AWS_SECRET_ACCESS_KEY` 和 `AWS_SESSION_TOKEN` 中读取 AWS 凭证。 | `false` | -| `use_insecure_imds_request` | 如果为 `true`,在从 Amazon EC2 元数据获取凭证时使用不安全的 IMDS 请求。 | `false` | -| `expiration_window_seconds` | 检查基于过期时间的凭证是否已过期的宽限期(秒)。 | `120` | -| `proxy` | S3 endpoint 的代理配置。`proxy` 块中的每个 `uri` 元素都应包含一个代理 URL。 | - | -| `connect_timeout_ms` | 套接字连接超时时间(毫秒)。 | `10000`(10 秒) | -| `request_timeout_ms` | 请求超时时间(毫秒)。 | `5000`(5 秒) | -| `retry_attempts` | 失败请求的重试次数。 | `10` | -| `single_read_retries` | 读操作期间连接中断时的重试次数。 | `4` | -| `min_bytes_for_seek` | 使用 seek 操作而不是顺序读取所需的最小字节数。 | `1 MB` | -| `metadata_path` | 用于存储 S3 元数据文件的本地文件系统路径。 | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | 如果为 `true`,在启动过程中跳过磁盘访问检查。 | `false` | -| `header` | 向请求添加指定的 HTTP 头。可以指定多次。 | - | -| `server_side_encryption_customer_key_base64` | 访问使用 SSE-C 加密的 S3 对象所需的 HTTP 头。 | - | -| `server_side_encryption_kms_key_id` | 访问使用 [SSE-KMS 加密](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html) 的 S3 对象所需的 HTTP 头。空字符串表示使用 AWS 管理的 S3 密钥。 | - | -| `server_side_encryption_kms_encryption_context` | SSE-KMS 的加密上下文 HTTP 头(与 `server_side_encryption_kms_key_id` 一起使用)。 | - | -| `server_side_encryption_kms_bucket_key_enabled` | 为 SSE-KMS 启用 S3 bucket key(与 `server_side_encryption_kms_key_id` 一起使用)。 | 与 bucket 级别设置保持一致 | -| `s3_max_put_rps` | 在触发限流前允许的最大 PUT 请求数(每秒)。 | `0`(不限) | -| `s3_max_put_burst` | 触及 RPS 限制前允许的最大并发 PUT 请求数。 | 与 `s3_max_put_rps` 相同 | -| `s3_max_get_rps` | 在触发限流前允许的最大 GET 请求数(每秒)。 | `0`(不限) | -| `s3_max_get_burst` | 触及 RPS 限制前允许的最大并发 GET 请求数。 | 与 `s3_max_get_rps` 相同 | -| `read_resource` | 用于[调度](/operations/workload-scheduling.md)读请求的资源名称。 | 空字符串(禁用) | -| `write_resource` | 用于[调度](/operations/workload-scheduling.md)写请求的资源名称。 | 空字符串(禁用) | -| `key_template` | 使用 [re2](https://github.com/google/re2/wiki/Syntax) 语法定义对象 key 生成格式。需要 `storage_metadata_write_full_object_key` 标志。与 `endpoint` 中的 `root path` 不兼容。需要 `key_compatibility_prefix`。 | - | -| `key_compatibility_prefix` | 与 `key_template` 一起使用。指定 `endpoint` 中之前的 `root path`,用于读取旧版本元数据。 | - | -| `read_only` | 只允许从该磁盘读取数据。 | - | -:::note -也支持使用类型 `s3` 的 Google Cloud Storage (GCS)。参见[基于 GCS 的 MergeTree](/integrations/gcs)。 -::: - -### 使用 Plain Storage {#plain-storage} - - - -在 `22.10` 中引入了一种新的磁盘类型 `s3_plain`,它提供只写一次的存储。 -其配置参数与 `s3` 磁盘类型相同。 -与 `s3` 磁盘类型不同,它按原样存储数据。换句话说, -它不会使用随机生成的 blob 名称,而是使用普通文件名 -(与 ClickHouse 在本地磁盘上存储文件的方式相同),并且不会在本地存储任何 -元数据。例如,这些元数据是从 `s3` 上的数据中推导而来。 - -这种磁盘类型允许保留表的静态版本,因为它不允许对现有数据执行合并, -也不允许插入新数据。该磁盘类型的一个用例是在其上创建备份, -可以通过 `BACKUP TABLE data TO Disk('plain_disk_name', 'backup_name')` 完成。 -之后,可以执行 -`RESTORE TABLE data AS data_restored FROM Disk('plain_disk_name', 'backup_name')` -或者使用 -`ATTACH TABLE data (...) ENGINE = MergeTree() SETTINGS disk = 'plain_disk_name'`。 - -配置: - -```xml - - s3_plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -从 `24.1` 版本开始,可以使用 `plain` 元数据类型来配置任意对象存储磁盘(`s3`、`azure`、`hdfs`(不支持)、`local`)。 - -配置: - -```xml - - object_storage - azure - plain - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -### 使用 S3 Plain Rewritable 存储 {#s3-plain-rewritable-storage} - -在 `24.4` 中引入了一种新的磁盘类型 `s3_plain_rewritable`。 -与 `s3_plain` 磁盘类型类似,它不需要额外的存储空间来保存 -元数据文件,而是将元数据存储在 S3 中。 -不同于 `s3_plain` 磁盘类型,`s3_plain_rewritable` 允许执行合并操作, -并支持 `INSERT` 操作。 -不支持[变更](/sql-reference/statements/alter#mutations)和表的复制。 - -此磁盘类型的一个使用场景是非复制的 `MergeTree` 表。尽管 -`s3` 磁盘类型适用于非复制的 `MergeTree` 表,如果不需要表的本地元数据, -并且可以接受受限的操作集,则可以选择使用 `s3_plain_rewritable` 磁盘类型。 -例如,这对于系统表可能会很有用。 - -配置: - -```xml - - s3_plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -等于 - -```xml - - object_storage - s3 - plain_rewritable - https://s3.eu-west-1.amazonaws.com/clickhouse-eu-west-1.clickhouse.com/data/ - 1 - -``` - -从 `24.5` 版本起,可以使用 `plain_rewritable` 元数据类型来配置任意对象存储磁盘(`s3`、`azure`、`local`)。 - -### 使用 Azure Blob Storage {#azure-blob-storage} - -`MergeTree` 系列的表引擎可以使用类型为 `azure_blob_storage` 的磁盘将数据存储到 [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/)。 - -配置示例: - - -```xml - - ... - - - azure_blob_storage - http://account.blob.core.windows.net - container - account - pass123 - /var/lib/clickhouse/disks/blob_storage_disk/ - /var/lib/clickhouse/disks/blob_storage_disk/cache/ - false - - - ... - -``` - -#### 连接参数 {#azure-blob-storage-connection-parameters} - -| 参数 | 描述 | 默认值 | -| -------------------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------- | -| `storage_account_url` (Required) | Azure Blob Storage 帐户 URL。例如:`http://account.blob.core.windows.net` 或 `http://azurite1:10000/devstoreaccount1`。 | - | -| `container_name` | 目标容器名称。 | `default-container` | -| `container_already_exists` | 控制容器创建行为:
- `false`:创建一个新容器
- `true`:直接连接到已存在的容器
- 未设置:检查容器是否存在,如不存在则创建 | - | - -身份验证参数(磁盘会尝试所有可用方法 **以及** Managed Identity Credential(托管身份凭据)): - -| 参数 | 描述 | -| ------------------- | ------------------------------------ | -| `connection_string` | 使用连接字符串进行身份验证。 | -| `account_name` | 使用共享密钥进行身份验证(与 `account_key` 配合使用)。 | -| `account_key` | 使用共享密钥进行身份验证(与 `account_name` 配合使用)。 | - -#### 限制参数 {#azure-blob-storage-limit-parameters} - -| 参数 | 描述 | -| ------------------------------------ | ------------------------------ | -| `s3_max_single_part_upload_size` | 向 Blob Storage 上传单个分块的最大大小。 | -| `min_bytes_for_seek` | 可随机访问(seekable)区域的最小大小。 | -| `max_single_read_retries` | 从 Blob Storage 读取一段数据的最大重试次数。 | -| `max_single_download_retries` | 从 Blob Storage 下载可读缓冲区的最大重试次数。 | -| `thread_pool_size` | 用于实例化 `IDiskRemote` 的最大线程数。 | -| `s3_max_inflight_parts_for_one_file` | 针对单个对象的最大并发 PUT 请求数。 | - -#### 其他参数 {#azure-blob-storage-other-parameters} - -| 参数 | 描述 | 默认值 | -| -------------------------------- | ---------------------------------------------------- | ---------------------------------------- | -| `metadata_path` | 用于存储 Blob Storage 元数据文件的本地文件系统路径。 | `/var/lib/clickhouse/disks//` | -| `skip_access_check` | 如果为 `true`,则在启动期间跳过磁盘访问检查。 | `false` | -| `read_resource` | 用于[调度](/operations/workload-scheduling.md)读取请求的资源名称。 | 空字符串(禁用) | -| `write_resource` | 用于[调度](/operations/workload-scheduling.md)写入请求的资源名称。 | 空字符串(禁用) | -| `metadata_keep_free_space_bytes` | 需要预留的元数据磁盘空闲空间大小。 | - | - -在集成测试目录中可以找到工作配置示例(例如 [test_merge_tree_azure_blob_storage](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_merge_tree_azure_blob_storage/configs/config.d/storage_conf.xml) 或 [test_azure_blob_storage_zero_copy_replication](https://github.com/ClickHouse/ClickHouse/blob/master/tests/integration/test_azure_blob_storage_zero_copy_replication/configs/config.d/storage_conf.xml))。 - -:::note 零拷贝复制尚未准备好用于生产环境 -在 ClickHouse 22.8 及更高版本中,零拷贝复制默认禁用。该功能不建议在生产环境中使用。 -::: - - -## 使用 HDFS 存储(不受支持) {#using-hdfs-storage-unsupported} - -在此示例配置中: - -* 磁盘类型为 `hdfs`(不受支持) -* 数据托管在 `hdfs://hdfs1:9000/clickhouse/` - -需要注意的是,HDFS 当前不受支持,因此在使用时可能会遇到问题。如果遇到问题并找到解决方案,欢迎提交 pull request 贡献修复。 - -```xml - - - - - hdfs - hdfs://hdfs1:9000/clickhouse/ - true - - - local - / - - - - - -
- hdfs -
- - hdd - -
-
-
-
-
-``` - -请注意,HDFS 在某些极端情况下可能无法正常工作。 - -### 使用数据加密 {#encrypted-virtual-file-system} - -可以对存储在 [S3](/engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-s3)、[HDFS](#using-hdfs-storage-unsupported)(不受支持)等外部磁盘,或本地磁盘上的数据进行加密。要启用加密模式,必须在配置文件中定义一个类型为 `encrypted` 的磁盘,并选择一个用于保存数据的底层磁盘。`encrypted` 磁盘会实时加密所有写入的文件,从 `encrypted` 磁盘读取文件时则会自动解密。因此,可以像使用普通磁盘一样使用 `encrypted` 磁盘。 - -磁盘配置示例: - -```xml - - - local - /path1/ - - - encrypted - disk1 - path2/ - _16_ascii_chars_ - - -``` - -例如,当 ClickHouse 将某个表中的数据写入文件 `store/all_1_1_0/data.bin` 到 `disk1` 时,该文件实际会写入物理磁盘路径 `/path1/store/all_1_1_0/data.bin`。 - -当将同一个文件写入 `disk2` 时,实际上会以加密方式写入物理磁盘路径 `/path1/path2/store/all_1_1_0/data.bin`。 - -### 必需参数 {#required-parameters-encrypted-disk} - -| Parameter | Type | Description | -| --------- | ------ | ------------------------------------------------------- | -| `type` | String | 必须设置为 `encrypted` 才能创建加密磁盘。 | -| `disk` | String | 用于底层存储的磁盘类型。 | -| `key` | Uint64 | 用于加密和解密的密钥。可以使用 `key_hex` 以十六进制形式指定。可以通过 `id` 属性指定多个密钥。 | - -### 可选参数 {#optional-parameters-encrypted-disk} - -| Parameter | Type | Default | Description | -| ---------------- | ------ | -------------- | -------------------------------------------------------------------------------------------------- | -| `path` | String | Root directory | 磁盘上保存数据的位置。 | -| `current_key_id` | String | - | 用于加密的密钥 ID。所有已指定的密钥都可用于解密。 | -| `algorithm` | Enum | `AES_128_CTR` | 加密算法。选项:
- `AES_128_CTR`(16 字节密钥)
- `AES_192_CTR`(24 字节密钥)
- `AES_256_CTR`(32 字节密钥) | - -磁盘配置示例: - - -```xml - - - - - s3 - ... - - - encrypted - disk_s3 - AES_128_CTR - 00112233445566778899aabbccddeeff - ffeeddccbbaa99887766554433221100 - 1 - - - - -``` - -### 使用本地缓存 {#using-local-cache} - -从 22.3 版本开始,可以在存储配置中为磁盘配置本地缓存。 -在 22.3 - 22.7 版本中,缓存仅支持 `s3` 磁盘类型。对于 >= 22.8 版本,缓存支持任意磁盘类型:S3、Azure、本地、加密等。 -对于 >= 23.5 版本,缓存仅支持远程磁盘类型:S3、Azure、HDFS(暂不支持)。 -缓存使用 `LRU` 缓存策略。 - -适用于 22.8 及以上版本的配置示例: - -```xml - - - - - s3 - ... - ... S3 配置 ... - - - cache - s3 - /s3_cache/ - 10Gi - - - - - -
- cache -
-
-
- -
-``` - -22.8 之前的版本配置示例: - -```xml - - - - - s3 - ... - ... S3 配置 ... - 1 - 10737418240 - - - - - -
- s3 -
-
-
- -
-``` - -文件缓存 **磁盘配置参数**: - -这些参数应在磁盘配置部分中定义。 - - -| Parameter | Type | Default | Description | -|---------------------------------------|---------|------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `path` | String | - | **必需**。用于存储缓存的目录路径。 | -| `max_size` | Size | - | **必需**。缓存的最大大小,可以是字节数或可读格式(例如 `10Gi`)。达到限制时,会使用 LRU 策略淘汰文件。支持 `ki`、`Mi`、`Gi` 格式(自 v22.10 起)。 | -| `cache_on_write_operations` | Boolean | `false` | 为 `INSERT` 查询和后台合并启用写穿缓存。可以通过 `enable_filesystem_cache_on_write_operations` 在每个查询级别进行覆盖。 | -| `enable_filesystem_query_cache_limit` | Boolean | `false` | 基于 `max_query_cache_size` 启用按查询的缓存大小限制。 | -| `enable_cache_hits_threshold` | Boolean | `false` | 启用后,仅在数据被多次读取后才会将其缓存。 | -| `cache_hits_threshold` | Integer | `0` | 将数据写入缓存前所需的读取次数(需要启用 `enable_cache_hits_threshold`)。 | -| `enable_bypass_cache_with_threshold` | Boolean | `false` | 对大范围读取跳过缓存。 | -| `bypass_cache_threshold` | Size | `256Mi` | 触发跳过缓存的读取范围大小(需要启用 `enable_bypass_cache_with_threshold`)。 | -| `max_file_segment_size` | Size | `8Mi` | 单个缓存文件的最大大小,可以是字节数或可读格式。 | -| `max_elements` | Integer | `10000000` | 最大缓存文件数量。 | -| `load_metadata_threads` | Integer | `16` | 启动时用于加载缓存元数据的线程数。 | - -> **注意**:Size 值支持 `ki`、`Mi`、`Gi` 等单位(例如 `10Gi`)。 - - - -## 文件缓存查询/配置文件设置 {#file-cache-query-profile-settings} - -| Setting | Type | Default | Description | -| ----------------------------------------------------------------------- | ------- | ----------------------- | --------------------------------------------------------------------------------------- | -| `enable_filesystem_cache` | Boolean | `true` | 针对单个查询启用或禁用缓存使用,即使使用的是 `cache` 磁盘类型。 | -| `read_from_filesystem_cache_if_exists_otherwise_bypass_cache` | Boolean | `false` | 启用后,仅在数据已存在于缓存中时才使用缓存;新的数据不会写入缓存。 | -| `enable_filesystem_cache_on_write_operations` | Boolean | `false` (Cloud: `true`) | 启用写穿缓存。需要在缓存配置中设置 `cache_on_write_operations`。 | -| `enable_filesystem_cache_log` | Boolean | `false` | 启用后会将详细的缓存使用情况记录到 `system.filesystem_cache_log`。 | -| `filesystem_cache_allow_background_download` | Boolean | `true` | 允许在后台完成部分已下载的分段。若禁用,则在当前查询/会话中始终在前台进行下载。 | -| `max_query_cache_size` | Size | `false` | 每个查询可使用的最大缓存大小。需要在缓存配置中启用 `enable_filesystem_query_cache_limit`。 | -| `filesystem_cache_skip_download_if_exceeds_per_query_cache_write_limit` | Boolean | `true` | 控制达到 `max_query_cache_size` 时的行为:
- `true`:停止下载新数据
- `false`:淘汰旧数据以为新数据腾出空间 | - -:::warning -缓存配置设置和缓存查询设置对应最新的 ClickHouse 版本, -在较早版本中,某些功能可能不受支持。 -::: - -#### 缓存系统表 {#cache-system-tables-file-cache} - -| Table Name | Description | Requirements | -| ----------------------------- | ------------------ | --------------------------------------- | -| `system.filesystem_cache` | 显示文件系统缓存的当前状态。 | 无 | -| `system.filesystem_cache_log` | 提供每个查询的详细缓存使用统计信息。 | 需要 `enable_filesystem_cache_log = true` | - -#### 缓存命令 {#cache-commands-file-cache} - -##### `SYSTEM DROP FILESYSTEM CACHE () (ON CLUSTER)` -- `ON CLUSTER` {#system-drop-filesystem-cache-on-cluster} - -仅当未提供 `` 时才支持此命令。 - -##### `SHOW FILESYSTEM CACHES` {#show-filesystem-caches} - -显示服务器上已配置的文件系统缓存列表。\ -(对于版本小于或等于 `22.8`,该命令名称为 `SHOW CACHES`) - -```sql title="Query" -显示文件系统缓存 -``` - -```text title="Response" -┌─缓存──────┐ -│ s3_cache │ -└───────────┘ -``` - -##### `DESCRIBE FILESYSTEM CACHE ''` {#describe-filesystem-cache} - -显示指定缓存的配置以及一些整体统计信息。 -缓存名称可以通过 `SHOW FILESYSTEM CACHES` 命令获取。(对于版本小于或等于 `22.8` 的 ClickHouse,该命令名为 `DESCRIBE CACHE`) - -```sql title="Query" -DESCRIBE FILESYSTEM CACHE 's3_cache' -``` - -```text title="Response" -┌────max_size─┬─max_elements─┬─max_file_segment_size─┬─boundary_alignment─┬─cache_on_write_operations─┬─cache_hits_threshold─┬─current_size─┬─current_elements─┬─path───────┬─background_download_threads─┬─enable_bypass_cache_with_threshold─┐ -│ 10000000000 │ 1048576 │ 104857600 │ 4194304 │ 1 │ 0 │ 3276 │ 54 │ /s3_cache/ │ 2 │ 0 │ -└─────────────┴──────────────┴───────────────────────┴────────────────────┴───────────────────────────┴──────────────────────┴──────────────┴──────────────────┴────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - - -| 缓存当前指标 | 缓存异步指标 | 缓存 Profile 事件 | -| ------------------------- | ---------------------- | ----------------------------------------------------------------------------------------- | -| `FilesystemCacheSize` | `FilesystemCacheBytes` | `CachedReadBufferReadFromSourceBytes`, `CachedReadBufferReadFromCacheBytes` | -| `FilesystemCacheElements` | `FilesystemCacheFiles` | `CachedReadBufferReadFromSourceMicroseconds`, `CachedReadBufferReadFromCacheMicroseconds` | -| | | `CachedReadBufferCacheWriteBytes`, `CachedReadBufferCacheWriteMicroseconds` | -| | | `CachedWriteBufferCacheWriteBytes`, `CachedWriteBufferCacheWriteMicroseconds` | - -### 使用静态 Web 存储(只读) {#web-storage} - -这是一个只读磁盘,其数据只会被读取,从不会被修改。通过 `ATTACH TABLE` 查询(见下方示例)将新表加载到该磁盘上。实际上不会使用本地磁盘,每个 `SELECT` 查询都会触发一次 `http` 请求以获取所需数据。所有对表数据的修改操作都会抛出异常,即不允许以下类型的查询:[`CREATE TABLE`](/sql-reference/statements/create/table.md)、[`ALTER TABLE`](/sql-reference/statements/alter/index.md)、[`RENAME TABLE`](/sql-reference/statements/rename#rename-table)、[`DETACH TABLE`](/sql-reference/statements/detach.md) 和 [`TRUNCATE TABLE`](/sql-reference/statements/truncate.md)。 -Web 存储适用于只读场景。典型用例包括托管示例数据或执行数据迁移。有一个名为 `clickhouse-static-files-uploader` 的工具,用于为给定表准备数据目录(`SELECT data_paths FROM system.tables WHERE name = 'table_name'`)。对于每个所需的表,都会得到一个包含文件的目录。这些文件可以上传到例如提供静态文件的 Web 服务器上。完成上述准备后,就可以通过 `DiskWeb` 将该表加载到任意 ClickHouse 服务器中。 - -在此示例配置中: - -* 磁盘类型为 `web` -* 数据托管在 `http://nginx:80/test1/` -* 使用了本地存储上的缓存 - -```xml - - - - - web - http://nginx:80/test1/ - - - cache - web - cached_web_cache/ - 100000000 - - - - - -
- web -
-
-
- - -
- cached_web -
-
-
-
-
-
-``` - -:::tip -如果某个 web 数据集预期不会被常规使用,也可以在查询中临时配置存储。 -参见 [dynamic configuration](#dynamic-configuration),即可跳过编辑配置文件的步骤。 - -一个 [示例数据集](https://github.com/ClickHouse/web-tables-demo) 托管在 GitHub 上。要将您自己的表准备好用于 web -存储,请参阅工具 [clickhouse-static-files-uploader](/operations/utilities/static-files-disk-uploader) -::: - -在这个 `ATTACH TABLE` 查询中,提供的 `UUID` 与数据所在目录的名称相匹配,endpoint 为指向 GitHub 原始内容的 URL。 - - -```sql --- highlight-next-line -ATTACH TABLE uk_price_paid UUID 'cf712b4f-2ca8-435c-ac23-c4393efe52f7' -( - price UInt32, - date Date, - postcode1 LowCardinality(String), - postcode2 LowCardinality(String), - type Enum8('other' = 0, 'terraced' = 1, 'semi-detached' = 2, 'detached' = 3, 'flat' = 4), - is_new UInt8, - duration Enum8('unknown' = 0, 'freehold' = 1, 'leasehold' = 2), - addr1 String, - addr2 String, - street LowCardinality(String), - locality LowCardinality(String), - town LowCardinality(String), - district LowCardinality(String), - county LowCardinality(String) -) -ENGINE = MergeTree -ORDER BY (postcode1, postcode2, addr1, addr2) - -- highlight-start - SETTINGS disk = disk( - type=web, - endpoint='https://raw.githubusercontent.com/ClickHouse/web-tables-demo/main/web/' - ); - -- highlight-end -``` - -一个现成的测试用例。你需要将如下配置添加到 config 中: - -```xml - - - - - web - https://clickhouse-datasets.s3.yandex.net/disk-with-static-files-tests/test-hits/ - - - - - -
- web -
-
-
-
-
-
-``` - -然后执行此查询: - - -```sql -ATTACH TABLE test_hits UUID '1ae36516-d62d-4218-9ae3-6516d62da218' -( - WatchID UInt64, - JavaEnable UInt8, - Title String, - GoodEvent Int16, - EventTime DateTime, - EventDate Date, - CounterID UInt32, - ClientIP UInt32, - ClientIP6 FixedString(16), - RegionID UInt32, - UserID UInt64, - CounterClass Int8, - OS UInt8, - UserAgent UInt8, - URL String, - Referer String, - URLDomain String, - RefererDomain String, - Refresh UInt8, - IsRobot UInt8, - RefererCategories Array(UInt16), - URLCategories Array(UInt16), - URLRegions Array(UInt32), - RefererRegions Array(UInt32), - ResolutionWidth UInt16, - ResolutionHeight UInt16, - ResolutionDepth UInt8, - FlashMajor UInt8, - FlashMinor UInt8, - FlashMinor2 String, - NetMajor UInt8, - NetMinor UInt8, - UserAgentMajor UInt16, - UserAgentMinor FixedString(2), - CookieEnable UInt8, - JavascriptEnable UInt8, - IsMobile UInt8, - MobilePhone UInt8, - MobilePhoneModel String, - Params String, - IPNetworkID UInt32, - TraficSourceID Int8, - SearchEngineID UInt16, - SearchPhrase String, - AdvEngineID UInt8, - IsArtifical UInt8, - WindowClientWidth UInt16, - WindowClientHeight UInt16, - ClientTimeZone Int16, - ClientEventTime DateTime, - SilverlightVersion1 UInt8, - SilverlightVersion2 UInt8, - SilverlightVersion3 UInt32, - SilverlightVersion4 UInt16, - PageCharset String, - CodeVersion UInt32, - IsLink UInt8, - IsDownload UInt8, - IsNotBounce UInt8, - FUniqID UInt64, - HID UInt32, - IsOldCounter UInt8, - IsEvent UInt8, - IsParameter UInt8, - DontCountHits UInt8, - WithHash UInt8, - HitColor FixedString(1), - UTCEventTime DateTime, - Age UInt8, - Sex UInt8, - Income UInt8, - Interests UInt16, - Robotness UInt8, - GeneralInterests Array(UInt16), - RemoteIP UInt32, - RemoteIP6 FixedString(16), - WindowName Int32, - OpenerName Int32, - HistoryLength Int16, - BrowserLanguage FixedString(2), - BrowserCountry FixedString(2), - SocialNetwork String, - SocialAction String, - HTTPError UInt16, - SendTiming Int32, - DNSTiming Int32, - ConnectTiming Int32, - ResponseStartTiming Int32, - ResponseEndTiming Int32, - FetchTiming Int32, - RedirectTiming Int32, - DOMInteractiveTiming Int32, - DOMContentLoadedTiming Int32, - DOMCompleteTiming Int32, - LoadEventStartTiming Int32, - LoadEventEndTiming Int32, - NSToDOMContentLoadedTiming Int32, - FirstPaintTiming Int32, - RedirectCount Int8, - SocialSourceNetworkID UInt8, - SocialSourcePage String, - ParamPrice Int64, - ParamOrderID String, - ParamCurrency FixedString(3), - ParamCurrencyID UInt16, - GoalsReached Array(UInt32), - OpenstatServiceName String, - OpenstatCampaignID String, - OpenstatAdID String, - OpenstatSourceID String, - UTMSource String, - UTMMedium String, - UTMCampaign String, - UTMContent String, - UTMTerm String, - FromTag String, - HasGCLID UInt8, - RefererHash UInt64, - URLHash UInt64, - CLID UInt32, - YCLID UInt64, - ShareService String, - ShareURL String, - ShareTitle String, - ParsedParams Nested( - Key1 String, - Key2 String, - Key3 String, - Key4 String, - Key5 String, - ValueDouble Float64), - IslandID FixedString(16), - RequestNum UInt32, - RequestTry UInt8 -) -ENGINE = MergeTree() -PARTITION BY toYYYYMM(EventDate) -ORDER BY (CounterID, EventDate, intHash32(UserID)) -SAMPLE BY intHash32(UserID) -SETTINGS storage_policy='web'; -``` - -#### 必填参数 {#static-web-storage-required-parameters} - -| Parameter | Description | -| ---------- | ------------------------------------------------------ | -| `type` | `web`。否则不会创建磁盘。 | -| `endpoint` | 使用 `path` 格式的端点 URL。端点 URL 必须包含一个用于存储数据的根路径,即数据上传到的路径。 | - -#### 可选参数 {#optional-parameters-web} - - -| 参数 | 描述 | 默认值 | -|-------------------------------------|--------------------------------------------------------------------------------|-----------------| -| `min_bytes_for_seek` | 使用 seek 操作而不是顺序读取所需的最小字节数 | `1` MB | -| `remote_fs_read_backoff_threashold` | 尝试从远程磁盘读取数据时的最大等待时间 | `10000` seconds | -| `remote_fs_read_backoff_max_tries` | 使用退避策略进行读取时的最大尝试次数 | `5` | - -如果查询失败并抛出异常 `DB:Exception Unreachable URL`,则可以尝试调整以下设置:[http_connection_timeout](/operations/settings/settings.md/#http_connection_timeout)、[http_receive_timeout](/operations/settings/settings.md/#http_receive_timeout)、[keep_alive_timeout](/operations/server-configuration-parameters/settings#keep_alive_timeout)。 - -要获取用于上传的文件,请运行: -`clickhouse static-files-disk-uploader --metadata-path --output-dir
`(`--metadata-path` 可通过查询 `SELECT data_paths FROM system.tables WHERE name = 'table_name'` 获取)。 - -当通过 `endpoint` 加载文件时,文件必须被加载到 `/store/` 路径下,但配置中只能包含 `endpoint`。 - -如果在服务器启动并加载表时无法访问磁盘上的 URL,则所有错误都会被捕获。如果在这种情况下发生了错误,可以通过 `DETACH TABLE table_name` -> `ATTACH TABLE table_name` 重新加载表(使其可见)。如果在服务器启动时元数据已成功加载,则表会立即可用。 - -使用 [http_max_single_read_retries](/operations/storing-data#web-storage) 设置来限制单次 HTTP 读取期间的最大重试次数。 - -### 零拷贝复制(尚未准备好用于生产环境) {#zero-copy} - -在 `S3` 磁盘以及(尚不支持的)`HDFS` 磁盘上可以进行零拷贝复制,但不推荐使用。零拷贝复制意味着:如果数据在多台机器上以远程方式存储且需要同步,则仅复制元数据(数据分片的路径),而不复制数据本身。 - -:::note 零拷贝复制尚未准备好用于生产环境 -在 ClickHouse 22.8 及更高版本中,零拷贝复制默认是禁用的。该功能不推荐用于生产环境。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md deleted file mode 100644 index 998028abac3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_insert_log.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '包含异步插入相关信息的系统表。每条记录代表一条被缓冲用于异步插入的 INSERT 查询。' -keywords: ['系统表', 'asynchronous_insert_log'] -slug: /operations/system-tables/asynchronous_insert_log -title: 'system.asynchronous_insert_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_insert_log {#systemasynchronous_insert_log} - - - -包含异步插入相关的信息。每条记录对应一条被缓冲到异步插入中的插入查询。 - -要开始记录日志,请在 [asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log) 部分中配置参数。 - -数据刷新周期通过服务器设置中 [asynchronous_insert_log](../../operations/server-configuration-parameters/settings.md#asynchronous_insert_log) 的 `flush_interval_milliseconds` 参数进行配置。要强制刷新,请使用 [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) 查询。 - -ClickHouse 不会自动从该表中删除数据。更多详情请参见 [Introduction](/operations/system-tables/overview#system-tables-introduction)。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 发生异步插入的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 异步插入执行完成的日期和时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 异步插入执行完成的日期和时间,精确到微秒。 -* `query` ([String](../../sql-reference/data-types/string.md)) — 查询字符串。 -* `database` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 -* `table` ([String](../../sql-reference/data-types/string.md)) — 表名。 -* `format` ([String](/sql-reference/data-types/string.md)) — 格式名称。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 初始查询的 ID。 -* `bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 插入的字节数。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 异常信息。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — 视图状态。取值: - * `'Ok' = 1` — 插入成功。 - * `'ParsingError' = 2` — 解析数据时抛出的异常。 - * `'FlushError' = 3` — 刷新数据时抛出的异常。 -* `flush_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 发生刷新操作的日期和时间。 -* `flush_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 发生刷新操作的日期和时间,精确到微秒。 -* `flush_query_id` ([String](../../sql-reference/data-types/string.md)) — 刷新查询的 ID。 - -**示例** - -查询: - -```sql -SELECT * FROM system.asynchronous_insert_log LIMIT 1 \G; -``` - -结果: - -```text -主机名: clickhouse.eu-central1.internal -事件日期: 2023-06-08 -事件时间: 2023-06-08 10:08:53 -事件时间_微秒: 2023-06-08 10:08:53.199516 -查询: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -数据库: public -表: data_guess -格式: CSV -查询_id: b46cd4c4-0269-4d0b-99f5-d27668c6102e -字节: 133223 -异常: -状态: Ok -刷新时间: 2023-06-08 10:08:55 -刷新时间_微秒: 2023-06-08 10:08:55.139676 -刷新查询_id: cd2c1e43-83f5-49dc-92e4-2fbc7f8d3716 -``` - -**另请参阅** - -* [system.query_log](../../operations/system-tables/query_log) — `query_log` 系统表的说明,其中包含关于查询执行的常规信息。 -* [system.asynchronous_inserts](/operations/system-tables/asynchronous_inserts) — 此表包含关于队列中待处理异步插入操作的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md deleted file mode 100644 index c0cce87f105..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_inserts.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '包含队列中待处理异步插入信息的系统表。' -keywords: ['系统表', 'asynchronous_inserts'] -slug: /operations/system-tables/asynchronous_inserts -title: 'system.asynchronous_inserts' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含关于队列中待处理异步插入的相关信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -查询: - -```sql -SELECT * FROM system.asynchronous_inserts LIMIT 1 \G; -``` - -结果: - -```text -Row 1: -────── -query: INSERT INTO public.data_guess (user_id, datasource_id, timestamp, path, type, num, str) FORMAT CSV -database: public -table: data_guess -format: CSV -first_update: 2023-06-08 10:08:54.199606 -total_bytes: 133223 -entries.query_id: ['b46cd4c4-0269-4d0b-99f5-d27668c6102e'] -entries.bytes: [133223] -``` - -**另请参阅** - -* [system.query_log](/operations/system-tables/query_log) — `query_log` 系统表的描述,其中包含有关查询执行的一般信息。 -* [system.asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) — 此表包含已执行异步插入操作的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md deleted file mode 100644 index f7acf5be651..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_loader.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: '包含关于最近异步作业(例如正在加载的表)的信息和状态的系统表。该表中每个作业对应一行记录。' -keywords: ['系统表', 'asynchronous_loader'] -slug: /operations/system-tables/asynchronous_loader -title: 'system.asynchronous_loader' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_loader {#systemasynchronous_loader} - - - -包含近期异步任务的信息和状态(例如表加载任务)。该表中每个任务对应一行记录。可以使用工具 `utils/async_loader_graph` 对该表中的信息进行可视化。 - -示例: - -```sql -SELECT * -FROM system.asynchronous_loader -LIMIT 1 -FORMAT Vertical -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -挂起的作业可能处于以下状态之一: - -* `is_executing` (`UInt8`) - 当前有 worker 正在执行该作业。 -* `is_blocked` (`UInt8`) - 该作业正在等待其依赖完成。 -* `is_ready` (`UInt8`) - 该作业已准备好执行并在等待 worker。 -* `elapsed` (`Float64`) - 自开始执行以来经过的秒数。如果作业尚未启动则为 0;如果作业已完成则为总执行时间。 - -每个作业都有一个与之关联的池,并在该池中启动。每个池都有一个固定的优先级,以及一个可变的最大 worker 数量。优先级越高(`priority` 值越小)的作业越先运行。当至少有一个高优先级作业处于就绪或执行状态时,不会启动低优先级作业。可以提升作业的优先级(但不能降低)。例如,如果某个查询需要某张表,那么该表的加载和启动作业将被优先处理。可以在作业执行期间提升其优先级,但作业不会从其 `execution_pool` 迁移到新分配的 `pool`。该作业在创建新作业时使用其 `pool`,以避免优先级反转。已经启动的作业不会被更高优先级的作业抢占,并且一旦开始就会始终运行至完成。 - -* `pool_id` (`UInt64`) - 当前分配给该作业的池的 ID。 - -* `pool` (`String`) - `pool_id` 对应池的名称。 - -* `priority` (`Int64`) - `pool_id` 对应池的优先级。 - -* `execution_pool_id` (`UInt64`) - 实际执行该作业的池的 ID。执行开始前等于最初分配的池。 - -* `execution_pool` (`String`) - `execution_pool_id` 对应池的名称。 - -* `execution_priority` (`Int64`) - `execution_pool_id` 对应池的优先级。 - -* `ready_seqno` (`Nullable(UInt64)`) - 对就绪作业为非空。worker 会从其池的就绪队列中拉取下一个将要执行的作业。如果存在多个就绪作业,则选择 `ready_seqno` 值最小的作业。 - -* `waiters` (`UInt64`) - 等待该作业的线程数量。 - -* `exception` (`Nullable(String)`) - 对失败和被取消的作业为非空。保存查询执行期间抛出的错误消息,或导致该作业被取消的错误,以及因依赖失败产生的作业名称链。 - -作业生命周期中的时间点: - -* `schedule_time` (`DateTime64`) - 作业被创建并调度执行的时间(通常包括其所有依赖)。 -* `enqueue_time` (`Nullable(DateTime64)`) - 作业变为就绪并被加入其池的就绪队列的时间。如果作业尚未就绪则为 Null。 -* `start_time` (`Nullable(DateTime64)`) - worker 从就绪队列中取出该作业并开始执行的时间。如果作业尚未开始则为 Null。 -* `finish_time` (`Nullable(DateTime64)`) - 作业执行完成的时间。如果作业尚未完成则为 Null。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md deleted file mode 100644 index e9a320d1c64..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metric_log.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '系统表,包含 `system.asynchronous_metrics` 的历史值,这些值会按时间间隔(默认每秒一次)保存' -keywords: ['系统表', 'asynchronous_metric_log'] -slug: /operations/system-tables/asynchronous_metric_log -title: 'system.asynchronous_metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含 `system.asynchronous_metrics` 的历史值,这些值会按固定时间间隔保存一次(默认每秒)。默认启用。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `metric` ([String](../../sql-reference/data-types/string.md)) — 指标名称。 -* `value` ([Float64](../../sql-reference/data-types/float.md)) — 指标值。 - -**示例** - -```sql -SELECT * FROM system.asynchronous_metric_log LIMIT 3 \G -``` - -```text -第 1 行: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:07 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0.001 - -第 2 行: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:08 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 - -第 3 行: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-11-14 -event_time: 2023-11-14 14:39:09 -metric: AsynchronousHeavyMetricsCalculationTimeSpent -value: 0 -``` - -**另请参阅** - -* [asynchronous_metric_log setting](../../operations/server-configuration-parameters/settings.md#asynchronous_metric_log) — 用于启用或禁用该设置。 -* [system.asynchronous_metrics](../system-tables/asynchronous_metrics.md) — 包含在后台周期性计算的指标。 -* [system.metric_log](../system-tables/metric_log.md) — 包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录,并会定期刷写到磁盘。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md deleted file mode 100644 index 2c900cdfc92..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/asynchronous_metrics.md +++ /dev/null @@ -1,633 +0,0 @@ ---- -description: '包含在后台定期计算得到的指标的系统表。例如,当前正在使用的 RAM 数量。' -keywords: ['system table', 'asynchronous_metrics'] -slug: /operations/system-tables/asynchronous_metrics -title: 'system.asynchronous_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.asynchronous_metrics {#systemasynchronous_metrics} - - - -包含在后台周期性计算得到的指标。例如,当前正在使用的内存量。 - -列: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — 指标名称。 -* `value` ([Float64](../../sql-reference/data-types/float.md)) — 指标值。 -* `description` ([String](../../sql-reference/data-types/string.md)) — 指标描述 - -**示例** - -```sql -SELECT * FROM system.asynchronous_metrics LIMIT 10 -``` - -```text -┌─metric──────────────────────────────────┬──────value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ AsynchronousMetricsCalculationTimeSpent │ 0.00179053 │ 计算异步指标所花费的时间(秒)(这是异步指标的开销)。 │ -│ NumberOfDetachedByUserParts │ 0 │ 用户通过 `ALTER TABLE DETACH` 查询从 MergeTree 表中分离的数据部分总数(相对于意外、损坏或被忽略的部分)。服务器不关心已分离的部分,可以将其删除。 │ -│ NumberOfDetachedParts │ 0 │ 从 MergeTree 表中分离的数据部分总数。数据部分可以由用户通过 `ALTER TABLE DETACH` 查询分离,也可以在部分损坏、意外或不需要时由服务器自动分离。服务器不关心已分离的部分,可以将其删除。 │ -│ TotalRowsOfMergeTreeTables │ 2781309 │ 所有 MergeTree 系列表中存储的行(记录)总数。 │ -│ TotalBytesOfMergeTreeTables │ 7741926 │ 所有 MergeTree 系列表中存储的字节总数(压缩后,包括数据和索引)。 │ -│ NumberOfTables │ 93 │ 服务器上所有数据库中的表总数,不包括无法包含 MergeTree 表的数据库。被排除的数据库引擎是那些动态生成表集合的引擎,例如 `Lazy`、`MySQL`、`PostgreSQL`、`SQlite`。 │ -│ NumberOfDatabases │ 6 │ 服务器上的数据库总数。 │ -│ MaxPartCountForPartition │ 6 │ 所有 MergeTree 系列表的所有分区中每个分区的最大数据部分数。值大于 300 表示配置错误、过载或大量数据加载。 │ -│ ReplicasSumMergesInQueue │ 0 │ 所有 Replicated 表的队列中待应用的合并操作总数。 │ -│ ReplicasSumInsertsInQueue │ 0 │ 所有 Replicated 表的队列中待复制的 INSERT 操作总数。 │ -└─────────────────────────────────────────┴────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -{/*- 与 system.events 和 system.metrics 不同,异步指标并不是在某个源代码文件中的简单列表里集中定义—— - 它们和 src/Interpreters/ServerAsynchronousMetrics.cpp 中的逻辑混在一起。 - 在这里专门将它们显式列出,便于读者查阅。 -*/ } - -## 指标说明 {#metric-descriptions} - -### AsynchronousHeavyMetricsCalculationTimeSpent {#asynchronousheavymetricscalculationtimespent} - -用于计算异步高开销(与表相关)指标所花费的时间(以秒为单位)(这是异步指标的额外开销)。 - -### AsynchronousHeavyMetricsUpdateInterval {#asynchronousheavymetricsupdateinterval} - -异步高开销(与表相关)指标的更新间隔。 - -### AsynchronousMetricsCalculationTimeSpent {#asynchronousmetricscalculationtimespent} - -用于计算异步指标所花费的时间(以秒为单位)(这是异步指标的额外开销)。 - -### AsynchronousMetricsUpdateInterval {#asynchronousmetricsupdateinterval} - -指标更新间隔。 - -### BlockActiveTime_*name* {#blockactivetime_name} - -块设备存在排队 I/O 请求的时间(以秒为单位)。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardBytes_*name* {#blockdiscardbytes_name} - -块设备上被丢弃的字节数。这些操作与 SSD 相关。丢弃操作不会被 ClickHouse 使用,但可以被系统中的其他进程使用。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardMerges_*name* {#blockdiscardmerges_name} - -从块设备请求的丢弃操作中,由操作系统 I/O 调度器合并后的操作数量。这些操作与 SSD 相关。丢弃操作不会被 ClickHouse 使用,但可以被系统中的其他进程使用。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardOps_*name* {#blockdiscardops_name} - -从块设备请求的丢弃操作数量。这些操作与 SSD 相关。丢弃操作不会被 ClickHouse 使用,但可以被系统中的其他进程使用。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockDiscardTime_*name* {#blockdiscardtime_name} - -在从块设备请求的丢弃操作中所花费的总时间(以秒为单位),为所有操作耗时之和。这些操作与 SSD 相关。丢弃操作不会被 ClickHouse 使用,但可以被系统中的其他进程使用。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockInFlightOps_*name* {#blockinflightops_name} - -该值统计已发送给设备驱动但尚未完成的 I/O 请求数量。不包括仍在队列中但尚未发送给设备驱动的 I/O 请求。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockQueueTime_*name* {#blockqueuetime_name} - -该值统计 I/O 请求在此块设备上等待的时间(以毫秒为单位)。如果有多个 I/O 请求在等待,该值将按“毫秒数 × 等待请求数”的乘积进行累加增长。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadBytes_*name* {#blockreadbytes_name} - -从块设备读取的字节数。由于使用了可以减少 I/O 的操作系统页缓存,该值可能低于从文件系统读取的字节数。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadMerges_*name* {#blockreadmerges_name} - -从块设备请求的读操作中,由操作系统 I/O 调度器合并后的操作数量。这是一个系统级指标,包含宿主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadOps_*name* {#blockreadops_name} - -### BlockReadOps_*name* {#blockreadops_name} - -从块设备请求的读操作次数。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockReadTime_*name* {#blockreadtime_name} - -在块设备上执行读操作所花费的时间(以秒为单位),为所有操作时间的总和。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteBytes_*name* {#blockwritebytes_name} - -写入块设备的字节数。由于使用了操作系统的页面缓存,该值可能小于写入文件系统的字节数,因为页面缓存可以减少 IO。由于直写缓存(write-through caching),对块设备的写入可能晚于对应的文件系统写入。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteMerges_*name* {#blockwritemerges_name} - -从块设备请求并由操作系统 IO 调度器合并在一起的写操作次数。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteOps_*name* {#blockwriteops_name} - -从块设备请求的写操作次数。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### BlockWriteTime_*name* {#blockwritetime_name} - -在块设备上执行写操作所花费的时间(以秒为单位),为所有操作时间的总和。这是一个系统范围指标,包含主机上所有进程,而不仅仅是 clickhouse-server。来源:`/sys/block`。参见 https://www.kernel.org/doc/Documentation/block/stat.txt - -### CPUFrequencyMHz_*name* {#cpufrequencymhz_name} - -CPU 的当前频率,单位为 MHz。大多数现代 CPU 会为节能和 Turbo Boost 而动态调整频率。 - -### DictionaryMaxUpdateDelay {#dictionarymaxlastsuccessfulupdatetime} - -字典更新的最大延迟时间(秒)。 - -### DictionaryTotalFailedUpdates {#dictionaryloadfailed} - -自上次成功加载以来,所有字典中的错误总次数。 - -### DiskAvailable_*name* {#diskavailable_name} - -磁盘(虚拟文件系统)上的可用字节数。远程文件系统可能会显示类似 16 EiB 这样很大的值。 - -### DiskTotal_*name* {#disktotal_name} - -磁盘(虚拟文件系统)的总大小(字节)。远程文件系统可能会显示类似 16 EiB 这样很大的值。 - -### DiskUnreserved_*name* {#diskunreserved_name} - -磁盘(虚拟文件系统)上可用的字节数,不包括为合并(merge)、拉取(fetch)和移动(move)预留的空间。远程文件系统可能会显示类似 16 EiB 这样很大的值。 - -### DiskUsed_*name* {#diskused_name} - -磁盘(虚拟文件系统)上已使用的字节数。远程文件系统并不总是提供该信息。 - -### FilesystemCacheBytes {#filesystemcachebytes} - -`cache` 虚拟文件系统中的总字节数。该缓存存储在磁盘上。 - -### FilesystemCacheFiles {#filesystemcachefiles} - -`cache` 虚拟文件系统中已缓存文件分段的总数量。该缓存存储在磁盘上。 - -### FilesystemLogsPathAvailableBytes {#filesystemlogspathavailablebytes} - -ClickHouse 日志路径所在卷上可用的字节数。如果该值接近零,应在配置文件中调整日志轮转策略。 - -### FilesystemLogsPathAvailableINodes {#filesystemlogspathavailableinodes} - -ClickHouse 日志路径所在卷上可用的 inode 数量。 - -### FilesystemLogsPathTotalBytes {#filesystemlogspathtotalbytes} - -ClickHouse 日志路径所在卷的大小(字节)。建议为日志预留至少 10 GB 的空间。 - -### FilesystemLogsPathTotalINodes {#filesystemlogspathtotalinodes} - -ClickHouse 日志路径所在卷上的 inode 总数。 - -### FilesystemLogsPathUsedBytes {#filesystemlogspathusedbytes} - -ClickHouse 日志路径所在卷上已使用的字节数。 - -### FilesystemLogsPathUsedINodes {#filesystemlogspathusedinodes} - -ClickHouse 日志路径所在卷上已使用的 inode 数量。 - -### FilesystemMainPathAvailableBytes {#filesystemmainpathavailablebytes} - -主 ClickHouse 路径所挂载卷上可用的字节数。 - -### FilesystemMainPathAvailableINodes {#filesystemmainpathavailableinodes} - -主 ClickHouse 路径所挂载卷上可用的 inode 数量。如果该值接近零,表示存在配置错误,即使磁盘未满也会收到 `no space left on device` 错误。 - -### FilesystemMainPathTotalBytes {#filesystemmainpathtotalbytes} - -主 ClickHouse 路径所挂载卷的总大小(以字节为单位)。 - -### FilesystemMainPathTotalINodes {#filesystemmainpathtotalinodes} - -主 ClickHouse 路径所挂载卷上的 inode 总数。如果小于 2500 万,表示存在配置错误。 - -### FilesystemMainPathUsedBytes {#filesystemmainpathusedbytes} - -主 ClickHouse 路径所挂载卷上已使用的字节数。 - -### FilesystemMainPathUsedINodes {#filesystemmainpathusedinodes} - -主 ClickHouse 路径所挂载卷上已使用的 inode 数量。该值大致对应文件的数量。 - -### HTTPThreads {#httpthreads} - -HTTP 接口服务器中的线程数(不含 TLS)。 - -### InterserverThreads {#interserverthreads} - -副本通信协议服务器中的线程数(不含 TLS)。 - -### Jitter {#jitter} - -用于计算异步指标的线程被调度唤醒的预期时间与实际被唤醒时间之间的时间差。是整体系统延迟和响应性的间接指标。 - -### LoadAverage*N* {#loadaveragen} - -整个系统在 1 分钟内使用指数平滑得到的平均负载。负载表示当前在所有进程中(即 OS 内核的调度实体)由 CPU 正在运行、等待 I/O,或已准备运行但此刻尚未被调度的线程数量。这个数字包含所有进程,而不仅仅是 clickhouse-server。如果系统过载,许多进程已准备运行但在等待 CPU 或 I/O,那么该数值可以大于 CPU 内核数。 - -### MaxPartCountForPartition {#maxpartcountforpartition} - -所有 MergeTree 系列表中所有分区中,每个分区包含的数据分片的最大数量。如果该值大于 300,表示存在配置错误、系统过载或正在进行大规模数据导入。 - -### MemoryCode {#memorycode} - -为服务器进程的机器代码页面映射的虚拟内存量,单位为字节。 - -### MemoryDataAndStack {#memorydataandstack} - -为栈使用以及已分配内存映射的虚拟内存量,单位为字节。是否包含每个线程的栈以及大部分通过 `mmap` 系统调用分配的内存未作明确规定。此指标仅为完整性而存在。建议使用 `MemoryResident` 指标进行监控。 - -### MemoryResidentMax {#memoryresidentmax} - -服务器进程使用的物理内存最大值,单位为字节。 - -### MemoryResident {#memoryresident} - -服务器进程当前使用的物理内存量,单位为字节。 - -### MemoryShared {#memoryshared} - -服务器进程使用且同时被其他进程共享的内存量,单位为字节。ClickHouse 不使用共享内存,但部分内存可能会因操作系统自身原因被标记为共享。监控该指标意义不大,它仅为完整性而存在。 - -### MemoryVirtual {#memoryvirtual} - -服务器进程已分配的虚拟地址空间大小,单位为字节。虚拟地址空间大小通常远大于实际物理内存消耗,不应将其用作内存消耗估算。该指标取值较大是完全正常的,仅在技术层面上有意义。 - -### MySQLThreads {#mysqlthreads} - -MySQL 兼容协议服务器中的线程数。 - -### NetworkReceiveBytes_*name* {#networkreceivebytes_name} - -通过网络接口接收的字节数。该指标是系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkReceiveDrop_*name* {#networkreceivedrop_name} - -通过网络接口接收时被丢弃的数据包的总字节数。该指标是系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkReceiveErrors_*name* {#networkreceiveerrors_name} - -通过网络接口接收时发生错误的次数。该指标是系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkReceivePackets_*name* {#networkreceivepackets_name} - -通过网络接口接收的网络数据包数量。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkSendBytes_*name* {#networksendbytes_name} - -通过网络接口发送的字节数。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkSendDrop_*name* {#networksenddrop_name} - -通过网络接口发送时被丢弃的数据包次数。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkSendErrors_*name* {#networksenderrors_name} - -通过网络接口发送时发生错误(例如 TCP 重传)的次数。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NetworkSendPackets_*name* {#networksendpackets_name} - -通过网络接口发送的网络数据包数量。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### NumberOfDatabases {#numberofdatabases} - -服务器上的数据库总数。 - -### NumberOfDetachedByUserParts {#numberofdetachedbyuserparts} - -由用户通过 `ALTER TABLE DETACH` 查询从 MergeTree 表中分离出来的部件总数(与异常的、损坏的或被忽略的部件相对)。服务器不关心已分离的部件,它们可以被删除。 - -### NumberOfDetachedParts {#numberofdetachedparts} - -从 MergeTree 表中分离出来的部件总数。部件可以由用户通过 `ALTER TABLE DETACH` 查询分离,或者在部件损坏、异常或不再需要时由服务器自身分离。服务器不关心已分离的部件,它们可以被删除。 - -### NumberOfTables {#numberoftables} - -服务器上所有数据库中的表的总数,不包括那些不能包含 MergeTree 表的数据库。被排除的数据库引擎是那些按需动态生成表集合的引擎,例如 `Lazy`、`MySQL`、`PostgreSQL`、`SQlite`。 - -### OSContextSwitches {#oscontextswitches} - -主机系统发生的上下文切换次数。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSGuestNiceTime {#osguestnicetime} - -在 Linux 内核控制下,为来宾操作系统运行虚拟 CPU 且来宾被设置为较高优先级时所花费时间的比例(参见 `man procfs`)。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。此指标与 ClickHouse 无关,仅为完整性而存在。单个 CPU 核心的取值位于区间 [0..1]。所有 CPU 核心的值是对各核心求和得到,位于区间 [0..num cores]。 - -### OSGuestNiceTimeCPU_*N* {#osguestnicetimecpu_n} - -在 Linux 内核控制下,为来宾操作系统运行虚拟 CPU 且来宾被设置为较高优先级时所花费时间的比例(参见 `man procfs`)。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。此指标与 ClickHouse 无关,仅为完整性而存在。单个 CPU 核心的取值位于区间 [0..1]。所有 CPU 核心的值是对各核心求和得到,位于区间 [0..num cores]。 - -### OSGuestNiceTimeNormalized {#osguestnicetimenormalized} - -该值类似于 `OSGuestNiceTime`,但会再除以 CPU 核心数,使其无论核心数量多少都位于 [0..1] 区间。这使可以在集群中跨多台服务器对该指标取平均,即使各服务器的核心数不一致,仍然可以得到平均资源利用率指标。 - -### OSGuestTime {#osguesttime} - -在 Linux 内核控制下,为来宾操作系统运行虚拟 CPU 所花费时间的比例(参见 `man procfs`)。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。此指标与 ClickHouse 无关,仅为完整性而存在。单个 CPU 核心的取值位于区间 [0..1]。所有 CPU 核心的值是对各核心求和得到,位于区间 [0..num cores]。 - -### OSGuestTimeCPU_*N* {#osguesttimecpu_n} - -在 Linux 内核控制下为来宾操作系统运行虚拟 CPU 所花费时间的比例(参见 `man procfs`)。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。此指标对 ClickHouse 本身并无实际意义,但为完整性起见仍然保留。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSGuestTimeNormalized {#osguesttimenormalized} - -该值类似于 `OSGuestTime`,但会除以 CPU 核心数量,使其在 [0..1] 区间内进行度量,而与核心数量无关。这使你能够在集群中跨多台服务器对该指标的值进行平均,即使核心数量不一致,仍然可以得到平均资源使用率指标。 - -### OSIOWaitTime {#osiowaittime} - -CPU 核心未运行代码、且由于进程在等待 IO,OS 内核没有在该 CPU 上运行任何其他进程的时间比例。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSIOWaitTimeCPU_*N* {#osiowaittimecpu_n} - -CPU 核心未运行代码、且由于进程在等待 IO,OS 内核没有在该 CPU 上运行任何其他进程的时间比例。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSIOWaitTimeNormalized {#osiowaittimenormalized} - -该值类似于 `OSIOWaitTime`,但会除以 CPU 核心数量,使其在 [0..1] 区间内进行度量,而与核心数量无关。这使你能够在集群中跨多台服务器对该指标的值进行平均,即使核心数量不一致,仍然可以得到平均资源使用率指标。 - -### OSIdleTime {#osidletime} - -从 OS 内核的角度看,CPU 核心处于空闲状态(甚至没有准备好去运行等待 IO 的进程)的时间比例。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。这不包括由于 CPU 内部原因(内存加载、流水线停顿、分支预测失败、运行另一个 SMT 线程)导致 CPU 未被充分利用的时间。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSIdleTimeCPU_*N* {#osidletimecpu_n} - -从 OS 内核的角度看,CPU 核心处于空闲状态(甚至没有准备好去运行等待 IO 的进程)的时间比例。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。这不包括由于 CPU 内部原因(内存加载、流水线停顿、分支预测失败、运行另一个 SMT 线程)导致 CPU 未被充分利用的时间。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSIdleTimeNormalized {#osidletimenormalized} - -该值类似于 `OSIdleTime`,但会除以 CPU 核心数量,使其在 [0..1] 区间内进行度量,而与核心数量无关。这使你能够在集群中跨多台服务器对该指标的值进行平均,即使核心数量不一致,仍然可以得到平均资源使用率指标。 - -### OSInterrupts {#osinterrupts} - -主机上的中断次数。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。 - -### OSIrqTime {#osirqtime} - -在 CPU 上处理硬件中断请求所花费时间的比例。这是一个系统级指标,包含主机上的所有进程,而不仅仅是 clickhouse-server。该指标数值过高可能表明硬件配置错误或非常高的网络负载。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对它们求和计算,范围为 [0..num cores]。 - -### OSIrqTimeCPU_*N* {#osirqtimecpu_n} - -CPU 上用于处理硬件中断请求的时间占比。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。该指标值过高可能表示硬件配置错误或异常高的网络负载。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值是对各核心值求和得到的,总区间为 [0..num cores]。 - -### OSIrqTimeNormalized {#osirqtimenormalized} - -该值与 `OSIrqTime` 类似,但除以 CPU 核心数,使其在 [0..1] 区间内度量,与核心数量无关。这样,即使集群中各服务器的核心数不一致,也可以对该指标在多台服务器之间求平均,并仍然得到平均资源利用率指标。 - -### OSMemoryAvailable {#osmemoryavailable} - -可供程序使用的内存总量,单位为字节。与 `OSMemoryFreePlusCached` 指标非常相似。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSMemoryBuffers {#osmemorybuffers} - -被操作系统内核缓冲区使用的内存量,单位为字节。该值通常应当较小,如果数值很大,可能表明操作系统配置不当。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSMemoryCached {#osmemorycached} - -被操作系统页缓存使用的内存量,单位为字节。通常,几乎所有可用内存都会被操作系统页缓存使用——因此该指标值较高是正常且预期的。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSMemoryFreePlusCached {#osmemoryfreepluscached} - -主机系统上的空闲内存加上操作系统页缓存内存之和,单位为字节。这部分内存可供程序使用。该值应与 `OSMemoryAvailable` 非常接近。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSMemoryFreeWithoutCached {#osmemoryfreewithoutcached} - -主机系统上的空闲内存量,单位为字节。不包括操作系统页缓存所使用的内存。页缓存内存同样可供程序使用,因此该指标值可能会让人困惑。建议参阅 `OSMemoryAvailable` 指标。为方便起见,我们还提供了 `OSMemoryFreePlusCached` 指标,其值应与 OSMemoryAvailable 大致相近。另请参考 https://www.linuxatemyram.com/。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSMemoryTotal {#osmemorytotal} - -主机系统上的内存总量,单位为字节。 - -### OSNiceTime {#osnicetime} - -CPU 核心运行高优先级用户态代码的时间占比。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值是对各核心值求和得到的,总区间为 [0..num cores]。 - -### OSNiceTimeCPU_*N* {#osnicetimecpu_n} - -CPU 核心运行高优先级用户态代码的时间占比。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值是对各核心值求和得到的,总区间为 [0..num cores]。 - -### OSNiceTimeNormalized {#osnicetimenormalized} - -该值与 `OSNiceTime` 类似,但除以 CPU 核心数,使其在 [0..1] 区间内度量,与核心数量无关。这样,即使集群中各服务器的核心数不一致,也可以对该指标在多台服务器之间求平均,并仍然得到平均资源利用率指标。 - -### OSOpenFiles {#osopenfiles} - -主机上已打开文件的总数量。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSProcessesBlocked {#osprocessesblocked} - -等待 I/O 完成而被阻塞的线程数量(参见 `man procfs`)。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。 - -### OSProcessesCreated {#osprocessescreated} - -创建的进程数量。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。 - -### OSProcessesRunning {#osprocessesrunning} - -操作系统中可运行(正在运行或准备运行)线程的数量。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。 - -### OSSoftIrqTime {#ossoftirqtime} - -CPU 用于处理软件中断请求的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。该指标数值较高可能表明系统上运行的软件效率较低。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSSoftIrqTimeCPU_*N* {#ossoftirqtimecpu_n} - -CPU 用于处理软件中断请求的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。该指标数值较高可能表明系统上运行的软件效率较低。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSSoftIrqTimeNormalized {#ossoftirqtimenormalized} - -该值与 `OSSoftIrqTime` 类似,但再除以 CPU 核心数,使其在 [0..1] 区间内度量,而不受核心数量影响。这样可以在集群中跨多台服务器对该指标进行平均,即使各服务器的核心数不一致,仍然能够得到平均资源利用率指标。 - -### OSStealTime {#osstealtime} - -在虚拟化环境中运行时,CPU 花费在其他操作系统上的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。并非所有虚拟化环境都会提供该指标,而且大多数都不会。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSStealTimeCPU_*N* {#osstealtimecpu_n} - -在虚拟化环境中运行时,CPU 花费在其他操作系统上的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。并非所有虚拟化环境都会提供该指标,而且大多数都不会。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSStealTimeNormalized {#osstealtimenormalized} - -该值与 `OSStealTime` 类似,但再除以 CPU 核心数,使其在 [0..1] 区间内度量,而不受核心数量影响。这样可以在集群中跨多台服务器对该指标进行平均,即使各服务器的核心数不一致,仍然能够得到平均资源利用率指标。 - -### OSSystemTime {#ossystemtime} - -CPU 核心运行操作系统内核(system)代码的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSSystemTimeCPU_*N* {#ossystemtimecpu_n} - -CPU 核心运行操作系统内核(system)代码的时间占比。该指标是系统范围的,包含主机上的所有进程,而不仅仅是 clickhouse-server。单个 CPU 核心的取值区间为 [0..1]。所有 CPU 核心的值计算为各核心值之和,区间为 [0..num cores]。 - -### OSSystemTimeNormalized {#ossystemtimenormalized} - -该值与 `OSSystemTime` 类似,但再除以 CPU 核心数,使其在 [0..1] 区间内度量,而不受核心数量影响。这样可以在集群中跨多台服务器对该指标进行平均,即使各服务器的核心数不一致,仍然能够得到平均资源利用率指标。 - -### OSThreadsRunnable {#osthreadsrunnable} - -OS 内核调度器所看到的“可运行”线程总数。 - -### OSThreadsTotal {#osthreadstotal} - -在操作系统内核调度器视角下的线程总数。 - -### OSUptime {#osuptime} - -主机服务器(运行 ClickHouse 的机器)的运行时间,以秒为单位。 - -### OSUserTime {#osusertime} - -CPU 核心运行用户态代码所占用时间的比例。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。这还包括由于 CPU 内部原因导致 CPU 未被充分利用的时间(内存加载、流水线停顿、分支预测失败、运行另一个 SMT 核心)。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对各核心求和计算得到,范围为 [0..num cores]。 - -### OSUserTimeCPU_*N* {#osusertimecpu_n} - -CPU 核心运行用户态代码所占用时间的比例。这是一个系统级指标,包含主机上所有进程,而不仅仅是 clickhouse-server。这还包括由于 CPU 内部原因导致 CPU 未被充分利用的时间(内存加载、流水线停顿、分支预测失败、运行另一个 SMT 核心)。单个 CPU 核心的取值范围为 [0..1]。所有 CPU 核心的取值通过对各核心求和计算得到,范围为 [0..num cores]。 - -### OSUserTimeNormalized {#osusertimenormalized} - -该值与 `OSUserTime` 类似,但会除以 CPU 核心数量,从而无论核心数量多少,其取值范围始终为 [0..1]。这允许你在集群中不同服务器之间对该指标进行平均,即使各服务器的核心数量不一致,仍然可以得到平均资源利用率指标。 - -### PostgreSQLThreads {#postgresqlthreads} - -PostgreSQL 兼容协议服务器中的线程数量。 - -### ReplicasMaxAbsoluteDelay {#replicasmaxabsolutedelay} - -在所有 Replicated 表中,最新的已复制数据 part 与仍待复制的最新数据 part 之间的最大时间差(以秒为单位)。一个非常大的值表明存在无数据的副本。 - -### ReplicasMaxInsertsInQueue {#replicasmaxinsertsinqueue} - -在所有 Replicated 表中,队列中(仍待复制)的 INSERT 操作的最大数量。 - -### ReplicasMaxMergesInQueue {#replicasmaxmergesinqueue} - -在所有 Replicated 表中,队列中(仍待应用)的合并操作的最大数量。 - -### ReplicasMaxQueueSize {#replicasmaxqueuesize} - -在所有 Replicated 表中,队列的最大大小(以 get、merge 等操作的数量计)。 - -### ReplicasMaxRelativeDelay {#replicasmaxrelativedelay} - -在所有 Replicated 表中,某个副本的延迟与同一张表中最新副本延迟之间的最大差值。 - -### ReplicasSumInsertsInQueue {#replicassuminsertsinqueue} - -在所有 Replicated 表中,队列中(仍待复制)的 INSERT 操作的总数量。 - -### ReplicasSumMergesInQueue {#replicassummergesinqueue} - -在所有 Replicated 表中,队列中(仍待应用)的合并操作的总数量。 - -### ReplicasSumQueueSize {#replicassumqueuesize} - -在所有 Replicated 表中,队列大小的总和(以 get、merge 等操作的数量计)。 - -### TCPThreads {#tcpthreads} - -TCP 协议服务器中的线程数量(不包括 TLS)。 - -### Temperature_*N* {#temperature_n} - -对应设备的温度,单位为 ℃。传感器可能返回不合理的数值。来源:`/sys/class/thermal` - -### Temperature_*name* {#temperature_name} - -对应硬件监控设备及其相关传感器报告的温度,单位为 ℃。传感器可能返回不合理的数值。来源:`/sys/class/hwmon` - -### TotalBytesOfMergeTreeTables {#totalbytesofmergetreetables} - -所有 MergeTree 系列表中存储的数据总字节数(压缩后,包括数据和索引)。 - -### TotalPartsOfMergeTreeTables {#totalpartsofmergetreetables} - -所有 MergeTree 系列表中的数据 part 总数。大于 10 000 的数值会对服务器启动时间产生负面影响,并且可能表示分区键选择不合理。 - -### TotalPrimaryKeyBytesInMemory {#totalprimarykeybytesinmemory} - -主键值使用的内存总量(以字节为单位)(仅考虑活动 part)。 - -### TotalPrimaryKeyBytesInMemoryAllocated {#totalprimarykeybytesinmemoryallocated} - -为主键值预留的内存总量(以字节为单位)(仅考虑活动 part)。 - -### TotalRowsOfMergeTreeTables {#totalrowsofmergetreetables} - -所有 MergeTree 系列表中存储的行(记录)总数。 - -### Uptime {#uptime} - -服务器运行时间(秒)。包括服务器在开始接受连接前进行初始化所花费的时间。 - -### jemalloc.active {#jemallocactive} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.allocated {#jemallocallocated} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.dirty_purged {#jemallocarenasalldirty_purged} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.muzzy_purged {#jemallocarenasallmuzzy_purged} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pactive {#jemallocarenasallpactive} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pdirty {#jemallocarenasallpdirty} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.arenas.all.pmuzzy {#jemallocarenasallpmuzzy} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.num_runs {#jemallocbackground_threadnum_runs} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.num_threads {#jemallocbackground_threadnum_threads} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.background_thread.run_intervals {#jemallocbackground_threadrun_intervals} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.epoch {#jemallocepoch} - -jemalloc(Jason Evans 的内存分配器)统计信息的内部递增更新编号,用于所有其他 `jemalloc` 指标中。 - -### jemalloc.mapped {#jemallocmapped} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.metadata {#jemallocmetadata} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.metadata_thp {#jemallocmetadata_thp} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.resident {#jemallocresident} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.retained {#jemallocretained} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -### jemalloc.prof.active {#jemallocprofactive} - -底层内存分配器(jemalloc)的内部指标。参见 https://jemalloc.net/jemalloc.3.html - -**另请参阅** - -- [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 -- [system.metrics](/operations/system-tables/metrics) — 包含实时计算得到的指标。 -- [system.events](/operations/system-tables/events) — 包含已发生事件的数量。 -- [system.metric_log](/operations/system-tables/metric_log) — 包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md deleted file mode 100644 index 658b44b9a72..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/azure_queue_settings.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '包含 AzureQueue 表设置相关信息的系统表。 - 从服务器版本 `24.10` 起可用。' -keywords: ['system table', 'azure_queue_settings'] -slug: /operations/system-tables/azure_queue_settings -title: 'system.azure_queue_settings' -doc_type: 'reference' ---- - -包含有关 [AzureQueue](../../engines/table-engines/integrations/azure-queue.md) 表设置的信息。 -从服务器版本 `24.10` 起可用。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md deleted file mode 100644 index 163ff4651ef..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backup_log.md +++ /dev/null @@ -1,169 +0,0 @@ ---- -description: '包含 `BACKUP` 和 `RESTORE` 操作相关日志条目的系统表。' -keywords: ['system table', 'backup_log'] -slug: /operations/system-tables/backup_log -title: 'system.backup_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.backup_log {#systembackup_log} - - - -包含 `BACKUP` 和 `RESTORE` 操作相关信息的日志记录。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 日志记录的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 日志记录的日期和时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 日志记录的时间(精确到微秒)。 -* `id` ([String](../../sql-reference/data-types/string.md)) — 备份或恢复操作的标识符。 -* `name` ([String](../../sql-reference/data-types/string.md)) — 备份存储的名称(`FROM` 或 `TO` 子句中的内容)。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — 操作状态。可能的取值: - * `'CREATING_BACKUP'` - * `'BACKUP_CREATED'` - * `'BACKUP_FAILED'` - * `'RESTORING'` - * `'RESTORED'` - * `'RESTORE_FAILED'` -* `error` ([String](../../sql-reference/data-types/string.md)) — 失败操作的错误信息(成功操作时为空字符串)。 -* `start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 操作开始时间。 -* `end_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 操作结束时间。 -* `num_files` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 备份中存储的文件数量。 -* `total_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 备份中存储的文件总大小。 -* `num_entries` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 备份中的条目数量,即如果备份以目录形式存储,则为目录中的文件数量;如果备份以归档形式存储,则为归档中的文件数量。如果是增量备份,或者包含空文件或重复文件,则该值与 `num_files` 不同。始终满足:`num_entries <= num_files`。 -* `uncompressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 备份的未压缩大小。 -* `compressed_size` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 备份的压缩大小。如果备份不是以归档形式存储,则该值等于 `uncompressed_size`。 -* `files_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 恢复操作期间读取的文件数量。 -* `bytes_read` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 恢复操作期间读取的文件总大小。 - -**示例** - -```sql -BACKUP TABLE test_db.my_table TO Disk('backups_disk', '1.zip') -``` - -```response -┌─id───────────────────────────────────┬─status─────────┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ BACKUP_CREATED │ -└──────────────────────────────────────┴────────────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'e5b74ecb-f6f1-426a-80be-872f90043885' ORDER BY event_date, event_time_microseconds \G -``` - -```response -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:05:21.998566 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: CREATING_BACKUP -error: -start_time: 2023-08-19 11:05:21 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Row 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time: 2023-08-19 11:08:56 -event_time_microseconds: 2023-08-19 11:08:56.916192 -id: e5b74ecb-f6f1-426a-80be-872f90043885 -name: Disk('backups_disk', '1.zip') -status: BACKUP_CREATED -error: -start_time: 2023-08-19 11:05:21 -end_time: 2023-08-19 11:08:56 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 3525068304 -files_read: 0 -bytes_read: 0 -``` - -```sql -RESTORE TABLE test_db.my_table FROM Disk('backups_disk', '1.zip') -``` - -```response -┌─id───────────────────────────────────┬─status───┐ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ RESTORED │ -└──────────────────────────────────────┴──────────┘ -``` - -```sql -SELECT * FROM system.backup_log WHERE id = 'cdf1f731-52ef-42da-bc65-2e1bfcd4ce90' ORDER BY event_date, event_time_microseconds \G -``` - -```response -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:19.718077 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORING -error: -start_time: 2023-08-19 11:09:19 -end_time: 1970-01-01 03:00:00 -num_files: 0 -total_size: 0 -num_entries: 0 -uncompressed_size: 0 -compressed_size: 0 -files_read: 0 -bytes_read: 0 - -Row 2: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-08-19 -event_time_microseconds: 2023-08-19 11:09:29.334234 -id: cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 -name: Disk('backups_disk', '1.zip') -status: RESTORED -error: -start_time: 2023-08-19 11:09:19 -end_time: 2023-08-19 11:09:29 -num_files: 57 -total_size: 4290364870 -num_entries: 46 -uncompressed_size: 4290362365 -compressed_size: 4290362365 -files_read: 57 -bytes_read: 4290364870 -``` - -这些内容本质上与系统表 `system.backups` 中记录的信息相同: - -```sql -SELECT * FROM system.backups ORDER BY start_time -``` - -```response -┌─id───────────────────────────────────┬─name──────────────────────────┬─status─────────┬─error─┬──────────start_time─┬────────────end_time─┬─num_files─┬─total_size─┬─num_entries─┬─uncompressed_size─┬─compressed_size─┬─files_read─┬─bytes_read─┐ -│ e5b74ecb-f6f1-426a-80be-872f90043885 │ Disk('backups_disk', '1.zip') │ BACKUP_CREATED │ │ 2023-08-19 11:05:21 │ 2023-08-19 11:08:56 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 3525068304 │ 0 │ 0 │ -│ cdf1f731-52ef-42da-bc65-2e1bfcd4ce90 │ Disk('backups_disk', '1.zip') │ RESTORED │ │ 2023-08-19 11:09:19 │ 2023-08-19 11:09:29 │ 57 │ 4290364870 │ 46 │ 4290362365 │ 4290362365 │ 57 │ 4290364870 │ -└──────────────────────────────────────┴───────────────────────────────┴────────────────┴───────┴─────────────────────┴─────────────────────┴───────────┴────────────┴─────────────┴───────────────────┴─────────────────┴────────────┴────────────┘ -``` - -**另请参阅** - -* [备份和恢复](../../operations/backup.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md deleted file mode 100644 index 71bf5b163f8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/backups.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -description: '包含 `BACKUP` 和 `RESTORE` 操作相关日志记录的系统表。' -keywords: ['system table', 'backups'] -slug: /operations/system-tables/backups -title: 'system.backups' -doc_type: 'reference' ---- - -# system.backups {#systembackups} - -包含所有 `BACKUP` 或 `RESTORE` 操作及其当前状态和其他属性的列表。注意,该表不是持久化的,只显示上次服务器重启之后执行的操作。 - -下面是包含名称和说明列的 markdown 表格: - -| Column | Description | -|---------------------|----------------------------------------------------------------------------------------------------------------------| -| `id` | 操作 ID,可以通过 SETTINGS id=... 传入,也可以是随机生成的 UUID。 | -| `name` | 操作名称,类似 `Disk('backups', 'my_backup')` 的字符串。 | -| `base_backup_name` | 基础备份操作名称,类似 `Disk('backups', 'my_base_backup')` 的字符串。 | -| `query_id` | 启动该备份的查询的 Query ID。 | -| `status` | 备份或恢复操作的状态。 | -| `error` | 错误信息(如果有)。 | -| `start_time` | 操作开始的时间。 | -| `end_time` | 操作结束的时间。 | -| `num_files` | 备份中存储的文件数量。 | -| `total_size` | 备份中存储的文件总大小。 | -| `num_entries` | 备份中的条目数量;如果备份以文件夹形式存储,则为该文件夹中的文件数量。 | -| `uncompressed_size` | 备份的未压缩大小。 | -| `compressed_size` | 备份的压缩大小。 | -| `files_read` | 从该备份执行 RESTORE 时读取的文件数量。 | -| `bytes_read` | 从该备份执行 RESTORE 时读取的文件总大小。 | -| `ProfileEvents` | 此操作期间捕获的所有 ProfileEvents 事件。 | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md deleted file mode 100644 index e8a3feceb42..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/blob_storage_log.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: '包含日志记录的系统表,记录各种 Blob 存储操作(如上传和删除)的相关信息。' -keywords: ['系统表', 'blob_storage_log'] -slug: /operations/system-tables/blob_storage_log -title: 'system.blob_storage_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含记录各种 blob 存储操作(如上传和删除)的日志记录。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒级精度的事件时间。 -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 事件类型。可能的取值: - * `'Upload'` - * `'Delete'` - * `'MultiPartUploadCreate'` - * `'MultiPartUploadWrite'` - * `'MultiPartUploadComplete'` - * `'MultiPartUploadAbort'` -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 与该事件关联的查询标识符(如果有)。 -* `thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 执行该操作的线程标识符。 -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — 执行该操作的线程名称。 -* `disk_name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — 关联磁盘的名称。 -* `bucket` ([String](../../sql-reference/data-types/string.md)) — bucket 的名称。 -* `remote_path` ([String](../../sql-reference/data-types/string.md)) — 远程资源的路径。 -* `local_path` ([String](../../sql-reference/data-types/string.md)) — 本地系统上引用远程资源的元数据文件路径。 -* `data_size` ([UInt32](/sql-reference/data-types/int-uint#integer-ranges)) — 上传事件中涉及的数据大小。 -* `error` ([String](../../sql-reference/data-types/string.md)) — 与该事件关联的错误消息(如果有)。 - -**示例** - -假设某个 blob 存储操作上传了一个文件,并记录了一个事件: - -```sql -SELECT * FROM system.blob_storage_log WHERE query_id = '7afe0450-504d-4e4b-9a80-cd9826047972' ORDER BY event_date, event_time_microseconds \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2023-10-31 -event_time: 2023-10-31 16:03:40 -event_time_microseconds: 2023-10-31 16:03:40.481437 -event_type: Upload -query_id: 7afe0450-504d-4e4b-9a80-cd9826047972 -thread_id: 2381740 -disk_name: disk_s3 -bucket: bucket1 -remote_path: rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe -local_path: store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt -data_size: 259 -error: -``` - -在此示例中,上传操作与 ID 为 `7afe0450-504d-4e4b-9a80-cd9826047972` 的 `INSERT` 查询相关联。本地元数据文件 `store/654/6549e8b3-d753-4447-8047-d462df6e6dbe/tmp_insert_all_1_1_0/checksums.txt` 对应于磁盘 `disk_s3` 上存储桶 `bucket1` 中的远程路径 `rrr/kxo/tbnqtrghgtnxkzgtcrlutwuslgawe`,文件大小为 259 字节。 - -**另请参阅** - -* [用于存储数据的外部磁盘](../../operations/storing-data.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md deleted file mode 100644 index 5e130315192..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/build_options.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '包含有关 ClickHouse 服务器构建选项的信息。' -slug: /operations/system-tables/build_options -title: 'system.build_options' -keywords: ['系统表', 'build_options'] -doc_type: 'reference' ---- - -包含有关 ClickHouse 服务器构建选项的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.build_options LIMIT 5 -``` - -```text -┌─name─────────────┬─value─┐ -│ USE_BROTLI │ 1 │ -│ USE_BZIP2 │ 1 │ -│ USE_CAPNP │ 1 │ -│ USE_CASSANDRA │ 1 │ -│ USE_DATASKETCHES │ 1 │ -└──────────────────┴───────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md deleted file mode 100644 index f61977af03e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/clusters.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '包含配置文件中可用集群及其服务器信息的系统表。' -keywords: ['system table', 'clusters'] -slug: /operations/system-tables/clusters -title: 'system.clusters' -doc_type: 'reference' ---- - -包含配置文件中可用集群及这些集群中服务器信息的系统表。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -查询: - -```sql -SELECT * FROM system.clusters LIMIT 2 FORMAT Vertical; -``` - -结果: - -```text -Row 1: -────── -cluster: test_cluster_two_shards -shard_num: 1 -shard_name: shard_01 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.1 -host_address: 127.0.0.1 -port: 9000 -is_local: 1 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL - -Row 2: -────── -cluster: test_cluster_two_shards -shard_num: 2 -shard_name: shard_02 -shard_weight: 1 -replica_num: 1 -host_name: 127.0.0.2 -host_address: 127.0.0.2 -port: 9000 -is_local: 0 -user: default -default_database: -errors_count: 0 -slowdowns_count: 0 -estimated_recovery_time: 0 -database_shard_name: -database_replica_name: -is_active: NULL -``` - -**另请参阅** - -* [Distributed 表引擎](../../engines/table-engines/special/distributed.md) -* [distributed_replica_error_cap 设置](../../operations/settings/settings.md#distributed_replica_error_cap) -* [distributed_replica_error_half_life 设置](../../operations/settings/settings.md#distributed_replica_error_half_life) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md deleted file mode 100644 index 8f2f054cd68..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/codecs.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '包含队列中编解码器信息的系统表。' -keywords: ['system table', 'codecs', 'compression'] -slug: /operations/system-tables/codecs -title: 'system.codecs' -doc_type: 'reference' ---- - -包含有关压缩和加密编解码器的信息。 - -您可以使用该表获取可用压缩和加密编解码器的信息。 - -`system.codecs` 表包含以下列(括号中为列类型): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -查询: - -```sql -SELECT * FROM system.codecs WHERE name='LZ4' -``` - -结果: - -```text -Row 1: -────── -name: LZ4 -method_byte: 130 -is_compression: 1 -is_generic_compression: 1 -is_encryption: 0 -is_timeseries_codec: 0 -is_experimental: 0 -description: 极快;压缩效果好;速度与效率兼顾。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md deleted file mode 100644 index 5e8b4c82516..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/columns.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '包含所有表中列信息的系统表' -keywords: ['system table', 'columns'] -slug: /operations/system-tables/columns -title: 'system.columns' -doc_type: 'reference' ---- - -包含所有表中列的信息。 - -你可以使用此表获取类似 [DESCRIBE TABLE](../../sql-reference/statements/describe-table.md) 查询返回的信息,但可以一次性针对多个表进行查询。 - -来自[临时表](../../sql-reference/statements/create/table.md#temporary-tables)的列仅在创建它们的会话中在 `system.columns` 中可见。此时这些列的 `database` 字段为空。 - -`system.columns` 表包含以下列(括号中为列类型): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.columns LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_catalog -type: String -position: 1 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ - -Row 2: -────── -database: INFORMATION_SCHEMA -table: COLUMNS -name: table_schema -type: String -position: 2 -default_kind: -default_expression: -data_compressed_bytes: 0 -data_uncompressed_bytes: 0 -marks_bytes: 0 -comment: -is_in_partition_key: 0 -is_in_sorting_key: 0 -is_in_primary_key: 0 -is_in_sampling_key: 0 -compression_codec: -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: ᴺᵁᴸᴸ -numeric_precision_radix: ᴺᵁᴸᴸ -numeric_scale: ᴺᵁᴸᴸ -datetime_precision: ᴺᵁᴸᴸ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md deleted file mode 100644 index 6a0e7dcfeec..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/contributors.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '包含贡献者信息的系统表。' -keywords: ['system table', 'contributors'] -slug: /operations/system-tables/contributors -title: 'system.contributors' -doc_type: 'reference' ---- - -包含贡献者的信息。在每次查询执行时,行的顺序是随机的。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERGED_END*/ } - -**示例** - -```sql -SELECT * FROM system.contributors LIMIT 10 -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -│ Max Vetrov │ -│ LiuYangkuan │ -│ svladykin │ -│ zamulla │ -│ Šimon Podlipský │ -│ BayoNet │ -│ Ilya Khomutov │ -│ Amy Krishnevsky │ -│ Loud_Scream │ -└──────────────────┘ -``` - -要在表中查找你自己的记录,请使用如下查询: - -```sql -SELECT * FROM system.contributors WHERE name = 'Olga Khvostikova' -``` - -```text -┌─name─────────────┐ -│ Olga Khvostikova │ -└──────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md deleted file mode 100644 index 212d9a72ec9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/crash_log.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '记录致命错误堆栈跟踪信息的系统表。' -keywords: ['system table', 'crash_log'] -slug: /operations/system-tables/crash_log -title: 'system.crash_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含有关致命错误的堆栈跟踪信息。该表在数据库中默认不存在,仅在发生致命错误时才会创建。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 带纳秒精度的事件时间戳。 -* `signal` ([Int32](../../sql-reference/data-types/int-uint.md)) — 信号编号。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 线程 ID。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询 ID。 -* `trace` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 崩溃瞬间的堆栈跟踪。每个元素是 ClickHouse 服务器进程中的虚拟内存地址。 -* `trace_full` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 崩溃瞬间的堆栈跟踪。每个元素包含 ClickHouse 服务器进程中被调用的方法。 -* `version` ([String](../../sql-reference/data-types/string.md)) — ClickHouse 服务器版本。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse 服务器修订号。 -* `build_id` ([String](../../sql-reference/data-types/string.md)) — 由编译器生成的 Build ID。 - -**示例** - -查询: - -```sql -SELECT * FROM system.crash_log ORDER BY event_time DESC LIMIT 1; -``` - -结果(部分输出): - -```text -第 1 行: -────── -主机名: clickhouse.eu-central1.internal -事件日期: 2020-10-14 -事件时间: 2020-10-14 15:47:40 -时间戳(纳秒): 1602679660271312710 -信号: 11 -线程 ID: 23624 -查询 ID: 428aab7c-8f5c-44e9-9607-d16b44467e69 -跟踪: [188531193,...] -完整跟踪: ['3. DB::(anonymous namespace)::FunctionFormatReadableTimeDelta::executeImpl(std::__1::vector >&, std::__1::vector > const&, unsigned long, unsigned long) const @ 0xb3cc1f9 in /home/username/work/ClickHouse/build/programs/clickhouse',...] -版本: ClickHouse 20.11.1.1 -修订版本: 54442 -构建 ID: -``` - -**另请参阅** - -* [trace_log](../../operations/system-tables/trace_log.md) 系统表 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md deleted file mode 100644 index 15f47467f33..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/current_roles.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: '包含当前用户当前启用角色的系统表。' -keywords: ['system table', 'current_roles'] -slug: /operations/system-tables/current_roles -title: 'system.current_roles' -doc_type: 'reference' ---- - -包含当前用户当前启用的角色。`SET ROLE` 会更改此表的内容。 - -列: - -- `role_name` ([String](../../sql-reference/data-types/string.md))) — 角色名称。 -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志位,指示 `current_role` 是否为具有 `ADMIN OPTION` 权限的角色。 -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志位,指示 `current_role` 是否为默认角色。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md deleted file mode 100644 index ae091f6a3e8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dashboards.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '包含可通过 HTTP 接口访问的 `/dashboard` 页面所使用的查询,对监控和故障排查很有用。' -keywords: ['system table', 'dashboards', 'monitoring', 'troubleshooting'] -slug: /operations/system-tables/dashboards -title: 'system.dashboards' -doc_type: 'reference' ---- - -包含 `/dashboard` 页面(可通过 [`HTTP 接口`](/interfaces/http.md) 访问)所使用的查询。 -该表对于监控和故障排查非常有用。表中每一行对应仪表盘中的一个图表。 - -:::note -`/dashboard` 页面不仅可以渲染来自 `system.dashboards` 的查询,还可以渲染来自任意具有相同表结构的表的查询。 -这对于创建自定义仪表盘很有帮助。 -::: - -示例: - -```sql -SELECT * -FROM system.dashboards -WHERE title ILIKE '%CPU%' -``` - -```text -Row 1: -────── -dashboard: overview -title: CPU 使用量(核心) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUVirtualTimeMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 2: -────── -dashboard: overview -title: CPU 等待时间 -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(ProfileEvent_OSCPUWaitMicroseconds) / 1000000 -FROM system.metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 3: -────── -dashboard: overview -title: 操作系统 CPU 使用量(用户空间) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSUserTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} - -Row 4: -────── -dashboard: overview -title: 操作系统 CPU 使用量(内核) -query: SELECT toStartOfInterval(event_time, INTERVAL {rounding:UInt32} SECOND)::INT AS t, avg(value) -FROM system.asynchronous_metric_log -WHERE event_date >= toDate(now() - {seconds:UInt32}) AND event_time >= now() - {seconds:UInt32} AND metric = 'OSSystemTimeNormalized' -GROUP BY t -ORDER BY t WITH FILL STEP {rounding:UInt32} -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md deleted file mode 100644 index 3e2572126ba..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_skipping_indices.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '包含所有表中现有的数据跳过索引的信息。' -keywords: ['system table', 'data_skipping_indices'] -slug: /operations/system-tables/data_skipping_indices -title: 'system.data_skipping_indices' -doc_type: 'reference' ---- - -包含所有表中现有的数据跳过索引的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.data_skipping_indices LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: user_actions -name: clicks_idx -type: minmax -type_full: minmax -expr: clicks -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 - -Row 2: -────── -database: default -table: users -name: contacts_null_idx -type: minmax -type_full: minmax -expr: assumeNotNull(contacts_null) -granularity: 1 -data_compressed_bytes: 58 -data_uncompressed_bytes: 6 -marks_bytes: 48 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md deleted file mode 100644 index ed61dd7d258..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/data_type_families.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '包含已支持数据类型信息的系统表' -keywords: ['system table', 'data_type_families'] -slug: /operations/system-tables/data_type_families -title: 'system.data_type_families' -doc_type: 'reference' ---- - -包含受支持[数据类型](../../sql-reference/data-types/index.md)的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.data_type_families WHERE alias_to = 'String' -``` - -```text -┌─name───────┬─case_insensitive─┬─alias_to─┐ -│ LONGBLOB │ 1 │ String │ -│ LONGTEXT │ 1 │ String │ -│ TINYTEXT │ 1 │ String │ -│ TEXT │ 1 │ String │ -│ VARCHAR │ 1 │ String │ -│ MEDIUMBLOB │ 1 │ String │ -│ BLOB │ 1 │ String │ -│ TINYBLOB │ 1 │ String │ -│ CHAR │ 1 │ String │ -│ MEDIUMTEXT │ 1 │ String │ -└────────────┴──────────────────┴──────────┘ -``` - -**另请参阅** - -* [Syntax](../../sql-reference/syntax.md) — 关于支持的语法的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md deleted file mode 100644 index 6977a1facc0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_engines.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '包含服务器所支持的数据库引擎列表的系统表。' -keywords: ['system table', 'database_engines'] -slug: /operations/system-tables/database_engines -title: 'system.database_engines' -doc_type: 'reference' ---- - -包含服务器所支持的数据库引擎列表。 - -该表包含以下列(列类型在括号中注明): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -示例: - -```sql -SELECT * -FROM system.database_engines -WHERE name IN ('Atomic', 'Lazy', 'Ordinary') -``` - -```text -┌─name─────┐ -│ 普通 │ -│ 原子 │ -│ 惰性 │ -└──────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md deleted file mode 100644 index 173f3779d65..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/database_replicas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: '包含 Replicated 数据库副本信息及状态的系统表。' -keywords: ['系统表', 'database_replicas'] -slug: /operations/system-tables/database_replicas -title: 'system.database_replicas' -doc_type: 'reference' ---- - -包含各个 Replicated 数据库副本的信息。 - -列: - -{/*AUTOGENERGED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.database_replicas FORMAT Vertical; -``` - -```text -Row 1: -────── -database: db_2 -is_readonly: 0 -max_log_ptr: 2 -replica_name: replica1 -replica_path: /test/db_2/replicas/shard1|replica1 -zookeeper_path: /test/db_2 -shard_name: shard1 -log_ptr: 2 -total_replicas: 1 -zookeeper_exception: -is_session_expired: 0 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md deleted file mode 100644 index c62cc7d1461..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/databases.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '系统表,包含当前用户可用数据库的信息。' -keywords: ['system table', 'databases'] -slug: /operations/system-tables/databases -title: 'system.databases' -doc_type: 'reference' ---- - -系统表,包含当前用户可用数据库的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -此系统表的 `name` 列用于实现 `SHOW DATABASES` 查询。 - -**示例** - -创建一个数据库。 - -```sql -CREATE DATABASE test; -``` - -查看当前用户可访问的所有数据库。 - -```sql -SELECT * FROM system.databases; -``` - -```text -┌─name────────────────┬─engine─────┬─data_path────────────────────┬─metadata_path─────────────────────────────────────────────────────────┬─uuid─────────────────────────────────┬─engine_full────────────────────────────────────────────┬─comment─┐ -│ INFORMATION_SCHEMA │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ default │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/f97/f97a3ceb-2e8a-4912-a043-c536e826a4d4/ │ f97a3ceb-2e8a-4912-a043-c536e826a4d4 │ Atomic │ │ -│ information_schema │ Memory │ /data/clickhouse_data/ │ │ 00000000-0000-0000-0000-000000000000 │ Memory │ │ -│ replicated_database │ Replicated │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/da8/da85bb71-102b-4f69-9aad-f8d6c403905e/ │ da85bb71-102b-4f69-9aad-f8d6c403905e │ Replicated('some/path/database', 'shard1', 'replica1') │ │ -│ system │ Atomic │ /data/clickhouse_data/store/ │ /data/clickhouse_data/store/b57/b5770419-ac7a-4b67-8229-524122024076/ │ b5770419-ac7a-4b67-8229-524122024076 │ Atomic │ │ -└─────────────────────┴────────────┴──────────────────────────────┴───────────────────────────────────────────────────────────────────────┴──────────────────────────────────────┴────────────────────────────────────────────────────────┴─────────┘ - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md deleted file mode 100644 index ff52b15cd1e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dead_letter_queue.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -description: '包含通过流式引擎接收且在解析时发生错误的消息信息的系统表。' -keywords: ['system table', 'dead_letter_queue'] -slug: /operations/system-tables/dead_letter_queue -title: 'system.dead_letter_queue' -doc_type: 'reference' ---- - -包含通过流式引擎接收且在解析时发生错误的消息信息。目前已为 Kafka 和 RabbitMQ 提供实现。 - -通过在特定引擎的 `handle_error_mode` 设置中指定 `dead_letter_queue` 来启用日志记录。 - -数据的刷新周期在服务器配置中 [dead_letter_queue](../../operations/server-configuration-parameters/settings.md#dead_letter_queue) 部分的 `flush_interval_milliseconds` 参数中设置。若要强制刷新,请使用 [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) 查询。 - -ClickHouse 不会自动从该表中删除数据。更多细节参见 [简介](../../operations/system-tables/overview.md#system-tables-introduction)。 - -列: - -* `table_engine` ([Enum8](../../sql-reference/data-types/enum.md)) - 流式引擎类型。可能的取值:`Kafka` 和 `RabbitMQ`。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) - 消息消费日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - 消息消费的日期和时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) - 具有微秒精度的消息消费时间。 -* `database` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) - 流式表所属的 ClickHouse 数据库。 -* `table` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) - ClickHouse 表名。 -* `error` ([String](../../sql-reference/data-types/string.md)) - 错误文本。 -* `raw_message` ([String](../../sql-reference/data-types/string.md)) - 消息体。 -* `kafka_topic_name` ([String](../../sql-reference/data-types/string.md)) - Kafka 主题名称。 -* `kafka_partition` ([UInt64](../../sql-reference/data-types/int-uint.md)) - 该主题的 Kafka 分区。 -* `kafka_offset` ([UInt64](../../sql-reference/data-types/int-uint.md)) - 消息的 Kafka 偏移量。 -* `kafka_key` ([String](../../sql-reference/data-types/string.md)) - 消息的 Kafka 键。 -* `rabbitmq_exchange_name` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ 交换机名称。 -* `rabbitmq_message_id` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ 消息 ID。 -* `rabbitmq_message_timestamp` ([DateTime](../../sql-reference/data-types/datetime.md)) - RabbitMQ 消息时间戳。 -* `rabbitmq_message_redelivered` ([UInt8](../../sql-reference/data-types/int-uint.md)) - RabbitMQ 重新投递标志。 -* `rabbitmq_message_delivery_tag` ([UInt64](../../sql-reference/data-types/int-uint.md)) - RabbitMQ 投递标签。 -* `rabbitmq_channel_id` ([String](../../sql-reference/data-types/string.md)) - RabbitMQ 通道 ID。 - -**示例** - -查询: - -```sql -SELECT * FROM system.dead_letter_queue LIMIT 1 \G; -``` - -结果: - - -```text -第 1 行: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910773 -database: default -table: kafka -error: 无法解析输入:应在 'qwertyuiop' 之前出现 '\t'(位于第 1 行) -: -第 1 行: -列 0, name: key, type: UInt64, 错误:文本 "qwertyuiop" 不是有效的 UInt64 值 -raw_message: qwertyuiop -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -第 2 行: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.910944 -database: default -table: kafka -error: 无法解析输入:应在 'asdfghjkl' 之前出现 '\t'(位于第 1 行) -: -第 1 行: -列 0, name: key, type: UInt64, 错误:文本 "asdfghjkl" 不是有效的 UInt64 值 -raw_message: asdfghjkl -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - -第 3 行: -────── -table_engine: Kafka -event_date: 2025-05-01 -event_time: 2025-05-01 10:34:53 -event_time_microseconds: 2025-05-01 10:34:53.911092 -database: default -table: kafka -error: 无法解析输入:应在 'zxcvbnm' 之前出现 '\t'(位于第 1 行) -: -第 1 行: -列 0, name: key, type: UInt64, 错误:文本 "zxcvbnm" 不是有效的 UInt64 值 -raw_message: zxcvbnm -kafka_topic_name: TSV_dead_letter_queue_err_1746095689 -kafka_partition: 0 -kafka_offset: 0 -kafka_key: -rabbitmq_exchange_name: -rabbitmq_message_id: -rabbitmq_message_timestamp: 1970-01-01 00:00:00 -rabbitmq_message_redelivered: 0 -rabbitmq_message_delivery_tag: 0 -rabbitmq_channel_id: - (test.py:78, dead_letter_queue_test) - -``` - -**另请参阅** - -* [Kafka](/engines/table-engines/integrations/kafka.md) — Kafka 引擎 -* [system.kafka_consumers](/operations/system-tables/kafka_consumers.md) — `kafka_consumers` 系统表的描述,该表包含有关 Kafka 消费者的统计数据、错误等信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md deleted file mode 100644 index 79fc16fca6d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/delta_metadata_log.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: '包含从 Delta Lake 表读取的元数据文件信息的系统表。每条记录对应一个根级元数据 JSON 文件。' -keywords: ['系统表', 'delta_lake_metadata_log'] -slug: /operations/system-tables/delta_lake_metadata_log -title: 'system.delta_lake_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.delta_lake_metadata_log {#systemdelta_lake_metadata_log} - -`system.delta_lake_metadata_log` 表会记录 ClickHouse 在读取 Delta Lake 表时的元数据访问和解析事件。它为每个元数据文件提供详细信息,对于调试、审计以及了解 Delta 表结构的演变非常有用。 - - - -## 目的 {#purpose} - -此表记录从 Delta Lake 表中读取的所有元数据文件。它帮助用户跟踪 ClickHouse 如何解释 Delta 表元数据,并诊断与模式演进、快照解析或查询规划相关的问题。 - -:::note -此表主要用于调试。 -::: - - - -## 列 {#columns} -| 名称 | 类型 | 描述 | -|----------------|-----------|----------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | 日志文件日期。 | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | 事件时间戳。 | -| `query_id` | [String](../../sql-reference/data-types/string.md) | 触发元数据读取的查询 ID。 | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Delta Lake 表的路径。 | -| `file_path` | [String](../../sql-reference/data-types/string.md) | 根元数据 JSON 文件的路径。 | -| `content` | [String](../../sql-reference/data-types/string.md) | JSON 格式的内容(来自 .json 的原始元数据)。 | - - - - - -## 控制日志详细程度 {#controlling-log-verbosity} - -可以使用 [`delta_lake_log_metadata`](../../operations/settings/settings.md#delta_lake_log_metadata) 设置来控制哪些元数据事件会被记录到日志中。 - -要记录当前查询中使用的所有元数据: - -```sql -SELECT * FROM my_delta_table SETTINGS delta_lake_log_metadata = 1; - -SYSTEM FLUSH LOGS delta_lake_metadata_log; - -SELECT * -FROM system.delta_lake_metadata_log -WHERE query_id = '{previous_query_id}'; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md deleted file mode 100644 index a9b41889bd8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_parts.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: '包含 MergeTree 表已分离部分信息的系统表' -keywords: ['system table', 'detached_parts'] -slug: /operations/system-tables/detached_parts -title: 'system.detached_parts' -doc_type: 'reference' ---- - -包含关于 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表已分离部分的信息。`reason` 列说明该部分被分离的原因。 - -对于用户手动分离的部分,该列为空。此类部分可以通过 [ALTER TABLE ATTACH PARTITION\|PART](/sql-reference/statements/alter/partition#attach-partitionpart) 命令重新附加。 - -关于其他列的说明,请参阅 [system.parts](../../operations/system-tables/parts.md)。 - -如果部分名称无效,则某些列的值可能为 `NULL`。此类部分可以通过 [ALTER TABLE DROP DETACHED PART](/sql-reference/statements/alter/partition#drop-detached-partitionpart) 命令删除。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md deleted file mode 100644 index 25afdd3e6bf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/detached_tables.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '包含每个已分离的表的信息。' -keywords: ['system table', 'detached_tables'] -slug: /operations/system-tables/detached_tables -title: 'system.detached_tables' -doc_type: 'reference' ---- - -包含每个已分离的表的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.detached_tables FORMAT Vertical; -``` - -```text -第 1 行: -────── -database: base -table: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -is_permanently: 1 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md deleted file mode 100644 index 6531199083d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dictionaries.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '包含字典相关信息的系统表' -keywords: ['system table', 'dictionaries'] -slug: /operations/system-tables/dictionaries -title: 'system.dictionaries' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含有关 [字典](../../sql-reference/dictionaries/index.md) 的信息。 - -列: - -{/*自动生成_开始*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -配置字典: - -```sql -CREATE DICTIONARY dictionary_with_comment -( - id UInt64, - value String -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(HOST 'localhost' PORT tcpPort() TABLE 'source_table')) -LAYOUT(FLAT()) -LIFETIME(MIN 0 MAX 1000) -COMMENT '临时字典'; -``` - -确保字典已加载。 - -```sql -SELECT * FROM system.dictionaries LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -name: dictionary_with_comment -uuid: 4654d460-0d03-433a-8654-d4600d03d33a -status: NOT_LOADED -origin: 4654d460-0d03-433a-8654-d4600d03d33a -type: -key.names: ['id'] -key.types: ['UInt64'] -attribute.names: ['value'] -attribute.types: ['String'] -bytes_allocated: 0 -query_count: 0 -hit_rate: 0 -found_rate: 0 -element_count: 0 -load_factor: 0 -source: -lifetime_min: 0 -lifetime_max: 0 -loading_start_time: 1970-01-01 00:00:00 -last_successful_update_time: 1970-01-01 00:00:00 -loading_duration: 0 -last_exception: -comment: 临时字典 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md deleted file mode 100644 index 2bf24add09c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dimensional_metrics.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '该表包含可实时计算并以 Prometheus 格式导出的维度指标,始终保持最新。' -keywords: ['system table', 'dimensional_metrics'] -slug: /operations/system-tables/dimensional_metrics -title: 'system.dimensional_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# dimensional_metrics {#dimensional_metrics} - - - -此表包含可以实时计算并以 Prometheus 格式导出的维度指标,且始终保持最新。 - -列: - -* `metric` ([String](../../sql-reference/data-types/string.md)) — 指标名称。 -* `value` ([Int64](../../sql-reference/data-types/int-uint.md)) — 指标值。 -* `description` ([String](../../sql-reference/data-types/string.md)) — 指标描述。 -* `labels` ([Map(String, String)](../../sql-reference/data-types/map.md)) — 指标标签。 -* `name` ([String](../../sql-reference/data-types/string.md)) — `metric` 的别名。 - -**Example** - -可以使用如下查询将所有维度指标以 Prometheus 格式导出。 - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'gauge' AS type -FROM system.dimensional_metrics -FORMAT Prometheus -``` - - -## 指标说明 {#metric_descriptions} - -### merge_failures {#merge_failures} -自启动以来所有合并失败的总次数。 - -### startup_scripts_failure_reason {#startup_scripts_failure_reason} -按错误类型指示启动脚本失败情况。当某个启动脚本失败时,该指标会被设置为 1,并使用错误名称进行标记。 - -### merge_tree_parts {#merge_tree_parts} -MergeTree 数据部分的数量,按部分状态、部分类型以及是否为投影部分进行标记。 - -**另请参阅** -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 包含周期性计算的指标。 -- [system.events](/operations/system-tables/events) — 包含已发生事件的数量。 -- [system.metric_log](/operations/system-tables/metric_log) — 包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md deleted file mode 100644 index 992e0ec4503..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/disks.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: '包含服务器配置中定义磁盘信息的系统表' -keywords: ['system table', 'disks'] -slug: /operations/system-tables/disks -title: 'system.disks' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含在[服务器配置](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure)中定义的磁盘的相关信息。 - -列: - -{/*AUTOGENERGED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.disks; -``` - -```response -┌─name────┬─path─────────────────┬───free_space─┬──total_space─┬─keep_free_space─┐ -│ default │ /var/lib/clickhouse/ │ 276392587264 │ 490652508160 │ 0 │ -└─────────┴──────────────────────┴──────────────┴──────────────┴─────────────────┘ - -返回 1 行。耗时: 0.001 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md deleted file mode 100644 index 33199209419..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distributed_ddl_queue.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '包含在集群上执行的分布式 DDL 查询(带有 ON CLUSTER 子句的查询)信息的系统表。' -keywords: ['system table', 'distributed_ddl_queue'] -slug: /operations/system-tables/distributed_ddl_queue -title: 'system.distributed_ddl_queue' -doc_type: 'reference' ---- - -包含在集群上执行的[分布式 DDL 查询(带有 ON CLUSTER 子句)](../../sql-reference/distributed-ddl.md)的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * -FROM system.distributed_ddl_queue -WHERE cluster = 'test_cluster' -LIMIT 2 -FORMAT Vertical - -查询 id: f544e72a-6641-43f1-836b-24baa1c9632a - -第 1 行: -────── -entry: query-0000000000 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: 已完成 -exception_code: 0 -exception_text: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -第 2 行: -────── -entry: query-0000000001 -entry_version: 5 -initiator_host: clickhouse01 -initiator_port: 9000 -cluster: test_cluster -query: CREATE DATABASE test_db UUID '4a82697e-c85e-4e5b-a01e-a36f2a758456' ON CLUSTER test_cluster -settings: {'max_threads':'16','use_uncompressed_cache':'0'} -query_create_time: 2023-09-01 16:15:14 -host: clickhouse-01 -port: 9000 -status: 已完成 -exception_code: 630 -exception_text: Code: 630. DB::Exception: 无法删除或重命名 test_db,因为某些表依赖于它: -query_finish_time: 2023-09-01 16:15:14 -query_duration_ms: 154 - -结果集包含 2 行。耗时:0.025 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md deleted file mode 100644 index e14124a1f24..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/distribution_queue.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: '包含队列中待发送到各分片的本地文件的信息。' -keywords: ['system table', 'distribution_queue'] -slug: /operations/system-tables/distribution_queue -title: 'system.distribution_queue' -doc_type: 'reference' ---- - -包含队列中待发送到各分片的本地文件的信息。这些本地文件包含通过以异步模式向 Distributed 表插入新数据而创建的新数据分片(parts)。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.distribution_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: dist -data_path: ./store/268/268bc070-3aad-4b1a-9cf2-4987580161af/default@127%2E0%2E0%2E2:9000/ -is_blocked: 1 -error_count: 0 -data_files: 1 -data_compressed_bytes: 499 -last_exception: -``` - -**另请参阅** - -* [分布式表引擎](../../engines/table-engines/special/distributed.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md deleted file mode 100644 index ee8a70830cb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dns_cache.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: '包含已缓存 DNS 记录信息的系统表。' -keywords: ['系统表', 'dns_cache'] -slug: /operations/system-tables/dns_cache -title: 'system.dns_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含缓存的 DNS 记录的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -查询: - -```sql -SELECT * FROM system.dns_cache; -``` - -结果: - -| hostname | ip_address | ip_family | cached_at | -| :-------- | :------------- | :------------ | :------------------ | -| localhost | ::1 | IPv6 | 2024-02-11 17:04:40 | -| localhost | 127.0.0.1 | IPv4 | 2024-02-11 17:04:40 | - -**另请参阅** - -* [disable_internal_dns_cache 设置](../../operations/server-configuration-parameters/settings.md#disable_internal_dns_cache) -* [dns_cache_max_entries 设置](../../operations/server-configuration-parameters/settings.md#dns_cache_max_entries) -* [dns_cache_update_period 设置](../../operations/server-configuration-parameters/settings.md#dns_cache_update_period) -* [dns_max_consecutive_failures 设置](../../operations/server-configuration-parameters/settings.md#dns_max_consecutive_failures) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md deleted file mode 100644 index d9d135f6e29..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: '包含已执行 DROP TABLE 语句但尚未完成数据清理的表的信息' -keywords: ['system table', 'dropped_tables'] -slug: /operations/system-tables/dropped_tables -title: 'system.dropped_tables' -doc_type: 'reference' ---- - -包含已执行 DROP TABLE 语句但尚未完成数据清理的表的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -以下示例演示如何获取关于 `dropped_tables` 的信息。 - -```sql -SELECT * -FROM system.dropped_tables\G -``` - -```text -Row 1: -────── -index: 0 -database: default -table: test -uuid: 03141bb2-e97a-4d7c-a172-95cc066bb3bd -engine: MergeTree -metadata_dropped_path: /data/ClickHouse/build/programs/data/metadata_dropped/default.test.03141bb2-e97a-4d7c-a172-95cc066bb3bd.sql -table_dropped_time: 2023-03-16 23:43:31 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md deleted file mode 100644 index bde06c4c7bf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/dropped_tables_parts.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '包含 `system.dropped_tables` 中 MergeTree 表各数据片段信息的系统表' -keywords: ['system table', 'dropped_tables_parts'] -slug: /operations/system-tables/dropped_tables_parts -title: 'system.dropped_tables_parts' -doc_type: 'reference' ---- - -包含从 [system.dropped_tables](./dropped_tables.md) 中记录的 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表的数据片段(parts)信息。 - -此表的结构与 [system.parts](./parts.md) 相同。 - -**另请参阅** - -- [MergeTree 系列表](../../engines/table-engines/mergetree-family/mergetree.md) -- [system.parts](./parts.md) -- [system.dropped_tables](./dropped_tables.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md deleted file mode 100644 index 172e01f9197..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/enabled_roles.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: '系统表,包含当前所有已启用的角色,包括当前用户的当前角色以及为当前角色授予的角色' -keywords: ['system table', 'enabled_roles'] -slug: /operations/system-tables/enabled_roles -title: 'system.enabled_roles' -doc_type: 'reference' ---- - -包含当前所有已启用的角色,包括当前用户的当前角色以及为当前角色授予的角色。 - -列: - -- `role_name` ([String](../../sql-reference/data-types/string.md))) — 角色名称。 -- `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志,指示该 `enabled_role` 是否为具有 `ADMIN OPTION` 权限的角色。 -- `is_current` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志,指示该 `enabled_role` 是否为当前用户的当前角色。 -- `is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志,指示该 `enabled_role` 是否为默认角色。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md deleted file mode 100644 index 2d624ccc95c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/error_log.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '系统表,包含表 `system.errors` 中错误值的历史记录,并定期刷新到磁盘。' -keywords: ['system table', 'error_log'] -slug: /operations/system-tables/system-error-log -title: 'system.error_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含来自表 `system.errors` 的错误统计历史记录,这些数据会定期写入磁盘。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `code` ([Int32](../../sql-reference/data-types/int-uint.md)) — 错误代码编号。 -* `error` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 错误名称。 -* `value` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 该错误发生的次数。 -* `remote` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 远程异常(即在某个分布式查询期间收到的异常)。 - -**示例** - -```sql -SELECT * FROM system.error_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2024-06-18 -event_time: 2024-06-18 07:32:39 -code: 999 -error: KEEPER_EXCEPTION -value: 2 -remote: 0 -``` - -**另请参阅** - -* [error_log 设置](../../operations/server-configuration-parameters/settings.md#error_log) — 控制该设置的启用和禁用。 -* [system.errors](../../operations/system-tables/errors.md) — 包含错误代码及其被触发次数。 -* [监控](../../operations/monitoring.md) — ClickHouse 监控的基础概念。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md deleted file mode 100644 index d38daa38424..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/errors.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '包含错误代码及其触发次数的系统表。' -keywords: ['system table', '错误'] -slug: /operations/system-tables/errors -title: 'system.errors' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含错误代码及其被触发的次数。 - -要显示所有可能的错误代码(包括那些未被触发的),请将设置 [system_events_show_zero_values](../settings/settings.md#system_events_show_zero_values) 设为 1。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -:::note -在成功执行查询时,某些错误计数器可能仍会增加。除非您能确定对应错误绝不可能是误报,否则不建议将此表用于服务器监控。 -::: - -**示例** - -```sql -SELECT name, code, value -FROM system.errors -WHERE value > 0 -ORDER BY code ASC -LIMIT 1 - -┌─name─────────────┬─code─┬─value─┐ -│ CANNOT_OPEN_FILE │ 76 │ 1 │ -└──────────────────┴──────┴───────┘ -``` - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), last_error_trace) AS all -SELECT name, arrayStringConcat(all, '\n') AS res -FROM system.errors -LIMIT 1 -SETTINGS allow_introspection_functions=1\G -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/events.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/events.md deleted file mode 100644 index 40832dc3e9c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/events.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: '包含有关系统中已发生事件数量信息的系统表。' -keywords: ['系统表', '事件'] -slug: /operations/system-tables/events -title: 'system.events' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含系统中发生的事件数量等信息。例如,在该表中,你可以查看自 ClickHouse 服务器启动以来已处理的 `SELECT` 查询数量。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -你可以在源码文件 [src/Common/ProfileEvents.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/ProfileEvents.cpp) 中找到所有支持的事件。 - -**示例** - -```sql -SELECT * FROM system.events LIMIT 5 -``` - -```text -┌─event─────────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ Query │ 12 │ 待解释并可能执行的查询数量。不包括解析失败的查询,或因 AST 大小限制、配额限制或并发查询数量限制而被拒绝的查询。可能包括 ClickHouse 自身发起的内部查询。不统计子查询。 │ -│ SelectQuery │ 8 │ 与 Query 相同,但仅适用于 SELECT 查询。 │ -│ FileOpen │ 73 │ 已打开的文件数量。 │ -│ ReadBufferFromFileDescriptorRead │ 155 │ 从文件描述符执行读取操作(read/pread)的次数。不包括套接字。 │ -│ ReadBufferFromFileDescriptorReadBytes │ 9931 │ 从文件描述符读取的字节数。如果文件已压缩,此处显示压缩后的数据大小。 │ -└───────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 包含周期性计算的指标。 -* [system.metrics](/operations/system-tables/metrics) — 包含即时计算的指标。 -* [system.metric_log](/operations/system-tables/metric_log) — 包含 `system.metrics` 和 `system.events` 两个表中的指标值历史记录。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基础概念。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md deleted file mode 100644 index 7b0924bf9f2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/functions.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: '包含普通函数和聚合函数信息的系统表。' -keywords: ['系统表', '函数'] -slug: /operations/system-tables/functions -title: 'system.functions' -doc_type: 'reference' ---- - -包含普通函数和聚合函数的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql - SELECT name, is_aggregate, is_deterministic, case_insensitive, alias_to FROM system.functions LIMIT 5; -``` - -```text -┌─name─────────────────────┬─is_aggregate─┬─is_deterministic─┬─case_insensitive─┬─alias_to─┐ -│ BLAKE3 │ 0 │ 1 │ 0 │ │ -│ sipHash128Reference │ 0 │ 1 │ 0 │ │ -│ mapExtractKeyLike │ 0 │ 1 │ 0 │ │ -│ sipHash128ReferenceKeyed │ 0 │ 1 │ 0 │ │ -│ mapPartialSort │ 0 │ 1 │ 0 │ │ -└──────────────────────────┴──────────────┴──────────────────┴──────────────────┴──────────┘ - -5 行在集合中。耗时: 0.002 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md deleted file mode 100644 index d827bdb346f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/grants.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: '显示 ClickHouse 用户账户已授予权限的系统表。' -keywords: ['system table', 'grants'] -slug: /operations/system-tables/grants -title: 'system.grants' -doc_type: 'reference' ---- - -授予 ClickHouse 用户账户的权限。 - -列: - -- `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 用户名。 - -- `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 分配给用户账户的角色。 - -- `access_type` ([Enum8](../../sql-reference/data-types/enum.md)) — ClickHouse 用户账户的访问类型。 - -- `database` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 数据库名称。 - -- `table` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 表名。 - -- `column` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 被授予访问权限的列名。 - -- `is_partial_revoke` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 逻辑值。用于指示是否有部分权限被撤销。可能的取值为: - - `0` — 该行描述的是一个授予操作。 - - `1` — 该行描述的是一个部分撤销操作。 - -- `grant_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 通过 `WITH GRANT OPTION` 授予的权限,参见 [GRANT](../../sql-reference/statements/grant.md#granting-privilege-syntax)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md deleted file mode 100644 index 79b044ec28c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/graphite_retentions.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: '包含 `graphite_rollup` 参数的信息,这些参数用于使用 `GraphiteMergeTree` 类型引擎的表。' -keywords: ['system table', 'graphite_retentions'] -slug: /operations/system-tables/graphite_retentions -title: 'system.graphite_retentions' -doc_type: 'reference' ---- - -包含 [graphite_rollup](../../operations/server-configuration-parameters/settings.md#graphite) 参数的信息,这些参数用于使用 [*GraphiteMergeTree](../../engines/table-engines/mergetree-family/graphitemergetree.md) 引擎的表。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md deleted file mode 100644 index 8dc283185e2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/histogram_metrics.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '此表包含可即时计算并以 Prometheus 格式导出的直方图指标,始终保持最新状态。' -keywords: ['system table', 'histogram_metrics'] -slug: /operations/system-tables/histogram_metrics -title: 'system.histogram_metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# histogram_metrics {#histogram_metrics} - - - -该表包含可即时计算并以 Prometheus 格式导出的直方图指标,其数据始终为最新。用于取代已弃用的 `system.latency_log`。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -可以使用如下所示的查询,将所有直方图指标以 Prometheus 格式导出。 - -```sql -SELECT - metric AS name, - toFloat64(value) AS value, - description AS help, - labels, - 'histogram' AS type -FROM system.histogram_metrics -FORMAT Prometheus -``` - - -## 指标说明 {#metric_descriptions} - -### keeper_response_time_ms_bucket {#keeper_response_time_ms_bucket} - -Keeper 的响应时间(单位:毫秒)。 - -**另请参阅** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 包含周期性计算的指标。 -- [system.events](/operations/system-tables/events) — 包含发生的各类事件的数量。 -- [system.metric_log](/operations/system-tables/metric_log) — 包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md deleted file mode 100644 index 3f802c43c1b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_history.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: 'system 表 Iceberg 快照历史记录' -keywords: ['system iceberg_history'] -slug: /operations/system-tables/iceberg_history -title: 'system.iceberg_history' -doc_type: 'reference' ---- - -# system.iceberg_history {#systemiceberg_history} - -此 system 表包含 ClickHouse 中 Iceberg 表的快照历史记录。如果 ClickHouse 中没有任何 Iceberg 表,则该表将为空。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md deleted file mode 100644 index 6cd7035e0ce..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/iceberg_metadata_log.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -description: '包含从 Iceberg 表中读取的元数据文件信息的系统表。每条记录表示一个根元数据文件、从 Avro 文件中提取的元数据,或某个 Avro 文件中的一条元数据记录。' -keywords: ['system table', 'iceberg_metadata_log'] -slug: /operations/system-tables/iceberg_metadata_log -title: 'system.iceberg_metadata_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.iceberg_metadata_log {#systemiceberg_metadata_log} - -`system.iceberg_metadata_log` 表记录了 ClickHouse 读取 Iceberg 表时的元数据访问和解析事件。它提供了每个已处理元数据文件或条目的详细信息,有助于调试、审计,以及理解 Iceberg 表结构的演变。 - - - -## 目的 {#purpose} - -此表会记录从 Iceberg 表中读取的每个元数据文件及条目,包括根元数据文件、manifest 列表以及 manifest 条目。它帮助用户跟踪 ClickHouse 如何解析 Iceberg 表元数据,并诊断与模式演进、文件解析或查询计划相关的问题。 - -:::note -此表主要用于调试。 -::: - - - -## 列 {#columns} - -| 名称 | 类型 | 描述 | -|----------------|-----------|----------------------------------------------------------------------------------------------| -| `event_date` | [Date](../../sql-reference/data-types/date.md) | 日志记录的日期。 | -| `event_time` | [DateTime](../../sql-reference/data-types/datetime.md) | 事件的时间戳。 | -| `query_id` | [String](../../sql-reference/data-types/string.md) | 触发元数据读取的查询 ID。 | -| `content_type` | [Enum8](../../sql-reference/data-types/enum.md) | 元数据内容的类型(见下文)。 | -| `table_path` | [String](../../sql-reference/data-types/string.md) | Iceberg 表的路径。 | -| `file_path` | [String](../../sql-reference/data-types/string.md) | 根元数据 JSON 文件、Avro 清单列表或清单文件的路径。 | -| `content` | [String](../../sql-reference/data-types/string.md) | JSON 格式的内容(来自 .json 的原始元数据、Avro 元数据或 Avro 条目)。 | -| `row_in_file` | [Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md)) | 文件中的行号(如适用)。对于 `ManifestListEntry` 和 `ManifestFileEntry` 类型的内容,该列存在。 | - - - -## `content_type` 值 {#content-type-values} - -- `None`: 无内容。 -- `Metadata`: 根元数据文件。 -- `ManifestListMetadata`: Manifest 列表的元数据。 -- `ManifestListEntry`: Manifest 列表条目。 -- `ManifestFileMetadata`: Manifest 文件的元数据。 -- `ManifestFileEntry`: Manifest 文件条目。 - - - - - -## 控制日志详细程度 {#controlling-log-verbosity} - -可以通过 [`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level) 设置来控制要记录哪些元数据事件。 - -要记录当前查询中使用的所有元数据: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'manifest_file_entry'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -若要仅记录当前查询使用的根元数据 JSON 文件: - -```sql -SELECT * FROM my_iceberg_table SETTINGS iceberg_metadata_log_level = 'metadata'; - -SYSTEM FLUSH LOGS iceberg_metadata_log; - -SELECT content_type, file_path, row_in_file -FROM system.iceberg_metadata_log -WHERE query_id = '{previous_query_id}'; -``` - -有关更多信息,请参阅 [`iceberg_metadata_log_level`](../../operations/settings/settings.md#iceberg_metadata_log_level) 设置的说明。 - -### 注意事项 {#good-to-know} - -* 仅在需要对 Iceberg 表进行深入排查时才在查询级别使用 `iceberg_metadata_log_level`。否则,可能会在日志表中填充过多元数据,从而导致性能下降。 -* 该表可能包含重复条目,因为它主要用于调试,并不保证每个实体的唯一性。 -* 如果使用的 `content_type` 比 `ManifestListMetadata` 更为详尽,则会对 manifest 列表禁用 Iceberg 元数据缓存。 -* 同样地,如果使用的 `content_type` 比 `ManifestFileMetadata` 更为详尽,则会对 manifest 文件禁用 Iceberg 元数据缓存。 - - -## 另请参阅 {#see-also} -- [Iceberg 表引擎](../../engines/table-engines/integrations/iceberg.md) -- [Iceberg 表函数](../../sql-reference/table-functions/iceberg.md) -- [system.iceberg_history](./iceberg_history.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/index.md deleted file mode 100644 index 04fc761af64..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/index.md +++ /dev/null @@ -1,135 +0,0 @@ ---- -description: '系统表的定义及其用途概览。' -keywords: ['system tables', 'overview'] -pagination_next: operations/system-tables/asynchronous_metric_log -sidebar_label: '概览' -sidebar_position: 52 -slug: /operations/system-tables/ -title: '系统表' -doc_type: 'reference' ---- - -{/* 本页的目录表由以下脚本根据 YAML front matter 字段 slug、description、title 自动生成: - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - - 如果你发现有错误,请直接编辑对应页面的 YML frontmatter。 - */ } - -{/*AUTOGENERATED_START*/ } - - -| 页面 | 说明 | -| ----------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- | -| [系统表概览](/operations/system-tables/overview) | 系统表是什么,以及它们为何有用的概述。 | -| [INFORMATION_SCHEMA](/operations/system-tables/information_schema) | 系统数据库,提供对数据库对象元数据的几乎标准化、与具体 DBMS 无关的视图。 | -| [system.asynchronous_insert_log](/operations/system-tables/asynchronous_insert_log) | 包含异步插入相关信息的系统表。每条记录表示一条被缓冲到异步插入中的插入查询。 | -| [system.asynchronous_inserts](/operations/system-tables/asynchronous_inserts) | 包含队列中待处理异步插入相关信息的系统表。 | -| [system.asynchronous_loader](/operations/system-tables/asynchronous_loader) | 包含最近异步作业信息及其状态的系统表(例如用于加载表的作业)。该表中每个作业对应一行。 | -| [system.asynchronous_metric_log](/operations/system-tables/asynchronous_metric_log) | 包含 `system.asynchronous_metrics` 历史值的系统表,这些值会按时间间隔保存一次(默认为每秒一次) | -| [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) | 包含在后台定期计算得到的各类指标的系统表。例如,当前使用的 RAM 大小。 | -| [system.azure_queue_settings](/operations/system-tables/azure_queue_settings) | 包含 AzureQueue 表设置信息的系统表。从服务器版本 `24.10` 起可用。 | -| [system.backup_log](/operations/system-tables/backup_log) | 包含 `BACKUP` 和 `RESTORE` 操作日志条目的系统表。 | -| [system.backups](/operations/system-tables/backups) | 包含有关 `BACKUP` 和 `RESTORE` 操作日志记录的系统表。 | -| [system.blob_storage_log](/operations/system-tables/blob_storage_log) | 用于记录日志条目的系统表,包含关于各种 blob 存储操作(例如上传和删除)的信息。 | -| [system.build_options](/operations/system-tables/build_options) | 包含 ClickHouse 服务器构建选项相关信息的 system 表。 | -| [system.clusters](/operations/system-tables/clusters) | 包含配置文件中可用集群及其中定义的服务器信息的系统表。 | -| [system.codecs](/operations/system-tables/codecs) | 包含队列中各编解码器信息的系统表。 | -| [system.columns](/operations/system-tables/columns) | 包含所有表的列信息的系统表 | -| [system.contributors](/operations/system-tables/contributors) | 包含贡献者信息的系统表。 | -| [system.crash_log](/operations/system-tables/crash_log) | 包含致命错误堆栈跟踪信息的系统表。 | -| [system.current_roles](/operations/system-tables/current_roles) | 记录当前用户已激活角色的系统表。 | -| [system.dashboards](/operations/system-tables/dashboards) | 包含 `/dashboard` 页面使用的查询,该页面可通过 HTTP 接口访问,可用于监控和故障排查。 | -| [system.data_skipping_indices](/operations/system-tables/data_skipping_indices) | 包含所有表上已存在的数据跳过索引信息的系统表。 | -| [system.data_type_families](/operations/system-tables/data_type_families) | 包含受支持数据类型信息的系统表 | -| [system.database_engines](/operations/system-tables/database_engines) | 包含服务器支持的数据库引擎列表的 system 表。 | -| [system.database_replicas](/operations/system-tables/database_replicas) | 包含有关复制数据库的信息和状态的系统表。 | -| [system.databases](/operations/system-tables/databases) | 包含当前用户可访问的数据库信息的系统表。 | -| [system.dead_letter_queue](/operations/system-tables/dead_letter_queue) | 包含通过流式引擎接收并在解析时出错的消息信息的系统表。 | -| [system.delta_lake_metadata_log](/operations/system-tables/delta_lake_metadata_log) | 包含从 Delta Lake 表中读取的元数据文件信息的系统表。每条记录对应一个根元数据 JSON 文件。 | -| [system.detached_parts](/operations/system-tables/detached_parts) | 包含 MergeTree 表中已分离分区片段信息的系统表 | -| [system.detached_tables](/operations/system-tables/detached_tables) | 包含每个已分离表相关信息的系统表。 | -| [system.dictionaries](/operations/system-tables/dictionaries) | 包含字典信息的系统表 | -| [system.dimensional_metrics](/operations/system-tables/dimensional_metrics) | 该表包含可即时计算并以 Prometheus 格式导出的维度型指标,始终保持最新状态。 | -| [system.disks](/operations/system-tables/disks) | 包含在服务器配置中定义的磁盘信息的系统表 | -| [system.distributed_ddl_queue](/operations/system-tables/distributed_ddl_queue) | 一个系统表,包含已在集群上执行的分布式 DDL 查询(使用 `ON CLUSTER` 子句的查询)的相关信息。 | -| [system.distribution_queue](/operations/system-tables/distribution_queue) | 包含队列中待发送至各分片的本地文件信息的系统表。 | -| [system.dns_cache](/operations/system-tables/dns_cache) | 包含缓存的 DNS 记录信息的系统表。 | -| [system.dropped_tables](/operations/system-tables/dropped_tables) | 包含已执行 `DROP TABLE` 但尚未完成数据清理的各表信息的系统表 | -| [system.dropped_tables_parts](/operations/system-tables/dropped_tables_parts) | 包含来自 `system.dropped_tables` 的已删除 MergeTree 表的分区片段信息的系统表 | -| [system.enabled_roles](/operations/system-tables/enabled_roles) | 系统表,包含当前所有激活的角色,包括当前用户当前正在使用的角色,以及授予该角色的其他角色 | -| [system.error_log](/operations/system-tables/system-error-log) | 系统表,包含来自 `system.errors` 表的错误值历史记录,并会定期刷写到磁盘。 | -| [system.errors](/operations/system-tables/errors) | 包含错误代码及其触发次数的系统表。 | -| [system.events](/operations/system-tables/events) | 用于记录系统中已发生事件数量的系统表。 | -| [system.functions](/operations/system-tables/functions) | 包含普通函数和聚合函数信息的系统表。 | -| [system.grants](/operations/system-tables/grants) | 用于展示已授予 ClickHouse 用户账户的权限的系统表。 | -| [system.graphite_retentions](/operations/system-tables/graphite_retentions) | 包含 `graphite_rollup` 参数信息的系统表,这些参数用于 `GraphiteMergeTree` 引擎类型的表中。 | -| [system.histogram_metrics](/operations/system-tables/histogram_metrics) | 该表包含可以即时计算的直方图指标,并可导出为 Prometheus 格式,始终保持最新。 | -| [system.iceberg_history](/operations/system-tables/iceberg_history) | 系统 Iceberg 快照历史 | -| [system.iceberg_metadata_log](/operations/system-tables/iceberg_metadata_log) | 包含关于从 Iceberg 表读取的元数据文件信息的系统表。每条记录表示一个根元数据文件、从某个 Avro 文件中提取的元数据,或某个 Avro 文件中的一条记录。 | -| [system.jemalloc_bins](/operations/system-tables/jemalloc_bins) | 系统表,包含通过 jemalloc 分配器在不同大小类别(bins)中完成的内存分配信息,并聚合自所有 arena。 | -| [system.kafka_consumers](/operations/system-tables/kafka_consumers) | 包含 Kafka 消费者相关信息的系统表。 | -| [system.licenses](/operations/system-tables/licenses) | 用于存放 ClickHouse 源码中 `contrib` 目录下第三方库许可证信息的系统表。 | -| [system.merge_tree_settings](/operations/system-tables/merge_tree_settings) | 包含有关 MergeTree 表设置的信息的系统表。 | -| [system.merges](/operations/system-tables/merges) | 包含当前正在对 MergeTree 系列表执行的合并和数据分片变更操作信息的系统表。 | -| [system.metric_log](/operations/system-tables/metric_log) | 系统表,包含来自 `system.metrics` 和 `system.events` 表的历史指标值,并会定期将其刷新到磁盘。 | -| [system.metrics](/operations/system-tables/metrics) | 包含可即时计算或具有当前值的指标的系统表。 | -| [system.moves](/operations/system-tables/moves) | 包含 MergeTree 表正在进行的数据部分移动信息的系统表。每个数据部分的移动对应表中的一行。 | -| [system.mutations](/operations/system-tables/mutations) | 包含 MergeTree 表变更及其执行进度信息的系统表。每条变更命令对应一行。 | -| [system.numbers](/operations/system-tables/numbers) | 系统表,包含一个名为 `number` 的 UInt64 单列,存储从零开始的几乎所有自然数。 | -| [system.numbers_mt](/operations/system-tables/numbers_mt) | 与 `system.numbers` 类似的 system 表,但读取是并行的,且数字可能以任意顺序返回。 | -| [system.one](/operations/system-tables/one) | 系统表,包含一行记录,该记录只有一个名为 `dummy` 的 UInt8 列,列值为 0。类似于其他 DBMS 中的 `DUAL` 表。 | -| [system.opentelemetry_span_log](/operations/system-tables/opentelemetry_span_log) | 包含关于已执行查询的 trace span 信息的系统表。 | -| [system.part_log](/operations/system-tables/part_log) | 包含 MergeTree 系列表中数据分区片段相关事件信息的 system 表,例如数据新增或合并等操作。 | -| [system.parts](/operations/system-tables/parts) | 包含 MergeTree 分区片段相关信息的系统表 | -| [system.parts_columns](/operations/system-tables/parts_columns) | 包含 MergeTree 表的分区片段和列信息的系统表。 | -| [system.processes](/operations/system-tables/processes) | 用于实现 `SHOW PROCESSLIST` 查询的系统表。 | -| [system.processors_profile_log](/operations/system-tables/processors_profile_log) | 包含处理器级性能分析信息的系统表(可在 `EXPLAIN PIPELINE` 中查看) | -| [system.projection_parts](/operations/system-tables/projection_parts) | 该系统表包含 MergeTree 系列表的投影分区片段信息。 | -| [system.projection_parts_columns](/operations/system-tables/projection_parts_columns) | 包含 MergeTree 系列表中投影分区片段内列信息的系统表 | -| [system.projections](/operations/system-tables/projections) | 包含所有表中现有 PROJECTION 信息的系统表。 | -| [system.query_cache](/operations/system-tables/query_cache) | 用于显示查询缓存内容的系统表。 | -| [system.query_condition_cache](/operations/system-tables/query_condition_cache) | 用于显示查询条件缓存内容的系统表。 | -| [system.query_log](/operations/system-tables/query_log) | 包含已执行查询信息的系统表,例如开始时间、处理时长和错误消息。 | -| [system.query_metric_log](/operations/system-tables/query_metric_log) | 一个系统表,用于存储来自 `system.events` 表的各个查询的内存和指标值历史记录,并会定期刷写到磁盘。 | -| [system.query_thread_log](/operations/system-tables/query_thread_log) | 包含执行查询的线程信息的系统表,例如线程名称、线程开始时间以及查询处理的持续时间。 | -| [system.query_views_log](/operations/system-tables/query_views_log) | 在运行查询时,包含已执行的依赖 VIEW 相关信息的系统表,例如 VIEW 类型或执行时间。 | -| [system.quota_limits](/operations/system-tables/quota_limits) | 包含所有 QUOTA 在各个时间区间最大值信息的 system 表。任意数量的行(包括零行)都可以对应于一个 QUOTA。 | -| [system.quota_usage](/operations/system-tables/quota_usage) | 包含当前 USER QUOTA 使用情况的系统表,例如 QUOTA 已使用多少以及剩余多少。 | -| [system.quotas](/operations/system-tables/quotas) | 存储 QUOTA 信息的系统表。 | -| [system.quotas_usage](/operations/system-tables/quotas_usage) | 记录所有用户 QUOTA 使用情况的系统表。 | -| [system.replicas](/operations/system-tables/replicas) | 包含本地服务器上复制表信息及其状态的 system 表,适用于监控。 | -| [system.replicated_fetches](/operations/system-tables/replicated_fetches) | 包含当前正在执行的后台拉取任务信息的系统表。 | -| [system.replication_queue](/operations/system-tables/replication_queue) | 包含 `ReplicatedMergeTree` 系列表在 ClickHouse Keeper 或 ZooKeeper 中复制队列任务信息的系统表。 | -| [system.resources](/operations/system-tables/resources) | 包含本地服务器上各资源信息的 system 表,每个资源对应一行。 | -| [system.role_grants](/operations/system-tables/role_grants) | 用于存储授予用户和角色的角色信息的系统表。 | -| [system.roles](/operations/system-tables/roles) | 包含已配置角色信息的系统表。 | -| [system.row_policies](/operations/system-tables/row_policies) | 包含某个特定表的过滤器,以及应使用该 ROW POLICY 的角色和/或用户列表的系统表。 | -| [system.s3_queue_settings](/operations/system-tables/s3_queue_settings) | 包含 S3Queue 表设置信息的系统表,自服务器版本 `24.10` 起可用。 | -| [system.scheduler](/operations/system-tables/scheduler) | 包含本地服务器上调度节点相关信息及其状态的系统表。 | -| [system.schema_inference_cache](/operations/system-tables/schema_inference_cache) | 包含所有已缓存文件模式信息的系统表。 | -| [system.server_settings](/operations/system-tables/server_settings) | 包含服务器全局设置的系统表,这些设置在 `config.xml` 中指定。 | -| [system.session_log](/operations/system-tables/session_log) | 包含所有成功和失败的登录和登出事件信息的系统表。 | -| [system.settings](/operations/system-tables/settings) | 包含当前用户会话设置信息的系统表。 | -| [system.settings_changes](/operations/system-tables/settings_changes) | 记录先前 ClickHouse 版本中 SETTING 变更信息的系统表。 | -| [system.settings_profile_elements](/operations/system-tables/settings_profile_elements) | 用于描述设置配置文件内容的系统表:该设置配置文件适用的约束、角色和用户,以及其父设置配置文件。 | -| [system.settings_profiles](/operations/system-tables/settings_profiles) | 用于存储已配置设置配置文件属性的系统表。 | -| [system.stack_trace](/operations/system-tables/stack_trace) | 包含所有服务器线程堆栈跟踪信息的系统表。允许开发人员检查服务器状态。 | -| [system.storage_policies](/operations/system-tables/storage_policies) | 包含有关在服务器配置中定义的存储策略和卷的信息的系统表。 | -| [system.symbols](/operations/system-tables/symbols) | 适用于 C++ 专家和 ClickHouse 工程师的系统表,包含用于对 `clickhouse` 可执行文件进行自省的信息。 | -| [system.table_engines](/operations/system-tables/table_engines) | 包含服务器所支持的表引擎及其各自支持特性描述的系统表。 | -| [system.tables](/operations/system-tables/tables) | 包含服务器已知的所有表元数据的 system 表。 | -| [system.text_log](/operations/system-tables/text_log) | 包含日志条目的系统表。 | -| [system.time_zones](/operations/system-tables/time_zones) | 一个系统表,其中包含 ClickHouse 服务器支持的时区列表。 | -| [system.trace_log](/operations/system-tables/trace_log) | 包含由采样查询分析器收集的堆栈跟踪的系统表。 | -| [system.unicode](/operations/system-tables/unicode) | 系统表,包含 Unicode 字符及其属性的列表。 | -| [system.user_processes](/operations/system-tables/user_processes) | 用于概览用户内存使用情况和 ProfileEvents 的信息所在的系统表。 | -| [system.users](/operations/system-tables/users) | 包含服务器上已配置的用户账号列表的系统表。 | -| [system.view_refreshes](/operations/system-tables/view_refreshes) | 包含可刷新materialized view相关信息的系统表。 | -| [system.warnings](/operations/system-tables/system_warnings) | 此表包含关于 ClickHouse 服务器的警告信息。 | -| [system.workloads](/operations/system-tables/workloads) | 包含本地服务器上工作负载信息的系统表。 | -| [system.zookeeper](/operations/system-tables/zookeeper) | 仅在配置了 ClickHouse Keeper 或 ZooKeeper 时才存在的系统表。它暴露来自配置文件中定义的 Keeper 集群的数据。 | -| [system.zookeeper_connection](/operations/system-tables/zookeeper_connection) | 仅在配置了 ZooKeeper 时才存在的系统表。显示当前与 ZooKeeper 的连接(包括辅助 ZooKeeper 实例)。 | -| [system.zookeeper_connection_log](/operations/system-tables/zookeeper_connection_log) | 显示 ZooKeeper 连接历史(包括辅助 ZooKeeper 实例)。 | -| [system.zookeeper_log](/operations/system-tables/zookeeper_log) | 包含关于向 ZooKeeper 服务器发起请求的参数及其响应信息的系统表。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md deleted file mode 100644 index 18809a584d5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/information_schema.md +++ /dev/null @@ -1,391 +0,0 @@ ---- -description: '提供几乎标准化、与 DBMS 无关的数据库对象元数据视图的系统数据库。' -keywords: ['system database', 'information_schema'] -slug: /operations/system-tables/information_schema -title: 'INFORMATION_SCHEMA' -doc_type: 'reference' ---- - -`INFORMATION_SCHEMA`(或:`information_schema`)是一个系统数据库,用于针对数据库对象元数据提供一个(某种程度上)标准化的、[与 DBMS 无关的视图](https://en.wikipedia.org/wiki/Information_schema)。`INFORMATION_SCHEMA` 中的视图通常不如普通系统表强大,但各类工具可以使用它们以跨 DBMS 的方式获取基本信息。`INFORMATION_SCHEMA` 中视图的结构和内容应当以向后兼容的方式演进,即只添加新功能,而不修改或移除现有功能。从内部实现角度看,`INFORMATION_SCHEMA` 中的视图通常映射到普通系统表,例如 [system.columns](../../operations/system-tables/columns.md)、[system.databases](../../operations/system-tables/databases.md) 和 [system.tables](../../operations/system-tables/tables.md)。 - -```sql -SHOW TABLES FROM INFORMATION_SCHEMA; - --- 或者: -SHOW TABLES FROM information_schema; -``` - -```text -┌─name────────────────────┐ -│ COLUMNS │ -│ KEY_COLUMN_USAGE │ -│ REFERENTIAL_CONSTRAINTS │ -│ SCHEMATA │ -| STATISTICS | -│ TABLES │ -│ VIEWS │ -│ columns │ -│ key_column_usage │ -│ referential_constraints │ -│ schemata │ -| statistics | -│ tables │ -│ views │ -└─────────────────────────┘ -``` - -`INFORMATION_SCHEMA` 包含以下视图: - -* [COLUMNS](#columns) -* [KEY_COLUMN_USAGE](#key_column_usage) -* [REFERENTIAL_CONSTRAINTS](#referential_constraints) -* [SCHEMATA](#schemata) -* [STATISTICS](#statistics) -* [TABLES](#tables) -* [VIEWS](#views) - -为与其他数据库兼容,提供了不区分大小写的等价视图,例如 `INFORMATION_SCHEMA.columns`。这些视图中的所有列也同样如此——同时提供小写(例如 `table_name`)和大写(例如 `TABLE_NAME`)两种形式。 - - -## 列 {#columns} - -包含从 [system.columns](../../operations/system-tables/columns.md) 系统表中读取的列,以及那些在 ClickHouse 中不受支持或没有意义(始终为 `NULL`),但根据标准仍必须保留的列。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -查询: - -```sql -SELECT table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - column_default, - is_nullable, - data_type, - character_maximum_length, - character_octet_length, - numeric_precision, - numeric_precision_radix, - numeric_scale, - datetime_precision, - character_set_catalog, - character_set_schema, - character_set_name, - collation_catalog, - collation_schema, - collation_name, - domain_catalog, - domain_schema, - domain_name, - column_comment, - column_type -FROM INFORMATION_SCHEMA.COLUMNS -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -结果: - -```text -第 1 行: -────── -table_catalog: default -table_schema: default -table_name: describe_example -column_name: id -ordinal_position: 1 -column_default: -is_nullable: 0 -data_type: UInt64 -character_maximum_length: ᴺᵁᴸᴸ -character_octet_length: ᴺᵁᴸᴸ -numeric_precision: 64 -numeric_precision_radix: 2 -numeric_scale: 0 -datetime_precision: ᴺᵁᴸᴸ -character_set_catalog: ᴺᵁᴸᴸ -character_set_schema: ᴺᵁᴸᴸ -character_set_name: ᴺᵁᴸᴸ -collation_catalog: ᴺᵁᴸᴸ -collation_schema: ᴺᵁᴸᴸ -collation_name: ᴺᵁᴸᴸ -domain_catalog: ᴺᵁᴸᴸ -domain_schema: ᴺᵁᴸᴸ -domain_name: ᴺᵁᴸᴸ -``` - - -## SCHEMATA {#schemata} - -包含从 [system.databases](../../operations/system-tables/databases.md) 系统表中读取的列,以及在 ClickHouse 中不受支持或没有意义(始终为 `NULL`),但根据标准要求必须存在的列。 - -列: - -* `catalog_name` ([String](../../sql-reference/data-types/string.md)) — 数据库名称。 -* `schema_name` ([String](../../sql-reference/data-types/string.md)) — 数据库名称。 -* `schema_owner` ([String](../../sql-reference/data-types/string.md)) — schema 所有者名称,始终为 `'default'`。 -* `default_character_set_catalog` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`,不受支持。 -* `default_character_set_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`,不受支持。 -* `default_character_set_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`,不受支持。 -* `sql_path` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — `NULL`,不受支持。 - -**示例** - -查询: - -```sql -SELECT catalog_name, - schema_name, - schema_owner, - default_character_set_catalog, - default_character_set_schema, - default_character_set_name, - sql_path -FROM information_schema.schemata -WHERE schema_name ILIKE 'information_schema' -LIMIT 1 -FORMAT Vertical; -``` - -结果: - -```text -Row 1: -────── -catalog_name: INFORMATION_SCHEMA -schema_name: INFORMATION_SCHEMA -schema_owner: default -default_character_set_catalog: ᴺᵁᴸᴸ -default_character_set_schema: ᴺᵁᴸᴸ -default_character_set_name: ᴺᵁᴸᴸ -sql_path: ᴺᵁᴸᴸ -``` - - -## TABLES {#tables} - -包含从 [system.tables](../../operations/system-tables/tables.md) 系统表中读取的列。 - -列: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — 表名。 -* `table_type` ([String](../../sql-reference/data-types/string.md)) — 表类型。可能的取值: - * `BASE TABLE` - * `VIEW` - * `FOREIGN TABLE` - * `LOCAL TEMPORARY` - * `SYSTEM VIEW` -* `table_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 总行数。如果无法确定则为 NULL。 -* `data_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 磁盘上的数据大小。如果无法确定则为 NULL。 -* `index_length` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 主键、二级索引以及所有标记的总大小。 -* `table_collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 表的默认排序规则。始终为 `utf8mb4_0900_ai_ci`。 -* `table_comment` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 创建表时使用的注释。 - -**示例** - -查询: - -```sql -SELECT table_catalog, - table_schema, - table_name, - table_type, - table_collation, - table_comment -FROM INFORMATION_SCHEMA.TABLES -WHERE (table_schema = currentDatabase() OR table_schema = '') - AND table_name NOT LIKE '%inner%' -LIMIT 1 -FORMAT Vertical; -``` - -结果: - -```text -第 1 行: -────── -table_catalog: default -table_schema: default -table_name: describe_example -table_type: BASE TABLE -table_collation: utf8mb4_0900_ai_ci -table_comment: -``` - - -## 视图 {#views} - -在使用 [View](../../engines/table-engines/special/view.md) 表引擎时,包含从 [system.tables](../../operations/system-tables/tables.md) 系统表读取的列。 - -列: - -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — 表名。 -* `view_definition` ([String](../../sql-reference/data-types/string.md)) — 视图的 `SELECT` 查询。 -* `check_option` ([String](../../sql-reference/data-types/string.md)) — `NONE`,不进行检查。 -* `is_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`,该视图不可更新。 -* `is_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — 指示创建的视图是否为[物化视图](/sql-reference/statements/create/view#materialized-view)。可能的值: - * `NO` — 创建的视图不是物化视图。 - * `YES` — 创建的视图是物化视图。 -* `is_trigger_updatable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`,该触发器不可更新。 -* `is_trigger_deletable` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`,该触发器不会被删除。 -* `is_trigger_insertable_into` ([Enum8](../../sql-reference/data-types/enum.md)) — `NO`,不会向该触发器插入数据。 - -**示例** - -查询: - -```sql -CREATE VIEW v (n Nullable(Int32), f Float64) AS SELECT n, f FROM t; -CREATE MATERIALIZED VIEW mv ENGINE = Null AS SELECT * FROM system.one; -SELECT table_catalog, - table_schema, - table_name, - view_definition, - check_option, - is_updatable, - is_insertable_into, - is_trigger_updatable, - is_trigger_deletable, - is_trigger_insertable_into -FROM information_schema.views -WHERE table_schema = currentDatabase() -LIMIT 1 -FORMAT Vertical; -``` - -结果: - -```text -Row 1: -────── -table_catalog: default -table_schema: default -table_name: mv -view_definition: SELECT * FROM system.one -check_option: NONE -is_updatable: NO -is_insertable_into: YES -is_trigger_updatable: NO -is_trigger_deletable: NO -is_trigger_insertable_into: NO -``` - - -## KEY_COLUMN_USAGE {#key_column_usage} - -包含 [system.tables](../../operations/system-tables/tables.md) 系统表中受约束限制的列。 - -列: - -* `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。始终为 `def`。 -* `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 约束所属模式(数据库)的名称。 -* `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 约束名称。 -* `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。始终为 `def`。 -* `table_schema` ([String](../../sql-reference/data-types/string.md)) — 表所属模式(数据库)的名称。 -* `table_name` ([String](../../sql-reference/data-types/string.md)) — 具有该约束的表的名称。 -* `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 具有该约束的列的名称。 -* `ordinal_position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 当前未使用。始终为 `1`。 -* `position_in_unique_constraint` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt32](../../sql-reference/data-types/int-uint.md))) — 当前未使用。始终为 `NULL`。 -* `referenced_table_schema` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。始终为 NULL。 -* `referenced_table_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。始终为 NULL。 -* `referenced_column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。始终为 NULL。 - -**示例** - -```sql -CREATE TABLE test (i UInt32, s String) ENGINE MergeTree ORDER BY i; -SELECT constraint_catalog, - constraint_schema, - constraint_name, - table_catalog, - table_schema, - table_name, - column_name, - ordinal_position, - position_in_unique_constraint, - referenced_table_schema, - referenced_table_name, - referenced_column_name -FROM information_schema.key_column_usage -WHERE table_name = 'test' -FORMAT Vertical; -``` - -结果: - -```response -Row 1: -────── -constraint_catalog: def -constraint_schema: default -constraint_name: PRIMARY -table_catalog: def -table_schema: default -table_name: test -column_name: i -ordinal_position: 1 -position_in_unique_constraint: ᴺᵁᴸᴸ -referenced_table_schema: ᴺᵁᴸᴸ -referenced_table_name: ᴺᵁᴸᴸ -referenced_column_name: ᴺᵁᴸᴸ -``` - - -## REFERENTIAL_CONSTRAINTS {#referential_constraints} - -包含外键信息。目前返回空结果(无行),这一点已足以与 Tableau Online 等第三方工具保持兼容。 - -列: - -- `constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 目前未使用。 -- `unique_constraint_catalog` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `unique_constraint_schema` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `unique_constraint_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 目前未使用。 -- `match_option` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `update_rule` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `delete_rule` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `table_name` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 -- `referenced_table_name` ([String](../../sql-reference/data-types/string.md)) — 目前未使用。 - - - -## STATISTICS {#statistics} - -提供有关表索引的信息。目前返回空结果(无任何行),仅用于与 Tableau Online 等第三方工具保持兼容。 - -列: - -- `table_catalog` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `table_schema` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `table_name` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `non_unique` ([Int32](../../sql-reference/data-types/int-uint.md)) — 当前未使用。 -- `index_schema` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `index_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。 -- `seq_in_index` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 当前未使用。 -- `column_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。 -- `collation` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。 -- `cardinality` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — 当前未使用。 -- `sub_part` ([Nullable](../../sql-reference/data-types/nullable.md)([Int64](../../sql-reference/data-types/int-uint.md))) — 当前未使用。 -- `packed` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。 -- `nullable` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `index_type` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `comment` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `index_comment` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `is_visible` ([String](../../sql-reference/data-types/string.md)) — 当前未使用。 -- `expression` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 当前未使用。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md deleted file mode 100644 index 56404f66e1b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/jemalloc_bins.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '系统表,包含通过 jemalloc 分配器在不同大小类别(bins)中的内存分配信息,这些信息汇总自所有 arena。' -keywords: ['system table', 'jemalloc_bins'] -slug: /operations/system-tables/jemalloc_bins -title: 'system.jemalloc_bins' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含由 `jemalloc` 分配器在不同大小类(bin)中完成的内存分配信息,这些信息是从所有 arena 聚合而来的。 -由于 `jemalloc` 中的线程本地缓存,这些统计数据可能并非绝对精确。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -找出对当前整体内存使用量贡献最大的内存分配大小。 - -```sql -SELECT - *, - allocations - deallocations AS active_allocations, - size * active_allocations AS allocated_bytes -FROM system.jemalloc_bins -WHERE allocated_bytes > 0 -ORDER BY allocated_bytes DESC -LIMIT 10 -``` - -```text -┌─索引─┬─大型─┬─────大小─┬─分配次数─┬─释放次数─┬─活跃分配数─┬─已分配字节─┐ -│ 82 │ 1 │ 50331648 │ 1 │ 0 │ 1 │ 50331648 │ -│ 10 │ 0 │ 192 │ 512336 │ 370710 │ 141626 │ 27192192 │ -│ 69 │ 1 │ 5242880 │ 6 │ 2 │ 4 │ 20971520 │ -│ 3 │ 0 │ 48 │ 16938224 │ 16559484 │ 378740 │ 18179520 │ -│ 28 │ 0 │ 4096 │ 122924 │ 119142 │ 3782 │ 15491072 │ -│ 61 │ 1 │ 1310720 │ 44569 │ 44558 │ 11 │ 14417920 │ -│ 39 │ 1 │ 28672 │ 1285 │ 913 │ 372 │ 10665984 │ -│ 4 │ 0 │ 64 │ 2837225 │ 2680568 │ 156657 │ 10026048 │ -│ 6 │ 0 │ 96 │ 2617803 │ 2531435 │ 86368 │ 8291328 │ -│ 36 │ 1 │ 16384 │ 22431 │ 21970 │ 461 │ 7553024 │ -└───────┴───────┴──────────┴──────────────┴───────────────┴────────────────────┴─────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md deleted file mode 100644 index f48f6157a8b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/kafka_consumers.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '包含 Kafka 消费者信息的系统表。' -keywords: ['system table', 'kafka_consumers'] -slug: /operations/system-tables/kafka_consumers -title: 'system.kafka_consumers' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -包含有关 Kafka 消费者的信息。 -适用于 [Kafka 表引擎](../../engines/table-engines/integrations/kafka)(ClickHouse 原生集成) - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -示例: - -```sql -SELECT * -FROM system.kafka_consumers -FORMAT Vertical -``` - -```text -第 1 行: -────── -database: test -table: kafka -consumer_id: ClickHouse-instance-test-kafka-1caddc7f-f917-4bb1-ac55-e28bd103a4a0 -assignments.topic: ['system_kafka_cons'] -assignments.partition_id: [0] -assignments.current_offset: [18446744073709550615] -exceptions.time: [] -exceptions.text: [] -last_poll_time: 2006-11-09 18:47:47 -num_messages_read: 4 -last_commit_time: 2006-11-10 04:39:40 -num_commits: 1 -last_rebalance_time: 1970-01-01 00:00:00 -num_rebalance_revocations: 0 -num_rebalance_assignments: 1 -is_currently_used: 1 -rdkafka_stat: {...} - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md deleted file mode 100644 index 2c325b10250..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/licenses.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -description: '包含位于 ClickHouse 源码的 contrib 目录中的第三方库的许可证的系统表。' -keywords: ['系统表', '许可证'] -slug: /operations/system-tables/licenses -title: 'system.licenses' -doc_type: 'reference' ---- - -# system.licenses {#systemlicenses} - -包含位于 ClickHouse 源码中 [contrib](https://github.com/ClickHouse/ClickHouse/tree/master/contrib) 目录内第三方库的许可证信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT library_name, license_type, license_path FROM system.licenses LIMIT 15 -``` - -```text -┌─library_name───────┬─license_type─┬─license_path────────────────────────┐ -│ aws-c-common │ Apache │ /contrib/aws-c-common/LICENSE │ -│ base64 │ BSD 2-clause │ /contrib/aklomp-base64/LICENSE │ -│ brotli │ MIT │ /contrib/brotli/LICENSE │ -│ [...] │ [...] │ [...] │ -└────────────────────┴──────────────┴─────────────────────────────────────┘ - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md deleted file mode 100644 index d2ade38eca4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merge_tree_settings.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: '包含 MergeTree 表设置信息的系统表。' -keywords: ['system table', 'merge_tree_settings'] -slug: /operations/system-tables/merge_tree_settings -title: 'system.merge_tree_settings' -doc_type: 'reference' ---- - -# system.merge_tree_settings {#systemmerge_tree_settings} - -包含 `MergeTree` 表相关设置的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.merge_tree_settings LIMIT 3 FORMAT Vertical; -``` - -```response -SELECT * -FROM system.merge_tree_settings -LIMIT 3 -FORMAT Vertical - -Query id: 2580779c-776e-465f-a90c-4b7630d0bb70 - -Row 1: -────── -name: min_compress_block_size -value: 0 -default: 0 -changed: 0 -description: 写入颗粒时,如果缓冲区中待压缩数据的大小大于或等于指定阈值,则对缓冲区中的数据进行压缩。如果未设置此项,则使用相应的全局设置。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 2: -────── -name: max_compress_block_size -value: 0 -default: 0 -changed: 0 -description: 如果缓冲区中待压缩数据的大小大于或等于指定阈值,则对其进行压缩。即使当前颗粒尚未完成,数据块也会被压缩。如果未设置此项,则使用相应的全局设置。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -Row 3: -────── -name: index_granularity -value: 8192 -default: 8192 -changed: 0 -description: 一个主键值对应的行数。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -is_obsolete: 0 -tier: Production - -3 rows in set. Elapsed: 0.001 sec. -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md deleted file mode 100644 index 3d28c082dc5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/merges.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: '包含当前正在对 MergeTree 系列表执行的合并和部件变更信息的系统表。' -keywords: ['系统表', '合并'] -slug: /operations/system-tables/merges -title: 'system.merges' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.merges {#systemmerges} - - - -包含关于 MergeTree 引擎系列表当前正在执行的合并(merge)和数据分片变更(part mutation)的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md deleted file mode 100644 index 576ff8427dc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metric_log.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '系统表,包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录,并会定期写入磁盘。' -keywords: ['system 表', 'metric_log'] -slug: /operations/system-tables/metric_log -title: 'system.metric_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.metric_log {#systemmetric_log} - - - -包含来自 `system.metrics` 和 `system.events` 表的指标值的历史记录,这些数据会定期刷写到磁盘。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒精度的事件时间。 - -**示例** - -```sql -SELECT * FROM system.metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -第 1 行: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -milliseconds: 196 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -... -CurrentMetric_Revision: 54439 -CurrentMetric_VersionInteger: 20009001 -CurrentMetric_RWLockWaitingReaders: 0 -CurrentMetric_RWLockWaitingWriters: 0 -CurrentMetric_RWLockActiveReaders: 0 -CurrentMetric_RWLockActiveWriters: 0 -CurrentMetric_GlobalThread: 74 -CurrentMetric_GlobalThreadActive: 26 -CurrentMetric_LocalThread: 0 -CurrentMetric_LocalThreadActive: 0 -CurrentMetric_DistributedFilesToInsert: 0 -``` - -**模式** - -可以使用 XML 标签 `` 将此表配置为不同的模式类型。默认模式类型为 `wide`,在该模式下,每个指标或 profile 事件都会作为单独的一列存储。此模式在仅读取单个列的场景下具有最高性能和效率。 - -`transposed` 模式以类似于 `system.asynchronous_metric_log` 的格式存储数据,其中指标和事件以行的形式存储。该模式适用于资源受限的环境,因为它在数据合并期间可以减少资源消耗。 - -还提供一种兼容性模式 `transposed_with_wide_view`,它使用 transposed 模式(`system.transposed_metric_log`)的表来存储实际数据,并在其之上基于 wide 模式创建一个视图。该视图会查询 transposed 表,因此在从 `wide` 模式迁移到 `transposed` 模式时非常有用。 - -**另请参阅** - -* [metric_log 设置](../../operations/server-configuration-parameters/settings.md#metric_log) — 启用和禁用该设置。 -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — 包含周期性计算的指标。 -* [system.events](/operations/system-tables/events) — 包含发生的一系列事件。 -* [system.metrics](../../operations/system-tables/metrics.md) — 包含即时计算的指标。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md deleted file mode 100644 index 8ac18eed951..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/metrics.md +++ /dev/null @@ -1,789 +0,0 @@ ---- -description: '系统表,包含可即时计算或表示当前值的指标。' -keywords: ['系统表', '指标'] -slug: /operations/system-tables/metrics -title: 'system.metrics' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.metrics {#systemmetrics} - - - -包含可以实时计算或具有当前值的指标。例如,同时处理的查询数量或当前副本延迟。此表始终保持最新。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -可以在源文件 [src/Common/CurrentMetrics.cpp](https://github.com/ClickHouse/ClickHouse/blob/master/src/Common/CurrentMetrics.cpp) 中查阅所有受支持的指标。 - -**示例** - -```sql -SELECT * FROM system.metrics LIMIT 10 -``` - -```text -┌─metric───────────────────────────────┬─value─┬─description────────────────────────────────────────────────────────────┐ -│ Query │ 1 │ 正在执行的查询数量 │ -│ Merge │ 0 │ 正在执行的后台合并数量 │ -│ PartMutation │ 0 │ 变更操作数量(ALTER DELETE/UPDATE) │ -│ ReplicatedFetch │ 0 │ 正在从副本获取的数据分片数量 │ -│ ReplicatedSend │ 0 │ 正在发送到副本的数据分片数量 │ -│ ReplicatedChecks │ 0 │ 正在进行一致性检查的数据分片数量 │ -│ BackgroundMergesAndMutationsPoolTask │ 0 │ 关联后台池中活跃的合并和变更任务数量 │ -│ BackgroundFetchesPoolTask │ 0 │ 关联后台池中活跃的获取任务数量 │ -│ BackgroundCommonPoolTask │ 0 │ 关联后台池中活跃的任务数量 │ -│ BackgroundMovePoolTask │ 0 │ BackgroundProcessingPool 中用于移动操作的活跃任务数量 │ -└──────────────────────────────────────┴───────┴────────────────────────────────────────────────────────────────────────┘ -``` - - -## 指标说明 {#metric-descriptions} - -### AggregatorThreads {#aggregatorthreads} - -Aggregator 线程池中的线程数量。 - -### AggregatorThreadsActive {#aggregatorthreadsactive} - -正在执行任务的 Aggregator 线程池中的线程数量。 - -### TablesLoaderForegroundThreads {#tablesloaderforegroundthreads} - -异步加载器前台线程池的线程数。 - -### TablesLoaderForegroundThreadsActive {#tablesloaderforegroundthreadsactive} - -异步加载器前台线程池中正在执行任务的线程数量。 - -### TablesLoaderBackgroundThreads {#tablesloaderbackgroundthreads} - -异步加载器后台线程池中的线程数。 - -### TablesLoaderBackgroundThreadsActive {#tablesloaderbackgroundthreadsactive} - -异步加载器后台线程池中正在执行任务的线程数量。 - -### AsyncInsertCacheSize {#asyncinsertcachesize} - -缓存中异步插入哈希 ID 的数量 - -### AsynchronousInsertThreads {#asynchronousinsertthreads} - -`AsynchronousInsert` 线程池中的线程数。 - -### AsynchronousInsertThreadsActive {#asynchronousinsertthreadsactive} - -AsynchronousInsert 线程池中正在执行任务的线程数量。 - -### AsynchronousReadWait {#asynchronousreadwait} - -等待异步读取的线程数量。 - -### BackgroundBufferFlushSchedulePoolSize {#backgroundbufferflushschedulepoolsize} - -BackgroundBufferFlushSchedulePool 中任务数的上限 - -### BackgroundBufferFlushSchedulePoolTask {#backgroundbufferflushschedulepooltask} - -BackgroundBufferFlushSchedulePool 中正在运行的任务数。该池用于定期刷新 Buffer 数据。 - -### BackgroundCommonPoolSize {#backgroundcommonpoolsize} - -关联后台线程池中任务数量的上限 - -### BackgroundCommonPoolTask {#backgroundcommonpooltask} - -关联的后台池中当前活动任务的数量 - -### BackgroundDistributedSchedulePoolSize {#backgrounddistributedschedulepoolsize} - -限制 BackgroundDistributedSchedulePool 中的任务数量 - -### BackgroundDistributedSchedulePoolTask {#backgrounddistributedschedulepooltask} - -BackgroundDistributedSchedulePool 中当前活动任务的数量。该池用于在后台执行分布式发送操作。 - -### BackgroundFetchesPoolSize {#backgroundfetchespoolsize} - -关联的后台池中可同时进行的获取操作数量上限 - -### BackgroundFetchesPoolTask {#backgroundfetchespooltask} - -关联后台池中的活动获取任务数量 - -### BackgroundMergesAndMutationsPoolSize {#backgroundmergesandmutationspoolsize} - -限制关联后台池中可同时进行的合并和变更任务数量 - -### BackgroundMergesAndMutationsPoolTask {#backgroundmergesandmutationspooltask} - -对应后台池中正在执行的合并与变更任务数量 - -### BackgroundMessageBrokerSchedulePoolSize {#backgroundmessagebrokerschedulepoolsize} - -用于消息流的 BackgroundProcessingPool 中任务数量的上限 - -### BackgroundMessageBrokerSchedulePoolTask {#backgroundmessagebrokerschedulepooltask} - -用于消息流的 BackgroundProcessingPool 中活动任务的数量 - -### BackgroundMovePoolSize {#backgroundmovepoolsize} - -限制 BackgroundProcessingPool 中用于 `move` 操作的任务数量 - -### BackgroundMovePoolTask {#backgroundmovepooltask} - -BackgroundProcessingPool 中用于执行 move 操作的活动任务数 - -### BackgroundSchedulePoolSize {#backgroundschedulepoolsize} - -BackgroundSchedulePool 中任务数量的上限。该池用于定期执行 ReplicatedMergeTree 相关任务,例如清理旧数据部件、变更数据部件、副本重新初始化等。 - -### BackgroundSchedulePoolTask {#backgroundschedulepooltask} - -BackgroundSchedulePool 中的活动任务数。该池用于周期性执行 ReplicatedMergeTree 相关任务,例如清理旧数据部件、变更数据部件、重新初始化副本等。 - -### BackupsIOThreads {#backupsiothreads} - -BackupsIO 线程池中的线程数。 - -### BackupsIOThreadsActive {#backupsiothreadsactive} - -BackupsIO 线程池中正在运行任务的线程数量。 - -### BackupsThreads {#backupsthreads} - -用于执行 BACKUP 操作的线程池线程数量。 - -### BackupsThreadsActive {#backupsthreadsactive} - -用于执行 BACKUP 任务的线程池中正在运行任务的线程数量。 - -### BrokenDistributedFilesToInsert {#brokendistributedfilestoinsert} - -已被标记为损坏的、用于异步写入到 `Distributed` 表的文件数量。该指标在启动时从 0 开始计数。各分片中的文件数量会被汇总。 - -### CacheDetachedFileSegments {#cachedetachedfilesegments} - -当前已分离的缓存文件段数量 - -### CacheDictionaryThreads {#cachedictionarythreads} - -`CacheDictionary` 线程池的线程数。 - -### CacheDictionaryThreadsActive {#cachedictionarythreadsactive} - -CacheDictionary 线程池中正在运行任务的线程数量。 - -### CacheDictionaryUpdateQueueBatches {#cachedictionaryupdatequeuebatches} - -CacheDictionaries 中更新队列中的“批次”(一组键)的数量。 - -### CacheDictionaryUpdateQueueKeys {#cachedictionaryupdatequeuekeys} - -CacheDictionary 更新队列中键的确切数量。 - -### CacheFileSegments {#cachefilesegments} - -现有缓存文件段的数量 - -### ContextLockWait {#contextlockwait} - -在 Context 中等待锁的线程数量。这是全局锁。 - -### DDLWorkerThreads {#ddlworkerthreads} - -用于处理 `ON CLUSTER` 查询的 DDLWorker 线程池线程数量。 - -### DDLWorkerThreadsActive {#ddlworkerthreadsactive} - -当前在为 ON CLUSTER 查询执行任务的 DDLWORKER 线程池中的线程数量。 - -### DatabaseCatalogThreads {#databasecatalogthreads} - -DatabaseCatalog 线程池中的线程数量。 - -### DatabaseCatalogThreadsActive {#databasecatalogthreadsactive} - -DatabaseCatalog 线程池中正在执行任务的线程数量。 - -### DatabaseOnDiskThreads {#databaseondiskthreads} - -DatabaseOnDisk 线程池中的线程数量。 - -### DatabaseOnDiskThreadsActive {#databaseondiskthreadsactive} - -DatabaseOnDisk 线程池中正在运行任务的线程数。 - -### DelayedInserts {#delayedinserts} - -由于 MergeTree 表某个分区的活动数据 part 数量过多而被限流的 INSERT 查询次数。 - -### DestroyAggregatesThreads {#destroyaggregatesthreads} - -用于销毁聚合状态的线程池中的线程数。 - -### DestroyAggregatesThreadsActive {#destroyaggregatesthreadsactive} - -用于销毁聚合状态的线程池中正在运行任务的线程数。 - -### DictCacheRequests {#dictcacherequests} - -正在发送到缓存类型字典的数据源的请求数量。 - -### DiskObjectStorageAsyncThreads {#diskobjectstorageasyncthreads} - -用于 DiskObjectStorage 的异步线程池中的线程数量。 - -### DiskObjectStorageAsyncThreadsActive {#diskobjectstorageasyncthreadsactive} - -DiskObjectStorage 的异步线程池中正在执行任务的线程数量。 - -### DiskSpaceReservedForMerge {#diskspacereservedformerge} - -为当前正在运行的后台合并操作预留的磁盘空间。该值略大于当前参与合并的各个 part 的总大小。 - -### DistributedFilesToInsert {#distributedfilestoinsert} - -用于异步插入到 Distributed 表的待处理文件数量。会对每个分片上的文件数量求和。 - -### DistributedSend {#distributedsend} - -向远程服务器发送插入到 Distributed 表中的数据时所建立的连接数量。适用于同步和异步两种模式。 - -### EphemeralNode {#ephemeralnode} - -当前在 ZooKeeper 中的临时节点数量。 - -### FilesystemCacheElements {#filesystemcacheelements} - -文件系统缓存元素(文件片段) - -### FilesystemCacheReadBuffers {#filesystemcachereadbuffers} - -活动缓存缓冲区数量 - -### FilesystemCacheSize {#filesystemcachesize} - -文件系统缓存大小(以字节为单位) - -### QueryCacheBytes {#querycachebytes} - -查询缓存的总大小,单位为字节。 - -### QueryCacheEntries {#querycacheentries} - -查询缓存中的总条目数。 - -### UncompressedCacheBytes {#uncompressedcachebytes} - -未压缩缓存的总大小(以字节为单位)。未压缩缓存通常不会带来性能提升,应尽量避免使用。 - -### UncompressedCacheCells {#uncompressedcachecells} - -### CompiledExpressionCacheBytes {#compiledexpressioncachebytes} - -JIT 编译代码缓存占用的总字节数。 - -### CompiledExpressionCacheCount {#compiledexpressioncachecount} - -JIT 编译代码缓存中的总条目数。 - -### MMapCacheCells {#mmapcachecells} - -使用 `mmap`(内存映射)打开的文件数量。该指标适用于将 `local_filesystem_read_method` 设置为 `mmap` 的查询。使用 `mmap` 打开的文件会保留在缓存中,以避免代价高昂的 TLB 刷新。 - -### MarkCacheBytes {#markcachebytes} - -标记缓存总大小(字节) - -### MarkCacheFiles {#markcachefiles} - -标记缓存中已缓存的标记文件的总数 - -### GlobalThread {#globalthread} - -全局线程池中的线程数。 - -### GlobalThreadActive {#globalthreadactive} - -当前在全局线程池中执行任务的线程数。 - -### HTTPConnection {#httpconnection} - -HTTP 服务器连接数 - -### HashedDictionaryThreads {#hasheddictionarythreads} - -HashedDictionary 线程池的线程数量。 - -### HashedDictionaryThreadsActive {#hasheddictionarythreadsactive} - -HashedDictionary 线程池中正在运行任务的线程数。 - -### IOPrefetchThreads {#ioprefetchthreads} - -IO 预取线程池中的线程数量。 - -### IOPrefetchThreadsActive {#ioprefetchthreadsactive} - -IO 预取线程池中正在运行任务的线程数量。 - -### IOThreads {#iothreads} - -IO 线程池中的线程数。 - -### IOThreadsActive {#iothreadsactive} - -IO 线程池中正在运行任务的线程数。 - -### IOUringInFlightEvents {#iouringinflightevents} - -正在进行的 io_uring SQE 数量 - -### IOUringPendingEvents {#iouringpendingevents} - -尚未提交的 io_uring SQE 数量 - -### IOWriterThreads {#iowriterthreads} - -I/O 写入线程池中的线程数。 - -### IOWriterThreadsActive {#iowriterthreadsactive} - -IO 写入线程池中正在运行任务的线程数量。 - -### InterserverConnection {#interserverconnection} - -来自其他副本用于获取数据片段的连接数 - -### KafkaAssignedPartitions {#kafkaassignedpartitions} - -当前分配给 Kafka 表的分区数量 - -### KafkaBackgroundReads {#kafkabackgroundreads} - -当前正在执行的后台读取操作数量(用于从 Kafka 填充物化视图) - -### KafkaConsumers {#kafkaconsumers} - -活动 Kafka 消费者数量 - -### KafkaConsumersInUse {#kafkaconsumersinuse} - -当前用于直接读取或后台读取的消费者数量 - -### KafkaConsumersWithAssignment {#kafkaconsumerswithassignment} - -具有分区分配的活动 Kafka 消费者数量。 - -### KafkaLibrdkafkaThreads {#kafkalibrdkafkathreads} - -当前活动的 librdkafka 线程数 - -### KafkaProducers {#kafkaproducers} - -已创建的活跃 Kafka 生产者数量 - -### KafkaWrites {#kafkawrites} - -当前正在执行的写入 Kafka 的插入操作数 - -### KeeperAliveConnections {#keeperaliveconnections} - -活动连接数 - -### KeeperOutstandingRequests {#keeperoutstandingrequests} - -未处理请求数 - -### LocalThread {#localthread} - -本地线程池中的线程数量。本地线程池中的线程从全局线程池中获取。 - -### LocalThreadActive {#localthreadactive} - -本地线程池中正在执行任务的线程数量。 - -### MMappedAllocBytes {#mmappedallocbytes} - -通过 mmap 分配的字节总数 - -### MMappedAllocs {#mmappedallocs} - -通过 mmap 进行映射分配的总次数 - -### MMappedFileBytes {#mmappedfilebytes} - -所有内存映射文件区域大小的总和。 - -### MMappedFiles {#mmappedfiles} - -内存映射文件(mmapped files)的总数。 - -### MarksLoaderThreads {#marksloaderthreads} - -用于加载标记的线程池线程数量。 - -### MarksLoaderThreadsActive {#marksloaderthreadsactive} - -用于加载标记的线程池中当前正在执行任务的线程数量。 - -### MaxDDLEntryID {#maxddlentryid} - -DDLWorker 已处理的最大 DDL 条目 ID。 - -### MaxPushedDDLEntryID {#maxpushedddlentryid} - -DDLWorker 推送到 ZooKeeper 的最大 DDL 条目 ID。 - -### MemoryTracking {#memorytracking} - -服务器已分配的内存总量(字节)。 - -### 合并 {#merge} - -正在执行的后台合并操作数量 - -### MergeTreeAllRangesAnnouncementsSent {#mergetreeallrangesannouncementssent} - -当前从远程服务器发送到发起服务器、关于数据部件集合(适用于 MergeTree 表)的正在发送中的公告数量。该指标在远程服务器端进行统计。 - -### MergeTreeBackgroundExecutorThreads {#mergetreebackgroundexecutorthreads} - -MergeTreeBackgroundExecutor 线程池中的线程数量。 - -### MergeTreeBackgroundExecutorThreadsActive {#mergetreebackgroundexecutorthreadsactive} - -MergeTreeBackgroundExecutor 线程池中正在运行任务的线程数量。 - -### MergeTreeDataSelectExecutorThreads {#mergetreedataselectexecutorthreads} - -MergeTreeDataSelectExecutor 线程池中的线程数。 - -### MergeTreeDataSelectExecutorThreadsActive {#mergetreedataselectexecutorthreadsactive} - -MergeTreeDataSelectExecutor 线程池中正在执行任务的线程数。 - -### MergeTreePartsCleanerThreads {#mergetreepartscleanerthreads} - -MergeTree 数据部分清理线程池中的线程数量。 - -### MergeTreePartsCleanerThreadsActive {#mergetreepartscleanerthreadsactive} - -MergeTree 数据片段清理线程池中当前正在执行任务的线程数。 - -### MergeTreePartsLoaderThreads {#mergetreepartsloaderthreads} - -MergeTree 部件加载器线程池中的线程数。 - -### MergeTreePartsLoaderThreadsActive {#mergetreepartsloaderthreadsactive} - -在 MergeTree 数据部分加载器线程池中正在执行任务的线程数。 - -### MergeTreeReadTaskRequestsSent {#mergetreereadtaskrequestssent} - -当前从远程服务器发回到发起服务器、用于选择读取任务(针对 MergeTree 表)的回调请求的数量(即正在进行中的请求数)。该指标在远程服务器端进行统计。 - -### Move {#move} - -当前正在执行的移动操作数 - -### MySQLConnection {#mysqlconnection} - -使用 MySQL 协议的客户端连接数量 - -### NetworkReceive {#networkreceive} - -从网络接收数据的线程数。仅包含与 ClickHouse 相关的网络交互,不包括由第三方库发起的网络交互。 - -### NetworkSend {#networksend} - -向网络发送数据的线程数。只统计与 ClickHouse 相关的网络交互,不包括由第三方库发起的网络交互。 - -### OpenFileForRead {#openfileforread} - -用于读取的已打开文件数量 - -### OpenFileForWrite {#openfileforwrite} - -打开用于写入的文件数量 - -### ParallelFormattingOutputFormatThreads {#parallelformattingoutputformatthreads} - -ParallelFormattingOutputFormatThreads 线程池中的线程数。 - -### ParallelFormattingOutputFormatThreadsActive {#parallelformattingoutputformatthreadsactive} - -ParallelFormattingOutputFormatThreads 线程池中正在执行任务的线程数。 - -### PartMutation {#partmutation} - -变更次数(ALTER DELETE/UPDATE) - -### 活跃数据部分 {#partsactive} - -当前和后续 SELECT 查询会使用的活跃数据部分。 - -### PartsCommitted {#partscommitted} - -已弃用。请参阅 PartsActive。 - -### PartsCompact {#partscompact} - -紧凑部件。 - -### PartsDeleteOnDestroy {#partsdeleteondestroy} - -数据分片已被移动到另一个磁盘,并应在其析构函数中自行删除。 - -### PartsDeleting {#partsdeleting} - -具有标识 `refcounter` 的非活动数据分片,当前正由清理器删除。 - -### PartsOutdated {#partsoutdated} - -不是活跃的数据分片,但可能仍在被当前的 `SELECT` 查询使用,待这些 `SELECT` 查询结束后即可删除。 - -### PartsPreActive {#partspreactive} - -该 part 位于 `data_parts` 中,但不会被 `SELECT` 查询使用。 - -### PartsPreCommitted {#partsprecommitted} - -已弃用。请参阅 PartsPreActive。 - -### PartsTemporary {#partstemporary} - -该 part 正在生成中,尚未出现在 data_parts 列表中。 - -### PartsWide {#partswide} - -宽格式数据片段。 - -### PendingAsyncInsert {#pendingasyncinsert} - -正在等待刷新的异步插入数量。 - -### PostgreSQLConnection {#postgresqlconnection} - -客户端通过 PostgreSQL 协议建立的连接数 - -### 查询 {#query} - -正在执行的查询数量 - -### QueryPreempted {#querypreempted} - -因 `priority` 设置而被中止并进入等待状态的查询数量。 - -### QueryThread {#querythread} - -查询处理线程的数量 - -### RWLockActiveReaders {#rwlockactivereaders} - -在表的 RWLock 上持有读锁的线程数量。 - -### RWLockActiveWriters {#rwlockactivewriters} - -持有表上 RWLock 写锁的线程数量。 - -### RWLockWaitingReaders {#rwlockwaitingreaders} - -等待获取表级 RWLock 读锁的线程数量。 - -### RWLockWaitingWriters {#rwlockwaitingwriters} - -在表的 RWLock 上等待写入的线程数量。 - -### Read {#read} - -正在进行的 read(read、pread、io_getevents 等)系统调用数量 - -### ReadTaskRequestsSent {#readtaskrequestssent} - -从远程服务器返回到发起端服务器、用于选择读取任务的在途回调请求的当前数量(适用于 `s3Cluster` 表函数及类似场景)。在远程服务器端进行度量。 - -### ReadonlyReplica {#readonlyreplica} - -由于在 ZooKeeper 会话丢失后重新初始化或在未配置 ZooKeeper 的情况下启动而处于只读状态的 Replicated 表的当前数量。 - -### RemoteRead {#remoteread} - -进行中的远程读取操作数量 - -### ReplicatedChecks {#replicatedchecks} - -用于一致性检查的数据分片数量 - -### ReplicatedFetch {#replicatedfetch} - -正在从副本拉取的数据分片数量 - -### ReplicatedSend {#replicatedsend} - -当前正在发送到副本的数据分片数量 - -### RestartReplicaThreads {#restartreplicathreads} - -RESTART REPLICA 线程池中的线程数。 - -### RestartReplicaThreadsActive {#restartreplicathreadsactive} - -RESTART REPLICA 线程池中正在执行任务的线程数量。 - -### RestoreThreads {#restorethreads} - -用于 RESTORE 操作的线程池中的线程数量。 - -### RestoreThreadsActive {#restorethreadsactive} - -用于执行 RESTORE 任务的线程池中的线程数。 - -### 修订号 {#revision} - -服务器的修订号。该数字在每次正式发行版或候选发行版发布时递增,补丁发行版除外。 - -### S3Requests {#s3requests} - -S3 请求 - -### SendExternalTables {#sendexternaltables} - -向远程服务器发送外部表数据的连接数。外部表用于实现带有分布式子查询的 `GLOBAL IN` 和 `GLOBAL JOIN` 运算符。 - -### SendScalars {#sendscalars} - -正在向远程服务器发送标量数据的连接数。 - -### StorageBufferBytes {#storagebufferbytes} - -Buffer 表缓冲区中存储的字节数 - -### StorageBufferRows {#storagebufferrows} - -Buffer 表缓冲区中的行数 - -### StorageDistributedThreads {#storagedistributedthreads} - -StorageDistributed 线程池中的线程数量。 - -### StorageDistributedThreadsActive {#storagedistributedthreadsactive} - -正在执行任务的 StorageDistributed 线程池中的线程数量。 - -### StorageHiveThreads {#storagehivethreads} - -StorageHive 线程池中的线程数。 - -### StorageHiveThreadsActive {#storagehivethreadsactive} - -StorageHive 线程池中正在执行任务的线程数。 - -### StorageS3Threads {#storages3threads} - -StorageS3 线程池的线程数量。 - -### StorageS3ThreadsActive {#storages3threadsactive} - -StorageS3 线程池中当前正在执行任务的线程数量。 - -### SystemReplicasThreads {#systemreplicasthreads} - -`system.replicas` 线程池中的线程数。 - -### SystemReplicasThreadsActive {#systemreplicasthreadsactive} - -`system.replicas` 线程池中正在运行任务的线程数量。 - -### TCPConnection {#tcpconnection} - -到 TCP 服务器的连接数(使用原生接口的客户端),也包括服务器之间的分布式查询连接 - -### TablesToDropQueueSize {#tablestodropqueuesize} - -已删除但仍在等待后台删除其数据的表的数量。 - -### TemporaryFilesForAggregation {#temporaryfilesforaggregation} - -外部聚合过程中创建的临时文件数量 - -### TemporaryFilesForJoin {#temporaryfilesforjoin} - -在 JOIN 过程中创建的临时文件数量 - -### TemporaryFilesForSort {#temporaryfilesforsort} - -外部排序过程中创建的临时文件数量 - -### TemporaryFilesUnknown {#temporaryfilesunknown} - -用途不明的临时文件数量 - -### ThreadPoolFSReaderThreads {#threadpoolfsreaderthreads} - -线程池中用于 `local_filesystem_read_method=threadpool` 的线程数量。 - -### ThreadPoolFSReaderThreadsActive {#threadpoolfsreaderthreadsactive} - -`local_filesystem_read_method=threadpool` 的线程池中正在执行任务的线程数量。 - -### ThreadPoolRemoteFSReaderThreads {#threadpoolremotefsreaderthreads} - -`remote_filesystem_read_method=threadpool` 所使用的线程池中的线程数量。 - -### ThreadPoolRemoteFSReaderThreadsActive {#threadpoolremotefsreaderthreadsactive} - -在 `remote_filesystem_read_method=threadpool` 模式下线程池中正在执行任务的线程数。 - -### ThreadsInOvercommitTracker {#threadsinovercommittracker} - -在 OvercommitTracker 中等待的线程数 - -### TotalTemporaryFiles {#totaltemporaryfiles} - -已创建的临时文件数量 - -### VersionInteger {#versioninteger} - -以 1000 进制表示的服务器版本号,使用单个整数值。例如,版本 11.22.33 会被转换为 11022033。 - -### Write {#write} - -处于进行中的 write(write、pwrite、io_getevents 等)系统调用数量 - -### ZooKeeperRequest {#zookeeperrequest} - -当前正在处理的 ZooKeeper 请求数量。 - -### ZooKeeperSession {#zookeepersession} - -到 ZooKeeper 的会话(连接)数量。不应超过一个,因为使用多个到 ZooKeeper 的连接可能会由于缺乏线性一致性(可能出现陈旧读),这是 ZooKeeper 一致性模型所允许的,而导致错误。 - -### ZooKeeperWatch {#zookeeperwatch} - -ZooKeeper 中的 watch(事件订阅)总数。 - -### ConcurrencyControlAcquired {#concurrencycontrolacquired} - -已获取的 CPU 槽位总数。 - -### ConcurrencyControlSoftLimit {#concurrencycontrolsoftlimit} - -CPU 插槽数量的软限制值。 - -**另请参阅** - -- [system.asynchronous_metrics](/operations/system-tables/asynchronous_metrics) — 包含周期性计算的指标。 -- [system.events](/operations/system-tables/events) — 包含已发生的各类事件。 -- [system.metric_log](/operations/system-tables/metric_log) — 包含来自 `system.metrics` 和 `system.events` 表的指标值历史记录。 -- [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md deleted file mode 100644 index 7f37908388f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/moves.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -description: '包含有关 MergeTree 表中正在进行的数据分片迁移的信息的 system 表。每个数据分片的移动对应一行记录。' -keywords: ['system 表', '移动'] -slug: /operations/system-tables/moves -title: 'system.moves' -doc_type: 'reference' ---- - -# system.moves {#systemmoves} - -该表包含 [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 表中正在进行的 [数据分片移动](/sql-reference/statements/alter/partition#move-partitionpart) 的信息。每个数据分片移动对应一行。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.moves -``` - -```response -┌─database─┬─table─┬─────elapsed─┬─target_disk_name─┬─target_disk_path─┬─part_name─┬─part_size─┬─thread_id─┐ -│ default │ test2 │ 1.668056039 │ s3 │ ./disks/s3/ │ all_3_3_0 │ 136 │ 296146 │ -└──────────┴───────┴─────────────┴──────────────────┴──────────────────┴───────────┴───────────┴───────────┘ -``` - -**另请参阅** - -* [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 表引擎 -* [在数据存储中使用多个块设备](/engines/table-engines/mergetree-family/mergetree#table_engine-mergetree-multiple-volumes) -* [ALTER TABLE ... MOVE PART](/sql-reference/statements/alter/partition#move-partitionpart) 命令 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md deleted file mode 100644 index c114a4abe59..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/mutations.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: '包含 MergeTree 表变更(mutation)及其进度信息的系统表。每条变更命令对应一行。' -keywords: ['system table', 'mutations'] -slug: /operations/system-tables/mutations -title: 'system.mutations' -doc_type: 'reference' ---- - -# system.mutations {#systemmutations} - -该表包含 [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 表的[变更](/sql-reference/statements/alter/index.md#mutations)及其执行进度的信息。每条变更命令在该表中对应一行记录。 - -## 列 {#columns} - -- `database` ([String](/sql-reference/data-types/string.md)) — 应用变更操作的数据库名称。 -- `table` ([String](/sql-reference/data-types/string.md)) — 应用变更操作的表名称。 -- `mutation_id` ([String](/sql-reference/data-types/string.md)) — 变更操作的 ID。对于复制表,这些 ID 对应于 ClickHouse Keeper 中 `/mutations/` 目录下的 znode 名称。对于非复制表,这些 ID 对应于表数据目录中的文件名。 -- `command` ([String](/sql-reference/data-types/string.md)) — 变更命令字符串(`ALTER TABLE [db.]table` 之后的查询部分)。 -- `create_time` ([DateTime](/sql-reference/data-types/datetime.md)) — 提交变更命令执行的日期和时间。 -- `block_numbers.partition_id` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — 对于复制表的变更操作,该数组包含分区的 ID(每个分区一条记录)。对于非复制表的变更操作,该数组为空。 -- `block_numbers.number` ([Array](/sql-reference/data-types/array.md)([Int64](/sql-reference/data-types/int-uint.md))) — 对于复制表的变更操作,该数组为每个分区包含一条记录,其中包含变更操作获取的块编号。分区中只有包含编号小于此编号的块的数据部分才会被变更。在非复制表中,所有分区的块编号形成单一序列。这意味着对于非复制表的变更操作,该列将包含一条记录,其中包含变更操作获取的单个块编号。 -- `parts_to_do_names` ([Array](/sql-reference/data-types/array.md)([String](/sql-reference/data-types/string.md))) — 需要变更才能完成变更操作的数据部分名称数组。 -- `parts_to_do` ([Int64](/sql-reference/data-types/int-uint.md)) — 需要变更才能完成变更操作的数据部分数量。 -- `is_killed` ([UInt8](/sql-reference/data-types/int-uint.md)) — 指示变更操作是否已被终止。**仅在 ClickHouse Cloud 中可用。** - -:::note -`is_killed=1` 并不一定意味着变更操作已完全结束。变更操作可能会在较长时间内保持 `is_killed=1` 和 `is_done=0` 的状态。如果另一个长时间运行的变更操作阻塞了已终止的变更操作,就会发生这种情况。这是正常情况。 -::: - -- `is_done` ([UInt8](/sql-reference/data-types/int-uint.md)) — 指示变更操作是否完成的标志。可能的值: - - `1` 表示变更操作已完成, - - `0` 表示变更操作仍在进行中。 - -:::note -即使 `parts_to_do = 0`,复制表的变更操作也可能尚未完成,因为长时间运行的 `INSERT` 查询会创建需要变更的新数据部分。 -::: - -如果在变更某些数据部分时出现问题,以下列包含附加信息: - -- `latest_failed_part` ([String](/sql-reference/data-types/string.md)) — 无法变更的最新数据部分的名称。 -- `latest_fail_time` ([DateTime](/sql-reference/data-types/datetime.md)) — 最近一次数据部分变更失败的日期和时间。 -- `latest_fail_reason` ([String](/sql-reference/data-types/string.md)) — 导致最近一次数据部分变更失败的异常消息。 - -## 监控 Mutation {#monitoring-mutations} - -要跟踪 `system.mutations` 表中的进度,请使用以下查询: - -```sql -SELECT * FROM clusterAllReplicas('cluster_name', 'system', 'mutations') -WHERE is_done = 0 AND table = 'tmp'; - --- 或 - -SELECT * FROM clusterAllReplicas('cluster_name', 'system.mutations') -WHERE is_done = 0 AND table = 'tmp'; -``` - -注意:这需要对 `system.*` 表具有读取权限。 - -:::tip Cloud 使用说明 -在 ClickHouse Cloud 中,每个节点上的 `system.mutations` 表都包含集群中的所有 mutation,因此无需使用 `clusterAllReplicas`。 -::: - -**另请参阅** - -- [Mutation](/sql-reference/statements/alter/index.md#mutations) -- [MergeTree](/engines/table-engines/mergetree-family/mergetree.md) 表引擎 -- [ReplicatedMergeTree](/engines/table-engines/mergetree-family/replication.md) 系列 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md deleted file mode 100644 index 4fae6981f27..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: '系统表,包含一个名为 `number` 的 UInt64 类型单列表,用于存储从零开始的几乎所有自然数。' -keywords: ['系统表', 'numbers'] -slug: /operations/system-tables/numbers -title: 'system.numbers' -doc_type: 'reference' ---- - -# system.numbers {#systemnumbers} - -该表包含一个名为 `number` 的 `UInt64` 列,存储从零开始的几乎所有自然数。 - -你可以将此表用于测试,或在需要进行暴力搜索时使用。 - -从该表读取数据时不支持并行化。 - -**示例** - -```sql -SELECT * FROM system.numbers LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -返回 10 行。耗时: 0.001 秒。 -``` - -你还可以使用谓词来限制输出结果。 - -```sql -SELECT * FROM system.numbers < 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -返回 10 行。耗时: 0.001 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md deleted file mode 100644 index b7a847cf5df..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/numbers_mt.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: '与 `system.numbers` 类似的系统表,但读取操作是并行的,且返回的数字顺序不固定。' -keywords: ['system table', 'numbers_mt'] -slug: /operations/system-tables/numbers_mt -title: 'system.numbers_mt' -doc_type: 'reference' ---- - -与 [`system.numbers`](../../operations/system-tables/numbers.md) 相同,但读取操作是并行的。返回的数字可以是任意顺序。 - -用于测试。 - -**示例** - -```sql -SELECT * FROM system.numbers_mt LIMIT 10; -``` - -```response -┌─number─┐ -│ 0 │ -│ 1 │ -│ 2 │ -│ 3 │ -│ 4 │ -│ 5 │ -│ 6 │ -│ 7 │ -│ 8 │ -│ 9 │ -└────────┘ - -返回 10 行。耗时: 0.001 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/one.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/one.md deleted file mode 100644 index d2628420c88..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/one.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: '系统表,包含一行且仅有一个名为 `dummy` 的 UInt8 列,列值为 0。类似于其他 DBMS 中的 `DUAL` 表。' -keywords: ['系统表', 'one'] -slug: /operations/system-tables/one -title: 'system.one' -doc_type: 'reference' ---- - -# system.one {#systemone} - -该表包含一行数据,只有一个名为 `dummy` 的 UInt8 列,列中的值为 0。 - -当 `SELECT` 查询中未指定 `FROM` 子句时,会使用该表。 - -这类似于其他 DBMS 中的 `DUAL` 表。 - -**示例** - -```sql -SELECT * FROM system.one LIMIT 10; -``` - -```response -┌─dummy─┐ -│ 0 │ -└───────┘ - -返回 1 行。用时:0.001 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md deleted file mode 100644 index 5366a1e9486..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/opentelemetry_span_log.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '包含已执行查询的跟踪 span 信息的系统表。' -keywords: ['系统表', 'opentelemetry_span_log'] -slug: /operations/system-tables/opentelemetry_span_log -title: 'system.opentelemetry_span_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.opentelemetry_span_log {#systemopentelemetry_span_log} - - - -包含已执行查询的 [trace span](https://opentracing.io/docs/overview/spans/) 相关信息。 - -列: - -* `trace_id` ([UUID](../../sql-reference/data-types/uuid.md)) — 已执行查询所属 trace 的 ID。 -* `span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` 的 ID。 -* `parent_span_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 父 `trace span` 的 ID。 -* `operation_name` ([String](../../sql-reference/data-types/string.md)) — 操作名称。 -* `kind` ([Enum8](../../sql-reference/data-types/enum.md)) — span 的 [SpanKind](https://opentelemetry.io/docs/reference/specification/trace/api/#spankind)。 - * `INTERNAL` — 表示该 span 代表应用程序内部操作。 - * `SERVER` — 表示该 span 覆盖同步 RPC 或其他远程请求的服务端处理。 - * `CLIENT` — 表示该 span 描述发往某个远程服务的请求。 - * `PRODUCER` — 表示该 span 描述异步请求的发起方。该父 span 通常会在对应的子 CONSUMER span 之前结束,甚至可能早于子 span 开始。 - * `CONSUMER` — 表示该 span 描述异步 PRODUCER 请求的子 span。 -* `start_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` 的开始时间(微秒)。 -* `finish_time_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `trace span` 的结束时间(微秒)。 -* `finish_date` ([Date](../../sql-reference/data-types/date.md)) — `trace span` 的结束日期。 -* `attribute.names` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 与 `trace span` 对应的[属性(Attribute)](https://opentelemetry.io/docs/go/instrumentation/#attributes)名称。根据 [OpenTelemetry](https://opentelemetry.io/) 标准中的建议进行填充。 -* `attribute.values` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 与 `trace span` 对应的属性值。根据 `OpenTelemetry` 标准中的建议进行填充。 - -**示例** - -查询: - -```sql -SELECT * FROM system.opentelemetry_span_log LIMIT 1 FORMAT Vertical; -``` - -结果: - -```text -第 1 行: -────── -trace_id: cdab0847-0d62-61d5-4d38-dd65b19a1914 -span_id: 701487461015578150 -parent_span_id: 2991972114672045096 -operation_name: DB::Block DB::InterpreterSelectQuery::getSampleBlockImpl() -kind: INTERNAL -start_time_us: 1612374594529090 -finish_time_us: 1612374594529108 -finish_date: 2021-02-03 -attribute.names: [] -attribute.values: [] -``` - -**另请参阅** - -* [OpenTelemetry](../../operations/opentelemetry.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md deleted file mode 100644 index 1d57dd437e4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/overview.md +++ /dev/null @@ -1,263 +0,0 @@ ---- -description: '系统表的概念简介及其用途。' -keywords: ['系统表', '概述'] -sidebar_label: '概述' -sidebar_position: 52 -slug: /operations/system-tables/overview -title: '系统表概述' -doc_type: 'reference' ---- - - - -## 系统表概览 {#system-tables-introduction} - -系统表提供以下信息: - -* 服务器状态、进程和环境。 -* 服务器的内部进程。 -* 构建 ClickHouse 二进制文件时使用的选项。 - -系统表: - -* 位于 `system` 数据库中。 -* 仅可用于读取数据。 -* 无法被 DROP 或 ALTER,但可以被 DETACH。 - -大多数系统表将数据存储在内存(RAM)中。ClickHouse 服务器在启动时会创建这些系统表。 - -与其他系统表不同,系统日志表 [metric_log](../../operations/system-tables/metric_log.md)、[query_log](../../operations/system-tables/query_log.md)、[query_thread_log](../../operations/system-tables/query_thread_log.md)、[trace_log](../../operations/system-tables/trace_log.md)、[part_log](../../operations/system-tables/part_log.md)、[crash_log](../../operations/system-tables/crash_log.md)、[text_log](../../operations/system-tables/text_log.md) 和 [backup_log](../../operations/system-tables/backup_log.md) 由 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表引擎驱动,并默认将数据存储在文件系统中。如果从文件系统中删除了一张表,ClickHouse 服务器会在下一次写入数据时重新创建一个空表。如果在新版本中系统表的表结构发生变化,ClickHouse 会重命名当前表并创建一个新表。 - -可以通过在 `/etc/clickhouse-server/config.d/` 下创建与表同名的配置文件,或者在 `/etc/clickhouse-server/config.xml` 中设置相应的元素,来自定义系统日志表。可自定义的元素包括: - -* `database`:系统日志表所属的数据库。该选项目前已废弃。所有系统日志表都位于 `system` 数据库下。 -* `table`:用于插入数据的表。 -* `partition_by`:指定 [PARTITION BY](../../engines/table-engines/mergetree-family/custom-partitioning-key.md) 表达式。 -* `ttl`:指定表的 [TTL](../../sql-reference/statements/alter/ttl.md) 表达式。 -* `flush_interval_milliseconds`:将数据刷新到磁盘的时间间隔。 -* `engine`:提供带参数的完整引擎表达式(以 `ENGINE =` 开头)。该选项与 `partition_by` 和 `ttl` 冲突。如果同时设置,服务器会抛出异常并退出。 - -示例: - -```xml - - - system - query_log
- toYYYYMM(event_date) - event_date + INTERVAL 30 DAY DELETE - - 7500 - 1048576 - 8192 - 524288 - false -
-
-``` - -默认情况下,表的大小不设上限。要控制表的大小,可以使用 [TTL](/sql-reference/statements/alter/ttl) 设置来删除过期的日志记录,还可以使用 `MergeTree` 引擎表的分区功能。 - - -## 系统指标的来源 {#system-tables-sources-of-system-metrics} - -为了收集系统指标,ClickHouse 服务器会使用: - -- `CAP_NET_ADMIN` 能力。 -- [procfs](https://en.wikipedia.org/wiki/Procfs)(仅限 Linux)。 - -**procfs** - -如果 ClickHouse 服务器不具备 `CAP_NET_ADMIN` 能力,它会尝试回退到使用 `ProcfsMetricsProvider`。`ProcfsMetricsProvider` 允许按查询粒度收集系统指标(用于 CPU 和 I/O)。 - -如果系统支持并启用了 procfs,ClickHouse 服务器会收集以下指标: - -- `OSCPUVirtualTimeMicroseconds` -- `OSCPUWaitMicroseconds` -- `OSIOWaitMicroseconds` -- `OSReadChars` -- `OSWriteChars` -- `OSReadBytes` -- `OSWriteBytes` - -:::note -在 Linux 内核 5.14.x 及之后的版本中,`OSIOWaitMicroseconds` 默认被禁用。 -可以通过执行 `sudo sysctl kernel.task_delayacct=1` 来启用它,或者在 `/etc/sysctl.d/` 中创建一个 `.conf` 文件,并写入 `kernel.task_delayacct = 1`。 -::: - - - -## ClickHouse Cloud 中的 system 表 {#system-tables-in-clickhouse-cloud} - -在 ClickHouse Cloud 中,system 表与在自托管部署中一样,为服务的状态和性能提供关键洞察。一些 system 表在整个集群范围内生效,尤其是那些从 Keeper 节点获取数据的表,这些节点负责管理分布式元数据。这些表反映了整个集群的状态,并且在各个节点上查询时其结果应当保持一致。例如,[`parts`](/operations/system-tables/parts) 在从任意节点查询时都应该是一致的: - -```sql -SELECT hostname(), count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-vccsrty-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.005 sec. - -SELECT - hostname(), - count() -FROM system.parts -WHERE `table` = 'pypi' - -┌─hostname()────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-w59bfco-0 │ 26 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.004 sec. -``` - -相反,其他一些 system 表是特定于节点的,例如仅存储在内存中,或使用 MergeTree 表引擎持久化其数据。这类用法通常适用于日志和指标等数据。这种持久化确保历史数据始终可用于分析。然而,这些特定于节点的表在每个节点上都是彼此独立的。 - -一般来说,在判断一个 system 表是否为特定于节点时,可以应用以下规则: - -* 带有 `_log` 后缀的 system 表。 -* 提供指标的 system 表,例如 `metrics`、`asynchronous_metrics`、`events`。 -* 提供正在运行的进程信息的 system 表,例如 `processes`、`merges`。 - -此外,由于升级或其 schema 的更改,可能会创建 system 表的新版本。这些版本通过数字后缀进行命名。 - -例如,以 `system.query_log` 表为例,它包含节点上执行的每个查询的一行记录: - -```sql -SHOW TABLES FROM system LIKE 'query_log%' - -┌─name─────────┐ -│ query_log │ -│ query_log_1 │ -│ query_log_10 │ -│ query_log_2 │ -│ query_log_3 │ -│ query_log_4 │ -│ query_log_5 │ -│ query_log_6 │ -│ query_log_7 │ -│ query_log_8 │ -│ query_log_9 │ -└──────────────┘ - -11 行结果,耗时 0.004 秒。 -``` - -### 跨多个版本查询 {#querying-multiple-versions} - -我们可以使用 [`merge`](/sql-reference/table-functions/merge) 函数对这些表进行跨表查询。例如,下面的查询会在每个 `query_log` 表中找出发送到目标节点的最新一次查询: - -```sql -SELECT - _table, - max(event_time) AS most_recent -FROM merge('system', '^query_log') -GROUP BY _table -ORDER BY most_recent DESC - -┌─_table───────┬─────────most_recent─┐ -│ query_log │ 2025-04-13 10:59:29 │ -│ query_log_1 │ 2025-04-09 12:34:46 │ -│ query_log_2 │ 2025-04-09 12:33:45 │ -│ query_log_3 │ 2025-04-07 17:10:34 │ -│ query_log_5 │ 2025-03-24 09:39:39 │ -│ query_log_4 │ 2025-03-24 09:38:58 │ -│ query_log_6 │ 2025-03-19 16:07:41 │ -│ query_log_7 │ 2025-03-18 17:01:07 │ -│ query_log_8 │ 2025-03-18 14:36:07 │ -│ query_log_10 │ 2025-03-18 14:01:33 │ -│ query_log_9 │ 2025-03-18 14:01:32 │ -└──────────────┴─────────────────────┘ -``` - - -11 行数据。耗时:0.373 秒。已处理 644 万行,25.77 MB(每秒 1,729 万行,69.17 MB/s)。 -峰值内存使用:28.45 MiB。 - -```` - -:::note 不要依赖数字后缀来确定顺序 -虽然表上的数字后缀可以暗示数据的顺序,但绝不应依赖它。因此,在查询特定日期范围时,应始终使用 merge 表函数结合日期过滤器。 -::: - -重要的是,这些表仍然是**每个节点的本地表**。 - -### 跨节点查询 {#querying-across-nodes} - -要全面查看整个集群,用户可以结合使用 [`clusterAllReplicas`](/sql-reference/table-functions/cluster) 函数和 `merge` 函数。`clusterAllReplicas` 函数允许在"default"集群内的所有副本上查询系统表,将各节点的数据整合为统一的结果。与 `merge` 函数结合使用时,可以查询集群中特定表的所有系统数据。 - -这种方法对于监控和调试集群范围的操作特别有价值,确保用户能够有效分析其 ClickHouse Cloud 部署的健康状况和性能。 - -:::note -ClickHouse Cloud 提供多副本集群以实现冗余和故障转移。这使其能够实现动态自动扩展和零停机升级等功能。在某个特定时刻,新节点可能正在添加到集群或从集群中移除。要跳过这些节点,请在使用 `clusterAllReplicas` 的查询中添加 `SETTINGS skip_unavailable_shards = 1`,如下所示。 -::: - -例如,考虑查询 `query_log` 表时的差异——该表通常对分析至关重要。 - -```sql -SELECT - hostname() AS host, - count() -FROM system.query_log -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -└───────────────────────────────┴─────────┘ - -1 row in set. Elapsed: 0.010 sec. Processed 17.87 thousand rows, 71.51 KB (1.75 million rows/s., 7.01 MB/s.) - -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', system.query_log) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 650543 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 656029 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 641155 │ -└───────────────────────────────┴─────────┘ - -3 rows in set. Elapsed: 0.026 sec. Processed 1.97 million rows, 7.88 MB (75.51 million rows/s., 302.05 MB/s.) -```` - -### 跨节点和版本查询 {#querying-across-nodes-and-versions} - -由于系统表存在版本控制,这仍然无法反映集群中的完整数据。将上述方法与 `merge` 函数结合使用后,我们就能在指定日期范围内获得精确结果: - -```sql -SELECT - hostname() AS host, - count() -FROM clusterAllReplicas('default', merge('system', '^query_log')) -WHERE (event_time >= '2025-04-01 00:00:00') AND (event_time <= '2025-04-12 00:00:00') -GROUP BY host SETTINGS skip_unavailable_shards = 1 - -┌─host──────────────────────────┬─count()─┐ -│ c-ecru-qn-34-server-s5bnysl-0 │ 3008000 │ -│ c-ecru-qn-34-server-6em4y4t-0 │ 3659443 │ -│ c-ecru-qn-34-server-iejrkg0-0 │ 1078287 │ -└───────────────────────────────┴─────────┘ -``` - - -3 行数据。耗时:0.462 秒。已处理 7.94 百万行,31.75 MB(17.17 百万行/秒,68.67 MB/秒)。 - -``` -``` - - -## 相关内容 {#related-content} - -- 博客:[系统表:窥探 ClickHouse 内部机制的窗口](https://clickhouse.com/blog/clickhouse-debugging-issues-with-system-tables) -- 博客:[核心监控查询 - 第 1 部分 - INSERT 查询](https://clickhouse.com/blog/monitoring-troubleshooting-insert-queries-clickhouse) -- 博客:[核心监控查询 - 第 2 部分 - SELECT 查询](https://clickhouse.com/blog/monitoring-troubleshooting-select-queries-clickhouse) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md deleted file mode 100644 index 66cf380d085..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/part_log.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: '记录 MergeTree 系列表中数据分片相关事件(例如分片的添加或合并)的系统表。' -keywords: ['system table', 'part_log'] -slug: /operations/system-tables/part_log -title: 'system.part_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.part_log {#systempart_log} - - - -只有在指定了 [part_log](/operations/server-configuration-parameters/settings#part_log) 服务器设置时,才会创建 `system.part_log` 表。 - -此表包含关于 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 系列表中[数据分片](../../engines/table-engines/mergetree-family/custom-partitioning-key.md)发生的事件的信息,例如数据的添加或合并。 - -`system.part_log` 表包含以下列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行该查询的服务器的主机名。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 用于创建此数据分片的 `INSERT` 查询的标识符。 -* `event_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 在数据部分上发生的事件类型。可以是以下值之一: - * `NewPart` — 插入新的数据分片。 - * `MergePartsStart` — 数据分片合并已开始。 - * `MergeParts` — 数据分片合并已完成。 - * `DownloadPart` — 下载数据分片。 - * `RemovePart` — 使用 [DETACH PARTITION](/sql-reference/statements/alter/partition#detach-partitionpart) 移除或分离数据分片。 - * `MutatePartStart` — 数据分片变更操作已开始。 - * `MutatePart` — 数据分片变更操作已完成。 - * `MovePart` — 将数据分片从一个磁盘移动到另一个磁盘。 -* `merge_reason`([Enum8](../../sql-reference/data-types/enum.md))— 类型为 `MERGE_PARTS` 的事件的原因。可以具有以下值之一: - * `NotAMerge` — 当前事件的类型不是 `MERGE_PARTS`。 - * `RegularMerge` — 一次常规合并。 - * `TTLDeleteMerge` — 清理过期数据。 - * `TTLRecompressMerge` — 对数据分片进行重新压缩。 -* `merge_algorithm` ([Enum8](../../sql-reference/data-types/enum.md)) — 类型为 `MERGE_PARTS` 的事件所使用的合并算法。可以取以下值之一: - * `未决定` - * `水平` - * `垂直` -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒级精度的事件时间。 -* `duration_ms` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 持续时间(毫秒)。 -* `database` ([String](../../sql-reference/data-types/string.md)) — 数据部分所在的数据库名称。 -* `table` ([String](../../sql-reference/data-types/string.md)) — 该数据分片所在表的名称。 -* `table_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — 该数据部分所属表的 UUID。 -* `part_name` ([String](../../sql-reference/data-types/string.md)) — 数据部分的名称。 -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — 数据部件写入到的分区 ID。如果按 `tuple()` 分区,则该列取值为 `all`。 -* `partition` ([String](../../sql-reference/data-types/string.md)) - 分区名称。 -* `part_type` ([String](../../sql-reference/data-types/string.md)) - part 的类型。可能的取值有:Wide 和 Compact。 -* `disk_name` ([String](../../sql-reference/data-types/string.md)) - 数据部件所在的磁盘名称。 -* `path_on_disk` ([String](../../sql-reference/data-types/string.md)) — 包含数据部件文件的目录的绝对路径。 -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 数据部分中的行数。 -* `size_in_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 数据部分的大小(以字节为单位)。 -* `merged_from` ([Array(String)](../../sql-reference/data-types/array.md)) — 当前数据片(合并后)由其来源数据片名称组成的数组。 -* `bytes_uncompressed` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 未压缩数据的字节数。 -* `read_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 在合并过程中读取的行数。 -* `read_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 合并过程中读取的字节数。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — 在此线程上下文中,已分配内存与已释放内存之间的最大差值。 -* `error` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 发生错误的代码号。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 发生错误时的文本信息。 -* `ProfileEvents`([Map(String, UInt64)](../../sql-reference/data-types/map.md))— 用于统计不同指标的 ProfileEvents。相关说明可在表 [system.events](/operations/system-tables/events) 中找到。 - -在首次向 `MergeTree` 表插入数据后,会创建 `system.part_log` 表。 - -**示例** - -```sql -SELECT * FROM system.part_log LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -query_id: -event_type: MergeParts -merge_reason: RegularMerge -merge_algorithm: Vertical -event_date: 2025-07-19 -event_time: 2025-07-19 23:54:19 -event_time_microseconds: 2025-07-19 23:54:19.710761 -duration_ms: 2158 -database: default -table: github_events -table_uuid: 1ad33424-f5f5-402b-ac03-ec82282634ab -part_name: all_1_7_1 -partition_id: all -partition: tuple() -part_type: Wide -disk_name: default -path_on_disk: ./data/store/1ad/1ad33424-f5f5-402b-ac03-ec82282634ab/all_1_7_1/ -rows: 3285726 -- 329 万 -size_in_bytes: 438968542 -- 4.39 亿 -merged_from: ['all_1_1_0','all_2_2_0','all_3_3_0','all_4_4_0','all_5_5_0','all_6_6_0','all_7_7_0'] -bytes_uncompressed: 1373137767 -- 13.7 亿 -read_rows: 3285726 -- 329 万 -read_bytes: 1429206946 -- 14.3 亿 -peak_memory_usage: 303611887 -- 3.04 亿 -error: 0 -exception: -ProfileEvents: {'FileOpen':703,'ReadBufferFromFileDescriptorRead':3824,'ReadBufferFromFileDescriptorReadBytes':439601681,'WriteBufferFromFileDescriptorWrite':592,'WriteBufferFromFileDescriptorWriteBytes':438988500,'ReadCompressedBytes':439601681,'CompressedReadBufferBlocks':6314,'CompressedReadBufferBytes':1539835748,'OpenedFileCacheHits':50,'OpenedFileCacheMisses':484,'OpenedFileCacheMicroseconds':222,'IOBufferAllocs':1914,'IOBufferAllocBytes':319810140,'ArenaAllocChunks':8,'ArenaAllocBytes':131072,'MarkCacheMisses':7,'CreatedReadBufferOrdinary':534,'DiskReadElapsedMicroseconds':139058,'DiskWriteElapsedMicroseconds':51639,'AnalyzePatchRangesMicroseconds':28,'ExternalProcessingFilesTotal':1,'RowsReadByMainReader':170857759,'WaitMarksLoadMicroseconds':988,'LoadedMarksFiles':7,'LoadedMarksCount':14,'LoadedMarksMemoryBytes':728,'Merge':2,'MergeSourceParts':14,'MergedRows':3285733,'MergedColumns':4,'GatheredColumns':51,'MergedUncompressedBytes':1429207058,'MergeTotalMilliseconds':2158,'MergeExecuteMilliseconds':2155,'MergeHorizontalStageTotalMilliseconds':145,'MergeHorizontalStageExecuteMilliseconds':145,'MergeVerticalStageTotalMilliseconds':2008,'MergeVerticalStageExecuteMilliseconds':2006,'MergeProjectionStageTotalMilliseconds':5,'MergeProjectionStageExecuteMilliseconds':4,'MergingSortedMilliseconds':7,'GatheringColumnMilliseconds':56,'ContextLock':2091,'PartsLockHoldMicroseconds':77,'PartsLockWaitMicroseconds':1,'RealTimeMicroseconds':2157475,'CannotWriteToWriteBufferDiscard':36,'LogTrace':6,'LogDebug':59,'LoggerElapsedNanoseconds':514040,'ConcurrencyControlSlotsGranted':53,'ConcurrencyControlSlotsAcquired':53} -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md deleted file mode 100644 index cec32be39b0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -description: '包含 MergeTree 数据分片信息的 system 表' -keywords: ['system 表', '数据分片'] -slug: /operations/system-tables/parts -title: 'system.parts' -doc_type: 'reference' ---- - -# system.parts {#systemparts} - -包含关于 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表数据分片(parts)的信息。 - -每一行描述一个数据分片。 - -列: - -* `partition` ([String](../../sql-reference/data-types/string.md)) – 分区名称。要了解什么是分区,请参阅 [ALTER](/sql-reference/statements/alter) 查询的说明。 - - 格式: - - * `YYYYMM`:按月自动分区。 - * `any_string`:手动分区时使用任意字符串。 - -* `name` ([String](../../sql-reference/data-types/string.md)) – 数据分片名称。分片命名结构可用于判断数据、摄取以及合并模式等多个方面。分片命名格式如下: - -```text -<分区ID>_<最小块号>_<最大块号>_<层级>_<数据版本> -``` - -* 定义: - * `partition_id` - 标识分区 ID - * `minimum_block_number` - 标识该 part 中的最小块号。ClickHouse 始终合并连续的块 - * `maximum_block_number` - 标识该 part 中的最大块号 - * `level` - 每对该 part 进行一次合并,该值加一。level 为 0 表示这是一个尚未被合并的新 part。需要牢记的是,ClickHouse 中的所有 part 始终都是不可变的 - * `data_version` - 可选值,在对 part 执行 mutate 操作时递增(同样,变更后的数据始终只会写入新的 part,因为 part 是不可变的) - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) - 数据部件的 UUID。 - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — 数据部分的存储格式。 - - 可选值: - - * `Wide` — 每一列都作为单独的文件存储在文件系统中。 - * `Compact` — 所有列都存储在文件系统中的同一个文件中。 - - 数据存储格式由 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表中的 `min_bytes_for_wide_part` 和 `min_rows_for_wide_part` 设置项控制。 - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) – 指示数据部分是否处于活动状态的标志位。处于活动状态的数据部分会在表中被使用,否则会被删除。合并后,非活动的数据部分仍会保留。 - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 标记数量。要获得一个数据部分中大致的行数,将 `marks` 乘以索引粒度(通常为 8192)(此提示不适用于自适应粒度)。 - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 行数。 - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 所有数据部分文件的总大小(字节)。 - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 数据分片中压缩数据的总大小。不包含任何辅助文件(例如标记文件)。 - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 数据片段中未压缩数据的总大小。不包括所有辅助文件(例如标记文件)。 - -* `primary_key_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 磁盘上 primary.idx/cidx 文件中主键值所占用的字节数。 - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 标记文件的大小(字节)。 - -* `secondary_indices_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 数据部分中二级索引压缩数据的总大小。不包括任何辅助文件(例如标记文件)。 - -* `secondary_indices_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 数据部分中二级索引未压缩数据的总大小。不包括任何辅助文件(例如标记文件)。 - -* `secondary_indices_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 二级索引标记文件的大小(字节数)。 - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – 数据部分所在目录的修改时间。该时间通常对应于数据部分的创建时间。 - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – 数据片段变为非活跃状态的时间。 - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) – 该数据分片被使用的引用次数。大于 2 的值表示该数据分片正被查询或合并操作使用。 - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) – 数据片段中日期键的最小值。 - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) – 数据部分中日期键的最大值。 - -* `min_time` ([DateTime](../../sql-reference/data-types/datetime.md)) – 数据部分中日期时间键的最小值。 - -* `max_time`([DateTime](../../sql-reference/data-types/datetime.md)) – 数据部分中日期时间键的最大值。 - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) – 分区 ID。 - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 合并后构成当前部分的数据块的最小编号。 - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 合并后构成当前数据片段的最大数据块编号。 - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) – 合并树的层级深度。零表示当前数据部件是通过插入创建的,而不是由其他部件合并得到的。 - -* `data_version` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 用于确定应对该数据部分应用哪些变更(会应用版本号高于当前 `data_version` 的变更)的数字。 - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 内存中主键值占用的字节数(当 `primary_key_lazy_load=1` 且 `use_primary_key_cache=1` 时,该值为 `0`)。 - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) – 为主键值预留的内存大小(以字节为单位,当 `primary_key_lazy_load=1` 且 `use_primary_key_cache=1` 时,该值为 `0`)。 - -* `is_frozen` ([UInt8](../../sql-reference/data-types/int-uint.md)) – 标志位,表示是否存在该分区的数据备份。1 表示备份存在,0 表示备份不存在。更多详情请参见 [FREEZE PARTITION](/sql-reference/statements/alter/partition#freeze-partition) - -* `database` ([String](../../sql-reference/data-types/string.md)) – 数据库名称。 - -* `table` ([String](../../sql-reference/data-types/string.md)) – 表名。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) – 表引擎的名称,不含参数。 - -* `path` ([String](../../sql-reference/data-types/string.md)) – 包含数据部分文件的文件夹的绝对路径。 - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) – 用于存储该数据部分的磁盘名称。 - -* `hash_of_all_files` ([String](../../sql-reference/data-types/string.md)) – 压缩文件的 [sipHash128](/sql-reference/functions/hash-functions#sipHash128) 哈希值。 - -* `hash_of_uncompressed_files`([String](../../sql-reference/data-types/string.md))– 未压缩文件(标记文件、索引文件等)的 [sipHash128](/sql-reference/functions/hash-functions#sipHash128) 哈希值。 - -* `uncompressed_hash_of_compressed_files` ([String](../../sql-reference/data-types/string.md)) – 将压缩文件中的数据视为未压缩时所计算的 [sipHash128](/sql-reference/functions/hash-functions#sipHash128) 哈希值。 - -* `delete_ttl_info_min` ([DateTime](../../sql-reference/data-types/datetime.md)) — [TTL DELETE 规则](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) 中日期时间键的最小值。 - -* `delete_ttl_info_max` ([DateTime](../../sql-reference/data-types/datetime.md)) — [TTL DELETE 规则](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)的日期时间键的最大值。 - -* `move_ttl_info.expression` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 由表达式组成的数组。每个表达式都定义一条 [TTL MOVE 规则](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)。 - -:::note -`move_ttl_info.expression` 数组主要是为了向后兼容而保留,现在检查 `TTL MOVE` 规则最简单的方式是使用 `move_ttl_info.min` 和 `move_ttl_info.max` 字段。 -::: - -* `move_ttl_info.min` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — 日期和时间值的数组。每个元素表示 [TTL MOVE 规则](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)的最小键值。 - -* `move_ttl_info.max` ([Array](../../sql-reference/data-types/array.md)([DateTime](../../sql-reference/data-types/datetime.md))) — 日期和时间值的数组。每个元素表示 [TTL MOVE 规则](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl)的最大键值。 - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) – `bytes_on_disk` 的别名。 - -* `marks_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) – `marks_bytes` 的别名。 - -**示例** - -```sql -SELECT * FROM system.parts LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_4_1_6 -part_type: Wide -active: 1 -marks: 2 -rows: 6 -bytes_on_disk: 310 -data_compressed_bytes: 157 -data_uncompressed_bytes: 91 -secondary_indices_compressed_bytes: 58 -secondary_indices_uncompressed_bytes: 6 -secondary_indices_marks_bytes: 48 -marks_bytes: 144 -modification_time: 2020-06-18 13:01:49 -remove_time: 1970-01-01 00:00:00 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -min_time: 1970-01-01 00:00:00 -max_time: 1970-01-01 00:00:00 -partition_id: all -min_block_number: 1 -max_block_number: 4 -level: 1 -data_version: 6 -primary_key_bytes_in_memory: 8 -primary_key_bytes_in_memory_allocated: 64 -is_frozen: 0 -database: default -table: months -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/months/all_1_4_1_6/ -hash_of_all_files: 2d0657a16d9430824d35e327fcbd87bf -hash_of_uncompressed_files: 84950cc30ba867c77a408ae21332ba29 -uncompressed_hash_of_compressed_files: 1ad78f1c6843bbfb99a2c931abe7df7d -delete_ttl_info_min: 1970-01-01 00:00:00 -delete_ttl_info_max: 1970-01-01 00:00:00 -move_ttl_info.expression: [] -move_ttl_info.min: [] -move_ttl_info.max: [] -``` - -**另请参阅** - -* [MergeTree 系列](../../engines/table-engines/mergetree-family/mergetree.md) -* [列和表的 TTL](../../engines/table-engines/mergetree-family/mergetree.md/#table_engine-mergetree-ttl) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md deleted file mode 100644 index 1ba8e5af741..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/parts_columns.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -description: '包含 MergeTree 表各数据部分及其列信息的系统表。' -keywords: ['系统表', 'parts_columns'] -slug: /operations/system-tables/parts_columns -title: 'system.parts_columns' -doc_type: 'reference' ---- - -# system.parts_columns {#systemparts_columns} - -包含 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表的各个数据 part 及其列的信息。 - -每一行对应一个数据 part。 - -列: - -* `partition` ([String](../../sql-reference/data-types/string.md)) — 分区名称。若要了解什么是分区,请参见 [ALTER](/sql-reference/statements/alter) 语句的说明。 - - 格式: - - * `YYYYMM` 表示按月自动分区。 - * `any_string` 表示手动指定分区。 - -* `name` ([String](../../sql-reference/data-types/string.md)) — 数据部分的名称。 - -* `part_type` ([String](../../sql-reference/data-types/string.md)) — 数据片段的存储格式。 - - 可能的值: - - * `Wide` — 在文件系统中,每一列都存储在单独的文件中。 - * `Compact` — 在文件系统中,所有列都存储在同一个文件中。 - - 数据存储格式由 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表的 `min_bytes_for_wide_part` 和 `min_rows_for_wide_part` 设置项控制。 - -* `active` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 指示数据片段是否处于活动状态的标志。如果数据片段处于活动状态,则会在表中使用;否则会被删除。合并后,非活动数据片段会保留。 - -* `marks` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 标记数。要估算一个数据部分中的大致行数,将 `marks` 乘以索引粒度(通常为 8192)(对于自适应粒度,此提示不适用)。 - -* `rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 行数。 - -* `bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 所有数据部分文件的总大小(以字节为单位)。 - -* `data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 数据分片中压缩数据的总大小。不包括所有辅助文件(例如标记文件)。 - -* `data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 数据部分中未压缩数据的总大小。不包括所有辅助文件(例如标记文件)。 - -* `marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `marks` 文件的大小。 - -* `modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 存放数据部分的目录的修改时间。通常对应于该数据部分的创建时间。 - -* `remove_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 数据片段变为非活跃状态的时间。 - -* `refcount` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 数据部分被引用的次数。值大于 2 表示该数据部分正在被查询或参与合并。 - -* `min_date` ([Date](../../sql-reference/data-types/date.md)) — 数据部分中日期键的最小值。 - -* `max_date` ([Date](../../sql-reference/data-types/date.md)) — 该数据片段中日期键的最大值。 - -* `partition_id` ([String](../../sql-reference/data-types/string.md)) — 分区 ID。 - -* `min_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 合并后构成当前数据部分的源数据部分的最小编号。 - -* `max_block_number` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 合并后构成当前数据片段的所有数据片段中编号的最大值。 - -* `level` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 合并树的深度。0 表示当前数据片段是由插入生成的,而不是通过合并其他片段生成的。 - -* `data_version`([UInt64](../../sql-reference/data-types/int-uint.md))— 用于确定应对数据分片应用哪些变更的数字(会应用版本号高于当前 `data_version` 的变更)。 - -* `primary_key_bytes_in_memory` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 主键值在内存中占用的字节数。 - -* `primary_key_bytes_in_memory_allocated` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 为主键值分配的内存字节数。 - -* `database` ([String](../../sql-reference/data-types/string.md)) — 数据库名称。 - -* `table` ([String](../../sql-reference/data-types/string.md)) — 表的名称。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) — 不带参数的表引擎的名称。 - -* `disk_name` ([String](../../sql-reference/data-types/string.md)) — 用于存储数据部分的磁盘名称。 - -* `path` ([String](../../sql-reference/data-types/string.md)) — 存放数据分片文件的文件夹的绝对路径。 - -* `column` ([String](../../sql-reference/data-types/string.md)) — 列名称。 - -* `type` ([String](../../sql-reference/data-types/string.md)) — 列的数据类型。 - -* `column_position` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列在表中的序号,从 1 开始。 - -* `default_kind`([String](../../sql-reference/data-types/string.md))— 默认值所使用的表达式类型(`DEFAULT`、`MATERIALIZED`、`ALIAS`),如果未定义则为空字符串。 - -* `default_expression` ([String](../../sql-reference/data-types/string.md)) — 默认值表达式;如果未定义,则为空字符串。 - -* `column_bytes_on_disk` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列在磁盘上的总大小,单位为字节。 - -* `column_data_compressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列中压缩数据的总大小,以字节为单位。 - -* `column_data_uncompressed_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列中解压缩后数据的总大小(以字节为单位)。 - -* `column_marks_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 列标记数据的大小(字节)。 - -* `bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — `bytes_on_disk` 的别名。 - -* `marks_size`([UInt64](../../sql-reference/data-types/int-uint.md))— `marks_bytes` 的别名。 - -**示例** - -```sql -SELECT * FROM system.parts_columns LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -partition: tuple() -name: all_1_2_1 -part_type: Wide -active: 1 -marks: 2 -rows: 2 -bytes_on_disk: 155 -data_compressed_bytes: 56 -data_uncompressed_bytes: 4 -marks_bytes: 96 -modification_time: 2020-09-23 10:13:36 -remove_time: 2106-02-07 06:28:15 -refcount: 1 -min_date: 1970-01-01 -max_date: 1970-01-01 -partition_id: all -min_block_number: 1 -max_block_number: 2 -level: 1 -data_version: 1 -primary_key_bytes_in_memory: 2 -primary_key_bytes_in_memory_allocated: 64 -database: default -table: 53r93yleapyears -engine: MergeTree -disk_name: default -path: /var/lib/clickhouse/data/default/53r93yleapyears/all_1_2_1/ -column: id -type: Int8 -column_position: 1 -default_kind: -default_expression: -column_bytes_on_disk: 76 -column_data_compressed_bytes: 28 -column_data_uncompressed_bytes: 2 -column_marks_bytes: 48 -``` - -**另请参阅** - -* [MergeTree 系列表引擎](../../engines/table-engines/mergetree-family/mergetree.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md deleted file mode 100644 index ca77ce22253..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processes.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: '用于实现 `SHOW PROCESSLIST` 查询的系统表。' -keywords: ['系统表', '进程'] -slug: /operations/system-tables/processes -title: 'system.processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processes {#systemprocesses} - - - -此系统表用于实现 `SHOW PROCESSLIST` 查询。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.processes LIMIT 10 FORMAT Vertical; -``` - -```response -第 1 行: -────── -is_initial_query: 1 -user: default -query_id: 35a360fa-3743-441d-8e1f-228c938268da -address: ::ffff:172.23.0.1 -port: 47588 -initial_user: default -initial_query_id: 35a360fa-3743-441d-8e1f-228c938268da -initial_address: ::ffff:172.23.0.1 -initial_port: 47588 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -elapsed: 0.000582537 -is_cancelled: 0 -is_all_data_sent: 0 -read_rows: 0 -read_bytes: 0 -total_rows_approx: 0 -written_rows: 0 -written_bytes: 0 -memory_usage: 0 -peak_memory_usage: 0 -query: SELECT * from system.processes LIMIT 10 FORMAT Vertical; -thread_ids: [67] -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -Settings: {'background_pool_size':'32','load_balancing':'random','allow_suspicious_low_cardinality_types':'1','distributed_aggregation_memory_efficient':'1','skip_unavailable_shards':'1','log_queries':'1','max_bytes_before_external_group_by':'20000000000','max_bytes_before_external_sort':'20000000000','allow_introspection_functions':'1'} - -结果集包含 1 行。用时:0.002 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md deleted file mode 100644 index 13cc6cce487..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/processors_profile_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: '包含处理器级别性能分析信息的系统表(可在 `EXPLAIN PIPELINE` 中查看)' -keywords: ['系统表', 'processors_profile_log', 'EXPLAIN PIPELINE'] -slug: /operations/system-tables/processors_profile_log -title: 'system.processors_profile_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.processors_profile_log {#systemprocessors_profile_log} - - - -此表包含处理器级别的性能分析信息(可以在 [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline) 中查看)。 - -Columns: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件发生的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件发生的日期和时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 事件发生的日期和时间(精确到微秒)。 -* `id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 处理器 ID。 -* `parent_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 父处理器 ID 列表。 -* `plan_step` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 创建此处理器的查询计划步骤 ID。如果处理器不是由任何步骤添加,则该值为零。 -* `plan_group` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 如果由查询计划步骤创建,则为处理器所属的分组。分组是对由同一查询计划步骤添加的处理器进行逻辑划分。分组仅用于美化 EXPLAIN PIPELINE 结果的展示。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初始查询的 ID(用于分布式查询执行)。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询 ID。 -* `name` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — 处理器名称。 -* `elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 此处理器执行所用的微秒数。 -* `input_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 此处理器等待数据(来自其他处理器)所用的微秒数。 -* `output_wait_elapsed_us` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 此处理器因输出端口已满而等待所用的微秒数。 -* `input_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 处理器消耗的行数。 -* `input_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 处理器消耗的字节数。 -* `output_rows` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 处理器生成的行数。 -* `output_bytes` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 处理器生成的字节数。 - -**Example** - -Query: - -```sql -EXPLAIN PIPELINE -SELECT sleep(1) -┌─explain─────────────────────────┐ -│ (Expression) │ -│ ExpressionTransform │ -│ (SettingQuotaAndLimits) │ -│ (ReadFromStorage) │ -│ SourceFromSingleChunk 0 → 1 │ -└─────────────────────────────────┘ - -SELECT sleep(1) -SETTINGS log_processors_profiles = 1 -Query id: feb5ed16-1c24-4227-aa54-78c02b3b27d4 -┌─sleep(1)─┐ -│ 0 │ -└──────────┘ -1 rows in set. Elapsed: 1.018 sec. - -SELECT - name, - elapsed_us, - input_wait_elapsed_us, - output_wait_elapsed_us -FROM system.processors_profile_log -WHERE query_id = 'feb5ed16-1c24-4227-aa54-78c02b3b27d4' -ORDER BY name ASC -``` - -结果: - -```text -┌─name────────────────────┬─elapsed_us─┬─input_wait_elapsed_us─┬─output_wait_elapsed_us─┐ -│ ExpressionTransform │ 1000497 │ 2823 │ 197 │ -│ LazyOutputFormat │ 36 │ 1002188 │ 0 │ -│ LimitsCheckingTransform │ 10 │ 1002994 │ 106 │ -│ NullSource │ 5 │ 1002074 │ 0 │ -│ NullSource │ 1 │ 1002084 │ 0 │ -│ SourceFromSingleChunk │ 45 │ 4736 │ 1000819 │ -└─────────────────────────┴────────────┴───────────────────────┴────────────────────────┘ -``` - -这里可以看到: - -* `ExpressionTransform` 在执行 `sleep(1)` 函数,因此它的 `work` 将耗时约 1e6 微秒,所以 `elapsed_us` > 1e6。 -* `SourceFromSingleChunk` 需要等待,因为 `ExpressionTransform` 在执行 `sleep(1)` 期间不接受任何数据,所以它将处于 `PortFull` 状态约 1e6 微秒,因此 `output_wait_elapsed_us` > 1e6。 -* `LimitsCheckingTransform`/`NullSource`/`LazyOutputFormat` 需要等到 `ExpressionTransform` 执行完 `sleep(1)` 后才能处理结果,因此 `input_wait_elapsed_us` > 1e6。 - -**另请参阅** - -* [`EXPLAIN PIPELINE`](../../sql-reference/statements/explain.md#explain-pipeline) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md deleted file mode 100644 index dbb073527df..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '包含 MergeTree 系列表投影部件信息的系统表。' -keywords: ['system table', 'projection_parts'] -slug: /operations/system-tables/projection_parts -title: 'system.projection_parts' -doc_type: 'reference' ---- - -# system.projection_parts {#systemprojection_parts} - -此表包含 MergeTree 系列表的投影数据片段信息。 - -## 列 {#columns} - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md deleted file mode 100644 index 5795cc919e3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projection_parts_columns.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '包含 MergeTree 家族表的投影数据片段中列信息的系统表' -keywords: ['系统表', 'projection_parts_columns'] -slug: /operations/system-tables/projection_parts_columns -title: 'system.projection_parts_columns' -doc_type: 'reference' ---- - -# system.projection_parts_columns {#systemprojection_parts_columns} - -此表包含 MergeTree 系列表的投影部件中各列的信息。 - -## 列 {#columns} - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md deleted file mode 100644 index 09c34e5dc7d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/projections.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: '包含所有表中已存在投影信息的系统表。' -keywords: ['系统表', '投影'] -slug: /operations/system-tables/projections -title: 'system.projections' -doc_type: 'reference' ---- - -# system.projections {#systemprojections} - -包含所有表中已存在的投影信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.projections LIMIT 2 FORMAT Vertical; -``` - -```text -行 1: -────── -database: default -table: landing -name: improved_sorting_key -type: Normal -sorting_key: ['user_id','date'] -query: SELECT * ORDER BY user_id, date - -行 2: -────── -database: default -table: landing -name: agg_no_key -type: Aggregate -sorting_key: [] -query: SELECT count() -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md deleted file mode 100644 index d54c2d4f62f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_cache.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: '用于显示查询缓存内容的系统表。' -keywords: ['system table', 'query_cache'] -slug: /operations/system-tables/query_cache -title: 'system.query_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_cache {#systemquery_cache} - - - -显示 [查询缓存](../query-cache.md) 的内容。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.query_cache FORMAT Vertical; -``` - -```text -第 1 行: -────── -query: SELECT 1 SETTINGS use_query_cache = 1 -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -result_size: 128 -tag: -stale: 0 -shared: 0 -compressed: 1 -expires_at: 2023-10-13 13:35:45 -key_hash: 12188185624808016954 - -返回 1 行。用时:0.004 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md deleted file mode 100644 index 4b78fbaae7f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_condition_cache.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '用于显示查询条件缓存内容的系统表。' -keywords: ['system table', 'query_condition_cache'] -slug: /operations/system-tables/query_condition_cache -title: 'system.query_condition_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_condition_cache {#systemquery_condition_cache} - - - -显示 [查询条件缓存](../query-condition-cache.md) 的内容。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.query_condition_cache FORMAT Vertical; -``` - -```text -第 1 行: -────── -table_uuid: 28270a24-ea27-49f6-99cd-97b9bee976ac -part_name: all_1_1_0 -condition: or(equals(b, 10000_UInt16), equals(c, 10000_UInt16)) -condition_hash: 5456494897146899690 -- 5.46 quintillion -entry_size: 40 -matching_marks: 111111110000000000000000000000000000000000000000000000000111111110000000000000000 - -结果集包含 1 行。用时:0.004 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md deleted file mode 100644 index 38432aff188..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_log.md +++ /dev/null @@ -1,242 +0,0 @@ ---- -description: '包含已执行查询信息的系统表,例如开始时间、处理时长和错误消息。' -keywords: ['系统表', 'query_log'] -slug: /operations/system-tables/query_log -title: 'system.query_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.query_log {#systemquery_log} - - - -存储已执行查询的元数据和统计信息,例如开始时间、持续时间、错误消息、资源使用情况以及其他执行细节。它不会存储查询结果。 - -可以在服务器配置的 [query_log](../../operations/server-configuration-parameters/settings.md#query_log) 部分更改查询日志记录相关设置。 - -可以通过将 [log_queries](/operations/settings/settings#log_queries) 设置为 `0` 来禁用查询日志记录。不建议关闭日志记录,因为此表中的信息对于排查问题非常重要。 - -数据刷新周期由服务器设置中 [query_log](../../operations/server-configuration-parameters/settings.md#query_log) 部分的 `flush_interval_milliseconds` 参数控制。要强制刷新,请使用 [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) 查询。 - -ClickHouse 不会自动从该表中删除数据。更多详情参见[简介](/operations/system-tables/overview#system-tables-introduction)。 - -`system.query_log` 表会记录两类查询: - -1. 由客户端直接运行的初始查询。 -2. 由其他查询发起的子查询(用于分布式查询执行)。对于此类查询,有关父查询的信息会显示在 `initial_*` 列中。 - -每个查询会在 `query_log` 表中创建一行或两行记录,具体取决于查询的状态(参见 `type` 列): - -1. 如果查询执行成功,则会创建两条类型为 `QueryStart` 和 `QueryFinish` 的记录。 -2. 如果在查询处理期间发生错误,则会创建两条类型为 `QueryStart` 和 `ExceptionWhileProcessing` 的记录。 -3. 如果在启动查询之前发生错误,则会创建一条类型为 `ExceptionBeforeStart` 的记录。 - -可以使用 [log_queries_probability](/operations/settings/settings#log_queries_probability) 设置来减少记录到 `query_log` 表中的查询数量。 - -可以使用 [log_formatted_queries](/operations/settings/settings#log_formatted_queries) 设置将格式化后的查询记录到 `formatted_query` 列中。 - - - -## 列 {#columns} - - - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — 执行查询时发生的事件类型。取值: - * `'QueryStart' = 1` — 成功开始执行查询。 - * `'QueryFinish' = 2` — 成功结束查询执行。 - * `'ExceptionBeforeStart' = 3` — 在开始执行查询前发生异常。 - * `'ExceptionWhileProcessing' = 4` — 在查询执行过程中发生异常。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 查询起始日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 查询的开始时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 查询开始时间,微秒级精度。 -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 查询开始执行的时间。 -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 查询执行的开始时间,具有微秒级精度。 -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 查询执行耗时,单位为毫秒。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 在查询中,从所有参与的表和表函数读取的总行数。它包括普通子查询以及用于 `IN` 和 `JOIN` 的子查询。对于分布式查询,`read_rows` 包含在所有副本上读取的行总数。每个副本都会发送自己的 `read_rows` 值,查询的发起服务器会汇总所有接收到的值和本地值。缓存的使用情况不会影响该值。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 从参与查询的所有表和表函数中读取的字节总数。包括普通子查询,以及用于 `IN` 和 `JOIN` 的子查询。对于分布式查询,`read_bytes` 包含在所有副本上读取的字节总数。每个副本会发送自己的 `read_bytes` 值,发起查询的服务器会汇总所有接收到的值以及本地的值。缓存中的数据量不会影响该值。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 对于 `INSERT` 查询,表示写入的行数。对于其他查询,该列的值为 0。` -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 对于 `INSERT` 查询,表示写入的未压缩字节数。对于其他查询,该列的值为 0。 -* `result_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — `SELECT` 查询结果中的行数,或 `INSERT` 查询写入的行数。 -* `result_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 在 RAM 中存储查询结果所占用的字节数。 -* `memory_usage` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 查询的内存使用量。 -* `current_database` ([String](../../sql-reference/data-types/string.md)) — 当前数据库的名称。 -* `query` ([String](../../sql-reference/data-types/string.md)) — 查询语句字符串。 -* `formatted_query` ([String](../../sql-reference/data-types/string.md)) — 格式化查询字符串。 -* `normalized_query_hash` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 一个数值哈希值,使得仅在字面量取值上不同的查询具有相同的值。 -* `query_kind` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — 查询类型。 -* `databases` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 查询中出现的数据库名称。 -* `tables` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 查询中涉及的表名。 -* `columns` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 查询中出现的列的名称。 -* `partitions` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 查询中包含的分区名称。 -* `projections` ([String](../../sql-reference/data-types/string.md)) — 查询执行期间使用的投影名称。 -* `views` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 查询中出现的(物化或实时)视图名称。 -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — 异常代码。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 异常信息。 -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [堆栈跟踪](https://en.wikipedia.org/wiki/Stack_trace)。如果查询成功完成,则该字段为空字符串。 -* `is_initial_query` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 查询类型。可能的取值如下: - * 1 — 查询由客户端发起。 - * 0 — 查询作为分布式查询执行的一部分,由另一条查询语句发起。 -* `user` ([String](../../sql-reference/data-types/string.md)) — 发起当前查询的用户名。` -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询 ID。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 用于执行该查询的 IP 地址。 -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 客户端用于发出该查询的端口。 -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — 执行初始查询的用户名称(用于分布式查询执行)。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初始查询的 ID(在分布式查询执行中)。 -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 父查询发起时的 IP 地址。 -* `initial_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 用于发起父查询的客户端端口。 -* `initial_query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 分布式查询执行时的初始查询开始时间。 -* `initial_query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 初始查询开始时间,精度为微秒(在分布式查询执行中使用)。 -* `interface` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 发起该查询的接口。可能的取值: - * 1 — TCP。 - * 2 — HTTP。 -* `os_user` ([String](../../sql-reference/data-types/string.md)) — 运行 [clickhouse-client](../../interfaces/cli.md) 的操作系统用户名。 -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — 客户端所在机器的主机名,即运行 [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的主机名。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的名称。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的修订版本号。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的主版本号。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的次要版本号。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端版本的补丁号部分。 -* `script_query_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 在包含多个查询的脚本中,用于 [clickhouse-client](../../interfaces/cli.md) 的查询编号。 -* `script_line_number` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 在包含多个查询的脚本中,该查询在脚本中的起始行号,供 [clickhouse-client](../../interfaces/cli.md) 使用。 -* `http_method` (UInt8) — 发起查询的 HTTP 方法。可能的取值如下: - * 0 — 查询通过 TCP 接口发起。 - * 1 — 使用了 `GET` 方法。 - * 2 — 使用了 `POST` 方法。 -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — 在 HTTP 请求中传递的 HTTP 头 `UserAgent`。 -* `http_referer` ([String](../../sql-reference/data-types/string.md)) — HTTP 首部字段 `Referer`,在 HTTP 请求中传递(包含发起该请求的页面的完整或部分地址)。 -* `forwarded_for` ([String](../../sql-reference/data-types/string.md)) — 在 HTTP 查询中传入的 HTTP 请求头 `X-Forwarded-For`。 -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — 在 [quotas](../../operations/quotas.md) 设置中指定的 `quota key`(参见 `keyed`)。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse 修订版本号。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/map.md)) — 用于度量不同指标的 ProfileEvents。其详细说明可以在表 [system.events](/operations/system-tables/events) 中找到 -* `Settings` ([Map(String, String)](../../sql-reference/data-types/map.md)) — 客户端执行查询时被更改的设置。要启用对设置变更的日志记录,请将 `log_query_settings` 参数设置为 1。 -* `log_comment` ([String](../../sql-reference/data-types/string.md)) — 日志备注。可以设置为任意字符串,长度不得超过 [max_query_size](../../operations/settings/settings.md#max_query_size)。如果未定义,则为空字符串。 -* `thread_ids` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 参与查询执行的线程ID。这些线程可能并未同时运行。 -* `peak_threads_usage` ([UInt64)](../../sql-reference/data-types/int-uint.md)) — 执行该查询时并发线程的最大数量。 -* `used_aggregate_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中所使用的 `aggregate functions`(聚合函数)的标准名称。 -* `used_aggregate_function_combinators` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中使用的 `aggregate functions combinators` 的规范名称。 -* `used_database_engines` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行期间使用的 `database engines` 的规范名称。 -* `used_data_type_families` ([Array(String)](../../sql-reference/data-types/array.md)) — 查询执行期间使用到的 `data type families` 的规范名称。 -* `used_dictionaries` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行期间使用的 `dictionaries` 的规范名称。对于使用 XML 文件配置的字典,其规范名称为该字典的名称;对于通过 SQL 语句创建的字典,其规范名称为完全限定的对象名称。` -* `used_formats` ([Array(String)](../../sql-reference/data-types/array.md)) — 查询执行期间使用的 `formats` 的规范名称。` -* `used_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中使用的 `functions` 的规范名称。 -* `used_storages` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中使用的 `storages` 的规范名称。 -* `used_table_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中使用的 `table functions` 的规范名称。 -* `used_executable_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行期间使用的“可执行用户自定义函数”(executable user defined functions)的规范名称。 -* `used_sql_user_defined_functions` ([Array(String)](../../sql-reference/data-types/array.md)) — 在查询执行过程中使用到的 SQL 用户自定义函数的规范名称。 -* `used_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) - 在查询执行过程中成功通过检查的权限。 -* `missing_privileges` ([Array(String)](../../sql-reference/data-types/array.md)) - 在查询执行期间缺少的权限。 -* `query_cache_usage` ([Enum8](../../sql-reference/data-types/enum.md)) — 查询执行期间对 [query cache](../query-cache.md) 的使用方式。可能的取值: - * `'Unknown'` = 状态未知。 - * `'None'` = 查询结果既未写入查询缓存,也未从查询缓存读取。 - * `'Write'` = 查询结果已写入查询缓存。 - * `'Read'` = 查询结果已从查询缓存读取。 - - - - - -## 示例 {#examples} - -**基本示例** - -```sql -SELECT * FROM system.query_log WHERE type = 'QueryFinish' ORDER BY query_start_time DESC LIMIT 1 FORMAT Vertical; -``` - -```text -第 1 行: -────── -hostname: clickhouse.eu-central1.internal -type: QueryFinish -event_date: 2021-11-03 -event_time: 2021-11-03 16:13:54 -event_time_microseconds: 2021-11-03 16:13:54.953024 -query_start_time: 2021-11-03 16:13:54 -query_start_time_microseconds: 2021-11-03 16:13:54.952325 -query_duration_ms: 0 -read_rows: 69 -read_bytes: 6187 -written_rows: 0 -written_bytes: 0 -result_rows: 69 -result_bytes: 48256 -memory_usage: 0 -current_database: default -query: DESCRIBE TABLE system.query_log -formatted_query: -normalized_query_hash: 8274064835331539124 -query_kind: -databases: [] -tables: [] -columns: [] -projections: [] -views: [] -exception_code: 0 -exception: -stack_trace: -is_initial_query: 1 -user: default -query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -address: ::ffff:127.0.0.1 -port: 40452 -initial_user: default -initial_query_id: 7c28bbbb-753b-4eba-98b1-efcbe2b9bdf6 -initial_address: ::ffff:127.0.0.1 -initial_port: 40452 -initial_query_start_time: 2021-11-03 16:13:54 -initial_query_start_time_microseconds: 2021-11-03 16:13:54.952325 -interface: 1 -os_user: sevirov -client_hostname: clickhouse.eu-central1.internal -client_name: ClickHouse -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 1 -http_method: 0 -http_user_agent: -http_referer: -forwarded_for: -quota_key: -revision: 54456 -log_comment: -thread_ids: [30776,31174] -ProfileEvents: {'Query':1,'NetworkSendElapsedMicroseconds':59,'NetworkSendBytes':2643,'SelectedRows':69,'SelectedBytes':6187,'ContextLock':9,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':817,'UserTimeMicroseconds':427,'SystemTimeMicroseconds':212,'OSCPUVirtualTimeMicroseconds':639,'OSReadChars':894,'OSWriteChars':319} -Settings: {'load_balancing':'random','max_memory_usage':'10000000000'} -used_aggregate_functions: [] -used_aggregate_function_combinators: [] -used_database_engines: [] -used_data_type_families: [] -used_dictionaries: [] -used_formats: [] -used_functions: [] -used_storages: [] -used_table_functions: [] -used_executable_user_defined_functions:[] -used_sql_user_defined_functions: [] -used_privileges: [] -missing_privileges: [] -query_cache_usage: None -``` - -**Cloud 示例** - -在 ClickHouse Cloud 中,`system.query_log` 是每个节点上的本地表;要查看所有记录,需要通过 [`clusterAllReplicas`](/sql-reference/table-functions/cluster) 进行查询。 - -例如,要汇总 “default” 集群中每个副本的 `query_log` 记录,可以编写如下查询: - -```sql -SELECT * -FROM clusterAllReplicas('default', system.query_log) -WHERE event_time >= now() - toIntervalHour(1) -LIMIT 10 -SETTINGS skip_unavailable_shards = 1; -``` - -**另请参阅** - -* [system.query_thread_log](/operations/system-tables/query_thread_log) — 此表包含关于每个查询执行线程的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md deleted file mode 100644 index 623065bc829..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_metric_log.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '系统表,包含针对各个查询、从 `system.events` 表收集的内存和指标值历史记录,并定期写入磁盘。' -keywords: ['系统表', 'query_metric_log'] -slug: /operations/system-tables/query_metric_log -title: 'system.query_metric_log' -doc_type: '参考' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_metric_log {#systemquery_metric_log} - - - -包含来自表 `system.events` 的各个查询的内存和指标值历史记录,并定期刷新到磁盘。 - -一旦查询开始,就会按照 `query_metric_log_interval` 毫秒的周期采集数据(默认值为 1000 毫秒)。如果查询耗时超过 `query_metric_log_interval`,则在查询结束时也会采集一次数据。 - -列: - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询的 ID。 -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行该查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 事件时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒精度的事件时间。 - -**示例** - -```sql -SELECT * FROM system.query_metric_log LIMIT 1 FORMAT Vertical; -``` - -```text -第 1 行: -────── -query_id: 97c8ba04-b6d4-4bd7-b13e-6201c5c6e49d -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-05 -event_time: 2020-09-05 16:22:33 -event_time_microseconds: 2020-09-05 16:22:33.196807 -memory_usage: 313434219 -peak_memory_usage: 598951986 -ProfileEvent_Query: 0 -ProfileEvent_SelectQuery: 0 -ProfileEvent_InsertQuery: 0 -ProfileEvent_FailedQuery: 0 -ProfileEvent_FailedSelectQuery: 0 -... -``` - -**另请参阅** - -* [query_metric_log setting](../../operations/server-configuration-parameters/settings.md#query_metric_log) — 启用或禁用该设置项。 -* [query_metric_log_interval](../../operations/settings/settings.md#query_metric_log_interval) -* [system.asynchronous_metrics](../../operations/system-tables/asynchronous_metrics.md) — 包含定期计算的指标。 -* [system.events](/operations/system-tables/events) — 包含已发生的一系列事件。 -* [system.metrics](../../operations/system-tables/metrics.md) — 包含实时计算的指标。 -* [Monitoring](../../operations/monitoring.md) — ClickHouse 监控的基本概念。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md deleted file mode 100644 index 6d5588d5956..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_thread_log.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -description: '存储执行查询的线程信息的系统表,例如线程名称、线程开始时间、查询处理时长。' -keywords: ['系统表', 'query_thread_log'] -slug: /operations/system-tables/query_thread_log -title: 'system.query_thread_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_thread_log {#systemquery_thread_log} - - - -包含执行查询的线程信息,例如线程名称、线程开始时间、查询处理持续时间等。 - -要开始记录日志: - -1. 在 [query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log) 部分配置相关参数。 -2. 将 [log_query_threads](/operations/settings/settings#log_query_threads) 设置为 1。 - -数据的刷新周期通过服务器设置中 [query_thread_log](/operations/server-configuration-parameters/settings#query_thread_log) 部分的 `flush_interval_milliseconds` 参数进行配置。要强制刷新,请使用 [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) 查询。 - -ClickHouse 不会自动从该表中删除数据。更多详细信息请参阅[简介](/operations/system-tables/overview#system-tables-introduction)。 - -你可以使用 [log_queries_probability](/operations/settings/settings#log_queries_probability) 设置来减少记录到 `query_thread_log` 表中的查询数量。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 线程完成查询执行时的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 线程完成查询执行时的日期和时间。 -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — 线程完成执行该查询时的日期和时间,精确到微秒。 -* `query_start_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 查询开始执行的时间。 -* `query_start_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 查询执行开始时间,精确到微秒。 -* `query_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 查询执行耗时。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 已读取的行数。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 已读取的字节数。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 在 `INSERT` 查询中,表示写入的行数。对于其他查询,该列的值为 0。 -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 对于 `INSERT` 查询,表示写入的字节数。对于其他查询,该列的值为 0。 -* `memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — 该线程上下文内已分配内存量与已释放内存量之间的差值。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — 在该线程上下文中,已分配内存量与已释放内存量之间的最大差值。 -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — 线程名。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 操作系统线程 ID。 -* `master_thread_id` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 初始线程在操作系统中的 ID。 -* `query` ([String](../../sql-reference/data-types/string.md)) — 查询字符串。 -* `is_initial_query` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 查询类型。可能的取值: - * 1 — 查询由客户端发起。 - * 0 — 查询在执行分布式查询时由其他查询发起。 -* `user` ([String](../../sql-reference/data-types/string.md)) — 发起当前查询的用户名称。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询 ID。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 用于执行查询的 IP 地址。 -* `port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — 客户端用于发起该查询的端口。 -* `initial_user` ([String](../../sql-reference/data-types/string.md)) — 运行初始查询的用户的名称(用于分布式查询执行)。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初始查询的 ID(在分布式查询执行中使用)。 -* `initial_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 父查询发起时的 IP 地址。 -* `initial_port` ([UInt16](/sql-reference/data-types/int-uint#integer-ranges)) — 用于发起父查询的客户端端口。 -* `interface` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 表示发起该查询的接口。可能的取值: - * 1 — TCP。 - * 2 — HTTP。 -* `os_user` ([String](../../sql-reference/data-types/string.md)) — 在操作系统中运行 [clickhouse-client](../../interfaces/cli.md) 的用户名。 -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — 运行 [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的机器的主机名。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的名称。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的修订号。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的主版本号。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的次版本号。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端版本中的补丁号部分。 -* `http_method` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 发起该查询的 HTTP 方法。可能的取值: - * 0 — 查询通过 TCP 接口发起。 - * 1 — 使用了 `GET` 方法。 - * 2 — 使用了 `POST` 方法。 -* `http_user_agent` ([String](../../sql-reference/data-types/string.md)) — 在 HTTP 请求中传递的 `User-Agent` 头部。 -* `quota_key` ([String](../../sql-reference/data-types/string.md)) — 在 [quotas](../../operations/quotas.md) 设置中指定的“quota key”(参见 `keyed`)。 -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse 修订号。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — 用于衡量该线程各类指标的 ProfileEvents。相关说明可在表 [system.events](/operations/system-tables/events) 中查阅。 - -**示例** - -```sql - SELECT * FROM system.query_thread_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-11 -event_time: 2020-09-11 10:08:17 -event_time_microseconds: 2020-09-11 10:08:17.134042 -query_start_time: 2020-09-11 10:08:17 -query_start_time_microseconds: 2020-09-11 10:08:17.063150 -query_duration_ms: 70 -read_rows: 0 -read_bytes: 0 -written_rows: 1 -written_bytes: 12 -memory_usage: 4300844 -peak_memory_usage: 4300844 -thread_name: TCPHandler -thread_id: 638133 -master_thread_id: 638133 -query: INSERT INTO test1 VALUES -is_initial_query: 1 -user: default -query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -address: ::ffff:127.0.0.1 -port: 33452 -initial_user: default -initial_query_id: 50a320fd-85a8-49b8-8761-98a86bcbacef -initial_address: ::ffff:127.0.0.1 -initial_port: 33452 -interface: 1 -os_user: bharatnc -client_hostname: tower -client_name: ClickHouse -client_revision: 54437 -client_version_major: 20 -client_version_minor: 7 -client_version_patch: 2 -http_method: 0 -http_user_agent: -quota_key: -revision: 54440 -ProfileEvents: {'Query':1,'SelectQuery':1,'ReadCompressedBytes':36,'CompressedReadBufferBlocks':1,'CompressedReadBufferBytes':10,'IOBufferAllocs':1,'IOBufferAllocBytes':89,'ContextLock':15,'RWLockAcquiredReadLocks':1} -``` - -**另请参阅** - -* [system.query_log](/operations/system-tables/query_log) — `query_log` 系统表的说明,其中包含关于查询执行的常规信息。 -* [system.query_views_log](/operations/system-tables/query_views_log) — 此表包含关于在查询期间执行的每个视图的信息。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md deleted file mode 100644 index e0240a11033..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/query_views_log.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -description: '在运行查询时记录已执行依赖视图信息的系统表,例如视图类型或执行时间。' -keywords: ['系统表', 'query_views_log'] -slug: /operations/system-tables/query_views_log -title: 'system.query_views_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.query_views_log {#systemquery_views_log} - - - -包含在运行查询时执行的依赖视图的信息,例如视图类型或执行时间。 - -要开始记录日志: - -1. 在 [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log) 部分配置参数。 -2. 将 [log_query_views](/operations/settings/settings#log_query_views) 设置为 1。 - -数据刷新的周期由服务器设置部分 [query_views_log](../../operations/server-configuration-parameters/settings.md#query_views_log) 中的 `flush_interval_milliseconds` 参数指定。要强制刷新,请使用 [SYSTEM FLUSH LOGS](/sql-reference/statements/system#flush-logs) 查询。 - -ClickHouse 不会自动从该表中删除数据。更多详情参见[简介](/operations/system-tables/overview#system-tables-introduction)。 - -可以使用 [log_queries_probability](/operations/settings/settings#log_queries_probability)) 设置来减少在 `query_views_log` 表中记录的查询数量。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 视图最后一次事件发生的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 视图完成执行的日期和时间。 -* `event_time_microseconds` ([DateTime](../../sql-reference/data-types/datetime.md)) — 视图完成执行的日期和时间(微秒精度)。 -* `view_duration_ms` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 视图执行的持续时间(各阶段之和),单位为毫秒。 -* `initial_query_id` ([String](../../sql-reference/data-types/string.md)) — 初始查询的 ID(用于分布式查询执行)。 -* `view_name` ([String](../../sql-reference/data-types/string.md)) — 视图名称。 -* `view_uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — 视图的 UUID。 -* `view_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 视图类型。取值: - * `'Default' = 1` — [默认视图](/sql-reference/statements/create/view#normal-view)。不应出现在该日志中。 - * `'Materialized' = 2` — [物化视图](/sql-reference/statements/create/view#materialized-view)。 - * `'Live' = 3` — [Live 视图](../../sql-reference/statements/create/view.md#live-view)。 -* `view_query` ([String](../../sql-reference/data-types/string.md)) — 由视图执行的查询。 -* `view_target` ([String](../../sql-reference/data-types/string.md)) — 视图目标表的名称。 -* `read_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 读取的行数。 -* `read_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 读取的字节数。 -* `written_rows` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 写入的行数。 -* `written_bytes` ([UInt64](/sql-reference/data-types/int-uint#integer-ranges)) — 写入的字节数。 -* `peak_memory_usage` ([Int64](../../sql-reference/data-types/int-uint.md)) — 在该视图上下文中,已分配与已释放内存量之间的最大差值。 -* `ProfileEvents` ([Map(String, UInt64)](../../sql-reference/data-types/array.md)) — 用于度量不同指标的 ProfileEvents。其说明可在表 [system.events](/operations/system-tables/events) 中找到。 -* `status` ([Enum8](../../sql-reference/data-types/enum.md)) — 视图状态。取值: - * `'QueryStart' = 1` — 成功开始视图的执行。不应出现在该日志中。 - * `'QueryFinish' = 2` — 视图执行成功结束。 - * `'ExceptionBeforeStart' = 3` — 在视图开始执行前发生异常。 - * `'ExceptionWhileProcessing' = 4` — 在视图执行过程中发生异常。 -* `exception_code` ([Int32](../../sql-reference/data-types/int-uint.md)) — 异常代码。 -* `exception` ([String](../../sql-reference/data-types/string.md)) — 异常消息。 -* `stack_trace` ([String](../../sql-reference/data-types/string.md)) — [堆栈跟踪](https://en.wikipedia.org/wiki/Stack_trace)。如果查询成功完成,则为空字符串。 - -**示例** - -查询: - -```sql -SELECT * FROM system.query_views_log LIMIT 1 \G; -``` - -结果: - -```text -第 1 行: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2021-06-22 -event_time: 2021-06-22 13:23:07 -event_time_microseconds: 2021-06-22 13:23:07.738221 -view_duration_ms: 0 -initial_query_id: c3a1ac02-9cad-479b-af54-9e9c0a7afd70 -view_name: default.matview_inner -view_uuid: 00000000-0000-0000-0000-000000000000 -view_type: Materialized -view_query: SELECT * FROM default.table_b -view_target: default.`.inner.matview_inner` -read_rows: 4 -read_bytes: 64 -written_rows: 2 -written_bytes: 32 -peak_memory_usage: 4196188 -ProfileEvents: {'FileOpen':2,'WriteBufferFromFileDescriptorWrite':2,'WriteBufferFromFileDescriptorWriteBytes':187,'IOBufferAllocs':3,'IOBufferAllocBytes':3145773,'FunctionExecute':3,'DiskWriteElapsedMicroseconds':13,'InsertedRows':2,'InsertedBytes':16,'SelectedRows':4,'SelectedBytes':48,'ContextLock':16,'RWLockAcquiredReadLocks':1,'RealTimeMicroseconds':698,'SoftPageFaults':4,'OSReadChars':463} -status: QueryFinish -exception_code: 0 -exception: -stack_trace: -``` - -**另请参见** - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md deleted file mode 100644 index 0584f1d6ebc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_limits.md +++ /dev/null @@ -1,28 +0,0 @@ ---- -description: '包含所有配额各时间区间最大值信息的系统表。一个配额可以对应任意数量的行(包括零行)。' -keywords: ['系统表', 'quota_limits'] -slug: /operations/system-tables/quota_limits -title: 'system.quota_limits' -doc_type: 'reference' ---- - -# system.quota_limits {#systemquota_limits} - -包含所有配额各个时间区间的最大限制信息。一个配额可以对应任意数量的行,也可以为零行。 - -列: - -* `quota_name` ([String](../../sql-reference/data-types/string.md)) — 配额名称。 -* `duration` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 用于计算资源消耗的时间区间长度,单位为秒。 -* `is_randomized_interval` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 逻辑值。指示该时间区间是否被随机化。如果未随机化,区间总是从同一时间开始。例如,1 分钟的区间总是从整数分钟开始(即可以从 11:20:00 开始,但绝不会从 11:20:01 开始),一天的区间总是从 UTC 午夜开始。如果区间被随机化,第一个区间从随机时间开始,后续区间依次顺延。取值: -* `0` — 区间未随机化。 -* `1` — 区间被随机化。 -* `max_queries` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 最大查询次数。 -* `max_query_selects` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 最大 `SELECT` 查询次数。 -* `max_query_inserts` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 最大 `INSERT` 查询次数。 -* `max_errors` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 最大错误次数。 -* `max_result_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 结果行数上限。 -* `max_result_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 用于存储查询结果的 RAM 使用量上限(字节数)。 -* `max_read_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 从查询中涉及的所有表和表函数读取的最大行数。 -* `max_read_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 从查询中涉及的所有表和表函数读取的最大字节数。 -* `max_execution_time` ([Nullable](../../sql-reference/data-types/nullable.md)([Float64](../../sql-reference/data-types/float.md))) — 查询执行时间上限,单位为秒。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md deleted file mode 100644 index 85b411d1556..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quota_usage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -description: '系统表,包含当前用户配额使用情况的信息,例如已使用多少配额以及剩余多少配额。' -keywords: ['系统表', 'quota_usage'] -slug: /operations/system-tables/quota_usage -title: 'system.quota_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - - -当前用户的配额使用情况:已用额度及剩余额度。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md deleted file mode 100644 index b777f77b1cd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: '包含配额信息的系统表。' -keywords: ['系统表', '配额', '配额'] -slug: /operations/system-tables/quotas -title: 'system.quotas' -doc_type: 'reference' ---- - - - -# system.quotas {#systemquotas} - -包含有关[配额](../../operations/system-tables/quotas.md)的信息。 - -列: -- `name` ([String](../../sql-reference/data-types/string.md)) — 配额名称。 -- `id` ([UUID](../../sql-reference/data-types/uuid.md)) — 配额 ID。 -- `storage`([String](../../sql-reference/data-types/string.md)) — 配额的存储方式。可能的值:"users.xml" 表示在 users.xml 文件中配置的配额,"disk" 表示通过 SQL 查询配置的配额。 -- `keys` ([Array](../../sql-reference/data-types/array.md)([Enum8](../../sql-reference/data-types/enum.md))) — 键指定配额应如何共享。如果两个连接使用相同的配额和键,它们共享相同数量的资源。取值: - - `[]` — 所有用户共享同一个配额。 - - `['user_name']` — 具有相同用户名的连接共享同一个配额。 - - `['ip_address']` — 来自同一 IP 的连接共享同一个配额。 - - `['client_key']` — 具有相同 key 的连接共享同一个配额。key 必须由客户端显式提供。使用 [clickhouse-client](../../interfaces/cli.md) 时,在 `--quota_key` 参数中传递 key 值,或在客户端配置文件中使用 `quota_key` 参数。使用 HTTP 接口时,使用 `X-ClickHouse-Quota` 请求头。 - - `['user_name', 'client_key']` — 具有相同 `client_key` 的连接共享同一个配额。如果客户端未提供 key,则按 `user_name` 跟踪配额。 - - `['client_key', 'ip_address']` — 具有相同 `client_key` 的连接共享同一个配额。如果客户端未提供 key,则按 `ip_address` 跟踪配额。 -- `durations` ([Array](../../sql-reference/data-types/array.md)([UInt64](../../sql-reference/data-types/int-uint.md))) — 以秒为单位的时间间隔长度。 -- `apply_to_all` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 逻辑值,指示配额适用于哪些用户。取值: - - `0` — 配额应用于 `apply_to_list` 中指定的用户。 - - `1` — 配额应用于除 `apply_to_except` 中列出的用户之外的所有用户。 -- `apply_to_list` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 应应用配额的用户名/[角色](../../guides/sre/user-management/index.md#role-management)列表。 -- `apply_to_except` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) — 不应应用配额的用户名/角色列表。 - - - -## 另请参阅 {#see-also} - -- [SHOW QUOTAS](/sql-reference/statements/show#show-quotas) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md deleted file mode 100644 index 46b6cccdb10..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/quotas_usage.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: '包含所有用户配额使用情况的系统表。' -keywords: ['system table', 'quotas_usage', 'quota'] -slug: /operations/system-tables/quotas_usage -title: 'system.quotas_usage' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - - -# system.quotas_usage {#systemquotas_usage} - - - -所有用户的配额使用信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW QUOTA](/sql-reference/statements/show#show-quota)) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md deleted file mode 100644 index fab35860a56..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicas.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: '包含本地服务器上各复制表信息和状态的系统表,适用于监控。' -keywords: ['system table', 'replicas'] -slug: /operations/system-tables/replicas -title: 'system.replicas' -doc_type: 'reference' ---- - -# system.replicas {#systemreplicas} - -包含本地服务器上所有复制表的相关信息和状态。 -此表可用于监控。表中每个使用 Replicated* 引擎的表对应一行记录。 - -示例: - -```sql -SELECT * -FROM system.replicas -WHERE table = 'test_table' -FORMAT Vertical -``` - -```text -Query id: dc6dcbcb-dc28-4df9-ae27-4354f5b3b13e - -Row 1: -─────── -database: db -table: test_table -engine: ReplicatedMergeTree -is_leader: 1 -can_become_leader: 1 -is_readonly: 0 -is_session_expired: 0 -future_parts: 0 -parts_to_check: 0 -zookeeper_path: /test/test_table -replica_name: r1 -replica_path: /test/test_table/replicas/r1 -columns_version: -1 -queue_size: 27 -inserts_in_queue: 27 -merges_in_queue: 0 -part_mutations_in_queue: 0 -queue_oldest_time: 2021-10-12 14:48:48 -inserts_oldest_time: 2021-10-12 14:48:48 -merges_oldest_time: 1970-01-01 03:00:00 -part_mutations_oldest_time: 1970-01-01 03:00:00 -oldest_part_to_get: 1_17_17_0 -oldest_part_to_merge_to: -oldest_part_to_mutate_to: -log_max_index: 206 -log_pointer: 207 -last_queue_update: 2021-10-12 14:50:08 -absolute_delay: 99 -total_replicas: 5 -active_replicas: 5 -lost_part_count: 0 -last_queue_update_exception: -zookeeper_exception: -replica_is_active: {'r1':1,'r2':1} -``` - -列: - -* `database` (`String`) - 数据库名称 -* `table` (`String`) - 表名 -* `engine` (`String`) - 表引擎名称 -* `is_leader` (`UInt8`) - 该副本是否为 leader。 - 多个副本可以同时为 leader。可以通过在 `merge_tree` 设置中使用 `replicated_can_become_leader` 来阻止某个副本成为 leader。leader 负责调度后台合并。 - 请注意,只要副本可用并且在 ZK 中有会话,就可以对其执行写入操作,而不论其是否为 leader。 -* `can_become_leader` (`UInt8`) - 该副本是否可以成为 leader。 -* `is_readonly` (`UInt8`) - 该副本是否处于只读模式。 - 如果配置中没有 ClickHouse Keeper 相关的配置段,或者在 ClickHouse Keeper 中重新初始化会话时发生未知错误,或者在 ClickHouse Keeper 中会话正在重新初始化时,该模式会被启用。 -* `is_session_expired` (`UInt8`) - 与 ClickHouse Keeper 的会话是否已过期。基本上与 `is_readonly` 含义相同。 -* `future_parts` (`UInt32`) - 作为尚未完成的 INSERT 或合并操作结果将出现的数据分片数量。 -* `parts_to_check` (`UInt32`) - 等待验证队列中的数据分片数量。如果怀疑某个分片可能已损坏,则会将其放入验证队列。 -* `zookeeper_path` (`String`) - 在 ClickHouse Keeper 中的表数据路径。 -* `replica_name` (`String`) - 在 ClickHouse Keeper 中的副本名称。同一张表的不同副本具有不同的名称。 -* `replica_path` (`String`) - 在 ClickHouse Keeper 中的副本数据路径。等同于拼接得到的路径 `'zookeeper_path/replicas/replica_path'`。 -* `columns_version` (`Int32`) - 表结构的版本号。表示执行 ALTER 的次数。如果副本具有不同的版本,说明某些副本尚未完成所有 ALTER 操作。 -* `queue_size` (`UInt32`) - 等待执行的操作队列大小。操作包括插入数据块、合并以及某些其他操作。通常该值与 `future_parts` 一致。 -* `inserts_in_queue` (`UInt32`) - 需要执行的数据块插入数量。插入通常会被相当快速地复制。如果该值较大,则表示存在问题。 -* `merges_in_queue` (`UInt32`) - 等待执行的合并数量。有时合并会很耗时,因此该值可能长时间大于零。 -* `part_mutations_in_queue` (`UInt32`) - 等待执行的变更(mutation)数量。 -* `queue_oldest_time` (`DateTime`) - 当 `queue_size` 大于 0 时,表示最早的操作被加入队列的时间。 -* `inserts_oldest_time` (`DateTime`) - 参见 `queue_oldest_time` -* `merges_oldest_time` (`DateTime`) - 参见 `queue_oldest_time` -* `part_mutations_oldest_time` (`DateTime`) - 参见 `queue_oldest_time` - -接下来的 4 列只有在与 ZK 存在活动会话时才为非零值。 - -* `log_max_index` (`UInt64`) - 全局活动日志中的最大条目编号。 -* `log_pointer` (`UInt64`) - 副本已复制到其执行队列中的全局活动日志最大条目编号再加一。如果 `log_pointer` 远小于 `log_max_index`,则说明存在问题。 -* `last_queue_update` (`DateTime`) - 队列最后一次更新的时间。 -* `absolute_delay` (`UInt64`) - 当前副本的滞后时长(秒)。 -* `total_replicas` (`UInt8`) - 该表已知副本的总数量。 -* `active_replicas` (`UInt8`) - 该表在 ClickHouse Keeper 中具有会话的副本数量(即正常工作的副本数量)。 -* `lost_part_count` (`UInt64`) - 自建表以来所有副本在该表中丢失的数据分片总数。该值保存在 ClickHouse Keeper 中,只会增加。 -* `last_queue_update_exception` (`String`) - 当队列中包含损坏条目时记录的最后异常消息。当 ClickHouse 在版本之间破坏向后兼容性,导致由新版本写入的日志条目无法被旧版本解析时,该字段尤其重要。 -* `zookeeper_exception` (`String`) - 在从 ClickHouse Keeper 获取信息时发生错误时记录的最后异常消息。 -* `replica_is_active` ([Map(String, UInt8)](../../sql-reference/data-types/map.md)) — 一个从副本名称到该副本是否处于活动状态的映射。 - -如果你查询所有列,访问该表的速度可能会稍微慢一些,因为每一行都需要从 ClickHouse Keeper 进行多次读取。 -如果你不查询最后 4 列(log_max_index、log_pointer、total_replicas、active_replicas),访问该表会更快。 - -例如,你可以像下面这样检查一切是否正常工作: - -```sql -SELECT - database, - table, - is_leader, - is_readonly, - is_session_expired, - future_parts, - parts_to_check, - columns_version, - queue_size, - inserts_in_queue, - merges_in_queue, - log_max_index, - log_pointer, - total_replicas, - active_replicas -FROM system.replicas -WHERE - is_readonly - OR is_session_expired - OR future_parts > 20 - OR parts_to_check > 10 - OR queue_size > 20 - OR inserts_in_queue > 10 - OR log_max_index - log_pointer > 10 - OR total_replicas < 2 - OR active_replicas < total_replicas -``` - -如果此查询未返回任何结果,就表示一切正常。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md deleted file mode 100644 index cdca6a8d84e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replicated_fetches.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '包含当前正在运行的后台数据拉取信息的系统表。' -keywords: ['system table', 'replicated_fetches'] -slug: /operations/system-tables/replicated_fetches -title: 'system.replicated_fetches' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.replicated_fetches {#systemreplicated_fetches} - - - -包含当前正在运行的后台拉取任务的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.replicated_fetches LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: default -table: t -elapsed: 7.243039876 -progress: 0.41832135995612835 -result_part_name: all_0_0_0 -result_part_path: /var/lib/clickhouse/store/700/70080a04-b2de-4adf-9fa5-9ea210e81766/all_0_0_0/ -partition_id: all -total_size_bytes_compressed: 1052783726 -bytes_read_compressed: 440401920 -source_replica_path: /clickhouse/test/t/replicas/1 -source_replica_hostname: node1 -source_replica_port: 9009 -interserver_scheme: http -URI: http://node1:9009/?endpoint=DataPartsExchange%3A%2Fclickhouse%2Ftest%2Ft%2Freplicas%2F1&part=all_0_0_0&client_protocol_version=4&compress=false -to_detached: 0 -thread_id: 54 -``` - -**另请参阅** - -* [管理 ReplicatedMergeTree 表](../../sql-reference/statements/system.md/#managing-replicatedmergetree-tables) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md deleted file mode 100644 index e23913b2500..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/replication_queue.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -description: '包含存储在 ClickHouse Keeper 或 ZooKeeper 中、针对 `ReplicatedMergeTree` - 系列表的复制队列中任务信息的系统表。' -keywords: ['system table', 'replication_queue'] -slug: /operations/system-tables/replication_queue -title: 'system.replication_queue' -doc_type: 'reference' ---- - -# system.replication_queue {#systemreplication_queue} - -包含关于存储在 ClickHouse Keeper 或 ZooKeeper 中、属于 `ReplicatedMergeTree` 系列表的复制队列任务的信息。 - -Columns: - -* `database` ([String](../../sql-reference/data-types/string.md)) — 数据库名称。 - -* `table` ([String](../../sql-reference/data-types/string.md)) — 表名。 - -* `replica_name` ([String](../../sql-reference/data-types/string.md)) — ClickHouse Keeper 中的副本名称。同一张表的不同副本具有不同的名称。 - -* `position` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 任务在队列中的位置。 - -* `node_name` ([String](../../sql-reference/data-types/string.md)) — ClickHouse Keeper 中的节点名称。 - -* `type` ([String](../../sql-reference/data-types/string.md)) — 队列中任务的类型,可能为以下之一: - - * `GET_PART` — 从另一个副本获取分片。 - * `ATTACH_PART` — 附加分片,可能来自本副本(如果在 `detached` 目录中找到)。可以将其视为带有一些优化的 `GET_PART`,因为它们几乎相同。 - * `MERGE_PARTS` — 合并分片。 - * `DROP_RANGE` — 删除指定分区中指定编号范围内的分片。 - * `CLEAR_COLUMN` — 注意:已弃用。从指定分区中删除特定列。 - * `CLEAR_INDEX` — 注意:已弃用。从指定分区中删除特定索引。 - * `REPLACE_RANGE` — 删除某一范围的分片并用新的分片替换。 - * `MUTATE_PART` — 对分片应用一个或多个变更。 - * `ALTER_METADATA` — 根据全局 /metadata 和 /columns 路径应用 ALTER 修改。 - -* `create_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 任务被提交执行的日期和时间。 - -* `required_quorum` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 等待任务完成并确认完成的副本数量。此列仅对 `GET_PARTS` 任务有意义。 - -* `source_replica` ([String](../../sql-reference/data-types/string.md)) — 源副本的名称。 - -* `new_part_name` ([String](../../sql-reference/data-types/string.md)) — 新分片的名称。 - -* `parts_to_merge` ([Array](../../sql-reference/data-types/array.md) ([String](../../sql-reference/data-types/string.md))) — 要合并或更新的分片名称。 - -* `is_detach` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 标志,指示队列中是否存在 `DETACH_PARTS` 任务。 - -* `is_currently_executing` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 标志,指示特定任务当前是否正在执行。 - -* `num_tries` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 为完成任务而失败的尝试次数。 - -* `last_exception` ([String](../../sql-reference/data-types/string.md)) — 最近一次发生错误的文本消息(如果有)。 - -* `last_attempt_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 上一次尝试执行该任务的日期和时间。 - -* `num_postponed` ([UInt32](../../sql-reference/data-types/int-uint.md)) — 该操作被推迟的次数。 - -* `postpone_reason` ([String](../../sql-reference/data-types/string.md)) — 任务被推迟的原因。 - -* `last_postpone_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 任务最近一次被推迟的日期和时间。 - -* `merge_type` ([String](../../sql-reference/data-types/string.md)) — 当前合并的类型。如果是变更操作则为空。 - -**Example** - -```sql -SELECT * FROM system.replication_queue LIMIT 1 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: merge -table: visits_v2 -replica_name: mtgiga001-1t -position: 15 -node_name: queue-0009325559 -type: MERGE_PARTS -create_time: 2020-12-07 14:04:21 -required_quorum: 0 -source_replica: mtgiga001-1t -new_part_name: 20201130_121373_121384_2 -parts_to_merge: ['20201130_121373_121378_1','20201130_121379_121379_0','20201130_121380_121380_0','20201130_121381_121381_0','20201130_121382_121382_0','20201130_121383_121383_0','20201130_121384_121384_0'] -is_detach: 0 -is_currently_executing: 0 -num_tries: 36 -last_exception: Code: 226, e.displayText() = DB::Exception: Marks file '/opt/clickhouse/data/merge/visits_v2/tmp_fetch_20201130_121373_121384_2/CounterID.mrk' does not exist (version 20.8.7.15 (official build)) -last_attempt_time: 2020-12-08 17:35:54 -num_postponed: 0 -postpone_reason: -last_postpone_time: 1970-01-01 03:00:00 -``` - -**另请参阅** - -* [管理 ReplicatedMergeTree 表](/sql-reference/statements/system#managing-replicatedmergetree-tables) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md deleted file mode 100644 index 0a080ad4989..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/resources.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: '包含本地服务器上各资源信息的系统表,每个资源对应一行。' -keywords: ['system table', 'resources'] -slug: /operations/system-tables/resources -title: 'system.resources' -doc_type: 'reference' ---- - -# system.resources {#systemresources} - -包含本地服务器上[资源](/operations/workload-scheduling.md#workload_entity_storage)的信息。该表中每个资源对应一行记录。 - -示例: - -```sql -SELECT * -FROM system.resources -FORMAT Vertical -``` - -```text -行 1: -────── -name: io_read -read_disks: ['s3'] -write_disks: [] -create_query: CREATE RESOURCE io_read (READ DISK s3) - -行 2: -────── -name: io_write -read_disks: [] -write_disks: ['s3'] -create_query: CREATE RESOURCE io_write (WRITE DISK s3) -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md deleted file mode 100644 index 10a7b76e797..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/role_grants.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -description: '包含用户与角色授权关系的系统表。' -keywords: ['system table', 'role_grants'] -slug: /operations/system-tables/role_grants -title: 'system.role_grants' -doc_type: 'reference' ---- - -# system.role_grants {#systemrole_grants} - -包含用户和角色的角色授权信息。要向此表添加记录,请使用 `GRANT role TO user`。 - -列: - -* `user_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 用户名。 - -* `role_name` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 角色名。 - -* `granted_role_name` ([String](../../sql-reference/data-types/string.md)) — 授予给 `role_name` 角色的角色名称。要将一个角色授予另一个角色,请使用 `GRANT role1 TO role2`。 - -* `granted_role_is_default` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志位,用于指示 `granted_role` 是否为默认角色。可能的值: - * 1 — `granted_role` 是默认角色。 - * 0 — `granted_role` 不是默认角色。 - -* `with_admin_option` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 标志位,用于指示 `granted_role` 是否为带有 [ADMIN OPTION](/sql-reference/statements/grant#admin-option) 权限的角色。可能的值: - * 1 — 该角色具有 `ADMIN OPTION` 权限。 - * 0 — 该角色不具有 `ADMIN OPTION` 权限。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md deleted file mode 100644 index b8272e1bed0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/roles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '包含已配置角色相关信息的系统表。' -keywords: ['系统表', '角色'] -slug: /operations/system-tables/roles -title: 'system.roles' -doc_type: 'reference' ---- - -# system.roles {#systemroles} - -包含已配置[角色](../../guides/sre/user-management/index.md#role-management)的信息。 - -列包括: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW ROLES](/sql-reference/statements/show#show-roles) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md deleted file mode 100644 index 8490bd17c38..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/row_policies.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '包含针对特定表的过滤条件,以及应使用该行级策略的角色和/或用户列表的系统表。' -keywords: ['系统表', 'row_policies'] -slug: /operations/system-tables/row_policies -title: 'system.row_policies' -doc_type: 'reference' ---- - -# system.row_policies {#systemrow_policies} - -包含针对某个特定表的过滤器,以及应使用此行策略的角色和/或用户列表。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW POLICIES](/sql-reference/statements/show#show-policies) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md deleted file mode 100644 index 50ee7d69976..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/s3_queue_settings.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -description: '包含 S3Queue 表设置信息的系统表。自服务器版本 `24.10` 起可用。' -keywords: ['system table', 's3_queue_settings'] -slug: /operations/system-tables/s3_queue_settings -title: 'system.s3_queue_settings' -doc_type: 'reference' ---- - -# system.s3_queue_settings {#systems3_queue_settings} - -包含 [S3Queue](../../engines/table-engines/integrations/s3queue.md) 表设置的信息。自服务器版本 `24.10` 起可用。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md deleted file mode 100644 index 659c94d5302..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/scheduler.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '包含本地服务器上调度节点的信息和状态的系统表。' -keywords: ['系统表', '调度器'] -slug: /operations/system-tables/scheduler -title: 'system.scheduler' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.scheduler {#systemscheduler} - - - -包含位于本地服务器上的[调度节点](/operations/workload-scheduling.md/#hierarchy)的相关信息和状态。 -此表可用于监控。表中每个调度节点对应一行。 - -示例: - -```sql -SELECT * -FROM system.scheduler -WHERE resource = 'network_read' AND path = '/prio/fair/prod' -FORMAT Vertical -``` - -```text -Row 1: -────── -resource: network_read -path: /prio/fair/prod -type: fifo -weight: 5 -priority: 0 -is_active: 0 -active_children: 0 -dequeued_requests: 67 -canceled_requests: 0 -dequeued_cost: 4692272 -canceled_cost: 0 -busy_periods: 63 -vruntime: 938454.1999999989 -system_vruntime: ᴺᵁᴸᴸ -queue_length: 0 -queue_cost: 0 -budget: -60524 -is_satisfied: ᴺᵁᴸᴸ -inflight_requests: ᴺᵁᴸᴸ -inflight_cost: ᴺᵁᴸᴸ -max_requests: ᴺᵁᴸᴸ -max_cost: ᴺᵁᴸᴸ -max_speed: ᴺᵁᴸᴸ -max_burst: ᴺᵁᴸᴸ -throttling_us: ᴺᵁᴸᴸ -tokens: ᴺᵁᴸᴸ -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md deleted file mode 100644 index e82170b5275..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/schema_inference_cache.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '包含所有已缓存文件模式信息的系统表。' -keywords: ['system table', 'schema_inference_cache'] -slug: /operations/system-tables/schema_inference_cache -title: 'system.schema_inference_cache' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.schema_inference_cache {#systemschema_inference_cache} - - - -包含所有已缓存文件 schema 的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -假设我们有一个名为 `data.jsonl` 的文件,其内容如下: - -```json -{"id" : 1, "age" : 25, "name" : "Josh", "hobbies" : ["足球", "烹饪", "音乐"]} -{"id" : 2, "age" : 19, "name" : "Alan", "hobbies" : ["网球", "艺术"]} -{"id" : 3, "age" : 32, "name" : "Lana", "hobbies" : ["健身", "阅读", "购物"]} -{"id" : 4, "age" : 47, "name" : "Brayan", "hobbies" : ["电影", "跳伞"]} -``` - -:::tip -将 `data.jsonl` 放在 `user_files_path` 目录中。你可以通过查看 -ClickHouse 的配置文件来找到该目录。默认值为: - -```sql -/var/lib/clickhouse/user_files/ -``` - -::: - -打开 `clickhouse-client` 并执行 `DESCRIBE` 查询: - -```sql -DESCRIBE file('data.jsonl') SETTINGS input_format_try_infer_integers=0; -``` - -```response -┌─name────┬─type────────────────────┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┬─ttl_expression─┐ -│ id │ Nullable(Float64) │ │ │ │ │ │ -│ age │ Nullable(Float64) │ │ │ │ │ │ -│ name │ Nullable(String) │ │ │ │ │ │ -│ hobbies │ Array(Nullable(String)) │ │ │ │ │ │ -└─────────┴─────────────────────────┴──────────────┴────────────────────┴─────────┴──────────────────┴────────────────┘ -``` - -我们来看一下 `system.schema_inference_cache` 表的内容: - -```sql -SELECT * -FROM system.schema_inference_cache -FORMAT Vertical -``` - -```response -第 1 行: -────── -storage: File -source: /home/droscigno/user_files/data.jsonl -format: JSONEachRow -additional_format_info: schema_inference_hints=, max_rows_to_read_for_schema_inference=25000, schema_inference_make_columns_nullable=true, try_infer_integers=false, try_infer_dates=true, try_infer_datetimes=true, try_infer_numbers_from_strings=true, read_bools_as_numbers=true, try_infer_objects=false -registration_time: 2022-12-29 17:49:52 -schema: id Nullable(Float64), age Nullable(Float64), name Nullable(String), hobbies Array(Nullable(String)) -``` - -**另请参阅** - -* [基于输入数据的自动模式推断](/interfaces/schema-inference.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md deleted file mode 100644 index f84d4c12de5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/server_settings.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '包含服务器全局设置信息的系统表,这些设置在 `config.xml` 中指定。' -keywords: ['系统表', 'server_settings'] -slug: /operations/system-tables/server_settings -title: 'system.server_settings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.server_settings {#systemserver_settings} - - - -包含在 `config.xml` 中指定的服务器全局设置的信息。 -当前,该表仅显示 `config.xml` 顶层中的设置,不支持嵌套配置(例如 [logger](../../operations/server-configuration-parameters/settings.md#logger))。 - -列: - -* `name` ([String](../../sql-reference/data-types/string.md)) — 服务器设置名称。 -* `value` ([String](../../sql-reference/data-types/string.md)) — 服务器设置的值。 -* `default` ([String](../../sql-reference/data-types/string.md)) — 服务器设置的默认值。 -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 指示某个设置是否在 `config.xml` 中被显式指定。 -* `description` ([String](../../sql-reference/data-types/string.md)) — 服务器设置的简要说明。 -* `type` ([String](../../sql-reference/data-types/string.md)) — 服务器设置值的类型。 -* `changeable_without_restart` ([Enum8](../../sql-reference/data-types/enum.md)) — 指示该设置是否可以在服务器运行时更改。取值: - * `'No' ` - * `'IncreaseOnly'` - * `'DecreaseOnly'` - * `'Yes'` -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) - 指示某个设置是否已废弃。 - -**示例** - -下面的示例展示了如何获取名称中包含 `thread_pool` 的服务器设置相关信息。 - -```sql -SELECT * -FROM system.server_settings -WHERE name LIKE '%thread_pool%' -``` - -```text -┌─name──────────────────────────────────────────┬─value─┬─default─┬─changed─┬─description─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─type───┬─changeable_without_restart─┬─is_obsolete─┐ -│ max_thread_pool_size │ 10000 │ 10000 │ 0 │ 可从操作系统分配用于查询执行和后台操作的最大线程数。 │ UInt64 │ No │ 0 │ -│ max_thread_pool_free_size │ 1000 │ 1000 │ 0 │ 分配后始终保留在全局线程池中的最大线程数,当任务数量不足时保持空闲。 │ UInt64 │ No │ 0 │ -│ thread_pool_queue_size │ 10000 │ 10000 │ 0 │ 可放置在队列中等待执行的最大任务数。 │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_size │ 100 │ 100 │ 0 │ 用于 IO 操作的最大线程数 │ UInt64 │ No │ 0 │ -│ max_io_thread_pool_free_size │ 0 │ 0 │ 0 │ IO 线程池的最大空闲容量。 │ UInt64 │ No │ 0 │ -│ io_thread_pool_queue_size │ 10000 │ 10000 │ 0 │ IO 线程池的队列大小。 │ UInt64 │ No │ 0 │ -│ max_active_parts_loading_thread_pool_size │ 64 │ 64 │ 0 │ 启动时用于加载活动数据部分集合(活动部分)的线程数。 │ UInt64 │ No │ 0 │ -│ max_outdated_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ 启动时用于加载非活动数据部分集合(过时部分)的线程数。 │ UInt64 │ No │ 0 │ -│ max_unexpected_parts_loading_thread_pool_size │ 32 │ 32 │ 0 │ 启动时用于加载非活动数据部分集合(意外部分)的线程数。 │ UInt64 │ No │ 0 │ -│ max_parts_cleaning_thread_pool_size │ 128 │ 128 │ 0 │ 用于并发删除非活动数据部分的线程数。 │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_size │ 1000 │ 1000 │ 0 │ 用于 BACKUP 查询 IO 操作的最大线程数 │ UInt64 │ No │ 0 │ -│ max_backups_io_thread_pool_free_size │ 0 │ 0 │ 0 │ 备份 IO 线程池的最大空闲容量。 │ UInt64 │ No │ 0 │ -│ backups_io_thread_pool_queue_size │ 0 │ 0 │ 0 │ 备份 IO 线程池的队列大小。 │ UInt64 │ No │ 0 │ -└───────────────────────────────────────────────┴───────┴─────────┴─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴────────┴────────────────────────────┴─────────────┘ - -``` - -使用 `WHERE changed` 在某些场景下会很有用,例如当你想检查配置文件中的设置是否已正确加载并实际生效时。 - -{/* */ } - -```sql -SELECT * FROM system.server_settings WHERE changed AND name='max_thread_pool_size' -``` - -**另请参见** - -* [设置](../../operations/system-tables/settings.md) -* [配置文件](../../operations/configuration-files.md) -* [服务器设置](../../operations/server-configuration-parameters/settings.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md deleted file mode 100644 index eac707ed3b6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/session_log.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: '记录所有登录和登出成功与失败事件信息的系统表。' -keywords: ['system table', 'session_log'] -slug: /operations/system-tables/session_log -title: 'system.session_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.session_log {#systemsession_log} - - - -包含所有成功和失败的登录和登出事件的信息。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) — 登录/登出结果。可能的值: - * `LoginFailure` — 登录错误。 - * `LoginSuccess` — 登录成功。 - * `Logout` — 从系统登出。 -* `auth_id` ([UUID](../../sql-reference/data-types/uuid.md)) — 身份验证 ID,每次用户登录时都会自动生成的 UUID。 -* `session_id` ([String](../../sql-reference/data-types/string.md)) — 由客户端通过 [HTTP](../../interfaces/http.md) 接口传递的会话 ID。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 登录/登出日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 登录/登出时间。 -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒精度的登录/登出起始时间。 -* `user` ([String](../../sql-reference/data-types/string.md)) — 用户名。 -* `auth_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 身份验证类型。可能的值: - * `NO_PASSWORD` - * `PLAINTEXT_PASSWORD` - * `SHA256_PASSWORD` - * `DOUBLE_SHA1_PASSWORD` - * `LDAP` - * `KERBEROS` - * `SSL_CERTIFICATE` -* `profiles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 为所有角色和/或用户设置的配置文件列表。 -* `roles` ([Array](../../sql-reference/data-types/array.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md))) — 应用了该配置文件的角色列表。 -* `settings` ([Array](../../sql-reference/data-types/array.md)([Tuple](../../sql-reference/data-types/tuple.md)([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md), [String](../../sql-reference/data-types/string.md)))) — 客户端登录/登出时更改的设置。 -* `client_address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 用于登录/登出的 IP 地址。 -* `client_port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 用于登录/登出的客户端端口。 -* `interface` ([Enum8](../../sql-reference/data-types/enum.md)) — 发起登录的接口。可能的值: - * `TCP` - * `HTTP` - * `gRPC` - * `MySQL` - * `PostgreSQL` -* `client_hostname` ([String](../../sql-reference/data-types/string.md)) — 运行 [clickhouse-client](../../interfaces/cli.md) 或其他 TCP 客户端的客户端机器主机名。 -* `client_name` ([String](../../sql-reference/data-types/string.md)) — `clickhouse-client` 或其他 TCP 客户端名称。 -* `client_revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` 或其他 TCP 客户端的修订版本号。 -* `client_version_major` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` 或其他 TCP 客户端的主版本号。 -* `client_version_minor` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` 或其他 TCP 客户端的次版本号。 -* `client_version_patch` ([UInt32](../../sql-reference/data-types/int-uint.md)) — `clickhouse-client` 或其他 TCP 客户端版本的补丁号。 -* `failure_reason` ([String](../../sql-reference/data-types/string.md)) — 包含登录/登出失败原因的异常信息。 - -**示例** - -查询: - -```sql -SELECT * FROM system.session_log LIMIT 1 FORMAT Vertical; -``` - -结果: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: LoginSuccess -auth_id: 45e6bd83-b4aa-4a23-85e6-bd83b4aa1a23 -session_id: -event_date: 2021-10-14 -event_time: 2021-10-14 20:33:52 -event_time_microseconds: 2021-10-14 20:33:52.104247 -user: default -auth_type: PLAINTEXT_PASSWORD -profiles: ['default'] -roles: [] -settings: [('load_balancing','random'),('max_memory_usage','10000000000')] -client_address: ::ffff:127.0.0.1 -client_port: 38490 -interface: TCP -client_hostname: -client_name: ClickHouse client -client_revision: 54449 -client_version_major: 21 -client_version_minor: 10 -client_version_patch: 0 -failure_reason: -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md deleted file mode 100644 index f52c0b79dfe..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -description: '包含当前用户会话设置信息的系统表。' -keywords: ['系统表', '设置'] -slug: /operations/system-tables/settings -title: 'system.settings' -doc_type: 'reference' ---- - -# system.settings {#systemsettings} - -包含当前用户会话参数设置信息。 - -列: - -* `name` ([String](../../sql-reference/data-types/string.md)) — 设置名称。 -* `value` ([String](../../sql-reference/data-types/string.md)) — 设置值。 -* `changed` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 指示该设置是否在配置中被显式定义或被显式更改。 -* `description` ([String](../../sql-reference/data-types/string.md)) — 设置的简要说明。 -* `min` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 通过[约束](/operations/settings/constraints-on-settings)为该设置指定的最小值(如果有)。如果设置没有最小值,则为 [NULL](/operations/settings/formats#input_format_null_as_default)。 -* `max` ([Nullable](../../sql-reference/data-types/nullable.md)([String](../../sql-reference/data-types/string.md))) — 通过[约束](/operations/settings/constraints-on-settings)为该设置指定的最大值(如果有)。如果设置没有最大值,则为 [NULL](/operations/settings/formats#input_format_null_as_default)。 -* `disallowed_values` ([Array](/sql-reference/data-types/array)([String](../../sql-reference/data-types/string.md))) — 不允许的取值列表。 -* `readonly` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) — 指示当前用户是否可以更改该设置: - * `0` — 当前用户可以更改该设置。 - * `1` — 当前用户不能更改该设置。 -* `default` ([String](../../sql-reference/data-types/string.md)) — 设置的默认值。 -* `alias_for` ([String](../../sql-reference/data-types/string.md)) — 如果该设置是其他设置的别名,则为原始设置的名称。 -* `is_obsolete` ([UInt8](/sql-reference/data-types/int-uint#integer-ranges)) - 指示该设置是否已废弃。 -* `tier` ([Enum8](../../sql-reference/data-types/enum.md)) — 此功能的支持级别。ClickHouse 的功能按层次组织,这些层次会根据当前的开发状态以及用户在使用它们时可以预期的行为而变化。取值: - * `'Production'` — 功能稳定、安全可用,并且与其他**生产级**功能交互时不存在问题。 - * `'Beta'` — 功能稳定且安全。但与其他功能一起使用时的结果未知,不保证正确性。欢迎测试和反馈。 - * `'Experimental'` — 功能仍在开发中。仅面向开发人员和 ClickHouse 爱好者。该功能可能有效也可能无效,并且可能在任何时候被移除。 - * `'Obsolete'` — 不再受支持。要么已经被移除,要么将在未来版本中移除。 - -**示例** - -下面的示例展示如何获取名称包含 `min_i` 的设置的信息。 - -```sql -SELECT * -FROM system.settings -WHERE name LIKE '%min_insert_block_size_%' -FORMAT Vertical -``` - -```text -Row 1: -────── -name: min_insert_block_size_rows -value: 1048449 -changed: 0 -description: 设置通过 `INSERT` 查询插入表中的数据块的最小行数。较小的数据块将被合并成较大的数据块。 - -可能的值: - -- 正整数。 -- 0 — 禁用合并。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 1048449 -alias_for: -is_obsolete: 0 -tier: 生产环境 - -Row 2: -────── -name: min_insert_block_size_bytes -value: 268402944 -changed: 0 -description: 设置通过 `INSERT` 查询插入表中的数据块的最小字节数。较小的数据块将被合并成较大的数据块。 - -可能的值: - -- 正整数。 -- 0 — 禁用合并。 -min: ᴺᵁᴸᴸ -max: ᴺᵁᴸᴸ -readonly: 0 -type: UInt64 -default: 268402944 -alias_for: -is_obsolete: 0 -tier: 生产环境 -``` - -Row 3: -────── -name: min_insert_block_size_rows_for_materialized_views -value: 0 -changed: 0 -description: 设置可通过 `INSERT` 查询插入到表中的数据块的最小行数。更小的数据块会被合并成更大的数据块。此设置仅对插入到[物化视图](../../sql-reference/statements/create/view.md)中的数据块生效。通过调整此设置,可以在向物化视图写入时控制数据块合并行为,并避免过度的内存使用。 - -Possible values: - -* 任意正整数。 -* 0 — 禁用合并。 - -**See Also** - -* [min_insert_block_size_rows](/operations/settings/settings#min_insert_block_size_rows) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -Row 4: -────── -name: min_insert_block_size_bytes_for_materialized_views -value: 0 -changed: 0 -description: 设置可通过 `INSERT` 查询插入到表中的数据块的最小字节数。更小的数据块会被合并成更大的数据块。此设置仅对插入到[物化视图](../../sql-reference/statements/create/view.md)中的数据块生效。通过调整此设置,可以在向物化视图写入时控制数据块合并行为,并避免过度的内存使用。 - -Possible values: - -* 任意正整数。 -* 0 — 禁用合并。 - -**See also** - -* [min_insert_block_size_bytes](/operations/settings/settings#min_insert_block_size_bytes) - min: ᴺᵁᴸᴸ - max: ᴺᵁᴸᴸ - readonly: 0 - type: UInt64 - default: 0 - alias_for:\ - is_obsolete: 0 - tier: Production - -```` - -使用 `WHERE changed` 在以下场景中很有用,例如需要检查: - -- 配置文件中的设置是否已正确加载并生效。 -- 当前会话中已变更的设置。 - - - -```sql -SELECT * FROM system.settings WHERE changed AND name='load_balancing' -```` - -**另请参阅** - -* [设置](/operations/system-tables/overview#system-tables-introduction) -* [查询权限](/operations/settings/permissions-for-queries) -* [设置限制](../../operations/settings/constraints-on-settings.md) -* [SHOW SETTINGS](../../sql-reference/statements/show.md#show-settings) 语句 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md deleted file mode 100644 index 71ac3eebeb3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_changes.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: '用于记录历史 ClickHouse 版本中设置变更信息的系统表。' -keywords: ['system table', 'settings_changes'] -slug: /operations/system-tables/settings_changes -title: 'system.settings_changes' -doc_type: 'reference' ---- - -# system.settings_changes {#systemsettings_changes} - -包含 ClickHouse 历史版本中设置变更的信息。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * -FROM system.settings_changes -WHERE version = '23.5' -FORMAT Vertical -``` - -```text -Row 1: -────── -type: Core -version: 23.5 -changes: [('input_format_parquet_preserve_order','1','0','允许 Parquet 读取器对行重新排序以提升并行处理性能。'),('parallelize_output_from_storages','0','1','允许在执行从 file/url/S3 等读取数据的查询时进行并行处理。这可能导致行重新排序。'),('use_with_fill_by_sorting_prefix','0','1','ORDER BY 子句中 WITH FILL 列之前的列构成排序前缀。排序前缀值不同的行将独立填充'),('output_format_parquet_compliant_nested_types','0','1','更改输出 Parquet 文件架构中的内部字段名称。')] -``` - -**另请参阅** - -* [设置](/operations/system-tables/overview#system-tables-introduction) -* [system.settings](settings.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md deleted file mode 100644 index 51e7382c12e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profile_elements.md +++ /dev/null @@ -1,21 +0,0 @@ ---- -description: '用于描述 SETTINGS PROFILE 内容的系统表:该设置所适用的约束、角色和用户,以及父 SETTINGS PROFILE。' -keywords: ['system table', 'settings_profile_elements'] -slug: /operations/system-tables/settings_profile_elements -title: 'system.settings_profile_elements' -doc_type: 'reference' ---- - -# system.settings_profile_elements {#systemsettings_profile_elements} - -描述设置配置文件的内容: - -* 约束。 -* 该设置适用的角色和用户。 -* 父级设置配置文件。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md deleted file mode 100644 index fadb0ead228..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/settings_profiles.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '包含已配置设置配置集属性的系统表。' -keywords: ['system table', 'settings_profiles'] -slug: /operations/system-tables/settings_profiles -title: 'system.settings_profiles' -doc_type: 'reference' ---- - -# system.settings_profiles {#systemsettings_profiles} - -包含已配置的设置配置文件的属性。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW PROFILES](/sql-reference/statements/show#show-profiles) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md deleted file mode 100644 index d0731420b88..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/stack_trace.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -description: '包含所有服务器线程堆栈跟踪的系统表。允许开发人员检查服务器状态。' -keywords: ['system table', 'stack_trace'] -slug: /operations/system-tables/stack_trace -title: 'system.stack_trace' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.stack_trace {#systemstack_trace} - - - -包含所有服务器线程的堆栈跟踪。用于帮助开发人员检查服务器状态。 - -要分析栈帧,请使用 `addressToLine`、`addressToLineWithInlines`、`addressToSymbol` 和 `demangle` [自省函数](../../sql-reference/functions/introspection.md)。 - -列: - -* `thread_name` ([String](../../sql-reference/data-types/string.md)) — 线程名称。 -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 线程标识符。 -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询标识符,可用于从 [query_log](../system-tables/query_log.md) 系统表中获取当时正在运行查询的详细信息。 -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 一个[堆栈跟踪](https://en.wikipedia.org/wiki/Stack_trace),表示被调用方法所在的物理地址列表。 - -:::tip -在知识库(Knowledge Base)中可以找到一些实用查询,包括[如何查看当前正在运行的线程](/knowledgebase/find-expensive-queries)以及[用于故障排查的常用查询](/knowledgebase/useful-queries-for-troubleshooting)。 -::: - -**示例** - -启用自省函数: - -```sql -SET allow_introspection_functions = 1; -``` - -从 ClickHouse 目标文件中获取符号信息: - -```sql -WITH arrayMap(x -> demangle(addressToSymbol(x)), trace) AS all SELECT thread_name, thread_id, query_id, arrayStringConcat(all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Row 1: -────── -thread_name: QueryPipelineEx -thread_id: 743490 -query_id: dc55a564-febb-4e37-95bb-090ef182c6f1 -res: memcpy -large_ralloc -arena_ralloc -do_rallocx -Allocator::realloc(void*, unsigned long, unsigned long, unsigned long) -HashTable, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>::resize(unsigned long, unsigned long) -void DB::Aggregator::executeImplBatch, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>>(DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>&, DB::AggregationMethodOneNumber, HashTableNoState, PairNoInit>, HashCRC32, HashTableGrowerWithPrecalculation<8ul>, Allocator>, true, false>::State&, DB::Arena*, unsigned long, unsigned long, DB::Aggregator::AggregateFunctionInstruction*, bool, char*) const -DB::Aggregator::executeImpl(DB::AggregatedDataVariants&, unsigned long, unsigned long, std::__1::vector>&, DB::Aggregator::AggregateFunctionInstruction*, bool, bool, char*) const -DB::Aggregator::executeOnBlock(std::__1::vector::immutable_ptr, std::__1::allocator::immutable_ptr>>, unsigned long, unsigned long, DB::AggregatedDataVariants&, std::__1::vector>&, std::__1::vector>, std::__1::allocator>>>&, bool&) const -DB::AggregatingTransform::work() -DB::ExecutionThreadContext::executeTask() -DB::PipelineExecutor::executeStepImpl(unsigned long, std::__1::atomic*) -void std::__1::__function::__policy_invoker::__call_impl>(std::__1::__function::__policy_storage const*) -ThreadPoolImpl>::worker(std::__1::__list_iterator, void*>) -void std::__1::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__1::__function::__policy_storage const*) -void* std::__1::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::__1::function, Priority, std::__1::optional, bool)::'lambda0'()>>(void*) -``` - -在 ClickHouse 源代码中获取文件名和行号: - -```sql -WITH arrayMap(x -> addressToLine(x), trace) AS all, arrayFilter(x -> x LIKE '%/dbms/%', all) AS dbms SELECT thread_name, thread_id, query_id, arrayStringConcat(notEmpty(dbms) ? dbms : all, '\n') AS res FROM system.stack_trace LIMIT 1 \G; -``` - -```text -Row 1: -────── -thread_name: clickhouse-serv - -thread_id: 686 -query_id: cad353e7-1c29-4b2e-949f-93e597ab7a54 -res: /lib/x86_64-linux-gnu/libc-2.27.so -/build/obj-x86_64-linux-gnu/../src/Storages/System/StorageSystemStackTrace.cpp:182 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/vector:656 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:1338 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectQuery.cpp:751 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/optional:224 -/build/obj-x86_64-linux-gnu/../src/Interpreters/InterpreterSelectWithUnionQuery.cpp:192 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:384 -/build/obj-x86_64-linux-gnu/../src/Interpreters/executeQuery.cpp:643 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:251 -/build/obj-x86_64-linux-gnu/../src/Server/TCPHandler.cpp:1197 -/build/obj-x86_64-linux-gnu/../contrib/poco/Net/src/TCPServerConnection.cpp:57 -/build/obj-x86_64-linux-gnu/../contrib/libcxx/include/atomic:856 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/Mutex_POSIX.h:59 -/build/obj-x86_64-linux-gnu/../contrib/poco/Foundation/include/Poco/AutoPtr.h:223 -/lib/x86_64-linux-gnu/libpthread-2.27.so -/lib/x86_64-linux-gnu/libc-2.27.so -``` - -**另请参阅** - -* [Introspection Functions](../../sql-reference/functions/introspection.md) — 可用的自省函数列表及其用法。 -* [system.trace_log](../system-tables/trace_log.md) — 包含由采样查询分析器收集的堆栈跟踪信息。 -* [arrayMap](/sql-reference/functions/array-functions#arrayMap) — `arrayMap` 函数的说明及使用示例。 -* [arrayFilter](/sql-reference/functions/array-functions#arrayFilter) — `arrayFilter` 函数的说明及使用示例。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md deleted file mode 100644 index dd19ed669bc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/storage_policies.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '包含在服务器配置中定义的存储策略和卷信息的系统表。' -keywords: ['系统表', 'storage_policies'] -slug: /operations/system-tables/storage_policies -title: 'system.storage_policies' -doc_type: 'reference' ---- - -# system.storage_policies {#systemstorage_policies} - -包含在[服务器配置](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes_configure)中定义的存储策略和卷的信息。 - -列: - -* `policy_name` ([String](../../sql-reference/data-types/string.md)) — 存储策略名称。 -* `volume_name` ([String](../../sql-reference/data-types/string.md)) — 在存储策略中定义的卷名称。 -* `volume_priority` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 卷在配置中的顺序号,数据会根据此优先级依次填充各卷,即在插入和合并过程中,数据会写入优先级较低的卷(同时会考虑其他规则:TTL、`max_data_part_size`、`move_factor`)。 -* `disks` ([Array(String)](../../sql-reference/data-types/array.md)) — 在存储策略中定义的磁盘名称。 -* `volume_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 卷类型。可以是以下值之一: - * `JBOD` - * `SINGLE_DISK` - * `UNKNOWN` -* `max_data_part_size` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 可以存储在该卷磁盘上的数据分片的最大大小(0 — 无限制)。 -* `move_factor` ([Float64](../../sql-reference/data-types/float.md)) — 空闲磁盘空间的占比。当该占比超过配置参数的值时,ClickHouse 会按顺序开始将数据移动到下一个卷。 -* `prefer_not_to_merge` ([UInt8](../../sql-reference/data-types/int-uint.md)) — `prefer_not_to_merge` 设置的值。应始终为 false。启用该设置说明这是一个错误配置。 -* `perform_ttl_move_on_insert` ([UInt8](../../sql-reference/data-types/int-uint.md)) — `perform_ttl_move_on_insert` 设置的值。用于在数据分片 INSERT 时禁用 TTL 移动。默认情况下,如果我们插入的某个数据分片已经符合 TTL 移动规则(已过期),它会立即被写入在移动规则中声明的卷/磁盘。如果目标卷/磁盘较慢(例如 S3),这可能会显著降低插入速度。 -* `load_balancing` ([Enum8](../../sql-reference/data-types/enum.md)) — 磁盘负载均衡策略。可以是以下值之一: - * `ROUND_ROBIN` - * `LEAST_USED` - -如果存储策略包含多个卷,则关于每个卷的信息会存储在表中的单独一行。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md deleted file mode 100644 index 4e22eb5d7ee..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/symbols.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: '对 C++ 专家和 ClickHouse 工程师有用的 system 表,包含用于对 `clickhouse` 可执行文件进行内省的信息。' -keywords: ['system 表', '符号'] -slug: /operations/system-tables/symbols -title: 'system.symbols' -doc_type: 'reference' ---- - -包含用于对 `clickhouse` 可执行文件进行内省的信息。访问该表需要具有 introspection 权限。 -此表仅适用于 C++ 专家和 ClickHouse 工程师。 - -列: - -* `symbol` ([String](../../sql-reference/data-types/string.md)) — 二进制文件中的符号名称。该名称是经过重整(mangled)的。你可以使用 `demangle(symbol)` 来获得可读名称。 -* `address_begin` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 符号在二进制文件中的起始地址。 -* `address_end` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 符号在二进制文件中的结束地址。 -* `name` ([String](../../sql-reference/data-types/string.md)) — `event` 的别名。 - -**示例** - -```sql -SELECT address_begin, address_end - address_begin AS size, demangle(symbol) FROM system.symbols ORDER BY size DESC LIMIT 10 -``` - -```text -┌─address_begin─┬─────size─┬─demangle(symbol)──────────────────────────────────────────────────────────────────┐ -│ 25000976 │ 29466000 │ icudt70_dat │ -│ 400605288 │ 2097272 │ arena_emap_global │ -│ 18760592 │ 1048576 │ CLD2::kQuadChrome1015_2 │ -│ 9807152 │ 884808 │ TopLevelDomainLookupHash::isValid(char const*, unsigned long)::wordlist │ -│ 57442432 │ 850608 │ llvm::X86Insts │ -│ 55682944 │ 681360 │ (anonymous namespace)::X86DAGToDAGISel::SelectCode(llvm::SDNode*)::MatcherTable │ -│ 55130368 │ 502840 │ (anonymous namespace)::X86InstructionSelector::getMatchTable() const::MatchTable0 │ -│ 402930616 │ 404032 │ qpl::ml::dispatcher::hw_dispatcher::get_instance()::instance │ -│ 274131872 │ 356795 │ DB::SettingsTraits::Accessor::instance()::$_0::operator()() const │ -│ 58293040 │ 249424 │ llvm::X86InstrNameData │ -└───────────────┴──────────┴───────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md deleted file mode 100644 index ca09cbc91a0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/system_warnings.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '此表包含有关 ClickHouse 服务器的警告消息。' -keywords: [ '系统表', '警告' ] -slug: /operations/system-tables/system_warnings -title: 'system.warnings' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.warnings {#systemwarnings} - - - -该表显示 ClickHouse 服务器的警告信息。 -相同类型的警告会合并为一条记录。 -例如,如果已挂载数据库的数量 N 超过可配置的阈值 T,则会显示一条包含当前值 N 的记录,而不是 N 条单独的记录。 -如果当前值降到阈值以下,则会从表中移除该记录。 - -可以通过以下设置配置该表: - -* [max_table_num_to_warn](../server-configuration-parameters/settings.md#max_table_num_to_warn) -* [max_database_num_to_warn](../server-configuration-parameters/settings.md#max_database_num_to_warn) -* [max_dictionary_num_to_warn](../server-configuration-parameters/settings.md#max_dictionary_num_to_warn) -* [max_view_num_to_warn](../server-configuration-parameters/settings.md#max_view_num_to_warn) -* [max_part_num_to_warn](../server-configuration-parameters/settings.md#max_part_num_to_warn) -* [max_pending_mutations_to_warn](../server-configuration-parameters/settings.md#max_pending_mutations_to_warn) -* [max_pending_mutations_execution_time_to_warn](/operations/server-configuration-parameters/settings#max_pending_mutations_execution_time_to_warn) -* [max_named_collection_num_to_warn](../server-configuration-parameters/settings.md#max_named_collection_num_to_warn) -* [resource_overload_warnings](/operations/settings/server-overload#resource-overload-warnings) - -列: - -* `message` ([String](../../sql-reference/data-types/string.md)) — 警告信息。 -* `message_format_string` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 用于格式化该警告信息的格式字符串。 - -**示例** - -查询: - -```sql - SELECT * FROM system.warnings LIMIT 2 \G; -``` - -结果: - -```text -Row 1: -────── -message: 活跃分区数超过 10 个。 -message_format_string: 活跃分区数超过 {}。 - -Row 2: -────── -message: 已挂载数据库数超过 2 个。 -message_format_string: 已挂载数据库数超过 {}。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md deleted file mode 100644 index ce4adc6f9c8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/table_engines.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -description: '包含服务器支持的表引擎及其所支持特性说明的系统表。' -keywords: ['system table', 'table_engines'] -slug: /operations/system-tables/table_engines -title: 'system.table_engines' -doc_type: 'reference' ---- - -# system.table_engines {#systemtable_engines} - -包含服务器支持的表引擎及其功能支持情况的描述。 - -该表包含以下列(括号中为列类型): - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -示例: - -```sql -SELECT * -FROM system.table_engines -WHERE name IN ('Kafka', 'MergeTree', 'ReplicatedCollapsingMergeTree') -``` - -```text -┌─name──────────────────────────┬─supports_settings─┬─supports_skipping_indices─┬─supports_sort_order─┬─supports_ttl─┬─supports_replication─┬─supports_deduplication─┬─supports_parallel_insert─┐ -│ MergeTree │ 1 │ 1 │ 1 │ 1 │ 0 │ 0 │ 1 │ -│ Kafka │ 1 │ 0 │ 0 │ 0 │ 0 │ 0 │ 0 │ -│ ReplicatedCollapsingMergeTree │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ 1 │ -└───────────────────────────────┴───────────────────┴───────────────────────────┴─────────────────────┴──────────────┴──────────────────────┴────────────────────────┴──────────────────────────┘ -``` - -**另请参阅** - -* MergeTree 系列[查询子句](../../engines/table-engines/mergetree-family/mergetree.md#mergetree-query-clauses) -* Kafka [设置](/engines/table-engines/integrations/kafka#creating-a-table) -* Join [设置](../../engines/table-engines/special/join.md#join-limitations-and-settings) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md deleted file mode 100644 index bfdc9fdb06d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/tables.md +++ /dev/null @@ -1,157 +0,0 @@ ---- -description: '包含服务器已知的所有表的元数据的系统表。' -keywords: ['system table', 'tables'] -slug: /operations/system-tables/tables -title: 'system.tables' -doc_type: '参考' ---- - -# system.tables {#systemtables} - -包含服务器已知的每个表的元数据。 - -[已分离](../../sql-reference/statements/detach.md) 的表不会在 `system.tables` 中显示。 - -[临时表](../../sql-reference/statements/create/table.md#temporary-tables) 只有在创建它们的会话中才会在 `system.tables` 中可见。它们会以 `database` 字段为空且 `is_temporary` 标志已开启的形式显示。 - -列: - -* `database` ([String](../../sql-reference/data-types/string.md)) — 表所在数据库的名称。 - -* `name` ([String](../../sql-reference/data-types/string.md)) — 表名。 - -* `uuid` ([UUID](../../sql-reference/data-types/uuid.md)) — 表的 UUID(Atomic 数据库)。 - -* `engine` ([String](../../sql-reference/data-types/string.md)) — 表引擎的名称(不含参数)。 - -* `is_temporary` ([UInt8](../../sql-reference/data-types/int-uint.md)) - 标志位,表示该表是否为临时表。 - -* `data_paths` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 表数据在文件系统中的路径。 - -* `metadata_path` ([String](../../sql-reference/data-types/string.md)) - 表在文件系统中的元数据路径。 - -* `metadata_modification_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - 表的元数据最后修改时间。 - -* `metadata_version` ([Int32](../../sql-reference/data-types/int-uint.md)) - ReplicatedMergeTree 表的元数据版本号;对于非 ReplicatedMergeTree 表,该值为 0。 - -* `dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 数据库依赖项。 - -* `dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 表的依赖关系(依赖于当前表的[物化视图](/sql-reference/statements/create/view#materialized-view))。 - -* `create_table_query` ([String](../../sql-reference/data-types/string.md)) - 用于创建该表的 SQL 查询语句。 - -* `engine_full` ([String](../../sql-reference/data-types/string.md)) - 表引擎参数。 - -* `as_select` ([String](../../sql-reference/data-types/string.md)) - 用于视图的 `SELECT` 查询。 - -* `parameterized_view_parameters` ([Array](../../sql-reference/data-types/array.md) of [Tuple](../../sql-reference/data-types/tuple.md)) — 参数化视图的参数。 - -* `partition_key` ([String](../../sql-reference/data-types/string.md)) - 在表中指定的分区键表达式。 - -* `sorting_key` ([String](../../sql-reference/data-types/string.md)) - 在表中指定的排序键表达式。 - -* `primary_key` ([String](../../sql-reference/data-types/string.md)) - 表中指定的主键表达式。 - -* `sampling_key` ([String](../../sql-reference/data-types/string.md)) - 在表中指定的采样键表达式。 - -* `storage_policy` ([String](../../sql-reference/data-types/string.md)) - 存储策略: - - * [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md#table_engine-mergetree-multiple-volumes) - * [分布式](/engines/table-engines/special/distributed) - -* `total_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 如果能够快速确定表中的精确行数,则为总行数,否则为 `NULL`(包括底层的 `Buffer` 表)。 - -* `total_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 字节总数(包括索引和投影)。如果可以快速确定存储中该表的精确字节数,则返回该值,否则为 `NULL`(不包括任何底层存储)。 - - * 如果表将数据存储在磁盘上,则返回磁盘上已用空间(即压缩后大小)。 - * 如果表将数据存储在内存中,则返回已使用内存字节数的近似值。 - -* `total_bytes_uncompressed` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 未压缩字节的总数(包括索引和投影)。如果可以根据存储中该表各个分片的校验和快速精确地计算字节数,则返回该值;否则为 `NULL`(不考虑底层存储(如果有))。 - -* `lifetime_rows` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 自服务器启动以来 INSERT 的总行数(仅用于 `Buffer` 表)。 - -* `lifetime_bytes` ([Nullable](../../sql-reference/data-types/nullable.md)([UInt64](../../sql-reference/data-types/int-uint.md))) - 自服务器启动以来已 INSERT 的总字节数(仅适用于 `Buffer` 表)。 - -* `comment` ([String](../../sql-reference/data-types/string.md)) - 表的注释。 - -* `has_own_data` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 标志用于指示该表本身是否在磁盘上存储一定数据,或者仅访问其他数据源。 - -* `loading_dependencies_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 数据库加载依赖项(在当前对象之前需要加载的对象列表)。 - -* `loading_dependencies_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 表加载依赖(需要在当前对象之前加载的对象列表)。 - -* `loading_dependent_database` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 加载所依赖的数据库。 - -* `loading_dependent_table` ([Array](../../sql-reference/data-types/array.md)([String](../../sql-reference/data-types/string.md))) - 依赖加载的表列表。 - -`system.tables` 表在实现 `SHOW TABLES` 查询时使用。 - -**示例** - -```sql -SELECT * FROM system.tables LIMIT 2 FORMAT Vertical; -``` - -```text -Row 1: -────── -database: base -name: t1 -uuid: 81b1c20a-b7c6-4116-a2ce-7583fb6b6736 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/store/81b/81b1c20a-b7c6-4116-a2ce-7583fb6b6736/'] -metadata_path: /var/lib/clickhouse/store/461/461cf698-fd0b-406d-8c01-5d8fd5748a91/t1.sql -metadata_modification_time: 2021-01-25 19:14:32 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE base.t1 (`n` UInt64) ENGINE = MergeTree ORDER BY n SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY n SETTINGS index_granularity = 8192 -as_select: SELECT database AS table_catalog -partition_key: -sorting_key: n -primary_key: n -sampling_key: -storage_policy: default -total_rows: 1 -total_bytes: 99 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] - -Row 2: -────── -database: default -name: 53r93yleapyears -uuid: 00000000-0000-0000-0000-000000000000 -engine: MergeTree -is_temporary: 0 -data_paths: ['/var/lib/clickhouse/data/default/53r93yleapyears/'] -metadata_path: /var/lib/clickhouse/metadata/default/53r93yleapyears.sql -metadata_modification_time: 2020-09-23 09:05:36 -dependencies_database: [] -dependencies_table: [] -create_table_query: CREATE TABLE default.`53r93yleapyears` (`id` Int8, `febdays` Int8) ENGINE = MergeTree ORDER BY id SETTINGS index_granularity = 8192 -engine_full: MergeTree ORDER BY id SETTINGS index_granularity = 8192 -as_select: SELECT name AS catalog_name -partition_key: -sorting_key: id -primary_key: id -sampling_key: -storage_policy: default -total_rows: 2 -total_bytes: 155 -lifetime_rows: ᴺᵁᴸᴸ -lifetime_bytes: ᴺᵁᴸᴸ -comment: -has_own_data: 0 -loading_dependencies_database: [] -loading_dependencies_table: [] -loading_dependent_database: [] -loading_dependent_table: [] -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md deleted file mode 100644 index 51931a80db2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/text_log.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -description: '记录日志条目的系统表。' -keywords: ['system table', 'text_log'] -slug: /operations/system-tables/text_log -title: 'system.text_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.text_log {#systemtext_log} - - - -包含日志记录条目。写入该表的日志级别可以通过服务器设置 `text_log.level` 进行限制。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `event_date` (Date) — 日志条目的日期。 -* `event_time` (DateTime) — 日志条目的时间。 -* `event_time_microseconds` (DateTime64) — 具有微秒精度的日志条目时间。 -* `microseconds` (UInt32) — 日志条目的微秒数。 -* `thread_name` (String) — 执行日志记录的线程名称。 -* `thread_id` (UInt64) — 操作系统线程 ID。 -* `level` (`Enum8`) — 日志条目级别。可能的取值: - * `1` 或 `'Fatal'`。 - * `2` 或 `'Critical'`。 - * `3` 或 `'Error'`。 - * `4` 或 `'Warning'`。 - * `5` 或 `'Notice'`。 - * `6` 或 `'Information'`。 - * `7` 或 `'Debug'`。 - * `8` 或 `'Trace'`。 -* `query_id` (String) — 查询的 ID。 -* `logger_name` (LowCardinality(String)) — 日志记录器名称(例如 `DDLWorker`)。 -* `message` (String) — 日志消息本身。 -* `revision` (UInt32) — ClickHouse 修订版本号。 -* `source_file` (LowCardinality(String)) — 产生该日志记录的源文件。 -* `source_line` (UInt64) — 产生该日志记录的源代码行号。 -* `message_format_string` (LowCardinality(String)) — 用于格式化消息的格式字符串。 -* `value1` (String) — 用于格式化消息的参数 1。 -* `value2` (String) — 用于格式化消息的参数 2。 -* `value3` (String) — 用于格式化消息的参数 3。 -* `value4` (String) — 用于格式化消息的参数 4。 -* `value5` (String) — 用于格式化消息的参数 5。 -* `value6` (String) — 用于格式化消息的参数 6。 -* `value7` (String) — 用于格式化消息的参数 7。 -* `value8` (String) — 用于格式化消息的参数 8。 -* `value9` (String) — 用于格式化消息的参数 9。 -* `value10` (String) — 用于格式化消息的参数 10。 - -**示例** - -```sql -SELECT * FROM system.text_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:07 -event_time_microseconds: 2020-09-10 11:23:07.871397 -microseconds: 871397 -thread_name: clickhouse-serv -thread_id: 564917 -level: Information -query_id: -logger_name: DNSCacheUpdater -message: 更新周期 15 秒 -revision: 54440 -source_file: /ClickHouse/src/Interpreters/DNSCacheUpdater.cpp; void DB::DNSCacheUpdater::start() -source_line: 45 -message_format_string: 更新周期 {} 秒 -value1: 15 -value2: -value3: -value4: -value5: -value6: -value7: -value8: -value9: -value10: -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md deleted file mode 100644 index 846706b402d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/time_zones.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: '包含 ClickHouse 服务器所支持的时区列表的 system 表。' -keywords: ['system 表', 'time_zones'] -slug: /operations/system-tables/time_zones -title: 'system.time_zones' -doc_type: 'reference' ---- - -# system.time_zones {#systemtime_zones} - -包含 ClickHouse 服务器所支持的时区列表。该时区列表可能会因 ClickHouse 版本不同而有所变化。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT * FROM system.time_zones LIMIT 10 -``` - -```text -┌─time_zone──────────┐ -│ Africa/Abidjan │ -│ Africa/Accra │ -│ Africa/Addis_Ababa │ -│ Africa/Algiers │ -│ Africa/Asmara │ -│ Africa/Asmera │ -│ Africa/Bamako │ -│ Africa/Bangui │ -│ Africa/Banjul │ -│ Africa/Bissau │ -└────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md deleted file mode 100644 index 6e256d9aeb0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/trace_log.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: '包含由查询采样分析器收集的堆栈跟踪的系统表。' -keywords: ['系统表', 'trace_log'] -slug: /operations/system-tables/trace_log -title: 'system.trace_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.trace_log {#systemtrace_log} - - - -包含由 [sampling query profiler](../../operations/optimizing-performance/sampling-query-profiler.md) 收集的堆栈跟踪信息。 - -当设置了服务器配置部分 [trace_log](../../operations/server-configuration-parameters/settings.md#trace_log) 时,ClickHouse 会创建此表。另请参见以下设置:[query_profiler_real_time_period_ns](../../operations/settings/settings.md#query_profiler_real_time_period_ns)、[query_profiler_cpu_time_period_ns](../../operations/settings/settings.md#query_profiler_cpu_time_period_ns)、[memory_profiler_step](../../operations/settings/settings.md#memory_profiler_step)、[memory_profiler_sample_probability](../../operations/settings/settings.md#memory_profiler_sample_probability)、[trace_profile_events](../../operations/settings/settings.md#trace_profile_events)。 - -要分析日志,请使用 `addressToLine`、`addressToLineWithInlines`、`addressToSymbol` 和 `demangle` 内省函数。 - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 - -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 采样时刻的日期。 - -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 采样时刻的时间戳。 - -* `event_time_microseconds` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 具有微秒精度的采样时刻时间戳。 - -* `timestamp_ns` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 以纳秒为单位的采样时刻时间戳。 - -* `revision` ([UInt32](../../sql-reference/data-types/int-uint.md)) — ClickHouse 服务器构建修订号。 - - 当通过 `clickhouse-client` 连接到服务器时,你会看到类似 `Connected to ClickHouse server version 19.18.1.` 的字符串。该字段包含的是服务器的 `revision`,而不是 `version`。 - -* `trace_type` ([Enum8](../../sql-reference/data-types/enum.md)) — 跟踪类型: - * `Real` 表示按墙钟时间收集堆栈跟踪。 - * `CPU` 表示按 CPU 时间收集堆栈跟踪。 - * `Memory` 表示当内存分配超过后续水位线时,收集分配和释放信息。 - * `MemorySample` 表示随机收集分配和释放信息。 - * `MemoryPeak` 表示收集内存峰值使用情况的更新。 - * `ProfileEvent` 表示收集 profile 事件增量。 - * `JemallocSample` 表示收集 jemalloc 样本。 - * `MemoryAllocatedWithoutCheck` 表示在忽略任何内存限制的情况下收集较大的内存分配(>16MiB)(仅供 ClickHouse 开发人员使用)。 - -* `thread_id` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 线程标识符。 - -* `query_id` ([String](../../sql-reference/data-types/string.md)) — 查询标识符,可用于从 [query_log](/operations/system-tables/query_log) 系统表中获取正在运行查询的详细信息。 - -* `trace` ([Array(UInt64)](../../sql-reference/data-types/array.md)) — 采样时刻的堆栈跟踪。每个元素是 ClickHouse 服务器进程内的虚拟内存地址。 - -* `size` ([Int64](../../sql-reference/data-types/int-uint.md)) — 对于 `Memory`、`MemorySample` 或 `MemoryPeak` 跟踪类型,该字段为分配的内存量;对于其他跟踪类型,该字段为 0。 - -* `event` ([LowCardinality(String)](../../sql-reference/data-types/lowcardinality.md)) — 对于 `ProfileEvent` 跟踪类型,该字段为已更新的 profile 事件名称;对于其他跟踪类型,该字段为空字符串。 - -* `increment` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 对于 `ProfileEvent` 跟踪类型,该字段为 profile 事件的增量值;对于其他跟踪类型,该字段为 0。 - -* `symbols`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)),如果启用了符号化,则包含与 `trace` 对应的已反混淆符号名称。 - -* `lines`, ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)),如果启用了符号化,则包含与 `trace` 对应的带行号的文件名字符串。 - -可以在服务器配置文件中 `trace_log` 部分下的 `symbolize` 设置中启用或禁用符号化。 - -**示例** - -```sql -SELECT * FROM system.trace_log LIMIT 1 \G -``` - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -event_date: 2020-09-10 -event_time: 2020-09-10 11:23:09 -event_time_microseconds: 2020-09-10 11:23:09.872924 -timestamp_ns: 1599762189872924510 -revision: 54440 -trace_type: Memory -thread_id: 564963 -query_id: -trace: [371912858,371912789,371798468,371799717,371801313,371790250,624462773,566365041,566440261,566445834,566460071,566459914,566459842,566459580,566459469,566459389,566459341,566455774,371993941,371988245,372158848,372187428,372187309,372187093,372185478,140222123165193,140222122205443] -size: 5244400 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md deleted file mode 100644 index 46eaeffc5ec..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/unicode.md +++ /dev/null @@ -1,178 +0,0 @@ ---- -description: '包含 Unicode 字符及其属性的系统表。' -keywords: ['系统表', 'unicode'] -slug: /operations/system-tables/unicode -title: 'system.unicode' -doc_type: 'reference' ---- - -# system.unicode {#systemunicode} - -`system.unicode` 表是一个虚拟表,用于提供关于 Unicode 字符及其属性的信息([https://unicode-org.github.io/icu/userguide/strings/properties.html](https://unicode-org.github.io/icu/userguide/strings/properties.html))。该表是动态生成的。 - -Columns - -:::note -ICU 文档中 Unicode 码点的属性名称会被转换为 snake_case。 -::: - -* `code_point` ([String](../../sql-reference/data-types/string.md)) — 码点的 UTF-8 表示。 -* `code_point_value` ([Int32](../../sql-reference/data-types/int-uint.md)) — 码点的数值。 -* `notation` ([String](../../sql-reference/data-types/string.md)) — 码点的 Unicode 表示形式。 -* Binary Properties ([UInt8](../../sql-reference/data-types/int-uint.md)) - 码点的二值属性。 - * `alphabetic`, `ascii_hex_digit`, `case_ignorable`... -* Enumerated Properties ([Int32](../../sql-reference/data-types/int-uint.md)) - 码点的枚举属性。 - * `bidi_class`, `bidi_paired_bracket_type`, `block`... -* String Properties ([String](../../sql-reference/data-types/string.md)) - 码点的字符串属性(ASCII 字符串、Unicode 字符串或码点) - * `case_folding`, `decomposition_mapping`, `name`... - -:::note -映射规则有一些特殊之处,详见 ICU 文档。例如,simple_uppercase_mapping 和 uppercase_mapping 并不完全相同。但未实现任何与语言相关的映射(例如在土耳其语中,i 的大写是 "İ" (U+0130))。 -::: - -* `numeric_value` ([Float64](../../sql-reference/data-types/float.md)) - 码点的数值。 -* `script_extensions` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) - 码点的书写系统扩展(script extensions)。 -* `identifier_type` ([Array(LowCardinality(String))](../../sql-reference/data-types/array.md)) - 码点的标识符类型。 -* `general_category_mask` ([Int32](../../sql-reference/data-types/int-uint.md)) - 码点的一般类别掩码。 - -**示例** - -```sql -SELECT * FROM system.unicode WHERE code_point = 'a' LIMIT 1; -``` - -```text -Row 1: -────── -code_point: a -code_point_value: 97 -notation: U+0061 -alphabetic: 1 -ascii_hex_digit: 1 -bidi_control: 0 -bidi_mirrored: 0 -dash: 0 -default_ignorable_code_point: 0 -deprecated: 0 -diacritic: 0 -extender: 0 -full_composition_exclusion: 0 -grapheme_base: 1 -grapheme_extend: 0 -grapheme_link: 0 -hex_digit: 1 -hyphen: 0 -id_continue: 1 -id_start: 1 -ideographic: 0 -ids_binary_operator: 0 -ids_trinary_operator: 0 -join_control: 0 -logical_order_exception: 0 -lowercase: 1 -math: 0 -noncharacter_code_point: 0 -quotation_mark: 0 -radical: 0 -soft_dotted: 0 -terminal_punctuation: 0 -unified_ideograph: 0 -uppercase: 0 -white_space: 0 -xid_continue: 1 -xid_start: 1 -case_sensitive: 1 -sentence_terminal: 0 -variation_selector: 0 -nfd_inert: 1 -nfkd_inert: 1 -nfc_inert: 0 -nfkc_inert: 0 -segment_starter: 1 -pattern_syntax: 0 -pattern_white_space: 0 -alnum: 1 -blank: 0 -graph: 1 -print: 1 -xdigit: 1 -cased: 1 -case_ignorable: 0 -changes_when_lowercased: 0 -changes_when_uppercased: 1 -changes_when_titlecased: 1 -changes_when_casefolded: 0 -changes_when_casemapped: 1 -changes_when_nfkc_casefolded: 0 -emoji: 0 -emoji_presentation: 0 -emoji_modifier: 0 -emoji_modifier_base: 0 -emoji_component: 0 -regional_indicator: 0 -prepended_concatenation_mark: 0 -extended_pictographic: 0 -basic_emoji: 0 -emoji_keycap_sequence: 0 -rgi_emoji_modifier_sequence: 0 -rgi_emoji_flag_sequence: 0 -rgi_emoji_tag_sequence: 0 -rgi_emoji_zwj_sequence: 0 -rgi_emoji: 0 -ids_unary_operator: 0 -id_compat_math_start: 0 -id_compat_math_continue: 0 -bidi_class: 0 -block: 1 -canonical_combining_class: 0 -decomposition_type: 0 -east_asian_width: 4 -general_category: 2 -joining_group: 0 -joining_type: 0 -line_break: 2 -numeric_type: 0 -script: 25 -hangul_syllable_type: 0 -nfd_quick_check: 1 -nfkd_quick_check: 1 -nfc_quick_check: 1 -nfkc_quick_check: 1 -lead_canonical_combining_class: 0 -trail_canonical_combining_class: 0 -grapheme_cluster_break: 0 -sentence_break: 4 -word_break: 1 -bidi_paired_bracket_type: 0 -indic_positional_category: 0 -indic_syllabic_category: 0 -vertical_orientation: 0 -identifier_status: 1 -general_category_mask: 4 -numeric_value: 0 -age: 1.1 -bidi_mirroring_glyph: a -case_folding: a -lowercase_mapping: a -name: LATIN SMALL LETTER A -simple_case_folding: a -simple_lowercase_mapping: a -simple_titlecase_mapping: A -simple_uppercase_mapping: A -titlecase_mapping: A -uppercase_mapping: A -bidi_paired_bracket: a -script_extensions: ['Latin'] -identifier_type: ['Recommended'] - -``` - -```sql -SELECT code_point, code_point_value, notation FROM system.unicode WHERE code_point = '😂'; -``` - -```text - ┌─code_point─┬─code_point_value─┬─notation─┐ -1. │ 😂 │ 128514 │ U+1F602 │ - └────────────┴──────────────────┴──────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md deleted file mode 100644 index 79ad722a99c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/user_processes.md +++ /dev/null @@ -1,36 +0,0 @@ ---- -description: '包含用于概览用户内存使用情况和 ProfileEvents 的系统表。' -keywords: ['system table', 'user_processes'] -slug: /operations/system-tables/user_processes -title: 'system.user_processes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.user_processes {#systemuser_processes} - - - -此系统表可用于查看用户的内存使用情况和 ProfileEvents 的整体概况。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -```sql -SELECT * FROM system.user_processes LIMIT 10 FORMAT Vertical; -``` - -```response -第 1 行: -────── -user: default -memory_usage: 9832 -peak_memory_usage: 9832 -ProfileEvents: {'Query':5,'SelectQuery':5,'QueriesWithSubqueries':38,'SelectQueriesWithSubqueries':38,'QueryTimeMicroseconds':842048,'SelectQueryTimeMicroseconds':842048,'ReadBufferFromFileDescriptorRead':6,'ReadBufferFromFileDescriptorReadBytes':234,'IOBufferAllocs':3,'IOBufferAllocBytes':98493,'ArenaAllocChunks':283,'ArenaAllocBytes':1482752,'FunctionExecute':670,'TableFunctionExecute':16,'DiskReadElapsedMicroseconds':19,'NetworkSendElapsedMicroseconds':684,'NetworkSendBytes':139498,'SelectedRows':6076,'SelectedBytes':685802,'ContextLock':1140,'RWLockAcquiredReadLocks':193,'RWLockReadersWaitMilliseconds':4,'RealTimeMicroseconds':1585163,'UserTimeMicroseconds':889767,'SystemTimeMicroseconds':13630,'SoftPageFaults':1947,'OSCPUWaitMicroseconds':6,'OSCPUVirtualTimeMicroseconds':903251,'OSReadChars':28631,'OSWriteChars':28888,'QueryProfilerRuns':3,'LogTrace':79,'LogDebug':24} - -结果集包含 1 行。用时:0.010 秒。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/users.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/users.md deleted file mode 100644 index 62f136c3765..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/users.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -description: '包含服务器上已配置用户账户列表的 system 表。' -keywords: ['system 表', 'users'] -slug: /operations/system-tables/users -title: 'system.users' -doc_type: 'reference' ---- - -# system.users {#systemusers} - -包含在服务器上配置的[用户账户](../../guides/sre/user-management/index.md#user-account-management)列表。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - - -## 另请参阅 {#see-also} - -- [SHOW USERS](/sql-reference/statements/show#show-users) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md deleted file mode 100644 index 252e1da9153..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/view_refreshes.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -description: '用于存储可刷新物化视图相关信息的系统表。' -keywords: ['system table', 'view_refreshes'] -slug: /operations/system-tables/view_refreshes -title: 'system.view_refreshes' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.view_refreshes {#systemview_refreshes} - - - -关于[可刷新物化视图(Refreshable Materialized Views)](../../sql-reference/statements/create/view.md#refreshable-materialized-view)的信息。包含所有可刷新物化视图,无论当前是否有刷新在进行。 - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } - -**示例** - -```sql -SELECT - database, - view, - status, - last_refresh_result, - last_refresh_time, - next_refresh_time -FROM system.view_refreshes - -┌─database─┬─view───────────────────────┬─status────┬─last_refresh_result─┬───last_refresh_time─┬───next_refresh_time─┐ -│ default │ hello_documentation_reader │ Scheduled │ Finished │ 2023-12-01 01:24:00 │ 2023-12-01 01:25:00 │ -└──────────┴────────────────────────────┴───────────┴─────────────────────┴─────────────────────┴─────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md deleted file mode 100644 index 9e6f721c156..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/workloads.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -description: '包含本地服务器工作负载信息的系统表。' -keywords: ['system table', 'workloads'] -slug: /operations/system-tables/workloads -title: 'system.workloads' -doc_type: 'reference' ---- - -# system.workloads {#systemworkloads} - -包含本地服务器上各个 [workload](/operations/workload-scheduling.md#workload_entity_storage) 的信息。该表中每一行对应一个 workload。 - -示例: - -```sql -SELECT * -FROM system.workloads -FORMAT Vertical -``` - -```text -第 1 行: -────── -name: production -parent: all -create_query: CREATE WORKLOAD production IN `all` SETTINGS weight = 9 - -第 2 行: -────── -name: development -parent: all -create_query: CREATE WORKLOAD development IN `all` - -第 3 行: -────── -name: all -parent: -create_query: CREATE WORKLOAD `all` -``` - -列: - -{/*AUTOGENERATED_START*/ } - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md deleted file mode 100644 index bc79217bc63..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: '仅在配置了 ClickHouse Keeper 或 ZooKeeper 时才存在的系统表。它提供对配置中定义的 Keeper 集群数据的访问。' -keywords: ['系统表', 'zookeeper'] -slug: /operations/system-tables/zookeeper -title: 'system.zookeeper' -doc_type: 'reference' ---- - -# system.zookeeper {#systemzookeeper} - -除非配置了 ClickHouse Keeper 或 ZooKeeper,否则该表不会存在。`system.zookeeper` 表会暴露配置中定义的 Keeper 集群中的数据。 -查询必须在 `WHERE` 子句中按如下所示包含 `path =` 条件或 `path IN` 条件。这对应于你希望获取其子节点数据的路径。 - -查询 `SELECT * FROM system.zookeeper WHERE path = '/clickhouse'` 会输出 `/clickhouse` 节点下所有子节点的数据。 -要输出所有根节点(根路径 `/` 下)的数据,请将 path 设置为 '/'。 -如果在 'path' 中指定的路径不存在,将抛出异常。 - -查询 `SELECT * FROM system.zookeeper WHERE path IN ('/', '/clickhouse')` 会输出 `/` 和 `/clickhouse` 节点下所有子节点的数据。 -如果在指定的 'path' 集合中某个路径不存在,将抛出异常。 -这可以用于批量执行 Keeper 路径查询。 - -查询 `SELECT * FROM system.zookeeper WHERE path = '/clickhouse' AND zookeeperName = 'auxiliary_cluster'` 会输出 `auxiliary_cluster` ZooKeeper 集群中的数据。 -如果指定的 'auxiliary_cluster' 不存在,将抛出异常。 - -列: - -* `name` (String) — 节点名称。 -* `path` (String) — 节点路径。 -* `value` (String) — 节点的值。 -* `zookeeperName` (String) — 默认 ZooKeeper 集群或某个辅助 ZooKeeper 集群的名称。 -* `dataLength` (Int32) — 值的大小。 -* `numChildren` (Int32) — 子孙节点数量。 -* `czxid` (Int64) — 创建该节点的事务 ID。 -* `mzxid` (Int64) — 最后一次修改该节点的事务 ID。 -* `pzxid` (Int64) — 最后一次删除或添加子孙节点的事务 ID。 -* `ctime` (DateTime) — 节点创建时间。 -* `mtime` (DateTime) — 节点最后修改时间。 -* `version` (Int32) — 节点版本:节点被修改的次数。 -* `cversion` (Int32) — 添加或删除子孙节点的次数。 -* `aversion` (Int32) — ACL 更改次数。 -* `ephemeralOwner` (Int64) — 对于临时节点,该节点所属会话的 ID。 - -示例: - -```sql -SELECT * -FROM system.zookeeper -WHERE path = '/clickhouse/tables/01-08/visits/replicas' -FORMAT Vertical -``` - -```text -Row 1: -────── -name: example01-08-1 -value: -czxid: 932998691229 -mzxid: 932998691229 -ctime: 2015-03-27 16:49:51 -mtime: 2015-03-27 16:49:51 -version: 0 -cversion: 47 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021031383 -path: /clickhouse/tables/01-08/visits/replicas - -Row 2: -────── -name: example01-08-2 -value: -czxid: 933002738135 -mzxid: 933002738135 -ctime: 2015-03-27 16:57:01 -mtime: 2015-03-27 16:57:01 -version: 0 -cversion: 37 -aversion: 0 -ephemeralOwner: 0 -dataLength: 0 -numChildren: 7 -pzxid: 987021252247 -path: /clickhouse/tables/01-08/visits/replicas -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md deleted file mode 100644 index a9f0f6089ac..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '仅在配置了 ZooKeeper 时存在的系统表。显示当前与 ZooKeeper 的连接(包括辅助 ZooKeeper 实例)。' -keywords: ['system table', 'zookeeper_connection'] -slug: /operations/system-tables/zookeeper_connection -title: 'system.zookeeper_connection' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection {#systemzookeeper_connection} - - - -在未配置 ZooKeeper 时,该表不存在。`system.zookeeper_connection` 表显示当前到 ZooKeeper 的连接(包括辅助 ZooKeeper)。每一行表示一个连接的信息。 - -列: - -* `name` ([String](../../sql-reference/data-types/string.md)) — ZooKeeper 集群的名称。 -* `host` ([String](../../sql-reference/data-types/string.md)) — ClickHouse 连接到的 ZooKeeper 节点的主机名/IP。 -* `port` ([UIn16](../../sql-reference/data-types/int-uint.md)) — ClickHouse 连接到的 ZooKeeper 节点的端口。 -* `index` ([Nullable(UInt8)](../../sql-reference/data-types/int-uint.md)) — ClickHouse 连接到的 ZooKeeper 节点在配置中的索引。如果未连接,此列为 NULL。 -* `connected_time` ([DateTime](../../sql-reference/data-types/datetime.md)) — 建立连接的时间。 -* `session_uptime_elapsed_seconds` ([UInt64](../../sql-reference/data-types/int-uint.md)) — 自连接建立以来经过的秒数。 -* `is_expired` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 当前连接是否已过期。 -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Keeper API 版本。 -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — 连接的会话 ID。 -* `xid` ([Int64](../../sql-reference/data-types/int-uint.md)) — 当前会话的 XID。 -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — 已启用的功能标志。仅适用于 ClickHouse Keeper。可能的取值为 `FILTERED_LIST`、`MULTI_READ`、`CHECK_NOT_EXISTS`、`CREATE_IF_NOT_EXISTS`、`REMOVE_RECURSIVE`。 -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — 可用区。 - -示例: - -```sql -SELECT * FROM system.zookeeper_connection; -``` - -```text -┌─name────┬─host──────┬─port─┬─index─┬──────connected_time─┬─session_uptime_elapsed_seconds─┬─is_expired─┬─keeper_api_version─┬─client_id─┬─xid─┬─enabled_feature_flags────────────────────────────────────────────────────┬─availability_zone─┐ -│ default │ 127.0.0.1 │ 2181 │ 0 │ 2025-04-10 14:30:00 │ 943 │ 0 │ 0 │ 420 │ 69 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS'] │ eu-west-1b │ -└─────────┴───────────┴──────┴───────┴─────────────────────┴────────────────────────────────┴────────────┴────────────────────┴───────────┴─────┴──────────────────────────────────────────────────────────────────────────┴───────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md deleted file mode 100644 index da4e36664dc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_connection_log.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '显示 ZooKeeper 连接历史(包括辅助 ZooKeeper)。' -keywords: ['system table', 'zookeeper_connection_log'] -slug: /operations/system-tables/zookeeper_connection_log -title: 'system.zookeeper_connection_log' -doc_type: 'reference' ---- - -import SystemTableCloud from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_system_table_cloud.md'; - -# system.zookeeper_connection_log {#systemzookeeper_connection_log} - - - -`system.zookeeper_connection_log` 表显示 ZooKeeper 连接(包括辅助 ZooKeeper)的历史记录。每一行对应一个与连接相关的事件。 - -:::note -该表不包含由于服务器关闭导致的断开连接事件。 -::: - -列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 与 ZooKeeper 建立连接或断开连接的服务器主机名。 -* `type` ([Enum8](../../sql-reference/data-types/enum.md)) - 事件类型。可能的取值:`Connected`、`Disconnected`。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) - 记录的日期。 -* `event_time` ([DateTime](../../sql-reference/data-types/datetime.md)) - 记录的时间。 -* `event_time_microseconds` ([Date](../../sql-reference/data-types/datetime64.md)) - 具有微秒精度的记录时间。 -* `name` ([String](../../sql-reference/data-types/string.md)) — ZooKeeper 集群名称。 -* `host` ([String](../../sql-reference/data-types/string.md)) — ClickHouse 所连接的 ZooKeeper 节点的主机名/IP。 -* `port` ([UIn16](../../sql-reference/data-types/int-uint.md)) — ClickHouse 所连接的 ZooKeeper 节点的端口。 -* `index` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ClickHouse 连接到或从其断开连接的 ZooKeeper 节点索引。该索引来自 ZooKeeper 配置。 -* `client_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — 该连接的会话 ID。 -* `keeper_api_version` ([UInt8](../../sql-reference/data-types/int-uint.md)) — Keeper API 版本。 -* `enabled_feature_flags` ([Array(Enum16)](../../sql-reference/data-types/array.md)) — 已启用的功能标志。仅适用于 ClickHouse Keeper。可能的取值为 `FILTERED_LIST`、`MULTI_READ`、`CHECK_NOT_EXISTS`、`CREATE_IF_NOT_EXISTS`、`REMOVE_RECURSIVE`。 -* `availability_zone` ([String](../../sql-reference/data-types/string.md)) — 可用区。 -* `reason` ([String](../../sql-reference/data-types/string.md)) — 连接或断开连接的原因。 - -Example: - -```sql -SELECT * FROM system.zookeeper_connection_log; -``` - -```text -┌─hostname─┬─type─────────┬─event_date─┬──────────event_time─┬────event_time_microseconds─┬─name───────────────┬─host─┬─port─┬─index─┬─client_id─┬─keeper_api_version─┬─enabled_feature_flags───────────────────────────────────────────────────────────────────────┬─availability_zone─┬─reason──────────────┐ - 1. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:35 │ 2025-05-12 19:49:35.713067 │ zk_conn_log_test_4 │ zoo2 │ 2181 │ 0 │ 10 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初始化 │ - 2. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:23 │ 2025-05-12 19:49:23.981570 │ default │ zoo1 │ 2181 │ 0 │ 4 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初始化 │ - 3. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:28 │ 2025-05-12 19:49:28.104021 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初始化 │ - 4. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.459251 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初始化 │ - 5. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.574312 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 初始化 │ - 6. │ node │ Disconnected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909890 │ default │ zoo1 │ 2181 │ 0 │ 5 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 配置变更 │ - 7. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.909895 │ default │ zoo2 │ 2181 │ 0 │ 8 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 配置变更 │ - 8. │ node │ Disconnected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912010 │ zk_conn_log_test_2 │ zoo2 │ 2181 │ 0 │ 6 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 配置变更 │ - 9. │ node │ Connected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912014 │ zk_conn_log_test_2 │ zoo3 │ 2181 │ 0 │ 9 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 配置变更 │ -10. │ node │ Disconnected │ 2025-05-12 │ 2025-05-12 19:49:29 │ 2025-05-12 19:49:29.912061 │ zk_conn_log_test_3 │ zoo3 │ 2181 │ 0 │ 7 │ 0 │ ['FILTERED_LIST','MULTI_READ','CHECK_NOT_EXISTS','CREATE_IF_NOT_EXISTS','REMOVE_RECURSIVE'] │ │ 从配置中移除 │ - └──────────┴──────────────┴────────────┴─────────────────────┴────────────────────────────┴────────────────────┴──────┴──────┴───────┴───────────┴────────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────┴───────────────────┴─────────────────────┘ -``` \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md deleted file mode 100644 index a27dd36587a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/system-tables/zookeeper_log.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: '包含发送到 ZooKeeper 服务器的请求参数及其响应参数信息的系统表。' -keywords: ['system table', 'zookeeper_log'] -slug: /operations/system-tables/zookeeper_log -title: 'system.zookeeper_log' -doc_type: 'reference' ---- - -# system.zookeeper_log {#systemzookeeper_log} - -该表包含向 ZooKeeper 服务器发起请求时的参数以及服务器响应相关的信息。 - -对于请求,只会填充包含请求参数的列,其余列填充默认值(`0` 或 `NULL`)。当收到响应时,会将响应中的数据补充到其他列中。 - -包含请求参数的列: - -* `hostname` ([LowCardinality(String)](../../sql-reference/data-types/string.md)) — 执行查询的服务器主机名。 -* `type` ([Enum](../../sql-reference/data-types/enum.md)) — ZooKeeper 客户端中的事件类型。可以具有以下值之一: - * `Request` — 请求已发送。 - * `Response` — 已接收到响应。 - * `Finalize` — 连接丢失,未接收到响应。 -* `event_date` ([Date](../../sql-reference/data-types/date.md)) — 事件发生的日期。 -* `event_time` ([DateTime64](../../sql-reference/data-types/datetime64.md)) — 事件发生的日期和时间。 -* `address` ([IPv6](../../sql-reference/data-types/ipv6.md)) — 用于发起请求的 ZooKeeper 服务器的 IP 地址。 -* `port` ([UInt16](../../sql-reference/data-types/int-uint.md)) — 用于发起请求的 ZooKeeper 服务器端口。 -* `session_id` ([Int64](../../sql-reference/data-types/int-uint.md)) — ZooKeeper 服务器为每个连接设置的会话 ID。 -* `xid` ([Int32](../../sql-reference/data-types/int-uint.md)) — 会话内请求的 ID。通常是按顺序递增的请求编号。对于请求行以及与之配对的 `response`/`finalize` 行,该值相同。 -* `has_watch` ([UInt8](../../sql-reference/data-types/int-uint.md)) — 请求是否设置了 [watch](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#ch_zkWatches)。 -* `op_num` ([Enum](../../sql-reference/data-types/enum.md)) — 请求或响应的类型。 -* `path` ([String](../../sql-reference/data-types/string.md)) — 请求中指定的 ZooKeeper 节点路径;如果请求不需要指定路径,则为空字符串。 -* `data` ([String](../../sql-reference/data-types/string.md)) — 写入 ZooKeeper 节点的数据(对于 `SET` 和 `CREATE` 请求 — 请求要写入的内容;对于 `GET` 请求的响应 — 实际读取到的内容),或为空字符串。 -* `is_ephemeral` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ZooKeeper 节点是否被创建为 [ephemeral](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Ephemeral+Nodes) 节点。 -* `is_sequential` ([UInt8](../../sql-reference/data-types/int-uint.md)) — ZooKeeper 节点是否被创建为 [sequential](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html#Sequence+Nodes+--+Unique+Naming) 节点。 -* `version` ([Nullable(Int32)](../../sql-reference/data-types/nullable.md)) — 请求在执行时期望的 ZooKeeper 节点版本。适用于 `CHECK`、`SET`、`REMOVE` 请求(如果请求不检查版本,则该字段为特殊值 `-1`;对于不支持版本检查的其他请求则为 `NULL`)。 -* `requests_size` ([UInt32](../../sql-reference/data-types/int-uint.md)) — multi 请求中包含的请求数量(这是一种特殊请求,由若干个连续的普通请求组成,并以原子方式执行它们)。multi 请求中包含的所有请求都具有相同的 `xid`。 -* `request_idx` ([UInt32](../../sql-reference/data-types/int-uint.md)) — multi 请求中某个子请求的序号(对 multi 请求本身为 `0`,随后子请求从 `1` 开始依次递增)。 - -包含请求响应参数的列: - -* `zxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — ZooKeeper 事务 ID。由 ZooKeeper 服务器在成功执行请求后发出的序列号(如果请求未执行/返回错误/客户端不知道请求是否已执行,则为 `0`)。 -* `error` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — 错误代码。可能取多个值,这里仅列出部分: - * `ZOK` — 请求执行成功。 - * `ZCONNECTIONLOSS` — 连接丢失。 - * `ZOPERATIONTIMEOUT` — 请求执行超时。 - * `ZSESSIONEXPIRED` — 会话已过期。 - * `NULL` — 请求已完成。 -* `watch_type` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — `watch` 事件的类型(对于 `op_num` = `Watch` 的响应),其他响应中为 `NULL`。 -* `watch_state` ([Nullable(Enum)](../../sql-reference/data-types/nullable.md)) — `watch` 事件的状态(对于 `op_num` = `Watch` 的响应),其他响应中为 `NULL`。 -* `path_created` ([String](../../sql-reference/data-types/string.md)) — 已创建 ZooKeeper 节点的路径(针对 `CREATE` 请求的响应),如果节点以 `sequential` 方式创建,则可能与 `path` 不同。 -* `stat_czxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — 导致创建此 ZooKeeper 节点的变更的 `zxid`。 -* `stat_mzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — 最近一次修改此 ZooKeeper 节点的变更的 `zxid`。 -* `stat_pzxid` ([Int64](../../sql-reference/data-types/int-uint.md)) — 最近一次修改此 ZooKeeper 节点子节点的变更的事务 ID。 -* `stat_version` ([Int32](../../sql-reference/data-types/int-uint.md)) — 针对此 ZooKeeper 节点数据进行的变更次数。 -* `stat_cversion` ([Int32](../../sql-reference/data-types/int-uint.md)) — 针对此 ZooKeeper 节点子节点进行的变更次数。 -* `stat_dataLength` ([Int32](../../sql-reference/data-types/int-uint.md)) — 此 ZooKeeper 节点数据字段的长度。 -* `stat_numChildren` ([Int32](../../sql-reference/data-types/int-uint.md)) — 此 ZooKeeper 节点的子节点数量。 -* `children` ([Array(String)](../../sql-reference/data-types/array.md)) — 子 ZooKeeper 节点列表(针对 `LIST` 请求的响应)。 - -**示例** - -查询: - -```sql -SELECT * FROM system.zookeeper_log WHERE (session_id = '106662742089334927') AND (xid = '10858') FORMAT Vertical; -``` - -结果: - -```text -Row 1: -────── -hostname: clickhouse.eu-central1.internal -type: Request -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.291792 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 0 -error: ᴺᵁᴸᴸ -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 0 -stat_mzxid: 0 -stat_pzxid: 0 -stat_version: 0 -stat_cversion: 0 -stat_dataLength: 0 -stat_numChildren: 0 -children: [] - -Row 2: -────── -type: Response -event_date: 2021-08-09 -event_time: 2021-08-09 21:38:30.292086 -address: :: -port: 2181 -session_id: 106662742089334927 -xid: 10858 -has_watch: 1 -op_num: List -path: /clickhouse/task_queue/ddl -data: -is_ephemeral: 0 -is_sequential: 0 -version: ᴺᵁᴸᴸ -requests_size: 0 -request_idx: 0 -zxid: 16926267 -error: ZOK -watch_type: ᴺᵁᴸᴸ -watch_state: ᴺᵁᴸᴸ -path_created: -stat_czxid: 16925469 -stat_mzxid: 16925469 -stat_pzxid: 16926179 -stat_version: 0 -stat_cversion: 7 -stat_dataLength: 0 -stat_numChildren: 7 -children: ['query-0000000006','query-0000000005','query-0000000004','query-0000000003','query-0000000002','query-0000000001','query-0000000000'] -``` - -**另请参阅** - -* [ZooKeeper](../../operations/tips.md#zookeeper) -* [ZooKeeper 指南](https://zookeeper.apache.org/doc/r3.3.3/zookeeperProgrammers.html) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/tips.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/tips.md deleted file mode 100644 index edbdf7f976f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/tips.md +++ /dev/null @@ -1,326 +0,0 @@ ---- -description: '介绍开源版 ClickHouse 使用建议的页面' -sidebar_label: 'OSS 使用建议' -sidebar_position: 58 -slug: /operations/tips -title: 'OSS 使用建议' -doc_type: 'guide' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_automated.md'; - - - - -## CPU 频率调节策略 {#cpu-scaling-governor} - -应始终使用 `performance` 频率调节策略。`on-demand` 频率调节策略在持续高负载场景下的效果要差得多。 - -```bash -$ echo 'performance' | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor -``` - - -## CPU 限制 {#cpu-limitations} - -处理器可能会过热。使用 `dmesg` 查看 CPU 的频率是否因过热而被限制。 -该限制也可能由数据中心层面的外部策略设置。可以在负载下使用 `turbostat` 对其进行监控。 - -## RAM {#ram} - -对于较小的数据量(压缩后最多约 200 GB),最好使用与数据量大致相当的内存。 -对于较大的数据量并且需要处理交互式(在线)查询时,应使用合理数量的 RAM(128 GB 或更多),以便热点数据子集能够装入页缓存中。 -即使在每台服务器数据量约为 50 TB 的情况下,使用 128 GB RAM 也能相比 64 GB 明显提升查询性能。 - -不要禁用 overcommit。`cat /proc/sys/vm/overcommit_memory` 的值应为 0 或 1。运行 - -```bash -$ echo 0 | sudo tee /proc/sys/vm/overcommit_memory -``` - -使用 `perf top` 观察内核在内存管理上的耗时。 -永久性 huge pages 也无需分配。 - - -### 使用少于 16GB 内存时 {#using-less-than-16gb-of-ram} - -推荐的内存大小为 32 GB 及以上。 - -如果系统的内存少于 16 GB,由于默认设置与该内存容量不匹配,可能会遇到各种内存异常。可以在内存较小的系统(最低到 2 GB)上使用 ClickHouse,但这些部署需要额外调优,并且只能以较低速率进行数据摄取。 - -在使用少于 16GB 内存运行 ClickHouse 时,建议执行以下操作: - -- 在 `config.xml` 中减小 mark cache 的大小。它可以设置得低至 500 MB,但不能设置为零。 -- 将查询处理线程数降低到 `1`。 -- 将 `max_block_size` 降低到 `8192`。即使低至 `1024` 的值在实践中也仍然可用。 -- 将 `max_download_threads` 降低到 `1`。 -- 将 `input_format_parallel_parsing` 和 `output_format_parallel_formatting` 设置为 `0`。 -- 禁用向 log 表写入数据,因为这会使后台合并任务为合并这些 log 表预留 RAM。禁用 `asynchronous_metric_log`、`metric_log`、`text_log`、`trace_log`。 - -附加说明: - -- 为了释放由内存分配器缓存的内存,可以运行 `SYSTEM JEMALLOC PURGE` 命令。 -- 不建议在内存较小的机器上使用 S3 或 Kafka 集成,因为它们的缓冲区需要占用大量内存。 - -## 存储子系统 {#storage-subsystem} - -如果预算允许,尽量使用 SSD。 -如果不允许,就使用 HDD。使用转速为 7200 RPM 的 SATA HDD 即可。 - -优先选择数量较多且带本地硬盘的服务器,而不是数量较少但连接外置磁盘柜的服务器。 -但对于仅偶尔被查询的归档数据,使用磁盘柜也是可行的。 - -## RAID {#raid} - -在使用 HDD 时,可以将其组合为 RAID-10、RAID-5、RAID-6 或 RAID-50。 -在 Linux 上,推荐使用软件 RAID(通过 `mdadm` 实现)。 -在创建 RAID-10 时,选择 `far` 布局。 -如果预算允许,优先选择 RAID-10。 - -单独使用 LVM(不配合 RAID 或 `mdadm`)是可以的,但用它做 RAID 或将其与 `mdadm` 组合使用是相对比较少见的方案,更容易出错 -(chunk 大小选择不当;chunk 未对齐;RAID 类型选择错误;忘记清理磁盘)。如果你对使用 LVM 很有把握,可以放心使用。 - -如果有超过 4 块磁盘,请使用 RAID-6(推荐)或 RAID-50,而不是 RAID-5。 -在使用 RAID-5、RAID-6 或 RAID-50 时,一定要增大 stripe_cache_size 参数,因为默认值通常并非最佳选择。 - -```bash -$ echo 4096 | sudo tee /sys/block/md2/md/stripe_cache_size -``` - -根据设备数量和块大小精确计算该数值,使用公式:`2 * num_devices * chunk_size_in_bytes / 4096`。 - -对于大多数 RAID 配置,64 KB 的块大小已经足够。clickhouse-server 的平均写入大小大约为 1 MB(1024 KB),因此推荐的条带大小也是 1 MB。如果需要进一步优化块大小,可将其设置为 1 MB 除以 RAID 阵列中非校验盘的数量,这样每次写入都可以在所有可用的非校验盘之间并行化。 -避免将块大小设置得过小或过大。 - -可以在 SSD 上使用 RAID-0。 -无论是否使用 RAID,都务必启用复制(replication)以保证数据安全。 - -启用带有较长队列的 NCQ。对于 HDD,选择 mq-deadline 或 CFQ 调度器;对于 SSD,选择 noop。不要降低 `readahead` 设置。 -对于 HDD,启用写缓存。 - -确保在操作系统中为 NVMe 和 SSD 磁盘启用了 [`fstrim`](https://en.wikipedia.org/wiki/Trim_\(computing\))(通常通过 cron 作业或 systemd 服务实现)。 - - -## 文件系统 {#file-system} - -Ext4 是最可靠的选择。将挂载选项设置为 `noatime`。XFS 的表现也很好。 -大多数其他文件系统通常也可以正常工作。 - -由于不支持硬链接,FAT-32 和 exFAT 不受支持。 - -不要使用压缩文件系统,因为 ClickHouse 自身已经进行了效果更好的压缩。 -不推荐使用加密文件系统,因为可以使用 ClickHouse 内置的加密功能,其效果更好。 - -虽然 ClickHouse 可以通过 NFS 工作,但这并不是最理想的选择。 - -## Linux 内核 {#linux-kernel} - -请勿使用已过时的 Linux 内核。 - -## 网络 {#network} - -如果使用 IPv6,请增大路由缓存的大小。 -3.2 之前版本的 Linux 内核在 IPv6 实现方面存在诸多问题。 - -如果可能,请至少使用 10 Gb 的网络。1 Gb 也能用,但在为包含数十 TB 数据的副本进行补丁更新,或处理具有大量中间数据的分布式查询时,效果会差得多。 - -## Huge Pages {#huge-pages} - -如果你使用的是较旧的 Linux 内核,应禁用 Transparent Huge Pages(透明大页)。它会干扰内存分配器,从而导致明显的性能下降。 -在较新的 Linux 内核中,Transparent Huge Pages 可以正常使用。 - -```bash -$ echo 'madvise' | sudo tee /sys/kernel/mm/transparent_hugepage/enabled -``` - -如果您想永久修改 transparent huge pages(透明大页)设置,请编辑 `/etc/default/grub`,在 `GRUB_CMDLINE_LINUX_DEFAULT` 选项中添加 `transparent_hugepage=madvise`: - -```bash -$ GRUB_CMDLINE_LINUX_DEFAULT="transparent_hugepage=madvise ..." -``` - -之后运行 `sudo update-grub` 命令,然后重启系统以使其生效。 - - -## 虚拟机管理程序配置 {#hypervisor-configuration} - -如果您使用 OpenStack,请设置 - -```ini -cpu_mode=host-passthrough -``` - -在 `nova.conf` 中。 - -如果使用 libvirt,请设置 - -```xml - -``` - -在 XML 配置中进行设置。 - -这对于 ClickHouse 能够通过 `cpuid` 指令获取正确信息非常重要。 -否则,如果在较旧的 CPU 型号上运行虚拟机管理程序,可能会触发 `Illegal instruction` 崩溃。 - - -## ClickHouse Keeper 和 ZooKeeper {#zookeeper} - -推荐在 ClickHouse 集群中使用 ClickHouse Keeper 替代 ZooKeeper。请参阅 [ClickHouse Keeper](../guides/sre/keeper/index.md) 文档。 - -如果希望继续使用 ZooKeeper,最好使用较新的 ZooKeeper 版本——3.4.9 或更高版本。稳定版 Linux 发行版中自带的版本可能已经过时。 - -绝不要使用手动编写的脚本在不同的 ZooKeeper 集群之间传输数据,因为对于顺序节点而言,结果将是不正确的。出于同样的原因,也绝不要使用 `zkcopy` 工具:[https://github.com/ksprojects/zkcopy/issues/15](https://github.com/ksprojects/zkcopy/issues/15) - -如果要把一个现有的 ZooKeeper 集群拆分为两个,正确的方法是先增加其副本数量,然后再将其重新配置为两个彼此独立的集群。 - -在测试环境或摄取速率较低的环境中,可以在与 ClickHouse 相同的服务器上运行 ClickHouse Keeper。 -对于生产环境,我们建议为 ClickHouse 和 ZooKeeper/Keeper 使用单独的服务器,或者将 ClickHouse 文件和 Keeper 文件放在不同的磁盘上。原因是 ZooKeeper/Keeper 对磁盘延迟非常敏感,而 ClickHouse 可能会占用所有可用的系统资源。 - -可以在 ZooKeeper 集群中配置 observer 节点,但 ClickHouse 服务器不应与 observer 交互。 - -不要修改 `minSessionTimeout` 设置,过大的值可能会影响 ClickHouse 重启时的稳定性。 - -在默认设置下,ZooKeeper 就像一个定时炸弹: - -> 在使用默认配置时(参见 `autopurge`),ZooKeeper 服务器不会删除旧的快照和日志文件,这项工作由运维人员负责。 - -必须拆除这枚炸弹。 - -下面的 ZooKeeper(3.5.1)配置用于一个大型生产环境: - -zoo.cfg: - -```bash -# http://hadoop.apache.org/zookeeper/docs/current/zookeeperAdmin.html {#httphadoopapacheorgzookeeperdocscurrentzookeeperadminhtml} - -# 每个 tick 的毫秒数 {#the-number-of-milliseconds-of-each-tick} -tickTime=2000 -# 初始同步阶段可以占用的 tick 数量 {#the-number-of-ticks-that-the-initial} -# 此值没有明确的依据 {#synchronization-phase-can-take} -initLimit=300 -# 发送请求和接收确认之间可以经过的 tick 数量 {#this-value-is-not-quite-motivated} -syncLimit=10 - -maxClientCnxns=2000 - -# 这是客户端可以请求且服务器将接受的最大值。 {#the-number-of-ticks-that-can-pass-between} -# 服务器上设置较高的 maxSessionTimeout 是可以的,以便允许客户端在需要时使用较高的会话超时。 {#sending-a-request-and-getting-an-acknowledgement} -# 但默认情况下我们请求 30 秒的会话超时(您可以在 ClickHouse 配置中使用 session_timeout_ms 更改它)。 {#it-is-the-maximum-value-that-client-may-request-and-the-server-will-accept} -maxSessionTimeout=60000000 -# 存储快照的目录。 {#it-is-ok-to-have-high-maxsessiontimeout-on-server-to-allow-clients-to-work-with-high-session-timeout-if-they-want} -dataDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/data -# 将 dataLogDir 放置在单独的物理磁盘上以获得更好的性能 {#but-we-request-session-timeout-of-30-seconds-by-default-you-can-change-it-with-session_timeout_ms-in-clickhouse-config} -dataLogDir=/opt/zookeeper/{{ '{{' }} cluster['name'] {{ '}}' }}/logs - -autopurge.snapRetainCount=10 -autopurge.purgeInterval=1 - - -# 为避免磁盘寻址,ZooKeeper 在事务日志文件中以 preAllocSize 千字节的块分配空间。 {#the-directory-where-the-snapshot-is-stored} -# 默认块大小为 64M。更改块大小的一个原因是在更频繁地创建快照时减小块大小。 {#place-the-datalogdir-to-a-separate-physical-disc-for-better-performance} -# (另请参阅 snapCount)。 {#to-avoid-seeks-zookeeper-allocates-space-in-the-transaction-log-file-in} -preAllocSize=131072 - -# 客户端提交请求的速度可能快于 ZooKeeper 处理它们的速度, {#blocks-of-preallocsize-kilobytes-the-default-block-size-is-64m-one-reason} -# 特别是在有大量客户端的情况下。为防止 ZooKeeper 因排队请求而耗尽内存, {#for-changing-the-size-of-the-blocks-is-to-reduce-the-block-size-if-snapshots} -# ZooKeeper 将限制客户端,使系统中未完成的请求不超过 globalOutstandingLimit。 {#are-taken-more-often-also-see-snapcount} -# 默认限制为 1000。 {#clients-can-submit-requests-faster-than-zookeeper-can-process-them} -# globalOutstandingLimit=1000 {#especially-if-there-are-a-lot-of-clients-to-prevent-zookeeper-from-running} - -# ZooKeeper 将事务记录到事务日志中。在将 snapCount 个事务写入日志文件后, {#out-of-memory-due-to-queued-requests-zookeeper-will-throttle-clients-so-that} -# 将启动快照并启动新的事务日志文件。默认 snapCount 为 100000。 {#there-is-no-more-than-globaloutstandinglimit-outstanding-requests-in-the} -snapCount=3000000 - -# 如果定义了此选项,请求将被记录到名为 traceFile.year.month.day 的跟踪文件中。 {#system-the-default-limit-is-1000} -#traceFile= - -# Leader 接受客户端连接。默认值为 "yes"。Leader 机器协调更新。 {#globaloutstandinglimit1000} -# 为了以略微牺牲读取吞吐量为代价获得更高的更新吞吐量, {#zookeeper-logs-transactions-to-a-transaction-log-after-snapcount-transactions} -# 可以将 leader 配置为不接受客户端并专注于协调。 {#are-written-to-a-log-file-a-snapshot-is-started-and-a-new-transaction-log-file} -leaderServes=yes - -standaloneEnabled=false -dynamicConfigFile=/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/zoo.cfg.dynamic -``` - -Java 版本: - - -```text -openjdk 11.0.5-shenandoah 2019-10-15 -OpenJDK Runtime Environment (build 11.0.5-shenandoah+10-adhoc.heretic.src) -OpenJDK 64-Bit Server VM (build 11.0.5-shenandoah+10-adhoc.heretic.src, mixed mode) -``` - -JVM 参数: - - -```bash -NAME=zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} -ZOOCFGDIR=/etc/$NAME/conf - -# TODO 此处代码较为冗余 {#is-started-the-default-snapcount-is-100000} -# 如何确定所需的 jar 文件? {#if-this-option-is-defined-requests-will-be-will-logged-to-a-trace-file-named} -# log4j 似乎要求 log4j.properties 文件必须位于 classpath 中 {#tracefileyearmonthday} -CLASSPATH="$ZOOCFGDIR:/usr/build/classes:/usr/build/lib/*.jar:/usr/share/zookeeper-3.6.2/lib/audience-annotations-0.5.0.jar:/usr/share/zookeeper-3.6.2/lib/commons-cli-1.2.jar:/usr/share/zookeeper-3.6.2/lib/commons-lang-2.6.jar:/usr/share/zookeeper-3.6.2/lib/jackson-annotations-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-core-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/jackson-databind-2.10.3.jar:/usr/share/zookeeper-3.6.2/lib/javax.servlet-api-3.1.0.jar:/usr/share/zookeeper-3.6.2/lib/jetty-http-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-io-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-security-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-server-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-servlet-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jetty-util-9.4.24.v20191120.jar:/usr/share/zookeeper-3.6.2/lib/jline-2.14.6.jar:/usr/share/zookeeper-3.6.2/lib/json-simple-1.1.1.jar:/usr/share/zookeeper-3.6.2/lib/log4j-1.2.17.jar:/usr/share/zookeeper-3.6.2/lib/metrics-core-3.2.5.jar:/usr/share/zookeeper-3.6.2/lib/netty-buffer-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-codec-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-handler-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-resolver-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-epoll-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/netty-transport-native-unix-common-4.1.50.Final.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_common-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_hotspot-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/simpleclient_servlet-0.6.0.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-api-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/slf4j-log4j12-1.7.25.jar:/usr/share/zookeeper-3.6.2/lib/snappy-java-1.1.7.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-jute-3.6.2.jar:/usr/share/zookeeper-3.6.2/lib/zookeeper-prometheus-metrics-3.6.2.jar:/usr/share/zookeeper-3.6.2/etc" - -ZOOCFG="$ZOOCFGDIR/zoo.cfg" -ZOO_LOG_DIR=/var/log/$NAME -USER=zookeeper -GROUP=zookeeper -PIDDIR=/var/run/$NAME -PIDFILE=$PIDDIR/$NAME.pid -SCRIPTNAME=/etc/init.d/$NAME -JAVA=/usr/local/jdk-11/bin/java -ZOOMAIN="org.apache.zookeeper.server.quorum.QuorumPeerMain" -ZOO_LOG4J_PROP="INFO,ROLLINGFILE" -JMXLOCALONLY=false -JAVA_OPTS="-Xms{{ '{{' }} cluster.get('xms','128M') {{ '}}' }} \ - -Xmx{{ '{{' }} cluster.get('xmx','1G') {{ '}}' }} \ - -Xlog:safepoint,gc*=info,age*=debug:file=/var/log/$NAME/zookeeper-gc.log:time,level,tags:filecount=16,filesize=16M - -verbose:gc \ - -XX:+UseG1GC \ - -Djute.maxbuffer=8388608 \ - -XX:MaxGCPauseMillis=50" -``` - -Salt 初始化: - -```text -description "zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} 集中式协调服务" - -start on runlevel [2345] -stop on runlevel [!2345] - -respawn - -limit nofile 8192 8192 - -pre-start script - [ -r "/etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment" ] || exit 0 - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -d $ZOO_LOG_DIR ] || mkdir -p $ZOO_LOG_DIR - chown $USER:$GROUP $ZOO_LOG_DIR -end script - -script - . /etc/zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }}/conf/environment - [ -r /etc/default/zookeeper ] && . /etc/default/zookeeper - if [ -z "$JMXDISABLE" ]; then - JAVA_OPTS="$JAVA_OPTS -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.local.only=$JMXLOCALONLY" - fi - exec start-stop-daemon --start -c $USER --exec $JAVA --name zookeeper-{{ '{{' }} cluster['name'] {{ '}}' }} \ - -- -cp $CLASSPATH $JAVA_OPTS -Dzookeeper.log.dir=${ZOO_LOG_DIR} \ - -Dzookeeper.root.logger=${ZOO_LOG4J_PROP} $ZOOMAIN $ZOOCFG -end script -``` - - -## Antivirus software {#antivirus-software} - -如果使用杀毒软件,请将其配置为忽略包含 ClickHouse 数据文件(`/var/lib/clickhouse`)的目录,否则可能会导致性能下降,并在数据摄取和后台合并任务过程中出现意外错误。 - -## 相关内容 {#related-content} - -- [刚开始接触 ClickHouse?这里有 13 个“致命错误”以及如何避免它们](https://clickhouse.com/blog/common-getting-started-issues-with-clickhouse) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/update.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/update.md deleted file mode 100644 index 79eed230932..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/update.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -description: '升级文档' -sidebar_title: '自托管升级' -slug: /operations/update -title: '自托管升级' -doc_type: 'guide' ---- - - - -## ClickHouse 升级概览 {#clickhouse-upgrade-overview} - -本文档包含: -- 通用指南 -- 推荐方案 -- 在系统上升级二进制文件的具体说明 - - - -## 一般指南 {#general-guidelines} - -这些说明有助于你进行规划,并理解在文档后面部分我们为何会给出相应的建议。 - -### 将 ClickHouse server 与 ClickHouse Keeper 或 ZooKeeper 分开升级 {#upgrade-clickhouse-server-separately-from-clickhouse-keeper-or-zookeeper} -除非 ClickHouse Keeper 或 Apache ZooKeeper 存在需要修复的安全问题,否则在升级 ClickHouse server 时没有必要同时升级 Keeper。升级过程中需要保持 Keeper 的稳定性,因此应先完成 ClickHouse server 的升级,再考虑升级 Keeper。 - -### 应经常进行小版本升级 {#minor-version-upgrades-should-be-adopted-often} -强烈建议在最新小版本发布后尽快升级到该版本。小版本发布不会包含破坏性变更,但会包含重要的缺陷修复(并且可能包含安全修复)。 - -### 在运行目标版本的独立 ClickHouse server 上测试实验特性 {#test-experimental-features-on-a-separate-clickhouse-server-running-the-target-version} - -实验特性的兼容性可能在任何时间、以任何方式被破坏。如果你在使用实验特性,请检查变更日志,并考虑搭建一个安装了目标版本的独立 ClickHouse server,在该实例上测试你对实验特性的使用情况。 - -### 降级 {#downgrades} -如果你升级后发现新版本与某些你依赖的特性不兼容,并且你尚未开始使用任何新特性,那么你可能可以降级到一个最近的版本(不超过一年前的版本)。一旦开始使用新特性,降级将无法进行。 - -### 集群中存在多个 ClickHouse server 版本 {#multiple-clickhouse-server-versions-in-a-cluster} - -我们会努力维持一年的兼容性窗口(其中包含 2 个 LTS 版本)。这意味着只要两个版本之间的发布时间差小于一年(或它们之间少于两个 LTS 版本),这两个版本就应该能够在同一集群中协同工作。不过,仍然建议尽快将集群中所有成员升级到相同版本,因为可能会出现一些小问题(例如分布式查询变慢、ReplicatedMergeTree 中某些后台操作出现可重试错误等)。 - -我们从不建议在同一集群中运行发布时间相差超过一年的不同版本。虽然我们不预期会发生数据丢失,但集群可能会变得不可用。如果版本之间相差超过一年,你应预期会遇到的问题包括: - -- 集群可能无法正常工作 -- 部分(甚至全部)查询可能会因各种错误而失败 -- 日志中可能出现各种错误/警告 -- 可能无法执行降级 - -### 增量升级 {#incremental-upgrades} - -如果当前版本与目标版本之间的差异超过一年,建议采取以下两种方式之一: -- 通过停机升级(停止所有 server、升级所有 server、重新启动所有 server)。 -- 或通过中间版本进行升级(选择一个比当前版本新但发布时间不超过一年的版本作为中间版本)。 - - - -## 推荐方案 {#recommended-plan} - -以下是实现 ClickHouse 零停机升级的推荐步骤: - -1. 确保你的配置更改不在默认的 `/etc/clickhouse-server/config.xml` 文件中,而是放在 `/etc/clickhouse-server/config.d/` 中,因为 `/etc/clickhouse-server/config.xml` 在升级过程中可能会被覆盖。 -2. 通读 [更新日志](/whats-new/changelog/index.md),查找不兼容变更(从目标版本往回查看到你当前使用的版本)。 -3. 根据不兼容变更说明,对可以在升级前完成的修改先行调整,并整理出升级完成后仍需进行的变更清单。 -4. 为每个分片确定一个或多个在升级期间保持运行的副本,同时升级该分片的其他副本。 -5. 对于要升级的副本,逐个执行: - -* 关闭 ClickHouse 服务器 -* 将服务器升级到目标版本 -* 启动 ClickHouse 服务器 -* 等待 Keeper 消息表明系统已稳定 -* 继续处理下一个副本6. 检查 Keeper 日志和 ClickHouse 日志中的错误 - -7. 将步骤 4 中确定的副本升级到新版本。 -8. 参考步骤 1 至 3 中整理的变更清单,执行需要在升级后完成的变更。 - -:::note -在复制环境中同时运行多个版本的 ClickHouse 时,出现此错误消息是预期行为。当所有副本都升级到同一版本后,该错误消息将不再出现。 - -```text -MergeFromLogEntryTask: Code: 40. DB::Exception: 数据分片校验和不匹配: -未压缩文件哈希值不匹配。(CHECKSUM_DOESNT_MATCH) 合并后的数据与其他副本的数据不完全一致。 -``` - -::: - - -## ClickHouse 服务器二进制升级流程 {#clickhouse-server-binary-upgrade-process} - -如果 ClickHouse 是通过 `deb` 软件包安装的,请在服务器上执行以下命令: - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-client clickhouse-server -$ sudo service clickhouse-server restart -``` - -如果你不是使用推荐的 `deb` 软件包来安装 ClickHouse,请使用相应的更新方法。 - -:::note -只要确保不存在某个分片的所有副本同时离线的时刻,你就可以一次性更新多个服务器。 -::: - -将旧版本的 ClickHouse 升级到指定版本: - -例如: - -`xx.yy.a.b` 是当前的稳定版本之一。最新的稳定版本可以在[这里](https://github.com/ClickHouse/ClickHouse/releases)找到。 - -```bash -$ sudo apt-get update -$ sudo apt-get install clickhouse-server=xx.yy.a.b clickhouse-client=xx.yy.a.b clickhouse-common-static=xx.yy.a.b -$ sudo service clickhouse-server restart -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md deleted file mode 100644 index 8a3a2558d92..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/userspace-page-cache.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: '一种缓存机制,可将数据缓存在进程内存中,而无需依赖操作系统的页面缓存。' -sidebar_label: '用户态页面缓存' -sidebar_position: 65 -slug: /operations/userspace-page-cache -title: '用户态页面缓存' -doc_type: 'reference' ---- - - - -# 用户态页缓存 {#userspace-page-cache} - - - -## 概览 {#overview} - -> 用户态页缓存是一种新的缓存机制,它允许将数据缓存到进程内存中,而不是依赖操作系统的页缓存。 - -ClickHouse 已经提供了 [Filesystem cache](/docs/operations/storing-data), -可用于在 Amazon S3、Google Cloud Storage (GCS) 或 Azure Blob Storage 等远程对象存储之上进行缓存。用户态页缓存旨在当常规操作系统缓存效果不佳时,加速对远程数据的访问。 - -它与文件系统缓存的不同之处在于: - -| Filesystem cache | 用户态页缓存 | -|---------------------------------------------------------|---------------------------------------| -| 将数据写入本地文件系统 | 仅存在于内存中 | -| 占用磁盘空间(也可在 tmpfs 上配置) | 独立于文件系统 | -| 在服务器重启后仍然存在 | 在服务器重启后不会保留 | -| 不会体现在服务器的内存使用中 | 会体现在服务器的内存使用中 | -| 适用于基于磁盘和基于内存(操作系统页缓存)的场景 | **适用于无磁盘服务器** | - - - -## 配置与使用 {#configuration-settings-and-usage} - -### 用法 {#usage} - -要启用用户态页缓存,首先需要在服务器上完成相应配置: - -```bash -cat config.d/page_cache.yaml -page_cache_max_size: 100G -``` - -:::note -用户态页缓存最多会使用指定数量的内存,但 -这部分内存并不会被预留。当服务器中其他组件需要内存时, -这些内存会被回收。 -::: - -接下来,在查询级别启用它: - -```sql -SET use_page_cache_for_disks_without_file_cache=1; -``` - -### 设置 {#settings} - -| Setting | Description | Default | -| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------- | -| `use_page_cache_for_disks_without_file_cache` | 对未启用文件系统缓存的远程磁盘使用用户态页缓存。 | `0` | -| `use_page_cache_with_distributed_cache` | 在使用分布式缓存时使用用户态页缓存。 | `0` | -| `read_from_page_cache_if_exists_otherwise_bypass_cache` | 以被动模式使用用户态页缓存,行为类似于 [`read_from_filesystem_cache_if_exists_otherwise_bypass_cache`](/docs/operations/settings/settings#read_from_filesystem_cache_if_exists_otherwise_bypass_cache)。 | `0` | -| `page_cache_inject_eviction` | 用户态页缓存会随机使部分页失效。用于测试目的。 | `0` | -| `page_cache_block_size` | 在用户态页缓存中存储的文件块大小(字节)。所有经过缓存的读取都会向上取整为该大小的倍数。 | `1048576` | -| `page_cache_history_window_ms` | 释放的内存在可被用户态页缓存重新使用前的延迟时间。 | `1000` | -| `page_cache_policy` | 用户态页缓存策略名称。 | `SLRU` | -| `page_cache_size_ratio` | 用户态页缓存中受保护队列相对于缓存总大小的比例。 | `0.5` | -| `page_cache_min_size` | 用户态页缓存的最小大小。 | `104857600` | -| `page_cache_max_size` | 用户态页缓存的最大大小。设置为 0 时禁用缓存。如果该值大于 `page_cache_min_size`,则缓存大小会在此区间内持续调整,以在保持总内存使用低于限制(`max_server_memory_usage`[`_to_ram_ratio`])的前提下尽可能利用可用内存。 | `0` | -| `page_cache_free_memory_ratio` | 在用户态页缓存中需要保持空闲的内存限额比例。类似于 Linux 的 `min_free_kbytes` 设置。 | `0.15` | -| `page_cache_lookahead_blocks` | 当用户态页缓存未命中时,如果后续连续块也不在缓存中,则一次性从底层存储中预读最多这么多连续块。每个块的大小为 `page_cache_block_size` 字节。 | `16` | -| `page_cache_shards` | 将用户态页缓存划分为相应数量的分片(shard),以减少互斥锁竞争。为实验性功能,不太可能提升性能。 | `4` | - - -## 相关内容 {#related-content} -- [文件系统缓存](/docs/operations/storing-data) -- [ClickHouse v25.3 版本发布网络研讨会](https://www.youtube.com/live/iCKEzp0_Z2Q?feature=shared&t=1320) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md deleted file mode 100644 index 18ae0dff315..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/backupview.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: 'clickhouse_backupview 文档 {#clickhouse_backupview}' -slug: /operations/utilities/backupview -title: 'clickhouse_backupview' -doc_type: 'reference' ---- - - - -# clickhouse_backupview {#clickhouse_backupview} - -用于分析由 [BACKUP](/operations/backup) 命令创建的备份的 Python 模块。 -主要目的是在不实际恢复备份的情况下,从备份中获取部分信息。 - -该模块提供的功能包括: -- 枚举备份中包含的文件 -- 读取备份中的文件 -- 以可读形式获取有关备份中包含的数据库、表和数据分片的有用信息 -- 检查备份的完整性 - - - -## 示例: {#example} - -```python -from clickhouse_backupview import open_backup, S3, FileInfo -``` - - -# 打开备份。同样也可以使用本地路径: {#open-a-backup-we-could-also-use-a-local-path} -# backup = open_backup("/backups/my_backup_1/") {#backup-open_backupbackupsmy_backup_1} -backup = open_backup(S3("uri", "access_key_id", "secret_access_key")) - - - -# 获取备份中的数据库列表。 {#get-a-list-of-databasess-inside-the-backup} -print(backup.get_databases())) - - - -# 获取备份中的表列表, {#get-a-list-of-tables-inside-the-backup} -# 并针对每个表输出其创建语句以及分区和数据分片列表。 {#and-for-each-table-its-create-query-and-a-list-of-parts-and-partitions} -for db in backup.get_databases(): - for tbl in backup.get_tables(database=db): - print(backup.get_create_query(database=db, table=tbl)) - print(backup.get_partitions(database=db, table=tbl)) - print(backup.get_parts(database=db, table=tbl)) - - - -# 从备份中提取全部内容。 {#extract-everything-from-the-backup} -backup.extract_all(table="mydb.mytable", out='/tmp/my_backup_1/all/') - - - -# 提取指定表的数据。 {#extract-the-data-of-a-specific-table} -backup.extract_table_data(table="mydb.mytable", out='/tmp/my_backup_1/mytable/') - - - -# 导出单个分区的数据。 {#extract-a-single-partition} -backup.extract_table_data(table="mydb.mytable", partition="202201", out='/tmp/my_backup_1/202201/') - - - -# 提取单个数据分片。 {#extract-a-single-part} - -backup.extract_table_data(table="mydb.mytable", part="202201_100_200_3", out='/tmp/my_backup_1/202201_100_200_3/') - -``` - -更多示例请参阅 [test](https://github.com/ClickHouse/ClickHouse/blob/master/utils/backupview/test/test.py)。 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md deleted file mode 100644 index da316188d7a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-benchmark.md +++ /dev/null @@ -1,172 +0,0 @@ ---- -description: 'clickhouse-benchmark 文档' -sidebar_label: 'clickhouse-benchmark' -sidebar_position: 61 -slug: /operations/utilities/clickhouse-benchmark -title: 'clickhouse-benchmark' -doc_type: 'reference' ---- - - - -# clickhouse-benchmark {#clickhouse-benchmark} - -连接到 ClickHouse 服务器并反复执行指定查询。 - -**语法** - -```bash -$ clickhouse-benchmark --query ["single query"] [keys] -``` - -或 - -```bash -$ echo "单条查询" | clickhouse-benchmark [keys] -``` - -或 - -```bash -$ clickhouse-benchmark [keys] <<< "单条查询语句" -``` - -如果你想发送一组查询,请创建一个文本文件,并在该文件中将每条查询语句放在单独的一行。例如: - -```sql -SELECT * FROM system.numbers LIMIT 10000000; -SELECT 1; -``` - -然后通过标准输入将该文件传递给 `clickhouse-benchmark`: - -```bash -clickhouse-benchmark [keys] < queries_file; -``` - - -## 命令行选项 {#clickhouse-benchmark-command-line-options} - -- `--query=QUERY` — 要执行的查询。如果未传递此参数,`clickhouse-benchmark` 将从标准输入读取查询。 -- `--query_id=ID` — 查询 ID。 -- `--query_id_prefix=ID_PREFIX` — 查询 ID 前缀。 -- `-c N`, `--concurrency=N` — `clickhouse-benchmark` 同时发送的查询数量。默认值:1。 -- `-C N`, `--max_concurrency=N` — 逐步增加并行查询数量直至指定值,并为每个并发级别生成一份报告。 -- `--precise` — 启用带加权指标的精确分段报告。 -- `-d N`, `--delay=N` — 中间报告之间的时间间隔(秒)(要禁用报告请设为 0)。默认值:1。 -- `-h HOST`, `--host=HOST` — 服务器主机名。默认值:`localhost`。在[对比模式](#clickhouse-benchmark-comparison-mode)下可以使用多个 `-h` 选项。 -- `-i N`, `--iterations=N` — 查询总数。默认值:0(无限重复)。 -- `-r`, `--randomize` — 当输入中有多个查询时,以随机顺序执行查询。 -- `-s`, `--secure` — 使用 `TLS` 连接。 -- `-t N`, `--timelimit=N` — 时间限制(秒)。达到指定时间限制后,`clickhouse-benchmark` 停止发送查询。默认值:0(禁用时间限制)。 -- `--port=N` — 服务器端口。默认值:9000。在[对比模式](#clickhouse-benchmark-comparison-mode)下可以使用多个 `--port` 选项。 -- `--confidence=N` — t 检验的置信水平。可选值:0 (80%)、1 (90%)、2 (95%)、3 (98%)、4 (99%)、5 (99.5%)。默认值:5。在[对比模式](#clickhouse-benchmark-comparison-mode)下,`clickhouse-benchmark` 会执行[双样本独立 Student t 检验](https://en.wikipedia.org/wiki/Student%27s_t-test#Independent_two-sample_t-test),以在选定置信水平下判断两个分布是否可以视为无显著差异。 -- `--cumulative` — 输出累积数据,而不是分段数据。 -- `--database=DATABASE_NAME` — ClickHouse 数据库名。默认值:`default`。 -- `--user=USERNAME` — ClickHouse 用户名。默认值:`default`。 -- `--password=PSWD` — ClickHouse 用户密码。默认值:空字符串。 -- `--stacktrace` — 输出堆栈跟踪。设置此选项后,`clickhouse-benchmark` 会输出异常的堆栈跟踪。 -- `--stage=WORD` — 服务器端的查询处理阶段。ClickHouse 会在指定阶段停止处理查询并向 `clickhouse-benchmark` 返回结果。可选值:`complete`、`fetch_columns`、`with_mergeable_state`。默认值:`complete`。 -- `--roundrobin` — 不对不同 `--host`/`--port` 的查询进行比较,而是为每个查询随机选择一个 `--host`/`--port` 并将查询发送到该地址。 -- `--reconnect=N` — 控制重连行为。可选值:0(从不重连)、1(每个查询都重连),或 N(每 N 个查询重连一次)。默认值:0。 -- `--max-consecutive-errors=N` — 允许的连续错误数量。默认值:0。 -- `--ignore-error`, `--continue_on_errors` — 即使查询失败也继续测试。 -- `--client-side-time` — 显示包含网络通信时间的耗时,而不是服务器端时间;注意,在 22.8 之前的服务器版本中,总是显示客户端时间。 -- `--proto-caps` — 启用/禁用数据传输中的分块。可选值(可用逗号分隔):`chunked_optional`、`notchunked`、`notchunked_optional`、`send_chunked`、`send_chunked_optional`、`send_notchunked`、`send_notchunked_optional`、`recv_chunked`、`recv_chunked_optional`、`recv_notchunked`、`recv_notchunked_optional`。默认值:`notchunked`。 -- `--help` — 显示帮助信息。 -- `--verbose` — 增加帮助信息的详细程度。 - -如果希望为查询应用一些[设置](/operations/settings/overview),可以以 `--= SETTING_VALUE` 的形式传递它们。例如:`--max_memory_usage=1048576`。 - - - -## 环境变量选项 {#clickhouse-benchmark-environment-variable-options} - -可以通过环境变量 `CLICKHOUSE_USER`、`CLICKHOUSE_PASSWORD` 和 `CLICKHOUSE_HOST` 设置用户名、密码和主机。 -命令行参数 `--user`、`--password` 或 `--host` 优先于环境变量。 - - - -## 输出 {#clickhouse-benchmark-output} - -默认情况下,`clickhouse-benchmark` 会在每个 `--delay` 时间间隔输出一份报告。 - -报告示例: - -```text -已执行查询数:10。 - -localhost:9000, 查询数 10,QPS: 6.772, RPS: 67904487.440, MiB/s: 518.070, result RPS: 67721584.984, result MiB/s: 516.675. - -0.000% 0.145 秒 -10.000% 0.146 秒 -20.000% 0.146 秒 -30.000% 0.146 秒 -40.000% 0.147 秒 -50.000% 0.148 秒 -60.000% 0.148 秒 -70.000% 0.148 秒 -80.000% 0.149 秒 -90.000% 0.150 秒 -95.000% 0.150 秒 -99.000% 0.150 秒 -99.900% 0.150 秒 -99.990% 0.150 秒 -``` - -在报告中可以看到: - -* `Queries executed:` 字段中的查询数量。 - -* 状态字符串包含(按顺序): - - * ClickHouse 服务器的端点。 - * 已处理的查询数量。 - * QPS:在 `--delay` 参数指定的时间段内,服务器每秒执行的查询数。 - * RPS:在 `--delay` 参数指定的时间段内,服务器每秒读取的行数。 - * MiB/s:在 `--delay` 参数指定的时间段内,服务器每秒读取的数据量(以 MiB 为单位)。 - * result RPS:在 `--delay` 参数指定的时间段内,服务器每秒写入到查询结果中的行数。 - * result MiB/s:在 `--delay` 参数指定的时间段内,服务器每秒写入到查询结果中的数据量(以 MiB 为单位)。 - -* 查询执行时间的分位数(百分位)。 - - -## 比较模式 {#clickhouse-benchmark-comparison-mode} - -`clickhouse-benchmark` 可以比较两个正在运行的 ClickHouse 服务器的性能。 - -要使用比较模式,请通过两组 `--host` 和 `--port` 参数来指定两个服务器的端点。参数按照它们在参数列表中的顺序进行一一匹配,第一个 `--host` 与第一个 `--port` 匹配,依此类推。`clickhouse-benchmark` 会同时与这两个服务器建立连接,然后发送查询。每条查询都会被发送到随机选取的其中一个服务器。结果会以表格形式显示。 - - - -## 示例 {#clickhouse-benchmark-example} - -```bash -$ echo "SELECT * FROM system.numbers LIMIT 10000000 OFFSET 10000000" | clickhouse-benchmark --host=localhost --port=9001 --host=localhost --port=9000 -i 10 -``` - -```text -已加载 1 个查询。 - -已执行的查询数:5 个。 - -localhost:9001,2 个查询,QPS:3.764,RPS:75446929.370,MiB/s:575.614,结果 RPS:37639659.982,结果 MiB/s:287.168。 -localhost:9000,3 个查询,QPS:3.815,RPS:76466659.385,MiB/s:583.394,结果 RPS:38148392.297,结果 MiB/s:291.049。 - -0.000% 0.258 秒 0.250 秒 -10.000% 0.258 秒 0.250 秒 -20.000% 0.258 秒 0.250 秒 -30.000% 0.258 秒 0.267 秒 -40.000% 0.258 秒 0.267 秒 -50.000% 0.273 秒 0.267 秒 -60.000% 0.273 秒 0.267 秒 -70.000% 0.273 秒 0.267 秒 -80.000% 0.273 秒 0.269 秒 -90.000% 0.273 秒 0.269 秒 -95.000% 0.273 秒 0.269 秒 -99.000% 0.273 秒 0.269 秒 -99.900% 0.273 秒 0.269 秒 -99.990% 0.273 秒 0.269 秒 - -在 99.5% 的置信度下,未能证明存在差异 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md deleted file mode 100644 index 6e8c96a2dc5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-compressor.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'ClickHouse Compressor 文档' -slug: /operations/utilities/clickhouse-compressor -title: 'clickhouse-compressor' -doc_type: 'reference' ---- - -用于数据压缩和解压缩的简单程序。 - -### 示例 {#examples} - -使用 LZ4 压缩数据: - -```bash -$ ./clickhouse-compressor < input_file > output_file -``` - -将 LZ4 格式的数据解压缩: - -```bash -$ ./clickhouse-compressor --decompress < input_file > output_file -``` - -使用 ZSTD 以压缩级别 5 压缩数据: - -```bash -$ ./clickhouse-compressor --codec 'ZSTD(5)' < input_file > output_file -``` - -使用 4 字节 Delta 和 ZSTD 级别 10 对数据进行压缩。 - -```bash -$ ./clickhouse-compressor --codec 'Delta(4)' --codec 'ZSTD(10)' < input_file > output_file -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md deleted file mode 100644 index f6301cd36cf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-disks.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse-disks 文档' -sidebar_label: 'clickhouse-disks' -sidebar_position: 59 -slug: /operations/utilities/clickhouse-disks -title: 'Clickhouse-disks' -doc_type: 'reference' ---- - - - -# Clickhouse-disks {#clickhouse-disks} - -一个为 ClickHouse 磁盘提供类文件系统操作的实用工具。它同时支持交互式和非交互式模式。 - - - -## 程序级通用选项 {#program-wide-options} - -* `--config-file, -C` -- ClickHouse 配置文件路径,默认为 `/etc/clickhouse-server/config.xml`。 -* `--save-logs` -- 将已执行命令的进度记录到 `/var/log/clickhouse-server/clickhouse-disks.log`。 -* `--log-level` -- 要记录的事件[类型](../server-configuration-parameters/settings#logger),默认为 `none`。 -* `--disk` -- 用于 `mkdir, move, read, write, remove` 命令的磁盘,默认为 `default`。 -* `--query, -q` -- 可在不启动交互模式的情况下执行的一条查询。 -* `--help, -h` -- 显示所有选项和命令及其说明。 - - - -## 延迟初始化 {#lazy-initialization} -配置中可用的所有磁盘都会采用延迟初始化方式。也就是说,只有在某个命令实际使用到某个磁盘时,才会为该磁盘初始化对应的对象。这样做是为了提高工具的健壮性,并避免去操作那些虽然在配置中声明、但用户并未使用、且可能在初始化过程中失败的磁盘。不过,在启动 clickhouse-disks 时,必须有一个磁盘会被立即初始化。该磁盘通过命令行参数 `--disk` 指定(默认值为 `default`)。 - - - -## 默认磁盘 {#default-disks} -启动后,会有两个未在配置中显式指定但可用于初始化的磁盘。 - -1. **`local` 磁盘**:该磁盘用于模拟启动 `clickhouse-disks` 工具时所在的本地文件系统。它的初始路径为启动 `clickhouse-disks` 时所在的目录,并挂载在文件系统根目录。 - -2. **`default` 磁盘**:该磁盘挂载到本地文件系统中由配置参数 `clickhouse/path` 指定的目录(默认值为 `/var/lib/clickhouse`)。它的初始路径被设置为 `/`。 - - - -## Clickhouse-disks 状态 {#clickhouse-disks-state} -对于添加的每个磁盘,该工具都会存储当前目录(类似于普通文件系统)。用户可以更改当前目录并在各个磁盘之间切换。 - -状态会显示在提示符中 "`disk_name`:`path_name`" - - - -## 命令 {#commands} - -在本说明文档中,所有必需的位置参数记作 ``,具名参数记作 `[--parameter value]`。所有位置参数都可以使用对应名称作为具名参数来指定。 - -* `cd (change-dir, change_dir) [--disk disk] ` - 将当前目录切换到磁盘 `disk` 上的路径 `path`(默认值为当前磁盘)。不会发生磁盘切换。 -* `copy (cp) [--disk-from disk_1] [--disk-to disk_2] `. - 递归地将磁盘 `disk_1` 上路径 `path-from` 的数据(默认值为当前磁盘(在非交互模式下为参数 `disk` 的值)) - 复制到磁盘 `disk_2` 上的路径 `path-to`(默认值为当前磁盘(在非交互模式下为参数 `disk` 的值))。 -* `current_disk_with_path (current, current_disk, current_path)` - 以如下格式打印当前状态: - `Disk: "current_disk" Path: "current path on current disk"` -* `help []` - 打印关于命令 `command` 的帮助信息。如果未指定 `command`,则打印所有命令的相关信息。 -* `move (mv) `. - 在当前磁盘内将文件或目录从 `path-from` 移动到 `path-to`。 -* `remove (rm, delete) `. - 在当前磁盘上递归删除 `path`。 -* `link (ln) `. - 在当前磁盘上从 `path-from` 到 `path-to` 创建一个硬链接。 -* `list (ls) [--recursive] ` - 列出当前磁盘上路径 `path` 下的文件。默认非递归。 -* `list-disks (list_disks, ls-disks, ls_disks)`. - 列出磁盘名称。 -* `mkdir [--recursive] ` 在当前磁盘上。 - 创建目录。默认非递归。 -* `read (r) [--path-to path]` - 将文件从 `path-from` 读取到 `path`(如果未提供则输出到 `stdout`)。 -* `switch-disk [--path path] ` - 在路径 `path` 上切换到磁盘 `disk`(如果未指定 `path`,默认值为之前在磁盘 `disk` 上的路径)。 -* `write (w) [--path-from path] `. - 将文件从 `path`(如果未提供 `path` 则从 `stdin` 读取,输入必须以 Ctrl+D 结束)写入到 `path-to`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md deleted file mode 100644 index e6ca4c07629..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-format.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: '使用 format 工具处理 ClickHouse 数据格式的指南' -slug: /operations/utilities/clickhouse-format -title: 'clickhouse-format' -doc_type: 'reference' ---- - -# clickhouse-format 实用工具 {#clickhouse-format-utility} - -用于格式化输入的查询语句。 - -参数: - -* `--help` 或 `-h` — 输出帮助信息。 -* `--query` — 格式化任意长度和复杂度的查询。 -* `--hilite` 或 `--highlight` — 使用 ANSI 终端转义序列添加语法高亮。 -* `--oneline` — 格式化为单行。 -* `--max_line_length` — 将长度小于指定值的查询格式化为单行。 -* `--comments` — 在输出中保留注释。 -* `--quiet` 或 `-q` — 仅检查语法,成功时不输出任何内容。 -* `--multiquery` 或 `-n` — 允许在同一文件中包含多个查询。 -* `--obfuscate` — 执行混淆而不是格式化。 -* `--seed <string>` — 指定任意字符串作为种子,用于决定混淆结果。 -* `--backslash` — 在格式化后的查询每一行末尾添加反斜杠。当你从网页或其他地方复制多行查询并希望在命令行中执行时,这会很有用。 -* `--semicolons_inline` — 在多查询模式下,将分号写在查询的最后一行,而不是单独起一行。 - -## 示例 {#examples} - -1. 格式化查询: - -```bash -$ clickhouse-format --query "select number from numbers(10) where number%2 order by number desc;" -``` - -结果: - -```bash -SELECT number -FROM numbers(10) -WHERE number % 2 -ORDER BY number DESC -``` - -2. 高亮显示与单行: - -```bash -$ clickhouse-format --oneline --hilite <<< "SELECT sum(number) FROM numbers(5);" -``` - -结果: - -```sql -SELECT sum(number) FROM numbers(5) -``` - -3. 多重查询: - -```bash -$ clickhouse-format -n <<< "SELECT min(number) FROM numbers(5); SELECT max(number) FROM numbers(5);" -``` - -结果: - -```sql -SELECT min(number) -FROM numbers(5) -; - -SELECT max(number) -FROM numbers(5) -; - -``` - -4. 混淆处理: - -```bash -$ clickhouse-format --seed Hello --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -结果: - -```sql -SELECT treasury_mammoth_hazelnut 在 nutmeg 和 span 之间,CASE WHEN chive >= 116 THEN switching ELSE ANYTHING END; -``` - -相同的查询,但使用另一个种子字符串: - -```bash -$ clickhouse-format --seed World --obfuscate <<< "SELECT cost_first_screen BETWEEN a AND b, CASE WHEN x >= 123 THEN y ELSE NULL END;" -``` - -结果: - -```sql -SELECT horse_tape_summer BETWEEN folklore AND moccasins, CASE WHEN intestine >= 116 THEN nonconformist ELSE FORESTRY END; -``` - -5. 添加反斜杠: - -```bash -$ clickhouse-format --backslash <<< "SELECT * FROM (SELECT 1 AS x UNION ALL SELECT 1 UNION DISTINCT SELECT 3);" -``` - -结果: - -```sql -SELECT * \ -FROM \ -( \ - SELECT 1 AS x \ - UNION ALL \ - SELECT 1 \ - UNION DISTINCT \ - SELECT 3 \ -) -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md deleted file mode 100644 index 012a94b276d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-keeper-client.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: 'ClickHouse Keeper 客户端工具文档' -sidebar_label: 'clickhouse-keeper-client' -slug: /operations/utilities/clickhouse-keeper-client -title: 'clickhouse-keeper-client 工具' -doc_type: 'reference' ---- - - - -# clickhouse-keeper-client 工具 {#clickhouse-keeper-client-utility} - -一个通过其原生协议与 clickhouse-keeper 交互的客户端工具。 - - - -## 参数 {#clickhouse-keeper-client} - -- `-q QUERY`, `--query=QUERY` — 要执行的查询。若未指定此参数,`clickhouse-keeper-client` 将以交互模式启动。 -- `-h HOST`, `--host=HOST` — 服务器地址。默认值:`localhost`。 -- `-p N`, `--port=N` — 服务器端口。默认值:9181 -- `-c FILE_PATH`, `--config-file=FILE_PATH` — 设置配置文件路径以获取连接字符串。默认值:`config.xml`。 -- `--connection-timeout=TIMEOUT` — 设置连接超时时间(以秒为单位)。默认值:10s。 -- `--session-timeout=TIMEOUT` — 设置会话超时时间(以秒为单位)。默认值:10s。 -- `--operation-timeout=TIMEOUT` — 设置操作超时时间(以秒为单位)。默认值:10s。 -- `--history-file=FILE_PATH` — 设置历史记录文件路径。默认值:`~/.keeper-client-history`。 -- `--log-level=LEVEL` — 设置日志级别。默认值:`information`。 -- `--no-confirmation` — 如果启用,则在若干命令上不再需要确认。交互模式下的默认值为 `false`,查询模式下的默认值为 `true`。 -- `--help` — 显示帮助信息。 - - - -## 示例 {#clickhouse-keeper-client-example} - -```bash -./clickhouse-keeper-client -h localhost -p 9181 --connection-timeout 30 --session-timeout 30 --operation-timeout 30 -已连接到 ZooKeeper,地址为 [::1]:9181,会话 ID 为 137 -/ :) ls -keeper foo bar -/ :) cd 'keeper' -/keeper :) ls -api_version -/keeper :) cd 'api_version' -/keeper/api_version :) ls - -/keeper/api_version :) cd 'xyz' -路径 /keeper/api_version/xyz 不存在 -/keeper/api_version :) cd ../../ -/ :) ls -keeper foo bar -/ :) get 'keeper/api_version' -2 -``` - - -## 命令 {#clickhouse-keeper-client-commands} - -- `ls '[path]'` -- 列出指定路径下的节点(默认:当前工作目录) -- `cd '[path]'` -- 切换工作路径(默认 `.`) -- `cp '' ''` -- 将 `src` 节点复制到 `dest` 路径 -- `cpr '' ''` -- 将 `src` 节点的子树复制到 `dest` 路径 -- `mv '' ''` -- 将 `src` 节点移动到 `dest` 路径 -- `mvr '' ''` -- 将 `src` 节点的子树移动到 `dest` 路径 -- `exists ''` -- 如果节点存在返回 `1`,否则返回 `0` -- `set '' [version]` -- 更新节点的值。仅当版本匹配时才更新(默认:-1) -- `create '' [mode]` -- 使用给定值创建新节点 -- `touch ''` -- 创建一个值为空字符串的新节点。如果节点已存在则不会抛出异常 -- `get ''` -- 返回节点的值 -- `rm '' [version]` -- 仅在版本匹配时删除该节点(默认:-1) -- `rmr '' [limit]` -- 当子树大小小于给定限制时递归删除路径。需要确认(默认限制 = 100) -- `flwc ` -- 执行 four-letter-word 命令 -- `help` -- 显示本帮助信息 -- `get_direct_children_number '[path]'` -- 获取指定路径下直接子节点的数量 -- `get_all_children_number '[path]'` -- 获取指定路径下所有子节点的数量 -- `get_stat '[path]'` -- 返回节点的状态信息(默认 `.`) -- `find_super_nodes '[path]'` -- 查找在给定路径下子节点数量大于阈值的节点(默认 `.`) -- `delete_stale_backups` -- 删除用于备份但已处于非活动状态的 ClickHouse 节点 -- `find_big_family [path] [n]` -- 返回子树中子节点数量最多的前 n 个节点(默认路径 = `.`,n = 10) -- `sync ''` -- 在进程与 leader 之间同步节点 -- `reconfig "" [version]` -- 重新配置 Keeper 集群。参见 /docs/en/guides/sre/keeper/clickhouse-keeper#reconfiguration diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md deleted file mode 100644 index daa89281540..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-local.md +++ /dev/null @@ -1,323 +0,0 @@ ---- -description: '使用 clickhouse-local 在无需 ClickHouse 服务器的情况下处理数据的指南' -sidebar_label: 'clickhouse-local' -sidebar_position: 60 -slug: /operations/utilities/clickhouse-local -title: 'clickhouse-local' -doc_type: 'reference' ---- - - - -# clickhouse-local {#clickhouse-local} - - - -## 何时使用 clickhouse-local 而不是 ClickHouse {#when-to-use-clickhouse-local-vs-clickhouse} - -`clickhouse-local` 是一个易于使用的 ClickHouse 版本,非常适合需要在本地和远程文件上使用 SQL 进行快速处理、且不必安装完整数据库服务器的开发人员。借助 `clickhouse-local`,开发人员可以直接在命令行中使用 SQL 命令(使用 [ClickHouse SQL 方言](../../sql-reference/index.md)),以一种简单高效的方式访问 ClickHouse 功能,而无需完整安装 ClickHouse。`clickhouse-local` 的主要优势之一是,它在安装 [clickhouse-client](/operations/utilities/clickhouse-local) 时已一并包含。这意味着开发人员可以快速开始使用 `clickhouse-local`,而不需要经历复杂的安装过程。 - -尽管 `clickhouse-local` 非常适合用于开发、测试以及文件处理,但它并不适合直接为终端用户或应用程序提供服务。在这些场景中,推荐使用开源的 [ClickHouse](/install)。ClickHouse 是一款功能强大的 OLAP 数据库,专为处理大规模分析型负载而设计。它能够对大型数据集上的复杂查询进行快速高效的处理,非常适合对高性能要求严格的生产环境。此外,ClickHouse 还提供了诸如复制、分片和高可用性等丰富特性,这些对于扩展以处理大规模数据集并为应用程序提供服务至关重要。如果你需要处理更大规模的数据集,或需要为终端用户或应用程序提供服务,我们建议使用开源的 ClickHouse,而不是 `clickhouse-local`。 - -请阅读下面的文档,这些文档展示了 `clickhouse-local` 的示例用例,例如[查询本地文件](#query_data_in_file)或[读取 S3 中的 Parquet 文件](#query-data-in-a-parquet-file-in-aws-s3)。 - - - -## 下载 clickhouse-local {#download-clickhouse-local} - -`clickhouse-local` 是通过与 ClickHouse 服务器和 `clickhouse-client` 相同的 `clickhouse` 二进制程序来执行的。下载最新版本最简单的方法是运行以下命令: - -```bash -curl https://clickhouse.com/ | sh -``` - -:::note -您刚刚下载的二进制文件可以运行各种 ClickHouse 工具和实用工具。如果您想以数据库服务器的方式运行 ClickHouse,请查看 [快速入门](/get-started/quick-start)。 -::: - - -## 使用 SQL 查询文件中的数据 {#query_data_in_file} - -`clickhouse-local` 的一个常见用途是对文件运行即席查询:这样你无需先将数据插入到表中。`clickhouse-local` 可以将文件中的数据以流式方式导入到一个临时表中,并执行你的 SQL。 - -如果文件与 `clickhouse-local` 位于同一台机器上,你只需指定要加载的文件即可。下面的 `reviews.tsv` 文件包含一部分 Amazon 商品评论示例: - -```bash -./clickhouse local -q "SELECT * FROM 'reviews.tsv'" -``` - -此命令是以下命令的简写: - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv')" -``` - -ClickHouse 会根据文件扩展名判断该文件使用的是制表符分隔格式。如果你需要显式指定格式,只需添加其中一种[ClickHouse 支持的众多输入格式](../../interfaces/formats.md): - -```bash -./clickhouse local -q "SELECT * FROM file('reviews.tsv', 'TabSeparated')" -``` - -`file` 表函数会创建一个表,你可以使用 `DESCRIBE` 查看推断出的表结构:` - -```bash -./clickhouse local -q "DESCRIBE file('reviews.tsv')" -``` - -:::tip -文件名中可以使用 glob 通配符(参见 [glob substitutions](/sql-reference/table-functions/file.md/#globs-in-path))。 - -示例: - -```bash -./clickhouse local -q "SELECT * FROM 'reviews*.jsonl'" -./clickhouse local -q "SELECT * FROM 'review_?.csv'" -./clickhouse local -q "SELECT * FROM 'review_{1..3}.csv'" -``` - -::: - -```response -marketplace Nullable(String) -customer_id Nullable(Int64) -review_id Nullable(String) -product_id Nullable(String) -product_parent Nullable(Int64) -product_title Nullable(String) -product_category Nullable(String) -star_rating Nullable(Int64) -helpful_votes Nullable(Int64) -total_votes Nullable(Int64) -vine Nullable(String) -verified_purchase Nullable(String) -review_headline Nullable(String) -review_body Nullable(String) -review_date Nullable(Date) -``` - -我们来找出评分最高的产品: - -```bash -./clickhouse local -q "SELECT - argMax(product_title,star_rating), - max(star_rating) -FROM file('reviews.tsv')" -``` - -```response -大富翁少年版桌游 5 -``` - - -## 在 AWS S3 中查询 Parquet 文件中的数据 {#query-data-in-a-parquet-file-in-aws-s3} - -如果你在 S3 中有一个文件,可以使用 `clickhouse-local` 和 `s3` 表函数来就地查询该文件(无需先将数据插入到 ClickHouse 表中)。我们在一个公共 bucket 中有一个名为 `house_0.parquet` 的文件,其中包含英国已售房产的房价数据。来看一下它有多少行: - -```bash -./clickhouse local -q " -SELECT count() -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -该文件有 270 万行记录: - -```response -2772030 -``` - -查看 ClickHouse 根据文件推断出的模式总是很有帮助: - -```bash -./clickhouse local -q "DESCRIBE s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet')" -``` - -```response -price Nullable(Int64) -date Nullable(UInt16) -postcode1 Nullable(String) -postcode2 Nullable(String) -type Nullable(String) -is_new Nullable(UInt8) -duration Nullable(String) -addr1 Nullable(String) -addr2 Nullable(String) -street Nullable(String) -locality Nullable(String) -town Nullable(String) -district Nullable(String) -county Nullable(String) -``` - -让我们来看哪些社区最昂贵: - -```bash -./clickhouse local -q " -SELECT - town, - district, - count() AS c, - round(avg(price)) AS price, - bar(price, 0, 5000000, 100) -FROM s3('https://datasets-documentation.s3.eu-west-3.amazonaws.com/house_parquet/house_0.parquet') -GROUP BY - town, - district -HAVING c >= 100 -ORDER BY price DESC -LIMIT 10" -``` - -```response -LONDON CITY OF LONDON 886 2271305 █████████████████████████████████████████████▍ -LEATHERHEAD ELMBRIDGE 206 1176680 ███████████████████████▌ -LONDON CITY OF WESTMINSTER 12577 1108221 ██████████████████████▏ -LONDON KENSINGTON AND CHELSEA 8728 1094496 █████████████████████▉ -HYTHE FOLKESTONE AND HYTHE 130 1023980 ████████████████████▍ -CHALFONT ST GILES CHILTERN 113 835754 ████████████████▋ -AMERSHAM BUCKINGHAMSHIRE 113 799596 ███████████████▉ -VIRGINIA WATER RUNNYMEDE 356 789301 ███████████████▊ -BARNET ENFIELD 282 740514 ██████████████▊ -NORTHWOOD THREE RIVERS 184 731609 ██████████████▋ -``` - -:::tip -当你准备好将文件写入 ClickHouse 时,启动一个 ClickHouse 服务器,并将 `file` 和 `s3` 表函数的结果插入到一个 `MergeTree` 表中。更多详情请参阅[快速开始](/get-started/quick-start)。 -::: - - -## 格式转换 {#format-conversions} - -你可以使用 `clickhouse-local` 在不同格式之间进行数据转换。示例: - -```bash -$ clickhouse-local --input-format JSONLines --output-format CSV --query "SELECT * FROM table" < data.json > data.csv -``` - -格式会根据文件扩展名自动识别: - -```bash -$ clickhouse-local --query "SELECT * FROM table" < data.json > data.csv -``` - -作为快捷方式,你可以使用 `--copy` 参数来简化写法: - -```bash -$ clickhouse-local --copy < data.json > data.csv -``` - - -## 使用方法 {#usage} - -默认情况下,`clickhouse-local` 可以访问同一主机上运行的 ClickHouse 服务器的数据,并且不会依赖于该服务器的配置。它也支持通过 `--config-file` 参数加载服务器配置。对于临时数据,默认会创建一个唯一的临时数据目录。 - -基本用法(Linux): - -```bash -$ clickhouse-local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -基本用法(Mac): - -```bash -$ ./clickhouse local --structure "table_structure" --input-format "format_of_incoming_data" --query "query" -``` - -:::note -`clickhouse-local` 也支持在 Windows 上通过 WSL2 使用。 -::: - -参数: - -* `-S`, `--structure` — 输入数据的表结构。 -* `--input-format` — 输入格式,默认为 `TSV`。 -* `-F`, `--file` — 数据路径,默认为 `stdin`。 -* `-q`, `--query` — 要执行的查询,以 `;` 作为分隔符。`--query` 可以被多次指定,例如 `--query "SELECT 1" --query "SELECT 2"`。不能与 `--queries-file` 同时使用。 -* `--queries-file` - 包含要执行查询的文件路径。`--queries-file` 可以被多次指定,例如 `--query queries1.sql --query queries2.sql`。不能与 `--query` 同时使用。 -* `--multiquery, -n` – 如果指定,则可以在 `--query` 选项之后列出多个以分号分隔的查询。为方便起见,也可以省略 `--query`,直接在 `--multiquery` 之后传入查询。 -* `-N`, `--table` — 存放输出数据的表名,默认是 `table`。 -* `-f`, `--format`, `--output-format` — 输出格式,默认为 `TSV`。 -* `-d`, `--database` — 默认数据库,默认为 `_local`。 -* `--stacktrace` — 是否在出现异常时转储调试输出。 -* `--echo` — 在执行前打印查询。 -* `--verbose` — 输出更多查询执行细节。 -* `--logger.console` — 将日志输出到控制台。 -* `--logger.log` — 日志文件名。 -* `--logger.level` — 日志级别。 -* `--ignore-error` — 在某个查询失败时不停止处理。 -* `-c`, `--config-file` — 配置文件路径,格式与 ClickHouse 服务器相同,默认情况下配置为空。 -* `--no-system-tables` — 不附加 system 表。 -* `--help` — 显示 `clickhouse-local` 的参数说明。 -* `-V`, `--version` — 打印版本信息并退出。 - -此外,还为每个 ClickHouse 配置变量提供了对应的参数,这些参数通常比 `--config-file` 更常用。 - - -## 示例 {#examples} - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local --structure "a Int64, b Int64" \ - --input-format "CSV" --query "SELECT * FROM table" -读取 2 行,32.00 B,耗时 0.000 秒,5182 行/秒,80.97 KiB/秒。 -1 2 -3 4 -``` - -前面的示例等价于: - -```bash -$ echo -e "1,2\n3,4" | clickhouse-local -n --query " - CREATE TABLE table (a Int64, b Int64) ENGINE = File(CSV, stdin); - SELECT a, b FROM table; - DROP TABLE table;" -已读取 2 行,32.00 B,用时 0.000 秒,4987 行/秒,77.93 KiB/秒。 -1 2 -3 4 -``` - -你不必使用 `stdin` 或 `--file` 参数,也可以通过 [`file` 表函数](../../sql-reference/table-functions/file.md) 打开任意数量的文件: - -```bash -$ echo 1 | tee 1.tsv -1 - -$ echo 2 | tee 2.tsv -2 - -$ clickhouse-local --query " - select * from file('1.tsv', TSV, 'a int') t1 - cross join file('2.tsv', TSV, 'b int') t2" -1 2 -``` - -现在让我们输出每个 Unix 用户的内存使用量: - -查询: - -```bash -$ ps aux | tail -n +2 | awk '{ printf("%s\t%s\n", $1, $4) }' \ - | clickhouse-local --structure "user String, mem Float64" \ - --query "SELECT user, round(sum(mem), 2) as memTotal - FROM table GROUP BY user ORDER BY memTotal DESC FORMAT Pretty" -``` - -结果: - -```text -读取 186 行,4.15 KiB,用时 0.035 秒,5302 行/秒,118.34 KiB/秒。 -┏━━━━━━━━━━┳━━━━━━━━━━┓ -┃ user ┃ memTotal ┃ -┡━━━━━━━━━━╇━━━━━━━━━━┩ -│ bayonet │ 113.5 │ -├──────────┼──────────┤ -│ root │ 8.8 │ -├──────────┼──────────┤ -... -``` - - -## 相关内容 {#related-content-1} - -- [使用 clickhouse-local 从本地文件中提取、转换和查询数据](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) -- [将数据导入 ClickHouse - 第 1 部分](https://clickhouse.com/blog/getting-data-into-clickhouse-part-1) -- [探索海量真实世界数据集:在 ClickHouse 中分析 100 多年的气象记录](https://clickhouse.com/blog/real-world-data-noaa-climate-data) -- 博客:[使用 clickhouse-local 从本地文件中提取、转换和查询数据](https://clickhouse.com/blog/extracting-converting-querying-local-files-with-sql-clickhouse-local) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md deleted file mode 100644 index 0462a6b2b8b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/clickhouse-obfuscator.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: 'ClickHouse Obfuscator 文档' -slug: /operations/utilities/clickhouse-obfuscator -title: 'clickhouse-obfuscator' -doc_type: 'reference' ---- - -用于对表中数据进行混淆的简单工具。 - -它读取一个输入表并生成一个输出表,输出表保留输入表的部分属性,但其中的数据已被替换。 -这使你可以在发布用于基准测试的数据时,使用几乎等同于真实生产环境的数据。 - -该工具旨在保留以下数据属性: - -- 每一列以及任意列组合的值基数(不同值的数量); -- 条件基数:在某一列的值满足特定条件时,另一列的不同值数量; -- 整数绝对值的概率分布;有符号整数的符号;浮点数的指数和符号; -- 字符串长度的概率分布; -- 数值为零的概率;空字符串和空数组,`NULL`; - -- 使用 LZ77 和熵编码族进行压缩时的数据压缩比; -- 整个表中时间值的连续性(差值的量级);浮点数值的连续性; -- `DateTime` 值中的日期部分; - -- 字符串值的 UTF-8 合法性; -- 字符串值看起来足够自然。 - -上述大多数属性都适用于性能测试: - -在保留基数、量级、压缩比等特性的前提下, -读取、过滤、聚合和排序等操作的执行速度将与原始数据几乎相同。 - -该工具以确定性的方式工作:你定义一个种子值,转换过程完全由输入数据和该种子共同决定。 -某些转换是一一映射且可逆的,因此你需要使用足够大的种子并妥善保密。 - -它使用了一些密码学原语来转换数据,但从密码学的严格意义上讲实现并不完备,因此除非有额外的安全保证,否则不应将结果视为安全。结果中可能仍会保留一些你不希望公开的数据。 - -它始终保持数值 0、1、-1,日期、数组长度以及空标记与源数据完全一致。 -例如,你的表中有一列 `IsMobile`,其取值为 0 和 1。在转换后的数据中,该列仍将保持相同的取值。 - -因此,用户仍然可以精确计算移动端流量占比。 - -再举一个例子:当你的表中包含一些私有数据,例如用户邮箱,而你不希望公开任何一个真实的邮箱地址时, -如果你的表足够大,包含多种不同的邮箱,而且不存在某个邮箱出现频率远高于其他邮箱的情况,那么所有数据都将被匿名化。但如果某一列中不同取值的数量较少,它可能会重现其中部分原始取值。 -你应当了解该工具的工作原理和算法,并根据需要微调其命令行参数。 - -此工具仅在数据量至少达到中等规模(至少数千行)时才能较好地工作。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/index.md deleted file mode 100644 index 38670f3a5e9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/index.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: '列出各种实用的 ClickHouse 工具和实用工具的页面。' -keywords: ['tools', 'utilities'] -sidebar_label: '工具和实用工具列表' -sidebar_position: 56 -slug: /operations/utilities/ -title: '工具和实用工具列表' -doc_type: 'landing-page' ---- - -| 工具/实用工具 | 描述 | -|------|-------------| -|[clickhouse-local](../../operations/utilities/clickhouse-local.md) | 允许在不启动 ClickHouse 服务器的情况下对数据运行 SQL 查询,类似于 `awk` 的用法。| -|[clickhouse-benchmark](../../operations/utilities/clickhouse-benchmark.md) | 使用自定义查询和设置对服务器进行压测。| -| [clickhouse-format](../../operations/utilities/clickhouse-format.md) | 支持对输入的查询语句进行格式化。| -|[ClickHouse obfuscator](../../operations/utilities/clickhouse-obfuscator.md) | 对数据进行混淆处理。| -|[ClickHouse compressor](../../operations/utilities/clickhouse-compressor.md) | 对数据进行压缩和解压缩。| -| [clickhouse-disks](../../operations/utilities/clickhouse-disks.md) | 为不同 ClickHouse 磁盘上的文件提供类似文件系统的操作能力。| -| [clickhouse-odbc-bridge](../../operations/utilities/odbc-bridge.md) | 用于 ODBC 驱动程序的代理服务器。| -| [clickhouse_backupview](../../operations/utilities/backupview.md) | 用于分析 ClickHouse 备份的 Python 模块。| \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md deleted file mode 100644 index 4cc41b92828..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/utilities/odbc-bridge.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: 'Odbc Bridge 文档' -slug: /operations/utilities/odbc-bridge -title: 'clickhouse-odbc-bridge' -doc_type: 'reference' ---- - -一个简单的 HTTP 服务器,用作 ODBC 驱动程序的代理。设计它的主要动机是,ODBC 实现中可能出现的段错误(segfault)或其他错误,可能会导致整个 clickhouse-server 进程崩溃。 - -该工具通过 HTTP 工作,而不是通过管道、共享内存或 TCP 进行通信,原因是: -- 实现更简单 -- 调试更简单 -- jdbc-bridge 可以以同样的方式实现 - - - -## 用法 {#usage} - -`clickhouse-server` 在 ODBC 表函数和 StorageODBC 引擎中使用此工具。 -不过它也可以作为独立工具从命令行使用,在 POST 请求的 URL 中指定以下参数: -- `connection_string` -- ODBC 连接字符串。 -- `sample_block` -- ClickHouse NamesAndTypesList 格式的列描述,名称使用反引号包裹, - 类型为字符串。名称和类型以空格分隔,各行以换行分隔。 -- `max_block_size` -- 可选参数,用于设置单个数据块的最大大小。 - -查询在 POST 请求的请求体中发送,响应以 RowBinary 格式返回。 - - - -## 示例: {#example} - -```bash -$ clickhouse-odbc-bridge --http-port 9018 --daemon - -$ curl -d "query=SELECT PageID, ImpID, AdType FROM Keys ORDER BY PageID, ImpID" --data-urlencode "connection_string=DSN=ClickHouse;DATABASE=stat" --data-urlencode "sample_block=columns format version: 1 -3 columns: -\`PageID\` String -\`ImpID\` String -\`AdType\` String -" "http://localhost:9018/" > result.txt - -$ cat result.txt -12246623837185725195925621517 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md b/i18n/zh/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md deleted file mode 100644 index c4fb5a8aebc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/operations/workload-scheduling.md +++ /dev/null @@ -1,420 +0,0 @@ ---- -description: '工作负载调度文档' -sidebar_label: 'Workload scheduling' -sidebar_position: 69 -slug: /operations/workload-scheduling -title: 'Workload scheduling' -doc_type: 'reference' ---- - -当 ClickHouse 同时执行多个查询时,它们可能会使用共享资源(例如磁盘和 CPU 核心)。可以应用调度约束和策略来规范如何在不同工作负载之间使用和共享这些资源。针对所有资源,可以配置统一的调度层级结构。层级结构的根节点代表共享资源,而叶子节点则对应特定的工作负载,用于承载超出资源容量的请求。 - -:::note -目前可以使用上述方法对 [远程磁盘 IO](#disk_config) 和 [CPU](#cpu_scheduling) 进行调度。有关灵活的内存限制,请参见 [Memory overcommit](settings/memory-overcommit.md) -::: - - - -## 磁盘配置 {#disk_config} - -要为特定磁盘启用 IO 工作负载调度,必须为 WRITE 和 READ 访问分别创建读写资源: - -```sql -CREATE RESOURCE resource_name (WRITE DISK disk_name, READ DISK disk_name) --- or -CREATE RESOURCE read_resource_name (WRITE DISK write_disk_name) -CREATE RESOURCE write_resource_name (READ DISK read_disk_name) -``` - -资源可用于任意数量的磁盘上,可用于 `READ`、`WRITE`,或同时用于 `READ` 和 `WRITE`。有一种语法可以将该资源应用到所有磁盘: - -```sql -CREATE RESOURCE all_io (READ ANY DISK, WRITE ANY DISK); -``` - -另一种指定某个资源使用哪些磁盘的方式,是通过服务器的 `storage_configuration`: - -:::warning -通过 ClickHouse 配置进行工作负载调度已被弃用。应改用 SQL 语法。 -::: - -要为特定磁盘启用 I/O 调度,必须在 `storage_configuration` 中指定 `read_resource` 和/或 `write_resource`。这会告诉 ClickHouse 对于给定磁盘的每个读写请求应使用哪种资源。读资源和写资源可以引用同一个资源名称,这对本地 SSD 或 HDD 很有用。多个不同的磁盘也可以引用同一个资源,这对远程磁盘很有用:例如,当你希望能在“production”和“development”工作负载之间公平分配网络带宽时。 - -示例: - -```xml - - - ... - - - s3 - https://clickhouse-public-datasets.s3.amazonaws.com/my-bucket/root-path/ - your_access_key_id - your_secret_access_key - network_read - network_write - - - - - -
- s3 -
-
-
-
-
-
-``` - -请注意,服务器端配置选项的优先级高于通过 SQL 方式定义资源。 - - -## 工作负载标记 {#workload_markup} - -可以通过设置 `workload` 为查询打标,以区分不同的工作负载。如果未设置 `workload`,则会使用值“default”。请注意,你也可以通过 settings profile 指定其他值。如果你希望某个用户发出的所有查询都带有固定的 `workload` 值,可以使用 setting constraint 将 `workload` 设为常量。 - -也可以为后台活动指定 `workload` 设置。合并(merge)和变更(mutation)分别使用服务器设置 `merge_workload` 和 `mutation_workload`。这些值同样可以通过 MergeTree 表设置项 `merge_workload` 和 `mutation_workload` 在特定表级别进行覆盖。 - -下面来看一个具有两种不同工作负载的系统示例:“production” 和 “development”。 - -```sql -SELECT count() FROM my_table WHERE value = 42 SETTINGS workload = 'production' -SELECT count() FROM my_table WHERE value = 13 SETTINGS workload = 'development' -``` - - -## 资源调度层次结构 {#hierarchy} - -从调度子系统的角度来看,资源是由调度节点组成的层次结构。 - -```mermaid -graph TD - subgraph network_read - nr_root(("/")) - -->|100 个并发请求| nr_fair("fair") - -->|75% 带宽| nr_prod["prod"] - nr_fair - -->|25% 带宽| nr_dev["dev"] - end - - subgraph network_write - nw_root(("/")) - -->|100 个并发请求| nw_fair("fair") - -->|75% 带宽| nw_prod["prod"] - nw_fair - -->|25% 带宽| nw_dev["dev"] - end -``` - -:::warning -使用 ClickHouse 配置进行工作负载调度已被弃用,应改用 SQL 语法。SQL 语法会自动创建所有必要的调度节点,下面的调度节点说明应视为较低层级的实现细节,可通过 [system.scheduler](/operations/system-tables/scheduler.md) 表访问。 -::: - -**可能的节点类型:** - -* `inflight_limit`(约束)- 当并发进行中的请求数量超过 `max_requests`,或它们的总代价超过 `max_cost` 时阻塞;必须只有一个子节点。 -* `bandwidth_limit`(约束)- 当当前带宽超过 `max_speed`(0 表示不限制)或突发流量超过 `max_burst`(默认等于 `max_speed`)时阻塞;必须只有一个子节点。 -* `fair`(策略)- 按照 max-min 公平性原则,从其子节点中选择下一个要处理的请求;子节点可以指定 `weight`(默认值为 1)。 -* `priority`(策略)- 按照静态优先级(数值越小优先级越高)从其子节点中选择下一个要处理的请求;子节点可以指定 `priority`(默认值为 0)。 -* `fifo`(队列)- 层级结构中的叶节点,用于存放超出资源容量的请求。 - -为了能够利用底层资源的全部能力,你应该使用 `inflight_limit`。注意,`max_requests` 或 `max_cost` 的取值过小可能导致资源未被充分利用,而取值过大则可能导致调度器内部队列为空,从而使子树中的策略被忽略(造成不公平或忽略优先级)。另一方面,如果你希望保护资源不被过度使用,则应使用 `bandwidth_limit`。当在 `duration` 秒内消耗的资源量超过 `max_burst + max_speed * duration` 字节时,它会进行限流。针对同一资源的两个 `bandwidth_limit` 节点可以用来分别限制短时间内的峰值带宽以及更长时间区间内的平均带宽。 - -下面的示例展示了如何定义图中所示的 IO 调度层级结构: - -```xml - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - - inflight_limit - 100 - - - fair - - - fifo - 3 - - - fifo - - - - -``` - - -## 工作负载分类器 {#workload_classifiers} - -:::warning -使用 ClickHouse 配置进行工作负载调度已弃用,应改为使用 SQL 语法。当使用 SQL 语法时,会自动创建分类器。 -::: - -工作负载分类器用于定义从查询中指定的 `workload` 到特定资源应使用的叶队列(leaf queues)的映射关系。目前,工作负载分类较为简单:仅支持静态映射。 - -示例: - -```xml - - - - /fair/prod - /fair/prod - - - /fair/dev - /fair/dev - - - /fair/dev - /fair/dev - - - -``` - - -## 工作负载层级结构 {#workloads} - -ClickHouse 提供了便捷的 SQL 语法来定义调度层级结构。所有通过 `CREATE RESOURCE` 创建的资源共享相同的层级结构,但在某些方面可能有所不同。每个通过 `CREATE WORKLOAD` 创建的工作负载,都会为每个资源维护若干自动创建的调度节点。可以在一个父工作负载中创建子工作负载。下面是一个示例,用于定义与上面的 XML 配置完全相同的层级结构: - -```sql -CREATE RESOURCE network_write (WRITE DISK s3) -CREATE RESOURCE network_read (READ DISK s3) -CREATE WORKLOAD all SETTINGS max_io_requests = 100 -CREATE WORKLOAD development IN all -CREATE WORKLOAD production IN all SETTINGS weight = 3 -``` - -没有子级的叶节点 workload 的名称可以在查询设置中使用:`SETTINGS workload = 'name'`。 - -要自定义 workload,可以使用以下设置: - -* `priority` - 同级 workloads 按静态优先级值进行服务(数值越低优先级越高)。 -* `weight` - 具有相同静态优先级的同级 workloads 按权重共享资源。 -* `max_io_requests` - 此 workload 中并发 IO 请求数量的上限。 -* `max_bytes_inflight` - 此 workload 中所有并发请求的在途字节总数上限。 -* `max_bytes_per_second` - 此 workload 的读或写字节速率上限。 -* `max_burst_bytes` - 在不被限流的情况下,此 workload 可以处理的最大字节数(对每种资源分别计算)。 -* `max_concurrent_threads` - 此 workload 中查询可使用的线程数上限。 -* `max_concurrent_threads_ratio_to_cores` - 与 `max_concurrent_threads` 相同,但会根据可用 CPU 核心数进行归一化。 -* `max_cpus` - 为此 workload 中的查询提供服务的 CPU 核心数上限。 -* `max_cpu_share` - 与 `max_cpus` 相同,但会根据可用 CPU 核心数进行归一化。 -* `max_burst_cpu_seconds` - 在不因 `max_cpus` 而被限流的情况下,此 workload 可以消耗的最大 CPU 秒数。 - -通过 workload 设置指定的所有限制对于每种资源都是相互独立的。例如,设置 `max_bytes_per_second = 10485760` 的 workload 将对每个读和写资源分别应用 10 MB/s 的带宽限制。如果需要对读写使用公共的总限额,请考虑对 READ 和 WRITE 访问使用同一个资源。 - -无法为不同资源指定不同的 workload 层级结构。但可以为特定资源指定不同的 workload 配置值: - -```sql -CREATE OR REPLACE WORKLOAD all SETTINGS max_io_requests = 100, max_bytes_per_second = 1000000 FOR network_read, max_bytes_per_second = 2000000 FOR network_write -``` - -还要注意,如果某个 workload 或 resource 正被另一个 workload 引用,则无法将其删除。若要更新某个 workload 的定义,请使用 `CREATE OR REPLACE WORKLOAD` 查询。 - -:::note -Workload 设置会被转换为一组对应的调度节点。有关更底层的细节,请参阅调度节点的[类型和选项](#hierarchy)说明。 -::: - - -## CPU 调度 {#cpu_scheduling} - -要为工作负载启用 CPU 调度,请创建 CPU 资源并设置并发线程数量上限: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 -``` - -当 ClickHouse 服务器使用[多线程](/operations/settings/settings.md#max_threads)执行大量并发查询且所有 CPU 插槽都已占用时,就会进入过载状态。在过载状态下,每一个释放的 CPU 插槽都会根据调度策略重新分配给合适的工作负载。对于共享同一工作负载的查询,插槽通过轮询(round robin)方式分配。对于处于不同工作负载中的查询,插槽则根据为各工作负载指定的权重、优先级和限制进行分配。 - -当线程未被阻塞并执行 CPU 密集型任务时,就会消耗 CPU 时间。出于调度目的,将线程分为两类: - -* 主线程(master thread)—— 首个开始处理查询或后台活动(例如合并或变更)的线程。 -* 工作线程(worker thread)—— 由主线程派生出来,用于处理 CPU 密集型任务的其他线程。 - -为了获得更好的响应性,将主线程和工作线程使用的资源隔离开可能是有益的。当使用较大的 `max_threads` 查询设置值时,大量工作线程很容易独占 CPU 资源。此时,新到的查询将被阻塞并等待一个 CPU 插槽,以便其主线程能够开始执行。为避免这种情况,可以使用如下配置: - -```sql -CREATE RESOURCE worker_cpu (WORKER THREAD) -CREATE RESOURCE master_cpu (MASTER THREAD) -CREATE WORKLOAD all SETTINGS max_concurrent_threads = 100 FOR worker_cpu, max_concurrent_threads = 1000 FOR master_cpu -``` - -它会分别对主线程和工作线程设置并发限制。即使所有 100 个工作线程 CPU 槽位都已占满,只要仍有可用的主线程 CPU 槽位,新查询也不会被阻塞,而是会先以单线程开始执行。之后如果有工作线程 CPU 槽位空闲,此类查询可以扩展并启动其对应的工作线程。另一方面,这种方法不会将槽位总数与 CPU 处理器的数量绑定在一起,过多的并发线程运行会影响性能。 - -限制主线程的并发数并不会限制并发查询的数量。CPU 槽位可以在查询执行过程中被释放,并被其他线程重新获取。例如,在主线程并发数限制为 2 的情况下,4 个并发查询仍然可以同时执行。在这种情况下,每个查询将获得一个 CPU 处理器 50% 的计算能力。应使用单独的机制来限制并发查询的数量,而这一点在当前的工作负载中尚不支持。 - -可以为不同的工作负载单独设置线程并发限制: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD all -CREATE WORKLOAD admin IN all SETTINGS max_concurrent_threads = 10 -CREATE WORKLOAD production IN all SETTINGS max_concurrent_threads = 100 -CREATE WORKLOAD analytics IN production SETTINGS max_concurrent_threads = 60, weight = 9 -CREATE WORKLOAD ingestion IN production -``` - -此配置示例为 admin 和 production 提供了相互独立的 CPU 插槽池。production 池在 analytics 和摄取之间共享。此外,如果 production 池过载,10 个释放插槽中的 9 个将在必要时重新分配给分析查询。在过载期间,摄取查询只会获得 10 个插槽中的 1 个。这可能改善面向用户的查询延迟。Analytics 本身有 60 个并发线程的上限,始终至少为摄取保留 40 个线程。当没有过载时,摄取可以使用全部 100 个线程。 - -要将某个查询从 CPU 调度中排除,请将查询设置 [use_concurrency_control](/operations/settings/settings.md/#use_concurrency_control) 设为 0。 - -CPU 调度目前尚不支持 merges 和 mutations。 - -为了为各类负载提供公平分配,有必要在查询执行期间进行抢占和缩减(down-scaling)。通过 `cpu_slot_preemption` 服务器设置启用抢占。如果启用,每个线程会根据 `cpu_slot_quantum_ns` 服务器设置,定期续约其 CPU 插槽占用。在 CPU 过载时,此类续约可能会阻塞执行。当执行被长时间阻塞时(参见 `cpu_slot_preemption_timeout_ms` 服务器设置),查询会进行缩减,并动态减少并发运行线程的数量。请注意,在不同负载之间可以保证 CPU 时间的公平性,但在同一负载内部的各个查询之间,在某些极端情况下可能无法完全保证。 - -:::warning -插槽调度提供了一种控制[查询并发度](/operations/settings/settings.md#max_threads)的方式,但除非将服务器设置 `cpu_slot_preemption` 设为 `true`,否则并不能保证 CPU 时间的公平分配;在未启用抢占时,公平性是基于竞争负载之间分配到的 CPU 插槽数量来提供的。这并不意味着获得相同数量的 CPU 秒数,因为在没有抢占的情况下,CPU 插槽可能被无限期持有。线程在开始执行时获取一个插槽,并在工作完成时释放。 -::: - - -:::note -声明 CPU 资源会使 [`concurrent_threads_soft_limit_num`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_num) 和 [`concurrent_threads_soft_limit_ratio_to_cores`](server-configuration-parameters/settings.md#concurrent_threads_soft_limit_ratio_to_cores) 设置不再生效。取而代之的是,使用工作负载设置 `max_concurrent_threads` 来限制为某个特定工作负载分配的 CPU 数量。要实现之前的行为,仅创建 WORKER THREAD 资源,将工作负载 `all` 的 `max_concurrent_threads` 设置为与 `concurrent_threads_soft_limit_num` 相同的值,并在查询中使用设置 `workload = "all"`。该配置等价于将 [`concurrent_threads_scheduler`](server-configuration-parameters/settings.md#concurrent_threads_scheduler) 设置为 "fair_round_robin"。 -::: - - - -## 线程 vs CPU {#threads_vs_cpus} - -有两种方式可以控制某个负载的 CPU 消耗: - -* 线程数量限制:`max_concurrent_threads` 和 `max_concurrent_threads_ratio_to_cores` -* CPU 节流:`max_cpus`、`max_cpu_share` 和 `max_burst_cpu_seconds` - -第一种方式允许根据当前服务器负载动态控制为一个查询创建多少线程。本质上,它降低了查询设置 `max_threads` 的实际生效上限。第二种方式使用令牌桶算法对该负载的 CPU 消耗进行节流控制。它不会直接影响线程数量,而是对该负载中所有线程的总 CPU 消耗进行限流。 - -使用 `max_cpus` 和 `max_burst_cpu_seconds` 的令牌桶节流机制含义如下:在任意长度为 `delta` 秒的时间区间内,该负载中所有查询的总 CPU 消耗不得超过 `max_cpus * delta + max_burst_cpu_seconds` 个 CPU 秒。从长期来看,它将平均消耗限制在 `max_cpus`,但在短期内可以超过该限制。例如,当 `max_burst_cpu_seconds = 60` 且 `max_cpus=0.001` 时,可以在不被节流的情况下运行 1 个线程 60 秒,或 2 个线程 30 秒,或 60 个线程 1 秒。`max_burst_cpu_seconds` 的默认值为 1 秒。对于并发线程较多的情况,如果该值过低,可能导致无法充分利用允许的 `max_cpus` 核心数。 - -:::warning -只有在开启 `cpu_slot_preemption` 服务器设置时,CPU 节流设置才会生效,否则会被忽略。 -::: - -在持有一个 CPU slot 时,一个线程可能处于以下三种主要状态之一: - -* **Running:** 实际在消耗 CPU 资源。线程处于该状态时花费的时间会被 CPU 节流机制计入。 -* **Ready:** 正在等待可用的 CPU。不会被 CPU 节流机制计入。 -* **Blocked:** 正在执行 IO 操作或其他阻塞系统调用(例如等待互斥量)。不会被 CPU 节流机制计入。 - -下面给出一个同时结合 CPU 节流和线程数量限制的配置示例: - -```sql -CREATE RESOURCE cpu (MASTER THREAD, WORKER THREAD) -CREATE WORKLOAD 全部 SETTINGS max_concurrent_threads_ratio_to_cores = 2 -CREATE WORKLOAD 管理 IN 全部 SETTINGS max_concurrent_threads = 2, priority = -1 -CREATE WORKLOAD 生产 IN 全部 SETTINGS weight = 4 -CREATE WORKLOAD 分析 IN 生产 SETTINGS max_cpu_share = 0.7, weight = 3 -CREATE WORKLOAD 摄取 IN 生产 -CREATE WORKLOAD 开发 IN 全部 SETTINGS max_cpu_share = 0.3 -``` - -在这里,我们将所有查询的线程总数限制为可用 CPU 数量的 2 倍。Admin 工作负载最多限制为 2 个线程,而不受可用 CPU 数量影响。Admin 的优先级为 -1(低于默认的 0),如果需要,它会优先获得任何 CPU 槽位。当 Admin 不执行查询时,CPU 资源会在生产和开发工作负载之间分配。基于权重(4 比 1)来保证 CPU 时间的份额:至少 80% 分配给生产(如果需要),至少 20% 分配给开发(如果需要)。权重决定了“保证份额”,而 CPU 限流决定了“上限”:生产没有上限,可以消耗 100%;开发则有 30% 的上限,即使没有来自其他工作负载的查询,该上限也会生效。生产工作负载不是叶节点,因此其资源会按权重(3 比 1)在 analytics 和摄取之间划分。这意味着 analytics 至少有 0.8 * 0.75 = 60% 的保证份额,并且基于 `max_cpu_share`,它的上限是总 CPU 资源的 70%。而摄取至少有 0.8 * 0.25 = 20% 的保证份额,但没有上限。 - -:::note -如果您希望最大化 ClickHouse 服务器上的 CPU 利用率,请避免为根工作负载 `all` 使用 `max_cpus` 和 `max_cpu_share`。相反,请将 `max_concurrent_threads` 设置为更高的值。例如,在一个拥有 8 个 CPU 的系统上,将 `max_concurrent_threads = 16`。这样可以让 8 个线程执行 CPU 任务,同时另外 8 个线程处理 I/O 操作。额外的线程会产生 CPU 压力,从而确保调度规则被严格执行。相对地,如果设置 `max_cpus = 8`,则永远不会产生 CPU 压力,因为服务器无法超出这 8 个可用 CPU。 -::: - - -## 查询插槽调度 {#query_scheduling} - -要为工作负载启用查询插槽调度,请创建 QUERY 资源,并为并发查询数量或每秒查询数量设置限制: - -```sql -CREATE RESOURCE query (QUERY) -CREATE WORKLOAD all SETTINGS max_concurrent_queries = 100, max_queries_per_second = 10, max_burst_queries = 20 -``` - -工作负载设置 `max_concurrent_queries` 用于限制某个工作负载下可同时运行的并发查询数量。它类似于查询级设置 [`max_concurrent_queries_for_all_users`](/operations/settings/settings#max_concurrent_queries_for_all_users) 和服务器级设置 [max_concurrent_queries](/operations/server-configuration-parameters/settings#max_concurrent_queries)。异步插入查询以及 KILL 等特定查询不计入该限制。 - -工作负载设置 `max_queries_per_second` 和 `max_burst_queries` 通过令牌桶限流器来限制该工作负载的查询数量。它保证在任意时间区间 `T` 内,开始执行的新查询数量不会超过 `max_queries_per_second * T + max_burst_queries`。 - -工作负载设置 `max_waiting_queries` 限制该工作负载的等待查询数量。当达到该限制时,服务器会返回错误 `SERVER_OVERLOADED`。 - -:::note -被阻塞的查询将无限期等待,并且在满足所有约束条件之前不会出现在 `SHOW PROCESSLIST` 中。 -::: - - -## 工作负载和资源存储 {#workload_entity_storage} - -所有工作负载和资源的定义,会以 `CREATE WORKLOAD` 和 `CREATE RESOURCE` 查询的形式持久化存储到磁盘中的 `workload_path`,或 ZooKeeper 中的 `workload_zookeeper_path`。建议使用 ZooKeeper 存储以在各节点之间实现一致性。或者,也可以在使用磁盘存储时配合 `ON CLUSTER` 子句。 - - - -## 基于配置的工作负载和资源 {#config_based_workloads} - -除了基于 SQL 的定义之外,还可以在服务器配置文件中预定义工作负载和资源。这在云环境中非常有用,在这类环境中,部分限制由基础设施决定,而其他限制可以由用户调整。基于配置定义的实体优先于通过 SQL 定义的实体,且不能通过 SQL 命令进行修改或删除。 - -### 配置格式 {#config_based_workloads_format} - -```xml - - - RESOURCE s3disk_read (READ DISK s3); - RESOURCE s3disk_write (WRITE DISK s3); - WORKLOAD all SETTINGS max_io_requests = 500 FOR s3disk_read, max_io_requests = 1000 FOR s3disk_write, max_bytes_per_second = 1342177280 FOR s3disk_read, max_bytes_per_second = 3355443200 FOR s3disk_write; - WORKLOAD production IN all SETTINGS weight = 3; - - -``` - -该配置使用与 `CREATE WORKLOAD` 和 `CREATE RESOURCE` 语句相同的 SQL 语法。所有查询都必须是合法的。 - -### 使用建议 {#config_based_workloads_usage_recommendations} - -对于云环境,典型的设置可能包括: - -1. 在配置中定义根工作负载和网络 IO 资源,用于设置基础设施资源上限 -2. 设置 `throw_on_unknown_workload` 以强制执行这些限制 -3. 创建一个 `CREATE WORKLOAD default IN all`,以自动对所有查询应用限制(因为 `workload` 查询设置的默认值是 'default') -4. 允许用户在已配置的层级结构中创建额外的工作负载 - -这样可以确保所有后台活动和查询都遵守基础设施限制,同时仍然为用户特定的调度策略保留灵活性。 - -另一种使用场景是在异构集群中为不同节点使用不同的配置。 - - -## 严格资源访问 {#strict_resource_access} - -要强制所有查询遵循资源调度策略,可以使用服务器端设置 `throw_on_unknown_workload`。如果将其设置为 `true`,则每个查询都必须使用有效的 `workload` 查询设置,否则会抛出 `RESOURCE_ACCESS_DENIED` 异常。如果将其设置为 `false`,则此类查询不会使用资源调度器,即它将获得对任意 `RESOURCE` 的无限制访问。查询设置 `use_concurrency_control = 0` 允许查询绕过 CPU 调度器并获得对 CPU 的无限制访问。要强制启用 CPU 调度,请创建一个设置约束,使 `use_concurrency_control` 保持为只读常量值。 - -:::note -除非已经执行了 `CREATE WORKLOAD default`,否则不要将 `throw_on_unknown_workload` 设置为 `true`。如果在服务器启动过程中执行了未显式设置 `workload` 的查询,可能会导致服务器启动失败。 -::: - - - -## 另请参阅 {#see-also} -- [system.scheduler](/operations/system-tables/scheduler.md) -- [system.workloads](/operations/system-tables/workloads.md) -- [system.resources](/operations/system-tables/resources.md) -- [merge_workload](/operations/settings/merge-tree-settings.md#merge_workload) MergeTree 设置项 -- [merge_workload](/operations/server-configuration-parameters/settings.md#merge_workload) 全局服务器设置项 -- [mutation_workload](/operations/settings/merge-tree-settings.md#mutation_workload) MergeTree 设置项 -- [mutation_workload](/operations/server-configuration-parameters/settings.md#mutation_workload) 全局服务器设置项 -- [workload_path](/operations/server-configuration-parameters/settings.md#workload_path) 全局服务器设置项 -- [workload_zookeeper_path](/operations/server-configuration-parameters/settings.md#workload_zookeeper_path) 全局服务器设置项 -- [cpu_slot_preemption](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption) 全局服务器设置项 -- [cpu_slot_quantum_ns](/operations/server-configuration-parameters/settings.md#cpu_slot_quantum_ns) 全局服务器设置项 -- [cpu_slot_preemption_timeout_ms](/operations/server-configuration-parameters/settings.md#cpu_slot_preemption_timeout_ms) 全局服务器设置项 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md deleted file mode 100644 index da9eb2e767b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/combinators.md +++ /dev/null @@ -1,362 +0,0 @@ ---- -description: '聚合函数组合器的文档' -sidebar_label: '组合器' -sidebar_position: 37 -slug: /sql-reference/aggregate-functions/combinators -title: '聚合函数组合器' -doc_type: 'reference' ---- - - - -# 聚合函数组合器 {#aggregate-function-combinators} - -聚合函数名可以追加一个后缀,从而改变该聚合函数的工作方式。 - - - -## -If {#-if} - -后缀 -If 可以附加到任意聚合函数的名称后面。此时,聚合函数会额外接受一个参数——条件(`UInt8` 类型)。聚合函数只会处理触发该条件的行。如果条件一次都没有被触发,则返回默认值(通常为 0 或空字符串)。 - -示例:`sumIf(column, cond)`、`countIf(cond)`、`avgIf(x, cond)`、`quantilesTimingIf(level1, level2)(x, cond)`、`argMinIf(arg, val, cond)` 等。 - -借助条件聚合函数,你可以在不使用子查询和 `JOIN` 的情况下,同时针对多个条件计算聚合值。例如,可以使用条件聚合函数实现分群对比功能。 - - - -## -Array {#-array} - -可以将 -Array 后缀附加到任意聚合函数上。在这种情况下,聚合函数接受类型为 `Array(T)`(数组)的参数,而不是类型为 `T` 的参数。如果聚合函数接受多个参数,则这些参数必须是长度相同的数组。在处理数组时,该聚合函数的行为与原始聚合函数在所有数组元素上的行为相同。 - -示例 1:`sumArray(arr)` —— 对所有 `arr` 数组中的所有元素求和。在这个例子中,也可以更简单地写成:`sum(arraySum(arr))`。 - -示例 2:`uniqArray(arr)` —— 计算所有 `arr` 数组中唯一元素的数量。这也可以用更简单的方式实现:`uniq(arrayJoin(arr))`,但并不总是可以在查询中添加 `arrayJoin`。 - --If 和 -Array 可以组合使用。不过,必须先使用 `Array`,再使用 `If`。示例:`uniqArrayIf(arr, cond)`、`quantilesTimingArrayIf(level1, level2)(arr, cond)`。由于这一顺序,`cond` 参数不会是数组。 - - - -## -Map {#-map} - -可以为任意聚合函数添加 `-Map` 后缀。这样会创建一个以 `Map` 类型作为参数的聚合函数,并使用指定的聚合函数分别聚合该 `Map` 中每个键对应的值。结果同样为 `Map` 类型。 - -**示例** - -```sql -CREATE TABLE map_map( - date Date, - timeslot DateTime, - status Map(String, UInt64) -) ENGINE = Log; - -INSERT INTO map_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', (['a', 'b', 'c'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', (['c', 'd', 'e'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['d', 'e', 'f'], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', (['f', 'g', 'g'], [10, 10, 10])); - -SELECT - timeslot, - sumMap(status), - avgMap(status), - minMap(status) -FROM map_map -GROUP BY timeslot; - -┌────────────timeslot─┬─sumMap(status)───────────────────────┬─avgMap(status)───────────────────────┬─minMap(status)───────────────────────┐ -│ 2000-01-01 00:00:00 │ {'a':10,'b':10,'c':20,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ {'a':10,'b':10,'c':10,'d':10,'e':10} │ -│ 2000-01-01 00:01:00 │ {'d':10,'e':10,'f':20,'g':20} │ {'d':10,'e':10,'f':10,'g':10} │ {'d':10,'e':10,'f':10,'g':10} │ -└─────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┴──────────────────────────────────────┘ -``` - - -## -SimpleState {#-simplestate} - -应用此组合子后,聚合函数会返回相同的值,但类型不同。它是一个可以存储在表中的 [SimpleAggregateFunction(...)](../../sql-reference/data-types/simpleaggregatefunction.md),用于与 [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) 表配合使用。 - -**语法** - -```sql -SimpleState(x) -``` - -**参数** - -* `x` — 聚合函数的参数。 - -**返回值** - -返回 `SimpleAggregateFunction(...)` 类型的聚合函数值。 - -**示例** - -查询: - -```sql -WITH anySimpleState(number) AS c SELECT toTypeName(c), c FROM numbers(1); -``` - -结果: - -```text -┌─toTypeName(c)────────────────────────┬─c─┐ -│ SimpleAggregateFunction(any, UInt64) │ 0 │ -└──────────────────────────────────────┴───┘ -``` - - -## -State {#-state} - -如果你应用这个组合子,聚合函数不会返回最终结果值(例如 [uniq](/sql-reference/aggregate-functions/reference/uniq) 函数的唯一值个数),而是返回聚合的中间状态(对于 `uniq`,这是用于计算唯一值个数的哈希表)。这是一个 `AggregateFunction(...)` 类型,可以用于后续处理,或者存储在表中以便稍后完成聚合。 - -:::note -请注意,由于中间状态中的数据顺序可能发生变化,-MapState 对于相同数据并不是不变的,不过这并不影响对此数据的摄取。 -::: - -要处理这些状态,请使用: - -- [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) 表引擎。 -- [finalizeAggregation](/sql-reference/functions/other-functions#finalizeAggregation) 函数。 -- [runningAccumulate](../../sql-reference/functions/other-functions.md#runningAccumulate) 函数。 -- [-Merge](#-merge) 组合子。 -- [-MergeState](#-mergestate) 组合子。 - - - -## -Merge {#-merge} - -如果使用此组合器,聚合函数会将中间聚合状态作为参数,合并这些状态以完成聚合,并返回最终结果值。 - - - -## -MergeState {#-mergestate} - -以与 -Merge 组合器相同的方式合并中间聚合状态。但它不会返回最终结果值,而是返回中间聚合状态,类似于 -State 组合器。 - - - -## -ForEach {#-foreach} - -将作用于表的聚合函数转换为作用于数组的聚合函数,对各数组中对应位置的元素进行聚合,并返回结果数组。例如,对于数组 `[1, 2]`、`[3, 4, 5]` 和 `[6, 7]`,`sumForEach` 在对对应位置的数组元素求和后返回结果 `[10, 13, 5]`。 - - - -## -Distinct {#-distinct} - -每个唯一的参数组合只会被聚合一次。重复的值会被忽略。 -示例:`sum(DISTINCT x)`(或 `sumDistinct(x)`)、`groupArray(DISTINCT x)`(或 `groupArrayDistinct(x)`)、`corrStable(DISTINCT x, y)`(或 `corrStableDistinct(x, y)`)等。 - - - -## -OrDefault {#-ordefault} - -修改聚合函数的行为。 - -如果聚合函数没有输入值,使用此组合器时,会返回其返回类型的默认值。适用于可以接受空输入数据的聚合函数。 - -`-OrDefault` 可以与其他组合器一起使用。 - -**语法** - -```sql -OrDefault(x) -``` - -**参数** - -* `x` — 聚合函数的参数。 - -**返回值** - -如果没有任何可聚合的数据,则返回聚合函数返回类型的默认值。 - -具体类型取决于所使用的聚合函数。 - -**示例** - -查询: - -```sql -SELECT avg(number), avgOrDefault(number) FROM numbers(0) -``` - -结果: - -```text -┌─avg(number)─┬─avgOrDefault(number)─┐ -│ nan │ 0 │ -└─────────────┴──────────────────────┘ -``` - -另外,`-OrDefault` 也可以与其他组合器一起使用。当聚合函数无法处理空输入时,这会很有用。 - -查询: - -```sql -SELECT avgOrDefaultIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -结果: - -```text -┌─avgOrDefaultIf(x, greater(x, 10))─┐ -│ 0.00 │ -└───────────────────────────────────┘ -``` - - -## -OrNull {#-ornull} - -修改聚合函数的行为。 - -此组合器将聚合函数的结果转换为 [Nullable](../../sql-reference/data-types/nullable.md) 数据类型。如果聚合函数没有可用于计算的值,则返回 [NULL](/operations/settings/formats#input_format_null_as_default)。 - -`-OrNull` 可以与其他组合器一起使用。 - -**语法** - -```sql -OrNull(x) -``` - -**参数** - -* `x` — 聚合函数的参数。 - -**返回值** - -* 聚合函数的结果,转换为 `Nullable` 数据类型。 -* 如果没有任何可聚合的数据,则返回 `NULL`。 - -类型:`Nullable(聚合函数返回类型)`。 - -**示例** - -在聚合函数名称末尾添加 `-orNull`。 - -查询: - -```sql -SELECT sumOrNull(number), toTypeName(sumOrNull(number)) FROM numbers(10) WHERE number > 10 -``` - -结果: - -```text -┌─sumOrNull(number)─┬─toTypeName(sumOrNull(number))─┐ -│ ᴺᵁᴸᴸ │ Nullable(UInt64) │ -└───────────────────┴───────────────────────────────┘ -``` - -`-OrNull` 也可以与其他组合器一起使用。当聚合函数不接受空输入时,这会很有用。 - -查询: - -```sql -SELECT avgOrNullIf(x, x > 10) -FROM -( - SELECT toDecimal32(1.23, 2) AS x -) -``` - -结果: - -```text -┌─avgOrNullIf(x, greater(x, 10))─┐ -│ ᴺᵁᴸᴸ │ -└────────────────────────────────┘ -``` - - -## -Resample {#-resample} - -可将数据划分为多个组,并分别对每个组中的数据进行聚合。分组是通过将某一列的取值划分为不同的区间来完成的。 - -```sql -Resample(start, end, step)(, resampling_key) -``` - -**参数** - -* `start` — `resampling_key` 值的完整所需区间的起始值。 -* `stop` — `resampling_key` 值的完整所需区间的结束值。该完整区间不包含 `stop` 值,即 `[start, stop)`。 -* `step` — 将完整区间划分为子区间时使用的步长。`aggFunction` 会在每个子区间上独立执行。 -* `resampling_key` — 用于将数据划分为各个区间的列。 -* `aggFunction_params` — 传给 `aggFunction` 的参数。 - -**返回值** - -* 一个数组,包含每个子区间上执行 `aggFunction` 的结果。 - -**示例** - -考虑包含如下数据的 `people` 表: - -```text -┌─姓名───┬─年龄─┬─工资─┐ -│ John │ 16 │ 10 │ -│ Alice │ 30 │ 15 │ -│ Mary │ 35 │ 8 │ -│ Evelyn │ 48 │ 11.5 │ -│ David │ 62 │ 9.9 │ -│ Brian │ 60 │ 16 │ -└────────┴─────┴──────┘ -``` - -让我们获取年龄位于 `[30,60)` 和 `[60,75)` 区间内的人的姓名。由于我们使用整数来表示年龄,因此实际得到的年龄区间为 `[30, 59]` 和 `[60,74]`。 - -要将姓名聚合到数组中,我们使用 [groupArray](/sql-reference/aggregate-functions/reference/grouparray) 聚合函数。它只接受一个参数,在我们的示例中就是 `name` 列。`groupArrayResample` 函数则使用 `age` 列按年龄对姓名进行聚合。要定义所需的区间,我们向 `groupArrayResample` 函数传递 `30, 75, 30` 这几个参数。 - -```sql -SELECT groupArrayResample(30, 75, 30)(name, age) FROM people -``` - -```text -┌─groupArrayResample(30, 75, 30)(name, age)─────┐ -│ [['Alice','Mary','Evelyn'],['David','Brian']] │ -└───────────────────────────────────────────────┘ -``` - -来看一下结果。 - -由于 `John` 年龄太小,他没有被纳入样本。其他人则按照指定的年龄区间进行了划分。 - -现在让我们计算在各个指定年龄区间内的总人数及其平均工资。 - -```sql -SELECT - countResample(30, 75, 30)(name, age) AS amount, - avgResample(30, 75, 30)(wage, age) AS avg_wage -FROM people -``` - -```text -┌─amount─┬─avg_wage──────────────────┐ -│ [3,2] │ [11.5,12.949999809265137] │ -└────────┴───────────────────────────┘ -``` - - -## -ArgMin {#-argmin} - -后缀 -ArgMin 可以附加到任意聚合函数的名称后使用。在这种情况下,该聚合函数会额外接受一个参数,该参数应当是任意可比较的表达式。聚合函数只会处理在该附加表达式对应的值上最小的那些行。 - -示例:`sumArgMin(column, expr)`、`countArgMin(expr)`、`avgArgMin(x, expr)` 等。 - - - -## -ArgMax {#-argmax} - -类似于后缀 -ArgMin,但只处理在指定附加表达式上具有最大值的行。 - - - -## 相关内容 {#related-content} - -- 博客文章:[在 ClickHouse 中使用聚合组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md deleted file mode 100644 index 8fe64b32f11..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/grouping_function.md +++ /dev/null @@ -1,369 +0,0 @@ ---- -description: 'GROUPING 聚合函数文档。' -slug: /sql-reference/aggregate-functions/grouping_function -title: 'GROUPING' -doc_type: 'reference' ---- - - - -# 分组 {#grouping} - - - -## GROUPING {#grouping} - -[ROLLUP](../statements/select/group-by.md/#rollup-modifier) 和 [CUBE](../statements/select/group-by.md/#cube-modifier) 是对 GROUP BY 的修饰符,它们都会计算小计。ROLLUP 接收一个有序的列列表,例如 `(day, month, year)`,并在聚合的每个层级计算小计,最后再计算总计。CUBE 则会针对所指定列的所有可能组合计算小计。GROUPING 用于识别由 ROLLUP 或 CUBE 返回的哪些行是更高层级的聚合行(superaggregate),哪些则是未使用修饰符的 GROUP BY 本应返回的普通分组结果行。 - -GROUPING 函数接收多个列作为参数,并返回一个位掩码(bitmask)值。 -- `1` 表示由 `ROLLUP` 或 `CUBE` 修饰的 `GROUP BY` 返回的该行是小计 -- `0` 表示由 `ROLLUP` 或 `CUBE` 返回的该行不是小计 - - - -## GROUPING SETS {#grouping-sets} - -默认情况下,CUBE 修饰符会对传入 CUBE 的所有列的所有可能组合计算小计。GROUPING SETS 允许你指定要计算的具体组合。 - -对层次化数据进行分析是使用 ROLLUP、CUBE 和 GROUPING SETS 修饰符的典型用例。这里的示例是一张表,包含了在两个数据中心中安装的 Linux 发行版及其版本的信息。按发行版、版本和数据中心位置来查看这些数据可能是有价值的。 - -### 加载示例数据 {#load-sample-data} - -```sql -CREATE TABLE servers ( datacenter VARCHAR(255), - distro VARCHAR(255) NOT NULL, - version VARCHAR(50) NOT NULL, - quantity INT - ) - ORDER BY (datacenter, distro, version) -``` - -```sql -INSERT INTO servers(datacenter, distro, version, quantity) -VALUES ('Schenectady', 'Arch','2022.08.05',50), - ('Westport', 'Arch','2022.08.05',40), - ('Schenectady','Arch','2021.09.01',30), - ('Westport', 'Arch','2021.09.01',20), - ('Schenectady','Arch','2020.05.01',10), - ('Westport', 'Arch','2020.05.01',5), - ('Schenectady','RHEL','9',60), - ('Westport','RHEL','9',70), - ('Westport','RHEL','7',80), - ('Schenectady','RHEL','7',80) -``` - -```sql -SELECT - * -FROM - servers; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─quantity─┐ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ 9 │ 70 │ -└─────────────┴────────┴────────────┴──────────┘ - -10 行数据。耗时: 0.409 秒。 -``` - -### 简单查询 {#simple-queries} - -按分布情况统计每个数据中心中的服务器数量: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ - -返回 4 行。耗时:0.212 秒。 -``` - -```sql -SELECT - datacenter, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter; -``` - -```response -┌─datacenter──┬─qty─┐ -│ Westport │ 215 │ -│ Schenectady │ 230 │ -└─────────────┴─────┘ - -返回 2 行。用时:0.277 秒。 -``` - -```sql -SELECT - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro; -``` - -```response - -┌─distro─┬─qty─┐ -│ Arch │ 155 │ -│ RHEL │ 290 │ -└────────┴─────┘ - -查询返回 2 行。用时:0.352 秒。 -``` - - -```sql -SELECT - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─qty─┐ -│ 445 │ -└─────┘ - -返回 1 行。耗时:0.244 秒。 -``` - -### 对比多个 GROUP BY 语句与 GROUPING SETS {#comparing-multiple-group-by-statements-with-grouping-sets} - -在不使用 CUBE、ROLLUP 或 GROUPING SETS 的情况下对数据进行拆分: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter, - distro -UNION ALL -SELECT - datacenter, - null, - SUM (quantity) qty -FROM - servers -GROUP BY - datacenter -UNION ALL -SELECT - null, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - distro -UNION ALL -SELECT - null, - null, - SUM(quantity) qty -FROM - servers; -``` - -```response -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ ᴺᵁᴸᴸ │ 215 │ -│ Schenectady │ ᴺᵁᴸᴸ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ ᴺᵁᴸᴸ │ Arch │ 155 │ -│ ᴺᵁᴸᴸ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -9 行在集合中。耗时:0.527 秒。 -``` - -通过 GROUPING SETS 获取相同的信息: - -```sql -SELECT - datacenter, - distro, - SUM (quantity) qty -FROM - servers -GROUP BY - GROUPING SETS( - (datacenter,distro), - (datacenter), - (distro), - () - ) -``` - -```response -┌─datacenter──┬─distro─┬─qty─┐ -│ Schenectady │ RHEL │ 140 │ -│ Westport │ Arch │ 65 │ -│ Schenectady │ Arch │ 90 │ -│ Westport │ RHEL │ 150 │ -└─────────────┴────────┴─────┘ -┌─datacenter──┬─distro─┬─qty─┐ -│ Westport │ │ 215 │ -│ Schenectady │ │ 230 │ -└─────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ │ 445 │ -└────────────┴────────┴─────┘ -┌─datacenter─┬─distro─┬─qty─┐ -│ │ Arch │ 155 │ -│ │ RHEL │ 290 │ -└────────────┴────────┴─────┘ - -9 行在集合中。耗时:0.427 秒。 -``` - -### 将 CUBE 与 GROUPING SETS 进行比较 {#comparing-cube-with-grouping-sets} - -下一条查询中的 CUBE,`CUBE(datacenter,distro,version)` 会生成一个可能不太合理的层次结构。跨这两个发行版比较版本并没有意义(因为 Arch 和 RHEL 的发布周期和版本命名规范并不相同)。后面的 GROUPING SETS 示例更为合适,因为它在同一个分组集合中同时包含了 `distro` 和 `version`。 - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM - servers -GROUP BY - CUBE(datacenter,distro,version) -ORDER BY - datacenter, - distro; -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ │ │ 7 │ 160 │ -│ │ │ 2020.05.01 │ 15 │ -│ │ │ 2021.09.01 │ 50 │ -│ │ │ 2022.08.05 │ 90 │ -│ │ │ 9 │ 130 │ -│ │ │ │ 445 │ -│ │ Arch │ 2021.09.01 │ 50 │ -│ │ Arch │ 2022.08.05 │ 90 │ -│ │ Arch │ 2020.05.01 │ 15 │ -│ │ Arch │ │ 155 │ -│ │ RHEL │ 9 │ 130 │ -│ │ RHEL │ 7 │ 160 │ -│ │ RHEL │ │ 290 │ -│ Schenectady │ │ 9 │ 60 │ -│ Schenectady │ │ 2021.09.01 │ 30 │ -│ Schenectady │ │ 7 │ 80 │ -│ Schenectady │ │ 2022.08.05 │ 50 │ -│ Schenectady │ │ 2020.05.01 │ 10 │ -│ Schenectady │ │ │ 230 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -│ Schenectady │ Arch │ │ 90 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ │ 9 │ 70 │ -│ Westport │ │ 2020.05.01 │ 5 │ -│ Westport │ │ 2022.08.05 │ 40 │ -│ Westport │ │ 7 │ 80 │ -│ Westport │ │ 2021.09.01 │ 20 │ -│ Westport │ │ │ 215 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Westport │ Arch │ │ 65 │ -│ Westport │ RHEL │ 9 │ 70 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴────────────┴───────────────┘ - -39 行结果集。执行时间:0.355 秒。 -``` - -:::note -当版本没有与发行版关联时,上面示例中的 version 可能就不太合适;如果我们跟踪的是内核版本,则可能更合理,因为内核版本可以与任一发行版关联。在这种情况下,使用 GROUPING SETS(如下一个示例所示)可能是更好的选择。 -::: - - -```sql -SELECT - datacenter, - distro, - version, - SUM(quantity) -FROM servers -GROUP BY - GROUPING SETS ( - (datacenter, distro, version), - (datacenter, distro)) -``` - -```response -┌─datacenter──┬─distro─┬─version────┬─sum(quantity)─┐ -│ Westport │ RHEL │ 9 │ 70 │ -│ Schenectady │ Arch │ 2022.08.05 │ 50 │ -│ Schenectady │ Arch │ 2021.09.01 │ 30 │ -│ Schenectady │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2020.05.01 │ 5 │ -│ Westport │ RHEL │ 7 │ 80 │ -│ Westport │ Arch │ 2021.09.01 │ 20 │ -│ Westport │ Arch │ 2022.08.05 │ 40 │ -│ Schenectady │ RHEL │ 9 │ 60 │ -│ Schenectady │ Arch │ 2020.05.01 │ 10 │ -└─────────────┴────────┴────────────┴───────────────┘ -┌─datacenter──┬─distro─┬─version─┬─sum(quantity)─┐ -│ Schenectady │ RHEL │ │ 140 │ -│ Westport │ Arch │ │ 65 │ -│ Schenectady │ Arch │ │ 90 │ -│ Westport │ RHEL │ │ 150 │ -└─────────────┴────────┴─────────┴───────────────┘ - -14 rows in set. Elapsed: 1.036 sec. -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md deleted file mode 100644 index 81b320a77c4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/index.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -description: '聚合函数参考文档' -sidebar_label: '聚合函数' -sidebar_position: 33 -slug: /sql-reference/aggregate-functions/ -title: '聚合函数' -doc_type: 'reference' ---- - -# 聚合函数 {#aggregate-functions} - -聚合函数以[常规](http://www.sql-tutorial.com/sql-aggregate-functions-sql-tutorial)方式工作,符合数据库专家对其的预期。 - -ClickHouse 还支持: - -* [参数化聚合函数](/sql-reference/aggregate-functions/parametric-functions),除了列之外还可以接受其他参数。 -* [组合器](/sql-reference/aggregate-functions/combinators),用于改变聚合函数的行为。 - -## NULL 处理 {#null-processing} - -在聚合过程中,所有 `NULL` 参数都会被跳过。如果聚合函数有多个参数,那么对于任意一个或多个参数为 NULL 的行,都会被忽略。 - -此规则有一个例外,即函数 [`first_value`](../../sql-reference/aggregate-functions/reference/first_value.md)、[`last_value`](../../sql-reference/aggregate-functions/reference/last_value.md) 及其别名(分别为 `any` 和 `anyLast`),当它们后面带有修饰符 `RESPECT NULLS` 时。例如,`FIRST_VALUE(b) RESPECT NULLS`。 - -**示例:** - -考虑下表: - -```text -┌─x─┬────y─┐ -│ 1 │ 2 │ -│ 2 │ ᴺᵁᴸᴸ │ -│ 3 │ 2 │ -│ 3 │ 3 │ -│ 3 │ ᴺᵁᴸᴸ │ -└───┴──────┘ -``` - -假设你需要对 `y` 列的值求总和: - -```sql -SELECT sum(y) FROM t_null_big -``` - -```text -┌─sum(y)─┐ -│ 7 │ -└────────┘ -``` - -现在,你可以使用 `groupArray` 函数根据 `y` 列生成一个数组: - -```sql -SELECT groupArray(y) FROM t_null_big -``` - -```text -┌─groupArray(y)─┐ -│ [2,2,3] │ -└───────────────┘ -``` - -`groupArray` 在结果数组中不会包含 `NULL`。 - -你可以使用 [COALESCE](../../sql-reference/functions/functions-for-nulls.md#coalesce) 将 NULL 转换为在你的使用场景中有意义的值。例如:`avg(COALESCE(column, 0))` 会在聚合中使用该列的值;如果为 NULL 则使用 0: - -```sql -SELECT - avg(y), - avg(coalesce(y, 0)) -FROM t_null_big -``` - -```text -┌─────────────avg(y)─┬─avg(coalesce(y, 0))─┐ -│ 2.3333333333333335 │ 1.4 │ -└────────────────────┴─────────────────────┘ -``` - -你也可以使用 [Tuple](sql-reference/data-types/tuple.md) 来绕过 NULL 跳过行为。仅包含一个 `NULL` 值的 `Tuple` 本身并不是 `NULL`,因此聚合函数不会因为这个 `NULL` 值而跳过该行。 - -```sql -SELECT - groupArray(y), - groupArray(tuple(y)).1 -FROM t_null_big; - -┌─groupArray(y)─┬─tupleElement(groupArray(tuple(y)), 1)─┐ -│ [2,2,3] │ [2,NULL,2,3,NULL] │ -└───────────────┴───────────────────────────────────────┘ -``` - -请注意,当这些列被用作聚合函数的参数时,相应的聚合将被跳过。例如,没有参数的 [`count`](../../sql-reference/aggregate-functions/reference/count.md)(`count()`)或仅带常量参数的形式(`count(1)`)会对数据块中的所有行进行计数(与 GROUP BY 列的取值无关,因为该列不是参数),而 `count(column)` 只会返回该列值不为 NULL 的行数。 - -```sql -SELECT - v, - count(1), - count(v) -FROM -( - SELECT if(number < 10, NULL, number % 3) AS v - FROM numbers(15) -) -GROUP BY v - -┌────v─┬─count()─┬─count(v)─┐ -│ ᴺᵁᴸᴸ │ 10 │ 0 │ -│ 0 │ 1 │ 1 │ -│ 1 │ 2 │ 2 │ -│ 2 │ 2 │ 2 │ -└──────┴─────────┴──────────┘ -``` - -下面是一个使用 `RESPECT NULLS` 的 `first_value` 示例,可以看到 NULL 输入会被保留(不会被跳过),并且它会返回读取到的第一个值,无论该值是否为 NULL: - -```sql -SELECT - col || '_' || ((col + 1) * 5 - 1) AS range, - first_value(odd_or_null) AS first, - first_value(odd_or_null) IGNORE NULLS as first_ignore_null, - first_value(odd_or_null) RESPECT NULLS as first_respect_nulls -FROM -( - SELECT - intDiv(number, 5) AS col, - if(number % 2 == 0, NULL, number) AS odd_or_null - FROM numbers(15) -) -GROUP BY col -ORDER BY col - -┌─range─┬─first─┬─first_ignore_null─┬─first_respect_nulls─┐ -│ 0_4 │ 1 │ 1 │ ᴺᵁᴸᴸ │ -│ 1_9 │ 5 │ 5 │ 5 │ -│ 2_14 │ 11 │ 11 │ ᴺᵁᴸᴸ │ -└───────┴───────┴───────────────────┴─────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md deleted file mode 100644 index 0d695688350..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/parametric-functions.md +++ /dev/null @@ -1,941 +0,0 @@ ---- -description: '参数化聚合函数文档' -sidebar_label: '参数化' -sidebar_position: 38 -slug: /sql-reference/aggregate-functions/parametric-functions -title: '参数化聚合函数' -doc_type: 'reference' ---- - -# 参数化聚合函数 {#parametric-aggregate-functions} - -某些聚合函数不仅可以接受参数列(用于压缩),还可以接受一组参数(用于初始化的常量)。其语法是使用两对括号而不是一对:第一对用于参数,第二对用于参数列。 - -## histogram {#histogram} - -计算自适应直方图。不保证结果精确。 - -```sql -histogram(number_of_bins)(values) -``` - -该函数使用 [A Streaming Parallel Decision Tree Algorithm](http://jmlr.org/papers/volume11/ben-haim10a/ben-haim10a.pdf)。当新数据进入函数时,会自动调整直方图箱的边界。通常情况下,各箱的宽度并不相等。 - -**参数** - -`values` — 计算得到输入值的[表达式](/sql-reference/syntax#expressions)。 - -**参数** - -`number_of_bins` — 直方图箱数量的上限。函数会自动计算箱的数量。它会尝试达到指定的箱数量,但如果无法达到,则会使用更少的箱。 - -**返回值** - -* 由以下格式的[元组](../../sql-reference/data-types/tuple.md)构成的[数组](../../sql-reference/data-types/array.md): - - ``` - [(lower_1, upper_1, height_1), ... (lower_N, upper_N, height_N)] - ``` - - * `lower` — 箱的下边界。 - * `upper` — 箱的上边界。 - * `height` — 箱的计算高度。 - -**示例** - -```sql -SELECT histogram(5)(number + 1) -FROM ( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─histogram(5)(plus(number, 1))───────────────────────────────────────────┐ -│ [(1,4.5,4),(4.5,8.5,4),(8.5,12.75,4.125),(12.75,17,4.625),(17,20,3.25)] │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -例如,可以使用 [bar](/sql-reference/functions/other-functions#bar) 函数来可视化直方图: - -```sql -WITH histogram(5)(rand() % 100) AS hist -SELECT - arrayJoin(hist).3 AS height, - bar(height, 0, 6, 5) AS bar -FROM -( - SELECT * - FROM system.numbers - LIMIT 20 -) -``` - -```text -┌─height─┬─bar───┐ -│ 2.125 │ █▋ │ -│ 3.25 │ ██▌ │ -│ 5.625 │ ████▏ │ -│ 5.625 │ ████▏ │ -│ 3.375 │ ██▌ │ -└────────┴───────┘ -``` - -在这种情况下,你需要记住自己并不知道直方图各分箱的边界。 - -## sequenceMatch {#sequencematch} - -检查序列中是否存在符合指定模式的事件链。 - -**语法** - -```sql -sequenceMatch(pattern)(timestamp, cond1, cond2, ...) -``` - -:::note -在同一秒内发生的事件在序列中的顺序可能不确定,从而影响结果。 -::: - -**参数** - -* `timestamp` — 包含时间数据的列。典型数据类型为 `Date` 和 `DateTime`。也可以使用任意受支持的 [UInt](../../sql-reference/data-types/int-uint.md) 数据类型。 - -* `cond1`, `cond2` — 描述事件链的条件。数据类型:`UInt8`。最多可以传递 32 个条件参数。函数只会考虑由这些条件描述的事件。如果序列中包含未被某个条件描述的数据,函数会跳过它们。 - -**参数** - -* `pattern` — 模式字符串。参见 [模式语法](#pattern-syntax)。 - -**返回值** - -* 如果匹配到模式,返回 1。 -* 如果未匹配到模式,返回 0。 - -类型:`UInt8`。 - -#### 模式语法 {#pattern-syntax} - -* `(?N)` — 匹配位置 `N` 处的条件参数。条件的编号范围为 `[1, 32]`。例如,`(?1)` 匹配传递给 `cond1` 参数的参数。 - -* `.*` — 匹配任意数量的事件。匹配该模式元素时不需要条件参数。 - -* `(?t operator value)` — 设置两个事件之间应相隔的时间(秒)。例如,模式 `(?1)(?t>1800)(?2)` 匹配彼此相隔超过 1800 秒的事件。在这些事件之间可以存在任意数量的任意事件。可以使用 `>=`、`>`、`<`、`<=`、`==` 运算符。 - -**示例** - -考虑表 `t` 中的如下数据: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -└──────┴────────┘ -``` - -运行以下查询: - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 1 │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -该函数找到了一个事件链,其中数字 2 紧跟在数字 1 之后。它跳过了中间的数字 3,因为该数字并未被描述为一个事件。如果我们希望在搜索示例中给出的该事件链时也将这个数字考虑在内,就需要为它单独添加一个条件。 - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 3) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 3))─┐ -│ 0 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -在这个例子中,函数无法找到与模式匹配的事件链,因为编号为 3 的事件发生在 1 和 2 之间。如果在相同情况下改为检查编号 4 的条件,则该序列就会与模式匹配。 - -```sql -SELECT sequenceMatch('(?1)(?2)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatch('(?1)(?2)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ 1 │ -└──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [sequenceCount](#sequencecount) - -## sequenceCount {#sequencecount} - -统计匹配该模式的事件链数量。该函数会搜索互不重叠的事件链。在当前事件链匹配完成后,才会开始搜索下一个事件链。 - -:::note -在同一秒内发生的事件在序列中的先后顺序可能不确定,从而影响结果。 -::: - -**语法** - -```sql -sequenceCount(pattern)(timestamp, cond1, cond2, ...) -``` - -**参数** - -* `timestamp` — 被视为包含时间数据的列。典型数据类型为 `Date` 和 `DateTime`。你也可以使用任意受支持的 [UInt](../../sql-reference/data-types/int-uint.md) 数据类型。 - -* `cond1`, `cond2` — 描述事件链的条件。数据类型:`UInt8`。最多可以传入 32 个条件参数。函数只会考虑这些条件中描述的事件。如果序列中包含未在任何条件中描述的数据,函数会跳过这些数据。 - -**参数说明** - -* `pattern` — 模式字符串。参见 [模式语法](#pattern-syntax)。 - -**返回值** - -* 匹配到的、互不重叠的事件链数量。 - -类型:`UInt64`。 - -**示例** - -假设表 `t` 中有如下数据: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -统计在数字 1 之后(中间可以有任意数量的其他数字)出现数字 2 的次数: - -```sql -SELECT sequenceCount('(?1).*(?2)')(time, number = 1, number = 2) FROM t -``` - -```text -┌─sequenceCount('(?1).*(?2)')(time, equals(number, 1), equals(number, 2))─┐ -│ 2 │ -└─────────────────────────────────────────────────────────────────────────┘ -``` - -## sequenceMatchEvents {#sequencematchevents} - -返回与模式匹配的最长事件链中各事件的时间戳。 - -:::note -在同一秒内发生的事件,其在序列中的先后顺序可能未定义,从而影响结果。 -::: - -**语法** - -```sql -sequenceMatchEvents(pattern)(timestamp, cond1, cond2, ...) -``` - -**参数** - -* `timestamp` — 被视为包含时间数据的列。典型数据类型为 `Date` 和 `DateTime`。也可以使用任意受支持的 [UInt](../../sql-reference/data-types/int-uint.md) 数据类型。 - -* `cond1`, `cond2` — 描述事件链的条件。数据类型:`UInt8`。最多可以传递 32 个条件参数。函数只会将这些条件中描述的事件纳入考虑。如果序列中包含未在条件中描述的数据,函数会跳过它们。 - -**参数说明** - -* `pattern` — 模式字符串。参见 [模式语法](#pattern-syntax)。 - -**返回值** - -* 由事件链中与条件参数 (?N) 匹配的时间戳组成的数组。数组中元素的位置与该条件参数在模式中的位置一一对应。 - -类型:Array。 - -**示例** - -考虑表 `t` 中的数据: - -```text -┌─time─┬─number─┐ -│ 1 │ 1 │ -│ 2 │ 3 │ -│ 3 │ 2 │ -│ 4 │ 1 │ -│ 5 │ 3 │ -│ 6 │ 2 │ -└──────┴────────┘ -``` - -返回最长链中事件的时间戳 - -```sql -SELECT sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, number = 1, number = 2, number = 4) FROM t -``` - -```text -┌─sequenceMatchEvents('(?1).*(?2).*(?1)(?3)')(time, equals(number, 1), equals(number, 2), equals(number, 4))─┐ -│ [1,3,4] │ -└────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [sequenceMatch](#sequencematch) - -## windowFunnel {#windowfunnel} - -在滑动时间窗口中搜索事件链,并计算该事件链中发生事件的最大数量。 - -该函数按照以下算法工作: - -* 函数首先搜索触发事件链中第一个条件的数据,并将事件计数器设为 1。此时即为滑动窗口开始的时间点。 - -* 如果事件链中的事件在窗口内按顺序发生,则计数器递增。如果事件顺序被打乱,则计数器不会递增。 - -* 如果数据中存在多个处于不同完成阶段的事件链,函数只会输出其中最长事件链的长度。 - -**语法** - -```sql -windowFunnel(window, [mode, [mode, ... ]])(timestamp, cond1, cond2, ..., condN) -``` - -**参数** - -* `timestamp` — 包含时间戳的列名。支持的数据类型:[Date](../../sql-reference/data-types/date.md)、[DateTime](/sql-reference/data-types/datetime) 以及其他无符号整数类型(注意:尽管时间戳支持 `UInt64` 类型,其数值不能超过 Int64 的最大值,即 2^63 - 1)。 -* `cond` — 描述事件链条件或数据的列。[UInt8](../../sql-reference/data-types/int-uint.md)。 - -**参数说明** - -* `window` — 滑动窗口的长度,即第一个条件与最后一个条件之间的时间间隔。`window` 的单位取决于 `timestamp` 本身,可能有所不同。通过以下表达式确定:`timestamp of cond1 <= timestamp of cond2 <= ... <= timestamp of condN <= timestamp of cond1 + window`。 -* `mode` — 可选参数。可以设置一个或多个模式。 - * `'strict_deduplication'` — 如果在事件序列中同一条件连续成立,则该重复事件会中断后续处理。注意:如果同一事件同时满足多个条件,则可能出现非预期结果。 - * `'strict_order'` — 不允许其他事件插入。例如在 `A->B->D->C` 的情况下,会在 `D` 处停止寻找 `A->B->C`,最大事件层级为 2。 - * `'strict_increase'` — 仅对时间戳严格递增的事件应用条件。 - * `'strict_once'` — 在事件链中每个事件只计数一次,即使它多次满足条件。 - -**返回值** - -在滑动时间窗口内,事件链中连续满足条件的最大数量。 -会分析结果集中所有的事件链。 - -类型:`Integer`。 - -**示例** - -判断在给定的一段时间内,是否足够让用户在网上商店中选购手机并完成两次购买。 - -设置如下事件链: - -1. 用户登录商店账户(`eventID = 1003`)。 -2. 用户搜索手机(`eventID = 1007, product = 'phone'`)。 -3. 用户下单(`eventID = 1009`)。 -4. 用户再次下单(`eventID = 1010`)。 - -输入表: - -```text -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-28 │ 1 │ 2019-01-29 10:00:00 │ 1003 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-31 │ 1 │ 2019-01-31 09:00:00 │ 1007 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-01-30 │ 1 │ 2019-01-30 08:00:00 │ 1009 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -┌─event_date─┬─user_id─┬───────────timestamp─┬─eventID─┬─product─┐ -│ 2019-02-01 │ 1 │ 2019-02-01 08:00:00 │ 1010 │ phone │ -└────────────┴─────────┴─────────────────────┴─────────┴─────────┘ -``` - -找出在 2019 年 1–2 月期间,用户 `user_id` 在这条链路中最多走到了哪一步。 - -查询: - -```sql -SELECT - level, - count() AS c -FROM -( - SELECT - user_id, - windowFunnel(6048000000000000)(timestamp, eventID = 1003, eventID = 1009, eventID = 1007, eventID = 1010) AS level - FROM trend - WHERE (event_date >= '2019-01-01') AND (event_date <= '2019-02-02') - GROUP BY user_id -) -GROUP BY level -ORDER BY level ASC; -``` - -结果: - -```text -┌─level─┬─c─┐ -│ 4 │ 1 │ -└───────┴───┘ -``` - -## retention {#retention} - -该函数接收一组 1 到 32 个 `UInt8` 类型的参数,这些参数指示事件是否满足某个条件。 -任意条件都可以作为参数指定(类似于 [WHERE](/sql-reference/statements/select/where) 子句中的条件)。 - -除第一个条件外,其余条件按成对方式应用:当第一个和第二个条件都为 true 时,第二个条件的结果为 true;当第一个和第三个条件都为 true 时,第三个条件的结果为 true;以此类推。 - -**语法** - -```sql -retention(cond1, cond2, ..., cond32); -``` - -**参数** - -* `cond` — 返回 `UInt8` 结果(1 或 0)的表达式。 - -**返回值** - -由 1 或 0 组成的数组。 - -* 1 — 该事件满足条件。 -* 0 — 该事件不满足条件。 - -类型:`UInt8`。 - -**示例** - -以下示例演示如何计算 `retention` 函数来分析网站流量。 - -**1.** 创建一个表用于演示。 - -```sql -CREATE TABLE retention_test(date Date, uid Int32) ENGINE = Memory; - -INSERT INTO retention_test SELECT '2020-01-01', number FROM numbers(5); -INSERT INTO retention_test SELECT '2020-01-02', number FROM numbers(10); -INSERT INTO retention_test SELECT '2020-01-03', number FROM numbers(15); -``` - -输入表: - -查询: - -```sql -SELECT * FROM retention_test -``` - -结果: - -```text -┌───────date─┬─uid─┐ -│ 2020-01-01 │ 0 │ -│ 2020-01-01 │ 1 │ -│ 2020-01-01 │ 2 │ -│ 2020-01-01 │ 3 │ -│ 2020-01-01 │ 4 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-02 │ 0 │ -│ 2020-01-02 │ 1 │ -│ 2020-01-02 │ 2 │ -│ 2020-01-02 │ 3 │ -│ 2020-01-02 │ 4 │ -│ 2020-01-02 │ 5 │ -│ 2020-01-02 │ 6 │ -│ 2020-01-02 │ 7 │ -│ 2020-01-02 │ 8 │ -│ 2020-01-02 │ 9 │ -└────────────┴─────┘ -┌───────date─┬─uid─┐ -│ 2020-01-03 │ 0 │ -│ 2020-01-03 │ 1 │ -│ 2020-01-03 │ 2 │ -│ 2020-01-03 │ 3 │ -│ 2020-01-03 │ 4 │ -│ 2020-01-03 │ 5 │ -│ 2020-01-03 │ 6 │ -│ 2020-01-03 │ 7 │ -│ 2020-01-03 │ 8 │ -│ 2020-01-03 │ 9 │ -│ 2020-01-03 │ 10 │ -│ 2020-01-03 │ 11 │ -│ 2020-01-03 │ 12 │ -│ 2020-01-03 │ 13 │ -│ 2020-01-03 │ 14 │ -└────────────┴─────┘ -``` - -**2.** 使用 `retention` 函数按照唯一 ID `uid` 对用户进行分组。 - -查询: - -```sql -SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r -FROM retention_test -WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') -GROUP BY uid -ORDER BY uid ASC -``` - -结果: - -```text -┌─uid─┬─r───────┐ -│ 0 │ [1,1,1] │ -│ 1 │ [1,1,1] │ -│ 2 │ [1,1,1] │ -│ 3 │ [1,1,1] │ -│ 4 │ [1,1,1] │ -│ 5 │ [0,0,0] │ -│ 6 │ [0,0,0] │ -│ 7 │ [0,0,0] │ -│ 8 │ [0,0,0] │ -│ 9 │ [0,0,0] │ -│ 10 │ [0,0,0] │ -│ 11 │ [0,0,0] │ -│ 12 │ [0,0,0] │ -│ 13 │ [0,0,0] │ -│ 14 │ [0,0,0] │ -└─────┴─────────┘ -``` - -**3.** 计算每日站点总访问量。 - -Query: - -```sql -SELECT - sum(r[1]) AS r1, - sum(r[2]) AS r2, - sum(r[3]) AS r3 -FROM -( - SELECT - uid, - retention(date = '2020-01-01', date = '2020-01-02', date = '2020-01-03') AS r - FROM retention_test - WHERE date IN ('2020-01-01', '2020-01-02', '2020-01-03') - GROUP BY uid -) -``` - -结果: - -```text -┌─r1─┬─r2─┬─r3─┐ -│ 5 │ 5 │ 5 │ -└────┴────┴────┘ -``` - -Where: - -* `r1`- 在 2020-01-01 当天访问该站点的独立访客数量(满足 `cond1` 条件)。 -* `r2`- 在 2020-01-01 至 2020-01-02 之间某一特定时间段内访问该站点的独立访客数量(同时满足 `cond1` 和 `cond2` 条件)。 -* `r3`- 在 2020-01-01 和 2020-01-03 某一特定时间段内访问该站点的独立访客数量(同时满足 `cond1` 和 `cond3` 条件)。 - -## uniqUpTo(N)(x) {#uniquptonx} - -计算参数的不同取值数量,最多计算到指定的上限 `N`。如果不同取值的数量大于 `N`,则该函数返回 `N` + 1,否则返回精确值。 - -推荐在 `N` 较小(最多为 10)时使用。`N` 的最大值为 100。 - -对于聚合函数的状态,此函数使用的内存量等于 1 + `N` * 单个值的字节大小。 -在处理字符串时,此函数会存储一个 8 字节的非加密散列值;对于字符串的计算是近似值。 - -例如,如果你有一张表,用来记录网站上用户发起的每一次搜索查询。表中的每一行代表一次搜索查询,包含用户 ID、搜索查询内容和查询时间戳等列。你可以使用 `uniqUpTo` 生成一份报表,仅展示被至少 5 个不同用户搜索过的关键词。 - -```sql -SELECT SearchPhrase -FROM SearchLog -GROUP BY SearchPhrase -HAVING uniqUpTo(4)(UserID) >= 5 -``` - -`uniqUpTo(4)(UserID)` 会为每个 `SearchPhrase` 计算不同 `UserID` 的数量,但最多只统计 4 个。如果某个 `SearchPhrase` 对应的不同 `UserID` 数量超过 4,该函数会返回 5(4 + 1)。随后,`HAVING` 子句会过滤掉那些不同 `UserID` 数量小于 5 的 `SearchPhrase`。这样就可以得到一份至少被 5 个不同用户使用过的搜索关键词列表。 - -## sumMapFiltered {#summapfiltered} - -此函数的行为与 [sumMap](/sql-reference/aggregate-functions/reference/summap) 相同,只是它还额外接受一个用于过滤的键数组作为参数。在处理高基数键时尤其有用。 - -**语法** - -`sumMapFiltered(keys_to_keep)(keys, values)` - -**参数** - -* `keys_to_keep`:用于过滤的键的 [Array](../data-types/array.md)。 -* `keys`:键的 [Array](../data-types/array.md)。 -* `values`:值的 [Array](../data-types/array.md)。 - -**返回值** - -* 返回一个包含两个数组的元组(tuple):按排序顺序的键,以及对应键的求和值。 - -**示例** - -查询: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt16, requests UInt64) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) FROM sum_map; -``` - -结果: - -```response - ┌─sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests)─┐ -1. │ ([1,4,8],[10,20,10]) │ - └─────────────────────────────────────────────────────────────────┘ -``` - -## sumMapFilteredWithOverflow {#summapfilteredwithoverflow} - -此函数的行为与 [sumMap](/sql-reference/aggregate-functions/reference/summap) 相同,但额外接受一个用于过滤的键数组作为参数。当键的基数很高时,这尤其有用。它与 [sumMapFiltered](#summapfiltered) 函数的不同之处在于,它执行的是允许溢出的求和运算——即求和结果的数据类型与参数的数据类型相同。 - -**语法** - -`sumMapFilteredWithOverflow(keys_to_keep)(keys, values)` - -**参数** - -* `keys_to_keep`: 用于过滤的键的 [Array](../data-types/array.md)。 -* `keys`: 键的 [Array](../data-types/array.md)。 -* `values`: 值的 [Array](../data-types/array.md)。 - -**返回值** - -* 返回由两个数组组成的元组:按排序顺序排列的键,以及对应键的累加值。 - -**示例** - -在此示例中,我们创建一张表 `sum_map`,向其中插入一些数据,然后同时使用 `sumMapFilteredWithOverflow` 和 `sumMapFiltered` 以及 `toTypeName` 函数来比较结果。在创建的表中,`requests` 的类型为 `UInt8`,`sumMapFiltered` 将累加值的类型提升为 `UInt64` 以避免溢出,而 `sumMapFilteredWithOverflow` 则保持类型为 `UInt8`,该类型不足以存储结果——即发生了溢出。 - -查询: - -```sql -CREATE TABLE sum_map -( - `date` Date, - `timeslot` DateTime, - `statusMap` Nested(status UInt8, requests UInt8) -) -ENGINE = Log - -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10]), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10]); -``` - -```sql -SELECT sumMapFilteredWithOverflow([1, 4, 8])(statusMap.status, statusMap.requests) as summap_overflow, toTypeName(summap_overflow) FROM sum_map; -``` - -```sql -SELECT sumMapFiltered([1, 4, 8])(statusMap.status, statusMap.requests) as summap, toTypeName(summap) FROM sum_map; -``` - -结果: - -```response - ┌─sum──────────────────┬─toTypeName(sum)───────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt8)) │ - └──────────────────────┴───────────────────────────────────┘ -``` - -```response - ┌─summap───────────────┬─toTypeName(summap)─────────────────┐ -1. │ ([1,4,8],[10,20,10]) │ Tuple(Array(UInt8), Array(UInt64)) │ - └──────────────────────┴────────────────────────────────────┘ -``` - -## sequenceNextNode {#sequencenextnode} - -返回匹配到的事件链中下一个事件的值。 - -*实验性函数,需通过 `SET allow_experimental_funnel_functions = 1` 启用。* - -**语法** - -```sql -sequenceNextNode(direction, base)(timestamp, event_column, base_condition, event1, event2, event3, ...) -``` - -**参数** - -* `direction` — 用于指定遍历方向。 - * forward — 向前移动。 - * backward — 向后移动。 - -* `base` — 用于设置基准点。 - * head — 将基准点设置为第一个事件。 - * tail — 将基准点设置为最后一个事件。 - * first_match — 将基准点设置为首个匹配到的 `event1`。 - * last_match — 将基准点设置为最后一个匹配到的 `event1`。 - -**参数说明** - -* `timestamp` — 包含时间戳的列名。支持的数据类型:[Date](../../sql-reference/data-types/date.md)、[DateTime](/sql-reference/data-types/datetime) 以及其他无符号整数类型。 -* `event_column` — 包含要返回的下一个事件值的列名。支持的数据类型:[String](../../sql-reference/data-types/string.md) 和 [Nullable(String)](../../sql-reference/data-types/nullable.md)。 -* `base_condition` — 基准点必须满足的条件。 -* `event1`, `event2`, ... — 描述事件链的条件,类型为 [UInt8](../../sql-reference/data-types/int-uint.md)。 - -**返回值** - -* `event_column[next_index]` — 当模式匹配且存在下一个值时返回。 -* `NULL` — 当模式未匹配或不存在下一个值时返回。 - -类型:[Nullable(String)](../../sql-reference/data-types/nullable.md)。 - -**示例** - -当事件为 A->B->C->D->E,且需要知道紧跟在 B->C 之后的事件(即 D)时,可以使用该函数。 - -用于查询紧跟在 A->B 之后的事件的语句如下: - -```sql -CREATE TABLE test_flow ( - dt DateTime, - id int, - page String) -ENGINE = MergeTree() -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow VALUES (1, 1, 'A') (2, 1, 'B') (3, 1, 'C') (4, 1, 'D') (5, 1, 'E'); - -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'A', page = 'A', page = 'B') as next_flow FROM test_flow GROUP BY id; -``` - -结果: - -```text -┌─id─┬─next_flow─┐ -│ 1 │ C │ -└────┴───────────┘ -``` - -**`forward` 与 `head` 的行为** - -```sql -ALTER TABLE test_flow DELETE WHERE 1 = 1 settings mutations_sync = 1; - -INSERT INTO test_flow VALUES (1, 1, 'Home') (2, 1, 'Gift') (3, 1, 'Exit'); -INSERT INTO test_flow VALUES (1, 2, 'Home') (2, 2, 'Home') (3, 2, 'Gift') (4, 2, 'Basket'); -INSERT INTO test_flow VALUES (1, 3, 'Gift') (2, 3, 'Home') (3, 3, 'Gift') (4, 3, 'Basket'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, page = 'Home', page = 'Home', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page - 1970-01-01 09:00:01 1 Home // 基准点,匹配 Home - 1970-01-01 09:00:02 1 Gift // 匹配 Gift - 1970-01-01 09:00:03 1 Exit // 结果 - - 1970-01-01 09:00:01 2 Home // 基准点,匹配 Home - 1970-01-01 09:00:02 2 Home // 不匹配 Gift - 1970-01-01 09:00:03 2 Gift - 1970-01-01 09:00:04 2 Basket -``` - -1970-01-01 09:00:01 3 Gift // 基准点,未与 Home 匹配 -1970-01-01 09:00:02 3 Home -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket - -```` - -**`backward` 和 `tail` 的行为** - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, page = 'Basket', page = 'Basket', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift -1970-01-01 09:00:03 1 Exit // 基准点,与 Basket 不匹配 - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // 结果 -1970-01-01 09:00:03 2 Gift // 与 Gift 匹配 -1970-01-01 09:00:04 2 Basket // 基准点,与 Basket 匹配 - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // 结果 -1970-01-01 09:00:03 3 Gift // Base point, Matched with Gift -1970-01-01 09:00:04 3 Basket // 基准点,与 Basket 匹配 -```` - -**`forward` 和 `first_match` 的行为** - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // 基点 -1970-01-01 09:00:03 1 Exit // 结果 - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // 基点 -1970-01-01 09:00:04 2 Basket // 结果 - -1970-01-01 09:00:01 3 Gift // 基点 -1970-01-01 09:00:02 3 Home // 结果 -1970-01-01 09:00:03 3 Gift -1970-01-01 09:00:04 3 Basket -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home -1970-01-01 09:00:02 1 Gift // 基准点 -1970-01-01 09:00:03 1 Exit // 与 Home 不匹配 - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home -1970-01-01 09:00:03 2 Gift // 基准点 -1970-01-01 09:00:04 2 Basket // 与 Home 不匹配 - -1970-01-01 09:00:01 3 Gift // 基准点 -1970-01-01 09:00:02 3 Home // 与 Home 匹配 -1970-01-01 09:00:03 3 Gift // 结果 -1970-01-01 09:00:04 3 Basket -``` - -**`backward` 和 `last_match` 的行为** - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift') FROM test_flow GROUP BY id; -``` - -dt id page -1970-01-01 09:00:01 1 Home // 结果 -1970-01-01 09:00:02 1 Gift // 基准点 -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home -1970-01-01 09:00:02 2 Home // 结果 -1970-01-01 09:00:03 2 Gift // 基准点 -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift -1970-01-01 09:00:02 3 Home // 结果 -1970-01-01 09:00:03 3 Gift // 基准点 -1970-01-01 09:00:04 3 Basket - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, page = 'Gift', page = 'Gift', page = 'Home') FROM test_flow GROUP BY id; - - dt id page -1970-01-01 09:00:01 1 Home // 与 Home 匹配,结果为 null -1970-01-01 09:00:02 1 Gift // 基点 -1970-01-01 09:00:03 1 Exit - -1970-01-01 09:00:01 2 Home // 结果 -1970-01-01 09:00:02 2 Home // 与 Home 匹配 -1970-01-01 09:00:03 2 Gift // 基点 -1970-01-01 09:00:04 2 Basket - -1970-01-01 09:00:01 3 Gift // 结果 -1970-01-01 09:00:02 3 Home // 与 Home 匹配 -1970-01-01 09:00:03 3 Gift // 基点 -1970-01-01 09:00:04 3 Basket -```` - -**`base_condition` 的行为** - -```sql -CREATE TABLE test_flow_basecond -( - `dt` DateTime, - `id` int, - `page` String, - `ref` String -) -ENGINE = MergeTree -PARTITION BY toYYYYMMDD(dt) -ORDER BY id; - -INSERT INTO test_flow_basecond VALUES (1, 1, 'A', 'ref4') (2, 1, 'A', 'ref3') (3, 1, 'B', 'ref2') (4, 1, 'B', 'ref1'); -``` - -```sql -SELECT id, sequenceNextNode('forward', 'head')(dt, page, ref = 'ref1', page = 'A') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 // 头部记录不能作为基准点,因为其 ref 列的值与 'ref1' 不匹配。 - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 -``` - -```sql -SELECT id, sequenceNextNode('backward', 'tail')(dt, page, ref = 'ref4', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 - 1970-01-01 09:00:03 1 B ref2 - 1970-01-01 09:00:04 1 B ref1 // 尾部不能作为基点,因为尾部的 ref 列与 'ref4' 不匹配。 -``` - -```sql -SELECT id, sequenceNextNode('forward', 'first_match')(dt, page, ref = 'ref3', page = 'A') FROM test_flow_basecond GROUP BY id; -``` - -dt id page ref -1970-01-01 09:00:01 1 A ref4 // 这一行不能作为基准点,因为 ref 列的值与 'ref3' 不一致。 -1970-01-01 09:00:02 1 A ref3 // 基准点 -1970-01-01 09:00:03 1 B ref2 // 结果 -1970-01-01 09:00:04 1 B ref1 - -```` - -```sql -SELECT id, sequenceNextNode('backward', 'last_match')(dt, page, ref = 'ref2', page = 'B') FROM test_flow_basecond GROUP BY id; - - dt id page ref - 1970-01-01 09:00:01 1 A ref4 - 1970-01-01 09:00:02 1 A ref3 // 结果 - 1970-01-01 09:00:03 1 B ref2 // 基准点 - 1970-01-01 09:00:04 1 B ref1 // 此行不能作为基准点,因为 ref 列与 'ref2' 不匹配。 -```` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md deleted file mode 100644 index 60d1ec92394..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/aggthrow.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -description: '此函数可用于测试异常安全性。它会在创建时以指定的概率抛出异常。' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/aggthrow -title: 'aggThrow' -doc_type: 'reference' ---- - -# aggThrow {#aggthrow} - -此函数可用于测试异常安全性。它会在创建时以指定概率抛出异常。 - -**语法** - -```sql -aggThrow(throw_prob) -``` - -**参数** - -* `throw_prob` — 在创建时抛出异常的概率。[Float64](../../data-types/float.md)。 - -**返回值** - -* 异常:`Code: 503. DB::Exception: Aggregate function aggThrow has thrown exception successfully`。 - -**示例** - -查询: - -```sql -SELECT number % 2 AS even, aggThrow(number) FROM numbers(10) GROUP BY even; -``` - -结果: - -```response -接收到异常: -代码:503. DB::Exception:聚合函数 aggThrow 已成功抛出异常:执行 AggregatingTransform 时发生。(AGGREGATE_FUNCTION_THROW) -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md deleted file mode 100644 index bf769176198..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/analysis_of_variance.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: '提供用于单因素方差分析(ANOVA 检验)的统计方法。它针对多组服从正态分布的观测值进行检验,以判断各组的均值是否相同。' -sidebar_position: 101 -slug: /sql-reference/aggregate-functions/reference/analysis_of_variance -title: 'analysisOfVariance' -doc_type: 'reference' ---- - -# analysisOfVariance {#analysisofvariance} - -提供用于单因素方差分析(ANOVA 检验)的统计检验方法。该检验针对多个服从正态分布的观测组,用于判断各组的均值是否相同。 - -**语法** - -```sql -analysisOfVariance(val, group_no) -``` - -别名:`anova` - -**参数** - -* `val`:数值。 -* `group_no`:`val` 所属的组号。 - -:::note -组从 0 开始编号,并且至少需要两个组才能执行检验。 -至少应有一个组的观测值数量大于 1。 -::: - -**返回值** - -* `(f_statistic, p_value)`。[Tuple](../../data-types/tuple.md)([Float64](../../data-types/float.md), [Float64](../../data-types/float.md))。 - -**示例** - -查询: - -```sql -SELECT analysisOfVariance(number, number % 2) FROM numbers(1048575); -``` - -结果: - -```response -┌─analysisOfVariance(number, modulo(number, 2))─┐ -│ (0,1) │ -└───────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md deleted file mode 100644 index 2ab90da2455..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/any.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: '选择列中第一个出现的值。' -sidebar_position: 102 -slug: /sql-reference/aggregate-functions/reference/any -title: 'any' -doc_type: 'reference' ---- - -# any {#any} - -选择列中首次遇到的值。 - -:::warning -由于查询可以以任意顺序执行,此函数的结果是非确定性的。 -如果您需要任意但确定性的结果,请使用函数 [`min`](../reference/min.md) 或 [`max`](../reference/max.md)。 -::: - -默认情况下,该函数从不返回 NULL,即会忽略输入列中的 NULL 值。 -但是,如果函数与 `RESPECT NULLS` 修饰符一起使用,则会返回读取到的第一个值,无论其是否为 NULL。 - -**语法** - -```sql -any(列) [RESPECT NULLS] -``` - -别名 `any(column)`(不带 `RESPECT NULLS`) - -* `any_value` -* [`first_value`](../reference/first_value.md)。 - -`any(column) RESPECT NULLS` 的别名 - -* `anyRespectNulls`, `any_respect_nulls` -* `firstValueRespectNulls`, `first_value_respect_nulls` -* `anyValueRespectNulls`, `any_value_respect_nulls` - -**参数** - -* `column`:列名。 - -**返回值** - -遇到的第一个值。 - -:::note -该函数的返回类型与输入相同,但会丢弃 LowCardinality。 -这意味着在没有任何输入行时,它会返回该类型的默认值(对于整数是 0,对于 `Nullable()` 列是 `Null`)。 -可以使用 `-OrNull` [组合器](../../../sql-reference/aggregate-functions/combinators.md) 来修改这种行为。 -::: - -**实现细节** - -在某些情况下,可以依赖于执行顺序。 -这适用于 `SELECT` 来自包含 `ORDER BY` 的子查询的情况。 - -当 `SELECT` 查询带有 `GROUP BY` 子句或至少一个聚合函数时,ClickHouse(与 MySQL 不同)要求 `SELECT`、`HAVING` 和 `ORDER BY` 子句中的所有表达式都必须由键或聚合函数计算得出。 -换句话说,从表中选出的每一列都必须要么用于键,要么出现在聚合函数中。 -要获得类似 MySQL 的行为,可以将其他列放入 `any` 聚合函数中。 - -**示例** - -查询: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES (NULL), ('Amsterdam'), ('New York'), ('Tokyo'), ('Valencia'), (NULL); - -SELECT any(city), anyRespectNulls(city) FROM tab; -``` - -```response -┌─any(city)─┬─anyRespectNulls(city)─┐ -│ Amsterdam │ ᴺᵁᴸᴸ │ -└───────────┴───────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md deleted file mode 100644 index aae8779f233..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anyheavy.md +++ /dev/null @@ -1,37 +0,0 @@ ---- -description: '使用 heavy hitters 算法选取一个经常出现的值。 - 如果在每个查询执行线程中,都存在一个值在超过一半的记录中出现,则返回该值。通常情况下,结果是非确定性的。' -sidebar_position: 104 -slug: /sql-reference/aggregate-functions/reference/anyheavy -title: 'anyHeavy' -doc_type: 'reference' ---- - -# anyHeavy {#anyheavy} - -使用 [heavy hitters](https://doi.org/10.1145/762471.762473) 算法选取一个高频出现的值。\ -如果存在某个值在查询的每个执行线程中出现的次数都超过该线程所处理记录数的一半,则返回该值。\ -通常,该结果是不确定的。 - -```sql -anyHeavy(column) -``` - -**参数** - -* `column` – 列名。 - -**示例** - -以 [OnTime](../../../getting-started/example-datasets/ontime.md) 数据集为例,从 `AirlineID` 列中选择任意一个经常出现的值。 - -```sql -SELECT anyHeavy(AirlineID) AS res -FROM ontime -``` - -```text -┌───res─┐ -│ 19690 │ -└───────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md deleted file mode 100644 index e3a82188493..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/anylast.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '选取列中最后一次出现的值。' -sidebar_position: 105 -slug: /sql-reference/aggregate-functions/reference/anylast -title: 'anyLast' -doc_type: 'reference' ---- - -# anyLast {#anylast} - -选择列中最后遇到的值。 - -:::warning -由于查询可以以任意顺序执行,此函数的结果是不确定的。 -如果您需要任意但确定的结果,请使用函数 [`min`](../reference/min.md) 或 [`max`](../reference/max.md)。 -::: - -默认情况下,该函数从不返回 NULL,即会忽略输入列中的 NULL 值。 -但是,如果函数与 `RESPECT NULLS` 修饰符一起使用,则会返回读取到的第一个值,无论其是否为 NULL。 - -**语法** - -```sql -anyLast(column) [RESPECT NULLS] -``` - -别名 `anyLast(column)`(不带 `RESPECT NULLS`) - -* [`last_value`](../reference/last_value.md)。 - -`anyLast(column) RESPECT NULLS` 的别名 - -* `anyLastRespectNulls`, `anyLast_respect_nulls` -* `lastValueRespectNulls`, `last_value_respect_nulls` - -**参数** - -* `column`:列名。 - -**返回值** - -* 最后遇到的值。 - -**示例** - -查询: - -```sql -CREATE TABLE tab (city Nullable(String)) ENGINE=Memory; - -INSERT INTO tab (city) VALUES ('Amsterdam'),(NULL),('New York'),('Tokyo'),('Valencia'),(NULL); - -SELECT anyLast(city), anyLastRespectNulls(city) FROM tab; -``` - -```response -┌─anyLast(city)─┬─anyLastRespectNulls(city)─┐ -│ Valencia │ ᴺᵁᴸᴸ │ -└───────────────┴───────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md deleted file mode 100644 index e6c8fdada9e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopk.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: '返回一个数组,包含指定列中近似出现频率最高的值及其计数。' -sidebar_position: 107 -slug: /sql-reference/aggregate-functions/reference/approxtopk -title: 'approx_top_k' -doc_type: 'reference' ---- - -# approx_top_k {#approx_top_k} - -返回一个数组,其中包含指定列中近似出现频率最高的值及其计数。结果数组按照值的近似出现频率降序排列(而不是按值本身排序)。 - -```sql -approx_top_k(N)(column) -approx_top_k(N, reserved)(column) -``` - -此函数不保证返回精确结果。在某些情况下可能会发生错误,并且可能返回的频繁值并不是最频繁出现的那些值。 - -`N` 的最大值为 `65536`。 - -**参数** - -* `N` — 要返回的元素个数。可选。默认值:10。 -* `reserved` — 为值预留的单元格数量。如果 uniq(column) > reserved,则 topK 函数的结果将是近似值。可选。默认值:N * 3。 - -**参数说明** - -* `column` — 要统计频率的值。 - -**示例** - -查询: - -```sql -SELECT approx_top_k(2)(k) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)); -``` - -结果: - -```text -┌─approx_top_k(2)(k)────┐ -│ [('y',3,0),('x',1,0)] │ -└───────────────────────┘ -``` - -# approx_top_count {#approx_top_count} - -是 `approx_top_k` 函数的别名。 - -**另请参阅** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md deleted file mode 100644 index ad56dc16fc6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/approxtopsum.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '返回指定列中近似出现次数最多的值及其计数的数组。' -sidebar_position: 108 -slug: /sql-reference/aggregate-functions/reference/approxtopsum -title: 'approx_top_sum' -doc_type: 'reference' ---- - -# approx_top_sum {#approx_top_sum} - -返回一个数组,其中包含指定列中近似最常出现的值及其计数。结果数组按值的近似出现频率降序排序(而不是按值本身排序)。此外,还会考虑各个值的权重。 - -```sql -approx_top_sum(N)(column, weight) -approx_top_sum(N, reserved)(column, weight) -``` - -此函数不保证返回结果的绝对准确性。在某些情况下,可能会产生误差,并且返回的高频值可能并不是实际出现频率最高的那些值。 - -`N` 的最大值为 `65536`。 - -**参数** - -* `N` — 要返回的元素数量。可选。默认值:10。 -* `reserved` — 为值预留的单元格数量。如果 uniq(column) > reserved,则 topK 函数的结果将是近似值。可选。默认值:N * 3。 - -**参数** - -* `column` — 用于计算频率的值。 -* `weight` — 权重。每个值在频率计算时按 `weight` 次计入。[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**示例** - -查询: - -```sql -SELECT approx_top_sum(2)(k, w) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -结果: - -```text -┌─approx_top_sum(2)(k, w)─┐ -│ [('z',10,0),('x',5,0)] │ -└─────────────────────────┘ -``` - -**另请参阅** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md deleted file mode 100644 index 10e42291b1d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmax.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -description: '计算最大 `val` 值对应的 `arg` 和 `val`。如果存在多行记录的 `val` 相同且为最大值,则返回哪一行对应的 `arg` 和 `val` 是不确定的。' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmax -title: 'argAndMax' -doc_type: 'reference' ---- - -# argAndMax {#argandmax} - -计算最大 `val` 值所对应的 `arg` 和 `val` 值。如果存在多行具有相同的最大 `val` 值,返回哪个关联的 `arg` 和 `val` 是不确定的。 -`arg` 和 `max` 两部分均作为[聚合函数](/sql-reference/aggregate-functions/index.md)运行,它们在处理过程中都会[跳过 `Null`](/sql-reference/aggregate-functions/index.md#null-processing),并在有非 `Null` 值可用时返回非 `Null` 值。 - -:::note -与 `argMax` 的唯一区别在于,`argAndMax` 会同时返回参数和值。 -::: - -**语法** - -```sql -argAndMax(arg, val) -``` - -**参数** - -* `arg` — 参数。 -* `val` — 值。 - -**返回值** - -* 与 `val` 的最大值对应的 `arg` 值。 -* `val` 的最大值 - -类型:元组,分别与 `arg`、`val` 的类型相匹配。 - -**示例** - -输入表: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -查询: - -```sql -SELECT argAndMax(user, salary) FROM salary; -``` - -结果: - -```text -┌─argAndMax(user, salary)─┐ -│ ('director',5000) │ -└─────────────────────────┘ -``` - -**扩展示例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), argAndMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─argAndMax(a, b)─┬─max(b)─┐ -│ b │ ('b',2) │ 3 │ -- argMax = b,因为它是第一个非 NULL 值,max(b) 来自另一行! -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMax(tuple(a), b) FROM test; -┌─argAndMax((a), b)─┐ -│ ((NULL),3) │-- 仅包含 `NULL` 值的 `Tuple` 本身不是 `NULL`,因此聚合函数不会因该 `NULL` 值而跳过该行 -└───────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA──┬─argMaxB─┐ -│ (NULL,3) │ 3 │ -- 可以使用 Tuple 获取对应 max(b) 的两列(所有列 - tuple(*)) -└──────────┴─────────┘ - -SELECT argAndMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argAndMax(a, b)─┬─max(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │-- 由于过滤条件,所有聚合行至少包含一个 `NULL` 值,因此所有行都被跳过,结果为 `NULL` -└─────────────────┴────────┘ - -SELECT argAndMax(a, (b,a)) FROM test; -┌─argAndMax(a, (b, a))─┐ -│ ('c',(2,'c')) │ -- 有两行 b=2,在 `Max` 中使用 `Tuple` 可以获取非第一个 `arg` -└──────────────────────┘ - -SELECT argAndMax(a, tuple(b)) FROM test; -┌─argAndMax(a, (b))─┐ -│ ('b',(2)) │ -- 可以在 `Max` 中使用 `Tuple` 以避免跳过 NULL 值 -└───────────────────┘ -``` - -**另请参阅** - -* [argMax](/sql-reference/aggregate-functions/reference/argmax.md) -* [Tuple](/sql-reference/data-types/tuple.md) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md deleted file mode 100644 index b5c97204188..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argandmin.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: '计算最小 `val` 值所对应的 `arg` 和 `val`。如果存在多行记录的最小 `val` 值相同,则返回哪一行对应的 `arg` 和 `val` 是不确定的。' -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/argandmin -title: 'argAndMin' -doc_type: 'reference' ---- - -# argAndMin {#argandmin} - -计算最小 `val` 值对应的 `arg` 和 `val`。如果有多行记录的最小 `val` 值相同,则返回哪一行对应的 `arg` 和 `val` 是不确定的。 -`arg` 和 `min` 两部分都作为[聚合函数](/sql-reference/aggregate-functions/index.md)起作用,在处理过程中都会[跳过 `Null`](/sql-reference/aggregate-functions/index.md#null-processing),并且在存在非 `Null` 值时返回非 `Null` 结果。 - -:::note -与 `argMin` 的唯一区别是 `argAndMin` 会同时返回参数和数值。 -::: - -**语法** - -```sql -argAndMin(arg, val) -``` - -**参数** - -* `arg` — 参数。 -* `val` — 值。 - -**返回值** - -* 与最小 `val` 值对应的 `arg` 值。 -* 最小的 `val` 值。 - -类型:与 `arg`、`val` 类型依次对应的元组。 - -**示例** - -输入表: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -查询: - -```sql -SELECT argAndMin(user, salary) FROM salary -``` - -结果: - -```text -┌─argAndMin(user, salary)─┐ -│ ('worker',1000) │ -└─────────────────────────┘ -``` - -**详细示例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a,b), argAndMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─argAndMin(a, b)─┬─min(b)─┐ -│ a │ ('a',1) │ 0 │ -- argMin = a,因为它是第一个非 `NULL` 值,而 min(b) 来自另一行! -└──────────────┴─────────────────┴────────┘ - -SELECT argAndMin(tuple(a), b) FROM test; -┌─argAndMin((a), b)─┐ -│ ((NULL),0) │ -- 'a' `Tuple` 仅包含一个 `NULL` 值,但其本身不是 `NULL`,因此聚合函数不会因该 `NULL` 值而跳过该行 -└───────────────────┘ - -SELECT (argAndMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA──┬─argMinB─┐ -│ (NULL,0) │ 0 │ -- 可以使用 `Tuple` 获取对应 min(b) 的两列(全部列 - tuple(*)) -└──────────┴─────────┘ - -SELECT argAndMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argAndMin(a, b)─┬─min(b)─┐ -│ ('',0) │ ᴺᵁᴸᴸ │ -- 由于过滤条件,所有聚合行至少包含一个 `NULL` 值,因此所有行都被跳过,结果为 `NULL` -└─────────────────┴────────┘ - -SELECT argAndMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin(a, (b, a))─┬─min((b, a))─┐ -│ ('a',(1,'a')) │ (0,NULL) │ -- 'a' 是 min 对应的第一个非 `NULL` 值 -└──────────────────────┴─────────────┘ - -SELECT argAndMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argAndMin((a, b), (b, a))─┬─min((b, a))─┐ -│ ((NULL,0),(0,NULL)) │ (0,NULL) │ -- argAndMin 在此返回 ((NULL,0),(0,NULL)),因为 `Tuple` 允许不跳过 `NULL` 值,此时 min(tuple(b, a)) 是该数据集的最小值 -└───────────────────────────┴─────────────┘ - -SELECT argAndMin(a, tuple(b)) FROM test; -┌─argAndMin(a, (b))─┐ -│ ('a',(1)) │ -- 可在 `min` 中使用 `Tuple` 以不跳过 b 为 `NULL` 值的行。 -└───────────────────┘ -``` - -**另请参阅** - -* [argMin](/sql-reference/aggregate-functions/reference/argmin.md) -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md deleted file mode 100644 index a40be1e45b5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmax.md +++ /dev/null @@ -1,112 +0,0 @@ ---- -description: '计算最大 `val` 值对应的 `arg` 值。' -sidebar_position: 109 -slug: /sql-reference/aggregate-functions/reference/argmax -title: 'argMax' -doc_type: 'reference' ---- - -# argMax {#argmax} - -计算最大 `val` 所对应的 `arg` 值。如果存在多行的最大 `val` 相同,则返回的对应 `arg` 值是不确定的。 -`arg` 和 `max` 两个部分都作为[聚合函数](/sql-reference/aggregate-functions/index.md)运行,在处理过程中都会[跳过 `Null`](/sql-reference/aggregate-functions/index.md#null-processing),并在存在非 `Null` 值时返回非 `Null` 值。 - -**语法** - -```sql -argMax(arg, val) -``` - -**参数** - -* `arg` — 参数。 -* `val` — 值。 - -**返回值** - -* 与最大 `val` 值对应的 `arg`。 - -类型:与 `arg` 的类型相同。 - -**示例** - -输入表: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -查询: - -```sql -SELECT argMax(user, salary) FROM salary; -``` - -结果: - -```text -┌─argMax(user, salary)─┐ -│ director │ -└──────────────────────┘ -``` - -**扩展示例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES(('a', 1), ('b', 2), ('c', 2), (NULL, 3), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ 3 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMax(a, b), max(b) FROM test; -┌─argMax(a, b)─┬─max(b)─┐ -│ b │ 3 │ -- argMax = 'b',因为它是第一个非 NULL 值,max(b) 来自另一行! -└──────────────┴────────┘ - -SELECT argMax(tuple(a), b) FROM test; -┌─argMax(tuple(a), b)─┐ -│ (NULL) │ -- 仅包含一个 `NULL` 值的 `Tuple` 本身不是 `NULL`,因此聚合函数不会因该 `NULL` 值而跳过该行 -└─────────────────────┘ - -SELECT (argMax((a, b), b) as t).1 argMaxA, t.2 argMaxB FROM test; -┌─argMaxA─┬─argMaxB─┐ -│ ᴺᵁᴸᴸ │ 3 │ -- 可以使用 Tuple 获取对应 max(b) 的两列(全部列 - tuple(*)) -└─────────┴─────────┘ - -SELECT argMax(a, b), max(b) FROM test WHERE a IS NULL AND b IS NULL; -┌─argMax(a, b)─┬─max(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- 由于过滤条件,所有聚合行都至少包含一个 `NULL` 值,因此所有行都被跳过,结果为 `NULL` -└──────────────┴────────┘ - -SELECT argMax(a, (b,a)) FROM test; -┌─argMax(a, tuple(b, a))─┐ -│ c │ -- 有两行 b=2,在 `Max` 中使用 `Tuple` 可以获取非第一个 `arg` -└────────────────────────┘ - -SELECT argMax(a, tuple(b)) FROM test; -┌─argMax(a, tuple(b))─┐ -│ b │ -- 可以在 `Max` 中使用 `Tuple` 以避免跳过 NULL 值 -└─────────────────────┘ -``` - -**另见** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md deleted file mode 100644 index 4101b6ff90d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/argmin.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: '计算最小 `val` 值对应的 `arg` 值。如果存在多行记录的 `val` 具有相同的最小值,则返回哪一个对应的 `arg` 是非确定性的。' -sidebar_position: 110 -slug: /sql-reference/aggregate-functions/reference/argmin -title: 'argMin' -doc_type: 'reference' ---- - -# argMin {#argmin} - -计算具有最小 `val` 值时对应的 `arg` 值。如果存在多行的 `val` 相同且为最小值,则最终返回哪一行的 `arg` 是不确定的。 -`arg` 部分和 `min` 部分都作为[聚合函数](/sql-reference/aggregate-functions/index.md)工作,它们在处理过程中都会[跳过 `Null`](/sql-reference/aggregate-functions/index.md#null-processing),并且在存在非 `Null` 值时返回非 `Null` 值。 - -**语法** - -```sql -argMin(arg, val) -``` - -**参数** - -* `arg` — 参数。 -* `val` — 值。 - -**返回值** - -* `val` 最小值对应的 `arg`。 - -类型:与 `arg` 相同。 - -**示例** - -输入表: - -```text -┌─user─────┬─salary─┐ -│ director │ 5000 │ -│ manager │ 3000 │ -│ worker │ 1000 │ -└──────────┴────────┘ -``` - -查询: - -```sql -SELECT argMin(user, salary) FROM salary -``` - -结果: - -```text -┌─argMin(user, salary)─┐ -│ worker │ -└──────────────────────┘ -``` - -**扩展示例** - -```sql -CREATE TABLE test -( - a Nullable(String), - b Nullable(Int64) -) -ENGINE = Memory AS -SELECT * -FROM VALUES((NULL, 0), ('a', 1), ('b', 2), ('c', 2), (NULL, NULL), ('d', NULL)); - -SELECT * FROM test; -┌─a────┬────b─┐ -│ ᴺᵁᴸᴸ │ 0 │ -│ a │ 1 │ -│ b │ 2 │ -│ c │ 2 │ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ d │ ᴺᵁᴸᴸ │ -└──────┴──────┘ - -SELECT argMin(a, b), min(b) FROM test; -┌─argMin(a, b)─┬─min(b)─┐ -│ a │ 0 │ -- argMin = a,因为它是第一个非 `NULL` 值,min(b) 来自另一行! -└──────────────┴────────┘ - -SELECT argMin(tuple(a), b) FROM test; -┌─argMin(tuple(a), b)─┐ -│ (NULL) │ -- 仅包含 `NULL` 值的 `Tuple` 本身不是 `NULL`,因此聚合函数不会因该 `NULL` 值而跳过该行 -└─────────────────────┘ - -SELECT (argMin((a, b), b) as t).1 argMinA, t.2 argMinB from test; -┌─argMinA─┬─argMinB─┐ -│ ᴺᵁᴸᴸ │ 0 │ -- 可以使用 `Tuple` 获取对应 max(b) 的所有列(all - tuple(*)) -└─────────┴─────────┘ - -SELECT argMin(a, b), min(b) FROM test WHERE a IS NULL and b IS NULL; -┌─argMin(a, b)─┬─min(b)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -- 由于过滤条件,所有聚合行都至少包含一个 `NULL` 值,因此所有行都被跳过,结果为 `NULL` -└──────────────┴────────┘ - -SELECT argMin(a, (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(a, tuple(b, a))─┬─min(tuple(b, a))─┐ -│ d │ (NULL,NULL) │ -- 'd' 是 min 对应的第一个非 `NULL` 值 -└────────────────────────┴──────────────────┘ - -SELECT argMin((a, b), (b, a)), min(tuple(b, a)) FROM test; -┌─argMin(tuple(a, b), tuple(b, a))─┬─min(tuple(b, a))─┐ -│ (NULL,NULL) │ (NULL,NULL) │ -- argMin 在此返回 (NULL,NULL),因为 `Tuple` 允许不跳过 `NULL`,而 min(tuple(b, a)) 在此情况下是该数据集的最小值 -└──────────────────────────────────┴──────────────────┘ - -SELECT argMin(a, tuple(b)) FROM test; -┌─argMin(a, tuple(b))─┐ -│ d │ -- `Tuple` 可用于 `min` 中以不跳过 b 为 `NULL` 值的行 -└─────────────────────┘ -``` - -**另请参见** - -* [Tuple](/sql-reference/data-types/tuple.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md deleted file mode 100644 index be36e5a80d1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avg.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: '计算算术平均数。' -sidebar_position: 112 -slug: /sql-reference/aggregate-functions/reference/avg -title: 'avg' -doc_type: 'reference' ---- - -# avg {#avg} - -计算算术平均值。 - -**语法** - -```sql -avg(x) -``` - -**参数** - -* `x` — 输入值,必须是 [Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 算术平均值,类型始终为 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入参数 `x` 为空,则返回 `NaN`。 - -**示例** - -查询: - -```sql -SELECT avg(x) FROM VALUES('x Int8', 0, 1, 2, 3, 4, 5); -``` - -结果: - -```text -┌─avg(x)─┐ -│ 2.5 │ -└────────┘ -``` - -**示例** - -创建一个临时表: - -查询语句: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -``` - -计算算术平均值: - -查询: - -```sql -SELECT avg(t) FROM test; -``` - -结果: - -```text -┌─avg(x)─┐ -│ nan │ -└────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md deleted file mode 100644 index 539b3ed1538..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/avgweighted.md +++ /dev/null @@ -1,101 +0,0 @@ ---- -description: '计算加权算术平均数。' -sidebar_position: 113 -slug: /sql-reference/aggregate-functions/reference/avgweighted -title: 'avgWeighted' -doc_type: 'reference' ---- - -# avgWeighted {#avgweighted} - -计算[加权算术平均值](https://en.wikipedia.org/wiki/Weighted_arithmetic_mean)。 - -**语法** - -```sql -avgWeighted(x, weight) -``` - -**参数** - -* `x` — 数值。 -* `weight` — 对应数值的权重。 - -`x` 和 `weight` 都必须是 -[整数类型 (Integer)](../../../sql-reference/data-types/int-uint.md) 或 [浮点类型 (floating-point)](../../../sql-reference/data-types/float.md), -但它们可以是不同的数据类型。 - -**返回值** - -* 当所有权重都等于 0,或传入的权重参数为空时,返回 `NaN`。 -* 否则返回加权平均值。 - -**返回类型** 始终为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (4, 1), (1, 0), (10, 2)) -``` - -结果: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**示例** - -查询: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Float64', (4, 1), (1, 0), (10, 2)) -``` - -结果: - -```text -┌─avgWeighted(x, weight)─┐ -│ 8 │ -└────────────────────────┘ -``` - -**示例** - -查询: - -```sql -SELECT avgWeighted(x, w) -FROM VALUES('x Int8, w Int8', (0, 0), (1, 0), (10, 0)) -``` - -结果: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` - -**示例** - -查询: - -```sql -CREATE TABLE test (t UInt8) ENGINE = Memory; -SELECT avgWeighted(t) FROM test -``` - -结果: - -```text -┌─avgWeighted(x, weight)─┐ -│ nan │ -└────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md deleted file mode 100644 index ad8f1bbea83..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/boundrat.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '在一组值中计算最左端和最右端点之间斜率的聚合函数。' -sidebar_position: 114 -slug: /sql-reference/aggregate-functions/reference/boundingRatio -title: 'boundingRatio' -doc_type: 'reference' ---- - -在一组值中计算最左端和最右端点之间斜率的聚合函数。 - -示例: - -示例数据: - -```sql -SELECT - number, - number * 1.5 -FROM numbers(10) -``` - -```response -┌─number─┬─multiply(number, 1.5)─┐ -│ 0 │ 0 │ -│ 1 │ 1.5 │ -│ 2 │ 3 │ -│ 3 │ 4.5 │ -│ 4 │ 6 │ -│ 5 │ 7.5 │ -│ 6 │ 9 │ -│ 7 │ 10.5 │ -│ 8 │ 12 │ -│ 9 │ 13.5 │ -└────────┴───────────────────────┘ -``` - -`boundingRatio()` 函数返回最左端点和最右端点之间连线的斜率。在上述数据中,这两个点分别是 `(0,0)` 和 `(9,13.5)`。 - -```sql -SELECT boundingRatio(number, number * 1.5) -FROM numbers(10) -``` - -```response -┌─boundingRatio(number, multiply(number, 1.5))─┐ -│ 1.5 │ -└──────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md deleted file mode 100644 index ad761e08c50..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/categoricalinformationvalue.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: '对每个类别计算 `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - - log(P(tag = 0)))` 的值。' -sidebar_position: 115 -slug: /sql-reference/aggregate-functions/reference/categoricalinformationvalue -title: 'categoricalInformationValue' -doc_type: 'reference' ---- - -对每个类别计算 `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` 的值。 - -```sql -categoricalInformationValue(category1, category2, ..., tag) -``` - -结果表明,离散(类别型)特征 `[category1, category2, ...]` 是如何对预测 `tag` 值的学习模型产生贡献的。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md deleted file mode 100644 index 407b473ccca..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/contingency.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '`contingency` 函数计算列联系数(contingency coefficient),用于度量数据表中两列之间的关联程度。其计算方式与 `cramersV` 函数类似,但平方根内的分母不同。' -sidebar_position: 116 -slug: /sql-reference/aggregate-functions/reference/contingency -title: 'contingency' -doc_type: 'reference' ---- - -# contingency {#contingency} - -`contingency` 函数用于计算[列联系数(contingency coefficient)](https://en.wikipedia.org/wiki/Contingency_table#Cram%C3%A9r's_V_and_the_contingency_coefficient_C),该值用于衡量表中两列之间的关联强度。它的计算方式与 [`cramersV` 函数](./cramersv.md)类似,但在平方根中的分母不同。 - -**语法** - -```sql -列联表(column1, column2) -``` - -**参数** - -* `column1` 和 `column2` 是要比较的列 - -**返回值** - -* 一个介于 0 和 1 之间的值。结果越大,两列之间的关联越紧密。 - -**返回类型** 始终为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -下面所比较的两列之间的关联度较弱。我们还给出了 `cramersV` 的结果(用于对比): - -```sql -SELECT - cramersV(a, b), - contingency(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -结果: - -```response -┌─────cramersV(a, b)─┬──contingency(a, b)─┐ -│ 0.5798088336225178 │ 0.0817230766271248 │ -└────────────────────┴────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md deleted file mode 100644 index 685fffa9b7b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corr.md +++ /dev/null @@ -1,64 +0,0 @@ ---- -description: '计算皮尔逊相关系数。' -sidebar_position: 117 -slug: /sql-reference/aggregate-functions/reference/corr -title: 'corr' -doc_type: 'reference' ---- - - - -# corr {#corr} - -计算[皮尔逊相关系数](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient): - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -
-:::note 此函数使用数值不稳定的算法。如果计算中需要[数值稳定性](https://en.wikipedia.org/wiki/Numerical_stability),请使用 [`corrStable`](../reference/corrstable.md) 函数。该函数速度较慢,但能提供更准确的结果。 ::: - -**语法** - -```sql -corr(x, y) -``` - -**参数** - -- `x` — 第一个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)。 -- `y` — 第二个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)。 - -**返回值** - -- 皮尔逊相关系数。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corr(x_value, y_value) -FROM series; -``` - -结果: - -```response -┌─corr(x_value, y_value)─┐ -│ 0.1730265755453256 │ -└────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md deleted file mode 100644 index fe4f65dbe2e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '计算 N 个变量的相关矩阵。' -sidebar_position: 118 -slug: /sql-reference/aggregate-functions/reference/corrmatrix -title: 'corrMatrix' -doc_type: 'reference' ---- - -# corrMatrix {#corrmatrix} - -计算 N 个变量的相关矩阵。 - -**语法** - -```sql -corrMatrix(x[, ...]) -``` - -**参数** - -* `x` — 数量不定的参数。[`(U)Int8/16/32/64`](../../data-types/int-uint.md)、[`Float*`](../../data-types/float.md)。 - -**返回值** - -* 相关矩阵。[`Array`](../../data-types/array.md)([`Array`](../../data-types/array.md)([`Float64`](../../data-types/float.md)))。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(corrMatrix(a, b, c, d))) AS corrMatrix -FROM test; -``` - -结果: - -```response - ┌─corrMatrix─────────────┐ -1. │ [1,-0.096,0.243,0.746] │ -2. │ [-0.096,1,0.173,0.106] │ -3. │ [0.243,0.173,1,0.258] │ -4. │ [0.746,0.106,0.258,1] │ - └────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md deleted file mode 100644 index 45dfacad064..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/corrstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: '计算皮尔逊相关系数,但使用数值上更稳定的算法。' -sidebar_position: 119 -slug: /sql-reference/aggregate-functions/reference/corrstable -title: 'corrStable' -doc_type: 'reference' ---- - - - -# corrStable {#corrstable} - -计算[皮尔逊相关系数](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient): - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{\sqrt{\Sigma{(x - \bar{x})^2} * \Sigma{(y - \bar{y})^2}}} -$$ - -与 [`corr`](../reference/corr.md) 函数类似,但使用数值稳定算法。因此,`corrStable` 比 `corr` 慢,但能产生更精确的结果。 - -**语法** - -```sql -corrStable(x, y) -``` - -**参数** - -- `x` — 第一个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 第二个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -- 皮尔逊相关系数。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series -( - i UInt32, - x_value Float64, - y_value Float64 -) -ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT corrStable(x_value, y_value) -FROM series; -``` - -结果: - -```response -┌─corrStable(x_value, y_value)─┐ -│ 0.17302657554532558 │ -└──────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md deleted file mode 100644 index 39b6a97b7da..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/count.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '计算行数或非 NULL 值的个数。' -sidebar_position: 120 -slug: /sql-reference/aggregate-functions/reference/count -title: 'count' -doc_type: 'reference' ---- - -# count {#count} - -统计行数或非 NULL 值的数量。 - -ClickHouse 支持以下 `count` 语法形式: - -* `count(expr)` 或 `COUNT(DISTINCT expr)`。 -* `count()` 或 `COUNT(*)`。`count()` 语法是 ClickHouse 特有的。 - -**参数** - -该函数可以接受: - -* 零个参数。 -* 一个[表达式](/sql-reference/syntax#expressions)。 - -**返回值** - -* 如果函数在没有参数的情况下被调用,则统计行数。 -* 如果传入了[表达式](/sql-reference/syntax#expressions),则函数统计该表达式返回非 NULL 的次数。如果表达式返回的是 [Nullable](../../../sql-reference/data-types/nullable.md) 类型的值,则 `count` 的结果仍然不是 `Nullable`。如果该表达式对所有行都返回 `NULL`,函数返回 0。 - -在这两种情况下,返回值的类型都是 [UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**详情** - -ClickHouse 支持 `COUNT(DISTINCT ...)` 语法。这种写法的行为取决于 [count_distinct_implementation](../../../operations/settings/settings.md#count_distinct_implementation) 设置。它定义执行该操作时使用哪个 [uniq*](/sql-reference/aggregate-functions/reference/uniq) 函数。默认使用 [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) 函数。 - -`SELECT count() FROM table` 查询默认会利用 MergeTree 的元数据进行优化。如果你需要使用行级安全控制(row-level security),请通过 [optimize_trivial_count_query](/operations/settings/settings#optimize_trivial_count_query) 设置禁用此优化。 - -同时,可以通过启用 [optimize_functions_to_subcolumns](/operations/settings/settings#optimize_functions_to_subcolumns) 设置来优化 `SELECT count(nullable_column) FROM table` 查询。当 `optimize_functions_to_subcolumns = 1` 时,函数只会读取 [null](../../../sql-reference/data-types/nullable.md#finding-null) 子列,而不是读取并处理整个列的数据。查询 `SELECT count(n) FROM table` 会被重写为 `SELECT sum(NOT n.null) FROM table`。 - -**改进 COUNT(DISTINCT expr) 的性能** - -如果你的 `COUNT(DISTINCT expr)` 查询速度较慢,考虑添加 [`GROUP BY`](/sql-reference/statements/select/group-by) 子句以提升并行化能力。你也可以使用[投影](../../../sql-reference/statements/alter/projection.md),在 `COUNT(DISTINCT target_col)` 所使用的目标列上创建索引。 - -**示例** - -示例 1: - -```sql -SELECT count() FROM t -``` - -```text -┌─count()─┐ -│ 5 │ -└─────────┘ -``` - -示例 2: - -```sql -SELECT name, value FROM system.settings WHERE name = 'count_distinct_implementation' -``` - -```text -┌─name──────────────────────────┬─value─────┐ -│ count_distinct_implementation │ uniqExact │ -└───────────────────────────────┴───────────┘ -``` - -```sql -SELECT count(DISTINCT num) FROM t -``` - -```text -┌─uniqExact(num)─┐ -│ 3 │ -└────────────────┘ -``` - -此示例表明,根据 `count_distinct_implementation` 设置的值,`count(DISTINCT num)` 会由 `uniqExact` 函数执行。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md deleted file mode 100644 index 25f7fc68823..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpop.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '计算总体协方差' -sidebar_position: 121 -slug: /sql-reference/aggregate-functions/reference/covarpop -title: 'covarPop' -doc_type: 'reference' ---- - - - -# covarPop {#covarpop} - -计算总体协方差: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -:::note -此函数使用数值不稳定的算法。如果计算中需要[数值稳定性](https://en.wikipedia.org/wiki/Numerical_stability),请使用 [`covarPopStable`](../reference/covarpopstable.md) 函数。该函数运行速度较慢,但提供更低的计算误差。 -::: - -**语法** - -```sql -covarPop(x, y) -``` - -**参数** - -- `x` — 第一个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 第二个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -- `x` 和 `y` 之间的总体协方差。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6, -4.4),(2, -9.6, 3),(3, -1.3, -4),(4, 5.3, 9.7),(5, 4.4, 0.037),(6, -8.6, -7.8),(7, 5.1, 9.3),(8, 7.9, -3.6),(9, -8.2, 0.62),(10, -3, 7.3); -``` - -```sql -SELECT covarPop(x_value, y_value) -FROM series; -``` - -结果: - -```reference -┌─covarPop(x_value, y_value)─┐ -│ 6.485648 │ -└────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md deleted file mode 100644 index fdfcd8dddce..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '返回 N 个变量之间的总体协方差矩阵。' -sidebar_position: 122 -slug: /sql-reference/aggregate-functions/reference/covarpopmatrix -title: 'covarPopMatrix' -doc_type: 'reference' ---- - -# covarPopMatrix {#covarpopmatrix} - -返回基于 N 个变量计算得到的总体协方差矩阵。 - -**语法** - -```sql -covarPopMatrix(x[, ...]) -``` - -**参数** - -* `x` — 一组可变数量的参数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -* 总体协方差矩阵。[Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md)))。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarPopMatrix(a, b, c, d))) AS covarPopMatrix -FROM test; -``` - -结果: - -```reference - ┌─covarPopMatrix────────────┐ -1. │ [8.25,-1.76,4.08,6.748] │ -2. │ [-1.76,41.07,6.486,2.132] │ -3. │ [4.08,6.486,34.21,4.755] │ -4. │ [6.748,2.132,4.755,9.93] │ - └───────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md deleted file mode 100644 index 2cfa9daf828..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarpopstable.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: '计算总体协方差值' -sidebar_position: 123 -slug: /sql-reference/aggregate-functions/reference/covarpopstable -title: 'covarPopStable' -doc_type: 'reference' ---- - - - -# covarPopStable {#covarpopstable} - -计算总体协方差的值: - -$$ -\frac{\Sigma{(x - \bar{x})(y - \bar{y})}}{n} -$$ - -该函数与 [covarPop](../reference/covarpop.md) 函数类似,但使用数值稳定的算法。因此,`covarPopStable` 比 `covarPop` 更慢,但能得到更精确的结果。 - -**语法** - -```sql -covarPop(x, y) -``` - -**参数** - -- `x` — 第一个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -- `y` — 第二个变量。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -- `x` 与 `y` 之间的总体协方差。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarPopStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -结果: - -```reference -┌─covarPopStable(x_value, y_value)─┐ -│ 6.485648 │ -└──────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md deleted file mode 100644 index 418aedde387..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsamp.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: '计算 `Σ((x - x̅)(y - y̅)) / (n - 1)` 的值' -sidebar_position: 124 -slug: /sql-reference/aggregate-functions/reference/covarsamp -title: 'covarSamp' -doc_type: 'reference' ---- - -# covarSamp {#covarsamp} - -计算 `Σ((x - x̅)(y - y̅)) / (n - 1)` 的值。 - -:::note -此函数使用数值不稳定的算法。如果在计算中需要[数值稳定性](https://en.wikipedia.org/wiki/Numerical_stability),请使用 [`covarSampStable`](../reference/covarsamp.md) 函数。该函数运行速度较慢,但能提供更低的计算误差。 -::: - -**语法** - -```sql -covarSamp(x, y) -``` - -**参数** - -* `x` — 第一个变量。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -* `y` — 第二个变量。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -* `x` 和 `y` 之间的样本协方差。当 `n <= 1` 时,返回 `nan`。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -结果: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ 7.206275555555556 │ -└─────────────────────────────┘ -``` - -查询: - -```sql -SELECT covarSamp(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); - -``` - -结果: - -```reference -┌─covarSamp(x_value, y_value)─┐ -│ nan │ -└─────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md deleted file mode 100644 index 70527227078..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampmatrix.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '返回 N 个变量之间的样本协方差矩阵。' -sidebar_position: 125 -slug: /sql-reference/aggregate-functions/reference/covarsampmatrix -title: 'covarSampMatrix' -doc_type: 'reference' ---- - -# covarSampMatrix {#covarsampmatrix} - -返回 N 个变量的样本协方差矩阵。 - -**语法** - -```sql -covarSampMatrix(x[, ...]) -``` - -**参数** - -* `x` — 数量可变的参数。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -* 样本协方差矩阵。[Array](../../data-types/array.md)([Array](../../data-types/array.md)([Float64](../../data-types/float.md)))。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test; -CREATE TABLE test -( - a UInt32, - b Float64, - c Float64, - d Float64 -) -ENGINE = Memory; -INSERT INTO test(a, b, c, d) VALUES (1, 5.6, -4.4, 2.6), (2, -9.6, 3, 3.3), (3, -1.3, -4, 1.2), (4, 5.3, 9.7, 2.3), (5, 4.4, 0.037, 1.222), (6, -8.6, -7.8, 2.1233), (7, 5.1, 9.3, 8.1222), (8, 7.9, -3.6, 9.837), (9, -8.2, 0.62, 8.43555), (10, -3, 7.3, 6.762); -``` - -```sql -SELECT arrayMap(x -> round(x, 3), arrayJoin(covarSampMatrix(a, b, c, d))) AS covarSampMatrix -FROM test; -``` - -结果: - -```reference - ┌─covarSampMatrix─────────────┐ -1. │ [9.167,-1.956,4.534,7.498] │ -2. │ [-1.956,45.634,7.206,2.369] │ -3. │ [4.534,7.206,38.011,5.283] │ -4. │ [7.498,2.369,5.283,11.034] │ - └─────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md deleted file mode 100644 index 374468c9bcb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/covarsampstable.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: '类似于 covarSamp,但计算速度更慢,计算误差更小。' -sidebar_position: 126 -slug: /sql-reference/aggregate-functions/reference/covarsampstable -title: 'covarSampStable' -doc_type: 'reference' ---- - -# covarSampStable {#covarsampstable} - -计算 `Σ((x - x̅)(y - y̅)) / (n - 1)` 的值。与 [covarSamp](../reference/covarsamp.md) 类似,但计算速度更慢,计算误差更小。 - -**语法** - -```sql -covarSampStable(x, y) -``` - -**参数** - -* `x` — 第一个变量。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 -* `y` — 第二个变量。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal](../../data-types/decimal.md)。 - -**返回值** - -* `x` 与 `y` 之间的样本协方差。若 `n <= 1`,则返回 `inf`。返回类型:[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS series; -CREATE TABLE series(i UInt32, x_value Float64, y_value Float64) ENGINE = Memory; -INSERT INTO series(i, x_value, y_value) VALUES (1, 5.6,-4.4),(2, -9.6,3),(3, -1.3,-4),(4, 5.3,9.7),(5, 4.4,0.037),(6, -8.6,-7.8),(7, 5.1,9.3),(8, 7.9,-3.6),(9, -8.2,0.62),(10, -3,7.3); -``` - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series -); -``` - -结果: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ 7.206275555555556 │ -└───────────────────────────────────┘ -``` - -查询: - -```sql -SELECT covarSampStable(x_value, y_value) -FROM -( - SELECT - x_value, - y_value - FROM series LIMIT 1 -); -``` - -结果: - -```reference -┌─covarSampStable(x_value, y_value)─┐ -│ inf │ -└───────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md deleted file mode 100644 index bd2096c9242..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersv.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -description: '`cramersV` 函数的结果范围为 0(表示变量之间没有关联)到 1,且只有在每个取值都能被另一个变量完全确定时才能达到 1。它可以被视为两个变量之间的关联程度,相对于它们在理论上可能达到的最大变异的百分比。' -sidebar_position: 127 -slug: /sql-reference/aggregate-functions/reference/cramersv -title: 'cramersV' -doc_type: 'reference' ---- - -# cramersV {#cramersv} - -[Cramer's V](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V)(有时称为 Cramer's phi)是一种用于衡量表中两列之间关联程度的统计量。`cramersV` 函数的结果范围从 0(表示变量之间没有关联)到 1,且只有当每个值都可以被另一个值完全确定时才会达到 1。它可以被视为两个变量之间的关联程度,占其最大可能变化范围的百分比。 - -:::note -关于偏差校正版本的 Cramer's V,请参见:[cramersVBiasCorrected](./cramersvbiascorrected.md) -::: - -**语法** - -```sql -cramersV(column1, column2) -``` - -**参数** - -* `column1`: 要比较的第一列。 -* `column2`: 要比较的第二列。 - -**返回值** - -* 一个介于 0(表示列值之间没有关联)到 1(完全相关)之间的值。 - -类型:固定为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -下面比较的这两列彼此之间没有关联,因此 `cramersV` 的结果为 0: - -查询: - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 3 AS a, - number % 5 AS b - FROM - numbers(150) - ); -``` - -结果: - -```response -┌─cramersV(a, b)─┐ -│ 0 │ -└────────────────┘ -``` - -下方这两列之间的相关性较强,因此 `cramersV` 的结果值较高: - -```sql -SELECT - cramersV(a, b) -FROM - ( - SELECT - number % 10 AS a, - if(number % 12 = 0, (number + 1) % 5, number % 5) AS b - FROM - numbers(150) - ); -``` - -结果: - -```response -┌─────cramersV(a, b)─┐ -│ 0.9066801892162646 │ -└────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md deleted file mode 100644 index f1d51c9010c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/cramersvbiascorrected.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -description: '计算 Cramer''s V 统计量,但进行了偏倚校正。' -sidebar_position: 128 -slug: /sql-reference/aggregate-functions/reference/cramersvbiascorrected -title: 'cramersVBiasCorrected' -doc_type: 'reference' ---- - -# cramersVBiasCorrected {#cramersvbiascorrected} - -Cramer's V 是用于衡量表中两列之间关联程度的指标。[`cramersV` 函数](./cramersv.md) 的结果范围从 0(表示变量之间没有关联)到 1,且只有在每个值都被另一个值完全决定时才会达到 1。该函数可能存在较大偏差,因此此版本的 Cramer's V 使用了[偏差校正](https://en.wikipedia.org/wiki/Cram%C3%A9r%27s_V#Bias_correction)。 - -**语法** - -```sql -cramersVBiasCorrected(column1, column2) -``` - -**参数** - -* `column1`: 要比较的第一列。 -* `column2`: 要比较的第二列。 - -**返回值** - -* 一个在 0(表示列值之间没有关联)到 1(完全关联)之间的数值。 - -类型:始终为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -下面被比较的这两列彼此之间具有中等程度的关联。请注意,`cramersVBiasCorrected` 的结果小于 `cramersV` 的结果: - -查询: - -```sql -SELECT - cramersV(a, b), - cramersVBiasCorrected(a ,b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -结果: - -```response -┌─────cramersV(a, b)─┬─cramersVBiasCorrected(a, b)─┐ -│ 0.5798088336225178 │ 0.5305112825189074 │ -└────────────────────┴─────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md deleted file mode 100644 index a888e874874..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '对相邻行之间的数值差求和。' -sidebar_position: 129 -slug: /sql-reference/aggregate-functions/reference/deltasum -title: 'deltaSum' -doc_type: 'reference' ---- - - - -# deltaSum {#deltasum} - -对相邻行之间的算术差值求和。如果差值为负数,则会被忽略。 - -:::note -为确保此函数正常工作,底层数据必须已排序。如果你打算在[物化视图](/sql-reference/statements/create/view#materialized-view)中使用此函数,通常更应使用 [deltaSumTimestamp](/sql-reference/aggregate-functions/reference/deltasumtimestamp) 方法。 -::: - -**语法** - -```sql -deltaSum(value) -``` - -**参数** - -* `value` — 输入值,必须是 [Integer](../../data-types/int-uint.md) 或 [Float](../../data-types/float.md) 类型。 - -**返回值** - -* 返回的算术差,类型为 `Integer` 或 `Float`。 - -**示例** - -查询: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3])); -``` - -结果: - -```text -┌─deltaSum(arrayJoin([1, 2, 3]))─┐ -│ 2 │ -└────────────────────────────────┘ -``` - -查询: - -```sql -SELECT deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3])); -``` - -结果: - -```text -┌─deltaSum(arrayJoin([1, 2, 3, 0, 3, 4, 2, 3]))─┐ -│ 7 │ -└───────────────────────────────────────────────┘ -``` - -查询: - -```sql -SELECT deltaSum(arrayJoin([2.25, 3, 4.5])); -``` - -结果: - -```text -┌─deltaSum(arrayJoin([2.25, 3, 4.5]))─┐ -│ 2.25 │ -└─────────────────────────────────────┘ -``` - - -## 另请参阅 {#see-also} - -- [runningDifference](/sql-reference/functions/other-functions#runningDifference) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md deleted file mode 100644 index 44cb47e3442..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/deltasumtimestamp.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '累加相邻行之间的差值。如果差值为负,则会被忽略。' -sidebar_position: 130 -slug: /sql-reference/aggregate-functions/reference/deltasumtimestamp -title: 'deltaSumTimestamp' -doc_type: 'reference' ---- - -累加相邻行之间的差值。如果差值为负,则会被忽略。 - -此函数主要用于按某个对齐到时间桶的时间戳排序并存储数据的[物化视图](/sql-reference/statements/create/view#materialized-view),例如 `toStartOfMinute` 时间桶。由于这样的物化视图中的行具有相同的时间戳,如果不存储原始、未取整的时间戳值,就无法以正确顺序合并这些行。`deltaSumTimestamp` 函数会跟踪它已经处理过的值对应的原始 `timestamp`,因此在数据分片合并期间,函数的值(状态)可以被正确计算。 - -要对有序集合计算增量和,你可以直接使用 [deltaSum](/sql-reference/aggregate-functions/reference/deltasum) 函数。 - -**语法** - -```sql -deltaSumTimestamp(value, timestamp) -``` - -**参数** - -* `value` — 输入值,必须是某个 [Integer](../../data-types/int-uint.md) 类型或 [Float](../../data-types/float.md) 类型,或者 [Date](../../data-types/date.md) 或 [DateTime](../../data-types/datetime.md)。 -* `timestamp` — 用于对值进行排序的参数,必须是某个 [Integer](../../data-types/int-uint.md) 类型或 [Float](../../data-types/float.md) 类型,或者 [Date](../../data-types/date.md) 或 [DateTime](../../data-types/datetime.md)。 - -**返回值** - -* 按 `timestamp` 参数排序后,相邻值之间差值的累计和。 - -类型:[Integer](../../data-types/int-uint.md) 或 [Float](../../data-types/float.md) 或 [Date](../../data-types/date.md) 或 [DateTime](../../data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT deltaSumTimestamp(value, timestamp) -FROM (SELECT number AS timestamp, [0, 4, 8, 3, 0, 0, 0, 1, 3, 5][number] AS value FROM numbers(1, 10)); -``` - -结果: - -```text -┌─deltaSumTimestamp(value, timestamp)─┐ -│ 13 │ -└─────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md deleted file mode 100644 index 8f96f1b667f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctdynamictypes.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '计算 Dynamic 列中存储的不同数据类型列表。' -sidebar_position: 215 -slug: /sql-reference/aggregate-functions/reference/distinctdynamictypes -title: 'distinctDynamicTypes' -doc_type: 'reference' ---- - -# distinctDynamicTypes {#distinctdynamictypes} - -计算存储在 [Dynamic](../../data-types/dynamic.md) 列中的去重数据类型列表。 - -**语法** - -```sql -distinctDynamicTypes(dynamic) -``` - -**参数** - -* `dynamic` — [Dynamic](../../data-types/dynamic.md) 列。 - -**返回值** - -* 排序后的数据类型名称列表 [Array(String)](../../data-types/array.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_dynamic; -CREATE TABLE test_dynamic(d Dynamic) ENGINE = Memory; -INSERT INTO test_dynamic VALUES (42), (NULL), ('Hello'), ([1, 2, 3]), ('2020-01-01'), (map(1, 2)), (43), ([4, 5]), (NULL), ('World'), (map(3, 4)) -``` - -```sql -SELECT distinctDynamicTypes(d) FROM test_dynamic; -``` - -结果: - -```reference -┌─distinctDynamicTypes(d)──────────────────────────────────────┐ -│ ['Array(Int64)','Date','Int64','Map(UInt8, UInt8)','String'] │ -└──────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md deleted file mode 100644 index 24659931454..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/distinctjsonpaths.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -description: '计算 JSON 列中所有不同路径的列表。' -sidebar_position: 216 -slug: /sql-reference/aggregate-functions/reference/distinctjsonpaths -title: 'distinctJSONPaths' -doc_type: 'reference' ---- - -# distinctJSONPaths {#distinctjsonpaths} - -计算存储在 [JSON](../../data-types/newjson.md) 列中的唯一路径列表。 - -**语法** - -```sql -distinctJSONPaths(json) -``` - -**参数** - -* `json` — [JSON](../../data-types/newjson.md) 列。 - -**返回值** - -* 排序后的路径列表 [Array(String)](../../data-types/array.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "Hello"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -结果: - -```reference -┌─distinctJSONPaths(json)───┐ -│ ['a','b','c.d.e','c.d.f'] │ -└───────────────────────────┘ -``` - -# distinctJSONPathsAndTypes {#distinctjsonpathsandtypes} - -计算存储在 [JSON](../../data-types/newjson.md) 列中的所有不同路径及其对应类型的列表。 - -**语法** - -```sql -distinctJSONPathsAndTypes(json) -``` - -**参数** - -* `json` — [JSON](../../data-types/newjson.md) 列。 - -**返回值** - -* 排序后的路径和类型映射 [Map(String, Array(String))](../../data-types/map.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"a" : 42, "b" : "Hello"}'), ('{"b" : [1, 2, 3], "c" : {"d" : {"e" : "2020-01-01"}}}'), ('{"a" : 43, "c" : {"d" : {"f" : [{"g" : 42}]}}}') -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -结果: - -```reference -┌─distinctJSONPathsAndTypes(json)───────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {'a':['Int64'],'b':['Array(Nullable(Int64))','String'],'c.d.e':['Date'],'c.d.f':['Array(JSON(max_dynamic_types=16, max_dynamic_paths=256))']} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -**注意** - -如果 JSON 声明中包含带有指定类型的路径,即使输入数据中没有这些路径的值,这些路径也会始终出现在 `distinctJSONPaths/distinctJSONPathsAndTypes` 函数的结果中。 - -```sql -DROP TABLE IF EXISTS test_json; -CREATE TABLE test_json(json JSON(a UInt32)) ENGINE = Memory; -INSERT INTO test_json VALUES ('{"b" : "Hello"}'), ('{"b" : "World", "c" : [1, 2, 3]}'); -``` - -```sql -SELECT json FROM test_json; -``` - -```text -┌─json──────────────────────────────────┐ -│ {"a":0,"b":"Hello"} │ -│ {"a":0,"b":"World","c":["1","2","3"]} │ -└───────────────────────────────────────┘ -``` - -```sql -SELECT distinctJSONPaths(json) FROM test_json; -``` - -```text -┌─distinctJSONPaths(json)─┐ -│ ['a','b','c'] │ -└─────────────────────────┘ -``` - -```sql -SELECT distinctJSONPathsAndTypes(json) FROM test_json; -``` - -```text -┌─distinctJSONPathsAndTypes(json)────────────────────────────────┐ -│ {'a':['UInt32'],'b':['String'],'c':['Array(Nullable(Int64))']} │ -└────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md deleted file mode 100644 index cdf3adb2861..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/entropy.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '计算某列值的香农熵。' -sidebar_position: 131 -slug: /sql-reference/aggregate-functions/reference/entropy -title: 'entropy' -doc_type: 'reference' ---- - -# entropy {#entropy} - -计算某列数值的[香农熵](https://en.wikipedia.org/wiki/Entropy_\(information_theory\))。 - -**语法** - -```sql -entropy(val) -``` - -**参数** - -* `val` — 任意类型的值列。 - -**返回值** - -* 香农熵。 - -类型:[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE entropy (`vals` UInt32,`strings` String) ENGINE = Memory; - -INSERT INTO entropy VALUES (1, 'A'), (1, 'A'), (1,'A'), (1,'A'), (2,'B'), (2,'B'), (2,'C'), (2,'D'); - -SELECT entropy(vals), entropy(strings) FROM entropy; -``` - -结果: - -```text -┌─entropy(vals)─┬─entropy(strings)─┐ -│ 1 │ 1.75 │ -└───────────────┴──────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md deleted file mode 100644 index a5f61c6f5ff..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/estimateCompressionRatio.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: '在不对列进行实际压缩的情况下,估算给定列的压缩比。' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/estimateCompressionRatio -title: 'estimateCompressionRatio' -doc_type: 'reference' ---- - -## estimateCompressionRatio {#estimatecompressionration} - -在不实际压缩的情况下估算给定列的压缩比。 - -**语法** - -```sql -estimateCompressionRatio(codec, block_size_bytes)(column) -``` - -**参数** - -* `column` - 任意类型的列 - -**参数说明** - -* `codec` - 一个[String](../../../sql-reference/data-types/string.md),其内容为一个[压缩编解码器](/sql-reference/statements/create/table#column_compression_codec)或多个以逗号分隔的编解码器。 -* `block_size_bytes` - 压缩数据的块大小。类似于同时设置[`max_compress_block_size`](../../../operations/settings/merge-tree-settings.md#max_compress_block_size)和[`min_compress_block_size`](../../../operations/settings/merge-tree-settings.md#min_compress_block_size)。默认值为 1 MiB(1048576 字节)。 - -这两个参数都是可选的。 - -**返回值** - -* 返回给定列的压缩比估计值。 - -类型:[Float64](/sql-reference/data-types/float)。 - -**示例** - -```sql title="Input table" -CREATE TABLE compression_estimate_example -( - `number` UInt64 -) -ENGINE = MergeTree() -ORDER BY number -SETTINGS min_bytes_for_wide_part = 0; - -INSERT INTO compression_estimate_example -SELECT number FROM system.numbers LIMIT 100_000; -``` - -```sql title="Query" -SELECT estimateCompressionRatio(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌───────────estimate─┐ -│ 1.9988506608699999 │ -└────────────────────┘ -``` - -:::note -上面的结果会因服务器的默认压缩编解码器而有所不同。参见[列压缩编解码器](/sql-reference/statements/create/table#column_compression_codec)。 -::: - -```sql title="Query" -SELECT estimateCompressionRatio('T64')(number) AS estimate FROM compression_estimate_example; -``` - -```text title="Response" -┌──────────estimate─┐ -│ 3.762758101688538 │ -└───────────────────┘ -``` - -此函数还可以指定多个编解码器: - -```sql title="Query" -SELECT estimateCompressionRatio('T64, ZSTD')(number) AS estimate FROM compression_estimate_example; -``` - -```response title="Response" -┌───────────estimate─┐ -│ 143.60078980434392 │ -└────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md deleted file mode 100644 index 6ef92f2ea26..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialmovingaverage.md +++ /dev/null @@ -1,203 +0,0 @@ ---- -description: '计算在指定时间段内各数值的指数移动平均。' -sidebar_position: 132 -slug: /sql-reference/aggregate-functions/reference/exponentialMovingAverage -title: 'exponentialMovingAverage' -doc_type: 'reference' ---- - -## exponentialMovingAverage {#exponentialmovingaverage} - -计算在指定时间段内数值的指数移动平均值。 - -**语法** - -```sql -exponentialMovingAverage(x)(value, timeunit) -``` - -每个 `value` 对应一个给定的 `timeunit`。半衰期 `x` 是指数权重衰减到一半时对应的时间滞后。该函数返回加权平均值:时间点越早,其对应的 `value` 所占权重就越小。 - -**参数** - -* `value` — 数值。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `timeunit` — 时间单位。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。`timeunit` 不是时间戳(秒),而是时间区间的索引。可以使用 [intDiv](/sql-reference/functions/arithmetic-functions#intDiv) 计算。 - -**函数参数** - -* `x` — 半衰期。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 在最新时间点上,返回过去 `x` 时间范围内的[指数平滑移动平均值](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average)。 - -类型:[Float64](/sql-reference/data-types/float)。 - -**示例** - -输入表: - -```text -┌──温度────────┬─时间戳─────┐ -│ 95 │ 1 │ -│ 95 │ 2 │ -│ 95 │ 3 │ -│ 96 │ 4 │ -│ 96 │ 5 │ -│ 96 │ 6 │ -│ 96 │ 7 │ -│ 97 │ 8 │ -│ 97 │ 9 │ -│ 97 │ 10 │ -│ 97 │ 11 │ -│ 98 │ 12 │ -│ 98 │ 13 │ -│ 98 │ 14 │ -│ 98 │ 15 │ -│ 99 │ 16 │ -│ 99 │ 17 │ -│ 99 │ 18 │ -│ 100 │ 19 │ -│ 100 │ 20 │ -└──────────────┴────────────┘ -``` - -查询: - -```sql -SELECT exponentialMovingAverage(5)(temperature, timestamp); -``` - -结果: - -```text -┌──exponentialMovingAverage(5)(temperature, timestamp)──┐ -│ 92.25779635374204 │ -└───────────────────────────────────────────────────────┘ -``` - -查询: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 1, 50) AS bar -FROM -( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialMovingAverage(10)(value, time) OVER (Rows BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -) -``` - -结果: - -```text -┌─值────┬─时间─┬─round(exp_smooth, 3)─┬─条形图─────────────────────────────────────┐ -│ 1 │ 0 │ 0.067 │ ███▎ │ -│ 0 │ 1 │ 0.062 │ ███ │ -│ 0 │ 2 │ 0.058 │ ██▊ │ -│ 0 │ 3 │ 0.054 │ ██▋ │ -│ 0 │ 4 │ 0.051 │ ██▌ │ -│ 0 │ 5 │ 0.047 │ ██▎ │ -│ 0 │ 6 │ 0.044 │ ██▏ │ -│ 0 │ 7 │ 0.041 │ ██ │ -│ 0 │ 8 │ 0.038 │ █▊ │ -│ 0 │ 9 │ 0.036 │ █▋ │ -│ 0 │ 10 │ 0.033 │ █▋ │ -│ 0 │ 11 │ 0.031 │ █▌ │ -│ 0 │ 12 │ 0.029 │ █▍ │ -│ 0 │ 13 │ 0.027 │ █▎ │ -│ 0 │ 14 │ 0.025 │ █▎ │ -│ 0 │ 15 │ 0.024 │ █▏ │ -│ 0 │ 16 │ 0.022 │ █ │ -│ 0 │ 17 │ 0.021 │ █ │ -│ 0 │ 18 │ 0.019 │ ▊ │ -│ 0 │ 19 │ 0.018 │ ▊ │ -│ 0 │ 20 │ 0.017 │ ▋ │ -│ 0 │ 21 │ 0.016 │ ▋ │ -│ 0 │ 22 │ 0.015 │ ▋ │ -│ 0 │ 23 │ 0.014 │ ▋ │ -│ 0 │ 24 │ 0.013 │ ▋ │ -│ 1 │ 25 │ 0.079 │ ███▊ │ -│ 1 │ 26 │ 0.14 │ ███████ │ -│ 1 │ 27 │ 0.198 │ █████████▊ │ -│ 1 │ 28 │ 0.252 │ ████████████▌ │ -│ 1 │ 29 │ 0.302 │ ███████████████ │ -│ 1 │ 30 │ 0.349 │ █████████████████▍ │ -│ 1 │ 31 │ 0.392 │ ███████████████████▌ │ -│ 1 │ 32 │ 0.433 │ █████████████████████▋ │ -│ 1 │ 33 │ 0.471 │ ███████████████████████▌ │ -│ 1 │ 34 │ 0.506 │ █████████████████████████▎ │ -│ 1 │ 35 │ 0.539 │ ██████████████████████████▊ │ -│ 1 │ 36 │ 0.57 │ ████████████████████████████▌ │ -│ 1 │ 37 │ 0.599 │ █████████████████████████████▊ │ -│ 1 │ 38 │ 0.626 │ ███████████████████████████████▎ │ -│ 1 │ 39 │ 0.651 │ ████████████████████████████████▌ │ -│ 1 │ 40 │ 0.674 │ █████████████████████████████████▋ │ -│ 1 │ 41 │ 0.696 │ ██████████████████████████████████▋ │ -│ 1 │ 42 │ 0.716 │ ███████████████████████████████████▋ │ -│ 1 │ 43 │ 0.735 │ ████████████████████████████████████▋ │ -│ 1 │ 44 │ 0.753 │ █████████████████████████████████████▋ │ -│ 1 │ 45 │ 0.77 │ ██████████████████████████████████████▍ │ -│ 1 │ 46 │ 0.785 │ ███████████████████████████████████████▎ │ -│ 1 │ 47 │ 0.8 │ ███████████████████████████████████████▊ │ -│ 1 │ 48 │ 0.813 │ ████████████████████████████████████████▋ │ -│ 1 │ 49 │ 0.825 │ █████████████████████████████████████████▎ │ -└───────┴──────┴──────────────────────┴────────────────────────────────────────────┘ -``` - -```sql -CREATE TABLE data -ENGINE = Memory AS -SELECT - 10 AS value, - toDateTime('2020-01-01') + (3600 * number) AS time -FROM numbers_mt(10); --- 使用 intDiv 计算时间单元 -SELECT - value, - time, - exponentialMovingAverage(1)(value, intDiv(toUInt32(time), 3600)) OVER (ORDER BY time ASC) AS res, - intDiv(toUInt32(time), 3600) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ --- 使用 toRelativeHourNum 计算时间单元 -SELECT - value, - time, - exponentialMovingAverage(1)(value, toRelativeHourNum(time)) OVER (ORDER BY time ASC) AS res, - toRelativeHourNum(time) AS timeunit -FROM data -ORDER BY time ASC; - -┌─value─┬────────────────time─┬─────────res─┬─timeunit─┐ -│ 10 │ 2020-01-01 00:00:00 │ 5 │ 438288 │ -│ 10 │ 2020-01-01 01:00:00 │ 7.5 │ 438289 │ -│ 10 │ 2020-01-01 02:00:00 │ 8.75 │ 438290 │ -│ 10 │ 2020-01-01 03:00:00 │ 9.375 │ 438291 │ -│ 10 │ 2020-01-01 04:00:00 │ 9.6875 │ 438292 │ -│ 10 │ 2020-01-01 05:00:00 │ 9.84375 │ 438293 │ -│ 10 │ 2020-01-01 06:00:00 │ 9.921875 │ 438294 │ -│ 10 │ 2020-01-01 07:00:00 │ 9.9609375 │ 438295 │ -│ 10 │ 2020-01-01 08:00:00 │ 9.98046875 │ 438296 │ -│ 10 │ 2020-01-01 09:00:00 │ 9.990234375 │ 438297 │ -└───────┴─────────────────────┴─────────────┴──────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md deleted file mode 100644 index 4ac3fc81d1b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedavg.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '返回时间序列在时间点 `t` 的值的指数平滑加权移动平均。' -sidebar_position: 133 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedAvg -title: 'exponentialTimeDecayedAvg' -doc_type: 'reference' ---- - -## exponentialTimeDecayedAvg {#exponentialtimedecayedavg} - -返回时间序列在时间点 `t` 处的指数加权平滑移动平均值。 - -**语法** - -```sql -exponentialTimeDecayedAvg(x)(v, t) -``` - -**参数** - -* `v` — 数值。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `t` — 时间。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**参数说明** - -* `x` — 半衰期周期。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 返回时间点 `t` 处的指数平滑加权移动平均值。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedAvg(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -响应: - -```sql -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ -1. │ 1 │ 0 │ 1 │ ██████████ │ -2. │ 0 │ 1 │ 0.475 │ ████▊ │ -3. │ 0 │ 2 │ 0.301 │ ███ │ -4. │ 0 │ 3 │ 0.214 │ ██▏ │ -5. │ 0 │ 4 │ 0.162 │ █▌ │ -6. │ 0 │ 5 │ 0.128 │ █▎ │ -7. │ 0 │ 6 │ 0.104 │ █ │ -8. │ 0 │ 7 │ 0.086 │ ▊ │ -9. │ 0 │ 8 │ 0.072 │ ▋ │ -0. │ 0 │ 9 │ 0.061 │ ▌ │ -1. │ 0 │ 10 │ 0.052 │ ▌ │ -2. │ 0 │ 11 │ 0.045 │ ▍ │ -3. │ 0 │ 12 │ 0.039 │ ▍ │ -4. │ 0 │ 13 │ 0.034 │ ▎ │ -5. │ 0 │ 14 │ 0.03 │ ▎ │ -6. │ 0 │ 15 │ 0.027 │ ▎ │ -7. │ 0 │ 16 │ 0.024 │ ▏ │ -8. │ 0 │ 17 │ 0.021 │ ▏ │ -9. │ 0 │ 18 │ 0.018 │ ▏ │ -0. │ 0 │ 19 │ 0.016 │ ▏ │ -1. │ 0 │ 20 │ 0.015 │ ▏ │ -2. │ 0 │ 21 │ 0.013 │ ▏ │ -3. │ 0 │ 22 │ 0.012 │ │ -4. │ 0 │ 23 │ 0.01 │ │ -5. │ 0 │ 24 │ 0.009 │ │ -6. │ 1 │ 25 │ 0.111 │ █ │ -7. │ 1 │ 26 │ 0.202 │ ██ │ -8. │ 1 │ 27 │ 0.283 │ ██▊ │ -9. │ 1 │ 28 │ 0.355 │ ███▌ │ -0. │ 1 │ 29 │ 0.42 │ ████▏ │ -1. │ 1 │ 30 │ 0.477 │ ████▊ │ -2. │ 1 │ 31 │ 0.529 │ █████▎ │ -3. │ 1 │ 32 │ 0.576 │ █████▊ │ -4. │ 1 │ 33 │ 0.618 │ ██████▏ │ -5. │ 1 │ 34 │ 0.655 │ ██████▌ │ -6. │ 1 │ 35 │ 0.689 │ ██████▉ │ -7. │ 1 │ 36 │ 0.719 │ ███████▏ │ -8. │ 1 │ 37 │ 0.747 │ ███████▍ │ -9. │ 1 │ 38 │ 0.771 │ ███████▋ │ -0. │ 1 │ 39 │ 0.793 │ ███████▉ │ -1. │ 1 │ 40 │ 0.813 │ ████████▏ │ -2. │ 1 │ 41 │ 0.831 │ ████████▎ │ -3. │ 1 │ 42 │ 0.848 │ ████████▍ │ -4. │ 1 │ 43 │ 0.862 │ ████████▌ │ -5. │ 1 │ 44 │ 0.876 │ ████████▊ │ -6. │ 1 │ 45 │ 0.888 │ ████████▉ │ -7. │ 1 │ 46 │ 0.898 │ ████████▉ │ -8. │ 1 │ 47 │ 0.908 │ █████████ │ -9. │ 1 │ 48 │ 0.917 │ █████████▏ │ -0. │ 1 │ 49 │ 0.925 │ █████████▏ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md deleted file mode 100644 index 3475a26e386..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedcount.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: '返回时间序列在时间索引为 `t` 时的累积指数衰减值。' -sidebar_position: 134 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedCount -title: 'exponentialTimeDecayedCount' -doc_type: 'reference' ---- - -## exponentialTimeDecayedCount {#exponentialtimedecayedcount} - -返回时间序列在时间点 `t` 处的累积指数衰减值。 - -**语法** - -```sql -指数时间衰减计数(x)(t) -``` - -**参数** - -* `t` — 时间。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md),[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**附加参数** - -* `x` — 半衰期。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 返回给定时间点的累积指数衰减值。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 20, 50) AS bar -FROM -( - SELECT - (number % 5) = 0 AS value, - number AS time, - exponentialTimeDecayedCount(10)(time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) -); -``` - -结果: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ ██▌ │ - 2. │ 0 │ 1 │ 1.905 │ ████▊ │ - 3. │ 0 │ 2 │ 2.724 │ ██████▊ │ - 4. │ 0 │ 3 │ 3.464 │ ████████▋ │ - 5. │ 0 │ 4 │ 4.135 │ ██████████▎ │ - 6. │ 1 │ 5 │ 4.741 │ ███████████▊ │ - 7. │ 0 │ 6 │ 5.29 │ █████████████▏ │ - 8. │ 0 │ 7 │ 5.787 │ ██████████████▍ │ - 9. │ 0 │ 8 │ 6.236 │ ███████████████▌ │ -10. │ 0 │ 9 │ 6.643 │ ████████████████▌ │ -11. │ 1 │ 10 │ 7.01 │ █████████████████▌ │ -12. │ 0 │ 11 │ 7.343 │ ██████████████████▎ │ -13. │ 0 │ 12 │ 7.644 │ ███████████████████ │ -14. │ 0 │ 13 │ 7.917 │ ███████████████████▊ │ -15. │ 0 │ 14 │ 8.164 │ ████████████████████▍ │ -16. │ 1 │ 15 │ 8.387 │ ████████████████████▉ │ -17. │ 0 │ 16 │ 8.589 │ █████████████████████▍ │ -18. │ 0 │ 17 │ 8.771 │ █████████████████████▉ │ -19. │ 0 │ 18 │ 8.937 │ ██████████████████████▎ │ -20. │ 0 │ 19 │ 9.086 │ ██████████████████████▋ │ -21. │ 1 │ 20 │ 9.222 │ ███████████████████████ │ -22. │ 0 │ 21 │ 9.344 │ ███████████████████████▎ │ -23. │ 0 │ 22 │ 9.455 │ ███████████████████████▋ │ -24. │ 0 │ 23 │ 9.555 │ ███████████████████████▉ │ -25. │ 0 │ 24 │ 9.646 │ ████████████████████████ │ -26. │ 1 │ 25 │ 9.728 │ ████████████████████████▎ │ -27. │ 0 │ 26 │ 9.802 │ ████████████████████████▌ │ -28. │ 0 │ 27 │ 9.869 │ ████████████████████████▋ │ -29. │ 0 │ 28 │ 9.93 │ ████████████████████████▊ │ -30. │ 0 │ 29 │ 9.985 │ ████████████████████████▉ │ -31. │ 1 │ 30 │ 10.035 │ █████████████████████████ │ -32. │ 0 │ 31 │ 10.08 │ █████████████████████████▏ │ -33. │ 0 │ 32 │ 10.121 │ █████████████████████████▎ │ -34. │ 0 │ 33 │ 10.158 │ █████████████████████████▍ │ -35. │ 0 │ 34 │ 10.191 │ █████████████████████████▍ │ -36. │ 1 │ 35 │ 10.221 │ █████████████████████████▌ │ -37. │ 0 │ 36 │ 10.249 │ █████████████████████████▌ │ -38. │ 0 │ 37 │ 10.273 │ █████████████████████████▋ │ -39. │ 0 │ 38 │ 10.296 │ █████████████████████████▋ │ -40. │ 0 │ 39 │ 10.316 │ █████████████████████████▊ │ -41. │ 1 │ 40 │ 10.334 │ █████████████████████████▊ │ -42. │ 0 │ 41 │ 10.351 │ █████████████████████████▉ │ -43. │ 0 │ 42 │ 10.366 │ █████████████████████████▉ │ -44. │ 0 │ 43 │ 10.379 │ █████████████████████████▉ │ -45. │ 0 │ 44 │ 10.392 │ █████████████████████████▉ │ -46. │ 1 │ 45 │ 10.403 │ ██████████████████████████ │ -47. │ 0 │ 46 │ 10.413 │ ██████████████████████████ │ -48. │ 0 │ 47 │ 10.422 │ ██████████████████████████ │ -49. │ 0 │ 48 │ 10.43 │ ██████████████████████████ │ -50. │ 0 │ 49 │ 10.438 │ ██████████████████████████ │ - └───────┴──────┴──────────────────────┴────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md deleted file mode 100644 index f06fd49cc9c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedmax.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '返回时间索引 `t` 处与 `t-1` 处计算得到的指数平滑移动平均值中的最大值。' -sidebar_position: 135 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedMax -title: 'exponentialTimeDecayedMax' -doc_type: 'reference' ---- - -## exponentialTimeDecayedMax {#exponentialtimedecayedmax} - -返回在时间索引为 `t` 处与时间索引为 `t-1` 处计算得到的指数平滑移动平均值中的最大值。 - -**语法** - -```sql -exponentialTimeDecayedMax(x)(value, timeunit) -``` - -**参数** - -* `value` — 数值。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `timeunit` — 时间单位。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**参数说明** - -* `x` — 半衰期。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 返回在时刻 `t` 和 `t-1` 的指数平滑加权移动平均值中的最大值。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 5, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedMax(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -结果: - -```response -┌─value─┬─time─┬─round(exp_smooth, 3)─┬─bar────────┐ - 1. │ 1 │ 0 │ 1 │ ██████████ │ - 2. │ 0 │ 1 │ 0.905 │ █████████ │ - 3. │ 0 │ 2 │ 0.819 │ ████████▏ │ - 4. │ 0 │ 3 │ 0.741 │ ███████▍ │ - 5. │ 0 │ 4 │ 0.67 │ ██████▋ │ - 6. │ 0 │ 5 │ 0.607 │ ██████ │ - 7. │ 0 │ 6 │ 0.549 │ █████▍ │ - 8. │ 0 │ 7 │ 0.497 │ ████▉ │ - 9. │ 0 │ 8 │ 0.449 │ ████▍ │ -10. │ 0 │ 9 │ 0.407 │ ████ │ -11. │ 0 │ 10 │ 0.368 │ ███▋ │ -12. │ 0 │ 11 │ 0.333 │ ███▎ │ -13. │ 0 │ 12 │ 0.301 │ ███ │ -14. │ 0 │ 13 │ 0.273 │ ██▋ │ -15. │ 0 │ 14 │ 0.247 │ ██▍ │ -16. │ 0 │ 15 │ 0.223 │ ██▏ │ -17. │ 0 │ 16 │ 0.202 │ ██ │ -18. │ 0 │ 17 │ 0.183 │ █▊ │ -19. │ 0 │ 18 │ 0.165 │ █▋ │ -20. │ 0 │ 19 │ 0.15 │ █▍ │ -21. │ 0 │ 20 │ 0.135 │ █▎ │ -22. │ 0 │ 21 │ 0.122 │ █▏ │ -23. │ 0 │ 22 │ 0.111 │ █ │ -24. │ 0 │ 23 │ 0.1 │ █ │ -25. │ 0 │ 24 │ 0.091 │ ▉ │ -26. │ 1 │ 25 │ 1 │ ██████████ │ -27. │ 1 │ 26 │ 1 │ ██████████ │ -28. │ 1 │ 27 │ 1 │ ██████████ │ -29. │ 1 │ 28 │ 1 │ ██████████ │ -30. │ 1 │ 29 │ 1 │ ██████████ │ -31. │ 1 │ 30 │ 1 │ ██████████ │ -32. │ 1 │ 31 │ 1 │ ██████████ │ -33. │ 1 │ 32 │ 1 │ ██████████ │ -34. │ 1 │ 33 │ 1 │ ██████████ │ -35. │ 1 │ 34 │ 1 │ ██████████ │ -36. │ 1 │ 35 │ 1 │ ██████████ │ -37. │ 1 │ 36 │ 1 │ ██████████ │ -38. │ 1 │ 37 │ 1 │ ██████████ │ -39. │ 1 │ 38 │ 1 │ ██████████ │ -40. │ 1 │ 39 │ 1 │ ██████████ │ -41. │ 1 │ 40 │ 1 │ ██████████ │ -42. │ 1 │ 41 │ 1 │ ██████████ │ -43. │ 1 │ 42 │ 1 │ ██████████ │ -44. │ 1 │ 43 │ 1 │ ██████████ │ -45. │ 1 │ 44 │ 1 │ ██████████ │ -46. │ 1 │ 45 │ 1 │ ██████████ │ -47. │ 1 │ 46 │ 1 │ ██████████ │ -48. │ 1 │ 47 │ 1 │ ██████████ │ -49. │ 1 │ 48 │ 1 │ ██████████ │ -50. │ 1 │ 49 │ 1 │ ██████████ │ - └───────┴──────┴──────────────────────┴────────────┘ -``` \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md deleted file mode 100644 index 6e30b72514d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/exponentialtimedecayedsum.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '返回时间序列在时间索引 `t` 处的指数平滑移动平均值之和。' -sidebar_position: 136 -slug: /sql-reference/aggregate-functions/reference/exponentialTimeDecayedSum -title: 'exponentialTimeDecayedSum' -doc_type: 'reference' ---- - -## exponentialTimeDecayedSum {#exponentialtimedecayedsum} - -返回时间序列在时间索引 `t` 处的指数平滑移动平均值的总和。 - -**语法** - -```sql -指数时间衰减和(x)(v, t) -``` - -**参数** - -* `v` — 数值。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `t` — 时间。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)、[DateTime](../../data-types/datetime.md)、[DateTime64](../../data-types/datetime64.md)。 - -**参数说明** - -* `x` — 使某个数值的权重衰减到 1/e 所需的时间差。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 返回在给定时间点的指数平滑移动平均值之和。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT - value, - time, - round(exp_smooth, 3), - bar(exp_smooth, 0, 10, 50) AS bar -FROM - ( - SELECT - (number = 0) OR (number >= 25) AS value, - number AS time, - exponentialTimeDecayedSum(10)(value, time) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS exp_smooth - FROM numbers(50) - ); -``` - -结果: - -```response -┌─值─┬─时间─┬─round(exp_smooth, 3)─┬─条形───────────────────────────────────────────────┐ - 1. │ 1 │ 0 │ 1 │ █████ │ - 2. │ 0 │ 1 │ 0.905 │ ████▌ │ - 3. │ 0 │ 2 │ 0.819 │ ████ │ - 4. │ 0 │ 3 │ 0.741 │ ███▋ │ - 5. │ 0 │ 4 │ 0.67 │ ███▎ │ - 6. │ 0 │ 5 │ 0.607 │ ███ │ - 7. │ 0 │ 6 │ 0.549 │ ██▋ │ - 8. │ 0 │ 7 │ 0.497 │ ██▍ │ - 9. │ 0 │ 8 │ 0.449 │ ██▏ │ -10. │ 0 │ 9 │ 0.407 │ ██ │ -11. │ 0 │ 10 │ 0.368 │ █▊ │ -12. │ 0 │ 11 │ 0.333 │ █▋ │ -13. │ 0 │ 12 │ 0.301 │ █▌ │ -14. │ 0 │ 13 │ 0.273 │ █▎ │ -15. │ 0 │ 14 │ 0.247 │ █▏ │ -16. │ 0 │ 15 │ 0.223 │ █ │ -17. │ 0 │ 16 │ 0.202 │ █ │ -18. │ 0 │ 17 │ 0.183 │ ▉ │ -19. │ 0 │ 18 │ 0.165 │ ▊ │ -20. │ 0 │ 19 │ 0.15 │ ▋ │ -21. │ 0 │ 20 │ 0.135 │ ▋ │ -22. │ 0 │ 21 │ 0.122 │ ▌ │ -23. │ 0 │ 22 │ 0.111 │ ▌ │ -24. │ 0 │ 23 │ 0.1 │ ▌ │ -25. │ 0 │ 24 │ 0.091 │ ▍ │ -26. │ 1 │ 25 │ 1.082 │ █████▍ │ -27. │ 1 │ 26 │ 1.979 │ █████████▉ │ -28. │ 1 │ 27 │ 2.791 │ █████████████▉ │ -29. │ 1 │ 28 │ 3.525 │ █████████████████▋ │ -30. │ 1 │ 29 │ 4.19 │ ████████████████████▉ │ -31. │ 1 │ 30 │ 4.791 │ ███████████████████████▉ │ -32. │ 1 │ 31 │ 5.335 │ ██████████████████████████▋ │ -33. │ 1 │ 32 │ 5.827 │ █████████████████████████████▏ │ -34. │ 1 │ 33 │ 6.273 │ ███████████████████████████████▎ │ -35. │ 1 │ 34 │ 6.676 │ █████████████████████████████████▍ │ -36. │ 1 │ 35 │ 7.041 │ ███████████████████████████████████▏ │ -37. │ 1 │ 36 │ 7.371 │ ████████████████████████████████████▊ │ -38. │ 1 │ 37 │ 7.669 │ ██████████████████████████████████████▎ │ -39. │ 1 │ 38 │ 7.939 │ ███████████████████████████████████████▋ │ -40. │ 1 │ 39 │ 8.184 │ ████████████████████████████████████████▉ │ -41. │ 1 │ 40 │ 8.405 │ ██████████████████████████████████████████ │ -42. │ 1 │ 41 │ 8.605 │ ███████████████████████████████████████████ │ -43. │ 1 │ 42 │ 8.786 │ ███████████████████████████████████████████▉ │ -44. │ 1 │ 43 │ 8.95 │ ████████████████████████████████████████████▊ │ -45. │ 1 │ 44 │ 9.098 │ █████████████████████████████████████████████▍ │ -46. │ 1 │ 45 │ 9.233 │ ██████████████████████████████████████████████▏ │ -47. │ 1 │ 46 │ 9.354 │ ██████████████████████████████████████████████▊ │ -48. │ 1 │ 47 │ 9.464 │ ███████████████████████████████████████████████▎ │ -49. │ 1 │ 48 │ 9.563 │ ███████████████████████████████████████████████▊ │ -50. │ 1 │ 49 │ 9.653 │ ████████████████████████████████████████████████▎ │ - └───────┴──────┴──────────────────────┴───────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md deleted file mode 100644 index a27c5d4e7e5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/first_value.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: '它是 `any` 的别名,为了与窗口函数兼容而引入。在窗口函数中,有时需要处理 `NULL` 值(默认情况下,所有 ClickHouse 聚合函数都会忽略 `NULL` 值)。' -sidebar_position: 137 -slug: /sql-reference/aggregate-functions/reference/first_value -title: 'first_value' -doc_type: 'reference' ---- - -# first_value {#first_value} - -它是 [`any`](../../../sql-reference/aggregate-functions/reference/any.md) 的别名,但之所以引入它,是为了兼容 [窗口函数](../../window-functions/index.md),在这些场景下有时需要处理 `NULL` 值(默认情况下,所有 ClickHouse 聚合函数都会忽略 NULL 值)。 - -它支持声明一个用于保留 NULL 值的修饰符(`RESPECT NULLS`),既可以在 [窗口函数](../../window-functions/index.md) 中使用,也可以在普通聚合中使用。 - -与 `any` 一样,如果不在窗口函数中使用且源数据流未排序,则结果是随机的,并且返回类型与输入类型一致(只有当输入是 Nullable 或者添加了 -OrNull 组合器时,才会返回 Null)。 - -## 示例 {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null); -``` - -### 示例 1 {#example1} - -默认情况下,NULL 值会被忽略。 - -```sql -SELECT first_value(b) FROM test_data; -``` - -```text -┌─any(b)─┐ -│ 3 │ -└────────┘ -``` - -### 示例 2 {#example2} - -NULL 值会被忽略。 - -```sql -SELECT first_value(b) ignore nulls FROM test_data -``` - -```text -┌─any(b) IGNORE NULLS ─┐ -│ 3 │ -└──────────────────────┘ -``` - -### 示例 3 {#example3} - -允许 NULL 值。 - -```sql -SELECT first_value(b) respect nulls FROM test_data -``` - -```text -┌─any(b) RESPECT NULLS ─┐ -│ ᴺᵁᴸᴸ │ -└───────────────────────┘ -``` - -### 示例 4 {#example4} - -通过在子查询中使用 `ORDER BY` 获得稳定的结果。 - -```sql -SELECT - first_value_respect_nulls(b), - first_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─any_respect_nulls(b)─┬─any(b)─┐ -│ ᴺᵁᴸᴸ │ 3 │ -└──────────────────────┴────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md deleted file mode 100644 index faa77379c6f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/flame_graph.md +++ /dev/null @@ -1,107 +0,0 @@ ---- -description: '基于堆栈跟踪列表构建火焰图的聚合函数。' -sidebar_position: 138 -slug: /sql-reference/aggregate-functions/reference/flame_graph -title: 'flameGraph' -doc_type: 'reference' ---- - - - -# flameGraph {#flamegraph} - -一种聚合函数,使用堆栈跟踪列表构建[火焰图(flamegraph)](https://www.brendangregg.com/flamegraphs.html)。输出字符串数组,可供 [flamegraph.pl 工具](https://github.com/brendangregg/FlameGraph) 使用,以渲染火焰图的 SVG。 - - - -## 语法 {#syntax} - -```sql -flameGraph(traces, [size], [ptr]) -``` - - -## 参数 {#parameters} - -- `traces` — 堆栈跟踪。[Array](../../data-types/array.md)([UInt64](../../data-types/int-uint.md))。 -- `size` — 用于内存分析的分配大小(可选,默认值为 `1`)。[UInt64](../../data-types/int-uint.md)。 -- `ptr` — 内存分配地址(可选,默认值为 `0`)。[UInt64](../../data-types/int-uint.md)。 - -:::note -当 `ptr != 0` 时,flameGraph 会将具有相同 size 和 ptr 的分配(size > 0)与释放(size < 0)进行映射。 -仅显示尚未被释放的分配。未被映射的释放操作将被忽略。 -::: - - - -## 返回值 {#returned-value} - -- 供 [flamegraph.pl 工具](https://github.com/brendangregg/FlameGraph) 使用的字符串数组。[Array](../../data-types/array.md)([String](../../data-types/string.md))。 - - - -## 示例 {#examples} - -### 基于 CPU 查询剖析器构建火焰图 {#building-a-flamegraph-based-on-a-cpu-query-profiler} - -```sql -SET query_profiler_cpu_time_period_ns=10000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(arrayReverse(trace))) from system.trace_log where trace_type = 'CPU' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl > flame_cpu.svg -``` - -### 使用内存查询分析器构建火焰图,展示所有内存分配情况 {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-all-allocations} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "select arrayJoin(flameGraph(trace, size)) from system.trace_log where trace_type = 'MemorySample' and query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem.svg -``` - -### 基于内存查询分析器构建火焰图,显示在查询上下文中未被释放的内存分配 {#building-a-flamegraph-based-on-a-memory-query-profiler-showing-allocations-which-were-not-deallocated-in-query-context} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1, use_uncompressed_cache=1, merge_tree_max_rows_to_use_cache=100000000000, merge_tree_max_bytes_to_use_cache=1000000000000; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx'" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_untracked.svg -``` - -### 基于内存查询分析器构建火焰图,展示某一固定时间点的活动内存分配 {#build-a-flamegraph-based-on-memory-query-profiler-showing-active-allocations-at-the-fixed-point-of-time} - -```sql -SET memory_profiler_sample_probability=1, max_untracked_memory=1; -SELECT SearchPhrase, COUNT(DISTINCT UserID) AS u FROM hits WHERE SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY u DESC LIMIT 10; -``` - -* 1 - 每秒内存使用情况 - -```sql -SELECT event_time, m, formatReadableSize(max(s) AS m) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample') GROUP BY event_time ORDER BY event_time; -``` - -* 2 - 找到内存使用量峰值所在的时间点 - -```sql -SELECT argMax(event_time, s), max(s) FROM (SELECT event_time, sum(size) OVER (ORDER BY event_time) AS s FROM system.trace_log WHERE query_id = 'xxx' AND trace_type = 'MemorySample'); -``` - -* 3 - 在某个固定时间点确定活动分配情况 - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time <= 'yyy' ORDER BY event_time)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_pos.svg -``` - -* 4 - 在特定时间点查找释放操作 - -```text -clickhouse client --allow_introspection_functions=1 -q "SELECT arrayJoin(flameGraph(trace, -size, ptr)) FROM (SELECT * FROM system.trace_log WHERE trace_type = 'MemorySample' AND query_id = 'xxx' AND event_time > 'yyy' ORDER BY event_time desc)" | ~/dev/FlameGraph/flamegraph.pl --countname=bytes --color=mem > flame_mem_time_point_neg.svg -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md deleted file mode 100644 index d964822627d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparray.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '创建一个由参数值组成的数组。可以以任意(未定义的)顺序将值添加到数组中。' -sidebar_position: 139 -slug: /sql-reference/aggregate-functions/reference/grouparray -title: 'groupArray' -doc_type: 'reference' ---- - -# groupArray {#grouparray} - -语法:`groupArray(x)` 或 `groupArray(max_size)(x)` - -创建一个由参数值构成的数组。 -可以以任意(非确定)顺序将值添加到该数组中。 - -第二种形式(带有 `max_size` 参数)会将结果数组的大小限制为最多 `max_size` 个元素。例如,`groupArray(1)(x)` 等价于 `[any(x)]`。 - -在某些情况下,仍然可以依赖执行顺序。这适用于 `SELECT` 语句的数据来自一个使用了 `ORDER BY` 的子查询且该子查询结果集足够小时的情形。 - -**示例** - -```text -SELECT * FROM default.ck; - -┌─id─┬─name─────┐ -│ 1 │ zhangsan │ -│ 1 │ ᴺᵁᴸᴸ │ -│ 1 │ lisi │ -│ 2 │ wangwu │ -└────┴──────────┘ - -``` - -查询: - -```sql -SELECT id, groupArray(10)(name) FROM default.ck GROUP BY id; -``` - -结果: - -```text -┌─id─┬─groupArray(10)(name)─┐ -│ 1 │ ['zhangsan','lisi'] │ -│ 2 │ ['wangwu'] │ -└────┴──────────────────────┘ -``` - -`groupArray` 函数会根据上述结果去除 `NULL` 值。 - -* 别名:`array_agg`。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md deleted file mode 100644 index 4e4e442aca1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayarray.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '将多个数组聚合为包含这些数组的更大数组。' -keywords: ['groupArrayArray', 'array_concat_agg'] -sidebar_position: 111 -slug: /sql-reference/aggregate-functions/reference/grouparrayarray -title: 'groupArrayArray' -doc_type: 'reference' ---- - -# groupArrayArray {#grouparrayarray} - -将多个数组聚合成一个由这些数组组成的更大数组。 -该函数将 [`groupArray`](/sql-reference/aggregate-functions/reference/grouparray) 与 [Array](/sql-reference/aggregate-functions/combinators#-array) 组合器结合使用。 - -别名:`array_concat_agg` - -**示例** - -我们有用于记录用户浏览会话的数据。每个会话都会记录某个特定用户访问页面的顺序。 -我们可以使用 `groupArrayArray` 函数来分析每个用户的页面访问模式。 - -```sql title="Setup" -CREATE TABLE 网站访问 ( - 用户ID UInt32, - 会话ID UInt32, - 页面访问 Array(String) -) ENGINE = Memory; - -INSERT INTO 网站访问 VALUES -(101, 1, ['首页', '产品', '结账']), -(101, 2, ['搜索', '产品详情', '联系我们']), -(102, 1, ['首页', '关于我们']), -(101, 3, ['博客', '首页']), -(102, 2, ['产品', '产品详情', '添加到购物车', '结账']); -``` - -```sql title="Query" -SELECT - user_id, - groupArrayArray(page_visits) AS user_session_page_sequences -FROM website_visits -GROUP BY user_id; -``` - -```sql title="Response" - ┌─user_id─┬─user_session_page_sequences───────────────────────────────────────────────────────────────┐ -1. │ 101 │ ['homepage','products','checkout','search','product_details','contact','blog','homepage'] │ -2. │ 102 │ ['homepage','about_us','products','product_details','add_to_cart','checkout'] │ - └─────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md deleted file mode 100644 index 6e5a7f11967..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayinsertat.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -description: '在指定位置向数组中插入值。' -sidebar_position: 140 -slug: /sql-reference/aggregate-functions/reference/grouparrayinsertat -title: 'groupArrayInsertAt' -doc_type: 'reference' ---- - -# groupArrayInsertAt {#grouparrayinsertat} - -在数组的指定位置插入一个值。 - -**语法** - -```sql -groupArrayInsertAt(default_x, size)(x, pos) -``` - -如果在一个查询中将多个值插入到同一位置,函数的行为如下: - -* 如果查询在单线程中执行,将使用第一个插入的值。 -* 如果查询在多线程中执行,结果值将是不确定的某个插入值。 - -**参数** - -* `x` — 要插入的值。[表达式](/sql-reference/syntax#expressions),其结果为某种[支持的数据类型](../../../sql-reference/data-types/index.md)。 -* `pos` — 要将指定元素 `x` 插入到的位置。数组的索引从零开始计数。[UInt32](/sql-reference/data-types/int-uint#integer-ranges)。 -* `default_x` — 用于填充空位置的默认值。可选参数。[表达式](/sql-reference/syntax#expressions),其结果的数据类型应与参数 `x` 配置的类型一致。如果未定义 `default_x`,则使用[默认值](/sql-reference/statements/create/table)。 -* `size` — 结果数组的长度。可选参数。使用该参数时,必须指定默认值 `default_x`。[UInt32](/sql-reference/data-types/int-uint#integer-ranges)。 - -**返回值** - -* 插入了值的数组。 - -类型:[Array](/sql-reference/data-types/array)。 - -**示例** - -查询: - -```sql -SELECT groupArrayInsertAt(toString(number), number * 2) FROM numbers(5); -``` - -结果: - -```text -┌─groupArrayInsertAt(toString(number), multiply(number, 2))─┐ -│ ['0','','1','','2','','3','','4'] │ -└───────────────────────────────────────────────────────────┘ -``` - -查询: - -```sql -SELECT groupArrayInsertAt('-')(toString(number), number * 2) FROM numbers(5); -``` - -结果: - -```text -┌─groupArrayInsertAt('-')(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2','-','3','-','4'] │ -└────────────────────────────────────────────────────────────────┘ -``` - -查询: - -```sql -SELECT groupArrayInsertAt('-', 5)(toString(number), number * 2) FROM numbers(5); -``` - -结果: - -```text -┌─groupArrayInsertAt('-', 5)(toString(number), multiply(number, 2))─┐ -│ ['0','-','1','-','2'] │ -└───────────────────────────────────────────────────────────────────┘ -``` - -多线程向同一位置插入多个元素。 - -查询: - -```sql -SELECT groupArrayInsertAt(number, 0) FROM numbers_mt(10) SETTINGS max_block_size = 1; -``` - -该查询会返回一个位于 `[0,9]` 范围内的随机整数。例如: - -```text -┌─groupArrayInsertAt(number, 0)─┐ -│ [7] │ -└───────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md deleted file mode 100644 index 507a4c014ec..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparrayintersect.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '返回给定数组的交集(返回所有在所有给定数组中都存在的元素)。' -sidebar_position: 141 -slug: /sql-reference/aggregate-functions/reference/grouparrayintersect -title: 'groupArrayIntersect' -doc_type: 'reference' ---- - -# groupArrayIntersect {#grouparrayintersect} - -返回给定数组的交集(即所有在每个数组中都出现的元素)。 - -**语法** - -```sql -groupArrayIntersect(x) -``` - -**参数** - -* `x` — 参数(列名或表达式)。 - -**返回值** - -* 一个数组,包含同时出现在所有数组中的元素。 - -类型:[Array](../../data-types/array.md)。 - -**示例** - -假设有表 `numbers`: - -```text -┌─a──────────────┐ -│ [1,2,4] │ -│ [1,5,2,8,-1,0] │ -│ [1,5,7,5,8,2] │ -└────────────────┘ -``` - -使用列名作为参数的查询: - -```sql -SELECT groupArrayIntersect(a) AS intersection FROM numbers; -``` - -结果: - -```text -┌─intersection──────┐ -│ [1, 2] │ -└───────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md deleted file mode 100644 index a792e3c4b37..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraylast.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: '生成包含最后一个参数值的数组。' -sidebar_position: 142 -slug: /sql-reference/aggregate-functions/reference/grouparraylast -title: 'groupArrayLast' -doc_type: 'reference' ---- - -# groupArrayLast {#grouparraylast} - -语法:`groupArrayLast(max_size)(x)` - -创建一个由最后若干个参数值组成的数组。 -例如,`groupArrayLast(1)(x)` 等价于 `[anyLast (x)]`。 - -在某些情况下,仍然可以依赖执行顺序。这适用于以下场景:当 `SELECT` 来自使用了 `ORDER BY` 的子查询,且该子查询的结果足够小时。 - -**示例** - -查询: - -```sql -SELECT groupArrayLast(2)(number+1) numbers FROM numbers(10) -``` - -结果: - -```text -┌─numbers─┐ -│ [9,10] │ -└─────────┘ -``` - -与 `groupArray` 相比: - -```sql -SELECT groupArray(2)(number+1) numbers FROM numbers(10) -``` - -```text -┌─numbers─┐ -│ [1,2] │ -└─────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md deleted file mode 100644 index 5bb037e729a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingavg.md +++ /dev/null @@ -1,82 +0,0 @@ ---- -description: '计算输入值的移动平均。' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingavg -title: 'groupArrayMovingAvg' -doc_type: 'reference' ---- - -# groupArrayMovingAvg {#grouparraymovingavg} - -计算输入值的移动平均。 - -```sql -groupArrayMovingAvg(numbers_for_summing) -groupArrayMovingAvg(window_size)(numbers_for_summing) -``` - -该函数可以将窗口大小作为参数传入。如果未指定窗口大小,函数会将其设为等于该列中的行数。 - -**参数** - -* `numbers_for_summing` — 结果为数值数据类型的[表达式](/sql-reference/syntax#expressions)。 -* `window_size` — 计算窗口的大小。 - -**返回值** - -* 与输入数据大小和类型相同的数组。 - -该函数使用[向零舍入](https://en.wikipedia.org/wiki/Rounding#Rounding_towards_zero),会截断对于结果数据类型而言无意义的小数位。 - -**示例** - -示例表 `b`: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -查询语句: - -```sql -SELECT - groupArrayMovingAvg(int) AS I, - groupArrayMovingAvg(float) AS F, - groupArrayMovingAvg(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F───────────────────────────────────┬─D─────────────────────┐ -│ [0,0,1,3] │ [0.275,0.82500005,1.9250001,3.8675] │ [0.27,0.82,1.92,3.86] │ -└───────────┴─────────────────────────────────────┴───────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingAvg(2)(int) AS I, - groupArrayMovingAvg(2)(float) AS F, - groupArrayMovingAvg(2)(dec) AS D -FROM t -``` - -```text -┌─I─────────┬─F────────────────────────────────┬─D─────────────────────┐ -│ [0,1,3,5] │ [0.55,1.6500001,3.3000002,6.085] │ [0.55,1.65,3.30,6.08] │ -└───────────┴──────────────────────────────────┴───────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md deleted file mode 100644 index c7234e11648..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraymovingsum.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '计算输入值的移动和。' -sidebar_position: 144 -slug: /sql-reference/aggregate-functions/reference/grouparraymovingsum -title: 'groupArrayMovingSum' -doc_type: 'reference' ---- - -# groupArrayMovingSum {#grouparraymovingsum} - -计算输入值的滑动和。 - -```sql -groupArrayMovingSum(numbers_for_summing) -groupArrayMovingSum(window_size)(numbers_for_summing) -``` - -该函数可以接收窗口大小作为参数。如果未指定,函数会将窗口大小默认为该列的行数。 - -**参数** - -* `numbers_for_summing` — 结果为数值数据类型值的[表达式](/sql-reference/syntax#expressions)。 -* `window_size` — 计算窗口的大小。 - -**返回值** - -* 与输入数据具有相同大小和类型的数组。 - -**示例** - -示例表: - -```sql -CREATE TABLE t -( - `int` UInt8, - `float` Float32, - `dec` Decimal32(2) -) -ENGINE = TinyLog -``` - -```text -┌─int─┬─float─┬──dec─┐ -│ 1 │ 1.1 │ 1.10 │ -│ 2 │ 2.2 │ 2.20 │ -│ 4 │ 4.4 │ 4.40 │ -│ 7 │ 7.77 │ 7.77 │ -└─────┴───────┴──────┘ -``` - -查询语句: - -```sql -SELECT - groupArrayMovingSum(int) AS I, - groupArrayMovingSum(float) AS F, - groupArrayMovingSum(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,7,14] │ [1.1,3.3000002,7.7000003,15.47] │ [1.10,3.30,7.70,15.47] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` - -```sql -SELECT - groupArrayMovingSum(2)(int) AS I, - groupArrayMovingSum(2)(float) AS F, - groupArrayMovingSum(2)(dec) AS D -FROM t -``` - -```text -┌─I──────────┬─F───────────────────────────────┬─D──────────────────────┐ -│ [1,3,6,11] │ [1.1,3.3000002,6.6000004,12.17] │ [1.10,3.30,6.60,12.17] │ -└────────────┴─────────────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md deleted file mode 100644 index c1cb1fc4a66..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysample.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: '创建一个包含示例参数值的数组。结果数组的大小限制为 `max_size` 个元素。参数值将被随机选择并添加到数组中。' -sidebar_position: 145 -slug: /sql-reference/aggregate-functions/reference/grouparraysample -title: 'groupArraySample' -doc_type: 'reference' ---- - -# groupArraySample {#grouparraysample} - -创建一个由参数值样本组成的数组。结果数组的大小限制为 `max_size` 个元素。参数值会被随机选取并添加到数组中。 - -**语法** - -```sql -groupArraySample(max_size[, seed])(x) -``` - -**参数** - -* `max_size` — 结果数组的最大长度。[UInt64](../../data-types/int-uint.md)。 -* `seed` — 随机数生成器的种子。可选。[UInt64](../../data-types/int-uint.md)。默认值:`123456`。 -* `x` — 参数(列名或表达式)。 - -**返回值** - -* 由随机选取的 `x` 参数组成的数组。 - -类型:[Array](../../data-types/array.md)。 - -**示例** - -设有表 `colors`: - -```text -┌─id─┬─color──┐ -│ 1 │ red │ -│ 2 │ blue │ -│ 3 │ green │ -│ 4 │ white │ -│ 5 │ orange │ -└────┴────────┘ -``` - -以列名作为参数的查询: - -```sql -SELECT groupArraySample(3)(color) as newcolors FROM colors; -``` - -结果: - -```text -┌─newcolors──────────────────┐ -│ ['white','blue','green'] │ -└────────────────────────────┘ -``` - -带列名且使用不同种子的查询: - -```sql -SELECT groupArraySample(3, 987654321)(color) as newcolors FROM colors; -``` - -结果: - -```text -┌─newcolors──────────────────┐ -│ ['red','orange','green'] │ -└────────────────────────────┘ -``` - -带表达式参数的查询: - -```sql -SELECT groupArraySample(3)(concat('light-', color)) as newcolors FROM colors; -``` - -结果: - -```text -┌─newcolors───────────────────────────────────┐ -│ ['light-blue','light-orange','light-green'] │ -└─────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md deleted file mode 100644 index 8a01831f0e1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/grouparraysorted.md +++ /dev/null @@ -1,47 +0,0 @@ ---- -description: '返回按升序排列的前 N 个元素组成的数组。' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/grouparraysorted -title: 'groupArraySorted' -doc_type: 'reference' ---- - -# groupArraySorted {#grouparraysorted} - -返回一个包含按升序排列的前 N 个元素的数组。 - -```sql -groupArraySorted(N)(column) -``` - -**参数** - -* `N` – 要返回的元素数量。 - -* `column` – 列的值(Integer、String、Float 以及其他通用类型)。 - -**示例** - -获取前 10 个数字: - -```sql -SELECT groupArraySorted(10)(number) FROM numbers(100) -``` - -```text -┌─groupArraySorted(10)(number)─┐ -│ [0,1,2,3,4,5,6,7,8,9] │ -└──────────────────────────────┘ -``` - -获取列中所有数字的 String 表示: - -```sql -SELECT groupArraySorted(5)(str) FROM (SELECT toString(number) AS str FROM numbers(5)); -``` - -```text -┌─groupArraySorted(5)(str)─┐ -│ ['0','1','2','3','4'] │ -└──────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md deleted file mode 100644 index 10053cdc2b5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitand.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '对一组数值执行按位 `AND` 运算。' -sidebar_position: 147 -slug: /sql-reference/aggregate-functions/reference/groupbitand -title: 'groupBitAnd' -doc_type: 'reference' ---- - -# groupBitAnd {#groupbitand} - -对一系列数值执行按位 `AND` 运算。 - -```sql -groupBitAnd(expr) -``` - -**参数** - -`expr` – 其计算结果为 `UInt*` 或 `Int*` 类型的表达式。 - -**返回值** - -`UInt*` 或 `Int*` 类型的值。 - -**示例** - -测试数据: - -```text -二进制 十进制 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -查询: - -```sql -SELECT groupBitAnd(num) FROM t -``` - -其中 `num` 是存放测试数据的列。 - -结果: - -```text -二进制 十进制 -00000100 = 4 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md deleted file mode 100644 index 2b35b114190..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmap.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '对无符号整数列执行 Bitmap 聚合计算,返回类型为 UInt64 的基数;如果添加后缀 -State,则返回一个 bitmap 对象' -sidebar_position: 148 -slug: /sql-reference/aggregate-functions/reference/groupbitmap -title: 'groupBitmap' -doc_type: 'reference' ---- - -# groupBitmap {#groupbitmap} - -对无符号整数列执行位图或聚合计算,返回值为 UInt64 类型的基数。如果添加后缀 -State,则返回[位图对象](../../../sql-reference/functions/bitmap-functions.md)。 - -```sql -groupBitmap(expr) -``` - -**参数** - -`expr` – 结果为 `UInt*` 类型的表达式。 - -**返回值** - -返回 `UInt64` 类型的值。 - -**示例** - -测试数据: - -```text -UserID -1 -1 -2 -3 -``` - -查询: - -```sql -SELECT groupBitmap(UserID) AS num FROM t -``` - -结果: - -```text -num -3 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md deleted file mode 100644 index 4501c8804cc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapand.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '计算位图列的按位与,并返回 UInt64 类型的基数;如果增加后缀 -State,则返回位图对象。' -sidebar_position: 149 -slug: /sql-reference/aggregate-functions/reference/groupbitmapand -title: 'groupBitmapAnd' -doc_type: 'reference' ---- - -计算位图列的按位与,并返回 UInt64 类型的基数;如果增加后缀 -State,则返回 [bitmap 对象](../../../sql-reference/functions/bitmap-functions.md)。 - -```sql -groupBitmapAnd(expr) -``` - -**参数** - -`expr` – 结果类型为 `AggregateFunction(groupBitmap, UInt*)` 的表达式。 - -**返回值** - -`UInt64` 类型的值。 - -**示例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapAnd(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapAnd(z)─┐ -│ 3 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapAndState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapAndState(z)))─┐ -│ [6,8,10] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md deleted file mode 100644 index 3fae95d80e0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '对 bitmap 列进行 OR 运算,返回 UInt64 类型的基数。如果添加 -State 后缀,则返回一个 bitmap 对象。这等价于 `groupBitmapMerge`。' -sidebar_position: 150 -slug: /sql-reference/aggregate-functions/reference/groupbitmapor -title: 'groupBitmapOr' -doc_type: 'reference' ---- - -# groupBitmapOr {#groupbitmapor} - -计算位图列的按位或,并以 UInt64 类型返回其基数;如果添加后缀 -State,则返回一个 [bitmap 对象](../../../sql-reference/functions/bitmap-functions.md)。这等价于 `groupBitmapMerge`。 - -```sql -groupBitmapOr(expr) -``` - -**参数** - -`expr` – 结果类型为 `AggregateFunction(groupBitmap, UInt*)` 的表达式。 - -**返回值** - -`UInt64` 类型的值。 - -**示例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapOr(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapOr(z)─┐ -│ 15 │ -└──────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapOrState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapOrState(z)))─┐ -│ [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] │ -└─────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md deleted file mode 100644 index 7df167fcc26..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitmapxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '对 bitmap 列执行 XOR 运算并返回 UInt64 类型的基数;如果带有 -State 后缀,则返回一个 bitmap 对象' -sidebar_position: 151 -slug: /sql-reference/aggregate-functions/reference/groupbitmapxor -title: 'groupBitmapXor' -doc_type: 'reference' ---- - -# groupBitmapXor {#groupbitmapxor} - -`groupBitmapXor` 计算 bitmap 列按位异或后的结果的基数,并以 UInt64 类型返回。如果使用后缀 -State,则返回一个 [bitmap 对象](../../../sql-reference/functions/bitmap-functions.md)。 - -```sql -groupBitmapXor(expr) -``` - -**参数** - -`expr` – 结果类型为 `AggregateFunction(groupBitmap, UInt*)` 的表达式。 - -**返回值** - -类型为 `UInt64` 的值。 - -**示例** - -```sql -DROP TABLE IF EXISTS bitmap_column_expr_test2; -CREATE TABLE bitmap_column_expr_test2 -( - tag_id String, - z AggregateFunction(groupBitmap, UInt32) -) -ENGINE = MergeTree -ORDER BY tag_id; - -INSERT INTO bitmap_column_expr_test2 VALUES ('tag1', bitmapBuild(cast([1,2,3,4,5,6,7,8,9,10] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag2', bitmapBuild(cast([6,7,8,9,10,11,12,13,14,15] AS Array(UInt32)))); -INSERT INTO bitmap_column_expr_test2 VALUES ('tag3', bitmapBuild(cast([2,4,6,8,10,12] AS Array(UInt32)))); - -SELECT groupBitmapXor(z) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─groupBitmapXor(z)─┐ -│ 10 │ -└───────────────────┘ - -SELECT arraySort(bitmapToArray(groupBitmapXorState(z))) FROM bitmap_column_expr_test2 WHERE like(tag_id, 'tag%'); -┌─arraySort(bitmapToArray(groupBitmapXorState(z)))─┐ -│ [1,3,5,6,8,10,11,13,14,15] │ -└──────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md deleted file mode 100644 index 7bbf1179ee4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '对一系列数字进行按位 `OR` 运算。' -sidebar_position: 152 -slug: /sql-reference/aggregate-functions/reference/groupbitor -title: 'groupBitOr' -doc_type: 'reference' ---- - -# groupBitOr {#groupbitor} - -对一系列数字执行按位 `OR` 运算。 - -```sql -groupBitOr(expr) -``` - -**参数** - -`expr` – 其结果为 `UInt*` 或 `Int*` 类型的表达式。 - -**返回值** - -`UInt*` 或 `Int*` 类型的值。 - -**示例** - -测试数据: - -```text -二进制 十进制 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -查询: - -```sql -SELECT groupBitOr(num) FROM t -``` - -其中 `num` 是测试数据所在的列。 - -结果: - -```text -二进制 十进制 -01111101 = 125 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md deleted file mode 100644 index 2897e5565bf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupbitxor.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -description: '对一系列数字应用按位 `XOR` 运算。' -sidebar_position: 153 -slug: /sql-reference/aggregate-functions/reference/groupbitxor -title: 'groupBitXor' -doc_type: 'reference' ---- - -# groupBitXor {#groupbitxor} - -对一组数值执行按位 `XOR` 运算。 - -```sql -groupBitXor(expr) -``` - -**参数** - -`expr` – 结果类型为 `UInt*` 或 `Int*` 的表达式。 - -**返回值** - -`UInt*` 或 `Int*` 类型的值。 - -**示例** - -测试数据: - -```text -二进制 十进制 -00101100 = 44 -00011100 = 28 -00001101 = 13 -01010101 = 85 -``` - -查询: - -```sql -SELECT groupBitXor(num) FROM t -``` - -其中 `num` 是测试数据所在的列。 - -结果: - -```text -二进制 十进制 -01101000 = 104 -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md deleted file mode 100644 index 977edf4ca23..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupconcat.md +++ /dev/null @@ -1,104 +0,0 @@ ---- -description: '从一组字符串生成拼接后的字符串,可选使用分隔符分隔,并可选限制元素的最大数量。' -sidebar_label: 'groupConcat' -sidebar_position: 363 -slug: /sql-reference/aggregate-functions/reference/groupconcat -title: 'groupConcat' -doc_type: 'reference' ---- - -从一组字符串生成拼接后的字符串,可选使用分隔符分隔,并可选限制元素的最大数量。 - -**语法** - -```sql -groupConcat[(分隔符 [, 限制])](表达式); -``` - -Alias: `group_concat` - -**Arguments** - -* `expression` — 输出为要被连接字符串的表达式或列名。 -* `delimiter` — 用于分隔被连接值的[字符串](../../../sql-reference/data-types/string.md)。此参数为可选项,未指定时默认为空字符串,或使用在 Parameters 中提供的分隔符。 - -**Parameters** - -* `delimiter` — 用于分隔被连接值的[字符串](../../../sql-reference/data-types/string.md)。此参数为可选项,未指定时默认为空字符串。 -* `limit` — 指定要连接的最大元素数量的正[整数](../../../sql-reference/data-types/int-uint.md)。如果存在更多元素,超出的元素将被忽略。此参数为可选项。 - -:::note -如果仅指定了 delimiter 而未指定 limit,则 delimiter 必须是第一个参数。如果同时指定了 delimiter 和 limit,则 delimiter 必须位于 limit 之前。 - -另外,如果在 Arguments 和 Parameters 中分别指定了不同的分隔符,将只使用 Arguments 中的分隔符。 -::: - -**Returned value** - -* 返回由列或表达式的连接值组成的[字符串](../../../sql-reference/data-types/string.md)。如果分组中没有元素或仅包含 null 元素,并且函数未指定仅 null 值的处理方式,则结果为一个值为 null 的可空字符串。 - -**Examples** - -输入表: - -```text -┌─id─┬─name─┐ -│ 1 │ John │ -│ 2 │ Jane │ -│ 3 │ Bob │ -└────┴──────┘ -``` - -1. 不带分隔符的基本用法: - -查询: - -```sql -SELECT groupConcat(Name) FROM Employees; -``` - -结果: - -```text -JohnJaneBob -``` - -这会将所有名称连接成一个没有任何分隔符的连续字符串。 - -2. 使用逗号作为分隔符: - -查询: - -```sql -SELECT groupConcat(', ')(Name) FROM Employees; -``` - -或 - -```sql -SELECT groupConcat(Name, ', ') FROM Employees; -``` - -结果: - -```text -John, Jane, Bob -``` - -输出结果显示名称之间以逗号和空格分隔。 - -3. 限制拼接元素的数量 - -查询: - -```sql -SELECT groupConcat(', ', 2)(Name) FROM Employees; -``` - -结果: - -```text -John, Jane -``` - -此查询仅返回前两个名称,即使表中还有更多名称。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md deleted file mode 100644 index 65e88ac78af..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/groupuniqarray.md +++ /dev/null @@ -1,16 +0,0 @@ ---- -description: '根据不同的参数值创建数组。' -sidebar_position: 154 -slug: /sql-reference/aggregate-functions/reference/groupuniqarray -title: 'groupUniqArray' -doc_type: 'reference' ---- - -# groupUniqArray {#groupuniqarray} - -语法:`groupUniqArray(x)` 或 `groupUniqArray(max_size)(x)` - -从不同的参数取值创建一个数组。内存消耗与 [uniqExact](../../../sql-reference/aggregate-functions/reference/uniqexact.md) 函数相同。 - -第二种形式(带有 `max_size` 参数)将结果数组的大小限制为 `max_size` 个元素。 -例如,`groupUniqArray(1)(x)` 等价于 `[any(x)]`。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md deleted file mode 100644 index 728ba4f7ad9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/index.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -description: '聚合函数概览页,包含所有聚合函数的完整列表' -sidebar_position: 36 -slug: /sql-reference/aggregate-functions/reference/ -title: '聚合函数' -toc_folder_title: '参考' -toc_hidden: true -doc_type: 'landing-page' ---- - -# 聚合函数 {#aggregate-functions} - -ClickHouse 支持所有标准 SQL 聚合函数([sum](../reference/sum.md)、[avg](../reference/avg.md)、[min](../reference/min.md)、[max](../reference/max.md)、[count](../reference/count.md)),以及大量其他聚合函数。 - -{/* 本页的目录表格由以下脚本自动生成: - https://github.com/ClickHouse/clickhouse-docs/blob/main/scripts/autogenerate-table-of-contents.sh - 根据 YAML front matter 中的 slug、description、title 字段生成。 - - 如果您发现错误,请直接编辑相应页面的 YAML front matter。 - */ } - -{/*AUTOGENERATED_START*/ } - -| 页面 | 说明 | -| --------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | -| [aggThrow](/sql-reference/aggregate-functions/reference/aggthrow) | 此函数可用于测试异常安全性。它会在创建时以指定概率抛出异常。 | -| [analysisOfVariance](/sql-reference/aggregate-functions/reference/analysis_of_variance) | 提供单因素方差分析(ANOVA)的统计检验。它用于对若干组服从正态分布的观测值进行检验,以判断各组的均值是否相同。 | -| [any](/sql-reference/aggregate-functions/reference/any) | 选取某列中首次出现的值。 | -| [anyHeavy](/sql-reference/aggregate-functions/reference/anyheavy) | 使用 heavy hitters 算法选择一个频繁出现的值。如果在每个查询执行线程中,都存在某个值在该线程中出现次数超过一半,则返回该值。通常,该结果是非确定性的。 | -| [anyLast](/sql-reference/aggregate-functions/reference/anylast) | 返回列中最后出现的值。 | -| [approx_top_k](/sql-reference/aggregate-functions/reference/approxtopk) | 返回一个数组,其中包含指定列中近似出现频率最高的值及其出现次数。 | -| [approx_top_sum](/sql-reference/aggregate-functions/reference/approxtopsum) | 返回一个数组,其中包含指定列中出现频率最高的值的近似结果及其对应计数。 | -| [argMax](/sql-reference/aggregate-functions/reference/argmax) | 计算使 `val` 取得最大值时的 `arg` 值。 | -| [argMin](/sql-reference/aggregate-functions/reference/argmin) | 计算最小 `val` 值对应的 `arg` 值。如果存在多行具有相同的 `val` 且该值为最大值,则返回哪一个关联的 `arg` 是不确定的。 | -| [argAndMax](/sql-reference/aggregate-functions/reference/argandmax) | 计算最大 `val` 值对应的 `arg` 和 `val`。如果存在多行记录的 `val` 相同且都是最大值,则返回哪一行对应的 `arg` 和 `val` 是不确定的。 | -| [argAndMin](/sql-reference/aggregate-functions/reference/argandmin) | 计算最小 `val` 值所对应的 `arg` 和 `val`。若存在多行记录的 `val` 相同且均为最小值,则最终返回哪一行对应的 `arg` 和 `val` 是不确定的。 | -| [groupArrayArray](/sql-reference/aggregate-functions/reference/grouparrayarray) | 将多个数组聚合成一个更大的数组。 | -| [avg](/sql-reference/aggregate-functions/reference/avg) | 计算算术平均值。 | -| [avgWeighted](/sql-reference/aggregate-functions/reference/avgweighted) | 计算加权算术平均值。 | -| [boundingRatio](/sql-reference/aggregate-functions/reference/boundingRatio) | 用于计算一组值中最左端点和最右端点之间斜率的聚合函数。 | -| [categoricalInformationValue](/sql-reference/aggregate-functions/reference/categoricalinformationvalue) | 为每个类别计算 `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` 的值。 | -| [contingency](/sql-reference/aggregate-functions/reference/contingency) | `contingency` 函数计算列联系数,该系数用于度量表中两列之间的关联程度。其计算方式与 `cramersV` 函数类似,但平方根中的分母不同。 | -| [corr](/sql-reference/aggregate-functions/reference/corr) | 计算皮尔逊相关系数。 | -| [corrMatrix](/sql-reference/aggregate-functions/reference/corrmatrix) | 计算 N 个变量的相关矩阵。 | -| [corrStable](/sql-reference/aggregate-functions/reference/corrstable) | 计算 Pearson 相关系数,但采用数值上更稳定的算法。 | -| [count](/sql-reference/aggregate-functions/reference/count) | 计算行数或非 NULL 值的数量。 | -| [covarPop](/sql-reference/aggregate-functions/reference/covarpop) | 计算总体协方差 | -| [covarPopMatrix](/sql-reference/aggregate-functions/reference/covarpopmatrix) | 返回 N 个变量之间的总体协方差矩阵。 | -| [covarPopStable](/sql-reference/aggregate-functions/reference/covarpopstable) | 计算总体协方差 | -| [covarSamp](/sql-reference/aggregate-functions/reference/covarsamp) | 计算 `Σ((x - x̅)(y - y̅)) / (n - 1)` 的值 | -| [covarSampMatrix](/sql-reference/aggregate-functions/reference/covarsampmatrix) | 返回 N 个变量的样本协方差矩阵。 | -| [covarSampStable](/sql-reference/aggregate-functions/reference/covarsampstable) | 类似于 covarSamp,但运行速度较慢,计算误差更小。 | -| [cramersV](/sql-reference/aggregate-functions/reference/cramersv) | `cramersV` 函数的结果范围从 0(表示变量之间没有关联)到 1,并且只有在每个值都完全由另一个值决定时才会达到 1。它可以被理解为两个变量之间关联程度相对于其最大可能变动的百分比。 | -| [cramersVBiasCorrected](/sql-reference/aggregate-functions/reference/cramersvbiascorrected) | 计算 Cramer's V,但使用了偏差校正。 | -| [deltaSum](/sql-reference/aggregate-functions/reference/deltasum) | 对连续行之间的算术差值进行求和。 | -| [deltaSumTimestamp](/sql-reference/aggregate-functions/reference/deltasumtimestamp) | 对相邻行之间的差值进行相加。如果差值为负,则会被忽略。 | -| [entropy](/sql-reference/aggregate-functions/reference/entropy) | 计算某列值的香农熵。 | -| [estimateCompressionRatio](/sql-reference/aggregate-functions/reference/estimateCompressionRatio) | 在不实际压缩给定列的情况下,估计其压缩比。 | -| [exponentialMovingAverage](/sql-reference/aggregate-functions/reference/exponentialMovingAverage) | 计算指定时间范围内数值的指数移动平均值。 | -| [exponentialTimeDecayedAvg](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedAvg) | 返回在时间点 `t` 处的时间序列值的指数平滑加权移动平均。 | -| [exponentialTimeDecayedCount](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedCount) | 返回时间序列在时间索引 `t` 处的累积指数衰减。 | -| [exponentialTimeDecayedMax](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedMax) | 返回时间索引为 `t` 与 `t-1` 时刻所计算的指数平滑移动平均值中的较大值。 | -| [exponentialTimeDecayedSum](/sql-reference/aggregate-functions/reference/exponentialTimeDecayedSum) | 返回时间序列在时间索引 `t` 处的指数平滑移动平均值之和。 | -| [first_value](/sql-reference/aggregate-functions/reference/first_value) | 它是 `any` 的别名,引入它是为了与窗口函数(Window Functions)兼容,因为在某些情况下需要处理 `NULL` 值(默认情况下,所有 ClickHouse 聚合函数都会忽略 `NULL` 值)。 | -| [flameGraph](/sql-reference/aggregate-functions/reference/flame_graph) | 根据堆栈跟踪列表构建火焰图的聚合函数。 | -| [groupArray](/sql-reference/aggregate-functions/reference/grouparray) | 创建一个包含参数值的数组。可以以任意顺序(顺序不固定)将值添加到该数组中。 | -| [groupArrayInsertAt](/sql-reference/aggregate-functions/reference/grouparrayinsertat) | 在数组的指定位置插入一个值。 | -| [groupArrayIntersect](/sql-reference/aggregate-functions/reference/grouparrayintersect) | 返回给定数组的交集(即所有给定数组中都包含的元素)。 | -| [groupArrayLast](/sql-reference/aggregate-functions/reference/grouparraylast) | 创建一个由最后一个参数的值组成的数组。 | -| [groupArrayMovingAvg](/sql-reference/aggregate-functions/reference/grouparraymovingavg) | 计算输入值的移动平均。 | -| [groupArrayMovingSum](/sql-reference/aggregate-functions/reference/grouparraymovingsum) | 计算输入值的滑动求和。 | -| [groupArraySample](/sql-reference/aggregate-functions/reference/grouparraysample) | 创建一个参数值样本数组。结果数组的大小上限为 `max_size` 个元素。参数值会被随机选取并添加到数组中。 | -| [groupArraySorted](/sql-reference/aggregate-functions/reference/grouparraysorted) | 返回一个按升序排列的前 N 个元素的数组。 | -| [timeSeriesGroupArray](/sql-reference/aggregate-functions/reference/timeSeriesGroupArray) | 按时间戳对时间序列进行升序排序。 | -| [groupBitAnd](/sql-reference/aggregate-functions/reference/groupbitand) | 对一组数字执行按位 `AND` 运算。 | -| [groupBitmap](/sql-reference/aggregate-functions/reference/groupbitmap) | 对无符号整数列进行 Bitmap 或聚合计算时,返回 UInt64 类型的基数;如果添加后缀 -State,则返回位图对象 | -| [groupBitmapAnd](/sql-reference/aggregate-functions/reference/groupbitmapand) | 对位图列执行 AND 运算,返回 UInt64 类型的基数值;如果添加后缀 -State,则返回一个位图对象。 | -| [groupBitmapOr](/sql-reference/aggregate-functions/reference/groupbitmapor) | 对位图列执行 OR 计算,返回基数(类型为 UInt64);如果添加后缀 -State,则返回一个位图对象。等同于 `groupBitmapMerge`。 | -| [groupBitmapXor](/sql-reference/aggregate-functions/reference/groupbitmapxor) | 计算 bitmap 列的 XOR,并以 UInt64 类型返回基数;如果使用 -State 后缀,则返回一个 bitmap 对象 | -| [groupBitOr](/sql-reference/aggregate-functions/reference/groupbitor) | 对一系列数值执行按位 `OR` 运算。 | -| [groupBitXor](/sql-reference/aggregate-functions/reference/groupbitxor) | 对一组数字执行按位 `XOR` 运算。 | -| [groupUniqArray](/sql-reference/aggregate-functions/reference/groupuniqarray) | 从多个参数值创建数组。 | -| [intervalLengthSum](/sql-reference/aggregate-functions/reference/intervalLengthSum) | 计算所有区间(数轴线段)的并集的总长度。 | -| [kolmogorovSmirnovTest](/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest) | 对来自两个总体的样本执行 Kolmogorov-Smirnov 检验。 | -| [kurtPop](/sql-reference/aggregate-functions/reference/kurtpop) | 计算序列的峰度值。 | -| [kurtSamp](/sql-reference/aggregate-functions/reference/kurtsamp) | 计算序列的样本峰度。 | -| [largestTriangleThreeBuckets](/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets) | 对输入数据应用 Largest-Triangle-Three-Buckets 算法。 | -| [last_value](/sql-reference/aggregate-functions/reference/last_value) | 选择最近一次出现的值,类似于 `anyLast`,但可以接受 NULL 值。 | -| [mannWhitneyUTest](/sql-reference/aggregate-functions/reference/mannwhitneyutest) | 对来自两个总体的样本执行 Mann-Whitney 秩检验。 | -| [max](/sql-reference/aggregate-functions/reference/max) | 计算一组值的最大值的聚合函数。 | -| [maxIntersections](/sql-reference/aggregate-functions/reference/maxintersections) | 计算一组区间之间相互相交的最大次数的聚合函数(在所有区间至少相交一次的前提下)。 | -| [maxIntersectionsPosition](/sql-reference/aggregate-functions/reference/maxintersectionsposition) | 用于计算 `maxIntersections` 函数出现位置的聚合函数。 | -| [maxMap](/sql-reference/aggregate-functions/reference/maxmap) | 根据 `key` 数组中的键,对 `value` 数组求最大值。 | -| [meanZTest](/sql-reference/aggregate-functions/reference/meanztest) | 对来自两个总体的样本进行均值z检验。 | -| [median](/sql-reference/aggregate-functions/reference/median) | `median*` 函数是对应 `quantile*` 函数的别名。它们用于计算数值型数据样本的中位数。 | -| [min](/sql-reference/aggregate-functions/reference/min) | 用于计算一组值中最小值的聚合函数。 | -| [minMap](/sql-reference/aggregate-functions/reference/minmap) | 按 `key` 数组中指定的键,从 `value` 数组中计算最小值。 | -| [quantile](/sql-reference/aggregate-functions/reference/quantile) | 计算数值序列的近似分位数。 | -| [quantileBFloat16](/sql-reference/aggregate-functions/reference/quantilebfloat16) | 计算由 bfloat16 数值构成的样本的近似分位数。 | -| [quantileDD](/sql-reference/aggregate-functions/reference/quantileddsketch) | 计算样本的近似分位数,并提供相对误差保证。 | -| [quantileDeterministic](/sql-reference/aggregate-functions/reference/quantiledeterministic) | 计算数值序列的近似分位数。 | -| [quantileExact 函数](/sql-reference/aggregate-functions/reference/quantileexact) | quantileExact、quantileExactLow、quantileExactHigh、quantileExactExclusive、quantileExactInclusive 函数 | -| [quantileExactWeighted](/sql-reference/aggregate-functions/reference/quantileexactweighted) | 精确计算数值数据序列的分位数,并考虑每个元素的权重。 | -| [quantileGK](/sql-reference/aggregate-functions/reference/quantileGK) | 使用 Greenwald-Khanna 算法计算数值序列的分位数。 | -| [quantileExactWeightedInterpolated](/sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated) | 使用线性插值计算数值数据序列的分位数,同时考虑每个元素的权重。 | -| [quantileInterpolatedWeighted](/sql-reference/aggregate-functions/reference/quantileInterpolatedWeighted) | 使用线性插值计算数值数据序列的分位数,并考虑各元素的权重。 | -| [quantiles 聚合函数](/sql-reference/aggregate-functions/reference/quantiles) | quantiles, quantilesExactExclusive, quantilesExactInclusive, quantilesGK | -| [quantileTDigest](/sql-reference/aggregate-functions/reference/quantiletdigest) | 使用 t-digest 算法计算数值序列的近似分位数。 | -| [quantileTDigestWeighted](/sql-reference/aggregate-functions/reference/quantiletdigestweighted) | 使用 t-digest 算法计算数值数据序列的近似分位数。 | -| [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) | 以指定精度计算数值数据序列的分位数。 | -| [quantileTimingWeighted](/sql-reference/aggregate-functions/reference/quantiletimingweighted) | 以指定的精度,根据每个序列元素的权重计算数值数据序列的分位数。 | -| [rankCorr](/sql-reference/aggregate-functions/reference/rankCorr) | 计算秩相关系数。 | -| [simpleLinearRegression](/sql-reference/aggregate-functions/reference/simplelinearregression) | 执行简单的一元线性回归。 | -| [singleValueOrNull](/sql-reference/aggregate-functions/reference/singlevalueornull) | 聚合函数 `singleValueOrNull` 用于实现子查询运算符,例如 `x = ALL (SELECT ...)`。它会检查数据中是否仅有一个唯一的非 NULL 值。 | -| [skewPop](/sql-reference/aggregate-functions/reference/skewpop) | 计算序列的偏度。 | -| [skewSamp](/sql-reference/aggregate-functions/reference/skewsamp) | 计算一组数据的样本偏度。 | -| [sparkbar](/sql-reference/aggregate-functions/reference/sparkbar) | 该函数在区间 `[min_x, max_x]` 上,根据取值 `x` 及其重复次数 `y` 绘制频率直方图。 | -| [stddevPop](/sql-reference/aggregate-functions/reference/stddevpop) | 结果等于 varPop 的平方根。 | -| [stddevPopStable](/sql-reference/aggregate-functions/reference/stddevpopstable) | 结果等于 varPop 的平方根。与 stddevPop 不同,此函数使用在数值上稳定的算法。 | -| [stddevSamp](/sql-reference/aggregate-functions/reference/stddevsamp) | 结果等于 varSamp 的平方根 | -| [stddevSampStable](/sql-reference/aggregate-functions/reference/stddevsampstable) | 结果等于 varSamp 的平方根。与 varSamp 不同的是,本函数使用数值稳定的算法。 | -| [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlinearregression) | 此函数实现了随机线性回归。它支持自定义学习率、L2 正则化系数、小批量大小等参数,并提供几种用于更新权重的方法(Adam、简单 SGD、Momentum、Nesterov)。 | -| [stochasticLogisticRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) | 此函数实现了随机逻辑回归。它可用于二元分类问题,支持与 stochasticLinearRegression 相同的自定义参数,且工作方式相同。 | -| [studentTTest](/sql-reference/aggregate-functions/reference/studentttest) | 对来自两个总体的样本进行 Student t 检验。 | -| [studentTTestOneSample](/sql-reference/aggregate-functions/reference/studentttestonesample) | 对样本与已知总体均值执行单样本 Student t 检验。 | -| [sum](/sql-reference/aggregate-functions/reference/sum) | 计算总和。仅适用于数值。 | -| [sumCount](/sql-reference/aggregate-functions/reference/sumcount) | 同时计算数值总和和行数。该函数由 ClickHouse 查询优化器使用:如果在一个查询中存在多个 `sum`、`count` 或 `avg` 函数,它们可以被替换为单个 `sumCount` 函数以重用计算结果。通常很少需要显式使用该函数。 | -| [sumKahan](/sql-reference/aggregate-functions/reference/sumkahan) | 使用 Kahan 补偿求和算法计算数值之和 | -| [sumMap](/sql-reference/aggregate-functions/reference/summap) | 根据 `key` 数组中指定的键,对一个或多个 `value` 数组求和。返回一个数组元组:依次为按排序顺序排列的键数组,以及对应键的求和值数组,且不会发生溢出。 | -| [sumMapWithOverflow](/sql-reference/aggregate-functions/reference/summapwithoverflow) | 根据 `key` 数组中指定的键对 `value` 数组求和。返回一个包含两个数组的元组:按排序顺序排列的键,以及对应键的求和值。与 `sumMap` 函数不同之处在于,它在求和时会发生溢出(不进行溢出检查)。 | -| [sumWithOverflow](/sql-reference/aggregate-functions/reference/sumwithoverflow) | 使用与输入参数相同的数据类型计算数值的总和。如果总和超过该数据类型的最大值,则会发生溢出。 | -| [theilsU](/sql-reference/aggregate-functions/reference/theilsu) | `theilsU` 函数计算 Theils' U 不确定性系数,用于度量表中两列之间的关联程度。 | -| [topK](/sql-reference/aggregate-functions/reference/topk) | 返回一个数组,其中包含指定列中近似出现频率最高的值。结果数组按值的近似出现频率降序排列(而不是按值本身排序)。 | -| [topKWeighted](/sql-reference/aggregate-functions/reference/topkweighted) | 返回一个数组,包含指定列中近似最常出现的值。结果数组按值的近似频率降序排序(而不是按值本身排序)。此外,还会考虑值的权重。 | -| [uniq](/sql-reference/aggregate-functions/reference/uniq) | 计算参数不同取值的大致个数。 | -| [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) | 计算参数不同取值的近似数量。 | -| [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) | 计算不同参数取值的近似数量。与 uniqCombined 相同,但对所有数据类型都使用 64 位哈希,而不仅仅是对 String 数据类型使用。 | -| [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) | 计算不同参数取值的精确个数。 | -| [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) | 使用 HyperLogLog 算法计算不同参数取值的近似数量。 | -| [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) | 使用 Theta Sketch Framework 近似计算不同参数取值的数量。 | -| [varPop](/sql-reference/aggregate-functions/reference/varPop) | 计算总体方差。 | -| [varPopStable](/sql-reference/aggregate-functions/reference/varpopstable) | 返回总体方差。与 varPop 不同,该函数采用数值更稳定的算法。虽然运行速度较慢,但计算误差更小。 | -| [varSamp](/sql-reference/aggregate-functions/reference/varSamp) | 计算一组数据的样本方差。 | -| [varSampStable](/sql-reference/aggregate-functions/reference/varsampstable) | 计算数据集的样本方差。与 `varSamp` 不同,此函数使用数值稳定的算法。虽然运行速度较慢,但计算误差更小。 | -| [welchTTest](/sql-reference/aggregate-functions/reference/welchttest) | 对来自两个总体的样本进行 Welch t 检验。 | -| [distinctDynamicTypes](/sql-reference/aggregate-functions/reference/distinctdynamictypes) | 计算 Dynamic 列中存储的不同数据类型列表。 | -| [distinctJSONPaths](/sql-reference/aggregate-functions/reference/distinctjsonpaths) | 计算 JSON 列中存储的唯一路径列表。 | -| [timeSeriesDeltaToGrid](/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid) | 用于在指定网格上对时间序列数据计算类似 PromQL 的 delta 的聚合函数。 | -| [timeSeriesInstantDeltaToGrid](/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid) | 用于在指定网格上,对时间序列数据计算类似 PromQL 的 idelta 的聚合函数。 | -| [timeSeriesInstantRateToGrid](/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid) | 用于在指定网格上对时间序列数据计算类似 PromQL 的 irate 的聚合函数。 | -| [timeSeriesLastTwoSamples](/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples) | 用于对时间序列数据进行重采样,以执行类似 PromQL 的 irate 和 idelta 计算的聚合函数 | -| [timeSeriesRateToGrid](/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid) | 在指定网格上对时间序列数据计算类似 PromQL 的 rate 的聚合函数。 | -| [timeSeriesResampleToGridWithStaleness](/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness) | 用于将时间序列数据重采样到指定网格的聚合函数。 | -| [timeSeriesDerivToGrid](/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid) | 用于在指定网格上对时间序列数据计算 PromQL 风格导数的聚合函数。 | -| [timeSeriesPredictLinearToGrid](/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid) | 用于在指定网格上对时间序列数据执行类似 PromQL 的线性预测计算的聚合函数。 | -| [timeSeriesChangesToGrid](/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid) | 一种聚合函数,用于在指定网格上对时间序列数据计算类似 PromQL 的变化。 | -| [timeSeriesResetsToGrid](/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid) | 一种聚合函数,用于按指定网格对时间序列数据计算类似 PromQL 的重置次数。 | -| [groupConcat](/sql-reference/aggregate-functions/reference/groupconcat) | 从一组字符串生成连接后的字符串,可选指定分隔符,也可选限制参与连接的最大元素数。 | -| [quantilePrometheusHistogram](/sql-reference/aggregate-functions/reference/quantilePrometheusHistogram) | 使用线性插值计算直方图的分位数。 | - -{/*AUTOGENERATED_END*/ } diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md deleted file mode 100644 index 1b9878f3a08..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/intervalLengthSum.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: '计算所有区间(数轴上的线段)并集的总长度。' -sidebar_label: 'intervalLengthSum' -sidebar_position: 155 -slug: /sql-reference/aggregate-functions/reference/intervalLengthSum -title: 'intervalLengthSum' -doc_type: 'reference' ---- - -计算所有区间(数轴上的线段)并集的总长度。 - -**语法** - -```sql -intervalLengthSum(start, end) -``` - -**参数** - -* `start` — 区间的起始值。[Int32](/sql-reference/data-types/int-uint#integer-ranges)、[Int64](/sql-reference/data-types/int-uint#integer-ranges)、[UInt32](/sql-reference/data-types/int-uint#integer-ranges)、[UInt64](/sql-reference/data-types/int-uint#integer-ranges)、[Float32](/sql-reference/data-types/float)、[Float64](/sql-reference/data-types/float)、[DateTime](/sql-reference/data-types/datetime) 或 [Date](/sql-reference/data-types/date)。 -* `end` — 区间的结束值。[Int32](/sql-reference/data-types/int-uint#integer-ranges)、[Int64](/sql-reference/data-types/int-uint#integer-ranges)、[UInt32](/sql-reference/data-types/int-uint#integer-ranges)、[UInt64](/sql-reference/data-types/int-uint#integer-ranges)、[Float32](/sql-reference/data-types/float)、[Float64](/sql-reference/data-types/float)、[DateTime](/sql-reference/data-types/datetime) 或 [Date](/sql-reference/data-types/date)。 - -:::note -两个参数的数据类型必须相同。否则会抛出异常。 -::: - -**返回值** - -* 所有区间(数轴上的线段)并集的总长度。根据参数的类型,返回值可能是 [UInt64](/sql-reference/data-types/int-uint#integer-ranges) 或 [Float64](/sql-reference/data-types/float) 类型。 - -**示例** - -1. 输入表: - -```text -┌─id─┬─start─┬─end─┐ -│ a │ 1.1 │ 2.9 │ -│ a │ 2.5 │ 3.2 │ -│ a │ 4 │ 5 │ -└────┴───────┴─────┘ -``` - -在这个示例中,使用的是 `Float32` 类型的参数。函数返回 `Float64` 类型的值。 - -结果为区间 `[1.1, 3.2]`(`[1.1, 2.9]` 和 `[2.5, 3.2]` 的并集)以及 `[4, 5]` 的长度之和。 - -查询: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM fl_interval GROUP BY id ORDER BY id; -``` - -结果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 3.1 │ Float64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -2. 输入表: - -```text -┌─id─┬───────────────start─┬─────────────────end─┐ -│ a │ 2020-01-01 01:12:30 │ 2020-01-01 02:10:10 │ -│ a │ 2020-01-01 02:05:30 │ 2020-01-01 02:50:31 │ -│ a │ 2020-01-01 03:11:22 │ 2020-01-01 03:23:31 │ -└────┴─────────────────────┴─────────────────────┘ -``` - -在本示例中,使用了 `DateTime` 类型的参数。该函数返回一个以秒为单位的值。 - -查询: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM dt_interval GROUP BY id ORDER BY id; -``` - -结果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 6610 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` - -3. 输入表: - - -```text -┌─id─┬──────开始─┬────────结束─┐ -│ a │ 2020-01-01 │ 2020-01-04 │ -│ a │ 2020-01-12 │ 2020-01-18 │ -└────┴────────────┴────────────┘ -``` - -在本示例中,使用了 `Date` 类型的参数。该函数返回的值以天为单位。 - -查询: - -```sql -SELECT id, intervalLengthSum(start, end), toTypeName(intervalLengthSum(start, end)) FROM date_interval GROUP BY id ORDER BY id; -``` - -结果: - -```text -┌─id─┬─intervalLengthSum(start, end)─┬─toTypeName(intervalLengthSum(start, end))─┐ -│ a │ 9 │ UInt64 │ -└────┴───────────────────────────────┴───────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md deleted file mode 100644 index b8b23386cb7..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -description: '对来自两个总体的样本进行 Kolmogorov-Smirnov 检验。' -sidebar_label: 'kolmogorovSmirnovTest' -sidebar_position: 156 -slug: /sql-reference/aggregate-functions/reference/kolmogorovsmirnovtest -title: 'kolmogorovSmirnovTest' -doc_type: 'reference' ---- - -# kolmogorovSmirnovTest {#kolmogorovsmirnovtest} - -将 Kolmogorov-Smirnov 检验应用于来自两个总体的样本。 - -**语法** - -```sql -kolmogorovSmirnovTest([alternative, computation_method])(sample_data, sample_index) -``` - -两个样本的值都在 `sample_data` 列中。如果 `sample_index` 等于 0,则该行中的值属于第一总体的样本,否则属于第二总体的样本。\ -样本必须来自连续的一维概率分布。 - -**参数** - -* `sample_data` — 样本数据。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — 样本索引。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**设置项** - -* `alternative` — 备择假设。(可选,默认:`'two-sided'`。)[String](../../../sql-reference/data-types/string.md)。\ - 设 F(x) 和 G(x) 分别为第一和第二分布的累积分布函数(CDF)。 - * `'two-sided'`\ - 原假设为样本来自同一分布,例如对所有 x 都有 `F(x) = G(x)`。\ - 备择假设为两个分布并不相同。 - * `'greater'`\ - 原假设为第一样本中的值在随机意义上 *小于* 第二样本中的值,\ - 即第一个分布的 CDF 位于第二个分布之上,因此也在其左侧。\ - 这实际上意味着对所有 x 都有 `F(x) >= G(x)`。在这种情况下,备择假设为至少存在一个 x 使得 `F(x) < G(x)`。 - * `'less'`。\ - 原假设为第一样本中的值在随机意义上 *大于* 第二样本中的值,\ - 即第一个分布的 CDF 位于第二个分布之下,因此也在其右侧。\ - 这实际上意味着对所有 x 都有 `F(x) <= G(x)`。在这种情况下,备择假设为至少存在一个 x 使得 `F(x) > G(x)`。 -* `computation_method` — 用于计算 p-value 的方法。(可选,默认:`'auto'`。)[String](../../../sql-reference/data-types/string.md)。 - * `'exact'` - 使用检验统计量的精确概率分布进行计算。除小样本外,计算开销较大且不划算。 - * `'asymp'` (`'asymptotic'`) - 使用近似方法进行计算。对于大样本,精确与渐近 p-value 非常接近。 - * `'auto'` - 当样本数量的最大值小于 10'000 时使用 `'exact'` 方法。 - -**返回值** - -包含两个元素的 [Tuple](../../../sql-reference/data-types/tuple.md): - -* 计算得到的统计量。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的 p-value。[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -查询: - -```sql -SELECT kolmogorovSmirnovTest('less', 'exact')(value, num) -FROM -( - SELECT - randNormal(0, 10) AS value, - 0 AS num - FROM numbers(10000) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(10000) -) -``` - -结果: - -```text -┌─kolmogorovSmirnovTest('less', 'exact')(value, num)─┐ -│ (0.009899999999999996,0.37528595205132287) │ -└────────────────────────────────────────────────────┘ -``` - -注意: -P 值大于 0.05(对应 95% 的置信水平),因此原假设不被拒绝。 - -查询: - -```sql -SELECT kolmogorovSmirnovTest('two-sided', 'exact')(value, num) -FROM -( - SELECT - randStudentT(10) AS value, - 0 AS num - FROM numbers(100) - UNION ALL - SELECT - randNormal(0, 10) AS value, - 1 AS num - FROM numbers(100) -) -``` - -结果: - -```text -┌─kolmogorovSmirnovTest('two-sided', 'exact')(value, num)─┐ -│ (0.4100000000000002,6.61735760482795e-8) │ -└─────────────────────────────────────────────────────────┘ -``` - -注: -P 值小于 0.05(对应 95% 的置信水平),因此应当拒绝原假设。 - -**另请参阅** - -* [Kolmogorov-Smirnov 检验](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test) \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md deleted file mode 100644 index 1747ca7d857..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: '计算数据序列的峰度。' -sidebar_position: 157 -slug: /sql-reference/aggregate-functions/reference/kurtpop -title: 'kurtPop' -doc_type: 'reference' ---- - -# kurtPop {#kurtpop} - -计算一个序列的[峰度](https://en.wikipedia.org/wiki/Kurtosis)。 - -```sql -kurtPop(expr) -``` - -**参数** - -`expr` — 返回数值的[表达式](/sql-reference/syntax#expressions)。 - -**返回值** - -给定分布的峰度。类型 — [Float64](../../../sql-reference/data-types/float.md) - -**示例** - -```sql -SELECT kurtPop(value) FROM series_with_value_column; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md deleted file mode 100644 index abc3e3644e9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/kurtsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '计算序列的样本峰度。' -sidebar_position: 158 -slug: /sql-reference/aggregate-functions/reference/kurtsamp -title: 'kurtSamp' -doc_type: 'reference' ---- - -# kurtSamp {#kurtsamp} - -计算序列的[样本峰度](https://en.wikipedia.org/wiki/Kurtosis)。 - -当传入的值构成某随机变量的样本时,该函数返回该随机变量峰度的无偏估计值。 - -```sql -kurtSamp(expr) -``` - -**参数** - -`expr` — 返回数字的[表达式](/sql-reference/syntax#expressions)。 - -**返回值** - -给定分布的峰度。返回类型为 [Float64](../../../sql-reference/data-types/float.md)。如果 `n <= 1`(`n` 为样本大小),则函数返回 `nan`。 - -**示例** - -```sql -SELECT kurtSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md deleted file mode 100644 index ac4969e124a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '将 Largest-Triangle-Three-Buckets 算法应用于输入数据。' -sidebar_label: 'largestTriangleThreeBuckets' -sidebar_position: 159 -slug: /sql-reference/aggregate-functions/reference/largestTriangleThreeBuckets -title: 'largestTriangleThreeBuckets' -doc_type: 'reference' ---- - -# largestTriangleThreeBuckets {#largesttrianglethreebuckets} - -将 [Largest-Triangle-Three-Buckets](https://skemman.is/bitstream/1946/15343/3/SS_MSthesis.pdf) 算法应用于输入数据。 -该算法用于对时间序列数据进行降采样,以便进行可视化。它被设计为在按 x 坐标排序的序列上运行。 -其工作方式是将已排序的序列划分为多个桶,然后在每个桶中找到面积最大的三角形。桶的数量等于结果序列中的点数。 -该函数会先按 `x` 对数据进行排序,然后对排序后的数据应用降采样算法。 - -**语法** - -```sql -largestTriangleThreeBuckets(n)(x, y) -``` - -别名:`lttb`。 - -**参数** - -* `x` — x 坐标。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、[Decimal](../../../sql-reference/data-types/decimal.md)、[Date](../../../sql-reference/data-types/date.md)、[Date32](../../../sql-reference/data-types/date32.md)、[DateTime](../../../sql-reference/data-types/datetime.md)、[DateTime64](../../../sql-reference/data-types/datetime64.md)。 -* `y` — y 坐标。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)、[Decimal](../../../sql-reference/data-types/decimal.md)、[Date](../../../sql-reference/data-types/date.md)、[Date32](../../../sql-reference/data-types/date32.md)、[DateTime](../../../sql-reference/data-types/datetime.md)、[DateTime64](../../../sql-reference/data-types/datetime64.md)。 - -在给定序列中,NaN 值会被忽略,即所有 NaN 值都会从分析中排除,从而确保函数仅在有效的数值数据上运行。 - -**参数** - -* `n` — 结果序列中的点数。[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返回值** - -由包含两个元素的 [Tuple](../../../sql-reference/data-types/tuple.md) 组成的 [Array](../../../sql-reference/data-types/array.md): - -**示例** - -输入表: - -```text -┌─────x───────┬───────y──────┐ -│ 1.000000000 │ 10.000000000 │ -│ 2.000000000 │ 20.000000000 │ -│ 3.000000000 │ 15.000000000 │ -│ 8.000000000 │ 60.000000000 │ -│ 9.000000000 │ 55.000000000 │ -│ 10.00000000 │ 70.000000000 │ -│ 4.000000000 │ 30.000000000 │ -│ 5.000000000 │ 40.000000000 │ -│ 6.000000000 │ 35.000000000 │ -│ 7.000000000 │ 50.000000000 │ -└─────────────┴──────────────┘ -``` - -查询语句: - -```sql -SELECT largestTriangleThreeBuckets(4)(x, y) FROM largestTriangleThreeBuckets_test; -``` - -结果: - -```text -┌────────largestTriangleThreeBuckets(4)(x, y)───────────┐ -│ [(1,10),(3,15),(9,55),(10,70)] │ -└───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md deleted file mode 100644 index c6a13ba43a4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/last_value.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: '选择最后出现的值,类似于 `anyLast`,但允许 NULL 值。' -sidebar_position: 160 -slug: /sql-reference/aggregate-functions/reference/last_value -title: 'last_value' -doc_type: 'reference' ---- - -# last_value {#last_value} - -选择最后出现的值,与 `anyLast` 类似,但可以接受 NULL。 -它主要应与[窗口函数](../../window-functions/index.md)一起使用。 -如果不使用窗口函数且源数据流未排序,则结果将是随机的。 - -## 示例 {#examples} - -```sql -CREATE TABLE test_data -( - a Int64, - b Nullable(Int64) -) -ENGINE = Memory; - -INSERT INTO test_data (a, b) VALUES (1,null), (2,3), (4, 5), (6,null) -``` - -### 示例 1 {#example1} - -默认情况下会忽略 NULL 值。 - -```sql -SELECT last_value(b) FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### 示例 2 {#example2} - -NULL 值会被忽略。 - -```sql -SELECT last_value(b) IGNORE NULLS FROM test_data -``` - -```text -┌─last_value_ignore_nulls(b)─┐ -│ 5 │ -└────────────────────────────┘ -``` - -### 示例 3 {#example3} - -接受 NULL 值。 - -```sql -SELECT last_value(b) RESPECT NULLS FROM test_data -``` - -```text -┌─last_value_respect_nulls(b)─┐ -│ ᴺᵁᴸᴸ │ -└─────────────────────────────┘ -``` - -### 示例 4 {#example4} - -使用包含 `ORDER BY` 的子查询来稳定结果。 - -```sql -SELECT - last_value_respect_nulls(b), - last_value(b) -FROM -( - SELECT * - FROM test_data - ORDER BY a ASC -) -``` - -```text -┌─last_value_respect_nulls(b)─┬─last_value(b)─┐ -│ ᴺᵁᴸᴸ │ 5 │ -└─────────────────────────────┴───────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md deleted file mode 100644 index eb810f417aa..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/mannwhitneyutest.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: '对来自两个总体的样本应用 Mann-Whitney U 秩检验。' -sidebar_label: 'mannWhitneyUTest' -sidebar_position: 161 -slug: /sql-reference/aggregate-functions/reference/mannwhitneyutest -title: 'mannWhitneyUTest' -doc_type: 'reference' ---- - -# mannWhitneyUTest {#mannwhitneyutest} - -对来自两个总体的样本进行 Mann-Whitney 秩检验。 - -**语法** - -```sql -mannWhitneyUTest[(alternative[, continuity_correction])](sample_data, sample_index) -``` - -两个样本的值都存储在 `sample_data` 列中。如果 `sample_index` 等于 0,则该行的值属于第一总体的样本;否则,它属于第二总体的样本。 - -原假设是两个总体在随机意义上(分布上)相同,同时也可以检验单侧假设。该检验不要求数据服从正态分布。 - -**参数** - -* `sample_data` — 样本数据。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — 样本索引。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**可选参数** - -* `alternative` — 备择假设。(可选,默认值:`'two-sided'`。)[String](../../../sql-reference/data-types/string.md)。 - * `'two-sided'`; - * `'greater'`; - * `'less'`。 -* `continuity_correction` — 如果不为 0,则在 p 值的正态近似中应用连续性校正。(可选,默认值:1。)[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返回值** - -包含两个元素的 [Tuple](../../../sql-reference/data-types/tuple.md): - -* 计算得到的 U 统计量。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的 p 值。[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -输入表: - -```text -┌─sample_data─┬─sample_index─┐ -│ 10 │ 0 │ -│ 11 │ 0 │ -│ 12 │ 0 │ -│ 1 │ 1 │ -│ 2 │ 1 │ -│ 3 │ 1 │ -└─────────────┴──────────────┘ -``` - -查询: - -```sql -SELECT mannWhitneyUTest('greater')(sample_data, sample_index) FROM mww_ttest; -``` - -结果: - -```text -┌─mannWhitneyUTest('greater')(sample_data, sample_index)─┐ -│ (9,0.04042779918503192) │ -└────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [Mann–Whitney U 检验](https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test) -* [随机序](https://en.wikipedia.org/wiki/Stochastic_ordering) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md deleted file mode 100644 index ea847144733..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/max.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: '用于对一组值计算最大值的聚合函数。' -sidebar_position: 162 -slug: /sql-reference/aggregate-functions/reference/max -title: 'max' -doc_type: 'reference' ---- - -用于对一组值计算最大值的聚合函数。 - -示例: - -```sql -SELECT max(salary) FROM employees; -``` - -```sql -SELECT department, max(salary) FROM employees GROUP BY department; -``` - -如果你需要一个非聚合函数在两个值中取最大值,请参阅 `greatest`: - -```sql -SELECT greatest(a, b) FROM table; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md deleted file mode 100644 index 7bea1a451e9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersections.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: '聚合函数,用于计算一组区间之间的最大相交次数(在所有区间至少有一次共同相交的前提下)。' -sidebar_position: 163 -slug: /sql-reference/aggregate-functions/reference/maxintersections -title: 'maxIntersections' -doc_type: 'reference' ---- - -# maxIntersections {#maxintersections} - -聚合函数,用于在所有区间至少相交一次的前提下,计算一组区间之间的最大相交数量。 - -语法为: - -```sql -maxIntersections(start_column, end_column) -``` - -**参数** - -* `start_column` – 表示每个区间起点的数值列。如果 `start_column` 为 `NULL` 或 0,则跳过该区间。 - -* `end_column` - 表示每个区间终点的数值列。如果 `end_column` 为 `NULL` 或 0,则跳过该区间。 - -**返回值** - -返回最大相交区间数。 - -**示例** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -这些区间如下: - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -这些区间中有三个具有一个共同的取值(该值是 `4`,但具体是哪一个值并不重要,我们关心的是相交次数的计数)。区间 `(1,3)` 和 `(3,7)` 共享一个端点,但在 `maxIntersections` 函数中并不被视为相交。 - -```sql -SELECT maxIntersections(start, end) FROM my_events; -``` - -响应: - -```response -3 -``` - -如果存在多个最大区间,可以使用 [`maxIntersectionsPosition` 函数](./maxintersectionsposition.md) 来确定它们的数量和位置。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md deleted file mode 100644 index 4c50c84f26c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxintersectionsposition.md +++ /dev/null @@ -1,67 +0,0 @@ ---- -description: '计算 maxIntersections 函数各次出现位置的聚合函数。' -sidebar_position: 164 -slug: /sql-reference/aggregate-functions/reference/maxintersectionsposition -title: 'maxIntersectionsPosition' -doc_type: 'reference' ---- - -# maxIntersectionsPosition {#maxintersectionsposition} - -聚合函数,用于计算[`maxIntersections` 函数](./maxintersections.md)结果中各次出现的位置。 - -语法如下所示: - -```sql -maxIntersectionsPosition(start_column, end_column) -``` - -**参数** - -* `start_column` – 表示每个区间起点的数值列。如果 `start_column` 为 `NULL` 或 0,则跳过该区间。 - -* `end_column` - 表示每个区间终点的数值列。如果 `end_column` 为 `NULL` 或 0,则跳过该区间。 - -**返回值** - -返回相交区间数量最大时的起始位置。 - -**示例** - -```sql -CREATE TABLE my_events ( - start UInt32, - end UInt32 -) -ENGINE = MergeTree -ORDER BY tuple(); - -INSERT INTO my_events VALUES - (1, 3), - (1, 6), - (2, 5), - (3, 7); -``` - -这些时间间隔如下: - -```response -1 - 3 -1 - - - - 6 - 2 - - 5 - 3 - - - 7 -``` - -请注意,其中有三个区间都共同包含数值 4,而且这一点是从第二个区间开始的: - -```sql -SELECT maxIntersectionsPosition(start, end) FROM my_events; -``` - -响应: - -```response -2 -``` - -换句话说,`(1,6)` 这一行是 3 个相交区间的起点,而 3 是同时相交的区间的最大数量。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md deleted file mode 100644 index 7a897001a14..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/maxmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '根据 `key` 数组中指定的键,从 `value` 数组中计算最大值。' -sidebar_position: 165 -slug: /sql-reference/aggregate-functions/reference/maxmap -title: 'maxMap' -doc_type: 'reference' ---- - -# maxMap {#maxmap} - -根据 `key` 数组中指定的键,在 `value` 数组中计算对应的最大值。 - -**语法** - -```sql -maxMap(key, value) -``` - -或 - -```sql -maxMap(Tuple(key, value)) -``` - -别名:`maxMappedArrays` - -:::note - -* 传入一个由键数组和值数组组成的元组,与分别传入两个数组(一个键数组和一个值数组)是等价的。 -* 对于每一行参与汇总的数据,`key` 和 `value` 中的元素数量必须相同。 - ::: - -**参数** - -* `key` — 键数组。[Array](../../data-types/array.md)。 -* `value` — 值数组。[Array](../../data-types/array.md)。 - -**返回值** - -* 返回一个由两个数组组成的元组:按排序顺序排列的键,以及为对应键计算得到的值。[Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md))。 - -**示例** - -查询: - -```sql -SELECT maxMap(a, b) -FROM VALUES('a Array(Char), b Array(Int64)', (['x', 'y'], [2, 2]), (['y', 'z'], [3, 1])) -``` - -结果: - -```text -┌─maxMap(a, b)───────────┐ -│ [['x','y','z'],[2,3,1]]│ -└────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md deleted file mode 100644 index b37738ccceb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/meanztest.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '对来自两个总体的样本进行均值 z 检验。' -sidebar_label: 'meanZTest' -sidebar_position: 166 -slug: /sql-reference/aggregate-functions/reference/meanztest -title: 'meanZTest' -doc_type: 'reference' ---- - -# meanZTest {#meanztest} - -对来自两个总体的样本执行均值 Z 检验。 - -**语法** - -```sql -meanZTest(总体方差_x, 总体方差_y, 置信度)(样本数据, 样本索引) -``` - -两个样本的值都在 `sample_data` 列中。如果 `sample_index` 等于 0,则该行的值属于第一个总体的样本;否则,该行的值属于第二个总体的样本。 - -原假设是两个总体的均值相等。假设总体服从正态分布。两个总体的方差可以不相等,且方差是已知的。 - -**参数** - -* `sample_data` — 样本数据。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — 样本索引。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**参数设置** - -* `population_variance_x` — 总体 x 的方差。[Float](../../../sql-reference/data-types/float.md)。 -* `population_variance_y` — 总体 y 的方差。[Float](../../../sql-reference/data-types/float.md)。 -* `confidence_level` — 用于计算置信区间的置信水平。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -包含四个元素的 [Tuple](../../../sql-reference/data-types/tuple.md): - -* 计算得到的 t-统计量。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的 p 值。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间下界。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间上界。[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -输入表: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.9 │ 0 │ -│ 22.1 │ 0 │ -│ 18.9 │ 1 │ -│ 19 │ 1 │ -│ 20.3 │ 1 │ -└─────────────┴──────────────┘ -``` - -查询: - -```sql -SELECT meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index) FROM mean_ztest -``` - -结果: - -```text -┌─meanZTest(0.7, 0.45, 0.95)(sample_data, sample_index)────────────────────────────┐ -│ (3.2841296025548123,0.0010229786769086013,0.8198428246768334,3.2468238419898365) │ -└──────────────────────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md deleted file mode 100644 index 8e9702fb4fc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/median.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -description: '`median*` 函数是对应 `quantile*` 函数的别名。它们用于计算数值型数据样本的中位数。' -sidebar_position: 167 -slug: /sql-reference/aggregate-functions/reference/median -title: 'median' -doc_type: 'reference' ---- - -# median {#median} - -`median*` 函数是对应 `quantile*` 函数的别名。它们用于计算数值型数据样本的中位数。 - -函数: - -* `median` — [quantile](/sql-reference/aggregate-functions/reference/quantile) 的别名。 -* `medianDeterministic` — [quantileDeterministic](/sql-reference/aggregate-functions/reference/quantiledeterministic) 的别名。 -* `medianExact` — [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 的别名。 -* `medianExactWeighted` — [quantileExactWeighted](/sql-reference/aggregate-functions/reference/quantileexactweighted) 的别名。 -* `medianTiming` — [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming) 的别名。 -* `medianTimingWeighted` — [quantileTimingWeighted](/sql-reference/aggregate-functions/reference/quantiletimingweighted) 的别名。 -* `medianTDigest` — [quantileTDigest](/sql-reference/aggregate-functions/reference/quantiletdigest) 的别名。 -* `medianTDigestWeighted` — [quantileTDigestWeighted](/sql-reference/aggregate-functions/reference/quantiletdigestweighted) 的别名。 -* `medianBFloat16` — [quantileBFloat16](/sql-reference/aggregate-functions/reference/quantilebfloat16) 的别名。 -* `medianDD` — [quantileDD](/sql-reference/aggregate-functions/reference/quantileddsketch) 的别名。 - -**示例** - -输入表: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -查询: - -```sql -SELECT medianDeterministic(val, 1) FROM t; -``` - -结果: - -```text -┌─medianDeterministic(val, 1)─┐ -│ 1.5 │ -└─────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md deleted file mode 100644 index 59e50f658c9..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/min.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -description: '计算一组值中最小值的聚合函数。' -sidebar_position: 168 -slug: /sql-reference/aggregate-functions/reference/min -title: 'min' -doc_type: 'reference' ---- - -计算一组值中最小值的聚合函数。 - -示例: - -```sql -SELECT min(salary) FROM employees; -``` - -```sql -SELECT department, min(salary) FROM employees GROUP BY department; -``` - -若需要使用非聚合函数在两个值中取最小值,请参阅 `least`: - -```sql -SELECT least(a, b) FROM table; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md deleted file mode 100644 index 3fc90abf5bb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/minmap.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -description: '根据 `key` 数组中指定的键,从 `value` 数组中计算对应的最小值。' -sidebar_position: 169 -slug: /sql-reference/aggregate-functions/reference/minmap -title: 'minMap' -doc_type: 'reference' ---- - -# minMap {#minmap} - -根据 `key` 数组中指定的键,从 `value` 数组中计算最小值。 - -**语法** - -```sql -`minMap(键, 值)` -``` - -或者 - -```sql -minMap(元组(键, 值)) -``` - -Alias: `minMappedArrays` - -:::note - -* 传递一个由键数组和值数组组成的元组,与分别传递键数组和值数组是等价的。 -* 对于每一行聚合数据,`key` 与 `value` 中的元素数量必须相同。 - ::: - -**参数** - -* `key` — 键的数组。[Array](../../data-types/array.md)。 -* `value` — 值的数组。[Array](../../data-types/array.md)。 - -**返回值** - -* 返回一个包含两个数组的元组:按排序顺序排列的键,以及为相应键计算得到的值。[Tuple](../../data-types/tuple.md)([Array](../../data-types/array.md), [Array](../../data-types/array.md))。 - -**示例** - -查询: - -```sql -SELECT minMap(a, b) -FROM VALUES('a Array(Int32), b Array(Int64)', ([1, 2], [2, 2]), ([2, 3], [1, 1])) -``` - -结果: - -```text -┌─minMap(a, b)──────┐ -│ ([1,2,3],[2,1,1]) │ -└───────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md deleted file mode 100644 index fbf0c41c2d0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantile.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '计算数值数据序列的近似分位数。' -sidebar_position: 170 -slug: /sql-reference/aggregate-functions/reference/quantile -title: 'quantile' -doc_type: 'reference' ---- - -# quantile {#quantile} - -计算数值数据序列的近似[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -此函数对数值序列应用[水库抽样](https://en.wikipedia.org/wiki/Reservoir_sampling),水库大小最多为 8192,并使用随机数生成器进行抽样。结果是非确定性的。若要获得精确分位数,请使用 [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 函数。 - -在查询中对不同分位水平使用多个 `quantile*` 函数时,其内部状态不会被合并(也就是说,该查询的执行效率低于理论最优)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -请注意,对于空的数值序列,`quantile` 将返回 NaN,而其 `quantile*` 变体则可能返回 NaN,或根据具体变体返回该序列类型的默认值。 - -**语法** - -```sql -quantile(level)(expr) -``` - -别名:`median`。 - -**参数** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 的常量浮点数。建议将 `level` 的取值范围设置为 `[0.01, 0.99]`。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 针对列值的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](/sql-reference/data-types/date) 或 [DateTime](/sql-reference/data-types/datetime)。 - -**返回值** - -* 指定级别的近似分位数。 - -类型: - -* 对于数值型输入数据类型,返回 [Float64](/sql-reference/data-types/float)。 -* 如果输入值类型为 `Date`,则返回 [Date](/sql-reference/data-types/date)。 -* 如果输入值类型为 `DateTime`,则返回 [DateTime](/sql-reference/data-types/datetime)。 - -**示例** - -输入表: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -查询: - -```sql -SELECT quantile(val) FROM t -``` - -结果: - -```text -┌─quantile(val)─┐ -│ 1.5 │ -└───────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md deleted file mode 100644 index 0c9b53b7809..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileGK.md +++ /dev/null @@ -1,76 +0,0 @@ ---- -description: '使用 Greenwald-Khanna 算法计算数值序列的分位数。' -sidebar_position: 175 -slug: /sql-reference/aggregate-functions/reference/quantileGK -title: 'quantileGK' -doc_type: 'reference' ---- - -# quantileGK {#quantilegk} - -使用 [Greenwald-Khanna](http://infolab.stanford.edu/~datar/courses/cs361a/papers/quantiles.pdf) 算法计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。Greenwald-Khanna 算法是一种用于在数据流上高效计算分位数的算法,由 Michael Greenwald 和 Sanjeev Khanna 于 2001 年提出。它广泛应用于数据库和大数据系统中,在这些场景下,需要对大规模数据流进行实时且较为精确的分位数计算。该算法在空间复杂度上仅为 O(log n),并且对每个元素的时间复杂度仅为 O(log log n)(其中 n 为输入数据量)。同时,该算法具有很高的精度,能够在高概率下给出近似的分位数值。 - -`quantileGK` 与 ClickHouse 中其他分位数函数不同,因为它允许用户控制近似分位数结果的精度。 - -**语法** - -```sql -quantileGK(accuracy, level)(expr) -``` - -别名:`medianGK`。 - -**参数** - -* `accuracy` — 分位数的精度。为正整数常量。精度值越大,误差越小。例如,如果将精度参数设置为 100,则计算得到的分位数在较高概率下误差不会超过 1%。计算得到的分位数精度与算法的计算复杂度之间存在权衡。更高的精度需要更多内存和计算资源来更准确地计算分位数,而较低的精度参数可以实现更快、更节省内存的计算,但精度会略有下降。 - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 - -* `expr` — 针对列值进行运算的表达式,其结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**返回值** - -* 指定级别和精度的分位数。 - -类型: - -* 对数值型输入数据类型,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值的类型为 `Date`,返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -```sql -SELECT quantileGK(1, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1, 0.25)(plus(number, 1))─┐ -│ 1 │ -└──────────────────────────────────────┘ - -SELECT quantileGK(10, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(10, 0.25)(plus(number, 1))─┐ -│ 156 │ -└───────────────────────────────────────┘ - -SELECT quantileGK(100, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(100, 0.25)(plus(number, 1))─┐ -│ 251 │ -└────────────────────────────────────────┘ - -SELECT quantileGK(1000, 0.25)(number + 1) -FROM numbers(1000) - -┌─quantileGK(1000, 0.25)(plus(number, 1))─┐ -│ 249 │ -└─────────────────────────────────────────┘ -``` - -**另请参见** - -* [median]/sql-reference/aggregate-functions/reference/median -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md deleted file mode 100644 index 7803c74236a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantilebfloat16.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '计算由 bfloat16 数值组成的样本的近似分位数。' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantilebfloat16 -title: 'quantileBFloat16' -doc_type: 'reference' ---- - -# quantileBFloat16Weighted {#quantilebfloat16weighted} - -类似于 `quantileBFloat16`,但会考虑序列中每个成员的权重。 - -对由 [bfloat16](https://en.wikipedia.org/wiki/Bfloat16_floating-point_format) 数值构成的样本计算近似的[分位数](https://en.wikipedia.org/wiki/Quantile)。`bfloat16` 是一种浮点数据类型,具有 1 位符号位、8 位指数位和 7 位尾数位。 -该函数将输入值转换为 32 位浮点数,并取其最高有效 16 位。然后计算 `bfloat16` 分位数值,并通过补零将结果转换为 64 位浮点数。 -该函数是一个快速的分位数估计器,其相对误差不超过 0.390625%。 - -**语法** - -```sql -quantileBFloat16[(level)](expr) -``` - -别名: `medianBFloat16` - -**参数** - -* `expr` — 包含数值数据的列。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)。 - -**参数说明** - -* `level` — 分位数水平。可选。取值范围为 0 到 1。默认值:0.5。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 指定分位数水平的近似值。 - -类型:[Float64](/sql-reference/data-types/float)。 - -**示例** - -输入表包含一个整数列和一个浮点数列: - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -用于计算 0.75 分位数(第三四分位数)的查询: - -```sql -SELECT quantileBFloat16(0.75)(a), quantileBFloat16(0.75)(b) FROM example_table; -``` - -结果: - -```text -┌─quantileBFloat16(0.75)(a)─┬─quantileBFloat16(0.75)(b)─┐ -│ 3 │ 1 │ -└───────────────────────────┴───────────────────────────┘ -``` - -请注意,在转换为 `bfloat16` 时,示例中的所有浮点值都会被截断为 1.0。 - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md deleted file mode 100644 index d2c8e57fa89..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileddsketch.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -description: '计算样本分布的近似分位数,并提供相对误差保证。' -sidebar_position: 171 -slug: /sql-reference/aggregate-functions/reference/quantileddsketch -title: 'quantileDD' -doc_type: 'reference' ---- - -计算样本分布的近似[分位数](https://en.wikipedia.org/wiki/Quantile),并提供相对误差保证。其工作原理是构建一个 [DD](https://www.vldb.org/pvldb/vol12/p2195-masson.pdf)。 - -**语法** - -```sql -quantileDD(relative_accuracy, [level])(expr) -``` - -**参数** - -* `expr` — 包含数值数据的列。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md)。 - -**参数说明** - -* `relative_accuracy` — 分位数的相对精度。取值范围为 0 到 1。[Float](../../../sql-reference/data-types/float.md)。sketch 的大小取决于数据范围和相对精度。范围越大且相对精度越小,sketch 越大。sketch 的大致内存占用为 `log(max_value/min_value)/relative_accuracy`。推荐值为 0.001 或更高。 - -* `level` — 分位数的级别。可选。取值范围为 0 到 1。默认值:0.5。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 指定级别的近似分位数值。 - -类型:[Float64](/sql-reference/data-types/float)。 - -**示例** - -输入表包含一个整数列和一个浮点列: - -```text -┌─a─┬─────b─┐ -│ 1 │ 1.001 │ -│ 2 │ 1.002 │ -│ 3 │ 1.003 │ -│ 4 │ 1.004 │ -└───┴───────┘ -``` - -用于计算 0.75 分位数(第三四分位数)的查询语句: - -```sql -SELECT quantileDD(0.01, 0.75)(a), quantileDD(0.01, 0.75)(b) FROM example_table; -``` - -结果: - -```text -┌─quantileDD(0.01, 0.75)(a)─┬─quantileDD(0.01, 0.75)(b)─┐ -│ 2.974233423476717 │ 1.01 │ -└─────────────────────────────────┴─────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md deleted file mode 100644 index fb94299de94..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiledeterministic.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '计算数值数据序列的近似分位数。' -sidebar_position: 172 -slug: /sql-reference/aggregate-functions/reference/quantiledeterministic -title: 'quantileDeterministic' -doc_type: 'reference' ---- - -# quantileDeterministic {#quantiledeterministic} - -计算数值数据序列的近似[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -此函数使用大小最多为 8192 的水库进行[水库抽样](https://en.wikipedia.org/wiki/Reservoir_sampling),并采用确定性的抽样算法,因此结果是确定的。若要获取精确分位数,请使用 [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 函数。 - -在查询中使用多个具有不同等级的 `quantile*` 函数时,它们的内部状态不会被合并(也就是说,查询的执行效率会低于本可达到的效率)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileDeterministic(level)(expr, determinator) -``` - -Alias: `medianDeterministic`. - -**Arguments** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议使用范围为 `[0.01, 0.99]` 的 `level` 值。默认值:0.5。在 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 作用于列值的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 -* `determinator` — 在水塘抽样算法中,其哈希值用于替代随机数生成器,使抽样结果具有确定性。作为 `determinator`,可以使用任意确定性的正数值,例如用户 ID 或事件 ID。如果同一个 `determinator` 值出现过于频繁,函数将无法正确工作。 - -**Returned value** - -* 指定级别的近似分位数。 - -Type: - -* 对数值数据类型输入返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值为 `Date` 类型,则返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值为 `DateTime` 类型,则返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**Example** - -输入表: - -```text -┌─val─┐ -│ 1 │ -│ 1 │ -│ 2 │ -│ 3 │ -└─────┘ -``` - -查询: - -```sql -SELECT quantileDeterministic(val, 1) FROM t -``` - -结果: - -```text -┌─quantileDeterministic(val, 1)─┐ -│ 1.5 │ -└───────────────────────────────┘ -``` - -**另见** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md deleted file mode 100644 index f5d3f5c841a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexact.md +++ /dev/null @@ -1,293 +0,0 @@ ---- -description: 'quantileExact、quantileExactLow、quantileExactHigh、quantileExactExclusive、 - quantileExactInclusive 函数' -sidebar_position: 173 -slug: /sql-reference/aggregate-functions/reference/quantileexact -title: 'quantileExact 函数' -doc_type: 'reference' ---- - -# quantileExact 精确分位数函数 {#quantileexact-functions} - -## quantileExact {#quantileexact} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了得到精确值,所有传入的值会被合并到一个数组中,然后对该数组进行部分排序。因此,该函数会消耗 `O(n)` 的内存,其中 `n` 是传入值的数量。不过,当值的数量较少时,该函数非常高效。 - -在一个查询中使用多个具有不同分位等级的 `quantile*` 函数时,其内部状态不会被合并(也就是说,该查询的执行效率低于理论上可能达到的效率)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileExact(level)(expr) -``` - -别名:`medianExact`。 - -**参数** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议使用 `[0.01, 0.99]` 范围内的 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 作用于列值的表达式,其结果类型为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**返回值** - -* 指定级别的分位数。 - -类型: - -* 对于数值数据类型,输出格式与输入格式相同。例如: - -```sql - -SELECT - toTypeName(quantileExact(number)) AS `quantile`, - toTypeName(quantileExact(number::Int32)) AS `quantile_int32`, - toTypeName(quantileExact(number::Float32)) AS `quantile_float32`, - toTypeName(quantileExact(number::Float64)) AS `quantile_float64`, - toTypeName(quantileExact(number::Int64)) AS `quantile_int64` -FROM numbers(1) - ┌─quantile─┬─quantile_int32─┬─quantile_float32─┬─quantile_float64─┬─quantile_int64─┐ -1. │ UInt64 │ Int32 │ Float32 │ Float64 │ Int64 │ - └──────────┴────────────────┴──────────────────┴──────────────────┴────────────────┘ - -返回 1 行。耗时: 0.002 秒。 -``` - -* 如果输入值的类型为 `Date`,则为 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,则为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantileExact(number) FROM numbers(10) -``` - -结果: - -```text -┌─quantileExact(number)─┐ -│ 5 │ -└───────────────────────┘ -``` - -## quantileExactLow {#quantileexactlow} - -与 `quantileExact` 类似,此函数计算数值数据序列的精确[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了获得精确值,所有传入的值会先合并为一个数组,然后对该数组进行完全排序。排序[算法](https://en.cppreference.com/w/cpp/algorithm/sort)的复杂度为 `O(N·log(N))`,其中 `N = std::distance(first, last)` 为元素个数。 - -返回值取决于分位数水平(level)和所选元素的数量。也就是说,如果 level 为 0.5,那么当元素个数为偶数时,函数返回较低的中位数值,当元素个数为奇数时返回中间的中位数值。中位数的计算方式与 Python 中使用的 [median_low](https://docs.python.org/3/library/statistics.html#statistics.median_low) 实现类似。 - -对于其他任意 level,会返回索引为 `level * size_of_array` 的元素。例如: - -```sql -SELECT quantileExactLow(0.1)(number) FROM numbers(10) - -┌─quantileExactLow(0.1)(number)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -在查询中使用多个分位数等级不同的 `quantile*` 函数时,其内部状态不会被合并(也就是说,查询的执行效率会低于理想水平)。在这种情况下,请使用 [quantiles](/sql-reference/aggregate-functions/reference/quantiles) 函数。 - -**语法** - -```sql -quantileExactLow(level)(expr) -``` - -别名:`medianExactLow`。 - -**参数** - -* `level` — 分位水平。可选参数。取值为 0 到 1 之间的常量浮点数。推荐在 `[0.01, 0.99]` 范围内使用 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 对列值进行计算的表达式,结果为数值[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**返回值** - -* 指定分位水平的分位数。 - -类型: - -* 对数值数据类型输入返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值的类型为 `Date`,则返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,则返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantileExactLow(number) FROM numbers(10) -``` - -结果: - -```text -┌─quantileExactLow(number)─┐ -│ 4 │ -└──────────────────────────┘ -``` - -## quantileExactHigh {#quantileexacthigh} - -与 `quantileExact` 类似,该函数计算数值数据序列的精确[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -所有传入的值会合并到一个数组中,然后对该数组进行完全排序,以获得精确值。排序[算法](https://en.cppreference.com/w/cpp/algorithm/sort)的复杂度为 `O(N·log(N))`,其中 `N = std::distance(first, last)` 为比较次数。 - -返回值取决于分位数水平(level)以及选定元素的数量。也就是说,如果 level 为 0.5,则当元素个数为偶数时,函数返回偏高的中位数值,当元素个数为奇数时,函数返回正中的中位数值。中位数的计算方式与 Python 中使用的 [median_high](https://docs.python.org/3/library/statistics.html#statistics.median_high) 实现类似。对于其他所有 level,返回索引为 `level * size_of_array` 对应位置的元素。 - -该实现与当前的 `quantileExact` 实现的行为完全一致。 - -在一个查询中使用多个带有不同 level 的 `quantile*` 函数时,其内部状态不会被合并(也就是说,该查询的执行效率低于理论最优)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileExactHigh(level)(expr) -``` - -别名:`medianExactHigh`。 - -**参数** - -* `level` — 分位数水平。可选参数,从 0 到 1 的常量浮点数。建议在 `[0.01, 0.99]` 范围内使用 `level` 值。默认值:0.5。当 `level = 0.5` 时,该函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 针对列值的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**返回值** - -* 指定水平的分位数。 - -类型: - -* 数值数据类型输入时返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 输入值为 `Date` 类型时返回 [Date](../../../sql-reference/data-types/date.md)。 -* 输入值为 `DateTime` 类型时返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantileExactHigh(number) FROM numbers(10) -``` - -结果: - -```text -┌─quantileExactHigh(number)─┐ -│ 5 │ -└───────────────────────────┘ -``` - -## quantileExactExclusive {#quantileexactexclusive} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了获得精确值,所有传入的值会被合并到一个数组中,并对该数组进行部分排序。因此,该函数会消耗 `O(n)` 的内存,其中 `n` 为传入值的数量。不过,在数据量较小时,该函数非常高效。 - -此函数等价于 Excel 中的 [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) 函数([R6 类型](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))。 - -在同一个查询中以不同分位点多次使用 `quantileExactExclusive` 函数时,其内部状态不会被合并(也就是说,该查询的执行效率会低于本可以达到的最优效率)。在这种情况下,请使用 [quantilesExactExclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactexclusive) 函数。 - -**语法** - -```sql -quantileExactExclusive(level)(expr) -``` - -**参数** - -* `expr` — 作用于列值的表达式,结果为数值型 [data types](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**参数说明** - -* `level` — 分位数级别。可选。取值范围:(0, 1) — 不包含边界。默认值:0.5。当 `level=0.5` 时,该函数计算[中位数](https://en.wikipedia.org/wiki/Median)。类型为 [Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 指定级别的分位数。 - -类型: - -* 对于数值型输入,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值类型为 `Date`,返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值类型为 `DateTime`,返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactExclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -结果: - -```text -┌─quantileExactExclusive(0.6)(x)─┐ -│ 599.6 │ -└────────────────────────────────┘ -``` - -## quantileExactInclusive {#quantileexactinclusive} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了得到精确值,所有传入的值会被收集到一个数组中,然后对该数组进行部分排序。因此,函数会消耗 `O(n)` 的内存,其中 `n` 是传入值的个数。不过,当值的数量较少时,该函数仍然非常高效。 - -此函数等价于 Excel 函数 [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed)([R7 类型](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))。 - -在同一个查询中使用多个具有不同 level 的 `quantileExactInclusive` 函数时,其内部状态不会被合并(即查询的执行效率低于理论上的最优)。在这种情况下,请使用 [quantilesExactInclusive](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantilesexactinclusive) 函数。 - -**语法** - -```sql -quantileExactInclusive(level)(expr) -``` - -**参数** - -* `expr` — 作用于列值的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**参数说明** - -* `level` — 分位数级别。可选。取值范围为 [0, 1](包含边界)。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。类型为 [Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 指定级别的分位数。 - -类型: - -* 对于数值数据类型输入,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值的类型为 `Date`,则返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,则返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantileExactInclusive(0.6)(x) FROM (SELECT number AS x FROM num); -``` - -结果: - -```text -┌─quantileExactInclusive(0.6)(x)─┐ -│ 599.4 │ -└────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md deleted file mode 100644 index df901b96920..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweighted.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '精确计算数值数据序列的分位数,并考虑每个元素的权重。' -sidebar_position: 174 -slug: /sql-reference/aggregate-functions/reference/quantileexactweighted -title: 'quantileExactWeighted' -doc_type: 'reference' ---- - -# quantileExactWeighted {#quantileexactweighted} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile),并考虑每个元素的权重。 - -为得到精确结果,会将传入的所有数值合并为一个数组,然后对该数组进行部分排序。每个值会按其权重进行计数,就好像它重复出现了 `weight` 次一样。算法中使用了哈希表。因此,如果传入的数值频繁重复,则该函数比 [quantileExact](/sql-reference/aggregate-functions/reference/quantileexact#quantileexact) 消耗更少的 RAM。可以使用该函数替代 `quantileExact`,并将权重设为 1。 - -在查询中使用多个具有不同 level 的 `quantile*` 函数时,其内部状态不会被合并(也就是说,查询的执行效率低于理论上可能达到的效率)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -加权精确分位数(level)(expr, weight) -``` - -别名:`medianExactWeighted`。 - -**参数** - -* `level` — 分位数水平。可选参数。取值为 0 到 1 的常量浮点数。建议在 `[0.01, 0.99]` 范围内使用 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 在列值上进行计算的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 -* `weight` — 序列成员的权重列。权重表示该值出现的次数,使用[无符号整数类型](../../../sql-reference/data-types/int-uint.md)表示。 - -**返回值** - -* 指定水平的分位数。 - -类型: - -* 数值数据类型输入时为 [Float64](../../../sql-reference/data-types/float.md)。 -* 输入值为 `Date` 类型时为 [Date](../../../sql-reference/data-types/date.md)。 -* 输入值为 `DateTime` 类型时为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -输入表: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -查询: - -```sql -SELECT quantileExactWeighted(n, val) FROM t -``` - -结果: - -```text -┌─quantileExactWeighted(n, val)─┐ -│ 1 │ -└───────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md deleted file mode 100644 index 09260f24ce5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileexactweightedinterpolated.md +++ /dev/null @@ -1,78 +0,0 @@ ---- -description: '采用线性插值方法并考虑每个元素的权重,计算数值数据序列的分位数。' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileExactWeightedInterpolated -title: 'quantileExactWeightedInterpolated' -doc_type: 'reference' ---- - -# quantileExactWeightedInterpolated {#quantileexactweightedinterpolated} - -使用线性插值计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile),并考虑每个元素的权重。 - -为了得到插值结果,首先将所有传入的数值组合成一个数组,然后按照其对应的权重进行排序。接着使用[加权百分位数方法](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method)进行分位数插值:根据权重构建累积分布,然后利用权重和值进行线性插值来计算分位数。 - -在查询中使用多个不同分位点的 `quantile*` 函数时,内部状态不会被合并(也就是说,该查询的执行效率会低于理论上的最优情况)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -我们强烈建议使用 `quantileExactWeightedInterpolated` 代替 `quantileInterpolatedWeighted`,因为 `quantileExactWeightedInterpolated` 比 `quantileInterpolatedWeighted` 更精确。下面是一个示例: - -```sql -SELECT - quantileExactWeightedInterpolated(0.99)(number, 1), - quantile(0.99)(number), - quantileInterpolatedWeighted(0.99)(number, 1) -FROM numbers(9) -┌─quantileExactWeightedInterpolated(0.99)(number, 1)─┬─quantile(0.99)(number)─┬─quantileInterpolatedWeighted(0.99)(number, 1)─┐ -│ 7.92 │ 7.92 │ 8 │ -└────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────────────────────────┘ -``` - -**语法** - -```sql -quantileExactWeightedInterpolated(level)(expr, weight) -``` - -别名:`medianExactWeightedInterpolated`。 - -**参数** - -* `level` — 分位数级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议使用 `[0.01, 0.99]` 范围内的 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 作用于列值的表达式,结果类型为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 -* `weight` — 包含序列成员权重的列。权重表示该值的出现次数,使用[无符号整数类型](../../../sql-reference/data-types/int-uint.md)表示。 - -**返回值** - -* 指定级别的分位数。 - -类型: - -* 数值数据类型输入时为 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值为 `Date` 类型,则为 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值为 `DateTime` 类型,则为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -输入表: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -结果: - -```text -┌─quantileExactWeightedInterpolated(n, val)─┐ -│ 1.5 │ -└───────────────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md deleted file mode 100644 index 9b1f0168ce0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileinterpolatedweighted.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -description: '使用线性插值法在数值数据序列上计算分位数,并考虑每个元素的权重。' -sidebar_position: 176 -slug: /sql-reference/aggregate-functions/reference/quantileInterpolatedWeighted -title: 'quantileInterpolatedWeighted' -doc_type: 'reference' ---- - -# quantileInterpolatedWeighted {#quantileinterpolatedweighted} - -使用线性插值计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile),并考虑每个元素的权重。 - -为了得到插值结果,首先将传入的所有值组合成一个数组,然后按照它们对应的权重进行排序。接着使用[加权百分位方法](https://en.wikipedia.org/wiki/Percentile#The_weighted_percentile_method)进行分位数插值:基于权重构建累积分布,再利用权重和值进行线性插值来计算分位数。 - -在查询中使用多个具有不同级别的 `quantile*` 函数时,它们的内部状态不会被合并(也就是说,该查询的执行效率会低于本可达到的最优效果)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileInterpolatedWeighted(级别)(表达式, 权重) -``` - -别名:`medianInterpolatedWeighted`。 - -**参数** - -* `level` — 分位水平。可选参数,为 0 到 1 之间的常量浮点数。建议在 `[0.01, 0.99]` 范围内选择 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 基于列值计算的表达式,其结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 -* `weight` — 包含序列成员权重的列。权重表示该值出现的次数。 - -**返回值** - -* 指定水平的分位数。 - -类型: - -* 对于数值型输入数据类型,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值的类型为 `Date`,返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -输入表: - -```text -┌─n─┬─val─┐ -│ 0 │ 3 │ -│ 1 │ 2 │ -│ 2 │ 1 │ -│ 5 │ 4 │ -└───┴─────┘ -``` - -查询: - -```sql -SELECT quantileInterpolatedWeighted(n, val) FROM t -``` - -结果: - -```text -┌─quantileInterpolatedWeighted(n, val)─┐ -│ 1 │ -└──────────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md deleted file mode 100644 index 70e341dd23e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantileprometheushistogram.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '使用线性插值计算直方图的分位数。' -sidebar_position: 364 -slug: /sql-reference/aggregate-functions/reference/quantilePrometheusHistogram -title: 'quantilePrometheusHistogram' -doc_type: 'reference' ---- - -# quantilePrometheusHistogram {#quantileprometheushistogram} - -使用线性插值计算直方图的[分位数](https://en.wikipedia.org/wiki/Quantile),同时考虑每个直方图桶的累积值和上界。 - -为了获得插值结果,所有传入的值会被合并成一个数组,然后按照其对应桶的上界值进行排序。之后会类似 PromQL 中针对经典直方图的 [histogram_quantile()](https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile) 函数执行分位数插值:在确定分位数所在的桶后,使用该桶的下界和上界进行线性插值。 - -**语法** - -```sql -quantilePrometheusHistogram(level)(bucket_upper_bound, cumulative_bucket_value) -``` - -**参数** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议将 `level` 值设置在 `[0.01, 0.99]` 范围内。默认值:`0.5`。在 `level=0.5` 时,该函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 - -* `bucket_upper_bound` — 直方图桶的上限。 - - * 最高桶的上限必须为 `+Inf`。 - -* `cumulative_bucket_value` — 直方图桶的累积 [UInt](../../../sql-reference/data-types/int-uint) 或 [Float64](../../../sql-reference/data-types/float.md) 值。 - - * 随着桶上限的增大,这些值必须单调递增。 - -**返回值** - -* 指定级别的分位数。 - -类型: - -* `Float64`。 - -**示例** - -输入表: - -```text - ┌─bucket_upper_bound─┬─cumulative_bucket_value─┐ -1. │ 0 │ 6 │ -2. │ 0.5 │ 11 │ -3. │ 1 │ 14 │ -4. │ inf │ 19 │ - └────────────────────┴─────────────────────────┘ -``` - -结果: - -```text - ┌─quantilePrometheusHistogram(bucket_upper_bound, cumulative_bucket_value)─┐ -1. │ 0.35 │ - └──────────────────────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md deleted file mode 100644 index 1a195eadacc..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiles.md +++ /dev/null @@ -1,180 +0,0 @@ ---- -description: 'quantiles, quantilesExactExclusive, quantilesExactInclusive, quantilesGK' -sidebar_position: 177 -slug: /sql-reference/aggregate-functions/reference/quantiles -title: 'quantiles 函数' -doc_type: 'reference' ---- - - - -# 分位数函数 {#quantiles-functions} - - - -## quantiles {#quantiles} - -语法:`quantiles(level1, level2, ...)(x)` - -所有分位数函数也都有对应的 `quantiles` 系列函数:`quantiles`、`quantilesDeterministic`、`quantilesTiming`、`quantilesTimingWeighted`、`quantilesExact`、`quantilesExactWeighted`、`quantileExactWeightedInterpolated`、`quantileInterpolatedWeighted`、`quantilesTDigest`、`quantilesBFloat16`、`quantilesDD`。这些函数在一次遍历中计算出所列各个分位水平的所有分位数,并返回包含结果值的数组。 - - - -## quantilesExactExclusive {#quantilesexactexclusive} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了获得精确值,所有传入的值会被合并为一个数组,然后对该数组进行部分排序。因此,该函数会消耗 `O(n)` 的内存,其中 `n` 是传入值的数量。不过,在数据量较小时,该函数非常高效。 - -此函数等价于 Excel 中的 [PERCENTILE.EXC](https://support.microsoft.com/en-us/office/percentile-exc-function-bbaa7204-e9e1-4010-85bf-c31dc5dce4ba) 函数([R6 类型](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))。 - -在处理一组分位等级(levels)时,相比于 [quantileExactExclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactexclusive),本函数效率更高。 - -**语法** - -```sql -quantilesExactExclusive(level1, level2, ...)(expr) -``` - -**参数** - -* `expr` — 对列值进行计算的表达式,结果为数值型 [数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**参数说明** - -* `level` — 分位数的水平。可取值范围为 (0, 1),不包含端点。类型为 [Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 由指定水平的分位数组成的 [Array](../../../sql-reference/data-types/array.md)。 - -数组元素类型: - -* 对于数值型输入,数组元素类型为 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值类型为 `Date`,则为 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值类型为 `DateTime`,则为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -结果: - -```text -┌─quantilesExactExclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.25,499.5,749.75,899.9,949.9499999999999,989.99,998.999] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesExactInclusive {#quantilesexactinclusive} - -精确计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -为了获得精确值,所有传入的值会被合并到一个数组中,然后对该数组进行部分排序。因此,该函数会消耗 `O(n)` 的内存,其中 `n` 是传入值的数量。不过,当传入值数量较少时,该函数非常高效。 - -此函数等价于 Excel 函数 [PERCENTILE.INC](https://support.microsoft.com/en-us/office/percentile-inc-function-680f9539-45eb-410b-9a5e-c1355e5fe2ed)([R7 类型](https://en.wikipedia.org/wiki/Quantile#Estimating_quantiles_from_a_sample))。 - -在处理一组分位等级时,比 [quantileExactInclusive](../../../sql-reference/aggregate-functions/reference/quantileexact.md#quantileexactinclusive) 更高效。 - -**语法** - -```sql -quantilesExactInclusive(level1, level2, ...)(expr) -``` - -**参数说明** - -* `expr` — 作用于列值的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**参数** - -* `level` — 分位数的水平。可选值:[0, 1](包含边界)。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -* 一个包含指定水平分位数的 [Array](../../../sql-reference/data-types/array.md)。 - -数组元素类型: - -* 数值型输入时为 [Float64](../../../sql-reference/data-types/float.md)。 -* 输入值为 `Date` 类型时为 [Date](../../../sql-reference/data-types/date.md)。 -* 输入值为 `DateTime` 类型时为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE num AS numbers(1000); - -SELECT quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x) FROM (SELECT number AS x FROM num); -``` - -结果: - -```text -┌─quantilesExactInclusive(0.25, 0.5, 0.75, 0.9, 0.95, 0.99, 0.999)(x)─┐ -│ [249.75,499.5,749.25,899.1,949.05,989.01,998.001] │ -└─────────────────────────────────────────────────────────────────────┘ -``` - - -## quantilesGK {#quantilesgk} - -`quantilesGK` 的工作方式与 `quantileGK` 类似,但它允许我们同时计算多个不同分位点的数值,并返回一个数组。 - -**语法** - -```sql -quantilesGK(accuracy, level1, level2, ...)(expr) -``` - -**返回值** - -* 包含指定分位水平的[数组](../../../sql-reference/data-types/array.md)。 - -数组元素类型: - -* 数值类型输入时为 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值为 `Date` 类型,则为 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值为 `DateTime` 类型,则为 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantilesGK(1, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [1,1,1] │ -└──────────────────────────────────────────────────┘ - -SELECT quantilesGK(10, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(10, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [156,413,659] │ -└───────────────────────────────────────────────────┘ -SELECT quantilesGK(100, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(100, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [251,498,741] │ -└────────────────────────────────────────────────────┘ - -SELECT quantilesGK(1000, 0.25, 0.5, 0.75)(number + 1) -FROM numbers(1000) - -┌─quantilesGK(1000, 0.25, 0.5, 0.75)(plus(number, 1))─┐ -│ [249,499,749] │ -└─────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md deleted file mode 100644 index dfe33399593..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigest.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: '使用 T-Digest 算法计算数值数据序列的近似分位数。' -sidebar_position: 178 -slug: /sql-reference/aggregate-functions/reference/quantiletdigest -title: 'quantileTDigest' -doc_type: 'reference' ---- - -# quantileTDigest {#quantiletdigest} - -使用 [t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf) 算法,对数值数据序列计算近似的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -内存消耗为 `log(n)`,其中 `n` 为值的数量。结果取决于查询的执行顺序,因此是非确定性的。 - -该函数的性能低于 [quantile](/sql-reference/aggregate-functions/reference/quantile) 或 [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming)。从状态大小与精度之间的比率来看,该函数要比 `quantile` 好得多。 - -在一个查询中使用多个具有不同 level 的 `quantile*` 函数时,其内部状态不会被合并(也就是说,查询的执行效率低于理论可达的效率)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileTDigest(level)(expr) -``` - -别名:`medianTDigest`。 - -**参数** - -* `level` — 分位数水平。可选参数。取值为 0 到 1 之间的常量浮点数。建议将 `level` 设置在 `[0.01, 0.99]` 范围内。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 基于列值的表达式,其结果为数值[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**返回值** - -* 给定水平的近似分位数。 - -类型: - -* 对于数值数据类型输入,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值为 `Date` 类型,则返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值为 `DateTime` 类型,则返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantileTDigest(number) FROM numbers(10) -``` - -结果: - -```text -┌─quantileTDigest(number)─┐ -│ 4.5 │ -└─────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](/sql-reference/aggregate-functions/reference/quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md deleted file mode 100644 index 83944a5be63..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletdigestweighted.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: '使用 t-digest 算法计算数值数据序列的近似分位数。' -sidebar_position: 179 -slug: /sql-reference/aggregate-functions/reference/quantiletdigestweighted -title: 'quantileTDigestWeighted' -doc_type: 'reference' ---- - -# quantileTDigestWeighted {#quantiletdigestweighted} - -使用 [t-digest](https://github.com/tdunning/t-digest/blob/master/docs/t-digest-paper/histo.pdf) 算法计算数值数据序列的近似[分位数](https://en.wikipedia.org/wiki/Quantile)。该函数会考虑序列中每个元素的权重。最大误差为 1%。内存消耗为 `log(n)`,其中 `n` 为数值的个数。 - -该函数的性能低于 [quantile](/sql-reference/aggregate-functions/reference/quantile) 或 [quantileTiming](/sql-reference/aggregate-functions/reference/quantiletiming)。从 State 状态大小与精度之间的比率来看,此函数相比 `quantile` 要好得多。 - -结果取决于查询的执行顺序,且是非确定性的。 - -在一个查询中使用多个不同分位水平的 `quantile*` 函数时,其内部状态不会合并(即查询的执行效率低于理论最优)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -:::note\ -[不推荐在极小数据集上使用 `quantileTDigestWeighted`](https://github.com/tdunning/t-digest/issues/167#issuecomment-828650275),否则可能产生显著误差。在这种情况下,请考虑改用 [`quantileTDigest`](../../../sql-reference/aggregate-functions/reference/quantiletdigest.md)。 -::: - -**语法** - -```sql -quantileTDigestWeighted(level)(expr, weight) -``` - -别名:`medianTDigestWeighted`。 - -**参数** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议在 `[0.01, 0.99]` 范围内使用 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 -* `expr` — 针对列值计算的表达式,结果为数值型[数据类型](/sql-reference/data-types)、[Date](../../../sql-reference/data-types/date.md) 或 [DateTime](../../../sql-reference/data-types/datetime.md)。 -* `weight` — 存储序列元素权重的列。权重表示每个值出现的次数。 - -**返回值** - -* 指定级别的近似分位数。 - -类型: - -* 对数值数据类型输入,返回 [Float64](../../../sql-reference/data-types/float.md)。 -* 如果输入值的类型为 `Date`,返回 [Date](../../../sql-reference/data-types/date.md)。 -* 如果输入值的类型为 `DateTime`,返回 [DateTime](../../../sql-reference/data-types/datetime.md)。 - -**示例** - -查询: - -```sql -SELECT quantileTDigestWeighted(number, 1) FROM numbers(10) -``` - -结果: - -```text -┌─quantileTDigestWeighted(number, 1)─┐ -│ 4.5 │ -└────────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md deleted file mode 100644 index 91f3e734fbf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletiming.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -description: '按预定精度计算数值序列的分位数。' -sidebar_position: 180 -slug: /sql-reference/aggregate-functions/reference/quantiletiming -title: 'quantileTiming' -doc_type: 'reference' ---- - -# quantileTiming {#quantiletiming} - -在给定精度下计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -结果是确定性的(不依赖于查询的处理顺序)。该函数针对处理描述分布的序列进行了优化,例如网页加载时间或后端响应时间。 - -在同一个查询中使用多个具有不同级别的 `quantile*` 函数时,它们的内部状态不会被合并(也就是说,查询的执行效率低于理论上的最优)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileTiming(level)(expr) -``` - -Alias: `medianTiming`. - -**参数** - -* `level` — 分位数的级别。可选参数。取值为 0 到 1 之间的常量浮点数。建议使用位于 `[0.01, 0.99]` 范围内的 `level` 值。默认值:0.5。在 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 - -* `expr` — 针对列值计算的[表达式](/sql-reference/syntax#expressions),返回一个 [Float*](../../../sql-reference/data-types/float.md) 类型的数值。 - - * 如果向函数传入负值,其行为未定义。 - * 如果值大于 30,000(页面加载时间超过 30 秒),则按 30,000 处理。 - -**精度** - -在以下情况下,计算结果是精确的: - -* 值的总数不超过 5670。 -* 值的总数超过 5670,但页面加载时间小于 1024 ms。 - -否则,计算结果会被四舍五入到最接近的 16 ms 的倍数。 - -:::note\ -对于页面加载时间分位数的计算,此函数比 [quantile](/sql-reference/aggregate-functions/reference/quantile) 更高效且更精确。 -::: - -**返回值** - -* 指定级别的分位数。 - -类型:`Float32`。 - -:::note\ -如果未向函数传入任何值(在使用 `quantileTimingIf` 时),将返回 [NaN](/sql-reference/data-types/float#nan-and-inf)。这样做的目的是将这些情况与结果为零的情况区分开来。关于对 `NaN` 值进行排序的说明,请参见 [ORDER BY 子句](/sql-reference/statements/select/order-by)。 -::: - -**示例** - -输入表: - -```text -┌─response_time─┐ -│ 72 │ -│ 112 │ -│ 126 │ -│ 145 │ -│ 104 │ -│ 242 │ -│ 313 │ -│ 168 │ -│ 108 │ -└───────────────┘ -``` - -查询: - -```sql -SELECT quantileTiming(response_time) FROM t -``` - -结果: - -```text -┌─quantileTiming(response_time)─┐ -│ 126 │ -└───────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md deleted file mode 100644 index f3c4e2fda9c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/quantiletimingweighted.md +++ /dev/null @@ -1,124 +0,0 @@ ---- -description: '在指定精度下,根据每个序列成员的权重计算数值数据序列的分位数。' -sidebar_position: 181 -slug: /sql-reference/aggregate-functions/reference/quantiletimingweighted -title: 'quantileTimingWeighted' -doc_type: 'reference' ---- - -# quantileTimingWeighted {#quantiletimingweighted} - -在给定精度下,根据序列中每个元素的权重,计算数值数据序列的[分位数](https://en.wikipedia.org/wiki/Quantile)。 - -结果是确定的(不依赖于查询的处理顺序)。该函数针对描述分布的序列进行了优化,例如网页加载时间或后端响应时间。 - -在一个查询中使用多个具有不同分位水平的 `quantile*` 函数时,其内部状态不会被合并(也就是说,该查询的执行效率会低于理论最优)。在这种情况下,请使用 [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) 函数。 - -**语法** - -```sql -quantileTimingWeighted(level)(expr, weight) -``` - -别名:`medianTimingWeighted`。 - -**参数** - -* `level` — 分位数的级别。可选参数。0 到 1 之间的常量浮点数。建议使用 `[0.01, 0.99]` 范围内的 `level` 值。默认值:0.5。当 `level=0.5` 时,函数计算[中位数](https://en.wikipedia.org/wiki/Median)。 - -* `expr` — 针对列值的[表达式](/sql-reference/syntax#expressions),返回 [Float*](../../../sql-reference/data-types/float.md) 类型的数值。 - - * 如果向函数传递负值,其行为未定义。 - * 如果值大于 30,000(页面加载时间超过 30 秒),则将其视为 30,000。 - -* `weight` — 包含序列元素权重的列。权重是该值出现的次数。 - -**精度** - -在以下情况下,计算是精确的: - -* 值的总数量不超过 5670。 -* 值的总数量超过 5670,但页面加载时间小于 1024 毫秒。 - -否则,计算结果将四舍五入到最接近的 16 毫秒的倍数。 - -:::note\ -对于页面加载时间分位数的计算,此函数比 [quantile](/sql-reference/aggregate-functions/reference/quantile) 更高效且更精确。 -::: - -**返回值** - -* 指定级别的分位数。 - -类型:`Float32`。 - -:::note\ -如果没有向函数传递任何值(在使用 `quantileTimingIf` 时),则返回 [NaN](/sql-reference/data-types/float#nan-and-inf)。这样做的目的是将这些情况与结果为零的情况区分开来。有关 `NaN` 值排序的说明,请参阅 [ORDER BY 子句](/sql-reference/statements/select/order-by)。 -::: - -**示例** - -输入表: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -查询: - -```sql -SELECT quantileTimingWeighted(response_time, weight) FROM t -``` - -结果: - -```text -┌─quantileTimingWeighted(response_time, weight)─┐ -│ 112 │ -└───────────────────────────────────────────────┘ -``` - -# quantilesTimingWeighted {#quantilestimingweighted} - -与 `quantileTimingWeighted` 相同,但接受多个带有分位数水平的参数,并返回一个数组,其中包含这些分位数对应的多个值。 - -**示例** - -输入表: - -```text -┌─response_time─┬─weight─┐ -│ 68 │ 1 │ -│ 104 │ 2 │ -│ 112 │ 3 │ -│ 126 │ 2 │ -│ 138 │ 1 │ -│ 162 │ 1 │ -└───────────────┴────────┘ -``` - -查询: - -```sql -SELECT quantilesTimingWeighted(0,5, 0.99)(response_time, weight) FROM t -``` - -结果: - -```text -┌─quantilesTimingWeighted(0.5, 0.99)(response_time, weight)─┐ -│ [112,162] │ -└───────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [median](/sql-reference/aggregate-functions/reference/median) -* [quantiles](../../../sql-reference/aggregate-functions/reference/quantiles.md#quantiles) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md deleted file mode 100644 index 994b3420536..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/rankCorr.md +++ /dev/null @@ -1,62 +0,0 @@ ---- -description: '计算秩相关系数。' -sidebar_position: 182 -slug: /sql-reference/aggregate-functions/reference/rankCorr -title: 'rankCorr' -doc_type: 'reference' ---- - -# rankCorr {#rankcorr} - -计算秩相关系数。 - -**语法** - -```sql -rankCorr(x, y) -``` - -**参数** - -* `x` — 任意值。[Float32](/sql-reference/data-types/float) 或 [Float64](/sql-reference/data-types/float)。 -* `y` — 任意值。[Float32](/sql-reference/data-types/float) 或 [Float64](/sql-reference/data-types/float)。 - -**返回值** - -* 返回 `x` 和 `y` 的秩之间的秩相关系数。相关系数的取值范围为 -1 到 +1。如果传入的参数少于两个,函数将抛出异常。值接近 +1 表示具有很强的线性关系,当一个随机变量增加时,另一个随机变量也随之增加。值接近 -1 表示具有很强的线性关系,当一个随机变量增加时,另一个随机变量随之减小。值接近或等于 0 表示这两个随机变量之间不存在相关关系。 - -类型:[Float64](/sql-reference/data-types/float)。 - -**示例** - -查询: - -```sql -SELECT rankCorr(number, number) FROM numbers(100); -``` - -结果: - -```text -┌─rankCorr(number, number)─┐ -│ 1 │ -└──────────────────────────┘ -``` - -查询: - -```sql -SELECT roundBankers(rankCorr(exp(number), sin(number)), 3) FROM numbers(100); -``` - -结果: - -```text -┌─roundBankers(rankCorr(exp(number), sin(number)), 3)─┐ -│ -0.037 │ -└─────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [斯皮尔曼秩相关系数](https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md deleted file mode 100644 index 610076b1bb5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/simplelinearregression.md +++ /dev/null @@ -1,46 +0,0 @@ ---- -description: '执行简单(单变量)线性回归。' -sidebar_position: 183 -slug: /sql-reference/aggregate-functions/reference/simplelinearregression -title: 'simpleLinearRegression' -doc_type: 'reference' ---- - -# simpleLinearRegression {#simplelinearregression} - -执行简单(一维)线性回归。 - -```sql -简单线性回归(x, y) -``` - -参数: - -* `x` — 自变量取值所在的列。 -* `y` — 因变量取值所在的列。 - -返回值: - -拟合直线 `y = k*x + b` 的常数 `(k, b)`。 - -**示例** - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [0, 1, 2, 3])─┐ -│ (1,0) │ -└───────────────────────────────────────────────────────────────────┘ -``` - -```sql -SELECT arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6]) -``` - -```text -┌─arrayReduce('simpleLinearRegression', [0, 1, 2, 3], [3, 4, 5, 6])─┐ -│ (1,3) │ -└───────────────────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md deleted file mode 100644 index 752e80a1935..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/singlevalueornull.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '聚合函数 `singleValueOrNull` 用于实现子查询运算符,例如 `x = ALL (SELECT ...)`。它用于检查数据中是否存在且仅存在一个唯一的非 NULL 值。' -sidebar_position: 184 -slug: /sql-reference/aggregate-functions/reference/singlevalueornull -title: 'singleValueOrNull' -doc_type: 'reference' ---- - -# singleValueOrNull {#singlevalueornull} - -聚合函数 `singleValueOrNull` 用于实现子查询运算符,例如 `x = ALL (SELECT ...)`。它会检查数据中是否恰好只有一个唯一且非 NULL 的值。 -如果只有一个唯一值,则返回该值。如果没有任何值,或者存在至少两个不同的值,则返回 NULL。 - -**语法** - -```sql -singleValueOrNull(x) -``` - -**参数** - -* `x` — 任意[数据类型](../../data-types/index.md)的列(但不包括 [Map](../../data-types/map.md)、[Array](../../data-types/array.md) 或 [Tuple](../../data-types/tuple),这些类型不能为 [Nullable](../../data-types/nullable.md))。 - -**返回值** - -* 当且仅当 `x` 中存在唯一一个非 NULL 值时,返回该唯一值。 -* 当有零个或至少两个不同的值时,返回 `NULL`。 - -**示例** - -查询: - -```sql -CREATE TABLE test (x UInt8 NULL) ENGINE=Log; -INSERT INTO test (x) VALUES (NULL), (NULL), (5), (NULL), (NULL); -SELECT singleValueOrNull(x) FROM test; -``` - -结果: - -```response -┌─singleValueOrNull(x)─┐ -│ 5 │ -└──────────────────────┘ -``` - -查询: - -```sql -INSERT INTO test (x) VALUES (10); -SELECT singleValueOrNull(x) FROM test; -``` - -结果: - -```response -┌─singleValueOrNull(x)─┐ -│ ᴺᵁᴸᴸ │ -└──────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md deleted file mode 100644 index 791c741a400..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewpop.md +++ /dev/null @@ -1,29 +0,0 @@ ---- -description: '计算序列的偏度。' -sidebar_position: 185 -slug: /sql-reference/aggregate-functions/reference/skewpop -title: 'skewPop' -doc_type: 'reference' ---- - -# skewPop {#skewpop} - -计算序列的[偏度](https://en.wikipedia.org/wiki/Skewness)。 - -```sql -skewPop(expr) -``` - -**参数** - -`expr` — 返回数字的[表达式](/sql-reference/syntax#expressions)。 - -**返回值** - -给定分布的偏度。返回类型为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -```sql -SELECT skewPop(value) FROM series_with_value_column; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md deleted file mode 100644 index 909b92b0da3..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/skewsamp.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -description: '计算序列的样本偏度。' -sidebar_position: 186 -slug: /sql-reference/aggregate-functions/reference/skewsamp -title: 'skewSamp' -doc_type: 'reference' ---- - -# skewSamp {#skewsamp} - -计算序列的[样本偏度](https://en.wikipedia.org/wiki/Skewness)。 - -如果传入的值构成某个随机变量的样本,则该函数返回该随机变量偏度的无偏估计值。 - -```sql -skewSamp(expr) -``` - -**参数** - -`expr` — 返回数值的[表达式](/sql-reference/syntax#expressions)。 - -**返回值** - -给定分布的偏度值。类型为 [Float64](../../../sql-reference/data-types/float.md)。如果 `n <= 1`(`n` 为样本大小),则函数返回 `nan`。 - -**示例** - -```sql -SELECT skewSamp(value) FROM series_with_value_column; -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md deleted file mode 100644 index 6a9a496f3cd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sparkbar.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -description: '该函数在区间 `[min_x, max_x]` 内,为取值 `x` 及其出现频率 `y` 绘制频率直方图。' -sidebar_label: 'sparkbar' -sidebar_position: 187 -slug: /sql-reference/aggregate-functions/reference/sparkbar -title: 'sparkbar' -doc_type: 'reference' ---- - -# sparkbar {#sparkbar} - -该函数在区间 `[min_x, max_x]` 内,为取值 `x` 及其重复次数 `y` 绘制频率直方图。 -所有落入同一桶中的 `x` 的重复次数会被取平均值,因此数据应当预先聚合。 -负值的重复次数会被忽略。 - -如果未指定区间,则使用最小的 `x` 作为区间起点,最大的 `x` 作为区间终点。 -否则,区间之外的值会被忽略。 - -**语法** - -```sql -sparkbar(buckets[, min_x, max_x])(x, y) -``` - -**参数** - -* `buckets` — 段(区间)的数量。类型:[Integer](../../../sql-reference/data-types/int-uint.md)。 -* `min_x` — 区间起点。可选参数。 -* `max_x` — 区间终点。可选参数。 - -**参数(Arguments)** - -* `x` — 包含数值的字段。 -* `y` — 包含数值出现频率的字段。 - -**返回值** - -* 频率直方图。 - -**示例** - -查询: - -```sql -CREATE TABLE spark_bar_data (`value` Int64, `event_date` Date) ENGINE = MergeTree ORDER BY event_date; - -INSERT INTO spark_bar_data VALUES (1,'2020-01-01'), (3,'2020-01-02'), (4,'2020-01-02'), (-3,'2020-01-02'), (5,'2020-01-03'), (2,'2020-01-04'), (3,'2020-01-05'), (7,'2020-01-06'), (6,'2020-01-07'), (8,'2020-01-08'), (2,'2020-01-11'); - -SELECT sparkbar(9)(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); - -SELECT sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date,cnt) FROM (SELECT sum(value) as cnt, event_date FROM spark_bar_data GROUP BY event_date); -``` - -结果: - -```text -┌─sparkbar(9)(event_date, cnt)─┐ -│ ▂▅▂▃▆█ ▂ │ -└──────────────────────────────┘ - -┌─sparkbar(9, toDate('2020-01-01'), toDate('2020-01-10'))(event_date, cnt)─┐ -│ ▂▅▂▃▇▆█ │ -└──────────────────────────────────────────────────────────────────────────┘ -``` - -该函数的别名是 sparkBar。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md deleted file mode 100644 index f2437b764f0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpop.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '结果等于 varPop 的平方根。' -sidebar_position: 188 -slug: /sql-reference/aggregate-functions/reference/stddevpop -title: 'stddevPop' -doc_type: 'reference' ---- - -# stddevPop {#stddevpop} - -结果等于 [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md) 的平方根。 - -别名:`STD`、`STDDEV_POP`。 - -:::note -此函数采用数值上不稳定的算法。如果在计算中需要更高的[数值稳定性](https://en.wikipedia.org/wiki/Numerical_stability),请使用 [`stddevPopStable`](../reference/stddevpopstable.md) 函数。它的运行速度较慢,但计算误差更小。 -::: - -**语法** - -```sql -stddevPop(x) -``` - -**参数** - -* `x`:要求标准差的值集合。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -* `x` 的标准差的平方根。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevPop(population) AS stddev -FROM test_data; -``` - -结果: - -```response -┌────────────stddev─┐ -│ 3.794733192202055 │ -└───────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md deleted file mode 100644 index f47c2861da4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevpopstable.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '结果等于 varPop 的平方根。与 stddevPop 不同,此函数采用数值稳定的算法实现。' -sidebar_position: 189 -slug: /sql-reference/aggregate-functions/reference/stddevpopstable -title: 'stddevPopStable' -doc_type: 'reference' ---- - -# stddevPopStable {#stddevpopstable} - -结果等于 [varPop](../../../sql-reference/aggregate-functions/reference/varpop.md) 的平方根。与 [`stddevPop`](../reference/stddevpop.md) 不同,该函数使用数值更稳定的算法。它执行速度较慢,但计算误差更小。 - -**语法** - -```sql -stddevPopStable(x) -``` - -**参数** - -* `x`: 要计算标准差的一组值(总体)。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -`x` 的方差的平方根。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population Float64, -) -ENGINE = Log; - -INSERT INTO test_data SELECT randUniform(5.5, 10) FROM numbers(1000000) - -SELECT - stddevPopStable(population) AS stddev -FROM test_data; -``` - -结果: - -```response -┌─────────────stddev─┐ -│ 1.2999977786592576 │ -└────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md deleted file mode 100644 index e549ef090c1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsamp.md +++ /dev/null @@ -1,58 +0,0 @@ ---- -description: '结果等于 varSamp 的平方根' -sidebar_position: 190 -slug: /sql-reference/aggregate-functions/reference/stddevsamp -title: 'stddevSamp' -doc_type: 'reference' ---- - -# stddevSamp {#stddevsamp} - -结果等于 [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md) 的平方根。 - -别名:`STDDEV_SAMP`。 - -:::note -此函数使用数值不稳定的算法。如果在计算中需要[数值稳定性](https://en.wikipedia.org/wiki/Numerical_stability),请使用 [`stddevSampStable`](../reference/stddevsampstable.md) 函数。它的运行速度较慢,但计算误差更小。 -::: - -**语法** - -```sql -stddevSamp(x) -``` - -**参数** - -* `x`: 用于计算样本方差平方根的值。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -`x` 的样本方差的平方根。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSamp(population) -FROM test_data; -``` - -结果: - -```response -┌─stddevSamp(population)─┐ -│ 4 │ -└────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md deleted file mode 100644 index 04d06ee27e8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stddevsampstable.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '结果等于 varSamp 的平方根。与 stddevSamp 不同,本函数使用数值更稳定的算法。' -sidebar_position: 191 -slug: /sql-reference/aggregate-functions/reference/stddevsampstable -title: 'stddevSampStable' -doc_type: 'reference' ---- - -# stddevSampStable {#stddevsampstable} - -结果等于 [varSamp](../../../sql-reference/aggregate-functions/reference/varsamp.md) 的平方根。与 [`stddevSamp`](../reference/stddevsamp.md) 不同,此函数使用在数值上更稳定的算法。其运行速度较慢,但数值误差更小。 - -**语法** - -```sql -stddevSampStable(x) -``` - -**参数** - -* `x`: 用于计算样本方差平方根的值。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -返回 `x` 的样本方差的平方根。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - population UInt8, -) -ENGINE = Log; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - stddevSampStable(population) -FROM test_data; -``` - -结果: - -```response -┌─stddevSampStable(population)─┐ -│ 4 │ -└──────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md deleted file mode 100644 index 75b4af0d550..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlinearregression.md +++ /dev/null @@ -1,79 +0,0 @@ ---- -description: '此函数实现随机线性回归。它支持自定义学习率、L2 正则化系数和小批量大小等参数,并提供多种用于更新权重的方法(Adam、简单 SGD、Momentum、Nesterov)。' -sidebar_position: 192 -slug: /sql-reference/aggregate-functions/reference/stochasticlinearregression -title: 'stochasticLinearRegression' -doc_type: 'reference' ---- - -# stochasticLinearRegression {#agg_functions_stochasticlinearregression_parameters} - -此函数实现随机线性回归。它支持自定义学习率、L2 正则化系数、mini-batch 大小等参数,并提供多种权重更新方法([Adam](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Adam)(默认)、[simple SGD](https://en.wikipedia.org/wiki/Stochastic_gradient_descent)、[Momentum](https://en.wikipedia.org/wiki/Stochastic_gradient_descent#Momentum) 和 [Nesterov](https://mipt.ru/upload/medialibrary/d7e/41-91.pdf))。 - -### Parameters {#parameters} - -有 4 个可自定义参数。它们按顺序传递给函数,但不必传入全部四个参数;未显式传入的将使用默认值。不过,要获得较好的模型效果,通常需要对这些参数进行调优。 - -```text -stochasticLinearRegression(0.00001, 0.1, 15, 'Adam') -``` - -1. `learning rate` 是在执行梯度下降步骤时的步长系数。过大的 learning rate 可能导致模型权重发散为无穷大。默认值为 `0.00001`。 -2. `l2 regularization coefficient`(L2 正则化系数),可以帮助防止过拟合。默认值为 `0.1`。 -3. `mini-batch size` 用于设置在执行一次梯度下降步骤时,要对多少个元素计算梯度并求和。纯随机梯度下降只使用一个元素,但使用较小的 batch(约 10 个元素)可以使梯度更新更加稳定。默认值为 `15`。 -4. `method for updating weights`(用于更新权重的方法),包括:`Adam`(默认)、`SGD`、`Momentum` 和 `Nesterov`。`Momentum` 和 `Nesterov` 需要稍多的计算和内存,但在随机梯度方法的收敛速度和稳定性方面通常更有优势。 - -### 用法 {#usage} - -`stochasticLinearRegression` 的使用包含两个步骤:拟合模型以及在新数据上进行预测。为了拟合模型并保存其状态以供后续使用,我们使用 `-State` 组合器,它会保存状态(例如模型权重)。 -为了进行预测,我们使用函数 [evalMLMethod](/sql-reference/functions/machine-learning-functions#evalmlmethod),该函数以状态作为参数,并接收用于预测的特征。 - -
- -**1.** 拟合 - -可以使用如下查询。 - -```sql -CREATE TABLE IF NOT EXISTS train_data -( - param1 Float64, - param2 Float64, - target Float64 -) ENGINE = Memory; - -CREATE TABLE your_model ENGINE = Memory AS SELECT -stochasticLinearRegressionState(0.1, 0.0, 5, 'SGD')(target, param1, param2) -AS state FROM train_data; -``` - -在这里,我们还需要向 `train_data` 表中插入数据。参数的数量不是固定的,只取决于传入 `linearRegressionState` 的参数个数。所有参数的值都必须是数值类型。 -请注意,包含目标值(即我们希望学习预测的值)的列应作为第一个参数传入。 - -**2.** 预测 - -在将状态保存到表中之后,我们可以多次使用该状态进行预测,甚至可以将其与其他状态合并,以创建新的、更优的模型。 - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -该查询将返回一列预测值。请注意,`evalMLMethod` 的第一个参数是 `AggregateFunctionState` 对象,后面的参数是特征列。 - -`test_data` 是一个与 `train_data` 类似的表,但可能不包含目标值。 - -### 注意事项 {#notes} - -1. 为了合并两个模型,用户可以创建如下查询: - `sql SELECT state1 + state2 FROM your_models` - 其中 `your_models` 表同时包含这两个模型。该查询将返回一个新的 `AggregateFunctionState` 对象。 - -2. 如果未使用 `-State` 组合器,用户可以在不保存模型的情况下,根据需要获取已创建模型的权重。 - `sql SELECT stochasticLinearRegression(0.01)(target, param1, param2) FROM train_data` - 此类查询会拟合模型并返回其权重——前几个值是对应模型参数的权重,最后一个是偏置项。因此,在上面的示例中,该查询将返回一列包含 3 个值。 - -**另请参阅** - -* [stochasticLogisticRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [线性回归与逻辑回归的区别](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md deleted file mode 100644 index ddf8f43bbe5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/stochasticlogisticregression.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -description: '此函数实现了随机逻辑回归。可用于二分类问题,支持与 stochasticLinearRegression 相同的自定义参数,且工作方式相同。' -sidebar_position: 193 -slug: /sql-reference/aggregate-functions/reference/stochasticlogisticregression -title: 'stochasticLogisticRegression' -doc_type: 'reference' ---- - -# stochasticLogisticRegression {#stochasticlogisticregression} - -该函数实现了随机逻辑回归,可用于二分类问题,支持与 stochasticLinearRegression 相同的自定义参数,工作方式也相同。 - -### 参数 {#parameters} - -参数与 stochasticLinearRegression 中的完全相同: -`learning rate`、`l2 regularization coefficient`、`mini-batch size`、`method for updating weights`。 -有关更多信息,请参见 [参数](../reference/stochasticlinearregression.md/#parameters)。 - -```text -stochasticLogisticRegression(1.0, 1.0, 10, 'SGD') -``` - -**1.** 拟合 - -{/* */ } - -请参阅 [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlinearregression) 描述中的 `Fitting` 部分。 - -预测标签必须在区间 [-1, 1] 内。 - -**2.** 预测 - -{/* */ } - -利用已保存的状态,我们可以预测某个对象被标记为 `1` 的概率。 - -```sql -WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) FROM test_data -``` - -该查询将返回一个概率列。注意,`evalMLMethod` 的第一个参数是 `AggregateFunctionState` 对象,后续参数为特征列。 - -我们还可以设置一个概率阈值,将元素分配到不同的标签中。 - -```sql -SELECT ans < 1.1 AND ans > 0.5 FROM -(WITH (SELECT state FROM your_model) AS model SELECT -evalMLMethod(model, param1, param2) AS ans FROM test_data) -``` - -则输出为标签。 - -`test_data` 是一个与 `train_data` 类似的表,但可能不包含目标变量的取值。 - -**另请参阅** - -* [stochasticLinearRegression](/sql-reference/aggregate-functions/reference/stochasticlogisticregression) -* [线性回归和逻辑回归之间的区别](https://stackoverflow.com/questions/12146914/what-is-the-difference-between-linear-regression-and-logistic-regression) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md deleted file mode 100644 index 86b33453135..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttest.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '对来自两个总体的样本进行 Student t 检验。' -sidebar_label: 'studentTTest' -sidebar_position: 194 -slug: /sql-reference/aggregate-functions/reference/studentttest -title: 'studentTTest' -doc_type: 'reference' ---- - -# studentTTest {#studentttest} - -对来自两个总体的样本进行 Student t 检验。 - -**语法** - -```sql -studentTTest([confidence_level])(sample_data, sample_index) -``` - -两个样本的数值都存储在 `sample_data` 列中。如果 `sample_index` 等于 0,则该行的数值属于第一总体的样本;否则,该数值属于第二总体的样本。\ -原假设是两个总体的均值相等。假定总体服从正态分布且方差相等。 - -**参数** - -* `sample_data` — 样本数据。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — 样本索引。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**可选参数** - -* `confidence_level` — 用于计算置信区间的置信水平。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -[Tuple](../../../sql-reference/data-types/tuple.md),包含两个或四个元素(如果指定了可选的 `confidence_level`): - -* 计算得到的 t-统计量。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的 p 值。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间下界。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间上界。[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -输入表: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 21.1 │ 0 │ -│ 21.9 │ 1 │ -│ 21.7 │ 0 │ -│ 19.9 │ 1 │ -│ 21.8 │ 1 │ -└─────────────┴──────────────┘ -``` - -查询: - -```sql -SELECT studentTTest(sample_data, sample_index) FROM student_ttest; -``` - -结果: - -```text -┌─studentTTest(sample_data, sample_index)───┐ -│ (-0.21739130434783777,0.8385421208415731) │ -└───────────────────────────────────────────┘ -``` - -**另请参阅** - -* [Student's t 检验](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [welchTTest 函数](/sql-reference/aggregate-functions/reference/welchttest) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md deleted file mode 100644 index 264845bd347..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/studentttestonesample.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -description: '对一个样本与已知总体均值进行单样本 Student t 检验。' -sidebar_label: 'studentTTestOneSample' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/studentttestonesample -title: 'studentTTestOneSample' -doc_type: 'reference' ---- - -# studentTTestOneSample {#studentttestonesample} - -对单个样本进行 Student t 检验,以确定该样本的均值是否不同于已知的总体均值。 - -假定数据服从正态分布。原假设为样本均值等于总体均值。 - -**语法** - -```sql -studentTTestOneSample([置信水平])(样本数据, 总体均值) -``` - -可选参数 `confidence_level` 用于启用置信区间计算。 - -**参数说明** - -* `sample_data` — 样本数据。Integer、Float 或 Decimal。 -* `population_mean` — 用于检验的已知总体均值。Integer、Float 或 Decimal(通常为常量)。 - -**可选参数** - -* `confidence_level` — 置信区间的置信水平。取值为区间 (0, 1) 内的 Float。 - -注意: - -* 至少需要 2 个观测值;否则结果为 `(nan, nan)`(如果请求返回置信区间,则区间值为 `nan`)。 -* 常量或近似常量的输入也会由于标准误为 0(或接近 0)而返回 `nan`。 - -**返回值** - -返回一个包含两个或四个元素的 [Tuple](../../../sql-reference/data-types/tuple.md)(如果指定了 `confidence_level`): - -* 计算得到的 t 统计量。Float64。 -* 计算得到的 p 值(双尾)。Float64。 -* 计算得到的置信区间下界。Float64。(可选) -* 计算得到的置信区间上界。Float64。(可选) - -置信区间是针对给定置信水平下的样本均值计算的。 - -**示例** - -输入表: - -```text -┌─value─┐ -│ 20.3 │ -│ 21.1 │ -│ 21.7 │ -│ 19.9 │ -│ 21.8 │ -└───────┘ -``` - -不含置信区间: - -```sql -SELECT studentTTestOneSample()(value, 20.0) FROM t; --- or simply -SELECT studentTTestOneSample(value, 20.0) FROM t; -``` - -95% 置信区间: - -```sql -SELECT studentTTestOneSample(0.95)(value, 20.0) FROM t; -``` - -**另请参阅** - -* [Student t 检验](https://en.wikipedia.org/wiki/Student%27s_t-test) -* [studentTTest 函数](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md deleted file mode 100644 index 20dd320c61e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sum.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '计算总和。只适用于数值类型。' -sidebar_position: 195 -slug: /sql-reference/aggregate-functions/reference/sum -title: 'sum' -doc_type: 'reference' ---- - -# sum {#sum} - -计算总和。仅适用于数值。 - -**语法** - -```sql -sum(num) -``` - -**参数** - -* `num`: 数值列。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -* 值的总和。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**示例** - -首先我们创建一个表 `employees`,并向其中插入一些虚构的员工数据。 - -查询: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `salary` UInt32 -) -ENGINE = Log -``` - -```sql -INSERT INTO employees VALUES - (87432, 'John Smith', 45680), - (59018, 'Jane Smith', 72350), - (20376, 'Ivan Ivanovich', 58900), - (71245, 'Anastasia Ivanovna', 89210); -``` - -我们使用 `sum` 函数查询员工薪资的总额。 - -查询: - -```sql -SELECT sum(salary) FROM employees; -``` - -结果: - -```response - ┌─sum(salary)─┐ -1. │ 266140 │ - └─────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md deleted file mode 100644 index 2fd20b6a12f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumcount.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -description: '同时计算数值之和和行数。该函数被 ClickHouse 查询优化器使用:如果一个查询中存在多个 `sum`、`count` 或 `avg` 函数,它们可以被替换为单个 `sumCount` 函数以重用计算。通常无需显式调用该函数。' -sidebar_position: 196 -slug: /sql-reference/aggregate-functions/reference/sumcount -title: 'sumCount' -doc_type: 'reference' ---- - -同时计算数值之和和行数。该函数被 ClickHouse 查询优化器使用:如果一个查询中存在多个 `sum`、`count` 或 `avg` 函数,它们可以被替换为单个 `sumCount` 函数以重用计算。通常无需显式调用该函数。 - -**语法** - -```sql -sumCount(x) -``` - -**参数** - -* `x` — 输入值,必须是 [Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 元组 `(sum, count)`,其中 `sum` 是数值之和,`count` 是非 NULL 值行的行数。 - -类型:[Tuple](../../../sql-reference/data-types/tuple.md)。 - -**示例** - -查询: - -```sql -CREATE TABLE s_table (x Int8) ENGINE = Log; -INSERT INTO s_table SELECT number FROM numbers(0, 20); -INSERT INTO s_table VALUES (NULL); -SELECT sumCount(x) FROM s_table; -``` - -结果: - -```text -┌─sumCount(x)─┐ -│ (190,20) │ -└─────────────┘ -``` - -**另请参阅** - -* [optimize_syntax_fuse_functions](../../../operations/settings/settings.md#optimize_syntax_fuse_functions) 设置。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md deleted file mode 100644 index f24cc7d3eba..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumkahan.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: '使用 Kahan 补偿求和算法计算数值之和' -sidebar_position: 197 -slug: /sql-reference/aggregate-functions/reference/sumkahan -title: 'sumKahan' -doc_type: 'reference' ---- - -使用 [Kahan 补偿求和算法](https://en.wikipedia.org/wiki/Kahan_summation_algorithm) 计算数值之和。 -比 [sum](./sum.md) 函数更慢。 -补偿仅适用于 [Float](../../../sql-reference/data-types/float.md) 类型。 - -**语法** - -```sql -sumKahan(x) -``` - -**参数** - -* `x` — 输入值,必须是 [Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 - -**返回值** - -* 数值之和,类型为 [Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md),具体取决于输入参数的类型。 - -**示例** - -查询: - -```sql -SELECT sum(0.1), sumKahan(0.1) FROM numbers(10); -``` - -结果: - -```text -┌───────────sum(0.1)─┬─sumKahan(0.1)─┐ -│ 0.9999999999999999 │ 1 │ -└────────────────────┴───────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md deleted file mode 100644 index 8aeb6d7c10f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summap.md +++ /dev/null @@ -1,126 +0,0 @@ ---- -description: '根据 `key` 数组中指定的键,对一个或多个 `value` 数组进行求和。返回一个由数组组成的元组:首先是按排序后的顺序排列的键数组,然后是对对应键求和且无溢出的值数组。' -sidebar_position: 198 -slug: /sql-reference/aggregate-functions/reference/summap -title: 'sumMap' -doc_type: 'reference' ---- - -# sumMap {#summap} - -根据 `key` 数组中指定的键,对一个或多个 `value` 数组进行求和。返回一个由数组组成的元组:第一个数组是排好序的键,后续数组是对应键的求和值,且不会发生溢出。 - -**语法** - -* `sumMap(key , value1 [, value2 , ...])` [Array 类型](../../data-types/array.md)。 -* `sumMap(Tuple(key [, value1 , value2 , ...]))` [Tuple 类型](../../data-types/tuple.md)。 - -别名:`sumMappedArrays`。 - -**参数** - -* `key`:键的 [Array](../../data-types/array.md)。 -* `value1`、`value2`、…:需要对每个键求和的值的 [Array](../../data-types/array.md)。 - -传入一个由键数组和值数组组成的 tuple,与分别传入一个键数组和若干值数组是等价的。 - -:::note -对于每一行参与汇总的数据,`key` 和所有 `value` 数组中的元素个数必须相同。 -::: - -**返回值** - -* 返回一个由数组组成的元组:第一个数组包含排好序的键,后续数组包含对应键的求和值。 - -**示例** - -首先,我们创建一张名为 `sum_map` 的表,并向其中插入一些数据。键和值的数组分别存储在名为 `statusMap` 的 [Nested](../../data-types/nested-data-structures/index.md) 类型列中,同时也以名为 `statusMapTuple` 的 [tuple](../../data-types/tuple.md) 类型列合并存储,用于演示上文所述此函数两种不同语法的用法。 - -查询: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt16, - requests UInt64 - ), - statusMapTuple Tuple(Array(Int32), Array(Int32)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -接下来,我们使用 `sumMap` 函数查询该表,同时采用数组和元组类型这两种语法形式: - -查询: - -```sql -SELECT - timeslot, - sumMap(statusMap.status, statusMap.requests), - sumMap(statusMapTuple) -FROM sum_map -GROUP BY timeslot -``` - -结果: - -```text -┌────────────timeslot─┬─sumMap(statusMap.status, statusMap.requests)─┬─sumMap(statusMapTuple)─────────┐ -│ 2000-01-01 00:00:00 │ ([1,2,3,4,5],[10,10,20,10,10]) │ ([1,2,3,4,5],[10,10,20,10,10]) │ -│ 2000-01-01 00:01:00 │ ([4,5,6,7,8],[10,10,20,10,10]) │ ([4,5,6,7,8],[10,10,20,10,10]) │ -└─────────────────────┴──────────────────────────────────────────────┴────────────────────────────────┘ -``` - -**包含多个值数组的示例** - -`sumMap` 也支持同时对多个值数组进行聚合。 -当存在共享相同键的相关指标时,这会非常有用。 - -```sql title="Query" -CREATE TABLE multi_metrics( - date Date, - browser_metrics Nested( - browser String, - impressions UInt32, - clicks UInt32 - ) -) -ENGINE = MergeTree() -ORDER BY tuple(); - -INSERT INTO multi_metrics VALUES - ('2000-01-01', ['Firefox', 'Chrome'], [100, 200], [10, 25]), - ('2000-01-01', ['Chrome', 'Safari'], [150, 50], [20, 5]), - ('2000-01-01', ['Firefox', 'Edge'], [80, 40], [8, 4]); - -SELECT - sumMap(browser_metrics.browser, browser_metrics.impressions, browser_metrics.clicks) AS result -FROM multi_metrics; -``` - -```text title="Response" -┌─result────────────────────────────────────────────────────────────────────────┐ -│ (['Chrome', 'Edge', 'Firefox', 'Safari'], [350, 40, 180, 50], [45, 4, 18, 5]) │ -└───────────────────────────────────────────────────────────────────────────────┘ -``` - -在此示例中: - -* 结果元组包含三个数组 -* 第一个数组:按排序后的键(浏览器名称) -* 第二个数组:每个浏览器的总展示次数 -* 第三个数组:每个浏览器的总点击次数 - -**另请参阅** - -* [用于 Map 数据类型的 Map 组合器](../combinators.md#-map) -* [sumMapWithOverflow](../reference/summapwithoverflow.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md deleted file mode 100644 index 156cac1a3f1..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/summapwithoverflow.md +++ /dev/null @@ -1,96 +0,0 @@ ---- -description: '根据 `key` 数组中指定的键,对 `value` 数组进行求和。返回一个包含两个数组的元组:按排序顺序排列的键数组,以及对应键的求和值数组。与 sumMap 函数不同之处在于,它执行的是允许溢出的求和。' -sidebar_position: 199 -slug: /sql-reference/aggregate-functions/reference/summapwithoverflow -title: 'sumMapWithOverflow' -doc_type: 'reference' ---- - -# sumMapWithOverflow {#summapwithoverflow} - -根据 `key` 数组中指定的键,对 `value` 数组进行求和。返回一个由两个数组组成的元组:排好序的键数组,以及对应键的求和值数组。 -它与 [sumMap](../reference/summap.md) 函数的区别在于,它执行的是允许溢出的求和——即求和结果的数据类型与参数的数据类型相同。 - -**语法** - -* `sumMapWithOverflow(key , value )` [Array 类型](../../data-types/array.md)。 -* `sumMapWithOverflow(Tuple(key , value ))` [Tuple 类型](../../data-types/tuple.md)。 - -**参数** - -* `key`:键的 [Array](../../data-types/array.md)。 -* `value`:值的 [Array](../../data-types/array.md)。 - -将由键数组和值数组组成的元组作为参数,与分别传入键数组和值数组是等价的。 - -:::note -对每一行进行汇总时,`key` 和 `value` 中的元素数量必须相同。 -::: - -**返回值** - -* 返回一个由两个数组组成的元组:排好序的键数组,以及对应键的求和值数组。 - -**示例** - -首先,我们创建一张名为 `sum_map` 的表,并向其中插入一些数据。键数组和值数组分别存储在 [Nested](../../data-types/nested-data-structures/index.md) 类型的 `statusMap` 列中,同时也以 [tuple](../../data-types/tuple.md) 类型组合存储在 `statusMapTuple` 列中,以说明上文所述的此函数两种不同语法的用法。 - -查询: - -```sql -CREATE TABLE sum_map( - date Date, - timeslot DateTime, - statusMap Nested( - status UInt8, - requests UInt8 - ), - statusMapTuple Tuple(Array(Int8), Array(Int8)) -) ENGINE = Log; -``` - -```sql -INSERT INTO sum_map VALUES - ('2000-01-01', '2000-01-01 00:00:00', [1, 2, 3], [10, 10, 10], ([1, 2, 3], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:00:00', [3, 4, 5], [10, 10, 10], ([3, 4, 5], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [4, 5, 6], [10, 10, 10], ([4, 5, 6], [10, 10, 10])), - ('2000-01-01', '2000-01-01 00:01:00', [6, 7, 8], [10, 10, 10], ([6, 7, 8], [10, 10, 10])); -``` - -如果我们使用数组类型语法配合 `sumMap`、`sumMapWithOverflow` 以及 `toTypeName` 函数来查询该表,可以看到, -对于 `sumMapWithOverflow` 函数,累加值数组的数据类型与参数类型相同,都是 `UInt8`(即求和是按可能发生溢出的方式进行的)。对于 `sumMap`,累加值数组的数据类型则从 `UInt8` 变为 `UInt64`,从而避免了溢出。 - -查询: - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMap.status, statusMap.requests)), - toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests)), -FROM sum_map -GROUP BY timeslot -``` - -同样地,我们可以使用 `tuple` 语法来得到相同的结果。 - -```sql -SELECT - timeslot, - toTypeName(sumMap(statusMapTuple)), - toTypeName(sumMapWithOverflow(statusMapTuple)), -FROM sum_map -GROUP BY timeslot -``` - -结果: - -```text - ┌────────────timeslot─┬─toTypeName(sumMap(statusMap.status, statusMap.requests))─┬─toTypeName(sumMapWithOverflow(statusMap.status, statusMap.requests))─┐ -1. │ 2000-01-01 00:01:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ -2. │ 2000-01-01 00:00:00 │ Tuple(Array(UInt8), Array(UInt64)) │ Tuple(Array(UInt8), Array(UInt8)) │ - └─────────────────────┴──────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [sumMap](../reference/summap.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md deleted file mode 100644 index 5333dc072bb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/sumwithoverflow.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '计算一组数字的总和,结果的数据类型与输入参数相同。如果总和超过该数据类型的最大值,则按该数据类型的溢出语义进行计算。' -sidebar_position: 200 -slug: /sql-reference/aggregate-functions/reference/sumwithoverflow -title: 'sumWithOverflow' -doc_type: 'reference' ---- - -# sumWithOverflow {#sumwithoverflow} - -计算数值的总和,结果的数据类型与输入参数相同。如果总和超过该数据类型的最大值,则会发生溢出并按溢出结果计算。 - -仅适用于数值类型。 - -**语法** - -```sql -sumWithOverflow(num) -``` - -**参数** - -* `num`: 数值列。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -* 所有值的总和。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**示例** - -首先,我们创建一张 `employees` 表,并向其中插入一些虚构的员工数据。在本示例中,我们将 `salary` 定义为 `UInt16`,以便在对这些值求和时可能产生溢出。 - -查询: - -```sql -CREATE TABLE employees -( - `id` UInt32, - `name` String, - `monthly_salary` UInt16 -) -ENGINE = Log -``` - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow) -FROM employees -``` - -我们使用 `sum` 和 `sumWithOverflow` 函数来查询员工工资的总额,并使用 `toTypeName` 函数显示其类型。 -对于 `sum` 函数,结果类型为 `UInt64`,足够大,可以容纳该总和;而对于 `sumWithOverflow`,结果类型仍然是 `UInt16`。 - -查询: - -```sql -SELECT - sum(monthly_salary) AS no_overflow, - sumWithOverflow(monthly_salary) AS overflow, - toTypeName(no_overflow), - toTypeName(overflow), -FROM employees; -``` - -结果: - -```response - ┌─no_overflow─┬─overflow─┬─toTypeName(no_overflow)─┬─toTypeName(overflow)─┐ -1. │ 118700 │ 53164 │ UInt64 │ UInt16 │ - └─────────────┴──────────┴─────────────────────────┴──────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md deleted file mode 100644 index a4dd8383be8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/theilsu.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -description: '`theilsU` 函数用于计算 Theil 的 U 不确定性系数,用来衡量表中两列之间的相关性。' -sidebar_position: 201 -slug: /sql-reference/aggregate-functions/reference/theilsu -title: 'theilsU' -doc_type: 'reference' ---- - -# theilsU {#theilsu} - -`theilsU` 函数计算 [Theil's U 不确定性系数](https://en.wikipedia.org/wiki/Contingency_table#Uncertainty_coefficient),用于度量表中两列之间的关联程度。其取值范围为 −1.0(100% 负相关,或完全反向)到 +1.0(100% 正相关,或完全一致)。值为 0.0 表示没有关联。 - -**语法** - -```sql -theilsU(column1, column2) -``` - -**参数** - -* `column1` 和 `column2` 是要进行比较的列 - -**返回值** - -* 介于 -1 和 1 之间的值 - -**返回类型** 始终为 [Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -下面被比较的这两列之间只有较弱的相关性,因此 `theilsU` 的值为负值: - -```sql -SELECT - theilsU(a, b) -FROM - ( - SELECT - number % 10 AS a, - number % 4 AS b - FROM - numbers(150) - ); -``` - -结果: - -```response -┌────────theilsU(a, b)─┐ -│ -0.30195720557678846 │ -└──────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md deleted file mode 100644 index 6384c8cb444..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '聚合函数,用于在指定网格上对时间序列数据计算类似 PromQL 的 changes。' -sidebar_position: 229 -slug: /sql-reference/aggregate-functions/reference/timeSeriesChangesToGrid -title: 'timeSeriesChangesToGrid' -doc_type: 'reference' ---- - -聚合函数,接收由时间戳和值组成的时间序列数据对,并在由起始时间戳、结束时间戳和步长描述的规则时间网格上,从这些数据中计算[类似 PromQL 的 changes](https://prometheus.io/docs/prometheus/latest/querying/functions/#changes)。对于网格上的每个点,用于计算 `changes` 的样本会在指定的时间窗口内进行选取和计算。 - -Parameters: - -* `start timestamp` - 指定网格的起始时间 -* `end timestamp` - 指定网格的结束时间 -* `grid step` - 指定网格的步长(秒) -* `staleness` - 指定参与计算样本允许的最大“陈旧时间”(秒) - -Arguments: - -* `timestamp` - 样本的时间戳 -* `value` - 对应该 `timestamp` 的时间序列值 - -Return value: -在指定网格上的 `changes` 值,类型为 `Array(Nullable(Float64))`。返回数组中每个元素对应一个时间网格点。如果在对应时间窗口内没有样本可用于计算某个网格点的 changes 值,则该元素为 NULL。 - -Example: -以下查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210, 225] 上计算 `changes` 值: - -```sql -WITH - -- 注意:130 和 190 之间的间隔用于演示根据窗口参数为 ts = 180 填充值的方式 - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 135 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -也可以将多组时间戳和数值样本作为长度相同的数组传入。使用数组参数的同一查询如下: - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesChangesToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -此函数为实验性功能,可通过将 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md deleted file mode 100644 index 2504e8d7eff..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '在指定网格上对时间序列数据计算类似 PromQL 的 delta 的聚合函数。' -sidebar_position: 221 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDeltaToGrid -title: 'timeSeriesDeltaToGrid' -doc_type: 'reference' ---- - -该聚合函数接收由时间戳与数值构成的时间序列数据对,并在由起始时间戳、结束时间戳和步长描述的等间隔时间网格上,从这些数据中计算[类似 PromQL 的 delta](https://prometheus.io/docs/prometheus/latest/querying/functions/#delta)。对于网格上的每个点,用于计算 `delta` 的样本都在指定的时间窗口内予以考虑。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(单位:秒)。 -* `staleness` - 指定所考虑样本的最大“陈旧度”(单位:秒)。陈旧度窗口是一个左开右闭区间。 - -函数参数(Arguments): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列数值 - -返回值: -在指定网格上的 `delta` 值,类型为 `Array(Nullable(Float64))`。返回的数组为每个时间网格点包含一个值。如果在窗口内没有足够的样本来计算某个网格点的 delta 值,则该值为 NULL。 - -示例: -下面的查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算 `delta` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于演示如何根据窗口参数为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 120 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesDeltaToGr⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,3,4.5,3.75,NULL,NULL,3.75] │ - └─────────────────────────────────────────┘ -``` - -同样可以以大小相同的数组形式传入多个时间戳和值样本。使用数组参数时,相同的查询如下: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -此函数为实验性功能,可通过将 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md deleted file mode 100644 index 1d8b33ebdb5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '在指定网格上,对时间序列数据计算类似 PromQL 的导数的聚合函数。' -sidebar_position: 227 -slug: /sql-reference/aggregate-functions/reference/timeSeriesDerivToGrid -title: 'timeSeriesDerivToGrid' -doc_type: 'reference' ---- - -聚合函数,接收由时间戳和值组成的时间序列数据对作为输入,并在由起始时间戳、结束时间戳和步长描述的规则时间网格上,从这些数据计算出[类似 PromQL 的导数](https://prometheus.io/docs/prometheus/latest/querying/functions/#deriv)。对于网格上的每个点,用于计算 `deriv` 的样本都限制在指定的时间窗口内。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(秒)。 -* `staleness` - 指定被考虑样本的最大“陈旧度”(秒)。陈旧度窗口是一个左开右闭区间。 - -参数(Arguments): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -返回值: -在指定网格上的 `deriv` 值,类型为 `Array(Nullable(Float64))`。返回数组对每个时间网格点包含一个值。如果在窗口中没有足够的样本来计算某个特定网格点的导数值,则该位置的值为 NULL。 - -示例: -以下查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算 `deriv` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于演示如何根据窗口参数为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 120 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,0.1,0.11,0.15,NULL,NULL,0.15] │ - └─────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -也可以将多个时间戳和对应的值样本作为大小相同的数组传递。使用数组参数时,同一查询如下所示: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesDerivToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -该函数为实验性功能,可通过设置 `allow_experimental_ts_to_grid_aggregate_function=true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md deleted file mode 100644 index 004350fdaf8..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesGroupArray.md +++ /dev/null @@ -1,65 +0,0 @@ ---- -description: '将时间序列按时间戳升序排序。' -sidebar_position: 146 -slug: /sql-reference/aggregate-functions/reference/timeSeriesGroupArray -title: 'timeSeriesGroupArray' -doc_type: 'reference' ---- - -# timeSeriesGroupArray {#timeseriesgrouparray} - -按时间戳升序排序时间序列。 - -**语法** - -```sql -timeSeriesGroupArray(时间戳, 值) -``` - -**参数** - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -**返回值** - -该函数返回一个由元组 (`timestamp`, `value`) 组成的数组,并按 `timestamp` 升序排序。 -如果同一 `timestamp` 存在多个值,则函数会从中选择最大值。 - -**示例** - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- 与上述时间戳一一对应的数值数组 -SELECT timeSeriesGroupArray(timestamp, value) -FROM -( - -- 此子查询将时间戳和数值这两个数组展开为多行 `timestamp`、`value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesGroupArray(timestamp, value)───────┐ -1. │ [(100,5),(110,1),(120,6),(130,8),(140,19)] │ - └──────────────────────────────────────────────┘ -``` - -也可以将多个时间戳和值的样本以长度相同的数组形式传入。使用数组参数时,同一查询如下: - -```sql -WITH - [110, 120, 130, 140, 140, 100]::Array(UInt32) AS timestamps, - [1, 6, 8, 17, 19, 5]::Array(Float32) AS values -- 对应于上述时间戳的数值数组 -SELECT timeSeriesGroupArray(timestamps, values); -``` - -:::note -该函数处于实验阶段,可通过设置 `allow_experimental_ts_to_grid_aggregate_function=true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md deleted file mode 100644 index cf19c882cbd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '在指定网格上,对时间序列数据计算类似 PromQL 的 idelta 的聚合函数。' -sidebar_position: 222 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantDeltaToGrid -title: 'timeSeriesInstantDeltaToGrid' -doc_type: 'reference' ---- - -该聚合函数接收由时间戳和值成对组成的时间序列数据,并在由起始时间戳、结束时间戳和步长所描述的规则时间网格上,从这些数据中计算[类似 PromQL 的 idelta](https://prometheus.io/docs/prometheus/latest/querying/functions/#idelta)。对于网格上的每个点,会在指定时间窗口内选取样本用于计算 `idelta`。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(以秒为单位)。 -* `staleness` - 指定被考虑样本的最大“陈旧度”(以秒为单位)。陈旧度窗口是左开右闭区间。 - -参数(函数参数): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -返回值: -指定网格上的 `idelta` 值,类型为 `Array(Nullable(Float64))`。返回的数组中,每个时间网格点对应一个值。如果在窗口内没有足够的样本来计算某个网格点的瞬时增量值,则该值为 NULL。 - -示例: -下面的查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算 `idelta` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于演示如何根据窗口参数为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 120 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesInsta⋯stamps, values)─┐ -1. │ [NULL,NULL,0,2,1,1,NULL,NULL,3] │ - └─────────────────────────────────┘ -``` - -还可以将多个时间戳和值样本作为长度相同的数组传递。使用数组参数的同一查询如下: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -此函数为实验性功能,可通过设置 `allow_experimental_ts_to_grid_aggregate_function=true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md deleted file mode 100644 index ed2c3aa3d0f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '在指定网格上的时间序列数据上计算类似 PromQL irate 的聚合函数。' -sidebar_position: 223 -slug: /sql-reference/aggregate-functions/reference/timeSeriesInstantRateToGrid -title: 'timeSeriesInstantRateToGrid' -doc_type: 'reference' ---- - -该聚合函数将时间序列数据作为时间戳与数值的成对输入,并在由起始时间戳、结束时间戳和步长定义的规则时间网格上,从这些数据中计算 [类似 PromQL 的 irate](https://prometheus.io/docs/prometheus/latest/querying/functions/#irate)。对于网格上的每个点,用于计算 `irate` 的样本都在指定的时间窗口内进行选取。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(单位:秒)。 -* `staleness` - 指定被纳入计算的样本允许的最大“陈旧度”(单位:秒)。陈旧度窗口是左开右闭区间。 - -参数(Arguments): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列数值 - -返回值: -指定网格上的 `irate` 值,类型为 `Array(Nullable(Float64))`。返回的数组中每个时间网格点对应一个值。如果在窗口内没有足够的样本为某个网格点计算瞬时速率值,则该值为 NULL。 - -示例: -以下查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算 `irate` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于演示如何根据窗口参数为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 120 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesInstantRa⋯timestamps, values)─┐ -1. │ [NULL,NULL,0,0.2,0.1,0.1,NULL,NULL,0.3] │ - └─────────────────────────────────────────┘ -``` - -同样还可以将多个时间戳和值样本以长度相同的数组形式传入。使用数组参数的同一查询如下: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -该函数为实验性功能,可通过将参数 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md deleted file mode 100644 index 2d0c1182f84..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples.md +++ /dev/null @@ -1,164 +0,0 @@ ---- -description: '用于对时间序列数据进行重采样,以支持类似 PromQL 中 irate 和 idelta 的计算的聚合函数' -sidebar_position: 224 -slug: /sql-reference/aggregate-functions/reference/timeSeriesLastTwoSamples -title: 'timeSeriesLastTwoSamples' -doc_type: 'reference' ---- - -一个聚合函数,用于接收时间戳和值成对的时间序列数据,并且最多只存储最近的 2 个样本。 - -参数: - -* `timestamp` - 样本的时间戳 -* `value` - 对应于该 `timestamp` 的时间序列值\ - 也可以将多个时间戳和数值样本作为大小相同的 Array 传入。 - -返回值:\ -一个 `Tuple(Array(DateTime), Array(Float64))` —— 一对长度为 0 到 2 的等长数组。第一个数组包含采样后时间序列的时间戳,第二个数组包含时间序列对应的数值。 - -示例:\ -该聚合函数旨在与物化视图和用于存储针对网格对齐时间戳的重采样时间序列数据的聚合表一起使用。\ -考虑下面这个原始数据示例表,以及一个用于存储重采样数据的表: - -```sql --- 原始数据表 -CREATE TABLE t_raw_timeseries -( - metric_id UInt64, - timestamp DateTime64(3, 'UTC') CODEC(DoubleDelta, ZSTD), - value Float64 CODEC(DoubleDelta) -) -ENGINE = MergeTree() -ORDER BY (metric_id, timestamp); - --- 重采样至更大时间步长(15 秒)的数据表 -CREATE TABLE t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), -- 对齐至 15 秒的时间戳 - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -ENGINE = AggregatingMergeTree() -ORDER BY (metric_id, grid_timestamp); - --- 用于填充重采样表的物化视图 -CREATE MATERIALIZED VIEW mv_resampled_timeseries TO t_resampled_timeseries_15_sec -( - metric_id UInt64, - grid_timestamp DateTime('UTC') CODEC(DoubleDelta, ZSTD), - samples AggregateFunction(timeSeriesLastTwoSamples, DateTime64(3, 'UTC'), Float64) -) -AS SELECT - metric_id, - ceil(toUnixTimestamp(timestamp + interval 999 millisecond) / 15, 0) * 15 AS grid_timestamp, -- 将时间戳向上舍入至下一个网格点 - initializeAggregation('timeSeriesLastTwoSamplesState', timestamp, value) AS samples -FROM t_raw_timeseries -ORDER BY metric_id, grid_timestamp; -``` - -插入一些测试数据,并读取时间在 '2024-12-12 12:00:12' 到 '2024-12-12 12:00:30' 之间的数据 - -```sql --- 插入数据 -INSERT INTO t_raw_timeseries(metric_id, timestamp, value) SELECT number%10 AS metric_id, '2024-12-12 12:00:00'::DateTime64(3, 'UTC') + interval ((number/10)%100)*900 millisecond as timestamp, number%3+number%29 AS value FROM numbers(1000); - --- 查看原始数据 -SELECT * -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN '2024-12-12 12:00:12' AND '2024-12-12 12:00:31' -ORDER BY metric_id, timestamp; -``` - - -```response -3 2024-12-12 12:00:12.870 29 -3 2024-12-12 12:00:13.770 8 -3 2024-12-12 12:00:14.670 19 -3 2024-12-12 12:00:15.570 30 -3 2024-12-12 12:00:16.470 9 -3 2024-12-12 12:00:17.370 20 -3 2024-12-12 12:00:18.270 2 -3 2024-12-12 12:00:19.170 10 -3 2024-12-12 12:00:20.070 21 -3 2024-12-12 12:00:20.970 3 -3 2024-12-12 12:00:21.870 11 -3 2024-12-12 12:00:22.770 22 -3 2024-12-12 12:00:23.670 4 -3 2024-12-12 12:00:24.570 12 -3 2024-12-12 12:00:25.470 23 -3 2024-12-12 12:00:26.370 5 -3 2024-12-12 12:00:27.270 13 -3 2024-12-12 12:00:28.170 24 -3 2024-12-12 12:00:29.069 6 -3 2024-12-12 12:00:29.969 14 -3 2024-12-12 12:00:30.869 25 -``` - -查询时间戳为 '2024-12-12 12:00:15' 和 '2024-12-12 12:00:30' 的最近 2 个样本: - -```sql --- 检查重新采样的数据 -SELECT metric_id, grid_timestamp, (finalizeAggregation(samples).1 as timestamp, finalizeAggregation(samples).2 as value) -FROM t_resampled_timeseries_15_sec -WHERE metric_id = 3 AND grid_timestamp BETWEEN '2024-12-12 12:00:15' AND '2024-12-12 12:00:30' -ORDER BY metric_id, grid_timestamp; -``` - -```response -3 2024-12-12 12:00:15 (['2024-12-12 12:00:14.670','2024-12-12 12:00:13.770'],[19,8]) -3 2024-12-12 12:00:30 (['2024-12-12 12:00:29.969','2024-12-12 12:00:29.069'],[14,6]) -``` - -聚合表仅为每个按 15 秒对齐的时间戳存储最近的 2 个值。这样在计算 PromQL 风格的 `irate` 和 `idelta` 时,只需读取的数据量就远小于原始表中的数据量。 - -```sql --- 从原始数据计算 idelta 和 irate -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- 时间戳网格起始时间 - start_ts + INTERVAL 60 SECOND AS end_ts, -- 时间戳网格结束时间 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM t_raw_timeseries -WHERE metric_id = 3 AND timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - - -```sql --- 从重新采样的数据计算 idelta 和 irate -WITH - '2024-12-12 12:00:15'::DateTime64(3,'UTC') AS start_ts, -- 时间戳网格起始点 - start_ts + INTERVAL 60 SECOND AS end_ts, -- 时间戳网格结束点 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "陈旧性"窗口 -SELECT - metric_id, - timeSeriesInstantDeltaToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values), - timeSeriesInstantRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values) -FROM ( - SELECT - metric_id, - finalizeAggregation(samples).1 AS timestamps, - finalizeAggregation(samples).2 AS values - FROM t_resampled_timeseries_15_sec - WHERE metric_id = 3 AND grid_timestamp BETWEEN start_ts - interval window_seconds seconds AND end_ts -) -GROUP BY metric_id; -``` - -```response -3 [11,8,-18,8,11] [12.222222222222221,8.88888888888889,1.1111111111111112,8.88888888888889,12.222222222222221] -``` - -:::note -此函数为实验性功能,可通过设置 `allow_experimental_ts_to_grid_aggregate_function=true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md deleted file mode 100644 index 2f6315a73df..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid.md +++ /dev/null @@ -1,75 +0,0 @@ ---- -description: '在指定网格上的时间序列数据上计算类似 PromQL 的线性预测的聚合函数。' -sidebar_position: 228 -slug: /sql-reference/aggregate-functions/reference/timeSeriesPredictLinearToGrid -title: 'timeSeriesPredictLinearToGrid' -doc_type: 'reference' ---- - -该聚合函数接收由时间戳和值组成的时间序列数据对,并在由起始时间戳、结束时间戳和步长描述的规则时间网格上,计算具有指定预测时间偏移量的[类似 PromQL 的线性预测](https://prometheus.io/docs/prometheus/latest/querying/functions/#predict_linear)。对于网格上的每个点,用于计算 `predict_linear` 的样本都限定在指定的时间窗口内。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(以秒为单位)。 -* `staleness` - 指定所考虑样本的最大“陈旧度”(以秒为单位)。陈旧度窗口是一个左开右闭区间。 -* `predict_offset` - 指定要添加到预测时间的偏移秒数。 - -参数(函数参数): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -返回值: -指定网格上的 `predict_linear` 值,类型为 `Array(Nullable(Float64))`。返回的数组包含每个时间网格点的一个值。如果在窗口内没有足够的样本来计算某个网格点的速率值,则该值为 NULL。 - -示例: -下面的查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算具有 60 秒偏移量的 `predict_linear` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于演示如何根据窗口参数为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上述时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 120 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds, -- "陈旧性"窗口 - 60 AS predict_offset -- 预测时间偏移量 -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamp, value)─┐ -1. │ [NULL,NULL,1,9.166667,11.6,16.916666,NULL,NULL,16.5] │ - └─────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -同样也可以将多个时间戳和数值样本作为相同长度的数组传入。使用数组参数的等价查询如下所示: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds, - 60 AS predict_offset -SELECT timeSeriesPredictLinearToGrid(start_ts, end_ts, step_seconds, window_seconds, predict_offset)(timestamps, values); -``` - -:::note -此函数为实验性功能,可通过将 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md deleted file mode 100644 index b02e64dcf57..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesRateToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '在指定网格上的时间序列数据上计算类似 PromQL 的 rate 的聚合函数。' -sidebar_position: 225 -slug: /sql-reference/aggregate-functions/reference/timeSeriesRateToGrid -title: 'timeSeriesRateToGrid' -doc_type: 'reference' ---- - -聚合函数,接受由时间戳和值构成的时间序列数据对,并在由起始时间戳、结束时间戳和步长定义的规则时间网格上,从这些数据计算[类似 PromQL 的 rate](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate)。对于网格上的每个点,用于计算 `rate` 的样本会被限制在指定的时间窗口内。 - -参数: - -* `start timestamp` - 指定网格的起始时间。 -* `end timestamp` - 指定网格的结束时间。 -* `grid step` - 指定网格的步长(单位:秒)。 -* `staleness` - 指定被考虑样本的最大“陈旧度”(单位:秒)。陈旧度窗口是一个左开右闭区间。 - -参数(Arguments): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -返回值: -在指定网格上的 `rate` 值,类型为 `Array(Nullable(Float64))`。返回的数组对每个时间网格点包含一个值。如果在窗口内没有足够的样本来计算某个网格点的 rate 值,则该值为 NULL。 - -示例: -以下查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上计算 `rate` 值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于展示将根据窗口参数,如何为 ts = 150、165、180 填充值 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上面各时间戳对应的值数组 - 90 AS start_ts, -- 时间戳网格的起始点 - 90 + 120 AS end_ts, -- 时间戳网格的结束点 - 15 AS step_seconds, -- 时间戳网格的步长(秒) - 45 AS window_seconds -- “数据陈旧”窗口 -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和数值数组展开为多行记录,每行包含 `timestamp` 和 `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesRateToGrid(start_ts, ⋯w_seconds)(timestamps, values)─┐ -1. │ [NULL,NULL,0,0.06666667,0.1,0.083333336,NULL,NULL,0.083333336] │ - └────────────────────────────────────────────────────────────────┘ -``` - -也可以将多个时间戳和数值样本以大小相同的数组形式传入。同一个查询使用数组参数时如下所示: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesRateToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -此函数为实验性功能,可通过将 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md deleted file mode 100644 index 309b565b6bb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '将时间序列数据重新采样到指定网格的聚合函数。' -sidebar_position: 226 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResampleToGridWithStaleness -title: 'timeSeriesResampleToGridWithStaleness' -doc_type: 'reference' ---- - -该聚合函数将时间序列数据作为时间戳和值的成对数据进行处理,并根据由起始时间戳、结束时间戳和步长描述的等间隔时间网格对数据进行重新采样。对于网格上的每个点,会在给定时间窗口内选择最近的样本。 - -别名:`timeSeriesLastToGrid`。 - -参数(Parameters): - -* `start timestamp` - 指定网格的起始时间戳 -* `end timestamp` - 指定网格的结束时间戳 -* `grid step` - 指定网格的步长(秒) -* `staleness window` - 指定最近样本允许的最大“陈旧”时间(秒) - -自变量(Arguments): - -* `timestamp` - 样本的时间戳 -* `value` - 与该 `timestamp` 对应的时间序列值 - -返回值: -将时间序列值重新采样到指定网格后的结果,类型为 `Array(Nullable(Float64))`。返回的数组中,每个时间网格点对应一个值。如果某个网格点没有样本,则该值为 NULL。 - -示例: -下面的查询将时间序列数据重新采样到网格 [90, 105, 120, 135, 150, 165, 180, 195, 210] 上,并为网格上的每个点选择时间戳不早于该网格点 30 秒之前的值: - -```sql -WITH - -- 注意:140 和 190 之间的间隔用于展示在给定 staleness 窗口参数下,ts = 150、165、180 的值是如何被填充的 - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, -- 与上方 timestamps 一一对应的值数组 - 90 AS start_ts, -- 时间戳网格起点 - 90 + 120 AS end_ts, -- 时间戳网格终点 - 15 AS step_seconds, -- 时间戳网格的步长 - 30 AS window_seconds -- “staleness” 时间窗口 -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将 timestamps 和 values 数组展开为多行,每行包含 `timestamp` 和 `value` - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesResa⋯stamp, value)─┐ -1. │ [NULL,NULL,1,3,4,4,NULL,5,8] │ - └──────────────────────────────┘ -``` - -还可以将多个时间戳和数值样本作为长度相同的数组传入。使用数组参数的相同查询如下: - -```sql -WITH - [110, 120, 130, 140, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 1, 3, 4, 5, 5, 8, 12, 13]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 120 AS end_ts, - 15 AS step_seconds, - 30 AS window_seconds -SELECT timeSeriesResampleToGridWithStaleness(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -该函数为实验性特性,可通过设置 `allow_experimental_ts_to_grid_aggregate_function=true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md deleted file mode 100644 index 43cfde5acba..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '一个聚合函数,用于在指定网格上对时间序列数据计算类似 PromQL 的 resets。' -sidebar_position: 230 -slug: /sql-reference/aggregate-functions/reference/timeSeriesResetsToGrid -title: 'timeSeriesResetsToGrid' -doc_type: 'reference' ---- - -聚合函数,将时间序列数据作为时间戳与数值的成对输入,并在由起始时间戳、结束时间戳和步长描述的规则时间网格上,从这些数据中计算[类似 PromQL 的 `resets`](https://prometheus.io/docs/prometheus/latest/querying/functions/#resets)。对于网格上的每一个点,用于计算 `resets` 的样本都会在指定的时间窗口内进行考虑。 - -参数: - -* `start timestamp` - 指定网格的起始时间 -* `end timestamp` - 指定网格的结束时间 -* `grid step` - 指定网格的步长(以秒为单位) -* `staleness` - 指定被考虑样本的最大“陈旧时间”(staleness,单位为秒) - -参数: - -* `timestamp` - 样本的时间戳 -* `value` - 时间序列在该 `timestamp` 上对应的数值 - -返回值: -指定网格上的 `resets` 值,类型为 `Array(Nullable(Float64))`。返回的数组对每一个时间网格点包含一个值。如果在某个网格点对应的窗口内没有样本可用于计算该点的 `resets` 值,则该值为 NULL。 - -示例: -下面的查询在网格 [90, 105, 120, 135, 150, 165, 180, 195, 210, 225] 上计算 `resets` 值: - -```sql -WITH - -- 注意:130 和 190 之间的间隔用于演示根据窗口参数为 ts = 180 填充值的方式 - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, -- 对应上述时间戳的值数组 - 90 AS start_ts, -- 时间戳网格起始值 - 90 + 135 AS end_ts, -- 时间戳网格结束值 - 15 AS step_seconds, -- 时间戳网格步长 - 45 AS window_seconds -- "过期"窗口 -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value) -FROM -( - -- 此子查询将时间戳和值数组转换为 `timestamp`、`value` 行 - SELECT - arrayJoin(arrayZip(timestamps, values)) AS ts_and_val, - ts_and_val.1 AS timestamp, - ts_and_val.2 AS value -); -``` - -响应: - -```response - ┌─timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamp, value)─┐ -1. │ [NULL,NULL,0,1,1,1,NULL,0,1,2] │ - └──────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -也可以将多组时间戳和值样本作为大小相同的数组传入。使用数组参数时,相同的查询如下: - -```sql -WITH - [110, 120, 130, 190, 200, 210, 220, 230]::Array(DateTime) AS timestamps, - [1, 3, 2, 6, 6, 4, 2, 0]::Array(Float32) AS values, - 90 AS start_ts, - 90 + 135 AS end_ts, - 15 AS step_seconds, - 45 AS window_seconds -SELECT timeSeriesResetsToGrid(start_ts, end_ts, step_seconds, window_seconds)(timestamps, values); -``` - -:::note -此函数为实验特性,可通过将 `allow_experimental_ts_to_grid_aggregate_function` 设置为 `true` 来启用。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md deleted file mode 100644 index 8ee1f52f971..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topk.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: '返回一个包含指定列中近似出现频率最高值的数组。结果数组按这些值的近似出现频率降序排列(而不是按值本身排序)。' -sidebar_position: 202 -slug: /sql-reference/aggregate-functions/reference/topk -title: 'topK' -doc_type: 'reference' ---- - -# topK {#topk} - -返回一个数组,其中包含指定列中近似最常出现的值。结果数组按这些值的近似出现频率降序排序(而不是按值本身排序)。 - -实现了用于计算 TopK 的 [Filtered Space-Saving](https://doi.org/10.1016/j.ins.2010.08.024) 算法,基于 [Parallel Space Saving](https://doi.org/10.1016/j.ins.2015.09.003) 中的 reduce-and-combine 算法。 - -```sql -topK(N)(column) -topK(N, load_factor)(column) -topK(N, load_factor, 'counts')(column) -``` - -此函数不保证结果是精确的。在某些情况下,可能会产生误差,并且可能返回的一些高频值并不是最频繁的值。 - -`N` 的最大值为 `65536`。 - -**参数** - -* `N` — 要返回的元素数量。可选。默认值:10。 -* `load_factor` — 定义为值预留多少单元格。如果 `uniq(column) > N * load_factor`,则 `topK` 函数的结果将是近似值。可选。默认值:3。 -* `counts` — 定义结果中是否应包含近似计数和误差值。 - -**参数** - -* `column` — 要计算频率的值。 - -**示例** - -使用 [OnTime](../../../getting-started/example-datasets/ontime.md) 数据集,并在 `AirlineID` 列中选择出现频率最高的三个值。 - -```sql -SELECT topK(3)(AirlineID) AS res -FROM ontime -``` - -```text -┌─res─────────────────┐ -│ [19393,19790,19805] │ -└─────────────────────┘ -``` - -**另请参阅** - -* [topKWeighted](../../../sql-reference/aggregate-functions/reference/topkweighted.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md deleted file mode 100644 index ed4ad8abea4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/topkweighted.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -description: '返回指定列中近似出现最频繁值的数组。结果数组按值的近似出现频率降序排列(而不是按值本身排序)。此外,还会考虑值的权重。' -sidebar_position: 203 -slug: /sql-reference/aggregate-functions/reference/topkweighted -title: 'topKWeighted' -doc_type: 'reference' ---- - -# topKWeighted {#topkweighted} - -返回一个数组,其中包含指定列中近似出现频率最高的值。结果数组按照值的近似出现频率降序排列(而不是按值本身排序)。同时会考虑各个值的权重。 - -**语法** - -```sql -topKWeighted(N)(column, weight) -topKWeighted(N, load_factor)(column, weight) -topKWeighted(N, load_factor, 'counts')(column, weight) -``` - -**参数** - -* `N` — 要返回的元素个数。可选。默认值:10。 -* `load_factor` — 定义为存储值预留的单元格数量。如果 uniq(column) > N * load_factor,则 topK 函数的结果是近似值。可选。默认值:3。 -* `counts` — 定义结果中是否应包含近似计数和误差值。 - -**参数说明** - -* `column` — 值。 -* `weight` — 权重。每个值在频率计算中按 `weight` 次计入。[UInt64](../../../sql-reference/data-types/int-uint.md)。 - -**返回值** - -返回一个数组,包含其权重近似总和最大的值。 - -**示例** - -查询: - -```sql -SELECT topKWeighted(2)(k, w) FROM -VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -结果: - -```text -┌─topKWeighted(2)(k, w)──┐ -│ ['z','x'] │ -└────────────────────────┘ -``` - -查询: - -```sql -SELECT topKWeighted(2, 10, 'counts')(k, w) -FROM VALUES('k Char, w UInt64', ('y', 1), ('y', 1), ('x', 5), ('y', 1), ('z', 10)) -``` - -结果: - -```text -┌─topKWeighted(2, 10, 'counts')(k, w)─┐ -│ [('z',10,0),('x',5,0)] │ -└─────────────────────────────────────┘ -``` - -**另请参阅** - -* [topK](../../../sql-reference/aggregate-functions/reference/topk.md) -* [approx_top_k](../../../sql-reference/aggregate-functions/reference/approxtopk.md) -* [approx_top_sum](../../../sql-reference/aggregate-functions/reference/approxtopsum.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md deleted file mode 100644 index 100dd1bb56d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniq.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -description: '计算参数中不同值的近似个数。' -sidebar_position: 204 -slug: /sql-reference/aggregate-functions/reference/uniq -title: 'uniq' -doc_type: 'reference' ---- - -# uniq {#uniq} - -计算参数中不同值的近似个数。 - -```sql -uniq(x[, ...]) -``` - -**参数** - -该函数的参数数量可变。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**返回值** - -* 一个 [UInt64](../../../sql-reference/data-types/int-uint.md) 类型的数值。 - -**实现细节** - -函数: - -* 计算聚合中所有参数的哈希值,并在后续计算中使用该哈希值。 - -* 使用自适应采样算法。对于计算状态,函数会维护最多 65536 个元素哈希值的样本。该算法在 CPU 上既非常精确又非常高效。当查询中包含多个此类函数时,使用 `uniq` 的速度几乎与使用其他聚合函数一样快。 - -* 保证结果具有确定性(不依赖于查询处理顺序)。 - -我们建议在几乎所有场景中使用此函数。 - -**另请参阅** - -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md deleted file mode 100644 index 4da8ea5cce2..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -description: '计算不同参数值的近似数量。' -sidebar_position: 205 -slug: /sql-reference/aggregate-functions/reference/uniqcombined -title: 'uniqCombined' -doc_type: 'reference' ---- - -# uniqCombined {#uniqcombined} - -计算参数不同取值的近似数量。 - -```sql -uniqCombined(HLL_precision)(x[, ...]) -``` - -函数 `uniqCombined` 是计算不同值数量的一个不错选择。 - -**参数** - -* `HLL_precision`: [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) 中单元数量的以 2 为底的对数。可选,你可以以 `uniqCombined(x[, ...])` 的形式使用该函数。`HLL_precision` 的默认值为 17,在实际中占用约 96 KiB 空间(2^17 个单元,每个 6 位)。 -* `X`:可变数量的参数。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**返回值** - -* 一个 [UInt64](../../../sql-reference/data-types/int-uint.md) 类型的数值。 - -**实现细节** - -`uniqCombined` 函数: - -* 为聚合中的所有参数计算哈希(`String` 使用 64 位哈希,其他类型使用 32 位哈希),并在计算中使用该哈希值。 -* 使用三种算法的组合:数组、哈希表以及带误差修正表的 HyperLogLog。 - * 对于不同元素数量较少的情况,使用数组。 - * 当集合规模更大时,使用哈希表。 - * 对于元素数量更多的情况,使用 HyperLogLog,其将占用固定大小的内存。 -* 以确定性的方式给出结果(不依赖查询处理顺序)。 - -:::note\ -由于对非 `String` 类型使用 32 位哈希,当基数显著大于 `UINT_MAX` 时,结果误差会非常大(在数百亿个不同值之后误差会快速增大),因此在这种情况下你应使用 [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64)。 -::: - -与 [uniq](/sql-reference/aggregate-functions/reference/uniq) 函数相比,`uniqCombined` 函数: - -* 内存消耗可降低数倍。 -* 计算精度可提高数倍。 -* 通常性能略低。在某些场景下,`uniqCombined` 的性能可以优于 `uniq`,例如在通过网络传输大量聚合状态的分布式查询中。 - -**示例** - -查询: - -```sql -SELECT uniqCombined(number) FROM numbers(1e6); -``` - -结果: - -```response -┌─uniqCombined(number)─┐ -│ 1001148 │ -- 100 万 -└──────────────────────┘ -``` - -请参阅 [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) 的示例部分,以了解在更大规模输入数据时 `uniqCombined` 与 `uniqCombined64` 之间差异的示例。 - -**另请参阅** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md deleted file mode 100644 index 74eeaf6118e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqcombined64.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -description: '计算不同参数值的近似数量。它与 uniqCombined 相同,但对所有数据类型都使用 64 位哈希值,而不仅仅是对 String 数据类型使用。' -sidebar_position: 206 -slug: /sql-reference/aggregate-functions/reference/uniqcombined64 -title: 'uniqCombined64' -doc_type: 'reference' ---- - -# uniqCombined64 {#uniqcombined64} - -计算不同参数值的近似数量。与 [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) 相同,但对所有数据类型都使用 64 位哈希,而不仅仅是对 String 类型使用。 - -```sql -uniqCombined64(HLL_precision)(x[, ...]) -``` - -**参数** - -* `HLL_precision`: [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) 中单元数量的以 2 为底的对数。你也可以以 `uniqCombined64(x[, ...])` 的形式调用该函数。`HLL_precision` 的默认值为 17,这在实际中对应 96 KiB 的空间(2^17 个单元,每个 6 bit)。 -* `X`:可变数量的参数。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**返回值** - -* 一个 [UInt64](../../../sql-reference/data-types/int-uint.md) 类型的数值。 - -**实现细节** - -`uniqCombined64` 函数: - -* 对聚合中的所有参数计算哈希(对所有数据类型使用 64 位哈希),然后在计算中使用该哈希值。 -* 使用三种算法的组合:数组、哈希表和带误差修正表的 HyperLogLog。 - * 对少量不同元素,使用数组。 - * 当集合规模更大时,使用哈希表。 - * 对更多数量的元素,使用 HyperLogLog,它会占用固定大小的内存。 -* 以确定性的方式提供结果(不依赖于查询处理顺序)。 - -:::note -由于对所有类型都使用 64 位哈希,对于明显大于 `UINT_MAX` 的基数,结果不会像 [uniqCombined](../../../sql-reference/aggregate-functions/reference/uniqcombined.md) 那样出现非常高的误差;`uniqCombined` 对非 `String` 类型使用的是 32 位哈希。 -::: - -与 [uniq](/sql-reference/aggregate-functions/reference/uniq) 函数相比,`uniqCombined64` 函数: - -* 内存占用要少几倍。 -* 计算精度要高几倍。 - -**示例** - -在下面的示例中,`uniqCombined64` 对 `1e10` 个不同数字进行计算,返回了一个与不同参数值个数非常接近的近似值。 - -查询: - -```sql -SELECT uniqCombined64(number) FROM numbers(1e10); -``` - -结果: - -```response -┌─uniqCombined64(number)─┐ -│ 9998568925 │ -- 100 亿 -└────────────────────────┘ -``` - -相比之下,就这个规模的输入而言,`uniqCombined` 函数返回的近似结果相当不理想。 - -查询: - -```sql -SELECT uniqCombined(number) FROM numbers(1e10); -``` - -结果: - -```response -┌─uniqCombined(number)─┐ -│ 5545308725 │ -- 55.45 亿 -└──────────────────────┘ -``` - -**另请参阅** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md deleted file mode 100644 index 22a17e01e4e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqexact.md +++ /dev/null @@ -1,42 +0,0 @@ ---- -description: '计算不同参数取值的精确数量。' -sidebar_position: 207 -slug: /sql-reference/aggregate-functions/reference/uniqexact -title: 'uniqExact' -doc_type: 'reference' ---- - -# uniqExact {#uniqexact} - -计算参数的不同取值的精确数量。 - -```sql -uniqExact(x[, ...]) -``` - -如果确实需要精确结果,请使用 `uniqExact` 函数。否则,请使用 [uniq](/sql-reference/aggregate-functions/reference/uniq) 函数。 - -`uniqExact` 函数比 `uniq` 使用更多内存,因为其状态大小会随着不同取值数量的增加而无限增长。 - -**参数** - -该函数接受可变数量的参数。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**示例** - -在本示例中,我们将使用 `uniqExact` 函数来统计 [opensky 数据集](https://sql.clickhouse.com?query=U0VMRUNUIHVuaXFFeGFjdCh0eXBlY29kZSkgRlJPTSBvcGVuc2t5Lm9wZW5za3k&) 中唯一机型代码(用于标识飞机类型的简短标识符)的数量。 - -```sql title="Query" -SELECT uniqExact(typecode) FROM opensky.opensky -``` - -```response title="Response" -1106 -``` - -**另请参阅** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md deleted file mode 100644 index f8068069388..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqhll12.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -description: '使用 HyperLogLog 算法计算不同参数取值的近似数量。' -sidebar_position: 208 -slug: /sql-reference/aggregate-functions/reference/uniqhll12 -title: 'uniqHLL12' -doc_type: 'reference' ---- - -# uniqHLL12 {#uniqhll12} - -使用 [HyperLogLog](https://en.wikipedia.org/wiki/HyperLogLog) 算法计算不同参数取值的近似个数。 - -```sql -uniqHLL12(x[, ...]) -``` - -**参数** - -该函数接受可变数量的参数。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**返回值** - -* 一个 [UInt64](../../../sql-reference/data-types/int-uint.md) 类型的数值。 - -**实现细节** - -函数: - -* 对聚合中的所有参数计算哈希值,并在后续计算中使用该哈希值。 - -* 使用 HyperLogLog 算法来估算不同参数取值的数量。 - - 使用 2^12 个 5 位单元。状态大小略大于 2.5 KB。对于小数据集(<10K 个元素),结果精度不高(误差可达约 10%)。但对于高基数数据集(10K-100M),结果相当准确,最大误差约为 1.6%。从 100M 开始,估算误差会增大;对于基数极高的数据集(1B+ 个元素),函数将返回非常不准确的结果。 - -* 提供确定性结果(不依赖于查询处理顺序)。 - -不建议使用此函数。在大多数情况下,应使用 [uniq](/sql-reference/aggregate-functions/reference/uniq) 或 [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) 函数。 - -**另请参阅** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) -* [uniqTheta](/sql-reference/aggregate-functions/reference/uniqthetasketch) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md deleted file mode 100644 index e64e09a5c74..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/uniqthetasketch.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: '使用 Theta Sketch Framework 估算不同参数取值的数量。' -sidebar_position: 209 -slug: /sql-reference/aggregate-functions/reference/uniqthetasketch -title: 'uniqTheta' -doc_type: 'reference' ---- - -使用 [Theta Sketch Framework](https://datasketches.apache.org/docs/Theta/ThetaSketches.html#theta-sketch-framework) 估算不同参数取值的数量。 - -```sql -uniqTheta(x[, ...]) -``` - -**参数** - -该函数接受可变数量的参数。参数可以是 `Tuple`、`Array`、`Date`、`DateTime`、`String` 或数值类型。 - -**返回值** - -* 一个 [UInt64](../../../sql-reference/data-types/int-uint.md) 类型的数字。 - -**实现细节** - -函数: - -* 为聚合中的所有参数计算哈希值,然后在计算中使用该哈希值。 - -* 使用 [KMV](https://datasketches.apache.org/docs/Theta/InverseEstimate.html) 算法来近似估算不同参数值的数量。 - - 使用了 4096 (2^12) 个 64 位 sketch。状态占用空间约为 41 KB。 - -* 相对误差为 3.125%(95% 置信度),详细信息参见[相对误差表](https://datasketches.apache.org/docs/Theta/ThetaErrorTable.html)。 - -**另请参阅** - -* [uniq](/sql-reference/aggregate-functions/reference/uniq) -* [uniqCombined](/sql-reference/aggregate-functions/reference/uniqcombined) -* [uniqCombined64](/sql-reference/aggregate-functions/reference/uniqcombined64) -* [uniqHLL12](/sql-reference/aggregate-functions/reference/uniqhll12) -* [uniqExact](/sql-reference/aggregate-functions/reference/uniqexact) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md deleted file mode 100644 index 3e7178a0e6c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpop.md +++ /dev/null @@ -1,60 +0,0 @@ ---- -description: '计算总体方差。' -sidebar_position: 210 -slug: /sql-reference/aggregate-functions/reference/varPop -title: 'varPop' -doc_type: 'reference' ---- - - - -## varPop {#varpop} - -计算总体方差: - -$$ -\frac{\Sigma{(x - \bar{x})^2}}{n} -$$ - -**语法** - -```sql -varPop(x) -``` - -别名:`VAR_POP`。 - -**参数** - -- `x`:用于计算总体方差的数值总体。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal\*](../../data-types/decimal.md)。 - -**返回值** - -- 返回 `x` 的总体方差。[`Float64`](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3), (3), (3), (4), (4), (5), (5), (7), (11), (15); - -SELECT - varPop(x) AS var_pop -FROM test_data; -``` - -结果: - -```response -┌─var_pop─┐ -│ 14.4 │ -└─────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md deleted file mode 100644 index 56e1f8d6c3b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varpopstable.md +++ /dev/null @@ -1,54 +0,0 @@ ---- -description: '返回总体方差。与 varPop 不同,此函数使用数值稳定的算法。其执行速度较慢,但计算误差更低。' -sidebar_position: 211 -slug: /sql-reference/aggregate-functions/reference/varpopstable -title: 'varPopStable' -doc_type: 'reference' ---- - -## varPopStable {#varpopstable} - -返回总体方差。与 [`varPop`](../reference/varpop.md) 不同,此函数使用[数值稳定](https://en.wikipedia.org/wiki/Numerical_stability)的算法。它的运行速度更慢,但能提供更小的计算误差。 - -**语法** - -```sql -varPopStable(x) -``` - -别名:`VAR_POP_STABLE`。 - -**参数** - -* `x`:要计算总体方差的一组总体数据。[(U)Int*](../../data-types/int-uint.md)、[Float*](../../data-types/float.md)、[Decimal*](../../data-types/decimal.md)。 - -**返回值** - -* 返回 `x` 的总体方差。[Float64](../../data-types/float.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x UInt8, -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (3),(3),(3),(4),(4),(5),(5),(7),(11),(15); - -SELECT - varPopStable(x) AS var_pop_stable -FROM test_data; -``` - -结果: - -```response -┌─var_pop_stable─┐ -│ 14.4 │ -└────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md deleted file mode 100644 index 29d812ab89e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsamp.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -description: '计算数据集的样本方差。' -sidebar_position: 212 -slug: /sql-reference/aggregate-functions/reference/varSamp -title: 'varSamp' -doc_type: 'reference' ---- - - - -## varSamp {#varsamp} - -计算数据集的样本方差。 - -**语法** - -```sql -varSamp(x) -``` - -别名:`VAR_SAMP`。 - -**参数** - -- `x`:需要计算样本方差的数据集。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal\*](../../data-types/decimal.md)。 - -**返回值** - -- 返回输入数据集 `x` 的样本方差。[Float64](../../data-types/float.md)。 - -**实现细节** - -`varSamp` 函数使用以下公式计算样本方差: - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -其中: - -- `x` 为数据集中的每个数据点。 -- `mean(x)` 为数据集的算术平均值。 -- `n` 为数据集中的数据点数量。 - -该函数假定输入数据集是从更大总体中抽取的样本。如果需要计算整个总体的方差(即拥有完整数据集时),应使用 [`varPop`](../reference/varpop.md)。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSamp(x),3) AS var_samp FROM test_data; -``` - -结果: - -```response -┌─var_samp─┐ -│ 0.865 │ -└──────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md deleted file mode 100644 index 094cdb8cd34..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/varsampstable.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -description: '计算数据集的样本方差。与 `varSamp` 不同,此函数使用数值更稳定的算法。它的运行速度较慢,但计算误差更小。' -sidebar_position: 213 -slug: /sql-reference/aggregate-functions/reference/varsampstable -title: 'varSampStable' -doc_type: 'reference' ---- - - - -## varSampStable {#varsampstable} - -计算数据集的样本方差。与 [`varSamp`](../reference/varsamp.md) 不同,该函数使用数值稳定算法。虽然运行速度较慢,但计算误差更小。 - -**语法** - -```sql -varSampStable(x) -``` - -别名:`VAR_SAMP_STABLE` - -**参数** - -- `x`:需要计算样本方差的数据集。[(U)Int\*](../../data-types/int-uint.md)、[Float\*](../../data-types/float.md)、[Decimal\*](../../data-types/decimal.md)。 - -**返回值** - -- 返回输入数据集的样本方差。[Float64](../../data-types/float.md)。 - -**实现细节** - -`varSampStable` 函数使用与 [`varSamp`](../reference/varsamp.md) 相同的公式计算样本方差: - -$$ -\sum\frac{(x - \text{mean}(x))^2}{(n - 1)} -$$ - -其中: - -- `x` 为数据集中的每个数据点。 -- `mean(x)` 为数据集的算术平均值。 -- `n` 为数据集中的数据点数量。 - -**示例** - -查询: - -```sql -DROP TABLE IF EXISTS test_data; -CREATE TABLE test_data -( - x Float64 -) -ENGINE = Memory; - -INSERT INTO test_data VALUES (10.5), (12.3), (9.8), (11.2), (10.7); - -SELECT round(varSampStable(x),3) AS var_samp_stable FROM test_data; -``` - -结果: - -```response -┌─var_samp_stable─┐ -│ 0.865 │ -└─────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md deleted file mode 100644 index 15484270894..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/aggregate-functions/reference/welchttest.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: '对来自两个总体的样本执行 Welch t 检验。' -sidebar_label: 'welchTTest' -sidebar_position: 214 -slug: /sql-reference/aggregate-functions/reference/welchttest -title: 'welchTTest' -doc_type: 'reference' ---- - -# welchTTest {#welchttest} - -将 Welch t 检验应用于来自两个总体的样本。 - -**语法** - -```sql -welchTTest([confidence_level])(sample_data, sample_index) -``` - -两个样本的值都在 `sample_data` 列中。如果 `sample_index` 等于 0,则该行中的值属于来自第一总体的样本,否则属于来自第二总体的样本。 - -原假设是两个总体的均值相等。假定总体服从正态分布,总体方差可以不相等。 - -**参数** - -* `sample_data` — 样本数据。[Integer](../../../sql-reference/data-types/int-uint.md)、[Float](../../../sql-reference/data-types/float.md) 或 [Decimal](../../../sql-reference/data-types/decimal.md)。 -* `sample_index` — 样本索引。[Integer](../../../sql-reference/data-types/int-uint.md)。 - -**参数** - -* `confidence_level` — 用于计算置信区间的置信水平(可选)。[Float](../../../sql-reference/data-types/float.md)。 - -**返回值** - -包含两个或四个元素的 [Tuple](../../../sql-reference/data-types/tuple.md)(如果指定了可选的 `confidence_level`)。 - -* 计算得到的 t 统计量。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的 p 值。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间下界。[Float64](../../../sql-reference/data-types/float.md)。 -* 计算得到的置信区间上界。[Float64](../../../sql-reference/data-types/float.md)。 - -**示例** - -输入表: - -```text -┌─sample_data─┬─sample_index─┐ -│ 20.3 │ 0 │ -│ 22.1 │ 0 │ -│ 21.9 │ 0 │ -│ 18.9 │ 1 │ -│ 20.3 │ 1 │ -│ 19 │ 1 │ -└─────────────┴──────────────┘ -``` - -查询: - -```sql -SELECT welchTTest(sample_data, sample_index) FROM welch_ttest; -``` - -结果: - -```text -┌─welchTTest(sample_data, sample_index)─────┐ -│ (2.7988719532211235,0.051807360348581945) │ -└───────────────────────────────────────────┘ -``` - -**另请参阅** - -* [Welch t 检验](https://en.wikipedia.org/wiki/Welch%27s_t-test) -* [studentTTest 函数](/sql-reference/aggregate-functions/reference/studentttest) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md deleted file mode 100644 index 6cbb98b6837..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/aggregatefunction.md +++ /dev/null @@ -1,106 +0,0 @@ ---- -description: 'ClickHouse 中 AggregateFunction 数据类型的文档,其用于存储聚合函数的中间状态' -keywords: ['AggregateFunction', 'Type'] -sidebar_label: 'AggregateFunction' -sidebar_position: 46 -slug: /sql-reference/data-types/aggregatefunction -title: 'AggregateFunction 类型' -doc_type: 'reference' ---- - -# AggregateFunction 数据类型 {#aggregatefunction-type} - -## 描述 {#description} - -ClickHouse 中的所有[聚合函数](/sql-reference/aggregate-functions)都有一个特定于实现的中间状态,可以序列化为 -`AggregateFunction` 数据类型并存储在表中。这通常通过 -[物化视图](../../sql-reference/statements/create/view.md) 来实现。 - -有两个常与 `AggregateFunction` 类型搭配使用的聚合函数[组合器](/sql-reference/aggregate-functions/combinators): - -- [`-State`](/sql-reference/aggregate-functions/combinators#-state) 聚合函数组合器,将其附加到聚合函数名后时,会生成 `AggregateFunction` 的中间状态。 -- [`-Merge`](/sql-reference/aggregate-functions/combinators#-merge) 聚合函数组合器,用于从中间状态中获取聚合的最终结果。 - -## 语法 {#syntax} - -```sql -AggregateFunction(aggregate_function_name, types_of_arguments...) -``` - -**参数** - -* `aggregate_function_name` - 聚合函数的名称。如果该函数是参数化的,则还需要指定其参数。 -* `types_of_arguments` - 聚合函数参数的类型。 - -例如: - -```sql -CREATE TABLE t -( - column1 AggregateFunction(uniq, UInt64), - column2 AggregateFunction(anyIf, String, UInt8), - column3 AggregateFunction(quantiles(0.5, 0.9), UInt64) -) ENGINE = ... -``` - - -## 使用方法 {#usage} - -### 数据插入 {#data-insertion} - -要向包含 `AggregateFunction` 类型列的表中插入数据,可以使用 `INSERT SELECT` 语句,结合聚合函数以及 -[`-State`](/sql-reference/aggregate-functions/combinators#-state) 聚合函数组合器。 - -例如,如果要向类型为 `AggregateFunction(uniq, UInt64)` 和 -`AggregateFunction(quantiles(0.5, 0.9), UInt64)` 的列中插入数据,则需要使用以下带有组合器的聚合函数。 - -```sql -uniqState(UserID) -quantilesState(0.5, 0.9)(SendTiming) -``` - -与函数 `uniq` 和 `quantiles` 相比,`uniqState` 和 `quantilesState` -(追加了 `-State` 组合器)返回的是状态,而不是最终值。 -换句话说,它们返回的是 `AggregateFunction` 类型的值。 - -在 `SELECT` 查询的结果中,`AggregateFunction` 类型的值在所有 ClickHouse 输出格式中 -都有与具体实现相关的二进制表示。 - -有一个特殊的会话级别设置 `aggregate_function_input_format`,允许从输入值构建状态。 -它支持以下格式: - -* `state` - 带有序列化状态的二进制字符串(默认)。 - 如果你使用 `SELECT` 查询将数据导出成例如 `TabSeparated` 格式, - 那么这个导出结果可以通过 `INSERT` 查询重新加载回去。 -* `value` - 该格式要求聚合函数参数的单个值,或者在参数为多个时,一个包含这些值的元组;这些值会被反序列化以构造相应的状态 -* `array` - 该格式要求一个值的 Array,如上面 `value` 选项所述;数组中的所有元素会被聚合以形成状态 - - -### 数据选择 {#data-selection} - -从 `AggregatingMergeTree` 表中查询数据时,使用 `GROUP BY` 子句, -并使用与插入数据时相同的聚合函数,但要使用 -[`-Merge`](/sql-reference/aggregate-functions/combinators#-merge) 组合器。 - -在聚合函数后附加 `-Merge` 组合器时,该函数会接收一组状态值,对其进行合并, -并返回完整数据聚合的结果。 - -例如,下面这两个查询将返回相同的结果: - -```sql -SELECT uniq(UserID) FROM table - -SELECT uniqMerge(state) FROM (SELECT uniqState(UserID) AS state FROM table GROUP BY RegionID) -``` - - -## 使用示例 {#usage-example} - -参见 [AggregatingMergeTree](../../engines/table-engines/mergetree-family/aggregatingmergetree.md) 引擎描述。 - -## 相关内容 {#related-content} - -- 博客:[在 ClickHouse 中使用聚合组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -- [MergeState](/sql-reference/aggregate-functions/combinators#-mergestate) - 组合器。 -- [State](/sql-reference/aggregate-functions/combinators#-state) 组合器。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md deleted file mode 100644 index 2d03c40c480..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/array.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -description: '关于 ClickHouse 中 Array 数据类型的文档' -sidebar_label: 'Array(T)' -sidebar_position: 32 -slug: /sql-reference/data-types/array -title: 'Array(T)' -doc_type: 'reference' ---- - -# Array(T) {#arrayt} - -由 `T` 类型元素组成的数组,数组的起始索引为 1。`T` 可以是任何数据类型,包括数组。 - -## 创建数组 {#creating-an-array} - -你可以使用函数创建数组: - -```sql -array(T) -``` - -也可以用方括号。 - -```sql -[] -``` - -创建数组示例: - -```sql -SELECT array(1, 2) AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName(array(1, 2))─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴─────────────────────────┘ -``` - -```sql -SELECT [1, 2] AS x, toTypeName(x) -``` - -```text -┌─x─────┬─toTypeName([1, 2])─┐ -│ [1,2] │ Array(UInt8) │ -└───────┴────────────────────┘ -``` - -## 使用数据类型 {#working-with-data-types} - -在临时创建数组时,ClickHouse 会自动将参数类型推断为能够容纳所有列出参数的最窄数据类型。如果其中包含任何 [Nullable](/sql-reference/data-types/nullable) 或字面量 [NULL](/operations/settings/formats#input_format_null_as_default) 值,数组元素的类型也会变成 [Nullable](../../sql-reference/data-types/nullable.md)。 - -如果 ClickHouse 无法确定数据类型,它会抛出异常。例如,当尝试同时使用字符串和数字创建数组时,就会发生这种情况(`SELECT array(1, 'a')`)。 - -自动数据类型检测示例: - -```sql -SELECT array(1, 2, NULL) AS x, toTypeName(x) -``` - -```text -┌─x──────────┬─toTypeName(array(1, 2, NULL))─┐ -│ [1,2,NULL] │ Array(Nullable(UInt8)) │ -└────────────┴───────────────────────────────┘ -``` - -如果尝试创建一个包含不兼容数据类型的数组,ClickHouse 会抛出异常: - -```sql -SELECT array(1, 'a') -``` - -```text -从服务器收到异常(版本 1.1.54388): -代码: 386. DB::Exception: 从 localhost:9000, 127.0.0.1 收到。DB::Exception: 类型 UInt8 和 String 之间不存在公共超类型,因为其中部分为 String/FixedString 类型,而其余部分不是。 -``` - -## 数组大小 {#array-size} - -可以在不读取整列数据的情况下,使用 `size0` 子列获取数组的长度。对于多维数组,可以使用 `sizeN-1` 子列,其中 `N` 表示目标维度。 - -**示例** - -查询: - -```sql -CREATE TABLE t_arr (`arr` Array(Array(Array(UInt32)))) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO t_arr VALUES ([[[12, 13, 0, 1],[12]]]); - -SELECT arr.size0, arr.size1, arr.size2 FROM t_arr; -``` - -结果: - -```text -┌─arr.size0─┬─arr.size1─┬─arr.size2─┐ -│ 1 │ [2] │ [[4,1]] │ -└───────────┴───────────┴───────────┘ -``` - -## 从 Array 读取嵌套子列 {#reading-nested-subcolumns-from-array} - -如果 `Array` 中的嵌套类型 `T` 本身包含子列(例如,它是一个[命名元组](./tuple.md)),则可以通过具有相同子列名称的 `Array(T)` 类型来读取这些子列。此时子列的类型将是“原始子列类型”的 `Array`。 - -**示例** - -```sql -CREATE TABLE t_arr (arr Array(Tuple(field1 UInt32, field2 String))) ENGINE = MergeTree ORDER BY tuple(); -INSERT INTO t_arr VALUES ([(1, 'Hello'), (2, 'World')]), ([(3, 'This'), (4, 'is'), (5, 'subcolumn')]); -SELECT arr.field1, toTypeName(arr.field1), arr.field2, toTypeName(arr.field2) from t_arr; -``` - -```test -┌─arr.field1─┬─toTypeName(arr.field1)─┬─arr.field2────────────────┬─toTypeName(arr.field2)─┐ -│ [1,2] │ Array(UInt32) │ ['Hello','World'] │ Array(String) │ -│ [3,4,5] │ Array(UInt32) │ ['This','is','subcolumn'] │ Array(String) │ -└────────────┴────────────────────────┴───────────────────────────┴────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md deleted file mode 100644 index 5b6163fdd5c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/boolean.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -description: 'ClickHouse 中 Boolean 类型的文档' -sidebar_label: 'Boolean' -sidebar_position: 33 -slug: /sql-reference/data-types/boolean -title: 'Bool(布尔)' -doc_type: 'reference' ---- - -# Bool {#bool} - -`bool` 类型在内部存储为 UInt8。可能的取值为 `true`(1)、`false`(0)。 - -```sql -SELECT true AS col, toTypeName(col); -┌─col──┬─toTypeName(true)─┐ -│ true │ Bool │ -└──────┴──────────────────┘ - -select true == 1 as col, toTypeName(col); -┌─col─┬─toTypeName(equals(true, 1))─┐ -│ 1 │ UInt8 │ -└─────┴─────────────────────────────┘ -``` - -```sql -CREATE TABLE test_bool -( - `A` Int64, - `B` Bool -) -ENGINE = Memory; - -INSERT INTO test_bool VALUES (1, true),(2,0); - -SELECT * FROM test_bool; -┌─A─┬─B─────┐ -│ 1 │ true │ -│ 2 │ false │ -└───┴───────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md deleted file mode 100644 index e75c0a167c6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/data-types-binary-encoding.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -description: '数据类型二进制编码规范文档' -sidebar_label: '数据类型二进制编码规范' -sidebar_position: 56 -slug: /sql-reference/data-types/data-types-binary-encoding -title: '数据类型二进制编码规范' -doc_type: 'reference' ---- - -# 数据类型二进制编码规范 {#data-types-binary-encoding-specification} - -本规范描述了一种用于对 ClickHouse 数据类型进行二进制编码和解码的二进制格式。该格式用于 `Dynamic` 列的[二进制序列化](dynamic.md#binary-output-format),并且在相应设置下可用于输入/输出格式 [RowBinaryWithNamesAndTypes](/interfaces/formats/RowBinaryWithNamesAndTypes) 和 [Native](/interfaces/formats/Native)。 - -下表描述了每种数据类型在二进制格式中的表示方式。每种数据类型的编码由 1 个字节的类型标识和一些可选的附加信息组成。 -二进制编码中的 `var_uint` 表示大小使用可变长度整数(Variable-Length Quantity)压缩方式进行编码。 - -| ClickHouse 数据类型 | 二进制编码 | -| --------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `Nothing` | `0x00` | -| `UInt8` | `0x01` | -| `UInt16` | `0x02` | -| `UInt32` | `0x03` | -| `UInt64` | `0x04` | -| `UInt128` | `0x05` | -| `UInt256` | `0x06` | -| `Int8` | `0x07` | -| `Int16` | `0x08` | -| `Int32` | `0x09` | -| `Int64` | `0x0A` | -| `Int128` | `0x0B` | -| `Int256` | `0x0C` | -| `Float32` | `0x0D` | -| `Float64` | `0x0E` | -| `Date` | `0x0F` | -| `Date32` | `0x10` | -| `DateTime` | `0x11` | -| `DateTime(time_zone)` | `0x12` | -| `DateTime64(P)` | `0x13` | -| `DateTime64(P, time_zone)` | `0x14` | -| `String` | `0x15` | -| `FixedString(N)` | `0x16` | -| `Enum8` | `0x17...` | -| `Enum16` | `0x18...>` | -| `Decimal32(P, S)` | `0x19` | -| `Decimal64(P, S)` | `0x1A` | -| `Decimal128(P, S)` | `0x1B` | -| `Decimal256(P, S)` | `0x1C` | -| `UUID` | `0x1D` | -| `Array(T)` | `0x1E` | -| `Tuple(T1, ..., TN)` | `0x1F...` | -| `Tuple(name1 T1, ..., nameN TN)` | `0x20...` | -| `Set` | `0x21` | -| `间隔` | `0x22`(参见 [interval_kind 的二进制编码](#interval-kind-binary-encoding)) | -| `Nullable(T)` | `0x23` | -| `Function` | `0x24...` | -| `AggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x25......`(参见 [聚合函数参数的二进制编码](#aggregate-function-parameter-binary-encoding)) | -| `LowCardinality(T)` | `0x26` | -| `Map(K, V)` | `0x27` | -| `IPv4` | `0x28` | -| `IPv6` | `0x29` | -| `Variant(T1, ..., TN)` | `0x2A...` | -| `Dynamic(max_types=N)` | `0x2B` | -| `Custom type`(`Ring`、`Polygon` 等) | `0x2C` | -| `Bool` | `0x2D` | -| `SimpleAggregateFunction(function_name(param_1, ..., param_N), arg_T1, ..., arg_TN)` | `0x2E......`(参见[聚合函数参数二进制编码](#aggregate-function-parameter-binary-encoding)) | -| `Nested(name1 T1, ..., nameN TN)` | `0x2F...` | -| `JSON(max_dynamic_paths=N, max_dynamic_types=M, path Type, SKIP skip_path, SKIP REGEXP skip_path_regexp)` | `0x30.........` | -| `BFloat16` | `0x31` | -| `时间` | `0x32` | -| `Time64(P)` | `0x34` | -| `QBit(T, N)` | `0x36` | - -对于类型 `JSON`,字节 `uint8_serialization_version` 表示序列化版本。当前版本固定为 0,但如果将来为 `JSON` 类型引入新的参数,该值可能会发生变化。 - -### Interval 间隔种类的二进制编码 {#interval-kind-binary-encoding} - -下表说明了 `Interval` 数据类型中不同间隔种类的二进制编码方式。 - -| 间隔种类 (Interval kind) | 二进制编码 (Binary encoding) | -|--------------------------|------------------------------| -| `Nanosecond` | `0x00` | -| `Microsecond` | `0x01` | -| `Millisecond` | `0x02` | -| `Second` | `0x03` | -| `Minute` | `0x04` | -| `Hour` | `0x05` | -| `Day` | `0x06` | -| `Week` | `0x07` | -| `Month` | `0x08` | -| `Quarter` | `0x09` | -| `Year` | `0x1A` | - -### 聚合函数参数的二进制编码 {#aggregate-function-parameter-binary-encoding} - -下表描述了 `AggregateFunction` 和 `SimpleAggregateFunction` 的参数是如何进行编码的。 -每个参数的编码由 1 个用于指示参数类型的字节和参数值本身组成。 - -| 参数类型 | 二进制编码 | -|--------------------------|--------------------------------------------------------------------------------------------------------------------------------| -| `Null` | `0x00` | -| `UInt64` | `0x01` | -| `Int64` | `0x02` | -| `UInt128` | `0x03` | -| `Int128` | `0x04` | -| `UInt256` | `0x05` | -| `Int256` | `0x06` | -| `Float64` | `0x07` | -| `Decimal32` | `0x08` | -| `Decimal64` | `0x09` | -| `Decimal128` | `0x0A` | -| `Decimal256` | `0x0B` | -| `String` | `0x0C` | -| `Array` | `0x0D...` | -| `Tuple` | `0x0E...` | -| `Map` | `0x0F...` | -| `IPv4` | `0x10` | -| `IPv6` | `0x11` | -| `UUID` | `0x12` | -| `Bool` | `0x13` | -| `Object` | `0x14...` | -| `AggregateFunctionState` | `0x15` | -| `Negative infinity` | `0xFE` | -| `Positive infinity` | `0xFF` | \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md deleted file mode 100644 index db59cbdd6f6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date.md +++ /dev/null @@ -1,53 +0,0 @@ ---- -description: '关于 ClickHouse 中 Date 数据类型的文档' -sidebar_label: 'Date' -sidebar_position: 12 -slug: /sql-reference/data-types/date -title: 'Date' -doc_type: 'reference' ---- - -# Date {#date} - -一个日期值。以两个字节存储,为自 1970-01-01 起的天数(无符号)。允许存储的取值范围从 Unix 纪元起点之后,到在编译阶段由常量定义的上限(目前为 2149 年,但最后一个完全受支持的年份是 2148 年)。 - -支持的取值范围:[1970-01-01, 2149-06-06]。 - -日期值在存储时不包含时区信息。 - -**示例** - -创建一个包含 `Date` 类型列的表并向其中插入数据: - -```sql -CREATE TABLE dt -( - `timestamp` Date, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 解析日期 --- - 从字符串, --- - 从"小"整数(解释为自 1970-01-01 以来的天数),以及 --- - 从"大"整数(解释为自 1970-01-01 以来的秒数)。 -INSERT INTO dt VALUES ('2019-01-01', 1), (17897, 2), (1546300800, 3); - -SELECT * FROM dt; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2019-01-01 │ 1 │ -│ 2019-01-01 │ 2 │ -│ 2019-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**另请参阅** - -* [用于处理日期和时间的函数](../../sql-reference/functions/date-time-functions.md) -* [用于处理日期和时间的运算符](../../sql-reference/operators#operators-for-working-with-dates-and-times) -* [`DateTime` 数据类型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md deleted file mode 100644 index 2bae733c1b6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/date32.md +++ /dev/null @@ -1,49 +0,0 @@ ---- -description: 'ClickHouse 中 Date32 数据类型的文档,用于存储比 Date 具有更大取值范围的日期' -sidebar_label: 'Date32' -sidebar_position: 14 -slug: /sql-reference/data-types/date32 -title: 'Date32' -doc_type: 'reference' ---- - -# Date32 {#date32} - -一种日期类型。支持与 [DateTime64](../../sql-reference/data-types/datetime64.md) 相同的日期范围。以本机字节序的有符号 32 位整数形式存储,其数值表示自 `1900-01-01` 起经过的天数。**重要!** 数值 0 表示 `1970-01-01`,负值表示 `1970-01-01` 之前的天数。 - -**示例** - -创建一个包含 `Date32` 类型列的表并向其中插入数据: - -```sql -CREATE TABLE dt32 -( - `timestamp` Date32, - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 解析日期 --- - 从字符串解析 --- - 从"小"整数解析(解释为自 1970-01-01 以来的天数) --- - 从"大"整数解析(解释为自 1970-01-01 以来的秒数) -INSERT INTO dt32 VALUES ('2100-01-01', 1), (47482, 2), (4102444800, 3); - -SELECT * FROM dt32; -``` - -```text -┌──timestamp─┬─event_id─┐ -│ 2100-01-01 │ 1 │ -│ 2100-01-01 │ 2 │ -│ 2100-01-01 │ 3 │ -└────────────┴──────────┘ -``` - -**另请参阅** - -* [toDate32](../../sql-reference/functions/type-conversion-functions.md#todate32) -* [toDate32OrZero](/sql-reference/functions/type-conversion-functions#todate32orzero) -* [toDate32OrNull](/sql-reference/functions/type-conversion-functions#todate32ornull) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md deleted file mode 100644 index 2de78a1f53b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime.md +++ /dev/null @@ -1,208 +0,0 @@ ---- -description: '关于 ClickHouse 中 DateTime 数据类型的文档,用于存储秒级精度的时间戳' -sidebar_label: 'DateTime' -sidebar_position: 16 -slug: /sql-reference/data-types/datetime -title: 'DateTime' -doc_type: 'reference' ---- - - - -# DateTime {#datetime} - -用于存储某一时刻,该时刻可以表示为日历日期和一天中的时间。 - -语法: - -```sql -DateTime([timezone]) -``` - -支持的取值范围:[1970-01-01 00:00:00, 2106-02-07 06:28:15]。 - -时间精度:1 秒。 - - -## 速度 {#speed} - -在 _大多数_ 情况下,`Date` 数据类型比 `DateTime` 更快。 - -`Date` 类型只需要 2 字节存储,而 `DateTime` 需要 4 字节。不过,在压缩时,`Date` 和 `DateTime` 之间的大小差异会变得更为显著。这种差异被放大是因为 `DateTime` 中的分钟和秒不如日期部分易于压缩。对 `Date` 而不是 `DateTime` 进行过滤和聚合也会更快。 - - - -## 使用说明 {#usage-remarks} - -时间点一律以[Unix 时间戳](https://en.wikipedia.org/wiki/Unix_time)的形式存储,与时区或夏令时无关。时区会影响 `DateTime` 类型值在文本格式中的显示方式,以及以字符串形式指定的值(如 `2020-01-01 05:00:01`)的解析方式。 - -与时区无关的 Unix 时间戳会被存储在表中,而时区则用于在数据导入/导出期间将其与文本格式互相转换,或者用于对这些值进行日历计算(例如:`toDate`、`toHour` 函数等)。时区不会存储在表的行中(或结果集中),而是存储在列的元数据中。 - -支持的时区列表可以在 [IANA Time Zone Database](https://www.iana.org/time-zones) 中找到,也可以通过 `SELECT * FROM system.time_zones` 查询。[该列表](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) 也可以在 Wikipedia 上查看。 - -在创建表时,可以为 `DateTime` 类型的列显式设置时区。例如:`DateTime('UTC')`。如果未设置时区,ClickHouse 会使用服务器设置中 [timezone](../../operations/server-configuration-parameters/settings.md#timezone) 参数的值,或者 ClickHouse 服务器启动时操作系统中的时区设置。 - -如果在初始化数据类型时没有显式设置时区,[clickhouse-client](../../interfaces/cli.md) 会默认应用服务器时区。要使用客户端时区,请在运行 `clickhouse-client` 时加上 `--use_client_time_zone` 参数。 - -ClickHouse 会根据 [date_time_output_format](../../operations/settings/settings-formats.md#date_time_output_format) 设置的值输出时间值。默认的文本格式为 `YYYY-MM-DD hh:mm:ss`。此外,还可以使用 [formatDateTime](../../sql-reference/functions/date-time-functions.md#formatDateTime) 函数更改输出格式。 - -向 ClickHouse 插入数据时,可以使用不同格式的日期和时间字符串,具体取决于 [date_time_input_format](../../operations/settings/settings-formats.md#date_time_input_format) 设置的值。 - - - -## 示例 {#examples} - -**1.** 创建一个包含 `DateTime` 类型列的表,并向其中插入数据: - -```sql -CREATE TABLE dt -( - `timestamp` DateTime('Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 解析 DateTime --- - 从字符串解析, --- - 从整数解析(解释为自 1970-01-01 以来的秒数)。 -INSERT INTO dt VALUES ('2019-01-01 00:00:00', 1), (1546300800, 2); - -SELECT * FROM dt; -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -│ 2019-01-01 03:00:00 │ 2 │ -└─────────────────────┴──────────┘ -``` - -* 将整数插入到 `DateTime` 列时,会被视为 Unix 时间戳(UTC)。`1546300800` 表示 UTC 时间 `'2019-01-01 00:00:00'`。但是,由于 `timestamp` 列指定了 `Asia/Istanbul`(UTC+3)时区,在以字符串形式输出时,该值会显示为 `'2019-01-01 03:00:00'`。 -* 将字符串值插入到 `DateTime` 列时,会被视为处于该列的时区内。`'2019-01-01 00:00:00'` 会被视为 `Asia/Istanbul` 时区的时间,并以 `1546290000` 存储。 - -**2.** 基于 `DateTime` 值进行过滤 - -```sql -SELECT * FROM dt WHERE timestamp = toDateTime('2019-01-01 00:00:00', 'Asia/Istanbul') -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -可以在 `WHERE` 子句中使用字符串值来过滤 `DateTime` 列的值。该字符串会被自动转换为 `DateTime` 类型: - -```sql -SELECT * FROM dt WHERE timestamp = '2019-01-01 00:00:00' -``` - -```text -┌───────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00 │ 1 │ -└─────────────────────┴──────────┘ -``` - -**3.** 获取 `DateTime` 类型列的时区: - -```sql -SELECT toDateTime(now(), 'Asia/Istanbul') AS column, toTypeName(column) AS x -``` - -```text -┌──────────────column─┬─x─────────────────────────┐ -│ 2019-10-16 04:12:04 │ DateTime('Asia/Istanbul') │ -└─────────────────────┴───────────────────────────┘ -``` - -**4.** 时区转换 - -```sql -SELECT -toDateTime(timestamp, 'Europe/London') AS lon_time, -toDateTime(timestamp, 'Asia/Istanbul') AS mos_time -FROM dt -``` - -```text -┌───────────lon_time──┬────────────mos_time─┐ -│ 2019-01-01 00:00:00 │ 2019-01-01 03:00:00 │ -│ 2018-12-31 21:00:00 │ 2019-01-01 00:00:00 │ -└─────────────────────┴─────────────────────┘ -``` - -由于时区转换只会更改元数据,因此该操作不会产生计算开销。 - - -## 对时区支持的限制 {#limitations-on-time-zones-support} - -部分时区可能无法得到完全支持,主要存在以下几种情况: - -如果与 UTC 的偏移量不是 15 分钟的整数倍,则小时和分钟的计算可能不正确。比如,利比里亚蒙罗维亚的时区在 1972 年 1 月 7 日之前的偏移量是 UTC -0:44:30。如果你对蒙罗维亚的历史时间进行计算,时间处理函数可能会返回错误结果。不过,在 1972 年 1 月 7 日之后的结果则是正确的。 - -如果时间转换(由于夏令时或其他原因)发生在某个不是 15 分钟整数倍的时间点上,那么在这一天的这一特定时刻,你同样可能得到不正确的结果。 - -非单调的日历日期。例如,在 Happy Valley - Goose Bay,当地时间在 2010 年 11 月 7 日 00:01:00(午夜过后一分钟)向后拨回了一小时。因此,在 11 月 6 日结束后,人们先经历了完整的一分钟 11 月 7 日时间,随后时间又被拨回到 11 月 6 日 23:01,再过 59 分钟之后,11 月 7 日又重新开始。ClickHouse 目前尚不支持这种“有趣”的情况。在这些日期内,时间处理函数的结果可能会略有偏差。 - -类似的问题也出现在 2010 年的 Casey 南极站。他们在 3 月 5 日 02:00 将时间往回拨了三小时。如果你在南极站工作,请放心使用 ClickHouse。只要确保将时区设置为 UTC,或者清楚了解可能存在的误差即可。 - -跨多天的时间偏移。一些太平洋岛屿将其时区偏移从 UTC+14 改为 UTC-12。这本身没有问题,但如果你在处理这些地区时区的历史时间点,且时间恰好处在转换日期附近,可能会出现一定的不准确。 - - - -## 处理夏令时(DST) {#handling-daylight-saving-time-dst} - -ClickHouse 的带时区 `DateTime` 类型在夏令时(DST)切换期间可能会表现出意外行为,特别是在以下情况: - -* [`date_time_output_format`](../../operations/settings/settings-formats.md#date_time_output_format) 被设置为 `simple`。 -* 时钟被向后拨(“Fall Back”),导致出现一小时的重叠时间。 -* 时钟被向前拨(“Spring Forward”),导致出现一小时的时间缺口。 - -默认情况下,ClickHouse 始终选择重叠时间中较早的一次,并且在向前拨时可能会将本不应存在的时间解释为有效时间。 - -例如,考虑以下从夏令时切换到标准时间的场景。 - -* 在 2023 年 10 月 29 日 02:00:00,时钟被向后拨到 01:00:00(BST → GMT)。 -* 01:00:00 – 01:59:59 这一小时会出现两次(一次在 BST,一次在 GMT)。 -* ClickHouse 始终选择第一次出现的时间(BST),在增加时间间隔时会导致意外结果。 - -```sql -SELECT '2023-10-29 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-10-29 01:30:00 │ 2023-10-29 01:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -类似地,在从标准时间切换到夏令时的过程中,看起来会“跳过”一个小时。 - -例如: - -* 在 2023 年 3 月 26 日 `00:59:59` 时,时钟会向前跳到 02:00:00(GMT → BST)。 -* 时间段 `01:00:00` – `01:59:59` 实际上不存在。 - -```sql -SELECT '2023-03-26 01:30:00'::DateTime('Europe/London') AS time, time + toIntervalHour(1) AS one_hour_later - -┌────────────────time─┬──────one_hour_later─┐ -│ 2023-03-26 00:30:00 │ 2023-03-26 02:30:00 │ -└─────────────────────┴─────────────────────┘ -``` - -在这种情况下,ClickHouse 会将不存在的时间 `2023-03-26 01:30:00` 调整为 `2023-03-26 00:30:00`。 - - -## 另请参阅 {#see-also} - -- [类型转换函数](../../sql-reference/functions/type-conversion-functions.md) -- [日期和时间函数](../../sql-reference/functions/date-time-functions.md) -- [数组函数](../../sql-reference/functions/array-functions.md) -- [`date_time_input_format` 设置](../../operations/settings/settings-formats.md#date_time_input_format) -- [`date_time_output_format` 设置](../../operations/settings/settings-formats.md#date_time_output_format) -- [`timezone` 服务器配置参数](../../operations/server-configuration-parameters/settings.md#timezone) -- [`session_timezone` 设置](../../operations/settings/settings.md#session_timezone) -- [日期和时间运算符](../../sql-reference/operators#operators-for-working-with-dates-and-times) -- [`Date` 数据类型](../../sql-reference/data-types/date.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md deleted file mode 100644 index 509b440b73c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/datetime64.md +++ /dev/null @@ -1,131 +0,0 @@ ---- -description: 'ClickHouse 中 DateTime64 数据类型文档,该类型用于存储具有亚秒级精度的时间戳' -sidebar_label: 'DateTime64' -sidebar_position: 18 -slug: /sql-reference/data-types/datetime64 -title: 'DateTime64' -doc_type: 'reference' ---- - -# DateTime64 {#datetime64} - -用于存储某一瞬时时刻,该时刻可以表示为日历日期和一天中的时间,并具有可配置的子秒级精度。 - -时间刻度(精度):10-precision 秒。有效范围:[ 0 : 9 ]。 -常用的取值有 3(毫秒)、6(微秒)、9(纳秒)。 - -**语法:** - -```sql -DateTime64(精度, [时区]) -``` - -在内部,数据以自纪元开始(1970-01-01 00:00:00 UTC)以来的若干个“tick”形式存储为 Int64。tick 的时间分辨率由 `precision` 参数决定。此外,`DateTime64` 类型可以存储一个对整列统一生效的时区,该时区会影响 `DateTime64` 类型值在文本格式中的显示方式,以及将字符串形式的值(如 `2020-01-01 05:00:01.000`)解析为 `DateTime64` 时的方式。时区不会存储在表的行(或结果集)中,而是存储在列的元数据中。详情参见 [DateTime](../../sql-reference/data-types/datetime.md)。 - -支持的取值范围:[1900-01-01 00:00:00, 2299-12-31 23:59:59.999999999] - -小数点后的位数取决于 `precision` 参数。 - -注意:最大值的精度为 8。如果使用 9 位(纳秒级)的最大精度,则在 UTC 时区下支持的最大值为 `2262-04-11 23:47:16`。 - -## 示例 {#examples} - -1. 创建一个包含 `DateTime64` 类型列的表,并向其中插入数据: - -```sql -CREATE TABLE dt64 -( - `timestamp` DateTime64(3, 'Asia/Istanbul'), - `event_id` UInt8 -) -ENGINE = TinyLog; -``` - -```sql --- 解析 DateTime --- - 从整数解析,此整数被解释为自 1970-01-01 起的微秒数(精度为 3), --- - 从小数解析,小数点前部分被解释为秒数,小数点后部分则根据精度表示小数秒, --- - 从字符串解析。 -INSERT INTO dt64 VALUES (1546300800123, 1), (1546300800.123, 2), ('2019-01-01 00:00:00', 3); - -SELECT * FROM dt64; -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -* 当以整数形式插入 datetime 时,它会被视为按相应精度缩放的 Unix 时间戳(UTC)。`1546300800000`(精度为 3)表示 UTC 的 `'2019-01-01 00:00:00'`。但是,由于 `timestamp` 列指定的时区是 `Asia/Istanbul`(UTC+3),在以字符串形式输出时,该值会显示为 `'2019-01-01 03:00:00'`。当以小数形式插入 datetime 时,其处理方式与整数类似,只是小数点前的值是精确到秒的 Unix 时间戳,小数点后的部分将被视为精度。 -* 当以字符串形式插入 datetime 值时,它会被视为处于该列所使用的时区中。`'2019-01-01 00:00:00'` 将被视为处于 `Asia/Istanbul` 时区,并以 `1546290000000` 的形式存储。 - -2. 对 `DateTime64` 值进行过滤 - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64('2019-01-01 00:00:00', 3, 'Asia/Istanbul'); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 00:00:00.000 │ 3 │ -└─────────────────────────┴──────────┘ -``` - -与 `DateTime` 不同,`DateTime64` 类型的值不会自动由 `String` 转换而来。 - -```sql -SELECT * FROM dt64 WHERE timestamp = toDateTime64(1546300800.123, 3); -``` - -```text -┌───────────────timestamp─┬─event_id─┐ -│ 2019-01-01 03:00:00.123 │ 1 │ -│ 2019-01-01 03:00:00.123 │ 2 │ -└─────────────────────────┴──────────┘ -``` - -与插入操作不同,`toDateTime64` 函数会将所有值视为小数形式,因此需要在小数点后指定精度。 - -3. 获取 `DateTime64` 类型值的时区: - -```sql -SELECT toDateTime64(now(), 3, 'Asia/Istanbul') AS column, toTypeName(column) AS x; -``` - -```text -┌──────────────────column─┬─x──────────────────────────────┐ -│ 2023-06-05 00:09:52.000 │ DateTime64(3, 'Asia/Istanbul') │ -└─────────────────────────┴────────────────────────────────┘ -``` - -4. 时区转换 - -```sql -SELECT -toDateTime64(timestamp, 3, 'Europe/London') AS lon_time, -toDateTime64(timestamp, 3, 'Asia/Istanbul') AS istanbul_time -FROM dt64; -``` - -```text -┌────────────────lon_time─┬───────────istanbul_time─┐ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2019-01-01 00:00:00.123 │ 2019-01-01 03:00:00.123 │ -│ 2018-12-31 21:00:00.000 │ 2019-01-01 00:00:00.000 │ -└─────────────────────────┴─────────────────────────┘ -``` - -**另请参阅** - -* [类型转换函数](../../sql-reference/functions/type-conversion-functions.md) -* [用于处理日期和时间的函数](../../sql-reference/functions/date-time-functions.md) -* [`date_time_input_format` 设置](../../operations/settings/settings-formats.md#date_time_input_format) -* [`date_time_output_format` 设置](../../operations/settings/settings-formats.md#date_time_output_format) -* [`timezone` 服务器配置参数](../../operations/server-configuration-parameters/settings.md#timezone) -* [`session_timezone` 设置](../../operations/settings/settings.md#session_timezone) -* [用于处理日期和时间的运算符](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [`Date` 数据类型](../../sql-reference/data-types/date.md) -* [`DateTime` 数据类型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md deleted file mode 100644 index 97ffdca0b33..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/decimal.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -description: 'ClickHouse 中 Decimal 数据类型的文档,这些类型提供具有可配置精度的定点数运算' -sidebar_label: 'Decimal' -sidebar_position: 6 -slug: /sql-reference/data-types/decimal -title: 'Decimal、Decimal(P)、Decimal(P, S)、Decimal32(S)、Decimal64(S)、Decimal128(S)、Decimal256(S)' -doc_type: 'reference' ---- - - - -# Decimal, Decimal(P), Decimal(P, S), Decimal32(S), Decimal64(S), Decimal128(S), Decimal256(S) {#decimal-decimalp-decimalp-s-decimal32s-decimal64s-decimal128s-decimal256s} - -有符号定点数,在执行加法、减法和乘法运算时保持精度。对于除法运算,最低有效数字会被直接截断(不进行四舍五入)。 - - - -## 参数 {#parameters} - -- P - 精度。有效范围:\[ 1 : 76 \]。决定数字最多可以包含多少位十进制数字(包括小数部分)。默认精度为 10。 -- S - 标度。有效范围:\[ 0 : P \]。决定小数部分最多可以包含多少位十进制数字。 - -Decimal(P) 等价于 Decimal(P, 0)。类似地,语法 Decimal 等价于 Decimal(10, 0)。 - -根据参数 P 的取值,Decimal(P, S) 等价于: -- P 在 \[ 1 : 9 \] 范围内 —— Decimal32(S) -- P 在 \[ 10 : 18 \] 范围内 —— Decimal64(S) -- P 在 \[ 19 : 38 \] 范围内 —— Decimal128(S) -- P 在 \[ 39 : 76 \] 范围内 —— Decimal256(S) - - - -## Decimal 值范围 {#decimal-value-ranges} - -- Decimal(P, S) - ( -1 \* 10^(P - S), 1 \* 10^(P - S) ) -- Decimal32(S) - ( -1 \* 10^(9 - S), 1 \* 10^(9 - S) ) -- Decimal64(S) - ( -1 \* 10^(18 - S), 1 \* 10^(18 - S) ) -- Decimal128(S) - ( -1 \* 10^(38 - S), 1 \* 10^(38 - S) ) -- Decimal256(S) - ( -1 \* 10^(76 - S), 1 \* 10^(76 - S) ) - -例如,Decimal32(4) 的取值范围为 -99999.9999 到 99999.9999,步长为 0.0001。 - - - -## 内部表示 {#internal-representation} - -在内部,数据表示为具有相应位宽的普通有符号整数。可在内存中存储的实际取值范围比上面指定的范围略大,只会在从字符串转换时进行范围检查。 - -由于现代 CPU 并不原生支持 128 位和 256 位整数,对 Decimal128 和 Decimal256 的运算是通过仿真实现的。因此,Decimal128 和 Decimal256 的运行速度明显慢于 Decimal32/Decimal64。 - - - -## 运算和结果类型 {#operations-and-result-type} - -在 Decimal 上执行二元运算时,结果类型会被提升为位宽更大的类型(与参数顺序无关)。 - -- `Decimal64(S1) Decimal32(S2) -> Decimal64(S)` -- `Decimal128(S1) Decimal32(S2) -> Decimal128(S)` -- `Decimal128(S1) Decimal64(S2) -> Decimal128(S)` -- `Decimal256(S1) Decimal<32|64|128>(S2) -> Decimal256(S)` - -scale 的规则: - -- 加、减:S = max(S1, S2)。 -- 乘:S = S1 + S2。 -- 除:S = S1。 - -对于 Decimal 与整数之间的类似运算,结果是与参与运算的 Decimal 参数位宽相同的 Decimal 类型。 - -Decimal 与 Float32/Float64 之间的运算未定义。如有需要,可以显式使用 toDecimal32、toDecimal64、toDecimal128 或 toFloat32、toFloat64 内置函数对其中一个参数进行类型转换。请注意,结果会丢失精度,并且类型转换是计算开销较大的操作。 - -某些作用于 Decimal 的函数会返回 Float64 结果(例如 var 或 stddev)。中间计算仍可能以 Decimal 进行,这可能导致在数值相同的前提下,使用 Float64 输入与使用 Decimal 输入得到的结果不同。 - - - -## 溢出检查 {#overflow-checks} - -在对 Decimal 类型进行计算时,可能会发生整数溢出。小数部分中的多余位数会被直接截断(不会进行四舍五入)。整数部分中的多余位数会导致抛出异常。 - -:::warning -Decimal128 和 Decimal256 尚未实现溢出检查。在发生溢出的情况下,会返回不正确的结果,不会抛出异常。 -::: - -```sql -SELECT toDecimal32(2, 4) AS x, x / 3 -``` - -```text -┌──────x─┬─divide(toDecimal32(2, 4), 3)─┐ -│ 2.0000 │ 0.6666 │ -└────────┴──────────────────────────────┘ -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, x * x -``` - -```text -DB::Exception: 小数位数超出范围。 -``` - -```sql -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -DB::Exception: 十进制数值运算溢出。 -``` - -溢出检查会导致运算变慢。如果能够确定不会发生溢出,则可以通过 `decimal_check_overflow` 设置来禁用检查。当检查被禁用且发生溢出时,结果将会不正确: - -```sql -SET decimal_check_overflow = 0; -SELECT toDecimal32(4.2, 8) AS x, 6 * x -``` - -```text -┌──────────x─┬─multiply(6, toDecimal32(4.2, 8))─┐ -│ 4.20000000 │ -17.74967296 │ -└────────────┴──────────────────────────────────┘ -``` - -溢出检查不仅会在算术运算中进行,也会在值比较时进行: - -```sql -SELECT toDecimal32(1, 8) < 100 -``` - -```text -DB::Exception: 无法进行比较。 -``` - -**另请参阅** - -* [isDecimalOverflow](/sql-reference/functions/other-functions#isDecimalOverflow) -* [countDigits](/sql-reference/functions/other-functions#countDigits) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md deleted file mode 100644 index 8bb5e9b95f6..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/domains/index.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -description: 'ClickHouse 中域类型的概述,它们在基础类型之上扩展了额外功能' -sidebar_label: '域' -sidebar_position: 56 -slug: /sql-reference/data-types/domains/ -title: '域' -doc_type: 'reference' ---- - - - -# 域(Domains) {#domains} - -域是一种具有特定用途的类型,它在现有基础类型之上增加了额外功能,同时保持底层数据类型在网络传输和磁盘存储时的格式不变。目前,ClickHouse 不支持用户自定义域。 - -在任何可以使用对应基础类型的地方,都可以使用域,例如: - -- 创建一个域类型的列 -- 从域列读取值 / 向域列写入值 -- 如果基础类型可以用作索引,则也可以将该域用作索引 -- 在调用函数时使用域列的值 - -### 域的额外功能 {#extra-features-of-domains} - -- 在 `SHOW CREATE TABLE` 或 `DESCRIBE TABLE` 中显示明确的列类型名称 -- 以更适合人工阅读的格式输入:`INSERT INTO domain_table(domain_column) VALUES(...)` -- 以更适合人工阅读的格式输出:`SELECT domain_column FROM domain_table` -- 以更适合人工阅读的格式,从外部数据源加载数据:`INSERT INTO domain_table FORMAT CSV ...` - -### 限制 {#limitations} - -- 无法通过 `ALTER TABLE` 将基础类型的索引列转换为域类型。 -- 在从其他列或表插入数据时,无法将字符串值隐式转换为域值。 -- 域不会对存储的值施加任何约束。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md deleted file mode 100644 index 3ce64bf9a21..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/dynamic.md +++ /dev/null @@ -1,716 +0,0 @@ ---- -description: 'ClickHouse 中 Dynamic 数据类型的文档,它能够在单个列中存储多种类型的值' -sidebar_label: 'Dynamic' -sidebar_position: 62 -slug: /sql-reference/data-types/dynamic -title: 'Dynamic' -doc_type: 'guide' ---- - -# Dynamic {#dynamic} - -此类型允许在其中存储任意类型的值,而无需预先了解或枚举所有可能的类型。 - -要声明一个 `Dynamic` 类型的列,请使用以下语法: - -```sql -<列名> Dynamic(max_types=N) -``` - -其中 `N` 是一个可选参数,取值范围为 `0` 到 `254`,用于指示在一个单独存储的数据块内(例如 MergeTree 表的单个数据分片中),类型为 `Dynamic` 的列中可以作为独立子列存储的不同数据类型的数量上限。如果超过此限制,所有具有新类型的值将以二进制形式一起存储在一个特殊的共享数据结构中。`max_types` 的默认值为 `32`。 - -## 创建 Dynamic 类型 {#creating-dynamic} - -在表的列定义中使用 `Dynamic` 类型: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ Hello, World! │ String │ -│ [1,2,3] │ Array(Int64) │ -└───────────────┴────────────────┘ -``` - -在普通列上使用 CAST: - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -对 `Variant` 列使用 CAST: - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 3) = 0, number, (number % 3) = 1, range(number + 1), NULL)::Dynamic AS d, dynamicType(d) FROM numbers(3) -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 0 │ UInt64 │ -│ [0,1] │ Array(UInt64) │ -│ ᴺᵁᴸᴸ │ None │ -└───────┴────────────────┘ -``` - -## 以子列形式读取 Dynamic 中的嵌套类型 {#reading-dynamic-nested-types-as-subcolumns} - -`Dynamic` 类型支持通过类型名作为子列名,从 `Dynamic` 列中读取单个嵌套类型。 -因此,如果存在一列 `d Dynamic`,可以使用语法 `d.T` 读取任意有效类型 `T` 的子列, -当 `T` 可以被 `Nullable` 包裹时,该子列的类型为 `Nullable(T)`,否则为 `T`。该子列的大小将与原始 `Dynamic` 列相同, -并且在原始 `Dynamic` 列中不包含类型 `T` 的所有行里,该子列将包含 `NULL` 值(或者在 `T` 不能被 `Nullable` 包裹时为空值)。 - -也可以使用函数 `dynamicElement(dynamic_column, type_name)` 来读取 `Dynamic` 的子列。 - -示例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT d, dynamicType(d), d.String, d.Int64, d.`Array(Int64)`, d.Date, d.`Array(String)` FROM test; -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─d.String──────┬─d.Int64─┬─d.Array(Int64)─┬─d.Date─┬─d.Array(String)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴───────────────┴─────────┴────────────────┴────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(d.String), toTypeName(d.Int64), toTypeName(d.`Array(Int64)`), toTypeName(d.Date), toTypeName(d.`Array(String)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(d.String)─┬─toTypeName(d.Int64)─┬─toTypeName(d.Array(Int64))─┬─toTypeName(d.Date)─┬─toTypeName(d.Array(String))─┐ -│ Nullable(String) │ Nullable(Int64) │ Array(Int64) │ Nullable(Date) │ Array(String) │ -└──────────────────────┴─────────────────────┴────────────────────────────┴────────────────────┴─────────────────────────────┘ -``` - -````sql -SELECT d, dynamicType(d), dynamicElement(d, 'String'), dynamicElement(d, 'Int64'), dynamicElement(d, 'Array(Int64)'), dynamicElement(d, 'Date'), dynamicElement(d, 'Array(String)') FROM test;``` -```` - -```text -┌─d─────────────┬─dynamicType(d)─┬─dynamicElement(d, 'String')─┬─dynamicElement(d, 'Int64')─┬─dynamicElement(d, 'Array(Int64)')─┬─dynamicElement(d, 'Date')─┬─dynamicElement(d, 'Array(String)')─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ [] │ ᴺᵁᴸᴸ │ [] │ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ ᴺᵁᴸᴸ │ [] │ -└───────────────┴────────────────┴─────────────────────────────┴────────────────────────────┴───────────────────────────────────┴───────────────────────────┴────────────────────────────────────┘ -``` - -要了解每一行中存储的是哪种变体类型,可以使用函数 `dynamicType(dynamic_column)`。它会返回一个 `String`,表示每一行的值类型名称(如果该行为 `NULL`,则返回 `'None'`)。 - -示例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT dynamicType(d) FROM test; -``` - -```text -┌─dynamicType(d)─┐ -│ None │ -│ Int64 │ -│ String │ -│ Array(Int64) │ -└────────────────┘ -``` - -## Dynamic 列与其他列之间的转换 {#conversion-between-dynamic-column-and-other-columns} - -`Dynamic` 列支持 4 种转换方式。 - -### 将普通列转换为 Dynamic 列 {#converting-an-ordinary-column-to-a-dynamic-column} - -```sql -SELECT 'Hello, World!'::Dynamic AS d, dynamicType(d); -``` - -```text -┌─d─────────────┬─dynamicType(d)─┐ -│ Hello, World! │ String │ -└───────────────┴────────────────┘ -``` - -### 通过解析将 String 列转换为 Dynamic 列 {#converting-a-string-column-to-a-dynamic-column-through-parsing} - -要从 `String` 列中解析出 `Dynamic` 类型的值,可以开启 `cast_string_to_dynamic_use_inference` 设置: - -```sql -SET cast_string_to_dynamic_use_inference = 1; -SELECT CAST(materialize(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01')), 'Map(String, Dynamic)') as map_of_dynamic, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamic) as map_of_dynamic_types; -``` - -```text -┌─map_of_dynamic──────────────────────────────┬─map_of_dynamic_types─────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'Int64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴──────────────────────────────────────────────┘ -``` - -### 将 `Dynamic` 列转换为普通列 {#converting-a-dynamic-column-to-an-ordinary-column} - -可以将 `Dynamic` 列转换为普通列。此时,所有嵌套类型都会被转换为目标类型: - -```sql -CREATE TABLE test (d Dynamic) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'), (true), ('e10'); -SELECT d::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(d, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -│ 1 │ -│ 0 │ -└──────────────────────────────┘ -``` - -### 将 Variant 列转换为 Dynamic 列 {#converting-a-variant-column-to-dynamic-column} - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'), ([1, 2, 3]); -SELECT v::Dynamic AS d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ UInt64 │ -│ String │ String │ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴────────────────┘ -``` - -### 将 Dynamic(max_types=N) 列转换为另一列 Dynamic(max_types=K) {#converting-a-dynamicmax_typesn-column-to-another-dynamicmax_typesk} - -如果 `K >= N`,则在转换过程中数据不会被更改: - -```sql -CREATE TABLE test (d Dynamic(max_types=3)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true); -SELECT d::Dynamic(max_types=5) as d2, dynamicType(d2) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ 42.42 │ String │ -│ true │ Bool │ -└───────┴────────────────┘ -``` - -如果 `K < N`,则具有最罕见类型的值将被插入到一个特殊子列中,但这些值仍然是可访问的: - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=2) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ false │ -│ 43 │ Int64 │ 43 │ Int64 │ false │ -│ 42.42 │ String │ 42.42 │ String │ false │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -函数 `isDynamicElementInSharedData` 对存储在 `Dynamic` 内部特殊共享数据结构中的行返回 `true`,并且如我们所见,结果列中只包含 2 种未存储在该共享数据结构中的类型。 - -如果 `K=0`,所有类型都会被插入到单个特殊子列中: - -```text -CREATE TABLE test (d Dynamic(max_types=4)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), (43), ('42.42'), (true), ([1, 2, 3]); -SELECT d, dynamicType(d), d::Dynamic(max_types=0) as d2, dynamicType(d2), isDynamicElementInSharedData(d2) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┬─d2──────┬─dynamicType(d2)─┬─isDynamicElementInSharedData(d2)─┐ -│ ᴺᵁᴸᴸ │ None │ ᴺᵁᴸᴸ │ None │ false │ -│ 42 │ Int64 │ 42 │ Int64 │ true │ -│ 43 │ Int64 │ 43 │ Int64 │ true │ -│ 42.42 │ String │ 42.42 │ String │ true │ -│ true │ Bool │ true │ Bool │ true │ -│ [1,2,3] │ Array(Int64) │ [1,2,3] │ Array(Int64) │ true │ -└─────────┴────────────────┴─────────┴─────────────────┴──────────────────────────────────┘ -``` - -## 从数据中读取 Dynamic 类型 {#reading-dynamic-type-from-the-data} - -所有文本格式(TSV、CSV、CustomSeparated、Values、JSONEachRow 等)都支持读取 `Dynamic` 类型。在解析数据时,ClickHouse 会尝试推断每个值的类型,并在向 `Dynamic` 列插入数据时使用该类型。 - -示例: - -```sql -SELECT - d, - dynamicType(d), - dynamicElement(d, 'String') AS str, - dynamicElement(d, 'Int64') AS num, - dynamicElement(d, 'Float64') AS float, - dynamicElement(d, 'Date') AS date, - dynamicElement(d, 'Array(Int64)') AS arr -FROM format(JSONEachRow, 'd Dynamic', $$ -{"d" : "Hello, World!"}, -{"d" : 42}, -{"d" : 42.42}, -{"d" : "2020-01-01"}, -{"d" : [1, 2, 3]} -$$) -``` - -```text -┌─d─────────────┬─dynamicType(d)─┬─str───────────┬──num─┬─float─┬───────date─┬─arr─────┐ -│ Hello, World! │ String │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ Int64 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ Float64 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 │ Date │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ [] │ -│ [1,2,3] │ Array(Int64) │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴────────────────┴───────────────┴──────┴───────┴────────────┴─────────┘ -``` - -## 在函数中使用 Dynamic 类型 {#using-dynamic-type-in-functions} - -大多数函数都支持类型为 `Dynamic` 的参数。在这种情况下,函数会分别在 `Dynamic` 列中存储的每种内部数据类型上执行。 -当函数的结果类型依赖于参数类型时,使用 `Dynamic` 参数执行此类函数的结果类型将是 `Dynamic`。当函数的结果类型不依赖于参数类型时,结果类型将是 `Nullable(T)`,其中 `T` 是该函数在普通(非 Dynamic)情况下的结果类型。 - -示例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (NULL), (1::Int8), (2::Int16), (3::Int32), (4::Int64); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ 1 │ Int8 │ -│ 2 │ Int16 │ -│ 3 │ Int32 │ -│ 4 │ Int64 │ -└──────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 3 │ Dynamic │ Int32 │ -│ 3 │ 4 │ Dynamic │ Int64 │ -│ 4 │ 5 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d + d AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ 1 │ 2 │ Dynamic │ Int16 │ -│ 2 │ 4 │ Dynamic │ Int32 │ -│ 3 │ 6 │ Dynamic │ Int64 │ -│ 4 │ 8 │ Dynamic │ Int64 │ -└──────┴──────┴─────────────────┴──────────────────┘ -``` - -```sql -SELECT d, d < 3 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(UInt8) │ -│ 1 │ 1 │ Nullable(UInt8) │ -│ 2 │ 1 │ Nullable(UInt8) │ -│ 3 │ 0 │ Nullable(UInt8) │ -│ 4 │ 0 │ Nullable(UInt8) │ -└──────┴──────┴─────────────────┘ -``` - -```sql -SELECT d, exp2(d) AS res, toTypeName(res) FROM test; -``` - -```sql -┌─d────┬──res─┬─toTypeName(res)───┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Float64) │ -│ 1 │ 2 │ Nullable(Float64) │ -│ 2 │ 4 │ Nullable(Float64) │ -│ 3 │ 8 │ Nullable(Float64) │ -│ 4 │ 16 │ Nullable(Float64) │ -└──────┴──────┴───────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ('str_1'), ('str_2'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ str_1 │ String │ -│ str_2 │ String │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, upper(d) AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res───┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ STR_1 │ Nullable(String) │ -│ str_2 │ STR_2 │ Nullable(String) │ -└───────┴───────┴──────────────────┘ -``` - -```sql -SELECT d, extract(d, '([0-3])') AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)──┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(String) │ -│ str_1 │ 1 │ Nullable(String) │ -│ str_2 │ 2 │ Nullable(String) │ -└───────┴──────┴──────────────────┘ -``` - -```sql -TRUNCATE TABLE test; -INSERT INTO test VALUES (NULL), ([1, 2]), ([3, 4]); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d[1] AS res, toTypeName(res), dynamicType(res) FROM test; -``` - -```text -┌─d─────┬─res──┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Dynamic │ None │ -│ [1,2] │ 1 │ Dynamic │ Int64 │ -│ [3,4] │ 3 │ Dynamic │ Int64 │ -└───────┴──────┴─────────────────┴──────────────────┘ -``` - -如果某个函数无法对 `Dynamic` 列中的某种类型执行,就会抛出异常: - -```sql -INSERT INTO test VALUES (42), (43), ('str_1'); -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d─────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ str_1 │ String │ -└───────┴────────────────┘ -┌─d─────┬─dynamicType(d)─┐ -│ ᴺᵁᴸᴸ │ None │ -│ [1,2] │ Array(Int64) │ -│ [3,4] │ Array(Int64) │ -└───────┴────────────────┘ -``` - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(d) FROM test; -``` - -```text -收到异常: -Code: 43. DB::Exception: 函数 plus 的参数类型不合法:Array(Int64) 和 UInt8,执行 'FUNCTION plus(__table1.d : 3, 1_UInt8 :: 1) -> plus(__table1.d, 1_UInt8) Dynamic : 0' 时。(ILLEGAL_TYPE_OF_ARGUMENT) -``` - -我们可以过滤掉不需要的类型: - -```sql -SELECT d, d + 1 AS res, toTypeName(res), dynamicType(res) FROM test WHERE dynamicType(d) NOT IN ('String', 'Array(Int64)', 'None') -``` - -```text -┌─d──┬─res─┬─toTypeName(res)─┬─dynamicType(res)─┐ -│ 42 │ 43 │ Dynamic │ Int64 │ -│ 43 │ 44 │ Dynamic │ Int64 │ -└────┴─────┴─────────────────┴──────────────────┘ -``` - -或者将所需的类型提取到子列中: - -```sql -SELECT d, d.Int64 + 1 AS res, toTypeName(res) FROM test; -``` - -```text -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ 42 │ 43 │ Nullable(Int64) │ -│ 43 │ 44 │ Nullable(Int64) │ -│ str_1 │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -┌─d─────┬──res─┬─toTypeName(res)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [1,2] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -│ [3,4] │ ᴺᵁᴸᴸ │ Nullable(Int64) │ -└───────┴──────┴─────────────────┘ -``` - -## 在 ORDER BY 和 GROUP BY 中使用 Dynamic 类型 {#using-dynamic-type-in-order-by-and-group-by} - -在执行 `ORDER BY` 和 `GROUP BY` 时,`Dynamic` 类型的值会以类似于 `Variant` 类型的方式进行比较: -对于类型为 `Dynamic` 的值 `d1`(其底层类型为 `T1`)和 `d2`(其底层类型为 `T2`),运算符 `<` 的结果定义如下: - -* 如果 `T1 = T2 = T`,则结果为 `d1.T < d2.T`(比较底层值)。 -* 如果 `T1 != T2`,则结果为 `T1 < T2`(比较类型名称)。 - -默认情况下,在 `GROUP BY`/`ORDER BY` 键中不允许使用 `Dynamic` 类型;如果确有需要,请结合其上述特殊比较规则,并启用 `allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by` 设置。 - -示例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (42), (43), ('abc'), ('abd'), ([1, 2, 3]), ([]), (NULL); -``` - -```sql -SELECT d, dynamicType(d) FROM test; -``` - -```text -┌─d───────┬─dynamicType(d)─┐ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ [1,2,3] │ Array(Int64) │ -│ [] │ Array(Int64) │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```sql -┌─d───────┬─dynamicType(d)─┐ -│ [] │ Array(Int64) │ -│ [1,2,3] │ Array(Int64) │ -│ 42 │ Int64 │ -│ 43 │ Int64 │ -│ abc │ String │ -│ abd │ String │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴────────────────┘ -``` - -**注意:** dynamic 类型中具有不同数值类型的值会被视为不同的值,彼此之间不会进行比较,而是仅比较它们的类型名称。 - -示例: - -```sql -CREATE TABLE test (d Dynamic) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT d, dynamicType(d) FROM test ORDER BY d SETTINGS allow_suspicious_types_in_order_by=1; -``` - -```text -┌─v───┬─dynamicType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -```sql -SELECT d, dynamicType(d) FROM test GROUP BY d SETTINGS allow_suspicious_types_in_group_by=1; -``` - -```text -┌─d───┬─dynamicType(d)─┐ -│ 1 │ Int64 │ -│ 100 │ UInt32 │ -│ 1 │ UInt32 │ -│ 100 │ Int64 │ -└─────┴────────────────┘ -``` - -**注意:** 由于对 `Dynamic` 类型的函数有[特殊处理](#using-dynamic-type-in-functions),此处描述的比较规则在执行 `<`/`>`/`=` 等比较函数时不会生效。 - -## 在 Dynamic 中存储的不同数据类型数量达到上限 {#reaching-the-limit-in-number-of-different-data-types-stored-inside-dynamic} - -`Dynamic` 数据类型只能将有限数量的不同数据类型作为单独的子列进行存储。默认情况下,该上限为 32,但你可以在类型声明中使用语法 `Dynamic(max_types=N)` 进行修改,其中 N 的取值范围是 0 到 254(由于实现细节的限制,`Dynamic` 中作为单独子列存储的不同数据类型数量不可能超过 254)。 -当达到该上限时,插入到 `Dynamic` 列中的所有新的数据类型的值将被写入到单一的共享数据结构中,该结构以二进制形式存储具有不同数据类型的值。 - -下面看看在不同场景下达到上限时会发生什么。 - -### 在数据解析过程中达到上限 {#reaching-the-limit-during-data-parsing} - -在从数据中解析 `Dynamic` 值的过程中,当当前数据块中的类型数量达到上限时,所有新的值都会被插入到共享数据结构中: - -```sql -SELECT d, dynamicType(d), isDynamicElementInSharedData(d) FROM format(JSONEachRow, 'd Dynamic(max_types=3)', ' -{"d" : 42} -{"d" : [1, 2, 3]} -{"d" : "Hello, World!"} -{"d" : "2020-01-01"} -{"d" : ["str1", "str2", "str3"]} -{"d" : {"a" : 1, "b" : [1, 2, 3]}} -') -``` - -```text -┌─d──────────────────────┬─dynamicType(d)─────────────────┬─isDynamicElementInSharedData(d)─┐ -│ 42 │ Int64 │ false │ -│ [1,2,3] │ Array(Int64) │ false │ -│ Hello, World! │ String │ false │ -│ 2020-01-01 │ Date │ true │ -│ ['str1','str2','str3'] │ Array(String) │ true │ -│ (1,[1,2,3]) │ Tuple(a Int64, b Array(Int64)) │ true │ -└────────────────────────┴────────────────────────────────┴─────────────────────────────────┘ -``` - -可以看到,在插入 3 种不同的数据类型 `Int64`、`Array(Int64)` 和 `String` 之后,所有新类型都被插入到了特殊的共享数据结构中。 - -### 在 MergeTree 表引擎中合并数据片段时 {#during-merges-of-data-parts-in-mergetree-table-engines} - -在 MergeTree 表中合并多个数据片段时,结果数据片段中的 `Dynamic` 列可能会达到其内部可存储为独立子列的不同数据类型数量上限,从而无法再将所有类型都作为来自源数据片段的子列进行存储。 -在这种情况下,ClickHouse 会选择在合并后哪些类型继续作为独立子列保留,哪些类型需要被插入到共享数据结构中。在大多数情况下,ClickHouse 会尝试保留最常见的类型,并将最罕见的类型存入共享数据结构,但这取决于具体实现。 - -下面来看一个这样的合并示例。首先,创建一个包含 `Dynamic` 列的表,将不同数据类型数量的上限设置为 `3`,并插入包含 `5` 种不同类型的值: - -```sql -CREATE TABLE test (id UInt64, d Dynamic(max_types=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, number FROM numbers(5); -INSERT INTO test SELECT number, range(number) FROM numbers(4); -INSERT INTO test SELECT number, toDate(number) FROM numbers(3); -INSERT INTO test SELECT number, map(number, number) FROM numbers(2); -INSERT INTO test SELECT number, 'str_' || toString(number) FROM numbers(1); -``` - -每次插入都会创建一个单独的数据片段,其中 `Dynamic` 列只包含一种类型: - -```sql -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count(); -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_1_0 │ -│ 4 │ Array(UInt64) │ false │ all_2_2_0 │ -│ 3 │ Date │ false │ all_3_3_0 │ -│ 2 │ Map(UInt64, UInt64) │ false │ all_4_4_0 │ -│ 1 │ String │ false │ all_5_5_0 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -现在,我们把所有部分合在一起,看看会发生什么: - -```sql -SYSTEM START MERGES test; -OPTIMIZE TABLE test FINAL; -SELECT count(), dynamicType(d), isDynamicElementInSharedData(d), _part FROM test GROUP BY _part, dynamicType(d), isDynamicElementInSharedData(d) ORDER BY _part, count() desc; -``` - -```text -┌─count()─┬─dynamicType(d)──────┬─isDynamicElementInSharedData(d)─┬─_part─────┐ -│ 5 │ UInt64 │ false │ all_1_5_2 │ -│ 4 │ Array(UInt64) │ false │ all_1_5_2 │ -│ 3 │ Date │ false │ all_1_5_2 │ -│ 2 │ Map(UInt64, UInt64) │ true │ all_1_5_2 │ -│ 1 │ String │ true │ all_1_5_2 │ -└─────────┴─────────────────────┴─────────────────────────────────┴───────────┘ -``` - -正如我们所见,ClickHouse 将最常见的类型 `UInt64` 和 `Array(UInt64)` 保留为子列,并将所有其他类型写入共享数据区。 - -## 支持 Dynamic 类型的 JSONExtract 函数 {#jsonextract-functions-with-dynamic} - -所有 `JSONExtract*` 函数都支持 `Dynamic` 类型: - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Dynamic') AS dynamic, dynamicType(dynamic) AS dynamic_type; -``` - -```text -┌─dynamic─┬─dynamic_type───────────┐ -│ [1,2,3] │ Array(Nullable(Int64)) │ -└─────────┴────────────────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Dynamic)') AS map_of_dynamics, mapApply((k, v) -> (k, dynamicType(v)), map_of_dynamics) AS map_of_dynamic_types -``` - -```text -┌─map_of_dynamics──────────────────┬─map_of_dynamic_types────────────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'Int64','b':'String','c':'Array(Nullable(Int64))'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────────────┘ -``` - -````sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Dynamic') AS dynamics, arrayMap(x -> (x.1, dynamicType(x.2)), dynamics) AS dynamic_types``` -```` - -```text -┌─dynamics───────────────────────────────┬─dynamic_types─────────────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','Int64'),('b','String'),('c','Array(Nullable(Int64))')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────────────┘ -``` - -### 二进制输出格式 {#binary-output-format} - -在 RowBinary 格式中,`Dynamic` 类型的值以如下格式序列化: - -```text - -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md deleted file mode 100644 index bf8df43c99e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/enum.md +++ /dev/null @@ -1,165 +0,0 @@ ---- -description: 'ClickHouse 中 Enum 数据类型的文档,用于表示一组具名常量值' -sidebar_label: 'Enum' -sidebar_position: 20 -slug: /sql-reference/data-types/enum -title: 'Enum' -doc_type: 'reference' ---- - -# Enum {#enum} - -由具名值组成的枚举类型。 - -具名值可以声明为 `'string' = integer` 键值对,或仅声明为 `'string'` 名称。ClickHouse 只存储数字,但支持通过名称对这些值进行操作。 - -ClickHouse 支持: - -* 8 位 `Enum`,最多可以包含 256 个值,枚举范围为 `[-128, 127]`。 -* 16 位 `Enum`,最多可以包含 65536 个值,枚举范围为 `[-32768, 32767]`。 - -在插入数据时,ClickHouse 会自动选择 `Enum` 的类型。也可以使用 `Enum8` 或 `Enum16` 类型来确保存储大小。 - -## 使用示例 {#usage-examples} - -这里我们创建一个表,其中包含一个类型为 `Enum8('hello' = 1, 'world' = 2)` 的列: - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world' = 2) -) -ENGINE = TinyLog -``` - -同样可以省略编号。ClickHouse 会自动分配连续编号。默认情况下,从 1 开始编号。 - -```sql -CREATE TABLE t_enum -( - x Enum('hello', 'world') -) -ENGINE = TinyLog -``` - -你也可以为第一个名称指定一个合法的起始编号。 - -```sql -CREATE TABLE t_enum -( - x Enum('hello' = 1, 'world') -) -ENGINE = TinyLog -``` - -```sql -CREATE TABLE t_enum -( - x Enum8('hello' = -129, 'world') -) -ENGINE = TinyLog -``` - -```text -服务器异常: -代码:69. DB::Exception:元素 'hello' 的值 -129 超出了 Enum8 的取值范围。 -``` - -列 `x` 只能存储在类型定义中列出的值:`'hello'` 或 `'world'`。如果尝试保存其他任何值,ClickHouse 会引发异常。该 `Enum` 的 8 位大小是自动确定的。 - -```sql -INSERT INTO t_enum VALUES ('hello'), ('world'), ('hello') -``` - -```text -Ok. -``` - -```sql -INSERT INTO t_enum VALUES('a') -``` - -```text -客户端异常: -代码:49. DB::Exception:类型 Enum('hello' = 1, 'world' = 2) 的未知元素 'a' -``` - -当从该表查询数据时,ClickHouse 会输出 `Enum` 中的字符串值。 - -```sql -SELECT * FROM t_enum -``` - -```text -┌─x─────┐ -│ hello │ -│ world │ -│ hello │ -└───────┘ -``` - -如果你需要查看这些行对应的数值表示,必须将 `Enum` 值强制转换为整数类型。 - -```sql -SELECT CAST(x, 'Int8') FROM t_enum -``` - -```text -┌─CAST(x, 'Int8')─┐ -│ 1 │ -│ 2 │ -│ 1 │ -└─────────────────┘ -``` - -在查询中创建 Enum 值时,也需要使用 `CAST`。 - -```sql -SELECT toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)')) -``` - -```text -┌─toTypeName(CAST('a', 'Enum(\'a\' = 1, \'b\' = 2)'))─┐ -│ Enum8('a' = 1, 'b' = 2) │ -└─────────────────────────────────────────────────────┘ -``` - -## 一般规则和用法 {#general-rules-and-usage} - -对于 `Enum8`,每个值被分配到 `-128 ... 127` 范围内的一个数字;对于 `Enum16`,则分配到 `-32768 ... 32767` 范围内的一个数字。所有字符串和数字都必须互不相同。允许使用空字符串。如果在(表定义中)指定了这种类型,数字可以按任意顺序给出。不过,顺序并不重要。 - -在 `Enum` 中,字符串值和数值都不能为 [NULL](../../sql-reference/syntax.md)。 - -`Enum` 可以用作 [Nullable](../../sql-reference/data-types/nullable.md) 类型的内部类型。因此,如果你使用如下查询来创建一张表: - -```sql -CREATE TABLE t_enum_nullable -( - x Nullable( Enum8('hello' = 1, 'world' = 2) ) -) -ENGINE = TinyLog -``` - -它不仅可以存储 `'hello'` 和 `'world'`,还可以存储 `NULL` 值。 - -```sql -INSERT INTO t_enum_nullable VALUES('hello'),('world'),(NULL) -``` - -在内存中,`Enum` 列的存储方式与对应数值的 `Int8` 或 `Int16` 相同。 - -以文本形式读取时,ClickHouse 会先将该值解析为字符串,并在 Enum 值集合中查找对应的字符串。如果未找到,则抛出异常。以文本格式读取时,本质上是读取字符串并查找对应的数值;如果未找到,同样会抛出异常。 -以文本形式写入时,会将该值写为对应的字符串。如果列数据包含无效数据(不在合法集合中的数字),将抛出异常。以二进制形式读写时,其行为与 `Int8` 和 `Int16` 数据类型相同。 -隐式默认值是数值最小的那个值。 - -在 `ORDER BY`、`GROUP BY`、`IN`、`DISTINCT` 等操作期间,Enum 的行为与对应的数值相同。例如,ORDER BY 会按数值对它们进行排序。相等和比较运算符在 Enum 上的工作方式与在其底层数值上的工作方式相同。 - -Enum 值不能与数字进行比较。Enum 可以与常量字符串进行比较。如果用于比较的字符串不是该 Enum 的有效值,将抛出异常。`IN` 运算符支持左侧为 Enum,右侧为字符串集合。这些字符串是对应 Enum 的各个取值。 - -大多数数值和字符串运算对 Enum 值都是未定义的,例如,对 Enum 加上一个数字,或将字符串与 Enum 进行拼接。 -不过,Enum 提供了一个内置的 `toString` 函数,该函数返回其字符串值。 - -Enum 值也可以使用 `toT` 函数转换为数值类型,其中 T 为某个数值类型。当 T 与 Enum 的底层数值类型相对应时,此转换是零开销的。 -如果仅修改值集合,Enum 类型可以通过 ALTER 以零开销进行更改。可以使用 ALTER 添加和删除 Enum 成员(只有在某个被删除的值从未在表中使用过时,删除才是安全的)。作为保护措施,更改先前已定义 Enum 成员的数值将会抛出异常。 - -使用 ALTER,可以像将 `Int8` 更改为 `Int16` 一样,将 `Enum8` 更改为 `Enum16`,或反之亦然。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md deleted file mode 100644 index a1755ec009f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/fixedstring.md +++ /dev/null @@ -1,111 +0,0 @@ ---- -description: 'ClickHouse 中 FixedString 数据类型的文档' -sidebar_label: 'FixedString(N)' -sidebar_position: 10 -slug: /sql-reference/data-types/fixedstring -title: 'FixedString(N)' -doc_type: 'reference' ---- - -# FixedString(N) {#fixedstringn} - -一个固定长度为 `N` 字节的字符串(不是按字符数或码点数计算)。 - -要声明一个 `FixedString` 类型的列,请使用以下语法: - -```sql -<列名> FixedString(N) -``` - -其中 `N` 是自然数。 - -当数据的长度恰好为 `N` 字节时,`FixedString` 类型是高效的。在其他情况下,它可能会降低效率。 - -以下是一些可以高效存储在 `FixedString` 类型列中的值示例: - -* IP 地址的二进制表示(IPv6 使用 `FixedString(16)`)。 -* 语言代码(ru_RU、en_US 等)。 -* 货币代码(USD、RUB 等)。 -* 哈希的二进制表示(MD5 使用 `FixedString(16)`,SHA256 使用 `FixedString(32)`)。 - -要存储 UUID 值,请使用 [UUID](../../sql-reference/data-types/uuid.md) 数据类型。 - -在插入数据时,ClickHouse 会: - -* 如果字符串少于 `N` 字节,则用空字节补足字符串。 -* 如果字符串多于 `N` 字节,则抛出 `Too large value for FixedString(N)` 异常。 - -考虑如下表,它只有一个 `FixedString(2)` 列: - -```sql - - -INSERT INTO FixedStringTable VALUES ('a'), ('ab'), (''); -``` - -```sql -SELECT - name, - toTypeName(name), - length(name), - empty(name) -FROM FixedStringTable; -``` - -```text -┌─name─┬─toTypeName(name)─┬─length(name)─┬─empty(name)─┐ -│ a │ FixedString(2) │ 2 │ 0 │ -│ ab │ FixedString(2) │ 2 │ 0 │ -│ │ FixedString(2) │ 2 │ 1 │ -└──────┴──────────────────┴──────────────┴─────────────┘ -``` - -请注意,`FixedString(N)` 值的长度是固定的。即使 `FixedString(N)` 值仅由空字节填充,[length](/sql-reference/functions/array-functions#length) 函数也会返回 `N`,但在这种情况下,[empty](/sql-reference/functions/array-functions#empty) 函数返回 `1`。 - -使用 `WHERE` 子句选择数据时,结果会根据条件的写法而有所不同: - -* 如果使用相等运算符 `=`、`==` 或 `equals` 函数,ClickHouse *不会* 将 `\0` 字符考虑在内。也就是说,查询 `SELECT * FROM FixedStringTable WHERE name = 'a';` 和 `SELECT * FROM FixedStringTable WHERE name = 'a\0';` 会返回相同的结果。 -* 如果使用 `LIKE` 子句,ClickHouse *会* 将 `\0` 字符考虑在内,因此可能需要在过滤条件中显式指定 `\0` 字符。 - -```sql -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name = 'a' -FORMAT JSONStringsEachRow - -Query id: c32cec28-bb9e-4650-86ce-d74a1694d79e - -{"name":"a\u0000"} - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a' -FORMAT JSONStringsEachRow - -结果集中包含 0 行。 - - -SELECT name -FROM FixedStringTable -WHERE name LIKE 'a\0' -FORMAT JSONStringsEachRow - -{"name":"a\u0000"} -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md deleted file mode 100644 index a8d5b100c51..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/float.md +++ /dev/null @@ -1,132 +0,0 @@ ---- -description: 'ClickHouse 中浮点数据类型的文档:Float32、Float64 和 BFloat16' -sidebar_label: 'Float32 | Float64 | BFloat16' -sidebar_position: 4 -slug: /sql-reference/data-types/float -title: 'Float32 | Float64 | BFloat16 类型' -doc_type: 'reference' ---- - -:::note -如果您需要进行精确计算,特别是处理需要高精度的金融或业务数据时,应考虑改用 [Decimal](../data-types/decimal.md)。 - -[浮点数](https://en.wikipedia.org/wiki/IEEE_754) 可能会导致不精确的结果,如下所示: - -```sql -CREATE TABLE IF NOT EXISTS float_vs_decimal -( - my_float Float64, - my_decimal Decimal64(3) -) -ENGINE=MergeTree -ORDER BY tuple(); -``` - - -# 生成 1 000 000 个保留 2 位小数的随机数,并分别以 float 和 decimal 类型存储 {#generate-1-000-000-random-numbers-with-2-decimal-places-and-store-them-as-a-float-and-as-a-decimal} - -INSERT INTO float_vs_decimal SELECT round(randCanonical(), 3) AS res, res FROM system.numbers LIMIT 1000000; - -```` -```sql -SELECT sum(my_float), sum(my_decimal) FROM float_vs_decimal; - -┌──────sum(my_float)─┬─sum(my_decimal)─┐ -│ 499693.60500000004 │ 499693.605 │ -└────────────────────┴─────────────────┘ - -SELECT sumKahan(my_float), sumKahan(my_decimal) FROM float_vs_decimal; - -┌─sumKahan(my_float)─┬─sumKahan(my_decimal)─┐ -│ 499693.605 │ 499693.605 │ -└────────────────────┴──────────────────────┘ -```` - -::: - -ClickHouse 中的浮点类型在 C 语言中的等价类型如下: - -* `Float32` — `float`。 -* `Float64` — `double`。 - -ClickHouse 中的浮点类型具有以下别名: - -* `Float32` — `FLOAT`、`REAL`、`SINGLE`。 -* `Float64` — `DOUBLE`、`DOUBLE PRECISION`。 - -在创建表时,可以为浮点数指定数值参数(例如 `FLOAT(12)`、`FLOAT(15, 22)`、`DOUBLE(12)`、`DOUBLE(4, 18)`),但 ClickHouse 会忽略这些参数。 - - -## 使用浮点数 {#using-floating-point-numbers} - -* 浮点数运算可能会产生舍入误差。 - -{/* */ } - -```sql -SELECT 1 - 0.9 - -┌───────minus(1, 0.9)─┐ -│ 0.09999999999999998 │ -└─────────────────────┘ -``` - -* 计算结果取决于计算方式(包括计算机系统的处理器类型和架构)。 -* 浮点运算可能产生诸如无穷大(`Inf`)和“非数字值”(`NaN`)之类的结果。在处理计算结果时应考虑这一点。 -* 当从文本解析浮点数时,结果可能并不是最接近的机器可表示数值。 - - -## NaN 和 Inf {#nan-and-inf} - -与标准 SQL 相比,ClickHouse 支持以下几类浮点数: - -* `Inf` – 无穷大。 - -{/* */ } - -```sql -SELECT 0.5 / 0 - -┌─divide(0.5, 0)─┐ -│ inf │ -└────────────────┘ -``` - -* `-Inf` — 负无穷大。 - -{/* */ } - -```sql -SELECT -0.5 / 0 - -┌─divide(-0.5, 0)─┐ -│ -inf │ -└─────────────────┘ -``` - -* `NaN` — 表示“不是数字”(Not a Number)。 - -{/* */ } - -```sql -SELECT 0 / 0 - -┌─divide(0, 0)─┐ -│ nan │ -└──────────────┘ -``` - -请参阅 [ORDER BY 子句](../../sql-reference/statements/select/order-by.md) 部分中关于 `NaN` 排序的规则。 - - -## BFloat16 {#bfloat16} - -`BFloat16` 是一种 16 位浮点数数据类型,包含 8 位指数、1 位符号位和 7 位尾数位。 -它在机器学习和 AI 应用中非常实用。 - -ClickHouse 支持 `Float32` 与 `BFloat16` 之间的相互转换, -可以使用 [`toFloat32()`](../functions/type-conversion-functions.md/#tofloat32) 或 [`toBFloat16`](../functions/type-conversion-functions.md/#tobfloat16) 函数来完成。 - -:::note -大多数其他操作当前尚不支持。 -::: diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md deleted file mode 100644 index 0fbc5a2df7f..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/geo.md +++ /dev/null @@ -1,206 +0,0 @@ ---- -description: 'ClickHouse 中用于表示地理对象和位置的几何数据类型' -sidebar_label: '地理' -sidebar_position: 54 -slug: /sql-reference/data-types/geo -title: '几何数据类型' -doc_type: 'reference' ---- - -ClickHouse 支持用于表示地理对象(例如位置、区域等)的数据类型。 - -**另请参阅** -- [简单地理要素的表示](https://en.wikipedia.org/wiki/GeoJSON)。 - - - -## Point {#point} - -`Point` 由其 X 和 Y 坐标表示,存储为 [Tuple](tuple.md)([Float64](float.md), [Float64](float.md))。 - -**示例** - -查询: - -```sql -CREATE TABLE geo_point (p Point) ENGINE = Memory(); -INSERT INTO geo_point VALUES((10, 10)); -SELECT p, toTypeName(p) FROM geo_point; -``` - -结果: - -```text -┌─p───────┬─toTypeName(p)─┐ -│ (10,10) │ Point │ -└─────────┴───────────────┘ -``` - - -## 环 {#ring} - -`Ring` 是一种没有孔洞的简单多边形,表示为点的数组:[Array](array.md)([Point](#point))。 - -**示例** - -查询: - -```sql -CREATE TABLE geo_ring (r Ring) ENGINE = Memory(); -INSERT INTO geo_ring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT r, toTypeName(r) FROM geo_ring; -``` - -结果: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ Ring │ -└───────────────────────────────┴───────────────┘ -``` - - -## LineString {#linestring} - -`LineString` 是以点数组形式存储的一条线:[Array](array.md)([Point](#point))。 - -**示例** - -查询: - -```sql -CREATE TABLE geo_linestring (l LineString) ENGINE = Memory(); -INSERT INTO geo_linestring VALUES([(0, 0), (10, 0), (10, 10), (0, 10)]); -SELECT l, toTypeName(l) FROM geo_linestring; -``` - -结果: - -```text -┌─r─────────────────────────────┬─toTypeName(r)─┐ -│ [(0,0),(10,0),(10,10),(0,10)] │ LineString │ -└───────────────────────────────┴───────────────┘ -``` - - -## MultiLineString {#multilinestring} - -`MultiLineString` 是由多条线构成的 `LineString` 数组:[Array](array.md)([LineString](#linestring))。 - -**示例** - -查询: - -```sql -CREATE TABLE geo_multilinestring (l MultiLineString) ENGINE = Memory(); -INSERT INTO geo_multilinestring VALUES([[(0, 0), (10, 0), (10, 10), (0, 10)], [(1, 1), (2, 2), (3, 3)]]); -SELECT l, toTypeName(l) FROM geo_multilinestring; -``` - -结果: - -```text -┌─l───────────────────────────────────────────────────┬─toTypeName(l)───┐ -│ [[(0,0),(10,0),(10,10),(0,10)],[(1,1),(2,2),(3,3)]] │ MultiLineString │ -└─────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Polygon {#polygon} - -`Polygon` 是一种带孔多边形,存储为由环组成的数组:[Array](array.md)([Ring](#ring))。外层数组的第一个元素是多边形的外边界,其后的所有元素表示孔。 - -**示例** - -这是一个带有一个孔的多边形: - -```sql -CREATE TABLE geo_polygon (pg Polygon) ENGINE = Memory(); -INSERT INTO geo_polygon VALUES([[(20, 20), (50, 20), (50, 50), (20, 50)], [(30, 30), (50, 50), (50, 30)]]); -SELECT pg, toTypeName(pg) FROM geo_polygon; -``` - -Result: - -```text -┌─pg────────────────────────────────────────────────────────────┬─toTypeName(pg)─┐ -│ [[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]] │ Polygon │ -└───────────────────────────────────────────────────────────────┴────────────────┘ -``` - - -## MultiPolygon {#multipolygon} - -`MultiPolygon` 由多个多边形组成,并以多边形数组的形式存储:[Array](array.md)([Polygon](#polygon))。 - -**示例** - -此 MultiPolygon 由两个独立的多边形组成——第一个没有空洞,第二个有一个空洞: - -```sql -CREATE TABLE geo_multipolygon (mpg MultiPolygon) ENGINE = Memory(); -INSERT INTO geo_multipolygon VALUES([[[(0, 0), (10, 0), (10, 10), (0, 10)]], [[(20, 20), (50, 20), (50, 50), (20, 50)],[(30, 30), (50, 50), (50, 30)]]]); -SELECT mpg, toTypeName(mpg) FROM geo_multipolygon; -``` - -结果: - -```text -┌─mpg─────────────────────────────────────────────────────────────────────────────────────────────┬─toTypeName(mpg)─┐ -│ [[[(0,0),(10,0),(10,10),(0,10)]],[[(20,20),(50,20),(50,50),(20,50)],[(30,30),(50,50),(50,30)]]] │ MultiPolygon │ -└─────────────────────────────────────────────────────────────────────────────────────────────────┴─────────────────┘ -``` - - -## Geometry {#geometry} - -`Geometry` 是上述所有类型的通用类型。它等价于这些类型的 `Variant`。 - -**示例** - -```sql -CREATE TABLE IF NOT EXISTS geo (geom Geometry) ENGINE = Memory(); -INSERT INTO geo VALUES ((1, 2)); -SELECT * FROM geo; -``` - -结果: - -```text - ┌─geom──┐ -1. │ (1,2) │ - └───────┘ -``` - -{/* */ } - -```sql -CREATE TABLE IF NOT EXISTS geo_dst (geom Geometry) ENGINE = Memory(); - -CREATE TABLE IF NOT EXISTS geo (geom String, id Int) ENGINE = Memory(); -INSERT INTO geo VALUES ('POLYGON((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 1); -INSERT INTO geo VALUES ('POINT(0 0)', 2); -INSERT INTO geo VALUES ('MULTIPOLYGON(((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4)),((-10 -10,-10 -9,-9 10,-10 -10)))', 3); -INSERT INTO geo VALUES ('LINESTRING(1 0,10 0,10 10,0 10,1 0)', 4); -INSERT INTO geo VALUES ('MULTILINESTRING((1 0,10 0,10 10,0 10,1 0),(4 4,5 4,5 5,4 5,4 4))', 5); -INSERT INTO geo_dst SELECT readWkt(geom) FROM geo ORDER BY id; - -SELECT * FROM geo_dst; -``` - -结果: - -```text - ┌─geom─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -1. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ -2. │ (0,0) │ -3. │ [[[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]],[[(-10,-10),(-10,-9),(-9,10),(-10,-10)]]] │ -4. │ [(1,0),(10,0),(10,10),(0,10),(1,0)] │ -5. │ [[(1,0),(10,0),(10,10),(0,10),(1,0)],[(4,4),(5,4),(5,5),(4,5),(4,4)]] │ - └──────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - - -## 相关内容 {#related-content} - -- [探索海量真实世界数据集:ClickHouse 中逾 100 年的气象记录](https://clickhouse.com/blog/real-world-data-noaa-climate-data) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md deleted file mode 100644 index f51cf6db844..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/index.md +++ /dev/null @@ -1,15 +0,0 @@ ---- -description: 'ClickHouse 数据类型文档' -sidebar_label: '数据类型列表' -sidebar_position: 1 -slug: /sql-reference/data-types/ -title: 'ClickHouse 中的数据类型' -doc_type: 'reference' ---- - -# ClickHouse 中的数据类型 {#data-types-in-clickhouse} - -本节介绍 ClickHouse 支持的数据类型,例如[整数](int-uint.md)、[浮点数](float.md)和[字符串](string.md)。 - -系统表 [system.data_type_families](/operations/system-tables/data_type_families) 提供了所有可用数据类型的概览。 -它还显示某个数据类型是否为另一数据类型的别名,以及其名称是否区分大小写(例如 `bool` 与 `BOOL`)。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md deleted file mode 100644 index 11a4b15437a..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/int-uint.md +++ /dev/null @@ -1,61 +0,0 @@ ---- -description: '关于 ClickHouse 中有符号和无符号整数数据类型的文档,范围从 8 位到 256 位' -sidebar_label: 'Int | UInt' -sidebar_position: 2 -slug: /sql-reference/data-types/int-uint -title: 'Int | UInt 类型' -doc_type: 'reference' ---- - -ClickHouse 提供了多种固定长度的整数类型, -包括带符号(`Int`)和无符号(`UInt`),范围从 1 字节到 32 字节。 - -在创建表时,可以为整数类型指定数值长度参数(例如 `TINYINT(8)`、`SMALLINT(16)`、`INT(32)`、`BIGINT(64)`),但 ClickHouse 会忽略这些参数。 - - - -## 整数范围 {#integer-ranges} - -整数类型的取值范围如下: - -| 类型 | 取值范围 | -|----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `Int8` | \[-128 : 127\] | -| `Int16` | \[-32768 : 32767\] | -| `Int32` | \[-2147483648 : 2147483647\] | -| `Int64` | \[-9223372036854775808 : 9223372036854775807\] | -| `Int128` | \[-170141183460469231731687303715884105728 : 170141183460469231731687303715884105727\] | -| `Int256` | \[-57896044618658097711785492504343953926634992332820282019728792003956564819968 : 57896044618658097711785492504343953926634992332820282019728792003956564819967\] | - -无符号整数类型的取值范围如下: - -| 类型 | 取值范围 | -|-----------|----------------------------------------------------------------------------------------| -| `UInt8` | \[0 : 255\] | -| `UInt16` | \[0 : 65535\] | -| `UInt32` | \[0 : 4294967295\] | -| `UInt64` | \[0 : 18446744073709551615\] | -| `UInt128` | \[0 : 340282366920938463463374607431768211455\] | -| `UInt256` | \[0 : 115792089237316195423570985008687907853269984665640564039457584007913129639935\] | - - - -## 整数类型别名 {#integer-aliases} - -整数类型有以下别名: - -| 类型 | 别名 | -|---------|----------------------------------------------------------------------------------| -| `Int8` | `TINYINT`, `INT1`, `BYTE`, `TINYINT SIGNED`, `INT1 SIGNED` | -| `Int16` | `SMALLINT`, `SMALLINT SIGNED` | -| `Int32` | `INT`, `INTEGER`, `MEDIUMINT`, `MEDIUMINT SIGNED`, `INT SIGNED`, `INTEGER SIGNED` | -| `Int64` | `BIGINT`, `SIGNED`, `BIGINT SIGNED`, `TIME` | - -无符号整数类型有以下别名: - -| 类型 | 别名 | -|----------|--------------------------------------------------------| -| `UInt8` | `TINYINT UNSIGNED`, `INT1 UNSIGNED` | -| `UInt16` | `SMALLINT UNSIGNED` | -| `UInt32` | `MEDIUMINT UNSIGNED`, `INT UNSIGNED`, `INTEGER UNSIGNED` | -| `UInt64` | `UNSIGNED`, `BIGINT UNSIGNED`, `BIT`, `SET` | diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md deleted file mode 100644 index 635369cda5e..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv4.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: 'ClickHouse 中 IPv4 数据类型文档' -sidebar_label: 'IPv4' -sidebar_position: 28 -slug: /sql-reference/data-types/ipv4 -title: 'IPv4' -doc_type: 'reference' ---- - -## IPv4 {#ipv4} - -IPv4 地址。占用 4 个字节,存储为 UInt32 类型。 - -### 基本用法 {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv4 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -或者可以使用 IPv4 域名作为键: - -```sql -CREATE TABLE hits (url String, from IPv4) ENGINE = MergeTree() ORDER BY from; -``` - -`IPv4` 域支持使用 IPv4 字符串作为自定义输入格式: - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '116.253.40.133')('https://clickhouse.com', '183.247.232.58')('https://clickhouse.com/docs/en/', '116.106.34.242'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬───────────from─┐ -│ https://clickhouse.com/docs/en/ │ 116.106.34.242 │ -│ https://wikipedia.org │ 116.253.40.133 │ -│ https://clickhouse.com │ 183.247.232.58 │ -└────────────────────────────────────┴────────────────┘ -``` - -值以紧凑的二进制格式存储: - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)─┐ -│ IPv4 │ B7F7E83A │ -└──────────────────┴───────────┘ -``` - -IPv4 地址可以直接与 IPv6 地址进行比较: - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [用于处理 IPv4 和 IPv6 地址的函数](../functions/ip-address-functions.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md deleted file mode 100644 index 811be4c22f0..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/ipv6.md +++ /dev/null @@ -1,77 +0,0 @@ ---- -description: '本文档介绍 ClickHouse 中的 IPv6 数据类型,该类型将 IPv6 地址存储为 16 字节的值' -sidebar_label: 'IPv6' -sidebar_position: 30 -slug: /sql-reference/data-types/ipv6 -title: 'IPv6' -doc_type: 'reference' ---- - -## IPv6 {#ipv6} - -IPv6 地址。以大端序的 16 字节 UInt128 形式存储。 - -### 基本用法 {#basic-usage} - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY url; - -DESCRIBE TABLE hits; -``` - -```text -┌─name─┬─type───┬─default_type─┬─default_expression─┬─comment─┬─codec_expression─┐ -│ url │ String │ │ │ │ │ -│ from │ IPv6 │ │ │ │ │ -└──────┴────────┴──────────────┴────────────────────┴─────────┴──────────────────┘ -``` - -或者可以使用 `IPv6` 域名作为键: - -```sql -CREATE TABLE hits (url String, from IPv6) ENGINE = MergeTree() ORDER BY from; -``` - -`IPv6` 域支持使用 IPv6 字符串作为自定义输入: - -```sql -INSERT INTO hits (url, from) VALUES ('https://wikipedia.org', '2a02:aa08:e000:3100::2')('https://clickhouse.com', '2001:44c8:129:2632:33:0:252:2')('https://clickhouse.com/docs/en/', '2a02:e980:1e::1'); - -SELECT * FROM hits; -``` - -```text -┌─url────────────────────────────────┬─from──────────────────────────┐ -│ https://clickhouse.com │ 2001:44c8:129:2632:33:0:252:2 │ -│ https://clickhouse.com/docs/en/ │ 2a02:e980:1e::1 │ -│ https://wikipedia.org │ 2a02:aa08:e000:3100::2 │ -└────────────────────────────────────┴───────────────────────────────┘ -``` - -数值以紧凑的二进制格式存储: - -```sql -SELECT toTypeName(from), hex(from) FROM hits LIMIT 1; -``` - -```text -┌─toTypeName(from)─┬─hex(from)────────────────────────┐ -│ IPv6 │ 200144C8012926320033000002520002 │ -└──────────────────┴──────────────────────────────────┘ -``` - -IPv6 地址可以直接与 IPv4 地址比较: - -```sql -SELECT toIPv4('127.0.0.1') = toIPv6('::ffff:127.0.0.1'); -``` - -```text -┌─equals(toIPv4('127.0.0.1'), toIPv6('::ffff:127.0.0.1'))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────┘ -``` - -**另请参阅** - -* [用于处理 IPv4 和 IPv6 地址的函数](../functions/ip-address-functions.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md deleted file mode 100644 index b614ef2e54c..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/lowcardinality.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -description: 'LowCardinality 字符串列优化文档' -sidebar_label: 'LowCardinality(T)' -sidebar_position: 42 -slug: /sql-reference/data-types/lowcardinality -title: 'LowCardinality(T)' -doc_type: 'reference' ---- - - - -# LowCardinality(T) {#lowcardinalityt} - -将其他数据类型的内部表示转换为字典编码。 - - - -## 语法 {#syntax} - -```sql -LowCardinality(data_type) -``` - -**参数** - -* `data_type` — [String](../../sql-reference/data-types/string.md)、[FixedString](../../sql-reference/data-types/fixedstring.md)、[Date](../../sql-reference/data-types/date.md)、[DateTime](../../sql-reference/data-types/datetime.md),以及除 [Decimal](../../sql-reference/data-types/decimal.md) 之外的其他数值类型。`LowCardinality` 对某些数据类型的效率较低,参见 [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types) 设置说明。 - - -## 描述 {#description} - -`LowCardinality` 是一种上层结构,用于改变数据的存储方式和处理规则。ClickHouse 会对 `LowCardinality` 列应用[字典编码](https://en.wikipedia.org/wiki/Dictionary_coder)。在许多应用场景下,基于字典编码数据进行 [SELECT](../../sql-reference/statements/select/index.md) 查询可以显著提升性能。 - -使用 `LowCardinality` 数据类型的效率取决于数据的多样性。如果字典包含少于 10,000 个不同取值,那么在大多数情况下,ClickHouse 在数据读取和存储方面会表现出更高的效率。如果字典包含超过 100,000 个不同取值,那么与使用普通数据类型相比,ClickHouse 的性能可能更差。 - -在处理字符串时,可以考虑使用 `LowCardinality` 来替代 [Enum](../../sql-reference/data-types/enum.md)。`LowCardinality` 在使用上提供了更高的灵活性,并且通常可以达到同等或更高的效率。 - - - -## 示例 {#example} - -创建一个包含 `LowCardinality` 列的表: - -```sql -CREATE TABLE lc_t -( - `id` UInt16, - `strings` LowCardinality(String) -) -ENGINE = MergeTree() -ORDER BY id -``` - - -## 相关设置和函数 {#related-settings-and-functions} - -设置: - -- [low_cardinality_max_dictionary_size](../../operations/settings/settings.md#low_cardinality_max_dictionary_size) -- [low_cardinality_use_single_dictionary_for_part](../../operations/settings/settings.md#low_cardinality_use_single_dictionary_for_part) -- [low_cardinality_allow_in_native_format](../../operations/settings/settings.md#low_cardinality_allow_in_native_format) -- [allow_suspicious_low_cardinality_types](../../operations/settings/settings.md#allow_suspicious_low_cardinality_types) -- [output_format_arrow_low_cardinality_as_dictionary](/operations/settings/formats#output_format_arrow_low_cardinality_as_dictionary) - -函数: - -- [toLowCardinality](../../sql-reference/functions/type-conversion-functions.md#tolowcardinality) - - - -## 相关内容 {#related-content} - -- 博客:[使用模式和编解码器优化 ClickHouse](https://clickhouse.com/blog/optimize-clickhouse-codecs-compression-schema) -- 博客:[在 ClickHouse 中处理时间序列数据](https://clickhouse.com/blog/working-with-time-series-data-and-functions-ClickHouse) -- [字符串优化(俄语视频演讲)](https://youtu.be/rqf-ILRgBdY?list=PL0Z2YDlm0b3iwXCpEFiOOYmwXzVmjJfEt)。[英文幻灯片](https://github.com/ClickHouse/clickhouse-presentations/raw/master/meetup19/string_optimization.pdf) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md deleted file mode 100644 index d4473170ce4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/map.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -description: 'ClickHouse 中 Map 数据类型文档' -sidebar_label: 'Map(K, V)' -sidebar_position: 36 -slug: /sql-reference/data-types/map -title: 'Map(K, V)' -doc_type: 'reference' ---- - - - -# Map(K, V) {#mapk-v} - -数据类型 `Map(K, V)` 用于存储键值对。 - -与其他数据库不同,在 ClickHouse 中 Map 中的键不要求唯一,也就是说,一个 Map 可以包含两个具有相同键的元素。 -(这是因为 Map 在内部实现为 `Array(Tuple(K, V))`。) - -你可以使用语法 `m[k]` 来获取 Map `m` 中键 `k` 对应的值。 -同时,`m[k]` 会顺序扫描整个 Map,即该操作的运行时间与 Map 的大小成线性关系。 - -**参数** - -* `K` — Map 键的类型。除 [Nullable](../../sql-reference/data-types/nullable.md) 和嵌套了 [Nullable](../../sql-reference/data-types/nullable.md) 类型的 [LowCardinality](../../sql-reference/data-types/lowcardinality.md) 之外,可以是任意类型。 -* `V` — Map 值的类型。可以是任意类型。 - -**示例** - -创建一个包含 Map 类型列的表: - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':1, 'key2':10}), ({'key1':2,'key2':20}), ({'key1':3,'key2':30}); -``` - -若要选择 `key2` 的值: - -```sql -SELECT m['key2'] FROM tab; -``` - -结果: - -```text -┌─arrayElement(m, 'key2')─┐ -│ 10 │ -│ 20 │ -│ 30 │ -└─────────────────────────┘ -``` - -如果访问的键 `k` 不在 map 中,`m[k]` 会返回该值类型的默认值,例如整数类型为 `0`,字符串类型为 `''`。 -要检查某个键是否存在于 map 中,可以使用函数 [mapContains](../../sql-reference/functions/tuple-map-functions#mapcontains)。 - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE=Memory; -INSERT INTO tab VALUES ({'key1':100}), ({}); -SELECT m['key1'] FROM tab; -``` - -结果: - -```text -┌─arrayElement(m, 'key1')─┐ -│ 100 │ -│ 0 │ -└─────────────────────────┘ -``` - - -## 将 Tuple 转换为 Map {#converting-tuple-to-map} - -类型为 `Tuple()` 的值可以通过函数 [CAST](/sql-reference/functions/type-conversion-functions#cast) 转换为 `Map()` 类型的值: - -**示例** - -查询: - -```sql -SELECT CAST(([1, 2, 3], ['Ready', 'Steady', 'Go']), 'Map(UInt8, String)') AS map; -``` - -结果: - -```text -┌─map───────────────────────────┐ -│ {1:'Ready',2:'Steady',3:'Go'} │ -└───────────────────────────────┘ -``` - - -## 读取 Map 的子列 {#reading-subcolumns-of-map} - -在某些情况下,为了避免读取整个 Map,你可以使用 `keys` 和 `values` 这两个子列。 - -**示例** - -查询: - -```sql -CREATE TABLE tab (m Map(String, UInt64)) ENGINE = Memory; -INSERT INTO tab VALUES (map('key1', 1, 'key2', 2, 'key3', 3)); - -SELECT m.keys FROM tab; -- 等同于 mapKeys(m) -SELECT m.values FROM tab; -- 等同于 mapValues(m) -``` - -结果: - -```text -┌─m.keys─────────────────┐ -│ ['key1','key2','key3'] │ -└────────────────────────┘ - -┌─m.values─┐ -│ [1,2,3] │ -└──────────┘ -``` - -**另请参阅** - -* [map()](/sql-reference/functions/tuple-map-functions#map) 函数 -* [CAST()](/sql-reference/functions/type-conversion-functions#cast) 函数 -* [用于 Map 数据类型的 -Map 组合器](../aggregate-functions/combinators.md#-map) - - -## 相关内容 {#related-content} - -- 博客:[使用 ClickHouse 构建可观测性解决方案 - 第 2 部分 - 链路追踪](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md deleted file mode 100644 index b3f958b45cd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nested-data-structures/index.md +++ /dev/null @@ -1,108 +0,0 @@ ---- -description: 'ClickHouse 中嵌套数据结构的概述' -sidebar_label: 'Nested(Name1 Type1, Name2 Type2, ...)' -sidebar_position: 57 -slug: /sql-reference/data-types/nested-data-structures/nested -title: 'Nested' -doc_type: 'guide' ---- - -# 嵌套 {#nested} - -## Nested(name1 Type1, Name2 Type2, ...) {#nestedname1-type1-name2-type2-} - -嵌套数据结构就像表格单元格中的一张表。嵌套数据结构的参数——列名和类型——的指定方式与 [CREATE TABLE](../../../sql-reference/statements/create/table.md) 查询中相同。表中的每一行都可以对应嵌套数据结构中的任意数量的行。 - -示例: - -```sql -CREATE TABLE test.visits -( - CounterID UInt32, - StartDate Date, - Sign Int8, - IsNew UInt8, - VisitID UInt64, - UserID UInt64, - ... - Goals Nested - ( - ID UInt32, - Serial UInt32, - EventTime DateTime, - Price Int64, - OrderID String, - CurrencyID UInt32 - ), - ... -) ENGINE = CollapsingMergeTree(StartDate, intHash32(UserID), (CounterID, StartDate, intHash32(UserID), VisitID), 8192, Sign) -``` - -此示例声明了 `Goals` 嵌套数据结构,其中包含关于转化(已达成目标)的数据。`visits` 表中的每一行都可以对应零个或任意数量的转化。 - -当将 [flatten_nested](/operations/settings/settings#flatten_nested) 设置为 `0`(默认值并非如此)时,将支持任意层级的嵌套。 - -在大多数情况下,处理嵌套数据结构时,其列通过用点分隔的列名来指定。这些列都是相同类型的数组。单个嵌套数据结构中的所有数组列长度都相同。 - -示例: - -```sql -SELECT - Goals.ID, - Goals.EventTime -FROM test.visits -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goals.ID───────────────────────┬─Goals.EventTime───────────────────────────────────────────────────────────────────────────┐ -│ [1073752,591325,591325] │ ['2014-03-17 16:38:10','2014-03-17 16:38:48','2014-03-17 16:42:27'] │ -│ [1073752] │ ['2014-03-17 00:28:25'] │ -│ [1073752] │ ['2014-03-17 10:46:20'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:59:20','2014-03-17 22:17:55','2014-03-17 22:18:07','2014-03-17 22:18:51'] │ -│ [] │ [] │ -│ [1073752,591325,591325] │ ['2014-03-17 11:37:06','2014-03-17 14:07:47','2014-03-17 14:36:21'] │ -│ [] │ [] │ -│ [] │ [] │ -│ [591325,1073752] │ ['2014-03-17 00:46:05','2014-03-17 00:46:05'] │ -│ [1073752,591325,591325,591325] │ ['2014-03-17 13:28:33','2014-03-17 13:30:26','2014-03-17 18:51:21','2014-03-17 18:51:45'] │ -└────────────────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -最容易的方式是将嵌套数据结构看作一组长度相同的多列数组。 - -在 `SELECT` 查询中,只有在 `ARRAY JOIN` 子句中才能指定整个嵌套数据结构的名称,而不是单独的列名。有关更多信息,请参阅“ARRAY JOIN 子句”。示例: - -```sql -SELECT - Goal.ID, - Goal.EventTime -FROM test.visits -ARRAY JOIN Goals AS Goal -WHERE CounterID = 101500 AND length(Goals.ID) < 5 -LIMIT 10 -``` - -```text -┌─Goal.ID─┬──────Goal.EventTime─┐ -│ 1073752 │ 2014-03-17 16:38:10 │ -│ 591325 │ 2014-03-17 16:38:48 │ -│ 591325 │ 2014-03-17 16:42:27 │ -│ 1073752 │ 2014-03-17 00:28:25 │ -│ 1073752 │ 2014-03-17 10:46:20 │ -│ 1073752 │ 2014-03-17 13:59:20 │ -│ 591325 │ 2014-03-17 22:17:55 │ -│ 591325 │ 2014-03-17 22:18:07 │ -│ 591325 │ 2014-03-17 22:18:51 │ -│ 1073752 │ 2014-03-17 11:37:06 │ -└─────────┴─────────────────────┘ -``` - -不能对整个嵌套数据结构执行 SELECT 查询,只能显式列出其中的各个单独列。 - -对于 INSERT 查询,需要分别传递嵌套数据结构中各个组成列的数组(就像它们是独立的列数组一样)。在插入时,系统会检查它们的长度是否一致。 - -对于 DESCRIBE 查询,嵌套数据结构中的列也会以同样的方式分别列出。 - -对嵌套数据结构中元素执行的 ALTER 查询存在一定限制。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md deleted file mode 100644 index c78e5390874..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/newjson.md +++ /dev/null @@ -1,1060 +0,0 @@ ---- -description: 'ClickHouse 中 JSON 数据类型的参考文档,该类型在处理 JSON 数据时提供原生支持' -keywords: ['json', 'data type'] -sidebar_label: 'JSON' -sidebar_position: 63 -slug: /sql-reference/data-types/newjson -title: 'JSON 数据类型' -doc_type: 'reference' ---- - -import {CardSecondary} from '@clickhouse/click-ui/bundled'; -import Link from '@docusaurus/Link' - - - - - -
- -`JSON` 类型会在单个列中存储 JavaScript Object Notation (JSON) 文档。 - -:::note -在 ClickHouse 开源版中,JSON 数据类型从 25.3 版本开始被标记为可用于生产环境。不建议在更早的版本中在生产环境使用此类型。 -::: - -要声明一个 `JSON` 类型的列,可以使用以下语法: - -```sql -<列名> JSON -( - max_dynamic_paths=N, - max_dynamic_types=M, - some.path 类型名, - SKIP path.to.skip, - SKIP REGEXP 'paths_regexp' -) -``` - -上述语法中的各参数定义如下: - -| Parameter | Description | Default Value | -| --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------- | -| `max_dynamic_paths` | 一个可选参数,表示在单个独立存储的数据块中(例如 MergeTree 表的单个 data part),最多可以有多少个路径以子列形式单独存储。

如果超过此限制,所有其余路径将合并存储在一个单一结构中。 | `1024` | -| `max_dynamic_types` | 一个取值范围在 `1` 到 `255` 之间的可选参数,表示在单个独立存储的数据块中(例如 MergeTree 表的单个 data part),在类型为 `Dynamic` 的单个路径列中最多可以存储多少种不同的数据类型。

如果超过此限制,所有新增类型都会被转换为 `String` 类型。 | `32` | -| `some.path TypeName` | 针对 JSON 中特定路径的可选类型提示。此类路径将始终作为具有指定类型的子列进行存储。 | | -| `SKIP path.to.skip` | 针对在 JSON 解析期间应跳过的特定路径的可选提示。此类路径将永远不会存储在 JSON 列中。若指定路径是一个嵌套 JSON 对象,则整个嵌套对象都会被跳过。 | | -| `SKIP REGEXP 'path_regexp'` | 使用正则表达式在 JSON 解析期间跳过路径的可选提示。所有匹配此正则表达式的路径将永远不会存储在 JSON 列中。 | | - - -## 创建 JSON {#creating-json} - -本节将介绍创建 `JSON` 的多种方法。 - -### 在表的列定义中使用 `JSON` {#using-json-in-a-table-column-definition} - -```sql title="Query (Example 1)" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 1)" -┌─json────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ -│ {"f":"你好,世界!"} │ -│ {"a":{"b":"43","e":"10"},"c":["4","5","6"]} │ -└─────────────────────────────────────────────┘ -``` - -```sql title="Query (Example 2)" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42}, "c" : [1, 2, 3]}'), ('{"f" : "Hello, World!"}'), ('{"a" : {"b" : 43, "e" : 10}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response (Example 2)" -┌─json──────────────────────────────┐ -│ {"a":{"b":42},"c":["1","2","3"]} │ -│ {"a":{"b":0},"f":"你好,世界!"} │ -│ {"a":{"b":43},"c":["4","5","6"]} │ -└───────────────────────────────────┘ -``` - -### 使用 `::JSON` 进行 CAST {#using-cast-with-json} - -可以使用特殊语法 `::JSON` 对各种类型执行类型转换(CAST)。 - -#### 使用 CAST 将 `String` 转换为 `JSON` {#cast-from-string-to-json} - -```sql title="Query" -SELECT '{"a" : {"b" : 42},"c" : [1, 2, 3], "d" : "Hello, World!"}'::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"你好,世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### 使用 CAST 将 `Tuple` 转换为 `JSON` {#cast-from-tuple-to-json} - -```sql title="Query" -SET enable_named_columns_in_function_tuple = 1; -SELECT (tuple(42 AS b) AS a, [1, 2, 3] AS c, 'Hello, World!' AS d)::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"你好,世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -#### 将 `Map` 类型 CAST 为 `JSON` {#cast-from-map-to-json} - -```sql title="Query" -SET use_variant_as_common_type=1; -SELECT map('a', map('b', 42), 'c', [1,2,3], 'd', 'Hello, World!')::JSON AS json; -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────┐ -│ {"a":{"b":"42"},"c":["1","2","3"],"d":"你好,世界!"} │ -└────────────────────────────────────────────────────────┘ -``` - -:::note -JSON 路径会被存储为扁平结构。这意味着,当根据类似 `a.b.c` 这样的路径来构造一个 JSON 对象时, -无法确定该对象应被构造为 `{ "a.b.c" : ... }` 还是 `{ "a": { "b": { "c": ... } } }`。 -我们的实现始终会采用后者的形式。 - -例如: - -```sql -SELECT CAST('{"a.b.c" : 42}', 'JSON') AS json -``` - -将会返回: - -```response - ┌─json───────────────────┐ -1. │ {"a":{"b":{"c":"42"}}} │ - └────────────────────────┘ -``` - -而**不是**: - - -```sql - ┌─json───────────┐ -1. │ {"a.b.c":"42"} │ - └────────────────┘ -``` - -::: - - -## 将 JSON 路径读取为子列 {#reading-json-paths-as-sub-columns} - -`JSON` 类型支持将 JSON 中的每个路径读取为独立的子列。 -如果在 JSON 类型的声明中未指定所请求路径的类型, -则该路径对应的子列类型始终为 [Dynamic](/sql-reference/data-types/dynamic.md)。 - -例如: - -```sql title="Query" -CREATE TABLE test (json JSON(a.b UInt32, SKIP a.e)) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : 42, "g" : 42.42}, "c" : [1, 2, 3], "d" : "2020-01-01"}'), ('{"f" : "Hello, World!", "d" : "2020-01-02"}'), ('{"a" : {"b" : 43, "e" : 10, "g" : 43.43}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────┐ -│ {"a":{"b":42,"g":42.42},"c":["1","2","3"],"d":"2020-01-01"} │ -│ {"a":{"b":0},"d":"2020-01-02","f":"Hello, World!"} │ -│ {"a":{"b":43,"g":43.43},"c":["4","5","6"]} │ -└─────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query (Reading JSON paths as sub-columns)" -SELECT json.a.b, json.a.g, json.c, json.d FROM test; -``` - -```text title="Response (Reading JSON paths as sub-columns)" -┌─json.a.b─┬─json.a.g─┬─json.c──┬─json.d─────┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└──────────┴──────────┴─────────┴────────────┘ -``` - -你还可以使用 `getSubcolumn` 函数从 JSON 类型中读取子列: - -```sql title="Query" -SELECT getSubcolumn(json, 'a.b'), getSubcolumn(json, 'a.g'), getSubcolumn(json, 'c'), getSubcolumn(json, 'd') FROM test; -``` - -```text title="Response" -┌─getSubcolumn(json, 'a.b')─┬─getSubcolumn(json, 'a.g')─┬─getSubcolumn(json, 'c')─┬─getSubcolumn(json, 'd')─┐ -│ 42 │ 42.42 │ [1,2,3] │ 2020-01-01 │ -│ 0 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-02 │ -│ 43 │ 43.43 │ [4,5,6] │ ᴺᵁᴸᴸ │ -└───────────────────────────┴───────────────────────────┴─────────────────────────┴─────────────────────────┘ -``` - -如果在数据中找不到请求的路径,将会被填充为 `NULL` 值: - -```sql title="Query" -SELECT json.non.existing.path FROM test; -``` - -```text title="Response" -┌─json.non.existing.path─┐ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -│ ᴺᵁᴸᴸ │ -└────────────────────────┘ -``` - -让我们检查一下返回的子列的数据类型: - -```sql title="Query" -SELECT toTypeName(json.a.b), toTypeName(json.a.g), toTypeName(json.c), toTypeName(json.d) FROM test; -``` - - -```text title="Response" -┌─toTypeName(json.a.b)─┬─toTypeName(json.a.g)─┬─toTypeName(json.c)─┬─toTypeName(json.d)─┐ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -│ UInt32 │ Dynamic │ Dynamic │ Dynamic │ -└──────────────────────┴──────────────────────┴────────────────────┴────────────────────┘ -``` - -如上所示,对于 `a.b`,其类型为 `UInt32`,这是在 JSON 类型声明中显式指定的; -而所有其他子列的类型则为 `Dynamic`。 - -也可以使用特殊语法 `json.some.path.:TypeName` 来读取 `Dynamic` 类型的子列: - -```sql title="Query" -SELECT - json.a.g.:Float64, - dynamicType(json.a.g), - json.d.:Date, - dynamicType(json.d) -FROM test -``` - -```text title="Response" -┌─json.a.g.:`Float64`─┬─dynamicType(json.a.g)─┬─json.d.:`Date`─┬─dynamicType(json.d)─┐ -│ 42.42 │ Float64 │ 2020-01-01 │ Date │ -│ ᴺᵁᴸᴸ │ None │ 2020-01-02 │ Date │ -│ 43.43 │ Float64 │ ᴺᵁᴸᴸ │ None │ -└─────────────────────┴───────────────────────┴────────────────┴─────────────────────┘ -``` - -`Dynamic` 子列可以转换为任意数据类型。若 `Dynamic` 中的内部类型无法转换为请求的类型,则会抛出异常: - -```sql title="Query" -SELECT json.a.g::UInt64 AS uint -FROM test; -``` - -```text title="Response" -┌─uint─┐ -│ 42 │ -│ 0 │ -│ 43 │ -└──────┘ -``` - -```sql title="Query" -SELECT json.a.g::UUID AS float -FROM test; -``` - -```text title="Response" -服务器返回异常: -Code: 48. DB::Exception: Received from localhost:9000. DB::Exception: -不支持数值类型与 UUID 之间的转换。 -传入的 UUID 可能未加引号: -执行 'FUNCTION CAST(__table1.json.a.g :: 2, 'UUID'_String :: 1) -> CAST(__table1.json.a.g, 'UUID'_String) UUID : 0' 时发生错误。 -(NOT_IMPLEMENTED) -``` - -:::note -要高效地从 Compact MergeTree 数据片段中读取子列,请确保已启用 MergeTree 设置 [write_marks_for_substreams_in_compact_parts](../../operations/settings/merge-tree-settings.md#write_marks_for_substreams_in_compact_parts)。 -::: - - -## 将 JSON 子对象读取为子列 {#reading-json-sub-objects-as-sub-columns} - -`JSON` 类型支持使用特殊语法 `json.^some.path` 将嵌套对象读取为 `JSON` 类型的子列: - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES ('{"a" : {"b" : {"c" : 42, "g" : 42.42}}, "c" : [1, 2, 3], "d" : {"e" : {"f" : {"g" : "Hello, World", "h" : [1, 2, 3]}}}}'), ('{"f" : "Hello, World!", "d" : {"e" : {"f" : {"h" : [4, 5, 6]}}}}'), ('{"a" : {"b" : {"c" : 43, "e" : 10, "g" : 43.43}}, "c" : [4, 5, 6]}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":"42","g":42.42}},"c":["1","2","3"],"d":{"e":{"f":{"g":"Hello, World","h":["1","2","3"]}}}} │ -│ {"d":{"e":{"f":{"h":["4","5","6"]}}},"f":"Hello, World!"} │ -│ {"a":{"b":{"c":"43","e":"10","g":43.43}},"c":["4","5","6"]} │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.^a.b, json.^d.e.f FROM test; -``` - -```text title="Response" -┌─json.^`a`.b───────────────────┬─json.^`d`.e.f──────────────────────────┐ -│ {"c":"42","g":42.42} │ {"g":"Hello, World","h":["1","2","3"]} │ -│ {} │ {"h":["4","5","6"]} │ -│ {"c":"43","e":"10","g":43.43} │ {} │ -└───────────────────────────────┴────────────────────────────────────────┘ -``` - -:::note -将子对象作为子列读取可能效率较低,因为这可能需要几乎对整个 JSON 数据进行扫描。 -::: - - -## 路径的类型推断 {#type-inference-for-paths} - -在解析 `JSON` 时,ClickHouse 会尝试为每个 JSON 路径推断出最合适的数据类型。 -其工作方式类似于[基于输入数据自动推断表结构](/interfaces/schema-inference.md), -并且由相同的设置控制: - -* [input_format_try_infer_dates](/operations/settings/formats#input_format_try_infer_dates) -* [input_format_try_infer_datetimes](/operations/settings/formats#input_format_try_infer_datetimes) -* [schema_inference_make_columns_nullable](/operations/settings/formats#schema_inference_make_columns_nullable) -* [input_format_json_try_infer_numbers_from_strings](/operations/settings/formats#input_format_json_try_infer_numbers_from_strings) -* [input_format_json_infer_incomplete_types_as_strings](/operations/settings/formats#input_format_json_infer_incomplete_types_as_strings) -* [input_format_json_read_numbers_as_strings](/operations/settings/formats#input_format_json_read_numbers_as_strings) -* [input_format_json_read_bools_as_strings](/operations/settings/formats#input_format_json_read_bools_as_strings) -* [input_format_json_read_bools_as_numbers](/operations/settings/formats#input_format_json_read_bools_as_numbers) -* [input_format_json_read_arrays_as_strings](/operations/settings/formats#input_format_json_read_arrays_as_strings) -* [input_format_json_infer_array_of_dynamic_from_array_of_different_types](/operations/settings/formats#input_format_json_infer_array_of_dynamic_from_array_of_different_types) - -下面来看一些示例: - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=1, input_format_try_infer_datetimes=1; -``` - -```text title="Response" -┌─paths_with_types─────────────────┐ -│ {'a':'Date','b':'DateTime64(9)'} │ -└──────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : "2020-01-01", "b" : "2020-01-01 10:00:00"}'::JSON) AS paths_with_types SETTINGS input_format_try_infer_dates=0, input_format_try_infer_datetimes=0; -``` - -```text title="Response" -┌─paths_with_types────────────┐ -│ {'a':'String','b':'String'} │ -└─────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types SETTINGS schema_inference_make_columns_nullable=1; -``` - -```text title="Response" -┌─paths_with_types───────────────┐ -│ {'a':'Array(Nullable(Int64))'} │ -└────────────────────────────────┘ -``` - -```sql title="Query" -SELECT JSONAllPathsWithTypes('{"a" : [1, 2, 3]}'::JSON) AS paths_with_types settings schema_inference_make_columns_nullable=0; -``` - -```text title="Response" -┌─paths_with_types─────┐ -│ {'a':'Array(Int64)'} │ -└──────────────────────┘ -``` - - -## 处理 JSON 对象数组 {#handling-arrays-of-json-objects} - -包含对象数组的 JSON 路径会被解析为 `Array(JSON)` 类型,并插入到该路径对应的 `Dynamic` 列中。 -要读取对象数组,可以从 `Dynamic` 列中将其提取为子列: - -```sql title="Query" -CREATE TABLE test (json JSON) ENGINE = Memory; -INSERT INTO test VALUES -('{"a" : {"b" : [{"c" : 42, "d" : "Hello", "f" : [[{"g" : 42.42}]], "k" : {"j" : 1000}}, {"c" : 43}, {"e" : [1, 2, 3], "d" : "My", "f" : [[{"g" : 43.43, "h" : "2020-01-01"}]], "k" : {"j" : 2000}}]}}'), -('{"a" : {"b" : [1, 2, 3]}}'), -('{"a" : {"b" : [{"c" : 44, "f" : [[{"h" : "2020-01-02"}]]}, {"e" : [4, 5, 6], "d" : "World", "f" : [[{"g" : 44.44}]], "k" : {"j" : 3000}}]}}'); -SELECT json FROM test; -``` - -```text title="Response" -┌─json────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐ -│ {"a":{"b":[{"c":"42","d":"你好","f":[[{"g":42.42}]],"k":{"j":"1000"}},{"c":"43"},{"d":"我的","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}]}} │ -│ {"a":{"b":["1","2","3"]}} │ -│ {"a":{"b":[{"c":"44","f":[[{"h":"2020-01-02"}]]},{"d":"世界","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}]}} │ -└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘ -``` - -```sql title="Query" -SELECT json.a.b, dynamicType(json.a.b) FROM test; -``` - -```text title="Response" -┌─json.a.b──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┬─dynamicType(json.a.b)────────────────────────────────────┐ -│ ['{"c":"42","d":"Hello","f":[[{"g":42.42}]],"k":{"j":"1000"}}','{"c":"43"}','{"d":"My","e":["1","2","3"],"f":[[{"g":43.43,"h":"2020-01-01"}]],"k":{"j":"2000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -│ [1,2,3] │ Array(Nullable(Int64)) │ -│ ['{"c":"44","f":[[{"h":"2020-01-02"}]]}','{"d":"World","e":["4","5","6"],"f":[[{"g":44.44}]],"k":{"j":"3000"}}'] │ Array(JSON(max_dynamic_types=16, max_dynamic_paths=256)) │ -└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴──────────────────────────────────────────────────────────┘ -``` - -正如你可能已经注意到的,嵌套 `JSON` 类型的 `max_dynamic_types`/`max_dynamic_paths` 参数相比默认值被调低了。 -这是为了避免在嵌套的 JSON 对象数组中,子列数量不受控制地增长。 - -让我们尝试从一个嵌套的 `JSON` 列中读取子列: - -```sql title="Query" -SELECT json.a.b.:`Array(JSON)`.c, json.a.b.:`Array(JSON)`.f, json.a.b.:`Array(JSON)`.d FROM test; -``` - - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -我们可以使用一种特殊语法来避免手动指定 `Array(JSON)` 子列名: - -```sql title="Query" -SELECT json.a.b[].c, json.a.b[].f, json.a.b[].d FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c─┬─json.a.b.:`Array(JSON)`.f───────────────────────────────────┬─json.a.b.:`Array(JSON)`.d─┐ -│ [42,43,NULL] │ [[['{"g":42.42}']],NULL,[['{"g":43.43,"h":"2020-01-01"}']]] │ ['Hello',NULL,'My'] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[['{"h":"2020-01-02"}']],[['{"g":44.44}']]] │ [NULL,'World'] │ -└───────────────────────────┴─────────────────────────────────────────────────────────────┴───────────────────────────┘ -``` - -路径后面的 `[]` 数量表示数组的嵌套层级。例如,`json.path[][]` 会被转换为 `json.path.:Array(Array(JSON))` - -让我们检查一下 `Array(JSON)` 中的路径和类型: - -```sql title="Query" -SELECT DISTINCT arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b[]))) FROM test; -``` - -```text title="Response" -┌─arrayJoin(JSONAllPathsWithTypes(arrayJoin(json.a.b.:`Array(JSON)`)))──┐ -│ ('c','Int64') │ -│ ('d','String') │ -│ ('f','Array(Array(JSON(max_dynamic_types=8, max_dynamic_paths=64)))') │ -│ ('k.j','Int64') │ -│ ('e','Array(Nullable(Int64))') │ -└───────────────────────────────────────────────────────────────────────┘ -``` - -让我们从 `Array(JSON)` 列中读取子列: - -```sql title="Query" -SELECT json.a.b[].c.:Int64, json.a.b[].f[][].g.:Float64, json.a.b[].f[][].h.:Date FROM test; -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.c.:`Int64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.g.:`Float64`─┬─json.a.b.:`Array(JSON)`.f.:`Array(Array(JSON))`.h.:`Date`─┐ -│ [42,43,NULL] │ [[[42.42]],[],[[43.43]]] │ [[[NULL]],[],[['2020-01-01']]] │ -│ [] │ [] │ [] │ -│ [44,NULL] │ [[[NULL]],[[44.44]]] │ [[['2020-01-02']],[[NULL]]] │ -└────────────────────────────────────┴──────────────────────────────────────────────────────────────┴───────────────────────────────────────────────────────────┘ -``` - -我们还可以从嵌套的 `JSON` 列中读取子对象的子列: - -```sql title="Query" -SELECT json.a.b[].^k FROM test -``` - -```text title="Response" -┌─json.a.b.:`Array(JSON)`.^`k`─────────┐ -│ ['{"j":"1000"}','{}','{"j":"2000"}'] │ -│ [] │ -│ ['{}','{"j":"3000"}'] │ -└──────────────────────────────────────┘ -``` - - -## 处理包含 NULL 的 JSON 键 {#handling-json-keys-with-nulls} - -在我们的 JSON 实现中,`null` 与值的缺失被视为等同: - -```sql title="Query" -SELECT '{}'::JSON AS json1, '{"a" : null}'::JSON AS json2, json1 = json2 -``` - -```text title="Response" -┌─json1─┬─json2─┬─equals(json1, json2)─┐ -│ {} │ {} │ 1 │ -└───────┴───────┴──────────────────────┘ -``` - -这意味着无法确定原始 JSON 数据中,是包含某个路径且其值为 NULL,还是根本不包含该路径。 - - -## 处理包含点号的 JSON 键 {#handling-json-keys-with-dots} - -在内部实现中,JSON 列会以扁平化的形式存储所有路径和值。这意味着在默认情况下,下面这两个对象会被视为相同: - -```json -{"a" : {"b" : 42}} -{"a.b" : 42} -``` - -它们在内部都会以路径和值的二元组形式存储,即路径 `a.b` 和值 `42`。在格式化 JSON 时,我们始终基于以点分隔的路径片段来构造嵌套对象: - -```sql title="Query" -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a":{"b":"42"}} │ ['a.b'] │ ['a.b'] │ -└──────────────────┴──────────────────┴─────────────────────┴─────────────────────┘ -``` - -如你所见,最初的 JSON `{"a.b" : 42}` 现在被格式化为 `{"a" : {"b" : 42}}`。 - -这一限制还会导致在解析如下这种有效 JSON 对象时失败: - -```sql title="Query" -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json; -``` - -```text title="Response" -代码:117. DB::Exception:无法将数据插入 JSON 列:解析 JSON 对象时发现重复路径:a.b。您可以启用 type_json_skip_duplicated_paths 设置以在插入时跳过重复路径:在作用域 SELECT CAST('{"a.b" : 42, "a" : {"b" : "Hello, World"}}', 'JSON') AS json 中。(INCORRECT_DATA) -``` - -如果你希望保留带点号的键并避免将其格式化为嵌套对象,可以启用设置 [json_type_escape_dots_in_keys](/operations/settings/formats#json_type_escape_dots_in_keys)(自 `25.8` 版本起可用)。在这种情况下,解析时 JSON 键中的所有点号都会被转义为 `%2E`,并在格式化时再还原回来。 - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a" : {"b" : 42}}'::JSON AS json1, '{"a.b" : 42}'::JSON AS json2, JSONAllPaths(json1), JSONAllPaths(json2); -``` - -```text title="Response" -┌─json1────────────┬─json2────────┬─JSONAllPaths(json1)─┬─JSONAllPaths(json2)─┐ -│ {"a":{"b":"42"}} │ {"a.b":"42"} │ ['a.b'] │ ['a%2Eb'] │ -└──────────────────┴──────────────┴─────────────────────┴─────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, JSONAllPaths(json); -``` - -```text title="Response" -┌─json──────────────────────────────────┬─JSONAllPaths(json)─┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ ['a%2Eb','a.b'] │ -└───────────────────────────────────────┴────────────────────┘ -``` - -要将包含转义点号的键作为子列读取时,必须在子列名称中同样使用转义点号: - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┘ -``` - -注意:由于标识符解析器和分析器的限制,子列 `` json.`a.b` `` 等价于子列 `json.a.b`,且无法读取带有转义点号的路径: - - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON AS json, json.`a%2Eb`, json.`a.b`, json.a.b; -``` - -```text title="Response" -┌─json──────────────────────────────────┬─json.a%2Eb─┬─json.a.b─────┬─json.a.b─────┐ -│ {"a.b":"42","a":{"b":"Hello World!"}} │ 42 │ Hello World! │ Hello World! │ -└───────────────────────────────────────┴────────────┴──────────────┴──────────────┘ -``` - -此外,如果你想为键名中包含点号的 JSON 路径指定提示(或在 `SKIP`/`SKIP REGEX` 部分中使用它),则必须在提示中将点号写为转义形式: - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(`a%2Eb` UInt8) as json, json.`a%2Eb`, toTypeName(json.`a%2Eb`); -``` - -```text title="Response" -┌─json────────────────────────────────┬─json.a%2Eb─┬─toTypeName(json.a%2Eb)─┐ -│ {"a.b":42,"a":{"b":"Hello World!"}} │ 42 │ UInt8 │ -└─────────────────────────────────────┴────────────┴────────────────────────┘ -``` - -```sql title="Query" -SET json_type_escape_dots_in_keys=1; -SELECT '{"a.b" : 42, "a" : {"b" : "Hello World!"}}'::JSON(SKIP `a%2Eb`) as json, json.`a%2Eb`; -``` - -```text title="Response" -┌─json───────────────────────┬─json.a%2Eb─┐ -│ {"a":{"b":"你好世界!"}} │ ᴺᵁᴸᴸ │ -└────────────────────────────┴────────────┘ -``` - - -## 从数据中读取 JSON 类型 {#reading-json-type-from-data} - -所有文本格式 -([`JSONEachRow`](/interfaces/formats/JSONEachRow), -[`TSV`](/interfaces/formats/TabSeparated), -[`CSV`](/interfaces/formats/CSV), -[`CustomSeparated`](/interfaces/formats/CustomSeparated), -[`Values`](/interfaces/formats/Values) 等) 都支持读取 `JSON` 类型的数据。 - -示例: - -```sql title="Query" -SELECT json FROM format(JSONEachRow, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP d.e, SKIP REGEXP \'b.*\')', ' -{"json" : {"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}}} -{"json" : {"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}}} -{"json" : {"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"}} -{"json" : {"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43}} -{"json" : {"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}} -') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"你好,世界!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - -对于 `CSV`/`TSV`/等文本格式,会从包含 JSON 对象的字符串中解析 `JSON`: - - -```sql title="Query" -SELECT json FROM format(TSV, 'json JSON(a.b.c UInt32, SKIP a.b.d, SKIP REGEXP \'b.*\')', -'{"a" : {"b" : {"c" : 1, "d" : [0, 1]}}, "b" : "2020-01-01", "c" : 42, "d" : {"e" : {"f" : ["s1", "s2"]}, "i" : [1, 2, 3]}} -{"a" : {"b" : {"c" : 2, "d" : [2, 3]}}, "b" : [1, 2, 3], "c" : null, "d" : {"e" : {"g" : 43}, "i" : [4, 5, 6]}} -{"a" : {"b" : {"c" : 3, "d" : [4, 5]}}, "b" : {"c" : 10}, "e" : "Hello, World!"} -{"a" : {"b" : {"c" : 4, "d" : [6, 7]}}, "c" : 43} -{"a" : {"b" : {"c" : 5, "d" : [8, 9]}}, "b" : {"c" : 11, "j" : [1, 2, 3]}, "d" : {"e" : {"f" : ["s3", "s4"], "g" : 44}, "h" : "2020-02-02 10:00:00"}}') -``` - -```text title="Response" -┌─json──────────────────────────────────────────────────────────┐ -│ {"a":{"b":{"c":1}},"c":"42","d":{"i":["1","2","3"]}} │ -│ {"a":{"b":{"c":2}},"d":{"i":["4","5","6"]}} │ -│ {"a":{"b":{"c":3}},"e":"你好,世界!"} │ -│ {"a":{"b":{"c":4}},"c":"43"} │ -│ {"a":{"b":{"c":5}},"d":{"h":"2020-02-02 10:00:00.000000000"}} │ -└───────────────────────────────────────────────────────────────┘ -``` - - -## 达到 JSON 中动态路径数量的上限 {#reaching-the-limit-of-dynamic-paths-inside-json} - -`JSON` 数据类型在内部只能将有限数量的路径存储为单独的子列。 -默认情况下,此上限为 `1024`,但你可以在类型声明中通过参数 `max_dynamic_paths` 来修改。 - -当达到上限时,插入到 `JSON` 列中的所有新路径都会存储在一个共享的数据结构中。 -这些路径仍然可以作为子列读取, -但效率可能会较低([参见关于共享数据的章节](#shared-data-structure))。 -设置这个上限是为了避免出现数量巨大的不同子列,从而导致表无法正常使用。 - -下面我们来看看在几种不同场景下达到该上限时会发生什么。 - -### 在数据解析过程中达到上限 {#reaching-the-limit-during-data-parsing} - -在从数据中解析 `JSON` 对象时,当当前数据块的路径数量达到上限后, -所有新路径都会存储在一个共享数据结构中。我们可以使用以下两个内省函数 `JSONDynamicPaths`、`JSONSharedDataPaths`: - -```sql title="Query" -SELECT json, JSONDynamicPaths(json), JSONSharedDataPaths(json) FROM format(JSONEachRow, 'json JSON(max_dynamic_paths=3)', ' -{"json" : {"a" : {"b" : 42}, "c" : [1, 2, 3]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-01"}} -{"json" : {"a" : {"b" : 44}, "c" : [4, 5, 6]}} -{"json" : {"a" : {"b" : 43}, "d" : "2020-01-02", "e" : "Hello", "f" : {"g" : 42.42}}} -{"json" : {"a" : {"b" : 43}, "c" : [7, 8, 9], "f" : {"g" : 43.43}, "h" : "World"}} -') -``` - -```text title="Response" -┌─json───────────────────────────────────────────────────────────┬─JSONDynamicPaths(json)─┬─JSONSharedDataPaths(json)─┐ -│ {"a":{"b":"42"},"c":["1","2","3"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-01"} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"44"},"c":["4","5","6"]} │ ['a.b','c','d'] │ [] │ -│ {"a":{"b":"43"},"d":"2020-01-02","e":"Hello","f":{"g":42.42}} │ ['a.b','c','d'] │ ['e','f.g'] │ -│ {"a":{"b":"43"},"c":["7","8","9"],"f":{"g":43.43},"h":"World"} │ ['a.b','c','d'] │ ['f.g','h'] │ -└────────────────────────────────────────────────────────────────┴────────────────────────┴───────────────────────────┘ -``` - -正如我们所见,在插入路径 `e` 和 `f.g` 之后,已经达到了限制, -它们被写入到一个共享的数据结构中。 - -### 在 MergeTree 表引擎中合并数据部分时 {#during-merges-of-data-parts-in-mergetree-table-engines} - -在 `MergeTree` 表中合并若干数据部分的过程中,结果数据部分中的 `JSON` 列可能会达到动态路径数量的上限, -从而无法将源数据部分中的所有路径都作为子列进行存储。 -在这种情况下,ClickHouse 会选择哪些路径在合并后继续作为子列保留,哪些路径则存储在共享的数据结构中。 -在大多数情况下,ClickHouse 会尽量保留包含 -最多非空值的路径,并将最不常见的路径移到共享的数据结构中。不过,这仍然取决于具体实现。 - -我们来看一个这样的合并示例。 -首先,我们创建一个带有 `JSON` 列的表,将动态路径的限制设置为 `3`,然后插入包含 `5` 个不同路径的值: - - -```sql title="Query" -CREATE TABLE test (id UInt64, json JSON(max_dynamic_paths=3)) ENGINE=MergeTree ORDER BY id; -SYSTEM STOP MERGES test; -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as a) FROM numbers(5); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as b) FROM numbers(4); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as c) FROM numbers(3); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as d) FROM numbers(2); -INSERT INTO test SELECT number, formatRow('JSONEachRow', number as e) FROM numbers(1); -``` - -每次插入都会创建一个单独的数据片段,其中 `JSON` 列只包含一个路径: - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 5 │ ['a'] │ [] │ all_1_1_0 │ -│ 4 │ ['b'] │ [] │ all_2_2_0 │ -│ 3 │ ['c'] │ [] │ all_3_3_0 │ -│ 2 │ ['d'] │ [] │ all_4_4_0 │ -│ 1 │ ['e'] │ [] │ all_5_5_0 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -现在,让我们把所有部分合并起来,看看会发生什么: - -```sql title="Query" -SELECT - count(), - groupArrayArrayDistinct(JSONDynamicPaths(json)) AS dynamic_paths, - groupArrayArrayDistinct(JSONSharedDataPaths(json)) AS shared_data_paths, - _part -FROM test -GROUP BY _part -ORDER BY _part ASC -``` - -```text title="Response" -┌─count()─┬─dynamic_paths─┬─shared_data_paths─┬─_part─────┐ -│ 15 │ ['a','b','c'] │ ['d','e'] │ all_1_5_2 │ -└─────────┴───────────────┴───────────────────┴───────────┘ -``` - -正如我们看到的,ClickHouse 保留了最常见的路径 `a`、`b` 和 `c`,并将路径 `d` 和 `e` 放入了一个共享的数据结构中。 - - -## 共享数据结构 {#shared-data-structure} - -如前一节所述,当达到 `max_dynamic_paths` 限制时,所有新的路径都会存储在一个共享数据结构中。 -本节将详细介绍该共享数据结构的实现方式,以及如何从中读取路径子列。 - -有关用于检查 JSON 列内容的函数的详细信息,请参见[“自省函数”](/sql-reference/data-types/newjson#introspection-functions)一节。 - -### 内存中的共享数据结构 {#shared-data-structure-in-memory} - -在内存中,共享数据结构只是一个类型为 `Map(String, String)` 的子列,用于存储从扁平化 JSON 路径到二进制编码值的映射。 -要从中提取路径子列,只需遍历该 `Map` 列中的所有行,并尝试查找请求的路径及其对应的值。 - -### MergeTree 部件中的共享数据结构 {#shared-data-structure-in-merge-tree-parts} - -在 [MergeTree](../../engines/table-engines/mergetree-family/mergetree.md) 表中,我们将数据存储在数据部件中,这些部件会将所有内容存储在磁盘上(本地或远程)。而磁盘上的数据存储方式可能与内存中的不同。 -目前,在 MergeTree 数据部件中存在 3 种不同的共享数据结构序列化方式:`map`、`map_with_buckets` -和 `advanced`。 - -序列化版本由 MergeTree 设置 -[object_shared_data_serialization_version](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version) -和 [object_shared_data_serialization_version_for_zero_level_parts](../../operations/settings/merge-tree-settings.md#object_shared_data_serialization_version_for_zero_level_parts) 控制 -(零级部件是在向表中插入数据时创建的部件,在合并过程中生成的部件级别更高)。 - -注意:仅当使用 `v3` [对象序列化版本](../../operations/settings/merge-tree-settings.md#object_serialization_version) 时,才支持更改共享数据结构的序列化方式。 - -#### Map {#shared-data-map} - -在 `map` 序列化版本中,共享数据会序列化为一个类型为 `Map(String, String)` 的单列,其存储方式与内存中相同。 -要从这种序列化方式中读取路径子列,ClickHouse 会读取整个 `Map` 列,并在内存中提取请求的路径。 - -这种序列化方式在写入数据和读取整个 `JSON` 列时效率较高,但在读取路径子列时效率不高。 - -#### 带桶的 Map {#shared-data-map-with-buckets} - -在 `map_with_buckets` 序列化版本中,共享数据会序列化为 `N` 列(“桶”),每列的类型为 `Map(String, String)`。 -每个桶仅包含路径的一个子集。要从这种序列化方式中读取路径子列,ClickHouse -会从单个桶中读取整个 `Map` 列,并在内存中提取请求的路径。 - -这种序列化方式在写入数据和读取整个 `JSON` 列时效率较低,但在读取路径子列时效率更高, -因为它只会从所需的桶中读取数据。 - -桶的数量 `N` 由 MergeTree 设置 [object_shared_data_buckets_for_compact_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_compact_part)(默认值为 8) -和 [object_shared_data_buckets_for_wide_part]( -../../operations/settings/merge-tree-settings.md#object_shared_data_buckets_for_wide_part)(默认值为 32)控制。 - -#### Advanced {#shared-data-advanced} - -在 `advanced` 序列化版本中,共享数据会序列化为一种专门的数据结构,通过存储一些额外信息来最大化路径子列的读取性能,从而只读取请求路径的数据。 -这种序列化方式同样支持桶,因此每个桶也只包含路径的一个子集。 - -这种序列化方式在写入数据时效率较低(因此不建议将其用于零级部件),读取整个 `JSON` 列时的效率相比 `map` 序列化略低,但在读取路径子列时非常高效。 - -注意:由于在数据结构中存储了额外信息,与 `map` 和 `map_with_buckets` 序列化方式相比,这种序列化在磁盘上的存储空间占用更高。 - -如需更详细地了解新的共享数据序列化方式及其实现细节,请参阅这篇[博客文章](https://clickhouse.com/blog/json-data-type-gets-even-better)。 - - - -## 自省函数 {#introspection-functions} - -有多个函数可以用于查看 JSON 列的内容: - -* [`JSONAllPaths`](../functions/json-functions.md#JSONAllPaths) -* [`JSONAllPathsWithTypes`](../functions/json-functions.md#JSONAllPathsWithTypes) -* [`JSONDynamicPaths`](../functions/json-functions.md#JSONDynamicPaths) -* [`JSONDynamicPathsWithTypes`](../functions/json-functions.md#JSONDynamicPathsWithTypes) -* [`JSONSharedDataPaths`](../functions/json-functions.md#JSONSharedDataPaths) -* [`JSONSharedDataPathsWithTypes`](../functions/json-functions.md#JSONSharedDataPathsWithTypes) -* [`distinctDynamicTypes`](../aggregate-functions/reference/distinctdynamictypes.md) -* [`distinctJSONPaths and distinctJSONPathsAndTypes`](../aggregate-functions/reference/distinctjsonpaths.md) - -**示例** - -让我们来分析一下 [GH Archive](https://www.gharchive.org/) 数据集中日期为 `2020-01-01` 的内容: - -```sql title="Query" -SELECT arrayJoin(distinctJSONPaths(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -``` - -```text title="Response" -┌─arrayJoin(distinctJSONPaths(json))─────────────────────────┐ -│ actor.avatar_url │ -│ actor.display_login │ -│ actor.gravatar_id │ -│ actor.id │ -│ actor.login │ -│ actor.url │ -│ created_at │ -│ id │ -│ org.avatar_url │ -│ org.gravatar_id │ -│ org.id │ -│ org.login │ -│ org.url │ -│ payload.action │ -│ payload.before │ -│ payload.comment._links.html.href │ -│ payload.comment._links.pull_request.href │ -│ payload.comment._links.self.href │ -│ payload.comment.author_association │ -│ payload.comment.body │ -│ payload.comment.commit_id │ -│ payload.comment.created_at │ -│ payload.comment.diff_hunk │ -│ payload.comment.html_url │ -│ payload.comment.id │ -│ payload.comment.in_reply_to_id │ -│ payload.comment.issue_url │ -│ payload.comment.line │ -│ payload.comment.node_id │ -│ payload.comment.original_commit_id │ -│ payload.comment.original_position │ -│ payload.comment.path │ -│ payload.comment.position │ -│ payload.comment.pull_request_review_id │ -... -│ payload.release.node_id │ -│ payload.release.prerelease │ -│ payload.release.published_at │ -│ payload.release.tag_name │ -│ payload.release.tarball_url │ -│ payload.release.target_commitish │ -│ payload.release.upload_url │ -│ payload.release.url │ -│ payload.release.zipball_url │ -│ payload.size │ -│ public │ -│ repo.id │ -│ repo.name │ -│ repo.url │ -│ type │ -└─arrayJoin(distinctJSONPaths(json))─────────────────────────┘ -``` - -```sql -SELECT arrayJoin(distinctJSONPathsAndTypes(json)) -FROM s3('s3://clickhouse-public-datasets/gharchive/original/2020-01-01-*.json.gz', JSONAsObject) -SETTINGS date_time_input_format = 'best_effort' -``` - - -```text -┌─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┐ -│ ('actor.avatar_url',['String']) │ -│ ('actor.display_login',['String']) │ -│ ('actor.gravatar_id',['String']) │ -│ ('actor.id',['Int64']) │ -│ ('actor.login',['String']) │ -│ ('actor.url',['String']) │ -│ ('created_at',['DateTime']) │ -│ ('id',['String']) │ -│ ('org.avatar_url',['String']) │ -│ ('org.gravatar_id',['String']) │ -│ ('org.id',['Int64']) │ -│ ('org.login',['String']) │ -│ ('org.url',['String']) │ -│ ('payload.action',['String']) │ -│ ('payload.before',['String']) │ -│ ('payload.comment._links.html.href',['String']) │ -│ ('payload.comment._links.pull_request.href',['String']) │ -│ ('payload.comment._links.self.href',['String']) │ -│ ('payload.comment.author_association',['String']) │ -│ ('payload.comment.body',['String']) │ -│ ('payload.comment.commit_id',['String']) │ -│ ('payload.comment.created_at',['DateTime']) │ -│ ('payload.comment.diff_hunk',['String']) │ -│ ('payload.comment.html_url',['String']) │ -│ ('payload.comment.id',['Int64']) │ -│ ('payload.comment.in_reply_to_id',['Int64']) │ -│ ('payload.comment.issue_url',['String']) │ -│ ('payload.comment.line',['Int64']) │ -│ ('payload.comment.node_id',['String']) │ -│ ('payload.comment.original_commit_id',['String']) │ -│ ('payload.comment.original_position',['Int64']) │ -│ ('payload.comment.path',['String']) │ -│ ('payload.comment.position',['Int64']) │ -│ ('payload.comment.pull_request_review_id',['Int64']) │ -... -│ ('payload.release.node_id',['String']) │ -│ ('payload.release.prerelease',['Bool']) │ -│ ('payload.release.published_at',['DateTime']) │ -│ ('payload.release.tag_name',['String']) │ -│ ('payload.release.tarball_url',['String']) │ -│ ('payload.release.target_commitish',['String']) │ -│ ('payload.release.upload_url',['String']) │ -│ ('payload.release.url',['String']) │ -│ ('payload.release.zipball_url',['String']) │ -│ ('payload.size',['Int64']) │ -│ ('public',['Bool']) │ -│ ('repo.id',['Int64']) │ -│ ('repo.name',['String']) │ -│ ('repo.url',['String']) │ -│ ('type',['String']) │ -└─arrayJoin(distinctJSONPathsAndTypes(json))──────────────────┘ -``` - - -## 使用 ALTER MODIFY COLUMN 修改为 JSON 类型 {#alter-modify-column-to-json-type} - -可以对现有表执行修改操作,将列的类型更改为新的 `JSON` 类型。目前仅支持对 `String` 类型列执行 `ALTER` 操作。 - -**示例** - -```sql title="Query" -CREATE TABLE test (json String) ENGINE=MergeTree ORDER BY tuple(); -INSERT INTO test VALUES ('{"a" : 42}'), ('{"a" : 43, "b" : "Hello"}'), ('{"a" : 44, "b" : [1, 2, 3]}'), ('{"c" : "2020-01-01"}'); -ALTER TABLE test MODIFY COLUMN json JSON; -SELECT json, json.a, json.b, json.c FROM test; -``` - -```text title="Response" -┌─json─────────────────────────┬─json.a─┬─json.b──┬─json.c─────┐ -│ {"a":"42"} │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ -│ {"a":"43","b":"Hello"} │ 43 │ Hello │ ᴺᵁᴸᴸ │ -│ {"a":"44","b":["1","2","3"]} │ 44 │ [1,2,3] │ ᴺᵁᴸᴸ │ -│ {"c":"2020-01-01"} │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 │ -└──────────────────────────────┴────────┴─────────┴────────────┘ -``` - - -## JSON 类型值的比较 {#comparison-between-values-of-the-json-type} - -JSON 对象的比较规则与 Map 类型类似。 - -例如: - -```sql title="Query" -CREATE TABLE test (json1 JSON, json2 JSON) ENGINE=Memory; -INSERT INTO test FORMAT JSONEachRow -{"json1" : {}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : [1, 2, 3]}} -{"json1" : {"a" : 42}, "json2" : {"a" : "Hello"}} -{"json1" : {"a" : 42}, "json2" : {"b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 42, "b" : 42}} -{"json1" : {"a" : 42}, "json2" : {"a" : 41, "b" : 42}} - -SELECT json1, json2, json1 < json2, json1 = json2, json1 > json2 FROM test; -``` - -```text title="Response" -┌─json1──────┬─json2───────────────┬─less(json1, json2)─┬─equals(json1, json2)─┬─greater(json1, json2)─┐ -│ {} │ {} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"41"} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"42"} │ 0 │ 1 │ 0 │ -│ {"a":"42"} │ {"a":["1","2","3"]} │ 0 │ 0 │ 1 │ -│ {"a":"42"} │ {"a":"Hello"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"42","b":"42"} │ 1 │ 0 │ 0 │ -│ {"a":"42"} │ {"a":"41","b":"42"} │ 0 │ 0 │ 1 │ -└────────────┴─────────────────────┴────────────────────┴──────────────────────┴───────────────────────┘ -``` - -**注意:** 当两个路径中包含不同数据类型的值时,将根据 `Variant` 数据类型的[比较规则](/sql-reference/data-types/variant#comparing-values-of-variant-data)进行比较。 - - -## 更高效使用 JSON 类型的技巧 {#tips-for-better-usage-of-the-json-type} - -在创建 `JSON` 列并向其中加载数据之前,请考虑以下几点建议: - -- 先分析你的数据,并尽可能为更多路径提供带类型的路径提示(path hint)。这将显著提升存储和读取效率。 -- 考虑在实际使用中会需要哪些路径,以及哪些路径基本不会被使用。对于不需要的路径,在 `SKIP` 部分中进行指定;如有必要,再在 `SKIP REGEXP` 部分中指定。这将改善存储效果。 -- 不要将 `max_dynamic_paths` 参数设置得过高,否则可能会降低存储和读取效率。 - 虽然具体数值高度依赖于内存、CPU 等系统参数,但一个经验法则是:对于本地文件系统存储,不要将 `max_dynamic_paths` 设置为大于 10 000;对于远程文件系统存储,不要设置为大于 1024。 - - - -## 延伸阅读 {#further-reading} - -- [我们如何为 ClickHouse 构建强大的全新 JSON 数据类型](https://clickhouse.com/blog/a-new-powerful-json-data-type-for-clickhouse) -- [十亿文档 JSON 挑战:ClickHouse 对比 MongoDB、Elasticsearch 等](https://clickhouse.com/blog/json-bench-clickhouse-vs-mongodb-elasticsearch-duckdb-postgresql) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md deleted file mode 100644 index 39035adb49b..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/nullable.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: 'ClickHouse 中 Nullable 数据类型修饰符文档' -sidebar_label: 'Nullable(T)' -sidebar_position: 44 -slug: /sql-reference/data-types/nullable -title: 'Nullable(T)' -doc_type: 'reference' ---- - - - -# Nullable(T) {#nullablet} - -允许在 `T` 类型的正常值之外,额外存储表示“缺失值”的特殊标记([NULL](../../sql-reference/syntax.md))。例如,类型为 `Nullable(Int8)` 的列既可以存储 `Int8` 类型的值,而没有值的行则会存储 `NULL`。 - -`T` 不能是以下复合数据类型之一:[Array](../../sql-reference/data-types/array.md)、[Map](../../sql-reference/data-types/map.md) 和 [Tuple](../../sql-reference/data-types/tuple.md),但复合数据类型可以包含 `Nullable` 类型的值,例如 `Array(Nullable(Int8))`。 - -`Nullable` 类型的字段不能用于表索引中。 - -除非在 ClickHouse 服务器配置中另有指定,对于任意 `Nullable` 类型,`NULL` 是其默认值。 - - - -## 存储特性 {#storage-features} - -为了在表的列中存储 `Nullable` 类型的值,ClickHouse 除了用于存储实际值的普通文件外,还会使用一个单独的 `NULL` 掩码文件。掩码文件中的条目使 ClickHouse 能够区分每一行中该数据类型的 `NULL` 值和其默认值。由于需要额外的文件,`Nullable` 列相比类似的普通列会消耗更多的存储空间。 - -:::note -使用 `Nullable` 几乎总是会对性能产生负面影响,在设计数据库时请牢记这一点。 -::: - - - -## 查找 NULL {#finding-null} - -可以通过使用 `null` 子列,在无需读取整个列的情况下查找列中的 `NULL` 值。当对应的值为 `NULL` 时返回 `1`,否则返回 `0`。 - -**示例** - -查询: - -```sql -CREATE TABLE nullable (`n` Nullable(UInt32)) ENGINE = MergeTree ORDER BY tuple(); - -INSERT INTO nullable VALUES (1) (NULL) (2) (NULL); - -SELECT n.null FROM nullable; -``` - -结果: - -```text -┌─n.null─┐ -│ 0 │ -│ 1 │ -│ 0 │ -│ 1 │ -└────────┘ -``` - - -## 使用示例 {#usage-example} - -```sql -CREATE TABLE t_null(x Int8, y Nullable(Int8)) ENGINE TinyLog -``` - -```sql -INSERT INTO t_null VALUES (1, NULL), (2, 3) -``` - -```sql -SELECT x + y FROM t_null -``` - -```text -┌─plus(x, y)─┐ -│ ᴺᵁᴸᴸ │ -│ 5 │ -└────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md deleted file mode 100644 index c12491268b7..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/qbit.md +++ /dev/null @@ -1,81 +0,0 @@ ---- -description: '介绍 ClickHouse 中 QBit 数据类型的文档,该类型支持用于近似向量搜索的细粒度量化' -keywords: ['qbit', '数据类型'] -sidebar_label: 'QBit' -sidebar_position: 64 -slug: /sql-reference/data-types/qbit -title: 'QBit 数据类型' -doc_type: 'reference' ---- - -import ExperimentalBadge from '@theme/badges/ExperimentalBadge'; - - - -`QBit` 数据类型通过重新组织向量的存储方式来加速近似搜索。它不是将每个向量的所有元素存放在一起,而是把所有向量中相同二进制位的位置分组存储。 -这种方式在保持向量以全精度存储的同时,允许你在查询时选择细粒度的量化级别:读取更少的比特位可减少 I/O 并加快计算,读取更多比特位则可提高精度。你既能获得量化带来的数据传输和计算开销降低的速度优势,又能在需要时访问所有原始数据。 - -:::note -`QBit` 数据类型及其相关的距离函数目前为实验性功能。 -要启用它们,请先执行 `SET allow_experimental_qbit_type = 1`。 -如果你遇到问题,请在 [ClickHouse 仓库](https://github.com/clickhouse/clickhouse/issues) 中提交 issue。 -::: - -要声明一个 `QBit` 类型的列,请使用以下语法: - -```sql -column_name QBit(element_type, dimension) -``` - -* `element_type` – 每个向量元素的类型。允许的类型包括 `BFloat16`、`Float32` 和 `Float64`。 -* `dimension` – 每个向量中的元素数量。 - - -## 创建 QBit {#creating-qbit} - -在表的列定义中使用 `QBit` 类型: - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [1, 2, 3, 4, 5, 6, 7, 8]), (2, [9, 10, 11, 12, 13, 14, 15, 16]); -SELECT vec FROM test ORDER BY id; -``` - -```text -┌─vec──────────────────────┐ -│ [1,2,3,4,5,6,7,8] │ -│ [9,10,11,12,13,14,15,16] │ -└──────────────────────────┘ -``` - - -## QBit 子列 {#qbit-subcolumns} - -`QBit` 实现了一种子列访问模式,允许你访问已存储向量的各个位平面。每个比特位都可以使用 `.N` 语法进行访问,其中 `N` 是该比特位的位置: - -```sql -CREATE TABLE test (id UInt32, vec QBit(Float32, 8)) ENGINE = Memory; -INSERT INTO test VALUES (1, [0, 0, 0, 0, 0, 0, 0, 0]); -INSERT INTO test VALUES (1, [-0, -0, -0, -0, -0, -0, -0, -0]); -SELECT bin(vec.1) FROM test; -``` - -```text -┌─bin(tupleElement(vec, 1))─┐ -│ 00000000 │ -│ 11111111 │ -└───────────────────────────┘ -``` - -可访问的子列数量取决于元素类型: - -* `BFloat16`: 16 个子列(1-16) -* `Float32`: 32 个子列(1-32) -* `Float64`: 64 个子列(1-64) - - -## 向量搜索函数 {#vector-search-functions} - -以下是使用 `QBit` 数据类型进行向量相似度搜索的距离函数: - -* [`L2DistanceTransposed`](../functions/distance-functions.md#L2DistanceTransposed) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md deleted file mode 100644 index 2298a470d9d..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/simpleaggregatefunction.md +++ /dev/null @@ -1,85 +0,0 @@ ---- -description: 'SimpleAggregateFunction 数据类型的文档' -sidebar_label: 'SimpleAggregateFunction' -sidebar_position: 48 -slug: /sql-reference/data-types/simpleaggregatefunction -title: 'SimpleAggregateFunction 类型' -doc_type: 'reference' ---- - - - -# SimpleAggregateFunction 类型 {#simpleaggregatefunction-type} - - - -## 描述 {#description} - -`SimpleAggregateFunction` 数据类型用于存储聚合函数的中间状态,但不会像 [`AggregateFunction`](../../sql-reference/data-types/aggregatefunction.md) 类型那样存储其完整状态。 - -此类优化适用于满足以下性质的函数: - -> 将函数 `f` 应用于行集 `S1 UNION ALL S2` 的结果,可以通过分别对行集的各个部分应用 `f`,然后再对这些结果应用一次 `f` 获得:`f(S1 UNION ALL S2) = f(f(S1) UNION ALL f(S2))`。 - -该性质保证仅使用部分聚合结果就足以计算出合并后的结果,因此无需存储和处理额外数据。例如,`min` 或 `max` 函数在从中间结果计算最终结果时不需要额外步骤;而 `avg` 函数则需要同时记录总和与计数,在最终的 `Merge` 步骤中合并中间状态后,再将两者相除即可得到平均值。 - -聚合函数的值通常是通过调用在函数名后追加 [`-SimpleState`](/sql-reference/aggregate-functions/combinators#-simplestate) 组合器的聚合函数来生成的。 - - - -## 语法 {#syntax} - -```sql -SimpleAggregateFunction(聚合函数名, 参数类型...) -``` - -**参数** - -* `aggregate_function_name` - 聚合函数名称。 -* `Type` - 聚合函数参数类型。 - - -## 支持的函数 {#supported-functions} - -支持以下聚合函数: - -- [`any`](/sql-reference/aggregate-functions/reference/any) -- [`any_respect_nulls`](/sql-reference/aggregate-functions/reference/any) -- [`anyLast`](/sql-reference/aggregate-functions/reference/anylast) -- [`anyLast_respect_nulls`](/sql-reference/aggregate-functions/reference/anylast) -- [`min`](/sql-reference/aggregate-functions/reference/min) -- [`max`](/sql-reference/aggregate-functions/reference/max) -- [`sum`](/sql-reference/aggregate-functions/reference/sum) -- [`sumWithOverflow`](/sql-reference/aggregate-functions/reference/sumwithoverflow) -- [`groupBitAnd`](/sql-reference/aggregate-functions/reference/groupbitand) -- [`groupBitOr`](/sql-reference/aggregate-functions/reference/groupbitor) -- [`groupBitXor`](/sql-reference/aggregate-functions/reference/groupbitxor) -- [`groupArrayArray`](/sql-reference/aggregate-functions/reference/grouparray) -- [`groupUniqArrayArray`](../../sql-reference/aggregate-functions/reference/groupuniqarray.md) -- [`groupUniqArrayArrayMap`](../../sql-reference/aggregate-functions/combinators#-map) -- [`sumMap`](/sql-reference/aggregate-functions/reference/summap) -- [`minMap`](/sql-reference/aggregate-functions/reference/minmap) -- [`maxMap`](/sql-reference/aggregate-functions/reference/maxmap) - -:::note -`SimpleAggregateFunction(func, Type)` 的值都为相同的 `Type`, -因此与 `AggregateFunction` 类型不同,无需应用 `-Merge`/`-State` 组合器。 - -在使用相同聚合函数时,`SimpleAggregateFunction` 类型相比 `AggregateFunction` -具有更好的性能。 -::: - - - -## 示例 {#example} - - - -```sql -CREATE TABLE simple (id UInt64, val SimpleAggregateFunction(sum, Double)) ENGINE=AggregatingMergeTree ORDER BY id; -``` - -## 相关内容 {#related-content} - -* 博文:[在 ClickHouse 中使用聚合函数组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) - 博文:[在 ClickHouse 中使用聚合函数组合器](https://clickhouse.com/blog/aggregate-functions-combinators-in-clickhouse-for-arrays-maps-and-states) -* [AggregateFunction](/sql-reference/data-types/aggregatefunction) 类型。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md deleted file mode 100644 index f38b69c1ffd..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/expression.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'Expression 特殊数据类型文档' -sidebar_label: 'Expression' -sidebar_position: 58 -slug: /sql-reference/data-types/special-data-types/expression -title: 'Expression' -doc_type: 'reference' ---- - -# 表达式 {#expression} - -在高阶函数中,表达式用于表示 lambda。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md deleted file mode 100644 index 8f61060fd94..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/index.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: 'ClickHouse 中用于在查询执行期间存储中间结果的特殊数据类型概览' -sidebar_label: '特殊数据类型' -sidebar_position: 55 -slug: /sql-reference/data-types/special-data-types/ -title: '特殊数据类型' -doc_type: 'reference' ---- - -# 特殊数据类型 {#special-data-types} - -特殊数据类型的值无法被序列化以保存到表中或输出到查询结果中,但可以在查询执行过程中作为中间结果使用。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md deleted file mode 100644 index 6ec3d8c4985..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/interval.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -description: 'Interval 特殊数据类型文档' -sidebar_label: 'Interval' -sidebar_position: 61 -slug: /sql-reference/data-types/special-data-types/interval -title: 'Interval' -doc_type: 'reference' ---- - - - -# Interval {#interval} - -表示时间和日期间隔的一类数据类型。[INTERVAL](/sql-reference/operators#interval) 运算符所产生的结果类型。 - -结构: - -* 以无符号整数值表示的时间间隔。 -* 间隔的类型。 - -支持的间隔类型: - -* `NANOSECOND` -* `MICROSECOND` -* `MILLISECOND` -* `SECOND` -* `MINUTE` -* `HOUR` -* `DAY` -* `WEEK` -* `MONTH` -* `QUARTER` -* `YEAR` - -对于每种间隔类型,都有一个对应的数据类型。例如,`DAY` 间隔对应 `IntervalDay` 数据类型: - -```sql -SELECT toTypeName(INTERVAL 4 DAY) -``` - -```text -┌─toTypeName(toIntervalDay(4))─┐ -│ IntervalDay │ -└──────────────────────────────┘ -``` - - -## 使用注意事项 {#usage-remarks} - -可以将 `Interval` 类型的值与 [Date](../../../sql-reference/data-types/date.md) 和 [DateTime](../../../sql-reference/data-types/datetime.md) 类型的值一起用于算术运算。例如,可以在当前时间的基础上加 4 天: - -```sql -SELECT now() AS current_date_time, current_date_time + INTERVAL 4 DAY -``` - -```text -┌───current_date_time─┬─plus(now(), toIntervalDay(4))─┐ -│ 2019-10-23 10:58:45 │ 2019-10-27 10:58:45 │ -└─────────────────────┴───────────────────────────────┘ -``` - -也可以同时使用多个时间区间: - -```sql -SELECT now() AS current_date_time, current_date_time + (INTERVAL 4 DAY + INTERVAL 3 HOUR) -``` - -```text -┌───current_date_time─┬─plus(current_date_time, plus(toIntervalDay(4), toIntervalHour(3)))─┐ -│ 2024-08-08 18:31:39 │ 2024-08-12 21:31:39 │ -└─────────────────────┴────────────────────────────────────────────────────────────────────┘ -``` - -以及比较不同时间区间的数值: - -```sql -SELECT toIntervalMicrosecond(3600000000) = toIntervalHour(1); -``` - -```text -┌─less(toIntervalMicrosecond(179999999), toIntervalMinute(3))─┐ -│ 1 │ -└─────────────────────────────────────────────────────────────┘ -``` - - -## 另请参阅 {#see-also} - -- [INTERVAL](/sql-reference/operators#interval) 运算符 -- [toInterval](/sql-reference/functions/type-conversion-functions#tointervalyear) 类型转换函数 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md deleted file mode 100644 index 70a8479f517..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/nothing.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -description: 'Nothing 特殊数据类型文档' -sidebar_label: 'Nothing' -sidebar_position: 60 -slug: /sql-reference/data-types/special-data-types/nothing -title: 'Nothing' -doc_type: 'reference' ---- - -# Nothing {#nothing} - -此数据类型的唯一作用是表示不期望有值的情况。因此,无法创建一个类型为 `Nothing` 的值。 - -例如,字面量 [NULL](/sql-reference/syntax#null) 的类型是 `Nullable(Nothing)`。更多信息参见 [Nullable](../../../sql-reference/data-types/nullable.md)。 - -`Nothing` 类型也可用于表示空数组: - -```sql -SELECT toTypeName(array()) -``` - -```text -┌─toTypeName(array())─┐ -│ Array(Nothing) │ -└─────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md deleted file mode 100644 index c2bd037afaf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/special-data-types/set.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -description: '关于用于 IN 表达式的 Set 特殊数据类型的文档' -sidebar_label: 'Set' -sidebar_position: 59 -slug: /sql-reference/data-types/special-data-types/set -title: 'Set' -doc_type: 'reference' ---- - -# Set {#set} - -用于 [IN](/sql-reference/operators/in) 表达式右侧的集合。 \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md deleted file mode 100644 index ef5437df094..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/string.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -description: 'ClickHouse 中 String 数据类型文档' -sidebar_label: 'String' -sidebar_position: 8 -slug: /sql-reference/data-types/string -title: 'String' -doc_type: 'reference' ---- - - - -# String {#string} - -任意长度的字符串。长度不受限制。其值可以包含任意字节序列,包括空字节(null byte)。 -`String` 类型替代了其他数据库管理系统中的 `VARCHAR`、`BLOB`、`CLOB` 等类型。 - -在创建表时,可以为字符串字段指定数值参数(例如 `VARCHAR(255)`),但 ClickHouse 会忽略这些参数。 - -别名: - -- `String` — `LONGTEXT`、`MEDIUMTEXT`、`TINYTEXT`、`TEXT`、`LONGBLOB`、`MEDIUMBLOB`、`TINYBLOB`、`BLOB`、`VARCHAR`、`CHAR`、`CHAR LARGE OBJECT`、`CHAR VARYING`、`CHARACTER LARGE OBJECT`、`CHARACTER VARYING`、`NCHAR LARGE OBJECT`、`NCHAR VARYING`、`NATIONAL CHARACTER LARGE OBJECT`、`NATIONAL CHARACTER VARYING`、`NATIONAL CHAR VARYING`、`NATIONAL CHARACTER`、`NATIONAL CHAR`、`BINARY LARGE OBJECT`、`BINARY VARYING`, - - - -## 编码 {#encodings} - -ClickHouse 没有“编码”这一概念。`String` 类型的值可以包含任意字节序列,并会按原样存储和输出。 -如果你需要存储文本,我们建议使用 UTF-8 编码。至少在你的终端使用 UTF-8(同样是推荐的)时,你可以在不进行转换的情况下读写这些值。 -同样地,一些用于处理字符串的函数提供了不同的变体,这些变体在假定字符串包含的是表示 UTF-8 编码文本的字节序列的前提下工作。 -例如,[length](/sql-reference/functions/array-functions#length) 函数按字节计算字符串长度,而 [lengthUTF8](../functions/string-functions.md#lengthUTF8) 函数在假定该值为 UTF-8 编码的前提下,按 Unicode 码点计算字符串长度。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md deleted file mode 100644 index 684fb3c9bcf..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time.md +++ /dev/null @@ -1,133 +0,0 @@ ---- -description: 'ClickHouse 中 Time 数据类型的文档,该类型以秒级精度存储时间' -slug: /sql-reference/data-types/time -sidebar_position: 15 -sidebar_label: 'Time' -title: 'Time' -doc_type: 'reference' ---- - - - -# Time {#time} - -数据类型 `Time` 表示一个由小时、分钟和秒组成的时间。 -它独立于任何日历日期,适用于不需要日、月、年部分的取值。 - -语法: - -```sql -时间 -``` - -文本表示的范围:[-999:59:59, 999:59:59]。 - -精度:1 秒。 - - -## 实现细节 {#implementation-details} - -**表示与性能**。 -数据类型 `Time` 在内部存储为一个带符号的 32 位整数,用于编码秒数。 -`Time` 和 `DateTime` 类型的值具有相同的字节大小,因此性能相当。 - -**规范化**。 -在将字符串解析为 `Time` 时,时间组件会被规范化,但不会进行有效性校验。 -例如,`25:70:70` 会被解释为 `26:11:10`。 - -**负值**。 -前导负号会被支持并保留。 -负值通常源自对 `Time` 值进行算术运算。 -对于 `Time` 类型,负输入在文本(例如 `'-01:02:03'`)和数值输入(例如 `-3723`)两种形式中都会被保留。 - -**饱和**。 -一天中的时间组件会被限制在 [-999:59:59, 999:59:59] 范围内。 -小时数超出 999(或小于 -999) 的值,在文本表示和往返时都会被表示为 `999:59:59`(或 `-999:59:59`)。 - -**时区**。 -`Time` 不支持时区,即在解释 `Time` 值时不带区域上下文。 -将时区作为类型参数指定给 `Time`,或在创建值时指定时区,都会抛出错误。 -同样,对 `Time` 列应用或更改时区的尝试也不被支持,并会导致错误。 -`Time` 值不会在不同的时区下被静默重新解释。 - - - -## 示例 {#examples} - -**1.** 创建一个包含 `Time` 类型列的表,并向其中插入数据: - -```sql -CREATE TABLE tab -( - `event_id` UInt8, - `time` Time -) -ENGINE = TinyLog; -``` - -```sql --- 解析时间 --- - 从字符串解析, --- - 从整数解析(解释为自 00:00:00 起的秒数)。 -INSERT INTO tab VALUES (1, '14:30:25'), (2, 52225); - -SELECT * FROM tab ORDER BY event_id; -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**2.** 按 `Time` 值过滤 - -```sql -SELECT * FROM tab WHERE time = toTime('14:30:25') -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -可以在 `WHERE` 谓词中使用字符串值来过滤 `Time` 列的值,它会被自动转换为 `Time`: - -```sql -SELECT * FROM tab WHERE time = '14:30:25' -``` - -```text - ┌─event_id─┬──────time─┐ -1. │ 1 │ 14:30:25 │ -2. │ 2 │ 14:30:25 │ - └──────────┴───────────┘ -``` - -**3.** 查看结果类型: - -```sql -SELECT CAST('14:30:25' AS Time) AS column, toTypeName(column) AS type -``` - -```text - ┌────column─┬─type─┐ -1. │ 14:30:25 │ Time │ - └───────────┴──────┘ -``` - - -## 另请参阅 {#see-also} - -- [类型转换函数](../functions/type-conversion-functions.md) -- [用于处理日期和时间的函数](../functions/date-time-functions.md) -- [用于处理数组的函数](../functions/array-functions.md) -- [`date_time_input_format` 设置](../../operations/settings/settings-formats.md#date_time_input_format) -- [`date_time_output_format` 设置](../../operations/settings/settings-formats.md#date_time_output_format) -- [`timezone` 服务器配置参数](../../operations/server-configuration-parameters/settings.md#timezone) -- [`session_timezone` 设置](../../operations/settings/settings.md#session_timezone) -- [`DateTime` 数据类型](datetime.md) -- [`Date` 数据类型](date.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md deleted file mode 100644 index ed50ce5c204..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/time64.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'ClickHouse 中 Time64 数据类型的文档,用于以亚秒级精度存储时间' -slug: /sql-reference/data-types/time64 -sidebar_position: 17 -sidebar_label: 'Time64' -title: 'Time64' -doc_type: 'reference' ---- - - - -# Time64 {#time64} - -数据类型 `Time64` 表示带有小数秒的一天中的时刻(time-of-day)。 -它不包含任何日历日期组件(日、月、年)。 -参数 `precision` 定义小数位数,因此也就定义了最小时间粒度(tick size)。 - -时间粒度(精度):10-precision 秒。有效范围:0..9。常用取值为 3(毫秒)、6(微秒)和 9(纳秒)。 - -**语法:** - -```sql -Time64(精度) -``` - -在内部,`Time64` 以有符号 64 位十进制数(Decimal64)的形式存储秒的小数部分。 -时间精度由 `precision` 参数决定。 -不支持时区:为 `Time64` 指定时区会抛出错误。 - -与 `DateTime64` 不同,`Time64` 不存储日期部分。 -另见 [`Time`](../../sql-reference/data-types/time.md)。 - -文本表示范围:当 `precision = 3` 时为 [-999:59:59.000, 999:59:59.999]。一般情况下,最小值为 `-999:59:59`,最大值为 `999:59:59`,并且最多带有 `precision` 位小数(例如,当 `precision = 9` 时,最小值为 `-999:59:59.999999999`)。 - - -## 实现细节 {#implementation-details} - -**表示形式**。 -带符号的 `Decimal64` 值,用于表示具有 `precision` 位小数的秒的小数部分。 - -**规范化**。 -将字符串解析为 `Time64` 时,时间组件会被规范化,而不会进行合法性校验。 -例如,`25:70:70` 会被解释为 `26:11:10`。 - -**负值**。 -支持并保留前导负号。 -负值通常来源于对 `Time64` 值进行算术运算。 -对于 `Time64`,文本输入(例如 `'-01:02:03.123'`)和数值输入(例如 `-3723.123`)中的负值都会被保留。 - -**饱和**。 -在拆解为组件或序列化为文本时,一天中的时间部分会被限制在区间 [-999:59:59.xxx, 999:59:59.xxx] 内。 -存储的数值可能超出该范围;然而,任何组件提取(小时、分钟、秒)和文本表示都会使用饱和值。 - -**时区**。 -`Time64` 不支持时区。 -在创建 `Time64` 类型或值时指定时区会抛出错误。 -同样,尝试对 `Time64` 列应用或更改时区也不被支持,并会导致错误。 - - - -## 示例 {#examples} - -1. 创建一个包含 `Time64` 类型列的表,并向其中插入数据: - -```sql -CREATE TABLE tab64 -( - `event_id` UInt8, - `time` Time64(3) -) -ENGINE = TinyLog; -``` - -```sql --- 解析 Time64 --- - 从字符串解析, --- - 从自 00:00:00 起的秒数解析(小数部分根据精度确定)。 -INSERT INTO tab64 VALUES (1, '14:30:25'), (2, 52225.123), (3, '14:30:25'); - -SELECT * FROM tab64 ORDER BY event_id; -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 2 │ 14:30:25.123 │ -3. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -2. 按 `Time64` 值进行过滤 - -```sql -SELECT * FROM tab64 WHERE time = toTime64('14:30:25', 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 1 │ 14:30:25.000 │ -2. │ 3 │ 14:30:25.000 │ - └──────────┴──────────────┘ -``` - -```sql -SELECT * FROM tab64 WHERE time = toTime64(52225.123, 3); -``` - -```text - ┌─event_id─┬────────time─┐ -1. │ 2 │ 14:30:25.123 │ - └──────────┴──────────────┘ -``` - -注意:`toTime64` 会根据指定的精度,将数字字面量解析为带有小数部分的秒数,因此请显式提供预期的小数位数。 - -3. 检查结果类型: - -```sql -SELECT CAST('14:30:25.250' AS Time64(3)) AS column, toTypeName(column) AS type; -``` - -```text - ┌────────column─┬─type──────┐ -1. │ 14:30:25.250 │ Time64(3) │ - └───────────────┴───────────┘ -``` - -**另请参阅** - -* [类型转换函数](../../sql-reference/functions/type-conversion-functions.md) -* [处理日期和时间的函数](../../sql-reference/functions/date-time-functions.md) -* [`date_time_input_format` 设置](../../operations/settings/settings-formats.md#date_time_input_format) -* [`date_time_output_format` 设置](../../operations/settings/settings-formats.md#date_time_output_format) -* [`timezone` 服务器配置参数](../../operations/server-configuration-parameters/settings.md#timezone) -* [`session_timezone` 设置](../../operations/settings/settings.md#session_timezone) -* [处理日期和时间的运算符](../../sql-reference/operators/index.md#operators-for-working-with-dates-and-times) -* [`Date` 数据类型](../../sql-reference/data-types/date.md) -* [`Time` 数据类型](../../sql-reference/data-types/time.md) -* [`DateTime` 数据类型](../../sql-reference/data-types/datetime.md) diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md deleted file mode 100644 index 2f12b25a5d4..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/tuple.md +++ /dev/null @@ -1,184 +0,0 @@ ---- -description: 'ClickHouse 中 Tuple 数据类型文档' -sidebar_label: 'Tuple(T1, T2, ...)' -sidebar_position: 34 -slug: /sql-reference/data-types/tuple -title: 'Tuple(T1, T2, ...)' -doc_type: 'reference' ---- - -# Tuple(T1, T2, ...) {#tuplet1-t2} - -一个由多个元素组成的元组,每个元素都有各自的[类型](/sql-reference/data-types)。元组必须至少包含一个元素。 - -元组用于对列进行临时分组。在查询中使用 IN 表达式时可以对列进行分组,也可用于指定 lambda 函数的某些形式参数。更多信息请参阅 [IN 运算符](../../sql-reference/operators/in.md) 和 [高阶函数](/sql-reference/functions/overview#higher-order-functions) 章节。 - -元组可以作为查询结果返回。在这种情况下,对于除 JSON 以外的文本格式,值会以逗号分隔并包裹在圆括号中。在 JSON 格式中,元组会以数组形式输出(使用方括号)。 - -## 创建元组 {#creating-tuples} - -你可以使用函数创建元组: - -```sql -tuple(T1, T2, ...) -``` - -创建元组的示例: - -```sql -SELECT tuple(1, 'a') AS x, toTypeName(x) -``` - -```text -┌─x───────┬─toTypeName(tuple(1, 'a'))─┐ -│ (1,'a') │ Tuple(UInt8, String) │ -└─────────┴───────────────────────────┘ -``` - -一个元组(Tuple)可以只包含一个元素 - -示例: - -```sql -SELECT tuple('a') AS x; -``` - -```text -┌─x─────┐ -│ ('a') │ -└───────┘ -``` - -语法 `(tuple_element1, tuple_element2)` 可用于在不调用 `tuple()` 函数的情况下创建一个包含多个元素的元组。 - -示例: - -```sql -SELECT (1, 'a') AS x, (today(), rand(), 'someString') AS y, ('a') AS not_a_tuple; -``` - -```text -┌─x───────┬─y──────────────────────────────────────┬─not_a_tuple─┐ -│ (1,'a') │ ('2022-09-21',2006973416,'someString') │ a │ -└─────────┴────────────────────────────────────────┴─────────────┘ -``` - -## 数据类型检测 {#data-type-detection} - -在动态创建 tuple 时,ClickHouse 会将 tuple 参数的类型推断为能够容纳给定参数值的最小类型。 如果该值为 [NULL](/operations/settings/formats#input_format_null_as_default),则推断出的类型为 [Nullable](../../sql-reference/data-types/nullable.md)。 - -自动数据类型检测示例: - -```sql -SELECT tuple(1, NULL) AS x, toTypeName(x) -``` - -```text -┌─x─────────┬─toTypeName(tuple(1, NULL))──────┐ -│ (1, NULL) │ Tuple(UInt8, Nullable(Nothing)) │ -└───────────┴─────────────────────────────────┘ -``` - -## 引用 Tuple 元素 {#referring-to-tuple-elements} - -Tuple 元素可以通过名称或索引进行引用: - -```sql -CREATE TABLE named_tuples (`a` Tuple(s String, i Int64)) ENGINE = Memory; -INSERT INTO named_tuples VALUES (('y', 10)), (('x',-10)); - -SELECT a.s FROM named_tuples; -- 按名称访问 -SELECT a.2 FROM named_tuples; -- 按索引访问 -``` - -结果: - -```text -┌─a.s─┐ -│ y │ -│ x │ -└─────┘ - -┌─tupleElement(a, 2)─┐ -│ 10 │ -│ -10 │ -└────────────────────┘ -``` - -## 使用 Tuple 的比较操作 {#comparison-operations-with-tuple} - -两个 Tuple 的比较是通过从左到右依次比较它们的元素来完成的。若第一个 Tuple 的某个元素大于(小于)第二个 Tuple 中对应的元素,则认为第一个 Tuple 大于(小于)第二个 Tuple;否则(当这两个元素相等时),继续比较下一个元素。 - -示例: - -```sql -SELECT (1, 'z') > (1, 'a') c1, (2022, 01, 02) > (2023, 04, 02) c2, (1,2,3) = (3,2,1) c3; -``` - -```text -┌─c1─┬─c2─┬─c3─┐ -│ 1 │ 0 │ 0 │ -└────┴────┴────┘ -``` - -实际案例: - -```sql -CREATE TABLE test -( - `year` Int16, - `month` Int8, - `day` Int8 -) -ENGINE = Memory AS -SELECT * -FROM values((2022, 12, 31), (2000, 1, 1)); - -SELECT * FROM test; - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -│ 2000 │ 1 │ 1 │ -└──────┴───────┴─────┘ - -SELECT * -FROM test -WHERE (year, month, day) > (2010, 1, 1); - -┌─year─┬─month─┬─day─┐ -│ 2022 │ 12 │ 31 │ -└──────┴───────┴─────┘ -CREATE TABLE test -( - `key` Int64, - `duration` UInt32, - `value` Float64 -) -ENGINE = Memory AS -SELECT * -FROM values((1, 42, 66.5), (1, 42, 70), (2, 1, 10), (2, 2, 0)); - -SELECT * FROM test; - -┌─key─┬─duration─┬─value─┐ -│ 1 │ 42 │ 66.5 │ -│ 1 │ 42 │ 70 │ -│ 2 │ 1 │ 10 │ -│ 2 │ 2 │ 0 │ -└─────┴──────────┴───────┘ - --- 查找每个键中持续时间最大的值,如果持续时间相同,则选择数值最大的 - -SELECT - key, - max(duration), - argMax(value, (duration, value)) -FROM test -GROUP BY key -ORDER BY key ASC; - -┌─key─┬─max(duration)─┬─argMax(value, tuple(duration, value))─┐ -│ 1 │ 42 │ 70 │ -│ 2 │ 2 │ 0 │ -└─────┴───────────────┴───────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md deleted file mode 100644 index 001aca6c2d5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/uuid.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -description: 'ClickHouse 中 UUID 数据类型文档' -sidebar_label: 'UUID' -sidebar_position: 24 -slug: /sql-reference/data-types/uuid -title: 'UUID' -doc_type: 'reference' ---- - - - -# UUID {#uuid} - -通用唯一标识符(UUID)是一种用于标识记录的 16 字节值。有关 UUID 的详细信息,请参阅 [维基百科](https://en.wikipedia.org/wiki/Universally_unique_identifier)。 - -尽管存在不同的 UUID 变体(参见[此处](https://datatracker.ietf.org/doc/html/draft-ietf-uuidrev-rfc4122bis)),ClickHouse 并不会校验插入的 UUID 是否符合某个特定变体。 -在 SQL 层面,UUID 在内部被视为由 16 个随机字节组成的序列,并采用 [8-4-4-4-12 的表示形式](https://en.wikipedia.org/wiki/Universally_unique_identifier#Textual_representation)。 - -UUID 值示例: - -```text -61f0c404-5cb3-11e7-907b-a6006ad3dba0 -``` - -默认的 UUID 全为零。例如,在插入一条新记录但未为 UUID 列提供值时,将使用该值: - -```text -00000000-0000-0000-0000-000000000000 -``` - -因历史原因,UUID 在排序时是依据其后半部分进行排序的。 -因此,UUID 不应直接用作表的主键、排序键或分区键。 - -示例: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY uuid; -``` - -结果: - -```text -┌─uuid─────────────────────────────────┐ -│ 36a0b67c-b74a-4640-803b-e44bb4547e3c │ -│ 3a00aeb8-2605-4eec-8215-08c0ecb51112 │ -│ 3fda7c49-282e-421a-85ab-c5684ef1d350 │ -│ 16ab55a7-45f6-44a8-873c-7a0b44346b3e │ -│ e3776711-6359-4f22-878d-bf290d052c85 │ -│ [...] │ -│ 9eceda2f-6946-40e3-b725-16f2709ca41a │ -│ 03644f74-47ba-4020-b865-be5fd4c8c7ff │ -│ ce3bc93d-ab19-4c74-b8cc-737cb9212099 │ -│ b7ad6c91-23d6-4b5e-b8e4-a52297490b56 │ -│ 06892f64-cc2d-45f3-bf86-f5c5af5768a9 │ -└──────────────────────────────────────┘ -``` - -作为一种变通方案,可以将 UUID 转换为具有更直观排序顺序的类型。 - -示例:转换为 UInt128: - -```sql -CREATE TABLE tab (uuid UUID) ENGINE = Memory; -INSERT INTO tab SELECT generateUUIDv4() FROM numbers(50); -SELECT * FROM tab ORDER BY toUInt128(uuid); -``` - -结果: - -```sql -┌─uuid─────────────────────────────────┐ -│ 018b81cd-aca1-4e9c-9e56-a84a074dc1a8 │ -│ 02380033-c96a-438e-913f-a2c67e341def │ -│ 057cf435-7044-456a-893b-9183a4475cea │ -│ 0a3c1d4c-f57d-44cc-8567-60cb0c46f76e │ -│ 0c15bf1c-8633-4414-a084-7017eead9e41 │ -│ [...] │ -│ f808cf05-ea57-4e81-8add-29a195bde63d │ -│ f859fb5d-764b-4a33-81e6-9e4239dae083 │ -│ fb1b7e37-ab7b-421a-910b-80e60e2bf9eb │ -│ fc3174ff-517b-49b5-bfe2-9b369a5c506d │ -│ fece9bf6-3832-449a-b058-cd1d70a02c8b │ -└──────────────────────────────────────┘ -``` - - -## 生成 UUID {#generating-uuids} - -ClickHouse 提供了 [generateUUIDv4](../../sql-reference/functions/uuid-functions.md) 函数,用于生成随机的第 4 版 UUID 值。 - - - -## 使用示例 {#usage-example} - -**示例 1** - -此示例演示如何创建一个带有 UUID 列的表,并向该表插入一个值。 - -```sql -CREATE TABLE t_uuid (x UUID, y String) ENGINE=TinyLog - -INSERT INTO t_uuid SELECT generateUUIDv4(), '示例 1' - -SELECT * FROM t_uuid -``` - -结果: - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ 示例 1 │ -└──────────────────────────────────────┴───────────┘ -``` - -**示例 2** - -在此示例中,插入记录时未指定 UUID 列的值,因此将插入默认的 UUID 值: - -```sql -INSERT INTO t_uuid (y) VALUES ('示例 2') - -SELECT * FROM t_uuid -``` - -```text -┌────────────────────────────────────x─┬─y─────────┐ -│ 417ddc5d-e556-4d27-95dd-a34d84e46a50 │ 示例 1 │ -│ 00000000-0000-0000-0000-000000000000 │ 示例 2 │ -└──────────────────────────────────────┴───────────┘ -``` - - -## 限制 {#restrictions} - -`UUID` 数据类型只支持 [`String`](../../sql-reference/data-types/string.md) 数据类型也支持的函数(例如 [`min`](/sql-reference/aggregate-functions/reference/min)、[`max`](/sql-reference/aggregate-functions/reference/max) 和 [`count`](/sql-reference/aggregate-functions/reference/count))。 - -`UUID` 数据类型不支持算术运算(例如 [`abs`](/sql-reference/functions/arithmetic-functions#abs))或聚合函数,例如 [`sum`](/sql-reference/aggregate-functions/reference/sum) 和 [`avg`](/sql-reference/aggregate-functions/reference/avg)。 diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md deleted file mode 100644 index c1fe3fefefb..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/data-types/variant.md +++ /dev/null @@ -1,490 +0,0 @@ ---- -description: 'ClickHouse 中 Variant 数据类型文档' -sidebar_label: 'Variant(T1, T2, ...)' -sidebar_position: 40 -slug: /sql-reference/data-types/variant -title: 'Variant(T1, T2, ...)' -doc_type: 'reference' ---- - -# Variant(T1, T2, ...) {#variantt1-t2} - -此类型表示由其他数据类型构成的联合类型。类型 `Variant(T1, T2, ..., TN)` 表示该类型的每一行都包含一个值,该值要么是类型 `T1`,要么是 `T2`,……,要么是 `TN`,或者都不是(即为 `NULL` 值)。 - -嵌套类型的顺序无关紧要:Variant(T1, T2) = Variant(T2, T1)。 -嵌套类型可以是除 Nullable(...)、LowCardinality(Nullable(...)) 和 Variant(...) 之外的任意类型。 - -:::note -不建议将相似的类型作为变体(例如不同的数值类型,如 `Variant(UInt32, Int64)`,或不同的日期类型,如 `Variant(Date, DateTime)`), -因为处理这类类型的值可能会导致歧义。默认情况下,创建此类 `Variant` 类型会抛出异常,但可以通过设置 `allow_suspicious_variant_types` 来放宽此限制。 -::: - -## 创建 Variant 类型 {#creating-variant} - -在表的列定义中使用 `Variant` 类型: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v FROM test; -``` - -```text -┌─v─────────────┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ Hello, World! │ -│ [1,2,3] │ -└───────────────┘ -``` - -对普通列使用 CAST: - -```sql -SELECT toTypeName(variant) AS type_name, '你好,世界!'::Variant(UInt64, String, Array(UInt64)) as variant; -``` - -```text -┌─type_name──────────────────────────────┬─variant───────┐ -│ Variant(Array(UInt64), String, UInt64) │ Hello, World! │ -└────────────────────────────────────────┴───────────────┘ -``` - -在参数之间不存在公共类型时使用函数 `if/multiIf`(需要启用设置 `use_variant_as_common_type`): - -```sql -SET use_variant_as_common_type = 1; -SELECT if(number % 2, number, range(number)) as variant FROM numbers(5); -``` - -```text -┌─variant───┐ -│ [] │ -│ 1 │ -│ [0,1] │ -│ 3 │ -│ [0,1,2,3] │ -└───────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT multiIf((number % 4) = 0, 42, (number % 4) = 1, [1, 2, 3], (number % 4) = 2, 'Hello, World!', NULL) AS variant FROM numbers(4); -``` - -```text -┌─variant───────┐ -│ 42 │ -│ [1,2,3] │ -│ Hello, World! │ -│ ᴺᵁᴸᴸ │ -└───────────────┘ -``` - -如果数组元素或映射值没有共同类型,请使用函数 'array/map'(需启用设置 `use_variant_as_common_type`): - -```sql -SET use_variant_as_common_type = 1; -SELECT array(range(number), number, 'str_' || toString(number)) as array_of_variants FROM numbers(3); -``` - -```text -┌─array_of_variants─┐ -│ [[],0,'str_0'] │ -│ [[0],1,'str_1'] │ -│ [[0,1],2,'str_2'] │ -└───────────────────┘ -``` - -```sql -SET use_variant_as_common_type = 1; -SELECT map('a', range(number), 'b', number, 'c', 'str_' || toString(number)) as map_of_variants FROM numbers(3); -``` - -```text -┌─map_of_variants───────────────┐ -│ {'a':[],'b':0,'c':'str_0'} │ -│ {'a':[0],'b':1,'c':'str_1'} │ -│ {'a':[0,1],'b':2,'c':'str_2'} │ -└───────────────────────────────┘ -``` - -## 以子列形式读取 Variant 中的嵌套类型 {#reading-variant-nested-types-as-subcolumns} - -Variant 类型支持使用类型名作为子列,从 Variant 列中读取单个嵌套类型。 -因此,如果有列 `variant Variant(T1, T2, T3)`,可以使用语法 `variant.T2` 读取类型为 `T2` 的子列。 -当 `T2` 可以出现在 `Nullable` 中时,该子列的类型为 `Nullable(T2)`,否则为 `T2`。该子列的 -大小与原始 `Variant` 列相同,在原始 `Variant` 列不是类型 `T2` 的所有行中,它将包含 `NULL` 值 -(如果 `T2` 不能出现在 `Nullable` 中,则为对应的空值)。 - -也可以使用函数 `variantElement(variant_column, type_name)` 读取 Variant 子列。 - -示例: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT v, v.String, v.UInt64, v.`Array(UInt64)` FROM test; -``` - -```text -┌─v─────────────┬─v.String──────┬─v.UInt64─┬─v.Array(UInt64)─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴───────────────┴──────────┴─────────────────┘ -``` - -```sql -SELECT toTypeName(v.String), toTypeName(v.UInt64), toTypeName(v.`Array(UInt64)`) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(v.String)─┬─toTypeName(v.UInt64)─┬─toTypeName(v.Array(UInt64))─┐ -│ Nullable(String) │ Nullable(UInt64) │ Array(UInt64) │ -└──────────────────────┴──────────────────────┴─────────────────────────────┘ -``` - -```sql -SELECT v, variantElement(v, 'String'), variantElement(v, 'UInt64'), variantElement(v, 'Array(UInt64)') FROM test; -``` - -```text -┌─v─────────────┬─variantElement(v, 'String')─┬─variantElement(v, 'UInt64')─┬─variantElement(v, 'Array(UInt64)')─┐ -│ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ [] │ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└───────────────┴─────────────────────────────┴─────────────────────────────┴────────────────────────────────────┘ -``` - -要了解每一行中存储的是哪种变体类型,可以使用函数 `variantType(variant_column)`。它返回一个 `Enum`,其中包含每行对应的变体类型名称(如果该行为 `NULL`,则为 `'None'`)。 - -示例: - -```sql -CREATE TABLE test (v Variant(UInt64, String, Array(UInt64))) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('Hello, World!'), ([1, 2, 3]); -SELECT variantType(v) FROM test; -``` - -```text -┌─variantType(v)─┐ -│ None │ -│ UInt64 │ -│ String │ -│ Array(UInt64) │ -└────────────────┘ -``` - -```sql -SELECT toTypeName(variantType(v)) FROM test LIMIT 1; -``` - -```text -┌─toTypeName(variantType(v))──────────────────────────────────────────┐ -│ Enum8('None' = -1, 'Array(UInt64)' = 0, 'String' = 1, 'UInt64' = 2) │ -└─────────────────────────────────────────────────────────────────────┘ -``` - -## 在 Variant 列与其他列之间进行转换 {#conversion-between-a-variant-column-and-other-columns} - -对于类型为 `Variant` 的列,可以执行 4 种转换。 - -### 将 String 列转换为 Variant 列 {#converting-a-string-column-to-a-variant-column} - -从 `String` 到 `Variant` 的转换是通过从字符串值中解析出一个 `Variant` 类型的值来完成的: - -```sql -SELECT '42'::Variant(String, UInt64) AS variant, variantType(variant) AS variant_type -``` - -```text -┌─variant─┬─variant_type─┐ -│ 42 │ UInt64 │ -└─────────┴──────────────┘ -``` - -```sql -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt64) │ -└─────────┴───────────────┘ -``` - -````sql -SELECT CAST(map('key1', '42', 'key2', 'true', 'key3', '2020-01-01'), 'Map(String, Variant(UInt64, Bool, Date))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types``` -```` - -```text -┌─map_of_variants─────────────────────────────┬─map_of_variant_types──────────────────────────┐ -│ {'key1':42,'key2':true,'key3':'2020-01-01'} │ {'key1':'UInt64','key2':'Bool','key3':'Date'} │ -└─────────────────────────────────────────────┴───────────────────────────────────────────────┘ -``` - -要在将 `String` 转换为 `Variant` 时禁用解析,可以关闭 `cast_string_to_dynamic_use_inference` 设置: - -```sql -SET cast_string_to_variant_use_inference = 0; -SELECT '[1, 2, 3]'::Variant(String, Array(UInt64)) as variant, variantType(variant) as variant_type -``` - -```text -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### 将普通列转换为 Variant 列 {#converting-an-ordinary-column-to-a-variant-column} - -可以将类型为 `T` 的普通列转换为包含该类型的 `Variant` 列: - -```sql -SELECT toTypeName(variant) AS type_name, [1,2,3]::Array(UInt64)::Variant(UInt64, String, Array(UInt64)) as variant, variantType(variant) as variant_name -``` - -```text -┌─type_name──────────────────────────────┬─variant─┬─variant_name──┐ -│ Variant(Array(UInt64), String, UInt64) │ [1,2,3] │ Array(UInt64) │ -└────────────────────────────────────────┴─────────┴───────────────┘ -``` - -注意:从 `String` 类型转换时始终是通过解析完成的,如果你需要在不解析的情况下将 `String` 列转换为 `Variant` 的 `String` 变体,可以按如下方式进行: - -```sql -SELECT '[1, 2, 3]'::Variant(String)::Variant(String, Array(UInt64), UInt64) as variant, variantType(variant) as variant_type -``` - -```sql -┌─variant───┬─variant_type─┐ -│ [1, 2, 3] │ String │ -└───────────┴──────────────┘ -``` - -### 将 Variant 列转换为普通列 {#converting-a-variant-column-to-an-ordinary-column} - -可以将 `Variant` 列转换为普通列。在这种情况下,所有嵌套的 Variant 值都会被转换为目标类型: - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('42.42'); -SELECT v::Nullable(Float64) FROM test; -``` - -```text -┌─CAST(v, 'Nullable(Float64)')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ 42.42 │ -└──────────────────────────────┘ -``` - -### 将一个 Variant 转换为另一个 Variant {#converting-a-variant-to-another-variant} - -可以将一个 `Variant` 列转换为另一个 `Variant` 列,但只有当目标 `Variant` 列包含源 `Variant` 中的所有嵌套类型时才可以: - -```sql -CREATE TABLE test (v Variant(UInt64, String)) ENGINE = Memory; -INSERT INTO test VALUES (NULL), (42), ('String'); -SELECT v::Variant(UInt64, String, Array(UInt64)) FROM test; -``` - -```text -┌─CAST(v, 'Variant(UInt64, String, Array(UInt64))')─┐ -│ ᴺᵁᴸᴸ │ -│ 42 │ -│ String │ -└───────────────────────────────────────────────────┘ -``` - -## 从数据中读取 Variant 类型 {#reading-variant-type-from-the-data} - -所有文本格式(TSV、CSV、CustomSeparated、Values、JSONEachRow 等)都支持读取 `Variant` 类型。在解析数据时,ClickHouse 会尝试将每个值写入最合适的 Variant 成员类型中。 - -示例: - -```sql -SELECT - v, - variantElement(v, 'String') AS str, - variantElement(v, 'UInt64') AS num, - variantElement(v, 'Float64') AS float, - variantElement(v, 'DateTime') AS date, - variantElement(v, 'Array(UInt64)') AS arr -FROM format(JSONEachRow, 'v Variant(String, UInt64, Float64, DateTime, Array(UInt64))', $$ -{"v" : "Hello, World!"}, -{"v" : 42}, -{"v" : 42.42}, -{"v" : "2020-01-01 00:00:00"}, -{"v" : [1, 2, 3]} -$$) -``` - -```text -┌─v───────────────────┬─str───────────┬──num─┬─float─┬────────────────date─┬─arr─────┐ -│ Hello, World! │ Hello, World! │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42 │ ᴺᵁᴸᴸ │ 42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [] │ -│ 42.42 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 42.42 │ ᴺᵁᴸᴸ │ [] │ -│ 2020-01-01 00:00:00 │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ 2020-01-01 00:00:00 │ [] │ -│ [1,2,3] │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ ᴺᵁᴸᴸ │ [1,2,3] │ -└─────────────────────┴───────────────┴──────┴───────┴─────────────────────┴─────────┘ -``` - -## 比较 Variant 类型的值 {#comparing-values-of-variant-data} - -`Variant` 类型的值只能与具有相同 `Variant` 类型的值进行比较。 - -对于类型为 `Variant(..., T1, ... T2, ...)` 的值 `v1`(其底层类型为 `T1`)和 `v2`(其底层类型为 `T2`),运算符 `<` 的结果定义如下: - -* 如果 `T1 = T2 = T`,结果为 `v1.T < v2.T`(比较底层值)。 -* 如果 `T1 != T2`,结果为 `T1 < T2`(比较类型名)。 - -示例: - -```sql -CREATE TABLE test (v1 Variant(String, UInt64, Array(UInt32)), v2 Variant(String, UInt64, Array(UInt32))) ENGINE=Memory; -INSERT INTO test VALUES (42, 42), (42, 43), (42, 'abc'), (42, [1, 2, 3]), (42, []), (42, NULL); -``` - -```sql -SELECT v2, variantType(v2) AS v2_type FROM test ORDER BY v2; -``` - -```text -┌─v2──────┬─v2_type───────┐ -│ [] │ Array(UInt32) │ -│ [1,2,3] │ Array(UInt32) │ -│ abc │ String │ -│ 42 │ UInt64 │ -│ 43 │ UInt64 │ -│ ᴺᵁᴸᴸ │ None │ -└─────────┴───────────────┘ -``` - -```sql -SELECT v1, variantType(v1) AS v1_type, v2, variantType(v2) AS v2_type, v1 = v2, v1 < v2, v1 > v2 FROM test; -``` - -```text -┌─v1─┬─v1_type─┬─v2──────┬─v2_type───────┬─equals(v1, v2)─┬─less(v1, v2)─┬─greater(v1, v2)─┐ -│ 42 │ UInt64 │ 42 │ UInt64 │ 1 │ 0 │ 0 │ -│ 42 │ UInt64 │ 43 │ UInt64 │ 0 │ 1 │ 0 │ -│ 42 │ UInt64 │ abc │ String │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [1,2,3] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ [] │ Array(UInt32) │ 0 │ 0 │ 1 │ -│ 42 │ UInt64 │ ᴺᵁᴸᴸ │ None │ 0 │ 1 │ 0 │ -└────┴─────────┴─────────┴───────────────┴────────────────┴──────────────┴─────────────────┘ - -``` - -若需要查找具有特定 `Variant` 值的行,可以采用以下任一方法: - -* 将该值转换为相应的 `Variant` 类型: - -```sql -SELECT * FROM test WHERE v2 == [1,2,3]::Array(UInt32)::Variant(String, UInt64, Array(UInt32)); -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -* 将 `Variant` 子列与目标类型进行比较: - -```sql -SELECT * FROM test WHERE v2.`Array(UInt32)` == [1,2,3] -- 或者使用 variantElement(v2, 'Array(UInt32)') -``` - -```text -┌─v1─┬─v2──────┐ -│ 42 │ [1,2,3] │ -└────┴─────────┘ -``` - -有时对 Variant 类型进行额外检查是有用的,因为具有 `Array/Map/Tuple` 等复杂类型的子列不能置于 `Nullable` 中,对于类型不同的行,这些子列会使用默认值而不是 `NULL`: - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE v2.`Array(UInt32)` == []; -``` - -```text -┌─v2───┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ 42 │ [] │ UInt64 │ -│ 43 │ [] │ UInt64 │ -│ abc │ [] │ String │ -│ [] │ [] │ Array(UInt32) │ -│ ᴺᵁᴸᴸ │ [] │ None │ -└──────┴──────────────────┴─────────────────┘ -``` - -```sql -SELECT v2, v2.`Array(UInt32)`, variantType(v2) FROM test WHERE variantType(v2) == 'Array(UInt32)' AND v2.`Array(UInt32)` == []; -``` - -```text -┌─v2─┬─v2.Array(UInt32)─┬─variantType(v2)─┐ -│ [] │ [] │ Array(UInt32) │ -└────┴──────────────────┴─────────────────┘ -``` - -**注意:** 具有不同数值类型的 `Variant` 值会被视为不同的变体,彼此之间不会进行比较,而是比较它们的类型名称。 - -示例: - -```sql -SET allow_suspicious_variant_types = 1; -CREATE TABLE test (v Variant(UInt32, Int64)) ENGINE=Memory; -INSERT INTO test VALUES (1::UInt32), (1::Int64), (100::UInt32), (100::Int64); -SELECT v, variantType(v) FROM test ORDER by v; -``` - -```text -┌─v───┬─variantType(v)─┐ -│ 1 │ Int64 │ -│ 100 │ Int64 │ -│ 1 │ UInt32 │ -│ 100 │ UInt32 │ -└─────┴────────────────┘ -``` - -**注意** 默认情况下,不允许将 `Variant` 类型用作 `GROUP BY`/`ORDER BY` 键;如果你要使用它,请注意其特殊的比较规则,并启用 `allow_suspicious_types_in_group_by`/`allow_suspicious_types_in_order_by` 设置。 - -## 支持 Variant 类型的 JSONExtract 函数 {#jsonextract-functions-with-variant} - -所有 `JSONExtract*` 函数都支持 `Variant` 类型: - -```sql -SELECT JSONExtract('{"a" : [1, 2, 3]}', 'a', 'Variant(UInt32, String, Array(UInt32))') AS variant, variantType(variant) AS variant_type; -``` - -```text -┌─variant─┬─variant_type──┐ -│ [1,2,3] │ Array(UInt32) │ -└─────────┴───────────────┘ -``` - -```sql -SELECT JSONExtract('{"obj" : {"a" : 42, "b" : "Hello", "c" : [1,2,3]}}', 'obj', 'Map(String, Variant(UInt32, String, Array(UInt32)))') AS map_of_variants, mapApply((k, v) -> (k, variantType(v)), map_of_variants) AS map_of_variant_types -``` - -```text -┌─map_of_variants──────────────────┬─map_of_variant_types────────────────────────────┐ -│ {'a':42,'b':'Hello','c':[1,2,3]} │ {'a':'UInt32','b':'String','c':'Array(UInt32)'} │ -└──────────────────────────────────┴─────────────────────────────────────────────────┘ -``` - -```sql -SELECT JSONExtractKeysAndValues('{"a" : 42, "b" : "Hello", "c" : [1,2,3]}', 'Variant(UInt32, String, Array(UInt32))') AS variants, arrayMap(x -> (x.1, variantType(x.2)), variants) AS variant_types -``` - -```text -┌─variants───────────────────────────────┬─variant_types─────────────────────────────────────────┐ -│ [('a',42),('b','Hello'),('c',[1,2,3])] │ [('a','UInt32'),('b','String'),('c','Array(UInt32)')] │ -└────────────────────────────────────────┴───────────────────────────────────────────────────────┘ -``` diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md deleted file mode 100644 index 61cea2bd6f5..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md +++ /dev/null @@ -1,4 +0,0 @@ -:::tip -如果你在 ClickHouse Cloud 中使用字典,请使用 DDL 查询方式来创建字典,并使用用户 `default` 创建字典。 -另外,请在 [Cloud Compatibility 指南](/whats-new/cloud-compatibility) 中核对受支持的字典数据源列表。 -::: \ No newline at end of file diff --git a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md b/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md deleted file mode 100644 index f9f7ca5a1af..00000000000 --- a/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/index.md +++ /dev/null @@ -1,2528 +0,0 @@ ---- -description: 'ClickHouse 中外部字典功能概述' -sidebar_label: '定义字典' -sidebar_position: 35 -slug: /sql-reference/dictionaries -title: '字典' -doc_type: 'reference' ---- - -import SelfManaged from '@site/i18n/zh/docusaurus-plugin-content-docs/current/_snippets/_self_managed_only_no_roadmap.md'; -import CloudDetails from '@site/i18n/zh/docusaurus-plugin-content-docs/current/sql-reference/dictionaries/_snippet_dictionary_in_cloud.md'; -import CloudNotSupportedBadge from '@theme/badges/CloudNotSupportedBadge'; - -# 字典 {#dictionaries} - -字典是一种映射(`key -> attributes`),便于维护各种类型的参考列表。 - -ClickHouse 支持在查询中用于操作字典的特殊函数。与对参考表进行 `JOIN` 相比,配合这些函数使用字典更加简单且更高效。 - -ClickHouse 支持: - -- 具备[一组函数](../../sql-reference/functions/ext-dict-functions.md)的字典。 -- 具备特定[函数集](../../sql-reference/functions/embedded-dict-functions.md)的[嵌入式字典](#embedded-dictionaries)。 - -:::tip 教程 -如果您刚开始在 ClickHouse 中使用字典,我们提供了一个介绍该主题的教程。请查看[这里](tutorial.md)。 -::: - -您可以基于各种数据源添加自定义字典。字典的源可以是 ClickHouse 表、本地文本或可执行文件、HTTP(s) 资源,或其他 DBMS(数据库管理系统)。有关更多信息,请参阅“[字典源](#dictionary-sources)”。 - -ClickHouse: - -- 将字典全部或部分存储在 RAM 中。 -- 定期更新字典并动态加载缺失的值。换句话说,字典可以被动态加载。 -- 允许使用 XML 文件或 [DDL 查询](../../sql-reference/statements/create/dictionary.md)创建字典。 - -字典的配置可以位于一个或多个 XML 文件中。配置路径在 [dictionaries_config](../../operations/server-configuration-parameters/settings.md#dictionaries_config) 参数中指定。 - -字典可以在服务器启动时或首次使用时加载,具体取决于 [dictionaries_lazy_load](../../operations/server-configuration-parameters/settings.md#dictionaries_lazy_load) 设置。 - -[dictionaries](/operations/system-tables/dictionaries) 系统表包含服务器上已配置字典的信息。对于每个字典,您可以在其中查看: - -- 字典的状态。 -- 配置参数。 -- 各类指标,例如为该字典分配的 RAM 容量,或自字典成功加载以来的查询次数。 - - - -## 使用 DDL 查询创建字典 {#creating-a-dictionary-with-a-ddl-query} - -可以使用 [DDL 查询](../../sql-reference/statements/create/dictionary.md)来创建字典,这是推荐的方法,因为通过 DDL 创建的字典具有以下优点: -- 服务器配置文件中不会新增额外的记录。 -- 这些字典可以作为一等公民对象使用,就像表或视图一样。 -- 可以直接读取数据,使用熟悉的 SELECT,而不是字典表函数。请注意,当通过 SELECT 语句直接访问字典时,带缓存的字典只会返回已缓存的数据,而不带缓存的字典则会返回其存储的全部数据。 -- 可以轻松重命名这些字典。 - -## 使用配置文件创建字典 {#creating-a-dictionary-with-a-configuration-file} - - - -:::note -使用配置文件创建字典不适用于 ClickHouse Cloud。请使用 DDL(见上文),并以用户 `default` 创建字典。 -::: - -字典配置文件的格式如下: - -```xml - - 可选元素,可包含任意内容。ClickHouse 服务器将忽略此元素。 - - - /etc/metrika.xml - - - - - - - - -``` - -您可以在同一个文件中[配置](#configuring-a-dictionary)任意数量的字典。 - -:::note -对于较小的字典,您可以在 `SELECT` 查询中对其进行描述来转换其值(参见 [transform](../../sql-reference/functions/other-functions.md) 函数)。该功能与字典无关。 -::: - -## 配置字典 {#configuring-a-dictionary} - - - -如果通过 XML 文件配置字典,则字典配置的结构如下: - -```xml - - dict_name - - - - - - - - - - - - - - - - - -``` - -相应的 [DDL 查询](../../sql-reference/statements/create/dictionary.md) 的结构如下: - -```sql -CREATE DICTIONARY dict_name -( - ... -- 属性 -) -PRIMARY KEY ... -- 复合主键或单主键配置 -SOURCE(...) -- 数据源配置 -LAYOUT(...) -- 内存布局配置 -LIFETIME(...) -- 字典内存生命周期 -``` - -## 在内存中存储字典 {#storing-dictionaries-in-memory} - -在内存中有多种方式可以存储字典。 - -我们推荐使用 [flat](#flat)、[hashed](#hashed) 和 [complex_key_hashed](#complex_key_hashed),它们能够提供最佳的处理速度。 - -不推荐使用缓存,因为其性能可能较差且难以选择合适的参数。更多信息参见 [cache](#cache) 部分。 - -可以通过以下几种方式提升字典性能: - -* 在 `GROUP BY` 之后再调用用于操作字典的函数。 -* 将要提取的属性标记为 injective(单射)。如果不同的键对应不同的属性值,则称该属性为单射。因此,当 `GROUP BY` 中使用按键获取属性值的函数时,该函数会被自动从 `GROUP BY` 中提取出来单独执行。 - -ClickHouse 会针对字典相关的错误抛出异常。错误示例: - -* 无法加载被访问的字典。 -* 查询 `cached` 字典时出错。 - -可以在 [system.dictionaries](../../operations/system-tables/dictionaries.md) 表中查看字典列表及其状态。 - - - -配置如下所示: - -```xml - - - ... - - - - - - ... - - -``` - -对应的 [DDL 查询](../../sql-reference/statements/create/dictionary.md): - -```sql -CREATE DICTIONARY (...) -... -LAYOUT(LAYOUT_TYPE(param value)) -- 布局设置 -... -``` - -在布局名称中不包含单词 `complex-key*` 的字典,其键类型为 [UInt64](../../sql-reference/data-types/int-uint.md),而布局名称包含 `complex-key*` 的字典则具有复合键(复杂键,可由任意类型组合构成)。 - -在 XML 字典中,使用 `` 标签来定义 [UInt64](../../sql-reference/data-types/int-uint.md) 类型的键。 - -配置示例(列 `key_column` 的类型为 UInt64): - -```xml -... - - - key_column - -... -``` - -复合 `complex` 键的 XML 字典通过 `` 标签进行定义。 - -复合键的配置示例(键包含一个 [String](../../sql-reference/data-types/string.md) 类型的元素): - -```xml -... - - - - country_code - String - - -... -``` - -## 在内存中存储字典的方式 {#ways-to-store-dictionaries-in-memory} - -在内存中存储字典数据的各种方法,在 CPU 和 RAM 使用方面各有取舍。字典相关[博客文章](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse)中 [Choosing a Layout](https://clickhouse.com/blog/faster-queries-dictionaries-clickhouse#choosing-a-layout) 一节给出的决策树,是选择使用哪种布局的良好起点。 - -* [flat](#flat) -* [hashed](#hashed) -* [sparse_hashed](#sparse_hashed) -* [complex_key_hashed](#complex_key_hashed) -* [complex_key_sparse_hashed](#complex_key_sparse_hashed) -* [hashed_array](#hashed_array) -* [complex_key_hashed_array](#complex_key_hashed_array) -* [range_hashed](#range_hashed) -* [complex_key_range_hashed](#complex_key_range_hashed) -* [cache](#cache) -* [complex_key_cache](#complex_key_cache) -* [ssd_cache](#ssd_cache) -* [complex_key_ssd_cache](#complex_key_ssd_cache) -* [direct](#direct) -* [complex_key_direct](#complex_key_direct) -* [ip_trie](#ip_trie) - -### flat {#flat} - -字典以扁平数组的形式完全存储在内存中。字典会使用多少内存?其占用量与最大键的大小(占用的空间)成正比。 - -字典键使用 [UInt64](../../sql-reference/data-types/int-uint.md) 类型,值的数量被限制为 `max_array_size`(默认 — 500,000)。如果在创建字典时发现更大的键,ClickHouse 会抛出异常且不会创建字典。字典扁平数组的初始大小由 `initial_array_size` 设置控制(默认 — 1024)。 - -支持所有类型的数据源。在更新时,会将数据(来自文件或表)完整读入内存。 - -这种方法在所有可用的字典存储方式中性能最佳。 - -配置示例: - -```xml - - - 50000 - 5000000 - - -``` - -或 - -```sql -LAYOUT(FLAT(INITIAL_ARRAY_SIZE 50000 MAX_ARRAY_SIZE 5000000)) -``` - -### hashed {#hashed} - -字典以哈希表的形式完整存储在内存中。字典可以包含任意数量、具有任意标识符的元素。在实际应用中,键的数量可以达到数千万级。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -支持所有类型的数据源。在更新时,会将数据(来自文件或表)完整读取一遍。 - -配置示例: - -```xml - - - -``` - -或 - -```sql -LAYOUT(HASHED()) -``` - -配置示例: - -```xml - - - - 10 - - - 10000 - - - 0.5 - - -``` - -或 - -```sql -LAYOUT(HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### sparse_hashed {#sparse_hashed} - -类似于 `hashed`,但以更高的 CPU 开销为代价使用更少的内存。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -配置示例: - -```xml - - - - - - - -``` - -或者 - -```sql -LAYOUT(SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -对于此类字典也可以使用 `shards`。同样,对于 `sparse_hashed` 来说,这比对 `hashed` 更关键,因为 `sparse_hashed` 更慢。 - -### complex_key_hashed {#complex_key_hashed} - -此类存储用于复合[键](#dictionary-key-and-fields),与 `hashed` 类似。 - -配置示例: - -```xml - - - - - - - -``` - -或 - -```sql -LAYOUT(COMPLEX_KEY_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### complex_key_sparse_hashed {#complex_key_sparse_hashed} - -该存储类型用于复合[键](#dictionary-key-and-fields)。与 [sparse_hashed](#sparse_hashed) 类似。 - -配置示例: - -```xml - - - - - - - -``` - -或 - -```sql -LAYOUT(COMPLEX_KEY_SPARSE_HASHED([SHARDS 1] [SHARD_LOAD_QUEUE_BACKLOG 10000] [MAX_LOAD_FACTOR 0.5])) -``` - -### hashed_array {#hashed_array} - -该字典完全存储在内存中。每个属性都存储在一个数组中。键属性以哈希表的形式存储,其中值为属性数组中的索引。字典可以包含具有任意标识符的任意数量的元素。在实际使用中,键的数量可以达到数千万级。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -支持所有类型的数据源。更新时会完整读取数据(无论来自文件还是表)。 - -配置示例: - -```xml - - - - -``` - -或者 - -```sql -LAYOUT(HASHED_ARRAY([SHARDS 1])) -``` - -### complex_key_hashed_array {#complex_key_hashed_array} - -此存储类型用于复合[键](#dictionary-key-and-fields)。类似于 [hashed_array](#hashed_array)。 - -配置示例: - -```xml - - - -``` - -或 - -```sql -LAYOUT(COMPLEX_KEY_HASHED_ARRAY([SHARDS 1])) -``` - -### range_hashed {#range_hashed} - -字典在内存中以哈希表的形式存储,并包含一个有序的范围数组及其对应的值。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 -这种存储方式与 `hashed` 相同,并且在键之外还允许使用日期/时间(任意数值类型)范围。 - -示例:表中按如下格式存储了每个广告主的折扣信息: - -```text -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 123 │ 2015-01-16 │ 2015-01-31 │ 0.25 │ -│ 123 │ 2015-01-01 │ 2015-01-15 │ 0.15 │ -│ 456 │ 2015-01-01 │ 2015-01-15 │ 0.05 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ -``` - -要在日期范围上使用采样,请在[结构](#dictionary-key-and-fields)中定义 `range_min` 和 `range_max` 元素。这些元素必须包含 `name` 和 `type` 字段(如果未指定 `type`,则会使用默认类型 Date)。`type` 可以是任意数值类型(Date / DateTime / UInt64 / Int32 / 其他)。 - -:::note -`range_min` 和 `range_max` 的值必须能用 `Int64` 类型表示。 -::: - -示例: - -```xml - - - - min - - - - - advertiser_id - - - discount_start_date - Date - - - discount_end_date - Date - - ... -``` - -或 - -```sql -CREATE DICTIONARY discounts_dict ( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Date, - amount Float64 -) -PRIMARY KEY id -SOURCE(CLICKHOUSE(TABLE 'discounts')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(RANGE_HASHED(range_lookup_strategy 'max')) -RANGE(MIN discount_start_date MAX discount_end_date) -``` - -要使用这些字典,你需要向 `dictGet` 函数传递一个额外的参数,并为该参数选取一个范围: - -```sql -dictGet('dict_name', 'attr_name', id, date) -``` - -查询示例: - -```sql -SELECT dictGet('discounts_dict', 'amount', 1, '2022-10-20'::Date); -``` - -此函数返回在包含所传入日期的日期范围内,指定 `id` 对应的值。 - -算法细节: - -* 如果找不到该 `id`,或者为该 `id` 找不到范围,则返回该属性类型的默认值。 -* 如果存在重叠范围且 `range_lookup_strategy=min`,则返回一个匹配范围,其 `range_min` 最小;如果找到多个范围,则返回其中 `range_max` 最小的范围;如果仍然找到多个范围(多个范围具有相同的 `range_min` 和 `range_max`),则从中随机返回一个范围。 -* 如果存在重叠范围且 `range_lookup_strategy=max`,则返回一个匹配范围,其 `range_min` 最大;如果找到多个范围,则返回其中 `range_max` 最大的范围;如果仍然找到多个范围(多个范围具有相同的 `range_min` 和 `range_max`),则从中随机返回一个范围。 -* 如果 `range_max` 为 `NULL`,则该范围在上界是开放的(无上界)。`NULL` 被视为可能的最大值。对于 `range_min`,可以使用 `1970-01-01` 或 `0`(-MAX_INT)作为表示无下界的值。 - -配置示例: - -```xml - - - ... - - - - - - - - Abcdef - - - StartTimeStamp - UInt64 - - - EndTimeStamp - UInt64 - - - XXXType - String - - - - - - -``` - -或 - -```sql -CREATE DICTIONARY somedict( - Abcdef UInt64, - StartTimeStamp UInt64, - EndTimeStamp UInt64, - XXXType String DEFAULT '' -) -PRIMARY KEY Abcdef -RANGE(MIN StartTimeStamp MAX EndTimeStamp) -``` - -包含重叠区间和开区间的配置示例: - -```sql -CREATE TABLE discounts -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -ENGINE = Memory; -``` - -INSERT INTO discounts VALUES (1, '2015-01-01', Null, 0.1); -INSERT INTO discounts VALUES (1, '2015-01-15', Null, 0.2); -INSERT INTO discounts VALUES (2, '2015-01-01', '2015-01-15', 0.3); -INSERT INTO discounts VALUES (2, '2015-01-04', '2015-01-10', 0.4); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-15', 0.5); -INSERT INTO discounts VALUES (3, '1970-01-01', '2015-01-10', 0.6); - -SELECT * FROM discounts ORDER BY advertiser_id, discount_start_date; -┌─advertiser_id─┬─discount_start_date─┬─discount_end_date─┬─amount─┐ -│ 1 │ 2015-01-01 │ ᴺᵁᴸᴸ │ 0.1 │ -│ 1 │ 2015-01-15 │ ᴺᵁᴸᴸ │ 0.2 │ -│ 2 │ 2015-01-01 │ 2015-01-15 │ 0.3 │ -│ 2 │ 2015-01-04 │ 2015-01-10 │ 0.4 │ -│ 3 │ 1970-01-01 │ 2015-01-15 │ 0.5 │ -│ 3 │ 1970-01-01 │ 2015-01-10 │ 0.6 │ -└───────────────┴─────────────────────┴───────────────────┴────────┘ - --- RANGE_LOOKUP_STRATEGY 'max' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'max')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- 只有一个匹配的区间:2015-01-01 - Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.2 │ -- 有两个匹配的区间,range_min 2015-01-15 (0.2) 大于 2015-01-01 (0.1) -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.4 │ -- 有两个匹配的区间,range_min 2015-01-04 (0.4) 大于 2015-01-01 (0.3) -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.5 │ -- 有两个匹配的区间,range_min 相同,2015-01-15 (0.5) 大于 2015-01-10 (0.6) -└─────┘ - -DROP DICTIONARY discounts_dict; - --- RANGE_LOOKUP_STRATEGY 'min' - -CREATE DICTIONARY discounts_dict -( - advertiser_id UInt64, - discount_start_date Date, - discount_end_date Nullable(Date), - amount Float64 -) -PRIMARY KEY advertiser_id -SOURCE(CLICKHOUSE(TABLE discounts)) -LIFETIME(MIN 600 MAX 900) -LAYOUT(RANGE_HASHED(RANGE_LOOKUP_STRATEGY 'min')) -RANGE(MIN discount_start_date MAX discount_end_date); - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-14')) res; -┌─res─┐ -│ 0.1 │ -- 只有一个匹配的区间:2015-01-01 - Null -└─────┘ - -select dictGet('discounts_dict', 'amount', 1, toDate('2015-01-16')) res; -┌─res─┐ -│ 0.1 │ -- 有两个区间匹配,range_min 2015-01-01 (0.1) 小于 2015-01-15 (0.2) -└─────┘ - -select dictGet('discounts_dict', 'amount', 2, toDate('2015-01-06')) res; -┌─res─┐ -│ 0.3 │ -- 有两个区间匹配,range_min 2015-01-01 (0.3) 小于 2015-01-04 (0.4) -└─────┘ - -select dictGet('discounts_dict', 'amount', 3, toDate('2015-01-01')) res; -┌─res─┐ -│ 0.6 │ -- 有两个区间匹配,range_min 相同,2015-01-10 (0.6) 小于 2015-01-15 (0.5) -└─────┘ - -```` - -### complex_key_range_hashed {#complex_key_range_hashed} - -字典以哈希表的形式存储在内存中,包含一个有序的范围数组及其对应的值(参见 [range_hashed](#range_hashed))。此存储类型用于组合[键](#dictionary-key-and-fields)。 - -配置示例: - -```sql -CREATE DICTIONARY range_dictionary -( - CountryID UInt64, - CountryKey String, - StartDate Date, - EndDate Date, - Tax Float64 DEFAULT 0.2 -) -PRIMARY KEY CountryID, CountryKey -SOURCE(CLICKHOUSE(TABLE 'date_table')) -LIFETIME(MIN 1 MAX 1000) -LAYOUT(COMPLEX_KEY_RANGE_HASHED()) -RANGE(MIN StartDate MAX EndDate); -```` - -### cache {#cache} - -字典存储在具有固定数量单元格(槽位)的缓存中。这些单元格保存经常使用的元素。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -在查询字典时,系统会首先在缓存中查找。对于每个数据块,所有在缓存中未找到或已过期的键都会通过 `SELECT attrs... FROM db.table WHERE id IN (k1, k2, ...)` 向源端请求。接收到的数据随后写入缓存。 - -如果在字典中未找到键,则会创建缓存更新任务并将其添加到更新队列中。可以通过设置 `max_update_queue_size`、`update_queue_push_timeout_milliseconds`、`query_wait_timeout_milliseconds`、`max_threads_for_updates` 来控制更新队列的属性。 - -对于缓存型(cache)字典,可以设置缓存中数据的过期[生命周期](#refreshing-dictionary-data-using-lifetime)。如果自单元格中的数据加载后经过的时间超过 `lifetime`,则不再使用该单元格的值,并将该键标记为过期。下一次需要使用该键时会重新请求。可以通过 `allow_read_expired_keys` 设置来配置此行为。 - -这是所有字典存储方式中效率最低的一种。缓存的性能在很大程度上取决于正确的设置和使用场景。仅当缓存命中率足够高时(推荐 99% 及以上),缓存型(cache)字典才能有良好表现。可以在 [system.dictionaries](../../operations/system-tables/dictionaries.md) 表中查看平均命中率。 - -如果将 `allow_read_expired_keys` 设置为 1(默认值为 0),则字典可以支持异步更新。如果客户端请求的键全部在缓存中,但其中一些已过期,则字典会将这些过期值返回给客户端,并在后台异步向源端请求这些键的数据。 - -要提高缓存性能,请在子查询中使用 `LIMIT`,并在外层查询中调用使用该字典的函数。 - -支持所有类型的数据源。 - -设置示例: - -```xml - - - - 1000000000 - - 0 - - 100000 - - 10 - - 60000 - - 4 - - -``` - -或 - -```sql -LAYOUT(CACHE(SIZE_IN_CELLS 1000000000)) -``` - -设置足够大的缓存大小。你需要通过试验来选择 cell 的数量: - -1. 设置一个初始值。 -2. 运行查询,直到缓存被完全填满。 -3. 使用 `system.dictionaries` 表评估内存消耗。 -4. 增加或减少 cell 数量,直到达到所需的内存占用。 - -:::note -不要使用 ClickHouse 作为数据源,因为它在处理包含随机读取的查询时速度较慢。 -::: - -### complex_key_cache {#complex_key_cache} - -该存储类型用于复合[键](#dictionary-key-and-fields)。类似于 `cache`。 - -### ssd_cache {#ssd_cache} - -类似于 `cache`,但将数据存储在 SSD 上,将索引存储在 RAM 中。所有与更新队列相关的缓存字典设置同样适用于 SSD 缓存字典。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -```xml - - - - 4096 - - 16777216 - - 131072 - - 1048576 - - /var/lib/clickhouse/user_files/test_dict - - -``` - -或 - -```sql -LAYOUT(SSD_CACHE(BLOCK_SIZE 4096 FILE_SIZE 16777216 READ_BUFFER_SIZE 1048576 - PATH '/var/lib/clickhouse/user_files/test_dict')) -``` - -### complex_key_ssd_cache {#complex_key_ssd_cache} - -此类存储用于复合 [keys](#dictionary-key-and-fields)。与 `ssd_cache` 类似。 - -### direct {#direct} - -字典不会存储在内存中,而是在处理请求时直接访问源。 - -字典键的类型为 [UInt64](../../sql-reference/data-types/int-uint.md)。 - -除本地文件外,支持所有类型的 [数据源](#dictionary-sources)。 - -配置示例: - -```xml - - - -``` - -或 - -```sql -LAYOUT(DIRECT()) -``` - -### complex_key_direct {#complex_key_direct} - -此类存储用于复合[键](#dictionary-key-and-fields),与 `direct` 类似。 - -### ip_trie {#ip_trie} - -此字典专为通过网络前缀进行 IP 地址查找而设计。它以 CIDR 表示法存储 IP 范围,并支持快速确定给定 IP 所属的前缀(例如子网或 ASN 范围),非常适合用于基于 IP 的搜索,如地理定位或网络分类。 - -